resilien
/
terraform
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

"Add cloud integration option" 

A more native integration for Terraform Cloud and its CLI-driven run workflow.

Instead of a backend, users declare a special block in the top-level terraform settings
block to configure Terraform Cloud, then run terraform init.

Full documentation will follow in later commits.
This commit is contained in:
Martin Atkins 2021-10-28 18:30:01 -07:00 committed by GitHub
parent 9b62dd6901 369264865e
commit f266d1ee82
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
143 changed files with 12578 additions and 625 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
go.mod
View File

@ -4,6 +4,7 @@ require (
cloud.google.com/go/storage v1.10.0
github.com/Azure/azure-sdk-for-go v52.5.0+incompatible
github.com/Azure/go-autorest/autorest v0.11.18
github.com/Netflix/go-expect v0.0.0-20211003183012-e1a7c020ce25
github.com/agext/levenshtein v1.2.3
github.com/aliyun/alibaba-cloud-sdk-go v0.0.0-20190329064014-6e358769c32a
github.com/aliyun/aliyun-oss-go-sdk v0.0.0-20190103054945-8205d1f41e70
@ -22,7 +23,7 @@ require (
github.com/davecgh/go-spew v1.1.1
github.com/dylanmei/winrmtest v0.0.0-20190225150635-99b7fe2fddf1
github.com/go-test/deep v1.0.3
github.com/golang/mock v1.5.0
github.com/golang/mock v1.6.0
github.com/golang/protobuf v1.5.2
github.com/google/go-cmp v0.5.5
github.com/google/uuid v1.2.0
@ -39,9 +40,9 @@ require (
github.com/hashicorp/go-hclog v0.15.0
github.com/hashicorp/go-multierror v1.1.1
github.com/hashicorp/go-plugin v1.4.3
github.com/hashicorp/go-retryablehttp v0.5.2
github.com/hashicorp/go-tfe v0.15.0
github.com/hashicorp/go-uuid v1.0.1
github.com/hashicorp/go-retryablehttp v0.7.0
github.com/hashicorp/go-tfe v0.19.1-0.20211020175229-e52963e079d0
github.com/hashicorp/go-uuid v1.0.2
github.com/hashicorp/go-version v1.2.1
github.com/hashicorp/hcl v0.0.0-20170504190234-a4b07c25de5f
github.com/hashicorp/hcl/v2 v2.10.1
@ -143,9 +144,9 @@ require (
github.com/hashicorp/go-msgpack v0.5.4 // indirect
github.com/hashicorp/go-rootcerts v1.0.2 // indirect
github.com/hashicorp/go-safetemp v1.0.0 // indirect
github.com/hashicorp/go-slug v0.4.1 // indirect
github.com/hashicorp/go-slug v0.7.0 // indirect
github.com/hashicorp/golang-lru v0.5.1 // indirect
github.com/hashicorp/jsonapi v0.0.0-20210518035559-1e50d74c8db3 // indirect
github.com/hashicorp/jsonapi v0.0.0-20210826224640-ee7dae0fb22d // indirect
github.com/hashicorp/serf v0.9.5 // indirect
github.com/hashicorp/yamux v0.0.0-20181012175058-2f1d1f20f75d // indirect
github.com/huandu/xstrings v1.3.2 // indirect
@ -154,6 +155,7 @@ require (
github.com/jstemmer/go-junit-report v0.9.1 // indirect
github.com/jtolds/gls v4.2.1+incompatible // indirect
github.com/klauspost/compress v1.11.2 // indirect
github.com/kr/pty v1.1.1 // indirect
github.com/masterzen/simplexml v0.0.0-20190410153822-31eea3082786 // indirect
github.com/mattn/go-colorable v0.1.6 // indirect
github.com/mitchellh/go-testing-interface v1.0.0 // indirect

30
go.sum
View File

@ -89,6 +89,8 @@ github.com/Masterminds/sprig v2.22.0+incompatible/go.mod h1:y6hNFY5UBTIWBxnzTeuN
github.com/Microsoft/go-winio v0.5.0 h1:Elr9Wn+sGKPlkaBvwu4mTrxtmOp3F3yV9qhaHbXGjwU=
github.com/Microsoft/go-winio v0.5.0/go.mod h1:JPGBdM1cNvN/6ISo+n8V5iA4v8pBzdOpzfwIujj1a84=
github.com/NYTimes/gziphandler v0.0.0-20170623195520-56545f4a5d46/go.mod h1:3wb06e3pkSAbeQ52E9H9iFoQsEEwGN64994WTCIhntQ=
github.com/Netflix/go-expect v0.0.0-20211003183012-e1a7c020ce25 h1:hWfsqBaNZUHztXA78g7Y2Jj3rDQaTCZhhFwz43i2VlA=
github.com/Netflix/go-expect v0.0.0-20211003183012-e1a7c020ce25/go.mod h1:68ORG0HSEWDuH5Eh73AFbYWZ1zT4Y+b0vhOa+vZRUdI=
github.com/PuerkitoBio/purell v1.0.0/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0=
github.com/PuerkitoBio/urlesc v0.0.0-20160726150825-5bd2802263f2/go.mod h1:uGdkoq3SwY9Y+13GIhn11/XLaGBb4BfwItxLd5jeuXE=
github.com/QcloudApi/qcloud_sign_golang v0.0.0-20141224014652-e4130a326409/go.mod h1:1pk82RBxDY/JZnPQrtqHlUFfCctgdorsd9M06fMynOM=
@ -241,8 +243,9 @@ github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFU
github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.4 h1:l75CXGRSwbaYNpl/Z2X1XIIAMSCquvXgpVZDhwEIJsc=
github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4=
github.com/golang/mock v1.6.0 h1:ErTB+efbowRARo13NNdxyJji2egdxLGQhRaY+DUumQc=
github.com/golang/mock v1.6.0/go.mod h1:p6yTPP+5HYm5mzsMV8JkE6ZKdX+/wYM6Hr+LicevLPs=
github.com/golang/protobuf v0.0.0-20161109072736-4bd1920723d7/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.1.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
@ -349,6 +352,7 @@ github.com/hashicorp/go-cleanhttp v0.5.2 h1:035FKYIWjmULyFRBKPs8TBQoi0x6d9G4xc9n
github.com/hashicorp/go-cleanhttp v0.5.2/go.mod h1:kO/YDlP8L1346E6Sodw+PrpBSV4/SoxCXGY6BqNFT48=
github.com/hashicorp/go-getter v1.5.2 h1:XDo8LiAcDisiqZdv0TKgz+HtX3WN7zA2JD1R1tjsabE=
github.com/hashicorp/go-getter v1.5.2/go.mod h1:orNH3BTYLu/fIxGIdLjLoAJHWMDQ/UKQr5O4m3iBuoo=
github.com/hashicorp/go-hclog v0.9.2/go.mod h1:5CU+agLiy3J7N7QjHK5d05KxGsuXiQLrjA0H7acj2lQ=
github.com/hashicorp/go-hclog v0.12.0/go.mod h1:whpDNt7SSdeAju8AWKIWsul05p54N/39EeqMAyrmvFQ=
github.com/hashicorp/go-hclog v0.14.1/go.mod h1:whpDNt7SSdeAju8AWKIWsul05p54N/39EeqMAyrmvFQ=
github.com/hashicorp/go-hclog v0.15.0 h1:qMuK0wxsoW4D0ddCCYwPSTm4KQv1X1ke3WmPWZ0Mvsk=
@ -364,22 +368,23 @@ github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+l
github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM=
github.com/hashicorp/go-plugin v1.4.3 h1:DXmvivbWD5qdiBts9TpBC7BYL1Aia5sxbRgQB+v6UZM=
github.com/hashicorp/go-plugin v1.4.3/go.mod h1:5fGEH17QVwTTcR0zV7yhDPLLmFX9YSZ38b18Udy6vYQ=
github.com/hashicorp/go-retryablehttp v0.5.2 h1:AoISa4P4IsW0/m4T6St8Yw38gTl5GtBAgfkhYh1xAz4=
github.com/hashicorp/go-retryablehttp v0.5.2/go.mod h1:9B5zBasrRhHXnJnui7y6sL7es7NDiJgTc6Er0maI1Xs=
github.com/hashicorp/go-retryablehttp v0.7.0 h1:eu1EI/mbirUgP5C8hVsTNaGZreBDlYiwC1FZWkvQPQ4=
github.com/hashicorp/go-retryablehttp v0.7.0/go.mod h1:vAew36LZh98gCBJNLH42IQ1ER/9wtLZZ8meHqQvEYWY=
github.com/hashicorp/go-rootcerts v1.0.2 h1:jzhAVGtqPKbwpyCPELlgNWhE1znq+qwJtW5Oi2viEzc=
github.com/hashicorp/go-rootcerts v1.0.2/go.mod h1:pqUvnprVnM5bf7AOirdbb01K4ccR319Vf4pU3K5EGc8=
github.com/hashicorp/go-safetemp v1.0.0 h1:2HR189eFNrjHQyENnQMMpCiBAsRxzbTMIgBhEyExpmo=
github.com/hashicorp/go-safetemp v1.0.0/go.mod h1:oaerMy3BhqiTbVye6QuFhFtIceqFoDHxNAB65b+Rj1I=
github.com/hashicorp/go-slug v0.4.1 h1:/jAo8dNuLgSImoLXaX7Od7QB4TfYCVPam+OpAt5bZqc=
github.com/hashicorp/go-slug v0.4.1/go.mod h1:I5tq5Lv0E2xcNXNkmx7BSfzi1PsJ2cNjs3cC3LwyhK8=
github.com/hashicorp/go-slug v0.7.0 h1:8HIi6oreWPtnhpYd8lIGQBgp4rXzDWQTOhfILZm+nok=
github.com/hashicorp/go-slug v0.7.0/go.mod h1:Ib+IWBYfEfJGI1ZyXMGNbu2BU+aa3Dzu41RKLH301v4=
github.com/hashicorp/go-sockaddr v1.0.0 h1:GeH6tui99pF4NJgfnhp+L6+FfobzVW3Ah46sLo0ICXs=
github.com/hashicorp/go-sockaddr v1.0.0/go.mod h1:7Xibr9yA9JjQq1JpNB2Vw7kxv8xerXegt+ozgdvDeDU=
github.com/hashicorp/go-syslog v1.0.0/go.mod h1:qPfqrKkXGihmCqbJM2mZgkZGvKG1dFdvsLplgctolz4=
github.com/hashicorp/go-tfe v0.15.0 h1:vdnz1NjOhvmap+cj8iPsL8SbS4iFFVuNYFkGpF5SdoA=
github.com/hashicorp/go-tfe v0.15.0/go.mod h1:c8glB5p6XzocEWLNkuy5RxcjqN5X2PpY6NF3f2W6nIo=
github.com/hashicorp/go-tfe v0.19.1-0.20211020175229-e52963e079d0 h1:64o1fy8p4a9J/5Gooy1S9iuEJQqXTVGYk5ZrXwFDZ8w=
github.com/hashicorp/go-tfe v0.19.1-0.20211020175229-e52963e079d0/go.mod h1:gyXLXbpBVxA2F/6opah8XBsOkZJxHYQmghl0OWi8keI=
github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
github.com/hashicorp/go-uuid v1.0.1 h1:fv1ep09latC32wFoVwnqcnKJGnMSdBanPczbHAYm1BE=
github.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
github.com/hashicorp/go-uuid v1.0.2 h1:cfejS+Tpcp13yd5nYHWDI6qVCny6wyX2Mt5SGur2IGE=
github.com/hashicorp/go-uuid v1.0.2/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
github.com/hashicorp/go-version v1.0.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
github.com/hashicorp/go-version v1.1.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
github.com/hashicorp/go-version v1.2.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
@ -393,8 +398,8 @@ github.com/hashicorp/hcl v0.0.0-20170504190234-a4b07c25de5f/go.mod h1:oZtUIOe8dh
github.com/hashicorp/hcl/v2 v2.0.0/go.mod h1:oVVDG71tEinNGYCxinCYadcmKU9bglqW9pV3txagJ90=
github.com/hashicorp/hcl/v2 v2.10.1 h1:h4Xx4fsrRE26ohAk/1iGF/JBqRQbyUqu5Lvj60U54ys=
github.com/hashicorp/hcl/v2 v2.10.1/go.mod h1:FwWsfWEjyV/CMj8s/gqAuiviY72rJ1/oayI9WftqcKg=
github.com/hashicorp/jsonapi v0.0.0-20210518035559-1e50d74c8db3 h1:mzwkutymYIXR5oQT9YnfbLuuw7LZmksiHKRPUTN5ijo=
github.com/hashicorp/jsonapi v0.0.0-20210518035559-1e50d74c8db3/go.mod h1:Yog5+CPEM3c99L1CL2CFCYoSzgWm5vTU58idbRUaLik=
github.com/hashicorp/jsonapi v0.0.0-20210826224640-ee7dae0fb22d h1:9ARUJJ1VVynB176G1HCwleORqCaXm/Vx0uUi0dL26I0=
github.com/hashicorp/jsonapi v0.0.0-20210826224640-ee7dae0fb22d/go.mod h1:Yog5+CPEM3c99L1CL2CFCYoSzgWm5vTU58idbRUaLik=
github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64=
github.com/hashicorp/mdns v1.0.1/go.mod h1:4gW7WsVCke5TE7EPeYliwHlRUyBtfCwuFwuMg2DmyNY=
github.com/hashicorp/memberlist v0.2.2 h1:5+RffWKwqJ71YPu9mWsF7ZOscZmwfasdA8kbdC7AO2g=
@ -452,6 +457,7 @@ github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFB
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.2.0 h1:s5hAObm+yFO5uHYt5dYjxi2rXrsnmRpJx4OYvIWUaQs=
github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pty v1.1.1 h1:VkoXIwSboBpnk99O/KFauAEILuNHv5DVFKZMBN/gUgw=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
@ -610,8 +616,9 @@ github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXf
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.6.1 h1:hDPOHmpOpP40lSULcqw7IrRb/u7w6RpDC9399XyoNd0=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common v1.0.194/go.mod h1:7sCQWVkxcsR38nffDW057DRGk8mUjK1Ing/EFOK8s8Y=
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common v1.0.232 h1:kwsWbh4rEw42ZDe9/812ebhbwNZxlQyZ2sTmxBOKhN4=
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common v1.0.232/go.mod h1:7sCQWVkxcsR38nffDW057DRGk8mUjK1Ing/EFOK8s8Y=
@ -932,6 +939,7 @@ golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4f
golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=
golang.org/x/tools v0.1.1/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.4/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.7 h1:6j8CgantCy3yc8JGBqkDLMKWqZ0RDU2g1HVgacojGWQ=
golang.org/x/tools v0.1.7/go.mod h1:LGqMHiF4EqQNHR1JncWGqT5BVaXmza+X+BDGol+dOxo=

6
internal/backend/init/init.go
View File

@ -27,6 +27,7 @@ import (
backendPg "github.com/hashicorp/terraform/internal/backend/remote-state/pg"
backendS3 "github.com/hashicorp/terraform/internal/backend/remote-state/s3"
backendSwift "github.com/hashicorp/terraform/internal/backend/remote-state/swift"
backendCloud "github.com/hashicorp/terraform/internal/cloud"
)
// backends is the list of available backends. This is a global variable
@ -49,7 +50,6 @@ func Init(services *disco.Disco) {
defer backendsLock.Unlock()
backends = map[string]backend.InitFn{
// Enhanced backends.
"local": func() backend.Backend { return backendLocal.New() },
"remote": func() backend.Backend { return backendRemote.New(services) },
@ -70,6 +70,10 @@ func Init(services *disco.Disco) {
"s3": func() backend.Backend { return backendS3.New() },
"swift": func() backend.Backend { return backendSwift.New() },
// Terraform Cloud 'backend'
// This is an implementation detail only, used for the cloud package
"cloud": func() backend.Backend { return backendCloud.New(services) },
// Deprecated backends.
"azure": func() backend.Backend {
return deprecateBackend(

33
internal/backend/remote/backend_apply_test.go
View File

@ -14,6 +14,7 @@ import (
version "github.com/hashicorp/go-version"
"github.com/hashicorp/terraform/internal/addrs"
"github.com/hashicorp/terraform/internal/backend"
"github.com/hashicorp/terraform/internal/cloud"
"github.com/hashicorp/terraform/internal/command/arguments"
"github.com/hashicorp/terraform/internal/command/clistate"
"github.com/hashicorp/terraform/internal/command/views"
@ -308,11 +309,11 @@ func TestRemote_applyWithoutRefresh(t *testing.T) {
// We should find a run inside the mock client that has refresh set
// to false.
runsAPI := b.client.Runs.(*mockRuns)
if got, want := len(runsAPI.runs), 1; got != want {
runsAPI := b.client.Runs.(*cloud.MockRuns)
if got, want := len(runsAPI.Runs), 1; got != want {
t.Fatalf("wrong number of runs in the mock client %d; want %d", got, want)
}
for _, run := range runsAPI.runs {
for _, run := range runsAPI.Runs {
if diff := cmp.Diff(false, run.Refresh); diff != "" {
t.Errorf("wrong Refresh setting in the created run\n%s", diff)
}
@ -377,11 +378,11 @@ func TestRemote_applyWithRefreshOnly(t *testing.T) {
// We should find a run inside the mock client that has refresh-only set
// to true.
runsAPI := b.client.Runs.(*mockRuns)
if got, want := len(runsAPI.runs), 1; got != want {
runsAPI := b.client.Runs.(*cloud.MockRuns)
if got, want := len(runsAPI.Runs), 1; got != want {
t.Fatalf("wrong number of runs in the mock client %d; want %d", got, want)
}
for _, run := range runsAPI.runs {
for _, run := range runsAPI.Runs {
if diff := cmp.Diff(true, run.RefreshOnly); diff != "" {
t.Errorf("wrong RefreshOnly setting in the created run\n%s", diff)
}
@ -448,11 +449,11 @@ func TestRemote_applyWithTarget(t *testing.T) {
// We should find a run inside the mock client that has the same
// target address we requested above.
runsAPI := b.client.Runs.(*mockRuns)
if got, want := len(runsAPI.runs), 1; got != want {
runsAPI := b.client.Runs.(*cloud.MockRuns)
if got, want := len(runsAPI.Runs), 1; got != want {
t.Fatalf("wrong number of runs in the mock client %d; want %d", got, want)
}
for _, run := range runsAPI.runs {
for _, run := range runsAPI.Runs {
if diff := cmp.Diff([]string{"null_resource.foo"}, run.TargetAddrs); diff != "" {
t.Errorf("wrong TargetAddrs in the created run\n%s", diff)
}
@ -523,11 +524,11 @@ func TestRemote_applyWithReplace(t *testing.T) {
// We should find a run inside the mock client that has the same
// refresh address we requested above.
runsAPI := b.client.Runs.(*mockRuns)
if got, want := len(runsAPI.runs), 1; got != want {
runsAPI := b.client.Runs.(*cloud.MockRuns)
if got, want := len(runsAPI.Runs), 1; got != want {
t.Fatalf("wrong number of runs in the mock client %d; want %d", got, want)
}
for _, run := range runsAPI.runs {
for _, run := range runsAPI.Runs {
if diff := cmp.Diff([]string{"null_resource.foo"}, run.ReplaceAddrs); diff != "" {
t.Errorf("wrong ReplaceAddrs in the created run\n%s", diff)
}
@ -776,9 +777,7 @@ func TestRemote_applyApprovedExternally(t *testing.T) {
wl, err := b.client.Workspaces.List(
ctx,
b.organization,
tfe.WorkspaceListOptions{
ListOptions: tfe.ListOptions{PageNumber: 2, PageSize: 10},
},
tfe.WorkspaceListOptions{},
)
if err != nil {
t.Fatalf("unexpected error listing workspaces: %v", err)
@ -852,9 +851,7 @@ func TestRemote_applyDiscardedExternally(t *testing.T) {
wl, err := b.client.Workspaces.List(
ctx,
b.organization,
tfe.WorkspaceListOptions{
ListOptions: tfe.ListOptions{PageNumber: 2, PageSize: 10},
},
tfe.WorkspaceListOptions{},
)
if err != nil {
t.Fatalf("unexpected error listing workspaces: %v", err)

29
internal/backend/remote/backend_plan_test.go
View File

@ -13,6 +13,7 @@ import (
tfe "github.com/hashicorp/go-tfe"
"github.com/hashicorp/terraform/internal/addrs"
"github.com/hashicorp/terraform/internal/backend"
"github.com/hashicorp/terraform/internal/cloud"
"github.com/hashicorp/terraform/internal/command/arguments"
"github.com/hashicorp/terraform/internal/command/clistate"
"github.com/hashicorp/terraform/internal/command/views"
@ -313,11 +314,11 @@ func TestRemote_planWithoutRefresh(t *testing.T) {
// We should find a run inside the mock client that has refresh set
// to false.
runsAPI := b.client.Runs.(*mockRuns)
if got, want := len(runsAPI.runs), 1; got != want {
runsAPI := b.client.Runs.(*cloud.MockRuns)
if got, want := len(runsAPI.Runs), 1; got != want {
t.Fatalf("wrong number of runs in the mock client %d; want %d", got, want)
}
for _, run := range runsAPI.runs {
for _, run := range runsAPI.Runs {
if diff := cmp.Diff(false, run.Refresh); diff != "" {
t.Errorf("wrong Refresh setting in the created run\n%s", diff)
}
@ -382,11 +383,11 @@ func TestRemote_planWithRefreshOnly(t *testing.T) {
// We should find a run inside the mock client that has refresh-only set
// to true.
runsAPI := b.client.Runs.(*mockRuns)
if got, want := len(runsAPI.runs), 1; got != want {
runsAPI := b.client.Runs.(*cloud.MockRuns)
if got, want := len(runsAPI.Runs), 1; got != want {
t.Fatalf("wrong number of runs in the mock client %d; want %d", got, want)
}
for _, run := range runsAPI.runs {
for _, run := range runsAPI.Runs {
if diff := cmp.Diff(true, run.RefreshOnly); diff != "" {
t.Errorf("wrong RefreshOnly setting in the created run\n%s", diff)
}
@ -432,7 +433,7 @@ func TestRemote_planWithTarget(t *testing.T) {
// When the backend code creates a new run, we'll tweak it so that it
// has a cost estimation object with the "skipped_due_to_targeting" status,
// emulating how a real server is expected to behave in that case.
b.client.Runs.(*mockRuns).modifyNewRun = func(client *mockClient, options tfe.RunCreateOptions, run *tfe.Run) {
b.client.Runs.(*cloud.MockRuns).ModifyNewRun = func(client *cloud.MockClient, options tfe.RunCreateOptions, run *tfe.Run) {
const fakeID = "fake"
// This is the cost estimate object embedded in the run itself which
// the backend will use to learn the ID to request from the cost
@ -446,7 +447,7 @@ func TestRemote_planWithTarget(t *testing.T) {
// the same ID indicated in the object above, where we'll then return
// the status "skipped_due_to_targeting" to trigger the special skip
// message in the backend output.
client.CostEstimates.estimations[fakeID] = &tfe.CostEstimate{
client.CostEstimates.Estimations[fakeID] = &tfe.CostEstimate{
ID: fakeID,
Status: "skipped_due_to_targeting",
}
@ -483,11 +484,11 @@ func TestRemote_planWithTarget(t *testing.T) {
// We should find a run inside the mock client that has the same
// target address we requested above.
runsAPI := b.client.Runs.(*mockRuns)
if got, want := len(runsAPI.runs), 1; got != want {
runsAPI := b.client.Runs.(*cloud.MockRuns)
if got, want := len(runsAPI.Runs), 1; got != want {
t.Fatalf("wrong number of runs in the mock client %d; want %d", got, want)
}
for _, run := range runsAPI.runs {
for _, run := range runsAPI.Runs {
if diff := cmp.Diff([]string{"null_resource.foo"}, run.TargetAddrs); diff != "" {
t.Errorf("wrong TargetAddrs in the created run\n%s", diff)
}
@ -558,11 +559,11 @@ func TestRemote_planWithReplace(t *testing.T) {
// We should find a run inside the mock client that has the same
// refresh address we requested above.
runsAPI := b.client.Runs.(*mockRuns)
if got, want := len(runsAPI.runs), 1; got != want {
runsAPI := b.client.Runs.(*cloud.MockRuns)
if got, want := len(runsAPI.Runs), 1; got != want {
t.Fatalf("wrong number of runs in the mock client %d; want %d", got, want)
}
for _, run := range runsAPI.runs {
for _, run := range runsAPI.Runs {
if diff := cmp.Diff([]string{"null_resource.foo"}, run.ReplaceAddrs); diff != "" {
t.Errorf("wrong ReplaceAddrs in the created run\n%s", diff)
}

3
internal/backend/remote/backend_state_test.go
View File

@ -6,6 +6,7 @@ import (
"testing"
"github.com/hashicorp/terraform/internal/backend"
"github.com/hashicorp/terraform/internal/cloud"
"github.com/hashicorp/terraform/internal/states"
"github.com/hashicorp/terraform/internal/states/remote"
"github.com/hashicorp/terraform/internal/states/statefile"
@ -39,7 +40,7 @@ func TestRemoteClient_stateLock(t *testing.T) {
func TestRemoteClient_withRunID(t *testing.T) {
// Set the TFE_RUN_ID environment variable before creating the client!
if err := os.Setenv("TFE_RUN_ID", generateID("run-")); err != nil {
if err := os.Setenv("TFE_RUN_ID", cloud.GenerateID("run-")); err != nil {
t.Fatalf("error setting env var TFE_RUN_ID: %v", err)
}

24
internal/backend/remote/testing.go
View File

@ -8,12 +8,14 @@ import (
"net/http/httptest"
"path"
"testing"
"time"
tfe "github.com/hashicorp/go-tfe"
svchost "github.com/hashicorp/terraform-svchost"
"github.com/hashicorp/terraform-svchost/auth"
"github.com/hashicorp/terraform-svchost/disco"
"github.com/hashicorp/terraform/internal/backend"
"github.com/hashicorp/terraform/internal/cloud"
"github.com/hashicorp/terraform/internal/configs"
"github.com/hashicorp/terraform/internal/configs/configschema"
"github.com/hashicorp/terraform/internal/httpclient"
@ -39,6 +41,26 @@ var (
})
)
// mockInput is a mock implementation of terraform.UIInput.
type mockInput struct {
answers map[string]string
}
func (m *mockInput) Input(ctx context.Context, opts *terraform.InputOpts) (string, error) {
v, ok := m.answers[opts.Id]
if !ok {
return "", fmt.Errorf("unexpected input request in test: %s", opts.Id)
}
if v == "wait-for-external-update" {
select {
case <-ctx.Done():
case <-time.After(time.Minute):
}
}
delete(m.answers, opts.Id)
return v, nil
}
func testInput(t *testing.T, answers map[string]string) *mockInput {
return &mockInput{answers: answers}
}
@ -111,7 +133,7 @@ func testBackend(t *testing.T, obj cty.Value) (*Remote, func()) {
}
// Get a new mock client.
mc := newMockClient()
mc := cloud.NewMockClient()
// Replace the services we use with our mock services.
b.CLI = cli.NewMockUi()

193
internal/backend/unparsed_value.go
View File

@ -25,6 +25,121 @@ type UnparsedVariableValue interface {
ParseVariableValue(mode configs.VariableParsingMode) (*terraform.InputValue, tfdiags.Diagnostics)
}
// ParseUndeclaredVariableValues processes a map of unparsed variable values
// and returns an input values map of the ones not declared in the specified
// declaration map along with detailed diagnostics about values of undeclared
// variables being present, depending on the source of these values. If more
// than two undeclared values are present in file form (config, auto, -var-file)
// the remaining errors are summarized to avoid a massive list of errors.
func ParseUndeclaredVariableValues(vv map[string]UnparsedVariableValue, decls map[string]*configs.Variable) (terraform.InputValues, tfdiags.Diagnostics) {
var diags tfdiags.Diagnostics
ret := make(terraform.InputValues, len(vv))
seenUndeclaredInFile := 0
for name, rv := range vv {
if _, declared := decls[name]; declared {
// Only interested in parsing undeclared variables
continue
}
val, valDiags := rv.ParseVariableValue(configs.VariableParseLiteral)
if valDiags.HasErrors() {
continue
}
ret[name] = val
switch val.SourceType {
case terraform.ValueFromConfig, terraform.ValueFromAutoFile, terraform.ValueFromNamedFile:
// We allow undeclared names for variable values from files and warn in case
// users have forgotten a variable {} declaration or have a typo in their var name.
// Some users will actively ignore this warning because they use a .tfvars file
// across multiple configurations.
if seenUndeclaredInFile < 2 {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Warning,
"Value for undeclared variable",
fmt.Sprintf("The root module does not declare a variable named %q but a value was found in file %q. If you meant to use this value, add a \"variable\" block to the configuration.\n\nTo silence these warnings, use TF_VAR_... environment variables to provide certain \"global\" settings to all configurations in your organization. To reduce the verbosity of these warnings, use the -compact-warnings option.", name, val.SourceRange.Filename),
))
}
seenUndeclaredInFile++
case terraform.ValueFromEnvVar:
// We allow and ignore undeclared names for environment
// variables, because users will often set these globally
// when they are used across many (but not necessarily all)
// configurations.
case terraform.ValueFromCLIArg:
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Value for undeclared variable",
fmt.Sprintf("A variable named %q was assigned on the command line, but the root module does not declare a variable of that name. To use this value, add a \"variable\" block to the configuration.", name),
))
default:
// For all other source types we are more vague, but other situations
// don't generally crop up at this layer in practice.
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Value for undeclared variable",
fmt.Sprintf("A variable named %q was assigned a value, but the root module does not declare a variable of that name. To use this value, add a \"variable\" block to the configuration.", name),
))
}
}
if seenUndeclaredInFile > 2 {
extras := seenUndeclaredInFile - 2
diags = diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagWarning,
Summary: "Values for undeclared variables",
Detail: fmt.Sprintf("In addition to the other similar warnings shown, %d other variable(s) defined without being declared.", extras),
})
}
return ret, diags
}
// ParseDeclaredVariableValues processes a map of unparsed variable values
// and returns an input values map of the ones declared in the specified
// variable declaration mapping. Diagnostics will be populating with
// any variable parsing errors encountered within this collection.
func ParseDeclaredVariableValues(vv map[string]UnparsedVariableValue, decls map[string]*configs.Variable) (terraform.InputValues, tfdiags.Diagnostics) {
var diags tfdiags.Diagnostics
ret := make(terraform.InputValues, len(vv))
for name, rv := range vv {
var mode configs.VariableParsingMode
config, declared := decls[name]
if declared {
mode = config.ParsingMode
} else {
// Only interested in parsing declared variables
continue
}
val, valDiags := rv.ParseVariableValue(mode)
diags = diags.Append(valDiags)
if valDiags.HasErrors() {
continue
}
ret[name] = val
}
return ret, diags
}
// Checks all given terraform.InputValues variable maps for the existance of
// a named variable
func isDefinedAny(name string, maps ...terraform.InputValues) bool {
for _, m := range maps {
if _, defined := m[name]; defined {
return true
}
}
return false
}
// ParseVariableValues processes a map of unparsed variable values by
// correlating each one with the given variable declarations which should
// be from a root module.
@ -42,87 +157,17 @@ type UnparsedVariableValue interface {
// that were successfully processed, allowing for careful analysis of the
// partial result.
func ParseVariableValues(vv map[string]UnparsedVariableValue, decls map[string]*configs.Variable) (terraform.InputValues, tfdiags.Diagnostics) {
var diags tfdiags.Diagnostics
ret := make(terraform.InputValues, len(vv))
ret, diags := ParseDeclaredVariableValues(vv, decls)
undeclared, diagsUndeclared := ParseUndeclaredVariableValues(vv, decls)
// Currently we're generating only warnings for undeclared variables
// defined in files (see below) but we only want to generate a few warnings
// at a time because existing deployments may have lots of these and
// the result can therefore be overwhelming.
seenUndeclaredInFile := 0
for name, rv := range vv {
var mode configs.VariableParsingMode
config, declared := decls[name]
if declared {
mode = config.ParsingMode
} else {
mode = configs.VariableParseLiteral
}
val, valDiags := rv.ParseVariableValue(mode)
diags = diags.Append(valDiags)
if valDiags.HasErrors() {
continue
}
if !declared {
switch val.SourceType {
case terraform.ValueFromConfig, terraform.ValueFromAutoFile, terraform.ValueFromNamedFile:
// We allow undeclared names for variable values from files and warn in case
// users have forgotten a variable {} declaration or have a typo in their var name.
// Some users will actively ignore this warning because they use a .tfvars file
// across multiple configurations.
if seenUndeclaredInFile < 2 {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Warning,
"Value for undeclared variable",
fmt.Sprintf("The root module does not declare a variable named %q but a value was found in file %q. If you meant to use this value, add a \"variable\" block to the configuration.\n\nTo silence these warnings, use TF_VAR_... environment variables to provide certain \"global\" settings to all configurations in your organization. To reduce the verbosity of these warnings, use the -compact-warnings option.", name, val.SourceRange.Filename),
))
}
seenUndeclaredInFile++
case terraform.ValueFromEnvVar:
// We allow and ignore undeclared names for environment
// variables, because users will often set these globally
// when they are used across many (but not necessarily all)
// configurations.
case terraform.ValueFromCLIArg:
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Value for undeclared variable",
fmt.Sprintf("A variable named %q was assigned on the command line, but the root module does not declare a variable of that name. To use this value, add a \"variable\" block to the configuration.", name),
))
default:
// For all other source types we are more vague, but other situations
// don't generally crop up at this layer in practice.
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Value for undeclared variable",
fmt.Sprintf("A variable named %q was assigned a value, but the root module does not declare a variable of that name. To use this value, add a \"variable\" block to the configuration.", name),
))
}
continue
}
ret[name] = val
}
if seenUndeclaredInFile > 2 {
extras := seenUndeclaredInFile - 2
diags = diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagWarning,
Summary: "Values for undeclared variables",
Detail: fmt.Sprintf("In addition to the other similar warnings shown, %d other variable(s) defined without being declared.", extras),
})
}
diags = diags.Append(diagsUndeclared)
// By this point we should've gathered all of the required root module
// variables from one of the many possible sources. We'll now populate
// any we haven't gathered as their defaults and fail if any of the
// missing ones are required.
for name, vc := range decls {
if _, defined := ret[name]; defined {
if isDefinedAny(name, ret, undeclared) {
continue
}

206
internal/backend/unparsed_value_test.go
View File

@ -13,7 +13,7 @@ import (
"github.com/hashicorp/terraform/internal/tfdiags"
)
func TestParseVariableValuesUndeclared(t *testing.T) {
func TestUnparsedValue(t *testing.T) {
vv := map[string]UnparsedVariableValue{
"undeclared0": testUnparsedVariableValue("0"),
"undeclared1": testUnparsedVariableValue("1"),
@ -59,66 +59,164 @@ func TestParseVariableValuesUndeclared(t *testing.T) {
},
}
gotVals, diags := ParseVariableValues(vv, decls)
for _, diag := range diags {
t.Logf("%s: %s", diag.Description().Summary, diag.Description().Detail)
}
if got, want := len(diags), 4; got != want {
t.Fatalf("wrong number of diagnostics %d; want %d", got, want)
}
const undeclSingular = `Value for undeclared variable`
const undeclPlural = `Values for undeclared variables`
const missingRequired = `No value for required variable`
if got, want := diags[0].Description().Summary, undeclSingular; got != want {
t.Errorf("wrong summary for diagnostic 0\ngot: %s\nwant: %s", got, want)
}
if got, want := diags[1].Description().Summary, undeclSingular; got != want {
t.Errorf("wrong summary for diagnostic 1\ngot: %s\nwant: %s", got, want)
}
if got, want := diags[2].Description().Summary, undeclPlural; got != want {
t.Errorf("wrong summary for diagnostic 2\ngot: %s\nwant: %s", got, want)
}
if got, want := diags[2].Description().Detail, "3 other variable(s)"; !strings.Contains(got, want) {
t.Errorf("wrong detail for diagnostic 2\ngot: %s\nmust contain: %s", got, want)
}
if got, want := diags[3].Description().Summary, missingRequired; got != want {
t.Errorf("wrong summary for diagnostic 3\ngot: %s\nwant: %s", got, want)
}
t.Run("ParseDeclaredVariableValues", func(t *testing.T) {
gotVals, diags := ParseDeclaredVariableValues(vv, decls)
wantVals := terraform.InputValues{
"declared1": {
Value: cty.StringVal("5"),
SourceType: terraform.ValueFromNamedFile,
SourceRange: tfdiags.SourceRange{
Filename: "fake.tfvars",
Start: tfdiags.SourcePos{Line: 1, Column: 1, Byte: 0},
End: tfdiags.SourcePos{Line: 1, Column: 1, Byte: 0},
if got, want := len(diags), 0; got != want {
t.Fatalf("wrong number of diagnostics %d; want %d", got, want)
}
wantVals := terraform.InputValues{
"declared1": {
Value: cty.StringVal("5"),
SourceType: terraform.ValueFromNamedFile,
SourceRange: tfdiags.SourceRange{
Filename: "fake.tfvars",
Start: tfdiags.SourcePos{Line: 1, Column: 1, Byte: 0},
End: tfdiags.SourcePos{Line: 1, Column: 1, Byte: 0},
},
},
},
"missing1": {
Value: cty.DynamicVal,
SourceType: terraform.ValueFromConfig,
SourceRange: tfdiags.SourceRange{
Filename: "fake.tf",
Start: tfdiags.SourcePos{Line: 3, Column: 1, Byte: 0},
End: tfdiags.SourcePos{Line: 3, Column: 1, Byte: 0},
}
if diff := cmp.Diff(wantVals, gotVals, cmp.Comparer(cty.Value.RawEquals)); diff != "" {
t.Errorf("wrong result\n%s", diff)
}
})
t.Run("ParseUndeclaredVariableValues", func(t *testing.T) {
gotVals, diags := ParseUndeclaredVariableValues(vv, decls)
if got, want := len(diags), 3; got != want {
t.Fatalf("wrong number of diagnostics %d; want %d", got, want)
}
if got, want := diags[0].Description().Summary, undeclSingular; got != want {
t.Errorf("wrong summary for diagnostic 0\ngot: %s\nwant: %s", got, want)
}
if got, want := diags[1].Description().Summary, undeclSingular; got != want {
t.Errorf("wrong summary for diagnostic 1\ngot: %s\nwant: %s", got, want)
}
if got, want := diags[2].Description().Summary, undeclPlural; got != want {
t.Errorf("wrong summary for diagnostic 2\ngot: %s\nwant: %s", got, want)
}
wantVals := terraform.InputValues{
"undeclared0": {
Value: cty.StringVal("0"),
SourceType: terraform.ValueFromNamedFile,
SourceRange: tfdiags.SourceRange{
Filename: "fake.tfvars",
Start: tfdiags.SourcePos{Line: 1, Column: 1},
End: tfdiags.SourcePos{Line: 1, Column: 1},
},
},
},
"missing2": {
Value: cty.StringVal("default for missing2"),
SourceType: terraform.ValueFromConfig,
SourceRange: tfdiags.SourceRange{
Filename: "fake.tf",
Start: tfdiags.SourcePos{Line: 4, Column: 1, Byte: 0},
End: tfdiags.SourcePos{Line: 4, Column: 1, Byte: 0},
"undeclared1": {
Value: cty.StringVal("1"),
SourceType: terraform.ValueFromNamedFile,
SourceRange: tfdiags.SourceRange{
Filename: "fake.tfvars",
Start: tfdiags.SourcePos{Line: 1, Column: 1},
End: tfdiags.SourcePos{Line: 1, Column: 1},
},
},
},
}
if diff := cmp.Diff(wantVals, gotVals, cmp.Comparer(cty.Value.RawEquals)); diff != "" {
t.Errorf("wrong result\n%s", diff)
}
"undeclared2": {
Value: cty.StringVal("2"),
SourceType: terraform.ValueFromNamedFile,
SourceRange: tfdiags.SourceRange{
Filename: "fake.tfvars",
Start: tfdiags.SourcePos{Line: 1, Column: 1},
End: tfdiags.SourcePos{Line: 1, Column: 1},
},
},
"undeclared3": {
Value: cty.StringVal("3"),
SourceType: terraform.ValueFromNamedFile,
SourceRange: tfdiags.SourceRange{
Filename: "fake.tfvars",
Start: tfdiags.SourcePos{Line: 1, Column: 1},
End: tfdiags.SourcePos{Line: 1, Column: 1},
},
},
"undeclared4": {
Value: cty.StringVal("4"),
SourceType: terraform.ValueFromNamedFile,
SourceRange: tfdiags.SourceRange{
Filename: "fake.tfvars",
Start: tfdiags.SourcePos{Line: 1, Column: 1},
End: tfdiags.SourcePos{Line: 1, Column: 1},
},
},
}
if diff := cmp.Diff(wantVals, gotVals, cmp.Comparer(cty.Value.RawEquals)); diff != "" {
t.Errorf("wrong result\n%s", diff)
}
})
t.Run("ParseVariableValues", func(t *testing.T) {
gotVals, diags := ParseVariableValues(vv, decls)
for _, diag := range diags {
t.Logf("%s: %s", diag.Description().Summary, diag.Description().Detail)
}
if got, want := len(diags), 4; got != want {
t.Fatalf("wrong number of diagnostics %d; want %d", got, want)
}
const missingRequired = `No value for required variable`
if got, want := diags[0].Description().Summary, undeclSingular; got != want {
t.Errorf("wrong summary for diagnostic 0\ngot: %s\nwant: %s", got, want)
}
if got, want := diags[1].Description().Summary, undeclSingular; got != want {
t.Errorf("wrong summary for diagnostic 1\ngot: %s\nwant: %s", got, want)
}
if got, want := diags[2].Description().Summary, undeclPlural; got != want {
t.Errorf("wrong summary for diagnostic 2\ngot: %s\nwant: %s", got, want)
}
if got, want := diags[2].Description().Detail, "3 other variable(s)"; !strings.Contains(got, want) {
t.Errorf("wrong detail for diagnostic 2\ngot: %s\nmust contain: %s", got, want)
}
if got, want := diags[3].Description().Summary, missingRequired; got != want {
t.Errorf("wrong summary for diagnostic 3\ngot: %s\nwant: %s", got, want)
}
wantVals := terraform.InputValues{
"declared1": {
Value: cty.StringVal("5"),
SourceType: terraform.ValueFromNamedFile,
SourceRange: tfdiags.SourceRange{
Filename: "fake.tfvars",
Start: tfdiags.SourcePos{Line: 1, Column: 1, Byte: 0},
End: tfdiags.SourcePos{Line: 1, Column: 1, Byte: 0},
},
},
"missing1": {
Value: cty.DynamicVal,
SourceType: terraform.ValueFromConfig,
SourceRange: tfdiags.SourceRange{
Filename: "fake.tf",
Start: tfdiags.SourcePos{Line: 3, Column: 1, Byte: 0},
End: tfdiags.SourcePos{Line: 3, Column: 1, Byte: 0},
},
},
"missing2": {
Value: cty.StringVal("default for missing2"),
SourceType: terraform.ValueFromConfig,
SourceRange: tfdiags.SourceRange{
Filename: "fake.tf",
Start: tfdiags.SourcePos{Line: 4, Column: 1, Byte: 0},
End: tfdiags.SourcePos{Line: 4, Column: 1, Byte: 0},
},
},
}
if diff := cmp.Diff(wantVals, gotVals, cmp.Comparer(cty.Value.RawEquals)); diff != "" {
t.Errorf("wrong result\n%s", diff)
}
})
}
type testUnparsedVariableValue string

1029
internal/cloud/backend.go Normal file
View File

File diff suppressed because it is too large Load Diff

182
internal/cloud/backend_apply.go Normal file
View File

@ -0,0 +1,182 @@
package cloud
import (
"bufio"
"context"
"io"
"log"
tfe "github.com/hashicorp/go-tfe"
"github.com/hashicorp/terraform/internal/backend"
"github.com/hashicorp/terraform/internal/plans"
"github.com/hashicorp/terraform/internal/terraform"
"github.com/hashicorp/terraform/internal/tfdiags"
)
func (b *Cloud) opApply(stopCtx, cancelCtx context.Context, op *backend.Operation, w *tfe.Workspace) (*tfe.Run, error) {
log.Printf("[INFO] cloud: starting Apply operation")
var diags tfdiags.Diagnostics
// We should remove the `CanUpdate` part of this test, but for now
// (to remain compatible with tfe.v2.1) we'll leave it in here.
if !w.Permissions.CanUpdate && !w.Permissions.CanQueueApply {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Insufficient rights to apply changes",
"The provided credentials have insufficient rights to apply changes. In order "+
"to apply changes at least write permissions on the workspace are required.",
))
return nil, diags.Err()
}
if w.VCSRepo != nil {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Apply not allowed for workspaces with a VCS connection",
"A workspace that is connected to a VCS requires the VCS-driven workflow "+
"to ensure that the VCS remains the single source of truth.",
))
return nil, diags.Err()
}
if b.ContextOpts != nil && b.ContextOpts.Parallelism != defaultParallelism {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Custom parallelism values are currently not supported",
`Terraform Cloud does not support setting a custom parallelism `+
`value at this time.`,
))
}
if op.PlanFile != nil {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Applying a saved plan is currently not supported",
`Terraform Cloud currently requires configuration to be present and `+
`does not accept an existing saved plan as an argument at this time.`,
))
}
if !op.HasConfig() && op.PlanMode != plans.DestroyMode {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"No configuration files found",
`Apply requires configuration to be present. Applying without a configuration `+
`would mark everything for destruction, which is normally not what is desired. `+
`If you would like to destroy everything, please run 'terraform destroy' which `+
`does not require any configuration files.`,
))
}
// Return if there are any errors.
if diags.HasErrors() {
return nil, diags.Err()
}
// Run the plan phase.
r, err := b.plan(stopCtx, cancelCtx, op, w)
if err != nil {
return r, err
}
// This check is also performed in the plan method to determine if
// the policies should be checked, but we need to check the values
// here again to determine if we are done and should return.
if !r.HasChanges || r.Status == tfe.RunCanceled || r.Status == tfe.RunErrored {
return r, nil
}
// Retrieve the run to get its current status.
r, err = b.client.Runs.Read(stopCtx, r.ID)
if err != nil {
return r, generalError("Failed to retrieve run", err)
}
// Return if the run cannot be confirmed.
if !op.AutoApprove && !r.Actions.IsConfirmable {
return r, nil
}
mustConfirm := (op.UIIn != nil && op.UIOut != nil) && !op.AutoApprove
if mustConfirm {
opts := &terraform.InputOpts{Id: "approve"}
if op.PlanMode == plans.DestroyMode {
opts.Query = "\nDo you really want to destroy all resources in workspace \"" + op.Workspace + "\"?"
opts.Description = "Terraform will destroy all your managed infrastructure, as shown above.\n" +
"There is no undo. Only 'yes' will be accepted to confirm."
} else {
opts.Query = "\nDo you want to perform these actions in workspace \"" + op.Workspace + "\"?"
opts.Description = "Terraform will perform the actions described above.\n" +
"Only 'yes' will be accepted to approve."
}
err = b.confirm(stopCtx, op, opts, r, "yes")
if err != nil && err != errRunApproved {
return r, err
}
} else {
// If we don't need to ask for confirmation, insert a blank
// line to separate the ouputs.
if b.CLI != nil {
b.CLI.Output("")
}
}
if !op.AutoApprove && err != errRunApproved {
if err = b.client.Runs.Apply(stopCtx, r.ID, tfe.RunApplyOptions{}); err != nil {
return r, generalError("Failed to approve the apply command", err)
}
}
r, err = b.waitForRun(stopCtx, cancelCtx, op, "apply", r, w)
if err != nil {
return r, err
}
logs, err := b.client.Applies.Logs(stopCtx, r.Apply.ID)
if err != nil {
return r, generalError("Failed to retrieve logs", err)
}
reader := bufio.NewReaderSize(logs, 64*1024)
if b.CLI != nil {
skip := 0
for next := true; next; {
var l, line []byte
for isPrefix := true; isPrefix; {
l, isPrefix, err = reader.ReadLine()
if err != nil {
if err != io.EOF {
return r, generalError("Failed to read logs", err)
}
next = false
}
line = append(line, l...)
}
// Skip the first 3 lines to prevent duplicate output.
if skip < 3 {
skip++
continue
}
if next || len(line) > 0 {
b.CLI.Output(b.Colorize().Color(string(line)))
}
}
}
return r, nil
}
const applyDefaultHeader = `
[reset][yellow]Running apply in Terraform Cloud. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.[reset]
Preparing the remote apply...
`

1612
internal/cloud/backend_apply_test.go Normal file
View File

File diff suppressed because it is too large Load Diff

21
internal/cloud/backend_cli.go Normal file
View File

@ -0,0 +1,21 @@
package cloud
import (
"github.com/hashicorp/terraform/internal/backend"
)
// CLIInit implements backend.CLI
func (b *Cloud) CLIInit(opts *backend.CLIOpts) error {
if cli, ok := b.local.(backend.CLI); ok {
if err := cli.CLIInit(opts); err != nil {
return err
}
}
b.CLI = opts.CLI
b.CLIColor = opts.CLIColor
b.ContextOpts = opts.ContextOpts
b.runningInAutomation = opts.RunningInAutomation
return nil
}

50
internal/cloud/backend_colorize.go Normal file
View File

@ -0,0 +1,50 @@
package cloud
import (
"regexp"
"github.com/mitchellh/colorstring"
)
// TODO SvH: This file should be deleted and the type cliColorize should be
// renamed back to Colorize as soon as we can pass -no-color to the backend.
// colorsRe is used to find ANSI escaped color codes.
var colorsRe = regexp.MustCompile("\033\\[\\d{1,3}m")
// Colorer is the interface that must be implemented to colorize strings.
type Colorer interface {
Color(v string) string
}
// Colorize is used to print output when the -no-color flag is used. It will
// strip all ANSI escaped color codes which are set while the operation was
// executed in Terraform Enterprise.
//
// When Terraform Enterprise supports run specific variables, this code can be
// removed as we can then pass the CLI flag to the backend and prevent the color
// codes from being written to the output.
type Colorize struct {
cliColor *colorstring.Colorize
}
// Color will strip all ANSI escaped color codes and return a uncolored string.
func (c *Colorize) Color(v string) string {
return colorsRe.ReplaceAllString(c.cliColor.Color(v), "")
}
// Colorize returns the Colorize structure that can be used for colorizing
// output. This is guaranteed to always return a non-nil value and so is useful
// as a helper to wrap any potentially colored strings.
func (b *Cloud) Colorize() Colorer {
if b.CLIColor != nil && !b.CLIColor.Disable {
return b.CLIColor
}
if b.CLIColor != nil {
return &Colorize{cliColor: b.CLIColor}
}
return &Colorize{cliColor: &colorstring.Colorize{
Colors: colorstring.DefaultColors,
Disable: true,
}}
}

536
internal/cloud/backend_common.go Normal file
View File

@ -0,0 +1,536 @@
package cloud
import (
"bufio"
"context"
"errors"
"fmt"
"io"
"math"
"strconv"
"strings"
"time"
tfe "github.com/hashicorp/go-tfe"
"github.com/hashicorp/terraform/internal/backend"
"github.com/hashicorp/terraform/internal/plans"
"github.com/hashicorp/terraform/internal/terraform"
)
var (
errApplyDiscarded = errors.New("Apply discarded.")
errDestroyDiscarded = errors.New("Destroy discarded.")
errRunApproved = errors.New("approved using the UI or API")
errRunDiscarded = errors.New("discarded using the UI or API")
errRunOverridden = errors.New("overridden using the UI or API")
)
var (
backoffMin = 1000.0
backoffMax = 3000.0
runPollInterval = 3 * time.Second
)
// backoff will perform exponential backoff based on the iteration and
// limited by the provided min and max (in milliseconds) durations.
func backoff(min, max float64, iter int) time.Duration {
backoff := math.Pow(2, float64(iter)/5) * min
if backoff > max {
backoff = max
}
return time.Duration(backoff) * time.Millisecond
}
func (b *Cloud) waitForRun(stopCtx, cancelCtx context.Context, op *backend.Operation, opType string, r *tfe.Run, w *tfe.Workspace) (*tfe.Run, error) {
started := time.Now()
updated := started
for i := 0; ; i++ {
select {
case <-stopCtx.Done():
return r, stopCtx.Err()
case <-cancelCtx.Done():
return r, cancelCtx.Err()
case <-time.After(backoff(backoffMin, backoffMax, i)):
// Timer up, show status
}
// Retrieve the run to get its current status.
r, err := b.client.Runs.Read(stopCtx, r.ID)
if err != nil {
return r, generalError("Failed to retrieve run", err)
}
// Return if the run is no longer pending.
if r.Status != tfe.RunPending && r.Status != tfe.RunConfirmed {
if i == 0 && opType == "plan" && b.CLI != nil {
b.CLI.Output(b.Colorize().Color(fmt.Sprintf("Waiting for the %s to start...\n", opType)))
}
if i > 0 && b.CLI != nil {
// Insert a blank line to separate the ouputs.
b.CLI.Output("")
}
return r, nil
}
// Check if 30 seconds have passed since the last update.
current := time.Now()
if b.CLI != nil && (i == 0 || current.Sub(updated).Seconds() > 30) {
updated = current
position := 0
elapsed := ""
// Calculate and set the elapsed time.
if i > 0 {
elapsed = fmt.Sprintf(
" (%s elapsed)", current.Sub(started).Truncate(30*time.Second))
}
// Retrieve the workspace used to run this operation in.
w, err = b.client.Workspaces.Read(stopCtx, b.organization, w.Name)
if err != nil {
return nil, generalError("Failed to retrieve workspace", err)
}
// If the workspace is locked the run will not be queued and we can
// update the status without making any expensive calls.
if w.Locked && w.CurrentRun != nil {
cr, err := b.client.Runs.Read(stopCtx, w.CurrentRun.ID)
if err != nil {
return r, generalError("Failed to retrieve current run", err)
}
if cr.Status == tfe.RunPending {
b.CLI.Output(b.Colorize().Color(
"Waiting for the manually locked workspace to be unlocked..." + elapsed))
continue
}
}
// Skip checking the workspace queue when we are the current run.
if w.CurrentRun == nil || w.CurrentRun.ID != r.ID {
found := false
options := tfe.RunListOptions{}
runlist:
for {
rl, err := b.client.Runs.List(stopCtx, w.ID, options)
if err != nil {
return r, generalError("Failed to retrieve run list", err)
}
// Loop through all runs to calculate the workspace queue position.
for _, item := range rl.Items {
if !found {
if r.ID == item.ID {
found = true
}
continue
}
// If the run is in a final state, ignore it and continue.
switch item.Status {
case tfe.RunApplied, tfe.RunCanceled, tfe.RunDiscarded, tfe.RunErrored:
continue
case tfe.RunPlanned:
if op.Type == backend.OperationTypePlan {
continue
}
}
// Increase the workspace queue position.
position++
// Stop searching when we reached the current run.
if w.CurrentRun != nil && w.CurrentRun.ID == item.ID {
break runlist
}
}
// Exit the loop when we've seen all pages.
if rl.CurrentPage >= rl.TotalPages {
break
}
// Update the page number to get the next page.
options.PageNumber = rl.NextPage
}
if position > 0 {
b.CLI.Output(b.Colorize().Color(fmt.Sprintf(
"Waiting for %d run(s) to finish before being queued...%s",
position,
elapsed,
)))
continue
}
}
options := tfe.RunQueueOptions{}
search:
for {
rq, err := b.client.Organizations.RunQueue(stopCtx, b.organization, options)
if err != nil {
return r, generalError("Failed to retrieve queue", err)
}
// Search through all queued items to find our run.
for _, item := range rq.Items {
if r.ID == item.ID {
position = item.PositionInQueue
break search
}
}
// Exit the loop when we've seen all pages.
if rq.CurrentPage >= rq.TotalPages {
break
}
// Update the page number to get the next page.
options.PageNumber = rq.NextPage
}
if position > 0 {
c, err := b.client.Organizations.Capacity(stopCtx, b.organization)
if err != nil {
return r, generalError("Failed to retrieve capacity", err)
}
b.CLI.Output(b.Colorize().Color(fmt.Sprintf(
"Waiting for %d queued run(s) to finish before starting...%s",
position-c.Running,
elapsed,
)))
continue
}
b.CLI.Output(b.Colorize().Color(fmt.Sprintf(
"Waiting for the %s to start...%s", opType, elapsed)))
}
}
}
func (b *Cloud) costEstimate(stopCtx, cancelCtx context.Context, op *backend.Operation, r *tfe.Run) error {
if r.CostEstimate == nil {
return nil
}
msgPrefix := "Cost estimation"
started := time.Now()
updated := started
for i := 0; ; i++ {
select {
case <-stopCtx.Done():
return stopCtx.Err()
case <-cancelCtx.Done():
return cancelCtx.Err()
case <-time.After(backoff(backoffMin, backoffMax, i)):
}
// Retrieve the cost estimate to get its current status.
ce, err := b.client.CostEstimates.Read(stopCtx, r.CostEstimate.ID)
if err != nil {
return generalError("Failed to retrieve cost estimate", err)
}
// If the run is canceled or errored, but the cost-estimate still has
// no result, there is nothing further to render.
if ce.Status != tfe.CostEstimateFinished {
if r.Status == tfe.RunCanceled || r.Status == tfe.RunErrored {
return nil
}
}
// checking if i == 0 so as to avoid printing this starting horizontal-rule
// every retry, and that it only prints it on the first (i=0) attempt.
if b.CLI != nil && i == 0 {
b.CLI.Output("\n------------------------------------------------------------------------\n")
}
switch ce.Status {
case tfe.CostEstimateFinished:
delta, err := strconv.ParseFloat(ce.DeltaMonthlyCost, 64)
if err != nil {
return generalError("Unexpected error", err)
}
sign := "+"
if delta < 0 {
sign = "-"
}
deltaRepr := strings.Replace(ce.DeltaMonthlyCost, "-", "", 1)
if b.CLI != nil {
b.CLI.Output(b.Colorize().Color(msgPrefix + ":\n"))
b.CLI.Output(b.Colorize().Color(fmt.Sprintf("Resources: %d of %d estimated", ce.MatchedResourcesCount, ce.ResourcesCount)))
b.CLI.Output(b.Colorize().Color(fmt.Sprintf(" $%s/mo %s$%s", ce.ProposedMonthlyCost, sign, deltaRepr)))
if len(r.PolicyChecks) == 0 && r.HasChanges && op.Type == backend.OperationTypeApply {
b.CLI.Output("\n------------------------------------------------------------------------")
}
}
return nil
case tfe.CostEstimatePending, tfe.CostEstimateQueued:
// Check if 30 seconds have passed since the last update.
current := time.Now()
if b.CLI != nil && (i == 0 || current.Sub(updated).Seconds() > 30) {
updated = current
elapsed := ""
// Calculate and set the elapsed time.
if i > 0 {
elapsed = fmt.Sprintf(
" (%s elapsed)", current.Sub(started).Truncate(30*time.Second))
}
b.CLI.Output(b.Colorize().Color(msgPrefix + ":\n"))
b.CLI.Output(b.Colorize().Color("Waiting for cost estimate to complete..." + elapsed + "\n"))
}
continue
case tfe.CostEstimateSkippedDueToTargeting:
b.CLI.Output(b.Colorize().Color(msgPrefix + ":\n"))
b.CLI.Output("Not available for this plan, because it was created with the -target option.")
b.CLI.Output("\n------------------------------------------------------------------------")
return nil
case tfe.CostEstimateErrored:
b.CLI.Output(msgPrefix + " errored.\n")
b.CLI.Output("\n------------------------------------------------------------------------")
return nil
case tfe.CostEstimateCanceled:
return fmt.Errorf(msgPrefix + " canceled.")
default:
return fmt.Errorf("Unknown or unexpected cost estimate state: %s", ce.Status)
}
}
}
func (b *Cloud) checkPolicy(stopCtx, cancelCtx context.Context, op *backend.Operation, r *tfe.Run) error {
if b.CLI != nil {
b.CLI.Output("\n------------------------------------------------------------------------\n")
}
for i, pc := range r.PolicyChecks {
// Read the policy check logs. This is a blocking call that will only
// return once the policy check is complete.
logs, err := b.client.PolicyChecks.Logs(stopCtx, pc.ID)
if err != nil {
return generalError("Failed to retrieve policy check logs", err)
}
reader := bufio.NewReaderSize(logs, 64*1024)
// Retrieve the policy check to get its current status.
pc, err := b.client.PolicyChecks.Read(stopCtx, pc.ID)
if err != nil {
return generalError("Failed to retrieve policy check", err)
}
// If the run is canceled or errored, but the policy check still has
// no result, there is nothing further to render.
if r.Status == tfe.RunCanceled || r.Status == tfe.RunErrored {
switch pc.Status {
case tfe.PolicyPending, tfe.PolicyQueued, tfe.PolicyUnreachable:
continue
}
}
var msgPrefix string
switch pc.Scope {
case tfe.PolicyScopeOrganization:
msgPrefix = "Organization policy check"
case tfe.PolicyScopeWorkspace:
msgPrefix = "Workspace policy check"
default:
msgPrefix = fmt.Sprintf("Unknown policy check (%s)", pc.Scope)
}
if b.CLI != nil {
b.CLI.Output(b.Colorize().Color(msgPrefix + ":\n"))
}
if b.CLI != nil {
for next := true; next; {
var l, line []byte
for isPrefix := true; isPrefix; {
l, isPrefix, err = reader.ReadLine()
if err != nil {
if err != io.EOF {
return generalError("Failed to read logs", err)
}
next = false
}
line = append(line, l...)
}
if next || len(line) > 0 {
b.CLI.Output(b.Colorize().Color(string(line)))
}
}
}
switch pc.Status {
case tfe.PolicyPasses:
if (r.HasChanges && op.Type == backend.OperationTypeApply || i < len(r.PolicyChecks)-1) && b.CLI != nil {
b.CLI.Output("\n------------------------------------------------------------------------")
}
continue
case tfe.PolicyErrored:
return fmt.Errorf(msgPrefix + " errored.")
case tfe.PolicyHardFailed:
return fmt.Errorf(msgPrefix + " hard failed.")
case tfe.PolicySoftFailed:
runUrl := fmt.Sprintf(runHeader, b.hostname, b.organization, op.Workspace, r.ID)
if op.Type == backend.OperationTypePlan || op.UIOut == nil || op.UIIn == nil ||
!pc.Actions.IsOverridable || !pc.Permissions.CanOverride {
return fmt.Errorf(msgPrefix + " soft failed.\n" + runUrl)
}
if op.AutoApprove {
if _, err = b.client.PolicyChecks.Override(stopCtx, pc.ID); err != nil {
return generalError(fmt.Sprintf("Failed to override policy check.\n%s", runUrl), err)
}
} else {
opts := &terraform.InputOpts{
Id: "override",
Query: "\nDo you want to override the soft failed policy check?",
Description: "Only 'override' will be accepted to override.",
}
err = b.confirm(stopCtx, op, opts, r, "override")
if err != nil && err != errRunOverridden {
return fmt.Errorf(
fmt.Sprintf("Failed to override: %s\n%s\n", err.Error(), runUrl),
)
}
if err != errRunOverridden {
if _, err = b.client.PolicyChecks.Override(stopCtx, pc.ID); err != nil {
return generalError(fmt.Sprintf("Failed to override policy check.\n%s", runUrl), err)
}
} else {
b.CLI.Output(fmt.Sprintf("The run needs to be manually overridden or discarded.\n%s\n", runUrl))
}
}
if b.CLI != nil {
b.CLI.Output("------------------------------------------------------------------------")
}
default:
return fmt.Errorf("Unknown or unexpected policy state: %s", pc.Status)
}
}
return nil
}
func (b *Cloud) confirm(stopCtx context.Context, op *backend.Operation, opts *terraform.InputOpts, r *tfe.Run, keyword string) error {
doneCtx, cancel := context.WithCancel(stopCtx)
result := make(chan error, 2)
go func() {
// Make sure we cancel doneCtx before we return
// so the input command is also canceled.
defer cancel()
for {
select {
case <-doneCtx.Done():
return
case <-stopCtx.Done():
return
case <-time.After(runPollInterval):
// Retrieve the run again to get its current status.
r, err := b.client.Runs.Read(stopCtx, r.ID)
if err != nil {
result <- generalError("Failed to retrieve run", err)
return
}
switch keyword {
case "override":
if r.Status != tfe.RunPolicyOverride {
if r.Status == tfe.RunDiscarded {
err = errRunDiscarded
} else {
err = errRunOverridden
}
}
case "yes":
if !r.Actions.IsConfirmable {
if r.Status == tfe.RunDiscarded {
err = errRunDiscarded
} else {
err = errRunApproved
}
}
}
if err != nil {
if b.CLI != nil {
b.CLI.Output(b.Colorize().Color(
fmt.Sprintf("[reset][yellow]%s[reset]", err.Error())))
}
if err == errRunDiscarded {
err = errApplyDiscarded
if op.PlanMode == plans.DestroyMode {
err = errDestroyDiscarded
}
}
result <- err
return
}
}
}
}()
result <- func() error {
v, err := op.UIIn.Input(doneCtx, opts)
if err != nil && err != context.Canceled && stopCtx.Err() != context.Canceled {
return fmt.Errorf("Error asking %s: %v", opts.Id, err)
}
// We return the error of our parent channel as we don't
// care about the error of the doneCtx which is only used
// within this function. So if the doneCtx was canceled
// because stopCtx was canceled, this will properly return
// a context.Canceled error and otherwise it returns nil.
if doneCtx.Err() == context.Canceled || stopCtx.Err() == context.Canceled {
return stopCtx.Err()
}
// Make sure we cancel the context here so the loop that
// checks for external changes to the run is ended before
// we start to make changes ourselves.
cancel()
if v != keyword {
// Retrieve the run again to get its current status.
r, err = b.client.Runs.Read(stopCtx, r.ID)
if err != nil {
return generalError("Failed to retrieve run", err)
}
// Make sure we discard the run if possible.
if r.Actions.IsDiscardable {
err = b.client.Runs.Discard(stopCtx, r.ID, tfe.RunDiscardOptions{})
if err != nil {
if op.PlanMode == plans.DestroyMode {
return generalError("Failed to discard destroy", err)
}
return generalError("Failed to discard apply", err)
}
}
// Even if the run was discarded successfully, we still
// return an error as the apply command was canceled.
if op.PlanMode == plans.DestroyMode {
return errDestroyDiscarded
}
return errApplyDiscarded
}
return nil
}()
return <-result
}

279
internal/cloud/backend_context.go Normal file
View File

@ -0,0 +1,279 @@
package cloud
import (
"context"
"fmt"
"log"
tfe "github.com/hashicorp/go-tfe"
"github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/hclsyntax"
"github.com/hashicorp/terraform/internal/backend"
"github.com/hashicorp/terraform/internal/configs"
"github.com/hashicorp/terraform/internal/states/statemgr"
"github.com/hashicorp/terraform/internal/terraform"
"github.com/hashicorp/terraform/internal/tfdiags"
"github.com/zclconf/go-cty/cty"
)
// LocalRun implements backend.Local
func (b *Cloud) LocalRun(op *backend.Operation) (*backend.LocalRun, statemgr.Full, tfdiags.Diagnostics) {
var diags tfdiags.Diagnostics
ret := &backend.LocalRun{
PlanOpts: &terraform.PlanOpts{
Mode: op.PlanMode,
Targets: op.Targets,
},
}
op.StateLocker = op.StateLocker.WithContext(context.Background())
// Get the remote workspace name.
remoteWorkspaceName := b.getRemoteWorkspaceName(op.Workspace)
// Get the latest state.
log.Printf("[TRACE] cloud: requesting state manager for workspace %q", remoteWorkspaceName)
stateMgr, err := b.StateMgr(op.Workspace)
if err != nil {
diags = diags.Append(fmt.Errorf("error loading state: %w", err))
return nil, nil, diags
}
log.Printf("[TRACE] cloud: requesting state lock for workspace %q", remoteWorkspaceName)
if diags := op.StateLocker.Lock(stateMgr, op.Type.String()); diags.HasErrors() {
return nil, nil, diags
}
defer func() {
// If we're returning with errors, and thus not producing a valid
// context, we'll want to avoid leaving the remote workspace locked.
if diags.HasErrors() {
diags = diags.Append(op.StateLocker.Unlock())
}
}()
log.Printf("[TRACE] cloud: reading remote state for workspace %q", remoteWorkspaceName)
if err := stateMgr.RefreshState(); err != nil {
diags = diags.Append(fmt.Errorf("error loading state: %w", err))
return nil, nil, diags
}
// Initialize our context options
var opts terraform.ContextOpts
if v := b.ContextOpts; v != nil {
opts = *v
}
// Copy set options from the operation
opts.UIInput = op.UIIn
// Load the latest state. If we enter contextFromPlanFile below then the
// state snapshot in the plan file must match this, or else it'll return
// error diagnostics.
log.Printf("[TRACE] cloud: retrieving remote state snapshot for workspace %q", remoteWorkspaceName)
ret.InputState = stateMgr.State()
log.Printf("[TRACE] cloud: loading configuration for the current working directory")
config, configDiags := op.ConfigLoader.LoadConfig(op.ConfigDir)
diags = diags.Append(configDiags)
if configDiags.HasErrors() {
return nil, nil, diags
}
ret.Config = config
if op.AllowUnsetVariables {
// If we're not going to use the variables in an operation we'll be
// more lax about them, stubbing out any unset ones as unknown.
// This gives us enough information to produce a consistent context,
// but not enough information to run a real operation (plan, apply, etc)
ret.PlanOpts.SetVariables = stubAllVariables(op.Variables, config.Module.Variables)
} else {
// The underlying API expects us to use the opaque workspace id to request
// variables, so we'll need to look that up using our organization name
// and workspace name.
remoteWorkspaceID, err := b.getRemoteWorkspaceID(context.Background(), op.Workspace)
if err != nil {
diags = diags.Append(fmt.Errorf("error finding remote workspace: %w", err))
return nil, nil, diags
}
log.Printf("[TRACE] cloud: retrieving variables from workspace %s/%s (%s)", remoteWorkspaceName, b.organization, remoteWorkspaceID)
tfeVariables, err := b.client.Variables.List(context.Background(), remoteWorkspaceID, tfe.VariableListOptions{})
if err != nil && err != tfe.ErrResourceNotFound {
diags = diags.Append(fmt.Errorf("error loading variables: %w", err))
return nil, nil, diags
}
if tfeVariables != nil {
if op.Variables == nil {
op.Variables = make(map[string]backend.UnparsedVariableValue)
}
for _, v := range tfeVariables.Items {
if v.Category == tfe.CategoryTerraform {
op.Variables[v.Key] = &remoteStoredVariableValue{
definition: v,
}
}
}
}
if op.Variables != nil {
variables, varDiags := backend.ParseVariableValues(op.Variables, config.Module.Variables)
diags = diags.Append(varDiags)
if diags.HasErrors() {
return nil, nil, diags
}
ret.PlanOpts.SetVariables = variables
}
}
tfCtx, ctxDiags := terraform.NewContext(&opts)
diags = diags.Append(ctxDiags)
ret.Core = tfCtx
log.Printf("[TRACE] cloud: finished building terraform.Context")
return ret, stateMgr, diags
}
func (b *Cloud) getRemoteWorkspaceName(localWorkspaceName string) string {
switch {
case localWorkspaceName == backend.DefaultStateName:
// The default workspace name is a special case
return b.WorkspaceMapping.Name
default:
return localWorkspaceName
}
}
func (b *Cloud) getRemoteWorkspace(ctx context.Context, localWorkspaceName string) (*tfe.Workspace, error) {
remoteWorkspaceName := b.getRemoteWorkspaceName(localWorkspaceName)
log.Printf("[TRACE] cloud: looking up workspace for %s/%s", b.organization, remoteWorkspaceName)
remoteWorkspace, err := b.client.Workspaces.Read(ctx, b.organization, remoteWorkspaceName)
if err != nil {
return nil, err
}
return remoteWorkspace, nil
}
func (b *Cloud) getRemoteWorkspaceID(ctx context.Context, localWorkspaceName string) (string, error) {
remoteWorkspace, err := b.getRemoteWorkspace(ctx, localWorkspaceName)
if err != nil {
return "", err
}
return remoteWorkspace.ID, nil
}
func stubAllVariables(vv map[string]backend.UnparsedVariableValue, decls map[string]*configs.Variable) terraform.InputValues {
ret := make(terraform.InputValues, len(decls))
for name, cfg := range decls {
raw, exists := vv[name]
if !exists {
ret[name] = &terraform.InputValue{
Value: cty.UnknownVal(cfg.Type),
SourceType: terraform.ValueFromConfig,
}
continue
}
val, diags := raw.ParseVariableValue(cfg.ParsingMode)
if diags.HasErrors() {
ret[name] = &terraform.InputValue{
Value: cty.UnknownVal(cfg.Type),
SourceType: terraform.ValueFromConfig,
}
continue
}
ret[name] = val
}
return ret
}
// remoteStoredVariableValue is a backend.UnparsedVariableValue implementation
// that translates from the go-tfe representation of stored variables into
// the Terraform Core backend representation of variables.
type remoteStoredVariableValue struct {
definition *tfe.Variable
}
var _ backend.UnparsedVariableValue = (*remoteStoredVariableValue)(nil)
func (v *remoteStoredVariableValue) ParseVariableValue(mode configs.VariableParsingMode) (*terraform.InputValue, tfdiags.Diagnostics) {
var diags tfdiags.Diagnostics
var val cty.Value
switch {
case v.definition.Sensitive:
// If it's marked as sensitive then it's not available for use in
// local operations. We'll use an unknown value as a placeholder for
// it so that operations that don't need it might still work, but
// we'll also produce a warning about it to add context for any
// errors that might result here.
val = cty.DynamicVal
if !v.definition.HCL {
// If it's not marked as HCL then we at least know that the
// value must be a string, so we'll set that in case it allows
// us to do some more precise type checking.
val = cty.UnknownVal(cty.String)
}
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Warning,
fmt.Sprintf("Value for var.%s unavailable", v.definition.Key),
fmt.Sprintf("The value of variable %q is marked as sensitive in the remote workspace. This operation always runs locally, so the value for that variable is not available.", v.definition.Key),
))
case v.definition.HCL:
// If the variable value is marked as being in HCL syntax, we need to
// parse it the same way as it would be interpreted in a .tfvars
// file because that is how it would get passed to Terraform CLI for
// a remote operation and we want to mimic that result as closely as
// possible.
var exprDiags hcl.Diagnostics
expr, exprDiags := hclsyntax.ParseExpression([]byte(v.definition.Value), "<remote workspace>", hcl.Pos{Line: 1, Column: 1})
if expr != nil {
var moreDiags hcl.Diagnostics
val, moreDiags = expr.Value(nil)
exprDiags = append(exprDiags, moreDiags...)
} else {
// We'll have already put some errors in exprDiags above, so we'll
// just stub out the value here.
val = cty.DynamicVal
}
// We don't have sufficient context to return decent error messages
// for syntax errors in the remote values, so we'll just return a
// generic message instead for now.
// (More complete error messages will still result from true remote
// operations, because they'll run on the remote system where we've
// materialized the values into a tfvars file we can report from.)
if exprDiags.HasErrors() {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
fmt.Sprintf("Invalid expression for var.%s", v.definition.Key),
fmt.Sprintf("The value of variable %q is marked in the remote workspace as being specified in HCL syntax, but the given value is not valid HCL. Stored variable values must be valid literal expressions and may not contain references to other variables or calls to functions.", v.definition.Key),
))
}
default:
// A variable value _not_ marked as HCL is always be a string, given
// literally.
val = cty.StringVal(v.definition.Value)
}
return &terraform.InputValue{
Value: val,
// We mark these as "from input" with the rationale that entering
// variable values into the Terraform Cloud or Enterprise UI is,
// roughly speaking, a similar idea to entering variable values at
// the interactive CLI prompts. It's not a perfect correspondance,
// but it's closer than the other options.
SourceType: terraform.ValueFromInput,
}, diags
}

235
internal/cloud/backend_context_test.go Normal file
View File

@ -0,0 +1,235 @@
package cloud
import (
"context"
"testing"
tfe "github.com/hashicorp/go-tfe"
"github.com/hashicorp/terraform/internal/backend"
"github.com/hashicorp/terraform/internal/command/arguments"
"github.com/hashicorp/terraform/internal/command/clistate"
"github.com/hashicorp/terraform/internal/command/views"
"github.com/hashicorp/terraform/internal/configs"
"github.com/hashicorp/terraform/internal/initwd"
"github.com/hashicorp/terraform/internal/states/statemgr"
"github.com/hashicorp/terraform/internal/terminal"
"github.com/zclconf/go-cty/cty"
)
func TestRemoteStoredVariableValue(t *testing.T) {
tests := map[string]struct {
Def *tfe.Variable
Want cty.Value
WantError string
}{
"string literal": {
&tfe.Variable{
Key: "test",
Value: "foo",
HCL: false,
Sensitive: false,
},
cty.StringVal("foo"),
``,
},
"string HCL": {
&tfe.Variable{
Key: "test",
Value: `"foo"`,
HCL: true,
Sensitive: false,
},
cty.StringVal("foo"),
``,
},
"list HCL": {
&tfe.Variable{
Key: "test",
Value: `[]`,
HCL: true,
Sensitive: false,
},
cty.EmptyTupleVal,
``,
},
"null HCL": {
&tfe.Variable{
Key: "test",
Value: `null`,
HCL: true,
Sensitive: false,
},
cty.NullVal(cty.DynamicPseudoType),
``,
},
"literal sensitive": {
&tfe.Variable{
Key: "test",
HCL: false,
Sensitive: true,
},
cty.UnknownVal(cty.String),
``,
},
"HCL sensitive": {
&tfe.Variable{
Key: "test",
HCL: true,
Sensitive: true,
},
cty.DynamicVal,
``,
},
"HCL computation": {
// This (stored expressions containing computation) is not a case
// we intentionally supported, but it became possible for remote
// operations in Terraform 0.12 (due to Terraform Cloud/Enterprise
// just writing the HCL verbatim into generated `.tfvars` files).
// We support it here for consistency, and we continue to support
// it in both places for backward-compatibility. In practice,
// there's little reason to do computation in a stored variable
// value because references are not supported.
&tfe.Variable{
Key: "test",
Value: `[for v in ["a"] : v]`,
HCL: true,
Sensitive: false,
},
cty.TupleVal([]cty.Value{cty.StringVal("a")}),
``,
},
"HCL syntax error": {
&tfe.Variable{
Key: "test",
Value: `[`,
HCL: true,
Sensitive: false,
},
cty.DynamicVal,
`Invalid expression for var.test: The value of variable "test" is marked in the remote workspace as being specified in HCL syntax, but the given value is not valid HCL. Stored variable values must be valid literal expressions and may not contain references to other variables or calls to functions.`,
},
"HCL with references": {
&tfe.Variable{
Key: "test",
Value: `foo.bar`,
HCL: true,
Sensitive: false,
},
cty.DynamicVal,
`Invalid expression for var.test: The value of variable "test" is marked in the remote workspace as being specified in HCL syntax, but the given value is not valid HCL. Stored variable values must be valid literal expressions and may not contain references to other variables or calls to functions.`,
},
}
for name, test := range tests {
t.Run(name, func(t *testing.T) {
v := &remoteStoredVariableValue{
definition: test.Def,
}
// This ParseVariableValue implementation ignores the parsing mode,
// so we'll just always parse literal here. (The parsing mode is
// selected by the remote server, not by our local configuration.)
gotIV, diags := v.ParseVariableValue(configs.VariableParseLiteral)
if test.WantError != "" {
if !diags.HasErrors() {
t.Fatalf("missing expected error\ngot: <no error>\nwant: %s", test.WantError)
}
errStr := diags.Err().Error()
if errStr != test.WantError {
t.Fatalf("wrong error\ngot: %s\nwant: %s", errStr, test.WantError)
}
} else {
if diags.HasErrors() {
t.Fatalf("unexpected error\ngot: %s\nwant: <no error>", diags.Err().Error())
}
got := gotIV.Value
if !test.Want.RawEquals(got) {
t.Errorf("wrong result\ngot: %#v\nwant: %#v", got, test.Want)
}
}
})
}
}
func TestRemoteContextWithVars(t *testing.T) {
catTerraform := tfe.CategoryTerraform
catEnv := tfe.CategoryEnv
tests := map[string]struct {
Opts *tfe.VariableCreateOptions
WantError string
}{
"Terraform variable": {
&tfe.VariableCreateOptions{
Category: &catTerraform,
},
`Value for undeclared variable: A variable named "key" was assigned a value, but the root module does not declare a variable of that name. To use this value, add a "variable" block to the configuration.`,
},
"environment variable": {
&tfe.VariableCreateOptions{
Category: &catEnv,
},
``,
},
}
for name, test := range tests {
t.Run(name, func(t *testing.T) {
configDir := "./testdata/empty"
b, bCleanup := testBackendWithName(t)
defer bCleanup()
_, configLoader, configCleanup := initwd.MustLoadConfigForTests(t, configDir)
defer configCleanup()
workspaceID, err := b.getRemoteWorkspaceID(context.Background(), testBackendSingleWorkspaceName)
if err != nil {
t.Fatal(err)
}
streams, _ := terminal.StreamsForTesting(t)
view := views.NewStateLocker(arguments.ViewHuman, views.NewView(streams))
op := &backend.Operation{
ConfigDir: configDir,
ConfigLoader: configLoader,
StateLocker: clistate.NewLocker(0, view),
Workspace: testBackendSingleWorkspaceName,
}
v := test.Opts
if v.Key == nil {
key := "key"
v.Key = &key
}
b.client.Variables.Create(context.TODO(), workspaceID, *v)
_, _, diags := b.LocalRun(op)
if test.WantError != "" {
if !diags.HasErrors() {
t.Fatalf("missing expected error\ngot: <no error>\nwant: %s", test.WantError)
}
errStr := diags.Err().Error()
if errStr != test.WantError {
t.Fatalf("wrong error\ngot: %s\nwant: %s", errStr, test.WantError)
}
// When Context() returns an error, it should unlock the state,
// so re-locking it is expected to succeed.
stateMgr, _ := b.StateMgr(testBackendSingleWorkspaceName)
if _, err := stateMgr.Lock(statemgr.NewLockInfo()); err != nil {
t.Fatalf("unexpected error locking state: %s", err.Error())
}
} else {
if diags.HasErrors() {
t.Fatalf("unexpected error\ngot: %s\nwant: <no error>", diags.Err().Error())
}
// When Context() succeeds, this should fail w/ "workspace already locked"
stateMgr, _ := b.StateMgr(testBackendSingleWorkspaceName)
if _, err := stateMgr.Lock(statemgr.NewLockInfo()); err == nil {
t.Fatal("unexpected success locking state after Context")
}
}
})
}
}

373
internal/cloud/backend_plan.go Normal file
View File

@ -0,0 +1,373 @@
package cloud
import (
"bufio"
"context"
"errors"
"fmt"
"io"
"io/ioutil"
"log"
"os"
"path/filepath"
"strings"
"syscall"
"time"
tfe "github.com/hashicorp/go-tfe"
"github.com/hashicorp/terraform/internal/backend"
"github.com/hashicorp/terraform/internal/plans"
"github.com/hashicorp/terraform/internal/tfdiags"
)
var planConfigurationVersionsPollInterval = 500 * time.Millisecond
func (b *Cloud) opPlan(stopCtx, cancelCtx context.Context, op *backend.Operation, w *tfe.Workspace) (*tfe.Run, error) {
log.Printf("[INFO] cloud: starting Plan operation")
var diags tfdiags.Diagnostics
if !w.Permissions.CanQueueRun {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Insufficient rights to generate a plan",
"The provided credentials have insufficient rights to generate a plan. In order "+
"to generate plans, at least plan permissions on the workspace are required.",
))
return nil, diags.Err()
}
if b.ContextOpts != nil && b.ContextOpts.Parallelism != defaultParallelism {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Custom parallelism values are currently not supported",
`Terraform Cloud does not support setting a custom parallelism `+
`value at this time.`,
))
}
if op.PlanFile != nil {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Displaying a saved plan is currently not supported",
`Terraform Cloud currently requires configuration to be present and `+
`does not accept an existing saved plan as an argument at this time.`,
))
}
if op.PlanOutPath != "" {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Saving a generated plan is currently not supported",
`Terraform Cloud does not support saving the generated execution `+
`plan locally at this time.`,
))
}
if !op.HasConfig() && op.PlanMode != plans.DestroyMode {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"No configuration files found",
`Plan requires configuration to be present. Planning without a configuration `+
`would mark everything for destruction, which is normally not what is desired. `+
`If you would like to destroy everything, please run plan with the "-destroy" `+
`flag or create a single empty configuration file. Otherwise, please create `+
`a Terraform configuration file in the path being executed and try again.`,
))
}
// Return if there are any errors.
if diags.HasErrors() {
return nil, diags.Err()
}
return b.plan(stopCtx, cancelCtx, op, w)
}
func (b *Cloud) plan(stopCtx, cancelCtx context.Context, op *backend.Operation, w *tfe.Workspace) (*tfe.Run, error) {
if b.CLI != nil {
header := planDefaultHeader
if op.Type == backend.OperationTypeApply || op.Type == backend.OperationTypeRefresh {
header = applyDefaultHeader
}
b.CLI.Output(b.Colorize().Color(strings.TrimSpace(header) + "\n"))
}
configOptions := tfe.ConfigurationVersionCreateOptions{
AutoQueueRuns: tfe.Bool(false),
Speculative: tfe.Bool(op.Type == backend.OperationTypePlan),
}
cv, err := b.client.ConfigurationVersions.Create(stopCtx, w.ID, configOptions)
if err != nil {
return nil, generalError("Failed to create configuration version", err)
}
var configDir string
if op.ConfigDir != "" {
// De-normalize the configuration directory path.
configDir, err = filepath.Abs(op.ConfigDir)
if err != nil {
return nil, generalError(
"Failed to get absolute path of the configuration directory: %v", err)
}
// Make sure to take the working directory into account by removing
// the working directory from the current path. This will result in
// a path that points to the expected root of the workspace.
configDir = filepath.Clean(strings.TrimSuffix(
filepath.Clean(configDir),
filepath.Clean(w.WorkingDirectory),
))
// If the workspace has a subdirectory as its working directory then
// our configDir will be some parent directory of the current working
// directory. Users are likely to find that surprising, so we'll
// produce an explicit message about it to be transparent about what
// we are doing and why.
if w.WorkingDirectory != "" && filepath.Base(configDir) != w.WorkingDirectory {
if b.CLI != nil {
b.CLI.Output(fmt.Sprintf(strings.TrimSpace(`
The remote workspace is configured to work with configuration at
%s relative to the target repository.
Terraform will upload the contents of the following directory,
excluding files or directories as defined by a .terraformignore file
at %s/.terraformignore (if it is present),
in order to capture the filesystem context the remote workspace expects:
%s
`), w.WorkingDirectory, configDir, configDir) + "\n")
}
}
} else {
// We did a check earlier to make sure we either have a config dir,
// or the plan is run with -destroy. So this else clause will only
// be executed when we are destroying and doesn't need the config.
configDir, err = ioutil.TempDir("", "tf")
if err != nil {
return nil, generalError("Failed to create temporary directory", err)
}
defer os.RemoveAll(configDir)
// Make sure the configured working directory exists.
err = os.MkdirAll(filepath.Join(configDir, w.WorkingDirectory), 0700)
if err != nil {
return nil, generalError(
"Failed to create temporary working directory", err)
}
}
err = b.client.ConfigurationVersions.Upload(stopCtx, cv.UploadURL, configDir)
if err != nil {
return nil, generalError("Failed to upload configuration files", err)
}
uploaded := false
for i := 0; i < 60 && !uploaded; i++ {
select {
case <-stopCtx.Done():
return nil, context.Canceled
case <-cancelCtx.Done():
return nil, context.Canceled
case <-time.After(planConfigurationVersionsPollInterval):
cv, err = b.client.ConfigurationVersions.Read(stopCtx, cv.ID)
if err != nil {
return nil, generalError("Failed to retrieve configuration version", err)
}
if cv.Status == tfe.ConfigurationUploaded {
uploaded = true
}
}
}
if !uploaded {
return nil, generalError(
"Failed to upload configuration files", errors.New("operation timed out"))
}
runOptions := tfe.RunCreateOptions{
ConfigurationVersion: cv,
Refresh: tfe.Bool(op.PlanRefresh),
Workspace: w,
AutoApply: tfe.Bool(op.AutoApprove),
}
switch op.PlanMode {
case plans.NormalMode:
// okay, but we don't need to do anything special for this
case plans.RefreshOnlyMode:
runOptions.RefreshOnly = tfe.Bool(true)
case plans.DestroyMode:
runOptions.IsDestroy = tfe.Bool(true)
default:
// Shouldn't get here because we should update this for each new
// plan mode we add, mapping it to the corresponding RunCreateOptions
// field.
return nil, generalError(
"Invalid plan mode",
fmt.Errorf("Terraform Cloud doesn't support %s", op.PlanMode),
)
}
if len(op.Targets) != 0 {
runOptions.TargetAddrs = make([]string, 0, len(op.Targets))
for _, addr := range op.Targets {
runOptions.TargetAddrs = append(runOptions.TargetAddrs, addr.String())
}
}
if len(op.ForceReplace) != 0 {
runOptions.ReplaceAddrs = make([]string, 0, len(op.ForceReplace))
for _, addr := range op.ForceReplace {
runOptions.ReplaceAddrs = append(runOptions.ReplaceAddrs, addr.String())
}
}
config, _, configDiags := op.ConfigLoader.LoadConfigWithSnapshot(op.ConfigDir)
if configDiags.HasErrors() {
return nil, fmt.Errorf("error loading config with snapshot: %w", configDiags.Errs()[0])
}
variables, varDiags := ParseCloudRunVariables(op.Variables, config.Module.Variables)
if varDiags.HasErrors() {
return nil, varDiags.Err()
}
runVariables := make([]*tfe.RunVariable, len(variables))
for name, value := range variables {
runVariables = append(runVariables, &tfe.RunVariable{
Key: name,
Value: value,
})
}
runOptions.Variables = runVariables
r, err := b.client.Runs.Create(stopCtx, runOptions)
if err != nil {
return r, generalError("Failed to create run", err)
}
// When the lock timeout is set, if the run is still pending and
// cancellable after that period, we attempt to cancel it.
if lockTimeout := op.StateLocker.Timeout(); lockTimeout > 0 {
go func() {
select {
case <-stopCtx.Done():
return
case <-cancelCtx.Done():
return
case <-time.After(lockTimeout):
// Retrieve the run to get its current status.
r, err := b.client.Runs.Read(cancelCtx, r.ID)
if err != nil {
log.Printf("[ERROR] error reading run: %v", err)
return
}
if r.Status == tfe.RunPending && r.Actions.IsCancelable {
if b.CLI != nil {
b.CLI.Output(b.Colorize().Color(strings.TrimSpace(lockTimeoutErr)))
}
// We abuse the auto aprove flag to indicate that we do not
// want to ask if the remote operation should be canceled.
op.AutoApprove = true
p, err := os.FindProcess(os.Getpid())
if err != nil {
log.Printf("[ERROR] error searching process ID: %v", err)
return
}
p.Signal(syscall.SIGINT)
}
}
}()
}
if b.CLI != nil {
b.CLI.Output(b.Colorize().Color(strings.TrimSpace(fmt.Sprintf(
runHeader, b.hostname, b.organization, op.Workspace, r.ID)) + "\n"))
}
r, err = b.waitForRun(stopCtx, cancelCtx, op, "plan", r, w)
if err != nil {
return r, err
}
logs, err := b.client.Plans.Logs(stopCtx, r.Plan.ID)
if err != nil {
return r, generalError("Failed to retrieve logs", err)
}
reader := bufio.NewReaderSize(logs, 64*1024)
if b.CLI != nil {
for next := true; next; {
var l, line []byte
for isPrefix := true; isPrefix; {
l, isPrefix, err = reader.ReadLine()
if err != nil {
if err != io.EOF {
return r, generalError("Failed to read logs", err)
}
next = false
}
line = append(line, l...)
}
if next || len(line) > 0 {
b.CLI.Output(b.Colorize().Color(string(line)))
}
}
}
// Retrieve the run to get its current status.
r, err = b.client.Runs.Read(stopCtx, r.ID)
if err != nil {
return r, generalError("Failed to retrieve run", err)
}
// If the run is canceled or errored, we still continue to the
// cost-estimation and policy check phases to ensure we render any
// results available. In the case of a hard-failed policy check, the
// status of the run will be "errored", but there is still policy
// information which should be shown.
// Show any cost estimation output.
if r.CostEstimate != nil {
err = b.costEstimate(stopCtx, cancelCtx, op, r)
if err != nil {
return r, err
}
}
// Check any configured sentinel policies.
if len(r.PolicyChecks) > 0 {
err = b.checkPolicy(stopCtx, cancelCtx, op, r)
if err != nil {
return r, err
}
}
return r, nil
}
const planDefaultHeader = `
[reset][yellow]Running plan in Terraform Cloud. Output will stream here. Pressing Ctrl-C
will stop streaming the logs, but will not stop the plan running remotely.[reset]
Preparing the remote plan...
`
const runHeader = `
[reset][yellow]To view this run in a browser, visit:
https://%s/app/%s/%s/runs/%s[reset]
`
// The newline in this error is to make it look good in the CLI!
const lockTimeoutErr = `
[reset][red]Lock timeout exceeded, sending interrupt to cancel the remote operation.
[reset]
`

1114
internal/cloud/backend_plan_test.go Normal file
View File

File diff suppressed because it is too large Load Diff

79
internal/cloud/backend_refresh_test.go Normal file
View File

@ -0,0 +1,79 @@
package cloud
import (
"context"
"strings"
"testing"
"time"
"github.com/hashicorp/terraform/internal/backend"
"github.com/hashicorp/terraform/internal/command/arguments"
"github.com/hashicorp/terraform/internal/command/clistate"
"github.com/hashicorp/terraform/internal/command/views"
"github.com/hashicorp/terraform/internal/initwd"
"github.com/hashicorp/terraform/internal/plans"
"github.com/hashicorp/terraform/internal/states/statemgr"
"github.com/hashicorp/terraform/internal/terminal"
"github.com/mitchellh/cli"
)
func testOperationRefresh(t *testing.T, configDir string) (*backend.Operation, func(), func(*testing.T) *terminal.TestOutput) {
t.Helper()
return testOperationRefreshWithTimeout(t, configDir, 0)
}
func testOperationRefreshWithTimeout(t *testing.T, configDir string, timeout time.Duration) (*backend.Operation, func(), func(*testing.T) *terminal.TestOutput) {
t.Helper()
_, configLoader, configCleanup := initwd.MustLoadConfigForTests(t, configDir)
streams, done := terminal.StreamsForTesting(t)
view := views.NewView(streams)
stateLockerView := views.NewStateLocker(arguments.ViewHuman, view)
operationView := views.NewOperation(arguments.ViewHuman, false, view)
return &backend.Operation{
ConfigDir: configDir,
ConfigLoader: configLoader,
PlanRefresh: true,
StateLocker: clistate.NewLocker(timeout, stateLockerView),
Type: backend.OperationTypeRefresh,
View: operationView,
}, configCleanup, done
}
func TestCloud_refreshBasicActuallyRunsApplyRefresh(t *testing.T) {
b, bCleanup := testBackendWithName(t)
defer bCleanup()
op, configCleanup, done := testOperationRefresh(t, "./testdata/refresh")
defer configCleanup()
defer done(t)
op.UIOut = b.CLI
b.CLIColor = b.cliColorize()
op.PlanMode = plans.RefreshOnlyMode
op.Workspace = testBackendSingleWorkspaceName
run, err := b.Operation(context.Background(), op)
if err != nil {
t.Fatalf("error starting operation: %v", err)
}
<-run.Done()
if run.Result != backend.OperationSuccess {
t.Fatalf("operation failed: %s", b.CLI.(*cli.MockUi).ErrorWriter.String())
}
output := b.CLI.(*cli.MockUi).OutputWriter.String()
if !strings.Contains(output, "Proceeding with 'terraform apply -refresh-only -auto-approve'") {
t.Fatalf("expected TFC header in output: %s", output)
}
stateMgr, _ := b.StateMgr(testBackendSingleWorkspaceName)
// An error suggests that the state was not unlocked after apply
if _, err := stateMgr.Lock(statemgr.NewLockInfo()); err != nil {
t.Fatalf("unexpected error locking state after apply: %s", err.Error())
}
}

182
internal/cloud/backend_state.go Normal file
View File

@ -0,0 +1,182 @@
package cloud
import (
"bytes"
"context"
"crypto/md5"
"encoding/base64"
"fmt"
tfe "github.com/hashicorp/go-tfe"
"github.com/hashicorp/terraform/internal/states/remote"
"github.com/hashicorp/terraform/internal/states/statefile"
"github.com/hashicorp/terraform/internal/states/statemgr"
)
type remoteClient struct {
client *tfe.Client
lockInfo *statemgr.LockInfo
organization string
runID string
stateUploadErr bool
workspace *tfe.Workspace
forcePush bool
}
// Get the remote state.
func (r *remoteClient) Get() (*remote.Payload, error) {
ctx := context.Background()
sv, err := r.client.StateVersions.Current(ctx, r.workspace.ID)
if err != nil {
if err == tfe.ErrResourceNotFound {
// If no state exists, then return nil.
return nil, nil
}
return nil, fmt.Errorf("Error retrieving state: %v", err)
}
state, err := r.client.StateVersions.Download(ctx, sv.DownloadURL)
if err != nil {
return nil, fmt.Errorf("Error downloading state: %v", err)
}
// If the state is empty, then return nil.
if len(state) == 0 {
return nil, nil
}
// Get the MD5 checksum of the state.
sum := md5.Sum(state)
return &remote.Payload{
Data: state,
MD5: sum[:],
}, nil
}
// Put the remote state.
func (r *remoteClient) Put(state []byte) error {
ctx := context.Background()
// Read the raw state into a Terraform state.
stateFile, err := statefile.Read(bytes.NewReader(state))
if err != nil {
return fmt.Errorf("Error reading state: %s", err)
}
options := tfe.StateVersionCreateOptions{
Lineage: tfe.String(stateFile.Lineage),
Serial: tfe.Int64(int64(stateFile.Serial)),
MD5: tfe.String(fmt.Sprintf("%x", md5.Sum(state))),
State: tfe.String(base64.StdEncoding.EncodeToString(state)),
Force: tfe.Bool(r.forcePush),
}
// If we have a run ID, make sure to add it to the options
// so the state will be properly associated with the run.
if r.runID != "" {
options.Run = &tfe.Run{ID: r.runID}
}
// Create the new state.
_, err = r.client.StateVersions.Create(ctx, r.workspace.ID, options)
if err != nil {
r.stateUploadErr = true
return fmt.Errorf("Error uploading state: %v", err)
}
return nil
}
// Delete the remote state.
func (r *remoteClient) Delete() error {
err := r.client.Workspaces.Delete(context.Background(), r.organization, r.workspace.Name)
if err != nil && err != tfe.ErrResourceNotFound {
return fmt.Errorf("Error deleting workspace %s: %v", r.workspace.Name, err)
}
return nil
}
// EnableForcePush to allow the remote client to overwrite state
// by implementing remote.ClientForcePusher
func (r *remoteClient) EnableForcePush() {
r.forcePush = true
}
// Lock the remote state.
func (r *remoteClient) Lock(info *statemgr.LockInfo) (string, error) {
ctx := context.Background()
lockErr := &statemgr.LockError{Info: r.lockInfo}
// Lock the workspace.
_, err := r.client.Workspaces.Lock(ctx, r.workspace.ID, tfe.WorkspaceLockOptions{
Reason: tfe.String("Locked by Terraform"),
})
if err != nil {
if err == tfe.ErrWorkspaceLocked {
lockErr.Info = info
err = fmt.Errorf("%s (lock ID: \"%s/%s\")", err, r.organization, r.workspace.Name)
}
lockErr.Err = err
return "", lockErr
}
r.lockInfo = info
return r.lockInfo.ID, nil
}
// Unlock the remote state.
func (r *remoteClient) Unlock(id string) error {
ctx := context.Background()
// We first check if there was an error while uploading the latest
// state. If so, we will not unlock the workspace to prevent any
// changes from being applied until the correct state is uploaded.
if r.stateUploadErr {
return nil
}
lockErr := &statemgr.LockError{Info: r.lockInfo}
// With lock info this should be treated as a normal unlock.
if r.lockInfo != nil {
// Verify the expected lock ID.
if r.lockInfo.ID != id {
lockErr.Err = fmt.Errorf("lock ID does not match existing lock")
return lockErr
}
// Unlock the workspace.
_, err := r.client.Workspaces.Unlock(ctx, r.workspace.ID)
if err != nil {
lockErr.Err = err
return lockErr
}
return nil
}
// Verify the optional force-unlock lock ID.
if r.organization+"/"+r.workspace.Name != id {
lockErr.Err = fmt.Errorf(
"lock ID %q does not match existing lock ID \"%s/%s\"",
id,
r.organization,
r.workspace.Name,
)
return lockErr
}
// Force unlock the workspace.
_, err := r.client.Workspaces.ForceUnlock(ctx, r.workspace.ID)
if err != nil {
lockErr.Err = err
return lockErr
}
return nil
}

58
internal/cloud/backend_state_test.go Normal file
View File

@ -0,0 +1,58 @@
package cloud
import (
"bytes"
"os"
"testing"
"github.com/hashicorp/terraform/internal/states"
"github.com/hashicorp/terraform/internal/states/remote"
"github.com/hashicorp/terraform/internal/states/statefile"
)
func TestRemoteClient_impl(t *testing.T) {
var _ remote.Client = new(remoteClient)
}
func TestRemoteClient(t *testing.T) {
client := testRemoteClient(t)
remote.TestClient(t, client)
}
func TestRemoteClient_stateLock(t *testing.T) {
b, bCleanup := testBackendWithName(t)
defer bCleanup()
s1, err := b.StateMgr(testBackendSingleWorkspaceName)
if err != nil {
t.Fatalf("expected no error, got %v", err)
}
s2, err := b.StateMgr(testBackendSingleWorkspaceName)
if err != nil {
t.Fatalf("expected no error, got %v", err)
}
remote.TestRemoteLocks(t, s1.(*remote.State).Client, s2.(*remote.State).Client)
}
func TestRemoteClient_withRunID(t *testing.T) {
// Set the TFE_RUN_ID environment variable before creating the client!
if err := os.Setenv("TFE_RUN_ID", GenerateID("run-")); err != nil {
t.Fatalf("error setting env var TFE_RUN_ID: %v", err)
}
// Create a new test client.
client := testRemoteClient(t)
// Create a new empty state.
sf := statefile.New(states.NewState(), "", 0)
var buf bytes.Buffer
statefile.Write(sf, &buf)
// Store the new state to verify (this will be done
// by the mock that is used) that the run ID is set.
if err := client.Put(buf.Bytes()); err != nil {
t.Fatalf("expected no error, got %v", err)
}
}

857
internal/cloud/backend_test.go Normal file
View File

@ -0,0 +1,857 @@
package cloud
import (
"context"
"fmt"
"net/http"
"os"
"strings"
"testing"
tfe "github.com/hashicorp/go-tfe"
version "github.com/hashicorp/go-version"
"github.com/hashicorp/terraform/internal/backend"
"github.com/hashicorp/terraform/internal/tfdiags"
tfversion "github.com/hashicorp/terraform/version"
"github.com/zclconf/go-cty/cty"
backendLocal "github.com/hashicorp/terraform/internal/backend/local"
)
func TestCloud(t *testing.T) {
var _ backend.Enhanced = New(nil)
var _ backend.CLI = New(nil)
}
func TestCloud_backendWithName(t *testing.T) {
b, bCleanup := testBackendWithName(t)
defer bCleanup()
workspaces, err := b.Workspaces()
if err != nil {
t.Fatalf("error: %v", err)
}
if len(workspaces) != 1 || workspaces[0] != testBackendSingleWorkspaceName {
t.Fatalf("should only have a single configured workspace matching the configured 'name' strategy, but got: %#v", workspaces)
}
if _, err := b.StateMgr("foo"); err != backend.ErrWorkspacesNotSupported {
t.Fatalf("expected fetching a state which is NOT the single configured workspace to have an ErrWorkspacesNotSupported error, but got: %v", err)
}
if err := b.DeleteWorkspace(testBackendSingleWorkspaceName); err != backend.ErrWorkspacesNotSupported {
t.Fatalf("expected deleting the single configured workspace name to result in an error, but got: %v", err)
}
if err := b.DeleteWorkspace("foo"); err != backend.ErrWorkspacesNotSupported {
t.Fatalf("expected deleting a workspace which is NOT the configured workspace name to result in an error, but got: %v", err)
}
}
func TestCloud_backendWithTags(t *testing.T) {
b, bCleanup := testBackendWithTags(t)
defer bCleanup()
backend.TestBackendStates(t, b)
// Test pagination works
for i := 0; i < 25; i++ {
_, err := b.StateMgr(fmt.Sprintf("foo-%d", i+1))
if err != nil {
t.Fatalf("error: %s", err)
}
}
workspaces, err := b.Workspaces()
if err != nil {
t.Fatalf("error: %s", err)
}
actual := len(workspaces)
if actual != 26 {
t.Errorf("expected 26 workspaces (over one standard paginated response), got %d", actual)
}
}
func TestCloud_PrepareConfig(t *testing.T) {
cases := map[string]struct {
config cty.Value
expectedErr string
}{
"null organization": {
config: cty.ObjectVal(map[string]cty.Value{
"organization": cty.NullVal(cty.String),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.StringVal("prod"),
"tags": cty.NullVal(cty.Set(cty.String)),
}),
}),
expectedErr: `Invalid organization value: The "organization" attribute value must not be empty.`,
},
"null workspace": {
config: cty.ObjectVal(map[string]cty.Value{
"organization": cty.StringVal("org"),
"workspaces": cty.NullVal(cty.String),
}),
expectedErr: `Invalid workspaces configuration: Missing workspace mapping strategy. Either workspace "tags" or "name" is required.`,
},
"workspace: empty tags, name": {
config: cty.ObjectVal(map[string]cty.Value{
"organization": cty.StringVal("org"),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.NullVal(cty.String),
"tags": cty.NullVal(cty.Set(cty.String)),
}),
}),
expectedErr: `Invalid workspaces configuration: Missing workspace mapping strategy. Either workspace "tags" or "name" is required.`,
},
"workspace: name present": {
config: cty.ObjectVal(map[string]cty.Value{
"organization": cty.StringVal("org"),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.StringVal("prod"),
"tags": cty.NullVal(cty.Set(cty.String)),
}),
}),
expectedErr: `Invalid workspaces configuration: Only one of workspace "tags" or "name" is allowed.`,
},
"workspace: name and tags present": {
config: cty.ObjectVal(map[string]cty.Value{
"organization": cty.StringVal("org"),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.StringVal("prod"),
"tags": cty.SetVal(
[]cty.Value{
cty.StringVal("billing"),
},
),
}),
}),
expectedErr: `Invalid workspaces configuration: Only one of workspace "tags" or "name" is allowed.`,
},
}
for name, tc := range cases {
s := testServer(t)
b := New(testDisco(s))
// Validate
_, valDiags := b.PrepareConfig(tc.config)
if valDiags.Err() != nil && tc.expectedErr != "" {
actualErr := valDiags.Err().Error()
if !strings.Contains(actualErr, tc.expectedErr) {
t.Fatalf("%s: unexpected validation result: %v", name, valDiags.Err())
}
}
}
}
func TestCloud_config(t *testing.T) {
cases := map[string]struct {
config cty.Value
confErr string
valErr string
}{
"with_a_nonexisting_organization": {
config: cty.ObjectVal(map[string]cty.Value{
"hostname": cty.NullVal(cty.String),
"organization": cty.StringVal("nonexisting"),
"token": cty.NullVal(cty.String),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.StringVal("prod"),
"tags": cty.NullVal(cty.Set(cty.String)),
}),
}),
confErr: "organization \"nonexisting\" at host app.terraform.io not found",
},
"with_an_unknown_host": {
config: cty.ObjectVal(map[string]cty.Value{
"hostname": cty.StringVal("nonexisting.local"),
"organization": cty.StringVal("hashicorp"),
"token": cty.NullVal(cty.String),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.StringVal("prod"),
"tags": cty.NullVal(cty.Set(cty.String)),
}),
}),
confErr: "Failed to request discovery document",
},
// localhost advertises TFE services, but has no token in the credentials
"without_a_token": {
config: cty.ObjectVal(map[string]cty.Value{
"hostname": cty.StringVal("localhost"),
"organization": cty.StringVal("hashicorp"),
"token": cty.NullVal(cty.String),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.StringVal("prod"),
"tags": cty.NullVal(cty.Set(cty.String)),
}),
}),
confErr: "terraform login localhost",
},
"with_tags": {
config: cty.ObjectVal(map[string]cty.Value{
"hostname": cty.NullVal(cty.String),
"organization": cty.StringVal("hashicorp"),
"token": cty.NullVal(cty.String),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.NullVal(cty.String),
"tags": cty.SetVal(
[]cty.Value{
cty.StringVal("billing"),
},
),
}),
}),
},
"with_a_name": {
config: cty.ObjectVal(map[string]cty.Value{
"hostname": cty.NullVal(cty.String),
"organization": cty.StringVal("hashicorp"),
"token": cty.NullVal(cty.String),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.StringVal("prod"),
"tags": cty.NullVal(cty.Set(cty.String)),
}),
}),
},
"without_a_name_tags": {
config: cty.ObjectVal(map[string]cty.Value{
"hostname": cty.NullVal(cty.String),
"organization": cty.StringVal("hashicorp"),
"token": cty.NullVal(cty.String),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.NullVal(cty.String),
"tags": cty.NullVal(cty.Set(cty.String)),
}),
}),
valErr: `Missing workspace mapping strategy.`,
},
"with_both_a_name_and_tags": {
config: cty.ObjectVal(map[string]cty.Value{
"hostname": cty.NullVal(cty.String),
"organization": cty.StringVal("hashicorp"),
"token": cty.NullVal(cty.String),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.StringVal("prod"),
"tags": cty.SetVal(
[]cty.Value{
cty.StringVal("billing"),
},
),
}),
}),
valErr: `Only one of workspace "tags" or "name" is allowed.`,
},
"null config": {
config: cty.NullVal(cty.EmptyObject),
},
}
for name, tc := range cases {
s := testServer(t)
b := New(testDisco(s))
// Validate
_, valDiags := b.PrepareConfig(tc.config)
if (valDiags.Err() != nil || tc.valErr != "") &&
(valDiags.Err() == nil || !strings.Contains(valDiags.Err().Error(), tc.valErr)) {
t.Fatalf("%s: unexpected validation result: %v", name, valDiags.Err())
}
// Configure
confDiags := b.Configure(tc.config)
if (confDiags.Err() != nil || tc.confErr != "") &&
(confDiags.Err() == nil || !strings.Contains(confDiags.Err().Error(), tc.confErr)) {
t.Fatalf("%s: unexpected configure result: %v", name, confDiags.Err())
}
}
}
func TestCloud_configVerifyMinimumTFEVersion(t *testing.T) {
config := cty.ObjectVal(map[string]cty.Value{
"hostname": cty.NullVal(cty.String),
"organization": cty.StringVal("hashicorp"),
"token": cty.NullVal(cty.String),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.NullVal(cty.String),
"tags": cty.SetVal(
[]cty.Value{
cty.StringVal("billing"),
},
),
}),
})
handlers := map[string]func(http.ResponseWriter, *http.Request){
"/api/v2/ping": func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.Header().Set("TFP-API-Version", "2.4")
},
}
s := testServerWithHandlers(handlers)
b := New(testDisco(s))
confDiags := b.Configure(config)
if confDiags.Err() == nil {
t.Fatalf("expected configure to error")
}
expected := `The 'cloud' option is not supported with this version of Terraform Enterprise.`
if !strings.Contains(confDiags.Err().Error(), expected) {
t.Fatalf("expected configure to error with %q, got %q", expected, confDiags.Err().Error())
}
}
func TestCloud_configVerifyMinimumTFEVersionInAutomation(t *testing.T) {
config := cty.ObjectVal(map[string]cty.Value{
"hostname": cty.NullVal(cty.String),
"organization": cty.StringVal("hashicorp"),
"token": cty.NullVal(cty.String),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.NullVal(cty.String),
"tags": cty.SetVal(
[]cty.Value{
cty.StringVal("billing"),
},
),
}),
})
handlers := map[string]func(http.ResponseWriter, *http.Request){
"/api/v2/ping": func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.Header().Set("TFP-API-Version", "2.4")
},
}
s := testServerWithHandlers(handlers)
b := New(testDisco(s))
b.runningInAutomation = true
confDiags := b.Configure(config)
if confDiags.Err() == nil {
t.Fatalf("expected configure to error")
}
expected := `This version of Terraform Cloud/Enterprise does not support the state mechanism
attempting to be used by the platform. This should never happen.`
if !strings.Contains(confDiags.Err().Error(), expected) {
t.Fatalf("expected configure to error with %q, got %q", expected, confDiags.Err().Error())
}
}
func TestCloud_setUnavailableTerraformVersion(t *testing.T) {
// go-tfe returns an error IRL if you try to set a Terraform version that's
// not available in your TFC instance. To test this, tfe_client_mock errors if
// you try to set any Terraform version for this specific workspace name.
workspaceName := "unavailable-terraform-version"
config := cty.ObjectVal(map[string]cty.Value{
"hostname": cty.NullVal(cty.String),
"organization": cty.StringVal("hashicorp"),
"token": cty.NullVal(cty.String),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.NullVal(cty.String),
"tags": cty.SetVal(
[]cty.Value{
cty.StringVal("sometag"),
},
),
}),
})
b, bCleanup := testBackend(t, config)
defer bCleanup()
// Make sure the workspace doesn't exist yet -- otherwise, we can't test what
// happens when a workspace gets created. This is why we can't use "name" in
// the backend config above, btw: if you do, testBackend() creates the default
// workspace before we get a chance to do anything.
_, err := b.client.Workspaces.Read(context.Background(), b.organization, workspaceName)
if err != tfe.ErrResourceNotFound {
t.Fatalf("the workspace we were about to try and create (%s/%s) already exists in the mocks somehow, so this test isn't trustworthy anymore", b.organization, workspaceName)
}
_, err = b.StateMgr(workspaceName)
if err != nil {
t.Fatalf("expected no error from StateMgr, despite not being able to set remote Terraform version: %#v", err)
}
// Make sure the workspace was created:
workspace, err := b.client.Workspaces.Read(context.Background(), b.organization, workspaceName)
if err != nil {
t.Fatalf("b.StateMgr() didn't actually create the desired workspace")
}
// Make sure our mocks still error as expected, using the same update function b.StateMgr() would call:
_, err = b.client.Workspaces.UpdateByID(
context.Background(),
workspace.ID,
tfe.WorkspaceUpdateOptions{TerraformVersion: tfe.String("1.1.0")},
)
if err == nil {
t.Fatalf("the mocks aren't emulating a nonexistent remote Terraform version correctly, so this test isn't trustworthy anymore")
}
}
func TestCloud_setConfigurationFields(t *testing.T) {
originalForceBackendEnv := os.Getenv("TF_FORCE_LOCAL_BACKEND")
cases := map[string]struct {
obj cty.Value
expectedHostname string
expectedOrganziation string
expectedWorkspaceName string
expectedWorkspaceTags []string
expectedForceLocal bool
setEnv func()
resetEnv func()
expectedErr string
}{
"with hostname set": {
obj: cty.ObjectVal(map[string]cty.Value{
"organization": cty.StringVal("hashicorp"),
"hostname": cty.StringVal("hashicorp.com"),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.StringVal("prod"),
"tags": cty.NullVal(cty.Set(cty.String)),
}),
}),
expectedHostname: "hashicorp.com",
expectedOrganziation: "hashicorp",
},
"with hostname not set, set to default hostname": {
obj: cty.ObjectVal(map[string]cty.Value{
"organization": cty.StringVal("hashicorp"),
"hostname": cty.NullVal(cty.String),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.StringVal("prod"),
"tags": cty.NullVal(cty.Set(cty.String)),
}),
}),
expectedHostname: defaultHostname,
expectedOrganziation: "hashicorp",
},
"with workspace name set": {
obj: cty.ObjectVal(map[string]cty.Value{
"organization": cty.StringVal("hashicorp"),
"hostname": cty.StringVal("hashicorp.com"),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.StringVal("prod"),
"tags": cty.NullVal(cty.Set(cty.String)),
}),
}),
expectedHostname: "hashicorp.com",
expectedOrganziation: "hashicorp",
expectedWorkspaceName: "prod",
},
"with workspace tags set": {
obj: cty.ObjectVal(map[string]cty.Value{
"organization": cty.StringVal("hashicorp"),
"hostname": cty.StringVal("hashicorp.com"),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.NullVal(cty.String),
"tags": cty.SetVal(
[]cty.Value{
cty.StringVal("billing"),
},
),
}),
}),
expectedHostname: "hashicorp.com",
expectedOrganziation: "hashicorp",
expectedWorkspaceTags: []string{"billing"},
},
"with force local set": {
obj: cty.ObjectVal(map[string]cty.Value{
"organization": cty.StringVal("hashicorp"),
"hostname": cty.StringVal("hashicorp.com"),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.NullVal(cty.String),
"tags": cty.NullVal(cty.Set(cty.String)),
}),
}),
expectedHostname: "hashicorp.com",
expectedOrganziation: "hashicorp",
setEnv: func() {
os.Setenv("TF_FORCE_LOCAL_BACKEND", "1")
},
resetEnv: func() {
os.Setenv("TF_FORCE_LOCAL_BACKEND", originalForceBackendEnv)
},
expectedForceLocal: true,
},
}
for name, tc := range cases {
b := &Cloud{}
// if `setEnv` is set, then we expect `resetEnv` to also be set
if tc.setEnv != nil {
tc.setEnv()
defer tc.resetEnv()
}
errDiags := b.setConfigurationFields(tc.obj)
if errDiags.HasErrors() || tc.expectedErr != "" {
actualErr := errDiags.Err().Error()
if !strings.Contains(actualErr, tc.expectedErr) {
t.Fatalf("%s: unexpected validation result: %v", name, errDiags.Err())
}
}
if tc.expectedHostname != "" && b.hostname != tc.expectedHostname {
t.Fatalf("%s: expected hostname %s to match configured hostname %s", name, b.hostname, tc.expectedHostname)
}
if tc.expectedOrganziation != "" && b.organization != tc.expectedOrganziation {
t.Fatalf("%s: expected organization (%s) to match configured organization (%s)", name, b.organization, tc.expectedOrganziation)
}
if tc.expectedWorkspaceName != "" && b.WorkspaceMapping.Name != tc.expectedWorkspaceName {
t.Fatalf("%s: expected workspace name mapping (%s) to match configured workspace name (%s)", name, b.WorkspaceMapping.Name, tc.expectedWorkspaceName)
}
if len(tc.expectedWorkspaceTags) > 0 {
presentSet := make(map[string]struct{})
for _, tag := range b.WorkspaceMapping.Tags {
presentSet[tag] = struct{}{}
}
expectedSet := make(map[string]struct{})
for _, tag := range tc.expectedWorkspaceTags {
expectedSet[tag] = struct{}{}
}
var missing []string
var unexpected []string
for _, expected := range tc.expectedWorkspaceTags {
if _, ok := presentSet[expected]; !ok {
missing = append(missing, expected)
}
}
for _, actual := range b.WorkspaceMapping.Tags {
if _, ok := expectedSet[actual]; !ok {
unexpected = append(missing, actual)
}
}
if len(missing) > 0 {
t.Fatalf("%s: expected workspace tag mapping (%s) to contain the following tags: %s", name, b.WorkspaceMapping.Tags, missing)
}
if len(unexpected) > 0 {
t.Fatalf("%s: expected workspace tag mapping (%s) to NOT contain the following tags: %s", name, b.WorkspaceMapping.Tags, unexpected)
}
}
if tc.expectedForceLocal != false && b.forceLocal != tc.expectedForceLocal {
t.Fatalf("%s: expected force local backend to be set ", name)
}
}
}
func TestCloud_localBackend(t *testing.T) {
b, bCleanup := testBackendWithName(t)
defer bCleanup()
local, ok := b.local.(*backendLocal.Local)
if !ok {
t.Fatalf("expected b.local to be \"*local.Local\", got: %T", b.local)
}
cloud, ok := local.Backend.(*Cloud)
if !ok {
t.Fatalf("expected local.Backend to be *cloud.Cloud, got: %T", cloud)
}
}
func TestCloud_addAndRemoveWorkspacesDefault(t *testing.T) {
b, bCleanup := testBackendWithName(t)
defer bCleanup()
if _, err := b.StateMgr(testBackendSingleWorkspaceName); err != nil {
t.Fatalf("expected no error, got %v", err)
}
if err := b.DeleteWorkspace(testBackendSingleWorkspaceName); err != backend.ErrWorkspacesNotSupported {
t.Fatalf("expected error %v, got %v", backend.ErrWorkspacesNotSupported, err)
}
}
func TestCloud_StateMgr_versionCheck(t *testing.T) {
b, bCleanup := testBackendWithName(t)
defer bCleanup()
// Some fixed versions for testing with. This logic is a simple string
// comparison, so we don't need many test cases.
v0135 := version.Must(version.NewSemver("0.13.5"))
v0140 := version.Must(version.NewSemver("0.14.0"))
// Save original local version state and restore afterwards
p := tfversion.Prerelease
v := tfversion.Version
s := tfversion.SemVer
defer func() {
tfversion.Prerelease = p
tfversion.Version = v
tfversion.SemVer = s
}()
// For this test, the local Terraform version is set to 0.14.0
tfversion.Prerelease = ""
tfversion.Version = v0140.String()
tfversion.SemVer = v0140
// Update the mock remote workspace Terraform version to match the local
// Terraform version
if _, err := b.client.Workspaces.Update(
context.Background(),
b.organization,
b.WorkspaceMapping.Name,
tfe.WorkspaceUpdateOptions{
TerraformVersion: tfe.String(v0140.String()),
},
); err != nil {
t.Fatalf("error: %v", err)
}
// This should succeed
if _, err := b.StateMgr(testBackendSingleWorkspaceName); err != nil {
t.Fatalf("expected no error, got %v", err)
}
// Now change the remote workspace to a different Terraform version
if _, err := b.client.Workspaces.Update(
context.Background(),
b.organization,
b.WorkspaceMapping.Name,
tfe.WorkspaceUpdateOptions{
TerraformVersion: tfe.String(v0135.String()),
},
); err != nil {
t.Fatalf("error: %v", err)
}
// This should fail
want := `Remote workspace Terraform version "0.13.5" does not match local Terraform version "0.14.0"`
if _, err := b.StateMgr(testBackendSingleWorkspaceName); err.Error() != want {
t.Fatalf("wrong error\n got: %v\nwant: %v", err.Error(), want)
}
}
func TestCloud_StateMgr_versionCheckLatest(t *testing.T) {
b, bCleanup := testBackendWithName(t)
defer bCleanup()
v0140 := version.Must(version.NewSemver("0.14.0"))
// Save original local version state and restore afterwards
p := tfversion.Prerelease
v := tfversion.Version
s := tfversion.SemVer
defer func() {
tfversion.Prerelease = p
tfversion.Version = v
tfversion.SemVer = s
}()
// For this test, the local Terraform version is set to 0.14.0
tfversion.Prerelease = ""
tfversion.Version = v0140.String()
tfversion.SemVer = v0140
// Update the remote workspace to the pseudo-version "latest"
if _, err := b.client.Workspaces.Update(
context.Background(),
b.organization,
b.WorkspaceMapping.Name,
tfe.WorkspaceUpdateOptions{
TerraformVersion: tfe.String("latest"),
},
); err != nil {
t.Fatalf("error: %v", err)
}
// This should succeed despite not being a string match
if _, err := b.StateMgr(testBackendSingleWorkspaceName); err != nil {
t.Fatalf("expected no error, got %v", err)
}
}
func TestCloud_VerifyWorkspaceTerraformVersion(t *testing.T) {
testCases := []struct {
local string
remote string
operations bool
wantErr bool
}{
{"0.13.5", "0.13.5", true, false},
{"0.14.0", "0.13.5", true, true},
{"0.14.0", "0.13.5", false, false},
{"0.14.0", "0.14.1", true, false},
{"0.14.0", "1.0.99", true, false},
{"0.14.0", "1.1.0", true, false},
{"0.14.0", "1.2.0", true, true},
{"1.2.0", "1.2.99", true, false},
{"1.2.0", "1.3.0", true, true},
{"0.15.0", "latest", true, false},
{"1.1.5", "~> 1.1.1", true, false},
{"1.1.5", "> 1.1.0, < 1.3.0", true, false},
{"1.1.5", "~> 1.0.1", true, true},
// pre-release versions are comparable within their pre-release stage (dev,
// alpha, beta), but not comparable to different stages and not comparable
// to final releases.
{"1.1.0-beta1", "~> 1.1.0-beta", true, false},
{"1.1.0", "~> 1.1.0-beta", true, true},
{"1.1.0-beta1", "~> 1.1.0-dev", true, true},
}
for _, tc := range testCases {
t.Run(fmt.Sprintf("local %s, remote %s", tc.local, tc.remote), func(t *testing.T) {
b, bCleanup := testBackendWithName(t)
defer bCleanup()
local := version.Must(version.NewSemver(tc.local))
// Save original local version state and restore afterwards
p := tfversion.Prerelease
v := tfversion.Version
s := tfversion.SemVer
defer func() {
tfversion.Prerelease = p
tfversion.Version = v
tfversion.SemVer = s
}()
// Override local version as specified
tfversion.Prerelease = ""
tfversion.Version = local.String()
tfversion.SemVer = local
// Update the mock remote workspace Terraform version to the
// specified remote version
if _, err := b.client.Workspaces.Update(
context.Background(),
b.organization,
b.WorkspaceMapping.Name,
tfe.WorkspaceUpdateOptions{
Operations: tfe.Bool(tc.operations),
TerraformVersion: tfe.String(tc.remote),
},
); err != nil {
t.Fatalf("error: %v", err)
}
diags := b.VerifyWorkspaceTerraformVersion(backend.DefaultStateName)
if tc.wantErr {
if len(diags) != 1 {
t.Fatal("expected diag, but none returned")
}
if got := diags.Err().Error(); !strings.Contains(got, "Incompatible Terraform version") {
t.Fatalf("unexpected error: %s", got)
}
} else {
if len(diags) != 0 {
t.Fatalf("unexpected diags: %s", diags.Err())
}
}
})
}
}
func TestCloud_VerifyWorkspaceTerraformVersion_workspaceErrors(t *testing.T) {
b, bCleanup := testBackendWithName(t)
defer bCleanup()
// Attempting to check the version against a workspace which doesn't exist
// should result in no errors
diags := b.VerifyWorkspaceTerraformVersion("invalid-workspace")
if len(diags) != 0 {
t.Fatalf("unexpected error: %s", diags.Err())
}
// Use a special workspace ID to trigger a 500 error, which should result
// in a failed check
diags = b.VerifyWorkspaceTerraformVersion("network-error")
if len(diags) != 1 {
t.Fatal("expected diag, but none returned")
}
if got := diags.Err().Error(); !strings.Contains(got, "Error looking up workspace: Workspace read failed") {
t.Fatalf("unexpected error: %s", got)
}
// Update the mock remote workspace Terraform version to an invalid version
if _, err := b.client.Workspaces.Update(
context.Background(),
b.organization,
b.WorkspaceMapping.Name,
tfe.WorkspaceUpdateOptions{
TerraformVersion: tfe.String("1.0.cheetarah"),
},
); err != nil {
t.Fatalf("error: %v", err)
}
diags = b.VerifyWorkspaceTerraformVersion(backend.DefaultStateName)
if len(diags) != 1 {
t.Fatal("expected diag, but none returned")
}
if got := diags.Err().Error(); !strings.Contains(got, "Incompatible Terraform version: The remote workspace specified") {
t.Fatalf("unexpected error: %s", got)
}
}
func TestCloud_VerifyWorkspaceTerraformVersion_ignoreFlagSet(t *testing.T) {
b, bCleanup := testBackendWithName(t)
defer bCleanup()
// If the ignore flag is set, the behaviour changes
b.IgnoreVersionConflict()
// Different local & remote versions to cause an error
local := version.Must(version.NewSemver("0.14.0"))
remote := version.Must(version.NewSemver("0.13.5"))
// Save original local version state and restore afterwards
p := tfversion.Prerelease
v := tfversion.Version
s := tfversion.SemVer
defer func() {
tfversion.Prerelease = p
tfversion.Version = v
tfversion.SemVer = s
}()
// Override local version as specified
tfversion.Prerelease = ""
tfversion.Version = local.String()
tfversion.SemVer = local
// Update the mock remote workspace Terraform version to the
// specified remote version
if _, err := b.client.Workspaces.Update(
context.Background(),
b.organization,
b.WorkspaceMapping.Name,
tfe.WorkspaceUpdateOptions{
TerraformVersion: tfe.String(remote.String()),
},
); err != nil {
t.Fatalf("error: %v", err)
}
diags := b.VerifyWorkspaceTerraformVersion(backend.DefaultStateName)
if len(diags) != 1 {
t.Fatal("expected diag, but none returned")
}
if got, want := diags[0].Severity(), tfdiags.Warning; got != want {
t.Errorf("wrong severity: got %#v, want %#v", got, want)
}
if got, want := diags[0].Description().Summary, "Incompatible Terraform version"; got != want {
t.Errorf("wrong summary: got %s, want %s", got, want)
}
wantDetail := "The local Terraform version (0.14.0) does not meet the version requirements for remote workspace hashicorp/app-prod (0.13.5)."
if got := diags[0].Description().Detail; got != wantDetail {
t.Errorf("wrong summary: got %s, want %s", got, wantDetail)
}
}

52
internal/cloud/cloud_variables.go Normal file
View File

@ -0,0 +1,52 @@
package cloud
import (
"encoding/json"
"fmt"
"github.com/hashicorp/terraform/internal/backend"
"github.com/hashicorp/terraform/internal/configs"
"github.com/hashicorp/terraform/internal/terraform"
"github.com/hashicorp/terraform/internal/tfdiags"
ctyjson "github.com/zclconf/go-cty/cty/json"
)
func allowedSourceType(source terraform.ValueSourceType) bool {
return source == terraform.ValueFromNamedFile || source == terraform.ValueFromCLIArg || source == terraform.ValueFromEnvVar
}
// ParseCloudRunVariables accepts a mapping of unparsed values and a mapping of variable
// declarations and returns a name/value variable map appropriate for an API run context,
// that is, containing declared string variables only sourced from non-file inputs like CLI args
// and environment variables. However, all variable parsing diagnostics are returned
// in order to allow callers to short circuit cloud runs that contain variable
// declaration or parsing errors. The only exception is that missing required values are not
// considered errors because they may be defined within the cloud workspace.
func ParseCloudRunVariables(vv map[string]backend.UnparsedVariableValue, decls map[string]*configs.Variable) (map[string]string, tfdiags.Diagnostics) {
declared, diags := backend.ParseDeclaredVariableValues(vv, decls)
_, undedeclaredDiags := backend.ParseUndeclaredVariableValues(vv, decls)
diags = diags.Append(undedeclaredDiags)
ret := make(map[string]string, len(declared))
// Even if there are parsing or declaration errors, populate the return map with the
// variables that could be used for cloud runs
for name, v := range declared {
if !allowedSourceType(v.SourceType) {
continue
}
valueData, err := ctyjson.Marshal(v.Value, v.Value.Type())
if err != nil {
return nil, diags.Append(fmt.Errorf("error marshaling input variable value as json: %w", err))
}
var variableValue string
if err = json.Unmarshal(valueData, &variableValue); err != nil {
// This should never happen since cty marshaled the value to begin with without error
return nil, diags.Append(fmt.Errorf("error unmarshaling run variable: %w", err))
}
ret[name] = variableValue
}
return ret, diags
}

119
internal/cloud/cloud_variables_test.go Normal file
View File

@ -0,0 +1,119 @@
package cloud
import (
"testing"
"github.com/google/go-cmp/cmp"
"github.com/hashicorp/hcl/v2"
"github.com/hashicorp/terraform/internal/backend"
"github.com/hashicorp/terraform/internal/configs"
"github.com/hashicorp/terraform/internal/terraform"
"github.com/hashicorp/terraform/internal/tfdiags"
"github.com/zclconf/go-cty/cty"
)
func TestParseCloudRunVariables(t *testing.T) {
t.Run("populates variables from allowed sources", func(t *testing.T) {
vv := map[string]backend.UnparsedVariableValue{
"undeclared": testUnparsedVariableValue{source: terraform.ValueFromCLIArg, value: "0"},
"declaredFromConfig": testUnparsedVariableValue{source: terraform.ValueFromConfig, value: "1"},
"declaredFromNamedFile": testUnparsedVariableValue{source: terraform.ValueFromNamedFile, value: "2"},
"declaredFromCLIArg": testUnparsedVariableValue{source: terraform.ValueFromCLIArg, value: "3"},
"declaredFromEnvVar": testUnparsedVariableValue{source: terraform.ValueFromEnvVar, value: "4"},
}
decls := map[string]*configs.Variable{
"declaredFromConfig": {
Name: "declaredFromConfig",
Type: cty.String,
ConstraintType: cty.String,
ParsingMode: configs.VariableParseLiteral,
DeclRange: hcl.Range{
Filename: "fake.tf",
Start: hcl.Pos{Line: 2, Column: 1, Byte: 0},
End: hcl.Pos{Line: 2, Column: 1, Byte: 0},
},
},
"declaredFromNamedFile": {
Name: "declaredFromNamedFile",
Type: cty.String,
ConstraintType: cty.String,
ParsingMode: configs.VariableParseLiteral,
DeclRange: hcl.Range{
Filename: "fake.tf",
Start: hcl.Pos{Line: 2, Column: 1, Byte: 0},
End: hcl.Pos{Line: 2, Column: 1, Byte: 0},
},
},
"declaredFromCLIArg": {
Name: "declaredFromCLIArg",
Type: cty.String,
ConstraintType: cty.String,
ParsingMode: configs.VariableParseLiteral,
DeclRange: hcl.Range{
Filename: "fake.tf",
Start: hcl.Pos{Line: 2, Column: 1, Byte: 0},
End: hcl.Pos{Line: 2, Column: 1, Byte: 0},
},
},
"declaredFromEnvVar": {
Name: "declaredFromEnvVar",
Type: cty.String,
ConstraintType: cty.String,
ParsingMode: configs.VariableParseLiteral,
DeclRange: hcl.Range{
Filename: "fake.tf",
Start: hcl.Pos{Line: 2, Column: 1, Byte: 0},
End: hcl.Pos{Line: 2, Column: 1, Byte: 0},
},
},
"missing": {
Name: "missing",
Type: cty.String,
ConstraintType: cty.String,
Default: cty.StringVal("2"),
ParsingMode: configs.VariableParseLiteral,
DeclRange: hcl.Range{
Filename: "fake.tf",
Start: hcl.Pos{Line: 2, Column: 1, Byte: 0},
End: hcl.Pos{Line: 2, Column: 1, Byte: 0},
},
},
}
wantVals := make(map[string]string)
wantVals["declaredFromNamedFile"] = "2"
wantVals["declaredFromCLIArg"] = "3"
wantVals["declaredFromEnvVar"] = "4"
gotVals, diags := ParseCloudRunVariables(vv, decls)
if diff := cmp.Diff(wantVals, gotVals, cmp.Comparer(cty.Value.RawEquals)); diff != "" {
t.Errorf("wrong result\n%s", diff)
}
if got, want := len(diags), 1; got != want {
t.Fatalf("expected 1 variable error: %v, got %v", diags.Err(), want)
}
if got, want := diags[0].Description().Summary, "Value for undeclared variable"; got != want {
t.Errorf("wrong summary for diagnostic 0\ngot: %s\nwant: %s", got, want)
}
})
}
type testUnparsedVariableValue struct {
source terraform.ValueSourceType
value string
}
func (v testUnparsedVariableValue) ParseVariableValue(mode configs.VariableParsingMode) (*terraform.InputValue, tfdiags.Diagnostics) {
return &terraform.InputValue{
Value: cty.StringVal(v.value),
SourceType: v.source,
SourceRange: tfdiags.SourceRange{
Filename: "fake.tfvars",
Start: tfdiags.SourcePos{Line: 1, Column: 1, Byte: 0},
End: tfdiags.SourcePos{Line: 1, Column: 1, Byte: 0},
},
}, nil
}

256
internal/cloud/e2e/apply_auto_approve_test.go Normal file
View File

@ -0,0 +1,256 @@
//go:build e2e
// +build e2e
package main
import (
"context"
"io/ioutil"
"log"
"os"
"testing"
expect "github.com/Netflix/go-expect"
tfe "github.com/hashicorp/go-tfe"
"github.com/hashicorp/terraform/internal/e2e"
tfversion "github.com/hashicorp/terraform/version"
)
func Test_terraform_apply_autoApprove(t *testing.T) {
skipWithoutRemoteTerraformVersion(t)
ctx := context.Background()
cases := map[string]struct {
operations []operationSets
validations func(t *testing.T, orgName string)
}{
"workspace manual apply, terraform apply without auto-approve, expect prompt": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
wsName := "app"
_ = createWorkspace(t, orgName, tfe.WorkspaceCreateOptions{
Name: tfe.String(wsName),
TerraformVersion: tfe.String(tfversion.String()),
AutoApply: tfe.Bool(false),
})
tfBlock := terraformConfigCloudBackendName(orgName, wsName)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Terraform Cloud has been successfully initialized!`,
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions in workspace "app"?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
},
},
},
validations: func(t *testing.T, orgName string) {
workspace, err := tfeClient.Workspaces.ReadWithOptions(ctx, orgName, "app", &tfe.WorkspaceReadOptions{Include: "current_run"})
if err != nil {
t.Fatal(err)
}
if workspace.CurrentRun == nil {
t.Fatal("Expected workspace to have run, but got nil")
}
if workspace.CurrentRun.Status != tfe.RunApplied {
t.Fatalf("Expected run status to be `applied`, but is %s", workspace.CurrentRun.Status)
}
},
},
"workspace auto apply, terraform apply without auto-approve, expect prompt": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
wsName := "app"
_ = createWorkspace(t, orgName, tfe.WorkspaceCreateOptions{
Name: tfe.String(wsName),
TerraformVersion: tfe.String(tfversion.String()),
AutoApply: tfe.Bool(true),
})
tfBlock := terraformConfigCloudBackendName(orgName, wsName)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Terraform Cloud has been successfully initialized!`,
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions in workspace "app"?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
},
},
},
validations: func(t *testing.T, orgName string) {
workspace, err := tfeClient.Workspaces.ReadWithOptions(ctx, orgName, "app", &tfe.WorkspaceReadOptions{Include: "current_run"})
if err != nil {
t.Fatal(err)
}
if workspace.CurrentRun == nil {
t.Fatal("Expected workspace to have run, but got nil")
}
if workspace.CurrentRun.Status != tfe.RunApplied {
t.Fatalf("Expected run status to be `applied`, but is %s", workspace.CurrentRun.Status)
}
},
},
"workspace manual apply, terraform apply with auto-approve, no prompt": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
wsName := "app"
_ = createWorkspace(t, orgName, tfe.WorkspaceCreateOptions{
Name: tfe.String(wsName),
TerraformVersion: tfe.String(tfversion.String()),
AutoApply: tfe.Bool(false),
})
tfBlock := terraformConfigCloudBackendName(orgName, wsName)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Terraform Cloud has been successfully initialized!`,
},
{
command: []string{"apply", "-auto-approve"},
expectedCmdOutput: `Apply complete!`,
},
},
},
},
validations: func(t *testing.T, orgName string) {
workspace, err := tfeClient.Workspaces.ReadWithOptions(ctx, orgName, "app", &tfe.WorkspaceReadOptions{Include: "current_run"})
if err != nil {
t.Fatal(err)
}
if workspace.CurrentRun == nil {
t.Fatal("Expected workspace to have run, but got nil")
}
if workspace.CurrentRun.Status != tfe.RunApplied {
t.Fatalf("Expected run status to be `applied`, but is %s", workspace.CurrentRun.Status)
}
},
},
"workspace auto apply, terraform apply with auto-approve, no prompt": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
wsName := "app"
_ = createWorkspace(t, orgName, tfe.WorkspaceCreateOptions{
Name: tfe.String(wsName),
TerraformVersion: tfe.String(tfversion.String()),
AutoApply: tfe.Bool(true),
})
tfBlock := terraformConfigCloudBackendName(orgName, wsName)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Terraform Cloud has been successfully initialized!`,
},
{
command: []string{"apply", "-auto-approve"},
expectedCmdOutput: `Apply complete!`,
},
},
},
},
validations: func(t *testing.T, orgName string) {
workspace, err := tfeClient.Workspaces.ReadWithOptions(ctx, orgName, "app", &tfe.WorkspaceReadOptions{Include: "current_run"})
if err != nil {
t.Fatal(err)
}
if workspace.CurrentRun == nil {
t.Fatal("Expected workspace to have run, but got nil")
}
if workspace.CurrentRun.Status != tfe.RunApplied {
t.Fatalf("Expected run status to be `applied`, but is %s", workspace.CurrentRun.Status)
}
},
},
}
for name, tc := range cases {
log.Println("Test: ", name)
organization, cleanup := createOrganization(t)
defer cleanup()
exp, err := expect.NewConsole(expect.WithStdout(os.Stdout), expect.WithDefaultTimeout(expectConsoleTimeout))
if err != nil {
t.Fatal(err)
}
defer exp.Close()
tmpDir, err := ioutil.TempDir("", "terraform-test")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmpDir)
tf := e2e.NewBinary(terraformBin, tmpDir)
tf.AddEnv("TF_LOG=info")
tf.AddEnv(cliConfigFileEnv)
defer tf.Close()
for _, op := range tc.operations {
op.prep(t, organization.Name, tf.WorkDir())
for _, tfCmd := range op.commands {
cmd := tf.Cmd(tfCmd.command...)
cmd.Stdin = exp.Tty()
cmd.Stdout = exp.Tty()
cmd.Stderr = exp.Tty()
err = cmd.Start()
if err != nil {
t.Fatal(err)
}
if tfCmd.expectedCmdOutput != "" {
_, err := exp.ExpectString(tfCmd.expectedCmdOutput)
if err != nil {
t.Fatal(err)
}
}
lenInput := len(tfCmd.userInput)
lenInputOutput := len(tfCmd.postInputOutput)
if lenInput > 0 {
for i := 0; i < lenInput; i++ {
input := tfCmd.userInput[i]
exp.SendLine(input)
// use the index to find the corresponding
// output that matches the input.
if lenInputOutput-1 >= i {
output := tfCmd.postInputOutput[i]
_, err := exp.ExpectString(output)
if err != nil {
t.Fatal(err)
}
}
}
}
err = cmd.Wait()
if err != nil {
t.Fatal(err)
}
}
}
if tc.validations != nil {
tc.validations(t, organization.Name)
}
}
}

143
internal/cloud/e2e/backend_apply_before_init_test.go Normal file
View File

@ -0,0 +1,143 @@
//go:build e2e
// +build e2e
package main
import (
"fmt"
"io/ioutil"
"os"
"testing"
expect "github.com/Netflix/go-expect"
"github.com/hashicorp/terraform/internal/e2e"
)
func Test_backend_apply_before_init(t *testing.T) {
skipWithoutRemoteTerraformVersion(t)
cases := map[string]struct {
operations []operationSets
}{
"terraform apply with cloud block - blank state": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
wsName := "new-workspace"
tfBlock := terraformConfigCloudBackendName(orgName, wsName)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"apply"},
expectedCmdOutput: `Terraform Cloud initialization required, please run "terraform init"`,
expectError: true,
},
},
},
},
},
"terraform apply with cloud block - local state": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
tfBlock := terraformConfigLocalBackend()
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Successfully configured the backend "local"!`,
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
},
},
{
prep: func(t *testing.T, orgName, dir string) {
wsName := "new-workspace"
tfBlock := terraformConfigCloudBackendName(orgName, wsName)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"apply"},
expectedCmdOutput: `Terraform Cloud initialization required, please run "terraform init"`,
expectError: true,
},
},
},
},
},
}
for name, tc := range cases {
fmt.Println("Test: ", name)
organization, cleanup := createOrganization(t)
defer cleanup()
exp, err := expect.NewConsole(expect.WithStdout(os.Stdout), expect.WithDefaultTimeout(expectConsoleTimeout))
if err != nil {
t.Fatal(err)
}
defer exp.Close()
tmpDir, err := ioutil.TempDir("", "terraform-test")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmpDir)
tf := e2e.NewBinary(terraformBin, tmpDir)
tf.AddEnv("TF_LOG=info")
tf.AddEnv(cliConfigFileEnv)
defer tf.Close()
for _, op := range tc.operations {
op.prep(t, organization.Name, tf.WorkDir())
for _, tfCmd := range op.commands {
cmd := tf.Cmd(tfCmd.command...)
cmd.Stdin = exp.Tty()
cmd.Stdout = exp.Tty()
cmd.Stderr = exp.Tty()
err = cmd.Start()
if err != nil {
t.Fatal(err)
}
if tfCmd.expectedCmdOutput != "" {
_, err := exp.ExpectString(tfCmd.expectedCmdOutput)
if err != nil {
t.Fatal(err)
}
}
lenInput := len(tfCmd.userInput)
lenInputOutput := len(tfCmd.postInputOutput)
if lenInput > 0 {
for i := 0; i < lenInput; i++ {
input := tfCmd.userInput[i]
exp.SendLine(input)
// use the index to find the corresponding
// output that matches the input.
if lenInputOutput-1 >= i {
output := tfCmd.postInputOutput[i]
_, err := exp.ExpectString(output)
if err != nil {
t.Fatal(err)
}
}
}
}
err = cmd.Wait()
if err != nil && !tfCmd.expectError {
t.Fatal(err)
}
}
}
}
}

234
internal/cloud/e2e/helper_test.go Normal file
View File

@ -0,0 +1,234 @@
//go:build e2e
// +build e2e
package main
import (
"context"
"fmt"
"os"
"testing"
"time"
tfe "github.com/hashicorp/go-tfe"
"github.com/hashicorp/go-uuid"
tfversion "github.com/hashicorp/terraform/version"
)
const (
expectConsoleTimeout = 15 * time.Second
)
type tfCommand struct {
command []string
expectedCmdOutput string
expectedErr string
expectError bool
userInput []string
postInputOutput []string
}
type operationSets struct {
commands []tfCommand
prep func(t *testing.T, orgName, dir string)
}
type testCases map[string]struct {
operations []operationSets
validations func(t *testing.T, orgName string)
}
func createOrganization(t *testing.T) (*tfe.Organization, func()) {
ctx := context.Background()
org, err := tfeClient.Organizations.Create(ctx, tfe.OrganizationCreateOptions{
Name: tfe.String("tst-" + randomString(t)),
Email: tfe.String(fmt.Sprintf("%s@tfe.local", randomString(t))),
CostEstimationEnabled: tfe.Bool(false),
})
if err != nil {
t.Fatal(err)
}
_, err = tfeClient.Admin.Organizations.Update(ctx, org.Name, tfe.AdminOrganizationUpdateOptions{
AccessBetaTools: tfe.Bool(true),
})
if err != nil {
t.Fatal(err)
}
return org, func() {
if err := tfeClient.Organizations.Delete(ctx, org.Name); err != nil {
t.Errorf("Error destroying organization! WARNING: Dangling resources\n"+
"may exist! The full error is shown below.\n\n"+
"Organization: %s\nError: %s", org.Name, err)
}
}
}
func createWorkspace(t *testing.T, orgName string, wOpts tfe.WorkspaceCreateOptions) *tfe.Workspace {
ctx := context.Background()
w, err := tfeClient.Workspaces.Create(ctx, orgName, wOpts)
if err != nil {
t.Fatal(err)
}
return w
}
func getWorkspace(workspaces []*tfe.Workspace, workspace string) (*tfe.Workspace, bool) {
for _, ws := range workspaces {
if ws.Name == workspace {
return ws, false
}
}
return nil, true
}
func randomString(t *testing.T) string {
v, err := uuid.GenerateUUID()
if err != nil {
t.Fatal(err)
}
return v
}
func terraformConfigLocalBackend() string {
return fmt.Sprintf(`
terraform {
backend "local" {
}
}
output "val" {
value = "${terraform.workspace}"
}
`)
}
func terraformConfigRemoteBackendName(org, name string) string {
return fmt.Sprintf(`
terraform {
backend "remote" {
hostname = "%s"
organization = "%s"
workspaces {
name = "%s"
}
}
}
output "val" {
value = "${terraform.workspace}"
}
`, tfeHostname, org, name)
}
func terraformConfigRemoteBackendPrefix(org, prefix string) string {
return fmt.Sprintf(`
terraform {
backend "remote" {
hostname = "%s"
organization = "%s"
workspaces {
prefix = "%s"
}
}
}
output "val" {
value = "${terraform.workspace}"
}
`, tfeHostname, org, prefix)
}
func terraformConfigCloudBackendTags(org, tag string) string {
return fmt.Sprintf(`
terraform {
cloud {
hostname = "%s"
organization = "%s"
workspaces {
tags = ["%s"]
}
}
}
output "tag_val" {
value = "%s"
}
`, tfeHostname, org, tag, tag)
}
func terraformConfigCloudBackendName(org, name string) string {
return fmt.Sprintf(`
terraform {
cloud {
hostname = "%s"
organization = "%s"
workspaces {
name = "%s"
}
}
}
output "val" {
value = "${terraform.workspace}"
}
`, tfeHostname, org, name)
}
func writeMainTF(t *testing.T, block string, dir string) {
f, err := os.Create(fmt.Sprintf("%s/main.tf", dir))
if err != nil {
t.Fatal(err)
}
_, err = f.WriteString(block)
if err != nil {
t.Fatal(err)
}
f.Close()
}
// Ensure that TFC/E has a particular terraform version.
func skipWithoutRemoteTerraformVersion(t *testing.T) {
version := tfversion.String()
opts := tfe.AdminTerraformVersionsListOptions{
ListOptions: tfe.ListOptions{
PageNumber: 1,
PageSize: 100,
},
}
hasVersion := false
findTfVersion:
for {
// TODO: update go-tfe Read() to retrieve a terraform version by name.
// Currently you can only retrieve by ID.
tfVersionList, err := tfeClient.Admin.TerraformVersions.List(context.Background(), opts)
if err != nil {
t.Fatalf("Could not retrieve list of terraform versions: %v", err)
}
for _, item := range tfVersionList.Items {
if item.Version == version {
hasVersion = true
break findTfVersion
}
}
// Exit the loop when we've seen all pages.
if tfVersionList.CurrentPage >= tfVersionList.TotalPages {
break
}
// Update the page number to get the next page.
opts.PageNumber = tfVersionList.NextPage
}
if !hasVersion {
t.Skip(fmt.Sprintf("Skipping test because TFC/E does not have current Terraform version to test with (%s)", version))
}
}

141
internal/cloud/e2e/main_test.go Normal file
View File

@ -0,0 +1,141 @@
//go:build e2e
// +build e2e
package main
import (
"fmt"
"io/ioutil"
"log"
"os"
"os/exec"
"strings"
"testing"
tfe "github.com/hashicorp/go-tfe"
tfversion "github.com/hashicorp/terraform/version"
)
var terraformBin string
var cliConfigFileEnv string
var tfeClient *tfe.Client
var tfeHostname string
var tfeToken string
func TestMain(m *testing.M) {
log.SetFlags(log.LstdFlags | log.Lshortfile)
if !accTest() {
// if TF_ACC is not set, we want to skip all these tests.
return
}
teardown := setup()
code := m.Run()
teardown()
os.Exit(code)
}
func accTest() bool {
// TF_ACC is set when we want to run acceptance tests, meaning it relies on
// network access.
return os.Getenv("TF_ACC") != ""
}
func setup() func() {
setTfeClient()
teardown := setupBinary()
return func() {
teardown()
}
}
func setTfeClient() {
hostname := os.Getenv("TFE_HOSTNAME")
token := os.Getenv("TFE_TOKEN")
if hostname == "" {
log.Fatal("hostname cannot be empty")
}
if token == "" {
log.Fatal("token cannot be empty")
}
tfeHostname = hostname
tfeToken = token
cfg := &tfe.Config{
Address: fmt.Sprintf("https://%s", hostname),
Token: token,
}
// Create a new TFE client.
client, err := tfe.NewClient(cfg)
if err != nil {
log.Fatal(err)
}
tfeClient = client
}
func setupBinary() func() {
log.Println("Setting up terraform binary")
tmpTerraformBinaryDir, err := ioutil.TempDir("", "terraform-test")
if err != nil {
log.Fatal(err)
}
log.Println(tmpTerraformBinaryDir)
currentDir, err := os.Getwd()
defer os.Chdir(currentDir)
if err != nil {
log.Fatal(err)
}
// Getting top level dir
dirPaths := strings.Split(currentDir, "/")
log.Println(currentDir)
topLevel := len(dirPaths) - 3
topDir := strings.Join(dirPaths[0:topLevel], "/")
if err := os.Chdir(topDir); err != nil {
log.Fatal(err)
}
cmd := exec.Command(
"go",
"build",
"-o", tmpTerraformBinaryDir,
"-ldflags", fmt.Sprintf("-X \"github.com/hashicorp/terraform/version.Prerelease=%s\"", tfversion.Prerelease),
)
err = cmd.Run()
if err != nil {
log.Fatal(err)
}
credFile := fmt.Sprintf("%s/dev.tfrc", tmpTerraformBinaryDir)
writeCredRC(credFile)
terraformBin = fmt.Sprintf("%s/terraform", tmpTerraformBinaryDir)
cliConfigFileEnv = fmt.Sprintf("TF_CLI_CONFIG_FILE=%s", credFile)
return func() {
os.RemoveAll(tmpTerraformBinaryDir)
}
}
func writeCredRC(file string) {
creds := credentialBlock()
f, err := os.Create(file)
if err != nil {
log.Fatal(err)
}
_, err = f.WriteString(creds)
if err != nil {
log.Fatal(err)
}
f.Close()
}
func credentialBlock() string {
return fmt.Sprintf(`
credentials "%s" {
token = "%s"
}`, tfeHostname, tfeToken)
}

438
internal/cloud/e2e/migrate_state_multi_to_tfc_test.go Normal file
View File

@ -0,0 +1,438 @@
//go:build e2e
// +build e2e
package main
import (
"context"
"io/ioutil"
"os"
"testing"
expect "github.com/Netflix/go-expect"
tfe "github.com/hashicorp/go-tfe"
"github.com/hashicorp/terraform/internal/e2e"
)
func Test_migrate_multi_to_tfc_cloud_name_strategy(t *testing.T) {
skipWithoutRemoteTerraformVersion(t)
ctx := context.Background()
cases := map[string]struct {
operations []operationSets
validations func(t *testing.T, orgName string)
}{
"migrating multiple workspaces to cloud using name strategy; current workspace is 'default'": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
tfBlock := terraformConfigLocalBackend()
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Successfully configured the backend "local"!`,
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
{
command: []string{"workspace", "new", "prod"},
expectedCmdOutput: `Created and switched to workspace "prod"!`,
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
{
command: []string{"workspace", "select", "default"},
expectedCmdOutput: `Switched to workspace "default".`,
},
},
},
{
prep: func(t *testing.T, orgName, dir string) {
wsName := "new-workspace"
tfBlock := terraformConfigCloudBackendName(orgName, wsName)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init", "-migrate-state"},
expectedCmdOutput: `Do you want to copy only your current workspace?`,
userInput: []string{"yes", "yes"},
postInputOutput: []string{
`Do you want to copy existing state to Terraform Cloud?`,
`Terraform Cloud has been successfully initialized!`},
},
{
command: []string{"workspace", "show"},
expectedCmdOutput: `new-workspace`, // this comes from the `prep` function
},
{
command: []string{"output"},
expectedCmdOutput: `val = "default"`, // this was the output of the current workspace selected before migration
},
},
},
},
validations: func(t *testing.T, orgName string) {
wsList, err := tfeClient.Workspaces.List(ctx, orgName, tfe.WorkspaceListOptions{})
if err != nil {
t.Fatal(err)
}
if len(wsList.Items) != 1 {
t.Fatalf("Expected the number of workspaces to be 1, but got %d", len(wsList.Items))
}
ws := wsList.Items[0]
// this workspace name is what exists in the cloud backend configuration block
if ws.Name != "new-workspace" {
t.Fatalf("Expected workspace to be `new-workspace`, but is %s", ws.Name)
}
},
},
"migrating multiple workspaces to cloud using name strategy; current workspace is 'prod'": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
tfBlock := terraformConfigLocalBackend()
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Successfully configured the backend "local"!`,
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
{
command: []string{"workspace", "new", "prod"},
expectedCmdOutput: `Created and switched to workspace "prod"!`,
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
},
},
{
prep: func(t *testing.T, orgName, dir string) {
wsName := "new-workspace"
tfBlock := terraformConfigCloudBackendName(orgName, wsName)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init", "-migrate-state"},
expectedCmdOutput: `Do you want to copy only your current workspace?`,
userInput: []string{"yes", "yes"},
postInputOutput: []string{
`Do you want to copy existing state to Terraform Cloud?`,
`Terraform Cloud has been successfully initialized!`},
},
{
command: []string{"workspace", "list"},
expectedCmdOutput: `new-workspace`, // this comes from the `prep` function
},
{
command: []string{"output"},
expectedCmdOutput: `val = "prod"`,
},
},
},
},
validations: func(t *testing.T, orgName string) {
wsList, err := tfeClient.Workspaces.List(ctx, orgName, tfe.WorkspaceListOptions{})
if err != nil {
t.Fatal(err)
}
ws := wsList.Items[0]
// this workspace name is what exists in the cloud backend configuration block
if ws.Name != "new-workspace" {
t.Fatalf("Expected workspace to be `new-workspace`, but is %s", ws.Name)
}
},
},
}
for name, tc := range cases {
t.Log("Test: ", name)
organization, cleanup := createOrganization(t)
defer cleanup()
exp, err := expect.NewConsole(expect.WithStdout(os.Stdout), expect.WithDefaultTimeout(expectConsoleTimeout))
if err != nil {
t.Fatal(err)
}
defer exp.Close()
tmpDir, err := ioutil.TempDir("", "terraform-test")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmpDir)
tf := e2e.NewBinary(terraformBin, tmpDir)
defer tf.Close()
tf.AddEnv("TF_LOG=INFO")
tf.AddEnv(cliConfigFileEnv)
for _, op := range tc.operations {
op.prep(t, organization.Name, tf.WorkDir())
for _, tfCmd := range op.commands {
t.Log("Running commands: ", tfCmd.command)
tfCmd.command = append(tfCmd.command)
cmd := tf.Cmd(tfCmd.command...)
cmd.Stdin = exp.Tty()
cmd.Stdout = exp.Tty()
cmd.Stderr = exp.Tty()
err = cmd.Start()
if err != nil {
t.Fatal(err)
}
if tfCmd.expectedCmdOutput != "" {
_, err := exp.ExpectString(tfCmd.expectedCmdOutput)
if err != nil {
t.Fatal(err)
}
}
lenInput := len(tfCmd.userInput)
lenInputOutput := len(tfCmd.postInputOutput)
if lenInput > 0 {
for i := 0; i < lenInput; i++ {
input := tfCmd.userInput[i]
exp.SendLine(input)
// use the index to find the corresponding
// output that matches the input.
if lenInputOutput-1 >= i {
output := tfCmd.postInputOutput[i]
_, err := exp.ExpectString(output)
if err != nil {
t.Fatal(err)
}
}
}
}
err = cmd.Wait()
if err != nil {
t.Fatal(err)
}
}
}
if tc.validations != nil {
tc.validations(t, organization.Name)
}
}
}
func Test_migrate_multi_to_tfc_cloud_tags_strategy(t *testing.T) {
skipWithoutRemoteTerraformVersion(t)
ctx := context.Background()
cases := map[string]struct {
operations []operationSets
validations func(t *testing.T, orgName string)
}{
"migrating multiple workspaces to cloud using tags strategy; pattern is using prefix `app-*`": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
tfBlock := terraformConfigLocalBackend()
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Successfully configured the backend "local"!`,
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
{
command: []string{"workspace", "new", "prod"},
expectedCmdOutput: `Created and switched to workspace "prod"!`,
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
{
command: []string{"workspace", "select", "default"},
expectedCmdOutput: `Switched to workspace "default".`,
},
{
command: []string{"output"},
expectedCmdOutput: `val = "default"`,
},
{
command: []string{"workspace", "select", "prod"},
expectedCmdOutput: `Switched to workspace "prod".`,
},
{
command: []string{"output"},
expectedCmdOutput: `val = "prod"`,
},
},
},
{
prep: func(t *testing.T, orgName, dir string) {
tag := "app"
tfBlock := terraformConfigCloudBackendTags(orgName, tag)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init", "-migrate-state"},
expectedCmdOutput: `Terraform Cloud requires all workspaces to be given an explicit name.`,
userInput: []string{"dev", "1", "app-*", "1"},
postInputOutput: []string{
`Would you like to rename your workspaces?`,
"What pattern would you like to add to all your workspaces?",
"The currently selected workspace (prod) does not exist.",
"Terraform Cloud has been successfully initialized!"},
},
{
command: []string{"workspace", "select", "app-prod"},
expectedCmdOutput: `Switched to workspace "app-prod".`,
},
{
command: []string{"output"},
expectedCmdOutput: `val = "prod"`,
},
{
command: []string{"workspace", "select", "app-dev"},
expectedCmdOutput: `Switched to workspace "app-dev".`,
},
{
command: []string{"output"},
expectedCmdOutput: `val = "default"`,
},
},
},
},
validations: func(t *testing.T, orgName string) {
wsList, err := tfeClient.Workspaces.List(ctx, orgName, tfe.WorkspaceListOptions{
Tags: tfe.String("app"),
})
if err != nil {
t.Fatal(err)
}
if len(wsList.Items) != 2 {
t.Fatalf("Expected the number of workspaecs to be 2, but got %d", len(wsList.Items))
}
expectedWorkspaceNames := []string{"app-prod", "app-dev"}
for _, ws := range wsList.Items {
hasName := false
for _, expectedNames := range expectedWorkspaceNames {
if expectedNames == ws.Name {
hasName = true
}
}
if !hasName {
t.Fatalf("Worksapce %s is not in the expected list of workspaces", ws.Name)
}
}
},
},
}
for name, tc := range cases {
t.Log("Test: ", name)
organization, cleanup := createOrganization(t)
t.Log(organization.Name)
defer cleanup()
exp, err := expect.NewConsole(expect.WithStdout(os.Stdout), expect.WithDefaultTimeout(expectConsoleTimeout))
if err != nil {
t.Fatal(err)
}
defer exp.Close()
tmpDir, err := ioutil.TempDir("", "terraform-test")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmpDir)
tf := e2e.NewBinary(terraformBin, tmpDir)
defer tf.Close()
tf.AddEnv("TF_LOG=INFO")
tf.AddEnv(cliConfigFileEnv)
for _, op := range tc.operations {
op.prep(t, organization.Name, tf.WorkDir())
for _, tfCmd := range op.commands {
t.Log("running commands: ", tfCmd.command)
cmd := tf.Cmd(tfCmd.command...)
cmd.Stdin = exp.Tty()
cmd.Stdout = exp.Tty()
cmd.Stderr = exp.Tty()
err = cmd.Start()
if err != nil {
t.Fatal(err)
}
if tfCmd.expectedCmdOutput != "" {
_, err := exp.ExpectString(tfCmd.expectedCmdOutput)
if err != nil {
t.Fatal(err)
}
}
lenInput := len(tfCmd.userInput)
lenInputOutput := len(tfCmd.postInputOutput)
if lenInput > 0 {
for i := 0; i < lenInput; i++ {
input := tfCmd.userInput[i]
exp.SendLine(input)
// use the index to find the corresponding
// output that matches the input.
if lenInputOutput-1 >= i {
output := tfCmd.postInputOutput[i]
if output == "" {
continue
}
_, err := exp.ExpectString(output)
if err != nil {
t.Fatal(err)
}
}
}
}
err = cmd.Wait()
if err != nil {
t.Fatal(err)
}
}
}
if tc.validations != nil {
tc.validations(t, organization.Name)
}
}
}

925
internal/cloud/e2e/migrate_state_remote_backend_to_tfc_test.go Normal file
View File

@ -0,0 +1,925 @@
//go:build e2e
// +build e2e
package main
import (
"context"
"io/ioutil"
"os"
"testing"
expect "github.com/Netflix/go-expect"
tfe "github.com/hashicorp/go-tfe"
"github.com/hashicorp/terraform/internal/e2e"
)
func Test_migrate_remote_backend_name_to_tfc_name(t *testing.T) {
skipWithoutRemoteTerraformVersion(t)
ctx := context.Background()
cases := map[string]struct {
operations []operationSets
validations func(t *testing.T, orgName string)
}{
"backend name strategy, to cloud with name strategy": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
remoteWorkspace := "remote-workspace"
tfBlock := terraformConfigRemoteBackendName(orgName, remoteWorkspace)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Successfully configured the backend "remote"!`,
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions in workspace "remote-workspace"?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
},
},
{
prep: func(t *testing.T, orgName, dir string) {
wsName := "cloud-workspace"
tfBlock := terraformConfigCloudBackendName(orgName, wsName)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init", "-migrate-state", "-ignore-remote-version"},
expectedCmdOutput: `Do you want to copy existing state to Terraform Cloud?`,
userInput: []string{"yes"},
postInputOutput: []string{`Terraform Cloud has been successfully initialized!`},
},
{
command: []string{"workspace", "show"},
expectedCmdOutput: `cloud-workspace`,
},
},
},
},
validations: func(t *testing.T, orgName string) {
expectedName := "cloud-workspace"
ws, err := tfeClient.Workspaces.Read(ctx, orgName, expectedName)
if err != nil {
t.Fatal(err)
}
if ws == nil {
t.Fatalf("Expected workspace %s to be present, but is not.", expectedName)
}
},
},
"backend name strategy, to cloud name strategy, using the same name": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
remoteWorkspace := "remote-workspace"
tfBlock := terraformConfigRemoteBackendName(orgName, remoteWorkspace)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Successfully configured the backend "remote"!`,
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions in workspace "remote-workspace"?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
},
},
{
prep: func(t *testing.T, orgName, dir string) {
wsName := "remote-workspace"
tfBlock := terraformConfigCloudBackendName(orgName, wsName)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init", "-migrate-state", "-ignore-remote-version"},
expectedCmdOutput: `Terraform Cloud has been successfully initialized!`,
},
{
command: []string{"workspace", "show"},
expectedCmdOutput: `remote-workspace`,
},
},
},
},
validations: func(t *testing.T, orgName string) {
expectedName := "remote-workspace"
ws, err := tfeClient.Workspaces.Read(ctx, orgName, expectedName)
if err != nil {
t.Fatal(err)
}
if ws == nil {
t.Fatalf("Expected workspace %s to be present, but is not.", expectedName)
}
},
},
}
for name, tc := range cases {
t.Log("Test: ", name)
exp, err := expect.NewConsole(expect.WithStdout(os.Stdout), expect.WithDefaultTimeout(expectConsoleTimeout))
if err != nil {
t.Fatal(err)
}
defer exp.Close()
tmpDir, err := ioutil.TempDir("", "terraform-test")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmpDir)
tf := e2e.NewBinary(terraformBin, tmpDir)
tf.AddEnv("TF_LOG=INFO")
tf.AddEnv(cliConfigFileEnv)
defer tf.Close()
organization, cleanup := createOrganization(t)
defer cleanup()
for _, op := range tc.operations {
op.prep(t, organization.Name, tf.WorkDir())
for _, tfCmd := range op.commands {
cmd := tf.Cmd(tfCmd.command...)
cmd.Stdin = exp.Tty()
cmd.Stdout = exp.Tty()
cmd.Stderr = exp.Tty()
err = cmd.Start()
if err != nil {
t.Fatal(err)
}
if tfCmd.expectedCmdOutput != "" {
_, err := exp.ExpectString(tfCmd.expectedCmdOutput)
if err != nil {
t.Fatal(err)
}
}
lenInput := len(tfCmd.userInput)
lenInputOutput := len(tfCmd.postInputOutput)
if lenInput > 0 {
for i := 0; i < lenInput; i++ {
input := tfCmd.userInput[i]
exp.SendLine(input)
// use the index to find the corresponding
// output that matches the input.
if lenInputOutput-1 >= i {
output := tfCmd.postInputOutput[i]
_, err := exp.ExpectString(output)
if err != nil {
t.Fatal(err)
}
}
}
}
err = cmd.Wait()
if err != nil {
t.Fatal(err)
}
}
}
if tc.validations != nil {
tc.validations(t, organization.Name)
}
}
}
func Test_migrate_remote_backend_name_to_tfc_name_different_org(t *testing.T) {
skipWithoutRemoteTerraformVersion(t)
ctx := context.Background()
cases := map[string]struct {
operations []operationSets
validations func(t *testing.T, orgName string)
}{
"backend name strategy, to cloud name strategy, using the same name, different organization": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
remoteWorkspace := "remote-workspace"
tfBlock := terraformConfigRemoteBackendName(orgName, remoteWorkspace)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Successfully configured the backend "remote"!`,
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions in workspace "remote-workspace"?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
},
},
{
prep: func(t *testing.T, orgName, dir string) {
wsName := "remote-workspace"
tfBlock := terraformConfigCloudBackendName(orgName, wsName)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init", "-migrate-state", "-ignore-remote-version"},
expectedCmdOutput: `Do you want to copy existing state to Terraform Cloud?`,
userInput: []string{"yes"},
postInputOutput: []string{`Terraform Cloud has been successfully initialized!`},
},
{
command: []string{"workspace", "show"},
expectedCmdOutput: `remote-workspace`,
},
},
},
},
validations: func(t *testing.T, orgName string) {
expectedName := "remote-workspace"
ws, err := tfeClient.Workspaces.Read(ctx, orgName, expectedName)
if err != nil {
t.Fatal(err)
}
if ws == nil {
t.Fatalf("Expected workspace %s to be present, but is not.", expectedName)
}
},
},
}
for name, tc := range cases {
t.Log("Test: ", name)
exp, err := expect.NewConsole(expect.WithStdout(os.Stdout), expect.WithDefaultTimeout(expectConsoleTimeout))
if err != nil {
t.Fatal(err)
}
defer exp.Close()
tmpDir, err := ioutil.TempDir("", "terraform-test")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmpDir)
tf := e2e.NewBinary(terraformBin, tmpDir)
tf.AddEnv("TF_LOG=INFO")
tf.AddEnv(cliConfigFileEnv)
defer tf.Close()
orgOne, cleanupOne := createOrganization(t)
orgTwo, cleanupTwo := createOrganization(t)
defer cleanupOne()
defer cleanupTwo()
orgs := []string{orgOne.Name, orgTwo.Name}
var orgName string
for index, op := range tc.operations {
orgName = orgs[index]
op.prep(t, orgName, tf.WorkDir())
for _, tfCmd := range op.commands {
cmd := tf.Cmd(tfCmd.command...)
cmd.Stdin = exp.Tty()
cmd.Stdout = exp.Tty()
cmd.Stderr = exp.Tty()
err = cmd.Start()
if err != nil {
t.Fatal(err)
}
if tfCmd.expectedCmdOutput != "" {
_, err := exp.ExpectString(tfCmd.expectedCmdOutput)
if err != nil {
t.Fatal(err)
}
}
lenInput := len(tfCmd.userInput)
lenInputOutput := len(tfCmd.postInputOutput)
if lenInput > 0 {
for i := 0; i < lenInput; i++ {
input := tfCmd.userInput[i]
exp.SendLine(input)
// use the index to find the corresponding
// output that matches the input.
if lenInputOutput-1 >= i {
output := tfCmd.postInputOutput[i]
_, err := exp.ExpectString(output)
if err != nil {
t.Fatal(err)
}
}
}
}
err = cmd.Wait()
if err != nil {
t.Fatal(err)
}
}
}
if tc.validations != nil {
tc.validations(t, orgName)
}
}
}
func Test_migrate_remote_backend_name_to_tfc_tags(t *testing.T) {
skipWithoutRemoteTerraformVersion(t)
ctx := context.Background()
cases := map[string]struct {
operations []operationSets
validations func(t *testing.T, orgName string)
}{
"single workspace with backend name strategy, to cloud with tags strategy": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
remoteWorkspace := "remote-workspace"
tfBlock := terraformConfigRemoteBackendName(orgName, remoteWorkspace)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Successfully configured the backend "remote"!`,
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions in workspace "remote-workspace"?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
{
command: []string{"workspace", "show"},
expectedCmdOutput: `default`,
},
},
},
{
prep: func(t *testing.T, orgName, dir string) {
tag := "app"
tfBlock := terraformConfigCloudBackendTags(orgName, tag)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init", "-migrate-state", "-ignore-remote-version"},
expectedCmdOutput: `Terraform Cloud requires all workspaces to be given an explicit name.`,
userInput: []string{"cloud-workspace", "yes"},
postInputOutput: []string{
`Do you want to copy existing state to Terraform Cloud?`,
`Terraform Cloud has been successfully initialized!`},
},
{
command: []string{"workspace", "show"},
expectedCmdOutput: `cloud-workspace`,
},
},
},
},
validations: func(t *testing.T, orgName string) {
wsList, err := tfeClient.Workspaces.List(ctx, orgName, tfe.WorkspaceListOptions{
Tags: tfe.String("app"),
})
if err != nil {
t.Fatal(err)
}
if len(wsList.Items) != 1 {
t.Fatalf("Expected number of workspaces to be 1, but got %d", len(wsList.Items))
}
ws := wsList.Items[0]
if ws.Name != "cloud-workspace" {
t.Fatalf("Expected workspace to be `cloud-workspace`, but is %s", ws.Name)
}
},
},
}
for name, tc := range cases {
t.Log("Test: ", name)
exp, err := expect.NewConsole(expect.WithStdout(os.Stdout), expect.WithDefaultTimeout(expectConsoleTimeout))
if err != nil {
t.Fatal(err)
}
defer exp.Close()
tmpDir, err := ioutil.TempDir("", "terraform-test")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmpDir)
tf := e2e.NewBinary(terraformBin, tmpDir)
tf.AddEnv("TF_LOG=INFO")
tf.AddEnv(cliConfigFileEnv)
defer tf.Close()
organization, cleanup := createOrganization(t)
defer cleanup()
for _, op := range tc.operations {
op.prep(t, organization.Name, tf.WorkDir())
for _, tfCmd := range op.commands {
cmd := tf.Cmd(tfCmd.command...)
cmd.Stdin = exp.Tty()
cmd.Stdout = exp.Tty()
cmd.Stderr = exp.Tty()
err = cmd.Start()
if err != nil {
t.Fatal(err)
}
if tfCmd.expectedCmdOutput != "" {
_, err := exp.ExpectString(tfCmd.expectedCmdOutput)
if err != nil {
t.Fatal(err)
}
}
lenInput := len(tfCmd.userInput)
lenInputOutput := len(tfCmd.postInputOutput)
if lenInput > 0 {
for i := 0; i < lenInput; i++ {
input := tfCmd.userInput[i]
exp.SendLine(input)
// use the index to find the corresponding
// output that matches the input.
if lenInputOutput-1 >= i {
output := tfCmd.postInputOutput[i]
_, err := exp.ExpectString(output)
if err != nil {
t.Fatal(err)
}
}
}
}
err = cmd.Wait()
if err != nil {
t.Fatal(err)
}
}
}
if tc.validations != nil {
tc.validations(t, organization.Name)
}
}
}
func Test_migrate_remote_backend_prefix_to_tfc_name(t *testing.T) {
skipWithoutRemoteTerraformVersion(t)
ctx := context.Background()
cases := map[string]struct {
operations []operationSets
validations func(t *testing.T, orgName string)
}{
"single workspace with backend prefix strategy, to cloud with name strategy": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
_ = createWorkspace(t, orgName, tfe.WorkspaceCreateOptions{Name: tfe.String("app-one")})
prefix := "app-"
tfBlock := terraformConfigRemoteBackendPrefix(orgName, prefix)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Terraform has been successfully initialized!`,
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions in workspace "app-one"?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
},
},
{
prep: func(t *testing.T, orgName, dir string) {
wsName := "cloud-workspace"
tfBlock := terraformConfigCloudBackendName(orgName, wsName)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init", "-migrate-state", "-ignore-remote-version"},
expectedCmdOutput: `Do you want to copy existing state to Terraform Cloud?`,
userInput: []string{"yes"},
postInputOutput: []string{
`Terraform Cloud has been successfully initialized!`},
},
{
command: []string{"workspace", "show"},
expectedCmdOutput: `cloud-workspace`,
},
},
},
},
validations: func(t *testing.T, orgName string) {
expectedName := "cloud-workspace"
ws, err := tfeClient.Workspaces.Read(ctx, orgName, expectedName)
if err != nil {
t.Fatal(err)
}
if ws == nil {
t.Fatalf("Expected workspace %s to be present, but is not.", expectedName)
}
},
},
"multiple workspaces with backend prefix strategy, to cloud with name strategy": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
_ = createWorkspace(t, orgName, tfe.WorkspaceCreateOptions{Name: tfe.String("app-one")})
_ = createWorkspace(t, orgName, tfe.WorkspaceCreateOptions{Name: tfe.String("app-two")})
prefix := "app-"
tfBlock := terraformConfigRemoteBackendPrefix(orgName, prefix)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `The currently selected workspace (default) does not exist.`,
userInput: []string{"1"},
postInputOutput: []string{`Terraform has been successfully initialized!`},
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions in workspace "app-one"?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
{
command: []string{"workspace", "list"},
expectedCmdOutput: "* one", // app name retrieved via prefix
},
{
command: []string{"workspace", "select", "two"},
expectedCmdOutput: `Switched to workspace "two".`, // app name retrieved via prefix
},
},
},
{
prep: func(t *testing.T, orgName, dir string) {
wsName := "cloud-workspace"
tfBlock := terraformConfigCloudBackendName(orgName, wsName)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init", "-migrate-state", "-ignore-remote-version"},
expectedCmdOutput: `Do you want to copy only your current workspace?`,
userInput: []string{"yes"},
postInputOutput: []string{
`Terraform Cloud has been successfully initialized!`},
},
{
command: []string{"workspace", "show"},
expectedCmdOutput: `cloud-workspace`,
},
},
},
},
validations: func(t *testing.T, orgName string) {
expectedName := "cloud-workspace"
ws, err := tfeClient.Workspaces.Read(ctx, orgName, expectedName)
if err != nil {
t.Fatal(err)
}
if ws == nil {
t.Fatalf("Expected workspace %s to be present, but is not.", expectedName)
}
wsList, err := tfeClient.Workspaces.List(ctx, orgName, tfe.WorkspaceListOptions{})
if err != nil {
t.Fatal(err)
}
if len(wsList.Items) != 3 {
t.Fatalf("expected number of workspaces in this org to be 3, but got %d", len(wsList.Items))
}
ws, empty := getWorkspace(wsList.Items, "cloud-workspace")
if empty {
t.Fatalf("expected workspaces to include 'cloud-workspace' but didn't.")
}
ws, empty = getWorkspace(wsList.Items, "app-one")
if empty {
t.Fatalf("expected workspaces to include 'app-one' but didn't.")
}
ws, empty = getWorkspace(wsList.Items, "app-two")
if empty {
t.Fatalf("expected workspaces to include 'app-two' but didn't.")
}
},
},
}
for name, tc := range cases {
t.Log("Test: ", name)
exp, err := expect.NewConsole(expect.WithStdout(os.Stdout), expect.WithDefaultTimeout(expectConsoleTimeout))
if err != nil {
t.Fatal(err)
}
defer exp.Close()
tmpDir, err := ioutil.TempDir("", "terraform-test")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmpDir)
tf := e2e.NewBinary(terraformBin, tmpDir)
tf.AddEnv("TF_LOG=INFO")
tf.AddEnv(cliConfigFileEnv)
defer tf.Close()
organization, cleanup := createOrganization(t)
defer cleanup()
for _, op := range tc.operations {
op.prep(t, organization.Name, tf.WorkDir())
for _, tfCmd := range op.commands {
cmd := tf.Cmd(tfCmd.command...)
cmd.Stdin = exp.Tty()
cmd.Stdout = exp.Tty()
cmd.Stderr = exp.Tty()
err = cmd.Start()
if err != nil {
t.Fatal(err)
}
if tfCmd.expectedCmdOutput != "" {
_, err := exp.ExpectString(tfCmd.expectedCmdOutput)
if err != nil {
t.Fatal(err)
}
}
lenInput := len(tfCmd.userInput)
lenInputOutput := len(tfCmd.postInputOutput)
if lenInput > 0 {
for i := 0; i < lenInput; i++ {
input := tfCmd.userInput[i]
exp.SendLine(input)
// use the index to find the corresponding
// output that matches the input.
if lenInputOutput-1 >= i {
output := tfCmd.postInputOutput[i]
_, err := exp.ExpectString(output)
if err != nil {
t.Fatal(err)
}
}
}
}
err = cmd.Wait()
if err != nil {
t.Fatal(err)
}
}
}
if tc.validations != nil {
tc.validations(t, organization.Name)
}
}
}
func Test_migrate_remote_backend_prefix_to_tfc_tags(t *testing.T) {
skipWithoutRemoteTerraformVersion(t)
ctx := context.Background()
cases := map[string]struct {
operations []operationSets
validations func(t *testing.T, orgName string)
}{
"single workspace with backend prefix strategy, to cloud with tags strategy": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
_ = createWorkspace(t, orgName, tfe.WorkspaceCreateOptions{Name: tfe.String("app-one")})
prefix := "app-"
tfBlock := terraformConfigRemoteBackendPrefix(orgName, prefix)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Terraform has been successfully initialized!`,
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions in workspace "app-one"?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
},
},
{
prep: func(t *testing.T, orgName, dir string) {
tag := "app"
tfBlock := terraformConfigCloudBackendTags(orgName, tag)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init", "-migrate-state", "-ignore-remote-version"},
expectedCmdOutput: `Terraform Cloud requires all workspaces to be given an explicit name.`,
userInput: []string{"cloud-workspace", "yes"},
postInputOutput: []string{
`Do you want to copy existing state to Terraform Cloud?`,
`Terraform Cloud has been successfully initialized!`},
},
{
command: []string{"workspace", "list"},
expectedCmdOutput: `cloud-workspace`,
},
},
},
},
validations: func(t *testing.T, orgName string) {
expectedName := "cloud-workspace"
ws, err := tfeClient.Workspaces.Read(ctx, orgName, expectedName)
if err != nil {
t.Fatal(err)
}
if ws == nil {
t.Fatalf("Expected workspace %s to be present, but is not.", expectedName)
}
},
},
"multiple workspaces with backend prefix strategy, to cloud with tags strategy": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
_ = createWorkspace(t, orgName, tfe.WorkspaceCreateOptions{Name: tfe.String("app-one")})
_ = createWorkspace(t, orgName, tfe.WorkspaceCreateOptions{Name: tfe.String("app-two")})
prefix := "app-"
tfBlock := terraformConfigRemoteBackendPrefix(orgName, prefix)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `The currently selected workspace (default) does not exist.`,
userInput: []string{"1"},
postInputOutput: []string{`Terraform has been successfully initialized!`},
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions in workspace "app-one"?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
{
command: []string{"workspace", "select", "two"},
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions in workspace "app-two"?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
},
},
{
prep: func(t *testing.T, orgName, dir string) {
tag := "app"
tfBlock := terraformConfigCloudBackendTags(orgName, tag)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init", "-migrate-state", "-ignore-remote-version"},
expectedCmdOutput: `Would you like to rename your workspaces?`,
userInput: []string{"1", "*"},
postInputOutput: []string{`What pattern would you like to add to all your workspaces?`,
`Terraform Cloud has been successfully initialized!`},
},
{
command: []string{"workspace", "show"},
expectedCmdOutput: "two", // this comes from the original workspace name from the previous backend.
},
{
command: []string{"workspace", "select", "one"},
expectedCmdOutput: `Switched to workspace "one".`, // this comes from the original workspace name from the previous backend.
},
},
},
},
validations: func(t *testing.T, orgName string) {
wsList, err := tfeClient.Workspaces.List(ctx, orgName, tfe.WorkspaceListOptions{
Tags: tfe.String("app"),
})
if err != nil {
t.Fatal(err)
}
if len(wsList.Items) != 2 {
t.Logf("Expected the number of workspaces to be 2, but got %d", len(wsList.Items))
}
ws, empty := getWorkspace(wsList.Items, "one")
if empty {
t.Fatalf("expected workspaces to include 'one' but didn't.")
}
if len(ws.TagNames) == 0 {
t.Fatalf("expected workspaces 'one' to have tags.")
}
ws, empty = getWorkspace(wsList.Items, "two")
if empty {
t.Fatalf("expected workspaces to include 'two' but didn't.")
}
if len(ws.TagNames) == 0 {
t.Fatalf("expected workspaces 'two' to have tags.")
}
},
},
}
for name, tc := range cases {
t.Log("Test: ", name)
exp, err := expect.NewConsole(expect.WithStdout(os.Stdout), expect.WithDefaultTimeout(expectConsoleTimeout))
if err != nil {
t.Fatal(err)
}
defer exp.Close()
tmpDir, err := ioutil.TempDir("", "terraform-test")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmpDir)
tf := e2e.NewBinary(terraformBin, tmpDir)
tf.AddEnv("TF_LOG=INFO")
tf.AddEnv(cliConfigFileEnv)
defer tf.Close()
organization, cleanup := createOrganization(t)
defer cleanup()
for _, op := range tc.operations {
op.prep(t, organization.Name, tf.WorkDir())
for _, tfCmd := range op.commands {
cmd := tf.Cmd(tfCmd.command...)
cmd.Stdin = exp.Tty()
cmd.Stdout = exp.Tty()
cmd.Stderr = exp.Tty()
err = cmd.Start()
if err != nil {
t.Fatal(err)
}
if tfCmd.expectedCmdOutput != "" {
_, err := exp.ExpectString(tfCmd.expectedCmdOutput)
if err != nil {
t.Fatal(err)
}
}
lenInput := len(tfCmd.userInput)
lenInputOutput := len(tfCmd.postInputOutput)
if lenInput > 0 {
for i := 0; i < lenInput; i++ {
input := tfCmd.userInput[i]
exp.SendLine(input)
// use the index to find the corresponding
// output that matches the input.
if lenInputOutput-1 >= i {
output := tfCmd.postInputOutput[i]
_, err := exp.ExpectString(output)
if err != nil {
t.Fatal(err)
}
}
}
}
err = cmd.Wait()
if err != nil {
t.Fatal(err)
}
}
}
if tc.validations != nil {
tc.validations(t, organization.Name)
}
}
}

204
internal/cloud/e2e/migrate_state_single_to_tfc_test.go Normal file
View File

@ -0,0 +1,204 @@
//go:build e2e
// +build e2e
package main
import (
"context"
"io/ioutil"
"os"
"testing"
expect "github.com/Netflix/go-expect"
tfe "github.com/hashicorp/go-tfe"
"github.com/hashicorp/terraform/internal/e2e"
)
func Test_migrate_single_to_tfc(t *testing.T) {
skipWithoutRemoteTerraformVersion(t)
ctx := context.Background()
cases := map[string]struct {
operations []operationSets
validations func(t *testing.T, orgName string)
}{
"migrate using cloud workspace name strategy": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
tfBlock := terraformConfigLocalBackend()
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Successfully configured the backend "local"!`,
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
},
},
{
prep: func(t *testing.T, orgName, dir string) {
wsName := "new-workspace"
tfBlock := terraformConfigCloudBackendName(orgName, wsName)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init", "-migrate-state"},
expectedCmdOutput: `Do you want to copy existing state to Terraform Cloud?`,
userInput: []string{"yes"},
postInputOutput: []string{`Terraform Cloud has been successfully initialized!`},
},
{
command: []string{"workspace", "list"},
expectedCmdOutput: `new-workspace`,
},
},
},
},
validations: func(t *testing.T, orgName string) {
wsList, err := tfeClient.Workspaces.List(ctx, orgName, tfe.WorkspaceListOptions{})
if err != nil {
t.Fatal(err)
}
ws := wsList.Items[0]
if ws.Name != "new-workspace" {
t.Fatalf("Expected workspace to be `new-workspace`, but is %s", ws.Name)
}
},
},
"migrate using cloud workspace tags strategy": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
tfBlock := terraformConfigLocalBackend()
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Successfully configured the backend "local"!`,
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
},
},
{
prep: func(t *testing.T, orgName, dir string) {
tag := "app"
tfBlock := terraformConfigCloudBackendTags(orgName, tag)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init", "-migrate-state"},
expectedCmdOutput: `Terraform Cloud requires all workspaces to be given an explicit name.`,
userInput: []string{"new-workspace", "yes"},
postInputOutput: []string{
`Do you want to copy existing state to Terraform Cloud?`,
`Terraform Cloud has been successfully initialized!`},
},
{
command: []string{"workspace", "list"},
expectedCmdOutput: `new-workspace`,
},
},
},
},
validations: func(t *testing.T, orgName string) {
wsList, err := tfeClient.Workspaces.List(ctx, orgName, tfe.WorkspaceListOptions{
Tags: tfe.String("app"),
})
if err != nil {
t.Fatal(err)
}
ws := wsList.Items[0]
if ws.Name != "new-workspace" {
t.Fatalf("Expected workspace to be `new-workspace`, but is %s", ws.Name)
}
},
},
}
for name, tc := range cases {
t.Log("Test: ", name)
organization, cleanup := createOrganization(t)
defer cleanup()
exp, err := expect.NewConsole(expect.WithStdout(os.Stdout), expect.WithDefaultTimeout(expectConsoleTimeout))
if err != nil {
t.Fatal(err)
}
defer exp.Close()
tmpDir, err := ioutil.TempDir("", "terraform-test")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmpDir)
tf := e2e.NewBinary(terraformBin, tmpDir)
tf.AddEnv("TF_LOG=info")
tf.AddEnv(cliConfigFileEnv)
defer tf.Close()
for _, op := range tc.operations {
op.prep(t, organization.Name, tf.WorkDir())
for _, tfCmd := range op.commands {
cmd := tf.Cmd(tfCmd.command...)
cmd.Stdin = exp.Tty()
cmd.Stdout = exp.Tty()
cmd.Stderr = exp.Tty()
err = cmd.Start()
if err != nil {
t.Fatal(err)
}
if tfCmd.expectedCmdOutput != "" {
_, err := exp.ExpectString(tfCmd.expectedCmdOutput)
if err != nil {
t.Fatal(err)
}
}
lenInput := len(tfCmd.userInput)
lenInputOutput := len(tfCmd.postInputOutput)
if lenInput > 0 {
for i := 0; i < lenInput; i++ {
input := tfCmd.userInput[i]
exp.SendLine(input)
// use the index to find the corresponding
// output that matches the input.
if lenInputOutput-1 >= i {
output := tfCmd.postInputOutput[i]
_, err := exp.ExpectString(output)
if err != nil {
t.Fatal(err)
}
}
}
}
err = cmd.Wait()
if err != nil {
t.Fatal(err)
}
}
}
if tc.validations != nil {
tc.validations(t, organization.Name)
}
}
}

117
internal/cloud/e2e/migrate_state_tfc_to_other_test.go Normal file
View File

@ -0,0 +1,117 @@
//go:build e2e
// +build e2e
package main
import (
"fmt"
"io/ioutil"
"os"
"testing"
expect "github.com/Netflix/go-expect"
"github.com/hashicorp/terraform/internal/e2e"
)
func Test_migrate_tfc_to_other(t *testing.T) {
cases := map[string]struct {
operations []operationSets
}{
"migrate from cloud to local backend": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
wsName := "new-workspace"
tfBlock := terraformConfigCloudBackendName(orgName, wsName)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Terraform Cloud has been successfully initialized!`,
},
},
},
{
prep: func(t *testing.T, orgName, dir string) {
tfBlock := terraformConfigLocalBackend()
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init", "-migrate-state"},
expectedCmdOutput: `Migrating state from Terraform Cloud to another backend is not yet implemented.`,
expectError: true,
},
},
},
},
},
}
for name, tc := range cases {
fmt.Println("Test: ", name)
organization, cleanup := createOrganization(t)
defer cleanup()
exp, err := expect.NewConsole(expect.WithStdout(os.Stdout), expect.WithDefaultTimeout(expectConsoleTimeout))
if err != nil {
t.Fatal(err)
}
defer exp.Close()
tmpDir, err := ioutil.TempDir("", "terraform-test")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmpDir)
tf := e2e.NewBinary(terraformBin, tmpDir)
tf.AddEnv("TF_LOG=info")
tf.AddEnv(cliConfigFileEnv)
defer tf.Close()
for _, op := range tc.operations {
op.prep(t, organization.Name, tf.WorkDir())
for _, tfCmd := range op.commands {
cmd := tf.Cmd(tfCmd.command...)
cmd.Stdin = exp.Tty()
cmd.Stdout = exp.Tty()
cmd.Stderr = exp.Tty()
err = cmd.Start()
if err != nil {
t.Fatal(err)
}
if tfCmd.expectedCmdOutput != "" {
_, err := exp.ExpectString(tfCmd.expectedCmdOutput)
if err != nil {
t.Fatal(err)
}
}
lenInput := len(tfCmd.userInput)
lenInputOutput := len(tfCmd.postInputOutput)
if lenInput > 0 {
for i := 0; i < lenInput; i++ {
input := tfCmd.userInput[i]
exp.SendLine(input)
// use the index to find the corresponding
// output that matches the input.
if lenInputOutput-1 >= i {
output := tfCmd.postInputOutput[i]
_, err := exp.ExpectString(output)
if err != nil {
t.Fatal(err)
}
}
}
}
err = cmd.Wait()
if err != nil && !tfCmd.expectError {
t.Fatal(err)
}
}
}
}
}

564
internal/cloud/e2e/migrate_state_tfc_to_tfc_test.go Normal file
View File

@ -0,0 +1,564 @@
//go:build e2e
// +build e2e
package main
import (
"context"
"io/ioutil"
"os"
"testing"
expect "github.com/Netflix/go-expect"
tfe "github.com/hashicorp/go-tfe"
"github.com/hashicorp/terraform/internal/e2e"
tfversion "github.com/hashicorp/terraform/version"
)
func Test_migrate_tfc_to_tfc_single_workspace(t *testing.T) {
skipWithoutRemoteTerraformVersion(t)
ctx := context.Background()
cases := map[string]struct {
setup func(t *testing.T) (string, func())
operations []operationSets
validations func(t *testing.T, orgName string)
}{
"migrating from name to name": {
setup: func(t *testing.T) (string, func()) {
organization, cleanup := createOrganization(t)
return organization.Name, cleanup
},
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
wsName := "prod"
// Creating the workspace here instead of it being created
// dynamically in the Cloud StateMgr because we want to ensure that
// the terraform version selected for the workspace matches the
// terraform version of this current branch.
_ = createWorkspace(t, orgName, tfe.WorkspaceCreateOptions{
Name: tfe.String("prod"),
TerraformVersion: tfe.String(tfversion.String()),
})
tfBlock := terraformConfigCloudBackendName(orgName, wsName)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Terraform Cloud has been successfully initialized!`,
},
{
command: []string{"workspace", "show"},
expectedCmdOutput: `prod`, // this comes from the `prep` function
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions in workspace "prod"?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
},
},
{
prep: func(t *testing.T, orgName, dir string) {
wsName := "dev"
_ = createWorkspace(t, orgName, tfe.WorkspaceCreateOptions{
Name: tfe.String(wsName),
TerraformVersion: tfe.String(tfversion.String()),
})
tfBlock := terraformConfigCloudBackendName(orgName, wsName)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init", "-migrate-state", "-ignore-remote-version"},
expectedCmdOutput: `Do you want to copy existing state to Terraform Cloud?`,
userInput: []string{"yes"},
postInputOutput: []string{`Terraform Cloud has been successfully initialized!`},
},
{
command: []string{"workspace", "show"},
expectedCmdOutput: `dev`, // this comes from the `prep` function
},
},
},
},
validations: func(t *testing.T, orgName string) {
wsList, err := tfeClient.Workspaces.List(ctx, orgName, tfe.WorkspaceListOptions{})
if err != nil {
t.Fatal(err)
}
// this workspace name is what exists in the cloud backend configuration block
if len(wsList.Items) != 2 {
t.Fatal("Expected number of workspaces to be 2")
}
},
},
"migrating from name to tags": {
setup: func(t *testing.T) (string, func()) {
organization, cleanup := createOrganization(t)
return organization.Name, cleanup
},
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
wsName := "prod"
_ = createWorkspace(t, orgName, tfe.WorkspaceCreateOptions{
Name: tfe.String("prod"),
TerraformVersion: tfe.String(tfversion.String()),
})
tfBlock := terraformConfigCloudBackendName(orgName, wsName)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Terraform Cloud has been successfully initialized!`,
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions in workspace "prod"?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
},
},
{
prep: func(t *testing.T, orgName, dir string) {
tag := "app"
tfBlock := terraformConfigCloudBackendTags(orgName, tag)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init", "-migrate-state", "-ignore-remote-version"},
expectedCmdOutput: `Terraform Cloud requires all workspaces to be given an explicit name.`,
userInput: []string{"new-workspace", "yes"},
postInputOutput: []string{
`Do you want to copy existing state to Terraform Cloud?`,
`Terraform Cloud has been successfully initialized!`},
},
{
command: []string{"workspace", "show"},
expectedCmdOutput: `new-workspace`, // this comes from the `prep` function
},
},
},
},
validations: func(t *testing.T, orgName string) {
wsList, err := tfeClient.Workspaces.List(ctx, orgName, tfe.WorkspaceListOptions{
Tags: tfe.String("app"),
})
if err != nil {
t.Fatal(err)
}
// this workspace name is what exists in the cloud backend configuration block
if len(wsList.Items) != 1 {
t.Fatal("Expected number of workspaces to be 1")
}
},
},
"migrating from name to tags without ignore-version flag": {
setup: func(t *testing.T) (string, func()) {
organization, cleanup := createOrganization(t)
return organization.Name, cleanup
},
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
wsName := "prod"
_ = createWorkspace(t, orgName, tfe.WorkspaceCreateOptions{
Name: tfe.String("prod"),
TerraformVersion: tfe.String(tfversion.String()),
})
tfBlock := terraformConfigCloudBackendName(orgName, wsName)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Terraform Cloud has been successfully initialized!`,
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions in workspace "prod"?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
},
},
{
prep: func(t *testing.T, orgName, dir string) {
tag := "app"
// This is only here to ensure that the updated terraform version is
// present in the workspace, and it does not default to a lower
// version that does not support `cloud`.
_ = createWorkspace(t, orgName, tfe.WorkspaceCreateOptions{
Name: tfe.String("new-workspace"),
TerraformVersion: tfe.String(tfversion.String()),
})
tfBlock := terraformConfigCloudBackendTags(orgName, tag)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init", "-migrate-state"},
expectedCmdOutput: `Terraform Cloud requires all workspaces to be given an explicit name.`,
expectError: true,
userInput: []string{"new-workspace", "yes"},
},
},
},
},
validations: func(t *testing.T, orgName string) {
wsList, err := tfeClient.Workspaces.List(ctx, orgName, tfe.WorkspaceListOptions{
Tags: tfe.String("app"),
})
if err != nil {
t.Fatal(err)
}
// The migration never occured, so we have no workspaces with this tag.
if len(wsList.Items) != 0 {
t.Fatalf("Expected number of workspaces to be 0, but got %d", len(wsList.Items))
}
},
},
}
for name, tc := range cases {
t.Log("Test: ", name)
exp, err := expect.NewConsole(expect.WithStdout(os.Stdout), expect.WithDefaultTimeout(expectConsoleTimeout))
if err != nil {
t.Fatal(err)
}
defer exp.Close()
tmpDir, err := ioutil.TempDir("", "terraform-test")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmpDir)
tf := e2e.NewBinary(terraformBin, tmpDir)
defer tf.Close()
tf.AddEnv("TF_LOG=INFO")
tf.AddEnv(cliConfigFileEnv)
orgName, cleanup := tc.setup(t)
defer cleanup()
for _, op := range tc.operations {
op.prep(t, orgName, tf.WorkDir())
for _, tfCmd := range op.commands {
t.Log("Running commands: ", tfCmd.command)
cmd := tf.Cmd(tfCmd.command...)
cmd.Stdin = exp.Tty()
cmd.Stdout = exp.Tty()
cmd.Stderr = exp.Tty()
err = cmd.Start()
if err != nil {
t.Fatal(err)
}
if tfCmd.expectedCmdOutput != "" {
_, err := exp.ExpectString(tfCmd.expectedCmdOutput)
if err != nil {
t.Fatal(err)
}
}
lenInput := len(tfCmd.userInput)
lenInputOutput := len(tfCmd.postInputOutput)
if lenInput > 0 {
for i := 0; i < lenInput; i++ {
input := tfCmd.userInput[i]
exp.SendLine(input)
// use the index to find the corresponding
// output that matches the input.
if lenInputOutput-1 >= i {
output := tfCmd.postInputOutput[i]
_, err := exp.ExpectString(output)
if err != nil {
t.Fatal(err)
}
}
}
}
err = cmd.Wait()
if err != nil && !tfCmd.expectError {
t.Fatal(err.Error())
}
}
}
if tc.validations != nil {
tc.validations(t, orgName)
}
}
}
func Test_migrate_tfc_to_tfc_multiple_workspace(t *testing.T) {
skipWithoutRemoteTerraformVersion(t)
ctx := context.Background()
cases := map[string]struct {
setup func(t *testing.T) (string, func())
operations []operationSets
validations func(t *testing.T, orgName string)
}{
"migrating from multiple workspaces via tags to name": {
setup: func(t *testing.T) (string, func()) {
organization, cleanup := createOrganization(t)
return organization.Name, cleanup
},
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
tag := "app"
_ = createWorkspace(t, orgName, tfe.WorkspaceCreateOptions{
Name: tfe.String("app-prod"),
Tags: []*tfe.Tag{{Name: tag}},
TerraformVersion: tfe.String(tfversion.String()),
})
_ = createWorkspace(t, orgName, tfe.WorkspaceCreateOptions{
Name: tfe.String("app-staging"),
Tags: []*tfe.Tag{{Name: tag}},
TerraformVersion: tfe.String(tfversion.String()),
})
tfBlock := terraformConfigCloudBackendTags(orgName, tag)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `The currently selected workspace (default) does not exist.`,
userInput: []string{"1"},
postInputOutput: []string{`Terraform Cloud has been successfully initialized!`},
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions in workspace "app-prod"?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
{
command: []string{"workspace", "select", "app-staging"},
expectedCmdOutput: `Switched to workspace "app-staging".`,
},
{
command: []string{"apply"},
expectedCmdOutput: `Do you want to perform these actions in workspace "app-staging"?`,
userInput: []string{"yes"},
postInputOutput: []string{`Apply complete!`},
},
{
command: []string{"output"},
expectedCmdOutput: `tag_val = "app"`,
},
},
},
{
prep: func(t *testing.T, orgName, dir string) {
name := "service"
// Doing this here instead of relying on dynamic workspace creation
// because we want to set the terraform version here so that it is
// using the right version for post init operations.
_ = createWorkspace(t, orgName, tfe.WorkspaceCreateOptions{
Name: tfe.String(name),
TerraformVersion: tfe.String(tfversion.String()),
})
tfBlock := terraformConfigCloudBackendName(orgName, name)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init", "-migrate-state", "-ignore-remote-version"},
expectedCmdOutput: `Do you want to copy only your current workspace?`,
userInput: []string{"yes", "yes"},
postInputOutput: []string{
`Do you want to copy existing state to Terraform Cloud?`,
`Terraform Cloud has been successfully initialized!`},
},
{
command: []string{"workspace", "show"},
expectedCmdOutput: `service`, // this comes from the `prep` function
},
{
command: []string{"output"},
expectedCmdOutput: `tag_val = "app"`,
},
},
},
},
validations: func(t *testing.T, orgName string) {
ws, err := tfeClient.Workspaces.Read(ctx, orgName, "service")
if err != nil {
t.Fatal(err)
}
if ws == nil {
t.Fatal("Expected to workspace not be empty")
}
},
},
"migrating from multiple workspaces via tags to other tags": {
setup: func(t *testing.T) (string, func()) {
organization, cleanup := createOrganization(t)
return organization.Name, cleanup
},
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
tag := "app"
_ = createWorkspace(t, orgName, tfe.WorkspaceCreateOptions{
Name: tfe.String("app-prod"),
Tags: []*tfe.Tag{{Name: tag}},
TerraformVersion: tfe.String(tfversion.String()),
})
_ = createWorkspace(t, orgName, tfe.WorkspaceCreateOptions{
Name: tfe.String("app-staging"),
Tags: []*tfe.Tag{{Name: tag}},
TerraformVersion: tfe.String(tfversion.String()),
})
tfBlock := terraformConfigCloudBackendTags(orgName, tag)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `The currently selected workspace (default) does not exist.`,
userInput: []string{"1"},
postInputOutput: []string{`Terraform Cloud has been successfully initialized!`},
},
{
command: []string{"apply", "-auto-approve"},
expectedCmdOutput: `Apply complete!`,
},
{
command: []string{"workspace", "select", "app-staging"},
expectedCmdOutput: `Switched to workspace "app-staging".`,
},
{
command: []string{"apply", "-auto-approve"},
expectedCmdOutput: `Apply complete!`,
},
},
},
{
prep: func(t *testing.T, orgName, dir string) {
tag := "billing"
tfBlock := terraformConfigCloudBackendTags(orgName, tag)
writeMainTF(t, tfBlock, dir)
t.Log(orgName)
},
commands: []tfCommand{
{
command: []string{"init", "-migrate-state", "-ignore-remote-version"},
expectedCmdOutput: `Would you like to rename your workspaces?`,
userInput: []string{"1", "new-*", "1"},
postInputOutput: []string{
`What pattern would you like to add to all your workspaces?`,
`The currently selected workspace (app-staging) does not exist.`,
`Terraform Cloud has been successfully initialized!`},
},
},
},
},
validations: func(t *testing.T, orgName string) {
wsList, err := tfeClient.Workspaces.List(ctx, orgName, tfe.WorkspaceListOptions{
Tags: tfe.String("billing"),
})
if err != nil {
t.Fatal(err)
}
if len(wsList.Items) != 2 {
t.Logf("Expected the number of workspaces to be 2, but got %d", len(wsList.Items))
}
_, empty := getWorkspace(wsList.Items, "new-app-prod")
if empty {
t.Fatalf("expected workspaces to include 'new-app-prod' but didn't.")
}
_, empty = getWorkspace(wsList.Items, "new-app-staging")
if empty {
t.Fatalf("expected workspaces to include 'new-app-staging' but didn't.")
}
},
},
}
for name, tc := range cases {
t.Log("Test: ", name)
exp, err := expect.NewConsole(expect.WithStdout(os.Stdout), expect.WithDefaultTimeout(expectConsoleTimeout))
if err != nil {
t.Fatal(err)
}
defer exp.Close()
tmpDir, err := ioutil.TempDir("", "terraform-test")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmpDir)
tf := e2e.NewBinary(terraformBin, tmpDir)
defer tf.Close()
tf.AddEnv("TF_LOG=INFO")
tf.AddEnv(cliConfigFileEnv)
orgName, cleanup := tc.setup(t)
defer cleanup()
for _, op := range tc.operations {
op.prep(t, orgName, tf.WorkDir())
for _, tfCmd := range op.commands {
t.Log("Running commands: ", tfCmd.command)
cmd := tf.Cmd(tfCmd.command...)
cmd.Stdin = exp.Tty()
cmd.Stdout = exp.Tty()
cmd.Stderr = exp.Tty()
err = cmd.Start()
if err != nil {
t.Fatal(err)
}
if tfCmd.expectedCmdOutput != "" {
_, err := exp.ExpectString(tfCmd.expectedCmdOutput)
if err != nil {
t.Fatal(err)
}
}
lenInput := len(tfCmd.userInput)
lenInputOutput := len(tfCmd.postInputOutput)
if lenInput > 0 {
for i := 0; i < lenInput; i++ {
input := tfCmd.userInput[i]
exp.SendLine(input)
// use the index to find the corresponding
// output that matches the input.
if lenInputOutput-1 >= i {
output := tfCmd.postInputOutput[i]
_, err := exp.ExpectString(output)
if err != nil {
t.Fatal(err)
}
}
}
}
t.Log(cmd.Stderr)
err = cmd.Wait()
if err != nil {
t.Fatal(err.Error())
}
}
}
if tc.validations != nil {
tc.validations(t, orgName)
}
}
}

151
internal/cloud/e2e/run_variables_test.go Normal file
View File

@ -0,0 +1,151 @@
//go:build e2e
// +build e2e
package main
import (
"fmt"
"io/ioutil"
"os"
"testing"
expect "github.com/Netflix/go-expect"
"github.com/hashicorp/terraform/internal/e2e"
)
func terraformConfigRequiredVariable(org, name string) string {
return fmt.Sprintf(`
terraform {
cloud {
hostname = "%s"
organization = "%s"
workspaces {
name = "%s"
}
}
}
variable "foo" {
type = string
}
variable "baz" {
type = string
}
output "test_cli" {
value = var.foo
}
output "test_env" {
value = var.baz
}
`, tfeHostname, org, name)
}
func Test_cloud_run_variables(t *testing.T) {
skipWithoutRemoteTerraformVersion(t)
cases := testCases{
"run variables from CLI arg": {
operations: []operationSets{
{
prep: func(t *testing.T, orgName, dir string) {
wsName := "new-workspace"
tfBlock := terraformConfigRequiredVariable(orgName, wsName)
writeMainTF(t, tfBlock, dir)
},
commands: []tfCommand{
{
command: []string{"init"},
expectedCmdOutput: `Terraform Cloud has been successfully initialized!`,
},
{
command: []string{"plan", "-var", "foo=bar"},
expectedCmdOutput: ` + test_cli = "bar"`,
},
{
command: []string{"plan", "-var", "foo=bar"},
expectedCmdOutput: ` + test_env = "qux"`,
},
},
},
},
},
}
for name, tc := range cases {
fmt.Println("Test: ", name)
organization, cleanup := createOrganization(t)
defer cleanup()
exp, err := expect.NewConsole(expect.WithStdout(os.Stdout), expect.WithDefaultTimeout(expectConsoleTimeout))
if err != nil {
t.Fatal(err)
}
defer exp.Close()
tmpDir, err := ioutil.TempDir("", "terraform-test")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmpDir)
tf := e2e.NewBinary(terraformBin, tmpDir)
tf.AddEnv("TF_LOG=info")
tf.AddEnv("TF_CLI_ARGS=-no-color")
tf.AddEnv("TF_VAR_baz=qux")
tf.AddEnv(cliConfigFileEnv)
defer tf.Close()
for _, op := range tc.operations {
op.prep(t, organization.Name, tf.WorkDir())
for _, tfCmd := range op.commands {
cmd := tf.Cmd(tfCmd.command...)
cmd.Stdin = exp.Tty()
cmd.Stdout = exp.Tty()
cmd.Stderr = exp.Tty()
err = cmd.Start()
if err != nil {
t.Fatal(err)
}
if tfCmd.expectedCmdOutput != "" {
_, err := exp.ExpectString(tfCmd.expectedCmdOutput)
if err != nil {
t.Fatal(err)
}
}
lenInput := len(tfCmd.userInput)
lenInputOutput := len(tfCmd.postInputOutput)
if lenInput > 0 {
for i := 0; i < lenInput; i++ {
input := tfCmd.userInput[i]
exp.SendLine(input)
// use the index to find the corresponding
// output that matches the input.
if lenInputOutput-1 >= i {
output := tfCmd.postInputOutput[i]
_, err := exp.ExpectString(output)
if err != nil {
t.Fatal(err)
}
}
}
}
err = cmd.Wait()
if err != nil && !tfCmd.expectError {
t.Fatal(err)
}
}
if tc.validations != nil {
tc.validations(t, organization.Name)
}
}
}
}

45
internal/cloud/errors.go Normal file
View File

@ -0,0 +1,45 @@
package cloud
import (
"fmt"
"strings"
"github.com/hashicorp/terraform/internal/tfdiags"
"github.com/zclconf/go-cty/cty"
)
var (
invalidOrganizationConfigMissingValue = tfdiags.AttributeValue(
tfdiags.Error,
"Invalid organization value",
`The "organization" attribute value must not be empty.\n\n%s`,
cty.Path{cty.GetAttrStep{Name: "organization"}},
)
invalidWorkspaceConfigMissingValues = tfdiags.AttributeValue(
tfdiags.Error,
"Invalid workspaces configuration",
fmt.Sprintf("Missing workspace mapping strategy. Either workspace \"tags\" or \"name\" is required.\n\n%s", workspaceConfigurationHelp),
cty.Path{cty.GetAttrStep{Name: "workspaces"}},
)
invalidWorkspaceConfigMisconfiguration = tfdiags.AttributeValue(
tfdiags.Error,
"Invalid workspaces configuration",
fmt.Sprintf("Only one of workspace \"tags\" or \"name\" is allowed.\n\n%s", workspaceConfigurationHelp),
cty.Path{cty.GetAttrStep{Name: "workspaces"}},
)
)
const ignoreRemoteVersionHelp = "If you're sure you want to upgrade the state, you can force Terraform to continue using the -ignore-remote-version flag. This may result in an unusable workspace."
func incompatibleWorkspaceTerraformVersion(message string, ignoreVersionConflict bool) tfdiags.Diagnostic {
severity := tfdiags.Error
suggestion := ignoreRemoteVersionHelp
if ignoreVersionConflict {
severity = tfdiags.Warning
suggestion = ""
}
description := strings.TrimSpace(fmt.Sprintf("%s\n\n%s", message, suggestion))
return tfdiags.Sourceless(severity, "Incompatible Terraform version", description)
}

25
internal/cloud/remote_test.go Normal file
View File

@ -0,0 +1,25 @@
package cloud
import (
"flag"
"os"
"testing"
"time"
_ "github.com/hashicorp/terraform/internal/logging"
)
func TestMain(m *testing.M) {
flag.Parse()
// Make sure TF_FORCE_LOCAL_BACKEND is unset
os.Unsetenv("TF_FORCE_LOCAL_BACKEND")
// Reduce delays to make tests run faster
backoffMin = 1.0
backoffMax = 1.0
planConfigurationVersionsPollInterval = 1 * time.Millisecond
runPollInterval = 1 * time.Millisecond
os.Exit(m.Run())
}

1
internal/cloud/testdata/.gitignore vendored Normal file
View File

@ -0,0 +1 @@
!*.log

7
internal/cloud/testdata/apply-destroy/apply.log vendored Normal file
View File

@ -0,0 +1,7 @@
Terraform v0.11.10
Initializing plugins and modules...
null_resource.hello: Destroying... (ID: 8657651096157629581)
null_resource.hello: Destruction complete after 0s
Apply complete! Resources: 0 added, 0 changed, 1 destroyed.

1
internal/cloud/testdata/apply-destroy/main.tf vendored Normal file
View File

@ -0,0 +1 @@
resource "null_resource" "foo" {}

22
internal/cloud/testdata/apply-destroy/plan.log vendored Normal file
View File

@ -0,0 +1,22 @@
Terraform v0.11.7
Configuring remote state backend...
Initializing Terraform configuration...
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
null_resource.hello: Refreshing state... (ID: 8657651096157629581)
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
- destroy
Terraform will perform the following actions:
- null_resource.hello
Plan: 0 to add, 0 to change, 1 to destroy.

1
internal/cloud/testdata/apply-no-changes/main.tf vendored Normal file
View File

@ -0,0 +1 @@
resource "null_resource" "foo" {}

17
internal/cloud/testdata/apply-no-changes/plan.log vendored Normal file
View File

@ -0,0 +1,17 @@
Terraform v0.11.7
Configuring remote state backend...
Initializing Terraform configuration...
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
null_resource.hello: Refreshing state... (ID: 8657651096157629581)
------------------------------------------------------------------------
No changes. Infrastructure is up-to-date.
This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.

12
internal/cloud/testdata/apply-no-changes/policy.log vendored Normal file
View File

@ -0,0 +1,12 @@
Sentinel Result: true
This result means that Sentinel policies returned true and the protected
behavior is allowed by Sentinel policies.
1 policies evaluated.
## Policy 1: Passthrough.sentinel (soft-mandatory)
Result: true
TRUE - Passthrough.sentinel:1:1 - Rule "main"

1
internal/cloud/testdata/apply-policy-hard-failed/main.tf vendored Normal file
View File

@ -0,0 +1 @@
resource "null_resource" "foo" {}

21
internal/cloud/testdata/apply-policy-hard-failed/plan.log vendored Normal file
View File

@ -0,0 +1,21 @@
Terraform v0.11.7
Configuring remote state backend...
Initializing Terraform configuration...
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ null_resource.foo
id: <computed>
Plan: 1 to add, 0 to change, 0 to destroy.

12
internal/cloud/testdata/apply-policy-hard-failed/policy.log vendored Normal file
View File

@ -0,0 +1,12 @@
Sentinel Result: false
Sentinel evaluated to false because one or more Sentinel policies evaluated
to false. This false was not due to an undefined value or runtime error.
1 policies evaluated.
## Policy 1: Passthrough.sentinel (hard-mandatory)
Result: false
FALSE - Passthrough.sentinel:1:1 - Rule "main"

7
internal/cloud/testdata/apply-policy-passed/apply.log vendored Normal file
View File

@ -0,0 +1,7 @@
Terraform v0.11.10
Initializing plugins and modules...
null_resource.hello: Creating...
null_resource.hello: Creation complete after 0s (ID: 8657651096157629581)
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

1
internal/cloud/testdata/apply-policy-passed/main.tf vendored Normal file
View File

@ -0,0 +1 @@
resource "null_resource" "foo" {}

21
internal/cloud/testdata/apply-policy-passed/plan.log vendored Normal file
View File

@ -0,0 +1,21 @@
Terraform v0.11.7
Configuring remote state backend...
Initializing Terraform configuration...
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ null_resource.foo
id: <computed>
Plan: 1 to add, 0 to change, 0 to destroy.

12
internal/cloud/testdata/apply-policy-passed/policy.log vendored Normal file
View File

@ -0,0 +1,12 @@
Sentinel Result: true
This result means that Sentinel policies returned true and the protected
behavior is allowed by Sentinel policies.
1 policies evaluated.
## Policy 1: Passthrough.sentinel (soft-mandatory)
Result: true
TRUE - Passthrough.sentinel:1:1 - Rule "main"

7
internal/cloud/testdata/apply-policy-soft-failed/apply.log vendored Normal file
View File

@ -0,0 +1,7 @@
Terraform v0.11.10
Initializing plugins and modules...
null_resource.hello: Creating...
null_resource.hello: Creation complete after 0s (ID: 8657651096157629581)
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

1
internal/cloud/testdata/apply-policy-soft-failed/main.tf vendored Normal file
View File

@ -0,0 +1 @@
resource "null_resource" "foo" {}

21
internal/cloud/testdata/apply-policy-soft-failed/plan.log vendored Normal file
View File

@ -0,0 +1,21 @@
Terraform v0.11.7
Configuring remote state backend...
Initializing Terraform configuration...
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ null_resource.foo
id: <computed>
Plan: 1 to add, 0 to change, 0 to destroy.

12
internal/cloud/testdata/apply-policy-soft-failed/policy.log vendored Normal file
View File

@ -0,0 +1,12 @@
Sentinel Result: false
Sentinel evaluated to false because one or more Sentinel policies evaluated
to false. This false was not due to an undefined value or runtime error.
1 policies evaluated.
## Policy 1: Passthrough.sentinel (soft-mandatory)
Result: false
FALSE - Passthrough.sentinel:1:1 - Rule "main"

7
internal/cloud/testdata/apply-variables/apply.log vendored Normal file
View File

@ -0,0 +1,7 @@
Terraform v0.11.10
Initializing plugins and modules...
null_resource.hello: Creating...
null_resource.hello: Creation complete after 0s (ID: 8657651096157629581)
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

4
internal/cloud/testdata/apply-variables/main.tf vendored Normal file
View File

@ -0,0 +1,4 @@
variable "foo" {}
variable "bar" {}
resource "null_resource" "foo" {}

21
internal/cloud/testdata/apply-variables/plan.log vendored Normal file
View File

@ -0,0 +1,21 @@
Terraform v0.11.7
Configuring remote state backend...
Initializing Terraform configuration...
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ null_resource.foo
id: <computed>
Plan: 1 to add, 0 to change, 0 to destroy.

5
internal/cloud/testdata/apply-with-error/main.tf vendored Normal file
View File

@ -0,0 +1,5 @@
resource "null_resource" "foo" {
triggers {
random = "${guid()}"
}
}

10
internal/cloud/testdata/apply-with-error/plan.log vendored Normal file
View File

@ -0,0 +1,10 @@
Terraform v0.11.7
Configuring remote state backend...
Initializing Terraform configuration...
Error: null_resource.foo: 1 error(s) occurred:
* null_resource.foo: 1:3: unknown function called: guid in:
${guid()}

7
internal/cloud/testdata/apply/apply.log vendored Normal file
View File

@ -0,0 +1,7 @@
Terraform v0.11.10
Initializing plugins and modules...
null_resource.hello: Creating...
null_resource.hello: Creation complete after 0s (ID: 8657651096157629581)
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

1
internal/cloud/testdata/apply/main.tf vendored Normal file
View File

@ -0,0 +1 @@
resource "null_resource" "foo" {}

21
internal/cloud/testdata/apply/plan.log vendored Normal file
View File

@ -0,0 +1,21 @@
Terraform v0.11.7
Configuring remote state backend...
Initializing Terraform configuration...
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ null_resource.foo
id: <computed>
Plan: 1 to add, 0 to change, 0 to destroy.

0
internal/cloud/testdata/empty/.gitignore vendored Normal file
View File

6
internal/cloud/testdata/plan-cost-estimation/ce.log vendored Normal file
View File

@ -0,0 +1,6 @@
+---------+------+-----+-------------+----------------------+
| PRODUCT | NAME | SKU | DESCRIPTION | DELTA |
+---------+------+-----+-------------+----------------------+
+---------+------+-----+-------------+----------------------+
| TOTAL | $0.000 USD / 720 HRS |
+---------+------+-----+-------------+----------------------+

5
internal/cloud/testdata/plan-cost-estimation/cost-estimate.log vendored Normal file
View File

@ -0,0 +1,5 @@
Cost estimation:
Waiting for cost estimation to complete...
Resources: 1 of 1 estimated
$25.488/mo +$25.488

1
internal/cloud/testdata/plan-cost-estimation/main.tf vendored Normal file
View File

@ -0,0 +1 @@
resource "null_resource" "foo" {}

20
internal/cloud/testdata/plan-cost-estimation/plan.log vendored Normal file
View File

@ -0,0 +1,20 @@
Terraform v0.12.9
Configuring remote state backend...
Initializing Terraform configuration...
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ null_resource.foo
id: <computed>
Plan: 1 to add, 0 to change, 0 to destroy.

5
internal/cloud/testdata/plan-long-line/main.tf vendored Normal file
View File

File diff suppressed because one or more lines are too long

23
internal/cloud/testdata/plan-long-line/plan.log vendored Normal file
View File

File diff suppressed because one or more lines are too long

1
internal/cloud/testdata/plan-no-changes/main.tf vendored Normal file
View File

@ -0,0 +1 @@
resource "null_resource" "foo" {}

17
internal/cloud/testdata/plan-no-changes/plan.log vendored Normal file
View File

@ -0,0 +1,17 @@
Terraform v0.11.7
Configuring remote state backend...
Initializing Terraform configuration...
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
null_resource.hello: Refreshing state... (ID: 8657651096157629581)
------------------------------------------------------------------------
No changes. Infrastructure is up-to-date.
This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.

12
internal/cloud/testdata/plan-no-changes/policy.log vendored Normal file
View File

@ -0,0 +1,12 @@
Sentinel Result: true
This result means that Sentinel policies returned true and the protected
behavior is allowed by Sentinel policies.
1 policies evaluated.
## Policy 1: Passthrough.sentinel (soft-mandatory)
Result: true
TRUE - Passthrough.sentinel:1:1 - Rule "main"

1
internal/cloud/testdata/plan-policy-hard-failed/main.tf vendored Normal file
View File

@ -0,0 +1 @@
resource "null_resource" "foo" {}

21
internal/cloud/testdata/plan-policy-hard-failed/plan.log vendored Normal file
View File

@ -0,0 +1,21 @@
Terraform v0.11.7
Configuring remote state backend...
Initializing Terraform configuration...
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ null_resource.foo
id: <computed>
Plan: 1 to add, 0 to change, 0 to destroy.

12
internal/cloud/testdata/plan-policy-hard-failed/policy.log vendored Normal file
View File

@ -0,0 +1,12 @@
Sentinel Result: false
Sentinel evaluated to false because one or more Sentinel policies evaluated
to false. This false was not due to an undefined value or runtime error.
1 policies evaluated.
## Policy 1: Passthrough.sentinel (hard-mandatory)
Result: false
FALSE - Passthrough.sentinel:1:1 - Rule "main"

1
internal/cloud/testdata/plan-policy-passed/main.tf vendored Normal file
View File

@ -0,0 +1 @@
resource "null_resource" "foo" {}

21
internal/cloud/testdata/plan-policy-passed/plan.log vendored Normal file
View File

@ -0,0 +1,21 @@
Terraform v0.11.7
Configuring remote state backend...
Initializing Terraform configuration...
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ null_resource.foo
id: <computed>
Plan: 1 to add, 0 to change, 0 to destroy.

12
internal/cloud/testdata/plan-policy-passed/policy.log vendored Normal file
View File

@ -0,0 +1,12 @@
Sentinel Result: true
This result means that Sentinel policies returned true and the protected
behavior is allowed by Sentinel policies.
1 policies evaluated.
## Policy 1: Passthrough.sentinel (soft-mandatory)
Result: true
TRUE - Passthrough.sentinel:1:1 - Rule "main"

1
internal/cloud/testdata/plan-policy-soft-failed/main.tf vendored Normal file
View File

@ -0,0 +1 @@
resource "null_resource" "foo" {}

21
internal/cloud/testdata/plan-policy-soft-failed/plan.log vendored Normal file
View File

@ -0,0 +1,21 @@
Terraform v0.11.7
Configuring remote state backend...
Initializing Terraform configuration...
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ null_resource.foo
id: <computed>
Plan: 1 to add, 0 to change, 0 to destroy.

12
internal/cloud/testdata/plan-policy-soft-failed/policy.log vendored Normal file
View File

@ -0,0 +1,12 @@
Sentinel Result: false
Sentinel evaluated to false because one or more Sentinel policies evaluated
to false. This false was not due to an undefined value or runtime error.
1 policies evaluated.
## Policy 1: Passthrough.sentinel (soft-mandatory)
Result: false
FALSE - Passthrough.sentinel:1:1 - Rule "main"

4
internal/cloud/testdata/plan-variables/main.tf vendored Normal file
View File

@ -0,0 +1,4 @@
variable "foo" {}
variable "bar" {}
resource "null_resource" "foo" {}

21
internal/cloud/testdata/plan-variables/plan.log vendored Normal file
View File

@ -0,0 +1,21 @@
Terraform v0.11.7
Configuring remote state backend...
Initializing Terraform configuration...
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ null_resource.foo
id: <computed>
Plan: 1 to add, 0 to change, 0 to destroy.

5
internal/cloud/testdata/plan-with-error/main.tf vendored Normal file
View File

@ -0,0 +1,5 @@
resource "null_resource" "foo" {
triggers {
random = "${guid()}"
}
}

10
internal/cloud/testdata/plan-with-error/plan.log vendored Normal file
View File

@ -0,0 +1,10 @@
Terraform v0.11.7
Configuring remote state backend...
Initializing Terraform configuration...
Error: null_resource.foo: 1 error(s) occurred:
* null_resource.foo: 1:3: unknown function called: guid in:
${guid()}

1
internal/cloud/testdata/plan-with-working-directory/terraform/main.tf vendored Normal file
View File

@ -0,0 +1 @@
resource "null_resource" "foo" {}

21
internal/cloud/testdata/plan-with-working-directory/terraform/plan.log vendored Normal file
View File

@ -0,0 +1,21 @@
Terraform v0.11.7
Configuring remote state backend...
Initializing Terraform configuration...
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ null_resource.foo
id: <computed>
Plan: 1 to add, 0 to change, 0 to destroy.

1
internal/cloud/testdata/plan/main.tf vendored Normal file
View File

@ -0,0 +1 @@
resource "null_resource" "foo" {}

21
internal/cloud/testdata/plan/plan.log vendored Normal file
View File

@ -0,0 +1,21 @@
Terraform v0.11.7
Configuring remote state backend...
Initializing Terraform configuration...
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ null_resource.foo
id: <computed>
Plan: 1 to add, 0 to change, 0 to destroy.

6
internal/cloud/testdata/refresh/main.tf vendored Normal file
View File

@ -0,0 +1,6 @@
resource "random_pet" "always_new" {
keepers = {
uuid = uuid() # Force a new name each time
}
length = 3
}

341
internal/cloud/testing.go Normal file
View File

@ -0,0 +1,341 @@
package cloud
import (
"context"
"fmt"
"io"
"net/http"
"net/http/httptest"
"path"
"testing"
"time"
tfe "github.com/hashicorp/go-tfe"
svchost "github.com/hashicorp/terraform-svchost"
"github.com/hashicorp/terraform-svchost/auth"
"github.com/hashicorp/terraform-svchost/disco"
"github.com/hashicorp/terraform/internal/backend"
"github.com/hashicorp/terraform/internal/configs"
"github.com/hashicorp/terraform/internal/configs/configschema"
"github.com/hashicorp/terraform/internal/httpclient"
"github.com/hashicorp/terraform/internal/providers"
"github.com/hashicorp/terraform/internal/states/remote"
"github.com/hashicorp/terraform/internal/terraform"
"github.com/hashicorp/terraform/internal/tfdiags"
"github.com/hashicorp/terraform/version"
"github.com/mitchellh/cli"
"github.com/zclconf/go-cty/cty"
backendLocal "github.com/hashicorp/terraform/internal/backend/local"
)
const (
testCred = "test-auth-token"
)
var (
tfeHost = svchost.Hostname(defaultHostname)
credsSrc = auth.StaticCredentialsSource(map[svchost.Hostname]map[string]interface{}{
tfeHost: {"token": testCred},
})
testBackendSingleWorkspaceName = "app-prod"
)
// mockInput is a mock implementation of terraform.UIInput.
type mockInput struct {
answers map[string]string
}
func (m *mockInput) Input(ctx context.Context, opts *terraform.InputOpts) (string, error) {
v, ok := m.answers[opts.Id]
if !ok {
return "", fmt.Errorf("unexpected input request in test: %s", opts.Id)
}
if v == "wait-for-external-update" {
select {
case <-ctx.Done():
case <-time.After(time.Minute):
}
}
delete(m.answers, opts.Id)
return v, nil
}
func testInput(t *testing.T, answers map[string]string) *mockInput {
return &mockInput{answers: answers}
}
func testBackendWithName(t *testing.T) (*Cloud, func()) {
obj := cty.ObjectVal(map[string]cty.Value{
"hostname": cty.NullVal(cty.String),
"organization": cty.StringVal("hashicorp"),
"token": cty.NullVal(cty.String),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.StringVal(testBackendSingleWorkspaceName),
"tags": cty.NullVal(cty.Set(cty.String)),
}),
})
return testBackend(t, obj)
}
func testBackendWithTags(t *testing.T) (*Cloud, func()) {
obj := cty.ObjectVal(map[string]cty.Value{
"hostname": cty.NullVal(cty.String),
"organization": cty.StringVal("hashicorp"),
"token": cty.NullVal(cty.String),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.NullVal(cty.String),
"tags": cty.SetVal(
[]cty.Value{
cty.StringVal("billing"),
},
),
}),
})
return testBackend(t, obj)
}
func testBackendNoOperations(t *testing.T) (*Cloud, func()) {
obj := cty.ObjectVal(map[string]cty.Value{
"hostname": cty.NullVal(cty.String),
"organization": cty.StringVal("no-operations"),
"token": cty.NullVal(cty.String),
"workspaces": cty.ObjectVal(map[string]cty.Value{
"name": cty.StringVal(testBackendSingleWorkspaceName),
"tags": cty.NullVal(cty.Set(cty.String)),
}),
})
return testBackend(t, obj)
}
func testRemoteClient(t *testing.T) remote.Client {
b, bCleanup := testBackendWithName(t)
defer bCleanup()
raw, err := b.StateMgr(testBackendSingleWorkspaceName)
if err != nil {
t.Fatalf("error: %v", err)
}
return raw.(*remote.State).Client
}
func testBackend(t *testing.T, obj cty.Value) (*Cloud, func()) {
s := testServer(t)
b := New(testDisco(s))
// Configure the backend so the client is created.
newObj, valDiags := b.PrepareConfig(obj)
if len(valDiags) != 0 {
t.Fatalf("testBackend: backend.PrepareConfig() failed: %s", valDiags.ErrWithWarnings())
}
obj = newObj
confDiags := b.Configure(obj)
if len(confDiags) != 0 {
t.Fatalf("testBackend: backend.Configure() failed: %s", confDiags.ErrWithWarnings())
}
// Get a new mock client.
mc := NewMockClient()
// Replace the services we use with our mock services.
b.CLI = cli.NewMockUi()
b.client.Applies = mc.Applies
b.client.ConfigurationVersions = mc.ConfigurationVersions
b.client.CostEstimates = mc.CostEstimates
b.client.Organizations = mc.Organizations
b.client.Plans = mc.Plans
b.client.PolicyChecks = mc.PolicyChecks
b.client.Runs = mc.Runs
b.client.StateVersions = mc.StateVersions
b.client.Variables = mc.Variables
b.client.Workspaces = mc.Workspaces
// Set local to a local test backend.
b.local = testLocalBackend(t, b)
ctx := context.Background()
// Create the organization.
_, err := b.client.Organizations.Create(ctx, tfe.OrganizationCreateOptions{
Name: tfe.String(b.organization),
})
if err != nil {
t.Fatalf("error: %v", err)
}
// Create the default workspace if required.
if b.WorkspaceMapping.Name != "" {
_, err = b.client.Workspaces.Create(ctx, b.organization, tfe.WorkspaceCreateOptions{
Name: tfe.String(b.WorkspaceMapping.Name),
})
if err != nil {
t.Fatalf("error: %v", err)
}
}
return b, s.Close
}
func testLocalBackend(t *testing.T, cloud *Cloud) backend.Enhanced {
b := backendLocal.NewWithBackend(cloud)
// Add a test provider to the local backend.
p := backendLocal.TestLocalProvider(t, b, "null", &terraform.ProviderSchema{
ResourceTypes: map[string]*configschema.Block{
"null_resource": {
Attributes: map[string]*configschema.Attribute{
"id": {Type: cty.String, Computed: true},
},
},
},
})
p.ApplyResourceChangeResponse = &providers.ApplyResourceChangeResponse{NewState: cty.ObjectVal(map[string]cty.Value{
"id": cty.StringVal("yes"),
})}
return b
}
// testServer returns a started *httptest.Server used for local testing with the default set of
// request handlers.
func testServer(t *testing.T) *httptest.Server {
return testServerWithHandlers(testDefaultRequestHandlers)
}
// testServerWithHandlers returns a started *httptest.Server with the given set of request handlers
// overriding any default request handlers (testDefaultRequestHandlers).
func testServerWithHandlers(handlers map[string]func(http.ResponseWriter, *http.Request)) *httptest.Server {
mux := http.NewServeMux()
for route, handler := range handlers {
mux.HandleFunc(route, handler)
}
for route, handler := range testDefaultRequestHandlers {
if handlers[route] == nil {
mux.HandleFunc(route, handler)
}
}
return httptest.NewServer(mux)
}
// testDefaultRequestHandlers is a map of request handlers intended to be used in a request
// multiplexer for a test server. A caller may use testServerWithHandlers to start a server with
// this base set of routes, and override a particular route for whatever edge case is being tested.
var testDefaultRequestHandlers = map[string]func(http.ResponseWriter, *http.Request){
// Respond to service discovery calls.
"/well-known/terraform.json": func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
io.WriteString(w, `{
"tfe.v2": "/api/v2/",
}`)
},
// Respond to service version constraints calls.
"/v1/versions/": func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
io.WriteString(w, fmt.Sprintf(`{
"service": "%s",
"product": "terraform",
"minimum": "0.1.0",
"maximum": "10.0.0"
}`, path.Base(r.URL.Path)))
},
// Respond to pings to get the API version header.
"/api/v2/ping": func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.Header().Set("TFP-API-Version", "2.5")
},
// Respond to the initial query to read the hashicorp org entitlements.
"/api/v2/organizations/hashicorp/entitlement-set": func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/vnd.api+json")
io.WriteString(w, `{
"data": {
"id": "org-GExadygjSbKP8hsY",
"type": "entitlement-sets",
"attributes": {
"operations": true,
"private-module-registry": true,
"sentinel": true,
"state-storage": true,
"teams": true,
"vcs-integrations": true
}
}
}`)
},
// Respond to the initial query to read the no-operations org entitlements.
"/api/v2/organizations/no-operations/entitlement-set": func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/vnd.api+json")
io.WriteString(w, `{
"data": {
"id": "org-ufxa3y8jSbKP8hsT",
"type": "entitlement-sets",
"attributes": {
"operations": false,
"private-module-registry": true,
"sentinel": true,
"state-storage": true,
"teams": true,
"vcs-integrations": true
}
}
}`)
},
// All tests that are assumed to pass will use the hashicorp organization,
// so for all other organization requests we will return a 404.
"/api/v2/organizations/": func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(404)
io.WriteString(w, `{
"errors": [
{
"status": "404",
"title": "not found"
}
]
}`)
},
}
// testDisco returns a *disco.Disco mapping app.terraform.io and
// localhost to a local test server.
func testDisco(s *httptest.Server) *disco.Disco {
services := map[string]interface{}{
"tfe.v2": fmt.Sprintf("%s/api/v2/", s.URL),
}
d := disco.NewWithCredentialsSource(credsSrc)
d.SetUserAgent(httpclient.TerraformUserAgent(version.String()))
d.ForceHostServices(svchost.Hostname(defaultHostname), services)
d.ForceHostServices(svchost.Hostname("localhost"), services)
return d
}
type unparsedVariableValue struct {
value string
source terraform.ValueSourceType
}
func (v *unparsedVariableValue) ParseVariableValue(mode configs.VariableParsingMode) (*terraform.InputValue, tfdiags.Diagnostics) {
return &terraform.InputValue{
Value: cty.StringVal(v.value),
SourceType: v.source,
}, tfdiags.Diagnostics{}
}
// testVariable returns a backend.UnparsedVariableValue used for testing.
func testVariables(s terraform.ValueSourceType, vs ...string) map[string]backend.UnparsedVariableValue {
vars := make(map[string]backend.UnparsedVariableValue, len(vs))
for _, v := range vs {
vars[v] = &unparsedVariableValue{
value: v,
source: s,
}
}
return vars
}

499
internal/backend/remote/backend_mock.go → internal/cloud/tfe_client_mock.go
View File

File diff suppressed because it is too large Load Diff

5
internal/command/apply.go
View File

@ -29,8 +29,9 @@ func (c *ApplyCommand) Run(rawArgs []string) int {
common, rawArgs := arguments.ParseView(rawArgs)
c.View.Configure(common)
// Propagate -no-color for the remote backend's legacy use of Ui. This
// should be removed when the remote backend is migrated to views.
// Propagate -no-color for legacy use of Ui. The remote backend and
// cloud package use this; it should be removed when/if they are
// migrated to views.
c.Meta.color = !common.NoColor
c.Meta.Color = c.Meta.color

2
internal/command/console.go
View File

@ -72,7 +72,7 @@ func (c *ConsoleCommand) Run(args []string) int {
}
// This is a read-only command
c.ignoreRemoteBackendVersionConflict(b)
c.ignoreRemoteVersionConflict(b)
// Build the operation
opReq := c.Operation(b)

2
internal/command/graph.go
View File

@ -88,7 +88,7 @@ func (c *GraphCommand) Run(args []string) int {
}
// This is a read-only command
c.ignoreRemoteBackendVersionConflict(b)
c.ignoreRemoteVersionConflict(b)
// Build the operation
opReq := c.Operation(b)

2
internal/command/import.go
View File

@ -204,7 +204,7 @@ func (c *ImportCommand) Run(args []string) int {
opReq.View = views.NewOperation(arguments.ViewHuman, c.RunningInAutomation, c.View)
// Check remote Terraform version is compatible
remoteVersionDiags := c.remoteBackendVersionCheck(b, opReq.Workspace)
remoteVersionDiags := c.remoteVersionCheck(b, opReq.Workspace)
diags = diags.Append(remoteVersionDiags)
c.showDiagnostics(diags)
if diags.HasErrors() {

71
internal/command/init.go
View File

@ -15,6 +15,7 @@ import (
"github.com/hashicorp/terraform/internal/addrs"
"github.com/hashicorp/terraform/internal/backend"
backendInit "github.com/hashicorp/terraform/internal/backend/init"
"github.com/hashicorp/terraform/internal/cloud"
"github.com/hashicorp/terraform/internal/configs"
"github.com/hashicorp/terraform/internal/configs/configschema"
"github.com/hashicorp/terraform/internal/getproviders"
@ -209,8 +210,20 @@ func (c *InitCommand) Run(args []string) int {
}
var back backend.Backend
if flagBackend {
switch {
case config.Module.CloudConfig != nil:
be, backendOutput, backendDiags := c.initCloud(config.Module)
diags = diags.Append(backendDiags)
if backendDiags.HasErrors() {
c.showDiagnostics(diags)
return 1
}
if backendOutput {
header = true
}
back = be
case flagBackend:
be, backendOutput, backendDiags := c.initBackend(config.Module, flagConfigExtra)
diags = diags.Append(backendDiags)
if backendDiags.HasErrors() {
@ -221,7 +234,7 @@ func (c *InitCommand) Run(args []string) int {
header = true
}
back = be
} else {
default:
// load the previously-stored backend config
be, backendDiags := c.Meta.backendFromState()
diags = diags.Append(backendDiags)
@ -251,7 +264,7 @@ func (c *InitCommand) Run(args []string) int {
// on a previous run) we'll use the current state as a potential source
// of provider dependencies.
if back != nil {
c.ignoreRemoteBackendVersionConflict(back)
c.ignoreRemoteVersionConflict(back)
workspace, err := c.Workspace()
if err != nil {
c.Ui.Error(fmt.Sprintf("Error selecting workspace: %s", err))
@ -292,12 +305,23 @@ func (c *InitCommand) Run(args []string) int {
// by errors then we'll output them here so that the success message is
// still the final thing shown.
c.showDiagnostics(diags)
c.Ui.Output(c.Colorize().Color(strings.TrimSpace(outputInitSuccess)))
_, cloud := back.(*cloud.Cloud)
output := outputInitSuccess
if cloud {
output = outputInitSuccessCloud
}
c.Ui.Output(c.Colorize().Color(strings.TrimSpace(output)))
if !c.RunningInAutomation {
// If we're not running in an automation wrapper, give the user
// some more detailed next steps that are appropriate for interactive
// shell usage.
c.Ui.Output(c.Colorize().Color(strings.TrimSpace(outputInitSuccessCLI)))
output = outputInitSuccessCLI
if cloud {
output = outputInitSuccessCLICloud
}
c.Ui.Output(c.Colorize().Color(strings.TrimSpace(output)))
}
return 0
}
@ -337,6 +361,21 @@ func (c *InitCommand) getModules(path string, earlyRoot *tfconfig.Module, upgrad
return true, diags
}
func (c *InitCommand) initCloud(root *configs.Module) (be backend.Backend, output bool, diags tfdiags.Diagnostics) {
c.Ui.Output(c.Colorize().Color("\n[reset][bold]Initializing Terraform Cloud..."))
backendConfig := root.CloudConfig.ToBackendConfig()
opts := &BackendOpts{
Config: &backendConfig,
Init: true,
}
back, backDiags := c.Backend(opts)
diags = diags.Append(backDiags)
return back, true, diags
}
func (c *InitCommand) initBackend(root *configs.Module, extraConfig rawFlags) (be backend.Backend, output bool, diags tfdiags.Diagnostics) {
c.Ui.Output(c.Colorize().Color("\n[reset][bold]Initializing the backend..."))
@ -344,6 +383,16 @@ func (c *InitCommand) initBackend(root *configs.Module, extraConfig rawFlags) (b
var backendConfigOverride hcl.Body
if root.Backend != nil {
backendType := root.Backend.Type
if backendType == "cloud" {
diags = diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Unsupported backend type",
Detail: fmt.Sprintf("There is no explicit backend type named %q. To configure Terraform Cloud, declare a 'cloud' block instead.", backendType),
Subject: &root.Backend.TypeRange,
})
return nil, true, diags
}
bf := backendInit.Backend(backendType)
if bf == nil {
diags = diags.Append(&hcl.Diagnostic{
@ -1050,6 +1099,10 @@ const outputInitSuccess = `
[reset][bold][green]Terraform has been successfully initialized![reset][green]
`
const outputInitSuccessCloud = `
[reset][bold][green]Terraform Cloud has been successfully initialized![reset][green]
`
const outputInitSuccessCLI = `[reset][green]
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
@ -1060,6 +1113,14 @@ rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
`
const outputInitSuccessCLICloud = `[reset][green]
You may now begin working with Terraform Cloud. Try running "terraform plan" to
see any changes that are required for your infrastructure.
If you ever set or change modules or Terraform Settings, run "terraform init"
again to reinitialize your working directory.
`
// providerProtocolTooOld is a message sent to the CLI UI if the provider's
// supported protocol versions are too old for the user's version of terraform,
// but a newer version of the provider is compatible.

137
internal/command/meta_backend.go
View File

@ -17,7 +17,7 @@ import (
"github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/hcldec"
"github.com/hashicorp/terraform/internal/backend"
remoteBackend "github.com/hashicorp/terraform/internal/backend/remote"
"github.com/hashicorp/terraform/internal/cloud"
"github.com/hashicorp/terraform/internal/command/arguments"
"github.com/hashicorp/terraform/internal/command/clistate"
"github.com/hashicorp/terraform/internal/command/views"
@ -55,6 +55,13 @@ type BackendOpts struct {
ForceLocal bool
}
// BackendWithRemoteTerraformVersion is a shared interface between the 'remote' and 'cloud' backends
// for simplified type checking when calling functions common to those particular backends.
type BackendWithRemoteTerraformVersion interface {
IgnoreVersionConflict()
VerifyWorkspaceTerraformVersion(workspace string) tfdiags.Diagnostics
}
// Backend initializes and returns the backend for this CLI session.
//
// The backend is used to perform the actual Terraform operations. This
@ -589,12 +596,21 @@ func (m *Meta) backendFromConfig(opts *BackendOpts) (backend.Backend, tfdiags.Di
case c != nil && s.Backend.Empty():
log.Printf("[TRACE] Meta.Backend: moving from default local state only to %q backend", c.Type)
if !opts.Init {
initReason := fmt.Sprintf("Initial configuration of the requested backend %q", c.Type)
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Backend initialization required, please run \"terraform init\"",
fmt.Sprintf(strings.TrimSpace(errBackendInit), initReason),
))
if c.Type == "cloud" {
initReason := "Initial configuration of Terraform Cloud"
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Terraform Cloud initialization required, please run \"terraform init\"",
fmt.Sprintf(strings.TrimSpace(errBackendInitCloud), initReason),
))
} else {
initReason := fmt.Sprintf("Initial configuration of the requested backend %q", c.Type)
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Backend initialization required, please run \"terraform init\"",
fmt.Sprintf(strings.TrimSpace(errBackendInit), initReason),
))
}
return nil, diags
}
@ -619,22 +635,39 @@ func (m *Meta) backendFromConfig(opts *BackendOpts) (backend.Backend, tfdiags.Di
}
log.Printf("[TRACE] Meta.Backend: backend configuration has changed (from type %q to type %q)", s.Backend.Type, c.Type)
initReason := fmt.Sprintf("Backend configuration changed for %q", c.Type)
if s.Backend.Type != c.Type {
initReason := ""
switch {
case c.Type == "cloud":
initReason = fmt.Sprintf("Backend configuration changed from %q to Terraform Cloud", s.Backend.Type)
case s.Backend.Type != c.Type:
initReason = fmt.Sprintf("Backend configuration changed from %q to %q", s.Backend.Type, c.Type)
default:
initReason = fmt.Sprintf("Backend configuration changed for %q", c.Type)
}
if !opts.Init {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Backend initialization required, please run \"terraform init\"",
fmt.Sprintf(strings.TrimSpace(errBackendInit), initReason),
))
if c.Type == "cloud" {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Terraform Cloud initialization required, please run \"terraform init\"",
fmt.Sprintf(strings.TrimSpace(errBackendInitCloud), initReason),
))
} else {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Backend initialization required, please run \"terraform init\"",
fmt.Sprintf(strings.TrimSpace(errBackendInit), initReason),
))
}
return nil, diags
}
if !m.migrateState {
diags = diags.Append(migrateOrReconfigDiag)
if c.Type == "cloud" {
diags = diags.Append(migrateOrReconfigDiagCloud)
} else {
diags = diags.Append(migrateOrReconfigDiag)
}
return nil, diags
}
@ -746,7 +779,11 @@ func (m *Meta) backend_c_r_S(c *configs.Backend, cHash int, sMgr *clistate.Local
// Get the backend type for output
backendType := s.Backend.Type
m.Ui.Output(fmt.Sprintf(strings.TrimSpace(outputBackendMigrateLocal), s.Backend.Type))
if s.Backend.Type == "cloud" {
m.Ui.Output(strings.TrimSpace(outputBackendMigrateLocalFromCloud))
} else {
m.Ui.Output(fmt.Sprintf(strings.TrimSpace(outputBackendMigrateLocal), s.Backend.Type))
}
// Grab a purely local backend to get the local state if it exists
localB, diags := m.Backend(&BackendOpts{ForceLocal: true, Init: true})
@ -915,9 +952,12 @@ func (m *Meta) backend_C_r_s(c *configs.Backend, cHash int, sMgr *clistate.Local
return nil, diags
}
// By now the backend is successfully configured.
m.Ui.Output(m.Colorize().Color(fmt.Sprintf(
"[reset][green]\n"+strings.TrimSpace(successBackendSet), s.Backend.Type)))
// By now the backend is successfully configured. If using Terraform Cloud, the success
// message is handled as part of the final init message
if _, ok := b.(*cloud.Cloud); !ok {
m.Ui.Output(m.Colorize().Color(fmt.Sprintf(
"[reset][green]\n"+strings.TrimSpace(successBackendSet), s.Backend.Type)))
}
return b, diags
}
@ -942,7 +982,11 @@ func (m *Meta) backend_C_r_S_changed(c *configs.Backend, cHash int, sMgr *clista
// no need to confuse the user if the backend types are the same
if s.Backend.Type != c.Type {
m.Ui.Output(strings.TrimSpace(fmt.Sprintf(outputBackendMigrateChange, s.Backend.Type, c.Type)))
output := fmt.Sprintf(outputBackendMigrateChange, s.Backend.Type, c.Type)
if c.Type == "cloud" {
output = fmt.Sprintf(outputBackendMigrateChangeCloud, s.Backend.Type)
}
m.Ui.Output(strings.TrimSpace(output))
}
// Grab the existing backend
@ -1001,8 +1045,12 @@ func (m *Meta) backend_C_r_S_changed(c *configs.Backend, cHash int, sMgr *clista
}
if output {
m.Ui.Output(m.Colorize().Color(fmt.Sprintf(
"[reset][green]\n"+strings.TrimSpace(successBackendSet), s.Backend.Type)))
// By now the backend is successfully configured. If using Terraform Cloud, the success
// message is handled as part of the final init message
if _, ok := b.(*cloud.Cloud); !ok {
m.Ui.Output(m.Colorize().Color(fmt.Sprintf(
"[reset][green]\n"+strings.TrimSpace(successBackendSet), s.Backend.Type)))
}
}
return b, diags
@ -1168,32 +1216,32 @@ func (m *Meta) backendInitFromConfig(c *configs.Backend) (backend.Backend, cty.V
return b, configVal, diags
}
// Helper method to ignore remote backend version conflicts. Only call this
// Helper method to ignore remote/cloud backend version conflicts. Only call this
// for commands which cannot accidentally upgrade remote state files.
func (m *Meta) ignoreRemoteBackendVersionConflict(b backend.Backend) {
if rb, ok := b.(*remoteBackend.Remote); ok {
rb.IgnoreVersionConflict()
func (m *Meta) ignoreRemoteVersionConflict(b backend.Backend) {
if back, ok := b.(BackendWithRemoteTerraformVersion); ok {
back.IgnoreVersionConflict()
}
}
// Helper method to check the local Terraform version against the configured
// version in the remote workspace, returning diagnostics if they conflict.
func (m *Meta) remoteBackendVersionCheck(b backend.Backend, workspace string) tfdiags.Diagnostics {
func (m *Meta) remoteVersionCheck(b backend.Backend, workspace string) tfdiags.Diagnostics {
var diags tfdiags.Diagnostics
if rb, ok := b.(*remoteBackend.Remote); ok {
if back, ok := b.(BackendWithRemoteTerraformVersion); ok {
// Allow user override based on command-line flag
if m.ignoreRemoteVersion {
rb.IgnoreVersionConflict()
back.IgnoreVersionConflict()
}
// If the override is set, this check will return a warning instead of
// an error
versionDiags := rb.VerifyWorkspaceTerraformVersion(workspace)
versionDiags := back.VerifyWorkspaceTerraformVersion(workspace)
diags = diags.Append(versionDiags)
// If there are no errors resulting from this check, we do not need to
// check again
if !diags.HasErrors() {
rb.IgnoreVersionConflict()
back.IgnoreVersionConflict()
}
}
@ -1283,6 +1331,19 @@ hasn't changed and try again. At this point, no changes to your existing
configuration or state have been made.
`
const errBackendInitCloud = `
Reason: %s
Changes to the Terraform Cloud configuration block require reinitialization.
This allows Terraform to set up the new configuration, copy existing state, etc.
Please run "terraform init" with either the "-reconfigure" or "-migrate-state"
flags to use the current configuration.
If the change reason above is incorrect, please verify your configuration
hasn't changed and try again. At this point, no changes to your existing
configuration or state have been made.
`
const errBackendWriteSaved = `
Error saving the backend configuration: %s
@ -1296,9 +1357,16 @@ const outputBackendMigrateChange = `
Terraform detected that the backend type changed from %q to %q.
`
const outputBackendMigrateChangeCloud = `
Terraform detected that the backend type changed from %q to Terraform Cloud.
`
const outputBackendMigrateLocal = `
Terraform has detected you're unconfiguring your previously set %q backend.
`
const outputBackendMigrateLocalFromCloud = `
Terraform has detected you're unconfiguring Terraform Cloud.
`
const outputBackendReconfigure = `
[reset][bold]Backend configuration changed![reset]
@ -1322,3 +1390,10 @@ var migrateOrReconfigDiag = tfdiags.Sourceless(
"A change in the backend configuration has been detected, which may require migrating existing state.\n\n"+
"If you wish to attempt automatic migration of the state, use \"terraform init -migrate-state\".\n"+
`If you wish to store the current configuration with no changes to the state, use "terraform init -reconfigure".`)
var migrateOrReconfigDiagCloud = tfdiags.Sourceless(
tfdiags.Error,
"Terraform Cloud configuration changed",
"A change in the Terraform Cloud configuration has been detected, which may require migrating existing state.\n\n"+
"If you wish to attempt automatic migration of the state, use \"terraform init -migrate-state\".\n"+
`If you wish to store the current configuration with no changes to the state, use "terraform init -reconfigure".`)

410
internal/command/meta_backend_migrate.go
View File

@ -12,6 +12,7 @@ import (
"strings"
"github.com/hashicorp/terraform/internal/backend"
"github.com/hashicorp/terraform/internal/cloud"
"github.com/hashicorp/terraform/internal/command/arguments"
"github.com/hashicorp/terraform/internal/command/clistate"
"github.com/hashicorp/terraform/internal/command/views"
@ -43,28 +44,21 @@ type backendMigrateOpts struct {
//
// This will attempt to lock both states for the migration.
func (m *Meta) backendMigrateState(opts *backendMigrateOpts) error {
log.Printf("[TRACE] backendMigrateState: need to migrate from %q to %q backend config", opts.SourceType, opts.DestinationType)
log.Printf("[INFO] backendMigrateState: need to migrate from %q to %q backend config", opts.SourceType, opts.DestinationType)
// We need to check what the named state status is. If we're converting
// from multi-state to single-state for example, we need to handle that.
var sourceSingleState, destinationSingleState bool
sourceWorkspaces, err := opts.Source.Workspaces()
if err == backend.ErrWorkspacesNotSupported {
sourceSingleState = true
err = nil
}
if err != nil {
return fmt.Errorf(strings.TrimSpace(
errMigrateLoadStates), opts.SourceType, err)
}
var sourceSingleState, destinationSingleState, sourceTFC, destinationTFC bool
destinationWorkspaces, err := opts.Destination.Workspaces()
if err == backend.ErrWorkspacesNotSupported {
destinationSingleState = true
err = nil
}
_, sourceTFC = opts.Source.(*cloud.Cloud)
_, destinationTFC = opts.Destination.(*cloud.Cloud)
sourceWorkspaces, sourceSingleState, err := retrieveWorkspaces(opts.Source, opts.SourceType)
if err != nil {
return fmt.Errorf(strings.TrimSpace(
errMigrateLoadStates), opts.DestinationType, err)
return err
}
destinationWorkspaces, destinationSingleState, err := retrieveWorkspaces(opts.Destination, opts.SourceType)
if err != nil {
return err
}
// Set up defaults
@ -75,25 +69,27 @@ func (m *Meta) backendMigrateState(opts *backendMigrateOpts) error {
// Disregard remote Terraform version for the state source backend. If it's a
// Terraform Cloud remote backend, we don't care about the remote version,
// as we are migrating away and will not break a remote workspace.
m.ignoreRemoteBackendVersionConflict(opts.Source)
m.ignoreRemoteVersionConflict(opts.Source)
// Disregard remote Terraform version if instructed to do so via CLI flag.
if m.ignoreRemoteVersion {
m.ignoreRemoteBackendVersionConflict(opts.Destination)
m.ignoreRemoteVersionConflict(opts.Destination)
} else {
// Check the remote Terraform version for the state destination backend. If
// it's a Terraform Cloud remote backend, we want to ensure that we don't
// break the workspace by uploading an incompatible state file.
for _, workspace := range destinationWorkspaces {
diags := m.remoteBackendVersionCheck(opts.Destination, workspace)
diags := m.remoteVersionCheck(opts.Destination, workspace)
if diags.HasErrors() {
return diags.Err()
}
}
// If there are no specified destination workspaces, perform a remote
// backend version check with the default workspace.
if len(destinationWorkspaces) == 0 {
diags := m.remoteBackendVersionCheck(opts.Destination, backend.DefaultStateName)
// Ensure that we are not dealing with Terraform Cloud migrations, as it
// does not support the default name.
if len(destinationWorkspaces) == 0 && !destinationTFC {
diags := m.remoteVersionCheck(opts.Destination, backend.DefaultStateName)
if diags.HasErrors() {
return diags.Err()
}
@ -103,6 +99,9 @@ func (m *Meta) backendMigrateState(opts *backendMigrateOpts) error {
// Determine migration behavior based on whether the source/destination
// supports multi-state.
switch {
case sourceTFC || destinationTFC:
return m.backendMigrateTFC(opts)
// Single-state to single-state. This is the easiest case: we just
// copy the default state directly.
case sourceSingleState && destinationSingleState:
@ -157,7 +156,7 @@ func (m *Meta) backendMigrateState(opts *backendMigrateOpts) error {
// Multi-state to multi-state.
func (m *Meta) backendMigrateState_S_S(opts *backendMigrateOpts) error {
log.Print("[TRACE] backendMigrateState: migrating all named workspaces")
log.Print("[INFO] backendMigrateState: migrating all named workspaces")
migrate := opts.force
if !migrate {
@ -212,9 +211,9 @@ func (m *Meta) backendMigrateState_S_S(opts *backendMigrateOpts) error {
// Multi-state to single state.
func (m *Meta) backendMigrateState_S_s(opts *backendMigrateOpts) error {
log.Printf("[TRACE] backendMigrateState: destination backend type %q does not support named workspaces", opts.DestinationType)
log.Printf("[INFO] backendMigrateState: destination backend type %q does not support named workspaces", opts.DestinationType)
currentEnv, err := m.Workspace()
currentWorkspace, err := m.Workspace()
if err != nil {
return err
}
@ -231,7 +230,7 @@ func (m *Meta) backendMigrateState_S_s(opts *backendMigrateOpts) error {
opts.DestinationType),
Description: fmt.Sprintf(
strings.TrimSpace(inputBackendMigrateMultiToSingle),
opts.SourceType, opts.DestinationType, currentEnv),
opts.SourceType, opts.DestinationType, currentWorkspace),
})
if err != nil {
return fmt.Errorf(
@ -244,7 +243,7 @@ func (m *Meta) backendMigrateState_S_s(opts *backendMigrateOpts) error {
}
// Copy the default state
opts.sourceWorkspace = currentEnv
opts.sourceWorkspace = currentWorkspace
// now switch back to the default env so we can acccess the new backend
m.SetWorkspace(backend.DefaultStateName)
@ -254,7 +253,7 @@ func (m *Meta) backendMigrateState_S_s(opts *backendMigrateOpts) error {
// Single state to single state, assumed default state name.
func (m *Meta) backendMigrateState_s_s(opts *backendMigrateOpts) error {
log.Printf("[TRACE] backendMigrateState: migrating %q workspace to %q workspace", opts.sourceWorkspace, opts.destinationWorkspace)
log.Printf("[INFO] backendMigrateState: single-to-single migrating %q workspace to %q workspace", opts.sourceWorkspace, opts.destinationWorkspace)
sourceState, err := opts.Source.StateMgr(opts.sourceWorkspace)
if err != nil {
@ -278,16 +277,9 @@ func (m *Meta) backendMigrateState_s_s(opts *backendMigrateOpts) error {
// for a new name and migrate the default state to the given named state.
destinationState, err = func() (statemgr.Full, error) {
log.Print("[TRACE] backendMigrateState: destination doesn't support a default workspace, so we must prompt for a new name")
name, err := m.UIInput().Input(context.Background(), &terraform.InputOpts{
Id: "new-state-name",
Query: fmt.Sprintf(
"[reset][bold][yellow]The %q backend configuration only allows "+
"named workspaces![reset]",
opts.DestinationType),
Description: strings.TrimSpace(inputBackendNewWorkspaceName),
})
name, err := m.promptNewWorkspaceName(opts.DestinationType)
if err != nil {
return nil, fmt.Errorf("Error asking for new state name: %s", err)
return nil, err
}
// Update the name of the destination state.
@ -444,12 +436,21 @@ func (m *Meta) backendMigrateState_s_s(opts *backendMigrateOpts) error {
}
func (m *Meta) backendMigrateEmptyConfirm(source, destination statemgr.Full, opts *backendMigrateOpts) (bool, error) {
inputOpts := &terraform.InputOpts{
Id: "backend-migrate-copy-to-empty",
Query: "Do you want to copy existing state to the new backend?",
Description: fmt.Sprintf(
strings.TrimSpace(inputBackendMigrateEmpty),
opts.SourceType, opts.DestinationType),
var inputOpts *terraform.InputOpts
if opts.DestinationType == "cloud" {
inputOpts = &terraform.InputOpts{
Id: "backend-migrate-copy-to-empty-cloud",
Query: "Do you want to copy existing state to Terraform Cloud?",
Description: fmt.Sprintf(strings.TrimSpace(inputBackendMigrateEmptyCloud), opts.SourceType),
}
} else {
inputOpts = &terraform.InputOpts{
Id: "backend-migrate-copy-to-empty",
Query: "Do you want to copy existing state to the new backend?",
Description: fmt.Sprintf(
strings.TrimSpace(inputBackendMigrateEmpty),
opts.SourceType, opts.DestinationType),
}
}
return m.confirm(inputOpts)
@ -485,18 +486,266 @@ func (m *Meta) backendMigrateNonEmptyConfirm(
}
// Ask for confirmation
inputOpts := &terraform.InputOpts{
Id: "backend-migrate-to-backend",
Query: "Do you want to copy existing state to the new backend?",
Description: fmt.Sprintf(
strings.TrimSpace(inputBackendMigrateNonEmpty),
opts.SourceType, opts.DestinationType, sourcePath, destinationPath),
var inputOpts *terraform.InputOpts
if opts.DestinationType == "cloud" {
inputOpts = &terraform.InputOpts{
Id: "backend-migrate-to-tfc",
Query: "Do you want to copy existing state to Terraform Cloud?",
Description: fmt.Sprintf(
strings.TrimSpace(inputBackendMigrateNonEmptyCloud),
opts.SourceType, sourcePath, destinationPath),
}
} else {
inputOpts = &terraform.InputOpts{
Id: "backend-migrate-to-backend",
Query: "Do you want to copy existing state to the new backend?",
Description: fmt.Sprintf(
strings.TrimSpace(inputBackendMigrateNonEmpty),
opts.SourceType, opts.DestinationType, sourcePath, destinationPath),
}
}
// Confirm with the user that the copy should occur
return m.confirm(inputOpts)
}
func retrieveWorkspaces(back backend.Backend, sourceType string) ([]string, bool, error) {
var singleState bool
var err error
workspaces, err := back.Workspaces()
if err == backend.ErrWorkspacesNotSupported {
singleState = true
err = nil
}
if err != nil {
return nil, singleState, fmt.Errorf(strings.TrimSpace(
errMigrateLoadStates), sourceType, err)
}
return workspaces, singleState, err
}
func (m *Meta) backendMigrateTFC(opts *backendMigrateOpts) error {
_, sourceTFC := opts.Source.(*cloud.Cloud)
cloudBackendDestination, destinationTFC := opts.Destination.(*cloud.Cloud)
sourceWorkspaces, sourceSingleState, err := retrieveWorkspaces(opts.Source, opts.SourceType)
if err != nil {
return err
}
//to be used below, not yet implamented
// destinationWorkspaces, destinationSingleState
_, _, err = retrieveWorkspaces(opts.Destination, opts.SourceType)
if err != nil {
return err
}
// from TFC to non-TFC backend
if sourceTFC && !destinationTFC {
// From Terraform Cloud to another backend. This is not yet implemented, and
// we recommend people to use the TFC API.
return fmt.Errorf(strings.TrimSpace(errTFCMigrateNotYetImplemented))
}
// Everything below, by the above two conditionals, now assumes that the
// destination is always Terraform Cloud (TFC).
sourceSingle := sourceSingleState || (len(sourceWorkspaces) == 1)
if sourceSingle {
if cloudBackendDestination.WorkspaceMapping.Strategy() == cloud.WorkspaceNameStrategy {
// If we know the name via WorkspaceNameStrategy, then set the
// destinationWorkspace to the new Name and skip the user prompt. Here the
// destinationWorkspace is not set to `default` thereby we will create it
// in TFC if it does not exist.
opts.destinationWorkspace = cloudBackendDestination.WorkspaceMapping.Name
}
currentWorkspace, err := m.Workspace()
if err != nil {
return err
}
opts.sourceWorkspace = currentWorkspace
log.Printf("[INFO] backendMigrateTFC: single-to-single migration from source %s to destination %q", opts.sourceWorkspace, opts.destinationWorkspace)
// Run normal single-to-single state migration
// This will handle both situations where the new cloud backend
// configuration is using a workspace.name strategy or workspace.tags
// strategy.
return m.backendMigrateState_s_s(opts)
}
destinationTagsStrategy := cloudBackendDestination.WorkspaceMapping.Strategy() == cloud.WorkspaceTagsStrategy
destinationNameStrategy := cloudBackendDestination.WorkspaceMapping.Strategy() == cloud.WorkspaceNameStrategy
multiSource := !sourceSingleState && len(sourceWorkspaces) > 1
if multiSource && destinationNameStrategy {
currentWorkspace, err := m.Workspace()
if err != nil {
return err
}
opts.sourceWorkspace = currentWorkspace
opts.destinationWorkspace = cloudBackendDestination.WorkspaceMapping.Name
if err := m.promptMultiToSingleCloudMigration(opts); err != nil {
return err
}
log.Printf("[INFO] backendMigrateTFC: multi-to-single migration from source %s to destination %q", opts.sourceWorkspace, opts.destinationWorkspace)
return m.backendMigrateState_s_s(opts)
}
// Multiple sources, and using tags strategy. So migrate every source
// workspace over to new one, prompt for workspace name pattern (*),
// and start migrating, and create tags for each workspace.
if multiSource && destinationTagsStrategy {
log.Printf("[INFO] backendMigrateTFC: multi-to-multi migration from source workspaces %q", sourceWorkspaces)
return m.backendMigrateState_S_TFC(opts, sourceWorkspaces)
}
// TODO(omar): after the check for sourceSingle is done, everything following
// it has to be multi. So rework the code to not need to check for multi, adn
// return m.backendMigrateState_S_TFC here.
return nil
}
// migrates a multi-state backend to Terraform Cloud
func (m *Meta) backendMigrateState_S_TFC(opts *backendMigrateOpts, sourceWorkspaces []string) error {
log.Print("[TRACE] backendMigrateState: migrating all named workspaces")
// This map is used later when doing the migration per source/destination.
// If a source has 'default', then we ask what the new name should be.
// And further down when we actually run state migration for each
// sourc/destination workspce, we use this new name (where source is 'default')
// and set as destinationWorkspace.
defaultNewName := map[string]string{}
for i := 0; i < len(sourceWorkspaces); i++ {
if sourceWorkspaces[i] == backend.DefaultStateName {
newName, err := m.promptNewWorkspaceName(opts.DestinationType)
if err != nil {
return err
}
defaultNewName[sourceWorkspaces[i]] = newName
}
}
pattern, err := m.promptMultiStateMigrationPattern(opts.SourceType)
if err != nil {
return err
}
// Go through each and migrate
for _, name := range sourceWorkspaces {
// Copy the same names
opts.sourceWorkspace = name
if newName, ok := defaultNewName[name]; ok {
// this has to be done before setting destinationWorkspace
name = newName
}
opts.destinationWorkspace = strings.Replace(pattern, "*", name, -1)
// Force it, we confirmed above
opts.force = true
// Perform the migration
log.Printf("[INFO] backendMigrateTFC: multi-to-multi migration, source workspace %q to destination workspace %q", opts.sourceWorkspace, opts.destinationWorkspace)
if err := m.backendMigrateState_s_s(opts); err != nil {
return fmt.Errorf(strings.TrimSpace(
errMigrateMulti), name, opts.SourceType, opts.DestinationType, err)
}
}
// After migrating multiple workspaces, we want to ensure that a workspace is
// set or we prompt the user to set a workspace.
err = m.selectWorkspace(opts.Destination)
if err != nil {
return err
}
return nil
}
// Multi-state to single state.
func (m *Meta) promptMultiToSingleCloudMigration(opts *backendMigrateOpts) error {
migrate := opts.force
if !migrate {
var err error
// Ask the user if they want to migrate their existing remote state
migrate, err = m.confirm(&terraform.InputOpts{
Id: "backend-migrate-multistate-to-single",
Query: "Do you want to copy only your current workspace?",
Description: fmt.Sprintf(
strings.TrimSpace(tfcInputBackendMigrateMultiToSingle),
opts.SourceType, opts.destinationWorkspace),
})
if err != nil {
return fmt.Errorf("Error asking for state migration action: %s", err)
}
}
if !migrate {
return fmt.Errorf("Migration aborted by user.")
}
return nil
}
func (m *Meta) promptNewWorkspaceName(destinationType string) (string, error) {
message := fmt.Sprintf("[reset][bold][yellow]The %q backend configuration only allows "+
"named workspaces![reset]", destinationType)
if destinationType == "cloud" {
message = `[reset][bold][yellow]Terraform Cloud requires all workspaces to be given an explicit name.[reset]`
}
name, err := m.UIInput().Input(context.Background(), &terraform.InputOpts{
Id: "new-state-name",
Query: message,
Description: strings.TrimSpace(inputBackendNewWorkspaceName),
})
if err != nil {
return "", fmt.Errorf("Error asking for new state name: %s", err)
}
return name, nil
}
func (m *Meta) promptMultiStateMigrationPattern(sourceType string) (string, error) {
renameWorkspaces, err := m.UIInput().Input(context.Background(), &terraform.InputOpts{
Id: "backend-migrate-multistate-to-tfc",
Query: fmt.Sprintf("[reset][bold][yellow]%s[reset]", "Would you like to rename your workspaces?"),
Description: fmt.Sprintf(strings.TrimSpace(tfcInputBackendMigrateMultiToMulti), sourceType),
})
if err != nil {
return "", fmt.Errorf("Error asking for state migration action: %s", err)
}
if renameWorkspaces != "2" && renameWorkspaces != "1" {
return "", fmt.Errorf("Please select 1 or 2 as part of this option.")
}
if renameWorkspaces == "2" {
// this means they did not want to rename their workspaces, and we are
// returning a generic '*' that means use the same workspace name during
// migration.
return "*", nil
}
pattern, err := m.UIInput().Input(context.Background(), &terraform.InputOpts{
Id: "backend-migrate-multistate-to-tfc-pattern",
Query: fmt.Sprintf("[reset][bold][yellow]%s[reset]", "What pattern would you like to add to all your workspaces?"),
Description: strings.TrimSpace(tfcInputBackendMigrateMultiToMultiPattern),
})
if err != nil {
return "", fmt.Errorf("Error asking for state migration action: %s", err)
}
if !strings.Contains(pattern, "*") {
return "", fmt.Errorf("The pattern must have an '*'")
}
if count := strings.Count(pattern, "*"); count > 1 {
return "", fmt.Errorf("The pattern '*' cannot be used more than once.")
}
return pattern, nil
}
const errMigrateLoadStates = `
Error inspecting states in the %q backend:
%s
@ -541,6 +790,46 @@ The state in the previous backend remains intact and unmodified. Please resolve
the error above and try again.
`
const errTFCMigrateNotYetImplemented = `
Migrating state from Terraform Cloud to another backend is not yet implemented.
Please use the API to do this: https://www.terraform.io/docs/cloud/api/state-versions.html
`
const tfcInputBackendMigrateMultiToMultiPattern = `
If you choose to NOT rename your workspaces, just input "*".
The asterisk "*" represents your workspace name. Here are a few examples
if a workspace was named 'prod':
* input: 'app-*'; output: 'app-prod'
* input: '*-app', output: 'prod-app'
* input: 'app-*-service', output: 'app-prod-service'
* input: '*'; output: 'prod'
`
const tfcInputBackendMigrateMultiToMulti = `
When migrating existing workspaces from the backend %[1]q to Terraform Cloud, would you like to
rename your workspaces?
Unlike typical Terraform workspaces representing an environment associated with a particular
configuration (e.g. production, staging, development), Terraform Cloud workspaces are named uniquely
across all configurations used within an organization. A typical strategy to start with is
<COMPONENT>-<ENVIRONMENT>-<REGION> (e.g. networking-prod-us-east, networking-staging-us-east).
For more information on workspace naming, see https://www.terraform.io/docs/cloud/workspaces/naming.html
1. Yes, rename workspaces according to a pattern.
2. No, I would not like to rename my workspaces. Migrate them as currently named.
`
const tfcInputBackendMigrateMultiToSingle = `
The previous backend %[1]q has multiple workspaces, but Terraform Cloud has been
configured to use a single workspace (%[2]q). By continuing, you will only
migrate your current workspace. If you wish to migrate all workspaces from the
previous backend, use the 'tags' strategy in your workspace configuration block
instead.
`
const inputBackendMigrateEmpty = `
Pre-existing state was found while migrating the previous %q backend to the
newly configured %q backend. No existing state was found in the newly
@ -548,6 +837,12 @@ configured %[2]q backend. Do you want to copy this state to the new %[2]q
backend? Enter "yes" to copy and "no" to start with an empty state.
`
const inputBackendMigrateEmptyCloud = `
Pre-existing state was found while migrating the previous %q backend to Terraform Cloud.
No existing state was found in Terraform Cloud. Do you want to copy this state to Terraform Cloud?
Enter "yes" to copy and "no" to start with an empty state.
`
const inputBackendMigrateNonEmpty = `
Pre-existing state was found while migrating the previous %q backend to the
newly configured %q backend. An existing non-empty state already exists in
@ -562,6 +857,19 @@ Enter "yes" to copy and "no" to start with the existing state in the newly
configured %[2]q backend.
`
const inputBackendMigrateNonEmptyCloud = `
Pre-existing state was found while migrating the previous %q backend to
Terraform Cloud. An existing non-empty state already exists in Terraform Cloud.
The two states have been saved to temporary files that will be removed after
responding to this query.
Previous (type %[1]q): %[2]s
New (Terraform Cloud): %[3]s
Do you want to overwrite the state in Terraform Cloud with the previous state?
Enter "yes" to copy and "no" to start with the existing state in Terraform Cloud.
`
const inputBackendMigrateMultiToSingle = `
The existing %[1]q backend supports workspaces and you currently are
using more than one. The newly configured %[2]q backend doesn't support
@ -587,7 +895,7 @@ If you answer "yes", Terraform will migrate all states. If you answer
const inputBackendNewWorkspaceName = `
Please provide a new workspace name (e.g. dev, test) that will be used
to migrate the existing default workspace.
to migrate the existing default workspace.
`
const inputBackendSelectWorkspace = `

Some files were not shown because too many files have changed in this diff Show More