Commit Graph

787 Commits

Author SHA1 Message Date
Tom Harvey a1c7c0dfa4
Update internal/backend/remote-state/azure/backend.go
Co-authored-by: Tom Bamford <tbamford@hashicorp.com>
2021-11-17 20:24:22 +01:00
tombuildsstuff ffe9a41468 internal/backend/remote-state/azure: adding tests covering MSAL 2021-11-17 18:45:44 +01:00
tombuildsstuff a2cad7d579 internal/backend/remote-state/azure: prefixing the existing tests with ADAL 2021-11-17 18:43:39 +01:00
tombuildsstuff 9f710558ce internal/backend/remote-state/azure: adding a feature-toggle to use MSAL 2021-11-17 18:42:01 +01:00
tombuildsstuff 27a34d6c2c internal/backend/remote-state/azure: updating the link to the client secret
This redirects to the registry so we should point there directly
2021-11-17 18:28:47 +01:00
tombuildsstuff 5133d30b71 internal/backend/remote-state/azure: code changes for go-azure-helpers v0.18.0 2021-11-17 18:26:23 +01:00
Jared Ledvina 61a5808680 k8s-backend - Initial context support for library update
Signed-off-by: Jared Ledvina <jared.ledvina@datadoghq.com>
2021-11-16 14:43:50 -05:00
Barrett Clark 02aacb09e6
Merge pull request #29863 from hashicorp/barrettclark/migrate-empty-default-workspace
Cloud: Migrate empty default workspace prompt
2021-11-16 13:27:39 -06:00
Barrett Clark a146a2746e Add clarifying commend and e2e test
This pull request focuses on removing the prompt to rename the default
workspace when it is empty. Functionality already exists to not migrate
an empty workspace. This commit adds some clarifying language in the
comment where we do the evaluation to know whether to ask for a new name
or not. I also added an end to end test, which I should have added to
begin with.
2021-11-16 13:05:26 -06:00
kmoe 18ab3512fa
Merge pull request #29943 from hashicorp/kmoe/moved-block-resource-type
refactoring: error when changing resource type during move
2021-11-16 18:25:27 +00:00
Katy Moe aeecc6a627
add AffectedAbsResource to interface 2021-11-16 18:19:11 +00:00
Barrett Clark a9eb62d692 Cloud: Migrate empty default workspace prompt
Given: You have multiple explicit local workspaces, and the `default`
workspace is empty.
When: You migrate the workspaces to Terraform Cloud.
Then: Terraform should _not_ ask for a workspace to migrate the
`default` workspace to in Terraform Cloud.
2021-11-16 10:33:46 -06:00
Simão Gomes Viana 5bbd108572
funcs: defer close file in funcs
Files opened by these two functions were not being closed,
leaking file descriptors. Close files that were opened when the
function exist.
2021-11-16 09:25:49 +01:00
Barrett Clark 388c430ece
Merge pull request #29883 from hashicorp/barrettclark/cloud-e2e-tests
Cloud: Make e2e Tests Less Chatty, and More Stable
2021-11-15 10:45:14 -06:00
Omar Ismail 9b675f8b70 Add skip for cloud e2e tests when env vars missing 2021-11-15 10:36:19 -05:00
Chris Arcand bf76cc98ef command: Suggestions on migration copy from user feedback
Some small edits to the TFC migration copy.
2021-11-15 09:21:31 -06:00
Katy Moe 11566b1d11
introduce AbsMoveableResource
AbsMoveableResource is an AbsMoveable that is either a resource or a resource
instance. This interface represents a moveable that has a resource type.
2021-11-15 11:05:33 +00:00
Katy Moe 7162e79fdf
return error on resource type mismatch
A resource move is invalid if the two resource( instance)s are of different
resource types. Check for this during move validation.
2021-11-15 11:05:33 +00:00
Katy Moe dc3ed6271c
add failing test for resource type mismatch 2021-11-15 11:05:33 +00:00
Katy Moe 4305271cff
remove occurrences of AbsMovable 2021-11-15 11:05:33 +00:00
Katy Moe 1dabdf0dcd
normalise test names 2021-11-14 21:51:53 +00:00
James Bardin 33bcc715a0
Merge pull request #29928 from hashicorp/jbardin/ignore_null_map
`ignore_changes` converting null maps to empty maps, and applying marks.
2021-11-12 14:32:31 -05:00
Omar Ismail 57a4b51e87 remove build tags for cloud e2e 2021-11-11 16:33:26 -05:00
Barrett Clark 5ef82ddd2f Cloud: fix e2e tests
- Fix tests and remove commented code
- Remove parallel for some flaky tests
- Add README
2021-11-11 14:37:21 -05:00
Brandon Croft 1f01ba4dbc
fix(run variables): support all variable types (map, list, bool, number, null, string)
All run variables remain encoded as strings in the API but will now be expressed as an HCL value to be evaluated correctly by the remote terraform. Previously, only strings were supported.

Examples:
string: `"quoted literal"` (strings must be quoted)
map:  `{ foo = "bar" }`
list: `["foo", "bar"]`
bool: `true`
null: `null`
number: `0.0001`

This requires the API to anticipate that all run variables will be HCL values
2021-11-11 10:36:01 -07:00
Brandon Croft 9441666d9a
Merge pull request #29903 from hashicorp/pull-mock-gotfe-changes
Pull changes made to mocks in go-tfe
2021-11-11 10:12:09 -07:00
James Bardin 9d19086c8e test for null map and fix lost map marks
Add a test to ensure ignore_changes does not change null maps to empty
maps.

Fix when a marked map revers the changes ignored within that map.
2021-11-11 10:44:39 -05:00
James Bardin 40174b634a ignore_changes changing a null map to empty
Fix an error where using `ignore_changes` would cause some null maps to
be saved as an empty map.
2021-11-11 10:44:05 -05:00
kmoe 64635edfb9
Merge pull request #29885 from hashicorp/kmoe/more-init-interrupts
command: make module installation interruptible
2021-11-11 12:37:49 +00:00
Katy Moe 6da61bf07b
revert change to installModules diag handling
Error diags from c.installModules() no longer cause getModules() to exit early.
Whether installModules completed successfully, errored, or was cancelled, we
try to update the manifest as best we can, preferring incomplete information
to none.
2021-11-11 12:28:16 +00:00
kmoe 40ec62c139
command: make module installation interruptible
Earlier work to make "terraform init" interruptible made the getproviders
package context-aware in order to allow provider installation to be cancelled.

Here we make a similar change for module installation, which is now also
cancellable with SIGINT. This involves plumbing context through initwd and
getmodules. Functions which can make network requests now include a context
parameter whose cancellation cancels those requests.

Since the module installation code is shared, "terraform get" is now
also interruptible during module installation.
2021-11-11 12:28:10 +00:00
Krista LaFentres c44c589f0a Error if backup or backup-out options are used without the state option on non-local backends for the state mv command 2021-11-09 13:09:36 -06:00
Chris Arcand 21750037d5 cloud: Help output typos 2021-11-08 22:07:37 -06:00
uturunku1 58304b678c import path to go mocks in tfe 2021-11-08 08:58:56 -08:00
Luces Huayhuaca 4e3218b4d5
cloud: convert uses of worspaces.operations into workspaces.executionMode (#29844)
* convert uses of worspaces.operations into workspaces.executionMode

The cloud package currently uses a deprecated API on workspaces to determine a workspace's execution mode.

Deprecated: Operations (boolean)
New hotness: Execution mode (string - "local", "remote", or "agent")

More details: https://www.terraform.io/docs/cloud/api/workspaces.html#request-body

All uses of Operations field coming from the client (within the cloud package) should be converted to the appropriate ExecutionMode equivalent.
Also, we need to update all acknowledgment of operations field on the tests that are testing the behavior of workspaces.

Co-authored-by: Nick Fagerlund <nick.fagerlund@gmail.com>

Co-authored-by: Nick Fagerlund <nick.fagerlund@gmail.com>
2021-11-08 07:20:15 -08:00
Chris Arcand 779c958fbf cloud: Add streamlined 'remote' backend state migration path
For Terraform Cloud users using the 'remote' backend, the existing
'pattern' prompt should work just fine - but because their workspaces
are already present in TFC, the 'migration' here is really just
realigning their local workspaces with Terraform Cloud. Instead of
forcing users to do the mental gymnastics of what it means to migrate
from 'prefix' - and because their remote workspaces probably already exist and
already conform to Terraform Cloud's naming concerns - streamline the
process for them and calculate the necessary pattern to migrate as-is,
without any user intervention necessary.
2021-11-03 15:07:33 -05:00
Chris Arcand 2690f738aa
Merge pull request #29878 from hashicorp/tweak-multi-to-multi-migration-tfc
cloud: Autoselect migrated current workspace + migrate UX adjustments
2021-11-03 12:12:24 -05:00
Chris Arcand 6f25ad3e17 cloud: Display the newly renamed workspaces after a migration 2021-11-03 11:27:35 -05:00
Chris Arcand c3bf5d4512 cloud: Automatically select renamed current workspace
After migrating to TFC with renamed workspaces, automatically select
what was the previous current workspace on behalf of the user. We don't
need to make the user reselect.
2021-11-03 11:27:35 -05:00
Chris Arcand f80b2177c8 cloud: Tweak multi-to-multi migration UX
Note these change do break the internal/cloud/e2e tests; they are in a
sad state that needs adjusting anyway, so I'm not updating them for
these changes at this time.
2021-11-03 11:26:52 -05:00
James Bardin 5a8021f03d
Merge pull request #29868 from xiaozhu36/xiaozhu
backend/oss: Removes the ConflictWith tag which on the attributes assume_role* to fix the incompatible error
2021-11-03 09:14:12 -04:00
James Bardin 913f9cc44a
Merge pull request #29862 from hashicorp/jbardin/validate-resource-values
generate precise resource types during validate
2021-11-03 08:59:09 -04:00
xiaozhu36 e0a2716601 backend/oss: Removes the ConflictWith tag which on the attributes assume_role* to fix the incompatible error 2021-11-03 11:17:35 +08:00
Chris Arcand 858dc96859
Merge pull request #29853 from hashicorp/cloud-tags-workspace
Cloud: Update Workspace during Tags strategy
2021-11-02 20:33:08 -05:00
Omar Ismail 1da7031855 cloud: Add tags to workspace if necessary when fetching state 2021-11-02 16:18:17 -05:00
Chris Arcand f04ea2bd3d
Merge pull request #29860 from hashicorp/fix-init-after-error-take-two
command: Update backend hash value only after a successful migration
2021-11-02 15:28:58 -05:00
Chris Arcand 7c0c2e952f command: Don't always update backend hash when fetching the saved backend
The Meta.backend_C_r_S_unchanged() method was sadly a bit of a mess.

It seems to have originally been used as a method to be called
when the backend is not changing, with an extra assumption that if the
configured backend's hash doesn't match the one in state, surely the
hash should just be updated as an option might have been moved to
command line flags.

However, this function was used throughout this file as 'the method to
load the initialized (but not necessarily configured) backend',
regardless of whether or not it is the same (unchanged). This is in
addition to Meta.backendFromState(), which is used to load the same
thing except in the main codepath of 'init -backend=false'.

These changes separate the concerns of backend_C_r_S_unchanged() by

1) Fetching the saved backend (savedBackend())
2) Updating the hash value in the backend cache when appropriate (either
   by leaving it to the caller to do themselves or by calling
   updateSavedupdateSavedBackendHash())

This allows migration codepaths to *not* update the hash value until
after a migration has successfully taken place.
2021-11-02 15:23:58 -05:00
James Bardin 647d36062c we don't need an abs provider address
We only lookup providers by provider type to get the schema, so there's
no reason to generate anything more specific.
2021-11-02 15:38:53 -04:00
James Bardin 6b8b0617bb generate precise resource types during validate
Allow `GetResource` to return correct types values during validation,
rather than relying on `cty.DynamicVal` as a placeholder. This allows
other dependent expressions to be more correctly evaluated.
2021-11-02 15:38:36 -04:00
Łukasz Sierant 19cce931a8 Test case for changing backend hash during aborted state migration
Pulled and updated from
https://github.com/hashicorp/terraform/pull/26260
2021-11-02 10:20:37 -05:00
Billy Keyes 52ca30273f
command/format: improve list nested attribute rendering (#29827)
* command/format: fix list nested attr diff rendering

Previously, diffs only rendered correctly if all changed elements
appeared before all unchanged elements. Once an unchanged element was
found, all remaining elements were skipped. This usually led to the
output being an empty list with a weird amount of space between the
brackets.

* command/format: improve list nested attr rendering

This makes several changes that make diffs for lists clearer and more
consistent:

  * Separate items with brackets instead of only new lines. This better
    matches the input syntax and avoids confusion from the first and
    last brackets implying there is a single item.
  * Render an action symbol for each element of the list
  * Use the correct action symbol for new and deleted items
  * Fix the alignment of opening and closing brackets

I also refactored the structure so it is similar to the set and map
cases to minimize duplication of the new prints.

* Fix re-use of blockBodyDiffResult struct
2021-11-02 11:13:56 -04:00
kmoe ba4b6652fa
Merge pull request #29849 from hashicorp/kmoe/ignore_changes-override
configs: fix ignore_changes config override bug
2021-11-01 19:54:04 +00:00
Barrett Clark 13ec9b539c
Merge pull request #29846 from hashicorp/barrettclark/parrelize-cloud-e2e-tests
cloud: Allow cloud tests to be run in parallel
2021-11-01 14:39:25 -05:00
James Bardin 834f61cd54
Merge pull request #29841 from xiaozhu36/xiaozhu
backend/oss: Supports the new attribute sts_endpoint
2021-11-01 15:31:51 -04:00
Katy Moe 8e640ae151
fix typo 2021-11-01 19:09:14 +00:00
Katy Moe 257c24ff75
IgnoreAllChanges should override during merge 2021-11-01 19:08:30 +00:00
Katy Moe df6dad5070
add failing test for IgnoreAllChanges override 2021-11-01 19:00:54 +00:00
Barrett Clark 9c13521d76 Allow cloud tests to be run in parallel
Add the `-parallel N` switch to tell the tests to run in N processes:

```
TFE_TOKEN=$TFE_TOKEN TFE_HOSTNAME=$TFE_HOSTNAME TF_ACC=1 go test -v \
  -tags=e2e ./internal/cloud/e2e/... -parallel 4
```
2021-11-01 13:58:29 -05:00
Nick Fagerlund 02e62c9851
Cloud: Init without erroring when no workspaces match the `tags` (#29835)
Previously, `terraform init` was throwing an error if you configured the cloud
block with `tags` and there weren't any tagged workspaces yet. Confusing and
alienating, since that that's a fairly normal situation! Basically TFC was
handling an empty list of workspaces worse than other backends, because it
doesn't support an unnamed default workspace.

This commit catches that condition during `Meta.selectBackend()` and asks the
user to pick a name for their first tagged workspace. If they cancel out, we
still error, but if we know what name they want, we can handle it the same way
as a nonexistent workspace specified in `name` -- just pass it to
`Meta.SetWorkspace()`, and let the workspace get implicitly created when
`InitCommand.Run()` eventually calls `StateMgr()`.
2021-11-01 10:20:15 -07:00
James Bardin b91d9435ea
Merge pull request #29832 from hashicorp/jbardin/nullable-variable
configs: explicitly nullable variable values
2021-11-01 12:46:31 -04:00
James Bardin 6d050c166e udpate diagnostic message 2021-11-01 12:28:55 -04:00
Martin Atkins 94cbc8fb5d experiments: config_driven_move has concluded
Based on feedback during earlier alpha releases, we've decided to move
forward with the current design for the first phase of config-driven
refactoring.

Therefore here we've marked the experiment as concluded with no changes
to the most recent incarnation of the functionality. The other changes
here are all just updating test fixtures to no longer declare that they
are using experimental features.
2021-11-01 08:46:15 -07:00
Alisdair McDiarmid a61bd8a96b cloud: Fix prerelease version constraint checks 2021-11-01 11:12:58 -04:00
xiaozhu36 5d19a34f88 backend/oss: Supports the new attribute sts_endpoint 2021-11-01 22:52:45 +08:00
Chris Arcand 3cd6c0be7a
Merge pull request #29836 from hashicorp/chrisarcand/fix-tfc-resource-counts
command: Adjust skipping of resource counts for any remote implementation
2021-11-01 09:50:13 -05:00
James Bardin 2ebdc099ac
Merge pull request #29307 from hayorov/main
Flatten `assume_role` block for OSS backend
2021-11-01 09:18:25 -04:00
James Bardin 1f873776e8 update null variable error text 2021-11-01 09:12:28 -04:00
James Bardin b71b393cf6 move nullable check to variable input evaluation 2021-11-01 09:02:32 -04:00
Chris Arcand 511afcd43a command: Adjust skipping of resource counts for any remote implementation
When using the Terraform Cloud integration - like the 'remote'
backend - resource count output should be suppressed if those counts are
being rendered remotely. This generalizes this to the shared
BackendWithRemoteTerraformVersion interface.
2021-10-29 21:23:28 -05:00
James Bardin 7b7972ac95 allow nullable override 2021-10-29 17:20:14 -04:00
James Bardin 81e709d185
Merge pull request #29825 from hashicorp/jbardin/no-panicwrap
Remove the use of panicwrap in the Terraform CLI
2021-10-29 14:41:05 -04:00
James Bardin f0a64eb456 configs: explicitly nullable variable values
The current behavior of module input variables is to allow users to
override a default by assigning `null`, which works contrary to the
behavior of resource attributes, and prevents explicitly accepting a
default when the input must be defined in the configuration.

Add a new variable attribute called `nullable` will allow explicitly
defining when a variable can be set to null or not. The current default
behavior is that of `nullable=true`.

Setting `nullable=false` in a variable block indicates that the variable
value can never be null. This either requires a non-null input value, or
a non-null default value. In the case of the latter, we also opt-in to
the new behavior of a `null` input value taking the default rather than
overriding it.

In a future language edition where we make `nullable=false` the default,
setting `nullable=true` will allow the legacy behavior of `null`
overriding a default value. The only future configuration in which this
would be required even if the legacy behavior were not desired is when
setting an optional+nullable value. In that case `default=null` would
also be needed and we could therefor imply `nullable=true` without
requiring it in the configuration.
2021-10-29 13:59:46 -04:00
Martin Atkins f266d1ee82
"Add cloud integration option"
A more native integration for Terraform Cloud and its CLI-driven run workflow.

Instead of a backend, users declare a special block in the top-level terraform settings
block to configure Terraform Cloud, then run terraform init.

Full documentation will follow in later commits.
2021-10-28 18:30:01 -07:00
Chris Arcand 369264865e internal/plugin[6]: Add generated mocks 2021-10-28 20:16:26 -05:00
Chris Arcand 6dc1fed6d5 Clarify legacy Ui comments 2021-10-28 19:29:21 -05:00
Chris Arcand 14260a5b4c Adjust default workspace naming prompt
The previous version is a little overdramatic and somewhat accusatory.
Just lay it out how it is: TFC needs workspaces to be named.
2021-10-28 19:29:21 -05:00
Brandon Croft 6d9c919f55 adds X-Terraform-Integration header
This can help differentiate cloud integration API requests from normal remote backend requests
2021-10-28 19:29:20 -05:00
Chris Arcand 3d23a8e062 cloud: Fix version check regression
When the 'select the exact version if possible' behavior was added, the
version check below it was never updated to take the newly updated
version in to account, resulting in a failed version check even as the
remote workspace updated to the correct version necessary.
2021-10-28 19:29:20 -05:00
Chris Arcand 652a68715f cloud/e2e: Misc test fixes 2021-10-28 19:29:20 -05:00
Chris Arcand f881c2d794 cloud: Skip E2E tests without an exact version 2021-10-28 19:29:20 -05:00
Chris Arcand e364ef2905 cloud/e2e: Disable cost estimation
E2E tests including cost estimation should indeed be added, but the
default case should be disabled; lots of cycles lost to pointless cost
estimates on null and random resources.
2021-10-28 19:29:19 -05:00
Omar Ismail fc5863844c Cloud migration: ignore backend version check when empty worksapces 2021-10-28 19:29:19 -05:00
Barrett Clark f5366468b4 Cloud Backend reference migrating away from TFC 2021-10-28 19:29:19 -05:00
Chris Arcand 2c0294c7e3 Tweak configuration copy for TFC
This aligns more with the existing copy for backends
2021-10-28 19:29:19 -05:00
Nick Fagerlund fabd8eb0b6 Fix broken rename behavior in tfe_client_mock.go
The delete + assign at the end of `Update` and `UpdateByID` are meant to handle
renaming a workspace — (remove old name), (insert new name).

However, `UpdateByID` was doing (remove new name), (insert new name) and leaving
the old name in place. This commit changes it to match `Update` by grabbing the
original name off the workspace object _before_ potentially renaming it.
2021-10-28 19:29:18 -05:00
Chris Arcand 50997d9a32 Restore API version checking, with internal usage accounted for 2021-10-28 19:29:18 -05:00
Nick Fagerlund cb26cbba22 Add tests for cloud.StateMgr() when TFC lacks local tf version
Alas, there's not a very good way to test the message we're supposed to print to
the console in this situation; we just don't appear to have a mock terminal that
the test can read from. But we can at least test that the function returns
without erroring under the exact conditions where it was erroring before.

Note that the behaviors of mc.Workspaces.Update and UpdateByID were already
starting to drift, so I consolidated their actual attribute update logic into a
helper function before they drifted much further.
2021-10-28 19:29:18 -05:00
Nick Fagerlund c99f5972de cloud.StateMgr(): Set terraform version AFTER creating workspace
Previously, if the remote TFC/TFE instance doesn't happen to have a tool_version
record whose name exactly matches the value of `tfversion.String()`, Terraform
would be completely blocked from using the `terraform workspace new` command
(when configured with the tags strategy) — the API would give a 422 to the
whole create request.

This commit changes the StateMgr() function to do the work in two passes; first
create the workspace (which should work fine regardless), THEN update the
Terraform version and print a warning to the terminal if it fails (which 99% of
the time is a benign failure with little impact on your future CLI usage).
2021-10-28 19:29:18 -05:00
Chris Arcand aace0015c2 cloud: Fix E2E variables test 2021-10-28 19:29:17 -05:00
Brandon Croft 64fbb0b613 remove reference to AccessInternalTools 2021-10-28 19:29:17 -05:00
Omar Ismail 1ff7827416 Multi-state migration prompt fix 2021-10-28 19:29:17 -05:00
Barrett Clark cc6de251d8 Update init reconfigure error message
If you move from the remote backend to the cloud block you will see this
error message.
2021-10-28 19:29:17 -05:00
Chris Arcand 12a4af3e72 Temporarily skip API version checking 2021-10-28 19:29:16 -05:00
Barrett Clark e16c53b561 Simplify/Consolidate the cloud conditionals 2021-10-28 19:29:16 -05:00
Barrett Clark ab304d831f Found another path to backend init error
There are actually a few different ways to get to this message.

1. Blank state — no previous terraform applied. Start with a cloud block.
1. Implicit local — start with no backend specified. This actually goes
   through the same code execution path as the first scenario.
1. Explicit local — start with a backend local block that has been
   applied, then change from the local backend to a cloud block. This
   will recognize the state, and is a different path through the code in
   the meta backend.

This commit handles the last case. The messaging has also been tweaked.

End to end test included as well.
2021-10-28 19:29:16 -05:00
Barrett Clark d29532cfeb tfc apply before init
https://app.asana.com/0/1199201948575144/1201019450474843/f
2021-10-28 19:29:16 -05:00
Barrett Clark e49f271eb5 Update copy for terraform init with cloud block 2021-10-28 19:29:16 -05:00
Omar Ismail 5ff2495ffa Update HasResource to HasManagedResourceInstanceObjects 2021-10-28 19:29:15 -05:00
Omar Ismail 86f93a2cbc Cloud UX test fixes 2021-10-28 19:29:15 -05:00
Chris Arcand 09d5b70b7a command/meta_backend_migrate:Tweak multi-to-single To TFC copy 2021-10-28 19:29:15 -05:00
Omar Ismail dfb4609be2 Backend State Migration from cloud to cloud.
* Add test for tfc to tfc mgiration
* Fix old tests, and remove unused code.
2021-10-28 19:29:15 -05:00
Barrett Clark 93bfcff61a More Terraform Cloud UX changes
* Update e2e tests to specify Terraform Cloud
* Update more of the terraform command flow to specify Terraform Cloud
2021-10-28 19:29:14 -05:00
Barrett Clark 261a2b49d3 UX for terraform init that copies state to TFC 2021-10-28 19:29:14 -05:00
Barrett Clark d04c01573c Fix iterator in a couple of e2e tests 2021-10-28 19:29:14 -05:00
Barrett Clark 0cd9490838 PR Feedback: cleanup variable setter 2021-10-28 19:29:14 -05:00
Chris Arcand f8256f6634 Update ux for "terraform init [-reconfigure]" 2021-10-28 19:29:14 -05:00
Omar Ismail dc76bbee73 Backend State Migration: Add remote backend test 2021-10-28 19:29:14 -05:00
Omar Ismail 3fedd6898c Backend State Migration to `cloud`: Multiple Workspaces
* Handle when there are multiple workspaces migrating to cloud,
using both the cloud name strategy and cloud tags strategy.
* Add e2e tests
2021-10-28 19:29:13 -05:00
Nick Fagerlund baa72ce235 Simplify logic flow: everything is a constraint
Explicit version strings are actually also version constraints! And the special
comparisons we were doing to allow a range of compatible versions can also be
expressed as version constraints.

Bonus: also simplify the way we handle version check errors, by composing the
messages inline and only extracting the repetitive parts into a function.
2021-10-28 19:29:13 -05:00
Barrett Clark 83538fdd6b Shift errors to error file, improve error text 2021-10-28 19:29:13 -05:00
Nick Fagerlund 9857097b34 Add tests for version constraints in workspace terraform-versions 2021-10-28 19:29:13 -05:00
Nick Fagerlund b43daeaa8d Cloud backend: accept version constraints from workspaces
The cloud backend (and remote before it) previously expected a TFC workspace's
`terraform-version` attribute to be either the magic string `"latest"` or an
explicit semver value. But a workspace might have a version constraint instead
(like `~> 1.1.0`), in which case the version check would blow up.

This commit checks whether `terraform-version` is a valid version constraint
before erroring out, and if so, returns success if the local version meets the
constraint.

Because it's not practical to deeply introspect the slice of version space
defined by a constraint, this check is slightly less robust than the version
comparisons below it:

- It can give a false OK on open-ended constraints like `>= 1.1.0`. Say you're
  running 1.3.0, it changed the state format, and the TFE instance admin has
  not yet added any 1.3.x Terraform versions; your workspace will now break.

- It will give a false not-OK when using different minor versions within a range
  that we know to be compatible, e.g. remote constraint of `~> 0.15.0` and local
  version of 1.1.0.

- This would be totally useless with the pre-0.14 versions of Terraform, where
  patch releases could change state format... but we're not going back in time
  to add this feature to them anyway.

Still, in the most common likely case (`~> x.y.z`), it'll complain at you (with
an error you can choose to override) if you're not using the same minor version,
and that seems proportionate, useful, and expected.
2021-10-28 19:29:13 -05:00
Barrett Clark 83337de654 Remove prefix from the cloud backend config
Now that we have tags we no longer need prefix.
2021-10-28 19:29:12 -05:00
Barrett Clark edbc84420c Run apply -refresh-state instead of refresh
When a user runs `terraform refresh` we give them an error message that
tells them to run `terraform apply -refresh-state`. We could just run
that command for them, though. That is what this PR does.
2021-10-28 19:29:12 -05:00
Brandon Croft bf02b5cb53 e2e test for cloud run variables 2021-10-28 19:29:12 -05:00
Brandon Croft 9963bf940a refactor: only fetch variables if unset variables are not allowed 2021-10-28 19:29:12 -05:00
Omar Ismail dd856f8a1b Add run variables to cloud plan operations from non-file sources
Run variables allow the run API to accept input variables not found
in the configuration slug (the file-based ones plus workspace vars)
2021-10-28 19:29:12 -05:00
Brandon Croft dbbfae5a1c refactor ParseVariableValues into separate operations
1. ParseDeclaredValues: parses unparsed variables into terraform.InputValues
2. ProbeUndeclaredVariableValues: compares variable declarations with unparsed values to warn/error about undeclared variables
2021-10-28 19:29:12 -05:00
Omar Ismail 55fc590904 Teraform Cloud Backend State Migration
* determining source or destination to cloud
* handling single to single state migrations to cloud,
using a name strategy or a tags strategy
* Add end-to-end tests for state migration.
2021-10-28 19:29:12 -05:00
Chris Arcand 7cc53fe163 cloud: Set minimum TFE version
These changes remove all of the preexisting version checking for
individual features, wiping the slate clean with an overall minimum
requirement of a future TFP-API-Version 2.5, which at the time of this
writing is expected to be TFE v202112-1.

It also actually provides that expected TFE version as an actionable
error message, rather than generically saying that it isn't supported or
using the somewhat opaque API version header.
2021-10-28 19:29:12 -05:00
Chris Arcand 46e47ed379 Use the 'tfe' service for everything.
The 'tfe' service was appended to with various versions to denote a new
'feature' implemented by a new 'service'. This quickly proved to not be
scalable, as adding an entry to the discovery document from every
feature is bad.

The new mechanism added was checking the TFP-API-Version header on
requests for a version, instead.

So we'll remove the separation here between different tfe service
'versions' and the separate 'state' service and Just Use TFE, as well as
the TFP-API-Version header for all feature versioning., as well as the
TFP-API-Version header for all feature versioning.
2021-10-28 19:29:12 -05:00
Chris Arcand 412679522d Remove all constraint checking
This is an outdated mechanism that isn't used anymore.
2021-10-28 19:29:11 -05:00
Chris Arcand a4e196d067 Port backend/local: Check dependency lock consistency before any operations
This is a port of the changes made in #29683
2021-10-28 19:29:11 -05:00
Omar Ismail d5567b479c Fix terraform version in cloud e2e tests to ignore prerelease. (#29) 2021-10-28 19:29:11 -05:00
Omar Ismail a387af6c61 Add auto-approve logic, e2e tests 2021-10-28 19:29:11 -05:00
Chris Arcand ee384e8716 Attempt to set current Terraform version on new workspaces, always 2021-10-28 19:29:11 -05:00
Chris Arcand aba7d96596 cloud: Support interop from 0.14 to 1.1
The previous conservative guarantee that we would not make backwards
incompatible changes to the state file format until at least Terraform
1.1 can now be extended. Terraform 0.14 through 1.1 will be able to
interoperably use state files, so we can update the remote backend
version compatibility check accordingly.

This is a port of https://github.com/hashicorp/terraform/pull/29645
2021-10-28 19:29:11 -05:00
Chris Arcand 74e087dc29 Rename cloud.testBackendDefault -> cloud.testBackendWithName 2021-10-28 19:29:11 -05:00
Chris Arcand d2b6b5f2b3 cloud: Align local and remote workspace name with 'name' strategy
This changes the 'name' strategy to always align the local configured
workspace name and the remote Terraform Cloud workspace, rather than the
implicit use of the 'default' unnamed workspace being used instead.

What this essentially means is that the Cloud integration does not fully
support workspaces when configured for a single TFC workspace (as was
the case with the 'remote' backend), but *does* use the
backend.Workspaces() interface to allow for normal local behaviors like
terraform.workspace to resolve to the correct name. It does this by
always setting the local workspace name when the 'name' strategy is
used, as a part of initialization.

Part of the diff here is exporting all the previously unexported types
for mapping workspaces. The command package (and init in particular)
needs to be able to handle setting the local workspace in this
particular scenario.
2021-10-28 19:29:11 -05:00
Omar Ismail a94b2405b1 static checker 2021-10-28 19:29:11 -05:00
Omar Ismail 594a390f84 Use context parallelism 2021-10-28 19:29:10 -05:00
Chris Arcand 1791b71196 cloud: TestCloud_backendWithTags
Implementing this test was quite a rabbithole, as in order to satisfy
backendTestBackendStates() the workspaces returned from
backend.Workspaces() must match exactly, and the shortcut taken to test
pagination in 3cc58813f0 created an
impossible circumstance that got plastered over with the fact that
prefix filtering is done clientside, not by the API as it should be.

Tagging does not rely on clientside filtering, and expects that the
request made to the TFC API returns exactly those workspaces with the
given tags.

These changes include a better way to test pagination, wherein we
actually create over a page worth of valid workspaces in the mock client
and implement a simplified pagination behavior to match how the TFC API
actually works.
2021-10-28 19:29:10 -05:00
Chris Arcand a97a1c8f66 cloud.MockClient: Use Contains() to implement Search
The TFC API doesn't behave this way; a search term just signals the
substring was found within the name.
2021-10-28 19:29:10 -05:00
Chris Arcand 9bdf1d2579 Rename testBackendNoDefault
"NoDefault" is now ambiguous with tags, and it does not imply using a
prefix.
2021-10-28 19:29:10 -05:00
Chris Arcand 6dcd0db265 Map CLI workspaces by TFC tags 2021-10-28 19:29:10 -05:00
Chris Arcand 7a243379fb cloud: Refactor workspaceMapping concerns into strategy() 2021-10-28 19:29:10 -05:00
Chris Arcand 922a8e4488 Refactor private workspace fields into workspaceMapping
A mostly cosemetic change; The fields 'workspace' and 'prefix' don't
really describe well what they are from a caller, so change these to use
a workspaceMapping struct to convey they are for implementing workspace
mapping strategies from CLI -> TFC
2021-10-28 19:29:10 -05:00
Omar Ismail 0a125c0400 Cloud: refactor setting attributes, add tests 2021-10-28 19:29:10 -05:00
Omar Ismail 8f49757f89 Add testing for Cloud PrepareConfig 2021-10-28 19:29:10 -05:00
Chris Arcand 471dd479e8 Update go-tfe to 26689e
These changes include additions to fulfill the interface for the client
mock, plus moving all that logic (which needn't be duplicated across
both the remote and cloud packages) over to the cloud package under a
dedicated mock client file.
2021-10-28 19:29:10 -05:00
Chris Arcand 34597c237f cloud: Don't ignore .log golden files 2021-10-28 19:29:10 -05:00
Chris Arcand 85f37f1aa5 Add missing log mocks
I missed adding these in the original porting commit from the remote
backend, because *.log is added to gitignore.
2021-10-28 19:29:10 -05:00
Martin Atkins f4758803fe cloud: Backend implements new version of the backend.Local interface
This makes the new backend compatible with the new terraform.Context API,
which has changed in main.
2021-10-28 19:29:09 -05:00
Chris Arcand 18d54c1129 Allow cloud block overrides
These changes allow cloud blocks to be overridden by backend blocks and
vice versa; the logic follows the current backend behavior of a block
overriding a preceding block in full, with no merges.
2021-10-28 19:29:09 -05:00
Chris Arcand 07b3d015d1 Defer cloud block overrides
This restriction is temporary. Overrides should be allowed, but have the
added complexity of needing to also override a 'backend' block, so this
work is being deferred for now.
2021-10-28 19:29:09 -05:00
Chris Arcand fccc873a3d Remove ability to declare a 'cloud' backend
With the alternative block introduced in 7bf9b2c7b, this removes the
ability to explicitly declare the 'cloud' backend. The literal backend
interface is an implementation detail and no longer a user-level
concept when using Terraform Cloud.
2021-10-28 19:29:09 -05:00
Chris Arcand a4c24e3147 Add cloud {} configuration block for Terraform Cloud
This is a replacement declaration for using Terraform Cloud as a remote
backend, leaving the literal backend as an implementation detail and not
a user-level concept.
2021-10-28 19:29:09 -05:00
Chris Arcand a5f063625a Replace generic host error messages for feature support 2021-10-28 19:29:09 -05:00
Chris Arcand ea8ad0b15a Initial commit of 'cloud' package
The cloud package intends to implement a new integration for
Terraform Cloud/Enterprise. The purpose of this integration is to better
support TFC users; it will shed some overly generic UX and architecture,
behavior changes that are otherwise backwards incompatible in the remote
backend, and technical debt - all of which are vestiges from before
Terraform Cloud existed.

This initial commit is largely a porting of the existing 'remote'
backend, which will serve as an underlying implementation detail and not
be a typical user-level backend. This is because to re-implement the
literal backend interface is orthogonal to the purpose of this
integration, and can always be migrated away from later.

As this backend is considered an implementation detail, it will not be
registered as a declarable backend. Within these changes it is, for easy
of initial development and a clean diff.
2021-10-28 19:29:09 -05:00
James Bardin 9b62dd6901
Merge pull request #29648 from hashicorp/jbardin/tfproto6.1
tfproto version 6.1
2021-10-28 16:00:02 -04:00