Commit Graph

52 Commits

Author SHA1 Message Date
Alisdair McDiarmid 536c80da23 backend: Add per-operation diagnostic rendering
The enhanced backends (local and remote) need to be able to render
diagnostics during operations. Prior to this commit, this functionality
was supported with a per-backend `ShowDiagnostics` function pointer.

In order to allow users of these backends to control how diagnostics are
rendered, this commit moves that function pointer to the `Operation`
type. This means that a diagnostic renderer is configured for each
operation, rather than once per backend initialization.

Some secondary consequences of this change:

- The `ReportResult` method on the backend is now moved to the
  `Operation` type, as it needs to access the `ShowDiagnostics` callback
  (and nothing else from the backend);
- Tests which assumed that diagnostics would be written to the backend's
  `cli.Ui` instance are migrated to using a new record/playback diags
  helper function;
- Apply, plan, and refresh commands now pass a pointer to the `Meta`
  struct's `showDiagnostics` method.

This commit should not change how Terraform works, and is refactoring in
preparation for more changes which move UI code out of the backend.
2021-02-12 14:30:35 -05:00
Alisdair McDiarmid 7e09cd1228 backend/remote: No version check for local ops
If the remote backend is connected to a Terraform Cloud workspace in
local operations mode, we disable the version check, as the remote
Terraform version is meaningless.
2021-01-05 09:11:19 -05:00
Alisdair McDiarmid 619c6727ef backend/remote: No version check for operations
Terraform remote version conflicts are not a concern for operations. We
are in one of three states:

- Running remotely, in which case the local version is irrelevant;
- Workspace configured for local operations, in which case the remote
  version is meaningless;
- Forcing local operations with a remote backend, which should only
  happen in the Terraform Cloud worker, in which case the Terraform
  versions by definition match.

This commit therefore disables the version check for operations (plan
and apply), which has the consequence of disabling it in Terraform Cloud
and Enterprise runs. In turn this enables Terraform Enterprise runs with
bundles which have a version that doesn't exactly match the bundled
Terraform version.
2020-12-17 12:58:38 -05:00
Alisdair McDiarmid 6e0d8cde91 backend/remote: Fix for "latest" workspace version
Terraform Cloud/Enterprise support a pseudo-version of "latest" for the
configured workspace Terraform version. If this is chosen, we abandon
the attempt to verify the versions are compatible, as the meaning of
"latest" cannot be predicted.

This affects both the StateMgr check (used for commands which execute
remotely) and the full version check (for local commands).
2020-12-08 15:34:23 -05:00
Alisdair McDiarmid c5c1f31db3 backend: Validate remote backend Terraform version
When using the enhanced remote backend, a subset of all Terraform
operations are supported. Of these, only plan and apply can be executed
on the remote infrastructure (e.g. Terraform Cloud). Other operations
run locally and use the remote backend for state storage.

This causes problems when the local version of Terraform does not match
the configured version from the remote workspace. If the two versions
are incompatible, an `import` or `state mv` operation can cause the
remote workspace to be unusable until a manual fix is applied.

To prevent this from happening accidentally, this commit introduces a
check that the local Terraform version and the configured remote
workspace Terraform version are compatible. This check is skipped for
commands which do not write state, and can also be disabled by the use
of a new command-line flag, `-ignore-remote-version`.

Terraform version compatibility is defined as:

- For all releases before 0.14.0, local must exactly equal remote, as
  two different versions cannot share state;
- 0.14.0 to 1.0.x are compatible, as we will not change the state
  version number until at least Terraform 1.1.0;
- Versions after 1.1.0 must have the same major and minor versions, as
  we will not change the state version number in a patch release.

If the two versions are incompatible, a diagnostic is displayed,
advising that the error can be suppressed with `-ignore-remote-version`.
When this flag is used, the diagnostic is still displayed, but as a
warning instead of an error.

Commands which will not write state can assert this fact by calling the
helper `meta.ignoreRemoteBackendVersionConflict`, which will disable the
checks. Those which can write state should instead call the helper
`meta.remoteBackendVersionCheck`, which will return diagnostics for
display.

In addition to these explicit paths for managing the version check, we
have an implicit check in the remote backend's state manager
initialization method. Both of the above helpers will disable this
check. This fallback is in place to ensure that future code paths which
access state cannot accidentally skip the remote version check.
2020-11-19 13:19:40 -05:00
Kristin Laemmert 6621501ae3
state: remove deprecated state package (#25490)
Most of the state package has been deprecated by the states package.
This PR replaces all the references to the old state package that
can be done simply - the low-hanging fruit.

* states: move state.Locker to statemgr

The state.Locker interface was a wrapper around a statemgr.Full, so
moving this was relatively straightforward.

* command: remove unnecessary use of state package for writing local terraform state files

* move state.LocalState into terraform package

state.LocalState is responsible for managing terraform.States, so it
made sense (to me) to move it into the terraform package.

* slight change of heart: move state.LocalState into clistate instead of
terraform
2020-08-11 11:43:01 -04:00
Kristin Laemmert 86e9ba3d65
* backend/local: push responsibility for unlocking state into individual operations
* unlock the state if Context() has an error, exactly as backend/remote does today
* terraform console and terraform import will exit before unlocking state in case of error in Context()
* responsibility for unlocking state in the local backend is pushed down the stack, out of backend.go and into each individual state operation
* add tests confirming that state is not locked after apply and plan

* backend/local: add checks that the state is unlocked after operations

This adds tests to plan, apply and refresh which validate that the state
is unlocked after all operations, regardless of exit status. I've also
added specific tests that force Context() to fail during each operation
to verify that locking behavior specifically.
2020-08-11 11:23:42 -04:00
James Bardin 35d232481f prevent panic in remote backend retry
Ensure that the *http.Response is not nil before checking the status.
2020-06-22 15:02:23 -04:00
Kristin Laemmert e6cf6cd758
backend/remote: do not panic if PrepareConfig or Configure receive null (#25135)
* backend/remote: do not panic if PrepareConfig or Configure receive null
objects

If a user cancels (ctrl-c) terraform init while it is requesting missing
configuration options for the remote backend, the PrepareConfig and
Configure functions would receive a null cty.Value which would result in
panics. This PR adds a check for null objects to the two functions in
question.

Fixes #23992
2020-06-05 09:11:44 -04:00
Kritonas 71395a0656 Fixed minor grammatical errors 2020-05-04 17:34:50 +03:00
Alisdair McDiarmid f15e58adf8 Improve remote backend missing token error
Prompt the user to run terraform login to generate and store a token for
the configured remote backend.
2020-02-05 13:25:29 -05:00
Radek Simko 32f9722d9d
Replace import paths & set UA string where necessary 2019-10-11 22:40:54 +01:00
Kristin Laemmert 412d459292
backend/remote: remove milseading contents from error message (#22148)
Previously, terraform was returning a potentially-misleading error
message in response to anything other than a 404 from the
b.client.Workspaces.Read operation. This PR simplifies Terraform's error
message with the intent of encouraging those who encounter it to focus
on the error message returned from the tfe client.

The added test is odd, and a bit hacky, and possibly overkill.
2019-07-22 09:06:39 -04:00
Sander van Harmelen 4628fbcc65
Merge pull request #20615 from hashicorp/svh/f-input-context
core: add a context to the UIInput interface
2019-03-08 18:44:28 +01:00
Sander van Harmelen 973e2a7cf9 core: add a context to the UIInput interface 2019-03-08 10:24:40 +01:00
Sander van Harmelen 0232d84a0d backend/remote: also retry on server errors
Enably retrying on server errors in the updated `go-tfe` client and add a retry log hook for writing retry messages to the CLI.
2019-03-06 13:36:06 +01:00
Sander van Harmelen 63e2dcef8a
Merge pull request #20481 from hashicorp/svh/b-exit-code
backend/remote: exit with 1 when a run is canceled
2019-02-27 07:52:20 +01:00
Sander van Harmelen 01f17fa0ca backend/remote: exit with 1 when a run is canceled 2019-02-26 21:00:07 +01:00
James Bardin c814f2da37 Change backend.ValidateConfig to PrepareConfig
This mirrors the change made for providers, so that default values can
be inserted into the config by the backend implementation. This is only
the interface and method name changes, it does not yet add any default
values.
2019-02-25 18:37:20 -05:00
Sander van Harmelen 54736b068b backend/remote: use `state.v2` for remote state only
The API surface area is much smaller when we use the remote backend for remote state only.

So in order to try and prevent any backwards incompatibilities when TF runs inside of TFE, we’ve split up the discovery services into `state.v2` (which can be used for remote state only configurations, so when running in TFE) and `tfe.v2.1` (which can be used for all remote configurations).
2019-02-19 10:59:51 +01:00
Sander van Harmelen 47a00ea34b backend/remote: cleanup test connections
Cleanup test connection to prevent file descriptor issues when running the tests on a Mac.
2019-02-07 09:55:19 +01:00
Sander van Harmelen 1e4c20686e backend/remote: make sure we show the correct error
Previously we would show two errors when there was a version constraint
error. But of course one is enough.
2019-01-23 15:09:42 +01:00
Sander van Harmelen 8937fedb76 backend/remote: fix a small typo 2019-01-18 19:40:57 +01:00
Sander van Harmelen 7b51af72b2 backend/remote: compare versions without the prerelease 2018-12-19 19:06:22 +01:00
Sander van Harmelen 55b6153b04 backend/remote: fix an error that prevents checking constraints 2018-12-15 21:36:47 +01:00
Sander van Harmelen 8f04e93739 backend/remote: return detailed incompatibility info 2018-12-14 21:11:41 +01:00
Sander van Harmelen 268c0f85ce Add a method to retrieve version contraints 2018-12-14 12:17:31 +01:00
Sander van Harmelen a5a2156584 core: enhance service discovery
This PR improves the error handling so we can provide better feedback about any service discovery errors that occured.

Additionally it adds logic to test for specific versions when discovering a service using `service.vN`. This will enable more informational errors which can indicate any version incompatibilities.
2018-12-10 20:52:05 +01:00
Sander van Harmelen 9062d887b8 backend/remote: use entitlements to select backends
Use the entitlements to a) determine if the organization exists, and b) as a means to select which backend to use (the local backend with remote state, or the remote backend).
2018-12-05 12:29:08 +01:00
Sander van Harmelen fe05609c5e backend/remote: support the new force-unlock API
Add support for the new `force-unlock` API and at the same time improve
performance a bit by reducing the amount of API calls made when using
the remote backend for state storage only.
2018-11-30 19:39:18 +01:00
Sander van Harmelen 35d9ce3f92 backend/remote: implement the Local interface 2018-11-26 20:50:25 +01:00
Sander van Harmelen a17f317025 Change how to fall back from remote to local backend
In order to support free organizations, we need a way to load the `remote` backend and then, depending on the used offering/plan, enable or disable remote operations.

In other words, we should be able to dynamically fall back to the `local` backend if needed, after first configuring the `remote` backend.

To make this works we need to change the way this was done previously when the env var `TF_FORCE_LOCAL_BACKEND` was set. The clear difference of course being that the env var would be available on startup, while the used offering/plan is only known after being able to connect to TFE.
2018-11-20 22:25:52 +01:00
Sander van Harmelen 52a1b22f7a Implement the remote enhanced backend
This is a refactored version of the `remote` backend that was initially added to Terraform v0.11.8 which should now be compatible with v0.12.0.
2018-11-06 16:29:46 +01:00
Martin Atkins 541952bb8f Revert some work that happened since v0.12-dev branched
This work was done against APIs that were already changed in the branch
before work began, and so it doesn't apply to the v0.12 development work.

To allow v0.12 to merge down to master, we'll revert this work out for now
and then re-introduce equivalent functionality in later commits that works
against the new APIs.
2018-10-16 19:48:28 -07:00
Sander van Harmelen 8875fa660f Make sure we also output policies while planning 2018-10-16 17:16:28 +02:00
Sander van Harmelen 775f8a9626 Make sure we always set a custom header
This is for TFE to recognize were the calls come from.
2018-10-15 20:33:42 +02:00
Sander van Harmelen b1fdbd7db8 Allow enhanced backends to pass custom exit codes
In some cases this is needed to keep the UX clean and to make sure any remote exit codes are passed through to the local process.

The most obvious example for this is when using the "remote" backend. This backend runs Terraform remotely and stream the output back to the local terminal.

When an error occurs during the remote execution, all the needed error information will already be in the streamed output. So if we then return an error ourselves, users will get the same errors twice.

By allowing the backend to specify the correct exit code, the UX remains the same while preserving the correct exit codes.
2018-10-05 20:44:12 +02:00
Sander van Harmelen 67db9da000 Add checks for all flags we currently don’t support
For Plan only:
-module-depth=n

For Plan & Apply
-parallelism=m
-refresh=false
-var “foo=bar” and -var-file=foo
2018-10-05 20:16:34 +02:00
Sander van Harmelen a2241e7c43 backend/remote: introduce support for `-no-color`
This is a bit of a hack to support the `-no-color` flag while we don’t have an option to set run variables.

That is also the reason why the orginal method is commented out instead of deleted. This will be reverted when the TFE starts supporting run variables.
2018-10-04 18:01:11 +02:00
Sander van Harmelen 3979aec0ae Ask to cancel a pending remote operation
Except when a lock-timeout has exceeded or auto-approve is set.
2018-10-04 17:16:45 +02:00
Sander van Harmelen b28f47055d backend/remote: extend mocks and add sentinel tests 2018-09-26 22:34:32 +02:00
Sander van Harmelen 2bd1040bbd backend/remote: extend mocks and add apply tests 2018-09-26 21:35:41 +02:00
Sander van Harmelen 9f9bbcb0e7 backend/remote: lots of improvements
This commit adds:

- support for `-lock-timeout`
- custom error message when a 404 is received
- canceling a pending run when TF is Ctrl-C’ed
- discard a run when the apply is not approved
2018-09-22 11:49:42 +02:00
Sander van Harmelen 621d589189 backend/remote: add support for the apply operation 2018-09-22 11:49:42 +02:00
Sander van Harmelen 4aeb67b8ff backend/remote: handle empty results correctly
The pagination info of a list call that returns an empty list contains:

```go
CurrentPage: 1
TotalPages: 0
```

So checking if we have seen all pages using `CurrentPage == TotalPages` will not work and will result in an endless loop.

The tests are updated so they will fail (timeout after 1m) if this is handled incorreclty.
2018-09-17 16:22:54 +02:00
Sander van Harmelen 7c1da2ffaa
Merge pull request #18818 from hashicorp/b-associate-state
backend/remote: add the run ID to associate state
2018-09-10 19:13:53 +02:00
Sander van Harmelen cd6d75bc03 backend/remote: add the run ID to associate state
If a run ID is available, we need to make sure we pass that when creating a new state version so the state will be properly associated with the run.
2018-09-09 21:30:42 +02:00
Sander van Harmelen 3cc58813f0 backend/remote: use a search query and use pagination
To prevent making unnecessary heavy calls to the backend, we should use a search query to limit the result.

But even if we use a search query, we should still use the pagination details to make sure we retrieved all items.
2018-09-09 21:29:50 +02:00
Sander van Harmelen 62ab84a319 backend/remote: compat changes for latest `go-tfe` 2018-09-08 11:47:18 +02:00
Sander van Harmelen 7049d973a9 Do not use the TFE_TOKEN env variable
Instead promote the use of shared credentials using the CLI Config File
2018-08-10 20:20:16 +02:00