Commit Graph

721 Commits

Author SHA1 Message Date
James Bardin 1338502c7b
Merge pull request #26924 from remilapeyre/concurrent-locks-pg
Use a global sequence to create the IDs for each workspace
2021-03-16 11:28:04 -04:00
Alisdair McDiarmid 4b159416ff backend/remote: Fix new workspace state migration
When migrating state to a new workspace, the version check would error
due to a 404 error on fetching the workspace record. This would result
in failed state migration.

Instead we should look specifically for a 404 error, and allow migration
to continue. If we're just about to create the workspace, there can't be
a version incompatibility problem.
2021-03-15 15:48:14 -04:00
Omar Ismail e9c7f37b8c
Enable --auto-approve for Policy checks on Remote Backend (Terraform Cloud) (#27804)
* Fix auto-approve for soft-policy
* Update error handling
* update testing string for consistency
2021-03-01 08:54:30 -05:00
Alisdair McDiarmid 37fff9336a
Merge pull request #27929 from hashicorp/alisdair/command-views-cleanup
Some command views cleanup
2021-02-25 16:26:08 -05:00
Alisdair McDiarmid 7c0ec0107f backend: Replace ShowDiagnostics with view.Diagnostics 2021-02-25 11:26:05 -05:00
James Bardin 936f597ba1 check errors before using configSnap
configSnap may be nil if there are errors loading it from the plan file.
2021-02-25 09:54:02 -05:00
Alisdair McDiarmid 02eb283ad4 backend/remote: Fix broken state lock retry
When using the -lock-timeout option with the remote backend configured
in local operations mode, Terraform would fail to retry acquiring the
lock. This was caused by the lock error message having a missing Info
field, which the state manager requires to be present in order to
attempt retries.
2021-02-19 15:47:18 -05:00
Alisdair McDiarmid 68558ccd54 backend/local: Replace CLI with view instance
This commit extracts the remaining UI logic from the local backend,
and removes access to the direct CLI output. This is replaced with an
instance of a `views.Operation` interface, which codifies the current
requirements for the local backend to interact with the user.

The exception to this at present is interactivity: approving a plan
still depends on the `UIIn` field for the backend. This is out of scope
for this commit and can be revisited separately, at which time the
`UIOut` field can also be removed.

Changes in support of this:

- Some instances of direct error output have been replaced with
  diagnostics, most notably in the emergency state backup handler. This
  requires reformatting the error messages to allow the diagnostic
  renderer to line-wrap them;
- The "in-automation" logic has moved out of the backend and into the
  view implementation;
- The plan, apply, refresh, and import commands instantiate a view and
  set it on the `backend.Operation` struct, as these are the only code
  paths which call the `local.Operation()` method that requires it;
- The show command requires the plan rendering code which is now in the
  views package, so there is a stub implementation of a `views.Show`
  interface there.

Other refactoring work in support of migrating these commands to the
common views code structure will come in follow-up PRs, at which point
we will be able to remove the UI instances from the unit tests for those
commands.
2021-02-18 12:08:08 -05:00
Pam Selle 8d2fbd4cd8
Remove deprecation on undeclared variable, docs, and summary adjustment (#27795)
* Remove deprecation on undeclared variable

Remove deprecation and add docs specific to the behavior around
undeclared variable values

* Limit full warnings to 2 instances, then summary

This way, the third warning is a summary, rather than the fourth
warning being the summary
2021-02-18 11:11:52 -05:00
Kristin Laemmert f6505870cc
Mildwonkey/providers interface renaming (#27805)
* providers.Interface: huge renamification

This commit renames a handful of functions in the providers.Interface to
match changes made in protocol v6. The following commit implements this
change across the rest of the codebase; I put this in a separate commit
for ease of reviewing and will squash these together when merging.

One noteworthy detail: protocol v6 removes the config from the
ValidateProviderConfigResponse, since it's never been used. I chose to
leave that in place in the interface until we deprecate support for
protocol v5 entirely.

Note that none of these changes impact current providers using protocol
v5; the protocol is unchanged. Only the translation layer between the
proto and terraform have changed.
2021-02-18 10:13:43 -05:00
Matthew Frahry 85b9bdea96
backend/azure: azure state refreshes outside of grabbing the lock #26561 2021-02-17 15:14:47 -08:00
James Bardin b0094dbf36
Merge pull request #27678 from GreenHedgehog/master
Fix possible nil pointer dereference in azure sdk
2021-02-17 08:49:49 -05:00
Matthew Frahry 81dc028689
backend/azurerm: updating the dependencies for the Azure Backend #26721 2021-02-16 23:24:56 -08:00
Alisdair McDiarmid 6375c6ce6b
Merge pull request #27787 from hashicorp/alisdair/command-views-state-locker
clistate: Update clistate.Locker for command views
2021-02-16 17:37:18 -05:00
Alisdair McDiarmid 1ae3d30383
Merge pull request #27760 from hashicorp/alisdair/command-views-ui-hook
cli: Migrate Terraform UI hook to command views
2021-02-16 09:36:47 -05:00
Alisdair McDiarmid 3257f31aa7 backend/local: Return diag for refresh empty state
The warning diag added when refreshing an empty state file was never
rendered, and instead a custom (and incorrect) warning was output to the
UI. This commit fixes the dropped diag and removes the custom warning.
2021-02-16 07:23:31 -05:00
Alisdair McDiarmid 2d1976bbda clistate: Update clistate.Locker for command views
The clistate package includes a Locker interface which provides a simple
way for the local backend to lock and unlock state, while providing
feedback to the user if there is a delay while waiting for the lock.
Prior to this commit, the backend was responsible for initializing the
Locker, passing through direct access to the cli.Ui instance.

This structure prevented commands from implementing different
implementations of the state locker UI. In this commit, we:

- Move the responsibility of creating the appropriate Locker to the
  source of the Operation;
- Add the ability to set the context for a Locker via a WithContext
  method;
- Replace the Locker's cli.Ui and Colorize members with a StateLocker
  view;
- Implement views.StateLocker for human-readable UI;
- Update the Locker interface to return detailed diagnostics instead of
  errors, reducing its direct interactions with UI;
- Add a Timeout() method on Locker to allow the remote backend to
  continue to misuse the -lock-timeout flag to cancel pending runs.

When an Operation is created, the StateLocker field must now be
populated with an implementation of Locker. For situations where locking
is disabled, this can be a no-op locker.

This change has no significant effect on the operation of Terraform,
with the exception of slightly different formatting of errors when state
locking or unlocking fails.
2021-02-16 07:19:22 -05:00
Alisdair McDiarmid a7b7cd29fc cli: Migrate Terraform UI hook to command views
Move the code which renders Terraform hook callbacks as UI into the
views package, backed by a views.View instead of a cli.Ui. Update test
setup accordingly.

To allow commands to control this hook, we add a hooks member on the
backend Operation struct. This supersedes the hooks in the Terraform
context, which is not directly controlled by the command logic.

This commit should not change how Terraform works, and is refactoring in
preparation for more changes which move UI code out of the backend.
2021-02-16 07:18:22 -05:00
Matthew Frahry cb041053e9 Merge with master 2021-02-15 14:37:40 -08:00
Matthew Frahry 2b42445454
Update client.go 2021-02-15 12:09:31 -08:00
Matthew Frahry f52a18b134
Update backend/remote-state/azure/client.go
Co-authored-by: Tom Harvey <tombuildsstuff@users.noreply.github.com>
2021-02-15 12:04:07 -08:00
Alisdair McDiarmid 536c80da23 backend: Add per-operation diagnostic rendering
The enhanced backends (local and remote) need to be able to render
diagnostics during operations. Prior to this commit, this functionality
was supported with a per-backend `ShowDiagnostics` function pointer.

In order to allow users of these backends to control how diagnostics are
rendered, this commit moves that function pointer to the `Operation`
type. This means that a diagnostic renderer is configured for each
operation, rather than once per backend initialization.

Some secondary consequences of this change:

- The `ReportResult` method on the backend is now moved to the
  `Operation` type, as it needs to access the `ShowDiagnostics` callback
  (and nothing else from the backend);
- Tests which assumed that diagnostics would be written to the backend's
  `cli.Ui` instance are migrated to using a new record/playback diags
  helper function;
- Apply, plan, and refresh commands now pass a pointer to the `Meta`
  struct's `showDiagnostics` method.

This commit should not change how Terraform works, and is refactoring in
preparation for more changes which move UI code out of the backend.
2021-02-12 14:30:35 -05:00
Omar Ismail 4ac095b63f only print horizontal rule at first attempt 2021-02-10 20:47:50 -05:00
Omar Ismail d96c90b6af Cost Estimation CLI output unnecessary 2021-02-10 11:17:05 -05:00
Alisdair McDiarmid b2ba650c21 cli: Remove deprecated destroy -force flag
This dramatically simplifies the logic around auto-approve, which is
nice.

Also add test coverage for the manual approve step, for both apply and
destroy, answering both yes and no.
2021-02-03 15:05:05 -05:00
GreenHedgehog d4db1d1342 fix possible nil pointer dereference in azure sdk 2021-02-03 21:09:05 +03:00
Alisdair McDiarmid 5ca118b4e6 cli: Move resource count code to command package
CountHook is an implementation of terraform.Hook which is used to
calculate how many resources were added, changed, or destroyed during an
apply. This hook was previously injected in the local backend code,
which means that the apply command code has no access to these counts.

This commit moves the CountHook code into the command package, and
removes an unused instance of the hook in the plan code path. The goal
here is moving UI code into the command package.
2021-01-29 15:29:35 -05:00
Jonathan Hall 49ee3d3ef8 Grammar nit: "setup" as a verb should be spelled "set up" 2021-01-26 20:39:11 +01:00
Nick Fagerlund ed7f97c34f Update language docs URLs in code and tests 2021-01-22 12:22:21 -08:00
James Bardin 9f2a6d33be move remaining helper packages to internal 2021-01-20 13:54:00 -05:00
Martin Atkins e6a516d87e backend/local: Use terminal properties to tweak the plan output
We now require the output to accept UTF-8 and we can determine how wide
the terminal (if any) is, so here we begin to make use of that for the
"terraform plan" command.

The horizontal rule is now made of box drawing characters instead of
hyphens and fills the whole terminal width.

The paragraphs of text in the output are now also wrapped to fill the
terminal width, instead of the hard-wrapping we did before.

This is just a start down the road of making better use of the terminal
capabilities. Lots of other commands could benefit from updates like these
too.
2021-01-13 15:37:04 -08:00
Martin Atkins d2c3403ab6 command: Use the new terminal.Streams object
Here we propagate in the initialized terminal.Streams from package main,
and then onwards to backends running in CLI mode.

This also replaces our use of helper/wrappedstreams to determine whether
stdin is a terminal or a pipe. helper/wrappedstreams returns incorrect
file descriptors on Windows, causing StdinPiped to always return false on
that platform and thus causing one of the odd behaviors discussed in

Finally, this includes some wrappers around the ability to look up the
number of columns in the terminal in preparation for use elsewhere. These
wrappers deal with the fact that our unit tests typically won't populate
meta.Streams.
2021-01-13 15:37:04 -08:00
James Bardin 3bc7d77230 update MockProvider usage 2021-01-12 17:47:55 -05:00
James Bardin e614fb9aed refresh is expected for destroy
These tests were not previously running a refresh, and hence did not
expect the resources to be read.
2021-01-08 13:29:54 -05:00
Pam Selle c9f372a62b
Merge pull request #24896 from bendrucker/validate-ignore-empty-provider
validate: ignore providers with no configuration
2021-01-07 15:30:57 -05:00
Alisdair McDiarmid 7e09cd1228 backend/remote: No version check for local ops
If the remote backend is connected to a Terraform Cloud workspace in
local operations mode, we disable the version check, as the remote
Terraform version is meaningless.
2021-01-05 09:11:19 -05:00
Alisdair McDiarmid 619c6727ef backend/remote: No version check for operations
Terraform remote version conflicts are not a concern for operations. We
are in one of three states:

- Running remotely, in which case the local version is irrelevant;
- Workspace configured for local operations, in which case the remote
  version is meaningless;
- Forcing local operations with a remote backend, which should only
  happen in the Terraform Cloud worker, in which case the Terraform
  versions by definition match.

This commit therefore disables the version check for operations (plan
and apply), which has the consequence of disabling it in Terraform Cloud
and Enterprise runs. In turn this enables Terraform Enterprise runs with
bundles which have a version that doesn't exactly match the bundled
Terraform version.
2020-12-17 12:58:38 -05:00
Ben Drucker a39273cfa3 Merge remote-tracking branch 'origin/master' into validate-ignore-empty-provider 2020-12-14 14:39:48 -08:00
James Bardin 7eac9e1d89 set SkipRefresh for plan and apply
The option is set in the same place for plan and apply.
2020-12-10 09:47:13 -05:00
Alisdair McDiarmid 6e0d8cde91 backend/remote: Fix for "latest" workspace version
Terraform Cloud/Enterprise support a pseudo-version of "latest" for the
configured workspace Terraform version. If this is chosen, we abandon
the attempt to verify the versions are compatible, as the meaning of
"latest" cannot be predicted.

This affects both the StateMgr check (used for commands which execute
remotely) and the full version check (for local commands).
2020-12-08 15:34:23 -05:00
Ben Drucker 2549e53aed add backend refresh test with provider config 2020-12-06 10:02:26 -08:00
Ben Drucker 7e11b97923 Merge remote-tracking branch 'origin/master' into validate-ignore-empty-provider 2020-12-06 09:47:24 -08:00
James Bardin 96436f526a unused err value and nil context
Incorrect err assignment had no effect.

Never use a nil context.
2020-12-02 13:59:19 -05:00
James Bardin d2c2d58f09 unused 2020-12-02 13:59:19 -05:00
James Bardin de5b022a3b legacy types in backend/local 2020-12-02 12:33:18 -05:00
James Bardin 2635b3b023 update remote state to use legacy types 2020-12-02 12:16:35 -05:00
Alisdair McDiarmid c5c1f31db3 backend: Validate remote backend Terraform version
When using the enhanced remote backend, a subset of all Terraform
operations are supported. Of these, only plan and apply can be executed
on the remote infrastructure (e.g. Terraform Cloud). Other operations
run locally and use the remote backend for state storage.

This causes problems when the local version of Terraform does not match
the configured version from the remote workspace. If the two versions
are incompatible, an `import` or `state mv` operation can cause the
remote workspace to be unusable until a manual fix is applied.

To prevent this from happening accidentally, this commit introduces a
check that the local Terraform version and the configured remote
workspace Terraform version are compatible. This check is skipped for
commands which do not write state, and can also be disabled by the use
of a new command-line flag, `-ignore-remote-version`.

Terraform version compatibility is defined as:

- For all releases before 0.14.0, local must exactly equal remote, as
  two different versions cannot share state;
- 0.14.0 to 1.0.x are compatible, as we will not change the state
  version number until at least Terraform 1.1.0;
- Versions after 1.1.0 must have the same major and minor versions, as
  we will not change the state version number in a patch release.

If the two versions are incompatible, a diagnostic is displayed,
advising that the error can be suppressed with `-ignore-remote-version`.
When this flag is used, the diagnostic is still displayed, but as a
warning instead of an error.

Commands which will not write state can assert this fact by calling the
helper `meta.ignoreRemoteBackendVersionConflict`, which will disable the
checks. Those which can write state should instead call the helper
`meta.remoteBackendVersionCheck`, which will return diagnostics for
display.

In addition to these explicit paths for managing the version check, we
have an implicit check in the remote backend's state manager
initialization method. Both of the above helpers will disable this
check. This fallback is in place to ensure that future code paths which
access state cannot accidentally skip the remote version check.
2020-11-19 13:19:40 -05:00
Alisdair McDiarmid eee57280f6 backend: Faster remote backend tests
The remote backend tests spent most of their execution time sleeping in
various polling and backoff waits. This is unnecessary when testing
against a mock server, so reduce all of these delays when under test to
much lower values.

Only one remaining test has an artificial delay: verifying the discovery
of services against an unknown hostname. This times out at DNS
resolution, which is more difficult to fix than seems worth it at this
time.
2020-11-18 16:00:05 -05:00
James Bardin 8e7a9b6312 output test for plan with no root output changes
Module outputs do not show up in the plan, and are not rendered in the
UI.
2020-11-17 16:11:57 -05:00
Rémi Lapeyre d81d521bcd Use a global sequence to create the IDs for each workspace
Until now the default workspace for every project would have the ID 1,
which would make it impossible to lock them at the same time since we
use the ID to identify the lock. With a global sequence to generate the
IDs, the default workspace will now have a different ID for each project
and it will be possible to lock multiple unrelated projects at the same
time.

If an old version of Terraform tries to get the lock on a project created
with this new version it will work as we continue to use the ID of the
workspace, we just change the way we generate them.

If this version tries to get a lock on a project created by an old
version of Terraform it will work as usual, but we may encounter a
conflict with another unrelated project. This is already the current
behavior so it's not an issue to persist this behavior. As users migrate
to an up-to-date version of Terraform this will stop.

Projects already present in the database will keep their conflicting IDs,
I did not wanted to change them as users may be reading the states
directly in the database for some reason. They can if they want change
them manually to remove conflicts, newly created projects will work
without manual intervention.

Closes https://github.com/hashicorp/terraform/issues/22833
2020-11-15 14:54:57 +01:00