Commit Graph

42 Commits

Author SHA1 Message Date
Alisdair McDiarmid c5c1f31db3 backend: Validate remote backend Terraform version
When using the enhanced remote backend, a subset of all Terraform
operations are supported. Of these, only plan and apply can be executed
on the remote infrastructure (e.g. Terraform Cloud). Other operations
run locally and use the remote backend for state storage.

This causes problems when the local version of Terraform does not match
the configured version from the remote workspace. If the two versions
are incompatible, an `import` or `state mv` operation can cause the
remote workspace to be unusable until a manual fix is applied.

To prevent this from happening accidentally, this commit introduces a
check that the local Terraform version and the configured remote
workspace Terraform version are compatible. This check is skipped for
commands which do not write state, and can also be disabled by the use
of a new command-line flag, `-ignore-remote-version`.

Terraform version compatibility is defined as:

- For all releases before 0.14.0, local must exactly equal remote, as
  two different versions cannot share state;
- 0.14.0 to 1.0.x are compatible, as we will not change the state
  version number until at least Terraform 1.1.0;
- Versions after 1.1.0 must have the same major and minor versions, as
  we will not change the state version number in a patch release.

If the two versions are incompatible, a diagnostic is displayed,
advising that the error can be suppressed with `-ignore-remote-version`.
When this flag is used, the diagnostic is still displayed, but as a
warning instead of an error.

Commands which will not write state can assert this fact by calling the
helper `meta.ignoreRemoteBackendVersionConflict`, which will disable the
checks. Those which can write state should instead call the helper
`meta.remoteBackendVersionCheck`, which will return diagnostics for
display.

In addition to these explicit paths for managing the version check, we
have an implicit check in the remote backend's state manager
initialization method. Both of the above helpers will disable this
check. This fallback is in place to ensure that future code paths which
access state cannot accidentally skip the remote version check.
2020-11-19 13:19:40 -05:00
Martin Atkins c94a6102df command: Improve consistency of the command short descriptions
The short description of our commands (as shown in the main help output
from "terraform") was previously very inconsistent, using different
tense/mood for different commands. Some of the commands were also using
some terminology choices inconsistent with how we currently talk about
the related ideas in our documentation.

Here I've tried to add some consistency by first rewriting them all in
the imperative mood (except the ones that just are just subcommand
groupings), and tweaking some of the terminology to hopefully gel better
with how we present similar ideas in our recently-updated docs.

While working on this I inevitably spotted some similar inconsistencies
in the longer-form help output of some of the commands. I've not reviewed
all of these for consistency, but I did update some where the wording
was either left inconsstent with the short form changes I'd made or
where the prose stood out to me as particularly inconsistent with our
current usual documentation language style.

All of this is subjective, so I expect we'll continue to tweak these over
time as we continue to develop our documentation writing style based on
user questions and feedback.
2020-10-26 09:55:21 -07:00
Alisdair McDiarmid 67203dade8 command: Simplify Meta.process helper method
After some refactoring, this helper method had an unused argument (vars)
and an always-nil error return value. This commit cleans this up.
2020-04-01 15:01:08 -04:00
Alisdair McDiarmid 6413313529 command/graph: Remove no-op state unlock
The graph command never locks the state, so this unlock operation was a
no-op and is a bit of a red herring. Remove it.
2020-02-14 14:48:50 -05:00
Pam Selle cd6c93774a Update docs to reflect current behavior 2020-01-08 16:51:42 -05:00
Martin Atkins 8664749b59 backend: Allow certain commands to opt out of required variable checks
Terraform Core expects all variables to be set, but for some ancillary
commands it's fine for them to just be set to placeholders because the
variable values themselves are not key to the command's functionality
as long as the terraform.Context is still self-consistent.

For such commands, rather than prompting for interactive input for
required variables we'll just stub them out as unknowns to reflect that
they are placeholders for values that a user would normally need to
provide.

This achieves a similar effect to how these commands behaved before, but
without the tendency to produce a slightly invalid terraform.Context that
would fail in strange ways when asked to run certain operations.
2019-10-10 10:07:01 -07:00
Kristin Laemmert c9d62bb2f6
command: discard output from flags package and return errs directly (#22373)
Any command using meta.defaultFlagSet *might* occasionally exit before
the flag package's output got written. This caused flag error messages
to get lost. This PR discards the flag package output in favor of
directly returning the error to the end user.
2019-08-16 08:31:21 -04:00
Przemysław Dąbek 9605b093d9 command/graph: use user-supplied plugin path when running graph command (#18083) 2019-04-17 13:48:11 -04:00
Sander van Harmelen ef9054562e commands: make sure the correct flagset is used
A lot of commands used `c.Meta.flagSet()` to create the initial flagset for the command, while quite a few of them didn’t actually use or support the flags that are then added.

So I updated a few commands to use `flag.NewFlagSet()` instead to only add the flags that are actually needed/supported.

Additionally this prevents a few commands from using locking while they actually don’t need locking (as locking is enabled as a default in `c.Meta.flagSet()`.
2018-11-23 16:13:34 +01:00
Sander van Harmelen 178ec8f7b4 Remove support for the -module-depth flag
# Conflicts:
#	backend/backend.go
2018-11-02 18:44:04 +01:00
Martin Atkins a3403f2766 terraform: Ugly huge change to weave in new State and Plan types
Due to how often the state and plan types are referenced throughout
Terraform, there isn't a great way to switch them out gradually. As a
consequence, this huge commit gets us from the old world to a _compilable_
new world, but still has a large number of known test failures due to
key functionality being stubbed out.

The stubs here are for anything that interacts with providers, since we
now need to do the follow-up work to similarly replace the old
terraform.ResourceProvider interface with its replacement in the new
"providers" package. That work, along with work to fix the remaining
failing tests, will follow in subsequent commits.

The aim here was to replace all references to terraform.State and its
downstream types with states.State, terraform.Plan with plans.Plan,
state.State with statemgr.State, and switch to the new implementations of
the state and plan file formats. However, due to the number of times those
types are used, this also ended up affecting numerous other parts of core
such as terraform.Hook, the backend.Backend interface, and most of the CLI
commands.

Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize
in advance to the person who inevitably just found this huge commit while
spelunking through the commit history.
2018-10-16 19:11:09 -07:00
Martin Atkins c937c06a03 terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.

The three main goals here are:
- Use the configuration models from the "configs" package instead of the
  older models in the "config" package, which is now deprecated and
  preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
  new "lang" package, instead of the Interpolator type and related
  functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
  rather than hand-constructed strings. This is not critical to support
  the above, but was a big help during the implementation of these other
  points since it made it much more explicit what kind of address is
  expected in each context.

Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.

I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-10-16 18:46:46 -07:00
Martin Atkins ebafa51723 command: Various updates for the new backend package API
This is a rather-messy, complex change to get the "command" package
building again against the new backend API that was updated for
the new configuration loader.

A lot of this is mechanical rewriting to the new API, but
meta_config.go and meta_backend.go in particular saw some major
changes to interface with the new loader APIs and to deal with
the change in order of steps in the backend API.
2018-10-16 18:44:26 -07:00
James Bardin 90a75422fb unlock state in console, import, graph, and push
The state locking improvements for the regular command had the side
effect of locking the state in the console, import, graph and push
commands. Those commands had been updated to get a state via the
Backend.Context method, which locks the state whenever possible, and now
need to call Unlock directly.

Add Unlock calls to all commands that call Context directly.
2018-03-21 12:13:40 -04:00
Martin Atkins 9a5c865040 command: validate config as part of loading it
Previously we required callers to separately call .Validate on the root
module to determine if there were any value errors, but we did that
inconsistently and would thus see crashes in some cases where later code
would try to use invalid configuration as if it were valid.

Now we run .Validate automatically after config loading, returning the
resulting diagnostics. Since we return a diagnostics here, it's possible
to return both warnings and errors.

We return the loaded module even if it's invalid, so callers are free to
ignore returned errors and try to work with the config anyway, though they
will need to be defensive against invalid configuration themselves in
that case.

As a result of this, all of the commands that load configuration now need
to use diagnostic printing to signal errors. For the moment this just
allows us to return potentially-multiple config errors/warnings in full
fidelity, but also sets us up for later when more subsystems are able
to produce rich diagnostics so we can show them all together.

Finally, this commit also removes some stale, commented-out code for the
"legacy" (pre-0.8) graph implementation, which has not been available
for some time.
2017-12-07 14:28:43 -08:00
James Bardin f10163ecc7 graph should not panic with no config
The backends replace a nil module tree with an empty one before building
the graph, so the graph command needs to do the same.
2017-07-18 13:03:57 -04:00
Robert Liebowitz 006744bfe0 Use all tfvars files in working directory
As a side effect, several commands that previously did not have a failure
state can now fail during meta-parameter processing.
2017-07-05 17:24:17 -07:00
James Bardin 718ede0636 have Meta.Backend use a Config rather than loading
Instead of providing the a path in BackendOpts, provide a loaded
*config.Config instead. This reduces the number of places where
configuration is loaded.
2017-06-09 14:03:59 -07:00
Mitchell Hashimoto 09242fab09
terraform: remove legacy graph builder 2017-01-26 15:18:42 -08:00
Mitchell Hashimoto ad7b063262
command: convert to use backends 2017-01-26 14:33:49 -08:00
Mitchell Hashimoto 22e868b966
command/graph: update for new graphs 2016-12-03 15:17:10 -08:00
James Bardin 28d406c040 Provider a marshaler for dag.Graph
The dot format generation was done with a mix of code from the terraform
package and the dot package. Unify the dot generation code, and it into
the dag package.

Use an intermediate structure to allow a dag.Graph to marshal itself
directly. This structure will be ablt to marshal directly to JSON, or be
translated to dot format. This was we can record more information about
the graph in the debug logs, and provide a way to translate those logged
structures to dot, which is convenient for viewing the graphs.
2016-11-14 08:50:33 -05:00
Paul Hinze e67fc0fe9b command: Change module-depth default to -1
This means that terraform commands like `plan`, `apply`, `show`, and
`graph` will expand all modules by default.

While modules-as-black-boxes is still very true in the conceptual design
of modules, feedback on this behavior has consistently suggested that
users would prefer to see more verbose output by default.

The `-module-depth` flag and env var are retained to allow output to be
optionally limited / summarized by these commands.
2016-01-20 13:58:02 -06:00
Radek Simko b7d41d2eed Add -no-color to help text 2015-06-22 13:14:01 +01:00
Paul Hinze 29d34cd5a4 command: allow module depth to be set via env var
Another convienence env var here with TF_MODULE_DEPTH.

Works like you'd expect it to!
2015-04-30 16:19:43 -05:00
Paul Hinze ce49dd6080 core: graph command gets -verbose and -draw-cycles
When you specify `-verbose` you'll get the whole graph of operations,
which gives a better idea of the operations terraform performs and in
what order.

The DOT graph is now generated with a small internal library instead of
simple string building. This allows us to ensure the graph generation is
as consistent as possible, among other benefits.

We set `newrank = true` in the graph, which I've found does just as good
a job organizing things visually as manually attempting to rank the nodes
based on depth.

This also fixes `-module-depth`, which was broken post-AST refector.
Modules are now expanded into subgraphs with labels and borders. We
have yet to regain the plan graphing functionality, so I removed that
from the docs for now.

Finally, if `-draw-cycles` is added, extra colored edges will be drawn
to indicate the path of any cycles detected in the graph.

A notable implementation change included here is that
{Reverse,}DepthFirstWalk has been made deterministic. (Before it was
dependent on `map` ordering.) This turned out to be unnecessary to gain
determinism in the final DOT-level implementation, but it seemed
a desirable enough of a property that I left it in.
2015-04-27 09:23:47 -05:00
Paul Hinze d4b9362518 core: validate on verbose graph to detect some cycles earlier
Most CBD-related cycles include destroy nodes, and destroy nodes were
all being pruned from the graph before staring the Validate walk.

In practice this meant that we had scenarios that would error out with
graph cycles on Apply that _seemed_ fine during Plan.

This introduces a Verbose option to the GraphBuilder that tells it to
generate a "worst-case" graph. Validate sets this to true so that cycle
errors will always trigger at this step if they're going to happen.

(This Verbose option will be exposed as a CLI flag to `terraform graph`
in a second incoming PR.)

refs #1651
2015-04-23 11:07:13 -05:00
Mitchell Hashimoto c2593f6ada terraform: re-enable dot-graphs 2015-02-19 23:00:29 -08:00
Mitchell Hashimoto b1e5b32322 terraform: Graph returns *Graph for now 2015-02-19 12:08:32 -08:00
Mitchell Hashimoto 72e6f97093 terraform: support graphing modules 2014-09-24 17:36:27 -07:00
Mitchell Hashimoto ed538a9594 command: Get command, not functional yet. Converted to use modules. 2014-09-22 10:56:50 -07:00
Mitchell Hashimoto 642fed0356 command: terraform.tfvars loaded by default if it exists 2014-08-05 09:32:01 -07:00
Mitchell Hashimoto ef6fba754d command: refactor so Context never plans 2014-07-26 17:51:15 -07:00
Mitchell Hashimoto 6c8c09c784 command/*: only Plan on the Apply 2014-07-14 11:48:03 -07:00
Mitchell Hashimoto ad3c0593a3 terraform: GraphDot 2014-07-14 11:34:52 -07:00
Mitchell Hashimoto 3a851bece0 command: convert all to use the new Meta thing 2014-07-12 20:37:30 -07:00
Mitchell Hashimoto 3d35158170 command: update synopsis to be better 2014-07-12 19:28:38 -07:00
Mitchell Hashimoto 8e100869a4 command/graph: can graph plans 2014-07-12 19:25:50 -07:00
Mitchell Hashimoto 235a253848 command/graph: no args means pwd 2014-07-11 20:41:47 -07:00
Mitchell Hashimoto 8f7244695f command/graph: takes config dir as arg 2014-07-11 20:38:03 -07:00
Mitchell Hashimoto a6ae7230d1 command: use new API 2014-07-03 11:46:40 -07:00
Mitchell Hashimoto 93fbb9ea8f command/graph 2014-07-01 10:02:13 -07:00