Commit Graph

31 Commits

Author SHA1 Message Date
Sander van Harmelen 52a1b22f7a Implement the remote enhanced backend
This is a refactored version of the `remote` backend that was initially added to Terraform v0.11.8 which should now be compatible with v0.12.0.
2018-11-06 16:29:46 +01:00
Martin Atkins 98bbd560b5 command: Fix most (but not all) "terraform plan" tests 2018-10-16 19:14:11 -07:00
Martin Atkins 2b80df0163 backend/local: Require caller to set PlanOutBackend with PlanOutPath
We can't generate a valid plan file without a backend configuration to
write into it, but it's the responsibility of the caller (the command
package) to manage the backend configuration mechanism, so we require it
to tell us what to write here.

This feels a little strange because the backend in principle knows its
own config, but in practice the backend only knows the _processed_ version
of the config, not the raw configuration value that was used to configure
it.
2018-10-16 19:14:11 -07:00
Kristin Laemmert 6a37ee9277 backend/local: more tests passing
I have no confidence in the change to plans/planfile/tfplan.go. The
tests were passing an empty backend config, which planfile  was able to
write to a file but not read from the same file. This change let me move
past that and it did not break any tests in the planfile package, but I
am concerned that it introduces undesired behavior.
2018-10-16 19:14:11 -07:00
Kristin Laemmert c661157999 plans/objchange: further harden ProposedNewObject against ~weird~
incoming values

Addresses an odd state where the priorV of an object to be changed is
known but null.

While this situation should not happen, it seemed prudent to ensure that
core is resilient to providers sending incorrect values (which might
also occur with manually edited state).
2018-10-16 19:14:11 -07:00
Martin Atkins a43b7df282 core: Handle forced-create_before_destroy during the plan walk
Previously we used a single plan action "Replace" to represent both the
destroy-before-create and the create-before-destroy variants of replacing.
However, this forces the apply graph builder to jump through a lot of
hoops to figure out which nodes need it forced on and rebuild parts of
the graph to represent that.

If we instead decide between these two cases at plan time, the actual
determination of it is more straightforward because each resource is
represented by only one node in the plan graph, and then we can ensure
we put the right nodes in the graph during DiffTransformer and thus avoid
the logic for dealing with deposed instances being spread across various
different transformers and node types.

As a nice side-effect, this also allows us to show the difference between
destroy-then-create and create-then-destroy in the rendered diff in the
CLI, although this change doesn't fully implement that yet.
2018-10-16 19:14:11 -07:00
Martin Atkins 5390fb1eed backend/local: Don't count outputs for choosing diff action symbols
We're not yet showing outputs in the rendered diff, so it doesn't make
sense to count them for the purpose of deciding which change action
symbols to include in the legend.
2018-10-16 19:14:11 -07:00
Martin Atkins 20318ca193 backend/local: Sort planned resource changes before rendering them
As in the old plan renderer, they are ordered by the natural ordering of
the resource addresses.
2018-10-16 19:14:11 -07:00
Martin Atkins 239a54ad6f command: initial structural diff rendering
This is a light adaptation of our earlier prototype of structural diff
rendering, as a starting point for what we'll actually ship. This is not
consistent with the latest mocks, so will need some additional work before
it is ready, but integrating this allows us to at least see the plan
contents while fixing up remaining issues elsewhere.
2018-10-16 19:14:11 -07:00
Martin Atkins a3403f2766 terraform: Ugly huge change to weave in new State and Plan types
Due to how often the state and plan types are referenced throughout
Terraform, there isn't a great way to switch them out gradually. As a
consequence, this huge commit gets us from the old world to a _compilable_
new world, but still has a large number of known test failures due to
key functionality being stubbed out.

The stubs here are for anything that interacts with providers, since we
now need to do the follow-up work to similarly replace the old
terraform.ResourceProvider interface with its replacement in the new
"providers" package. That work, along with work to fix the remaining
failing tests, will follow in subsequent commits.

The aim here was to replace all references to terraform.State and its
downstream types with states.State, terraform.Plan with plans.Plan,
state.State with statemgr.State, and switch to the new implementations of
the state and plan file formats. However, due to the number of times those
types are used, this also ended up affecting numerous other parts of core
such as terraform.Hook, the backend.Backend interface, and most of the CLI
commands.

Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize
in advance to the person who inevitably just found this huge commit while
spelunking through the commit history.
2018-10-16 19:11:09 -07:00
Martin Atkins c937c06a03 terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.

The three main goals here are:
- Use the configuration models from the "configs" package instead of the
  older models in the "config" package, which is now deprecated and
  preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
  new "lang" package, instead of the Interpolator type and related
  functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
  rather than hand-constructed strings. This is not critical to support
  the above, but was a big help during the implementation of these other
  points since it made it much more explicit what kind of address is
  expected in each context.

Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.

I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-10-16 18:46:46 -07:00
Martin Atkins 5782357c28 backend: Update interface and implementations for new config loader
The new config loader requires some steps to happen in a different
order, particularly in regard to knowing the schema in order to
decode the configuration.

Here we lean directly on the configschema package, rather than
on helper/schema.Backend as before, because it's generally
sufficient for our needs here and this prepares us for the
helper/schema package later moving out into its own repository
to seed a "plugin SDK".
2018-10-16 18:39:12 -07:00
James Bardin e9a76808df create clistate.Locker interface
Simplify the use of clistate.Lock by creating a clistate.Locker
instance, which stores the context of locking a state, to allow unlock
to be called without knowledge of how the state was locked.

This alows the backend code to bring the needed UI methods to the point
where the state is locked, and still unlock the state from an outer
scope.

Provide a NoopLocker as well, so that callers can always call Unlock
without verifying the status of the lock.

Add the StateLocker field to the backend.Operation, so that the state
lock can be carried between the different function scopes of the backend
code. This will allow the backend context to lock the state before it's
read, while allowing the different operations to unlock the state when
they complete.
2018-02-23 16:48:15 -05:00
James Bardin ef8ed1e275 coalesce the backened interrupt code
Moves the nested select statements for backend operations into a single
function. The only difference in this part was that apply called
PersistState, which should be harmless regardless of the type of
operation being run.
2018-02-12 11:56:54 -05:00
James Bardin 7cba68326a always wait for a RunningOperation to return
If the user wishes to interrupt the running operation, only the first
interrupt was communicated to the operation by canceling the provided
context. A second interrupt would start the shutdown process, but not
communicate this to the running operation. This order of event could
cause partial writes of state.

What would happen is that once the command returns, the plugin system
would stop the provider processes. Once the provider processes dies, all
pending Eval operations would return return with an error, and quickly
cause the operation to complete. Since the backend code didn't know that
the process was shutting down imminently, it would continue by
attempting to write out the last known state. Under the right
conditions, the process would exit part way through the writing of the
state file.

Add Stop and Cancel CancelFuncs to the RunningOperation, to allow it to
easily differentiate between the two signals. The backend will then be
able to detect a shutdown and abort more gracefully.

In order to ensure that the backend is not in the process of writing the
state out, the command will always attempt to wait for the process to
complete after cancellation.
2018-02-12 11:56:03 -05:00
James Bardin 85295e5c23 watch for cancellation in plan and refresh
Cancellation in the local backend was only implemented for apply.
2017-12-05 10:17:20 -05:00
Martin Atkins 0fe43c8977 cli: allow disabling "next steps" message in terraform plan
In #15884 we adjusted the plan output to give an explicit command to run
to apply a plan, whereas before this command was just alluded to in the
prose.

Since releasing that, we've got good feedback that it's confusing to
include such instructions when Terraform is running in a workflow
automation tool, because such tools usually abstract away exactly what
commands are run and require users to take different actions to
proceed through the workflow.

To accommodate such environments while retaining helpful messages for
normal CLI usage, here we introduce a new environment variable
TF_IN_AUTOMATION which, when set to a non-empty value, is a hint to
Terraform that it isn't being run in an interactive command shell and
it should thus tone down the "next steps" messaging.

The documentation for this setting is included as part of the "...in
automation" guide since it's not generally useful in other cases. We also
intentionally disclaim comprehensive support for this since we want to
avoid creating an extreme number of "if running in automation..."
codepaths that would increase the testing matrix and hurt maintainability.

The focus is specifically on the output of the three commands we give in
the automation guide, which at present means the following two situations:

* "terraform init" does not include the final paragraphs that suggest
  running "terraform plan" and tell you in what situations you might need
  to re-run "terraform init".
* "terraform plan" does not include the final paragraphs that either
  warn about not specifying "-out=..." or instruct to run
  "terraform apply" with the generated plan file.
2017-09-14 10:51:41 -07:00
Martin Atkins 83414beb8f command: various adjustments to the diff presentation
The previous diff presentation was rather "wordy", and not very friendly
to those who can't see color either because they have color-blindness or
because they don't have a color-supporting terminal.

This new presentation uses the actual symbols used in the plan output
and tries to be more concise. It also uses some framing characters to
try to separate the different stages of "terraform plan" to make it
easier to visually navigate.

The apply command also adopts this new plan presentation, in preparation
for "terraform apply" (with interactive plan confirmation) becoming the
primary, safe workflow in the next major release.

Finally, we standardize on the terminology "perform" and "actions" rather
than "execute" and "changes" to reflect the fact that reading is now an
action and that isn't actually a _change_.
2017-09-01 17:55:05 -07:00
Martin Atkins 3ea159297c command/format: improve consistency of plan results
Previously the rendered plan output was constructed directly from the
core plan and then annotated with counts derived from the count hook.
At various places we applied little adjustments to deal with the fact that
the user-facing diff model is not identical to the internal diff model,
including the special handling of data source reads and destroys. Since
this logic was just muddled into the rendering code, it behaved
inconsistently with the tally of adds, updates and deletes.

This change reworks the plan formatter so that it happens in two stages:
- First, we produce a specialized Plan object that is tailored for use
  in the UI. This applies all the relevant logic to transform the
  physical model into the user model.
- Second, we do a straightforward visual rendering of the display-oriented
  plan object.

For the moment this is slightly overkill since there's only one rendering
path, but it does give us the benefit of letting the counts be derived
from the same data as the full detailed diff, ensuring that they'll stay
consistent.

Later we may choose to have other UIs for plans, such as a
machine-readable output intended to drive a web UI. In that case, we'd
want the web UI to consume a serialization of the _display-oriented_ plan
so that it doesn't need to re-implement all of these UI special cases.

This introduces to core a new diff action type for "refresh". Currently
this is used _only_ in the UI layer, to represent data source reads.
Later it would be good to use this type for the core diff as well, to
improve consistency, but that is left for another day to keep this change
focused on the UI.
2017-09-01 17:55:05 -07:00
James Bardin 305ef43aa6 provide contexts to clistate.Lock calls
Add fields required to create an appropriate context for all calls to
clistate.Lock.

Add missing checks for Meta.stateLock, where we would attempt to lock,
even if locking should be skipped.
2017-04-01 17:09:20 -04:00
James Bardin 3f0dcd1308 Have the clistate Lock use LockWithContext
- Have the ui Lock helper use state.LockWithContext.
- Rename the message package to clistate, since that's how it's imported
  everywhere.
- Use a more idiomatic placement of the Context in the LockWithContext
  args.
2017-04-01 17:09:20 -04:00
Mitchell Hashimoto d01886a644
command: remove legacy remote state on migration
Fixes #12871

We were forgetting to remove the legacy remote state from the actual
state value when migrating. This only causes an issue when saving a plan
since the plan contains the state itself and causes an error where both
a backend + legacy state exist.

If saved plans aren't used this causes no noticable issue.

Due to buggy upgrades already existing in the wild, I also added code to
clear the remote section if it exists in a standard unchanged backend
2017-03-20 10:14:59 -07:00
James Bardin f2e496a14c Have backend operations properly unlock state
Make sure unlock is called with the correct LockID during operations
2017-02-15 14:41:55 -05:00
James Bardin 67dc16c9ca Make backend/local test pass 2017-02-15 14:41:55 -05:00
Mitchell Hashimoto 235b7eb38e Merge pull request #11944 from hashicorp/f-state-slow
show message if state lock acquisition/release is slow
2017-02-14 14:00:23 -08:00
Mitchell Hashimoto 5e4f6cf2b1
backend/local: fix could not to did not to prevent error look 2017-02-14 12:09:45 -08:00
Mitchell Hashimoto 65982bd412
backend/local: use new command/state package for better UX 2017-02-14 11:17:28 -08:00
James Bardin 9cdba1f199 enable local state locking for apply
Have the LocalBackend lock the state during operations, and enble this
for the apply comand.
2017-02-02 18:08:28 -05:00
Mitchell Hashimoto a424203ea3
backend/local: validate module exists for plan
Fixes #11504

The local backend should error if `terraform plan` is called in a
directory with no Terraform config files (same behavior as 0.8.x).
**New behavior:** We now allow `terraform plan -destroy` with no
configuration files since that seems reasonable.
2017-01-29 20:02:12 -08:00
Mitchell Hashimoto 31f7cca77f
backend/local: fix crash (in tests) due to not guarding nil CLI 2017-01-26 14:33:50 -08:00
Mitchell Hashimoto 397e1b3132
backend/local
The local backend implementation is an implementation of
backend.Enhanced that recreates all the behavior of the CLI but through
the backend interface.
2017-01-26 14:33:49 -08:00