Commit Graph

78 Commits

Author SHA1 Message Date
Martin Atkins 896b6bc897 core: Fix test build for ./plans/planfile
This was missed in the splitting of "Replace" into "DeleteThenCreate" and
"CreateThenDelete".
2018-10-16 19:14:11 -07:00
Martin Atkins 9179cdcbc6 plans/planfile: allow resource instances with no instance key
This happens for resources that don't have either "count" or "for_each"
set.
2018-10-16 19:14:11 -07:00
Martin Atkins 3e9d23b120 plans: Regenerate Action.String for new action values
This was missed on the initial update of these because at the time the
package wasn't generate-able due to other problems.
2018-10-16 19:14:11 -07:00
Martin Atkins a43b7df282 core: Handle forced-create_before_destroy during the plan walk
Previously we used a single plan action "Replace" to represent both the
destroy-before-create and the create-before-destroy variants of replacing.
However, this forces the apply graph builder to jump through a lot of
hoops to figure out which nodes need it forced on and rebuild parts of
the graph to represent that.

If we instead decide between these two cases at plan time, the actual
determination of it is more straightforward because each resource is
represented by only one node in the plan graph, and then we can ensure
we put the right nodes in the graph during DiffTransformer and thus avoid
the logic for dealing with deposed instances being spread across various
different transformers and node types.

As a nice side-effect, this also allows us to show the difference between
destroy-then-create and create-then-destroy in the rendered diff in the
CLI, although this change doesn't fully implement that yet.
2018-10-16 19:14:11 -07:00
James Bardin b738cdb19f make Changes.Empty() not count NoOps
This make the plan `Empty` concept more closely match the legacy diff
behavior.

Remove old unknown field from plan_test
2018-10-16 19:14:11 -07:00
Martin Atkins d53c3d5c1b plans: Retain output value changes for all outputs in memory
During the plan operation we need to retain _somewhere_ the planned
changes for all outputs so we can refer to them during expression
evaluation. For consistency with how we handle resource instance changes,
we'll keep them in the plan so we can properly retain unknown values,
which cannot be written to state.

As with output values in the state, only root output plans are retained
in a round-trip through the on-disk plan file format, but that's okay
because we can trivially re-calculate all of these during apply. We
include the _root_ outputs in the plan file only because they are
externally-visible side effects that ought to be included in any rendering
of the plan made from the plan file for user inspection.
2018-10-16 19:14:11 -07:00
Martin Atkins 8048e9a585 plans/objchange: Don't panic if old or new values are null 2018-10-16 19:14:11 -07:00
Martin Atkins 6fd82ef97e core: Split Replace changes into separate Delete/Create changes
Since we do our deletes using a separate graph node from all of the other
actions, and a "Replace" change implies both a delete _and_ a create, we
need to pretend at apply time that a single replace change was actually
two separate changes.

This will also early-exit eval if a destroy node finds a non-Delete change
or if an apply node finds a Delete change. These should not happen in
practice because we leave these nodes out of the graph when they are not
needed for the given action, but we do this here for robustness so as not
to have an invisible dependency between the graph builder and the eval
phase.
2018-10-16 19:14:11 -07:00
Martin Atkins 1aa9ac14cc plans/objchange: LongestCommonSubsequence
This algorithm is the usual first step when generating diffs. This package
is a bit of a strange home for it, but since it works with changes to
cty.Value this feels more natural than any other place it could be.
2018-10-16 19:14:11 -07:00
Martin Atkins 1ced176fc6 plans: Track RequiredReplace as a cty.PathSet
We were previously tracking this as a []cty.Path, but having it turned
into a pathset on creation makes downstream use of it more convenient and
ensures that it'll obey expected invariants like not containing the same
path twice.
2018-10-16 19:14:11 -07:00
Martin Atkins 3f8a973846 plans/objchange: when prior is null, computed attributes are unknown 2018-10-16 19:14:11 -07:00
Martin Atkins 1d672623eb core: Remove changes from the plan after they are applied 2018-10-16 19:14:11 -07:00
Martin Atkins 2f0e5d93c8 plans: ChangesSync.GetResourceInstanceChange must copy the change
This is promised in its doc comment, but wasn't actually done in practice.
2018-10-16 19:14:11 -07:00
Martin Atkins 3d86dc51e0 core: Re-implement EvalReadDiff for the new plan types
This also includes passing in the provider schema to a few more EvalNodes
that were expecting it but not getting it, in order to be able to
successfully test the implementation of EvalReadDiff here.
2018-10-16 19:14:11 -07:00
Martin Atkins bd299b9a22 core: Re-implement EvalWriteDiff to work with new plan types 2018-10-16 19:14:11 -07:00
Martin Atkins 44bc7519a6 terraform: More wiring in of new provider types
This doesn't actually work yet, but it builds and then panics in a pretty
satisfying way.
2018-10-16 19:12:54 -07:00
Martin Atkins 549544f201 configschema: Handle nested blocks containing dynamic-typed attributes
We need to make the collection itself be a tuple or object rather than
list or map in this case, since otherwise all of the elements of the
collection are constrained to be of the same type and that isn't the
intent of a provider indicating that it accepts any type.
2018-10-16 19:11:09 -07:00
Martin Atkins 4c78539c2b plans/objchange: AssertObjectSuperset function
This function's goal is to ensure that the "final" plan value produced
by a provider during the apply step is always consistent with the known
parts of the planned value produced during the plan step.

Any error produced here indicates a bug in the provider.
2018-10-16 19:11:09 -07:00
Martin Atkins 0317da9911 plans/objchange: logic for merging prior state with config
This produces a "proposed new state", which already has prior computed
values propagated into it (since that behavior is standard for all
resource types) but could be customized further by the provider to make
the "_planned_ new state".

In the process of implementing this it became clear that our configschema
DecoderSpec behavior is incorrect, since it's producing list values for
NestingList and map values for NestingMap. While that seems like it should
be right, we should actually be using tuple and object types respectively
to allow each block to have a different runtime type in situations where
an attribute is given the type cty.DynamicPseudoType. That's not fixed
here, and so without a further fix list and map blocks will panic here.
The DecoderSpec implementation will be fixed in a subsequent commit.
2018-10-16 19:11:09 -07:00
Martin Atkins dfe0988f10 plans: Record private data bytes as part of resource change
This allows a provider to retain arbitrary extra data in the plan and
make use of it during apply. The contents are not used by Terraform and
never shown to the user.
2018-10-16 19:11:09 -07:00
Martin Atkins a3403f2766 terraform: Ugly huge change to weave in new State and Plan types
Due to how often the state and plan types are referenced throughout
Terraform, there isn't a great way to switch them out gradually. As a
consequence, this huge commit gets us from the old world to a _compilable_
new world, but still has a large number of known test failures due to
key functionality being stubbed out.

The stubs here are for anything that interacts with providers, since we
now need to do the follow-up work to similarly replace the old
terraform.ResourceProvider interface with its replacement in the new
"providers" package. That work, along with work to fix the remaining
failing tests, will follow in subsequent commits.

The aim here was to replace all references to terraform.State and its
downstream types with states.State, terraform.Plan with plans.Plan,
state.State with statemgr.State, and switch to the new implementations of
the state and plan file formats. However, due to the number of times those
types are used, this also ended up affecting numerous other parts of core
such as terraform.Hook, the backend.Backend interface, and most of the CLI
commands.

Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize
in advance to the person who inevitably just found this huge commit while
spelunking through the commit history.
2018-10-16 19:11:09 -07:00
Martin Atkins b7db32b819 plans: separate types for encoded and decoded changes
The types here were originally written to allow us to defer decoding of
object values until schemas are available, but it turns out that this was
forcing us to defer decoding longer than necessary and potentially decode
the same value multiple times.

To avoid this, we create pairs of types to represent the encoded and
decoded versions and methods for moving between them. These types are
identical to one another apart from how the dynamic values are
represented.
2018-10-16 18:58:49 -07:00
Martin Atkins 074db88636 plans: Include target addresses in the plan
We shouldn't really need these because the plan is already filtered to
include diffs only for targeted resources, but we currently rely on this
to filter out non-resource items from the diff, and so we'll retain it
for now to avoid reworking how the apply-time graph builder works.
2018-10-16 18:50:29 -07:00
Martin Atkins d9dfd135c6 plans: Include backend settings in plan and plan files
On the first pass here we erroneously assumed that this was redundant with
the backend settings embedded in the configuration itself. In practice,
users can override backend configuration when running "terraform init"
and so we need to record the _effective_ backend configuration.

Along with this, we also return the selected workspace name at the time
the plan was created so we'll later be able to produce a specialized error
for the situation of having the wrong workspace selected. This isn't
strictly required because we'll also check the lineage of the state, but
the error message that would result from that failure would be relatively
opaque and thus less helpful to the user.
2018-10-16 18:50:29 -07:00
Martin Atkins f77e7a61b0 various: helpers for collecting necessary provider types
Since schemas are required to interpret provider, resource, and
provisioner attributes in configs, states, and plans, these helpers intend
to make it easier to gather up the the necessary provider types in order
to preload all of the needed schemas before beginning further processing.

Config.ProviderTypes returns directly the list of provider types, since
at this level further detail is not useful: we've not yet run the
provider allocation algorithm, and so the only thing we can reliably
extract here is provider types themselves.

State.ProviderAddrs and Plan.ProviderAddrs each return a list of
absolute provider addresses, which can then be turned into a list of
provider types using the new helper providers.AddressedTypesAbs.

Since we're already using configs.Config throughout core, this also
updates the terraform.LoadSchemas helper to use Config.ProviderTypes
to find the necessary providers, rather than implementing its own
discovery logic. states.State is not yet plumbed in, so we cannot yet
use State.ProviderAddrs to deal with the state but there's a TODO comment
to remind us to update that in a later commit when we swap out
terraform.State for states.State.

A later commit will probably refactor this further so that we can easily
obtain schema for the providers needed to interpret a plan too, but that
is deferred here because further work is required to make core work with
the new plan types first. At that point, terraform.LoadSchemas may become
providers.LoadSchemas with a different interface that just accepts lists
of provider and provisioner names that have been gathered by the caller
using these new helpers.
2018-10-16 18:50:29 -07:00
Martin Atkins 51aeba2cf2 plans: Record provider config address in each resource change
This allows us to record the resource-to-provider associations we made
during the plan phase and ensure that we use the same config during
apply.

This isn't technically necessary, since we can in principle just repeat
the same matching algorithm against the config, but that algorithm is
relatively complicated and so if we execute it just once during plan we
remove the risk of bugs causing different decisions to be made during
the apply phase.

This also includes updates to the plan file format to include the provider
addresses. Since we've not actually shipped any version of Terraform
using our protobuf schema yet, we renumber here all of the fields after
the new one to keep them incrementing consecutively. This example should
not be followed after we have a released version of Terraform using this,
as a courtesy to anyone trying to parse these files (even though we're
not yet guaranteeing compatibility between releases).
2018-10-16 18:50:29 -07:00
Martin Atkins a2eb462f5d plans/planfile: Reading and writing the new plan format
The new format is radically different in than the old in physical
structure, but still has the same logical parts: the plan itself, a
snapshot of the input configuration, and a snapshot of the state as it
existed when the plan was created.

Rather than creating plan-specific serializations of state and config, the
new format instead leans on the existing file formats implemented
elsewhere, wrapping the result up in a zip archive with some internal file
naming conventions.

The plan portion of the file is serialized with protobuf, consistent with
our general strategy of replacing all use of encoding/gob with protobuf
moving forward.
2018-10-16 18:50:29 -07:00
Martin Atkins 7357e7f734 plans: New package for in-memory plan models
The types in this package are intended to replace plan- and diff-related
types from the "terraform" package, although those older types must remain
for now so that they can be used to implement shims for older codepaths.

type "Changes" is approximately equivalent to terraform.Diff, but renamed
since it now describes whole objects before and after rather than an
attribute-level diff as before. The term "diff" is now reserved for the
visual rendition of the changes we'll display to the user, although
rendering of this new Changes model is not yet implemented.
2018-10-16 18:50:29 -07:00