Until now we've not really cared much about the state snapshot produced
by the previous Terraform operation, except to use it as a jumping-off
point for our refresh step.
However, we'd like to be able to report to an end-user whenever Terraform
detects a change that occurred outside of Terraform, because that's often
helpful context for understanding why a plan contains changes that don't
seem to have corresponding changes in the configuration.
As part of reporting that we'll need to keep track of the state as it
was before we did any refreshing work, so we can then compare that against
the state after refreshing. To retain enough data to achieve that, the
existing Plan field State is now two fields: PrevRunState and PriorState.
This also includes a very shallow change in the core package to make it
populate something somewhat-reasonable into this field so that integration
tests can function reasonably. However, this shallow implementation isn't
really sufficient for real-world use of PrevRunState because we'll
actually need to update PrevRunState as part of planning in order to
incorporate the results of any provider-specific state upgrades to make
the PrevRunState objects compatible with the current provider schema, or
else our diffs won't be valid. This deeper awareness of PrevRunState in
Terraform Core will follow in a subsequent commit, prior to anything else
making use of Plan.PrevRunState.
Previously we were repeating some logic in the UI layer in order to
recover relevant additional context about a change to report to a user.
In order to help keep things consistent, and to have a clearer path for
adding more such things in the future, here we capture this user-facing
idea of an "action reason" within the plan model, and then use that
directly in order to decide how to describe the change to the user.
For the moment the "tainted" situation is the only one that gets a special
message, matching what we had before, but we can expand on this in future
in order to give better feedback about the other replace situations too.
This also preemptively includes the "replacing by request" reason, which
is currently not reachable but will be used in the near future as part of
implementing the -replace=... plan command line option to allow forcing
a particular object to be replaced.
So far we don't have any special reasons for anything other than replacing,
which makes sense because replacing is the only one that is in a sense
a special case of another action (Update), but this could expand to
other kinds of reasons in the future, such as explaining which of the
few different reasons a data source read might be deferred until the
apply step.
Previously there were only two planning modes: normal mode and destroy
mode. In that context it made sense for these to be distinguished only by
a boolean flag.
We're now getting ready to add our third mode, "refresh only". This
establishes the idea that planning can be done in one of a number of
mutually-exclusive "modes", which are related to but separate from the
various other options that serve as modifiers for the plan operation.
This commit only introduces the new plans.Mode type and replaces the
existing "destroy" flag with a variable of that type. This doesn't cause
any change in effective behavior because Terraform Core still supports
only NormalMode and DestroyMode, with NewContext rejecting an attempt to
create a RefreshMode context for now.
It is in retrospect a little odd that the "destroy" flag was part of
ContextOpts rather than just an argument to the Plan method, but
refactoring that would be too invasive a change for right now so we'll
leave this as a field of the context for now and save revisiting that for
another day.
* providers.Interface: huge renamification
This commit renames a handful of functions in the providers.Interface to
match changes made in protocol v6. The following commit implements this
change across the rest of the codebase; I put this in a separate commit
for ease of reviewing and will squash these together when merging.
One noteworthy detail: protocol v6 removes the config from the
ValidateProviderConfigResponse, since it's never been used. I chose to
leave that in place in the interface until we deprecate support for
protocol v5 entirely.
Note that none of these changes impact current providers using protocol
v5; the protocol is unchanged. Only the translation layer between the
proto and terraform have changed.
The existing context test files are becoming quite unwieldy.
Start new ones both to make editing easier, and to help discourage the
copy+pasting of older test patterns we no longer need.
Rather than using a prior value of null to indicate create, which is
imprecise because null is a valid output value, only plan values that
didn't exist in the prior state as Create changes.
The planning logic here was inspired by resources, but unlike resources
a null value is valid for an output and does not necessarily indicate it
is being removed. If both before and after are null, the change should
be NoOp. When an output is removed from the configuration, we have a
separate node to create a Delete action to remove it from the state.
Remove any calls to testDiffFn and testApplyFn which don't effect the
test result. This way we have more tests using predictable provider
behavior, which is more likely to uncover legitimate regressions as the
particular behavior of the legacy mock provider logic does not need to
be taken into account.
Add reasonable default behavior to the mock provider, so that may do not
need to depend on the idiosyncrasies of the old (though updated) test
testDiffFn and to a lesser extend the testApplyFn. This behavior is
based directly upon the documented resource lifecycle, rather
than be an ad-hoc collection of behaviors from old tests.
As a proof of concept, remove all uses of testDiffFn from the plan
context tests that don't cause the tests to fail.
This allows up to detect an unset value from the zero value so that
defaults can be implemented, while still allowing tests to return
specific values of their choosing.
Binding a sensitive value to a variable with custom validation rules
would cause a panic, as the validation expression carries the sensitive
mark when it is evaluated for truthiness. This commit drops the marks
before testing, which fixes the issue.
Isolate the test schema expansion, because having NestingSet
in the schema actually necessitates [] values in the AttrsJson.
While this didn't fail any tests on its addition, that
is scary and so isolate this to the one test using it.
Because of the composition pattern used within core, we can't easily
remove a behavior from an embedded type. Rather than trying to
re-implement all necessary methods on the
NodePlannableResourceInstnaceOrphan to exclude orphaned data resources
from GraphNodeProviderConsumer, we can modify ProvidedBy to indicate
when there is no provider required.
The ignore_changes option `all` can cause computed attributes to show up
in the validation configuration, which will be rejected by the provider
SDK. Validate the config before applying the ignore_changes options.
In the future it we should probably have a way for processIgnoreChanges
to skip computed values based on the schema. Since we also want a way to
more easily query the schema for "computed-ness" to validate the
ignore_changes arguments for computed values, we can fix these at the
same time with a future change to configschema. This will most likely
require some sort of method to retrieve info from the configschema.Block
via cty.Path, which we cannot easily do right now.
Because we allow legacy providers to depart from the contract and return
changes to non-computed values, the plan response may have altered
values that were already suppressed with ignore_changes. A prime example
of this is where providers attempt to obfuscate config data by turning
the config value into a hash and storing the hash value in the state.
There are enough cases of this in existing providers that we must
accommodate the behavior for now, so for ignore_changes to work at all
on these values, we will revert the ignored values once more on the
planned state.
Core is only using the PrepareProviderConfig call for the validation
part of the method, but we should be re-validating the final config
immediately before Configure.
This change elects to not start using the PreparedConfig here, since
there is no useful reason for it at this point, and it would
introduce a functional difference between terraform releases that can be
avoided.
Because ignore_changes configuration can refer to resource arguments
which are assigned sensitive values, we need to unmark the resource
object before processing.
Replace the old mock provider test functions with modern equivalents.
There were a lot of inconsistencies in how they were used, so we needed
to update a lot of tests to match the correct behavior.
The loading of the initial instance state was inadvertently skipped when
-refresh=false, causing all resources to appear to be missing from the
state during plan.
If a data source refers to a indexed managed resource, we need to
re-target that reference to the containing resource for planning. Since
data sources use the same mechanism as depends_on for managed resource
references, they can only refer to resources as a whole.
In order to ensure all the starting values agree, and since
ignore_changes is only meant to apply to the configuration, we need to
process the ignore_changes values on the config itself rather than the
proposed value.
This ensures the proposed new value and the config value seen by
providers are coordinated, and still allows us to use the rules laid out
by objchange.AssertPlanValid to compare the result to the configuration.
A data source referencing another data source through depends_on should
not be forced to defer until apply. Data sources have no side effects,
so nothing should need to be applied. If the dependency has a
planned change due to a managed resource, the original data source will
also encounter that further down the list of dependencies.
This prevents a data source being read during plan for any reason from
causing other data sources to be deferred until apply. It does not
change the behavior noticeably in 0.14, but because 0.13 still had
separate refresh and plan phases which could read the data source, the
deferral could cause many things downstream to become unexpectedly
unknown until apply.
Now that we don't have to handle data sources that may or may not have
been updated during a refresh phase, and the plan phase can save the
data source to the refreshed state, we can remove a lot of the logic
involved in detecting whether the data source needs to be planned or
not.
When there is no separate refresh phase, we always must attempt to read
the data source during planning, and the only conditions are based on
having a known configuration, and not having any dependencies on which
we're waiting. If the data source is read during plan, we can now save
that directly to the refreshed state, and don't need to smuggle the
value as a change to be saved during apply.
Treat any reference from a data source to a managed resource as a
dependency on the entire resource. While a resource's
attribute may be statically resolvable from the configuration, if the
user added a reference to that resource, it stands to reason that the
user intended there to be a dependency which we need to wait on.
This is an extension of an implicit behavior that existed previously in
Terraform, but was lost in the 0.13 release. That behavior was emergent
from the fact that the Refresh walk did not process the configuration
for managed resources, so any new resources in the config would be
evaluate as entirely unknown during Refresh, even if some attributes
were statically resolvable at that point.
This new implementation restores the old behavior, and extends it to
updates and replacements of the referenced resource.
This only changes the refreshed state stored in the plan file. Since the
change is stored in the plan, the applied result would be the same, but
we should still store the refreshed data in the plan file for tools that
consume the plan file.
This will also be needed in order to implement a new refresh command
based on the plan itself.
Test that the changes which use the sensitive variable have
corresponding path value marks. Also remove the unrelated validate
function from this test.
This also unearthed that the marking must happen
earlier in the eval_diff in order to produce a valid plan
(so that the planned marked value matches the marked config
value)
When working with a ConfigResource, the generalization of a
ModuleInstance to a Module was inadvertently dropped, and there was to
test coverage for that type of target.
Ensure we can target a specific module instance alone.
Before expansion happens, we only have expansion resource nodes that
know their ConfigResource address. In order to properly compare these to
targets within a module instance, we need to generalize the target to
also be a ConfigResource.
We can also remove the IgnoreIndices field from the transformer, since
we have addresses that are properly scoped and can compare them in the
correct context.
When looking up a resource during plan, we need to return an empty
container type when we're certain there are going to be no instances.
It's now more common to reference resources in a context that needs to
be known during plan (e.g. for_each), and always returning a DynamicVal
her would block plan from succeeding.
If a data source is storing a value that doesn't comply precisely with
the schema, it will now show up as a perpetual diff during plan.
Since we can easily detect if there is no resulting change from the
stored value, rather than presenting a planned read each time, we can
change the plan to a NoOp and log the incongruity as a warning.
Martin Atkins