Commit Graph

140 Commits

Author SHA1 Message Date
Martin Atkins 31349a9c3a Move configs/ to internal/configs/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins b9a93a0fe7 Move addrs/ to internal/addrs/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 05caff2ca3 Move tfdiags/ to internal/tfdiags/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins ce69c3903f command/views: Show refresh-detected changes as part of a plan
We've always had a mechanism to synchronize the Terraform state with
remote objects before creating a plan, but we previously kept the result
of that to ourselves, and so it would sometimes lead to Terraform
generating a planned action to undo some upstream drift, but Terraform
would give no context as to why that action was planned even though the
relevant part of the configuration hadn't changed.

Now we'll detect any differences between the previous run state and the
refreshed state and, if any managed resources now look different, show
an additional note about it as extra context for the planned changes that
follow.

This appears as an optional extra block of information before the normal
plan output. It'll appear the same way in all of the contexts where we
render plans, including "terraform show" for saved plans.
2021-05-13 09:05:06 -07:00
James Bardin bac4e0a3de fix ResourceInstanceObject.DeepCopy
missing CreateBeforeDestroy
2021-03-25 17:39:53 -04:00
Alex Chan 4ec80fa18d Correct the spelling of heirarchy/hierarchy throughout 2021-02-05 15:07:04 +00:00
James Bardin c6cea8efc3 states/statefile: staticcheck 2020-12-02 13:59:19 -05:00
James Bardin c2f8b06594 states/statemgr: staticcheck 2020-12-02 13:59:19 -05:00
James Bardin 0fb7d19191 states/statemgr: t.Fatal from goroutine 2020-12-02 13:59:19 -05:00
James Bardin 44ccb94e2e remove unused 2020-12-02 13:59:18 -05:00
James Bardin 729c9b6f10 consistent reciever names 2020-12-02 13:59:18 -05:00
Alisdair McDiarmid dc2e7520e5 states: Disable Terraform version check
For this version of Terraform and forward, we no longer refuse to read
compatible state files written by future versions of Terraform. This is
a commitment that any changes to the semantics or format of the state
file after this commit will require a new state file version 5.

The result of this is that users of this Terraform version will be able
to share remote state with users of future versions, and all users will
be able to read and write state. This will be true until the next major
state file version is required.

This does not affect users of previous versions of Terraform, which will
continue to refuse to read state written by later versions.
2020-10-28 16:52:35 -04:00
James Bardin 0b31ffa587 use a single log writer
Use a single log writer instance for all std library logging.

Setup the std log writer in the logging package, and remove boilerplate
from test packages.
2020-10-19 14:29:54 -04:00
James Bardin 6ca477f042 move helper/logging to internal
remove a dead code file too
2020-10-19 14:27:53 -04:00
Pam Selle 923f9f226d Update some comments in states package 2020-10-07 10:57:25 -04:00
Kristin Laemmert 3933cbd491
remove LegacyProvider (#26433) 2020-10-05 08:33:49 -04:00
Pam Selle 3e7be13dff Update ordering for marking/unmarking and asserting plan valid
Update when we unmark objects so we can assert the plan is valid,
and process UnknownAsNull on the unmarked value
2020-10-02 13:03:11 -04:00
Pam Selle 0a02e7040f
Store sensitive attribute paths in state (#26338)
* Add creation test and simplify in-place test

* Add deletion test

* Start adding marking from state

Start storing paths that should be marked
when pulled out of state. Implements deep
copy for attr paths. This commit also includes some
comment noise from investigations, and fixing the diff test

* Fix apply stripping marks

* Expand diff tests

* Basic apply test

* Update comments on equality checks to clarify current understanding

* Add JSON serialization for sensitive paths

We need to serialize a slice of cty.Path values to be used to re-mark
the sensitive values of a resource instance when loading the state file.
Paths consist of a list of steps, each of which may be either getting an
attribute value by name, or indexing into a collection by string or
number.

To serialize these without building a complex parser for a compact
string form, we render a nested array of small objects, like so:

[
  [
    { type: "get_attr", value: "foo" },
    { type: "index", value: { "type": "number", "value": 2 } }
  ]
]

The above example is equivalent to a path `foo[2]`.

* Format diffs with map types

Comparisons need unmarked values to operate on,
so create unmarked values for those operations. Additionally,
change diff to cover map types

* Remove debugging printing

* Fix bug with marking non-sensitive values

When pulling a sensitive value from state,
we were previously using those marks to remark
the planned new value, but that new value
might *not* be sensitive, so let's not do that

* Fix apply test

Apply was not passing the second state
through to the third pass at apply

* Consistency in checking for length of paths vs inspecting into value

* In apply, don't mark with before paths

* AttrPaths test coverage for DeepCopy

* Revert format changes

Reverts format changes in format/diff for this
branch so those changes can be discussed on a separate PR

* Refactor name of AttrPaths to AttrSensitivePaths

* Rename AttributePaths/attributePaths for naming consistency

Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
2020-09-24 12:40:17 -04:00
James Bardin 53a8e321eb remove mod=vendor in statemgr test 2020-09-24 08:44:49 -04:00
Pam Selle 843ed8911b Don't save PathValueMarks on instance_object 2020-09-10 11:04:17 -04:00
Pam Selle bc55b6a28b Use UnmarkDeepWithPaths and MarkWithPaths
Updates existing code to use the new Value
methods for unmarking/marking and removes
panics/workarounds in cty marshall methods
2020-09-10 11:04:17 -04:00
Pam Selle 2c35869a32 Fix bug for force push for backends besides the remote backend
In refactoring the force push code when implementing force push
for the Terraform remote backend, a bug was introduced that
meant that backends that don't implement the EnableForcePush
method would still have their state validated. This commit
fixes that, and adds test coverage such that there is a separate
mockRemoteClient that has this method implemented.
2020-09-10 09:13:57 -04:00
Alisdair McDiarmid e02e53ed4d states: Add roundtrip v3->v4 with builtin provider 2020-08-14 15:40:48 -04:00
Kristin Laemmert 95eca06782
statemgr: add a NewUnlockErrorFull state manager for tests (#25823)
* statemgr: add a NewUnlockErrorFull state manager for tests

I've frequently needed to coerce Unlock() errors for tests and it's been
awkward and fraught every time, so I decided to add a full state manger
that returns *mostly* errors. I intend to use this in conjunction with
the clistate.Locker interface, which first calls Lock() (to block if the
mutex is in use) at the start of Unlock(), so Lock() rather awkwardly needed to succeed.
2020-08-14 14:14:51 -04:00
Kristin Laemmert 5964e928dc
states/statefile: consistently sort resources across modules (#25498)
If a statefile had resources with the same name in different modules,
the sort order could be inconsistent between state refreshes. This adds
the module to the Less() function used in sorting and a minimal test to
verify consistent ordering.
2020-08-11 11:50:07 -04:00
Kristin Laemmert 6621501ae3
state: remove deprecated state package (#25490)
Most of the state package has been deprecated by the states package.
This PR replaces all the references to the old state package that
can be done simply - the low-hanging fruit.

* states: move state.Locker to statemgr

The state.Locker interface was a wrapper around a statemgr.Full, so
moving this was relatively straightforward.

* command: remove unnecessary use of state package for writing local terraform state files

* move state.LocalState into terraform package

state.LocalState is responsible for managing terraform.States, so it
made sense (to me) to move it into the terraform package.

* slight change of heart: move state.LocalState into clistate instead of
terraform
2020-08-11 11:43:01 -04:00
Kristin Laemmert 58bcc2e9bb
addrs: detect builtin provider when parsing legacy provider string (#25154)
* addrs: detect builtin provider when parsing legacy provider string

The ParseLegacyAbsProviderConfig was not detecting builtin providers
("terraform"), which caused issues for all users with 0.12 state and
the "terraform_remote_state" data source. Since "terraform" is the only
built-in provider this adds a very simple check to the parser so it
properly returns the builtin FQN.

* add tests to the addrs package
2020-06-08 08:27:36 -04:00
James Bardin 08128fe775 add omitempty to create_before_destroy
no need to litter the state file with "create_before_destroy":false
2020-05-18 11:31:36 -04:00
James Bardin 9fe87fe520 remove unused depends_on field and add test
The depends_on fields was deprecated during 0.12, and is no longer used.

Add a roundtrip test for the create_before_destroy status.
2020-05-14 15:46:08 -04:00
James Bardin 23f5fc3e5a thread create_before_destroy to the state file 2020-05-14 15:46:08 -04:00
James Bardin 15a95031e5 remove a few traces of states.EachMode 2020-04-30 09:22:15 -04:00
James Bardin f915c5d957 remove EachMode from resource state
Due to the fact that resources can transition between each modes, trying
to track the mode for a resource as a whole in state doesn't work,
because there may be instances with a mode different from the resource
as a whole. This is difficult for core to track, as this metadata being
changed as a side effect from multiple places often causes core to see
the incorrect mode when evaluating instances.

Since core can always determine the correct mode to evaluate from the
configuration, we don't need to interrogate the state to know the mode.
Once core no longer needs to reference EachMode from states, the
resource state can simply be a container for instances, and doesn't need
to try and track the "current" mode.
2020-04-30 09:22:14 -04:00
James Bardin 6c0f7703a6
Merge pull request #24697 from hashicorp/jbardin/get-module-data
Always return all module instances during evaluation
2020-04-22 09:49:45 -04:00
James Bardin 2c4c027a97 Add ModuleOutputs method to states
In order to efficiently build the module objects for evaluation, we need
to collect the outputs from a set of module instances. The ModuleOutputs
method will return a copy of the state outputs, while not requiring the
unnecessary copying of each entire module.
2020-04-13 17:59:09 -04:00
James Bardin e9eb8e04cc add AbsOutputAddrs to state outputs
We need all module instance outputs to build the objects for evaluation,
but there is no need to copy all the resource instances along with that.
This allows us to only return the output states, with enough information
to connect them with their module instances.
2020-04-13 16:37:59 -04:00
Mikhail Mazurskiy c9ce1f0c1f
Fix typos (#24531) 2020-04-13 08:34:40 -04:00
James Bardin a0f92f9d6a remove extra brackets in state string output 2020-04-09 15:36:16 -04:00
James Bardin 3ee9cf49ce missing wg.Wait in concurrent test 2020-04-08 10:55:20 -04:00
Kristin Laemmert c4873778c8 Mildwonkey/tests (#24548)
* helper/resource: remove provider resolver test
* repl tests passing
* helper/resource: add some extra shimming to ShimLegacyState

Some of the tests in helper/resource have to shim between legacy and
modern states. Terraform expects modern states to have the Provider set
for each resource (and not be a legacy provider). This PR adds a wrapper
around terraform.ShimLegacyState which iterates through the resources
in the state (after the shim) and adds the provider FQN.
2020-04-06 09:24:23 -07:00
Kristin Laemmert d95667f264 states/version3 upgrade: revert provider FQN changes (#24504)
This commit reverts an earlier change which automatically converted
provider strings to legacy provider FQNs. It has become apparent that a
state upgrade step will be required before upgrading to v0.13.
2020-04-06 09:24:23 -07:00
Martin Atkins 4061cbed38 internal/getproviders: A new shared model for provider requirements
We've been using the models from the "moduledeps" package to represent our
provider dependencies everywhere since the idea of provider dependencies
was introduced in Terraform 0.10, but that model is not convenient to use
for any use-case other than the "terraform providers" command that needs
individual-module-level detail.

To make things easier for new codepaths working with the new-style
provider installer, here we introduce a new model type
getproviders.Requirements which is based on the type the new installer was
already taking as its input. We have new methods in the states, configs,
and earlyconfig packages to produce values of this type, and a helper
to merge Requirements together so we can combine config-derived and
state-derived requirements together during installation.

The advantage of this new model over the moduledeps one is that all of
recursive module walking is done up front and we produce a simple, flat
structure that is more convenient for the main use-cases of selecting
providers for installation and then finding providers in the local cache
to use them for other operations.

This new model is _not_ suitable for implementing "terraform providers"
because it does not retain module-specific requirement details. Therefore
we will likely keep using moduledeps for "terraform providers" for now,
and then possibly at a later time consider specializing the moduledeps
logic for only what "terraform providers" needs, because it seems to be
the only use-case that needs to retain that level of detail.
2020-03-27 09:01:32 -07:00
James Bardin b3fc0dab94 use addrs.ConfigResource for dependency tracking
We can't get module instances during transformation, so we need to
reduce the Dependencies to using `addrs.ConfigResource` for now.
2020-03-25 17:03:06 -04:00
Kristin Laemmert ed1aebbeda
terraform: large refactor to use Provider from configs.Resource (#24396)
* terraform: large refactor to use Provider from configs.Resource

configs.Resource.ImpliedProvider() now returns a string; it is the
callers' responsibility to turn that into an addrs.Provider if needed.

GraphNodeProviderConsumer ProvidedBy() no longer returns nil (reverting
to earlier, pre-provider-fqn behavior): it will return either the
provider set in config, provider set in state, or the default provider.
2020-03-18 08:58:20 -04:00
James Bardin ea51b790bc states.Resource needs to record its module too
The ModuleInstance is known while building the state resource, but it's
not recorded. Since a resource may be retrieved via a ConfigResource
address, we need to know from which module instance it was loaded.
2020-03-16 11:16:23 -04:00
James Bardin 3729e6a705 update MaybeFixUpResourceInstanceAddressForCount
We need to fixup count by config path, not by absolute path, so update
MaybeFixUpResourceInstanceAddressForCount to work with
addrs.ConfigResource.
2020-03-16 11:16:23 -04:00
James Bardin e13eecbc5b finish provider ModuleInstance replacement 2020-03-11 14:19:52 -04:00
James Bardin 8497adcb6e AbsProviderConfig to use addrs.Module
Change ModuleInstance to Module in AbsProviderConfig, because providers
need to be handled before module expansion, and should not be used
defined inside an expanded module at all.

Renaming of the addrs type can happen later, when there's less work
in-flight around provider configuration.
2020-03-10 20:25:44 -04:00
James Bardin 654e880bb8
Merge pull request #24084 from hashicorp/jbardin/cbd-instance-state
Add CreateBeforeDestroy to instance state
2020-03-09 13:16:29 -04:00
Alisdair McDiarmid f54e2a62bf states: Fix coverage blips in statemgr/filesystem
Something non-deterministic in the test suite is causing a coverage blip
in a line in the filesystem state manager. This commit adds a test which
specifically covers that line, which hopefully pleases the Codecov robot.
2020-03-06 17:55:28 -05:00
James Bardin dd8ab5812e add CreateBeforeDestroy to state
Added to src and object, but still need serialization and tests.
2020-02-13 21:04:56 -05:00