Commit Graph

112 Commits

Author SHA1 Message Date
James Bardin 9fe87fe520 remove unused depends_on field and add test
The depends_on fields was deprecated during 0.12, and is no longer used.

Add a roundtrip test for the create_before_destroy status.
2020-05-14 15:46:08 -04:00
James Bardin 23f5fc3e5a thread create_before_destroy to the state file 2020-05-14 15:46:08 -04:00
James Bardin 15a95031e5 remove a few traces of states.EachMode 2020-04-30 09:22:15 -04:00
James Bardin f915c5d957 remove EachMode from resource state
Due to the fact that resources can transition between each modes, trying
to track the mode for a resource as a whole in state doesn't work,
because there may be instances with a mode different from the resource
as a whole. This is difficult for core to track, as this metadata being
changed as a side effect from multiple places often causes core to see
the incorrect mode when evaluating instances.

Since core can always determine the correct mode to evaluate from the
configuration, we don't need to interrogate the state to know the mode.
Once core no longer needs to reference EachMode from states, the
resource state can simply be a container for instances, and doesn't need
to try and track the "current" mode.
2020-04-30 09:22:14 -04:00
James Bardin 6c0f7703a6
Merge pull request #24697 from hashicorp/jbardin/get-module-data
Always return all module instances during evaluation
2020-04-22 09:49:45 -04:00
James Bardin 2c4c027a97 Add ModuleOutputs method to states
In order to efficiently build the module objects for evaluation, we need
to collect the outputs from a set of module instances. The ModuleOutputs
method will return a copy of the state outputs, while not requiring the
unnecessary copying of each entire module.
2020-04-13 17:59:09 -04:00
James Bardin e9eb8e04cc add AbsOutputAddrs to state outputs
We need all module instance outputs to build the objects for evaluation,
but there is no need to copy all the resource instances along with that.
This allows us to only return the output states, with enough information
to connect them with their module instances.
2020-04-13 16:37:59 -04:00
Mikhail Mazurskiy c9ce1f0c1f
Fix typos (#24531) 2020-04-13 08:34:40 -04:00
James Bardin a0f92f9d6a remove extra brackets in state string output 2020-04-09 15:36:16 -04:00
James Bardin 3ee9cf49ce missing wg.Wait in concurrent test 2020-04-08 10:55:20 -04:00
Kristin Laemmert c4873778c8 Mildwonkey/tests (#24548)
* helper/resource: remove provider resolver test
* repl tests passing
* helper/resource: add some extra shimming to ShimLegacyState

Some of the tests in helper/resource have to shim between legacy and
modern states. Terraform expects modern states to have the Provider set
for each resource (and not be a legacy provider). This PR adds a wrapper
around terraform.ShimLegacyState which iterates through the resources
in the state (after the shim) and adds the provider FQN.
2020-04-06 09:24:23 -07:00
Kristin Laemmert d95667f264 states/version3 upgrade: revert provider FQN changes (#24504)
This commit reverts an earlier change which automatically converted
provider strings to legacy provider FQNs. It has become apparent that a
state upgrade step will be required before upgrading to v0.13.
2020-04-06 09:24:23 -07:00
Martin Atkins 4061cbed38 internal/getproviders: A new shared model for provider requirements
We've been using the models from the "moduledeps" package to represent our
provider dependencies everywhere since the idea of provider dependencies
was introduced in Terraform 0.10, but that model is not convenient to use
for any use-case other than the "terraform providers" command that needs
individual-module-level detail.

To make things easier for new codepaths working with the new-style
provider installer, here we introduce a new model type
getproviders.Requirements which is based on the type the new installer was
already taking as its input. We have new methods in the states, configs,
and earlyconfig packages to produce values of this type, and a helper
to merge Requirements together so we can combine config-derived and
state-derived requirements together during installation.

The advantage of this new model over the moduledeps one is that all of
recursive module walking is done up front and we produce a simple, flat
structure that is more convenient for the main use-cases of selecting
providers for installation and then finding providers in the local cache
to use them for other operations.

This new model is _not_ suitable for implementing "terraform providers"
because it does not retain module-specific requirement details. Therefore
we will likely keep using moduledeps for "terraform providers" for now,
and then possibly at a later time consider specializing the moduledeps
logic for only what "terraform providers" needs, because it seems to be
the only use-case that needs to retain that level of detail.
2020-03-27 09:01:32 -07:00
James Bardin b3fc0dab94 use addrs.ConfigResource for dependency tracking
We can't get module instances during transformation, so we need to
reduce the Dependencies to using `addrs.ConfigResource` for now.
2020-03-25 17:03:06 -04:00
Kristin Laemmert ed1aebbeda
terraform: large refactor to use Provider from configs.Resource (#24396)
* terraform: large refactor to use Provider from configs.Resource

configs.Resource.ImpliedProvider() now returns a string; it is the
callers' responsibility to turn that into an addrs.Provider if needed.

GraphNodeProviderConsumer ProvidedBy() no longer returns nil (reverting
to earlier, pre-provider-fqn behavior): it will return either the
provider set in config, provider set in state, or the default provider.
2020-03-18 08:58:20 -04:00
James Bardin ea51b790bc states.Resource needs to record its module too
The ModuleInstance is known while building the state resource, but it's
not recorded. Since a resource may be retrieved via a ConfigResource
address, we need to know from which module instance it was loaded.
2020-03-16 11:16:23 -04:00
James Bardin 3729e6a705 update MaybeFixUpResourceInstanceAddressForCount
We need to fixup count by config path, not by absolute path, so update
MaybeFixUpResourceInstanceAddressForCount to work with
addrs.ConfigResource.
2020-03-16 11:16:23 -04:00
James Bardin e13eecbc5b finish provider ModuleInstance replacement 2020-03-11 14:19:52 -04:00
James Bardin 8497adcb6e AbsProviderConfig to use addrs.Module
Change ModuleInstance to Module in AbsProviderConfig, because providers
need to be handled before module expansion, and should not be used
defined inside an expanded module at all.

Renaming of the addrs type can happen later, when there's less work
in-flight around provider configuration.
2020-03-10 20:25:44 -04:00
James Bardin 654e880bb8
Merge pull request #24084 from hashicorp/jbardin/cbd-instance-state
Add CreateBeforeDestroy to instance state
2020-03-09 13:16:29 -04:00
Alisdair McDiarmid f54e2a62bf states: Fix coverage blips in statemgr/filesystem
Something non-deterministic in the test suite is causing a coverage blip
in a line in the filesystem state manager. This commit adds a test which
specifically covers that line, which hopefully pleases the Codecov robot.
2020-03-06 17:55:28 -05:00
James Bardin dd8ab5812e add CreateBeforeDestroy to state
Added to src and object, but still need serialization and tests.
2020-02-13 21:04:56 -05:00
Kristin Laemmert 47a16b0937
addrs: embed Provider in AbsProviderConfig instead of Type
a large refactor to addrs.AbsProviderConfig, embedding the addrs.Provider instead of a Type string. I've added and updated tests, added some Legacy functions to support older state formats and shims, and added a normalization step when reading v4 (current) state files (not the added tests under states/statefile/roundtrip which work with both current and legacy-style AbsProviderConfig strings).

The remaining 'fixme' and 'todo' comments are mostly going to be addressed in a subsequent PR and involve looking up a given local provider config's FQN. This is fine for now as we are only working with default assumption.
2020-02-13 15:32:58 -05:00
Martin Atkins 8b511524d6
Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978)
* Introduce "Local" terminology for non-absolute provider config addresses

In a future change AbsProviderConfig and LocalProviderConfig are going to
become two entirely distinct types, rather than Abs embedding Local as
written here. This naming change is in preparation for that subsequent
work, which will also include introducing a new "ProviderConfig" type
that is an interface that AbsProviderConfig and LocalProviderConfig both
implement.

This is intended to be largely just a naming change to get started, so
we can deal with all of the messy renaming. However, this did also require
a slight change in modeling where the Resource.DefaultProviderConfig
method has become Resource.DefaultProvider returning a Provider address
directly, because this method doesn't have enough information to construct
a true and accurate LocalProviderConfig -- it would need to refer to the
configuration to know what this module is calling the provider it has
selected.

In order to leave a trail to follow for subsequent work, all of the
changes here are intended to ensure that remaining work will become
obvious via compile-time errors when all of the following changes happen:
- The concept of "legacy" provider addresses is removed from the addrs
  package, including removing addrs.NewLegacyProvider and
  addrs.Provider.LegacyString.
- addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded
  in it and has an addrs.Provider and a string alias directly instead.
- The provider-schema-handling parts of Terraform core are updated to
  work with addrs.Provider to identify providers, rather than legacy
  strings.

In particular, there are still several codepaths here making legacy
provider address assumptions (in order to limit the scope of this change)
but I've made sure each one is doing something that relies on at least
one of the above changes not having been made yet.

* addrs: ProviderConfig interface

In a (very) few special situations in the main "terraform" package we need
to make runtime decisions about whether a provider config is absolute
or local.

We currently do that by exploiting the fact that AbsProviderConfig has
LocalProviderConfig nested inside of it and so in the local case we can
just ignore the wrapping AbsProviderConfig and use the embedded value.

In a future change we'll be moving away from that embedding and making
these two types distinct in order to represent that mapping between them
requires consulting a lookup table in the configuration, and so here we
introduce a new interface type ProviderConfig that can represent either
AbsProviderConfig or LocalProviderConfig decided dynamically at runtime.

This also includes the Config.ResolveAbsProviderAddr method that will
eventually be responsible for that local-to-absolute translation, so
that callers with access to the configuration can normalize to an
addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's
currently unused because existing callers are still relying on the
simplistic structural transform, but we'll switch them over in a later
commit.

* rename LocalType to LocalName

Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-01-31 08:23:07 -05:00
Kristin Laemmert 6541775ce4
addrs: roll back change to Type field in ProviderConfig (#23937) 2020-01-28 08:13:30 -05:00
Kristin Laemmert 67fc4dd5a1 configs: move ProviderConfigCompact[Str] from addrs to configs
The configs package is aware of provider name and type (which are the
same thing today, but expected to be two different things in a future
release), and should be the source of truth for a provider config
address.
2019-12-09 08:30:08 -05:00
Kristin Laemmert e3416124cc
addrs: replace "Type string" with "Type Provider" in ProviderConfig
* huge change to weave new addrs.Provider into addrs.ProviderConfig
* terraform: do not include an empty string in the returned Providers /
Provisioners
- Fixed a minor bug where results included an extra empty string
2019-12-06 08:00:18 -05:00
Pam Selle e4d2469698 Update comment 2019-12-04 12:08:34 -05:00
Pam Selle de953eca6f Fixup for allowing empty lists/sets of resources 2019-12-03 16:29:01 -05:00
Pam Selle 19b408e053 Only need this one call, don't call the meta func 2019-12-03 14:27:18 -05:00
Pam Selle 61ce1cb3ad Fix provider related tests 2019-12-03 14:27:18 -05:00
Pam Selle 01da7b4f7f Tested and confirmed we cannot delete this, so removed TODO 2019-12-03 14:27:18 -05:00
Pam Selle 87fdcd0064 Add a new method cause doing things in module is weird 2019-12-03 14:27:18 -05:00
Pam Selle 3b4382082d More comments and names 2019-12-03 14:27:18 -05:00
Pam Selle fd34def9dc More comment updates, a lil refactor 2019-12-03 14:27:18 -05:00
Pam Selle ca9da51516 Cleanup 2019-12-03 14:27:18 -05:00
Pam Selle 9482bb8eb0 Comment cleanup 2019-12-03 14:27:18 -05:00
Pam Selle d144b83d50 This works, with lots of noise 2019-12-03 14:27:18 -05:00
Martin Atkins 7cf782c1bf states/statefile: Better error messages for some unsupported v3 cases
There are a few situations that we've seen arise quite commonly for folks
upgrading from Terraform 0.11 to 0.12. These particular problems are not
things that Terraform 0.12 can fix automatically during upgrading, but
we can at least give some better feedback to users that they ought to be
addressed _before_ upgrading.

The provider address problem is already detected and flagged by the
"terraform 0.11checklist" command that folks should run as part of their
upgrade process, but the module address problem is not something we
noticed was lacking validation in 0.11 and so the checklist tool doesn't
cover it. Due to the lack of coverage in the checklist tool, this commit
also includes an additional section in the upgrade guide that mentions
the problem and gives instructions on how to address it.
2019-11-21 09:18:49 -08:00
Martin Atkins 1ae47ae314 states/statefile: tolerate and ignore invalid depends_on in version 3
The state refactoring command "terraform state mv" in Terraform 0.11 does
not update existing dependency addresses recorded in the state when it
moves objects around, and Terraform only updates the dependency addresses
in the state when it performs a full update on a resource instance, and
so it's a common problem for folks updating from Terraform 0.11 with
resource names that are not valid identifiers to run into state upgrade
errors even though they have followed the instructions produced by
"terraform 0.12checklist".

Dependencies are synced from config during every refresh walk anyway, so
in practice we can get away with just discarding invalid dependency
addresses and letting the refresh walk update them. In practice these
addresses are unlikely to be pointing at a resource that actually exists
anyway, because if so Terraform 0.12's configuration parser wouldn't be
able to interpret it.

Discarding invalid dependency addresses allows the state upgrade to
complete successfully in such cases and thus gives the refresh step an
opportunity to repair the problem.
2019-11-21 09:18:49 -08:00
aqche bcaf7f7ba7 states/statemgr: Properly return error when Unlock detects invalid lock id 2019-11-11 15:24:53 -08:00
James Bardin 5a0a0020a0 read+write the new dependencies in the statefile
The test fixture did not like having modules when using the generic json
map, so read and compare the states in the final *File datastructure.
2019-11-07 17:49:03 -05:00
James Bardin 2c3c011f20 change state dependencies to AbsResource addrs
We need to be able to reference all possible dependencies for ordering
when the configuration is no longer present, which means that absolute
addresses must be used. Since this is only to recreate the proper
ordering for instance destruction, only resources addresses need to be
listed rather than individual instance addresses.
2019-10-30 17:25:53 -04:00
Radek Simko 7860f55e4f
Version tools per Go convention under tools.go 2019-10-17 22:23:39 +02:00
Radek Simko 95cbf1c282
Fix copy-paste errors in test data 2019-08-21 11:45:06 +01:00
Pam Selle e6817f6319 Reordering, comment update 2019-08-13 17:22:14 -04:00
Alex Pilon 4bf43efcfd
move hcl2shim package to configs 2019-08-06 19:58:58 -04:00
Martin Atkins d41fc8d517 states/statefile: additional context for some v3 upgrade error messages
Some of our errors returned here were lacking context about what part of
the file was problematic, which led to some useless error reporting for
some real-world situations that this upgrade process doesn't seem to be
catching.

Here we add additional context to those error cases, as a step towards
tracking down exactly which upgrade cases are missing here so that we can
potentially fix them in a subsequent release.
2019-07-18 18:20:53 -07:00
James Bardin 2988e373d6
Merge pull request #21484 from minamijoyo/fix-statev2-hash-decode
states/statefile: decode backend hash as uint64
2019-06-10 17:52:33 -04:00
James Bardin ac2219ba6e don't lose Private state data during copy
Fix the scope of the private data copy in DeepCopy.

Make sure Dependencies matches nil vs empty so that Equal compares
correctly between copied states
2019-06-05 19:22:46 -04:00