Commit Graph

103 Commits

Author SHA1 Message Date
James Bardin 3ee9cf49ce missing wg.Wait in concurrent test 2020-04-08 10:55:20 -04:00
Kristin Laemmert c4873778c8 Mildwonkey/tests (#24548)
* helper/resource: remove provider resolver test
* repl tests passing
* helper/resource: add some extra shimming to ShimLegacyState

Some of the tests in helper/resource have to shim between legacy and
modern states. Terraform expects modern states to have the Provider set
for each resource (and not be a legacy provider). This PR adds a wrapper
around terraform.ShimLegacyState which iterates through the resources
in the state (after the shim) and adds the provider FQN.
2020-04-06 09:24:23 -07:00
Kristin Laemmert d95667f264 states/version3 upgrade: revert provider FQN changes (#24504)
This commit reverts an earlier change which automatically converted
provider strings to legacy provider FQNs. It has become apparent that a
state upgrade step will be required before upgrading to v0.13.
2020-04-06 09:24:23 -07:00
Martin Atkins 4061cbed38 internal/getproviders: A new shared model for provider requirements
We've been using the models from the "moduledeps" package to represent our
provider dependencies everywhere since the idea of provider dependencies
was introduced in Terraform 0.10, but that model is not convenient to use
for any use-case other than the "terraform providers" command that needs
individual-module-level detail.

To make things easier for new codepaths working with the new-style
provider installer, here we introduce a new model type
getproviders.Requirements which is based on the type the new installer was
already taking as its input. We have new methods in the states, configs,
and earlyconfig packages to produce values of this type, and a helper
to merge Requirements together so we can combine config-derived and
state-derived requirements together during installation.

The advantage of this new model over the moduledeps one is that all of
recursive module walking is done up front and we produce a simple, flat
structure that is more convenient for the main use-cases of selecting
providers for installation and then finding providers in the local cache
to use them for other operations.

This new model is _not_ suitable for implementing "terraform providers"
because it does not retain module-specific requirement details. Therefore
we will likely keep using moduledeps for "terraform providers" for now,
and then possibly at a later time consider specializing the moduledeps
logic for only what "terraform providers" needs, because it seems to be
the only use-case that needs to retain that level of detail.
2020-03-27 09:01:32 -07:00
James Bardin b3fc0dab94 use addrs.ConfigResource for dependency tracking
We can't get module instances during transformation, so we need to
reduce the Dependencies to using `addrs.ConfigResource` for now.
2020-03-25 17:03:06 -04:00
Kristin Laemmert ed1aebbeda
terraform: large refactor to use Provider from configs.Resource (#24396)
* terraform: large refactor to use Provider from configs.Resource

configs.Resource.ImpliedProvider() now returns a string; it is the
callers' responsibility to turn that into an addrs.Provider if needed.

GraphNodeProviderConsumer ProvidedBy() no longer returns nil (reverting
to earlier, pre-provider-fqn behavior): it will return either the
provider set in config, provider set in state, or the default provider.
2020-03-18 08:58:20 -04:00
James Bardin ea51b790bc states.Resource needs to record its module too
The ModuleInstance is known while building the state resource, but it's
not recorded. Since a resource may be retrieved via a ConfigResource
address, we need to know from which module instance it was loaded.
2020-03-16 11:16:23 -04:00
James Bardin 3729e6a705 update MaybeFixUpResourceInstanceAddressForCount
We need to fixup count by config path, not by absolute path, so update
MaybeFixUpResourceInstanceAddressForCount to work with
addrs.ConfigResource.
2020-03-16 11:16:23 -04:00
James Bardin e13eecbc5b finish provider ModuleInstance replacement 2020-03-11 14:19:52 -04:00
James Bardin 8497adcb6e AbsProviderConfig to use addrs.Module
Change ModuleInstance to Module in AbsProviderConfig, because providers
need to be handled before module expansion, and should not be used
defined inside an expanded module at all.

Renaming of the addrs type can happen later, when there's less work
in-flight around provider configuration.
2020-03-10 20:25:44 -04:00
James Bardin 654e880bb8
Merge pull request #24084 from hashicorp/jbardin/cbd-instance-state
Add CreateBeforeDestroy to instance state
2020-03-09 13:16:29 -04:00
Alisdair McDiarmid f54e2a62bf states: Fix coverage blips in statemgr/filesystem
Something non-deterministic in the test suite is causing a coverage blip
in a line in the filesystem state manager. This commit adds a test which
specifically covers that line, which hopefully pleases the Codecov robot.
2020-03-06 17:55:28 -05:00
James Bardin dd8ab5812e add CreateBeforeDestroy to state
Added to src and object, but still need serialization and tests.
2020-02-13 21:04:56 -05:00
Kristin Laemmert 47a16b0937
addrs: embed Provider in AbsProviderConfig instead of Type
a large refactor to addrs.AbsProviderConfig, embedding the addrs.Provider instead of a Type string. I've added and updated tests, added some Legacy functions to support older state formats and shims, and added a normalization step when reading v4 (current) state files (not the added tests under states/statefile/roundtrip which work with both current and legacy-style AbsProviderConfig strings).

The remaining 'fixme' and 'todo' comments are mostly going to be addressed in a subsequent PR and involve looking up a given local provider config's FQN. This is fine for now as we are only working with default assumption.
2020-02-13 15:32:58 -05:00
Martin Atkins 8b511524d6
Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978)
* Introduce "Local" terminology for non-absolute provider config addresses

In a future change AbsProviderConfig and LocalProviderConfig are going to
become two entirely distinct types, rather than Abs embedding Local as
written here. This naming change is in preparation for that subsequent
work, which will also include introducing a new "ProviderConfig" type
that is an interface that AbsProviderConfig and LocalProviderConfig both
implement.

This is intended to be largely just a naming change to get started, so
we can deal with all of the messy renaming. However, this did also require
a slight change in modeling where the Resource.DefaultProviderConfig
method has become Resource.DefaultProvider returning a Provider address
directly, because this method doesn't have enough information to construct
a true and accurate LocalProviderConfig -- it would need to refer to the
configuration to know what this module is calling the provider it has
selected.

In order to leave a trail to follow for subsequent work, all of the
changes here are intended to ensure that remaining work will become
obvious via compile-time errors when all of the following changes happen:
- The concept of "legacy" provider addresses is removed from the addrs
  package, including removing addrs.NewLegacyProvider and
  addrs.Provider.LegacyString.
- addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded
  in it and has an addrs.Provider and a string alias directly instead.
- The provider-schema-handling parts of Terraform core are updated to
  work with addrs.Provider to identify providers, rather than legacy
  strings.

In particular, there are still several codepaths here making legacy
provider address assumptions (in order to limit the scope of this change)
but I've made sure each one is doing something that relies on at least
one of the above changes not having been made yet.

* addrs: ProviderConfig interface

In a (very) few special situations in the main "terraform" package we need
to make runtime decisions about whether a provider config is absolute
or local.

We currently do that by exploiting the fact that AbsProviderConfig has
LocalProviderConfig nested inside of it and so in the local case we can
just ignore the wrapping AbsProviderConfig and use the embedded value.

In a future change we'll be moving away from that embedding and making
these two types distinct in order to represent that mapping between them
requires consulting a lookup table in the configuration, and so here we
introduce a new interface type ProviderConfig that can represent either
AbsProviderConfig or LocalProviderConfig decided dynamically at runtime.

This also includes the Config.ResolveAbsProviderAddr method that will
eventually be responsible for that local-to-absolute translation, so
that callers with access to the configuration can normalize to an
addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's
currently unused because existing callers are still relying on the
simplistic structural transform, but we'll switch them over in a later
commit.

* rename LocalType to LocalName

Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-01-31 08:23:07 -05:00
Kristin Laemmert 6541775ce4
addrs: roll back change to Type field in ProviderConfig (#23937) 2020-01-28 08:13:30 -05:00
Kristin Laemmert 67fc4dd5a1 configs: move ProviderConfigCompact[Str] from addrs to configs
The configs package is aware of provider name and type (which are the
same thing today, but expected to be two different things in a future
release), and should be the source of truth for a provider config
address.
2019-12-09 08:30:08 -05:00
Kristin Laemmert e3416124cc
addrs: replace "Type string" with "Type Provider" in ProviderConfig
* huge change to weave new addrs.Provider into addrs.ProviderConfig
* terraform: do not include an empty string in the returned Providers /
Provisioners
- Fixed a minor bug where results included an extra empty string
2019-12-06 08:00:18 -05:00
Pam Selle e4d2469698 Update comment 2019-12-04 12:08:34 -05:00
Pam Selle de953eca6f Fixup for allowing empty lists/sets of resources 2019-12-03 16:29:01 -05:00
Pam Selle 19b408e053 Only need this one call, don't call the meta func 2019-12-03 14:27:18 -05:00
Pam Selle 61ce1cb3ad Fix provider related tests 2019-12-03 14:27:18 -05:00
Pam Selle 01da7b4f7f Tested and confirmed we cannot delete this, so removed TODO 2019-12-03 14:27:18 -05:00
Pam Selle 87fdcd0064 Add a new method cause doing things in module is weird 2019-12-03 14:27:18 -05:00
Pam Selle 3b4382082d More comments and names 2019-12-03 14:27:18 -05:00
Pam Selle fd34def9dc More comment updates, a lil refactor 2019-12-03 14:27:18 -05:00
Pam Selle ca9da51516 Cleanup 2019-12-03 14:27:18 -05:00
Pam Selle 9482bb8eb0 Comment cleanup 2019-12-03 14:27:18 -05:00
Pam Selle d144b83d50 This works, with lots of noise 2019-12-03 14:27:18 -05:00
Martin Atkins 7cf782c1bf states/statefile: Better error messages for some unsupported v3 cases
There are a few situations that we've seen arise quite commonly for folks
upgrading from Terraform 0.11 to 0.12. These particular problems are not
things that Terraform 0.12 can fix automatically during upgrading, but
we can at least give some better feedback to users that they ought to be
addressed _before_ upgrading.

The provider address problem is already detected and flagged by the
"terraform 0.11checklist" command that folks should run as part of their
upgrade process, but the module address problem is not something we
noticed was lacking validation in 0.11 and so the checklist tool doesn't
cover it. Due to the lack of coverage in the checklist tool, this commit
also includes an additional section in the upgrade guide that mentions
the problem and gives instructions on how to address it.
2019-11-21 09:18:49 -08:00
Martin Atkins 1ae47ae314 states/statefile: tolerate and ignore invalid depends_on in version 3
The state refactoring command "terraform state mv" in Terraform 0.11 does
not update existing dependency addresses recorded in the state when it
moves objects around, and Terraform only updates the dependency addresses
in the state when it performs a full update on a resource instance, and
so it's a common problem for folks updating from Terraform 0.11 with
resource names that are not valid identifiers to run into state upgrade
errors even though they have followed the instructions produced by
"terraform 0.12checklist".

Dependencies are synced from config during every refresh walk anyway, so
in practice we can get away with just discarding invalid dependency
addresses and letting the refresh walk update them. In practice these
addresses are unlikely to be pointing at a resource that actually exists
anyway, because if so Terraform 0.12's configuration parser wouldn't be
able to interpret it.

Discarding invalid dependency addresses allows the state upgrade to
complete successfully in such cases and thus gives the refresh step an
opportunity to repair the problem.
2019-11-21 09:18:49 -08:00
aqche bcaf7f7ba7 states/statemgr: Properly return error when Unlock detects invalid lock id 2019-11-11 15:24:53 -08:00
James Bardin 5a0a0020a0 read+write the new dependencies in the statefile
The test fixture did not like having modules when using the generic json
map, so read and compare the states in the final *File datastructure.
2019-11-07 17:49:03 -05:00
James Bardin 2c3c011f20 change state dependencies to AbsResource addrs
We need to be able to reference all possible dependencies for ordering
when the configuration is no longer present, which means that absolute
addresses must be used. Since this is only to recreate the proper
ordering for instance destruction, only resources addresses need to be
listed rather than individual instance addresses.
2019-10-30 17:25:53 -04:00
Radek Simko 7860f55e4f
Version tools per Go convention under tools.go 2019-10-17 22:23:39 +02:00
Radek Simko 95cbf1c282
Fix copy-paste errors in test data 2019-08-21 11:45:06 +01:00
Pam Selle e6817f6319 Reordering, comment update 2019-08-13 17:22:14 -04:00
Alex Pilon 4bf43efcfd
move hcl2shim package to configs 2019-08-06 19:58:58 -04:00
Martin Atkins d41fc8d517 states/statefile: additional context for some v3 upgrade error messages
Some of our errors returned here were lacking context about what part of
the file was problematic, which led to some useless error reporting for
some real-world situations that this upgrade process doesn't seem to be
catching.

Here we add additional context to those error cases, as a step towards
tracking down exactly which upgrade cases are missing here so that we can
potentially fix them in a subsequent release.
2019-07-18 18:20:53 -07:00
James Bardin 2988e373d6
Merge pull request #21484 from minamijoyo/fix-statev2-hash-decode
states/statefile: decode backend hash as uint64
2019-06-10 17:52:33 -04:00
James Bardin ac2219ba6e don't lose Private state data during copy
Fix the scope of the private data copy in DeepCopy.

Make sure Dependencies matches nil vs empty so that Equal compares
correctly between copied states
2019-06-05 19:22:46 -04:00
Masayuki Morita ac83827c90 states/statefile: decode backend hash as uint64
Fixes #21478

In #19688, `terraform.BackendState.Hash` was fixed.
We also need to fix `states/statefile.backendStateV2.Hash`
2019-05-28 23:59:36 +09:00
James Bardin 1e1dcfd95a always write the current state terraform_version
Any time a new state is written, the current terraform version should be
recorded.
2019-05-21 18:55:53 -04:00
Radek Simko 8a6d1d62b6
stringer: Regenerate files with latest version 2019-05-13 15:34:27 +01:00
Kristin Laemmert d7dda4e436
states/statefile: upgrade legacy dependency syntax (#21159)
We used to allow "foo.1" etc as a reference, but now it's "foo[1]".
2019-04-30 14:34:01 -04:00
Paul Tyng ab2d09b0cb
godoc typo 2019-04-17 13:45:58 -04:00
Sander van Harmelen 21cca34716 Fixup the docs for WriteStateForMigration 2019-03-27 17:41:11 +01:00
Sander van Harmelen 57f6e01830 backend/local: preserve serial and lineage on failure
When failing to write the state, the local backend writes the state to a local file called `errrored.tfstate`. Previously it would do so by creating a new state file which would use a new serial and lineage. By exorting the existing state file and directly assigning the new state, the serial and lineage are preserved.
2019-03-27 16:15:16 +01:00
Martin Atkins c39905e1a8 command: Fix various issues in the "terraform state ..." subcommands
In earlier refactoring we updated these commands to support the new
address and state types, but attempted to partially retain the old-style
"StateFilter" abstraction that originally lived in the Terraform package,
even though that was no longer being used for any other functionality.

Unfortunately the adaptation of the existing filtering to the new types
wasn't exact and so these commands ended up having a few bugs that were
not covered by the existing tests.

Since the old StateFilter behavior was the source of various misbehavior
anyway, here it's removed altogether and replaced with some simpler
functions in the state_meta.go file that are tailored to the use-cases of
these sub-commands.

As well as just generally behaving more consistently with the other
parts of Terraform that use the new resource address types, this commit
fixes the following bugs:

- A resource address of aws_instance.foo would previously match an
  resource of that type and name in any module, which disagreed with the
  expected interpretation elsewhere of meaning a single resource in the
  root module.

- The "terraform state mv" command was not supporting moves from a single
  resource address to an indexed address and vice-versa, because the old
  logic didn't need to make that distinction while they are two separate
  address types in the new logic. Now we allow resources that do not have
  count/for_each to be treated as if they are instances for the purposes
  of this command, which is a better match for likely user intent and for
  the old behavior.

Finally, we also clean up a little some of the usage output from these
commands, which hasn't been updated for some time and so had both some
stale information and some inaccurate terminology.
2019-03-18 09:19:55 -07:00
Andriy Borodiychuk a0f3a2daa0 Update doc.go, fix typo (#20529) 2019-03-01 08:59:01 -08:00