Commit Graph

97 Commits

Author SHA1 Message Date
Martin Atkins 4c5b866dea Move registry/ to internal/registry/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 73dda868cc Move backend/ to internal/backend/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 7c6e78bcb0 plans: Track both the previous run and prior states in the plan
Until now we've not really cared much about the state snapshot produced
by the previous Terraform operation, except to use it as a jumping-off
point for our refresh step.

However, we'd like to be able to report to an end-user whenever Terraform
detects a change that occurred outside of Terraform, because that's often
helpful context for understanding why a plan contains changes that don't
seem to have corresponding changes in the configuration.

As part of reporting that we'll need to keep track of the state as it
was before we did any refreshing work, so we can then compare that against
the state after refreshing. To retain enough data to achieve that, the
existing Plan field State is now two fields: PrevRunState and PriorState.

This also includes a very shallow change in the core package to make it
populate something somewhat-reasonable into this field so that integration
tests can function reasonably. However, this shallow implementation isn't
really sufficient for real-world use of PrevRunState because we'll
actually need to update PrevRunState as part of planning in order to
incorporate the results of any provider-specific state upgrades to make
the PrevRunState objects compatible with the current provider schema, or
else our diffs won't be valid. This deeper awareness of PrevRunState in
Terraform Core will follow in a subsequent commit, prior to anything else
making use of Plan.PrevRunState.
2021-05-05 15:11:05 -07:00
Alisdair McDiarmid c5a6aa31d3 cli: Add initial command views abstraction
Terraform supports multiple output formats for several sub-commands.
The default format is user-readable text, but many sub-commands support
a `-json` flag to output a machine-readable format for the result. The
output command also supports a `-raw` flag for a simpler, scripting-
focused machine readable format.

This commit adds a "views" abstraction, intended to help ensure
consistency between the various output formats. This extracts the render
specific code from the command package, and moves it into a views
package. Each command is expected to create an interface for its view,
and one or more implementations of that interface.

By doing so, we separate the concerns of generating the sub-command
result from rendering the result in the specified output format. This
should make it easier to ensure that all output formats will be updated
together when changes occur in the result-generating phase.

There are some other consequences of this restructuring:

- Views now directly access the terminal streams, rather than the
  now-redundant cli.Ui instance;
- With the reorganization of commands, parsing CLI arguments is now the
  responsibility of a separate "arguments" package.

For now, views are added only for the output sub-command, as an example.
Because this command uses code which is shared with the apply and
refresh commands, those are also partially updated.
2021-02-11 15:06:39 -05:00
Jonathan Hall 49ee3d3ef8 Grammar nit: "setup" as a verb should be spelled "set up" 2021-01-26 20:39:11 +01:00
James Bardin 1c58c6ba48 command staticcheck 2020-12-02 13:59:19 -05:00
James Bardin e998882824 remove legacy types from command package 2020-12-02 12:33:18 -05:00
James Bardin 0b31ffa587 use a single log writer
Use a single log writer instance for all std library logging.

Setup the std log writer in the logging package, and remove boilerplate
from test packages.
2020-10-19 14:29:54 -04:00
James Bardin 6ca477f042 move helper/logging to internal
remove a dead code file too
2020-10-19 14:27:53 -04:00
Kristin Laemmert 04be220f5f deprecate helper/copy
helper/copy CopyDir was used heavily in tests. It differes from
internal/copydir in a few ways, the main one being that it creates the
dst directory while the internal version expected the dst to exist
(there are other differences, which is why I did not just switch tests
to using internal's CopyDir).

I moved the CopyDir func from helper/copy into command_test.go; I could
also have moved it into internal/copy and named it something like
CreateDirAndCopy so if that seems like a better option please let me
know.

helper/copy/CopyFile was used in a couple of spots so I moved it into
internal, at which point I thought it made more sense to rename the
package copy (instead of copydir).

There's also a `go mod tidy` included.
2020-10-08 08:42:16 -04:00
Kristin Laemmert 3933cbd491
remove LegacyProvider (#26433) 2020-10-05 08:33:49 -04:00
Nick McClendon 529ee04269
Fix taint and untaint commands when in a workspace (#22467)
* Fix taint and untaint commands when in a workspace

Fixes #22157. Removes DefaultStateFilepath as the default for the
-state flag, allowing workspaces to be used properly.

* update test with modern state types

Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-09-28 13:09:37 -04:00
James Bardin 2e5366ac69 remove -mod=vendor from command tests 2020-09-23 17:56:19 -04:00
James Bardin c3182bd589 mock provider needs to return a valid response 2020-09-21 16:17:45 -04:00
Alisdair McDiarmid fc7e467d19 command: Add redirect support to 0.13upgrade
If a provider changes namespace in the registry, we can detect this when
running the 0.13upgrade command. As long as there is a version matching
the user's constraints, we now use the provider's new source address.
Otherwise, warn the user that the provider has moved and a version
upgrade is necessary to move to it.
2020-08-31 14:53:35 -04:00
Alisdair McDiarmid 0ed04d05aa command: Fix command fixture modify-in-place bugs
Some of the tests in the command package were running directly on the
fixture directories, and modifying or locking files within them. This
could cause state to leak between tests.

This commit cleans up all such cases that I could find.
2020-06-18 17:09:20 -04:00
Alisdair McDiarmid ca40107066 command/init: Better diagnostics for provider 404s
Fetching a default namespace provider from the public registry can
result in 404 Not Found error. This might be caused by a previously-
default provider moving to a new namespace, which means that the
configuration needs to be upgraded to use an explicit provider source.

This commit adds a more detailed diagnostic for this situation,
suggesting that the intended provider might be in a new namespace. The
recommended course of action is to run the 0.13upgrade command to
generate the correct required_providers configuration.
2020-05-28 09:24:32 -04:00
James Bardin 756e889697 remove unused DependsOn usage in state 2020-05-14 15:46:08 -04:00
Kristin Laemmert 27a794062e Mildwonkey/command tests (#24535)
* command: refactor testBackendState to write states.State

testBackendState was using the older terraform.State format, which is no
longer sufficient for most tests since the state upgrader does not
encode provider FQNs automatically. Users will run `terraform
0.13upgrade` to update their state to include provider FQNs in
resources, but tests need to use the modern state format instead of
relying on the automatic upgrade.

* plan tests passing
* graph tests passing
* json packages test update
* command test updates
* update show test fixtures
* state show tests passing
2020-04-06 09:24:23 -07:00
Kristin Laemmert 0af09b23ca command: apply and most of import tests passing 2020-04-06 09:24:23 -07:00
Martin Atkins 549aede792 Remove terraform.ResourceProvider, use providercache.Installer instead
Back when we first introduced provider versioning in Terraform 0.10, we
did the provider version resolution in terraform.NewContext because we
weren't sure yet how exactly our versioning model was going to play out
(whether different versions could be selected per provider configuration,
for example) and because we were building around the limitations of our
existing filesystem-based plugin discovery model.

However, the new installer codepath is new able to do all of the
selections up front during installation, so we don't need such a heavy
inversion of control abstraction to get this done: the command package can
select the exact provider versions and pass their factories directly
to terraform.NewContext as a simple static map.

The result of this commit is that CLI commands other than "init" are now
able to consume the local cache directory and selections produced by the
installation process in "terraform init", passing all of the selected
providers down to the terraform.NewContext function for use in
implementing the main operations.

This commit is just enough to get the providers passing into the
terraform.Context. There's still plenty more to do here, including to
repair all of the tests this change has additionally broken.
2020-04-06 09:24:23 -07:00
James Bardin b3fc0dab94 use addrs.ConfigResource for dependency tracking
We can't get module instances during transformation, so we need to
reduce the Dependencies to using `addrs.ConfigResource` for now.
2020-03-25 17:03:06 -04:00
James Bardin e13eecbc5b finish provider ModuleInstance replacement 2020-03-11 14:19:52 -04:00
Kristin Laemmert 47a16b0937
addrs: embed Provider in AbsProviderConfig instead of Type
a large refactor to addrs.AbsProviderConfig, embedding the addrs.Provider instead of a Type string. I've added and updated tests, added some Legacy functions to support older state formats and shims, and added a normalization step when reading v4 (current) state files (not the added tests under states/statefile/roundtrip which work with both current and legacy-style AbsProviderConfig strings).

The remaining 'fixme' and 'todo' comments are mostly going to be addressed in a subsequent PR and involve looking up a given local provider config's FQN. This is fine for now as we are only working with default assumption.
2020-02-13 15:32:58 -05:00
Martin Atkins 8b511524d6
Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978)
* Introduce "Local" terminology for non-absolute provider config addresses

In a future change AbsProviderConfig and LocalProviderConfig are going to
become two entirely distinct types, rather than Abs embedding Local as
written here. This naming change is in preparation for that subsequent
work, which will also include introducing a new "ProviderConfig" type
that is an interface that AbsProviderConfig and LocalProviderConfig both
implement.

This is intended to be largely just a naming change to get started, so
we can deal with all of the messy renaming. However, this did also require
a slight change in modeling where the Resource.DefaultProviderConfig
method has become Resource.DefaultProvider returning a Provider address
directly, because this method doesn't have enough information to construct
a true and accurate LocalProviderConfig -- it would need to refer to the
configuration to know what this module is calling the provider it has
selected.

In order to leave a trail to follow for subsequent work, all of the
changes here are intended to ensure that remaining work will become
obvious via compile-time errors when all of the following changes happen:
- The concept of "legacy" provider addresses is removed from the addrs
  package, including removing addrs.NewLegacyProvider and
  addrs.Provider.LegacyString.
- addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded
  in it and has an addrs.Provider and a string alias directly instead.
- The provider-schema-handling parts of Terraform core are updated to
  work with addrs.Provider to identify providers, rather than legacy
  strings.

In particular, there are still several codepaths here making legacy
provider address assumptions (in order to limit the scope of this change)
but I've made sure each one is doing something that relies on at least
one of the above changes not having been made yet.

* addrs: ProviderConfig interface

In a (very) few special situations in the main "terraform" package we need
to make runtime decisions about whether a provider config is absolute
or local.

We currently do that by exploiting the fact that AbsProviderConfig has
LocalProviderConfig nested inside of it and so in the local case we can
just ignore the wrapping AbsProviderConfig and use the embedded value.

In a future change we'll be moving away from that embedding and making
these two types distinct in order to represent that mapping between them
requires consulting a lookup table in the configuration, and so here we
introduce a new interface type ProviderConfig that can represent either
AbsProviderConfig or LocalProviderConfig decided dynamically at runtime.

This also includes the Config.ResolveAbsProviderAddr method that will
eventually be responsible for that local-to-absolute translation, so
that callers with access to the configuration can normalize to an
addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's
currently unused because existing callers are still relying on the
simplistic structural transform, but we'll switch them over in a later
commit.

* rename LocalType to LocalName

Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-01-31 08:23:07 -05:00
Kristin Laemmert 6541775ce4
addrs: roll back change to Type field in ProviderConfig (#23937) 2020-01-28 08:13:30 -05:00
James Bardin 98c02ac114 remove stale dependencies on `state mv`
Clear any Dependencies if there is an entry matching a `state mv` from
address. While stale dependencies won't directly effect any current
operations, clearing the list will allow them to be recreated in their
entirety during refresh. This will help future releases that may rely
solely on the pre-calculated dependencies for destruction ordering.
2020-01-06 15:06:41 -05:00
James Bardin 42a2bb4da3 expand test tempdir symlinks to fix tests on macos 2019-12-06 10:56:47 -05:00
Kristin Laemmert e3416124cc
addrs: replace "Type string" with "Type Provider" in ProviderConfig
* huge change to weave new addrs.Provider into addrs.ProviderConfig
* terraform: do not include an empty string in the returned Providers /
Provisioners
- Fixed a minor bug where results included an extra empty string
2019-12-06 08:00:18 -05:00
Kristin Laemmert 9891d0354a
providers: use addrs.Provider as map keys for provider.Factory (#23548)
* terraform/context: use new addrs.Provider as map key in provider factories
* added NewLegacyProviderType and LegacyString funcs to make it explicit that these are temporary placeholders

This PR introduces a new concept, provider fully-qualified name (FQN), encapsulated by the `addrs.Provider` struct.
2019-12-04 11:30:20 -05:00
James Bardin 46dbb3dde5 use Dependencies to connect creator and destroyer
The DestroyEdgeTransformer cannot determine ordering from the graph when
the destroyers are from orphaned resources, because there are no
references to resolve. The new stored Dependencies provides what we need
to connect the instances in this case.

We also add the StateDependencies method directly in the
GraphNodeResourceInstance interface, since all instances already
implement this, and we don't need another optional interface to check.

The old code in DestroyEdgeTransformer may no longer be needed in the
long run, but that can be determined separately, since too many of the
tests start with an incomplete state and rely on the Dependencies being
determined from the configuration alone.
2019-11-07 17:49:03 -05:00
Radek Simko 5b9f2fafc8 Standardise directory name for test data 2019-06-30 10:16:15 +02:00
James Bardin ac2219ba6e don't lose Private state data during copy
Fix the scope of the private data copy in DeepCopy.

Make sure Dependencies matches nil vs empty so that Equal compares
correctly between copied states
2019-06-05 19:22:46 -04:00
Martin Atkins 86c02d5c35 command: "terraform init" can partially initialize for 0.12upgrade
There are a few constructs from 0.11 and prior that cause 0.12 parsing to
fail altogether, which previously created a chicken/egg problem because
we need to install the providers in order to run "terraform 0.12upgrade"
and thus fix the problem.

This changes "terraform init" to use the new "early configuration" loader
for module and provider installation. This is built on the more permissive
parser in the terraform-config-inspect package, and so it allows us to
read out the top-level blocks from the configuration while accepting
legacy HCL syntax.

In the long run this will let us do version compatibility detection before
attempting a "real" config load, giving us better error messages for any
future syntax additions, but in the short term the key thing is that it
allows us to install the dependencies even if the configuration isn't
fully valid.

Because backend init still requires full configuration, this introduces a
new mode of terraform init where it detects heuristically if it seems like
we need to do a configuration upgrade and does a partial init if so,
before finally directing the user to run "terraform 0.12upgrade" before
running any other commands.

The heuristic here is based on two assumptions:
- If the "early" loader finds no errors but the normal loader does, the
  configuration is likely to be valid for Terraform 0.11 but not 0.12.
- If there's already a version constraint in the configuration that
  excludes Terraform versions prior to v0.12 then the configuration is
  probably _already_ upgraded and so it's just a normal syntax error,
  even if the early loader didn't detect it.

Once the upgrade process is removed in 0.13.0 (users will be required to
go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can
be simplified to remove that special mode, but the idea of doing the
dependency version checks against the liberal parser will remain valuable
to increase our chances of reporting version-based incompatibilities
rather than syntax errors as we add new features in future.
2019-01-14 11:33:21 -08:00
Martin Atkins 0c0a437bcb Move module install functionality over to internal/initwd 2019-01-14 11:33:21 -08:00
James Bardin 149ccd929e missing commits from 19688 2018-12-18 16:54:09 -08:00
Sander van Harmelen 79a9a15879 command/state: lock when pushing state
Next to adding the locking for the `state push` command, this commit also fixes a small bug where the lock would not be propertly released when running the `state show` command.

And finally it renames some variables in the `[un]taint` code in order to try to standardize the var names of a few frequently used variables (e.g. statemgr.Full, states.State, states.SyncState).
2018-11-20 11:15:16 +01:00
Martin Atkins 884aa387b8 command: Use vendoring when building helper programs in tests
In a couple places in tests we execute a child "go build" to make a helper
program. Now that we're running in module mode, "go build" will normally
default to downloading and caching dependencies, which we don't want
because we're still using vendoring for the moment.

Therefore we need to instruct these child builds to use vendoring too,
avoiding the need to download all of the dependencies and ensuring that
we'll be building with the same dependencies that we'd use for a normal
build.
2018-11-19 11:41:52 -08:00
Martin Atkins ad276142b4 command: Fix the command package test build
Some merging conflict shenanigans here led to this usage not lining up
with the imported symbol name, meaning that the tests couldn't compile any
more.
2018-11-08 14:39:24 -08:00
Martin Atkins c3d11b762b command: Fix testBackendState
The hashing function for cached backend configuration is different now, so
our hard-coded hash of the HTTP backend address wasn't working anymore.

Here we update the hash so that tests using this test backend will work
again. Rather than leaving it hard-coded, we'll instead compute it the
same way as "terraform init" would.

In practice only one test is actually using this function right now, so
we also update the test fixture for that test (TestPlan_outBackend) to
match the new expectations, though as of this commit it's still failing
with an unrelated error.
2018-11-08 08:57:11 -08:00
Sander van Harmelen 52a1b22f7a Implement the remote enhanced backend
This is a refactored version of the `remote` backend that was initially added to Terraform v0.11.8 which should now be compatible with v0.12.0.
2018-11-06 16:29:46 +01:00
Martin Atkins 541952bb8f Revert some work that happened since v0.12-dev branched
This work was done against APIs that were already changed in the branch
before work began, and so it doesn't apply to the v0.12 development work.

To allow v0.12 to merge down to master, we'll revert this work out for now
and then re-introduce equivalent functionality in later commits that works
against the new APIs.
2018-10-16 19:48:28 -07:00
Martin Atkins 7b77e20bdc command: Fix several "terraform init" tests 2018-10-16 19:14:11 -07:00
Martin Atkins 89d0944e5b command: Fix even more of the "terraform refresh" tests 2018-10-16 19:14:11 -07:00
Martin Atkins 98bbd560b5 command: Fix most (but not all) "terraform plan" tests 2018-10-16 19:14:11 -07:00
Martin Atkins 741d334ee4 command: Even more fixes for "apply" command tests 2018-10-16 19:14:11 -07:00
Martin Atkins fc2614c939 command: Fix some tests for the "apply" command 2018-10-16 19:14:11 -07:00
Martin Atkins d9051be66d command: Use new provider and provisioner types in test helpers
This moves our build errors from the test helpers out into the tests
themselves. We'll update the tests in subsequent commits.
2018-10-16 19:14:11 -07:00
Martin Atkins a3403f2766 terraform: Ugly huge change to weave in new State and Plan types
Due to how often the state and plan types are referenced throughout
Terraform, there isn't a great way to switch them out gradually. As a
consequence, this huge commit gets us from the old world to a _compilable_
new world, but still has a large number of known test failures due to
key functionality being stubbed out.

The stubs here are for anything that interacts with providers, since we
now need to do the follow-up work to similarly replace the old
terraform.ResourceProvider interface with its replacement in the new
"providers" package. That work, along with work to fix the remaining
failing tests, will follow in subsequent commits.

The aim here was to replace all references to terraform.State and its
downstream types with states.State, terraform.Plan with plans.Plan,
state.State with statemgr.State, and switch to the new implementations of
the state and plan file formats. However, due to the number of times those
types are used, this also ended up affecting numerous other parts of core
such as terraform.Hook, the backend.Backend interface, and most of the CLI
commands.

Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize
in advance to the person who inevitably just found this huge commit while
spelunking through the commit history.
2018-10-16 19:11:09 -07:00
Martin Atkins a270a18a4d command: Update tests for changes in "terraform" package
This is not exhaustive, but it gets the tests in this package compiling
again and gets most of them working.
2018-10-16 18:49:20 -07:00