Commit Graph

130 Commits

Author SHA1 Message Date
Martin Atkins d9051be66d command: Use new provider and provisioner types in test helpers
This moves our build errors from the test helpers out into the tests
themselves. We'll update the tests in subsequent commits.
2018-10-16 19:14:11 -07:00
Martin Atkins 44bc7519a6 terraform: More wiring in of new provider types
This doesn't actually work yet, but it builds and then panics in a pretty
satisfying way.
2018-10-16 19:12:54 -07:00
Martin Atkins a3403f2766 terraform: Ugly huge change to weave in new State and Plan types
Due to how often the state and plan types are referenced throughout
Terraform, there isn't a great way to switch them out gradually. As a
consequence, this huge commit gets us from the old world to a _compilable_
new world, but still has a large number of known test failures due to
key functionality being stubbed out.

The stubs here are for anything that interacts with providers, since we
now need to do the follow-up work to similarly replace the old
terraform.ResourceProvider interface with its replacement in the new
"providers" package. That work, along with work to fix the remaining
failing tests, will follow in subsequent commits.

The aim here was to replace all references to terraform.State and its
downstream types with states.State, terraform.Plan with plans.Plan,
state.State with statemgr.State, and switch to the new implementations of
the state and plan file formats. However, due to the number of times those
types are used, this also ended up affecting numerous other parts of core
such as terraform.Hook, the backend.Backend interface, and most of the CLI
commands.

Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize
in advance to the person who inevitably just found this huge commit while
spelunking through the commit history.
2018-10-16 19:11:09 -07:00
Martin Atkins 0742e756e5 tfdiags: Sort order for diagnostics
Because we gather together diagnostics from many different parts of the
codebase, the list often ends up being in a non-ideal order. Here we
define a partial ordering for diagnostics that should hopefully make them
easier to scan when many are present, by grouping together diagnostics
that are of the same severity and belong to the same file.

We use sort.Stable here because we have a partial order and so we need
to make sure that diagnostics that do not have a relative ordering will
remain in their original order.

This sorting is applied just in time before rendering the diagnostics
in command.Meta.showDiagnostics.
2018-10-16 18:50:29 -07:00
Martin Atkins c937c06a03 terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.

The three main goals here are:
- Use the configuration models from the "configs" package instead of the
  older models in the "config" package, which is now deprecated and
  preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
  new "lang" package, instead of the Interpolator type and related
  functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
  rather than hand-constructed strings. This is not critical to support
  the above, but was a big help during the implementation of these other
  points since it made it much more explicit what kind of address is
  expected in each context.

Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.

I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-10-16 18:46:46 -07:00
Martin Atkins ebafa51723 command: Various updates for the new backend package API
This is a rather-messy, complex change to get the "command" package
building again against the new backend API that was updated for
the new configuration loader.

A lot of this is mechanical rewriting to the new API, but
meta_config.go and meta_backend.go in particular saw some major
changes to interface with the new loader APIs and to deal with
the change in order of steps in the backend API.
2018-10-16 18:44:26 -07:00
Martin Atkins fd5b7b42b8 command: new Meta methods for accessing the new config loader
We need to share a single config loader across all callers because that
allows us to maintain the source code cache we'll use for snippets in
error messages.

Nothing calls this yet. Callers will be gradually updated away from Module
and Config in subsequent commits.
2018-10-16 18:20:32 -07:00
Martin Atkins bd10b84a8e command/format: include source snippets in diagnostics
If we get a diagnostic message that references a source range, and if the
source code for the referenced file is available, we'll show a snippet of
the source code with the source range highlighted.

At the moment we have no cache of source code, so in practice this
codepath can never be visited. Callers to format.Diagnostic will be
gradually updated in subsequent commits.
2018-10-16 18:20:32 -07:00
Sander van Harmelen 179b32d426 Add a `CredentialsForHost` method to disco.Disco
By adding this method you now only have to pass a `*disco.Disco` object around in order to do discovery and use any configured credentials for the discovered hosts.

Of course you can also still pass around both a `*disco.Disco` and a `auth.CredentialsSource` object if there is a need or a reason for that!
2018-08-03 11:29:11 +02:00
James Bardin 67a6152091 move backend operation cancellation into meta
Create a single command method for running and operation with
cancellation.
2018-02-12 11:56:54 -05:00
James Bardin 7d5f7cb22f
Merge pull request #16961 from hashicorp/jbardin/mock-provider-race
minor race issue in mockResourceProvider
2018-01-08 16:47:58 -05:00
James Bardin e63a3474d5 kill the flag error writer after 2 seconds
There's no point in trying to track these, they're lost after each test.
Kill them after a short delay so we don't have goroutines from every single
command test to wade through if we have a stack dump.
2017-12-20 15:52:43 -05:00
James Bardin 885e4cde81 don't loop indefinitely in confirm method
Only check for input twice in the meta.confirm method. This prevents an
errant newline from aborting the run while allowing Terraform to exit if
there is no input available. We don't just check for a tty, since we
still rely on being able to pipe input in for testing.

Remove the redundant confirmation loops in the migration code, and only
use the confirm method.
2017-12-18 18:39:21 -05:00
James Bardin 2941ed464c replace the testShutdownHook with a check for Stop
Now that the local backend can be cancelled during plan and refresh, we
don't really need the testShutdownHook. Simplify the tests by just
checking for Stop being called on the provider.
2017-12-05 10:17:20 -05:00
James Bardin e2501d7830 make apply shutdown test completely deterministic
Add a shutdown hook to verify that a context has been correctly
cancelled, so we can remove the sleep and stop guessing.

Add a plan version of the shutdown test as well.
2017-12-01 15:56:49 -05:00
James Bardin 3aaa1e9d04 make plans cancellable
There was no cancellation context for a plan, so it would always have to
run to completion as SIGINT was being swallowed.

Move the shutdown channel to the command Meta since it's used in
multiple commands.
2017-12-01 13:14:44 -05:00
Martin Atkins 3da5fefdc1 command: Allow TF_DATA_DIR env var to override data directory
This allows the user to customize the location where Terraform stores
the files normally placed in the ".terraform" subdirectory, if e.g. the
current working directory is not writable.
2017-11-01 16:55:23 -07:00
James Bardin 5203c66116 pass command credentials into module.Storage 2017-10-27 16:16:35 -04:00
James Bardin 3a495ffe56 rename ModuleStorage to Storage
get rid of stutter and use module.Storage
2017-10-27 13:11:21 -04:00
James Bardin f2a7b94692 use the new ModuleStorage in the command package
Update the command package to use the new module storage. Move the old
command output strings into the module storage itself. This could be
moved back later either by using ui callbacks, or designing a module
storage interface once we know what the final requirements will look
like.
2017-10-27 12:58:24 -04:00
Martin Atkins 865e61b4ea main+command: provide service disco and creds to commands
The command package is the main place we need access to these, so that
we can use them during init (to install packages, for example) and so that
we can use them to configure remote backends.

For the moment we're just providing an empty credentials object, which
will start to include both statically-configured and
helper-program-provided credentials sources in subsequent commits.
2017-10-19 11:18:43 -07:00
Martin Atkins ea81e75a4e command: utility for rendering tfdiag diagnostics
This new method showDiagnostics takes any value that would be accepted by
tfdiags.Append and renders it to the UI.

This is intended to encourage consistent handling of the different kinds
of errors and diagnostics that can be produced, and allow richer error
objects like the HCL2 diagnostics to be easily unwrapped and shown in
their full-fidelity.
2017-10-06 11:46:07 -07:00
Martin Atkins 5c0670fdf4 command: allow callers to activate a plugin cache
Here we add a new caller-settable field to command.Meta that activates
a read-through cache directory for plugin installation.
2017-09-29 14:03:09 -07:00
Jack Bruno 3f2136d7ee cli: terraform import -ignore-missing-config
This new option allows importing without configuration present.

Configuration is required by default as a confirmation that the provided resource name is correct, but it can be useful to override this in tools that wrap Terraform to do more involved operations.
2017-09-18 11:41:30 -07:00
Martin Atkins 0fe43c8977 cli: allow disabling "next steps" message in terraform plan
In #15884 we adjusted the plan output to give an explicit command to run
to apply a plan, whereas before this command was just alluded to in the
prose.

Since releasing that, we've got good feedback that it's confusing to
include such instructions when Terraform is running in a workflow
automation tool, because such tools usually abstract away exactly what
commands are run and require users to take different actions to
proceed through the workflow.

To accommodate such environments while retaining helpful messages for
normal CLI usage, here we introduce a new environment variable
TF_IN_AUTOMATION which, when set to a non-empty value, is a hint to
Terraform that it isn't being run in an interactive command shell and
it should thus tone down the "next steps" messaging.

The documentation for this setting is included as part of the "...in
automation" guide since it's not generally useful in other cases. We also
intentionally disclaim comprehensive support for this since we want to
avoid creating an extreme number of "if running in automation..."
codepaths that would increase the testing matrix and hurt maintainability.

The focus is specifically on the output of the three commands we give in
the automation guide, which at present means the following two situations:

* "terraform init" does not include the final paragraphs that suggest
  running "terraform plan" and tell you in what situations you might need
  to re-run "terraform init".
* "terraform plan" does not include the final paragraphs that either
  warn about not specifying "-out=..." or instruct to run
  "terraform apply" with the generated plan file.
2017-09-14 10:51:41 -07:00
James Bardin f7f1e8e406 Sort arguments in Meta.process
Meta.process was relying on the system readdir to order the arguments,
but readdir doesn't guarantee any ordering. Read the directory contents
as a whole and sort them in place before adding the tfvars files.
2017-07-06 11:34:47 -04:00
Robert Liebowitz e67ecb5ce2 Restructure Meta.process to logically group code 2017-07-05 17:24:17 -07:00
Robert Liebowitz 8d98fdecac Autoload only .auto.tfvars files 2017-07-05 17:24:17 -07:00
Robert Liebowitz 006744bfe0 Use all tfvars files in working directory
As a side effect, several commands that previously did not have a failure
state can now fail during meta-parameter processing.
2017-07-05 17:24:17 -07:00
Jake Champlin 9944ea6886
core: Skip provider checksum validation based on env var
Skips checksum validation if the `TF_SKIP_PROVIDER_VERIFY` environment variable is set. Undocumented variable, as the primary goal is to significantly improve the local provider development workflow.
2017-07-03 13:59:13 -04:00
Martin Atkins 3df164502a command: don't prompt for state migration if TF_INPUT is set
The "confirm" method was directly checking the meta struct's input field,
but that only represents the -input command line flag, and doesn't
respect the TF_INPUT environment variable.

By calling the Input method instead, we check both.

This fixes #15338.
2017-06-26 11:22:29 -07:00
James Bardin 000e860706 Add plugin dir scaffolding
add pluginDir to command.Meta, the flag to initialize it, and the
methods to save and restore it.
2017-06-15 14:26:12 -04:00
James Bardin 55bf19e548 always write to dataDir in the current directory
Now that init can take a directory for configuration, the old behavior
of writing the .terraform data directory into the target path no longer
makes sense. Don't change the dataDir field during init, and write to
the default location.

Clean up all references to Meta.dataDir, and only use the getter method
in case we chose to dynamically override this at some point.
2017-06-14 15:14:26 -04:00
James Bardin cbbbcea4b9 stop err scanner goroutines between tests 2017-06-12 13:52:30 -04:00
Martin Atkins 418a8a8bc9 command + backend: rename various API objects to "Workspace" terminology
We're shifting terminology from "environment" to "workspace". This takes
care of some of the main internal API surface that was using the old
terminology, though is not intended to be entirely comprehensive and is
mainly just to minimize the amount of confusion for maintainers as we
continue moving towards eliminating the old terminology.
2017-06-09 16:26:25 -07:00
Martin Atkins 31d556894f command: shallow UI-focused rename of "environment" to "workspace"
Feedback after 0.9 was that the term "environment" was confusing due to
it colliding with several other concepts, such as OS environment
variables, a non-aligned Terraform Enterprise concept, and differing ideas
of "environment" within various organizations.

This new term "workspace" is intended to ease some of that confusion. This
term is not used anywhere else in Terraform today, and we expect it to not
be used in a manner that would be confusing within user organizations.

This begins a deprecation cycle for the "terraform env" family of commands,
instead moving to an equivalent set of "terraform workspace" commands.

There are some remaining references to the old "environment" concept in
the code, which will be cleaned up in a separate change. This change is
instead focused on text visible in the UI and wording within code comments
for the benefit of human maintainers of the code.
2017-06-09 15:01:39 -07:00
David Glasser c25d848ffb core: allow overriding environment name via env var
This allows you to run multiple concurrent terraform operations against
different environments from the same source directory.

Fixes #14447.

Also removes some dead code which appears to do the same thing as the function I
modified.
2017-06-09 15:01:39 -07:00
Martin Atkins 4571a16b15 command: remove Meta.forceProviderSHA256s
This was added with the idea of using it to override the SHA256 hashes
to match those hypothetically stored in a plan, but we already have a
mechanism elsewhere for populating context fields from plan fields, so
this is not actually necessary.
2017-06-09 14:03:59 -07:00
Martin Atkins 6ba6508ec9 command: pass the locked plugin hashes into ContextOpts
By reading our lock file and passing this into the context, we ensure that
only the plugins referenced in the lock file can be used. As of this
commit there is no way to create that lock file, but that will follow soon
as part of "terraform init".

We also provide a way to force a particular set of SHA256s. The main use
for this is to allow us to persist a set of plugins in the plan and
check the same plugins are used during apply, but it may also be useful
for automated tests.
2017-06-09 14:03:59 -07:00
Martin Atkins 7ca592ac06 core: use ResourceProviderResolver to resolve providers
Previously the set of providers was fixed early on in the command package
processing. In order to be version-aware we need to defer this work until
later, so this interface exists so we can hold on to the possibly-many
versions of plugins we have available and then later, once we've finished
determining the provider dependencies, select the appropriate version of
each provider to produce the final set of providers to use.

This commit establishes the use of this new mechanism, and thus populates
the provider factory map with only the providers that result from the
dependency resolution process.

This disables support for internal provider plugins, though the
mechanisms for building and launching these are still here vestigially,
to be cleaned up in a subsequent commit.

This also adds a new awkward quirk to the "terraform import" workflow
where one can't import a resource from a provider that isn't already
mentioned (implicitly or explicitly) in config. We will do some UX work
in subsequent commits to make this behavior better.

This breaks many tests due to the change in interface, but to keep this
particular diff reasonably easy to read the test fixes are split into
a separate commit.
2017-06-09 14:03:59 -07:00
Martin Atkins 8364383c35 Push plugin discovery down into command package
Previously we did plugin discovery in the main package, but as we move
towards versioned plugins we need more information available in order to
resolve plugins, so we move this responsibility into the command package
itself.

For the moment this is just preserving the existing behavior as long as
there are only internal and unversioned plugins present. This is the
final state for provisioners in 0.10, since we don't want to support
versioned provisioners yet. For providers this is just a checkpoint along
the way, since further work is required to apply version constraints from
configuration and support additional plugin search directories.

The automatic plugin discovery behavior is not desirable for tests because
we want to mock the plugins there, so we add a new backdoor for the tests
to use to skip the plugin discovery and just provide their own mock
implementations. Most of this diff is thus noisy rework of the tests to
use this new mechanism.
2017-06-09 14:03:59 -07:00
James Bardin 7aa2ce8341 add -reconfigure option for init
The reconfigure flag will force init to ignore any saved backend state.
This is useful when a user does not want any backend migration to
happen, or if the saved configuration can't be loaded at all for some
reason.
2017-04-20 18:15:46 -04:00
James Bardin aad143b6d1 set stateLock to true when building meta flagSet
Any commands that use `stateLock` should have a flag to set that
value, but set a failsafe to true just in case.
2017-04-04 14:44:58 -04:00
James Bardin 305ef43aa6 provide contexts to clistate.Lock calls
Add fields required to create an appropriate context for all calls to
clistate.Lock.

Add missing checks for Meta.stateLock, where we would attempt to lock,
even if locking should be skipped.
2017-04-01 17:09:20 -04:00
James Bardin 50023e9a60 honor `input=false` in state migration
return an error when confirming a copy if -input=false
2017-03-29 18:11:45 -04:00
James Bardin 54e536cfe0 add `-force-copy` option to init command
The `-force-copy` option will suppress confirmation for copying state
data.

Modify some tests to use the option, making sure to leave coverage of
the Input code path.
2017-03-22 08:47:26 -04:00
Mitchell Hashimoto d475fc29a8
command: test that terraform meta information is passed through 2017-03-13 16:31:35 -07:00
James Bardin 4dac986a91 Local.StatePaths doesn't need to reutrn an error
add a test to ensure we have consistent output
2017-02-28 19:18:16 -05:00
James Bardin b53704ed87 Thread the environment through all commands
Add Env and SetEnv methods to command.Meta to retrieve the current
environment name inside any command.

Make sure all calls to Backend.State contain an environment name, and
make the package compile against the update backend package.
2017-02-28 16:35:46 -05:00
James Bardin 07903189f1 s/Meta.lockState/Meta.stateLock/g 2017-02-06 09:58:04 -05:00