Commit Graph

1708 Commits

Author SHA1 Message Date
Alisdair McDiarmid fc7e467d19 command: Add redirect support to 0.13upgrade
If a provider changes namespace in the registry, we can detect this when
running the 0.13upgrade command. As long as there is a version matching
the user's constraints, we now use the provider's new source address.
Otherwise, warn the user that the provider has moved and a version
upgrade is necessary to move to it.
2020-08-31 14:53:35 -04:00
Pam Selle f2d213c461
Merge pull request #25657 from pdecat/typo_hierarchical
Typo: heirarchical => hierarchical
2020-08-28 12:37:49 -04:00
Alisdair McDiarmid 2323e9370a command: Warn instead of error for empty output
When the output subcommand is called with no arguments, and there are no
outputs to show, we previously rendered an error message but returned a
non-error status code. This is confusing.

This commit changes the text UI to use a warning diagnostic, which makes
it clearer that this is a non-error situation. We do not change the exit
code or the text of the warning, so hopefully this is not considered a
breaking change.
2020-08-28 11:05:46 -04:00
Alisdair McDiarmid 45437721c9 command: Add test for backend config validation 2020-08-26 11:37:11 -04:00
Alisdair McDiarmid f028b0a2bf command: Fix backend config schema validation
When applying a backend config override file, we must not check for the
presence of all required fields, as the override can be a partial
configuration. It is only valid to check for required fields after all
overrides have been merged, which init already does.
2020-08-26 10:50:47 -04:00
Alisdair McDiarmid c5d9935c0e
Merge pull request #25960 from hashicorp/alisdair/backend-config-override-fix
command: Fix backend config override validation
2020-08-24 10:14:16 -04:00
Alisdair McDiarmid 35125717b2
Merge pull request #25847 from zachwhaley/fix-tf-cli-args-no-color
Fix error when multiple -no-color arguments are used
2020-08-24 10:13:59 -04:00
Alisdair McDiarmid 19efd7bc3b
Merge pull request #25262 from hashicorp/alisdair/always-validate-workspace-name
command: Always validate workspace name
2020-08-24 10:11:13 -04:00
Alisdair McDiarmid 677aabc767 command: Fix backend config override validation
When loading a backend config override file, init was doing two things
wrong:

- First, if the file failed to parse, we accidentally didn't return,
  which caused a panic due to the parsed body being nil;
- Secondly, we were overzealous with the validation of the file,
  allowing only attributes. While most backend configs are attributes
  only, the enhanced remote backend body also contains a `workspaces`
  block, which we need to support here.

This commit fixes the first bug with an early return and adds test cases
for missing file and intentionally-blank filename (to clear the config).

We also add a schema validation for the backend block, based on the
backend schema itself. This requires constructing an HCL body schema so
that we can call `Content` and check for diagnostic errors.

The result is more useful errors when an invalid backend config override
file is used, while also supporting the enhanced remote backend config
fully.

Does not include tests specific to the remote backend, because the
mocking involved to allow the backend to fully initialize is too
involved to be worth it.
2020-08-21 16:21:13 -04:00
Cameron Stitt 54e32652f7
Ensure depends_on is in module calls for config 2020-08-20 07:49:03 +10:00
Alisdair McDiarmid 30c7dfca62
Merge pull request #25898 from hashicorp/alisdair/fix-required-version-diags
terraform: Fix required version constraint diags
2020-08-19 11:26:03 -04:00
Alisdair McDiarmid 3114e2ad7c
Merge pull request #25890 from hashicorp/import-our-nemesis
terraform: Eval module call arguments for import
2020-08-19 11:25:38 -04:00
kmoe 6486b857d6
Merge pull request #25811 from hashicorp/terraform-version-order
terraform version: fix output order
2020-08-19 13:54:18 +01:00
Alisdair McDiarmid c98f352dc8 terraform: Fix required version constraint diags
If a module has multiple terraform.required_version constraints, any
failures would point at the last constraint in the error diagnostics. If
an earlier constraint was the actual problem, this leads to confusing
errors like this:

    Error: Unsupported Terraform Core version

      on main.tf line 6, in terraform:
       6:   required_version = ">= 0.13.0"

    This configuration does not support Terraform version 0.13.0.

The error was due to storing the declaration range of the constraint as
a pointer to the contents of a loop variable, which was later
overwritten in later iterations of the loop.  Instead we now use HCL's
handy Ptr() method to create a direct pointer to the range struct.
2020-08-18 09:35:32 -04:00
Alisdair McDiarmid d8e9964363 terraform: Eval module call arguments for import
Include the import walk in the list of operations for which we create an
EvalModuleCallArgument node. This causes module call arguments to be
evaluated even if the module variables have defaults, ensuring that
invalid default values (such as the common "{}" for variables thought of
as maps) do not cause failures specific to import.

This fixes a bug where a child module evaluates an input variable in its
locals block, assuming that it is a nested object structure. The bug
report includes a default value of "{}", which is overridden by a root
variable value. Without the eval node added in this commit, the default
value is used and the local evaluation errors.
2020-08-17 17:14:12 -04:00
Zachary Whaley af8d5a69d8 Fix error when TF_CLI_ARGS adds -no-color argument
Fixes #25845
2020-08-13 13:41:34 -05:00
Katy Moe e59553ef02
test outdated version output 2020-08-12 21:22:39 +01:00
James Bardin 1c09df1a66
Merge pull request #25779 from hashicorp/jbardin/remove-state-attrs
Remove resource state attributes that are no longer in the schema
2020-08-12 10:49:44 -04:00
Katy Moe 25337b7217
fix outdated message output order 2020-08-11 23:27:29 +01:00
Alisdair McDiarmid 96d2265ddb command: Add test for invalid selected workspace
If somehow an invalid workspace has been selected, the Meta.Workspace
method should not return an error, to ensure that we don't break any
existing workflows with invalid workspace names.
2020-08-11 12:33:12 -04:00
Alisdair McDiarmid 8252920e9a command: Allow workspace delete with invalid name
We are validating the workspace name for all workspace commands. Due to
a bug with the TF_WORKSPACE environment variable, it has been possible
to accidentally create a workspace with an invalid name.

This commit removes the valid workspace name check for workspace delete
to allow users to clean up any invalid workspaces.
2020-08-11 12:33:12 -04:00
Alisdair McDiarmid b239570abb command: Always validate workspace name
The workspace name can be overridden by setting a TF_WORKSPACE
environment variable. If this is done, we should still validate the
resulting workspace name; otherwise, we could end up with an invalid and
unselectable workspace.

This change updates the Meta.Workspace function to return an error, and
handles that error wherever necessary.
2020-08-11 12:33:12 -04:00
Alisdair McDiarmid ef65f552c8
Merge pull request #25647 from hashicorp/alisdair/remove-stubbed-out-apply-diff-code
command: Remove stubbed-out apply diff output code
2020-08-11 12:18:49 -04:00
Alisdair McDiarmid 17e1c9dd05 command: Fix state mv for only resource in module
When moving a resource block with multiple instances to a new address
within the same module, we need to ensure that the target module is
present as late as possible. Otherwise, deleting the resource from the
original address triggers pruning, and the module is removed just before
we try to add the resource to it, which causes a crash.

Includes regression test which panics without this code change.
2020-08-11 11:54:35 -04:00
Kristin Laemmert 6621501ae3
state: remove deprecated state package (#25490)
Most of the state package has been deprecated by the states package.
This PR replaces all the references to the old state package that
can be done simply - the low-hanging fruit.

* states: move state.Locker to statemgr

The state.Locker interface was a wrapper around a statemgr.Full, so
moving this was relatively straightforward.

* command: remove unnecessary use of state package for writing local terraform state files

* move state.LocalState into terraform package

state.LocalState is responsible for managing terraform.States, so it
made sense (to me) to move it into the terraform package.

* slight change of heart: move state.LocalState into clistate instead of
terraform
2020-08-11 11:43:01 -04:00
Kristin Laemmert 86e9ba3d65
* backend/local: push responsibility for unlocking state into individual operations
* unlock the state if Context() has an error, exactly as backend/remote does today
* terraform console and terraform import will exit before unlocking state in case of error in Context()
* responsibility for unlocking state in the local backend is pushed down the stack, out of backend.go and into each individual state operation
* add tests confirming that state is not locked after apply and plan

* backend/local: add checks that the state is unlocked after operations

This adds tests to plan, apply and refresh which validate that the state
is unlocked after all operations, regardless of exit status. I've also
added specific tests that force Context() to fail during each operation
to verify that locking behavior specifically.
2020-08-11 11:23:42 -04:00
James Bardin 99cd3ab223 fix command tests
A number of tests had invalid configs or providers, but were never
properly validated
2020-08-07 14:13:57 -04:00
FGtatsuro b6cf1c3673 Use valid heredoc begin symbol in diff output. 2020-08-03 00:33:35 +09:00
Patrick Decat 062865735f Typo: heirarchical => hierarchical 2020-07-23 15:09:22 +02:00
Alisdair McDiarmid ee19b0d6c5 command: Remove stubbed-out apply diff output code
This code was made to do nothing pre-0.12, and we have no plans to
reintroduce a diff in the apply output, so it seems reasonable to now
remove it altogether.
2020-07-22 14:19:00 -04:00
Kristin Laemmert f8e3456867
command/show: fix bug displaying provider config in json output of tf plan (#25577)
A lingering FIXME caused missing configuration from provider config
blocks in the json output of terraform plan. This fixes the regression
and adds a test. For the sake of testing, I added an optional attribute
to the show test provider, which resulted in the providers schema test
getting an update - not a bad addition, but we can always add a
test-specific provider schema as needed.
2020-07-14 15:28:31 -04:00
Alisdair McDiarmid 53e587e1a6
Merge pull request #25504 from hashicorp/alisdair/post-install-cache-validate
Add post-install provider cache validation and error reporting
2020-07-09 14:55:17 -04:00
Martin Atkins 7909dd318d command/init: Specialized error message for host that isn't registry
If the user specifies a host that isn't a provider registry in a provider
source address then we'll print out some specialized error messages for
different variants of that situation.

In particular, this includes a special case for when the error is on the
hostname "github.com", in anticipation of folks incorrectly attempting to
use GitHub repository URLs (or Go-style module paths that happen to be
on GitHub) to specify providers, so we can give a more specific hint about
that.

This is just a different presentation of an existing error case that we
are already covering in the installer tests, so there are no new tests
here. We could in principle have a test covering the exact text of these
error messages, but we don't have much precedent for command package tests
covering that level of cosmetic detail.
2020-07-08 10:18:55 -07:00
Martin Atkins 80ab867e57 command/init: Remove special 0.12upgrade heuristic
For Terraform v0.12 we introduced a special loading mode where we would
use the 0.11-syntax-compatible "earlyconfig" package as a heuristic to
identify situations where it was likely that the user was trying to use
0.11-only syntax that the upgrade tool might help with.

However, as the language has moved on that is no longer a suitable
heuristic in Terraform 0.13 and later: other new additions to the
language can cause the main loader to disagree with earlyconfig, which
would lead us to give poor advice about how to respond.

Instead, we'll now return the same generic "there are errors" message in
all syntax error cases. We have an extra message for errors in this
case (as compared to other commands) because "terraform init" is usually
the first command a new user interacts with and so this message gives some
extra explanation about what "terraform init" will do with the
configuration once it's valid.

This also includes a reset control character in the output of the message
as part of our ongoing mission to stop Terraform printing out whole
paragraphs of colored text, which can often be hard to read for various
reasons.
2020-07-08 10:18:55 -07:00
Alisdair McDiarmid 87d1fb4006 command/init: Display provider validation errors
After installing providers, we validate the presence of an executable
file, and generate a selected versions lockfile. If this process fails,
notify the user. One possible cause for this is an invalid provider
package with a missing or misnamed executable file.
2020-07-07 15:20:20 -04:00
Alisdair McDiarmid a18b531b14 getproviders: FakeInstallablePackageMeta filename
Add an optional execFilename argument to the test helper function
FakeInstallablePackageMeta, which allows the creation of invalid
packages.
2020-07-07 15:18:30 -04:00
Alisdair McDiarmid 8e87ccb689 providercache: Lazily detect executable file
Instead of searching the installed provider package directory for a
binary as we install it, we can lazily detect the executable as it is
required. Doing so allows us to separately report an invalid unpacked
package, giving the user more actionable error messages.
2020-07-07 15:18:27 -04:00
Kristin Laemmert 604e65bb62 Revert "backend/local: release lock if there is an error in Context() (#25427)"
This reverts commit 1ba0d615e7.
2020-06-30 14:12:32 -04:00
Kristin Laemmert 1ba0d615e7
backend/local: release lock if there is an error in Context() (#25427)
* command/console: return in case of errors before trying to unlock remote
state

The remote backend `Context` would exit without an active lock if there
was an error, while the local backend `Context` exited *with* a lock. This
caused a problem in `terraform console`, which would call unlock
regardless of error status.

This commit makes the local and remote backend consistently unlock the
state incase of error, and updates terraform console to check for errors
before trying to unlock the state.

* adding tests for remote and local backends
2020-06-29 14:57:42 -04:00
Kristin Laemmert df244b87c2
command/init: return an error with invalid -backend-config files (#25411)
* command/init: return an error with invalid -backend-config files

The -backend-config flag expects a set of key-value pairs or a file
containing key-value pairs. If the file instead contains a full backend
configuration block, it was silently ignored. This commit adds a check
for blocks in the file and returns an error if they are encountered.

Fixes #24845

* emphasize backend configuration file in docs
2020-06-26 12:49:31 -04:00
Alisdair McDiarmid b4cc77b6cd
Merge pull request #25379 from hashicorp/alisdair/yes
command/login: Require "yes" to confirm
2020-06-25 13:00:02 -04:00
Alisdair McDiarmid f6f707c669
Merge pull request #25381 from hashicorp/alisdair/013upgrade-preserves-more-comments
command: Fix 0.13upgrade to preserve more comments
2020-06-25 12:56:34 -04:00
Alisdair McDiarmid 779fe37a1c command/login: Require "yes" to confirm
This is for consistency with other commands which use prompts, all of
which require "yes" rather than "y" to confirm.

We also migrate the login command to use UIInput, which now supports
securely asking for passwords or secrets via the speakeasy library.
2020-06-25 11:46:51 -04:00
Kristin Laemmert 47e657c611
internal/getproviders: decode and return any registry warnings (#25337)
* internal/getproviders: decode and return any registry warnings

The public registry may include a list of warnings in the "versions"
response for any given provider. This PR adds support for warnings from
the registry and an installer event to return those warnings to the
user.
2020-06-25 10:49:48 -04:00
James Bardin 98ff2065bc
Merge pull request #25377 from hashicorp/jbardin/remote-state
Correct remote state return value
2020-06-25 10:43:22 -04:00
Alisdair McDiarmid b6739829e7 command: Fix 0.13upgrade to preserve more comments
Previously, any comments inside the required provider configuration for
a given provider would be wiped out upon rerunning the 0.13upgrade
command. This commit attempts to preserve those comments if the existing
entry is semantically equivalent to the entry we are about to write.
2020-06-24 15:54:46 -04:00
James Bardin 731b19ab46 e2e test for remote state read 2020-06-24 14:09:59 -04:00
João G. Packer fd47260fdc
command: adjust exit code of state rm (#22300)
* command: adjust exit code of state rm and state mv

Commands `state rm` and `state mv` will now exit with code 1 when the
target resource is not found in the current state.
This is consistent with `terraform state show non_existent_resource`.

Fixes #17800
2020-06-24 14:03:23 -04:00
Alisdair McDiarmid f714a9da79
Merge pull request #25352 from hashicorp/alisdair/fix-import-allow-missing-config
command/import: Fix allow-missing-config option
2020-06-24 13:37:13 -04:00
Alisdair McDiarmid 550d75f5ce
Merge pull request #25300 from hashicorp/alisdair/fix-init-flags-silent-exit
command: Fix init flags silent exit bug
2020-06-24 10:33:09 -04:00
Alisdair McDiarmid 9ab9ef6291 command/import: Fix allow-missing-config option
We previously intentionally removed support for the allow-missing-config
option to terraform import, requiring that all imported resources have
matching config. See #24412.

However, the option was not removed from the import command, and it is
widely used. This commit reintroduces support for importing with a
missing configuration by falling back to implying the provider FQN based
on the resource type.
2020-06-23 14:20:50 -04:00
Alisdair McDiarmid 820ed48813 command: Fix init flags silent exit bug
When using `-flag=value` with Powershell, unquoted values are broken
into separate arguments. This means that the following command:

  terraform init -backend-config=./backend.conf

is interpreted by Terraform as:

  terraform init -backend-config= ./backend.conf

This results in an empty backend-config setting (which is semantically
valid!) followed by a custom configuration path (pointing at a file).

Due to a bug where we could exit without printing diagnostics, this
would result in a silent failure that was very difficult to diagnose.
2020-06-18 17:58:45 -04:00
Alisdair McDiarmid 0ed04d05aa command: Fix command fixture modify-in-place bugs
Some of the tests in the command package were running directly on the
fixture directories, and modifying or locking files within them. This
could cause state to leak between tests.

This commit cleans up all such cases that I could find.
2020-06-18 17:09:20 -04:00
Alisdair McDiarmid 163c943e9b command: Fix bug with -v/-version/--version flags
We globally support a -v/-version/--version flag, which triggers the
version subcommand. The recent introduction of JSON output support meant
we started parsing the flags for the first time, but we didn't add flags
for these global version arguments.

This commit adds those flags (but doesn't check them, since they have no
effect on the version command itself). Also adds usage information for
terraform version.
2020-06-17 15:24:15 -04:00
Katy Moe 6b3d0ee64b add test for terraform version 2020-06-16 14:47:57 -04:00
Katy Moe dbe139e616 add test for terraform version -json 2020-06-16 14:47:57 -04:00
Katy Moe d8e22f1804 add terraform version -json 2020-06-16 14:47:57 -04:00
Alisdair McDiarmid d2b215f4d5 command: Fix test cleanup
This test runs `init`, which causes some dangling `.terraform` state
after running tests.
2020-06-16 14:11:08 -04:00
James Bardin b35a56cf6e command tests looked for providers in the graphs
The quotes in provider names needs to be escaped, so these tests were
looking for the wrong output to begin with.
2020-06-15 14:40:59 -04:00
Alisdair McDiarmid 7d4cab126f
Merge pull request #25233 from hashicorp/alisdair/remove-implicit-init-then-apply
command/apply: Remove implicit init call
2020-06-12 15:52:06 -04:00
Kristin Laemmert b611bd7209 reproduction test 2020-06-12 15:39:55 -04:00
Alisdair McDiarmid 0d614a8d01 command/apply: Remove implicit init call
The positional argument passed to apply was once used to specify a
source for a Terraform module to fetch and initialize (#337). This
functionality was removed from the init command later (#15032) but not
completely removed from apply.

This code was non-functional but largely not harmful, except for a very
specific case: when passing an absolute path to a plan file as the
positional argument on Windows, the getter.Detect code would incorrectly
interpret the path as a URL. This caused init to fail and the apply
command would exit with code 1 but without diagnostics.

This commit removes this codepath, which fixes this bug, and should
otherwise have no effect on the supported behaviour of apply.
2020-06-12 12:39:15 -04:00
Alisdair McDiarmid 08b735984a
Merge pull request #25191 from hashicorp/alisdair/better-provider-upgrade-hints-on-init
command/init: Improve diags for legacy providers
2020-06-12 12:31:33 -04:00
Martin Atkins 17feb2abfc vendor: go get github.com/apparentlymart/go-versions@v1.0.0
This new version permits omitting the space between the operator and the
boundary in a ruby-style version constraint, like ">1.0.0" instead of
"> 1.0.0".
2020-06-12 08:45:14 -07:00
Alisdair McDiarmid 9263b28e99 command/init: Improve diags for legacy providers
When initializing a configuration which refers to re-namespaced legacy
providers, we attempt to detect this and display a diagnostic message.
Previously this message would direct the user to run the 0.13upgrade
command, but without specifying in which directories.

This commit detects which modules are using the providers in question,
and for local modules displays a list of upgrade commands which specify
the source directories of these modules.

For remote modules, we display a separate list noting that they need to
be upgraded elsewhere, providing both the local module call name and the
module source address.
2020-06-12 09:57:01 -04:00
Kristin Laemmert 5450e8515d
command/013upgrade: detect builtin terraform provider (#25215)
* command/013upgrade: detect builtin terraform provider
2020-06-11 14:10:47 -04:00
Alisdair McDiarmid 1c1e4a4de0 command/providers: Show provider requirements tree
Providers can be required from multiple sources. The previous
implementation of the providers sub-command displayed only a flat list
of provider requirements, which made it difficult to see which modules
required each provider.

This commit reintroduces the tree display of provider requirements, and
adds a separate output block for providers required by existing state.
2020-06-09 14:21:53 -04:00
Masayuki Morita ee913a1c87
command/0.13upgrade: make confirmation more user-friendly (#25142)
I feel the current confirmation prompt for 0.13upgrade command is
ambiguous what is expected. Actually, when I used it for the first time,
I cancelled it by typing `y` instead of `yes`.

I believe it would be great if the 0.13upgrade command tell us the
expected value for confirmation like 0.12upgrade.
2020-06-08 13:36:19 -04:00
Alisdair McDiarmid c87c0a9d2a command/format: Fix empty overlap diagnostics
Diagnostics where the highlight range has an empty overlap with a line
would skip lines of the output. This is because if two ranges abut each
other, they can be considered to overlap, but that overlap is empty.
This results in an edge case in the diagnostic printer which causes the
line not to be printed.
2020-06-05 16:17:27 -04:00
Shunsuke Miyoshi dc3ce1bbf7
command/refresh: fix state file path (#22885)
* fix default state file path for refresh command
2020-06-04 10:24:53 -04:00
Alisdair McDiarmid fbb966b97b command: Fix 0.13upgrade usage 2020-06-04 08:51:24 -04:00
Masayuki Morita f266956a5d Add `-yes` flag to 0.13upgrade help message
It seems to be implemented but not shown in help message.

FYI: A help message for 0.12upgrade command on the 0.12 branch.
https://github.com/hashicorp/terraform/blob/v0.12.26/command/012_config_upgrade.go#L235-L243
2020-06-04 11:11:46 +09:00
Martin Atkins 49e2e00231 command: terraform providers mirror
This new command is intended to make it easy to create or update a mirror
directory containing suitable providers for the current configuration,
producing a layout that is appropriate both for a filesystem mirror or,
if copied into the document root of an HTTP server, a network mirror.

This initial version is not customizable aside from being able to select
multiple platforms to install packages for.

Future iterations of this could include commands to turn the JSON index
generation on and off, or to instruct it to produce the unpacked directory
layout instead of the packed directory layout as it currently does. Both
of those options would make the generated directory unsuitable to be
a network mirror, but it would still work as a filesystem mirror.

In the long run this will hopefully form part of a replacement workflow to
terraform-bundle as a way to put copies of providers somewhere so we don't
need to re-download them every time, but some other changes will be needed
outside of just this command before that'd be true, such as adding support
for network and/or filesystem mirrors in Terraform Enterprise.
2020-06-01 14:49:43 -07:00
Alisdair McDiarmid 5e2b11657e command: Fix 0.12upgrade stub 2020-06-01 16:12:30 -04:00
Kristin Laemmert 020084f6d0 update e2etests for windows compatibility 2020-05-29 11:57:50 -04:00
Alisdair McDiarmid 83482770c4
Merge pull request #25055 from hashicorp/alisdair/detected-legacy-provider-on-init
command/init: Better diagnostics for provider 404s
2020-05-29 11:57:18 -04:00
Martin Atkins 31a4b44d2e backend/local: treat output changes as side-effects to be applied
This is a baby-step towards an intended future where all Terraform actions
which have side-effects in either remote objects or the Terraform state
can go through the plan+apply workflow.

This initial change is focused only on allowing plan+apply for changes to
root module output values, so that these can be written into a new state
snapshot (for consumption by terraform_remote_state elsewhere) without
having to go outside of the primary workflow by running
"terraform refresh".

This is also better than "terraform refresh" because it gives an
opportunity to review the proposed changes before applying them, as we're
accustomed to with resource changes.

The downside here is that Terraform Core was not designed to produce
accurate changesets for root module outputs. Although we added a place for
it in the plan model in Terraform 0.12, Terraform Core currently produces
inaccurate changesets there which don't properly track the prior values.

We're planning to rework Terraform Core's evaluation approach in a
forthcoming release so it would itself be able to distinguish between the
prior state and the planned new state to produce an accurate changeset,
but this commit introduces a temporary stop-gap solution of implementing
the logic up in the local backend code, where we can freeze a snapshot of
the prior state before we take any other actions and then use that to
produce an accurate output changeset to decide whether the plan has
externally-visible side-effects and render any changes to output values.

This temporary approach should be replaced by a more appropriately-placed
solution in Terraform Core in a release, which should then allow further
behaviors in similar vein, such as user-visible drift detection for
resource instances.
2020-05-29 07:36:40 -07:00
Alisdair McDiarmid ca40107066 command/init: Better diagnostics for provider 404s
Fetching a default namespace provider from the public registry can
result in 404 Not Found error. This might be caused by a previously-
default provider moving to a new namespace, which means that the
configuration needs to be upgraded to use an explicit provider source.

This commit adds a more detailed diagnostic for this situation,
suggesting that the intended provider might be in a new namespace. The
recommended course of action is to run the 0.13upgrade command to
generate the correct required_providers configuration.
2020-05-28 09:24:32 -04:00
Alisdair McDiarmid ef28671b34
Merge pull request #24932 from hashicorp/signing-language
Modify language for reporting signing state
2020-05-28 09:09:34 -04:00
Paddy 5127f1ef8b
command: Unmanaged providers
This adds supports for "unmanaged" providers, or providers with process
lifecycles not controlled by Terraform. These providers are assumed to
be started before Terraform is launched, and are assumed to shut
themselves down after Terraform has finished running.

To do this, we must update the go-plugin dependency to v1.3.0, which
added support for the "test mode" plugin serving that powers all this.

As a side-effect of not needing to manage the process lifecycle anymore,
Terraform also no longer needs to worry about the provider's binary, as
it won't be used for anything anymore. Because of this, we can disable
the init behavior that concerns itself with downloading that provider's
binary, checking its version, and otherwise managing the binary.

This is all managed on a per-provider basis, so managed providers that
Terraform downloads, starts, and stops can be used in the same commands
as unmanaged providers. The TF_REATTACH_PROVIDERS environment variable
is added, and is a JSON encoding of the provider's address to the
information we need to connect to it.

This change enables two benefits: first, delve and other debuggers can
now be attached to provider server processes, and Terraform can connect.
This allows for attaching debuggers to provider processes, which before
was difficult to impossible. Second, it allows the SDK test framework to
host the provider in the same process as the test driver, while running
a production Terraform binary against the provider. This allows for Go's
built-in race detector and test coverage tooling to work as expected in
provider tests.

Unmanaged providers are expected to work in the exact same way as
managed providers, with one caveat: Terraform kills provider processes
and restarts them once per graph walk, meaning multiple times during
most Terraform CLI commands. As unmanaged providers can't be killed by
Terraform, and have no visibility into graph walks, unmanaged providers
are likely to have differences in how their global mutable state behaves
when compared to managed providers. Namely, unmanaged providers are
likely to retain global state when managed providers would have reset
it. Developers relying on global state should be aware of this.
2020-05-26 17:48:57 -07:00
Alisdair McDiarmid 5d0b75df7a
Merge pull request #25013 from hashicorp/alisdair/ui-data-source-read
command: Add UI hooks for read actions
2020-05-26 15:59:30 -04:00
Paul Tyng 22ef5cc99c Modify language for reporting signing state
Be more explicit about the signing status of fetched plugins and provide documentation about the different signing options.
2020-05-26 13:14:05 -04:00
Alisdair McDiarmid 62d826e066 command/init: Use full config for provider reqs
Relying on the early config for provider requirements was necessary in
Terraform 0.12, to allow the 0.12upgrade command to run after init
installs providers.

However in 0.13, the same restrictions do not apply, and the detection
of provider requirements has changed. As a result, the early config
loader gives incorrect provider requirements in some circumstances,
such as those in the new test in this commit.

Therefore we are changing the init command to use the requirements found
by the full configuration loader. This also means that we can remove the
internal initwd CheckCoreVersionRequirements function.
2020-05-25 16:50:12 -04:00
Alisdair McDiarmid 841ff590ea command: Add UI hooks for read actions 2020-05-21 16:02:18 -04:00
Kristin Laemmert eead4c49fe command/init: add e2e tests for provider not found messages 2020-05-20 11:04:11 -04:00
Kristin Laemmert 8d28d73de3 getproviders: add a registry-specific error and modify output when a
provider is not found.

Previously a user would see the following error even if terraform was
only searching the local filesystem:

"provider registry registry.terraform.io does not have a provider named
...."

This PR adds a registry-specific error type and modifies the MultiSource
installer to check for registry errors. It will return the
registry-specific error message if there is one, but if not the error
message will list all locations searched.
2020-05-20 11:04:11 -04:00
Kristin Laemmert a33a613703 command/init: add debug messages when -plugin-dir is set 2020-05-20 11:04:11 -04:00
Kristin Laemmert 0d620018fe
provider cache: log errors and validate dir exists (#24993)
* providercache: add logging for errors from getproviders.SearchLocalDirectory

providercache.fillMetaCache() was silently swallowing errors when
searching the cache directory. This commit logs the error without
changing the behavior otherwise.

* command/cliconfig: validate plugin cache dir exists

The plugin cache directory must exist for terraform to use it, so we
will add a check at the begining.
2020-05-19 15:32:36 -04:00
James Bardin 756e889697 remove unused DependsOn usage in state 2020-05-14 15:46:08 -04:00
Kristin Laemmert 862dc36884
command/init: include config loader diagnostics in output (#24921)
Previously the diagnostics from the config loaders (earlyconfig and
regular) were only appended to the overall diags if an error was found.
This adds all diagnostics from the regular config loader so that any
generated warnings will be displayed, even if there are no errors.

I did not add the `earlyconfig` warnings since they will be displayed if
there is an error and are likely to be duplicated by the config loader.
2020-05-12 08:39:12 -04:00
Kristin Laemmert 60321b41e8
getproviders: move protocol compatibility functions into registry client (#24846)
* internal/registry source: return error if requested provider version protocols are not supported

* getproviders: move responsibility for protocol compatibility checks into the registry client

The original implementation had the providercache checking the provider
metadata for protocol compatibility, but this is only relevant for the
registry source so it made more sense to move the logic into
getproviders.

This also addresses an issue where we were pulling the metadata for
every provider version until we found one that was supported. I've
extended the registry client to unmarshal the protocols in
`ProviderVersions` so we can filter through that list, instead of
pulling each version's metadata.
2020-05-11 13:49:12 -04:00
Alisdair McDiarmid 82ebbf9a23 command: Add prompt & confirmation to 0.13upgrade 2020-05-11 09:32:11 -04:00
Alisdair McDiarmid 62b0cbed12 internal: Fix LookupLegacyProvider
When looking up the namespace for a legacy provider source, we need to
use the /v1/providers/-/{name}/versions endpoint. For non-HashiCorp
providers, the /v1/providers/-/{name} endpoint returns a 404.

This commit updates the LegacyProviderDefaultNamespace method and the
mock registry servers accordingly.
2020-05-08 12:29:25 -04:00
Alisdair McDiarmid 1fdcbc4825 command: Fix 0.13upgrade bug with multiple blocks
If a configuration had multiple blocks in the versions.tf file, it would
be added to the `rewritePaths` list multiple times. We would then remove
it from this slice, but only once, and so the output file would later be
rewritten to remove the required providers block.

This commit uses a set instead of a list to prevent this case, and adds
a regression test.
2020-05-07 20:11:44 -04:00
Alisdair McDiarmid e2be704d81 command: Extract reused hclwrite helper code 2020-05-07 20:11:44 -04:00
Alisdair McDiarmid a740b739e0 command: Change 0.13upgrade default to versions.tf
Instead of using providers.tf as the default output file for the
upgrader, we now default to versions.tf. This means that if the
configuration has no `required_providers` blocks at all, or has
multiple, the provider version requirements will be stored in the
versions.tf file.

We now also update the versions.tf file to set a `required_version`
attribute in the first `terraform` block, with value ">= 0.13". This
is similar to the behaviour of the 0.12upgrade command, and signals that
the configuration should not be used with older versions of Terraform.
2020-05-07 15:45:48 -04:00
Alisdair McDiarmid 01a3376ead command: Check required_version before upgrading
If a configuration has a version constraint which prevents use with
Terraform 0.13, the upgrade command should exit before making any
changes.
2020-05-07 15:45:48 -04:00
Alisdair McDiarmid ae98bd12a7 command: Rework 0.13upgrade sub-command
This commit implements most of the intended functionality of the upgrade
command for rewriting configurations.

For a given module, it makes a list of all providers in use. Then it
attempts to detect the source address for providers without an explicit
source.

Once this step is complete, the tool rewrites the relevant configuration
files. This results in a single "required_providers" block for the
module, with a source for each provider.

Any providers for which the source cannot be detected (for example,
unofficial providers) will need a source to be defined by the user. The
tool writes an explanatory comment to the configuration to help with
this.
2020-05-07 11:38:55 -04:00
Mike Morris 9568de6b90
command: Fix OAuth2 PKCE arguments
Providers like Okta and AWS Cognito expect that the PKCE challenge
uses base64 URL Encoding without any padding (base64.RawURLEncoding)

Additionally, Okta strictly adheres to section 4.2 of RFC 7636 and
requires that the unencoded key for the PKCE data is at least 43
characters in length.
2020-05-05 09:58:48 -07:00
Kristin Laemmert 0b76100da0
init: return proper config errors (#24865)
Fixed a bug where we were returning earlyConfDiags instead of confDiags.
2020-05-05 10:08:05 -04:00
James Bardin 9debd341bc
Merge pull request #24807 from hashicorp/jbardin/remove-each-mode
Remove EachMode from state
2020-05-03 10:54:07 -04:00
Kristin Laemmert ce03f1255f
internal/providercache: fix error message for protocol mismatch (#24818)
There was a bug in the installer trying to pass a nil error.
2020-04-30 11:12:04 -04:00
James Bardin 2bfaddcf57 fix state mv to work without EachMode
The only situation where `state mv` needs to understand the each mode is
when with resource addresses that may reference a single instance, or a
group of for_each or count instances. In this case we can differentiate
the two by checking the existence of the NoKey instance key.
2020-04-30 09:22:15 -04:00
James Bardin 15a95031e5 remove a few traces of states.EachMode 2020-04-30 09:22:15 -04:00
Pam Selle 87bce5f9dd
Support reading module outputs in terraform console (#24808)
* Include eval in output walk

This allows outputs to be evaluated in the evalwalk,
impacting terraform console. Outputs are still not evaluated
for terraform console in the root module, so this has
no impact on writing to state (as child module outputs are not
written to state). Also adds test coverage to the console command,
including for evaluating locals (another use of the evalwalk)
2020-04-30 09:21:42 -04:00
Alisdair McDiarmid 9266e944fa
Merge pull request #24752 from hashicorp/alisdair/terraform-state-replace-provider-docs
website: Add docs for state replace-provider
2020-04-24 16:33:08 -04:00
Martin Atkins 1ce3c60693
command/cliconfig: Explicit provider installation method configuration
This set of commits allows explicit configuration of provider installation methods
in the CLI config, overriding the implicit method selections.
2020-04-23 10:58:00 -07:00
Martin Atkins 622abf707d command/cliconfig: Remove redundant struct types
These were being used in an earlier iteration of the provider installation
configuration but it was all collapsed down into a single
ProviderInstallationMethod type later, making these redundant.
2020-04-23 10:52:01 -07:00
Martin Atkins 6b2050f42a main: Properly handle provider installation method exclusions
Previously we were incorrectly using the Include configuration for both
the include and exclude list, making the include portion totally
ineffective.
2020-04-23 10:52:01 -07:00
Martin Atkins 3167067029 command/e2etest: provider installation with explicit install methods
This exercises the ability to customize the installation methods used by
the provider plugin installer, in this case forcing the use of a custom
local directory with a result essentially the same as what happens when
you pass -plugin-dir to "terraform init".
2020-04-23 10:52:01 -07:00
Martin Atkins c7fe6b9160 command/cliconfig: handle provider_installation block in JSON syntax
The CLI config can be written in both native HCL and HCL JSON syntaxes, so
the provider_installation block must be expressible using JSON too. Our
previous checks to approximate HCL 2-level strictness were too strict for
HCL JSON where things are more ambiguous even in HCL 2, so this includes
some additional relaxations if we detect that we're decoding an AST
produced from a JSON file.

This is still subject to the quirky ways HCL 1 handles JSON though, so
the JSON value must be structured in a way that doesn't trigger HCL's
heuristics that try to guess what is a block and what is an attribute.
(This is the issue that HCL 2 fixes by always decoding using a schema;
there's more context on this in:
  https://log.martinatkins.me/2019/04/25/hcl-json/ )
2020-04-23 10:52:01 -07:00
Martin Atkins f5012c12da command/cliconfig: Installation methods, not installation sources
Unfortunately in the user model the noun "source" is already used for the
argument in the required_providers block to specify which provider to use,
so it's confusing to use the same noun to also refer to the method used to
obtain that provider.

In the hope of mitigating that confusion, here we use the noun "method",
as in "installation method", to talk about the decision between getting
a provider directly from its origin registry or getting it from some
mirror. This is distinct from the provider's "source", which is the
location where a provider _originates_ (prior to mirroring).

This noun is also not super awesome, but better than overloading an
existing term in the same feature.
2020-04-23 10:52:01 -07:00
Martin Atkins e872ec4461 command/cliconfig: Remove remnant extraArg checks in provider_installation
In the first pass of implementing this it was strict about what arguments
are allowed inside source blocks, but that was counter to our usual design
principles for CLI config where we tend to ignore unrecognized things to
allow for some limited kinds of future expansion without breaking
compatibility with older versions of Terraform that will be sharing the
same CLI configuration files with newer versions.

However, I'd removed the tracking of that prior to the initial commit. I
missed some leftover parts when doing that removal, so this cleans up the
rest of it.
2020-04-23 10:52:01 -07:00
Martin Atkins 8b75d1498f command/cliconfig: Use existing HTTP mirror source rather than new stub
An earlier commit added a redundant stub for a new network mirror source
that was already previously stubbed as HTTPMirrorSource.

This commit removes the unnecessary extra stub and changes the CLI config
handling to use it instead. Along the way this also switches to using a
full base URL rather than just a hostname for the mirror, because using
the usual "Terraform-native service discovery" protocol here doesn't isn't
as useful as in the places we normally use it (the mirror mechanism is
already serving as an indirection over the registry protocol) and using
a direct base URL will make it easier to deploy an HTTP mirror under
a path prefix on an existing static file server.
2020-04-23 10:52:01 -07:00
Martin Atkins b8856c677c cliconfig: Ignore config dir if TF_CLI_CONFIG_FILE envvar is set
When we originally introduced this environment variable it was intended to
solve for the use-case where a particular invocation of Terraform needs
a different CLI configuration than usual, such as if Terraform is being
run as part of an automated test suite or other sort of automated
situation with different needs than normal use.

However, we accidentally had it only override the original singleton CLI
config file, while leaving the CLI configuration directory still enabled.
Now we'll take the CLI configuration out of the equation too, so that only
the single specified configuration file and any other environment-sourced
settings will be included.
2020-04-23 10:52:01 -07:00
Alisdair McDiarmid 52d64127f7 website: Add docs for state replace-provider 2020-04-23 11:41:57 -04:00
Kristin Laemmert 21b9da5a02
internal/providercache: verify that the provider protocol version is compatible (#24737)
* internal/providercache: verify that the provider protocol version is
compatible

The public registry includes a list of supported provider protocol
versions for each provider version. This change adds verification of
support and adds a specific error message pointing users to the closest
matching version.
2020-04-23 08:21:56 -04:00
Martin Atkins 2c535d829d command/cliconfig: Decode provider_installation blocks
This new CLI config block type allows explicitly specifying where
Terraform should look to find provider plugins for installation. This is
not used anywhere as of this commit, but in a future commit we'll change
package main to treat the presence of a block of this type as a request
to disable the default set of provider sources and use these explicitly-
specified ones instead.
2020-04-21 15:48:07 -07:00
Kristin Laemmert 8108face36
terraform: return `initialization required` error when provider schemas not found (#24715)
A side effect of the various changes to the provider installer included losing the initialization required error message which would occur if a user removed or modified the .terraform directory.

Previously, plugin factories were created after the configuration was loaded, in terraform.NewContext. Terraform would compare the required providers (from config and state) to the available providers and return the aforementioned error if a provider was missing.

Provider factories are now loaded at the beginning of any terraform command, before terraform even loads the configuration, and therefore before terraform has a list of required providers.

This commit replaces the current error when a providers' schema cannot be found in the provider factories with the init error, and adds a command test (to plan tests, for no real reason other than that's what I thought of first).
2020-04-21 16:29:27 -04:00
Alisdair McDiarmid e32e7e2c4b
Merge pull request #24617 from hashicorp/alisdair/provider-installer-signature-verification
internal: Verify provider signatures on install
2020-04-20 12:11:40 -04:00
Martin Atkins 92d6a30bb4 main: skip direct provider installation for providers available locally
This more closely replicates the 0.12-and-earlier behavior, where having
at least one version of a provider installed locally would totally disable
any attempt to look for newer versions remotely.

This is just for the implicit default behavior. Assumption is that later
we'll have an explicit configuration mechanism that will allow the user
to specify exactly where to look for what, and thus avoid tricky
heuristics like this.
2020-04-17 13:55:11 -07:00
Alisdair McDiarmid a5b3d497cc internal: Verify provider signatures on install
Providers installed from the registry are accompanied by a list of
checksums (the "SHA256SUMS" file), which is cryptographically signed to
allow package authentication. The process of verifying this has multiple
steps:

- First we must verify that the SHA256 hash of the package archive
  matches the expected hash. This could be done for local installations
  too, in the future.
- Next we ensure that the expected hash returned as part of the registry
  API response matches an entry in the checksum list.
- Finally we verify the cryptographic signature of the checksum list,
  using the public keys provided by the registry.

Each of these steps is implemented as a separate PackageAuthentication
type. The local archive installation mechanism uses only the archive
checksum authenticator, and the HTTP installation uses all three in the
order given.

The package authentication system now also returns a result value, which
is used by command/init to display the result of the authentication
process.

There are three tiers of signature, each of which is presented
differently to the user:

- Signatures from the embedded HashiCorp public key indicate that the
  provider is officially supported by HashiCorp;
- If the signing key is not from HashiCorp, it may have an associated
  trust signature, which indicates that the provider is from one of
  HashiCorp's trusted partners;
- Otherwise, if the signature is valid, this is a community provider.
2020-04-17 13:57:19 -04:00
Kristin Laemmert f09ae6f862
provider source tests: added test suite to exercise hyphenated providers (#24685) 2020-04-16 15:54:33 -04:00
Kristin Laemmert 269d511481 command/providers: refactor with new provider types and functions
The providers command has been refactored to use the modern provider types and
ProviderRequirements() functions. This resulted in a breaking change to
the output: it no longer outputs the providers by module and no longer
prints `(inherited)` or `(from state)` to show why a provider is
included. We decided that at this time it was best to stick with the
existing functions and make this change, but if we get feedback from the
community we will revisit.

Additional tests to exercise providers in modules and providers from
state have been included.
2020-04-10 15:08:10 -04:00
Kristin Laemmert 0a5fb40fdf configs: include provider configs in ProviderRequirements()
This PR adds iteration through any provider configuration blocks in the
config in addProviderRequirements().

A stale comment (of mine!) would leave one expecting the
module.ProviderRequirements to include any requirements from provider
configs. The comment was inaccurate and has been updated.
2020-04-10 15:08:10 -04:00
James Bardin b84e6f7f95 remove race from closed-over err variable 2020-04-08 10:12:46 -04:00
Martin Atkins 297a3a5db9 command/init: Restore the unconstrained provider warnings
When a provider dependency is implicit rather than explicit, or otherwise
when version constraints are lacking, we produce a warning recommending
the addition of explicit version constraints in the configuration.

This restores the warning functionality from previous Terraform versions,
adapting it slightly to account for the new provider FQN syntax and to
recommend using a required_providers block rather than version constraints
in "provider" blocks, because the latter is no longer recommended in the
documentation.
2020-04-06 09:24:23 -07:00
Martin Atkins a6f63c4891 command/e2etest: update "init" tests for abbreviated provider addresses
We're now longer showing the default registry hostname as part of
addresses coming from that registry.
2020-04-06 09:24:23 -07:00
Kristin Laemmert 5852a5c33d Mildwonkey/last tests (#24553)
* show text fixture update
* temporarily disable providers tests
2020-04-06 09:24:23 -07:00
Alisdair McDiarmid 0f5a38b384 internal: Fix init provider lockfile test
The fake installable package meta used a ZIP archive which gave
different checksums between macOS and Linux targets. This commit removes
the target from the contents of this archive, and updates the golden
hash value in the test to match. This test should now pass on both
platforms.
2020-04-06 09:24:23 -07:00
Alisdair McDiarmid b233aa39e9 addrs: Simplify presentation of provider FQNs
The provider fully-qualified name string used in configuration is very
long, and since most providers are hosted in the public registry, most
of that length is redundant. This commit adds and uses a `ForDisplay`
method, which simplifies the presentation of provider FQNs.

If the hostname is the default hostname, we now display only the
namespace and type. This is only used in UI, but should still be
unambiguous, as it matches the FQN string parsing behaviour.
2020-04-06 09:24:23 -07:00
Martin Atkins 8c928e8358 main: Consult local directories as potential mirrors of providers
This restores some of the local search directories we used to include when
searching for provider plugins in Terraform 0.12 and earlier. The
directory structures we are expecting in these are different than before,
so existing directory contents will not be compatible without
restructuring, but we need to retain support for these local directories
so that users can continue to sideload third-party provider plugins until
the explicit, first-class provider mirrors configuration (in CLI config)
is implemented, at which point users will be able to override these to
whatever directories they want.

This also includes some new search directories that are specific to the
operating system where Terraform is running, following the documented
layout conventions of that platform. In particular, this follows the
XDG Base Directory specification on Unix systems, which has been a
somewhat-common request to better support "sideloading" of packages via
standard Linux distribution package managers and other similar mechanisms.
While it isn't strictly necessary to add that now, it seems ideal to do
all of the changes to our search directory layout at once so that our
documentation about this can cleanly distinguish "0.12 and earlier" vs.
"0.13 and later", rather than having to document a complex sequence of
smaller changes.

Because this behavior is a result of the integration of package main with
package command, this behavior is verified using an e2etest rather than
a unit test. That test, TestInitProvidersVendored, is also fixed here to
create a suitable directory structure for the platform where the test is
being run. This fixes TestInitProvidersVendored.
2020-04-06 09:24:23 -07:00
Martin Atkins fcb8c53454 command/jsonconfig: Use correct provider address to access schema
There was a remaining TODO in this package to find the true provider FQN
when looking up the schema for a resource type. We now have that data
available in the Provider field of configs.Resource, so we can now
complete that change.

The tests for this functionality actually live in the parent "command"
package as part of the tests for the "terraform show" command, so this
fix is verified by all of the TestShow... tests now passing except one,
and that remaining one is failing for some other reason which we'll
address in a later commit.
2020-04-06 09:24:23 -07:00
Martin Atkins 69cae48a11 command/import: fix TestImport_initializationErrorShouldUnlock
This was checking for a specific output error message which has changed
due to our new provider installer/selection approach.
2020-04-06 09:24:23 -07:00
Martin Atkins 958ea4f7d1 internal/providercache: Handle built-in providers
Built-in providers are special providers that are distributed as part of
Terraform CLI itself, rather than being installed separately. They always
live in the terraform.io/builtin/... namespace so it's easier to see that
they are special, and currently there is only one built-in provider named
"terraform".

Previous commits established the addressing scheme for built-in providers.
This commit makes the installer aware of them to the extent that it knows
not to try to install them the usual way and it's able to report an error
if the user requests a built-in provider that doesn't exist or tries to
impose a particular version constraint for a built-in provider.

For the moment the tests for this are the ones in the "command" package
because that's where the existing testing infrastructure for this
functionality lives. A later commit should add some more focused unit
tests here in the internal/providercache package, too.
2020-04-06 09:24:23 -07:00
Kristin Laemmert 27a794062e Mildwonkey/command tests (#24535)
* command: refactor testBackendState to write states.State

testBackendState was using the older terraform.State format, which is no
longer sufficient for most tests since the state upgrader does not
encode provider FQNs automatically. Users will run `terraform
0.13upgrade` to update their state to include provider FQNs in
resources, but tests need to use the modern state format instead of
relying on the automatic upgrade.

* plan tests passing
* graph tests passing
* json packages test update
* command test updates
* update show test fixtures
* state show tests passing
2020-04-06 09:24:23 -07:00
Martin Atkins de6c9ccec1 command/init: Move "vendored provider" test to e2etests
In the new design the ProviderSource is decided by package main, not by
the "command" package, and so making sure the vendor directory is included
is the responsibility of that package instead. Therefore we can no longer
test this at the "command" package level, but we'll retain a test for it
in e2etests to record that it isn't currently working, so that we have
a prompt to fix it before releasing.
2020-04-06 09:24:23 -07:00
Martin Atkins f35ebe2d65 internal/providercache: Fix incorrect logic in Installer.SetGlobalCacheDir
Due to some incomplete rework of this function in an earlier commit, the
safety check for using the same directory as both the target and the
cache was inverted and was raising an error _unless_ they matched, rather
than _if_ they matched.

This change is verified by the e2etest TestInitProviders_pluginCache,
which is also updated to use the new-style cache directory layout as part
of this commit.
2020-04-06 09:24:23 -07:00
Martin Atkins 14d456372a command/e2etest: Update expected output for new plugin installer
These tests make assertions against specific user-oriented output from the
"terraform init" command, but we've intentionally changed some of these
messages as part of introducing support for the decentralized provider
namespace.
2020-04-06 09:24:23 -07:00
Kristin Laemmert 0af09b23ca command: apply and most of import tests passing 2020-04-06 09:24:23 -07:00
Martin Atkins fe9105abfe command: Fix TestInit_getProviderMissing
This one just needs a tweak for the new wording of the error message.
2020-04-06 09:24:23 -07:00
Martin Atkins add7006de6 command: Fix TestInit_pluginDirProviders and _pluginDirProvidersDoesNotGet
Both of these are attempting to test -plugin-dir, which means we need some
additional help to populate some suitable directories for -plugin-dir to
refer to. The new installFakeProviderPackagesElsewhere helper generalizes
the earlier installFakeProviderPackages to allow installing fake provider
packages to an arbitrary other directory.
2020-04-06 09:24:23 -07:00
Martin Atkins c81eebe0ac command/init: Fix TestInit_providerSource
This test is focused on making sure that the required_providers syntax
is working, so the rewritten version does not include any special handling
of pre-installed packages or "vendored" packages. Pre-installed plugins
are tested in other tests such as TestInit_getUpgradePlugins.
2020-04-06 09:24:23 -07:00
Martin Atkins 14701b8300 command/init: Fix TestInit_getUpgradePlugins
This test now requires a bit of a different approach because it was
previously directly constructing a cache directory but we now use a
different directory layout.

Rather than manually constructing the new heirarchical directory layout
(which would've required a lot more inline code), this introduces a helper
function installFakeProviderPackages that installs a fake provider package
directly into the local cache directory associated with a Meta object,
with the correct directory layout.
2020-04-06 09:24:23 -07:00
Martin Atkins c4fb22863c command/init: Support -plugin-dir again
This is a slightly different approach than we used to take for this
option: rather than disabling the installer and causing all future
commands to look elsewhere for plugins, we'll now leave the installer
enabled by constrain it to only look at the given directories.

This is overall simpler because it doesn't require any special tracking
of the plugin directories for subsequent commands. Instead, the selections
file generated by the installer will record the versions it selected from
the specified directories, and we'll link them in to the local cache just
as we would normally so that other commands don't need to do anything
special to select the right plugins in either case.
2020-04-06 09:24:23 -07:00
Martin Atkins d40085f374 command: Make the tests compile again
They still aren't passing, but this is just enough updating to make the
test program compile successfully after the refactoring related to
provider installation. They are now using the mock provider source offered
by the getproviders package, which is similar but not totally identical
to the idea of mocking the entire installer as these tests used to do, and
so many of them need further adjustment to still be testing what they
intended to test under this new architecture.

Subsequent commits will gradually repair the failing tests.
2020-04-06 09:24:23 -07:00
Kristin Laemmert 3f6ce3c588 Mildwonkey/tests (#24522)
* terraform: add helper functions for creating test state

testSetResourceInstanceCurrent and testSetResourceInstanceTainted are
wrapper functions around states.Module.SetResourceInstanceCurrent()
used to set a resource in state. They work with current, non-deposed
resources with no dependencies.

testSetResourceInstanceDeposed can be used to set a desosed resource in state.

* terraform: update all tests to use modern providers and state
2020-04-06 09:24:23 -07:00
Martin Atkins 549aede792 Remove terraform.ResourceProvider, use providercache.Installer instead
Back when we first introduced provider versioning in Terraform 0.10, we
did the provider version resolution in terraform.NewContext because we
weren't sure yet how exactly our versioning model was going to play out
(whether different versions could be selected per provider configuration,
for example) and because we were building around the limitations of our
existing filesystem-based plugin discovery model.

However, the new installer codepath is new able to do all of the
selections up front during installation, so we don't need such a heavy
inversion of control abstraction to get this done: the command package can
select the exact provider versions and pass their factories directly
to terraform.NewContext as a simple static map.

The result of this commit is that CLI commands other than "init" are now
able to consume the local cache directory and selections produced by the
installation process in "terraform init", passing all of the selected
providers down to the terraform.NewContext function for use in
implementing the main operations.

This commit is just enough to get the providers passing into the
terraform.Context. There's still plenty more to do here, including to
repair all of the tests this change has additionally broken.
2020-04-06 09:24:23 -07:00
Martin Atkins f113a7c22d command/init: Collect provider dependencies using our new helpers
This produces a value shaped the way the provider installer expects
without the need for further flattening and preprocessing.
2020-04-06 09:24:23 -07:00
Martin Atkins e6df3905c9 command/init: Generate progress output during provider installation
This restores some sequential event log output similar to what we had in
the previous implementation of plugin installation.
2020-04-06 09:24:23 -07:00
Martin Atkins 94e1ac2d07 command: Minimal integration of new provider installer in "init"
There's still a lot of work to do here around both the UX and the
follow-up steps that need to happen after installation completes, but this
is enough to faciliate some initial end-to-end testing of the new-style
install process.
2020-04-06 09:24:23 -07:00
Alisdair McDiarmid 7165d6c429 command: Add state replace-provider subcommand
Terraform 0.13 will allow the installation of providers from various
sources. If a user updates their configuration to change the source of
an in-use provider (for example, if the provider namespace changes),
they will also need to update the state file accordingly.

This commit introduces a new `state replace-provider` subcommand which
supports this. All resources using the `from` provider will be updated
to use the `to` provider.
2020-04-02 08:15:52 -04:00
Alisdair McDiarmid 67203dade8 command: Simplify Meta.process helper method
After some refactoring, this helper method had an unused argument (vars)
and an always-nil error return value. This commit cleans this up.
2020-04-01 15:01:08 -04:00
Alisdair McDiarmid 7b94ddd2f6
Merge pull request #24471 from hashicorp/alisdair/fmt-source
command/fmt: Include source snippets in errors
2020-03-27 09:19:12 -04:00
Alisdair McDiarmid 7302bfb484
Merge pull request #24473 from hashicorp/alisdair/fix-multiline-diagnostic-output
command/format: Fix multi-line diagnostic output
2020-03-27 09:18:51 -04:00
Alisdair McDiarmid a23c3e3c81 command/format: Fix multi-line diagnostic output
Previously, if a diagnostic context spanned multiple lines, any lines
which did not overlap with the highlight range would be displayed as
blank. This commit fixes the bug.

The problem was caused by the unconditional use of `PartitionAround` to
split the line into before/highlighted/after ranges. When two ranges
don't overlap, this method returns empty ranges, which results in a
blank line. Instead, we first check if the ranges do overlap, and if not
we print the entire line from the context.
2020-03-26 15:53:52 -04:00
Alisdair McDiarmid 206e2e6d6a command/fmt: Include source snippets in errors
Previously, diagnostic errors would display the filename and line
number, along with "(source code not available)". This is because the
fmt command directly loads and parses the configuration, instead of
using the config loader.

This commit registers the manually parsed source as a synthetic
configuration file, so that the diagnostic formatter can look up the
source for the range with the error and display it.
2020-03-26 14:00:26 -04:00
James Bardin b3fc0dab94 use addrs.ConfigResource for dependency tracking
We can't get module instances during transformation, so we need to
reduce the Dependencies to using `addrs.ConfigResource` for now.
2020-03-25 17:03:06 -04:00
Martin Atkins 754b7ebb65 command: Expose providercache package objects for use elsewhere
These new functions allow command implementations to get hold of the
providercache objects and installation source object derived from the
current CLI configuration.
2020-03-25 11:29:48 -07:00
Kristin Laemmert 4f141ae365
rename provider-specific functions (#24417)
missingPlugins was hard-coded to work only with provider plugins, so I
renamed it to clarify the usage.

Also renamed a test provider from greater_than to greater-than as the
underscore is an invalid provider name character and this will become a
hard error in the near future.
2020-03-20 13:59:59 -04:00
Kristin Laemmert c8d64846ad
Mildwonkey/ps import (#24412)
* import: remove Config from ImportOpts

`Config` in ImportOpts was any provider configuration provided by the
user on the command line. This option has already been removed in favor
of only taking the provider from the configuration loaded in the current
context.

* terrafrom: add Config to ImportStateTransformer and refactor Transform
to get the resource provider FQN from the Config
2020-03-20 08:15:29 -04:00
Kristin Laemmert 5f313a65ad
command: remove 0.12upgrade (#24403)
* command: remove 0.12upgrade and related `configupgrade` library
* leave deprecation warning for 0.12upgrade to point users to v0.12
2020-03-19 08:01:16 -04:00
Alisdair McDiarmid 8c7a44355b command: Fix bug when using terraform login on Windows 2020-03-18 09:08:40 -04:00
Alisdair McDiarmid fb7035ac3e
Merge pull request #24364 from hashicorp/alisdair/013upgrade
command: Add scaffold for 0.13upgrade command
2020-03-18 09:07:58 -04:00
James Bardin a8b9547e0d fixup states.Resource change throughout packages 2020-03-16 16:50:48 -04:00
Alisdair McDiarmid 3b0b29ef52 command: Add scaffold for 0.13upgrade command 2020-03-16 12:50:24 -04:00
James Bardin e13eecbc5b finish provider ModuleInstance replacement 2020-03-11 14:19:52 -04:00
Kristin Laemmert 5901952882
command: tests should not leave dirs behind thank you (#24340) 2020-03-10 16:32:22 -04:00
Kristin Laemmert add16fc67b
jsonstate: sort child modules by address for consistency (#24329)
* jsonstate: sort child modules by address for consistency
2020-03-09 15:57:14 -04:00
Paddy e6592dc710
Add support for provider metadata to modules. (#22583)
Implement a new provider_meta block in the terraform block of modules, allowing provider-keyed metadata to be communicated from HCL to provider binaries.

Bundled in this change for minimal protocol version bumping is the addition of markdown support for attribute descriptions and the ability to indicate when an attribute is deprecated, so this information can be shown in the schema dump.

Co-authored-by: Paul Tyng <paul@paultyng.net>
2020-03-05 16:53:24 -08:00
Kristin Laemmert 7f1b0a4681
command/jsonstate: fix inconsistency with resource address (#24256)
* command/jsonstate: fix inconsistency with resource address

Resource addresses in state output were not including index for
instances created with for_each or count, while the index was appearing
in the plan output. This PR fixes that inconsistency, adds tests, and
updates the existing tests.

Fixes #24110

* add tests showing expected prior state resource addressing
* added example of show json state output with modules
2020-03-05 08:13:45 -05:00
James Bardin 10d926904f state mv should always target instance each mode
When doing a state mv of an instance, the resulting each mode should
always be taken from the target address.
2020-03-02 14:45:03 -05:00
Martin Atkins f899f5aa42 command/init: new error message for signature verification failure
Our initial Terraform 0.13.0 release will continue to support only the
hard-coded official HashiCorp signing key, with support for other keys to
follow in a later release once the trust infrastructure is in place to
support that.

This change is intended to (marginally) improve the UX for a possible
future situation where a HashiCorp-distributed provider makes a released
signed with a new key and a prior version of Terraform ends up trying to
install it due to incorrect version constraints. With this new text we
hope to give the user a better prompt for onward troubleshooting, but
in a sort of hedging way because we have not yet finalized the details of
how new keys might roll out in practice.

Hopefully a user seeing this message would consult the release notes for
Terraform itself and for the provider in question and find some
as-yet-undetermined information about how to proceed.

If the decentralized trust model design comes together before the v0.13.0
release then we may make further amendments here to prepare for that, but
that work should not block the v0.13.0 release if other work concludes
first.
2020-02-25 10:31:54 -05:00
Pam Selle c249943360
Module Expansion: Part 2 (#24154)
* WIP: dynamic expand

* WIP: add variable and local support

* WIP: outputs

* WIP: Add referencer

* String representation, fixing tests it impacts

* Fixes TestContext2Apply_outputOrphanModule

* Fix TestContext2Apply_plannedDestroyInterpolatedCount

* Update DestroyOutputTransformer and associated types to reflect PlannableOutputs

* Remove comment about locals

* Remove module count enablement

* Removes allowing count for modules, and reverts the test,
while adding a Skip()'d test that works when you re-enable
the config

* update TargetDownstream signature to match master

* remove unnecessary method

Co-authored-by: James Bardin <j.bardin@gmail.com>
2020-02-24 17:42:32 -05:00
James Bardin 1b45b744c3 remove json2dot command
There's no need for the json2dot command since we can't create json
debug graphs.
2020-02-19 14:53:19 -05:00
Alisdair McDiarmid 678760b61a
Merge pull request #24122 from hashicorp/alisdair/unlock-commentary
command: Comment to explain why unlock is required
2020-02-14 17:04:47 -05:00
Alisdair McDiarmid 7696c14661
Merge pull request #24107 from hashicorp/alisdair/terraform-login-wsl-browser-launcher-fix
command/login: Fix browser launcher for WSL users
2020-02-14 16:32:35 -05:00
Alisdair McDiarmid aa38cbe3bc command: Comment to explain why unlock is required 2020-02-14 16:29:07 -05:00
Alisdair McDiarmid f0cf57e5d3 Remove unused import-provider-mismatch fixture 2020-02-14 14:49:26 -05:00
Alisdair McDiarmid 6413313529 command/graph: Remove no-op state unlock
The graph command never locks the state, so this unlock operation was a
no-op and is a bit of a red herring. Remove it.
2020-02-14 14:48:50 -05:00
Alisdair McDiarmid 701d095808 command: Fix stale lock when exiting early
If an error occurs on creating the context for console or import, we
would fail to unlock the state. Fix this by unlocking slightly earlier.
Affects console and import commands.

Fixes #23318
2020-02-14 14:48:30 -05:00
Kristin Laemmert add134298a
addrs: ProviderConfig fixups (#24115)
* fix outdated syntax in comments
* test for non-strings in ParseAbsProviderConfig
* ProviderConfigDefault and ProviderConfigAliased now take Providers
instead of strings
2020-02-14 09:06:50 -05:00
James Bardin dc8cdd260c add missing deps to targeted destroy test 2020-02-13 15:43:52 -05:00
Kristin Laemmert 47a16b0937
addrs: embed Provider in AbsProviderConfig instead of Type
a large refactor to addrs.AbsProviderConfig, embedding the addrs.Provider instead of a Type string. I've added and updated tests, added some Legacy functions to support older state formats and shims, and added a normalization step when reading v4 (current) state files (not the added tests under states/statefile/roundtrip which work with both current and legacy-style AbsProviderConfig strings).

The remaining 'fixme' and 'todo' comments are mostly going to be addressed in a subsequent PR and involve looking up a given local provider config's FQN. This is fine for now as we are only working with default assumption.
2020-02-13 15:32:58 -05:00
Alisdair McDiarmid 12e090ce48 command/login: Fix browser launcher for WSL users
With the current implementation of terraform login, Windows Subsystem
for Linux fails to open a browser due to lack of support for xdg-open.
This commit reuses a fix from pkg/browser#8 which detects a WSL
environment and uses cmd.exe to open the URL instead.
2020-02-13 15:02:40 -05:00
Kristin Laemmert 1c7167a9a1
command/import: remove -provider command line argument (#24090)
Now that #22862 has been merged, terraform will properly pick up the
resource provider configuration from state. We can remove the deprecated
`-provider` flag.
2020-02-12 14:45:41 -05:00
Kristin Laemmert 80862f3436
command/import: attach references before validating provider (#22862)
There was an order-of-operations bug where the import graph builder was
validating that the provider did not have any resource references before
references were actually being attached. This PR fixes the order of
operations and adds a test (in the command package).

Fixes #22804
2020-02-12 14:00:08 -05:00
Li Kexian f3bbe8d33f
command/workspace delete: release lock after workspace removal warning (#24085) 2020-02-12 10:34:51 -05:00
Alisdair McDiarmid 7ff58780d4 Remove unnecessary type assertion checks
The type assertion checks on the credentials source are unnecessary, and
the alternative code path they allow would panic.
2020-02-06 15:30:49 -05:00
Alisdair McDiarmid 325f8a84f8 command/logout: Exit early if already logged out 2020-02-06 15:00:55 -05:00
Alisdair McDiarmid 081f02971d command/logout: Add terraform logout command
Use terraform logout to remove stored credentials for a remote service
host.
2020-02-06 15:00:55 -05:00
Alisdair McDiarmid e57685d8fc
Merge pull request #24030 from hashicorp/alisdair/terraform-login-token-validation
Add token validation for manual terraform login
2020-02-05 10:07:28 -05:00
Kristin Laemmert 7eed30595a
moduledeps: replace ProviderInstance with addrs.Provider (#24017)
* addrs: add ParseProviderSourceString function to parse fqns from
tfconfig-inspect
* moduledeps: use addrs.Provider instead of ProviderInstance
2020-02-05 09:27:32 -05:00
Alisdair McDiarmid c77cfaafc2 Add token validation for manual terraform login
When a token is pasted by the user, we make a request to the
TFE API /account/details endpoint to verify its validity. If successful,
we display the logged-in username as confirmation. If not, we refuse to
store the invalid token and display an error message.

This commit also trims whitespace from around the pasted value, to
reduce the likelihood of a copy & paste error.
2020-02-04 17:24:25 -05:00
Kristin Laemmert 927999a820
command/state show: use configured provider (#24027)
The `state show`  command was not checking if a given resource had a
configured provider, and instead was only using the default provider
config. This PR checks for a configured provider, using the default
provider if one is not set.

Fixes #22010
2020-02-04 12:07:59 -05:00
Alisdair McDiarmid f34cba407f
Merge pull request #23995 from hashicorp/alisdair/terraform-login
Enable login subcommand, add manual token support
2020-02-04 11:28:27 -05:00
Kristin Laemmert 80ab551867
terraform: use addrs.Provider as map keys for provider schemas (#24002)
This is a stepping-stone PR for the provider source project. In this PR
"legcay-stype" FQNs are created from the provider name string. Future
work involves encoding the FQN directly in the AbsProviderConfig and
removing the calls to addrs.NewLegacyProvider().
2020-02-03 08:18:04 -05:00
Martin Atkins 8b511524d6
Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978)
* Introduce "Local" terminology for non-absolute provider config addresses

In a future change AbsProviderConfig and LocalProviderConfig are going to
become two entirely distinct types, rather than Abs embedding Local as
written here. This naming change is in preparation for that subsequent
work, which will also include introducing a new "ProviderConfig" type
that is an interface that AbsProviderConfig and LocalProviderConfig both
implement.

This is intended to be largely just a naming change to get started, so
we can deal with all of the messy renaming. However, this did also require
a slight change in modeling where the Resource.DefaultProviderConfig
method has become Resource.DefaultProvider returning a Provider address
directly, because this method doesn't have enough information to construct
a true and accurate LocalProviderConfig -- it would need to refer to the
configuration to know what this module is calling the provider it has
selected.

In order to leave a trail to follow for subsequent work, all of the
changes here are intended to ensure that remaining work will become
obvious via compile-time errors when all of the following changes happen:
- The concept of "legacy" provider addresses is removed from the addrs
  package, including removing addrs.NewLegacyProvider and
  addrs.Provider.LegacyString.
- addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded
  in it and has an addrs.Provider and a string alias directly instead.
- The provider-schema-handling parts of Terraform core are updated to
  work with addrs.Provider to identify providers, rather than legacy
  strings.

In particular, there are still several codepaths here making legacy
provider address assumptions (in order to limit the scope of this change)
but I've made sure each one is doing something that relies on at least
one of the above changes not having been made yet.

* addrs: ProviderConfig interface

In a (very) few special situations in the main "terraform" package we need
to make runtime decisions about whether a provider config is absolute
or local.

We currently do that by exploiting the fact that AbsProviderConfig has
LocalProviderConfig nested inside of it and so in the local case we can
just ignore the wrapping AbsProviderConfig and use the embedded value.

In a future change we'll be moving away from that embedding and making
these two types distinct in order to represent that mapping between them
requires consulting a lookup table in the configuration, and so here we
introduce a new interface type ProviderConfig that can represent either
AbsProviderConfig or LocalProviderConfig decided dynamically at runtime.

This also includes the Config.ResolveAbsProviderAddr method that will
eventually be responsible for that local-to-absolute translation, so
that callers with access to the configuration can normalize to an
addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's
currently unused because existing callers are still relying on the
simplistic structural transform, but we'll switch them over in a later
commit.

* rename LocalType to LocalName

Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-01-31 08:23:07 -05:00
Alisdair McDiarmid b75201acc2 Enable login subcommand, add manual token support 2020-01-30 09:55:38 -05:00
Kristin Laemmert 6541775ce4
addrs: roll back change to Type field in ProviderConfig (#23937) 2020-01-28 08:13:30 -05:00