Commit Graph

1590 Commits

Author SHA1 Message Date
Alisdair McDiarmid 60c469b4a5
Merge pull request #26345 from hashicorp/alisdair/taint-should-respect-required-version
command: Taint should respect required_version
2020-09-24 15:52:23 -04:00
Pam Selle 0a02e7040f
Store sensitive attribute paths in state (#26338)
* Add creation test and simplify in-place test

* Add deletion test

* Start adding marking from state

Start storing paths that should be marked
when pulled out of state. Implements deep
copy for attr paths. This commit also includes some
comment noise from investigations, and fixing the diff test

* Fix apply stripping marks

* Expand diff tests

* Basic apply test

* Update comments on equality checks to clarify current understanding

* Add JSON serialization for sensitive paths

We need to serialize a slice of cty.Path values to be used to re-mark
the sensitive values of a resource instance when loading the state file.
Paths consist of a list of steps, each of which may be either getting an
attribute value by name, or indexing into a collection by string or
number.

To serialize these without building a complex parser for a compact
string form, we render a nested array of small objects, like so:

[
  [
    { type: "get_attr", value: "foo" },
    { type: "index", value: { "type": "number", "value": 2 } }
  ]
]

The above example is equivalent to a path `foo[2]`.

* Format diffs with map types

Comparisons need unmarked values to operate on,
so create unmarked values for those operations. Additionally,
change diff to cover map types

* Remove debugging printing

* Fix bug with marking non-sensitive values

When pulling a sensitive value from state,
we were previously using those marks to remark
the planned new value, but that new value
might *not* be sensitive, so let's not do that

* Fix apply test

Apply was not passing the second state
through to the third pass at apply

* Consistency in checking for length of paths vs inspecting into value

* In apply, don't mark with before paths

* AttrPaths test coverage for DeepCopy

* Revert format changes

Reverts format changes in format/diff for this
branch so those changes can be discussed on a separate PR

* Refactor name of AttrPaths to AttrSensitivePaths

* Rename AttributePaths/attributePaths for naming consistency

Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
2020-09-24 12:40:17 -04:00
James Bardin 2e5366ac69 remove -mod=vendor from command tests 2020-09-23 17:56:19 -04:00
Alisdair McDiarmid 14a233b019 command: Taint should respect required_version
Despite not requiring the configuration for any other reason, the taint
subcommand should not execute if the required_version constraints cannot
be met. Doing so can result in an undesirable state file upgrade.
2020-09-22 17:33:09 -04:00
Alisdair McDiarmid 18f9ea53b9 command: Providers schema shows required_providers
The providers schema command is using the Config.ProviderTypes method,
which had not been kept up to date with the changes to provider
requirements detection made in Config.ProviderRequirements. This
resulted in any currently-unused providers being omitted from the
output.

This commit changes the ProviderTypes method to use the same underlying
logic as ProviderRequirements, which ensures that `required_providers`
blocks are taken into account.

Includes an integration test case to verify that this fixes the provider
schemas command bug.
2020-09-22 10:28:32 -04:00
James Bardin ab06f0c9f8 we can roll back the e2e tests
the data sources no longer show up in the tests
2020-09-22 09:55:19 -04:00
James Bardin c3182bd589 mock provider needs to return a valid response 2020-09-21 16:17:45 -04:00
James Bardin 3b3ff98356 Revert "fix show -json tests"
This reverts commit e54949f2e1.

Changes incorrectly applied to the planned state tests
2020-09-21 16:17:45 -04:00
James Bardin 86dd8938c9 data sources now show up in the initial plan 2020-09-17 09:55:00 -04:00
James Bardin e54949f2e1 fix show -json tests
The prior state recorded in the plans did not match the actual prior
state. Make the plans and state match depending on whether there was
existing state or not.
2020-09-17 09:55:00 -04:00
James Bardin ad5899d8bb ReadResource is called during plan but not destroy 2020-09-17 09:55:00 -04:00
Joe Ceresini 04b39372b0 Add scopes support to password grant type 2020-09-15 19:32:41 -04:00
Joe Ceresini 2ffa84a4de Add command/login test for scopes 2020-09-15 19:30:48 -04:00
Joe Ceresini 2acb7fac9f Include scopes in login URL 2020-09-14 12:21:20 -04:00
Alisdair McDiarmid 8b2b569d6e repl: Improved value renderer for console outputs
Use a slightly modified value renderer from terraform-provider-testing
to display values in the console REPL, as well as outputs from the apply
and outputs subcommands.

Derived from code in this repository, MIT licensed:

https://github.com/apparentlymart/terraform-provider-testing

Note that this is technically a breaking change for the console
subcommand, which would previously error if the user attempted to render
an unknown value (such as an unset variable). This was marked as an
unintentional side effect, with the goal being the new behaviour of
rendering "(unknown)", which is why I changed the behaviour in this
commit.
2020-09-14 09:47:12 -04:00
Pam Selle 6a126df0c6
Merge pull request #26183 from hashicorp/pselle/sensitive-values
Add sensitive attribute to variables
2020-09-11 11:24:18 -04:00
Pam Selle 8d8389da74 Add diff test with a sensitive change
Adds a diff test for a changed value,
and modifies the diff file to cover variable
diffs on sensitive values
2020-09-10 16:45:31 -04:00
Alisdair McDiarmid 59662c01af vendor: Upgrade mitchellh/cli
Update tests to match the fix in mitchellh/cli#71, which aligns MockUi
with BasicUi and allows newlines in user input.

We are not using the new ErrorWriter, added in mitchellh/cli#81, as it
does not appear to interact correctly with panicwrap. All error output
from CLI parsing will continue to appear on stdout, not stderr.
2020-09-10 15:23:02 -04:00
Pam Selle e4e16ccbd3 Rebase fix 2020-09-10 11:06:40 -04:00
Pam Selle 61c78fd3b9 Add case to compactValueStr not to expose sensitive vals in diagnostics 2020-09-10 11:04:17 -04:00
Pam Selle bc55b6a28b Use UnmarkDeepWithPaths and MarkWithPaths
Updates existing code to use the new Value
methods for unmarking/marking and removes
panics/workarounds in cty marshall methods
2020-09-10 11:04:17 -04:00
Pam Selle 896d277a69 If the path is empty, we should not be marking the path 2020-09-10 11:04:17 -04:00
Pam Selle 84d118e18f Track sensitivity through evaluation
Mark sensitivity on a value. However, when the value is encoded to send to the
provider to produce a changeset we must remove the marks, so unmark the value
and remark it with the saved path afterwards
2020-09-10 11:04:17 -04:00
Alisdair McDiarmid 09d8355f43 command: Add experimental concise diff renderer
When rendering a diff between current state and projected state, we only
show resources and outputs which have changes. However, we show a full
structural diff for these values, which includes all attributes and
blocks for a changed resource or output. The result can be a very long
diff, which makes it difficult to verify what the changed fields are.

This commit adds an experimental concise diff renderer, which suppresses
most unchanged fields, only displaying the most relevant changes and
some identifying context. This means:

- Always show all identifying attributes, initially defined as `id`,
  `name`, and `tags`, even if unchanged;
- Only show changed, added, or removed primitive values: `string`,
  `number`, or `bool`;
- Only show added or removed elements in unordered collections and
  structural types: `map`, `set`, and `object`;
- Show added or removed elements with any surrounding unchanged elements
  for sequence types: `list` and `tuple`;
- Only show added or removed nested blocks, or blocks with changed
  attributes.

If any attributes, collection elements, or blocks are hidden, a count
is kept and displayed at the end of the parent scope. This ensures that
it is clear that the diff is only displaying a subset of the resource.

The experiment is currently enabled by default, but can be disabled by
setting the TF_X_CONCISE_DIFF environment variable to 0.
2020-09-10 10:35:55 -04:00
Alisdair McDiarmid 1c8a7f65d0
Merge pull request #26137 from hashicorp/alisdair/output-changes-tests
command: Add tests for format.OutputChanges
2020-09-08 09:45:05 -04:00
Martin Atkins efe78b2910 main: new global option -chdir
This new option is intended to address the previous inconsistencies where
some older subcommands supported partially changing the target directory
(where Terraform would use the new directory inconsistently) where newer
commands did not support that override at all.

Instead, now Terraform will accept a -chdir command at the start of the
command line (before the subcommand) and will interpret it as a request
to direct all actions that would normally be taken in the current working
directory into the target directory instead. This is similar to options
offered by some other similar tools, such as the -C option in "make".

The new option is only accepted at the start of the command line (before
the subcommand) as a way to reflect that it is a global command (not
specific to a particular subcommand) and that it takes effect _before_
executing the subcommand. This also means it'll be forced to appear before
any other command-specific arguments that take file paths, which hopefully
communicates that those other arguments are interpreted relative to the
overridden path.

As a measure of pragmatism for existing uses, the path.cwd object in
the Terraform language will continue to return the _original_ working
directory (ignoring -chdir), in case that is important in some exceptional
workflows. The path.root object gives the root module directory, which
will always match the overriden working directory unless the user
simultaneously uses one of the legacy directory override arguments, which
is not a pattern we intend to support in the long run.

As a first step down the deprecation path, this commit adjusts the
documentation to de-emphasize the inconsistent old command line arguments,
including specific guidance on what to use instead for the main three
workflow commands, but all of those options remain supported in the same
way as they were before. In a later commit we'll make those arguments
produce a visible deprecation warning in Terraform's output, and then
in an even later commit we'll remove them entirely so that -chdir is the
single supported way to run Terraform from a directory other than the
one containing the root module configuration.
2020-09-04 15:31:08 -07:00
Alisdair McDiarmid ba9baac36d command: Add tests for format.OutputChanges
Most of the functionality for rendering output changes is covered by the
tests for ResourceChanges, as they both share the same diff renderer.
This commit adds a few tests to cover some of the output specific code.
2020-09-04 16:22:23 -04:00
Graham Hargreaves e9394dfb38
command/clistate: Return an error on unlock failure (#25729)
* Return an error on unlock failure

When the lock can't be released return the err even if there is no previous error with the current action. This allows faster failure in CI/CD systems. Without this failure to remove the lock would result in the failure happening on a subsequent plan or apply which slows down the feedback loop in automated systems.

* Update command/clistate/state.go

Accept review suggestion

Co-authored-by: ZymoticB <ZymoticB@users.noreply.github.com>

* add test

Co-authored-by: ZymoticB <ZymoticB@users.noreply.github.com>
Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-09-03 16:05:16 -04:00
Alisdair McDiarmid 9f824c53a5 command: Better in-house provider install errors
When init attempts to install a legacy provider required by state and
fails, but another provider with the same type is successfully
installed, this almost definitely means that the user is migrating an
in-house provider. The solution here is to use the `terraform state
replace-provider` subcommand.

This commit makes that next step clearer, by detecting this specific
case, and displaying a list of commands to fix the existing state
provider references.
2020-09-01 14:02:19 -04:00
Alisdair McDiarmid 3547f9e368 format: Don't wrap space-prefixed diag details
Diagnostic detail lines sometimes contain lines which include commands
suggested for the user to execute. By convention, these start with
leading whitespace to indicate that they are not prose.

This commit changes the diagnostic formatter to wrap each line of the
detail separately, and skips word wrapping for lines prefixed with
space. This prevents ugly and confusing wrapping of long command lines.
2020-09-01 14:02:19 -04:00
Alisdair McDiarmid 6d228cc560
Merge pull request #26036 from hashicorp/alisdair/output-empty-should-be-warning
command: Warn instead of error for empty output
2020-09-01 10:23:51 -04:00
Alisdair McDiarmid 89e8d0868a
Merge pull request #26061 from hashicorp/alisdair/013upgrade-registry-provider-redirect
command: Add redirect support to 0.13upgrade
2020-09-01 08:59:35 -04:00
Kristin Laemmert 196c183dda
terraform: remove state from `validate` graph walk (#26063)
This pull reverts a recent change to backend/local which created two context, one with and one without state. Instead I have removed the state entirely from the validate graph (by explicitly passing a states.NewState() to the validate graph builder).

This changed caused a test failure, which (ty so much for the help) @jbardin discovered was inaccurate all along: the test's call to `Validate()` was actually what was removing the output from state. The new expected test output matches terraform's actual behavior on the command line: if you use -target to destroy a resource, an output that references only that resource is *not* removed from state even though that test would lead you to believe it did.

This includes two tests to cover the expected behavior:

TestPlan_varsUnset has been updated so it will panic if it gets more than one request to input a variable
TestPlan_providerArgumentUnset covers #26035

Fixes #26035, #26027
2020-08-31 15:45:39 -04:00
Alisdair McDiarmid fc7e467d19 command: Add redirect support to 0.13upgrade
If a provider changes namespace in the registry, we can detect this when
running the 0.13upgrade command. As long as there is a version matching
the user's constraints, we now use the provider's new source address.
Otherwise, warn the user that the provider has moved and a version
upgrade is necessary to move to it.
2020-08-31 14:53:35 -04:00
Pam Selle f2d213c461
Merge pull request #25657 from pdecat/typo_hierarchical
Typo: heirarchical => hierarchical
2020-08-28 12:37:49 -04:00
Alisdair McDiarmid 2323e9370a command: Warn instead of error for empty output
When the output subcommand is called with no arguments, and there are no
outputs to show, we previously rendered an error message but returned a
non-error status code. This is confusing.

This commit changes the text UI to use a warning diagnostic, which makes
it clearer that this is a non-error situation. We do not change the exit
code or the text of the warning, so hopefully this is not considered a
breaking change.
2020-08-28 11:05:46 -04:00
Alisdair McDiarmid 45437721c9 command: Add test for backend config validation 2020-08-26 11:37:11 -04:00
Alisdair McDiarmid f028b0a2bf command: Fix backend config schema validation
When applying a backend config override file, we must not check for the
presence of all required fields, as the override can be a partial
configuration. It is only valid to check for required fields after all
overrides have been merged, which init already does.
2020-08-26 10:50:47 -04:00
Alisdair McDiarmid c5d9935c0e
Merge pull request #25960 from hashicorp/alisdair/backend-config-override-fix
command: Fix backend config override validation
2020-08-24 10:14:16 -04:00
Alisdair McDiarmid 35125717b2
Merge pull request #25847 from zachwhaley/fix-tf-cli-args-no-color
Fix error when multiple -no-color arguments are used
2020-08-24 10:13:59 -04:00
Alisdair McDiarmid 19efd7bc3b
Merge pull request #25262 from hashicorp/alisdair/always-validate-workspace-name
command: Always validate workspace name
2020-08-24 10:11:13 -04:00
Alisdair McDiarmid 677aabc767 command: Fix backend config override validation
When loading a backend config override file, init was doing two things
wrong:

- First, if the file failed to parse, we accidentally didn't return,
  which caused a panic due to the parsed body being nil;
- Secondly, we were overzealous with the validation of the file,
  allowing only attributes. While most backend configs are attributes
  only, the enhanced remote backend body also contains a `workspaces`
  block, which we need to support here.

This commit fixes the first bug with an early return and adds test cases
for missing file and intentionally-blank filename (to clear the config).

We also add a schema validation for the backend block, based on the
backend schema itself. This requires constructing an HCL body schema so
that we can call `Content` and check for diagnostic errors.

The result is more useful errors when an invalid backend config override
file is used, while also supporting the enhanced remote backend config
fully.

Does not include tests specific to the remote backend, because the
mocking involved to allow the backend to fully initialize is too
involved to be worth it.
2020-08-21 16:21:13 -04:00
Cameron Stitt 54e32652f7
Ensure depends_on is in module calls for config 2020-08-20 07:49:03 +10:00
Alisdair McDiarmid 30c7dfca62
Merge pull request #25898 from hashicorp/alisdair/fix-required-version-diags
terraform: Fix required version constraint diags
2020-08-19 11:26:03 -04:00
Alisdair McDiarmid 3114e2ad7c
Merge pull request #25890 from hashicorp/import-our-nemesis
terraform: Eval module call arguments for import
2020-08-19 11:25:38 -04:00
kmoe 6486b857d6
Merge pull request #25811 from hashicorp/terraform-version-order
terraform version: fix output order
2020-08-19 13:54:18 +01:00
Alisdair McDiarmid c98f352dc8 terraform: Fix required version constraint diags
If a module has multiple terraform.required_version constraints, any
failures would point at the last constraint in the error diagnostics. If
an earlier constraint was the actual problem, this leads to confusing
errors like this:

    Error: Unsupported Terraform Core version

      on main.tf line 6, in terraform:
       6:   required_version = ">= 0.13.0"

    This configuration does not support Terraform version 0.13.0.

The error was due to storing the declaration range of the constraint as
a pointer to the contents of a loop variable, which was later
overwritten in later iterations of the loop.  Instead we now use HCL's
handy Ptr() method to create a direct pointer to the range struct.
2020-08-18 09:35:32 -04:00
Alisdair McDiarmid d8e9964363 terraform: Eval module call arguments for import
Include the import walk in the list of operations for which we create an
EvalModuleCallArgument node. This causes module call arguments to be
evaluated even if the module variables have defaults, ensuring that
invalid default values (such as the common "{}" for variables thought of
as maps) do not cause failures specific to import.

This fixes a bug where a child module evaluates an input variable in its
locals block, assuming that it is a nested object structure. The bug
report includes a default value of "{}", which is overridden by a root
variable value. Without the eval node added in this commit, the default
value is used and the local evaluation errors.
2020-08-17 17:14:12 -04:00
Zachary Whaley af8d5a69d8 Fix error when TF_CLI_ARGS adds -no-color argument
Fixes #25845
2020-08-13 13:41:34 -05:00
Katy Moe e59553ef02
test outdated version output 2020-08-12 21:22:39 +01:00