Commit Graph

1854 Commits

Author SHA1 Message Date
James Bardin ced676d50d fixed configloader doesn't add unnecessary diags 2021-05-07 12:28:09 -04:00
Alisdair McDiarmid 91cdde1d67
Merge pull request #28608 from hashicorp/alisdair/json-plan-replace-paths
jsonplan: Add replace_paths
2021-05-06 08:23:09 -04:00
Martin Atkins 7c6e78bcb0 plans: Track both the previous run and prior states in the plan
Until now we've not really cared much about the state snapshot produced
by the previous Terraform operation, except to use it as a jumping-off
point for our refresh step.

However, we'd like to be able to report to an end-user whenever Terraform
detects a change that occurred outside of Terraform, because that's often
helpful context for understanding why a plan contains changes that don't
seem to have corresponding changes in the configuration.

As part of reporting that we'll need to keep track of the state as it
was before we did any refreshing work, so we can then compare that against
the state after refreshing. To retain enough data to achieve that, the
existing Plan field State is now two fields: PrevRunState and PriorState.

This also includes a very shallow change in the core package to make it
populate something somewhat-reasonable into this field so that integration
tests can function reasonably. However, this shallow implementation isn't
really sufficient for real-world use of PrevRunState because we'll
actually need to update PrevRunState as part of planning in order to
incorporate the results of any provider-specific state upgrades to make
the PrevRunState objects compatible with the current provider schema, or
else our diffs won't be valid. This deeper awareness of PrevRunState in
Terraform Core will follow in a subsequent commit, prior to anything else
making use of Plan.PrevRunState.
2021-05-05 15:11:05 -07:00
Kristin Laemmert 5f30efe857
command tests: plan and init (#28616)
* command/init: add test for reconfigure

* command/plan: adding tests

* command/apply: tests

* command: show and refresh tests
2021-05-05 14:13:20 -04:00
Alisdair McDiarmid a5b7394f9a command/jsonplan: Add replace_paths
The set of paths which caused a resource update to require replacement
has been stored in the plan since 0.15.0 (#28201). This commit adds a
simple JSON representation of these paths, allowing consumers of this
format to determine exactly which paths caused the resource to be
replaced.

This representation is intentionally more loosely encoded than the JSON
state serialization of paths used for sensitive attributes. Instead of a
path step being represented by an object with type and value, we use a
more-JavaScripty heterogenous array of numbers and strings. Any
practical consumer of this format will likely traverse an object tree
using the index operator, which should work more easily with this
format. It also allows easy prefix comparison for consumers which are
tracking paths.

While updating the documentation to include this new field, I noticed
that some others were missing, so added them too.
2021-05-04 16:51:51 -04:00
Alisdair McDiarmid 53fab10b26
Merge pull request #28598 from hashicorp/alisdair/fix-diagnostic-snippet-crash
cli: Fix diagnostic snippet crash
2021-05-04 13:01:58 -04:00
Kristin Laemmert 3679de0630
command/format: fix repetitive "unchanged attribute hidden" message (#28589)
writeNestedAttrDiff and writeAttrDiff were both printing the "unchanged attribute" message.  This removes one of the redundant prints.

Fixing this led me (in a very roundabout way) to realize that NestedType attributes were printing a sum total of unchanged attributes, including those in entirely unchanged elements, while *not* printing the total of unchanged elements. I added the necessary logic to count and print the number of unchanged elements for maps and lists.
2021-05-04 10:23:50 -04:00
Alisdair McDiarmid 25f99857cf cli: Fix crash with invalid JSON diagnostics
If a JSON diagnostic value has a highlight end offset which is before
the highlight start offset, this would previously panic. This commit
adds a normalization step to prevent the crash.
2021-05-04 08:35:42 -04:00
Alisdair McDiarmid 3a9b369b43 views/json: Fix diag crash with invalid highlight
Some diagnostic sources (I'm looking at you, HCL) fail to set the end of
the subject range. This is a bug in those code paths, but we can ensure
that we generate valid JSON diagnostics by checking for it here.

By doing so before the range normalization, we ensure that we generate a
unit width highlight whenever possible, so that at least something
useful is displayed.
2021-05-04 08:35:42 -04:00
Alisdair McDiarmid e9c9092c40 Fix missing trailing newline 2021-05-04 08:31:39 -04:00
Martin Atkins 1d3e34e35e command: New -replace=... planning option
This allows a similar effect to pre-tainting an object but does the action
within the context of a normal plan and apply, avoiding the need for an
intermediate state where the old object still exists but is marked as
tainted.

The core functionality for this was already present, so this commit is
just the UI-level changes to make that option available for use and to
explain how it contributed to the resulting plan in Terraform's output.
2021-05-03 15:43:23 -07:00
Alisdair McDiarmid 7f39f19ec7
Merge pull request #28584 from hashicorp/alisdair/add-reason-to-json-planned-change
command/views: Add reason to JSON planned change
2021-05-03 09:37:27 -04:00
Alisdair McDiarmid 7b23fa7877 command/views: Add reason to JSON planned change
Now that we have extra information about the reason for a given resource
action, include that in the JSON log output for planned changes.
2021-05-03 06:49:42 -04:00
Alisdair McDiarmid c95e9ada6b cli: Show forces replacement for sensitive attrs
When rendering a plan diff, sensitive resource attributes would
previously omit the "forces replacement" comment, which can lead to
confusion when the only reason for a resource being replaced is a
sensitive attribute.
2021-05-03 06:48:10 -04:00
Alisdair McDiarmid f72730a02b cli: Add JSON logs for operations commands 2021-04-30 11:37:36 -04:00
Martin Atkins b802237e03 plans: Track an optional extra "reason" for some planned actions
Previously we were repeating some logic in the UI layer in order to
recover relevant additional context about a change to report to a user.
In order to help keep things consistent, and to have a clearer path for
adding more such things in the future, here we capture this user-facing
idea of an "action reason" within the plan model, and then use that
directly in order to decide how to describe the change to the user.

For the moment the "tainted" situation is the only one that gets a special
message, matching what we had before, but we can expand on this in future
in order to give better feedback about the other replace situations too.

This also preemptively includes the "replacing by request" reason, which
is currently not reachable but will be used in the near future as part of
implementing the -replace=... plan command line option to allow forcing
a particular object to be replaced.

So far we don't have any special reasons for anything other than replacing,
which makes sense because replacing is the only one that is in a sense
a special case of another action (Update), but this could expand to
other kinds of reasons in the future, such as explaining which of the
few different reasons a data source read might be deferred until the
apply step.
2021-04-29 17:50:46 -07:00
Alisdair McDiarmid 4b1ff4eda7
Merge pull request #28523 from hashicorp/alisdair/json-plan-sensitive-provider-attrs
core: Add sensitive provider attrs to JSON plan
2021-04-28 15:49:47 -04:00
Martin Atkins 89f986ded6 command+backend: generalized "plan mode"
So far we've only had "normal mode" and "destroy mode", where the latter
is activated either by "terraform plan -destroy" or "terraform destroy".

In preparation for introducing a third mode "refresh only" this
generalizes how we handle modes so we can potentially deal with an
arbitrary number of modes, although for now we only intend to have three.

Mostly this is just a different implementation of the same old behavior,
but there is one small user-visible difference here: the "terraform apply"
command now accepts a -destroy option, mirroring the option of the same
name on "terraform plan", which in turn makes "terraform destroy"
effectively a shorthand for "terraform apply -destroy".

This is intended to make us consistent that "terraform apply" without a
plan file argument accepts all of the same plan-customization options that
"terraform plan" does, which will in turn avoid us having to add a new
alias of "terraform plan" for each new plan mode we might add. The -help
output is changed in that vein here, although we'll wait for subsequent
commit to make a similar change to the website documentation just so we
can deal with the "refresh only mode" docs at the same time.
2021-04-27 08:23:54 -07:00
Martin Atkins c6a7d080d9 core: Generalize the idea of a "plan mode", vs just destroy flag
Previously there were only two planning modes: normal mode and destroy
mode. In that context it made sense for these to be distinguished only by
a boolean flag.

We're now getting ready to add our third mode, "refresh only". This
establishes the idea that planning can be done in one of a number of
mutually-exclusive "modes", which are related to but separate from the
various other options that serve as modifiers for the plan operation.

This commit only introduces the new plans.Mode type and replaces the
existing "destroy" flag with a variable of that type. This doesn't cause
any change in effective behavior because Terraform Core still supports
only NormalMode and DestroyMode, with NewContext rejecting an attempt to
create a RefreshMode context for now.

It is in retrospect a little odd that the "destroy" flag was part of
ContextOpts rather than just an argument to the Plan method, but
refactoring that would be too invasive a change for right now so we'll
leave this as a field of the context for now and save revisiting that for
another day.
2021-04-27 08:23:54 -07:00
Alisdair McDiarmid c89004d223 core: Add sensitive provider attrs to JSON plan
When rendering a stored plan file as JSON, we include a data structure
representing the sensitivity of the changed resource values. Prior to
this commit, this was a direct representation of the sensitivity marks
applied to values via mechanisms such as sensitive variables, sensitive
outputs, and the `sensitive` function.

This commit extends this to include sensitivity based on the provider
schema. This is in line with the UI rendering of the plan, which
considers these two different types of sensitivity to be equivalent.

Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2021-04-27 10:29:34 -04:00
Kristin Laemmert f6af7b4f7a
lang/funcs: add (console-only) TypeFunction (#28501)
* lang/funcs: add (console-only) TypeFunction

The type() function, which is only available for terraform console,
prints out the type of a given value. This is mainly intended for
debugging - it's nice to be able to print out terraform's understanding
of a complex variable.

This introduces a new field for Scope: ConsoleMode. When ConsoleMode is true, any additional functions intended for use in the console (only) may be added.
2021-04-23 10:29:50 -04:00
Chris Arcand 15b6a1614c
Merge pull request #28479 from hashicorp/motd-for-tfc
New login success output for TFC/E
2021-04-22 09:13:56 -05:00
Chris Arcand 58ec68063b Split off error MOTD error logging
The error logging in outputDefaultTFCLoginSuccess() is an unrelated side
effect.
2021-04-22 09:05:57 -05:00
Chris Arcand 4b97411890 Add purple to the list of colorstring codes
This is the first color code introduced for 256 color terminals, and
should be a safe addition when considering the oldest supported
platform.
2021-04-21 21:35:45 -05:00
Chris Arcand 58cba1dea3 Add customized login success output for TFC/E
When logging in to Terraform Cloud or Terraform Enterprise, change the
success output to be a bit more customized for the platform. For
Terraform Cloud, fetch a dynamic welcome banner that intentionally fails
open and defaults to a hardcoded message if its not available for any
reason.
2021-04-21 21:23:42 -05:00
Martin Atkins 8f233cde4c cli: Diagnostics can include collections with sensitive elements
We previously had a shallow IsMarked call in compactValueStr's caller but
then a more-conservative deep ContainsMarked call inside compactValueStr
with a different resulting message. As well as causing an inconsistency
in messages, this was also a bit confusing because it made it seem like
a non-sensitive collection containing a sensitive element value was wholly
sensitive, making the debug information in the diagnostic messages not
trustworthy for debugging certain varieties of problem.

I originally considered just removing the redundant check in
compactValueStr here, but ultimately I decided to keep it as a sort of
defense in depth in case a future refactoring disconnects these two
checks. This should also serve as a prompt to someone making later changes
to compactValueStr to think about the implications of sensitive values
in there, which otherwise wouldn't be mentioned at all.

Disclosing information about a collection containing sensitive values is
safe here because compactValueStr only discloses information about the
value's type and element keys, and neither of those can be sensitive in
isolation. (Constructing a map with sensitive keys reduces to a sensitive
overall map.)
2021-04-19 11:58:49 -07:00
Alisdair McDiarmid 8dcf768f4e backend/remote: Add IsLocalOperations
To ensure that the apply command can determine whether an operation is
executed locally or remotely, we add an IsLocalOperations method on the
remote backend. This returns the internal forceLocal boolean.

We also update this flag after checking if the corresponding remote
workspace is in local operations mode or not. This ensures that we know
if an operation is running locally (entirely on the practitioner's
machine), pseudo-locally (on a Terraform Cloud worker), or remotely
(executing on a worker, rendering locally).
2021-04-16 11:43:57 -04:00
Alisdair McDiarmid 69e7922a33 cli: Fix missing apply summary for remote runs
Disabling the resource count and outputs rendering when the remote
backend is in use causes them to be omitted from Terraform Cloud runs.
This commit changes the condition to render these values if either the
remote backend is not in use, or the command is running in automation
via the TF_IN_AUTOMATION flag. As this is intended to be set by
Terraform Cloud and other remote backend implementations, this addresses
the problem.
2021-04-16 10:03:22 -04:00
Alisdair McDiarmid fad305f884 cli: Fix remote backend UI issues
Fix two bugs which surface when using the remote backend:

- When migrating to views, we removed the call to `(*Meta).process`
  which initialized the color boolean. This resulted in the legacy UI
  calls in the remote backend stripping color codes. To fix this, we
  populate this boolean from the common arguments.
- Remote apply will output the resource summary and output changes, and
  these are rendered via the remote backend streaming. We need to
  special case this in the apply command and prevent displaying a
  zero-change summary line.

Neither of these are coverable by automated tests, as we don't have any
command-package level testing for the remote backend. Manually verified.
2021-04-16 08:28:33 -04:00
James Bardin d04999863c "with" formatting 2021-04-06 15:50:30 -04:00
James Bardin 265b5106ca call the InConfigBody with addresses 2021-04-06 15:15:52 -04:00
James Bardin 406ac97965 add the address field to the view diagnostics 2021-04-06 15:15:52 -04:00
Alisdair McDiarmid ff32fab41a command/jsonplan: Fix sensitive/unknown crash
When rendering the JSON plan sensitivity output, if the plan contained
unknown collection or structural types, Terraform would crash. We need
to detect unknown values before attempting to iterate them.

Unknown collection or structural values cannot have sensitive contents
accidentally displayed, as those values are not known until after apply.
As a result we return an empty value of the appropriate type for the
sensitivity mapping.
2021-03-31 14:29:15 -04:00
Alisdair McDiarmid 786d99207e cli: Only rewrite provider locks file if changed
If the provider locks have not changed, there is no need to rewrite the
locks file. Preventing this needless rewrite should allow Terraform to
operate in a read-only directory, so long as the provider requirements
don't change.
2021-03-29 16:09:07 -04:00
Alisdair McDiarmid 340543d122
Merge pull request #28201 from hashicorp/alisdair/planfile-sensitive-required-replace
Add sensitivity and required-replace fields to plan file, and expose sensitivity to JSON plan
2021-03-29 15:08:36 -04:00
Alisdair McDiarmid 5e30d58dc2 command/jsonplan: Add output change sensitivity
When an output value changes, we have a small amount of information we
can convey about its sensitivity. If either the output was previously
marked sensitive, or is currently marked sensitive in the config, this
is tracked in the output change data.

This commit encodes this boolean in the change struct's
`before_sensitive` and `after_sensitive` fields, in the a way which
matches resource value sensitivity. Since we have so little information
to work with, these two values will always be booleans, and always equal
each.

This is logically consistent with how else we want to obscure sensitive
data: a changing output which was or is marked sensitive should not have
the value shown in human-readable output.
2021-03-26 19:26:11 -04:00
Alisdair McDiarmid 63613ca1b0 command/jsonconfig: Add variable sensitive flag 2021-03-26 19:26:11 -04:00
Alisdair McDiarmid e27aacebf9 command/jsonplan: Add sensitive value mapping data
Similar to `after_unknown`, `before_sensitive` and `after_sensitive` are
values with similar structure to `before` and `after` which encode the
presence of sensitive values in a planned change. These should be used
to obscure sensitive values from human-readable output.

These values follow the same structure as the `before` and `after`
values, replacing sensitive values with `true`, and non-sensitive values
with `false`. Following the `after_unknown` precedent, we omit
non-sensitive `false` values for object attributes/map values, to make
serialization more compact.

One difference from `after_unknown` is that a sensitive complex value
(collection or structural type) is replaced with `true`. If the complex
value itself is sensitive, all of its contents should be obscured.
2021-03-26 19:26:10 -04:00
Alisdair McDiarmid fd7c4105f2
Merge pull request #28202 from hashicorp/alisdair/fmt-multiline-value-expr-unwrap
cli: Fix fmt output for multi-line value exprs
2021-03-26 11:33:14 -04:00
Martin Atkins 6f35c2847b command: Reorganize docs of the local backend's legacy CLI options
We have these funny extra options that date back to before Terraform even
had remote state, which we've preserved along the way by most recently
incorporating them as special-case overrides for the local backend.

The documentation we had for these has grown less accurate over time as
the details have shifted, and was in many cases missing the requisite
caveats that they are only for the local backend and that backend
configuration is the modern, preferred way to deal with the use-cases they
were intended for.

We always have a bit of a tension with this sort of legacy option because
we want to keep them documented just enough to be useful to someone who
finds an existing script/etc using them and wants to know what they do,
but not to take up so much space that they might distract users from
finding the modern alternative they should consider instead.

As a compromise in that vein here I've created a new section about these
options under the local backend documentation, which then gives us the
space to go into some detail about the various behaviors and interactions
and also to discuss their history and our recommended alternatives. I then
simplified all of the other mentions of these in command documentation
to just link to or refer to the local backend documentation. My hope then
is that folks who need to know what these do can still find the docs, but
that information can be kept out of the direct path of new users so they
can focus on learning about remote backends instead.

This is certainly not the most ideal thing ever, but it seemed like the
best compromise between the competing priorities I described above.
2021-03-25 13:56:48 -07:00
Alisdair McDiarmid d71f0c6149 cli: Fix fmt output for multi-line value exprs
The formatter for value expressions which use legacy interpolation
syntax was previously behaving incorrectly with some multi-line
expressions. Any HCL expression which requires parenthesis to be allowed
to span multiple lines could be skip those parens if already inside
string interpolation (`"${}"`).

When removing string interpolation, we now check for a resulting
multi-line expression, and conservatively ensure that it starts and ends
with parenthesis. These may be redundant, as not all expressions require
parens to permit spanning multiple lines, but at least it will be valid
output.
2021-03-25 15:40:54 -04:00
Alisdair McDiarmid 841cced6c9
Merge pull request #28138 from hashicorp/alisdair/command-views-validate
cli: Migrate validate command to views
2021-03-23 13:13:06 -04:00
Alisdair McDiarmid 7242d9af2b Fix typo in plan no changes output message 2021-03-22 16:39:53 -04:00
Alisdair McDiarmid 46c2dce0fb Fix broken test 2021-03-22 16:38:56 -04:00
Kristin Laemmert b9138f4465
terraform: validate providers' schemas during NewContext (#28124)
* checkpoint save: update InternalValidate tests to compare exact error

* configschema: extract and extend attribute validation

This commit adds an attribute-specific InternalValidate which was extracted directly from the block.InternalValidate logic and extended to verify any NestedTypes inside an Attribute. Only one error message changed, since it is now valid to have a cty.NilType for Attribute.Type as long as NestedType is set.

* terraform: validate provider schema's during NewContext

We haven't been able to guarantee that providers are validating their own schemas using (some version of) InternalValidate since providers were split out of the main codebase. This PR adds a call to InternalValidate when provider schemas are initially loaded by NewContext, which required a few other changes:

InternalValidate's handling of errors vs multierrors was a little weird - before this PR, it was occasionally returning a non-nil error which only stated "0 errors occurred" - so I addressed that in InternalValidate. I then tested this with a configuration that was using all of our most popular providers, and found that at least on provider had some invalid attribute names, so I commented that particular validation out. Adding that in would be a breaking change which we would have to coordinate with enablement and providers and (especially in this case) make sure it's well communicated to external provider developers.

I ran a few very unscientific tests comparing the timing with and without this validation, and it appeared to only cause a sub-second increase.

* refactor validate error message to closer match the sdk's message

* better error message

* tweak error message: move the instruction to run init to the end of the message, after the specific error.
2021-03-22 13:17:50 -04:00
Kristin Laemmert 77562d9b57
command/jsonprovider: bump format version (#28115)
Support for attributes with NestedTypes was added in https://github.com/hashicorp/terraform/pull/28055, and should have included a format version bump: this is a backwards-compatible change, but consumers will need to be updated in order to properly decode attributes (with NestedTypes) going forward.
2021-03-22 11:45:36 -04:00
Alisdair McDiarmid dd380d0b58 cli: Migrate validate command to views 2021-03-18 15:56:28 -04:00
Alisdair McDiarmid 46a29b13ed cli: Add format version to validate -json output
In line with the other complex JSON output formats for plan and provider
schema, here we add an explicit `format_version` field to the JSON
output of terraform validate.
2021-03-16 09:46:36 -04:00
Kristin Laemmert fac60ab124
Add formatting for attributes with nested types (#28069)
* format/diff: extract attributes-writing logic to a function

This is a stepping-stone commit (for easier reviewability, and to prove that tests did not change) as part of writing a NestedType-specific diff printer.

* command/format: add support for formatting attributes with NestedTypes

This commit adds custom formatting for NestedType attributes. THe logic was mostly copied from the block diff printer, with minor tweaks here and there. I used the (excellent) existing test coverage and added a NestedType attribute to every test.

Since the (nested-block specific) test schemas were nearly identical, I added a function that returns the schema with the requested NestingMode.
2021-03-15 09:31:23 -04:00
Alisdair McDiarmid c92826c14d cli: Use standard JSON diagnostics for validate
Now that we have a comprehensive JSON diagnostic structure, we can use
it in the `validate -json` output instead of the inline version. Note
that this changes the output of `validate -json` in two ways:

1. We fix some off-by-one errors caused by zero-width highlight ranges.
   This aligns the JSON diagnostic output with the text output seen by
   most Terraform users, so I consider this a bug fix.
2. We add the `snippet` field to the JSON diagnostics where available.
   This is purely additive and is permitted under our JSON format
   stability guarantees.
2021-03-12 13:12:37 -05:00