Commit Graph

1255 Commits

Author SHA1 Message Date
James Bardin e71e3d85a9
Merge pull request #21611 from hashicorp/jbardin/private-data-read
Make sure resource private data is carried through the entire resource lifecycle
2019-06-05 19:36:55 -04:00
James Bardin ac2219ba6e don't lose Private state data during copy
Fix the scope of the private data copy in DeepCopy.

Make sure Dependencies matches nil vs empty so that Equal compares
correctly between copied states
2019-06-05 19:22:46 -04:00
Ivan Kalita 5b6b1663ef backend/http: implement retries for the http backend (#19702)
Fixes #19619
2019-06-05 16:12:07 -04:00
Kristin Laemmert b9f114aa25
command/show (json): marshal the state snapshot included with the plan file (#21597)
* command/show: marshal the state snapshot from the planfile

The planfile contains a state snapshot with certain resources updated
(outputs and datasources). Previously `terraform show -json PLANFILE`
was using the current state instead of the state inside the plan as
intended.

This caused an issue when the state included a terraform_remote_state
datasource. The datasource's state gets refreshed - and therefore
upgraded to the current state version - during plan, but that won't
persist to state until apply.

* update comment to reflect new return
2019-06-05 07:29:02 -04:00
Kristin Laemmert 06a3333316 note: this is an unusual edgecase in the category of "probably should not happen"
If https://github.com/hashicorp/terraform/issues/21543 is
fixed, we can remove this check.
2019-06-03 14:00:46 -04:00
Kristin Laemmert 703f024cbd command/show (-json): fix panic if a moduleCall has a nil config
In the unlikely event that a moduleCall has a nil config - for example,
if a nested module call includes a variable with a typo in an
attribute - continue gracefully.
2019-06-03 11:19:03 -04:00
Kristin Laemmert 65fc037ecb
command/show (plan -json): fix panic (#21541)
* command/show -json: fix panic

afterUnknown should return only bools, not values.

* command/jsonplan: let's delete some redundant code!

the plan output was somewhat inconsistent with return values for
"after_unknown". This strives to fix that. If all "after" values are
known, return an empty object instead of iterating over values.

Also fixing some typos and general copypasta.
2019-06-03 07:14:23 -05:00
James Bardin 7f5ffbfe90
Merge pull request #21439 from hashicorp/jbardin/init-backend
init always prompts for migration with most remote backend configurations
2019-05-29 16:01:40 -05:00
James Bardin 06dfc4abd8 allow setting -backend-config='' to unset override
There is currently no way to unset -backend-config during init, since
not setting that option assumes the user will use the saved config.
Allow setting `-backend-config=""` to specify no overrides.
2019-05-29 12:58:04 -05:00
Martin Atkins d512584497 command/jsonplan: Don't panic with mixtures of known/unknown/empty
The omitUnknowns and unknownAsBool functions were previously trying hard
to preserve the same collection types in the output as they had in the
input, by attempting to keep everything matched up so that the results
would be valid.

Unfortunately, this turns out to be a harder problem than we originally
thought: it was possible for a collection value going in to produce
inconsistent element types out (and thus a panic) in the following
situations:
- when a collection with mixed known and unknown values was passed in
  to omitUnknowns.
- when a collection of collections where the inner collections are a
  mixture of empty and not empty in unknownAsNull.

The results of these functions are only used to marshal to JSON anyway,
and JSON serialization can't distinguish between the three sequence types
or the two mapping types, so in practice we can just standardize on
converting all sequences to tuple and all mappings to object here and not
change the resulting output at all, and then we don't have to worry about
making sure all of the inner types get preserved exactly.

A nice consequence of that relaxation is that we can now do what we
originally wanted to do with unknownAsBool, and omit map keys and
object attributes altogether if their values would've been false,
producing a much more compact result. This is easiest to do now when
there's only one known user of this JSON plan output, and we know that
user will treat both false and omitted as the same here.
2019-05-28 19:33:21 -07:00
James Bardin c017149b31 don't store prepared backend config
The backend gets to "prepare" the configuration before Configure is
called, in order to validate the values and insert defaults. We don't
want to store this value in the "config state", because it will often
not match the raw config after it is prepared, forcing unecessary
backend migrations during init.

Since PrepareConfig is always called before Configure, we can store the
config value directly, and assume that it will be prepared in the same
manner each time.
2019-05-24 14:51:18 -04:00
James Bardin ee9a618369 don't migrate backend during init without override
If the backend config hashes match during init, and there are no new
backend override options, then we assume the existing config is OK.
Since init should be idempotent, we should be able to run init with no
options or config changes, and not effect the backends at all.
2019-05-24 11:31:04 -04:00
James Bardin cefc927e48 failing test for backend re-init 2019-05-23 18:21:52 -04:00
Sander van Harmelen 8fd3aa91a0
Merge pull request #21234 from hashicorp/svh/f-init-v0.12
Always try to select a workspace after initialization (v0.12)
2019-05-23 14:34:31 +02:00
Martin Atkins 63fa1ac418 vendor: go get github.com/hashicorp/hcl2@master
This includes a small fix to ensure the parser doesn't produce an invalid
body for block parsing syntax errors, and instead produces an incomplete
result that calling applications like Terraform can still analyze.

The problem here was affecting our version-constraint-sniffing code, which
intentionally tried to find a core version constraint even if there's a
syntax error so that it can report that a new version of Terraform is a
likely cause of the syntax error. It was working in most cases, unless
it was the "terraform" block itself that contained the error, because then
we'd try to analyze a broken hcl.Block with a nil body.

This includes a new test for "terraform init" that exercises this
recovery codepath.
2019-05-14 15:37:46 -07:00
Sander van Harmelen d9ba0c69ea Always try to select a workspace after initialization
There are a number of use cases that can require a user to select a workspace after initializing Terraform.

To make sure we cover all these use cases, we will always call the selectWorkspace method to verify a valid workspace is already selected or (if needed) offer to select one before moving on.
2019-05-07 21:46:45 +02:00
Martin Atkins 6adcc7ab73 vendor: go get github.com/zclconf/go-cty@master
cty now guarantees that sets of primitive values will iterate in a
reasonable order. Previously it was the caller's responsibility to deal
with that, but we invariably neglected to do so, causing inconsistent
ordering. Since cty prioritizes consistent behavior over performance, it
now imposes its own sort on set elements as part of iterating over them so
that calling applications don't have to worry so much about it.

This change also causes cty to consistently push unknown and null values
in sets to the end of iteration, where before that was undefined. This
means that our diff output will now consistently list additions before
removals when showing sets, rather than the ordering being undefined as
before.

The ordering of known, non-null, non-primitive values is still not
contractually fixed but remains consistent for a particular version of
cty.
2019-04-30 15:49:28 -07:00
Martin Atkins eed605ac05 [WIP] Re-enable the end-to-end tests (#20044)
* internal/initwd: Allow deprecated relative module paths

In Terraform 0.11 we deprecated this form but didn't have any explicit
warning for it. Now we'll still accept it but generate a warning. In a
future major release we will drop this form altogether, since it is
ambiguous with registry module source addresses.

This codepath is covered by the command/e2etest suite.

* e2e: Skip copying .exists file, if present

We use this only in the "empty" test fixture in order to let git know that
the directory exists. We need to skip copying it so that we can test
"terraform init -from-module=...", which expects to find an empty
directory.

* command/e2etests: Re-enable and fix up the e2etest "acctests"

We disabled all of the tests that accessed remote services like the
Terraform Registry while they were being updated to support the new
protocols we now expect. With those services now in place, we can
re-enable these tests.

Some details of exactly what output we print, etc, have intentionally
changed since these tests were last updated.

* e2e: refactor for modern states and plans

* command/e2etest: re-enable e2etests and update for tf 0.12 compatibility
plugin/discovery: mkdirAll instead of mkdir when creating cache dir
2019-04-29 13:03:24 -04:00
Sander van Harmelen e41c2ecdce
Merge pull request #21139 from hashicorp/svh/b-stdin
Make sure UIInput keeps working after being canceled
2019-04-29 17:58:29 +02:00
Sander van Harmelen 9ab2e9d8b2 Make sure UIInput keeps working after being canceled
Once you start reading from stdin, that is a blocking call that will
never finish. So when a context is canceled causing the input method to
return, the read will remain blocking in the running goroutine.

There isn't a real solution for it (e.g. its not possible to unblock the
read) so the only solution is to make the reader reusable.
2019-04-29 15:15:26 +02:00
James Bardin f79a768a4e command/format: take noop changes from lcs
When rendering the diff, the NoOp changes should come from the LCS
sequence, rather than the new sequence. The two indexes will not align
in many cases, adding the wrong new object or indexing out of bounds.
2019-04-27 11:28:02 -04:00
Przemysław Dąbek 9605b093d9 command/graph: use user-supplied plugin path when running graph command (#18083) 2019-04-17 13:48:11 -04:00
Kristin Laemmert c1079b59bd
command/state_list.go: fix bug loading user-defined state (#21015)
* command/state_list.go: fix bug loading user-defined state

If the user supplied a state path via the `-state` flag and terraform
was running in a workspace other than `default`, the state was not being
loaded properly. Fixes #19920
2019-04-15 12:22:07 -04:00
Kristin Laemmert 1c95b21c6c
command/output: get the state path from the workspace or CLI argument (#20994)
Previously this command was setting the state path to the default state
file, instead of honoring the backend configuration.
2019-04-12 07:37:27 -04:00
Martin Atkins 88e76fa9ef configs/configschema: Introduce the NestingGroup mode for blocks
In study of existing providers we've found a pattern we werent previously
accounting for of using a nested block type to represent a group of
arguments that relate to a particular feature that is always enabled but
where it improves configuration readability to group all of its settings
together in a nested block.

The existing NestingSingle was not a good fit for this because it is
designed under the assumption that the presence or absence of the block
has some significance in enabling or disabling the relevant feature, and
so for these always-active cases we'd generate a misleading plan where
the settings for the feature appear totally absent, rather than showing
the default values that will be selected.

NestingGroup is, therefore, a slight variation of NestingSingle where
presence vs. absence of the block is not distinguishable (it's never null)
and instead its contents are treated as unset when the block is absent.
This then in turn causes any default values associated with the nested
arguments to be honored and displayed in the plan whenever the block is
not explicitly configured.

The current SDK cannot activate this mode, but that's okay because its
"legacy type system" opt-out flag allows it to force a block to be
processed in this way anyway. We're adding this now so that we can
introduce the feature in a future SDK without causing a breaking change
to the protocol, since the set of possible block nesting modes is not
extensible.
2019-04-10 14:53:52 -07:00
Pam Selle 346e341ff2 Only display status link if public registry used
Updates to throw a specific error if using the defined public
registry vs. another registry.
2019-04-05 16:49:27 -04:00
Kristin Laemmert f8a5e17d3d
command/format: improve "source" of error messages regarding missing arguments (#20907)
* vendor: update hcl2 dependency
* command/format: revert diagnostic format behavior if snippet or highlight range is empty
2019-04-03 14:04:59 -04:00
Frederic 0f1f504c22 command/format: indicate in diff when adding an attribute forces replacement 2019-03-29 14:52:13 -07:00
Pam Selle ff7245f27c Add status link to make message more helpful 2019-03-27 16:22:37 -04:00
Pam Selle d72456d188 Add friendly error for when registry unresponsive
If the registry is unresponsive, you will now get an error
specific to this, rather than a misleading "provider unavailable" type
error. Also adds debug logging for when errors like this may occur
2019-03-27 14:39:14 -04:00
Kristin Laemmert d5bb09b13b
command/show: refresh is required to show remote state (#20818) 2019-03-25 16:28:35 -04:00
Justin Downing 1e32ae243c grammatical updates to comments and docs (#20195) 2019-03-21 14:05:41 -07:00
findkim 161fe47b34 plugin/discovery: improve providery discovery verification errors 2019-03-21 14:38:51 -05:00
Martin Atkins e918fa83ec command/format: Don't panic when item removed from list of objects
Due to these tests happening in the wrong order, removing an object from
the end of a sequence of objects would previously cause a bounds-check
panic.

Rather than a more severe rework of the logic here, for now we'll just
introduce an extra precondition to prevent the panic. The code that
follows already handles the case where there _is_ no new object (i.e. the
"old" object is being deleted) as long as we're able to pass through this
type-checking logic.

The new "JSON list of objects - removing item" test covers this problem
by rendering a diff for an object being removed from the end of a list
of objects within a JSON value.
2019-03-19 15:46:40 -07:00
Justin Campbell e696e063dc
Merge pull request #20674 from hashicorp/registry-warnings
plugin/discovery: Parse warnings from TF Registry
2019-03-18 13:12:26 -04:00
Justin Campbell e6316c9de6 plugin/discovery: Parse warnings from TF Registry
Terraform Registry (and other registry implementations) can now return
an array of warnings with the versions response. These warnings are now
displayed to the user during a `terraform init`.
2019-03-18 12:21:27 -04:00
Justin Campbell 24e13d8ec1 plugin/discovery: Return tfdiags from Get
Allows us to surface warnings to the user using the tfdiags interfaces.
2019-03-18 12:21:27 -04:00
Martin Atkins c39905e1a8 command: Fix various issues in the "terraform state ..." subcommands
In earlier refactoring we updated these commands to support the new
address and state types, but attempted to partially retain the old-style
"StateFilter" abstraction that originally lived in the Terraform package,
even though that was no longer being used for any other functionality.

Unfortunately the adaptation of the existing filtering to the new types
wasn't exact and so these commands ended up having a few bugs that were
not covered by the existing tests.

Since the old StateFilter behavior was the source of various misbehavior
anyway, here it's removed altogether and replaced with some simpler
functions in the state_meta.go file that are tailored to the use-cases of
these sub-commands.

As well as just generally behaving more consistently with the other
parts of Terraform that use the new resource address types, this commit
fixes the following bugs:

- A resource address of aws_instance.foo would previously match an
  resource of that type and name in any module, which disagreed with the
  expected interpretation elsewhere of meaning a single resource in the
  root module.

- The "terraform state mv" command was not supporting moves from a single
  resource address to an indexed address and vice-versa, because the old
  logic didn't need to make that distinction while they are two separate
  address types in the new logic. Now we allow resources that do not have
  count/for_each to be treated as if they are instances for the purposes
  of this command, which is a better match for likely user intent and for
  the old behavior.

Finally, we also clean up a little some of the usage output from these
commands, which hasn't been updated for some time and so had both some
stale information and some inaccurate terminology.
2019-03-18 09:19:55 -07:00
Kristin Laemmert c73a5fc07d
command/show: add provider name to resource changes (#20711) 2019-03-15 13:20:43 -07:00
Kristin Laemmert 9d0d564ec7
`terraform show` and `terraform providers schema -json` should return valid json (#20697)
* command/providers schemas: return empty json object if config parses successfully but no providers found
* command/show (state): return an empty object if state is nil
2019-03-14 14:52:07 -07:00
James Bardin 0569e39788 don't try to treat "null" as json in diff output
Trying to decode and write "null" as json will panic, since it decodes
to nil.
2019-03-14 17:20:42 -04:00
Kristin Laemmert a15a4acf2f
configs/configupgrade: detect possible relative module sources (#20646)
* configs/configupgrade: detect possible relative module sources

If a module source appears to be a relative local path but does not have
a preceding ./, print a #TODO message for the user.

* internal/initwd: limit go-getter detectors to those supported by terraform
* internal/initwd: move isMaybeRelativeLocalPath check into getWithGoGetter

To avoid making two calls to getter.Detect, which potentially makes
non-trivial API calls, the "isMaybeRelativeLocalPath" check was moved to
a later step and a custom error type was added so user-friendly
diagnostics could be displayed in the event that a possible relative local
path was detected.
2019-03-13 11:17:14 -07:00
James Bardin a8df5586c0 check for errors when loading a backend config
We can't dereference the config module if there were any errors loading
it.
2019-03-12 17:57:26 -04:00
Martin Atkins 2d41c1009b command/format: Diffs for NestingMap block types
Our initial prototype of new-style diff rendering excluded this because
the old SDK has no support for this construct. However, we want to be able
to introduce this construct in the new SDK without breaking compatibility
with existing versions of Terraform Core, so we need to implement it now
so it's ready to be used once the SDK implements it.

The key associated with each block allows us to properly correlate the
items to recognize the difference between an in-place update of an
existing block and the addition/deletion of a block.
2019-03-11 08:18:26 -07:00
Martin Atkins dd1fa322a7 command/format: Support list/map blocks with dynamic-typed attrs
Our null-to-empty normalization was previously assuming these would always
be collection types, but that isn't true when a block contains something
dynamic since we must then use tuple or object types instead to properly
represent all of the individual element types.
2019-03-11 08:18:26 -07:00
Martin Atkins 69772b11b1 command/format: test for diff rendering with dynamic-typed subattrs
We use cty a little differently when a nested list block contains a
dynamically-typed attribute: it appears as a tuple value instead of a
list value so that we can retain the individual types of each element.

Here we introduce a test for that case, but doing so required also making
the runTestCases function handle types in a stricter way so that it will
produce planned values that match how Terraform Core would do it,
including the necessary late-bound type information for the
dynamically-typed attribute.
2019-03-11 08:18:26 -07:00
Kristin Laemmert 30672faebe
remove indentation from json output of `show` and `providers schema` (#20623)
* command/providers schema: correct help text

* command/show: remove indentation from json output

* command/providers schema: remove indentation from json output
2019-03-11 07:29:36 -07:00
James Bardin b8e53255b4 Revert "remove NormalizeObjectFromLegacySDK from diff"
This reverts commit 5c40d6610c.
2019-03-08 17:32:41 -05:00
Sander van Harmelen 973e2a7cf9 core: add a context to the UIInput interface 2019-03-08 10:24:40 +01:00
James Bardin a95d97f066
Merge pull request #20595 from hashicorp/jbardin/normalize-objects
normalize all objects read from the provider
2019-03-06 17:13:23 -05:00