Commit Graph

23495 Commits

Author SHA1 Message Date
James Bardin 1592b4aa67
Merge pull request #20094 from hashicorp/jbardin/missing-map-values
handle empty map values
2019-01-23 20:28:34 -05:00
James Bardin 775df57217 add more tests
verify that changes to defaults are detected
2019-01-23 20:03:10 -05:00
James Bardin 7dd0acc46b don't count empty containers in diff.Apply
If there were no matching keys, and there was no diff at all, don't set
a zero count for the container. Normally Providers can't reliably detect
empty vs unset here, but there are some cases that worked.
2019-01-23 19:34:11 -05:00
Kristin Laemmert e9099b4fcc
command/jsonplan: fix panic when filteredAfter is null (#20096)
* command/jsonplan: fix panic when filteredAfter is null
* command/jsonconfig: provider short name is required to properly look up resource schema
2019-01-23 16:14:34 -08:00
Martin Atkins c70954aeab tools/terraform-bundle: refuse to bundle versions <0.12.0
Since terraform-bundle is just a different frontend to Terraform's module
installer, it is subject to the same installation constraints as Terraform
itself.

Terraform 0.12 cannot install providers targeting Terraform 0.11 and
earlier, and so therefore terraform-bundle built with Terraform 0.12
cannot either. A build of terraform-bundle from the v0.11 line must be
used instead.

Without this change, the latest revisions of terraform-bundle would
install plugins for Terraform 0.12 to bundle along with Terraform 0.10 or
0.11, which will not work at runtime due to the plugin protocol mismatch.

Until now, terraform-bundle was incorrectly labelled with its own version
number even though in practice it has no version identity separate from
Terraform itself. Part of this change, then, is to make the
terraform-bundle version match the Terraform version it was built against,
though any prior builds will of course continue to refer to themselves
as 0.0.1.

If asked to create a bundle for a version of Terraform v0.12 or greater,
an error will be returned instructing the user to use a build from the
v0.11 branch or one of the v0.11.x tags in order to bundle those versions.

This also includes a small fix for a bug where the tool would not fail
properly when the requested Terraform version is not available for
installation, instead just producing a zip file with no "terraform"
executable inside at all. Now it will fail, allowing automated build
processes to detect it and not produce a broken archive for distribution.
2019-01-23 14:43:52 -08:00
James Bardin 37b5e2dc87 don't remove empty diff values
Our new diff handling no longer requires stripping the empty diffs out,
and provider may be relying on some of the empty-value quirks in
helper/schema.
2019-01-23 17:33:23 -05:00
James Bardin 675d700a5f test for missing map entries 2019-01-23 17:04:17 -05:00
James Bardin 9b30da500d missing prefix in recounted map
Missing prefix in map recount. This generally passes tests since the
actual count should already be there and be correct, then ethe extra key
is ignored by the shims.
2019-01-23 14:57:04 -05:00
Kristin Laemmert f00fcb90bf
mildwonkey/b-show-state (#20032)
* command/show: properly marshal attribute values to json

marshalAttributeValues in jsonstate and jsonplan packages was returning
a cty.Value, which json/encoding could not marshal. These functions now
convert those cty.Values into json.RawMessages.

* command/jsonplan: planned values should include resources that are not changing
* command/jsonplan: return a filtered list of proposed 'after' attributes

Previously, proposed 'after' attributes were not being shown if the
attributes were not WhollyKnown. jsonplan now iterates through all the
`after` attributes, omitting those which are not wholly known.

The same was roughly true for after_unknown, and that structure is now
correctly populated. In the future we may choose to filter the
after_unknown structure to _only_ display unknown attributes, instead of
all attributes.

* command/jsonconfig: use a unique key for providers so that aliased
providers don't get munged together

This now uses the same "provider" key from configs.Module, e.g.
`providername.provideralias`.

* command/jsonplan: unknownAsBool needs to iterate through objects that are not wholly known

* command/jsonplan: properly display actions as strings according to the RFC,
instead of a plans.Action string.

For example:
a plans.Action string DeleteThenCreate should be displayed as ["delete",
"create"]

Tests have been updated to reflect this.

* command/jsonplan: return "null" for unknown list items.

The length of a list could be meaningful on its own, so we will turn
unknowns into "null". The same is less likely true for maps and objects,
so we will continue to omit unknown values from those.
2019-01-23 11:46:53 -08:00
Radek Simko b492c3662c
Merge pull request #20089 from hashicorp/t-cmd-fmt-sensitive-update
command/format: Add test to cover update of sensitive field
2019-01-23 15:55:53 +00:00
Radek Simko ffb887f8b7
Update CHANGELOG.md 2019-01-23 15:37:53 +00:00
Radek Simko 71d07832e2
Merge pull request #20071 from hashicorp/b-fix-json-diff-formatting
command/format: Fix nested (JSON) object formatting
2019-01-23 15:37:03 +00:00
Radek Simko f04d0b48bc
command/format: Add test to cover update of sensitive field 2019-01-23 15:32:13 +00:00
Sander van Harmelen 1e4c20686e backend/remote: make sure we show the correct error
Previously we would show two errors when there was a version constraint
error. But of course one is enough.
2019-01-23 15:09:42 +01:00
Tom Harvey ec0419fe91
backend/azurerm: fixing the syntax highlighting (#20085) 2019-01-23 13:33:11 +00:00
Radek Simko 953eae7e4b
command/format: Fix rendering of different types 2019-01-23 13:13:48 +00:00
Radek Simko f3d1565d6f
command/format: Fix tuple diff formatting 2019-01-23 11:17:55 +00:00
Martin Atkins 995042666a config/hcl2shim: ValuesSDKEquivalent float64 comparison of numbers
The SDK uses only the native int and float64 types internally for values
that are specified as being "number" in schema, so for SDK purposes only
a float64 level of precision is significant.

To avoid any weirdness introduced as we shim and un-shim numbers, we'll
reduce floating point numbers to float64 precision before comparing them
to try to mimic the result the SDK itself would've gotten from comparing
its own float64 versions of these values using the Go "==" operator.
2019-01-22 18:45:21 -08:00
James Bardin 46a4628782
Merge pull request #20081 from hashicorp/jbardin/list-block
New Diff.Apply method
2019-01-22 19:20:53 -05:00
Martin Atkins f65b7c5372 helper/plugin: Discard meaningless differences from provider planning
Due to various inprecisions in the old SDK implementation, applying the
generated diff can potentially make changes to the data structure that
have no real effect, such as replacing an empty list with a null list or
vice-versa.

Although we can't totally eliminate such diff noise, here we attempt to
avoid it in situations where there are _only_ meaningless changes -- where
the prior state and planned state are equivalent -- by just echoing back
the prior state verbatim to ensure that Terraform will treat it as a noop
change.

If there _are_ some legitimate changes then the result may still contain
meaningless changes alongside it, but that is just a cosmetic problem for
the diff renderer, because the meaningless changes will be ignored
altogether during a subsequent apply anyway. The primary goal here is just
to ensure we can converge on a fixpoint when there are no explicit changes
in the configuration.
2019-01-22 15:41:10 -08:00
Martin Atkins 6b03c81375 config/hcl2shim: ValuesSDKEquivalent function
This is a first pass of an "approximately equal" function that tries to
mimic the reduced precision caused by the field reader abstraction in
helper/schema so that we can distinguish between meaningful changes to
the proposed new state and incidental ones that just result from the loss
of precision in the SDK implementation.
2019-01-22 15:41:10 -08:00
James Bardin 273f20ec8b update comment and fix core test
One terraform test was broken when the result became more correct.
2019-01-22 18:38:17 -05:00
James Bardin 93d78c4ee7 disable broken import test for now 2019-01-22 18:10:12 -05:00
James Bardin 7257258f18 new Diff.Apply
The previous version assumed the diff could be applied verbatim, and
only used the schema at the top level since diffs are "flat". This
turned out to not work reliably with nested blocks. The new Apply method
is driven completely by the schema, and handles nested blocks separately
from other collections.
2019-01-22 18:10:12 -05:00
James Bardin c37147d876 fix computed set keys in shims
When generated a config, the computed set keys were missing the leading
set name.
2019-01-22 18:10:12 -05:00
James Bardin 8d302c5bd2 update grpc_provider for new diffs
Keep the diff as-is before applying.
2019-01-22 18:10:12 -05:00
James Bardin f78b5045d0 add failing test for lost elements in list blocks
Modifying an element loses the modification, and other elements in a
TypeList.
2019-01-22 18:10:12 -05:00
Radek Simko 0dff8fe5e0
Add failing test case for tuple 2019-01-22 16:49:49 +00:00
Radek Simko 3d0a25c65d
command/format: Fix nested (JSON) object formatting 2019-01-22 16:26:28 +00:00
Radek Simko f341624df7
Update CHANGELOG.md 2019-01-22 16:10:30 +00:00
Radek Simko 7e0be7d8b9
Merge pull request #20067 from hashicorp/b-cmd-fmt-unknown-set
command/format: Fix rendering of unknown elements in set/map/list
2019-01-22 16:09:44 +00:00
Martin Atkins 10bf4c763b backend: Undeclared variables in -var-file is a warning, not an error
In Terraform 0.11 and earlier we just silently ignored undeclared
variables in -var-file and the automatically-loaded .tfvars files. This
was a bad user experience for anyone who made a typo in a variable name
and got no feedback about it, so we made this an error for 0.12.

However, several users are now relying on the silent-ignore behavior for
automation scenarios where they pass the same .tfvars file to all
configurations in their organization and expect Terraform to ignore any
settings that are not relevant to a specific configuration. We never
intentionally supported that, but we don't want to immediately break that
workflow during 0.12 upgrade.

As a compromise, then, we'll make this a warning for v0.12.0 that contains
a deprecation notice suggesting to move to using environment variables
for this "cross-configuration variables" use-case. We don't produce errors
for undeclared variables in environment variables, even though that
potentially causes the same UX annoyance as ignoring them in vars files,
because environment variables are assumed to live in the user's session
and this it would be very inconvenient to have to unset such variables
when moving between directories. Their "ambientness" makes them a better
fit for these automatically-assigned general variable values that may or
may not be used by a particular configuration.

This can revert to being an error in a future major release, after users
have had the opportunity to migrate their automation solutions over to
use environment variables.

We don't seem to have any tests covering this specific situation right
now. That isn't ideal, but this change is so straightforward that it would
be relatively expensive to build new targeted test cases for it and so
I instead just hand-tested that it is indeed now producing a warning where
we were previously producing an error. Hopefully if there is any more
substantial work done on this codepath in future that will be our prompt
to add some unit tests for this.
2019-01-22 07:46:51 -08:00
Radek Simko c5ba7469be
command/format: Fix rendering of unknown elements in set/map/list 2019-01-21 15:05:59 +00:00
Radek Simko 98cc99e632
command/format: Add crashing test with unknown element in map 2019-01-21 15:04:13 +00:00
Radek Simko 73225c7aeb
command/format: Add crashing test with unknown element in list 2019-01-21 15:04:13 +00:00
Radek Simko e7e8b7358f
command/format: Add crashing test with unknown element in set 2019-01-21 15:04:13 +00:00
Martin Atkins 15cd6d8300 core: Retain prior state if update fails with no new state
In an ideal world, providers are supposed to respond to errors during
apply by returning a partial new state alongside the error diagnostics.
In practice though, our SDK leaves the new value set to nil for certain
errors, which was causing Terraform to "forget" the object altogether by
assuming that the provider intended to say "null".

We now adjust that assumption to apply only in the delete case. In all
other cases (including updates) we retain the prior state if the new
state is given as nil. Although we could potentially fix this in the SDK
itself, I expect this is a likely bug in other future SDKs for other
languages too, so this new assumption is a safer one to make to be
resilient to data loss when providers don't behave perfectly.

Providers that return both nil new value and no errors are considered
buggy, but unfortunately that applies to the mocks in many of our tests,
so for pragmatic reasons we can't generate an error for that case as we do
for other "should never happen" situations. Instead, we'll just retain the
prior value in the state so the user can retry.
2019-01-18 16:54:52 -08:00
Sander van Harmelen db36ccb316
Merge pull request #20054 from hashicorp/svh/b-typo
backend/remote: fix a small typo
2019-01-18 19:56:32 +01:00
Sander van Harmelen 8937fedb76 backend/remote: fix a small typo 2019-01-18 19:40:57 +01:00
Martin Atkins 8b094f48f7 command: Update "terraform get" to use the new module installer
We missed this on the initial update pass because this was calling
directly into the module package API rather than going through the Meta
methods that we updated for the new config loader.

m.installModules here is the same method that "terraform init" is using
for this purpose, ensuring the two will behave the same way. This changes
the output a little compared to the old installer, but it still includes
the important information about where each module is coming from.
2019-01-17 16:52:34 -08:00
James Bardin 565eeac5d1
Merge pull request #20041 from hashicorp/jbardin/shims
Some fixes for the provider shims
2019-01-17 19:34:54 -05:00
James Bardin c045d3e6a3 disable known failing tests
We need these changes in master for testing, worry about these test
after.
2019-01-17 19:19:13 -05:00
James Bardin 286cb0a39d clean out diff a little more before checking
Check if there wasn't any real diff attributes first, before returning
the original state in PlanResourceChange.
2019-01-17 19:19:13 -05:00
James Bardin 4f691c5988 don't replace null strings with empty strings
This adds unexpected values in some cases, and since the case this
handles is only within set objects, we'll deal woth this when tackling
the sets themselves.
2019-01-17 19:19:13 -05:00
James Bardin 2cc651124e don't overwrite values in plan
Plan can change known values too, which we can't match in sets. We'll
find another way to normalize these eithout losing plan values.
2019-01-17 18:51:18 -05:00
James Bardin 4439a7dcf4 add tests for nested default values
Don't lose default values set within a nested block.
2019-01-17 18:51:18 -05:00
Martin Atkins 176a5abfd3 command: Restore single-file support in "terraform fmt"
This possibility was lost in the rewrite to use HCL2, but it's used by
a number of external utilities and text editor integrations, so we'll
restore it here.

Using the stdin/stdout mode is generally preferable for text editor use
since it allows formatting of the in-memory buffer rather than directly
the file on disk, but for editors that don't have support for that sort of
tooling it can be convenient to just launch a single command and directly
modify the on-disk file.
2019-01-17 14:21:18 -08:00
Martin Atkins b0a43cab84 command: "terraform fmt" must fail if file has invalid syntax
Since the HCL formatter only works with tokens, it can in principle be
called with any input and produce some output. However, when given invalid
syntax it will tend to produce nonsensical results that may drastically
change the input file and be hard for the user to undo.

Since there's no strong reason to try to format an invalid or incomplete
file, we'll instead try parsing first and fail if parsing does not
complete successfully.

Since we talk directly to the HCL API here this is only a _syntax_ check,
and so it can be applied to files that are invalid in other ways as far
as Terraform is concerned, such as using unsupported top-level block types,
resource types that don't exist, etc.
2019-01-17 14:21:18 -08:00
Martin Atkins a10f9a18cf vendor: go get github.com/zclconf/go-cty@master
There are no code changes in this update. This just catches up with the
latest master commit in the upstream repo, which has only seen
documentation updates since the prior commit.
2019-01-17 14:21:18 -08:00
Martin Atkins 787c176063 vendor: go get github.com/hashicorp/hcl2@master
This includes a missing feature for the splat syntax (null.* returns an
empty tuple) and also fixes a bug in the source code formatter where it
was inserting spaces between two consecutive interpolation sequences, like
"${foo}${bar}".
2019-01-17 14:21:18 -08:00