Commit Graph

23479 Commits

Author SHA1 Message Date
Radek Simko f3d1565d6f
command/format: Fix tuple diff formatting 2019-01-23 11:17:55 +00:00
Martin Atkins 995042666a config/hcl2shim: ValuesSDKEquivalent float64 comparison of numbers
The SDK uses only the native int and float64 types internally for values
that are specified as being "number" in schema, so for SDK purposes only
a float64 level of precision is significant.

To avoid any weirdness introduced as we shim and un-shim numbers, we'll
reduce floating point numbers to float64 precision before comparing them
to try to mimic the result the SDK itself would've gotten from comparing
its own float64 versions of these values using the Go "==" operator.
2019-01-22 18:45:21 -08:00
James Bardin 46a4628782
Merge pull request #20081 from hashicorp/jbardin/list-block
New Diff.Apply method
2019-01-22 19:20:53 -05:00
Martin Atkins f65b7c5372 helper/plugin: Discard meaningless differences from provider planning
Due to various inprecisions in the old SDK implementation, applying the
generated diff can potentially make changes to the data structure that
have no real effect, such as replacing an empty list with a null list or
vice-versa.

Although we can't totally eliminate such diff noise, here we attempt to
avoid it in situations where there are _only_ meaningless changes -- where
the prior state and planned state are equivalent -- by just echoing back
the prior state verbatim to ensure that Terraform will treat it as a noop
change.

If there _are_ some legitimate changes then the result may still contain
meaningless changes alongside it, but that is just a cosmetic problem for
the diff renderer, because the meaningless changes will be ignored
altogether during a subsequent apply anyway. The primary goal here is just
to ensure we can converge on a fixpoint when there are no explicit changes
in the configuration.
2019-01-22 15:41:10 -08:00
Martin Atkins 6b03c81375 config/hcl2shim: ValuesSDKEquivalent function
This is a first pass of an "approximately equal" function that tries to
mimic the reduced precision caused by the field reader abstraction in
helper/schema so that we can distinguish between meaningful changes to
the proposed new state and incidental ones that just result from the loss
of precision in the SDK implementation.
2019-01-22 15:41:10 -08:00
James Bardin 273f20ec8b update comment and fix core test
One terraform test was broken when the result became more correct.
2019-01-22 18:38:17 -05:00
James Bardin 93d78c4ee7 disable broken import test for now 2019-01-22 18:10:12 -05:00
James Bardin 7257258f18 new Diff.Apply
The previous version assumed the diff could be applied verbatim, and
only used the schema at the top level since diffs are "flat". This
turned out to not work reliably with nested blocks. The new Apply method
is driven completely by the schema, and handles nested blocks separately
from other collections.
2019-01-22 18:10:12 -05:00
James Bardin c37147d876 fix computed set keys in shims
When generated a config, the computed set keys were missing the leading
set name.
2019-01-22 18:10:12 -05:00
James Bardin 8d302c5bd2 update grpc_provider for new diffs
Keep the diff as-is before applying.
2019-01-22 18:10:12 -05:00
James Bardin f78b5045d0 add failing test for lost elements in list blocks
Modifying an element loses the modification, and other elements in a
TypeList.
2019-01-22 18:10:12 -05:00
Radek Simko 0dff8fe5e0
Add failing test case for tuple 2019-01-22 16:49:49 +00:00
Radek Simko 3d0a25c65d
command/format: Fix nested (JSON) object formatting 2019-01-22 16:26:28 +00:00
Radek Simko f341624df7
Update CHANGELOG.md 2019-01-22 16:10:30 +00:00
Radek Simko 7e0be7d8b9
Merge pull request #20067 from hashicorp/b-cmd-fmt-unknown-set
command/format: Fix rendering of unknown elements in set/map/list
2019-01-22 16:09:44 +00:00
Martin Atkins 10bf4c763b backend: Undeclared variables in -var-file is a warning, not an error
In Terraform 0.11 and earlier we just silently ignored undeclared
variables in -var-file and the automatically-loaded .tfvars files. This
was a bad user experience for anyone who made a typo in a variable name
and got no feedback about it, so we made this an error for 0.12.

However, several users are now relying on the silent-ignore behavior for
automation scenarios where they pass the same .tfvars file to all
configurations in their organization and expect Terraform to ignore any
settings that are not relevant to a specific configuration. We never
intentionally supported that, but we don't want to immediately break that
workflow during 0.12 upgrade.

As a compromise, then, we'll make this a warning for v0.12.0 that contains
a deprecation notice suggesting to move to using environment variables
for this "cross-configuration variables" use-case. We don't produce errors
for undeclared variables in environment variables, even though that
potentially causes the same UX annoyance as ignoring them in vars files,
because environment variables are assumed to live in the user's session
and this it would be very inconvenient to have to unset such variables
when moving between directories. Their "ambientness" makes them a better
fit for these automatically-assigned general variable values that may or
may not be used by a particular configuration.

This can revert to being an error in a future major release, after users
have had the opportunity to migrate their automation solutions over to
use environment variables.

We don't seem to have any tests covering this specific situation right
now. That isn't ideal, but this change is so straightforward that it would
be relatively expensive to build new targeted test cases for it and so
I instead just hand-tested that it is indeed now producing a warning where
we were previously producing an error. Hopefully if there is any more
substantial work done on this codepath in future that will be our prompt
to add some unit tests for this.
2019-01-22 07:46:51 -08:00
Radek Simko c5ba7469be
command/format: Fix rendering of unknown elements in set/map/list 2019-01-21 15:05:59 +00:00
Radek Simko 98cc99e632
command/format: Add crashing test with unknown element in map 2019-01-21 15:04:13 +00:00
Radek Simko 73225c7aeb
command/format: Add crashing test with unknown element in list 2019-01-21 15:04:13 +00:00
Radek Simko e7e8b7358f
command/format: Add crashing test with unknown element in set 2019-01-21 15:04:13 +00:00
Martin Atkins 15cd6d8300 core: Retain prior state if update fails with no new state
In an ideal world, providers are supposed to respond to errors during
apply by returning a partial new state alongside the error diagnostics.
In practice though, our SDK leaves the new value set to nil for certain
errors, which was causing Terraform to "forget" the object altogether by
assuming that the provider intended to say "null".

We now adjust that assumption to apply only in the delete case. In all
other cases (including updates) we retain the prior state if the new
state is given as nil. Although we could potentially fix this in the SDK
itself, I expect this is a likely bug in other future SDKs for other
languages too, so this new assumption is a safer one to make to be
resilient to data loss when providers don't behave perfectly.

Providers that return both nil new value and no errors are considered
buggy, but unfortunately that applies to the mocks in many of our tests,
so for pragmatic reasons we can't generate an error for that case as we do
for other "should never happen" situations. Instead, we'll just retain the
prior value in the state so the user can retry.
2019-01-18 16:54:52 -08:00
Sander van Harmelen db36ccb316
Merge pull request #20054 from hashicorp/svh/b-typo
backend/remote: fix a small typo
2019-01-18 19:56:32 +01:00
Sander van Harmelen 8937fedb76 backend/remote: fix a small typo 2019-01-18 19:40:57 +01:00
Martin Atkins 8b094f48f7 command: Update "terraform get" to use the new module installer
We missed this on the initial update pass because this was calling
directly into the module package API rather than going through the Meta
methods that we updated for the new config loader.

m.installModules here is the same method that "terraform init" is using
for this purpose, ensuring the two will behave the same way. This changes
the output a little compared to the old installer, but it still includes
the important information about where each module is coming from.
2019-01-17 16:52:34 -08:00
James Bardin 565eeac5d1
Merge pull request #20041 from hashicorp/jbardin/shims
Some fixes for the provider shims
2019-01-17 19:34:54 -05:00
James Bardin c045d3e6a3 disable known failing tests
We need these changes in master for testing, worry about these test
after.
2019-01-17 19:19:13 -05:00
James Bardin 286cb0a39d clean out diff a little more before checking
Check if there wasn't any real diff attributes first, before returning
the original state in PlanResourceChange.
2019-01-17 19:19:13 -05:00
James Bardin 4f691c5988 don't replace null strings with empty strings
This adds unexpected values in some cases, and since the case this
handles is only within set objects, we'll deal woth this when tackling
the sets themselves.
2019-01-17 19:19:13 -05:00
James Bardin 2cc651124e don't overwrite values in plan
Plan can change known values too, which we can't match in sets. We'll
find another way to normalize these eithout losing plan values.
2019-01-17 18:51:18 -05:00
James Bardin 4439a7dcf4 add tests for nested default values
Don't lose default values set within a nested block.
2019-01-17 18:51:18 -05:00
Martin Atkins 176a5abfd3 command: Restore single-file support in "terraform fmt"
This possibility was lost in the rewrite to use HCL2, but it's used by
a number of external utilities and text editor integrations, so we'll
restore it here.

Using the stdin/stdout mode is generally preferable for text editor use
since it allows formatting of the in-memory buffer rather than directly
the file on disk, but for editors that don't have support for that sort of
tooling it can be convenient to just launch a single command and directly
modify the on-disk file.
2019-01-17 14:21:18 -08:00
Martin Atkins b0a43cab84 command: "terraform fmt" must fail if file has invalid syntax
Since the HCL formatter only works with tokens, it can in principle be
called with any input and produce some output. However, when given invalid
syntax it will tend to produce nonsensical results that may drastically
change the input file and be hard for the user to undo.

Since there's no strong reason to try to format an invalid or incomplete
file, we'll instead try parsing first and fail if parsing does not
complete successfully.

Since we talk directly to the HCL API here this is only a _syntax_ check,
and so it can be applied to files that are invalid in other ways as far
as Terraform is concerned, such as using unsupported top-level block types,
resource types that don't exist, etc.
2019-01-17 14:21:18 -08:00
Martin Atkins a10f9a18cf vendor: go get github.com/zclconf/go-cty@master
There are no code changes in this update. This just catches up with the
latest master commit in the upstream repo, which has only seen
documentation updates since the prior commit.
2019-01-17 14:21:18 -08:00
Martin Atkins 787c176063 vendor: go get github.com/hashicorp/hcl2@master
This includes a missing feature for the splat syntax (null.* returns an
empty tuple) and also fixes a bug in the source code formatter where it
was inserting spaces between two consecutive interpolation sequences, like
"${foo}${bar}".
2019-01-17 14:21:18 -08:00
Kim Ngo b88ef2b47c
Update CHANGELOG.md 2019-01-17 12:09:13 -06:00
Martin Atkins 6ab15a80a4
Update CHANGELOG.md 2019-01-17 10:03:30 -08:00
Martin Atkins 2f8f7d6f4d lang/funcs: Type conversion functions
It's not normally necessary to make explicit type conversions in Terraform
because the language implicitly converts as necessary, but explicit
conversions are useful in a few specialized cases:

- When defining output values for a reusable module, it may be desirable
  to force a "cleaner" output type than would naturally arise from a
  computation, such as forcing a string containing digits into a number.
- Our 0.12upgrade mechanism will use some of these to replace use of the
  undocumented, hidden type conversion functions in HIL, and force
  particular type interpretations in some tricky cases.
- We've found that type conversion functions can be useful as _temporary_
  workarounds for bugs in Terraform and in providers where implicit type
  conversion isn't working correctly or a type constraint isn't specified
  precisely enough for the automatic conversion behavior.

These all follow the same convention of being named "to" followed by a
short type name. Since we've had a long-standing convention of running all
the words together in lowercase in function names, we stick to that here
even though some of these names are quite strange, because these should
be rarely-used functions anyway.
2019-01-17 10:01:47 -08:00
Kim Ngo ba6e243bd9
Merge pull request #20030 from findkim/bump-plugin-install-version
Bump provider installer protocol version to 5
2019-01-17 11:37:48 -06:00
Radek Simko 5d4946e27a
Merge pull request #20004 from hashicorp/issue-template-labels
Add labels to issue templates
2019-01-17 15:22:49 +00:00
Radek Simko b661a983b8
Add labels to issues templates 2019-01-17 15:04:16 +00:00
findkim 073d367018 tools/terraform-bundle: match plugin installer protocol with terraform's installer 2019-01-16 16:15:52 -06:00
findkim 7816e61614 Bump installer protocol version to 5 and separate client and server protocol references 2019-01-16 15:07:57 -06:00
Radek Simko 2ad9f0513a
Update CHANGELOG.md 2019-01-16 19:31:28 +00:00
Radek Simko aaf4319953
Merge pull request #20005 from hashicorp/b-statemgr-avoid-html-escaping
states/statemgr: Avoid HTML escaping when printing LockInfo
2019-01-16 19:30:50 +00:00
Martin Atkins 4e14ab7557 config: Remove "setproduct" function, which is now in lang/funcs 2019-01-16 09:57:16 -08:00
Martin Atkins da51e72cbb lang/functions: set functions from cty
The sethaselement, setintersection, and setunion functions are defined in
the cty stdlib. Making them available in Terraform will make it easier to
work with sets, and complement the currently-Terraform-specific setproduct
function.

In the long run setproduct should probably move into the cty stdlib too,
but since it was submitted as a Terraform function originally we'll leave
it here now for simplicity's sake and reorganize later.
2019-01-16 09:57:16 -08:00
Martin Atkins edb5f82de1 lang/funcs: Convert the "setproduct" function to the new approach
In our new world it produces either a set of a tuple type or a list of a
tuple type, depending on the given argument types.

The resulting collection's element tuple type is decided by the element
types of the given collections, allowing type information to propagate
even if unknown values are present.
2019-01-16 09:57:16 -08:00
Martin Atkins 9d11d427a8 config: Rename "product" function to "setproduct"
This hopefully makes it clearer that it works with sets rather than real
numbers, avoiding confusion with "sum", "min", "max", etc.
2019-01-16 09:57:16 -08:00
Fábio Matavelli acd17d9075 Create product interpolation function
This creates the product interpolation function that returns the cartesian product of a list of lists.
2019-01-16 09:57:16 -08:00
Kim Ngo 2bca828e46
Merge pull request #20015 from findkim/rm-dup-discovery-logging
Remove duplicate discovery logging
2019-01-15 16:12:54 -06:00