Commit Graph

28160 Commits

Author SHA1 Message Date
James Bardin f8493bf5cd update hcl
update to v2.10.0
2021-04-20 17:04:56 -04:00
James Bardin d15f7394a1
Merge pull request #28457 from hashicorp/jbardin/provisioner-null-checks
additional null checks in provisioners
2021-04-20 16:25:34 -04:00
James Bardin 7f571b5ebb additional null checks in provisioners
Now that provisioners for directly with the plugin API and cty data
types, we need to add a few null checks to catch invalid input that
would have otherwise been masked by the legacy SDK.
2021-04-20 12:31:32 -04:00
John Houston fabdf0bea1
Add config_paths and drop KUBECONFIG env variable in kubernetes backend (#26997) 2021-04-20 10:05:45 -04:00
Martin Atkins 14336ae6f8 lang/funcs: Conversion functions can handle sensitive values
In order to avoid updating every one of our existing functions with
explicit support for sensitive values, there's a default rule in the
functions system which makes the result of a function sensitive if any
of its arguments contain sensitive values.

We were applying that default to the various type conversion functions,
like tomap and tolist, which meant that converting a complex-typed value
with a sensitive value anywhere inside it would result in a
wholly-sensitive result.

That's unnecessarily conservative because the cty conversion layer (which
these functions are wrapping) already knows how to handle sensitivity
in a more precise way. Therefore we can opt in to handling marked values
(which Terraform uses for sensitivity) here and the only special thing
we need to do is handle errors related to sensitive values differently,
so we won't print their values out literally in case of an error (and so
that the attempt to print them out literally won't panic trying to
extract the marked values).
2021-04-19 12:10:50 -07:00
Martin Atkins 8f233cde4c cli: Diagnostics can include collections with sensitive elements
We previously had a shallow IsMarked call in compactValueStr's caller but
then a more-conservative deep ContainsMarked call inside compactValueStr
with a different resulting message. As well as causing an inconsistency
in messages, this was also a bit confusing because it made it seem like
a non-sensitive collection containing a sensitive element value was wholly
sensitive, making the debug information in the diagnostic messages not
trustworthy for debugging certain varieties of problem.

I originally considered just removing the redundant check in
compactValueStr here, but ultimately I decided to keep it as a sort of
defense in depth in case a future refactoring disconnects these two
checks. This should also serve as a prompt to someone making later changes
to compactValueStr to think about the implications of sensitive values
in there, which otherwise wouldn't be mentioned at all.

Disclosing information about a collection containing sensitive values is
safe here because compactValueStr only discloses information about the
value's type and element keys, and neither of those can be sensitive in
isolation. (Constructing a map with sensitive keys reduces to a sensitive
overall map.)
2021-04-19 11:58:49 -07:00
sanflores 751fba49a7
website: v0.15 upgrade guide had invalid example for tomap(...)
tomap expects an object value with braces, not a tuple value with brackets.
2021-04-19 10:20:03 -07:00
James Bardin a0f221d04b
Merge pull request #28444 from hashicorp/jbardin/destroy-plan-provider
wrong operation during destroy plan walk
2021-04-19 13:15:42 -04:00
James Bardin 877348c031 wrong operation during destroy plan walk
The destroy plan walk was identifying itself as a normal plan, and
causing providers to be configured when they were not needed. Since the
provider configuration may not be complete during the minimal destroy
plan walk, validation or configuration may fail.
2021-04-19 12:35:10 -04:00
Rachel Sharp 56483d10d2
Update debugging.html.md 2021-04-19 12:18:02 -04:00
Kristin Laemmert 85adad0ec7
docs: small update for provider binary locations (#28413)
* docs: add note that provider binaries need to be placed in appropriate subdirectories under the default locations
2021-04-19 09:04:46 -04:00
James Bardin 6839170274
Merge pull request #28414 from hashicorp/jbardin/config-provider-fqns
resolve provider types when building the config
2021-04-16 13:20:57 -04:00
Alisdair McDiarmid 7b2c7dddf3
Merge pull request #28412 from hashicorp/alisdair/fix-missing-apply-output-for-remote-operations
cli: Fix missing apply summary for remote runs
2021-04-16 12:51:07 -04:00
James Bardin d0cc7f1d5e resolve provider types when building the config
All the information is available to resolve provider types when building
the configuration, but some provider references still had no FQN. This
caused validation to assume a default type, and incorrectly reject valid
module calls with non-default namespaced providers.

Resolve as much provider type information as possible when loading the
config. Only use this internally for now, but this should be useful
outside of the package to avoid re-resolving the providers later on. We
can come back and find where this might be useful elsewhere, but for now
keep the change as small as possible to avoid any changes in behavior.
2021-04-16 12:37:50 -04:00
Alisdair McDiarmid 8dcf768f4e backend/remote: Add IsLocalOperations
To ensure that the apply command can determine whether an operation is
executed locally or remotely, we add an IsLocalOperations method on the
remote backend. This returns the internal forceLocal boolean.

We also update this flag after checking if the corresponding remote
workspace is in local operations mode or not. This ensures that we know
if an operation is running locally (entirely on the practitioner's
machine), pseudo-locally (on a Terraform Cloud worker), or remotely
(executing on a worker, rendering locally).
2021-04-16 11:43:57 -04:00
Alisdair McDiarmid 69e7922a33 cli: Fix missing apply summary for remote runs
Disabling the resource count and outputs rendering when the remote
backend is in use causes them to be omitted from Terraform Cloud runs.
This commit changes the condition to render these values if either the
remote backend is not in use, or the command is running in automation
via the TF_IN_AUTOMATION flag. As this is intended to be set by
Terraform Cloud and other remote backend implementations, this addresses
the problem.
2021-04-16 10:03:22 -04:00
Alisdair McDiarmid 23800438ab
Merge pull request #28409 from hashicorp/alisdair/fix-remote-backend-ui-issues
cli: Fix remote backend UI issues
2021-04-16 09:03:34 -04:00
Alisdair McDiarmid fad305f884 cli: Fix remote backend UI issues
Fix two bugs which surface when using the remote backend:

- When migrating to views, we removed the call to `(*Meta).process`
  which initialized the color boolean. This resulted in the legacy UI
  calls in the remote backend stripping color codes. To fix this, we
  populate this boolean from the common arguments.
- Remote apply will output the resource summary and output changes, and
  these are rendered via the remote backend streaming. We need to
  special case this in the apply command and prevent displaying a
  zero-change summary line.

Neither of these are coverable by automated tests, as we don't have any
command-package level testing for the remote backend. Manually verified.
2021-04-16 08:28:33 -04:00
James Bardin d351d712c4 dynamic block MinItems MaxItems validation test 2021-04-15 17:34:33 -04:00
James Bardin 3e49c4b388 MinItems and MaxItems can be validated once again
The new hcldec dynamic block behavior no longer tried to validate
MinItems and MaxItems when the number of values is unknown.
2021-04-15 17:17:36 -04:00
Martin Atkins dedac2cdd6 website: v0.15 Upgrade Guide entry for Azure Backend arguments
Terraform v0.15 includes the conclusion of the deprecation cycle for some
renamed arguments in the "azure" backend.

We missed this on the first draft of the upgrade guide because this change
arrived along with various other more innocuous updates and so we didn't
spot it during our change review.
2021-04-15 10:30:11 -07:00
Martin Atkins 035d1648e4 website: Link to the v0.15 upgrade guide
Unfortunately it seems that this link got lost in a merge conflict when
we did the big nav refactor earlier in the v0.15 cycle, so here we'll
retoractively add it to the new location for upgrade guide nav, in the
language layout rather than the downloads layout.
2021-04-15 10:17:02 -07:00
Alisdair McDiarmid ec001d3e18
Merge pull request #28381 from hashicorp/alisdair/fix-double-mark-sensitive-attrs
core: Fix double-marked sensitive attributes
2021-04-15 10:13:29 -04:00
Alisdair McDiarmid 2390a11d60 core: Fix double-marked sensitive attributes
We need to unmark the decoded state and merge the marks with those from
the resource schema.

Co-authored-by: James Bardin <j.bardin@gmail.com>
2021-04-15 09:30:13 -04:00
James Bardin 2cd1619c40
Merge pull request #28329 from serejkus/dag/set-tiny-optimisations
tiny optimisations of dag.Set
2021-04-14 16:23:43 -04:00
James Bardin 59c8281378 use new dynamic decoding behavior
Dynamic blocks with unknown for_each expressions are now decoded into an
unknown value rather than using a sentinel object with unknown
and null attributes. This will allow providers to precisely plan the
block values, rather than trying to heuristically paper over the
incorrect plans when dynamic is in use.
2021-04-14 12:13:22 -04:00
Alisdair McDiarmid 2c8c387540
Merge pull request #28363 from hashicorp/alisdair/here-is-a-newline
command: Add a newline before confirming apply
2021-04-14 09:38:32 -04:00
Alisdair McDiarmid e4031eaccf command: Add a newline before confirming apply
This blank line delineating the plan and the query was accidentally
dropped as part of the views migration.
2021-04-14 09:30:49 -04:00
Martin Atkins 5f5432e8ea
website: v0.15 upgrade guide for sensitive resource attributes (#28355)
* website: v0.15 upgrade guide for sensitive resource attributes

Our earlier draft of this guide didn't include a section about the
stabilization of the "provider_sensitive_attrs" language experiment. This
new section aims to address the situation where a module might previously
have been returning a sensitive value without having marked it as such,
and thus that module will begin returning an error after upgrading to
Terraform v0.15.

As part of that, I also reviewed the existing documentation about these
features and made some edits aiming to make these four different sections
work well together if users refer to them all at once, as they are likely
to do if they follow the new links from the upgrade guide. I aimed to
retain all of the content we had before, but some of it is now in a new
location.

In particular, I moved the discussion about the v0.14 language experiment
into the upgrade guide, because it seems like a topic only really relevant
to those upgrading from an earlier version and not something folks need to
know about if they are using Terraform for the first time in v0.15 or
later.

* minor fixups

Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2021-04-14 09:04:40 -04:00
James Bardin b515ab583a make blocktoattr an hcldec.UnknownBody
This will allow any dynamic blocks that are fixed up as a blocktoattr
still decode into an unknown value.
2021-04-13 18:42:15 -04:00
Martin Atkins 140c613ae8 lang/funcs: "one" function
In the Terraform language we typically use lists of zero or one values in
some sense interchangably with single values that might be null, because
various Terraform language constructs are designed to work with
collections rather than with nullable values.

In Terraform v0.12 we made the splat operator [*] have a "special power"
of concisely converting from a possibly-null single value into a
zero-or-one list as a way to make that common operation more concise.

In a sense this "one" function is the opposite operation to that special
power: it goes from a zero-or-one collection (list, set, or tuple) to a
possibly-null single value.

This is a concise alternative to the following clunky conditional
expression, with the additional benefit that the following expression is
also not viable for set values, and it also properly handles the case
where there's unexpectedly more than one value:

    length(var.foo) != 0 ? var.foo[0] : null

Instead, we can write:

    one(var.foo)

As with the splat operator, this is a tricky tradeoff because it could be
argued that it's not something that'd be immediately intuitive to someone
unfamiliar with Terraform. However, I think that's justified given how
often zero-or-one collections arise in typical Terraform configurations.
Unlike the splat operator, it should at least be easier to search for its
name and find its documentation the first time you see it in a
configuration.

My expectation that this will become a common pattern is also my
justification for giving it a short, concise name. Arguably it could be
better named something like "oneornull", but that's a pretty clunky name
and I'm not convinced it really adds any clarity for someone who isn't
already familiar with it.
2021-04-12 15:32:03 -07:00
Alisdair McDiarmid 33e5d111fe
Merge pull request #28326 from hashicorp/alisdair/allow-nonsensitive-on-non-sensitive-values
lang/funcs: Make nonsensitive more permissive
2021-04-12 14:00:41 -04:00
Alisdair McDiarmid c1f7193454 lang/funcs: Make nonsensitive more permissive
Calling the nonsensitive function with values which are not sensitive
will result in an error. This restriction was added with the goal of
preventing confusingly redundant use of this function.

Unfortunately, this breaks when using nonsensitive to reveal the value of
sensitive resource attributes. This is because the validate walk does
not (and cannot) mark attributes as sensitive based on the schema,
because the resource value itself is unknown.

This commit therefore alters this restriction such that it permits
nonsensitive unknown values, and adds a test case to cover this specific
scenario.
2021-04-12 12:31:59 -04:00
Sergey Elantsev 1ad01debf3 tiny optimisations of dag.Set
1. Use hint for map size in Set.Copy().
2. Use Set.Copy() if Set.Difference() argument is empty.
2021-04-09 22:59:30 +03:00
James Bardin 1212bbec9f
Merge pull request #28317 from hashicorp/jbardin/delete-error-dependencies
restore saved dependencies on delete error
2021-04-08 11:38:57 -04:00
James Bardin 4bfabbaee4 restore saved dependencies on delete error
Is a resource delete action fails and the provider returned a new state,
we need to ensure the stored dependencies are retained.
2021-04-08 09:57:14 -04:00
James Bardin b7fb533bd2
Merge pull request #28275 from hashicorp/jbardin/diagnostic-addresses
Add addresses to diagnostics
2021-04-06 16:09:39 -04:00
James Bardin d04999863c "with" formatting 2021-04-06 15:50:30 -04:00
James Bardin 9e5baf4662 use WholeContainingBody instead of Sourceless
When returning generic grpc errors from a provider, use
WholeContainingBody so that callers can annotate the error with all the
available contextual information. This can help troubleshoot problems by
narrowing down problems to a particular configuration or specific
resource instance.
2021-04-06 15:15:52 -04:00
James Bardin 265b5106ca call the InConfigBody with addresses 2021-04-06 15:15:52 -04:00
James Bardin 406ac97965 add the address field to the view diagnostics 2021-04-06 15:15:52 -04:00
James Bardin 503c413de2 add addresses to diagnostics
Add an address argument to tfdiags.InConfigBody, and store the address
string the diagnostics details. Since nearly every place where we want
to annotate the diagnostics with the config context we also have some
sort of address, we can use the same call to insert them both into the
diagnostic.

Perhaps we should rename InConfigBody and ElaborateFromConfigBody to
reflect the additional address parameter, but for now we can verify this
is a pattern that suits us.
2021-04-06 15:15:52 -04:00
Víctor Felipe Godoy Hernández a9487c7674
Fix yamldecode example from json to yaml (#28220)
* Fix yamldecode example from json to yaml

* inline yaml example
2021-04-05 13:41:07 -04:00
Matthew Hooker 49c984ff77
bump go-getter to 1.5.2 (#28189) 2021-04-05 09:01:47 -04:00
Dennis Gursky 4e42d21837
Improve ModuleInstance String() performance (#28246)
* Optimize (m ModuleInstance) String()

Optimize (m ModuleInstance) String() to preallocate the buffer and use strings.Builder instead of bytes.Buffer

This leads to a common case only doing a single allocation as opposed to a few allocations which the bytes.Buffer is doing.

* adding a benchmark test

Result:

```
$ go test -bench=String ./addrs -benchmem 
BenchmarkStringShort-12         18271692                56.52 ns/op           16 B/op          1 allocs/op
BenchmarkStringLong-12           8057071               158.5 ns/op            96 B/op          1 allocs/op
PASS
$ git checkout main addrs/module_instance.go
$ go test -bench=String ./addrs -benchmem 
BenchmarkStringShort-12          7690818               162.0 ns/op            80 B/op          2 allocs/op
BenchmarkStringLong-12           2922117               414.1 ns/op           288 B/op          3 allocs/op
```

* Update module_instance_test.go

switch spaces to tabs
2021-04-05 08:44:27 -04:00
James Bardin 973ed6eae6
Merge pull request #28272 from hashicorp/jbardin/plan-destroy-skip-refesh
ensure plans always have a stored state
2021-04-01 16:49:09 -04:00
James Bardin b6b7f78b49 ensure plans always have a stored state
When refresh was skipped for a destroy plan, there was no state stored
in the plan.
2021-04-01 16:44:32 -04:00
James Bardin 1e1bb00e5b
Merge pull request #28267 from hashicorp/jbardin/data-depends_on-refs
Don't force data reads from dependency changes in sibling modules
2021-04-01 13:39:05 -04:00
James Bardin 349e99bb0c don't force data reads from sibling module changes
Dependencies are tracked via configuration addresses, but when dealing
with depends_on references they can only apply to resources within the
same module instance. When determining if a data source can be read
during planning, verify that the dependency change is coming from the
same module instance.
2021-04-01 12:03:34 -04:00
James Bardin 8871eff495 remove debug Println 2021-04-01 12:03:34 -04:00