Commit Graph

28255 Commits

Author SHA1 Message Date
Alisdair McDiarmid d37c3c597a
Merge pull request #28245 from hashicorp/alisdair/fix-sensitive-value-sensitive-attribute-conflict
core: Fix sensitive value/attribute conflict
2021-03-31 09:29:53 -04:00
Alisdair McDiarmid c54c18680e core: Fix sensitive value/attribute conflict
When applying sensitivity marks to resources, we previously would first
mark any provider-denoted sensitive attributes, then apply the set of
planned-change sensitive value marks. This would cause a panic if a
provider marked an iterable value as sensitive, because it is invalid to
call `MarkWithPaths` against a marked iterable value.

Instead, we now merge the marks from the provider schema and the planned
change into a single set, and apply them with one call. The included
test panics without this change.
2021-03-30 15:51:09 -04:00
Martin Atkins f47c4efb1b website: Dynamic blocks can for_each any collection type
We previously added a hint to both resource for_each and dynamic blocks
about using the "flatten" and "setproduct" situations to construct
suitable collections to repeat over.

However, we used the same text in both places which ended up stating that
dynamic blocks can only accept map or set values, which is a constraint
that applies to resource for_each (because we need to assign a unique
identifier to each instance) and not to dynamic blocks (which don't have
any uniqueness enforced by Terraform Core itself).

To remove that contradiction with the text above which talks about what
is valid here, I've just generalized this to say "collection", because
the primary point of this paragraph is the "one element per desired nested
block" part, not specifically what sort of collections are permitted in
this location. (Text further up describes the supported types.)
2021-03-30 09:43:33 -07:00
James Bardin 7ab4f1fb7d
Merge pull request #28240 from hashicorp/jbardin/test-builds
remove unsupported platforms from CI test builds
2021-03-30 11:22:09 -04:00
James Bardin 23e08cd064 remove unsupported platforms from tests
Rather than increase resources for more cross-platform builds, opt to
remove platforms that are not officially supported.
2021-03-30 11:10:03 -04:00
James Bardin 1f53cfcd43
Merge pull request #28238 from hashicorp/jbardin/circle-resource-class
provide more memory for builds
2021-03-30 10:48:24 -04:00
James Bardin 6ef5ed1060 provide more memory for builds
Cross-platform builds are running out of resources. Try using a little
more memory.
2021-03-30 10:36:16 -04:00
James Bardin 1a158743b3
Merge pull request #28228 from hashicorp/jbardin/destroy-deps
Stored dependency handling during plan
2021-03-30 09:19:14 -04:00
Alisdair McDiarmid 4fb505631c
Merge pull request #28230 from hashicorp/alisdair/only-rewrite-provider-locks-file-if-changed
cli: Only rewrite provider locks file if changed
2021-03-30 09:12:45 -04:00
Alisdair McDiarmid 786d99207e cli: Only rewrite provider locks file if changed
If the provider locks have not changed, there is no need to rewrite the
locks file. Preventing this needless rewrite should allow Terraform to
operate in a read-only directory, so long as the provider requirements
don't change.
2021-03-29 16:09:07 -04:00
Alisdair McDiarmid 340543d122
Merge pull request #28201 from hashicorp/alisdair/planfile-sensitive-required-replace
Add sensitivity and required-replace fields to plan file, and expose sensitivity to JSON plan
2021-03-29 15:08:36 -04:00
James Bardin 7964328f34 merge dependencies when refreshing
The resource configuration was always being used to determine
dependencies during refresh, because if there were no changes to a
resource, there was no chance to replace any incorrect stored
dependencies. Now that we are refreshing during plan, we can determine
that a resource has no changes and opt to store the new dependencies
immediately.

Here we clean up the writeResourceInstanceState calls to no longer
modify the resource instance state, removing the `dependencies`
argument. Callers are now expected to set the Dependencies field as
needed.
2021-03-29 14:34:01 -04:00
Robin Norwood e3bd661470
Merge pull request #28164 from hashicorp/rln-add-resource-targeting-tutorial-callout
Add callout to resource targeting tutorial
2021-03-29 09:12:30 -05:00
Alisdair McDiarmid 5e30d58dc2 command/jsonplan: Add output change sensitivity
When an output value changes, we have a small amount of information we
can convey about its sensitivity. If either the output was previously
marked sensitive, or is currently marked sensitive in the config, this
is tracked in the output change data.

This commit encodes this boolean in the change struct's
`before_sensitive` and `after_sensitive` fields, in the a way which
matches resource value sensitivity. Since we have so little information
to work with, these two values will always be booleans, and always equal
each.

This is logically consistent with how else we want to obscure sensitive
data: a changing output which was or is marked sensitive should not have
the value shown in human-readable output.
2021-03-26 19:26:11 -04:00
Alisdair McDiarmid 63613ca1b0 command/jsonconfig: Add variable sensitive flag 2021-03-26 19:26:11 -04:00
Alisdair McDiarmid e27aacebf9 command/jsonplan: Add sensitive value mapping data
Similar to `after_unknown`, `before_sensitive` and `after_sensitive` are
values with similar structure to `before` and `after` which encode the
presence of sensitive values in a planned change. These should be used
to obscure sensitive values from human-readable output.

These values follow the same structure as the `before` and `after`
values, replacing sensitive values with `true`, and non-sensitive values
with `false`. Following the `after_unknown` precedent, we omit
non-sensitive `false` values for object attributes/map values, to make
serialization more compact.

One difference from `after_unknown` is that a sensitive complex value
(collection or structural type) is replaced with `true`. If the complex
value itself is sensitive, all of its contents should be obscured.
2021-03-26 19:26:10 -04:00
Matthew Frahry 15779013da
backend/azurerm: Bug fixes and updated dependencies #28181 2021-03-26 14:07:56 -07:00
Judith Malnick 75fbbe8065
clarify version that pull upgrades local state to (#28204) 2021-03-26 13:57:44 -07:00
Alisdair McDiarmid fd7c4105f2
Merge pull request #28202 from hashicorp/alisdair/fmt-multiline-value-expr-unwrap
cli: Fix fmt output for multi-line value exprs
2021-03-26 11:33:14 -04:00
James Bardin 985124bfa8 failing test for removed cbd reference
A stored dependency is documented as being honored even after it is
removed from the configuration, until the referenced resource is
destroyed.
2021-03-25 17:39:53 -04:00
James Bardin bac4e0a3de fix ResourceInstanceObject.DeepCopy
missing CreateBeforeDestroy
2021-03-25 17:39:53 -04:00
Martin Atkins 6f35c2847b command: Reorganize docs of the local backend's legacy CLI options
We have these funny extra options that date back to before Terraform even
had remote state, which we've preserved along the way by most recently
incorporating them as special-case overrides for the local backend.

The documentation we had for these has grown less accurate over time as
the details have shifted, and was in many cases missing the requisite
caveats that they are only for the local backend and that backend
configuration is the modern, preferred way to deal with the use-cases they
were intended for.

We always have a bit of a tension with this sort of legacy option because
we want to keep them documented just enough to be useful to someone who
finds an existing script/etc using them and wants to know what they do,
but not to take up so much space that they might distract users from
finding the modern alternative they should consider instead.

As a compromise in that vein here I've created a new section about these
options under the local backend documentation, which then gives us the
space to go into some detail about the various behaviors and interactions
and also to discuss their history and our recommended alternatives. I then
simplified all of the other mentions of these in command documentation
to just link to or refer to the local backend documentation. My hope then
is that folks who need to know what these do can still find the docs, but
that information can be kept out of the direct path of new users so they
can focus on learning about remote backends instead.

This is certainly not the most ideal thing ever, but it seemed like the
best compromise between the competing priorities I described above.
2021-03-25 13:56:48 -07:00
Matthew Frahry 13b41d59f5 Website Test Fix 2021-03-25 13:47:12 -07:00
Alisdair McDiarmid d71f0c6149 cli: Fix fmt output for multi-line value exprs
The formatter for value expressions which use legacy interpolation
syntax was previously behaving incorrectly with some multi-line
expressions. Any HCL expression which requires parenthesis to be allowed
to span multiple lines could be skip those parens if already inside
string interpolation (`"${}"`).

When removing string interpolation, we now check for a resulting
multi-line expression, and conservatively ensure that it starts and ends
with parenthesis. These may be redundant, as not all expressions require
parens to permit spanning multiple lines, but at least it will be valid
output.
2021-03-25 15:40:54 -04:00
Alisdair McDiarmid a12c413b84 plans/planfile: Add required-replace and sensitive
The stored planfile now serializes the required-replace path set and the
collection of before/after sensitivity marks. This ensures that storing
a plan and displaying it with `terraform show` renders the same output
for plans with required-replace resources, and those with sensitive
values in the diff.
2021-03-25 14:42:34 -04:00
Dennis Gursky 550de86135
Set Intersection #performance (#28183)
* Set Intersection #performance

Intersection is faster for sets of different sizes if one iterates over the shorter set and checks the presence of an element in a larger one. For an edge case consider `s` having 1M entries and `other` no entries at all. In this case original code will iterate over 1M entries in `s` not finding anything and then returning an empty result set. In the patched code the iteration won't happen at all and result is returned immediately.

This change is inspired by profiling a relatively large terraform configuration, where the time to validate was sped up 4x with this change.
2021-03-24 13:04:37 -04:00
James Bardin 07ecfb13f0
Merge pull request #28180 from hashicorp/jbardin/mark-provider-sensitive
check for unknowns when marking resource values
2021-03-24 10:26:07 -04:00
James Bardin 8cba2bfc39 check for unknowns when marking resource values
When we map schema sensitivity to resource values, there may be unknowns
when dealing with planned objects. Check for unknowns before iterating
over block values.
2021-03-23 17:06:18 -04:00
Alisdair McDiarmid 841cced6c9
Merge pull request #28138 from hashicorp/alisdair/command-views-validate
cli: Migrate validate command to views
2021-03-23 13:13:06 -04:00
James Bardin b5e5948b64
Merge pull request #28176 from hashicorp/jbardin/destroy-doc
update destroying doc to show show more CBD detail
2021-03-23 13:04:56 -04:00
James Bardin 44199ae4ba update destroying doc to show show more CBD detail
Update the create-before-destroy cases in the destroying.md document.
2021-03-23 11:21:43 -04:00
James Bardin 0ebc71383b
Merge pull request #28165 from hashicorp/jbardin/destroy-update-dep
Destroy then update dependency ordering
2021-03-23 11:04:55 -04:00
Alisdair McDiarmid 0e43fa9834
Merge pull request #28166 from hashicorp/alisdair/fix-typo
Fix typo in plan no changes output message
2021-03-23 09:31:43 -04:00
Matthew Frahry 0bbb0dc200 Fix for #27809 2021-03-22 14:20:54 -07:00
Alisdair McDiarmid 7242d9af2b Fix typo in plan no changes output message 2021-03-22 16:39:53 -04:00
Alisdair McDiarmid 46c2dce0fb Fix broken test 2021-03-22 16:38:56 -04:00
James Bardin dcb12195a3 update destroying.md
Update the full-replacement example graph to show the transitive
dependency that is required for the destroy-then-update case. Add
another example describing the case where we reduce the graph to
only an update and replace and the dependency on the destroy node
remains.
2021-03-22 15:29:51 -04:00
James Bardin 0bc64e3cc4 tests for destroy-then-update dependency ordering 2021-03-22 14:18:54 -04:00
Matthew Frahry 3546650ac6 backend/azurerm: adding the right role name 2021-03-22 10:51:01 -07:00
Matthew Frahry 3722b1b613 backend/azurerm: support for using azuread authentication for blobs 2021-03-22 10:49:34 -07:00
Kristin Laemmert b9138f4465
terraform: validate providers' schemas during NewContext (#28124)
* checkpoint save: update InternalValidate tests to compare exact error

* configschema: extract and extend attribute validation

This commit adds an attribute-specific InternalValidate which was extracted directly from the block.InternalValidate logic and extended to verify any NestedTypes inside an Attribute. Only one error message changed, since it is now valid to have a cty.NilType for Attribute.Type as long as NestedType is set.

* terraform: validate provider schema's during NewContext

We haven't been able to guarantee that providers are validating their own schemas using (some version of) InternalValidate since providers were split out of the main codebase. This PR adds a call to InternalValidate when provider schemas are initially loaded by NewContext, which required a few other changes:

InternalValidate's handling of errors vs multierrors was a little weird - before this PR, it was occasionally returning a non-nil error which only stated "0 errors occurred" - so I addressed that in InternalValidate. I then tested this with a configuration that was using all of our most popular providers, and found that at least on provider had some invalid attribute names, so I commented that particular validation out. Adding that in would be a breaking change which we would have to coordinate with enablement and providers and (especially in this case) make sure it's well communicated to external provider developers.

I ran a few very unscientific tests comparing the timing with and without this validation, and it appeared to only cause a sub-second increase.

* refactor validate error message to closer match the sdk's message

* better error message

* tweak error message: move the instruction to run init to the end of the message, after the specific error.
2021-03-22 13:17:50 -04:00
Matthew Frahry 341479087c backend/azurerm: adding support for azuread authentication 2021-03-22 10:15:41 -07:00
Robin Norwood 31323b911b Add callout to resource targeting tutorial 2021-03-22 11:55:41 -05:00
Matthew Frahry a978d4ee99 website: adding the new fields to azurerm 2021-03-22 09:53:52 -07:00
Matthew Frahry b0b0a44a67 backend/azurerm: added a feature flag for using AzureAD to authenticate 2021-03-22 09:33:57 -07:00
Matthew Frahry 05b45ab4f3 backend/azurerm: removing support for the deprecated fields 2021-03-22 09:26:06 -07:00
Matthew Frahry 3961f08e63 dependencies: upgrade all the azure things 2021-03-22 09:22:16 -07:00
Kristin Laemmert 77562d9b57
command/jsonprovider: bump format version (#28115)
Support for attributes with NestedTypes was added in https://github.com/hashicorp/terraform/pull/28055, and should have included a format version bump: this is a backwards-compatible change, but consumers will need to be updated in order to properly decode attributes (with NestedTypes) going forward.
2021-03-22 11:45:36 -04:00
Tej-Singh-Rana 2a49d908b8
fix the typo (#28140) 2021-03-22 10:51:45 -04:00
Kristin Laemmert 125e9d69d4
terraform-bundle: return an error if "versions" argument is omitted (#28158) 2021-03-22 10:51:03 -04:00