Commit Graph

32 Commits

Author SHA1 Message Date
Kristin Laemmert b9138f4465
terraform: validate providers' schemas during NewContext (#28124)
* checkpoint save: update InternalValidate tests to compare exact error

* configschema: extract and extend attribute validation

This commit adds an attribute-specific InternalValidate which was extracted directly from the block.InternalValidate logic and extended to verify any NestedTypes inside an Attribute. Only one error message changed, since it is now valid to have a cty.NilType for Attribute.Type as long as NestedType is set.

* terraform: validate provider schema's during NewContext

We haven't been able to guarantee that providers are validating their own schemas using (some version of) InternalValidate since providers were split out of the main codebase. This PR adds a call to InternalValidate when provider schemas are initially loaded by NewContext, which required a few other changes:

InternalValidate's handling of errors vs multierrors was a little weird - before this PR, it was occasionally returning a non-nil error which only stated "0 errors occurred" - so I addressed that in InternalValidate. I then tested this with a configuration that was using all of our most popular providers, and found that at least on provider had some invalid attribute names, so I commented that particular validation out. Adding that in would be a breaking change which we would have to coordinate with enablement and providers and (especially in this case) make sure it's well communicated to external provider developers.

I ran a few very unscientific tests comparing the timing with and without this validation, and it appeared to only cause a sub-second increase.

* refactor validate error message to closer match the sdk's message

* better error message

* tweak error message: move the instruction to run init to the end of the message, after the specific error.
2021-03-22 13:17:50 -04:00
Kristin Laemmert 2b4e389839
Mildwonkey/nested type format (#27793)
* command/format: check for sensitive NestedTypes

Eventually, the diff formatter will need to be updated to properly
handle NestedTypes, but for now we can let the existing function deal
with them as regular cty.Object-type attributes.

To avoid printing sensitive nested attributes, we will treat any
attribute with at least one sensitive nested attribute as an entirely
sensitive attribute.

* bugfix for Object ImpliedType()

ImpliedType() was returning too early when the given object had optional
attributes, therefore skipping the incredibly important step of
accounting for the nesting mode when returning said type.
2021-02-18 08:48:52 -05:00
Kristin Laemmert 1cf4909b28 configschema: fix various issues with NestedTypes
A handful of bugs popped up while extending the testing in
plans/objchange. The main themes were failing to recurse through deeply
nested NestedType attributes and improperly building up the ImpliedType.
This commit fixes those issues and extends the test coverage to match.
2021-02-12 13:34:25 -05:00
Kristin Laemmert 6aa90a51d0 ContainsSensitive must descend into nested objects 2021-02-11 10:32:31 -05:00
Kristin Laemmert 3ad720e9dc configs/configschema: add new NestedType to attribute
This commit adds a new field, NestedType, to the Attribute schema, and
extends the current Attribute decoderSpec to account for the new type.
The codepaths are mostly unused and included in a separate commit to
verify that the included changes do not impact any other tests yet.
2021-02-05 13:34:55 -05:00
James Bardin e01d37d0dc Block.AttribuuteByPath
There are a few places where we want to perform some transformation on a
cty.Value, but require information from the schema. Rather than create
bespoke functions to walk the cty.Value and schema in concert, we can
provide Attribute information from a cty.Path allowing the use of
Value.Transform in these cases.
2021-01-12 16:31:18 -05:00
James Bardin 9c3ece40f7 configs/configschema: staticcheck 2020-12-02 13:59:19 -05:00
James Bardin e27ecba6e4 extended cache comments 2020-10-14 12:45:06 -04:00
James Bardin bb76c3b50c
Update configs/configschema/decoder_spec.go
Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-10-14 10:33:44 -04:00
James Bardin d40e2fb8d1 cache DecoderSpec calls
DecoderSpec may be called many times, and deeply recursive calls are
expensive. Since we cannot synchronize the Blocks themselves due to them
being copied in parts of the code, we use a separate cache to store the
generated Specs.
2020-10-14 09:19:26 -04:00
Martin Atkins 593cf7b4d5 didyoumean: move from "helper" to "internal"
This new-ish package ended up under "helper" during the 0.12 cycle for
want of some other place to put it, but in retrospect that was an odd
choice because the "helper/" tree is otherwise a bunch of legacy code from
when the SDK lived in this repository.

Here we move it over into the "internal" directory just to distance it
from the guidance of not using "helper/" packages in new projects;
didyoumean is a package we actively use as part of error message hints.
2020-10-02 13:35:07 -07:00
Paddy e6592dc710
Add support for provider metadata to modules. (#22583)
Implement a new provider_meta block in the terraform block of modules, allowing provider-keyed metadata to be communicated from HCL to provider binaries.

Bundled in this change for minimal protocol version bumping is the addition of markdown support for attribute descriptions and the ability to indicate when an attribute is deprecated, so this information can be shown in the schema dump.

Co-authored-by: Paul Tyng <paul@paultyng.net>
2020-03-05 16:53:24 -08:00
Radek Simko 7860f55e4f
Version tools per Go convention under tools.go 2019-10-17 22:23:39 +02:00
Martin Atkins 39e609d5fd vendor: switch to HCL 2.0 in the HCL repository
Previously we were using the experimental HCL 2 repository, but now we'll
shift over to the v2 import path within the main HCL repository as part of
actually releasing HCL 2.0 as stable.

This is a mechanical search/replace to the new import paths. It also
switches to the v2.0.0 release of HCL, which includes some new code that
Terraform didn't previously have but should not change any behavior that
matters for Terraform's purposes.

For the moment the experimental HCL2 repository is still an indirect
dependency via terraform-config-inspect, so it remains in our go.sum and
vendor directories for the moment. Because terraform-config-inspect uses
a much smaller subset of the HCL2 functionality, this does still manage
to prune the vendor directory a little. A subsequent release of
terraform-config-inspect should allow us to completely remove that old
repository in a future commit.
2019-10-02 15:10:21 -07:00
James Bardin 13e2e10577 fix Min/Max validation during decoding
We can only validate MinItems >= 1 (equiv to "Required") during
decoding, as dynamic blocks each only decode as a single block. MaxItems
cannot be validated at all, also because of dynamic blocks, which may
have any number of blocks in the config.
2019-08-20 10:13:21 -04:00
James Bardin 731d4226d3 do not validate Min/Max Items in CoerceValue
Due to both the nature of dynamic blocks, and the need for resources to
sometimes communicate incomplete values, we cannot validate MinItems and
MaxItems in CoerceValue.
2019-08-20 10:13:15 -04:00
James Bardin 554cedab8a don't validate Min/Max block vals in CoerceValue
A provider may not have the data to fill in required block values in all
cases during the resource Read operation. This is more common in import,
because there is no initial configuration or state, and it's possible
some values are only provided in the configuration.

The original intent of MinItems and MaxItems in the schema was to
enforce configuration constraints, not to enforce what the resource
could save in the state.  Since the configuration is already statically
validated, and the Schema is validated against the configuration in a
separate step, we can drop these extra validation constraints in
CoerceValue and relax it to only ensure the types conform to what is
expected.
2019-08-15 10:02:39 -04:00
James Bardin 67dbd6d345 don't check MinItems with unknowns in blocks
If a block was defined via "dynamic", there will be only one block value
until the expansion is known. Since we can't detect dynamic blocks at
this point, don't verify MinItems while there are unknown values in the
config.

The decoder spec can also only check for existence of a block, so limit
the check to 0 or 1.
2019-07-27 11:50:28 -07:00
Martin Atkins 88e76fa9ef configs/configschema: Introduce the NestingGroup mode for blocks
In study of existing providers we've found a pattern we werent previously
accounting for of using a nested block type to represent a group of
arguments that relate to a particular feature that is always enabled but
where it improves configuration readability to group all of its settings
together in a nested block.

The existing NestingSingle was not a good fit for this because it is
designed under the assumption that the presence or absence of the block
has some significance in enabling or disabling the relevant feature, and
so for these always-active cases we'd generate a misleading plan where
the settings for the feature appear totally absent, rather than showing
the default values that will be selected.

NestingGroup is, therefore, a slight variation of NestingSingle where
presence vs. absence of the block is not distinguishable (it's never null)
and instead its contents are treated as unset when the block is absent.
This then in turn causes any default values associated with the nested
arguments to be honored and displayed in the plan whenever the block is
not explicitly configured.

The current SDK cannot activate this mode, but that's okay because its
"legacy type system" opt-out flag allows it to force a block to be
processed in this way anyway. We're adding this now so that we can
introduce the feature in a future SDK without causing a breaking change
to the protocol, since the set of possible block nesting modes is not
extensible.
2019-04-10 14:53:52 -07:00
Martin Atkins a20084dc0e configs/configschema: EmptyValue methods
These helpers determine the value that would be used for a particular
schema construct if the configuration construct it represents is not
present (or, in the case of *Block, empty) in the configuration.

This is different than cty.NullVal on the implied type because it might
return non-null "empty" values for certain constructs if their absence
would be reported as such during a decode with no required attributes or
blocks.
2019-04-10 14:53:52 -07:00
Martin Atkins b217624d83 config/configupgrade: Test to show that list unwrapping works for sets
This was already working but we had no tests to prove it.
2019-02-22 17:40:40 -08:00
Martin Atkins e831182c8d plans/objchange: Hide sensitive attribute values in error messages
Since these error messages get printed in Terraform's output and we
encourage users to share them as part of bug reports, we should avoid
including sensitive information in them to reduce the risk of accidental
exposure.
2019-02-11 17:26:50 -08:00
James Bardin c20164ab31 fix CoerceValue to handle changing dynamic types
Objects with DynamicPseudoType attributes can't be coerced within a map
if a concrete type is set. Change the Value type used to an Object when
there is a type mismatch.
2019-02-08 16:36:27 -05:00
Martin Atkins bbbf22d8e4 configs/configschema: Block.StaticValidateTraversal method
This allows basic static validation of a traversal against a schema, to
verify that it represents a valid path through the structural parts of
the schema.

The main purpose of this is to produce better error messages (using our
knowledge of the schema) than we'd be able to achieve by just relying
on HCL expression evaluation errors. This is particularly important for
nested blocks because it may not be obvious whether one is represented
internally by a set or a list, and incorrect usage would otherwise produce
a confusing HCL-oriented error message.
2018-11-26 08:25:03 -08:00
James Bardin ebc9745788 fix "too many items" error message 2018-11-13 18:41:53 -05:00
James Bardin b3a491d035 use correct block types in CoerceValue
When creating Null or Unknown values during CoerceValue, the the outer
block type was being used rather than the current block type.
2018-10-18 18:09:43 -04:00
Martin Atkins 549544f201 configschema: Handle nested blocks containing dynamic-typed attributes
We need to make the collection itself be a tuple or object rather than
list or map in this case, since otherwise all of the elements of the
collection are constrained to be of the same type and that isn't the
intent of a provider indicating that it accepts any type.
2018-10-16 19:11:09 -07:00
Martin Atkins 0317da9911 plans/objchange: logic for merging prior state with config
This produces a "proposed new state", which already has prior computed
values propagated into it (since that behavior is standard for all
resource types) but could be customized further by the provider to make
the "_planned_ new state".

In the process of implementing this it became clear that our configschema
DecoderSpec behavior is incorrect, since it's producing list values for
NestingList and map values for NestingMap. While that seems like it should
be right, we should actually be using tuple and object types respectively
to allow each block to have a different runtime type in situations where
an attribute is given the type cty.DynamicPseudoType. That's not fixed
here, and so without a further fix list and map blocks will panic here.
The DecoderSpec implementation will be fixed in a subsequent commit.
2018-10-16 19:11:09 -07:00
James Bardin 8f295fcb22 CoerceValue should insert Null for unset attrs
This matches the decoder spec, where a value unset in the configuration
is always Null.
2018-10-16 18:53:51 -07:00
James Bardin 50e099ad10 don't set Unknown for computed values
Any value that is not set in the configuration should decode as a Null
value. Unknown should still be returned if a computed value expression
is unknown.
2018-10-16 18:53:51 -07:00
Martin Atkins d8bf3cc4e0 configschema: Fix ConfigSchema bugs with nested blocks
We were iterating over the wrong value to recursively coerce content for
nested blocks, and also incorrectly constructing the cty.Path used in
errors.
2018-10-16 18:53:51 -07:00
Martin Atkins 479c6b2466 move "configschema" from "config" to "configs"
The "config" package is no longer used and will be removed as part
of the 0.12 release cleanup. Since configschema is part of the
"new world" of configuration modelling, it makes more sense for
it to live as a subdirectory of the newer "configs" package.
2018-10-16 18:50:29 -07:00