Commit Graph

309 Commits

Author SHA1 Message Date
Martin Atkins 4c5b866dea Move registry/ to internal/registry/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00
Martin Atkins 27ad9861ce configs: Meta-argument escaping blocks
Several top-level block types in the Terraform language have a body where
two different schemas are overlayed on top of one another: Terraform first
looks for "meta-arguments" that are built into the language, and then
evaluates all of the remaining arguments against some externally-defined
schema whose content is not fully controlled by Terraform.

So far we've been cautiously adding new meta-arguments in these namespaces
after research shows us that there are relatively few existing providers
or modules that would have functionality masked by those additions, but
that isn't really a viable path forward as we prepare to make stronger
compatibility promises.

In an earlier commit we've introduced the foundational parts of a new
language versioning mechanism called "editions" which should allow us to
make per-module-opt-in breaking changes in the future, but these shared
namespaces remain a liability because it would be annoying if adopting a
new edition made it impossible to use a feature of a third-party provider
or module that was already using a name that has now become reserved in
the new edition.

This commit introduces a new syntax intended to be a rarely-used escape
hatch for that situation. When we're designing new editions we will do our
best to choose names that don't conflict with commonly-used providers and
modules, but there are many providers and modules that we cannot see and
so there is a risk that any name we might choose could collide with at
least one existing provider or module. The automatic migration tool to
upgrade an existing module to a new edition should therefore detect that
situation and make use of this escaping block syntax in order to retain
the existing functionality until all the called providers or modules are
updated to no longer use conflicting names.

Although we can't put in technical constraints on using this feature for
other purposes (because we don't know yet what future editions will add),
this mechanism is intentionally not documented for now because it serves
no immediate purpose. In effect, this change is just squatting on the
syntax of a special block type named "_" so that later editions can make
use of it without it _also_ conflicting, creating a confusing nested
escaping situation. However, the first time a new edition actually makes
use of this syntax we should then document alongside the meta-arguments
so folks can understand the meaning of escaping blocks produced by
edition upgrade tools.
2021-05-17 11:17:25 -07:00
James Bardin 655f18c393 don't build a config if it didn't load properly 2021-05-07 11:52:06 -04:00
James Bardin d76a8fcf6c null check in ValueMarks 2021-05-06 11:08:20 -04:00
James Bardin 168296b507 validate that modules name providers passed to mod
Passing a provider into a module requires that it be named within the
module. This would previously pass validation, however core would fail
to resolve the provider resulting in an unclear "provider not found"
error.
2021-05-04 16:41:04 -04:00
Alisdair McDiarmid c89004d223 core: Add sensitive provider attrs to JSON plan
When rendering a stored plan file as JSON, we include a data structure
representing the sensitivity of the changed resource values. Prior to
this commit, this was a direct representation of the sensitivity marks
applied to values via mechanisms such as sensitive variables, sensitive
outputs, and the `sensitive` function.

This commit extends this to include sensitivity based on the provider
schema. This is in line with the UI rendering of the plan, which
considers these two different types of sensitivity to be equivalent.

Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2021-04-27 10:29:34 -04:00
James Bardin 4997b9b5bb
Merge pull request #28424 from hashicorp/jbardin/dynamic
Better planing of unknown dynamic blocks
2021-04-23 18:26:21 -04:00
James Bardin d0cc7f1d5e resolve provider types when building the config
All the information is available to resolve provider types when building
the configuration, but some provider references still had no FQN. This
caused validation to assume a default type, and incorrectly reject valid
module calls with non-default namespaced providers.

Resolve as much provider type information as possible when loading the
config. Only use this internally for now, but this should be useful
outside of the package to avoid re-resolving the providers later on. We
can come back and find where this might be useful elsewhere, but for now
keep the change as small as possible to avoid any changes in behavior.
2021-04-16 12:37:50 -04:00
James Bardin 3e49c4b388 MinItems and MaxItems can be validated once again
The new hcldec dynamic block behavior no longer tried to validate
MinItems and MaxItems when the number of values is unknown.
2021-04-15 17:17:36 -04:00
Kristin Laemmert b9138f4465
terraform: validate providers' schemas during NewContext (#28124)
* checkpoint save: update InternalValidate tests to compare exact error

* configschema: extract and extend attribute validation

This commit adds an attribute-specific InternalValidate which was extracted directly from the block.InternalValidate logic and extended to verify any NestedTypes inside an Attribute. Only one error message changed, since it is now valid to have a cty.NilType for Attribute.Type as long as NestedType is set.

* terraform: validate provider schema's during NewContext

We haven't been able to guarantee that providers are validating their own schemas using (some version of) InternalValidate since providers were split out of the main codebase. This PR adds a call to InternalValidate when provider schemas are initially loaded by NewContext, which required a few other changes:

InternalValidate's handling of errors vs multierrors was a little weird - before this PR, it was occasionally returning a non-nil error which only stated "0 errors occurred" - so I addressed that in InternalValidate. I then tested this with a configuration that was using all of our most popular providers, and found that at least on provider had some invalid attribute names, so I commented that particular validation out. Adding that in would be a breaking change which we would have to coordinate with enablement and providers and (especially in this case) make sure it's well communicated to external provider developers.

I ran a few very unscientific tests comparing the timing with and without this validation, and it appeared to only cause a sub-second increase.

* refactor validate error message to closer match the sdk's message

* better error message

* tweak error message: move the instruction to run init to the end of the message, after the specific error.
2021-03-22 13:17:50 -04:00
James Bardin 759b76436a validate provider_meta contains no interpolations
The provider_meta specification does not allow interpolation, but we
were not preventing it in the configuration.
2021-03-12 12:10:25 -05:00
Martin Atkins b5adc33075 configs: Accept and minimally validate a "language" argument
We expect that in order to continue to evolve the language without
breaking existing modules we will at some point need to have a way to mark
when a particular module is expecting a newer interpretation of the
language.

Although it's too early to do any deep preparation for that, this commit
aims to proactively reserve an argument named "language" inside
"terraform" blocks, which currently only accepts the keyword TF2021 that
is intended to represent "the edition of the Terraform language as defined
in 2021".

That argument also defaults to TF2021 if not set, so in practice there's
no real reason to set this today, but this minimal validation today is
intended to give better feedback to users of older Terraform versions in
the event that we introduce a new language edition later and they try to
use an module incompatible with their Terraform version.
2021-02-26 11:43:32 -08:00
Martin Atkins 54cc4dadf6 configs: A better error message for custom variable validation
Our previous message conflated the requirement for a full sentence with
the suggestion to write in a style similar to Terraform's built-in error
messages, which created a sense that the system would actively reject an
error message written in another language.

There's no intent here to block writing error messages in other languages,
but there is a practical consideration that Terraform's UI output is
currently not localized and so consistency with Terraform's other output,
if that's important to a module author, will typically mean writing the
error message in English.
2021-02-26 09:36:42 -08:00
Pam Selle 8f7807684a Upgrade to quoted keywords to error
The warning about deprecation is upgraded to an error
2021-02-21 20:27:20 -05:00
Pam Selle 61bee3cd44
Merge pull request #27834 from hashicorp/pselle/ignore-changes-wildcard
Upgrade ignore_changes wildcard from warning to error
2021-02-19 16:16:39 -05:00
Pam Selle fa7c3d7e10 Remove interpolation-only warning
These interpolations are removed when upgrading using 0.12upgrade,
and are removed in terraform fmt in many cases
2021-02-19 10:59:09 -05:00
Pam Selle 230658f2b2 Upgrade ignore_changes wildcard from warning to error
The syntax ignore_changes = ["*"] was deprecated and now errors.
Use = all instead.
2021-02-19 10:34:20 -05:00
Kristin Laemmert 2b4e389839
Mildwonkey/nested type format (#27793)
* command/format: check for sensitive NestedTypes

Eventually, the diff formatter will need to be updated to properly
handle NestedTypes, but for now we can let the existing function deal
with them as regular cty.Object-type attributes.

To avoid printing sensitive nested attributes, we will treat any
attribute with at least one sensitive nested attribute as an entirely
sensitive attribute.

* bugfix for Object ImpliedType()

ImpliedType() was returning too early when the given object had optional
attributes, therefore skipping the incredibly important step of
accounting for the nesting mode when returning said type.
2021-02-18 08:48:52 -05:00
Kristin Laemmert 7943312ebd
Merge pull request #27699 from hashicorp/mildwonkey/protocolv6
Mildwonkey/protocolv6
2021-02-16 14:51:53 -05:00
Alisdair McDiarmid bb3f59b016
Merge pull request #27697 from hashicorp/alisdair/json-plan-provider-required-versions
cli: Fix for missing provider requirements in JSON plan
2021-02-16 09:05:14 -05:00
Kristin Laemmert 1cf4909b28 configschema: fix various issues with NestedTypes
A handful of bugs popped up while extending the testing in
plans/objchange. The main themes were failing to recurse through deeply
nested NestedType attributes and improperly building up the ImpliedType.
This commit fixes those issues and extends the test coverage to match.
2021-02-12 13:34:25 -05:00
James Bardin 4195531925 old comment 2021-02-11 17:39:51 -05:00
Kristin Laemmert 6aa90a51d0 ContainsSensitive must descend into nested objects 2021-02-11 10:32:31 -05:00
James Bardin da252de1a0 add provider validation
Add validation which was removed from the configload package, along with
additional validation checks. The output is slightly different, as
instead of validating whether the modules are allowed to have provider
configurations, we validate the various combinations of provider
structures themselves.
2021-02-11 10:22:18 -05:00
James Bardin 7aaffac223 configload should not be doing validation
The configload package should only be responsible for locating and
loading the configuration, and not be further inspecting the config
source itself. Moving the validating into the configs package.
2021-02-10 10:20:40 -05:00
James Bardin ac585be079 initial support for parsing configuration_aliases
Add support for parsing configuration_aliases in required_providers
entries. The decoder needed to be re-written here in order to support
the bare reference style usage of provider names so that they match the
usage in other location within configuration. The only change to
existing handling of the required_providers block is more precise error
locations in a couple cases.
2021-02-09 08:38:30 -05:00
Alisdair McDiarmid 7cae76383a cli: Fix for provider requirements in JSON plan
The JSON plan output format includes a serialized, simplified version of
the configuration. One component of this config is a map of provider
configurations, which includes version constraints.

Until now, only version constraints specified in the provider config
blocks were exposed in the JSON plan output. This is a deprecated method
of specifying provider versions, and the recommended use of a
required_providers block resulted in the version constraints being
omitted.

This commit fixes this with two changes:

- When processing the provider configurations from a module, output the
  fully-merged version constraints for the entire module, instead of any
  constraints set in the provider configuration block itself;
- After all provider configurations are processed, iterate over the
  required_providers entries to ensure that any configuration-less
  providers are output to the JSON plan too.

No changes are necessary to the structure of the JSON plan output, so
this is effectively a semantic level bug fix.
2021-02-05 14:01:58 -05:00
Kristin Laemmert 3ad720e9dc configs/configschema: add new NestedType to attribute
This commit adds a new field, NestedType, to the Attribute schema, and
extends the current Attribute decoderSpec to account for the new type.
The codepaths are mostly unused and included in a separate commit to
verify that the included changes do not impact any other tests yet.
2021-02-05 13:34:55 -05:00
James Bardin e01d37d0dc Block.AttribuuteByPath
There are a few places where we want to perform some transformation on a
cty.Value, but require information from the schema. Rather than create
bespoke functions to walk the cty.Value and schema in concert, we can
provide Attribute information from a cty.Path allowing the use of
Value.Transform in these cases.
2021-01-12 16:31:18 -05:00
James Bardin aeb96384fd configs/configload: staticcheck 2020-12-02 13:59:19 -05:00
James Bardin 9c3ece40f7 configs/configschema: staticcheck 2020-12-02 13:59:19 -05:00
James Bardin 82305bf1c8 configs: staticcheck 2020-12-02 13:59:19 -05:00
Pam Selle c6ab9b1553
Merge pull request #26938 from hashicorp/pselle/remove-vendor-provisioners
Remove vendor provisioners
2020-12-02 11:48:40 -05:00
Martin Atkins eed6b3c48b configs: Emit a warning for backends in nested modules
Terraform considers backend configurations only in the root module, so any
declarations in child modules are entirely ignored.

To avoid users mistakenly thinking that a root module backend
configuration has taken effect, we'll now emit a warning about it. This is
a warning rather than an error because it's reasonable to call a module
that would normally be a root module instead as a child module when
writing a wrapper module to handle integration testing.
2020-11-18 08:20:25 -08:00
Martin Atkins eac85b506b configs: Specialized warning for single-interpolation object keys
We have an existing warning message to encourage moving away from the old
0.11-and-earlier style of redundantly wrapping standalone expressions in
templates, but due to the special rules for object keys the warning
message was giving misleading advice in that context: a user following the
advice as given would then encounter an error about the object key being
ambiguous.

To account for that, this introduces a special alternative version of the
warning just for that particular position, directing the user to replace
the template interpolation markers with parenthesis instead. That will
then get the same result as the former interpolation sequence, rather than
producing the ambiguity error.
2020-11-18 08:01:05 -08:00
Pam Selle e39e0e3d04 Remove vendor provisioners and add fmt Make target
Remove chef, habitat, puppet, and salt-masterless provsioners,
which follows their deprecation. Update the documentatin for these
provisioners to clarify that they have been removed from later versions
of Terraform. Adds the fmt Make target back and updates fmtcheck script
for correctness.
2020-11-17 11:22:03 -05:00
Alisdair McDiarmid 45671a354d configs: Fix provider lookup local name mismatch
When a resource has no `provider` argument specified, its provider is
derived from the implied provider type based on the resource type. For
example, a `boop_instance` resource has an implied provider local name
of `boop`. Correspondingly, its provider configuration is specified with
a `provider "boop"` block.

However, users can use the `required_providers` configuration to give a
different local name to a given provider than its defined type. For
example, a provider may be published at `foobar/beep`, but provide
resources such as `boop_instance`. The most convenient way to use this
provider is with a `required_providers` map:

terraform {
  required_providers {
    boop = {
      source = "foobar/beep"
    }
  }
}

Once that local name is defined, it is used for provider configuration
(a `provider "boop"` block, not `provider "beep"`). It should also be
used when looking up a resource's provider configuration or provider.

This commit fixes a bug with this edge case, where previously we were
looking up the local provider configuration block using the resource's
assigned provider type. Instead, if no provider argument is specified,
we should be using the implied provider type, as that is what binds the
resource to the local provider configuration.
2020-11-10 15:25:02 -05:00
James Bardin e27ecba6e4 extended cache comments 2020-10-14 12:45:06 -04:00
James Bardin bb76c3b50c
Update configs/configschema/decoder_spec.go
Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-10-14 10:33:44 -04:00
James Bardin d40e2fb8d1 cache DecoderSpec calls
DecoderSpec may be called many times, and deeply recursive calls are
expensive. Since we cannot synchronize the Blocks themselves due to them
being copied in parts of the code, we use a separate cache to store the
generated Specs.
2020-10-14 09:19:26 -04:00
Martin Atkins 0bbbb9c64b configs: Experimental support for optional object type attributes
This builds on an experimental feature in the underlying cty library which
allows marking specific attribtues of an object type constraint as
optional, which in turn modifies how the cty conversion package handles
missing attributes in a source value: it will silently substitute a null
value of the appropriate type rather than returning an error.

In order to implement the experiment this commit temporarily forks the
HCL typeexpr extension package into a local internal/typeexpr package,
where I've extended the type constraint syntax to allow annotating object
type attributes as being optional using the HCL function call syntax.
If the experiment is successful -- both at the Terraform layer and in
the underlying cty library -- we'll likely send these modifications to
upstream HCL so that other HCL-based languages can potentially benefit
from this new capability.

Because it's experimental, the optional attribute modifier is allowed only
with an explicit opt-in to the module_variable_optional_attrs experiment.
2020-10-12 10:12:28 -07:00
Pam Selle c1957fcaf0 Add JSON test case for variable sensitive flag 2020-10-08 13:13:32 -04:00
Pam Selle bc57c20d10 Remove sensitive_variables experiment
Ahead of the beta, remove the sensitive_variable experiment
and update tests accordingly
2020-10-08 11:22:20 -04:00
Kristin Laemmert 3933cbd491
remove LegacyProvider (#26433) 2020-10-05 08:33:49 -04:00
Martin Atkins 593cf7b4d5 didyoumean: move from "helper" to "internal"
This new-ish package ended up under "helper" during the 0.12 cycle for
want of some other place to put it, but in retrospect that was an odd
choice because the "helper/" tree is otherwise a bunch of legacy code from
when the SDK lived in this repository.

Here we move it over into the "internal" directory just to distance it
from the guidance of not using "helper/" packages in new projects;
didyoumean is a package we actively use as part of error message hints.
2020-10-02 13:35:07 -07:00
Pam Selle e0e6f4fd13
Merge pull request #26443 from hashicorp/pselle/sensitive-var-module-merge
Implement module merge for sensitive variable config
2020-10-01 10:40:47 -04:00
Pam Selle f2fe0ceb0a Implement module merge for sensitive config
Implements merging behavior for when sensitive
is set on a variable and adds testing accordingly
2020-09-30 17:25:54 -04:00
Pam Selle ada6f45871 Add test file for bad sensitive value
Adds a test file for an incorrectly typed
(non-boolean) sensitive value to ensure it errors
2020-09-30 13:58:01 -04:00
Alisdair McDiarmid 13ed5af5c8 configs: Deprecate nested redundant interpolations
Previous deprecations only included direct assignment of template-only
expressions to arguments. That is, this was not deprecated:

locals {
  foo = ["${var.foo}"]
}

This commit uses hclsyntax.VisitAll to detect and show deprecations for
all template-only expressions, no matter how deep they are in a given
expression.
2020-09-22 13:40:08 -04:00
Alisdair McDiarmid 18f9ea53b9 command: Providers schema shows required_providers
The providers schema command is using the Config.ProviderTypes method,
which had not been kept up to date with the changes to provider
requirements detection made in Config.ProviderRequirements. This
resulted in any currently-unused providers being omitted from the
output.

This commit changes the ProviderTypes method to use the same underlying
logic as ProviderRequirements, which ensures that `required_providers`
blocks are taken into account.

Includes an integration test case to verify that this fixes the provider
schemas command bug.
2020-09-22 10:28:32 -04:00