Commit Graph

61 Commits

Author SHA1 Message Date
James Bardin d0cc7f1d5e resolve provider types when building the config
All the information is available to resolve provider types when building
the configuration, but some provider references still had no FQN. This
caused validation to assume a default type, and incorrectly reject valid
module calls with non-default namespaced providers.

Resolve as much provider type information as possible when loading the
config. Only use this internally for now, but this should be useful
outside of the package to avoid re-resolving the providers later on. We
can come back and find where this might be useful elsewhere, but for now
keep the change as small as possible to avoid any changes in behavior.
2021-04-16 12:37:50 -04:00
James Bardin 759b76436a validate provider_meta contains no interpolations
The provider_meta specification does not allow interpolation, but we
were not preventing it in the configuration.
2021-03-12 12:10:25 -05:00
Martin Atkins b5adc33075 configs: Accept and minimally validate a "language" argument
We expect that in order to continue to evolve the language without
breaking existing modules we will at some point need to have a way to mark
when a particular module is expecting a newer interpretation of the
language.

Although it's too early to do any deep preparation for that, this commit
aims to proactively reserve an argument named "language" inside
"terraform" blocks, which currently only accepts the keyword TF2021 that
is intended to represent "the edition of the Terraform language as defined
in 2021".

That argument also defaults to TF2021 if not set, so in practice there's
no real reason to set this today, but this minimal validation today is
intended to give better feedback to users of older Terraform versions in
the event that we introduce a new language edition later and they try to
use an module incompatible with their Terraform version.
2021-02-26 11:43:32 -08:00
Pam Selle 8f7807684a Upgrade to quoted keywords to error
The warning about deprecation is upgraded to an error
2021-02-21 20:27:20 -05:00
Pam Selle 61bee3cd44
Merge pull request #27834 from hashicorp/pselle/ignore-changes-wildcard
Upgrade ignore_changes wildcard from warning to error
2021-02-19 16:16:39 -05:00
Pam Selle fa7c3d7e10 Remove interpolation-only warning
These interpolations are removed when upgrading using 0.12upgrade,
and are removed in terraform fmt in many cases
2021-02-19 10:59:09 -05:00
Pam Selle 230658f2b2 Upgrade ignore_changes wildcard from warning to error
The syntax ignore_changes = ["*"] was deprecated and now errors.
Use = all instead.
2021-02-19 10:34:20 -05:00
James Bardin 4195531925 old comment 2021-02-11 17:39:51 -05:00
James Bardin da252de1a0 add provider validation
Add validation which was removed from the configload package, along with
additional validation checks. The output is slightly different, as
instead of validating whether the modules are allowed to have provider
configurations, we validate the various combinations of provider
structures themselves.
2021-02-11 10:22:18 -05:00
James Bardin ac585be079 initial support for parsing configuration_aliases
Add support for parsing configuration_aliases in required_providers
entries. The decoder needed to be re-written here in order to support
the bare reference style usage of provider names so that they match the
usage in other location within configuration. The only change to
existing handling of the required_providers block is more precise error
locations in a couple cases.
2021-02-09 08:38:30 -05:00
Pam Selle c6ab9b1553
Merge pull request #26938 from hashicorp/pselle/remove-vendor-provisioners
Remove vendor provisioners
2020-12-02 11:48:40 -05:00
Martin Atkins eed6b3c48b configs: Emit a warning for backends in nested modules
Terraform considers backend configurations only in the root module, so any
declarations in child modules are entirely ignored.

To avoid users mistakenly thinking that a root module backend
configuration has taken effect, we'll now emit a warning about it. This is
a warning rather than an error because it's reasonable to call a module
that would normally be a root module instead as a child module when
writing a wrapper module to handle integration testing.
2020-11-18 08:20:25 -08:00
Pam Selle e39e0e3d04 Remove vendor provisioners and add fmt Make target
Remove chef, habitat, puppet, and salt-masterless provsioners,
which follows their deprecation. Update the documentatin for these
provisioners to clarify that they have been removed from later versions
of Terraform. Adds the fmt Make target back and updates fmtcheck script
for correctness.
2020-11-17 11:22:03 -05:00
Martin Atkins 0bbbb9c64b configs: Experimental support for optional object type attributes
This builds on an experimental feature in the underlying cty library which
allows marking specific attribtues of an object type constraint as
optional, which in turn modifies how the cty conversion package handles
missing attributes in a source value: it will silently substitute a null
value of the appropriate type rather than returning an error.

In order to implement the experiment this commit temporarily forks the
HCL typeexpr extension package into a local internal/typeexpr package,
where I've extended the type constraint syntax to allow annotating object
type attributes as being optional using the HCL function call syntax.
If the experiment is successful -- both at the Terraform layer and in
the underlying cty library -- we'll likely send these modifications to
upstream HCL so that other HCL-based languages can potentially benefit
from this new capability.

Because it's experimental, the optional attribute modifier is allowed only
with an explicit opt-in to the module_variable_optional_attrs experiment.
2020-10-12 10:12:28 -07:00
Pam Selle c1957fcaf0 Add JSON test case for variable sensitive flag 2020-10-08 13:13:32 -04:00
Pam Selle bc57c20d10 Remove sensitive_variables experiment
Ahead of the beta, remove the sensitive_variable experiment
and update tests accordingly
2020-10-08 11:22:20 -04:00
Pam Selle e0e6f4fd13
Merge pull request #26443 from hashicorp/pselle/sensitive-var-module-merge
Implement module merge for sensitive variable config
2020-10-01 10:40:47 -04:00
Pam Selle f2fe0ceb0a Implement module merge for sensitive config
Implements merging behavior for when sensitive
is set on a variable and adds testing accordingly
2020-09-30 17:25:54 -04:00
Pam Selle ada6f45871 Add test file for bad sensitive value
Adds a test file for an incorrectly typed
(non-boolean) sensitive value to ensure it errors
2020-09-30 13:58:01 -04:00
Alisdair McDiarmid 13ed5af5c8 configs: Deprecate nested redundant interpolations
Previous deprecations only included direct assignment of template-only
expressions to arguments. That is, this was not deprecated:

locals {
  foo = ["${var.foo}"]
}

This commit uses hclsyntax.VisitAll to detect and show deprecations for
all template-only expressions, no matter how deep they are in a given
expression.
2020-09-22 13:40:08 -04:00
Alisdair McDiarmid 18f9ea53b9 command: Providers schema shows required_providers
The providers schema command is using the Config.ProviderTypes method,
which had not been kept up to date with the changes to provider
requirements detection made in Config.ProviderRequirements. This
resulted in any currently-unused providers being omitted from the
output.

This commit changes the ProviderTypes method to use the same underlying
logic as ProviderRequirements, which ensures that `required_providers`
blocks are taken into account.

Includes an integration test case to verify that this fixes the provider
schemas command bug.
2020-09-22 10:28:32 -04:00
Alisdair McDiarmid 803c95e552
Merge pull request #26105 from hashicorp/alisdair/more-interpolation-only-expression-deprecations
configs: More interpolation-only expr deprecations
2020-09-16 16:06:45 -04:00
Pam Selle 81a6cde3cf
Merge pull request #26203 from hashicorp/pselle/deprecate-vendor-provisioners
Deprecate vendor (3rd party) provisioners
2020-09-15 13:52:10 -04:00
Pam Selle 02c1bddfe1 Create experiment for sensitive attribute 2020-09-10 11:04:17 -04:00
Pam Selle 7fef1db20d Add sensitive variable configs test coverage 2020-09-10 11:04:17 -04:00
Kristin Laemmert 923e157b5c
configs: deprecate version argument inside provider configuration blocks (#26135)
The version argument is deprecated in Terraform v0.14 in favor of
required_providers and will be removed in a future version of terraform
(expected to be v0.15). The provider configuration documentation already
discourages use of 'version' inside provider configuration blocks, so it
only needed an extra note that it is actively deprecated.
2020-09-08 08:19:00 -04:00
Alisdair McDiarmid e693c14e5a configs: More interpolation-only expr deprecations
Extend the deprecation for interpolation-only expressions to include
module calls, data sources, outputs, and locals.
2020-09-02 16:36:47 -04:00
Pam Selle edc670d079 Add deprecation warning for vendor provisioners
Adds a warning for chef, habitat, puppet, and salt-masterless
provisioners, and a corresponding test file to test for the warning
2020-08-26 10:46:04 -04:00
Kristin Laemmert df4627fecb
configs: include "providers" when overriding modules (#25496) 2020-08-11 11:46:40 -04:00
Kristin Laemmert 9cb8456f3d
configs: prevent panic with invalid type name (#25562)
An invalid type name in a resource (or data source) could cause a panic
when determining the implied provider for the resource. This commit adds
verification that the type name is valid. It does not add a diagnostic,
since the invalid type name would have already been caught by the
parser.

Fixes #25560
2020-07-13 09:47:16 -04:00
Martin Atkins 0a46ded3e7 addrs: Disallow provider source addresses starting with terraform-
The main motivation here is to produce a helpful error if a user
incorrectly uses the terraform-provider- prefix (which we see on provider
VCS repositories and plugin executables) as part of the source address.

However, this also more broadly blocks "terraform-" as a prefix in
anticipation of whatever instinct causes the phenomenon where e.g.
Python's PyPI has thousands of packages whose names start with "python-",
even though everything on PyPI is for Python by definition. This is
definitely not _necessary_, but it's better to be restrictive at first
and weaken later as needed.
2020-07-08 10:18:55 -07:00
Alisdair McDiarmid 5e3d2dbdfa configs: Fail early for invalid resource provider
If a resource's "provider" reference is invalid and cannot be parsed, we
should not store the reference as part of a `ProviderConfigRef`. Doing
so creates an invalid data structure, which prevents us from using
`MustParseProviderPart` with the name in later steps.

The invalid test files added in this commit will cause a panic without
the code change.
2020-06-26 09:47:58 -04:00
Alisdair McDiarmid 45f7da9678 configs: Fix nested provider requirements bug
In a recent PR, we changed the provider requirements code to permit
per-module requirements gathering, to enhance the provider command
output. This had an incorrect implementation of recursive requirements
gathering for the normal case, which resulted in only depth-1 modules
being inspected.

This commit fixes the broken recursion and adds a grandchild module to
the unit tests as test coverage. This also demanded fixing the
testNestedModuleConfigFromDir helper function to cope with nested
modules in test configs.
2020-06-22 12:16:22 -04:00
Alisdair McDiarmid 1c1e4a4de0 command/providers: Show provider requirements tree
Providers can be required from multiple sources. The previous
implementation of the providers sub-command displayed only a flat list
of provider requirements, which made it difficult to see which modules
required each provider.

This commit reintroduces the tree display of provider requirements, and
adds a separate output block for providers required by existing state.
2020-06-09 14:21:53 -04:00
Martin Atkins d1bc412220 configs: Custom variable validation is no longer experimental
All of the feedback from the experiment described enhancements that can
potentially be added later without breaking changes, so this change simply
removes the experiment gate from the feature as originally implemented
with no changes to its functionality.

Further enhancements may follow in later releases, but the goal of this
change is just to ship the feature exactly as it was under the experiment.

Most of the changes here are cleaning up the experiment opt-ins from our
test cases. The most important parts are in configs/experiments.go and in
experiments/experiment.go .
2020-05-28 16:07:59 -07:00
Kristin Laemmert 041f4dd8ca
configs: require normalized provider local names (#24945)
* addrs: replace NewLegacyProvider with NewDefaultProvider in ParseProviderSourceString

ParseProviderSourceString was still defaulting to NewLegacyProvider when
encountering single-part strings. This has been fixed.

This commit also adds a new function, IsProviderPartNormalized, which
returns a bool indicating if the string given is the same as a
normalized version (as normalized by ParseProviderPart) or an error.
This is intended for use by the configs package when decoding provider
configurations.

* terraform: fix provider local names in tests

* configs: validate that all provider names are normalized

The addrs package normalizes all source strings, but not the local
names. This caused very odd behavior if for e.g. a provider local name
was capitalized in one place and not another. We considered enabling
case-sensitivity for provider local names, but decided that since this
was not something that worked in previous versions of terraform (and we
have yet to encounter any use cases for this feature) we could generate
an error if the provider local name is not normalized. This error also
provides instructions on how to fix it.

* configs: refactor decodeProviderRequirements to consistently not set an FQN when there are errors
2020-05-14 09:00:58 -04:00
Alisdair McDiarmid dcb8b45e0f configs: Fix for resources with implied providers
Previously, resources without explicit provider configuration (i.e. a
`provider =` attribute) would be assigned a default provider based upon
the resource type. For example, a resource `foo_bar` would be assigned
provider `hashicorp/foo`.

This behaviour did not work well with community or partner providers,
with sources configured in `terraform.required_providers` blocks. With
the following configuration:

    terraform {
      required_providers {
        foo = {
          source = "acme/foo"
        }
      }
    }

    resource foo_bar "a" { }

the resource would be configured with the `hashicorp/foo` provider.

This commit fixes this implied provider behaviour. First we look for a
provider with local name matching the resource type in the module's
required providers map. If one is found, this provider is assigned to
the resource. Otherwise, we still fall back to a default provider.
2020-04-28 14:54:31 -04:00
Alisdair McDiarmid 7ca7b1f0fe configs: Simplify required_providers blocks
We now permit at most one `required_providers` block per module (except
for overrides). This prevents users (and Terraform) from struggling to
understand how to merge multiple `required_providers` configurations,
with `version` and `source` attributes split across multiple blocks.

Because only one `required_providers` block is permitted, there is no
need to concatenate version constraints and resolve them. This allows us
to simplify the structs used to represent provider requirements,
aligning more closely with other structs in this package.

This commit also fixes a semantic use-before-initialize bug, where
resources defined before a `required_providers` block would be unable to
use its source attribute. We achieve this by processing the module's
`required_providers` configuration (and overrides) before resources.

Overrides for `required_providers` work as before, replacing the entire
block per provider.
2020-04-24 13:44:08 -04:00
Kristin Laemmert 269d511481 command/providers: refactor with new provider types and functions
The providers command has been refactored to use the modern provider types and
ProviderRequirements() functions. This resulted in a breaking change to
the output: it no longer outputs the providers by module and no longer
prints `(inherited)` or `(from state)` to show why a provider is
included. We decided that at this time it was best to stick with the
existing functions and make this change, but if we get feedback from the
community we will revisit.

Additional tests to exercise providers in modules and providers from
state have been included.
2020-04-10 15:08:10 -04:00
Martin Atkins 3e3d8f6764 configs: Hint for a misplaced top-level required_providers block
With provider dependencies now appearing inside a nested block, it seems
likely that configuration examples showing dependencies out of context
will sometimes mislead users into thinking that required_providers is
toplevel.

To give better feedback in that situation, we'll produce a specialized
error in that case hinting the correct structure to the user.
2020-04-06 09:24:23 -07:00
Martin Atkins 7caf0b9246 addrs: ImpliedProviderForUnqualifiedType function
This encapsulates the logic for selecting an implied FQN for an
unqualified type name, which could either come from a local name used in
a module without specifying an explicit source for it or from the prefix
of a resource type on a resource that doesn't explicitly set "provider".

This replaces the previous behavior of just directly calling
NewDefaultProvider everywhere so that we can use a different implication
for the local name "terraform", to refer to the built-in terraform
provider rather than the stale one that's on registry.terraform.io for
compatibility with other Terraform versions.
2020-04-06 09:24:23 -07:00
Martin Atkins 4061cbed38 internal/getproviders: A new shared model for provider requirements
We've been using the models from the "moduledeps" package to represent our
provider dependencies everywhere since the idea of provider dependencies
was introduced in Terraform 0.10, but that model is not convenient to use
for any use-case other than the "terraform providers" command that needs
individual-module-level detail.

To make things easier for new codepaths working with the new-style
provider installer, here we introduce a new model type
getproviders.Requirements which is based on the type the new installer was
already taking as its input. We have new methods in the states, configs,
and earlyconfig packages to produce values of this type, and a helper
to merge Requirements together so we can combine config-derived and
state-derived requirements together during installation.

The advantage of this new model over the moduledeps one is that all of
recursive module walking is done up front and we produce a simple, flat
structure that is more convenient for the main use-cases of selecting
providers for installation and then finding providers in the local cache
to use them for other operations.

This new model is _not_ suitable for implementing "terraform providers"
because it does not retain module-specific requirement details. Therefore
we will likely keep using moduledeps for "terraform providers" for now,
and then possibly at a later time consider specializing the moduledeps
logic for only what "terraform providers" needs, because it seems to be
the only use-case that needs to retain that level of detail.
2020-03-27 09:01:32 -07:00
Kristin Laemmert ed1aebbeda
terraform: large refactor to use Provider from configs.Resource (#24396)
* terraform: large refactor to use Provider from configs.Resource

configs.Resource.ImpliedProvider() now returns a string; it is the
callers' responsibility to turn that into an addrs.Provider if needed.

GraphNodeProviderConsumer ProvidedBy() no longer returns nil (reverting
to earlier, pre-provider-fqn behavior): it will return either the
provider set in config, provider set in state, or the default provider.
2020-03-18 08:58:20 -04:00
Kristin Laemmert ef19fb6203
configs: attach provider fqn to Resource (#24382)
* configs: attach provider fqn to Resource
2020-03-16 14:36:16 -04:00
Kristin Laemmert 1c78b26012
terraform: provider source test (#24342)
* configs: parse provider source string during module merge

This was the smallest unit of work needed to start writing provider
source tests!

* Update configs/parser_test.go

Co-Authored-By: Alisdair McDiarmid <alisdair@users.noreply.github.com>
2020-03-12 12:00:00 -04:00
Martin Atkins 86f0b5191c addrs: Stronger validation and normalization of provider namespace/type
The provider FQN is becoming our primary identifier for a provider, so
it's important that we are clear about the equality rules for these
addresses and what characters are valid within them.

We previously had a basic regex permitting ASCII letters and digits for
validation and no normalization at all. We need to do at least case
folding and UTF-8 normalization because these names will appear in file
and directory names in case-insensitive filesystems and in repository
names such as on GitHub.

Since we're already using DNS-style normalization and validation rules
for the hostname part, rather than defining an entirely new set of rules
here we'll just treat the provider namespace and type as if they were
single labels in a DNS name. Aside from some internal consistency, that
also works out nicely because systems like GitHub use organization and
repository names as part of hostnames (e.g. with GitHub Pages) and so
tend to apply comparable constraints themselves.

This introduces the possibility of names containing letters from alphabets
other than the latin alphabet, and for latin letters with diacritics.
That's consistent with our introduction of similar support for identifiers
in the language in Terraform 0.12, and is intended to be more friendly to
Terraform users throughout the world that might prefer to name their
products using a different alphabet. This is also a further justification
for using the DNS normalization rules: modern companies tend to choose
product names that make good domain names, and now such names will be
usable as Terraform provider names too.
2020-02-18 15:42:09 -08:00
James Bardin 84d1f5c688 convert destroy provisioner warnings to errors 2020-02-13 15:42:10 -05:00
Martin Atkins 8b511524d6
Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978)
* Introduce "Local" terminology for non-absolute provider config addresses

In a future change AbsProviderConfig and LocalProviderConfig are going to
become two entirely distinct types, rather than Abs embedding Local as
written here. This naming change is in preparation for that subsequent
work, which will also include introducing a new "ProviderConfig" type
that is an interface that AbsProviderConfig and LocalProviderConfig both
implement.

This is intended to be largely just a naming change to get started, so
we can deal with all of the messy renaming. However, this did also require
a slight change in modeling where the Resource.DefaultProviderConfig
method has become Resource.DefaultProvider returning a Provider address
directly, because this method doesn't have enough information to construct
a true and accurate LocalProviderConfig -- it would need to refer to the
configuration to know what this module is calling the provider it has
selected.

In order to leave a trail to follow for subsequent work, all of the
changes here are intended to ensure that remaining work will become
obvious via compile-time errors when all of the following changes happen:
- The concept of "legacy" provider addresses is removed from the addrs
  package, including removing addrs.NewLegacyProvider and
  addrs.Provider.LegacyString.
- addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded
  in it and has an addrs.Provider and a string alias directly instead.
- The provider-schema-handling parts of Terraform core are updated to
  work with addrs.Provider to identify providers, rather than legacy
  strings.

In particular, there are still several codepaths here making legacy
provider address assumptions (in order to limit the scope of this change)
but I've made sure each one is doing something that relies on at least
one of the above changes not having been made yet.

* addrs: ProviderConfig interface

In a (very) few special situations in the main "terraform" package we need
to make runtime decisions about whether a provider config is absolute
or local.

We currently do that by exploiting the fact that AbsProviderConfig has
LocalProviderConfig nested inside of it and so in the local case we can
just ignore the wrapping AbsProviderConfig and use the embedded value.

In a future change we'll be moving away from that embedding and making
these two types distinct in order to represent that mapping between them
requires consulting a lookup table in the configuration, and so here we
introduce a new interface type ProviderConfig that can represent either
AbsProviderConfig or LocalProviderConfig decided dynamically at runtime.

This also includes the Config.ResolveAbsProviderAddr method that will
eventually be responsible for that local-to-absolute translation, so
that callers with access to the configuration can normalize to an
addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's
currently unused because existing callers are still relying on the
simplistic structural transform, but we'll switch them over in a later
commit.

* rename LocalType to LocalName

Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-01-31 08:23:07 -05:00
James Bardin 35107acc46 allow "terraform" in destroy provisioners
The values supplied in terraform are static, and do not create any
dependencies.
2020-01-15 11:39:01 -05:00
James Bardin c376905adc it's safe for destroy provisioners to access path
The path values are statically loaded, and do not create any
dependencies that could cause problems with destroy provisioners.
2020-01-13 16:44:44 -05:00