Commit Graph

35 Commits

Author SHA1 Message Date
Martin Atkins 27ad9861ce configs: Meta-argument escaping blocks
Several top-level block types in the Terraform language have a body where
two different schemas are overlayed on top of one another: Terraform first
looks for "meta-arguments" that are built into the language, and then
evaluates all of the remaining arguments against some externally-defined
schema whose content is not fully controlled by Terraform.

So far we've been cautiously adding new meta-arguments in these namespaces
after research shows us that there are relatively few existing providers
or modules that would have functionality masked by those additions, but
that isn't really a viable path forward as we prepare to make stronger
compatibility promises.

In an earlier commit we've introduced the foundational parts of a new
language versioning mechanism called "editions" which should allow us to
make per-module-opt-in breaking changes in the future, but these shared
namespaces remain a liability because it would be annoying if adopting a
new edition made it impossible to use a feature of a third-party provider
or module that was already using a name that has now become reserved in
the new edition.

This commit introduces a new syntax intended to be a rarely-used escape
hatch for that situation. When we're designing new editions we will do our
best to choose names that don't conflict with commonly-used providers and
modules, but there are many providers and modules that we cannot see and
so there is a risk that any name we might choose could collide with at
least one existing provider or module. The automatic migration tool to
upgrade an existing module to a new edition should therefore detect that
situation and make use of this escaping block syntax in order to retain
the existing functionality until all the called providers or modules are
updated to no longer use conflicting names.

Although we can't put in technical constraints on using this feature for
other purposes (because we don't know yet what future editions will add),
this mechanism is intentionally not documented for now because it serves
no immediate purpose. In effect, this change is just squatting on the
syntax of a special block type named "_" so that later editions can make
use of it without it _also_ conflicting, creating a confusing nested
escaping situation. However, the first time a new edition actually makes
use of this syntax we should then document alongside the meta-arguments
so folks can understand the meaning of escaping blocks produced by
edition upgrade tools.
2021-05-17 11:17:25 -07:00
James Bardin d0cc7f1d5e resolve provider types when building the config
All the information is available to resolve provider types when building
the configuration, but some provider references still had no FQN. This
caused validation to assume a default type, and incorrectly reject valid
module calls with non-default namespaced providers.

Resolve as much provider type information as possible when loading the
config. Only use this internally for now, but this should be useful
outside of the package to avoid re-resolving the providers later on. We
can come back and find where this might be useful elsewhere, but for now
keep the change as small as possible to avoid any changes in behavior.
2021-04-16 12:37:50 -04:00
Pam Selle 61bee3cd44
Merge pull request #27834 from hashicorp/pselle/ignore-changes-wildcard
Upgrade ignore_changes wildcard from warning to error
2021-02-19 16:16:39 -05:00
Pam Selle fa7c3d7e10 Remove interpolation-only warning
These interpolations are removed when upgrading using 0.12upgrade,
and are removed in terraform fmt in many cases
2021-02-19 10:59:09 -05:00
Pam Selle 230658f2b2 Upgrade ignore_changes wildcard from warning to error
The syntax ignore_changes = ["*"] was deprecated and now errors.
Use = all instead.
2021-02-19 10:34:20 -05:00
Alisdair McDiarmid 45671a354d configs: Fix provider lookup local name mismatch
When a resource has no `provider` argument specified, its provider is
derived from the implied provider type based on the resource type. For
example, a `boop_instance` resource has an implied provider local name
of `boop`. Correspondingly, its provider configuration is specified with
a `provider "boop"` block.

However, users can use the `required_providers` configuration to give a
different local name to a given provider than its defined type. For
example, a provider may be published at `foobar/beep`, but provide
resources such as `boop_instance`. The most convenient way to use this
provider is with a `required_providers` map:

terraform {
  required_providers {
    boop = {
      source = "foobar/beep"
    }
  }
}

Once that local name is defined, it is used for provider configuration
(a `provider "boop"` block, not `provider "beep"`). It should also be
used when looking up a resource's provider configuration or provider.

This commit fixes a bug with this edge case, where previously we were
looking up the local provider configuration block using the resource's
assigned provider type. Instead, if no provider argument is specified,
we should be using the implied provider type, as that is what binds the
resource to the local provider configuration.
2020-11-10 15:25:02 -05:00
Alisdair McDiarmid e693c14e5a configs: More interpolation-only expr deprecations
Extend the deprecation for interpolation-only expressions to include
module calls, data sources, outputs, and locals.
2020-09-02 16:36:47 -04:00
Alisdair McDiarmid 5e3d2dbdfa configs: Fail early for invalid resource provider
If a resource's "provider" reference is invalid and cannot be parsed, we
should not store the reference as part of a `ProviderConfigRef`. Doing
so creates an invalid data structure, which prevents us from using
`MustParseProviderPart` with the name in later steps.

The invalid test files added in this commit will cause a panic without
the code change.
2020-06-26 09:47:58 -04:00
Kristin Laemmert 041f4dd8ca
configs: require normalized provider local names (#24945)
* addrs: replace NewLegacyProvider with NewDefaultProvider in ParseProviderSourceString

ParseProviderSourceString was still defaulting to NewLegacyProvider when
encountering single-part strings. This has been fixed.

This commit also adds a new function, IsProviderPartNormalized, which
returns a bool indicating if the string given is the same as a
normalized version (as normalized by ParseProviderPart) or an error.
This is intended for use by the configs package when decoding provider
configurations.

* terraform: fix provider local names in tests

* configs: validate that all provider names are normalized

The addrs package normalizes all source strings, but not the local
names. This caused very odd behavior if for e.g. a provider local name
was capitalized in one place and not another. We considered enabling
case-sensitivity for provider local names, but decided that since this
was not something that worked in previous versions of terraform (and we
have yet to encounter any use cases for this feature) we could generate
an error if the provider local name is not normalized. This error also
provides instructions on how to fix it.

* configs: refactor decodeProviderRequirements to consistently not set an FQN when there are errors
2020-05-14 09:00:58 -04:00
Kristin Laemmert ed1aebbeda
terraform: large refactor to use Provider from configs.Resource (#24396)
* terraform: large refactor to use Provider from configs.Resource

configs.Resource.ImpliedProvider() now returns a string; it is the
callers' responsibility to turn that into an addrs.Provider if needed.

GraphNodeProviderConsumer ProvidedBy() no longer returns nil (reverting
to earlier, pre-provider-fqn behavior): it will return either the
provider set in config, provider set in state, or the default provider.
2020-03-18 08:58:20 -04:00
Kristin Laemmert ef19fb6203
configs: attach provider fqn to Resource (#24382)
* configs: attach provider fqn to Resource
2020-03-16 14:36:16 -04:00
Martin Atkins 8b511524d6
Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978)
* Introduce "Local" terminology for non-absolute provider config addresses

In a future change AbsProviderConfig and LocalProviderConfig are going to
become two entirely distinct types, rather than Abs embedding Local as
written here. This naming change is in preparation for that subsequent
work, which will also include introducing a new "ProviderConfig" type
that is an interface that AbsProviderConfig and LocalProviderConfig both
implement.

This is intended to be largely just a naming change to get started, so
we can deal with all of the messy renaming. However, this did also require
a slight change in modeling where the Resource.DefaultProviderConfig
method has become Resource.DefaultProvider returning a Provider address
directly, because this method doesn't have enough information to construct
a true and accurate LocalProviderConfig -- it would need to refer to the
configuration to know what this module is calling the provider it has
selected.

In order to leave a trail to follow for subsequent work, all of the
changes here are intended to ensure that remaining work will become
obvious via compile-time errors when all of the following changes happen:
- The concept of "legacy" provider addresses is removed from the addrs
  package, including removing addrs.NewLegacyProvider and
  addrs.Provider.LegacyString.
- addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded
  in it and has an addrs.Provider and a string alias directly instead.
- The provider-schema-handling parts of Terraform core are updated to
  work with addrs.Provider to identify providers, rather than legacy
  strings.

In particular, there are still several codepaths here making legacy
provider address assumptions (in order to limit the scope of this change)
but I've made sure each one is doing something that relies on at least
one of the above changes not having been made yet.

* addrs: ProviderConfig interface

In a (very) few special situations in the main "terraform" package we need
to make runtime decisions about whether a provider config is absolute
or local.

We currently do that by exploiting the fact that AbsProviderConfig has
LocalProviderConfig nested inside of it and so in the local case we can
just ignore the wrapping AbsProviderConfig and use the embedded value.

In a future change we'll be moving away from that embedding and making
these two types distinct in order to represent that mapping between them
requires consulting a lookup table in the configuration, and so here we
introduce a new interface type ProviderConfig that can represent either
AbsProviderConfig or LocalProviderConfig decided dynamically at runtime.

This also includes the Config.ResolveAbsProviderAddr method that will
eventually be responsible for that local-to-absolute translation, so
that callers with access to the configuration can normalize to an
addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's
currently unused because existing callers are still relying on the
simplistic structural transform, but we'll switch them over in a later
commit.

* rename LocalType to LocalName

Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-01-31 08:23:07 -05:00
Kristin Laemmert 6541775ce4
addrs: roll back change to Type field in ProviderConfig (#23937) 2020-01-28 08:13:30 -05:00
Kristin Laemmert e3416124cc
addrs: replace "Type string" with "Type Provider" in ProviderConfig
* huge change to weave new addrs.Provider into addrs.ProviderConfig
* terraform: do not include an empty string in the returned Providers /
Provisioners
- Fixed a minor bug where results included an extra empty string
2019-12-06 08:00:18 -05:00
James Bardin 8547603ff5 deprecation warning for destroy provisioner refs
Add deprecation warning for references from destroy provisioners or
their connections to external resources or values. In order to ensure
resource destruction can be completed correctly, destroy nodes must be
able to evaluate with only their instance state.

We have sufficient information to validate destroy-time provisioners
early on during the config loading process. Later on these can be
converted to hard errors, and only allow self, count.index, and each.key
in destroy provisioners. Limited the provisioner and block evaluation
scope later on is tricky, but if the references can never be loaded,
then they will never be encountered during evaluation.
2019-12-04 11:14:37 -05:00
Martin Atkins 91752f02da configs: Warn for deprecated interpolation and quoted type constraints
Following on from de652e22a26b, this introduces deprecation warnings for
when an attribute value expression is a template with only a single
interpolation sequence, and for variable type constraints given in quotes.

As with the previous commit, we allowed these deprecated forms with no
warning for a few releases after v0.12.0 to ensure that folks who need to
write cross-compatible modules for a while during upgrading would be able
to do so, but we're now marking these as explicitly deprecated to guide
users towards the new idiomatic forms.

The "terraform 0.12upgrade" tool would've already updated configurations
to not hit these warnings for those who had pre-existing configurations
written for Terraform 0.11.

The main target audience for these warnings are newcomers to Terraform who
are learning from existing examples already published in various spots on
the wider internet that may be showing older Terraform syntax, since those
folks will not be running their configurations through the upgrade tool.
These warnings will hopefully guide them towards modern Terraform usage
during their initial experimentation, and thus reduce the chances of
inadvertently adopting the less-readable legacy usage patterns in
greenfield projects.
2019-11-13 07:55:55 -08:00
Martin Atkins 39e609d5fd vendor: switch to HCL 2.0 in the HCL repository
Previously we were using the experimental HCL 2 repository, but now we'll
shift over to the v2 import path within the main HCL repository as part of
actually releasing HCL 2.0 as stable.

This is a mechanical search/replace to the new import paths. It also
switches to the v2.0.0 release of HCL, which includes some new code that
Terraform didn't previously have but should not change any behavior that
matters for Terraform's purposes.

For the moment the experimental HCL2 repository is still an indirect
dependency via terraform-config-inspect, so it remains in our go.sum and
vendor directories for the moment. Because terraform-config-inspect uses
a much smaller subset of the HCL2 functionality, this does still manage
to prune the vendor directory a little. A subsequent release of
terraform-config-inspect should allow us to completely remove that old
repository in a future commit.
2019-10-02 15:10:21 -07:00
Thayne McCombs 7c678d104f Add support for for_each for data blocks.
This also fixes a few things with resource for_each:

It makes validation more like validation for count.

It makes sure the index is stored in the state properly.
2019-07-25 16:59:06 -04:00
Pam Selle 7d905f6777 Resource for_each 2019-07-22 10:51:16 -04:00
Martin Atkins 0681935df5 configs: Reserve various names for future use
We want the forthcoming v0.12.0 release to be the last significant
breaking change to our main configuration constructs for a long time, but
not everything could be implemented in that release.

As a compromise then, we reserve various names we have some intent of
using in a future release so that such future uses will not be a further
breaking change later.

Some of these names are associated with specific short-term plans, while
others are reserved conservatively for possible later work and may be
"un-reserved" in a later release if we don't end up using them. The ones
that we expect to use in the near future were already being handled, so
we'll continue to decode them at the config layer but also produce an
error so that we don't get weird behavior downstream where the
corresponding features don't work yet.
2018-11-26 08:25:03 -08:00
James Bardin e0e177374f don't hanlde "type" when parsing connection blocks
There's no provisioner schema yet, and we need to handle the type
dynamically during evaluation.
2018-10-16 18:49:20 -07:00
Martin Atkins c4fac74371 configs: Fix crasher in provider configuration reference parsing
Due to a logic error, this would panic if given an empty traversal.
2018-10-16 18:46:46 -07:00
Martin Atkins cd584309b9 configs: Fix typo in the deprecation warning about ignore_changes = ["*"] 2018-10-16 18:46:46 -07:00
Martin Atkins 5cf791861f configs: Allow looking up resources by resource addresses.
Throughout the main "terraform" package we identify resources using the
address types, and so this helper is useful to make concise transitions
between the address types and the configuration types.

As part of this, we use the address types to produce the keys used in our
resource maps. This has no visible change in behavior since the prior
implementation produced an equal result, but this change ensures that
ResourceByAddr cannot be broken by hypothetical future changes to the
key serialization.
2018-10-16 18:46:46 -07:00
Martin Atkins 24dce0c624 configs: Helper methods to integrate with "addrs" package
Our new "addrs" package gives us some nice representations of various
kinds of "address" within Terraform. To talk to APIs that use these, it's
convenient to be able to easily derive such addresses from the
configuration objects.

These new methods, along with a recasting of the existing
Resource.ProviderConfigKey method to Resource.ProviderConfigAddr, give us
some key integration points to support the configuration graph transforms
in the main "terraform" package.
2018-10-16 18:44:26 -07:00
Martin Atkins b6fdd0446e configs: parse the "providers" map for module calls
This was accidentally missed on the first pass of module call decoding.
As before, this is a map from child provider config address to parent
provider config address, allowing the set of providers to be projected in
arbitrary ways into a child module.
2018-10-16 18:44:26 -07:00
Martin Atkins c07b0a7806 configs: Re-unify the ManagedResource and DataResource types
Initially the intent here was to tease these apart a little more since
they don't really share much behavior in common in core, but in practice
it'll take a lot of refactoring to tease apart these assumptions in core
right now and so we'll keep these things unified at the configuration
layer in the interests of minimizing disruption at the core layer.

The two types are still kept in separate maps to help reinforce the fact
that they are separate concepts with some behaviors in common, rather than
the same concept.
2018-10-16 18:44:26 -07:00
Martin Atkins 4ed06a9227 terraform: HCL2-flavored module dependency resolver
For the moment this is just a lightly-adapted copy of
ModuleTreeDependencies named ConfigTreeDependencies, with the goal that
the two can live concurrently for the moment while not all callers are yet
updated and then we can drop ModuleTreeDependencies and its helper
functions altogether in a later commit.

This can then be used to make "terraform init" and "terraform providers"
work properly with the HCL2-powered configuration loader.
2018-10-16 18:44:26 -07:00
Martin Atkins a26ff56f01 configs: highlight resource name in diags when invalid
Previously we were erroneously highlighting the resource type name instead.
2018-03-08 15:41:19 -08:00
Martin Atkins 4fa8c16ead configs: support ignore_changes wildcards
The initial pass of implementation here missed the special case where
ignore_changes can, in the old parser, be set to ["*"] to ignore changes
to all attributes.

Since that syntax is awkward and non-obvious, our new decoder will instead
expect ignore_changes = all, using HCL2's capability to interpret an
expression as a literal keyword. For compatibility with old configurations
we will still accept the ["*"] form but emit a deprecation warning to
encourage moving to the new form.
2018-02-15 15:56:39 -08:00
Martin Atkins 36fb5b52e7 configs: quoted keywords/references are warnings, not errors
In our new loader we are changing certain values in configuration to be
naked keywords or references rather than quoted strings as before. Since
many of these have been shown in books, tutorials, and our own
documentation we will make the old forms generate deprecation warnings
rather than errors so that newcomers starting from older documentation
can be eased into the new syntax, rather than getting blocked.

This will also avoid creating a hard compatibility wall for reusable
modules that are already published, allowing them to still be used in
spite of these warnings and then fixed when the maintainer is able.
2018-02-15 15:56:39 -08:00
Martin Atkins 7c8efe103e configs: allow overrides files to omit args that primary files can't
Some of the fields in our config structs are either mandatory in primary
files or there is a default value that we apply if absent.

Unfortunately override files impose the additional constraint that we
be allowed to omit required fields (which have presumably already been
set in the primary files) and that we are able to distinguish between a
default value and omitting a value entirely.

Since most of our fields were already acceptable for override files, here
we just add some new fields to deal with the few cases where special
handling is required and a helper function to disable the "Required" flag
on attributes in a given schema.
2018-02-15 15:56:38 -08:00
Martin Atkins 4e5efa498a configs: Parser.LoadConfigDir
This method wraps LoadConfigFile to load all of the .tf and .tf.json files
in a given directory and then bundle them together into a Module object.

This function also deals with the distinction between primary and override
files, first appending together the primary files in lexicographic order
by filename, and then merging in override files in the same order.

The merging behavior is not fully implemented as of this commit, and so
will be expanded in future commits.
2018-02-15 15:56:37 -08:00
Martin Atkins e15ec486bf configs: Parser.LoadConfigFile
This is a first pass of decoding of the main Terraform configuration file
format. It hasn't yet been tested with any real-world configurations, so
it will need to be revised further as we test it more thoroughly.
2018-02-15 15:56:37 -08:00
Martin Atkins 13fa73c63e configs: stub out main configuration structs
These types represent the individual elements within configuration, the
modules a configuration is made of, and the configuration (static module
tree) itself.
2018-02-15 15:56:37 -08:00