Commit Graph

250 Commits

Author SHA1 Message Date
Kristin Laemmert 923e157b5c
configs: deprecate version argument inside provider configuration blocks (#26135)
The version argument is deprecated in Terraform v0.14 in favor of
required_providers and will be removed in a future version of terraform
(expected to be v0.15). The provider configuration documentation already
discourages use of 'version' inside provider configuration blocks, so it
only needed an extra note that it is actively deprecated.
2020-09-08 08:19:00 -04:00
Martin Atkins b0da5b1ce5 core: Remove the last few HIL remnants
We've not been using HIL in the main codepaths since Terraform 0.12, but
some references to it (and some supporting functionality in Terraform)
stuck around due to interactions with types we'd kept around to support
legacy shims.

However, removing the configs.RawConfig field from
terraform.ResourceConfig disconnects that subtree of dependencies from
everything else, allowing us to remove it. This is safe because the only
remaining uses of terraform.ResourceConfig are shims from values that
were already evaluated using the HCL 2 API, and thus they never need
the "just in time" HIL evaluation that ResourceConfig.interpolateForce
used to do.

We also had some HIL references in configs/hcl2shim that were previously
in support of the "terraform 0.12upgrade" command, but the implementation
of that command is now removed.

There was one remaining reference to HIL in a now-unused function in the
helper/schema package, which I removed entirely here.

This then allows us to remove the HIL dependency entirely, and also to
clean up some remaining old remants of the legacy "config" package that
we'd recently moved into the "configs" package pending further pruning.
2020-09-02 15:53:33 -07:00
Pam Selle 73cba31602
Merge pull request #25605 from davidair/patch-1
Tiny typo in an error message
2020-08-28 12:38:55 -04:00
Kristin Laemmert 23a8bdd522
configs: finish deprecation of the config package by removing the remaining used functions into configs (#25996) 2020-08-26 14:39:18 -04:00
Kristin Laemmert df4627fecb
configs: include "providers" when overriding modules (#25496) 2020-08-11 11:46:40 -04:00
davidair 5cde8b999b
Tiny typo in an error message
Changing "An module" to "A module" in an error message
2020-07-17 11:09:45 -04:00
Kristin Laemmert 9cb8456f3d
configs: prevent panic with invalid type name (#25562)
An invalid type name in a resource (or data source) could cause a panic
when determining the implied provider for the resource. This commit adds
verification that the type name is valid. It does not add a diagnostic,
since the invalid type name would have already been caught by the
parser.

Fixes #25560
2020-07-13 09:47:16 -04:00
Martin Atkins 0a46ded3e7 addrs: Disallow provider source addresses starting with terraform-
The main motivation here is to produce a helpful error if a user
incorrectly uses the terraform-provider- prefix (which we see on provider
VCS repositories and plugin executables) as part of the source address.

However, this also more broadly blocks "terraform-" as a prefix in
anticipation of whatever instinct causes the phenomenon where e.g.
Python's PyPI has thousands of packages whose names start with "python-",
even though everything on PyPI is for Python by definition. This is
definitely not _necessary_, but it's better to be restrictive at first
and weaken later as needed.
2020-07-08 10:18:55 -07:00
Alisdair McDiarmid 5e3d2dbdfa configs: Fail early for invalid resource provider
If a resource's "provider" reference is invalid and cannot be parsed, we
should not store the reference as part of a `ProviderConfigRef`. Doing
so creates an invalid data structure, which prevents us from using
`MustParseProviderPart` with the name in later steps.

The invalid test files added in this commit will cause a panic without
the code change.
2020-06-26 09:47:58 -04:00
Alisdair McDiarmid 9a9f4e2696 configs: Fix provider requirements panics
When parsing provider requirements we should check the type of the
source and version attributes rather than assuming that they are
strings. Otherwise an invalid attribute value will cause a panic.
2020-06-24 10:12:29 -04:00
James Bardin 3f22bbf8d5 don't allow providers in modules using depends_on
Providers themselves don't support depends_on, and therefor a module
with providers cannot use depends_on.
2020-06-23 09:56:00 -04:00
Alisdair McDiarmid 45f7da9678 configs: Fix nested provider requirements bug
In a recent PR, we changed the provider requirements code to permit
per-module requirements gathering, to enhance the provider command
output. This had an incorrect implementation of recursive requirements
gathering for the normal case, which resulted in only depth-1 modules
being inspected.

This commit fixes the broken recursion and adds a grandchild module to
the unit tests as test coverage. This also demanded fixing the
testNestedModuleConfigFromDir helper function to cope with nested
modules in test configs.
2020-06-22 12:16:22 -04:00
Pam Selle 199157a51a
Validation for provider blocks in expanding modules (nested) (#25248)
* Refactor provider validation into separate func & recurse

Refactors the validate provider functions into a separate function
that can recursively search above a module to check and see if
any parents of the module contain count/for_each configs to be
considered
2020-06-16 13:52:41 -04:00
Alisdair McDiarmid 08b735984a
Merge pull request #25191 from hashicorp/alisdair/better-provider-upgrade-hints-on-init
command/init: Improve diags for legacy providers
2020-06-12 12:31:33 -04:00
Martin Atkins 7ab914491b configs: Don't panic if new version constraint parser raises an error
The new provider installer code is using a new version constraint parser
because it produces better error messages than the one we were using
before. However, it has some cases where it returns errors that the old
parser (which was entirely regex-match-based) didn't catch.

In the long run we should consistently use the new parser everywhere, but
until then we'll avoid panicking then the two disagree, by returning
diagnostic messages instead of using MustParseVersionConstraints.

For now, we only hit these error cases if the user enters something that
the old parser allows but the new parser does not.
2020-06-12 08:45:14 -07:00
Alisdair McDiarmid 9263b28e99 command/init: Improve diags for legacy providers
When initializing a configuration which refers to re-namespaced legacy
providers, we attempt to detect this and display a diagnostic message.
Previously this message would direct the user to run the 0.13upgrade
command, but without specifying in which directories.

This commit detects which modules are using the providers in question,
and for local modules displays a list of upgrade commands which specify
the source directories of these modules.

For remote modules, we display a separate list noting that they need to
be upgraded elsewhere, providing both the local module call name and the
module source address.
2020-06-12 09:57:01 -04:00
Alisdair McDiarmid 9204498fe5 configs/configload: Remove unused SourceAddr funcs
isLocalSourceAddr and isRegistrySourceAddr are also present in
internal/initwd/getter.go, and they are unused in this package.
2020-06-10 16:26:04 -04:00
Alisdair McDiarmid 1c1e4a4de0 command/providers: Show provider requirements tree
Providers can be required from multiple sources. The previous
implementation of the providers sub-command displayed only a flat list
of provider requirements, which made it difficult to see which modules
required each provider.

This commit reintroduces the tree display of provider requirements, and
adds a separate output block for providers required by existing state.
2020-06-09 14:21:53 -04:00
samir elsharkawy 5dfc266da9 config/name_values: fix index out of range in looksLikeSentences 2020-06-05 13:29:43 +02:00
Kristin Laemmert 6fbd3942ea configs: fix panic with provider aliases
addProviderRequirements() was incorrectly using the map keys from the module
provider configs when looking up the provider FQN. The map keys include
alias, so this resulted in a panic. Update addProviderRequirements() to
use the provider's name (only) when looking up the FQN.
2020-06-02 10:55:31 -04:00
James Bardin 8ba63110ec
Merge pull request #25005 from hashicorp/jbardin/module-depends-on
Module depends_on
2020-05-28 21:29:04 -04:00
Martin Atkins d1bc412220 configs: Custom variable validation is no longer experimental
All of the feedback from the experiment described enhancements that can
potentially be added later without breaking changes, so this change simply
removes the experiment gate from the feature as originally implemented
with no changes to its functionality.

Further enhancements may follow in later releases, but the goal of this
change is just to ship the feature exactly as it was under the experiment.

Most of the changes here are cleaning up the experiment opt-ins from our
test cases. The most important parts are in configs/experiments.go and in
experiments/experiment.go .
2020-05-28 16:07:59 -07:00
James Bardin 1b8fb4083a allow depends_on in module call 2020-05-20 13:46:13 -04:00
Kristin Laemmert 041f4dd8ca
configs: require normalized provider local names (#24945)
* addrs: replace NewLegacyProvider with NewDefaultProvider in ParseProviderSourceString

ParseProviderSourceString was still defaulting to NewLegacyProvider when
encountering single-part strings. This has been fixed.

This commit also adds a new function, IsProviderPartNormalized, which
returns a bool indicating if the string given is the same as a
normalized version (as normalized by ParseProviderPart) or an error.
This is intended for use by the configs package when decoding provider
configurations.

* terraform: fix provider local names in tests

* configs: validate that all provider names are normalized

The addrs package normalizes all source strings, but not the local
names. This caused very odd behavior if for e.g. a provider local name
was capitalized in one place and not another. We considered enabling
case-sensitivity for provider local names, but decided that since this
was not something that worked in previous versions of terraform (and we
have yet to encounter any use cases for this feature) we could generate
an error if the provider local name is not normalized. This error also
provides instructions on how to fix it.

* configs: refactor decodeProviderRequirements to consistently not set an FQN when there are errors
2020-05-14 09:00:58 -04:00
Pam Selle f82700bc56
Disallow provider configuration in expanding modules (#24892)
Validate providers in expanding modules. Expanding modules cannot have provider configurations with non-empty configs, which includes having a version configured. If an empty or alias-only block is passed, the provider must be passed through the providers argument on the module call
2020-05-08 11:35:28 -04:00
Alisdair McDiarmid ae98bd12a7 command: Rework 0.13upgrade sub-command
This commit implements most of the intended functionality of the upgrade
command for rewriting configurations.

For a given module, it makes a list of all providers in use. Then it
attempts to detect the source address for providers without an explicit
source.

Once this step is complete, the tool rewrites the relevant configuration
files. This results in a single "required_providers" block for the
module, with a source for each provider.

Any providers for which the source cannot be detected (for example,
unofficial providers) will need a source to be defined by the user. The
tool writes an explanatory comment to the configuration to help with
this.
2020-05-07 11:38:55 -04:00
Alisdair McDiarmid dcb8b45e0f configs: Fix for resources with implied providers
Previously, resources without explicit provider configuration (i.e. a
`provider =` attribute) would be assigned a default provider based upon
the resource type. For example, a resource `foo_bar` would be assigned
provider `hashicorp/foo`.

This behaviour did not work well with community or partner providers,
with sources configured in `terraform.required_providers` blocks. With
the following configuration:

    terraform {
      required_providers {
        foo = {
          source = "acme/foo"
        }
      }
    }

    resource foo_bar "a" { }

the resource would be configured with the `hashicorp/foo` provider.

This commit fixes this implied provider behaviour. First we look for a
provider with local name matching the resource type in the module's
required providers map. If one is found, this provider is assigned to
the resource. Otherwise, we still fall back to a default provider.
2020-04-28 14:54:31 -04:00
Alisdair McDiarmid 7ca7b1f0fe configs: Simplify required_providers blocks
We now permit at most one `required_providers` block per module (except
for overrides). This prevents users (and Terraform) from struggling to
understand how to merge multiple `required_providers` configurations,
with `version` and `source` attributes split across multiple blocks.

Because only one `required_providers` block is permitted, there is no
need to concatenate version constraints and resolve them. This allows us
to simplify the structs used to represent provider requirements,
aligning more closely with other structs in this package.

This commit also fixes a semantic use-before-initialize bug, where
resources defined before a `required_providers` block would be unable to
use its source attribute. We achieve this by processing the module's
`required_providers` configuration (and overrides) before resources.

Overrides for `required_providers` work as before, replacing the entire
block per provider.
2020-04-24 13:44:08 -04:00
Kristin Laemmert 269d511481 command/providers: refactor with new provider types and functions
The providers command has been refactored to use the modern provider types and
ProviderRequirements() functions. This resulted in a breaking change to
the output: it no longer outputs the providers by module and no longer
prints `(inherited)` or `(from state)` to show why a provider is
included. We decided that at this time it was best to stick with the
existing functions and make this change, but if we get feedback from the
community we will revisit.

Additional tests to exercise providers in modules and providers from
state have been included.
2020-04-10 15:08:10 -04:00
Kristin Laemmert 0a5fb40fdf configs: include provider configs in ProviderRequirements()
This PR adds iteration through any provider configuration blocks in the
config in addProviderRequirements().

A stale comment (of mine!) would leave one expecting the
module.ProviderRequirements to include any requirements from provider
configs. The comment was inaccurate and has been updated.
2020-04-10 15:08:10 -04:00
James Bardin 939d045a0b don't allow count && for_each 2020-04-07 14:18:08 -04:00
Martin Atkins 3e3d8f6764 configs: Hint for a misplaced top-level required_providers block
With provider dependencies now appearing inside a nested block, it seems
likely that configuration examples showing dependencies out of context
will sometimes mislead users into thinking that required_providers is
toplevel.

To give better feedback in that situation, we'll produce a specialized
error in that case hinting the correct structure to the user.
2020-04-06 09:24:23 -07:00
Martin Atkins 7caf0b9246 addrs: ImpliedProviderForUnqualifiedType function
This encapsulates the logic for selecting an implied FQN for an
unqualified type name, which could either come from a local name used in
a module without specifying an explicit source for it or from the prefix
of a resource type on a resource that doesn't explicitly set "provider".

This replaces the previous behavior of just directly calling
NewDefaultProvider everywhere so that we can use a different implication
for the local name "terraform", to refer to the built-in terraform
provider rather than the stale one that's on registry.terraform.io for
compatibility with other Terraform versions.
2020-04-06 09:24:23 -07:00
Kristin Laemmert e683a6adef Mildwonkey/terraform tests (targeting integration branch) (#24513)
* configs: remove `Legacy*` Provider functions, switch to default
* terraform context test updates
2020-04-06 09:24:23 -07:00
Martin Atkins 4061cbed38 internal/getproviders: A new shared model for provider requirements
We've been using the models from the "moduledeps" package to represent our
provider dependencies everywhere since the idea of provider dependencies
was introduced in Terraform 0.10, but that model is not convenient to use
for any use-case other than the "terraform providers" command that needs
individual-module-level detail.

To make things easier for new codepaths working with the new-style
provider installer, here we introduce a new model type
getproviders.Requirements which is based on the type the new installer was
already taking as its input. We have new methods in the states, configs,
and earlyconfig packages to produce values of this type, and a helper
to merge Requirements together so we can combine config-derived and
state-derived requirements together during installation.

The advantage of this new model over the moduledeps one is that all of
recursive module walking is done up front and we produce a simple, flat
structure that is more convenient for the main use-cases of selecting
providers for installation and then finding providers in the local cache
to use them for other operations.

This new model is _not_ suitable for implementing "terraform providers"
because it does not retain module-specific requirement details. Therefore
we will likely keep using moduledeps for "terraform providers" for now,
and then possibly at a later time consider specializing the moduledeps
logic for only what "terraform providers" needs, because it seems to be
the only use-case that needs to retain that level of detail.
2020-03-27 09:01:32 -07:00
James Bardin cd045f6f4e enable count and for_each in configuration 2020-03-25 17:03:06 -04:00
Kristin Laemmert 5b427ec648
addrs: Provider.IsLegacy and IsDefault functions
* add IsLegacy and IsDefault funcs to addrs.Provider
* add some test coverage
2020-03-23 09:50:35 -04:00
Kristin Laemmert 5f313a65ad
command: remove 0.12upgrade (#24403)
* command: remove 0.12upgrade and related `configupgrade` library
* leave deprecation warning for 0.12upgrade to point users to v0.12
2020-03-19 08:01:16 -04:00
Kristin Laemmert ed1aebbeda
terraform: large refactor to use Provider from configs.Resource (#24396)
* terraform: large refactor to use Provider from configs.Resource

configs.Resource.ImpliedProvider() now returns a string; it is the
callers' responsibility to turn that into an addrs.Provider if needed.

GraphNodeProviderConsumer ProvidedBy() no longer returns nil (reverting
to earlier, pre-provider-fqn behavior): it will return either the
provider set in config, provider set in state, or the default provider.
2020-03-18 08:58:20 -04:00
Kristin Laemmert ef19fb6203
configs: attach provider fqn to Resource (#24382)
* configs: attach provider fqn to Resource
2020-03-16 14:36:16 -04:00
Martin Atkins 946eda3f3c configs: Return diagnostics (almost) directly from ParseProviderSourceString
This function can already produce suitable diagnostic messages which we'd
like to preserve, but it cannot produce source location information, and
so we'll amend the diagnostics to include that on the way out while
retaining all of the other values in the diagnostics.
2020-03-12 11:11:29 -07:00
Kristin Laemmert 1c78b26012
terraform: provider source test (#24342)
* configs: parse provider source string during module merge

This was the smallest unit of work needed to start writing provider
source tests!

* Update configs/parser_test.go

Co-Authored-By: Alisdair McDiarmid <alisdair@users.noreply.github.com>
2020-03-12 12:00:00 -04:00
James Bardin 8497adcb6e AbsProviderConfig to use addrs.Module
Change ModuleInstance to Module in AbsProviderConfig, because providers
need to be handled before module expansion, and should not be used
defined inside an expanded module at all.

Renaming of the addrs type can happen later, when there's less work
in-flight around provider configuration.
2020-03-10 20:25:44 -04:00
Kristin Laemmert c7cc0afb80
Mildwonkey/ps schema (#24312)
* add Config to AttachSchemaTransformer for providerFqn lookup
* terraform: refactor ProvidedBy() to return nil when provider is not set
in config or state
2020-03-10 14:43:57 -04:00
Paddy e6592dc710
Add support for provider metadata to modules. (#22583)
Implement a new provider_meta block in the terraform block of modules, allowing provider-keyed metadata to be communicated from HCL to provider binaries.

Bundled in this change for minimal protocol version bumping is the addition of markdown support for attribute descriptions and the ability to indicate when an attribute is deprecated, so this information can be shown in the schema dump.

Co-authored-by: Paul Tyng <paul@paultyng.net>
2020-03-05 16:53:24 -08:00
Martin Atkins 86f0b5191c addrs: Stronger validation and normalization of provider namespace/type
The provider FQN is becoming our primary identifier for a provider, so
it's important that we are clear about the equality rules for these
addresses and what characters are valid within them.

We previously had a basic regex permitting ASCII letters and digits for
validation and no normalization at all. We need to do at least case
folding and UTF-8 normalization because these names will appear in file
and directory names in case-insensitive filesystems and in repository
names such as on GitHub.

Since we're already using DNS-style normalization and validation rules
for the hostname part, rather than defining an entirely new set of rules
here we'll just treat the provider namespace and type as if they were
single labels in a DNS name. Aside from some internal consistency, that
also works out nicely because systems like GitHub use organization and
repository names as part of hostnames (e.g. with GitHub Pages) and so
tend to apply comparable constraints themselves.

This introduces the possibility of names containing letters from alphabets
other than the latin alphabet, and for latin letters with diacritics.
That's consistent with our introduction of similar support for identifiers
in the language in Terraform 0.12, and is intended to be more friendly to
Terraform users throughout the world that might prefer to name their
products using a different alphabet. This is also a further justification
for using the DNS normalization rules: modern companies tend to choose
product names that make good domain names, and now such names will be
usable as Terraform provider names too.
2020-02-18 15:42:09 -08:00
Kristin Laemmert 6e2618d9be terraform: ProviderTransform gets provider fqn from module
Added configs.Module.ProviderForLocalProviderConfig which allows
terraform.ProviderTransformer to get the provider FQN from the module,
instead of assuming NewLegacyProvider.
2020-02-14 15:41:31 -08:00
Kristin Laemmert 72ec579807
configs/decodeRequiredProviders test: before comparing slices (#24114) 2020-02-14 08:26:14 -05:00
James Bardin 37e60cd883 fix comment text 2020-02-13 20:46:48 -05:00
James Bardin 84d1f5c688 convert destroy provisioner warnings to errors 2020-02-13 15:42:10 -05:00