Commit Graph

272 Commits

Author SHA1 Message Date
James Bardin e27ecba6e4 extended cache comments 2020-10-14 12:45:06 -04:00
James Bardin bb76c3b50c
Update configs/configschema/decoder_spec.go
Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-10-14 10:33:44 -04:00
James Bardin d40e2fb8d1 cache DecoderSpec calls
DecoderSpec may be called many times, and deeply recursive calls are
expensive. Since we cannot synchronize the Blocks themselves due to them
being copied in parts of the code, we use a separate cache to store the
generated Specs.
2020-10-14 09:19:26 -04:00
Martin Atkins 0bbbb9c64b configs: Experimental support for optional object type attributes
This builds on an experimental feature in the underlying cty library which
allows marking specific attribtues of an object type constraint as
optional, which in turn modifies how the cty conversion package handles
missing attributes in a source value: it will silently substitute a null
value of the appropriate type rather than returning an error.

In order to implement the experiment this commit temporarily forks the
HCL typeexpr extension package into a local internal/typeexpr package,
where I've extended the type constraint syntax to allow annotating object
type attributes as being optional using the HCL function call syntax.
If the experiment is successful -- both at the Terraform layer and in
the underlying cty library -- we'll likely send these modifications to
upstream HCL so that other HCL-based languages can potentially benefit
from this new capability.

Because it's experimental, the optional attribute modifier is allowed only
with an explicit opt-in to the module_variable_optional_attrs experiment.
2020-10-12 10:12:28 -07:00
Pam Selle c1957fcaf0 Add JSON test case for variable sensitive flag 2020-10-08 13:13:32 -04:00
Pam Selle bc57c20d10 Remove sensitive_variables experiment
Ahead of the beta, remove the sensitive_variable experiment
and update tests accordingly
2020-10-08 11:22:20 -04:00
Kristin Laemmert 3933cbd491
remove LegacyProvider (#26433) 2020-10-05 08:33:49 -04:00
Martin Atkins 593cf7b4d5 didyoumean: move from "helper" to "internal"
This new-ish package ended up under "helper" during the 0.12 cycle for
want of some other place to put it, but in retrospect that was an odd
choice because the "helper/" tree is otherwise a bunch of legacy code from
when the SDK lived in this repository.

Here we move it over into the "internal" directory just to distance it
from the guidance of not using "helper/" packages in new projects;
didyoumean is a package we actively use as part of error message hints.
2020-10-02 13:35:07 -07:00
Pam Selle e0e6f4fd13
Merge pull request #26443 from hashicorp/pselle/sensitive-var-module-merge
Implement module merge for sensitive variable config
2020-10-01 10:40:47 -04:00
Pam Selle f2fe0ceb0a Implement module merge for sensitive config
Implements merging behavior for when sensitive
is set on a variable and adds testing accordingly
2020-09-30 17:25:54 -04:00
Pam Selle ada6f45871 Add test file for bad sensitive value
Adds a test file for an incorrectly typed
(non-boolean) sensitive value to ensure it errors
2020-09-30 13:58:01 -04:00
Alisdair McDiarmid 13ed5af5c8 configs: Deprecate nested redundant interpolations
Previous deprecations only included direct assignment of template-only
expressions to arguments. That is, this was not deprecated:

locals {
  foo = ["${var.foo}"]
}

This commit uses hclsyntax.VisitAll to detect and show deprecations for
all template-only expressions, no matter how deep they are in a given
expression.
2020-09-22 13:40:08 -04:00
Alisdair McDiarmid 18f9ea53b9 command: Providers schema shows required_providers
The providers schema command is using the Config.ProviderTypes method,
which had not been kept up to date with the changes to provider
requirements detection made in Config.ProviderRequirements. This
resulted in any currently-unused providers being omitted from the
output.

This commit changes the ProviderTypes method to use the same underlying
logic as ProviderRequirements, which ensures that `required_providers`
blocks are taken into account.

Includes an integration test case to verify that this fixes the provider
schemas command bug.
2020-09-22 10:28:32 -04:00
Alisdair McDiarmid 803c95e552
Merge pull request #26105 from hashicorp/alisdair/more-interpolation-only-expression-deprecations
configs: More interpolation-only expr deprecations
2020-09-16 16:06:45 -04:00
Pam Selle 81a6cde3cf
Merge pull request #26203 from hashicorp/pselle/deprecate-vendor-provisioners
Deprecate vendor (3rd party) provisioners
2020-09-15 13:52:10 -04:00
Pam Selle 6a126df0c6
Merge pull request #26183 from hashicorp/pselle/sensitive-values
Add sensitive attribute to variables
2020-09-11 11:24:18 -04:00
Pam Selle 02c1bddfe1 Create experiment for sensitive attribute 2020-09-10 11:04:17 -04:00
Pam Selle 7fef1db20d Add sensitive variable configs test coverage 2020-09-10 11:04:17 -04:00
Pam Selle 862ddf73e2 Add a sensitive attribute 2020-09-10 11:02:54 -04:00
Alisdair McDiarmid 898b459a03 configs: Error on invalid required_providers attrs
A few users have recently been confused about the purpose of the
required_providers objects, adding provider configuration parameters in
addition to version and source. This previously did not cause an error
so would result in a confusingly distant failure.

This commit adds a single diagnostic for any required_providers object
which includes attributes other than version or source.
2020-09-09 11:52:47 -04:00
Kristin Laemmert 923e157b5c
configs: deprecate version argument inside provider configuration blocks (#26135)
The version argument is deprecated in Terraform v0.14 in favor of
required_providers and will be removed in a future version of terraform
(expected to be v0.15). The provider configuration documentation already
discourages use of 'version' inside provider configuration blocks, so it
only needed an extra note that it is actively deprecated.
2020-09-08 08:19:00 -04:00
Martin Atkins b0da5b1ce5 core: Remove the last few HIL remnants
We've not been using HIL in the main codepaths since Terraform 0.12, but
some references to it (and some supporting functionality in Terraform)
stuck around due to interactions with types we'd kept around to support
legacy shims.

However, removing the configs.RawConfig field from
terraform.ResourceConfig disconnects that subtree of dependencies from
everything else, allowing us to remove it. This is safe because the only
remaining uses of terraform.ResourceConfig are shims from values that
were already evaluated using the HCL 2 API, and thus they never need
the "just in time" HIL evaluation that ResourceConfig.interpolateForce
used to do.

We also had some HIL references in configs/hcl2shim that were previously
in support of the "terraform 0.12upgrade" command, but the implementation
of that command is now removed.

There was one remaining reference to HIL in a now-unused function in the
helper/schema package, which I removed entirely here.

This then allows us to remove the HIL dependency entirely, and also to
clean up some remaining old remants of the legacy "config" package that
we'd recently moved into the "configs" package pending further pruning.
2020-09-02 15:53:33 -07:00
Alisdair McDiarmid e693c14e5a configs: More interpolation-only expr deprecations
Extend the deprecation for interpolation-only expressions to include
module calls, data sources, outputs, and locals.
2020-09-02 16:36:47 -04:00
Pam Selle 73cba31602
Merge pull request #25605 from davidair/patch-1
Tiny typo in an error message
2020-08-28 12:38:55 -04:00
Kristin Laemmert 23a8bdd522
configs: finish deprecation of the config package by removing the remaining used functions into configs (#25996) 2020-08-26 14:39:18 -04:00
Pam Selle edc670d079 Add deprecation warning for vendor provisioners
Adds a warning for chef, habitat, puppet, and salt-masterless
provisioners, and a corresponding test file to test for the warning
2020-08-26 10:46:04 -04:00
Kristin Laemmert df4627fecb
configs: include "providers" when overriding modules (#25496) 2020-08-11 11:46:40 -04:00
davidair 5cde8b999b
Tiny typo in an error message
Changing "An module" to "A module" in an error message
2020-07-17 11:09:45 -04:00
Kristin Laemmert 9cb8456f3d
configs: prevent panic with invalid type name (#25562)
An invalid type name in a resource (or data source) could cause a panic
when determining the implied provider for the resource. This commit adds
verification that the type name is valid. It does not add a diagnostic,
since the invalid type name would have already been caught by the
parser.

Fixes #25560
2020-07-13 09:47:16 -04:00
Martin Atkins 0a46ded3e7 addrs: Disallow provider source addresses starting with terraform-
The main motivation here is to produce a helpful error if a user
incorrectly uses the terraform-provider- prefix (which we see on provider
VCS repositories and plugin executables) as part of the source address.

However, this also more broadly blocks "terraform-" as a prefix in
anticipation of whatever instinct causes the phenomenon where e.g.
Python's PyPI has thousands of packages whose names start with "python-",
even though everything on PyPI is for Python by definition. This is
definitely not _necessary_, but it's better to be restrictive at first
and weaken later as needed.
2020-07-08 10:18:55 -07:00
Alisdair McDiarmid 5e3d2dbdfa configs: Fail early for invalid resource provider
If a resource's "provider" reference is invalid and cannot be parsed, we
should not store the reference as part of a `ProviderConfigRef`. Doing
so creates an invalid data structure, which prevents us from using
`MustParseProviderPart` with the name in later steps.

The invalid test files added in this commit will cause a panic without
the code change.
2020-06-26 09:47:58 -04:00
Alisdair McDiarmid 9a9f4e2696 configs: Fix provider requirements panics
When parsing provider requirements we should check the type of the
source and version attributes rather than assuming that they are
strings. Otherwise an invalid attribute value will cause a panic.
2020-06-24 10:12:29 -04:00
James Bardin 3f22bbf8d5 don't allow providers in modules using depends_on
Providers themselves don't support depends_on, and therefor a module
with providers cannot use depends_on.
2020-06-23 09:56:00 -04:00
Alisdair McDiarmid 45f7da9678 configs: Fix nested provider requirements bug
In a recent PR, we changed the provider requirements code to permit
per-module requirements gathering, to enhance the provider command
output. This had an incorrect implementation of recursive requirements
gathering for the normal case, which resulted in only depth-1 modules
being inspected.

This commit fixes the broken recursion and adds a grandchild module to
the unit tests as test coverage. This also demanded fixing the
testNestedModuleConfigFromDir helper function to cope with nested
modules in test configs.
2020-06-22 12:16:22 -04:00
Pam Selle 199157a51a
Validation for provider blocks in expanding modules (nested) (#25248)
* Refactor provider validation into separate func & recurse

Refactors the validate provider functions into a separate function
that can recursively search above a module to check and see if
any parents of the module contain count/for_each configs to be
considered
2020-06-16 13:52:41 -04:00
Alisdair McDiarmid 08b735984a
Merge pull request #25191 from hashicorp/alisdair/better-provider-upgrade-hints-on-init
command/init: Improve diags for legacy providers
2020-06-12 12:31:33 -04:00
Martin Atkins 7ab914491b configs: Don't panic if new version constraint parser raises an error
The new provider installer code is using a new version constraint parser
because it produces better error messages than the one we were using
before. However, it has some cases where it returns errors that the old
parser (which was entirely regex-match-based) didn't catch.

In the long run we should consistently use the new parser everywhere, but
until then we'll avoid panicking then the two disagree, by returning
diagnostic messages instead of using MustParseVersionConstraints.

For now, we only hit these error cases if the user enters something that
the old parser allows but the new parser does not.
2020-06-12 08:45:14 -07:00
Alisdair McDiarmid 9263b28e99 command/init: Improve diags for legacy providers
When initializing a configuration which refers to re-namespaced legacy
providers, we attempt to detect this and display a diagnostic message.
Previously this message would direct the user to run the 0.13upgrade
command, but without specifying in which directories.

This commit detects which modules are using the providers in question,
and for local modules displays a list of upgrade commands which specify
the source directories of these modules.

For remote modules, we display a separate list noting that they need to
be upgraded elsewhere, providing both the local module call name and the
module source address.
2020-06-12 09:57:01 -04:00
Alisdair McDiarmid 9204498fe5 configs/configload: Remove unused SourceAddr funcs
isLocalSourceAddr and isRegistrySourceAddr are also present in
internal/initwd/getter.go, and they are unused in this package.
2020-06-10 16:26:04 -04:00
Alisdair McDiarmid 1c1e4a4de0 command/providers: Show provider requirements tree
Providers can be required from multiple sources. The previous
implementation of the providers sub-command displayed only a flat list
of provider requirements, which made it difficult to see which modules
required each provider.

This commit reintroduces the tree display of provider requirements, and
adds a separate output block for providers required by existing state.
2020-06-09 14:21:53 -04:00
samir elsharkawy 5dfc266da9 config/name_values: fix index out of range in looksLikeSentences 2020-06-05 13:29:43 +02:00
Kristin Laemmert 6fbd3942ea configs: fix panic with provider aliases
addProviderRequirements() was incorrectly using the map keys from the module
provider configs when looking up the provider FQN. The map keys include
alias, so this resulted in a panic. Update addProviderRequirements() to
use the provider's name (only) when looking up the FQN.
2020-06-02 10:55:31 -04:00
James Bardin 8ba63110ec
Merge pull request #25005 from hashicorp/jbardin/module-depends-on
Module depends_on
2020-05-28 21:29:04 -04:00
Martin Atkins d1bc412220 configs: Custom variable validation is no longer experimental
All of the feedback from the experiment described enhancements that can
potentially be added later without breaking changes, so this change simply
removes the experiment gate from the feature as originally implemented
with no changes to its functionality.

Further enhancements may follow in later releases, but the goal of this
change is just to ship the feature exactly as it was under the experiment.

Most of the changes here are cleaning up the experiment opt-ins from our
test cases. The most important parts are in configs/experiments.go and in
experiments/experiment.go .
2020-05-28 16:07:59 -07:00
James Bardin 1b8fb4083a allow depends_on in module call 2020-05-20 13:46:13 -04:00
Kristin Laemmert 041f4dd8ca
configs: require normalized provider local names (#24945)
* addrs: replace NewLegacyProvider with NewDefaultProvider in ParseProviderSourceString

ParseProviderSourceString was still defaulting to NewLegacyProvider when
encountering single-part strings. This has been fixed.

This commit also adds a new function, IsProviderPartNormalized, which
returns a bool indicating if the string given is the same as a
normalized version (as normalized by ParseProviderPart) or an error.
This is intended for use by the configs package when decoding provider
configurations.

* terraform: fix provider local names in tests

* configs: validate that all provider names are normalized

The addrs package normalizes all source strings, but not the local
names. This caused very odd behavior if for e.g. a provider local name
was capitalized in one place and not another. We considered enabling
case-sensitivity for provider local names, but decided that since this
was not something that worked in previous versions of terraform (and we
have yet to encounter any use cases for this feature) we could generate
an error if the provider local name is not normalized. This error also
provides instructions on how to fix it.

* configs: refactor decodeProviderRequirements to consistently not set an FQN when there are errors
2020-05-14 09:00:58 -04:00
Pam Selle f82700bc56
Disallow provider configuration in expanding modules (#24892)
Validate providers in expanding modules. Expanding modules cannot have provider configurations with non-empty configs, which includes having a version configured. If an empty or alias-only block is passed, the provider must be passed through the providers argument on the module call
2020-05-08 11:35:28 -04:00
Alisdair McDiarmid ae98bd12a7 command: Rework 0.13upgrade sub-command
This commit implements most of the intended functionality of the upgrade
command for rewriting configurations.

For a given module, it makes a list of all providers in use. Then it
attempts to detect the source address for providers without an explicit
source.

Once this step is complete, the tool rewrites the relevant configuration
files. This results in a single "required_providers" block for the
module, with a source for each provider.

Any providers for which the source cannot be detected (for example,
unofficial providers) will need a source to be defined by the user. The
tool writes an explanatory comment to the configuration to help with
this.
2020-05-07 11:38:55 -04:00
Alisdair McDiarmid dcb8b45e0f configs: Fix for resources with implied providers
Previously, resources without explicit provider configuration (i.e. a
`provider =` attribute) would be assigned a default provider based upon
the resource type. For example, a resource `foo_bar` would be assigned
provider `hashicorp/foo`.

This behaviour did not work well with community or partner providers,
with sources configured in `terraform.required_providers` blocks. With
the following configuration:

    terraform {
      required_providers {
        foo = {
          source = "acme/foo"
        }
      }
    }

    resource foo_bar "a" { }

the resource would be configured with the `hashicorp/foo` provider.

This commit fixes this implied provider behaviour. First we look for a
provider with local name matching the resource type in the module's
required providers map. If one is found, this provider is assigned to
the resource. Otherwise, we still fall back to a default provider.
2020-04-28 14:54:31 -04:00
Alisdair McDiarmid 7ca7b1f0fe configs: Simplify required_providers blocks
We now permit at most one `required_providers` block per module (except
for overrides). This prevents users (and Terraform) from struggling to
understand how to merge multiple `required_providers` configurations,
with `version` and `source` attributes split across multiple blocks.

Because only one `required_providers` block is permitted, there is no
need to concatenate version constraints and resolve them. This allows us
to simplify the structs used to represent provider requirements,
aligning more closely with other structs in this package.

This commit also fixes a semantic use-before-initialize bug, where
resources defined before a `required_providers` block would be unable to
use its source attribute. We achieve this by processing the module's
`required_providers` configuration (and overrides) before resources.

Overrides for `required_providers` work as before, replacing the entire
block per provider.
2020-04-24 13:44:08 -04:00