Commit Graph

192 Commits

Author SHA1 Message Date
Kristin Laemmert 272cb44d3d
configs: extend module.ProviderRequirements to include the addrs.Provider instead of just version constraints. (#23843)
Renamed file.ProviderRequirements to file.RequiredProviders to match the
name of the block in the configuration. file.RequiredProviders contains
the contents of the file(s); module.ProviderRequirements contains the
parsed and merged provider requirements.

Extended decodeRequiredProvidersBlock to parse the new provider source
syntax (version only, it will ignore any other attributes).

Added some tests; swapped deep.Equal with cmp.Equal in the
terraform/module_dependencies_test.go because deep was not catching
incorrect constraints.
2020-01-13 11:31:47 -05:00
Martin Atkins ff4ea042c2 config: Allow module authors to specify validation rules for variables
The existing "type" argument allows specifying a type constraint that
allows for some basic validation, but often there are more constraints on
a variable value than just its type.

This new feature (requiring an experiment opt-in for now, while we refine
it) allows specifying arbitrary validation rules for any variable which
can then cause custom error messages to be returned when a caller provides
an inappropriate value.

    variable "example" {
      validation {
        condition = var.example != "nope"
        error_message = "Example value must not be \"nope\"."
      }
    }

The core parts of this are designed to do as little new work as possible
when no validations are specified, and thus the main new checking codepath
here can therefore only run when the experiment is enabled in order to
permit having validations.
2020-01-10 15:23:25 -08:00
Kristin Laemmert 18dd1bb4d6
Mildwonkey/tfconfig upgrade (#23670)
* deps: bump terraform-config-inspect library
* configs: parse `version` in new required_providers block

With the latest version of `terraform-config-inspect`, the
required_providers attribute can now be a string or an object with
attributes "source" and "version". This change allows parsing the
version constraint from the new object while ignoring any given source attribute.
2020-01-10 11:54:53 -05:00
Martin Atkins 2b2ac1f6de configs: use local set of go-getter detectors
In an earlier change we switched to defining our own sets of detectors,
getters, etc for go-getter in order to insulate us from upstream changes
to those sets that might otherwise change the user-visible behavior of
Terraform's module installer.

However, we apparently neglected to actually refer to our local set of
detectors, and continued to refer to the upstream set. Here we catch up
with the latest detectors from upstream (taken from the version of
go-getter we currently have vendored) and start using that fixed set.

Currently we are maintaining these custom go-getter sets in two places
due to the configload vs. initwd distinction. That was already true for
goGetterGetters and goGetterDecompressors, and so I've preserved that for
now just to keep this change relatively simple; in later change it would
be nice to factor these "get with go getter" functions out into a shared
location which we can call from both configload and initwd.
2020-01-07 09:38:46 -08:00
Kristin Laemmert 49fc53d1d1
provider source enhancements
* configs: move ProviderConfigCompact[Str] from addrs to configs

The configs package is aware of provider name and type (which are the
same thing today, but expected to be two different things in a future
release), and should be the source of truth for a provider config
address. This is an intermediate step; the next step will change the returned types to something based in the configs package.

* command: rename choosePlugins to chooseProviders to clarify scope of function

* use `Provider.LegacyString()` (instead of `Provider.Type`) consistently
* explicitly create legacy-style provider (continuing from above change)
2019-12-11 08:35:55 -05:00
Martin Atkins b90fb25321 experiments: a mechanism for opt-in experimental language features
Traditionally we've preferred to release new language features in major
releases only, because we can then use the beta cycle to gather feedback
on the feature and learn about any usability challenges or other
situations we didn't consider during our design in time to make those
changes before inclusion in a stable release.

This "experiments" feature is intended to decouple the feedback cycle for
new features from the major release rhythm, and thus allow us to release
new features in minor releases by first releasing them as experimental for
a minor release or two, adjust for any feedback gathered during that
period, and then finally remove the experiment gate and enable the feature
for everyone.

The intended model here is that anything behind an experiment gate is
subject to breaking changes even in patch releases, and so any module
using these experimental features will be broken by a future Terraform
upgrade.

The behavior implemented here is:

- Recognize a new "experiments" setting in the "terraform" block which
  allows module authors to explicitly opt in to experimental features.

  terraform {
    experiments = [resource_for_each]
  }

- Generate a warning whenever loading a module that has experiments
  enabled, to avoid accidentally depending on experimental features and
  thus risking unexpected breakage on next Terraform upgrade.

- We check the enabled experiments against the configuration at module
  load time, which means that experiments are scoped to a particular
  module. Enabling an experiment in one module does not automatically
  enable it in any other module.

This experiments mechanism is itself an experiment, and so I'd like to
use the resource for_each feature to trial it. Because any configuration
using experiments is subject to breaking changes, we are free to adjust
this experiments feature in future releases as we see fit, but once
for_each is shipped without an experiment gate we'll be blocked from
making significant changes to it until the next major release at least.
2019-12-10 09:27:05 -08:00
Kristin Laemmert 67fc4dd5a1 configs: move ProviderConfigCompact[Str] from addrs to configs
The configs package is aware of provider name and type (which are the
same thing today, but expected to be two different things in a future
release), and should be the source of truth for a provider config
address.
2019-12-09 08:30:08 -05:00
James Bardin 2fdf984cce update destroy provisioner warning text
Make it a little more user-oriented
2019-12-06 10:20:23 -05:00
Kristin Laemmert e3416124cc
addrs: replace "Type string" with "Type Provider" in ProviderConfig
* huge change to weave new addrs.Provider into addrs.ProviderConfig
* terraform: do not include an empty string in the returned Providers /
Provisioners
- Fixed a minor bug where results included an extra empty string
2019-12-06 08:00:18 -05:00
James Bardin 6817c844bc
Merge pull request #23559 from hashicorp/jbardin/deprecate-destroy-references
deprecation warning for destroy provisioner refs
2019-12-05 18:07:48 -05:00
Kristin Laemmert 9891d0354a
providers: use addrs.Provider as map keys for provider.Factory (#23548)
* terraform/context: use new addrs.Provider as map key in provider factories
* added NewLegacyProviderType and LegacyString funcs to make it explicit that these are temporary placeholders

This PR introduces a new concept, provider fully-qualified name (FQN), encapsulated by the `addrs.Provider` struct.
2019-12-04 11:30:20 -05:00
James Bardin 8547603ff5 deprecation warning for destroy provisioner refs
Add deprecation warning for references from destroy provisioners or
their connections to external resources or values. In order to ensure
resource destruction can be completed correctly, destroy nodes must be
able to evaluate with only their instance state.

We have sufficient information to validate destroy-time provisioners
early on during the config loading process. Later on these can be
converted to hard errors, and only allow self, count.index, and each.key
in destroy provisioners. Limited the provisioner and block evaluation
scope later on is tricky, but if the references can never be loaded,
then they will never be encountered during evaluation.
2019-12-04 11:14:37 -05:00
James Bardin 6caa5d23e2 fix diagnostics handling
Located all non-test paths where a Diagnostic type was assigned to an
error variable.
2019-11-21 09:14:50 -05:00
Kristin Laemmert 2c78414724
configs/configupgrade: do not panic when int value is out of range (#23394)
`terraform 0.12upgrade` assumes that the configuration has passed 0.11
init, but did not explicitly check that the configuration was valid.
Certain issues would not get caught because the configuration was
syntactically valid. In this case, int or float values out of range
resulted in a panic from `Value()`.

Since running a 0.11 validate command is a breaking change, this PR
merely moves the `Value()` logic for ints and floats into `configupgrade` so
the error can be returned to the user, instead of causing a panic.
2019-11-15 11:02:59 -05:00
Martin Atkins 91752f02da configs: Warn for deprecated interpolation and quoted type constraints
Following on from de652e22a26b, this introduces deprecation warnings for
when an attribute value expression is a template with only a single
interpolation sequence, and for variable type constraints given in quotes.

As with the previous commit, we allowed these deprecated forms with no
warning for a few releases after v0.12.0 to ensure that folks who need to
write cross-compatible modules for a while during upgrading would be able
to do so, but we're now marking these as explicitly deprecated to guide
users towards the new idiomatic forms.

The "terraform 0.12upgrade" tool would've already updated configurations
to not hit these warnings for those who had pre-existing configurations
written for Terraform 0.11.

The main target audience for these warnings are newcomers to Terraform who
are learning from existing examples already published in various spots on
the wider internet that may be showing older Terraform syntax, since those
folks will not be running their configurations through the upgrade tool.
These warnings will hopefully guide them towards modern Terraform usage
during their initial experimentation, and thus reduce the chances of
inadvertently adopting the less-readable legacy usage patterns in
greenfield projects.
2019-11-13 07:55:55 -08:00
Calle Pettersson 73d574913d configs: Mention leading underscores in invalid identifier message 2019-11-11 16:22:33 -08:00
Martin Atkins 79dc808614 configs: Emit warnings for deprecated quoted references/keywords
Terraform 0.12.0 removed the need for putting references and keywords
in quotes, but we disabled the deprecation warnings for the initial
release in order to avoid creating noise for folks who were intentionally
attempting to maintain modules that were cross-compatible with both
Terraform 0.11 and Terraform 0.12.

However, with Terraform 0.12 now more widely used, the lack of these
warnings seems to be causing newcomers to copy the quoted versions from
existing examples on the internet, which is perpetuating the old and
confusing quoted form in newer configurations.

In preparation for phasing out these deprecated forms altogether in a
future major release, and for the shorter-term benefit of giving better
feedback to newcomers when they are learning from outdated examples, we'll
now re-enable those deprecation warnings, and be explicit that the old
forms are intended for removal in a future release.

In order to properly test this, we establish a new set of test
configurations that explicitly mark which warnings they are expecting and
verify that they do indeed produce those expected warnings. We also
verify that the "success" tests do _not_ produce warnings, while removing
the ones that were previously written to succeed but have their warnings
ignored.
2019-11-11 10:17:34 -08:00
Pam Selle cca36025d6
Merge pull request #22946 from hashicorp/kmoe/copy_dir_dotfiles
remove dotfile exclusion in copyDir
2019-10-24 12:01:42 -04:00
Radek Simko 7860f55e4f
Version tools per Go convention under tools.go 2019-10-17 22:23:39 +02:00
Radek Simko 32f9722d9d
Replace import paths & set UA string where necessary 2019-10-11 22:40:54 +01:00
Martin Atkins e21f0fa61e backend/local: Handle interactive prompts for variables in UI layer
During the 0.12 work we intended to move all of the variable value
collection logic into the UI layer (command package and backend packages)
and present them all together as a unified data structure to Terraform
Core. However, we didn't quite succeed because the interactive prompts
for unset required variables were still being handled _after_ calling
into Terraform Core.

Here we complete that earlier work by moving the interactive prompts for
variables out into the UI layer too, thus allowing us to handle final
validation of the variables all together in one place and do so in the UI
layer where we have the most context still available about where all of
these values are coming from.

This allows us to fix a problem where previously disabling input with
-input=false on the command line could cause Terraform Core to receive an
incomplete set of variable values, and fail with a bad error message.

As a consequence of this refactoring, the scope of terraform.Context.Input
is now reduced to only gathering provider configuration arguments. Ideally
that too would move into the UI layer somehow in a future commit, but
that's a problem for another day.
2019-10-10 10:07:01 -07:00
James Bardin 1ee851f256
Merge pull request #22846 from hashicorp/jbardin/evaluate-resource
Always evaluate resources in their entirety
2019-10-08 07:57:15 -04:00
Martin Atkins 39e609d5fd vendor: switch to HCL 2.0 in the HCL repository
Previously we were using the experimental HCL 2 repository, but now we'll
shift over to the v2 import path within the main HCL repository as part of
actually releasing HCL 2.0 as stable.

This is a mechanical search/replace to the new import paths. It also
switches to the v2.0.0 release of HCL, which includes some new code that
Terraform didn't previously have but should not change any behavior that
matters for Terraform's purposes.

For the moment the experimental HCL2 repository is still an indirect
dependency via terraform-config-inspect, so it remains in our go.sum and
vendor directories for the moment. Because terraform-config-inspect uses
a much smaller subset of the HCL2 functionality, this does still manage
to prune the vendor directory a little. A subsequent release of
terraform-config-inspect should allow us to completely remove that old
repository in a future commit.
2019-10-02 15:10:21 -07:00
Katy Moe 2c44a7aacb
remove dotfile exclusion in copyDir
copyDir is used in configload/getter.go to copy previously downloaded modules instead of using the go-getter client every time. The go-getter client downloads dotfiles, but copyDir did not copy dotfiles, leading to inconsistent behaviour when reusing the same module source.
2019-09-30 13:11:57 +01:00
James Bardin 86bf674246 change GetResourceInstance to GetResource
In order to allow lazy evaluation of resource indexes, we can't index
resources immediately via GetResourceInstance. Change the evaluation to
always return whole Resources via GetResource, and index individual
instances during expression evaluation.

This will allow us to always check for invalid index errors rather than
returning an unknown value and ignoring it during apply.
2019-09-19 09:19:14 -04:00
James Bardin 13e2e10577 fix Min/Max validation during decoding
We can only validate MinItems >= 1 (equiv to "Required") during
decoding, as dynamic blocks each only decode as a single block. MaxItems
cannot be validated at all, also because of dynamic blocks, which may
have any number of blocks in the config.
2019-08-20 10:13:21 -04:00
James Bardin 731d4226d3 do not validate Min/Max Items in CoerceValue
Due to both the nature of dynamic blocks, and the need for resources to
sometimes communicate incomplete values, we cannot validate MinItems and
MaxItems in CoerceValue.
2019-08-20 10:13:15 -04:00
James Bardin 3e03213485
Merge pull request #22478 from hashicorp/jbardin/coerce-value
don't validate Min/Max block vals in CoerceValue
2019-08-16 15:58:53 -04:00
Daniel Schroeder 628b50f900 configs/configupgrade: typo fix (#22469) 2019-08-16 10:38:38 -04:00
James Bardin 554cedab8a don't validate Min/Max block vals in CoerceValue
A provider may not have the data to fill in required block values in all
cases during the resource Read operation. This is more common in import,
because there is no initial configuration or state, and it's possible
some values are only provided in the configuration.

The original intent of MinItems and MaxItems in the schema was to
enforce configuration constraints, not to enforce what the resource
could save in the state.  Since the configuration is already statically
validated, and the Schema is validated against the configuration in a
separate step, we can drop these extra validation constraints in
CoerceValue and relax it to only ensure the types conform to what is
expected.
2019-08-15 10:02:39 -04:00
Pam Selle 9631e4c73d
Merge pull request #20571 from sergkondr/fix_misspelling
fix misspelling
2019-08-13 17:13:13 -04:00
Alex Pilon 4bf43efcfd
move hcl2shim package to configs 2019-08-06 19:58:58 -04:00
James Bardin 5878527732
Merge pull request #22221 from hashicorp/jbardin/min-items
MinItems with dynamic blocks
2019-07-29 09:45:42 -07:00
James Bardin 67dbd6d345 don't check MinItems with unknowns in blocks
If a block was defined via "dynamic", there will be only one block value
until the expansion is known. Since we can't detect dynamic blocks at
this point, don't verify MinItems while there are unknown values in the
config.

The decoder spec can also only check for existence of a block, so limit
the check to 0 or 1.
2019-07-27 11:50:28 -07:00
Pam Selle 360068b3cb
Merge pull request #21922 from pselle/resource_for_each
Resource for_each
2019-07-26 11:41:56 -04:00
Pam Selle e7d8ac5ad7 Remove panic, update comment 2019-07-26 11:22:10 -04:00
Thayne McCombs 7c678d104f Add support for for_each for data blocks.
This also fixes a few things with resource for_each:

It makes validation more like validation for count.

It makes sure the index is stored in the state properly.
2019-07-25 16:59:06 -04:00
Kristin Laemmert 66f4a48b8c
configs/configupgrade: fix panic on nil hilNode (#22181)
In some cases (see #22020 for a specific example), the parsed hilNode
can be nil. This causes a series of panics. Instead, return an error and
move on.
2019-07-23 13:05:37 -04:00
Pam Selle 7d905f6777 Resource for_each 2019-07-22 10:51:16 -04:00
Alex Pilon 0450f487fa
move IsEmptyDir to configs package 2019-07-18 13:07:10 -04:00
James Bardin 8111050c66 ensure we record diagnostics from nested modules
When loading nested modules, the child module diagnostics were dropped
in the recursive function. This mean that the config from the submodules
wasn't fully loaded, even though no errors were reported to the user.

This caused further problems if the plan was stored in a plan file, when
means only the partial configuration was stored for the subsequent apply
operation, which would result in unexplained "Resource node has no
configuration attached" errors later on.

Also due to the child module diagnostics being lost, any newly added
nested modules would be silently ignored until `init` was run again
manually.
2019-07-16 19:06:48 -04:00
Radek Simko 5b9f2fafc8 Standardise directory name for test data 2019-06-30 10:16:15 +02:00
Kristin Laemmert 2a457115a3
configs: fix panic when the value is missing from version attribute in a provider block 2019-06-21 14:30:17 -04:00
Pam Selle 59c5cc4788
Merge pull request #21254 from davewongillies/gcs
Add GCS source support for modules
2019-06-13 10:24:38 -04:00
Martin Atkins e85093ce08 configs/configload: Don't panic when version constraint is added
Previously, adding a version constraint to a module that was previously
recorded without a version in the module manifest would cause a panic.

Instead, we now use a slight variant of the "dependencies have changed"
error that doesn't try to print out a specific version number.
2019-06-03 09:45:30 -07:00
David Gillies 8b45443b21
Add GCS source support for modules 2019-05-21 12:18:49 -07:00
Martin Atkins 004c2056a7 configs/configupgrade: Use single-line syntax for empty object exprs 2019-05-16 07:29:42 -07:00
Kristin Laemmert 14d625c850
configs/configupgrade: preserve in-line comments for lists (#21299)
* configs/configupgrade: preserve in-line comments for lists

The configupgrade tool was not writing `LineComments` for lists. Now it
is!

Fixes #21155
2019-05-14 16:19:31 -04:00
Radek Simko a2ee9fc8f9
Merge pull request #21282 from hashicorp/configupgrade-err-msg-fmt
configupgrade: Improve error message formatting
2019-05-13 16:17:30 +01:00
Radek Simko 8a6d1d62b6
stringer: Regenerate files with latest version 2019-05-13 15:34:27 +01:00