Commit Graph

165 Commits

Author SHA1 Message Date
James Bardin 3e03213485
Merge pull request #22478 from hashicorp/jbardin/coerce-value
don't validate Min/Max block vals in CoerceValue
2019-08-16 15:58:53 -04:00
Daniel Schroeder 628b50f900 configs/configupgrade: typo fix (#22469) 2019-08-16 10:38:38 -04:00
James Bardin 554cedab8a don't validate Min/Max block vals in CoerceValue
A provider may not have the data to fill in required block values in all
cases during the resource Read operation. This is more common in import,
because there is no initial configuration or state, and it's possible
some values are only provided in the configuration.

The original intent of MinItems and MaxItems in the schema was to
enforce configuration constraints, not to enforce what the resource
could save in the state.  Since the configuration is already statically
validated, and the Schema is validated against the configuration in a
separate step, we can drop these extra validation constraints in
CoerceValue and relax it to only ensure the types conform to what is
expected.
2019-08-15 10:02:39 -04:00
Pam Selle 9631e4c73d
Merge pull request #20571 from sergkondr/fix_misspelling
fix misspelling
2019-08-13 17:13:13 -04:00
Alex Pilon 4bf43efcfd
move hcl2shim package to configs 2019-08-06 19:58:58 -04:00
James Bardin 5878527732
Merge pull request #22221 from hashicorp/jbardin/min-items
MinItems with dynamic blocks
2019-07-29 09:45:42 -07:00
James Bardin 67dbd6d345 don't check MinItems with unknowns in blocks
If a block was defined via "dynamic", there will be only one block value
until the expansion is known. Since we can't detect dynamic blocks at
this point, don't verify MinItems while there are unknown values in the
config.

The decoder spec can also only check for existence of a block, so limit
the check to 0 or 1.
2019-07-27 11:50:28 -07:00
Pam Selle 360068b3cb
Merge pull request #21922 from pselle/resource_for_each
Resource for_each
2019-07-26 11:41:56 -04:00
Pam Selle e7d8ac5ad7 Remove panic, update comment 2019-07-26 11:22:10 -04:00
Thayne McCombs 7c678d104f Add support for for_each for data blocks.
This also fixes a few things with resource for_each:

It makes validation more like validation for count.

It makes sure the index is stored in the state properly.
2019-07-25 16:59:06 -04:00
Kristin Laemmert 66f4a48b8c
configs/configupgrade: fix panic on nil hilNode (#22181)
In some cases (see #22020 for a specific example), the parsed hilNode
can be nil. This causes a series of panics. Instead, return an error and
move on.
2019-07-23 13:05:37 -04:00
Pam Selle 7d905f6777 Resource for_each 2019-07-22 10:51:16 -04:00
Alex Pilon 0450f487fa
move IsEmptyDir to configs package 2019-07-18 13:07:10 -04:00
James Bardin 8111050c66 ensure we record diagnostics from nested modules
When loading nested modules, the child module diagnostics were dropped
in the recursive function. This mean that the config from the submodules
wasn't fully loaded, even though no errors were reported to the user.

This caused further problems if the plan was stored in a plan file, when
means only the partial configuration was stored for the subsequent apply
operation, which would result in unexplained "Resource node has no
configuration attached" errors later on.

Also due to the child module diagnostics being lost, any newly added
nested modules would be silently ignored until `init` was run again
manually.
2019-07-16 19:06:48 -04:00
Radek Simko 5b9f2fafc8 Standardise directory name for test data 2019-06-30 10:16:15 +02:00
Kristin Laemmert 2a457115a3
configs: fix panic when the value is missing from version attribute in a provider block 2019-06-21 14:30:17 -04:00
Pam Selle 59c5cc4788
Merge pull request #21254 from davewongillies/gcs
Add GCS source support for modules
2019-06-13 10:24:38 -04:00
Martin Atkins e85093ce08 configs/configload: Don't panic when version constraint is added
Previously, adding a version constraint to a module that was previously
recorded without a version in the module manifest would cause a panic.

Instead, we now use a slight variant of the "dependencies have changed"
error that doesn't try to print out a specific version number.
2019-06-03 09:45:30 -07:00
David Gillies 8b45443b21
Add GCS source support for modules 2019-05-21 12:18:49 -07:00
Martin Atkins 004c2056a7 configs/configupgrade: Use single-line syntax for empty object exprs 2019-05-16 07:29:42 -07:00
Kristin Laemmert 14d625c850
configs/configupgrade: preserve in-line comments for lists (#21299)
* configs/configupgrade: preserve in-line comments for lists

The configupgrade tool was not writing `LineComments` for lists. Now it
is!

Fixes #21155
2019-05-14 16:19:31 -04:00
Radek Simko a2ee9fc8f9
Merge pull request #21282 from hashicorp/configupgrade-err-msg-fmt
configupgrade: Improve error message formatting
2019-05-13 16:17:30 +01:00
Radek Simko 8a6d1d62b6
stringer: Regenerate files with latest version 2019-05-13 15:34:27 +01:00
Radek Simko 81c20ed7ae
configupgrade: Improve error message formatting 2019-05-13 13:14:59 +01:00
Radek Simko 42ba7a3e00
configupgrade: Upgrade indexing of splat syntax 2019-04-26 23:27:32 +01:00
Radek Simko 1f5cadeec0
0.12upgrade: Return error for invalid reference 2019-04-26 23:27:31 +01:00
Martin Atkins d7f23f0beb configs/configupgrade: Don't panic if analyzer fails
Previously we were trying to access a field of the analysis object before
checking if analysis produced errors. The analysis function usually
returns a nil analysis on error, so this would result in a panic whenever
that happened.

Now we'll dereference the analysis object pointer only after checking for
errors, so we'll get a chance to report the analysis error to the user.
2019-04-17 10:08:54 -07:00
Martin Atkins 1bb47ab9a5 configs/configupgrade: Preserve comments on items in object exprs
The expression upgrade functionality mostly ignores comments because in
the old language the syntax prevented comments from appearing in the
middle of expressions, but there was one exception: object expressions.

Because HCL 1 used ObjectType both for blocks and for object expressions,
that is the one situation where something we consider to be an expression
could have inline attached comments in the old language.

We migrate these here so we don't lose these comments that don't appear
anywhere else. Other comments get gathered up into a general comments
set maintained inside the analysis object and so will be printed out as
required _between_ expressions, just as they did before.
2019-04-17 07:48:57 -07:00
Martin Atkins bb118c37a2 configs: Handle "dynamic" blocks as special during override merging
Previously we were treating "dynamic" blocks in configuration the same as
any other block type when merging config bodies, so that dynamic blocks
in the override would override any dynamic blocks present in the base,
without considering the dynamic block type.

It's more useful and intuitive for us to treat dynamic blocks as if they
are instances of their given block type for the purpose of overriding.
That means a foo block can be overridden by a dynamic "foo" block and
vice-versa, and dynamic blocks of different types do not interact at all
during overriding.

This requires us to recognize dynamic blocks and treat them specially
during decoding of a merged body. We leave them unexpanded here because
this package is not responsible for dynamic block expansion (that happens
in the sibling "lang" package) but we do decode them enough to recognize
their labels so we can treat them as if they were blocks of the labelled
type.
2019-04-16 06:58:45 -07:00
Martin Atkins 88e76fa9ef configs/configschema: Introduce the NestingGroup mode for blocks
In study of existing providers we've found a pattern we werent previously
accounting for of using a nested block type to represent a group of
arguments that relate to a particular feature that is always enabled but
where it improves configuration readability to group all of its settings
together in a nested block.

The existing NestingSingle was not a good fit for this because it is
designed under the assumption that the presence or absence of the block
has some significance in enabling or disabling the relevant feature, and
so for these always-active cases we'd generate a misleading plan where
the settings for the feature appear totally absent, rather than showing
the default values that will be selected.

NestingGroup is, therefore, a slight variation of NestingSingle where
presence vs. absence of the block is not distinguishable (it's never null)
and instead its contents are treated as unset when the block is absent.
This then in turn causes any default values associated with the nested
arguments to be honored and displayed in the plan whenever the block is
not explicitly configured.

The current SDK cannot activate this mode, but that's okay because its
"legacy type system" opt-out flag allows it to force a block to be
processed in this way anyway. We're adding this now so that we can
introduce the feature in a future SDK without causing a breaking change
to the protocol, since the set of possible block nesting modes is not
extensible.
2019-04-10 14:53:52 -07:00
Martin Atkins a20084dc0e configs/configschema: EmptyValue methods
These helpers determine the value that would be used for a particular
schema construct if the configuration construct it represents is not
present (or, in the case of *Block, empty) in the configuration.

This is different than cty.NullVal on the implied type because it might
return non-null "empty" values for certain constructs if their absence
would be reported as such during a decode with no required attributes or
blocks.
2019-04-10 14:53:52 -07:00
Martin Atkins 26c1e40ad7 configs/configupgrade: Normalize number literals to decimal
The v0.12 language supports numeric constants only in decimal notation, as
a simplification. For rare situations where a different base is more
appropriate, such as unix-style file modes, we've found it better for
providers to accept a string containing a representation in the
appropriate base, since that way the interpretation can be validated and
it will be displayed in the same way in the rendered plan diff, in
outputs, etc.

We use tv.Value() here to mimick how HCL 1 itself would have interpreted
these, and then format them back out in the canonical form, which
implicitly converts any non-decimal constants to decimal on the way
through.
2019-04-04 18:09:44 -07:00
Kristin Laemmert 3aa4ac43f9
configs/configupgrade: return if hil.Parse() produces an error. (#20920)
Fixes #20917
2019-04-03 14:20:59 -04:00
Martin Atkins 6c5819f910 configs/configupgrade: Prefer block syntax for list-of-object attributes
In order to preserve pre-v0.12 idiom for list-of-object attributes, we'll
prefer to use block syntax for them except for the special situation where
the user explicitly assigns an empty list, where attribute syntax is
required in order to allow existing provider logic to differentiate from
an implicit lack of blocks.
2019-04-01 13:30:24 -07:00
Kristin Laemmert da52170797
configupgrade: fix test (#20863) 2019-03-28 18:00:33 -04:00
Kristin Laemmert 1baa1b907e
configs/configupgrade: detect invalid resource names and print a TODO (#20856)
* configs/configupgrade: detect invalid resource names and print a TODO
message

In terraform 0.11 and prior it was possible to start a resource name
with a number. This is no longer valid, as the resource name would would
be ambiguous with number values in HCL expressions.

Fixes #19919

* Update configs/configupgrade/test-fixtures/valid/invalid-resource-name/want/resource.tf

Co-Authored-By: mildwonkey <mildwonkey@users.noreply.github.com>
2019-03-28 13:48:35 -04:00
Justin Downing 1e32ae243c grammatical updates to comments and docs (#20195) 2019-03-21 14:05:41 -07:00
Martin Atkins ea1d5f8fcb vendor: go get github.com/hashicorp/hcl2@master
This includes two upstream fixes:

- Handle explicit JSON "null" consistently during decode of JSON syntax.
- Properly detect the end of a "heredoc" when formatting to avoid messing
  up indentation of other lines following the heredoc.
2019-03-15 13:55:30 -07:00
Kristin Laemmert 2c60140cad
configs: do not panic if module version is a variable (#20681)
Fixes #20653
2019-03-14 09:12:27 -07:00
Kristin Laemmert a15a4acf2f
configs/configupgrade: detect possible relative module sources (#20646)
* configs/configupgrade: detect possible relative module sources

If a module source appears to be a relative local path but does not have
a preceding ./, print a #TODO message for the user.

* internal/initwd: limit go-getter detectors to those supported by terraform
* internal/initwd: move isMaybeRelativeLocalPath check into getWithGoGetter

To avoid making two calls to getter.Detect, which potentially makes
non-trivial API calls, the "isMaybeRelativeLocalPath" check was moved to
a later step and a custom error type was added so user-friendly
diagnostics could be displayed in the event that a possible relative local
path was detected.
2019-03-13 11:17:14 -07:00
Kristin Laemmert 0c5fd835ce
copyDir: detect if the module install path is a symlink to a directory (#20603)
configs/configload and internal/initwd both had a copyDir function that
would fail if the source directory contained a symlinked directory,
because the os.FileMode.IsDir() returns false for symlinks.

This PR adds a check for a symlink and copies that symlink in the
target directory. It handles symlinks for both files and directories
(with included tests).

Fixes #20539
2019-03-07 12:59:48 -08:00
Sergey Kondrashov 43e7a7b552 fix misspelling 2019-03-05 16:12:52 +03:00
Martin Atkins b217624d83 config/configupgrade: Test to show that list unwrapping works for sets
This was already working but we had no tests to prove it.
2019-02-22 17:40:40 -08:00
Martin Atkins dd43926761 configs/configupgrade: Fix up uses of the .count pseudo-attribute
Terraform 0.11 and prior had an odd special case where a resource
attribute access for "count" would be resolved as the count for the
whole resource, rather than as an attribute of an individual instance as
for all other attributes.

Because Terraform 0.12 makes test_instance.foo appear as a list when count
is set (so it can be used in other expressions), it's no longer possible
to have an attribute in that position: lists don't have attributes.
Fortunately we don't really need that special case anymore since it
doesn't do anything we can't now do with the length(...) function.

This upgrade rule, then, detects references like test_instance.foo.count
and rewrites to length(test_instance.foo). As a special case, if
test_instance.foo doesn't have "count" set then it just rewrites as the
constant 1, which mimics what would've happened in that case in Terraform
0.11.
2019-02-22 16:18:53 -08:00
Martin Atkins 966eb39427 configs/configupgrade: Default arguments in "connection" blocks
Prior to Terraform v0.12 it was possible for a provider to secretly set
some default arguments for the "connection" block, which most commonly
included a hard-coded type of "ssh" and a value from "host".

In the interests of "explicit is better than implicit", Terraform 0.12 no
longer has this feature and instead requires connection settings to be
written explicitly in terms of the resource's exported attributes. For
compatibility though, the upgrade tool will insert expressions that are
as close as possible to the logic the provider formerly implemented, or
in a few rare cases a TF-UPGRADE-TODO comment to fix it up manually.

Some of the existing resource type implementations have incredibly
complicated implementations of selecting a single host IP address to use
and don't expose the result of that as an attribute, so for now we handle
those via a complicated Terraform language expression achieving the same
result. Ideally these providers would introduce a new attribute that
exports the same address formerly exported as the hostname before their
initial v0.12-compatible release, in which case we can simplify these to
just reference the attribute in question. That would be preferable also
because it would allow use of that exported attribute in other contexts,
such as in a null_resource provisioner somewhere else or in an output
to allow a caller to deal with the SSH part itself.
2019-02-22 12:32:56 -08:00
Martin Atkins ac6e0e42dc configs/configupgrade: Upgrade the bodies of "connection" blocks
This uses the fixed "superset" schema from the main terraform package to
apply our standard expression mapping, with the exception of "type" where
interpolation sequences are not supported due to the type being evaluated
early to retrieve the schema for decoding the rest.
2019-02-22 12:32:56 -08:00
Martin Atkins e2ef51800a configs/configupgrade: Upgrade the bodies of "provisioner" blocks
Aside from the two special meta-arguments "connection" and "provisioner"
this is just our standard mapping from schema to conversion rules, using
the provisioner's configuration schema.
2019-02-22 12:32:56 -08:00
Martin Atkins cdca8fbfe8 configs/configupgrade: Correct ignore_changes error message
Due to a copy-paste error, this was using the message from the providers
map in a "module" block.

This new message is not particularly helpful, but we should only see it
for a configuration that wouldn't have been valid in 0.11 either, and so
it's unlikely to be displayed.
2019-02-22 12:32:56 -08:00
Martin Atkins 0095a944cd configs/configupgrade: Include provisioner schemas in analysis
We'll need these to migrate any "provisioner" blocks we find in the input
configuration.
2019-02-22 12:32:56 -08:00
Martin Atkins fa0d6484df configs/configupgrade: Detect and fix element(...) usage with sets
Although sets do not have indexed elements, in Terraform 0.11 and earlier
element(...) would work with sets because we'd automatically convert them
to lists on entry to HIL -- with an arbitrary-but-consistent ordering --
and this return an arbitrary-but-consistent element from the list.

The element(...) function in Terraform 0.12 does not allow this because it
is not safe in general, but there was an existing pattern relying on this
in Terraform 0.11 configs which this upgrade rule is intended to preserve:

    resource "example" "example" {
      count = "${length(any_set_attribute)}"

      foo = "${element(any_set_attribute, count.index}"
    }

The above works because the exact indices assigned in the conversion are
irrelevant: we're just asking Terraform to create one resource for each
distinct element in the set.

This upgrade rule therefore inserts an explicit conversion to list if it
is able to successfully provide that the given expression will return a
set type:

    foo = "${element(tolist(any_set_attribute), count.index}"

This makes the conversion explicit, allowing users to decide if it is
safe and rework the configuration if not. Since our static type analysis
functionality focuses mainly on resource type attributes, in practice this
rule will only apply when the given expression is a statically-checkable
resource reference. Since sets are an SDK-only concept in Terraform 0.11
and earlier anyway, in practice that works out just right: it's not
possible for sets to appear anywhere else in older versions anyway.
2019-02-21 09:39:55 -08:00