Commit Graph

18 Commits

Author SHA1 Message Date
Kristin Laemmert a15a4acf2f
configs/configupgrade: detect possible relative module sources (#20646)
* configs/configupgrade: detect possible relative module sources

If a module source appears to be a relative local path but does not have
a preceding ./, print a #TODO message for the user.

* internal/initwd: limit go-getter detectors to those supported by terraform
* internal/initwd: move isMaybeRelativeLocalPath check into getWithGoGetter

To avoid making two calls to getter.Detect, which potentially makes
non-trivial API calls, the "isMaybeRelativeLocalPath" check was moved to
a later step and a custom error type was added so user-friendly
diagnostics could be displayed in the event that a possible relative local
path was detected.
2019-03-13 11:17:14 -07:00
Martin Atkins 966eb39427 configs/configupgrade: Default arguments in "connection" blocks
Prior to Terraform v0.12 it was possible for a provider to secretly set
some default arguments for the "connection" block, which most commonly
included a hard-coded type of "ssh" and a value from "host".

In the interests of "explicit is better than implicit", Terraform 0.12 no
longer has this feature and instead requires connection settings to be
written explicitly in terms of the resource's exported attributes. For
compatibility though, the upgrade tool will insert expressions that are
as close as possible to the logic the provider formerly implemented, or
in a few rare cases a TF-UPGRADE-TODO comment to fix it up manually.

Some of the existing resource type implementations have incredibly
complicated implementations of selecting a single host IP address to use
and don't expose the result of that as an attribute, so for now we handle
those via a complicated Terraform language expression achieving the same
result. Ideally these providers would introduce a new attribute that
exports the same address formerly exported as the hostname before their
initial v0.12-compatible release, in which case we can simplify these to
just reference the attribute in question. That would be preferable also
because it would allow use of that exported attribute in other contexts,
such as in a null_resource provisioner somewhere else or in an output
to allow a caller to deal with the SSH part itself.
2019-02-22 12:32:56 -08:00
Martin Atkins e2ef51800a configs/configupgrade: Upgrade the bodies of "provisioner" blocks
Aside from the two special meta-arguments "connection" and "provisioner"
this is just our standard mapping from schema to conversion rules, using
the provisioner's configuration schema.
2019-02-22 12:32:56 -08:00
Martin Atkins fa0d6484df configs/configupgrade: Detect and fix element(...) usage with sets
Although sets do not have indexed elements, in Terraform 0.11 and earlier
element(...) would work with sets because we'd automatically convert them
to lists on entry to HIL -- with an arbitrary-but-consistent ordering --
and this return an arbitrary-but-consistent element from the list.

The element(...) function in Terraform 0.12 does not allow this because it
is not safe in general, but there was an existing pattern relying on this
in Terraform 0.11 configs which this upgrade rule is intended to preserve:

    resource "example" "example" {
      count = "${length(any_set_attribute)}"

      foo = "${element(any_set_attribute, count.index}"
    }

The above works because the exact indices assigned in the conversion are
irrelevant: we're just asking Terraform to create one resource for each
distinct element in the set.

This upgrade rule therefore inserts an explicit conversion to list if it
is able to successfully provide that the given expression will return a
set type:

    foo = "${element(tolist(any_set_attribute), count.index}"

This makes the conversion explicit, allowing users to decide if it is
safe and rework the configuration if not. Since our static type analysis
functionality focuses mainly on resource type attributes, in practice this
rule will only apply when the given expression is a statically-checkable
resource reference. Since sets are an SDK-only concept in Terraform 0.11
and earlier anyway, in practice that works out just right: it's not
possible for sets to appear anywhere else in older versions anyway.
2019-02-21 09:39:55 -08:00
Martin Atkins f93f7e5b5c configs/configupgrade: Remove redundant list brackets
In early versions of Terraform where the interpolation language didn't
have any real list support, list brackets around a single string was the
signal to split the string on a special uuid separator to produce a list
just in time for processing, giving expressions like this:

    foo = ["${test_instance.foo.*.id}"]

Logically this is weird because it looks like it should produce a list
of lists of strings. When we added real list support in Terraform 0.7 we
retained support for this behavior by trimming off extra levels of list
during evaluation, and inadvertently continued relying on this notation
for correct type checking.

During the Terraform 0.10 line we fixed the type checker bugs (a few
remaining issues notwithstanding) so that it was finally possible to
use the more intuitive form:

    foo = "${test_instance.foo.*.id}"

...but we continued trimming off extra levels of list for backward
compatibility.

Terraform 0.12 finally removes that compatibility shim, causing redundant
list brackets to be interpreted as a list of lists.

This upgrade rule attempts to identify situations that are relying on the
old compatibility behavior and trim off the redundant extra brackets. It's
not possible to do this fully-generally using only static analysis, but
we can gather enough information through or partial type inference
mechanism here to deal with the most common situations automatically and
produce a TF-UPGRADE-TODO comment for more complex scenarios where the
user intent isn't decidable with only static analysis.

In particular, this handles by far the most common situation of wrapping
list brackets around a splat expression like the first example above.
After this and the other upgrade rules are applied, the first example
above will become:

    foo = test_instance.foo.*.id
2018-12-07 17:05:36 -08:00
Martin Atkins ceedeb69a9 configs/configupgrade: Normalize and upgrade reference expressions
The reference syntax is not significantly changed, but there are some
minor additional restrictions on identifiers in HCL2 and as a special case
we need to rewrite references to data.terraform_remote_state .

Along with those mandatory upgrades, we will also switch references to
using normal index syntax where it's safe to do so, as part of
de-emphasizing the old strange integer attribute syntax (like foo.0.bar).
2018-12-04 11:37:39 -08:00
Martin Atkins 6596d031d9 configs/configupgrade: Convert block-as-attr to dynamic blocks
Users discovered that they could exploit some missing validation in
Terraform v0.11 and prior to treat block types as if they were attributes
and assign dynamic expressions to them, with some significant caveats and
gotchas resulting from the fact that this was never intended to work.

However, since such patterns are in use in the wild we'll convert them
to a dynamic block during upgrade. With only static analysis we must
unfortunately generate a very conservative, ugly dynamic block with
every possible argument set. Users ought to then clean up the generated
configuration after confirming which arguments are actually required.
2018-12-04 11:37:39 -08:00
Martin Atkins bdb724562c configs/configupgrade: Test that map attrs as blocks are fixed
The old parser was forgiving in allowing the use of block syntax where a
map attribute was expected, but the new parser is not (in order to allow
for dynamic map keys, for expressions, etc) and so the upgrade tool must
fix these to use attribute syntax.
2018-12-04 11:37:39 -08:00
Martin Atkins 1aa368d0d8 configs/configupgrade: Add some logging and enable it for tests 2018-12-04 11:37:39 -08:00
Martin Atkins e8999eefdc configs/configupgrade: Populate the test provider schema
This will allow us to test some schema-sensitive migration rules.
2018-12-04 11:37:39 -08:00
Martin Atkins 8e594f32aa configs/configupgrade: Upgrade rules for the "terraform" block type
This includes the backend configuration, so we need to load the requested
backend and migrate the included configuration per that schema.
2018-12-04 11:37:39 -08:00
Martin Atkins cf52e224f6 configs/configupgrade: Basic migration of provider blocks
This involved some refactoring of how block bodies are migrated, which
still needs some additional work to deal with meta-arguments but is now
at least partially generalized to support both resource and provider
blocks.
2018-12-04 11:37:39 -08:00
Martin Atkins 1a654e9e1c configs/configupgrade: Disable the tests for now
The tests in here are illustrating that this package is not yet finished,
but we plan to run a release before we finish this and so we'll skip those
tests for now with the intent of reinstating this again once we return
to finish this up.
2018-10-16 19:14:11 -07:00
Martin Atkins 961056c08d configs/configupgrade: Use mock provider instead of test provider
The test provider comes with a lot of baggage since it's designed to be
used as a plugin, so instead we'll just use the mock provider
implementation directly, and so we can (in a later commit) configure it
appropriately for what our tests need here.
2018-10-16 19:14:11 -07:00
Martin Atkins 85aa8769db configs/configupgrade: Partially fix TestUpgradeValid
This is still not compileable because the test provider needs to be
updated to the new provider interface, but all the rest of the types are
now correct so we can update the test provider in a later commit to make
this work again.
2018-10-16 19:14:11 -07:00
Martin Atkins adb88eaa16 configupgrade: Analysis of input configuration
In order to properly migrate the contents of resource, data, provider and
provisioner blocks we will need the provider's schema in order to
understand what is expected, so we can resolve some ambiguities inherent
in the legacy HCL AST.

This includes an initial prototype of migrating the content of resource
blocks just to verify that the information is being gathered correctly.
As with the rest of the upgrade_native.go file, this will be reorganized
significantly once the basic end-to-end flow is established and we can
see how to organize this code better.
2018-10-16 18:50:29 -07:00
Martin Atkins 95b7b883a3 configupgrade: Basic expression formatting
This covers all of the expression node types in HIL's AST, and also
includes initial support for some of our top-level blocks so that we can
easily test that.

The initial implementations of the "variable" and "output" blocks are
pretty redundant and messy, so we can hopefully improve on these in a
later pass.
2018-10-16 18:50:29 -07:00
Martin Atkins a345533573 configupgrade: Beginnings of Upgrade function
This function is the main functionality of this package. So far it just
deals with detecting and renaming JSON files that are mislabeled as
native syntax files. Other functionality will follow in later commits.
2018-10-16 18:50:29 -07:00