Commit Graph

792 Commits

Author SHA1 Message Date
James Bardin 0bc279557e add more metadata to the Tree and the storage path
Submodules were located by using their module path as the storage key.
Now that modules may have versions, a submodule needs to know how to
locate the corect source depending on the versions of its ancestors in
the tree.

Add a version field to each Tree, and a pointer back to the parent Tree
to step back through the ancestors. The new versionedPathKey method uses
this information to build a unique key for each module, dependent on the
ancestor versions.

Not only do stored modules need to know their version if it exists, but
any relative source needs to know all the ancestor versions in order to
resolve correctly.
2017-10-27 09:08:15 -04:00
James Bardin 1f44fd8eb2 move getStorage to moduleStorage too
Add a test to ensure we don't store different modules with the same
relative path in the same location.
2017-10-27 09:08:15 -04:00
James Bardin 3d163917c7 move the getter.Storage into our moduleStorge type
Wrap the Storage interfaces together.
2017-10-27 09:08:15 -04:00
James Bardin 7416e7a6e0 update the module manifest implementation
The getter.Storage abstraction is proving entirely inadequate here, but
we can't replace it wholesale at the moment.

The Tree loader needs to know the location of the manifest before it can
start loading any modules. Since the version will have to be part of the
hashed storage key, there is no way to know what version of each module
are stored. The storageDir function will extract the StorageDir field
from the underlying FolderStorage instance for the tree to locate the
manifest.
2017-10-27 09:08:15 -04:00
James Bardin 283d88551a Add a new module storage manifest
To add registry support, a workaround in the local module storage was
added to record the subdirectory containing the module source from
within the archive file. Here we replace that temporary implementation
with the full manifest needed to record the necessary module metadata
for module loading.

In order to support versioned modules, the actual stored version needs
to be recorded. This can't be derived from the configuration, because
the configuration only contains the constraints, and at load time we need
to be able to enumerate the stored modules and all versions in order to
resolve them.

While the local storage key will be derived from the source and version,
that information is lost once it's hashed. While the entire storage
layer could be replaced to encode the needed data in the path itself,
this provides a minimal change to work with the existing storage code.
2017-10-27 09:08:15 -04:00
James Bardin 0962792257 surface registry errors to the user
Now that we can enforce local modules being relative or absolute paths,
we can be assured that any module source matching a registry pattern
must be found in the registry. This allows us to surface more useful
errors to the user, rather than simply stating that a source string
isn't valid.
2017-10-27 09:08:15 -04:00
James Bardin 40c36b48dc missing test-fixture files 2017-10-27 09:08:15 -04:00
James Bardin 8933ef8f70 make //subdir notation work with registry modules
Append the original source subdir to the discovered subdir when locating
the files in the module tree.
2017-10-27 09:08:15 -04:00
James Bardin a359c4ee2f check the registry detector before local files
Breaking change for 0.11.

Local files were checked first to avoid the possibility of breaking a
module with a local source that looked like a registry ID. Now we can
enfore that any source iwth the pattern "namespace/identifier/provider"
must be a registry module.
2017-10-27 09:08:15 -04:00
James Bardin 6088270c66 skip lookup for paths starting with ../
Prevent an extra registry lookup for sources starting with ../
2017-10-27 09:08:15 -04:00
James Bardin b0163e69d1 update comments 2017-10-27 09:08:15 -04:00
James Bardin 4ab70ba09d rename confusing test module name 2017-10-27 09:08:15 -04:00
James Bardin fe0957deb7 hcl2 maps now work 2017-10-27 09:08:15 -04:00
James Bardin 2e505083cd change ProviderConfig.Scope to Path
Though it's intended for "interpolation scope", Path is generally used
for this elsewhere.
2017-10-27 09:08:15 -04:00
James Bardin cb0e37a870 implement provider inheritence during loading
This implements provider inheritance during config loading, rather than
during graph evaluation. At this point it's much simpler to find the
desired configuration, and once all providers are declared, all the
inheritance code in the graph can be removed.

The inheritance is dome by simply copying the RawConfig from the parent
ProviderConfig into the module. Since this happens before any
evaluation, we record the original interpolation scope in the
ProviderConfig so that it can be properly resolved later on.
2017-10-27 09:08:15 -04:00
James Bardin 29e5a355b9 add "version" and "provider" to module config
Add the Version and Providers fields to the module config.

Add ProviderConfig.Scope, which will be used to record the original
path of a ProviderConfig for interpolation.
2017-10-27 09:08:15 -04:00
Radek Simko 7fceccfbf7
helper/schema: Loosen validation for 'id' field 2017-10-26 09:37:38 +01:00
Lyle Franklin 43dcaa1a00 config: Don't panic if config directory contains only overrides 2017-10-23 15:46:15 -07:00
Martin Atkins ccb328cc1f config: source code ranges for InterpolatedVariable
Having a reference to the originating source range will allow us to
generate decent error messages if certain references can't be resolved
at interpolation time.

This is not yet populated or used. It will never be populated nor used by
the current HCL/HIL-based interpolation path, but will be used by the
experimental HCL2-based interpolation path to give it the necessary info
to produce diagnostics.
2017-10-17 07:20:17 -07:00
Martin Atkins 71e989ba3e config/hcl2shim: make some of the HCL2 shim functions public
The value-conversion machinery is also needed in the main "terraform"
package to help us populate our HCL2 evaluation scope, so a subset of the
shim functions move here into a new package where they can be public.

Some of them remain private within the config package since they depend
on some other symbols in the config package, and they are not needed
by outside callers anyway.
2017-10-16 17:54:02 -07:00
Martin Atkins 22fb82963c config: when copying a HCL2 RawConfig, don't corrupt it
Previously we were demoting HCL2 RawConfigs into empty old-school
RawConfigs on copy.
2017-10-16 17:52:23 -07:00
Gauthier Wallet ed9ba576e3 core: New interpolation function "chunklist"
This turns a list into a list of lists with each element (apart from possibly the last) being the given length.
2017-10-10 11:56:13 -07:00
Stefan Staudenmeyer 14792035cf Aliasing the module to omit errors thrown at build time
Signed-off-by: Stefan Staudenmeyer <stefan.staudenmeyer@instana.com>
2017-10-09 16:44:58 +02:00
Martin Atkins fc20f419dd config and command: use errwrap to propagate config load errors
Previously we were using fmt.Sprintf and thus forcing the stringification
of the wrapped error.

Using errwrap allows us to unpack the original error at the top of the
stack, which is useful when the wrapped error is really a hcl.Diagnostics
containing potentially-multiple errors and possibly warnings.
2017-10-06 11:46:07 -07:00
Martin Atkins b91bd62747 config/configschema: Sensitive flag for attributes
We don't currently have any need for this information, but we're
propagating it out of helper/schema here pre-emptively so that once we
later have a use for it we will not need to rebuild the providers to gain
access to it.

The long-term expected use-case for this is to have Terraform Core use
static analysis techniques to trace the path of sensitive data through
interpolations so that intermediate results can be flagged as sensitive
too, but we have a lot more work to do before such a thing would actually
be possible.
2017-10-04 16:35:11 -07:00
Martin Atkins 0ef985cada config/configschema: Block.ImpliedType delegates to zcldec
zcldec now has its own function for computing the implied type for a spec,
so we can use that instead of our own logic.

The zcldec logic is more general since its spec model is more general than
our schema model here, but it produces the same results for the subset
of specifications that our DecoderSpec method produces.
2017-10-04 16:35:11 -07:00
Martin Atkins 2b622fe31a config/configschema: Block.DecoderSpec
This returns a decoding specification that can be used with the hcldec
package to decode a body into a cty.Value of an object type.
2017-10-04 16:35:11 -07:00
Martin Atkins d0d829848a config/configschema: Block.ImpliedType method
This returns a cty.Type that the caller can expect to recieve when
decoding a value using the (not yet implemented) decoder specification
for a given schema.
2017-10-04 16:35:11 -07:00
Martin Atkins f117906bdb config/configschema: InternalValidate for blocks
This checks that a schema complies with the documented constraints on
which values are valid. It is primarily intended for use in tests.
2017-10-04 16:35:11 -07:00
Martin Atkins d712a04c32 config/configschema: MinItems and MaxItems for nested blocks 2017-10-04 16:35:11 -07:00
Martin Atkins dadec5210b config/configschema: stub out initial API 2017-10-04 16:35:11 -07:00
Martin Atkins b851fa71c9 config: Make HIL-based functions available to HCL2 via a shim
Terraform has a _lot_ of functions written against HIL's function API, and
we're not ready to rewrite them all yet, so instead we shim the HIL
function API to conform to the HCL2 (really: cty) function API and thus
allow most of our existing functions to work as expected when called from
HCL2-based config files.

Not all of the functions can be fully shimmed in this way due to depending
on HIL implementation details that we can't mimic through the HCL2 API.
We don't attempt to address that yet, and instead just let them fail when
called. We will eventually address this by using first-class HCL2
functions for these few cases, thus avoiding the HIL API altogether where
we need to. (The methodology for that is already illustrated here in the
provision of jsonencode and jsondecode functions that are HCL2-native.)
2017-10-03 17:47:01 -07:00
Martin Atkins 34e9de605c config: tests for HIL-to-HCL2 value shimming 2017-10-03 17:47:01 -07:00
Martin Atkins bbf9725134 config: Validate resource "count" for HCL2-specified resources
This early validation uses interpolation of a placeholder value to achieve
some "best effort" validation of the validity of the count attribute.
Since HCL2-specified resources can't be interpolated using the main
interpolator, here we branch and use the HCL2 API to do a
largely-equivalent (though slightly less accurate) check.

In the long run we don't really need this extra check at all, since the
validation walk does a more accurate version of the same thing. However,
we're preserving this for now in the interests of minimizing the amount
of change for the main codepath during our experiment.
2017-10-03 17:47:01 -07:00
Martin Atkins 71e68f06c4 config: allow tests to force using the HCL2 loader
Currently the default for tests is to use the old HCL loader, but we need
to be able to test aspects of the new loader as we work through the
experimental phase. This new function testConfigHCL2 is the same as
testConfig except that it forces the use of HCL2 even if the opt-in
comment isn't present, thus allowing us to implement tests that ensure
that the exact same file works in both the old and new cases.

Once the HCL2 loader becomes the default this function will be removed
and callers will start calling into the normal testConfig function.
2017-10-03 17:47:01 -07:00
Martin Atkins d91327eaa0 config: allow HCL2 experiment opt-in (build-time flag to enable)
Use the new HCL2 config loader when the opt-in comment #terraform:hcl2 is
present in a .tf file.

For now this is disabled for "normal" builds and enabled only if
explicitly configured via a linker flag during build. This is because it's
not yet in a good state to be released: the HCL2 loader produces RawConfig
objects that the validator and interpolator can't yet deal with, and so
using HCL2 for anything non-trivial currently causes Terraform to crash
in real use.
2017-10-03 17:47:01 -07:00
Martin Atkins b0215fcd0f config: HCL2 config loader
This loader uses the HCL2 parser and decoder to process a config file,
and then transforms the result into the same shape as would be produced
by the HCL config loader.

To avoid making changes to the existing config structures (which are
depended on across much of the codebase) we first decode into a set of
HCL2-tailored structs and then process them into the public-facing structs
that a loader is expected to return. This is a compromise to keep the
config package API broadly unchanged for now. Once we're ready to remove
the old HCL loader (which implies that we're ready to support HCL2
natively elsewhere in the codebase) we will be able to simplify this
quite considerably.

Due to some mismatches of abstraction between HCL/HIL and HCL2, some
shimming is required to get the required result.
2017-10-03 17:47:01 -07:00
Martin Atkins edbbe41b44 config: allow a HCL2 body to piggy-back on a RawConfig
At this time we're not ready to refactor the various uses of RawConfig
in Terraform core, so we'll smuggle a HCL2 body within a degenerate
RawConfig object that we can then recognize and unpack once this object
is returned to us in an interpolation call.
2017-10-03 17:47:01 -07:00
James Bardin 08ac4f3a2e Merge pull request #16089 from wgliang/optimization-merge-slice
Optimization merge slice
2017-10-02 16:40:13 -04:00
James Bardin 1fbb2d611a remove extra slash from registry urls
A refactor introduced an extra `/` in the download url, which causes an
extra redirect during discovery.

Improve a registry test to verify that detection doesn't require the
registry after the modules have been fetched.
2017-09-28 14:45:02 -04:00
Ariel Alonso 550ae05819 config: new "transpose" interpolation function
This function takes a map of lists of strings and inverts it so that
the string values become keys and the keys become items within the
corresponding lists.
2017-09-28 10:29:25 -07:00
Martin Atkins a28b5d295e config: improve interpolation function test output
These tests were written before subtest support was available. By running
them as subtests we can get better output in the event of an error, or
in verbose mode.
2017-09-26 14:31:43 -07:00
James Bardin d78b575536 Merge pull request #16160 from hashicorp/jbardin/get-subdir
Handle module source subdirectories in Terraform
2017-09-26 09:23:19 -04:00
James Bardin 2b3e48aa70 fix test comments 2017-09-26 09:12:14 -04:00
Krzysztof Wilczynski 6e7e03f4ea config: new "abs" interpolation function
This new function returns the absolute value for a given number.
2017-09-25 14:19:18 -07:00
James Bardin f6e89bb8c3 record the subdirectory in the FolderStorage
Module detection currently requires calling the registry to determine
the subdirectory. Since we're not directly accessing the subdirectory
through FolderStorage, and now handling it within terraform so modules can
reference sibling paths, we need to call out to the registry every
time we load a configuration to verify the subdirectory for the module,
which is returned during the Detect.

Record the subdirectories for each module in the top-level of the
FolderStorage path for retrieval during Tree.Load. This lets us bypass
Detection altogether, modules can be loaded without redetecting.
2017-09-22 22:03:38 -04:00
James Bardin 1b01f18920 handle go-getter subdirs in Tree.Load
In order to remain backward compatible with some modules, we need to
handle subdirs during Load. This means duplicating part of the go-getter
code path for subDir handling so we can resolve any subDirs and globs
internally, while keeping the entire remote directory structure within
the file storage.
2017-09-22 20:32:44 -04:00
James Bardin 59089ef461 add test loading an archive with `//*` subdir
The registry uses the `//*` subdir format to unpack github archives. Add
a test to cover this pattern.
2017-09-21 15:32:55 -05:00
James Bardin 38569c8508 add tests for get from tar subdir
Test that we can get a subdirectory from a tarball (or any other
"packed" source that we support).

The 'tar-subdir-to-parent' test highlights a regression where the
subdirectory module references a module in its parent directory. This
breaks the intended use ofr the subdirectory and the implementation in
go-getter. We need to fix this in terraform, and possible plan warnings
and deprecations for this type of source.
2017-09-21 13:12:33 -05:00
James Bardin 3d3992454e Add basic ACC test
This just does a lookup in the registry to verify that the download api
works.
2017-09-15 16:16:29 -04:00
James Bardin ba14cf9511 add test fixture for new getter discovery behavior 2017-09-15 16:16:29 -04:00
James Bardin f9fb2b4c9e move local module detection ahead of the registry
The getter.FileDetector was intended to be the final detector, only
converting a path to a file URL and returning a true in all cases. We
want to check for a local module before checking the registry so no
local modules that happen to match a registry module are broken.

Wrap the getter.FileDetector to check the module source's existence
before delegating the search to the registry.
2017-09-15 16:16:28 -04:00
James Bardin d216d19f21 change the FolderStorage key
updating the key will cause the FolderStorage hash to change forcing
modules to be re-fetched. This is required because any configurations
using the subDir notation will have the configuration in the wrong
directory.
2017-09-15 16:16:28 -04:00
James Bardin 6c20141c30 remove subdir handling from Tree.Load
Terraform was redundantly handling `//dir` notation which should be
handled by go-getter. Rather than allowing go-getter to unpack a subdir
as expected, the subdir was stripped off and accessed through the module
configuration.

This scheme will no longer works now that go-getter supports `*`
subdirectories
    (e.g. `//*` would be analogous to `tar --strip-components=1`).

Even though this allows Terraform to use go-getter's native unpacking,
detection is still done separately because Detect requires a `pwd` which
is dependent on the configuration directory and not known to the
global FolderStorage.
2017-09-15 16:16:28 -04:00
James Bardin a83ff57aea Add registry detector
Add a getter.Detector for detecting registry modules and looking up
the download location of the latest version. This is essentially a
temporary API until constraint solving is supported by the registry, as
then we'll have to supply the full set of known contraints to the
registry at once for resolution and we will fetch specific versions of
modules.
2017-09-15 16:16:28 -04:00
Micah Hausler 7fa4b648bb config: multi-line "indent" function
This function prepends a number of spaces on to all but the first line
of a string containing multiple lines of text.
2017-09-14 09:47:48 -07:00
Wang Guoliang 9a97c348f0
optimization:Merge slices 2017-09-13 21:08:41 +08:00
Martin Atkins 0a342e8dc2 config: allow local value interpolations in count
There is some additional, early validation on the "count" meta-argument
that verifies that only suitable variable types are used, and adding local
values to this whitelist was missed in the initial implementation.
2017-09-01 17:54:05 -07:00
Martin Atkins 8cd0ee80e5 config: merge/append for local values
It seems that this somehow got lost in the commit/rebase shuffle and
wasn't caught by the tests that _did_ make it because they were all using
just one file.

As a result of this bug, locals would fail to work correctly in any
configuration with more than one .tf file.

Along with restoring the append/merge behavior, this also reworks some of
the tests to exercise the multi-file case as better insurance against
regressions of this sort in future.

This fixes #15969.
2017-09-01 17:51:13 -07:00
Martin Atkins c12d64f340 Use t.Helper() in our test helpers
Go 1.9 adds this new function which, when called, marks the caller as
being a "helper function". Helper function stack frames are then skipped
when trying to find a line of test code to blame for a test failure, so
that the code in the main test function appears in the test failure output
rather than a line within the helper function itself.

This covers many -- but probaly not all -- of our test helpers across
various packages.
2017-08-28 09:59:30 -07:00
Mitchell Hashimoto adcf41f076
config: parse description field for outputs
We added the description field in 0.9 but we never parsed it because we
didn't have a use for it. As we prepare to use this field, let's start
parsing it out
2017-08-28 09:42:03 -07:00
Sunny 06fd5d7a3e config: urlencode interpolation function
This escapes all characters that might have a special interpretation when embedded into a portion of a URL, including slashes, equals signs and ampersands.
2017-08-22 11:26:09 -07:00
Gorka Lerchundi Osa a303817e03 config: base64gzip interpolation function
Since Terraform's internals are not 8-bit clean (it assumes UTF-8
strings), we can't implement raw gzip directly. We're going to add
support where it makes sense for passing data to attributes as
base64 so that the result of this function can be used.
2017-08-22 10:44:55 -07:00
Martin Atkins d41d58967f config: parsing of local.foo variables for interpolation 2017-08-21 15:15:25 -07:00
Martin Atkins f6797d6cb0 config: parsing of "locals" blocks in configuration 2017-08-21 15:15:25 -07:00
Sebastian Maj 28b5708fef config: jsonencode function to support nested lists and maps 2017-08-18 09:57:57 -07:00
JWal31 117f44bcda config: Add "flatten" interpolation function
This function turns a list of lists or any arbitrary number of nested
lists into a flat list of primitive values.
2017-08-16 13:46:19 -07:00
Radek Simko 67ceb1ab07
config: Make 'id' a reserved field name 2017-08-02 20:02:44 +02:00
Radek Simko 07cbd54fbc Actively disallow reserved field names in schema (#15522) 2017-07-10 21:51:55 -07:00
Robert Liebowitz 006744bfe0 Use all tfvars files in working directory
As a side effect, several commands that previously did not have a failure
state can now fail during meta-parameter processing.
2017-07-05 17:24:17 -07:00
trung 681661a539 #15291: config/interpolate_funcs: Added contains() function to test if a given element is present in the list 2017-06-16 15:05:19 -07:00
Martin Atkins f70318097a config: fix provider version constraint validation
Previously we were using the "semver" library to parse version
constraints, but we switched over to go-version and encapsulated it
inside our own plugin/discovery package to reduce dependency sprawl in
the code.

This particular situation was missed when updating references to the new
path, which meant that our validation code disagreed with the rest of
the code about what is considered a valid version constraint string.
By using the correct function, we ensure that we catch early any invalid
versions.
2017-06-09 14:03:59 -07:00
Martin Atkins a8a64c66c0 config/module: helper to visit all modules in a tree 2017-06-09 14:03:59 -07:00
Martin Atkins 32a5c62639 config: parse provider version constraints into a constraint map 2017-06-09 14:03:59 -07:00
Martin Atkins 0b14c2cdb3 Resolve resource provider types in config package
Previously the logic for inferring a provider type from a resource name
was buried a utility function in the 'terraform' package. Instead here we
lift it up into the 'config' package where we can make broader use of it
and where it's easier to discover.
2017-06-09 14:03:59 -07:00
Martin Atkins 7e7d4c70df config: allow version constraints on providers, but validate them
We now accept syntactically-valid version constraints on provider blocks,
though we still don't actually do anything with them.
2017-06-09 14:03:59 -07:00
Martin Atkins 73fc9985b2 config: add "version" argument to provider blocks, disabled
In future we will support version constraints on providers, so we're
reserving this attribute name that is currently not used by any builtin
providers.

For now using this will produce an error, since the rest of Terraform
(outside of the config parser) doesn't currently have this notion and we
don't want people to start trying to use it until its behavior is fully
defined and implemented.

It may be used by third-party providers, so this is a breaking change
worth warning about in CHANGELOG but one whose impact should be small.
Any third-party providers using this name should migrate to using a new
attribute name instead moving forward.
2017-06-09 14:03:59 -07:00
Radek Simko 1244309579 Fix stringer comments (#15069) 2017-06-05 10:17:35 +01:00
He Guimin 87562be855 provider/alicloud: Add the function of replacing ecs instance's system disk (#15048)
* add replacing system disk function for ecs

* remove ForceNew of system_disk_size
2017-06-05 11:27:49 +03:00
Jake Champlin ac177492fb
core: Revert stringer changes from earlier commits 2017-06-01 11:37:12 -04:00
Thomas Schaaf 79c91e11c8 provider/aws: Add aws elastic beanstalk solution stack (#14944)
* Add aws elastic beanstalk solution stack

Signed-off-by: Thomas Schaaf <thomaschaaf@Thomass-MBP.fritz.box>

* Fix incorrect naming

Signed-off-by: Thomas Schaaf <thomaschaaf@Thomass-MBP.fritz.box>

* Use unique go variable/function names

Signed-off-by: Thomas Schaaf <thomaschaaf@Thomass-MacBook-Pro.local>

* Add docs to sidebar

* Sort provider by alphabet

* Fix indent

* Add required statement

* Fix acceptance test
2017-06-01 02:23:06 +03:00
Jasmin Gacic 5d33023d99 Merge pull request #14725 from StackPointCloud/bcrypt
`bcrypt` builtin function
2017-05-30 21:39:41 +02:00
Martin Atkins 410b60cb7f Stop requiring multi-vars (splats) to be in array brackets
Prior to Terraform 0.7, lists in Terraform were just a shallow abstraction
on top of strings with a magic delimiter between items. Wrapping a single
string in brackets in the configuration was Terraform's prompt that it
needed to split the string on that delimiter during interpolation.

In 0.7, when first-class lists were added, this convention was preserved
by flattening lists-of-lists by one level when they were encountered in
configuration. However, there was an oversight in that change where it
did not correctly handle the case where the inner list was unknown.

In #14135 we removed some code that was flattening partially-unknown lists
into fully-unknown (untyped) values. This inadvertently exposed the missed
case from the previous paragraph, causing issues for list-wrapped splat
expressions with unknown members. While this worked fine for resources,
due to some fixup done inside helper/schema, this did not work for other
interpolation contexts such as module blocks.

Various attempts to fix this up and restore the flattening behavior
selectively were unsuccessful, due to a proliferation of assumptions all
over the core code that would be too risky to change just to fix this bug.

This change, then, takes the different approach of removing the
requirement that splats be presented inside list brackets. This
requirement didn't make much sense anymore anyway, since no other
list-returning expression had this constraint and so the rest of Terraform
was already successfully dealing with both cases.

This leaves us with two different scenarios:

- For resource arguments, existing normalization code in helper/schema
  does its own flattening that preserves compatibility with the common
  practice of using bracketed splats. This change proves this with a test
  within the "test" provider that exercises the whole Terraform core and
  helper/schema stack that assigns bracketed splats to list and set
  attributes.

- For arguments in other blocks, such as in module callsites, the
  interpolator's own flattening behavior applies to known lists,
  preserving compatibility with configurations from before
  partially-computed splats were possible, but those wishing to use
  partially-computed splats are required to drop the surrounding brackets.
  This is less concerning because this scenario was introduced only in
  0.9.5, so the scope for breakage is limited to those who adopted this
  new feature quickly after upgrading.

As of this commit, the recommendation is to stop using brackets around
splats but the old form continues to be supported for backward
compatibility. In a future _major_ version of Terraform we will probably
phase out this legacy form to improve consistency, but for now both
forms are acceptable at the expense of some (pre-existing) weird behavior
when _actual_ lists-of-lists are used.

This addresses #14521 by officially adopting the suggested workaround of
dropping the brackets around the splat. However, it doesn't yet allow
passing of a partially-unknown list between modules: that still violates
assumptions in Terraform's core, so for the moment partially-unknown lists
work only within a _single_ interpolation expression, and cannot be
passed around between expressions. Until more holistic work is done to
improve Terraform's type handling, passing a partially-unknown splat
through to a module will result in a fully-unknown list emerging on
the other side, just as was the case before #14135; this change just
addresses the fact that this was failing with an error in 0.9.5.
2017-05-23 11:22:37 -07:00
jasminSPC bca159160e Added missing dependencies 2017-05-22 12:02:32 +02:00
Pradeep Bhadani 2c39887706 Add fail test - one parameter, non-numeric parameter 2017-05-18 14:30:10 +00:00
Pradeep Bhadani 47b1aaea3a Add pow function 2017-05-17 21:46:33 +00:00
Patrick Decat 392166faea Typo: now => no (#14493)
> This validation checks that there are now splat variables referencing ourself. This currently is not allowed.

=> 

> This validation checks that there are no splat variables referencing ourself. This currently is not allowed.
2017-05-15 15:39:55 +03:00
Joshua Spence 38d4d2f9d4 Add a logarithm function
Fixes #9498. Adds a `log` function for calculating logarithms.
2017-05-05 17:06:53 +10:00
Martins Sipenko c7fb9808ef config: sha512 hash functions (#14100) 2017-05-02 16:35:23 -07:00
Martin Atkins 81b0c4b28d config: generate errors for unnamed blocks of various sources
We've been incorrectly validating (or not validating at all) the
requirement that certain blocks be followed by a name string, to prohibit
e.g. this:

    variable {}

and:

    variable = ""

Before this change we were catching this for most constructs only if
there were no _valid_ blocks of the same name in the same file. For
modules in particular, we were not catching this at all.

Now we detect this for all kinds of block (resources had a pre-existing
check, so aren't touched here) and produce a different error message
depending on which of the above incorrect forms are used.

This fixes #13575.
2017-05-02 16:29:57 -07:00
Martin Atkins b1763e262a Restore stringer-generated files back to new version
stringer has changed the boilerplate it generates in a recent version.
We'd previously updated to the new format but accientally rolled back
to the old while merging a long-running feature branch.

This restores us back to the new format again.
2017-04-21 14:49:18 -07:00
Jake Champlin 70bc1e1dcc Merge pull request #12537 from Pryz/coalescelist
Add coalescelist interpolation function
2017-04-21 15:23:21 -04:00
Pavel Khusainov f9fb6010ee config: "matchkeys" interpolation function
This new function allows using a search within one list to filter another list. For example, it can be used to find the ids of EC2 instances in a particular AZ.

The interface is made slightly awkward by the constraints of HIL's featureset.

#13847
2017-04-21 10:04:49 -07:00
Jasmin Gacic 61499cfcf0 Provider Oneandone (#13633)
* Terraform Provider 1&1

* Addressing pull request remarks

* Fixed imports

* Fixing remarks

* Test optimiziation
2017-04-21 17:19:10 +03:00
Justin Campbell c1c3127d8e config: Generate error copy from valid types map
Renders as:

```
Variable 'invalid_type' type must be one of [string, map, list] - 'not_a_type' is not a valid type
```
2017-04-20 13:30:22 -07:00
Justin Campbell cf775ded0e config: Add 'list' to variable type error message 2017-04-20 13:30:22 -07:00
tmshn 86d7c47c0a Change cidrhost() to get IP from end of the range when negative number given
Ref: https://github.com/apparentlymart/go-cidr/pull/2
2017-04-19 09:41:41 -07:00
Joern Barthel 9622b49c45 Support for Windows newlines. 2017-04-07 10:41:55 +02:00
Joern Barthel 059a1b2c0f Added chomp interpolation function. 2017-04-06 13:17:25 +02:00
James Bardin ff2d753062 add Rehash to terraform.BackendState
This method mirrors that of config.Backend, so we can compare the
configration of a backend read from a config vs that of a backend read
from a state. This will prevent init from reinitializing when using
`-backend-config` options that match the existing state.
2017-03-29 15:53:42 -04:00
Martin Atkins 76dca009e0 Allow escaped interpolation-like sequences in variable defaults
The variable validator assumes that any AST node it gets from an
interpolation walk is an indicator of an interpolation. Unfortunately,
back in f223be15 we changed the interpolation walker to emit a LiteralNode
as a way to signal that the result is a literal but not identical to the
input due to escapes.

The existence of this issue suggests a bit of a design smell in that the
interpolation walker interface at first glance appears to skip over all
literals, but it actually emits them in this one situation. In the long
run we should perhaps think about whether the abstraction is right here,
but this is a shallow, tactical change that fixes #13001.
2017-03-29 09:25:57 -07:00
Martin Atkins 21cd5595e2 Update stringer-generated files to new boilerplate
golang/tools commit 23ca8a263 changed the format of the leading comment
to comply with some new standards discussed here:
https://golang.org/issue/13560

This is the result of running generate with the latest version of
stringer. Everyone working on Terraform will need to update stringer
after this is merged, to avoid reverting this:
    go get -u golang.org/x/tools/cmd/stringer
2017-03-29 08:07:06 -07:00
Joshua Spence af2c84de5a Add `basename` and `dirname` functions
Adds `basename` and `dirname` interpolation. I want to add a `stack` tag to our infrastructure, the value of which is set to `${basename(path.cwd)}`. We currently use `${replace(path.cwd, "/^.+\\//", "")}` instead, but this is extremeley unreadable. The existance of a `basename` function would be very useful for this use case.

I don't have an immediate use case for a `dirname` function, but it seemed reasonable to add it as well.
2017-03-28 09:29:26 -07:00
Joshua Spence e71d6d92ad Add a substring interpolation function (#12870)
Adds a new `substr` interpolation function which can be used to truncate a string.
2017-03-22 11:30:39 -04:00
James Bardin 579e15c97c Merge pull request #12942 from hashicorp/jbardin/GH-12905
merge config.Terraform fields in config.Append
2017-03-22 09:11:42 -04:00
James Bardin bcbcc65f7d add terraform config merge logic to config.Merge 2017-03-22 09:01:54 -04:00
James Bardin 8bcb9e19ca restructure JSON terraform config block AST
When configuration is read out of JSON, HCL assumes that empty levels of
objects can be flattened, but this removes too much to decode into a
config.Terraform struct.

Reconstruct the appropriate AST to decode the config struct.
2017-03-21 18:15:58 -04:00
James Bardin b38e620b2f merge config.Terraform fields in config.Append
Ensure that fields set in an earlier Terraform config block aren't
removed by Append when encountering another Terraform block. When
multiple blocks contain the same field, the later one still wins.
2017-03-21 15:43:55 -04:00
Benjamin Boudreau 073fa873ac Fix receive typo (#12881) 2017-03-20 13:15:27 -04:00
Mitchell Hashimoto 68ee4e0480
config/module: don't panic when referencing undefined module
Fixes #12788

We would panic when referencing an output from an undefined module. The
panic above this is correct but in this case Load will not catch
interpolated variables that _reference_ an unloaded/undefined module.
Test included.
2017-03-16 20:14:20 -07:00
Mitchell Hashimoto f7da5d323c
config: test that JSON loading terraform backend info works 2017-03-16 14:51:26 -07:00
James Bardin 2e3579b058 Merge pull request #12383 from hashicorp/jbardin/multi-var-errs
report all errors from module validation
2017-03-14 15:36:32 -04:00
Mitchell Hashimoto e2ca2c5911
config: allow TerraformVars in count 2017-03-13 16:38:54 -07:00
Mitchell Hashimoto 786334b643
config: parse TerraformVariables 2017-03-13 16:09:06 -07:00
Pryz 733f1ca1e7 Add coalescelist interpolation function 2017-03-08 12:38:08 -08:00
James Bardin a111635908 Fix panic in interpolate_walk
Verify that we have enough containers in the stack to look for a map in
replaceCurrent.
2017-03-07 15:35:40 -05:00
James Bardin 3c41a7ca1e Add test for Validate crash
Crash during Validate walk with nested variable default.
2017-03-07 15:01:29 -05:00
James Bardin 2a949093ed report all errors from module validation
It can be tedious fixing a new module with many errors when Terraform
only outputs the first random error it encounters.

Accumulate all errors from validation, and format them for the user.
2017-03-02 15:16:05 -05:00
Mitchell Hashimoto b502643863 Merge pull request #12067 from hashicorp/b-backend-interp
config: validate backend configuration can't contain interpolations
2017-02-26 21:44:45 -08:00
John Murphy 628d46ac53 Fixed broken build for netbsd 2017-02-22 00:30:18 +08:00
Mitchell Hashimoto 6b4c007894
config: validate backend configuration can't contain interpolations 2017-02-17 16:27:01 -08:00
Kit Ewbank 549cff56d0 Add 'slice' interpolation function. (#9729) 2017-02-13 21:20:02 +00:00
Mitchell Hashimoto b6bfc4798d
config: Resource.Count should type check
Fixes #11800

Type check the value of count so we don't panic on the conversion.

I wondered "why didn't we do this before?" There is no excuse for NOT
doing it at all but the reasoning was beacuse prior to the list/map work
in 0.7, the value couldn't be anything other than a string since any
primitive can turn into a string.

Regardless, we should've always done this.
2017-02-10 10:41:41 -08:00
Mitchell Hashimoto cf46e1c3e0
terraform: don't validate computed values in validate
This disables the computed value check for `count` during the validation
pass. This enables partial support for #3888 or #1497: as long as the
value is non-computed during the plan, complex values will work in
counts.

**Notably, this allows data source values to be present in counts!**

The "count" value can be disabled during validation safely because we
can treat it as if any field that uses `count.index` is computed for
validation. We then validate a single instance (as if `count = 1`) just
to make sure all required fields are set.
2017-01-27 21:15:43 -08:00
Mitchell Hashimoto 7b342100d0
config: add "backend" loading to the Terraform section 2017-01-26 14:33:49 -08:00
Mitchell Hashimoto 928fce71f7
config: parse "when" and "on_failure" on provisioners 2017-01-19 18:10:21 -08:00
Jake Champlin 0b73b92830
core: Add pathexpand interpolation function
Adds the `pathexpand` interpolation function to allow users to expand `~` to the home directory in filepath strings.
2017-01-18 16:11:19 -05:00
Mitchell Hashimoto 095b7e7831
config/module: disallow root modules named "root"
Fixes #11038

This is a **short term fix**.

Terraform core doesn't currently handle root modules named "root" well
because the prefix `[]string{"root"}` has special meaning and Terraform
core [currently] can't disambiguate between the root module and a module
named "root" in the root module.

This PR introduces a short term fix by simply disallowing root modules
named "root". This shouldn't break any BC because since 0.8.0 this
didn't work at all in many broken ways (including crashes).

Longer term, this should be fixed by removing the special prefix at all
and having empty paths be root. I started down this path but the core
changes necessary are far too scary for a patch release. We can aim for
0.9.
2017-01-08 15:39:57 -08:00
Mitchell Hashimoto 0c30caec7d
config: smarter provider alias usage validation
Fixes #4789

This improves the validation that valid provider aliases are used.

Previously, we required that provider aliases be defined in every module
they're used. This isn't correct because the alias may be used in a
parent module and inherited.

This removes that validation and creates the validation that a provider
alias must be defined in the used module or _any parent_. This allows
inheritance to work properly.

We've always had this type of validation for aliases because we believe
its a good UX tradeoff: typo-ing an alias is really painful, so we
require declaration of alias usage. It may add a small burden to
declare, but since relatively few aliases are used, it improves the
scenario where a user fat-fingers an alias name.
2016-12-16 16:47:32 -08:00
Mitchell Hashimoto c2c5668a8d
config: Append supports `terraform` 2016-12-13 21:53:02 -08:00
Mitchell Hashimoto 3878b8b093
config: Merge respects Terraform blocks, provider aliases, and more
Fixes #10715

`config.Merge` was not updated to support a number of new features. This
updates the codepath to merge various fields, including the `terraform`
block which was the issue in #10715.

The `Merge` API is called when an `_override` file is present to _merge_
configurations. Normally configurations are _appended_. Only an override
file triggers a _merge_.

I started working on a generic library to do this automatically awhile
back but never finished it. This might motivate me to do so. In the
interest of getting a fix out though, we'll continue the manual
approach.
2016-12-13 21:48:59 -08:00
Mitchell Hashimoto 8a102799c2 Merge pull request #10658 from hashicorp/b-var-keys
config: validate invalid variable keys
2016-12-12 10:53:07 -08:00
Mitchell Hashimoto 3ba9720b3e
config: validate invalid variable keys
Fixes #9416

A simple change to verify that only valid keys for `variable` blocks are
used.
2016-12-10 19:27:01 -05:00
Mitchell Hashimoto 5d684b399c
config: error when loading multiple lifecycle blocks
Fixes #8776

This introduces an error when multiple `lifecycle` blocks exist on a
resource in the configuration.
2016-12-10 18:52:13 -05:00
Mitchell Hashimoto b5f1738e17
config: allow IDs to start with numbers 2016-12-09 12:35:23 -05:00
Mitchell Hashimoto 8b9e2c17cc
config: fix NameRegexp validation to allow single-char names 2016-12-08 23:13:19 -05:00
Mitchell Hashimoto 6b458160b9
config: disallow names starting with ints
Fixes #10597

This disallows any names for variables, modules, etc. starting with
ints. This causes parse errors with the new HIL parser and actually
causes long term ambiguities if we allow this.

I've also updated the upgrade guide to note this as a backwards
compatibility and how people can fix this going forward.
2016-12-08 23:01:51 -05:00
Mitchell Hashimoto 14b371d533
config: validate that RawConfig.Copy doesn't copy the interpolated
values
2016-12-02 13:25:32 -05:00
Mitchell Hashimoto 692eadd763
config: nitpicks from #10475
/cc @grubernaut - we put stdlibs above 3rd party libs separated by a
space
2016-12-01 18:02:39 -05:00
Jake Champlin 1f33952330
Add the timestamp interpolation function.
Adds the timestamp interpolation function, tests, and documentation to allow a user to insert an RFC 3339 formatted UTC timestamp.
2016-12-01 14:51:01 -05:00
Mitchell Hashimoto 3665fea2db
config: validate that data sources don't have provisioners 2016-11-23 08:46:13 -08:00
Mitchell Hashimoto 9cc52d83de
config: test that null characters show up as errors in parse 2016-11-21 18:11:11 -08:00
Martin Atkins 399542a168 core: allow outputs to have descriptions (#9722)
We allow variables to have descriptions specified, as additional context
for a module user as to what should be provided for a given variable.

We previously lacked a similar mechanism for outputs. Since they too are
part of a module's public interface, it makes sense to be able to add
descriptions for these for symmetry's sake.

This change makes a "description" attribute valid within an "output"
configuration block and stores it within the configuration data structure,
but doesn't yet do anything further with it. For now this is useful only
for third-party tools that might parse a module's config to generate
user documentation; later we could expose the descriptions as part of
the "apply" output, but that is left for a separate change.
2016-11-18 19:09:43 +02:00
Mitchell Hashimoto b3f80b9469
config: maintain slice index accounting for computed keys
Fixes #10075
Fixes #10013

When interpolating, we were only maintaining the last known slice index.
If you had sibling slices then you could lose your slice index when
exiting the slice. The resulting behavior was that no some runs the
computed key would be: "slice.0.attr" and on others would be
"slice.attr", the latter being incorrect.

We now maintain a list of slice indexes so that as we unnest, we
properly restore the old value.

Surprisingly unrelated to the graph but the shadow graph caught this
which is great. :)
2016-11-14 18:33:29 -08:00
Mitchell Hashimoto 25d19ef3d0 Merge pull request #10080 from hashicorp/f-tf-version
terraform: support version requirement in configuration
2016-11-14 11:53:30 -08:00
Mitchell Hashimoto df34fa88ce Merge pull request #10076 from hashicorp/f-depend-module
terraform: depends_on can reference entire modules
2016-11-14 11:53:12 -08:00
Mitchell Hashimoto e3a01ccfd8 Merge pull request #10072 from hashicorp/f-output-depends-on
terraform: output nodes can have `depends_on`
2016-11-14 11:52:18 -08:00
Mitchell Hashimoto 15979435d6
config: update error string for new HIL merge 2016-11-13 10:28:31 -08:00
Mitchell Hashimoto 85d3439fa0
config: parse and validate terraform.required_version 2016-11-12 16:22:35 -08:00
Mitchell Hashimoto 576b61a21d
config: validate depends_on with module values 2016-11-12 08:21:27 -08:00