Commit Graph

512 Commits

Author SHA1 Message Date
Mitchell Hashimoto 706b2e2aea Merge pull request #8482 from hashicorp/b-output-dup
config: variable names and outputs must be unique
2016-08-26 13:57:59 -07:00
Mitchell Hashimoto 0fceeaaeb0
config: test for var uniqueness in overrides 2016-08-26 13:48:21 -07:00
Mitchell Hashimoto 4e7f2dd01e Merge pull request #8398 from hashicorp/b-module-collide
config/module: use source as part of key
2016-08-26 13:27:56 -07:00
Mitchell Hashimoto 8a3559560d
config: JSON resource keys with only one item load properly GH-5140
When a resource has only a single key set, the HCL parser treats that
key as part of the overall set of object keys. This isn't valid since
we expect resources to have exactly two keys. In this scenario, we have
to "unwrap" the keys back into a set of objects.
2016-08-25 17:18:18 -07:00
Mitchell Hashimoto fbf06e2a59
config: vars must be unique 2016-08-25 14:51:49 -07:00
Mitchell Hashimoto 099293b690
config: outputs must be unique 2016-08-25 14:43:57 -07:00
Mitchell Hashimoto 92336281a5
config/module: use source as part of key 2016-08-22 14:07:28 -07:00
Mitchell Hashimoto f4faf2274b
config: count can't be a SimpleVariable 2016-08-16 13:48:12 -07:00
James Bardin 81bb6b7264 Merge pull request #7905 from hashicorp/jbardin/merge
core: Add merge interpolation function
2016-08-02 09:45:20 -04:00
James Bardin 39bbbb8da6 Add merge interpolation function
Add a `merge` interpolation function, which merges any number of maps.
Duplicate keys are OK, with the last write winning.
2016-08-01 18:30:58 -04:00
James Bardin 1af7ee87a2 Silence log output when not verbose
Set the default log package output to iotuil.Discard during tests if the
`-v` flag isn't set. If we are verbose, then apply the filter according
to the TF_LOG env variable.
2016-08-01 17:19:14 -04:00
James Bardin c9e1522103 Use HIL to limit concat to ast.TypeList
we can remove some type checks in the concat function
2016-08-01 15:24:18 -04:00
James Bardin c15c0eb0cb Disallow strings as arguments to concat
The concat interpolation function now only accepts list arguments.
Strings are no longer supported, for concatenation or appending to
lists. All arguments must be a list, and single elements can be promoted
with the `list` interpolation function.
2016-08-01 15:06:20 -04:00
James Nugent 796ce7bdd7 build: Opt-in to building rather than opt-out 2016-07-29 18:40:31 -05:00
James Nugent b2d5b66508 Merge pull request #7880 from hashicorp/b-fix-freebsd
build: Fix errors in FreeBSD build
2016-07-29 18:30:55 -05:00
James Nugent 2eb00c9184 build: Fix errors in FreeBSD build
Fixes the following error when cross compiling:

```
--> freebsd/amd64 error: exit status 2
Stderr: # github.com/hashicorp/terraform/config/module
config/module/inode.go:18: cannot use st.Ino (type uint32) as type uint64 in return argument
```
2016-07-29 18:26:22 -05:00
James Bardin 5802f76eaa Make all terraform package tests pass under -race
This isn't a pretty refactor, but fixes the race issues in this package
for now.

Fix race on RawConfig.Config()

fix command package races
2016-07-29 16:12:21 -04:00
Paul Hinze 88030764ff
config: Audit all interpolation functions for list/map behavior
- `distinct()` - error on non-flat lists
 - `element()` - error on non-flat lists
 - `join()` - error on non-flat lists
 - `length()` - support maps
 - `lookup()` - error on non-flat maps
 - `values()` - error on non-flat maps
2016-07-28 09:56:30 -05:00
Paul Hinze 1425b34562
config: Add map() interpolation function
* `map(key, value, ...)` - Returns a map consisting of the key/value pairs
  specified as arguments. Every odd argument must be a string key, and every
  even argument must have the same type as the other values specified.
  Duplicate keys are not allowed. Examples:
  * `map("hello", "world")`
  * `map("us-east", list("a", "b", "c"), "us-west", list("b", "c", "d"))`
2016-07-27 13:03:08 -05:00
James Bardin 8dcbc0b0a0 Add concat to accept lists of lists and maps
This will allow the concat interpolation function to accept lists of
lists, and lists of maps as well as strings. We still allow bare strings
for backwards compatibility, but remove some of the old comment wording
as it could cause confusion of this function with actual string
concatenation.

Since maps are now supported in the config, this removes the superfluous
(and failing) TestInterpolationFuncConcatListOfMaps.
2016-07-19 17:45:50 -04:00
James Bardin 2bd7cfd5fe Expand list interpolation to lists and maps
Allow lists and maps within the list interpolation function via variable
interpolation. Since this requires setting the variadic type to TypeAny,
we check for non-heterogeneous lists in the callback.
2016-07-19 13:44:37 -04:00
James Nugent 58dd41f3b1 core: Add list() interpolation function
The list() interpolation function provides a way to add support for list
literals (of strings) to HIL without having to invent new syntax for it
and modify the HIL parser.

It presents as a function, thus:

    - list() -> []
    - list("a") -> ["a"]
    - list("a", "b") -> ["a", "b"]

Thanks to @wr0ngway for the idea of this approach, fixes #7460.
2016-07-18 18:12:11 -04:00
James Nugent c6e03cba96 core: Fix slice element keys on interpolateWalk
Part of the interpolation walk is to detect keys which involve computed
values and therefore cannot be resolved at this time. The interplation
walker keeps sufficient state to be able to populate the ResourceConfig
with a slice of such keys.

Previously they didn't take slice indexes into account, so in the
following case:

```
"services": []interface{}{
    map[string]interface{}{
        "elb": "___something computed___",
    },
    map[string]interface{}{
        "elb": "___something else computed___",
    },
    map[string]interface{}{
        "elb": "not computed",
    },
}
```

Unknown keys would be populated as follows:

```
services.elb
services.elb
```

This is not sufficient information to be useful, as it is impossible to
distinguish which of the `services.elb`s are unknown vs not.

This commit therefore retains the slice indexes as part of the key for
unknown keys - producing for the example above:

```
services.0.elb
services.1.elb
```
2016-07-08 16:43:42 +01:00
James Bardin e78dc291af Add test to ensure key/values interp order
Test that interpolated values are returned in the order of the sorted
keys.
2016-06-29 15:06:59 -04:00
James Nugent 8284b79957 Merge pull request #7298 from hashicorp/jbardin/GH-7273-fixup
code: Add proper build constraints for GH-7273
2016-06-24 14:33:10 +03:00
James Nugent 8403a465bc core: Add test and fix for element with empty list
The incldued test previously caused a panic, it now returns an error
message explaining the issue.
2016-06-23 21:15:33 +01:00
James Bardin 525485c213 Add proper build constraints for GH-7273
The syscall.Stat_t type doesn't exist on windows, so the inode lookup
needs to be in a file with proper build constraints.
2016-06-23 08:02:32 -04:00
James Bardin 558e023e26 Don't try to copy files over themselves
When copying a config module, make sure the full path for src and dst
files don't match, and also check the inode in case we resolved a
different path to the same file.

Make a note about the unsafe usage of reusing a tempDir path.
2016-06-22 11:25:42 -04:00
James Nugent 9d6c629689 config: Update test to reflect new behaviour
Escaped quotes are no longer supported as HIL syntax (as of the last
update to HIL), so this commit changes the Terraform config-layer test
to verify the non-presence of this behaviour for 0.7.
2016-06-20 19:06:03 +03:00
James Nugent 4b6a632246 core: Rename uniq -> distinct and add docs 2016-06-15 13:25:17 +02:00
Jan Schumann df3e017f6c fix #7106 2016-06-15 13:25:16 +02:00
Paul Hinze 9bc980f569
core: Fix panic on concat() w/ list of nonprimitives
The `concat()` interpolation function does not yet support types other
than strings / lists of strings. Make it an error message instead of a
panic when a list of non-primitives is supplied.

Fixes the panic in #7030
2016-06-12 13:29:06 -05:00
Paul Hinze ffa29090ec
core: Better error for dot indexing on user vars
Dot indexing worked in the "regexps and strings" world of 0.6.x, but it
no longer works on the 0.7 series w/ proper List / Map types.

There is plenty of dot-indexed config out in the wild, so we need to do
what we can to point users to the new syntax.

Here is one place we can do it for user variables (`var.somemap`). We'll
also need to address Resource Variables and Module Variables in a
separate PR.

This fixes the panic in #7103 - a proper error message is now returned.
2016-06-12 10:45:48 -05:00
James Nugent 01cd596c60 core: Fix detection of empty list/map defaults
This commit changes config parsing from weak decoding lists and maps
into []string and map[string]string respectively to decode into
[]interface{} and map[string]interface{} respectively. This is in order
to take advantage of the work integrated in #7082 to defeat the backward
compatibility features of the mapstructure library.

Test coverage of loading empty variables and validating their default
types against expectation.
2016-06-12 11:19:03 +02:00
James Nugent 578ff9569e core: Add sort() interpolation function 2016-06-11 18:03:52 +01:00
James Nugent cb9ef298f3 core: Defeat backward compatibilty in mapstructure
The mapstructure library has a regrettable backward compatibility
concern whereby a WeakDecode of []interface{}{} into a target of
map[string]interface{} yields an empty map rather than an error. One
possibility is to switch to using Decode instead of WeakDecode, but this
loses the nice handling of type conversion, requiring a large volume of
code to be added to Terraform or HIL in order to retain that behaviour.

Instead we add a DecodeHook to our usage of the mapstructure library
which checks for decoding []interface{}{} or []string{} into a map and
returns an error instead.

This has the effect of defeating the code added to retain backwards
compatibility in mapstructure, giving us the correct (for our
circumstances) behaviour of Decode for empty structures and the type
conversion of WeakDecode.

The code is identical to that in the HIL library, and packaged into a
helper.
2016-06-08 18:38:41 +01:00
David Adams b5d1279107 Allow specifying a default value to lookup()
Fixes #4474, where lookup() calls fail out the entire interpolation when
the provided key value is not found in the map. This will allow using
coalesce() along with lookup() to greatly improve module flexibility.
2016-05-25 19:25:15 -05:00
David Glasser 594ea105d8 config: support lists and maps in jsonencode
For now we only support lists and maps whose values are strings, not
deeply nested data.
2016-05-18 10:46:06 -07:00
Martin Atkins 60c24e3319 command: Prevent data resources from being tainted
Since the data resource lifecycle contains no steps to deal with tainted
instances, we must make sure that they never get created.

Doing this out in the command layer is not the best, but this is currently
the only layer that has enough information to make this decision and so
this simple solution was preferred over a more disruptive refactoring,
under the assumption that this taint functionality eventually gets
reworked in terms of StateFilter anyway.
2016-05-14 08:26:37 -07:00
Martin Atkins 718cdda77b config: Parsing of data.TYPE.NAME.FIELD variables
This allows ${data.TYPE.NAME.FIELD} interpolation syntax at the
configuration level, though since there is no special handling of them
in the core package this currently just acts as an alias for
${TYPE.NAME.FIELD}.
2016-05-14 08:26:35 -07:00
Martin Atkins 860140074f config: Data source loading
This allows the config loader to read "data" blocks from the config and
turn them into DataSource objects.

This just reads the data from the config file. It doesn't validate the
data nor do anything useful with it.
2016-05-14 08:26:35 -07:00
Martin Atkins fc4fa10981 config: "ResourceMode" concept for resources
Previously resources were assumed to always support the full set of
create, read, update and delete operations, and Terraform's resource
management lifecycle.

Data sources introduce a new kind of resource that only supports the
"read" operation. To support this, a new "Mode" field is added to
the Resource concept within the config layer, which can be set to
ManagedResourceMode (to indicate the only mode previously possible) or
DataResourceMode (to indicate that only "read" is supported).

To support both managed and data resources in the tests, the
stringification of resources in config_string.go is adjusted slightly
to use the Id() method rather than the unusual type[name] serialization
from before, causing a simple mechanical adjustment to the loader tests'
expected result strings.
2016-05-14 08:26:35 -07:00
Mitchell Hashimoto 27452f0043
terraform: Module option to Import to add module to graph 2016-05-11 13:02:37 -07:00
Mitchell Hashimoto 3c9a92e04a
command: Context buliding allows empty module trees 2016-05-11 13:02:34 -07:00
James Nugent f1d0fc46aa core: Fix go vet issues shown by Travis 2016-05-10 16:00:28 -04:00
James Nugent 244da895cd core: Remove StringList
Much celebration may now ensue! ♪┏(°.°)┛┗(°.°)┓┗(°.°)┛┏(°.°)┓ ♪
2016-05-10 14:49:14 -04:00
James Nugent f49583d25a core: support native list variables in config
This commit adds support for native list variables and outputs, building
up on the previous change to state. Interpolation functions now return
native lists in preference to StringList.

List variables are defined like this:

variable "test" {
    # This can also be inferred
    type = "list"
    default = ["Hello", "World"]
}

output "test_out" {
    value = "${var.a_list}"
}
This results in the following state:

```
...
            "outputs": {
                "test_out": [
                    "hello",
                    "world"
                ]
            },
...
```

And the result of terraform output is as follows:

```
$ terraform output
test_out = [
  hello
  world
]
```

Using the output name, an xargs-friendly representation is output:

```
$ terraform output test_out
hello
world
```

The output command also supports indexing into the list (with
appropriate range checking and no wrapping):

```
$ terraform output test_out 1
world
```

Along with maps, list outputs from one module may be passed as variables
into another, removing the need for the `join(",", var.list_as_string)`
and `split(",", var.list_as_string)` which was previously necessary in
Terraform configuration.

This commit also updates the tests and implementations of built-in
interpolation functions to take and return native lists where
appropriate.

A backwards compatibility note: previously the concat interpolation
function was capable of concatenating either strings or lists. The
strings use case was deprectated a long time ago but still remained.
Because we cannot return `ast.TypeAny` from an interpolation function,
this use case is no longer supported for strings - `concat` is only
capable of concatenating lists. This should not be a huge issue - the
type checker picks up incorrect parameters, and the native HIL string
concatenation - or the `join` function - can be used to replicate the
missing behaviour.
2016-05-10 14:49:14 -04:00
James Nugent e57a399d71 core: Use native HIL maps instead of flatmaps
This changes the representation of maps in the interpolator from the
dotted flatmap form of a string variable named "var.variablename.key"
per map element to use native HIL maps instead.

This involves porting some of the interpolation functions in order to
keep the tests green, and adding support for map outputs.

There is one backwards incompatibility: as a result of an implementation
detail of maps, one could access an indexed map variable using the
syntax "${var.variablename.key}".

This is no longer possible - instead HIL native syntax -
"${var.variablename["key"]}" must be used. This was previously
documented, (though not heavily used) so it must be noted as a backward
compatibility issue for Terraform 0.7.
2016-05-10 14:49:13 -04:00
James Nugent b62f6af158 core: Add support for marking outputs as sensitive (#6559)
* core: Add support for marking outputs as sensitive

This commit allows an output to be marked "sensitive", in which case the
value is redacted in the post-refresh and post-apply list of outputs.

For example, the configuration:

```
variable "input" {
    default = "Hello world"
}

output "notsensitive" {
    value = "${var.input}"
}

output "sensitive" {
    sensitive = true
    value = "${var.input}"
}
```

Would result in the output:

```
terraform apply

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

  notsensitive = Hello world
  sensitive    = <sensitive>
```

The `terraform output` command continues to display the value as before.

Limitations: Note that sensitivity is not tracked internally, so if the
output is interpolated in another module into a resource, the value will
be displayed. The value is still present in the state.
2016-05-09 15:46:07 -04:00
James Nugent a0cc7115b3 deps: Update call sites of hil.Eval from update
hil.Eval() now returns (hil.EvaluationResult, error) instead of (value,
type, error). This commit updates the call sites, but retains all
previous behaviour. Tests are also updated.
2016-04-18 16:37:12 -07:00