Commit Graph

527 Commits

Author SHA1 Message Date
James Nugent cb9ef298f3 core: Defeat backward compatibilty in mapstructure
The mapstructure library has a regrettable backward compatibility
concern whereby a WeakDecode of []interface{}{} into a target of
map[string]interface{} yields an empty map rather than an error. One
possibility is to switch to using Decode instead of WeakDecode, but this
loses the nice handling of type conversion, requiring a large volume of
code to be added to Terraform or HIL in order to retain that behaviour.

Instead we add a DecodeHook to our usage of the mapstructure library
which checks for decoding []interface{}{} or []string{} into a map and
returns an error instead.

This has the effect of defeating the code added to retain backwards
compatibility in mapstructure, giving us the correct (for our
circumstances) behaviour of Decode for empty structures and the type
conversion of WeakDecode.

The code is identical to that in the HIL library, and packaged into a
helper.
2016-06-08 18:38:41 +01:00
David Adams b5d1279107 Allow specifying a default value to lookup()
Fixes #4474, where lookup() calls fail out the entire interpolation when
the provided key value is not found in the map. This will allow using
coalesce() along with lookup() to greatly improve module flexibility.
2016-05-25 19:25:15 -05:00
David Glasser 594ea105d8 config: support lists and maps in jsonencode
For now we only support lists and maps whose values are strings, not
deeply nested data.
2016-05-18 10:46:06 -07:00
Martin Atkins 60c24e3319 command: Prevent data resources from being tainted
Since the data resource lifecycle contains no steps to deal with tainted
instances, we must make sure that they never get created.

Doing this out in the command layer is not the best, but this is currently
the only layer that has enough information to make this decision and so
this simple solution was preferred over a more disruptive refactoring,
under the assumption that this taint functionality eventually gets
reworked in terms of StateFilter anyway.
2016-05-14 08:26:37 -07:00
Martin Atkins 718cdda77b config: Parsing of data.TYPE.NAME.FIELD variables
This allows ${data.TYPE.NAME.FIELD} interpolation syntax at the
configuration level, though since there is no special handling of them
in the core package this currently just acts as an alias for
${TYPE.NAME.FIELD}.
2016-05-14 08:26:35 -07:00
Martin Atkins 860140074f config: Data source loading
This allows the config loader to read "data" blocks from the config and
turn them into DataSource objects.

This just reads the data from the config file. It doesn't validate the
data nor do anything useful with it.
2016-05-14 08:26:35 -07:00
Martin Atkins fc4fa10981 config: "ResourceMode" concept for resources
Previously resources were assumed to always support the full set of
create, read, update and delete operations, and Terraform's resource
management lifecycle.

Data sources introduce a new kind of resource that only supports the
"read" operation. To support this, a new "Mode" field is added to
the Resource concept within the config layer, which can be set to
ManagedResourceMode (to indicate the only mode previously possible) or
DataResourceMode (to indicate that only "read" is supported).

To support both managed and data resources in the tests, the
stringification of resources in config_string.go is adjusted slightly
to use the Id() method rather than the unusual type[name] serialization
from before, causing a simple mechanical adjustment to the loader tests'
expected result strings.
2016-05-14 08:26:35 -07:00
Mitchell Hashimoto 27452f0043
terraform: Module option to Import to add module to graph 2016-05-11 13:02:37 -07:00
Mitchell Hashimoto 3c9a92e04a
command: Context buliding allows empty module trees 2016-05-11 13:02:34 -07:00
James Nugent f1d0fc46aa core: Fix go vet issues shown by Travis 2016-05-10 16:00:28 -04:00
James Nugent 244da895cd core: Remove StringList
Much celebration may now ensue! ♪┏(°.°)┛┗(°.°)┓┗(°.°)┛┏(°.°)┓ ♪
2016-05-10 14:49:14 -04:00
James Nugent f49583d25a core: support native list variables in config
This commit adds support for native list variables and outputs, building
up on the previous change to state. Interpolation functions now return
native lists in preference to StringList.

List variables are defined like this:

variable "test" {
    # This can also be inferred
    type = "list"
    default = ["Hello", "World"]
}

output "test_out" {
    value = "${var.a_list}"
}
This results in the following state:

```
...
            "outputs": {
                "test_out": [
                    "hello",
                    "world"
                ]
            },
...
```

And the result of terraform output is as follows:

```
$ terraform output
test_out = [
  hello
  world
]
```

Using the output name, an xargs-friendly representation is output:

```
$ terraform output test_out
hello
world
```

The output command also supports indexing into the list (with
appropriate range checking and no wrapping):

```
$ terraform output test_out 1
world
```

Along with maps, list outputs from one module may be passed as variables
into another, removing the need for the `join(",", var.list_as_string)`
and `split(",", var.list_as_string)` which was previously necessary in
Terraform configuration.

This commit also updates the tests and implementations of built-in
interpolation functions to take and return native lists where
appropriate.

A backwards compatibility note: previously the concat interpolation
function was capable of concatenating either strings or lists. The
strings use case was deprectated a long time ago but still remained.
Because we cannot return `ast.TypeAny` from an interpolation function,
this use case is no longer supported for strings - `concat` is only
capable of concatenating lists. This should not be a huge issue - the
type checker picks up incorrect parameters, and the native HIL string
concatenation - or the `join` function - can be used to replicate the
missing behaviour.
2016-05-10 14:49:14 -04:00
James Nugent e57a399d71 core: Use native HIL maps instead of flatmaps
This changes the representation of maps in the interpolator from the
dotted flatmap form of a string variable named "var.variablename.key"
per map element to use native HIL maps instead.

This involves porting some of the interpolation functions in order to
keep the tests green, and adding support for map outputs.

There is one backwards incompatibility: as a result of an implementation
detail of maps, one could access an indexed map variable using the
syntax "${var.variablename.key}".

This is no longer possible - instead HIL native syntax -
"${var.variablename["key"]}" must be used. This was previously
documented, (though not heavily used) so it must be noted as a backward
compatibility issue for Terraform 0.7.
2016-05-10 14:49:13 -04:00
James Nugent b62f6af158 core: Add support for marking outputs as sensitive (#6559)
* core: Add support for marking outputs as sensitive

This commit allows an output to be marked "sensitive", in which case the
value is redacted in the post-refresh and post-apply list of outputs.

For example, the configuration:

```
variable "input" {
    default = "Hello world"
}

output "notsensitive" {
    value = "${var.input}"
}

output "sensitive" {
    sensitive = true
    value = "${var.input}"
}
```

Would result in the output:

```
terraform apply

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

  notsensitive = Hello world
  sensitive    = <sensitive>
```

The `terraform output` command continues to display the value as before.

Limitations: Note that sensitivity is not tracked internally, so if the
output is interpolated in another module into a resource, the value will
be displayed. The value is still present in the state.
2016-05-09 15:46:07 -04:00
James Nugent a0cc7115b3 deps: Update call sites of hil.Eval from update
hil.Eval() now returns (hil.EvaluationResult, error) instead of (value,
type, error). This commit updates the call sites, but retains all
previous behaviour. Tests are also updated.
2016-04-18 16:37:12 -07:00
James Nugent d7d39702c0 Type check variables between modules (#6185)
These tests demonstrates a problem where the types to a module input are 
not checked. For example, if a module - inner - defines a variable
"should_be_a_map" as a map, or with a default variable of map, we do not
fail if the user sets the variable value in the outer module to a string
value. This is also a problem in nested modules.

The implementation changes add a type checking step into the graph
evaluation process to ensure invalid types are not passed.
2016-04-15 12:07:54 -07:00
David Glasser 6cf06bb3ab config: new interpolation function jsonencode 2016-03-29 07:38:58 -07:00
Paul Hinze 293c6ca68c Revert "Revert "core: Add uuid() interpolate function.""
This reverts commit 661be01d9b.
2016-03-21 15:14:30 -05:00
Paul Hinze 567a9b9e06 config: remove missing equals test to fix build
This is behavior that's covered in the parser now - and the error
message is nicer to boot!
2016-03-21 10:39:20 -05:00
James Nugent 87550b2b72 Merge pull request #5263 from uber/b-element-negative
Error out on negative indices in element()
2016-03-16 09:39:35 +00:00
Paul Hinze 661be01d9b Revert "core: Add uuid() interpolate function." 2016-03-15 18:39:34 -05:00
Paul Hinze 1e0b8ea478 core: Add uuid() interpolate function.
Utilizes hashicorp's go-uuid library for proper random seeding setup.
2016-03-10 18:39:07 -06:00
Radek Simko 664ba5f5a6 config: Add new interpolation function - md5 2016-02-24 13:01:05 +00:00
Radek Simko 573d3bd7ab config: Sort functions mapping alphabetically 2016-02-24 13:01:05 +00:00
Bill Fumerola c0034e672b Error out on negative indices in element() 2016-02-22 15:58:47 -08:00
Jeff Zohrab 33d30761dd Add hint to download modules for new users. 2016-02-17 11:32:44 -05:00
Paul Hinze 9a00675262 Merge pull request #5026 from hashicorp/phinze/destroy-node-copies
core: Make copies when creating destroy nodes
2016-02-09 11:12:01 -06:00
Paul Hinze 3f72837f4b core: Make copies when creating destroy nodes
Fixes an interpolation race that was occurring when a tainted destroy
node and a primary destroy node both tried to interpolate a computed
count in their config. Since they were sharing a pointer to the _same_
config, depending on how the race played out one of them could catch the
config uninterpolated and would then throw a syntax error.

The `Copy()` tree implemented for this fix can probably be used
elsewhere - basically we should copy the config whenever we drop nodes
into the graph - but for now I'm just applying it to the place that
fixes this bug.

Fixes #4982 - Includes a test covering that race condition.
2016-02-09 09:25:16 -06:00
Radek Simko 4edf782260 Merge pull request #4854 from jfromaniello/add_signum_interpolation
Add signum interpolation function
2016-02-07 19:44:16 +00:00
Mitchell Hashimoto 5f3de02fa9 remove config/lang, use hashicorp/hil 2016-02-03 13:24:04 -05:00
Radek Simko ecedcd0032 config: Add base64sha256() function 2016-01-30 13:19:10 +01:00
Colin Hebert 61a40dce13 Update the test file 2016-01-30 20:52:45 +11:00
Colin Hebert d92d205dd9 rename trim to trimspace 2016-01-30 20:51:28 +11:00
Colin Hebert d45b7b2ddc Fix name from strip to trim 2016-01-30 10:32:18 +11:00
Colin Hebert f5074cd521 Add the trim() interpolation function 2016-01-30 10:28:04 +11:00
José F. Romaniello c8795b8565 Add signum interpolation function
This function returns -1 for negative numbers, 0 for 0 and 1 for positive numbers.

Useful when you need to set a value for the first resource and a different value for the rest of the resources.

Example: `${element(split(",", var.r53_failover_policy), signum(count.index))}`
2016-01-27 12:49:52 -03:00
James Nugent 3bdd1ac8b3 core: Display invalid type in error message
If a variable type which is invalid (e.g. "stringg") is declared, we now
include the invalid type description in the error message to make it
easier to track down the source of the error in the source file.
2016-01-25 10:21:12 -06:00
James Nugent cb6cb8b96a core: Support explicit variable type declaration
This commit adds support for declaring variable types in Terraform
configuration. Historically, the type has been inferred from the default
value, defaulting to string if no default was supplied. This has caused
users to devise workarounds if they wanted to declare a map but provide
values from a .tfvars file (for example).

The new syntax adds the "type" key to variable blocks:

```
variable "i_am_a_string" {
    type = "string"
}

variable "i_am_a_map" {
    type = "map"
}
```

This commit does _not_ extend the type system to include bools, integers
or floats - the only two types available are maps and strings.

Validation is performed if a default value is provided in order to
ensure that the default value type matches the declared type.

In the case that a type is not declared, the old logic is used for
determining the type. This allows backwards compatiblity with previous
Terraform configuration.
2016-01-24 11:40:02 -06:00
Paul Hinze 87a9701f91 config: validation error when output is missing value field
Also lists out invalid keys in errmsg when they are present

Closes #4398
2016-01-20 14:00:36 -06:00
Paul Hinze 47b521ebaf Merge pull request #4745 from hashicorp/b-lifecycle-keys
config: validate lifecycle keys [GH-4413]
2016-01-19 18:24:54 -06:00
Paul Hinze 911575b7d6 Merge pull request #4747 from hashicorp/b-escaped
Literals with escaped interpolations work
2016-01-19 18:23:08 -06:00
Mitchell Hashimoto 693736b52a config: fix tests 2016-01-19 13:17:47 -08:00
Mitchell Hashimoto f223be15cd config: eval HEL as long as the result changes [GH-2909] 2016-01-19 12:51:56 -08:00
Mitchell Hashimoto 8e7b0d90e3 config: detect provisioner-only resource in JSON and error [GH-4385] 2016-01-19 12:19:32 -08:00
Mitchell Hashimoto 99fbb91ba2 config: validate lifecycle keys [GH-4413] 2016-01-19 11:28:45 -08:00
Matt Moyer c17a6ceb2a Add a sha256(...) interpolation function. 2016-01-16 23:54:04 +00:00
Paul Hinze 0739cf2348 provider/template: fix race causing panic in template_file
The render code path in `template_file` was doing unsynchronized access
to a shared mapping of functions in `config.Func`.

This caused a race condition that was most often triggered when a
`template_file` had a `count` of more than one, and expressed itself as
a panic in the plugin followed by a cascade of "unexpected EOF" errors
through the plugin system.

Here, we simply turn the FuncMap from shared state into a generated
value, which avoids the race. We do more re-initialization of the data
structure, but the performance implications are minimal, and we can
always revisit with a perf pass later now that the race is fixed.
2016-01-15 16:34:46 -05:00
Joseph Kordish e1b62c76ad add sha1 interpolation 2016-01-06 15:10:43 -06:00
Jesse Szwedko 41f9ebc667 Add support for unary operators + and -
This adds support to the configuration interpolation syntax for + and -
as unary operators, specifically to represent negative numbers.
2015-12-18 18:05:25 +00:00
Paul Hinze b6626eed57 config: friendlier error message on resource arity mismatch
closes #2072
2015-12-09 18:05:49 -06:00
James Nugent 5ea25363a1 Add regression test for #4069
This may be brittle as it makes use of .gitattributes to override the
autocrlf setting in order to have an input file with Windows line
endings across multiple platforms.
2015-12-01 13:37:18 -05:00
Paul Hinze d90eb2d88e config: test replicating #4079
Should help cover terraform against regression once
https://github.com/hashicorp/hcl/pull/70 lands.
2015-12-01 10:31:05 -06:00
James Nugent 7f5f8d300d Add failing test replicating #4065 2015-11-26 15:08:48 +02:00
Paul Hinze afb5136ac2 Merge pull request #3986 from hashicorp/phinze/hcl-escaped-quotes
config: test covering escaped quotes syntax error
2015-11-19 12:32:30 -06:00
Paul Hinze 15e7927009 config: test covering escaped quotes syntax error
This was never intended to be valid syntax, but it worked in the old HCL
parser, and we've found a decent number of examples of it in the wild.

Fixed in https://github.com/hashicorp/hcl/pull/62 and we'll keep this
test in Terraform to cover the behavior.
2015-11-19 12:11:42 -06:00
James Nugent 6ae3218f8a Add failing tests for JSON configuration parsing
Reproduces the issue reported by @svanharmelen in #3964.
2015-11-19 16:06:30 +02:00
Paul Hinze 928f534cfc template_file: source contents instead of path
Building on the work of #3846, deprecate `filename` in favor of a
`template` attribute that accepts file contents instead of a path.

Required a bit of work in the interpolation code to prevent Terraform
from assuming that template interpolations were resource variables that
needed to be resolved. Leaving them as "Unknown Variables" prevents
interpolation from happening early and lets the `template_file` resource
do its thing.
2015-11-13 11:24:20 -06:00
James Nugent f4164b5322 Add resource with heredoc to config load tests
This test reproduces the issue which is likely the root cause of #3840.
Test is currently failing with an "illegal character" message
corresponding with the location of the heredoc, which is also seen in
various acceptance tests for providers.
2015-11-10 18:12:21 -05:00
Mitchell Hashimoto deb17b90eb Merge pull request #3813 from hashicorp/b-new-hcl
Use new HCL API
2015-11-09 10:34:48 -08:00
Martin Atkins 988baa584b Merge #3814: 'coalesce' interpolation func 2015-11-09 09:31:18 -08:00
James Nugent f4c03ec2a6 Reflect new comment format in stringer.go
As of November 8th 2015, (4b07c5ce8a), the word "Code" is prepended to
the comments in Go source files generated by the stringer utility.
2015-11-09 11:38:51 -05:00
Matt Morrison 6ecec7fe83 Add coalesce func 2015-11-08 19:34:56 +13:00
Mitchell Hashimoto 13c5fdb154 config: remove debug line 2015-11-07 16:55:07 -08:00
Mitchell Hashimoto 02f512d4bd config: new HCL API 2015-11-07 16:53:42 -08:00
Paul Hinze c56245ce34 Merge pull request #2973 from bobtfish/length_empty_split_zero
Split of "" should be empty. Length of empty array should be 0
2015-10-29 14:58:36 -05:00
Paul Hinze af04321723 config: return to the go1.5 generated lang/y.go
It has improvements to error messaging that we want.

We'll use this occasion begin developing / building with Go 1.5 from
here on out. Build times will be slower, but we have core development
plans that will help mitigate that.

/cc @hashicorp/terraform-committers
2015-10-28 12:20:18 -04:00
Tomas Doran 96275ee66a Add an explicit test for PR #2973 2015-10-25 18:55:21 +00:00
Martin Atkins ef161e1c1b Various interpolation functions for CIDR range manipulation.
These new functions allow Terraform to be used for network address space
planning tasks, and make it easier to produce reusable modules that
contain or depend on network infrastructure.

For example:
- cidrsubnet allows an aws_subnet to derive its
  CIDR prefix from its parent aws_vpc.
- cidrhost allows a fixed IP address for a resource to be assigned within
  an address range defined elsewhere.
- cidrnetmask provides the dotted-decimal form of a prefix length that is
  accepted by some systems such as routing tables and static network
  interface configuration files.

The bulk of the work here is done by an external library I authored called
go-cidr. It is MIT licensed and was implemented primarily for the purpose
of using it within Terraform. It has its own unit tests and so the unit
tests within this change focus on simple success cases and on the correct
handling of the various error cases.
2015-10-22 08:10:52 -07:00
Matt Morrison cccc5d03e3 Add lower / upper interpolation functions 2015-10-21 08:16:24 -07:00
Paul Hinze 77847b1572 config/lang: restore go1.4.3 generated code
my theory is that @mitchellh checked in a go1.5 generated file in
344e7c26b5
2015-10-21 09:49:23 -05:00
Mitchell Hashimoto 344e7c26b5 fix a bunch of tests from go-getter import 2015-10-15 13:48:58 -07:00
Mitchell Hashimoto 06f4ac8166 config/module: use go-getter 2015-10-15 13:36:58 -07:00
Rob Zienert a1939e70f7 Adding ignore_changes lifecycle meta property 2015-10-14 16:34:27 -05:00
Martin Atkins 3c939f9b26 Merge #3239: "compact" interpolation function 2015-10-10 15:18:10 -07:00
Martin Atkins 16b11e443d go fmt the "compact" function changes. 2015-10-10 15:17:25 -07:00
Svend Sorensen 73b51698ad Replace simple case with if 2015-10-08 17:58:19 +02:00
Svend Sorensen 8e4a313f17 Return an empty slice for empty string lists 2015-10-08 17:37:13 +02:00
Svend Sorensen 53f44878ff Add tests for empty string lists 2015-10-08 17:32:54 +02:00
Anthony Stanton f2f4ded970 Initialize list as an empty slice 2015-10-08 16:59:57 +02:00
Panagiotis Moustafellos e4845f75cc removed extra parentheses 2015-10-08 15:48:04 +03:00
Anthony Stanton aed3f98703 Rename func which is now a method. 2015-10-08 10:01:21 +02:00
Anthony Stanton 95b2a60b29 Use {a,b} instead of {b,c}
How does the alphabet even?
2015-10-08 10:01:21 +02:00
Anthony Stanton ef2b0a0b71 Order functions alphabetically 2015-10-08 10:01:21 +02:00
Anthony Stanton 735803ef09 Test cases for compact() 2015-10-08 10:00:32 +02:00
Anthony Stanton 7610874264 Initial implementation of compact() interpolation function 2015-10-08 10:00:32 +02:00
Martin Atkins 3fde993978 Merge #3336: Remove local multierror package.
Instead, use ``github.com/hashicorp/go-multierror``.
2015-10-03 17:53:36 -07:00
Martin Atkins 0b85d35e87 Rename base64enc/dec to encode/decode.
There isn't any precedent for abbreviating words in the interpolation
function names, and it may not be clear to all users what "enc" and "dec"
are short for, so instead we'll prefer to spell out the whole words for
improved readability.
2015-10-03 15:12:51 -07:00
Sander van Harmelen 2ba8dc38fa Switch to go-multierror
It seems there are 4 locations left that use the `helper/multierror`
package, where the rest is TF settled on the `hashicorp/go-multierror`
package.

Functionally this doesn’t change anything, so I suggest to delete the
builtin version as it can only cause confusion (both packages have the
same name, but are still different types according to Go’s type system.
2015-09-27 18:58:48 -07:00
Gorka Lerchundi Osa 70522fb770 implements base64{enc,dec} interpolation funcs
fixes #3320

Signed-off-by: Gorka Lerchundi Osa <glertxundi@gmail.com>
2015-09-25 09:23:36 +02:00
Anthony Scalisi 198e1a5186 remove various typos 2015-09-11 11:56:20 -07:00
Radek Simko 563fd3872b Add a few tests for string_list 2015-08-27 13:02:02 +01:00
Jason Gedge cf5926ddad Add a function to find the index of an element in a list. 2015-07-13 14:11:30 -04:00
Mitchell Hashimoto 89d44e5088 Merge pull request #2505 from hashicorp/b-var-splat
config/lang: don't see * as part of var name [GH-2046]
2015-06-26 08:20:25 -07:00
Mitchell Hashimoto 97d2c4a6de config/lang: don't see * as part of var name [GH-2046] 2015-06-25 19:52:50 -07:00
Paul Hinze c95f21cec1 config: make formatlist work on lists of length 1
removes treat-lists-as-scalar special casing for formatlist

/cc @radeksimko

fixes #2240
2015-06-25 18:55:28 -05:00
Paul Hinze e88aeede9b core: allow distinguishing between empty lists and strings
Had to handle a lot of implicit leaning on a few properties of the old
representation:

 * Old representation allowed plain strings to be treated as lists
   without problem (i.e. shoved into strings.Split), now strings need to
   be checked whether they are a list before they are treated as one
   (i.e. shoved into StringList(s).Slice()).
 * Tested behavior of 0 and 1 length lists in formatlist() was a side
   effect of the representation. Needs to be special cased now to
   maintain the behavior.
 * Found a pretty old context test failure that was wrong in several
   different ways. It's covered by TestContext2Apply_multiVar so I
   removed it.
2015-06-25 18:53:35 -05:00
Paul Hinze 7238b3b4af core: encapsulate representation of StringList
Now the only code that cares about how StringLists are represented lives
inside string_list.go

...which gives us the ability to change it! :)
2015-06-25 17:55:57 -05:00
Paul Hinze 10b3abf405 config: introduce StringList to abstract over list hack
This is the initial pure "all tests passing without a diff" stage. The
plan is to change the internal representation of StringList to include a
suffix delimiter, which will allow us to recognize empty and
single-element lists.
2015-06-25 17:55:56 -05:00
Mitchell Hashimoto 29eadb8194 terraform: missing provider should add missing aliases [GH-2023] 2015-06-24 20:58:52 -07:00
Mitchell Hashimoto 461f6557c7 config: fix test error 2015-06-23 22:30:41 -07:00