Commit Graph

896 Commits

Author SHA1 Message Date
James Bardin caadb4297f make sure a computed list is can be RequiresNew
If a schema.TypeList had a Schema with ForceNew, and if that list was
NewComputed, the diff would not have RequiresNew set. This causes apply
to fail when the diffs didn't match because of the change to
RequiresNew.

Set the RequiresNew field on the list's ResourceAttrDiff based on the
Schema value.
2017-04-21 17:51:15 -04:00
Martin Atkins b1763e262a Restore stringer-generated files back to new version
stringer has changed the boilerplate it generates in a recent version.
We'd previously updated to the new format but accientally rolled back
to the old while merging a long-running feature branch.

This restores us back to the new format again.
2017-04-21 14:49:18 -07:00
Jasmin Gacic 61499cfcf0 Provider Oneandone (#13633)
* Terraform Provider 1&1

* Addressing pull request remarks

* Fixed imports

* Fixing remarks

* Test optimiziation
2017-04-21 17:19:10 +03:00
James Bardin f5cda342f7 Merge pull request #13778 from hashicorp/jbardin/GH-13617
improve resource.WaitForState and add refreshGracePeriod
2017-04-19 18:23:00 -04:00
James Bardin 4c3a053f0c lint errors 2017-04-19 18:19:48 -04:00
James Bardin 14bea66f4e add test for proper cancelation 2017-04-19 18:07:00 -04:00
James Bardin eb4b45941c fix tests affected by refreshGracePeriod
A couple tests require lowering the grace period to keep the test from
taking the full 30s timeout.

The Retry_hang test also needed to be removed from the Parallel group,
becuase it modifies the global refreshGracePeriod variable.
2017-04-19 18:07:00 -04:00
James Bardin af1628eaa4 add refreshGracePeriod
Refresh calls may have side effects that need to be recorded if it
succeeds, especially common when when WaitForState is called from
resource.Retry.

If the WaitForState timeout is reached and there is a Refresh call
in-flight, wait up to refreshGracePeriod (set to 30s) for it to
complete.
2017-04-19 18:07:00 -04:00
= 1608f5544f Added error check 2017-04-19 14:22:35 -06:00
= 6bc3610c09 removing extraneous code 2017-04-19 09:18:30 -06:00
James Bardin 6601b9b8dd adjust the inconsistent_negative test to match
This test unfortunately relies on the timing of the loops in
WaitForState, and the text of the error message. Adjust the timing so
the timeout isn't an even multiple of the poll interval, and make sure
we reach a minimum number of retries.
2017-04-19 10:53:06 -04:00
James Bardin af5e22cf94 don't leave WaitForState goroutine running
Make sure that we can cancel the WaitForState refresh loop when reaching
a timeout, otherwise it may run indefinitely. There's no need to try and
store and read the Result concurrently, just pass the value over a
channel.
2017-04-19 10:41:51 -04:00
Sander van Harmelen 3d0073e05c core: fix a crash by suggesting a different approach to solve #11170 (#13541)
* Revert #11245, #11321, #11498 and #11757

These PR’s are all related to issue #11170 for which I would like to propose a different solution then the one currently implemented.

* A different approach to solve #11170

This approach has (IMHO) a few advantages with regards to the solution currently implemented. I will elaborate on this in the PR.
2017-04-14 22:32:30 +02:00
Clint 9ef9501e65 provider/aws: Fix EMR Bootstrap Action Ordering (#13580)
* provider/aws: Add failing test for EMR Bootstrap Actions

* aws_emr_cluster: Fix bootstrap action parameter ordering

* provider/aws: Fix EMR Bootstrap arguments

* provider/aws: Args needs to be ForceNew, because we can't update them
2017-04-12 14:19:38 -05:00
James Bardin 735dfc416d Merge pull request #13427 from hashicorp/jbardin/context-keys
Fix context key types in schema
2017-04-10 15:38:27 -04:00
Martin Atkins a0269c688c helper/schema: Clarify the expectations for DefaultFunc
Discussion in #9512 revealed that some of the comments here were
inaccurate and that the comments here did not paint a complete enough
picture of the behavior and expectations of Default and DefaultFunc.

This is a comments-only change that aims to clarify the situation and
call attention to the fact that the defaults only affect the handling of
the configuration and that changes to defaults may require migration of
existing resource states.

This closes #9512.
2017-04-06 09:51:43 -07:00
James Bardin 8a1089a161 convert the other context keys to the correct type 2017-04-06 10:51:24 -04:00
James Bardin 812f9fb253 don't use primitive types for context value keys
A context value key should be typed within the package namespace,
otherwise different packages could have colliding values in a context.
2017-04-06 10:36:06 -04:00
Jake Champlin fed7b6e781 Merge pull request #13129 from tombuildsstuff/validation
provider/azurerm: Moving Reused JSON Validation -> Core
2017-04-04 16:11:13 -04:00
Jake Champlin f24087ee54
core: Add ImportStateIdPrefix field for testing
Adds the `ImportStateIdPrefix` field for import acceptance tests. There are (albeit fairly rare) import cases where a resource needs to be imported with a combination of the resource's ID and a known string prefix. This allows the developer to specify the known prefix, and omit the `ImportStateId` field.

```
$ make test TEST=./helper/resource TESTARGS="-run=TestTest_importStateIdPrefix"
==> Checking that code complies with gofmt requirements...
==> Checking AWS provider for unchecked errors...
==> NOTE: at this time we only look for uncheck errors in the AWS package
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/03/30 18:08:36 Generated command/internal_plugin_list.go
go test -i ./helper/resource || exit 1
echo ./helper/resource | \
                xargs -t -n4 go test -run=TestTest_importStateIdPrefix -timeout=60s -parallel=4
go test -run=TestTest_importStateIdPrefix -timeout=60s -parallel=4 ./helper/resource
ok      github.com/hashicorp/terraform/helper/resource  0.025s
```
2017-03-30 18:11:10 -04:00
Martin Atkins 21cd5595e2 Update stringer-generated files to new boilerplate
golang/tools commit 23ca8a263 changed the format of the leading comment
to comply with some new standards discussed here:
https://golang.org/issue/13560

This is the result of running generate with the latest version of
stringer. Everyone working on Terraform will need to update stringer
after this is merged, to avoid reverting this:
    go get -u golang.org/x/tools/cmd/stringer
2017-03-29 08:07:06 -07:00
James Nugent 0e3a7e6d0d helper/resource: Allow unknown pending states (#13099)
Sometimes when waiting on a target state, the set of valid states
through which a value will transition is unknown. This commit adds
support for an empty Pending slice and will treat any states that are not
the target as valid provided the timeouts are not exceeded.
2017-03-28 22:38:42 +03:00
tombuildsstuff 64dda483dd Adding tests for each helper method 2017-03-28 12:59:46 +01:00
James Nugent 5d734cee4e helper/acctest: Add RandIntRange helper function
This commit adds a new RandIntRange function to the helper/acctest
package for generating random integers in a given range. This is useful
when randomizing test spaces with a constrained range (e.g. 0-4095).
2017-03-27 20:30:30 -04:00
tombuildsstuff 6efdc94ae3 Splitting the structure tests 2017-03-27 17:54:21 +01:00
tombuildsstuff 459c9a0a5a Returning a blank string if there's nothing to serialize 2017-03-27 16:22:44 +01:00
tombuildsstuff 277d1b6b2d Refactoring the schema diff/validation -> core 2017-03-27 15:41:55 +01:00
James Bardin 2e6a44d5ff reify the list values before validation
If the list was marked as computed, all values will be raw config
values. Fetch the individual keys from the config to get any known
values before validating.
2017-03-24 12:04:18 -04:00
James Bardin 99a12f5df3 interpolation strings were beeing validated
Interpolation strings for non-computed values in a list were being
passed to the schema's ValidateFunc.
2017-03-24 12:04:18 -04:00
Joseph Anthony Pasquale Holsten 855adb47ed alicloud: simplify validators (#12982) 2017-03-23 10:57:11 +02:00
clint shryock 61355c33c5 testing: Add option to run only a plan on a TestStep configuration 2017-03-22 15:38:21 -05:00
James Nugent 9476fc4418 helper/acctest: Add NewSSHKeyPair function (#12894)
Many cloud services prevent duplicate key pairs with different names.
Among them are Digital Ocean, Joyent Triton and Packet. Consequently, if
tests leave dangling resources it is not enough to simply randomise the
name, the entire key material must be regenerated.

This commit adds a helper method that returns a new randomly generated
key pair, where the public key material is formatted in OpenSSH
"authorized keys" format, and the private key material is PEM encoded.
2017-03-20 20:42:34 +00:00
James Bardin efd0f5b0db Fix logic when skipping schema input
The Required||Optional logic in schemaMap.Input was incorrect, causing
it to always request input. Fix the logic, and the associated tests
which were passing "just because".
2017-03-17 14:55:24 -04:00
Mitchell Hashimoto df8529719c
command/init: backend-config accepts key=value pairs
This augments backend-config to also accept key=value pairs.
This should make Terraform easier to script rather than having to
generate a JSON file.

You must still specify the backend type as a minimal amount in
configurations, example:

```
terraform { backend "consul" {} }
```

This is required because Terraform needs to be able to detect the
_absense_ of that value for unsetting, if that is necessary at some
point.
2017-03-16 23:27:05 -07:00
Radek Simko 1df1c21d5b schema: Allow *Resource as Elem of TypeMap in validation (#12722) 2017-03-15 14:54:41 +00:00
Radek Simko afd34f79df schema: Enable map value validation (#12638) 2017-03-13 15:58:58 +00:00
Clint 3fdeacdca7 helper/schema: Rename Timeout resource block to Timeouts (#12533)
helper/schema: Rename Timeout resource block to Timeouts

- Pluralize configuration argument name to better represent that there is
one block for many timeouts
- use a const for the configuration timeouts key
- update docs
2017-03-09 14:40:14 -06:00
James Bardin ecb1944c31 Merge pull request #12498 from hashicorp/jbardin/test-reset
Add schema.Provider.TestReset to reset StopContext between tests
2017-03-09 08:34:54 -05:00
James Bardin 5238f51dc7 move TestReset mock from terraform to helper
the terraform package doesn't know about TestProvider, so don't put the
hooks in terraform.MockResourceProvider. Wrap the mock in the test where
we need to check the TestProvider functionality.
2017-03-08 17:48:11 -05:00
James Bardin 6fcb55d09e reword TestProvider doc 2017-03-08 17:41:35 -05:00
James Bardin 4b2e96b2e2 test for TestReset and fixed resource factories 2017-03-07 11:39:12 -05:00
James Bardin 1eb9a2d083 Reset ResourceProviders
Call all ResourceProviderFactories and reset the providers before tests.
Store the provider and/or the error in a fixed factory function to be
returned again later.
2017-03-07 11:39:12 -05:00
James Bardin 1d9d7be28c Add schema.Provider.TestReset method
Provider.TestReset resets the internal state of the Provider at the
start of a test. This also adds a MetaReset function field to
schema.Provider, which is called by TestReset and can be used to reset
any other tsated stored in the provider metadata.

This is currently used to reset the internal Context returned by
StopContext between tests, and should  be implemented by a provider if
it stores a Context from a previous test.
2017-03-07 11:39:11 -05:00
clint shryock aa3677cd89 helper/schema: Guard against a panic if Timeout is not properly
structured
2017-03-07 10:25:32 -06:00
Clint 2fe5976aec helper/schema: Add configurable Timeouts (#12311)
* helper/schema: Add custom Timeout block for resources

* refactor DefaultTimeout to suuport multiple types. Load meta in Refresh from Instance State

* update vpc but it probably wont last anyway

* refactor test into table test for more cases

* rename constant keys

* refactor configdecode

* remove VPC demo

* remove comments

* remove more comments

* refactor some

* rename timeKeys to timeoutKeys

* remove note

* documentation/resources: Document the Timeout block

* document timeouts

* have a test case that covers 'hours'

* restore a System default timeout of 20 minutes, instead of 0

* restore system default timeout of 20 minutes, refactor tests, add test method to handle system default

* rename timeout key constants

* test applying timeout to state

* refactor test

* Add resource Diff test

* clarify docs

* update to use constants
2017-03-02 11:07:49 -06:00
Mitchell Hashimoto 3342aa580c
terraform: InstanceState.Meta is value type interface{}
This changes the type of values in Meta for InstanceState to
`interface{}`. They were `string` before.

This will allow richer structures to be persisted to this without
flatmapping them (down with flatmap!). The documentation clearly states
that only primitives/collections are allowed here.

The only thing using this was helper/schema for schema versioning.
Appropriate type checking was added to make this change safe.

The timeout work @catsby is doing will use this for a richer structure.
2017-02-23 10:44:05 -08:00
Mitchell Hashimoto c6d0333dc0
flatmap: mark computed list as a computed value in Expand
Fixes #12183

The fix is in flatmap for this but the entire issue is a bit more
complex. Given a schema with a computed set, if you reference it like
this:

    lookup(attr[0], "field")

And "attr" contains a computed set within it, it would panic even though
"field" is available. There were a couple avenues I could've taken to
fix this:

1.) Any complex value containing any unknown value at any point is
entirely unknown.

2.) Only the specific part of the complex value is unknown.

I took route 2 so that the above works without any computed (since
"name" is not computed but something else is). This may actually have an
effect on other parts of Terraform configs, however those similar
configs would've simply crashed previously so it shouldn't break any
pre-existing configs.
2017-02-23 10:03:59 -08:00
Mitchell Hashimoto 2124135287
helper/slowmessage 2017-02-14 09:27:23 -08:00
Kit Ewbank 64fda44b00 Add 'aws_vpn_gateway' data source. (#11886) 2017-02-13 16:24:55 +00:00
James Bardin ced4c53324 Merge pull request #11757 from hashicorp/jbardin/GH-11588
Make sure to diff all nested schema.Set elements
2017-02-07 17:05:32 -05:00
James Bardin 7359a18a71 Make sure to diff all nested schema.Set elements
This follows on GH-11498, using the same method to ensure all set
elements are marked as NewRemoved if the set is being removed in the
diff.
2017-02-07 16:55:20 -05:00
Mitchell Hashimoto 864c79396d
terraform: default shadow to false
To avoid chasing down issues like #11635 I'm proposing we disable the
shadow graph for end users now that we have merged in all the new
graphs. I've kept it around and default-on for tests so that we can use
it to test new features as we build them. I think it'll still have value
going forward but I don't want to hold us for making it work 100% with
all of Terraform at all times.

I propose backporting this to 0-8-stable, too.
2017-02-06 18:02:32 -08:00
Mitchell Hashimoto 61881d2795 Merge pull request #10934 from hashicorp/f-provisioner-stop
core: stoppable provisioners, helper/schema for provisioners
2017-01-30 12:53:15 -08:00
Radek Simko d5ac48de2a helper/schema: Remove missed subfields when parent list is removed (#11498) 2017-01-29 21:15:00 +00:00
Mitchell Hashimoto 09242fab09
terraform: remove legacy graph builder 2017-01-26 15:18:42 -08:00
Mitchell Hashimoto 487a37b0dd
helper/schema: PromoteSingle for legacy support of "maybe list" types 2017-01-26 15:09:15 -08:00
Mitchell Hashimoto 3c0c81957a
provisioners/remote-exec: switch to helper/schema 2017-01-26 15:09:15 -08:00
Mitchell Hashimoto a2e044829b
provisioners/file: use the old communicator.New just to minimize risk 2017-01-26 15:03:28 -08:00
Mitchell Hashimoto a1da59a73e
helper/schema: provisioner allows for nil state 2017-01-26 15:03:28 -08:00
Mitchell Hashimoto b2891bc9ef
helper/schema: Provisioner support 2017-01-26 15:03:27 -08:00
Mitchell Hashimoto 251e5c6f87
helper/schema: framework for Backends 2017-01-26 14:33:49 -08:00
Kit Ewbank 87d98b1ff1 Correct aws_s3_bucket_object data source acceptance tests. (#11346) 2017-01-23 21:53:02 +00:00
Radek Simko 6222b3e91d
helper/resource: Add retries to err message 2017-01-20 11:09:04 -08:00
Radek Simko be4e5e53a9
Add more details (Retries) to resource.NotFoundError 2017-01-20 07:08:30 -08:00
Conor Mongey 77c8683281 provider/vault: Remove user input for optional vault provider fields (#11082)
* Remove the need to input vault optional settings

* Allow TypeList to skip input

* Remove conflicts on vault ca_cert_* fields
2017-01-17 12:06:55 +00:00
Sean Chittenden 7862e5b499
Differentiate between lookup failure and value inequality.
Before this patch it was not possible to test for a key in a map where
the value is an empty string.  With this patch, however, it is now
possible to write a check like:

```
resource.TestCheckResourceAttr("res.name", "mymap.KeyWithEmptyValue", ""),
```

To test that `KeyWithEmptyValue` is a valid key in `mymap`.
2017-01-11 23:28:48 -08:00
clint shryock 6bafd4c896 fix typo 2017-01-10 11:06:51 -06:00
Mitchell Hashimoto 6d594b3bc6
helper/variables: trim whitespace on key before empty string comp 2016-12-14 13:52:34 -08:00
Mitchell Hashimoto b7f6f8eb2a
helper/variables: trim whitespace around the key in -var
Fixes #10716

This trims whitespace around the key in the `-var` flag.

This is a regression from 0.7.x.

The value is whitespace sensitive unless double-quoted. This is the same
behavior as 0.7.x. I considered rejecting whitespace around the '='
completely but I don't want to introduce BC and the behavior actually
seems quite obvious to me.
2016-12-13 21:01:01 -08:00
Mitchell Hashimoto 42edd22566
helper/variables: address go vet 2016-12-10 20:32:50 -05:00
Mitchell Hashimoto 83e72b0361
helper/variables: ParseInput for consistent parsing 2016-12-10 14:44:17 -05:00
Mitchell Hashimoto 39e3d8ea9b
helper/variables: helpers for working with vars 2016-12-10 14:30:36 -05:00
James Nugent 3177fc0765 provider/aws: Extract helper/encryption library
This commit extracts the GPG code used for aws_iam_user_login_profile
into a library that can be reused for other resources, and updates the
call sites appropriately.
2016-12-06 14:24:21 -06:00
James Bardin 7677bd94ed Merge pull request #10325 from hashicorp/jbardin/GH-10187
Fix some cases for nested maps and lists
2016-11-29 12:24:53 -05:00
James Bardin 01be1a5ecd Check for interpolated values when reading a map
Accessing an interpolated value in a map through ConfigFieldReader can
fail, because GetRaw can't access interpolated values, so check if the
value exists at all by looking in the config. If the config has a value,
assume our map's value is interpolated and proceed as such.

We also need to lookup the correct schema to properly read a field from
a nested structure.

- Maps previously always defaulted to TypeString. Now check if Elem is a
  ValueType and use that if applicable
- Lists now return the schema for nested element types, defaulting to a
  TypeString like maps.

This only allows maps and lists to be nested one level deep, and the
inner map or list must only contain string values.
2016-11-28 09:04:24 -05:00
Paul Tyng fead481002 Fix typo in test method name (#10370) 2016-11-25 16:20:12 +00:00
Mitchell Hashimoto 0634aada69
Revert "core: Ensure hasComputedSubKeys iterates over Sets and Lists properly"
This reverts commit 4d8208d840.
2016-11-21 17:31:19 -08:00
Mitchell Hashimoto f2fcf1494a
Merge branch 'fix-field-reader-setsubkeys-computed' of https://github.com/vancluever/terraform into vancluever-fix-field-reader-setsubkeys-computed 2016-11-21 17:24:26 -08:00
Mitchell Hashimoto fd36b548c5
helper/wrappedstreams: get original console input/output on Windows
Fixes #10266

panicwrap was using Extrafiles to get the original standard streams for
`terraform console`. This doesn't work on Windows. Instead, we must use
the Win32 APIs to get the exact handles.
2016-11-21 10:44:01 -08:00
James Bardin 24ebb72920 Merge pull request #10152 from hashicorp/jbardin/unique-id
Fix resource.UniqueId to be properly ordered
2016-11-21 09:13:26 -05:00
Chris Marchesi 8d06d68d0f core: Backport NewComputed change to nested list/set tests
Needed due to work done in 95d37ea, we may need to adjust
hasComputedSubKeys to propagate NewComputed in the same way that we
have added "~", however will wait for comment from @mitchellh.
2016-11-19 09:29:48 -08:00
Chris Marchesi f258056731 core: Tests for hasComputedSubKeys fix
This covers:

 * Complex sets with computed fields in a set
 * Complex lists with computed fields in a set

Adding a test to test basic lists with computed fields seemed to fail,
but possibly for an unrelated reason (the list returned as nil). The fix
to this inparticular case may be out of the scope of this specific
issue.

Reference gist and details in hashicorp/terraform#9171.
2016-11-19 08:56:16 -08:00
Chris Marchesi 4d8208d840 core: Ensure hasComputedSubKeys iterates over Sets and Lists properly
This fixes some edge-ish cases where a set in a config has a set or list
in it that contains computed values, but non-set or list values in the
parent do not.

This can cause "diffs didn't match during apply" errors in a scenario
such as when a set's hash is calculated off of child items (including
any sub-lists or sets, as it should be), and the hash changes between
the plan and apply diffs due to the computed values present in the
sub-list or set items. These will be marked as computed, but due to the
fact that the function was not iterating over the list or set items
properly (ie: not adding the item number to the address, so
set.0.set.foo was being yielded instead of set.0.set.0.foo), these
computed values were not being properly propagated to the parent set to
be marked as computed.

Fixes hashicorp/terraform#6527.
Fixes hashicorp/terraform#8271.

This possibly fixes other non-CloudFront related issues too.
2016-11-19 08:56:16 -08:00
James Bardin c4eefd4b5e Allow primitive type in maps via all FieldReaders
Now that we can read primitive type out of a map, each field reader
needs to be able to set the proper type as expected.
2016-11-17 15:35:08 -05:00
James Bardin 730014b33e Convert the map fields values when reading diff
Convert the value to the correct type when reading a diff and the map
schema has an primitive Elem type.
2016-11-17 14:34:18 -05:00
Radek Simko ad34f1ec74 Add (failing) test for map w/ non-string values 2016-11-17 11:46:32 -05:00
James Bardin e28e11d44c Fix resource.UniqueId to be properly ordered
UniqueId attempted to provide an ordered unique id by using a nanosecond
timestamp, but doesn't take into account that time is not monotonic
increasing. This provides an implementation that will always be
increasing.
2016-11-15 21:58:23 -05:00
Mitchell Hashimoto 39542898b0
helper/schema: mark diff as forcenew if element is computed
Fixes #10125

If the elements are computed and the field is ForceNew, then we should
mark the computed count as potentially forcing a new operation.

Example, assuming `groups` forces new...

**Step 1:**

    groups = ["1", "2", "3"]

At this point, the resource isn't create, so this should result in a
diff like:

    CREATE resource:
      groups: "" => ["1", "2", "3"]

**Step 2:**

    groups = ["${computedvar}"]

The OLD behavior was:

    UPDATE resource
      groups.#: "3" => "computed"

This would cause a diff mismatch because if `${computedvar}` was
different then it should force new. The NEW behavior is:

    DESTROY/CREATE resource:
      groups.#: "3" => "computed" (forces new)
2016-11-15 11:02:14 -08:00
Mitchell Hashimoto 6557a3de18
helper/shadow: Close for auto-closing all values
Fixes #10122

The simple fix was that we forgot to close `ReadDataApply` for the
provider. But I've always felt that this section of the code was brittle
and I wanted to put in a more robust solution. The `shadow.Close` method
uses reflection to automatically close all values.
2016-11-15 08:54:26 -08:00
Mitchell Hashimoto 1a6056b287
command: split out and tag code so compilation works on Solaris
The readline library doesn't support Solaris. For now, we'll just not
support console there.
2016-11-14 00:32:01 -08:00
Mitchell Hashimoto a867457d75
command/console 2016-11-13 23:17:04 -08:00
Mitchell Hashimoto 73a1564dac
helper/wrappedreadline: helper for dealing with wrapped standard streams 2016-11-13 23:17:03 -08:00
Mitchell Hashimoto 785cc7b78a
terraform: default new graphs on, old graphs behind -Xlegacy-graph
This turns the new graphs on by default and puts the old graphs behind a
flag `-Xlegacy-graph`. This effectively inverts the current 0.7.x
behavior with the new graphs.

We've incubated most of these for a few weeks now. We've found issues
and we've fixed them and we've been using these graphs internally for
awhile without any major issue. Its time to default them on and get them
part of a beta.
2016-11-10 21:53:20 -08:00
Mitchell Hashimoto 15f50b86bf
helper/schema: passing tests for computed complex sets 2016-11-10 08:52:37 -08:00
Mitchell Hashimoto 59bd1a22f4
helper/diff: handle unknownvariablevalue 2016-11-09 14:28:16 -08:00
Mitchell Hashimoto 29485f6167
terraform: ResourceConfig.IsComputed cases 2016-11-09 14:28:15 -08:00
Mitchell Hashimoto 5792b2cba2
helper/schema: convert _Diff to subtests 2016-11-09 14:28:15 -08:00
Mitchell Hashimoto 2b7177cfe7 Merge pull request #9607 from hashicorp/f-provider-stop-redo
terraform: ResourceProvider.Stop (redo)
2016-11-08 15:58:48 -08:00
Mitchell Hashimoto e45debe0e5
helper/schema: only mark "ForceNew" on resources that cause the ForceNew
Fixes #2748

This changes the diff to only mark "forces new resource" on the fields
that actually caused the new resource, not every field that changed.
This makes diffs much more accurate.

I'd like to request a review but I'm going to defer merging until
Terraform 0.8. Changes like this are very possible to cause "diffs
didn't match" errors and I want some real world testing in a beta before
we hit prod with this.
2016-11-08 15:49:28 -08:00
Mitchell Hashimoto b7bee66df5
helper/schema: sort errors in helper/schema test for deterministic tests 2016-11-04 16:51:26 -07:00
Mitchell Hashimoto f0abe6d1a0 Merge pull request #9812 from hashicorp/b-bool-computed-crash
helper/schema: computed bool fields should not crash
2016-11-04 08:47:49 -07:00
Mitchell Hashimoto 65b17ccd06
helper/schema: allow ConflictsWith and Computed Optional fields 2016-11-02 22:24:34 -07:00
Mitchell Hashimoto 7834cf7190
helper/schema: computed bool fields should not crash
Fixes #7715

If a bool field was computed and the raw value was not convertable to a
boolean, helper/schema would crash. The correct behavior is to try not
to read the raw value when the value is computed and to simply mark that
it is computed. This does that (and matches the behavior of the other
primitives).
2016-11-02 13:25:23 -07:00
Mitchell Hashimoto 2d84582881 Merge pull request #9699 from hashicorp/b-removed-forcenew
helper/schema: removed optional items force new
2016-10-31 13:24:36 -07:00
Mitchell Hashimoto f7a234bc71
helper/schema: validate Read, Delete are set 2016-10-30 15:04:32 -07:00
Mitchell Hashimoto 5489d8c549
helper/schema: removed optional items force new
Fixes #5138

If an item is optional and is removed completely from the configuration,
it should still trigger a destroy/create if the field itself was marked
as "ForceNew".

See the example in #5138.
2016-10-28 18:45:12 -04:00
Mitchell Hashimoto 3f36787207 Merge pull request #9618 from hashicorp/b-computed-prim
helper/schema,terraform: handle computed primitives in diffs
2016-10-28 10:44:13 -04:00
Mitchell Hashimoto af82be19ea
helper/experiment: a helper for setting, making experiments
This creates a standard package and interface for defining, querying,
setting experiments (`-X` flags).

I expect we'll want to continue to introduce various features behind
experimental flags. I want to make doing this as easy as possible and I
want to make _removing_ experiments as easy as possible as well.

The goal with this packge has been to rely on the compiler enforcing our
experiment references as much as possible. This means that every
experiment is a global variable that must be referenced directly, so
when it is removed you'll get compiler errors where the experiment is
referenced.

This also unifies and makes it easy to grab CLI flags to enable/disable
experiments as well as env vars! This way defining an experiment is just
a couple lines of code (documented on the package).
2016-10-26 15:47:58 -04:00
Mitchell Hashimoto 95d37ea79c
helper/schema,terraform: handle computed primtives in diffs
Fixes #3309

There are two primary changes, one to how helper/schema creates diffs
and one to how Terraform compares diffs. Both require careful
understanding.

== 1. helper/schema Changes

helper/schema, given any primitive field (string, int, bool, etc.)
_used to_ create a basic diff when given a computed new value (i.e. from
an unkown interpolation). This would put in the plan that the old value
is whatever the old value was, and the new value was the actual
interpolation. For example, from #3309, the diff showed the following:

```
~ module.test.aws_eip.test-instance.0
    instance: "<INSTANCE ID>" => "${element(aws_instance.test-instance.*.id, count.index)}"
```

Then, when running `apply`, the diff would be realized and you would get
a diff mismatch error because it would realize the final value is the
same and remove it from the diff.

**The change:** `helper/schema` now marks unknown primitive values with
`NewComputed` set to true. Semantically this is correct for the diff to
have this information.

== 2. Terraform Diff.Same Changes

Next, the way Terraform compares diffs needed to be updated

Specifically, the case where the diff from the plan had a NewComputed
primitive and the diff from the apply _no longer has that value_. This
is possible if the computed value ended up being the same as the old
value. This is allowed to pass through.

Together, these fix #3309.
2016-10-25 22:36:59 -04:00
Mitchell Hashimoto ad5a82de75
helper/resource: remove StopCh, use contexts instead 2016-10-25 12:08:36 -07:00
Mitchell Hashimoto 5ee8042dff
helper/schema: expose stop information as a Context 2016-10-25 12:08:36 -07:00
Mitchell Hashimoto be34dfe7c1
helper/resource: StopCh as a helper for provider stopCh + timeout 2016-10-25 12:08:35 -07:00
Mitchell Hashimoto d1bb2f3487
helper/schema: support Stop() 2016-10-25 12:08:35 -07:00
Mitchell Hashimoto 60140b28f4
Revert "Merge pull request #9536 from hashicorp/f-provider-stop"
This reverts commit c3a4cff133, reversing
changes made to 791a02e6e4.

This change requires plugin recompilation and we should hold off until a
minor release for that.
2016-10-25 12:00:36 -07:00
Mitchell Hashimoto 61bbaf6f85
helper/resource: remove StopCh, use contexts instead 2016-10-25 11:32:30 -07:00
Mitchell Hashimoto 86eb30b8a2
helper/schema: expose stop information as a Context 2016-10-25 11:32:30 -07:00
Mitchell Hashimoto 89647745b0
helper/resource: StopCh as a helper for provider stopCh + timeout 2016-10-25 11:31:56 -07:00
Mitchell Hashimoto 8c11f137f5
helper/schema: support Stop() 2016-10-25 11:31:55 -07:00
James Nugent 43dd13cd36 helper/schema: Introduce Noop and RemoveFromState
This commit implements reusable functions for when resources have no
need to implement a particular operation:

- Noop - does nothing and returns no error.
- RemoveFromState - sets the resource ID to empty string (removing it
  from state) and returns no error.
2016-10-25 11:47:03 -05:00
Mitchell Hashimoto fa9758e162
helper/schema: test with DiffSuppress and Default 2016-10-24 22:23:13 -07:00
Mitchell Hashimoto de827887bf
helper/shadow: keyedValue.WaitForChange must unlock 2016-10-21 17:57:00 -07:00
Mitchell Hashimoto 0fe51b334c Merge pull request #9334 from hashicorp/f-shadow-graph
terraform: Shadow Graph
2016-10-19 13:36:10 -07:00
Sean Chittenden b1c3649eac
Append to debug log files, don't clobber. 2016-10-18 13:08:23 -07:00
Mitchell Hashimoto 10bcdd04d4
helper/shadow: KeyedValue.Init 2016-10-12 18:45:40 +08:00
James Bardin 404a76e2b9 Merge pull request #9144 from hashicorp/b-source
config/module: use the raw source as part of the key
2016-10-11 12:29:42 -04:00
Mitchell Hashimoto 50e0647c53
helper/shadow: ComparedValue 2016-10-11 22:17:30 +08:00
Mitchell Hashimoto c92ee5a8bd
helper/shadow: KeyedValue.WaitForChange returns immediately if closed 2016-10-11 22:17:29 +08:00
Mitchell Hashimoto 0408c2dfb2
helper/shadow: KeyedValue.WaitForChange 2016-10-11 22:17:29 +08:00
Mitchell Hashimoto 47f4343bf5
helper/shadow: KeyedValue add test case to avoid panic 2016-10-11 22:17:29 +08:00
Mitchell Hashimoto 136ac4728d
helper/shadow: KeyedValue.Close 2016-10-11 22:17:29 +08:00
Mitchell Hashimoto d2fb630df8
helper/shadow: Value.Close 2016-10-11 22:17:29 +08:00
Mitchell Hashimoto d6168edc50
helper/shadow: KeyedValue.ValueOk 2016-10-11 22:17:28 +08:00
Mitchell Hashimoto bd69e41c14
helper/shadow: KeyedValue 2016-10-11 22:17:27 +08:00
Mitchell Hashimoto 8426cea6b0
helper/shadow: OrderedValue 2016-10-11 22:17:27 +08:00
Mitchell Hashimoto 1df3bbdc37
terraform: working on the resource provider shadow, not working yet 2016-10-11 22:17:27 +08:00
Paul Stack 4ac7d0a6bb Merge pull request #8103 from BedeGaming/validation-helper
helper: create validation package to provide common validation functions
2016-10-07 13:42:29 +01:00
stack72 5a537cdbf9
helper/schema: Adding of MinItems as a validation to Lists and Maps
This is required for the times when the configuration cannot have an
empty configuration. An example would be in AzureRM, when you create a
LoadBalancer with a configuration, you can delete *all* but 1 of these
configurations
2016-10-04 18:57:58 +01:00
Paul Hinze 534a5a3b75 Merge pull request #8891 from gotascii/fix-diff-set-update
Unchanged NestedSets are not returned by DiffFieldReader.
2016-10-03 11:06:40 -05:00
Mitchell Hashimoto 098225dc0d
config/module: use the raw source as part of the key
This changes the key for the storage to be the _raw_ source from the
module, not the fully expanded source. Example: it'll be a relative path
instead of an absolute path.

This allows the ".terraform/modules" directory to be portable when
moving to other machines. This was a behavior that existed in <= 0.7.2
and was broken with #8398. This amends that and adds a test to verify.
2016-09-30 10:44:46 -07:00
Justin Marney 895d10a627 Use DeepEqual instead of multiple if statements to reduce noise. 2016-09-17 17:41:44 +00:00
Justin Marney 53e10dfa89 Name test appropriately. 2016-09-17 16:56:12 +00:00
Justin Marney 9217f5a063 Unchanged NestedSets are not returned by DiffFieldReader. 2016-09-16 23:57:33 +00:00
Radek Simko 86acdccaf2 helper/resource: Add timeout to TimeoutError msg (#8773) 2016-09-15 10:53:25 +01:00
James Nugent e0226c9039 core: Check for attrV being nil before dereference
This can be an issue with unset computed fields.

Fixes #8815.
2016-09-14 09:51:15 +01:00
James Nugent 6c23181686 Merge pull request #8383 from kjmkznr/import-aws-s3-b-notification
provider/aws: Support import `aws_s3_bucket_notification`
2016-09-03 15:50:25 -07:00
Paul Stack bf755bb5c9 Merge pull request #8585 from hashicorp/f-diff-suppression
helper/schema: Add diff suppression callback
2016-09-01 14:46:03 +01:00
James Nugent 85ec09111b helper/schema: Add diff suppression callback
This commit adds a new callback, DiffSuppressFunc, to  the schema.Schema
structure. If set for a given schema, a callback to the user-supplied
function will be made for each attribute for which the default
type-based diff mechanism produces an attribute diff. Returning `true`
from the callback will suppress the diff (i.e. pretend there was no
diff), and returning false will retain it as part of the plan.

There are a number of motivating examples for this - one of which is
included as an example:

1. On SSH public keys, trailing whitespace does not matter in many
   cases - and in some cases it is added by provider APIs. For
   digitalocean_ssh_key resources we previously had a StateFunc that
   trimmed the whitespace - we now have a DiffSuppressFunc which
   verifies whether the trimmed strings are equivalent.

2. IAM policy equivalence for AWS. A good proportion of AWS issues
   relate to IAM policies which have been "normalized" (used loosely)
   by the IAM API endpoints. This can make the JSON strings differ
   from those generated by iam_policy_document resources or template
   files, even though the semantics are the same (for example,
   reordering of `bucket-prefix/` and `bucket-prefix/*` in an S3
   bucket policy. DiffSupressFunc can be used to test for semantic
   equivalence rather than pure text equivalence, but without having to
   deal with the complexity associated with a full "provider-land" diff
   implementation without helper/schema.
2016-08-31 19:13:53 -05:00
James Bardin e0014198e1 Merge pull request #8560 from hashicorp/jbardin/race2
Fix races in WaitForState
2016-08-31 14:02:43 -04:00
Mitchell Hashimoto aaaed823af Merge pull request #8508 from TimeIncOSS/f-schema-all-validation-errors
schema: Return all validation errors together from InternalValidate
2016-08-31 10:44:13 -07:00
James Bardin 481f12cf2c Remove `first` variable an extra if block
clean up the code slighly by moving the Sleep in WaitForState to the end
of the loop.
2016-08-30 17:56:26 -04:00
James Bardin 82be35a797 Fix races in WaitForState
The WaitForState method can't read the result values in a timeout
because they are still owned by the running goroutine. Keep all values
scoped inside the goroutine, and save them into an atomic.Value to be
returned.

Fixes race introduced in #8510
2016-08-30 16:22:21 -04:00
Radek Simko 506c118383
helper/resource: Show last state in timeout err message 2016-08-27 21:25:14 +01:00
Radek Simko aef2513b44
helper/resource: Fix WaitForState tests 2016-08-27 15:11:09 +01:00
Radek Simko b90ab0b705
schema: Return all validation errors together from InternalValidate 2016-08-27 10:34:16 +01:00
James Nugent df06d5623d Merge pull request #8249 from meteor/glasser/name-prefix-timestamp
core: name_prefix names now start with a timestamp
2016-08-23 14:37:56 +02:00
Kazunori Kojima ed05161fd0
provider/aws: Support import `aws_s3_bucket_notification` 2016-08-23 08:19:06 +09:00
James Nugent 6ae7140251 Merge pull request #8370 from hashicorp/dtolnoy-wait
Do not sleep between first attempt or between successful attempts
2016-08-22 14:30:54 +02:00
James Nugent 0a5acaa0d8 Change error message return structure 2016-08-22 10:34:32 +02:00
David Tolnay 9d3f40a513 Fix bad message from TimeoutError
Before:

    Error creating IAM Role my-role: timeout while waiting for state
    to become 'success'. last error: %!s(<nil>)
2016-08-22 10:23:53 +02:00
David Tolnay e27a735da2 Do not sleep between first attempt or between successful attempts 2016-08-22 10:17:23 +02:00
David Glasser 4ad825fe08 core: name_prefix names now start with a timestamp
This means that two resources created by the same rule will get names
 which sort in the order they are created.

The rest of the ID is still random base32 characters; we no longer set
the bit values that denote UUIDv4.

The length of the ID returned by PrefixedUniqueId is not changed by this
commit; that way we don't break any resources where the underlying
resource has a name length limit.

Fixes #8143.
2016-08-17 11:06:28 -07:00
James Nugent b551981cc7 testing: Add ComposeAggregateTestFunc
This commit adds a function which composes a series of TestFuncs, but
will run all tests before returning an error, unlike ComposeTestFunc.
This is useful when verifying contents of state in acceptance tests and
it is desirable to see all the failing cases in one run for slow
resources.
2016-08-16 19:56:18 +01:00
Mitchell Hashimoto f26d1b40e0 Merge pull request #8215 from hashicorp/f-signalwrapper
helper/signalwrapper and azurerm_storage_account listens for signals
2016-08-16 09:57:54 -07:00
Mitchell Hashimoto 8dafcb36fd
providers/azurerm: cancellable storage account creation 2016-08-15 21:12:32 -07:00
Mitchell Hashimoto d8337920f9
helper/signalwrapper: more tests 2016-08-15 20:58:48 -07:00
Mitchell Hashimoto bdcea55117
helper/signalwrapper 2016-08-15 20:55:35 -07:00
James Nugent b681ab23d3 testing: Add TestCheckResourceAttrSet helper
This commit adds a TestCheckFunc which ensures that a value is set for a
given name/key combination. It is primarily useful for ensuring that
computed values are set where it is not possible to know the expected
value ahead of time.
2016-08-15 15:42:25 -05:00
Paul Hinze 3dccfa0cc9
terraform: diffs with only tainted set are non-empty
Fixes issue where a resource marked as tainted with no other attribute
diffs would never show up in the plan or apply as needing to be
replaced.

One unrelated test needed updating due to a quirk in the testDiffFn
logic - it adds a "type" field diff if the diff is non-Empty. NBD
2016-08-12 17:37:49 -05:00
Peter McAtominey 8abec085ec helper: create validation package to provide common validation functions 2016-08-10 13:27:49 +01:00
James Bardin 1af7ee87a2 Silence log output when not verbose
Set the default log package output to iotuil.Discard during tests if the
`-v` flag isn't set. If we are verbose, then apply the filter according
to the TF_LOG env variable.
2016-08-01 17:19:14 -04:00
James Bardin 074be9ae56 Another race in resource.Retry 2016-07-29 18:35:54 -04:00
Paul Hinze 261043fd1a Merge pull request #7523 from hashicorp/f-aws-beanstalk-env-poll-timing
provider/aws: Beanstalk environments, bump the minimum timeout between API calls
2016-07-21 16:43:21 -05:00
Paul Hinze 614806d59f
helper/resource: Fix import test harness, which was modifying state
Maps are reference types, it turns out :D
2016-07-15 13:15:47 -06:00
clint shryock de60481428 provider/aws: Rework Beanstalk optional polling
expose a poll_interval for users to configure polling for updates
2016-07-13 15:38:23 -06:00
David Tolnay 7096e4d3da Return nonnegative hash if int is 32 bits 2016-07-01 14:40:53 -07:00
Paul Hinze 3b732131d2 Merge pull request #7446 from hashicorp/b-jit-resource-validate
core: rerun resource validation before plan and apply
2016-07-01 15:00:40 -05:00
Paul Hinze ad27190852 Merge pull request #7448 from dtolnay/timeout
Timing out is not success
2016-07-01 14:28:11 -05:00
Paul Hinze afccf62e3e Merge pull request #7459 from hashicorp/r-resource-unit-test
helper/resource: Consolidate unit test override
2016-07-01 13:23:13 -05:00
Paul Hinze 4a1b36ac0d
core: rerun resource validation before plan and apply
In #7170 we found two scenarios where the type checking done during the
`context.Validate()` graph walk was circumvented, and the subsequent
assumption of type safety in the provider's `Diff()` implementation
caused panics.

Both scenarios have to do with interpolations that reference Computed
values. The sentinel we use to indicate that a value is Computed does
not carry any type information with it yet.

That means that an incorrect reference to a list or a map in a string
attribute can "sneak through" validation only to crop up...

 1. ...during Plan for Data Source References
 2. ...during Apply for Resource references

In order to address this, we:

 * add high-level tests for each of these two scenarios in `provider/test`
 * add context-level tests for the same two scenarios in `terraform`
   (these tests proved _really_ tricky to write!)
 * place an `EvalValidateResource` just before `EvalDiff` and `EvalApply` to
   catch these errors
 * add some plumbing to `Plan()` and `Apply()` to return validation
   errors, which were previously only generated during `Validate()`
 * wrap unit-tests around `EvalValidateResource`
 * add an `IgnoreWarnings` option to `EvalValidateResource` to prevent
   active warnings from halting execution on the second-pass validation

Eventually, we might be able to attach type information to Computed
values, which would allow for these errors to be caught earlier. For
now, this solution keeps us safe from panics and raises the proper
errors to the user.

Fixes #7170
2016-07-01 13:12:57 -05:00
Paul Hinze 5656d7388c
helper/resource: Consolidate unit test override
I noticed we had two mechanisms for unit test override. One that dropped
a sentinel into the env var, and another with a struct member on
TestCase. This consolidates the two, using the cleaner struct member
internal mechanism and the nicer `resource.UnitTest()` entry point.
2016-07-01 13:08:43 -05:00
David Tolnay 7e1cd34819 Timing out is not success 2016-06-30 18:07:56 -07:00
Clint a84aa5e914 Revert "helper/schema: Make nested Set(s) in List(s) work" (#7436) 2016-06-30 10:48:52 -05:00
Radek Simko 37d57f4a85 Improve naming convention (resultSet -> exists) 2016-06-28 17:40:45 +01:00
Radek Simko 917ad44cf0 helper/schema: Fix readSet implementation (DiffFieldReader) 2016-06-28 17:40:44 +01:00
Radek Simko c738c5a9a3 helper/schema: Implement reader-specific readList method 2016-06-28 17:40:44 +01:00
Radek Simko 378b526dc3 helper/schema: Add regression tests for nested Set/List
Although DiffFieldReader was the one mostly responsible for a buggy behaviour
more tests were added throughout the debugging process most of which
would fail without the bugfix.

 - ResourceData
 - MultiLevelFieldReader
 - MapFieldReader
 - DiffFieldReader
2016-06-28 17:40:44 +01:00
Patrick Sodré 80936e3562 provider/triton: add machine domain names (#7149)
* vendor joyent/gosdc/cloudapi

* provider/triton: Add machine domain names

 - Includes acceptance test.
2016-06-26 22:18:17 +01:00
James Nugent dbf725bd68 core: Allow dynamic attributes in helper/schema
The helper/schema framework for building providers previously validated
in all cases that each field being set in state was in the schema.
However, in order to support remote state in a usable fashion, the need
has arisen for the top level attributes of the resource to be created
dynamically. In order to still be able to use helper/schema, this commit
adds the capability to assign additional fields.

Though I do not forsee this being used by providers other than remote
state (and that eventually may move into Terraform Core rather than
being a provider), the usage and semantics are:

To opt into dynamic attributes, add a schema attribute named
"__has_dynamic_attributes", and make it an optional string with no
default value, in order that it does not appear in diffs:

        "__has_dynamic_attributes": {
            Type: schema.TypeString
            Optional: true
        }

In the read callback, use the d.UnsafeSetFieldRaw(key, value) function
to set the dynamic attributes.

Note that other fields in the schema _are_ copied into state, and that
the names of the schema fields cannot currently be used as dynamic
attribute names, as we check to ensure a value is not already set for a
given key.
2016-06-11 13:29:05 +01:00
James Nugent 9c7cf639b3 testing: Allow acceptance test to run as unit test
This commit adds a flag to acceptance tests in order to make
appropriately named tests work during `make test` irrespective of the
TF_ACC environment variable. This should only be used on tests which are
known to be fast.
2016-06-11 12:55:14 +01:00
clint shryock 7d71b8cc3c helper and terraform interpolate test update 2016-06-10 10:07:17 -05:00
James Bardin 2c7b702d1f Merge pull request #7091 from hashicorp/jbardin/serialize
Serialization for hash panics on TypeMap
2016-06-09 16:16:41 -04:00
James Bardin bab031aac5 Add test for TypeMap in a Schema 2016-06-09 16:00:33 -04:00
James Bardin d8fbaa7924 Serialization for hash panics on TypeMap
The serializeCollectionMemberForHash helper can't be called for the
MapType values, because MapType doesn't have a schema.Elem. Instead, we
can write the key/value pairs directly to the buffer. This still doesn't
allow for nested maps or lists, but we need to define that use case
before committing to it here.
2016-06-09 13:37:58 -04:00
James Nugent 074545e536 core: Use .% instead of .# for maps in state
The flatmapped representation of state prior to this commit encoded maps
and lists (and therefore by extension, sets) with a key corresponding to
the number of elements, or the unknown variable indicator under a .# key
and then individual items. For example, the list ["a", "b", "c"] would
have been encoded as:

    listname.# = 3
    listname.0 = "a"
    listname.1 = "b"
    listname.2 = "c"

And the map {"key1": "value1", "key2", "value2"} would have been encoded
as:

    mapname.# = 2
    mapname.key1 = "value1"
    mapname.key2 = "value2"

Sets use the hash code as the key - for example a set with a (fictional)
hashcode calculation may look like:

    setname.# = 2
    setname.12312512 = "value1"
    setname.56345233 = "value2"

Prior to the work done to extend the type system, this was sufficient
since the internal representation of these was effectively the same.
However, following the separation of maps and lists into distinct
first-class types, this encoding presents a problem: given a state file,
it is impossible to tell the encoding of an empty list and an empty map
apart. This presents problems for the type checker during interpolation,
as many interpolation functions will operate on only one of these two
structures.

This commit therefore changes the representation in state of maps to use
a "%" as the key for the number of elements. Consequently the map above
will now be encoded as:

    mapname.% = 2
    mapname.key1 = "value1"
    mapname.key2 = "value2"

This has the effect of an empty list (or set) now being encoded as:

    listname.# = 0

And an empty map now being encoded as:

    mapname.% = 0

Therefore we can eliminate some nasty guessing logic from the resource
variable supplier for interpolation, at the cost of having to migrate
state up front (to follow in a subsequent commit).

In order to reduce the number of potential situations in which resources
would be "forced new", we continue to accept "#" as the count key when
reading maps via helper/schema. There is no situation under which we can
allow "#" as an actual map key in any case, as it would not be
distinguishable from a list or set in state.
2016-06-09 10:49:42 +01:00
James Nugent cb9ef298f3 core: Defeat backward compatibilty in mapstructure
The mapstructure library has a regrettable backward compatibility
concern whereby a WeakDecode of []interface{}{} into a target of
map[string]interface{} yields an empty map rather than an error. One
possibility is to switch to using Decode instead of WeakDecode, but this
loses the nice handling of type conversion, requiring a large volume of
code to be added to Terraform or HIL in order to retain that behaviour.

Instead we add a DecodeHook to our usage of the mapstructure library
which checks for decoding []interface{}{} or []string{} into a map and
returns an error instead.

This has the effect of defeating the code added to retain backwards
compatibility in mapstructure, giving us the correct (for our
circumstances) behaviour of Decode for empty structures and the type
conversion of WeakDecode.

The code is identical to that in the HIL library, and packaged into a
helper.
2016-06-08 18:38:41 +01:00
James Nugent 91587a49f3 core: Remove unnecessary debug logging
Some unnecessary debug logging was introduced in 7b6df27e4, this commit
removes it so as not to clutter logs.
2016-06-08 18:38:41 +01:00
Radek Simko 1ea5cff9b3 Merge pull request #6946 from TimeIncOSS/f-aws-s3-object-data-source
provider/aws: Add aws_s3_bucket_object data source
2016-06-02 07:04:46 +01:00
Radek Simko bf8931b1ab provider/aws: Mark Lambda function as gone when it's gone (#6924)
* helper/error: Introduce TimeoutError & UnexpectedStateError

* provider/aws: Mark Lambda function as gone when it's gone
2016-06-02 00:50:43 +01:00
Radek Simko 84ab00d92e helper/resource: Add TestStep.PreventPostDestroyRefresh
- This is to allow easier testing of data sources which read data from resources created in the same scope
2016-06-01 19:13:00 +01:00
Chris Marchesi 9d7fb89114 core: Adding Sensitive attribute to resource schema
This an effort to address hashicorp/terraform#516.

Adding the Sensitive attribute to the resource schema, opening up the
ability for resource maintainers to mark some fields as sensitive.
Sensitive fields are hidden in the output, and, possibly in the future,
could be encrypted.
2016-05-29 22:18:44 -07:00
Sander van Harmelen 8560f50cbc
Change taint behaviour to act as a normal resource
This means it’s shown correctly in a plan and takes into account any
actions that are dependant on the tainted resource and, vice verse, any
actions that the tainted resource depends on.

So this changes the behaviour from saying this resource is tainted so
just forget about it and make sure it gets deleted in the background,
to saying I want that resource to be recreated (taking into account the
existing resource and it’s place in the graph).
2016-05-26 19:55:26 -05:00
Martin Atkins 031b561ef0 helper/resource: ignore data resource diffs during destroy
When testing destroy, the test harness calls Refresh followed by Plan,
with the expectation that the resulting diff will be empty.

Data resources challenge this expectation, because they will always be
instantiated during refresh if their configuration isn't computed, and so
the subsequent diff will want to destroy what was instantiated.

To work around this, we make an exception that data resource destroy
diffs may appear in the plan but nothing else.

This fixes #6713.
2016-05-21 14:16:02 -07:00
Mitchell Hashimoto 55583baa7e
Merge branch 'f-aws-import' 2016-05-18 15:28:12 -06:00
James Nugent 3ea3c657b5 core: Use OutputState in JSON instead of map
This commit forward ports the changes made for 0.6.17, in order to store
the type and sensitive flag against outputs.

It also refactors the logic of the import for V0 to V1 state, and
fixes up the call sites of the new format for outputs in V2 state.

Finally we fix up tests which did not previously set a state version
where one is required.
2016-05-18 13:25:20 -05:00
Mitchell Hashimoto 884980da1a
providers/aws: instance, nat, internet gateway 2016-05-16 10:03:57 -07:00
Mitchell Hashimoto b7d4767dd6
helper/schema: pass through import state func 2016-05-16 10:03:57 -07:00
Martin Atkins 6cd22a4c9a helper/schema: emit warning when using data source resource shim
For backward compatibility we will continue to support using the data
sources that were formerly logical resources as resources for the moment,
but we want to warn the user about it since this support is likely to
be removed in future.

This is done by adding a new "deprecation message" feature to
schema.Resource, but for the moment this is done as an internal feature
(not usable directly by plugins) so that we can collect additional
use-cases and design a more general interface before creating a
compatibility constraint.
2016-05-14 08:26:36 -07:00
Martin Atkins fb262d0dbe helper/schema: shim for making data sources act like resources
Historically we've had some "read-only" and "logical" resources. With the
addition of the data source concept these will gradually become data
sources, but we need to retain backward compatibility with existing
configurations that use the now-deprecated resources.

This shim is intended to allow us to easily create a resource from a
data source implementation. It adjusts the schema as needed and adds
stub Create and Delete implementations.

This would ideally also produce a deprecation warning whenever such a
shimmed resource is used, but the schema system doesn't currently have
a mechanism for resource-specific validation, so that remains just a TODO
for the moment.
2016-05-14 08:26:36 -07:00
Martin Atkins 6a468dcd83 helper/schema: Resource can be writable or not
In the "schema" layer a Resource is just any "thing" that has a schema
and supports some or all of the CRUD operations. Data sources introduce
a new use of Resource to represent read-only resources, which require
some different InternalValidate logic.
2016-05-14 08:26:36 -07:00
Martin Atkins 0e0e3d73af core: New ResourceProvider methods for data resources
This is a breaking change to the ResourceProvider interface that adds the
new operations relating to data sources.

DataSources, ValidateDataSource, ReadDataDiff and ReadDataApply are the
data source equivalents of Resources, Validate, Diff and Apply (respectively)
for managed resources.

The diff/apply model seems at first glance a rather strange workflow for
read-only resources, but implementing data resources in this way allows them
to fit cleanly into the standard plan/apply lifecycle in cases where the
configuration contains computed arguments and thus the read must be deferred
until apply time.

Along with breaking the interface, we also fix up the plugin client/server
and helper/schema implementations of it, which are all of the callers
used when provider plugins use helper/schema. This would be a breaking
change for any provider plugin that directly implements the provider
interface, but no known plugins do this and it is not recommended.

At the helper/schema layer the implementer sees ReadDataApply as a "Read",
as opposed to "Create" or "Update" as in the managed resource Apply
implementation. The planning mechanics are handled entirely within
helper/schema, so that complexity is hidden from the provider implementation
itself.
2016-05-14 08:26:36 -07:00
Mitchell Hashimoto 6a675b4a15
helper/resource: ImportState test can verify states 2016-05-11 13:02:37 -07:00
Mitchell Hashimoto ec02c8c0e2
helper/resource: testing of almost all aspects of ImportState tests 2016-05-11 13:02:37 -07:00
Mitchell Hashimoto 2d99c451fb
helper/resource: basic ImportState acceptance testing
Still some TODOs, and more test cases to write, but the basics are all
here.
2016-05-11 13:02:37 -07:00
Mitchell Hashimoto 9bd1c9e7ca
helper/resource: reshuffling to prepare for importstate testing 2016-05-11 13:02:37 -07:00
Mitchell Hashimoto b728e55861
helper/schema: Resource.Data should set latest schema version 2016-05-11 13:02:36 -07:00
Mitchell Hashimoto c02c6c3f9c
helper/schema: default state func for import 2016-05-11 13:02:35 -07:00
Mitchell Hashimoto 2bb814e3de
helper/schema: adapt to ID being arg to ImportState 2016-05-11 13:02:35 -07:00
Mitchell Hashimoto 03931bfda9
helper/schema: ImportState must set ID on the resource data 2016-05-11 13:02:34 -07:00
Mitchell Hashimoto 445194ebdf
helper/schema: test ImportState 2016-05-11 13:02:33 -07:00
Mitchell Hashimoto 1685054a9a
helper/schema: cleaner way to store Ephemeral 2016-05-11 13:02:31 -07:00
Mitchell Hashimoto 84531a3fd5
helper/schema: sets Importable to true for resources that have importer 2016-05-11 13:02:30 -07:00
Mitchell Hashimoto 19609bde0e
helper/schema: can specify Importer on Resource, InternalValidate 2016-05-11 13:02:30 -07:00
Mitchell Hashimoto b8121ea63e
helper/schema: Resource.Data to return a ResourceData for a Resource 2016-05-11 13:02:30 -07:00
Mitchell Hashimoto 4e9877179c
helper/schema: start the resource importer 2016-05-11 13:02:29 -07:00
James Nugent 7b6df27e4a helper/schema: Read native maps from configuration
This adds a test and the support necessary to read from native maps
passed as variables via interpolation - for example:

```
resource ...... {
     mapValue = "${var.map}"
}
```

We also add support for interpolating maps from the flat-mapped resource
config, which is necessary to support assignment of computed maps, which
is now valid.

Unfortunately there is no good way to distinguish between a list and a
map in the flatmap. In lieu of changing that representation (which is
risky), we assume that if all the keys are numeric, this is intended to
be a list, and if not it is intended to be a map. This does preclude
maps which have purely numeric keys, which should be noted as a
backwards compatibility concern.
2016-05-10 14:49:14 -04:00
James Nugent f49583d25a core: support native list variables in config
This commit adds support for native list variables and outputs, building
up on the previous change to state. Interpolation functions now return
native lists in preference to StringList.

List variables are defined like this:

variable "test" {
    # This can also be inferred
    type = "list"
    default = ["Hello", "World"]
}

output "test_out" {
    value = "${var.a_list}"
}
This results in the following state:

```
...
            "outputs": {
                "test_out": [
                    "hello",
                    "world"
                ]
            },
...
```

And the result of terraform output is as follows:

```
$ terraform output
test_out = [
  hello
  world
]
```

Using the output name, an xargs-friendly representation is output:

```
$ terraform output test_out
hello
world
```

The output command also supports indexing into the list (with
appropriate range checking and no wrapping):

```
$ terraform output test_out 1
world
```

Along with maps, list outputs from one module may be passed as variables
into another, removing the need for the `join(",", var.list_as_string)`
and `split(",", var.list_as_string)` which was previously necessary in
Terraform configuration.

This commit also updates the tests and implementations of built-in
interpolation functions to take and return native lists where
appropriate.

A backwards compatibility note: previously the concat interpolation
function was capable of concatenating either strings or lists. The
strings use case was deprectated a long time ago but still remained.
Because we cannot return `ast.TypeAny` from an interpolation function,
this use case is no longer supported for strings - `concat` is only
capable of concatenating lists. This should not be a huge issue - the
type checker picks up incorrect parameters, and the native HIL string
concatenation - or the `join` function - can be used to replicate the
missing behaviour.
2016-05-10 14:49:14 -04:00
Mitchell Hashimoto 35c87836b4 core: Add terraform_version to state
This adds a field terraform_version to the state that represents the
Terraform version that wrote that state. If Terraform encounters a state
written by a future version, it will error. You must use at least the
version that wrote that state.

Internally we have fields to override this behavior (StateFutureAllowed),
but I chose not to expose them as CLI flags, since the user can just
modify the state directly. This is tricky, but should be tricky to
represent the horrible disaster that can happen by enabling it.

We didn't have to bump the state format version since the absense of the
field means it was written by version "0.0.0" which will always be
older. In effect though this change will always apply to version 2 of
the state since it appears in 0.7 which bumped the version for other
purposes.
2016-05-10 14:40:11 -04:00
Paul Hinze b4df304b47
helper/schema: Normalize bools to "true"/"false" in diffs
For a long time now, the diff logic has relied on the behavior of
`mapstructure.WeakDecode` to determine how various primitives are
converted into strings.  The `schema.DiffString` function is used for
all primitive field types: TypeBool, TypeInt, TypeFloat, and TypeString.

The `mapstructure` library's string representation of booleans is "0"
and "1", which differs from `strconv.FormatBool`'s "false" and "true"
(which is used in writing out boolean fields to the state).

Because of this difference, diffs have long had the potential for
cosmetically odd but semantically neutral output like:

    "true" => "1"
    "false" => "0"

So long as `mapstructure.Decode` or `strconv.ParseBool` are used to
interpret these strings, there's no functional problem.

We had our first clear functional problem with #6005 and friends, where
users noticed diffs like the above showing up unexpectedly and causing
troubles when `ignore_changes` was in play.

This particular bug occurs down in Terraform core's EvalIgnoreChanges.
There, the diff is modified to account for ignored attributes, and
special logic attempts to handle properly the situation where the
ignored attribute was going to trigger a resource replacement. That
logic relies on the string representations of the Old and New fields in
the diff to be the same so that it filters properly.

So therefore, we now get a bug when a diff includes `Old: "0", New:
"false"` since the strings do not match, and `ignore_changes` is not
properly handled.

Here, we introduce `TypeBool`-specific normalizing into `finalizeDiff`.
I spiked out a full `diffBool` function, but figuring out which pieces
of `diffString` to duplicate there got hairy. This seemed like a simpler
and more direct solution.

Fixes #6005 (and potentially others!)
2016-05-05 09:00:58 -05:00
Mitchell Hashimoto d85df63526
providers/aws: aws_instance id-only 2016-04-22 09:37:41 -07:00
Mitchell Hashimoto e0da21d381
helper/resource: make id-only check opt-in
As I've been working through the resources, I'm finding that a lot are
going to need some serious work. Given we have hundreds, I think it
might be prudent to make this opt-in for now and we can revisit
automatic/opt-out at some future point.

Importability will likely be opt-in it appears so this will match up
with that.
2016-04-21 08:37:08 -07:00
Mitchell Hashimoto ff7b58f032
providers/aws: peering connection id-only test settings 2016-04-20 12:19:21 -07:00
Mitchell Hashimoto 0ef1b3b84a
providers/aws: response value for DescribeVpcAttribute needs to be
.Value
2016-04-20 11:35:43 -07:00
Mitchell Hashimoto 1db1bf6639
helper/resource: use the full config for id-only checks
Originally I used an empty config module. This caused problems since
important provider configurations weren't available. Instead, I now set
it to use the full config. This isn't an issue since the attributes
themselves aren't available to Refresh anyways.
2016-04-20 11:18:13 -07:00
Mitchell Hashimoto 0c8b0bff2c
helper/resource: can specify specific name to id refresh test 2016-04-20 11:12:30 -07:00
Mitchell Hashimoto 1a9fae6b2e
helper/resource: can disable ID refresh check 2016-04-20 11:09:54 -07:00
Mitchell Hashimoto 35f4201b9e
providers/aws: instance_tenancy is computed, set 2016-04-20 10:48:22 -07:00
Mitchell Hashimoto f2c4f8e9ba
helper/resource: fix tests 2016-04-20 10:08:34 -07:00
Mitchell Hashimoto 5c4f78796b
helper/resource: don't need to id-only check if no test steps 2016-04-20 09:52:53 -07:00
Mitchell Hashimoto a285c04dc9
helper/resource: only verify id-only run if no error 2016-04-20 09:50:59 -07:00
Mitchell Hashimoto baac14aaeb
helper/resource: guard id-only by acc var 2016-04-20 09:34:54 -07:00
Mitchell Hashimoto cb32cb8947
helper/resource: error if id-only check didn't run 2016-04-20 09:25:23 -07:00
Mitchell Hashimoto 86e0c853db
helper/resource: test for failing id-only refresh check 2016-04-20 09:18:25 -07:00
Mitchell Hashimoto 060b43fbd9
helper/resource: remove debug 2016-04-20 09:17:04 -07:00
Mitchell Hashimoto 4f6edf4fe4
helper/resource: id-only refresh testing 2016-04-20 09:16:48 -07:00
James Nugent cb8a0549e1 Merge pull request #5757 from hashicorp/f-output-testing
Add TestCheckOutput helper to resource testing
2016-03-21 17:19:08 +00:00
James Nugent 35f9d2e081 Add TestCheckOutput helper to resource testing
This allows outputs in test configuration to have test functions written
conveniently. Useful for azurerm_template_deployment.
2016-03-21 16:54:02 +00:00
Radek Simko bf9f5879ca helper/resource: Remove NotFoundError function 2016-03-21 16:47:50 +00:00
James Nugent f946695187 Merge pull request #5444 from TimeIncOSS/f-aws-logs-metric-filter
provider/aws: Add support for CloudWatch Log Metric Filter
2016-03-21 16:43:38 +00:00
Paul Hinze c3e27b3e0a provider/test: a test provider
Here we also introduce a `test` provider meant as an aid to exposing
via automated tests issues involving interactions between
`helper/schema` and Terraform core.

This has been helpful so far in diagnosing `ignore_changes` problems,
and I imagine it will be helpful in other contexts as well.

We'll have to be careful to prevent the `test` provider from becoming a
dumping ground for poorly specified tests that have a clear home
elsewhere. But for bug exposure I think it's useful to have.
2016-03-21 08:59:54 -05:00
Radek Simko fd2fcaf1d5 helper/resource: Implement resource-wide NotFoundError 2016-03-15 13:58:25 +00:00