Commit Graph

992 Commits

Author SHA1 Message Date
Chris Marchesi e33aa9dc2e
helper/validation: Add StringMatch
StringMatch returns a validation function that can be used to match a
string against a regular expression. This can be used for simple
substring validations or more complex validation scenarios. Optionally,
an error message can be returned so that the user is returned a better
error message other than that their field did not match a regular
expression that they might not be able to understand.
2017-11-27 17:16:15 -08:00
Radek Simko 2974d63e75
Merge pull request #16588 from hashicorp/f-panic-on-invalid-rd-set
helper/schema: Opt-in panic on invalid ResourceData.Set
2017-11-08 19:17:46 +00:00
Nándor István Krácser e5cc8af7f3 helper/resource: fix ungrammatical doc comment in StateChangeConf 2017-11-08 09:43:36 -08:00
Radek Simko e93d64b18c
helper/schema: Opt-in panic on invalid ResourceData.Set 2017-11-08 10:05:11 +00:00
Chris Marchesi 3fa11e456b
helper/schema: Clear existing map/set/list contents before overwriting
There are situations where one may need to write to a set, list, or map
more than once per single TF operation (apply/refresh/etc). In these
cases, further writes using Set (example: d.Set("some_set", newSet))
currently create unstable results in the set writer (the name of the
writer layer that holds the data set by these calls) because old keys
are not being cleared out first.

This bug is most visible when using sets. Example: First write to set
writes elements that have been hashed at 10 and 20, and the second write
writes elements that have been hashed at 30 and 40. While the set length
has been correctly set at 2, since a set is basically a map (as is the
entire map writer) and map results are non-deterministic, reads to this
set will now deliver unstable results in a random but predictable
fashion as the map results are delivered to the caller non-deterministic
- sometimes you may correctly get 30 and 40, but sometimes you may get
10 and 20, or even 10 and 30, etc.

This problem propagates to state which is even more damaging as unstable
results are set to state where they become part of the permanent data
set going forward.

The problem also applies to lists and maps. This is probably more of an
issue with maps as a map can contain any key/value combination and hence
there is no predictable pattern where keys would be overwritten with
default or zero values. This is contrary to complex lists, which has
this problem as well, but since lists are deterministic and the length
of a list properly gets updated during the overwrite, the problem is
masked by the fact that a read will only read to the boundary of the
list, skipping any bad data that may still be available due past the
list boundary.

This update clears the child contents of any set, list, or map before
beginning a new write to address this issue. Tests are included for all
three data types.
2017-11-05 12:04:23 -08:00
Chris Marchesi 0c0ae3ca7c helper/schema: CustomizeDiff allowed on writable resources only
This keeps CustomizeDiff from being defined on data sources, where it
would be useless. We just catch this in InternalValidate like the rest
of the CRUD functions that are not used in data sources.
2017-11-01 15:42:24 -07:00
Chris Marchesi 2c541e8c97 helper/schema: Add behaviour detail to various custom diff comments
Added some more detailed comments to CustomizeDiff's comments. The new
comments detail how CustomizeDiff will be called in the event of
different scenarios like creating a new resource, diffing an existing
one, diffing an existing resource that has a change that requires a new
resource, and destroy/tainted resources.

Also added similar detail to ForceNew in ResourceDiff.

This should help mitigate any confusion that may come up when using
CustomizeDiff, especially in the ForceNew scenario when the second run
happens with no state.
2017-11-01 15:42:24 -07:00
Chris Marchesi f5bdbc0911 helper/schema: Simplify setDiff, remove exported SetDiff mentions
setDiff does not make use of its new parameter anymore, so it has been
removed. Also, as there is no more SetDiff (exported) function, mentions
of that have been removed from comments too.
2017-11-01 14:25:32 -07:00
Chris Marchesi 3ac0cdf0c0 helper/schema: Better ResourceDiff schema key validation
This fixes nil pointer issues that could come up if an invalid key was
referenced (ie: not one in the schema). Also ships a helper validation
function to streamline things.
2017-11-01 14:25:32 -07:00
Chris Marchesi 529d7e6dae helper/schema: Review -> CustomizeDiff
Restoring the naming of this field in the resource back to
CustomizeDiff, as this is generally more descriptive of the process
that's happening, despite the lengthy name.
2017-11-01 14:25:32 -07:00
Chris Marchesi 09e2109ff8 helper/schema: Resouce.Diff no longer ResourceProvider API compatible
The old comments said that this interface was API compatible with
terraform.ResourceProvider's Diff method - with the addition of passing
down meta to it, this is no longer the case.

Not too sure if this is really a big deal - schema.Resource never fully
implemented terraform.ResourceProvider, from what I can see, and the
path from Provdier.Diff to Resource.Diff is still pretty clear. Just
wanted to remove an outdated comment.
2017-11-01 14:25:32 -07:00
Chris Marchesi 9625830980 helper/schema: Move computedKeys init to init function
This simplifies the new value initialization and future-proofs it
against more complex initialization functionality.
2017-11-01 14:25:32 -07:00
Chris Marchesi f0aafe4d67 helper/schema: Restore new value for complex set test 2017-11-01 14:25:32 -07:00
Chris Marchesi 3444549d60 helper/schema: Move computed key reader logic to childAddrOf
This will make this check safer and will remove the risk of it passing
on keys with a similar prefix.
2017-11-01 14:25:32 -07:00
Chris Marchesi 36aa63b338 helper/schema: Guard against out of range on childAddrOf
This could panic if we sent a parent that was actually longer than the
child (should almost never come up, but the guard will make it safe
anyway).

Also fixed a slice style lint warning in UpdatedKeys.
2017-11-01 14:25:32 -07:00
Chris Marchesi 931b0e1441 helper/schema: Remove exported SetDiff method
The consensus is that it's a generally better idea to move setting the
functionality of old values completely to the refresh/read process,
hence it's moot to have this function around anymore. This also means we
don't need the old value reader/writer anymore, which simplifies things.
2017-11-01 14:25:32 -07:00
Chris Marchesi ad98471559 helper/schema: Correct some comments
Removed some outdated comments for SetNew, and normalized the computed
key note for SetNew, SetNewComputed, and SetDiff.
2017-11-01 14:25:32 -07:00
Chris Marchesi 8a7c9a6f02 helper/schema: frist -> first 2017-11-01 14:25:32 -07:00
Chris Marchesi 7d5f9ed6b1 helper/schema: NewComputed values should be nil
When working on this initially, I think I thought that since NewComputed
values in the diff were empty strings, that it was using the zero value.
After review, it doesn't seem like this is the case - so I have adjusted
NewComputed to pass nil values. There is also a guard now that keeps the
new value writer from accepting computed fields with non-nil values.
2017-11-01 14:25:32 -07:00
Chris Marchesi 6f422d8c44 helper/schema: Remove unused log line
Meant to remove this before finalizing the PR. :P
2017-11-01 14:25:32 -07:00
Chris Marchesi c6647a3bb7 helper/schema: CustomizeDiff -> Review
To keep with the current convention of most other schema.Resource
functional fields being fairly short, CustomizeDiff has been changed to
"Review". It would be "Diff", however it is already used by existing
functions in schema.Provider and schema.Resource.
2017-11-01 14:25:32 -07:00
Chris Marchesi f7e42728b6 helper/schema: Remove unused Destroy check for CustomizeDiff
Both Destroy and DestroyDeposed are not propagated down the diff stack,
meaning that there is no way we can tell at this point if an instance is
being destroyed or deposed, so this check would never be used.

In this regard, Destroy never runs a diff down the stack at all, and a
deposition check is not run until *after* the provider's diff function
is called. To answer this question and close it off, we could either
determine if a resource is deposed earlier, and propagate that down, or
treat deposed resources like full destroy nodes, and not diff them at
all (but rather making a diff with the only thing in it being
DestroyDeposed flagged).
2017-11-01 14:25:32 -07:00
Chris Marchesi fa1fc2ca8e helper/schema: CustomizeDiff invocation test
Just one more test to check to make sure that CustomizeDiff is called
on resource level.
2017-11-01 14:25:32 -07:00
Chris Marchesi ee769188d7 helper/schema: More CustomizeDiff test cases
Added a few more test cases for CustomizeDiff, caught another error in
the process. I think this is ready for review now, and possibly some
real-world testing of the waters by way of porting some resources that
would benefit from the feature.
2017-11-01 14:25:32 -07:00
Chris Marchesi 8af9610b87 helper/schema: Hook CustomizeDiffFunc into diff logic
It's alive! CustomizeDiff logic now has been inserted into the diff
process. The test_resource_with_custom_diff resource provides some basic
testing and a reference implementation.

There should now be plenty of test coverage for this feature via the
tests added for ResourceDiff, and the basic test added to the
schemaMap.Diff test, and the test resource, but more can be added to
test any specific case that comes up otherwise.
2017-11-01 14:25:32 -07:00
Chris Marchesi 8126ee8401 helper/schema: Fix supplied config to tests 2017-11-01 14:25:32 -07:00
Chris Marchesi 196d7e63fe helper/schema: Drop ClearAll from ResourceDiff
As the interface has been specced out for ResourceDiff, the only diff
operation that can function on a non-computed key is ForceNew, and that
is only if a diff exists for that key. As such, allowing the user to
clear keys that cannot be operated on afterwards does not really make
much sense.

Will re-add this function if it is determined it's needed after all on
review.
2017-11-01 14:25:32 -07:00
Chris Marchesi 22220fd0f7 helper/schema: Final set of ResourceDiff tests, bug fixes
Final set of coverage for ResourceDiff and bug fixes to correct issues
the test cases brought up.

Will be dropping ClearAll in next commit, as with how this interface is
intended to be used, it does not really make much sense.
2017-11-01 14:25:32 -07:00
Chris Marchesi a5fc664ea6 helper/schema: Add schemaMap.DeepCopy
This provides a deep copy of a schemaMap, which will be needed for the
diff customization process as ResourceDiff will be able to flag fields
as ForceNew - we don't want to affect the source schema.
2017-11-01 14:25:32 -07:00
Chris Marchesi 6a4f7b0dce helper/schema: Don't ignore diffs for certain NewComputed keys
In diffString, removed values are marked if the old value is non-nil and
the new value is nil, regardless of if the new value is marked as a
computed value. With the new diff override behaviour, this can lead to
issues where a nil attribute may get inserted into the diff if the new
value has been marked as computed (as the value will be marked as
missing, but computed).

This propagates into finalizeDiff in some respects because even if
NewComputed is manually set in the diff that gets passed in, it is
ignored, and the schema becomes the only source of truth. Since our new
diff behaviour is mainly designed to be supported on computed keys only,
it stands to reason that we there will be a time where we want to set a
diff as NewComputed on a key, and see that change in the diff.

These two small changes makes that happen. No regressions in tests have
been observed via this change, so it seems safe and non-invasive.
2017-11-01 14:25:32 -07:00
Chris Marchesi 64cc4084d2 helper/schema: ResourceDiff tests, bug fixes
The beginning of tests for SetNew. Noticed that removed values are not
logged for computed keys in a diff - not too sure if that is going to be
a problem (possibly not as GetChange in ResourceData should still
reference state for the old value). I came on this most notably in the
"TestSetNew/complex-ish_set_diff" test, for reference.

More tests pending for other functions for coverage of other functions
and test cases as defined in #8769.
2017-11-01 14:25:32 -07:00
Chris Marchesi f5f4e0329f helper/schema: Track updated keys in ResourceDiff
This ensures that when we hook this into the main diff logic, that we
can just re-diff the keys that are modified, ensuring that the diff is
not re-run on keys that were not touched, or on keys that were
intentionally removed.
2017-11-01 14:25:32 -07:00
Chris Marchesi aeb793f968 helper/schema: Add Clear function to ResourceDiff
This should complete the feature set of the prototype. This function
removes a specific key from the existing diff, preventing conflicts.

Further functionality might be needed to make this behave as expected,
namely ensuring that we don't re-process the whole diff after the
CustomizeDiffFunc runs.
2017-11-01 14:25:32 -07:00
Chris Marchesi 1e8cfc52e9 helper/schema: Updated ResourceDiff prototype
This new prototype removes the dependence on a underlying ResourceData,
moving several items up to ensure an approach that more matches
ResourceData. Namely, we create our own MultiLevelFieldReader that
also layers updated diff values on top. New values use a small
re-implementation of the MapFieldWriter/Reader that allows computed
values to be set.

The assumption here now is that a second diff will happen after the
first one is done, processing the new values set in the 2 new
reader/writer levels and updating the diff as necessary.
2017-11-01 14:25:32 -07:00
Chris Marchesi b99c615ee6 helper/schema: New ResourceDiff object
This adds a new object, ResourceDiff, to the schema package. This
object, in conjunction with a function defined in CustomizeDiff in the
resource schema, allows for the in-flight customization of a Terraform
diff. This helps support use cases such as when there are necessary
changes to a resource that cannot be detected in config, such as via
computed fields (most of the utility in this object works on computed
fields only). It also allows for a wholesale wipe of the diff to allow
for diff logic completely offloaded to an external API, if it is a
better use case for a specific provider.

As part of this work, many internal diff functions have been moved to
use a special resourceDiffer interface, to allow for shared
functionality between ResourceDiff and ResourceData. This may be
extended to the DiffSuppressFunc as well which would restrict use of
ResourceData in DiffSuppressFunc to generally read-only fields.

This work is not yet in its final state - CustomizeDiff is not yet
implemented in the general diff workflow, new functions may be added
(notably Clear() for a single key), and functionality may be altered.
Tests will follow as well.
2017-11-01 14:25:32 -07:00
Radek Simko d39025b9d8
helper: Escape test name in TF_LOG_PATH_MASK 2017-10-31 06:48:57 +00:00
Radek Simko 48fbd60e33
helper: Fall back to TF_LOG_PATH instead of os.Stderr 2017-10-30 15:58:13 +00:00
James Bardin 3a495ffe56 rename ModuleStorage to Storage
get rid of stutter and use module.Storage
2017-10-27 13:11:21 -04:00
James Bardin f2a7b94692 use the new ModuleStorage in the command package
Update the command package to use the new module storage. Move the old
command output strings into the module storage itself. This could be
moved back later either by using ui callbacks, or designing a module
storage interface once we know what the final requirements will look
like.
2017-10-27 12:58:24 -04:00
Dana Hoffman a7e308fc2c helper/schema: Schema.Elem documentation for TypeMap
Updated it to state that Elem is valid for TypeMaps too, and that TypeSets can use a Resource as an Elem.
2017-10-26 10:29:47 -07:00
Radek Simko 7fceccfbf7
helper/schema: Loosen validation for 'id' field 2017-10-26 09:37:38 +01:00
Arthur Burkart 013df9350b hashcode: "Strings" function for hashing slices of strings 2017-10-20 14:54:23 -07:00
Martin Atkins 183833affc core: terraform.ResourceProvider.GetSchema method
In order to parse provider, resource and data source configuration from
HCL2 config files, we need to know the relevant configuration schema.
This new method allows Terraform Core to request these from a provider.

This is a breaking change to this interface, so all of its implementers
in this package are updated too. This includes concrete implementations
of the new method in helper/schema that use the schema conversion code
added in an earlier commit to produce a configschema.Block automatically.

Plugins compiled against prior versions of helper/schema will not have
support for this method, and so calls to them will fail. Callers of
this new method will therefore need to sniff for support using the
SchemaAvailable field added to both ResourceType and DataSource.

This careful handling will need to persist until next time we increment
the plugin protocol version, at which point we can make the breaking
change of requiring this information to be available.
2017-10-17 07:23:41 -07:00
Martin Atkins ccc20fdad1 helper/didyoumean: helper library for "Did you mean ...? suggestions
Uses Levenshtein distance to decide if the input is similar enough to one
of the given suggestions, and returns that suggestion if so.

The distance threshold of three was arrived at experimentally, and has
no objective basis.
2017-10-16 17:50:57 -07:00
Radek Simko 0388bc1b23 Merge pull request #16356 from hashicorp/t-log-path-mask
helper: Allow logs isolation per acceptance test
2017-10-16 20:32:38 +03:00
Radek Simko 4052a8f956
helper: Allow logs isolation per acceptance test 2017-10-14 16:39:11 +03:00
Martin Atkins b91bd62747 config/configschema: Sensitive flag for attributes
We don't currently have any need for this information, but we're
propagating it out of helper/schema here pre-emptively so that once we
later have a use for it we will not need to rebuild the providers to gain
access to it.

The long-term expected use-case for this is to have Terraform Core use
static analysis techniques to trace the path of sensitive data through
interpolations so that intermediate results can be flagged as sensitive
too, but we have a lot more work to do before such a thing would actually
be possible.
2017-10-04 16:35:11 -07:00
Martin Atkins 69650b0bbc helper/schema: conversion of Schema to configschema.Block
As part of moving to the next-generation HCL implementation,
Terraform Core is getting its own representation of configuration schema
that is tailored for configuration-processing use-cases. The capabilities
of this are a subset of the helper/schema model primarily concerned with
the configuration structure and value types, leaving detailed validation
and defaults for helper/schema to still solve.

These new methods allow mechanical creation of a schema in the new Core
schema model from a schema expressed in the helper/schema model. This is
not yet used as of this commit, but will be used later to implement some
new ResourceProvider methods that will allow core to obtain the schema
for provider, resource and data source configuration while remaining
source-compatible with existing provider implementations.
2017-10-04 16:35:11 -07:00
Chris Marchesi edb8e8904c
helper/validation: Add NoZeroValues
This adds NoZeroValues, a small SchemaValidateFunc that checks that a
defined value is not a zero value. It's useful for situations where you
want to keep someone from explicitly entering a zero value (ie:
literally "0", or an empty string) on a required field, and want to
catch it in the validation stage, versus during apply using GetOk.
2017-09-25 17:19:23 -07:00
Chris Marchesi dc1f155728
helper/resource: Failure case for ImportStateIdFunc
Just to make sure returning an error was working.
2017-09-06 19:51:01 -07:00
Chris Marchesi 7bba1d0c95
helper/resource: Add ImportStateIdFunc
Add an ImportStateIdFunc field to the ImportState testing functionality.
This will allow for more powerful generation of complex import state IDs
that can't be accomplished by ImportStateId or ImportStateIdPrefix
themselves.
2017-09-06 19:44:43 -07:00
Radek Simko db97555e3e
helper/resource: Add TestStep.SkipFunc 2017-08-30 10:24:02 +02:00
Chris Marchesi 61971f2dde
helper/validation: Fix validation package
That'll teach me to use the GitHub merge conflict resolver :P
2017-08-28 14:41:01 -07:00
Chris Marchesi b0152f6be3 Add newline to end of validation.go 2017-08-28 13:49:11 -07:00
Chris Marchesi f17e14f576 Merge branch 'master' into f-validate-list-no-duplicates 2017-08-28 13:45:16 -07:00
Chris Marchesi bd368580c6 Merge pull request #15931 from hashicorp/f-validate-regex-func
helper/validation: Add ValidateRegexp
2017-08-28 13:30:02 -07:00
Martin Atkins c12d64f340 Use t.Helper() in our test helpers
Go 1.9 adds this new function which, when called, marks the caller as
being a "helper function". Helper function stack frames are then skipped
when trying to find a line of test code to blame for a test failure, so
that the code in the main test function appears in the test failure output
rather than a line within the helper function itself.

This covers many -- but probaly not all -- of our test helpers across
various packages.
2017-08-28 09:59:30 -07:00
Chris Marchesi aacfc01945
helper/validation: Add ValidateListUniqueStrings
Add the ValidateListUniqueStrings function, which is a ValidateFunc that
ensures a list has no duplicate items in it. It's useful for when a list
is needed over a set because order matters, yet the items still need to
be unique.
2017-08-27 09:50:21 -07:00
Chris Marchesi afeeecd10d
helper/validation: Add ValidateRegexp
This adds a ValidateRegexp validation helper to check to see if a field
has a valid regular expression.

Includes test for good regexp, and test for bad regexp.
2017-08-26 10:07:10 -07:00
Chris Marchesi 287a5eb66a
helper/schema: More tests for Set.HashEqual
Just to make sure equality on outer and inner values does not affect
anything.
2017-08-15 21:56:01 -07:00
Chris Marchesi ca42980e49
helper/schema: Add Set.HashEqual
Equality of schema.Sets gets tricky when dealing with nested sets -
Set.Equal only superficially compares the underlying maps and hence any
sets nested under the root sets cause issues.

This adds a simple method, HashEqual, that does a top-level hash
comparison, helping to work around this without any complex re-invention
of things like reflect.DeepEqual.

Of course, in order to make effective use of this function, the user
needs to make sure they are properly hashing their nested sets, however
this is trivial with things like HashResource.
2017-08-15 21:50:52 -07:00
Jake Champlin 3600f0b730 Merge pull request #15723 from hashicorp/f-get-boolean-non-default
core: Add `GetOkExists` schema function
2017-08-03 18:06:17 -04:00
Jake Champlin d969f97e73
update tests 2017-08-03 17:53:07 -04:00
Jake Champlin fa272e8c9c
Add more specific exists tests 2017-08-03 14:14:39 -04:00
Jake Champlin 268138dbd4
Rename to GetOkExists 2017-08-03 12:05:19 -04:00
Jake Champlin 270bbdb19b
core: Add `GetOkRaw` schema function
Adds `GetOkRaw` as a schema function. This should only be used to verify
boolean attributes are either set or not set, regardless of their zero
value for their type. There are a few small use cases outside of the boolean
type where this will be helpful as well.

Overall, this shouldn't detract from the zero-value checks that `GetOK()`
currently has, and should only be used when absolutely needed. However,
there are enough use-cases for this addition without checking for the
zero-value of the type, that this is needed.

Primary use case is for a boolean attribute that is `Optional` and `Computed`,
without a default value. There's currently no way to verify that the boolean
attribute was explicitly set to the zero-value literal with the current
`GetOk()` function. This new function allows for that check, keeping the
`Computed` check for the returned `exists` boolean.

```
$ make test TEST=./helper/schema TESTARGS="-run=TestResourceDataGetOkRaw"
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/08/02 11:17:32 Generated command/internal_plugin_list.go
go test -i ./helper/schema || exit 1
echo ./helper/schema | \
        xargs -t -n4 go test -run=TestResourceDataGetOkRaw -timeout=60s -parallel=4
go test -run=TestResourceDataGetOkRaw -timeout=60s -parallel=4 ./helper/schema
ok      github.com/hashicorp/terraform/helper/schema    0.005s
```
2017-08-02 11:18:59 -04:00
James Bardin ca68723c91 memoize DiffFieldReader.ReadField
The field reader code path is extremely inefficient, but refactoring
it all is much to invasive a change at the moment.

Have DiffFieldReader internally cache results for ReadField.
2017-07-28 14:26:36 -04:00
Lars Lehtonen e34f194b31
Fix swallowed err in acctest package 2017-07-19 16:24:41 -07:00
Radek Simko f979b8feef Enforce field names to be alphanum lowercase + underscores (#15562) 2017-07-17 08:37:46 +01:00
Radek Simko 07cbd54fbc Actively disallow reserved field names in schema (#15522) 2017-07-10 21:51:55 -07:00
James Bardin 60ea42cd1c Add testProviderConfig to import tests
Provider import tests previously didn't have to supply a config, but
terraform now requires the provider to be declared for discovery.

testProviderConfig returns a stub config with provider blocks based
on the TestCase Providers. This allows basic import tests in providers
to remain unchanged.
2017-07-07 16:16:35 -04:00
James Bardin 657932261b Make sure shadow.closeWalker doesn't copy Mutexes
The Close methods on shadow.Values require pointer receivers because
they contain a sync.Mutex, but that value was being copied through
Value.Interface by the closeWalker.  Because reflectwalk passes the
struct fields to the StructField method as they are defined in the
struct, and they may have been read as a value, we can't immediately
call Interface() to check the method set without possibly copying the
internal mutex values. Use the Implements method to first check if we
need to call Interface, and if it's not, then we can check if the value
is addressable.

Because of this use of reflection, we can't vet for the copying of these
locks. The minimal amount of code in the Close method left us only with
a race detected within the mutex itself, which leads to a stacktrace
pointing to the runtime rather than our code.
2017-07-07 11:20:54 -04:00
James Bardin c66dd48b6e make shadow.Value a Locker
This way it's correctly handled by CopyStructure
2017-07-06 16:58:29 -04:00
Riley Karson 8cf04f8e4e Added IntAtLeast and IntAtMost validation functions. (#15403) 2017-06-27 17:25:51 +03:00
joel.ringuette ffddf96603 Fix issue with reading timeouts on Delete
Original fix by @jringuette but I couldn't get his patch to apply after
the repo split :/
2017-06-23 09:15:29 -05:00
clint shryock 11dc0c5fdf Regression test for #767 (https://github.com/hashicorp/terraform/issues/14444) 2017-06-23 09:15:29 -05:00
Sander van Harmelen 7e180aec92 Refactor the provisioner validation function (#15273)
It turns out that `d.GetOk` also returns `false` when the user _did_ actually supply a value for it in the config, but the value itself needs to be evaluated before it can be used.

So instead of passing a `ResourceData` we now pass a `ResourceConfig`
which makes much more sense for doing the validation anyway.
2017-06-15 19:57:04 +02:00
David Glasser 0a1f9156dc Fix resource.UniqueId to be properly ordered over multiple runs
The timestamp prefix added in #8249 was removed in #10152 to ensure that
returned IDs really are properly ordered.  However, this meant that IDs were no
longer ordered over multiple invocations of terraform, which was the main
motivation for adding the timestamp in the first place.  This commit does a
hybrid: timestamp-plus-incrementing-counter instead of just incrementing counter
or timestamp-plus-random.
2017-06-15 08:09:23 -07:00
Martin Atkins c835ef8ff3 Update tests for the new ProviderResolver interface
Rather than providing an already-resolved map of plugins to core, we now
provide a "provider resolver" which knows how to resolve a set of provider
dependencies, to be determined later, and produce that map.

This requires the context to be instantiated in a different way, so this
very noisy diff is a mostly-mechanical update of all of the existing
places where contexts get created for testing, using some adapted versions
of the pre-existing utilities for passing in mock providers.
2017-06-09 14:03:59 -07:00
Martin Atkins 0573ff6793 helper/resource: pass config when testing import
Previously having a config was mutually exclusive with running an import,
but we need to provide a config so that the provider is declared, or else
we can't actually complete the import in the future world where providers
are installed dynamically based on their declarations.
2017-06-09 14:03:59 -07:00
Clint 372a80bc42 provider/aws: Add Sweeper setup, Sweepers for DB Option Group, Key Pair (#14773)
* provider/aws: Add Sweeper setup, Sweepers for DB Option Group, Key Pair

* provider/google: Add sweeper for any leaked databases
* more recursion and added LC sweeper, to test out the Dependency path

* implement a dependency example

* implement sweep-run flag to filter runs

* stub a test for TestMain

* test for multiple -sweep-run list
2017-06-06 10:34:17 -05:00
Radek Simko 1244309579 Fix stringer comments (#15069) 2017-06-05 10:17:35 +01:00
He Guimin 87562be855 provider/alicloud: Add the function of replacing ecs instance's system disk (#15048)
* add replacing system disk function for ecs

* remove ForceNew of system_disk_size
2017-06-05 11:27:49 +03:00
Jake Champlin ac177492fb
core: Revert stringer changes from earlier commits 2017-06-01 11:37:12 -04:00
Thomas Schaaf 79c91e11c8 provider/aws: Add aws elastic beanstalk solution stack (#14944)
* Add aws elastic beanstalk solution stack

Signed-off-by: Thomas Schaaf <thomaschaaf@Thomass-MBP.fritz.box>

* Fix incorrect naming

Signed-off-by: Thomas Schaaf <thomaschaaf@Thomass-MBP.fritz.box>

* Use unique go variable/function names

Signed-off-by: Thomas Schaaf <thomaschaaf@Thomass-MacBook-Pro.local>

* Add docs to sidebar

* Sort provider by alphabet

* Fix indent

* Add required statement

* Fix acceptance test
2017-06-01 02:23:06 +03:00
Jake Champlin 7894478c8c Merge pull request #14681 from svanharmelen/f-review
Use `helpers.shema.Provisoner` in Chef provisioner V2
2017-05-30 14:26:51 -04:00
Jake Champlin 24202fb3c1 Merge pull request #14578 from caiofilipini/digitalocean-certs
provider/digitalocean: Add support for certificates
2017-05-26 13:07:15 -04:00
James Bardin 6bc52be0a5 check for IsComputed when validating a schema obj
GH-14784 allowed nested structures to be validate, rather than relying
on the raw value. However this still returns the same validation error
if the structures contain a computed value, since Get will return the
raw string in that case.

This simply skips the validation in the IsComputed case, since there's
nothing that can be checked.
2017-05-24 12:34:53 -04:00
Matt Robenolt b9a3433f6b helper/schema: fix validating nested objects
When interpreting a nested object, we were validating against the "raw"
value, and not the interpolated value, causing incorrect errors.

This affects structures such as:

```tf
tags = "${list(map("foo", "bar"))}"
```

Prior to this, a complaint about "expected object, got string" since the
raw value is obviously a string, when the interpolated value is the
correct shape.
2017-05-24 01:57:13 -07:00
Sander van Harmelen 0e422737ba Fix and refactor the Chef provisioner
The tests did pass, but that was because they only tested part of the changes. By using the `schema.TestResourceDataRaw` function the schema and config are better tested and so they pointed out a problem with the schema of the Chef provisioner.

The `Elem` fields did not have a `*schema.Schema` but a `schema.Schema` and in an `Elem` schema only the `Type` field may (and must) be set. Any other fields like `Optional` are not allowed here.

Next to fixing that problem I also did a little refactoring and cleaning up. Mainly making the `ProvisionerS` private (`provisioner`) and removing the deprecated fields.
2017-05-19 21:05:21 +02:00
Vladislav Rassokhin f5449a62e0 Various built-in provisioners improvements:
1. Migrate `chef` provisioner to `schema.Provisioner`:

 * `chef.Provisioner` structure was renamed to `ProvisionerS`and  now it's decoded from `schema.ResourceData` instead of `terraform.ResourceConfig` using simple copy-paste-based solution;
 * Added simple schema without any validation yet.

 2. Support `ValidateFunc` validate function : implemented in `file` and `chef` provisioners.
2017-05-19 20:43:51 +02:00
Caio Filipini 45ad54c816 provider/digitalocean: Add support for certificates
Besides the support for DO certificates themselves, this commit also
includes:

1) A new `RandTLSCert` function that generates a valid, self-signed TLS
certificate to be used in the test
2) A fix for the PEM encoding of the private key generated in
`RandSSHKeyPair`: the PEM was always empty
2017-05-17 17:20:40 +02:00
Radek Simko f868a59ffa provider/google: Log HTTP requests and responses in DEBUG mode (#14281) 2017-05-10 21:16:43 +02:00
Radek Simko 44a99e0ae5 core: Avoid crash on empty TypeSet blocks (#14305) 2017-05-09 20:45:53 +02:00
Radek Simko 9867ce4dde
helper/schema: Disallow validation+diff suppression on computed-only fields 2017-04-23 12:25:40 +02:00
James Bardin 9aaf220efb Merge pull request #13863 from hashicorp/jbardin/computed-list-requires-new
make sure a computed list is can be RequiresNew
2017-04-21 18:02:14 -04:00
James Bardin caadb4297f make sure a computed list is can be RequiresNew
If a schema.TypeList had a Schema with ForceNew, and if that list was
NewComputed, the diff would not have RequiresNew set. This causes apply
to fail when the diffs didn't match because of the change to
RequiresNew.

Set the RequiresNew field on the list's ResourceAttrDiff based on the
Schema value.
2017-04-21 17:51:15 -04:00
Martin Atkins b1763e262a Restore stringer-generated files back to new version
stringer has changed the boilerplate it generates in a recent version.
We'd previously updated to the new format but accientally rolled back
to the old while merging a long-running feature branch.

This restores us back to the new format again.
2017-04-21 14:49:18 -07:00
Jasmin Gacic 61499cfcf0 Provider Oneandone (#13633)
* Terraform Provider 1&1

* Addressing pull request remarks

* Fixed imports

* Fixing remarks

* Test optimiziation
2017-04-21 17:19:10 +03:00
James Bardin f5cda342f7 Merge pull request #13778 from hashicorp/jbardin/GH-13617
improve resource.WaitForState and add refreshGracePeriod
2017-04-19 18:23:00 -04:00
James Bardin 4c3a053f0c lint errors 2017-04-19 18:19:48 -04:00
James Bardin 14bea66f4e add test for proper cancelation 2017-04-19 18:07:00 -04:00
James Bardin eb4b45941c fix tests affected by refreshGracePeriod
A couple tests require lowering the grace period to keep the test from
taking the full 30s timeout.

The Retry_hang test also needed to be removed from the Parallel group,
becuase it modifies the global refreshGracePeriod variable.
2017-04-19 18:07:00 -04:00
James Bardin af1628eaa4 add refreshGracePeriod
Refresh calls may have side effects that need to be recorded if it
succeeds, especially common when when WaitForState is called from
resource.Retry.

If the WaitForState timeout is reached and there is a Refresh call
in-flight, wait up to refreshGracePeriod (set to 30s) for it to
complete.
2017-04-19 18:07:00 -04:00
= 1608f5544f Added error check 2017-04-19 14:22:35 -06:00
= 6bc3610c09 removing extraneous code 2017-04-19 09:18:30 -06:00
James Bardin 6601b9b8dd adjust the inconsistent_negative test to match
This test unfortunately relies on the timing of the loops in
WaitForState, and the text of the error message. Adjust the timing so
the timeout isn't an even multiple of the poll interval, and make sure
we reach a minimum number of retries.
2017-04-19 10:53:06 -04:00
James Bardin af5e22cf94 don't leave WaitForState goroutine running
Make sure that we can cancel the WaitForState refresh loop when reaching
a timeout, otherwise it may run indefinitely. There's no need to try and
store and read the Result concurrently, just pass the value over a
channel.
2017-04-19 10:41:51 -04:00
Sander van Harmelen 3d0073e05c core: fix a crash by suggesting a different approach to solve #11170 (#13541)
* Revert #11245, #11321, #11498 and #11757

These PR’s are all related to issue #11170 for which I would like to propose a different solution then the one currently implemented.

* A different approach to solve #11170

This approach has (IMHO) a few advantages with regards to the solution currently implemented. I will elaborate on this in the PR.
2017-04-14 22:32:30 +02:00
Clint 9ef9501e65 provider/aws: Fix EMR Bootstrap Action Ordering (#13580)
* provider/aws: Add failing test for EMR Bootstrap Actions

* aws_emr_cluster: Fix bootstrap action parameter ordering

* provider/aws: Fix EMR Bootstrap arguments

* provider/aws: Args needs to be ForceNew, because we can't update them
2017-04-12 14:19:38 -05:00
James Bardin 735dfc416d Merge pull request #13427 from hashicorp/jbardin/context-keys
Fix context key types in schema
2017-04-10 15:38:27 -04:00
Martin Atkins a0269c688c helper/schema: Clarify the expectations for DefaultFunc
Discussion in #9512 revealed that some of the comments here were
inaccurate and that the comments here did not paint a complete enough
picture of the behavior and expectations of Default and DefaultFunc.

This is a comments-only change that aims to clarify the situation and
call attention to the fact that the defaults only affect the handling of
the configuration and that changes to defaults may require migration of
existing resource states.

This closes #9512.
2017-04-06 09:51:43 -07:00
James Bardin 8a1089a161 convert the other context keys to the correct type 2017-04-06 10:51:24 -04:00
James Bardin 812f9fb253 don't use primitive types for context value keys
A context value key should be typed within the package namespace,
otherwise different packages could have colliding values in a context.
2017-04-06 10:36:06 -04:00
Jake Champlin fed7b6e781 Merge pull request #13129 from tombuildsstuff/validation
provider/azurerm: Moving Reused JSON Validation -> Core
2017-04-04 16:11:13 -04:00
Jake Champlin f24087ee54
core: Add ImportStateIdPrefix field for testing
Adds the `ImportStateIdPrefix` field for import acceptance tests. There are (albeit fairly rare) import cases where a resource needs to be imported with a combination of the resource's ID and a known string prefix. This allows the developer to specify the known prefix, and omit the `ImportStateId` field.

```
$ make test TEST=./helper/resource TESTARGS="-run=TestTest_importStateIdPrefix"
==> Checking that code complies with gofmt requirements...
==> Checking AWS provider for unchecked errors...
==> NOTE: at this time we only look for uncheck errors in the AWS package
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/03/30 18:08:36 Generated command/internal_plugin_list.go
go test -i ./helper/resource || exit 1
echo ./helper/resource | \
                xargs -t -n4 go test -run=TestTest_importStateIdPrefix -timeout=60s -parallel=4
go test -run=TestTest_importStateIdPrefix -timeout=60s -parallel=4 ./helper/resource
ok      github.com/hashicorp/terraform/helper/resource  0.025s
```
2017-03-30 18:11:10 -04:00
Martin Atkins 21cd5595e2 Update stringer-generated files to new boilerplate
golang/tools commit 23ca8a263 changed the format of the leading comment
to comply with some new standards discussed here:
https://golang.org/issue/13560

This is the result of running generate with the latest version of
stringer. Everyone working on Terraform will need to update stringer
after this is merged, to avoid reverting this:
    go get -u golang.org/x/tools/cmd/stringer
2017-03-29 08:07:06 -07:00
James Nugent 0e3a7e6d0d helper/resource: Allow unknown pending states (#13099)
Sometimes when waiting on a target state, the set of valid states
through which a value will transition is unknown. This commit adds
support for an empty Pending slice and will treat any states that are not
the target as valid provided the timeouts are not exceeded.
2017-03-28 22:38:42 +03:00
tombuildsstuff 64dda483dd Adding tests for each helper method 2017-03-28 12:59:46 +01:00
James Nugent 5d734cee4e helper/acctest: Add RandIntRange helper function
This commit adds a new RandIntRange function to the helper/acctest
package for generating random integers in a given range. This is useful
when randomizing test spaces with a constrained range (e.g. 0-4095).
2017-03-27 20:30:30 -04:00
tombuildsstuff 6efdc94ae3 Splitting the structure tests 2017-03-27 17:54:21 +01:00
tombuildsstuff 459c9a0a5a Returning a blank string if there's nothing to serialize 2017-03-27 16:22:44 +01:00
tombuildsstuff 277d1b6b2d Refactoring the schema diff/validation -> core 2017-03-27 15:41:55 +01:00
James Bardin 2e6a44d5ff reify the list values before validation
If the list was marked as computed, all values will be raw config
values. Fetch the individual keys from the config to get any known
values before validating.
2017-03-24 12:04:18 -04:00
James Bardin 99a12f5df3 interpolation strings were beeing validated
Interpolation strings for non-computed values in a list were being
passed to the schema's ValidateFunc.
2017-03-24 12:04:18 -04:00
Joseph Anthony Pasquale Holsten 855adb47ed alicloud: simplify validators (#12982) 2017-03-23 10:57:11 +02:00
clint shryock 61355c33c5 testing: Add option to run only a plan on a TestStep configuration 2017-03-22 15:38:21 -05:00
James Nugent 9476fc4418 helper/acctest: Add NewSSHKeyPair function (#12894)
Many cloud services prevent duplicate key pairs with different names.
Among them are Digital Ocean, Joyent Triton and Packet. Consequently, if
tests leave dangling resources it is not enough to simply randomise the
name, the entire key material must be regenerated.

This commit adds a helper method that returns a new randomly generated
key pair, where the public key material is formatted in OpenSSH
"authorized keys" format, and the private key material is PEM encoded.
2017-03-20 20:42:34 +00:00
James Bardin efd0f5b0db Fix logic when skipping schema input
The Required||Optional logic in schemaMap.Input was incorrect, causing
it to always request input. Fix the logic, and the associated tests
which were passing "just because".
2017-03-17 14:55:24 -04:00
Mitchell Hashimoto df8529719c
command/init: backend-config accepts key=value pairs
This augments backend-config to also accept key=value pairs.
This should make Terraform easier to script rather than having to
generate a JSON file.

You must still specify the backend type as a minimal amount in
configurations, example:

```
terraform { backend "consul" {} }
```

This is required because Terraform needs to be able to detect the
_absense_ of that value for unsetting, if that is necessary at some
point.
2017-03-16 23:27:05 -07:00
Radek Simko 1df1c21d5b schema: Allow *Resource as Elem of TypeMap in validation (#12722) 2017-03-15 14:54:41 +00:00
Radek Simko afd34f79df schema: Enable map value validation (#12638) 2017-03-13 15:58:58 +00:00
Clint 3fdeacdca7 helper/schema: Rename Timeout resource block to Timeouts (#12533)
helper/schema: Rename Timeout resource block to Timeouts

- Pluralize configuration argument name to better represent that there is
one block for many timeouts
- use a const for the configuration timeouts key
- update docs
2017-03-09 14:40:14 -06:00
James Bardin ecb1944c31 Merge pull request #12498 from hashicorp/jbardin/test-reset
Add schema.Provider.TestReset to reset StopContext between tests
2017-03-09 08:34:54 -05:00
James Bardin 5238f51dc7 move TestReset mock from terraform to helper
the terraform package doesn't know about TestProvider, so don't put the
hooks in terraform.MockResourceProvider. Wrap the mock in the test where
we need to check the TestProvider functionality.
2017-03-08 17:48:11 -05:00
James Bardin 6fcb55d09e reword TestProvider doc 2017-03-08 17:41:35 -05:00
James Bardin 4b2e96b2e2 test for TestReset and fixed resource factories 2017-03-07 11:39:12 -05:00
James Bardin 1eb9a2d083 Reset ResourceProviders
Call all ResourceProviderFactories and reset the providers before tests.
Store the provider and/or the error in a fixed factory function to be
returned again later.
2017-03-07 11:39:12 -05:00
James Bardin 1d9d7be28c Add schema.Provider.TestReset method
Provider.TestReset resets the internal state of the Provider at the
start of a test. This also adds a MetaReset function field to
schema.Provider, which is called by TestReset and can be used to reset
any other tsated stored in the provider metadata.

This is currently used to reset the internal Context returned by
StopContext between tests, and should  be implemented by a provider if
it stores a Context from a previous test.
2017-03-07 11:39:11 -05:00
clint shryock aa3677cd89 helper/schema: Guard against a panic if Timeout is not properly
structured
2017-03-07 10:25:32 -06:00
Clint 2fe5976aec helper/schema: Add configurable Timeouts (#12311)
* helper/schema: Add custom Timeout block for resources

* refactor DefaultTimeout to suuport multiple types. Load meta in Refresh from Instance State

* update vpc but it probably wont last anyway

* refactor test into table test for more cases

* rename constant keys

* refactor configdecode

* remove VPC demo

* remove comments

* remove more comments

* refactor some

* rename timeKeys to timeoutKeys

* remove note

* documentation/resources: Document the Timeout block

* document timeouts

* have a test case that covers 'hours'

* restore a System default timeout of 20 minutes, instead of 0

* restore system default timeout of 20 minutes, refactor tests, add test method to handle system default

* rename timeout key constants

* test applying timeout to state

* refactor test

* Add resource Diff test

* clarify docs

* update to use constants
2017-03-02 11:07:49 -06:00
Mitchell Hashimoto 3342aa580c
terraform: InstanceState.Meta is value type interface{}
This changes the type of values in Meta for InstanceState to
`interface{}`. They were `string` before.

This will allow richer structures to be persisted to this without
flatmapping them (down with flatmap!). The documentation clearly states
that only primitives/collections are allowed here.

The only thing using this was helper/schema for schema versioning.
Appropriate type checking was added to make this change safe.

The timeout work @catsby is doing will use this for a richer structure.
2017-02-23 10:44:05 -08:00
Mitchell Hashimoto c6d0333dc0
flatmap: mark computed list as a computed value in Expand
Fixes #12183

The fix is in flatmap for this but the entire issue is a bit more
complex. Given a schema with a computed set, if you reference it like
this:

    lookup(attr[0], "field")

And "attr" contains a computed set within it, it would panic even though
"field" is available. There were a couple avenues I could've taken to
fix this:

1.) Any complex value containing any unknown value at any point is
entirely unknown.

2.) Only the specific part of the complex value is unknown.

I took route 2 so that the above works without any computed (since
"name" is not computed but something else is). This may actually have an
effect on other parts of Terraform configs, however those similar
configs would've simply crashed previously so it shouldn't break any
pre-existing configs.
2017-02-23 10:03:59 -08:00
Mitchell Hashimoto 2124135287
helper/slowmessage 2017-02-14 09:27:23 -08:00
Kit Ewbank 64fda44b00 Add 'aws_vpn_gateway' data source. (#11886) 2017-02-13 16:24:55 +00:00
James Bardin ced4c53324 Merge pull request #11757 from hashicorp/jbardin/GH-11588
Make sure to diff all nested schema.Set elements
2017-02-07 17:05:32 -05:00
James Bardin 7359a18a71 Make sure to diff all nested schema.Set elements
This follows on GH-11498, using the same method to ensure all set
elements are marked as NewRemoved if the set is being removed in the
diff.
2017-02-07 16:55:20 -05:00
Mitchell Hashimoto 864c79396d
terraform: default shadow to false
To avoid chasing down issues like #11635 I'm proposing we disable the
shadow graph for end users now that we have merged in all the new
graphs. I've kept it around and default-on for tests so that we can use
it to test new features as we build them. I think it'll still have value
going forward but I don't want to hold us for making it work 100% with
all of Terraform at all times.

I propose backporting this to 0-8-stable, too.
2017-02-06 18:02:32 -08:00
Mitchell Hashimoto 61881d2795 Merge pull request #10934 from hashicorp/f-provisioner-stop
core: stoppable provisioners, helper/schema for provisioners
2017-01-30 12:53:15 -08:00
Radek Simko d5ac48de2a helper/schema: Remove missed subfields when parent list is removed (#11498) 2017-01-29 21:15:00 +00:00
Mitchell Hashimoto 09242fab09
terraform: remove legacy graph builder 2017-01-26 15:18:42 -08:00
Mitchell Hashimoto 487a37b0dd
helper/schema: PromoteSingle for legacy support of "maybe list" types 2017-01-26 15:09:15 -08:00
Mitchell Hashimoto 3c0c81957a
provisioners/remote-exec: switch to helper/schema 2017-01-26 15:09:15 -08:00
Mitchell Hashimoto a2e044829b
provisioners/file: use the old communicator.New just to minimize risk 2017-01-26 15:03:28 -08:00
Mitchell Hashimoto a1da59a73e
helper/schema: provisioner allows for nil state 2017-01-26 15:03:28 -08:00
Mitchell Hashimoto b2891bc9ef
helper/schema: Provisioner support 2017-01-26 15:03:27 -08:00
Mitchell Hashimoto 251e5c6f87
helper/schema: framework for Backends 2017-01-26 14:33:49 -08:00
Kit Ewbank 87d98b1ff1 Correct aws_s3_bucket_object data source acceptance tests. (#11346) 2017-01-23 21:53:02 +00:00
Radek Simko 6222b3e91d
helper/resource: Add retries to err message 2017-01-20 11:09:04 -08:00
Radek Simko be4e5e53a9
Add more details (Retries) to resource.NotFoundError 2017-01-20 07:08:30 -08:00
Conor Mongey 77c8683281 provider/vault: Remove user input for optional vault provider fields (#11082)
* Remove the need to input vault optional settings

* Allow TypeList to skip input

* Remove conflicts on vault ca_cert_* fields
2017-01-17 12:06:55 +00:00
Sean Chittenden 7862e5b499
Differentiate between lookup failure and value inequality.
Before this patch it was not possible to test for a key in a map where
the value is an empty string.  With this patch, however, it is now
possible to write a check like:

```
resource.TestCheckResourceAttr("res.name", "mymap.KeyWithEmptyValue", ""),
```

To test that `KeyWithEmptyValue` is a valid key in `mymap`.
2017-01-11 23:28:48 -08:00
clint shryock 6bafd4c896 fix typo 2017-01-10 11:06:51 -06:00
Mitchell Hashimoto 6d594b3bc6
helper/variables: trim whitespace on key before empty string comp 2016-12-14 13:52:34 -08:00
Mitchell Hashimoto b7f6f8eb2a
helper/variables: trim whitespace around the key in -var
Fixes #10716

This trims whitespace around the key in the `-var` flag.

This is a regression from 0.7.x.

The value is whitespace sensitive unless double-quoted. This is the same
behavior as 0.7.x. I considered rejecting whitespace around the '='
completely but I don't want to introduce BC and the behavior actually
seems quite obvious to me.
2016-12-13 21:01:01 -08:00
Mitchell Hashimoto 42edd22566
helper/variables: address go vet 2016-12-10 20:32:50 -05:00
Mitchell Hashimoto 83e72b0361
helper/variables: ParseInput for consistent parsing 2016-12-10 14:44:17 -05:00
Mitchell Hashimoto 39e3d8ea9b
helper/variables: helpers for working with vars 2016-12-10 14:30:36 -05:00
James Nugent 3177fc0765 provider/aws: Extract helper/encryption library
This commit extracts the GPG code used for aws_iam_user_login_profile
into a library that can be reused for other resources, and updates the
call sites appropriately.
2016-12-06 14:24:21 -06:00
James Bardin 7677bd94ed Merge pull request #10325 from hashicorp/jbardin/GH-10187
Fix some cases for nested maps and lists
2016-11-29 12:24:53 -05:00
James Bardin 01be1a5ecd Check for interpolated values when reading a map
Accessing an interpolated value in a map through ConfigFieldReader can
fail, because GetRaw can't access interpolated values, so check if the
value exists at all by looking in the config. If the config has a value,
assume our map's value is interpolated and proceed as such.

We also need to lookup the correct schema to properly read a field from
a nested structure.

- Maps previously always defaulted to TypeString. Now check if Elem is a
  ValueType and use that if applicable
- Lists now return the schema for nested element types, defaulting to a
  TypeString like maps.

This only allows maps and lists to be nested one level deep, and the
inner map or list must only contain string values.
2016-11-28 09:04:24 -05:00
Paul Tyng fead481002 Fix typo in test method name (#10370) 2016-11-25 16:20:12 +00:00
Mitchell Hashimoto 0634aada69
Revert "core: Ensure hasComputedSubKeys iterates over Sets and Lists properly"
This reverts commit 4d8208d840.
2016-11-21 17:31:19 -08:00
Mitchell Hashimoto f2fcf1494a
Merge branch 'fix-field-reader-setsubkeys-computed' of https://github.com/vancluever/terraform into vancluever-fix-field-reader-setsubkeys-computed 2016-11-21 17:24:26 -08:00
Mitchell Hashimoto fd36b548c5
helper/wrappedstreams: get original console input/output on Windows
Fixes #10266

panicwrap was using Extrafiles to get the original standard streams for
`terraform console`. This doesn't work on Windows. Instead, we must use
the Win32 APIs to get the exact handles.
2016-11-21 10:44:01 -08:00
James Bardin 24ebb72920 Merge pull request #10152 from hashicorp/jbardin/unique-id
Fix resource.UniqueId to be properly ordered
2016-11-21 09:13:26 -05:00
Chris Marchesi 8d06d68d0f core: Backport NewComputed change to nested list/set tests
Needed due to work done in 95d37ea, we may need to adjust
hasComputedSubKeys to propagate NewComputed in the same way that we
have added "~", however will wait for comment from @mitchellh.
2016-11-19 09:29:48 -08:00
Chris Marchesi f258056731 core: Tests for hasComputedSubKeys fix
This covers:

 * Complex sets with computed fields in a set
 * Complex lists with computed fields in a set

Adding a test to test basic lists with computed fields seemed to fail,
but possibly for an unrelated reason (the list returned as nil). The fix
to this inparticular case may be out of the scope of this specific
issue.

Reference gist and details in hashicorp/terraform#9171.
2016-11-19 08:56:16 -08:00
Chris Marchesi 4d8208d840 core: Ensure hasComputedSubKeys iterates over Sets and Lists properly
This fixes some edge-ish cases where a set in a config has a set or list
in it that contains computed values, but non-set or list values in the
parent do not.

This can cause "diffs didn't match during apply" errors in a scenario
such as when a set's hash is calculated off of child items (including
any sub-lists or sets, as it should be), and the hash changes between
the plan and apply diffs due to the computed values present in the
sub-list or set items. These will be marked as computed, but due to the
fact that the function was not iterating over the list or set items
properly (ie: not adding the item number to the address, so
set.0.set.foo was being yielded instead of set.0.set.0.foo), these
computed values were not being properly propagated to the parent set to
be marked as computed.

Fixes hashicorp/terraform#6527.
Fixes hashicorp/terraform#8271.

This possibly fixes other non-CloudFront related issues too.
2016-11-19 08:56:16 -08:00
James Bardin c4eefd4b5e Allow primitive type in maps via all FieldReaders
Now that we can read primitive type out of a map, each field reader
needs to be able to set the proper type as expected.
2016-11-17 15:35:08 -05:00
James Bardin 730014b33e Convert the map fields values when reading diff
Convert the value to the correct type when reading a diff and the map
schema has an primitive Elem type.
2016-11-17 14:34:18 -05:00
Radek Simko ad34f1ec74 Add (failing) test for map w/ non-string values 2016-11-17 11:46:32 -05:00
James Bardin e28e11d44c Fix resource.UniqueId to be properly ordered
UniqueId attempted to provide an ordered unique id by using a nanosecond
timestamp, but doesn't take into account that time is not monotonic
increasing. This provides an implementation that will always be
increasing.
2016-11-15 21:58:23 -05:00
Mitchell Hashimoto 39542898b0
helper/schema: mark diff as forcenew if element is computed
Fixes #10125

If the elements are computed and the field is ForceNew, then we should
mark the computed count as potentially forcing a new operation.

Example, assuming `groups` forces new...

**Step 1:**

    groups = ["1", "2", "3"]

At this point, the resource isn't create, so this should result in a
diff like:

    CREATE resource:
      groups: "" => ["1", "2", "3"]

**Step 2:**

    groups = ["${computedvar}"]

The OLD behavior was:

    UPDATE resource
      groups.#: "3" => "computed"

This would cause a diff mismatch because if `${computedvar}` was
different then it should force new. The NEW behavior is:

    DESTROY/CREATE resource:
      groups.#: "3" => "computed" (forces new)
2016-11-15 11:02:14 -08:00
Mitchell Hashimoto 6557a3de18
helper/shadow: Close for auto-closing all values
Fixes #10122

The simple fix was that we forgot to close `ReadDataApply` for the
provider. But I've always felt that this section of the code was brittle
and I wanted to put in a more robust solution. The `shadow.Close` method
uses reflection to automatically close all values.
2016-11-15 08:54:26 -08:00
Mitchell Hashimoto 1a6056b287
command: split out and tag code so compilation works on Solaris
The readline library doesn't support Solaris. For now, we'll just not
support console there.
2016-11-14 00:32:01 -08:00
Mitchell Hashimoto a867457d75
command/console 2016-11-13 23:17:04 -08:00
Mitchell Hashimoto 73a1564dac
helper/wrappedreadline: helper for dealing with wrapped standard streams 2016-11-13 23:17:03 -08:00
Mitchell Hashimoto 785cc7b78a
terraform: default new graphs on, old graphs behind -Xlegacy-graph
This turns the new graphs on by default and puts the old graphs behind a
flag `-Xlegacy-graph`. This effectively inverts the current 0.7.x
behavior with the new graphs.

We've incubated most of these for a few weeks now. We've found issues
and we've fixed them and we've been using these graphs internally for
awhile without any major issue. Its time to default them on and get them
part of a beta.
2016-11-10 21:53:20 -08:00
Mitchell Hashimoto 15f50b86bf
helper/schema: passing tests for computed complex sets 2016-11-10 08:52:37 -08:00
Mitchell Hashimoto 59bd1a22f4
helper/diff: handle unknownvariablevalue 2016-11-09 14:28:16 -08:00
Mitchell Hashimoto 29485f6167
terraform: ResourceConfig.IsComputed cases 2016-11-09 14:28:15 -08:00
Mitchell Hashimoto 5792b2cba2
helper/schema: convert _Diff to subtests 2016-11-09 14:28:15 -08:00
Mitchell Hashimoto 2b7177cfe7 Merge pull request #9607 from hashicorp/f-provider-stop-redo
terraform: ResourceProvider.Stop (redo)
2016-11-08 15:58:48 -08:00
Mitchell Hashimoto e45debe0e5
helper/schema: only mark "ForceNew" on resources that cause the ForceNew
Fixes #2748

This changes the diff to only mark "forces new resource" on the fields
that actually caused the new resource, not every field that changed.
This makes diffs much more accurate.

I'd like to request a review but I'm going to defer merging until
Terraform 0.8. Changes like this are very possible to cause "diffs
didn't match" errors and I want some real world testing in a beta before
we hit prod with this.
2016-11-08 15:49:28 -08:00
Mitchell Hashimoto b7bee66df5
helper/schema: sort errors in helper/schema test for deterministic tests 2016-11-04 16:51:26 -07:00
Mitchell Hashimoto f0abe6d1a0 Merge pull request #9812 from hashicorp/b-bool-computed-crash
helper/schema: computed bool fields should not crash
2016-11-04 08:47:49 -07:00
Mitchell Hashimoto 65b17ccd06
helper/schema: allow ConflictsWith and Computed Optional fields 2016-11-02 22:24:34 -07:00
Mitchell Hashimoto 7834cf7190
helper/schema: computed bool fields should not crash
Fixes #7715

If a bool field was computed and the raw value was not convertable to a
boolean, helper/schema would crash. The correct behavior is to try not
to read the raw value when the value is computed and to simply mark that
it is computed. This does that (and matches the behavior of the other
primitives).
2016-11-02 13:25:23 -07:00
Mitchell Hashimoto 2d84582881 Merge pull request #9699 from hashicorp/b-removed-forcenew
helper/schema: removed optional items force new
2016-10-31 13:24:36 -07:00
Mitchell Hashimoto f7a234bc71
helper/schema: validate Read, Delete are set 2016-10-30 15:04:32 -07:00
Mitchell Hashimoto 5489d8c549
helper/schema: removed optional items force new
Fixes #5138

If an item is optional and is removed completely from the configuration,
it should still trigger a destroy/create if the field itself was marked
as "ForceNew".

See the example in #5138.
2016-10-28 18:45:12 -04:00
Mitchell Hashimoto 3f36787207 Merge pull request #9618 from hashicorp/b-computed-prim
helper/schema,terraform: handle computed primitives in diffs
2016-10-28 10:44:13 -04:00
Mitchell Hashimoto af82be19ea
helper/experiment: a helper for setting, making experiments
This creates a standard package and interface for defining, querying,
setting experiments (`-X` flags).

I expect we'll want to continue to introduce various features behind
experimental flags. I want to make doing this as easy as possible and I
want to make _removing_ experiments as easy as possible as well.

The goal with this packge has been to rely on the compiler enforcing our
experiment references as much as possible. This means that every
experiment is a global variable that must be referenced directly, so
when it is removed you'll get compiler errors where the experiment is
referenced.

This also unifies and makes it easy to grab CLI flags to enable/disable
experiments as well as env vars! This way defining an experiment is just
a couple lines of code (documented on the package).
2016-10-26 15:47:58 -04:00
Mitchell Hashimoto 95d37ea79c
helper/schema,terraform: handle computed primtives in diffs
Fixes #3309

There are two primary changes, one to how helper/schema creates diffs
and one to how Terraform compares diffs. Both require careful
understanding.

== 1. helper/schema Changes

helper/schema, given any primitive field (string, int, bool, etc.)
_used to_ create a basic diff when given a computed new value (i.e. from
an unkown interpolation). This would put in the plan that the old value
is whatever the old value was, and the new value was the actual
interpolation. For example, from #3309, the diff showed the following:

```
~ module.test.aws_eip.test-instance.0
    instance: "<INSTANCE ID>" => "${element(aws_instance.test-instance.*.id, count.index)}"
```

Then, when running `apply`, the diff would be realized and you would get
a diff mismatch error because it would realize the final value is the
same and remove it from the diff.

**The change:** `helper/schema` now marks unknown primitive values with
`NewComputed` set to true. Semantically this is correct for the diff to
have this information.

== 2. Terraform Diff.Same Changes

Next, the way Terraform compares diffs needed to be updated

Specifically, the case where the diff from the plan had a NewComputed
primitive and the diff from the apply _no longer has that value_. This
is possible if the computed value ended up being the same as the old
value. This is allowed to pass through.

Together, these fix #3309.
2016-10-25 22:36:59 -04:00
Mitchell Hashimoto ad5a82de75
helper/resource: remove StopCh, use contexts instead 2016-10-25 12:08:36 -07:00
Mitchell Hashimoto 5ee8042dff
helper/schema: expose stop information as a Context 2016-10-25 12:08:36 -07:00
Mitchell Hashimoto be34dfe7c1
helper/resource: StopCh as a helper for provider stopCh + timeout 2016-10-25 12:08:35 -07:00
Mitchell Hashimoto d1bb2f3487
helper/schema: support Stop() 2016-10-25 12:08:35 -07:00
Mitchell Hashimoto 60140b28f4
Revert "Merge pull request #9536 from hashicorp/f-provider-stop"
This reverts commit c3a4cff133, reversing
changes made to 791a02e6e4.

This change requires plugin recompilation and we should hold off until a
minor release for that.
2016-10-25 12:00:36 -07:00
Mitchell Hashimoto 61bbaf6f85
helper/resource: remove StopCh, use contexts instead 2016-10-25 11:32:30 -07:00
Mitchell Hashimoto 86eb30b8a2
helper/schema: expose stop information as a Context 2016-10-25 11:32:30 -07:00
Mitchell Hashimoto 89647745b0
helper/resource: StopCh as a helper for provider stopCh + timeout 2016-10-25 11:31:56 -07:00
Mitchell Hashimoto 8c11f137f5
helper/schema: support Stop() 2016-10-25 11:31:55 -07:00
James Nugent 43dd13cd36 helper/schema: Introduce Noop and RemoveFromState
This commit implements reusable functions for when resources have no
need to implement a particular operation:

- Noop - does nothing and returns no error.
- RemoveFromState - sets the resource ID to empty string (removing it
  from state) and returns no error.
2016-10-25 11:47:03 -05:00
Mitchell Hashimoto fa9758e162
helper/schema: test with DiffSuppress and Default 2016-10-24 22:23:13 -07:00
Mitchell Hashimoto de827887bf
helper/shadow: keyedValue.WaitForChange must unlock 2016-10-21 17:57:00 -07:00
Mitchell Hashimoto 0fe51b334c Merge pull request #9334 from hashicorp/f-shadow-graph
terraform: Shadow Graph
2016-10-19 13:36:10 -07:00
Sean Chittenden b1c3649eac
Append to debug log files, don't clobber. 2016-10-18 13:08:23 -07:00
Mitchell Hashimoto 10bcdd04d4
helper/shadow: KeyedValue.Init 2016-10-12 18:45:40 +08:00
James Bardin 404a76e2b9 Merge pull request #9144 from hashicorp/b-source
config/module: use the raw source as part of the key
2016-10-11 12:29:42 -04:00
Mitchell Hashimoto 50e0647c53
helper/shadow: ComparedValue 2016-10-11 22:17:30 +08:00
Mitchell Hashimoto c92ee5a8bd
helper/shadow: KeyedValue.WaitForChange returns immediately if closed 2016-10-11 22:17:29 +08:00
Mitchell Hashimoto 0408c2dfb2
helper/shadow: KeyedValue.WaitForChange 2016-10-11 22:17:29 +08:00
Mitchell Hashimoto 47f4343bf5
helper/shadow: KeyedValue add test case to avoid panic 2016-10-11 22:17:29 +08:00
Mitchell Hashimoto 136ac4728d
helper/shadow: KeyedValue.Close 2016-10-11 22:17:29 +08:00
Mitchell Hashimoto d2fb630df8
helper/shadow: Value.Close 2016-10-11 22:17:29 +08:00
Mitchell Hashimoto d6168edc50
helper/shadow: KeyedValue.ValueOk 2016-10-11 22:17:28 +08:00
Mitchell Hashimoto bd69e41c14
helper/shadow: KeyedValue 2016-10-11 22:17:27 +08:00
Mitchell Hashimoto 8426cea6b0
helper/shadow: OrderedValue 2016-10-11 22:17:27 +08:00
Mitchell Hashimoto 1df3bbdc37
terraform: working on the resource provider shadow, not working yet 2016-10-11 22:17:27 +08:00
Paul Stack 4ac7d0a6bb Merge pull request #8103 from BedeGaming/validation-helper
helper: create validation package to provide common validation functions
2016-10-07 13:42:29 +01:00
stack72 5a537cdbf9
helper/schema: Adding of MinItems as a validation to Lists and Maps
This is required for the times when the configuration cannot have an
empty configuration. An example would be in AzureRM, when you create a
LoadBalancer with a configuration, you can delete *all* but 1 of these
configurations
2016-10-04 18:57:58 +01:00
Paul Hinze 534a5a3b75 Merge pull request #8891 from gotascii/fix-diff-set-update
Unchanged NestedSets are not returned by DiffFieldReader.
2016-10-03 11:06:40 -05:00
Mitchell Hashimoto 098225dc0d
config/module: use the raw source as part of the key
This changes the key for the storage to be the _raw_ source from the
module, not the fully expanded source. Example: it'll be a relative path
instead of an absolute path.

This allows the ".terraform/modules" directory to be portable when
moving to other machines. This was a behavior that existed in <= 0.7.2
and was broken with #8398. This amends that and adds a test to verify.
2016-09-30 10:44:46 -07:00
Justin Marney 895d10a627 Use DeepEqual instead of multiple if statements to reduce noise. 2016-09-17 17:41:44 +00:00
Justin Marney 53e10dfa89 Name test appropriately. 2016-09-17 16:56:12 +00:00
Justin Marney 9217f5a063 Unchanged NestedSets are not returned by DiffFieldReader. 2016-09-16 23:57:33 +00:00
Radek Simko 86acdccaf2 helper/resource: Add timeout to TimeoutError msg (#8773) 2016-09-15 10:53:25 +01:00
James Nugent e0226c9039 core: Check for attrV being nil before dereference
This can be an issue with unset computed fields.

Fixes #8815.
2016-09-14 09:51:15 +01:00
James Nugent 6c23181686 Merge pull request #8383 from kjmkznr/import-aws-s3-b-notification
provider/aws: Support import `aws_s3_bucket_notification`
2016-09-03 15:50:25 -07:00
Paul Stack bf755bb5c9 Merge pull request #8585 from hashicorp/f-diff-suppression
helper/schema: Add diff suppression callback
2016-09-01 14:46:03 +01:00
James Nugent 85ec09111b helper/schema: Add diff suppression callback
This commit adds a new callback, DiffSuppressFunc, to  the schema.Schema
structure. If set for a given schema, a callback to the user-supplied
function will be made for each attribute for which the default
type-based diff mechanism produces an attribute diff. Returning `true`
from the callback will suppress the diff (i.e. pretend there was no
diff), and returning false will retain it as part of the plan.

There are a number of motivating examples for this - one of which is
included as an example:

1. On SSH public keys, trailing whitespace does not matter in many
   cases - and in some cases it is added by provider APIs. For
   digitalocean_ssh_key resources we previously had a StateFunc that
   trimmed the whitespace - we now have a DiffSuppressFunc which
   verifies whether the trimmed strings are equivalent.

2. IAM policy equivalence for AWS. A good proportion of AWS issues
   relate to IAM policies which have been "normalized" (used loosely)
   by the IAM API endpoints. This can make the JSON strings differ
   from those generated by iam_policy_document resources or template
   files, even though the semantics are the same (for example,
   reordering of `bucket-prefix/` and `bucket-prefix/*` in an S3
   bucket policy. DiffSupressFunc can be used to test for semantic
   equivalence rather than pure text equivalence, but without having to
   deal with the complexity associated with a full "provider-land" diff
   implementation without helper/schema.
2016-08-31 19:13:53 -05:00
James Bardin e0014198e1 Merge pull request #8560 from hashicorp/jbardin/race2
Fix races in WaitForState
2016-08-31 14:02:43 -04:00
Mitchell Hashimoto aaaed823af Merge pull request #8508 from TimeIncOSS/f-schema-all-validation-errors
schema: Return all validation errors together from InternalValidate
2016-08-31 10:44:13 -07:00
James Bardin 481f12cf2c Remove `first` variable an extra if block
clean up the code slighly by moving the Sleep in WaitForState to the end
of the loop.
2016-08-30 17:56:26 -04:00
James Bardin 82be35a797 Fix races in WaitForState
The WaitForState method can't read the result values in a timeout
because they are still owned by the running goroutine. Keep all values
scoped inside the goroutine, and save them into an atomic.Value to be
returned.

Fixes race introduced in #8510
2016-08-30 16:22:21 -04:00
Radek Simko 506c118383
helper/resource: Show last state in timeout err message 2016-08-27 21:25:14 +01:00
Radek Simko aef2513b44
helper/resource: Fix WaitForState tests 2016-08-27 15:11:09 +01:00
Radek Simko b90ab0b705
schema: Return all validation errors together from InternalValidate 2016-08-27 10:34:16 +01:00