Commit Graph

492 Commits

Author SHA1 Message Date
Paddy Carver b7d19e40f8 Deprecated -> DeprecationMessage.
At the Enablement team's request, change from using `Deprecated` to
`DeprecationMessage`, as it's a string value, not a boolean.
2018-06-20 11:21:46 -07:00
Paddy Carver 0dd2408d65 Export resource deprecation.
We already had the functionality to make resources deprecated, which was
used when migrating resources to data sources, but the functionality was
unexported, so only the schema package could do it. Now it's exported,
meaning providers can mark entire resources as deprecated. I also added
a test in hopefully-the-right place?
2018-06-19 12:02:55 -07:00
kt a182a53108
Add a HashInt helper/schema function 2018-04-27 00:04:41 -07:00
Chris Marchesi d7048cb640
helper/schema: ResourceDiff ForceNew attribute correctness
A couple of bugs have been discovered in ResourceDiff.ForceNew:

* NewRemoved is not preserved when a diff for a key is already present.
This is because the second diff that happens after customization
performs a second getChange on not just state and config, but also on
the pre-existing diff. This results in Exists == true, meaning nil is
never returned as a new value.
* ForceNew was doing the work of adding the key to the list of changed
keys by doing a full SetNew on the existing value. This has a side
effect of fetching zero values from what were otherwise undefined values
and creating diffs for these values where there should not have been
(example: "" => "0").

This update fixes these scenarios by:

* Adding a new private function to check the existing diff for
NewRemoved keys. This is included in the check on new values in
diffChange.
* Keys that have been flagged as ForceNew (or parent keys of lists and
sets that have been flagged as ForceNew) are now maintained in a
separate map. UpdatedKeys now returns the results of both of these maps,
but otherwise these keys are ignored by ResourceDiff.
* Pursuant the above, values are no longer pushed into the newDiff
writer by ForceNew. This prevents the zero value problem, and makes for
a cleaner implementation where the provider has to "manually" SetNew to
update the appropriate values in the writer. It also prevents
non-computed keys from winding up in the diff, which ResourceDiff
normally blocks by design.

There are also a couple of tests for cases that should never come up
right now involving Optional/Computed values and NewRemoved, for which
explanations are given in annotations of each test. These are here to
guard against future regressions.
2018-04-08 07:50:41 -07:00
James Bardin 61eae050ab
Merge pull request #17588 from creack/creack/fix-deepcopy-type
Fix issue with deepcopy returning wrong type causing panic
2018-04-05 09:24:08 -04:00
Chris Marchesi db5ed51703
Merge pull request #17667 from hashicorp/f-resource-diff-getokexists-computed
helper/schema: Add Computed to ResourceDiff, expose GetOkExists
2018-04-02 07:32:04 -07:00
Adam Lewandowski 56a330c533 Remove attribute values from ConflictsWith validation message 2018-03-30 07:53:59 -04:00
Chris Marchesi fdfe89ed1d
helper/schema: Computed -> NewValueKnown, add comments to GetOkExists
Added some more context to GetOkExists, moved Computed to NewValueKnown
to accommodate some changes that will be coming up in HCL2 that may make
"Computed" less intuitive of a function name, and updated the docs for
NewValueKnown as well.
2018-03-29 15:18:35 -07:00
Chris Marchesi 274b933077
helper/schema: Add Computed to ResourceDiff, expose GetOkExists
This adds a new method to ResourceDiff: Computed, which exposes the
computed read result field to ResourceDiff. In the context of
customizing the diff, this is important as interpolated and otherwise
computed values will show up in the diff as blank, with no way of
determining if the value is actually blank or if it's a computed value
not available at diff customization time. Currently assumptions need to
be made on this, but this does not help in validation scenarios where
one needs to differentiate between an actual blank value and a value
that will be available later.

This is exposed for the most part via NewComputed in the diff, but the
tests cover both the config reader as well (with no diff, even though
this should not come up in normal operation) and also the newDiff reader
when someone sets a new value using SetNew and SetNewComputed.

This commit also exposes GetOkExists. The tests were mostly pulled from
ResourceData but a few were added to ensure that config was being
properly covered as well, in addition to covering SetNew and
SetNewComputed.
2018-03-29 12:04:36 -07:00
James Bardin 798df9dafa make sure ResourceData timeouts are always set
Return the global default timeout if the ResourceData timeouts are nil.

Set the timeouts from the Resource when calling Resource.Data, so that
the config values are always available.
2018-03-22 15:10:43 -04:00
Guillaume J. Charmes a4f941368b
Fix issue with deepcopy returning wrong type causing panic
Signed-off-by: Guillaume J. Charmes <gcharmes@magicleap.com>
2018-03-14 19:34:23 -04:00
James McGill 035d56409f helper/schema: handle TypeMap elem consistently with other collection types
For historical reasons, the handling of element types for maps is inconsistent with other collection types.

Here we begin a multi-step process to make it consistent, starting by supporting both the "consistent" form of using a schema.Schema and an existing erroneous form of using a schema.Type directly. In subsequent commits we will phase out the erroneous form and require the schema.Schema approach, the same as we do for TypeList and TypeSet.
2018-03-14 14:50:41 -07:00
Radek Simko ccf8a31cbb
helper/schema: Allow ResourceDiff.ForceNew on nested fields (avoid crash) 2018-03-01 15:51:01 +00:00
Radek Simko 7af1c2b3a4
helper/schema: Prevent crash on removal of computed field in CustomizeDiff 2018-02-01 12:05:22 +00:00
Stefan Schmidt ee6c8a777a s/repalce/replace 2018-01-23 17:41:03 +01:00
James Bardin 4b49a323c3 go fmt
slight change to go fmt coming in 0.10
2017-12-26 13:26:38 -05:00
James Bardin 0df8da59f7 add FIXMEs
This new codepath with the getDiff "customzed" return value, along with
the associated test need to be removed as soon as we can support unset
fields from the config, so we don't continue to carry this broken
behavior forward any longer than needed.
2017-12-20 08:51:00 -05:00
Chris Marchesi cb5ce1d35e
helper/schema: Extend diffChange and bubble up customized values
This extends the internal diffChange method so that ResourceDiff's
implementation of it can report back whether or not the value came from
a customized diff.

This is an effort to work to preserve the pre-ResourceDiff behaviour
that ignores the diff for computed keys when the old value was populated
but the new value wasn't - this behaviour is actually being depended on
by users that are using it to exploit using zero values in modules. This
should allow both scenarios to co-exist by shifting the NewComputed
exemption over to exempting values that come from diff customization.
2017-12-19 16:06:57 -08:00
James Bardin 643ef4334f revert the change that broke the test case
This reverts one of the changes from 6a4f7b0, which broke empty strings
being seen as unset for computed values.

This breaks a number of other tests, and is only an intermediate change
for evaluating other solutions.
2017-12-19 16:14:07 -05:00
James Bardin 7a8a443994 add failing test case
This case should be expected to fail with the current diff algorithm,
but the existing behavior was widely relied upon so we need to roll this
back until there is a representable nil value.
2017-12-19 15:14:58 -05:00
Rob Campbell 5daeee5f6d Update various files for new version of "stringer"
The latest version of stringer now uses strconv instead of fmt.
2017-12-11 13:26:29 -08:00
Radek Simko 2974d63e75
Merge pull request #16588 from hashicorp/f-panic-on-invalid-rd-set
helper/schema: Opt-in panic on invalid ResourceData.Set
2017-11-08 19:17:46 +00:00
Radek Simko e93d64b18c
helper/schema: Opt-in panic on invalid ResourceData.Set 2017-11-08 10:05:11 +00:00
Chris Marchesi 3fa11e456b
helper/schema: Clear existing map/set/list contents before overwriting
There are situations where one may need to write to a set, list, or map
more than once per single TF operation (apply/refresh/etc). In these
cases, further writes using Set (example: d.Set("some_set", newSet))
currently create unstable results in the set writer (the name of the
writer layer that holds the data set by these calls) because old keys
are not being cleared out first.

This bug is most visible when using sets. Example: First write to set
writes elements that have been hashed at 10 and 20, and the second write
writes elements that have been hashed at 30 and 40. While the set length
has been correctly set at 2, since a set is basically a map (as is the
entire map writer) and map results are non-deterministic, reads to this
set will now deliver unstable results in a random but predictable
fashion as the map results are delivered to the caller non-deterministic
- sometimes you may correctly get 30 and 40, but sometimes you may get
10 and 20, or even 10 and 30, etc.

This problem propagates to state which is even more damaging as unstable
results are set to state where they become part of the permanent data
set going forward.

The problem also applies to lists and maps. This is probably more of an
issue with maps as a map can contain any key/value combination and hence
there is no predictable pattern where keys would be overwritten with
default or zero values. This is contrary to complex lists, which has
this problem as well, but since lists are deterministic and the length
of a list properly gets updated during the overwrite, the problem is
masked by the fact that a read will only read to the boundary of the
list, skipping any bad data that may still be available due past the
list boundary.

This update clears the child contents of any set, list, or map before
beginning a new write to address this issue. Tests are included for all
three data types.
2017-11-05 12:04:23 -08:00
Chris Marchesi 0c0ae3ca7c helper/schema: CustomizeDiff allowed on writable resources only
This keeps CustomizeDiff from being defined on data sources, where it
would be useless. We just catch this in InternalValidate like the rest
of the CRUD functions that are not used in data sources.
2017-11-01 15:42:24 -07:00
Chris Marchesi 2c541e8c97 helper/schema: Add behaviour detail to various custom diff comments
Added some more detailed comments to CustomizeDiff's comments. The new
comments detail how CustomizeDiff will be called in the event of
different scenarios like creating a new resource, diffing an existing
one, diffing an existing resource that has a change that requires a new
resource, and destroy/tainted resources.

Also added similar detail to ForceNew in ResourceDiff.

This should help mitigate any confusion that may come up when using
CustomizeDiff, especially in the ForceNew scenario when the second run
happens with no state.
2017-11-01 15:42:24 -07:00
Chris Marchesi f5bdbc0911 helper/schema: Simplify setDiff, remove exported SetDiff mentions
setDiff does not make use of its new parameter anymore, so it has been
removed. Also, as there is no more SetDiff (exported) function, mentions
of that have been removed from comments too.
2017-11-01 14:25:32 -07:00
Chris Marchesi 3ac0cdf0c0 helper/schema: Better ResourceDiff schema key validation
This fixes nil pointer issues that could come up if an invalid key was
referenced (ie: not one in the schema). Also ships a helper validation
function to streamline things.
2017-11-01 14:25:32 -07:00
Chris Marchesi 529d7e6dae helper/schema: Review -> CustomizeDiff
Restoring the naming of this field in the resource back to
CustomizeDiff, as this is generally more descriptive of the process
that's happening, despite the lengthy name.
2017-11-01 14:25:32 -07:00
Chris Marchesi 09e2109ff8 helper/schema: Resouce.Diff no longer ResourceProvider API compatible
The old comments said that this interface was API compatible with
terraform.ResourceProvider's Diff method - with the addition of passing
down meta to it, this is no longer the case.

Not too sure if this is really a big deal - schema.Resource never fully
implemented terraform.ResourceProvider, from what I can see, and the
path from Provdier.Diff to Resource.Diff is still pretty clear. Just
wanted to remove an outdated comment.
2017-11-01 14:25:32 -07:00
Chris Marchesi 9625830980 helper/schema: Move computedKeys init to init function
This simplifies the new value initialization and future-proofs it
against more complex initialization functionality.
2017-11-01 14:25:32 -07:00
Chris Marchesi f0aafe4d67 helper/schema: Restore new value for complex set test 2017-11-01 14:25:32 -07:00
Chris Marchesi 3444549d60 helper/schema: Move computed key reader logic to childAddrOf
This will make this check safer and will remove the risk of it passing
on keys with a similar prefix.
2017-11-01 14:25:32 -07:00
Chris Marchesi 36aa63b338 helper/schema: Guard against out of range on childAddrOf
This could panic if we sent a parent that was actually longer than the
child (should almost never come up, but the guard will make it safe
anyway).

Also fixed a slice style lint warning in UpdatedKeys.
2017-11-01 14:25:32 -07:00
Chris Marchesi 931b0e1441 helper/schema: Remove exported SetDiff method
The consensus is that it's a generally better idea to move setting the
functionality of old values completely to the refresh/read process,
hence it's moot to have this function around anymore. This also means we
don't need the old value reader/writer anymore, which simplifies things.
2017-11-01 14:25:32 -07:00
Chris Marchesi ad98471559 helper/schema: Correct some comments
Removed some outdated comments for SetNew, and normalized the computed
key note for SetNew, SetNewComputed, and SetDiff.
2017-11-01 14:25:32 -07:00
Chris Marchesi 8a7c9a6f02 helper/schema: frist -> first 2017-11-01 14:25:32 -07:00
Chris Marchesi 7d5f9ed6b1 helper/schema: NewComputed values should be nil
When working on this initially, I think I thought that since NewComputed
values in the diff were empty strings, that it was using the zero value.
After review, it doesn't seem like this is the case - so I have adjusted
NewComputed to pass nil values. There is also a guard now that keeps the
new value writer from accepting computed fields with non-nil values.
2017-11-01 14:25:32 -07:00
Chris Marchesi 6f422d8c44 helper/schema: Remove unused log line
Meant to remove this before finalizing the PR. :P
2017-11-01 14:25:32 -07:00
Chris Marchesi c6647a3bb7 helper/schema: CustomizeDiff -> Review
To keep with the current convention of most other schema.Resource
functional fields being fairly short, CustomizeDiff has been changed to
"Review". It would be "Diff", however it is already used by existing
functions in schema.Provider and schema.Resource.
2017-11-01 14:25:32 -07:00
Chris Marchesi f7e42728b6 helper/schema: Remove unused Destroy check for CustomizeDiff
Both Destroy and DestroyDeposed are not propagated down the diff stack,
meaning that there is no way we can tell at this point if an instance is
being destroyed or deposed, so this check would never be used.

In this regard, Destroy never runs a diff down the stack at all, and a
deposition check is not run until *after* the provider's diff function
is called. To answer this question and close it off, we could either
determine if a resource is deposed earlier, and propagate that down, or
treat deposed resources like full destroy nodes, and not diff them at
all (but rather making a diff with the only thing in it being
DestroyDeposed flagged).
2017-11-01 14:25:32 -07:00
Chris Marchesi fa1fc2ca8e helper/schema: CustomizeDiff invocation test
Just one more test to check to make sure that CustomizeDiff is called
on resource level.
2017-11-01 14:25:32 -07:00
Chris Marchesi ee769188d7 helper/schema: More CustomizeDiff test cases
Added a few more test cases for CustomizeDiff, caught another error in
the process. I think this is ready for review now, and possibly some
real-world testing of the waters by way of porting some resources that
would benefit from the feature.
2017-11-01 14:25:32 -07:00
Chris Marchesi 8af9610b87 helper/schema: Hook CustomizeDiffFunc into diff logic
It's alive! CustomizeDiff logic now has been inserted into the diff
process. The test_resource_with_custom_diff resource provides some basic
testing and a reference implementation.

There should now be plenty of test coverage for this feature via the
tests added for ResourceDiff, and the basic test added to the
schemaMap.Diff test, and the test resource, but more can be added to
test any specific case that comes up otherwise.
2017-11-01 14:25:32 -07:00
Chris Marchesi 8126ee8401 helper/schema: Fix supplied config to tests 2017-11-01 14:25:32 -07:00
Chris Marchesi 196d7e63fe helper/schema: Drop ClearAll from ResourceDiff
As the interface has been specced out for ResourceDiff, the only diff
operation that can function on a non-computed key is ForceNew, and that
is only if a diff exists for that key. As such, allowing the user to
clear keys that cannot be operated on afterwards does not really make
much sense.

Will re-add this function if it is determined it's needed after all on
review.
2017-11-01 14:25:32 -07:00
Chris Marchesi 22220fd0f7 helper/schema: Final set of ResourceDiff tests, bug fixes
Final set of coverage for ResourceDiff and bug fixes to correct issues
the test cases brought up.

Will be dropping ClearAll in next commit, as with how this interface is
intended to be used, it does not really make much sense.
2017-11-01 14:25:32 -07:00
Chris Marchesi a5fc664ea6 helper/schema: Add schemaMap.DeepCopy
This provides a deep copy of a schemaMap, which will be needed for the
diff customization process as ResourceDiff will be able to flag fields
as ForceNew - we don't want to affect the source schema.
2017-11-01 14:25:32 -07:00
Chris Marchesi 6a4f7b0dce helper/schema: Don't ignore diffs for certain NewComputed keys
In diffString, removed values are marked if the old value is non-nil and
the new value is nil, regardless of if the new value is marked as a
computed value. With the new diff override behaviour, this can lead to
issues where a nil attribute may get inserted into the diff if the new
value has been marked as computed (as the value will be marked as
missing, but computed).

This propagates into finalizeDiff in some respects because even if
NewComputed is manually set in the diff that gets passed in, it is
ignored, and the schema becomes the only source of truth. Since our new
diff behaviour is mainly designed to be supported on computed keys only,
it stands to reason that we there will be a time where we want to set a
diff as NewComputed on a key, and see that change in the diff.

These two small changes makes that happen. No regressions in tests have
been observed via this change, so it seems safe and non-invasive.
2017-11-01 14:25:32 -07:00
Chris Marchesi 64cc4084d2 helper/schema: ResourceDiff tests, bug fixes
The beginning of tests for SetNew. Noticed that removed values are not
logged for computed keys in a diff - not too sure if that is going to be
a problem (possibly not as GetChange in ResourceData should still
reference state for the old value). I came on this most notably in the
"TestSetNew/complex-ish_set_diff" test, for reference.

More tests pending for other functions for coverage of other functions
and test cases as defined in #8769.
2017-11-01 14:25:32 -07:00