Commit Graph

67 Commits

Author SHA1 Message Date
James Bardin 2448d1d38b move timeout handling to grpc_provider
simpleDiff is only called from the grpc_provider, and we always need to
make sure the timeouts are encoded in the private data.
2019-06-19 22:48:15 -04:00
Martin Atkins bd1a215580 helper/resource: Ignore Removed attributes for ImportStateVerify
Due to the lossiness of our legacy models for diff and state, shimming a
diff and then creating a state from it produces a different result than
shimming a state directly. That means that ImportStateVerify no longer
works as expected if there are any Computed attributes in the schema where
d.Set isn't called during Read.

Fixing that for every case would require some risky changes to the shim
behavior, so we're instead going to ask provider developers to address it
by adding `d.Set` calls where needed, since that is the contract for
"Computed" anyway -- a default value should be produced during Create, and
thus by extension during Import.

However, since a common situation where this occurs is attributes marked
as "Removed", since all of the code that deals with them has generally
been deleted, we'll avoid problems in that case here by treating Removed
attributes as ignored for the purposes of ImportStateVerify.

This required exporting some functionality that was formerly unexported
in helper/schema, but it's a relatively harmless schema introspection
function so shouldn't be a big deal to export it.
2019-04-16 11:14:49 -07:00
James Bardin 1a9c06d0f5 Revert "helper/schema: Implementation of the AsSingle mechanism"
This reverts commit 1987a92386.
2019-04-08 16:45:35 -04:00
Martin Atkins 1987a92386 helper/schema: Implementation of the AsSingle mechanism
The previous commit added a new flag to schema.Schema which is documented
to make a list with MaxItems: 1 be presented to Terraform Core as a single
value instead, giving a way to switch to non-list nested resources without
it being a breaking change for Terraform v0.11 users as long as it's done
prior to a provider's first v0.12-compatible release.

This is the implementation of that mechanism. It's intentionally
implemented as a suite of extra fixups rather than direct modifications to
existing shim code because we want to ensure that this has no effect
whatsoever on the result of a resource type that _isn't_ using AsSingle.

Although there is some small unit test coverage of the fixup steps here,
the primary testing for this is in the test provider since the integration
of all of these fixup steps in the correct order is the more important
result than any of the intermediate fixup steps.
2019-03-14 15:36:15 -07:00
James Bardin df04e2e7a6 move InstanceState shim into schema.Resource
This was the resource can rebuild the flatmapped state using the
schema and ResourceData, providing us the the correct set key values.
2018-11-16 09:59:03 -05:00
James Bardin d50a152f8b check for a nil diff in simpleDiff 2018-10-19 14:25:20 -04:00
James Bardin ff4e81cc2b add old values when computing the new InstanceDiff
This was previously done in the RequiresNew code, which is skipped in
new style provider.
2018-10-18 20:05:33 -04:00
James Bardin 46b4c27dbe create a SimpleDiff for the new provider shims
Terraform now handles any actual "diffing" of resource, and the existing
Diff functions are only used to shim the schema.Provider to the new
methods. Since terraform is handling what used to be the Diff, the
provider now should not modify the diff based on RequiresNew due to it
interfering with the ignore_changes handling.
2018-10-16 19:14:11 -07:00
James Bardin b88410984b legacy provider needs to handle StateUpgraders
In order to not require state migrations to be supported in both
MigrateState and StateUpgraders, the legacy provider codepath needs to
handle the StateUpgraders transparently during Refresh.
2018-10-16 18:56:50 -07:00
James Bardin 7d24936507 updates to teh StateUpgraders
Fix documentation.
Require StateUpgraders to be add in order, and test in validation.
2018-10-16 18:53:51 -07:00
James Bardin 0c33b26e04 StateUpgrade redux
It turns out that state upgrades need to be handled differently since
providers are going to be backwards compatible. This means that new
state upgrades may still be stored in the flatmap format when used wih
terraform 0.11. Because we can't account for the specific version which
could produce a legacy state, all future state upgrades need to record
the schema types for decoding.

Rather than defining a single Upgrade function for states, we now have a
list of functions, each of which handle upgrading a specific version to
the next. In practice this isn't much different from the way many
resources implement upgrades themselves, with a separate function for
each version dispatched from the MigrateState function. The only added
burden is the recording of the schema type, and we intend to supply
tools and helper function to prevent the need to copy the entire
existing schema in all cases.
2018-10-16 18:53:51 -07:00
James Bardin 9eef5e3f91 implement UpgradeState for schema.Resource
This is the provider-side UpgradeState implementation for a particular
resource. This new function will be called to upgrade a saved state with
an old schema version to the current schema.

UpgradeState also requires a record of the last schema and version that
could have been stored as a flatmapped state. If the stored state is in
the legacy flatmap format, this will allow the provider to properly
decode the flatmapped state into the expected structure for the new json
encoded state. If the stored state's version is below that of the
LegacySchema.Version value, it will first be processed by the legacy
MigrateState function.
2018-10-16 18:53:51 -07:00
Paddy Carver b7d19e40f8 Deprecated -> DeprecationMessage.
At the Enablement team's request, change from using `Deprecated` to
`DeprecationMessage`, as it's a string value, not a boolean.
2018-06-20 11:21:46 -07:00
Paddy Carver 0dd2408d65 Export resource deprecation.
We already had the functionality to make resources deprecated, which was
used when migrating resources to data sources, but the functionality was
unexported, so only the schema package could do it. Now it's exported,
meaning providers can mark entire resources as deprecated. I also added
a test in hopefully-the-right place?
2018-06-19 12:02:55 -07:00
James Bardin 798df9dafa make sure ResourceData timeouts are always set
Return the global default timeout if the ResourceData timeouts are nil.

Set the timeouts from the Resource when calling Resource.Data, so that
the config values are always available.
2018-03-22 15:10:43 -04:00
James Bardin 4b49a323c3 go fmt
slight change to go fmt coming in 0.10
2017-12-26 13:26:38 -05:00
Chris Marchesi 0c0ae3ca7c helper/schema: CustomizeDiff allowed on writable resources only
This keeps CustomizeDiff from being defined on data sources, where it
would be useless. We just catch this in InternalValidate like the rest
of the CRUD functions that are not used in data sources.
2017-11-01 15:42:24 -07:00
Chris Marchesi 2c541e8c97 helper/schema: Add behaviour detail to various custom diff comments
Added some more detailed comments to CustomizeDiff's comments. The new
comments detail how CustomizeDiff will be called in the event of
different scenarios like creating a new resource, diffing an existing
one, diffing an existing resource that has a change that requires a new
resource, and destroy/tainted resources.

Also added similar detail to ForceNew in ResourceDiff.

This should help mitigate any confusion that may come up when using
CustomizeDiff, especially in the ForceNew scenario when the second run
happens with no state.
2017-11-01 15:42:24 -07:00
Chris Marchesi 529d7e6dae helper/schema: Review -> CustomizeDiff
Restoring the naming of this field in the resource back to
CustomizeDiff, as this is generally more descriptive of the process
that's happening, despite the lengthy name.
2017-11-01 14:25:32 -07:00
Chris Marchesi 09e2109ff8 helper/schema: Resouce.Diff no longer ResourceProvider API compatible
The old comments said that this interface was API compatible with
terraform.ResourceProvider's Diff method - with the addition of passing
down meta to it, this is no longer the case.

Not too sure if this is really a big deal - schema.Resource never fully
implemented terraform.ResourceProvider, from what I can see, and the
path from Provdier.Diff to Resource.Diff is still pretty clear. Just
wanted to remove an outdated comment.
2017-11-01 14:25:32 -07:00
Chris Marchesi c6647a3bb7 helper/schema: CustomizeDiff -> Review
To keep with the current convention of most other schema.Resource
functional fields being fairly short, CustomizeDiff has been changed to
"Review". It would be "Diff", however it is already used by existing
functions in schema.Provider and schema.Resource.
2017-11-01 14:25:32 -07:00
Chris Marchesi 8af9610b87 helper/schema: Hook CustomizeDiffFunc into diff logic
It's alive! CustomizeDiff logic now has been inserted into the diff
process. The test_resource_with_custom_diff resource provides some basic
testing and a reference implementation.

There should now be plenty of test coverage for this feature via the
tests added for ResourceDiff, and the basic test added to the
schemaMap.Diff test, and the test resource, but more can be added to
test any specific case that comes up otherwise.
2017-11-01 14:25:32 -07:00
Chris Marchesi b99c615ee6 helper/schema: New ResourceDiff object
This adds a new object, ResourceDiff, to the schema package. This
object, in conjunction with a function defined in CustomizeDiff in the
resource schema, allows for the in-flight customization of a Terraform
diff. This helps support use cases such as when there are necessary
changes to a resource that cannot be detected in config, such as via
computed fields (most of the utility in this object works on computed
fields only). It also allows for a wholesale wipe of the diff to allow
for diff logic completely offloaded to an external API, if it is a
better use case for a specific provider.

As part of this work, many internal diff functions have been moved to
use a special resourceDiffer interface, to allow for shared
functionality between ResourceDiff and ResourceData. This may be
extended to the DiffSuppressFunc as well which would restrict use of
ResourceData in DiffSuppressFunc to generally read-only fields.

This work is not yet in its final state - CustomizeDiff is not yet
implemented in the general diff workflow, new functions may be added
(notably Clear() for a single key), and functionality may be altered.
Tests will follow as well.
2017-11-01 14:25:32 -07:00
Radek Simko 7fceccfbf7
helper/schema: Loosen validation for 'id' field 2017-10-26 09:37:38 +01:00
Radek Simko 07cbd54fbc Actively disallow reserved field names in schema (#15522) 2017-07-10 21:51:55 -07:00
joel.ringuette ffddf96603 Fix issue with reading timeouts on Delete
Original fix by @jringuette but I couldn't get his patch to apply after
the repo split :/
2017-06-23 09:15:29 -05:00
Clint 2fe5976aec helper/schema: Add configurable Timeouts (#12311)
* helper/schema: Add custom Timeout block for resources

* refactor DefaultTimeout to suuport multiple types. Load meta in Refresh from Instance State

* update vpc but it probably wont last anyway

* refactor test into table test for more cases

* rename constant keys

* refactor configdecode

* remove VPC demo

* remove comments

* remove more comments

* refactor some

* rename timeKeys to timeoutKeys

* remove note

* documentation/resources: Document the Timeout block

* document timeouts

* have a test case that covers 'hours'

* restore a System default timeout of 20 minutes, instead of 0

* restore system default timeout of 20 minutes, refactor tests, add test method to handle system default

* rename timeout key constants

* test applying timeout to state

* refactor test

* Add resource Diff test

* clarify docs

* update to use constants
2017-03-02 11:07:49 -06:00
Mitchell Hashimoto 3342aa580c
terraform: InstanceState.Meta is value type interface{}
This changes the type of values in Meta for InstanceState to
`interface{}`. They were `string` before.

This will allow richer structures to be persisted to this without
flatmapping them (down with flatmap!). The documentation clearly states
that only primitives/collections are allowed here.

The only thing using this was helper/schema for schema versioning.
Appropriate type checking was added to make this change safe.

The timeout work @catsby is doing will use this for a richer structure.
2017-02-23 10:44:05 -08:00
Mitchell Hashimoto f7a234bc71
helper/schema: validate Read, Delete are set 2016-10-30 15:04:32 -07:00
James Nugent 43dd13cd36 helper/schema: Introduce Noop and RemoveFromState
This commit implements reusable functions for when resources have no
need to implement a particular operation:

- Noop - does nothing and returns no error.
- RemoveFromState - sets the resource ID to empty string (removing it
  from state) and returns no error.
2016-10-25 11:47:03 -05:00
Martin Atkins 6cd22a4c9a helper/schema: emit warning when using data source resource shim
For backward compatibility we will continue to support using the data
sources that were formerly logical resources as resources for the moment,
but we want to warn the user about it since this support is likely to
be removed in future.

This is done by adding a new "deprecation message" feature to
schema.Resource, but for the moment this is done as an internal feature
(not usable directly by plugins) so that we can collect additional
use-cases and design a more general interface before creating a
compatibility constraint.
2016-05-14 08:26:36 -07:00
Martin Atkins 6a468dcd83 helper/schema: Resource can be writable or not
In the "schema" layer a Resource is just any "thing" that has a schema
and supports some or all of the CRUD operations. Data sources introduce
a new use of Resource to represent read-only resources, which require
some different InternalValidate logic.
2016-05-14 08:26:36 -07:00
Martin Atkins 0e0e3d73af core: New ResourceProvider methods for data resources
This is a breaking change to the ResourceProvider interface that adds the
new operations relating to data sources.

DataSources, ValidateDataSource, ReadDataDiff and ReadDataApply are the
data source equivalents of Resources, Validate, Diff and Apply (respectively)
for managed resources.

The diff/apply model seems at first glance a rather strange workflow for
read-only resources, but implementing data resources in this way allows them
to fit cleanly into the standard plan/apply lifecycle in cases where the
configuration contains computed arguments and thus the read must be deferred
until apply time.

Along with breaking the interface, we also fix up the plugin client/server
and helper/schema implementations of it, which are all of the callers
used when provider plugins use helper/schema. This would be a breaking
change for any provider plugin that directly implements the provider
interface, but no known plugins do this and it is not recommended.

At the helper/schema layer the implementer sees ReadDataApply as a "Read",
as opposed to "Create" or "Update" as in the managed resource Apply
implementation. The planning mechanics are handled entirely within
helper/schema, so that complexity is hidden from the provider implementation
itself.
2016-05-14 08:26:36 -07:00
Mitchell Hashimoto b728e55861
helper/schema: Resource.Data should set latest schema version 2016-05-11 13:02:36 -07:00
Mitchell Hashimoto 19609bde0e
helper/schema: can specify Importer on Resource, InternalValidate 2016-05-11 13:02:30 -07:00
Mitchell Hashimoto b8121ea63e
helper/schema: Resource.Data to return a ResourceData for a Resource 2016-05-11 13:02:30 -07:00
Radek Simko 8bdd92187c Merge pull request #4446 from TimeIncOSS/f-schema-new-resource
helper/schema: Allow identification of a new resource in update func
2016-02-29 20:07:00 +00:00
Paul Hinze c70eab6500 aws: undeprecate min_elb_capacity; restore min capacity waiting
It was a mistake to switched fully to `==` when activating waiting for
capacity on updates in #3947. Users that didn't set `min_elb_capacity ==
desired_capacity` and instead treated it as an actual "minimum" would
see timeouts for every create, since their target numbers would never be
reached exactly.

Here, we fix that regression by restoring the minimum waiting behavior
during creates.

In order to preserve all the stated behavior, I had to split out
different criteria for create and update, criteria which are now
exhaustively unit tested.

The set of fields that affect capacity waiting behavior has become a bit
of a mess. Next major release I'd like to rework all of these into a
more consistently named block of config. For now, just getting the
behavior correct and documented.

(Also removes all the fixed names from the ASG tests as I was hitting
collision issues running them over here.)

Fixes #4792
2016-01-27 13:30:44 -06:00
Radek Simko 4c6ceef9b8 helper/schema: Allow identification of a new resource in update func 2015-12-27 14:01:03 +01:00
Panagiotis Moustafellos e4845f75cc removed extra parentheses 2015-10-08 15:48:04 +03:00
Radek Simko 641b701830 schema: Make validation more strict 2015-10-03 14:29:19 -07:00
Paul Hinze 7eb72e7a12 helper/schema: record schema version when destroy fails
This was just a missed exit from the resource.Apply function -
subsequent refreshes would add the SchemaVersion back into the state,
but having the state recorded once without the meta information can
cause problems with Atlas's remote state checksumming.
2015-08-03 15:53:15 -05:00
Mitchell Hashimoto dd24ed4b76 helper/schema: blank ID refresh doesn't exist [GH-1905] 2015-05-13 20:15:13 -07:00
Paul Hinze 1e3d1b07e6 helper/schema: validate ConflictsWith against top-level
The runtime impl of ConfictsWith uses Resource.Get(), which makes it
work with any other attribute of the resource - the InternalValidate was
only checking against the local schemaMap though, preventing subResource
from using ConflictsWith properly.

It's a lot of wiring and it's a bit ugly, but it's not runtime code, so
I'm a bit less concerned about that aspect.

This should take care of the problem mentioned in #1909
2015-05-12 09:45:15 -05:00
Paul Hinze 66dbf91ffd helper/schema: ensure ForceNew set when Update is not
If a given resource does not define an `Update` function, then all of
its attributes must be specified as `ForceNew`, lest Applys fail with
"doesn't support update" like #1367.

This is something we can detect automatically, so this adds a check for
it when we validate provider implementations.
2015-04-03 09:57:30 -05:00
Paul Hinze 3ba8ed536b helper/schema: record schema version on apply
We were previously only recording the schema version on refresh. This
caused the state to be incorrectly written after a `terraform apply`
causing subsequent commands to run the state through an unnecessary
migration.
2015-03-18 19:08:48 -05:00
Paul Hinze 3d4b55e557 helper/schema: schema versioning & migration
Providers get a per-resource SchemaVersion integer that they can bump
when a resource's schema changes format. Each InstanceState with an
older recorded SchemaVersion than the cureent one is yielded to a
`MigrateSchema` function to be transformed such that it can be addressed
by the current version of the resource's Schema.
2015-03-06 16:26:11 -06:00
Mitchell Hashimoto 48b9614556 helper/schema: Exists API 2015-01-10 15:39:29 -08:00
Mitchell Hashimoto d0ce67a5b7 helper/schema: on destroy/create, reset state to be empty [GH-464] 2014-10-21 00:28:53 -07:00
Mitchell Hashimoto 9b2b3a963f ResourceDiff => InstanceDiff 2014-09-17 16:33:24 -07:00