Commit Graph

523 Commits

Author SHA1 Message Date
James Bardin d50a152f8b check for a nil diff in simpleDiff 2018-10-19 14:25:20 -04:00
James Bardin ff4e81cc2b add old values when computing the new InstanceDiff
This was previously done in the RequiresNew code, which is skipped in
new style provider.
2018-10-18 20:05:33 -04:00
James Bardin 38163f2b37 use SimpleDiff and set "id" as RequiresReplace
Use the new SimpleDiff method of the provider so that the diff isn't
altered by ForceNew attributes.

Always set an "id" as RequiresReplace so core knows an instance will be
replaced, even if all ForceNew attributes are filtered out due to
ignore_changes.
2018-10-16 19:14:54 -07:00
James Bardin 46b4c27dbe create a SimpleDiff for the new provider shims
Terraform now handles any actual "diffing" of resource, and the existing
Diff functions are only used to shim the schema.Provider to the new
methods. Since terraform is handling what used to be the Diff, the
provider now should not modify the diff based on RequiresNew due to it
interfering with the ignore_changes handling.
2018-10-16 19:14:11 -07:00
James Bardin 0d4d572c39 start work on helper/resource test fixtures
The helper resource tests won't pass for now, as they use a
terraform.MockProvider which can't be used in the schema.Provider shims.
2018-10-16 19:14:11 -07:00
Martin Atkins 33151f5011 core: Move StateValueFromInstanceState shim from helper/schema
This one doesn't depend on any helper/schema specific bits and it'll also
be useful for the shims in our mock provider in core.
2018-10-16 19:14:11 -07:00
Martin Atkins 76d11f44cc core: Move some of the helper/schema shims so provider mock can use them
The old names are now wrappers around these new functions.
2018-10-16 19:14:11 -07:00
James Bardin a87470cc15 resource ids must always have a value
The "id" field is assumed to always exist, and must have a valid value.
Set "id" to unknown when planning new resource changes to indicate that
it will be computed.
2018-10-16 19:14:11 -07:00
James Bardin b88410984b legacy provider needs to handle StateUpgraders
In order to not require state migrations to be supported in both
MigrateState and StateUpgraders, the legacy provider codepath needs to
handle the StateUpgraders transparently during Refresh.
2018-10-16 18:56:50 -07:00
James Bardin dcaf5aa262 add some of the shims used for the grpc provider
This adds some of the required shim functions to the schema package.
While this further bloats the already huge package, adding the helpers
here was significantly less disruptive than refactoring types into
separate packages to prevent import cycles.

The majority of tests here are directly adapted from existing schema
tests to provide as many known good values to the shims as possible.
2018-10-16 18:56:50 -07:00
James Bardin 7d24936507 updates to teh StateUpgraders
Fix documentation.
Require StateUpgraders to be add in order, and test in validation.
2018-10-16 18:53:51 -07:00
James Bardin 0c33b26e04 StateUpgrade redux
It turns out that state upgrades need to be handled differently since
providers are going to be backwards compatible. This means that new
state upgrades may still be stored in the flatmap format when used wih
terraform 0.11. Because we can't account for the specific version which
could produce a legacy state, all future state upgrades need to record
the schema types for decoding.

Rather than defining a single Upgrade function for states, we now have a
list of functions, each of which handle upgrading a specific version to
the next. In practice this isn't much different from the way many
resources implement upgrades themselves, with a separate function for
each version dispatched from the MigrateState function. The only added
burden is the recording of the schema type, and we intend to supply
tools and helper function to prevent the need to copy the entire
existing schema in all cases.
2018-10-16 18:53:51 -07:00
James Bardin 9eef5e3f91 implement UpgradeState for schema.Resource
This is the provider-side UpgradeState implementation for a particular
resource. This new function will be called to upgrade a saved state with
an old schema version to the current schema.

UpgradeState also requires a record of the last schema and version that
could have been stored as a flatmapped state. If the stored state is in
the legacy flatmap format, this will allow the provider to properly
decode the flatmapped state into the expected structure for the new json
encoded state. If the stored state's version is below that of the
LegacySchema.Version value, it will first be processed by the legacy
MigrateState function.
2018-10-16 18:53:51 -07:00
James Bardin bcc8be7400 add schema.InternalMap
This exposes the internal schemaMap for use by the new provider shims.
2018-10-16 18:53:51 -07:00
James Bardin 0120d53baf only add "id" to top-level resources
Make sure we only add "id" to the top-level resource, since Resource is
also used for nested blocks.
2018-10-16 18:53:51 -07:00
James Bardin 15ccf2dda5 use a custom comparer for cty.Type
Make sure we also compare cty.Types in `cmp.Equal`, even though they
contain unexported fields.
2018-10-16 18:50:57 -07:00
James Bardin 3112b707be SetId should set the attribute as well
The update protocol shims will also check for this this, but eventually
"id" will only be a normal attribute, and we shouldn't have to special
case this.
2018-10-16 18:50:57 -07:00
James Bardin befa81c74f add implicit "id" to resource attribute schemas
When converting a legacy schemaMap to a configschema, we need to add
"id" as a required attribute to top-level resources if it's not
declared.

The "id" field will be required to interoperate with the legacy helper
schema, since the presence of an id was used to indicate the existence
of a resource.
2018-10-16 18:50:57 -07:00
Martin Atkins 479c6b2466 move "configschema" from "config" to "configs"
The "config" package is no longer used and will be removed as part
of the 0.12 release cleanup. Since configschema is part of the
"new world" of configuration modelling, it makes more sense for
it to live as a subdirectory of the newer "configs" package.
2018-10-16 18:50:29 -07:00
Martin Atkins 3c10a3b213 helper/schema: Tolerate incorrectly-specified collection elems
We historically tolerated this, so we need to tolerate it here too in
order to work correctly with existing provider code.
2018-10-16 18:49:20 -07:00
Martin Atkins c937c06a03 terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.

The three main goals here are:
- Use the configuration models from the "configs" package instead of the
  older models in the "config" package, which is now deprecated and
  preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
  new "lang" package, instead of the Interpolator type and related
  functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
  rather than hand-constructed strings. This is not critical to support
  the above, but was a big help during the implementation of these other
  points since it made it much more explicit what kind of address is
  expected in each context.

Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.

I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-10-16 18:46:46 -07:00
Martin Atkins 5782357c28 backend: Update interface and implementations for new config loader
The new config loader requires some steps to happen in a different
order, particularly in regard to knowing the schema in order to
decode the configuration.

Here we lean directly on the configschema package, rather than
on helper/schema.Backend as before, because it's generally
sufficient for our needs here and this prepares us for the
helper/schema package later moving out into its own repository
to seed a "plugin SDK".
2018-10-16 18:39:12 -07:00
Martin Atkins 1fb714ea3b helper/schema: CoreConfigSchema method for Backend
This is just like the method of the same name on Resource, adapting the
helper/schema model of schema into Terraform Core's idea of schema.
2018-10-16 18:24:47 -07:00
Martin Atkins d6c6f8852c configschema: include description in schema
We will need access to this information in order to render interactive
input prompts, and it will also be useful in returning schema information
to external tools such as text editors that have autocomplete-like
functionality.
2018-10-16 18:24:47 -07:00
Paddy 35d82b0555
Merge pull request #18795 from hashicorp/paddy_diff_nested_checkKey
Allow for nested fields with checkKey on ResourceDiffs.
2018-09-26 14:21:28 -07:00
Paddy Carver 2eadc8f625 Don't allow sub-blocks for SetNew.
You can't set individual items in lists, you can only set the list as a
whole, so we shouldn't allow sub-blocks to SetNew or SetNewComputed.
2018-09-26 12:38:38 -07:00
Paddy Carver 393c32624a Add tests to ensure clearing sub-blocks works. 2018-09-20 10:52:36 -07:00
Sean Chittenden d749420a25
Fix drift caused from gofmt when running make dev and go 1.11.
A fresh checkout of `origin/master` does not build atm using the `dev`
target because `master` has not been formatted using `gofmt` from Go
1.11 (tis has been the case for a while if you've been running devel).

None of the drift in question is especially new but now that Go 1.11
has been released and gofmt's formatting guidelines have been updated,
it would be *really* nice if the code in `master` reflected the current
tooling in order to avoid having to fight this drift locally.

* 8mo: https://github.com/hashicorp/terraform/blame/master/backend/remote-state/s3/backend_test.go#L260-L261
* 6mo: https://github.com/hashicorp/terraform/blame/master/builtin/provisioners/chef/linux_provisioner_test.go#L124
* 1yr: 7cfeffe36b/command/init.go (L75-L76)
* 12d: 7cfeffe36b/command/meta_backend_test.go (L1437)
* 2yr: 7cfeffe36b/helper/schema/resource_timeout_test.go (L26)
* 4yr: 7cfeffe36b/helper/schema/schema_test.go (L2059)
* 1yr: 7cfeffe36b/plugin/discovery/get_test.go (L151)
2018-09-09 10:18:08 -07:00
Paddy Carver 7ee4339555 Allow for nested fields with checkKey on ResourceDiffs.
When checking if a ResourceDiff key is valid, allow for keys that exist
on sub-blocks. This allows things like diff.Clear to be called on
sub-block fields, instead of just on top-level fields.
2018-09-05 17:29:03 -07:00
Sander van Harmelen 3e935c846f terraform/terraform_remote_state: accept complex configs
The `remote` backend config contains an attribute that is defined as a `*schema.Set`, but currently only `string` values are accepted as the `config` attribute is defined as a `schema.TypeMap`.

Additionally the `b.Validate()` method wasn’t called to prevent a possible panic in case of unexpected configurations being passed to `b.Configure()`.

This commit is a bit of a hack to be able to support this in the 0.11 series. The 0.12 series will have proper support, so when merging 0.12 this should be reverted again.
2018-08-29 20:21:47 +02:00
Paul Tyng 74eae6825a
Fix typo in comment 2018-08-16 16:29:29 -04:00
Paddy Carver b7d19e40f8 Deprecated -> DeprecationMessage.
At the Enablement team's request, change from using `Deprecated` to
`DeprecationMessage`, as it's a string value, not a boolean.
2018-06-20 11:21:46 -07:00
Paddy Carver 0dd2408d65 Export resource deprecation.
We already had the functionality to make resources deprecated, which was
used when migrating resources to data sources, but the functionality was
unexported, so only the schema package could do it. Now it's exported,
meaning providers can mark entire resources as deprecated. I also added
a test in hopefully-the-right place?
2018-06-19 12:02:55 -07:00
kt a182a53108
Add a HashInt helper/schema function 2018-04-27 00:04:41 -07:00
Chris Marchesi d7048cb640
helper/schema: ResourceDiff ForceNew attribute correctness
A couple of bugs have been discovered in ResourceDiff.ForceNew:

* NewRemoved is not preserved when a diff for a key is already present.
This is because the second diff that happens after customization
performs a second getChange on not just state and config, but also on
the pre-existing diff. This results in Exists == true, meaning nil is
never returned as a new value.
* ForceNew was doing the work of adding the key to the list of changed
keys by doing a full SetNew on the existing value. This has a side
effect of fetching zero values from what were otherwise undefined values
and creating diffs for these values where there should not have been
(example: "" => "0").

This update fixes these scenarios by:

* Adding a new private function to check the existing diff for
NewRemoved keys. This is included in the check on new values in
diffChange.
* Keys that have been flagged as ForceNew (or parent keys of lists and
sets that have been flagged as ForceNew) are now maintained in a
separate map. UpdatedKeys now returns the results of both of these maps,
but otherwise these keys are ignored by ResourceDiff.
* Pursuant the above, values are no longer pushed into the newDiff
writer by ForceNew. This prevents the zero value problem, and makes for
a cleaner implementation where the provider has to "manually" SetNew to
update the appropriate values in the writer. It also prevents
non-computed keys from winding up in the diff, which ResourceDiff
normally blocks by design.

There are also a couple of tests for cases that should never come up
right now involving Optional/Computed values and NewRemoved, for which
explanations are given in annotations of each test. These are here to
guard against future regressions.
2018-04-08 07:50:41 -07:00
James Bardin 61eae050ab
Merge pull request #17588 from creack/creack/fix-deepcopy-type
Fix issue with deepcopy returning wrong type causing panic
2018-04-05 09:24:08 -04:00
Chris Marchesi db5ed51703
Merge pull request #17667 from hashicorp/f-resource-diff-getokexists-computed
helper/schema: Add Computed to ResourceDiff, expose GetOkExists
2018-04-02 07:32:04 -07:00
Adam Lewandowski 56a330c533 Remove attribute values from ConflictsWith validation message 2018-03-30 07:53:59 -04:00
Chris Marchesi fdfe89ed1d
helper/schema: Computed -> NewValueKnown, add comments to GetOkExists
Added some more context to GetOkExists, moved Computed to NewValueKnown
to accommodate some changes that will be coming up in HCL2 that may make
"Computed" less intuitive of a function name, and updated the docs for
NewValueKnown as well.
2018-03-29 15:18:35 -07:00
Chris Marchesi 274b933077
helper/schema: Add Computed to ResourceDiff, expose GetOkExists
This adds a new method to ResourceDiff: Computed, which exposes the
computed read result field to ResourceDiff. In the context of
customizing the diff, this is important as interpolated and otherwise
computed values will show up in the diff as blank, with no way of
determining if the value is actually blank or if it's a computed value
not available at diff customization time. Currently assumptions need to
be made on this, but this does not help in validation scenarios where
one needs to differentiate between an actual blank value and a value
that will be available later.

This is exposed for the most part via NewComputed in the diff, but the
tests cover both the config reader as well (with no diff, even though
this should not come up in normal operation) and also the newDiff reader
when someone sets a new value using SetNew and SetNewComputed.

This commit also exposes GetOkExists. The tests were mostly pulled from
ResourceData but a few were added to ensure that config was being
properly covered as well, in addition to covering SetNew and
SetNewComputed.
2018-03-29 12:04:36 -07:00
James Bardin 798df9dafa make sure ResourceData timeouts are always set
Return the global default timeout if the ResourceData timeouts are nil.

Set the timeouts from the Resource when calling Resource.Data, so that
the config values are always available.
2018-03-22 15:10:43 -04:00
Guillaume J. Charmes a4f941368b
Fix issue with deepcopy returning wrong type causing panic
Signed-off-by: Guillaume J. Charmes <gcharmes@magicleap.com>
2018-03-14 19:34:23 -04:00
James McGill 035d56409f helper/schema: handle TypeMap elem consistently with other collection types
For historical reasons, the handling of element types for maps is inconsistent with other collection types.

Here we begin a multi-step process to make it consistent, starting by supporting both the "consistent" form of using a schema.Schema and an existing erroneous form of using a schema.Type directly. In subsequent commits we will phase out the erroneous form and require the schema.Schema approach, the same as we do for TypeList and TypeSet.
2018-03-14 14:50:41 -07:00
Radek Simko ccf8a31cbb
helper/schema: Allow ResourceDiff.ForceNew on nested fields (avoid crash) 2018-03-01 15:51:01 +00:00
Radek Simko 7af1c2b3a4
helper/schema: Prevent crash on removal of computed field in CustomizeDiff 2018-02-01 12:05:22 +00:00
Stefan Schmidt ee6c8a777a s/repalce/replace 2018-01-23 17:41:03 +01:00
James Bardin 4b49a323c3 go fmt
slight change to go fmt coming in 0.10
2017-12-26 13:26:38 -05:00
James Bardin 0df8da59f7 add FIXMEs
This new codepath with the getDiff "customzed" return value, along with
the associated test need to be removed as soon as we can support unset
fields from the config, so we don't continue to carry this broken
behavior forward any longer than needed.
2017-12-20 08:51:00 -05:00
Chris Marchesi cb5ce1d35e
helper/schema: Extend diffChange and bubble up customized values
This extends the internal diffChange method so that ResourceDiff's
implementation of it can report back whether or not the value came from
a customized diff.

This is an effort to work to preserve the pre-ResourceDiff behaviour
that ignores the diff for computed keys when the old value was populated
but the new value wasn't - this behaviour is actually being depended on
by users that are using it to exploit using zero values in modules. This
should allow both scenarios to co-exist by shifting the NewComputed
exemption over to exempting values that come from diff customization.
2017-12-19 16:06:57 -08:00
James Bardin 643ef4334f revert the change that broke the test case
This reverts one of the changes from 6a4f7b0, which broke empty strings
being seen as unset for computed values.

This breaks a number of other tests, and is only an intermediate change
for evaluating other solutions.
2017-12-19 16:14:07 -05:00