Commit Graph

7739 Commits

Author SHA1 Message Date
Alisdair McDiarmid 107dc0c34f Fix terraform_remote_state backend version check
The change recently introduced to ensure that remote backend users do
not accidentally upgrade their state file needs to be disabled for all
read-only uses, including the builtin terraform_remote_state data
source.
2020-12-08 14:18:04 -05:00
James Bardin df56a5f8ae remove dead code from tf provider 2020-12-02 13:07:10 -05:00
James Bardin b32362da9d internal/legacy/builtin/providers/test
Move the test provider into the interal/legacy directory, until we can
factor it out of the e2e test.
2020-12-02 12:16:35 -05:00
Radek Simko bc87be384b
builtin/provider/terraform: Add field descriptions 2020-11-06 08:51:07 +00:00
James Bardin c61a893590 unused tests
these are no longer relevant
2020-10-14 14:08:09 -04:00
Martin Atkins 6b5ca49578 helper: remove a bunch of unused packages, types and functions
With the SDK moving out into its own repository, a lot of the packages
under "helper/" are no longer needed. We still need to keep around just
enough legacy SDK to support the "test" provider and some little bits and
bobs in the backends and provisioners, but a lot of this is now just dead
code.

One of the test provider tests was depending on some validation functions
only because the schema in there was originally copied from a "real"
provider. The validation rules are not actually important for the test,
so I removed them. Otherwise, this removes only dead code.
2020-10-02 09:01:37 -07:00
Kristin Laemmert 479655ad47 refactor tests to use modern states.State in favor of terraform.State where possible 2020-09-30 16:07:54 -04:00
James Bardin a0fad2c6b4 always return original workspace, even when null
The returned value type needs to strictly match the proposed value type.
2020-06-24 13:41:51 -04:00
James Bardin fb06063b2b don't set remote workspace in state
The workspace attribute is not computed, and cannot be changed from the
configuration.
2020-06-18 16:34:57 -04:00
Paddy 5127f1ef8b
command: Unmanaged providers
This adds supports for "unmanaged" providers, or providers with process
lifecycles not controlled by Terraform. These providers are assumed to
be started before Terraform is launched, and are assumed to shut
themselves down after Terraform has finished running.

To do this, we must update the go-plugin dependency to v1.3.0, which
added support for the "test mode" plugin serving that powers all this.

As a side-effect of not needing to manage the process lifecycle anymore,
Terraform also no longer needs to worry about the provider's binary, as
it won't be used for anything anymore. Because of this, we can disable
the init behavior that concerns itself with downloading that provider's
binary, checking its version, and otherwise managing the binary.

This is all managed on a per-provider basis, so managed providers that
Terraform downloads, starts, and stops can be used in the same commands
as unmanaged providers. The TF_REATTACH_PROVIDERS environment variable
is added, and is a JSON encoding of the provider's address to the
information we need to connect to it.

This change enables two benefits: first, delve and other debuggers can
now be attached to provider server processes, and Terraform can connect.
This allows for attaching debuggers to provider processes, which before
was difficult to impossible. Second, it allows the SDK test framework to
host the provider in the same process as the test driver, while running
a production Terraform binary against the provider. This allows for Go's
built-in race detector and test coverage tooling to work as expected in
provider tests.

Unmanaged providers are expected to work in the exact same way as
managed providers, with one caveat: Terraform kills provider processes
and restarts them once per graph walk, meaning multiple times during
most Terraform CLI commands. As unmanaged providers can't be killed by
Terraform, and have no visibility into graph walks, unmanaged providers
are likely to have differences in how their global mutable state behaves
when compared to managed providers. Namely, unmanaged providers are
likely to retain global state when managed providers would have reset
it. Developers relying on global state should be aware of this.
2020-05-26 17:48:57 -07:00
Martin Atkins f897863288 providers/terraform: test that validation does not configure backend 2020-05-07 11:08:10 -07:00
Ben Drucker db1a623ed4 backend/terraform: additional test coverage for error cases in getBackend 2020-05-07 11:08:10 -07:00
Ben Drucker 81b8891b90 providers/terraform: don't call backend.Configure to validate terraform_remote_state
Validation is supposed to be a local-only operation, but Configure implementations
are allowed to make outgoing requests to remote APIs to validate settings.
2020-05-07 11:08:10 -07:00
Kristin Laemmert c8d64846ad
Mildwonkey/ps import (#24412)
* import: remove Config from ImportOpts

`Config` in ImportOpts was any provider configuration provided by the
user on the command line. This option has already been removed in favor
of only taking the provider from the configuration loaded in the current
context.

* terrafrom: add Config to ImportStateTransformer and refactor Transform
to get the resource provider FQN from the Config
2020-03-20 08:15:29 -04:00
Paddy e6592dc710
Add support for provider metadata to modules. (#22583)
Implement a new provider_meta block in the terraform block of modules, allowing provider-keyed metadata to be communicated from HCL to provider binaries.

Bundled in this change for minimal protocol version bumping is the addition of markdown support for attribute descriptions and the ability to indicate when an attribute is deprecated, so this information can be shown in the schema dump.

Co-authored-by: Paul Tyng <paul@paultyng.net>
2020-03-05 16:53:24 -08:00
Kristin Laemmert 9891d0354a
providers: use addrs.Provider as map keys for provider.Factory (#23548)
* terraform/context: use new addrs.Provider as map key in provider factories
* added NewLegacyProviderType and LegacyString funcs to make it explicit that these are temporary placeholders

This PR introduces a new concept, provider fully-qualified name (FQN), encapsulated by the `addrs.Provider` struct.
2019-12-04 11:30:20 -05:00
James Bardin a3c374b227 don't lose track of private data in the ACC tests
If the resource schema version is > 0, the metadata containing the
timeout values was overwritten during acceptance tests.
2019-09-18 13:59:12 -04:00
Pam Selle 2e5a8c0f6e Update when ignore_changes are evaluated, to impact customizediff 2019-09-09 09:35:10 -04:00
James Bardin a94f5ee132 prevent panics when encountering nil diffs
While we can't change the behavior of helper/schema at this point, we
can protect against panics in the case of unexpected nils in the
instance diff.
2019-09-04 16:51:42 -04:00
Alex Pilon 4bf43efcfd
move hcl2shim package to configs 2019-08-06 19:58:58 -04:00
James Bardin 682286e184 test for Required win MinItems > 1 2019-07-26 14:36:19 -07:00
Alex Pilon 7f8f198719
remove UnknownVariabeValue from config and update references to shim 2019-07-17 22:41:24 -04:00
James Bardin efd2a6cef6 dynamic set block test 2019-07-12 16:48:49 -04:00
James Bardin a0338df4d4 update ignore_changes to use cty.Path.Equals
Remove reflect.DeepEqual from path comparisons to get reliable results.

The equality issues were only noticed going the grpc interface, so add a
corresponding test to the test provider.
2019-07-10 14:49:37 -04:00
Radek Simko 5b9f2fafc8 Standardise directory name for test data 2019-06-30 10:16:15 +02:00
James Bardin 0d2363a058 add tests to preserve existing Set nil behavior
While it may not be intuitive, providers expect that setting a `nil`
value will appear as an empty string in state.
2019-06-28 12:09:50 -04:00
James Bardin 75602df5ef Revert "Remove removed attribute from applied state"
This reverts commit 2e2a363052.
2019-06-28 11:51:52 -04:00
James Bardin 0d9f84414a
Merge pull request #21879 from hashicorp/jbardin/sdk-panics
prevent sdk panics in 2 specific cases
2019-06-25 15:38:15 -04:00
James Bardin fe11724678 test for panic when readin empty map 2019-06-25 14:24:31 -04:00
James Bardin f362198467
Merge pull request #21806 from hashicorp/jbardin/lost-empty-strings
prevent an empty string from being lost
2019-06-25 12:32:06 -04:00
James Bardin c4874aa5b3 add more timeout provider tests 2019-06-19 22:48:15 -04:00
James Bardin 4dcda04d96 prevent an empty string from being lost
The helper/schema diff process loses empty strings, causing them to show
up as unset (null) during apply. Besides failing to show as set by
GetOk, the absence of the value also triggers the schema to insert a
default value again during apply.

It would also be be preferable if the defaults weren't re-evaluated
again during ApplyResourceChange, but that would require a more invasive
patch to the field readers, and ensuring the empty string is stored in
the plan should block the default.
2019-06-19 17:42:00 -04:00
James Bardin 8ae31aa2db normalize empty blocks during import
Like Upgrade, Import is another case where we don't have the context of
the configuration and need to ensure missing blocks are normalized.
2019-06-17 20:29:01 -04:00
James Bardin 2e2a363052 Remove removed attribute from applied state
When a Diff contains a NewRemoved attribute (which would have been null
in the planned state), the final value is often the "zero" value string
for the type, which the provider itself still applies to the state.
Rather than risking a change of behavior in helper/schema by fixing the
inconsistency, we'll remove the NewRemoved attributes after apply to
prevent further issues resulting from the change in planned value.
2019-06-13 17:29:25 -04:00
James Bardin d33c5163a7
Merge pull request #21555 from hashicorp/jbardin/re-validate
Allow providers to re-validate the final resource config
2019-06-07 16:43:38 -04:00
James Bardin a056b84cdd add delete timeout test
The timeout value is still not being persisted.

While it doesn't fix this issue, make sure to always return Private
during Plan.
2019-06-05 19:22:46 -04:00
James Bardin c658fe173a test a dynamic block with MinItems in the schema
A dynamic block is always going to be a single value during validation,
but should not fail when min-items > 1 is specified.
2019-05-31 20:07:41 -05:00
James Bardin 6055cb632e filter unknowns from simple lists and maps in sdk
While there was already a check in the sdk to filter unknowns from
validation, it missed the case where those were in simple lists and maps.
2019-05-28 09:58:07 -04:00
James Bardin c391f3a1a3 failing test for cty conversion bug 2019-05-16 16:40:17 -04:00
James Bardin b9e5745b3c remove SkipCoreTypeCheck and LegacySchema
Update the test provider to match the sdk.
2019-05-14 18:19:18 -04:00
James Bardin 6bc36d3321 validate integers when using protoV5
The new type system only has a Number type, but helper schema
differentiates between Int and Float values. Verify that a new config
value is an integer during Validate, because the existing WeakDecode
validation will decode a float value into an integer while the config
FieldReader will attempt to parse the float exactly.

Since we're limiting this to protoV5, we can be certain that any valid
config value will be converted to an `int` type by the shims. The only
case where an integral float value will appear is if the integer is out
of range for the systems `int` type, but we also need to prevent that
anyway since it would fail to read in the same manner.
2019-05-11 09:34:28 -04:00
James Bardin 7075bc9a4d restrict the ComputedKeys usage to containers
Computed primitive values must see the UnknownConfigValue or they are
assumed to be unchanged. Restrict the usage of the protov5 ComputedKeys
to containers.
2019-05-06 19:19:10 -04:00
Martin Atkins 083af21d30 providers/terraform: Explicit validate step
We were previously catching some errors at read time, but some type errors
were panicking because the cty.DynamicPseudoType arguments have no
automatic pre-type-checking done but this code was assuming they would
be objects.

Here we add an explicit validation step that includes both the backend
validation we were previously doing during read and some additional
type checking to ensure the two dynamic arguments are suitably-typed.
Having the separate validation step means that these problems can be
detected by "terraform validate", rather than only in "terraform plan"
or "terraform apply".
2019-05-04 21:06:31 -07:00
James Bardin f9dfa98533 test to make sure a data source is planned
The data source here needs to be re-read during apply, because its
config is changing.
2019-05-03 16:25:37 -04:00
Martin Atkins 332010fd56 plans/objchange: Fix handling of dynamic block placeholders
If a dynamic block (in the HCL dynamic block extension sense) has an
unknown value for its for_each argument, it gets expanded to a single
placeholder block with all of its attributes set to a unknown values.

We can use this as part of a heuristic to relax our object compatibility
checks for situations where the plan included an object that appears to
be (but isn't necessarily) such a placeholder, allowing for the fact that
the one placeholder block could be replaced with zero or more real blocks
once the for_each value is known.

Previously our heuristic was too strict: it would match only if the only
block present was a dynamic placeholder. In practice, users may mix
dynamic blocks with static blocks of the same type, so we need to be more
liberal to avoid generating incorrect incompatibility errors in such
cases.
2019-05-02 14:08:40 -07:00
James Bardin 0f883b118a test that computed maps are applied correctly
Verify that a computed map which is not correctly marked as unknown in
the plan is still applied correctly.
2019-04-23 12:31:19 -04:00
James Bardin 67395306e1 delete unknown values from apply config altogether
removeConfigUnknowns need to remove the value completely from the config
map. Removing this value allows GetOk and GetOkExists to indicate if the
value was set in the config in the case of an Optional+Computed
attribute.
2019-04-22 18:06:26 -04:00
Martin Atkins 861a2ebf26 helper/schema: Use a more targeted shim for nested set diff applying
We previously attempted to make the special diff apply behavior for nested
sets of objects work with attribute mode by totally discarding attribute
mode for all shims.

In practice, that is too broad a solution: there are lots of other shimming
behaviors that we _don't_ want when attribute mode is enabled. In
particular, we need to make sure that the difference between null and
empty can be seen in configuration.

As a compromise then, we will give all of the shims access to the real
ConfigMode and then do a more specialized fixup within the diff-apply
logic: we'll construct a synthetic nested block schema and then use that
to run our existing logic to deal with nested sets of objects, while
using the previous behavior in all other cases.

In effect, this means that the special new behavior only applies when the
provider uses the opt-in ConfigMode setting on a particular attribute,
and thus this change has much less risk of causing broad, unintended
regressions elsewhere.
2019-04-17 07:47:31 -07:00
Martin Atkins ff2de9c818 core: Keep old value on error even for delete
When an operation fails, providers may return a null new value rather than
returning a partial state. In that case, we'd prefer to keep the old value
so that we stand the best chance of being able to retry on a subsequent
run.

Previously we were making an exception for the delete action, allowing
the result of that to be null even when an error is returned. In practice
that was a bad idea because it would cause Terraform to lose track of the
object even though it might not actually have been deleted.

Now we'll retain the old object even in the delete case. Providers can
still return partial new objects if they were able to partially complete
a delete operation, in which case we'll discard what we had before, but
if the result is null with errors then we'll assume the delete failed
entirely and so just keep the old state as-is, giving us the opportunity
to refresh it on the next run to see if anything actually happened after
all.

(This also includes a new resource in the test provider which isn't used
by the patch but was useful for some manual UX testing here, so I thought
I'd include it in case it's similarly useful in future, given how simple
its implementation is.)
2019-04-17 07:40:15 -07:00
Martin Atkins bd1a215580 helper/resource: Ignore Removed attributes for ImportStateVerify
Due to the lossiness of our legacy models for diff and state, shimming a
diff and then creating a state from it produces a different result than
shimming a state directly. That means that ImportStateVerify no longer
works as expected if there are any Computed attributes in the schema where
d.Set isn't called during Read.

Fixing that for every case would require some risky changes to the shim
behavior, so we're instead going to ask provider developers to address it
by adding `d.Set` calls where needed, since that is the contract for
"Computed" anyway -- a default value should be produced during Create, and
thus by extension during Import.

However, since a common situation where this occurs is attributes marked
as "Removed", since all of the code that deals with them has generally
been deleted, we'll avoid problems in that case here by treating Removed
attributes as ignored for the purposes of ImportStateVerify.

This required exporting some functionality that was formerly unexported
in helper/schema, but it's a relatively harmless schema introspection
function so shouldn't be a big deal to export it.
2019-04-16 11:14:49 -07:00