* Add support for plugin protocol v6
This PR turns on support for plugin protocol v6. A provider can
advertise itself as supporting protocol version 6 and terraform will
use the correct client.
Todo:
The "unmanaged" providers functionality does not support protocol
version, so at the moment terraform will continue to assume that
"unmanaged" providers are on protocol v5. This will require some
upstream work on go-plugin (I believe).
I would like to convert the builtin providers to use protocol v6 in a
future PR; however it is not necessary until we remove protocol v6.
* add e2e test for using both plugin protocol versions
- copied grpcwrap and made a version that returns protocol v6 provider
- copied the test provider, provider-simple, and made a version that's
using protocol v6 with the above fun
- added an e2etest
* providers.Interface: huge renamification
This commit renames a handful of functions in the providers.Interface to
match changes made in protocol v6. The following commit implements this
change across the rest of the codebase; I put this in a separate commit
for ease of reviewing and will squash these together when merging.
One noteworthy detail: protocol v6 removes the config from the
ValidateProviderConfigResponse, since it's never been used. I chose to
leave that in place in the interface until we deprecate support for
protocol v5 entirely.
Note that none of these changes impact current providers using protocol
v5; the protocol is unchanged. Only the translation layer between the
proto and terraform have changed.
Add a test for a diagnostic with no steps to make sure it still gets
associated with the resource in the config. Follow up to #27710 using
@alisdair's suggested testing.
Right now, there's a bug that if a diagnostic comes back from the
provider with an AttributePath set, but no steps in the AttributePath,
Terraform _thinks_ it's an attribute-specific diagnostic and not a
whole-resource diagnostic, but then doesn't associate it with any
specific attribute, meaning the diagnostic doesn't get associated with
the config at all.
This PR changes things to check if there are any steps in the
AttributePath before deciding this isn't a whole-resource diagnostic,
and if there aren't, treats it as a whole-resource diagnostic, instead.
See hashicorp/terraform-plugin-sdk#561 for more details on how this
surfaces in the wild.
Provisioner plugins were the last remaining use of the legacy netrpc
plugin interface (even though it was wrapped in shimmed into the grpc
protocol). Now that these types are no longer being used, we can remove
the old code from the plugin package entirely.
Terraform does not use rpc errors for any error communication, so these
are always something that went wrong in outside of the plugin protocol.
The most common example of which is a provider crash, which would return
"rpc error: code = Unavailable desc = transport is closing". Replace
these error codes with something a little more presentable for the user,
and insert the calling method name to help correlate it to the
operation that failed.
Core was previously ignoring JSON-encoded dynamic values, but these are
technically supported, so we must either error or accept the value.
Since we already have the decoder for Json state, it's minimal effort to
support this on all plugin methods too.
This change also gives providers an easy way to implement the
UpgradeResourceState method. The obvious implementation of returning the same
JSON-encoded value has tripped up a few providers not using the legacy
SDK already, and we should have at least indicated that the value was
being lost.
Implement a new provider_meta block in the terraform block of modules, allowing provider-keyed metadata to be communicated from HCL to provider binaries.
Bundled in this change for minimal protocol version bumping is the addition of markdown support for attribute descriptions and the ability to indicate when an attribute is deprecated, so this information can be shown in the schema dump.
Co-authored-by: Paul Tyng <paul@paultyng.net>
* deps: bump terraform-config-inspect library
* configs: parse `version` in new required_providers block
With the latest version of `terraform-config-inspect`, the
required_providers attribute can now be a string or an object with
attributes "source" and "version". This change allows parsing the
version constraint from the new object while ignoring any given source attribute.
* terraform/context: use new addrs.Provider as map key in provider factories
* added NewLegacyProviderType and LegacyString funcs to make it explicit that these are temporary placeholders
This PR introduces a new concept, provider fully-qualified name (FQN), encapsulated by the `addrs.Provider` struct.
Send Private data blob through ReadResource as well. This will allow for
extra flexibility for future providers that may want to pass data out of
band through to their resource Read functions.
The grpc server does not shutdown when the listener is closed. Since
tests aren't run through go-plugin, which has a separate RPC Shutdown
channel to stop the server, we need to track and stop the server
directly.
* internal/initwd: Allow deprecated relative module paths
In Terraform 0.11 we deprecated this form but didn't have any explicit
warning for it. Now we'll still accept it but generate a warning. In a
future major release we will drop this form altogether, since it is
ambiguous with registry module source addresses.
This codepath is covered by the command/e2etest suite.
* e2e: Skip copying .exists file, if present
We use this only in the "empty" test fixture in order to let git know that
the directory exists. We need to skip copying it so that we can test
"terraform init -from-module=...", which expects to find an empty
directory.
* command/e2etests: Re-enable and fix up the e2etest "acctests"
We disabled all of the tests that accessed remote services like the
Terraform Registry while they were being updated to support the new
protocols we now expect. With those services now in place, we can
re-enable these tests.
Some details of exactly what output we print, etc, have intentionally
changed since these tests were last updated.
* e2e: refactor for modern states and plans
* command/e2etest: re-enable e2etests and update for tf 0.12 compatibility
plugin/discovery: mkdirAll instead of mkdir when creating cache dir
In study of existing providers we've found a pattern we werent previously
accounting for of using a nested block type to represent a group of
arguments that relate to a particular feature that is always enabled but
where it improves configuration readability to group all of its settings
together in a nested block.
The existing NestingSingle was not a good fit for this because it is
designed under the assumption that the presence or absence of the block
has some significance in enabling or disabling the relevant feature, and
so for these always-active cases we'd generate a misleading plan where
the settings for the feature appear totally absent, rather than showing
the default values that will be selected.
NestingGroup is, therefore, a slight variation of NestingSingle where
presence vs. absence of the block is not distinguishable (it's never null)
and instead its contents are treated as unset when the block is absent.
This then in turn causes any default values associated with the nested
arguments to be honored and displayed in the plan whenever the block is
not explicitly configured.
The current SDK cannot activate this mode, but that's okay because its
"legacy type system" opt-out flag allows it to force a block to be
processed in this way anyway. We're adding this now so that we can
introduce the feature in a future SDK without causing a breaking change
to the protocol, since the set of possible block nesting modes is not
extensible.
Some providers may generate quite large schemas, and the internal
default grpc response size limit is 4MB. 64MB should cover most any use
case, and if we get providers nearing that we may want to consider a
finer-grained API to fetch individual resource schemas.
If the registry is unresponsive, you will now get an error
specific to this, rather than a misleading "provider unavailable" type
error. Also adds debug logging for when errors like this may occur
Terraform Registry (and other registry implementations) can now return
an array of warnings with the versions response. These warnings are now
displayed to the user during a `terraform init`.
Kristin Laemmert