This includes a bugfix to the cty/msgpack package to ensure correct
decoding of unknown and null values.
This also includes updates to cty's dependencies.
govendor's lack of understanding of transitive versions was making it hell
to figure out some dependency hell between etcd, grpc, and protobuf. After
fighting with it for a few hours, I decided to give Go 1.11rc2 a try since
previous experiments had been promising on the 1.11 tree in master.
The dependencies all worked out first time when managed using the Go
Modules code, and so we'll run with this now to continue to make progress
though we may wish to back out of this nearer to release and return to
govendor for a while until other projects have caught up.
However, since this commit includes a vendor directory produced using Go
Modules it doesn't actually _require_ Go 1.11 to build, and instead
requires it only to make further changes to the selected versions in the
vendor dir. Go 1.10's vendoring support will still find the modules in
their expected locations within the vendor dir.
We need to make the collection itself be a tuple or object rather than
list or map in this case, since otherwise all of the elements of the
collection are constrained to be of the same type and that isn't the
intent of a provider indicating that it accepts any type.
This function's goal is to ensure that the "final" plan value produced
by a provider during the apply step is always consistent with the known
parts of the planned value produced during the plan step.
Any error produced here indicates a bug in the provider.
This produces a "proposed new state", which already has prior computed
values propagated into it (since that behavior is standard for all
resource types) but could be customized further by the provider to make
the "_planned_ new state".
In the process of implementing this it became clear that our configschema
DecoderSpec behavior is incorrect, since it's producing list values for
NestingList and map values for NestingMap. While that seems like it should
be right, we should actually be using tuple and object types respectively
to allow each block to have a different runtime type in situations where
an attribute is given the type cty.DynamicPseudoType. That's not fixed
here, and so without a further fix list and map blocks will panic here.
The DecoderSpec implementation will be fixed in a subsequent commit.
This allows a provider to retain arbitrary extra data in the plan and
make use of it during apply. The contents are not used by Terraform and
never shown to the user.
We're going to allow the provider to encode whatever it wants in here, so
a provider can use whatever is most convenient for its implementation
language and to avoid some of the bugs we saw with the prior model where
the forced round-trip through JSON and back into interface{} would cause
some loss of fidelity, leading to bugs.
Chaange ResourceProvider to providers.Interface starting from the
context, and fix all type errors.
This only replaced some of method calls directly applicable to the
providers themselves. The resource methods will follow.
This is a general catchup of some developments in the HCL2 codebase, but
in particular includes:
- Recording expression and evalcontext as part of diagnostics, so that
variable value information can be included alongside diagnostic
snippets.
- hcldec supports decoding blocks into tuple and object values as well as
list and map values, which then allows cty.DynamicPseudoType nested
attributes to work properly.
These new types are adaptations of terraform.ProviderResolver and
terraform.ResourceProviderFactory respectively. In a later commit we will
update users of the old types to use these new types, possibly in a
"gradual repair" sort of fashion by initially using type aliases, though
that won't 100% solve the problem because the types themselves have
changed in this move to return providers.Interface instead of
terraform.ResourceProvider.
MockProvider and MockProvisioner implement the new plugin interfaces,
and are built following the patterns used by the legacy
MockResourceProvider and MockResourceProvisioner
Close was previously an optional interface, because it's only applicable
on the client side but the types were shared accross all packages. Since
all plugins will now implement Close, it no longer needs to be optional.
Managing which function need to be shared between the terraform plugin
and the helper plugin without creating cycles was becoming difficult.
Move all functions related to converting between terraform and proto
type into plugin/convert.
Due to how often the state and plan types are referenced throughout
Terraform, there isn't a great way to switch them out gradually. As a
consequence, this huge commit gets us from the old world to a _compilable_
new world, but still has a large number of known test failures due to
key functionality being stubbed out.
The stubs here are for anything that interacts with providers, since we
now need to do the follow-up work to similarly replace the old
terraform.ResourceProvider interface with its replacement in the new
"providers" package. That work, along with work to fix the remaining
failing tests, will follow in subsequent commits.
The aim here was to replace all references to terraform.State and its
downstream types with states.State, terraform.Plan with plans.Plan,
state.State with statemgr.State, and switch to the new implementations of
the state and plan file formats. However, due to the number of times those
types are used, this also ended up affecting numerous other parts of core
such as terraform.Hook, the backend.Backend interface, and most of the CLI
commands.
Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize
in advance to the person who inevitably just found this huge commit while
spelunking through the commit history.
Our existing core tests make extensive use of the string representation
of a state for comparison purposes, because they were written before we
began making use of helper packages like "cmp".
To avoid the need to rewrite all of those tests and potentially break
them, we will instead port that particular rendering as closely as
possible but mark it with a comment sternly warning not to use it for
anything new.
We don't want to use this moving forward for a number of reasons, but
most notably:
- printing out whole before and after state representations makes it
hard to find a subtle difference in outcome when a test fails, while
"cmp" can provide us with a real diff.
- this string serialization is constrained by the capabilities of
Terraform prior to our new state models, and so it does not
comprehensively represent all possibilities in the new world.
- it will probably behave oddly/poorly when given states containing
features that arrived after it was written, even though I made a
best effort here to make it do something reasonable in situations
I thought about.
The types here were originally written to allow us to defer decoding of
object values until schemas are available, but it turns out that this was
forcing us to defer decoding longer than necessary and potentially decode
the same value multiple times.
To avoid this, we create pairs of types to represent the encoded and
decoded versions and methods for moving between them. These types are
identical to one another apart from how the dynamic values are
represented.
The types here were originally written to allow us to defer decoding of
object values until schemas are available, but it turns out that this was
forcing us to defer decoding longer than necessary and potentially decode
the same value multiple times.
To avoid this, we create pairs of types to represent the encoded and
decoded versions and methods for moving between them. These types are
identical to one another apart from how the dynamic values are
represented.
In practice these pairs of functions are often used together when working
with a "full" statemgr, so these helper wrappers allow us to do that more
conveniently.
This also introduces a new interface statemgr.Storage, which represents
a state manager that has all of the storage capabilities but does not
necessarily support locking. In practice callers will usually just use
statemgr.Full, but these more-specific interfaces allow us to reflect
in APIs which subset of the statemgr functionality each function depends
on.
The new helper/plugin package contains the grpc servers for handling the
new plugin protocol
The GRPCProviderServer and GRPCProvisionerServer handle the grpc plugin
protocol, and convert the requests to the legacy schema.Provider and
schema.Provisioner methods.
Here we add the GRPCProvisioner and GRPCProvider which implement the
core provisioners.Interface and providers.Interface, and translate
betweeen the core types and the grpc protocol.
These will allow easier testing of the grpc endpoints in isolation.
Mocks are generated for ProviderClient, ProvisionerClient,
Provisioner_ProvisionResourceClient, and
Provisioner_ProvisionResourceServer using `go generate`
support the requested platform.
If the newest version of a provider which matches the version
constraints does not support the requested platform, filter the list of
available versions by platform support and try again.
In order to not require state migrations to be supported in both
MigrateState and StateUpgraders, the legacy provider codepath needs to
handle the StateUpgraders transparently during Refresh.
This adds some of the required shim functions to the schema package.
While this further bloats the already huge package, adding the helpers
here was significantly less disruptive than refactoring types into
separate packages to prevent import cycles.
The majority of tests here are directly adapted from existing schema
tests to provide as many known good values to the shims as possible.
Martin Atkins