Commit Graph

21840 Commits

Author SHA1 Message Date
Martin Atkins c97dd145b9 addrs: Module.Call method
This is a helper for splitting a non-root module path into a callee and
call pair, similar to the method of the same name on ModuleInstance.
2018-10-16 18:46:46 -07:00
Martin Atkins d4cfe85361 core: pass InstanceKey to EvaluateBlock
This gives us the value we need to evaluate a "count.index" reference, and
later also the equivalent for the "for_each" argument once implemented.
2018-10-16 18:46:46 -07:00
Martin Atkins d4285dd27f core: Attach resource and provider config schemas during graph build
This is a little awkward since we need to instantiate the providers much
earlier than before. To avoid a lot of reshuffling here we just spin each
one up and then immediately shut it down again, letting our existing init
functionality during the graph walk still do the main initialization.
2018-10-16 18:46:46 -07:00
Martin Atkins ca67899827 core: NewLegacyResourceInstanceAddress correct handling of addrs.NoKey
Previously we were setting zero here, but the correct way to represent
"no index" in a *ResourceAddress is to set the index to -1.
2018-10-16 18:46:46 -07:00
Martin Atkins a28c93c3ce core: render variables, locals and outputs nicely in "terraform graph" 2018-10-16 18:46:46 -07:00
Martin Atkins 5cf791861f configs: Allow looking up resources by resource addresses.
Throughout the main "terraform" package we identify resources using the
address types, and so this helper is useful to make concise transitions
between the address types and the configuration types.

As part of this, we use the address types to produce the keys used in our
resource maps. This has no visible change in behavior since the prior
implementation produced an equal result, but this change ensures that
ResourceByAddr cannot be broken by hypothetical future changes to the
key serialization.
2018-10-16 18:46:46 -07:00
Martin Atkins ccc1b6990f config/hcl2shim: shims from flatmap to cty and vice-versa
These particular shims will have a pretty limited lifetime in mainline
Terraform code (primarily to stub out the new expression evaluator against
the old state structs until the new format is implemented) but will live
on for some time in state migration and provider plugin compatibility
shims.
2018-10-16 18:46:46 -07:00
Martin Atkins 9ca13d5c1d command/validate: make sure diagnostics is always present and an array
Previously an empty diagnostics would appear as "null" in the JSON output,
since that is how encoding/json serializes a nil slice. It's more
convenient for users of dynamic languages to keep the type consistent
in all cases, since they can then just iterate the list without needing a
special case for when it is null.
2018-10-16 18:46:46 -07:00
Martin Atkins 70f1635416 configs: record the source directory for modules
We can only do this when modules are loaded with Parser.LoadConfigDir,
but in practice this is the common case anyway.

This is important to support the path.module and path.root expressions in
configuration.
2018-10-16 18:46:46 -07:00
Martin Atkins fa2a76fa23 configs: use addrs.Module for module path, rather than []string
addrs.Module is itself internally just []string, but this better
communicates our intent here and makes this integrate better with other
code which is using this type for this purposes.
2018-10-16 18:46:46 -07:00
Martin Atkins 226d66bac8 core: include "root" element when constructing legacy ModuleState path
We've not yet adjusted any of the state structs to reflect our new address
types because they are used with encoding/json to produce our state file
format, but the shimming here previously was incorrect because it failed
to include the special "root" string that's always required at element
zero of a module path in the state.
2018-10-16 18:46:46 -07:00
Martin Atkins c937c06a03 terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.

The three main goals here are:
- Use the configuration models from the "configs" package instead of the
  older models in the "config" package, which is now deprecated and
  preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
  new "lang" package, instead of the Interpolator type and related
  functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
  rather than hand-constructed strings. This is not critical to support
  the above, but was a big help during the implementation of these other
  points since it made it much more explicit what kind of address is
  expected in each context.

Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.

I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-10-16 18:46:46 -07:00
Martin Atkins 2c70d884d6 dag: node visits return diagnostics rather than errors
This allows node visits to also return warnings.
2018-10-16 18:44:26 -07:00
Martin Atkins 24dce0c624 configs: Helper methods to integrate with "addrs" package
Our new "addrs" package gives us some nice representations of various
kinds of "address" within Terraform. To talk to APIs that use these, it's
convenient to be able to easily derive such addresses from the
configuration objects.

These new methods, along with a recasting of the existing
Resource.ProviderConfigKey method to Resource.ProviderConfigAddr, give us
some key integration points to support the configuration graph transforms
in the main "terraform" package.
2018-10-16 18:44:26 -07:00
Martin Atkins 072322336e addrs: AbsProviderConfig.Inherited method
This helper deals with the address wrangling required to find the address
that a provider configuration might inherit from if no explicit
configuration is given and instead configuration is taken from the
parent module.

This method is not generally useful, and is here mainly just to help the
provider-related graph transformations in the main terraform package.
2018-10-16 18:44:26 -07:00
Martin Atkins 02b25e7057 addrs: flesh out functionality ready to replace terraform.ResourceAddress
This "kitchen sink" commit is mainly focused on supporting "targets" as
a new sub-category of addresses, for use-case like the -target CLI option,
but also includes some other functionality to get closer to replacing
terraform.ResourceAddress and fill out some missing parts for representing
various other address types that are currently represented as strings
in the "terraform" package.
2018-10-16 18:44:26 -07:00
Martin Atkins b6fdd0446e configs: parse the "providers" map for module calls
This was accidentally missed on the first pass of module call decoding.
As before, this is a map from child provider config address to parent
provider config address, allowing the set of providers to be projected in
arbitrary ways into a child module.
2018-10-16 18:44:26 -07:00
Martin Atkins c6598a3f86 addrs: ParseAbsProviderConfig function
This is for parsing the type of provider configuration address we write
into state in order to remember which provider configuration is
responsible for each resource.
2018-10-16 18:44:26 -07:00
Martin Atkins 8ca174b133 addrs: implement String for AbsResource and AbsResourceInstance 2018-10-16 18:44:26 -07:00
Martin Atkins a09498a8a3 core: load a provider's schema at initialization
This is currently not very ergonomic due to the API exposed by providers.
We'll smooth this out in a later change to improve the provider API, since
we know we always want the entire schema.
2018-10-16 18:44:26 -07:00
Martin Atkins 3822650e15 tfdiags: Diagnostics.ErrWithWarnings and .NonFatalErr
There is some existing practice in the "terraform" package of returning
a special error type ValidationError from EvalNode implementations in
order to return warnings without halting the graph walk even though a
non-nil error was returned.

This is a diagnostics-flavored version of that approach, allowing us to
avoid totally reworking the EvalNode concept around diagnostics and
retaining the ability to return non-fatal errors.

NonFatalErr is equivalent to the former terraform.ValidationError, while
ErrWithWarnings is a helper that automatically treats any errors as
fatal but returns NonFatalError if the diagnostics contains only warnings.
2018-10-16 18:44:26 -07:00
Martin Atkins fca07d1a61 addrs: AbsProviderConfig type
Fitting with the usual naming scheme, this is the combination of a
module instance address and a provider config.
2018-10-16 18:44:26 -07:00
Martin Atkins dd62cd97c9 configs: DisabledModuleWalker
This is a built-in implementation of ModuleWalker that just returns an
error any time it's asked for a module. This is intended for simple unit
tests where no child modules are needed anyway.
2018-10-16 18:44:26 -07:00
Martin Atkins 2eba023537 configs: NewEmptyConfig function
This is useful for creating a valid placeholder configuration, but not
much else. Most callers should use BuildConfig to build a configuration
that actually has something in it.
2018-10-16 18:44:26 -07:00
Martin Atkins c07b0a7806 configs: Re-unify the ManagedResource and DataResource types
Initially the intent here was to tease these apart a little more since
they don't really share much behavior in common in core, but in practice
it'll take a lot of refactoring to tease apart these assumptions in core
right now and so we'll keep these things unified at the configuration
layer in the interests of minimizing disruption at the core layer.

The two types are still kept in separate maps to help reinforce the fact
that they are separate concepts with some behaviors in common, rather than
the same concept.
2018-10-16 18:44:26 -07:00
Martin Atkins cd51864d84 configs: Start using the new "addrs" package types for modules
We initially just mimicked our old practice of using []string for module
paths here, but the addrs package now gives us a pair of types that better
capture the two different kinds of module addresses we are dealing with:
static addresses (nodes in the configuration tree) and dynamic/instance
addresses (which can represent the situation where multiple instances are
created from a single module call).

This distinction still remains rather artificial since we don't yet have
support for count or for_each on module calls, but this is intended to lay
the foundations for that to be added later, and in the mean time just
gives us some handy helper functions for parsing and formatting these
address types.
2018-10-16 18:44:26 -07:00
Martin Atkins a16ca2ec53 lang: new package for the runtime parts of the config language
Whereas package "configs" deals with the static structure of the
configuration language, this new package "lang" deals with the dynamic
aspects such as expression evaluation.

So far this mainly consists of populating a hcl.EvalContext that contains
the values necessary to evaluate a block or an expression. There is also
special handling here for dynamic block generation using the HCL
"dynblock" extension, which is exposed in the public interface (rather
than hiding it as an implementation detail of EvalBlock) so that the
caller can then extract proper source locations for any result values
using the expanded body.

This also includes the beginnings of a replacement for the function table
handling that currently lives in the old "config" package, but most of
the functions are not yet ported and so this will expand in subsequent
commits.
2018-10-16 18:44:26 -07:00
Martin Atkins e5dfa17433 addrs: All Referenceable implementations support String()
This gives us a convenient way to convert back to a string representation
that matches what the user would've written in configuration.
2018-10-16 18:44:26 -07:00
Martin Atkins f2809854a2 govendor fetch github.com/hashicorp/hcl2/ext/dynblock/... 2018-10-16 18:44:26 -07:00
Martin Atkins 63041ffa09 addrs: ParseRef function, for parsing references in expressions
This function corresponds to terraform.NewInterpolatedVariable, but built
with HCL2 primitives. It accepts a hcl.Traversal, which is what is
returned from the HCL2 API functions to find which variables are
referenced in a given expression.
2018-10-16 18:44:26 -07:00
Martin Atkins b9d84f2944 addrs: new package for representing addresses
This package is intended to contain all the functionality for parsing,
representing, and formatting addresses of objects within Terraform.

It will eventually subsume the responsibilities of both the
InterpolatedVariable and ResourceAddress types in the "terraform" package,
but for the moment is just a set of types for representing these things,
lacking any way to parse or format them. The remaining functionality
will follow in subsequent commits.
2018-10-16 18:44:26 -07:00
Martin Atkins 4ed06a9227 terraform: HCL2-flavored module dependency resolver
For the moment this is just a lightly-adapted copy of
ModuleTreeDependencies named ConfigTreeDependencies, with the goal that
the two can live concurrently for the moment while not all callers are yet
updated and then we can drop ModuleTreeDependencies and its helper
functions altogether in a later commit.

This can then be used to make "terraform init" and "terraform providers"
work properly with the HCL2-powered configuration loader.
2018-10-16 18:44:26 -07:00
Martin Atkins ebafa51723 command: Various updates for the new backend package API
This is a rather-messy, complex change to get the "command" package
building again against the new backend API that was updated for
the new configuration loader.

A lot of this is mechanical rewriting to the new API, but
meta_config.go and meta_backend.go in particular saw some major
changes to interface with the new loader APIs and to deal with
the change in order of steps in the backend API.
2018-10-16 18:44:26 -07:00
Martin Atkins 5782357c28 backend: Update interface and implementations for new config loader
The new config loader requires some steps to happen in a different
order, particularly in regard to knowing the schema in order to
decode the configuration.

Here we lean directly on the configschema package, rather than
on helper/schema.Backend as before, because it's generally
sufficient for our needs here and this prepares us for the
helper/schema package later moving out into its own repository
to seed a "plugin SDK".
2018-10-16 18:39:12 -07:00
Martin Atkins 591aaf1e6a configload: helper functions for tests
These utility functions are intended to allow concisely loading a
configuration from a fixture directory in a test, bailing out early if
there are any unexpected errors.
2018-10-16 18:24:47 -07:00
Martin Atkins 1fb714ea3b helper/schema: CoreConfigSchema method for Backend
This is just like the method of the same name on Resource, adapting the
helper/schema model of schema into Terraform Core's idea of schema.
2018-10-16 18:24:47 -07:00
Martin Atkins 22d140e883 configschema: Block.CoerceValue method
The usual way to use a configschema.Block is to obtain a hcldec spec from
it and then decode an hcl.Body. There are inevitably situations though
where a body has already been decoded into a cty.Value before we know
which schema we need to use.

This new method CoerceValue is intended to deal with this case, applying
the schema to an already-decoded value in what should be an intuitive way
for most situations.
2018-10-16 18:24:47 -07:00
Martin Atkins 6cf9346dfd configs/configload: Helper for recognizing a config dir (or not)
This is just a small wrapper around the parser's IsConfigDir method, for
convenience and for law of demeter.
2018-10-16 18:24:47 -07:00
Martin Atkins 5dd6b839d0 configs: Export MergeBodies and new SynthBody function
We have a few special use-cases in Terraform where an object is
constructed from a mixture of different sources, such as a configuration
file, command line arguments, and environment variables.

To represent this within the HCL model, we introduce a new "synthetic"
HCL body type that just represents a map of values that are interpreted
as attributes.

We then export the previously-private MergeBodies function to allow the
synthetic body to be used as an override for a "real" body, which then
allows us to combine these various sources together while still retaining
the proper source location information for each individual attribute.

Since a synthetic body doesn't actually exist in configuration, it does
not produce source locations that can be turned into source snippets but
we can still use placeholder strings to help the user to understand
which of the many different sources a particular value came from.
2018-10-16 18:24:47 -07:00
Martin Atkins d6c6f8852c configschema: include description in schema
We will need access to this information in order to render interactive
input prompts, and it will also be useful in returning schema information
to external tools such as text editors that have autocomplete-like
functionality.
2018-10-16 18:24:47 -07:00
James Bardin 0a6938e90a update tests to match pre-release constraint rules 2018-10-16 18:24:47 -07:00
James Bardin 398a6ef31a update go-version
The latest go-version update properly handles pre-release versions in
constraints.
2018-10-16 18:24:47 -07:00
Martin Atkins 618883596a command: remove "terraform push"
The remote API this talks to will be going away very soon, before our next
major release, and so we'll remove the command altogether in that release.

This also removes the "encodeHCL" function, which was used only for
adding a .tfvars-formatted file to the uploaded archive.
2018-10-16 18:24:47 -07:00
Martin Atkins c51d22bdeb config/hcl2shim: don't put nils in ConfigValue maps
Absent values are omitted by the old code we are emulating in HCL, so we
must do the same here in order to avoid breaking assumptions in the
helper/schema layer.
2018-10-16 18:24:10 -07:00
Martin Atkins 860a57d104 tfdiags: Simple helper for creating "sourceless" diagnostics
While diagnostics are primarily designed for reporting problems in
configuration, they can also be used for errors and warnings about the
environment Terraform is running in, such as inability to reach a remote
service, etc.

Function Sourceless makes it easy to produce such diagnostics with the
customary summary/detail structure. When these diagnostics are rendered
they will have no source code snippet and will instead just include the
English-language content.
2018-10-16 18:24:10 -07:00
Martin Atkins 9bd47bf17c tfdiags: helper functions for nicer display of cty.PathError
cty doesn't have a native string representation of a cty.Path because it
is the layer below any particular syntax, but up here in Terraform land
we know that we use HCL native syntax and so we can format a string in
a HCL-ish way for familiarity to the user.

We'll use this form automatically when such an error is used directly as a
diagnostic, but we also expose the function publicly so that other code
that incorporates errors into diagnostic detail strings can apply the
same formatting.
2018-10-16 18:24:10 -07:00
Martin Atkins e309675853 tfdiags: Contextual diagnostics
The usual usage of diagnostics requires us to pass around source location
information to everywhere that might generate a diagnostic, and that is
always the best way to get the most precise diagnostic source locations.

However, it's impractical to require source location information to be
retained in every Terraform subsystem, and so this new idea of "contextual
diagnostics" allows us to separate the generation of a diagnostic from
the resolution of its source location, instead resolving the location
information as a post-processing step once the call stack unwinds to a
place where there is enough context to find it.

This is necessarily a less precise approach than reading the source ranges
directly from the configuration AST, but gives us an alternative to no
diagnostics at all in portions of Terraform where full location
information is not available.

This is a best-effort sort of thing which will get as precise as it can
but may return a range in a parent block if the precise location of a
particular attribute cannot be found. Diagnostics that rely on this
mechanism should include some other contextual information in the detail
message to make up for any loss of precision that results.
2018-10-16 18:24:10 -07:00
Martin Atkins b41f08d79d govendor fetch github.com/zclconf/go-cty/... 2018-10-16 18:24:10 -07:00
James Bardin a79d620f03 remove legacy remote state code 2018-10-16 18:24:10 -07:00
James Bardin 979faa5dbe move artifactory remote state to backend 2018-10-16 18:23:14 -07:00