Commit Graph

24 Commits

Author SHA1 Message Date
Pam Selle c55b69e30a Fix diags non-assignment bugs
Fix places where diags was not reassigned when diags were added.
2020-11-20 13:37:23 -05:00
James Bardin 7c4d00c04c allow path and terraform in self-block eval
We can insert the terraform and path values into EvalSelfBlock, since
these are static and always known during evaluation.
2020-11-02 14:00:58 -05:00
Martin Atkins e2c64bc255 core: Annotate for_each errors with expression info
Our diagnostics model allows for optionally annotating an error or warning
with information about the expression and eval context it was generated
from, which the diagnostic renderer for the UI will then use to give the
user some additional hints about what values may have contributed to the
error.

We previously didn't have those annotations on the results of evaluating
for_each expressions though, because in that case we were using the helper
function to evaluate an expression in one shot and thus we didn't ever
have a reference to the EvalContext in order to include it in the
diagnostic values.

Now, at the expense of having to handle the evaluation at a slightly lower
level of abstraction, we'll annotate all of the for_each error messages
with source expression information. This is valuable because we see users
often confused as to how their complex for_each expressions ended up being
invalid, and hopefully giving some information about what the inputs were
will allow more users to self-solve.
2020-10-29 09:07:48 -07:00
James Bardin 64c3a8f550 lang.Scope.EvalSelfBlock
In order to properly evaluate a destroy provisioner, we cannot rely on
the usual evaluation context, because the resource has already been
removed from the state.

EvalSelfBlock evaluates an hcl.Body in the limited scope of a single
object as "self", with the added values of "count.index" and "each.key".
2020-10-02 11:25:21 -04:00
James Bardin 42cee86ee2 remove GetModuleInstanceOutput
There is no codepath that can use this any longer, since we need to
evaluate the modules as whole objects.

This means we're going to have to live for now with invalid module
output references returning "object" errors rather that "module".
2020-04-14 14:49:10 -04:00
James Bardin 600d4c3e1f eval Data needs to operate on whole modules
In order to be able to use module values, and handle operations like
possibly invalid module indexes in conditional statements, whole modules
must always be returned during evaluation.
2020-04-12 10:50:31 -04:00
Pam Selle f738f85241 Create non-specific ModuleCallOutput 2020-03-26 13:29:38 -04:00
James Bardin 3839405d3d correctly evaluate self in for_each resources
The fallback type for GetResource from an EachMap is a cty.Object,
because resource schemas may contain dynamically typed attributes.
Check for an Object type in the evaluation of self, to use the proper
GetAttr method when extracting the value.
2019-10-29 12:44:42 -04:00
James Bardin 1ee851f256
Merge pull request #22846 from hashicorp/jbardin/evaluate-resource
Always evaluate resources in their entirety
2019-10-08 07:57:15 -04:00
James Bardin bfce78064b lang/eval: more evalContext fixups
self references do not need to be added to `managedResource`, and in
fact that could cause issues later if self is allowed in contexts other
than managed resources.

Coalesce 2 cases in the Referenceable switch, be take the
ContainingResource address of an instance beforehand.
2019-10-07 18:13:20 -04:00
Martin Atkins 39e609d5fd vendor: switch to HCL 2.0 in the HCL repository
Previously we were using the experimental HCL 2 repository, but now we'll
shift over to the v2 import path within the main HCL repository as part of
actually releasing HCL 2.0 as stable.

This is a mechanical search/replace to the new import paths. It also
switches to the v2.0.0 release of HCL, which includes some new code that
Terraform didn't previously have but should not change any behavior that
matters for Terraform's purposes.

For the moment the experimental HCL2 repository is still an indirect
dependency via terraform-config-inspect, so it remains in our go.sum and
vendor directories for the moment. Because terraform-config-inspect uses
a much smaller subset of the HCL2 functionality, this does still manage
to prune the vendor directory a little. A subsequent release of
terraform-config-inspect should allow us to completely remove that old
repository in a future commit.
2019-10-02 15:10:21 -07:00
James Bardin d4031918d1 parse resource refs correctly
Now that we only evaluate whole resources, we can parse resource refs
correct as the resource, rather than an unknown instance.
2019-09-19 11:46:09 -04:00
James Bardin c49f976308 change lang eval to also only lookup resources
Continue only evaluating resource at a whole and push the indexing of
the resource down into the expression evaluation.

The exception here is that `self` must be an instance which must be
extracted from the resource. We now also add the entire resource to the
context, which was previously only partially populated with the self
referenced instance.
2019-09-19 11:46:09 -04:00
James Bardin 86bf674246 change GetResourceInstance to GetResource
In order to allow lazy evaluation of resource indexes, we can't index
resources immediately via GetResourceInstance. Change the evaluation to
always return whole Resources via GetResource, and index individual
instances during expression evaluation.

This will allow us to always check for invalid index errors rather than
returning an unknown value and ignoring it during apply.
2019-09-19 09:19:14 -04:00
Pam Selle 7d905f6777 Resource for_each 2019-07-22 10:51:16 -04:00
Martin Atkins 4d52999538 lang: EvalBlock should use ReferencesInBlock
Previously it was calling directly to hcldec.Variables, and thus missing
the special fixups we do inside ReferencesInBlock to deal with
Terraform-specific concerns such as our attribute-as-blocks preprocessing.
2019-04-03 15:57:17 -07:00
Martin Atkins 87786484ea lang/eval: Apply attr-as-nested-block fixup in EvalBlock
For any block content we evaluate dynamically via this API, we'll make a
special allowance for users to optionally write members of a list
attribute instead as a sequence of nested blocks, thus allowing some
existing provider features that were assuming this capability to continue
to support it after v0.12.

This should not be used for any new provider features, and should ideally
be eventually phased out so that there aren't two
similar-but-slightly-different syntaxes for saying the same thing.
2019-03-28 10:41:01 -07:00
Martin Atkins 838a42d218 vendor: go get github.com/hashicorp/hcl2@master
This includes improved functionality for HCL's "dynamic block extension",
which will allow us (in a subsequent commit) to properly detect
dependencies inside nested "dynamic" blocks, where currently they get
missed.

For this commit though, we just upgrade HCL to a version that includes it
and make a small change to our "lang" package to align with an upstream
renaming.
2019-03-19 10:04:45 -07:00
Martin Atkins 2be524d6ac core: Validate depends_on and ignore_changes traversals
Both depends_on and ignore_changes contain references to objects that we
can validate.

Historically Terraform has not validated these, instead just ignoring
references to non-existent objects. Since there is no reason to refer to
something that doesn't exist, we'll now verify this and return errors so
that users get explicit feedback on any typos they may have made, rather
than just wondering why what they added seems to have no effect.

This is particularly important for ignore_changes because users have
historically used strange values here to try to exploit the fact that
Terraform was resolving ignore_changes against a flatmap. This will give
them explicit feedback for any odd constructs that the configuration
upgrade tool doesn't know how to detect and fix.
2018-12-17 09:02:25 -08:00
Martin Atkins e63a1dfb96 lang: EvalExpr only convert if wantType is not dynamic
This actually seems to be a bug in the underlying cty Convert function
since converting to cty.DynamicPseudoType should always just return the
input verbatim, but it seems like it's actually converting unknown values
of any type to be cty.DynamicVal, losing the type information.

We should eventually fix this in cty too, but having this extra check in
the Terraform layer is harmless and allows us to make progress without
context-switching.
2018-12-07 17:05:36 -08:00
Martin Atkins 3b49028b77 core: Static-validate resource references against schemas
In the initial move to HCL2 we started relying only on full expression
evaluation to catch attribute errors, but that's not sufficient for
resource attributes in practice because during validation we can't know
yet whether a resource reference evaluates to a single object or to a
list of objects (if count is set).

To address this, here we reinstate some static validation of resource
references by analyzing directly the reference objects, disregarding any
instance index if present, and produce errors if the remaining subsequent
traversal steps do not correspond to items within the resource type
schema.

This also allows us to produce some more specialized error messages for
certain situations. In particular, we can recognize a reference like
aws_instance.foo.count, which in 0.11 and prior was a weird special case
for determining the count value of a resource block, and offer a helpful
error showing the new length(aws_instance.foo) usage pattern.

This eventually delegates to the static traversal validation logic that
was added to the configschema package in a previous commit, which also
includes some specialized error messages that distinguish between
attributes and block types in the schema so that the errors relate more
directly to constructs the user can see in the configuration.

In future we could potentially move more of the checks from the dynamic
schema construction step to the static validation step, but resources
are the reference type that most needs this immediately due to the
ambiguity caused by the instance indexing syntax. We can safely refactor
other reference types to be statically validated in later releases.

This is verified by two pre-existing context validate tests which we
temporarily disabled during earlier work (now re-enabled) and also by a
new validate test aimed specifically at the special case for the "count"
attribute.
2018-11-26 08:25:03 -08:00
Martin Atkins 479c6b2466 move "configschema" from "config" to "configs"
The "config" package is no longer used and will be removed as part
of the 0.12 release cleanup. Since configschema is part of the
"new world" of configuration modelling, it makes more sense for
it to live as a subdirectory of the newer "configs" package.
2018-10-16 18:50:29 -07:00
Martin Atkins c937c06a03 terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.

The three main goals here are:
- Use the configuration models from the "configs" package instead of the
  older models in the "config" package, which is now deprecated and
  preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
  new "lang" package, instead of the Interpolator type and related
  functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
  rather than hand-constructed strings. This is not critical to support
  the above, but was a big help during the implementation of these other
  points since it made it much more explicit what kind of address is
  expected in each context.

Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.

I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-10-16 18:46:46 -07:00
Martin Atkins a16ca2ec53 lang: new package for the runtime parts of the config language
Whereas package "configs" deals with the static structure of the
configuration language, this new package "lang" deals with the dynamic
aspects such as expression evaluation.

So far this mainly consists of populating a hcl.EvalContext that contains
the values necessary to evaluate a block or an expression. There is also
special handling here for dynamic block generation using the HCL
"dynblock" extension, which is exposed in the public interface (rather
than hiding it as an implementation detail of EvalBlock) so that the
caller can then extract proper source locations for any result values
using the expanded body.

This also includes the beginnings of a replacement for the function table
handling that currently lives in the old "config" package, but most of
the functions are not yet ported and so this will expand in subsequent
commits.
2018-10-16 18:44:26 -07:00