Most of the time we're converting from HCL diagnostics to tfdiags as we
expose diagnostics directly from HCL, but occasionally we need to to the
reverse.
For example, our configs package uses hcl.Diagnostics by convention
because it's primarily working with HCL, but sometimes it interacts with
functions elsewhere (like in the "addrs" package) that return
tfdiags.Diagnostics, where they need to be adapted to return in an HCL
shape.
This should be used with some care because, similar to Diagnostics.ForRPC,
it forces immediate flattening of all of the diagnostics to a single
type and so can potentially lose internal tracking information that
appears in other tfdiags.Diagnostic information, such as the additional
metadata tracked in the ConsolidateWarnings result to allow later
appending to existing groups.
When warnings appear in isolation (not accompanied by an error) it's
reasonable to want to defer resolving them for a while because they are
not actually blocking immediate work.
However, our warning messages tend to be long by default in order to
include all of the necessary context to understand the implications of
the warning, and that can make them overwhelming when combined with other
output.
As a compromise, this adds a new CLI option -compact-warnings which is
supported for all the main operation commands and which uses a more
compact format to print out warnings as long as they aren't also
accompanied by errors.
The default remains unchanged except that the threshold for consolidating
warning messages is reduced to one so that we'll now only show one of
each distinct warning summary.
Full warning messages are always shown if there's at least one error
included in the diagnostic set too, because in that case the warning
message could contain additional context to help understand the error.
This detects when there are many warning diagnostics with the same summary
and consolidates some of them together into a single diagnostic in order
to make the resulting output less overwhelming when presented in CLI
output where other information is competing for attention with the
warnings.
This is not yet used anywhere. Usage of it will follow in a subsequent
commit.
Previously we were using the experimental HCL 2 repository, but now we'll
shift over to the v2 import path within the main HCL repository as part of
actually releasing HCL 2.0 as stable.
This is a mechanical search/replace to the new import paths. It also
switches to the v2.0.0 release of HCL, which includes some new code that
Terraform didn't previously have but should not change any behavior that
matters for Terraform's purposes.
For the moment the experimental HCL2 repository is still an indirect
dependency via terraform-config-inspect, so it remains in our go.sum and
vendor directories for the moment. Because terraform-config-inspect uses
a much smaller subset of the HCL2 functionality, this does still manage
to prune the vendor directory a little. A subsequent release of
terraform-config-inspect should allow us to completely remove that old
repository in a future commit.
HCL diagnostics relating to expression evaluation include these, so this
is really just to pass that information through our Terraform-specific
diagnostics interface.
Even if a provider doesn't indicate a specific attribute as the cause of
a resource operation error, we know the error relates to some aspect of
the resource, so we'll include that approximate information in the result
so that we don't produce user-hostile error messages with no context
whatsoever.
Later we can hopefully refine this to place the source range on the header
of the configuration block rather than on an empty part of the body, but
that'll require some more complex rework here and so for now we'll just
accept this as an interim state so that the user can at least figure out
which resource block the error is coming from.
When presenting an error that may be a PathError, the error's path is
usually relative to some other value. If the caller is able to express
that value (or, more often, a reference to it) in HCL syntax then this
method will produce a complete expression in the error message,
concatenating any path information from the error to the end of the given
prefix string.
The only contextual information returned by the plugins in a diagnostic
is an attribute path. In order to test the conversions, we need to be
able to extract the cty.Path from a diagnostic if it exists.
Because we gather together diagnostics from many different parts of the
codebase, the list often ends up being in a non-ideal order. Here we
define a partial ordering for diagnostics that should hopefully make them
easier to scan when many are present, by grouping together diagnostics
that are of the same severity and belong to the same file.
We use sort.Stable here because we have a partial order and so we need
to make sure that diagnostics that do not have a relative ordering will
remain in their original order.
This sorting is applied just in time before rendering the diagnostics
in command.Meta.showDiagnostics.
This is a variant of diagnosticsAsError that we use to signal to informed
callers that there might just be warnings inside, but we should also do
the right thing if a caller just appends it to an existing diagnostics
without checking first.
There is some existing practice in the "terraform" package of returning
a special error type ValidationError from EvalNode implementations in
order to return warnings without halting the graph walk even though a
non-nil error was returned.
This is a diagnostics-flavored version of that approach, allowing us to
avoid totally reworking the EvalNode concept around diagnostics and
retaining the ability to return non-fatal errors.
NonFatalErr is equivalent to the former terraform.ValidationError, while
ErrWithWarnings is a helper that automatically treats any errors as
fatal but returns NonFatalError if the diagnostics contains only warnings.
While diagnostics are primarily designed for reporting problems in
configuration, they can also be used for errors and warnings about the
environment Terraform is running in, such as inability to reach a remote
service, etc.
Function Sourceless makes it easy to produce such diagnostics with the
customary summary/detail structure. When these diagnostics are rendered
they will have no source code snippet and will instead just include the
English-language content.
cty doesn't have a native string representation of a cty.Path because it
is the layer below any particular syntax, but up here in Terraform land
we know that we use HCL native syntax and so we can format a string in
a HCL-ish way for familiarity to the user.
We'll use this form automatically when such an error is used directly as a
diagnostic, but we also expose the function publicly so that other code
that incorporates errors into diagnostic detail strings can apply the
same formatting.
The usual usage of diagnostics requires us to pass around source location
information to everywhere that might generate a diagnostic, and that is
always the best way to get the most precise diagnostic source locations.
However, it's impractical to require source location information to be
retained in every Terraform subsystem, and so this new idea of "contextual
diagnostics" allows us to separate the generation of a diagnostic from
the resolution of its source location, instead resolving the location
information as a post-processing step once the call stack unwinds to a
place where there is enough context to find it.
This is necessarily a less precise approach than reading the source ranges
directly from the configuration AST, but gives us an alternative to no
diagnostics at all in portions of Terraform where full location
information is not available.
This is a best-effort sort of thing which will get as precise as it can
but may return a range in a parent block if the precise location of a
particular attribute cannot be found. Diagnostics that rely on this
mechanism should include some other contextual information in the detail
message to make up for any loss of precision that results.
Previously we were showing only the summaries when converting to a string
error, but HCL generates summaries that indicate only the _type_ of error,
expecting that the detail will give the details, and so we need to show
both in order to produce a useful error message.
HCL will be the most frequent origin of a source range, so this helper
should make it easier to translate these between the two worlds so we
can more easily use them in native Terraform diagnostics and other
messaging.
Due to the use of interfaces, Diagnostics is not super-friendly to the gob
encoding we currently use for plugin RPC. To mitigate this, we provide
a helper that converts all of the wrapped objects into a predictable flat
structure that we can pre-emptively register with gob.
This means that the decoded Diagnostics still has the same meaning as
the original, though the original wrapped errors (if any) are lost and
thus our errwrap integration won't be effective any longer.
Currently we lean heavily on the Go error type as our primary means of
describing errors, and along with that use several more specialized
implementations of it in different spots for additional capabilities such
as multiple errors in one object, source code range references, etc.
We also have a rather ad-hoc approach of returning an array of warnings
from certain functions along with one or multiple errors.
This rather-disorganized approach makes it hard for us to present
user-facing error messages consistently. As a step towards mitigating
this, package tfdiags provides a model for user-facing error and warning
messages and helper functions for creating them from various other
error and warning types used elsewhere in Terraform.
This mechanism is intended to be used to report errors and warnings where
the audience is the Terraform user, and so it may go a few layers deep
down the call stack into codepaths like config parsing, interpolation, etc
but is primarily a UX concern. The deepest reaches of Terraform core will
continue using "error" as normal, with higher layers preparing error
messages for presentation to the user.
To avoid needing to change the interface of every function that might
generate error diagnostics, the Diagnostics type can be "smuggled" via
an error value through other APIs and then unwrapped at the other end,
though it will lose any naked warnings (without at least one error) along
the way, and so codepaths that are expected to generate warnings
(validation, primarily) should use the concrete Diagnostics type
throughout the call chain.
Martin Atkins