Commit Graph

22 Commits

Author SHA1 Message Date
Martin Atkins d76759a6a9 configs/configload: snapshotDir must be used via pointer
A snapshotDir tracks its current position as part of its state, so we need
to use it via pointer rather than value so that Readdirnames can actually
update that position, or else we'll just get stuck at position zero.

In practice this wasn't hurting anything because we only call Readdir once
on our snapshots, to read the whole directory at once. Still nice to fix
to avoid a gotcha for future maintenence, though.
2021-10-12 11:42:17 -07:00
Martin Atkins df578afd7e backend/local: Check dependency lock consistency before any operations
In historical versions of Terraform the responsibility to check this was
inside the terraform.NewContext function, along with various other
assorted concerns that made that function particularly complicated.

More recently, we reduced the responsibility of the "terraform" package
only to instantiating particular named plugins, assuming that its caller
is responsible for selecting appropriate versions of any providers that
_are_ external. However, until this commit we were just assuming that
"terraform init" had correctly selected appropriate plugins and recorded
them in the lock file, and so nothing was dealing with the problem of
ensuring that there haven't been any changes to the lock file or config
since the most recent "terraform init" which would cause us to need to
re-evaluate those decisions.

Part of the game here is to slightly extend the role of the dependency
locks object to also carry information about a subset of provider
addresses whose lock entries we're intentionally disregarding as part of
the various little edge-case features we have for overridding providers:
dev_overrides, "unmanaged providers", and the testing overrides in our
own unit tests. This is an in-memory-only annotation, never included in
the serialized plan files on disk.

I had originally intended to create a new package to encapsulate all of
this plugin-selection logic, including both the version constraint
checking here and also the handling of the provider factory functions, but
as an interim step I've just made version constraint consistency checks
the responsibility of the backend/local package, which means that we'll
always catch problems as part of preparing for local operations, while
not imposing these additional checks on commands that _don't_ run local
operations, such as "terraform apply" when in remote operations mode.
2021-10-01 14:43:58 -07:00
James Bardin a53faf43f6 return partial config from LoadConfig with errors
LoadConfig should return any parsed configuration in order for the
caller to verify `required_version`.
2021-09-28 13:30:03 -04:00
James Bardin 9029870613 handle NestedTypes in Block.CoerceValue
The CoerceValue code was not updated to handle NestedTypes, and while
none of the new codepaths make use of this method, there are still some
internal uses.
2021-09-14 16:02:50 -04:00
James Bardin 53a73a8ab6 configs: add ConstraintType to config.Variable
In order to handle optional attributes, the Variable type needs to keep
track of the type constraint for decoding and conversion, as well as the
concrete type for creating values and type comparison.

Since the Type field is referenced throughout the codebase, and for
future refactoring if the handling of optional attributes changes
significantly, the constraint is now loaded into an entirely new field
called ConstraintType. This prevents types containing
ObjectWithOptionalAttrs from escaping the decode/conversion codepaths
into the rest of the codebase.
2021-09-13 08:51:32 -04:00
James Bardin af4f4540a9 configschema: do not expose optional attributes
Objects with optional attributes are only used for the decoding of
HCL, and those types should never be exposed elsewhere within
terraform.

Separate the external ImpliedType method from the cty.Type generated
internally for the decoder spec. This unfortunately causes our
ImpliedType method to return a different type than the
hcldec.ImpliedType function, but the former is only used within
terraform for concrete values, while the latter is used to decode
HCL. Renaming the ImpliedType methods could be done to further
differentiate them, but that does cause fairly large diff in the
codebase that does not seem worth the effort at this time.
2021-09-13 08:51:32 -04:00
Alisdair McDiarmid 06956773dc configs: Disable experiment warnings at link time
This package level variable can be overridden at link time to allow
temporarily disabling the UI warning when experimental features are
enabled. This makes it easier to understand how UI will render when the
feature is no longer experimental.

This change is only for those developing Terraform.
2021-09-03 12:11:06 -04:00
James Bardin 863963e7a6 de-linting 2021-09-01 11:36:21 -04:00
Martin Atkins 383bbdeebc Upgrade to Go 1.17
This includes the addition of the new "//go:build" comment form in addition
to the legacy "// +build" notation, as produced by gofmt to ensure
consistent behavior between Go versions. The new directives are all
equivalent to what was present before, so there's no change in behavior.

Go 1.17 continues to use the Unicode 13 tables as in Go 1.16, so this
upgrade does not require also upgrading our Unicode-related dependencies.

This upgrade includes the following breaking changes which will also
appear as breaking changes for Terraform users, but that are consistent
with the Terraform v1.0 compatibility promises.

- On MacOS, Terraform now requires macOS 10.13 High Sierra or later.

This upgrade also includes the following breaking changes which will
appear as breaking changes for Terraform users that are inconsistent with
our compatibility promises, but have justified exceptions as follows:

- cidrsubnet, cidrhost, and cidrnetmask will now reject IPv4 CIDR
  addresses whose decimal components have leading zeros, where previously
  they would just silently ignore those leading zeros.

  This is a security-motivated exception to our compatibility promises,
  because some external systems interpret zero-prefixed octets as octal
  numbers rather than decimal, and thus the previous lenient parsing could
  lead to a different interpretation of the address between systems, and
  thus potentially allow bypassing policy when configuring firewall rules
  etc.

This upgrade also includes the following breaking changes which could
_potentially_ appear as breaking changes for Terraform users, but that do
not in practice for the reasons given:

- The Go net/url package no longer allows query strings with pairs
  separated by semicolons instead of ampersands. This primarily affects
  HTTP servers written in Go, and Terraform includes a special temporary
  HTTP server as part of its implementation of OAuth for "terraform login",
  but that server only needs to accept URLs created by Terraform itself
  and Terraform does not generate any URLs that would be rejected.
2021-08-17 15:20:05 -07:00
Kristin Laemmert 0729e9fdd7
configs/configschema: extend block.AttributeByPath to descend into Objects (#29222)
* configs/configschema: extend block.AttributeByPath to descend into Objects

This commit adds a recursive Object.AttributeByPath function which will step through Attributes with NestedTypes. If an Attribute without a NestedType is encountered while there is still more to the path, it will return nil.
2021-07-22 09:45:33 -04:00
James Bardin aaf03d3251 update test error for hclv2.10.1 2021-07-21 09:09:30 -04:00
Kristin Laemmert 0d80a74539
configs/configschema: fix missing "computed" attributes from NestedObject's ImpliedType (#29172)
* configs/configschema: fix missing "computed" attributes from NestedObject's ImpliedType

listOptionalAttrsFromObject was not including "computed" attributes in the list of optional object attributes. This is now fixed. I've also added some tests and fixed some panics and otherwise bad behavior when bad input is given. One natable change is in ImpliedType, which was panicking on an invalid nesting mode. The comment expressly states that it will return a result even when the schema is inconsistent, so I removed the panic and instead return an empty object.
2021-07-15 13:00:07 -04:00
Martin Atkins 6b8e103d6a configs: Include "moved" blocks when merging multiple files into a module
An earlier commit added logic to decode "moved" blocks and do static
validation of them. Here we now include that result also in modules
produced from those files, which we can then use in Terraform Core to
actually implement the moves.

This also places the feature behind an active experiment keyword called
config_driven_move. For now activating this doesn't actually achieve
anything except let you include moved blocks that Terraform will summarily
ignore, but we'll expand the scope of this in later commits to eventually
reach the point where it's really usable.
2021-07-01 08:28:02 -07:00
Martin Atkins d92b5e5f5e configs: valid-modules test ignores experimental features warning
A common source of churn when we're running experiments is that a module
that would otherwise be valid ends up generating a warning merely because
the experiment is active. That means we end up needing to shuffle the
test files around if the feature ultimately graduates to stable.

To reduce that churn in simple cases, we'll make an exception to disregard
the "Experiment is active" warning for any experiment that a module has
intentionally opted into, because those warnings are always expected and
not a cause for concern.

It's still possible to test those warnings explicitly using the
testdata/warning-files directory, if needed.
2021-07-01 08:28:02 -07:00
Martin Atkins 708003b035 configs: For Moved blocks, use addrs.MoveEndpoint instead of addrs.Target
Although addrs.Target can in principle capture the information we need to
represent move endpoints, it's semantically confusing because
addrs.Targetable uses addrs.Abs... types which are typically for absolute
addresses, but we were using them for relative addresses here.

We now have specialized address types for representing moves and probably
other things which have similar requirements later on. These types
largely communicate the same information in the end, but aim to do so in
a way that's explicit about which addresses are relative and which are
absolute, to make it less likely that we'd inadvertently misuse these
addresses.
2021-07-01 08:28:02 -07:00
James Bardin d9dfd451ea update to use typed sensitive marks 2021-06-25 12:49:07 -04:00
Kristin Laemmert 3acb5e2841
configs: add decodeMovedBlock behind a locked gate. (#28973)
This PR adds decoding for the upcoming "moved" blocks in configuration. This code is gated behind an experiment called EverythingIsAPlan, but the experiment is not registered as an active experiment, so it will never run (there is a test in place which will fail if the experiment is ever registered).

This also adds a new function to the Targetable interface, AddrType, to simplifying comparing two addrs.Targetable.

There is some validation missing still: this does not (yet) descend into resources to see if the actual resource types are the same (I've put this off in part because we will eventually need the provider schema to verify aliased resources, so I suspect this validation will have to happen later on).
2021-06-21 10:53:16 -04:00
Martin Atkins 51b0aee36c addrs: ModuleRegistryPackage for representing module registry packages
Previously we had a separation between ModuleSourceRemote and
ModulePackage as a way to represent within the type system that there's an
important difference between a module source address and a package address,
because module packages often contain multiple modules and so a
ModuleSourceRemote combines a ModulePackage with a subdirectory to
represent one specific module.

This commit applies that same strategy to ModuleSourceRegistry, creating
a new type ModuleRegistryPackage to represent the different sort of
package that we use for registry modules. Again, the main goal here is
to try to reflect the conceptual modelling more directly in the type
system so that we can more easily verify that uses of these different
address types are correct.

To make use of that, I've also lightly reworked initwd's module installer
to use addrs.ModuleRegistryPackage directly, instead of a string
representation thereof. This was in response to some earlier commits where
I found myself accidentally mixing up package addresses and source
addresses in the installRegistryModule method; with this new organization
those bugs would've been caught at compile time, rather than only at
unit and integration testing time.

While in the area anyway, I also took this opportunity to fix some
historical confusing names of fields in initwd.ModuleInstaller, to be
clearer that they are only for registry packages and not for all module
source address types.
2021-06-03 08:50:34 -07:00
Martin Atkins 17b766c3ea configs: EntersNewPackage methods for descendant modules
Now that we (in the previous commit) refactored how we deal with module
sources to do the parsing at config loading time rather than at module
installation time, we can expose a method to centralize the determination
for whether a particular module call (and its resulting Config object)
enters a new external package.

We don't use this for anything yet, but in later commits we will use this
for some cross-module features that are available only for modules
belonging to the same package, because we assume that modules grouped
together in a package can change together and thus it's okay to permit a
little more coupling of internal details in that case, which would not
be appropriate between modules that are versioned separately.
2021-06-03 08:50:34 -07:00
Martin Atkins 1a8da65314 Refactoring of module source addresses and module installation
It's been a long while since we gave close attention to the codepaths for
module source address parsing and external module package installation.
Due to their age, these codepaths often diverged from our modern practices
such as representing address types in the addrs package, and encapsulating
package installation details only in a particular location.

In particular, this refactor makes source address parsing a separate step
from module installation, which therefore makes the result of that parsing
available to other Terraform subsystems which work with the configuration
representation objects.

This also presented the opportunity to better encapsulate our use of
go-getter into a new package "getmodules" (echoing "getproviders"), which
is intended to be the only part of Terraform that directly interacts with
go-getter.

This is largely just a refactor of the existing functionality into a new
code organization, but there is one notable change in behavior here: the
source address parsing now happens during configuration loading rather
than module installation, which may cause errors about invalid addresses
to be returned in different situations than before. That counts as
backward compatible because we only promise to remain compatible with
configurations that are _valid_, which means that they can be initialized,
planned, and applied without any errors. This doesn't introduce any new
error cases, and instead just makes a pre-existing error case be detected
earlier.

Our module registry client is still using its own special module address
type from registry/regsrc for now, with a small shim from the new
addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also
rework the registry client to work with the new address type, but this
commit is already big enough as it is.
2021-06-03 08:50:34 -07:00
Martin Atkins e5f52e56f8 addrs: Rename DefaultRegistryHost to DefaultProviderRegistryHost
As the comment notes, this hostname is the default for provide source
addresses. We'll shortly be adding some address types to represent module
source addresses too, and so we'll also have DefaultModuleRegistryHost
for that situation.

(They'll actually both contain the the same hostname, but that's a
coincidence rather than a requirement.)
2021-06-03 08:50:34 -07:00
Martin Atkins 31349a9c3a Move configs/ to internal/configs/
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.

If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
2021-05-17 14:09:07 -07:00