terraform/internal/configs/module_call.go

301 lines
9.5 KiB
Go
Raw Permalink Normal View History

package configs
import (
"fmt"
"github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/gohcl"
"github.com/hashicorp/hcl/v2/hclsyntax"
Refactoring of module source addresses and module installation It's been a long while since we gave close attention to the codepaths for module source address parsing and external module package installation. Due to their age, these codepaths often diverged from our modern practices such as representing address types in the addrs package, and encapsulating package installation details only in a particular location. In particular, this refactor makes source address parsing a separate step from module installation, which therefore makes the result of that parsing available to other Terraform subsystems which work with the configuration representation objects. This also presented the opportunity to better encapsulate our use of go-getter into a new package "getmodules" (echoing "getproviders"), which is intended to be the only part of Terraform that directly interacts with go-getter. This is largely just a refactor of the existing functionality into a new code organization, but there is one notable change in behavior here: the source address parsing now happens during configuration loading rather than module installation, which may cause errors about invalid addresses to be returned in different situations than before. That counts as backward compatible because we only promise to remain compatible with configurations that are _valid_, which means that they can be initialized, planned, and applied without any errors. This doesn't introduce any new error cases, and instead just makes a pre-existing error case be detected earlier. Our module registry client is still using its own special module address type from registry/regsrc for now, with a small shim from the new addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also rework the registry client to work with the new address type, but this commit is already big enough as it is.
2021-05-28 04:24:59 +02:00
"github.com/hashicorp/terraform/internal/addrs"
"github.com/hashicorp/terraform/internal/getmodules"
)
// ModuleCall represents a "module" block in a module or file.
type ModuleCall struct {
Name string
Refactoring of module source addresses and module installation It's been a long while since we gave close attention to the codepaths for module source address parsing and external module package installation. Due to their age, these codepaths often diverged from our modern practices such as representing address types in the addrs package, and encapsulating package installation details only in a particular location. In particular, this refactor makes source address parsing a separate step from module installation, which therefore makes the result of that parsing available to other Terraform subsystems which work with the configuration representation objects. This also presented the opportunity to better encapsulate our use of go-getter into a new package "getmodules" (echoing "getproviders"), which is intended to be the only part of Terraform that directly interacts with go-getter. This is largely just a refactor of the existing functionality into a new code organization, but there is one notable change in behavior here: the source address parsing now happens during configuration loading rather than module installation, which may cause errors about invalid addresses to be returned in different situations than before. That counts as backward compatible because we only promise to remain compatible with configurations that are _valid_, which means that they can be initialized, planned, and applied without any errors. This doesn't introduce any new error cases, and instead just makes a pre-existing error case be detected earlier. Our module registry client is still using its own special module address type from registry/regsrc for now, with a small shim from the new addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also rework the registry client to work with the new address type, but this commit is already big enough as it is.
2021-05-28 04:24:59 +02:00
SourceAddr addrs.ModuleSource
SourceAddrRaw string
SourceAddrRange hcl.Range
SourceSet bool
Config hcl.Body
Version VersionConstraint
Count hcl.Expression
ForEach hcl.Expression
Providers []PassedProviderConfig
DependsOn []hcl.Traversal
DeclRange hcl.Range
}
func decodeModuleBlock(block *hcl.Block, override bool) (*ModuleCall, hcl.Diagnostics) {
var diags hcl.Diagnostics
mc := &ModuleCall{
Name: block.Labels[0],
DeclRange: block.DefRange,
}
schema := moduleBlockSchema
if override {
schema = schemaForOverrides(schema)
}
content, remain, moreDiags := block.Body.PartialContent(schema)
diags = append(diags, moreDiags...)
mc.Config = remain
if !hclsyntax.ValidIdentifier(mc.Name) {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid module instance name",
Detail: badIdentifierDetail,
Subject: &block.LabelRanges[0],
})
}
haveVersionArg := false
if attr, exists := content.Attributes["version"]; exists {
var versionDiags hcl.Diagnostics
mc.Version, versionDiags = decodeVersionConstraint(attr)
diags = append(diags, versionDiags...)
haveVersionArg = true
}
if attr, exists := content.Attributes["source"]; exists {
mc.SourceSet = true
mc.SourceAddrRange = attr.Expr.Range()
Refactoring of module source addresses and module installation It's been a long while since we gave close attention to the codepaths for module source address parsing and external module package installation. Due to their age, these codepaths often diverged from our modern practices such as representing address types in the addrs package, and encapsulating package installation details only in a particular location. In particular, this refactor makes source address parsing a separate step from module installation, which therefore makes the result of that parsing available to other Terraform subsystems which work with the configuration representation objects. This also presented the opportunity to better encapsulate our use of go-getter into a new package "getmodules" (echoing "getproviders"), which is intended to be the only part of Terraform that directly interacts with go-getter. This is largely just a refactor of the existing functionality into a new code organization, but there is one notable change in behavior here: the source address parsing now happens during configuration loading rather than module installation, which may cause errors about invalid addresses to be returned in different situations than before. That counts as backward compatible because we only promise to remain compatible with configurations that are _valid_, which means that they can be initialized, planned, and applied without any errors. This doesn't introduce any new error cases, and instead just makes a pre-existing error case be detected earlier. Our module registry client is still using its own special module address type from registry/regsrc for now, with a small shim from the new addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also rework the registry client to work with the new address type, but this commit is already big enough as it is.
2021-05-28 04:24:59 +02:00
valDiags := gohcl.DecodeExpression(attr.Expr, nil, &mc.SourceAddrRaw)
diags = append(diags, valDiags...)
if !valDiags.HasErrors() {
var addr addrs.ModuleSource
var err error
if haveVersionArg {
addr, err = addrs.ParseModuleSourceRegistry(mc.SourceAddrRaw)
} else {
addr, err = addrs.ParseModuleSource(mc.SourceAddrRaw)
}
mc.SourceAddr = addr
if err != nil {
// NOTE: We leave mc.SourceAddr as nil for any situation where the
// source attribute is invalid, so any code which tries to carefully
// use the partial result of a failed config decode must be
// resilient to that.
mc.SourceAddr = nil
// NOTE: In practice it's actually very unlikely to end up here,
// because our source address parser can turn just about any string
// into some sort of remote package address, and so for most errors
// we'll detect them only during module installation. There are
// still a _few_ purely-syntax errors we can catch at parsing time,
// though, mostly related to remote package sub-paths and local
// paths.
switch err := err.(type) {
case *getmodules.MaybeRelativePathErr:
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid module source address",
Detail: fmt.Sprintf(
"Terraform failed to determine your intended installation method for remote module package %q.\n\nIf you intended this as a path relative to the current module, use \"./%s\" instead. The \"./\" prefix indicates that the address is a relative filesystem path.",
err.Addr, err.Addr,
),
Subject: mc.SourceAddrRange.Ptr(),
})
default:
if haveVersionArg {
// In this case we'll include some extra context that
// we assumed a registry source address due to the
// version argument.
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid registry module source address",
Detail: fmt.Sprintf("Failed to parse module registry address: %s.\n\nTerraform assumed that you intended a module registry source address because you also set the argument \"version\", which applies only to registry modules.", err),
Subject: mc.SourceAddrRange.Ptr(),
})
} else {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid module source address",
Detail: fmt.Sprintf("Failed to parse module source address: %s.", err),
Subject: mc.SourceAddrRange.Ptr(),
})
}
}
Refactoring of module source addresses and module installation It's been a long while since we gave close attention to the codepaths for module source address parsing and external module package installation. Due to their age, these codepaths often diverged from our modern practices such as representing address types in the addrs package, and encapsulating package installation details only in a particular location. In particular, this refactor makes source address parsing a separate step from module installation, which therefore makes the result of that parsing available to other Terraform subsystems which work with the configuration representation objects. This also presented the opportunity to better encapsulate our use of go-getter into a new package "getmodules" (echoing "getproviders"), which is intended to be the only part of Terraform that directly interacts with go-getter. This is largely just a refactor of the existing functionality into a new code organization, but there is one notable change in behavior here: the source address parsing now happens during configuration loading rather than module installation, which may cause errors about invalid addresses to be returned in different situations than before. That counts as backward compatible because we only promise to remain compatible with configurations that are _valid_, which means that they can be initialized, planned, and applied without any errors. This doesn't introduce any new error cases, and instead just makes a pre-existing error case be detected earlier. Our module registry client is still using its own special module address type from registry/regsrc for now, with a small shim from the new addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also rework the registry client to work with the new address type, but this commit is already big enough as it is.
2021-05-28 04:24:59 +02:00
}
}
}
if attr, exists := content.Attributes["count"]; exists {
mc.Count = attr.Expr
}
if attr, exists := content.Attributes["for_each"]; exists {
2020-04-07 20:18:08 +02:00
if mc.Count != nil {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: `Invalid combination of "count" and "for_each"`,
Detail: `The "count" and "for_each" meta-arguments are mutually-exclusive, only one should be used to be explicit about the number of resources to be created.`,
Subject: &attr.NameRange,
})
}
mc.ForEach = attr.Expr
}
if attr, exists := content.Attributes["depends_on"]; exists {
deps, depsDiags := decodeDependsOn(attr)
diags = append(diags, depsDiags...)
mc.DependsOn = append(mc.DependsOn, deps...)
}
if attr, exists := content.Attributes["providers"]; exists {
seen := make(map[string]hcl.Range)
pairs, pDiags := hcl.ExprMap(attr.Expr)
diags = append(diags, pDiags...)
for _, pair := range pairs {
key, keyDiags := decodeProviderConfigRef(pair.Key, "providers")
diags = append(diags, keyDiags...)
value, valueDiags := decodeProviderConfigRef(pair.Value, "providers")
diags = append(diags, valueDiags...)
if keyDiags.HasErrors() || valueDiags.HasErrors() {
continue
}
matchKey := key.String()
if prev, exists := seen[matchKey]; exists {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Duplicate provider address",
Detail: fmt.Sprintf("A provider configuration was already passed to %s at %s. Each child provider configuration can be assigned only once.", matchKey, prev),
Subject: pair.Value.Range().Ptr(),
})
continue
}
rng := hcl.RangeBetween(pair.Key.Range(), pair.Value.Range())
seen[matchKey] = rng
mc.Providers = append(mc.Providers, PassedProviderConfig{
InChild: key,
InParent: value,
})
}
}
configs: Meta-argument escaping blocks Several top-level block types in the Terraform language have a body where two different schemas are overlayed on top of one another: Terraform first looks for "meta-arguments" that are built into the language, and then evaluates all of the remaining arguments against some externally-defined schema whose content is not fully controlled by Terraform. So far we've been cautiously adding new meta-arguments in these namespaces after research shows us that there are relatively few existing providers or modules that would have functionality masked by those additions, but that isn't really a viable path forward as we prepare to make stronger compatibility promises. In an earlier commit we've introduced the foundational parts of a new language versioning mechanism called "editions" which should allow us to make per-module-opt-in breaking changes in the future, but these shared namespaces remain a liability because it would be annoying if adopting a new edition made it impossible to use a feature of a third-party provider or module that was already using a name that has now become reserved in the new edition. This commit introduces a new syntax intended to be a rarely-used escape hatch for that situation. When we're designing new editions we will do our best to choose names that don't conflict with commonly-used providers and modules, but there are many providers and modules that we cannot see and so there is a risk that any name we might choose could collide with at least one existing provider or module. The automatic migration tool to upgrade an existing module to a new edition should therefore detect that situation and make use of this escaping block syntax in order to retain the existing functionality until all the called providers or modules are updated to no longer use conflicting names. Although we can't put in technical constraints on using this feature for other purposes (because we don't know yet what future editions will add), this mechanism is intentionally not documented for now because it serves no immediate purpose. In effect, this change is just squatting on the syntax of a special block type named "_" so that later editions can make use of it without it _also_ conflicting, creating a confusing nested escaping situation. However, the first time a new edition actually makes use of this syntax we should then document alongside the meta-arguments so folks can understand the meaning of escaping blocks produced by edition upgrade tools.
2021-05-15 01:09:51 +02:00
var seenEscapeBlock *hcl.Block
for _, block := range content.Blocks {
configs: Meta-argument escaping blocks Several top-level block types in the Terraform language have a body where two different schemas are overlayed on top of one another: Terraform first looks for "meta-arguments" that are built into the language, and then evaluates all of the remaining arguments against some externally-defined schema whose content is not fully controlled by Terraform. So far we've been cautiously adding new meta-arguments in these namespaces after research shows us that there are relatively few existing providers or modules that would have functionality masked by those additions, but that isn't really a viable path forward as we prepare to make stronger compatibility promises. In an earlier commit we've introduced the foundational parts of a new language versioning mechanism called "editions" which should allow us to make per-module-opt-in breaking changes in the future, but these shared namespaces remain a liability because it would be annoying if adopting a new edition made it impossible to use a feature of a third-party provider or module that was already using a name that has now become reserved in the new edition. This commit introduces a new syntax intended to be a rarely-used escape hatch for that situation. When we're designing new editions we will do our best to choose names that don't conflict with commonly-used providers and modules, but there are many providers and modules that we cannot see and so there is a risk that any name we might choose could collide with at least one existing provider or module. The automatic migration tool to upgrade an existing module to a new edition should therefore detect that situation and make use of this escaping block syntax in order to retain the existing functionality until all the called providers or modules are updated to no longer use conflicting names. Although we can't put in technical constraints on using this feature for other purposes (because we don't know yet what future editions will add), this mechanism is intentionally not documented for now because it serves no immediate purpose. In effect, this change is just squatting on the syntax of a special block type named "_" so that later editions can make use of it without it _also_ conflicting, creating a confusing nested escaping situation. However, the first time a new edition actually makes use of this syntax we should then document alongside the meta-arguments so folks can understand the meaning of escaping blocks produced by edition upgrade tools.
2021-05-15 01:09:51 +02:00
switch block.Type {
case "_":
if seenEscapeBlock != nil {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Duplicate escaping block",
Detail: fmt.Sprintf(
"The special block type \"_\" can be used to force particular arguments to be interpreted as module input variables rather than as meta-arguments, but each module block can have only one such block. The first escaping block was at %s.",
seenEscapeBlock.DefRange,
),
Subject: &block.DefRange,
})
continue
}
seenEscapeBlock = block
// When there's an escaping block its content merges with the
// existing config we extracted earlier, so later decoding
// will see a blend of both.
mc.Config = hcl.MergeBodies([]hcl.Body{mc.Config, block.Body})
default:
// All of the other block types in our schema are reserved.
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Reserved block type name in module block",
Detail: fmt.Sprintf("The block type name %q is reserved for use by Terraform in a future version.", block.Type),
Subject: &block.TypeRange,
})
}
}
return mc, diags
}
// EntersNewPackage returns true if this call is to an external module, either
// directly via a remote source address or indirectly via a registry source
// address.
//
// Other behaviors in Terraform may treat package crossings as a special
// situation, because that indicates that the caller and callee can change
// independently of one another and thus we should disallow using any features
// where the caller assumes anything about the callee other than its input
// variables, required provider configurations, and output values.
func (mc *ModuleCall) EntersNewPackage() bool {
return moduleSourceAddrEntersNewPackage(mc.SourceAddr)
}
// PassedProviderConfig represents a provider config explicitly passed down to
// a child module, possibly giving it a new local address in the process.
type PassedProviderConfig struct {
InChild *ProviderConfigRef
InParent *ProviderConfigRef
}
var moduleBlockSchema = &hcl.BodySchema{
Attributes: []hcl.AttributeSchema{
{
Name: "source",
Required: true,
},
{
Name: "version",
},
{
Name: "count",
},
{
Name: "for_each",
},
{
Name: "depends_on",
},
{
Name: "providers",
},
},
Blocks: []hcl.BlockHeaderSchema{
configs: Meta-argument escaping blocks Several top-level block types in the Terraform language have a body where two different schemas are overlayed on top of one another: Terraform first looks for "meta-arguments" that are built into the language, and then evaluates all of the remaining arguments against some externally-defined schema whose content is not fully controlled by Terraform. So far we've been cautiously adding new meta-arguments in these namespaces after research shows us that there are relatively few existing providers or modules that would have functionality masked by those additions, but that isn't really a viable path forward as we prepare to make stronger compatibility promises. In an earlier commit we've introduced the foundational parts of a new language versioning mechanism called "editions" which should allow us to make per-module-opt-in breaking changes in the future, but these shared namespaces remain a liability because it would be annoying if adopting a new edition made it impossible to use a feature of a third-party provider or module that was already using a name that has now become reserved in the new edition. This commit introduces a new syntax intended to be a rarely-used escape hatch for that situation. When we're designing new editions we will do our best to choose names that don't conflict with commonly-used providers and modules, but there are many providers and modules that we cannot see and so there is a risk that any name we might choose could collide with at least one existing provider or module. The automatic migration tool to upgrade an existing module to a new edition should therefore detect that situation and make use of this escaping block syntax in order to retain the existing functionality until all the called providers or modules are updated to no longer use conflicting names. Although we can't put in technical constraints on using this feature for other purposes (because we don't know yet what future editions will add), this mechanism is intentionally not documented for now because it serves no immediate purpose. In effect, this change is just squatting on the syntax of a special block type named "_" so that later editions can make use of it without it _also_ conflicting, creating a confusing nested escaping situation. However, the first time a new edition actually makes use of this syntax we should then document alongside the meta-arguments so folks can understand the meaning of escaping blocks produced by edition upgrade tools.
2021-05-15 01:09:51 +02:00
{Type: "_"}, // meta-argument escaping block
// These are all reserved for future use.
{Type: "lifecycle"},
{Type: "locals"},
{Type: "provider", LabelNames: []string{"type"}},
},
}
func moduleSourceAddrEntersNewPackage(addr addrs.ModuleSource) bool {
switch addr.(type) {
case nil:
// There are only two situations where we should get here:
// - We've been asked about the source address of the root module,
// which is always nil.
// - We've been asked about a ModuleCall that is part of the partial
// result of a failed decode.
// The root module exists outside of all module packages, so we'll
// just return false for that case. For the error case it doesn't
// really matter what we return as long as we don't panic, because
// we only make a best-effort to allow careful inspection of objects
// representing invalid configuration.
return false
case addrs.ModuleSourceLocal:
// Local source addresses are the only address type that remains within
// the same package.
return false
default:
// All other address types enter a new package.
return true
}
}