terraform/internal/earlyconfig/config.go

211 lines
7.2 KiB
Go
Raw Normal View History

package earlyconfig
import (
"fmt"
"sort"
version "github.com/hashicorp/go-version"
"github.com/hashicorp/terraform-config-inspect/tfconfig"
"github.com/hashicorp/terraform/internal/addrs"
internal/getproviders: A new shared model for provider requirements We've been using the models from the "moduledeps" package to represent our provider dependencies everywhere since the idea of provider dependencies was introduced in Terraform 0.10, but that model is not convenient to use for any use-case other than the "terraform providers" command that needs individual-module-level detail. To make things easier for new codepaths working with the new-style provider installer, here we introduce a new model type getproviders.Requirements which is based on the type the new installer was already taking as its input. We have new methods in the states, configs, and earlyconfig packages to produce values of this type, and a helper to merge Requirements together so we can combine config-derived and state-derived requirements together during installation. The advantage of this new model over the moduledeps one is that all of recursive module walking is done up front and we produce a simple, flat structure that is more convenient for the main use-cases of selecting providers for installation and then finding providers in the local cache to use them for other operations. This new model is _not_ suitable for implementing "terraform providers" because it does not retain module-specific requirement details. Therefore we will likely keep using moduledeps for "terraform providers" for now, and then possibly at a later time consider specializing the moduledeps logic for only what "terraform providers" needs, because it seems to be the only use-case that needs to retain that level of detail.
2020-03-26 20:04:48 +01:00
"github.com/hashicorp/terraform/internal/getproviders"
"github.com/hashicorp/terraform/internal/moduledeps"
"github.com/hashicorp/terraform/internal/plugin/discovery"
"github.com/hashicorp/terraform/internal/tfdiags"
)
// A Config is a node in the tree of modules within a configuration.
//
// The module tree is constructed by following ModuleCall instances recursively
// through the root module transitively into descendent modules.
type Config struct {
// RootModule points to the Config for the root module within the same
// module tree as this module. If this module _is_ the root module then
// this is self-referential.
Root *Config
// ParentModule points to the Config for the module that directly calls
// this module. If this is the root module then this field is nil.
Parent *Config
// Path is a sequence of module logical names that traverse from the root
// module to this config. Path is empty for the root module.
//
// This should only be used to display paths to the end-user in rare cases
// where we are talking about the static module tree, before module calls
// have been resolved. In most cases, an addrs.ModuleInstance describing
// a node in the dynamic module tree is better, since it will then include
// any keys resulting from evaluating "count" and "for_each" arguments.
Path addrs.Module
// ChildModules points to the Config for each of the direct child modules
// called from this module. The keys in this map match the keys in
// Module.ModuleCalls.
Children map[string]*Config
// Module points to the object describing the configuration for the
// various elements (variables, resources, etc) defined by this module.
Module *tfconfig.Module
// CallPos is the source position for the header of the module block that
// requested this module.
//
// This field is meaningless for the root module, where its contents are undefined.
CallPos tfconfig.SourcePos
// SourceAddr is the source address that the referenced module was requested
// from, as specified in configuration.
//
// This field is meaningless for the root module, where its contents are undefined.
SourceAddr string
// Version is the specific version that was selected for this module,
// based on version constraints given in configuration.
//
// This field is nil if the module was loaded from a non-registry source,
// since versions are not supported for other sources.
//
// This field is meaningless for the root module, where it will always
// be nil.
Version *version.Version
}
internal/getproviders: A new shared model for provider requirements We've been using the models from the "moduledeps" package to represent our provider dependencies everywhere since the idea of provider dependencies was introduced in Terraform 0.10, but that model is not convenient to use for any use-case other than the "terraform providers" command that needs individual-module-level detail. To make things easier for new codepaths working with the new-style provider installer, here we introduce a new model type getproviders.Requirements which is based on the type the new installer was already taking as its input. We have new methods in the states, configs, and earlyconfig packages to produce values of this type, and a helper to merge Requirements together so we can combine config-derived and state-derived requirements together during installation. The advantage of this new model over the moduledeps one is that all of recursive module walking is done up front and we produce a simple, flat structure that is more convenient for the main use-cases of selecting providers for installation and then finding providers in the local cache to use them for other operations. This new model is _not_ suitable for implementing "terraform providers" because it does not retain module-specific requirement details. Therefore we will likely keep using moduledeps for "terraform providers" for now, and then possibly at a later time consider specializing the moduledeps logic for only what "terraform providers" needs, because it seems to be the only use-case that needs to retain that level of detail.
2020-03-26 20:04:48 +01:00
// ProviderRequirements searches the full tree of modules under the receiver
// for both explicit and implicit dependencies on providers.
//
// The result is a full manifest of all of the providers that must be available
// in order to work with the receiving configuration.
//
// If the returned diagnostics includes errors then the resulting Requirements
// may be incomplete.
func (c *Config) ProviderRequirements() (getproviders.Requirements, tfdiags.Diagnostics) {
reqs := make(getproviders.Requirements)
diags := c.addProviderRequirements(reqs)
return reqs, diags
}
// addProviderRequirements is the main part of the ProviderRequirements
// implementation, gradually mutating a shared requirements object to
// eventually return.
func (c *Config) addProviderRequirements(reqs getproviders.Requirements) tfdiags.Diagnostics {
var diags tfdiags.Diagnostics
// First we'll deal with the requirements directly in _our_ module...
for localName, providerReqs := range c.Module.RequiredProviders {
var fqn addrs.Provider
if source := providerReqs.Source; source != "" {
addr, moreDiags := addrs.ParseProviderSourceString(source)
if moreDiags.HasErrors() {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Invalid provider source address",
fmt.Sprintf("Invalid source %q for provider %q in %s", source, localName, c.Path),
))
continue
}
fqn = addr
}
if fqn.IsZero() {
fqn = addrs.ImpliedProviderForUnqualifiedType(localName)
internal/getproviders: A new shared model for provider requirements We've been using the models from the "moduledeps" package to represent our provider dependencies everywhere since the idea of provider dependencies was introduced in Terraform 0.10, but that model is not convenient to use for any use-case other than the "terraform providers" command that needs individual-module-level detail. To make things easier for new codepaths working with the new-style provider installer, here we introduce a new model type getproviders.Requirements which is based on the type the new installer was already taking as its input. We have new methods in the states, configs, and earlyconfig packages to produce values of this type, and a helper to merge Requirements together so we can combine config-derived and state-derived requirements together during installation. The advantage of this new model over the moduledeps one is that all of recursive module walking is done up front and we produce a simple, flat structure that is more convenient for the main use-cases of selecting providers for installation and then finding providers in the local cache to use them for other operations. This new model is _not_ suitable for implementing "terraform providers" because it does not retain module-specific requirement details. Therefore we will likely keep using moduledeps for "terraform providers" for now, and then possibly at a later time consider specializing the moduledeps logic for only what "terraform providers" needs, because it seems to be the only use-case that needs to retain that level of detail.
2020-03-26 20:04:48 +01:00
}
if _, ok := reqs[fqn]; !ok {
// We'll at least have an unconstrained dependency then, but might
// add to this in the loop below.
reqs[fqn] = nil
}
for _, constraintsStr := range providerReqs.VersionConstraints {
if constraintsStr != "" {
constraints, err := getproviders.ParseVersionConstraints(constraintsStr)
if err != nil {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Invalid provider version constraint",
fmt.Sprintf("Provider %q in %s has invalid version constraint %q: %s.", localName, c.Path, constraintsStr, err),
))
continue
}
reqs[fqn] = append(reqs[fqn], constraints...)
}
}
}
// ...and now we'll recursively visit all of the child modules to merge
// in their requirements too.
for _, childConfig := range c.Children {
moreDiags := childConfig.addProviderRequirements(reqs)
diags = diags.Append(moreDiags)
}
return diags
}
// ProviderDependencies is a deprecated variant of ProviderRequirements which
// uses the moduledeps models for representation. This is preserved to allow
// a gradual transition over to ProviderRequirements, but note that its
// support for fully-qualified provider addresses has some idiosyncracies.
func (c *Config) ProviderDependencies() (*moduledeps.Module, tfdiags.Diagnostics) {
var diags tfdiags.Diagnostics
var name string
if len(c.Path) > 0 {
name = c.Path[len(c.Path)-1]
}
ret := &moduledeps.Module{
Name: name,
}
providers := make(moduledeps.Providers)
for name, reqs := range c.Module.RequiredProviders {
addrs: Stronger validation and normalization of provider namespace/type The provider FQN is becoming our primary identifier for a provider, so it's important that we are clear about the equality rules for these addresses and what characters are valid within them. We previously had a basic regex permitting ASCII letters and digits for validation and no normalization at all. We need to do at least case folding and UTF-8 normalization because these names will appear in file and directory names in case-insensitive filesystems and in repository names such as on GitHub. Since we're already using DNS-style normalization and validation rules for the hostname part, rather than defining an entirely new set of rules here we'll just treat the provider namespace and type as if they were single labels in a DNS name. Aside from some internal consistency, that also works out nicely because systems like GitHub use organization and repository names as part of hostnames (e.g. with GitHub Pages) and so tend to apply comparable constraints themselves. This introduces the possibility of names containing letters from alphabets other than the latin alphabet, and for latin letters with diacritics. That's consistent with our introduction of similar support for identifiers in the language in Terraform 0.12, and is intended to be more friendly to Terraform users throughout the world that might prefer to name their products using a different alphabet. This is also a further justification for using the DNS normalization rules: modern companies tend to choose product names that make good domain names, and now such names will be usable as Terraform provider names too.
2020-02-15 03:10:03 +01:00
var fqn addrs.Provider
if source := reqs.Source; source != "" {
2020-12-01 22:27:29 +01:00
addr, parseDiags := addrs.ParseProviderSourceString(source)
if parseDiags.HasErrors() {
addrs: Stronger validation and normalization of provider namespace/type The provider FQN is becoming our primary identifier for a provider, so it's important that we are clear about the equality rules for these addresses and what characters are valid within them. We previously had a basic regex permitting ASCII letters and digits for validation and no normalization at all. We need to do at least case folding and UTF-8 normalization because these names will appear in file and directory names in case-insensitive filesystems and in repository names such as on GitHub. Since we're already using DNS-style normalization and validation rules for the hostname part, rather than defining an entirely new set of rules here we'll just treat the provider namespace and type as if they were single labels in a DNS name. Aside from some internal consistency, that also works out nicely because systems like GitHub use organization and repository names as part of hostnames (e.g. with GitHub Pages) and so tend to apply comparable constraints themselves. This introduces the possibility of names containing letters from alphabets other than the latin alphabet, and for latin letters with diacritics. That's consistent with our introduction of similar support for identifiers in the language in Terraform 0.12, and is intended to be more friendly to Terraform users throughout the world that might prefer to name their products using a different alphabet. This is also a further justification for using the DNS normalization rules: modern companies tend to choose product names that make good domain names, and now such names will be usable as Terraform provider names too.
2020-02-15 03:10:03 +01:00
diags = diags.Append(wrapDiagnostic(tfconfig.Diagnostic{
Severity: tfconfig.DiagError,
Summary: "Invalid provider source",
Detail: fmt.Sprintf("Invalid source %q for provider", name),
}))
continue
}
fqn = addr
}
if fqn.IsZero() {
2020-10-05 14:33:49 +02:00
fqn = addrs.NewDefaultProvider(name)
}
var constraints version.Constraints
for _, reqStr := range reqs.VersionConstraints {
command: "terraform init" can partially initialize for 0.12upgrade There are a few constructs from 0.11 and prior that cause 0.12 parsing to fail altogether, which previously created a chicken/egg problem because we need to install the providers in order to run "terraform 0.12upgrade" and thus fix the problem. This changes "terraform init" to use the new "early configuration" loader for module and provider installation. This is built on the more permissive parser in the terraform-config-inspect package, and so it allows us to read out the top-level blocks from the configuration while accepting legacy HCL syntax. In the long run this will let us do version compatibility detection before attempting a "real" config load, giving us better error messages for any future syntax additions, but in the short term the key thing is that it allows us to install the dependencies even if the configuration isn't fully valid. Because backend init still requires full configuration, this introduces a new mode of terraform init where it detects heuristically if it seems like we need to do a configuration upgrade and does a partial init if so, before finally directing the user to run "terraform 0.12upgrade" before running any other commands. The heuristic here is based on two assumptions: - If the "early" loader finds no errors but the normal loader does, the configuration is likely to be valid for Terraform 0.11 but not 0.12. - If there's already a version constraint in the configuration that excludes Terraform versions prior to v0.12 then the configuration is probably _already_ upgraded and so it's just a normal syntax error, even if the early loader didn't detect it. Once the upgrade process is removed in 0.13.0 (users will be required to go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can be simplified to remove that special mode, but the idea of doing the dependency version checks against the liberal parser will remain valuable to increase our chances of reporting version-based incompatibilities rather than syntax errors as we add new features in future.
2019-01-14 20:11:00 +01:00
if reqStr != "" {
constraint, err := version.NewConstraint(reqStr)
if err != nil {
diags = diags.Append(wrapDiagnostic(tfconfig.Diagnostic{
Severity: tfconfig.DiagError,
Summary: "Invalid provider version constraint",
2020-10-05 14:33:49 +02:00
Detail: fmt.Sprintf("Invalid version constraint %q for provider %s.", reqStr, fqn.String()),
}))
continue
}
constraints = append(constraints, constraint...)
}
}
providers[fqn] = moduledeps.ProviderDependency{
Constraints: discovery.NewConstraints(constraints),
Reason: moduledeps.ProviderDependencyExplicit,
}
}
ret.Providers = providers
childNames := make([]string, 0, len(c.Children))
for name := range c.Children {
childNames = append(childNames, name)
}
sort.Strings(childNames)
for _, name := range childNames {
child, childDiags := c.Children[name].ProviderDependencies()
ret.Children = append(ret.Children, child)
diags = diags.Append(childDiags)
}
return ret, diags
}