resilien
/
terraform
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

website: Generalized advice on modules with provider configs 

As part of documenting the new module for_each capabilities we added a
section noting that shared modules using the legacy pattern of declaring
their own provider configurations would not be compatible with them.

However, that also applies to the new module depends_on and several folks
participating in the beta pointed out that the documentation wasn't
discussing that at all.

In order to generalize the advice, I've moved the old content we had
(since v0.11) recommending against provider configurations in shared
modules out into its own section, now being more explicit that it is
a legacy pattern and not recommended, and then folded the content about
for_each and count, now also including depends_on, into that expanded
section.

As is often the case, that had some knock-on effects on the content on
the rest of this page, so there's some general editing and reorganization
here. In particular, I moved the "Multiple Instances of a Module" section
much further up the page because it's content relevant to users of
shared modules, while the later content on this page is more aimed at
authors of shared modules, including the new section about the legacy
pattern.
This commit is contained in:
Martin Atkins 2020-07-24 12:25:34 -07:00
parent c2d6276238
commit ef071f3d0e
1 changed files with 186 additions and 117 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

303
website/docs/configuration/modules.html.md
View File

@ -128,6 +128,72 @@ modules sourced from local file paths do not support `version`; since
they're loaded from the same source repository, they always share the same
version as their caller.
## Multiple Instances of a Module
Use the `for_each` or the `count` argument to create multiple instances of a
module from a single `module` block. These arguments have the same syntax and
type constraints as
[`for_each`](./resources.html#for_each-multiple-resource-instances-defined-by-a-map-or-set-of-strings)
and
[`count`](./resources.html#count-multiple-resource-instances-by-count)
when used with resources.
```hcl
# my_buckets.tf
module "bucket" {
for_each = toset(["assets", "media"])
source = "./publish_bucket"
name = "${each.key}_bucket"
}
```
```hcl
# publish_bucket/bucket-and-cloudfront.tf
variable "name" {} # this is the input parameter of the module
resource "aws_s3_bucket" "example" {
# Because var.name includes each.key in the calling
# module block, its value will be different for
# each instance of this module.
bucket = var.name
# ...
}
resource "aws_iam_user" "deploy_user" {
# ...
}
```
This example defines a local child module in the `./publish_bucket`
subdirectory. That module has configuration to create an S3 bucket. The module
wraps the bucket and all the other implementation details required to configure
a bucket.
We declare multiple module instances by using the `for_each` attribute,
which accepts a map (with string keys) or a set of strings as its value. Additionally,
we use the `each.key` in our module block, because the
[`each`](/docs/configuration/resources.html#the-each-object) object is available when
we have declared `for_each` on the module block. When using the `count` argument, the
[`count`](/docs/configuration/resources.html#the-count-object) object is available.
Resources from child modules are prefixed with `module.module_name[module index]`
when displayed in plan output and elsewhere in the UI. For a module with without
`count` or `for_each`, the address will not contain the module index as the module's
name suffices to reference the module.
In our example, the `./publish_bucket` module contains `aws_s3_bucket.example`, and so the two
instances of this module produce S3 bucket resources with [resource addresses](/docs/internals/resource-addressing.html) of `module.bucket["assets"].aws_s3_bucket.example`
and `module.bucket["media"].aws_s3_bucket.example` respectively. These full addresses
are used within the UI and on the command line, but only [outputs](docs/configuration/outputs.html)
from a module can be referenced from elsewhere in your configuration.
When refactoring an existing configuration to introduce modules, moving
resource blocks between modules causes Terraform to see the new location
as an entirely separate resource to the old. Always check the execution plan
after performing such actions to ensure that no resources are surprisingly
deleted.
## Other Meta-arguments
Along with the `source` meta-argument described above, module blocks have
@ -161,56 +227,83 @@ about how modules can be used, created, and published is included in
In a configuration with multiple modules, there are some special considerations
for how resources are associated with provider configurations.
While in principle `provider` blocks can appear in any module, it is recommended
that they be placed only in the _root_ module of a configuration, since this
approach allows users to configure providers just once and re-use them across
all descendent modules.
Each resource in the configuration must be associated with one provider
configuration, which may either be within the same module as the resource
or be passed from the parent module. Providers can be passed down to descendent
modules in two ways: either _implicitly_ through inheritance, or _explicitly_
via the `providers` argument within a `module` block. These two options are
discussed in more detail in the following sections.
configuration. Provider configurations, unlike most other concepts in
Terraform, are global to an entire Terraform configuration and can be shared
across module boundaries. Provider configurations can be defined only in a
root Terraform module.
In all cases it is recommended to keep explicit provider configurations only in
the root module and pass them (whether implicitly or explicitly) down to
descendent modules. This avoids the provider configurations from being "lost"
when descendent modules are removed from the configuration. It also allows
the user of a configuration to determine which providers require credentials
by inspecting only the root module.
Providers can be passed down to descendent modules in two ways: either
_implicitly_ through inheritance, or _explicitly_ via the `providers` argument
within a `module` block. These two options are discussed in more detail in the
following sections.
A module intended to be called by one or more other modules must not contain
any `provider` blocks, with the exception of the special
"proxy provider blocks" discussed under
_[Passing Providers Explicitly](#passing-providers-explicitly)_
below.
For backward compatibility with configurations targeting Terraform v0.10 and
earlier Terraform does not produce an error for a `provider` block in a shared
module if the `module` block only uses features available in Terraform v0.10,
but that is a legacy usage pattern that is no longer recommended and a legacy
module containing its own provider configurations is not compatible with the
`for_each`, `count`, and `depends_on` arguments that were introduced in
Terraform v0.13. For more information, see
[Legacy Shared Modules with Provider Configurations](#legacy-shared-modules-with-provider-configurations).
Provider configurations are used for all operations on associated resources,
including destroying remote objects and refreshing state. Terraform retains, as
part of its state, a reference to the provider configuration that was most
recently used to apply changes to each resource. When a `resource` block is
removed from the configuration, this record in the state is used to locate the
appropriate configuration because the resource's `provider` argument (if any)
is no longer present in the configuration.
removed from the configuration, this record in the state will be used to locate
the appropriate configuration because the resource's `provider` argument
(if any) will no longer be present in the configuration.
As a consequence, it is required that all resources created for a particular
provider configuration must be destroyed before that provider configuration is
removed, unless the related resources are re-configured to use a different
provider configuration first.
As a consequence, you must ensure that all resources that belong to a
particular provider configuration are destroyed before you can remove that
provider configuration's block from your configuration. If Terraform finds
a resource instance tracked in the state whose provider configuration block is
no longer available then it will return an error during planning, prompting you
to reintroduce the provider configuration.
### Provider Version Constraints in Modules
Although provider _configurations_ are shared between modules, each module must
declare its own [provider requirements](provider-requirements.html), so that
Terraform can ensure that there is a single version of the provider that is
compatible with all modules in the configuration and to specify the
[source address](provider-requirements.html#source-addresses) that serves as
the global (module-agnostic) identifier for a provider.
To declare that a module requires particular versions of a specific provider,
use a [`required_providers`](terraform.html#specifying-required-provider-versions)
block inside a `terraform` block:
use a `required_providers` block inside a `terraform` block:
```hcl
terraform {
required_providers {
aws = ">= 2.7.0"
aws = {
source = "hashicorp/aws"
version = ">= 2.7.0"
}
}
}
```
Shared modules should constrain only the minimum allowed version, using a `>=`
constraint. This specifies the minimum version the provider is compatible
with while allowing users to upgrade to newer provider versions without
altering the module source code.
A provider requirement says, for example, "This module requires version v2.7.0
of the provider `hashicorp/aws` and will refer to it as `aws`." It doesn't,
however, specify any of the configuration settings that determine what remote
endpoints the provider will access, such as an AWS region; configuration
settings come from provider _configurations_, and a particular overall Terraform
configuration can potentially have
[several different configurations for the same provider](providers.html#alias-multiple-provider-instances).
If you are writing a shared Terraform module, constrain only the minimum
required provider version using a `>=` constraint. This should specify the
minimum version containing the features your module relies on, and thus allow a
user of your module to potentially select a newer provider version if other
features are needed by other parts of their overall configuration.
### Implicit Provider Inheritance
@ -349,78 +442,47 @@ configuration block is valid, it is not necessary: proxy configuration blocks
are needed only to establish which _alias_ provider configurations a child
module is expecting.
A proxy configuration block must not include the `version` argument. To specify
version constraints for a particular child module without creating a local
module configuration, use the [`required_providers`](/docs/configuration/terraform.html#specifying-required-provider-versions)
setting inside a `terraform` block.
A proxy configuration block declares that a module is expecting to be
explicitly passed an additional (aliased) provider configuration. Don't use a
proxy configuration block if a module only needs a single default provider
configuration, and don't use proxy configuration blocks only to imply
[provider requirements](provider-requirements.html).
## Multiple Instances of a Module
## Legacy Shared Modules with Provider Configurations
Use the `count` or `for_each` arguments to create multiple instances of a module.
These arguments have the same syntax and type constraints as
[`count`](./resources.html#count-multiple-resource-instances-by-count) and
[`for_each`](./resources.html#for_each-multiple-resource-instances-defined-by-a-map-or-set-of-strings)
as defined for managed resources.
In Terraform v0.10 and earlier there was no explicit way to use different
configurations of a provider in different modules in the same configuration,
and so module authors commonly worked around this by writing `provider` blocks
directly inside their modules, making the module have its own separate
provider configurations separate from those declared in the root module.
```hcl
# my_buckets.tf
module "bucket" {
for_each = toset(["assets", "media"])
source = "./publish_bucket"
name = "${each.key}_bucket"
}
```
However, that pattern had a significant drawback: because a provider
configuration is required to destroy the remote object associated with a
resource instance as well as to create or update it, a provider configuration
must always stay present in the overall Terraform configuration for longer
than all of the resources it manages. If a particular module includes
both resources and the provider configurations for those resources then
removing the module from its caller would violate that constraint: both the
resources and their associated providers would, in effect, be removed
simultaneously.
```hcl
# publish_bucket/bucket-and-cloudfront.tf
variable "name" {} # this is the input parameter of the module
Terraform v0.11 introduced the mechanisms described in earlier sections to
allow passing provider configurations between modules in a structured way, and
thus we explicitly recommended against writing a child module with its own
provider configuration blocks. However, that legacy pattern continued to work
for compatibility purposes -- though with the same drawback -- until Terraform
v0.13.
resource "aws_s3_bucket" "example" {
# ...
}
Terraform v0.13 introduced the possibility for a module itself to use the
`for_each`, `count`, and `depends_on` arguments, but the implementation of
those unfortunately conflicted with the support for the legacy pattern.
resource "aws_iam_user" "deploy_user" {
# ...
}
```
This example defines a local child module in the `./publish_bucket`
subdirectory. That module has configuration to create an S3 bucket. The module
wraps the bucket and all the other implementation details required to configure
a bucket.
We declare multiple module instances by using the `for_each` attribute,
which accepts a map (with string keys) or a set of strings as its value. Additionally,
we use the `each.key` in our module block, because the
[`each`](/docs/configuration/resources.html#the-each-object) object is available when
we have declared `for_each` on the module block. When using the `count` argument, the
[`count`](/docs/configuration/resources.html#the-count-object) object is available.
Resources from child modules are prefixed with `module.module_name[module index]`
when displayed in plan output and elsewhere in the UI. For a module with without
`count` or `for_each`, the address will not contain the module index as the module's
name suffices to reference the module.
In our example, the `./publish_bucket` module contains `aws_s3_bucket.example`, and so the two
instances of this module produce S3 bucket resources with [resource addresses](/docs/internals/resource-addressing.html) of `module.bucket["assets"].aws_s3_bucket.example`
and `module.bucket["media"].aws_s3_bucket.example` respectively. These full addresses
are used within the UI and on the command line, but only [outputs](docs/configuration/outputs.html)
from a module can be referenced from elsewhere in your configuration.
When refactoring an existing configuration to introduce modules, moving
resource blocks between modules causes Terraform to see the new location
as an entirely separate resource to the old. Always check the execution plan
after performing such actions to ensure that no resources are surprisingly
deleted.
### Limitations when using module expansion
Modules using `count` or `for_each` cannot include configured `provider` blocks within the module.
Only [proxy configuration blocks](#proxy-configuration-blocks) are allowed.
If a module contains proxy configuration blocks, the calling module block must be have the
corresponding providers passed to the `providers` argument. If you attempt to use `count` or
`for_each` with a module that does not satisfy this requirement, you will see an error:
To retain the backward compatibility as much as possible, Terraform v0.13
continues to support the legacy pattern for module blocks that do not use these
new features, but a module with its own provider configurations is not
compatible with `for_each`, `count`, or `depends_on` and so Terraform will
produce an error announcing that if you attempt to combine these features. For
example:
```
Error: Module does not support count
@ -436,10 +498,23 @@ its provider configurations from the calling module, by using the "providers"
argument in the calling module block.
```
Assuming the child module only has proxy configuration blocks, the calling
module block could be adjusted like so to remove this error:
To make a module compatible with the new features, you must either remove all
of the `provider` blocks from its definition or, if you need multiple
configurations for the same provider, replace them with
_proxy configuration blocks_ as described in
[Passing Providers Explicitly](#passing-providers-explicitly).
If the new version of the module uses proxy configuration blocks, or if the
calling module needs the child module to use different provider configurations
than its own default provider configurations, the calling module must then
include an explicit `providers` argument to describe which provider
configurations the child module will use:
```hcl
provider "aws" {
region = "us-west-1"
}
```
provider "aws" {
region = "us-east-1"
alias = "east"
@ -448,24 +523,24 @@ provider "aws" {
module "child" {
count = 2
providers = {
# By default, the child module would use the
# default (unaliased) AWS provider configuration
# using us-west-1, but this will override it
# to use the additional "east" configuration
# for its resources instead.
aws = aws.east
}
}
```
Note how we are now [passing the providers](#passing-providers-explicitly) to the child module.
In addition, modules using `count` or `for_each` cannot pass different sets of providers
to different instances. For example, you cannot interpolate variables in the `providers`
block on a module.
This is because when a module instance is destroyed (such as a key-value being removed from the
`for_each` map), the appropriate provider must be available in order to perform the destroy.
You can pass different sets of providers to different module instances by using multiple `module` blocks:
```
# my_buckets.tf
Due to the association between resources and provider configurations being
static, module calls using `for_each` or `count` cannot pass different
provider configurations to different instances. If you need different
instances of your module to use different provider configurations then you
must use a separate `module` block for each distinct set of provider
configurations:
```hcl
provider "aws" {
alias = "usw1"
region = "us-west-1"
@ -509,12 +584,6 @@ module "bucket_w2" {
}
```
Each module block may optionally have different providers passed to it
using the [`providers`](/docs/configuration/modules.html#passing-providers-explicitly)
argument. This can be useful in situations where, for example, a duplicated set of
resources must be created across several regions or datacenters.
## Tainting resources within a module
The [taint command](/docs/commands/taint.html) can be used to _taint_ specific