terraform/internal/terraform/transform_targets.go

160 lines
4.2 KiB
Go
Raw Normal View History

package terraform
import (
"log"
"github.com/hashicorp/terraform/internal/addrs"
"github.com/hashicorp/terraform/internal/dag"
)
// GraphNodeTargetable is an interface for graph nodes to implement when they
// need to be told about incoming targets. This is useful for nodes that need
// to respect targets as they dynamically expand. Note that the list of targets
// provided will contain every target provided, and each implementing graph
// node must filter this list to targets considered relevant.
type GraphNodeTargetable interface {
terraform: ugly huge change to weave in new HCL2-oriented types Due to how deeply the configuration types go into Terraform Core, there isn't a great way to switch out to HCL2 gradually. As a consequence, this huge commit gets us from the old state to a _compilable_ new state, but does not yet attempt to fix any tests and has a number of known missing parts and bugs. We will continue to iterate on this in forthcoming commits, heading back towards passing tests and making Terraform fully-functional again. The three main goals here are: - Use the configuration models from the "configs" package instead of the older models in the "config" package, which is now deprecated and preserved only to help us write our migration tool. - Do expression inspection and evaluation using the functionality of the new "lang" package, instead of the Interpolator type and related functionality in the main "terraform" package. - Represent addresses of various objects using types in the addrs package, rather than hand-constructed strings. This is not critical to support the above, but was a big help during the implementation of these other points since it made it much more explicit what kind of address is expected in each context. Since our new packages are built to accommodate some future planned features that are not yet implemented (e.g. the "for_each" argument on resources, "count"/"for_each" on modules), and since there's still a fair amount of functionality still using old-style APIs, there is a moderate amount of shimming here to connect new assumptions with old, hopefully in a way that makes it easier to find and eliminate these shims later. I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.
2018-04-30 19:33:53 +02:00
SetTargets([]addrs.Targetable)
}
// TargetsTransformer is a GraphTransformer that, when the user specifies a
// list of resources to target, limits the graph to only those resources and
// their dependencies.
type TargetsTransformer struct {
// List of targeted resource names specified by the user
terraform: ugly huge change to weave in new HCL2-oriented types Due to how deeply the configuration types go into Terraform Core, there isn't a great way to switch out to HCL2 gradually. As a consequence, this huge commit gets us from the old state to a _compilable_ new state, but does not yet attempt to fix any tests and has a number of known missing parts and bugs. We will continue to iterate on this in forthcoming commits, heading back towards passing tests and making Terraform fully-functional again. The three main goals here are: - Use the configuration models from the "configs" package instead of the older models in the "config" package, which is now deprecated and preserved only to help us write our migration tool. - Do expression inspection and evaluation using the functionality of the new "lang" package, instead of the Interpolator type and related functionality in the main "terraform" package. - Represent addresses of various objects using types in the addrs package, rather than hand-constructed strings. This is not critical to support the above, but was a big help during the implementation of these other points since it made it much more explicit what kind of address is expected in each context. Since our new packages are built to accommodate some future planned features that are not yet implemented (e.g. the "for_each" argument on resources, "count"/"for_each" on modules), and since there's still a fair amount of functionality still using old-style APIs, there is a moderate amount of shimming here to connect new assumptions with old, hopefully in a way that makes it easier to find and eliminate these shims later. I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.
2018-04-30 19:33:53 +02:00
Targets []addrs.Targetable
}
func (t *TargetsTransformer) Transform(g *Graph) error {
terraform: ugly huge change to weave in new HCL2-oriented types Due to how deeply the configuration types go into Terraform Core, there isn't a great way to switch out to HCL2 gradually. As a consequence, this huge commit gets us from the old state to a _compilable_ new state, but does not yet attempt to fix any tests and has a number of known missing parts and bugs. We will continue to iterate on this in forthcoming commits, heading back towards passing tests and making Terraform fully-functional again. The three main goals here are: - Use the configuration models from the "configs" package instead of the older models in the "config" package, which is now deprecated and preserved only to help us write our migration tool. - Do expression inspection and evaluation using the functionality of the new "lang" package, instead of the Interpolator type and related functionality in the main "terraform" package. - Represent addresses of various objects using types in the addrs package, rather than hand-constructed strings. This is not critical to support the above, but was a big help during the implementation of these other points since it made it much more explicit what kind of address is expected in each context. Since our new packages are built to accommodate some future planned features that are not yet implemented (e.g. the "for_each" argument on resources, "count"/"for_each" on modules), and since there's still a fair amount of functionality still using old-style APIs, there is a moderate amount of shimming here to connect new assumptions with old, hopefully in a way that makes it easier to find and eliminate these shims later. I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.
2018-04-30 19:33:53 +02:00
if len(t.Targets) > 0 {
targetedNodes, err := t.selectTargetedNodes(g, t.Targets)
if err != nil {
return err
}
for _, v := range g.Vertices() {
if !targetedNodes.Include(v) {
log.Printf("[DEBUG] Removing %q, filtered by targeting.", dag.VertexName(v))
g.Remove(v)
}
}
}
return nil
}
terraform: ugly huge change to weave in new HCL2-oriented types Due to how deeply the configuration types go into Terraform Core, there isn't a great way to switch out to HCL2 gradually. As a consequence, this huge commit gets us from the old state to a _compilable_ new state, but does not yet attempt to fix any tests and has a number of known missing parts and bugs. We will continue to iterate on this in forthcoming commits, heading back towards passing tests and making Terraform fully-functional again. The three main goals here are: - Use the configuration models from the "configs" package instead of the older models in the "config" package, which is now deprecated and preserved only to help us write our migration tool. - Do expression inspection and evaluation using the functionality of the new "lang" package, instead of the Interpolator type and related functionality in the main "terraform" package. - Represent addresses of various objects using types in the addrs package, rather than hand-constructed strings. This is not critical to support the above, but was a big help during the implementation of these other points since it made it much more explicit what kind of address is expected in each context. Since our new packages are built to accommodate some future planned features that are not yet implemented (e.g. the "for_each" argument on resources, "count"/"for_each" on modules), and since there's still a fair amount of functionality still using old-style APIs, there is a moderate amount of shimming here to connect new assumptions with old, hopefully in a way that makes it easier to find and eliminate these shims later. I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.
2018-04-30 19:33:53 +02:00
// Returns a set of targeted nodes. A targeted node is either addressed
// directly, address indirectly via its container, or it's a dependency of a
// targeted node.
func (t *TargetsTransformer) selectTargetedNodes(g *Graph, addrs []addrs.Targetable) (dag.Set, error) {
targetedNodes := make(dag.Set)
core: Allow downstream targeting of certain node types The previous behavior of targets was that targeting a particular node would implicitly target everything it depends on. This makes sense when the dependencies in question are between resources, since we need to make sure all of a resource's dependencies are in place before we can create or update it. However, it had the undesirable side-effect that targeting a resource would _exclude_ any outputs referring to it, since the dependency edge goes from output to resource. This then causes the output to be "stale", which is problematic when outputs are being consumed by downstream configs using terraform_remote_state. GraphNodeTargetDownstream allows nodes to opt-in to a new behavior where they can be targeted by _inverted_ dependency edges. That is, it allows outputs to be considered targeted if anything they directly depend on is targeted. This is different than the implied targeting behavior in the other direction because transitive dependencies are not considered unless the intermediate nodes themselves have TargetDownstream. This means that an output1→output2→resource chain can implicitly target both outputs, but an output→resource1→resource2 chain _won't_ target the output if only resource2 is targeted. This behavior creates a scenario where an output can be visited before all of its dependencies are ready, since it may have a mixture of both targeted and untargeted dependencies. This is fine for outputs because they silently ignore any errors encountered during interpolation anyway, but other hypothetical future implementers of this interface may need to be more careful. This fixes #14186.
2017-05-11 03:27:49 +02:00
vertices := g.Vertices()
for _, v := range vertices {
if t.nodeIsTarget(v, addrs) {
targetedNodes.Add(v)
// We inform nodes that ask about the list of targets - helps for nodes
// that need to dynamically expand. Note that this only occurs for nodes
// that are already directly targeted.
if tn, ok := v.(GraphNodeTargetable); ok {
tn.SetTargets(addrs)
}
deps, _ := g.Ancestors(v)
for _, d := range deps {
targetedNodes.Add(d)
}
}
}
// It is expected that outputs which are only derived from targeted
// resources are also updated. While we don't include any other possible
// side effects from the targeted nodes, these are added because outputs
// cannot be targeted on their own.
// Start by finding the root module output nodes themselves
for _, v := range vertices {
// outputs are all temporary value types
tv, ok := v.(graphNodeTemporaryValue)
if !ok {
continue
}
core: Allow downstream targeting of certain node types The previous behavior of targets was that targeting a particular node would implicitly target everything it depends on. This makes sense when the dependencies in question are between resources, since we need to make sure all of a resource's dependencies are in place before we can create or update it. However, it had the undesirable side-effect that targeting a resource would _exclude_ any outputs referring to it, since the dependency edge goes from output to resource. This then causes the output to be "stale", which is problematic when outputs are being consumed by downstream configs using terraform_remote_state. GraphNodeTargetDownstream allows nodes to opt-in to a new behavior where they can be targeted by _inverted_ dependency edges. That is, it allows outputs to be considered targeted if anything they directly depend on is targeted. This is different than the implied targeting behavior in the other direction because transitive dependencies are not considered unless the intermediate nodes themselves have TargetDownstream. This means that an output1→output2→resource chain can implicitly target both outputs, but an output→resource1→resource2 chain _won't_ target the output if only resource2 is targeted. This behavior creates a scenario where an output can be visited before all of its dependencies are ready, since it may have a mixture of both targeted and untargeted dependencies. This is fine for outputs because they silently ignore any errors encountered during interpolation anyway, but other hypothetical future implementers of this interface may need to be more careful. This fixes #14186.
2017-05-11 03:27:49 +02:00
// root module outputs indicate that while they are an output type,
// they not temporary and will return false here.
if tv.temporaryValue() {
continue
}
core: Allow downstream targeting of certain node types The previous behavior of targets was that targeting a particular node would implicitly target everything it depends on. This makes sense when the dependencies in question are between resources, since we need to make sure all of a resource's dependencies are in place before we can create or update it. However, it had the undesirable side-effect that targeting a resource would _exclude_ any outputs referring to it, since the dependency edge goes from output to resource. This then causes the output to be "stale", which is problematic when outputs are being consumed by downstream configs using terraform_remote_state. GraphNodeTargetDownstream allows nodes to opt-in to a new behavior where they can be targeted by _inverted_ dependency edges. That is, it allows outputs to be considered targeted if anything they directly depend on is targeted. This is different than the implied targeting behavior in the other direction because transitive dependencies are not considered unless the intermediate nodes themselves have TargetDownstream. This means that an output1→output2→resource chain can implicitly target both outputs, but an output→resource1→resource2 chain _won't_ target the output if only resource2 is targeted. This behavior creates a scenario where an output can be visited before all of its dependencies are ready, since it may have a mixture of both targeted and untargeted dependencies. This is fine for outputs because they silently ignore any errors encountered during interpolation anyway, but other hypothetical future implementers of this interface may need to be more careful. This fixes #14186.
2017-05-11 03:27:49 +02:00
// If this output is descended only from targeted resources, then we
// will keep it
deps, _ := g.Ancestors(v)
found := 0
for _, d := range deps {
switch d.(type) {
case GraphNodeResourceInstance:
case GraphNodeConfigResource:
default:
core: Allow downstream targeting of certain node types The previous behavior of targets was that targeting a particular node would implicitly target everything it depends on. This makes sense when the dependencies in question are between resources, since we need to make sure all of a resource's dependencies are in place before we can create or update it. However, it had the undesirable side-effect that targeting a resource would _exclude_ any outputs referring to it, since the dependency edge goes from output to resource. This then causes the output to be "stale", which is problematic when outputs are being consumed by downstream configs using terraform_remote_state. GraphNodeTargetDownstream allows nodes to opt-in to a new behavior where they can be targeted by _inverted_ dependency edges. That is, it allows outputs to be considered targeted if anything they directly depend on is targeted. This is different than the implied targeting behavior in the other direction because transitive dependencies are not considered unless the intermediate nodes themselves have TargetDownstream. This means that an output1→output2→resource chain can implicitly target both outputs, but an output→resource1→resource2 chain _won't_ target the output if only resource2 is targeted. This behavior creates a scenario where an output can be visited before all of its dependencies are ready, since it may have a mixture of both targeted and untargeted dependencies. This is fine for outputs because they silently ignore any errors encountered during interpolation anyway, but other hypothetical future implementers of this interface may need to be more careful. This fixes #14186.
2017-05-11 03:27:49 +02:00
continue
}
if !targetedNodes.Include(d) {
// this dependency isn't being targeted, so we can't process this
// output
found = 0
break
core: Allow downstream targeting of certain node types The previous behavior of targets was that targeting a particular node would implicitly target everything it depends on. This makes sense when the dependencies in question are between resources, since we need to make sure all of a resource's dependencies are in place before we can create or update it. However, it had the undesirable side-effect that targeting a resource would _exclude_ any outputs referring to it, since the dependency edge goes from output to resource. This then causes the output to be "stale", which is problematic when outputs are being consumed by downstream configs using terraform_remote_state. GraphNodeTargetDownstream allows nodes to opt-in to a new behavior where they can be targeted by _inverted_ dependency edges. That is, it allows outputs to be considered targeted if anything they directly depend on is targeted. This is different than the implied targeting behavior in the other direction because transitive dependencies are not considered unless the intermediate nodes themselves have TargetDownstream. This means that an output1→output2→resource chain can implicitly target both outputs, but an output→resource1→resource2 chain _won't_ target the output if only resource2 is targeted. This behavior creates a scenario where an output can be visited before all of its dependencies are ready, since it may have a mixture of both targeted and untargeted dependencies. This is fine for outputs because they silently ignore any errors encountered during interpolation anyway, but other hypothetical future implementers of this interface may need to be more careful. This fixes #14186.
2017-05-11 03:27:49 +02:00
}
found++
}
if found > 0 {
// we found an output we can keep; add it, and all it's dependencies
targetedNodes.Add(v)
for _, d := range deps {
targetedNodes.Add(d)
}
}
}
return targetedNodes, nil
}
terraform: ugly huge change to weave in new HCL2-oriented types Due to how deeply the configuration types go into Terraform Core, there isn't a great way to switch out to HCL2 gradually. As a consequence, this huge commit gets us from the old state to a _compilable_ new state, but does not yet attempt to fix any tests and has a number of known missing parts and bugs. We will continue to iterate on this in forthcoming commits, heading back towards passing tests and making Terraform fully-functional again. The three main goals here are: - Use the configuration models from the "configs" package instead of the older models in the "config" package, which is now deprecated and preserved only to help us write our migration tool. - Do expression inspection and evaluation using the functionality of the new "lang" package, instead of the Interpolator type and related functionality in the main "terraform" package. - Represent addresses of various objects using types in the addrs package, rather than hand-constructed strings. This is not critical to support the above, but was a big help during the implementation of these other points since it made it much more explicit what kind of address is expected in each context. Since our new packages are built to accommodate some future planned features that are not yet implemented (e.g. the "for_each" argument on resources, "count"/"for_each" on modules), and since there's still a fair amount of functionality still using old-style APIs, there is a moderate amount of shimming here to connect new assumptions with old, hopefully in a way that makes it easier to find and eliminate these shims later. I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.
2018-04-30 19:33:53 +02:00
func (t *TargetsTransformer) nodeIsTarget(v dag.Vertex, targets []addrs.Targetable) bool {
var vertexAddr addrs.Targetable
switch r := v.(type) {
case GraphNodeResourceInstance:
vertexAddr = r.ResourceInstanceAddr()
case GraphNodeConfigResource:
terraform: ugly huge change to weave in new HCL2-oriented types Due to how deeply the configuration types go into Terraform Core, there isn't a great way to switch out to HCL2 gradually. As a consequence, this huge commit gets us from the old state to a _compilable_ new state, but does not yet attempt to fix any tests and has a number of known missing parts and bugs. We will continue to iterate on this in forthcoming commits, heading back towards passing tests and making Terraform fully-functional again. The three main goals here are: - Use the configuration models from the "configs" package instead of the older models in the "config" package, which is now deprecated and preserved only to help us write our migration tool. - Do expression inspection and evaluation using the functionality of the new "lang" package, instead of the Interpolator type and related functionality in the main "terraform" package. - Represent addresses of various objects using types in the addrs package, rather than hand-constructed strings. This is not critical to support the above, but was a big help during the implementation of these other points since it made it much more explicit what kind of address is expected in each context. Since our new packages are built to accommodate some future planned features that are not yet implemented (e.g. the "for_each" argument on resources, "count"/"for_each" on modules), and since there's still a fair amount of functionality still using old-style APIs, there is a moderate amount of shimming here to connect new assumptions with old, hopefully in a way that makes it easier to find and eliminate these shims later. I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.
2018-04-30 19:33:53 +02:00
vertexAddr = r.ResourceAddr()
terraform: ugly huge change to weave in new HCL2-oriented types Due to how deeply the configuration types go into Terraform Core, there isn't a great way to switch out to HCL2 gradually. As a consequence, this huge commit gets us from the old state to a _compilable_ new state, but does not yet attempt to fix any tests and has a number of known missing parts and bugs. We will continue to iterate on this in forthcoming commits, heading back towards passing tests and making Terraform fully-functional again. The three main goals here are: - Use the configuration models from the "configs" package instead of the older models in the "config" package, which is now deprecated and preserved only to help us write our migration tool. - Do expression inspection and evaluation using the functionality of the new "lang" package, instead of the Interpolator type and related functionality in the main "terraform" package. - Represent addresses of various objects using types in the addrs package, rather than hand-constructed strings. This is not critical to support the above, but was a big help during the implementation of these other points since it made it much more explicit what kind of address is expected in each context. Since our new packages are built to accommodate some future planned features that are not yet implemented (e.g. the "for_each" argument on resources, "count"/"for_each" on modules), and since there's still a fair amount of functionality still using old-style APIs, there is a moderate amount of shimming here to connect new assumptions with old, hopefully in a way that makes it easier to find and eliminate these shims later. I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.
2018-04-30 19:33:53 +02:00
default:
// Only resource and resource instance nodes can be targeted.
return false
}
terraform: ugly huge change to weave in new HCL2-oriented types Due to how deeply the configuration types go into Terraform Core, there isn't a great way to switch out to HCL2 gradually. As a consequence, this huge commit gets us from the old state to a _compilable_ new state, but does not yet attempt to fix any tests and has a number of known missing parts and bugs. We will continue to iterate on this in forthcoming commits, heading back towards passing tests and making Terraform fully-functional again. The three main goals here are: - Use the configuration models from the "configs" package instead of the older models in the "config" package, which is now deprecated and preserved only to help us write our migration tool. - Do expression inspection and evaluation using the functionality of the new "lang" package, instead of the Interpolator type and related functionality in the main "terraform" package. - Represent addresses of various objects using types in the addrs package, rather than hand-constructed strings. This is not critical to support the above, but was a big help during the implementation of these other points since it made it much more explicit what kind of address is expected in each context. Since our new packages are built to accommodate some future planned features that are not yet implemented (e.g. the "for_each" argument on resources, "count"/"for_each" on modules), and since there's still a fair amount of functionality still using old-style APIs, there is a moderate amount of shimming here to connect new assumptions with old, hopefully in a way that makes it easier to find and eliminate these shims later. I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.
2018-04-30 19:33:53 +02:00
for _, targetAddr := range targets {
switch vertexAddr.(type) {
case addrs.ConfigResource:
// Before expansion happens, we only have nodes that know their
// ConfigResource address. We need to take the more specific
// target addresses and generalize them in order to compare with a
// ConfigResource.
switch target := targetAddr.(type) {
case addrs.AbsResourceInstance:
targetAddr = target.ContainingResource().Config()
case addrs.AbsResource:
targetAddr = target.Config()
case addrs.ModuleInstance:
targetAddr = target.Module()
terraform: ugly huge change to weave in new HCL2-oriented types Due to how deeply the configuration types go into Terraform Core, there isn't a great way to switch out to HCL2 gradually. As a consequence, this huge commit gets us from the old state to a _compilable_ new state, but does not yet attempt to fix any tests and has a number of known missing parts and bugs. We will continue to iterate on this in forthcoming commits, heading back towards passing tests and making Terraform fully-functional again. The three main goals here are: - Use the configuration models from the "configs" package instead of the older models in the "config" package, which is now deprecated and preserved only to help us write our migration tool. - Do expression inspection and evaluation using the functionality of the new "lang" package, instead of the Interpolator type and related functionality in the main "terraform" package. - Represent addresses of various objects using types in the addrs package, rather than hand-constructed strings. This is not critical to support the above, but was a big help during the implementation of these other points since it made it much more explicit what kind of address is expected in each context. Since our new packages are built to accommodate some future planned features that are not yet implemented (e.g. the "for_each" argument on resources, "count"/"for_each" on modules), and since there's still a fair amount of functionality still using old-style APIs, there is a moderate amount of shimming here to connect new assumptions with old, hopefully in a way that makes it easier to find and eliminate these shims later. I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.
2018-04-30 19:33:53 +02:00
}
core: -target option to also select resources in descendant modules Previously the behavior for -target when given a module address was to target only resources directly within that module, ignoring any resources defined in child modules. This behavior turned out to be counter-intuitive, since users expected the -target address to be interpreted hierarchically. We'll now use the new "Contains" function for addresses, which provides a hierarchical "containment" concept that is more consistent with user expectations. In particular, it allows module.foo to match module.foo.module.bar.aws_instance.baz, where before that would not have been true. Since Contains isn't commutative (unlike Equals) this requires some special handling for targeting specific indices. When given an argument like -target=aws_instance.foo[0], the initial graph construction (for both plan and refresh) is for the resource nodes from configuration, which have not yet been expanded to separate indexed instances. Thus we need to do the first pass of TargetsTransformer in mode where indices are ignored, with the work then completed by the DynamicExpand method which re-applies the TargetsTransformer in index-sensitive mode. This is a breaking change for anyone depending on the previous behavior of -target, since it will now select more resources than before. There is no way provided to obtain the previous behavior. Eventually we may support negative targeting, which could then combine with positive targets to regain the previous behavior as an explicit choice.
2017-06-16 03:15:41 +02:00
}
terraform: ugly huge change to weave in new HCL2-oriented types Due to how deeply the configuration types go into Terraform Core, there isn't a great way to switch out to HCL2 gradually. As a consequence, this huge commit gets us from the old state to a _compilable_ new state, but does not yet attempt to fix any tests and has a number of known missing parts and bugs. We will continue to iterate on this in forthcoming commits, heading back towards passing tests and making Terraform fully-functional again. The three main goals here are: - Use the configuration models from the "configs" package instead of the older models in the "config" package, which is now deprecated and preserved only to help us write our migration tool. - Do expression inspection and evaluation using the functionality of the new "lang" package, instead of the Interpolator type and related functionality in the main "terraform" package. - Represent addresses of various objects using types in the addrs package, rather than hand-constructed strings. This is not critical to support the above, but was a big help during the implementation of these other points since it made it much more explicit what kind of address is expected in each context. Since our new packages are built to accommodate some future planned features that are not yet implemented (e.g. the "for_each" argument on resources, "count"/"for_each" on modules), and since there's still a fair amount of functionality still using old-style APIs, there is a moderate amount of shimming here to connect new assumptions with old, hopefully in a way that makes it easier to find and eliminate these shims later. I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.
2018-04-30 19:33:53 +02:00
if targetAddr.TargetContains(vertexAddr) {
return true
}
}
return false
}