From 5f5432e8ea443daa1a77eb9abd65445d3583f6c8 Mon Sep 17 00:00:00 2001 From: Martin Atkins Date: Wed, 14 Apr 2021 06:04:40 -0700 Subject: [PATCH] website: v0.15 upgrade guide for sensitive resource attributes (#28355) * website: v0.15 upgrade guide for sensitive resource attributes Our earlier draft of this guide didn't include a section about the stabilization of the "provider_sensitive_attrs" language experiment. This new section aims to address the situation where a module might previously have been returning a sensitive value without having marked it as such, and thus that module will begin returning an error after upgrading to Terraform v0.15. As part of that, I also reviewed the existing documentation about these features and made some edits aiming to make these four different sections work well together if users refer to them all at once, as they are likely to do if they follow the new links from the upgrade guide. I aimed to retain all of the content we had before, but some of it is now in a new location. In particular, I moved the discussion about the v0.14 language experiment into the upgrade guide, because it seems like a topic only really relevant to those upgrading from an earlier version and not something folks need to know about if they are using Terraform for the first time in v0.15 or later. * minor fixups Co-authored-by: Kristin Laemmert --- .../language/expressions/references.html.md | 71 +++++----- website/docs/language/values/outputs.html.md | 23 ++-- .../docs/language/values/variables.html.md | 47 ++++--- website/upgrade-guides/0-15.html.markdown | 122 ++++++++++++++++++ 4 files changed, 191 insertions(+), 72 deletions(-) diff --git a/website/docs/language/expressions/references.html.md b/website/docs/language/expressions/references.html.md index 2b1bb7366..f69420f92 100644 --- a/website/docs/language/expressions/references.html.md +++ b/website/docs/language/expressions/references.html.md @@ -277,6 +277,36 @@ Note that unlike `count`, splat expressions are _not_ directly applicable to res * `values(aws_instance.example)[*].id` +### Sensitive Resource Attributes + +When defining the schema for a resource type, a provider developer can mark +certain attributes as _sensitive_, in which case Terraform will show a +placeholder marker `(sensitive)` instead of the actual value when rendering +a plan involving that attribute. + +A provider attribute marked as sensitive behaves similarly to an +[an input variable declared as sensitive](/docs/language/values/variables.html#suppressing-values-in-cli-output), +where Terraform will hide the value in the plan and apply messages and will +also hide any other values you derive from it as sensitive. +However, there are some limitations to that behavior as described in +[Cases where Terraform may disclose a sensitive variable](/docs/language/values/variables.html#cases-where-terraform-may-disclose-a-sensitive-variable). + +If you use a sensitive value from a resource attribute as part of an +[output value](/docs/language/values/outputs.html) then Terraform will require +you to also mark the output value itself as sensitive, to confirm that you +intended to export it. + +Terraform will still record sensitive values in the [state](/docs/language/state/index.html), +and so anyone who can access the state data will have access to the sensitive +values in cleartext. For more information, see +[_Sensitive Data in State_](/docs/language/state/sensitive-data.html). + +-> **Note:** Treating values derived from a sensitive resource attribute as +sensitive themselves was introduced in Terraform v0.15. Earlier versions of +Terraform will obscure the direct value of a sensitive resource attribute, +but will _not_ automatically obscure other values derived from sensitive +resource attributes. + ### Values Not Yet Known When Terraform is planning a set of changes that will apply your configuration, @@ -317,44 +347,3 @@ effect: until the apply phase, causing the apply to fail. Unknown values appear in the `terraform plan` output as `(not yet known)`. - -### Sensitive Resource Attributes - -When defining the schema for a resource type, a provider developer can mark -certain attributes as _sensitive_, in which case Terraform will show a -placeholder marker `(sensitive)` instead of the actual value when rendering -a plan involving that attribute. - -The treatment of these particular sensitive values is currently different than -for values in -[input variables](/docs/language/values/variables.html) -and -[output values](/docs/language/values/outputs.html) -that have `sensitive = true` set. Sensitive resource attributes will be -obscured in the plan when they appear directly, but other values that you -_derive_ from a sensitive resource attribute will not themselves be considered -sensitive, and so Terraform will include those derived values in its output -without redacting them. - -Terraform v0.15.0 and later treats resource attributes that are marked as -sensitive (by the provider) in the same way as sensitive input variables and -output values, so that Terraform will consider any derived values as sensitive too. - -If you are using Terraform v0.14.x, this feature is considered experimental. -You can activate that experiment for your module using the -`provider_sensitive_attrs` experiment keyword: - -```hcl -terraform { - experiments = [provider_sensitive_attrs] -} -``` - -The behavior of this experiment might change even in future patch releases of -Terraform, so we don't recommend using this experiment in modules you use -to describe production infrastructure. - -If you enable this experiment and you have exported any sensitive resource -attributes via your module's output values then you will see an error unless -you also mark the output value as `sensitive = true`, confirming your intent -to export it. diff --git a/website/docs/language/values/outputs.html.md b/website/docs/language/values/outputs.html.md index 60450f83d..35b8aa441 100644 --- a/website/docs/language/values/outputs.html.md +++ b/website/docs/language/values/outputs.html.md @@ -102,9 +102,10 @@ output "db_password" { } ``` -Setting an output value as sensitive prevents Terraform from showing its value -in `plan` and `apply`. In the following scenario, our root module has an output declared as sensitive -and a module call with a sensitive output, which we then use in a resource attribute. +Terraform will hide values marked as sensitive in the messages from +`terraform plan` and `terraform apply`. In the following scenario, our root +module has an output declared as sensitive and a module call with a +sensitive output, which we then use in a resource attribute. ```hcl # main.tf @@ -130,11 +131,9 @@ output "a" { } ``` -When we run a `plan` or `apply`, the sensitive value is redacted from output: +When we run a plan or apply, the sensitive value is redacted from output: ``` -# CLI output - Terraform will perform the following actions: # test_instance.x will be created @@ -148,11 +147,15 @@ Changes to Outputs: + out = (sensitive value) ``` --> **Note:** In Terraform versions prior to Terraform 0.14, setting an output value in the root module as sensitive would prevent Terraform from showing its value in the list of outputs at the end of `terraform apply`. However, the value could still display in the CLI output for other reasons, like if the value is referenced in an expression for a resource argument. +-> **Note:** In Terraform versions prior to Terraform 0.14, setting an output +value in the root module as sensitive would prevent Terraform from showing its +value in the list of outputs at the end of `terraform apply`. However, the +value could still display in the CLI output for other reasons, like if the +value is referenced in an expression for a resource argument. -Sensitive output values are still recorded in the -[state](/docs/language/state/index.html), and so will be visible to anyone who is able -to access the state data. For more information, see +Terraform will still record sensitive values in the [state](/docs/language/state/index.html), +and so anyone who can access the state data will have access to the sensitive +values in cleartext. For more information, see [_Sensitive Data in State_](/docs/language/state/sensitive-data.html). diff --git a/website/docs/language/values/variables.html.md b/website/docs/language/values/variables.html.md index f911b0406..3c167eb1d 100644 --- a/website/docs/language/values/variables.html.md +++ b/website/docs/language/values/variables.html.md @@ -211,13 +211,16 @@ using a sentence structure similar to the above examples. > **Hands-on:** Try the [Protect Sensitive Input Variables](https://learn.hashicorp.com/tutorials/terraform/sensitive-variables?in=terraform/configuration-language&utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS) tutorial on HashiCorp Learn. -Setting a variable as `sensitive` prevents Terraform from showing its value in the `plan` or `apply` output, when that variable is used within a configuration. +Setting a variable as `sensitive` prevents Terraform from showing its value in +the `plan` or `apply` output, when you use that variable elsewhere in your +configuration. -Sensitive values are still recorded in the [state](/docs/language/state/index.html), and so will be visible to anyone who is able to access the state data. For more information, see [_Sensitive Data in State_](/docs/language/state/sensitive-data.html). +Terraform will still record sensitive values in the [state](/docs/language/state/index.html), +and so anyone who can access the state data will have access to the sensitive +values in cleartext. For more information, see +[_Sensitive Data in State_](/docs/language/state/sensitive-data.html). -A provider can define [an attribute as sensitive](/docs/extend/best-practices/sensitive-state.html#using-the-sensitive-flag), which prevents the value of that attribute from being displayed in logs or regular output. The `sensitive` argument on variables allows users to replicate this behavior for values in their configuration, by defining a variable as `sensitive`. - -Define a variable as sensitive by setting the `sensitive` argument to `true`: +Declare a variable as sensitive by setting the `sensitive` argument to `true`: ``` variable "user_information" { @@ -234,7 +237,9 @@ resource "some_resource" "a" { } ``` -Using this variable throughout your configuration will obfuscate the value from display in `plan` or `apply` output: +Any expressions whose result depends on the sensitive variable will be treated +as sensitive themselves, and so in the above example the two arguments of +`resource "some_resource" "a"` will also be hidden in the plan output: ``` Terraform will perform the following actions: @@ -248,22 +253,12 @@ Terraform will perform the following actions: Plan: 1 to add, 0 to change, 0 to destroy. ``` -In some cases where a sensitive variable is used in a nested block, the whole block can be redacted. This happens with resources which can have multiple blocks of the same type, where the values must be unique. This looks like: +In some cases where you use a sensitive variable inside a nested block, Terraform +may treat the entire block as redacted. This happens for resource types where +all of the blocks of a particular type are required to be unique, and so +disclosing the content of one block might imply the content of a sibling block. ``` -# main.tf - -resource "some_resource" "a" { - nested_block { - user_information = var.user_information # a sensitive variable - other_information = "not sensitive data" - } -} - -# CLI output - -Terraform will perform the following actions: - # some_resource.a will be updated in-place ~ resource "some_resource" "a" { ~ nested_block { @@ -271,9 +266,19 @@ Terraform will perform the following actions: # so its contents will not be displayed. } } - ``` +A provider can also +[declare an attribute as sensitive](/docs/extend/best-practices/sensitive-state.html#using-the-sensitive-flag), +which will cause Terraform to hide it from regular output regardless of how +you assign it a value. For more information, see +[Sensitive Resource Attributes](/docs/language/expressions/references.html#sensitive-resource-attributes). + +If you use a sensitive value from as part of an +[output value](/docs/language/values/outputs.html) then Terraform will require +you to also mark the output value itself as sensitive, to confirm that you +intended to export it. + #### Cases where Terraform may disclose a sensitive variable A `sensitive` variable is a configuration-centered concept, and values are sent to providers without any obfuscation. A provider error could disclose a value if that value is included in the error message. For example, a provider might return the following error even if "foo" is a sensitive value: `"Invalid value 'foo' for field"` diff --git a/website/upgrade-guides/0-15.html.markdown b/website/upgrade-guides/0-15.html.markdown index aa79773a0..d8de73a3b 100644 --- a/website/upgrade-guides/0-15.html.markdown +++ b/website/upgrade-guides/0-15.html.markdown @@ -46,12 +46,134 @@ may be able to reproduce it and offer advice. Upgrade guide sections: +* [Sensitive Output Values](#sensitive-output-values) * [Legacy Configuration Language Features](#legacy-configuration-language-features) * [Alternative (Aliased) Provider Configurations Within Modules](#alternative-provider-configurations-within-modules) * [Commands Accepting a Configuration Directory Argument](#commands-accepting-a-configuration-directory-argument) * [Microsoft Windows Terminal Support](#microsoft-windows-terminal-support) * [Other Minor Command Line Behavior Changes](#other-minor-command-line-behavior-changes) +## Sensitive Output Values + +Terraform v0.14 previously introduced the ability for Terraform to track and +propagate the "sensitivity" of values through expressions that include +references to sensitive input variables and output values. For example: + +```hcl +variable "example" { + type = string + sensitive = true +} + +resource "example" "example" { + # The following value is also treated as sensitive, because it's derived + # from var.example. + name = "foo-${var.example}" +} +``` + +As part of that feature, Terraform also began requiring you to mark an output +value as sensitive if its definition includes any sensitive values itself: + +```hcl +output "example" { + value = "foo-${var.example}" + + # Must mark this output value as sensitive, because it's derived from + # var.example that is declared as sensitive. + sensitive = true +} +``` + +Terraform v0.15 extends this mechanism to also work for values derived from +resource attributes that the provider has declared as being sensitive. +Provider developers will typically mark an attribute as sensitive if the +remote system has documented the corresponding field as being sensitive, such +as if the attribute always contains a password or a private key. + +As a result of that, after upgrading to Terraform v0.15 you may find that +Terraform now reports some of your output values as invalid, if they were +derived from sensitive attributes without also being marked as sensitive: + +``` +╷ +│ Error: Output refers to sensitive values +│ +│ on sensitive-resource-attr.tf line 5: +│ 5: output "private_key" { +│ +│ Expressions used in outputs can only refer to sensitive values if the +│ sensitive attribute is true. +╵ +``` + +If you were intentionally exporting a sensitive value, you can address the +error by adding an explicit declaration `sensitive = true` to the output +value declaration: + +```hcl +output "private_key" { + value = tls_private_key.example.private_key_pem + sensitive = true +} +``` + +With that addition, if this output value was a root module output value then +Terraform will hide its value in the `terraform plan` and `terraform apply` +output: + +``` +Changes to Outputs: + + private_key = (sensitive value) +``` + +-> **Note:** The declaration of an output value as sensitive must be made +within the module that declares the output, so if you depend on a third-party +module that has a sensitive output value that is lacking this declaration then +you will need to wait for a new version of that module before you can upgrade +to Terraform v0.15. + +The value is only hidden in the main UI, and so the sensitive value +will still be recorded in the state. If you declared this output value in order +to use it as part of integration with other software, you can still retrieve +the cleartext value using commands intended for machine rather than human +consumption, such as `terraform output -json` or `terraform output -raw`: + +```shellsession +$ terraform output -raw private_key +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEAoahsvJ1rIxTIOOmJZ7yErs5eOq/Kv9+5l3h0LbxW78K8//Kb +OMU3v8F3h8jp+AB/1zGr5UBYfnYp5ncJm/OTCXLFAHxGibEbRnf1m2A3o0hEaWsw +# (etc...) +``` + +If you consider Terraform's treatment of a sensitive value to be too +conservative and you'd like to force Terraform to treat a sensitive value as +non-sensitive, you can use +[the `nonsensitive` function](/docs/language/functions/nonsensitive.html) to +override Terraform's automatic detection: + +```hcl +output "private_key" { + # WARNING: Terraform will display this result as cleartext + value = nonsensitive(tls_private_key.example.private_key_pem) +} +``` + +For more information on the various situations where sensitive values can +originate in Terraform, refer to the following sections: + +* [Sensitive Input Variables](/docs/language/values/variables.html#suppressing-values-in-cli-output) +* [Sensitive Resource Attributes](/docs/language/expressions/references.html#sensitive-resource-attributes) +* [Sensitive Output Values](/docs/language/values/outputs.html#sensitive) + +-> **Note:** The new behavior described in this section was previously +available in Terraform v0.14 as the +[language experiment](/docs/language/settings/#experimental-language-features) +`provider_sensitive_attrs`. That experiment has now concluded, so if you were +participating in that experiment then you'll now need to remove the experiment +opt-in from your module as part of upgrading to Terraform v0.15. + ## Legacy Configuration Language Features Terraform v0.12 introduced new syntax for a variety of existing Terraform