diff --git a/website/docs/language/resources/provisioners/connection.mdx b/website/docs/language/resources/provisioners/connection.mdx index 704012199..eb1ccff5f 100644 --- a/website/docs/language/resources/provisioners/connection.mdx +++ b/website/docs/language/resources/provisioners/connection.mdx @@ -102,6 +102,9 @@ block would create a dependency cycle. Defaults to 5 minutes. * `script_path` - The path used to copy scripts meant for remote execution. + For more information, see + [How Provisioners Execute Remote Scripts](#how-provisioners-execute-remote-scripts) + below. **Additional arguments only supported by the `ssh` connection type:** @@ -123,9 +126,7 @@ block would create a dependency cycle. * `host_key` - The public key from the remote host or the signing CA, used to verify the connection. -* `target_platform` - The target platform to connect to. Valid values are `windows` and `unix`. Defaults to `unix` if not set. - - If the platform is set to `windows`, the default `script_path` is `c:\windows\temp\terraform_%RAND%.cmd`, assuming [the SSH default shell](https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_server_configuration#configuring-the-default-shell-for-openssh-in-windows) is `cmd.exe`. If the SSH default shell is PowerShell, set `script_path` to `"c:/windows/temp/terraform_%RAND%.ps1"` +* `target_platform` - The target platform to connect to. Valid values are `"windows"` and `"unix"`. Defaults to `"unix"` if not set. **Additional arguments only supported by the `winrm` connection type:** @@ -137,6 +138,11 @@ block would create a dependency cycle. * `cacert` - The CA certificate to validate against. +Provisioners typically assume that the remote system runs Microsoft Windows +when using the `winrm` connection type. Behaviors which would vary based on +the `target_platform` option if using SSH will instead force the +Windows-specific behavior when using WinRM, unless otherwise specified. + ## Connecting through a Bastion Host with SSH @@ -167,3 +173,78 @@ The `ssh` connection also supports the following fields to facilitate connnectio * `bastion_certificate` - The contents of a signed CA Certificate. The certificate argument must be used in conjunction with a `bastion_private_key`. These can be loaded from a file on disk using the [the `file` function](/language/functions/file). + +## How Provisioners Execute Remote Scripts + +Provisioners which execute commands on a remote system via a protocol such as +SSH typically achieve that by uploading a script file to the remote system +and then asking the default shell to execute it. Provisioners use this strategy +because it then allows you to use all of the typical scripting techniques +supported by that shell, including preserving environment variable values +and other context between script statements. + +However, this approach does have some consequences which can be relevant in +some unusual situations, even though this is just an implementation detail +for typical use. + +Most importantly, there must be a suitable location in the remote filesystem +where the provisioner can create the script file. By default, Terraform +chooses a path containing a random number using the following patterns +depending on how `target_platform` is set: + +* `"unix"`: `/tmp/terraform_%RAND%.sh` +* `"windows"`: `C:/windows/temp/terraform_%RAND%.cmd` + +In both cases above, the provisioner replaces the sequence `%RAND%` with +some randomly-chosen decimal digits. + +Provisioners cannot react directly to remote environment variables such as +`TMPDIR` or use functions like `mktemp` because they run on the system where +Terraform is running, not on the remote system. Therefore if your remote +system doesn't use the filesystem layout expected by these default paths +then you can override it using the `script_path` option in your `connection` +block: + +```hcl +connection { + # ... + script_path = "H:/terraform-temp/script_%RAND%.sh" +} +``` + +As with the default patterns, provisioners will replace the sequence `%RAND%` +with randomly-selected decimal digits, to reduce the likelihood of collisions +between multiple provisioners running concurrently. + +If your target system is running Windows, we recommend uses forward slashes +instead of backslashes, despite the typical convention on Windows, because +the Terraform language uses backslash as the quoted string escape character. + +### Executing Scripts using SSH/SCP + +When using the SSH protocol, provisioners upload their script files using +the Secure Copy Protocol (SCP), which requires that the remote system have +the `scp` service program installed to act as the server for that protocol. + +Provisioners will pass the chosen script path (after `%RAND%` +expansion) directly to the remote `scp` process, which is responsible for +interpreting it. With the default configuration of `scp` as distributed with +OpenSSH, you can place temporary scripts in the home directory of the remote +user by specifying a relative path: + +```hcl +connection { + type = "ssh" + # ... + script_path = "terraform_provisioner_%RAND%.sh" +} +``` + +-> **Warning:** In Terraform v1.0 and earlier, the built-in provisioners +incorrectly passed the `script_path` value to `scp` through a remote shell and +thus allowed it to be subject to arbitrary shell expansion, and thus created an +unintended opportunity for remote code execution. Terraform v1.1 and later +will now correctly quote and escape the script path to ensure that the +remote `scp` process can always interpret it literally. For modules that will +be used with Terraform v1.0 and earlier, avoid using untrusted external +values as part of the `script_path` argument. diff --git a/website/docs/language/resources/provisioners/file.mdx b/website/docs/language/resources/provisioners/file.mdx index 06b22758b..83dfb7a90 100644 --- a/website/docs/language/resources/provisioners/file.mdx +++ b/website/docs/language/resources/provisioners/file.mdx @@ -52,41 +52,77 @@ resource "aws_instance" "web" { The following arguments are supported: -* `source` - This is the source file or folder. It can be specified as - relative to the current working directory or as an absolute path. This - attribute cannot be specified with `content`. +* `source` - The source file or directory. Specify it either relative to the + current working directory or as an absolute path. + This argument cannot be combined with `content`. -* `content` - This is the content to copy on the destination. If destination is a file, - the content will be written on that file, in case of a directory a file named - `tf-file-content` is created. It's recommended to use a file as the destination. A - [`template_file`](https://registry.terraform.io/providers/hashicorp/template/latest/docs/data-sources/file) might be referenced in here, or - any interpolation syntax. This attribute cannot be specified with `source`. +* `content` - The direct content to copy on the destination. + If destination is a file, the content will be written on that file. In case + of a directory, a file named `tf-file-content` is created inside that + directory. We recommend using a file as the destination when using `content`. + This argument cannot be combined with `source`. -* `destination` - (Required) This is the destination path. It must be specified as an - absolute path. +* `destination` - (Required) The destination path to write to on the remote + system. See [Destination Paths](#destination-paths) below for more + information. + +## Destination Paths + +The path you provide in the `destination` argument will be evaluated by the +remote system, rather than by Terraform itself. Therefore the valid values +for that argument can vary depending on the operating system and remote access +software running on the target. + +When connecting over SSH, the `file` provisioner passes the given destination +path verbatim to the `scp` program on the remote host. By default, OpenSSH's +`scp` implementation runs in the remote user's home directory and so you can +specify a relative path to upload into that home directory, or an absolute +path to upload to some other location. The remote `scp` process will run with +the access level of the user specified in the `connection` block, and so +permissions may prevent writing directly to locations outside of the home +directory. + +Because WinRM has no corresponding file transfer protocol, for WinRM +connections the `file` provisioner uses a more complex process: + +1. Generate a temporary filename in the directory given in the remote system's + `TEMP` environment variable, using a pseudorandom UUID for uniqueness. +2. Use sequential generated `echo` commands over WinRM to gradually append + base64-encoded chunks of the source file to the chosen temporary file. +3. Use an uploaded PowerShell script to read the temporary file, base64-decode, + and write the raw result into the destination file. + +In the WinRM case, the destination path is therefore interpreted by PowerShell +and so you must take care not to use any meta-characters that PowerShell might +interpret. In particular, avoid including any untrusted external input in +your `destination` argument when using WinRM, because it can serve as a vector +for arbitrary PowerShell code execution on the remote system. + +Modern Windows systems support running an OpenSSH server, so we strongly +recommend choosing SSH over WinRM whereever possible, and using WinRM only as +a last resort when working with obsolete Windows versions. ## Directory Uploads -The file provisioner is also able to upload a complete directory to the remote machine. -When uploading a directory, there are a few important things you should know. +The `file` provisioner can upload a complete directory to the remote machine. +When uploading a directory, there are some additional considerations. -First, when using the `ssh` connection type the destination directory must already exist. -If you need to create it, use a remote-exec provisioner just prior to the file provisioner -in order to create the directory. When using the `winrm` connection type the destination -directory will be created for you if it doesn't already exist. +When using the `ssh` connection type the destination directory must already +exist. If you need to create it, use a remote-exec provisioner just prior to +the file provisioner in order to create the directory -Next, the existence of a trailing slash on the source path will determine whether the -directory name will be embedded within the destination, or whether the destination will -be created. An example explains this best: +When using the `winrm` connection type the destination directory will be +created for you if it doesn't already exist. -If the source is `/foo` (no trailing slash), and the destination is `/tmp`, then the contents -of `/foo` on the local machine will be uploaded to `/tmp/foo` on the remote machine. The -`foo` directory on the remote machine will be created by Terraform. +The existence of a trailing slash on the source path will determine whether the +directory name will be embedded within the destination, or whether the +destination will be created. For example: -If the source, however, is `/foo/` (a trailing slash is present), and the destination is -`/tmp`, then the contents of `/foo` will be uploaded directly into `/tmp`. +* If the source is `/foo` (no trailing slash), and the destination is `/tmp`, + then the contents of `/foo` on the local machine will be uploaded to + `/tmp/foo` on the remote machine. The `foo` directory on the remote machine + will be created by Terraform. -This behavior was adopted from the standard behavior of -[rsync](https://linux.die.net/man/1/rsync). - --> **Note:** Under the covers, rsync may or may not be used. +* If the source, however, is `/foo/` (a trailing slash is present), and the + destination is `/tmp`, then the contents of `/foo` will be uploaded directly + into `/tmp`. diff --git a/website/docs/language/upgrade-guides/1-1.mdx b/website/docs/language/upgrade-guides/1-1.mdx index faafcf147..aa16a4838 100644 --- a/website/docs/language/upgrade-guides/1-1.mdx +++ b/website/docs/language/upgrade-guides/1-1.mdx @@ -23,6 +23,7 @@ small number of users, described in the following sections. * [Terraform requires macOS 10.13 High Sierra or later](#terraform-requires-macos-1013-high-sierra-or-later) * [Preparation for removing Azure AD Graph support in the AzureRM Backend](#preparation-for-removing-azure-ad-graph-support-in-the-azurerm-backend) +* [Interpretation of remote file paths in the `remote-exec` and `file` provisioners](#interpretation-of-remote-file-paths-in-the-remote-exec-and-file-provisioners) * [Changes to `terraform graph`](#changes-to-terraform-graph) * [Changes to `terraform state mv`](#changes-to-terraform-state-mv) * [Provider checksum verification in `terraform apply`](#provider-checksum-verification-in-terraform-apply) @@ -54,6 +55,66 @@ in the near future to prepare for the final removal of Azure AD Graph support in a later Terraform release. However, no immediate change is required before upgrading to Terraform v1.1. +## Interpretation of remote file paths in the `remote-exec` and `file` provisioners + +When using Terraform's built-in `remote-exec` and `file` provisioners, there +are two situations where Terraform internally uses +[Secure Copy Protocol](https://en.wikipedia.org/wiki/Secure_copy_protocol) +(SCP) to upload files to the remote system at a configuration-specified +location: + +* For [the `file` provisioner](/language/resources/provisioners/file), + the primary functionality is to upload a file using SCP, and the + `destination` argument specifies the remote path where the file is to be + written. +* For [the `remote-exec` provisioner](/language/resources/provisioners/remote-exec), + internally the provisioner works by uploading the given scripts to files + on the remote system and then executing them. By default the provisioner + selects a temporary filename automatically, but a module author can + potentially override that location using the `script_path` argument in the + associated [`connection` block](https://www.terraform.io/language/resources/provisioners/connection). + +If you are not using either of the specific arguments mentioned above, no +configuration changes will be required to upgrade to Terraform v1.1. + +These provisioners both passing the specified remote paths to the `scp` service +program on the remote system. In Terraform v1.0 and earlier, the provisioners +were passing the paths to `scp` in a way that was inadvertently subject to +_shell expansion_. That inadvertently allowed for convenient shorthands +such as `~/example` and `$HOME/example` to write into the target user's +home directory, but also offered an undesirable opportunity for accidental +remote code execution, such as `$(arbitrary-program)`. + +In Terraform v1.1 both of the above remote path arguments are passed _verbatim_ +to the remote `scp` service, without any prior shell expansion. For that reason, +shell-defined expansion tokens such as `~` and environment variable references +will no longer be evaluated. + +By default, the OpenSSH server and the program `scp` together already interpret +relative paths as relative to the target user's home directory, and so +module authors can specify relative paths without any special metacharacters +in order to request uploading into that default location: + +```hcl + provisioner "file" { + source = "local.txt" + destination = "remote.txt" + } +``` + +If you maintain a module that was depending on expansion of `~/`, `$HOME/`, +`${HOME}`/ or similar, remove that prefix so that your module instead specifies +just a relative path. + +This is an intentional compatibility regression which we accepted after due +consideration of +[the pragmatic exceptions to our compatibility promises](/language/v1-compatibility-promises#pragmatic-exceptions). +Specifically, this behavior offered an unintended and non-obvious avenue for +arbitrary code execution on the remote system if either of the above arguments +were populated from outside input, and an alternative approach is available +which doesn't have that drawback, and this is therefore justified on security +grounds. + ## Changes to `terraform graph` The `terraform graph` command exists to help with debugging and so it