Commit Graph

415 Commits

Author SHA1 Message Date
Martin Atkins 954d38e870 lang: New file-hashing functions
In prior versions, we recommended using hash functions in conjunction with
the file function as an idiom for detecting changes to upstream blobs
without fetching and comparing the whole blob.

That approach relied on us being able to return raw binary data from
file(...). Since Terraform strings pass through intermediate
representations that are not binary-safe (e.g. the JSON state), there was
a risk of string corruption in prior versions which we have avoided for
0.12 by requiring that file(...) be used only with UTF-8 text files.

The specific case of returning a string and immediately passing it into
another function was not actually subject to that corruption risk, since
the HIL interpreter would just pass the string through verbatim, but this
is still now forbidden as a result of the stricter handling of file(...).

To avoid breaking these use-cases, here we introduce variants of the hash
functions a with "file" prefix that take a filename for a disk file to
hash rather than hashing the given string directly. The configuration
upgrade tool also now includes a rule to detect the documented idiom and
rewrite it into a single function call for one of these new functions.

This does cause a bit of function sprawl, but that seems preferable to
introducing more complex rules for when file(...) can and cannot read
binary files, making the behavior of these various functions easier to
understand in isolation.
2019-01-25 10:18:44 -08:00
Tom Harvey ec0419fe91
backend/azurerm: fixing the syntax highlighting (#20085) 2019-01-23 13:33:11 +00:00
Martin Atkins 2f8f7d6f4d lang/funcs: Type conversion functions
It's not normally necessary to make explicit type conversions in Terraform
because the language implicitly converts as necessary, but explicit
conversions are useful in a few specialized cases:

- When defining output values for a reusable module, it may be desirable
  to force a "cleaner" output type than would naturally arise from a
  computation, such as forcing a string containing digits into a number.
- Our 0.12upgrade mechanism will use some of these to replace use of the
  undocumented, hidden type conversion functions in HIL, and force
  particular type interpretations in some tricky cases.
- We've found that type conversion functions can be useful as _temporary_
  workarounds for bugs in Terraform and in providers where implicit type
  conversion isn't working correctly or a type constraint isn't specified
  precisely enough for the automatic conversion behavior.

These all follow the same convention of being named "to" followed by a
short type name. Since we've had a long-standing convention of running all
the words together in lowercase in function names, we stick to that here
even though some of these names are quite strange, because these should
be rarely-used functions anyway.
2019-01-17 10:01:47 -08:00
Martin Atkins da51e72cbb lang/functions: set functions from cty
The sethaselement, setintersection, and setunion functions are defined in
the cty stdlib. Making them available in Terraform will make it easier to
work with sets, and complement the currently-Terraform-specific setproduct
function.

In the long run setproduct should probably move into the cty stdlib too,
but since it was submitted as a Terraform function originally we'll leave
it here now for simplicity's sake and reorganize later.
2019-01-16 09:57:16 -08:00
Martin Atkins edb5f82de1 lang/funcs: Convert the "setproduct" function to the new approach
In our new world it produces either a set of a tuple type or a list of a
tuple type, depending on the given argument types.

The resulting collection's element tuple type is decided by the element
types of the given collections, allowing type information to propagate
even if unknown values are present.
2019-01-16 09:57:16 -08:00
Sander van Harmelen e08a7e979e backend/remote: use the correct test operation 2019-01-15 16:13:16 +01:00
Nick Fagerlund ac45e43fda website: Finish moving "writing custom providers" to /docs/extend/
This document was previously copied to the "Extending Terraform" section (in the
terraform-website repo), and the old URL was redirected so that the copy in
/guides can no longer be reached on the website. But the old copy of the file
remained, and now it runs the risk of confusing contributors, since the copy in
terraform-website/.../docs/extend is the more up-to-date version.
2019-01-11 16:36:40 -08:00
Brian Flad 05691a978e
Merge pull request #19951 from hashicorp/s3-backend-nosuchbucket-enhanced-error
backend/s3: Configure AWS Client MaxRetries and provide enhanced S3 NoSuchBucket error message
2019-01-11 10:04:16 -05:00
Chris Griggs 0cc057b873
Merge branch 'master' into cgriggs01-comm-5 2019-01-09 14:31:47 -08:00
Tom Harvey 177ca3711d
Adding the AzureAD entry to the providers page (#19946)
Adding the AzureAD entry to the providers page
2019-01-09 22:28:25 +00:00
cgriggs01 a9069cab7b edit fmt 2019-01-09 11:40:34 -08:00
cgriggs01 885a366f2e add <tr> 2019-01-09 11:21:02 -08:00
cgriggs01 830e23b217 add new community providers 2019-01-09 10:39:32 -08:00
Brian Flad ed37d07632
backend/s3: Configure AWS Client MaxRetries and provide enhanced S3 NoSuchBucket error message
The AWS Go SDK automatically provides a default request retryer with exponential backoff that is invoked via setting `MaxRetries` or leaving it `nil` will default to 3. The terraform-aws-provider `config.Client()` sets `MaxRetries` to 0 unless explicitly configured above 0. Previously, we were not overriding this behavior by setting the configuration and therefore not invoking the default request retryer.

The default retryer already handles HTTP error codes above 500, including S3's InternalError response, so the extraneous handling can be removed. This will also start automatically retrying many additional cases, such as temporary networking issues or other retryable AWS service responses.

Changes:
* s3/backend: Add `max_retries` argument
* s3/backend: Enhance S3 NoSuchBucket error to include additional information
2019-01-09 13:01:37 -05:00
Kristin Laemmert cdf7cc2449
command/json*: updating documentation and adding tests (#19944)
A few minor fixes and cleanups as a result of said tests. Hooray for
eventual consistency!
2019-01-09 08:59:11 -08:00
Martin Atkins d0e6a4c69a lang: Add "formatdate" function
We missed this one on a previous pass of bringing in most of the cty
stdlib functions.

This will resolve #17625 by allowing conversion from Terraform's
conventional RFC 3339 timestamps into various other formats.
2019-01-07 09:10:14 -08:00
Martin Atkins c753df6a93 lang/funcs: templatefile function
This function is similar to the template_file data source offered by the
template provider, but having it built in to the language makes it more
convenient to use, allowing templates to be rendered from files anywhere
an inline template would normally be allowed:

    user_data = templatefile("${path.module}/userdata.tmpl", {
      hostname = format("petserver%02d", count.index)
    })

Unlike the template_file data source, this function allows values of any
type in its variables map, passing them through verbatim to the template.
Its tighter integration with Terraform also allows it to return better
error messages with source location information from the template itself.

The template_file data source was originally created to work around the
fact that HIL didn't have any support for map values at the time, and
even once map support was added it wasn't very usable. With HCL2
expressions, there's little reason left to use a data source to render
a template; the only remaining reason left to use template_file is to
render a template that is constructed dynamically during the Terraform
run, which is a very rare need.
2018-12-21 08:06:14 -08:00
Nick Fagerlund 17873ae61b website: Update functions section page titles 2018-12-20 14:20:42 -08:00
Nick Fagerlund 9f85e6d5cc website: Update configuration language section page titles 2018-12-20 14:20:42 -08:00
Nick Fagerlund d2abdc23ac website: Structural edit of configuration language docs
This commit is a wide-ranging set of edits to the pages under
/docs/configuration. Among other things, it

- Separates style conventions out into their own page.
- Separates type constraints and conversion info into their own page.
- Conflates similar complex types a little more freely, since the distinction is
  only relevant when restricting inputs for a reusable module or resource.
- Clarifies several concepts that confused me during edits.
2018-12-20 14:20:42 -08:00
cgriggs01 da370cd689 fix spacing in website docs 2018-12-20 13:27:15 -08:00
cgriggs01 9df6e2df42 add hedvig provider links 2018-12-20 10:54:20 -08:00
cgriggs01 79f93ed695 links for ucloud provider 2018-12-19 13:11:31 -08:00
Sander van Harmelen da8e02eb2e
Merge pull request #19647 from hashicorp/svh/f-versions
core:  add a method to the disco package retrieve version constraints
2018-12-14 12:38:52 +01:00
Sander van Harmelen 268c0f85ce Add a method to retrieve version contraints 2018-12-14 12:17:31 +01:00
Brian Flad 058434d28b
Merge pull request #19571 from hashicorp/f-backend-s3-other-endpoints
backend/s3: Support DynamoDB, IAM, and STS endpoint configurations
2018-12-10 19:28:01 -05:00
Tom Harvey 383bc98f5c
backend/azurerm: Support for authenticating using the Azure CLI (#19465)
* Upgrading to 2.0.0 of github.com/hashicorp/go-azure-helpers

* Support for authenticating using Azure CLI

* backend/azurerm: support for authenticating using the Azure CLI
2018-12-10 21:23:30 +00:00
cgriggs01 41897bb437 six new community providers 2018-12-10 11:13:45 -08:00
Brian Flad 9a3b02cd6c
backend/s3: Support DynamoDB, IAM, and STS endpoint configurations
This change enables a few related use cases:
* AWS has partitions outside Commercial, GovCloud (US), and China, which are the only endpoints automatically handled by the AWS Go SDK. DynamoDB locking and credential verification can not currently be enabled in those regions.
* Allows usage of any DynamoDB-compatible API for state locking
* Allows usage of any IAM/STS-compatible API for credential verification
2018-12-07 03:10:51 -05:00
cgriggs01 808c76fa9b Add skytap links 2018-12-05 15:38:47 -08:00
Chris Griggs 89347e9c81
Updates path name 2018-11-28 16:55:03 -08:00
cgriggs01 c943083a4b selvpc -> selectel 2018-11-28 13:30:48 -08:00
Daniel Schroeder 65080b9ce3 website: Fix redundant "be" in workspaces documentation 2018-11-28 07:59:37 -08:00
Chris Griggs e32c5de440
Merge pull request #19390 from cgriggs01/cgriggs01-selvpc
[Website] New certified and community provider
2018-11-26 13:03:50 -08:00
cgriggs01 f013942150 formatting edits 2018-11-26 13:01:55 -08:00
Tom Harvey 6d4f702467
backend/azurerm: support for custom resource manager endpoints (#19460)
* backend/azurerm: removing the `arm_` prefix from keys

* removing the deprecated fields test because the deprecation makes it fail

* authentication: support for custom resource manager endpoints

* Adding debug prefixes to the log statements
2018-11-26 14:42:16 +01:00
Tom Harvey d580f30e03
backend/azurerm: removing the `arm_` prefix from keys (#19448)
* backend/azurerm: removing the `arm_` prefix from keys

* removing the deprecated fields test because the deprecation makes it fail
2018-11-26 11:19:43 +01:00
Tom Harvey 96b1c951fa
backend/azurerm: support for authenticating via SAS Tokens (#19440)
* adding acceptance tests for msi auth

* including the resource group name in the tests

* backend/azurerm: support for authenticating using a SAS Token

* resolving merge conflicts

* moving the defer to prior to the error
2018-11-22 18:02:33 +01:00
Tom Harvey c928962f44
backend/azurerm: support for authenticating via msi (#19433)
* backend/azurerm: support for authenticating via msi

* adding acceptance tests for msi auth

* including the resource group name in the tests

* support for using the test client via msi
2018-11-22 16:52:27 +01:00
Tom Harvey 0ec109bdc0
backend/azurerm: upgrading the SDK / support for proxies (#19414)
* vendor updates

- updating to v21.3.0 of github.com/Azure/azure-sdk-for-go
- updating to v10.15.4 of github.com/Azure/go-autorest
- vendoring github.com/hashicorp/go-azure-helpers @ 0.1.1

* backend/azurerm: refactoring to use the new auth package

- refactoring the backend to use a shared client via the new auth package
- adding tests covering both Service Principal and Access Key auth
- support for authenticating using a proxy
- rewriting the backend documentation to include examples of both authentication types

* switching to use the build-in logging function

* documenting it's also possible to retrieve the access key from an env var
2018-11-21 22:06:03 +01:00
Nick Fagerlund 7d5db9522f
website: Fix plugin path on Windows (#19423)
...and one other reference to the application data directory.

Context:

https://docs.microsoft.com/en-us/windows/desktop/shell/knownfolderid#folderid_roamingappdata

In newer Windows versions, the folder accessible as `%APPDATA%` (and via various
APIs) is actually at something like "documents and settings\user\application
data\roaming", while earlier versions omit the "\roaming" part of the path. This
means you can confuse people by referring to the "application data" directory by
its human name, because "roaming" is the real application data directory, but it
looks like a subdirectory of "application data".

Thus, it's less confusing to just use the `%APPDATA%` variable, with the added
benefit that you can copy and paste the path and it'll just work in most places.
2018-11-20 16:54:18 -08:00
cgriggs01 28de3823ac new provider and community links 2018-11-16 11:35:33 -08:00
Kristin Laemmert 1be09745d7
Merge pull request #19007 from cardoe/fix-swift-backend-docs
Fix backend/swift docs
2018-11-08 06:34:09 -08:00
Nick Fagerlund 3b02214d8c
website: clarify behavior of terraform_remote_state data sources (#19227) 2018-11-07 14:01:58 -08:00
Tom Harvey 49c42b9830
docs: add warning to backend/swift about auto-expire
If the user uses the auto-expire value in the backend/swift settings
then swift will automatically delete their Statefile which is likely
something the user doesn't want given how Terraform works.
2018-11-06 18:18:46 -06:00
Doug Goldstein 817be7b23f
website: update backend/swift examples to use versioning
Since object versioning is a best practice the docs should have all the
examples containing it by default.
2018-11-06 18:18:46 -06:00
Doug Goldstein 58cb47d108
website: backend/swift fix bad link for object versioning
The displayed link said `expire_after` but really is a link to
`archive_container` so update the link to read the right data.
fixes #19005
2018-11-06 18:18:46 -06:00
Doug Goldstein ddc30b6546
website: backend/swift add docs link for expire_after
Provide a link to the OpenStack Swift docs for the object expiration
feature that is used by the `expire_after` field.
2018-11-06 18:18:45 -06:00
Doug Goldstein b31aab4469
website: fix backend/swift links to Swift docs
The links to the OpenStack Swift documentation were broken due to
changes on the OpenStack website.
2018-11-06 18:18:45 -06:00
Doug Goldstein 027b107268
website: update deprecated backend/swift docs
Update the examples and docs to not directly reference deprecated
fields.
2018-11-06 18:18:45 -06:00