Commit Graph

432 Commits

Author SHA1 Message Date
James Bardin 6cc3e1d0bd move init error to where it is generated
The init error was output deep in the backend by detecting a
special ResourceProviderError and formatted directly to the CLI.

Create some Diagnostics closer to where the problem is detected, and
passed that back through the normal diagnostic flow. While the output
isn't as nice yet, this restores the helpful error message and makes the
code easier to maintain. Better formatting can be handled later.
2019-02-20 14:18:37 -05:00
Brian Flad 185a3304e5
Merge pull request #20374 from hashicorp/td-backend-s3-dependencies
backend/s3: Switch from github.com/terraform-providers/terraform-provider-aws to github.com/hashicorp/aws-sdk-go-base
2019-02-20 08:42:07 -08:00
Brian Flad a41e545198
backend/s3: Reinstate region validation and update copy for skip_requesting_account_id deprecation message 2019-02-20 08:29:09 -08:00
Sander van Harmelen 43eb7c08e0
Merge pull request #20379 from hashicorp/svh/f-state-service
backend/remote: use `state.v2` for remote state only
2019-02-20 10:48:05 +01:00
Brian Flad 43f12bbfe0
backend/s3: Update copy for skip_get_ec2_platforms and skip_requesting_account_id argument deprecations 2019-02-19 10:54:53 -08:00
Sander van Harmelen 54736b068b backend/remote: use `state.v2` for remote state only
The API surface area is much smaller when we use the remote backend for remote state only.

So in order to try and prevent any backwards incompatibilities when TF runs inside of TFE, we’ve split up the discovery services into `state.v2` (which can be used for remote state only configurations, so when running in TFE) and `tfe.v2.1` (which can be used for all remote configurations).
2019-02-19 10:59:51 +01:00
Brian Flad 1aaac172b0
backend/s3: Switch from github.com/terraform-providers/terraform-provider-aws to github.com/hashicorp/aws-sdk-go-base
Output from acceptance testing (no new failures):

```
--- PASS: TestBackend_impl (0.00s)
--- PASS: TestBackendConfig (0.37s)
--- PASS: TestBackendConfig_invalidKey (0.00s)
--- PASS: TestBackend (3.26s)
--- PASS: TestBackendLocked (6.80s)
--- FAIL: TestBackendExtraPaths (2.32s)
--- PASS: TestBackendPrefixInWorkspace (2.06s)
--- PASS: TestKeyEnv (8.20s)
--- PASS: TestRemoteClient_impl (0.00s)
--- PASS: TestRemoteClient (2.42s)
--- PASS: TestRemoteClientLocks (6.33s)
--- PASS: TestForceUnlock (13.31s)
--- PASS: TestRemoteClient_clientMD5 (11.75s)
--- PASS: TestRemoteClient_stateChecksum (10.07s)
```
2019-02-18 02:30:30 -05:00
Martin Atkins 6eb7bfbdfb
Merge #20265: Don't presume unknown for values unset in config
This changes the contract for `PlanResourceChange` so that the provider is now responsible
for populating all default values during plan, including inserting any unknown values for
defaults it will fill in at apply time.
2019-02-08 13:54:55 -08:00
Martin Atkins e3618f915b backend/local: Fix mock provider in tests
We've changed the contract for PlanResourceChange to now require the
provider to populate any default values (including unknowns) it wants to
set for computed arguments, so our mock provider here now needs to be a
little more complex to deal with that.

This fixes several of the tests in this package. A minor change to
TestLocal_applyEmptyDirDestroy was required to make it properly configure
the mock provider so PlanResourceChange can access the schema.
2019-02-08 12:48:32 -08:00
Radek Simko e6777105b7
Merge pull request #20275 from hashicorp/vendor-openstack-upgrade
vendor: Bump openstack provider to v1.15.0
2019-02-08 16:22:35 +00:00
Sander van Harmelen aefbec63b1 backend/remote: update the test logic 2019-02-08 16:56:37 +01:00
Sander van Harmelen 3b80f69eec
Merge pull request #20242 from hashicorp/svh/b-scanner-buffer-v0.12
backend/remote: fix bufio.Scanner: token too long
2019-02-08 16:52:22 +01:00
Radek Simko a7f0722729
backend/swift: Fix interface after upgrade 2019-02-08 11:03:03 +00:00
Sander van Harmelen 47a00ea34b backend/remote: cleanup test connections
Cleanup test connection to prevent file descriptor issues when running the tests on a Mac.
2019-02-07 09:55:19 +01:00
Sander van Harmelen 5249d0fe83 backend/remote: fix bufio.Scanner: token too long 2019-02-07 09:54:43 +01:00
Radek Simko 6365fffe4f
Fix interface after upgrade 2019-02-01 10:48:56 +00:00
Sander van Harmelen 6dd9f22780
Merge pull request #20086 from hashicorp/svh/f-service-constraints
backend/remote: make sure we show the correct error
2019-01-24 20:22:20 +01:00
Sander van Harmelen 1e4c20686e backend/remote: make sure we show the correct error
Previously we would show two errors when there was a version constraint
error. But of course one is enough.
2019-01-23 15:09:42 +01:00
Martin Atkins 10bf4c763b backend: Undeclared variables in -var-file is a warning, not an error
In Terraform 0.11 and earlier we just silently ignored undeclared
variables in -var-file and the automatically-loaded .tfvars files. This
was a bad user experience for anyone who made a typo in a variable name
and got no feedback about it, so we made this an error for 0.12.

However, several users are now relying on the silent-ignore behavior for
automation scenarios where they pass the same .tfvars file to all
configurations in their organization and expect Terraform to ignore any
settings that are not relevant to a specific configuration. We never
intentionally supported that, but we don't want to immediately break that
workflow during 0.12 upgrade.

As a compromise, then, we'll make this a warning for v0.12.0 that contains
a deprecation notice suggesting to move to using environment variables
for this "cross-configuration variables" use-case. We don't produce errors
for undeclared variables in environment variables, even though that
potentially causes the same UX annoyance as ignoring them in vars files,
because environment variables are assumed to live in the user's session
and this it would be very inconvenient to have to unset such variables
when moving between directories. Their "ambientness" makes them a better
fit for these automatically-assigned general variable values that may or
may not be used by a particular configuration.

This can revert to being an error in a future major release, after users
have had the opportunity to migrate their automation solutions over to
use environment variables.

We don't seem to have any tests covering this specific situation right
now. That isn't ideal, but this change is so straightforward that it would
be relatively expensive to build new targeted test cases for it and so
I instead just hand-tested that it is indeed now producing a warning where
we were previously producing an error. Hopefully if there is any more
substantial work done on this codepath in future that will be our prompt
to add some unit tests for this.
2019-01-22 07:46:51 -08:00
Sander van Harmelen 8937fedb76 backend/remote: fix a small typo 2019-01-18 19:40:57 +01:00
Sander van Harmelen e08a7e979e backend/remote: use the correct test operation 2019-01-15 16:13:16 +01:00
Martin Atkins 0c0a437bcb Move module install functionality over to internal/initwd 2019-01-14 11:33:21 -08:00
Brian Flad ed37d07632
backend/s3: Configure AWS Client MaxRetries and provide enhanced S3 NoSuchBucket error message
The AWS Go SDK automatically provides a default request retryer with exponential backoff that is invoked via setting `MaxRetries` or leaving it `nil` will default to 3. The terraform-aws-provider `config.Client()` sets `MaxRetries` to 0 unless explicitly configured above 0. Previously, we were not overriding this behavior by setting the configuration and therefore not invoking the default request retryer.

The default retryer already handles HTTP error codes above 500, including S3's InternalError response, so the extraneous handling can be removed. This will also start automatically retrying many additional cases, such as temporary networking issues or other retryable AWS service responses.

Changes:
* s3/backend: Add `max_retries` argument
* s3/backend: Enhance S3 NoSuchBucket error to include additional information
2019-01-09 13:01:37 -05:00
Sander van Harmelen bba03384d5 backend/remote: log early to indicate execution started 2019-01-08 17:06:24 +01:00
Sander van Harmelen 7b51af72b2 backend/remote: compare versions without the prerelease 2018-12-19 19:06:22 +01:00
Sander van Harmelen 55b6153b04 backend/remote: fix an error that prevents checking constraints 2018-12-15 21:36:47 +01:00
Sander van Harmelen 2ef8315885
Merge pull request #19659 from hashicorp/svh/f-check-constraints
backend/remote: return detailed incompatibility info
2018-12-14 21:26:02 +01:00
Sander van Harmelen 8f04e93739 backend/remote: return detailed incompatibility info 2018-12-14 21:11:41 +01:00
Radek Simko f64978b64c
backend/local: Render CBD replacement (+/-) correctly (#19642)
* backend/local: Render CBD replacement (+/-) correctly

* command/format: Use IsReplace helper function
2018-12-14 13:45:47 +00:00
Sander van Harmelen da8e02eb2e
Merge pull request #19647 from hashicorp/svh/f-versions
core:  add a method to the disco package retrieve version constraints
2018-12-14 12:38:52 +01:00
Sander van Harmelen 268c0f85ce Add a method to retrieve version contraints 2018-12-14 12:17:31 +01:00
Radek Simko 3ab4739ba4
backend/local: Avoid rendering data sources on destroy 2018-12-12 18:21:49 +00:00
Radek Simko e55079abc9
backend/local: Fix incorrect destroy/update count on apply 2018-12-11 18:06:53 +00:00
Brian Flad 058434d28b
Merge pull request #19571 from hashicorp/f-backend-s3-other-endpoints
backend/s3: Support DynamoDB, IAM, and STS endpoint configurations
2018-12-10 19:28:01 -05:00
Tom Harvey 383bc98f5c
backend/azurerm: Support for authenticating using the Azure CLI (#19465)
* Upgrading to 2.0.0 of github.com/hashicorp/go-azure-helpers

* Support for authenticating using Azure CLI

* backend/azurerm: support for authenticating using the Azure CLI
2018-12-10 21:23:30 +00:00
Sander van Harmelen a5a2156584 core: enhance service discovery
This PR improves the error handling so we can provide better feedback about any service discovery errors that occured.

Additionally it adds logic to test for specific versions when discovering a service using `service.vN`. This will enable more informational errors which can indicate any version incompatibilities.
2018-12-10 20:52:05 +01:00
Brian Flad 9a3b02cd6c
backend/s3: Support DynamoDB, IAM, and STS endpoint configurations
This change enables a few related use cases:
* AWS has partitions outside Commercial, GovCloud (US), and China, which are the only endpoints automatically handled by the AWS Go SDK. DynamoDB locking and credential verification can not currently be enabled in those regions.
* Allows usage of any DynamoDB-compatible API for state locking
* Allows usage of any IAM/STS-compatible API for credential verification
2018-12-07 03:10:51 -05:00
Sander van Harmelen 9062d887b8 backend/remote: use entitlements to select backends
Use the entitlements to a) determine if the organization exists, and b) as a means to select which backend to use (the local backend with remote state, or the remote backend).
2018-12-05 12:29:08 +01:00
Kristin Laemmert 67aa944166 backend/local: decode variables with cty.DynamicPseudoType
Variables values are marshalled with an explicit type of
cty.DynamicPseudoType, but were being decoded using `Implied Type` to
try and guess the type. This was causing errors because `Implied Type`
does not expect to find a late-bound value.
2018-11-30 15:15:28 -08:00
Martin Atkins 12572e97bc core: Automatically upgrade resource instance states on read
If an instance object in state has an earlier schema version number then
it is likely that the schema we're holding won't be able to decode the
raw data that is stored. Instead, we must ask the provider to upgrade it
for us first, which might also include translating it from flatmap form
if it was last updated with a Terraform version earlier than v0.12.

This ends up being a "seam" between our use of int64 for schema versions
in the providers package and uint64 everywhere else. We intend to
standardize on int64 everywhere eventually, but for now this remains
consistent with existing usage in each layer to keep the type conversion
noise contained here and avoid mass-updates to other Terraform components
at this time.

This also includes a minor change to the test helpers for the
backend/local package, which were inexplicably setting a SchemaVersion of
1 on the basic test state but setting the mock schema version to zero,
creating an invalid situation where the state would need to be downgraded.
2018-11-30 11:22:39 -08:00
Sander van Harmelen fe05609c5e backend/remote: support the new force-unlock API
Add support for the new `force-unlock` API and at the same time improve
performance a bit by reducing the amount of API calls made when using
the remote backend for state storage only.
2018-11-30 19:39:18 +01:00
Sander van Harmelen 4c878db8fd
Merge pull request #19464 from hashicorp/svh/f-context
backend/remote: implement the Local interface
2018-11-28 20:16:58 +01:00
Sander van Harmelen a061725132
Merge pull request #19403 from hashicorp/svh/f-policies
backend/remote: also show policies when there are no changes
2018-11-28 16:30:31 +01:00
Martin Atkins 168d84b3c4 core: Make resource type schema versions visible to callers
Previously we were fetching these from the provider but then immediately
discarding the version numbers because the schema API had nowhere to put
them.

To avoid a late-breaking change to the internal structure of
terraform.ProviderSchema (which is constructed directly all over the
tests) we're retaining the resource type schemas in a new map alongside
the existing one with the same keys, rather than just switching to
using the providers.Schema struct directly there.

The methods that return resource type schemas now return two arguments,
intentionally creating a little API friction here so each new caller can
be reminded to think about whether they need to do something with the
schema version, though it can be ignored by many callers.

Since this was a breaking change to the Schemas API anyway, this also
fixes another API wart where there was a separate method for fetching
managed vs. data resource types and thus every caller ended up having a
switch statement on "mode". Now we just accept mode as an argument and
do the switch statement within the single SchemaForResourceType method.
2018-11-27 15:53:54 -08:00
Sander van Harmelen 35d9ce3f92 backend/remote: implement the Local interface 2018-11-26 20:50:25 +01:00
Tom Harvey 6d4f702467
backend/azurerm: support for custom resource manager endpoints (#19460)
* backend/azurerm: removing the `arm_` prefix from keys

* removing the deprecated fields test because the deprecation makes it fail

* authentication: support for custom resource manager endpoints

* Adding debug prefixes to the log statements
2018-11-26 14:42:16 +01:00
Tom Harvey d580f30e03
backend/azurerm: removing the `arm_` prefix from keys (#19448)
* backend/azurerm: removing the `arm_` prefix from keys

* removing the deprecated fields test because the deprecation makes it fail
2018-11-26 11:19:43 +01:00
Tom Harvey 96b1c951fa
backend/azurerm: support for authenticating via SAS Tokens (#19440)
* adding acceptance tests for msi auth

* including the resource group name in the tests

* backend/azurerm: support for authenticating using a SAS Token

* resolving merge conflicts

* moving the defer to prior to the error
2018-11-22 18:02:33 +01:00
Tom Harvey e437feadc2
backend/azurerm: fixing a bug where locks couldn't be unlocked (#19441)
Fixes #17046
2018-11-22 17:47:35 +01:00
Tom Harvey c928962f44
backend/azurerm: support for authenticating via msi (#19433)
* backend/azurerm: support for authenticating via msi

* adding acceptance tests for msi auth

* including the resource group name in the tests

* support for using the test client via msi
2018-11-22 16:52:27 +01:00