Commit Graph

584 Commits

Author SHA1 Message Date
Radek Simko 7860f55e4f
Version tools per Go convention under tools.go 2019-10-17 22:23:39 +02:00
Pam Selle a5f2f4cbba Update messaging for .terraformignore 2019-10-17 11:12:00 -04:00
Radek Simko 32f9722d9d
Replace import paths & set UA string where necessary 2019-10-11 22:40:54 +01:00
Martin Atkins 8664749b59 backend: Allow certain commands to opt out of required variable checks
Terraform Core expects all variables to be set, but for some ancillary
commands it's fine for them to just be set to placeholders because the
variable values themselves are not key to the command's functionality
as long as the terraform.Context is still self-consistent.

For such commands, rather than prompting for interactive input for
required variables we'll just stub them out as unknowns to reflect that
they are placeholders for values that a user would normally need to
provide.

This achieves a similar effect to how these commands behaved before, but
without the tendency to produce a slightly invalid terraform.Context that
would fail in strange ways when asked to run certain operations.
2019-10-10 10:07:01 -07:00
Martin Atkins e21f0fa61e backend/local: Handle interactive prompts for variables in UI layer
During the 0.12 work we intended to move all of the variable value
collection logic into the UI layer (command package and backend packages)
and present them all together as a unified data structure to Terraform
Core. However, we didn't quite succeed because the interactive prompts
for unset required variables were still being handled _after_ calling
into Terraform Core.

Here we complete that earlier work by moving the interactive prompts for
variables out into the UI layer too, thus allowing us to handle final
validation of the variables all together in one place and do so in the UI
layer where we have the most context still available about where all of
these values are coming from.

This allows us to fix a problem where previously disabling input with
-input=false on the command line could cause Terraform Core to receive an
incomplete set of variable values, and fail with a bad error message.

As a consequence of this refactoring, the scope of terraform.Context.Input
is now reduced to only gathering provider configuration arguments. Ideally
that too would move into the UI layer somehow in a future commit, but
that's a problem for another day.
2019-10-10 10:07:01 -07:00
Martin Atkins 39e609d5fd vendor: switch to HCL 2.0 in the HCL repository
Previously we were using the experimental HCL 2 repository, but now we'll
shift over to the v2 import path within the main HCL repository as part of
actually releasing HCL 2.0 as stable.

This is a mechanical search/replace to the new import paths. It also
switches to the v2.0.0 release of HCL, which includes some new code that
Terraform didn't previously have but should not change any behavior that
matters for Terraform's purposes.

For the moment the experimental HCL2 repository is still an indirect
dependency via terraform-config-inspect, so it remains in our go.sum and
vendor directories for the moment. Because terraform-config-inspect uses
a much smaller subset of the HCL2 functionality, this does still manage
to prune the vendor directory a little. A subsequent release of
terraform-config-inspect should allow us to completely remove that old
repository in a future commit.
2019-10-02 15:10:21 -07:00
Collin J. Doering 6d838ed76c
Allow s3 backends to contain more then 1000 workspaces
* backend/remote-state/s3/backend_state.go: Prior to this commit, the terraform s3 backend did
  not paginate calls to s3 when finding workspaces, which resulted in workspaces 'disappearing'
  once they are switched away from, even though the state file still exists. This is due to the
  ListBucket operation defaulting MaxItems to 1000, so terraform s3 backends that contained
  more then 1000 workspaces did not function as expected. This rectifies this situation by
  paginating calls to s3 when finding workspaces.

Signed-off-by: Collin J. Doering <collin@rekahsoft.ca>
2019-10-01 11:46:34 -04:00
Paul Thrasher f2ef8ef317
slow down tfce polling to 1s
Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-09-30 14:22:15 -07:00
Paul Thrasher 4e308ef362
typos. some code, some text.
Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-09-30 13:44:23 -07:00
Paul Thrasher d2eaffabea
vendor latest go-tfe
Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-09-27 15:13:40 -07:00
Paul Thrasher d3fc3dee6e
clean up go mod for go-tfe
Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-09-27 14:16:41 -07:00
Paul Thrasher e2831b3d1e
tfce test additions
Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-09-27 14:16:41 -07:00
Paul Thrasher bc2d888b7b
update to go-tfe 0.3.23
this fixes the attr types

Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-09-27 14:16:05 -07:00
Paul Thrasher 39817ac476
cost estimation status polling
Properly wait for cost estimation to finish running before outputting
the results. Waits 500 milliseconds between checks, rather than backing
off exponentially, because we are not in a run queue. At the point we're
waiting, we expect cost estimation to be run in a timely manner.
2019-09-27 14:16:05 -07:00
Paul Thrasher 979bba0f32
tfce mock fix
Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-09-27 14:11:49 -07:00
Kristin Laemmert 006ef022e6
Merge pull request #22918 from hashicorp/mildwonkey/to-e2e-or-not-e2e
Sundry fixes to e2e tests
2019-09-27 08:46:04 -04:00
Kristin Laemmert 68dfc3046d backend/remote-state: etcdv3, oss, and manta acc tests should fail
faster

The acceptance tests for etcdv3, oss and manta were not validating
required env variablea, chosing to assume that if one was running
acceptance tests they had already configured the credentials.

It was not always clear if this was a bug in the tests or the provider,
so I opted to make the tests fail faster when required attributes were
unset (or "").
2019-09-27 08:45:12 -04:00
Kristin Laemmert 5300f85a79 backend/manta: fix panic when insecure_skip_tls_verify was not set
The DefaultFunc for insecure_skip_tls_verify was sending an empty string
instead of a bool. Fixes to default to `false`.
2019-09-26 10:08:29 -04:00
Paddy 9b24ef7870
Merge pull request #21772 from luis-silva/21680/GCS_OAUTH
Add OAuth2 token support for GCS backend
2019-09-25 16:09:35 -07:00
Martin Atkins 7e29b9b5d4 core: Warn when creating and applying with -target
The documentation for the -target option warns that it's intended for
exceptional circumstances only and not for routine use, but that's not a
very prominent location for that warning and so some users miss it.

Here we make the warning more prominent by including it directly in the
Terraform output when -target is in use. We first warn during planning
that the plan might be incomplete, and then warn again after apply
concludes and direct the user to run "terraform plan" to make sure that
there are no further changes outstanding. The latter message is intended
to reinforce that -target should only be a one-off operation and that you
should always run without it soon after to ensure that the workspace is
left in a consistent, converged state.
2019-09-17 14:36:05 -07:00
Luis Silva 9ae8eca55d
Merge branch 'master' into 21680/GCS_OAUTH 2019-08-27 16:56:27 +01:00
yanndegat be5280e4e1 remote-state/pg: add option to skip schema creation (#21607)
* add `skip_schema_creation` option
* add sanity check to avoid situations where postgres users
  hasn't been granted the "CREATE SCHEMA" right

closes #21604

Signed-off-by: yann degat <yann@2kmail.net>
2019-08-27 11:14:32 -04:00
yuanye b69c0b4199 oss backend support profile 2019-08-23 21:04:34 +08:00
James Bardin 10d94fb764
Merge pull request #21967 from williams-brian/SSE-C_Remote_State
Add support for SSE-C to S3 backend
2019-08-22 17:30:28 -04:00
Radek Simko f3357aad45
vendor: Downgrade Azure dependencies
This is to allow Terraform providers to upgrade to at least
one more minor version of the plugin SDK without major UX hiccups.

This concludes (unsuccessful) experiments involving upgrades
to SDK with https://github.com/Azure/go-autorest/pull/455

Even with that patch all providers still experience broken UX
as described in https://github.com/hashicorp/terraform/pull/22490

This downgrade reduces the uncomfort to only a handful of providers
from >100s. The affected providers more or less directly depend on
Azure SDK(s), which is ~8.

Affected providers practically cannot consume Terraform Plugin SDK
with this patch (downgraded Azure SDKs) and can just wait for
extracted Terraform Plugin SDK which is planned to be released soon.

This reverts the following PRs:

 - https://github.com/hashicorp/terraform/pull/22247
 - https://github.com/hashicorp/terraform/pull/22248
 - https://github.com/hashicorp/terraform/pull/22524
 - https://github.com/hashicorp/terraform/pull/22525

and it is otherwise result of the following commands

```
go get github.com/Azure/azure-sdk-for-go@v21.3.0
go get github.com/hashicorp/go-azure-helpers@166dfd221bb2
go mod tidy
```
2019-08-20 15:52:11 +01:00
Radek Simko dee450cf8c
vendor: Bump go-azure-helpers to 0.7.0 2019-08-20 09:32:02 +01:00
Roberto Jung Drebes b459a92575 backend/gcs: Use new endpoints for Google OAuth
This is a consequence of googleapis/google-cloud-common#260.
2019-08-13 15:24:34 -07:00
Pam Selle 9631e4c73d
Merge pull request #20571 from sergkondr/fix_misspelling
fix misspelling
2019-08-13 17:13:13 -04:00
Alex Pilon 4bf43efcfd
move hcl2shim package to configs 2019-08-06 19:58:58 -04:00
Chris Marchesi d43fc71135
modules: update go-azure-helpers to v0.5.0
To help address the issues posed on #22087 and #22085.
2019-08-01 11:57:34 -07:00
He Guimin a490dfa495 backend/oss: Support for assume role config 2019-07-30 23:27:17 +08:00
Kristin Laemmert 412d459292
backend/remote: remove milseading contents from error message (#22148)
Previously, terraform was returning a potentially-misleading error
message in response to anything other than a 404 from the
b.client.Workspaces.Read operation. This PR simplifies Terraform's error
message with the intent of encouraging those who encounter it to focus
on the error message returned from the tfe client.

The added test is odd, and a bit hacky, and possibly overkill.
2019-07-22 09:06:39 -04:00
Kristin Laemmert 190ef537ec backend/remote: notify users when uploading something other than cwd
When a TFC workspace is configured without a VCS root, and with a
working directory, and a user is running `terraform init` from that same
directory, TFC uploads the entire configuration directory, not only the
user's cwd. This is not obvious to the user, so we are adding a descriptive
message explaining what is being uploaded, and why.
2019-07-18 09:50:17 -07:00
Kristin Laemmert 89eeaed0a0
[WIP] backend/enhanced: start with absolute configuration path (#22096)
* backend/enhanced: start with absolute config path

We recently started normalizing the config path before all "command"
operations, which was necessary for consistency but had unexpected
consequences for remote backend operations, specifically when a vcs root
with a working directory are configured.

This PR de-normalizes the path back to an absolute path.

* Check the error and add a test

It turned out all required logic was already present, so I just needed to add a test for this specific use case.
2019-07-17 08:39:37 -04:00
Pam Selle 23a187d85d
Merge pull request #21739 from hasheddan/s3-backend-comment-typo
s3 backend: minor typo in putMD5 comment
2019-07-12 14:01:26 -04:00
Colin Fowler cd7bfba141 rebased to terraform master branch 2019-07-10 18:05:10 +01:00
Brian Williams 5e3c3bafb8 Add support for SSE-C to S3 backend
These changes add support for encrypting terraform remote-state in S3 using customer-supplied encryption keys (SSE-C).
2019-07-06 10:10:54 -05:00
Luis Silva f6c90c1d96 Add OAuth2 token support for GCS backend 2019-07-05 10:06:44 +01:00
Radek Simko 5b9f2fafc8 Standardise directory name for test data 2019-06-30 10:16:15 +02:00
Daniel Mangum 3bc14620c0 backend/remote-state: remove dead code
This remote-state adapter is no longer used, because the old remote state
mechanism was removed in Terraform v0.12.
2019-06-18 14:18:05 -07:00
hasheddan 961d43142c s3 backend: minor typo in putMD5 comment
Signed-off-by: hasheddan <georgedanielmangum@gmail.com>
2019-06-15 08:20:37 -05:00
Mary Cutrali e44ca40702 update remote apply language to use proper its/it's 2019-06-14 12:22:21 -05:00
Pam Selle cb593c25aa
Merge pull request #21381 from davidcelis/remove-cost-estimation
Remove pre-alpha cost estimation logic from remote backend
2019-06-13 10:59:56 -04:00
kayrus d06609dd23 Swift backend: add application credential support 2019-06-12 17:23:13 +02:00
Joe Topjian d8343aa95f backend/swift: Authentication updates
Support for cross-domain authentication has been added and mapping
environment variables to the correct domain settings has been
fixed.

In addition, support for clouds.yaml files has been added.
2019-06-12 08:05:54 -07:00
James Bardin 0c91d227fa
Merge branch 'master' into master 2019-06-10 15:50:59 -04:00
Ivan Kalita 5b6b1663ef backend/http: implement retries for the http backend (#19702)
Fixes #19619
2019-06-05 16:12:07 -04:00
Martin Atkins b1213f7f6c backend/local: don't panic when an instance has only a deposed object
This unusual situation isn't supposed to arise in normal use, but it can
come up in practice in some edge-case scenarios where Terraform fails in
a severe way during a create_before_destroy.

Some earlier versions of Terraform also had bugs in their handling of
deposed objects, so this may also arise if upgrading from one of those
older versions with some leftover deposed objects in the state.
2019-06-04 09:23:29 -07:00
Mateusz Gozdek b6de825723 backend/gcs: make Unlock error message more clear 2019-05-27 20:07:14 -07:00
David Celis 5ccccfb91f
Remove pre-alpha cost estimation logic 2019-05-20 16:48:38 -07:00
Radek Simko 8a6d1d62b6
stringer: Regenerate files with latest version 2019-05-13 15:34:27 +01:00
Stephen Buergler fc5b186e8d Don't leak so many connections in the pg backend
This change fixes an error I get:
Error: pq: too many connections for role "asdf"
because I can only have so many connections.
2019-04-30 23:34:51 -05:00
Sander van Harmelen 394f20f59c backend/remote: do not unlock after a failed upload
When changes are made and we failed to upload the state, we should not
try to unlock the workspace. Leaving the workspace locked is a good
indication something went wrong and also prevents other changes from
being applied before the newest state is properly uploaded.

Additionally we now output the lock ID when a lock or force-unlock
action failed.
2019-04-29 21:23:33 +02:00
Paul Thrasher 151c91ffda
use scanner for reading logs
Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-04-25 11:17:08 -07:00
Paul Thrasher 0e27a8862f
remove duplicate remote output line
Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-04-25 10:52:19 -07:00
Sander van Harmelen bb12206bca Fixup the tests 2019-04-25 10:32:00 +02:00
Sander van Harmelen 7cf744241a Do not use a scanner to read the logs
Using a scanner can cause issues when reading long lines. Also make sure we return the error correctly while planning.
2019-04-25 09:51:52 +02:00
Paul Thrasher c7a023a95c
update test for new go-tfe version
Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-04-24 16:46:17 -07:00
Paul Thrasher 53f977bee2
TFCE api correct pluralization
Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-04-24 16:44:00 -07:00
Paul Thrasher e479bd5dc3
update to latest go-tfe
same version number but pointing to a new sha

Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-04-24 16:44:00 -07:00
Sander van Harmelen 90bc237b7b Prevent a panic caused by writing to a nil map 2019-04-11 14:34:14 +02:00
He Guimin 3f44dd9dec Add tablestore config to store state lock 2019-04-11 07:11:10 +08:00
He Guimin b887d44712 Add Alibaba Cloud backend OSS with lock 2019-04-11 07:06:43 +08:00
Sander van Harmelen 39a95e4222 backend/remote: correctly load remote variables
When using `terraform console` in combination with the remote backend, variables defined in Terraform Enterprise were load loaded correctly.
2019-03-28 17:23:48 +01:00
Sander van Harmelen 57f6e01830 backend/local: preserve serial and lineage on failure
When failing to write the state, the local backend writes the state to a local file called `errrored.tfstate`. Previously it would do so by creating a new state file which would use a new serial and lineage. By exorting the existing state file and directly assigning the new state, the serial and lineage are preserved.
2019-03-27 16:15:16 +01:00
Justin Downing 1e32ae243c grammatical updates to comments and docs (#20195) 2019-03-21 14:05:41 -07:00
Stefan Schmidt 3ca1253796 backend/gcs: Mark the unused attributes project and region as removed.
These two attributes are not used since aec45e6967.
2019-03-19 17:29:04 -07:00
Sander van Harmelen 9f6a126293 backend/remote: check for external updates 2019-03-08 19:18:07 +01:00
Sander van Harmelen 4628fbcc65
Merge pull request #20615 from hashicorp/svh/f-input-context
core: add a context to the UIInput interface
2019-03-08 18:44:28 +01:00
Sander van Harmelen e75e845804
Merge pull request #20588 from hashicorp/svh/f-retry
backend/remote: also retry on server errors
2019-03-08 18:17:29 +01:00
Sander van Harmelen 973e2a7cf9 core: add a context to the UIInput interface 2019-03-08 10:24:40 +01:00
Sander van Harmelen 0232d84a0d backend/remote: also retry on server errors
Enably retrying on server errors in the updated `go-tfe` client and add a retry log hook for writing retry messages to the CLI.
2019-03-06 13:36:06 +01:00
Kristin Laemmert b9d8e96e0c
command/plan: plan output should indicate if a resource is being (#20580)
replaced because the instance was tainted.
2019-03-05 16:18:55 -08:00
Martin Atkins 709487b4f1 backend: Cap number of "undeclared variable" warnings at four
For users who in previous versions have relied on our lack of checking for
whether variables are declared, they may previously have seen an
overwhelming number of warnings when running Terraform v0.12.

Here we cap that number at three specific warnings and then one general
warning, so we can still give a specific source location for the first
couple (for users who have genuinely made a typo) but summarize away a
large number for those who are seeing this because they've not yet
migrated to using environment variables.
2019-03-05 15:42:08 -08:00
Mars Hall 34fa67c6b0 Correct pg backend error info 2019-03-05 11:37:08 -08:00
Mars Hall ccf549a62d Stricter pg backend locking during workspace creation 2019-03-05 11:19:56 -08:00
Sergey Kondrashov 43e7a7b552 fix misspelling 2019-03-05 16:12:52 +03:00
Mars Hall 31c9776d55 Switch pg backend to session-level advisory locking, to avoid rollback of partial state updates 2019-03-04 17:01:08 -08:00
Brian Flad e34a97abca
backend/s3: Add debug logging and user agent
Porting over previous behavior from terraform-provider-aws.
2019-02-27 12:59:08 -05:00
Sander van Harmelen 63e2dcef8a
Merge pull request #20481 from hashicorp/svh/b-exit-code
backend/remote: exit with 1 when a run is canceled
2019-02-27 07:52:20 +01:00
James Bardin eada955721
Merge pull request #19070 from mars/postgres-backend
Postgres backend
2019-02-26 20:20:01 -05:00
Mars Hall 6d9e2048f4 Skip more pg backend acceptance tests 2019-02-26 16:50:36 -08:00
Mars Hall 920e7a7acc Mutex pg backend‘s database transaction 2019-02-26 13:45:16 -08:00
Mars Hall c6bf3442fc Lock() in pg backend now returns LockError for retryable errors 2019-02-26 13:20:40 -08:00
Mars Hall a075a68c58 Correct name of pg backend‘s default schema 2019-02-26 12:59:31 -08:00
Sander van Harmelen 01f17fa0ca backend/remote: exit with 1 when a run is canceled 2019-02-26 21:00:07 +01:00
James Bardin dd16455a79 fix s3 default region to match provider 2019-02-25 20:27:21 -05:00
Mars Hall 2621f95bd2 📚 doc corrections 2019-02-25 16:30:30 -08:00
Mars Hall b9a91b7c1e Switch pg backend to use native Postgres locks 2019-02-25 16:05:53 -08:00
James Bardin c814f2da37 Change backend.ValidateConfig to PrepareConfig
This mirrors the change made for providers, so that default values can
be inserted into the config by the backend implementation. This is only
the interface and method name changes, it does not yet add any default
values.
2019-02-25 18:37:20 -05:00
Sander van Harmelen 435b82d6d0
Merge pull request #20460 from hashicorp/svh/f-permissions
backend/remote: use the can-queue-apply permission
2019-02-25 19:32:20 +01:00
James Bardin aebfecf871
Merge pull request #20211 from yanndegat/swift_remote_backend
Swift remote backend support for workspaces & locking
2019-02-25 12:58:59 -05:00
Sander van Harmelen 58961026a2 backend/remote: use the can-queue-apply permission
Previously we checked can-update in order to determine if a user had the
required permissions to apply a run, but that wasn't sufficient. So we
added a new permission, can-queue-apply, that we now use instead.
2019-02-25 14:45:40 +01:00
Mars Hall 8cb2943b6b pg backend tests run when TF_ACC is set (no more TF_PG_TEST) 2019-02-22 14:42:24 -08:00
Mars Hall c8b8227967 Compatibility fixes for 0.12.0 2019-02-22 13:54:41 -08:00
Mars Hall d4d798c25e Fix to use existing pg indexes 2019-02-22 13:51:57 -08:00
Mars Hall 9feda7aab3 Correct code comment for pg 2019-02-22 13:51:57 -08:00
Mars Hall 6d7e8268e7 Include pg backend in main init 2019-02-22 13:51:57 -08:00
Mars Hall 9043005cfb Postgres database backend 2019-02-22 13:48:49 -08:00
James Bardin 31ca293777 fix slash handling around workspace_key_prefix
The handling of slashes was broken around listing workspaces in
workspace_key_prefix. While it worked in most places by splitting an
extra time around the spurious slashes, it failed in the case that the
prefix ended with a slash of its own.

A test was temporarily added to verify that the backend works with the
unusual keys, but rather than risking silent breakage around prefixes
with trailing slashes, we also add validation to prevent users from
entering keys with trailing slashes at all.
2019-02-22 10:29:10 -05:00