Commit Graph

288 Commits

Author SHA1 Message Date
stack72 a6f76aa0af backend/manta: Manta Backend was not dealing with a ResourceNotFound
Fixes: #17314

We now deal correctly with the creation of the state file - we were
not dealing well with a ResourceNotFound error

Now that this has been changed around, we try and create the statefile
and if there is an error, we look for an existing statefile - previously
this was not the order of operations
2018-03-12 17:27:57 +02:00
Martin Atkins 6aefa5835c Merge #17218: Add -auto-approve to "terraform destroy" for consistency 2018-03-08 17:42:15 -08:00
Paul Tyng c868092d2d
Standardize http.Client creation with User-Agent 2018-02-28 12:09:50 -05:00
James Bardin e9a76808df create clistate.Locker interface
Simplify the use of clistate.Lock by creating a clistate.Locker
instance, which stores the context of locking a state, to allow unlock
to be called without knowledge of how the state was locked.

This alows the backend code to bring the needed UI methods to the point
where the state is locked, and still unlock the state from an outer
scope.

Provide a NoopLocker as well, so that callers can always call Unlock
without verifying the status of the lock.

Add the StateLocker field to the backend.Operation, so that the state
lock can be carried between the different function scopes of the backend
code. This will allow the backend context to lock the state before it's
read, while allowing the different operations to unlock the state when
they complete.
2018-02-23 16:48:15 -05:00
James Bardin 8242c773b8 missed the local state backend 2018-02-20 22:09:54 -05:00
James Bardin 57cbcbe99d update remote state tests to use new test helpers 2018-02-20 21:05:58 -05:00
James Bardin 112c37458f report the proper lock id from a state lock error
Fix the now failing state unlock test by reporting the correct ID.

The ID used by GCS is the generation number of the info object, which
isn't known until the info is already written out. While we can't get
the correct ID from the info data for the error rmessage, we can update
it with the generation number after it's read.
2018-02-20 20:54:01 -05:00
James Bardin 1fee5ae172 update gcs to use the new backend tests 2018-02-20 20:36:08 -05:00
James Bardin 85d6b1d9cc add test for lock error and force-unlock
This adds a general test to verify that a remote state backend returns
the expected error type when it cannot lock a state. It then extracts
the ID reported in the error, and attempts to unlock the state using
that ID, which simulated the force-unlock scenario. This is a separate
test, since not all backends have persistent locks that can be unlocked
later.

We also split out the backend test to be called individually as needed.
2018-02-20 20:32:07 -05:00
James Bardin ef8ed1e275 coalesce the backened interrupt code
Moves the nested select statements for backend operations into a single
function. The only difference in this part was that apply called
PersistState, which should be harmless regardless of the type of
operation being run.
2018-02-12 11:56:54 -05:00
James Bardin 7cba68326a always wait for a RunningOperation to return
If the user wishes to interrupt the running operation, only the first
interrupt was communicated to the operation by canceling the provided
context. A second interrupt would start the shutdown process, but not
communicate this to the running operation. This order of event could
cause partial writes of state.

What would happen is that once the command returns, the plugin system
would stop the provider processes. Once the provider processes dies, all
pending Eval operations would return return with an error, and quickly
cause the operation to complete. Since the backend code didn't know that
the process was shutting down imminently, it would continue by
attempting to write out the last known state. Under the right
conditions, the process would exit part way through the writing of the
state file.

Add Stop and Cancel CancelFuncs to the RunningOperation, to allow it to
easily differentiate between the two signals. The backend will then be
able to detect a shutdown and abort more gracefully.

In order to ensure that the backend is not in the process of writing the
state out, the command will always attempt to wait for the process to
complete after cancellation.
2018-02-12 11:56:03 -05:00
Laura Martin 6e1e614a56 Change -force to -auto-approve when destroying
Since an early version of Terraform, the `destroy` command has always
had the `-force` flag to allow an auto approval of the interactive
prompt. 0.11 introduced `-auto-approve` as default to `false` when using
the `apply` command.

The `-auto-approve` flag was introduced to reduce ambiguity of it's
function, but the `-force` flag was never updated for a destroy.

People often use wrappers when automating commands in Terraform, and the
inconsistency between `apply` and `destroy` means that additional logic
must be added to the wrappers to do similar functions. Both commands are
more or less able to run with similar syntax, and also heavily share
their code.

This commit updates the command in `destroy` to use the `-auto-approve` flag
making working with the Terraform CLI a more consistent experience.

We leave in `-force` in `destroy` for the time-being and flag it as
deprecated to ensure a safe switchover period.
2018-02-01 00:14:42 +00:00
James Bardin a5ed7d0ae4
Merge pull request #17086 from hashicorp/jbardin/GH-17083
only split the prefix string once from the s3 key
2018-01-17 18:51:43 -05:00
stack72 e9476c6765 backend/manta: Update manta dependencies
Internally, triton-go has changed how it handles errors. We can now get rid of
checking strings for errors, and we have introduced an errors library that
wraps some of the major errors we encounter and test for
2018-01-17 11:34:02 -08:00
Masayuki Morita f440dba137 Standardize on log level "WARN" rather than "WARNING" 2018-01-16 18:05:26 -08:00
James Bardin 43d000625d only split the prefix string once from the s3 key
Ensure that the prefix is only split off a single time when the
workspace_key_prefix is a substring of the workspace or key name.
2018-01-11 10:08:02 -05:00
Stefan Schmidt c200c170ad Handle refresh errors. 2018-01-10 16:40:20 +01:00
Paddy e4cdbd6c9f
Merge pull request #16936 from negz/gcskeys
Support 'customer supplied encryption keys' in the GCS backend
2018-01-09 01:17:35 -08:00
James Bardin 7c703f2ab2
Merge pull request #16932 from rv-jmaggio/master
Fixing issues with workspace_key_prefix for s3 backend
2018-01-05 13:24:40 -05:00
James Bardin dd5882e5b2
Merge pull request #16992 from hashicorp/jbardin/go1.10-prep
go1.10 prep
2018-01-03 15:41:41 -05:00
goraxe fb6b349e58 backend/s3: Allow forcing use of the S3 path-style API
This is useful when the underlying API is an S3-workalike service, rather than official S3.
2018-01-03 12:15:47 -08:00
Paul Stack 191cf283d5 backend/manta: Support Triton RBAC
Triton Manta allows an account other than the main triton account to be used via RBAC.

Here we expose the SDC_USER / TRITON_USER options to the backend so that a user can be specified.
2018-01-03 12:12:46 -08:00
James Bardin 4b49a323c3 go fmt
slight change to go fmt coming in 0.10
2017-12-26 13:26:38 -05:00
rv-jmaggio b313ce80c4 Changing prefix for empty workspace prefix 2017-12-19 13:14:31 -05:00
Henry Bell 56357c0ab7 Fix typo ('depracted' -> 'deprecated') 2017-12-19 16:43:06 +00:00
rv-jmaggio bef64cfe91 Fixing implementation for empty string and making acceptance test work 2017-12-19 09:31:53 -05:00
rv-jmaggio b02a1c8a46 clarifying tests and using SplitN in implementation 2017-12-18 16:24:34 -05:00
Nic Cope 011841124b Support 'customer supplied encryption keys' in the GCS backend
https://cloud.google.com/storage/docs/encryption#customer-supplied

GCS state created using customer supplied encryption keys can only be read or
modified using the same key.
2017-12-17 19:27:52 -08:00
rv-jmaggio 7f8d686074 refactor and add a test 2017-12-15 21:04:15 -05:00
rv-jmaggio 0889c118a8 Fixing issues with workspace_key_prefix 2017-12-15 17:50:36 -05:00
Rob Campbell 5daeee5f6d Update various files for new version of "stringer"
The latest version of stringer now uses strconv instead of fmt.
2017-12-11 13:26:29 -08:00
James Bardin 52eced589f accept a path or contents for credentials
Match the operation of the google provider, by accepting either a file
path or contents for both `credentials` and `GOOGLE_CREDENTIALS`
2017-12-06 18:33:59 -05:00
James Bardin 2932203492 verify that a state can be read even when locked
This should only happen when a state is loaded via the backend, as well
as a remote state.
2017-12-06 18:33:59 -05:00
James Bardin 9dea2f78d4 create unique buckets for each test, and clean up
This creates a unique bucket name for each test, so that the tests in
parallel don't collide, and buckets left over from interrupted tests
don't cause future failures.

Also make sure that buckets are removed, regardless of content.
2017-12-06 18:33:59 -05:00
James Bardin aec45e6967 backends should never create unmanaged resources
The backend was creating bucket named in the configuration if it didn't
exist. We don't allow other backends to do this, because these are not
managed resources that terraform can control.
2017-12-06 18:32:41 -05:00
Kaveh Mousavi Zamani 7507e3cd21 backend/gcs: fix locking issue when used with terraform_remote_state
Previously there was a problem with double-locking when using the GCS backend with the terraform_remote_state data source.

Here we adjust the locking methodology to avoid that problem.
2017-12-06 09:36:16 -08:00
James Bardin 12b7dac124
Merge pull request #16833 from hashicorp/jbardin/plan-shutdown
Fully enable shutdown for plan and refresh in the local backend
2017-12-05 16:48:34 -05:00
Sander van Harmelen aaedf255c3 backend/s3: allow skipping the region check (#16757)
Without the possibility to skip this check, it’s not possible to use a custom region with a third-party service that mimicks the S3 API.
2017-12-05 11:19:36 -08:00
James Bardin 85295e5c23 watch for cancellation in plan and refresh
Cancellation in the local backend was only implemented for apply.
2017-12-05 10:17:20 -05:00
Martin Atkins ba0514106a return tfdiags.Diagnostics from validation methods
Validation is the best time to return detailed diagnostics
to the user since we're much more likely to have source
location information, etc than we are in later operations.

This change doesn't actually add any detail to the messages
yet, but it changes the interface so that we can gradually
introduce more detailed diagnostics over time.

While here there are some minor adjustments to some of the
messages to improve their consistency with terminology we
use elsewhere.
2017-11-28 11:15:29 -08:00
James Bardin 00b7715710
Merge pull request #16586 from hashicorp/jbardin/providers
Store resolved providers in state
2017-11-08 14:27:48 -05:00
Nic Cope 27ba7de77d backend/gcs: automatically add trailing slash to prefix
The backend doesn't function correctly if a trailing slash is not present, due to how workspaces are enumerated.
2017-11-08 11:24:36 -08:00
James Bardin 09180a10ff cannot destroy without a config 2017-11-07 21:23:37 -05:00
James Bardin d62e9217ae update test state strings for backend/local 2017-11-07 21:23:10 -05:00
Malik Bougacha ed0d57d1b7 Get the enviroment from the environment variable (#16326)
* Get the enviroment from the enviroment variable

* typo
2017-11-06 16:59:48 +00:00
stack72 1fd0f803e4 Migrate Manta Remote state to be a backend
This PR changes manta from being a legacy remote state client to a new backend type. This also includes creating a simple lock within manta

This PR also unifies the way the triton client is configured (the schema) and also uses the same env vars to set the backend up

It is important to note that if the remote state path does not exist, then the backend will create that path. This means the user doesn't need to fall into a chicken and egg situation of creating the directory in advance before interacting with it
2017-10-30 18:36:50 +02:00
James Bardin cf54ca3b0f update tests for new consul packages
Reuse the running consul server for all tests.

Update the lostLockConnection package, since the api client should no
longer lose a lock immediately on network errors.
2017-10-29 12:40:44 -04:00
James Bardin 3a03d3683e update consul api packages
This is from a commit just after the v1.0.0 release, because it removes
the Porter service dependency for tests. The client api package was not
changed.
2017-10-28 19:44:10 -04:00
Martin Atkins 671aace8ec backend/local: disable local backup of remote state
Previously we forced all remote state backends to be wrapped in a
BackupState wrapper that generates a local "terraform.tfstate.backup"
file before updating the remote state.

This backup mechanism was motivated by allowing users to recover a
previous state if user error caused an undesirable change such as loss
of the record of one or more resources. However, it also has the downside
of flushing a possibly-sensitive state to local disk in a location where
users may not realize its purpose and accidentally check it into version
control. Those using remote state would generally prefer that state never
be flushed to local disk at all.

The use-case of recovering older states can be dealt with for remote
backends by selecting a backend that has preservation of older versions
as a first-class feature, such as S3 versioning or Terraform Enterprise's
first-class historical state versioning mechanism.

There remains still one case where state can be flushed to local disk: if
a write to the remote backend fails during "terraform apply" then we will
still create the "errored.tfstate" file to allow the user to recover. This
seems like a reasonable compromise because this is done only in an
_exceptional_ case, and the console output makes it very clear that this
file has been created.

Fixes #15339.
2017-10-27 17:06:33 -07:00
James Bardin 55089e472d
Merge pull request #16484 from hashicorp/f-gcloud-backend
Convert gcloud backend
2017-10-27 17:23:22 -04:00