Commit Graph

162 Commits

Author SHA1 Message Date
James Bardin eeddc3f8ea add some nil checks for unexpected lock failures 2017-04-05 09:06:47 -04:00
James Bardin af2e289212 remove Sleep from TestLockWithContext 2017-04-03 14:46:22 -04:00
James Bardin d1460d8c82 test LockWithContext 2017-04-03 11:50:19 -04:00
James Bardin 93b1dd6323 give LockWithContext a little backoff
Backoff the Lock calls exponentially, to a reasonable limit.
2017-04-01 18:01:49 -04:00
James Bardin 3f0dcd1308 Have the clistate Lock use LockWithContext
- Have the ui Lock helper use state.LockWithContext.
- Rename the message package to clistate, since that's how it's imported
  everywhere.
- Use a more idiomatic placement of the Context in the LockWithContext
  args.
2017-04-01 17:09:20 -04:00
James Bardin 826771a830 add state.LockWithContext
LockWithContext will retry a lock until the context expires or is
cancelled. This will let us implement a `-lock-timeout` flag, and make
use of existing contexts when applicable.
2017-04-01 17:01:55 -04:00
James Bardin 75458a182d remove extra state.Locker assertions
All states are lockers, so get rid of extra asertions.
2017-04-01 17:01:45 -04:00
James Bardin bf6384a163 All states are lockers
Since moving to the new backends, all states (except InmemState) are
Lockers. Add the methods to the State interface to remove a heap of
assertion checks.
2017-04-01 17:01:12 -04:00
James Bardin 9f5cf2b105 convert S3 remote state to a backend
Move the S3 State from a legacy remote state to an official backend.

This increases test coverage, uses a set schema for configuration, and
will allow new backend features to be implemented for the S3 state, e.g.
"environments".
2017-03-22 10:59:37 -04:00
Mitchell Hashimoto 1daff7a826
backend/consul: support "lock" option to disable locking
This adds a "lock" config (default true) to allow users to optionally
disable state locking with Consul. This is necessary if the token given
doesn't have session permission and is necessary for backwards
compatibility.
2017-03-14 17:59:10 -07:00
Ash Berlin 70de22253a Fix error message in Azure state backend (#12424)
The error led me to try adding `resource_group` but the code wanted `resource_group_name`. This fixes the error message to match the code.
2017-03-04 20:27:37 +02:00
Peter McAtominey cbed463b44 state/azure: add environment option for non-public cloud usage (#12364) 2017-03-02 04:58:24 +00:00
Mitchell Hashimoto 942572b574
backend/init: add atlas, remove legacy atlas remote state 2017-03-01 13:29:16 -08:00
James Bardin 7f40f90c4c remove the legacy Inmem remote state
moved to a new backend
2017-02-20 14:50:31 -05:00
James Bardin a4283d23f6 fix windows locking
Don't create a new windows handle for the lock.
2017-02-17 13:29:48 -05:00
James Bardin c0dda6a7b3 test failure for windows local locks 2017-02-17 13:29:47 -05:00
Mitchell Hashimoto 03f6c650ca
state/remote: ClientLocker is just a Client that is a state.Locker 2017-02-15 14:27:02 -08:00
Mitchell Hashimoto 9451acc5da
state/remote: add unit test to verify s3 is a ClientLocker as well 2017-02-15 14:25:53 -08:00
Mitchell Hashimoto efe754183b
state/remote: export ClientLocker, test for implementation
This adds unit tests (that will fail at compile time) if various structs
don't implement the right interfaces for locking
2017-02-15 14:20:59 -08:00
James Bardin ec00564be6 Clean up LockInfo and LockError and use them
Gove LockInfo a Marshal method for easy serialization, and a String
method for more readable output.

Have the state.Locker implementations use LockError when possible to
return LockInfo and an error.
2017-02-15 14:44:43 -05:00
James Bardin 888af93356 Have S3 check the lockID on Unlock
This needs to be improved to happen in a transaction, but it gets the
implementation passing the new tests.
2017-02-15 14:44:43 -05:00
James Bardin 67bbebce08 Have consul state reutrn the lock ID
The lock ID isn't used because the lock is tied to the client, but
return the lock ID to match the behavior of other locks.
2017-02-15 14:44:43 -05:00
James Bardin 08cff7cc13 have LocalState check Lock ID on Unlock
Have LocalState store and check the lock ID, and strictly enforce
unlocking with the correct ID.

This isn't required for local lock correctness, as we track the file
descriptor to unlock, but it does provide a varification that locking
and unlocking is done correctly throughout terraform.
2017-02-15 14:41:55 -05:00
James Bardin 6aa1066f7c Store and use the correct IDs in TestRemoteLocks 2017-02-15 14:41:55 -05:00
James Bardin f5ed8cd288 Use NewLockInfo to get a pre-populated value
Using NewLockInfo ensure we start with all required fields filled.
2017-02-15 14:41:55 -05:00
James Bardin 52b2343672 make state test pass with new state.Locker 2017-02-15 14:41:55 -05:00
James Bardin 4f0c465187 make command tests pass with new state.Locker 2017-02-15 14:41:55 -05:00
James Bardin 6d32b70637 Make S3 remote state pass tests
TODO: update S3Client to make full use of the state.Locker interface
2017-02-15 14:41:55 -05:00
James Bardin 0ad6f08204 Make remote state test run
Make them compile against the new interface.
The tests will be updated later to check new behavior.
2017-02-15 14:41:55 -05:00
James Bardin 200c8de4e9 Update the state.Locker interface
Remove CacheState rather than update it, since it's no longer used.
2017-02-15 14:41:55 -05:00
Mitchell Hashimoto a7cfb8062b
state: just style changes in the code itself (newlines, moving methods) 2017-02-14 08:51:20 -08:00
James Bardin 4d00c29706 Merge pull request #11836 from hashicorp/jbardin/state-locking
Add locking during backend configuration
2017-02-09 18:30:10 -05:00
James Bardin 0c1b138719 Add state locking during backend init
During backend initialization, especially during a migration, there is a
chance that an existing state could be overwritten.

Attempt to get a locks when writing the new state. It would be nice to
always have a lock when reading the states, but the recursive structure
of the Meta.Backend config functions makes that quite complex.
2017-02-09 15:47:27 -05:00
Daren Desjardins 5718294386 state/remote/s3: Fix Bug with Assume Role for Federated IAM Account (#10067)
* Enable remote s3 state support for assume role

- provide role_arn in backend config to enable assume role

Fixes #8739

* Check for errors after obtaining credentials
2017-02-09 16:54:49 +00:00
James Bardin 5c2e945b3c Fix format string and typos 2017-02-09 10:27:42 -05:00
James Bardin 14d965722e Use single state.LockInfo struct
Remove redundant structures
2017-02-08 11:34:31 -05:00
James Bardin 9b76f6e138 Move TestRemoteLocks to state/remote
This was legacy remote state client and backends can use this test
function without an import cycle.
2017-02-08 11:25:52 -05:00
James Bardin e92559f518 Cleanup state file during Unlock
Close and remove the file descriptor from LocalState if we Unlock the
state. Also remove an empty state file if we created it and it was never
written to. This is mostly to clean up after tests, but doesn't hurt to
not leave empty files around.
2017-02-03 18:58:18 -05:00
James Bardin a2b5811f50 Remove "expires" from lock info.
We are not going to handle lock expiration, at least at this time, so
remove the Expires fields to avoid any confusion.
2017-02-03 14:55:21 -05:00
James Bardin 1078781487 Change lock reason -> info
This makes it more apparent that the information passed in isn't
required nor will it conform to any standard. There may be call sites
that can't provide good contextual info, and we don't want to count on
that value.
2017-02-02 18:08:28 -05:00
James Bardin ebd88f8f8a add msdn link for LockFileEx 2017-02-01 12:38:10 -05:00
James Bardin 370a4ca70b add mutex for windows lockedFiles map 2017-02-01 12:36:08 -05:00
James Bardin 7590154974 Don't create empty backups
Not really a problem, but created unnecessary files and changes existing
behavior.
2017-01-30 17:16:57 -05:00
James Bardin 1646310e68 Allow a non-existent state file
A missing state file was allowed, and treated as an empty state.
2017-01-30 17:16:57 -05:00
James Bardin 3fdcbda3aa Switch from Path to PathOut on LocalState.written
After LocalState writes to a state file, we will refresh off the new
state file rather than the original Path argument.
2017-01-30 17:16:57 -05:00
James Bardin 8f7f1917f2 Remove state file data when writing a nil state
The old behavior in this situation was to simply delete the file. Since
we now have a lock on this file we don't want to close or delete it, so
instead truncate the file at offset 0.

Fix a number of related tests
2017-01-30 17:16:57 -05:00
James Bardin da0c325e5c Silence state package logs during tests
Output log output when testing is verbose
2017-01-30 17:16:57 -05:00
James Bardin 10f6d7f30f Add locking for s3 state
Use a DynamoDB table to coodinate state locking in S3.

We use a simple strategy here, defining a key containing the value of
the bucket/key of the state file as the lock. If the keys exists, the
locks fails.

TODO: decide if locks should automatically be expired, or require manual
intervention.
2017-01-30 17:16:57 -05:00
James Bardin 35307d5a60 Add remote state locking
In order to provide lockign for remote states, the Cache state,
remote.State need to expose Lock and Unlock methods. The actual locking
will be done by the remote.Client, which can implement the same
state.Locker methods.
2017-01-30 17:16:57 -05:00
James Bardin 6162cde6ff Add basic local state locking
Add the LockUnlock methods to LocalState and BackupState.

The implementation for LocalState will be platform specific. We will use
OS-native locking on the state files, speficially locking whichever
state file we intend to write to.
2017-01-30 17:16:57 -05:00