Commit Graph

290 Commits

Author SHA1 Message Date
James Bardin 9c431aee1b only list environments when the keyName matches
Prevent extra keys in the s3 envPrefix path from showing up as
listed environments.

Better handle keys containing slashes

Add tests for unexpected keys in s3.
2017-04-12 13:57:22 -04:00
James Bardin 5621d97925 cleanup consul lock entries
This matches the consul cli behavior, where locks are cleaned up after
use.

Return an error from re-locking the state. This isn't required by the
Locker interface, but it's an added sanity check for state operations.
What was incorrect here was returning an empty ID and error, which would
indicate that Lock/Unlock isn't supported.
2017-04-06 14:19:55 -04:00
James Bardin 6e136c848a use the aws provider client initialization
Use the aws provider code to create the clients for the s3 backend, so
that all the behavior matches that of the provider.

Remove the fake creds from the test, as the aws provider will attempt to
validate them.
2017-04-05 12:39:50 -04:00
James Bardin 54aa466b74 initialize the s3 lock path in one place 2017-04-04 14:44:58 -04:00
James Bardin d059939f88 Merge pull request #13262 from hashicorp/jbardin/lock-timeouts
lock timeouts
2017-04-04 14:30:20 -04:00
James Bardin 305ef43aa6 provide contexts to clistate.Lock calls
Add fields required to create an appropriate context for all calls to
clistate.Lock.

Add missing checks for Meta.stateLock, where we would attempt to lock,
even if locking should be skipped.
2017-04-01 17:09:20 -04:00
James Bardin 3f0dcd1308 Have the clistate Lock use LockWithContext
- Have the ui Lock helper use state.LockWithContext.
- Rename the message package to clistate, since that's how it's imported
  everywhere.
- Use a more idiomatic placement of the Context in the LockWithContext
  args.
2017-04-01 17:09:20 -04:00
James Bardin 75458a182d remove extra state.Locker assertions
All states are lockers, so get rid of extra asertions.
2017-04-01 17:01:45 -04:00
Paul Stack 4501be7e5c backend/remote-state: Add support for assume role extensions to s3 backend (#13236)
Fixes: #13234

This now matches the AWS provider for the Assume Role support
2017-03-31 19:59:29 +03:00
Martin Atkins 21cd5595e2 Update stringer-generated files to new boilerplate
golang/tools commit 23ca8a263 changed the format of the leading comment
to comply with some new standards discussed here:
https://golang.org/issue/13560

This is the result of running generate with the latest version of
stringer. Everyone working on Terraform will need to update stringer
after this is merged, to avoid reverting this:
    go get -u golang.org/x/tools/cmd/stringer
2017-03-29 08:07:06 -07:00
James Bardin 7b8e1aff3d fix local backend test
The local backend can't define a StateOut path if we want to test
writing multiple named state files. Use a default local backend.
2017-03-23 11:15:46 -04:00
James Bardin 49b9a6ad92 test for proper state persistence
The backend state tests weren't properly checking for persistence.
Update the test to persist states and fetch them again from the backend,
checking that lineage is preserved.
2017-03-23 10:06:07 -04:00
James Bardin 575e7f1811 Properly create a new named state in s3
If the state doesn't exist, we need to initialize one so that it can be
listed be States.
2017-03-23 10:03:22 -04:00
James Bardin fa4dc01cf4 add named state support to the s3 backend
This adds named state (environment) support to the S3 backend.

A state NAME will prepend the configured s3 key with `env:/NAME/`.
The default state will remain rooted in the bucket for backwards
compatibility.

Locks in DynamoDB use the S3 key as the as the primary key value, so
locking will work as expected for multiple states.
2017-03-22 16:59:13 -04:00
James Bardin 4980fa20e7 move s3 config from client to backend
The RemoteClient needs to be configured for the named state, so move the
general config to the backend.

Rename some fields for consistency.
2017-03-22 15:52:55 -04:00
James Bardin 9f5cf2b105 convert S3 remote state to a backend
Move the S3 State from a legacy remote state to an official backend.

This increases test coverage, uses a set schema for configuration, and
will allow new backend features to be implemented for the S3 state, e.g.
"environments".
2017-03-22 10:59:37 -04:00
Mitchell Hashimoto d01886a644
command: remove legacy remote state on migration
Fixes #12871

We were forgetting to remove the legacy remote state from the actual
state value when migrating. This only causes an issue when saving a plan
since the plan contains the state itself and causes an error where both
a backend + legacy state exist.

If saved plans aren't used this causes no noticable issue.

Due to buggy upgrades already existing in the wild, I also added code to
clear the remote section if it exists in a standard unchanged backend
2017-03-20 10:14:59 -07:00
Mitchell Hashimoto 23dd2a0ee5
backend/atlas: accept ATLAS_ADDRESS env var for address
This is required for on-premise TFE.
2017-03-16 23:04:07 -07:00
Mitchell Hashimoto 2be1f55cbb
backend/local: allow refresh on empty/non-existent state
This allows a refresh on a non-existent or empty state file. We changed
this in 0.9.0 to error which seemed reasonable but it turns out this
complicates automation that runs refresh since it now needed to
determine if the state file was empty before running.

Its easier to just revert this into a warning with exit code zero.

The reason this changed is because in 0.8.x and earlier, the output
would be simply empty with exit code zero which seemed odd.
2017-03-16 12:11:31 -07:00
Mitchell Hashimoto 2969b29d9b
backend/local: call new test API correctly 2017-03-15 08:46:58 -07:00
Mitchell Hashimoto 1ca0352e5f Merge pull request #12558 from hashicorp/f-backend-testing
backend/local: run backend.TestBackend
2017-03-15 08:45:20 -07:00
Mitchell Hashimoto 0cd15176ff
backend/consul: rebase and modify gzip test 2017-03-14 17:59:54 -07:00
Mitchell Hashimoto 1daff7a826
backend/consul: support "lock" option to disable locking
This adds a "lock" config (default true) to allow users to optionally
disable state locking with Consul. This is necessary if the token given
doesn't have session permission and is necessary for backwards
compatibility.
2017-03-14 17:59:10 -07:00
James Bardin 08d2b44ada add gzip to consul backend tests 2017-03-14 15:41:00 -04:00
James Bardin 1527eae6b7 Merge pull request #8748 from christoe/christoe/8491-gzip-consul-remote-state
state/remote: Add gzip support to consul remote state (#8491)
2017-03-14 15:27:46 -04:00
James Bardin 86f711f6fc Make consul backend tests opt-in
This way we don't require contributers to have consul installed to run
make test.
2017-03-14 14:44:03 -04:00
James Bardin fa7743b627 quiet the consul server during backend tests
Don't display logs unless using `-v`
2017-03-14 10:49:38 -04:00
James Bardin 90055c6ae2 convert the consul backend to use consul/testutil
Start up our own consul server for unit tests.
2017-03-13 18:25:58 -04:00
Christoffer Kylvåg abfa35db7c backend/remote-state/consul: Make gzip compression configurable (#8491) 2017-03-13 08:17:33 +01:00
Mitchell Hashimoto f63e8b3398
backend/local: run backend.TestBackend
This verifies that local behaves in an expected way by the backend
package.
2017-03-09 16:17:21 +05:30
Christoffer Kylvåg e098c7c24a backend/remote-state/consul: Add gzip support to consul backend client (#8491) 2017-03-09 08:00:19 +01:00
Mitchell Hashimoto 35afb7bc86
backend/consul: some comments 2017-03-01 23:01:28 -08:00
Mitchell Hashimoto b842fd0c27
backend/consul: support named states 2017-03-01 22:58:51 -08:00
Mitchell Hashimoto 3db55cf747
backend/consul: build your own backend 2017-03-01 22:19:36 -08:00
Mitchell Hashimoto 08b47cf73b
backend/atlas: adhere to new multi-state interface 2017-03-01 13:31:52 -08:00
Mitchell Hashimoto e6a87cf8de
backend/atlas: adhere to backend.CLI 2017-03-01 13:29:16 -08:00
Mitchell Hashimoto 942572b574
backend/init: add atlas, remove legacy atlas remote state 2017-03-01 13:29:16 -08:00
Mitchell Hashimoto d00d6bc7fe
backend/atlas: initial state working 2017-03-01 13:29:16 -08:00
Mitchell Hashimoto bdde7d845a Merge pull request #12347 from hashicorp/b-env-migrate
command: migrating envs when changing backends
2017-03-01 12:58:40 -08:00
Mitchell Hashimoto 549d525487 Merge pull request #12320 from hashicorp/b-legacy-state
command: fix loading legacy remote state + migration guide
2017-03-01 12:39:17 -08:00
Mitchell Hashimoto e75b666591
command: test multi-state to single state 2017-03-01 11:34:45 -08:00
Mitchell Hashimoto 1d8b76c89d
command: initial work on migrating envs, basic cases first 2017-03-01 10:59:17 -08:00
Mitchell Hashimoto 6ea0dc0964
backend/consul: make address have a default to prevent input 2017-02-28 19:29:05 -08:00
James Bardin 4dac986a91 Local.StatePaths doesn't need to reutrn an error
add a test to ensure we have consistent output
2017-02-28 19:18:16 -05:00
James Bardin b53704ed87 Thread the environment through all commands
Add Env and SetEnv methods to command.Meta to retrieve the current
environment name inside any command.

Make sure all calls to Backend.State contain an environment name, and
make the package compile against the update backend package.
2017-02-28 16:35:46 -05:00
James Bardin f866bb545c update remote-state/consul 2017-02-28 16:35:46 -05:00
James Bardin 597eb6c918 update remote-state/inmem client 2017-02-28 16:35:45 -05:00
James Bardin 8fdf3a42b8 update remote-state.Backend 2017-02-28 16:35:45 -05:00
James Bardin 5762878eba Make backcend/legacy match new Backend iface
move the unsupported error value to backend.ErrNamedStatesNotSupported
to be used by any backend implementation.
2017-02-28 16:35:45 -05:00
James Bardin 65527f35a4 update local.Local to match the latest Backend
Update the methods, remove the handling of "current", and make tests
pass.
2017-02-28 16:07:31 -05:00
James Bardin 96194fbc0d Update Backend interface to latest iteration
What will hopfully be the final version of the Backend interface. This
combines the MultiState interface into Backend since it will be required
to implement, and simplifies the interface because the Backend is no
longer responsible for tracking the current state.
2017-02-28 16:07:07 -05:00
James Bardin 7f453f3341 remove some leftover methods in the legacy backend
These were left from the initial implementation, but are not used.
2017-02-28 16:07:07 -05:00
James Bardin fbc11c7961 fix incorrect current state in local backend
Forgot to remove the currentState field, which was not always set. The
current state should always just be read from the environment file.

Always return the default state name when we can't determine the state.
2017-02-28 16:07:07 -05:00
James Bardin e6eb71dde5 Add tests to check Backend delegation
Ensure that when MultiState methods are properly delegated when there is
a defined Local.Backend.
2017-02-28 16:07:06 -05:00
James Bardin 0933541a8c Split out the backend environment interface
Split the interface to change environments out from the minimal Backend
interface, to make it optional for backend implementations. If
backend.MultiState isn't implemented, return a "not implemented" from
environment related methods.

Have the Local backend delegate the MultiState methods to the proper
backend.
2017-02-28 16:06:14 -05:00
James Bardin dbc45b907c Make the Local backend handle its own named states
Add the functionality required for terraform environments
2017-02-28 16:03:36 -05:00
James Bardin 761c63d14a Update Backend to incorporate environments
Add the missing methods/arguments to handle Terraform environments in
Backends. Extra functionality simply returns defaults for now.
2017-02-28 16:03:36 -05:00
Mitchell Hashimoto 3cedfa00f4
command: use backend.CLIIinit
I made this interface way back with the original backend work and I
guess I forgot to hook it up! This is becoming an issue as I'm working
on our 2nd enhanced backend that requires this information and I
realized it was hardcoded before.

This propertly uses the CLIInit interface allowing any backend to gain
access to this data.
2017-02-28 10:58:29 -08:00
Mitchell Hashimoto 8f11068ab2 Merge pull request #12173 from hashicorp/b-remote-state-ds
providers/terraform: remote state data source supports backends
2017-02-22 18:43:06 -08:00
Mitchell Hashimoto d2d87bccf0 Merge pull request #12155 from hashicorp/b-state-backend
command: refresh state in old commands for backend
2017-02-22 18:40:55 -08:00
Mitchell Hashimoto 9574f16f92
backend/local: refresh with no config should not crash on input
Fixes #12174

You're allowed to refresh with a nil module (no configs) as long as you
have state. However, if `-input=true` (default) then this would crash
since the input attempts to read the configs.

The API contract with `terraform.Context` says that the module tree must
be non-nil and loaded. To do this for other commands we create an empty
module tree. We do that here now.
2017-02-22 13:10:08 -08:00
Mitchell Hashimoto ebb22d3ecd
backend/local: don't RefreshState on State API 2017-02-22 13:01:16 -08:00
Mitchell Hashimoto 52720ce880
providers/terraform: data source uses backends for state loading 2017-02-22 11:37:56 -08:00
Mitchell Hashimoto f79e04500f
backend/init: a package for storing the factories for backends 2017-02-22 11:17:06 -08:00
James Bardin 2392455a67 Add inmem remote backend
Add an Inmem remote state backend.

The only config option right now is a lock_id, which will instantiate
the backend in a locked state for testing.
2017-02-20 14:49:46 -05:00
Mitchell Hashimoto d443bf1b56
backend/local: allow nil modules (no config) if executing a plan 2017-02-16 10:56:39 -08:00
Mitchell Hashimoto 1480d0c5b8
backend/local: check for empty config on apply
This prevents Terraform from crashing on apply/destroy with a directory
with no Terraform configuration files. We allow a destroy with no files
but not an apply.
2017-02-15 16:00:59 -08:00
Mitchell Hashimoto efe754183b
state/remote: export ClientLocker, test for implementation
This adds unit tests (that will fail at compile time) if various structs
don't implement the right interfaces for locking
2017-02-15 14:20:59 -08:00
James Bardin ec00564be6 Clean up LockInfo and LockError and use them
Gove LockInfo a Marshal method for easy serialization, and a String
method for more readable output.

Have the state.Locker implementations use LockError when possible to
return LockInfo and an error.
2017-02-15 14:44:43 -05:00
James Bardin 67bbebce08 Have consul state reutrn the lock ID
The lock ID isn't used because the lock is tied to the client, but
return the lock ID to match the behavior of other locks.
2017-02-15 14:44:43 -05:00
James Bardin f2e496a14c Have backend operations properly unlock state
Make sure unlock is called with the correct LockID during operations
2017-02-15 14:41:55 -05:00
James Bardin f5ed8cd288 Use NewLockInfo to get a pre-populated value
Using NewLockInfo ensure we start with all required fields filled.
2017-02-15 14:41:55 -05:00
James Bardin cd233fef6a make consul client pass state.Locker tests 2017-02-15 14:41:55 -05:00
James Bardin 67dc16c9ca Make backend/local test pass 2017-02-15 14:41:55 -05:00
Mitchell Hashimoto 235b7eb38e Merge pull request #11944 from hashicorp/f-state-slow
show message if state lock acquisition/release is slow
2017-02-14 14:00:23 -08:00
Mitchell Hashimoto 5e4f6cf2b1
backend/local: fix could not to did not to prevent error look 2017-02-14 12:09:45 -08:00
Mitchell Hashimoto 65982bd412
backend/local: use new command/state package for better UX 2017-02-14 11:17:28 -08:00
James Bardin 80fab23e04 Don't test consul using demo.consul.io
We shoudn't require an external service for unit test.

TODO: create some proper acceptance tests for consul
2017-02-08 11:34:31 -05:00
James Bardin 14d965722e Use single state.LockInfo struct
Remove redundant structures
2017-02-08 11:34:31 -05:00
James Bardin 9b76f6e138 Move TestRemoteLocks to state/remote
This was legacy remote state client and backends can use this test
function without an import cycle.
2017-02-08 11:25:52 -05:00
James Bardin 54cac349a3 Add state locking to consul backend
Use consul locks to implement state locking. The lock path is state path
+ "/.lock" which matches the consul cli default for locks. Lockinfo is
stored at path + "/.lockinfo".
2017-02-08 11:25:52 -05:00
Mitchell Hashimoto bdca9bffe4
backend/local: output warnings
Fixes #11628

This is a simple fix to output warnings. I originally forgot to do this
since the local backend didn't have a CLI UI at the time. It does now so
this is an easy fix.
2017-02-07 13:22:28 -08:00
James Bardin 0d7752b0f5 Update runningOp.Err with State.Unlock error
Have the defer'ed State.Unlock call append any error to the
RunningOperation.Err field. Local error would be rare and
self-correcting, but when the backend.Local is using a remote state the
error may require user intervention.
2017-02-06 09:54:15 -05:00
James Bardin 9cdba1f199 enable local state locking for apply
Have the LocalBackend lock the state during operations, and enble this
for the apply comand.
2017-02-02 18:08:28 -05:00
Mitchell Hashimoto a424203ea3
backend/local: validate module exists for plan
Fixes #11504

The local backend should error if `terraform plan` is called in a
directory with no Terraform config files (same behavior as 0.8.x).
**New behavior:** We now allow `terraform plan -destroy` with no
configuration files since that seems reasonable.
2017-01-29 20:02:12 -08:00
Mitchell Hashimoto 31f7cca77f
backend/local: fix crash (in tests) due to not guarding nil CLI 2017-01-26 14:33:50 -08:00
Mitchell Hashimoto 1f5d425428
backend/remote-state
This allows migration of the remote state implementations to a richer
experience including input asking.
2017-01-26 14:33:49 -08:00
Mitchell Hashimoto 13c34b16e8
backend/legacy
This allows using legacy remote state backends with the new backend
interface.
2017-01-26 14:33:49 -08:00
Mitchell Hashimoto 397e1b3132
backend/local
The local backend implementation is an implementation of
backend.Enhanced that recreates all the behavior of the CLI but through
the backend interface.
2017-01-26 14:33:49 -08:00
Mitchell Hashimoto 8a070ddef0
backend: introduce the backend set of interfaces
Backends are a mechanism that allow abstracting the behavior of
Terraform CLI from the actual core. This allows us to slip in special
behavior such as state loading, remote operations, etc.
2017-01-26 14:33:49 -08:00