Commit Graph

41 Commits

Author SHA1 Message Date
James Bardin 2635b3b023 update remote state to use legacy types 2020-12-02 12:16:35 -05:00
Upo 3b9c5e5bbb
backend/gcs: remove deprecated "path" config argument (#26841) 2020-11-06 16:35:40 -05:00
Upo 4ccc63d79d
backend: Add service account impersonation to GCS Backend and update the docs (#26700) 2020-11-06 09:06:07 -05:00
Kristin Laemmert 6f4f6fbc72 helper: deprecate pathorcontents
pathorcontents was solely used by the gcs backend. I moved the function
into the backend package so it could still be used by other backends for
good measure.
2020-10-08 08:42:16 -04:00
Kristin Laemmert 6621501ae3
state: remove deprecated state package (#25490)
Most of the state package has been deprecated by the states package.
This PR replaces all the references to the old state package that
can be done simply - the low-hanging fruit.

* states: move state.Locker to statemgr

The state.Locker interface was a wrapper around a statemgr.Full, so
moving this was relatively straightforward.

* command: remove unnecessary use of state package for writing local terraform state files

* move state.LocalState into terraform package

state.LocalState is responsible for managing terraform.States, so it
made sense (to me) to move it into the terraform package.

* slight change of heart: move state.LocalState into clistate instead of
terraform
2020-08-11 11:43:01 -04:00
Paddy Carver b8752c7610 Add a backend-specific env var for the GCS backend.
Right now, the only environment variable available is the same
environment variable that will be picked up by the GCP provider. Users
would like to be able to store state in separate projects or accounts or
otherwise authenticate to the provider with a service account that
doesn't have access to the state. This seems like a reasonable enough
practice to me, and the solution seems straightforward--offer an
environment variable that doesn't mean anything to the provider to
configure the backend credentials. I've added GOOGLE_BACKEND_CREDENTIALS
to manage just the backend credentials, and documented it appropriately.
2019-12-12 03:35:39 -08:00
Luis Silva 9ae8eca55d
Merge branch 'master' into 21680/GCS_OAUTH 2019-08-27 16:56:27 +01:00
Roberto Jung Drebes b459a92575 backend/gcs: Use new endpoints for Google OAuth
This is a consequence of googleapis/google-cloud-common#260.
2019-08-13 15:24:34 -07:00
Luis Silva f6c90c1d96 Add OAuth2 token support for GCS backend 2019-07-05 10:06:44 +01:00
Mateusz Gozdek b6de825723 backend/gcs: make Unlock error message more clear 2019-05-27 20:07:14 -07:00
Stefan Schmidt 3ca1253796 backend/gcs: Mark the unused attributes project and region as removed.
These two attributes are not used since aec45e6967.
2019-03-19 17:29:04 -07:00
Martin Atkins aaf405b662 backend/remote-state: Get all the backend tests building again
The state manager refactoring in an earlier commit was reflected in the
implementations of these backends, but not in their tests. This gets us
back to a state where the backend tests will compile, and gets _most_ of
them passing again, with a few exceptions that will be addressed in a
subsequent commit.
2018-10-16 19:14:11 -07:00
Martin Atkins a3403f2766 terraform: Ugly huge change to weave in new State and Plan types
Due to how often the state and plan types are referenced throughout
Terraform, there isn't a great way to switch them out gradually. As a
consequence, this huge commit gets us from the old world to a _compilable_
new world, but still has a large number of known test failures due to
key functionality being stubbed out.

The stubs here are for anything that interacts with providers, since we
now need to do the follow-up work to similarly replace the old
terraform.ResourceProvider interface with its replacement in the new
"providers" package. That work, along with work to fix the remaining
failing tests, will follow in subsequent commits.

The aim here was to replace all references to terraform.State and its
downstream types with states.State, terraform.Plan with plans.Plan,
state.State with statemgr.State, and switch to the new implementations of
the state and plan file formats. However, due to the number of times those
types are used, this also ended up affecting numerous other parts of core
such as terraform.Hook, the backend.Backend interface, and most of the CLI
commands.

Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize
in advance to the person who inevitably just found this huge commit while
spelunking through the commit history.
2018-10-16 19:11:09 -07:00
Martin Atkins 5782357c28 backend: Update interface and implementations for new config loader
The new config loader requires some steps to happen in a different
order, particularly in regard to knowing the schema in order to
decode the configuration.

Here we lean directly on the configschema package, rather than
on helper/schema.Backend as before, because it's generally
sufficient for our needs here and this prepares us for the
helper/schema package later moving out into its own repository
to seed a "plugin SDK".
2018-10-16 18:39:12 -07:00
Sander van Harmelen 495d1ea350 Use New() instead of `once.Do(b.init)` 2018-08-03 11:29:11 +02:00
Paul Tyng c868092d2d
Standardize http.Client creation with User-Agent 2018-02-28 12:09:50 -05:00
James Bardin 112c37458f report the proper lock id from a state lock error
Fix the now failing state unlock test by reporting the correct ID.

The ID used by GCS is the generation number of the info object, which
isn't known until the info is already written out. While we can't get
the correct ID from the info data for the error rmessage, we can update
it with the generation number after it's read.
2018-02-20 20:54:01 -05:00
James Bardin 1fee5ae172 update gcs to use the new backend tests 2018-02-20 20:36:08 -05:00
Nic Cope 011841124b Support 'customer supplied encryption keys' in the GCS backend
https://cloud.google.com/storage/docs/encryption#customer-supplied

GCS state created using customer supplied encryption keys can only be read or
modified using the same key.
2017-12-17 19:27:52 -08:00
James Bardin 52eced589f accept a path or contents for credentials
Match the operation of the google provider, by accepting either a file
path or contents for both `credentials` and `GOOGLE_CREDENTIALS`
2017-12-06 18:33:59 -05:00
James Bardin 9dea2f78d4 create unique buckets for each test, and clean up
This creates a unique bucket name for each test, so that the tests in
parallel don't collide, and buckets left over from interrupted tests
don't cause future failures.

Also make sure that buckets are removed, regardless of content.
2017-12-06 18:33:59 -05:00
James Bardin aec45e6967 backends should never create unmanaged resources
The backend was creating bucket named in the configuration if it didn't
exist. We don't allow other backends to do this, because these are not
managed resources that terraform can control.
2017-12-06 18:32:41 -05:00
Kaveh Mousavi Zamani 7507e3cd21 backend/gcs: fix locking issue when used with terraform_remote_state
Previously there was a problem with double-locking when using the GCS backend with the terraform_remote_state data source.

Here we adjust the locking methodology to avoid that problem.
2017-12-06 09:36:16 -08:00
Nic Cope 27ba7de77d backend/gcs: automatically add trailing slash to prefix
The backend doesn't function correctly if a trailing slash is not present, due to how workspaces are enumerated.
2017-11-08 11:24:36 -08:00
Florian Forster afa13a3d8e backend/remote-state/gcs: Move toBucketName to the tests. 2017-10-27 16:52:21 -04:00
Florian Forster 3023ae2813 backend/remote-state/gcs: Include project ID in bucket names when testing.
Since bucket names must be *globally* unique. By including the project
ID in the bucket name we ensure that people don't step on each other's
feet when testing.
2017-10-27 16:52:21 -04:00
Florian Forster dcb84ee2c2 backend/remote-state/gcs: Sanitize bucket names. 2017-10-27 16:52:21 -04:00
Florian Forster 454d6bbe2a backend/remote-state/gcs: Delete test buckets after tests complete.
This way tests clean up after themselves and don't leak buckets.
2017-10-27 16:52:21 -04:00
Florian Forster a6669c9e99 backend/remote-state/gcs: Don't enable versioning on new buckets.
Enabling versioning without setting up lifecycle management leads to
every lock file being archived, slowly accruing useless data.
2017-10-27 16:52:21 -04:00
Florian Forster a84823f597 backend/remote-state/gcs: Require TF_ACC for tests using the network. 2017-10-27 16:52:21 -04:00
Florian Forster 72ccf22a92 backend/remote-state/gcs: Implement additional tests.
This calls backend.TestBackend() and remote.TestRemoteLocks() for
standardized acceptance tests. It removes custom listing tests since
those are performed by backend.TestBackend(), too.

Since each tests uses its own bucket, all tests can be run in parallel.
2017-10-27 16:52:21 -04:00
Florian Forster 927085289d backend/remote-state/gcs: Implement the "region" config option.
This allows to select the region in which a bucket is created.
This copies behavior from the Google Cloud provider.
2017-10-27 16:52:21 -04:00
Florian Forster 9583d0945c backend/remote-state/gcs: Add support for the GOOGLE_PROJECT environment variable.
This copies behavior from the Google Cloud provider.
2017-10-27 16:52:21 -04:00
Florian Forster 37dc95158d backend/remote-state/gcs: Enable versioning on automatically created buckets. 2017-10-27 16:52:21 -04:00
Florian Forster df386d3133 backend/remote-state/gcs: Automatically create the bucket if needed.
This resurrects the previously documented but unused "project" option.
This option is required to create buckets (so they are associated with the
right cloud project) but not to access the buckets later on (because their
names are globally unique).
2017-10-27 16:52:21 -04:00
Florian Forster 14263223e7 backend/remote-state/gcs: Simplify initialization of the GCS client.
This also implements the (already documented) behavior of checking the
GOOGLE_CREDENTIALS environment variable.
2017-10-27 16:52:21 -04:00
Florian Forster 816c98f387 backend/remote-state/gcs: Read credentials with ioutil.ReadFile().
We never expect the raw JSON to appear in the config, so pathorcontents is
not the right package here.
2017-10-27 16:52:21 -04:00
Florian Forster c00e929ee5 backend/remote-state/gcs: Mark the "path" option as deprecated. 2017-10-27 16:52:21 -04:00
Florian Forster 91b1a39a40 backend/remote-state/gcs: Implement an end-to-end test.
The code is loosely based on state/remote/gcs_test.go. If the
GOOGLE_PROJECT environment variable is set, this test will

1) create a new bucket; error out if the bucket already exists.
2) create a new state
3) list states and ensure that the newly created state is listed
4) ensure that an object with the expected name exists
5) rum "state/remote".TestClient()
6) delete the state

The bucket is deleted when the test exits, though this may fail if the
bucket it not empty.
2017-10-27 16:51:21 -04:00
Florian Forster 14bfbf0617 backend/remote-state/gcs: Document the "prefix" option.
"state_dir" has been renamed to "prefix" to better fix the GCS
terminology.
2017-10-27 16:51:21 -04:00
Florian Forster c054bd0939 backend/remote-state/gcs: Rename "gcloud" to "gcs" for backwards compatibility. 2017-10-27 16:51:21 -04:00