Commit Graph

195 Commits

Author SHA1 Message Date
James Bardin b20ab9f0bb Merge pull request #16243 from hashicorp/jbardin/s3-errors
retry on s3 backend internal errors
2017-10-04 09:26:26 -04:00
bclodius fba02f0bea retry on s3 backend internal errors
Internal errors from S3 are usually transient, and can be immediately retried.
Make 2 attempts at retreiving the state object before returning an error.
2017-10-04 09:11:30 -04:00
James Bardin 91442b7146 Merge pull request #15680 from brunomcustodio/etcdv3-backend
[WIP] etcd v3 backend with lock support.
2017-10-03 14:15:58 -04:00
James Bardin d477d1f6d4 Merge pull request #15553 from bonifaido/custom_s3_backend
Allow non-AWS S3 backends
2017-10-02 19:39:48 -04:00
James Bardin e0ee1e4d6e Merge pull request #16070 from octo/defaultdatadir
backend/local: Remove unused const DefaultDataDir.
2017-10-02 16:27:23 -04:00
Martin Atkins 0fe43c8977 cli: allow disabling "next steps" message in terraform plan
In #15884 we adjusted the plan output to give an explicit command to run
to apply a plan, whereas before this command was just alluded to in the
prose.

Since releasing that, we've got good feedback that it's confusing to
include such instructions when Terraform is running in a workflow
automation tool, because such tools usually abstract away exactly what
commands are run and require users to take different actions to
proceed through the workflow.

To accommodate such environments while retaining helpful messages for
normal CLI usage, here we introduce a new environment variable
TF_IN_AUTOMATION which, when set to a non-empty value, is a hint to
Terraform that it isn't being run in an interactive command shell and
it should thus tone down the "next steps" messaging.

The documentation for this setting is included as part of the "...in
automation" guide since it's not generally useful in other cases. We also
intentionally disclaim comprehensive support for this since we want to
avoid creating an extreme number of "if running in automation..."
codepaths that would increase the testing matrix and hurt maintainability.

The focus is specifically on the output of the three commands we give in
the automation guide, which at present means the following two situations:

* "terraform init" does not include the final paragraphs that suggest
  running "terraform plan" and tell you in what situations you might need
  to re-run "terraform init".
* "terraform plan" does not include the final paragraphs that either
  warn about not specifying "-out=..." or instruct to run
  "terraform apply" with the generated plan file.
2017-09-14 10:51:41 -07:00
Florian Forster ce85600cd4 backend/local: Remove unused const DefaultDataDir.
Not to be confused with the const of the same name in the "command"
package.
2017-09-12 08:20:12 +02:00
Bruno Miguel Custodio 524c3934c6
Add TLS support. 2017-09-09 08:03:59 +01:00
Bruno Miguel Custodio c8ff10f603
Reuse 'etcd' client. 2017-09-09 00:21:23 +01:00
Bruno Miguel Custodio 6daf1d9d84
Allow for username and password to be read from envvars. 2017-09-08 23:41:27 +01:00
Bruno Miguel Custodio 8f7b315037
Add documentation. 2017-09-08 23:40:05 +01:00
Bruno Miguel Custodio 54dc50ec5c
Cleanup 'etcd' after every test. 2017-09-08 22:49:23 +01:00
Bruno Miguel Custodio 3c21b9c56f
Get rid of 'keyEnvPrefix'. 2017-09-08 22:34:15 +01:00
Bruno Miguel Custodio 6570948bea
Fixing the test suite. 2017-09-08 17:46:39 +01:00
Bruno Miguel Custodio 038f5eb638
Remove 'lockSuffix' as not needed in 'etcd' v3. 2017-09-08 16:59:39 +01:00
Bruno Miguel Custodio bb4dec6032
Make sure we don't relock. 2017-09-08 16:22:14 +01:00
Bruno Miguel Custodio b8f4f6d3e2
Delete lock info when unlocking. 2017-09-08 15:21:06 +01:00
Bruno Miguel Custodio b896348230
Sort the result of 'Backend.States()'. 2017-09-08 12:25:15 +01:00
Bruno Miguel Custodio 70aad79b6e
Make 'endpoints' a schema.TypeList. 2017-09-08 12:16:00 +01:00
Bruno Miguel Custodio fee5cabeda
Make 'gofmt' happy. 2017-09-08 10:22:58 +01:00
Bruno Miguel Custodio 52c97e9fc9
First steps towards an 'etcdv3' backend. 2017-09-08 09:43:33 +01:00
tombuildsstuff d074b0da29 Obtaining the current metadata before setting it 2017-09-06 12:41:05 -07:00
tombuildsstuff a10d23dd95 Removing dead code 2017-09-06 12:41:05 -07:00
Minkyu Kim f12a237747 Fix crash when using consul backend 2017-09-02 20:05:30 +09:00
Martin Atkins 83414beb8f command: various adjustments to the diff presentation
The previous diff presentation was rather "wordy", and not very friendly
to those who can't see color either because they have color-blindness or
because they don't have a color-supporting terminal.

This new presentation uses the actual symbols used in the plan output
and tries to be more concise. It also uses some framing characters to
try to separate the different stages of "terraform plan" to make it
easier to visually navigate.

The apply command also adopts this new plan presentation, in preparation
for "terraform apply" (with interactive plan confirmation) becoming the
primary, safe workflow in the next major release.

Finally, we standardize on the terminology "perform" and "actions" rather
than "execute" and "changes" to reflect the fact that reading is now an
action and that isn't actually a _change_.
2017-09-01 17:55:05 -07:00
Martin Atkins 3ea159297c command/format: improve consistency of plan results
Previously the rendered plan output was constructed directly from the
core plan and then annotated with counts derived from the count hook.
At various places we applied little adjustments to deal with the fact that
the user-facing diff model is not identical to the internal diff model,
including the special handling of data source reads and destroys. Since
this logic was just muddled into the rendering code, it behaved
inconsistently with the tally of adds, updates and deletes.

This change reworks the plan formatter so that it happens in two stages:
- First, we produce a specialized Plan object that is tailored for use
  in the UI. This applies all the relevant logic to transform the
  physical model into the user model.
- Second, we do a straightforward visual rendering of the display-oriented
  plan object.

For the moment this is slightly overkill since there's only one rendering
path, but it does give us the benefit of letting the counts be derived
from the same data as the full detailed diff, ensuring that they'll stay
consistent.

Later we may choose to have other UIs for plans, such as a
machine-readable output intended to drive a web UI. In that case, we'd
want the web UI to consume a serialization of the _display-oriented_ plan
so that it doesn't need to re-implement all of these UI special cases.

This introduces to core a new diff action type for "refresh". Currently
this is used _only_ in the UI layer, to represent data source reads.
Later it would be good to use this type for the core diff as well, to
improve consistency, but that is left for another day to keep this change
focused on the UI.
2017-09-01 17:55:05 -07:00
James Bardin 76eb65c911 Fix crash in consul backend
A TLS config was being assigned to a Transport in a nil http.Client. The
Transport is built in the consul config by default, but the http.Client
is not built until later in NewClient.
2017-08-31 09:29:43 -04:00
Martin Atkins c12d64f340 Use t.Helper() in our test helpers
Go 1.9 adds this new function which, when called, marks the caller as
being a "helper function". Helper function stack frames are then skipped
when trying to find a line of test code to blame for a test failure, so
that the code in the main test function appears in the test failure output
rather than a line within the helper function itself.

This covers many -- but probaly not all -- of our test helpers across
various packages.
2017-08-28 09:59:30 -07:00
Loïc Albertin 21b287e919 backend/consul: allow specifying various Consul client security options
These new options are equivalent to the consul CLI arguments -ca-file, -client-cert and -client-key.
2017-08-23 09:47:21 -07:00
James Bardin 5d50e764ea deprecate the "azure" backend name
Alert users to use the current "azurerm" naming.
2017-08-17 12:49:23 -04:00
James Bardin 0e873d70ed Merge pull request #15840 from hashicorp/jbardin/deprecate-backend
add the ability to mark a backend as deprecated
2017-08-17 12:39:05 -04:00
James Bardin cc454fcc56 add the ability to mark a backend as deprecated
This is a simple wrapper to allow a backend to be marked as deprecated
during initialization.
2017-08-17 12:29:19 -04:00
Peter McAtominey b74b26bcfa backend: alias azurerm to azure 2017-08-17 16:37:51 +01:00
Peter McAtominey 6b9b5ec297 backend/azure: update to latest SDK 2017-08-17 16:32:18 +01:00
Peter McAtominey f9e8e54835 backend: convert Azure remote state to a backend
Added locking support via blob leasing (requires that an empty state is
created before any lock can be acquired.

Added support for "environments" in much the same way as the S3 backend.
2017-08-17 16:32:17 +01:00
James Bardin 2bb5007690 Merge pull request #15683 from hashicorp/jbardin/remote-state-lineage
Remove strict lineage check in remote.State
2017-08-09 17:49:19 -04:00
James Bardin 4c7cd549cd don't allow leading slashes in s3 remote state key
S3 accepts objects with a leading slash and strips them off. This works
fine except in our workspace hierarchy, which then can no longer find
suffixes matching the full key name.
2017-08-08 09:47:44 -04:00
James Bardin dc30761497 Merge pull request #15567 from hashicorp/jbardin/consul-update
Update consul vendor and remote state
2017-08-03 13:20:31 -04:00
James Bardin c3e943bed2 add another failing test for remote.State lineage
Want to make sure we don't hit this again.
2017-08-01 19:34:21 -04:00
James Bardin 18d71f273e make inmem behave more like remote backends
When remote backend imeplemtations create a new named state, they may
need to acquire a lock and/or save an actual empty state to the backend.
Copy this behavior in the inmem backend for testing.
2017-08-01 19:34:21 -04:00
James Bardin 5deef9621d export Reset()
This package is used for testing, so there needs to be an easy method
for reinitializing the stored data between tests.
2017-08-01 19:34:21 -04:00
James Bardin ac60ddcd40 Support named states in inmeme backend
Used to expand test coverage
2017-08-01 19:34:21 -04:00
James Bardin 1070d04b07 Remove consul context adapter for RenewPeriodic
Updated the vendored consul which no longer requires the channel adapter
to convert a `chan stuct{}` to a `<-chan struct{}`.

Call testutil.NewTestServerConfigT with the new signature.
2017-07-20 11:09:45 -04:00
James Bardin 9a7ffbfb1b Merge pull request #15566 from hashicorp/jbardin/state-serial
Add warning to mismatched plan state
2017-07-18 18:09:00 -04:00
James Bardin dd9a54c661 Merge pull request #15435 from hashicorp/jbardin/state-path-message
only show state path help if state is local
2017-07-18 18:08:18 -04:00
James Bardin 50f412bff4 Merge pull request #15557 from hashicorp/jbardin/consul-lock-sessions
have the consul client manage the lock session
2017-07-17 10:56:30 -04:00
James Bardin a1727ec4c2 Add warning to mismatched plan state
Forward-port the plan state check from the 0.9 series.
0.10 has improved the serial handling for the state, so this adds
relevant comments and some more test coverage for the case of an
incrementing serial during apply.
2017-07-17 10:41:29 -04:00
James Bardin bcb11f6d89 have the consul client manage the lock session
When a consul lock is lost, there is a possibility that the associated
session is still active. Most commonly, the long request to watch the
lock key may error out, while the session is continually refreshed at a
rate of TTL/2.

First have the lock monitor retry the lock internally for at least 10
seconds (5 attempts with the default 2 second wait time). In most cases
this will reconnect on the first try, keeping the lock channel open.

If the consul lock can't recover itself, then cancel the session as soon
as possible (terminating the PreiodicRenew will call Session.Destroy),
and start over. In the worse case, the consul agents were split, and the
session still exists on the leader so we may need to wait for the old
session TTL, plus the LockWait time to renew the lock.

We use a Context for the cancellation channels here, because that
removes the need to worry about double-closes and nil channels. It
requires an awkward adapter goroutine for now to convert the Done()
`<-chan` to a `chan` for PeriodicRenew, but makes the rest of the code
safer in the long run.
2017-07-14 14:42:42 -04:00
James Bardin 193d4b868c backend state tests must honor lineage
Remote state implementations may initialize a lineage when creating a
new named state (i.e. "workspace"). The tests were ignoring that initial
lineage to write a new state to the backend.
2017-07-14 13:50:26 -04:00
Nandor Kracser f6c77339f2 Allow non-AWS S3 backends
This commit makes sts, metadata and other AWS related API calls optional, thus the backend initialization will not send non-AWS API tokens to AWS APIs
2017-07-14 13:08:47 +02:00