Commit Graph

828 Commits

Author SHA1 Message Date
James Bardin eadda50f02 Merge pull request #15652 from hashicorp/jbardin/state-command
state commands with remote state backends
2017-08-03 13:29:45 -04:00
James Bardin 55d18dcef2 update state rm amd mv docs
Update the documentation to match the current behavior, and make the
usage output and website docs match.
2017-08-03 13:24:23 -04:00
Martin Atkins 8a7a0a7459 command: terraform init -from-module=...
This restores the earlier behavior of the first positional argument to
terraform init in 0.9, but as a command line option.

The positional argument was removed to improve consistency with other
commands that take a working directory as their first positional argument.
It was originally intended that this functionality would return in a
later release along with some other general improvements to Terraform's
module handling, but we're introducing here an interim solution that
uses the existing module source concept, to allow for easier porting of
workflows that previously depended on the automatic copy behavior.

In a future release this feature may change again as the module
improvements design firms up, but we expect it to be broadly compatible
with this temporary state.
2017-07-28 15:23:29 -07:00
James Bardin 51547ad2ba add tests for state commands through a backend 2017-07-27 18:06:47 -04:00
James Bardin 5b4ae36cb0 don't print help for state loading errors
These already include detailed messages, and it's not a usage issue,
it's a config or file location issue.
2017-07-27 18:06:47 -04:00
James Bardin 33ba6774e0 Make the state commands use the real command.Meta
In order to use a backend for the state commands, we need an initialized
meta. Use a single Meta instance rather than temporary ones to make sure
the backends are initialized properly.
2017-07-27 15:33:50 -04:00
James Bardin 45a9edb763 make state mv and rm work with remote states
The default value for the -state flag was overriding the location of any
remote state.
2017-07-27 09:41:39 -04:00
James Bardin 0bca383b6b Merge pull request #15528 from alrs/command-swallowed-errors
Fix swallowed errors in command package.
2017-07-20 08:57:10 -04:00
Lars Lehtonen 119996b815
Return immediately after newly-added error condition 2017-07-19 15:46:21 -07:00
James Bardin eb02467298 Merge pull request #15588 from hashicorp/jbardin/no-graph-config
graph should not panic with no config
2017-07-18 13:10:00 -04:00
James Bardin f10163ecc7 graph should not panic with no config
The backends replace a nil module tree with an empty one before building
the graph, so the graph command needs to do the same.
2017-07-18 13:03:57 -04:00
Martin Atkins 23f9c8785e command/e2etest: an initial test for the primary workflow
This e2etest runs an init, plan, apply, destroy sequence against a test
configuration using the real template and null providers downloaded from
the official repository.

This test _does_ trample a bit on the scope of some already-existing
tests, but this is mainly just to check our assumptions about how
Terraform behaves to ensure that we can reach our main conclusion here:
that the main Terraform workflow commands interact correctly with each
other in real use and we can complete the full workflow.
2017-07-17 14:25:33 -07:00
Martin Atkins 52df81ee49 command/e2etest: test that we can install provider plugins
We already have good tests for the business logic around provider
installation, but the existing tests all stub out the main repository
server. This test completes that coverage by verifying that the installer
is able to run against the real repository and install an official release
of the template provider.
2017-07-17 14:25:33 -07:00
Martin Atkins 0e0b0d125a command/e2etest: "terraform version" test
This basic test is here primarily because it's one of the few that can
run without reaching out to external services, and so it means our usual
test runs will catch situations where the main executable build is
somehow broken.

The version command itself is not very interesting to test, but it's
convenient in that its behavior is very predictable and self-contained.
2017-07-17 14:25:33 -07:00
Martin Atkins fee61a44b4 command/e2etest: end-to-end testing harness
Previously we had no automated testing of whether we can produce a
Terraform executable that actually works. Our various functional tests
have good coverage of specific Terraform features and whole operations,
but we lacked end-to-end testing of actual usage of the generated binary,
without any stubbing.

This package is intended as a vehicle for such end-to-end testing. When
run normally under "go test" it will produce a build of the main Terraform
binary and make it available for tests to execute. The harness exposes
a flag for whether tests are allowed to reach out to external network
services, controlled with our standard TF_ACC environment variable, so
that basic local tests can be safely run as part of "make test" while
more elaborate tests can be run easily when desired.

It also provides a separate mode of operation where the included script
make-archive.sh can be used to produce a self-contained test archive that
can be copied to another system to run the tests there. This is intended
to allow testing of cross-compiled binaries, by shipping them over to
the target OS and architecture to run without requiring a full Go compiler
installation on the target system.

The goal here is not to test again functionality that's already
well-covered by our existing tests, but rather to test chains of normal
operations against the build binary that are not otherwise tested
together.
2017-07-17 14:25:33 -07:00
Lars Lehtonen 8501d83e6c
Fix swallowed errors in command package. 2017-07-11 08:01:02 -07:00
James Bardin 583cc350a9 fix minor races in workspace tests
The improved err scanner loop in meta causes these to race. There's no
need to write back to the same commands struct, so just use a new
instance in each iteration.
2017-07-06 13:49:32 -04:00
James Bardin f7f1e8e406 Sort arguments in Meta.process
Meta.process was relying on the system readdir to order the arguments,
but readdir doesn't guarantee any ordering. Read the directory contents
as a whole and sort them in place before adding the tfvars files.
2017-07-06 11:34:47 -04:00
James Bardin 0fdcf2c01e properly cleanup and print correct error messages
Some Meta tests were not cleaning up their temp directories.

The process test wasn't printing the correct arguments in the error
messages.
2017-07-06 11:33:32 -04:00
Robert Liebowitz 4bf2269784 Add test case for autoloading tfvars 2017-07-05 17:24:17 -07:00
Robert Liebowitz e67ecb5ce2 Restructure Meta.process to logically group code 2017-07-05 17:24:17 -07:00
Robert Liebowitz 8d98fdecac Autoload only .auto.tfvars files 2017-07-05 17:24:17 -07:00
Robert Liebowitz 006744bfe0 Use all tfvars files in working directory
As a side effect, several commands that previously did not have a failure
state can now fail during meta-parameter processing.
2017-07-05 17:24:17 -07:00
Martin Atkins 39c4d6ab1f Merge #15424: Improve robustness of state persistence handling
Previously the APIs for state persistence and management had some problematic cases where we depended on hidden mutations of the state structure as side-effects of otherwise-innocent-looking operations, which was a frequent cause of accidental regressions due to faulty assumptions.

This new model attempts to isolate certain state mutations to just within the state managers, and makes the state managers work on separated snapshots of the state rather than on the "live" object to reduce the risk of race conditions.
2017-07-05 16:27:08 -07:00
Martin Atkins fee1197cf9 command: terraform state rm to require at least one argument
Due to how the state filter machinery works, passing no arguments is valid
and matches _all_ resources.

It is very unlikely that someone wants to remove everything from state, so
this ends up being a very dangerous default for the "terraform state rm"
command, and surprising for someone who perhaps runs it looking for the
usage information.

So we'll be pragmatic here and reject the no-arguments case for this
command, accepting that it makes the unlikely case of intentionally
deleting all resources harder in order to make it less likely that it
will happen _unintentionally_.

If someone does really want to remove all resources from the state, they
can provide an explicit empty string argument, but this isn't documented
because it's a weird case that doesn't seem worth mentioning.

This fixes #15283.
2017-07-05 16:19:32 -07:00
James Bardin fb397060eb add some Serial checks to apply and refresh tests 2017-07-05 18:18:28 -04:00
James Bardin 054716c397 use testStateRead helper in apply tests
We have a helper function that we hardly ever use.
TODO: convert the rest of the manual ReadState calls eventually.
2017-07-05 18:18:16 -04:00
James Bardin 501cbeaffe testState shouldn't rely on mods from WriteState
The state returned from the testState helper shouldn't rely on any
mutations caused by WriteState. The Init function (which is analogous to
NewState) shoudl set any required fields.
2017-07-05 17:47:05 -04:00
Mike Helmick 9d7fce2f69 command: "terraform workspace show" to print current workspace name
This command serves as an alternative to the human-oriented list of workspaces for scripting use-cases where it's useful to know the _current_ workspace name.
2017-07-05 14:35:46 -07:00
Martin Atkins 4d53eaa6df state: more robust handling of state Serial
Previously we relied on a constellation of coincidences for everything to
work out correctly with state serials. In particular, callers needed to
be very careful about mutating states (or not) because many different bits
of code shared pointers to the same objects.

Here we move to a model where all of the state managers always use
distinct instances of state, copied when WriteState is called. This means
that they are truly a snapshot of the state as it was at that call, even
if the caller goes on mutating the state that was passed.

We also adjust the handling of serials so that the state managers ignore
any serials in incoming states and instead just treat each Persist as
the next version after what was most recently Refreshed.

(An exception exists for when nothing has been refreshed, e.g. because
we are writing a state to a location for the first time. In that case
we _do_ trust the caller, since the given state is either a new state
or it's a copy of something we're migrating from elsewhere with its
state and lineage intact.)

The intent here is to allow the rest of Terraform to not worry about
serials and state identity, and instead just treat the state as a mutable
structure. We'll just snapshot it occasionally, when WriteState is called,
and deal with serials _only_ at persist time.

This is intended as a more robust version of #15423, which was a quick
hotfix to an issue that resulted from our previous slopping handling
of state serials but arguably makes the problem worse by depending on
an additional coincidental behavior of the local backend's apply
implementation.
2017-07-05 12:34:30 -07:00
Radek Simko 14614a5423 command/validate: Add flag to check that all variables are specified (#13872)
* command/validate: Add flag to check that all variables are specified

* Rename config-only to check-variables
2017-07-05 17:32:29 +01:00
Jake Champlin 9944ea6886
core: Skip provider checksum validation based on env var
Skips checksum validation if the `TF_SKIP_PROVIDER_VERIFY` environment variable is set. Undocumented variable, as the primary goal is to significantly improve the local provider development workflow.
2017-07-03 13:59:13 -04:00
James Bardin da1c45b566 Merge pull request #15412 from hashicorp/jbardin/windows-delete-workspace
Unlock state before workspace deletion
2017-06-27 17:47:34 -04:00
James Bardin 1c78dfb286 Unlock state before workspace deletion
We need to release the lock just before deleting the state, in case the backend
can't remove the resource while holding the lock. This is currently true for
Windows local files.

TODO: While there is little safety in locking while deleting the state, it
might be nice to be able to coordinate processes around state deletion, i.e. in
a CI environment. Adding Delete() as a required method of States would allow
the removal of the resource to be delegated from the Backend to the State
itself.
2017-06-27 14:31:08 -04:00
David Glasser 14af879fe0 command: also print plan for destroy 2017-06-27 11:22:31 -07:00
David Glasser 039d36bf91 command: add "apply -auto-approve=false" flag
A common reason to want to use `terraform plan` is to have a chance to
review and confirm a plan before running it.  If in fact that is the
only reason you are running plan, this new `terraform apply -auto-approve=false`
flag provides an easier alternative to

    P=$(mktemp -t plan)
    terraform refresh
    terraform plan -refresh=false -out=$P
    terraform apply $P
    rm $P

The flag defaults to true for now, but in a future version of Terraform it will
default to false.
2017-06-27 11:22:26 -07:00
James Bardin 22074f1029 Merge pull request #15408 from hashicorp/jbardin/import-plugin-dir
Make sure import uses complete plugin path
2017-06-27 13:07:47 -04:00
Patrick Aikens 7775ef809b Display correct to and from backends in copy message when migrating to new remote state (#15318) 2017-06-27 19:05:45 +03:00
James Bardin 85b9069eb7 test import with -plugin-dir
The import command wasn't loading the full plugin path for discovery.
Run a basic plugin init sequence, and verify we can find a plugin (even
though the plugin is invalid and will fail).
2017-06-27 11:07:45 -04:00
James Bardin 0cbd51c365 find -plugin-dir during import
The import command wasn't loading the plugin path at all, relying on the
local directory for binaries.

Load the plugin dir into Meta, and pass in ForceLocal for consistency.
The Backend returned was going to be a Local anyway, so the added check
wasn't ensuring anything.
2017-06-26 18:28:45 -04:00
Martin Atkins 3df164502a command: don't prompt for state migration if TF_INPUT is set
The "confirm" method was directly checking the meta struct's input field,
but that only represents the -input command line flag, and doesn't
respect the TF_INPUT environment variable.

By calling the Input method instead, we check both.

This fixes #15338.
2017-06-26 11:22:29 -07:00
James Bardin 833cc9a6c5 Fix state mv/rm -backup documentation
There is only one backup made, contrary to the help text. We may want to
implement that, but the documentation should match the current behavior.
2017-06-23 14:46:09 -04:00
James Bardin 6e7baaaeff don't load the backend when -state is provided
When using a `state` command, if the `-state` flag is provided we do not
want to modify the Backend state. In this case we should always create a
local state instance.

The backup flag was also being ignored, and some tests were relying on
that, which have been fixed.
2017-06-23 14:41:49 -04:00
James Bardin 5f939b42fe test that `state mv -state` doesn't use Backend
If we provide a -state flag to a state command, we do not want terraform
to modify the backend state. This test fails since the state specified
in the backend doesn't exist
2017-06-23 14:39:37 -04:00
James Bardin 4893fcc24f improve plugin error formatting
Remove "checksum" from the error, and only indicate that the plugin has
changed.

Always show requested versions even if it's "any", and found versions of
plugins.
2017-06-22 15:31:36 -04:00
James Bardin 103ab20b00 add required and found versions to error output
Provide the user with feedback showing the version requirements and the
versions found when there's is a plugin error
2017-06-22 14:20:50 -04:00
James Bardin 7a955f990c make display plugin checksum error to user
The error follows a generic message, so can be ignored by users who may
not understand the implications.
2017-06-22 13:38:55 -04:00
Martin Atkins 0dc6d97a37 command/format: minor adjustments to plan rendering
This change makes various minor adjustments to the rendering of plans
in the output of "terraform plan":

- Resources are identified using the standard resource address syntax,
  rather than exposing the legacy internal representation used in the
  module diff resource keys. This fixes #8713.

- Subjectively, having square brackets in the addresses made it look more
  visually "off" when the same name but with different indices were
  shown together with differing-length "symbols", so the symbols are now
  all padded and right-aligned to three characters for consistent layout
  across all operations.

- The -/+ action is now more visually distinct, using several different
  colors to help communicate what it will do and including a more obvious
  "(new resource required)" marker to help draw attention to this not
  being just an update diff. This fixes #15350.

- The resources are now sorted in a manner that sorts index [10] after
  index [9], rather than after index [1] as we did before. This makes it
  easier to scan the list and avoids the common confusion where it seems
  that there are only 10 items when in fact there are 11-20 items with
  all the tens hiding further up in the list.
2017-06-22 07:03:23 -07:00
Martin Atkins 40e2fbb8e9 command: init to allow plugin init without backend init
Previously init would crash if given these options:
    -backend=false -get-plugins=true

This is because the state is used as a source of provider dependency
information, and we need to instantiate the backend to get the state.

To avoid the crash, we now use the following adjusted behavior:

- if -backend=true, we behave as before

- if -backend=false, we instead try to instantiate the backend the same
  way any other command would, without modifying its configuration

- if we're able to instantiate the backend, we use it to fetch state
  for dependency resolution purposes

- if the backend is not instantiable then we assume it's not yet
  configured and proceed with a nil state, which may cause us to see an
  incomplete picture of the dependencies but still allows the install
  to succeed. Subsequently running "terraform plan" will not work until
  the backend is (re-)initialized, so the incomplete picture of required
  plugins is safe.
2017-06-21 11:43:28 -07:00
Martin Atkins 3dc7d618f7 command: avoid empty string constraints in plugin install errors
This takes care of a few dangling cases where we were still stringifying
empty version constraints, which creates confusing error messages due to
it stringing as the empty string.

For the "no suitable versions available" message, we fall back on the
"provider not found" message if no versions were found even though it's
unconstrained. This should only happen in an edge case where the
provider's index page exists on the releases server but no versions are
yet present.

For the message about plugin protocol versions, this again is an edge
case since with no constraints this should happen only if we release
an incompatible Terraform version but don't release a new version of the
plugin that's compatible. In this case we just show the constraint as
"(any version)" to make sure we always show _something_.
2017-06-21 09:18:17 -07:00