Commit Graph

1224 Commits

Author SHA1 Message Date
Radek Simko 9e5677adeb
Merge pull request #19453 from hashicorp/b-whole-body-diag-rendering
command/format: Fix rendering of attribute-agnostic diagnostics
2018-11-26 23:52:21 +00:00
Radek Simko 501fcd1e21
Fix tests after upgrading hcl 2018-11-26 23:38:37 +00:00
Radek Simko 4f2868a019
command/format: Fix rendering of attribute-agnostic diagnostics 2018-11-26 23:38:37 +00:00
Sander van Harmelen ef9054562e commands: make sure the correct flagset is used
A lot of commands used `c.Meta.flagSet()` to create the initial flagset for the command, while quite a few of them didn’t actually use or support the flags that are then added.

So I updated a few commands to use `flag.NewFlagSet()` instead to only add the flags that are actually needed/supported.

Additionally this prevents a few commands from using locking while they actually don’t need locking (as locking is enabled as a default in `c.Meta.flagSet()`.
2018-11-23 16:13:34 +01:00
Sander van Harmelen 79a9a15879 command/state: lock when pushing state
Next to adding the locking for the `state push` command, this commit also fixes a small bug where the lock would not be propertly released when running the `state show` command.

And finally it renames some variables in the `[un]taint` code in order to try to standardize the var names of a few frequently used variables (e.g. statemgr.Full, states.State, states.SyncState).
2018-11-20 11:15:16 +01:00
Martin Atkins 884aa387b8 command: Use vendoring when building helper programs in tests
In a couple places in tests we execute a child "go build" to make a helper
program. Now that we're running in module mode, "go build" will normally
default to downloading and caching dependencies, which we don't want
because we're still using vendoring for the moment.

Therefore we need to instruct these child builds to use vendoring too,
avoiding the need to download all of the dependencies and ensuring that
we'll be building with the same dependencies that we'd use for a normal
build.
2018-11-19 11:41:52 -08:00
Martin Atkins 5255e85238 "go fmt" fixups
Apparently my editor is still not reliably formatting on save, so I missed
a few formatting quirks in these files.
2018-11-19 09:02:35 -08:00
Martin Atkins 73c9521a04 command/e2etest: Temporarily disable tests that access network
Several of these tests rely on external services (e.g. Terraform Registry)
that have not yet been updated to support the needs of Terraform v0.12.0,
so for now we'll skip all of these tests and wait until those systems have
been updated.

This should be removed before Terraform v0.12.0 final to enable these
tests to be used as part of pre-release smoke testing.
2018-11-19 09:02:35 -08:00
Martin Atkins 37bc187f95 command: "terraform output" mustn't panic when no state is present
This is verified by TestOutput_noArgs.
2018-11-19 09:02:35 -08:00
Martin Atkins 53b5b95ef5 command: Fix TestRefresh_backup
The local filesystem state manager no longer creates backup files eagerly,
instead creating them only if on first write there is already a snapshot
present in the target file.

Therefore for this test to exercise the codepaths it intends to we must
create an initial state snapshot for it to overwrite, creating the backup
in the process.

There are several other tests for this behavior elsewhere, so this test
is primarily to verify that the refresh command is configuring the backend
appropriately to get the backups written in the desired location.
2018-11-19 09:02:35 -08:00
Martin Atkins 762a173c7f command: Fix TestRefresh_outPath
We now only create a backup state file if the given output file already
exists, which it does not in this test.

(The behavior of creating the backup files is already covered by other
tests, so no need for this one go out of its way to do it.)
2018-11-19 09:02:35 -08:00
Martin Atkins e2ba90fdfa command: Fix TestMetaBackend_planLocalMatch
We now don't create a local state backup until the first snapshot write,
so we don't expect there to be a backup file until the end of the test.
(There is already a check at the end there, unmodified by this change.)
2018-11-19 09:02:35 -08:00
Martin Atkins 48601d261d states/statemgr: In Filesystem, back up output file, not input file
The filesystem backend has the option of using a different file for its
initial read.

Previously we were incorrectly writing the contents of that file out into
the backup file, rather than the prior contents of the output file. Now
we will always read the output file in RefreshState in order to decide
what we will back up but then we will optionally additionally read the
input file and prefer its content as the "current" state snapshot.

This is verified by command.TestMetaBackend_planLocalStatePath and
TestMetaBackend_configureNew, which are both now passing.
2018-11-19 09:02:35 -08:00
Martin Atkins 27abd9c6b8 command: Fix TestMetaBackend_localDoesNotDeleteLocal
The changes to how we handle setting the state path on the local backend
broke the heuristic we were using here for detecting migration from one
local backend to another with the same state path, which would by default
end up deleting the state altogether after migration.

We now use the StatePaths method to do this, which takes into account
both the default values and any settings that have been set.

Additionally this addresses a flaw in the old method which could
potentially have deleted all non-default workspace state files if the
"path" setting were changed without also changing the "workspace_dir"
setting. This new approach is conservative because it will preserve all
of the files if any one overlaps.
2018-11-19 09:02:35 -08:00
Martin Atkins be79bf0412 command: Fix TestPlan_outBackend
In an earlier change we fixed the "backendFromConfig" codepath to be
able to properly detect changes to the -backend-config arguments during
"terraform init", but this detection is too strict for the normal case
of running an operation in a previously-initialized directory.

Before any of the recent changes, the logic here was to selectively update
the hash to include -backend-config settings in the init case. Since
that late hash recalculation was confusing, here we take the alternative
path of using the hash only in the normal case and full value comparison
in the init case. Treating both of these cases separately makes things
marginally easier to follow here.
2018-11-19 09:02:35 -08:00
Martin Atkins b316e4ab56 command: Fix TestImport_remoteState
The import command was imposing the default state path at the CLI level,
rather than leaving that to be handled by the backend. As a result, the
output state was always forced to be terraform.tfstate, regardless of
the backend settings.
2018-11-19 09:02:35 -08:00
Martin Atkins 2b9f92be31 command: Partially fix TestMetaBackend_planLocalStatePath
This test is testing some strange implementation details of the old
local backend which do not hold with the new filesystem state manager.
Specifically, it was expecting state to be read from the stateOutPath
rather than the statePath, which makes no sense here because the backend
is configured to read from the default terraform.tfstate file (which does
not exist.)

There is another problem with this test which will be addressed in a
subsequent commit.
2018-11-19 09:02:35 -08:00
Martin Atkins aecb66d3db command: Fix TestMetaBackend_configuredChangeCopy_multiToNoDefaultWithoutDefault
As part of integrating the new "remote" backend we relaxed the requirement
that a "default" workspace must exist in all backends and now skip
migrating empty workspace states to avoid creating unnecessary "default"
workspaces when switching between backends that require it and backends
that don't, such as when switching from the local backend (which always
has a "default" workspace) to Terraform Enterprise.
2018-11-19 09:02:35 -08:00
Martin Atkins ec27526cc3 command: Fix TestMetaBackend_configuredChangeCopy_multiToMulti
This was failing because we now handle the settings for the local backend
a little differently as a result of decoding it with the HCL2 machinery.

Specifically, the backend.State* fields are now assumed to be what is
given in configuration, and any CLI overrides are maintained separately
in OverrideState* fields so that they can be imposed "just in time" in
StatePaths.

This is particularly important because OverrideStatePath (when set) is
used regardless of workspace name, while StatePath is a suitable value
only for the "default" workspace, with others needing to be constructed
from StateWorkspaceDir instead.
2018-11-19 09:02:35 -08:00
Martin Atkins 6c7cecfbd8 command: More logging during migration
This just finishes off the logging added in earlier commits to get all
the way through to the actual migration call.
2018-11-19 09:02:35 -08:00
Martin Atkins f39a5d0962 command: Fix various TestMetaBackend-prefix tests
Our new state model has a different implementation of "empty" that doesn't
consider lineage/serial, so we need to have some actual content in these
state fixtures to avoid them being skipped during state migrations.
2018-11-19 09:02:35 -08:00
Martin Atkins 22c84c71a4 command: Use statemgr.Import and statemgr.Export for state push and pull
We previously hacked around the import/export functionality being missing
in the statemgr layer after refactoring, but now it's been reintroduced
to fix functionality elsewhere we should use the centralized Import and
Export functions to ensure consistent behavior.

In particular, this pushes the logic for checking lineage and serial
during push down into the state manager itself, which is better because
all other details about lineage and serial are managed within the state
managers.
2018-11-19 09:02:35 -08:00
Martin Atkins aacbe1d14b command: Fix TestMetaBackend_configureNewWithState
This test was initially failing because its fixture had a state which our
new state models consider to be "empty", and thus it was not migrated.

After fixing that (by adding an output to the fixture), this revealed a
bug that the lineage was not being persisted through the migration. This
is fixed by using the statemgr.Migrate method instead of writing via the
normal Writer interface, which allows two cooperating state managers to
properly transfer the lineage and serial along with the state snapshot.
2018-11-19 09:02:35 -08:00
Martin Atkins 24046ab833 command: More TRACE logging for the Backend instantiation codepaths 2018-11-19 09:02:35 -08:00
Martin Atkins 2293391241 command: Fix TestMetaBackend_emptyWithExplicitState
This test was incorrectly updated in a previous iteration, with it
creating a modified state to write but then not actually writing it,
writing an empty test state instead.

This made the test fail because a backup state file is created only if
the new state snapshot is different to the old when written.
2018-11-19 09:02:35 -08:00
Martin Atkins 8c54da0ad2 command: TestInit_fromModule_explicitDest guard against other tests
Some other test is leaving behind a terraform.tfstate after it concludes,
which can cause this test to fail in a strange way due to picking up
extra provider requirements from that state.

This check doesn't fix that problem, but it at least makes the test fail
in a more helpful way to avoid time wasted trying to debug this test when
it's some other test that actually has the bug.
2018-11-12 15:19:55 -08:00
Martin Atkins ced06a4ca3 command: fix panic in TestStatePush_lineageMismatch
This test is currently failing due to the command completing successfully,
which would previously cause a panic because we didn't properly initialize
the MockUi and so its error buffer is nil unless written to.

(The failure this was masking will be fixed in a subsequent commit.)
2018-11-12 15:19:55 -08:00
Martin Atkins 544c2932ce command: Fix TestInit_checkRequiredVersion
In prior refactoring we lost the required core version check from
"terraform init", which we restore here.

Additionally, this test used to have an incorrect name that suggested it
was testing something in the "getProvider" codepath, but version checking
happens regardless of what other options are selected.
2018-11-12 15:19:55 -08:00
Martin Atkins 9ba399bca8 command: Fix TestInit_getProvider
After all of the refactoring we were no longer checking the Terraform
version field in a state file, causing this test to fail.

This restores that check, though with a slightly different error message.
2018-11-12 15:19:55 -08:00
Martin Atkins f6d468ffd5 command: Fix TestInit_inputFalse
This test was using old-style state files as its input, differing only by
lineage. Since lineages are now managed within the state manager itself,
the test can't use that to distinguish the two files and so we put a
different output in each one instead.

This also introduces some TRACE logging to the migration codepaths.
There's some hard-to-follow control flow here and so this extra logging
helps to understand the reason for a particular outcome, and since this
codepath is visited only in "terraform init" anyway it doesn't hurt to
be a bit more verbose here.
2018-11-12 15:19:55 -08:00
Martin Atkins c0b7f58143 command: Fix detection of necessary backend migration
In the refactoring for new HCL this codepath stopped taking into account
changes to the CLI -backend-config options when deciding if a backend
migration is required.

This restores that behavior in a different way than it used to be: rather
than re-hashing the merged config and comparing the hashes, we instead
just compare directly the configuration values, which must be exactly
equal in order to skip migration.

This change is covered by the test TestInit_inputFalse, although as of
this commit it is still not passing due a downstream problem within the
migration code itself.
2018-11-12 15:19:55 -08:00
Sander van Harmelen 5e9414bf03 backend/migrations: only select workspaces if supported
If the backend does not support workspaces, we don't have to try to
select a workspace and we should not return an error.
2018-11-12 16:51:32 +01:00
Martin Atkins 297b3b8830 command: Fix TestInit_backendReinitConfigToExtra
This test was re-using the same InitCommand value to run multiple times,
which is not realistic. Since we now cache configuration source code
inside command.Meta on load, it's important that we use a fresh
InitCommand instance here so it'll see the modified configuration file
we've left on disk.
2018-11-09 09:48:03 -08:00
Martin Atkins b5547f00f6 command: backendConfig must mutate its copy, not the original
Here we were going to the trouble of copying the body so we could mutate
it, but then ended up mutating the original anyway and then returning the
unmodified copy. Whoops!

This fix is verified by a number of "init" command tests that exercise the
-backend-config option, including TestInit_backendConfigFile and several
others whose names have the prefix TestInit_backendConfig .
2018-11-09 09:48:03 -08:00
Martin Atkins 8b603e4877 command: Name the Terraform Registry specifically in error message
When we originally wrote this message we struggled a bit for how to refer
to the releases server without writing an awkwardly-ungrammatical
sentence, and so "the official repository" became a placeholder name for
it.

Now that we'll be looking in Terraform Registry this gives us a nice
proper noun to use. This message will need to evolve more as our
integration with the registry gets more sophisticated, but for now this
works.
2018-11-09 09:48:03 -08:00
Martin Atkins 592850e22c command: Fix TestApply_plan_remoteState
Some over-zealous bulk updating of this test file caused this test to be
producing a remote state config cache file on disk when it doesn't
actually need one: the backend config comes from the plan file when
applying a saved plan.
2018-11-09 09:48:03 -08:00
Martin Atkins c05a556c0e command: Fix TestRefresh_backup
This test intentionally overrides the backup file location using the
-backup option, so the backup file is not in the default location for this
one.
2018-11-09 09:48:03 -08:00
Martin Atkins ad276142b4 command: Fix the command package test build
Some merging conflict shenanigans here led to this usage not lining up
with the imported symbol name, meaning that the tests couldn't compile any
more.
2018-11-08 14:39:24 -08:00
Martin Atkins 1e45d30036 command: Fix TestPlan_outBackend
We missed fixing this up during the big updates for the new plan/state
models since the failures were being masked by testBackendState being
broken.

This is the same sort of update made to many other tests: add schema to
the mock provider, adjust for the new plan/state types, and make
allowances for the new built-in diffing behavior in core.
2018-11-08 08:57:11 -08:00
Martin Atkins c3d11b762b command: Fix testBackendState
The hashing function for cached backend configuration is different now, so
our hard-coded hash of the HTTP backend address wasn't working anymore.

Here we update the hash so that tests using this test backend will work
again. Rather than leaving it hard-coded, we'll instead compute it the
same way as "terraform init" would.

In practice only one test is actually using this function right now, so
we also update the test fixture for that test (TestPlan_outBackend) to
match the new expectations, though as of this commit it's still failing
with an unrelated error.
2018-11-08 08:57:11 -08:00
Martin Atkins e20346bf4f command: fix TestMeta_process
The mission of this process method used to include dealing with
auto-loaded tfvars files, but it doesn't do that anymore.

It does still deal with the -no-color option, but the test wasn't
exercising that part before.

Now the test here focuses on the -no-color behavior.

The process method still has a "vars" flag argument which is no longer
used. Since this is an unexported method we could potentially address this
but this commit is intentionally limited only to fixing the test.
2018-11-08 08:57:11 -08:00
Martin Atkins fcf3f643ce command: Fix TestPlan_shutdown
Comments here indicate that this was erroneously returning an error but
we accepted it anyway to get the tests passing again after other work.
The tests over in the "terraform" package agree that cancelling should be
a successful outcome rather than an error.

I think that cancelling _should_ actually be an error, since Terraform did
not complete the operation it set out to complete, but that's a change
we'd need to make cautiously since automation wrapper scripts may be
depending on the success-on-cancel behavior.

Therefore this just fixes the command package test to agree with the
Terraform package tests and adds some FIXME notes to capture the potential
that we might want to update this later.
2018-11-08 08:57:11 -08:00
Martin Atkins 0ea8aa6fe5 command: Fix TestWorkspace_createWithState
The State.Equal function is now more precise than this test needs. It's
only trying to distinguish between an empty state and a non-empty state,
so the string representation of state is good enough to get that done
while disregarding other subtle differences.
2018-11-08 08:57:11 -08:00
Sander van Harmelen 52a1b22f7a Implement the remote enhanced backend
This is a refactored version of the `remote` backend that was initially added to Terraform v0.11.8 which should now be compatible with v0.12.0.
2018-11-06 16:29:46 +01:00
Sander van Harmelen b62a22ab62 Add a VariableSourceType for names .tfvars files
This new source type should be used for variables loaded from .tfvars files that were explicitly passed as command line arguments (e.g. -var-file=foo.tfvars)
2018-11-05 19:29:34 +01:00
Sander van Harmelen 178ec8f7b4 Remove support for the -module-depth flag
# Conflicts:
#	backend/backend.go
2018-11-02 18:44:04 +01:00
Sander van Harmelen 643fccc923
Merge pull request #19250 from hashicorp/f-inputs
Fix the ability to ask for and save user input
2018-11-01 20:14:52 +01:00
Sander van Harmelen 5944e8e34f Fix the ability to ask for and save user input 2018-11-01 20:00:08 +01:00
Sander van Harmelen 5458a91985 command/state: update and fix the state show command 2018-10-27 15:15:25 +02:00
Sander van Harmelen 7ec3f96e3a command/state: update and fix the state mv command 2018-10-27 15:01:07 +02:00
Sander van Harmelen 19c1241a50 command/state: update and fix the state rm command 2018-10-24 10:59:33 +02:00
James Bardin e93d69f18b more nil/known checks before val.LengthInt 2018-10-19 16:51:15 -04:00
Sander van Harmelen 536c2fe6f1 Make `state mv` use the new `states.Filter` 2018-10-19 19:19:49 +02:00
Sander van Harmelen 5e11de460a
Merge pull request #19130 from hashicorp/f-state-push-pull
command/state: update and fix the state push and pull
2018-10-19 19:16:00 +02:00
Sander van Harmelen 7fbd93b5cd command/state: update and fix the state push and pull 2018-10-19 19:12:23 +02:00
Sander van Harmelen a88ba31b63
Merge pull request #19129 from hashicorp/f-show-list
command/state: update and fix the state list command
2018-10-19 19:10:09 +02:00
Sander van Harmelen 83168ea25f
Merge pull request #19127 from hashicorp/b-prepare-provider
Make the test compile again
2018-10-19 19:05:18 +02:00
Sander van Harmelen af1a471a05 command/state: update and fix the state list command 2018-10-19 16:31:12 +02:00
Sander van Harmelen 4d6626b3a8 Terraform show didn’t show absolute paths
Without using absolute paths any module info is lost in the output. And the attributes were randomly ordered and so changed between different executions of the command.
2018-10-19 15:38:11 +02:00
Sander van Harmelen b846c5f653 Make the test compile again
Not sure if these checks still make sense, but without this change the test don’t compile for the related packages.
2018-10-19 15:29:14 +02:00
Martin Atkins 17b883f592 command/format: Include variable values in diagnostic messages
When HCL encounters an error during expression evaluation, it annotates
its diagnostics with information about the expression that was being
evaluated and the EvalContext it was evaluated in.

This gives us enough information to show helpful hints to the user about
the final values of any reference expressions that are present in the
expression, which is very useful extra context for expressions that get
evaluated multiple times, such as:
- Any expression in a block with "count" or "for_each" set
- The sub-expressions within a "for" expression
2018-10-18 17:12:01 -07:00
James Bardin e08a388d3c check IsKnown on values that may panic 2018-10-18 19:21:32 -04:00
Sander van Harmelen 48ef7ecfa6 Updates after running `make fmt` with Go v1.11.1 2018-10-17 14:11:08 -07:00
James Bardin fe9ed37dfc minor fixes for command Apply tests 2018-10-17 09:42:15 -04:00
Martin Atkins 541952bb8f Revert some work that happened since v0.12-dev branched
This work was done against APIs that were already changed in the branch
before work began, and so it doesn't apply to the v0.12 development work.

To allow v0.12 to merge down to master, we'll revert this work out for now
and then re-introduce equivalent functionality in later commits that works
against the new APIs.
2018-10-16 19:48:28 -07:00
Martin Atkins 8e51363f04 command: Don't allow -var and -var-file when applying saved plan
This reinstates an old behavior that was lost in the reorganization of how
we deal with the -var and -var-file options.

This fix is verified by TestApply_planVars now passing.
2018-10-16 19:14:11 -07:00
Martin Atkins 7abf81d8da command: Restore support for terraform.tfvars.json
In the new implementation of collecting variables I initially forgot the
JSON variant of terraform.tfvars.

This fix is verified by TestApply_varFileDefaultJSON now passing.
2018-10-16 19:14:11 -07:00
Martin Atkins 66f96cf842 command: Un-stub and reimplement "terraform state rm"
This was previously targeting the old state manager and state types, so it
needed some considerable rework to get it working again with the new state
types.

Since our new resource address syntax lacks the weird extra .deposed
special case we had before, we instead interpret addresses as
whole-instance addresses here and remove the deposed objects along with
the current one (if present), since this is more likely to match with
user expectations because we don't consider deposed objects to be
independently addressable in any other situation.

With that said, to be more explicit about what is going on we do now have
a -dry-run mode and maintain separate counts of current and deposed
instances so that we can expose that in the UI where relevant.
2018-10-16 19:14:11 -07:00
Martin Atkins 40eda180d6 command: go fmt in import_test.go and refresh_test.go 2018-10-16 19:14:11 -07:00
Martin Atkins 8b466811d2 command: Fix a few tests from meta_backend_test.go 2018-10-16 19:14:11 -07:00
Martin Atkins 86b7963bba command: Fix tests for "terraform providers" 2018-10-16 19:14:11 -07:00
Martin Atkins 5b61cc919b command: Fix "terraform import" tests 2018-10-16 19:14:11 -07:00
Martin Atkins 7b77e20bdc command: Fix several "terraform init" tests 2018-10-16 19:14:11 -07:00
Martin Atkins 89d0944e5b command: Fix even more of the "terraform refresh" tests 2018-10-16 19:14:11 -07:00
Martin Atkins 5d1d6a95f9 command: "go fmt" of plan_test.go 2018-10-16 19:14:11 -07:00
Martin Atkins e54848b86f command: Fix many (but not all) "terraform refresh" tests 2018-10-16 19:14:11 -07:00
Martin Atkins 73abb6e8f4 command: Re-enable showing outputs after successful "apply"
We temporarily disabled this because it needed some further work to update
it for the new state models, which has now been done.

We no longer need the configuration objects for the outputs because the
state itself contains all of the information needed for displaying these.
2018-10-16 19:14:11 -07:00
Martin Atkins 98bbd560b5 command: Fix most (but not all) "terraform plan" tests 2018-10-16 19:14:11 -07:00
Martin Atkins ca314afc0d command: Make suitable provider schemas available for all "plan" tests 2018-10-16 19:14:11 -07:00
Martin Atkins 73318a436b command: go fmt 2018-10-16 19:14:11 -07:00
Martin Atkins 53bb3f57e6 command: Remove tests related to legacy remote state migrations, etc
We no longer support legacy remote state, so the behaviors these tests
were covering are no longer present.
2018-10-16 19:14:11 -07:00
Martin Atkins 275a44f552 command: Reinstate object ids in the UIHook progress logs
We used to treat the "id" attribute of a resource as special and elevate
it into its own struct field "ID" in the state, but the new state format
and provider protocol treats it just as any other attribute.

However, it's still useful to show the value of a single identifying
attribute when there isn't room in the UI for showing all of the
attributes, and so here we take a new strategy of considering "id" along
with some other conventional names as special only in the UI layer.

This new heuristic approach can be adjusted over time as new provider
patterns emerge, but for now it covers some common conventions we've seen
in real providers.

With that said, since all existing providers made for Terraform versions
prior to v0.12 were forced to set "id", we won't see any use of other
attributes here until providers are updated to remove the placeholder
ids they were generating in cases where an id was not actually relevant
but was forced by the old protocol. At that point the UX should be
improved by showing a more relevant attribute instead.

We now also allow for the possibility of no id at all, since that is valid
for resources that exist only within the Terraform state, like the ones
from the "random" and "tls" providers.
2018-10-16 19:14:11 -07:00
Martin Atkins 43d5206d82 command: Stub the "terraform state mv" tests to remind to fix
This command isn't yet updated for the new state types, but since we were
not returning a non-successful error status here the tests were just
failing in a weird way instead. Now we'll fail with a message that makes
it clear there is still work to do in the real implementation here.
2018-10-16 19:14:11 -07:00
Martin Atkins 1d64ce8ec0 command: Fix "terraform validate" tests
We now need to give a provider schema that the test fixtures conform to,
so we can validate against it.
2018-10-16 19:14:11 -07:00
Martin Atkins 025540789c command: Restore the "terraform output" functionality
We previously stubbed most of this out because it hadn't yet been updated
to support the new state types, etc.

This restores all of the previous behavior as covered by the tests.

We intentionally remove one behavior that was not covered by the tests:
we used to allow retrieval of outputs from non-root modules using the
-module option, but since we no longer persist non-root outputs in the
state we can no longer support this without a full expression evaluation
walk, and that'd be overkill for this otherwise-simple command. Descendant
module outputs are not part of the public interface of a configuration
anyway, so accessing them from outside in this way is an anti-pattern.

(For debugging scenarios it is still possible to access these from
"terraform console", which _does_ do a full evaluation graph walk to
prepare its evaluation scope.)
2018-10-16 19:14:11 -07:00
Martin Atkins feff10dcb5 command: Fix TestGraph_plan
The plan file writer requires a backend config to be present, but we don't
really need one for the sake of _this_ test, since we don't activate the
backend to render a plan graph, and so we just write in a placeholder.
2018-10-16 19:14:11 -07:00
Martin Atkins c0baedac84 command: Fix tests for "terraform fmt" command
Some underlying assumptions have shifted here, so although the behavior
is still broadly the same we need to accommodate some different details.
2018-10-16 19:14:11 -07:00
Martin Atkins 74582447bb command: Make input variable values available to "terraform console"
This fixes TestConsole_tfvars.
2018-10-16 19:14:11 -07:00
Martin Atkins 741d334ee4 command: Even more fixes for "apply" command tests 2018-10-16 19:14:11 -07:00
Martin Atkins 34a29315f7 command: More fixes (but still not all) for "apply" command tests 2018-10-16 19:14:11 -07:00
Martin Atkins de3944b9bf command: collect root module variable values for apply/plan/refresh
This connects a missing link left by earlier refactoring: the command
package is responsible for gathering up variable values provided by the
user and passing them through to the backend to use in operations.
2018-10-16 19:14:11 -07:00
Martin Atkins fc2614c939 command: Fix some tests for the "apply" command 2018-10-16 19:14:11 -07:00
Martin Atkins 34c4fd316d command: Update backend-unchanged fixture for new expectations
Our serialization of the backend configuration has changed slightly for
Terraform 0.12 due to reimplementing it in terms of the HCL2 types, so
the base case that should be unchanged during the test needs to be
changed.
2018-10-16 19:14:11 -07:00
Martin Atkins 34ebde0b95 command/format: be resilient to incomplete schema when formatting state
In all real cases the schemas should be populated here, but we don't want
to panic in UI rendering code if there's a bug here.

This can also be tripped up by tests with incomplete mocks. It's
unfortunate that this can therefore mask some problems in tests, but tests
can protect against it by asserting on specific output text rather than
just assuming that a zero exit status is a pass.
2018-10-16 19:14:11 -07:00
Kristin Laemmert 3cf1b001c2 command/format: revert indentation change for consistency's sake 2018-10-16 19:14:11 -07:00
Kristin Laemmert db26324b3c command/format: fix indentation
also run go mod tidy
2018-10-16 19:14:11 -07:00
Martin Atkins d63c2fdd8d command: Show snippet of invalid resource addresses in import
If we fail to parse the resource address given to "terraform import" then
it's helpful to produce a "source code" snippet of what the user provided
so they might see more precisely which part of the address was invalid.
2018-10-16 19:14:11 -07:00
Martin Atkins 27d086d8bd command: Fix TestUntaint 2018-10-16 19:14:11 -07:00
Martin Atkins 1bc10180cb command: Fix all of the "taint" command tests
Most of this is just updates to allow for the fact that we now always save
the provider address as part of resource state, whereas before it was only
saved conditionally.

This also updates TestTaint_module for the intentional change that it now
expects a child module to be specified using normal resource address
syntax, rather than as a separate -module option.
2018-10-16 19:14:11 -07:00
Martin Atkins 8321c3614f command: Fix TestUntaint_module
The untaint command now uses normal resource address syntax to specify the
module path, rather than passing it in as a separate argument.
2018-10-16 19:14:11 -07:00
Martin Atkins 3cee60ee3a command: Fix various "untaint" command tests
Resource instance state now always includes the provider address fo each
resource, so our expected state output strings needed to be adjusted.
2018-10-16 19:14:11 -07:00
Martin Atkins 55b9c7eacd command: Actually include the error when backend config decoding fails 2018-10-16 19:14:11 -07:00
Kristin Laemmert a4c3022cf8 command/show: config files are assumed to be in cwd 2018-10-16 19:14:11 -07:00
Kristin Laemmert d08fe7a91f cli: format/state_test.go
Added a very simple test with state and schema.
TODO: if tests are added we should test using golden files (and example
state files, instead of strings). This seemed unnecessary with the
simple test cases.
2018-10-16 19:14:11 -07:00
Kristin Laemmert 14c28b8de4 cli: format/state refactor to use blockBodyDiffPrinter
Use functions from format/diff to print values
2018-10-16 19:14:11 -07:00
Kristin Laemmert 8063f69e5c cli: format/state refactor for new state format
format/state now requires provider schemas to properly format output
state. command/show has been modified to pass a context to format.State.
2018-10-16 19:14:11 -07:00
Martin Atkins e8240087fe command: Populate backend configuration in plan files
In previous work we didn't quite connect these dots. The connection here
is sub-awesome since the existing interfaces here had some unfortunate
assumptions that we'd like to move away from (like the idea of a "nil
backend" implying the local backend) but we're accepting this for now to
avoid another big round of refactoring.

The main implication of this is that we will now always include a backend
configuration in the plan, though it might just be a placeholder config
for the local backend in the remaining cases where that's still implicitly
set. Later we will change this so that there is no implicit local backend
at all (terraform init is always required, _it_ will deal with setting
implicitly setting the local backend when appropriate), which will allow
us to rework this to be more straightforward and less "spooky".
2018-10-16 19:14:11 -07:00
Martin Atkins cbc548eb36 command: Do CLI init of backend loaded from plan
If we don't do this, we can't produce any output when applying a saved
plan file.

Here we also introduce a check to the local backend's ReportResult
function so that it won't panic if CLI init is skipped, although that
will no longer happen in the apply-from-file case due to the change
described in the previous paragraph.
2018-10-16 19:14:11 -07:00
James Bardin 52c0032aed update provisioners for multiple processes
The "internal" provisioners are still run in a separate process, and
need to be updated to restart on each walk.
2018-10-16 19:14:11 -07:00
James Bardin 795161f615 update to start a new process for each plugin
Modify the plugin factories to create a new plugin process for each
individual plugin.
2018-10-16 19:14:11 -07:00
Martin Atkins 412439ae01 command: Un-stub tests that were previously hanging on the inmem backend
The underlying bug has now been fixed,
in b3b62429a7bbeb852f9b26299732b265d9d3299f .
2018-10-16 19:14:11 -07:00
Martin Atkins 058a1d81b0 command/format: Remove tests for plan.go
plan.go as a whole will be removed before too long, so there's no point
in us updating all of these tests to use the new plan types.
2018-10-16 19:14:11 -07:00
Martin Atkins 3b8d46418f command/format: print correctly nested blocks with labels 2018-10-16 19:14:11 -07:00
Martin Atkins b0016e9cf6 command: Allow tests to run to completion without panics or hangs
There are still 160 test failures as of this commit, but at least the test
program can run to completion and list out all the failures.
2018-10-16 19:14:11 -07:00
Martin Atkins d446d32d96 command: Fix TestApply for new mock provider interface
It must now provide a basic implementation of plan and apply for its
mock provider, which in this case can just pass through the proposed value
generated by core because there are no computed attributes in this schema.
2018-10-16 19:14:11 -07:00
Martin Atkins 5021e0e098 command: Fix TestApply_destroyTargeted for new provider and state types 2018-10-16 19:14:11 -07:00
Martin Atkins db3ea65e8b command: Fix TestApply_destroy for new provider and state types 2018-10-16 19:14:11 -07:00
Martin Atkins be7bce493e command: Update plugins_test.go for new provider types
This only makes it compile. It does not yet ensure that the tests pass.
2018-10-16 19:14:11 -07:00
Martin Atkins dc0b5292a6 command: Update refresh_test.go for new provider types
This only makes it compile. It does not yet ensure that the tests pass.
2018-10-16 19:14:11 -07:00
Martin Atkins b195c712a3 command: Update plan_test.go for new provider types
This only makes it compile. It does not yet ensure that the tests pass.
2018-10-16 19:14:11 -07:00
Martin Atkins 54c1616d73 command: Update import_test.go for new provider types
This only makes it compile. It does not yet ensure that the tests pass.
2018-10-16 19:14:11 -07:00
Martin Atkins 65341b0973 command: Update apply_test.go for new provider types
This only makes it compile. It does not yet ensure that the tests pass.
2018-10-16 19:14:11 -07:00
Martin Atkins d9051be66d command: Use new provider and provisioner types in test helpers
This moves our build errors from the test helpers out into the tests
themselves. We'll update the tests in subsequent commits.
2018-10-16 19:14:11 -07:00
Martin Atkins a43b7df282 core: Handle forced-create_before_destroy during the plan walk
Previously we used a single plan action "Replace" to represent both the
destroy-before-create and the create-before-destroy variants of replacing.
However, this forces the apply graph builder to jump through a lot of
hoops to figure out which nodes need it forced on and rebuild parts of
the graph to represent that.

If we instead decide between these two cases at plan time, the actual
determination of it is more straightforward because each resource is
represented by only one node in the plan graph, and then we can ensure
we put the right nodes in the graph during DiffTransformer and thus avoid
the logic for dealing with deposed instances being spread across various
different transformers and node types.

As a nice side-effect, this also allows us to show the difference between
destroy-then-create and create-then-destroy in the rendered diff in the
CLI, although this change doesn't fully implement that yet.
2018-10-16 19:14:11 -07:00
Martin Atkins b428746cfd command: UiHook to report when it gets incorrect values
For PreApply hook purposes we only actually use the Delete, Create, and
Update actions, because other actions are handled in different ways than
a direct call to ApplyResourceChange.

However, if there's a bug in core that causes it to pass a different
action, it's better for us to mark it as being explicitly unknown in the
UI rather than simply defaulting to "Modifying...", which can thus obscure
the problem and make for a confusing result.
2018-10-16 19:14:11 -07:00
Martin Atkins a37d5268eb command/format: Render unified diff for list and tuple value changes 2018-10-16 19:14:11 -07:00
Martin Atkins 30a46ed8ed command/format: Further refinement on the heuristic JSON-based diff
We'll now show an "update" symbol prior to the argument to this synthetic
jsonencode(...) call, for consistency with how we show nested values in
other cases and to attach a verb to any "# forces replacement".

We'll also show a special form in the case where the value seems to differ
only in whitespace, so users can understand what's going on in that
hopefully-rare situation, particularly if those whitespace-only changes
end up forcing us to replace a remote object.
2018-10-16 19:14:11 -07:00
Martin Atkins 783205948c command/format: JSON value formatting heuristic not for primitive values
Since our own syntax for primitive values is similar to that of JSON, and
since we permit automatic conversions from number and bool to string, we
must do this special JSON value diff formatting only if the value is a
JSON array or object to avoid confusing results.
2018-10-16 19:14:11 -07:00
Martin Atkins 4406b06ef8 command/format: Don't include commas after elements in map diffs
Although commas are allowed in this context, it's not idiomatic to use
them.
2018-10-16 19:14:11 -07:00
Martin Atkins ce157c7f23 command/format: show deep diffs for old/new JSON strings
Because so far we've not supported dynamically-typed complex data
structures, several providers have used strings containing JSON to stand
in for these.

In order to get a readable diff in those cases, we'll recognize situations
where old and new are both JSON and present a diff of the effective value
of the JSON, using a faux call to the jsonencode(...) function to indicate
when we've done so.

This is a bit of a "cute" heuristic, but is important at least for now
until we can migrate away from that practice of passing large JSON strings
to providers and use dynamically-typed attributes instead.
2018-10-16 19:14:11 -07:00
Martin Atkins 9706a00b3a command/format: correct alignment for multi-line values in map diff 2018-10-16 19:14:11 -07:00
Martin Atkins 04d8c17be8 command/format: Nice rendering for changes to map values 2018-10-16 19:14:11 -07:00
Martin Atkins a2f4b8b058 command/format: Show resource diff with header "comment"
This extra comment line gives us a place to show the full resource address
(since the block header line only includes type and name) and also allows
us to explain in long form the meaning of the change icon on the following
line.
2018-10-16 19:14:11 -07:00
Martin Atkins 239a54ad6f command: initial structural diff rendering
This is a light adaptation of our earlier prototype of structural diff
rendering, as a starting point for what we'll actually ship. This is not
consistent with the latest mocks, so will need some additional work before
it is ready, but integrating this allows us to at least see the plan
contents while fixing up remaining issues elsewhere.
2018-10-16 19:14:11 -07:00
Martin Atkins a7342de274 core: Properly handle no-op changes in plan
Previously we just left these out of the plan altogether, but in the new
plan types we intentionally include change information for every resource
instance, even if no changes are actually planned, to allow alternative
plan file viewers to show what isn't changing as well as what is.
2018-10-16 19:14:11 -07:00
Martin Atkins 5731703de5 command/format: show planned changes in rendered plan
This codepath is going to be significantly changed before release to make
it support structural diff of the new data types, but this lets us lean on
the old renderer to produce partial output in the mean time while we
continue to work on getting things working end-to-end after the
considerable refactoring that's been going on.
2018-10-16 19:14:11 -07:00
Martin Atkins 44bc7519a6 terraform: More wiring in of new provider types
This doesn't actually work yet, but it builds and then panics in a pretty
satisfying way.
2018-10-16 19:12:54 -07:00
Martin Atkins a3403f2766 terraform: Ugly huge change to weave in new State and Plan types
Due to how often the state and plan types are referenced throughout
Terraform, there isn't a great way to switch them out gradually. As a
consequence, this huge commit gets us from the old world to a _compilable_
new world, but still has a large number of known test failures due to
key functionality being stubbed out.

The stubs here are for anything that interacts with providers, since we
now need to do the follow-up work to similarly replace the old
terraform.ResourceProvider interface with its replacement in the new
"providers" package. That work, along with work to fix the remaining
failing tests, will follow in subsequent commits.

The aim here was to replace all references to terraform.State and its
downstream types with states.State, terraform.Plan with plans.Plan,
state.State with statemgr.State, and switch to the new implementations of
the state and plan file formats. However, due to the number of times those
types are used, this also ended up affecting numerous other parts of core
such as terraform.Hook, the backend.Backend interface, and most of the CLI
commands.

Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize
in advance to the person who inevitably just found this huge commit while
spelunking through the commit history.
2018-10-16 19:11:09 -07:00
Kristin Laemmert ce5e66e178 plugin/discovery provider installer: download providers from the registry
Terraform will query the public registry at
https://registry.terraform.io for providers, instead of
https://releases.hashicorp.com.
2018-10-16 18:56:50 -07:00
Martin Atkins 479c6b2466 move "configschema" from "config" to "configs"
The "config" package is no longer used and will be removed as part
of the 0.12 release cleanup. Since configschema is part of the
"new world" of configuration modelling, it makes more sense for
it to live as a subdirectory of the newer "configs" package.
2018-10-16 18:50:29 -07:00
Martin Atkins adb88eaa16 configupgrade: Analysis of input configuration
In order to properly migrate the contents of resource, data, provider and
provisioner blocks we will need the provider's schema in order to
understand what is expected, so we can resolve some ambiguities inherent
in the legacy HCL AST.

This includes an initial prototype of migrating the content of resource
blocks just to verify that the information is being gathered correctly.
As with the rest of the upgrade_native.go file, this will be reorganized
significantly once the basic end-to-end flow is established and we can
see how to organize this code better.
2018-10-16 18:50:29 -07:00
Martin Atkins eac8779870 command: validate must set values for root variables
Since the intent of the validate command is to check config validity
regardless of context (input variables, state, etc), we use unknown values
of the requested type here, which will then allow us to complete type
checking against the specified types of the variables without assuming
any particular values.
2018-10-16 18:50:29 -07:00
Martin Atkins ffe5f7c4e6 command: 0.12upgrade command
This is the frontend to the work-in-progress codepath for upgrading the
source code for a module written for Terraform v0.11 or earlier to use
the new syntax and idiom of v0.12.

The underlying upgrade code is not yet complete as of this commit, and
so the command is not yet very useful. We will continue to iterate on
the upgrade code in subsequent commits.
2018-10-16 18:50:29 -07:00
Martin Atkins 0742e756e5 tfdiags: Sort order for diagnostics
Because we gather together diagnostics from many different parts of the
codebase, the list often ends up being in a non-ideal order. Here we
define a partial ordering for diagnostics that should hopefully make them
easier to scan when many are present, by grouping together diagnostics
that are of the same severity and belong to the same file.

We use sort.Stable here because we have a partial order and so we need
to make sure that diagnostics that do not have a relative ordering will
remain in their original order.

This sorting is applied just in time before rendering the diagnostics
in command.Meta.showDiagnostics.
2018-10-16 18:50:29 -07:00
Martin Atkins 2ef56b3e05 Fix up some missed "go fmt"
Because of the size of some of these files, automatic format-on-save was
implicitly disabled in my editor, which I didn't notice before committing.
2018-10-16 18:49:20 -07:00
Martin Atkins dc7f793be9 command: terraform fmt to use new HCL formatter
This doesn't yet include test updates, since there are problems in core
currently blocking these tests from running. The tests will therefore be
updated in a subsequent commit.
2018-10-16 18:49:20 -07:00
Martin Atkins a270a18a4d command: Update tests for changes in "terraform" package
This is not exhaustive, but it gets the tests in this package compiling
again and gets most of them working.
2018-10-16 18:49:20 -07:00
Martin Atkins fc7871c5ee command/import: default provider config in same module
Previously we were defaulting the provider configuration selection to a
provider in the root module inferred from the resource type name.

This is close, but not quite right: we need to _start_ with a provider
configuration in the same module as we're importing into, and then our
provider resolution steps during import graph construction will use that
as a starting point for a walk up the tree to find the nearest matching
configuration (which might eventually still be in the root, but not
necessarily).
2018-10-16 18:48:28 -07:00
Martin Atkins f085af4ba6 command: update "terraform console" for HCL2
This now uses the HCL2 parser and evaluator APIs and evaluates in terms
of a new-style *lang.Scope, rather than the old terraform.Interpolator
type that is no longer functional.

The Context.Eval method used here behaves differently than the
Context.Interpolater method used previously: it performs a graph walk
to populate transient values such as input variables, local values, and
output values, and produces its scope in terms of the result of that
graph walk. Because of this, it is a lot more robust than the prior method
when asked to resolve references other than those that are persisted
in the state.
2018-10-16 18:46:46 -07:00
Martin Atkins 9ca13d5c1d command/validate: make sure diagnostics is always present and an array
Previously an empty diagnostics would appear as "null" in the JSON output,
since that is how encoding/json serializes a nil slice. It's more
convenient for users of dynamic languages to keep the type consistent
in all cases, since they can then just iterate the list without needing a
special case for when it is null.
2018-10-16 18:46:46 -07:00