Commit Graph

118 Commits

Author SHA1 Message Date
Martin Atkins 63fa1ac418 vendor: go get github.com/hashicorp/hcl2@master
This includes a small fix to ensure the parser doesn't produce an invalid
body for block parsing syntax errors, and instead produces an incomplete
result that calling applications like Terraform can still analyze.

The problem here was affecting our version-constraint-sniffing code, which
intentionally tried to find a core version constraint even if there's a
syntax error so that it can report that a new version of Terraform is a
likely cause of the syntax error. It was working in most cases, unless
it was the "terraform" block itself that contained the error, because then
we'd try to analyze a broken hcl.Block with a nil body.

This includes a new test for "terraform init" that exercises this
recovery codepath.
2019-05-14 15:37:46 -07:00
Kristin Laemmert c73a5fc07d
command/show: add provider name to resource changes (#20711) 2019-03-15 13:20:43 -07:00
Kristin Laemmert 9d0d564ec7
`terraform show` and `terraform providers schema -json` should return valid json (#20697)
* command/providers schemas: return empty json object if config parses successfully but no providers found
* command/show (state): return an empty object if state is nil
2019-03-14 14:52:07 -07:00
Kristin Laemmert c4151b7c7c
command/show: fixing bugs in modulecalls (#20513)
* command/show: fixing bugs in modulecalls

jsonconfig and jsonplan both had subtle bugs with the logic for
marshaling module calls that only showed up when multiple modules were
referenced. This PR fixes those bugs and extends the existing tests to
include multiple modules.

* sort all the things, mostly for tests
2019-03-01 13:59:12 -08:00
Kristin Laemmert 16823f43de
command/jsonprovider: export providers schemas to json (#20446)
* command/jsonprovider: a new package for exporting providers schemas as JSON
2019-02-25 13:32:47 -08:00
Kristin Laemmert 0c94e20a83
command/show enhancements and bugfixes
* command/jsonconfig: provider config marshaling enhancements

This PR fixes a bug wherein the keys in "provider_config" were the
"addrs.ProviderConfig", and therefore being overwritten for each module,
instead of the intended "addrs.AbsProviderConfig".

We realized that there was still opportunity for ambiguity, for example
if a user made a provider alias that was the same name as a module, so
we opted to use the syntax `modulename:providername(.provideralias)`

* command/json*: fixed a bug where we were attempting to lookup schemas
with the provider name, instead of provider type.
2019-02-20 14:27:49 -08:00
Kristin Laemmert b14472f22c
command/jsonconfig: add missing fields from configuration output (#20387)
Display depends_on for resources and outputs, and description for
outputs.
2019-02-19 16:31:10 -08:00
Kristin Laemmert c59a274e96
command/jsonplan: "deposed" should be a string, not a bool. (#20351) 2019-02-19 13:55:49 -08:00
Kristin Laemmert 818b4ec068
command/show: add "module_version" to "module_calls" in config (#20367)
* command/show: add "module_version" to "module_calls" in config portion
of `terraform show`.

Also extended the `terraform show -json` test to run `init` so we could
add examples with modules. This does _not_ test the "module_version"
yet, but it _did_ help expose a bug in jsonplan where modules were
duplicated. This is also fixed in this PR.

* command/jsonconfig: rename version to version_constraint and
resolved_source to source.
2019-02-19 08:12:33 -08:00
Kristin Laemmert f783ed0d45
command/jsonconfig: display module variables in config output (#20311)
* command/jsonconfig: display module variables in config output

The tests have been updated to reflect this change.

* command/jsonconfig: properly handle variables with nil defaults
2019-02-12 12:03:07 -08:00
Kristin Laemmert f2f35265bc
command/show: json output enhancements (#20291)
* command/jsonplan: 
- add variables to plan output
- print known planned values for resources

Previously, resource attribute values were only displayed if the values
were wholly known. Now we will filter the unknown values out of the
change and print the known values.

* command/jsonstate: added depends_on and tainted fields
* command/show: update tests to reflect added fields
2019-02-11 13:17:03 -08:00
Martin Atkins 7e186f72d9 command: Update "terraform show -json" tests for changed provider contract
We now require a provider to populate all of its defaults -- including
unknown value placeholders -- during PlanResourceChange. That means the
mock provider for testing "terraform show -json" must now manage the
population of the computed "id" attribute during plan.

To make this logic a little easier, we also change the ApplyResourceChange
implementation to fill in a non-null id, since that makes it easier for
the mock PlanResourceChange to recognize when it needs to populate that
default value during an update.
2019-02-08 11:58:21 -08:00
Kristin Laemmert 6e057c529e
command/jsonplan: sort resources by address (#20113)
* command/jsonplan: sort resources by address
* command/show: extend test case to include resources with count
* command/json*: document resource ordering as consistent but undefined
2019-01-25 09:17:40 -08:00
Kristin Laemmert f00fcb90bf
mildwonkey/b-show-state (#20032)
* command/show: properly marshal attribute values to json

marshalAttributeValues in jsonstate and jsonplan packages was returning
a cty.Value, which json/encoding could not marshal. These functions now
convert those cty.Values into json.RawMessages.

* command/jsonplan: planned values should include resources that are not changing
* command/jsonplan: return a filtered list of proposed 'after' attributes

Previously, proposed 'after' attributes were not being shown if the
attributes were not WhollyKnown. jsonplan now iterates through all the
`after` attributes, omitting those which are not wholly known.

The same was roughly true for after_unknown, and that structure is now
correctly populated. In the future we may choose to filter the
after_unknown structure to _only_ display unknown attributes, instead of
all attributes.

* command/jsonconfig: use a unique key for providers so that aliased
providers don't get munged together

This now uses the same "provider" key from configs.Module, e.g.
`providername.provideralias`.

* command/jsonplan: unknownAsBool needs to iterate through objects that are not wholly known

* command/jsonplan: properly display actions as strings according to the RFC,
instead of a plans.Action string.

For example:
a plans.Action string DeleteThenCreate should be displayed as ["delete",
"create"]

Tests have been updated to reflect this.

* command/jsonplan: return "null" for unknown list items.

The length of a list could be meaningful on its own, so we will turn
unknowns into "null". The same is less likely true for maps and objects,
so we will continue to omit unknown values from those.
2019-01-23 11:46:53 -08:00
Martin Atkins 86c02d5c35 command: "terraform init" can partially initialize for 0.12upgrade
There are a few constructs from 0.11 and prior that cause 0.12 parsing to
fail altogether, which previously created a chicken/egg problem because
we need to install the providers in order to run "terraform 0.12upgrade"
and thus fix the problem.

This changes "terraform init" to use the new "early configuration" loader
for module and provider installation. This is built on the more permissive
parser in the terraform-config-inspect package, and so it allows us to
read out the top-level blocks from the configuration while accepting
legacy HCL syntax.

In the long run this will let us do version compatibility detection before
attempting a "real" config load, giving us better error messages for any
future syntax additions, but in the short term the key thing is that it
allows us to install the dependencies even if the configuration isn't
fully valid.

Because backend init still requires full configuration, this introduces a
new mode of terraform init where it detects heuristically if it seems like
we need to do a configuration upgrade and does a partial init if so,
before finally directing the user to run "terraform 0.12upgrade" before
running any other commands.

The heuristic here is based on two assumptions:
- If the "early" loader finds no errors but the normal loader does, the
  configuration is likely to be valid for Terraform 0.11 but not 0.12.
- If there's already a version constraint in the configuration that
  excludes Terraform versions prior to v0.12 then the configuration is
  probably _already_ upgraded and so it's just a normal syntax error,
  even if the early loader didn't detect it.

Once the upgrade process is removed in 0.13.0 (users will be required to
go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can
be simplified to remove that special mode, but the idea of doing the
dependency version checks against the liberal parser will remain valuable
to increase our chances of reporting version-based incompatibilities
rather than syntax errors as we add new features in future.
2019-01-14 11:33:21 -08:00
Kristin Laemmert 5df9cd0f52
command/show: tests for -json output (#19980)
* command/show: added test scaffold for json output

More test cases will be added once the basic shape of the tests is
validated.

- command/json* packages now sort resources by address, matching
behavior elsewhere
- using cmp in tests instead of reflect.DeepEqual for the diffs
- updating expected output in tests to match sorting
2019-01-11 15:13:55 -08:00
Kristin Laemmert 126e5f337f
json output of terraform plan (#19687)
* command/show: adding functions to aid refactoring

The planfile -> statefile -> state logic path was getting hard to follow
with blurry human eyes. The getPlan... and getState... functions were
added to help streamline the logic flow. Continued refactoring may follow.

* command/show: use ctx.Config() instead of a config snapshot

As originally written, the jsonconfig marshaller was getting an error
when loading configs that included one or more modules. It's not clear
if that was an error in the function call or in the configloader itself,
  but as a simpler solution existed I did not dig too far.

* command/jsonplan: implement jsonplan.Marshal

Split the `config` portion into a discrete package to aid in naming
sanity (so we could have for example jsonconfig.Resource instead of
jsonplan.ConfigResource) and to enable marshaling the config on it's
own.
2018-12-19 11:08:25 -08:00
Martin Atkins f39a5d0962 command: Fix various TestMetaBackend-prefix tests
Our new state model has a different implementation of "empty" that doesn't
consider lineage/serial, so we need to have some actual content in these
state fixtures to avoid them being skipped during state migrations.
2018-11-19 09:02:35 -08:00
Martin Atkins 22c84c71a4 command: Use statemgr.Import and statemgr.Export for state push and pull
We previously hacked around the import/export functionality being missing
in the statemgr layer after refactoring, but now it's been reintroduced
to fix functionality elsewhere we should use the centralized Import and
Export functions to ensure consistent behavior.

In particular, this pushes the logic for checking lineage and serial
during push down into the state manager itself, which is better because
all other details about lineage and serial are managed within the state
managers.
2018-11-19 09:02:35 -08:00
Martin Atkins aacbe1d14b command: Fix TestMetaBackend_configureNewWithState
This test was initially failing because its fixture had a state which our
new state models consider to be "empty", and thus it was not migrated.

After fixing that (by adding an output to the fixture), this revealed a
bug that the lineage was not being persisted through the migration. This
is fixed by using the statemgr.Migrate method instead of writing via the
normal Writer interface, which allows two cooperating state managers to
properly transfer the lineage and serial along with the state snapshot.
2018-11-19 09:02:35 -08:00
Martin Atkins c3d11b762b command: Fix testBackendState
The hashing function for cached backend configuration is different now, so
our hard-coded hash of the HTTP backend address wasn't working anymore.

Here we update the hash so that tests using this test backend will work
again. Rather than leaving it hard-coded, we'll instead compute it the
same way as "terraform init" would.

In practice only one test is actually using this function right now, so
we also update the test fixture for that test (TestPlan_outBackend) to
match the new expectations, though as of this commit it's still failing
with an unrelated error.
2018-11-08 08:57:11 -08:00
Sander van Harmelen 52a1b22f7a Implement the remote enhanced backend
This is a refactored version of the `remote` backend that was initially added to Terraform v0.11.8 which should now be compatible with v0.12.0.
2018-11-06 16:29:46 +01:00
Sander van Harmelen 5e11de460a
Merge pull request #19130 from hashicorp/f-state-push-pull
command/state: update and fix the state push and pull
2018-10-19 19:16:00 +02:00
Sander van Harmelen 7fbd93b5cd command/state: update and fix the state push and pull 2018-10-19 19:12:23 +02:00
Sander van Harmelen af1a471a05 command/state: update and fix the state list command 2018-10-19 16:31:12 +02:00
Martin Atkins 8b466811d2 command: Fix a few tests from meta_backend_test.go 2018-10-16 19:14:11 -07:00
Martin Atkins 98bbd560b5 command: Fix most (but not all) "terraform plan" tests 2018-10-16 19:14:11 -07:00
Martin Atkins ca314afc0d command: Make suitable provider schemas available for all "plan" tests 2018-10-16 19:14:11 -07:00
Martin Atkins fc2614c939 command: Fix some tests for the "apply" command 2018-10-16 19:14:11 -07:00
Martin Atkins 34c4fd316d command: Update backend-unchanged fixture for new expectations
Our serialization of the backend configuration has changed slightly for
Terraform 0.12 due to reimplementing it in terms of the HCL2 types, so
the base case that should be unchanged during the test needs to be
changed.
2018-10-16 19:14:11 -07:00
Martin Atkins 65341b0973 command: Update apply_test.go for new provider types
This only makes it compile. It does not yet ensure that the tests pass.
2018-10-16 19:14:11 -07:00
Martin Atkins bfd9392eb8 command: beginnings of new config loader in "terraform validate"
As part of some light reorganization of our commands, this new
implementation no longer does validation of variables and will thus avoid
the need to spin up a fully-valid context. Instead, its focus is on
validating the configuration itself, regardless of any variables, state,
etc.

This change anticipates us later adding a -validate-only flag to
"terraform plan" which will then take over the related use-case of
checking if a particular execution of Terraform is valid, _including_ the
state, variables, etc.

Although leaving variables out of validate feels pretty arbitrary today
while all of the variable sources are local anyway, we have plans to
allow per-workspace variables to be stored in the backend in future and
at that point it will no longer be possible to fully validate variables
without accessing the backend. The "terraform plan" command explicitly
requires access to the backend, while "terraform validate" is now
explicitly for local-only validation of a single module.

In a future commit this will be extended to do basic type checking of
the configuration based on provider schemas, etc.
2018-10-16 18:20:32 -07:00
Sander van Harmelen f410a5bb26 backend/migrations: migrate the default state
Certain backends (currently only the `remote` backend) do not support using both the default and named workspaces at the same time.

To make the migration easier for users that currently use both types of workspaces, this commit adds logic to ask the user for a new workspace name during the migration process.
2018-08-29 21:37:39 +02:00
Sander van Harmelen 7fb2d1b8de Implement the Enterprise enhanced remote backend 2018-08-03 22:22:55 +02:00
James Bardin 5a975d9997 add test for internal plugins with -plugin-dir
-plugin-dir was short-circuiting the discovery for internal plugins
2018-01-05 11:51:09 -05:00
Martin Atkins f1079257ac command: show a special error message when importing in an empty dir
If users run "terraform import" in a directory with no Terraform
configuration files, it's likely that they've made a mistake either by
being in the wrong directory or forgetting to use the -config option
on the command line.

To help users find their mistake in this case, we'll now produce a
specialized error message for this situation:

    Error: No Terraform configuration files

    The directory /home/user/example does not contain any Terraform
    configuration files (.tf or .tf.json). To specify a different
    configuration directory, use the -config="..." command line option.

While here, this also converts some of the other existing messages to
diagnostics so that we can show any configuration warnings along with
the error message, and move towards the new standard error presentation.
2017-12-11 16:08:33 -08:00
Martin Atkins d4ee58ce59 Re-integrate the "terraform" provider into the main binary
As part of the 0.10 core/provider split we moved this provider, along with
all the others, out into its own repository.

In retrospect, the "terraform" provider doesn't really make sense to be
separated since it's just a thin wrapper around some core code anyway,
and so re-integrating it into core avoids the confusion that results when
Terraform Core and the terraform provider have inconsistent versions of
the backend code and dependencies.

There is no good reason to use a different version of the backend code
in the provider than in core, so this new "internal provider" mechanism
is stricter than the old one: it's not possible to use an external build
of this provider at all, and version constraints for it are rejected as
a result.

This provider is also run in-process rather than in a child process, since
again it's just a very thin wrapper around code that's already running
in Terraform core anyway, and so the process barrier between the two does
not create enough advantage to warrant the additional complexity.
2017-11-03 11:36:31 -07:00
Martin Atkins 86343fa895 Remove test output file accidentally added in 52c97e9f
Also, fix the missing gitignore rule that caused this to be added in the
first place.
2017-10-03 17:32:26 -07:00
Bruno Miguel Custodio 52c97e9fc9
First steps towards an 'etcdv3' backend. 2017-09-08 09:43:33 +01:00
Sunny 3a1582c1b9 command/validate: read terraform.tfvars file for variable values
This is now consistent with the handling of this file for other commands.
2017-08-28 12:01:11 -07:00
Sunny 2d849f8650 command/init: check required_version
Previously we were checking required_version only during "real" operations, and not during initialization. Catching it during init is better because that's the first command users run on a new working directory.
2017-08-28 11:25:16 -07:00
James Bardin ea3e87b584 Merge pull request #15768 from hashicorp/jbardin/remote-import
Don't ForceLocal for the import backend
2017-08-11 09:12:46 -04:00
James Bardin 5bcc1bae59 Merge pull request #15769 from hashicorp/jbardin/discovery-paths
load legacy plugin paths
2017-08-09 17:54:59 -04:00
James Bardin fa20d43d80 test loading of Meta.PluginOverrides
These are currently being skipped in discovery
2017-08-09 11:13:54 -04:00
James Bardin e3748901b4 Don't ForceLocal for the import backend
While the `local.Local` backend is the only implementation of
`backend.Local`, creating the backend with `ForceLocal` bypasses the
`backend.Backend` in the `local.Local` causing a local state to be
implicitly created rather than using the configured state backend.

Add a test that imports into a configured backend (using the "local"
backend as a remote state proxy). This further confirms the confusing
nature of ForceLocal, as the backend _is_ local, but not from the
viewpoint of meta.Backend.
2017-08-09 10:24:32 -04:00
James Bardin 16e8e405c7 create failing test cases for remote lineage issue
Some remote backend would fail on `-state push -force`, or `workspace
new -state` because of a new strict lineage check in remote.State.
2017-08-01 19:34:21 -04:00
Radek Simko 14614a5423 command/validate: Add flag to check that all variables are specified (#13872)
* command/validate: Add flag to check that all variables are specified

* Rename config-only to check-variables
2017-07-05 17:32:29 +01:00
Martin Atkins 032f71f1ff command: produce provider lock file during "terraform init"
Once we've installed the necessary plugins, we'll do one more walk of
the available plugins and record the SHA256 hashes of all of the plugins
we select in the provider lock file.

The file we write here gets read when we're building ContextOpts to
initialize the main terraform context, so any command that works with
the context will then fail if any of the provider binaries change.
2017-06-09 14:03:59 -07:00
Martin Atkins 9a398a7793 command: require resource to be in config before import
Previously we encouraged users to import a resource and _then_ write the
configuration block for it. This ordering creates lots of risk, since
for various reasons users can end up subsequently running Terraform
without any configuration in place, which then causes Terraform to want
to destroy the resource that was imported.

Now we invert this and require a minimal configuration block be written
first. This helps ensure that the user ends up with a correlated resource
config and state, protecting against any inconsistency caused by typos.

This addresses #11835.
2017-06-09 14:03:59 -07:00
Martin Atkins 7d8719150c command: validate import resource address early
Previously we deferred validation of the resource address on the import
command until we were in the core guts, which caused the error responses
to be rather unhelpful.

By validating these things early we can give better feedback to the user.
2017-06-09 14:03:59 -07:00