Commit Graph

1799 Commits

Author SHA1 Message Date
Alisdair McDiarmid 7c0ec0107f backend: Replace ShowDiagnostics with view.Diagnostics 2021-02-25 11:26:05 -05:00
Alisdair McDiarmid 3737ee5081 views: Remove outdated comment 2021-02-25 09:34:32 -05:00
Alisdair McDiarmid 4271b0e11d cli: Remove -var/-var-file flags from validate
These flags are not used and have been deprecated since 0.12.11.
2021-02-24 12:29:56 -05:00
Alisdair McDiarmid 7a03c3153e
Merge pull request #27861 from hashicorp/alisdair/help-global-options
cli: Add reference to global options to help text
2021-02-23 14:19:08 -05:00
James Bardin c103242bef
Merge pull request #27885 from hashicorp/jbardin/show-json
jsonstate: indicate schema version mismatch during encoding
2021-02-23 12:57:06 -05:00
James Bardin 3449a8aa35 don't marshal state with the wrong schema
Instead of returning an error with no context about unexpected
attributes or incorrect types, notify users that the schema stored in
the state does not match the current provider.

User can only encounter this error if the providers have updated their
schemas since the state was stored. This would appears when running
`terraform show -json` to display the current state, or
`terraform show -json planfile` if that plan was created with
`-refresh=false`. In either case, the state must be refreshed in order
to properly json encoded.
2021-02-23 10:19:24 -05:00
Alisdair McDiarmid b7f54b30d5 cli: Fix overly broad auto-approve argument
The auto-approve argument was part of the arguments.Operation type,
which resulted in adding a silent -auto-approve flag to plan and
refresh. This was unintended, and is fixed in this commit by moving the
flag to the arguments.Apply type and updating the downstream callers.
2021-02-23 10:09:30 -05:00
Alisdair McDiarmid 8d9a08e3a1 cli: Migrate plan to command views 2021-02-23 10:09:25 -05:00
Alisdair McDiarmid eeb5dfbf76
Merge pull request #27865 from hashicorp/alisdair/command-views-refresh
cli: Migrate refresh to command views
2021-02-23 09:27:33 -05:00
Martin Atkins 7f78342953 command: Experimental "terraform test" command
This is just a prototype to gather some feedback in our ongoing research
on integration testing of Terraform modules. The hope is that by having a
command integrated into Terraform itself it'll be easier for interested
module authors to give it a try, and also easier for us to iterate quickly
based on feedback without having to coordinate across multiple codebases.

Everything about this is subject to change even in future patch releases.
Since it's a CLI command rather than a configuration language feature it's
not using the language experiments mechanism, but generates a warning
similar to the one language experiments generate in order to be clear that
backward compatibility is not guaranteed.
2021-02-22 14:21:45 -08:00
Alisdair McDiarmid 50d58b592a cli: Migrate refresh to command views 2021-02-22 11:58:52 -05:00
Alisdair McDiarmid 56b756cfd9
Merge pull request #27841 from hashicorp/alisdair/command-views-apply
cli: Migrate apply to command views
2021-02-22 11:53:57 -05:00
Alisdair McDiarmid 6bef157f8e
Merge pull request #27860 from hashicorp/alisdair/remove-noop-plan-backend
cli: Remove no-op plan backend setup
2021-02-22 11:48:58 -05:00
Alisdair McDiarmid 43028c4912 cli: Migrate apply to command views 2021-02-22 11:47:40 -05:00
Kristin Laemmert 7b7273e3aa
Add support for plugin protocol v6 (#27826)
* Add support for plugin protocol v6

This PR turns on support for plugin protocol v6. A provider can
advertise itself as supporting protocol version 6 and terraform will
use the correct client.

Todo:

The "unmanaged" providers functionality does not support protocol
version, so at the moment terraform will continue to assume that
"unmanaged" providers are on protocol v5. This will require some
upstream work on go-plugin (I believe).

I would like to convert the builtin providers to use protocol v6 in a
future PR; however it is not necessary until we remove protocol v6.

* add e2e test for using both plugin protocol versions

- copied grpcwrap and made a version that returns protocol v6 provider
- copied the test provider, provider-simple, and made a version that's
  using protocol v6 with the above fun
- added an e2etest
2021-02-22 10:22:45 -05:00
Alisdair McDiarmid 94340b5940 cli: Add reference to global options to help text 2021-02-22 09:25:56 -05:00
Alisdair McDiarmid b976178018 cli: Remove no-op plan backend setup
This code does not appear to have any effect. The operation request has
its PlanOutBackend field populated directly in the Meta.Operation
method, by calling m.backendForState.ForPlan().

I tested a trivial null-resource config with a Consul backend, and the
saved plans with and without this code present were identical.
2021-02-22 07:43:18 -05:00
Pam Selle 8f7807684a Upgrade to quoted keywords to error
The warning about deprecation is upgraded to an error
2021-02-21 20:27:20 -05:00
Pam Selle fa7c3d7e10 Remove interpolation-only warning
These interpolations are removed when upgrading using 0.12upgrade,
and are removed in terraform fmt in many cases
2021-02-19 10:59:09 -05:00
James Bardin 3da5d2bdf6
Merge pull request #27692 from alexwlchan/abolish-the-heirarchy
Correct the spelling of heirarchy/hierarchy throughout
2021-02-18 16:34:28 -05:00
Pam Selle 156b1615a2 Remove 0.13upgrade test fixtures 2021-02-18 12:57:58 -05:00
Alisdair McDiarmid 68558ccd54 backend/local: Replace CLI with view instance
This commit extracts the remaining UI logic from the local backend,
and removes access to the direct CLI output. This is replaced with an
instance of a `views.Operation` interface, which codifies the current
requirements for the local backend to interact with the user.

The exception to this at present is interactivity: approving a plan
still depends on the `UIIn` field for the backend. This is out of scope
for this commit and can be revisited separately, at which time the
`UIOut` field can also be removed.

Changes in support of this:

- Some instances of direct error output have been replaced with
  diagnostics, most notably in the emergency state backup handler. This
  requires reformatting the error messages to allow the diagnostic
  renderer to line-wrap them;
- The "in-automation" logic has moved out of the backend and into the
  view implementation;
- The plan, apply, refresh, and import commands instantiate a view and
  set it on the `backend.Operation` struct, as these are the only code
  paths which call the `local.Operation()` method that requires it;
- The show command requires the plan rendering code which is now in the
  views package, so there is a stub implementation of a `views.Show`
  interface there.

Other refactoring work in support of migrating these commands to the
common views code structure will come in follow-up PRs, at which point
we will be able to remove the UI instances from the unit tests for those
commands.
2021-02-18 12:08:08 -05:00
Kristin Laemmert f6505870cc
Mildwonkey/providers interface renaming (#27805)
* providers.Interface: huge renamification

This commit renames a handful of functions in the providers.Interface to
match changes made in protocol v6. The following commit implements this
change across the rest of the codebase; I put this in a separate commit
for ease of reviewing and will squash these together when merging.

One noteworthy detail: protocol v6 removes the config from the
ValidateProviderConfigResponse, since it's never been used. I chose to
leave that in place in the interface until we deprecate support for
protocol v5 entirely.

Note that none of these changes impact current providers using protocol
v5; the protocol is unchanged. Only the translation layer between the
proto and terraform have changed.
2021-02-18 10:13:43 -05:00
Kristin Laemmert 2b4e389839
Mildwonkey/nested type format (#27793)
* command/format: check for sensitive NestedTypes

Eventually, the diff formatter will need to be updated to properly
handle NestedTypes, but for now we can let the existing function deal
with them as regular cty.Object-type attributes.

To avoid printing sensitive nested attributes, we will treat any
attribute with at least one sensitive nested attribute as an entirely
sensitive attribute.

* bugfix for Object ImpliedType()

ImpliedType() was returning too early when the given object had optional
attributes, therefore skipping the incredibly important step of
accounting for the nesting mode when returning said type.
2021-02-18 08:48:52 -05:00
Pam Selle 48bfde2e28
Merge pull request #27798 from hashicorp/pselle/upgrade_command_removal
Remove removed `upgrade` commands
2021-02-17 14:42:33 -05:00
Pam Selle 08b649b6f9 Remove removed upgrade commands
Delete commands that are no longer in Terraform and have been
removed for a full release cycle.
2021-02-17 14:34:58 -05:00
Pam Selle b582b19359 Remove duplicate declaration 2021-02-16 18:08:36 -05:00
Alisdair McDiarmid 6375c6ce6b
Merge pull request #27787 from hashicorp/alisdair/command-views-state-locker
clistate: Update clistate.Locker for command views
2021-02-16 17:37:18 -05:00
Alisdair McDiarmid 0aa227e0e3 cli: Fix misapplied terminal colors on run logs
Due to calling the Colorize function with the full string instead of the
format string, plan/apply logs which include resource instance keys or
IDs which happen to match color formatting would be rendered
incorrectly.

This commit fixes this by only colorizing the known-safe format string.
We also add full test coverage for the UI hook, although only one of the
hooks is tested for this color bugfix due to verbosity of the test.

We also add the bold coloring to the provisioner output prefix, which
seems to have been an oversight.
2021-02-16 15:05:22 -05:00
Alisdair McDiarmid 1ae3d30383
Merge pull request #27760 from hashicorp/alisdair/command-views-ui-hook
cli: Migrate Terraform UI hook to command views
2021-02-16 09:36:47 -05:00
Alisdair McDiarmid 20cb28b25f
Merge pull request #27788 from hashicorp/alisdair/command-views-output-tests
views: Expand test coverage for views.Output
2021-02-16 09:07:18 -05:00
Alisdair McDiarmid bb3f59b016
Merge pull request #27697 from hashicorp/alisdair/json-plan-provider-required-versions
cli: Fix for missing provider requirements in JSON plan
2021-02-16 09:05:14 -05:00
Alisdair McDiarmid 96cf29111d views: Expand test coverage for views.Output
Now that the view code is separated, we can increase test coverage in
unit tests. This commit moves some tests from the command package which
were testing only view code, and adds more new test cases.
2021-02-16 08:48:31 -05:00
Alisdair McDiarmid 2d1976bbda clistate: Update clistate.Locker for command views
The clistate package includes a Locker interface which provides a simple
way for the local backend to lock and unlock state, while providing
feedback to the user if there is a delay while waiting for the lock.
Prior to this commit, the backend was responsible for initializing the
Locker, passing through direct access to the cli.Ui instance.

This structure prevented commands from implementing different
implementations of the state locker UI. In this commit, we:

- Move the responsibility of creating the appropriate Locker to the
  source of the Operation;
- Add the ability to set the context for a Locker via a WithContext
  method;
- Replace the Locker's cli.Ui and Colorize members with a StateLocker
  view;
- Implement views.StateLocker for human-readable UI;
- Update the Locker interface to return detailed diagnostics instead of
  errors, reducing its direct interactions with UI;
- Add a Timeout() method on Locker to allow the remote backend to
  continue to misuse the -lock-timeout flag to cancel pending runs.

When an Operation is created, the StateLocker field must now be
populated with an implementation of Locker. For situations where locking
is disabled, this can be a no-op locker.

This change has no significant effect on the operation of Terraform,
with the exception of slightly different formatting of errors when state
locking or unlocking fails.
2021-02-16 07:19:22 -05:00
Alisdair McDiarmid a7b7cd29fc cli: Migrate Terraform UI hook to command views
Move the code which renders Terraform hook callbacks as UI into the
views package, backed by a views.View instead of a cli.Ui. Update test
setup accordingly.

To allow commands to control this hook, we add a hooks member on the
backend Operation struct. This supersedes the hooks in the Terraform
context, which is not directly controlled by the command logic.

This commit should not change how Terraform works, and is refactoring in
preparation for more changes which move UI code out of the backend.
2021-02-16 07:18:22 -05:00
Alisdair McDiarmid fa774dafc0
Merge pull request #27761 from hashicorp/alisdair/command-views-operation-diagnostics
backend: Add per-operation diagnostic rendering
2021-02-16 07:17:31 -05:00
Alisdair McDiarmid 3cd086f46c views: Use new streams helper functions 2021-02-12 15:37:02 -05:00
Alisdair McDiarmid 536c80da23 backend: Add per-operation diagnostic rendering
The enhanced backends (local and remote) need to be able to render
diagnostics during operations. Prior to this commit, this functionality
was supported with a per-backend `ShowDiagnostics` function pointer.

In order to allow users of these backends to control how diagnostics are
rendered, this commit moves that function pointer to the `Operation`
type. This means that a diagnostic renderer is configured for each
operation, rather than once per backend initialization.

Some secondary consequences of this change:

- The `ReportResult` method on the backend is now moved to the
  `Operation` type, as it needs to access the `ShowDiagnostics` callback
  (and nothing else from the backend);
- Tests which assumed that diagnostics would be written to the backend's
  `cli.Ui` instance are migrated to using a new record/playback diags
  helper function;
- Apply, plan, and refresh commands now pass a pointer to the `Meta`
  struct's `showDiagnostics` method.

This commit should not change how Terraform works, and is refactoring in
preparation for more changes which move UI code out of the backend.
2021-02-12 14:30:35 -05:00
Alisdair McDiarmid 57879bfb71 cli: Add global view arguments parser
Rather than modifying and relying on the existing Meta.process
argument extractor, we can more clearly handle global CLI flags using
a separate parser step. This allows us to explicitly configure the view
in the command.
2021-02-12 09:56:02 -05:00
Alisdair McDiarmid c5a6aa31d3 cli: Add initial command views abstraction
Terraform supports multiple output formats for several sub-commands.
The default format is user-readable text, but many sub-commands support
a `-json` flag to output a machine-readable format for the result. The
output command also supports a `-raw` flag for a simpler, scripting-
focused machine readable format.

This commit adds a "views" abstraction, intended to help ensure
consistency between the various output formats. This extracts the render
specific code from the command package, and moves it into a views
package. Each command is expected to create an interface for its view,
and one or more implementations of that interface.

By doing so, we separate the concerns of generating the sub-command
result from rendering the result in the specified output format. This
should make it easier to ensure that all output formats will be updated
together when changes occur in the result-generating phase.

There are some other consequences of this restructuring:

- Views now directly access the terminal streams, rather than the
  now-redundant cli.Ui instance;
- With the reorganization of commands, parsing CLI arguments is now the
  responsibility of a separate "arguments" package.

For now, views are added only for the output sub-command, as an example.
Because this command uses code which is shared with the apply and
refresh commands, those are also partially updated.
2021-02-11 15:06:39 -05:00
Alisdair McDiarmid 943df48491 cli: Optimize for large multi-line string outputs 2021-02-11 10:42:06 -05:00
Alisdair McDiarmid 4991cc4835 cli: Improve error for invalid -target flags
Errors encountered when parsing flags for apply, plan, and refresh were
being suppressed. This resulted in a generic usage error when using an
invalid `-target` flag.

This commit makes several changes to address this. First, these commands
now output the flag parse error before exiting, leaving at least some
hint about the error. You can verify this manually with something like:

    terraform apply -invalid-flag

We also change how target attributes are parsed, moving the
responsibility from the flags instance to the command. This allows us to
customize the diagnostic output to be more user friendly. The
diagnostics now look like:

```shellsession
$ terraform apply -no-color -target=foo

Error: Invalid target "foo"

Resource specification must include a resource type and name.
```

Finally, we add test coverage for both parsing of target flags, and at
the command level for successful use of resource targeting. These tests
focus on the UI output (via the change summary and refresh logs), as the
functionality of targeting is covered by the context tests in the
terraform package.
2021-02-08 13:48:04 -05:00
Alisdair McDiarmid 7cae76383a cli: Fix for provider requirements in JSON plan
The JSON plan output format includes a serialized, simplified version of
the configuration. One component of this config is a map of provider
configurations, which includes version constraints.

Until now, only version constraints specified in the provider config
blocks were exposed in the JSON plan output. This is a deprecated method
of specifying provider versions, and the recommended use of a
required_providers block resulted in the version constraints being
omitted.

This commit fixes this with two changes:

- When processing the provider configurations from a module, output the
  fully-merged version constraints for the entire module, instead of any
  constraints set in the provider configuration block itself;
- After all provider configurations are processed, iterate over the
  required_providers entries to ensure that any configuration-less
  providers are output to the JSON plan too.

No changes are necessary to the structure of the JSON plan output, so
this is effectively a semantic level bug fix.
2021-02-05 14:01:58 -05:00
Alex Chan 4ec80fa18d Correct the spelling of heirarchy/hierarchy throughout 2021-02-05 15:07:04 +00:00
Alisdair McDiarmid d7613a0aac cli: Better diagnostics for apply positional args
The previous changes removing support for using the trailing positional
argument as a working directory missed a spot in the apply/destroy
command implementation. We still support this argument for applying a
saved plan:

    terraform apply foo.tfplan

However, if you pass a positional path which doesn't "look like" a plan
(for example, the path to a configuration directory), Terraform would
silently ignore it and continue.

This commit fixes that by adding an error message if the user specifies
a path which the plan loader rejects as not "looking like" a plan. This
message includes a reference to the `-chdir` flag as a pointer about
what to do next.

We also rearrange the error message when calling `terraform destroy`
with a plan file argument, and add test coverage for the above. While
we're here, update the destroy tests to copy the fixture directory,
chdir, and defer cleanup.
2021-02-04 10:27:18 -05:00
Alisdair McDiarmid b2ba650c21 cli: Remove deprecated destroy -force flag
This dramatically simplifies the logic around auto-approve, which is
nice.

Also add test coverage for the manual approve step, for both apply and
destroy, answering both yes and no.
2021-02-03 15:05:05 -05:00
James Bardin 1e5a8e4dae fix concurrent test relying on sleep
Make an old concurrent test deterministic, and not rely on sleep for any
synchronization.
2021-02-03 09:20:08 -05:00
Alisdair McDiarmid 888f36aebb cli: Remove positional plan argument from graph
To make the command arguments easier to understand and extend, we are
moving away from positional arguments. This commit changes the graph
command to take a `-plan` flag instead of an optional trailing path.
2021-02-02 13:21:26 -05:00
Alisdair McDiarmid ca23a096d8 cli: Remove legacy positional path arguments
Several commands continued to support the legacy positional path
argument to specify a working directory. This functionality has been
replaced with the global -chdir flag, which is specified before any
other arguments, including the sub-command name.

This commit removes support for the trailing path parameter from
most commands. The only command which still supports a path argument is
fmt, which also supports "-" to indicate receiving configuration from
standard input.

Any invocation of a command with an invalid trailing path parameter will
result in a short error message, pointing at the -chdir alternative.

There are many test updates in this commit, almost all of which are
migrations from using positional arguments to specify a working
directory. Because of the layer at which these tests run, we are unable
to use the -chdir argument, so the churn in test files is larger than
ideal. Sorry!
2021-02-02 13:21:26 -05:00
Brian Flad 70aac9c551
command: Use different warning messaging in init command and move above potentially confusing output (#27514)
Reference: https://github.com/hashicorp/terraform/issues/27459
Reference: https://github.com/hashicorp/terraform/pull/27507

This follows the change in the plan command to move the development overrides warning above other output from the command that may exhibit unexpected behavior.
2021-02-01 10:50:08 -05:00