Commit Graph

341 Commits

Author SHA1 Message Date
Martin Atkins 44bc7519a6 terraform: More wiring in of new provider types
This doesn't actually work yet, but it builds and then panics in a pretty
satisfying way.
2018-10-16 19:12:54 -07:00
Martin Atkins a3403f2766 terraform: Ugly huge change to weave in new State and Plan types
Due to how often the state and plan types are referenced throughout
Terraform, there isn't a great way to switch them out gradually. As a
consequence, this huge commit gets us from the old world to a _compilable_
new world, but still has a large number of known test failures due to
key functionality being stubbed out.

The stubs here are for anything that interacts with providers, since we
now need to do the follow-up work to similarly replace the old
terraform.ResourceProvider interface with its replacement in the new
"providers" package. That work, along with work to fix the remaining
failing tests, will follow in subsequent commits.

The aim here was to replace all references to terraform.State and its
downstream types with states.State, terraform.Plan with plans.Plan,
state.State with statemgr.State, and switch to the new implementations of
the state and plan file formats. However, due to the number of times those
types are used, this also ended up affecting numerous other parts of core
such as terraform.Hook, the backend.Backend interface, and most of the CLI
commands.

Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize
in advance to the person who inevitably just found this huge commit while
spelunking through the commit history.
2018-10-16 19:11:09 -07:00
Martin Atkins 479c6b2466 move "configschema" from "config" to "configs"
The "config" package is no longer used and will be removed as part
of the 0.12 release cleanup. Since configschema is part of the
"new world" of configuration modelling, it makes more sense for
it to live as a subdirectory of the newer "configs" package.
2018-10-16 18:50:29 -07:00
Martin Atkins ebc6238bee backend/local: Update tests for changes in "terraform" package
This mainly consists of adding schema information to the mock providers.
2018-10-16 18:49:20 -07:00
Martin Atkins c937c06a03 terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.

The three main goals here are:
- Use the configuration models from the "configs" package instead of the
  older models in the "config" package, which is now deprecated and
  preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
  new "lang" package, instead of the Interpolator type and related
  functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
  rather than hand-constructed strings. This is not critical to support
  the above, but was a big help during the implementation of these other
  points since it made it much more explicit what kind of address is
  expected in each context.

Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.

I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-10-16 18:46:46 -07:00
Martin Atkins ebafa51723 command: Various updates for the new backend package API
This is a rather-messy, complex change to get the "command" package
building again against the new backend API that was updated for
the new configuration loader.

A lot of this is mechanical rewriting to the new API, but
meta_config.go and meta_backend.go in particular saw some major
changes to interface with the new loader APIs and to deal with
the change in order of steps in the backend API.
2018-10-16 18:44:26 -07:00
Martin Atkins 5782357c28 backend: Update interface and implementations for new config loader
The new config loader requires some steps to happen in a different
order, particularly in regard to knowing the schema in order to
decode the configuration.

Here we lean directly on the configschema package, rather than
on helper/schema.Backend as before, because it's generally
sufficient for our needs here and this prepares us for the
helper/schema package later moving out into its own repository
to seed a "plugin SDK".
2018-10-16 18:39:12 -07:00
James Bardin a79d620f03 remove legacy remote state code 2018-10-16 18:24:10 -07:00
James Bardin 979faa5dbe move artifactory remote state to backend 2018-10-16 18:23:14 -07:00
James Bardin 18ef072325 move legacy http remote state to a backend 2018-10-16 18:22:37 -07:00
James Bardin fe527ec9d7 move legacy etcd remote state to a backend 2018-10-16 18:21:51 -07:00
Martin Atkins bd10b84a8e command/format: include source snippets in diagnostics
If we get a diagnostic message that references a source range, and if the
source code for the referenced file is available, we'll show a snippet of
the source code with the source range highlighted.

At the moment we have no cache of source code, so in practice this
codepath can never be visited. Callers to format.Diagnostic will be
gradually updated in subsequent commits.
2018-10-16 18:20:32 -07:00
Sander van Harmelen 8875fa660f Make sure we also output policies while planning 2018-10-16 17:16:28 +02:00
Sander van Harmelen 775f8a9626 Make sure we always set a custom header
This is for TFE to recognize were the calls come from.
2018-10-15 20:33:42 +02:00
Sander van Harmelen 0a59e54933 Improve the output just a bit 2018-10-15 19:56:59 +02:00
Sander van Harmelen c08cd597c5 Omit any empty lines containing STX/ETX markers 2018-10-11 22:08:09 +02:00
Sander van Harmelen fb0af07696 Print status updates while waiting for the run to start 2018-10-11 13:41:48 +02:00
Sander van Harmelen d78470ad5a Don’t ask questions when -auto-approve is set
We previously asked to override a soft-failed policy, even wehn -auto-approve was set. That is now fixed by returning a policy failed error.
2018-10-09 20:12:33 +02:00
Sander van Harmelen 53a8aaaf85
Merge pull request #19022 from hashicorp/f-auto-apply
backend/remote: properly handle workspaces that auto apply changes
2018-10-09 09:43:25 +02:00
Sander van Harmelen 194863db4e Properly handle workspaces that auto apply changes
This commit adds logic to check if a workspace is configured to auto apply changes. And if it does, we make sure we output all steps correctly.
2018-10-08 12:31:21 +02:00
Sander van Harmelen b1fdbd7db8 Allow enhanced backends to pass custom exit codes
In some cases this is needed to keep the UX clean and to make sure any remote exit codes are passed through to the local process.

The most obvious example for this is when using the "remote" backend. This backend runs Terraform remotely and stream the output back to the local terminal.

When an error occurs during the remote execution, all the needed error information will already be in the streamed output. So if we then return an error ourselves, users will get the same errors twice.

By allowing the backend to specify the correct exit code, the UX remains the same while preserving the correct exit codes.
2018-10-05 20:44:12 +02:00
Sander van Harmelen 67db9da000 Add checks for all flags we currently don’t support
For Plan only:
-module-depth=n

For Plan & Apply
-parallelism=m
-refresh=false
-var “foo=bar” and -var-file=foo
2018-10-05 20:16:34 +02:00
Sander van Harmelen ffc67a8e90 Prevent running plan or apply without permissions 2018-10-05 12:06:00 +02:00
Sander van Harmelen 53d322ec69 Test lock timeout errors when running a plan 2018-10-05 11:23:27 +02:00
Sander van Harmelen c12f0355a7 Revert "Merge pull request #18980 from hashicorp/f-policy-output"
This reverts commit f09c2db8d2, reversing
changes made to 8394dc797d.
2018-10-04 23:03:40 +02:00
Sander van Harmelen a2241e7c43 backend/remote: introduce support for `-no-color`
This is a bit of a hack to support the `-no-color` flag while we don’t have an option to set run variables.

That is also the reason why the orginal method is commented out instead of deleted. This will be reverted when the TFE starts supporting run variables.
2018-10-04 18:01:11 +02:00
Sander van Harmelen f09c2db8d2
Merge pull request #18980 from hashicorp/f-policy-output
backend/remote: only show the full policy output when it fails
2018-10-04 17:29:14 +02:00
Sander van Harmelen 3979aec0ae Ask to cancel a pending remote operation
Except when a lock-timeout has exceeded or auto-approve is set.
2018-10-04 17:16:45 +02:00
Sander van Harmelen 37f5ab3500 Only show the full policy output when it fails
If the policy passes, only show that instead of the full check output to prevent cluttering the output. So a passing policy will only show:

-----------------------------------------------
Organization policy check: passed
-----------------------------------------------
2018-10-02 19:16:17 +02:00
Sander van Harmelen b28f47055d backend/remote: extend mocks and add sentinel tests 2018-09-26 22:34:32 +02:00
Sander van Harmelen 2bd1040bbd backend/remote: extend mocks and add apply tests 2018-09-26 21:35:41 +02:00
Sander van Harmelen 9f9bbcb0e7 backend/remote: lots of improvements
This commit adds:

- support for `-lock-timeout`
- custom error message when a 404 is received
- canceling a pending run when TF is Ctrl-C’ed
- discard a run when the apply is not approved
2018-09-22 11:49:42 +02:00
Sander van Harmelen 621d589189 backend/remote: add support for the apply operation 2018-09-22 11:49:42 +02:00
Kristin Laemmert 813b5fd27f
Merge pull request #18825 from sean-/master-make-dev-drift
build: Updates required to bump golang version to 1.11
2018-09-17 09:00:04 -07:00
Sander van Harmelen 4aeb67b8ff backend/remote: handle empty results correctly
The pagination info of a list call that returns an empty list contains:

```go
CurrentPage: 1
TotalPages: 0
```

So checking if we have seen all pages using `CurrentPage == TotalPages` will not work and will result in an endless loop.

The tests are updated so they will fail (timeout after 1m) if this is handled incorreclty.
2018-09-17 16:22:54 +02:00
Sander van Harmelen ea88daa499 backend/remote: add support for state locking 2018-09-10 19:49:53 +02:00
Sander van Harmelen 7c1da2ffaa
Merge pull request #18818 from hashicorp/b-associate-state
backend/remote: add the run ID to associate state
2018-09-10 19:13:53 +02:00
Sander van Harmelen cd6d75bc03 backend/remote: add the run ID to associate state
If a run ID is available, we need to make sure we pass that when creating a new state version so the state will be properly associated with the run.
2018-09-09 21:30:42 +02:00
Sander van Harmelen 3cc58813f0 backend/remote: use a search query and use pagination
To prevent making unnecessary heavy calls to the backend, we should use a search query to limit the result.

But even if we use a search query, we should still use the pagination details to make sure we retrieved all items.
2018-09-09 21:29:50 +02:00
Sean Chittenden d749420a25
Fix drift caused from gofmt when running make dev and go 1.11.
A fresh checkout of `origin/master` does not build atm using the `dev`
target because `master` has not been formatted using `gofmt` from Go
1.11 (tis has been the case for a while if you've been running devel).

None of the drift in question is especially new but now that Go 1.11
has been released and gofmt's formatting guidelines have been updated,
it would be *really* nice if the code in `master` reflected the current
tooling in order to avoid having to fight this drift locally.

* 8mo: https://github.com/hashicorp/terraform/blame/master/backend/remote-state/s3/backend_test.go#L260-L261
* 6mo: https://github.com/hashicorp/terraform/blame/master/builtin/provisioners/chef/linux_provisioner_test.go#L124
* 1yr: 7cfeffe36b/command/init.go (L75-L76)
* 12d: 7cfeffe36b/command/meta_backend_test.go (L1437)
* 2yr: 7cfeffe36b/helper/schema/resource_timeout_test.go (L26)
* 4yr: 7cfeffe36b/helper/schema/schema_test.go (L2059)
* 1yr: 7cfeffe36b/plugin/discovery/get_test.go (L151)
2018-09-09 10:18:08 -07:00
Sander van Harmelen 62ab84a319 backend/remote: compat changes for latest `go-tfe` 2018-09-08 11:47:18 +02:00
Sander van Harmelen 3cfc83e335 backend/remote: take working directories into account
In TFE you can configure a workspace to use a custom working directory. When determining which directory that needs to be uploaded to TFE, this working directory should be taken into account to make sure we are uploading the correct root directory for the workspace.
2018-08-31 20:35:16 +02:00
Sander van Harmelen ce2869dced
Merge pull request #18760 from hashicorp/f-remote-backend
backend/migrations: migrate the default state
2018-08-30 11:16:25 +02:00
Sander van Harmelen f410a5bb26 backend/migrations: migrate the default state
Certain backends (currently only the `remote` backend) do not support using both the default and named workspaces at the same time.

To make the migration easier for users that currently use both types of workspaces, this commit adds logic to ask the user for a new workspace name during the migration process.
2018-08-29 21:37:39 +02:00
Sander van Harmelen 77b9fad7f0 backend/manta: deprecate camelcase attribute name
This change is needed (next to making it consistent as its the only attribute name that uses camelcase) because passing that attribute objectName in the update `terraform_remote_state` resource, ended up giving me this error: https://github.com/hashicorp/terraform/blob/master/helper/schema/schema.go#L736
2018-08-29 20:22:04 +02:00
Sander van Harmelen 7049d973a9 Do not use the TFE_TOKEN env variable
Instead promote the use of shared credentials using the CLI Config File
2018-08-10 20:20:16 +02:00
Sander van Harmelen 83d5f4147b backend/remote: use schema max/min items options 2018-08-05 15:30:27 +02:00
Sander van Harmelen 7fb2d1b8de Implement the Enterprise enhanced remote backend 2018-08-03 22:22:55 +02:00
Sander van Harmelen 495d1ea350 Use New() instead of `once.Do(b.init)` 2018-08-03 11:29:11 +02:00
Sander van Harmelen 97d1c46602 Update the backend import names
It’s a purely cosmetic change, but I find it easier to read them like this.
2018-08-03 11:29:11 +02:00