Commit Graph

110 Commits

Author SHA1 Message Date
Kristin Laemmert 604e65bb62 Revert "backend/local: release lock if there is an error in Context() (#25427)"
This reverts commit 1ba0d615e7.
2020-06-30 14:12:32 -04:00
Kristin Laemmert 1ba0d615e7
backend/local: release lock if there is an error in Context() (#25427)
* command/console: return in case of errors before trying to unlock remote
state

The remote backend `Context` would exit without an active lock if there
was an error, while the local backend `Context` exited *with* a lock. This
caused a problem in `terraform console`, which would call unlock
regardless of error status.

This commit makes the local and remote backend consistently unlock the
state incase of error, and updates terraform console to check for errors
before trying to unlock the state.

* adding tests for remote and local backends
2020-06-29 14:57:42 -04:00
James Bardin 35d232481f prevent panic in remote backend retry
Ensure that the *http.Response is not nil before checking the status.
2020-06-22 15:02:23 -04:00
Kristin Laemmert e6cf6cd758
backend/remote: do not panic if PrepareConfig or Configure receive null (#25135)
* backend/remote: do not panic if PrepareConfig or Configure receive null
objects

If a user cancels (ctrl-c) terraform init while it is requesting missing
configuration options for the remote backend, the PrepareConfig and
Configure functions would receive a null cty.Value which would result in
panics. This PR adds a check for null objects to the two functions in
question.

Fixes #23992
2020-06-05 09:11:44 -04:00
CJ Horton e1dcae17b7 add sad path tests for the TFP API version check 2020-05-19 11:14:48 -07:00
CJ Horton 68f199fc76 replace status string with go-tfe enum 2020-05-19 11:14:36 -07:00
Martin Atkins 8e1615a802 backend/remote: Handle cost estimation skipped due to targeting
The remote server might choose to skip running cost estimation for a
targeted plan, in which case we'll show a note about it in the UI and then
move on, rather than returning an "invalid status" error.

This new status isn't yet available in the go-tfe library as a constant,
so for now we have the string directly in our switch statement. This is
a pragmatic way to expedite getting the "critical path" of this feature
in place without blocking on changes to ancillary codebases. A subsequent
commit should switch this over to tfe.CostEstimateSkippedDueToTargeting
once that's available in a go-tfe release.
2020-05-18 15:12:44 -07:00
CJ Horton 0eea4e7c62 prevent targeting for unsupported API versions 2020-05-15 16:57:18 -07:00
Mike Nomitch 650a272a1d Change message sent to remote backend if -target used 2020-05-15 15:58:45 -07:00
Martin Atkins 16f1f3b739 backend/remote: Support -target on plan and apply
Previously we did not allow -target to be used with the remote backend
because there was no way to send the targets to Terraform Cloud/Enterprise
via the API.

There is now an attribute in the request for creating a plan that allows
us to send target addresses, so we'll remove that restriction and copy
the given target addresses into the API request.
2020-05-15 15:58:45 -07:00
Martin Atkins db4f3f8bc5 vendor: go get github.com/hashicorp/go-tfe@v0.8.0
This includes a new TargetAddrs field on both Run and RunCreateOptions
which we'll use to send resource addresses that were specified using
-target on the CLI command line when using the remote backend.

There were some unrelated upstream breaking changes compared to the last
version we had vendored, so this commit also includes some changes to the
backend/remote package to work with this new API, which now requires the
remote backend to be aware of the remote system's opaque workspace id.
2020-05-15 15:58:01 -07:00
Pam Selle 60b3815af4
Merge pull request #24696 from hashicorp/leetrout/remote-state-force-push
Add support for force pushing with the remote backend
2020-05-06 15:23:28 -04:00
Lee Trout cb0e20ca2b Add support for force pushing with the remote backend
Both differing serials and lineage protections should be bypassed
with the -force flag (in addition to resources).

Compared to other backends we aren’t just shipping over the state
bytes in a simple payload during the persistence phase of the push
command and the force flag added to the Go TFE client needs to be
specified at that time.

To prevent changing every method signature of PersistState of the
remote client I added an optional interface that provides a hook
to flag the Client as operating in a force push context. Changing
the method signature would be more explicit at the cost of not
being used anywhere else currently or the optional interface pattern
could be applied to the state itself so it could be upgraded to
support PersistState(force bool) only when needed.

Prior to this only the resources of the state were checked for
changes not the lineage or the serial. To bring this in line with
documented behavior noted above those attributes also have a “read”
counterpart just like state has. These are now checked along with
state to determine if the state as a whole is unchanged.

Tests were altered to table driven test format and testing was
expanded to include WriteStateForMigration and its interaction
with a ClientForcePusher type.
2020-05-06 12:07:43 -04:00
Kritonas 71395a0656 Fixed minor grammatical errors 2020-05-04 17:34:50 +03:00
Ryan Uber 105fcb3cac backend/remote: move reading policy check logs earlier 2020-04-15 12:31:44 -07:00
Ryan Uber 93906e1ad8 backend/remote: display cost estimate and policy check whenever available 2020-04-13 13:04:53 -07:00
Alisdair McDiarmid f15e58adf8 Improve remote backend missing token error
Prompt the user to run terraform login to generate and store a token for
the configured remote backend.
2020-02-05 13:25:29 -05:00
Robert Tillery af77d1d22c backend/remote: Filter environment variables when loading context (#23358)
* backend/remote: Filter environment variables when loading context

Following up on #23122, the remote system (Terraform Cloud or
Enterprise) serves environment and Terraform variables using a single
type of object. We only should load Terraform variables into the
Terraform context.

Fixes https://github.com/hashicorp/terraform/issues/23283.
2019-11-13 11:34:09 -05:00
Martin Atkins 8f27409007 backend/remote: Support HCL variable values in local operations
For remote operations, the remote system (Terraform Cloud or Enterprise)
writes the stored variable values into a .tfvars file before running the
remote copy of Terraform CLI.

By contrast, for operations that only run locally (like
"terraform import"), we fetch the stored variable values from the remote
API and add them into the set of available variables directly as part
of creating the local execution context.

Previously in the local-only case we were assuming that all stored
variables are strings, which isn't true: the Terraform Cloud/Enterprise UI
allows users to specify that a particular variable is given as an HCL
expression, in which case the correct behavior is to parse and evaluate
the expression to obtain the final value.

This also addresses a related issue whereby previously we were forcing
all sensitive values to be represented as a special string "<sensitive>".
That leads to type checking errors for any variable specified as having
a type other than string, so instead here we use an unknown value as a
placeholder so that type checking can pass.

Unpopulated sensitive values may cause errors downstream though, so we'll
also produce a warning for each of them to let the user know that those
variables are not available for local-only operations. It's a warning
rather than an error so that operations that don't rely on known values
for those variables can potentially complete successfully.

This can potentially produce errors in situations that would've been
silently ignored before: if a remote variable is marked as being HCL
syntax but is not valid HCL then it will now fail parsing at this early
stage, whereas previously it would've just passed through as a string
and failed only if the operation tried to interpret it as a non-string.
However, in situations like these the remote operations like
"terraform plan" would already have been failing with an equivalent
error message anyway, so it's unlikely that any existing workspace that
is being used for routine operations would have such a broken
configuration.
2019-10-31 09:45:50 -07:00
Martin Atkins 9f9f22091e backend/remote: Unlock workspace if Context fails
Previously any error case in the Context method would cause us to leave
the remote workspace locked on exit, requiring manual action to unlock it.
2019-10-31 09:45:50 -07:00
Martin Atkins b10f058cbb backend/remote: Only load variables if we're going to use them
Some commands don't use variables at all or use them in a way that doesn't
require them to all be fully valid and consistent. For those, we don't
want to fetch variable values from the remote system and try to validate
them because that's wasteful and likely to cause unnecessary error
messages.

Furthermore, the variables endpoint in Terraform Cloud and Enterprise only
works for personal access tokens, so it's important that we don't assume
we can _always_ use it. If we do, then we'll see problems when commands
are run inside Terraform Cloud and Enterprise remote execution contexts,
where the variables map always comes back as empty.
2019-10-18 11:31:19 -07:00
Martin Atkins a8d01e3940 backend/remote: Report invalid variables only remotely
The remote backend uses backend.ParseVariableValues locally only to decide
if the user seems to be trying to use -var or -var-file options locally,
since those are not supported for the remote backend.

Other than detecting those, we don't actually have any need to use the
results of backend.ParseVariableValues, and so it's better for us to
ignore any errors it produces itself and prefer to just send a
potentially-invalid request to the remote system and let the remote system
be responsible for validating it.

This then avoids issues caused by the fact that when remote operations are
in use the local system does not have all of the required context: it
can't see which environment variables will be set in the remote execution
context nor which variables the remote system will set using its own
generated -var-file based on the workspace stored variables.
2019-10-18 11:31:19 -07:00
Pam Selle a5f2f4cbba Update messaging for .terraformignore 2019-10-17 11:12:00 -04:00
Radek Simko 32f9722d9d
Replace import paths & set UA string where necessary 2019-10-11 22:40:54 +01:00
Paul Thrasher f2ef8ef317
slow down tfce polling to 1s
Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-09-30 14:22:15 -07:00
Paul Thrasher 4e308ef362
typos. some code, some text.
Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-09-30 13:44:23 -07:00
Paul Thrasher d2eaffabea
vendor latest go-tfe
Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-09-27 15:13:40 -07:00
Paul Thrasher d3fc3dee6e
clean up go mod for go-tfe
Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-09-27 14:16:41 -07:00
Paul Thrasher e2831b3d1e
tfce test additions
Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-09-27 14:16:41 -07:00
Paul Thrasher bc2d888b7b
update to go-tfe 0.3.23
this fixes the attr types

Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-09-27 14:16:05 -07:00
Paul Thrasher 39817ac476
cost estimation status polling
Properly wait for cost estimation to finish running before outputting
the results. Waits 500 milliseconds between checks, rather than backing
off exponentially, because we are not in a run queue. At the point we're
waiting, we expect cost estimation to be run in a timely manner.
2019-09-27 14:16:05 -07:00
Paul Thrasher 979bba0f32
tfce mock fix
Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-09-27 14:11:49 -07:00
Kristin Laemmert 412d459292
backend/remote: remove milseading contents from error message (#22148)
Previously, terraform was returning a potentially-misleading error
message in response to anything other than a 404 from the
b.client.Workspaces.Read operation. This PR simplifies Terraform's error
message with the intent of encouraging those who encounter it to focus
on the error message returned from the tfe client.

The added test is odd, and a bit hacky, and possibly overkill.
2019-07-22 09:06:39 -04:00
Kristin Laemmert 190ef537ec backend/remote: notify users when uploading something other than cwd
When a TFC workspace is configured without a VCS root, and with a
working directory, and a user is running `terraform init` from that same
directory, TFC uploads the entire configuration directory, not only the
user's cwd. This is not obvious to the user, so we are adding a descriptive
message explaining what is being uploaded, and why.
2019-07-18 09:50:17 -07:00
Kristin Laemmert 89eeaed0a0
[WIP] backend/enhanced: start with absolute configuration path (#22096)
* backend/enhanced: start with absolute config path

We recently started normalizing the config path before all "command"
operations, which was necessary for consistency but had unexpected
consequences for remote backend operations, specifically when a vcs root
with a working directory are configured.

This PR de-normalizes the path back to an absolute path.

* Check the error and add a test

It turned out all required logic was already present, so I just needed to add a test for this specific use case.
2019-07-17 08:39:37 -04:00
Radek Simko 5b9f2fafc8 Standardise directory name for test data 2019-06-30 10:16:15 +02:00
Mary Cutrali e44ca40702 update remote apply language to use proper its/it's 2019-06-14 12:22:21 -05:00
David Celis 5ccccfb91f
Remove pre-alpha cost estimation logic 2019-05-20 16:48:38 -07:00
Sander van Harmelen 394f20f59c backend/remote: do not unlock after a failed upload
When changes are made and we failed to upload the state, we should not
try to unlock the workspace. Leaving the workspace locked is a good
indication something went wrong and also prevents other changes from
being applied before the newest state is properly uploaded.

Additionally we now output the lock ID when a lock or force-unlock
action failed.
2019-04-29 21:23:33 +02:00
Paul Thrasher 151c91ffda
use scanner for reading logs
Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-04-25 11:17:08 -07:00
Paul Thrasher 0e27a8862f
remove duplicate remote output line
Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-04-25 10:52:19 -07:00
Sander van Harmelen bb12206bca Fixup the tests 2019-04-25 10:32:00 +02:00
Sander van Harmelen 7cf744241a Do not use a scanner to read the logs
Using a scanner can cause issues when reading long lines. Also make sure we return the error correctly while planning.
2019-04-25 09:51:52 +02:00
Paul Thrasher c7a023a95c
update test for new go-tfe version
Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-04-24 16:46:17 -07:00
Paul Thrasher 53f977bee2
TFCE api correct pluralization
Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-04-24 16:44:00 -07:00
Paul Thrasher e479bd5dc3
update to latest go-tfe
same version number but pointing to a new sha

Signed-off-by: Paul Thrasher <pthrasher@hashicorp.com>
2019-04-24 16:44:00 -07:00
Sander van Harmelen 90bc237b7b Prevent a panic caused by writing to a nil map 2019-04-11 14:34:14 +02:00
Sander van Harmelen 39a95e4222 backend/remote: correctly load remote variables
When using `terraform console` in combination with the remote backend, variables defined in Terraform Enterprise were load loaded correctly.
2019-03-28 17:23:48 +01:00
Sander van Harmelen 9f6a126293 backend/remote: check for external updates 2019-03-08 19:18:07 +01:00
Sander van Harmelen 4628fbcc65
Merge pull request #20615 from hashicorp/svh/f-input-context
core: add a context to the UIInput interface
2019-03-08 18:44:28 +01:00