Commit Graph

197 Commits

Author SHA1 Message Date
James Bardin 3fbdee0777 clean up remote.Cmd api
Combine the ExitStatus and Err values from remote.Cmd into an error
returned by Wait, better matching the behavior of the os/exec package.

Non-zero exit codes are returned from Wait as a remote.ExitError.
Communicator related errors are returned directly.

Clean up all the error handling in the provisioners using a
communicator. Also remove the extra copyOutput synchronization that was
copied from package to package.
2018-03-16 14:29:48 -04:00
James Bardin a1061ed931 update the chef and habitat error handling
Use the new ExitStatus method, and also check the cmd.Err() method for
errors.

Remove leaks from the output goroutines in both provisioners by
deferring their cleanup, and returning early on all error conditions.
2018-03-15 16:04:05 -04:00
James Bardin a715430d24 fix exit status handling in salt-masterless
Convert to the new Cmd.ExitStatus() method in the salt-masterless
provisioner. Add calls to Wait and remove race conditions around setting
the status.
2018-03-15 16:04:00 -04:00
James Bardin af132a186d remove timeout from remote-exec command context
The timeout for the remote command was taken from the wrong config
field, and the connection timeout was being used which is 5 min. Any
remote command taking more than 5 min would be terminated by
disconnecting the communicator. Remove the timeout from the context, and
rely on the global timeout provided by terraform.

There was no way to get the error from the communicator previously, so
the broken connection was silently ignored and the provisioner returned
successfully. Now we can use the new cmd.Err() method to retrieve any
errors encountered during execution.
2018-03-15 16:03:40 -04:00
Kristiyan Nikolov 999f9096c1 provisioner/local-exec: Allow passing environment variables 2018-03-05 15:58:49 -08:00
Paweł Socha be8d39210d Wait for ssh connection 2018-02-26 15:40:26 +01:00
Paweł Socha 627bb24ea6 fix ssh problem with communicator.Retry 2018-02-26 10:27:14 +01:00
Rob Campbell bbd3d7f7a2 Updates the capitalization of the bind documenation for habitat
provisioner. Also fixes an issue where channels and URLs are
not honored in the initial package install.

Signed-off-by: Rob Campbell <rcampbell@chef.io>
2018-02-21 11:47:45 -05:00
chrisjob1021 10bb21e9c4 provisioner/local-exec: Optional "working_dir" argument
This new argument allows overriding of the working directory of the child process, with the default still being the working directory of Terraform itself.
2018-02-16 11:31:11 -08:00
James Bardin 0345d960b2 simplify remote-exec runScripts
There no reason to retry around the execution of remote scripts. We've
already established a connection, so the only that could happen here is
to continually retry uploading or executing a script that can't succeed.

This also simplifies the streaming output from the command, which
doesn't need such explicit synchronization. Closing the output pipes is
sufficient to stop the copyOutput functions, and they don't close around
any values that are accessed again after the command executes.
2018-02-15 16:14:38 -05:00
James Bardin c1b35ad69b have the ssh communicator return fatal errors
This will let the retry loop abort when there are errors which aren't
going to ever be corrected.
2018-02-15 16:14:33 -05:00
James Bardin e331ae9842 remove retryFunc
it's now in the communicator package
2018-02-14 18:32:29 -05:00
James Bardin d02250c2b9 remove retryFunc
it's now in the communicator package
2018-02-14 18:30:20 -05:00
James Bardin 89a0ac6e89 remove retryFunc
It's now in the communicator package
2018-02-14 18:25:05 -05:00
James Bardin f5b8091e2c remove retryFunc
It's now in the communicator package
2018-02-14 18:21:26 -05:00
Nolan Davidson f43e592849 [provisioner-habitat] Detect if hab user exists (#17195)
Currently the provisioner will fail if the `hab` user already exists on
the target system.

This adds a check to see if we need to create the user before trying to
add it.

Fixes #17159

Signed-off-by: Nolan Davidson <ndavidson@chef.io>
2018-02-13 14:13:22 -06:00
Nolan Davidson 848375b9a6 [provisioner-habitat] Allow custom service name (#17196)
This change allows the Habitat supervisor service name to be
configurable. Currently it is hard coded to `hab-supervisor`.

Signed-off-by: Nolan Davidson <ndavidson@chef.io>
2018-02-13 14:11:59 -06:00
Jeremiah Snapp 7595e27772 Fix chef provisioner validateFn
Correctly validate Chef provisioner's `use_policyfile`
field even if its value is a string type.

Signed-off-by: Jeremiah Snapp <jeremiah@chef.io>
2018-01-25 09:24:05 -05:00
Masayuki Morita f440dba137 Standardize on log level "WARN" rather than "WARNING" 2018-01-16 18:05:26 -08:00
James Bardin 4b49a323c3 go fmt
slight change to go fmt coming in 0.10
2017-12-26 13:26:38 -05:00
Rob Campbell 29f70bc112 Adds build_auth_token to Habitat Provisioner
First successful run with private origin and HAB_AUTH_TOKEN set

Update struct, schema, and decodeConfig names to more sensible versions

Cleaned up formatting

Update habitat provisioner docs

Remove unused unitstring
2017-12-12 19:46:42 -05:00
Nolan Davidson a50a383946 Additional work on the habitat provisioner.
Signed-off-by: Nolan Davidson <ndavidson@chef.io>
2017-12-07 16:29:30 -08:00
Nolan Davidson 653db95df7 Initial implementation of a habitat provisioner
First pass at loading the config data using the TF schema.

Signed-off-by: Nolan Davidson <ndavidson@chef.io>
2017-12-07 16:29:30 -08:00
Subba Rao Pasupuleti 44cb98e04f provisioner/salt-masterless: Wait for operations to complete
Previously the provisioner did not wait until the Salt operation had completed before returning, causing some operations not to be applied, and causing the output to get swallowed.

Now we wait until the remote work is complete, and copy output into the Terraform log in a similar way as is done for other provisioners.
2017-12-05 09:26:40 -08:00
Christoph Kappel af206e7543 Clean clients from chef-vault on recreate_client enabled (#16357)
Fixes #15921

When terraform re-creates an existing node/client with chef provisioner,
the already existing client (which has old keys) must be removed from
the vault items. Afterwards, the chef-vault will be updated with the
newly created client (which has the new keys). Therefore, the recreated
client will be able to decrypt the vault items properly.
2017-10-20 17:46:29 +02:00
Martin Atkins 6a4498ba76 provisioner/salt-masterless: add "_file" suffix to "minion_config"
In #15870 we got good feedback that it'd be more useful to have the
various filename-accepting arguments on this provisioner instead accept
strings that represent the contents of such files, so that they can be
generated from elsewhere in the Terraform config.

This change does not achieve that, but it does make room for doing this
later by renaming "minion_config" to "minion_config_file" so that we
can later add a "minion_config" option alongside that takes the file
content, and deprecate "minion_config_file".

Ideally we'd just implement the requested change immediately, but
unfortunately the release schedule doesn't have time for this so this is
a pragmatic change to allow us to make the full requested change at a
later date without backward incompatibilities.

This change is safe because the salt-masterless provisioner has not yet
been included in a release at the time of this commit.
2017-08-30 13:55:28 -07:00
Martin Atkins 4aa67f0bfc provisioner/salt-masterless: fix crash processing connection config
The code here was previously assuming that d.State() was equivalent to
the schema.ProvRawStateKey due to them both being of type InstanceState,
but that is in fact not true since a state object contains some transient
information that is _not_ part of the persisted state, including the
connection information we need here.

Calling ResourceData.State() constructs a _new_ state based on its stored
values, so the constructed object is lacking this transient information.
We need to use the specific state object provided by the caller in order
to get access to the transient connection configuration.

Unfortunately there is no automated test coverage for this because we have
no good story for testing provisioners that use "communicator". While such
tests could potentially be written, we'd like to get this in somewhat
quickly to unblock a release, rather than delaying to design and implement
some sort of mocking system for this.
2017-08-30 13:50:30 -07:00
Stephen Weatherford 5d5f8224d2 provisioner/local-exec: allow user to specify interpreter 2017-08-22 10:16:36 -07:00
Lars Lehtonen 47484e23fe provisioner/local-exec: Drop untestable error
TestResourceProvider_stop uses a goroutine, which means that any function with *testing.T as its receiver within that goroutine will silently fail.

Now the test to accepts that an error that occurs within the goroutine is lost. It also adds some more verbose logs to explain what is happening.
2017-08-21 19:05:49 -07:00
James Bardin 18af7750d4 Merge pull request #14720 from sevagh/master
Add salt-masterless provisioner
2017-08-16 16:47:46 -04:00
James Bardin db42015342 backoff retries in remote-exec provisioner
Add a simple backoff to the remote-exec retryFunc.
Backoff between tries, up to a 10s max.
2017-08-09 15:54:08 -04:00
Sevag Hanssian 3477868804 PR feedback 2017-08-07 12:21:27 -04:00
Sevag Hanssian 867760ed56 Add salt-masterless provisioner 2017-08-07 10:00:29 -04:00
Sander van Harmelen b01f68f343 provisioner/chef: fix panic
Fixes #15431
Fixes #15500
2017-07-22 09:58:58 +02:00
Sander van Harmelen 7e180aec92 Refactor the provisioner validation function (#15273)
It turns out that `d.GetOk` also returns `false` when the user _did_ actually supply a value for it in the config, but the value itself needs to be evaluated before it can be used.

So instead of passing a `ResourceData` we now pass a `ResourceConfig`
which makes much more sense for doing the validation anyway.
2017-06-15 19:57:04 +02:00
Sander van Harmelen 21a646f6fe Use the InstanceState to query any connection details (#15271)
Fixes #15205 #15270
2017-06-14 21:40:31 +02:00
Sander van Harmelen 0e422737ba Fix and refactor the Chef provisioner
The tests did pass, but that was because they only tested part of the changes. By using the `schema.TestResourceDataRaw` function the schema and config are better tested and so they pointed out a problem with the schema of the Chef provisioner.

The `Elem` fields did not have a `*schema.Schema` but a `schema.Schema` and in an `Elem` schema only the `Type` field may (and must) be set. Any other fields like `Optional` are not allowed here.

Next to fixing that problem I also did a little refactoring and cleaning up. Mainly making the `ProvisionerS` private (`provisioner`) and removing the deprecated fields.
2017-05-19 21:05:21 +02:00
Vladislav Rassokhin f5449a62e0 Various built-in provisioners improvements:
1. Migrate `chef` provisioner to `schema.Provisioner`:

 * `chef.Provisioner` structure was renamed to `ProvisionerS`and  now it's decoded from `schema.ResourceData` instead of `terraform.ResourceConfig` using simple copy-paste-based solution;
 * Added simple schema without any validation yet.

 2. Support `ValidateFunc` validate function : implemented in `file` and `chef` provisioners.
2017-05-19 20:43:51 +02:00
Jake Champlin 7e5eeb2268
provisioner/remote-exec: Fix panic from remote_exec provisioner
Fixes panic on `nil` values of `inline` and `scripts` from improper interface casts.

Fixes: #13970
2017-05-01 16:48:42 -04:00
Sander van Harmelen 7304fe5c14 Make sure we add new Chef-Vault clients as clients (#13525)
This is possible using the newly released Chef-Vault 3.0 gem. Before we could only add new clients as admins.

Fixes #9137
2017-04-11 10:36:05 +02:00
James Bardin fd2ea3cd82 remove log.Fatal from test
also increase the timeout for slow test hosts.
2017-02-24 18:48:48 -05:00
James Bardin 197f6cab79 Cannot store multiple types in atomic.Value
Storing error values to atomic.Value may fail if they have different
dynamic types. Wrap error value in a consistent struct type to avoid
panics.

Make sure we return a nil error on success
2017-02-07 19:02:04 -05:00
Mitchell Hashimoto 640faf18c3
Revert "provisioner/remote-exec: fail on first inline script with bad exit code (#11155)"
This reverts commit d2047d714e.
2017-02-06 16:51:51 -08:00
James Bardin ff2936bb3f Fix cancellation when spawning a subprocess
If the shell spawns a subprocess which doesn't close the output file
descriptors, the exec.Cmd will block on Wait() (see
golang.org/issue/18874). Use an os.Pipe to provide the command with a
real file descriptor so the exec package doesn't need to do the copy
manually. This in turn may block our own reading goroutine, but we can
select on that and leave it for cleanup later.
2017-02-01 12:01:29 -05:00
James Bardin e0325d9b8f fix race with multiple calls to cmd.Wait()
There was still a race around the local-exec Command, where we were
calling Wait in 2 places which you can't do.
2017-01-31 18:07:26 -05:00
Mitchell Hashimoto 61881d2795 Merge pull request #10934 from hashicorp/f-provisioner-stop
core: stoppable provisioners, helper/schema for provisioners
2017-01-30 12:53:15 -08:00
Mitchell Hashimoto 3776d31d69
provisioners/local-exec: remove data race by setting err only once 2017-01-30 10:21:05 -08:00
Mitchell Hashimoto 5fc516f99d
provisioners/chef: check IsComputed for JSON attributes
Fixes #10788

This checks `IsComputed` prior to attempting to use the JSON
configurations. Due to a change in 0.8, the prior check for simply map
existence would always succeed even with a computed value (as designed),
but we forgot to update provisioners to not do that.

There are other provisioners that also do this but to no ill effect
currently. I've only changed Chef since we know that is an issue.

This issue doesn't affect 0.9 due to helper/schema doing this
automatically for provisioners.
2017-01-29 12:30:44 -08:00
Mitchell Hashimoto 142df657c3
provisioners/remote-exec: listen to Stop 2017-01-26 15:10:30 -08:00
Mitchell Hashimoto 487a37b0dd
helper/schema: PromoteSingle for legacy support of "maybe list" types 2017-01-26 15:09:15 -08:00
Mitchell Hashimoto f29845e54e
update privisioner bins to use new functions 2017-01-26 15:09:15 -08:00
Mitchell Hashimoto 3c0c81957a
provisioners/remote-exec: switch to helper/schema 2017-01-26 15:09:15 -08:00
Mitchell Hashimoto 27c19af9ff
provisioners/file: support Stop 2017-01-26 15:03:28 -08:00
Mitchell Hashimoto a2e044829b
provisioners/file: use the old communicator.New just to minimize risk 2017-01-26 15:03:28 -08:00
Mitchell Hashimoto 02a4adc07c
provisioners/file: convert to helper/schema 2017-01-26 15:03:28 -08:00
Mitchell Hashimoto 0fb87cd96b
provisioners/local-exec: stoppable
This modifies local-exec to be stoppable with the new Stop API call that
provisioners can listen to.
2017-01-26 15:03:28 -08:00
Mitchell Hashimoto c5b784c33f
provisioners/local-exec: switch to helper/schema 2017-01-26 15:03:28 -08:00
Peter McAtominey d2047d714e provisioner/remote-exec: fail on first inline script with bad exit code (#11155)
The provisioner collected all inline commands into a single script which meant
only the exit code of the last command was actually checked for an error.
2017-01-20 14:04:43 +00:00
Kyle Persohn b300cac97b provisioner/chef: Support named run-lists for Policyfiles (#11215)
* provisioner/chef: Support named run-lists for Policyfiles

Add an optional argument for overriding the Chef Client's initial
run with a named run-list specified by the Policyfile. This is useful
for bootstrapping a node with a one-time setup recipe that deviates
from a policy's normal run-list.

* Update chef client cmd building per review feedback.
2017-01-19 09:03:45 +01:00
Mitchell Hashimoto 4b1d9cfd7d
communicator/ssh: don't share rand object to guarantee unique values
Fixes #10463

I'm really surprised this flew under the radar for years...

By having unique PRNGs, the SSH communicator could and would
generate identical ScriptPaths and two provisioners running in parallel
could overwrite each other and execute the same script. This would
happen because they're both seeded by the current time which could
potentially be identical if done in parallel...

Instead, we share the rand now so that the sequence is guaranteed
unique. As an extra measure of robustness, we also multiple by the PID
so that we're also protected against two processes at the same time.
2016-12-06 00:21:49 -08:00
Sam Dunne 628743b24c Add ability to skip chef registration
This change allows a user to skip the bootstrap stage for a machine.
2016-10-01 00:49:39 +02:00
Sander van Harmelen c307dc9557 Accept both slices and strings in vault_json (#9114)
Fixes #9105 by allowing the `vault_json` to contain either slices or strings.

And fixes #8932 by changing to way we cleanup the user key.
2016-10-01 00:35:27 +02:00
Sander van Harmelen 968472a63e Support recreating clients and configuring Chef Vaults (#8577)
Fixes #3605 and adds the functionality suggested in PR #7440.

This PR is using a different appraoch that (IMHO) feels cleaner and (even more important) adds support for Windows at the same time.
2016-09-15 14:20:18 +02:00
Sander van Harmelen 4004790247 Make using `ssl_verify_mode` more robust (#7769)
And prettify the template output by removing additions empty lines.
2016-07-22 16:01:48 +02:00
James Nugent 8beafe25ae provisioner/file: Clean up temporary files 2016-07-08 19:35:00 +01:00
jorge.marey 9b065f32e2 Include content option for file provisioner
- Include new option in file provisioner. Now content or source can be
  provided. Content will create a temp file and copy there the contents.
- Later that file will be used as source.
- Include test to check that changes are working correctly.
2016-07-08 19:34:59 +01:00
Paul Hinze 579680cebc provisioner/remote-exec: Move script cleanup after command wait
The script cleanup step added in #5577 was positioned before the
`cmd.Wait()` call to ensure the command completes. This was causing
non-deterministic failures, especially for longer running scripts.

Fixes #5699
Fixes #5737
2016-03-21 10:49:32 -05:00
Paul Hinze 96ebf0af0e provisioner/remote-exec: Clear out scripts after uploading
Prevents residual script contents from remaining on machine.

Fixes #482
2016-03-11 10:25:11 -06:00
Ben a97f1a557f Provide correct command to fetchChefCertificatesFunc
fetchChefCertificatesFunc expects the knife command path, not the chef command path.  Update the code to pass the correct command's path to the method.
2016-02-25 12:10:50 -07:00
Sander van Harmelen ac0cbd400e Add `attributes_json` param for consistency
Add `attributes_json` param for both consistency and easier management
of deprecating the old `attributes` param.
2016-02-09 11:11:46 +01:00
Sander van Harmelen 79e2642dab Fix issue #4881
This fixes issue #4881 by adding an option to fetch the Chef SSL
certificates.
2016-02-04 15:31:24 +01:00
Sander van Harmelen da927fcd08 Make the Chef `attributes` param also accept a raw JSON string
See the updated docs for more details and examples, but in short this
enables the `attributes` param from the Chef provisioner to accept a
raw JSON string.

Fixes #3074
Fixes #3572
2016-01-29 18:41:14 +01:00
Jason Riddle ca39512fa7 Fix the failing chef provisioner test 2016-01-14 13:51:23 -05:00
Jason Riddle 5cb4b70e7e Add ENV['no_proxy'] to chef provisioner 2016-01-13 09:51:08 -05:00
Sander van Harmelen 986245c553 provisioner/chef: fixes #4262
This small tweak fixes #4262 by making sure files can be uploaded
correctly.
2016-01-09 15:56:49 +01:00
Sander van Harmelen 71ffb6caa0 Add the option to add arbitrary `client.rb` options
Fixes #3630
2016-01-09 00:42:02 +01:00
Petr Artamonov 76913703a9 modified to have less code and not to verify incoming string 2016-01-08 20:00:11 +01:00
Petr Artamonov 64f19c0dc3 enable reporting flag 2016-01-07 18:13:30 +01:00
Sander van Harmelen b007d4f77a Revert changing `path.Join()` to `filepath.Join()` (PR #3896)
PR #3896 added support for passing keys by content, but in this same PR
all references to `path.Join()` where changed to `filepath.join()`.
There is however a significant difference between these two calls and
using the latter one now causes issues when running the Chef
provisioner on Windows (see issue #4039).
2015-11-24 14:40:46 +01:00
Paul Hinze 73ce6d184a chef: read key contents instead of paths
Builds on the work of #3846, shifting the Chef provisioner's
configuration options from `secret_key_path` and `validation_key_path`
over to `secret_key` and `validation_key`.
2015-11-12 16:11:44 -06:00
Joel Moss b1d731bd6f [chef provisioning] When use_policyfile is given, the run list is not used, so don't require it 2015-10-14 19:05:38 +01:00
Radek Simko f9efede852 gofmt files from recently merged PRs 2015-10-07 13:35:06 -07:00
Joel Moss 20362e7506 Added Policyfile support to the Chef provisioner
This Adds three new arguments `use_policyfile`, `policy_group` and `policy_name` to the Chef
provisioner. If `use_policyfile` == true, then the other arguments are required.
2015-09-26 00:56:24 +01:00
Sander van Harmelen 162568e682 Fixes issue #2872
Still not a 100% fix, but that would require some more hacking in core
TF. If time permits I’ll have a look at that later on… But for now this
is a good fix to be able to close #2872
2015-08-21 17:26:32 +02:00
Sander van Harmelen 885b4e9278 Updating the test accordingly... 2015-07-20 18:34:44 +02:00
Sander van Harmelen df909ca3ca Fix an issue with `sudo` and `hints`
Fixes issue #2781
2015-07-20 18:31:32 +02:00
Sander van Harmelen 4a8ef78d33 Fixes #2676 by prefixing all Windows commands
By prefixing them with `cmd /c` it will work with both `winner` and
`ssh` connection types.

This PR also reverts some bad stringer changes made in PR #2673
2015-07-10 12:56:27 +02:00
Sander van Harmelen 97fd4f5b7d Tweaking the tests 2015-07-09 21:29:27 +02:00
Joshua Seidel cda814d8b3 No need to do this as they both are in their own scope 2015-07-09 09:32:13 -04:00
Joshua Seidel 4070805fcd ran gofmt removed required, added IF 2015-07-09 09:05:12 -04:00
Joshua Seidel 61f47d440d correct space, fix reading of file 2015-07-08 20:06:37 -04:00
Joshua Seidel 090248fab1 fix errors 2015-07-08 19:41:38 -04:00
Joshua Seidel cd58da773c remove unused if 2015-07-08 15:25:00 -04:00
Joshua Seidel 7f4a5ac413 fix if statement and condition 2015-07-08 15:09:17 -04:00
Joshua Seidel 6e21ca50a0 add chef secret key 2015-07-08 14:52:23 -04:00
Jeroen de Korte 1c14bfd04d Added the HTTPS proxy option 2015-07-03 16:27:52 +02:00
Jeroen de Korte 007ab6e503 Fixed the http_proxy 2015-07-03 15:04:58 +02:00
Sander van Harmelen 3d03f4b2aa Fixing the tests 2015-06-30 14:13:36 +02:00
Sander van Harmelen b7e981c6d1 Fixes issue #2568
When surrounding the version with quotes, even no version (an empty
string) will be accepted as parameter. The install.sh script treats an
empty version string the same as no when version is set. So it will
then just use the latest available version.
2015-06-30 14:03:06 +02:00
Sander van Harmelen 61517f68b1 Add an `ohai_hints` option to upload hint files
This option takes a list of hints that will be uploaded to the new node
before starting the initial Chef run.
2015-06-25 16:32:36 +02:00