Commit Graph

56 Commits

Author SHA1 Message Date
James Bardin 2954d9849a add some more functionality to MockCommunicator 2018-03-20 14:23:32 -04:00
James Bardin 2d7dc605a0 get communicator errors from a remote.Cmd
The remote.Cmd struct could not convey any transport related error to
the caller, meaning that interrupted commands would show that they
succeeded.

Change Cmd.SetExited to accept an exit status, as well as an error to
store for the caller.  Make the status and error fields internal,
require serialized access through the getter methods.

Users of remote.Cmd should not check both Cmd.Err() and Cmd.ExitStatus()
until after Wait returns.

Require communicators to call Cmd.Init before executing the command.
This will indicate incorrect usage of the remote.Cmd by causing a panic
in SetExitStatus.
2018-03-15 16:03:20 -04:00
James Bardin e41b29d096
Merge pull request #17354 from hashicorp/jbardin/known_hosts
Verify host keys in ssh connections
2018-02-15 18:33:41 -05:00
James Bardin c1b35ad69b have the ssh communicator return fatal errors
This will let the retry loop abort when there are errors which aren't
going to ever be corrected.
2018-02-15 16:14:33 -05:00
James Bardin e06f76b90f Fix type assertion when loading stored error
Fix a bug where the last error was not retrieved from errVal.Load
due to an incorrect type assertion.
2018-02-15 15:59:34 -05:00
James Bardin bc90eca19f add the remote-exec retry function to communicator
Every provisioner that uses communicator implements its own retryFunc.
Take the remote-exec implementation (since it's the most complete) and
put it in the communicator package for each provisioner to use.

Add a public interface `communicator.Fatal`, which can wrap an error to
indicate a fatal error that should not be retried.
2018-02-14 18:18:12 -05:00
James Bardin bdfa97dbdb add tests for signed host certs
This checks that we can verify host certificates signed by a CA
2018-02-14 15:35:41 -05:00
James Bardin 1e7fd1c4ea add test for host key validation
This tests basic known_hosts validation for the ssh communicator.
2018-02-14 15:35:41 -05:00
James Bardin 1a68fdb4f6 add support for ssh host key checking
Add `host_key` and `bastion_host_key` fields to the ssh communicator
config for strict host key checking.

Both fields expect the contents of an openssh formated public key. This
key can either be the remote host's public key, or the public key of the
CA which signed the remote host certificate.

Support for signed certificates is limited, because the provisioner
usually connects to a remote host by ip address rather than hostname, so
the certificate would need to be signed appropriately. Connecting via
a hostname needs to currently be done through a secondary provisioner,
like one attached to a null_resource.
2018-02-14 15:35:41 -05:00
James Bardin daf05e65e0 test identity file parsing 2017-12-26 16:27:18 -05:00
James Bardin 8c8847e1cf sort ssh agent signers by requested id
It's becoming more common for users to have many ssh keys loaded into an
agent, and with the default max auth attempts of an openssh server at 6,
one often needs to specify which id to use in order to avoid a `too many
authentication failures` error.

Add a connection field called `agent_identity` which will function
similarly to the ssh_config IdentityFile when used in conjunction with
an ssh agent. This uses `agent_identity` rather than `identity_file` to
specify that the file is not used directly for authentication, rather
it's used to choose which identity returned from the agent to
authenticate with first.

This feature tries a number of different methods to match the agent
identity. First the provisioner attempts to read the id file and extract
the public key. If that isn't available, we look for a .pub authorized
key file. Either of these will result in a public key that can be
matched directly against the agent keys. Finally we fall back to
matching the comment string exactly, and the id as a suffix. The only
result of using the agent_identity is the reordering of the public keys
used for authentication, and if there is no exact match the client
will still attempt remaining keys until there is an error.
2017-12-26 16:27:18 -05:00
jd3nn1s 21c9c2ce00 communicator/winrm: pass cacert option correctly
It appears that the cacert option for the winrm provisioner was
not getting passed correctly to the winrm package. Log output
showed that CACert was false regardless of configuration.

While the validation of the connector looked for cacert, the winrm
communicator looked for ca_cert.
2017-10-23 13:28:41 -07:00
Adam Shannon c9c2823f62 communicator/ssh: add what error details we can for the user
ssh.Waitmsg's String() method provides output which can include the
process status, signal, and message
2017-10-03 09:06:02 -05:00
Wang Guoliang 383b0c176c
optimization:use bytes.Equal instead of bytes.Compare 2017-09-13 20:58:14 +08:00
Jake Champlin 6e599672e1
Remove LGPL dependencies
This changeset performs the following:

- Updates `masterzen/winrm` vendor to include change from (https://github.com/masterzen/winrm/pull/73)
- Updates `dylanmei/winrmtest` vendor to include change from (https://github.com/dylanmei/winrmtest/pull/4)
- Updates `packer-community/winrmcp` vendor to include the removal of the `masterzen/winrm/winrm` sub-class as a result of the `winrm` CLI tool being removed from the `masterzen/winrm` repository.
- Changes `communicator/winrm/communicator.go` to conform to the new ABI in the `masterzen/winrm` library.

This should completely remove any LGPL licensed dependencies inside of the Terraform project.

```
$ make test
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/08/20 13:40:16 Generated command/internal_plugin_list.go
go test -i $(go list ./... | grep -v '/terraform/vendor/' | grep -v '/builtin/bins/') || exit 1
echo $(go list ./... | grep -v '/terraform/vendor/' | grep -v '/builtin/bins/') | \
        xargs -t -n4 go test  -timeout=60s -parallel=4
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform github.com/hashicorp/terraform/backend github.com/hashicorp/terraform/backend/atlas github.com/hashicorp/terraform/backend/init
ok      github.com/hashicorp/terraform  0.011s
ok      github.com/hashicorp/terraform/backend  0.020s
ok      github.com/hashicorp/terraform/backend/atlas    0.634s
ok      github.com/hashicorp/terraform/backend/init     0.007s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/backend/legacy github.com/hashicorp/terraform/backend/local github.com/hashicorp/terraform/backend/remote-state github.com/hashicorp/terraf
orm/backend/remote-state/azure
ok      github.com/hashicorp/terraform/backend/legacy   0.009s
ok      github.com/hashicorp/terraform/backend/local    0.211s
ok      github.com/hashicorp/terraform/backend/remote-state     0.006s
ok      github.com/hashicorp/terraform/backend/remote-state/azure       0.010s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/backend/remote-state/consul github.com/hashicorp/terraform/backend/remote-state/inmem github.com/hashicorp/terraform/backend/remote-state/s
3 github.com/hashicorp/terraform/backend/remote-state/swift
ok      github.com/hashicorp/terraform/backend/remote-state/consul      0.007s
ok      github.com/hashicorp/terraform/backend/remote-state/inmem       0.013s
ok      github.com/hashicorp/terraform/backend/remote-state/s3  0.007s
ok      github.com/hashicorp/terraform/backend/remote-state/swift       0.013s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/builtin/providers/test github.com/hashicorp/terraform/builtin/provisioners/chef github.com/hashicorp/terraform/builtin/provisioners/file gi
thub.com/hashicorp/terraform/builtin/provisioners/local-exec
ok      github.com/hashicorp/terraform/builtin/providers/test   1.544s
ok      github.com/hashicorp/terraform/builtin/provisioners/chef        0.017s
ok      github.com/hashicorp/terraform/builtin/provisioners/file        0.006s
ok      github.com/hashicorp/terraform/builtin/provisioners/local-exec  0.078s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/builtin/provisioners/remote-exec github.com/hashicorp/terraform/builtin/provisioners/salt-masterless github.com/hashicorp/terraform/command
 github.com/hashicorp/terraform/command/clistate
ok      github.com/hashicorp/terraform/builtin/provisioners/remote-exec 1.037s
ok      github.com/hashicorp/terraform/builtin/provisioners/salt-masterless     0.008s
ok      github.com/hashicorp/terraform/command  14.589s
?       github.com/hashicorp/terraform/command/clistate [no test files]
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/command/e2etest github.com/hashicorp/terraform/command/format github.com/hashicorp/terraform/communicator github.com/hashicorp/terraform/co
mmunicator/remote
ok      github.com/hashicorp/terraform/command/e2etest  3.729s
ok      github.com/hashicorp/terraform/command/format   0.004s
ok      github.com/hashicorp/terraform/communicator     0.005s
ok      github.com/hashicorp/terraform/communicator/remote      0.003s [no tests to run]
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/communicator/shared github.com/hashicorp/terraform/communicator/ssh github.com/hashicorp/terraform/communicator/winrm github.com/hashicorp/
terraform/config
ok      github.com/hashicorp/terraform/communicator/shared      0.007s
ok      github.com/hashicorp/terraform/communicator/ssh 0.016s
ok      github.com/hashicorp/terraform/communicator/winrm       0.018s
ok      github.com/hashicorp/terraform/config   0.213s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/config/module github.com/hashicorp/terraform/dag github.com/hashicorp/terraform/digraph github.com/hashicorp/terraform/flatmap
ok      github.com/hashicorp/terraform/config/module    0.044s
ok      github.com/hashicorp/terraform/dag      0.010s
ok      github.com/hashicorp/terraform/digraph  0.002s
ok      github.com/hashicorp/terraform/flatmap  0.002s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/helper/acctest github.com/hashicorp/terraform/helper/config github.com/hashicorp/terraform/helper/copy github.com/hashicorp/terraform/helpe
r/diff
?       github.com/hashicorp/terraform/helper/acctest   [no test files]
ok      github.com/hashicorp/terraform/helper/config    0.005s
?       github.com/hashicorp/terraform/helper/copy      [no test files]
ok      github.com/hashicorp/terraform/helper/diff      0.005s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/helper/encryption github.com/hashicorp/terraform/helper/experiment github.com/hashicorp/terraform/helper/hashcode github.com/hashicorp/terr
aform/helper/hilmapstructure
?       github.com/hashicorp/terraform/helper/encryption        [no test files]
ok      github.com/hashicorp/terraform/helper/experiment        0.001s
ok      github.com/hashicorp/terraform/helper/hashcode  0.001s
?       github.com/hashicorp/terraform/helper/hilmapstructure   [no test files]
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/helper/logging github.com/hashicorp/terraform/helper/mutexkv github.com/hashicorp/terraform/helper/pathorcontents github.com/hashicorp/terr
aform/helper/resource
?       github.com/hashicorp/terraform/helper/logging   [no test files]
ok      github.com/hashicorp/terraform/helper/mutexkv   0.055s
ok      github.com/hashicorp/terraform/helper/pathorcontents    0.002s
ok      github.com/hashicorp/terraform/helper/resource  2.659s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/helper/schema github.com/hashicorp/terraform/helper/shadow github.com/hashicorp/terraform/helper/signalwrapper github.com/hashicorp/terrafo
rm/helper/slowmessage
ok      github.com/hashicorp/terraform/helper/schema    0.063s
ok      github.com/hashicorp/terraform/helper/shadow    0.156s
ok      github.com/hashicorp/terraform/helper/signalwrapper     0.022s
ok      github.com/hashicorp/terraform/helper/slowmessage       0.102s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/helper/structure github.com/hashicorp/terraform/helper/validation github.com/hashicorp/terraform/helper/variables github.com/hashicorp/terr
aform/helper/wrappedreadline
ok      github.com/hashicorp/terraform/helper/structure 0.004s
ok      github.com/hashicorp/terraform/helper/validation        0.004s
ok      github.com/hashicorp/terraform/helper/variables 0.006s
?       github.com/hashicorp/terraform/helper/wrappedreadline   [no test files]
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/helper/wrappedstreams github.com/hashicorp/terraform/moduledeps github.com/hashicorp/terraform/plugin github.com/hashicorp/terraform/plugin
/discovery
?       github.com/hashicorp/terraform/helper/wrappedstreams    [no test files]
ok      github.com/hashicorp/terraform/moduledeps       0.004s
ok      github.com/hashicorp/terraform/plugin   0.046s
ok      github.com/hashicorp/terraform/plugin/discovery 0.029s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/repl github.com/hashicorp/terraform/scripts github.com/hashicorp/terraform/state github.com/hashicorp/terraform/state/remote
ok      github.com/hashicorp/terraform/repl     0.006s
ok      github.com/hashicorp/terraform/scripts  0.008s
ok      github.com/hashicorp/terraform/state    2.617s
ok      github.com/hashicorp/terraform/state/remote     0.025s
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/terraform github.com/hashicorp/terraform/tools/terraform-bundle
go test -timeout=60s -parallel=4 github.com/hashicorp/terraform/terraform github.com/hashicorp/terraform/tools/terraform-bundle
ok      github.com/hashicorp/terraform/terraform        4.222s
?       github.com/hashicorp/terraform/tools/terraform-bundle   [no test files]
```
2017-08-20 13:53:51 -04:00
James Bardin e6f58f5817 Add upload tests
These are skipped without the SSH_UPLOAD_TEST flag, since they require
connecting to a running ssh server on localhost as the current user, and
one creates a very large file.
2017-02-10 10:30:05 -05:00
James Bardin e1f06e5d0f Skip upload copy if we know the length
If the source length is known, we can skip copying the file.
2017-02-10 10:11:31 -05:00
Mitchell Hashimoto b486354a9c
communicator/ssh: Disconnect() should also kill the actual connection 2017-01-26 15:10:03 -08:00
Mitchell Hashimoto a2e044829b
provisioners/file: use the old communicator.New just to minimize risk 2017-01-26 15:03:28 -08:00
Mitchell Hashimoto 02a4adc07c
provisioners/file: convert to helper/schema 2017-01-26 15:03:28 -08:00
Mitchell Hashimoto 4b1d9cfd7d
communicator/ssh: don't share rand object to guarantee unique values
Fixes #10463

I'm really surprised this flew under the radar for years...

By having unique PRNGs, the SSH communicator could and would
generate identical ScriptPaths and two provisioners running in parallel
could overwrite each other and execute the same script. This would
happen because they're both seeded by the current time which could
potentially be identical if done in parallel...

Instead, we share the rand now so that the sequence is guaranteed
unique. As an extra measure of robustness, we also multiple by the PID
so that we're also protected against two processes at the same time.
2016-12-06 00:21:49 -08:00
James Bardin 8174037a2b Remove deprecated key_file and bastion_key_file
These were deprecated in the 0.6.x series. Their functionality was broken
in a recent release which whould have removed them as well.
2016-10-12 14:30:09 -04:00
Sander van Harmelen 968472a63e Support recreating clients and configuring Chef Vaults (#8577)
Fixes #3605 and adds the functionality suggested in PR #7440.

This PR is using a different appraoch that (IMHO) feels cleaner and (even more important) adds support for Windows at the same time.
2016-09-15 14:20:18 +02:00
Kristinn Örn Sigurðsson bc5518f993 provisioners: Allow provisioning over IPv6 2016-09-03 15:45:24 -07:00
Paul Hinze 2f237eca44
communicator/ssh: correct test typo
Copy pasta!!
2016-06-29 10:59:55 -05:00
Paul Hinze 96c20f0dd7
communicator/{ssh,winrm}: seed random script paths
Without a seed, the "random" script path locations for the remote-exec
provisioner were actually deterministic!

Every rand.Int31() would return the same pseudorandom chain starting w/
the numbers: 1298498081, 2019727887, 1427131847, 939984059, ...

So here we properly seed the communicators so the script paths are
actually random, and multiple runs on a single remote host have much
less chance of clobbering each other.

Fixes #4186

Kudos to @DustinChaloupka for the correct hunch leading to this fix!
2016-06-29 09:32:24 -05:00
James Nugent f2fef2556a Fix import formatting across code base 2016-04-18 17:28:46 -07:00
David Meyer f46a629d72 communicator/winrm: Fixed HTTPS when using copy client. 2016-03-21 18:20:48 +00:00
Paul Hinze 9cbaacad32 communicator/winrm: fix data race in io copy
As the command completes, the winrm.Command does not wait for its copy of
the remote.Command streams to report itself completed.

This adds an additional sync.WaitGroup to ensure that copy is finished up
before moving on, solving the data race.

Fixes the following data race:

```
==================
WARNING: DATA RACE
Read by goroutine 6:
  github.com/hashicorp/terraform/communicator/winrm.TestStart()
      /Users/phinze/go/src/github.com/hashicorp/terraform/communicator/winrm/communicator_test.go:79 +0xa44
  testing.tRunner()
      /private/var/folders/vd/7l9ys5k57l91x63sh28wl_kc0000gn/T/workdir/go/src/testing/testing.go:456 +0xdc

Previous write by goroutine 14:
  bytes.(*Buffer).ReadFrom()
      /private/var/folders/vd/7l9ys5k57l91x63sh28wl_kc0000gn/T/workdir/go/src/bytes/buffer.go:174 +0x465
  io.copyBuffer()
      /private/var/folders/vd/7l9ys5k57l91x63sh28wl_kc0000gn/T/workdir/go/src/io/io.go:375 +0x1a5
  io.Copy()
      /private/var/folders/vd/7l9ys5k57l91x63sh28wl_kc0000gn/T/workdir/go/src/io/io.go:351 +0x78

Goroutine 6 (running) created at:
  testing.RunTests()
      /private/var/folders/vd/7l9ys5k57l91x63sh28wl_kc0000gn/T/workdir/go/src/testing/testing.go:561 +0xaa3
  testing.(*M).Run()
      /private/var/folders/vd/7l9ys5k57l91x63sh28wl_kc0000gn/T/workdir/go/src/testing/testing.go:494 +0xe4
  main.main()
      github.com/hashicorp/terraform/communicator/winrm/_test/_testmain.go:62 +0x20f

Goroutine 14 (finished) created at:
  github.com/hashicorp/terraform/communicator/winrm.runCommand()
      /Users/phinze/go/src/github.com/hashicorp/terraform/communicator/winrm/communicator.go:151 +0xf8
==================
```
2016-01-16 13:33:24 -05:00
Sander van Harmelen c72342eefc Add SSH agent support for Windows
The Windows support is limited to the Pageant SSH authentication agent.
This fixes #3423
2015-12-15 16:39:23 +01:00
Paul Hinze b6aed3fec3 communicator/ssh: fix typos travis didn't catch
I think rebasing and splitting got me into a weird state. This should
fix the build failures on master.
2015-11-12 16:00:28 -06:00
Paul Hinze 7ffa66d1a5 ssh: accept private key contents instead of path
We've been moving away from config fields expecting file paths that
Terraform will load, instead prefering fields that expect file contents,
leaning on `file()` to do loading from a path.

This helps with consistency and also flexibility - since this makes it
easier to shift sensitive files into environment variables.

Here we add a little helper package to manage the transitional period
for these fields where we support both behaviors.

Also included is the first of several fields being shifted over - SSH
private keys in provisioner connection config.

We're moving to new field names so the behavior is more intuitive, so
instead of `key_file` it's `private_key` now.

Additional field shifts will be included in follow up PRs so they can be
reviewed and discussed individually.
2015-11-12 14:59:14 -06:00
Panagiotis Moustafellos e4845f75cc removed extra parentheses 2015-10-08 15:48:04 +03:00
Anthony Scalisi 198e1a5186 remove various typos 2015-09-11 11:56:20 -07:00
Paul Hinze 814d05b96e communicator/ssh: missing newline on bastion host output 2015-07-16 12:16:39 -05:00
Paul Hinze 98891aef96 Merge pull request #2684 from hashicorp/f-print-bastion-host-details
communicator/ssh: print ssh bastion host details to output
2015-07-16 12:13:32 -05:00
Paul Hinze 448a6a1b48 communicator/ssh: print ssh bastion host details to output 2015-07-10 13:02:53 -06:00
Paul Hinze 138018c896 communicator/ssh: agent forward failure is not fatal
On connections where no second hop is made, there's no problem if the
agent forwarding connection is denied, so we shouldn't treat that as a
fatal error.
2015-07-10 12:51:45 -06:00
Paul Hinze 801aaf1eec communicator/ssh: sort agent after static keyfile
In the SSH client configuration, we had SSH Agent authentication listed
before the static PrivateKey loaded from the `key_file` setting.
Switching the default of the `agent` setting exposed the fact that the
SSH agent overrides the `key_file` during the handshake. By listing the
`key_file` first, we catch the provided key before any query goes out to
the agent.

Adds a key-based authentication SSH test to cover this new behavior. It
fails without the reordering on any machine with an SSH agent running.

Fixes #2614
2015-07-02 15:41:23 -05:00
Paul Hinze a7cbbbd258 communicator/ssh: bastion host support
* adds `bastion_*` fields to `connection` which add configuration for a
   bastion host
 * if `bastion_host` is set, connect to that host first, then jump
   through it to make the SSH connection to `host`
 * enables SSH Agent forwarding by default
2015-06-22 18:00:18 -05:00
Paul Hinze 44de0996c5 communicator/ssh: switch agent default to true
This changes SSH Agent utilization from opt-in to opt-out, bringing
Terraform in line with the behavior of Packer and the `ssh` command
itself.

But skip SSH agent connections if the SSH_AUTH_SOCK env var doesn't exist,
whcih means there's no agent to connect to.
2015-06-19 17:14:43 -05:00
Sander van Harmelen f146a31ded Fixing #2033 by adding a small check on the input
If the input begins with a shebang, we’ll leave it as is. If not we’ll
add a default shebang…
2015-05-21 23:36:54 +02:00
Sander van Harmelen a8dacede24 Improved some logging...
I added a debug log line in the last commit, only to find out it’s now
logging the same info twice. So removed the double entry and tweaked
the existing once.
2015-05-11 15:18:32 +02:00
Sander van Harmelen d4150d5b1a Adding the tests... 2015-05-08 18:17:57 +02:00
Sander van Harmelen 3881c81a66 Added the exit code as well... 2015-05-05 21:56:20 +02:00
Sander van Harmelen 48f4d499f9 Fixing a small vet warning introduced by PR #1796
While fixing the vet warning also tried to improve any feedback by
showing the actual output/error instead of just the error code.

While testing this I noticed only adding stderr output is not enough as
not all error info is send to stderr, but sometimes also to stdout.
2015-05-05 21:26:28 +02:00
Luke Amdor fbae7884eb remote-exec (ssh): chmod'ing right path 2015-05-04 15:44:44 -05:00
Luke Amdor 11b4060fee remote-exec (ssh): checking chmod command 2015-05-04 15:44:39 -05:00
Sander van Harmelen e55169b39b Typo... 2015-05-01 22:28:12 +02:00
Sander van Harmelen 7f408cf8aa Adding an import needed for the tests
This will likely be a temp fix until `make updated eps` is made a
little smarter by @phinze 😉
2015-05-01 22:26:11 +02:00