Commit Graph

21364 Commits

Author SHA1 Message Date
James Bardin d3f7edeb27 add backend.Operation.StateLocker
Add the StateLocker field so that the state lock can be carried between
the different function scopes of the backend code. This will allow the
backend context to lock the state before it's read, while allowing the
different operations to unlock the state when they complete.
2018-02-23 11:22:28 -05:00
James Bardin e88bd74bb7 create clistate.Locker interface
Simplify the use of clistate.Lock by creating a clistate.Locker
instance, which stores the context of locking a state, to allow unlock
to be called without knowledge of how the state was locked.

This alows the backend code to bring the needed UI methods to the point
where the state is locked, and still unlock the state from an outer
scope.
2018-02-23 11:21:21 -05:00
James Bardin 7c6072c2a0
Merge pull request #17397 from hashicorp/jbardin/gcs-lock-info
Fix reported GCS state lock ID
2018-02-21 16:06:03 -05:00
James Bardin e5392a194e
Merge pull request #17408 from hashicorp/jbardin/makefile
finish removing vet from the makefile
2018-02-21 16:05:14 -05:00
James Bardin ec339a228e update CHANGELOG.md 2018-02-21 15:36:29 -05:00
James Bardin ec62819b3a finish removing vet from the makefile
The test target will already cover vet, since it's run as part of the
test command now.

Also remove the `go test -i` since it's no longer needed with the new
build cache.
2018-02-21 15:30:29 -05:00
James Bardin 6ebb7aab56
Merge pull request #17401 from hashicorp/jbardin/go1.10
update to go1.10
2018-02-21 15:27:39 -05:00
Michael Sambol abac457033 website: Fix typo in Getting Started guide 2018-02-21 11:42:22 -08:00
James Bardin d4415bdd7d try travis go version as string 2018-02-21 11:37:06 -05:00
James Bardin 3bc1dcf21a update travis to build with go1.10
Remove the vet target too, since vet is now always run with every test.
2018-02-21 11:27:46 -05:00
James Bardin a37acb1837 gofmt with go1.10 2018-02-21 10:22:08 -05:00
James Bardin 8242c773b8 missed the local state backend 2018-02-20 22:09:54 -05:00
James Bardin 57cbcbe99d update remote state tests to use new test helpers 2018-02-20 21:05:58 -05:00
James Bardin 112c37458f report the proper lock id from a state lock error
Fix the now failing state unlock test by reporting the correct ID.

The ID used by GCS is the generation number of the info object, which
isn't known until the info is already written out. While we can't get
the correct ID from the info data for the error rmessage, we can update
it with the generation number after it's read.
2018-02-20 20:54:01 -05:00
James Bardin 1fee5ae172 update gcs to use the new backend tests 2018-02-20 20:36:08 -05:00
James Bardin 85d6b1d9cc add test for lock error and force-unlock
This adds a general test to verify that a remote state backend returns
the expected error type when it cannot lock a state. It then extracts
the ID reported in the error, and attempts to unlock the state using
that ID, which simulated the force-unlock scenario. This is a separate
test, since not all backends have persistent locks that can be unlocked
later.

We also split out the backend test to be called individually as needed.
2018-02-20 20:32:07 -05:00
James Bardin 1d0f5fdac9 update CHANGELOG.md 2018-02-16 14:43:16 -05:00
Stuart Auld 7d1f594f54 helper/resource: don't panic if a requested module does not exist 2018-02-16 11:32:49 -08:00
chrisjob1021 10bb21e9c4 provisioner/local-exec: Optional "working_dir" argument
This new argument allows overriding of the working directory of the child process, with the default still being the working directory of Terraform itself.
2018-02-16 11:31:11 -08:00
Stuart Auld df9446f490 helper/resource: allow tests to opt out of temporary file cleanup
This is rarely needed, but sometimes tests need to create temporary files as part of their operation. This should be used sparingly, since it prevents the pro-active cleanup of the temporary working directory.
2018-02-16 11:24:01 -08:00
Martin Atkins 4f833d5682
Merge #17358: HCL2-powered configuration loader (not yet used) 2018-02-16 11:00:20 -08:00
Martin Atkins 4fa8c16ead configs: support ignore_changes wildcards
The initial pass of implementation here missed the special case where
ignore_changes can, in the old parser, be set to ["*"] to ignore changes
to all attributes.

Since that syntax is awkward and non-obvious, our new decoder will instead
expect ignore_changes = all, using HCL2's capability to interpret an
expression as a literal keyword. For compatibility with old configurations
we will still accept the ["*"] form but emit a deprecation warning to
encourage moving to the new form.
2018-02-15 15:56:39 -08:00
Martin Atkins c5f5340b15 configs: Additional guidance in doc.go
There's quite a lot in this package, so hopefully this additional
paragraph will help readers get oriented.
2018-02-15 15:56:39 -08:00
Martin Atkins 36fb5b52e7 configs: quoted keywords/references are warnings, not errors
In our new loader we are changing certain values in configuration to be
naked keywords or references rather than quoted strings as before. Since
many of these have been shown in books, tutorials, and our own
documentation we will make the old forms generate deprecation warnings
rather than errors so that newcomers starting from older documentation
can be eased into the new syntax, rather than getting blocked.

This will also avoid creating a hard compatibility wall for reusable
modules that are already published, allowing them to still be used in
spite of these warnings and then fixed when the maintainer is able.
2018-02-15 15:56:39 -08:00
Martin Atkins ea868026b8 configs/configload: installer tests inspect their result
Previously we were just loading the module and asserting no diagnostics,
but that is not really good enough since if we install modules incorrectly
it's possible that we are still able to load an empty configuration
successfully.

Now we'll do some basic inspecetion of the module tree that results from
loading what we installed, to ensure that all of the expected modules
are present at the right locations in the tree.
2018-02-15 15:56:39 -08:00
Martin Atkins d859bacbdd configs/configload: InitDirFromModule
This will provide the functionality of "terraform init -from-module=...",
which uses the contents of a given module to populate the working
directory.

This mechanism is intended for installing e.g. examples from Terraform
Registry or elsewhere. It's not fully-general since it can't reasonably
install a module from a subdir that refers up to a parent directory, but
that isn't an issue for all reasonable uses of this option.
2018-02-15 15:56:39 -08:00
Martin Atkins 74afcb4a7f configs/configload: some loaders can't install modules
Originally the hope was to use the afero filesystem abstraction for all
loader operations, but since we install modules using go-getter we cannot
(without a lot of refactoring) support vfs for installation.

The vfs use-case is for reading configuration from plan zip files anyway,
and so we have no real reason to support installation into a vfs. For now
at least we will just add the possibility that a loader might not be
install-capable. At the moment we have no non-install-capable loaders, but
we'll add one later once we get to loading configuration from plan files.
2018-02-15 15:56:39 -08:00
Martin Atkins 59939cf320 configs/configload: installation from registry and go-getter
Unlike the old installer in config/module, this uses new-style
installation directories that include the static module path so that paths
we show in diagnostics will be more meaningful to the user.

As before, we retrieve the entire "package" associated with the given
source string, rather than any given subdirectory directly, because the
retrieved module may contain ../ references into parent directories which
must be resolvable after extraction.
2018-02-15 15:56:39 -08:00
Martin Atkins 3d551e25e0 configs: BuildConfig sorts child modules by name
This is not strictly necessary, but since this is not a
performance-critical codepath we'll do this because it makes life easier
for callers that want to print out user-facing logs about build process,
or who are logging actions taken as part of a unit test.
2018-02-15 15:56:38 -08:00
Martin Atkins 51e5f7a56b govendor fetch github.com/go-test/deep
This will be used to produce readable diffs for deep data structures in
unit tests.
2018-02-15 15:56:38 -08:00
Martin Atkins 7feef98517 configs/configload: installation of local modules
Enough of the InstallModules method to install local modules (those with
relative paths). "Install" is actually a bit of an exaggeration for these
since we actually just record them in our manifest after verifying that
the source directory exists.

This is a change of behavior relative to the old module installer since
we no longer create a symlink to the module directory inside the
.terraform/modules directory. Instead, we record the module's true
location in our manifest so that the loader will find it later.

The use of a symlink here predated the manifest file. Now that we have a
manifest file the symlinks are redundant. Using the "natural" location of
the module leads to more helpful error messages, since we'll refer to
the module path as the user expects it, rather than to an internal alias.
2018-02-15 15:56:38 -08:00
Martin Atkins 72ad927c4d configs/configload: package for loading configurations
Previously the behavior for loading and installing modules was included in
the same package as the representation of the module tree (in the
config/module package).

In our new world, the model of a module tree (now called a "Config") is
included in "configs" along with the Module and File structs. This new
package replaces the loading and installation functionality previously
in config/module with new equivalents that work with the model objects
in "configs".

As of this commit, only the loading functionality is implemented. The
installation functionality will follow in subsequent commits.
2018-02-15 15:56:38 -08:00
Martin Atkins 9153bb448e configs: include the module call source range in our module tree 2018-02-15 15:56:38 -08:00
Martin Atkins 8929eca405 configs: BuildConfig function
BuildConfig creates a module tree by recursively walking through module
calls in the root module and any descendent modules. This is intended to
be used both for the simple case of loading already-installed modules and
the more complex case of installing modules inside "terraform init", both
of which will be dealt with in a separate package.
2018-02-15 15:56:38 -08:00
Martin Atkins cc38e91612 configs: Implementation of mergeBody
mergeBody is a hcl.Body implementation that deals with our override file
merging behavior for the portions of the configuration that are not
processed until full eval time.

Mimicking the behavior of our old config merge implementation from the
"config" package, the rules here are:

- Attributes in the override body hide attributes of the same name in
  the base body.
- Any block in the override body hides all blocks with the same type name
  that appear in the base body.

This is tested by a new test for the overriding of module arguments, which
asserts the correct behavior of the merged body as part of its work.
2018-02-15 15:56:38 -08:00
Martin Atkins 7c8efe103e configs: allow overrides files to omit args that primary files can't
Some of the fields in our config structs are either mandatory in primary
files or there is a default value that we apply if absent.

Unfortunately override files impose the additional constraint that we
be allowed to omit required fields (which have presumably already been
set in the primary files) and that we are able to distinguish between a
default value and omitting a value entirely.

Since most of our fields were already acceptable for override files, here
we just add some new fields to deal with the few cases where special
handling is required and a helper function to disable the "Required" flag
on attributes in a given schema.
2018-02-15 15:56:38 -08:00
Martin Atkins 4e5efa498a configs: Parser.LoadConfigDir
This method wraps LoadConfigFile to load all of the .tf and .tf.json files
in a given directory and then bundle them together into a Module object.

This function also deals with the distinction between primary and override
files, first appending together the primary files in lexicographic order
by filename, and then merging in override files in the same order.

The merging behavior is not fully implemented as of this commit, and so
will be expanded in future commits.
2018-02-15 15:56:37 -08:00
Martin Atkins 9be399d49c configs: Another simple test for _invalid_ config files
Much like TestParserLoadConfigFileSuccess, this is intended to be an
easy-to-maintain collection of bad examples to test different permutations
of our error handling.

As with TestParserLoadConfigFileSuccess, we should also have more specific
tests alongside this that check that the error outcome is what was
expected, since this test just accepts any error and may thus not be
testing what we think it is.
2018-02-15 15:56:37 -08:00
Martin Atkins e524d1eb95 configs: Simple test of loading valid configuration files
This test is intended to be an easy-to-maintain catalog of good examples
that we can use to catch certain parsing or decoding regressions easily.

It's not a fully-comprehensive test since it doesn't check the result
of decoding, instead just accepting any decode that completes without
errors. However, an easy-to-maintain test like this is a good complement
to some more specialized tests since we can easily collect good examples
over time and just add them in here.
2018-02-15 15:56:37 -08:00
Martin Atkins e15ec486bf configs: Parser.LoadConfigFile
This is a first pass of decoding of the main Terraform configuration file
format. It hasn't yet been tested with any real-world configurations, so
it will need to be revised further as we test it more thoroughly.
2018-02-15 15:56:37 -08:00
Martin Atkins b865d62bb8 govendor fetch github.com/hashicorp/hcl2/...
This just catches us up with the latest fixes and improvements in upstream
HCL2.
2018-02-15 15:56:37 -08:00
Martin Atkins 13fa73c63e configs: stub out main configuration structs
These types represent the individual elements within configuration, the
modules a configuration is made of, and the configuration (static module
tree) itself.
2018-02-15 15:56:37 -08:00
Martin Atkins 05e3b47ba1 configs: Parser.LoadValuesFile
This method loads a "values file" -- also known as a "tfvars file" -- and
returns the values found inside.

A values file is an HCL file (in either native or JSON syntax) whose
top-level body is treated as a set of arbitrary key/value pairs whose
values may not depend on any variables or functions.

We will load values files through a configs.Parser -- even though values
files are not strictly-speaking part of configuration -- because this
causes them to be registered in our source code cache so that we can
generate source code snippets if we need to report any diagnostics.
2018-02-15 15:56:37 -08:00
Martin Atkins a0f4a313ef configs: Parser type
configs.Parser is the entry-point for this package, providing functions to
load and parse HCL-based configuration files.

We use the library "afero" to decouple the parser from the physical OS
filesystem, which here allows us to easily use an in-memory filesystem
for testing and will, in future, allow us to read files from more unusual
places, such as configuration embedded in a plan file.
2018-02-15 15:56:36 -08:00
James Bardin e41b29d096
Merge pull request #17354 from hashicorp/jbardin/known_hosts
Verify host keys in ssh connections
2018-02-15 18:33:41 -05:00
James Bardin 162389b82b
Merge pull request #17362 from hashicorp/jbardin/etcd
update etcd documentation links
2018-02-15 18:11:52 -05:00
Martin Atkins 7987a2fdb2 configs: new package for HCL2-based configuration
There's a lot of complexity in our existing "config" package that results
from our approach to handling configuration with HCL and HIL. A lot of
that functionality is no longer needed -- or must work in a significantly
different way -- for HCL2.

The new package "configs", which is named following the convention of some
Go standard library packages like "strings", is a re-imagination of some
of the functionality from the "config" package for an HCL2-only world.

The scope of this package will be slightly smaller than "config", since
it only deals with config loading and not with expression evaluation.
Another package "lang" (mentioned in the docstring here but not yet added)
will deal with the more dynamic portions of of configuration handling,
including populating an hcl.EvalContext to evaluate expressions.
2018-02-15 15:10:01 -08:00
Martin Atkins 976d85ae92 govendor fetch github.com/spf13/afero/... 2018-02-15 15:10:01 -08:00
James Bardin 1c4f403386
Merge pull request #17359 from hashicorp/jbardin/provisioner-error
Halt on provisioner errors
2018-02-15 18:09:32 -05:00
James Bardin 5458ea9aff update etcd documentation links
Add missing link to ectdv3.

Update etcd v2 link to the current v2 README, which highlights the pending
deprecation.
2018-02-15 17:13:24 -05:00