Commit Graph

20422 Commits

Author SHA1 Message Date
James Bardin 8cf0a8ca9c faster DAG transitive reduction
In the case of highly-connected graphs, the TransitiveReduction process
was far too computationally intensive. Since no operations are applied
to the nodes, and the walk order is not even user visible, we don't need
to sort them n^2 times.
2017-10-03 11:24:59 -04:00
James Bardin d477d1f6d4 Merge pull request #15553 from bonifaido/custom_s3_backend
Allow non-AWS S3 backends
2017-10-02 19:39:48 -04:00
James Bardin 67be68b521 Merge pull request #16242 from hashicorp/jbardin/race
minor race fix in tests
2017-10-02 16:50:02 -04:00
James Bardin 08ac4f3a2e Merge pull request #16089 from wgliang/optimization-merge-slice
Optimization merge slice
2017-10-02 16:40:13 -04:00
James Bardin 3662aac93f Merge pull request #16204 from hashicorp/jbardin/outputs
make outputs error
2017-10-02 16:37:36 -04:00
James Bardin e0ee1e4d6e Merge pull request #16070 from octo/defaultdatadir
backend/local: Remove unused const DefaultDataDir.
2017-10-02 16:27:23 -04:00
James Bardin 3ed7b1abec fix the providerWithLocals test
This test was set to fail once this issue was fixed, and now it's fixed.
2017-10-02 16:21:37 -04:00
James Bardin 3c2b3df5a9 add some output error tests
Add some tests for output errors and catch the errors behind the output
errors feature flag.
2017-10-02 16:20:29 -04:00
James Bardin 715036d209 put output errors behind a feature flag
We're going to start merging breaking functgionality behind feature
flags, to reduce the need for long-lived feature branches.
2017-10-02 16:20:29 -04:00
James Bardin 35c6a4e89d add DestroyValueReferenceTransformer
DestroyValueReferenceTransformer is used during destroy to reverse the
edges for output and local values. Because destruction is going to
remove these from the state, nodes that depend on their value need to be
visited first.
2017-10-02 16:20:29 -04:00
James Bardin 3414e1b996 remove dead code 2017-10-02 16:20:29 -04:00
James Bardin 7fde7d9f20 make a destroy plan work the same as destroy
When working on an existing plan, the context always used walkApply,
even if the plan was for a full destroy. Mark in the plan if it was
icreated for a destroy, and transfer that to the context when reading
the plan.
2017-10-02 16:20:29 -04:00
James Bardin 78964d305c remove broken outputs from targeted graph
A Targeted graph may include outputs that were transitively included,
but if they are missing any dependencies they will fail to interpolate
later on.

Prune any outputs in the TargetsTransformer that have missing
dependencies, and are not depended on by any resource. This will
maintain the existing behavior of outputs failing silently ni most
cases, but allow errors to be surfaced where the output value is
required.
2017-10-02 16:20:29 -04:00
James Bardin a048bcffa0 continue on output errors during Input
Module outputs may not have complete information during Input, because
it happens before refresh. Continue process on output interpolation
errors during the Input walk.
2017-10-02 16:20:29 -04:00
James Bardin 1ad97f6be8 use an EvalOpFilter for module variables
Remove the Input flag threaded through the input graph creation process
to prevent interpolation failures on module variables.
Use an EvalOpFilter instead to inset the correct EvalNode during
walkInput. Remove the EvalTryInterpolate type, and use the same
ContinueOnErr flag as the output node for consistency and to try and
keep the number possible eval node types down.
2017-10-02 16:20:29 -04:00
James Bardin c4dd31e62b "destroy" outputs too
Ouptuts don't need to be re-evaluated during destroy, since everything
is already in the state, so we can simply remove them.
2017-10-02 16:20:29 -04:00
James Bardin fc9ada463c minor race fix in tests 2017-10-02 16:19:01 -04:00
James Bardin 71e47e56a0 release: clean up after v0.10.7 2017-10-02 18:54:33 +00:00
James Bardin 8ba8fc79c1
v0.10.7 2017-10-02 18:46:49 +00:00
James Bardin 5f9bf20ba6 update CHANGELOG.md 2017-10-02 14:39:47 -04:00
James Bardin d367afe6d4 Merge pull request #16233 from alex1x/master
Fixes #16226 - two changes to the Provisioners sidebar on the website
2017-10-02 13:44:58 -04:00
Alex Tasioulis 4e6289dad0 website: change order of provisioners in sidebar list
Assume null_resource should also be ordered alphabetically
2017-10-01 22:03:39 +01:00
Alex Tasioulis 825a098c56 website: add salt-masterless provisioner to sidebar
Fixes #16226
2017-10-01 22:02:43 +01:00
James Bardin cef16676f7 Merge pull request #16219 from hashicorp/jbardin/local-destroy
Improve the test to show a failure when using locals in a provider
2017-09-29 17:36:04 -04:00
James Bardin 4694f88896 Merge pull request #16161 from hashicorp/jbardin/go-getter
update go-getter
2017-09-29 17:24:36 -04:00
James Bardin da0e74276a fix provider with local value test and docs
Make sure this fails during destroy

Add a note in the documentation that local values may fail during
destroy.
2017-09-29 17:14:07 -04:00
Martin Atkins 4e04e4cfdd Update CHANGELOG.md 2017-09-29 14:06:13 -07:00
Martin Atkins 34956cd124 website: Document the plugin cache
Since we don't currently auto-install provisioner plugins this is
currently placed on the providers documentation page and referred to as
the "Provider Plugin Cache". In future this mechanism may also apply to
provisioners, in which case we'll figure out at that point where better
to place this information so it can be referenced from both the provider
and provisioner documentation pages.
2017-09-29 14:03:09 -07:00
Martin Atkins f19bf25945 website: don't recommend "providers" block in CLI config
This mechanism for configuring plugins is now deprecated, since it's not
capable of declaring plugin versions. Instead, we recommend just placing
plugins into a particular directory, which is now documented on the
main providers documentation page and linked from the more detailed docs
on plugins in general.
2017-09-29 14:03:09 -07:00
Martin Atkins adbb9d0cf9 website: checkpoint settings refer to CLI config file docs
Previously we described inline here where to put the .terraformrc file,
but now we have a separate page all about this file that gives us more
room to describe in more detail where the file is placed and what else it
can do.
2017-09-29 14:03:09 -07:00
Martin Atkins 35959b8689 website: document explicitly the CLI configuration file
Previously this file was mentioned in passing in a few locations. It now
has enough functionality to warrant its own page.
2017-09-29 14:03:09 -07:00
Martin Atkins 30e6f4e066 command/e2etest: test installation of plugins from cache
This is a tough one to unit tests because the behavior is tangled up in
the code that hits releases.hashicorp.com, so we'll add this e2etest as
some extra insurance that this works end-to-end.
2017-09-29 14:03:09 -07:00
Martin Atkins cb17a9a607 main: allow enabling plugin caching via config file or environment
Either the environment variable TF_PLUGIN_CACHE_DIR or a setting in the
CLI config file (~/.terraformrc on Unix) allow opting in to the plugin
caching behavior.

This is opt-in because for new users we don't want to pollute their system
with extra directories they don't know about. By opting in to caching, the
user is assuming the responsibility to occasionally prune the cache over
time as older plugins become stale and unused.
2017-09-29 14:03:09 -07:00
Martin Atkins 5c0670fdf4 command: allow callers to activate a plugin cache
Here we add a new caller-settable field to command.Meta that activates
a read-through cache directory for plugin installation.
2017-09-29 14:03:09 -07:00
Martin Atkins 879899d434 plugin/discovery: plugin caching mechanism
For users that have metered or slow internet connections it is annoying
to have Terraform constantly re-downloading the same files when they
initialize many separate directories.

To help such users, here we add an opt-in mechanism to use a local
directory as a read-through cache. When enabled, any plugin download will
be skipped if a suitable file already exists in the cache directory. If
the desired plugin isn't in the cache, it will be downloaded into the
cache for use next time.

This mechanism also serves to reduce total disk usage by allowing
plugin files to be shared between many configurations, as long as the
target system isn't Windows and supports either hardlinks or symlinks.
2017-09-29 14:03:09 -07:00
Martin Atkins 12d6bc8c30 plugin/discovery: ignore non-files when discovering
If we encounter something that isn't a file -- for example, a dangling
symlink whose referent has been deleted -- we'll ignore it so that we
can either later produce a "no such plugin" error or auto-install a plugin
that will actually work.
2017-09-29 14:03:09 -07:00
Martin Atkins 3f401f0cd4 main: make configuration available when initializing commands
This, in principle, allows us to make use of configuration information
when we populate the Meta structure, though we won't actually make use
of that until a subsequent commit.
2017-09-29 14:03:09 -07:00
James Bardin 60d8b6c4d7 Merge pull request #16214 from hashicorp/jbardin/registry-url
remove extra slash from registry urls
2017-09-28 17:52:34 -04:00
Martin Atkins f45aae9b70 build: run end-to-end tests during Travis run
We don't usually run "acceptance tests" during a Travis run, but this
particular suite doesn't require any special credentials since it just
accesses releases.hashicorp.com to download plugins, so therefore it's
safe to run in Travis at the expense of adding a few more seconds to
the runtime.

Running it in Travis can therefore give us some extra confidence for
pull requests that may inadvertently break certain details of the
workflow, as well as ensuring that these tests are kept up-to-date as
the system changes.
2017-09-28 14:35:51 -07:00
Martin Atkins 73d1298572 command/e2etest: test the "running in automation" workflow
Since we now have a guide that recommends some specific ways to run
Terraform in automation, we can mimic those suggestions in an e2e test and
thus ensure they keep working.

Here we test the three different approaches suggested in the guide:
- init, plan, apply (main case)
- init, apply (e.g. for deploying to a QA/staging environment)
- init, plan (e.g. for verifying a pull request)
2017-09-28 14:35:51 -07:00
Martin Atkins 0abedd8877 e2e: allow tests to set environment variables for command runs 2017-09-28 14:35:51 -07:00
Martin Atkins cb6d4e5f20 command/e2etest: fix TestPrimarySeparatePlan test
In 6712192724 we stopped counting data
source destroys in the destroy tally since they are an implementation
detail.

This caused this test to start failing, though since the new behavior is
correct here we just update the test to match.
2017-09-28 14:35:51 -07:00
James Bardin feb3883568 Merge pull request #16213 from hashicorp/jbardin/local-destroy
Don't evaluate locals during destroy
2017-09-28 16:13:31 -04:00
Martin Atkins c5d8f2ffd2 website: clarify that provider version does not accept interpolations
This fixes #15896.
2017-09-28 12:41:39 -07:00
James Bardin 061597304c add test for destroying with locals in a provider
Verify that locals aren't removed from the state before providers are
evaluated during destroy.
2017-09-28 15:31:41 -04:00
James Bardin 1fbb2d611a remove extra slash from registry urls
A refactor introduced an extra `/` in the download url, which causes an
extra redirect during discovery.

Improve a registry test to verify that detection doesn't require the
registry after the modules have been fetched.
2017-09-28 14:45:02 -04:00
Martin Atkins 7a2af23d25 Update CHANGELOG.md 2017-09-28 10:30:32 -07:00
Ariel Alonso 550ae05819 config: new "transpose" interpolation function
This function takes a map of lists of strings and inverts it so that
the string values become keys and the keys become items within the
corresponding lists.
2017-09-28 10:29:25 -07:00
James Bardin 77396107c4 don't evaluate locals during destroy
Locals don't need to be evaluated during destroy.  Rather than simply
skipping them, remove them from the state as they are encountered. Even
though they are not persisted in the state, it keeps the state up to
date as the destroy happens, and we reduce the chance of other
inconstancies later on.
2017-09-28 12:56:25 -04:00
James Bardin 9d8ab55658 Add failing test for destroy with locals 2017-09-28 11:06:37 -04:00