Commit Graph

26355 Commits

Author SHA1 Message Date
Martin Atkins 49e2e00231 command: terraform providers mirror
This new command is intended to make it easy to create or update a mirror
directory containing suitable providers for the current configuration,
producing a layout that is appropriate both for a filesystem mirror or,
if copied into the document root of an HTTP server, a network mirror.

This initial version is not customizable aside from being able to select
multiple platforms to install packages for.

Future iterations of this could include commands to turn the JSON index
generation on and off, or to instruct it to produce the unpacked directory
layout instead of the packed directory layout as it currently does. Both
of those options would make the generated directory unsuitable to be
a network mirror, but it would still work as a filesystem mirror.

In the long run this will hopefully form part of a replacement workflow to
terraform-bundle as a way to put copies of providers somewhere so we don't
need to re-download them every time, but some other changes will be needed
outside of just this command before that'd be true, such as adding support
for network and/or filesystem mirrors in Terraform Enterprise.
2020-06-01 14:49:43 -07:00
Martin Atkins 85af77386c internal/getproviders: PackageFilePathForPackage
This is the equivalent of UnpackedDirectoryPathForPackage when working
with the packed directory layout. It returns a path to a .zip file with
a name that would be detected by SearchLocalDirectory as a
PackageLocalArchive package.
2020-06-01 14:49:43 -07:00
Martin Atkins 9489672d54 internal/getproviders: Package hashing for local filesystem packages
We previously had this functionality available for cached packages in the
providercache package. This moves the main implementation of this over
to the getproviders package and then implements it also for PackageMeta,
allowing us to compute hashes in a consistent way across both of our
representations of a provider package.

The new methods on PackageMeta will only be effective for packages in the
local filesystem because we need direct access to the contents in order
to produce the hash. Hopefully in future the registry protocol will be
able to also provide hashes using this content-based (rather than
archive-based) algorithm and then we'll be able to make this work for
PackageMeta referring to a package obtained from a registry too, but
hashes for local packages only are still useful for some cases right now,
such as generating mirror directories in the "terraform providers mirror"
command.
2020-06-01 14:49:43 -07:00
Martin Atkins 67311f73fd website: Draw better attention to for and for_each patterns
When helping folks in the community forum, I commonly see questions around
more complex patterns in transforming deep data structures into different
shapes to work with for_each. We have examples of these patterns in the
docs for the functions that they rely on, but they were not previously
very discoverable in the main configuration language documentation
sections.

Here I've moved the "Using Expressions in for_each" subsection on the
Resources page above some of the other sub-sections to hopefully make it
easier to see, and written out in more detail the two specific patterns
that answer a significant number of for_each-related user questions in
the hope that readers will be more likely to realize that the links are
relevant to what their goals.

I also added some more elaboration about the behavior of converting from
list to set in the "Using Sets" subsection, because this feature is often
a user's first encounter with the set data type and I've inferred from
some of the questions I've answered that a number of Terraform users don't
have prior experience with set data types in other languages to draw
assumptions from.

Finally, I added some similar links to the for_each patterns within the
for expression documentation itself, to try to make those examples more
visible to those who might be discovering the documentation in a different
sequence, e.g. by following a deep link shared in an answer to a question
in the community forum.
2020-06-01 14:17:47 -07:00
Alisdair McDiarmid ded30b6836
Merge pull request #25103 from hashicorp/alisdair/fix-012upgrade-stub
command: Fix 0.12upgrade stub
2020-06-01 16:18:12 -04:00
Alisdair McDiarmid 5e2b11657e command: Fix 0.12upgrade stub 2020-06-01 16:12:30 -04:00
Kristin Laemmert f86fb6161b
Merge pull request #25087 from WillBrock/wb-provider-typo
website/docs: Fix doc typo replacing "and" with "as"
2020-06-01 12:57:51 -04:00
heroku-miraheze 8b70fbca6c
Add CloudSigma Provider 2020-05-31 10:15:54 +02:00
Will Brock b4dd2b8b2e
Fix typo replacing and with as 2020-05-30 18:10:51 -04:00
James Bardin 866449a78a
Merge pull request #25069 from hashicorp/jbardin/depends-on-docs
depends_on doc updates
2020-05-29 16:50:39 -04:00
Pam Selle 5883099ce5
Update CHANGELOG.md 2020-05-29 15:34:13 -04:00
Pam Selle 195b59e5f2
Merge pull request #25078 from hashicorp/pselle/jsonencode
[vendor] upgrade go-cty
2020-05-29 15:32:45 -04:00
Pam Selle 5e8b60ba07 Upgrade go-cty for jsonencode fix, closes #23062 2020-05-29 13:00:12 -04:00
James Bardin 3046b790b1
Merge pull request #25018 from hashicorp/jbardin/destroy-modules
Destroy nested, expanding modules
2020-05-29 12:44:13 -04:00
Kristin Laemmert 020084f6d0 update e2etests for windows compatibility 2020-05-29 11:57:50 -04:00
Alisdair McDiarmid 83482770c4
Merge pull request #25055 from hashicorp/alisdair/detected-legacy-provider-on-init
command/init: Better diagnostics for provider 404s
2020-05-29 11:57:18 -04:00
Martin Atkins fd26df0bc4
Update CHANGELOG.md 2020-05-29 07:42:46 -07:00
Martin Atkins 89c2a61b41 website: Small corrections to the "plan" and "apply" command docs
The "apply" documentation contained a simple typo, while the "plan"
documentation contained outdated information about using
"terraform plan PLANFILE" to view a plan. The latter is now a separate
command entirely, since Terraform 0.12: "terraform show PLANFILE".
2020-05-29 07:36:40 -07:00
Martin Atkins 31a4b44d2e backend/local: treat output changes as side-effects to be applied
This is a baby-step towards an intended future where all Terraform actions
which have side-effects in either remote objects or the Terraform state
can go through the plan+apply workflow.

This initial change is focused only on allowing plan+apply for changes to
root module output values, so that these can be written into a new state
snapshot (for consumption by terraform_remote_state elsewhere) without
having to go outside of the primary workflow by running
"terraform refresh".

This is also better than "terraform refresh" because it gives an
opportunity to review the proposed changes before applying them, as we're
accustomed to with resource changes.

The downside here is that Terraform Core was not designed to produce
accurate changesets for root module outputs. Although we added a place for
it in the plan model in Terraform 0.12, Terraform Core currently produces
inaccurate changesets there which don't properly track the prior values.

We're planning to rework Terraform Core's evaluation approach in a
forthcoming release so it would itself be able to distinguish between the
prior state and the planned new state to produce an accurate changeset,
but this commit introduces a temporary stop-gap solution of implementing
the logic up in the local backend code, where we can freeze a snapshot of
the prior state before we take any other actions and then use that to
produce an accurate output changeset to decide whether the plan has
externally-visible side-effects and render any changes to output values.

This temporary approach should be replaced by a more appropriately-placed
solution in Terraform Core in a release, which should then allow further
behaviors in similar vein, such as user-visible drift detection for
resource instances.
2020-05-29 07:36:40 -07:00
James Bardin 1bbb59fc1b remove extra whitespace 2020-05-29 10:31:10 -04:00
James Bardin acd3dca567 remove unused closer field 2020-05-29 10:29:27 -04:00
James Bardin 3dbd6ea3a9 remove stale comment 2020-05-29 10:18:33 -04:00
Alisdair McDiarmid cdfc2e9abd
Merge pull request #25065 from hashicorp/alisdair/remove-dead-code-plugin-discovery
plugin/discovery: Remove dead code
2020-05-29 10:18:15 -04:00
Chris Carpenter 8f4f181151
Update taint.html.markdown
Single qoute cause an `Invalid character` error.
2020-05-29 08:13:02 -05:00
James Bardin 46d27fe848 update CHANGELOG.md 2020-05-28 22:34:13 -04:00
James Bardin 9cb7851b02 update CHANGELOG.md 2020-05-28 22:33:21 -04:00
James Bardin 74ae34865d re-add test lost in merge 2020-05-28 21:41:15 -04:00
James Bardin b860b40127 re-enable depend_on test 2020-05-28 21:35:05 -04:00
James Bardin 10e8529471 add node pruning log line 2020-05-28 21:34:01 -04:00
James Bardin 75ec201e6a temporarily disable module depends_on test 2020-05-28 21:33:56 -04:00
James Bardin d97a210ad3 don't connect destroyers to module expanders
Resource destroy nodes can only depend on other resources. Connecting
them to their module expander can introduce cycles when the module
expander depends on resources in the destroyer's subgraph.
2020-05-28 21:33:08 -04:00
James Bardin a9e0f46f23 expanded module depends_on test 2020-05-28 21:33:05 -04:00
James Bardin 70ac00595b fix apply tests
sSme apply tests had outputs in empty modules, which won't be saved to
state.
2020-05-28 21:30:44 -04:00
James Bardin 48757bcbe0 get rid of the NodeOutputOrphan
We don't need another node type for orphaned outptus, they are just
outputs being removed for a different reason than destroy. Use the
NodeDestroyableOutput implementation.

Destroy outputs also don't need to be referencers, since they are being
removed.

Rename DestroyOutputTransformer to destroyRootOutputTransformer, and add
an explanation as to why it is the only transformer that requires an
exception to know when it's involved from the destroy command.
2020-05-28 21:30:44 -04:00
James Bardin dc1b133831 remove requiresInstanceExpansion
simplification allows us to settle on a single interface,
graphNodeExpandsInstances for all types if instance expanders. The only
other specific class of resource we need to detect during pruning is the
nodeExpandApplyableResource node, which is already classified under the
GraphNodeResourceInstance interface.
2020-05-28 21:30:44 -04:00
James Bardin 082f91cd85 fix ModulePath for nodeExpandModule
ModulePath was incorrectly returning the parent module, because it did
not implement ReferenceOutside. With ReferenceOutside working correctly,
we can have ModulePath return the real path and remove the special case
for this during pruning.
2020-05-28 21:30:44 -04:00
James Bardin c638252210 pruneUnusedNodesTransformer
Create a single transformer to remove all unused nodes from the apply
graph. This is similar to the combination of the resource pruning done
in the destroy edge transformer, and the unused values transformer. In
addition to resources, variables, locals, and outputs, we now need to
remove unused module expansion nodes as well. Since these can all be
interdependent, we need to process them as whole in a single
transformation.
2020-05-28 21:30:42 -04:00
James Bardin 7122b271f9 restore the ordering of nested modules
In order for depends_on to work, modules need to implicitly depend on
their child modules. This will have little effect on terraform's
concurrency, as configuration trees are always much wider than they are
deep.
2020-05-28 21:30:12 -04:00
James Bardin 63b54f2b72 remove unused transformer 2020-05-28 21:30:12 -04:00
James Bardin 5c401bead9 nodeCloseModule needs GraphNodeReferenceOutside
The ModulePath method was incorrect, and standing in for
GraphNodeReferenceOutside. Add ReferenceOutside so we can fix
ModulePath.
2020-05-28 21:30:12 -04:00
James Bardin 009a136fa2 requiresInstanceExpansion and instanceExpander
create interfaces that nodes can implement to declare whether they
expand into instances of some sort, using the instances.Expander, and/or
whether use the instances.Expander to find instances.

included is a rough transformer implementation to remove these nodes
from the apply graph.
2020-05-28 21:30:12 -04:00
James Bardin 741aab31d1 delete unused code
this search for orphans was never used
2020-05-28 21:30:12 -04:00
James Bardin 8ba63110ec
Merge pull request #25005 from hashicorp/jbardin/module-depends-on
Module depends_on
2020-05-28 21:29:04 -04:00
Martin Atkins b0839175b1
Update CHANGELOG.md 2020-05-28 16:19:02 -07:00
Martin Atkins ec7dce0974
Update CHANGELOG.md 2020-05-28 16:13:23 -07:00
Martin Atkins d1bc412220 configs: Custom variable validation is no longer experimental
All of the feedback from the experiment described enhancements that can
potentially be added later without breaking changes, so this change simply
removes the experiment gate from the feature as originally implemented
with no changes to its functionality.

Further enhancements may follow in later releases, but the goal of this
change is just to ship the feature exactly as it was under the experiment.

Most of the changes here are cleaning up the experiment opt-ins from our
test cases. The most important parts are in configs/experiments.go and in
experiments/experiment.go .
2020-05-28 16:07:59 -07:00
James Bardin 5d64029275 depends_on doc updates
Update depends_on for modules and data sources.
2020-05-28 16:17:01 -04:00
Alisdair McDiarmid 8b279b6f34 plugin/discovery: Remove dead code
Provider installation is now handled in the internal/getproviders
package instead.
2020-05-28 15:20:41 -04:00
Alisdair McDiarmid ca40107066 command/init: Better diagnostics for provider 404s
Fetching a default namespace provider from the public registry can
result in 404 Not Found error. This might be caused by a previously-
default provider moving to a new namespace, which means that the
configuration needs to be upgraded to use an explicit provider source.

This commit adds a more detailed diagnostic for this situation,
suggesting that the intended provider might be in a new namespace. The
recommended course of action is to run the 0.13upgrade command to
generate the correct required_providers configuration.
2020-05-28 09:24:32 -04:00
Alisdair McDiarmid 01f91316da
Update CHANGELOG.md 2020-05-28 09:11:55 -04:00