When using the enhanced remote backend, a subset of all Terraform
operations are supported. Of these, only plan and apply can be executed
on the remote infrastructure (e.g. Terraform Cloud). Other operations
run locally and use the remote backend for state storage.
This causes problems when the local version of Terraform does not match
the configured version from the remote workspace. If the two versions
are incompatible, an `import` or `state mv` operation can cause the
remote workspace to be unusable until a manual fix is applied.
To prevent this from happening accidentally, this commit introduces a
check that the local Terraform version and the configured remote
workspace Terraform version are compatible. This check is skipped for
commands which do not write state, and can also be disabled by the use
of a new command-line flag, `-ignore-remote-version`.
Terraform version compatibility is defined as:
- For all releases before 0.14.0, local must exactly equal remote, as
two different versions cannot share state;
- 0.14.0 to 1.0.x are compatible, as we will not change the state
version number until at least Terraform 1.1.0;
- Versions after 1.1.0 must have the same major and minor versions, as
we will not change the state version number in a patch release.
If the two versions are incompatible, a diagnostic is displayed,
advising that the error can be suppressed with `-ignore-remote-version`.
When this flag is used, the diagnostic is still displayed, but as a
warning instead of an error.
Commands which will not write state can assert this fact by calling the
helper `meta.ignoreRemoteBackendVersionConflict`, which will disable the
checks. Those which can write state should instead call the helper
`meta.remoteBackendVersionCheck`, which will return diagnostics for
display.
In addition to these explicit paths for managing the version check, we
have an implicit check in the remote backend's state manager
initialization method. Both of the above helpers will disable this
check. This fallback is in place to ensure that future code paths which
access state cannot accidentally skip the remote version check.
The Registry is a web service whose behavior isn't directly tied to Terraform
core's release cycle; therefore, its docs should be decoupled from that release
cycle as well.
https://github.com/hashicorp/terraform-website/pull/1517 adopts the registry
docs into hashicorp/terraform-website, which already hosts several other
corpuses of documentation that aren't tied to Terraform core's version (like
Terraform Cloud, Terraform Enterprise, and Extending Terraform). Once that PR is
merged, we should remove the registry docs from this repository to avoid
confusing anyone.
The local-exec provisioner documentation includes an example which refers
to an attribute of the current resource using its full traversal path,
rather than using "self" as we typically expect.
Due to some coincidences in how Terraform builds the dependency graph,
referring to the resource in this way happens to work when the resource
has only a single instance (the graph builder just skips that
self-referential dependency edge), but it fails if the user later tries
to add "count" or "for_each" to the resource, because at that point all
of the instances become dependent on one another, which creates a
dependency cycle.
Using "self" to access the current instance attributes is the usual
approach, so I've updated the documentation to show that.
As written previously this seemed to suggest using "app.terraform.io" (the
"hostname you use to access the Terraform Cloud application) to access a
private registry in Terraform Enterprise, but that isn't true and I assume
isn't what was intended.
Instead, the hostname for a Terraform Enterprise instance is the hostname
where the Terraform Enterprise application is running, which is both the
hostname where users would find its web UI and the hostname they'd use
to configure the "remote" backend for remote operations and state storage.
In order to be able to predict a result type even if arguments are not yet
known, coalesce requires all of its arguments to be of the same type. Our
usual automatic conversion rules mean that in some cases the result is
a silent type conversion rather than an explicit error, so we'll at least
document that so that folks who encounter it can understand what is
causing the likely-surprising behavior.
If we were building this function over again today I expect we'd make it
always return an error under type mismatch, but to do so now would be a
breaking change and the potential cost of that seems too high for
something that doesn't seem to arise incredibly often in practice.
This one is a lot like the previous two commits, but slightly more complex:
- Only adding one new meta-argument page, for `providers`; otherwise, it just
re-uses the dual-purpose pages I made in the resources commit.
- About that `providers` argument: The stuff that was relevant to consumers of a
module went in that meta-argument page, but there was also a huge deep dive on
how the _author_ of a re-usable module should handle provider configurations
in cases where inheriting the default providers isn't sufficient. THAT, I
moved into a new page in the module development section. (For the consumer of
a module, this should all be an implementation detail; the module README
should tell you which aliased providers you need to configure and pass, and
then you just do it, without worrying about proxy configuration blocks etc.)
- The "standard module structure" recommendations in the main module development
page gets a page of its own, to make it more prominent and discoverable.
- Same deal with using the old URL as a landing page, at least for the main
module calls page. It didn't seem necessary for the module development page.
- Resource behavior gets its own page.
- Meta-arguments all get their own pages.
- Stuff about resource syntax itself gets a page.
In the process of breaking the meta-arguments out into their own pages, I
revised them (with the exception of `provider`) so that they apply to both
resources and modules.
Like with Expressions, this commit repurposes the old resources.html URL as a
landing page for old links.
This commit converts the previous URL for this content to a landing page, which
captures all of the previous in-page anchors and directs readers to the new home
for each section.
For some time now we've been recommending explicitly passing data between
configurations using separate resource types and data sources, rather than
always using terraform_remote_state, for reasons including reducing
coupling between subsystems and allowing a configuration's state snapshots
to be under restrictive access controls.
However, those recommendations have so far not appeared directly in the
documentation for terraform_remote_state, and have instead just been
alluded to elsewhere in the documentation when discussing ways to pass
data between configurations.
This change, then, is an attempt to be clear and explicit about the
recommendation and to give a variety of specific examples of how to
implement it. The terraform_remote_state data source page is admittedly
not the most obvious place in the information architecture to put a set
of alternatives to it, but it does appear that this documentation page is
where people most commonly end up when researching options in this area
and so I've put this here in an attempt to "meet people where they are".
Possibly in a future documentation reorganization we might have an
separate page specifically about sharing data between configurations, but
we don't currently have time to do that bigger reorganization. If we do so
later, the content on this page could potentially be replaced with a
summary of the recommendation and a link to another place for the details,
but the goal here is to make this information visible in the existing
location people look for it, rather than blocking until there's a better
place for it to live.
This also includes a small amount of editing of some existing content on
the page to use terminology and style more similar to how our main
configuration language documentation is written,.
Remove chef, habitat, puppet, and salt-masterless provsioners,
which follows their deprecation. Update the documentatin for these
provisioners to clarify that they have been removed from later versions
of Terraform. Adds the fmt Make target back and updates fmtcheck script
for correctness.
Some hasty, incorrect merge conflict fixing caused this page to have a
strange mix of terminology between "system" and "provider". Along with
that, there were also several editorial errors caused by text on this
page having originally been derived from the provider registry
documentation.
This documentation will now consistently talk about being a module
registry protocol rather than a provider registry protocol, and it will
consistently use the term "system" as a generic term for the final part
of the module source address, aside from noting that there is an optional
convention to name it after the "type" part of an official provider when
possible.
This is a new part of the existing module_variable_optional_attrs
experiment, because it's intended to complement the ability to declare
an input variable whose type constraint is an object type with optional
attributes. Module authors can use this to replace null values (that were
either explicitly set or implied by attribute omission) with other
non-null values of the same type.
This function is a bit more type-fussy than our functions typically are
because it's intended for use primarily with input variables that have
fully-specified type constraints, and thus it uses that type information
to help inform how the defaults data structure should be interpreted.
Other uses of this function will probably be harder today because it takes
a lot of extra annotation to build a value of a specific type if it isn't
passing through a variable type constraint. Perhaps later language
features for more general type conversion will make this more applicable,
but for now the more general form of this problem is better solved other
ways.
The new nav structure demanded a few new pages that give context about a feature
or workflow. In a few cases, they take text from an existing page.
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Co-authored-by: Judith Malnick <judith.patudith@gmail.com>
We're splitting the current Terraform CLI docs into two top-level categories,
and these are the new nav sidebars for those sections.
As of this commit, they refer to some new "glue" pages that don't exist yet.
The HashiCorp engineering services team has set up APT and Yum
repositories as alternative installation methods for various HashiCorp
products, now including Terraform.
We don't really have a great place to talk about these in our current
website structure. There is a longer-term plan to revamp the downloads
page to include other options, but we are already getting lots of
questions about how to use these repositories and so my goal here is to
publish at least a first pass of documentation, linked from the Downloads
page sidebar as a placeholder for now, so we'll have somewhere to refer to
when answering such questions.
My intent is that even once we have a revamped Downloads page that
mentions these options more clearly, we'll still need to link out to
another page to talk about various details, and so the two new URLs this
creates would be the home of that content, even if we rewrite the specific
prose here to work better in the context of the new Downloads page.
The example configuration now uses Terraform 0.12+ syntax, and the
output examples are up to date with the current text UI. We also add an
explicit recommendation to use the `-json` option for a consistent and
stable output format, for use in automation.
Prior to Terraform 0.12 these two functions were the only way to construct
literal lists and maps (respectively) in HIL expressions. Terraform 0.12,
by switching to HCL 2, introduced first-class syntax for constructing
tuple and object values, which can then be converted into list and map
values using the tolist and tomap type conversion functions.
We marked both of these functions as deprecated in the Terraform v0.12
release and have since then mentioned in the docs that they will be
removed in a future Terraform version. The "terraform 0.12upgrade" tool
from Terraform v0.12 also included a rule to automatically rewrite uses
of these functions into equivalent new syntax.
The main motivation for removing these now is just to get this change made
prior to Terraform 1.0. as we'll be doing with various other deprecations.
However, a specific reason for these two functions in particular is that
their existence is what caused us to invent the idea of a "type expression"
as a distinct kind of expression in Terraform v0.12, and so removing them
now would allow potentially unifying type expressions with value
expressions in a future release.
We do not have any current specific plans to make that change, but one
potential motivation for doing so would be to take another attempt at a
generalized "convert" function which takes a type as one of its arguments.
Our previous attempt to implement such a function was foiled by the fact
that Terraform's expression validator doesn't have any way to know to
treat one argument of a particular function as special, and so it was
generating incorrect error messages. We won't necessarily do that, but
having these "list" and "map" functions out of the way leaves the option
open.
The documentation states that an explicit type conversion to set is needed, but it does not say why implicit type conversion does not work.
Co-authored-by: Nick Fagerlund <nick@hashicorp.com>
* The index must be non-negative integer
and added instructions on how to get the last value in the list.
* Typo fix
Co-authored-by: Nick Fagerlund <nick@hashicorp.com>
* Update module-registry-protocol.html.md
1: There is a mismatch in the segment labels for the version query URL (system vs provider)
2: There is a discrepancy between the documentation and the actual generated request for retrieving module source code (URL segments 4 vs 3)
- There is no segment for "provider"
* Update module-registry-protocol.html.md
Changed ```:system``` to ```:provider``` for versions and source API URLs
These pages are thoroughly obsolete. Later, we'll delete and redirect them; for
now, we'll make sure the relevant pages are front-and-center in the sidebar if
someone somehow ends up on here.
Guides was already mostly gone. The two major remaining bits (the "core
workflow" guide and the "recommended practices" guide, which deserves a rename)
make much more sense as highly technical marketing material, and thus have a
natural home in the intro section.
Several `terraform` subcommands include sub-sub-commands; with our old sidebar
system, viewing those took you to an isolated "island" nav sidebar, away from
the main docs. The new navigation will adopt all these pages, so we don't need
to exile the reader to odd places.
As of this commit, that layout doesn't exist yet, but I'm isolating the one-line
changes to their own commit to try and keep your eyes from glazing over.
We typically try to avoid making subjective, boasty claims in our
documentation in recent times, but there remained both some older
documentation that we've not recently revised and also some newer examples
that are, in retrospect, also perhaps more "boasty" than they need to be.
We prefer not to use this sort of boasty language because not everyone
using Terraform has the same background and experience, and so what is
"easy" or "intuitive" to one person may not be so to another person, and
that should not suggest that the second person is in any way wrong or
inadequate.
In reviewing some of our use of the word "easy" here I tried as much as
possible to surgically revise the existing content without getting drawn
into a big rewrite, but in some cases the content was either pretty
unsalvageable (due to talking about obsolete features that were removed
long ago) or required some broader changes to make the result hopefully
still get the same facts across. In those cases I've both removed some
content entirely or adjusted larger paragraphs.
This was not an exhaustive review and so I'm sure there's still plenty of
room for similar improvements elsewhere. I also resisted the urge to
update some pages that contain outdated information about currently-active
features.
My initial motivation here was to update the example output from
Terraform's top-level help list to match recent updates in the layout
and language used.
However, while here I took the opportunity to update some dated language
that was not consistent with our modern documentation writing style,
in particular including a totally unnecessary and potentially-alienating
claim that Terraform is "very easy to use". Our modern writing style
discourages this sort of "boastful" language and encourages us to focus on
the facts at hand.
This is an analog to the "alltrue" function, using OR as the reduce
operator rather than AND.
This also includes some simplification of the "alltrue" implementation
to implement it similarly as a sort of reduce operation with AND
as the reduce operator, but with the same effective behavior.
* Fixes#26684
* Update provider-requirements.html.md
Removing additional/extra newlines
* Update provider-requirements.html.md
And now some trailing spaces. le sigh
* Update config.html.md
When reading this page, I couldn't find the list of the "supported backends to the left". They're actually on a different page, so thought I'd update it so that others wouldn't find it confusing like me.
If this is ok with you, would it be possible to label this PR with 'hacktoberfest-accepted'? I'm still new to this. If not, I'd be alright. Thank you!
* Update config.html.md
Swapped the full URL in the link for a relative path
Co-authored-by: Petros Kolyvas <petros@hashicorp.com>
* ADD CLI option position for force-unlock command
* Update force-unlock.html.markdown
Made a change to also include the missing [DIR]
Co-authored-by: Petros Kolyvas <petros@hashicorp.com>
These were initially introduced as functions with "encode" and "decode"
prefixes, but that doesn't match with our existing convention of putting
the encoding format first so that the encode and decode functions will
group together in a alphabetically-ordered function list.
"text" is not really a defined serialization format, but it's a short word
that hopefully represents well enough what these functions are aiming to
encode and decode, while being consistent with existing functions like
jsonencode/jsondecode, yamlencode/yamldecode, etc.
The "base64" at the end here is less convincing because there is precedent
for that modifier to appear both at the beginning and the end in our
existing function names. I chose to put it at the end here because that
seems to be our emergent convention for situations where the base64
encoding is a sort of secondary modifier alongside the primary purpose
of the function, as we see with "filebase64". (base64gzip is an exception
here, but it seems outvoted by the others.)
* Add note to upgrade guide about provider sensitivity
Now that sensitivity follows attributes providers mark
as sensitive, add this note to the upgrade guide.
Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
For normal provider installation we want to associate each provider with
a selected version number and find a suitable package for that version
that conforms to the official hashes for that release.
Those requirements are very onerous for a provider developer currently
testing a not-yet-released build, though. To allow for that case this new
CLI configuration feature allows overriding specific providers to refer
to give local filesystem directories.
Any provider overridden in this way is not subject to the usual
restrictions about selected versions or checksum conformance, and
activating an override won't cause any changes to the selections recorded
in the lock file because it's intended to be a temporary setting for one
developer only.
This is, in a sense, a spiritual successor of an old capability we had to
override specific plugins in the CLI configuration file. There were
some vestiges of that left in the main package and CLI config package
but nothing has actually been honoring them for several versions now and
so this commit removes them to avoid confusion with the new mechanism.
Terraform v0.10 introduced .terraform/plugins as a cache directory for
automatically-installed plugins, Terraform v0.13 later reorganized the
directory structure inside but retained its purpose as a cache.
The local cache used to also serve as a record of specifically which
packages were selected in a particular working directory, with the intent
that a second run of "terraform init" would always select the same
packages again. That meant that in some sense it behaved a bit like a
local filesystem mirror directory, even though that wasn't its intended
purpose.
Due to some unfortunate miscommunications, somewhere a long the line we
published some documentation that _recommended_ using the cache directory
as if it were a filesystem mirror directory when working with Terraform
Cloud. That was really only working as an accident of implementation
details, and Terraform v0.14 is now going to break that because the source
of record for the currently-selected provider versions is now the
public-facing dependency lock file rather than the contents of an existing
local cache directory on disk.
After some consideration of how to move forward here, this commit
implements a compromise that tries to avoid silently doing anything
surprising while still giving useful guidance to folks who were previously
using the unsupported strategy. Specifically:
- The local cache directory will now be .terraform/providers rather than
.terraform/plugins, because .terraform/plugins is effectively "poisoned"
by the incorrect usage that we can't reliably distinguish from prior
version correct usage.
- The .terraform/plugins directory is now the "legacy cache directory". It
is intentionally _not_ now a filesystem mirror directory, because that
would risk incorrectly interpreting providers automatically installed
by Terraform v0.13 as if they were a local mirror, and thus upgrades
and checksum fetches from the origin registry would be blocked.
- Because of the previous two points, someone who _was_ trying to use the
legacy cache directory as a filesystem mirror would see installation
fail for any providers they manually added to the legacy directory.
To avoid leaving that user stumped as to what went wrong, there's a
heuristic for the case where a non-official provider fails installation
and yet we can see it in the legacy cache directory. If that heuristic
matches then we'll produce a warning message hinting to move the
provider under the terraform.d/plugins directory, which is a _correct_
location for "bundled" provider plugins that belong only to a single
configuration (as opposed to being installed globally on a system).
This does unfortunately mean that anyone who was following the
incorrectly-documented pattern will now encounter an error (and the
aforementioned warning hint) after upgrading to Terraform v0.14. This
seems like the safest compromise because Terraform can't automatically
infer the intent of files it finds in .terraform/plugins in order to
decide automatically how best to handle them.
The internals of the .terraform directory are always considered
implementation detail for a particular Terraform version and so switching
to a new directory for the _actual_ cache directory fits within our usual
set of guarantees, though it's definitely non-ideal in isolation but okay
when taken in the broader context of this problem, where the alternative
would be silent misbehavior when upgrading.
The upgrade requirements for this release are considerably more modest
than for Terraform v0.13, so this time we just have some notes about a
few changes in behavior that may be impactful to some users.
This first pass is intended to be included as part of a forthcoming beta
testers' guide as we begin the v0.14 beta testing period. We will make
further changes to this upgrade guide based on feedback from those who
participate in the beta process.
Note that this upgrade guide is not intended as release marketing material
and so its presentation is focused on addressing concerns users might
encounter while upgrading. We'll share highlights from the release in
other contexts, such as the changelog and in the product blog.
This builds on an experimental feature in the underlying cty library which
allows marking specific attribtues of an object type constraint as
optional, which in turn modifies how the cty conversion package handles
missing attributes in a source value: it will silently substitute a null
value of the appropriate type rather than returning an error.
In order to implement the experiment this commit temporarily forks the
HCL typeexpr extension package into a local internal/typeexpr package,
where I've extended the type constraint syntax to allow annotating object
type attributes as being optional using the HCL function call syntax.
If the experiment is successful -- both at the Terraform layer and in
the underlying cty library -- we'll likely send these modifications to
upstream HCL so that other HCL-based languages can potentially benefit
from this new capability.
Because it's experimental, the optional attribute modifier is allowed only
with an explicit opt-in to the module_variable_optional_attrs experiment.
This includes both the main documentation about the lock file itself and
changes to related documentation about Terraform commands that interact
with the lock file.
We will likely continue to update this first pass of documentation as we
get feedback and questions during the prerelease period.
* website: Update all Learn crosslinks
The URL structure on Learn recently changed, so it's time to update some URLs.
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Alisdair McDiarmid