Commit Graph

1130 Commits

Author SHA1 Message Date
Alisdair McDiarmid 09d8355f43 command: Add experimental concise diff renderer
When rendering a diff between current state and projected state, we only
show resources and outputs which have changes. However, we show a full
structural diff for these values, which includes all attributes and
blocks for a changed resource or output. The result can be a very long
diff, which makes it difficult to verify what the changed fields are.

This commit adds an experimental concise diff renderer, which suppresses
most unchanged fields, only displaying the most relevant changes and
some identifying context. This means:

- Always show all identifying attributes, initially defined as `id`,
  `name`, and `tags`, even if unchanged;
- Only show changed, added, or removed primitive values: `string`,
  `number`, or `bool`;
- Only show added or removed elements in unordered collections and
  structural types: `map`, `set`, and `object`;
- Show added or removed elements with any surrounding unchanged elements
  for sequence types: `list` and `tuple`;
- Only show added or removed nested blocks, or blocks with changed
  attributes.

If any attributes, collection elements, or blocks are hidden, a count
is kept and displayed at the end of the parent scope. This ensures that
it is clear that the diff is only displaying a subset of the resource.

The experiment is currently enabled by default, but can be disabled by
setting the TF_X_CONCISE_DIFF environment variable to 0.
2020-09-10 10:35:55 -04:00
Martin Atkins b0da5b1ce5 core: Remove the last few HIL remnants
We've not been using HIL in the main codepaths since Terraform 0.12, but
some references to it (and some supporting functionality in Terraform)
stuck around due to interactions with types we'd kept around to support
legacy shims.

However, removing the configs.RawConfig field from
terraform.ResourceConfig disconnects that subtree of dependencies from
everything else, allowing us to remove it. This is safe because the only
remaining uses of terraform.ResourceConfig are shims from values that
were already evaluated using the HCL 2 API, and thus they never need
the "just in time" HIL evaluation that ResourceConfig.interpolateForce
used to do.

We also had some HIL references in configs/hcl2shim that were previously
in support of the "terraform 0.12upgrade" command, but the implementation
of that command is now removed.

There was one remaining reference to HIL in a now-unused function in the
helper/schema package, which I removed entirely here.

This then allows us to remove the HIL dependency entirely, and also to
clean up some remaining old remants of the legacy "config" package that
we'd recently moved into the "configs" package pending further pruning.
2020-09-02 15:53:33 -07:00
Ganesh Prasad Kumble ae717e6175
Update hashcode.go
Updating the comment `we need and non negative integer` to `we need a non negative integer`
2020-05-22 11:28:56 +05:30
James Bardin a286ef6d9c remove depends_on from helper test shims 2020-05-14 15:57:16 -04:00
Mikhail Mazurskiy c9ce1f0c1f
Fix typos (#24531) 2020-04-13 08:34:40 -04:00
Kristin Laemmert c4873778c8 Mildwonkey/tests (#24548)
* helper/resource: remove provider resolver test
* repl tests passing
* helper/resource: add some extra shimming to ShimLegacyState

Some of the tests in helper/resource have to shim between legacy and
modern states. Terraform expects modern states to have the Provider set
for each resource (and not be a legacy provider). This PR adds a wrapper
around terraform.ShimLegacyState which iterates through the resources
in the state (after the shim) and adds the provider FQN.
2020-04-06 09:24:23 -07:00
Martin Atkins 549aede792 Remove terraform.ResourceProvider, use providercache.Installer instead
Back when we first introduced provider versioning in Terraform 0.10, we
did the provider version resolution in terraform.NewContext because we
weren't sure yet how exactly our versioning model was going to play out
(whether different versions could be selected per provider configuration,
for example) and because we were building around the limitations of our
existing filesystem-based plugin discovery model.

However, the new installer codepath is new able to do all of the
selections up front during installation, so we don't need such a heavy
inversion of control abstraction to get this done: the command package can
select the exact provider versions and pass their factories directly
to terraform.NewContext as a simple static map.

The result of this commit is that CLI commands other than "init" are now
able to consume the local cache directory and selections produced by the
installation process in "terraform init", passing all of the selected
providers down to the terraform.NewContext function for use in
implementing the main operations.

This commit is just enough to get the providers passing into the
terraform.Context. There's still plenty more to do here, including to
repair all of the tests this change has additionally broken.
2020-04-06 09:24:23 -07:00
James Bardin cd65b28da0 don't call os.NewFile on unknown FDs
os.NewFile was called on file descriptors 3, 4, and 5 during every init,
in case this process happened to be running inside panicwrap. If the
runtime has already chosen one of these file descriptors to use
internally, starting polling on them can cause the runtime to crash.

Initialize the file descriptors lazily, only if we know that they belong
to us, after Wrapped is checked.
2020-04-04 15:46:19 -04:00
Kristin Laemmert c8d64846ad
Mildwonkey/ps import (#24412)
* import: remove Config from ImportOpts

`Config` in ImportOpts was any provider configuration provided by the
user on the command line. This option has already been removed in favor
of only taking the provider from the configuration loaded in the current
context.

* terrafrom: add Config to ImportStateTransformer and refactor Transform
to get the resource provider FQN from the Config
2020-03-20 08:15:29 -04:00
James Bardin a8b9547e0d fixup states.Resource change throughout packages 2020-03-16 16:50:48 -04:00
James Bardin e13eecbc5b finish provider ModuleInstance replacement 2020-03-11 14:19:52 -04:00
Paddy e6592dc710
Add support for provider metadata to modules. (#22583)
Implement a new provider_meta block in the terraform block of modules, allowing provider-keyed metadata to be communicated from HCL to provider binaries.

Bundled in this change for minimal protocol version bumping is the addition of markdown support for attribute descriptions and the ability to indicate when an attribute is deprecated, so this information can be shown in the schema dump.

Co-authored-by: Paul Tyng <paul@paultyng.net>
2020-03-05 16:53:24 -08:00
Kristin Laemmert 47a16b0937
addrs: embed Provider in AbsProviderConfig instead of Type
a large refactor to addrs.AbsProviderConfig, embedding the addrs.Provider instead of a Type string. I've added and updated tests, added some Legacy functions to support older state formats and shims, and added a normalization step when reading v4 (current) state files (not the added tests under states/statefile/roundtrip which work with both current and legacy-style AbsProviderConfig strings).

The remaining 'fixme' and 'todo' comments are mostly going to be addressed in a subsequent PR and involve looking up a given local provider config's FQN. This is fine for now as we are only working with default assumption.
2020-02-13 15:32:58 -05:00
Martin Atkins 8b511524d6
Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978)
* Introduce "Local" terminology for non-absolute provider config addresses

In a future change AbsProviderConfig and LocalProviderConfig are going to
become two entirely distinct types, rather than Abs embedding Local as
written here. This naming change is in preparation for that subsequent
work, which will also include introducing a new "ProviderConfig" type
that is an interface that AbsProviderConfig and LocalProviderConfig both
implement.

This is intended to be largely just a naming change to get started, so
we can deal with all of the messy renaming. However, this did also require
a slight change in modeling where the Resource.DefaultProviderConfig
method has become Resource.DefaultProvider returning a Provider address
directly, because this method doesn't have enough information to construct
a true and accurate LocalProviderConfig -- it would need to refer to the
configuration to know what this module is calling the provider it has
selected.

In order to leave a trail to follow for subsequent work, all of the
changes here are intended to ensure that remaining work will become
obvious via compile-time errors when all of the following changes happen:
- The concept of "legacy" provider addresses is removed from the addrs
  package, including removing addrs.NewLegacyProvider and
  addrs.Provider.LegacyString.
- addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded
  in it and has an addrs.Provider and a string alias directly instead.
- The provider-schema-handling parts of Terraform core are updated to
  work with addrs.Provider to identify providers, rather than legacy
  strings.

In particular, there are still several codepaths here making legacy
provider address assumptions (in order to limit the scope of this change)
but I've made sure each one is doing something that relies on at least
one of the above changes not having been made yet.

* addrs: ProviderConfig interface

In a (very) few special situations in the main "terraform" package we need
to make runtime decisions about whether a provider config is absolute
or local.

We currently do that by exploiting the fact that AbsProviderConfig has
LocalProviderConfig nested inside of it and so in the local case we can
just ignore the wrapping AbsProviderConfig and use the embedded value.

In a future change we'll be moving away from that embedding and making
these two types distinct in order to represent that mapping between them
requires consulting a lookup table in the configuration, and so here we
introduce a new interface type ProviderConfig that can represent either
AbsProviderConfig or LocalProviderConfig decided dynamically at runtime.

This also includes the Config.ResolveAbsProviderAddr method that will
eventually be responsible for that local-to-absolute translation, so
that callers with access to the configuration can normalize to an
addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's
currently unused because existing callers are still relying on the
simplistic structural transform, but we'll switch them over in a later
commit.

* rename LocalType to LocalName

Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-01-31 08:23:07 -05:00
Kristin Laemmert 6541775ce4
addrs: roll back change to Type field in ProviderConfig (#23937) 2020-01-28 08:13:30 -05:00
Kristin Laemmert e3416124cc
addrs: replace "Type string" with "Type Provider" in ProviderConfig
* huge change to weave new addrs.Provider into addrs.ProviderConfig
* terraform: do not include an empty string in the returned Providers /
Provisioners
- Fixed a minor bug where results included an extra empty string
2019-12-06 08:00:18 -05:00
Martin Atkins 97d64581c6 helper/logging: Loosen the filtering heuristic even more
Now we'll consider any message that doesn't start with a digit as a
continuation. This is _definitely_ in loose, best-effort territory now
since a continuation line could very well start with a digit, but
ultimately this is still better than the totally broken behavior that
preceded it.

Just to make sure that none of these heuristics interfere with producing
complete, accurate logs for TRACE, we'll skip the LevelFilter altogether
in that case.
2019-12-05 15:22:03 -08:00
Martin Atkins faffb3a784 helper/logging: Warn about levels other than TRACE
The filtering for other log levels is unreliable and glitchy because it's
trying to infer information from the log stream that isn't reliably
represented.

Although the previous commit has improved the situation somewhat, it is
still a tricky and unreliable heuristic, so worth a warning to anyone who
is reading such a log that if they see something confusing it could be
a result of the heuristic not working fully.
2019-12-05 15:22:03 -08:00
Martin Atkins 30bf83cdeb helper/logging: Bring the LevelFilter into our own codebase
In order to make this work reasonably we can't avoid using some funny
heuristics, which are somewhat reasonable to apply within the context of
Terraform itself but would not be good to add to the general "logutils".

Specifically, this is adding the additional heuristic that lines starting
with spaces are continuation lines and so should inherit the log level
of the most recent non-continuation line.
2019-12-05 15:22:03 -08:00
Kristin Laemmert 9891d0354a
providers: use addrs.Provider as map keys for provider.Factory (#23548)
* terraform/context: use new addrs.Provider as map key in provider factories
* added NewLegacyProviderType and LegacyString funcs to make it explicit that these are temporary placeholders

This PR introduces a new concept, provider fully-qualified name (FQN), encapsulated by the `addrs.Provider` struct.
2019-12-04 11:30:20 -05:00
James Bardin bee703360c
Merge pull request #23252 from hashicorp/jbardin/abs-state-dependencies
store absolute addresses for resource dependencies in the state
2019-11-08 10:25:32 -05:00
James Bardin 16b368e292 only need to handle depends_on to shim old state 2019-11-07 17:49:03 -05:00
James Bardin 49439d02d1 sanitize provisioner output strings
The grpc protocol requires strings to be valid utf8, but because
provisioners often don't have control over the command output, invalid
utf8 sequences can make it into the response causing grpc transport
errors.

Replace all invalid utf sequences with the standard utf replacement
character in the provisioner output. The code is a direct copy from the
go1.13 std library, and can be replaced with strings.ToValidUTF8 once
it's available.
2019-11-06 14:57:52 -05:00
Radek Simko 7860f55e4f
Version tools per Go convention under tools.go 2019-10-17 22:23:39 +02:00
Martin Atkins 39e609d5fd vendor: switch to HCL 2.0 in the HCL repository
Previously we were using the experimental HCL 2 repository, but now we'll
shift over to the v2 import path within the main HCL repository as part of
actually releasing HCL 2.0 as stable.

This is a mechanical search/replace to the new import paths. It also
switches to the v2.0.0 release of HCL, which includes some new code that
Terraform didn't previously have but should not change any behavior that
matters for Terraform's purposes.

For the moment the experimental HCL2 repository is still an indirect
dependency via terraform-config-inspect, so it remains in our go.sum and
vendor directories for the moment. Because terraform-config-inspect uses
a much smaller subset of the HCL2 functionality, this does still manage
to prune the vendor directory a little. A subsequent release of
terraform-config-inspect should allow us to completely remove that old
repository in a future commit.
2019-10-02 15:10:21 -07:00
James Bardin a3c374b227 don't lose track of private data in the ACC tests
If the resource schema version is > 0, the metadata containing the
timeout values was overwritten during acceptance tests.
2019-09-18 13:59:12 -04:00
appilon d3dc1263bf
Merge pull request #22382 from hashicorp/appilon/delete-files
[CLEANUP] Prune internal usage of config package (with 1 exception)
2019-08-08 13:01:00 -04:00
Radek Simko ecbfc365a0
Merge pull request #20157 from shinmog/conflictswith
helper/schema: Add "part" to ConflictsWith validation error
2019-08-08 14:20:35 +01:00
Radek Simko db7d1826e4
Merge pull request #22312 from ewbankkit/add-rand-ip-address
helper/acctest: Add function to return random IP address
2019-08-08 13:53:51 +01:00
Radek Simko 7861cc6141
Merge pull request #20309 from chanzuckerberg/ryanking/remove_lifecycle_comment
helper/schema: Remove comment about nested schema.Resources having their own lifecycle
2019-08-08 13:48:22 +01:00
Radek Simko 350395d804
Add better comment 2019-08-08 13:21:04 +01:00
Radek Simko a08ad08169
Merge pull request #18277 from ndmckinley/patch-1
helper/resource: Minor docs tweak on ImportStateVerify
2019-08-08 13:08:19 +01:00
Alex Pilon 77757d9f5b
prune references to config/module
delete config/module
prune references to config except in terraform/resource.go
move, cleanup, and delete inert code
2019-08-07 17:50:59 -04:00
Alex Pilon 4bf43efcfd
move hcl2shim package to configs 2019-08-06 19:58:58 -04:00
Alex Pilon ec0402a238
add case to decoder to assert to slice, then each item to a map 2019-08-05 22:08:05 -04:00
Alex Pilon 6ae281b34f
set Err to true for test
This was likely meant to be set to Err true but
the old parser mistakenly didn't raise an error
2019-08-05 22:08:04 -04:00
Alex Pilon 6de3c32652
comment out failing test 2019-08-05 22:08:04 -04:00
Alex Pilon 83aa07f907
prune NewResourceConfig and update tests 2019-08-05 22:08:03 -04:00
Kit Ewbank 949a6ae695 Add 'RandIpAddress'. 2019-08-02 14:51:40 -04:00
The Terraform Team 176f790323
Merge pull request #22262 from hashicorp/mildwonkey/b-deprecation-warnings
helper/schema: don't skip deprecation check during validation
2019-07-30 21:27:51 +01:00
Kristin Laemmert bfd66083de helper/schema: don't skip deprecation check during validation
If an attribute was not wholly known, helper/schema was skipping the
`validateType` function which (among other things) returned deprecation
messages. This PR checks for deprecation before returning when skipping
validateType.
2019-07-30 16:00:43 -04:00
Paul Tyng e400c25358
Minor typo 2019-07-30 10:43:46 -04:00
Paul Tyng 366f70a71e
Update copy for Exists
You can signal the same information in `Read` with an empty ID if the object does not exist, Implementing `Exists` is not the only way to do so and in some providers is also not the preferred way.
2019-07-30 10:37:58 -04:00
James Bardin 7648f99bca
Merge pull request #22236 from hashicorp/jbardin/nil-in-config
don't reflect nil in schema validation
2019-07-29 14:08:37 -07:00
James Bardin 016c4f782d don't reflect nil in schema validation
Nil values were not previously expected during validation, but they can
appear in some situations with the new protocol. Add checks to prevent
using zero reflect.Values.
2019-07-29 12:38:35 -04:00
James Bardin 4bed030d40 don't validate MinItems with unknowns in a block
If there are unknowns, the block may have come from a dynamic
declaration, and we can't validate MinItems. Once the blocks are
expanded, we will get the full config for validation without any unknown
values.
2019-07-27 11:50:28 -07:00
Alex Pilon 7f8f198719
remove UnknownVariabeValue from config and update references to shim 2019-07-17 22:41:24 -04:00
James Bardin 75602df5ef Revert "Remove removed attribute from applied state"
This reverts commit 2e2a363052.
2019-06-28 11:51:52 -04:00
James Bardin 8e8eb6d6d3
Merge pull request #21884 from hashicorp/jbardin/validate-null-diags
don't append nil elements to a diagnostic cty.Path
2019-06-26 09:33:04 -04:00
James Bardin d4b0788854 don't append nil elements to a diagnostic cty.Path
If a set element is nil in validateConfigNulls, we don't want to
append that element to the diagnostic path, since it doesn't offer any
useful info to the user.
2019-06-25 18:05:51 -04:00