Commit Graph

81 Commits

Author SHA1 Message Date
Dennis Gursky 4e42d21837
Improve ModuleInstance String() performance (#28246)
* Optimize (m ModuleInstance) String()

Optimize (m ModuleInstance) String() to preallocate the buffer and use strings.Builder instead of bytes.Buffer

This leads to a common case only doing a single allocation as opposed to a few allocations which the bytes.Buffer is doing.

* adding a benchmark test

Result:

```
$ go test -bench=String ./addrs -benchmem 
BenchmarkStringShort-12         18271692                56.52 ns/op           16 B/op          1 allocs/op
BenchmarkStringLong-12           8057071               158.5 ns/op            96 B/op          1 allocs/op
PASS
$ git checkout main addrs/module_instance.go
$ go test -bench=String ./addrs -benchmem 
BenchmarkStringShort-12          7690818               162.0 ns/op            80 B/op          2 allocs/op
BenchmarkStringLong-12           2922117               414.1 ns/op           288 B/op          3 allocs/op
```

* Update module_instance_test.go

switch spaces to tabs
2021-04-05 08:44:27 -04:00
Alisdair McDiarmid a43b035a51 core: Reduce string allocations for addrs Equal
Generating strings and comparing them to implement Equal is a quick and
easy solution. Unfortunately when this code is in the hot path, it
becomes very expensive, so this commit changes some of those instances
to compare the values directly.

Combined with using addr.Equal instead of checking for string equality,
this makes Terraform dramatically faster for some operations, such as
generating large JSON plans.
2021-03-05 13:23:32 -05:00
James Bardin ddb5b9ee53 fix break from switch in loop 2020-12-02 13:59:19 -05:00
Kristin Laemmert 3933cbd491
remove LegacyProvider (#26433) 2020-10-05 08:33:49 -04:00
Martin Atkins 9c246e9a01 addrs: Resource.InModule, similar to Resource.Absolute
The Resource.Absolute function is there to conveniently construct an
AbsResource from a Resource by providing a module instance. Likewise, this
new InModule method allows conveniently constructing a ConfigResource from
a Resource by providing a module.
2020-08-24 11:41:28 -07:00
Alisdair McDiarmid d53a4ac588 addrs: Fix LegacyString for builtin provider addrs
Builtin provider addrs (i.e. "terraform.io/builtin/terraform") should be
able to convert to legacy string form (i.e. "terraform"). This ensures
that we can safely round-trip through ParseLegacyAbsProviderConfig,
which can return either a legacy or a builtin provider addr.
2020-08-14 14:56:35 -04:00
Martin Atkins 0a46ded3e7 addrs: Disallow provider source addresses starting with terraform-
The main motivation here is to produce a helpful error if a user
incorrectly uses the terraform-provider- prefix (which we see on provider
VCS repositories and plugin executables) as part of the source address.

However, this also more broadly blocks "terraform-" as a prefix in
anticipation of whatever instinct causes the phenomenon where e.g.
Python's PyPI has thousands of packages whose names start with "python-",
even though everything on PyPI is for Python by definition. This is
definitely not _necessary_, but it's better to be restrictive at first
and weaken later as needed.
2020-07-08 10:18:55 -07:00
James Bardin b861f5a4d7 unkeyed target ModulesInstance can be Modules
If the last step in a module target is an unkeyed instance, and it's
being compared against keyed instances, we have to assume it was
intended to be used as a Module rather than a ModuleInstance.
2020-06-09 21:58:58 -04:00
Kristin Laemmert 58bcc2e9bb
addrs: detect builtin provider when parsing legacy provider string (#25154)
* addrs: detect builtin provider when parsing legacy provider string

The ParseLegacyAbsProviderConfig was not detecting builtin providers
("terraform"), which caused issues for all users with 0.12 state and
the "terraform_remote_state" data source. Since "terraform" is the only
built-in provider this adds a very simple check to the parser so it
properly returns the builtin FQN.

* add tests to the addrs package
2020-06-08 08:27:36 -04:00
Kristin Laemmert 041f4dd8ca
configs: require normalized provider local names (#24945)
* addrs: replace NewLegacyProvider with NewDefaultProvider in ParseProviderSourceString

ParseProviderSourceString was still defaulting to NewLegacyProvider when
encountering single-part strings. This has been fixed.

This commit also adds a new function, IsProviderPartNormalized, which
returns a bool indicating if the string given is the same as a
normalized version (as normalized by ParseProviderPart) or an error.
This is intended for use by the configs package when decoding provider
configurations.

* terraform: fix provider local names in tests

* configs: validate that all provider names are normalized

The addrs package normalizes all source strings, but not the local
names. This caused very odd behavior if for e.g. a provider local name
was capitalized in one place and not another. We considered enabling
case-sensitivity for provider local names, but decided that since this
was not something that worked in previous versions of terraform (and we
have yet to encounter any use cases for this feature) we could generate
an error if the provider local name is not normalized. This error also
provides instructions on how to fix it.

* configs: refactor decodeProviderRequirements to consistently not set an FQN when there are errors
2020-05-14 09:00:58 -04:00
Kristin Laemmert 60321b41e8
getproviders: move protocol compatibility functions into registry client (#24846)
* internal/registry source: return error if requested provider version protocols are not supported

* getproviders: move responsibility for protocol compatibility checks into the registry client

The original implementation had the providercache checking the provider
metadata for protocol compatibility, but this is only relevant for the
registry source so it made more sense to move the logic into
getproviders.

This also addresses an issue where we were pulling the metadata for
every provider version until we found one that was supported. I've
extended the registry client to unmarshal the protocols in
`ProviderVersions` so we can filter through that list, instead of
pulling each version's metadata.
2020-05-11 13:49:12 -04:00
James Bardin 6c0f7703a6
Merge pull request #24697 from hashicorp/jbardin/get-module-data
Always return all module instances during evaluation
2020-04-22 09:49:45 -04:00
Kristin Laemmert f09ae6f862
provider source tests: added test suite to exercise hyphenated providers (#24685) 2020-04-16 15:54:33 -04:00
James Bardin a7507e140d parse module references as whole modules
Module references, like resource references, need to always return the
and object containing all instances in order to handle modules as single
values, and to postpone index evaluation to when the expression as whole
is evaluated.
2020-04-12 11:26:44 -04:00
James Bardin 46b85820ba
Update addrs/module_call.go
Co-Authored-By: Pam Selle <pam@hashicorp.com>
2020-04-10 14:45:36 -04:00
James Bardin b9ddec33e9 de-specify module output references
Like resource references, we need to make instance references less
specific when they aren't expanded yet during plan.
2020-04-09 15:41:35 -04:00
Alisdair McDiarmid b233aa39e9 addrs: Simplify presentation of provider FQNs
The provider fully-qualified name string used in configuration is very
long, and since most providers are hosted in the public registry, most
of that length is redundant. This commit adds and uses a `ForDisplay`
method, which simplifies the presentation of provider FQNs.

If the hostname is the default hostname, we now display only the
namespace and type. This is only used in UI, but should still be
unambiguous, as it matches the FQN string parsing behaviour.
2020-04-06 09:24:23 -07:00
Martin Atkins 7caf0b9246 addrs: ImpliedProviderForUnqualifiedType function
This encapsulates the logic for selecting an implied FQN for an
unqualified type name, which could either come from a local name used in
a module without specifying an explicit source for it or from the prefix
of a resource type on a resource that doesn't explicitly set "provider".

This replaces the previous behavior of just directly calling
NewDefaultProvider everywhere so that we can use a different implication
for the local name "terraform", to refer to the built-in terraform
provider rather than the stale one that's on registry.terraform.io for
compatibility with other Terraform versions.
2020-04-06 09:24:23 -07:00
Martin Atkins 5aa2e5ec8c addrs: "built-in" provider namespace
The introduction of a heirarchical addressing scheme for providers gives
us an opportunity to make more explicit the special case of "built-in"
providers.

Thus far we've just had a special case in the "command" package that the
provider named "terraform" is handled differently than all others, though
there's nothing especially obvious about that in the UI.

Moving forward we'll put such "built-in" providers under the special
namespace terraform.io/builtin/terraform, which will be visible in the UI
as being different than the other providers and we can use the namespace
itself (rather than a particular name) as the trigger for our special-case
behaviors around built-in plugins.

We have no plans to introduce any built-in providers other than
"terraform" in the foreseeable future, so any others will produce an
error.

This commit just establishes the addressing convention, without making use
of it anywhere yet. Subsequent commits will make the provider installer
and resolver codepaths aware of it, replacing existing checks for the
provider just being called "terraform".
2020-04-06 09:24:23 -07:00
Alisdair McDiarmid cadc133828 addrs: Fix diagnostic for invalid provider type
This previously rendered as `Invalid provider type ""`, as `name` was
empty if parsing failed. Using the source string is more helpful.
2020-04-02 08:15:52 -04:00
Pam Selle f738f85241 Create non-specific ModuleCallOutput 2020-03-26 13:29:38 -04:00
Kristin Laemmert 5b427ec648
addrs: Provider.IsLegacy and IsDefault functions
* add IsLegacy and IsDefault funcs to addrs.Provider
* add some test coverage
2020-03-23 09:50:35 -04:00
Kristin Laemmert ed1aebbeda
terraform: large refactor to use Provider from configs.Resource (#24396)
* terraform: large refactor to use Provider from configs.Resource

configs.Resource.ImpliedProvider() now returns a string; it is the
callers' responsibility to turn that into an addrs.Provider if needed.

GraphNodeProviderConsumer ProvidedBy() no longer returns nil (reverting
to earlier, pre-provider-fqn behavior): it will return either the
provider set in config, provider set in state, or the default provider.
2020-03-18 08:58:20 -04:00
James Bardin f0e175a835 add AbsResource.Config() ConfigResource 2020-03-16 11:16:23 -04:00
James Bardin e6bac359ed Missing ConfigResource checks in TargetContains
Adding some missing checks for ConfigResource, and check all
combinations of Resource types for consistency.
2020-03-13 19:01:23 -04:00
James Bardin e3ad9ffb77 added module targetting tests 2020-03-13 09:19:27 -04:00
James Bardin bf91bff2c8 TargetContains improvements
Simplify Module.TargetContains

Handle the case of a keyed instance address in
ModuleInstance.TargetContains.
2020-03-13 09:08:42 -04:00
Pam Selle 7407fee9c2 Make modules targetable 2020-03-13 09:08:42 -04:00
James Bardin 50077eabe9
Merge pull request #24362 from hashicorp/jbardin/module-expansion-some-more
implement addrs.ConfigResource
2020-03-13 09:08:26 -04:00
James Bardin 9054716caf implement addrs.ConfigResource
Core needs a way to address resources through unexpanded modules, as
they are present in the configuration. There are already some cases of
paring `addrs.Module` with `addrs.Resource` for this purpose, but it is
going to be helpful to have a single type to describe that pair, as
well as have the ability to use TargetContains.
2020-03-12 15:58:25 -04:00
Kristin Laemmert 1c78b26012
terraform: provider source test (#24342)
* configs: parse provider source string during module merge

This was the smallest unit of work needed to start writing provider
source tests!

* Update configs/parser_test.go

Co-Authored-By: Alisdair McDiarmid <alisdair@users.noreply.github.com>
2020-03-12 12:00:00 -04:00
James Bardin 8497adcb6e AbsProviderConfig to use addrs.Module
Change ModuleInstance to Module in AbsProviderConfig, because providers
need to be handled before module expansion, and should not be used
defined inside an expanded module at all.

Renaming of the addrs type can happen later, when there's less work
in-flight around provider configuration.
2020-03-10 20:25:44 -04:00
James Bardin 6ae9013c3f add addrs.Module.Equal
Mirror the addrs.ModuleInstance.Equal method
2020-03-10 17:25:11 -04:00
James Bardin b1df763541 remove UnkeyedInstanceShim from ref transformer
Since references are always within the scope of a single module, and we
can connect all module instance outputs for proper ordering, the
existing transformer works directly with only module paths as opposed to
module instances.

TODO: TransformApplyReferences for more precise module instance
targeting?
2020-03-10 17:25:11 -04:00
Martin Atkins b53e3074cf addrs: Provider.LessThan
When making lists of providers (or lists that contain providers) it's
helpful to have a canonical ordering in order to produce deterministic
results.

This ordering has no semantic meaning and is just here for the sake of
having a predictable standard.
2020-02-25 10:30:03 -05:00
Pam Selle c249943360
Module Expansion: Part 2 (#24154)
* WIP: dynamic expand

* WIP: add variable and local support

* WIP: outputs

* WIP: Add referencer

* String representation, fixing tests it impacts

* Fixes TestContext2Apply_outputOrphanModule

* Fix TestContext2Apply_plannedDestroyInterpolatedCount

* Update DestroyOutputTransformer and associated types to reflect PlannableOutputs

* Remove comment about locals

* Remove module count enablement

* Removes allowing count for modules, and reverts the test,
while adding a Skip()'d test that works when you re-enable
the config

* update TargetDownstream signature to match master

* remove unnecessary method

Co-authored-by: James Bardin <j.bardin@gmail.com>
2020-02-24 17:42:32 -05:00
James Bardin bf65b516c0
Merge pull request #24163 from hashicorp/jbardin/destroy-provisioner-keys
Destroy provisioner each.key
2020-02-20 08:41:55 -05:00
James Bardin 8421abaca0 Add InstanceKey.Value method
Have the InstanceKey implementations handle their own cty.Value
conversions.
2020-02-19 16:02:40 -05:00
Martin Atkins 86f0b5191c addrs: Stronger validation and normalization of provider namespace/type
The provider FQN is becoming our primary identifier for a provider, so
it's important that we are clear about the equality rules for these
addresses and what characters are valid within them.

We previously had a basic regex permitting ASCII letters and digits for
validation and no normalization at all. We need to do at least case
folding and UTF-8 normalization because these names will appear in file
and directory names in case-insensitive filesystems and in repository
names such as on GitHub.

Since we're already using DNS-style normalization and validation rules
for the hostname part, rather than defining an entirely new set of rules
here we'll just treat the provider namespace and type as if they were
single labels in a DNS name. Aside from some internal consistency, that
also works out nicely because systems like GitHub use organization and
repository names as part of hostnames (e.g. with GitHub Pages) and so
tend to apply comparable constraints themselves.

This introduces the possibility of names containing letters from alphabets
other than the latin alphabet, and for latin letters with diacritics.
That's consistent with our introduction of similar support for identifiers
in the language in Terraform 0.12, and is intended to be more friendly to
Terraform users throughout the world that might prefer to name their
products using a different alphabet. This is also a further justification
for using the DNS normalization rules: modern companies tend to choose
product names that make good domain names, and now such names will be
usable as Terraform provider names too.
2020-02-18 15:42:09 -08:00
Martin Atkins 657f9502e4 addrs: ModuleInstance.Module
This returns the address of the module that the module instance is an
instance of.
2020-02-14 15:20:07 -08:00
Martin Atkins 9f727b5799 addrs: ModuleInstanceStep.String
When ModuleInstanceStep values appear alone in debug messages, it's easier
to read them in a compact, HCL-like form than as the default struct
printing style.
2020-02-14 15:20:07 -08:00
Kristin Laemmert add134298a
addrs: ProviderConfig fixups (#24115)
* fix outdated syntax in comments
* test for non-strings in ParseAbsProviderConfig
* ProviderConfigDefault and ProviderConfigAliased now take Providers
instead of strings
2020-02-14 09:06:50 -05:00
Kristin Laemmert 47a16b0937
addrs: embed Provider in AbsProviderConfig instead of Type
a large refactor to addrs.AbsProviderConfig, embedding the addrs.Provider instead of a Type string. I've added and updated tests, added some Legacy functions to support older state formats and shims, and added a normalization step when reading v4 (current) state files (not the added tests under states/statefile/roundtrip which work with both current and legacy-style AbsProviderConfig strings).

The remaining 'fixme' and 'todo' comments are mostly going to be addressed in a subsequent PR and involve looking up a given local provider config's FQN. This is fine for now as we are only working with default assumption.
2020-02-13 15:32:58 -05:00
Kristin Laemmert 7eed30595a
moduledeps: replace ProviderInstance with addrs.Provider (#24017)
* addrs: add ParseProviderSourceString function to parse fqns from
tfconfig-inspect
* moduledeps: use addrs.Provider instead of ProviderInstance
2020-02-05 09:27:32 -05:00
Martin Atkins 8b511524d6
Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978)
* Introduce "Local" terminology for non-absolute provider config addresses

In a future change AbsProviderConfig and LocalProviderConfig are going to
become two entirely distinct types, rather than Abs embedding Local as
written here. This naming change is in preparation for that subsequent
work, which will also include introducing a new "ProviderConfig" type
that is an interface that AbsProviderConfig and LocalProviderConfig both
implement.

This is intended to be largely just a naming change to get started, so
we can deal with all of the messy renaming. However, this did also require
a slight change in modeling where the Resource.DefaultProviderConfig
method has become Resource.DefaultProvider returning a Provider address
directly, because this method doesn't have enough information to construct
a true and accurate LocalProviderConfig -- it would need to refer to the
configuration to know what this module is calling the provider it has
selected.

In order to leave a trail to follow for subsequent work, all of the
changes here are intended to ensure that remaining work will become
obvious via compile-time errors when all of the following changes happen:
- The concept of "legacy" provider addresses is removed from the addrs
  package, including removing addrs.NewLegacyProvider and
  addrs.Provider.LegacyString.
- addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded
  in it and has an addrs.Provider and a string alias directly instead.
- The provider-schema-handling parts of Terraform core are updated to
  work with addrs.Provider to identify providers, rather than legacy
  strings.

In particular, there are still several codepaths here making legacy
provider address assumptions (in order to limit the scope of this change)
but I've made sure each one is doing something that relies on at least
one of the above changes not having been made yet.

* addrs: ProviderConfig interface

In a (very) few special situations in the main "terraform" package we need
to make runtime decisions about whether a provider config is absolute
or local.

We currently do that by exploiting the fact that AbsProviderConfig has
LocalProviderConfig nested inside of it and so in the local case we can
just ignore the wrapping AbsProviderConfig and use the embedded value.

In a future change we'll be moving away from that embedding and making
these two types distinct in order to represent that mapping between them
requires consulting a lookup table in the configuration, and so here we
introduce a new interface type ProviderConfig that can represent either
AbsProviderConfig or LocalProviderConfig decided dynamically at runtime.

This also includes the Config.ResolveAbsProviderAddr method that will
eventually be responsible for that local-to-absolute translation, so
that callers with access to the configuration can normalize to an
addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's
currently unused because existing callers are still relying on the
simplistic structural transform, but we'll switch them over in a later
commit.

* rename LocalType to LocalName

Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-01-31 08:23:07 -05:00
Kristin Laemmert 6541775ce4
addrs: roll back change to Type field in ProviderConfig (#23937) 2020-01-28 08:13:30 -05:00
Martin Atkins 5771db6696 addrs: Fix infinite recursion in AbsInputVariableInstance.String 2020-01-10 15:23:25 -08:00
Kristin Laemmert 67fc4dd5a1 configs: move ProviderConfigCompact[Str] from addrs to configs
The configs package is aware of provider name and type (which are the
same thing today, but expected to be two different things in a future
release), and should be the source of truth for a provider config
address.
2019-12-09 08:30:08 -05:00
Kristin Laemmert e3416124cc
addrs: replace "Type string" with "Type Provider" in ProviderConfig
* huge change to weave new addrs.Provider into addrs.ProviderConfig
* terraform: do not include an empty string in the returned Providers /
Provisioners
- Fixed a minor bug where results included an extra empty string
2019-12-06 08:00:18 -05:00
Kristin Laemmert 9891d0354a
providers: use addrs.Provider as map keys for provider.Factory (#23548)
* terraform/context: use new addrs.Provider as map key in provider factories
* added NewLegacyProviderType and LegacyString funcs to make it explicit that these are temporary placeholders

This PR introduces a new concept, provider fully-qualified name (FQN), encapsulated by the `addrs.Provider` struct.
2019-12-04 11:30:20 -05:00