Commit Graph

63 Commits

Author SHA1 Message Date
Martin Atkins ba0514106a return tfdiags.Diagnostics from validation methods
Validation is the best time to return detailed diagnostics
to the user since we're much more likely to have source
location information, etc than we are in later operations.

This change doesn't actually add any detail to the messages
yet, but it changes the interface so that we can gradually
introduce more detailed diagnostics over time.

While here there are some minor adjustments to some of the
messages to improve their consistency with terminology we
use elsewhere.
2017-11-28 11:15:29 -08:00
James Bardin b15258dfec remove provider inheritance from Tree.Load
Now that resources can be connected to providers with different paths in
the core graph, handling the inheritance in config makes less sense.

Removing this to make room for core to walk the Tree and connect
resources directly to the proper provider instance.
2017-11-06 17:27:45 -05:00
James Bardin 2ed529f81a update from review
Reference modules by terraform address
Unquote versions
2017-11-03 10:50:19 -04:00
James Bardin c510473fd2 remove code added by stale vim buffer 2017-11-03 10:33:18 -04:00
James Bardin 1bf64ec788 add "Updating" output and fix output tests 2017-11-03 10:28:08 -04:00
James Bardin 9c334fe012 update init output
Change "Downloading" to 'Initializing" to match the provider loading
dialog.

List each module being loaded.

If a regisry module is being downloaded, list the registry host, and the
version discovered.

Show the source string from the config that is being fetched, rather
than the go-getter url. The full source can be found in the logs for
debugging.

Add much more extensive logging
2017-11-03 10:28:07 -04:00
James Bardin 838ffafc18 stop adding implicit provider configs
Now that providers in the graph can adopt resources without an explicit
provider, there's no need to add the implicit configs to the module.Tree
when loading.
2017-11-02 15:29:51 -04:00
James Bardin 3a495ffe56 rename ModuleStorage to Storage
get rid of stutter and use module.Storage
2017-10-27 13:11:21 -04:00
James Bardin 36eb40a432 export ModuleStorage and use it for Tree.Load
Exporting ModuleStorage allows us to explicitly pass in the storgae
location rather than extracting it out of the getter.Storage interface,
set a UI for communiating actions back to the user, and accepting a
services Disco for discovery.
2017-10-27 11:29:29 -04:00
James Bardin 9bda82ccf1 add example for the reason behind versionedPathKey 2017-10-27 09:08:15 -04:00
James Bardin 1756b3d09f ProviderConfig needs an Inherited flag
If a provider configuration is inherited from another module, any
interpolations in that config won't have variables declared locally. Let
the config only be validated in it's original location.
2017-10-27 09:08:15 -04:00
James Bardin 0afd4a9097 Lookup registry module versions during Tree.Load.
Registry modules can't be handled directly by the getter.Storage
implementation, which doesn't know how to handle versions. First see if
we have a matching module stored that satisfies our constraints. If
not, and we're getting or updating, we can look it up in the registry.

This essentially takes the place of a "registry detector" for go-getter,
but required the intermediate step of resolving the version dependency.

This also starts breaking up the huge Tree.Load method into more
manageable parts. It was sorely needed, as indicated by the difficulty
encountered in this refactor. There's still a lot that can be done to
improve this, but at least there are now a few easier to read methods
when we come back to it.
2017-10-27 09:08:15 -04:00
James Bardin 0d10564a74 remove the registryDetector
The detection of registry modules will have to happen in mutliple
phases. The go-getter interface requires that the detector return the
final URL, while we won't know that until we verify which version we
need. This leaves the regisry sources broken, to be re-integrated in a
following commit.
2017-10-27 09:08:15 -04:00
James Bardin ae2903b810 missing current module source in versionedPathKey
The reason versionedPathKey didn't solve the previous test case was a
missing source string for the current module.
2017-10-27 09:08:15 -04:00
James Bardin 0bc279557e add more metadata to the Tree and the storage path
Submodules were located by using their module path as the storage key.
Now that modules may have versions, a submodule needs to know how to
locate the corect source depending on the versions of its ancestors in
the tree.

Add a version field to each Tree, and a pointer back to the parent Tree
to step back through the ancestors. The new versionedPathKey method uses
this information to build a unique key for each module, dependent on the
ancestor versions.

Not only do stored modules need to know their version if it exists, but
any relative source needs to know all the ancestor versions in order to
resolve correctly.
2017-10-27 09:08:15 -04:00
James Bardin 1f44fd8eb2 move getStorage to moduleStorage too
Add a test to ensure we don't store different modules with the same
relative path in the same location.
2017-10-27 09:08:15 -04:00
James Bardin 3d163917c7 move the getter.Storage into our moduleStorge type
Wrap the Storage interfaces together.
2017-10-27 09:08:15 -04:00
James Bardin 7416e7a6e0 update the module manifest implementation
The getter.Storage abstraction is proving entirely inadequate here, but
we can't replace it wholesale at the moment.

The Tree loader needs to know the location of the manifest before it can
start loading any modules. Since the version will have to be part of the
hashed storage key, there is no way to know what version of each module
are stored. The storageDir function will extract the StorageDir field
from the underlying FolderStorage instance for the tree to locate the
manifest.
2017-10-27 09:08:15 -04:00
James Bardin 283d88551a Add a new module storage manifest
To add registry support, a workaround in the local module storage was
added to record the subdirectory containing the module source from
within the archive file. Here we replace that temporary implementation
with the full manifest needed to record the necessary module metadata
for module loading.

In order to support versioned modules, the actual stored version needs
to be recorded. This can't be derived from the configuration, because
the configuration only contains the constraints, and at load time we need
to be able to enumerate the stored modules and all versions in order to
resolve them.

While the local storage key will be derived from the source and version,
that information is lost once it's hashed. While the entire storage
layer could be replaced to encode the needed data in the path itself,
this provides a minimal change to work with the existing storage code.
2017-10-27 09:08:15 -04:00
James Bardin 8933ef8f70 make //subdir notation work with registry modules
Append the original source subdir to the discovered subdir when locating
the files in the module tree.
2017-10-27 09:08:15 -04:00
James Bardin b0163e69d1 update comments 2017-10-27 09:08:15 -04:00
James Bardin 2e505083cd change ProviderConfig.Scope to Path
Though it's intended for "interpolation scope", Path is generally used
for this elsewhere.
2017-10-27 09:08:15 -04:00
James Bardin cb0e37a870 implement provider inheritence during loading
This implements provider inheritance during config loading, rather than
during graph evaluation. At this point it's much simpler to find the
desired configuration, and once all providers are declared, all the
inheritance code in the graph can be removed.

The inheritance is dome by simply copying the RawConfig from the parent
ProviderConfig into the module. Since this happens before any
evaluation, we record the original interpolation scope in the
ProviderConfig so that it can be properly resolved later on.
2017-10-27 09:08:15 -04:00
James Bardin 29e5a355b9 add "version" and "provider" to module config
Add the Version and Providers fields to the module config.

Add ProviderConfig.Scope, which will be used to record the original
path of a ProviderConfig for interpolation.
2017-10-27 09:08:15 -04:00
Stefan Staudenmeyer 14792035cf Aliasing the module to omit errors thrown at build time
Signed-off-by: Stefan Staudenmeyer <stefan.staudenmeyer@instana.com>
2017-10-09 16:44:58 +02:00
James Bardin f6e89bb8c3 record the subdirectory in the FolderStorage
Module detection currently requires calling the registry to determine
the subdirectory. Since we're not directly accessing the subdirectory
through FolderStorage, and now handling it within terraform so modules can
reference sibling paths, we need to call out to the registry every
time we load a configuration to verify the subdirectory for the module,
which is returned during the Detect.

Record the subdirectories for each module in the top-level of the
FolderStorage path for retrieval during Tree.Load. This lets us bypass
Detection altogether, modules can be loaded without redetecting.
2017-09-22 22:03:38 -04:00
James Bardin 1b01f18920 handle go-getter subdirs in Tree.Load
In order to remain backward compatible with some modules, we need to
handle subdirs during Load. This means duplicating part of the go-getter
code path for subDir handling so we can resolve any subDirs and globs
internally, while keeping the entire remote directory structure within
the file storage.
2017-09-22 20:32:44 -04:00
James Bardin d216d19f21 change the FolderStorage key
updating the key will cause the FolderStorage hash to change forcing
modules to be re-fetched. This is required because any configurations
using the subDir notation will have the configuration in the wrong
directory.
2017-09-15 16:16:28 -04:00
James Bardin 6c20141c30 remove subdir handling from Tree.Load
Terraform was redundantly handling `//dir` notation which should be
handled by go-getter. Rather than allowing go-getter to unpack a subdir
as expected, the subdir was stripped off and accessed through the module
configuration.

This scheme will no longer works now that go-getter supports `*`
subdirectories
    (e.g. `//*` would be analogous to `tar --strip-components=1`).

Even though this allows Terraform to use go-getter's native unpacking,
detection is still done separately because Detect requires a `pwd` which
is dependent on the configuration directory and not known to the
global FolderStorage.
2017-09-15 16:16:28 -04:00
James Bardin a83ff57aea Add registry detector
Add a getter.Detector for detecting registry modules and looking up
the download location of the latest version. This is essentially a
temporary API until constraint solving is supported by the registry, as
then we'll have to supply the full set of known contraints to the
registry at once for resolution and we will fetch specific versions of
modules.
2017-09-15 16:16:28 -04:00
Martin Atkins a8a64c66c0 config/module: helper to visit all modules in a tree 2017-06-09 14:03:59 -07:00
Mitchell Hashimoto 68ee4e0480
config/module: don't panic when referencing undefined module
Fixes #12788

We would panic when referencing an output from an undefined module. The
panic above this is correct but in this case Load will not catch
interpolated variables that _reference_ an unloaded/undefined module.
Test included.
2017-03-16 20:14:20 -07:00
James Bardin 2a949093ed report all errors from module validation
It can be tedious fixing a new module with many errors when Terraform
only outputs the first random error it encounters.

Accumulate all errors from validation, and format them for the user.
2017-03-02 15:16:05 -05:00
Mitchell Hashimoto 095b7e7831
config/module: disallow root modules named "root"
Fixes #11038

This is a **short term fix**.

Terraform core doesn't currently handle root modules named "root" well
because the prefix `[]string{"root"}` has special meaning and Terraform
core [currently] can't disambiguate between the root module and a module
named "root" in the root module.

This PR introduces a short term fix by simply disallowing root modules
named "root". This shouldn't break any BC because since 0.8.0 this
didn't work at all in many broken ways (including crashes).

Longer term, this should be fixed by removing the special prefix at all
and having empty paths be root. I started down this path but the core
changes necessary are far too scary for a patch release. We can aim for
0.9.
2017-01-08 15:39:57 -08:00
Mitchell Hashimoto 0c30caec7d
config: smarter provider alias usage validation
Fixes #4789

This improves the validation that valid provider aliases are used.

Previously, we required that provider aliases be defined in every module
they're used. This isn't correct because the alias may be used in a
parent module and inherited.

This removes that validation and creates the validation that a provider
alias must be defined in the used module or _any parent_. This allows
inheritance to work properly.

We've always had this type of validation for aliases because we believe
its a good UX tradeoff: typo-ing an alias is really painful, so we
require declaration of alias usage. It may add a small burden to
declare, but since relatively few aliases are used, it improves the
scenario where a user fat-fingers an alias name.
2016-12-16 16:47:32 -08:00
Mitchell Hashimoto 55ef966b88
config/module: tree.Child on a nil tree works 2016-10-19 13:38:49 -07:00
Mitchell Hashimoto 098225dc0d
config/module: use the raw source as part of the key
This changes the key for the storage to be the _raw_ source from the
module, not the fully expanded source. Example: it'll be a relative path
instead of an absolute path.

This allows the ".terraform/modules" directory to be portable when
moving to other machines. This was a behavior that existed in <= 0.7.2
and was broken with #8398. This amends that and adds a test to verify.
2016-09-30 10:44:46 -07:00
Mitchell Hashimoto 92336281a5
config/module: use source as part of key 2016-08-22 14:07:28 -07:00
Mitchell Hashimoto 27452f0043
terraform: Module option to Import to add module to graph 2016-05-11 13:02:37 -07:00
Mitchell Hashimoto 3c9a92e04a
command: Context buliding allows empty module trees 2016-05-11 13:02:34 -07:00
Jeff Zohrab 33d30761dd Add hint to download modules for new users. 2016-02-17 11:32:44 -05:00
Mitchell Hashimoto 06f4ac8166 config/module: use go-getter 2015-10-15 13:36:58 -07:00
Mitchell Hashimoto 2378160803 config/module: determine storage folder by path 2015-04-07 16:44:24 -07:00
Mitchell Hashimoto f084d8d932 config/module: store the path with the module 2015-04-07 16:37:46 -07:00
Mitchell Hashimoto 1e00b4386c config/module: Child(nil) or empty will return self 2014-10-07 20:02:18 -07:00
Mitchell Hashimoto 267d45df86 config/module: Can look up Child with Tree.Child 2014-10-07 20:00:36 -07:00
Mitchell Hashimoto 3941dbac5d fmt 2014-09-28 11:51:49 -07:00
Mitchell Hashimoto 4fbd5abc63 config/module: support GitHub paths without // 2014-09-26 15:30:36 -07:00
Mitchell Hashimoto 7aa30fdf25 config: support subdirectories with the "//" syntax 2014-09-26 14:56:04 -07:00
Mitchell Hashimoto 1ef167602e config/module: validate that required parameters are passed through 2014-09-24 19:40:06 -07:00