Commit Graph

161 Commits

Author SHA1 Message Date
James Bardin a359c4ee2f check the registry detector before local files
Breaking change for 0.11.

Local files were checked first to avoid the possibility of breaking a
module with a local source that looked like a registry ID. Now we can
enfore that any source iwth the pattern "namespace/identifier/provider"
must be a registry module.
2017-10-27 09:08:15 -04:00
James Bardin 6088270c66 skip lookup for paths starting with ../
Prevent an extra registry lookup for sources starting with ../
2017-10-27 09:08:15 -04:00
James Bardin b0163e69d1 update comments 2017-10-27 09:08:15 -04:00
James Bardin 4ab70ba09d rename confusing test module name 2017-10-27 09:08:15 -04:00
James Bardin 2e505083cd change ProviderConfig.Scope to Path
Though it's intended for "interpolation scope", Path is generally used
for this elsewhere.
2017-10-27 09:08:15 -04:00
James Bardin cb0e37a870 implement provider inheritence during loading
This implements provider inheritance during config loading, rather than
during graph evaluation. At this point it's much simpler to find the
desired configuration, and once all providers are declared, all the
inheritance code in the graph can be removed.

The inheritance is dome by simply copying the RawConfig from the parent
ProviderConfig into the module. Since this happens before any
evaluation, we record the original interpolation scope in the
ProviderConfig so that it can be properly resolved later on.
2017-10-27 09:08:15 -04:00
James Bardin 29e5a355b9 add "version" and "provider" to module config
Add the Version and Providers fields to the module config.

Add ProviderConfig.Scope, which will be used to record the original
path of a ProviderConfig for interpolation.
2017-10-27 09:08:15 -04:00
Stefan Staudenmeyer 14792035cf Aliasing the module to omit errors thrown at build time
Signed-off-by: Stefan Staudenmeyer <stefan.staudenmeyer@instana.com>
2017-10-09 16:44:58 +02:00
James Bardin 1fbb2d611a remove extra slash from registry urls
A refactor introduced an extra `/` in the download url, which causes an
extra redirect during discovery.

Improve a registry test to verify that detection doesn't require the
registry after the modules have been fetched.
2017-09-28 14:45:02 -04:00
James Bardin 2b3e48aa70 fix test comments 2017-09-26 09:12:14 -04:00
James Bardin f6e89bb8c3 record the subdirectory in the FolderStorage
Module detection currently requires calling the registry to determine
the subdirectory. Since we're not directly accessing the subdirectory
through FolderStorage, and now handling it within terraform so modules can
reference sibling paths, we need to call out to the registry every
time we load a configuration to verify the subdirectory for the module,
which is returned during the Detect.

Record the subdirectories for each module in the top-level of the
FolderStorage path for retrieval during Tree.Load. This lets us bypass
Detection altogether, modules can be loaded without redetecting.
2017-09-22 22:03:38 -04:00
James Bardin 1b01f18920 handle go-getter subdirs in Tree.Load
In order to remain backward compatible with some modules, we need to
handle subdirs during Load. This means duplicating part of the go-getter
code path for subDir handling so we can resolve any subDirs and globs
internally, while keeping the entire remote directory structure within
the file storage.
2017-09-22 20:32:44 -04:00
James Bardin 59089ef461 add test loading an archive with `//*` subdir
The registry uses the `//*` subdir format to unpack github archives. Add
a test to cover this pattern.
2017-09-21 15:32:55 -05:00
James Bardin 38569c8508 add tests for get from tar subdir
Test that we can get a subdirectory from a tarball (or any other
"packed" source that we support).

The 'tar-subdir-to-parent' test highlights a regression where the
subdirectory module references a module in its parent directory. This
breaks the intended use ofr the subdirectory and the implementation in
go-getter. We need to fix this in terraform, and possible plan warnings
and deprecations for this type of source.
2017-09-21 13:12:33 -05:00
James Bardin 3d3992454e Add basic ACC test
This just does a lookup in the registry to verify that the download api
works.
2017-09-15 16:16:29 -04:00
James Bardin ba14cf9511 add test fixture for new getter discovery behavior 2017-09-15 16:16:29 -04:00
James Bardin f9fb2b4c9e move local module detection ahead of the registry
The getter.FileDetector was intended to be the final detector, only
converting a path to a file URL and returning a true in all cases. We
want to check for a local module before checking the registry so no
local modules that happen to match a registry module are broken.

Wrap the getter.FileDetector to check the module source's existence
before delegating the search to the registry.
2017-09-15 16:16:28 -04:00
James Bardin d216d19f21 change the FolderStorage key
updating the key will cause the FolderStorage hash to change forcing
modules to be re-fetched. This is required because any configurations
using the subDir notation will have the configuration in the wrong
directory.
2017-09-15 16:16:28 -04:00
James Bardin 6c20141c30 remove subdir handling from Tree.Load
Terraform was redundantly handling `//dir` notation which should be
handled by go-getter. Rather than allowing go-getter to unpack a subdir
as expected, the subdir was stripped off and accessed through the module
configuration.

This scheme will no longer works now that go-getter supports `*`
subdirectories
    (e.g. `//*` would be analogous to `tar --strip-components=1`).

Even though this allows Terraform to use go-getter's native unpacking,
detection is still done separately because Detect requires a `pwd` which
is dependent on the configuration directory and not known to the
global FolderStorage.
2017-09-15 16:16:28 -04:00
James Bardin a83ff57aea Add registry detector
Add a getter.Detector for detecting registry modules and looking up
the download location of the latest version. This is essentially a
temporary API until constraint solving is supported by the registry, as
then we'll have to supply the full set of known contraints to the
registry at once for resolution and we will fetch specific versions of
modules.
2017-09-15 16:16:28 -04:00
Martin Atkins a8a64c66c0 config/module: helper to visit all modules in a tree 2017-06-09 14:03:59 -07:00
Mitchell Hashimoto 68ee4e0480
config/module: don't panic when referencing undefined module
Fixes #12788

We would panic when referencing an output from an undefined module. The
panic above this is correct but in this case Load will not catch
interpolated variables that _reference_ an unloaded/undefined module.
Test included.
2017-03-16 20:14:20 -07:00
James Bardin 2a949093ed report all errors from module validation
It can be tedious fixing a new module with many errors when Terraform
only outputs the first random error it encounters.

Accumulate all errors from validation, and format them for the user.
2017-03-02 15:16:05 -05:00
John Murphy 628d46ac53 Fixed broken build for netbsd 2017-02-22 00:30:18 +08:00
Mitchell Hashimoto 7b342100d0
config: add "backend" loading to the Terraform section 2017-01-26 14:33:49 -08:00
Mitchell Hashimoto 095b7e7831
config/module: disallow root modules named "root"
Fixes #11038

This is a **short term fix**.

Terraform core doesn't currently handle root modules named "root" well
because the prefix `[]string{"root"}` has special meaning and Terraform
core [currently] can't disambiguate between the root module and a module
named "root" in the root module.

This PR introduces a short term fix by simply disallowing root modules
named "root". This shouldn't break any BC because since 0.8.0 this
didn't work at all in many broken ways (including crashes).

Longer term, this should be fixed by removing the special prefix at all
and having empty paths be root. I started down this path but the core
changes necessary are far too scary for a patch release. We can aim for
0.9.
2017-01-08 15:39:57 -08:00
Mitchell Hashimoto 0c30caec7d
config: smarter provider alias usage validation
Fixes #4789

This improves the validation that valid provider aliases are used.

Previously, we required that provider aliases be defined in every module
they're used. This isn't correct because the alias may be used in a
parent module and inherited.

This removes that validation and creates the validation that a provider
alias must be defined in the used module or _any parent_. This allows
inheritance to work properly.

We've always had this type of validation for aliases because we believe
its a good UX tradeoff: typo-ing an alias is really painful, so we
require declaration of alias usage. It may add a small burden to
declare, but since relatively few aliases are used, it improves the
scenario where a user fat-fingers an alias name.
2016-12-16 16:47:32 -08:00
Mitchell Hashimoto 55ef966b88
config/module: tree.Child on a nil tree works 2016-10-19 13:38:49 -07:00
Mitchell Hashimoto 098225dc0d
config/module: use the raw source as part of the key
This changes the key for the storage to be the _raw_ source from the
module, not the fully expanded source. Example: it'll be a relative path
instead of an absolute path.

This allows the ".terraform/modules" directory to be portable when
moving to other machines. This was a behavior that existed in <= 0.7.2
and was broken with #8398. This amends that and adds a test to verify.
2016-09-30 10:44:46 -07:00
Mitchell Hashimoto 92336281a5
config/module: use source as part of key 2016-08-22 14:07:28 -07:00
James Nugent 796ce7bdd7 build: Opt-in to building rather than opt-out 2016-07-29 18:40:31 -05:00
James Nugent 2eb00c9184 build: Fix errors in FreeBSD build
Fixes the following error when cross compiling:

```
--> freebsd/amd64 error: exit status 2
Stderr: # github.com/hashicorp/terraform/config/module
config/module/inode.go:18: cannot use st.Ino (type uint32) as type uint64 in return argument
```
2016-07-29 18:26:22 -05:00
James Bardin 525485c213 Add proper build constraints for GH-7273
The syscall.Stat_t type doesn't exist on windows, so the inode lookup
needs to be in a file with proper build constraints.
2016-06-23 08:02:32 -04:00
James Bardin 558e023e26 Don't try to copy files over themselves
When copying a config module, make sure the full path for src and dst
files don't match, and also check the inode in case we resolved a
different path to the same file.

Make a note about the unsafe usage of reusing a tempDir path.
2016-06-22 11:25:42 -04:00
Mitchell Hashimoto 27452f0043
terraform: Module option to Import to add module to graph 2016-05-11 13:02:37 -07:00
Mitchell Hashimoto 3c9a92e04a
command: Context buliding allows empty module trees 2016-05-11 13:02:34 -07:00
Jeff Zohrab 33d30761dd Add hint to download modules for new users. 2016-02-17 11:32:44 -05:00
Paul Hinze 87a9701f91 config: validation error when output is missing value field
Also lists out invalid keys in errmsg when they are present

Closes #4398
2016-01-20 14:00:36 -06:00
Mitchell Hashimoto 344e7c26b5 fix a bunch of tests from go-getter import 2015-10-15 13:48:58 -07:00
Mitchell Hashimoto 06f4ac8166 config/module: use go-getter 2015-10-15 13:36:58 -07:00
Panagiotis Moustafellos e4845f75cc removed extra parentheses 2015-10-08 15:48:04 +03:00
Mitchell Hashimoto 1099e3f59f config: add module raw configs to InterpolatedConfigs [GH-1448] 2015-04-29 17:12:28 -07:00
Radek Simko 8eb6984a89 Test helpers to print expected output 2015-04-13 14:12:43 +01:00
Mitchell Hashimoto aab7bc0da5 config/module: rename parameters 2015-04-07 17:07:50 -07:00
Mitchell Hashimoto 2378160803 config/module: determine storage folder by path 2015-04-07 16:44:24 -07:00
Mitchell Hashimoto f084d8d932 config/module: store the path with the module 2015-04-07 16:37:46 -07:00
Mitchell Hashimoto 44fce5ce60 config/module: remove debug 2015-03-26 09:31:58 -07:00
Mitchell Hashimoto 2e11ca68df config/module: go back to the original folder when doing parent
references
2015-03-26 09:30:32 -07:00
Mitchell Hashimoto bd4aaac71a config/module: failing unit test for GH-1232 2015-03-26 09:11:32 -07:00
Mitchell Hashimoto cab2e408d2 config/module: retry bitbucket test a few times [GH-1027] 2015-02-23 15:24:24 -08:00
Emil Hessman e7bbbfb098 helper/url: add Windows 'safe' URL Parse wrapper
Pull out the urlParse function, which was introduced in config/module,
into a helper package.
2015-02-05 11:16:54 +01:00
Emil Hessman a8db835a44 config/module: fix HgGetter test failures on Windows
HgGetter tests failed on windows/amd64 using Mercurial version 3.2.4:

--- FAIL: TestHgGetter (0.11s)
        get_hg_test.go:35: err: C:\Program Files\Mercurial\hg.exe exited with 255: abort: file:// URLs can only refer to localhost
--- FAIL: TestHgGetter_branch (0.11s)
        get_hg_test.go:62: err: C:\Program Files\Mercurial\hg.exe exited with 255: abort: file:// URLs can only refer to localhost
FAIL
FAIL    github.com/hashicorp/terraform/config/module    5.615s

This commit fixes the failures by adjusting the file:// URL to a form that
Mercurial expects.
2015-02-02 19:49:57 +01:00
Emil Hessman 5bbfc0d4e2 config/module: fix URL file path handling on Windows
Only adjust the URL Scheme when parsing drive letter file paths on
Windows, don't add a file scheme prefix.
FileDetector is responsible for adding the file scheme prefix.
2015-02-02 12:19:34 +01:00
Mitchell Hashimoto 2d9dd25493 config/module: fix regression of ignoring forced type if valid URL
/cc @ceh - Does this break Windows at all? This regressed with your
commit (we didn't have test coverage on it). I added a test now, though.
2015-02-01 20:17:56 +01:00
Emil Hessman 97227a5c70 config/module: fix detection of file paths on Windows
Absolute file paths were not correctly detected by module.Detect
when using url.Parse to parse the source URL. Wrap the detection
with urlParse to properly handle file path detections on Windows.

Fixes command test failures on Windows.
2015-01-28 10:40:32 +01:00
Emil Hessman 65177edd1e config/module: fix test failures on Windows
When parsing URLs on Windows, assume it is a drive letter path
if the second element is a ':' character. Format the drive letter
path as a "file:///"-path prior to parsing the URL.

Fixes test failures of the following form in command on Windows:

=== RUN TestApply_plan
--- FAIL: TestApply_plan (0.00s)
        apply_test.go:379: bad: 1

		module download not supported for scheme 'c'
2015-01-28 08:19:04 +01:00
Emil Hessman d5a49363d7 config/module: handle absolute file paths on Windows
Using url.Parse to parse an absolute file path on Windows yields
a URL type where the Path element is prefixed by a slash.

For example, parsing "file:///C:/Users/user" gives a URL type
with Path:"/C:/Users/user".

According to golang.org/issue/6027, the parsing is correct as is.

The leading slash on the Path must be eliminated before any file
operations.

This commit introduces a urlParse function which wraps the url.Parse
functionality and removes the leading slash in Path for absolute file
paths on Windows.

Fixes config/module test failures on Windows.
2015-01-27 23:18:47 +01:00
Emil Hessman 78d1fc742f config/module: adjust HttpGetter test to fix Windows test failure
Specify laddr on the form host:port in the call to net.Listen as
documented for net.Dial, see godoc.org/net#Dial

Fixes the following test failures on Windows:

> go test -run=TestHttpGetter
--- FAIL: TestHttpGetter_header (0.00s)
        get_http_test.go:31: err: Get http://[::]:52101/header?terraform-get=1: dial tcp [::]:52101: ConnectEx tcp: The requested address is not valid in its context.
--- FAIL: TestHttpGetter_meta (0.00s)
        get_http_test.go:55: err: Get http://[::]:52103/meta?terraform-get=1: dial tcp [::]:52103: ConnectEx tcp: The requested address is not valid in its context.
--- FAIL: TestHttpGetter_metaSubdir (0.00s)
        get_http_test.go:79: err: Get http://[::]:52105/meta-subdir?terraform-get=1: dial tcp [::]:52105: ConnectEx tcp: The requested address is not valid in its context.
FAIL
exit status 1
FAIL github.com/hashicorp/terraform/config/module    0.054s
2015-01-27 23:18:46 +01:00
Emil Hessman 74cf8fcabd config/module: adjust FileDetector tests for Windows
"/foo" is not an absolute path on Windows. Adjust the FileDetector
tests to take that into account when verifying the results.

Fixes FileDetector test failures on Windows.
2015-01-27 23:18:46 +01:00
Emil Hessman 160e4f926e config/module: fix panic on Windows when running tests
On Windows, Go returns paths with the `\` path separator.
Make sure we are using `/` even on Windows since URLs are `/`-based.
2015-01-27 07:52:51 +01:00
Seth Vargo 94e1eac051 Merge pull request #547 from ceh/issue-518
Fix issue 518
2014-11-20 13:30:14 -05:00
Emil Hessman a76290f278 config/module: don't walk dst when dst is in src
Fixes #518
2014-11-06 13:45:01 +01:00
Emil Hessman 4bfe18b40d argument and verb formatting fixes reported by go vet
builtin/providers/aws/tags_test.go:56: unrecognized printf verb 'i'
builtin/providers/aws/tags_test.go:59: unrecognized printf verb 'i'

config/config_test.go:101: possible formatting directive in Fatal call
config/config_test.go:157: possible formatting directive in Fatal call

config/module/get_file_test.go:91: missing argument for Fatalf(%s): format reads arg 1, have only 0 args

helper/schema/schema.go:341: arg v.Type for printf verb %s of wrong type: schema.ValueType
helper/schema/schema.go:656: missing argument for Errorf(%s): format reads arg 2, have only 1 args
helper/schema/schema.go:912: arg schema.Type for printf verb %s of wrong type: schema.ValueType

terraform/context.go:178: arg v.Type() for printf verb %s of wrong type: github.com/hashicorp/terraform/config.VariableType
terraform/context.go:486: arg c.Operation for printf verb %s of wrong type: terraform.walkOperation

terraform/diff_test.go💯 arg actual for printf verb %s of wrong type: terraform.DiffChangeType
terraform/diff_test.go:235: arg actual for printf verb %s of wrong type: terraform.DiffChangeType
2014-11-02 13:56:44 +01:00
Mitchell Hashimoto 369ba8cf07 config/module: fix issue where Get was copying subdir contents twice 2014-10-13 17:38:27 -07:00
Mitchell Hashimoto 6ee9c06cdc config/module: copyDir properly copies files in subdirectories 2014-10-12 21:12:42 -07:00
Mitchell Hashimoto 1e00b4386c config/module: Child(nil) or empty will return self 2014-10-07 20:02:18 -07:00
Mitchell Hashimoto 267d45df86 config/module: Can look up Child with Tree.Child 2014-10-07 20:00:36 -07:00
Mitchell Hashimoto 5c2182e316 config/module: only return SkipDir if it is dot-prefixed 2014-10-03 13:48:08 -07:00
Mitchell Hashimoto f35ebe7032 config/module: walk actually recurses into dirs, don't copy dot-prefix 2014-10-03 13:46:19 -07:00
Mitchell Hashimoto 3941dbac5d fmt 2014-09-28 11:51:49 -07:00
Mitchell Hashimoto 0a2d06268c config/module: detect supports subdirs 2014-09-27 09:29:12 -07:00
Mitchell Hashimoto 9689a34b28 config/module: GetCopy 2014-09-26 16:21:33 -07:00
Mitchell Hashimoto c91fd76fe8 config/module: Get can support subdirs 2014-09-26 16:11:13 -07:00
Mitchell Hashimoto 4fbd5abc63 config/module: support GitHub paths without // 2014-09-26 15:30:36 -07:00
Mitchell Hashimoto 21d90dcf4f config/module: HTTP supports subdirs 2014-09-26 15:22:26 -07:00
Mitchell Hashimoto 7aa30fdf25 config: support subdirectories with the "//" syntax 2014-09-26 14:56:04 -07:00
Mitchell Hashimoto 1ef167602e config/module: validate that required parameters are passed through 2014-09-24 19:40:06 -07:00
Mitchell Hashimoto 718fb42f4b terraform: Plan should use module.Tree 2014-09-24 14:56:48 -07:00
Mitchell Hashimoto 97da02c368 config/module: proper name for root 2014-09-22 16:39:01 -07:00
Mitchell Hashimoto ed538a9594 command: Get command, not functional yet. Converted to use modules. 2014-09-22 10:56:50 -07:00
Mitchell Hashimoto 9a626b3e8c config/module: support HTTP protocol 2014-09-16 13:44:12 -07:00
Mitchell Hashimoto fde151978e config/module: parallelize some things 2014-09-16 12:02:35 -07:00
Mitchell Hashimoto 27564fff2b config/module: detect BitBucket URLs 2014-09-16 11:55:14 -07:00
Mitchell Hashimoto 5480eb4e41 config/module: detect preserves forces 2014-09-16 10:54:23 -07:00
Mitchell Hashimoto 2a655bc7d9 config/module: detect GitHub URLs 2014-09-16 10:52:08 -07:00
Mitchell Hashimoto 9c74d6b5c0 config/module: hg supports branches/tags/etc. 2014-09-16 10:30:55 -07:00
Mitchell Hashimoto feb9a36597 config/module: tests to verify that params are preserved on files 2014-09-16 10:06:44 -07:00
Mitchell Hashimoto ac19a488d2 config/module: support branches in git 2014-09-16 10:02:11 -07:00
Mitchell Hashimoto 13d892217c config/module: fix some issues where tag re-pulling didnt' work 2014-09-16 09:59:09 -07:00
Mitchell Hashimoto dcb900470c config/module: git supports tags 2014-09-16 09:55:51 -07:00
Mitchell Hashimoto 7e94f7d4a9 config/module: Mercurial support 2014-09-16 09:34:23 -07:00
Mitchell Hashimoto fc71d7091f config/module: Git test... this is kind of ghetto 2014-09-16 00:03:20 -07:00
Mitchell Hashimoto 96385113e7 config/module: delete weird subproject business 2014-09-15 23:55:07 -07:00
Mitchell Hashimoto 3e2989daf1 config/module: test Git 2014-09-15 23:52:27 -07:00
Mitchell Hashimoto acb6d12a75 config/module: support forced getters with TYPE::URL syntax 2014-09-15 23:48:56 -07:00
Mitchell Hashimoto cf4885d2fd config/module: git support 2014-09-15 23:32:30 -07:00
Mitchell Hashimoto 7bbf6a0d3a config/module: NewTreeModule is easier to use 2014-09-15 15:49:07 -07:00
Mitchell Hashimoto c0a30d3337 config: TestString 2014-09-15 15:43:12 -07:00
Mitchell Hashimoto 292f57ea0a config/module: validate outputs line up with ModuleVariables 2014-09-15 13:57:07 -07:00
Mitchell Hashimoto 12e7c75211 config/module: validate that parameters into modules valid 2014-09-15 10:32:41 -07:00