resilien
/
terraform
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

website: updated docs for the module registry protocol 

Now includes more complete information on usage of private registries and
updates the module registry API documentation to include the new
version-enumeration endpoint along with describing Terraform's discovery
protocol.
This commit is contained in:
Martin Atkins 2017-10-31 16:05:22 -07:00
parent 56a9b1309a
commit 4fa2b8c519
5 changed files with 329 additions and 67 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

112
website/docs/internals/remote-service-discovery.html.md Normal file
View File

@ -0,0 +1,112 @@
---
layout: "docs"
page_title: "Internals: Remote Service Discovery"
sidebar_current: "docs-remote-service-discovery"
description: |-
Remote service discovery is a protocol used to locate Terraform-native
services provided at a user-friendly hostname.
---
# Remote Service Discovery
Terraform implements much of its functionality in terms of remote services.
While in many cases these are generic third-party services that are useful
to many applications, some of these services are tailored specifically to
Terraform's needs. We call these _Terraform-native services_, and Terraform
interacts with them via the remote service discovery protocol described below.
## User-facing Hostname
Terraform-native services are provided, from a user's perspective, at a
user-facing "friendly hostname" which serves as the key for configuration and
for any authentication credentials required.
The discovery protocol's purpose is to map from a user-provided hostname to
the base URL of a particular service. Each host can provide different
combinations of services -- or no services at all! -- and so the discovery
protocol has a secondary purpose of allowing Terraform to identify _which_
services are valid for a given hostname.
For example, module source strings can include a module registry hostname
as their first segment, like `example.com/namespace/name/provider`, and
Terraform uses service discovery to determine whether `example.com` _has_
a module registry, and if so where its API is available.
A user-facing hostname is a fully-specified
[internationalized domain name](https://en.wikipedia.org/wiki/Internationalized_domain_name)
expressed in its Unicode form (the corresponding "punycode" form is not allowed)
which must be resolvable in DNS to an address that has an HTTPS server running
on port 443.
User-facing hostnames are normalized for internal comparison using the
standard Unicode [Nameprep](https://en.wikipedia.org/wiki/Nameprep) algorithm,
which includes converting all letters to lowercase, normalizing combining
diacritics to precomposed form where possible, and various other normalization
steps.
## Discovery Process
Given a hostname, discovery begins by forming an initial discovery URL
using that hostname with the `https:` scheme and the fixed path
`/.well-known/terraform.json`.
For example, given the hostname `example.com` the initial discovery URL
would be `https://example.com/.well-known/terraform.json`.
Terraform then sends a `GET` request to this discovery URL and expects a
JSON response. If the response does not have status 200, does not have a media
type of `application/json` or, if the body cannot be parsed as a JSON object,
then discovery fails and Terraform considers the host to not support _any_
Terraform-native services.
If the response is an HTTP redirect then Terraform repeats this step with the
new location as its discovery URL. Terraform is guaranteed to follow at least
one redirect, but nested redirects are not guaranteed nor recommended.
If the response is a valid JSON object then its keys are Terraform native
service identifiers, consisting of a service type name and a version string
separated by a period. For example, the service identifier for version 1 of
the module registry protocol is `modules.v1`.
The value of each object element is the base URL for the service in question.
This URL may be either absolute or relative, and if relative it is resolved
against the final discovery URL (_after_ following redirects).
The following is an example discovery document declaring support for
version 1 of the module registry protocol:
```json
{
"modules.v1": "https://modules.example.com/v1/"
}
```
## Supported Services
At present, only one service identifier is in use:
* `modules.v1`: [module registry API version 1](/docs/registry/api.html)
## Authentication
If credentials for the given hostname are available in
[the CLI config](/docs/commands/cli-config.html) then they will be included
in the request for the discovery document.
The credentials may also be provided to endpoints declared in the discovery
document, depending on the requirements of the service in question.
## Non-standard Ports in User-facing Hostnames
It is strongly recommended to provide the discovery document for a hostname
on the standard HTTPS port 443. However, in development environments this is
not always possible or convenient, so Terraform allows a hostname to end
with a port specification consisting of a colon followed by one or more
decimal digits.
When a custom port number is present, the service on that port is expected to
implement HTTPS and respond to the same fixed discovery path.
For day-to-day use it is strongly recommended _not_ to rely on this mechanism
and to instead provide the discovery document on the standard port, since this
allows use of the most user-friendly hostname form.

235
website/docs/registry/api.html.md
View File

@ -8,13 +8,183 @@ description: |-
# HTTP API
The [Terraform Registry](https://registry.terraform.io) has an HTTP API for
reading and downloading registry modules.
When downloading modules from registry sources such as the public
[Terraform Registry](https://registry.terraform.io), Terraform expects
the given hostname to support the following module registry protocol.
Terraform interacts with the registry as read-only. Therefore, the documented
API is read-only. Any endpoints that aren't documented on this page can and will
likely change over time. This allows differing methods for getting modules into
the registry while keeping a consistent API for reading modules in the registry.
A registry module source is of the form `hostname/namespace/name/provider`,
where the initial hostname portion is implied to be `registry.terraform.io/`
if not specified. The public Terraform Registry is therefore the default
module source.
[Terraform Registry](https://registry.terraform.io) implements a superset
of this API to allow for importing new modules, etc, but any endpoints not
documented on this page are subject to change over time.
## Service Discovery
The hostname portion of a module source string is first passed to
[the service discovery protocol](/docs/internals/remote-service-discovery.html)
to determine if the given host has a module registry and, if so, the base
URL for its module registry endpoints.
The service identifier for this protocol is `modules.v1`, and the declared
URL should always end with a slash such that the paths shown in the following
sections can be appended to it.
For example, if discovery produces the URL `https://modules.example.com/v1/`
then this API would use full endpoint URLs like
`https://modules.example.com/v1/{namespace}/{name}/{provider}/versions`.
The example request URLs shown in this document are for the public
[Terraform Registry](https://registry.terraform.io), and use its API base
URL of `https://registry.terraform.io/v1/modules/` .
## List Available Versions for a Specific Module
This is the primary endpoint for resolving module sources, returning the
available versions for a given fully-qualified module.
| Method | Path | Produces |
| ------ | ------------------------------------- | -------------------------- |
| `GET` | `:namespace/:name/:provider/versions` | `application/json` |
### Parameters
- `namespace` `(string: <required>)` - The user or organization the module is
owned by. This is required and is specified as part of the URL path.
- `name` `(string: <required>)` - The name of the module.
This is required and is specified as part of the URL path.
- `provider` `(string: <required>)` - The name of the provider.
This is required and is specified as part of the URL path.
### Sample Request
```text
$ curl https://registry.terraform.io/v1/modules/hashicorp/consul/aws/versions
```
### Sample Response
The `modules` array in the response always includes the requested module as the
first element. Other elements of this list, if present, are dependencies of the
requested module that are provided to potentially avoid additional requests to
resolve these modules.
Additional modules are not required to be provided but, when present, can be
used by Terraform to optimize the module installation process.
Each returned module has an array of available versions, which Terraform
matches against any version constraints given in configuration.
```json
{
"modules": [
{
"source": "hashicorp/consul/aws",
"versions": [
{
"version": "0.0.1",
"submodules" : [
{
"path": "modules/consul-cluster",
"providers": [
{
"name": "aws",
"version": ""
}
],
"dependencies": []
},
{
"path": "modules/consul-security-group-rules",
"providers": [
{
"name": "aws",
"version": ""
}
],
"dependencies": []
},
{
"providers": [
{
"name": "aws",
"version": ""
}
],
"dependencies": [],
"path": "modules/consul-iam-policies"
}
],
"root": {
"dependencies": [],
"providers": [
{
"name": "template",
"version": ""
},
{
"name": "aws",
"version": ""
}
]
}
}
]
}
]
}
```
## Download Source Code for a Specific Module Version
This endpoint downloads the specified version of a module for a single provider.
A successful response has no body, and includes the location from which the module
version's source can be downloaded in the `X-Terraform-Get` header. Note that
this string may contain special syntax interpreted by Terraform via
[`go-getter`](https://github.com/hashicorp/go-getter). See the [`go-getter`
documentation](https://github.com/hashicorp/go-getter#url-format) for details.
The value of `X-Terraform-Get` may instead be a relative URL, indicated by
beginning with `/`, `./` or `../`, in which case it is resolved relative to
the full URL of the download endpoint.
| Method | Path | Produces |
| ------ | ---------------------------- | -------------------------- |
| `GET` | `:namespace/:name/:provider/:version/download` | `application/json` |
### Parameters
- `namespace` `(string: <required>)` - The user the module is owned by.
This is required and is specified as part of the URL path.
- `name` `(string: <required>)` - The name of the module.
This is required and is specified as part of the URL path.
- `provider` `(string: <required>)` - The name of the provider.
This is required and is specified as part of the URL path.
- `version` `(string: <required>)` - The version of the module.
This is required and is specified as part of the URL path.
### Sample Request
```text
$ curl -i \
https://registry.terraform.io/v1/modules/hashicorp/consul/aws/0.0.1/download
```
### Sample Response
```text
HTTP/1.1 204 No Content
Content-Length: 0
X-Terraform-Get: https://api.github.com/repos/hashicorp/terraform-aws-consul/tarball/v0.0.1//*?archive=tar.gz
```
## List Latest Version of Module for All Providers
@ -22,7 +192,7 @@ This endpoint returns the latest version of each provider for a module.
| Method | Path | Produces |
| ------ | ---------------------------- | -------------------------- |
| `GET` | `/v1/modules/:namespace/:name` | `application/json` |
| `GET` | `:namespace/:name` | `application/json` |
### Parameters
@ -82,13 +252,13 @@ $ curl \
}
```
## Latest Module for a Single Provider
## Latest Version for a Specific Module Provider
This endpoint returns the latest version of a module for a single provider.
| Method | Path | Produces |
| ------ | ---------------------------- | -------------------------- |
| `GET` | `/v1/modules/:namespace/:name/:provider` | `application/json` |
| `GET` | `:namespace/:name/:provider` | `application/json` |
### Parameters
@ -210,7 +380,7 @@ This endpoint returns the specified version of a module for a single provider.
| Method | Path | Produces |
| ------ | ---------------------------- | -------------------------- |
| `GET` | `/v1/modules/:namespace/:name/:provider/:version` | `application/json` |
| `GET` | `:namespace/:name/:provider/:version` | `application/json` |
### Parameters
@ -330,49 +500,6 @@ Note this response has has some fields trimmed for clarity.
}
```
## Download a Specific Module
This endpoint downloads the specified version of a module for a single provider.
A successful response has no body, and includes the URL from which the module
version's source can be downloaded in the `X-Terraform-Get` header. Note that
this URL may contain special syntax interpreted by Terraform via
[`go-getter`](https://github.com/hashicorp/go-getter). See the [`go-getter`
documentation](https://github.com/hashicorp/go-getter#url-format) for details.
| Method | Path | Produces |
| ------ | ---------------------------- | -------------------------- |
| `GET` | `/v1/modules/:namespace/:name/:provider/:version/download` | `application/json` |
### Parameters
- `namespace` `(string: <required>)` - The user the module is owned by.
This is required and is specified as part of the URL path.
- `name` `(string: <required>)` - The name of the module.
This is required and is specified as part of the URL path.
- `provider` `(string: <required>)` - The name of the provider.
This is required and is specified as part of the URL path.
- `version` `(string: <required>)` - The version of the module.
This is required and is specified as part of the URL path.
### Sample Request
```text
$ curl -i \
https://registry.terraform.io/v1/modules/hashicorp/consul/aws/0.0.1/download
```
### Sample Response
```text
HTTP/1.1 204 No Content
Content-Length: 0
X-Terraform-Get: https://api.github.com/repos/hashicorp/terraform-aws-consul/tarball/v0.0.1//*?archive=tar.gz
```
## Download the Latest Version of a Module
This endpoint downloads the latest version of a module for a single provider.
@ -382,7 +509,7 @@ download endpoint (above) for the latest version.
| Method | Path | Produces |
| ------ | ---------------------------- | -------------------------- |
| `GET` | `/v1/modules/:namespace/:name/:provider/download` | `application/json` |
| `GET` | `:namespace/:name/:provider/download` | `application/json` |
### Parameters

8
website/docs/registry/index.html.md
View File

@ -29,9 +29,9 @@ module "consul" {
You can also publish your own modules on the Terraform Registry. You may
use the [public registry](https://registry.terraform.io) for public modules.
For private modules, you must use [Terraform Enterprise](https://www.hashicorp.com/products/terraform).
You can use modules without a registry by
[sourcing modules directly](/docs/modules/sources.html), however non-registry
modules do not support versioning, documentation generation, and more.
For private modules, you can use a [Private Registry](/docs/registry/private.html),
or [reference repositories and other sources directly](/docs/modules/sources.html).
Some features are available only for registry modules, such as versioning
and documentation generation.
Use the navigation to the left to learn more about using the registry.

37
website/docs/registry/private.html.md
View File

@ -12,17 +12,36 @@ The registry at [registry.terraform.io](https://registry.terraform.io)
may only host public modules. Terraform is capable of loading modules from
private registries for private modules.
Official private registries are available via [Terraform Enterprise](#).
Official private registries are available via [Terraform Enterprise](https://www.hashicorp.com/products/terraform).
There are two tiers: Pro and Enterprise. The Pro version is only available
as a SaaS service whereas the Enterprise version is available for private
install. Both versions fully support private registries.
The Terraform project does not provide any free or open source solution to have
a private registry. Terraform only requires the [read API](/docs/registry/api.html)
to be available to load modules from a registry.
Support for specifying an alternative to the public registry will be available
in Terraform 0.11. We welcome the community to create their own private
registries by recreating this API.
Terraform interacts with module registries using [the registry API](/docs/registry/api.html).
The Terraform open source project does not provide a server implementation, but
we welcome community members to create their own private registries by following
the published protocol.
Modules can alternatively be referenced
[directly from version control and other sources](/docs/modules/sources.html),
but only registry modules support certain features such as
[version constraints](/docs/modules/usage.html#module-versions).
## Private Registry Module Sources
Public Terraform Registry modules have source strings of the form
`namespace/name/provider`. Private registries -- whether integrated into
Terraform Enterprise or via a third-party implementation -- require an
additional hostname prefix:
```hcl
module "example" {
source = "example.com/namespace/name/provider"
}
```
Private registry module sources are supported in Terraform v0.11.0 and
newer.
## Coming Soon
@ -33,5 +52,5 @@ by the end of 2017. In the mean time, if you're interested in private
registries and being part of the beta, please contact us at
[hello@hashicorp.com](mailto:hello@hashicorp.com).
When Terraform Enterprise is publicly available, the documentation will
be available here.
When Terraform Enterprise is publicly available, Private Registry documentation
will be available here.

4
website/layouts/docs.erb
View File

@ -589,6 +589,10 @@
<a href="/docs/internals/resource-addressing.html">Resource Addressing</a>
</li>
<li<%= sidebar_current("docs-remote-service-discovery") %>>
<a href="/docs/internals/remote-service-discovery.html">Remote Service Discovery</a>
</li>
<li<%= sidebar_current("docs-internals-plugins") %>>
<a href="/docs/internals/internal-plugins.html">Internal Plugins</a>
</li>