While the TLS handshakes are a fairly small overhead compared to
downloading the providers, clients in some situation are failing to
complete the TLS handshake in a timely manner. It's unclear if this is
because of heavily constrained clients are stalling while doing the
major crpto operations, or the edge servers are throttling repeated
requests from the same IPs.
This should allow reusing the open TLS connection to the release edge
servers during init.
For users that have metered or slow internet connections it is annoying
to have Terraform constantly re-downloading the same files when they
initialize many separate directories.
To help such users, here we add an opt-in mechanism to use a local
directory as a read-through cache. When enabled, any plugin download will
be skipped if a suitable file already exists in the cache directory. If
the desired plugin isn't in the cache, it will be downloaded into the
cache for use next time.
This mechanism also serves to reduce total disk usage by allowing
plugin files to be shared between many configurations, as long as the
target system isn't Windows and supports either hardlinks or symlinks.
If the release site is missing the "x-terraform-protocol-version"
header, we should fetch the latest spec'ed release. Downloading the
wrong protocol version can't do any damage, and the version present is
more than likely compatible.
Previously we forced only installing for the current GOOS and GOARCH. Now
we allow this to be optionally overridden, which allows building tools
that can, for example, populate a directory with plugins to run on a Linux
server while working on a Mac.
Some errors from Get are essentially user error, so we want to be able to
recognize them and give the user good feedback on how to proceed.
Although sentinel values are not an ideal solution to this, it's something
reasonably simple we can do to get this done without lots of refactoring.
Fetch the SHA256SUMS file and verify it's signature before downloading
any plugins.
This embeds the hashicorp public key in the binary. If the publickey is
replaced, new releases will need to be cut anyway. A
--verify-plugin=false flag will be added to skip signature verification
in these cases.
Previously we had a "getProvider" function type used to implement plugin
fetching. Here we replace that with an interface type, initially with
just a "Get" function.
For now this just simplifies the interface by allowing the target
directory and protocol version to be members of the struct rather than
passed as arguments.
A later change will extend this interface to also include a method to
purge unused plugins, so that upgrading frequently doesn't leave behind
a trail of unused executable files.
We can filter the allowed versions and sort them before checking the
protocol version, that way we can just return the first one found
reducing network requests.
Get provider needs to be provided with the plugin protocol version,
because it can't be imported here directly.
The plugin url types and methods were confusing; replace them with a few
functions to format the urls.
Add discovery.GetProviders to fetch plugins from the relases site.
This is an early version, with no tests, that only (probably) fetches
plugins from the default location. The URLs are still subject to change,
and since there are no plugin releases, it doesn't work at all yet.
Masayuki Morita