Commit Graph

50 Commits

Author SHA1 Message Date
James Goodhouse 25bfe7337b
lang: add setsubtract function (#23424)
* add setdifference and setsubtract functions and docs
* remove setdifference as it is not implemented correct in underlying lib

* Update setintersection.html.md
* Update setproduct.html.md
* Update setunion.html.md
2020-02-06 12:49:11 -05:00
Martin Atkins 02576988c1 lang: "try" and "can" functions
These are intended to make it easier to work with arbitrary data
structures whose shape might not be known statically, such as the result
of jsondecode(...) or yamldecode(...) of data from a separate system.

For example, in an object value which has attributes that may or may not
be set we can concisely provide a fallback value to use when the attribute
isn't set:

    try(local.example.foo, "fallback-foo")

Using a "try to evaluate" model rather than explicit testing fits better
with the usual programming model of the Terraform language where values
are normally automatically converted to the necessary type where possible:
the given expression is subject to all of the same normal type conversions,
which avoids inadvertently creating a more restrictive evaluation model
as might happen if this were handled using checks like a hypothetical
isobject(...) function, etc.
2020-01-10 15:23:25 -08:00
George Christou 23fc68cc91
lang: Fix new `trim*` function mappings
Fixes #23413
2019-11-19 10:13:14 +00:00
Martin Atkins f84ab99b7d lang/funcs: cidrsubnets function
This is a companion to cidrsubnet that allows bulk-allocation of multiple
subnet addresses at once, with automatic numbering.

Unlike cidrsubnet, cidrsubnets allows each of the allocations to have a
different prefix length, and will pack the networks consecutively into the
given address space. cidrsubnets can potentially create more complicated
addressing schemes than cidrsubnet alone can, because it's able to take
into account the full set of requested prefix lengths rather than just
one at a time.
2019-09-20 15:58:01 -07:00
Jeet Parekh bcc69c05bb lang/funcs: parseint function 2019-09-17 15:33:22 -07:00
Brian Flad d48d9ed766
lang/funcs: Add fileset function
Reference: https://github.com/hashicorp/terraform/issues/16697

Enumerates a set of regular file names from a given glob pattern. Implemented via the Go stdlib `path/filepath.Glob()` functionality. Notably, stdlib does not support `**` or `{}` extended patterns. See also: https://github.com/golang/go/issues/11862

To support the extended glob patterns, it will require adding a dependency on a third party library or adding our own matching code.
2019-08-20 04:50:01 -04:00
Martin Atkins 135afaeb9c lang: "regex" and "regexall" functions
These existing upstream cty functions allow matching strings against
regular expression patterns, which can be useful if you need to consume
a non-standard string format that Terraform doesn't (and can't) have a
built-in function for.
2019-08-06 11:52:14 -07:00
Andreas Sommer 042aead714 lang/funcs: add "abspath" function (#21409) 2019-07-02 08:30:30 -04:00
Lars Eric Scheidler aa07806bfc lang/funcs: New "uuidv5" function
This generates name-based uuids, rather than pseudorandom uuids as with the
"uuid" function.
2019-06-07 14:38:22 -07:00
Martin Atkins 382e1ca821 lang: yamldecode and yamlencode functions
These follow the same principle as jsondecode and jsonencode, but use
YAML instead of JSON.

YAML has a much more complex information model than JSON, so we can only
support a subset of it during decoding, but hopefully the subset supported
here is a useful one.

Because there are many different ways to _generate_ YAML, the yamlencode
function is forced to make some decisions, and those decisions are likely
to affect compatibility with other real-world YAML parsers. Although the
format here is intended to be generic and compatible, we may find that
there are problems with it that'll we'll want to adjust for in a future
release, so yamlencode is therefore marked as experimental for now until
the underlying library is ready to commit to ongoing byte-for-byte
compatibility in serialization.

The main use-case here is met by yamldecode, which will allow reading in
files written in YAML format by humans for use in Terraform modules, in
situations where a higher-level input format than direct Terraform
language declarations is helpful.
2019-06-04 16:24:09 -07:00
Martin Atkins f9a73d48db lang: "range" function
This is similar to the function of the same name in Python, generating a
sequence of numbers as a list that can then be used in other
sequence-oriented operations.

The primary use-case for it is to turn a count expressed as a number into
a list of that length, which can then be iterated over or passed to a
collection function to produce that number of something else, as shown
in the example at the end of its documentation page.
2019-06-04 16:20:17 -07:00
Kristin Laemmert b1d0b1383f
lang/funcs: remove sethaselement function and documentation (#21164)
`contains` and `sethaselement` are effectively the same function, and
`contains` works with `sets` thanks to automatic HCL conversion.
2019-05-02 10:47:19 -04:00
Kit Ewbank efc08de5d6 lang/funcs: add 'strrev' interpolation function. (#21091) 2019-04-24 14:52:39 -04:00
Kristin Laemmert d4669246c7
funcs/coalesce: return the first non-null, non-empty-string element from a sequence (#21002)
* funcs/coalesce: return the first non-null, non-empty element from a
sequence.

The go-cty coalesce function, which was originally used here, returns the
first non-null element from a sequence. Terraform 0.11's coalesce,
however, returns the first non-empty string from a list of strings.

This new coalesce function aims to preserve terraform's documented
functionality while adding support for additional argument types. The
tests include those in go-cty and adapted tests from the 0.11 version of
coalesce.

* website/docs: update coalesce function document
2019-04-12 13:57:52 -04:00
Martin Atkins 096b1bb87b lang/funcs: Port the "reverse" function from the old functions set
This has the same functionality as the "reverse" function that was
implemented in the "config" package, but adapted to the new language type
system.
2019-03-19 17:32:19 -07:00
Martin Atkins 954d38e870 lang: New file-hashing functions
In prior versions, we recommended using hash functions in conjunction with
the file function as an idiom for detecting changes to upstream blobs
without fetching and comparing the whole blob.

That approach relied on us being able to return raw binary data from
file(...). Since Terraform strings pass through intermediate
representations that are not binary-safe (e.g. the JSON state), there was
a risk of string corruption in prior versions which we have avoided for
0.12 by requiring that file(...) be used only with UTF-8 text files.

The specific case of returning a string and immediately passing it into
another function was not actually subject to that corruption risk, since
the HIL interpreter would just pass the string through verbatim, but this
is still now forbidden as a result of the stricter handling of file(...).

To avoid breaking these use-cases, here we introduce variants of the hash
functions a with "file" prefix that take a filename for a disk file to
hash rather than hashing the given string directly. The configuration
upgrade tool also now includes a rule to detect the documented idiom and
rewrite it into a single function call for one of these new functions.

This does cause a bit of function sprawl, but that seems preferable to
introducing more complex rules for when file(...) can and cannot read
binary files, making the behavior of these various functions easier to
understand in isolation.
2019-01-25 10:18:44 -08:00
Martin Atkins 2f8f7d6f4d lang/funcs: Type conversion functions
It's not normally necessary to make explicit type conversions in Terraform
because the language implicitly converts as necessary, but explicit
conversions are useful in a few specialized cases:

- When defining output values for a reusable module, it may be desirable
  to force a "cleaner" output type than would naturally arise from a
  computation, such as forcing a string containing digits into a number.
- Our 0.12upgrade mechanism will use some of these to replace use of the
  undocumented, hidden type conversion functions in HIL, and force
  particular type interpretations in some tricky cases.
- We've found that type conversion functions can be useful as _temporary_
  workarounds for bugs in Terraform and in providers where implicit type
  conversion isn't working correctly or a type constraint isn't specified
  precisely enough for the automatic conversion behavior.

These all follow the same convention of being named "to" followed by a
short type name. Since we've had a long-standing convention of running all
the words together in lowercase in function names, we stick to that here
even though some of these names are quite strange, because these should
be rarely-used functions anyway.
2019-01-17 10:01:47 -08:00
Martin Atkins da51e72cbb lang/functions: set functions from cty
The sethaselement, setintersection, and setunion functions are defined in
the cty stdlib. Making them available in Terraform will make it easier to
work with sets, and complement the currently-Terraform-specific setproduct
function.

In the long run setproduct should probably move into the cty stdlib too,
but since it was submitted as a Terraform function originally we'll leave
it here now for simplicity's sake and reorganize later.
2019-01-16 09:57:16 -08:00
Martin Atkins edb5f82de1 lang/funcs: Convert the "setproduct" function to the new approach
In our new world it produces either a set of a tuple type or a list of a
tuple type, depending on the given argument types.

The resulting collection's element tuple type is decided by the element
types of the given collections, allowing type information to propagate
even if unknown values are present.
2019-01-16 09:57:16 -08:00
Martin Atkins d0e6a4c69a lang: Add "formatdate" function
We missed this one on a previous pass of bringing in most of the cty
stdlib functions.

This will resolve #17625 by allowing conversion from Terraform's
conventional RFC 3339 timestamps into various other formats.
2019-01-07 09:10:14 -08:00
Martin Atkins c753df6a93 lang/funcs: templatefile function
This function is similar to the template_file data source offered by the
template provider, but having it built in to the language makes it more
convenient to use, allowing templates to be rendered from files anywhere
an inline template would normally be allowed:

    user_data = templatefile("${path.module}/userdata.tmpl", {
      hostname = format("petserver%02d", count.index)
    })

Unlike the template_file data source, this function allows values of any
type in its variables map, passing them through verbatim to the template.
Its tighter integration with Terraform also allows it to return better
error messages with source location information from the template itself.

The template_file data source was originally created to work around the
fact that HIL didn't have any support for map values at the time, and
even once map support was added it wasn't very usable. With HCL2
expressions, there's little reason left to use a data source to render
a template; the only remaining reason left to use template_file is to
render a template that is constructed dynamically during the Terraform
run, which is a very rare need.
2018-12-21 08:06:14 -08:00
Radek Simko edaa4bbc82
lang: Add fileexists function 2018-10-17 10:18:07 +01:00
Kristin Laemmert 4f5c03339a functions: ZipmapFunc 2018-10-16 18:49:20 -07:00
Kristin Laemmert 6463dd90e9 functions: TransposeFunc, SliceFunc 2018-10-16 18:49:20 -07:00
Kristin Laemmert 30671d85ad functions: MergeFunc 2018-10-16 18:49:20 -07:00
Kristin Laemmert aecd7b2e62 functions: LookupFunc 2018-10-16 18:49:20 -07:00
Kristin Laemmert 4d8c398f8e functions: KeysFunc 2018-10-16 18:49:20 -07:00
Kristin Laemmert 21daabe680 functions: MapFunc 2018-10-16 18:49:20 -07:00
Kristin Laemmert da02e0da4d functions: ListFunc 2018-10-16 18:49:20 -07:00
Kristin Laemmert 529c2c3cc9 functions: FlattenFunc 2018-10-16 18:49:20 -07:00
Kristin Laemmert 4dd3ffc127 porting matchkeys 2018-10-16 18:49:20 -07:00
Kristin Laemmert 0cbcd75ebb port distinct and chunklist functions 2018-10-16 18:49:20 -07:00
Kristin Laemmert 8aac7587f7 port index and contains functions 2018-10-16 18:49:20 -07:00
Kristin Laemmert e697e7d733 port compact function 2018-10-16 18:49:20 -07:00
Kristin Laemmert 04ac87747c base64decode: check that the decoded (not encoded) string is valid UTF-8 2018-10-16 18:49:20 -07:00
Kristin Laemmert b6d3d69d3a port cidr functions 2018-10-16 18:49:20 -07:00
Kristin Laemmert 10ef61c71c porting many functions 2018-10-16 18:49:20 -07:00
Kristin Laemmert 602b59cdc4 porting functions 2018-10-16 18:49:20 -07:00
Kristin Laemmert 4ad3676934 port ceil function 2018-10-16 18:49:20 -07:00
Kristin Laemmert c4f4dddff5 porting crypto functions 2018-10-16 18:49:20 -07:00
Kristin Laemmert eb1d8b7909 cleanup comments for nicer godocs 2018-10-16 18:49:20 -07:00
Kristin Laemmert 9aa9b18658 porting crypto functions 2018-10-16 18:49:20 -07:00
Kristin Laemmert 1a5299efcb porting encoding functions 2018-10-16 18:49:20 -07:00
Kristin Laemmert d4e703a5c1 base64decode 2018-10-16 18:48:28 -07:00
Kristin Laemmert a187c92f0e implement datetime functions 2018-10-16 18:48:28 -07:00
Kristin Laemmert 755b1e2497 implement pathexpand 2018-10-16 18:48:28 -07:00
Kristin Laemmert 2a2ffb6ef4 implement dirname function 2018-10-16 18:48:28 -07:00
Kristin Laemmert 8c1f0842b0 implement basename function 2018-10-16 18:48:28 -07:00
Martin Atkins 129f5fe74d lang/funcs: port some of Terraform's built-in functions
These implementations are adaptations of the existing implementations in
config/interpolate_funcs.go, updated to work with the cty API.

The set of functions chosen here was motivated mainly by what Terraform's
existing context tests depend on, so we can get the contexts tests back
into good shape before fleshing out the rest of these functions.
2018-10-16 18:48:28 -07:00
Martin Atkins a16ca2ec53 lang: new package for the runtime parts of the config language
Whereas package "configs" deals with the static structure of the
configuration language, this new package "lang" deals with the dynamic
aspects such as expression evaluation.

So far this mainly consists of populating a hcl.EvalContext that contains
the values necessary to evaluate a block or an expression. There is also
special handling here for dynamic block generation using the HCL
"dynblock" extension, which is exposed in the public interface (rather
than hiding it as an implementation detail of EvalBlock) so that the
caller can then extract proper source locations for any result values
using the expanded body.

This also includes the beginnings of a replacement for the function table
handling that currently lives in the old "config" package, but most of
the functions are not yet ported and so this will expand in subsequent
commits.
2018-10-16 18:44:26 -07:00