Because `aws_security_group_rule` resources are an abstraction on top of
Security Groups, they must interact with the AWS Security Group APIs in
a pattern that often results in lots of parallel requests interacting
with the same security group.
We've found that this pattern can trigger race conditions resulting in
inconsistent behavior, including:
* Rules that report as created but don't actually exist on AWS's side
* Rules that show up in AWS but don't register as being created
locally, resulting in follow up attempts to authorize the rule
failing w/ Duplicate errors
Here, we introduce a per-SG mutex that must be held by any security
group before it is allowed to interact with AWS APIs. This protects the
space between `DescribeSecurityGroup` and `Authorize*` / `Revoke*`
calls, ensuring that no other rules interact with the SG during that
span.
The included test exposes the race by applying a security group with
lots of rules, which based on the dependency graph can all be handled in
parallel. This fails most of the time without the new locking behavior.
I've omitted the mutex from `Read`, since it is only called during the
Refresh walk when no changes are being made, meaning a bunch of parallel
`DescribeSecurityGroup` API calls should be consistent in that case.
A "Layer" is a particular service that forms part of the infrastructure for
a set of applications. Some layers are application servers and others are
pure infrastructure, like MySQL servers or load balancers.
Although the AWS API only has one type called "Layer", it actually has
a number of different "soft" types that each have slightly different
validation rules and extra properties that are packed into the Attributes
map.
To make the validation rule differences explicit in Terraform, and to make
the Terraform structure more closely resemble the OpsWorks UI than its
API, we use a separate resource type per layer type, with the common code
factored out into a shared struct type.
"Stack" is the root concept in OpsWorks, and acts as a container for a number
of different "layers" that each provide some service for an application.
A stack isn't very interesting on its own, but it needs to be created before
any layers can be created.
AWS provides three different ways to create AMIs that each have different
inputs, but once they are complete the same management operations apply.
Thus these three resources each have a different "Create" implementation
but then share the same "Read", "Update" and "Delete" implementations.
* master: (720 commits)
Update CHANGELOG.md
Update CHANGELOG.md
dynamodb-local Update AWS config https://github.com/hashicorp/terraform/pull/2825#issuecomment-126353610
Make target_pools optional
Update CHANGELOG.md
code formatting
Update CHANGELOG.md
providers/google: Fix reading account_file path
providers/google: Fix error appending
providers/google: Return if we could parse JSON
providers/google: Change account_file to JSON
providers/google: Default account_file* to empty
providers/google: Add account_file/account_file_contents ConflictsWith
providers/google: Document account_file_contents
providers/google: Use account_file_contents if provided
providers/google: Add account_file_contents to provider
Update CHANGELOG.md
Update CHANGELOG.md
dynamodb-local Use ` instead of : to refer region to keep the consistency with the provider docs
dynamodb-local Update aws provider docs to include the `dynamodb_endpoint` argument
...
* master:
Update CHANGELOG.md
Update CHANGELOG.md
Added affinity group resource.
update link to actually work
provider/azure: Fix SQL client name to match upstream
add warning message to explain scenario of conflicting rules
typo
remove debugging
Update CHANGELOG.md
provider/aws: Add docs for autoscaling_policy + cloudwatch_metric_alarm
provider/aws: Add autoscaling_policy
provider/aws: Add cloudwatch_metric_alarm
rename method, update docs
clean up some conflicts with
clean up old, incompatible test
update tests with another example
update test
remove meta usage, stub test
fix existing tests
Consider security groups with source security groups when hashing
This is an iteration on the great work done by @dalehamel in PRs #2095
and #2109.
The core team went back and forth on how to best model Spot Instance
Requests, requesting and then rejecting a separate-resource
implementation in #2109.
After more internal discussion, we landed once again on a separate
resource to model Spot Instance Requests. Out of respect for
@dalehamel's already-significant donated time, with this I'm attempting
to pick up the work to take this across the finish line.
Important architectural decisions represented here:
* Spot Instance Requests are always of type "persistent", to properly
match Terraform's declarative model.
* The spot_instance_request resource exports several attributes that
are expected to be constantly changing as the spot market changes:
spot_bid_status, spot_request_state, and instance_id. Creating
additional resource dependencies based on these attributes is not
recommended, as Terraform diffs will be continually generated to keep
up with the live changes.
* When a Spot Instance Request is deleted/canceled, an attempt is made
to terminate the last-known attached spot instance. Race conditions
dictate that this attempt cannot guarantee that the associated spot
instance is terminated immediately.
Implementation notes:
* This version of aws_spot_instance_request borrows a lot of common
code from aws_instance.
* In order to facilitate borrowing, we introduce `awsInstanceOpts`, an
internal representation of instance details that's meant to be shared
between resources. The goal here would be to refactor ASG Launch
Configurations to use the same struct.
* The new aws_spot_instance_request acc. test is passing.
* All aws_instance acc. tests remain passing.
* upstream/master: (21 commits)
fix typo
fix typo, use awslabs/aws-sdk-go
Update CHANGELOG.md
More internal links in template documentation.
providers/aws: Requires ttl and records attributes if there isn't an ALIAS block.
Condense switch fallthroughs into expr lists
Fix docs for aws_route53_record params
Update CHANGELOG.md
provider/aws: Add IAM Server Certificate resource
aws_db_instance docs updated per #2070
providers/aws: Adds link to AWS docs about RDS parameters.
Downgrade middleman to 3.3.12 as 3.3.13 does not exist
providers/aws: Clarifies db_security_group usage.
"More more" no more!
Indentation issue
Export ARN in SQS queue and SNS topic / subscription; updated tests for new AWS SDK errors; updated documentation.
Changed Required: false to Optional: true in the SNS topic schema
Initial SNS support
correct resource name in example
added attributes reference section for AWS_EBS_VOLUME
...
aws hides its credentials in many places:
multiple env vars, config files,
ec2 metadata.
Terraform currently recognizes only the env vars;
to use the other options, you had to put in a
dummy empty value for access_key and secret_key.
Rather than duplicate all aws checks, ask the
aws sdk to fetch credentials earlier.
- Users
- Groups
- Roles
- Inline policies for the above three
- Instance profiles
- Managed policies
- Access keys
This is most of the data types provided by IAM. There are a few things
missing, but the functionality here is probably sufficient for 95% of
the cases. Makes a dent in #28.
This resource allows an existing Route Table to be assigned as the
"main" Route Table of a VPC. This means that the Route Table will be
used for any subnets within the VPC without an explicit Route Table
assigned [1].
This is particularly useful in getting an Internet Gateway in place as
the default for a VPC, since the automatically created Main Route Table
does not have one [2].
Note that this resource is an abstraction over an association and does not
map directly to a CRUD-able object in AWS. In order to retain a coherent
"Delete" operation for this resource, we remember the ID of the AWS-created
Route Table and reset the VPC's main Route Table to it when this
resource is deleted.
refs #843, #748
[1] http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Route_Tables.html#RouteTableDetails
[2] http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Internet_Gateway.html#Add_IGW_Routing
For now this only supports importing a key pair (by specifying a
public_key) property. In the future it'd be fairly trivial to support
key pair creation, with the private key returned as a computed property.
In real world usage you'd probably want to provide that public_key
property via a variable rather than hard-coding it into a terraform
config that'd end up in source control.
Rafal Jeczalik