backend/s3: Updates for Terraform v0.13.0 (#25134)

* deps: Update github.com/hashicorp/aws-sdk-go-base@v0.5.0

Updated via:

```
$ go get github.com/hashicorp/aws-sdk-go-base@v0.5.0
$ go mod tidy
$ go mod vendor
```

* backend/s3: Updates for Terraform v0.13.0

Reference: https://github.com/hashicorp/terraform/issues/13410
Reference: https://github.com/hashicorp/terraform/issues/18774
Reference: https://github.com/hashicorp/terraform/issues/19482
Reference: https://github.com/hashicorp/terraform/issues/20062
Reference: https://github.com/hashicorp/terraform/issues/20599
Reference: https://github.com/hashicorp/terraform/issues/22103
Reference: https://github.com/hashicorp/terraform/issues/22161
Reference: https://github.com/hashicorp/terraform/issues/22601
Reference: https://github.com/hashicorp/terraform/issues/22992
Reference: https://github.com/hashicorp/terraform/issues/24252
Reference: https://github.com/hashicorp/terraform/issues/24253
Reference: https://github.com/hashicorp/terraform/issues/24480
Reference: https://github.com/hashicorp/terraform/issues/25056

Changes:

```
NOTES

* backend/s3: Deprecated `lock_table`, `skip_get_ec2_platforms`, `skip_requesting_account_id` arguments have been removed
* backend/s3: Credential ordering has changed from static, environment, shared credentials, EC2 metadata, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata) to static, environment, shared credentials, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata)
* The `AWS_METADATA_TIMEOUT` environment variable no longer has any effect as we now depend on the default AWS Go SDK EC2 Metadata client timeout of one second with two retries

ENHANCEMENTS

* backend/s3: Always enable shared configuration file support (no longer require `AWS_SDK_LOAD_CONFIG` environment variable)
* backend/s3: Automatically expand `~` prefix for home directories in `shared_credentials_file` argument
* backend/s3: Add `assume_role_duration_seconds`, `assume_role_policy_arns`, `assume_role_tags`, and `assume_role_transitive_tag_keys` arguments

BUG FIXES

* backend/s3: Ensure configured profile is used
* backend/s3: Ensure configured STS endpoint is used during AssumeRole API calls
* backend/s3: Prefer AWS shared configuration over EC2 metadata credentials
* backend/s3: Prefer ECS credentials over EC2 metadata credentials
* backend/s3: Remove hardcoded AWS Provider messaging
```

Output from acceptance testing:

```
--- PASS: TestBackend (16.32s)
--- PASS: TestBackendConfig (0.58s)
--- PASS: TestBackendConfig_AssumeRole (0.02s)
--- PASS: TestBackendConfig_conflictingEncryptionSchema (0.00s)
--- PASS: TestBackendConfig_invalidKey (0.00s)
--- PASS: TestBackendConfig_invalidSSECustomerKeyEncoding (0.00s)
--- PASS: TestBackendConfig_invalidSSECustomerKeyLength (0.00s)
--- PASS: TestBackendExtraPaths (13.21s)
--- PASS: TestBackendLocked (28.98s)
--- PASS: TestBackendPrefixInWorkspace (5.65s)
--- PASS: TestBackendSSECustomerKey (17.60s)
--- PASS: TestBackend_impl (0.00s)
--- PASS: TestForceUnlock (17.50s)
--- PASS: TestKeyEnv (50.25s)
--- PASS: TestRemoteClient (4.78s)
--- PASS: TestRemoteClientLocks (16.85s)
--- PASS: TestRemoteClient_clientMD5 (12.08s)
--- PASS: TestRemoteClient_impl (0.00s)
--- PASS: TestRemoteClient_stateChecksum (17.92s)
```
This commit is contained in:
Brian Flad 2020-06-05 16:41:32 -04:00 committed by GitHub
parent 2dd64a7816
commit ba081aa10a
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
32 changed files with 2113 additions and 844 deletions

View File

@ -44,7 +44,7 @@ func New() backend.Backend {
"region": { "region": {
Type: schema.TypeString, Type: schema.TypeString,
Required: true, Required: true,
Description: "The region of the S3 bucket.", Description: "AWS region of the S3 Bucket and DynamoDB Table (if used).",
DefaultFunc: schema.MultiEnvDefaultFunc([]string{ DefaultFunc: schema.MultiEnvDefaultFunc([]string{
"AWS_REGION", "AWS_REGION",
"AWS_DEFAULT_REGION", "AWS_DEFAULT_REGION",
@ -114,14 +114,6 @@ func New() backend.Backend {
Default: "", Default: "",
}, },
"lock_table": {
Type: schema.TypeString,
Optional: true,
Description: "DynamoDB table for state locking",
Default: "",
Deprecated: "please use the dynamodb_table attribute",
},
"dynamodb_table": { "dynamodb_table": {
Type: schema.TypeString, Type: schema.TypeString,
Optional: true, Optional: true,
@ -157,14 +149,6 @@ func New() backend.Backend {
Default: false, Default: false,
}, },
"skip_get_ec2_platforms": {
Type: schema.TypeBool,
Optional: true,
Description: "Skip getting the supported EC2 platforms.",
Default: false,
Deprecated: "The S3 Backend does not require EC2 functionality and this attribute is no longer used.",
},
"skip_region_validation": { "skip_region_validation": {
Type: schema.TypeBool, Type: schema.TypeBool,
Optional: true, Optional: true,
@ -172,14 +156,6 @@ func New() backend.Backend {
Default: false, Default: false,
}, },
"skip_requesting_account_id": {
Type: schema.TypeBool,
Optional: true,
Description: "Skip requesting the account ID.",
Default: false,
Deprecated: "The S3 Backend no longer automatically looks up the AWS Account ID and this attribute is no longer used.",
},
"skip_metadata_api_check": { "skip_metadata_api_check": {
Type: schema.TypeBool, Type: schema.TypeBool,
Optional: true, Optional: true,
@ -223,13 +199,40 @@ func New() backend.Backend {
Default: "", Default: "",
}, },
"assume_role_duration_seconds": {
Type: schema.TypeInt,
Optional: true,
Description: "Seconds to restrict the assume role session duration.",
},
"assume_role_policy": { "assume_role_policy": {
Type: schema.TypeString, Type: schema.TypeString,
Optional: true, Optional: true,
Description: "The permissions applied when assuming a role.", Description: "IAM Policy JSON describing further restricting permissions for the IAM Role being assumed.",
Default: "", Default: "",
}, },
"assume_role_policy_arns": {
Type: schema.TypeSet,
Optional: true,
Description: "Amazon Resource Names (ARNs) of IAM Policies describing further restricting permissions for the IAM Role being assumed.",
Elem: &schema.Schema{Type: schema.TypeString},
},
"assume_role_tags": {
Type: schema.TypeMap,
Optional: true,
Description: "Assume role session tags.",
Elem: &schema.Schema{Type: schema.TypeString},
},
"assume_role_transitive_tag_keys": {
Type: schema.TypeSet,
Optional: true,
Description: "Assume role session tag keys to pass to any subsequent sessions.",
Elem: &schema.Schema{Type: schema.TypeString},
},
"workspace_key_prefix": { "workspace_key_prefix": {
Type: schema.TypeString, Type: schema.TypeString,
Optional: true, Optional: true,
@ -302,6 +305,7 @@ func (b *Backend) configure(ctx context.Context) error {
b.workspaceKeyPrefix = data.Get("workspace_key_prefix").(string) b.workspaceKeyPrefix = data.Get("workspace_key_prefix").(string)
b.serverSideEncryption = data.Get("encrypt").(bool) b.serverSideEncryption = data.Get("encrypt").(bool)
b.kmsKeyID = data.Get("kms_key_id").(string) b.kmsKeyID = data.Get("kms_key_id").(string)
b.ddbTable = data.Get("dynamodb_table").(string)
customerKeyString := data.Get("sse_customer_key").(string) customerKeyString := data.Get("sse_customer_key").(string)
if customerKeyString != "" { if customerKeyString != "" {
@ -316,18 +320,15 @@ func (b *Backend) configure(ctx context.Context) error {
} }
} }
b.ddbTable = data.Get("dynamodb_table").(string)
if b.ddbTable == "" {
// try the deprecated field
b.ddbTable = data.Get("lock_table").(string)
}
cfg := &awsbase.Config{ cfg := &awsbase.Config{
AccessKey: data.Get("access_key").(string), AccessKey: data.Get("access_key").(string),
AssumeRoleARN: data.Get("role_arn").(string), AssumeRoleARN: data.Get("role_arn").(string),
AssumeRoleDurationSeconds: data.Get("assume_role_duration_seconds").(int),
AssumeRoleExternalID: data.Get("external_id").(string), AssumeRoleExternalID: data.Get("external_id").(string),
AssumeRolePolicy: data.Get("assume_role_policy").(string), AssumeRolePolicy: data.Get("assume_role_policy").(string),
AssumeRoleSessionName: data.Get("session_name").(string), AssumeRoleSessionName: data.Get("session_name").(string),
CallerDocumentationURL: "https://www.terraform.io/docs/backends/types/s3.html",
CallerName: "S3 Backend",
CredsFilename: data.Get("shared_credentials_file").(string), CredsFilename: data.Get("shared_credentials_file").(string),
DebugLogging: logging.IsDebugOrHigher(), DebugLogging: logging.IsDebugOrHigher(),
IamEndpoint: data.Get("iam_endpoint").(string), IamEndpoint: data.Get("iam_endpoint").(string),
@ -346,9 +347,47 @@ func (b *Backend) configure(ctx context.Context) error {
}, },
} }
if policyARNSet := data.Get("assume_role_policy_arns").(*schema.Set); policyARNSet.Len() > 0 {
for _, policyARNRaw := range policyARNSet.List() {
policyARN, ok := policyARNRaw.(string)
if !ok {
continue
}
cfg.AssumeRolePolicyARNs = append(cfg.AssumeRolePolicyARNs, policyARN)
}
}
if tagMap := data.Get("assume_role_tags").(map[string]interface{}); len(tagMap) > 0 {
cfg.AssumeRoleTags = make(map[string]string)
for k, vRaw := range tagMap {
v, ok := vRaw.(string)
if !ok {
continue
}
cfg.AssumeRoleTags[k] = v
}
}
if transitiveTagKeySet := data.Get("assume_role_transitive_tag_keys").(*schema.Set); transitiveTagKeySet.Len() > 0 {
for _, transitiveTagKeyRaw := range transitiveTagKeySet.List() {
transitiveTagKey, ok := transitiveTagKeyRaw.(string)
if !ok {
continue
}
cfg.AssumeRoleTransitiveTagKeys = append(cfg.AssumeRoleTransitiveTagKeys, transitiveTagKey)
}
}
sess, err := awsbase.GetSession(cfg) sess, err := awsbase.GetSession(cfg)
if err != nil { if err != nil {
return err return fmt.Errorf("error configuring S3 Backend: %w", err)
} }
b.dynClient = dynamodb.New(sess.Copy(&aws.Config{ b.dynClient = dynamodb.New(sess.Copy(&aws.Config{

View File

@ -2,6 +2,7 @@ package s3
import ( import (
"fmt" "fmt"
"net/url"
"os" "os"
"reflect" "reflect"
"testing" "testing"
@ -10,12 +11,64 @@ import (
"github.com/aws/aws-sdk-go/aws" "github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/dynamodb" "github.com/aws/aws-sdk-go/service/dynamodb"
"github.com/aws/aws-sdk-go/service/s3" "github.com/aws/aws-sdk-go/service/s3"
awsbase "github.com/hashicorp/aws-sdk-go-base"
"github.com/hashicorp/terraform/backend" "github.com/hashicorp/terraform/backend"
"github.com/hashicorp/terraform/configs/hcl2shim" "github.com/hashicorp/terraform/configs/hcl2shim"
"github.com/hashicorp/terraform/state/remote" "github.com/hashicorp/terraform/state/remote"
"github.com/hashicorp/terraform/states" "github.com/hashicorp/terraform/states"
) )
const (
mockStsAssumeRoleArn = `arn:aws:iam::555555555555:role/AssumeRole`
mockStsAssumeRolePolicy = `{
"Version": "2012-10-17",
"Statement": {
"Effect": "Allow",
"Action": "*",
"Resource": "*",
}
}`
mockStsAssumeRolePolicyArn = `arn:aws:iam::555555555555:policy/AssumeRolePolicy1`
mockStsAssumeRoleSessionName = `AssumeRoleSessionName`
mockStsAssumeRoleTagKey = `AssumeRoleTagKey`
mockStsAssumeRoleTagValue = `AssumeRoleTagValue`
mockStsAssumeRoleTransitiveTagKey = `AssumeRoleTagKey`
mockStsAssumeRoleValidResponse = `<AssumeRoleResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
<AssumeRoleResult>
<AssumedRoleUser>
<Arn>arn:aws:sts::555555555555:assumed-role/role/AssumeRoleSessionName</Arn>
<AssumedRoleId>ARO123EXAMPLE123:AssumeRoleSessionName</AssumedRoleId>
</AssumedRoleUser>
<Credentials>
<AccessKeyId>AssumeRoleAccessKey</AccessKeyId>
<SecretAccessKey>AssumeRoleSecretKey</SecretAccessKey>
<SessionToken>AssumeRoleSessionToken</SessionToken>
<Expiration>2099-12-31T23:59:59Z</Expiration>
</Credentials>
</AssumeRoleResult>
<ResponseMetadata>
<RequestId>01234567-89ab-cdef-0123-456789abcdef</RequestId>
</ResponseMetadata>
</AssumeRoleResponse>`
mockStsGetCallerIdentityValidResponseBody = `<GetCallerIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
<GetCallerIdentityResult>
<Arn>arn:aws:iam::222222222222:user/Alice</Arn>
<UserId>AKIAI44QH8DHBEXAMPLE</UserId>
<Account>222222222222</Account>
</GetCallerIdentityResult>
<ResponseMetadata>
<RequestId>01234567-89ab-cdef-0123-456789abcdef</RequestId>
</ResponseMetadata>
</GetCallerIdentityResponse>`
)
var (
mockStsGetCallerIdentityRequestBody = url.Values{
"Action": []string{"GetCallerIdentity"},
"Version": []string{"2011-06-15"},
}.Encode()
)
// verify that we are doing ACC tests or the S3 tests specifically // verify that we are doing ACC tests or the S3 tests specifically
func testACC(t *testing.T) { func testACC(t *testing.T) {
skip := os.Getenv("TF_ACC") == "" && os.Getenv("TF_S3_TEST") == "" skip := os.Getenv("TF_ACC") == "" && os.Getenv("TF_S3_TEST") == ""
@ -66,6 +119,243 @@ func TestBackendConfig(t *testing.T) {
} }
} }
func TestBackendConfig_AssumeRole(t *testing.T) {
testACC(t)
testCases := []struct {
Config map[string]interface{}
Description string
MockStsEndpoints []*awsbase.MockEndpoint
}{
{
Config: map[string]interface{}{
"bucket": "tf-test",
"key": "state",
"region": "us-west-1",
"role_arn": mockStsAssumeRoleArn,
"session_name": mockStsAssumeRoleSessionName,
},
Description: "role_arn",
MockStsEndpoints: []*awsbase.MockEndpoint{
{
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: url.Values{
"Action": []string{"AssumeRole"},
"DurationSeconds": []string{"900"},
"RoleArn": []string{mockStsAssumeRoleArn},
"RoleSessionName": []string{mockStsAssumeRoleSessionName},
"Version": []string{"2011-06-15"},
}.Encode()},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"},
},
{
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
},
},
},
{
Config: map[string]interface{}{
"assume_role_duration_seconds": 3600,
"bucket": "tf-test",
"key": "state",
"region": "us-west-1",
"role_arn": mockStsAssumeRoleArn,
"session_name": mockStsAssumeRoleSessionName,
},
Description: "assume_role_duration_seconds",
MockStsEndpoints: []*awsbase.MockEndpoint{
{
Request: &awsbase.MockRequest{"POST", "/", url.Values{
"Action": []string{"AssumeRole"},
"DurationSeconds": []string{"3600"},
"RoleArn": []string{mockStsAssumeRoleArn},
"RoleSessionName": []string{mockStsAssumeRoleSessionName},
"Version": []string{"2011-06-15"},
}.Encode()},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"},
},
{
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
},
},
},
{
Config: map[string]interface{}{
"bucket": "tf-test",
"external_id": "AssumeRoleExternalId",
"key": "state",
"region": "us-west-1",
"role_arn": mockStsAssumeRoleArn,
"session_name": mockStsAssumeRoleSessionName,
},
Description: "external_id",
MockStsEndpoints: []*awsbase.MockEndpoint{
{
Request: &awsbase.MockRequest{"POST", "/", url.Values{
"Action": []string{"AssumeRole"},
"DurationSeconds": []string{"900"},
"ExternalId": []string{"AssumeRoleExternalId"},
"RoleArn": []string{mockStsAssumeRoleArn},
"RoleSessionName": []string{mockStsAssumeRoleSessionName},
"Version": []string{"2011-06-15"},
}.Encode()},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"},
},
{
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
},
},
},
{
Config: map[string]interface{}{
"assume_role_policy": mockStsAssumeRolePolicy,
"bucket": "tf-test",
"key": "state",
"region": "us-west-1",
"role_arn": mockStsAssumeRoleArn,
"session_name": mockStsAssumeRoleSessionName,
},
Description: "assume_role_policy",
MockStsEndpoints: []*awsbase.MockEndpoint{
{
Request: &awsbase.MockRequest{"POST", "/", url.Values{
"Action": []string{"AssumeRole"},
"DurationSeconds": []string{"900"},
"Policy": []string{mockStsAssumeRolePolicy},
"RoleArn": []string{mockStsAssumeRoleArn},
"RoleSessionName": []string{mockStsAssumeRoleSessionName},
"Version": []string{"2011-06-15"},
}.Encode()},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"},
},
{
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
},
},
},
{
Config: map[string]interface{}{
"assume_role_policy_arns": []interface{}{mockStsAssumeRolePolicyArn},
"bucket": "tf-test",
"key": "state",
"region": "us-west-1",
"role_arn": mockStsAssumeRoleArn,
"session_name": mockStsAssumeRoleSessionName,
},
Description: "assume_role_policy_arns",
MockStsEndpoints: []*awsbase.MockEndpoint{
{
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: url.Values{
"Action": []string{"AssumeRole"},
"DurationSeconds": []string{"900"},
"PolicyArns.member.1.arn": []string{mockStsAssumeRolePolicyArn},
"RoleArn": []string{mockStsAssumeRoleArn},
"RoleSessionName": []string{mockStsAssumeRoleSessionName},
"Version": []string{"2011-06-15"},
}.Encode()},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"},
},
{
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
},
},
},
{
Config: map[string]interface{}{
"assume_role_tags": map[string]interface{}{
mockStsAssumeRoleTagKey: mockStsAssumeRoleTagValue,
},
"bucket": "tf-test",
"key": "state",
"region": "us-west-1",
"role_arn": mockStsAssumeRoleArn,
"session_name": mockStsAssumeRoleSessionName,
},
Description: "assume_role_tags",
MockStsEndpoints: []*awsbase.MockEndpoint{
{
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: url.Values{
"Action": []string{"AssumeRole"},
"DurationSeconds": []string{"900"},
"RoleArn": []string{mockStsAssumeRoleArn},
"RoleSessionName": []string{mockStsAssumeRoleSessionName},
"Tags.member.1.Key": []string{mockStsAssumeRoleTagKey},
"Tags.member.1.Value": []string{mockStsAssumeRoleTagValue},
"Version": []string{"2011-06-15"},
}.Encode()},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"},
},
{
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
},
},
},
{
Config: map[string]interface{}{
"assume_role_tags": map[string]interface{}{
mockStsAssumeRoleTagKey: mockStsAssumeRoleTagValue,
},
"assume_role_transitive_tag_keys": []interface{}{mockStsAssumeRoleTagKey},
"bucket": "tf-test",
"key": "state",
"region": "us-west-1",
"role_arn": mockStsAssumeRoleArn,
"session_name": mockStsAssumeRoleSessionName,
},
Description: "assume_role_transitive_tag_keys",
MockStsEndpoints: []*awsbase.MockEndpoint{
{
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: url.Values{
"Action": []string{"AssumeRole"},
"DurationSeconds": []string{"900"},
"RoleArn": []string{mockStsAssumeRoleArn},
"RoleSessionName": []string{mockStsAssumeRoleSessionName},
"Tags.member.1.Key": []string{mockStsAssumeRoleTagKey},
"Tags.member.1.Value": []string{mockStsAssumeRoleTagValue},
"TransitiveTagKeys.member.1": []string{mockStsAssumeRoleTagKey},
"Version": []string{"2011-06-15"},
}.Encode()},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"},
},
{
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
},
},
},
}
for _, testCase := range testCases {
testCase := testCase
t.Run(testCase.Description, func(t *testing.T) {
closeSts, mockStsSession, err := awsbase.GetMockedAwsApiSession("STS", testCase.MockStsEndpoints)
defer closeSts()
if err != nil {
t.Fatalf("unexpected error creating mock STS server: %s", err)
}
if mockStsSession != nil && mockStsSession.Config != nil {
testCase.Config["sts_endpoint"] = aws.StringValue(mockStsSession.Config.Endpoint)
}
diags := New().Configure(hcl2shim.HCL2ValueFromConfigValue(testCase.Config))
if diags.HasErrors() {
for _, diag := range diags {
t.Errorf("unexpected error: %s", diag.Description().Summary)
}
}
})
}
}
func TestBackendConfig_invalidKey(t *testing.T) { func TestBackendConfig_invalidKey(t *testing.T) {
testACC(t) testACC(t)
cfg := hcl2shim.HCL2ValueFromConfigValue(map[string]interface{}{ cfg := hcl2shim.HCL2ValueFromConfigValue(map[string]interface{}{

4
go.mod
View File

@ -17,7 +17,7 @@ require (
github.com/armon/circbuf v0.0.0-20190214190532-5111143e8da2 github.com/armon/circbuf v0.0.0-20190214190532-5111143e8da2
github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da // indirect github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da // indirect
github.com/armon/go-radix v1.0.0 // indirect github.com/armon/go-radix v1.0.0 // indirect
github.com/aws/aws-sdk-go v1.30.12 github.com/aws/aws-sdk-go v1.31.9
github.com/baiyubin/aliyun-sts-go-sdk v0.0.0-20180326062324-cfa1a18b161f // indirect github.com/baiyubin/aliyun-sts-go-sdk v0.0.0-20180326062324-cfa1a18b161f // indirect
github.com/blang/semver v3.5.1+incompatible github.com/blang/semver v3.5.1+incompatible
github.com/bmatcuk/doublestar v1.1.5 github.com/bmatcuk/doublestar v1.1.5
@ -48,7 +48,7 @@ require (
github.com/grpc-ecosystem/go-grpc-middleware v1.0.0 // indirect github.com/grpc-ecosystem/go-grpc-middleware v1.0.0 // indirect
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 // indirect github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 // indirect
github.com/grpc-ecosystem/grpc-gateway v1.8.5 // indirect github.com/grpc-ecosystem/grpc-gateway v1.8.5 // indirect
github.com/hashicorp/aws-sdk-go-base v0.4.0 github.com/hashicorp/aws-sdk-go-base v0.5.0
github.com/hashicorp/consul v0.0.0-20171026175957-610f3c86a089 github.com/hashicorp/consul v0.0.0-20171026175957-610f3c86a089
github.com/hashicorp/errwrap v1.0.0 github.com/hashicorp/errwrap v1.0.0
github.com/hashicorp/go-azure-helpers v0.10.0 github.com/hashicorp/go-azure-helpers v0.10.0

10
go.sum
View File

@ -86,9 +86,8 @@ github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj
github.com/armon/go-radix v1.0.0 h1:F4z6KzEeeQIMeLFa97iZU6vupzoecKdU5TX24SNppXI= github.com/armon/go-radix v1.0.0 h1:F4z6KzEeeQIMeLFa97iZU6vupzoecKdU5TX24SNppXI=
github.com/armon/go-radix v1.0.0/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8= github.com/armon/go-radix v1.0.0/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8=
github.com/aws/aws-sdk-go v1.15.78/go.mod h1:E3/ieXAlvM0XWO57iftYVDLLvQ824smPP3ATZkfNZeM= github.com/aws/aws-sdk-go v1.15.78/go.mod h1:E3/ieXAlvM0XWO57iftYVDLLvQ824smPP3ATZkfNZeM=
github.com/aws/aws-sdk-go v1.25.3/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo= github.com/aws/aws-sdk-go v1.31.9 h1:n+b34ydVfgC30j0Qm69yaapmjejQPW2BoDBX7Uy/tLI=
github.com/aws/aws-sdk-go v1.30.12 h1:KrjyosZvkpJjcwMk0RNxMZewQ47v7+ZkbQDXjWsJMs8= github.com/aws/aws-sdk-go v1.31.9/go.mod h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZveU8YkpAk0=
github.com/aws/aws-sdk-go v1.30.12/go.mod h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZveU8YkpAk0=
github.com/baiyubin/aliyun-sts-go-sdk v0.0.0-20180326062324-cfa1a18b161f h1:ZNv7On9kyUzm7fvRZumSyy/IUiSC7AzL0I1jKKtwooA= github.com/baiyubin/aliyun-sts-go-sdk v0.0.0-20180326062324-cfa1a18b161f h1:ZNv7On9kyUzm7fvRZumSyy/IUiSC7AzL0I1jKKtwooA=
github.com/baiyubin/aliyun-sts-go-sdk v0.0.0-20180326062324-cfa1a18b161f/go.mod h1:AuiFmCCPBSrqvVMvuqFuk0qogytodnVFVSN5CeJB8Gc= github.com/baiyubin/aliyun-sts-go-sdk v0.0.0-20180326062324-cfa1a18b161f/go.mod h1:AuiFmCCPBSrqvVMvuqFuk0qogytodnVFVSN5CeJB8Gc=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973 h1:xJ4a3vCFaGF/jqvzLMYoU8P317H5OQ+Via4RmuPwCS0= github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973 h1:xJ4a3vCFaGF/jqvzLMYoU8P317H5OQ+Via4RmuPwCS0=
@ -199,8 +198,8 @@ github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 h1:Ovs26xHkKqVztRpIrF/92Bcuy
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk= github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk=
github.com/grpc-ecosystem/grpc-gateway v1.8.5 h1:2+KSC78XiO6Qy0hIjfc1OD9H+hsaJdJlb8Kqsd41CTE= github.com/grpc-ecosystem/grpc-gateway v1.8.5 h1:2+KSC78XiO6Qy0hIjfc1OD9H+hsaJdJlb8Kqsd41CTE=
github.com/grpc-ecosystem/grpc-gateway v1.8.5/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY= github.com/grpc-ecosystem/grpc-gateway v1.8.5/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
github.com/hashicorp/aws-sdk-go-base v0.4.0 h1:zH9hNUdsS+2G0zJaU85ul8D59BGnZBaKM+KMNPAHGwk= github.com/hashicorp/aws-sdk-go-base v0.5.0 h1:fk7ID0v3PWL/KNL8FvkBPu8Sm93EPUCCmtZCiTXLySE=
github.com/hashicorp/aws-sdk-go-base v0.4.0/go.mod h1:eRhlz3c4nhqxFZJAahJEFL7gh6Jyj5rQmQc7F9eHFyQ= github.com/hashicorp/aws-sdk-go-base v0.5.0/go.mod h1:2fRjWDv3jJBeN6mVWFHV6hFTNeFBx2gpDLQaZNxUVAY=
github.com/hashicorp/consul v0.0.0-20171026175957-610f3c86a089 h1:1eDpXAxTh0iPv+1kc9/gfSI2pxRERDsTk/lNGolwHn8= github.com/hashicorp/consul v0.0.0-20171026175957-610f3c86a089 h1:1eDpXAxTh0iPv+1kc9/gfSI2pxRERDsTk/lNGolwHn8=
github.com/hashicorp/consul v0.0.0-20171026175957-610f3c86a089/go.mod h1:mFrjN1mfidgJfYP1xrJCF+AfRhr6Eaqhb2+sfyn/OOI= github.com/hashicorp/consul v0.0.0-20171026175957-610f3c86a089/go.mod h1:mFrjN1mfidgJfYP1xrJCF+AfRhr6Eaqhb2+sfyn/OOI=
github.com/hashicorp/errwrap v1.0.0 h1:hLrqtEDnRye3+sgx6z4qVLNuviH3MR5aQ0ykNJa/UYA= github.com/hashicorp/errwrap v1.0.0 h1:hLrqtEDnRye3+sgx6z4qVLNuviH3MR5aQ0ykNJa/UYA=
@ -269,7 +268,6 @@ github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb/go.mod h1:+NfK9FKe
github.com/jhump/protoreflect v1.6.0 h1:h5jfMVslIg6l29nsMs0D8Wj17RDVdNYti0vDN/PZZoE= github.com/jhump/protoreflect v1.6.0 h1:h5jfMVslIg6l29nsMs0D8Wj17RDVdNYti0vDN/PZZoE=
github.com/jhump/protoreflect v1.6.0/go.mod h1:eaTn3RZAmMBcV0fifFvlm6VHNz3wSkYyXYWUh7ymB74= github.com/jhump/protoreflect v1.6.0/go.mod h1:eaTn3RZAmMBcV0fifFvlm6VHNz3wSkYyXYWUh7ymB74=
github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k= github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=
github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=
github.com/jmespath/go-jmespath v0.3.0 h1:OS12ieG61fsCg5+qLJ+SsW9NicxNkg3b25OyT2yCeUc= github.com/jmespath/go-jmespath v0.3.0 h1:OS12ieG61fsCg5+qLJ+SsW9NicxNkg3b25OyT2yCeUc=
github.com/jmespath/go-jmespath v0.3.0/go.mod h1:9QtRXoHjLGCJ5IBSaohpXITPlowMeeYCZ7fLUTSywik= github.com/jmespath/go-jmespath v0.3.0/go.mod h1:9QtRXoHjLGCJ5IBSaohpXITPlowMeeYCZ7fLUTSywik=
github.com/jonboulle/clockwork v0.1.0 h1:VKV+ZcuP6l3yW9doeqz6ziZGgcynBVQO+obU0+0hcPo= github.com/jonboulle/clockwork v0.1.0 h1:VKV+ZcuP6l3yW9doeqz6ziZGgcynBVQO+obU0+0hcPo=

View File

@ -169,6 +169,29 @@ type AssumeRoleProvider struct {
// size. // size.
Policy *string Policy *string
// The ARNs of IAM managed policies you want to use as managed session policies.
// The policies must exist in the same account as the role.
//
// This parameter is optional. You can provide up to 10 managed policy ARNs.
// However, the plain text that you use for both inline and managed session
// policies can't exceed 2,048 characters.
//
// An AWS conversion compresses the passed session policies and session tags
// into a packed binary format that has a separate limit. Your request can fail
// for this limit even if your plain text meets the other requirements. The
// PackedPolicySize response element indicates by percentage how close the policies
// and tags for your request are to the upper size limit.
//
// Passing policies to this operation returns new temporary credentials. The
// resulting session's permissions are the intersection of the role's identity-based
// policy and the session policies. You can use the role's temporary credentials
// in subsequent AWS API calls to access resources in the account that owns
// the role. You cannot use session policies to grant more permissions than
// those allowed by the identity-based policy of the role that is being assumed.
// For more information, see Session Policies (https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html#policies_session)
// in the IAM User Guide.
PolicyArns []*sts.PolicyDescriptorType
// The identification number of the MFA device that is associated with the user // The identification number of the MFA device that is associated with the user
// who is making the AssumeRole call. Specify this value if the trust policy // who is making the AssumeRole call. Specify this value if the trust policy
// of the role being assumed includes a condition that requires MFA authentication. // of the role being assumed includes a condition that requires MFA authentication.
@ -291,6 +314,7 @@ func (p *AssumeRoleProvider) RetrieveWithContext(ctx credentials.Context) (crede
RoleSessionName: aws.String(p.RoleSessionName), RoleSessionName: aws.String(p.RoleSessionName),
ExternalId: p.ExternalID, ExternalId: p.ExternalID,
Tags: p.Tags, Tags: p.Tags,
PolicyArns: p.PolicyArns,
TransitiveTagKeys: p.TransitiveTagKeys, TransitiveTagKeys: p.TransitiveTagKeys,
} }
if p.Policy != nil { if p.Policy != nil {

View File

@ -50,6 +50,7 @@ func (f FetchTokenPath) FetchToken(ctx credentials.Context) ([]byte, error) {
// an OIDC token. // an OIDC token.
type WebIdentityRoleProvider struct { type WebIdentityRoleProvider struct {
credentials.Expiry credentials.Expiry
PolicyArns []*sts.PolicyDescriptorType
client stsiface.STSAPI client stsiface.STSAPI
ExpiryWindow time.Duration ExpiryWindow time.Duration
@ -107,6 +108,7 @@ func (p *WebIdentityRoleProvider) RetrieveWithContext(ctx credentials.Context) (
sessionName = strconv.FormatInt(now().UnixNano(), 10) sessionName = strconv.FormatInt(now().UnixNano(), 10)
} }
req, resp := p.client.AssumeRoleWithWebIdentityRequest(&sts.AssumeRoleWithWebIdentityInput{ req, resp := p.client.AssumeRoleWithWebIdentityRequest(&sts.AssumeRoleWithWebIdentityInput{
PolicyArns: p.PolicyArns,
RoleArn: &p.roleARN, RoleArn: &p.roleARN,
RoleSessionName: &sessionName, RoleSessionName: &sessionName,
WebIdentityToken: aws.String(string(b)), WebIdentityToken: aws.String(string(b)),

View File

@ -93,7 +93,7 @@ func decodeV3Endpoints(modelDef modelDefinition, opts DecodeModelOptions) (Resol
} }
func custAddS3DualStack(p *partition) { func custAddS3DualStack(p *partition) {
if p.ID != "aws" { if !(p.ID == "aws" || p.ID == "aws-cn" || p.ID == "aws-us-gov") {
return return
} }

File diff suppressed because it is too large Load Diff

View File

@ -239,3 +239,26 @@ func (es errors) Error() string {
return strings.Join(parts, "\n") return strings.Join(parts, "\n")
} }
// CopySeekableBody copies the seekable body to an io.Writer
func CopySeekableBody(dst io.Writer, src io.ReadSeeker) (int64, error) {
curPos, err := src.Seek(0, sdkio.SeekCurrent)
if err != nil {
return 0, err
}
// copy errors may be assumed to be from the body.
n, err := io.Copy(dst, src)
if err != nil {
return n, err
}
// seek back to the first position after reading to reset
// the body for transmission.
_, err = src.Seek(curPos, sdkio.SeekStart)
if err != nil {
return n, err
}
return n, nil
}

View File

@ -5,4 +5,4 @@ package aws
const SDKName = "aws-sdk-go" const SDKName = "aws-sdk-go"
// SDKVersion is the version of this SDK // SDKVersion is the version of this SDK
const SDKVersion = "1.30.12" const SDKVersion = "1.31.9"

View File

@ -0,0 +1,53 @@
package checksum
import (
"crypto/md5"
"encoding/base64"
"fmt"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/aws/request"
)
const contentMD5Header = "Content-Md5"
// AddBodyContentMD5Handler computes and sets the HTTP Content-MD5 header for requests that
// require it.
func AddBodyContentMD5Handler(r *request.Request) {
// if Content-MD5 header is already present, return
if v := r.HTTPRequest.Header.Get(contentMD5Header); len(v) != 0 {
return
}
// if S3DisableContentMD5Validation flag is set, return
if aws.BoolValue(r.Config.S3DisableContentMD5Validation) {
return
}
// if request is presigned, return
if r.IsPresigned() {
return
}
// if body is not seekable, return
if !aws.IsReaderSeekable(r.Body) {
if r.Config.Logger != nil {
r.Config.Logger.Log(fmt.Sprintf(
"Unable to compute Content-MD5 for unseekable body, S3.%s",
r.Operation.Name))
}
return
}
h := md5.New()
if _, err := aws.CopySeekableBody(h, r.Body); err != nil {
r.Error = awserr.New("ContentMD5", "failed to compute body MD5", err)
return
}
// encode the md5 checksum in base64 and set the request header.
v := base64.StdEncoding.EncodeToString(h.Sum(nil))
r.HTTPRequest.Header.Set(contentMD5Header, v)
}

View File

@ -670,7 +670,7 @@ func (c *DynamoDB) CreateGlobalTableRequest(input *CreateGlobalTableInput) (req
// relationship between two or more DynamoDB tables with the same table name // relationship between two or more DynamoDB tables with the same table name
// in the provided Regions. // in the provided Regions.
// //
// This method only applies to Version 2017.11.29 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V1.html) // This operation only applies to Version 2017.11.29 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V1.html)
// of global tables. // of global tables.
// //
// If you want to add a new replica table to a global table, each of the following // If you want to add a new replica table to a global table, each of the following
@ -693,6 +693,14 @@ func (c *DynamoDB) CreateGlobalTableRequest(input *CreateGlobalTableInput) (req
// * The global secondary indexes must have the same hash key and sort key // * The global secondary indexes must have the same hash key and sort key
// (if present). // (if present).
// //
// If local secondary indexes are specified, then the following conditions must
// also be met:
//
// * The local secondary indexes must have the same name.
//
// * The local secondary indexes must have the same hash key and sort key
// (if present).
//
// Write capacity settings should be set consistently across your replica tables // Write capacity settings should be set consistently across your replica tables
// and secondary indexes. DynamoDB strongly recommends enabling auto scaling // and secondary indexes. DynamoDB strongly recommends enabling auto scaling
// to manage the write capacity settings for all of your global tables replicas // to manage the write capacity settings for all of your global tables replicas
@ -1839,8 +1847,10 @@ func (c *DynamoDB) DescribeGlobalTableRequest(input *DescribeGlobalTableInput) (
// //
// Returns information about the specified global table. // Returns information about the specified global table.
// //
// This method only applies to Version 2017.11.29 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V1.html) // This operation only applies to Version 2017.11.29 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V1.html)
// of global tables. // of global tables. If you are using global tables Version 2019.11.21 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V2.html)
// you can use DescribeTable (https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_DescribeTable.html)
// instead.
// //
// Returns awserr.Error for service API and SDK errors. Use runtime type assertions // Returns awserr.Error for service API and SDK errors. Use runtime type assertions
// with awserr.Error's Code and Message methods to get detailed information about // with awserr.Error's Code and Message methods to get detailed information about
@ -1949,7 +1959,7 @@ func (c *DynamoDB) DescribeGlobalTableSettingsRequest(input *DescribeGlobalTable
// //
// Describes Region-specific settings for a global table. // Describes Region-specific settings for a global table.
// //
// This method only applies to Version 2017.11.29 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V1.html) // This operation only applies to Version 2017.11.29 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V1.html)
// of global tables. // of global tables.
// //
// Returns awserr.Error for service API and SDK errors. Use runtime type assertions // Returns awserr.Error for service API and SDK errors. Use runtime type assertions
@ -2311,7 +2321,7 @@ func (c *DynamoDB) DescribeTableReplicaAutoScalingRequest(input *DescribeTableRe
// //
// Describes auto scaling settings across replicas of the global table at once. // Describes auto scaling settings across replicas of the global table at once.
// //
// This method only applies to Version 2019.11.21 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V2.html) // This operation only applies to Version 2019.11.21 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V2.html)
// of global tables. // of global tables.
// //
// Returns awserr.Error for service API and SDK errors. Use runtime type assertions // Returns awserr.Error for service API and SDK errors. Use runtime type assertions
@ -2912,7 +2922,7 @@ func (c *DynamoDB) ListGlobalTablesRequest(input *ListGlobalTablesInput) (req *r
// //
// Lists all global tables that have a replica in the specified Region. // Lists all global tables that have a replica in the specified Region.
// //
// This method only applies to Version 2017.11.29 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V1.html) // This operation only applies to Version 2017.11.29 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V1.html)
// of global tables. // of global tables.
// //
// Returns awserr.Error for service API and SDK errors. Use runtime type assertions // Returns awserr.Error for service API and SDK errors. Use runtime type assertions
@ -3325,9 +3335,15 @@ func (c *DynamoDB) PutItemRequest(input *PutItemInput) (req *request.Request, ou
// * PutItem in the AWS SDK for Ruby V2 (http://docs.aws.amazon.com/goto/SdkForRubyV2/dynamodb-2012-08-10/PutItem) // * PutItem in the AWS SDK for Ruby V2 (http://docs.aws.amazon.com/goto/SdkForRubyV2/dynamodb-2012-08-10/PutItem)
// //
// When you add an item, the primary key attributes are the only required attributes. // When you add an item, the primary key attributes are the only required attributes.
// Attribute values cannot be null. String and Binary type attributes must have // Attribute values cannot be null.
// lengths greater than zero. Set type attributes cannot be empty. Requests //
// with empty values will be rejected with a ValidationException exception. // Empty String and Binary attribute values are allowed. Attribute values of
// type String and Binary must have a length greater than zero if the attribute
// is used as a key attribute for a table or index. Set type attributes cannot
// be empty.
//
// Invalid Requests with empty values will be rejected with a ValidationException
// exception.
// //
// To prevent a new item from replacing an existing item, use a conditional // To prevent a new item from replacing an existing item, use a conditional
// expression that contains the attribute_not_exists function with the name // expression that contains the attribute_not_exists function with the name
@ -5709,7 +5725,7 @@ func (c *DynamoDB) UpdateTableReplicaAutoScalingRequest(input *UpdateTableReplic
// //
// Updates auto scaling settings on your global tables at once. // Updates auto scaling settings on your global tables at once.
// //
// This method only applies to Version 2019.11.21 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V2.html) // This operation only applies to Version 2019.11.21 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V2.html)
// of global tables. // of global tables.
// //
// Returns awserr.Error for service API and SDK errors. Use runtime type assertions // Returns awserr.Error for service API and SDK errors. Use runtime type assertions
@ -13564,6 +13580,10 @@ type PutItemInput struct {
// types for those attributes must match those of the schema in the table's // types for those attributes must match those of the schema in the table's
// attribute definition. // attribute definition.
// //
// Empty String and Binary attribute values are allowed. Attribute values of
// type String and Binary must have a length greater than zero if the attribute
// is used as a key attribute for a table or index.
//
// For more information about primary keys, see Primary Key (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html#HowItWorks.CoreComponents.PrimaryKey) // For more information about primary keys, see Primary Key (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html#HowItWorks.CoreComponents.PrimaryKey)
// in the Amazon DynamoDB Developer Guide. // in the Amazon DynamoDB Developer Guide.
// //

File diff suppressed because it is too large Load Diff

View File

@ -13,7 +13,6 @@ import (
"github.com/aws/aws-sdk-go/aws" "github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awserr" "github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/aws/request" "github.com/aws/aws-sdk-go/aws/request"
"github.com/aws/aws-sdk-go/internal/sdkio"
) )
const ( const (
@ -25,30 +24,6 @@ const (
appendMD5TxEncoding = "append-md5" appendMD5TxEncoding = "append-md5"
) )
// contentMD5 computes and sets the HTTP Content-MD5 header for requests that
// require it.
func contentMD5(r *request.Request) {
h := md5.New()
if !aws.IsReaderSeekable(r.Body) {
if r.Config.Logger != nil {
r.Config.Logger.Log(fmt.Sprintf(
"Unable to compute Content-MD5 for unseekable body, S3.%s",
r.Operation.Name))
}
return
}
if _, err := copySeekableBody(h, r.Body); err != nil {
r.Error = awserr.New("ContentMD5", "failed to compute body MD5", err)
return
}
// encode the md5 checksum in base64 and set the request header.
v := base64.StdEncoding.EncodeToString(h.Sum(nil))
r.HTTPRequest.Header.Set(contentMD5Header, v)
}
// computeBodyHashes will add Content MD5 and Content Sha256 hashes to the // computeBodyHashes will add Content MD5 and Content Sha256 hashes to the
// request. If the body is not seekable or S3DisableContentMD5Validation set // request. If the body is not seekable or S3DisableContentMD5Validation set
// this handler will be ignored. // this handler will be ignored.
@ -90,7 +65,7 @@ func computeBodyHashes(r *request.Request) {
dst = io.MultiWriter(hashers...) dst = io.MultiWriter(hashers...)
} }
if _, err := copySeekableBody(dst, r.Body); err != nil { if _, err := aws.CopySeekableBody(dst, r.Body); err != nil {
r.Error = awserr.New("BodyHashError", "failed to compute body hashes", err) r.Error = awserr.New("BodyHashError", "failed to compute body hashes", err)
return return
} }
@ -119,28 +94,6 @@ const (
sha256HexEncLen = sha256.Size * 2 // hex.EncodedLen sha256HexEncLen = sha256.Size * 2 // hex.EncodedLen
) )
func copySeekableBody(dst io.Writer, src io.ReadSeeker) (int64, error) {
curPos, err := src.Seek(0, sdkio.SeekCurrent)
if err != nil {
return 0, err
}
// hash the body. seek back to the first position after reading to reset
// the body for transmission. copy errors may be assumed to be from the
// body.
n, err := io.Copy(dst, src)
if err != nil {
return n, err
}
_, err = src.Seek(curPos, sdkio.SeekStart)
if err != nil {
return n, err
}
return n, nil
}
// Adds the x-amz-te: append_md5 header to the request. This requests the service // Adds the x-amz-te: append_md5 header to the request. This requests the service
// responds with a trailing MD5 checksum. // responds with a trailing MD5 checksum.
// //

View File

@ -33,12 +33,6 @@ func defaultInitRequestFn(r *request.Request) {
platformRequestHandlers(r) platformRequestHandlers(r)
switch r.Operation.Name { switch r.Operation.Name {
case opPutBucketCors, opPutBucketLifecycle, opPutBucketPolicy,
opPutBucketTagging, opDeleteObjects, opPutBucketLifecycleConfiguration,
opPutObjectLegalHold, opPutObjectRetention, opPutObjectLockConfiguration,
opPutBucketReplication:
// These S3 operations require Content-MD5 to be set
r.Handlers.Build.PushBack(contentMD5)
case opGetBucketLocation: case opGetBucketLocation:
// GetBucketLocation has custom parsing logic // GetBucketLocation has custom parsing logic
r.Handlers.Unmarshal.PushFront(buildGetBucketLocation) r.Handlers.Unmarshal.PushFront(buildGetBucketLocation)

View File

@ -2,6 +2,7 @@ package s3
import ( import (
"bytes" "bytes"
"io"
"io/ioutil" "io/ioutil"
"net/http" "net/http"
@ -24,17 +25,18 @@ func copyMultipartStatusOKUnmarhsalError(r *request.Request) {
r.HTTPResponse.Body = ioutil.NopCloser(body) r.HTTPResponse.Body = ioutil.NopCloser(body)
defer body.Seek(0, sdkio.SeekStart) defer body.Seek(0, sdkio.SeekStart)
if body.Len() == 0 {
// If there is no body don't attempt to parse the body.
return
}
unmarshalError(r) unmarshalError(r)
if err, ok := r.Error.(awserr.Error); ok && err != nil { if err, ok := r.Error.(awserr.Error); ok && err != nil {
if err.Code() == request.ErrCodeSerialization { if err.Code() == request.ErrCodeSerialization &&
err.OrigErr() != io.EOF {
r.Error = nil r.Error = nil
return return
} }
// if empty payload
if err.OrigErr() == io.EOF {
r.HTTPResponse.StatusCode = http.StatusInternalServerError
} else {
r.HTTPResponse.StatusCode = http.StatusServiceUnavailable r.HTTPResponse.StatusCode = http.StatusServiceUnavailable
} }
} }
}

View File

@ -1,6 +1,7 @@
package s3 package s3
import ( import (
"bytes"
"encoding/xml" "encoding/xml"
"fmt" "fmt"
"io" "io"
@ -45,17 +46,24 @@ func unmarshalError(r *request.Request) {
// Attempt to parse error from body if it is known // Attempt to parse error from body if it is known
var errResp xmlErrorResponse var errResp xmlErrorResponse
err := xmlutil.UnmarshalXMLError(&errResp, r.HTTPResponse.Body) var err error
if err == io.EOF { if r.HTTPResponse.StatusCode >= 200 && r.HTTPResponse.StatusCode < 300 {
// Only capture the error if an unmarshal error occurs that is not EOF, err = s3unmarshalXMLError(&errResp, r.HTTPResponse.Body)
// because S3 might send an error without a error message which causes } else {
// the XML unmarshal to fail with EOF. err = xmlutil.UnmarshalXMLError(&errResp, r.HTTPResponse.Body)
err = nil
} }
if err != nil { if err != nil {
var errorMsg string
if err == io.EOF {
errorMsg = "empty response payload"
} else {
errorMsg = "failed to unmarshal error message"
}
r.Error = awserr.NewRequestFailure( r.Error = awserr.NewRequestFailure(
awserr.New(request.ErrCodeSerialization, awserr.New(request.ErrCodeSerialization,
"failed to unmarshal error message", err), errorMsg, err),
r.HTTPResponse.StatusCode, r.HTTPResponse.StatusCode,
r.RequestID, r.RequestID,
) )
@ -86,3 +94,21 @@ type RequestFailure interface {
// Host ID is the S3 Host ID needed for debug, and contacting support // Host ID is the S3 Host ID needed for debug, and contacting support
HostID() string HostID() string
} }
// s3unmarshalXMLError is s3 specific xml error unmarshaler
// for 200 OK errors and response payloads.
// This function differs from the xmlUtil.UnmarshalXMLError
// func. It does not ignore the EOF error and passes it up.
// Related to bug fix for `s3 200 OK response with empty payload`
func s3unmarshalXMLError(v interface{}, stream io.Reader) error {
var errBuf bytes.Buffer
body := io.TeeReader(stream, &errBuf)
err := xml.NewDecoder(body).Decode(v)
if err != nil && err != io.EOF {
return awserr.NewUnmarshalError(err,
"failed to unmarshal error message", errBuf.Bytes())
}
return err
}

View File

@ -1788,7 +1788,7 @@ type AssumeRoleWithSAMLInput struct {
// in the IAM User Guide. // in the IAM User Guide.
// //
// SAMLAssertion is a required field // SAMLAssertion is a required field
SAMLAssertion *string `min:"4" type:"string" required:"true"` SAMLAssertion *string `min:"4" type:"string" required:"true" sensitive:"true"`
} }
// String returns the string representation // String returns the string representation
@ -2100,7 +2100,7 @@ type AssumeRoleWithWebIdentityInput struct {
// the application makes an AssumeRoleWithWebIdentity call. // the application makes an AssumeRoleWithWebIdentity call.
// //
// WebIdentityToken is a required field // WebIdentityToken is a required field
WebIdentityToken *string `min:"4" type:"string" required:"true"` WebIdentityToken *string `min:"4" type:"string" required:"true" sensitive:"true"`
} }
// String returns the string representation // String returns the string representation

View File

@ -6,14 +6,21 @@ linters:
disable-all: true disable-all: true
enable: enable:
- deadcode - deadcode
- dogsled
- errcheck - errcheck
- goconst
- gofmt - gofmt
- gomnd
- gosimple - gosimple
- ineffassign - ineffassign
- interfacer
- misspell - misspell
- scopelint
- staticcheck - staticcheck
- structcheck - structcheck
- unconvert - unconvert
- unparam
- unused - unused
- typecheck
- varcheck - varcheck
- vet - vet

View File

@ -1,20 +0,0 @@
dist: xenial
language: go
go:
- "1.13.x"
matrix:
fast_finish: true
allow_failures:
- go: tip
install:
- make tools
script:
- make lint
- go test -timeout=30s -parallel=4 -v ./...
branches:
only:
- master

View File

@ -1,3 +1,25 @@
# v0.5.0 (June 4, 2020)
BREAKING CHANGES
* Credential ordering has changed from static, environment, shared credentials, EC2 metadata, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata) to static, environment, shared credentials, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata). #20
* The `AWS_METADATA_TIMEOUT` environment variable no longer has any effect as we now depend on the default AWS Go SDK EC2 Metadata client timeout of one second with two retries. #20 / #44
ENHANCEMENTS
* Always enable AWS shared configuration file support (no longer require `AWS_SDK_LOAD_CONFIG` environment variable) #38
* Automatically expand `~` prefix for home directories in shared credentials filename handling #40
* Support assume role duration, policy ARNs, tags, and transitive tag keys via configuration #39
* Add `CannotAssumeRoleError` and `NoValidCredentialSourcesError` error types with helpers #42
BUG FIXES
* Properly use custom STS endpoint during AssumeRole API calls triggered by Terraform AWS Provider and S3 Backend configurations #32
* Properly use custom EC2 metadata endpoint during API calls triggered by fallback credentials lookup #32
* Prefer shared configuration handling over EC2 metadata #20
* Prefer ECS credentials over EC2 metadata #20
* Remove hardcoded AWS Provider messaging in error messages #31 / #42
# v0.4.0 (October 3, 2019) # v0.4.0 (October 3, 2019)
BUG FIXES BUG FIXES

View File

@ -13,29 +13,21 @@ import (
awsCredentials "github.com/aws/aws-sdk-go/aws/credentials" awsCredentials "github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/credentials/ec2rolecreds" "github.com/aws/aws-sdk-go/aws/credentials/ec2rolecreds"
"github.com/aws/aws-sdk-go/aws/credentials/stscreds" "github.com/aws/aws-sdk-go/aws/credentials/stscreds"
"github.com/aws/aws-sdk-go/aws/defaults"
"github.com/aws/aws-sdk-go/aws/ec2metadata" "github.com/aws/aws-sdk-go/aws/ec2metadata"
"github.com/aws/aws-sdk-go/aws/session" "github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/iam" "github.com/aws/aws-sdk-go/service/iam"
"github.com/aws/aws-sdk-go/service/sts" "github.com/aws/aws-sdk-go/service/sts"
"github.com/hashicorp/go-cleanhttp" "github.com/hashicorp/go-cleanhttp"
"github.com/hashicorp/go-multierror" "github.com/hashicorp/go-multierror"
homedir "github.com/mitchellh/go-homedir"
) )
const ( const (
// errMsgNoValidCredentialSources error getting credentials // Default amount of time for EC2/ECS metadata client operations.
errMsgNoValidCredentialSources = `No valid credential sources found for AWS Provider. // Keep this value low to prevent long delays in non-EC2/ECS environments.
Please see https://terraform.io/docs/providers/aws/index.html for more information on DefaultMetadataClientTimeout = 100 * time.Millisecond
providing credentials for the AWS Provider`
) )
var (
// ErrNoValidCredentialSources indicates that no credentials source could be found
ErrNoValidCredentialSources = errNoValidCredentialSources()
)
func errNoValidCredentialSources() error { return errors.New(errMsgNoValidCredentialSources) }
// GetAccountIDAndPartition gets the account ID and associated partition. // GetAccountIDAndPartition gets the account ID and associated partition.
func GetAccountIDAndPartition(iamconn *iam.IAM, stsconn *sts.STS, authProviderName string) (string, string, error) { func GetAccountIDAndPartition(iamconn *iam.IAM, stsconn *sts.STS, authProviderName string) (string, string, error) {
var accountID, partition string var accountID, partition string
@ -75,7 +67,7 @@ func GetAccountIDAndPartitionFromEC2Metadata() (string, string, error) {
setOptionalEndpoint(cfg) setOptionalEndpoint(cfg)
sess, err := session.NewSession(cfg) sess, err := session.NewSession(cfg)
if err != nil { if err != nil {
return "", "", fmt.Errorf("error creating EC2 Metadata session: %s", err) return "", "", fmt.Errorf("error creating EC2 Metadata session: %w", err)
} }
metadataClient := ec2metadata.New(sess) metadataClient := ec2metadata.New(sess)
@ -84,7 +76,7 @@ func GetAccountIDAndPartitionFromEC2Metadata() (string, string, error) {
// We can end up here if there's an issue with the instance metadata service // We can end up here if there's an issue with the instance metadata service
// or if we're getting credentials from AdRoll's Hologram (in which case IAMInfo will // or if we're getting credentials from AdRoll's Hologram (in which case IAMInfo will
// error out). // error out).
err = fmt.Errorf("failed getting account information via EC2 Metadata IAM information: %s", err) err = fmt.Errorf("failed getting account information via EC2 Metadata IAM information: %w", err)
log.Printf("[DEBUG] %s", err) log.Printf("[DEBUG] %s", err)
return "", "", err return "", "", err
} }
@ -107,7 +99,7 @@ func GetAccountIDAndPartitionFromIAMGetUser(iamconn *iam.IAM) (string, string, e
return "", "", nil return "", "", nil
} }
} }
err = fmt.Errorf("failed getting account information via iam:GetUser: %s", err) err = fmt.Errorf("failed getting account information via iam:GetUser: %w", err)
log.Printf("[DEBUG] %s", err) log.Printf("[DEBUG] %s", err)
return "", "", err return "", "", err
} }
@ -130,7 +122,7 @@ func GetAccountIDAndPartitionFromIAMListRoles(iamconn *iam.IAM) (string, string,
MaxItems: aws.Int64(int64(1)), MaxItems: aws.Int64(int64(1)),
}) })
if err != nil { if err != nil {
err = fmt.Errorf("failed getting account information via iam:ListRoles: %s", err) err = fmt.Errorf("failed getting account information via iam:ListRoles: %w", err)
log.Printf("[DEBUG] %s", err) log.Printf("[DEBUG] %s", err)
return "", "", err return "", "", err
} }
@ -151,7 +143,7 @@ func GetAccountIDAndPartitionFromSTSGetCallerIdentity(stsconn *sts.STS) (string,
output, err := stsconn.GetCallerIdentity(&sts.GetCallerIdentityInput{}) output, err := stsconn.GetCallerIdentity(&sts.GetCallerIdentityInput{})
if err != nil { if err != nil {
return "", "", fmt.Errorf("error calling sts:GetCallerIdentity: %s", err) return "", "", fmt.Errorf("error calling sts:GetCallerIdentity: %w", err)
} }
if output == nil || output.Arn == nil { if output == nil || output.Arn == nil {
@ -177,37 +169,30 @@ func parseAccountIDAndPartitionFromARN(inputARN string) (string, string, error)
func GetCredentialsFromSession(c *Config) (*awsCredentials.Credentials, error) { func GetCredentialsFromSession(c *Config) (*awsCredentials.Credentials, error) {
log.Printf("[INFO] Attempting to use session-derived credentials") log.Printf("[INFO] Attempting to use session-derived credentials")
var sess *session.Session // Avoid setting HTTPClient here as it will prevent the ec2metadata
var err error // client from automatically lowering the timeout to 1 second.
if c.Profile == "" {
sess, err = session.NewSession()
if err != nil {
return nil, ErrNoValidCredentialSources
}
} else {
options := &session.Options{ options := &session.Options{
Config: aws.Config{ Config: aws.Config{
HTTPClient: cleanhttp.DefaultClient(), EndpointResolver: c.EndpointResolver(),
MaxRetries: aws.Int(0), MaxRetries: aws.Int(0),
Region: aws.String(c.Region), Region: aws.String(c.Region),
}, },
Profile: c.Profile,
SharedConfigState: session.SharedConfigEnable,
} }
options.Profile = c.Profile
options.SharedConfigState = session.SharedConfigEnable
sess, err = session.NewSessionWithOptions(*options) sess, err := session.NewSessionWithOptions(*options)
if err != nil { if err != nil {
if IsAWSErr(err, "NoCredentialProviders", "") { if IsAWSErr(err, "NoCredentialProviders", "") {
return nil, ErrNoValidCredentialSources return nil, c.NewNoValidCredentialSourcesError(err)
}
return nil, fmt.Errorf("Error creating AWS session: %s", err)
} }
return nil, fmt.Errorf("Error creating AWS session: %w", err)
} }
creds := sess.Config.Credentials creds := sess.Config.Credentials
cp, err := sess.Config.Credentials.Get() cp, err := sess.Config.Credentials.Get()
if err != nil { if err != nil {
return nil, ErrNoValidCredentialSources return nil, c.NewNoValidCredentialSourcesError(err)
} }
log.Printf("[INFO] Successfully derived credentials from session") log.Printf("[INFO] Successfully derived credentials from session")
@ -216,10 +201,16 @@ func GetCredentialsFromSession(c *Config) (*awsCredentials.Credentials, error) {
} }
// GetCredentials gets credentials from the environment, shared credentials, // GetCredentials gets credentials from the environment, shared credentials,
// or the session (which may include a credential process). GetCredentials also // the session (which may include a credential process), or ECS/EC2 metadata endpoints.
// validates the credentials and the ability to assume a role or will return an // GetCredentials also validates the credentials and the ability to assume a role
// error if unsuccessful. // or will return an error if unsuccessful.
func GetCredentials(c *Config) (*awsCredentials.Credentials, error) { func GetCredentials(c *Config) (*awsCredentials.Credentials, error) {
sharedCredentialsFilename, err := homedir.Expand(c.CredsFilename)
if err != nil {
return nil, fmt.Errorf("error expanding shared credentials filename: %w", err)
}
// build a chain provider, lazy-evaluated by aws-sdk // build a chain provider, lazy-evaluated by aws-sdk
providers := []awsCredentials.Provider{ providers := []awsCredentials.Provider{
&awsCredentials.StaticProvider{Value: awsCredentials.Value{ &awsCredentials.StaticProvider{Value: awsCredentials.Value{
@ -229,70 +220,11 @@ func GetCredentials(c *Config) (*awsCredentials.Credentials, error) {
}}, }},
&awsCredentials.EnvProvider{}, &awsCredentials.EnvProvider{},
&awsCredentials.SharedCredentialsProvider{ &awsCredentials.SharedCredentialsProvider{
Filename: c.CredsFilename, Filename: sharedCredentialsFilename,
Profile: c.Profile, Profile: c.Profile,
}, },
} }
// Build isolated HTTP client to avoid issues with globally-shared settings
client := cleanhttp.DefaultClient()
// Keep the default timeout (100ms) low as we don't want to wait in non-EC2 environments
client.Timeout = 100 * time.Millisecond
const userTimeoutEnvVar = "AWS_METADATA_TIMEOUT"
userTimeout := os.Getenv(userTimeoutEnvVar)
if userTimeout != "" {
newTimeout, err := time.ParseDuration(userTimeout)
if err == nil {
if newTimeout.Nanoseconds() > 0 {
client.Timeout = newTimeout
} else {
log.Printf("[WARN] Non-positive value of %s (%s) is meaningless, ignoring", userTimeoutEnvVar, newTimeout.String())
}
} else {
log.Printf("[WARN] Error converting %s to time.Duration: %s", userTimeoutEnvVar, err)
}
}
log.Printf("[INFO] Setting AWS metadata API timeout to %s", client.Timeout.String())
cfg := &aws.Config{
HTTPClient: client,
}
usedEndpoint := setOptionalEndpoint(cfg)
// Add the default AWS provider for ECS Task Roles if the relevant env variable is set
if uri := os.Getenv("AWS_CONTAINER_CREDENTIALS_RELATIVE_URI"); len(uri) > 0 {
providers = append(providers, defaults.RemoteCredProvider(*cfg, defaults.Handlers()))
log.Print("[INFO] ECS container credentials detected, RemoteCredProvider added to auth chain")
}
if !c.SkipMetadataApiCheck {
// Real AWS should reply to a simple metadata request.
// We check it actually does to ensure something else didn't just
// happen to be listening on the same IP:Port
ec2Session, err := session.NewSession(cfg)
if err != nil {
return nil, fmt.Errorf("error creating EC2 Metadata session: %s", err)
}
metadataClient := ec2metadata.New(ec2Session)
if metadataClient.Available() {
providers = append(providers, &ec2rolecreds.EC2RoleProvider{
Client: metadataClient,
})
log.Print("[INFO] AWS EC2 instance detected via default metadata" +
" API endpoint, EC2RoleProvider added to the auth chain")
} else {
if usedEndpoint == "" {
usedEndpoint = "default location"
}
log.Printf("[INFO] Ignoring AWS metadata API endpoint at %s "+
"as it doesn't return any instance-id", usedEndpoint)
}
}
// Validate the credentials before returning them // Validate the credentials before returning them
creds := awsCredentials.NewChainCredentials(providers) creds := awsCredentials.NewChainCredentials(providers)
cp, err := creds.Get() cp, err := creds.Get()
@ -303,7 +235,7 @@ func GetCredentials(c *Config) (*awsCredentials.Credentials, error) {
return nil, err return nil, err
} }
} else { } else {
return nil, fmt.Errorf("Error loading credentials for AWS Provider: %s", err) return nil, fmt.Errorf("Error loading credentials for AWS Provider: %w", err)
} }
} else { } else {
log.Printf("[INFO] AWS Auth provider used: %q", cp.ProviderName) log.Printf("[INFO] AWS Auth provider used: %q", cp.ProviderName)
@ -316,11 +248,12 @@ func GetCredentials(c *Config) (*awsCredentials.Credentials, error) {
// Otherwise we need to construct an STS client with the main credentials, and verify // Otherwise we need to construct an STS client with the main credentials, and verify
// that we can assume the defined role. // that we can assume the defined role.
log.Printf("[INFO] Attempting to AssumeRole %s (SessionName: %q, ExternalId: %q, Policy: %q)", log.Printf("[INFO] Attempting to AssumeRole %s (SessionName: %q, ExternalId: %q)",
c.AssumeRoleARN, c.AssumeRoleSessionName, c.AssumeRoleExternalID, c.AssumeRolePolicy) c.AssumeRoleARN, c.AssumeRoleSessionName, c.AssumeRoleExternalID)
awsConfig := &aws.Config{ awsConfig := &aws.Config{
Credentials: creds, Credentials: creds,
EndpointResolver: c.EndpointResolver(),
Region: aws.String(c.Region), Region: aws.String(c.Region),
MaxRetries: aws.Int(c.MaxRetries), MaxRetries: aws.Int(c.MaxRetries),
HTTPClient: cleanhttp.DefaultClient(), HTTPClient: cleanhttp.DefaultClient(),
@ -329,7 +262,7 @@ func GetCredentials(c *Config) (*awsCredentials.Credentials, error) {
assumeRoleSession, err := session.NewSession(awsConfig) assumeRoleSession, err := session.NewSession(awsConfig)
if err != nil { if err != nil {
return nil, fmt.Errorf("error creating assume role session: %s", err) return nil, fmt.Errorf("error creating assume role session: %w", err)
} }
stsclient := sts.New(assumeRoleSession) stsclient := sts.New(assumeRoleSession)
@ -337,31 +270,60 @@ func GetCredentials(c *Config) (*awsCredentials.Credentials, error) {
Client: stsclient, Client: stsclient,
RoleARN: c.AssumeRoleARN, RoleARN: c.AssumeRoleARN,
} }
if c.AssumeRoleSessionName != "" {
assumeRoleProvider.RoleSessionName = c.AssumeRoleSessionName if c.AssumeRoleDurationSeconds > 0 {
assumeRoleProvider.Duration = time.Duration(c.AssumeRoleDurationSeconds) * time.Second
} }
if c.AssumeRoleExternalID != "" { if c.AssumeRoleExternalID != "" {
assumeRoleProvider.ExternalID = aws.String(c.AssumeRoleExternalID) assumeRoleProvider.ExternalID = aws.String(c.AssumeRoleExternalID)
} }
if c.AssumeRolePolicy != "" { if c.AssumeRolePolicy != "" {
assumeRoleProvider.Policy = aws.String(c.AssumeRolePolicy) assumeRoleProvider.Policy = aws.String(c.AssumeRolePolicy)
} }
if len(c.AssumeRolePolicyARNs) > 0 {
var policyDescriptorTypes []*sts.PolicyDescriptorType
for _, policyARN := range c.AssumeRolePolicyARNs {
policyDescriptorType := &sts.PolicyDescriptorType{
Arn: aws.String(policyARN),
}
policyDescriptorTypes = append(policyDescriptorTypes, policyDescriptorType)
}
assumeRoleProvider.PolicyArns = policyDescriptorTypes
}
if c.AssumeRoleSessionName != "" {
assumeRoleProvider.RoleSessionName = c.AssumeRoleSessionName
}
if len(c.AssumeRoleTags) > 0 {
var tags []*sts.Tag
for k, v := range c.AssumeRoleTags {
tag := &sts.Tag{
Key: aws.String(k),
Value: aws.String(v),
}
tags = append(tags, tag)
}
assumeRoleProvider.Tags = tags
}
if len(c.AssumeRoleTransitiveTagKeys) > 0 {
assumeRoleProvider.TransitiveTagKeys = aws.StringSlice(c.AssumeRoleTransitiveTagKeys)
}
providers = []awsCredentials.Provider{assumeRoleProvider} providers = []awsCredentials.Provider{assumeRoleProvider}
assumeRoleCreds := awsCredentials.NewChainCredentials(providers) assumeRoleCreds := awsCredentials.NewChainCredentials(providers)
_, err = assumeRoleCreds.Get() _, err = assumeRoleCreds.Get()
if err != nil { if err != nil {
if awsErr, ok := err.(awserr.Error); ok && awsErr.Code() == "NoCredentialProviders" { return nil, c.NewCannotAssumeRoleError(err)
return nil, fmt.Errorf("The role %q cannot be assumed.\n\n"+
" There are a number of possible causes of this - the most common are:\n"+
" * The credentials used in order to assume the role are invalid\n"+
" * The credentials do not have appropriate permission to assume the role\n"+
" * The role ARN is not valid",
c.AssumeRoleARN)
}
return nil, fmt.Errorf("Error loading credentials for AWS Provider: %s", err)
} }
return assumeRoleCreds, nil return assumeRoleCreds, nil

View File

@ -1,6 +1,7 @@
package awsbase package awsbase
import ( import (
"errors"
"strings" "strings"
"github.com/aws/aws-sdk-go/aws/awserr" "github.com/aws/aws-sdk-go/aws/awserr"
@ -11,19 +12,15 @@ import (
// * Error.Code() matches code // * Error.Code() matches code
// * Error.Message() contains message // * Error.Message() contains message
func IsAWSErr(err error, code string, message string) bool { func IsAWSErr(err error, code string, message string) bool {
awsErr, ok := err.(awserr.Error) var awsErr awserr.Error
if !ok { if errors.As(err, &awsErr) {
return false return awsErr.Code() == code && strings.Contains(awsErr.Message(), message)
} }
if awsErr.Code() != code {
return false return false
} }
return strings.Contains(awsErr.Message(), message)
}
// IsAWSErrExtended returns true if the error matches all these conditions: // IsAWSErrExtended returns true if the error matches all these conditions:
// * err is of type awserr.Error // * err is of type awserr.Error
// * Error.Code() matches code // * Error.Code() matches code
@ -33,5 +30,15 @@ func IsAWSErrExtended(err error, code string, message string, origErrMessage str
if !IsAWSErr(err, code, message) { if !IsAWSErr(err, code, message) {
return false return false
} }
return strings.Contains(err.(awserr.Error).OrigErr().Error(), origErrMessage)
if origErrMessage == "" {
return true
}
// Ensure OrigErr() is non-nil, to prevent panics
if origErr := err.(awserr.Error).OrigErr(); origErr != nil {
return strings.Contains(origErr.Error(), origErrMessage)
}
return false
} }

View File

@ -3,9 +3,15 @@ package awsbase
type Config struct { type Config struct {
AccessKey string AccessKey string
AssumeRoleARN string AssumeRoleARN string
AssumeRoleDurationSeconds int
AssumeRoleExternalID string AssumeRoleExternalID string
AssumeRolePolicy string AssumeRolePolicy string
AssumeRolePolicyARNs []string
AssumeRoleSessionName string AssumeRoleSessionName string
AssumeRoleTags map[string]string
AssumeRoleTransitiveTagKeys []string
CallerDocumentationURL string
CallerName string
CredsFilename string CredsFilename string
DebugLogging bool DebugLogging bool
IamEndpoint string IamEndpoint string

View File

@ -0,0 +1,46 @@
package awsbase
import (
"log"
"os"
"github.com/aws/aws-sdk-go/aws/ec2metadata"
"github.com/aws/aws-sdk-go/aws/endpoints"
"github.com/aws/aws-sdk-go/service/iam"
"github.com/aws/aws-sdk-go/service/sts"
)
func (c *Config) EndpointResolver() endpoints.Resolver {
resolver := func(service, region string, optFns ...func(*endpoints.Options)) (endpoints.ResolvedEndpoint, error) {
// Ensure we pass all existing information (e.g. SigningRegion) and
// only override the URL, otherwise a MissingRegion error can occur
// when aws.Config.Region is not defined.
resolvedEndpoint, err := endpoints.DefaultResolver().EndpointFor(service, region, optFns...)
if err != nil {
return resolvedEndpoint, err
}
switch service {
case ec2metadata.ServiceName:
if endpoint := os.Getenv("AWS_METADATA_URL"); endpoint != "" {
log.Printf("[INFO] Setting custom EC2 metadata endpoint: %s", endpoint)
resolvedEndpoint.URL = endpoint
}
case iam.ServiceName:
if endpoint := c.IamEndpoint; endpoint != "" {
log.Printf("[INFO] Setting custom IAM endpoint: %s", endpoint)
resolvedEndpoint.URL = endpoint
}
case sts.ServiceName:
if endpoint := c.StsEndpoint; endpoint != "" {
log.Printf("[INFO] Setting custom STS endpoint: %s", endpoint)
resolvedEndpoint.URL = endpoint
}
}
return resolvedEndpoint, nil
}
return endpoints.ResolverFunc(resolver)
}

76
vendor/github.com/hashicorp/aws-sdk-go-base/errors.go generated vendored Normal file
View File

@ -0,0 +1,76 @@
package awsbase
import (
"errors"
"fmt"
)
// CannotAssumeRoleError occurs when AssumeRole cannot complete.
type CannotAssumeRoleError struct {
Config *Config
Err error
}
func (e CannotAssumeRoleError) Error() string {
if e.Config == nil {
return fmt.Sprintf("cannot assume role: %s", e.Err)
}
return fmt.Sprintf(`IAM Role (%s) cannot be assumed.
There are a number of possible causes of this - the most common are:
* The credentials used in order to assume the role are invalid
* The credentials do not have appropriate permission to assume the role
* The role ARN is not valid
Error: %s
`, e.Config.AssumeRoleARN, e.Err)
}
func (e CannotAssumeRoleError) Unwrap() error {
return e.Err
}
// IsCannotAssumeRoleError returns true if the error contains the CannotAssumeRoleError type.
func IsCannotAssumeRoleError(err error) bool {
var e CannotAssumeRoleError
return errors.As(err, &e)
}
func (c *Config) NewCannotAssumeRoleError(err error) CannotAssumeRoleError {
return CannotAssumeRoleError{Config: c, Err: err}
}
// NoValidCredentialSourcesError occurs when all credential lookup methods have been exhausted without results.
type NoValidCredentialSourcesError struct {
Config *Config
Err error
}
func (e NoValidCredentialSourcesError) Error() string {
if e.Config == nil {
return fmt.Sprintf("no valid credential sources found: %s", e.Err)
}
return fmt.Sprintf(`no valid credential sources for %s found.
Please see %s
for more information about providing credentials.
Error: %s
`, e.Config.CallerName, e.Config.CallerDocumentationURL, e.Err)
}
func (e NoValidCredentialSourcesError) Unwrap() error {
return e.Err
}
// IsNoValidCredentialSourcesError returns true if the error contains the NoValidCredentialSourcesError type.
func IsNoValidCredentialSourcesError(err error) bool {
var e NoValidCredentialSourcesError
return errors.As(err, &e)
}
func (c *Config) NewNoValidCredentialSourcesError(err error) NoValidCredentialSourcesError {
return NoValidCredentialSourcesError{Config: c, Err: err}
}

View File

@ -1,12 +1,10 @@
module github.com/hashicorp/aws-sdk-go-base module github.com/hashicorp/aws-sdk-go-base
require ( require (
github.com/aws/aws-sdk-go v1.25.3 github.com/aws/aws-sdk-go v1.31.9
github.com/hashicorp/go-cleanhttp v0.5.0 github.com/hashicorp/go-cleanhttp v0.5.0
github.com/hashicorp/go-multierror v1.0.0 github.com/hashicorp/go-multierror v1.0.0
github.com/stretchr/testify v1.3.0 // indirect github.com/mitchellh/go-homedir v1.1.0
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd // indirect
golang.org/x/text v0.3.0 // indirect
) )
go 1.13 go 1.13

View File

@ -1,23 +1,31 @@
github.com/aws/aws-sdk-go v1.16.36 h1:POeH34ZME++pr7GBGh+ZO6Y5kOwSMQpqp5BGUgooJ6k= github.com/aws/aws-sdk-go v1.31.9 h1:n+b34ydVfgC30j0Qm69yaapmjejQPW2BoDBX7Uy/tLI=
github.com/aws/aws-sdk-go v1.16.36/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo= github.com/aws/aws-sdk-go v1.31.9/go.mod h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZveU8YkpAk0=
github.com/aws/aws-sdk-go v1.25.3 h1:uM16hIw9BotjZKMZlX05SN2EFtaWfi/NonPKIARiBLQ=
github.com/aws/aws-sdk-go v1.25.3/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8= github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/go-sql-driver/mysql v1.5.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg=
github.com/hashicorp/errwrap v1.0.0 h1:hLrqtEDnRye3+sgx6z4qVLNuviH3MR5aQ0ykNJa/UYA= github.com/hashicorp/errwrap v1.0.0 h1:hLrqtEDnRye3+sgx6z4qVLNuviH3MR5aQ0ykNJa/UYA=
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/go-cleanhttp v0.5.0 h1:wvCrVc9TjDls6+YGAF2hAifE1E5U1+b4tH6KdvN3Gig= github.com/hashicorp/go-cleanhttp v0.5.0 h1:wvCrVc9TjDls6+YGAF2hAifE1E5U1+b4tH6KdvN3Gig=
github.com/hashicorp/go-cleanhttp v0.5.0/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80= github.com/hashicorp/go-cleanhttp v0.5.0/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
github.com/hashicorp/go-multierror v1.0.0 h1:iVjPR7a6H0tWELX5NxNe7bYopibicUzc7uPribsnS6o= github.com/hashicorp/go-multierror v1.0.0 h1:iVjPR7a6H0tWELX5NxNe7bYopibicUzc7uPribsnS6o=
github.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk= github.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk=
github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af h1:pmfjZENx5imkbgOkpRUYLnmbU7UEFbjtDA2hxJ1ichM= github.com/jmespath/go-jmespath v0.3.0 h1:OS12ieG61fsCg5+qLJ+SsW9NicxNkg3b25OyT2yCeUc=
github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k= github.com/jmespath/go-jmespath v0.3.0/go.mod h1:9QtRXoHjLGCJ5IBSaohpXITPlowMeeYCZ7fLUTSywik=
github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q= github.com/stretchr/testify v1.5.1 h1:nOGnQDM7FYENwehXlg/kFVnos3rEvtKTjRvOWSzb6H4=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd h1:HuTn7WObtcDo9uEEU7rEqL0jYthdXAmZ6PP+meazmaU= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20200202094626-16171245cfb2 h1:CCH4IOTTfewWjGOlSp+zGcjutRKlBEZQ6wTn8ozI/nI=
golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg= golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=

View File

@ -2,6 +2,7 @@ package awsbase
import ( import (
"bytes" "bytes"
"encoding/json"
"fmt" "fmt"
"log" "log"
"net/http" "net/http"
@ -19,7 +20,7 @@ func MockAwsApiServer(svcName string, endpoints []*MockEndpoint) *httptest.Serve
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
buf := new(bytes.Buffer) buf := new(bytes.Buffer)
if _, err := buf.ReadFrom(r.Body); err != nil { if _, err := buf.ReadFrom(r.Body); err != nil {
w.WriteHeader(500) w.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(w, "Error reading from HTTP Request Body: %s", err) fmt.Fprintf(w, "Error reading from HTTP Request Body: %s", err)
return return
} }
@ -43,7 +44,7 @@ func MockAwsApiServer(svcName string, endpoints []*MockEndpoint) *httptest.Serve
} }
} }
w.WriteHeader(400) w.WriteHeader(http.StatusBadRequest)
})) }))
return ts return ts
@ -73,20 +74,43 @@ func awsMetadataApiMock(responses []*MetadataResponse) func() {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/plain") w.Header().Set("Content-Type", "text/plain")
w.Header().Add("Server", "MockEC2") w.Header().Add("Server", "MockEC2")
log.Printf("[DEBUG] Mocker server received request to %q", r.RequestURI) log.Printf("[DEBUG] Mock EC2 metadata server received request: %s", r.RequestURI)
for _, e := range responses { for _, e := range responses {
if r.RequestURI == e.Uri { if r.RequestURI == e.Uri {
fmt.Fprintln(w, e.Body) fmt.Fprintln(w, e.Body)
return return
} }
} }
w.WriteHeader(400) w.WriteHeader(http.StatusBadRequest)
})) }))
os.Setenv("AWS_METADATA_URL", ts.URL+"/latest") os.Setenv("AWS_METADATA_URL", ts.URL+"/latest")
return ts.Close return ts.Close
} }
// ecsCredentialsApiMock establishes a httptest server to mock out the ECS credentials API.
func ecsCredentialsApiMock() func() {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.Header().Add("Server", "MockECS")
log.Printf("[DEBUG] Mock ECS credentials server received request: %s", r.RequestURI)
if r.RequestURI == "/creds" {
_ = json.NewEncoder(w).Encode(map[string]string{
"AccessKeyId": "EcsCredentialsAccessKey",
"Expiration": time.Now().UTC().Format(time.RFC3339),
"RoleArn": "arn:aws:iam::000000000000:role/EcsCredentials",
"SecretAccessKey": "EcsCredentialsSecretKey",
"Token": "EcsCredentialsSessionToken",
})
return
}
w.WriteHeader(http.StatusBadRequest)
}))
os.Setenv("AWS_CONTAINER_CREDENTIALS_FULL_URI", ts.URL+"/creds")
return ts.Close
}
// MockEndpoint represents a basic request and response that can be used for creating simple httptest server routes. // MockEndpoint represents a basic request and response that can be used for creating simple httptest server routes.
type MockEndpoint struct { type MockEndpoint struct {
Request *MockRequest Request *MockRequest
@ -119,13 +143,17 @@ var ec2metadata_instanceIdEndpoint = &MetadataResponse{
} }
var ec2metadata_securityCredentialsEndpoints = []*MetadataResponse{ var ec2metadata_securityCredentialsEndpoints = []*MetadataResponse{
{
Uri: "/latest/api/token",
Body: "Ec2MetadataApiToken",
},
{ {
Uri: "/latest/meta-data/iam/security-credentials/", Uri: "/latest/meta-data/iam/security-credentials/",
Body: "test_role", Body: "test_role",
}, },
{ {
Uri: "/latest/meta-data/iam/security-credentials/test_role", Uri: "/latest/meta-data/iam/security-credentials/test_role",
Body: "{\"Code\":\"Success\",\"LastUpdated\":\"2015-12-11T17:17:25Z\",\"Type\":\"AWS-HMAC\",\"AccessKeyId\":\"somekey\",\"SecretAccessKey\":\"somesecret\",\"Token\":\"sometoken\"}", Body: "{\"Code\":\"Success\",\"LastUpdated\":\"2015-12-11T17:17:25Z\",\"Type\":\"AWS-HMAC\",\"AccessKeyId\":\"Ec2MetadataAccessKey\",\"SecretAccessKey\":\"Ec2MetadataSecretKey\",\"Token\":\"Ec2MetadataSessionToken\"}",
}, },
} }
@ -165,6 +193,54 @@ const iamResponse_GetUser_unauthorized = `<ErrorResponse xmlns="https://iam.amaz
<RequestId>7a62c49f-347e-4fc4-9331-6e8eEXAMPLE</RequestId> <RequestId>7a62c49f-347e-4fc4-9331-6e8eEXAMPLE</RequestId>
</ErrorResponse>` </ErrorResponse>`
var stsResponse_AssumeRole_valid = fmt.Sprintf(`<AssumeRoleResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
<AssumeRoleResult>
<AssumedRoleUser>
<Arn>arn:aws:sts::555555555555:assumed-role/role/AssumeRoleSessionName</Arn>
<AssumedRoleId>ARO123EXAMPLE123:AssumeRoleSessionName</AssumedRoleId>
</AssumedRoleUser>
<Credentials>
<AccessKeyId>AssumeRoleAccessKey</AccessKeyId>
<SecretAccessKey>AssumeRoleSecretKey</SecretAccessKey>
<SessionToken>AssumeRoleSessionToken</SessionToken>
<Expiration>%s</Expiration>
</Credentials>
</AssumeRoleResult>
<ResponseMetadata>
<RequestId>01234567-89ab-cdef-0123-456789abcdef</RequestId>
</ResponseMetadata>
</AssumeRoleResponse>`, time.Now().UTC().Format(time.RFC3339))
const stsResponse_AssumeRole_InvalidClientTokenId = `<ErrorResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
<Error>
<Type>Sender</Type>
<Code>InvalidClientTokenId</Code>
<Message>The security token included in the request is invalid.</Message>
</Error>
<RequestId>4d0cf5ec-892a-4d3f-84e4-30e9987d9bdd</RequestId>
</ErrorResponse>`
var stsResponse_AssumeRoleWithWebIdentity_valid = fmt.Sprintf(`<AssumeRoleWithWebIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
<AssumeRoleWithWebIdentityResult>
<SubjectFromWebIdentityToken>amzn1.account.AF6RHO7KZU5XRVQJGXK6HB56KR2A</SubjectFromWebIdentityToken>
<Audience>client.6666666666666666666.6666@apps.example.com</Audience>
<AssumedRoleUser>
<Arn>arn:aws:sts::666666666666:assumed-role/FederatedWebIdentityRole/AssumeRoleWithWebIdentitySessionName</Arn>
<AssumedRoleId>ARO123EXAMPLE123:AssumeRoleWithWebIdentitySessionName</AssumedRoleId>
</AssumedRoleUser>
<Credentials>
<SessionToken>AssumeRoleWithWebIdentitySessionToken</SessionToken>
<SecretAccessKey>AssumeRoleWithWebIdentitySecretKey</SecretAccessKey>
<Expiration>%s</Expiration>
<AccessKeyId>AssumeRoleWithWebIdentityAccessKey</AccessKeyId>
</Credentials>
<Provider>www.amazon.com</Provider>
</AssumeRoleWithWebIdentityResult>
<ResponseMetadata>
<RequestId>01234567-89ab-cdef-0123-456789abcdef</RequestId>
</ResponseMetadata>
</AssumeRoleWithWebIdentityResponse>`, time.Now().UTC().Format(time.RFC3339))
const stsResponse_GetCallerIdentity_valid = `<GetCallerIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/"> const stsResponse_GetCallerIdentity_valid = `<GetCallerIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
<GetCallerIdentityResult> <GetCallerIdentityResult>
<Arn>arn:aws:iam::222222222222:user/Alice</Arn> <Arn>arn:aws:iam::222222222222:user/Alice</Arn>
@ -228,3 +304,5 @@ const iamResponse_ListRoles_unauthorized = `<ErrorResponse xmlns="https://iam.am
</Error> </Error>
<RequestId>7a62c49f-347e-4fc4-9331-6e8eEXAMPLE</RequestId> <RequestId>7a62c49f-347e-4fc4-9331-6e8eEXAMPLE</RequestId>
</ErrorResponse>` </ErrorResponse>`
const webIdentityToken = `WebIdentityToken`

View File

@ -5,6 +5,7 @@ import (
"fmt" "fmt"
"log" "log"
"net/http" "net/http"
"os"
"github.com/aws/aws-sdk-go/aws" "github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/endpoints" "github.com/aws/aws-sdk-go/aws/endpoints"
@ -15,16 +16,28 @@ import (
"github.com/hashicorp/go-cleanhttp" "github.com/hashicorp/go-cleanhttp"
) )
const (
// Maximum network retries.
// We depend on the AWS Go SDK DefaultRetryer exponential backoff.
// Ensure that if the AWS Config MaxRetries is set high (which it is by
// default), that we only retry for a few seconds with typically
// unrecoverable network errors, such as DNS lookup failures.
MaxNetworkRetryCount = 9
)
// GetSessionOptions attempts to return valid AWS Go SDK session authentication // GetSessionOptions attempts to return valid AWS Go SDK session authentication
// options based on pre-existing credential provider, configured profile, or // options based on pre-existing credential provider, configured profile, or
// fallback to automatically a determined session via the AWS Go SDK. // fallback to automatically a determined session via the AWS Go SDK.
func GetSessionOptions(c *Config) (*session.Options, error) { func GetSessionOptions(c *Config) (*session.Options, error) {
options := &session.Options{ options := &session.Options{
Config: aws.Config{ Config: aws.Config{
EndpointResolver: c.EndpointResolver(),
HTTPClient: cleanhttp.DefaultClient(), HTTPClient: cleanhttp.DefaultClient(),
MaxRetries: aws.Int(0), MaxRetries: aws.Int(0),
Region: aws.String(c.Region), Region: aws.String(c.Region),
}, },
Profile: c.Profile,
SharedConfigState: session.SharedConfigEnable,
} }
// get and validate credentials // get and validate credentials
@ -53,6 +66,10 @@ func GetSessionOptions(c *Config) (*session.Options, error) {
// GetSession attempts to return valid AWS Go SDK session. // GetSession attempts to return valid AWS Go SDK session.
func GetSession(c *Config) (*session.Session, error) { func GetSession(c *Config) (*session.Session, error) {
if c.SkipMetadataApiCheck {
os.Setenv("AWS_EC2_METADATA_DISABLED", "true")
}
options, err := GetSessionOptions(c) options, err := GetSessionOptions(c)
if err != nil { if err != nil {
@ -62,9 +79,9 @@ func GetSession(c *Config) (*session.Session, error) {
sess, err := session.NewSessionWithOptions(*options) sess, err := session.NewSessionWithOptions(*options)
if err != nil { if err != nil {
if IsAWSErr(err, "NoCredentialProviders", "") { if IsAWSErr(err, "NoCredentialProviders", "") {
return nil, ErrNoValidCredentialSources return nil, c.NewNoValidCredentialSourcesError(err)
} }
return nil, fmt.Errorf("Error creating AWS session: %s", err) return nil, fmt.Errorf("Error creating AWS session: %w", err)
} }
if c.MaxRetries > 0 { if c.MaxRetries > 0 {
@ -82,9 +99,7 @@ func GetSession(c *Config) (*session.Session, error) {
// NOTE: This logic can be fooled by other request errors raising the retry count // NOTE: This logic can be fooled by other request errors raising the retry count
// before any networking error occurs // before any networking error occurs
sess.Handlers.Retry.PushBack(func(r *request.Request) { sess.Handlers.Retry.PushBack(func(r *request.Request) {
// We currently depend on the DefaultRetryer exponential backoff here. if r.RetryCount < MaxNetworkRetryCount {
// ~10 retries gives a fair backoff of a few seconds.
if r.RetryCount < 9 {
return return
} }
// RequestError: send request failed // RequestError: send request failed
@ -102,9 +117,8 @@ func GetSession(c *Config) (*session.Session, error) {
}) })
if !c.SkipCredsValidation { if !c.SkipCredsValidation {
stsClient := sts.New(sess.Copy(&aws.Config{Endpoint: aws.String(c.StsEndpoint)})) if _, _, err := GetAccountIDAndPartitionFromSTSGetCallerIdentity(sts.New(sess)); err != nil {
if _, _, err := GetAccountIDAndPartitionFromSTSGetCallerIdentity(stsClient); err != nil { return nil, fmt.Errorf("error validating provider credentials: %w", err)
return nil, fmt.Errorf("error using credentials to get account ID: %s", err)
} }
} }
@ -125,14 +139,14 @@ func GetSessionWithAccountIDAndPartition(c *Config) (*session.Session, string, s
return sess, accountID, partition, nil return sess, accountID, partition, nil
} }
iamClient := iam.New(sess.Copy(&aws.Config{Endpoint: aws.String(c.IamEndpoint)})) iamClient := iam.New(sess)
stsClient := sts.New(sess.Copy(&aws.Config{Endpoint: aws.String(c.StsEndpoint)})) stsClient := sts.New(sess)
if !c.SkipCredsValidation { if !c.SkipCredsValidation {
accountID, partition, err := GetAccountIDAndPartitionFromSTSGetCallerIdentity(stsClient) accountID, partition, err := GetAccountIDAndPartitionFromSTSGetCallerIdentity(stsClient)
if err != nil { if err != nil {
return nil, "", "", fmt.Errorf("error validating provider credentials: %s", err) return nil, "", "", fmt.Errorf("error validating provider credentials: %w", err)
} }
return sess, accountID, partition, nil return sess, accountID, partition, nil
@ -154,7 +168,7 @@ func GetSessionWithAccountIDAndPartition(c *Config) (*session.Session, string, s
return nil, "", "", fmt.Errorf( return nil, "", "", fmt.Errorf(
"AWS account ID not previously found and failed retrieving via all available methods. "+ "AWS account ID not previously found and failed retrieving via all available methods. "+
"See https://www.terraform.io/docs/providers/aws/index.html#skip_requesting_account_id for workaround and implications. "+ "See https://www.terraform.io/docs/providers/aws/index.html#skip_requesting_account_id for workaround and implications. "+
"Errors: %s", err) "Errors: %w", err)
} }
var partition string var partition string

5
vendor/modules.txt vendored
View File

@ -112,7 +112,7 @@ github.com/armon/circbuf
# github.com/armon/go-radix v1.0.0 # github.com/armon/go-radix v1.0.0
## explicit ## explicit
github.com/armon/go-radix github.com/armon/go-radix
# github.com/aws/aws-sdk-go v1.30.12 # github.com/aws/aws-sdk-go v1.31.9
## explicit ## explicit
github.com/aws/aws-sdk-go/aws github.com/aws/aws-sdk-go/aws
github.com/aws/aws-sdk-go/aws/arn github.com/aws/aws-sdk-go/aws/arn
@ -144,6 +144,7 @@ github.com/aws/aws-sdk-go/internal/sdkuri
github.com/aws/aws-sdk-go/internal/shareddefaults github.com/aws/aws-sdk-go/internal/shareddefaults
github.com/aws/aws-sdk-go/internal/strings github.com/aws/aws-sdk-go/internal/strings
github.com/aws/aws-sdk-go/internal/sync/singleflight github.com/aws/aws-sdk-go/internal/sync/singleflight
github.com/aws/aws-sdk-go/private/checksum
github.com/aws/aws-sdk-go/private/protocol github.com/aws/aws-sdk-go/private/protocol
github.com/aws/aws-sdk-go/private/protocol/eventstream github.com/aws/aws-sdk-go/private/protocol/eventstream
github.com/aws/aws-sdk-go/private/protocol/eventstream/eventstreamapi github.com/aws/aws-sdk-go/private/protocol/eventstream/eventstreamapi
@ -293,7 +294,7 @@ github.com/gophercloud/utils/terraform/auth
## explicit ## explicit
# github.com/grpc-ecosystem/grpc-gateway v1.8.5 # github.com/grpc-ecosystem/grpc-gateway v1.8.5
## explicit ## explicit
# github.com/hashicorp/aws-sdk-go-base v0.4.0 # github.com/hashicorp/aws-sdk-go-base v0.5.0
## explicit ## explicit
github.com/hashicorp/aws-sdk-go-base github.com/hashicorp/aws-sdk-go-base
# github.com/hashicorp/consul v0.0.0-20171026175957-610f3c86a089 # github.com/hashicorp/consul v0.0.0-20171026175957-610f3c86a089

View File

@ -140,61 +140,66 @@ data.terraform_remote_state.network:
public_subnet_id = subnet-1e05dd33 public_subnet_id = subnet-1e05dd33
``` ```
## Configuration variables ## Configuration
The following configuration options or environment variables are supported: This backend requires the configuration of the AWS Region and S3 state storage. Other configuration, such as enabling DynamoDB state locking, is optional.
* `bucket` - (Required) The name of the S3 bucket. ### Credentials and Shared Configuration
* `key` - (Required) The path to the state file inside the bucket. When using
a non-default [workspace](/docs/state/workspaces.html), the state path will The following configuration is required:
be `/workspace_key_prefix/workspace_name/key`
* `region` / `AWS_DEFAULT_REGION` - (Optional) The region of the S3 * `region` - (Required) AWS Region of the S3 Bucket and DynamoDB Table (if used). This can also be sourced from the `AWS_DEFAULT_REGION` and `AWS_REGION` environment variables.
bucket.
* `endpoint` / `AWS_S3_ENDPOINT` - (Optional) A custom endpoint for the The following configuration is optional:
S3 API.
* `encrypt` - (Optional) Whether to enable [server side * `access_key` - (Optional) AWS access key. If configured, must also configure `secret_key`. This can also be sourced from the `AWS_ACCESS_KEY_ID` environment variable, AWS shared credentials file (e.g. `~/.aws/credentials`), or AWS shared configuration file (e.g. `~/.aws/config`).
encryption](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html) * `secret_key` - (Optional) AWS access key. If configured, must also configure `access_key`. This can also be sourced from the `AWS_SECRET_ACCESS_KEY` environment variable, AWS shared credentials file (e.g. `~/.aws/credentials`), or AWS shared configuration file (e.g. `~/.aws/config`).
of the state file. * `iam_endpoint` - (Optional) Custom endpoint for the AWS Identity and Access Management (IAM) API. This can also be sourced from the `AWS_IAM_ENDPOINT` environment variable.
* `acl` - [Canned
ACL](https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl)
to be applied to the state file.
* `access_key` / `AWS_ACCESS_KEY_ID` - (Optional) AWS access key.
* `secret_key` / `AWS_SECRET_ACCESS_KEY` - (Optional) AWS secret access key.
* `kms_key_id` - (Optional) The ARN of a KMS Key to use for encrypting
the state.
* `lock_table` - (Optional, Deprecated) Use `dynamodb_table` instead.
* `dynamodb_table` - (Optional) The name of a DynamoDB table to use for state
locking and consistency. The table must have a primary key named LockID(`LockID` must have type of `string`). If
not present, locking will be disabled.
* `profile` - (Optional) This is the AWS profile name as set in the
shared credentials file. It can also be sourced from the `AWS_PROFILE`
environment variable if `AWS_SDK_LOAD_CONFIG` is set to a truthy value,
e.g. `AWS_SDK_LOAD_CONFIG=1`.
* `shared_credentials_file` - (Optional) This is the path to the
shared credentials file. If this is not set and a profile is specified,
`~/.aws/credentials` will be used.
* `token` - (Optional) Use this to set an MFA token. It can also be
sourced from the `AWS_SESSION_TOKEN` environment variable.
* `role_arn` - (Optional) The role to be assumed.
* `assume_role_policy` - (Optional) The permissions applied when assuming a role.
* `external_id` - (Optional) The external ID to use when assuming the role.
* `session_name` - (Optional) The session name to use when assuming the role.
* `workspace_key_prefix` - (Optional) The prefix applied to the state path
inside the bucket. This is only relevant when using a non-default workspace.
This defaults to "env:"
* `dynamodb_endpoint` / `AWS_DYNAMODB_ENDPOINT` - (Optional) A custom endpoint for the DynamoDB API.
* `iam_endpoint` / `AWS_IAM_ENDPOINT` - (Optional) A custom endpoint for the IAM API.
* `sts_endpoint` / `AWS_STS_ENDPOINT` - (Optional) A custom endpoint for the STS API.
* `force_path_style` - (Optional) Always use path-style S3 URLs (`https://<HOST>/<BUCKET>` instead of `https://<BUCKET>.<HOST>`).
* `skip_credentials_validation` - (Optional) Skip the credentials validation via the STS API.
* `skip_get_ec2_platforms` - (**DEPRECATED**, Optional) Skip getting the supported EC2 platforms.
* `skip_region_validation` - (**DEPRECATED**, Optional) Skip validation of provided region name.
* `skip_requesting_account_id` - (**DEPRECATED**, Optional) Skip requesting the account ID.
* `skip_metadata_api_check` - (Optional) Skip the AWS Metadata API check.
* `sse_customer_key` / `AWS_SSE_CUSTOMER_KEY` - (Optional) The key to use for encrypting state with [Server-Side Encryption with Customer-Provided Keys (SSE-C)](https://docs.aws.amazon.com/AmazonS3/latest/dev/ServerSideEncryptionCustomerKeys.html).
This is the base64-encoded value of the key, which must decode to 256 bits. Due to the sensitivity of the value, it is recommended to set it using the `AWS_SSE_CUSTOMER_KEY` environment variable.
Setting it inside a terraform file will cause it to be persisted to disk in `terraform.tfstate`.
* `max_retries` - (Optional) The maximum number of times an AWS API request is retried on retryable failure. Defaults to 5. * `max_retries` - (Optional) The maximum number of times an AWS API request is retried on retryable failure. Defaults to 5.
* `profile` - (Optional) Name of AWS profile in AWS shared credentials file (e.g. `~/.aws/credentials`) or AWS shared configuration file (e.g. `~/.aws/config`) to use for credentials and/or configuration. This can also be sourced from the `AWS_PROFILE` environment variable.
* `shared_credentials_file` - (Optional) Path to the AWS shared credentials file. Defaults to `~/.aws/credentials`.
* `skip_credentials_validation` - (Optional) Skip credentials validation via the STS API.
* `skip_region_validation` - (Optional) Skip validation of provided region name.
* `skip_metadata_api_check` - (Optional) Skip usage of EC2 Metadata API.
* `sts_endpoint` - (Optional) Custom endpoint for the AWS Security Token Service (STS) API. This can also be sourced from the `AWS_STS_ENDPOINT` environment variable.
* `token` - (Optional) Multi-Factor Authentication (MFA) token. This can also be sourced from the `AWS_SESSION_TOKEN` environment variable.
#### Assume Role Configuration
The following configuration is optional:
* `assume_role_duration_seconds` - (Optional) Number of seconds to restrict the assume role session duration.
* `assume_role_policy` - (Optional) IAM Policy JSON describing further restricting permissions for the IAM Role being assumed.
* `assume_role_policy_arns` - (Optional) Set of Amazon Resource Names (ARNs) of IAM Policies describing further restricting permissions for the IAM Role being assumed.
* `assume_role_tags` - (Optional) Map of assume role session tags.
* `assume_role_transitive_tag_keys` - (Optional) Set of assume role session tag keys to pass to any subsequent sessions.
* `external_id` - (Optional) External identifier to use when assuming the role.
* `role_arn` - (Optional) Amazon Resource Name (ARN) of the IAM Role to assume.
* `session_name` - (Optional) Session name to use when assuming the role.
### S3 State Storage
The following configuration is required:
* `bucket` - (Required) Name of the S3 Bucket.
* `key` - (Required) Path to the state file inside the S3 Bucket. When using a non-default [workspace](/docs/state/workspaces.html), the state path will be `/workspace_key_prefix/workspace_name/key` (see also the `workspace_key_prefix` configuration).
The following configuration is optional:
* `acl` - (Optional) [Canned ACL](https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl) to be applied to the state file.
* `encrypt` - (Optional) Enable [server side encryption](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html) of the state file.
* `endpoint` - (Optional) Custom endpoint for the AWS S3 API. This can also be sourced from the `AWS_S3_ENDPOINT` environment variable.
* `force_path_style` - (Optional) Enable path-style S3 URLs (`https://<HOST>/<BUCKET>` instead of `https://<BUCKET>.<HOST>`).
* `kms_key_id` - (Optional) Amazon Resource Name (ARN) of a Key Management Service (KMS) Key to use for encrypting the state.
* `sse_customer_key` - (Optional) The key to use for encrypting state with [Server-Side Encryption with Customer-Provided Keys (SSE-C)](https://docs.aws.amazon.com/AmazonS3/latest/dev/ServerSideEncryptionCustomerKeys.html). This is the base64-encoded value of the key, which must decode to 256 bits. This can also be sourced from the `AWS_SSE_CUSTOMER_KEY` environment variable, which is recommended due to the sensitivity of the value. Setting it inside a terraform file will cause it to be persisted to disk in `terraform.tfstate`.
* `workspace_key_prefix` - (Optional) Prefix applied to the state path inside the bucket. This is only relevant when using a non-default workspace. Defaults to `env:`.
### DynamoDB State Locking
The following configuration is optional:
* `dynamodb_endpoint` - (Optional) Custom endpoint for the AWS DynamoDB API. This can also be sourced from the `AWS_DYNAMODB_ENDPOINT` environment variable.
* `dynamodb_table` - (Optional) Name of DynamoDB Table to use for state locking and consistency. The table must have a primary key named `LockID` with type of `string`. If not configured, state locking will be disabled.
## Multi-account AWS Architecture ## Multi-account AWS Architecture