From ba081aa10a8b8a4a32648ad9f67670a1425dec0b Mon Sep 17 00:00:00 2001 From: Brian Flad Date: Fri, 5 Jun 2020 16:41:32 -0400 Subject: [PATCH] backend/s3: Updates for Terraform v0.13.0 (#25134) * deps: Update github.com/hashicorp/aws-sdk-go-base@v0.5.0 Updated via: ``` $ go get github.com/hashicorp/aws-sdk-go-base@v0.5.0 $ go mod tidy $ go mod vendor ``` * backend/s3: Updates for Terraform v0.13.0 Reference: https://github.com/hashicorp/terraform/issues/13410 Reference: https://github.com/hashicorp/terraform/issues/18774 Reference: https://github.com/hashicorp/terraform/issues/19482 Reference: https://github.com/hashicorp/terraform/issues/20062 Reference: https://github.com/hashicorp/terraform/issues/20599 Reference: https://github.com/hashicorp/terraform/issues/22103 Reference: https://github.com/hashicorp/terraform/issues/22161 Reference: https://github.com/hashicorp/terraform/issues/22601 Reference: https://github.com/hashicorp/terraform/issues/22992 Reference: https://github.com/hashicorp/terraform/issues/24252 Reference: https://github.com/hashicorp/terraform/issues/24253 Reference: https://github.com/hashicorp/terraform/issues/24480 Reference: https://github.com/hashicorp/terraform/issues/25056 Changes: ``` NOTES * backend/s3: Deprecated `lock_table`, `skip_get_ec2_platforms`, `skip_requesting_account_id` arguments have been removed * backend/s3: Credential ordering has changed from static, environment, shared credentials, EC2 metadata, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata) to static, environment, shared credentials, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata) * The `AWS_METADATA_TIMEOUT` environment variable no longer has any effect as we now depend on the default AWS Go SDK EC2 Metadata client timeout of one second with two retries ENHANCEMENTS * backend/s3: Always enable shared configuration file support (no longer require `AWS_SDK_LOAD_CONFIG` environment variable) * backend/s3: Automatically expand `~` prefix for home directories in `shared_credentials_file` argument * backend/s3: Add `assume_role_duration_seconds`, `assume_role_policy_arns`, `assume_role_tags`, and `assume_role_transitive_tag_keys` arguments BUG FIXES * backend/s3: Ensure configured profile is used * backend/s3: Ensure configured STS endpoint is used during AssumeRole API calls * backend/s3: Prefer AWS shared configuration over EC2 metadata credentials * backend/s3: Prefer ECS credentials over EC2 metadata credentials * backend/s3: Remove hardcoded AWS Provider messaging ``` Output from acceptance testing: ``` --- PASS: TestBackend (16.32s) --- PASS: TestBackendConfig (0.58s) --- PASS: TestBackendConfig_AssumeRole (0.02s) --- PASS: TestBackendConfig_conflictingEncryptionSchema (0.00s) --- PASS: TestBackendConfig_invalidKey (0.00s) --- PASS: TestBackendConfig_invalidSSECustomerKeyEncoding (0.00s) --- PASS: TestBackendConfig_invalidSSECustomerKeyLength (0.00s) --- PASS: TestBackendExtraPaths (13.21s) --- PASS: TestBackendLocked (28.98s) --- PASS: TestBackendPrefixInWorkspace (5.65s) --- PASS: TestBackendSSECustomerKey (17.60s) --- PASS: TestBackend_impl (0.00s) --- PASS: TestForceUnlock (17.50s) --- PASS: TestKeyEnv (50.25s) --- PASS: TestRemoteClient (4.78s) --- PASS: TestRemoteClientLocks (16.85s) --- PASS: TestRemoteClient_clientMD5 (12.08s) --- PASS: TestRemoteClient_impl (0.00s) --- PASS: TestRemoteClient_stateChecksum (17.92s) ``` --- backend/remote-state/s3/backend.go | 137 +-- backend/remote-state/s3/backend_test.go | 290 +++++++ go.mod | 4 +- go.sum | 10 +- .../stscreds/assume_role_provider.go | 24 + .../stscreds/web_identity_provider.go | 2 + .../aws/aws-sdk-go/aws/endpoints/decode.go | 2 +- .../aws/aws-sdk-go/aws/endpoints/defaults.go | 714 ++++++++++++++-- vendor/github.com/aws/aws-sdk-go/aws/types.go | 23 + .../github.com/aws/aws-sdk-go/aws/version.go | 2 +- .../private/checksum/content_md5.go | 53 ++ .../aws/aws-sdk-go/service/dynamodb/api.go | 40 +- .../aws/aws-sdk-go/service/s3/api.go | 809 +++++++++--------- .../aws/aws-sdk-go/service/s3/body_hash.go | 49 +- .../aws-sdk-go/service/s3/customizations.go | 6 - .../aws-sdk-go/service/s3/statusok_error.go | 16 +- .../aws-sdk-go/service/s3/unmarshal_error.go | 40 +- .../aws/aws-sdk-go/service/sts/api.go | 4 +- .../hashicorp/aws-sdk-go-base/.golangci.yml | 7 + .../hashicorp/aws-sdk-go-base/.travis.yml | 20 - .../hashicorp/aws-sdk-go-base/CHANGELOG.md | 22 + .../hashicorp/aws-sdk-go-base/awsauth.go | 210 ++--- .../hashicorp/aws-sdk-go-base/awserr.go | 25 +- .../hashicorp/aws-sdk-go-base/config.go | 44 +- .../hashicorp/aws-sdk-go-base/endpoints.go | 46 + .../hashicorp/aws-sdk-go-base/errors.go | 76 ++ .../hashicorp/aws-sdk-go-base/go.mod | 6 +- .../hashicorp/aws-sdk-go-base/go.sum | 28 +- .../hashicorp/aws-sdk-go-base/mock.go | 88 +- .../hashicorp/aws-sdk-go-base/session.go | 44 +- vendor/modules.txt | 5 +- website/docs/backends/types/s3.html.md | 111 +-- 32 files changed, 2113 insertions(+), 844 deletions(-) create mode 100644 vendor/github.com/aws/aws-sdk-go/private/checksum/content_md5.go delete mode 100644 vendor/github.com/hashicorp/aws-sdk-go-base/.travis.yml create mode 100644 vendor/github.com/hashicorp/aws-sdk-go-base/endpoints.go create mode 100644 vendor/github.com/hashicorp/aws-sdk-go-base/errors.go diff --git a/backend/remote-state/s3/backend.go b/backend/remote-state/s3/backend.go index 95dd2f6ca..1ccee3207 100644 --- a/backend/remote-state/s3/backend.go +++ b/backend/remote-state/s3/backend.go @@ -44,7 +44,7 @@ func New() backend.Backend { "region": { Type: schema.TypeString, Required: true, - Description: "The region of the S3 bucket.", + Description: "AWS region of the S3 Bucket and DynamoDB Table (if used).", DefaultFunc: schema.MultiEnvDefaultFunc([]string{ "AWS_REGION", "AWS_DEFAULT_REGION", @@ -114,14 +114,6 @@ func New() backend.Backend { Default: "", }, - "lock_table": { - Type: schema.TypeString, - Optional: true, - Description: "DynamoDB table for state locking", - Default: "", - Deprecated: "please use the dynamodb_table attribute", - }, - "dynamodb_table": { Type: schema.TypeString, Optional: true, @@ -157,14 +149,6 @@ func New() backend.Backend { Default: false, }, - "skip_get_ec2_platforms": { - Type: schema.TypeBool, - Optional: true, - Description: "Skip getting the supported EC2 platforms.", - Default: false, - Deprecated: "The S3 Backend does not require EC2 functionality and this attribute is no longer used.", - }, - "skip_region_validation": { Type: schema.TypeBool, Optional: true, @@ -172,14 +156,6 @@ func New() backend.Backend { Default: false, }, - "skip_requesting_account_id": { - Type: schema.TypeBool, - Optional: true, - Description: "Skip requesting the account ID.", - Default: false, - Deprecated: "The S3 Backend no longer automatically looks up the AWS Account ID and this attribute is no longer used.", - }, - "skip_metadata_api_check": { Type: schema.TypeBool, Optional: true, @@ -223,13 +199,40 @@ func New() backend.Backend { Default: "", }, + "assume_role_duration_seconds": { + Type: schema.TypeInt, + Optional: true, + Description: "Seconds to restrict the assume role session duration.", + }, + "assume_role_policy": { Type: schema.TypeString, Optional: true, - Description: "The permissions applied when assuming a role.", + Description: "IAM Policy JSON describing further restricting permissions for the IAM Role being assumed.", Default: "", }, + "assume_role_policy_arns": { + Type: schema.TypeSet, + Optional: true, + Description: "Amazon Resource Names (ARNs) of IAM Policies describing further restricting permissions for the IAM Role being assumed.", + Elem: &schema.Schema{Type: schema.TypeString}, + }, + + "assume_role_tags": { + Type: schema.TypeMap, + Optional: true, + Description: "Assume role session tags.", + Elem: &schema.Schema{Type: schema.TypeString}, + }, + + "assume_role_transitive_tag_keys": { + Type: schema.TypeSet, + Optional: true, + Description: "Assume role session tag keys to pass to any subsequent sessions.", + Elem: &schema.Schema{Type: schema.TypeString}, + }, + "workspace_key_prefix": { Type: schema.TypeString, Optional: true, @@ -302,6 +305,7 @@ func (b *Backend) configure(ctx context.Context) error { b.workspaceKeyPrefix = data.Get("workspace_key_prefix").(string) b.serverSideEncryption = data.Get("encrypt").(bool) b.kmsKeyID = data.Get("kms_key_id").(string) + b.ddbTable = data.Get("dynamodb_table").(string) customerKeyString := data.Get("sse_customer_key").(string) if customerKeyString != "" { @@ -316,29 +320,26 @@ func (b *Backend) configure(ctx context.Context) error { } } - b.ddbTable = data.Get("dynamodb_table").(string) - if b.ddbTable == "" { - // try the deprecated field - b.ddbTable = data.Get("lock_table").(string) - } - cfg := &awsbase.Config{ - AccessKey: data.Get("access_key").(string), - AssumeRoleARN: data.Get("role_arn").(string), - AssumeRoleExternalID: data.Get("external_id").(string), - AssumeRolePolicy: data.Get("assume_role_policy").(string), - AssumeRoleSessionName: data.Get("session_name").(string), - CredsFilename: data.Get("shared_credentials_file").(string), - DebugLogging: logging.IsDebugOrHigher(), - IamEndpoint: data.Get("iam_endpoint").(string), - MaxRetries: data.Get("max_retries").(int), - Profile: data.Get("profile").(string), - Region: data.Get("region").(string), - SecretKey: data.Get("secret_key").(string), - SkipCredsValidation: data.Get("skip_credentials_validation").(bool), - SkipMetadataApiCheck: data.Get("skip_metadata_api_check").(bool), - StsEndpoint: data.Get("sts_endpoint").(string), - Token: data.Get("token").(string), + AccessKey: data.Get("access_key").(string), + AssumeRoleARN: data.Get("role_arn").(string), + AssumeRoleDurationSeconds: data.Get("assume_role_duration_seconds").(int), + AssumeRoleExternalID: data.Get("external_id").(string), + AssumeRolePolicy: data.Get("assume_role_policy").(string), + AssumeRoleSessionName: data.Get("session_name").(string), + CallerDocumentationURL: "https://www.terraform.io/docs/backends/types/s3.html", + CallerName: "S3 Backend", + CredsFilename: data.Get("shared_credentials_file").(string), + DebugLogging: logging.IsDebugOrHigher(), + IamEndpoint: data.Get("iam_endpoint").(string), + MaxRetries: data.Get("max_retries").(int), + Profile: data.Get("profile").(string), + Region: data.Get("region").(string), + SecretKey: data.Get("secret_key").(string), + SkipCredsValidation: data.Get("skip_credentials_validation").(bool), + SkipMetadataApiCheck: data.Get("skip_metadata_api_check").(bool), + StsEndpoint: data.Get("sts_endpoint").(string), + Token: data.Get("token").(string), UserAgentProducts: []*awsbase.UserAgentProduct{ {Name: "APN", Version: "1.0"}, {Name: "HashiCorp", Version: "1.0"}, @@ -346,9 +347,47 @@ func (b *Backend) configure(ctx context.Context) error { }, } + if policyARNSet := data.Get("assume_role_policy_arns").(*schema.Set); policyARNSet.Len() > 0 { + for _, policyARNRaw := range policyARNSet.List() { + policyARN, ok := policyARNRaw.(string) + + if !ok { + continue + } + + cfg.AssumeRolePolicyARNs = append(cfg.AssumeRolePolicyARNs, policyARN) + } + } + + if tagMap := data.Get("assume_role_tags").(map[string]interface{}); len(tagMap) > 0 { + cfg.AssumeRoleTags = make(map[string]string) + + for k, vRaw := range tagMap { + v, ok := vRaw.(string) + + if !ok { + continue + } + + cfg.AssumeRoleTags[k] = v + } + } + + if transitiveTagKeySet := data.Get("assume_role_transitive_tag_keys").(*schema.Set); transitiveTagKeySet.Len() > 0 { + for _, transitiveTagKeyRaw := range transitiveTagKeySet.List() { + transitiveTagKey, ok := transitiveTagKeyRaw.(string) + + if !ok { + continue + } + + cfg.AssumeRoleTransitiveTagKeys = append(cfg.AssumeRoleTransitiveTagKeys, transitiveTagKey) + } + } + sess, err := awsbase.GetSession(cfg) if err != nil { - return err + return fmt.Errorf("error configuring S3 Backend: %w", err) } b.dynClient = dynamodb.New(sess.Copy(&aws.Config{ diff --git a/backend/remote-state/s3/backend_test.go b/backend/remote-state/s3/backend_test.go index 903cf237f..82a282652 100644 --- a/backend/remote-state/s3/backend_test.go +++ b/backend/remote-state/s3/backend_test.go @@ -2,6 +2,7 @@ package s3 import ( "fmt" + "net/url" "os" "reflect" "testing" @@ -10,12 +11,64 @@ import ( "github.com/aws/aws-sdk-go/aws" "github.com/aws/aws-sdk-go/service/dynamodb" "github.com/aws/aws-sdk-go/service/s3" + awsbase "github.com/hashicorp/aws-sdk-go-base" "github.com/hashicorp/terraform/backend" "github.com/hashicorp/terraform/configs/hcl2shim" "github.com/hashicorp/terraform/state/remote" "github.com/hashicorp/terraform/states" ) +const ( + mockStsAssumeRoleArn = `arn:aws:iam::555555555555:role/AssumeRole` + mockStsAssumeRolePolicy = `{ + "Version": "2012-10-17", + "Statement": { + "Effect": "Allow", + "Action": "*", + "Resource": "*", + } +}` + mockStsAssumeRolePolicyArn = `arn:aws:iam::555555555555:policy/AssumeRolePolicy1` + mockStsAssumeRoleSessionName = `AssumeRoleSessionName` + mockStsAssumeRoleTagKey = `AssumeRoleTagKey` + mockStsAssumeRoleTagValue = `AssumeRoleTagValue` + mockStsAssumeRoleTransitiveTagKey = `AssumeRoleTagKey` + mockStsAssumeRoleValidResponse = ` + + + arn:aws:sts::555555555555:assumed-role/role/AssumeRoleSessionName + ARO123EXAMPLE123:AssumeRoleSessionName + + + AssumeRoleAccessKey + AssumeRoleSecretKey + AssumeRoleSessionToken + 2099-12-31T23:59:59Z + + + + 01234567-89ab-cdef-0123-456789abcdef + +` + mockStsGetCallerIdentityValidResponseBody = ` + + arn:aws:iam::222222222222:user/Alice + AKIAI44QH8DHBEXAMPLE + 222222222222 + + + 01234567-89ab-cdef-0123-456789abcdef + +` +) + +var ( + mockStsGetCallerIdentityRequestBody = url.Values{ + "Action": []string{"GetCallerIdentity"}, + "Version": []string{"2011-06-15"}, + }.Encode() +) + // verify that we are doing ACC tests or the S3 tests specifically func testACC(t *testing.T) { skip := os.Getenv("TF_ACC") == "" && os.Getenv("TF_S3_TEST") == "" @@ -66,6 +119,243 @@ func TestBackendConfig(t *testing.T) { } } +func TestBackendConfig_AssumeRole(t *testing.T) { + testACC(t) + + testCases := []struct { + Config map[string]interface{} + Description string + MockStsEndpoints []*awsbase.MockEndpoint + }{ + { + Config: map[string]interface{}{ + "bucket": "tf-test", + "key": "state", + "region": "us-west-1", + "role_arn": mockStsAssumeRoleArn, + "session_name": mockStsAssumeRoleSessionName, + }, + Description: "role_arn", + MockStsEndpoints: []*awsbase.MockEndpoint{ + { + Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: url.Values{ + "Action": []string{"AssumeRole"}, + "DurationSeconds": []string{"900"}, + "RoleArn": []string{mockStsAssumeRoleArn}, + "RoleSessionName": []string{mockStsAssumeRoleSessionName}, + "Version": []string{"2011-06-15"}, + }.Encode()}, + Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"}, + }, + { + Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody}, + Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"}, + }, + }, + }, + { + Config: map[string]interface{}{ + "assume_role_duration_seconds": 3600, + "bucket": "tf-test", + "key": "state", + "region": "us-west-1", + "role_arn": mockStsAssumeRoleArn, + "session_name": mockStsAssumeRoleSessionName, + }, + Description: "assume_role_duration_seconds", + MockStsEndpoints: []*awsbase.MockEndpoint{ + { + Request: &awsbase.MockRequest{"POST", "/", url.Values{ + "Action": []string{"AssumeRole"}, + "DurationSeconds": []string{"3600"}, + "RoleArn": []string{mockStsAssumeRoleArn}, + "RoleSessionName": []string{mockStsAssumeRoleSessionName}, + "Version": []string{"2011-06-15"}, + }.Encode()}, + Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"}, + }, + { + Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody}, + Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"}, + }, + }, + }, + { + Config: map[string]interface{}{ + "bucket": "tf-test", + "external_id": "AssumeRoleExternalId", + "key": "state", + "region": "us-west-1", + "role_arn": mockStsAssumeRoleArn, + "session_name": mockStsAssumeRoleSessionName, + }, + Description: "external_id", + MockStsEndpoints: []*awsbase.MockEndpoint{ + { + Request: &awsbase.MockRequest{"POST", "/", url.Values{ + "Action": []string{"AssumeRole"}, + "DurationSeconds": []string{"900"}, + "ExternalId": []string{"AssumeRoleExternalId"}, + "RoleArn": []string{mockStsAssumeRoleArn}, + "RoleSessionName": []string{mockStsAssumeRoleSessionName}, + "Version": []string{"2011-06-15"}, + }.Encode()}, + Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"}, + }, + { + Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody}, + Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"}, + }, + }, + }, + { + Config: map[string]interface{}{ + "assume_role_policy": mockStsAssumeRolePolicy, + "bucket": "tf-test", + "key": "state", + "region": "us-west-1", + "role_arn": mockStsAssumeRoleArn, + "session_name": mockStsAssumeRoleSessionName, + }, + Description: "assume_role_policy", + MockStsEndpoints: []*awsbase.MockEndpoint{ + { + Request: &awsbase.MockRequest{"POST", "/", url.Values{ + "Action": []string{"AssumeRole"}, + "DurationSeconds": []string{"900"}, + "Policy": []string{mockStsAssumeRolePolicy}, + "RoleArn": []string{mockStsAssumeRoleArn}, + "RoleSessionName": []string{mockStsAssumeRoleSessionName}, + "Version": []string{"2011-06-15"}, + }.Encode()}, + Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"}, + }, + { + Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody}, + Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"}, + }, + }, + }, + { + Config: map[string]interface{}{ + "assume_role_policy_arns": []interface{}{mockStsAssumeRolePolicyArn}, + "bucket": "tf-test", + "key": "state", + "region": "us-west-1", + "role_arn": mockStsAssumeRoleArn, + "session_name": mockStsAssumeRoleSessionName, + }, + Description: "assume_role_policy_arns", + MockStsEndpoints: []*awsbase.MockEndpoint{ + { + Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: url.Values{ + "Action": []string{"AssumeRole"}, + "DurationSeconds": []string{"900"}, + "PolicyArns.member.1.arn": []string{mockStsAssumeRolePolicyArn}, + "RoleArn": []string{mockStsAssumeRoleArn}, + "RoleSessionName": []string{mockStsAssumeRoleSessionName}, + "Version": []string{"2011-06-15"}, + }.Encode()}, + Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"}, + }, + { + Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody}, + Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"}, + }, + }, + }, + { + Config: map[string]interface{}{ + "assume_role_tags": map[string]interface{}{ + mockStsAssumeRoleTagKey: mockStsAssumeRoleTagValue, + }, + "bucket": "tf-test", + "key": "state", + "region": "us-west-1", + "role_arn": mockStsAssumeRoleArn, + "session_name": mockStsAssumeRoleSessionName, + }, + Description: "assume_role_tags", + MockStsEndpoints: []*awsbase.MockEndpoint{ + { + Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: url.Values{ + "Action": []string{"AssumeRole"}, + "DurationSeconds": []string{"900"}, + "RoleArn": []string{mockStsAssumeRoleArn}, + "RoleSessionName": []string{mockStsAssumeRoleSessionName}, + "Tags.member.1.Key": []string{mockStsAssumeRoleTagKey}, + "Tags.member.1.Value": []string{mockStsAssumeRoleTagValue}, + "Version": []string{"2011-06-15"}, + }.Encode()}, + Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"}, + }, + { + Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody}, + Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"}, + }, + }, + }, + { + Config: map[string]interface{}{ + "assume_role_tags": map[string]interface{}{ + mockStsAssumeRoleTagKey: mockStsAssumeRoleTagValue, + }, + "assume_role_transitive_tag_keys": []interface{}{mockStsAssumeRoleTagKey}, + "bucket": "tf-test", + "key": "state", + "region": "us-west-1", + "role_arn": mockStsAssumeRoleArn, + "session_name": mockStsAssumeRoleSessionName, + }, + Description: "assume_role_transitive_tag_keys", + MockStsEndpoints: []*awsbase.MockEndpoint{ + { + Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: url.Values{ + "Action": []string{"AssumeRole"}, + "DurationSeconds": []string{"900"}, + "RoleArn": []string{mockStsAssumeRoleArn}, + "RoleSessionName": []string{mockStsAssumeRoleSessionName}, + "Tags.member.1.Key": []string{mockStsAssumeRoleTagKey}, + "Tags.member.1.Value": []string{mockStsAssumeRoleTagValue}, + "TransitiveTagKeys.member.1": []string{mockStsAssumeRoleTagKey}, + "Version": []string{"2011-06-15"}, + }.Encode()}, + Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"}, + }, + { + Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody}, + Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"}, + }, + }, + }, + } + + for _, testCase := range testCases { + testCase := testCase + + t.Run(testCase.Description, func(t *testing.T) { + closeSts, mockStsSession, err := awsbase.GetMockedAwsApiSession("STS", testCase.MockStsEndpoints) + defer closeSts() + + if err != nil { + t.Fatalf("unexpected error creating mock STS server: %s", err) + } + + if mockStsSession != nil && mockStsSession.Config != nil { + testCase.Config["sts_endpoint"] = aws.StringValue(mockStsSession.Config.Endpoint) + } + + diags := New().Configure(hcl2shim.HCL2ValueFromConfigValue(testCase.Config)) + + if diags.HasErrors() { + for _, diag := range diags { + t.Errorf("unexpected error: %s", diag.Description().Summary) + } + } + }) + } +} + func TestBackendConfig_invalidKey(t *testing.T) { testACC(t) cfg := hcl2shim.HCL2ValueFromConfigValue(map[string]interface{}{ diff --git a/go.mod b/go.mod index 858d52314..9ce9e8c49 100644 --- a/go.mod +++ b/go.mod @@ -17,7 +17,7 @@ require ( github.com/armon/circbuf v0.0.0-20190214190532-5111143e8da2 github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da // indirect github.com/armon/go-radix v1.0.0 // indirect - github.com/aws/aws-sdk-go v1.30.12 + github.com/aws/aws-sdk-go v1.31.9 github.com/baiyubin/aliyun-sts-go-sdk v0.0.0-20180326062324-cfa1a18b161f // indirect github.com/blang/semver v3.5.1+incompatible github.com/bmatcuk/doublestar v1.1.5 @@ -48,7 +48,7 @@ require ( github.com/grpc-ecosystem/go-grpc-middleware v1.0.0 // indirect github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 // indirect github.com/grpc-ecosystem/grpc-gateway v1.8.5 // indirect - github.com/hashicorp/aws-sdk-go-base v0.4.0 + github.com/hashicorp/aws-sdk-go-base v0.5.0 github.com/hashicorp/consul v0.0.0-20171026175957-610f3c86a089 github.com/hashicorp/errwrap v1.0.0 github.com/hashicorp/go-azure-helpers v0.10.0 diff --git a/go.sum b/go.sum index 4847f957e..4a0b6894b 100644 --- a/go.sum +++ b/go.sum @@ -86,9 +86,8 @@ github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj github.com/armon/go-radix v1.0.0 h1:F4z6KzEeeQIMeLFa97iZU6vupzoecKdU5TX24SNppXI= github.com/armon/go-radix v1.0.0/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8= github.com/aws/aws-sdk-go v1.15.78/go.mod h1:E3/ieXAlvM0XWO57iftYVDLLvQ824smPP3ATZkfNZeM= -github.com/aws/aws-sdk-go v1.25.3/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo= -github.com/aws/aws-sdk-go v1.30.12 h1:KrjyosZvkpJjcwMk0RNxMZewQ47v7+ZkbQDXjWsJMs8= -github.com/aws/aws-sdk-go v1.30.12/go.mod h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZveU8YkpAk0= +github.com/aws/aws-sdk-go v1.31.9 h1:n+b34ydVfgC30j0Qm69yaapmjejQPW2BoDBX7Uy/tLI= +github.com/aws/aws-sdk-go v1.31.9/go.mod h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZveU8YkpAk0= github.com/baiyubin/aliyun-sts-go-sdk v0.0.0-20180326062324-cfa1a18b161f h1:ZNv7On9kyUzm7fvRZumSyy/IUiSC7AzL0I1jKKtwooA= github.com/baiyubin/aliyun-sts-go-sdk v0.0.0-20180326062324-cfa1a18b161f/go.mod h1:AuiFmCCPBSrqvVMvuqFuk0qogytodnVFVSN5CeJB8Gc= github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973 h1:xJ4a3vCFaGF/jqvzLMYoU8P317H5OQ+Via4RmuPwCS0= @@ -199,8 +198,8 @@ github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 h1:Ovs26xHkKqVztRpIrF/92Bcuy github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk= github.com/grpc-ecosystem/grpc-gateway v1.8.5 h1:2+KSC78XiO6Qy0hIjfc1OD9H+hsaJdJlb8Kqsd41CTE= github.com/grpc-ecosystem/grpc-gateway v1.8.5/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY= -github.com/hashicorp/aws-sdk-go-base v0.4.0 h1:zH9hNUdsS+2G0zJaU85ul8D59BGnZBaKM+KMNPAHGwk= -github.com/hashicorp/aws-sdk-go-base v0.4.0/go.mod h1:eRhlz3c4nhqxFZJAahJEFL7gh6Jyj5rQmQc7F9eHFyQ= +github.com/hashicorp/aws-sdk-go-base v0.5.0 h1:fk7ID0v3PWL/KNL8FvkBPu8Sm93EPUCCmtZCiTXLySE= +github.com/hashicorp/aws-sdk-go-base v0.5.0/go.mod h1:2fRjWDv3jJBeN6mVWFHV6hFTNeFBx2gpDLQaZNxUVAY= github.com/hashicorp/consul v0.0.0-20171026175957-610f3c86a089 h1:1eDpXAxTh0iPv+1kc9/gfSI2pxRERDsTk/lNGolwHn8= github.com/hashicorp/consul v0.0.0-20171026175957-610f3c86a089/go.mod h1:mFrjN1mfidgJfYP1xrJCF+AfRhr6Eaqhb2+sfyn/OOI= github.com/hashicorp/errwrap v1.0.0 h1:hLrqtEDnRye3+sgx6z4qVLNuviH3MR5aQ0ykNJa/UYA= @@ -269,7 +268,6 @@ github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb/go.mod h1:+NfK9FKe github.com/jhump/protoreflect v1.6.0 h1:h5jfMVslIg6l29nsMs0D8Wj17RDVdNYti0vDN/PZZoE= github.com/jhump/protoreflect v1.6.0/go.mod h1:eaTn3RZAmMBcV0fifFvlm6VHNz3wSkYyXYWUh7ymB74= github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k= -github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k= github.com/jmespath/go-jmespath v0.3.0 h1:OS12ieG61fsCg5+qLJ+SsW9NicxNkg3b25OyT2yCeUc= github.com/jmespath/go-jmespath v0.3.0/go.mod h1:9QtRXoHjLGCJ5IBSaohpXITPlowMeeYCZ7fLUTSywik= github.com/jonboulle/clockwork v0.1.0 h1:VKV+ZcuP6l3yW9doeqz6ziZGgcynBVQO+obU0+0hcPo= diff --git a/vendor/github.com/aws/aws-sdk-go/aws/credentials/stscreds/assume_role_provider.go b/vendor/github.com/aws/aws-sdk-go/aws/credentials/stscreds/assume_role_provider.go index 73d9763c9..6846ef6f8 100644 --- a/vendor/github.com/aws/aws-sdk-go/aws/credentials/stscreds/assume_role_provider.go +++ b/vendor/github.com/aws/aws-sdk-go/aws/credentials/stscreds/assume_role_provider.go @@ -169,6 +169,29 @@ type AssumeRoleProvider struct { // size. Policy *string + // The ARNs of IAM managed policies you want to use as managed session policies. + // The policies must exist in the same account as the role. + // + // This parameter is optional. You can provide up to 10 managed policy ARNs. + // However, the plain text that you use for both inline and managed session + // policies can't exceed 2,048 characters. + // + // An AWS conversion compresses the passed session policies and session tags + // into a packed binary format that has a separate limit. Your request can fail + // for this limit even if your plain text meets the other requirements. The + // PackedPolicySize response element indicates by percentage how close the policies + // and tags for your request are to the upper size limit. + // + // Passing policies to this operation returns new temporary credentials. The + // resulting session's permissions are the intersection of the role's identity-based + // policy and the session policies. You can use the role's temporary credentials + // in subsequent AWS API calls to access resources in the account that owns + // the role. You cannot use session policies to grant more permissions than + // those allowed by the identity-based policy of the role that is being assumed. + // For more information, see Session Policies (https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html#policies_session) + // in the IAM User Guide. + PolicyArns []*sts.PolicyDescriptorType + // The identification number of the MFA device that is associated with the user // who is making the AssumeRole call. Specify this value if the trust policy // of the role being assumed includes a condition that requires MFA authentication. @@ -291,6 +314,7 @@ func (p *AssumeRoleProvider) RetrieveWithContext(ctx credentials.Context) (crede RoleSessionName: aws.String(p.RoleSessionName), ExternalId: p.ExternalID, Tags: p.Tags, + PolicyArns: p.PolicyArns, TransitiveTagKeys: p.TransitiveTagKeys, } if p.Policy != nil { diff --git a/vendor/github.com/aws/aws-sdk-go/aws/credentials/stscreds/web_identity_provider.go b/vendor/github.com/aws/aws-sdk-go/aws/credentials/stscreds/web_identity_provider.go index fda495194..6feb262b2 100644 --- a/vendor/github.com/aws/aws-sdk-go/aws/credentials/stscreds/web_identity_provider.go +++ b/vendor/github.com/aws/aws-sdk-go/aws/credentials/stscreds/web_identity_provider.go @@ -50,6 +50,7 @@ func (f FetchTokenPath) FetchToken(ctx credentials.Context) ([]byte, error) { // an OIDC token. type WebIdentityRoleProvider struct { credentials.Expiry + PolicyArns []*sts.PolicyDescriptorType client stsiface.STSAPI ExpiryWindow time.Duration @@ -107,6 +108,7 @@ func (p *WebIdentityRoleProvider) RetrieveWithContext(ctx credentials.Context) ( sessionName = strconv.FormatInt(now().UnixNano(), 10) } req, resp := p.client.AssumeRoleWithWebIdentityRequest(&sts.AssumeRoleWithWebIdentityInput{ + PolicyArns: p.PolicyArns, RoleArn: &p.roleARN, RoleSessionName: &sessionName, WebIdentityToken: aws.String(string(b)), diff --git a/vendor/github.com/aws/aws-sdk-go/aws/endpoints/decode.go b/vendor/github.com/aws/aws-sdk-go/aws/endpoints/decode.go index 343a2106f..654fb1ad5 100644 --- a/vendor/github.com/aws/aws-sdk-go/aws/endpoints/decode.go +++ b/vendor/github.com/aws/aws-sdk-go/aws/endpoints/decode.go @@ -93,7 +93,7 @@ func decodeV3Endpoints(modelDef modelDefinition, opts DecodeModelOptions) (Resol } func custAddS3DualStack(p *partition) { - if p.ID != "aws" { + if !(p.ID == "aws" || p.ID == "aws-cn" || p.ID == "aws-us-gov") { return } diff --git a/vendor/github.com/aws/aws-sdk-go/aws/endpoints/defaults.go b/vendor/github.com/aws/aws-sdk-go/aws/endpoints/defaults.go index 4173d38bc..1ececcff6 100644 --- a/vendor/github.com/aws/aws-sdk-go/aws/endpoints/defaults.go +++ b/vendor/github.com/aws/aws-sdk-go/aws/endpoints/defaults.go @@ -25,11 +25,12 @@ const ( ApSoutheast1RegionID = "ap-southeast-1" // Asia Pacific (Singapore). ApSoutheast2RegionID = "ap-southeast-2" // Asia Pacific (Sydney). CaCentral1RegionID = "ca-central-1" // Canada (Central). - EuCentral1RegionID = "eu-central-1" // EU (Frankfurt). - EuNorth1RegionID = "eu-north-1" // EU (Stockholm). - EuWest1RegionID = "eu-west-1" // EU (Ireland). - EuWest2RegionID = "eu-west-2" // EU (London). - EuWest3RegionID = "eu-west-3" // EU (Paris). + EuCentral1RegionID = "eu-central-1" // Europe (Frankfurt). + EuNorth1RegionID = "eu-north-1" // Europe (Stockholm). + EuSouth1RegionID = "eu-south-1" // Europe (Milan). + EuWest1RegionID = "eu-west-1" // Europe (Ireland). + EuWest2RegionID = "eu-west-2" // Europe (London). + EuWest3RegionID = "eu-west-3" // Europe (Paris). MeSouth1RegionID = "me-south-1" // Middle East (Bahrain). SaEast1RegionID = "sa-east-1" // South America (Sao Paulo). UsEast1RegionID = "us-east-1" // US East (N. Virginia). @@ -47,7 +48,7 @@ const ( // AWS GovCloud (US) partition's regions. const ( UsGovEast1RegionID = "us-gov-east-1" // AWS GovCloud (US-East). - UsGovWest1RegionID = "us-gov-west-1" // AWS GovCloud (US). + UsGovWest1RegionID = "us-gov-west-1" // AWS GovCloud (US-West). ) // AWS ISO (US) partition's regions. @@ -133,19 +134,22 @@ var awsPartition = partition{ Description: "Canada (Central)", }, "eu-central-1": region{ - Description: "EU (Frankfurt)", + Description: "Europe (Frankfurt)", }, "eu-north-1": region{ - Description: "EU (Stockholm)", + Description: "Europe (Stockholm)", + }, + "eu-south-1": region{ + Description: "Europe (Milan)", }, "eu-west-1": region{ - Description: "EU (Ireland)", + Description: "Europe (Ireland)", }, "eu-west-2": region{ - Description: "EU (London)", + Description: "Europe (London)", }, "eu-west-3": region{ - Description: "EU (Paris)", + Description: "Europe (Paris)", }, "me-south-1": region{ Description: "Middle East (Bahrain)", @@ -186,6 +190,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -216,6 +221,7 @@ var awsPartition = partition{ }, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -306,6 +312,29 @@ var awsPartition = partition{ "us-west-2": endpoint{}, }, }, + "api.detective": service{ + Defaults: endpoint{ + Protocols: []string{"https"}, + }, + Endpoints: endpoints{ + "ap-northeast-1": endpoint{}, + "ap-northeast-2": endpoint{}, + "ap-south-1": endpoint{}, + "ap-southeast-1": endpoint{}, + "ap-southeast-2": endpoint{}, + "ca-central-1": endpoint{}, + "eu-central-1": endpoint{}, + "eu-north-1": endpoint{}, + "eu-west-1": endpoint{}, + "eu-west-2": endpoint{}, + "eu-west-3": endpoint{}, + "sa-east-1": endpoint{}, + "us-east-1": endpoint{}, + "us-east-2": endpoint{}, + "us-west-1": endpoint{}, + "us-west-2": endpoint{}, + }, + }, "api.ecr": service{ Endpoints: endpoints{ @@ -369,6 +398,12 @@ var awsPartition = partition{ Region: "eu-north-1", }, }, + "eu-south-1": endpoint{ + Hostname: "api.ecr.eu-south-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "eu-south-1", + }, + }, "eu-west-1": endpoint{ Hostname: "api.ecr.eu-west-1.amazonaws.com", CredentialScope: credentialScope{ @@ -449,6 +484,29 @@ var awsPartition = partition{ }, }, }, + "api.elastic-inference": service{ + + Endpoints: endpoints{ + "ap-northeast-1": endpoint{ + Hostname: "api.elastic-inference.ap-northeast-1.amazonaws.com", + }, + "ap-northeast-2": endpoint{ + Hostname: "api.elastic-inference.ap-northeast-2.amazonaws.com", + }, + "eu-west-1": endpoint{ + Hostname: "api.elastic-inference.eu-west-1.amazonaws.com", + }, + "us-east-1": endpoint{ + Hostname: "api.elastic-inference.us-east-1.amazonaws.com", + }, + "us-east-2": endpoint{ + Hostname: "api.elastic-inference.us-east-2.amazonaws.com", + }, + "us-west-2": endpoint{ + Hostname: "api.elastic-inference.us-west-2.amazonaws.com", + }, + }, + }, "api.mediatailor": service{ Endpoints: endpoints{ @@ -532,6 +590,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -558,6 +617,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -670,6 +730,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -686,6 +747,7 @@ var awsPartition = partition{ Protocols: []string{"http", "https"}, }, Endpoints: endpoints{ + "ap-east-1": endpoint{}, "ap-northeast-1": endpoint{}, "ap-northeast-2": endpoint{}, "ap-south-1": endpoint{}, @@ -693,8 +755,12 @@ var awsPartition = partition{ "ap-southeast-2": endpoint{}, "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, + "eu-north-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, + "eu-west-3": endpoint{}, + "me-south-1": endpoint{}, + "sa-east-1": endpoint{}, "us-east-1": endpoint{}, "us-east-2": endpoint{}, "us-west-1": endpoint{}, @@ -817,6 +883,7 @@ var awsPartition = partition{ "cloud9": service{ Endpoints: endpoints{ + "ap-east-1": endpoint{}, "ap-northeast-1": endpoint{}, "ap-northeast-2": endpoint{}, "ap-south-1": endpoint{}, @@ -827,8 +894,12 @@ var awsPartition = partition{ "eu-north-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, + "eu-west-3": endpoint{}, + "me-south-1": endpoint{}, + "sa-east-1": endpoint{}, "us-east-1": endpoint{}, "us-east-2": endpoint{}, + "us-west-1": endpoint{}, "us-west-2": endpoint{}, }, }, @@ -859,6 +930,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -978,6 +1050,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -1102,6 +1175,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -1357,9 +1431,27 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, - "us-east-1": endpoint{}, - "us-east-2": endpoint{}, - "us-west-2": endpoint{}, + "fips-us-east-1": endpoint{ + Hostname: "comprehendmedical-fips.us-east-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-east-1", + }, + }, + "fips-us-east-2": endpoint{ + Hostname: "comprehendmedical-fips.us-east-2.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-east-2", + }, + }, + "fips-us-west-2": endpoint{ + Hostname: "comprehendmedical-fips.us-west-2.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-west-2", + }, + }, + "us-east-1": endpoint{}, + "us-east-2": endpoint{}, + "us-west-2": endpoint{}, }, }, "config": service{ @@ -1533,6 +1625,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -1571,8 +1664,10 @@ var awsPartition = partition{ "discovery": service{ Endpoints: endpoints{ + "ap-northeast-1": endpoint{}, "ap-southeast-2": endpoint{}, "eu-central-1": endpoint{}, + "us-east-1": endpoint{}, "us-west-2": endpoint{}, }, }, @@ -1595,6 +1690,7 @@ var awsPartition = partition{ }, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -1763,6 +1859,7 @@ var awsPartition = partition{ }, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -1820,6 +1917,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -1885,6 +1983,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -1920,27 +2019,46 @@ var awsPartition = partition{ "us-west-2": endpoint{}, }, }, - "elastic-inference": service{ - + "eks": service{ + Defaults: endpoint{ + Protocols: []string{"http", "https"}, + }, Endpoints: endpoints{ - "ap-northeast-1": endpoint{ - Hostname: "api.elastic-inference.ap-northeast-1.amazonaws.com", + "ap-east-1": endpoint{}, + "ap-northeast-1": endpoint{}, + "ap-northeast-2": endpoint{}, + "ap-south-1": endpoint{}, + "ap-southeast-1": endpoint{}, + "ap-southeast-2": endpoint{}, + "ca-central-1": endpoint{}, + "eu-central-1": endpoint{}, + "eu-north-1": endpoint{}, + "eu-west-1": endpoint{}, + "eu-west-2": endpoint{}, + "eu-west-3": endpoint{}, + "fips-us-east-1": endpoint{ + Hostname: "fips.eks.us-east-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-east-1", + }, }, - "ap-northeast-2": endpoint{ - Hostname: "api.elastic-inference.ap-northeast-2.amazonaws.com", + "fips-us-east-2": endpoint{ + Hostname: "fips.eks.us-east-2.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-east-2", + }, }, - "eu-west-1": endpoint{ - Hostname: "api.elastic-inference.eu-west-1.amazonaws.com", - }, - "us-east-1": endpoint{ - Hostname: "api.elastic-inference.us-east-1.amazonaws.com", - }, - "us-east-2": endpoint{ - Hostname: "api.elastic-inference.us-east-2.amazonaws.com", - }, - "us-west-2": endpoint{ - Hostname: "api.elastic-inference.us-west-2.amazonaws.com", + "fips-us-west-2": endpoint{ + Hostname: "fips.eks.us-west-2.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-west-2", + }, }, + "me-south-1": endpoint{}, + "sa-east-1": endpoint{}, + "us-east-1": endpoint{}, + "us-east-2": endpoint{}, + "us-west-2": endpoint{}, }, }, "elasticache": service{ @@ -1956,6 +2074,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -1986,6 +2105,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -2024,6 +2144,7 @@ var awsPartition = partition{ "elasticfilesystem": service{ Endpoints: endpoints{ + "af-south-1": endpoint{}, "ap-east-1": endpoint{}, "ap-northeast-1": endpoint{}, "ap-northeast-2": endpoint{}, @@ -2033,9 +2154,16 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, + "fips-af-south-1": endpoint{ + Hostname: "elasticfilesystem-fips.af-south-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "af-south-1", + }, + }, "fips-ap-east-1": endpoint{ Hostname: "elasticfilesystem-fips.ap-east-1.amazonaws.com", CredentialScope: credentialScope{ @@ -2090,6 +2218,12 @@ var awsPartition = partition{ Region: "eu-north-1", }, }, + "fips-eu-south-1": endpoint{ + Hostname: "elasticfilesystem-fips.eu-south-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "eu-south-1", + }, + }, "fips-eu-west-1": endpoint{ Hostname: "elasticfilesystem-fips.eu-west-1.amazonaws.com", CredentialScope: credentialScope{ @@ -2167,6 +2301,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -2220,6 +2355,7 @@ var awsPartition = partition{ SSLCommonName: "{service}.{region}.{dnsSuffix}", }, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -2310,6 +2446,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -2340,6 +2477,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -2617,6 +2755,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -2729,6 +2868,7 @@ var awsPartition = partition{ Endpoints: endpoints{ "ap-southeast-2": endpoint{}, "eu-north-1": endpoint{}, + "eu-west-1": endpoint{}, "me-south-1": endpoint{}, "us-east-2": endpoint{}, "us-west-2": endpoint{}, @@ -3057,6 +3197,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -3147,6 +3288,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -3168,8 +3310,11 @@ var awsPartition = partition{ "ap-southeast-2": endpoint{}, "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, + "eu-north-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, + "eu-west-3": endpoint{}, + "sa-east-1": endpoint{}, "us-east-1": endpoint{}, "us-east-2": endpoint{}, "us-west-1": endpoint{}, @@ -3189,6 +3334,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -3302,6 +3448,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -3320,6 +3467,25 @@ var awsPartition = partition{ "us-east-1": endpoint{}, }, }, + "macie": service{ + + Endpoints: endpoints{ + "fips-us-east-1": endpoint{ + Hostname: "macie-fips.us-east-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-east-1", + }, + }, + "fips-us-west-2": endpoint{ + Hostname: "macie-fips.us-west-2.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-west-2", + }, + }, + "us-east-1": endpoint{}, + "us-west-2": endpoint{}, + }, + }, "managedblockchain": service{ Endpoints: endpoints{ @@ -3367,14 +3533,45 @@ var awsPartition = partition{ "ap-southeast-2": endpoint{}, "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, + "eu-north-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, - "sa-east-1": endpoint{}, - "us-east-1": endpoint{}, - "us-east-2": endpoint{}, - "us-west-1": endpoint{}, - "us-west-2": endpoint{}, + "fips-ca-central-1": endpoint{ + Hostname: "mediaconvert-fips.ca-central-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "ca-central-1", + }, + }, + "fips-us-east-1": endpoint{ + Hostname: "mediaconvert-fips.us-east-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-east-1", + }, + }, + "fips-us-east-2": endpoint{ + Hostname: "mediaconvert-fips.us-east-2.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-east-2", + }, + }, + "fips-us-west-1": endpoint{ + Hostname: "mediaconvert-fips.us-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-west-1", + }, + }, + "fips-us-west-2": endpoint{ + Hostname: "mediaconvert-fips.us-west-2.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-west-2", + }, + }, + "sa-east-1": endpoint{}, + "us-east-1": endpoint{}, + "us-east-2": endpoint{}, + "us-west-1": endpoint{}, + "us-west-2": endpoint{}, }, }, "medialive": service{ @@ -3446,6 +3643,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -3460,8 +3658,10 @@ var awsPartition = partition{ "mgh": service{ Endpoints: endpoints{ + "ap-northeast-1": endpoint{}, "ap-southeast-2": endpoint{}, "eu-central-1": endpoint{}, + "us-east-1": endpoint{}, "us-west-2": endpoint{}, }, }, @@ -3502,6 +3702,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -3815,11 +4016,41 @@ var awsPartition = partition{ "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, - "me-south-1": endpoint{}, - "us-east-1": endpoint{}, - "us-east-2": endpoint{}, - "us-west-1": endpoint{}, - "us-west-2": endpoint{}, + "fips-ca-central-1": endpoint{ + Hostname: "outposts-fips.ca-central-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "ca-central-1", + }, + }, + "fips-us-east-1": endpoint{ + Hostname: "outposts-fips.us-east-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-east-1", + }, + }, + "fips-us-east-2": endpoint{ + Hostname: "outposts-fips.us-east-2.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-east-2", + }, + }, + "fips-us-west-1": endpoint{ + Hostname: "outposts-fips.us-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-west-1", + }, + }, + "fips-us-west-2": endpoint{ + Hostname: "outposts-fips.us-west-2.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-west-2", + }, + }, + "me-south-1": endpoint{}, + "us-east-1": endpoint{}, + "us-east-2": endpoint{}, + "us-west-1": endpoint{}, + "us-west-2": endpoint{}, }, }, "pinpoint": service{ @@ -4027,11 +4258,42 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, "me-south-1": endpoint{}, - "sa-east-1": endpoint{}, + "rds-fips.ca-central-1": endpoint{ + Hostname: "rds-fips.ca-central-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "ca-central-1", + }, + }, + "rds-fips.us-east-1": endpoint{ + Hostname: "rds-fips.us-east-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-east-1", + }, + }, + "rds-fips.us-east-2": endpoint{ + Hostname: "rds-fips.us-east-2.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-east-2", + }, + }, + "rds-fips.us-west-1": endpoint{ + Hostname: "rds-fips.us-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-west-1", + }, + }, + "rds-fips.us-west-2": endpoint{ + Hostname: "rds-fips.us-west-2.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-west-2", + }, + }, + "sa-east-1": endpoint{}, "us-east-1": endpoint{ SSLCommonName: "{service}.{dnsSuffix}", }, @@ -4053,6 +4315,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -4124,6 +4387,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -4315,6 +4579,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{ Hostname: "s3.eu-west-1.amazonaws.com", SignatureVersions: []string{"s3", "s3v4"}, @@ -4515,10 +4780,22 @@ var awsPartition = partition{ "schemas": service{ Endpoints: endpoints{ + "ap-east-1": endpoint{}, "ap-northeast-1": endpoint{}, + "ap-northeast-2": endpoint{}, + "ap-south-1": endpoint{}, + "ap-southeast-1": endpoint{}, + "ap-southeast-2": endpoint{}, + "ca-central-1": endpoint{}, + "eu-central-1": endpoint{}, + "eu-north-1": endpoint{}, "eu-west-1": endpoint{}, + "eu-west-2": endpoint{}, + "eu-west-3": endpoint{}, + "sa-east-1": endpoint{}, "us-east-1": endpoint{}, "us-east-2": endpoint{}, + "us-west-1": endpoint{}, "us-west-2": endpoint{}, }, }, @@ -4543,6 +4820,7 @@ var awsPartition = partition{ "secretsmanager": service{ Endpoints: endpoints{ + "af-south-1": endpoint{}, "ap-east-1": endpoint{}, "ap-northeast-1": endpoint{}, "ap-northeast-2": endpoint{}, @@ -4552,6 +4830,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -4590,6 +4869,7 @@ var awsPartition = partition{ "securityhub": service{ Endpoints: endpoints{ + "af-south-1": endpoint{}, "ap-east-1": endpoint{}, "ap-northeast-1": endpoint{}, "ap-northeast-2": endpoint{}, @@ -4599,6 +4879,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -4756,20 +5037,21 @@ var awsPartition = partition{ }, }, "shield": service{ - IsRegionalized: boxedFalse, + PartitionEndpoint: "aws-global", + IsRegionalized: boxedFalse, Defaults: endpoint{ SSLCommonName: "shield.us-east-1.amazonaws.com", Protocols: []string{"https"}, }, Endpoints: endpoints{ - "fips-us-east-1": endpoint{ - Hostname: "shield-fips.us-east-1.amazonaws.com", + "aws-global": endpoint{ + Hostname: "shield.us-east-1.amazonaws.com", CredentialScope: credentialScope{ Region: "us-east-1", }, }, - "us-east-1": endpoint{ - Hostname: "shield.us-east-1.amazonaws.com", + "fips-aws-global": endpoint{ + Hostname: "shield-fips.us-east-1.amazonaws.com", CredentialScope: credentialScope{ Region: "us-east-1", }, @@ -4789,6 +5071,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -4949,6 +5232,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -5000,6 +5284,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -5050,6 +5335,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -5122,6 +5408,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -5160,6 +5447,7 @@ var awsPartition = partition{ "storagegateway": service{ Endpoints: endpoints{ + "af-south-1": endpoint{}, "ap-east-1": endpoint{}, "ap-northeast-1": endpoint{}, "ap-northeast-2": endpoint{}, @@ -5169,6 +5457,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -5264,6 +5553,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -5324,6 +5614,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -5372,6 +5663,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -5806,6 +6098,7 @@ var awsPartition = partition{ "ca-central-1": endpoint{}, "eu-central-1": endpoint{}, "eu-north-1": endpoint{}, + "eu-south-1": endpoint{}, "eu-west-1": endpoint{}, "eu-west-2": endpoint{}, "eu-west-3": endpoint{}, @@ -5967,6 +6260,13 @@ var awscnPartition = partition{ "cn-northwest-1": endpoint{}, }, }, + "codecommit": service{ + + Endpoints: endpoints{ + "cn-north-1": endpoint{}, + "cn-northwest-1": endpoint{}, + }, + }, "codedeploy": service{ Endpoints: endpoints{ @@ -6050,6 +6350,15 @@ var awscnPartition = partition{ "cn-northwest-1": endpoint{}, }, }, + "eks": service{ + Defaults: endpoint{ + Protocols: []string{"http", "https"}, + }, + Endpoints: endpoints{ + "cn-north-1": endpoint{}, + "cn-northwest-1": endpoint{}, + }, + }, "elasticache": service{ Endpoints: endpoints{ @@ -6284,6 +6593,19 @@ var awscnPartition = partition{ "cn-northwest-1": endpoint{}, }, }, + "route53": service{ + PartitionEndpoint: "aws-cn-global", + IsRegionalized: boxedFalse, + + Endpoints: endpoints{ + "aws-cn-global": endpoint{ + Hostname: "route53.amazonaws.com.cn", + CredentialScope: credentialScope{ + Region: "cn-northwest-1", + }, + }, + }, + }, "runtime.sagemaker": service{ Endpoints: endpoints{ @@ -6295,6 +6617,9 @@ var awscnPartition = partition{ Defaults: endpoint{ Protocols: []string{"http", "https"}, SignatureVersions: []string{"s3v4"}, + + HasDualStack: boxedTrue, + DualStackHostname: "{service}.dualstack.{region}.{dnsSuffix}", }, Endpoints: endpoints{ "cn-north-1": endpoint{}, @@ -6305,6 +6630,9 @@ var awscnPartition = partition{ Defaults: endpoint{ Protocols: []string{"https"}, SignatureVersions: []string{"s3v4"}, + + HasDualStack: boxedTrue, + DualStackHostname: "{service}.dualstack.{region}.{dnsSuffix}", }, Endpoints: endpoints{ "cn-north-1": endpoint{ @@ -6507,7 +6835,7 @@ var awsusgovPartition = partition{ Description: "AWS GovCloud (US-East)", }, "us-gov-west-1": region{ - Description: "AWS GovCloud (US)", + Description: "AWS GovCloud (US-West)", }, }, Services: services{ @@ -6530,6 +6858,18 @@ var awsusgovPartition = partition{ Protocols: []string{"https"}, }, Endpoints: endpoints{ + "fips-us-gov-east-1": endpoint{ + Hostname: "acm-pca.us-gov-east-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-east-1", + }, + }, + "fips-us-gov-west-1": endpoint{ + Hostname: "acm-pca.us-gov-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, + }, "us-gov-east-1": endpoint{}, "us-gov-west-1": endpoint{}, }, @@ -6628,7 +6968,9 @@ var awsusgovPartition = partition{ "autoscaling": service{ Endpoints: endpoints{ - "us-gov-east-1": endpoint{}, + "us-gov-east-1": endpoint{ + Protocols: []string{"http", "https"}, + }, "us-gov-west-1": endpoint{ Protocols: []string{"http", "https"}, }, @@ -6671,8 +7013,18 @@ var awsusgovPartition = partition{ "cloudformation": service{ Endpoints: endpoints{ - "us-gov-east-1": endpoint{}, - "us-gov-west-1": endpoint{}, + "us-gov-east-1": endpoint{ + Hostname: "cloudformation.us-gov-east-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-east-1", + }, + }, + "us-gov-west-1": endpoint{ + Hostname: "cloudformation.us-gov-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, + }, }, }, "cloudhsm": service{ @@ -6695,8 +7047,18 @@ var awsusgovPartition = partition{ "cloudtrail": service{ Endpoints: endpoints{ - "us-gov-east-1": endpoint{}, - "us-gov-west-1": endpoint{}, + "us-gov-east-1": endpoint{ + Hostname: "cloudtrail.us-gov-east-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-east-1", + }, + }, + "us-gov-west-1": endpoint{ + Hostname: "cloudtrail.us-gov-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, + }, }, }, "codebuild": service{ @@ -6762,6 +7124,24 @@ var awsusgovPartition = partition{ "us-gov-west-1": endpoint{}, }, }, + "cognito-identity": service{ + + Endpoints: endpoints{ + "us-gov-west-1": endpoint{}, + }, + }, + "cognito-idp": service{ + + Endpoints: endpoints{ + "fips-us-gov-west-1": endpoint{ + Hostname: "cognito-idp-fips.us-gov-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, + }, + "us-gov-west-1": endpoint{}, + }, + }, "comprehend": service{ Defaults: endpoint{ Protocols: []string{"https"}, @@ -6923,6 +7303,15 @@ var awsusgovPartition = partition{ "us-gov-west-1": endpoint{}, }, }, + "eks": service{ + Defaults: endpoint{ + Protocols: []string{"http", "https"}, + }, + Endpoints: endpoints{ + "us-gov-east-1": endpoint{}, + "us-gov-west-1": endpoint{}, + }, + }, "elasticache": service{ Endpoints: endpoints{ @@ -6975,6 +7364,18 @@ var awsusgovPartition = partition{ "elasticloadbalancing": service{ Endpoints: endpoints{ + "fips-us-gov-east-1": endpoint{ + Hostname: "elasticloadbalancing-fips.us-gov-east-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-east-1", + }, + }, + "fips-us-gov-west-1": endpoint{ + Hostname: "elasticloadbalancing-fips.us-gov-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, + }, "us-gov-east-1": endpoint{}, "us-gov-west-1": endpoint{ Protocols: []string{"http", "https"}, @@ -6984,12 +7385,36 @@ var awsusgovPartition = partition{ "elasticmapreduce": service{ Endpoints: endpoints{ + "fips-us-gov-east-1": endpoint{ + Hostname: "elasticmapreduce.us-gov-east-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-east-1", + }, + }, + "fips-us-gov-west-1": endpoint{ + Hostname: "elasticmapreduce.us-gov-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, + }, "us-gov-east-1": endpoint{}, "us-gov-west-1": endpoint{ Protocols: []string{"https"}, }, }, }, + "email": service{ + + Endpoints: endpoints{ + "fips-us-gov-west-1": endpoint{ + Hostname: "email-fips.us-gov-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, + }, + "us-gov-west-1": endpoint{}, + }, + }, "es": service{ Endpoints: endpoints{ @@ -7006,8 +7431,18 @@ var awsusgovPartition = partition{ "events": service{ Endpoints: endpoints{ - "us-gov-east-1": endpoint{}, - "us-gov-west-1": endpoint{}, + "us-gov-east-1": endpoint{ + Hostname: "events.us-gov-east-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-east-1", + }, + }, + "us-gov-west-1": endpoint{ + Hostname: "events.us-gov-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, + }, }, }, "firehose": service{ @@ -7072,7 +7507,12 @@ var awsusgovPartition = partition{ Protocols: []string{"https"}, }, Endpoints: endpoints{ - "us-gov-west-1": endpoint{}, + "us-gov-west-1": endpoint{ + Hostname: "greengrass.us-gov-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, + }, }, }, "guardduty": service{ @@ -7082,6 +7522,12 @@ var awsusgovPartition = partition{ }, Endpoints: endpoints{ "us-gov-west-1": endpoint{}, + "us-gov-west-1-fips": endpoint{ + Hostname: "guardduty.us-gov-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, + }, }, }, "health": service{ @@ -7101,6 +7547,12 @@ var awsusgovPartition = partition{ Region: "us-gov-west-1", }, }, + "iam-govcloud-fips": endpoint{ + Hostname: "iam.us-gov.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, + }, }, }, "inspector": service{ @@ -7138,9 +7590,28 @@ var awsusgovPartition = partition{ "us-gov-west-1": endpoint{}, }, }, + "kafka": service{ + + Endpoints: endpoints{ + "us-gov-east-1": endpoint{}, + "us-gov-west-1": endpoint{}, + }, + }, "kinesis": service{ Endpoints: endpoints{ + "fips-us-gov-east-1": endpoint{ + Hostname: "kinesis-fips.us-gov-east-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-east-1", + }, + }, + "fips-us-gov-west-1": endpoint{ + Hostname: "kinesis-fips.us-gov-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, + }, "us-gov-east-1": endpoint{}, "us-gov-west-1": endpoint{}, }, @@ -7206,7 +7677,12 @@ var awsusgovPartition = partition{ "mediaconvert": service{ Endpoints: endpoints{ - "us-gov-west-1": endpoint{}, + "us-gov-west-1": endpoint{ + Hostname: "mediaconvert.us-gov-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, + }, }, }, "metering.marketplace": service{ @@ -7267,12 +7743,38 @@ var awsusgovPartition = partition{ Region: "us-gov-west-1", }, }, + "fips-aws-us-gov-global": endpoint{ + Hostname: "organizations.us-gov-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, + }, }, }, "outposts": service{ Endpoints: endpoints{ - "us-gov-east-1": endpoint{}, + "us-gov-east-1": endpoint{ + Hostname: "outposts.us-gov-east-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-east-1", + }, + }, + "us-gov-west-1": endpoint{ + Hostname: "outposts.us-gov-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, + }, + }, + }, + "pinpoint": service{ + Defaults: endpoint{ + CredentialScope: credentialScope{ + Service: "mobiletargeting", + }, + }, + Endpoints: endpoints{ "us-gov-west-1": endpoint{}, }, }, @@ -7298,6 +7800,18 @@ var awsusgovPartition = partition{ "rds": service{ Endpoints: endpoints{ + "rds.us-gov-east-1": endpoint{ + Hostname: "rds.us-gov-east-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-east-1", + }, + }, + "rds.us-gov-west-1": endpoint{ + Hostname: "rds.us-gov-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, + }, "us-gov-east-1": endpoint{}, "us-gov-west-1": endpoint{}, }, @@ -7373,6 +7887,9 @@ var awsusgovPartition = partition{ "s3": service{ Defaults: endpoint{ SignatureVersions: []string{"s3", "s3v4"}, + + HasDualStack: boxedTrue, + DualStackHostname: "{service}.dualstack.{region}.{dnsSuffix}", }, Endpoints: endpoints{ "fips-us-gov-west-1": endpoint{ @@ -7395,6 +7912,9 @@ var awsusgovPartition = partition{ Defaults: endpoint{ Protocols: []string{"https"}, SignatureVersions: []string{"s3v4"}, + + HasDualStack: boxedTrue, + DualStackHostname: "{service}.dualstack.{region}.{dnsSuffix}", }, Endpoints: endpoints{ "us-gov-east-1": endpoint{ @@ -7459,10 +7979,18 @@ var awsusgovPartition = partition{ }, Endpoints: endpoints{ "us-gov-east-1": endpoint{ + Hostname: "serverlessrepo.us-gov-east-1.amazonaws.com", Protocols: []string{"https"}, + CredentialScope: credentialScope{ + Region: "us-gov-east-1", + }, }, "us-gov-west-1": endpoint{ + Hostname: "serverlessrepo.us-gov-west-1.amazonaws.com", Protocols: []string{"https"}, + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, }, }, }, @@ -7526,25 +8054,67 @@ var awsusgovPartition = partition{ "sns": service{ Endpoints: endpoints{ - "us-gov-east-1": endpoint{}, + "us-gov-east-1": endpoint{ + Hostname: "sns.us-gov-east-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-east-1", + }, + }, "us-gov-west-1": endpoint{ + Hostname: "sns.us-gov-west-1.amazonaws.com", Protocols: []string{"http", "https"}, + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, }, }, }, "sqs": service{ Endpoints: endpoints{ - "us-gov-east-1": endpoint{}, + "us-gov-east-1": endpoint{ + Hostname: "sqs.us-gov-east-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-east-1", + }, + }, "us-gov-west-1": endpoint{ + Hostname: "sqs.us-gov-west-1.amazonaws.com", SSLCommonName: "{region}.queue.{dnsSuffix}", Protocols: []string{"http", "https"}, + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, }, }, }, "ssm": service{ Endpoints: endpoints{ + "fips-us-gov-east-1": endpoint{ + Hostname: "ssm.us-gov-east-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-east-1", + }, + }, + "fips-us-gov-west-1": endpoint{ + Hostname: "ssm.us-gov-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, + }, + "ssm-facade-fips-us-gov-east-1": endpoint{ + Hostname: "ssm-facade.us-gov-east-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-east-1", + }, + }, + "ssm-facade-fips-us-gov-west-1": endpoint{ + Hostname: "ssm-facade.us-gov-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, + }, "us-gov-east-1": endpoint{}, "us-gov-west-1": endpoint{}, }, @@ -7602,7 +8172,19 @@ var awsusgovPartition = partition{ Endpoints: endpoints{ "us-gov-east-1": endpoint{}, + "us-gov-east-1-fips": endpoint{ + Hostname: "sts.us-gov-east-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-east-1", + }, + }, "us-gov-west-1": endpoint{}, + "us-gov-west-1-fips": endpoint{ + Hostname: "sts.us-gov-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, + }, }, }, "support": service{ @@ -7615,6 +8197,12 @@ var awsusgovPartition = partition{ Region: "us-gov-west-1", }, }, + "fips-us-gov-west-1": endpoint{ + Hostname: "support.us-gov-west-1.amazonaws.com", + CredentialScope: credentialScope{ + Region: "us-gov-west-1", + }, + }, }, }, "swf": service{ @@ -7889,6 +8477,12 @@ var awsisoPartition = partition{ }, }, }, + "es": service{ + + Endpoints: endpoints{ + "us-iso-east-1": endpoint{}, + }, + }, "events": service{ Endpoints: endpoints{ diff --git a/vendor/github.com/aws/aws-sdk-go/aws/types.go b/vendor/github.com/aws/aws-sdk-go/aws/types.go index d542ef01b..98751ee84 100644 --- a/vendor/github.com/aws/aws-sdk-go/aws/types.go +++ b/vendor/github.com/aws/aws-sdk-go/aws/types.go @@ -239,3 +239,26 @@ func (es errors) Error() string { return strings.Join(parts, "\n") } + +// CopySeekableBody copies the seekable body to an io.Writer +func CopySeekableBody(dst io.Writer, src io.ReadSeeker) (int64, error) { + curPos, err := src.Seek(0, sdkio.SeekCurrent) + if err != nil { + return 0, err + } + + // copy errors may be assumed to be from the body. + n, err := io.Copy(dst, src) + if err != nil { + return n, err + } + + // seek back to the first position after reading to reset + // the body for transmission. + _, err = src.Seek(curPos, sdkio.SeekStart) + if err != nil { + return n, err + } + + return n, nil +} diff --git a/vendor/github.com/aws/aws-sdk-go/aws/version.go b/vendor/github.com/aws/aws-sdk-go/aws/version.go index 64b91ae74..1078db649 100644 --- a/vendor/github.com/aws/aws-sdk-go/aws/version.go +++ b/vendor/github.com/aws/aws-sdk-go/aws/version.go @@ -5,4 +5,4 @@ package aws const SDKName = "aws-sdk-go" // SDKVersion is the version of this SDK -const SDKVersion = "1.30.12" +const SDKVersion = "1.31.9" diff --git a/vendor/github.com/aws/aws-sdk-go/private/checksum/content_md5.go b/vendor/github.com/aws/aws-sdk-go/private/checksum/content_md5.go new file mode 100644 index 000000000..e045f38d8 --- /dev/null +++ b/vendor/github.com/aws/aws-sdk-go/private/checksum/content_md5.go @@ -0,0 +1,53 @@ +package checksum + +import ( + "crypto/md5" + "encoding/base64" + "fmt" + + "github.com/aws/aws-sdk-go/aws" + "github.com/aws/aws-sdk-go/aws/awserr" + "github.com/aws/aws-sdk-go/aws/request" +) + +const contentMD5Header = "Content-Md5" + +// AddBodyContentMD5Handler computes and sets the HTTP Content-MD5 header for requests that +// require it. +func AddBodyContentMD5Handler(r *request.Request) { + // if Content-MD5 header is already present, return + if v := r.HTTPRequest.Header.Get(contentMD5Header); len(v) != 0 { + return + } + + // if S3DisableContentMD5Validation flag is set, return + if aws.BoolValue(r.Config.S3DisableContentMD5Validation) { + return + } + + // if request is presigned, return + if r.IsPresigned() { + return + } + + // if body is not seekable, return + if !aws.IsReaderSeekable(r.Body) { + if r.Config.Logger != nil { + r.Config.Logger.Log(fmt.Sprintf( + "Unable to compute Content-MD5 for unseekable body, S3.%s", + r.Operation.Name)) + } + return + } + + h := md5.New() + + if _, err := aws.CopySeekableBody(h, r.Body); err != nil { + r.Error = awserr.New("ContentMD5", "failed to compute body MD5", err) + return + } + + // encode the md5 checksum in base64 and set the request header. + v := base64.StdEncoding.EncodeToString(h.Sum(nil)) + r.HTTPRequest.Header.Set(contentMD5Header, v) +} diff --git a/vendor/github.com/aws/aws-sdk-go/service/dynamodb/api.go b/vendor/github.com/aws/aws-sdk-go/service/dynamodb/api.go index e7983fa8e..ea2ca2c33 100644 --- a/vendor/github.com/aws/aws-sdk-go/service/dynamodb/api.go +++ b/vendor/github.com/aws/aws-sdk-go/service/dynamodb/api.go @@ -670,7 +670,7 @@ func (c *DynamoDB) CreateGlobalTableRequest(input *CreateGlobalTableInput) (req // relationship between two or more DynamoDB tables with the same table name // in the provided Regions. // -// This method only applies to Version 2017.11.29 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V1.html) +// This operation only applies to Version 2017.11.29 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V1.html) // of global tables. // // If you want to add a new replica table to a global table, each of the following @@ -693,6 +693,14 @@ func (c *DynamoDB) CreateGlobalTableRequest(input *CreateGlobalTableInput) (req // * The global secondary indexes must have the same hash key and sort key // (if present). // +// If local secondary indexes are specified, then the following conditions must +// also be met: +// +// * The local secondary indexes must have the same name. +// +// * The local secondary indexes must have the same hash key and sort key +// (if present). +// // Write capacity settings should be set consistently across your replica tables // and secondary indexes. DynamoDB strongly recommends enabling auto scaling // to manage the write capacity settings for all of your global tables replicas @@ -1839,8 +1847,10 @@ func (c *DynamoDB) DescribeGlobalTableRequest(input *DescribeGlobalTableInput) ( // // Returns information about the specified global table. // -// This method only applies to Version 2017.11.29 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V1.html) -// of global tables. +// This operation only applies to Version 2017.11.29 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V1.html) +// of global tables. If you are using global tables Version 2019.11.21 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V2.html) +// you can use DescribeTable (https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_DescribeTable.html) +// instead. // // Returns awserr.Error for service API and SDK errors. Use runtime type assertions // with awserr.Error's Code and Message methods to get detailed information about @@ -1949,7 +1959,7 @@ func (c *DynamoDB) DescribeGlobalTableSettingsRequest(input *DescribeGlobalTable // // Describes Region-specific settings for a global table. // -// This method only applies to Version 2017.11.29 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V1.html) +// This operation only applies to Version 2017.11.29 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V1.html) // of global tables. // // Returns awserr.Error for service API and SDK errors. Use runtime type assertions @@ -2311,7 +2321,7 @@ func (c *DynamoDB) DescribeTableReplicaAutoScalingRequest(input *DescribeTableRe // // Describes auto scaling settings across replicas of the global table at once. // -// This method only applies to Version 2019.11.21 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V2.html) +// This operation only applies to Version 2019.11.21 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V2.html) // of global tables. // // Returns awserr.Error for service API and SDK errors. Use runtime type assertions @@ -2912,7 +2922,7 @@ func (c *DynamoDB) ListGlobalTablesRequest(input *ListGlobalTablesInput) (req *r // // Lists all global tables that have a replica in the specified Region. // -// This method only applies to Version 2017.11.29 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V1.html) +// This operation only applies to Version 2017.11.29 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V1.html) // of global tables. // // Returns awserr.Error for service API and SDK errors. Use runtime type assertions @@ -3325,9 +3335,15 @@ func (c *DynamoDB) PutItemRequest(input *PutItemInput) (req *request.Request, ou // * PutItem in the AWS SDK for Ruby V2 (http://docs.aws.amazon.com/goto/SdkForRubyV2/dynamodb-2012-08-10/PutItem) // // When you add an item, the primary key attributes are the only required attributes. -// Attribute values cannot be null. String and Binary type attributes must have -// lengths greater than zero. Set type attributes cannot be empty. Requests -// with empty values will be rejected with a ValidationException exception. +// Attribute values cannot be null. +// +// Empty String and Binary attribute values are allowed. Attribute values of +// type String and Binary must have a length greater than zero if the attribute +// is used as a key attribute for a table or index. Set type attributes cannot +// be empty. +// +// Invalid Requests with empty values will be rejected with a ValidationException +// exception. // // To prevent a new item from replacing an existing item, use a conditional // expression that contains the attribute_not_exists function with the name @@ -5709,7 +5725,7 @@ func (c *DynamoDB) UpdateTableReplicaAutoScalingRequest(input *UpdateTableReplic // // Updates auto scaling settings on your global tables at once. // -// This method only applies to Version 2019.11.21 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V2.html) +// This operation only applies to Version 2019.11.21 (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V2.html) // of global tables. // // Returns awserr.Error for service API and SDK errors. Use runtime type assertions @@ -13564,6 +13580,10 @@ type PutItemInput struct { // types for those attributes must match those of the schema in the table's // attribute definition. // + // Empty String and Binary attribute values are allowed. Attribute values of + // type String and Binary must have a length greater than zero if the attribute + // is used as a key attribute for a table or index. + // // For more information about primary keys, see Primary Key (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html#HowItWorks.CoreComponents.PrimaryKey) // in the Amazon DynamoDB Developer Guide. // diff --git a/vendor/github.com/aws/aws-sdk-go/service/s3/api.go b/vendor/github.com/aws/aws-sdk-go/service/s3/api.go index 52e87308f..228cd3cfe 100644 --- a/vendor/github.com/aws/aws-sdk-go/service/s3/api.go +++ b/vendor/github.com/aws/aws-sdk-go/service/s3/api.go @@ -14,6 +14,7 @@ import ( "github.com/aws/aws-sdk-go/aws/awsutil" "github.com/aws/aws-sdk-go/aws/client" "github.com/aws/aws-sdk-go/aws/request" + "github.com/aws/aws-sdk-go/private/checksum" "github.com/aws/aws-sdk-go/private/protocol" "github.com/aws/aws-sdk-go/private/protocol/eventstream" "github.com/aws/aws-sdk-go/private/protocol/eventstream/eventstreamapi" @@ -217,7 +218,7 @@ func (c *S3) CompleteMultipartUploadRequest(input *CompleteMultipartUploadInput) // does not exist. The upload ID might be invalid, or the multipart upload // might have been aborted or completed. 404 Not Found // -// The following operations are related to DeleteBucketMetricsConfiguration: +// The following operations are related to CompleteMultipartUpload: // // * CreateMultipartUpload // @@ -305,20 +306,9 @@ func (c *S3) CopyObjectRequest(input *CopyObjectInput) (req *request.Request, ou // // You can store individual objects of up to 5 TB in Amazon S3. You create a // copy of your object up to 5 GB in size in a single atomic operation using -// this API. However, for copying an object greater than 5 GB, you must use -// the multipart upload Upload Part - Copy API. For more information, see Copy -// Object Using the REST Multipart Upload API (https://docs.aws.amazon.com/AmazonS3/latest/dev/CopyingObjctsUsingRESTMPUapi.html). -// -// When copying an object, you can preserve all metadata (default) or specify -// new metadata. However, the ACL is not preserved and is set to private for -// the user making the request. To override the default ACL setting, specify -// a new ACL when generating a copy request. For more information, see Using -// ACLs (https://docs.aws.amazon.com/AmazonS3/latest/dev/S3_ACLs_UsingACLs.html). -// -// Amazon S3 transfer acceleration does not support cross-region copies. If -// you request a cross-region copy using a transfer acceleration endpoint, you -// get a 400 Bad Request error. For more information about transfer acceleration, -// see Transfer Acceleration (https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html). +// this API. However, to copy an object greater than 5 GB, you must use the +// multipart upload Upload Part - Copy API. For more information, see Copy Object +// Using the REST Multipart Upload API (https://docs.aws.amazon.com/AmazonS3/latest/dev/CopyingObjctsUsingRESTMPUapi.html). // // All copy requests must be authenticated. Additionally, you must have read // access to the source object and write access to the destination bucket. For @@ -326,28 +316,6 @@ func (c *S3) CopyObjectRequest(input *CopyObjectInput) (req *request.Request, ou // Both the Region that you want to copy the object from and the Region that // you want to copy the object to must be enabled for your account. // -// To only copy an object under certain conditions, such as whether the Etag -// matches or whether the object was modified before or after a specified date, -// use the request parameters x-amz-copy-source-if-match, x-amz-copy-source-if-none-match, -// x-amz-copy-source-if-unmodified-since, or x-amz-copy-source-if-modified-since. -// -// All headers with the x-amz- prefix, including x-amz-copy-source, must be -// signed. -// -// You can use this operation to change the storage class of an object that -// is already stored in Amazon S3 using the StorageClass parameter. For more -// information, see Storage Classes (https://docs.aws.amazon.com/AmazonS3/latest/dev/storage-class-intro.html). -// -// The source object that you are copying can be encrypted or unencrypted. If -// the source object is encrypted, it can be encrypted by server-side encryption -// using AWS managed encryption keys or by using a customer-provided encryption -// key. When copying an object, you can request that Amazon S3 encrypt the target -// object by using either the AWS managed encryption keys or by using your own -// encryption key. You can do this regardless of the form of server-side encryption -// that was used to encrypt the source, or even if the source object was not -// encrypted. For more information about server-side encryption, see Using Server-Side -// Encryption (https://docs.aws.amazon.com/AmazonS3/latest/dev/serv-side-encryption.html). -// // A copy request might return an error when Amazon S3 receives the copy request // or while Amazon S3 is copying the files. If the error occurs before the copy // operation starts, you receive a standard Amazon S3 error. If the error occurs @@ -363,31 +331,104 @@ func (c *S3) CopyObjectRequest(input *CopyObjectInput) (req *request.Request, ou // it were not, it would not contain the content-length, and you would need // to read the entire body. // -// Consider the following when using request headers: +// The copy request charge is based on the storage class and Region that you +// specify for the destination object. For pricing information, see Amazon S3 +// pricing (https://aws.amazon.com/s3/pricing/). // -// * Consideration 1 – If both the x-amz-copy-source-if-match and x-amz-copy-source-if-unmodified-since -// headers are present in the request and evaluate as follows, Amazon S3 -// returns 200 OK and copies the data: x-amz-copy-source-if-match condition -// evaluates to true x-amz-copy-source-if-unmodified-since condition evaluates -// to false +// Amazon S3 transfer acceleration does not support cross-Region copies. If +// you request a cross-Region copy using a transfer acceleration endpoint, you +// get a 400 Bad Request error. For more information, see Transfer Acceleration +// (https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html). // -// * Consideration 2 – If both of the x-amz-copy-source-if-none-match and -// x-amz-copy-source-if-modified-since headers are present in the request -// and evaluate as follows, Amazon S3 returns the 412 Precondition Failed -// response code: x-amz-copy-source-if-none-match condition evaluates to -// false x-amz-copy-source-if-modified-since condition evaluates to true +// Metadata // -// The copy request charge is based on the storage class and Region you specify -// for the destination object. For pricing information, see Amazon S3 Pricing -// (https://aws.amazon.com/s3/pricing/). +// When copying an object, you can preserve all metadata (default) or specify +// new metadata. However, the ACL is not preserved and is set to private for +// the user making the request. To override the default ACL setting, specify +// a new ACL when generating a copy request. For more information, see Using +// ACLs (https://docs.aws.amazon.com/AmazonS3/latest/dev/S3_ACLs_UsingACLs.html). // -// Following are other considerations when using CopyObject: +// To specify whether you want the object metadata copied from the source object +// or replaced with metadata provided in the request, you can optionally add +// the x-amz-metadata-directive header. When you grant permissions, you can +// use the s3:x-amz-metadata-directive condition key to enforce certain metadata +// behavior when objects are uploaded. For more information, see Specifying +// Conditions in a Policy (https://docs.aws.amazon.com/AmazonS3/latest/dev/amazon-s3-policy-keys.html) +// in the Amazon S3 Developer Guide. For a complete list of Amazon S3-specific +// condition keys, see Actions, Resources, and Condition Keys for Amazon S3 +// (https://docs.aws.amazon.com/AmazonS3/latest/dev/list_amazons3.html). +// +// x-amz-copy-source-if Headers +// +// To only copy an object under certain conditions, such as whether the Etag +// matches or whether the object was modified before or after a specified date, +// use the following request parameters: +// +// * x-amz-copy-source-if-match +// +// * x-amz-copy-source-if-none-match +// +// * x-amz-copy-source-if-unmodified-since +// +// * x-amz-copy-source-if-modified-since +// +// If both the x-amz-copy-source-if-match and x-amz-copy-source-if-unmodified-since +// headers are present in the request and evaluate as follows, Amazon S3 returns +// 200 OK and copies the data: +// +// * x-amz-copy-source-if-match condition evaluates to true +// +// * x-amz-copy-source-if-unmodified-since condition evaluates to false +// +// If both the x-amz-copy-source-if-none-match and x-amz-copy-source-if-modified-since +// headers are present in the request and evaluate as follows, Amazon S3 returns +// the 412 Precondition Failed response code: +// +// * x-amz-copy-source-if-none-match condition evaluates to false +// +// * x-amz-copy-source-if-modified-since condition evaluates to true +// +// All headers with the x-amz- prefix, including x-amz-copy-source, must be +// signed. +// +// Encryption +// +// The source object that you are copying can be encrypted or unencrypted. The +// source object can be encrypted with server-side encryption using AWS managed +// encryption keys (SSE-S3 or SSE-KMS) or by using a customer-provided encryption +// key. With server-side encryption, Amazon S3 encrypts your data as it writes +// it to disks in its data centers and decrypts the data when you access it. +// +// You can optionally use the appropriate encryption-related headers to request +// server-side encryption for the target object. You have the option to provide +// your own encryption key or use SSE-S3 or SSE-KMS, regardless of the form +// of server-side encryption that was used to encrypt the source object. You +// can even request encryption if the source object was not encrypted. For more +// information about server-side encryption, see Using Server-Side Encryption +// (https://docs.aws.amazon.com/AmazonS3/latest/dev/serv-side-encryption.html). +// +// Access Control List (ACL)-Specific Request Headers +// +// When copying an object, you can optionally use headers to grant ACL-based +// permissions. By default, all objects are private. Only the owner has full +// access control. When adding a new object, you can grant permissions to individual +// AWS accounts or to predefined groups defined by Amazon S3. These permissions +// are then added to the ACL on the object. For more information, see Access +// Control List (ACL) Overview (https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html) +// and Managing ACLs Using the REST API (https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-using-rest-api.html). +// +// Storage Class Options +// +// You can use the CopyObject operation to change the storage class of an object +// that is already stored in Amazon S3 using the StorageClass parameter. For +// more information, see Storage Classes (https://docs.aws.amazon.com/AmazonS3/latest/dev/storage-class-intro.html) +// in the Amazon S3 Service Developer Guide. // // Versioning // // By default, x-amz-copy-source identifies the current version of an object -// to copy. (If the current version is a delete marker, Amazon S3 behaves as -// if the object was deleted.) To copy a different version, use the versionId +// to copy. If the current version is a delete marker, Amazon S3 behaves as +// if the object was deleted. To copy a different version, use the versionId // subresource. // // If you enable versioning on the target bucket, Amazon S3 generates a unique @@ -402,87 +443,6 @@ func (c *S3) CopyObjectRequest(input *CopyObjectInput) (req *request.Request, ou // of this object before you can use it as a source object for the copy operation. // For more information, see . // -// Access Permissions -// -// When copying an object, you can optionally specify the accounts or groups -// that should be granted specific permissions on the new object. There are -// two ways to grant the permissions using the request headers: -// -// * Specify a canned ACL with the x-amz-acl request header. For more information, -// see Canned ACL (https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#CannedACL). -// -// * Specify access permissions explicitly with the x-amz-grant-read, x-amz-grant-read-acp, -// x-amz-grant-write-acp, and x-amz-grant-full-control headers. These parameters -// map to the set of permissions that Amazon S3 supports in an ACL. For more -// information, see Access Control List (ACL) Overview (https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html). -// -// You can use either a canned ACL or specify access permissions explicitly. -// You cannot do both. -// -// Server-Side- Encryption-Specific Request Headers -// -// To encrypt the target object, you must provide the appropriate encryption-related -// request headers. The one you use depends on whether you want to use AWS managed -// encryption keys or provide your own encryption key. -// -// * To encrypt the target object using server-side encryption with an AWS -// managed encryption key, provide the following request headers, as appropriate. -// x-amz-server-side​-encryption x-amz-server-side-encryption-aws-kms-key-id -// x-amz-server-side-encryption-context If you specify x-amz-server-side-encryption:aws:kms, -// but don't provide x-amz-server-side-encryption-aws-kms-key-id, Amazon -// S3 uses the AWS managed CMK in AWS KMS to protect the data. If you want -// to use a customer managed AWS KMS CMK, you must provide the x-amz-server-side-encryption-aws-kms-key-id -// of the symmetric customer managed CMK. Amazon S3 only supports symmetric -// CMKs and not asymmetric CMKs. For more information, see Using Symmetric -// and Asymmetric Keys (https://docs.aws.amazon.com/kms/latest/developerguide/symmetric-asymmetric.html) -// in the AWS Key Management Service Developer Guide. All GET and PUT requests -// for an object protected by AWS KMS fail if you don't make them with SSL -// or by using SigV4. For more information about server-side encryption with -// CMKs stored in AWS KMS (SSE-KMS), see Protecting Data Using Server-Side -// Encryption with CMKs stored in KMS (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html). -// -// * To encrypt the target object using server-side encryption with an encryption -// key that you provide, use the following headers. x-amz-server-side​-encryption​-customer-algorithm -// x-amz-server-side​-encryption​-customer-key x-amz-server-side​-encryption​-customer-key-MD5 -// -// * If the source object is encrypted using server-side encryption with -// customer-provided encryption keys, you must use the following headers. -// x-amz-copy-source​-server-side​-encryption​-customer-algorithm x-amz-copy-source​-server-side​-encryption​-customer-key -// x-amz-copy-source-​server-side​-encryption​-customer-key-MD5 For -// more information about server-side encryption with CMKs stored in AWS -// KMS (SSE-KMS), see Protecting Data Using Server-Side Encryption with CMKs -// stored in Amazon KMS (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html). -// -// Access-Control-List (ACL)-Specific Request Headers -// -// You also can use the following access control–related headers with this -// operation. By default, all objects are private. Only the owner has full access -// control. When adding a new object, you can grant permissions to individual -// AWS accounts or to predefined groups defined by Amazon S3. These permissions -// are then added to the access control list (ACL) on the object. For more information, -// see Using ACLs (https://docs.aws.amazon.com/AmazonS3/latest/dev/S3_ACLs_UsingACLs.html). -// With this operation, you can grant access permissions using one of the following -// two methods: -// -// * Specify a canned ACL (x-amz-acl) — Amazon S3 supports a set of predefined -// ACLs, known as canned ACLs. Each canned ACL has a predefined set of grantees -// and permissions. For more information, see Canned ACL (https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#CannedACL). -// -// * Specify access permissions explicitly — To explicitly grant access -// permissions to specific AWS accounts or groups, use the following headers. -// Each header maps to specific permissions that Amazon S3 supports in an -// ACL. For more information, see Access Control List (ACL) Overview (https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html). -// In the header, you specify a list of grantees who get the specific permission. -// To grant permissions explicitly, use: x-amz-grant-read x-amz-grant-write -// x-amz-grant-read-acp x-amz-grant-write-acp x-amz-grant-full-control You -// specify each grantee as a type=value pair, where the type is one of the -// following: emailAddress – if the value specified is the email address -// of an AWS account id – if the value specified is the canonical user -// ID of an AWS account uri – if you are granting permissions to a predefined -// group For example, the following x-amz-grant-read header grants the AWS -// accounts identified by email addresses permissions to read object data -// and its metadata: x-amz-grant-read: emailAddress="xyz@amazon.com", emailAddress="abc@amazon.com" -// // The following operations are related to CopyObject: // // * PutObject @@ -581,8 +541,8 @@ func (c *S3) CreateBucketRequest(input *CreateBucketInput) (req *request.Request // can optionally specify a Region in the request body. You might choose a Region // to optimize latency, minimize costs, or address regulatory requirements. // For example, if you reside in Europe, you will probably find it advantageous -// to create buckets in the EU (Ireland) Region. For more information, see How -// to Select a Region for Your Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#access-bucket-intro). +// to create buckets in the Europe (Ireland) Region. For more information, see +// How to Select a Region for Your Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#access-bucket-intro). // // If you send your create bucket request to the s3.amazonaws.com endpoint, // the request goes to the us-east-1 Region. Accordingly, the signature calculations @@ -608,12 +568,19 @@ func (c *S3) CreateBucketRequest(input *CreateBucketInput) (req *request.Request // in an ACL. For more information, see Access Control List (ACL) Overview // (https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html). You // specify each grantee as a type=value pair, where the type is one of the -// following: emailAddress – if the value specified is the email address -// of an AWS account id – if the value specified is the canonical user -// ID of an AWS account uri – if you are granting permissions to a predefined -// group For example, the following x-amz-grant-read header grants the AWS -// accounts identified by email addresses permissions to read object data -// and its metadata: x-amz-grant-read: emailAddress="xyz@amazon.com", emailAddress="abc@amazon.com" +// following: id – if the value specified is the canonical user ID of an +// AWS account uri – if you are granting permissions to a predefined group +// emailAddress – if the value specified is the email address of an AWS +// account Using email addresses to specify a grantee is only supported in +// the following AWS Regions: US East (N. Virginia) US West (N. California) +// US West (Oregon) Asia Pacific (Singapore) Asia Pacific (Sydney) Asia Pacific +// (Tokyo) Europe (Ireland) South America (São Paulo) For a list of all +// the Amazon S3 supported Regions and endpoints, see Regions and Endpoints +// (https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region) in +// the AWS General Reference. For example, the following x-amz-grant-read +// header grants the AWS accounts identified by account IDs permissions to +// read object data and its metadata: x-amz-grant-read: id="11112222333", +// id="444455556666" // // You can use either a canned ACL or specify access permissions explicitly. // You cannot do both. @@ -832,12 +799,19 @@ func (c *S3) CreateMultipartUploadRequest(input *CreateMultipartUploadInput) (re // To grant permissions explicitly, use: x-amz-grant-read x-amz-grant-write // x-amz-grant-read-acp x-amz-grant-write-acp x-amz-grant-full-control You // specify each grantee as a type=value pair, where the type is one of the -// following: emailAddress – if the value specified is the email address -// of an AWS account id – if the value specified is the canonical user -// ID of an AWS account uri – if you are granting permissions to a predefined -// group For example, the following x-amz-grant-read header grants the AWS -// accounts identified by email addresses permissions to read object data -// and its metadata: x-amz-grant-read: emailAddress="xyz@amazon.com", emailAddress="abc@amazon.com" +// following: id – if the value specified is the canonical user ID of an +// AWS account uri – if you are granting permissions to a predefined group +// emailAddress – if the value specified is the email address of an AWS +// account Using email addresses to specify a grantee is only supported in +// the following AWS Regions: US East (N. Virginia) US West (N. California) +// US West (Oregon) Asia Pacific (Singapore) Asia Pacific (Sydney) Asia Pacific +// (Tokyo) Europe (Ireland) South America (São Paulo) For a list of all +// the Amazon S3 supported Regions and endpoints, see Regions and Endpoints +// (https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region) in +// the AWS General Reference. For example, the following x-amz-grant-read +// header grants the AWS accounts identified by account IDs permissions to +// read object data and its metadata: x-amz-grant-read: id="11112222333", +// id="444455556666" // // The following operations are related to CreateMultipartUpload: // @@ -1012,7 +986,7 @@ func (c *S3) DeleteBucketAnalyticsConfigurationRequest(input *DeleteBucketAnalyt // To use this operation, you must have permissions to perform the s3:PutAnalyticsConfiguration // action. The bucket owner has this permission by default. The bucket owner // can grant this permission to others. For more information about permissions, -// see Permissions Related to Bucket Subresource Operations (https://docs.aws.amazon.com/AmazonS3/latest/dev//using-with-s3-actions.html#using-with-s3-actions-related-to-bucket-subresources) +// see Permissions Related to Bucket Subresource Operations (https://docs.aws.amazon.com/AmazonS3/latest/dev/using-with-s3-actions.html#using-with-s3-actions-related-to-bucket-subresources) // and Managing Access Permissions to Your Amazon S3 Resources (https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-access-control.html). // // For information about the Amazon S3 analytics feature, see Amazon S3 Analytics @@ -1189,14 +1163,14 @@ func (c *S3) DeleteBucketEncryptionRequest(input *DeleteBucketEncryptionInput) ( // // This implementation of the DELETE operation removes default encryption from // the bucket. For information about the Amazon S3 default encryption feature, -// see Amazon S3 Default Bucket Encryption (https://docs.aws.amazon.com/AmazonS3/latest/dev//bucket-encryption.html) +// see Amazon S3 Default Bucket Encryption (https://docs.aws.amazon.com/AmazonS3/latest/dev/bucket-encryption.html) // in the Amazon Simple Storage Service Developer Guide. // // To use this operation, you must have permissions to perform the s3:PutEncryptionConfiguration // action. The bucket owner has this permission by default. The bucket owner // can grant this permission to others. For more information about permissions, -// see Permissions Related to Bucket Subresource Operations (https://docs.aws.amazon.com/AmazonS3/latest/dev//using-with-s3-actions.html#using-with-s3-actions-related-to-bucket-subresources) -// and Managing Access Permissions to your Amazon S3 Resources (https://docs.aws.amazon.com/AmazonS3/latest/dev//s3-access-control.html) +// see Permissions Related to Bucket Subresource Operations (https://docs.aws.amazon.com/AmazonS3/latest/dev/using-with-s3-actions.html#using-with-s3-actions-related-to-bucket-subresources) +// and Managing Access Permissions to your Amazon S3 Resources (https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-access-control.html) // in the Amazon Simple Storage Service Developer Guide. // // Related Resources @@ -2112,6 +2086,10 @@ func (c *S3) DeleteObjectsRequest(input *DeleteObjectsInput) (req *request.Reque output = &DeleteObjectsOutput{} req = c.newRequest(op, input, output) + req.Handlers.Build.PushBackNamed(request.NamedHandler{ + Name: "contentMd5Handler", + Fn: checksum.AddBodyContentMD5Handler, + }) return } @@ -2239,7 +2217,7 @@ func (c *S3) DeletePublicAccessBlockRequest(input *DeletePublicAccessBlockInput) // Subresource Operations (https://docs.aws.amazon.com/AmazonS3/latest/dev/using-with-s3-actions.html#using-with-s3-actions-related-to-bucket-subresources) // and Managing Access Permissions to Your Amazon S3 Resources (https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-access-control.html). // -// The following operations are related to DeleteBucketMetricsConfiguration: +// The following operations are related to DeletePublicAccessBlock: // // * Using Amazon S3 Block Public Access (https://docs.aws.amazon.com/AmazonS3/latest/dev/access-control-block-public-access.html) // @@ -2329,8 +2307,8 @@ func (c *S3) GetBucketAccelerateConfigurationRequest(input *GetBucketAccelerateC // To use this operation, you must have permission to perform the s3:GetAccelerateConfiguration // action. The bucket owner has this permission by default. The bucket owner // can grant this permission to others. For more information about permissions, -// see Permissions Related to Bucket Subresource Operations (https://docs.aws.amazon.com/AmazonS3/latest/dev//using-with-s3-actions.html#using-with-s3-actions-related-to-bucket-subresources) -// and Managing Access Permissions to your Amazon S3 Resources (https://docs.aws.amazon.com/AmazonS3/latest/dev//s3-access-control.html) +// see Permissions Related to Bucket Subresource Operations (https://docs.aws.amazon.com/AmazonS3/latest/dev/using-with-s3-actions.html#using-with-s3-actions-related-to-bucket-subresources) +// and Managing Access Permissions to your Amazon S3 Resources (https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-access-control.html) // in the Amazon Simple Storage Service Developer Guide. // // You set the Transfer Acceleration state of an existing bucket to Enabled @@ -2341,7 +2319,7 @@ func (c *S3) GetBucketAccelerateConfigurationRequest(input *GetBucketAccelerateC // state if a state has never been set on the bucket. // // For more information about transfer acceleration, see Transfer Acceleration -// (https://docs.aws.amazon.com/AmazonS3/latest/dev//transfer-acceleration.html) +// (https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html) // in the Amazon Simple Storage Service Developer Guide. // // Related Resources @@ -2997,7 +2975,7 @@ func (c *S3) GetBucketLifecycleConfigurationRequest(input *GetBucketLifecycleCon // configuration does not exist. HTTP Status Code: 404 Not Found SOAP Fault // Code Prefix: Client // -// The following operations are related to DeleteBucketMetricsConfiguration: +// The following operations are related to GetBucketLifecycleConfiguration: // // * GetBucketLifecycle // @@ -6430,6 +6408,10 @@ func (c *S3) PutBucketAclRequest(input *PutBucketAclInput) (req *request.Request output = &PutBucketAclOutput{} req = c.newRequest(op, input, output) req.Handlers.Unmarshal.Swap(restxml.UnmarshalHandler.Name, protocol.UnmarshalDiscardBodyHandler) + req.Handlers.Build.PushBackNamed(request.NamedHandler{ + Name: "contentMd5Handler", + Fn: checksum.AddBodyContentMD5Handler, + }) return } @@ -6473,14 +6455,20 @@ func (c *S3) PutBucketAclRequest(input *PutBucketAclInput) (req *request.Request // Amazon S3 supports in an ACL. For more information, see Access Control // List (ACL) Overview (https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html). // You specify each grantee as a type=value pair, where the type is one of -// the following: emailAddress – if the value specified is the email address -// of an AWS account id – if the value specified is the canonical user -// ID of an AWS account uri – if you are granting permissions to a predefined -// group For example, the following x-amz-grant-write header grants create, -// overwrite, and delete objects permission to LogDelivery group predefined -// by Amazon S3 and two AWS accounts identified by their email addresses. -// x-amz-grant-write: uri="http://acs.amazonaws.com/groups/s3/LogDelivery", -// emailAddress="xyz@amazon.com", emailAddress="abc@amazon.com" +// the following: id – if the value specified is the canonical user ID +// of an AWS account uri – if you are granting permissions to a predefined +// group emailAddress – if the value specified is the email address of +// an AWS account Using email addresses to specify a grantee is only supported +// in the following AWS Regions: US East (N. Virginia) US West (N. California) +// US West (Oregon) Asia Pacific (Singapore) Asia Pacific (Sydney) Asia Pacific +// (Tokyo) Europe (Ireland) South America (São Paulo) For a list of all +// the Amazon S3 supported Regions and endpoints, see Regions and Endpoints +// (https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region) in +// the AWS General Reference. For example, the following x-amz-grant-write +// header grants create, overwrite, and delete objects permission to LogDelivery +// group predefined by Amazon S3 and two AWS accounts identified by their +// email addresses. x-amz-grant-write: uri="http://acs.amazonaws.com/groups/s3/LogDelivery", +// id="111122223333", id="555566667777" // // You can use either a canned ACL or specify access permissions explicitly. // You cannot do both. @@ -6490,11 +6478,6 @@ func (c *S3) PutBucketAclRequest(input *PutBucketAclInput) (req *request.Request // You can specify the person (grantee) to whom you're assigning access rights // (using request elements) in the following ways: // -// * By Email address: <>Grantees@email.com<>lt;/Grantee> -// The grantee is resolved to the CanonicalUser and, in a response to a GET -// Object acl request, appears as the CanonicalUser. -// // * By the person's ID: <>ID<><>GranteesEmail<> // DisplayName is optional and ignored in the request @@ -6502,6 +6485,17 @@ func (c *S3) PutBucketAclRequest(input *PutBucketAclInput) (req *request.Request // * By URI: <>http://acs.amazonaws.com/groups/global/AuthenticatedUsers<> // +// * By Email address: <>Grantees@email.com<>lt;/Grantee> +// The grantee is resolved to the CanonicalUser and, in a response to a GET +// Object acl request, appears as the CanonicalUser. Using email addresses +// to specify a grantee is only supported in the following AWS Regions: US +// East (N. Virginia) US West (N. California) US West (Oregon) Asia Pacific +// (Singapore) Asia Pacific (Sydney) Asia Pacific (Tokyo) Europe (Ireland) +// South America (São Paulo) For a list of all the Amazon S3 supported Regions +// and endpoints, see Regions and Endpoints (https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region) +// in the AWS General Reference. +// // Related Resources // // * CreateBucket @@ -6697,6 +6691,10 @@ func (c *S3) PutBucketCorsRequest(input *PutBucketCorsInput) (req *request.Reque output = &PutBucketCorsOutput{} req = c.newRequest(op, input, output) req.Handlers.Unmarshal.Swap(restxml.UnmarshalHandler.Name, protocol.UnmarshalDiscardBodyHandler) + req.Handlers.Build.PushBackNamed(request.NamedHandler{ + Name: "contentMd5Handler", + Fn: checksum.AddBodyContentMD5Handler, + }) return } @@ -6814,6 +6812,10 @@ func (c *S3) PutBucketEncryptionRequest(input *PutBucketEncryptionInput) (req *r output = &PutBucketEncryptionOutput{} req = c.newRequest(op, input, output) req.Handlers.Unmarshal.Swap(restxml.UnmarshalHandler.Name, protocol.UnmarshalDiscardBodyHandler) + req.Handlers.Build.PushBackNamed(request.NamedHandler{ + Name: "contentMd5Handler", + Fn: checksum.AddBodyContentMD5Handler, + }) return } @@ -6824,7 +6826,8 @@ func (c *S3) PutBucketEncryptionRequest(input *PutBucketEncryptionInput) (req *r // // This implementation of the PUT operation sets default encryption for a bucket // using server-side encryption with Amazon S3-managed keys SSE-S3 or AWS KMS -// customer master keys (CMKs) (SSE-KMS). +// customer master keys (CMKs) (SSE-KMS). For information about the Amazon S3 +// default encryption feature, see Amazon S3 Default Bucket Encryption (https://docs.aws.amazon.com/AmazonS3/latest/dev/bucket-encryption.html). // // This operation requires AWS Signature Version 4. For more information, see // Authenticating Requests (AWS Signature Version 4) (sig-v4-authenticating-requests.html). @@ -6929,19 +6932,19 @@ func (c *S3) PutBucketInventoryConfigurationRequest(input *PutBucketInventoryCon // bucket where you want the inventory to be stored, and whether to generate // the inventory daily or weekly. You can also configure what object metadata // to include and whether to inventory all object versions or only current versions. -// For more information, see Amazon S3 Inventory (https://docs.aws.amazon.com/AmazonS3/latest/dev//storage-inventory.html) +// For more information, see Amazon S3 Inventory (https://docs.aws.amazon.com/AmazonS3/latest/dev/storage-inventory.html) // in the Amazon Simple Storage Service Developer Guide. // // You must create a bucket policy on the destination bucket to grant permissions // to Amazon S3 to write objects to the bucket in the defined location. For // an example policy, see Granting Permissions for Amazon S3 Inventory and Storage -// Class Analysis. (https://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html#example-bucket-policies-use-case-9) +// Class Analysis (https://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html#example-bucket-policies-use-case-9). // // To use this operation, you must have permissions to perform the s3:PutInventoryConfiguration // action. The bucket owner has this permission by default and can grant this // permission to others. For more information about permissions, see Permissions -// Related to Bucket Subresource Operations (https://docs.aws.amazon.com/AmazonS3/latest/dev//using-with-s3-actions.html#using-with-s3-actions-related-to-bucket-subresources) -// and Managing Access Permissions to Your Amazon S3 Resources (https://docs.aws.amazon.com/AmazonS3/latest/dev//s3-access-control.html) +// Related to Bucket Subresource Operations (https://docs.aws.amazon.com/AmazonS3/latest/dev/using-with-s3-actions.html#using-with-s3-actions-related-to-bucket-subresources) +// and Managing Access Permissions to Your Amazon S3 Resources (https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-access-control.html) // in the Amazon Simple Storage Service Developer Guide. // // Special Errors @@ -6954,7 +6957,7 @@ func (c *S3) PutBucketInventoryConfigurationRequest(input *PutBucketInventoryCon // // * HTTP 403 Forbidden Error Code: AccessDenied Cause: You are not the owner // of the specified bucket, or you do not have the s3:PutInventoryConfiguration -// bucket permission to set the configuration on the bucket +// bucket permission to set the configuration on the bucket. // // Related Resources // @@ -7037,6 +7040,10 @@ func (c *S3) PutBucketLifecycleRequest(input *PutBucketLifecycleInput) (req *req output = &PutBucketLifecycleOutput{} req = c.newRequest(op, input, output) req.Handlers.Unmarshal.Swap(restxml.UnmarshalHandler.Name, protocol.UnmarshalDiscardBodyHandler) + req.Handlers.Build.PushBackNamed(request.NamedHandler{ + Name: "contentMd5Handler", + Fn: checksum.AddBodyContentMD5Handler, + }) return } @@ -7049,7 +7056,7 @@ func (c *S3) PutBucketLifecycleRequest(input *PutBucketLifecycleInput) (req *req // // Creates a new lifecycle configuration for the bucket or replaces an existing // lifecycle configuration. For information about lifecycle configuration, see -// Object Lifecycle Management (https://docs.aws.amazon.com/AmazonS3/latest/dev//object-lifecycle-mgmt.html) +// Object Lifecycle Management (https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html) // in the Amazon Simple Storage Service Developer Guide. // // By default, all Amazon S3 resources, including buckets, objects, and related @@ -7071,11 +7078,11 @@ func (c *S3) PutBucketLifecycleRequest(input *PutBucketLifecycleInput) (req *req // * s3:PutLifecycleConfiguration // // For more information about permissions, see Managing Access Permissions to -// your Amazon S3 Resources (https://docs.aws.amazon.com/AmazonS3/latest/dev//s3-access-control.html) +// your Amazon S3 Resources (https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-access-control.html) // in the Amazon Simple Storage Service Developer Guide. // // For more examples of transitioning objects to storage classes such as STANDARD_IA -// or ONEZONE_IA, see Examples of Lifecycle Configuration (https://docs.aws.amazon.com/AmazonS3/latest/dev//intro-lifecycle-rules.html#lifecycle-configuration-examples). +// or ONEZONE_IA, see Examples of Lifecycle Configuration (https://docs.aws.amazon.com/AmazonS3/latest/dev/intro-lifecycle-rules.html#lifecycle-configuration-examples). // // Related Resources // @@ -7089,8 +7096,8 @@ func (c *S3) PutBucketLifecycleRequest(input *PutBucketLifecycleInput) (req *req // the AWS account that created the bucket—can perform any of the operations. // A resource owner can also grant others permission to perform the operation. // For more information, see the following topics in the Amazon Simple Storage -// Service Developer Guide: Specifying Permissions in a Policy (https://docs.aws.amazon.com/AmazonS3/latest/dev//using-with-s3-actions.html) -// Managing Access Permissions to your Amazon S3 Resources (https://docs.aws.amazon.com/AmazonS3/latest/dev//s3-access-control.html) +// Service Developer Guide: Specifying Permissions in a Policy (https://docs.aws.amazon.com/AmazonS3/latest/dev/using-with-s3-actions.html) +// Managing Access Permissions to your Amazon S3 Resources (https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-access-control.html) // // Returns awserr.Error for service API and SDK errors. Use runtime type assertions // with awserr.Error's Code and Message methods to get detailed information about @@ -7164,6 +7171,10 @@ func (c *S3) PutBucketLifecycleConfigurationRequest(input *PutBucketLifecycleCon output = &PutBucketLifecycleConfigurationOutput{} req = c.newRequest(op, input, output) req.Handlers.Unmarshal.Swap(restxml.UnmarshalHandler.Name, protocol.UnmarshalDiscardBodyHandler) + req.Handlers.Build.PushBackNamed(request.NamedHandler{ + Name: "contentMd5Handler", + Fn: checksum.AddBodyContentMD5Handler, + }) return } @@ -7301,6 +7312,10 @@ func (c *S3) PutBucketLoggingRequest(input *PutBucketLoggingInput) (req *request output = &PutBucketLoggingOutput{} req = c.newRequest(op, input, output) req.Handlers.Unmarshal.Swap(restxml.UnmarshalHandler.Name, protocol.UnmarshalDiscardBodyHandler) + req.Handlers.Build.PushBackNamed(request.NamedHandler{ + Name: "contentMd5Handler", + Fn: checksum.AddBodyContentMD5Handler, + }) return } @@ -7528,6 +7543,10 @@ func (c *S3) PutBucketNotificationRequest(input *PutBucketNotificationInput) (re output = &PutBucketNotificationOutput{} req = c.newRequest(op, input, output) req.Handlers.Unmarshal.Swap(restxml.UnmarshalHandler.Name, protocol.UnmarshalDiscardBodyHandler) + req.Handlers.Build.PushBackNamed(request.NamedHandler{ + Name: "contentMd5Handler", + Fn: checksum.AddBodyContentMD5Handler, + }) return } @@ -7730,6 +7749,10 @@ func (c *S3) PutBucketPolicyRequest(input *PutBucketPolicyInput) (req *request.R output = &PutBucketPolicyOutput{} req = c.newRequest(op, input, output) req.Handlers.Unmarshal.Swap(restxml.UnmarshalHandler.Name, protocol.UnmarshalDiscardBodyHandler) + req.Handlers.Build.PushBackNamed(request.NamedHandler{ + Name: "contentMd5Handler", + Fn: checksum.AddBodyContentMD5Handler, + }) return } @@ -7826,6 +7849,10 @@ func (c *S3) PutBucketReplicationRequest(input *PutBucketReplicationInput) (req output = &PutBucketReplicationOutput{} req = c.newRequest(op, input, output) req.Handlers.Unmarshal.Swap(restxml.UnmarshalHandler.Name, protocol.UnmarshalDiscardBodyHandler) + req.Handlers.Build.PushBackNamed(request.NamedHandler{ + Name: "contentMd5Handler", + Fn: checksum.AddBodyContentMD5Handler, + }) return } @@ -7950,6 +7977,10 @@ func (c *S3) PutBucketRequestPaymentRequest(input *PutBucketRequestPaymentInput) output = &PutBucketRequestPaymentOutput{} req = c.newRequest(op, input, output) req.Handlers.Unmarshal.Swap(restxml.UnmarshalHandler.Name, protocol.UnmarshalDiscardBodyHandler) + req.Handlers.Build.PushBackNamed(request.NamedHandler{ + Name: "contentMd5Handler", + Fn: checksum.AddBodyContentMD5Handler, + }) return } @@ -8035,6 +8066,10 @@ func (c *S3) PutBucketTaggingRequest(input *PutBucketTaggingInput) (req *request output = &PutBucketTaggingOutput{} req = c.newRequest(op, input, output) req.Handlers.Unmarshal.Swap(restxml.UnmarshalHandler.Name, protocol.UnmarshalDiscardBodyHandler) + req.Handlers.Build.PushBackNamed(request.NamedHandler{ + Name: "contentMd5Handler", + Fn: checksum.AddBodyContentMD5Handler, + }) return } @@ -8065,8 +8100,8 @@ func (c *S3) PutBucketTaggingRequest(input *PutBucketTaggingInput) (req *request // * Error code: InvalidTagError Description: The tag provided was not a // valid tag. This error can occur if the tag did not pass input validation. // For information about tag restrictions, see User-Defined Tag Restrictions -// (https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2//allocation-tag-restrictions.html) -// and AWS-Generated Cost Allocation Tag Restrictions (https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2//aws-tag-restrictions.html). +// (https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/allocation-tag-restrictions.html) +// and AWS-Generated Cost Allocation Tag Restrictions (https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/aws-tag-restrictions.html). // // * Error code: MalformedXMLError Description: The XML provided does not // match the schema. @@ -8151,6 +8186,10 @@ func (c *S3) PutBucketVersioningRequest(input *PutBucketVersioningInput) (req *r output = &PutBucketVersioningOutput{} req = c.newRequest(op, input, output) req.Handlers.Unmarshal.Swap(restxml.UnmarshalHandler.Name, protocol.UnmarshalDiscardBodyHandler) + req.Handlers.Build.PushBackNamed(request.NamedHandler{ + Name: "contentMd5Handler", + Fn: checksum.AddBodyContentMD5Handler, + }) return } @@ -8259,6 +8298,10 @@ func (c *S3) PutBucketWebsiteRequest(input *PutBucketWebsiteInput) (req *request output = &PutBucketWebsiteOutput{} req = c.newRequest(op, input, output) req.Handlers.Unmarshal.Swap(restxml.UnmarshalHandler.Name, protocol.UnmarshalDiscardBodyHandler) + req.Handlers.Build.PushBackNamed(request.NamedHandler{ + Name: "contentMd5Handler", + Fn: checksum.AddBodyContentMD5Handler, + }) return } @@ -8326,6 +8369,11 @@ func (c *S3) PutBucketWebsiteRequest(input *PutBucketWebsiteInput) (req *request // // * HttpRedirectCode // +// Amazon S3 has a limitation of 50 routing rules per website configuration. +// If you require more than 50 routing rules, you can use object redirect. For +// more information, see Configuring an Object Redirect (https://docs.aws.amazon.com/AmazonS3/latest/dev/how-to-page-redirect.html) +// in the Amazon Simple Storage Service Developer Guide. +// // Returns awserr.Error for service API and SDK errors. Use runtime type assertions // with awserr.Error's Code and Message methods to get detailed information about // the error. @@ -8415,12 +8463,12 @@ func (c *S3) PutObjectRequest(input *PutObjectInput) (req *request.Request, outp // you can calculate the MD5 while putting an object to Amazon S3 and compare // the returned ETag to the calculated MD5 value. // -// To configure your application to send the request headers before sending -// the request body, use the 100-continue HTTP status code. For PUT operations, -// this helps you avoid sending the message body if the message is rejected -// based on the headers (for example, because authentication fails or a redirect -// occurs). For more information on the 100-continue HTTP status code, see Section -// 8.2.3 of http://www.ietf.org/rfc/rfc2616.txt (http://www.ietf.org/rfc/rfc2616.txt). +// The Content-MD5 header is required for any request to upload an object with +// a retention period configured using Amazon S3 Object Lock. For more information +// about Amazon S3 Object Lock, see Amazon S3 Object Lock Overview (https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lock-overview.html) +// in the Amazon Simple Storage Service Developer Guide. +// +// Server-side Encryption // // You can optionally request server-side encryption. With server-side encryption, // Amazon S3 encrypts your data as it writes it to disks in its data centers @@ -8428,143 +8476,34 @@ func (c *S3) PutObjectRequest(input *PutObjectInput) (req *request.Request, outp // your own encryption key or use AWS managed encryption keys. For more information, // see Using Server-Side Encryption (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html). // -// Access Permissions +// Access Control List (ACL)-Specific Request Headers // -// You can optionally specify the accounts or groups that should be granted -// specific permissions on the new object. There are two ways to grant the permissions -// using the request headers: -// -// * Specify a canned ACL with the x-amz-acl request header. For more information, -// see Canned ACL (https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#CannedACL). -// -// * Specify access permissions explicitly with the x-amz-grant-read, x-amz-grant-read-acp, -// x-amz-grant-write-acp, and x-amz-grant-full-control headers. These parameters -// map to the set of permissions that Amazon S3 supports in an ACL. For more -// information, see Access Control List (ACL) Overview (https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html). -// -// You can use either a canned ACL or specify access permissions explicitly. -// You cannot do both. -// -// Server-Side- Encryption-Specific Request Headers -// -// You can optionally tell Amazon S3 to encrypt data at rest using server-side -// encryption. Server-side encryption is for data encryption at rest. Amazon -// S3 encrypts your data as it writes it to disks in its data centers and decrypts -// it when you access it. The option you use depends on whether you want to -// use AWS managed encryption keys or provide your own encryption key. -// -// * Use encryption keys managed by Amazon S3 or customer master keys (CMKs) -// stored in AWS Key Management Service (AWS KMS) – If you want AWS to -// manage the keys used to encrypt data, specify the following headers in -// the request. x-amz-server-side​-encryption x-amz-server-side-encryption-aws-kms-key-id -// x-amz-server-side-encryption-context If you specify x-amz-server-side-encryption:aws:kms, -// but don't provide x-amz-server-side-encryption-aws-kms-key-id, Amazon -// S3 uses the AWS managed CMK in AWS KMS to protect the data. If you want -// to use a customer managed AWS KMS CMK, you must provide the x-amz-server-side-encryption-aws-kms-key-id -// of the symmetric customer managed CMK. Amazon S3 only supports symmetric -// CMKs and not asymmetric CMKs. For more information, see Using Symmetric -// and Asymmetric Keys (https://docs.aws.amazon.com/kms/latest/developerguide/symmetric-asymmetric.html) -// in the AWS Key Management Service Developer Guide. All GET and PUT requests -// for an object protected by AWS KMS fail if you don't make them with SSL -// or by using SigV4. For more information about server-side encryption with -// CMKs stored in AWS KMS (SSE-KMS), see Protecting Data Using Server-Side -// Encryption with CMKs stored in AWS (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html). -// -// * Use customer-provided encryption keys – If you want to manage your -// own encryption keys, provide all the following headers in the request. -// x-amz-server-side​-encryption​-customer-algorithm x-amz-server-side​-encryption​-customer-key -// x-amz-server-side​-encryption​-customer-key-MD5 For more information -// about server-side encryption with CMKs stored in KMS (SSE-KMS), see Protecting -// Data Using Server-Side Encryption with CMKs stored in AWS (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html). -// -// Access-Control-List (ACL)-Specific Request Headers -// -// You also can use the following access control–related headers with this -// operation. By default, all objects are private. Only the owner has full access -// control. When adding a new object, you can grant permissions to individual -// AWS accounts or to predefined groups defined by Amazon S3. These permissions -// are then added to the Access Control List (ACL) on the object. For more information, -// see Using ACLs (https://docs.aws.amazon.com/AmazonS3/latest/dev/S3_ACLs_UsingACLs.html). -// With this operation, you can grant access permissions using one of the following -// two methods: -// -// * Specify a canned ACL (x-amz-acl) — Amazon S3 supports a set of predefined -// ACLs, known as canned ACLs. Each canned ACL has a predefined set of grantees -// and permissions. For more information, see Canned ACL (https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#CannedACL). -// -// * Specify access permissions explicitly — To explicitly grant access -// permissions to specific AWS accounts or groups, use the following headers. -// Each header maps to specific permissions that Amazon S3 supports in an -// ACL. For more information, see Access Control List (ACL) Overview (https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html). -// In the header, you specify a list of grantees who get the specific permission. -// To grant permissions explicitly use: x-amz-grant-read x-amz-grant-write -// x-amz-grant-read-acp x-amz-grant-write-acp x-amz-grant-full-control You -// specify each grantee as a type=value pair, where the type is one of the -// following: emailAddress – if the value specified is the email address -// of an AWS account Using email addresses to specify a grantee is only supported -// in the following AWS Regions: US East (N. Virginia) US West (N. California) -// US West (Oregon) Asia Pacific (Singapore) Asia Pacific (Sydney) Asia Pacific -// (Tokyo) EU (Ireland) South America (São Paulo) For a list of all the -// Amazon S3 supported Regions and endpoints, see Regions and Endpoints (https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region) -// in the AWS General Reference id – if the value specified is the canonical -// user ID of an AWS account uri – if you are granting permissions to a -// predefined group For example, the following x-amz-grant-read header grants -// the AWS accounts identified by email addresses permissions to read object -// data and its metadata: x-amz-grant-read: emailAddress="xyz@amazon.com", -// emailAddress="abc@amazon.com" -// -// Server-Side- Encryption-Specific Request Headers -// -// You can optionally tell Amazon S3 to encrypt data at rest using server-side -// encryption. Server-side encryption is for data encryption at rest. Amazon -// S3 encrypts your data as it writes it to disks in its data centers and decrypts -// it when you access it. The option you use depends on whether you want to -// use AWS-managed encryption keys or provide your own encryption key. -// -// * Use encryption keys managed by Amazon S3 or customer master keys (CMKs) -// stored in AWS Key Management Service (AWS KMS) – If you want AWS to -// manage the keys used to encrypt data, specify the following headers in -// the request. x-amz-server-side​-encryption x-amz-server-side-encryption-aws-kms-key-id -// x-amz-server-side-encryption-context If you specify x-amz-server-side-encryption:aws:kms, -// but don't provide x-amz-server-side-encryption-aws-kms-key-id, Amazon -// S3 uses the AWS managed CMK in AWS KMS to protect the data. If you want -// to use a customer managed AWS KMS CMK, you must provide the x-amz-server-side-encryption-aws-kms-key-id -// of the symmetric customer managed CMK. Amazon S3 only supports symmetric -// CMKs and not asymmetric CMKs. For more information, see Using Symmetric -// and Asymmetric Keys (https://docs.aws.amazon.com/kms/latest/developerguide/symmetric-asymmetric.html) -// in the AWS Key Management Service Developer Guide. All GET and PUT requests -// for an object protected by AWS KMS fail if you don't make them with SSL -// or by using SigV4. For more information about server-side encryption with -// CMKs stored in AWS KMS (SSE-KMS), see Protecting Data Using Server-Side -// Encryption with CMKs stored in AWS KMS (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html). -// -// * Use customer-provided encryption keys – If you want to manage your -// own encryption keys, provide all the following headers in the request. -// If you use this feature, the ETag value that Amazon S3 returns in the -// response is not the MD5 of the object. x-amz-server-side​-encryption​-customer-algorithm -// x-amz-server-side​-encryption​-customer-key x-amz-server-side​-encryption​-customer-key-MD5 -// For more information about server-side encryption with CMKs stored in -// AWS KMS (SSE-KMS), see Protecting Data Using Server-Side Encryption with -// CMKs stored in AWS KMS (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html). +// You can use headers to grant ACL- based permissions. By default, all objects +// are private. Only the owner has full access control. When adding a new object, +// you can grant permissions to individual AWS accounts or to predefined groups +// defined by Amazon S3. These permissions are then added to the ACL on the +// object. For more information, see Access Control List (ACL) Overview (https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html) +// and Managing ACLs Using the REST API (https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-using-rest-api.html). // // Storage Class Options // -// By default, Amazon S3 uses the Standard storage class to store newly created -// objects. The Standard storage class provides high durability and high availability. -// You can specify other storage classes depending on the performance needs. +// By default, Amazon S3 uses the STANDARD storage class to store newly created +// objects. The STANDARD storage class provides high durability and high availability. +// Depending on performance needs, you can specify a different storage class. // For more information, see Storage Classes (https://docs.aws.amazon.com/AmazonS3/latest/dev/storage-class-intro.html) -// in the Amazon Simple Storage Service Developer Guide. +// in the Amazon S3 Service Developer Guide. // // Versioning // // If you enable versioning for a bucket, Amazon S3 automatically generates // a unique version ID for the object being stored. Amazon S3 returns this ID -// in the response using the x-amz-version-id response header. If versioning -// is suspended, Amazon S3 always uses null as the version ID for the object -// stored. For more information about returning the versioning state of a bucket, -// see GetBucketVersioning. If you enable versioning for a bucket, when Amazon -// S3 receives multiple write requests for the same object simultaneously, it -// stores all of the objects. +// in the response. When you enable versioning for a bucket, if Amazon S3 receives +// multiple write requests for the same object simultaneously, it stores all +// of the objects. +// +// For more information about versioning, see Adding Objects to Versioning Enabled +// Buckets (https://docs.aws.amazon.com/AmazonS3/latest/dev/AddingObjectstoVersioningEnabledBuckets.html). +// For information about returning the versioning state of a bucket, see GetBucketVersioning. // // Related Resources // @@ -8639,6 +8578,10 @@ func (c *S3) PutObjectAclRequest(input *PutObjectAclInput) (req *request.Request output = &PutObjectAclOutput{} req = c.newRequest(op, input, output) + req.Handlers.Build.PushBackNamed(request.NamedHandler{ + Name: "contentMd5Handler", + Fn: checksum.AddBodyContentMD5Handler, + }) return } @@ -8651,7 +8594,9 @@ func (c *S3) PutObjectAclRequest(input *PutObjectAclInput) (req *request.Request // Depending on your application needs, you can choose to set the ACL on an // object using either the request body or the headers. For example, if you // have an existing application that updates a bucket ACL using the request -// body, you can continue to use that approach. +// body, you can continue to use that approach. For more information, see Access +// Control List (ACL) Overview (https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html) +// in the Amazon S3 Developer Guide. // // Access Permissions // @@ -8673,12 +8618,19 @@ func (c *S3) PutObjectAclRequest(input *PutObjectAclInput) (req *request.Request // S3 supports in an ACL. For more information, see Access Control List (ACL) // Overview (https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html). // You specify each grantee as a type=value pair, where the type is one of -// the following: emailAddress – if the value specified is the email address -// of an AWS account id – if the value specified is the canonical user -// ID of an AWS account uri – if you are granting permissions to a predefined -// group For example, the following x-amz-grant-read header grants list objects -// permission to the two AWS accounts identified by their email addresses. -// x-amz-grant-read: emailAddress="xyz@amazon.com", emailAddress="abc@amazon.com" +// the following: id – if the value specified is the canonical user ID +// of an AWS account uri – if you are granting permissions to a predefined +// group emailAddress – if the value specified is the email address of +// an AWS account Using email addresses to specify a grantee is only supported +// in the following AWS Regions: US East (N. Virginia) US West (N. California) +// US West (Oregon) Asia Pacific (Singapore) Asia Pacific (Sydney) Asia Pacific +// (Tokyo) Europe (Ireland) South America (São Paulo) For a list of all +// the Amazon S3 supported Regions and endpoints, see Regions and Endpoints +// (https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region) in +// the AWS General Reference. For example, the following x-amz-grant-read +// header grants list objects permission to the two AWS accounts identified +// by their email addresses. x-amz-grant-read: emailAddress="xyz@amazon.com", +// emailAddress="abc@amazon.com" // // You can use either a canned ACL or specify access permissions explicitly. // You cannot do both. @@ -8688,11 +8640,6 @@ func (c *S3) PutObjectAclRequest(input *PutObjectAclInput) (req *request.Request // You can specify the person (grantee) to whom you're assigning access rights // (using request elements) in the following ways: // -// * By Email address: <>Grantees@email.com<>lt;/Grantee> -// The grantee is resolved to the CanonicalUser and, in a response to a GET -// Object acl request, appears as the CanonicalUser. -// // * By the person's ID: <>ID<><>GranteesEmail<> // DisplayName is optional and ignored in the request. @@ -8700,6 +8647,17 @@ func (c *S3) PutObjectAclRequest(input *PutObjectAclInput) (req *request.Request // * By URI: <>http://acs.amazonaws.com/groups/global/AuthenticatedUsers<> // +// * By Email address: <>Grantees@email.com<>lt;/Grantee> +// The grantee is resolved to the CanonicalUser and, in a response to a GET +// Object acl request, appears as the CanonicalUser. Using email addresses +// to specify a grantee is only supported in the following AWS Regions: US +// East (N. Virginia) US West (N. California) US West (Oregon) Asia Pacific +// (Singapore) Asia Pacific (Sydney) Asia Pacific (Tokyo) Europe (Ireland) +// South America (São Paulo) For a list of all the Amazon S3 supported Regions +// and endpoints, see Regions and Endpoints (https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region) +// in the AWS General Reference. +// // Versioning // // The ACL of an object is set at the object version level. By default, PUT @@ -8784,6 +8742,10 @@ func (c *S3) PutObjectLegalHoldRequest(input *PutObjectLegalHoldInput) (req *req output = &PutObjectLegalHoldOutput{} req = c.newRequest(op, input, output) + req.Handlers.Build.PushBackNamed(request.NamedHandler{ + Name: "contentMd5Handler", + Fn: checksum.AddBodyContentMD5Handler, + }) return } @@ -8862,6 +8824,10 @@ func (c *S3) PutObjectLockConfigurationRequest(input *PutObjectLockConfiguration output = &PutObjectLockConfigurationOutput{} req = c.newRequest(op, input, output) + req.Handlers.Build.PushBackNamed(request.NamedHandler{ + Name: "contentMd5Handler", + Fn: checksum.AddBodyContentMD5Handler, + }) return } @@ -8945,6 +8911,10 @@ func (c *S3) PutObjectRetentionRequest(input *PutObjectRetentionInput) (req *req output = &PutObjectRetentionOutput{} req = c.newRequest(op, input, output) + req.Handlers.Build.PushBackNamed(request.NamedHandler{ + Name: "contentMd5Handler", + Fn: checksum.AddBodyContentMD5Handler, + }) return } @@ -9023,12 +8993,16 @@ func (c *S3) PutObjectTaggingRequest(input *PutObjectTaggingInput) (req *request output = &PutObjectTaggingOutput{} req = c.newRequest(op, input, output) + req.Handlers.Build.PushBackNamed(request.NamedHandler{ + Name: "contentMd5Handler", + Fn: checksum.AddBodyContentMD5Handler, + }) return } // PutObjectTagging API operation for Amazon Simple Storage Service. // -// Sets the supplied tag-set to an object that already exists in a bucket +// Sets the supplied tag-set to an object that already exists in a bucket. // // A tag is a key-value pair. You can associate tags with an object by sending // a PUT request against the tagging subresource that is associated with the @@ -9135,6 +9109,10 @@ func (c *S3) PutPublicAccessBlockRequest(input *PutPublicAccessBlockInput) (req output = &PutPublicAccessBlockOutput{} req = c.newRequest(op, input, output) req.Handlers.Unmarshal.Swap(restxml.UnmarshalHandler.Name, protocol.UnmarshalDiscardBodyHandler) + req.Handlers.Build.PushBackNamed(request.NamedHandler{ + Name: "contentMd5Handler", + Fn: checksum.AddBodyContentMD5Handler, + }) return } @@ -9246,9 +9224,9 @@ func (c *S3) RestoreObjectRequest(input *RestoreObjectInput) (req *request.Reque // * restore an archive - Restore an archived object // // To use this operation, you must have permissions to perform the s3:RestoreObject -// and s3:GetObject actions. The bucket owner has this permission by default -// and can grant this permission to others. For more information about permissions, -// see Permissions Related to Bucket Subresource Operations (https://docs.aws.amazon.com/AmazonS3/latest/dev/using-with-s3-actions.html#using-with-s3-actions-related-to-bucket-subresources) +// action. The bucket owner has this permission by default and can grant this +// permission to others. For more information about permissions, see Permissions +// Related to Bucket Subresource Operations (https://docs.aws.amazon.com/AmazonS3/latest/dev/using-with-s3-actions.html#using-with-s3-actions-related-to-bucket-subresources) // and Managing Access Permissions to Your Amazon S3 Resources (https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-access-control.html) // in the Amazon Simple Storage Service Developer Guide. // @@ -9290,8 +9268,8 @@ func (c *S3) RestoreObjectRequest(input *RestoreObjectInput) (req *request.Reque // the query.) You cannot mix ordinal positions with header column names. // SELECT s.Id, s.FirstName, s.SSN FROM S3Object s // -// For more information about using SQL with Glacier Select restore, see SQL -// Reference for Amazon S3 Select and Glacier Select (https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-glacier-select-sql-reference.html) +// For more information about using SQL with S3 Glacier Select restore, see +// SQL Reference for Amazon S3 Select and S3 Glacier Select (https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-glacier-select-sql-reference.html) // in the Amazon Simple Storage Service Developer Guide. // // When making a select request, you can also do the following: @@ -9344,12 +9322,12 @@ func (c *S3) RestoreObjectRequest(input *RestoreObjectInput) (req *request.Reque // retrievals and provisioned capacity are not available for the DEEP_ARCHIVE // storage class. // -// * Standard - Standard retrievals allow you to access any of your archived +// * Standard - S3 Standard retrievals allow you to access any of your archived // objects within several hours. This is the default option for the GLACIER // and DEEP_ARCHIVE retrieval requests that do not specify the retrieval -// option. Standard retrievals typically complete within 3-5 hours from the -// GLACIER storage class and typically complete within 12 hours from the -// DEEP_ARCHIVE storage class. +// option. S3 Standard retrievals typically complete within 3-5 hours from +// the GLACIER storage class and typically complete within 12 hours from +// the DEEP_ARCHIVE storage class. // // * Bulk - Bulk retrievals are Amazon S3 Glacier’s lowest-cost retrieval // option, enabling you to retrieve large amounts, even petabytes, of data @@ -9408,10 +9386,10 @@ func (c *S3) RestoreObjectRequest(input *RestoreObjectInput) (req *request.Reque // (This error does not apply to SELECT type requests.) HTTP Status Code: // 409 Conflict SOAP Fault Code Prefix: Client // -// * Code: GlacierExpeditedRetrievalNotAvailable Cause: Glacier expedited +// * Code: GlacierExpeditedRetrievalNotAvailable Cause: S3 Glacier expedited // retrievals are currently not available. Try again later. (Returned if // there is insufficient capacity to process the Expedited request. This -// error applies only to Expedited retrievals and not to Standard or Bulk +// error applies only to Expedited retrievals and not to S3 Standard or Bulk // retrievals.) HTTP Status Code: 503 SOAP Fault Code Prefix: N/A // // Related Resources @@ -9420,7 +9398,7 @@ func (c *S3) RestoreObjectRequest(input *RestoreObjectInput) (req *request.Reque // // * GetBucketNotificationConfiguration // -// * SQL Reference for Amazon S3 Select and Glacier Select (https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-glacier-select-sql-reference.html) +// * SQL Reference for Amazon S3 Select and S3 Glacier Select (https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-glacier-select-sql-reference.html) // in the Amazon Simple Storage Service Developer Guide // // Returns awserr.Error for service API and SDK errors. Use runtime type assertions @@ -9522,7 +9500,7 @@ func (c *S3) SelectObjectContentRequest(input *SelectObjectContentInput) (req *r // in the Amazon Simple Storage Service Developer Guide. // // For more information about using SQL with Amazon S3 Select, see SQL Reference -// for Amazon S3 Select and Glacier Select (https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-glacier-select-sql-reference.html) +// for Amazon S3 Select and S3 Glacier Select (https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-glacier-select-sql-reference.html) // in the Amazon Simple Storage Service Developer Guide. // // Permissions @@ -9572,8 +9550,8 @@ func (c *S3) SelectObjectContentRequest(input *SelectObjectContentInput) (req *r // The SelectObjectContent operation does not support the following GetObject // functionality. For more information, see GetObject. // -// * Range: While you can specify a scan range for a Amazon S3 Select request, -// see SelectObjectContentRequest$ScanRange in the request parameters below, +// * Range: Although you can specify a scan range for an Amazon S3 Select +// request (see SelectObjectContentRequest$ScanRange in the request parameters), // you cannot specify the range of bytes of an object to return. // // * GLACIER, DEEP_ARCHIVE and REDUCED_REDUNDANCY storage classes: You cannot @@ -9583,8 +9561,7 @@ func (c *S3) SelectObjectContentRequest(input *SelectObjectContentInput) (req *r // // Special Errors // -// For a list of special errors for this operation and for general information -// about Amazon S3 errors and a list of error codes, see ErrorResponses +// For a list of special errors for this operation, see SelectObjectContentErrorCodeList // // Related Resources // @@ -10615,8 +10592,11 @@ type AnalyticsS3BucketDestination struct { // Bucket is a required field Bucket *string `type:"string" required:"true"` - // The account ID that owns the destination bucket. If no account ID is provided, - // the owner will not be validated prior to exporting data. + // The account ID that owns the destination S3 bucket. If no account ID is provided, + // the owner is not validated before exporting data. + // + // Although this value is optional, we strongly recommend that you set it to + // help prevent problems if the destination bucket ownership changes. BucketAccountId *string `type:"string"` // Specifies the file format used when exporting data to Amazon S3. @@ -14575,9 +14555,9 @@ type Destination struct { // must be replicated. Must be specified together with a Metrics block. ReplicationTime *ReplicationTime `type:"structure"` - // The storage class to use when replicating objects, such as standard or reduced - // redundancy. By default, Amazon S3 uses the storage class of the source object - // to create the object replica. + // The storage class to use when replicating objects, such as S3 Standard or + // reduced redundancy. By default, Amazon S3 uses the storage class of the source + // object to create the object replica. // // For valid values, see the StorageClass element of the PUT Bucket replication // (https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTreplication.html) @@ -16110,6 +16090,7 @@ type GetBucketLocationOutput struct { // Specifies the Region where the bucket resides. For a list of all the Amazon // S3 supported location constraints by Region, see Regions and Endpoints (https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region). + // Buckets in Region us-east-1 have a LocationConstraint of null. LocationConstraint *string `type:"string" enum:"BucketLocationConstraint"` } @@ -16319,7 +16300,7 @@ func (s *GetBucketMetricsConfigurationOutput) SetMetricsConfiguration(v *Metrics type GetBucketNotificationConfigurationRequest struct { _ struct{} `locationName:"GetBucketNotificationConfigurationRequest" type:"structure"` - // Name of the bucket for which to get the notification configuration + // Name of the bucket for which to get the notification configuration. // // Bucket is a required field Bucket *string `location:"uri" locationName:"Bucket" type:"string" required:"true"` @@ -16967,10 +16948,10 @@ func (s *GetBucketWebsiteInput) hasEndpointARN() bool { type GetBucketWebsiteOutput struct { _ struct{} `type:"structure"` - // The name of the error document for the website. + // The object key name of the website error document to use for 4XX class errors. ErrorDocument *ErrorDocument `type:"structure"` - // The name of the index document for the website. + // The name of the index document for the website (for example index.html). IndexDocument *IndexDocument `type:"structure"` // Specifies the redirect behavior of all requests to a website endpoint of @@ -17207,7 +17188,10 @@ type GetObjectInput struct { PartNumber *int64 `location:"querystring" locationName:"partNumber" type:"integer"` // Downloads the specified range bytes of an object. For more information about - // the HTTP Range header, see http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35. + // the HTTP Range header, see https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35 + // (https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35). + // + // Amazon S3 doesn't support retrieving multiple ranges of data per GET request. Range *string `location:"header" locationName:"Range" type:"string"` // Confirms that the requester knows that they will be charged for the request. @@ -17756,7 +17740,7 @@ type GetObjectOutput struct { ServerSideEncryption *string `location:"header" locationName:"x-amz-server-side-encryption" type:"string" enum:"ServerSideEncryption"` // Provides storage class information of the object. Amazon S3 returns this - // header for all objects except for Standard storage class objects. + // header for all objects except for S3 Standard storage class objects. StorageClass *string `location:"header" locationName:"x-amz-storage-class" type:"string" enum:"StorageClass"` // The number of tags, if any, on the object. @@ -18441,11 +18425,11 @@ func (s *GetPublicAccessBlockOutput) SetPublicAccessBlockConfiguration(v *Public return s } -// Container for Glacier job parameters. +// Container for S3 Glacier job parameters. type GlacierJobParameters struct { _ struct{} `type:"structure"` - // Glacier retrieval tier at which the restore will be processed. + // S3 Glacier retrieval tier at which the restore will be processed. // // Tier is a required field Tier *string `type:"string" required:"true" enum:"Tier"` @@ -18536,6 +18520,29 @@ type Grantee struct { DisplayName *string `type:"string"` // Email address of the grantee. + // + // Using email addresses to specify a grantee is only supported in the following + // AWS Regions: + // + // * US East (N. Virginia) + // + // * US West (N. California) + // + // * US West (Oregon) + // + // * Asia Pacific (Singapore) + // + // * Asia Pacific (Sydney) + // + // * Asia Pacific (Tokyo) + // + // * Europe (Ireland) + // + // * South America (São Paulo) + // + // For a list of all the Amazon S3 supported Regions and endpoints, see Regions + // and Endpoints (https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region) + // in the AWS General Reference. EmailAddress *string `type:"string"` // The canonical user ID of the grantee. @@ -18716,6 +18723,8 @@ type HeadObjectInput struct { // Downloads the specified range bytes of an object. For more information about // the HTTP Range header, see http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35. + // + // Amazon S3 doesn't support retrieving multiple ranges of data per GET request. Range *string `location:"header" locationName:"Range" type:"string"` // Confirms that the requester knows that they will be charged for the request. @@ -19029,7 +19038,7 @@ type HeadObjectOutput struct { ServerSideEncryption *string `location:"header" locationName:"x-amz-server-side-encryption" type:"string" enum:"ServerSideEncryption"` // Provides storage class information of the object. Amazon S3 returns this - // header for all objects except for Standard storage class objects. + // header for all objects except for S3 Standard storage class objects. // // For more information, see Storage Classes (https://docs.aws.amazon.com/AmazonS3/latest/dev/storage-class-intro.html). StorageClass *string `location:"header" locationName:"x-amz-storage-class" type:"string" enum:"StorageClass"` @@ -19624,7 +19633,11 @@ func (s *InventoryFilter) SetPrefix(v string) *InventoryFilter { type InventoryS3BucketDestination struct { _ struct{} `type:"structure"` - // The ID of the account that owns the destination bucket. + // The account ID that owns the destination S3 bucket. If no account ID is provided, + // the owner is not validated before exporting data. + // + // Although this value is optional, we strongly recommend that you set it to + // help prevent problems if the destination bucket ownership changes. AccountId *string `type:"string"` // The Amazon Resource Name (ARN) of the bucket where inventory results will @@ -19781,7 +19794,8 @@ func (s *JSONInput) SetType(v string) *JSONInput { type JSONOutput struct { _ struct{} `type:"structure"` - // The value used to separate individual records in the output. + // The value used to separate individual records in the output. If no value + // is specified, Amazon S3 uses a newline character ('\n'). RecordDelimiter *string `type:"string"` } @@ -21017,11 +21031,12 @@ type ListObjectVersionsInput struct { // Specifies the key to start with when listing objects in a bucket. KeyMarker *string `location:"querystring" locationName:"key-marker" type:"string"` - // Sets the maximum number of keys returned in the response. The response might - // contain fewer keys but will never contain more. If additional keys satisfy - // the search criteria, but were not returned because max-keys was exceeded, - // the response contains true. To return the additional - // keys, see key-marker and version-id-marker. + // Sets the maximum number of keys returned in the response. By default the + // API returns up to 1,000 key names. The response might contain fewer keys + // but will never contain more. If additional keys satisfy the search criteria, + // but were not returned because max-keys was exceeded, the response contains + // true. To return the additional keys, see key-marker + // and version-id-marker. MaxKeys *int64 `location:"querystring" locationName:"max-keys" type:"integer"` // Use this parameter to select only those keys that begin with the specified @@ -21298,8 +21313,9 @@ type ListObjectsInput struct { // Specifies the key to start with when listing objects in a bucket. Marker *string `location:"querystring" locationName:"marker" type:"string"` - // Sets the maximum number of keys returned in the response. The response might - // contain fewer keys but will never contain more. + // Sets the maximum number of keys returned in the response. By default the + // API returns up to 1,000 key names. The response might contain fewer keys + // but will never contain more. MaxKeys *int64 `location:"querystring" locationName:"max-keys" type:"integer"` // Limits the response to keys that begin with the specified prefix. @@ -21561,8 +21577,9 @@ type ListObjectsV2Input struct { // true. FetchOwner *bool `location:"querystring" locationName:"fetch-owner" type:"boolean"` - // Sets the maximum number of keys returned in the response. The response might - // contain fewer keys but will never contain more. + // Sets the maximum number of keys returned in the response. By default the + // API returns up to 1,000 key names. The response might contain fewer keys + // but will never contain more. MaxKeys *int64 `location:"querystring" locationName:"max-keys" type:"integer"` // Limits the response to keys that begin with the specified prefix. @@ -21731,8 +21748,9 @@ type ListObjectsV2Output struct { // result will include less than equals 50 keys KeyCount *int64 `type:"integer"` - // Sets the maximum number of keys returned in the response. The response might - // contain fewer keys but will never contain more. + // Sets the maximum number of keys returned in the response. By default the + // API returns up to 1,000 key names. The response might contain fewer keys + // but will never contain more. MaxKeys *int64 `type:"integer"` // Bucket name. @@ -23976,7 +23994,7 @@ type PutBucketCorsInput struct { // Describes the cross-origin access configuration for objects in an Amazon // S3 bucket. For more information, see Enabling Cross-Origin Resource Sharing - // (https://docs.aws.amazon.com/AmazonS3/latest/dev//cors.html) in the Amazon + // (https://docs.aws.amazon.com/AmazonS3/latest/dev/cors.html) in the Amazon // Simple Storage Service Developer Guide. // // CORSConfiguration is a required field @@ -25767,8 +25785,8 @@ type PutObjectInput struct { // S3 (for example, AES256, aws:kms). ServerSideEncryption *string `location:"header" locationName:"x-amz-server-side-encryption" type:"string" enum:"ServerSideEncryption"` - // If you don't specify, Standard is the default storage class. Amazon S3 supports - // other storage classes. + // If you don't specify, S3 Standard is the default storage class. Amazon S3 + // supports other storage classes. StorageClass *string `location:"header" locationName:"x-amz-storage-class" type:"string" enum:"StorageClass"` // The tag-set for the object. The tag-set must be encoded as URL Query parameters. @@ -27784,7 +27802,7 @@ type RestoreRequest struct { // The optional description for the job. Description *string `type:"string"` - // Glacier related parameters pertaining to this job. Do not use with restores + // S3 Glacier related parameters pertaining to this job. Do not use with restores // that specify OutputLocation. GlacierJobParameters *GlacierJobParameters `type:"structure"` @@ -27794,7 +27812,7 @@ type RestoreRequest struct { // Describes the parameters for Select job types. SelectParameters *SelectParameters `type:"structure"` - // Glacier retrieval tier at which the restore will be processed. + // S3 Glacier retrieval tier at which the restore will be processed. Tier *string `type:"string" enum:"Tier"` // Type of restore request. @@ -27932,8 +27950,9 @@ func (s *RoutingRule) SetRedirect(v *Redirect) *RoutingRule { } // Specifies lifecycle rules for an Amazon S3 bucket. For more information, -// see PUT Bucket lifecycle (https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTlifecycle.html) -// in the Amazon Simple Storage Service API Reference. +// see Put Bucket Lifecycle Configuration (https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTlifecycle.html) +// in the Amazon Simple Storage Service API Reference. For examples, see Put +// Bucket Lifecycle Configuration Examples (https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketLifecycleConfiguration.html#API_PutBucketLifecycleConfiguration_Examples) type Rule struct { _ struct{} `type:"structure"` @@ -27978,7 +27997,10 @@ type Rule struct { // Status is a required field Status *string `type:"string" required:"true" enum:"ExpirationStatus"` - // Specifies when an object transitions to a specified storage class. + // Specifies when an object transitions to a specified storage class. For more + // information about Amazon S3 lifecycle configuration rules, see Transitioning + // Objects Using Amazon S3 Lifecycle (https://docs.aws.amazon.com/AmazonS3/latest/dev/lifecycle-transition-general-considerations.html) + // in the Amazon Simple Storage Service Developer Guide. Transition *Transition `type:"structure"` } @@ -28623,8 +28645,24 @@ func (s *SelectParameters) SetOutputSerialization(v *OutputSerialization) *Selec type ServerSideEncryptionByDefault struct { _ struct{} `type:"structure"` - // KMS master key ID to use for the default encryption. This parameter is allowed - // if and only if SSEAlgorithm is set to aws:kms. + // AWS Key Management Service (KMS) customer master key ID to use for the default + // encryption. This parameter is allowed if and only if SSEAlgorithm is set + // to aws:kms. + // + // You can specify the key ID or the Amazon Resource Name (ARN) of the CMK. + // However, if you are using encryption with cross-account operations, you must + // use a fully qualified CMK ARN. For more information, see Using encryption + // for cross-account operations (https://docs.aws.amazon.com/AmazonS3/latest/dev/bucket-encryption.html#bucket-encryption-update-bucket-policy). + // + // For example: + // + // * Key ID: 1234abcd-12ab-34cd-56ef-1234567890ab + // + // * Key ARN: arn:aws:kms:us-east-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab + // + // Amazon S3 only supports symmetric CMKs and not asymmetric CMKs. For more + // information, see Using Symmetric and Asymmetric Keys (https://docs.aws.amazon.com/kms/latest/developerguide/symmetric-asymmetric.html) + // in the AWS Key Management Service Developer Guide. KMSMasterKeyID *string `type:"string" sensitive:"true"` // Server-side encryption algorithm to use for the default encryption. @@ -29329,7 +29367,10 @@ func (s *TopicConfigurationDeprecated) SetTopic(v string) *TopicConfigurationDep return s } -// Specifies when an object transitions to a specified storage class. +// Specifies when an object transitions to a specified storage class. For more +// information about Amazon S3 lifecycle configuration rules, see Transitioning +// Objects Using Amazon S3 Lifecycle (https://docs.aws.amazon.com/AmazonS3/latest/dev/lifecycle-transition-general-considerations.html) +// in the Amazon Simple Storage Service Developer Guide. type Transition struct { _ struct{} `type:"structure"` diff --git a/vendor/github.com/aws/aws-sdk-go/service/s3/body_hash.go b/vendor/github.com/aws/aws-sdk-go/service/s3/body_hash.go index 5c8ce5cc8..407f06b6e 100644 --- a/vendor/github.com/aws/aws-sdk-go/service/s3/body_hash.go +++ b/vendor/github.com/aws/aws-sdk-go/service/s3/body_hash.go @@ -13,7 +13,6 @@ import ( "github.com/aws/aws-sdk-go/aws" "github.com/aws/aws-sdk-go/aws/awserr" "github.com/aws/aws-sdk-go/aws/request" - "github.com/aws/aws-sdk-go/internal/sdkio" ) const ( @@ -25,30 +24,6 @@ const ( appendMD5TxEncoding = "append-md5" ) -// contentMD5 computes and sets the HTTP Content-MD5 header for requests that -// require it. -func contentMD5(r *request.Request) { - h := md5.New() - - if !aws.IsReaderSeekable(r.Body) { - if r.Config.Logger != nil { - r.Config.Logger.Log(fmt.Sprintf( - "Unable to compute Content-MD5 for unseekable body, S3.%s", - r.Operation.Name)) - } - return - } - - if _, err := copySeekableBody(h, r.Body); err != nil { - r.Error = awserr.New("ContentMD5", "failed to compute body MD5", err) - return - } - - // encode the md5 checksum in base64 and set the request header. - v := base64.StdEncoding.EncodeToString(h.Sum(nil)) - r.HTTPRequest.Header.Set(contentMD5Header, v) -} - // computeBodyHashes will add Content MD5 and Content Sha256 hashes to the // request. If the body is not seekable or S3DisableContentMD5Validation set // this handler will be ignored. @@ -90,7 +65,7 @@ func computeBodyHashes(r *request.Request) { dst = io.MultiWriter(hashers...) } - if _, err := copySeekableBody(dst, r.Body); err != nil { + if _, err := aws.CopySeekableBody(dst, r.Body); err != nil { r.Error = awserr.New("BodyHashError", "failed to compute body hashes", err) return } @@ -119,28 +94,6 @@ const ( sha256HexEncLen = sha256.Size * 2 // hex.EncodedLen ) -func copySeekableBody(dst io.Writer, src io.ReadSeeker) (int64, error) { - curPos, err := src.Seek(0, sdkio.SeekCurrent) - if err != nil { - return 0, err - } - - // hash the body. seek back to the first position after reading to reset - // the body for transmission. copy errors may be assumed to be from the - // body. - n, err := io.Copy(dst, src) - if err != nil { - return n, err - } - - _, err = src.Seek(curPos, sdkio.SeekStart) - if err != nil { - return n, err - } - - return n, nil -} - // Adds the x-amz-te: append_md5 header to the request. This requests the service // responds with a trailing MD5 checksum. // diff --git a/vendor/github.com/aws/aws-sdk-go/service/s3/customizations.go b/vendor/github.com/aws/aws-sdk-go/service/s3/customizations.go index 036d0b2e0..a7698d5eb 100644 --- a/vendor/github.com/aws/aws-sdk-go/service/s3/customizations.go +++ b/vendor/github.com/aws/aws-sdk-go/service/s3/customizations.go @@ -33,12 +33,6 @@ func defaultInitRequestFn(r *request.Request) { platformRequestHandlers(r) switch r.Operation.Name { - case opPutBucketCors, opPutBucketLifecycle, opPutBucketPolicy, - opPutBucketTagging, opDeleteObjects, opPutBucketLifecycleConfiguration, - opPutObjectLegalHold, opPutObjectRetention, opPutObjectLockConfiguration, - opPutBucketReplication: - // These S3 operations require Content-MD5 to be set - r.Handlers.Build.PushBack(contentMD5) case opGetBucketLocation: // GetBucketLocation has custom parsing logic r.Handlers.Unmarshal.PushFront(buildGetBucketLocation) diff --git a/vendor/github.com/aws/aws-sdk-go/service/s3/statusok_error.go b/vendor/github.com/aws/aws-sdk-go/service/s3/statusok_error.go index f6a69aed1..247770e4c 100644 --- a/vendor/github.com/aws/aws-sdk-go/service/s3/statusok_error.go +++ b/vendor/github.com/aws/aws-sdk-go/service/s3/statusok_error.go @@ -2,6 +2,7 @@ package s3 import ( "bytes" + "io" "io/ioutil" "net/http" @@ -24,17 +25,18 @@ func copyMultipartStatusOKUnmarhsalError(r *request.Request) { r.HTTPResponse.Body = ioutil.NopCloser(body) defer body.Seek(0, sdkio.SeekStart) - if body.Len() == 0 { - // If there is no body don't attempt to parse the body. - return - } - unmarshalError(r) if err, ok := r.Error.(awserr.Error); ok && err != nil { - if err.Code() == request.ErrCodeSerialization { + if err.Code() == request.ErrCodeSerialization && + err.OrigErr() != io.EOF { r.Error = nil return } - r.HTTPResponse.StatusCode = http.StatusServiceUnavailable + // if empty payload + if err.OrigErr() == io.EOF { + r.HTTPResponse.StatusCode = http.StatusInternalServerError + } else { + r.HTTPResponse.StatusCode = http.StatusServiceUnavailable + } } } diff --git a/vendor/github.com/aws/aws-sdk-go/service/s3/unmarshal_error.go b/vendor/github.com/aws/aws-sdk-go/service/s3/unmarshal_error.go index 5b63fac72..6eecf6691 100644 --- a/vendor/github.com/aws/aws-sdk-go/service/s3/unmarshal_error.go +++ b/vendor/github.com/aws/aws-sdk-go/service/s3/unmarshal_error.go @@ -1,6 +1,7 @@ package s3 import ( + "bytes" "encoding/xml" "fmt" "io" @@ -45,17 +46,24 @@ func unmarshalError(r *request.Request) { // Attempt to parse error from body if it is known var errResp xmlErrorResponse - err := xmlutil.UnmarshalXMLError(&errResp, r.HTTPResponse.Body) - if err == io.EOF { - // Only capture the error if an unmarshal error occurs that is not EOF, - // because S3 might send an error without a error message which causes - // the XML unmarshal to fail with EOF. - err = nil + var err error + if r.HTTPResponse.StatusCode >= 200 && r.HTTPResponse.StatusCode < 300 { + err = s3unmarshalXMLError(&errResp, r.HTTPResponse.Body) + } else { + err = xmlutil.UnmarshalXMLError(&errResp, r.HTTPResponse.Body) } + if err != nil { + var errorMsg string + if err == io.EOF { + errorMsg = "empty response payload" + } else { + errorMsg = "failed to unmarshal error message" + } + r.Error = awserr.NewRequestFailure( awserr.New(request.ErrCodeSerialization, - "failed to unmarshal error message", err), + errorMsg, err), r.HTTPResponse.StatusCode, r.RequestID, ) @@ -86,3 +94,21 @@ type RequestFailure interface { // Host ID is the S3 Host ID needed for debug, and contacting support HostID() string } + +// s3unmarshalXMLError is s3 specific xml error unmarshaler +// for 200 OK errors and response payloads. +// This function differs from the xmlUtil.UnmarshalXMLError +// func. It does not ignore the EOF error and passes it up. +// Related to bug fix for `s3 200 OK response with empty payload` +func s3unmarshalXMLError(v interface{}, stream io.Reader) error { + var errBuf bytes.Buffer + body := io.TeeReader(stream, &errBuf) + + err := xml.NewDecoder(body).Decode(v) + if err != nil && err != io.EOF { + return awserr.NewUnmarshalError(err, + "failed to unmarshal error message", errBuf.Bytes()) + } + + return err +} diff --git a/vendor/github.com/aws/aws-sdk-go/service/sts/api.go b/vendor/github.com/aws/aws-sdk-go/service/sts/api.go index 7f60d4aa1..550b5f687 100644 --- a/vendor/github.com/aws/aws-sdk-go/service/sts/api.go +++ b/vendor/github.com/aws/aws-sdk-go/service/sts/api.go @@ -1788,7 +1788,7 @@ type AssumeRoleWithSAMLInput struct { // in the IAM User Guide. // // SAMLAssertion is a required field - SAMLAssertion *string `min:"4" type:"string" required:"true"` + SAMLAssertion *string `min:"4" type:"string" required:"true" sensitive:"true"` } // String returns the string representation @@ -2100,7 +2100,7 @@ type AssumeRoleWithWebIdentityInput struct { // the application makes an AssumeRoleWithWebIdentity call. // // WebIdentityToken is a required field - WebIdentityToken *string `min:"4" type:"string" required:"true"` + WebIdentityToken *string `min:"4" type:"string" required:"true" sensitive:"true"` } // String returns the string representation diff --git a/vendor/github.com/hashicorp/aws-sdk-go-base/.golangci.yml b/vendor/github.com/hashicorp/aws-sdk-go-base/.golangci.yml index 59ef674ca..687b0c780 100644 --- a/vendor/github.com/hashicorp/aws-sdk-go-base/.golangci.yml +++ b/vendor/github.com/hashicorp/aws-sdk-go-base/.golangci.yml @@ -6,14 +6,21 @@ linters: disable-all: true enable: - deadcode + - dogsled - errcheck + - goconst - gofmt + - gomnd - gosimple - ineffassign + - interfacer - misspell + - scopelint - staticcheck - structcheck - unconvert + - unparam - unused + - typecheck - varcheck - vet diff --git a/vendor/github.com/hashicorp/aws-sdk-go-base/.travis.yml b/vendor/github.com/hashicorp/aws-sdk-go-base/.travis.yml deleted file mode 100644 index f1aaa658b..000000000 --- a/vendor/github.com/hashicorp/aws-sdk-go-base/.travis.yml +++ /dev/null @@ -1,20 +0,0 @@ -dist: xenial -language: go -go: -- "1.13.x" - -matrix: - fast_finish: true - allow_failures: - - go: tip - -install: -- make tools - -script: -- make lint -- go test -timeout=30s -parallel=4 -v ./... - -branches: - only: - - master diff --git a/vendor/github.com/hashicorp/aws-sdk-go-base/CHANGELOG.md b/vendor/github.com/hashicorp/aws-sdk-go-base/CHANGELOG.md index 558e7e393..fd1b523fa 100644 --- a/vendor/github.com/hashicorp/aws-sdk-go-base/CHANGELOG.md +++ b/vendor/github.com/hashicorp/aws-sdk-go-base/CHANGELOG.md @@ -1,3 +1,25 @@ +# v0.5.0 (June 4, 2020) + +BREAKING CHANGES + +* Credential ordering has changed from static, environment, shared credentials, EC2 metadata, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata) to static, environment, shared credentials, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata). #20 +* The `AWS_METADATA_TIMEOUT` environment variable no longer has any effect as we now depend on the default AWS Go SDK EC2 Metadata client timeout of one second with two retries. #20 / #44 + +ENHANCEMENTS + +* Always enable AWS shared configuration file support (no longer require `AWS_SDK_LOAD_CONFIG` environment variable) #38 +* Automatically expand `~` prefix for home directories in shared credentials filename handling #40 +* Support assume role duration, policy ARNs, tags, and transitive tag keys via configuration #39 +* Add `CannotAssumeRoleError` and `NoValidCredentialSourcesError` error types with helpers #42 + +BUG FIXES + +* Properly use custom STS endpoint during AssumeRole API calls triggered by Terraform AWS Provider and S3 Backend configurations #32 +* Properly use custom EC2 metadata endpoint during API calls triggered by fallback credentials lookup #32 +* Prefer shared configuration handling over EC2 metadata #20 +* Prefer ECS credentials over EC2 metadata #20 +* Remove hardcoded AWS Provider messaging in error messages #31 / #42 + # v0.4.0 (October 3, 2019) BUG FIXES diff --git a/vendor/github.com/hashicorp/aws-sdk-go-base/awsauth.go b/vendor/github.com/hashicorp/aws-sdk-go-base/awsauth.go index 531162038..3363b1d5b 100644 --- a/vendor/github.com/hashicorp/aws-sdk-go-base/awsauth.go +++ b/vendor/github.com/hashicorp/aws-sdk-go-base/awsauth.go @@ -13,29 +13,21 @@ import ( awsCredentials "github.com/aws/aws-sdk-go/aws/credentials" "github.com/aws/aws-sdk-go/aws/credentials/ec2rolecreds" "github.com/aws/aws-sdk-go/aws/credentials/stscreds" - "github.com/aws/aws-sdk-go/aws/defaults" "github.com/aws/aws-sdk-go/aws/ec2metadata" "github.com/aws/aws-sdk-go/aws/session" "github.com/aws/aws-sdk-go/service/iam" "github.com/aws/aws-sdk-go/service/sts" "github.com/hashicorp/go-cleanhttp" "github.com/hashicorp/go-multierror" + homedir "github.com/mitchellh/go-homedir" ) const ( - // errMsgNoValidCredentialSources error getting credentials - errMsgNoValidCredentialSources = `No valid credential sources found for AWS Provider. - Please see https://terraform.io/docs/providers/aws/index.html for more information on - providing credentials for the AWS Provider` + // Default amount of time for EC2/ECS metadata client operations. + // Keep this value low to prevent long delays in non-EC2/ECS environments. + DefaultMetadataClientTimeout = 100 * time.Millisecond ) -var ( - // ErrNoValidCredentialSources indicates that no credentials source could be found - ErrNoValidCredentialSources = errNoValidCredentialSources() -) - -func errNoValidCredentialSources() error { return errors.New(errMsgNoValidCredentialSources) } - // GetAccountIDAndPartition gets the account ID and associated partition. func GetAccountIDAndPartition(iamconn *iam.IAM, stsconn *sts.STS, authProviderName string) (string, string, error) { var accountID, partition string @@ -75,7 +67,7 @@ func GetAccountIDAndPartitionFromEC2Metadata() (string, string, error) { setOptionalEndpoint(cfg) sess, err := session.NewSession(cfg) if err != nil { - return "", "", fmt.Errorf("error creating EC2 Metadata session: %s", err) + return "", "", fmt.Errorf("error creating EC2 Metadata session: %w", err) } metadataClient := ec2metadata.New(sess) @@ -84,7 +76,7 @@ func GetAccountIDAndPartitionFromEC2Metadata() (string, string, error) { // We can end up here if there's an issue with the instance metadata service // or if we're getting credentials from AdRoll's Hologram (in which case IAMInfo will // error out). - err = fmt.Errorf("failed getting account information via EC2 Metadata IAM information: %s", err) + err = fmt.Errorf("failed getting account information via EC2 Metadata IAM information: %w", err) log.Printf("[DEBUG] %s", err) return "", "", err } @@ -107,7 +99,7 @@ func GetAccountIDAndPartitionFromIAMGetUser(iamconn *iam.IAM) (string, string, e return "", "", nil } } - err = fmt.Errorf("failed getting account information via iam:GetUser: %s", err) + err = fmt.Errorf("failed getting account information via iam:GetUser: %w", err) log.Printf("[DEBUG] %s", err) return "", "", err } @@ -130,7 +122,7 @@ func GetAccountIDAndPartitionFromIAMListRoles(iamconn *iam.IAM) (string, string, MaxItems: aws.Int64(int64(1)), }) if err != nil { - err = fmt.Errorf("failed getting account information via iam:ListRoles: %s", err) + err = fmt.Errorf("failed getting account information via iam:ListRoles: %w", err) log.Printf("[DEBUG] %s", err) return "", "", err } @@ -151,7 +143,7 @@ func GetAccountIDAndPartitionFromSTSGetCallerIdentity(stsconn *sts.STS) (string, output, err := stsconn.GetCallerIdentity(&sts.GetCallerIdentityInput{}) if err != nil { - return "", "", fmt.Errorf("error calling sts:GetCallerIdentity: %s", err) + return "", "", fmt.Errorf("error calling sts:GetCallerIdentity: %w", err) } if output == nil || output.Arn == nil { @@ -177,37 +169,30 @@ func parseAccountIDAndPartitionFromARN(inputARN string) (string, string, error) func GetCredentialsFromSession(c *Config) (*awsCredentials.Credentials, error) { log.Printf("[INFO] Attempting to use session-derived credentials") - var sess *session.Session - var err error - if c.Profile == "" { - sess, err = session.NewSession() - if err != nil { - return nil, ErrNoValidCredentialSources - } - } else { - options := &session.Options{ - Config: aws.Config{ - HTTPClient: cleanhttp.DefaultClient(), - MaxRetries: aws.Int(0), - Region: aws.String(c.Region), - }, - } - options.Profile = c.Profile - options.SharedConfigState = session.SharedConfigEnable + // Avoid setting HTTPClient here as it will prevent the ec2metadata + // client from automatically lowering the timeout to 1 second. + options := &session.Options{ + Config: aws.Config{ + EndpointResolver: c.EndpointResolver(), + MaxRetries: aws.Int(0), + Region: aws.String(c.Region), + }, + Profile: c.Profile, + SharedConfigState: session.SharedConfigEnable, + } - sess, err = session.NewSessionWithOptions(*options) - if err != nil { - if IsAWSErr(err, "NoCredentialProviders", "") { - return nil, ErrNoValidCredentialSources - } - return nil, fmt.Errorf("Error creating AWS session: %s", err) + sess, err := session.NewSessionWithOptions(*options) + if err != nil { + if IsAWSErr(err, "NoCredentialProviders", "") { + return nil, c.NewNoValidCredentialSourcesError(err) } + return nil, fmt.Errorf("Error creating AWS session: %w", err) } creds := sess.Config.Credentials cp, err := sess.Config.Credentials.Get() if err != nil { - return nil, ErrNoValidCredentialSources + return nil, c.NewNoValidCredentialSourcesError(err) } log.Printf("[INFO] Successfully derived credentials from session") @@ -216,10 +201,16 @@ func GetCredentialsFromSession(c *Config) (*awsCredentials.Credentials, error) { } // GetCredentials gets credentials from the environment, shared credentials, -// or the session (which may include a credential process). GetCredentials also -// validates the credentials and the ability to assume a role or will return an -// error if unsuccessful. +// the session (which may include a credential process), or ECS/EC2 metadata endpoints. +// GetCredentials also validates the credentials and the ability to assume a role +// or will return an error if unsuccessful. func GetCredentials(c *Config) (*awsCredentials.Credentials, error) { + sharedCredentialsFilename, err := homedir.Expand(c.CredsFilename) + + if err != nil { + return nil, fmt.Errorf("error expanding shared credentials filename: %w", err) + } + // build a chain provider, lazy-evaluated by aws-sdk providers := []awsCredentials.Provider{ &awsCredentials.StaticProvider{Value: awsCredentials.Value{ @@ -229,70 +220,11 @@ func GetCredentials(c *Config) (*awsCredentials.Credentials, error) { }}, &awsCredentials.EnvProvider{}, &awsCredentials.SharedCredentialsProvider{ - Filename: c.CredsFilename, + Filename: sharedCredentialsFilename, Profile: c.Profile, }, } - // Build isolated HTTP client to avoid issues with globally-shared settings - client := cleanhttp.DefaultClient() - - // Keep the default timeout (100ms) low as we don't want to wait in non-EC2 environments - client.Timeout = 100 * time.Millisecond - - const userTimeoutEnvVar = "AWS_METADATA_TIMEOUT" - userTimeout := os.Getenv(userTimeoutEnvVar) - if userTimeout != "" { - newTimeout, err := time.ParseDuration(userTimeout) - if err == nil { - if newTimeout.Nanoseconds() > 0 { - client.Timeout = newTimeout - } else { - log.Printf("[WARN] Non-positive value of %s (%s) is meaningless, ignoring", userTimeoutEnvVar, newTimeout.String()) - } - } else { - log.Printf("[WARN] Error converting %s to time.Duration: %s", userTimeoutEnvVar, err) - } - } - - log.Printf("[INFO] Setting AWS metadata API timeout to %s", client.Timeout.String()) - cfg := &aws.Config{ - HTTPClient: client, - } - usedEndpoint := setOptionalEndpoint(cfg) - - // Add the default AWS provider for ECS Task Roles if the relevant env variable is set - if uri := os.Getenv("AWS_CONTAINER_CREDENTIALS_RELATIVE_URI"); len(uri) > 0 { - providers = append(providers, defaults.RemoteCredProvider(*cfg, defaults.Handlers())) - log.Print("[INFO] ECS container credentials detected, RemoteCredProvider added to auth chain") - } - - if !c.SkipMetadataApiCheck { - // Real AWS should reply to a simple metadata request. - // We check it actually does to ensure something else didn't just - // happen to be listening on the same IP:Port - ec2Session, err := session.NewSession(cfg) - - if err != nil { - return nil, fmt.Errorf("error creating EC2 Metadata session: %s", err) - } - - metadataClient := ec2metadata.New(ec2Session) - if metadataClient.Available() { - providers = append(providers, &ec2rolecreds.EC2RoleProvider{ - Client: metadataClient, - }) - log.Print("[INFO] AWS EC2 instance detected via default metadata" + - " API endpoint, EC2RoleProvider added to the auth chain") - } else { - if usedEndpoint == "" { - usedEndpoint = "default location" - } - log.Printf("[INFO] Ignoring AWS metadata API endpoint at %s "+ - "as it doesn't return any instance-id", usedEndpoint) - } - } - // Validate the credentials before returning them creds := awsCredentials.NewChainCredentials(providers) cp, err := creds.Get() @@ -303,7 +235,7 @@ func GetCredentials(c *Config) (*awsCredentials.Credentials, error) { return nil, err } } else { - return nil, fmt.Errorf("Error loading credentials for AWS Provider: %s", err) + return nil, fmt.Errorf("Error loading credentials for AWS Provider: %w", err) } } else { log.Printf("[INFO] AWS Auth provider used: %q", cp.ProviderName) @@ -316,20 +248,21 @@ func GetCredentials(c *Config) (*awsCredentials.Credentials, error) { // Otherwise we need to construct an STS client with the main credentials, and verify // that we can assume the defined role. - log.Printf("[INFO] Attempting to AssumeRole %s (SessionName: %q, ExternalId: %q, Policy: %q)", - c.AssumeRoleARN, c.AssumeRoleSessionName, c.AssumeRoleExternalID, c.AssumeRolePolicy) + log.Printf("[INFO] Attempting to AssumeRole %s (SessionName: %q, ExternalId: %q)", + c.AssumeRoleARN, c.AssumeRoleSessionName, c.AssumeRoleExternalID) awsConfig := &aws.Config{ - Credentials: creds, - Region: aws.String(c.Region), - MaxRetries: aws.Int(c.MaxRetries), - HTTPClient: cleanhttp.DefaultClient(), + Credentials: creds, + EndpointResolver: c.EndpointResolver(), + Region: aws.String(c.Region), + MaxRetries: aws.Int(c.MaxRetries), + HTTPClient: cleanhttp.DefaultClient(), } assumeRoleSession, err := session.NewSession(awsConfig) if err != nil { - return nil, fmt.Errorf("error creating assume role session: %s", err) + return nil, fmt.Errorf("error creating assume role session: %w", err) } stsclient := sts.New(assumeRoleSession) @@ -337,31 +270,60 @@ func GetCredentials(c *Config) (*awsCredentials.Credentials, error) { Client: stsclient, RoleARN: c.AssumeRoleARN, } - if c.AssumeRoleSessionName != "" { - assumeRoleProvider.RoleSessionName = c.AssumeRoleSessionName + + if c.AssumeRoleDurationSeconds > 0 { + assumeRoleProvider.Duration = time.Duration(c.AssumeRoleDurationSeconds) * time.Second } + if c.AssumeRoleExternalID != "" { assumeRoleProvider.ExternalID = aws.String(c.AssumeRoleExternalID) } + if c.AssumeRolePolicy != "" { assumeRoleProvider.Policy = aws.String(c.AssumeRolePolicy) } + if len(c.AssumeRolePolicyARNs) > 0 { + var policyDescriptorTypes []*sts.PolicyDescriptorType + + for _, policyARN := range c.AssumeRolePolicyARNs { + policyDescriptorType := &sts.PolicyDescriptorType{ + Arn: aws.String(policyARN), + } + policyDescriptorTypes = append(policyDescriptorTypes, policyDescriptorType) + } + + assumeRoleProvider.PolicyArns = policyDescriptorTypes + } + + if c.AssumeRoleSessionName != "" { + assumeRoleProvider.RoleSessionName = c.AssumeRoleSessionName + } + + if len(c.AssumeRoleTags) > 0 { + var tags []*sts.Tag + + for k, v := range c.AssumeRoleTags { + tag := &sts.Tag{ + Key: aws.String(k), + Value: aws.String(v), + } + tags = append(tags, tag) + } + + assumeRoleProvider.Tags = tags + } + + if len(c.AssumeRoleTransitiveTagKeys) > 0 { + assumeRoleProvider.TransitiveTagKeys = aws.StringSlice(c.AssumeRoleTransitiveTagKeys) + } + providers = []awsCredentials.Provider{assumeRoleProvider} assumeRoleCreds := awsCredentials.NewChainCredentials(providers) _, err = assumeRoleCreds.Get() if err != nil { - if awsErr, ok := err.(awserr.Error); ok && awsErr.Code() == "NoCredentialProviders" { - return nil, fmt.Errorf("The role %q cannot be assumed.\n\n"+ - " There are a number of possible causes of this - the most common are:\n"+ - " * The credentials used in order to assume the role are invalid\n"+ - " * The credentials do not have appropriate permission to assume the role\n"+ - " * The role ARN is not valid", - c.AssumeRoleARN) - } - - return nil, fmt.Errorf("Error loading credentials for AWS Provider: %s", err) + return nil, c.NewCannotAssumeRoleError(err) } return assumeRoleCreds, nil diff --git a/vendor/github.com/hashicorp/aws-sdk-go-base/awserr.go b/vendor/github.com/hashicorp/aws-sdk-go-base/awserr.go index 282954835..ce657679a 100644 --- a/vendor/github.com/hashicorp/aws-sdk-go-base/awserr.go +++ b/vendor/github.com/hashicorp/aws-sdk-go-base/awserr.go @@ -1,6 +1,7 @@ package awsbase import ( + "errors" "strings" "github.com/aws/aws-sdk-go/aws/awserr" @@ -11,17 +12,13 @@ import ( // * Error.Code() matches code // * Error.Message() contains message func IsAWSErr(err error, code string, message string) bool { - awsErr, ok := err.(awserr.Error) + var awsErr awserr.Error - if !ok { - return false + if errors.As(err, &awsErr) { + return awsErr.Code() == code && strings.Contains(awsErr.Message(), message) } - if awsErr.Code() != code { - return false - } - - return strings.Contains(awsErr.Message(), message) + return false } // IsAWSErrExtended returns true if the error matches all these conditions: @@ -33,5 +30,15 @@ func IsAWSErrExtended(err error, code string, message string, origErrMessage str if !IsAWSErr(err, code, message) { return false } - return strings.Contains(err.(awserr.Error).OrigErr().Error(), origErrMessage) + + if origErrMessage == "" { + return true + } + + // Ensure OrigErr() is non-nil, to prevent panics + if origErr := err.(awserr.Error).OrigErr(); origErr != nil { + return strings.Contains(origErr.Error(), origErrMessage) + } + + return false } diff --git a/vendor/github.com/hashicorp/aws-sdk-go-base/config.go b/vendor/github.com/hashicorp/aws-sdk-go-base/config.go index fb57e44e4..c5f9daafb 100644 --- a/vendor/github.com/hashicorp/aws-sdk-go-base/config.go +++ b/vendor/github.com/hashicorp/aws-sdk-go-base/config.go @@ -1,25 +1,31 @@ package awsbase type Config struct { - AccessKey string - AssumeRoleARN string - AssumeRoleExternalID string - AssumeRolePolicy string - AssumeRoleSessionName string - CredsFilename string - DebugLogging bool - IamEndpoint string - Insecure bool - MaxRetries int - Profile string - Region string - SecretKey string - SkipCredsValidation bool - SkipMetadataApiCheck bool - SkipRequestingAccountId bool - StsEndpoint string - Token string - UserAgentProducts []*UserAgentProduct + AccessKey string + AssumeRoleARN string + AssumeRoleDurationSeconds int + AssumeRoleExternalID string + AssumeRolePolicy string + AssumeRolePolicyARNs []string + AssumeRoleSessionName string + AssumeRoleTags map[string]string + AssumeRoleTransitiveTagKeys []string + CallerDocumentationURL string + CallerName string + CredsFilename string + DebugLogging bool + IamEndpoint string + Insecure bool + MaxRetries int + Profile string + Region string + SecretKey string + SkipCredsValidation bool + SkipMetadataApiCheck bool + SkipRequestingAccountId bool + StsEndpoint string + Token string + UserAgentProducts []*UserAgentProduct } type UserAgentProduct struct { diff --git a/vendor/github.com/hashicorp/aws-sdk-go-base/endpoints.go b/vendor/github.com/hashicorp/aws-sdk-go-base/endpoints.go new file mode 100644 index 000000000..c3f64d1cd --- /dev/null +++ b/vendor/github.com/hashicorp/aws-sdk-go-base/endpoints.go @@ -0,0 +1,46 @@ +package awsbase + +import ( + "log" + "os" + + "github.com/aws/aws-sdk-go/aws/ec2metadata" + "github.com/aws/aws-sdk-go/aws/endpoints" + "github.com/aws/aws-sdk-go/service/iam" + "github.com/aws/aws-sdk-go/service/sts" +) + +func (c *Config) EndpointResolver() endpoints.Resolver { + resolver := func(service, region string, optFns ...func(*endpoints.Options)) (endpoints.ResolvedEndpoint, error) { + // Ensure we pass all existing information (e.g. SigningRegion) and + // only override the URL, otherwise a MissingRegion error can occur + // when aws.Config.Region is not defined. + resolvedEndpoint, err := endpoints.DefaultResolver().EndpointFor(service, region, optFns...) + + if err != nil { + return resolvedEndpoint, err + } + + switch service { + case ec2metadata.ServiceName: + if endpoint := os.Getenv("AWS_METADATA_URL"); endpoint != "" { + log.Printf("[INFO] Setting custom EC2 metadata endpoint: %s", endpoint) + resolvedEndpoint.URL = endpoint + } + case iam.ServiceName: + if endpoint := c.IamEndpoint; endpoint != "" { + log.Printf("[INFO] Setting custom IAM endpoint: %s", endpoint) + resolvedEndpoint.URL = endpoint + } + case sts.ServiceName: + if endpoint := c.StsEndpoint; endpoint != "" { + log.Printf("[INFO] Setting custom STS endpoint: %s", endpoint) + resolvedEndpoint.URL = endpoint + } + } + + return resolvedEndpoint, nil + } + + return endpoints.ResolverFunc(resolver) +} diff --git a/vendor/github.com/hashicorp/aws-sdk-go-base/errors.go b/vendor/github.com/hashicorp/aws-sdk-go-base/errors.go new file mode 100644 index 000000000..2d5f1677c --- /dev/null +++ b/vendor/github.com/hashicorp/aws-sdk-go-base/errors.go @@ -0,0 +1,76 @@ +package awsbase + +import ( + "errors" + "fmt" +) + +// CannotAssumeRoleError occurs when AssumeRole cannot complete. +type CannotAssumeRoleError struct { + Config *Config + Err error +} + +func (e CannotAssumeRoleError) Error() string { + if e.Config == nil { + return fmt.Sprintf("cannot assume role: %s", e.Err) + } + + return fmt.Sprintf(`IAM Role (%s) cannot be assumed. + +There are a number of possible causes of this - the most common are: + * The credentials used in order to assume the role are invalid + * The credentials do not have appropriate permission to assume the role + * The role ARN is not valid + +Error: %s +`, e.Config.AssumeRoleARN, e.Err) +} + +func (e CannotAssumeRoleError) Unwrap() error { + return e.Err +} + +// IsCannotAssumeRoleError returns true if the error contains the CannotAssumeRoleError type. +func IsCannotAssumeRoleError(err error) bool { + var e CannotAssumeRoleError + return errors.As(err, &e) +} + +func (c *Config) NewCannotAssumeRoleError(err error) CannotAssumeRoleError { + return CannotAssumeRoleError{Config: c, Err: err} +} + +// NoValidCredentialSourcesError occurs when all credential lookup methods have been exhausted without results. +type NoValidCredentialSourcesError struct { + Config *Config + Err error +} + +func (e NoValidCredentialSourcesError) Error() string { + if e.Config == nil { + return fmt.Sprintf("no valid credential sources found: %s", e.Err) + } + + return fmt.Sprintf(`no valid credential sources for %s found. + +Please see %s +for more information about providing credentials. + +Error: %s +`, e.Config.CallerName, e.Config.CallerDocumentationURL, e.Err) +} + +func (e NoValidCredentialSourcesError) Unwrap() error { + return e.Err +} + +// IsNoValidCredentialSourcesError returns true if the error contains the NoValidCredentialSourcesError type. +func IsNoValidCredentialSourcesError(err error) bool { + var e NoValidCredentialSourcesError + return errors.As(err, &e) +} + +func (c *Config) NewNoValidCredentialSourcesError(err error) NoValidCredentialSourcesError { + return NoValidCredentialSourcesError{Config: c, Err: err} +} diff --git a/vendor/github.com/hashicorp/aws-sdk-go-base/go.mod b/vendor/github.com/hashicorp/aws-sdk-go-base/go.mod index 1af445d2b..d4cba362a 100644 --- a/vendor/github.com/hashicorp/aws-sdk-go-base/go.mod +++ b/vendor/github.com/hashicorp/aws-sdk-go-base/go.mod @@ -1,12 +1,10 @@ module github.com/hashicorp/aws-sdk-go-base require ( - github.com/aws/aws-sdk-go v1.25.3 + github.com/aws/aws-sdk-go v1.31.9 github.com/hashicorp/go-cleanhttp v0.5.0 github.com/hashicorp/go-multierror v1.0.0 - github.com/stretchr/testify v1.3.0 // indirect - golang.org/x/net v0.0.0-20190213061140-3a22650c66bd // indirect - golang.org/x/text v0.3.0 // indirect + github.com/mitchellh/go-homedir v1.1.0 ) go 1.13 diff --git a/vendor/github.com/hashicorp/aws-sdk-go-base/go.sum b/vendor/github.com/hashicorp/aws-sdk-go-base/go.sum index f06c2e910..eabfab4b5 100644 --- a/vendor/github.com/hashicorp/aws-sdk-go-base/go.sum +++ b/vendor/github.com/hashicorp/aws-sdk-go-base/go.sum @@ -1,23 +1,31 @@ -github.com/aws/aws-sdk-go v1.16.36 h1:POeH34ZME++pr7GBGh+ZO6Y5kOwSMQpqp5BGUgooJ6k= -github.com/aws/aws-sdk-go v1.16.36/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo= -github.com/aws/aws-sdk-go v1.25.3 h1:uM16hIw9BotjZKMZlX05SN2EFtaWfi/NonPKIARiBLQ= -github.com/aws/aws-sdk-go v1.25.3/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo= +github.com/aws/aws-sdk-go v1.31.9 h1:n+b34ydVfgC30j0Qm69yaapmjejQPW2BoDBX7Uy/tLI= +github.com/aws/aws-sdk-go v1.31.9/go.mod h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZveU8YkpAk0= github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/go-sql-driver/mysql v1.5.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg= github.com/hashicorp/errwrap v1.0.0 h1:hLrqtEDnRye3+sgx6z4qVLNuviH3MR5aQ0ykNJa/UYA= github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= github.com/hashicorp/go-cleanhttp v0.5.0 h1:wvCrVc9TjDls6+YGAF2hAifE1E5U1+b4tH6KdvN3Gig= github.com/hashicorp/go-cleanhttp v0.5.0/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80= github.com/hashicorp/go-multierror v1.0.0 h1:iVjPR7a6H0tWELX5NxNe7bYopibicUzc7uPribsnS6o= github.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk= -github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af h1:pmfjZENx5imkbgOkpRUYLnmbU7UEFbjtDA2hxJ1ichM= -github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k= +github.com/jmespath/go-jmespath v0.3.0 h1:OS12ieG61fsCg5+qLJ+SsW9NicxNkg3b25OyT2yCeUc= +github.com/jmespath/go-jmespath v0.3.0/go.mod h1:9QtRXoHjLGCJ5IBSaohpXITPlowMeeYCZ7fLUTSywik= +github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y= +github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= +github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= -github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q= -github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= -golang.org/x/net v0.0.0-20190213061140-3a22650c66bd h1:HuTn7WObtcDo9uEEU7rEqL0jYthdXAmZ6PP+meazmaU= -golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +github.com/stretchr/testify v1.5.1 h1:nOGnQDM7FYENwehXlg/kFVnos3rEvtKTjRvOWSzb6H4= +github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA= +golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= +golang.org/x/net v0.0.0-20200202094626-16171245cfb2 h1:CCH4IOTTfewWjGOlSp+zGcjutRKlBEZQ6wTn8ozI/nI= +golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw= +gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= diff --git a/vendor/github.com/hashicorp/aws-sdk-go-base/mock.go b/vendor/github.com/hashicorp/aws-sdk-go-base/mock.go index e3c802660..06ecd082b 100644 --- a/vendor/github.com/hashicorp/aws-sdk-go-base/mock.go +++ b/vendor/github.com/hashicorp/aws-sdk-go-base/mock.go @@ -2,6 +2,7 @@ package awsbase import ( "bytes" + "encoding/json" "fmt" "log" "net/http" @@ -19,7 +20,7 @@ func MockAwsApiServer(svcName string, endpoints []*MockEndpoint) *httptest.Serve ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { buf := new(bytes.Buffer) if _, err := buf.ReadFrom(r.Body); err != nil { - w.WriteHeader(500) + w.WriteHeader(http.StatusInternalServerError) fmt.Fprintf(w, "Error reading from HTTP Request Body: %s", err) return } @@ -43,7 +44,7 @@ func MockAwsApiServer(svcName string, endpoints []*MockEndpoint) *httptest.Serve } } - w.WriteHeader(400) + w.WriteHeader(http.StatusBadRequest) })) return ts @@ -73,20 +74,43 @@ func awsMetadataApiMock(responses []*MetadataResponse) func() { ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "text/plain") w.Header().Add("Server", "MockEC2") - log.Printf("[DEBUG] Mocker server received request to %q", r.RequestURI) + log.Printf("[DEBUG] Mock EC2 metadata server received request: %s", r.RequestURI) for _, e := range responses { if r.RequestURI == e.Uri { fmt.Fprintln(w, e.Body) return } } - w.WriteHeader(400) + w.WriteHeader(http.StatusBadRequest) })) os.Setenv("AWS_METADATA_URL", ts.URL+"/latest") return ts.Close } +// ecsCredentialsApiMock establishes a httptest server to mock out the ECS credentials API. +func ecsCredentialsApiMock() func() { + ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + w.Header().Add("Server", "MockECS") + log.Printf("[DEBUG] Mock ECS credentials server received request: %s", r.RequestURI) + if r.RequestURI == "/creds" { + _ = json.NewEncoder(w).Encode(map[string]string{ + "AccessKeyId": "EcsCredentialsAccessKey", + "Expiration": time.Now().UTC().Format(time.RFC3339), + "RoleArn": "arn:aws:iam::000000000000:role/EcsCredentials", + "SecretAccessKey": "EcsCredentialsSecretKey", + "Token": "EcsCredentialsSessionToken", + }) + return + } + w.WriteHeader(http.StatusBadRequest) + })) + + os.Setenv("AWS_CONTAINER_CREDENTIALS_FULL_URI", ts.URL+"/creds") + return ts.Close +} + // MockEndpoint represents a basic request and response that can be used for creating simple httptest server routes. type MockEndpoint struct { Request *MockRequest @@ -119,13 +143,17 @@ var ec2metadata_instanceIdEndpoint = &MetadataResponse{ } var ec2metadata_securityCredentialsEndpoints = []*MetadataResponse{ + { + Uri: "/latest/api/token", + Body: "Ec2MetadataApiToken", + }, { Uri: "/latest/meta-data/iam/security-credentials/", Body: "test_role", }, { Uri: "/latest/meta-data/iam/security-credentials/test_role", - Body: "{\"Code\":\"Success\",\"LastUpdated\":\"2015-12-11T17:17:25Z\",\"Type\":\"AWS-HMAC\",\"AccessKeyId\":\"somekey\",\"SecretAccessKey\":\"somesecret\",\"Token\":\"sometoken\"}", + Body: "{\"Code\":\"Success\",\"LastUpdated\":\"2015-12-11T17:17:25Z\",\"Type\":\"AWS-HMAC\",\"AccessKeyId\":\"Ec2MetadataAccessKey\",\"SecretAccessKey\":\"Ec2MetadataSecretKey\",\"Token\":\"Ec2MetadataSessionToken\"}", }, } @@ -165,6 +193,54 @@ const iamResponse_GetUser_unauthorized = ` + + + arn:aws:sts::555555555555:assumed-role/role/AssumeRoleSessionName + ARO123EXAMPLE123:AssumeRoleSessionName + + + AssumeRoleAccessKey + AssumeRoleSecretKey + AssumeRoleSessionToken + %s + + + + 01234567-89ab-cdef-0123-456789abcdef + +`, time.Now().UTC().Format(time.RFC3339)) + +const stsResponse_AssumeRole_InvalidClientTokenId = ` + + Sender + InvalidClientTokenId + The security token included in the request is invalid. + +4d0cf5ec-892a-4d3f-84e4-30e9987d9bdd +` + +var stsResponse_AssumeRoleWithWebIdentity_valid = fmt.Sprintf(` + + amzn1.account.AF6RHO7KZU5XRVQJGXK6HB56KR2A + client.6666666666666666666.6666@apps.example.com + + arn:aws:sts::666666666666:assumed-role/FederatedWebIdentityRole/AssumeRoleWithWebIdentitySessionName + ARO123EXAMPLE123:AssumeRoleWithWebIdentitySessionName + + + AssumeRoleWithWebIdentitySessionToken + AssumeRoleWithWebIdentitySecretKey + %s + AssumeRoleWithWebIdentityAccessKey + + www.amazon.com + + + 01234567-89ab-cdef-0123-456789abcdef + +`, time.Now().UTC().Format(time.RFC3339)) + const stsResponse_GetCallerIdentity_valid = ` arn:aws:iam::222222222222:user/Alice @@ -228,3 +304,5 @@ const iamResponse_ListRoles_unauthorized = ` 0 { @@ -82,9 +99,7 @@ func GetSession(c *Config) (*session.Session, error) { // NOTE: This logic can be fooled by other request errors raising the retry count // before any networking error occurs sess.Handlers.Retry.PushBack(func(r *request.Request) { - // We currently depend on the DefaultRetryer exponential backoff here. - // ~10 retries gives a fair backoff of a few seconds. - if r.RetryCount < 9 { + if r.RetryCount < MaxNetworkRetryCount { return } // RequestError: send request failed @@ -102,9 +117,8 @@ func GetSession(c *Config) (*session.Session, error) { }) if !c.SkipCredsValidation { - stsClient := sts.New(sess.Copy(&aws.Config{Endpoint: aws.String(c.StsEndpoint)})) - if _, _, err := GetAccountIDAndPartitionFromSTSGetCallerIdentity(stsClient); err != nil { - return nil, fmt.Errorf("error using credentials to get account ID: %s", err) + if _, _, err := GetAccountIDAndPartitionFromSTSGetCallerIdentity(sts.New(sess)); err != nil { + return nil, fmt.Errorf("error validating provider credentials: %w", err) } } @@ -125,14 +139,14 @@ func GetSessionWithAccountIDAndPartition(c *Config) (*session.Session, string, s return sess, accountID, partition, nil } - iamClient := iam.New(sess.Copy(&aws.Config{Endpoint: aws.String(c.IamEndpoint)})) - stsClient := sts.New(sess.Copy(&aws.Config{Endpoint: aws.String(c.StsEndpoint)})) + iamClient := iam.New(sess) + stsClient := sts.New(sess) if !c.SkipCredsValidation { accountID, partition, err := GetAccountIDAndPartitionFromSTSGetCallerIdentity(stsClient) if err != nil { - return nil, "", "", fmt.Errorf("error validating provider credentials: %s", err) + return nil, "", "", fmt.Errorf("error validating provider credentials: %w", err) } return sess, accountID, partition, nil @@ -154,7 +168,7 @@ func GetSessionWithAccountIDAndPartition(c *Config) (*session.Session, string, s return nil, "", "", fmt.Errorf( "AWS account ID not previously found and failed retrieving via all available methods. "+ "See https://www.terraform.io/docs/providers/aws/index.html#skip_requesting_account_id for workaround and implications. "+ - "Errors: %s", err) + "Errors: %w", err) } var partition string diff --git a/vendor/modules.txt b/vendor/modules.txt index 2cbe3f82b..2c433512b 100644 --- a/vendor/modules.txt +++ b/vendor/modules.txt @@ -112,7 +112,7 @@ github.com/armon/circbuf # github.com/armon/go-radix v1.0.0 ## explicit github.com/armon/go-radix -# github.com/aws/aws-sdk-go v1.30.12 +# github.com/aws/aws-sdk-go v1.31.9 ## explicit github.com/aws/aws-sdk-go/aws github.com/aws/aws-sdk-go/aws/arn @@ -144,6 +144,7 @@ github.com/aws/aws-sdk-go/internal/sdkuri github.com/aws/aws-sdk-go/internal/shareddefaults github.com/aws/aws-sdk-go/internal/strings github.com/aws/aws-sdk-go/internal/sync/singleflight +github.com/aws/aws-sdk-go/private/checksum github.com/aws/aws-sdk-go/private/protocol github.com/aws/aws-sdk-go/private/protocol/eventstream github.com/aws/aws-sdk-go/private/protocol/eventstream/eventstreamapi @@ -293,7 +294,7 @@ github.com/gophercloud/utils/terraform/auth ## explicit # github.com/grpc-ecosystem/grpc-gateway v1.8.5 ## explicit -# github.com/hashicorp/aws-sdk-go-base v0.4.0 +# github.com/hashicorp/aws-sdk-go-base v0.5.0 ## explicit github.com/hashicorp/aws-sdk-go-base # github.com/hashicorp/consul v0.0.0-20171026175957-610f3c86a089 diff --git a/website/docs/backends/types/s3.html.md b/website/docs/backends/types/s3.html.md index 4f9596d12..98107722d 100644 --- a/website/docs/backends/types/s3.html.md +++ b/website/docs/backends/types/s3.html.md @@ -140,61 +140,66 @@ data.terraform_remote_state.network: public_subnet_id = subnet-1e05dd33 ``` -## Configuration variables +## Configuration -The following configuration options or environment variables are supported: +This backend requires the configuration of the AWS Region and S3 state storage. Other configuration, such as enabling DynamoDB state locking, is optional. - * `bucket` - (Required) The name of the S3 bucket. - * `key` - (Required) The path to the state file inside the bucket. When using - a non-default [workspace](/docs/state/workspaces.html), the state path will - be `/workspace_key_prefix/workspace_name/key` - * `region` / `AWS_DEFAULT_REGION` - (Optional) The region of the S3 - bucket. - * `endpoint` / `AWS_S3_ENDPOINT` - (Optional) A custom endpoint for the - S3 API. - * `encrypt` - (Optional) Whether to enable [server side - encryption](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html) - of the state file. - * `acl` - [Canned - ACL](https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl) - to be applied to the state file. - * `access_key` / `AWS_ACCESS_KEY_ID` - (Optional) AWS access key. - * `secret_key` / `AWS_SECRET_ACCESS_KEY` - (Optional) AWS secret access key. - * `kms_key_id` - (Optional) The ARN of a KMS Key to use for encrypting - the state. - * `lock_table` - (Optional, Deprecated) Use `dynamodb_table` instead. - * `dynamodb_table` - (Optional) The name of a DynamoDB table to use for state - locking and consistency. The table must have a primary key named LockID(`LockID` must have type of `string`). If - not present, locking will be disabled. - * `profile` - (Optional) This is the AWS profile name as set in the - shared credentials file. It can also be sourced from the `AWS_PROFILE` - environment variable if `AWS_SDK_LOAD_CONFIG` is set to a truthy value, - e.g. `AWS_SDK_LOAD_CONFIG=1`. - * `shared_credentials_file` - (Optional) This is the path to the - shared credentials file. If this is not set and a profile is specified, - `~/.aws/credentials` will be used. - * `token` - (Optional) Use this to set an MFA token. It can also be - sourced from the `AWS_SESSION_TOKEN` environment variable. - * `role_arn` - (Optional) The role to be assumed. - * `assume_role_policy` - (Optional) The permissions applied when assuming a role. - * `external_id` - (Optional) The external ID to use when assuming the role. - * `session_name` - (Optional) The session name to use when assuming the role. - * `workspace_key_prefix` - (Optional) The prefix applied to the state path - inside the bucket. This is only relevant when using a non-default workspace. - This defaults to "env:" - * `dynamodb_endpoint` / `AWS_DYNAMODB_ENDPOINT` - (Optional) A custom endpoint for the DynamoDB API. - * `iam_endpoint` / `AWS_IAM_ENDPOINT` - (Optional) A custom endpoint for the IAM API. - * `sts_endpoint` / `AWS_STS_ENDPOINT` - (Optional) A custom endpoint for the STS API. - * `force_path_style` - (Optional) Always use path-style S3 URLs (`https:///` instead of `https://.`). - * `skip_credentials_validation` - (Optional) Skip the credentials validation via the STS API. - * `skip_get_ec2_platforms` - (**DEPRECATED**, Optional) Skip getting the supported EC2 platforms. - * `skip_region_validation` - (**DEPRECATED**, Optional) Skip validation of provided region name. - * `skip_requesting_account_id` - (**DEPRECATED**, Optional) Skip requesting the account ID. - * `skip_metadata_api_check` - (Optional) Skip the AWS Metadata API check. - * `sse_customer_key` / `AWS_SSE_CUSTOMER_KEY` - (Optional) The key to use for encrypting state with [Server-Side Encryption with Customer-Provided Keys (SSE-C)](https://docs.aws.amazon.com/AmazonS3/latest/dev/ServerSideEncryptionCustomerKeys.html). - This is the base64-encoded value of the key, which must decode to 256 bits. Due to the sensitivity of the value, it is recommended to set it using the `AWS_SSE_CUSTOMER_KEY` environment variable. - Setting it inside a terraform file will cause it to be persisted to disk in `terraform.tfstate`. - * `max_retries` - (Optional) The maximum number of times an AWS API request is retried on retryable failure. Defaults to 5. +### Credentials and Shared Configuration + +The following configuration is required: + +* `region` - (Required) AWS Region of the S3 Bucket and DynamoDB Table (if used). This can also be sourced from the `AWS_DEFAULT_REGION` and `AWS_REGION` environment variables. + +The following configuration is optional: + +* `access_key` - (Optional) AWS access key. If configured, must also configure `secret_key`. This can also be sourced from the `AWS_ACCESS_KEY_ID` environment variable, AWS shared credentials file (e.g. `~/.aws/credentials`), or AWS shared configuration file (e.g. `~/.aws/config`). +* `secret_key` - (Optional) AWS access key. If configured, must also configure `access_key`. This can also be sourced from the `AWS_SECRET_ACCESS_KEY` environment variable, AWS shared credentials file (e.g. `~/.aws/credentials`), or AWS shared configuration file (e.g. `~/.aws/config`). +* `iam_endpoint` - (Optional) Custom endpoint for the AWS Identity and Access Management (IAM) API. This can also be sourced from the `AWS_IAM_ENDPOINT` environment variable. +* `max_retries` - (Optional) The maximum number of times an AWS API request is retried on retryable failure. Defaults to 5. +* `profile` - (Optional) Name of AWS profile in AWS shared credentials file (e.g. `~/.aws/credentials`) or AWS shared configuration file (e.g. `~/.aws/config`) to use for credentials and/or configuration. This can also be sourced from the `AWS_PROFILE` environment variable. +* `shared_credentials_file` - (Optional) Path to the AWS shared credentials file. Defaults to `~/.aws/credentials`. +* `skip_credentials_validation` - (Optional) Skip credentials validation via the STS API. +* `skip_region_validation` - (Optional) Skip validation of provided region name. +* `skip_metadata_api_check` - (Optional) Skip usage of EC2 Metadata API. +* `sts_endpoint` - (Optional) Custom endpoint for the AWS Security Token Service (STS) API. This can also be sourced from the `AWS_STS_ENDPOINT` environment variable. +* `token` - (Optional) Multi-Factor Authentication (MFA) token. This can also be sourced from the `AWS_SESSION_TOKEN` environment variable. + +#### Assume Role Configuration + +The following configuration is optional: + +* `assume_role_duration_seconds` - (Optional) Number of seconds to restrict the assume role session duration. +* `assume_role_policy` - (Optional) IAM Policy JSON describing further restricting permissions for the IAM Role being assumed. +* `assume_role_policy_arns` - (Optional) Set of Amazon Resource Names (ARNs) of IAM Policies describing further restricting permissions for the IAM Role being assumed. +* `assume_role_tags` - (Optional) Map of assume role session tags. +* `assume_role_transitive_tag_keys` - (Optional) Set of assume role session tag keys to pass to any subsequent sessions. +* `external_id` - (Optional) External identifier to use when assuming the role. +* `role_arn` - (Optional) Amazon Resource Name (ARN) of the IAM Role to assume. +* `session_name` - (Optional) Session name to use when assuming the role. + +### S3 State Storage + +The following configuration is required: + +* `bucket` - (Required) Name of the S3 Bucket. +* `key` - (Required) Path to the state file inside the S3 Bucket. When using a non-default [workspace](/docs/state/workspaces.html), the state path will be `/workspace_key_prefix/workspace_name/key` (see also the `workspace_key_prefix` configuration). + +The following configuration is optional: + +* `acl` - (Optional) [Canned ACL](https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl) to be applied to the state file. +* `encrypt` - (Optional) Enable [server side encryption](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html) of the state file. +* `endpoint` - (Optional) Custom endpoint for the AWS S3 API. This can also be sourced from the `AWS_S3_ENDPOINT` environment variable. +* `force_path_style` - (Optional) Enable path-style S3 URLs (`https:///` instead of `https://.`). +* `kms_key_id` - (Optional) Amazon Resource Name (ARN) of a Key Management Service (KMS) Key to use for encrypting the state. +* `sse_customer_key` - (Optional) The key to use for encrypting state with [Server-Side Encryption with Customer-Provided Keys (SSE-C)](https://docs.aws.amazon.com/AmazonS3/latest/dev/ServerSideEncryptionCustomerKeys.html). This is the base64-encoded value of the key, which must decode to 256 bits. This can also be sourced from the `AWS_SSE_CUSTOMER_KEY` environment variable, which is recommended due to the sensitivity of the value. Setting it inside a terraform file will cause it to be persisted to disk in `terraform.tfstate`. +* `workspace_key_prefix` - (Optional) Prefix applied to the state path inside the bucket. This is only relevant when using a non-default workspace. Defaults to `env:`. + +### DynamoDB State Locking + +The following configuration is optional: + +* `dynamodb_endpoint` - (Optional) Custom endpoint for the AWS DynamoDB API. This can also be sourced from the `AWS_DYNAMODB_ENDPOINT` environment variable. +* `dynamodb_table` - (Optional) Name of DynamoDB Table to use for state locking and consistency. The table must have a primary key named `LockID` with type of `string`. If not configured, state locking will be disabled. ## Multi-account AWS Architecture