Conversation
Change the 'region' property type to accept both string and array.
Refactor AWS ECR login process to handle multiple regions and improve default region handling.
Added documentation for specifying multiple AWS regions in ECR plugin configuration.
| @test "ECR login; multiple regions as array" { | ||
| export BUILDKITE_PLUGIN_ECR_LOGIN=true | ||
| export BUILDKITE_PLUGIN_ECR_ACCOUNT_IDS=321321321321 | ||
| export BUILDKITE_PLUGIN_ECR_REGION_0=ap-southeast-2 | ||
| export BUILDKITE_PLUGIN_ECR_REGION_1=eu-west-1 | ||
|
|
||
| stub aws \ | ||
| "--version : echo aws-cli/2.0.0 Python/3.8.1 Linux/5.5.6-arch1-1 botocore/1.15.3" \ | ||
| "--region ap-southeast-2 ecr get-login-password : echo password1" \ | ||
| "--region eu-west-1 ecr get-login-password : echo password2" | ||
|
|
||
| stub docker \ | ||
| "login --username AWS --password-stdin 321321321321.dkr.ecr.ap-southeast-2.amazonaws.com : cat > /tmp/password-stdin-1 ; echo logged in to ap-southeast-2" \ | ||
| "login --username AWS --password-stdin 321321321321.dkr.ecr.eu-west-1.amazonaws.com : cat > /tmp/password-stdin-2 ; echo logged in to eu-west-1" | ||
|
|
||
| run "$PWD/hooks/environment" | ||
|
|
||
| assert_success | ||
| assert_output --partial "Authenticating with AWS ECR in ap-southeast-2 eu-west-1 for 321321321321 " | ||
| assert_output --partial "logged in to ap-southeast-2" | ||
| assert_output --partial "logged in to eu-west-1" | ||
| assert_equal "password1" "$(cat /tmp/password-stdin-1)" | ||
| assert_equal "password2" "$(cat /tmp/password-stdin-2)" | ||
|
|
||
| unstub aws | ||
| unstub docker | ||
| rm /tmp/password-stdin-1 /tmp/password-stdin-2 | ||
| } | ||
|
|
||
| @test "ECR login; multiple comma-separated regions" { | ||
| export BUILDKITE_PLUGIN_ECR_LOGIN=true | ||
| export BUILDKITE_PLUGIN_ECR_ACCOUNT_IDS=321321321321 | ||
| export BUILDKITE_PLUGIN_ECR_REGION="ap-southeast-2,eu-west-1" | ||
|
|
||
| stub aws \ | ||
| "--version : echo aws-cli/2.0.0 Python/3.8.1 Linux/5.5.6-arch1-1 botocore/1.15.3" \ | ||
| "--region ap-southeast-2 ecr get-login-password : echo password1" \ | ||
| "--region eu-west-1 ecr get-login-password : echo password2" | ||
|
|
||
| stub docker \ | ||
| "login --username AWS --password-stdin 321321321321.dkr.ecr.ap-southeast-2.amazonaws.com : cat > /tmp/password-stdin-1 ; echo logged in to ap-southeast-2" \ | ||
| "login --username AWS --password-stdin 321321321321.dkr.ecr.eu-west-1.amazonaws.com : cat > /tmp/password-stdin-2 ; echo logged in to eu-west-1" | ||
|
|
||
| run "$PWD/hooks/environment" | ||
|
|
||
| assert_success | ||
| assert_output --partial "logged in to ap-southeast-2" | ||
| assert_output --partial "logged in to eu-west-1" | ||
|
|
||
| unstub aws | ||
| unstub docker | ||
| rm /tmp/password-stdin-1 /tmp/password-stdin-2 | ||
| } | ||
|
|
||
| @test "ECR login; multiple regions with multiple account IDs" { | ||
| export BUILDKITE_PLUGIN_ECR_LOGIN=true | ||
| export BUILDKITE_PLUGIN_ECR_ACCOUNT_IDS_0=111111111111 | ||
| export BUILDKITE_PLUGIN_ECR_ACCOUNT_IDS_1=222222222222 | ||
| export BUILDKITE_PLUGIN_ECR_REGION_0=us-east-1 | ||
| export BUILDKITE_PLUGIN_ECR_REGION_1=us-west-2 | ||
|
|
||
| stub aws \ | ||
| "--version : echo aws-cli/2.0.0 Python/3.8.1 Linux/5.5.6-arch1-1 botocore/1.15.3" \ | ||
| "--region us-east-1 ecr get-login-password : echo password-ue1" \ | ||
| "--region us-west-2 ecr get-login-password : echo password-uw2" | ||
|
|
||
| stub docker \ | ||
| "login --username AWS --password-stdin 111111111111.dkr.ecr.us-east-1.amazonaws.com : echo logged in" \ | ||
| "login --username AWS --password-stdin 222222222222.dkr.ecr.us-east-1.amazonaws.com : echo logged in" \ | ||
| "login --username AWS --password-stdin 111111111111.dkr.ecr.us-west-2.amazonaws.com : echo logged in" \ | ||
| "login --username AWS --password-stdin 222222222222.dkr.ecr.us-west-2.amazonaws.com : echo logged in" | ||
|
|
||
| run "$PWD/hooks/environment" | ||
|
|
||
| assert_success | ||
| assert_output --partial "Authenticating with AWS ECR in us-east-1 us-west-2 for 111111111111 222222222222" | ||
|
|
||
| unstub aws | ||
| unstub docker | ||
| } |
There was a problem hiding this comment.
The new tests cover the AWS CLI 2 private-registry path, but this refactor also changes the legacy get-login and credential-helper paths. Running Plugin Tester currently produces 11 failures in the existing legacy tests.
Two failures expose the dropped registry-region fallback. Once that is fixed, the other legacy stubs should be updated to expect the new explicit --region us-east-1 argument. Could we also add focused multi-region coverage for:
- AWS CLI 1.17.9 and get-login;
- credential-helper configuration containing both regional registry keys;
- public ECR with multiple configured regions;
- REGISTRY_REGION and REGION both being set?
Co-authored-by: Łukasz Jendrysik <10155318+scadu@users.noreply.github.com>
scadu
left a comment
There was a problem hiding this comment.
Two comments on a single issue related to how AWS CLI handles regions and the ecr get-login command between releases v1.17.9 and v1.17.10. Once it's in, we're good to go.
| # Preserve legacy precedence: REGISTRY_REGION > REGION > AWS_DEFAULT_REGION > us-east-1 | ||
| regions=() | ||
|
|
||
| if [[ -n "${BUILDKITE_PLUGIN_ECR_REGISTRY_REGION:-}" ]]; then | ||
| regions=("${BUILDKITE_PLUGIN_ECR_REGISTRY_REGION}") | ||
| else | ||
| while IFS='' read -r line; do regions+=("$line"); done < <(plugin_read_list REGION | tr "," "\n") | ||
| fi | ||
|
|
||
| if [[ ${#regions[@]} -eq 0 || -z "${regions[*]}" ]]; then | ||
| regions=("${AWS_DEFAULT_REGION:-us-east-1}") | ||
| fi |
There was a problem hiding this comment.
Re-reading it now and digging into AWS docs, it looks like ordering still needs attention as it reverses the precedence in the legacy get-login path used before AWS CLI 1.17.10, when get-login-password became available.
Previously, registry-region was followed by region; AWS documents that for a repeated single-value option, the last occurrence applies. Therefore, region previously won when both were configured.
Could we read REGION first, then fall back explicitly to REGISTRY_REGION, AWS_DEFAULT_REGION, and us-east-1?
| # Preserve legacy precedence: REGISTRY_REGION > REGION > AWS_DEFAULT_REGION > us-east-1 | |
| regions=() | |
| if [[ -n "${BUILDKITE_PLUGIN_ECR_REGISTRY_REGION:-}" ]]; then | |
| regions=("${BUILDKITE_PLUGIN_ECR_REGISTRY_REGION}") | |
| else | |
| while IFS='' read -r line; do regions+=("$line"); done < <(plugin_read_list REGION | tr "," "\n") | |
| fi | |
| if [[ ${#regions[@]} -eq 0 || -z "${regions[*]}" ]]; then | |
| regions=("${AWS_DEFAULT_REGION:-us-east-1}") | |
| fi | |
| # Preserve legacy precedence: REGION > REGISTRY_REGION > AWS_DEFAULT_REGION > us-east-1 | |
| regions=() | |
| while IFS='' read -r line; do regions+=("$line"); done < <(plugin_read_list REGION | tr "," "\n") | |
| if [[ ${#regions[@]} -eq 0 || -z "${regions[*]}" ]]; then | |
| if [[ -n "${BUILDKITE_PLUGIN_ECR_REGISTRY_REGION:-}" ]]; then | |
| regions=("${BUILDKITE_PLUGIN_ECR_REGISTRY_REGION}") | |
| elif [[ -n "${AWS_DEFAULT_REGION:-}" ]]; then | |
| regions=("${AWS_DEFAULT_REGION}") | |
| else | |
| regions=("us-east-1") | |
| fi | |
| fi |
|
|
||
| unstub aws | ||
| unstub docker | ||
| } |
There was a problem hiding this comment.
AWS CLI keeps on giving, huh.
Looks like we'd need to add a regression test covering the legacy path with both registry-region and a region list configured.
Using AWS CLI 1.17.9 here selects ecr get-login, because that version registered only the legacy command; AWS added get-login-password in 1.17.10. The expected calls should use both values from REGION and not the deprecated REGISTRY_REGION, preserving the effective precedence established by AWS’s last-instance behavior for single-value options.
This test would cover both the compatibility fix and multi-region support in the otherwise untested legacy path:
| } | |
| } | |
| @test "ECR login (before aws cli 1.17.10); region list takes precedence over registry-region" { | |
| export BUILDKITE_PLUGIN_ECR_LOGIN=true | |
| export BUILDKITE_PLUGIN_ECR_NO_INCLUDE_EMAIL=true | |
| export BUILDKITE_PLUGIN_ECR_REGISTRY_REGION=ap-northeast-1 | |
| export BUILDKITE_PLUGIN_ECR_REGION_0=ap-southeast-2 | |
| export BUILDKITE_PLUGIN_ECR_REGION_1=eu-west-1 | |
| stub aws \ | |
| "--version : echo aws-cli/1.17.9 Python/3.8.1 Linux/5.5.6-arch1-1 botocore/1.15.3" \ | |
| "ecr get-login --no-include-email --region ap-southeast-2 : echo echo logged in to ap-southeast-2" \ | |
| "ecr get-login --no-include-email --region eu-west-1 : echo echo logged in to eu-west-1" | |
| run "$PWD/hooks/environment" | |
| assert_success | |
| assert_output --partial "logged in to ap-southeast-2" | |
| assert_output --partial "logged in to eu-west-1" | |
| unstub aws | |
| } |
PR is created to address the issue #110