Skip to content

SUP 2776: Support for multiple regions in one plugin call#131

Open
nsuma8989 wants to merge 28 commits into
masterfrom
sup-2776
Open

SUP 2776: Support for multiple regions in one plugin call#131
nsuma8989 wants to merge 28 commits into
masterfrom
sup-2776

Conversation

@nsuma8989

Copy link
Copy Markdown

PR is created to address the issue #110

nsuma8989 added 6 commits July 7, 2026 16:33
Change the 'region' property type to accept both string and array.
Refactor AWS ECR login process to handle multiple regions and improve default region handling.
Added documentation for specifying multiple AWS regions in ECR plugin configuration.
@nsuma8989 nsuma8989 requested a review from a team as a code owner July 8, 2026 07:11
Comment thread hooks/environment
Comment thread hooks/environment
Comment thread hooks/environment Outdated
Comment thread tests/run.bats
Comment on lines +1006 to +1085
@test "ECR login; multiple regions as array" {
export BUILDKITE_PLUGIN_ECR_LOGIN=true
export BUILDKITE_PLUGIN_ECR_ACCOUNT_IDS=321321321321
export BUILDKITE_PLUGIN_ECR_REGION_0=ap-southeast-2
export BUILDKITE_PLUGIN_ECR_REGION_1=eu-west-1

stub aws \
"--version : echo aws-cli/2.0.0 Python/3.8.1 Linux/5.5.6-arch1-1 botocore/1.15.3" \
"--region ap-southeast-2 ecr get-login-password : echo password1" \
"--region eu-west-1 ecr get-login-password : echo password2"

stub docker \
"login --username AWS --password-stdin 321321321321.dkr.ecr.ap-southeast-2.amazonaws.com : cat > /tmp/password-stdin-1 ; echo logged in to ap-southeast-2" \
"login --username AWS --password-stdin 321321321321.dkr.ecr.eu-west-1.amazonaws.com : cat > /tmp/password-stdin-2 ; echo logged in to eu-west-1"

run "$PWD/hooks/environment"

assert_success
assert_output --partial "Authenticating with AWS ECR in ap-southeast-2 eu-west-1 for 321321321321 "
assert_output --partial "logged in to ap-southeast-2"
assert_output --partial "logged in to eu-west-1"
assert_equal "password1" "$(cat /tmp/password-stdin-1)"
assert_equal "password2" "$(cat /tmp/password-stdin-2)"

unstub aws
unstub docker
rm /tmp/password-stdin-1 /tmp/password-stdin-2
}

@test "ECR login; multiple comma-separated regions" {
export BUILDKITE_PLUGIN_ECR_LOGIN=true
export BUILDKITE_PLUGIN_ECR_ACCOUNT_IDS=321321321321
export BUILDKITE_PLUGIN_ECR_REGION="ap-southeast-2,eu-west-1"

stub aws \
"--version : echo aws-cli/2.0.0 Python/3.8.1 Linux/5.5.6-arch1-1 botocore/1.15.3" \
"--region ap-southeast-2 ecr get-login-password : echo password1" \
"--region eu-west-1 ecr get-login-password : echo password2"

stub docker \
"login --username AWS --password-stdin 321321321321.dkr.ecr.ap-southeast-2.amazonaws.com : cat > /tmp/password-stdin-1 ; echo logged in to ap-southeast-2" \
"login --username AWS --password-stdin 321321321321.dkr.ecr.eu-west-1.amazonaws.com : cat > /tmp/password-stdin-2 ; echo logged in to eu-west-1"

run "$PWD/hooks/environment"

assert_success
assert_output --partial "logged in to ap-southeast-2"
assert_output --partial "logged in to eu-west-1"

unstub aws
unstub docker
rm /tmp/password-stdin-1 /tmp/password-stdin-2
}

@test "ECR login; multiple regions with multiple account IDs" {
export BUILDKITE_PLUGIN_ECR_LOGIN=true
export BUILDKITE_PLUGIN_ECR_ACCOUNT_IDS_0=111111111111
export BUILDKITE_PLUGIN_ECR_ACCOUNT_IDS_1=222222222222
export BUILDKITE_PLUGIN_ECR_REGION_0=us-east-1
export BUILDKITE_PLUGIN_ECR_REGION_1=us-west-2

stub aws \
"--version : echo aws-cli/2.0.0 Python/3.8.1 Linux/5.5.6-arch1-1 botocore/1.15.3" \
"--region us-east-1 ecr get-login-password : echo password-ue1" \
"--region us-west-2 ecr get-login-password : echo password-uw2"

stub docker \
"login --username AWS --password-stdin 111111111111.dkr.ecr.us-east-1.amazonaws.com : echo logged in" \
"login --username AWS --password-stdin 222222222222.dkr.ecr.us-east-1.amazonaws.com : echo logged in" \
"login --username AWS --password-stdin 111111111111.dkr.ecr.us-west-2.amazonaws.com : echo logged in" \
"login --username AWS --password-stdin 222222222222.dkr.ecr.us-west-2.amazonaws.com : echo logged in"

run "$PWD/hooks/environment"

assert_success
assert_output --partial "Authenticating with AWS ECR in us-east-1 us-west-2 for 111111111111 222222222222"

unstub aws
unstub docker
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new tests cover the AWS CLI 2 private-registry path, but this refactor also changes the legacy get-login and credential-helper paths. Running Plugin Tester currently produces 11 failures in the existing legacy tests.

Two failures expose the dropped registry-region fallback. Once that is fixed, the other legacy stubs should be updated to expect the new explicit --region us-east-1 argument. Could we also add focused multi-region coverage for:

  • AWS CLI 1.17.9 and get-login;
  • credential-helper configuration containing both regional registry keys;
  • public ECR with multiple configured regions;
  • REGISTRY_REGION and REGION both being set?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

@scadu scadu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two comments on a single issue related to how AWS CLI handles regions and the ecr get-login command between releases v1.17.9 and v1.17.10. Once it's in, we're good to go.

Comment thread hooks/environment
Comment on lines +158 to +169
# Preserve legacy precedence: REGISTRY_REGION > REGION > AWS_DEFAULT_REGION > us-east-1
regions=()

if [[ -n "${BUILDKITE_PLUGIN_ECR_REGISTRY_REGION:-}" ]]; then
regions=("${BUILDKITE_PLUGIN_ECR_REGISTRY_REGION}")
else
while IFS='' read -r line; do regions+=("$line"); done < <(plugin_read_list REGION | tr "," "\n")
fi

if [[ ${#regions[@]} -eq 0 || -z "${regions[*]}" ]]; then
regions=("${AWS_DEFAULT_REGION:-us-east-1}")
fi

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reading it now and digging into AWS docs, it looks like ordering still needs attention as it reverses the precedence in the legacy get-login path used before AWS CLI 1.17.10, when get-login-password became available.

Previously, registry-region was followed by region; AWS documents that for a repeated single-value option, the last occurrence applies. Therefore, region previously won when both were configured.

Could we read REGION first, then fall back explicitly to REGISTRY_REGION, AWS_DEFAULT_REGION, and us-east-1?

Suggested change
# Preserve legacy precedence: REGISTRY_REGION > REGION > AWS_DEFAULT_REGION > us-east-1
regions=()
if [[ -n "${BUILDKITE_PLUGIN_ECR_REGISTRY_REGION:-}" ]]; then
regions=("${BUILDKITE_PLUGIN_ECR_REGISTRY_REGION}")
else
while IFS='' read -r line; do regions+=("$line"); done < <(plugin_read_list REGION | tr "," "\n")
fi
if [[ ${#regions[@]} -eq 0 || -z "${regions[*]}" ]]; then
regions=("${AWS_DEFAULT_REGION:-us-east-1}")
fi
# Preserve legacy precedence: REGION > REGISTRY_REGION > AWS_DEFAULT_REGION > us-east-1
regions=()
while IFS='' read -r line; do regions+=("$line"); done < <(plugin_read_list REGION | tr "," "\n")
if [[ ${#regions[@]} -eq 0 || -z "${regions[*]}" ]]; then
if [[ -n "${BUILDKITE_PLUGIN_ECR_REGISTRY_REGION:-}" ]]; then
regions=("${BUILDKITE_PLUGIN_ECR_REGISTRY_REGION}")
elif [[ -n "${AWS_DEFAULT_REGION:-}" ]]; then
regions=("${AWS_DEFAULT_REGION}")
else
regions=("us-east-1")
fi
fi

Comment thread tests/run.bats

unstub aws
unstub docker
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AWS CLI keeps on giving, huh.
Looks like we'd need to add a regression test covering the legacy path with both registry-region and a region list configured.

Using AWS CLI 1.17.9 here selects ecr get-login, because that version registered only the legacy command; AWS added get-login-password in 1.17.10. The expected calls should use both values from REGION and not the deprecated REGISTRY_REGION, preserving the effective precedence established by AWS’s last-instance behavior for single-value options.

This test would cover both the compatibility fix and multi-region support in the otherwise untested legacy path:

Suggested change
}
}
@test "ECR login (before aws cli 1.17.10); region list takes precedence over registry-region" {
export BUILDKITE_PLUGIN_ECR_LOGIN=true
export BUILDKITE_PLUGIN_ECR_NO_INCLUDE_EMAIL=true
export BUILDKITE_PLUGIN_ECR_REGISTRY_REGION=ap-northeast-1
export BUILDKITE_PLUGIN_ECR_REGION_0=ap-southeast-2
export BUILDKITE_PLUGIN_ECR_REGION_1=eu-west-1
stub aws \
"--version : echo aws-cli/1.17.9 Python/3.8.1 Linux/5.5.6-arch1-1 botocore/1.15.3" \
"ecr get-login --no-include-email --region ap-southeast-2 : echo echo logged in to ap-southeast-2" \
"ecr get-login --no-include-email --region eu-west-1 : echo echo logged in to eu-west-1"
run "$PWD/hooks/environment"
assert_success
assert_output --partial "logged in to ap-southeast-2"
assert_output --partial "logged in to eu-west-1"
unstub aws
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants