-
Notifications
You must be signed in to change notification settings - Fork 0
chore: add cache to the build stage #57
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: go-integration-tests
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds caching mechanisms to improve CI/CD performance for a Go application workflow. It introduces GitHub Actions cache for Docker build operations and implements Docker image pre-pulling/caching for integration tests to reduce build times and network usage.
Key Changes:
- Adds GitHub Actions cache backend (
type=gha) to Docker build stage for layer caching - Implements Docker image caching for integration tests with disk space cleanup to handle large images
- Upgrades
docker/setup-buildx-actionfrom v2 to v3
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated 8 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Calculate cache slot (invalidate every 20 runs) | ||
| CACHE_SLOT=$(( ${{ github.run_number }} - (${{ github.run_number }} % 20) )) |
Copilot
AI
Dec 4, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The cache key rotation strategy using github.run_number will not work correctly across different branches. Run numbers are global to the repository, not per-branch, which means cache keys will invalidate unpredictably when PRs from different branches are running.
Consider using a time-based rotation (e.g., ${{ github.run_number / 20 }} rounded down) or a date-based approach like $(date +%Y%m%d) to ensure more predictable cache behavior across branches.
| # Calculate cache slot (invalidate every 20 runs) | |
| CACHE_SLOT=$(( ${{ github.run_number }} - (${{ github.run_number }} % 20) )) | |
| # Calculate cache slot (invalidate daily) | |
| CACHE_SLOT=$(date +%Y%m%d) |
| continue | ||
| fi | ||
| filename=$(echo "$image" | tr '/:' '_') |
Copilot
AI
Dec 4, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The filename sanitization using tr '/:' '_' is insufficient and could lead to collisions or filesystem issues. It doesn't handle other problematic characters like spaces, special characters, or very long names.
Consider using a more robust approach:
filename=$(echo "$image" | sed 's/[^a-zA-Z0-9._-]/_/g' | cut -c1-200)Or use a hash-based approach for guaranteed uniqueness:
filename=$(echo "$image" | sha256sum | cut -d' ' -f1)| filename=$(echo "$image" | tr '/:' '_') | |
| filename=$(echo "$image" | sha256sum | cut -d' ' -f1) |
| IMAGE="${ARTIFACT_REGISTRY}/kochava/integrations-testsuite:latest" | ||
| # Get digest without pulling the full image (only manifest) | ||
| # Extract the actual image digest from the manifest | ||
| DIGEST=$(docker manifest inspect ${IMAGE} | jq -r '.manifests[0].digest // .config.digest' | sed 's/sha256://') |
Copilot
AI
Dec 4, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The docker manifest inspect command output is piped to jq and sed without error checking. If the manifest inspection fails (e.g., due to network issues or authentication problems), the pipeline might produce an empty or invalid digest, which would be written to GITHUB_OUTPUT. This could cause cache key issues or failures in subsequent steps.
Add error checking after the pipeline:
DIGEST=$(docker manifest inspect ${IMAGE} | jq -r '.manifests[0].digest // .config.digest' | sed 's/sha256://')
if [ -z "$DIGEST" ] || [ "$DIGEST" = "null" ]; then
echo "Error: Failed to get image digest"
exit 1
fi
echo "digest=${DIGEST}" >> $GITHUB_OUTPUT| DIGEST=$(docker manifest inspect ${IMAGE} | jq -r '.manifests[0].digest // .config.digest' | sed 's/sha256://') | |
| DIGEST=$(docker manifest inspect ${IMAGE} | jq -r '.manifests[0].digest // .config.digest' | sed 's/sha256://') | |
| if [ -z "$DIGEST" ] || [ "$DIGEST" = "null" ]; then | |
| echo "Error: Failed to get image digest" | |
| exit 1 | |
| fi |
| - name: Get integrations-testsuite image digest | ||
| if: ${{ inputs.GO_TEST_INTEGRATION_ENABLED }} | ||
| id: image-digest | ||
| env: | ||
| ARTIFACT_REGISTRY: ${{ secrets.ARTIFACT_REGISTRY }} | ||
| run: | | ||
| IMAGE="${ARTIFACT_REGISTRY}/kochava/integrations-testsuite:latest" | ||
| # Get digest without pulling the full image (only manifest) | ||
| # Extract the actual image digest from the manifest | ||
| DIGEST=$(docker manifest inspect ${IMAGE} | jq -r '.manifests[0].digest // .config.digest' | sed 's/sha256://') | ||
| echo "digest=${DIGEST}" >> $GITHUB_OUTPUT |
Copilot
AI
Dec 4, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The docker manifest inspect command requires authentication when used with private registries, but this step runs before the Login to Artifact Registry step (lines 112-120) has authenticated. This will cause the command to fail with authentication errors for private images.
The authentication step should be moved before this step, or this step should be moved after the authentication step to ensure docker manifest inspect can access the private registry.
| IMAGE="${ARTIFACT_REGISTRY}/kochava/integrations-testsuite:latest" | ||
| # Get digest without pulling the full image (only manifest) | ||
| # Extract the actual image digest from the manifest | ||
| DIGEST=$(docker manifest inspect ${IMAGE} | jq -r '.manifests[0].digest // .config.digest' | sed 's/sha256://') |
Copilot
AI
Dec 4, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The jq query .manifests[0].digest // .config.digest assumes the manifest has a specific structure, but for multi-architecture images it may select an arbitrary architecture's digest (the first manifest). For single-architecture images, the fallback to .config.digest may not provide the correct image digest.
Consider using docker buildx imagetools inspect --raw which provides a more reliable way to get the image digest, or specify the platform explicitly to ensure consistent cache keys across runs.
| DIGEST=$(docker manifest inspect ${IMAGE} | jq -r '.manifests[0].digest // .config.digest' | sed 's/sha256://') | |
| DIGEST=$(docker buildx imagetools inspect --raw ${IMAGE} | grep -m1 '^Digest:' | awk '{print $2}' | sed 's/sha256://') |
No description provided.