Test rpm package _before_ pushing to remote repo - #373
JackPGreen wants to merge 3 commits into
Conversation
2168c10 to
b4d2f8b
Compare
b4d2f8b to
623f215
Compare
JackPGreen
changed the title
rpm package _before_ pushing to remote repo
623f215 to
1240a41
Compare
1240a41 to
32564f4
Compare
8dfdd8f to
12293db
Compare
12293db to
81f6dc8
Compare
There was a problem hiding this comment.
we should enable JFrog cache again?
@ldziedziul agreed?
There was a problem hiding this comment.
I think it's still a bit racy and I'd suggest leaving it as-is.
There was a problem hiding this comment.
I think only when you fetch immediately after update metadata but not sure
moreover, the disabling of cache hasn't helped so we should revert
for some reason there was no issue with deb and I believe because it uses local (not virtual)
There was a problem hiding this comment.
@ldziedziul WDYT?
also happens in sandbox #373 (comment)
both envs use local repos so no a cache issue
| - name: Calculate YUM Repository Metadata | ||
| run: | | ||
| # https://docs.jfrog.com/artifactory/reference/calculateyumrepositorymetadata | ||
| jf api --method POST "/artifactory/api/yum/${RPM_REPO}" |
There was a problem hiding this comment.
With this we don't test the install from the remote repo at all anymore (.repo file, recalculated metadata, repo_gpgcheck). Fair enough for gating the upload, but shouldn't we keep it as a non-blocking step at the end (continue-on-error: true, maybe with a short retry loop) so a broken repo is still visible in the run? WDYT?
There was a problem hiding this comment.
fyi https://hazelcast.atlassian.net/browse/DI-853
eventually remote will be tested
no harm keeping it here though but if its intermittent, then might be confusing
may be install from rpm-local?
I thought issue was with virtual repo
There was a problem hiding this comment.
With this we don't test the install from the remote repo at all anymore (
.repofile, recalculated metadata,repo_gpgcheck). Fair enough for gating the upload, but shouldn't we keep it as a non-blocking step at the end (continue-on-error: true, maybe with a short retry loop) so a broken repo is still visible in the run? WDYT?
I agree it would be nice to validate the image is actually usable - but we don't do that in Docker, simply trust our publishing tooling.
However, if we don't have stable tooling in place to support publish-and-test we'll end up with a flakey, often-failing, always-ignored test that adds no value... even if it's flagging a real problem.
I have concerns about this - lets discuss in refinement next time.
if its intermittent, then might be confusing may be install from
rpm-local? I thought issue was withvirtualrepo
Unfortunately, we don't know what the cause is - only that I can't reproduce in sandbox.
I've noticed the repository metadata recalculation takes variable amounts of time - <1 minute in sandbox, <3 minutes in sandbox when a genuinely new package is pushed, and >4 minutes in live when a new package is pushed.
I suspect the problem is that there's some background task that takes longer with a larger/longer-lived repo and that doesn't complete in time, leading to the out-of-date metadata. But without being able to reproduce, it's just speculation.
There was a problem hiding this comment.
I checked and we are using rpm-local in live
metadata update request is synchronous so likely some (recent) bug/issue on JFrog side. Its been working before
| ./build-hazelcast-rpm-package.sh | ||
| env: | ||
| JAVA_VERSION: ${{ inputs.JAVA_VERSION }} | ||
| source ./common.sh |
There was a problem hiding this comment.
Why delete the script rather than just drop jf rt upload from it and have it write echo "file=${RPM_FILE}" >> "${GITHUB_OUTPUT}"? The path is derivable from common.sh anyway. deb and brew still use build-hazelcast-*.sh, so rpm becomes the odd one out. WDYT?
There was a problem hiding this comment.
Why delete the script rather than just drop
jf rt uploadfrom it and have it writeecho "file=${RPM_FILE}" >> "${GITHUB_OUTPUT}"?
This was my first approach, but for some reason the variable wasn't accessible.
But I think my new approach is easier to reason - both while developing, or reading logs.
The path is derivable from
common.shanyway. deb and brew still usebuild-hazelcast-*.sh, so rpm becomes the odd one out. WDYT?
I think those should change too, but trying to avoid changing everything at once.
|
|
||
| jf rt upload \ | ||
| "${{ steps.create-rpm-package.outputs.file }}" \ | ||
| "${RPM_REPO}/${RELEASE_CHANNEL}/${HZ_DISTRIBUTION}-${RPM_PACKAGE_VERSION}.noarch.rpm" |
There was a problem hiding this comment.
nit: we already have the filename in the step output, couldn't we use $(basename "${{ steps.create-rpm-package.outputs.file }}") here (and in the delete step) instead of rebuilding it?
There was a problem hiding this comment.
I think it would be better if it was a constant env, but that blends into refactoring away common.sh which I'm trying to avoid doing at the same time.
|
This branch was successfully deployed
3 participants
Currently (for both regular builds and PRs) we:
This has two issues:
Hazelcast-Stable-RPM: using metadata from Tue Sep 8 01:28:49 2026in logs which is >12 hours out of date)Instead, we should test
rpmpackage before pushing to remote repo - which sidesteps the whole availability issue.Changes:
build-hazelcast-rpm-package.shto allow (local) RPM package location to be derived