Is there an existing issue for the same bug?
Branch Name
main / PR #25089 branch issue-24951-main
Commit ID
ae752dc
Other Environment Information
- Hardware parameters: GitHub-hosted runner
- OS type: Ubuntu / x86
- Others: Matrixone CI unit-test job
Actual Behavior
The Ubuntu/x86 unit-test job failed in TestGlobalCheckpoint2.
After ForceCheckpoint and ForceGlobalCheckpoint, the test waits for
AllCheckpointsFinished() and then immediately asserts that the scheduler
pending LSN count is zero. In this run, both assertions observed one pending
LSN:
pkg/vm/engine/tae/db/test/db_test.go:8248
expected: 0x0
actual : 0x1
pkg/vm/engine/tae/db/test/db_test.go:8254
expected: 0x0
actual : 0x1
Test: TestGlobalCheckpoint2
CI run:
https://github.com/matrixorigin/matrixone/actions/runs/29799310841/job/88537842971
This is unrelated to PR #25089: that PR does not modify
pkg/vm/engine/tae, checkpoint, WAL, or scheduler code. The same test has
also failed historically in #20008 and #21721.
Expected Behavior
TestGlobalCheckpoint2 should deterministically wait until all
checkpoint-related asynchronous work, including pending LSN processing, has
completed before asserting the pending count.
Steps to Reproduce
.agents/skills/mo-dev/scripts/mo-cgo-test -v -count=10 \
-run '^TestGlobalCheckpoint2$' -timeout=180s \
./pkg/vm/engine/tae/db/test
The local stress run passed 10/10 at the same PR head, indicating a
low-probability timing-dependent failure.
Additional information
The test currently waits for AllCheckpointsFinished(), but that condition
did not guarantee Runtime.Scheduler.GetPenddingLSNCnt() == 0 in the failing
CI run. Consider waiting explicitly for the pending LSN count or tightening
the completion contract so the assertion cannot race asynchronous scheduler
cleanup.
Is there an existing issue for the same bug?
Branch Name
main / PR #25089 branch
issue-24951-mainCommit ID
ae752dc
Other Environment Information
Actual Behavior
The Ubuntu/x86 unit-test job failed in
TestGlobalCheckpoint2.After
ForceCheckpointandForceGlobalCheckpoint, the test waits forAllCheckpointsFinished()and then immediately asserts that the schedulerpending LSN count is zero. In this run, both assertions observed one pending
LSN:
CI run:
https://github.com/matrixorigin/matrixone/actions/runs/29799310841/job/88537842971
This is unrelated to PR #25089: that PR does not modify
pkg/vm/engine/tae, checkpoint, WAL, or scheduler code. The same test hasalso failed historically in #20008 and #21721.
Expected Behavior
TestGlobalCheckpoint2should deterministically wait until allcheckpoint-related asynchronous work, including pending LSN processing, has
completed before asserting the pending count.
Steps to Reproduce
The local stress run passed 10/10 at the same PR head, indicating a
low-probability timing-dependent failure.
Additional information
The test currently waits for
AllCheckpointsFinished(), but that conditiondid not guarantee
Runtime.Scheduler.GetPenddingLSNCnt() == 0in the failingCI run. Consider waiting explicitly for the pending LSN count or tightening
the completion contract so the assertion cannot race asynchronous scheduler
cleanup.