Skip to content

[Bug]: TestDiskCacheReadEnsuresMemoryCacheCapacity leaks CachedData and flakes unrelated UTs #25840

Description

@LeftHandCold

Summary

TestDiskCacheReadEnsuresMemoryCacheCapacity leaves an allocator-backed CachedData object unreleased. The checked allocator reports the leak asynchronously from its finalizer, so a later and unrelated test such as TestDiskCacheQuotaExceeded can be reported as the failing test.

This occurred in the Ubuntu/x86 UT job for PR #25834: https://github.com/matrixorigin/matrixone/actions/runs/29564751079/job/87834783504?pr=25834

Root cause

pkg/fileservice/disk_cache_test.go:1121 allocates a 10-byte fscache.Data through AllocateCacheDataWithHint. DiskCache.Read stores that object in vec.Entries[0].CachedData and transfers result ownership to the caller's IOVector.

The caller-side release contract is IOVector.Release(), which releases both IOEntry.CachedData and any raw read buffer. TestDiskCacheReadEnsuresMemoryCacheCapacity verifies that CachedData was populated but returns without calling vec.Release().

The checked allocator attaches a finalizer to sampled allocations. When GC later collects the abandoned deallocator, the finalizer panics. This explains why the job labels TestDiskCacheQuotaExceeded as failed even though the allocation stack ends in TestDiskCacheReadEnsuresMemoryCacheCapacity.

The missing release was introduced with the test in #24387 (bdb848db22b468b98deedef227b948bbace68ffa). This is a test cleanup defect; the production DiskCache.Read ownership semantics are working as designed.

Impact

Confirmed impact:

  • FileService UT is flaky under the default sampled checked allocator.
  • The asynchronous finalizer can blame whichever test happens to run when GC occurs, obscuring the real source and causing unrelated PR CI failures.

No production behavior or cache ownership semantic is affected by the proposed fix.

Reproduction / evidence

On current main, run:

MO_MALLOC_DEBUG=1 GOFLAGS=-mod=mod go test ./pkg/fileservice \
  -run '^TestDiskCache(ReadEnsuresMemoryCacheCapacity|QuotaExceeded)$' -count=100

The process panics with missing free: ... size 10; the allocation stack points to TestDiskCacheReadEnsuresMemoryCacheCapacity.func1 at disk_cache_test.go:1121, followed by DiskCache.Read at line 1125.

The 10-byte size is exactly the value passed to AllocateCacheDataWithHint by that test. TestDiskCacheQuotaExceeded does not allocate CacheData through this path.

Proposed fix

Register vec.Release() immediately after constructing the result vector so cleanup also runs if a later assertion fails. Keep the production DiskCache.Read, IOEntry, allocator, and cache ownership logic unchanged.

Acceptance criteria

  • TestDiskCacheReadEnsuresMemoryCacheCapacity releases the caller-owned CachedData exactly once.
  • The two-test reproduction passes repeatedly with MO_MALLOC_DEBUG=1.
  • pkg/fileservice package tests pass.
  • Production FileService behavior and ownership semantics remain unchanged.

Metadata

Metadata

Assignees

Labels

kind/bugSomething isn't workingseverity/s0Active / top priority for current sprint. Owner has committed to working on it now.

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions