Repository navigation
Conversation
55ba1c2 to
2a82ae0
Compare
|
Rebased onto current main (v0.2.7). The only overlap was the DeepSeek pictures change: the auto-pin line keeps its new |
30cadc8 to
d62913f
Compare
|
Rebased onto v0.2.10. Two overlaps, both kept:
Locally, |
a04e87d to
48d86c8
Compare
|
CI note: the two failing jobs fail the same way on
All other jobs pass. |
|
Thank you for this pull request, and for the CI note. It keeps a request's pictures in the cache from encoding until the combined file is copied, so another request cannot free them in between. It is not in 0.2.17. It changes how every request holds the picture cache, and 0.2.17 already changed the same Vision code (the picture reader fix from #5900), so it needs a rebase on 0.2.17 and a test with more than 64 pictures before we take it. The PR stays open, and we will reply here once that has been done. |
Keep the FIFO and a reentrant image batch lock through encoding, validation and combined-file copying, defer eviction to the outermost batch, and trim back to the 64-entry bound afterward. Close encoder pipes on teardown; retry a native decoder refusal once through the existing Pillow normalization. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s no batch() A vision encoder without batch() (the HistoryReferences test stub) now prepares the request without the batch lock, as encode_all() already falls back to encode().
Tests that build a reader with Vision.__new__ (test_vision_workdir) have no cache_batches, so encode() raised AttributeError. A class attribute keeps the counter at 0 until batch() sets it on the instance. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
48d86c8 to
c82afd0
Compare
|
I rebased onto current On Linux with Python 3.12, This run used the mock engine and picture reader. I did not run the case on a GPU. The GitHub checks for the rebased head are separate from these local results. |
Problem and result
The image cache keeps its records during
encode_all, but the service releases the FIFO before it prepares the combined embedding file. Another request can evict those files between encoding and the stat/copy step, so the first request fails or copies a missing file. A refused request with more than 64 cached images can also leave the cache above its bound.After this change, a request owns its images from encoding through the combined-file copy. Eviction waits for the outermost batch, and the cache trims back to 64 afterward. Encoder pipes close on teardown, and failed startup cleans up. A native decoder refusal retries once through the existing Pillow normalization path; conversion failures still remove their temporary files.
This replaces #2286 (closed in the 0.2.0-beta tracker reset), rebased on current
main.Related discussion
#2286 (this change on the old base, with the review note about
Vision.__new__tests). #1072 and #1207 fixed retention during encoding; this covers the remaining interval through the combined-file copy.Approach
A reentrant batch counter (
cache_batches) defers eviction until the outermost batch exits, so nestedencode_all/prepare calls keep ownership without a second lock. The FIFO stays held through validation and copying. The existingVisionCacheEvictiontests that buildVisionwith__new__now setcache_batches(the review note on #2286). No config, default or API change. The longer FIFO hold protects ownership; its throughput cost is unmeasured.Validation
Passed:
python -m unittest serve.test_vision_cache serve.test_server.VisionDecode serve.test_server.ImageMarkers serve.test_server.VisionTempFiles serve.test_server.VisionShutdown serve.test_server.VisionCacheEviction serve.test_vision_elasticon this branch (main4b9e05a2+ this commit), Linux, Python 3.14, CPU only: 30 tests OK. On stock4b9e05a2with the same tests: 16 failures and 2 errors (eviction before stat, eviction between stat and copy, out-of-order records for >64 images, untrimmed failure paths, the decoder retry).python -m unittest discover -s serve -p 'test_*.py'on this branch: 1046 tests, 1 failure (below), 12 skipped.Failed:
serve.test_keepawake.KeepAwakeTests.test_service_holds_for_the_whole_request_including_early_closefails intermittently on this branch and on stock4b9e05a2alike (2 of 3 runs on stock here). It is a timing test unrelated to vision.Not run:
Limits
The FIFO now holds longer per image request; I did not measure the throughput cost.
🤖 Generated with Claude Code