Explore IBufferWriter / IBufferDistributedCache support for the L2 path

[slang25]

Draft, mostly to get some feedback on the direction before I take it any further. Related to #371 and #528, and there's already a prototype in #593 from @Inok that I leaned on for ideas 👍

The idea is to skip the intermediate byte[] on the L2 path when both sides can do better:

• New opt-in IBufferFusionCacheSerializer : IFusionCacheSerializer, with Serialize<T>(T, IBufferWriter<byte>) and Deserialize<T>(in ReadOnlySequence<byte>). Sync-only, since netstandard2.0 rules out default interface methods (same call HybridCache made).
• DistributedCacheAccessor picks the buffered path at runtime only when the L2 is an IBufferDistributedCache and the serializer implements the new interface, otherwise it falls back to the existing byte[] path.
• All six bundled serializers implement it: System.Text.Json, MemoryPack, MessagePack and protobuf-net through their native buffer APIs, Newtonsoft and ServiceStack through two new Stream adapters (BufferWriterStream / ReadOnlySequenceStream).
• FusionCacheOptions.DistributedCacheBufferPool controls the pool, defaulting to ArrayPool<byte>.Shared.

On my machine (M2 Pro, .NET 10, STJ) a buffered Set drops essentially all of the payload allocation (103KB → 1.35KB, and 4.3MB → 1.38KB), runs 5-10% faster and has no Gen2, while a buffered Get loses the payload-sized array (−27%). Both paths produce byte-identical output, so mixed fleets and rollbacks are safe, and there are tests covering that in both directions.

Outstanding actions:

• the Chaos package has no buffer-aware wrappers yet
• no IHybridCacheSerializer<T> bridging
• buffer lifetime on the failure path is dispose-on-success / leak-on-failure: a soft timeout can abandon an operation that is still writing into the buffer, so returning it to the pool didn't feel safe. I'd really like a second opinion on that bit.

So it's not bulletproof yet. Is this a direction you'd welcome, or is there a better shape for it?

[jodydonetti]

Hi @slang25 , first of all thanks!

Draft, mostly to get some feedback on the direction before I take it any further.

Spoiler: this looks really good.

Related to #371 and #528, and there's already a prototype in #593 from @Inok that I leaned on for ideas 👍

I just updated @Inok here.
I'll take a look at both PRs and see the contact points, then I'll update both of you.
In any case if we proceed to include this in FusionCache (quite likely) I'll make sure to have credits for both of you front and center.

The idea is to skip the intermediate byte[] on the L2 path when both sides can do better:

• New opt-in IBufferFusionCacheSerializer : IFusionCacheSerializer, with Serialize<T>(T, IBufferWriter<byte>) and Deserialize<T>(in ReadOnlySequence<byte>).
• DistributedCacheAccessor picks the buffered path at runtime only when the L2 is an IBufferDistributedCache and the serializer implements the new interface, otherwise it falls back to the existing byte[] path.

That's a great approach, and fwiw it's the same design I was thinking about: an additional opt-in interface that derives from IFusionCacheSerializer 👍

Sync-only, since netstandard2.0 rules out default interface methods (same call HybridCache made).

I need to take a better look at this.

• All six bundled serializers implement it: System.Text.Json, MemoryPack, MessagePack and protobuf-net through their native buffer APIs, Newtonsoft and ServiceStack through two new Stream adapters (BufferWriterStream / ReadOnlySequenceStream).

Great!

• FusionCacheOptions.DistributedCacheBufferPool controls the pool, defaulting to ArrayPool<byte>.Shared.

At first sight, not sure about exposing such an "internal detail", but I guess it can probably make sense.
Let me think about it.

On my machine (M2 Pro, .NET 10, STJ) a buffered Set drops essentially all of the payload allocation (103KB → 1.35KB, and 4.3MB → 1.38KB), runs 5-10% faster and has no Gen2, while a buffered Get loses the payload-sized array (−27%). Both paths produce byte-identical output, so mixed fleets and rollbacks are safe, and there are tests covering that in both directions.

This sounds honestly glorious.

Outstanding actions:

• the Chaos package has no buffer-aware wrappers yet

Not sure it would necessarily need it, thoughts?

• no IHybridCacheSerializer<T> bridging

That was not planned, care to elaborate about this?

• buffer lifetime on the failure path is dispose-on-success / leak-on-failure: a soft timeout can abandon an operation that is still writing into the buffer, so returning it to the pool didn't feel safe. I'd really like a second opinion on that bit.

I'll take a look at the code, but of course in general we should try to avoid leaks as much as possible.

So it's not bulletproof yet. Is this a direction you'd welcome, or is there a better shape for it?

Definitely the right path. Again, fwiw the approach is the same I was thinking about, although not yet in such a thought out way.

During this weekend I'll try to take a good deep look at it, then I'll let (both of) you know.

Thanks!

PS: one of the very first task on my backlog after finally releasing my course was a (potentially) great performance boost I am thinking about for some time now, but... that would require a bit of internal refactoring, and that in turn would probably not play so well with PRs, merges & friends so I'll wait a bit before doing it, I have no rush for that.