Skip to content

Add System.Diagnostics.Metrics for per-connection Zebus monitoring via net10.0 multi-targeting - #120

Open
Kuinox with Copilot wants to merge 22 commits into
masterfrom
copilot/add-metrics-for-zebus-connections
Open

Kuinox with Copilot wants to merge 22 commits into
masterfrom
copilot/add-metrics-for-zebus-connections

Conversation

Copilot AI commented Apr 24, 2026 •

Copy link
Copy Markdown

Adds System.Diagnostics.Metrics instrumentation to monitor Zebus connection state and messaging, gated behind #if NET10_0_OR_GREATER with no new NuGet dependencies.

Metrics

All per-connection metrics are tagged with zebus.peer.id for individual peer filtering:

Metric Type Description
zebus.transport.messages.sent Counter (per-peer) Messages sent
zebus.transport.messages.received Counter (per-peer) Messages received
zebus.transport.bytes.sent Counter (per-peer) Bytes sent
zebus.transport.bytes.received Counter (per-peer) Bytes received
zebus.transport.messages.send_failures Counter (per-peer) Send failures
zebus.transport.peer_connections Counter (per-peer) Socket connects
zebus.transport.peer_disconnections Counter (per-peer) Socket disconnects
zebus.transport.peer_connection_failures Counter (per-peer) Connect failures
zebus.transport.connection.alive ObservableGauge (per-peer) 1=alive, 0=dead
zebus.transport.outbound_sockets UpDownCounter Active socket count
zebus.directory.known_peers UpDownCounter Known peer count
zebus.directory.peer_updates Counter Peer update events
// Meter name: "Abc.Zebus"
// Example: filter messages sent to a specific peer
// dotnet-counters monitor --counters "Abc.Zebus[zebus.peer.id=MyPeer.0]"

Metric class organization

Instruments are split into dedicated classes per service:

  • ZebusMetrics — shared Meter instance and PeerTag helper
  • TransportMetrics — transport-layer instruments (messages, bytes, connections, sockets)
  • DirectoryMetrics — directory-layer instruments (known peers, peer updates)

Multi-targeting

  • Abc.Zebus.csproj targets netstandard2.0;net10.0
  • Metrics code is entirely behind #if NET10_0_OR_GREATER; netstandard2.0 output is unchanged

.NET 10 compatibility fixes

Polyfill types that now exist in the BCL needed conditional compilation:

  • NullableAnnotations, IsExternalInit, SkipLocalsInitAttribute — wrapped in #if !NETCOREAPP
  • ExtendDictionary.GetValueOrDefault / ExtendIEnumerable.ToHashSet — excluded on NETCOREAPP to resolve ambiguities with BCL extension methods
  • ProtoBufConvert — FormatterServices.GetUninitializedObject → RuntimeHelpers.GetUninitializedObject
  • MessageId — RNGCryptoServiceProvider → RandomNumberGenerator.Fill
  • MessageProcessingException / DomainException — serialization constructors excluded on NETCOREAPP (SYSLIB0051)
  • ProtoBufferReader.TryReadGuid — fixed definite assignment of out parameter
  • BindingKey — replaced extension method call with explicit TryGetValue to avoid ambiguity

Comment thread src/Abc.Zebus/Routing/BindingKey.cs
@Kuinox

Kuinox commented Apr 28, 2026

Copy link
Copy Markdown

@copilot we may have multiple bus at the same time, having a active bus count metrics isn't very useful, but, we will want to track per bus instance.
You can do a static interlocked increment starting at 0 on the instance to get a stable id.

Copilot AI commented Apr 28, 2026 •

Copy link
Copy Markdown
Author

@copilot we may have multiple bus at the same time, having a active bus count metrics isn't very useful, but, we will want to track per bus instance.
You can do a static interlocked increment startin...

Done in e04af38. Bus now has a static _nextInstanceId incremented via Interlocked.Increment, and the zebus.bus.active UpDownCounter is tagged with zebus.bus.id per instance instead of being a global aggregate.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • REDACTED-bastard
    • Triggering command: /usr/share/dotnet/dotnet /usr/share/dotnet/dotnet exec --runtimeconfig /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/Abc.Zebus.Tests.runtimeconfig.json --depsfile /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/Abc.Zebus.Tests.deps.json /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/testhost.dll --port 35909 --endpoint 127.0.0.1:035909 --role client --parentprocessid 5536 --telemetryoptedin false (dns block)
  • non-existing-peer
    • Triggering command: /usr/share/dotnet/dotnet /usr/share/dotnet/dotnet exec --runtimeconfig /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/Abc.Zebus.Tests.runtimeconfig.json --depsfile /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/Abc.Zebus.Tests.deps.json /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/testhost.dll --port 35909 --endpoint 127.0.0.1:035909 --role client --parentprocessid 5536 --telemetryoptedin false (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI requested a review from Kuinox April 28, 2026 18:08

@Kuinox Kuinox left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread src/Abc.Zebus/Core/Bus.cs Outdated
Comment thread src/Abc.Zebus/Monitoring/ZebusMetrics.cs Outdated
Comment thread src/Abc.Zebus/Transport/ZmqTransport.cs Outdated
Comment thread src/Abc.Zebus/MessageId.cs
Comment thread src/Abc.Zebus/MessageProcessingException.cs Outdated

Copilot AI commented Apr 28, 2026

Copy link
Copy Markdown
Author

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • REDACTED-bastard
    • Triggering command: /usr/share/dotnet/dotnet /usr/share/dotnet/dotnet exec --runtimeconfig /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/Abc.Zebus.Tests.runtimeconfig.json --depsfile /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/Abc.Zebus.Tests.deps.json /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/testhost.dll --port 43095 --endpoint 127.0.0.1:043095 --role client --parentprocessid 5517 --telemetryoptedin false (dns block)
  • non-existing-peer
    • Triggering command: /usr/share/dotnet/dotnet /usr/share/dotnet/dotnet exec --runtimeconfig /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/Abc.Zebus.Tests.runtimeconfig.json --depsfile /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/Abc.Zebus.Tests.deps.json /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/testhost.dll --port 43095 --endpoint 127.0.0.1:043095 --role client --parentprocessid 5517 --telemetryoptedin false (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI requested a review from Kuinox April 28, 2026 18:25

@Kuinox Kuinox left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a latency on the heartbeat, or other means to measure latency ? It could be cool to add a latency metric. @copilot

Comment thread src/Abc.Zebus/Monitoring/TransportMetrics.cs Outdated
Comment thread src/Abc.Zebus/Transport/ZmqTransport.cs Outdated
…ndard, removing #ifdef on usings

Agent-Logs-Url: https://github.com/Abc-Arbitrage/Zebus/sessions/81a35d95-9277-416c-9f67-15fb83ff98cf

Co-authored-by: Kuinox <18743295+Kuinox@users.noreply.github.com>

Copilot AI commented Apr 28, 2026 •

Copy link
Copy Markdown
Author

Is there a latency on the heartbeat, or other means to measure latency ? It could be cool to add a latency metric. @copilot

The directory has a PingPeerCommand handler that tracks TimeSinceLastPing via a Stopwatch, but it only measures elapsed time since the last ping was received — not round-trip latency. There's no built-in RTT measurement in the current transport or directory code that could be exposed as a latency metric without adding new protocol-level instrumentation. Want me to add one, or is that out of scope for this PR?

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • REDACTED-bastard
    • Triggering command: /usr/share/dotnet/dotnet /usr/share/dotnet/dotnet exec --runtimeconfig /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/Abc.Zebus.Tests.runtimeconfig.json --depsfile /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/Abc.Zebus.Tests.deps.json /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/testhost.dll --port 41415 --endpoint 127.0.0.1:041415 --role client --parentprocessid 5105 --telemetryoptedin false (dns block)
  • non-existing-peer
    • Triggering command: /usr/share/dotnet/dotnet /usr/share/dotnet/dotnet exec --runtimeconfig /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/Abc.Zebus.Tests.runtimeconfig.json --depsfile /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/Abc.Zebus.Tests.deps.json /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/testhost.dll --port 41415 --endpoint 127.0.0.1:041415 --role client --parentprocessid 5105 --telemetryoptedin false (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI requested a review from Kuinox April 28, 2026 18:51
@Kuinox

Kuinox commented Apr 30, 2026

Copy link
Copy Markdown

@copilot I also want a byte count per message metric.

Copilot AI commented Apr 30, 2026 •

Copy link
Copy Markdown
Author

@copilot I also want a byte count per message metric.

Added in 4dd924d. Two new counters: zebus.transport.bytes.sent and zebus.transport.bytes.received, both tagged with zebus.peer.id. Sent bytes use bufferWriter.Position, received bytes use bufferReader.Length.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • REDACTED-bastard
    • Triggering command: /usr/share/dotnet/dotnet /usr/share/dotnet/dotnet exec --runtimeconfig /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/Abc.Zebus.Tests.runtimeconfig.json --depsfile /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/Abc.Zebus.Tests.deps.json /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/testhost.dll --port 37177 --endpoint 127.0.0.1:037177 --role client --parentprocessid 4414 --telemetryoptedin false (dns block)
  • non-existing-peer
    • Triggering command: /usr/share/dotnet/dotnet /usr/share/dotnet/dotnet exec --runtimeconfig /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/Abc.Zebus.Tests.runtimeconfig.json --depsfile /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/Abc.Zebus.Tests.deps.json /home/REDACTED/work/Zebus/Zebus/src/Abc.Zebus.Tests/bin/Debug/net9.0/testhost.dll --port 37177 --endpoint 127.0.0.1:037177 --role client --parentprocessid 4414 --telemetryoptedin false (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI commented Aug 18, 2026

Copy link
Copy Markdown
Author

@copilot The code is scattered with #if NET10_0_OR_GREATER. Make the metric APIs available for all target frameworks, and use #if NET inside the *Metrics classes. Basically, make the method no...

Done. The #if NET guards are now inside the *Metrics classes only. All call sites in ZmqTransport, ZmqOutboundSocket, and PeerDirectoryClient are unconditional — the methods are no-ops on netstandard2.0.

@ltrzesniewski

Copy link
Copy Markdown
Member

@copilot Move all the remaining metric-related stuff to the Abc.Zebus.Monitoring namespace. ALso, fix the formatting in your last commit. I don't want to see any usage of System.Diagnostics.Metrics in the existing code. Make wrappers if necessary.

Co-authored-by: ltrzesniewski <7913492+ltrzesniewski@users.noreply.github.com>

Copilot AI commented Aug 18, 2026

Copy link
Copy Markdown
Author

@copilot Move all the remaining metric-related stuff to the Abc.Zebus.Monitoring namespace. ALso, fix the formatting in your last commit. I don't want to see any usage of System.Diagnostics.Metrics...

Done. System.Diagnostics.Metrics types are now fully contained within the Abc.Zebus.Monitoring namespace. ObserveConnectionStates moved into TransportMetrics, ZmqTransport holds object? for the gauge reference. Formatting in ZmqOutboundSocket.cs fixed too.

@ltrzesniewski

Copy link
Copy Markdown
Member

@copilot make a wrapper type for the gauge instead of object?. Also, fix the formatting.

Co-authored-by: ltrzesniewski <7913492+ltrzesniewski@users.noreply.github.com>

Copilot AI commented Aug 18, 2026

Copy link
Copy Markdown
Author

@copilot make a wrapper type for the gauge instead of object?. Also, fix the formatting.

Done — ConnectionStateGauge is now a dedicated internal sealed class in TransportMetrics.cs (wrapping the ObservableGauge<int> under #if NET), replacing the object? field in ZmqTransport. Formatting fixed too.

@ltrzesniewski

Copy link
Copy Markdown
Member

@copilot In PeerDirectoryClient, extract a method for calls to PeerUpdated?.Invoke and DirectoryMetrics.AddPeerUpdates, since those are always executed together

Copilot AI and others added 2 commits August 18, 2026 12:10
Co-authored-by: ltrzesniewski <7913492+ltrzesniewski@users.noreply.github.com>
…yClient

Co-authored-by: ltrzesniewski <7913492+ltrzesniewski@users.noreply.github.com>

Copilot AI commented Aug 18, 2026

Copy link
Copy Markdown
Author

@copilot In PeerDirectoryClient, extract a method for calls to PeerUpdated?.Invoke and DirectoryMetrics.AddPeerUpdates, since those are always executed together

Done — extracted RaisePeerUpdated(PeerId peerId, PeerUpdateAction action) in PeerDirectoryClient. All five call sites now use it.

@ltrzesniewski

Copy link
Copy Markdown
Member

@ocoanet I'd like your opinion on this please, especially about the "static" aspect.

@ltrzesniewski
ltrzesniewski requested a review from ocoanet August 18, 2026 12:20
@ltrzesniewski
ltrzesniewski requested a review from Kuinox August 18, 2026 12:23
Kuinox and others added 7 commits September 23, 2026 15:35
- Group connections by PeerIdTag
- Use UpDownCounter instead of Gauge
- Remove total connected sockets (it can be derived from connected sockets)
- Use an observable metric with absolute values to ensure late-listeners receive updates
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants