perf: improve varint writing performance

[anthony-swirldslabs]

Description:
Improving parts of varint writing code in PBJ in addition to the improvement already delivered at #614

Also, adding benchmarks for varints.

Most of the code is extracted from a draft PR at #612 with minor modifications to make it compile and to simplify running the benchmarks.

Results from the 4 benchmarks:

Benchmark                          (numOfBytes)  Mode  Cnt  Score   Error  Units
VarInt32SizeOfBench.google                    1  avgt    4  0.384 ± 0.008  ns/op
VarInt32SizeOfBench.google                    2  avgt    4  0.383 ± 0.003  ns/op
VarInt32SizeOfBench.google                    4  avgt    4  0.383 ± 0.006  ns/op
VarInt32SizeOfBench.kafka                     1  avgt    4  0.463 ± 0.011  ns/op
VarInt32SizeOfBench.kafka                     2  avgt    4  0.460 ± 0.001  ns/op
VarInt32SizeOfBench.kafka                     4  avgt    4  0.460 ± 0.003  ns/op
VarInt32SizeOfBench.pbjProtoTools             1  avgt    4  0.187 ± 0.002  ns/op
VarInt32SizeOfBench.pbjProtoTools             2  avgt    4  0.516 ± 0.061  ns/op
VarInt32SizeOfBench.pbjProtoTools             4  avgt    4  0.511 ± 0.007  ns/op

Benchmark                          (numOfBytes)  Mode  Cnt  Score   Error  Units
VarInt64SizeOfBench.google                    1  avgt    4  0.381 ± 0.013  ns/op
VarInt64SizeOfBench.google                    2  avgt    4  0.381 ± 0.018  ns/op
VarInt64SizeOfBench.google                    4  avgt    4  0.382 ± 0.006  ns/op
VarInt64SizeOfBench.google                    8  avgt    4  0.382 ± 0.006  ns/op
VarInt64SizeOfBench.kafka                     1  avgt    4  0.716 ± 0.017  ns/op
VarInt64SizeOfBench.kafka                     2  avgt    4  0.716 ± 0.017  ns/op
VarInt64SizeOfBench.kafka                     4  avgt    4  0.713 ± 0.004  ns/op
VarInt64SizeOfBench.kafka                     8  avgt    4  0.712 ± 0.007  ns/op
VarInt64SizeOfBench.pbjProtoTools             1  avgt    4  0.187 ± 0.002  ns/op
VarInt64SizeOfBench.pbjProtoTools             2  avgt    4  0.513 ± 0.011  ns/op
VarInt64SizeOfBench.pbjProtoTools             4  avgt    4  0.512 ± 0.017  ns/op
VarInt64SizeOfBench.pbjProtoTools             8  avgt    4  0.512 ± 0.005  ns/op

Benchmark                                     (numOfBytes)  Mode  Cnt   Score   Error  Units
VarIntReaderBench.dataBufferDirectRead                   1  avgt    5   1.605 ± 0.009  ns/op
VarIntReaderBench.dataBufferDirectRead                   2  avgt    5   1.607 ± 0.012  ns/op
VarIntReaderBench.dataBufferDirectRead                   4  avgt    5   1.605 ± 0.007  ns/op
VarIntReaderBench.dataBufferDirectRead                   8  avgt    5   1.605 ± 0.010  ns/op
VarIntReaderBench.dataBufferGet                          1  avgt    5   1.659 ± 0.033  ns/op
VarIntReaderBench.dataBufferGet                          2  avgt    5   1.658 ± 0.022  ns/op
VarIntReaderBench.dataBufferGet                          4  avgt    5   1.657 ± 0.006  ns/op
VarIntReaderBench.dataBufferGet                          8  avgt    5   1.658 ± 0.010  ns/op
VarIntReaderBench.dataBufferRead                         1  avgt    5   1.671 ± 0.007  ns/op
VarIntReaderBench.dataBufferRead                         2  avgt    5   1.679 ± 0.058  ns/op
VarIntReaderBench.dataBufferRead                         4  avgt    5   1.671 ± 0.015  ns/op
VarIntReaderBench.dataBufferRead                         8  avgt    5   1.673 ± 0.018  ns/op
VarIntReaderBench.dataBytesGet                           1  avgt    5   2.763 ± 0.042  ns/op
VarIntReaderBench.dataBytesGet                           2  avgt    5   2.759 ± 0.022  ns/op
VarIntReaderBench.dataBytesGet                           4  avgt    5   2.746 ± 0.051  ns/op
VarIntReaderBench.dataBytesGet                           8  avgt    5   2.782 ± 0.100  ns/op
VarIntReaderBench.dataNonSyncInputStreamRead             1  avgt    5   3.714 ± 0.415  ns/op
VarIntReaderBench.dataNonSyncInputStreamRead             2  avgt    5   3.655 ± 0.151  ns/op
VarIntReaderBench.dataNonSyncInputStreamRead             4  avgt    5   3.717 ± 0.135  ns/op
VarIntReaderBench.dataNonSyncInputStreamRead             8  avgt    5   3.719 ± 0.117  ns/op
VarIntReaderBench.dataSyncInputStreamRead                1  avgt    5  32.161 ± 0.244  ns/op
VarIntReaderBench.dataSyncInputStreamRead                2  avgt    5  32.585 ± 0.570  ns/op
VarIntReaderBench.dataSyncInputStreamRead                4  avgt    5  32.506 ± 0.862  ns/op
VarIntReaderBench.dataSyncInputStreamRead                8  avgt    5  32.408 ± 2.324  ns/op
VarIntReaderBench.googleDirecRead                        1  avgt    5   2.512 ± 0.156  ns/op
VarIntReaderBench.googleDirecRead                        2  avgt    5   2.486 ± 0.030  ns/op
VarIntReaderBench.googleDirecRead                        4  avgt    5   2.496 ± 0.112  ns/op
VarIntReaderBench.googleDirecRead                        8  avgt    5   2.497 ± 0.043  ns/op
VarIntReaderBench.googleRead                             1  avgt    5   1.431 ± 0.011  ns/op
VarIntReaderBench.googleRead                             2  avgt    5   1.430 ± 0.009  ns/op
VarIntReaderBench.googleRead                             4  avgt    5   1.429 ± 0.011  ns/op
VarIntReaderBench.googleRead                             8  avgt    5   1.430 ± 0.006  ns/op
VarIntReaderBench.googleSlowPathDirectRead               1  avgt    5   2.955 ± 0.050  ns/op
VarIntReaderBench.googleSlowPathDirectRead               2  avgt    5   2.962 ± 0.035  ns/op
VarIntReaderBench.googleSlowPathDirectRead               4  avgt    5   2.988 ± 0.118  ns/op
VarIntReaderBench.googleSlowPathDirectRead               8  avgt    5   2.958 ± 0.085  ns/op
VarIntReaderBench.googleSlowPathRead                     1  avgt    5   3.468 ± 0.026  ns/op
VarIntReaderBench.googleSlowPathRead                     2  avgt    5   3.476 ± 0.037  ns/op
VarIntReaderBench.googleSlowPathRead                     4  avgt    5   3.475 ± 0.053  ns/op
VarIntReaderBench.googleSlowPathRead                     8  avgt    5   3.463 ± 0.035  ns/op
VarIntReaderBench.richardGet                             1  avgt    5   1.607 ± 0.096  ns/op
VarIntReaderBench.richardGet                             2  avgt    5   1.600 ± 0.026  ns/op
VarIntReaderBench.richardGet                             4  avgt    5   1.615 ± 0.074  ns/op
VarIntReaderBench.richardGet                             8  avgt    5   1.737 ± 0.532  ns/op

Benchmark                                      (numOfBytes)  Mode  Cnt   Score   Error  Units
VarIntWriterBench.googleCodedByteArray                    4  avgt    5   3.495 ± 0.042  ns/op
VarIntWriterBench.googleCodedByteBufferDirect             4  avgt    5   3.041 ± 0.073  ns/op
VarIntWriterBench.googleCodedOutputStream                 4  avgt    5   5.035 ± 0.075  ns/op
VarIntWriterBench.kafkaByteBuffer                         4  avgt    5   5.576 ± 0.074  ns/op
VarIntWriterBench.pbjBufferedData                         4  avgt    5   5.408 ± 0.043  ns/op
VarIntWriterBench.pbjBufferedDataDirect                   4  avgt    5   5.103 ± 0.035  ns/op
VarIntWriterBench.pbjWritableStreamingData                4  avgt    5  39.237 ± 0.537  ns/op
VarIntWriterBench.richardStartinByteArray                 4  avgt    5   1.372 ± 0.012  ns/op

Related issue(s):

Fixes #596

Notes for reviewer:
All tests should pass.

Checklist

• Documented (Code comments, README, etc.)
• Tested (unit, integration, etc.)

[github-actions]

JUnit Test Report

   77 files  ±0     77 suites  ±0   2m 17s ⏱️ -54s
1 328 tests ±0  1 324 ✅ ±0   4 💤 ±0  0 ❌ ±0 
7 185 runs  ±0  7 165 ✅ ±0  20 💤 ±0  0 ❌ ±0 

Results for commit 278d3a1. ± Comparison against base commit fd93a3b.

This pull request removes 8 and adds 7 tests. Note that renamed tests count towards both.

, 1
com.hedera.pbj.runtime.ProtoWriterToolsTest ‑ [1] FLOAT, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c0449710@43201f84, [0.1, 0.5, 100.0], 12, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c0449920@2c51c756
com.hedera.pbj.runtime.ProtoWriterToolsTest ‑ [1] STRING, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c0456688@10280879, [string 1, testing here, testing there], com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c0456898@3472f3ab
com.hedera.pbj.runtime.ProtoWriterToolsTest ‑ [2] BYTES, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c0456aa8@37a939bc, [010203, ff7f0f, 42da07370bff], com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c0456cb8@74d776fb
com.hedera.pbj.runtime.ProtoWriterToolsTest ‑ [2] DOUBLE, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c0449b30@53f1fcc2, [0.1, 0.5, 100.0, 1.7653472635472653E240], 32, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c0449d40@6a261998
com.hedera.pbj.runtime.ProtoWriterToolsTest ‑ [3] BOOL, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c0449f50@49770ef9, [true, false, false, true, true, true], 6, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c044a160@15bcecf9
com.hedera.pbj.runtime.ProtoWriterToolsTest ‑ [4] ENUM, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c044a370@22ea6051, [Mock for EnumWithProtoMetadata, hashCode: 373462668, Mock for EnumWithProtoMetadata, hashCode: 1482874276, Mock for EnumWithProtoMetadata, hashCode: 698212351], 3, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c044a580@430db481
com.hedera.pbj.runtime.Utf8ToolsTest ‑ [4] 

com.hedera.pbj.runtime.ProtoWriterToolsTest ‑ [1] FLOAT, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c044c638@49770ef9, [0.1, 0.5, 100.0], 12, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c044c848@15bcecf9
com.hedera.pbj.runtime.ProtoWriterToolsTest ‑ [1] STRING, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c04556c0@c3ca482, [string 1, testing here, testing there], com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c04558d0@6b338591
com.hedera.pbj.runtime.ProtoWriterToolsTest ‑ [2] BYTES, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c0455ae0@786cc1df, [010203, ff7f0f, 42da07370bff], com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c0455cf0@1ddb6abb
com.hedera.pbj.runtime.ProtoWriterToolsTest ‑ [2] DOUBLE, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c044ca58@27b7e663, [0.1, 0.5, 100.0, 1.7653472635472653E240], 32, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c044cc68@5f2ad3d5
com.hedera.pbj.runtime.ProtoWriterToolsTest ‑ [3] BOOL, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c044ce78@674fd531, [true, false, false, true, true, true], 6, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c044d088@7f53b345
com.hedera.pbj.runtime.ProtoWriterToolsTest ‑ [4] ENUM, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c044d298@71817f66, [Mock for EnumWithProtoMetadata, hashCode: 1938741493, Mock for EnumWithProtoMetadata, hashCode: 274720798, Mock for EnumWithProtoMetadata, hashCode: 184109835], 3, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000007c044d4a8@a08e41b
com.hedera.pbj.runtime.Utf8ToolsTest ‑ [4] 
, 1

♻️ This comment has been updated with latest results.

[github-actions]

Integration Test Report

    405 files  ±0      405 suites  ±0   17m 15s ⏱️ + 2m 41s
114 836 tests ±0  114 836 ✅ ±0  0 💤 ±0  0 ❌ ±0 
115 077 runs  ±0  115 077 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 278d3a1. ± Comparison against base commit fd93a3b.

This pull request removes 3 and adds 3 tests. Note that renamed tests count towards both.

com.hedera.pbj.integration.test.ParserNeverWrapsTest ‑ [1] com.hedera.pbj.integration.test.ParserNeverWrapsTest$$Lambda/0x00007f8a90b86f68@71490286
com.hedera.pbj.integration.test.ParserNeverWrapsTest ‑ [2] com.hedera.pbj.integration.test.ParserNeverWrapsTest$$Lambda/0x00007f8a90b87198@16f5c321
com.hedera.pbj.integration.test.ParserNeverWrapsTest ‑ [3] com.hedera.pbj.integration.test.ParserNeverWrapsTest$$Lambda/0x00007f8a90b873c8@52c4d48d

com.hedera.pbj.integration.test.ParserNeverWrapsTest ‑ [1] com.hedera.pbj.integration.test.ParserNeverWrapsTest$$Lambda/0x00007fafb0b4f968@be13b7a
com.hedera.pbj.integration.test.ParserNeverWrapsTest ‑ [2] com.hedera.pbj.integration.test.ParserNeverWrapsTest$$Lambda/0x00007fafb0b4fb98@556e1e
com.hedera.pbj.integration.test.ParserNeverWrapsTest ‑ [3] com.hedera.pbj.integration.test.ParserNeverWrapsTest$$Lambda/0x00007fafb0b4fdc8@53f3b2b9

♻️ This comment has been updated with latest results.

[abies]

For the sake of interest, can you please attach the results from the jhm runs?

[anthony-swirldslabs]

@anies :

For the sake of interest, can you please attach the results from the jhm runs?

Good idea. I've updated the Description of this PR with the results. Please have a look. An uber comment for the results is that we're trying to optimize the most frequent case of 1 byte varint - this is because most field tags in protobuf models only occupy 1 byte, so we want to be super efficient when processing such short varints.