Skip to content

part: Transaction improvements - #208

Merged
joamaki merged 3 commits into
mainfrom
pr/joamaki/part-txn-fixes
Oct 13, 2026
Merged

joamaki merged 3 commits into
mainfrom
pr/joamaki/part-txn-fixes

Conversation

@joamaki

@joamaki joamaki commented Oct 12, 2026

Copy link
Copy Markdown
Contributor
  • Fix a minor issue with leaf.txnID() returning 0 leading to thinking that a leaf node doesn't need to be cloned. This could've caused a leaf watch channel to not close when the first transaction inserted the same key twice.
  • Reuse the grown deletion path buffer to reduce allocations and make sure we always clear it to not hold onto nodes.
  • Allocate the deleted parents buffer on demand rather than during newTxn
  • In the part iterator use a small inline buffer for edges to avoid allocation in shallow traversals

AIL:3

@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
$ make
go build ./...
go: downloading github.com/cilium/hive v1.0.4
go: downloading go.yaml.in/yaml/v3 v3.0.4
go: downloading golang.org/x/time v0.15.0
go: downloading github.com/spf13/cobra v1.10.2
go: downloading github.com/spf13/pflag v1.0.10
go: downloading github.com/cilium/stream v0.0.1
go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de
go: downloading github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc
go: downloading github.com/mitchellh/mapstructure v1.5.0
go: downloading go.uber.org/dig v1.17.1
go: downloading golang.org/x/term v0.16.0
go: downloading github.com/spf13/viper v1.18.2
go: downloading golang.org/x/sys v0.17.0
go: downloading golang.org/x/tools v0.17.0
go: downloading github.com/spf13/cast v1.6.0
go: downloading github.com/fsnotify/fsnotify v1.7.0
go: downloading github.com/sagikazarmark/slog-shim v0.1.0
go: downloading github.com/spf13/afero v1.11.0
go: downloading golang.org/x/text v0.14.0
go: downloading github.com/subosito/gotenv v1.6.0
go: downloading github.com/hashicorp/hcl v1.0.0
go: downloading gopkg.in/ini.v1 v1.67.0
go: downloading github.com/magiconair/properties v1.8.7
go: downloading github.com/pelletier/go-toml/v2 v2.1.0
go: downloading gopkg.in/yaml.v3 v3.0.1
STATEDB_VALIDATE=1 go test ./... -cover -vet=all -test.count 1
go: downloading github.com/stretchr/testify v1.11.1
go: downloading go.uber.org/goleak v1.3.0
go: downloading golang.org/x/exp v0.0.0-20240119083558-1b970713d09a
go: downloading github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2
ok  	github.com/cilium/statedb	325.853s	coverage: 79.6% of statements
ok  	github.com/cilium/statedb/index	0.005s	coverage: 48.6% of statements
ok  	github.com/cilium/statedb/internal	0.014s	coverage: 54.1% of statements
ok  	github.com/cilium/statedb/lpm	4.472s	coverage: 76.1% of statements
ok  	github.com/cilium/statedb/part	83.742s	coverage: 86.6% of statements
ok  	github.com/cilium/statedb/reconciler	0.303s	coverage: 93.3% of statements
	github.com/cilium/statedb/reconciler/benchmark		coverage: 0.0% of statements
	github.com/cilium/statedb/reconciler/example		coverage: 0.0% of statements
go test -race ./... -test.count 1
ok  	github.com/cilium/statedb	41.860s
ok  	github.com/cilium/statedb/index	1.014s
ok  	github.com/cilium/statedb/internal	1.042s
ok  	github.com/cilium/statedb/lpm	2.862s
ok  	github.com/cilium/statedb/part	38.241s
ok  	github.com/cilium/statedb/reconciler	1.455s
?   	github.com/cilium/statedb/reconciler/benchmark	[no test files]
?   	github.com/cilium/statedb/reconciler/example	[no test files]
go test ./... -bench . -benchmem -test.run xxx
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkDB_WriteTxn_1-4                      	  893239	      1239 ns/op	    807105 objects/sec	     632 B/op	      12 allocs/op
BenchmarkDB_WriteTxn_10-4                     	 2131884	       559.4 ns/op	   1787521 objects/sec	     349 B/op	       5 allocs/op
BenchmarkDB_WriteTxn_100-4                    	 2465965	       514.1 ns/op	   1945096 objects/sec	     317 B/op	       5 allocs/op
BenchmarkDB_WriteTxn_1000-4                   	 2334332	       515.0 ns/op	   1941700 objects/sec	     318 B/op	       5 allocs/op
BenchmarkDB_WriteTxn_100_SecondaryIndex-4     	 1507104	       795.9 ns/op	   1256436 objects/sec	     419 B/op	       7 allocs/op
BenchmarkDB_WriteTxn_CommitOnly_100Tables-4   	 1498098	       798.7 ns/op	    1088 B/op	       3 allocs/op
BenchmarkDB_WriteTxn_CommitOnly_1Table-4      	 2071094	       578.4 ns/op	     200 B/op	       3 allocs/op
BenchmarkDB_NewWriteTxn-4                     	 2312259	       521.3 ns/op	     200 B/op	       3 allocs/op
BenchmarkDB_WriteTxnCommit100-4               	 1522869	       788.0 ns/op	    1072 B/op	       3 allocs/op
BenchmarkDB_NewReadTxn-4                      	641526511	         1.870 ns/op	       0 B/op	       0 allocs/op
BenchmarkDB_Modify-4                          	    2154	    554742 ns/op	   1802641 objects/sec	  342485 B/op	    6072 allocs/op
BenchmarkDB_GetInsert-4                       	    1903	    630500 ns/op	   1586042 objects/sec	  326483 B/op	    6072 allocs/op
BenchmarkDB_ListInsert-4                      	    1478	    803685 ns/op	   1244268 objects/sec	  486514 B/op	   11072 allocs/op
BenchmarkDB_RandomInsert-4                    	    2270	    522411 ns/op	   1914202 objects/sec	  318476 B/op	    5072 allocs/op
BenchmarkDB_RandomReplace-4                   	    1183	   1007630 ns/op	    992427 objects/sec	  432300 B/op	    8088 allocs/op
BenchmarkDB_SequentialInsert-4                	    2245	    517122 ns/op	   1933780 objects/sec	  318477 B/op	    5072 allocs/op
BenchmarkDB_SequentialInsert_Prefix-4         	     540	   2259900 ns/op	    442497 objects/sec	 2793440 B/op	   41795 allocs/op
BenchmarkDB_Changes_Baseline-4                	    1867	    641280 ns/op	   1559381 objects/sec	  388220 B/op	    6162 allocs/op
BenchmarkDB_Changes-4                         	     991	   1205798 ns/op	    829326 objects/sec	  603949 B/op	    9323 allocs/op
BenchmarkDB_RandomLookup-4                    	   28387	     42443 ns/op	  23561028 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_SequentialLookup-4                	   36402	     32986 ns/op	  30315585 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_Prefix_SecondaryIndex-4           	    8445	    132322 ns/op	   7557302 objects/sec	  108952 B/op	      26 allocs/op
BenchmarkDB_LowerBound_SecondaryIndex-4       	    8990	    135591 ns/op	   7375109 objects/sec	  109240 B/op	      28 allocs/op
BenchmarkDB_FullIteration_All-4               	     801	   1452538 ns/op	  68845024 objects/sec	     104 B/op	       4 allocs/op
BenchmarkDB_FullIteration_Prefix-4            	     832	   1519002 ns/op	  65832694 objects/sec	     136 B/op	       5 allocs/op
BenchmarkDB_FullIteration_Get-4               	     225	   5283838 ns/op	  18925635 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_FullIteration_Get_Secondary-4     	     103	  11633561 ns/op	   8595820 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_FullIteration_ReadTxnGet-4        	     206	   5778482 ns/op	  17305582 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_PropagationDelay-4                	  872300	      1330 ns/op	        10.00 50th_µs	        14.00 90th_µs	        41.00 99th_µs	     755 B/op	      12 allocs/op
BenchmarkDB_WriteTxn_1_DefaultMetrics-4       	  935474	      1316 ns/op	     632 B/op	      12 allocs/op
BenchmarkDB_WriteTxn_1_ExpVarMetrics-4        	  714026	      1674 ns/op	     656 B/op	      15 allocs/op
BenchmarkDB_WriteTxn_100_LPMIndex-4           	  524848	      2351 ns/op	    425403 objects/sec	    1571 B/op	      35 allocs/op
BenchmarkDB_WriteTxn_1_LPMIndex-4             	  152239	     13644 ns/op	     73291 objects/sec	   13643 B/op	      64 allocs/op
BenchmarkDB_LPMIndex_Get-4                    	     402	   3075123 ns/op	   3251903 objects/sec	       0 B/op	       0 allocs/op
BenchmarkWatchSet_4-4                         	 2360458	       504.0 ns/op	     296 B/op	       4 allocs/op
BenchmarkWatchSet_16-4                        	  700884	      1548 ns/op	    1096 B/op	       5 allocs/op
BenchmarkWatchSet_128-4                       	   89139	     13497 ns/op	    8904 B/op	       5 allocs/op
BenchmarkWatchSet_1024-4                      	    8785	    134542 ns/op	   73743 B/op	       5 allocs/op
PASS
ok  	github.com/cilium/statedb	49.491s
PASS
ok  	github.com/cilium/statedb/index	0.004s
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb/internal
cpu: AMD EPYC 7763 64-Core Processor                
Benchmark_SortableMutex-4   	 6362112	       188.5 ns/op	       0 B/op	       0 allocs/op
PASS
ok  	github.com/cilium/statedb/internal	1.204s
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb/lpm
cpu: AMD EPYC 7763 64-Core Processor                
Benchmark_txn_insert/batchSize=1-4         	    1848	    638232 ns/op	   1566827 objects/sec	  822407 B/op	   13975 allocs/op
Benchmark_txn_insert/batchSize=10-4        	    3216	    371815 ns/op	   2689507 objects/sec	  369195 B/op	    6668 allocs/op
Benchmark_txn_insert/batchSize=100-4       	    3484	    346647 ns/op	   2884782 objects/sec	  329613 B/op	    6027 allocs/op
Benchmark_txn_delete/batchSize=1-4         	    1549	    815574 ns/op	   1226130 objects/sec	 1270475 B/op	   13976 allocs/op
Benchmark_txn_delete/batchSize=10-4        	    3248	    370226 ns/op	   2701050 objects/sec	  356419 B/op	    5769 allocs/op
Benchmark_txn_delete/batchSize=100-4       	    3468	    329915 ns/op	   3031085 objects/sec	  270754 B/op	    5038 allocs/op
Benchmark_LPM_Lookup-4                     	    9340	    127903 ns/op	   7818453 objects/sec	       0 B/op	       0 allocs/op
Benchmark_LPM_All-4                        	  134853	      8994 ns/op	 111182233 objects/sec	      32 B/op	       1 allocs/op
Benchmark_LPM_Prefix-4                     	  133364	      9028 ns/op	 110770047 objects/sec	      32 B/op	       1 allocs/op
Benchmark_LPM_LowerBound-4                 	  241626	      4895 ns/op	 102150606 objects/sec	     288 B/op	       2 allocs/op
PASS
ok  	github.com/cilium/statedb/lpm	12.000s
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb/part
cpu: AMD EPYC 7763 64-Core Processor                
Benchmark_Set_Singleton_Create-4              	21774256	        55.11 ns/op	      24 B/op	       1 allocs/op
Benchmark_Set_Singleton_Has-4                 	100000000	        11.27 ns/op	       0 B/op	       0 allocs/op
Benchmark_StringMap_Txn_Insert-4              	    7630	    149679 ns/op	   6680978 items/sec	   97738 B/op	    1305 allocs/op
Benchmark_Uint64Map_Random-4                  	    1892	    637872 ns/op	   1567712 items/sec	 1323522 B/op	    6044 allocs/op
Benchmark_Uint64Map_Sequential-4              	    2030	    603002 ns/op	   1658370 items/sec	 1702577 B/op	    5752 allocs/op
Benchmark_Uint64Map_Sequential_Insert-4       	    2206	    549135 ns/op	   1821045 items/sec	 1694573 B/op	    4751 allocs/op
Benchmark_Uint64Map_Sequential_Txn_Insert-4   	    9901	    119781 ns/op	   8348597 items/sec	   89952 B/op	    2030 allocs/op
Benchmark_Uint64Map_Random_Insert-4           	    2167	    558734 ns/op	   1789760 items/sec	 1312598 B/op	    5033 allocs/op
Benchmark_Uint64Map_Random_Txn_Insert-4       	    6308	    190089 ns/op	   5260685 items/sec	  116656 B/op	    2410 allocs/op
Benchmark_Insert_RootOnlyWatch-4              	    9966	    118580 ns/op	   8433145 objects/sec	   75040 B/op	    2035 allocs/op
Benchmark_Insert-4                            	    9843	    116057 ns/op	   8616487 objects/sec	   75584 B/op	    2046 allocs/op
Benchmark_WatchReplace-4                      	   10000	    104825 ns/op	   9539682 objects/sec	   57306 B/op	    1006 allocs/op
Benchmark_Modify-4                            	   12332	     97359 ns/op	  10271234 objects/sec	   58056 B/op	    1007 allocs/op
Benchmark_GetInsert-4                         	   10000	    115955 ns/op	   8624040 objects/sec	   58056 B/op	    1007 allocs/op
Benchmark_Replace-4                           	15903033	        74.51 ns/op	  13420676 objects/sec	      48 B/op	       1 allocs/op
Benchmark_Replace_RootOnlyWatch-4             	 8445102	       150.8 ns/op	   6632486 objects/sec	      88 B/op	       1 allocs/op
Benchmark_txn_1-4                             	 7802542	       152.5 ns/op	   6555863 objects/sec	      64 B/op	       3 allocs/op
Benchmark_txn_10-4                            	 9850712	       120.1 ns/op	   8324979 objects/sec	      76 B/op	       2 allocs/op
Benchmark_txn_100-4                           	11053016	       107.4 ns/op	   9311103 objects/sec	      67 B/op	       2 allocs/op
Benchmark_txn_1000-4                          	 9678577	       121.9 ns/op	   8206462 objects/sec	      65 B/op	       2 allocs/op
Benchmark_txn_delete_1-4                      	 4936452	       244.4 ns/op	   4091080 objects/sec	     632 B/op	       3 allocs/op
Benchmark_txn_delete_10-4                     	 9335412	       126.6 ns/op	   7900017 objects/sec	     103 B/op	       1 allocs/op
Benchmark_txn_delete_100-4                    	10763959	       104.7 ns/op	   9547149 objects/sec	      35 B/op	       1 allocs/op
Benchmark_txn_delete_1000-4                   	11423620	       104.3 ns/op	   9589950 objects/sec	      28 B/op	       1 allocs/op
BenchmarkDeleteParentCache/depth_8-4          	 4387662	       271.4 ns/op	      48 B/op	       1 allocs/op
BenchmarkDeleteParentCache/depth_100-4        	  456714	      2623 ns/op	      48 B/op	       1 allocs/op
Benchmark_Get-4                               	   60812	     19676 ns/op	  50823775 objects/sec	       0 B/op	       0 allocs/op
Benchmark_GetWatch-4                          	   47294	     25430 ns/op	  39323693 objects/sec	       0 B/op	       0 allocs/op
Benchmark_All-4                               	  141348	      8417 ns/op	 118808001 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Iterator_All-4                      	  157357	      7596 ns/op	 131645005 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Iterator_Next-4                     	  150582	      7937 ns/op	 125994045 objects/sec	     896 B/op	       1 allocs/op
Benchmark_LowerBound-4                        	   10000	    107679 ns/op	   9286822 objects/sec	  104000 B/op	    2000 allocs/op
Benchmark_LowerBound_Random-4                 	    7485	    155701 ns/op	   6422585 objects/sec	   96384 B/op	    1002 allocs/op
Benchmark_Hashmap_Insert-4                    	   14671	     81719 ns/op	  12237120 objects/sec	   74264 B/op	      20 allocs/op
Benchmark_Hashmap_Get_Uint64-4                	  137666	      8707 ns/op	 114845678 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Hashmap_Get_Bytes-4                 	  108068	     11041 ns/op	  90569967 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Delete_Random-4                     	      68	  18735464 ns/op	   5337471 objects/sec	 2539435 B/op	  102756 allocs/op
Benchmark_find16-4                            	225989640	         5.311 ns/op	       0 B/op	       0 allocs/op
Benchmark_findIndex16-4                       	85015240	        13.73 ns/op	       0 B/op	       0 allocs/op
Benchmark_find64-4                            	295349955	         4.058 ns/op	       0 B/op	       0 allocs/op
Benchmark_findIndex64_hit-4                   	295831776	         4.056 ns/op	       0 B/op	       0 allocs/op
Benchmark_findIndex64_miss-4                  	294755266	         4.064 ns/op	       0 B/op	       0 allocs/op
Benchmark_find4-4                             	426094305	         2.816 ns/op	       0 B/op	       0 allocs/op
Benchmark_findIndex4-4                        	319770602	         3.749 ns/op	       0 B/op	       0 allocs/op
BenchmarkSmallWriteTxn/updates_1-4            	  426124	      2700 ns/op	    3529 B/op	       4 allocs/op
BenchmarkSmallWriteTxn/updates_2-4            	  305924	      3716 ns/op	    4742 B/op	       6 allocs/op
BenchmarkSmallWriteTxn/updates_4-4            	  196714	      5893 ns/op	    7155 B/op	      11 allocs/op
BenchmarkSmallWriteTxn/updates_8-4            	  119084	      9802 ns/op	   11927 B/op	      20 allocs/op
BenchmarkSmallWriteTxn/updates_16-4           	   67908	     17827 ns/op	   21289 B/op	      38 allocs/op
BenchmarkFreshWriteTxn-4                      	 5434141	       217.3 ns/op	     185 B/op	       6 allocs/op
BenchmarkAtomicWatchPointerFirstChannel-4     	19964085	        59.78 ns/op	     120 B/op	       2 allocs/op
BenchmarkAtomicWatchPointerChannel-4          	479896081	         2.497 ns/op	       0 B/op	       0 allocs/op
PASS
ok  	github.com/cilium/statedb/part	63.723s
PASS
ok  	github.com/cilium/statedb/reconciler	0.004s
?   	github.com/cilium/statedb/reconciler/benchmark	[no test files]
?   	github.com/cilium/statedb/reconciler/example	[no test files]
go run ./reconciler/benchmark -quiet
1000000 objects reconciled in 1.64 seconds (batch size 1000)
Throughput 611499.65 objects per second
506MB total allocated, 5007203 in-use objects, 231MB bytes in use

@joamaki
joamaki marked this pull request as ready for review October 12, 2026 10:59
@joamaki
joamaki requested a review from a team as a code owner October 12, 2026 10:59
@joamaki
joamaki requested review from derailed and removed request for a team October 12, 2026 10:59

@bimmlerd bimmlerd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generally looks fine, the last commit is a bit too much on the complexity side of complexity for speed side IMO

Comment thread part/iterator.go Resolved
Leaves have no transaction ID and always report zero. Never treat them
as writable through the transaction ID shortcut, including in the first
transaction, so overwriting a leaf closes its previous watch.

AIL:3
Signed-off-by: Jussi Maki <jussi@isovalent.com>
Keep the grown parent-path slice after deep deletions so later deletes
reuse it. Clear visited entries on all return paths to avoid retaining
removed nodes through the transaction cache.

BenchmarkDeleteParentCache repeatedly deletes and reinserts the deepest
key among 8 or 100 nested prefixes in one transaction. Results from
linux/arm64, Go 1.25.10, benchtime=200ms, count=10:

                         │      old      │       new        │ vs base
                         │    sec/op     │     sec/op       │
depth_8                   │ 168.9ns ± 7% │  171.6ns ± 3%  │ ~ (p=0.353 n=10)
depth_100                 │ 2.179µs ± 3% │  1.502µs ± 3%  │ -31.08% (p=0.000 n=10)

                         │     old       │       new        │ vs base
                         │     B/op      │      B/op        │
depth_8                   │  48.00 ± 0%  │   48.00 ± 0%   │ ~ (p=1.000 n=10)
depth_100                 │ 3504.00 ± 0% │   48.00 ± 0%   │ -98.63% (p=0.000 n=10)

                         │     old       │       new        │ vs base
                         │   allocs/op   │    allocs/op     │
depth_8                   │  1.000 ± 0%  │   1.000 ± 0%   │ ~ (p=1.000 n=10)
depth_100                 │  3.000 ± 0%  │   1.000 ± 0%   │ -66.67% (p=0.000 n=10)

AIL:3
Signed-off-by: Jussi Maki <jussi@isovalent.com>
A new transaction need not allocate the parent-path buffer until it
deletes from a nonempty tree. Keep insertion-only transactions free of
this allocation.

BenchmarkFreshWriteTxn creates a new tree, inserts one key, and commits
on each iteration. Results from linux/arm64, Go 1.25.10,
benchtime=200ms, count=10:

               │      old      │      new       │ vs base
               │    sec/op     │    sec/op      │
FreshWriteTxn  │ 328.9ns ± 22% │ 135.0ns ± 9%  │ -58.96% (p=0.000 n=10)

               │     old       │      new       │ vs base
               │     B/op      │     B/op       │
FreshWriteTxn  │ 697.0 ± 0%   │ 185.0 ± 0%   │ -73.46% (p=0.000 n=10)

               │     old       │      new       │ vs base
               │   allocs/op   │   allocs/op    │
FreshWriteTxn  │ 7.000 ± 0%   │ 6.000 ± 0%   │ -14.29% (p=0.000 n=10)

AIL:3
Signed-off-by: Jussi Maki <jussi@isovalent.com>
@joamaki
joamaki force-pushed the pr/joamaki/part-txn-fixes branch from 2712090 to be53dfc Compare October 13, 2026 14:11
@joamaki

joamaki commented Oct 13, 2026

Copy link
Copy Markdown
Contributor Author

generally looks fine, the last commit is a bit too much on the complexity side of complexity for speed side IMO

mmm yeah, I think the idea is still worth doing. I dropped the commit and will take another look at whether this can be done in a much simpler way.

@joamaki
joamaki merged commit f859acc into main Oct 13, 2026
1 check passed
@joamaki
joamaki deleted the pr/joamaki/part-txn-fixes branch October 13, 2026 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants