Skip to content

v0.5: backport PR 214 - #217

Merged
joamaki merged 4 commits into
v0.5from
pr/joamaki/v0.5-backports
Nov 19, 2026
Merged

joamaki merged 4 commits into
v0.5from
pr/joamaki/v0.5-backports

Conversation

@joamaki

@joamaki joamaki commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Backport of PR #214 to bring the fix to Cilium v1.19.

…iled

[ upstream commit 6917b43 ]

[ backporter's notes: Port the pruning regressions to the internal test package and synchronize
  the reconciliation loop with channels instead of synctest, retaining
  Go 1.24 compatibility. Cover concurrent writes, failed attempts, empty
  tables with nonzero revisions, and both single and batch operations. ]

Regression test for checking that pruning only happens once reconciliation
has been attempted on all objects that existed at the time the table
become initialized.

AIL:3
Signed-off-by: Jussi Maki <jussi@isovalent.com>
[ upstream commit 2514f53 ]

[ backporter's notes: Use the v0.5 incremental.run API and change iterator. Omit the upstream
  revision/progress bookkeeping changes because v0.5 does not provide
  revision-based progress tracking; initial pruning uses snapshot exhaustion.
  AIL:3 for the backport ]

Pruning was invoked after the table had initialized but not before
the initial set was reconciled. This is an issue when the Update and
Prune operations have a dependency on each other, e.g. Prune reads
internal state populated by Update.

Fix this by only pruning once reconciliation has been attempted
at least up to the revision at which the table has been initialized.

AIL:0
Signed-off-by: Jussi Maki <jussi@isovalent.com>
[ upstrem commit d1a0840 ]

From now on also run CI checks for PRs targeting
v0.* and also after merging.

Signed-off-by: Jussi Maki <jussi@isovalent.com>
@joamaki
joamaki requested a review from pippolo84 November 19, 2026 14:33
@joamaki
joamaki requested review from a team as code owners November 19, 2026 14:33
@joamaki
joamaki requested review from Artyop and aanm and removed request for a team, Artyop and aanm November 19, 2026 14:33
@novikovfred

Copy link
Copy Markdown

Hi, as I see v1.18 on 0.4
and v1.19 on 0.5

@joamaki

joamaki commented Nov 19, 2026

Copy link
Copy Markdown
Contributor Author

Oops yeah should've been that way around.

[ upstream commit 3c6df79 ]

We started seeing the error of the pr workflow [^1]:

    Run thollander/actions-comment-pull-request@v2
    No comment has been found with asked pattern. Creating a new comment.
    Error: Resource not accessible by integration

According to the thollander/actions-comment-pull-request action's
README[^2] we need to add pull-requestes: write permissions to the
workflow.

Do that and also update the action to v3 while at it.

[^1]: #151 (comment)
[^2]: https://github.com/thollander/actions-comment-pull-request/blob/e4a76dd2b0a3c2027c3fd84147a67c22ee4c90fa/README.md?plain=1#L172-L181

Signed-off-by: Tobias Klauser <tobias@cilium.io>
Signed-off-by: Jussi Maki <jussi@isovalent.com>
@github-actions

Copy link
Copy Markdown
$ make
go build ./...
go: downloading go1.24.0 (linux/amd64)
go: downloading go.yaml.in/yaml/v3 v3.0.3
go: downloading github.com/cilium/hive v0.0.0-20250731144630-28e7a35ed227
go: downloading golang.org/x/time v0.5.0
go: downloading github.com/spf13/cobra v1.8.0
go: downloading github.com/spf13/pflag v1.0.5
go: downloading github.com/cilium/stream v0.0.0-20240209152734-a0792b51812d
go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de
go: downloading github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc
go: downloading github.com/mitchellh/mapstructure v1.5.0
go: downloading go.uber.org/dig v1.17.1
go: downloading golang.org/x/term v0.16.0
go: downloading github.com/spf13/viper v1.18.2
go: downloading golang.org/x/sys v0.17.0
go: downloading golang.org/x/tools v0.17.0
go: downloading github.com/spf13/cast v1.6.0
go: downloading github.com/fsnotify/fsnotify v1.7.0
go: downloading github.com/sagikazarmark/slog-shim v0.1.0
go: downloading github.com/spf13/afero v1.11.0
go: downloading github.com/subosito/gotenv v1.6.0
go: downloading github.com/hashicorp/hcl v1.0.0
go: downloading gopkg.in/ini.v1 v1.67.0
go: downloading github.com/magiconair/properties v1.8.7
go: downloading github.com/pelletier/go-toml/v2 v2.1.0
go: downloading gopkg.in/yaml.v3 v3.0.1
go: downloading golang.org/x/text v0.14.0
PART_VALIDATE=1 go test ./... -cover -vet=all -test.count 1
go: downloading github.com/stretchr/testify v1.8.4
go: downloading go.uber.org/goleak v1.3.0
go: downloading golang.org/x/exp v0.0.0-20240119083558-1b970713d09a
go: downloading github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2
ok  	github.com/cilium/statedb	161.810s	coverage: 81.4% of statements
ok  	github.com/cilium/statedb/index	0.012s	coverage: 28.7% of statements
ok  	github.com/cilium/statedb/internal	0.017s	coverage: 46.7% of statements
ok  	github.com/cilium/statedb/part	90.173s	coverage: 87.6% of statements
ok  	github.com/cilium/statedb/reconciler	0.291s	coverage: 91.0% of statements
	github.com/cilium/statedb/reconciler/benchmark		coverage: 0.0% of statements
	github.com/cilium/statedb/reconciler/example		coverage: 0.0% of statements
go test -race ./... -test.count 1
ok  	github.com/cilium/statedb	37.721s
ok  	github.com/cilium/statedb/index	1.023s
ok  	github.com/cilium/statedb/internal	1.044s
ok  	github.com/cilium/statedb/part	41.742s
ok  	github.com/cilium/statedb/reconciler	1.371s
?   	github.com/cilium/statedb/reconciler/benchmark	[no test files]
?   	github.com/cilium/statedb/reconciler/example	[no test files]
go test ./... -bench . -benchmem -test.run xxx
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkDB_WriteTxn_1-4                    	  558375	      1859 ns/op	    537851 objects/sec	    1096 B/op	      19 allocs/op
BenchmarkDB_WriteTxn_10-4                   	 1622689	       737.8 ns/op	   1355323 objects/sec	     456 B/op	       8 allocs/op
BenchmarkDB_WriteTxn_100-4                  	 1897592	       630.7 ns/op	   1585563 objects/sec	     479 B/op	       7 allocs/op
BenchmarkDB_WriteTxn_1000-4                 	 1671960	       718.8 ns/op	   1391206 objects/sec	     457 B/op	       7 allocs/op
BenchmarkDB_WriteTxn_100_SecondaryIndex-4   	  519834	      2126 ns/op	    470311 objects/sec	    1341 B/op	      37 allocs/op
BenchmarkDB_NewWriteTxn-4                   	 1754589	       682.7 ns/op	     336 B/op	       4 allocs/op
BenchmarkDB_NewReadTxn-4                    	515380174	         2.219 ns/op	       0 B/op	       0 allocs/op
BenchmarkDB_Modify-4                        	    1490	    810839 ns/op	   1233290 objects/sec	  537345 B/op	    8144 allocs/op
BenchmarkDB_GetInsert-4                     	    1287	    914233 ns/op	   1093813 objects/sec	  496874 B/op	    8144 allocs/op
BenchmarkDB_RandomInsert-4                  	    1603	    761570 ns/op	   1313076 objects/sec	  487831 B/op	    7143 allocs/op
BenchmarkDB_RandomReplace-4                 	     367	   3314377 ns/op	    301716 objects/sec	 1891247 B/op	   46192 allocs/op
BenchmarkDB_SequentialInsert-4              	    1662	    731827 ns/op	   1366443 objects/sec	  457104 B/op	    7117 allocs/op
BenchmarkDB_SequentialInsert_Prefix-4       	     418	   2842586 ns/op	    351792 objects/sec	 3498828 B/op	   56550 allocs/op
BenchmarkDB_Changes_Baseline-4              	    1238	    924813 ns/op	   1081299 objects/sec	  550457 B/op	   10199 allocs/op
BenchmarkDB_Changes-4                       	     702	   1691128 ns/op	    591321 objects/sec	  941545 B/op	   14400 allocs/op
BenchmarkDB_RandomLookup-4                  	   18068	     66353 ns/op	  15070797 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_SequentialLookup-4              	   21948	     54691 ns/op	  18284423 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_Prefix_SecondaryIndex-4         	    6783	    177207 ns/op	   5643115 objects/sec	  125114 B/op	    1031 allocs/op
BenchmarkDB_FullIteration_All-4             	     696	   1493764 ns/op	  66944976 objects/sec	     320 B/op	      11 allocs/op
BenchmarkDB_FullIteration_Get-4             	     178	   6725476 ns/op	  14868836 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_FullIteration_Get_Secondary-4   	      86	  12880312 ns/op	   7763787 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_FullIteration_ReadTxnGet-4      	     170	   6963588 ns/op	  14360413 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_PropagationDelay-4              	  579909	      1835 ns/op	        15.00 50th_µs	        19.00 90th_µs	        46.00 99th_µs	    1018 B/op	      20 allocs/op
BenchmarkWatchSet_4-4                       	 2194860	       538.9 ns/op	     320 B/op	       5 allocs/op
BenchmarkWatchSet_16-4                      	  778569	      1553 ns/op	    1096 B/op	       5 allocs/op
BenchmarkWatchSet_128-4                     	   89101	     13449 ns/op	    8904 B/op	       5 allocs/op
BenchmarkWatchSet_1024-4                    	    8985	    134507 ns/op	   73744 B/op	       5 allocs/op
PASS
ok  	github.com/cilium/statedb	34.496s
PASS
ok  	github.com/cilium/statedb/index	0.004s
PASS
ok  	github.com/cilium/statedb/internal	0.004s
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb/part
cpu: AMD EPYC 7763 64-Core Processor                
Benchmark_Uint64Map_Random-4                  	    1430	    798516 ns/op	   1252324 items/sec	 2567654 B/op	    7042 allocs/op
Benchmark_Uint64Map_Sequential-4              	    1806	    667886 ns/op	   1497261 items/sec	 2266257 B/op	    6755 allocs/op
Benchmark_Uint64Map_Sequential_Insert-4       	    1957	    615896 ns/op	   1623650 items/sec	 2258252 B/op	    5754 allocs/op
Benchmark_Uint64Map_Sequential_Txn_Insert-4   	   10000	    110751 ns/op	   9029253 items/sec	   88668 B/op	    2030 allocs/op
Benchmark_Uint64Map_Random_Insert-4           	    1620	    787586 ns/op	   1269702 items/sec	 2558536 B/op	    6035 allocs/op
Benchmark_Uint64Map_Random_Txn_Insert-4       	    6878	    174463 ns/op	   5731858 items/sec	  127884 B/op	    2460 allocs/op
Benchmark_Insert_RootOnlyWatch-4              	    9106	    124641 ns/op	   8023032 objects/sec	   73789 B/op	    2035 allocs/op
Benchmark_Insert-4                            	    6963	    171000 ns/op	   5847968 objects/sec	  189231 B/op	    3062 allocs/op
Benchmark_Modify-4                            	   10000	    105807 ns/op	   9451165 objects/sec	   70872 B/op	    1017 allocs/op
Benchmark_GetInsert-4                         	    9111	    123420 ns/op	   8102412 objects/sec	   70934 B/op	    1017 allocs/op
Benchmark_Replace-4                           	34015351	        35.17 ns/op	  28433730 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Replace_RootOnlyWatch-4             	33828966	        35.30 ns/op	  28328015 objects/sec	       0 B/op	       0 allocs/op
Benchmark_txn_1-4                             	 4357483	       272.7 ns/op	   3666869 objects/sec	     216 B/op	       4 allocs/op
Benchmark_txn_10-4                            	 9109153	       131.9 ns/op	   7582469 objects/sec	      90 B/op	       2 allocs/op
Benchmark_txn_100-4                           	10816917	       110.2 ns/op	   9077050 objects/sec	      89 B/op	       2 allocs/op
Benchmark_txn_1000-4                          	 9698311	       122.7 ns/op	   8147364 objects/sec	      77 B/op	       2 allocs/op
Benchmark_txn_delete_1-4                      	 1966590	       606.2 ns/op	   1649608 objects/sec	    2952 B/op	       5 allocs/op
Benchmark_txn_delete_10-4                     	 8578747	       138.9 ns/op	   7199602 objects/sec	     333 B/op	       1 allocs/op
Benchmark_txn_delete_100-4                    	11609823	       103.1 ns/op	   9703598 objects/sec	      70 B/op	       1 allocs/op
Benchmark_txn_delete_1000-4                   	13885326	        86.07 ns/op	  11618699 objects/sec	      26 B/op	       1 allocs/op
Benchmark_Get-4                               	   47114	     25499 ns/op	  39217515 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Iterate-4                           	  159915	      7650 ns/op	 130726638 objects/sec	     104 B/op	       4 allocs/op
Benchmark_Hashmap_Insert-4                    	   14995	     79903 ns/op	  12515212 objects/sec	   74265 B/op	      20 allocs/op
Benchmark_Hashmap_Get_Uint64-4                	  135806	      8679 ns/op	 115220443 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Hashmap_Get_Bytes-4                 	  111554	     10723 ns/op	  93258509 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Delete_Random-4                     	      18	  59832706 ns/op	   1671327 objects/sec	72000842 B/op	  138028 allocs/op
PASS
ok  	github.com/cilium/statedb/part	32.554s
PASS
ok  	github.com/cilium/statedb/reconciler	0.005s
?   	github.com/cilium/statedb/reconciler/benchmark	[no test files]
?   	github.com/cilium/statedb/reconciler/example	[no test files]
go run ./reconciler/benchmark -quiet
1000000 objects reconciled in 2.56 seconds (batch size 1000)
Throughput 390410.23 objects per second
951MB total allocated, 6015164 in-use objects, 338MB bytes in use

@joamaki
joamaki merged commit 42735e3 into v0.5 Nov 19, 2026
1 check passed
@joamaki
joamaki deleted the pr/joamaki/v0.5-backports branch November 19, 2026 15:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants