From 501c6a0249d69177b914340f38aa8a120b75bda7 Mon Sep 17 00:00:00 2001 From: Asaf Mahlev Date: Thu, 20 Aug 2026 10:39:03 +0300 Subject: [PATCH 1/2] chore(tools): add a test-result integrity + baseline gate Three false conclusions were drawn from Gradle test output in a single session, and none of them was flagged by the test run itself: - Gradle served results from an earlier run because the task was UP-TO-DATE, so a "passing" report described code that never executed. - An ad-hoc XML reader under-counted, reporting 4 failures where there were 12. Under-reporting is the dangerous direction: it looks like good news, and nothing contradicts it. - Failures were attributed to a code change when one of them reaches the network over HTTP and can flip with no code change at all. Each is a habit that has to be remembered, so each is now a check that runs whether or not anyone remembers. tools/check_test_results.py: - refuses to report if any result file predates the run (--started-after) - counts elements AND sums the tests=/failures= attributes the suites declare, and refuses to report if the two disagree, so an under-counting reader cannot pass silently - diffs failing test NAMES against a checked-in baseline rather than comparing pass counts - quarantines tests marked `net:` in the baseline, which are reported but never counted as a regression or as an attributable fix Exit codes are distinct: 0 clean, 1 new failures, 2 untrustworthy results. Wired into the Unit tests workflow as the authoritative gate. The Gradle step becomes continue-on-error so the checker decides; it is strictly stronger than the exit code it replaces, since a green Gradle run over stale results now fails. The checker's own tests run in the same job. tools/tests/test_check_test_results.py covers each failure mode with fixtures, including a file holding several elements and one whose declared totals exceed what it lists. Verified against real output: on the results where the ad-hoc reader said 4 failures, the checker reports 320 tests / 12 failed, matching the totals the suites declare. Exit codes confirmed 2 / 0 / 1 for stale, clean, and new-failure runs. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: a88bd77f-d993-44c5-9efc-c7124f0d825e --- .github/workflows/build-test-auto.yml | 24 +- AGENTS.md | 7 + .../check_test_results.cpython-313.pyc | Bin 0 -> 13062 bytes tools/check_test_results.py | 256 ++++++++++++++++++ tools/test_baselines/runTests-linux.txt | 15 + tools/test_baselines/runTests-windows.txt | 10 + .../test_check_test_results.cpython-313.pyc | Bin 0 -> 12082 bytes tools/tests/test_check_test_results.py | 181 +++++++++++++ 8 files changed, 492 insertions(+), 1 deletion(-) create mode 100644 tools/__pycache__/check_test_results.cpython-313.pyc create mode 100644 tools/check_test_results.py create mode 100644 tools/test_baselines/runTests-linux.txt create mode 100644 tools/test_baselines/runTests-windows.txt create mode 100644 tools/tests/__pycache__/test_check_test_results.cpython-313.pyc create mode 100644 tools/tests/test_check_test_results.py diff --git a/.github/workflows/build-test-auto.yml b/.github/workflows/build-test-auto.yml index 76a74d548..e48994cbb 100644 --- a/.github/workflows/build-test-auto.yml +++ b/.github/workflows/build-test-auto.yml @@ -39,8 +39,30 @@ jobs: - name: Grant execute permission for gradlew run: chmod +x gradlew + + - name: Record run start time + id: start + run: echo "ts=$(date +%s)" >> "$GITHUB_OUTPUT" + - name: Run unit tests - run: ./gradlew :app:testOfflineRunTestsUnitTest + continue-on-error: true + run: ./gradlew :app:testOfflineRunTestsUnitTest --continue + + # Authoritative gate. Strictly stronger than the Gradle exit code above: + # it also fails on results that are stale (task was UP-TO-DATE, so the + # report describes code that never ran) or self-inconsistent (enumerated + # testcases disagree with the totals the suites declare). Both of those + # have previously produced confident, wrong "it passes" conclusions. + - name: Verify test results and diff against baseline + run: | + python3 tools/check_test_results.py \ + --results-dir app/build/test-results/testOfflineRunTestsUnitTest \ + --baseline tools/test_baselines/runTests-linux.txt \ + --started-after ${{ steps.start.outputs.ts }} + + - name: Self-test the results checker + if: ${{ always() }} + run: python3 -m unittest discover -s tools/tests - name: Archive test reports uses: actions/upload-artifact@v4 diff --git a/AGENTS.md b/AGENTS.md index 4bd59613d..d8e5ff327 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -89,6 +89,13 @@ $env:JAVA_HOME = "C:\Program Files\Eclipse Adoptium\jdk-21.0.11.10-hotspot" - **Key tests:** `InputLogicTest.kt` (typing/autocorrect/combining-mode/Hangul), `SuggestTest.kt`, `WordComposerTest.java`, `DictionaryGroupTest.kt` (reflection + Mockito on the package-internal `DictionaryGroup`), `SettingsContainerTest.kt` (settings wiring), `KeyboardParserTest.kt`, `ClipboardDaoTest.kt`. - **Conventions:** `@Test`; method names use camelCase or backtick form; obtain `Context` via Robolectric; package-internal classes are exercised via reflection (`Class.forName(...).declaredConstructors`). - **Known failures:** the full debug unit suite has ~11 pre-existing failures (in `KeyboardParserTest`, `XLinkTest`, `StringUtilsTest` emoji, and `InputLogicTest` Hangul/autocorrect-revert/autospace-indicator) that are environment/data-dependent and usually unrelated to a change. The `runTests` build type exists to skip these on CI. **Verify a change by diffing failures against an `origin/main` baseline run, not by absolute pass count.** +- **Don't read the test report by hand — use the gate.** `tools/check_test_results.py` parses the JUnit XML and refuses to answer when the results can't be trusted: it fails if any result file predates the run (Gradle served an UP-TO-DATE task, so the report describes code that never ran) and if its own enumeration disagrees with the totals the suites declare (an under-counting reader looks like good news). It then diffs failing test **names** against a checked-in baseline, and quarantines `net:`-marked tests that reach the network so they are never counted as a regression or as an attributable fix. It runs automatically in the Unit tests workflow and is the authoritative gate there. Locally: + ```bash + python tools/check_test_results.py \ + --results-dir app/build/test-results/testOfflineRunTestsUnitTest \ + --baseline tools/test_baselines/runTests-windows.txt + ``` + Baselines live in `tools/test_baselines/`. If it reports new failures, don't paste them into the baseline — establish they aren't yours first, then `--update-baseline` and say why in the PR. The checker has its own tests: `python -m unittest discover -s tools/tests`. - **Coverage gap:** gesture/glide recognition needs the native engine, so JVM unit tests exercise tap-based logic, not native gesture recognition (a trace/replay harness is planned — see `docs/IMPROVEMENT_PLAN.md`). - **Expectation:** new behavior MUST add/update unit tests; any settings change updates `SettingsContainerTest.kt`. Keep PRs single-responsibility (see `CONTRIBUTING.md`). diff --git a/tools/__pycache__/check_test_results.cpython-313.pyc b/tools/__pycache__/check_test_results.cpython-313.pyc new file mode 100644 index 0000000000000000000000000000000000000000..71255be9e1410fd59d0d5759c33ce1f4699a7844 GIT binary patch literal 13062 zcmb7KeQ;CPm4A|+zCUft7@GhOW3UC5&1Zv6AP{UY#@NKqA|!@th4pL%vgE!e2dp%y zo84*crd`aYS)6HS;_h^|&dkn+&UUtR+S#%*`$wmLNs)uqS2ks*nVrdhq_Ev?_K)4) zxliv&IHFA-`03qy&pr3tbMLw5>pt{)T^yc;fBf3zr?+$5Z>gX^R_(*|@DKQSixWAS z6HTJ|l&Rk=n+5u|oU+Ik_RGsW`?bne_G|06D}d>D$W8{ypK|rPWp}?v_OP<`l(*j} z`|xd({pa25xn@qZH*un4zgZh~e~Ah?AO=Kd6L*=HYs4DS#lC9ATG7qE>cl$H!@h!I zQ1r5|da+(yC;I67vPoWdzCrYNpX0dLdX7799o|(!T=CP)onP0?=_BqpX-xt*VVSUm z8e)HMBngT%6_Xj7SQ3?E>C{+qT1li+if~nu zB_S@yuBL=>IX$TlLa1Ho>CDu0Mu<<#iPVIU1}%!DP`7p=MWyt(aP?AR>=H)7NQ{{2 zl#s}P@Nr>08Jn1p;@vLS?hYNRBFQgHah+ssz?d*4OD`wV(~2OYcbV`K%1lhTB3z9r zEx;hv)-z3qaM7Ggs4R15^ACYphBDlLOnNg0z97ct0KI!@D%WvH`|l3tc% zFh`Qd5{eY>aP8?3jtZAzsd!jUCzF6DhRRYb4lpH=lv0`Gj4+)-F+7%@#=6A2F*6~a zP+}8cX^~A)93&$ z(kh^fED<`kCc8o52>2>pi;ZQ3NofqkCX`75Yn;R?2#HCWa4eOfNv7D?;{?g3oJwad zfh~lrv<^gkI9!^M#w=P?V57y0$5IoLOluHN$kJGbt+Boo=$TBXudo$JCaz#drqgk7 z^s3S!h-vf(65{$)ro`rqJWYEL8$n28UrDLyNeSBzNEr_qKN+@XXw!^gqaPNeq%;Yp zE5gyi9zmI&)aasi2s6#?5lfm*%qn^>NOegS3(=0Jr=kYi;KS0v@zKEq}7)OM} zIO~|Xnl{$5!?m|VIBu*3Erl^moN&an%|t6OA<<%`0Dx79LHtT`K+8?rXY7(j4ly+) zVHEiaTlFz4(Rea3rEQy3`Z0_^7E2}}!M*4&0L9o9$U!oFm6pdCAgxI4!X!4Rz8S`1 z$qr$dNt_TH&tT)mG$|yJIq1T&;V=M4A*L$(#Q(#>hJ1m_}DxHkFD6LCWKY6BR2#P)7aH%qkeI&%DraCW9CzA0_ zg4df^VQ3uONJ*f6gqE0&H2To-V5=JEpEfQVQLj@vaa6F5VPKiQ){(hZ*;UEJWC(0n zW6JKQ_v}5eiyjx7&ejsM;y7M0T^x&$bhfs{kJ5ogL^1jN+$?x}>Kz>&m2=RF(^GMb zX3PxhjrCzWLD7cUC14fUaW3G%5SScc3uP|NuuU`>%ajgVoe^w8yM?l{n8XpmpxPjx z#sL#zaC-zHVJMiANC|CVRuUU)It3M>NU@7lfU==8Je){DRVYv>kW8Jjkn3a8Tp^R{ zj-_zUFu7I+LuS>9*~M_wDT-&KWKHlQabFvAV z-z=JB%Qj9l%lvsO^rcO-beUwk$e(wJ)>c45k2^&>N{&*=#Y)an$<0cxQpv+g?o!Fi zN}f{5$4XvS^2z=Qe<+~Zhc(`Ryc;8BJA0uqN{0qxO0BYsP9nd}bLf@M&Bx>{huq^LDUSwHPmU;3h^F?A55 zY=GFb6Ob4Vt?({4h`3PpEoS%$cN=oOa7icjo+6;?DxMtj3&dx>^^E`(eE7M_faY;pnEoA zz)PRTYLN`Ghe0;$rSk9SHvK!|9H!%L%_BNQr{N8i_Hk(^7^}~4>szy;|DVL|`Yi50 z*~k6aK4FYmqKOU?_B>%W9-ViGOgh1FoMYgmWq60)xBNRysMkEASH~gxNN`gwj8$!y z_;t8CuJ3igxf!22)}TYtSS@2eN8H1^tH1?xjANMR6JTl#7}{5d0Y`gePZeysL>sh6 zOvC#=3s-GdMqEBltOW(tc8tMeoJq#XB5dRJjhAMaZLEZY143joOGe0=yM@7z zN!nE>=~^~H)yh6K6U9-sFeF({%ZkdA<)E56KK?&Es)Jz3b5rdlNY%;!vlij75Zb^L zvuaadyh?GEpGc-JsxEN`mgFQEy{fr)M71(vR2N&iUK&NSA?y<{KEO=Xl8H^IHt8D7 zPvD?s!Dxw89P1)C5%K0kT(!jFam@f>mI;&?%ooByHj2vAh^keYN+vSQU{QI(pqVYS z2}2&$U9!1pFQ^t2R43RB`v?}5%1^<7Rrw5je$^sf8&i2AjSUFol9U2+aza%X`(c)s z$`fZ4LT@-U_>eadMP4>*-FcGxPg(zTc6u-^v=#Z8*`0A1>c#Rs|`@qsGg$;WaT5hbnZJx6g z9lq<{dG9U$$BrL5Zl5abIo41C`n#;Z?6Tr} z3_Hts$UtPKtBQ}eIGAVVviS;A)i7I)9#u?LD7=0g)i|E3nYOxtG>nMm+9$=IJK7Di zF;2>>1|6^2g=q%nDyA%yfhEF=yq^=T5flCHwdraB3TU9DY|JfO$d*0K960@S z)Q)31w5QUDMUW%Tj&B4K5O2e0B6S6kNO((Qh|MK39kKwrJ~azM)+fNHMHmr*S)4>d z*3z+SJX_a}5DlWTh?@Wdi#P}&o7|30G8q+g^29Q#196hcOadpT4bc#sV>063T*xXP zA{a00C&i;IQlUC9KK$=VMLt5UR@PA=8-yJr+WFzvyrFTNQdV<%7=7);4!G)2VFWz`@;nd=5 z%hIi5%cHpir+#rd_tJ&jsW0XN(cfDjPM_Ghn*9$voF_Ot#GIqY{32R#wz)Pzx3JA+ z!j?ki+6T3jo3hm0O_u*ovS_MpEAa`quqoglRl`?lH3%GFs%%Kq8s1Zx%FjVb6il55?DhaJ{AR&dP=cjJ%l_-J5(Y^9g!%CT_YAbmO&9QDm{)$;cG3}Uy9yp5knUE z9i!k6nJ%ePsFwv>$;{AV+uaN;nMt(!UFEQH8w{5#32f0gqOZ_ruu*gyNi$`rKSjr2 z+9;U>ETfAOp)E;8yNit|U7JGm7&u5t>K0R|ok?1me1yCZQCILYi6jDoDhPGzaY>@8 zBAm`>F+%8?;lw7Y2jZFO1wNE!f|w{c^>RW5nKk;MWWjwgsSg#ws3{|gi6vYyaSc!W97qCG|(31by4qPH7v8DP%U z?!#FP_^JUC(cEPkHo_75IGm9}gBJShk`}6>L-g?w^5FF`$-zKagTgm!9?@`Q3|P`o zaI31rkCJi69$o(!{ovR^tC97&=85p5v~k$;+5RHWbJcXAJ`a5y=($%hf~ool6;KPB zHq>|=vSvGmrOdRP654bjMVL~{Kf$BXQWep-L^=xIOsL~i>vU#3eBfP^>W~lyjzid0 z)7HU|Q!~9`mQ$ipb<(+j_~f+=EMT&-DWZq~FruD`DO6D?UZGr>>Kg1FiJlqmJ$~|> zd;m;PEy%J#z>d(jLnlggmS}<7U_n|^W^+WTAivEV2zd7e2w7ugviu`s)@lH@|Y@D@)31!_Iue&R=fWArZ!dc4H7oeL z%J;j!^OfA@&Ro;3)uu1xo4!zJI&jz1bsxHQV18gRc-OP(6F(O?1gZA~b8f^TH)-m# zQc?`{*O zFveZ`l!KR|HH=2lCfW(d1ZdFc80SKcK^S~TwhE_GNH~URUo+2|Tw5fcr4)1M(P96uBN0864 z!&9Y{ED0xMJ(|9vnp|=Ot@2sCLUzsCJxvu~#7nhl9bmX#Ay?W(wt=Lb4gBSFLKk+H zSde4X!^(u4*~Dw@T%Xq4knuF&{|?85PUaC^bqn&(?* zdq1=XiVp8}&%9>=ZtHq{|LFMbu9bbgg_`5<@9SOe`Swc-153drTcNf!zppo6b3E@j z4r`cqeErl{Pp$IxdA@!zzC40lL7v}3)k9w$TIjiX^2W(UsSs>g%H)GPZ%xb&z0V*1 zpWj)z+7l*5{7-|=)z}`hhmAZzKSUdO0#L3!)vVQKt}JlX)LuDbAe!koq}*7_1VvCq z`H8$}9W@*%(bi>w&{c=De{M)eS!s!!=oDRD{1dUdMUUuZqF!B58AljpJ%-7=TJsFo zt*Jm!SB1K&jU%!-TCc;2zUr9h;L;CF92!3Yj|N>lZodi0zQYMrte zjDQNtjJU2)S!)_LatdW?%Ew)q<_b(k^p|E*kvphLuZ{snxsiE@7;)5;HDY}Ny|sGJ z&q1$Mht+={>r+>*1-<393c{=s%CpwL;6c@+rCBeRf6=lIEiL65{R=G(XxXlp%fDz@ zzt_mdfUc0udbcIgo1-*cI2wPC3I z{T2=CF%C9X$Bnr)jqcU4Rv$^94`4U!HP#6`$6OsgVk6@sVyAb?aV}n!PHJ;Lp}(DG zfqaR};1ay~idBC~rwB9Dg_(=E^ja7mFpyL3iXIyy?#Jc#G|mMwKqAg3jIl|__ShK4 zF8H&?a2R8BJvNWc6`XhI^0h6c=^Gy24q3O+RJqtNPwwhqb(EIkX=(?tyJ z*4q@_u*!NLyNE)1+#w^NfNNcdp=Xd?&$`N&YqI`uxD;1(`FDo0kFremgG@Pp)q|{56mYCnCdUp}bl|kYOJ-fEP0@nD zwuHhmPu3}1z01ghgyVU7P1386ULw74@Re@bkh!E~u;Tbq9Xf5QUlX;mV@yV~W^PC6 zi}V>1UOWbWSvAL|*cWrwaR-crvs6#zoiF(>s8NS9A$;UcO+I`q z@9dd>Dym`&ajCOvi+sQHL@7G3@Y3q|+m zURm6+B>v!~@4d7%aLZkE`&Qi>^X`o~*QUiw2xPQAlW%<{xBbxV7btUdoRC;;k6he_ zwq^c3@4Matf){+lY~Ou5qKX^v|DIzVq9Cu{cy(#)2baHh`Bv!Xmw$3Ozo&1-agy;W zwA}Ju_}%dG=G*3MsvRND~=1) z@tM29wrV{?_x%BIH|N^D0#DHGVKE!JU`HD|8V&JGgf;oL$t=FcGT+DqQG$mxXcA44 zU-_GOed?q$husQ9cNRi+=xy@1@P*nVR6&OZ>yd5DOiv}HQT;L@?!K3<6WT@MVG8*f6peZJ|bXh|$f#@o9t$adS{6t5lvL zMCP=DtAR6$T8{*I2XgF^)X}S5+#aC>xs_F^?ut5@Y-Cs{`qfZLk$J*L9xO!|GM8`* zSDP!v5E%EgLyw(-?Bt_UPG(0L9U(G#5h@RBAj9qzYIpQDQKjPf?f-*Wz6bIxCLuHSIYIj;FPT-(2M>+gH(W+|L*cK^58W2$?= z;q}mh;sYu^+SG3HKl-A{X$n4CUyIicpK0@>W}oTV-#4E$nY<6qnK|Awmn~Qti@g2o WeXsY;kGygA13vh`!tqUv#Qz7r^%T$m literal 0 HcmV?d00001 diff --git a/tools/check_test_results.py b/tools/check_test_results.py new file mode 100644 index 000000000..958816ca4 --- /dev/null +++ b/tools/check_test_results.py @@ -0,0 +1,256 @@ +#!/usr/bin/env python3 +"""Integrity + baseline gate for Gradle JUnit XML test results. + +This exists because three separate false conclusions were drawn from Gradle test +output during one session, none of which the test run itself flagged: + +1. Gradle served results from a previous run because the test task was + UP-TO-DATE, so a "passing" report described code that was never exercised. +2. A hand-rolled XML reader silently under-counted: it disagreed with the + totals the results themselves declared, and reported 4 failures where there + were 12. The exact mechanism is less important than the fact that nothing + flagged it -- under-reporting is the dangerous direction, because it looks + like good news. So this script never trusts its own enumeration: it counts + elements AND sums the tests=/failures= attributes the suites + declare, and refuses to report anything if the two disagree. +3. Failures were attributed to a code change when at least one of them reaches + the network and can flip with no code change at all. + +Each check below is one of those, made mechanical. Run it after a test task; +it exits non-zero rather than relying on anyone remembering. + +Usage: + python tools/check_test_results.py \ + --results-dir app/build/test-results/testOfflineRunTestsUnitTest \ + --baseline tools/test_baselines/runTests-linux.txt \ + --started-after 1723800000 + + # after deliberately changing which tests fail: + python tools/check_test_results.py --results-dir ... --baseline ... --update-baseline + +Exit codes: + 0 results are trustworthy and match the baseline + 1 baseline mismatch (new failures) + 2 integrity failure (stale, unparseable, or self-inconsistent results) +""" + +from __future__ import annotations + +import argparse +import os +import sys +import xml.etree.ElementTree as ET +from dataclasses import dataclass, field +from pathlib import Path + +NET_PREFIX = "net:" + + +@dataclass +class Results: + """Everything parsed out of one results directory.""" + + # test ids that failed or errored, as "ClassName > test name" + failed: set[str] = field(default_factory=set) + # declared totals, summed from attributes + declared_tests: int = 0 + declared_failures: int = 0 + # observed totals, counted from elements + observed_tests: int = 0 + observed_failures: int = 0 + suites: int = 0 + files: int = 0 + + +def parse_results(results_dir: Path) -> Results: + """Parse every in every XML file under results_dir. + + Deliberately iterates all testsuite elements, not just the document root: + a single file may hold more than one, and missing them silently + under-reports failures. + """ + res = Results() + xml_files = sorted(results_dir.glob("**/*.xml")) + if not xml_files: + raise SystemExit(f"[integrity] no result XML found under {results_dir}") + + for path in xml_files: + try: + tree = ET.parse(path) + except ET.ParseError as exc: + raise SystemExit(f"[integrity] cannot parse {path}: {exc}") + res.files += 1 + + root = tree.getroot() + suites = [root] if root.tag == "testsuite" else [] + suites.extend(root.iter("testsuite") if root.tag != "testsuite" else []) + # a root may itself nest further children + if root.tag == "testsuite": + suites.extend(root.findall("testsuite")) + + seen = set() + for suite in suites: + if id(suite) in seen: + continue + seen.add(id(suite)) + res.suites += 1 + res.declared_tests += int(suite.get("tests", 0)) + res.declared_failures += int(suite.get("failures", 0)) + int( + suite.get("errors", 0) + ) + suite_name = (suite.get("name") or path.stem).split(".")[-1] + + for case in suite.findall("testcase"): + res.observed_tests += 1 + if case.find("failure") is not None or case.find("error") is not None: + res.observed_failures += 1 + res.failed.add(f"{suite_name} > {case.get('name')}") + + return res + + +def check_freshness(results_dir: Path, started_after: float) -> list[str]: + """Every result file must post-date the run we think produced it.""" + problems = [] + for path in sorted(results_dir.glob("**/*.xml")): + mtime = path.stat().st_mtime + if mtime < started_after: + problems.append( + f" {path.name} last written {mtime:.0f}, before run start {started_after:.0f}" + ) + return problems + + +def check_self_consistency(res: Results) -> list[str]: + """Declared totals must equal what we actually enumerated. + + This is the check that catches an under-counting reader: whatever the + mechanism (a skipped suite, a results file read while still being written, + an unexpected root element), the enumerated counts come out lower than the + totals the suites declare, and that disagreement is mechanically visible. + """ + problems = [] + if res.declared_tests != res.observed_tests: + problems.append( + f" declared {res.declared_tests} tests but enumerated {res.observed_tests}" + ) + if res.declared_failures != res.observed_failures: + problems.append( + f" declared {res.declared_failures} failures but enumerated " + f"{res.observed_failures}" + ) + return problems + + +def load_baseline(path: Path) -> tuple[set[str], set[str]]: + """Return (expected_failures, network_dependent).""" + expected: set[str] = set() + networked: set[str] = set() + if not path.exists(): + return expected, networked + for raw in path.read_text(encoding="utf-8").splitlines(): + line = raw.strip() + if not line or line.startswith("#"): + continue + if line.startswith(NET_PREFIX): + networked.add(line[len(NET_PREFIX):].strip()) + else: + expected.add(line) + return expected, networked + + +def write_baseline(path: Path, failed: set[str], networked: set[str]) -> None: + lines = [ + "# Known-failing tests. Generated by tools/check_test_results.py.", + "# One test id per line, as reported: 'ClassName > test name'.", + "# Prefix a line with 'net:' if the test reaches the network -- those are", + "# reported but never treated as a regression or as an attributable fix,", + "# because they can flip without any code change.", + "", + ] + lines += sorted(failed - networked) + if networked: + lines.append("") + lines += [f"{NET_PREFIX}{name}" for name in sorted(networked)] + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text("\n".join(lines) + "\n", encoding="utf-8") + + +def main(argv: list[str] | None = None) -> int: + ap = argparse.ArgumentParser(description=__doc__.split("\n")[0]) + ap.add_argument("--results-dir", required=True, type=Path) + ap.add_argument("--baseline", required=True, type=Path) + ap.add_argument( + "--started-after", + type=float, + default=None, + help="epoch seconds; every result file must be newer than this", + ) + ap.add_argument("--update-baseline", action="store_true") + args = ap.parse_args(argv) + + if not args.results_dir.is_dir(): + print(f"[integrity] results dir does not exist: {args.results_dir}") + return 2 + + # 1. staleness + if args.started_after is not None: + stale = check_freshness(args.results_dir, args.started_after) + if stale: + print("[integrity] STALE RESULTS -- the test task did not rerun:") + print("\n".join(stale)) + print(" re-run with --rerun-tasks, or pass the correct --started-after") + return 2 + + res = parse_results(args.results_dir) + + # 2. self-consistency + inconsistent = check_self_consistency(res) + if inconsistent: + print("[integrity] RESULTS DISAGREE WITH THEMSELVES:") + print("\n".join(inconsistent)) + print( + " the parser missed testcases -- most likely a file holding more than\n" + " one . Do not trust any count from this run." + ) + return 2 + + print( + f"[ok] {res.files} file(s), {res.suites} suite(s), " + f"{res.observed_tests} tests, {res.observed_failures} failed" + ) + + expected, networked = load_baseline(args.baseline) + + if args.update_baseline: + write_baseline(args.baseline, res.failed, networked & res.failed) + print(f"[ok] baseline written to {args.baseline} ({len(res.failed)} entries)") + return 0 + + # 3. baseline diff, by NAME, with network-dependent tests quarantined + new_failures = res.failed - expected - networked + fixed = expected - res.failed + net_failing = res.failed & networked + + if net_failing: + print("[note] network-dependent tests failing (not counted either way):") + for name in sorted(net_failing): + print(f" {name}") + + if fixed: + print("[note] no longer failing -- refresh the baseline if deliberate:") + for name in sorted(fixed): + print(f" {name}") + + if new_failures: + print(f"[FAIL] {len(new_failures)} test(s) failing that the baseline does not list:") + for name in sorted(new_failures): + print(f" {name}") + return 1 + + print("[ok] no new failures against baseline") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/test_baselines/runTests-linux.txt b/tools/test_baselines/runTests-linux.txt new file mode 100644 index 000000000..c7e7ad40d --- /dev/null +++ b/tools/test_baselines/runTests-linux.txt @@ -0,0 +1,15 @@ +# Known-failing tests. Generated by tools/check_test_results.py. +# One test id per line, as reported: 'ClassName > test name'. +# Prefix a line with 'net:' if the test reaches the network -- those are +# reported but never treated as a regression or as an attributable fix, +# because they can flip without any code change. +# +# This is the baseline for the `runTests` build type on Linux CI, which is the +# variant the Unit tests workflow gates on. It is intentionally empty: the +# `runTests` type self-skips the data/network-dependent tests, and the four +# ParserTest failures seen on Windows do not reproduce on Linux. +# +# If CI fails here listing test names, do not "fix" it by pasting them in. +# Work out whether the change caused them first; only then run +# python tools/check_test_results.py --results-dir --baseline --update-baseline +# and say in the PR why the baseline moved. diff --git a/tools/test_baselines/runTests-windows.txt b/tools/test_baselines/runTests-windows.txt new file mode 100644 index 000000000..34981573c --- /dev/null +++ b/tools/test_baselines/runTests-windows.txt @@ -0,0 +1,10 @@ +# Known-failing tests. Generated by tools/check_test_results.py. +# One test id per line, as reported: 'ClassName > test name'. +# Prefix a line with 'net:' if the test reaches the network -- those are +# reported but never treated as a regression or as an attributable fix, +# because they can flip without any code change. + +ParserTest > canLoadKeyboard +ParserTest > de_DE has extra keys +ParserTest > dvorak has 4 rows +ParserTest > popup key count does not depend on shift for (for simple layout) diff --git a/tools/tests/__pycache__/test_check_test_results.cpython-313.pyc b/tools/tests/__pycache__/test_check_test_results.cpython-313.pyc new file mode 100644 index 0000000000000000000000000000000000000000..d5a90b8dcf196251c0e288f3450691ffca079ffa GIT binary patch literal 12082 zcmd5?Yit`=cAnu3AEHQ!lr2kEqwV3h#InY zhO(vI7ET|Dyx2vQq+YAL0cvcF)<%QE$S$y1Y=O0t{@8zOSxTi&T({R^%-48e7Kc$`AOD19 zm@ACP5JqH0OCNj6LM#H-xjv3?^p__*{k0M+{k0Jr{k0Q2{PKN{Q)Q$~V21eMMzj0e zZ?Gm;H)|q04KK-HCoovmKD&la*dXo zhw4O6+c}1jsu%{^9$GoRk1?NFU`Dx7&R;eCdr?k|1T4c^>r>2^n&Tc6>Cg5oynj~-~l~iIX+#^WBh?G!eIITnzX*Hfy1Z_;xgbR`? zNKq{a1n2^%e%!aOd9P29m1r^+S4NNf!l5?;M|`iJa2})m zrsFV8MVgS0`^Mx%JUy|0-+|_Fd1^Q*kyvvA22q-O6?m}zOqW3L38lm+*vB`W+b7 zx|d$fK92hQH}R%KRGAd4KP+?U^~d*Ju>LFGfJWkzaVfFZE65wwSor)W&h)zeiSJ)+ z;-FD$O*|?o{Yfd-m(kF5Jb>}0*YmESZQk3adV)5Sr^vC%B$392F+hd`0!dz2QwQ@F z^p7#0k+>qSV@X!N(>#?1)tvs%bu0AhSaPmEu8bEb@PSnoVTA@4BS=z7Or2T#S#x8; zqvk!^31>9pzNlkC{|LwI<7TK_Va~CEa)wd3Fc-|YT-qul!WMAlmZKJd33E-1pBs2) zO=}~8BR5&yVZtncvhxA~TmZjwQS|rV36bR zuOdIV#4I~1FIQY@U*@gv_P^afTXXI3rT%$-_j0-GQqN;(!9-D*C9??ZJ}}gvoW2+B{k82p!Vt8@bM$jWLnYGhzyi& z%IF`tLL{z4qzHYK)_hZIexZB1pnLTyh)c|WyQ`PnJMX!7F1mN!7{2e>I2-)XG1GqE z<(=Jnw|d+4?=8Ca->=%VRMm8^s_ELukY=*7=6kTkVUkLvZ%J)0a=*tvm8r<8OBTYS*HxJA10_v(Dd~`qimL*U6`qOvUac z_pbkN?|P=9h5opssgvWr;GCUadui~P1vPz1;E@k6mK>Di41X{SD!~evd{J2eFW#aC za0n(_3M9Ckkpf7RzOO_)Sp+Vn?<)Z#~=YRy00SVi{Gbgof2`zit zs1IYEV2Jwwq5lD*vfo8&;fHKSk+VIQ*gJ5tzuPcAQim5IC}&7L3JiXAE2_Op(3z1f zD10D_Y^R>ufiwz8h32b9q$o+MDzz9@_pdaWRhpi`u73pL5_A7KmNGkZ{n(;wFSQz| zzqaUVd5+QiXedaR@Rnc!DjqpR7K&a?@1i`>$AJr@H9om%X z3nIYoB5}|#B|_sDy=A3#J@4AcEA9n z1qCK!^y)|iB4#ZfjYt|qn8Rr_y9haq4G@0l0+J0_L(vKUDgxd+%p*&gwTFGQ$zeVG z%|^}|Fq&hdDEwy&pgop#GSP}&Alsur$EQM+i$(X{A#zMOuwR%|A;FZMkO|;yOxS0TyXBx(7lz>(e(GkqlBJpgxKk2MblAQ@Ycz+z?62erwJ+ z$u@YM1VaGJQi4iMn6&%2G}1y_(ir`xj~s&-5TAI&=BVT+iL@;-V|`w7i%OwZGEd z#@%Uib~p;*9%CH}_v9t%ufpZ&WLA@57MYTZ*F;MIt}FanLk7T-RCUN3 zK$Rd2QuWs>4b2~cKwwOPU|^-1BeB%EWFl6q;adz1*Cc>#xE<5n5irl3c0(379|SYn zy=wb07)ZSUVs*{-RL@Ft_1C|@;5h)%Le-|L{tx^g?!Ly)S2fLa-gi~J*Yks(*|T%e zIp?CQF{9fbbuHEJy;r~YmiHH3zwG_A_vd}Lee(yp7V5hfU2m*Z_>QuUo!sX;ogIf( zQTUB7N#P;c!vrBIR9x0lYf#8+!k|>@px97Mg>1>H=C@Qy&QM9x3o|Z_zf>D3oi?x+ z=?6K5&AB3QRRLQNO%482Lp6=+Xjaow0L}XLQX00hT?#r=k?^dpSr~a{Ff+@9KME~1 z?7P>n@7C5|gnoJc)AK(cyxlr~uzR85jo(@pU42jOE5Nd@{U~?msI$Xvrwjy04mkqD zlA|cvP`rks8wHYl5=4RKTg+Ha;5ih?98?Uokg?dTVfK;7WvvyGD=_Vt5FCLW(PJMqUHP~fu z)$IBA8)iC|5%n+k{O}Zp!M@qVwLS=m@#=Tyy^Y3YNCV9UZta*kJ@0B;-q?6uyixV> zpWI0NTjy=duTS3H&^6NqCV#bWE;#SnGShW`L(SEr9~_+vefav*4Tgn7f+&+6YCqo5 zz@X5iLq>ReV)Ai7- z?YRmzu*#yfW)_W(0_-m(D>2w;$}S-UJn$wUahyxx>)$E4pzG-2hxqCY4g5n+1G^U7 z-7lhnm^P*6S^<``zGPLNx60cI!X3Ko4&IA;E7;1)#SW&|Du`RM>0Ov_rsUB#8c!(J zwLH$b?y$8W&H=BN^{klhC@P{=w25}~ii6n{Z>jwt_*lm=h~SeAD-=S!Wr zvw0Nu`A=9*Bc=?Lwq*NK>;RgWpDik3yXX?#=1P7`8e1_y;42`8`c~3|HDr!t3`N$t zl0U^m+(DeK369xRAiDD;gZ-Hl!d=L1fqn!qf$R;UggY z`YVb9xFPHaxx9htNR&w*ihdO6rVSeXU99~X3d~dk=R~{6`C&4Qhe%7v0E#GzaS$rf zghG$Jo7ys+c9v-)1V-=+BG~@P#B-1&iOKrW5=Q)Dx4w^~{|>}@{5u4H!*hE+aV>Zb zzCiF-%mD7z9lUKv3ea=kTYYu&2b-5YTdr@o7W}wo*;D^f^qTLdqsyMH*Y{u7{(0(W zQy+itR&ajD;f2N{5AA$?#bbuAs+{R~=w#g9%qCRun)hx~aM2|`Z72={I%_%)abFyA zcJVKtBxKs&6l5pb>Wbp~wb<)*S9t5M`G`Nm7Z_PTob8NPTQeeqf1a4opt z-uWV^jlnR7B;<6S>i~L{~i}SG5ns+uE9f`IUs~JV) z@2CIqM{dc@A#WeQIF5Ot43;AwXDP(yFx49fq*F0S?Pfz3N^$+pH6Rl4uLu)YCbTFV zA_x{Iv|gp)XAB$}&IDBv0$brR*jh;GdrIKls;&PC2ly1kdLYgSVXpTty7uLVFqw^S z$nah*`=D&@^-nr)S^nkZt)YM4f17-9c=?s)haCIL-p35<-M8Gj|B2P&KKRgA&6QBh2y zm`3rJC`u(c!gv7%<`dK%Al_k~Se(|zN3|BK@W??02Q_D2oOfLA8Cc*644ZCO}7zQOB01Q@Yn11NF|Hah)f#KeDyzO|m^6ko5_Wk|uYjZsx z{?$@VV7?}>P}8zhb9lby@Mr9NP20kT*A|&$Uo%@D+Uy?wlIvk@WhH;9{9zN1M^7q1 zm!7ybvw_(oOB}_Muva>x)8@J7G-1daQ hqleQh!@Fjt7r4#KyzAZ5e|`Gz&OGE8zK#Y|{|7SgEP?<4 literal 0 HcmV?d00001 diff --git a/tools/tests/test_check_test_results.py b/tools/tests/test_check_test_results.py new file mode 100644 index 000000000..0a0947ceb --- /dev/null +++ b/tools/tests/test_check_test_results.py @@ -0,0 +1,181 @@ +#!/usr/bin/env python3 +"""Self-tests for tools/check_test_results.py. + +Each test corresponds to a false conclusion that was actually drawn from Gradle +output, so the gate is itself gated. Run with: + + python -m unittest discover -s tools/tests -v +""" + +from __future__ import annotations + +import os +import sys +import tempfile +import time +import unittest +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parents[1])) + +import check_test_results as gate # noqa: E402 + + +SINGLE_SUITE = """ + + + + stack + + +""" + +# Gradle writes several elements into one file for +# parameterised/nested classes. Reading only the first one silently drops the +# rest -- exactly the bug this gate exists to catch. +MULTI_SUITE = """ + + + + + + + stack + + + stack + + + + +""" + +# declares more tests than it lists -- what a parser miss looks like from outside +INCONSISTENT = """ + + + +""" + + +def write(dirpath: Path, name: str, content: str) -> Path: + p = dirpath / name + p.write_text(content, encoding="utf-8") + return p + + +class ParsingTests(unittest.TestCase): + def test_counts_every_testsuite_in_a_file(self): + """The regression this gate was built for: 2 of 3 suites must not vanish.""" + with tempfile.TemporaryDirectory() as td: + d = Path(td) + write(d, "TEST-multi.xml", MULTI_SUITE) + res = gate.parse_results(d) + self.assertEqual(res.observed_tests, 4, "should see all 4 testcases") + self.assertEqual(res.observed_failures, 2) + self.assertIn("ParserTest > canLoadKeyboard", res.failed) + self.assertIn("ParserTest > dvorak has 4 rows", res.failed) + + def test_aggregates_across_files(self): + with tempfile.TemporaryDirectory() as td: + d = Path(td) + write(d, "TEST-a.xml", SINGLE_SUITE) + write(d, "TEST-b.xml", MULTI_SUITE) + res = gate.parse_results(d) + self.assertEqual(res.files, 2) + self.assertEqual(res.observed_tests, 6) + self.assertEqual(res.observed_failures, 3) + + def test_empty_results_dir_is_an_error(self): + with tempfile.TemporaryDirectory() as td: + with self.assertRaises(SystemExit): + gate.parse_results(Path(td)) + + +class IntegrityTests(unittest.TestCase): + def test_self_inconsistency_is_detected(self): + with tempfile.TemporaryDirectory() as td: + d = Path(td) + write(d, "TEST-x.xml", INCONSISTENT) + res = gate.parse_results(d) + problems = gate.check_self_consistency(res) + self.assertTrue(problems, "declared 43 vs enumerated 1 must be flagged") + + def test_stale_results_are_detected(self): + with tempfile.TemporaryDirectory() as td: + d = Path(td) + p = write(d, "TEST-a.xml", SINGLE_SUITE) + old = time.time() - 3600 + os.utime(p, (old, old)) + self.assertTrue(gate.check_freshness(d, time.time() - 60)) + + def test_fresh_results_pass(self): + with tempfile.TemporaryDirectory() as td: + d = Path(td) + write(d, "TEST-a.xml", SINGLE_SUITE) + self.assertEqual(gate.check_freshness(d, time.time() - 60), []) + + +class BaselineTests(unittest.TestCase): + def _run(self, results_xml, baseline_text, extra=None): + with tempfile.TemporaryDirectory() as td: + d = Path(td) + rd = d / "results" + rd.mkdir() + write(rd, "TEST-a.xml", results_xml) + bl = d / "baseline.txt" + bl.write_text(baseline_text, encoding="utf-8") + argv = ["--results-dir", str(rd), "--baseline", str(bl)] + argv += extra or [] + return gate.main(argv) + + def test_known_failure_passes(self): + rc = self._run(SINGLE_SUITE, "InputLogicTest > breaks\n") + self.assertEqual(rc, 0) + + def test_new_failure_fails(self): + rc = self._run(SINGLE_SUITE, "# nothing known to fail\n") + self.assertEqual(rc, 1) + + def test_network_dependent_failure_is_not_a_regression(self): + """A test that reaches the network can flip with no code change.""" + rc = self._run(SINGLE_SUITE, "net:InputLogicTest > breaks\n") + self.assertEqual(rc, 0) + + def test_stale_results_fail_with_integrity_code(self): + with tempfile.TemporaryDirectory() as td: + d = Path(td) + rd = d / "results" + rd.mkdir() + p = write(rd, "TEST-a.xml", SINGLE_SUITE) + old = time.time() - 3600 + os.utime(p, (old, old)) + bl = d / "baseline.txt" + bl.write_text("InputLogicTest > breaks\n", encoding="utf-8") + rc = gate.main([ + "--results-dir", str(rd), + "--baseline", str(bl), + "--started-after", str(time.time() - 60), + ]) + self.assertEqual(rc, 2, "stale results must not be reported as a pass") + + def test_inconsistent_results_fail_with_integrity_code(self): + rc = self._run(INCONSISTENT, "") + self.assertEqual(rc, 2) + + def test_update_baseline_roundtrip(self): + with tempfile.TemporaryDirectory() as td: + d = Path(td) + rd = d / "results" + rd.mkdir() + write(rd, "TEST-a.xml", SINGLE_SUITE) + bl = d / "baseline.txt" + self.assertEqual( + gate.main(["--results-dir", str(rd), "--baseline", str(bl), + "--update-baseline"]), 0) + self.assertEqual( + gate.main(["--results-dir", str(rd), "--baseline", str(bl)]), 0) + + +if __name__ == "__main__": + unittest.main() From d10a7977ab78bb1166bb3925911ba0e185d7eed2 Mon Sep 17 00:00:00 2001 From: Asaf Mahlev Date: Thu, 20 Aug 2026 10:39:17 +0300 Subject: [PATCH 2/2] chore(tools): stop tracking Python bytecode The pycache directories were committed by mistake; ignore them instead. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: a88bd77f-d993-44c5-9efc-c7124f0d825e --- .gitignore | 4 ++++ .../check_test_results.cpython-313.pyc | Bin 13062 -> 0 bytes tools/__pycache__/release.cpython-313.pyc | Bin 7737 -> 0 bytes .../test_check_test_results.cpython-313.pyc | Bin 12082 -> 0 bytes 4 files changed, 4 insertions(+) delete mode 100644 tools/__pycache__/check_test_results.cpython-313.pyc delete mode 100644 tools/__pycache__/release.cpython-313.pyc delete mode 100644 tools/tests/__pycache__/test_check_test_results.cpython-313.pyc diff --git a/.gitignore b/.gitignore index 06dd05602..e0f9f2f93 100755 --- a/.gitignore +++ b/.gitignore @@ -86,3 +86,7 @@ docs/superpowers/ # Temporary generated files docs/releasenote/release_notes_temp.md + +# Python bytecode from tools/ +__pycache__/ +*.pyc diff --git a/tools/__pycache__/check_test_results.cpython-313.pyc b/tools/__pycache__/check_test_results.cpython-313.pyc deleted file mode 100644 index 71255be9e1410fd59d0d5759c33ce1f4699a7844..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 13062 zcmb7KeQ;CPm4A|+zCUft7@GhOW3UC5&1Zv6AP{UY#@NKqA|!@th4pL%vgE!e2dp%y zo84*crd`aYS)6HS;_h^|&dkn+&UUtR+S#%*`$wmLNs)uqS2ks*nVrdhq_Ev?_K)4) zxliv&IHFA-`03qy&pr3tbMLw5>pt{)T^yc;fBf3zr?+$5Z>gX^R_(*|@DKQSixWAS z6HTJ|l&Rk=n+5u|oU+Ik_RGsW`?bne_G|06D}d>D$W8{ypK|rPWp}?v_OP<`l(*j} z`|xd({pa25xn@qZH*un4zgZh~e~Ah?AO=Kd6L*=HYs4DS#lC9ATG7qE>cl$H!@h!I zQ1r5|da+(yC;I67vPoWdzCrYNpX0dLdX7799o|(!T=CP)onP0?=_BqpX-xt*VVSUm z8e)HMBngT%6_Xj7SQ3?E>C{+qT1li+if~nu zB_S@yuBL=>IX$TlLa1Ho>CDu0Mu<<#iPVIU1}%!DP`7p=MWyt(aP?AR>=H)7NQ{{2 zl#s}P@Nr>08Jn1p;@vLS?hYNRBFQgHah+ssz?d*4OD`wV(~2OYcbV`K%1lhTB3z9r zEx;hv)-z3qaM7Ggs4R15^ACYphBDlLOnNg0z97ct0KI!@D%WvH`|l3tc% zFh`Qd5{eY>aP8?3jtZAzsd!jUCzF6DhRRYb4lpH=lv0`Gj4+)-F+7%@#=6A2F*6~a zP+}8cX^~A)93&$ z(kh^fED<`kCc8o52>2>pi;ZQ3NofqkCX`75Yn;R?2#HCWa4eOfNv7D?;{?g3oJwad zfh~lrv<^gkI9!^M#w=P?V57y0$5IoLOluHN$kJGbt+Boo=$TBXudo$JCaz#drqgk7 z^s3S!h-vf(65{$)ro`rqJWYEL8$n28UrDLyNeSBzNEr_qKN+@XXw!^gqaPNeq%;Yp zE5gyi9zmI&)aasi2s6#?5lfm*%qn^>NOegS3(=0Jr=kYi;KS0v@zKEq}7)OM} zIO~|Xnl{$5!?m|VIBu*3Erl^moN&an%|t6OA<<%`0Dx79LHtT`K+8?rXY7(j4ly+) zVHEiaTlFz4(Rea3rEQy3`Z0_^7E2}}!M*4&0L9o9$U!oFm6pdCAgxI4!X!4Rz8S`1 z$qr$dNt_TH&tT)mG$|yJIq1T&;V=M4A*L$(#Q(#>hJ1m_}DxHkFD6LCWKY6BR2#P)7aH%qkeI&%DraCW9CzA0_ zg4df^VQ3uONJ*f6gqE0&H2To-V5=JEpEfQVQLj@vaa6F5VPKiQ){(hZ*;UEJWC(0n zW6JKQ_v}5eiyjx7&ejsM;y7M0T^x&$bhfs{kJ5ogL^1jN+$?x}>Kz>&m2=RF(^GMb zX3PxhjrCzWLD7cUC14fUaW3G%5SScc3uP|NuuU`>%ajgVoe^w8yM?l{n8XpmpxPjx z#sL#zaC-zHVJMiANC|CVRuUU)It3M>NU@7lfU==8Je){DRVYv>kW8Jjkn3a8Tp^R{ zj-_zUFu7I+LuS>9*~M_wDT-&KWKHlQabFvAV z-z=JB%Qj9l%lvsO^rcO-beUwk$e(wJ)>c45k2^&>N{&*=#Y)an$<0cxQpv+g?o!Fi zN}f{5$4XvS^2z=Qe<+~Zhc(`Ryc;8BJA0uqN{0qxO0BYsP9nd}bLf@M&Bx>{huq^LDUSwHPmU;3h^F?A55 zY=GFb6Ob4Vt?({4h`3PpEoS%$cN=oOa7icjo+6;?DxMtj3&dx>^^E`(eE7M_faY;pnEoA zz)PRTYLN`Ghe0;$rSk9SHvK!|9H!%L%_BNQr{N8i_Hk(^7^}~4>szy;|DVL|`Yi50 z*~k6aK4FYmqKOU?_B>%W9-ViGOgh1FoMYgmWq60)xBNRysMkEASH~gxNN`gwj8$!y z_;t8CuJ3igxf!22)}TYtSS@2eN8H1^tH1?xjANMR6JTl#7}{5d0Y`gePZeysL>sh6 zOvC#=3s-GdMqEBltOW(tc8tMeoJq#XB5dRJjhAMaZLEZY143joOGe0=yM@7z zN!nE>=~^~H)yh6K6U9-sFeF({%ZkdA<)E56KK?&Es)Jz3b5rdlNY%;!vlij75Zb^L zvuaadyh?GEpGc-JsxEN`mgFQEy{fr)M71(vR2N&iUK&NSA?y<{KEO=Xl8H^IHt8D7 zPvD?s!Dxw89P1)C5%K0kT(!jFam@f>mI;&?%ooByHj2vAh^keYN+vSQU{QI(pqVYS z2}2&$U9!1pFQ^t2R43RB`v?}5%1^<7Rrw5je$^sf8&i2AjSUFol9U2+aza%X`(c)s z$`fZ4LT@-U_>eadMP4>*-FcGxPg(zTc6u-^v=#Z8*`0A1>c#Rs|`@qsGg$;WaT5hbnZJx6g z9lq<{dG9U$$BrL5Zl5abIo41C`n#;Z?6Tr} z3_Hts$UtPKtBQ}eIGAVVviS;A)i7I)9#u?LD7=0g)i|E3nYOxtG>nMm+9$=IJK7Di zF;2>>1|6^2g=q%nDyA%yfhEF=yq^=T5flCHwdraB3TU9DY|JfO$d*0K960@S z)Q)31w5QUDMUW%Tj&B4K5O2e0B6S6kNO((Qh|MK39kKwrJ~azM)+fNHMHmr*S)4>d z*3z+SJX_a}5DlWTh?@Wdi#P}&o7|30G8q+g^29Q#196hcOadpT4bc#sV>063T*xXP zA{a00C&i;IQlUC9KK$=VMLt5UR@PA=8-yJr+WFzvyrFTNQdV<%7=7);4!G)2VFWz`@;nd=5 z%hIi5%cHpir+#rd_tJ&jsW0XN(cfDjPM_Ghn*9$voF_Ot#GIqY{32R#wz)Pzx3JA+ z!j?ki+6T3jo3hm0O_u*ovS_MpEAa`quqoglRl`?lH3%GFs%%Kq8s1Zx%FjVb6il55?DhaJ{AR&dP=cjJ%l_-J5(Y^9g!%CT_YAbmO&9QDm{)$;cG3}Uy9yp5knUE z9i!k6nJ%ePsFwv>$;{AV+uaN;nMt(!UFEQH8w{5#32f0gqOZ_ruu*gyNi$`rKSjr2 z+9;U>ETfAOp)E;8yNit|U7JGm7&u5t>K0R|ok?1me1yCZQCILYi6jDoDhPGzaY>@8 zBAm`>F+%8?;lw7Y2jZFO1wNE!f|w{c^>RW5nKk;MWWjwgsSg#ws3{|gi6vYyaSc!W97qCG|(31by4qPH7v8DP%U z?!#FP_^JUC(cEPkHo_75IGm9}gBJShk`}6>L-g?w^5FF`$-zKagTgm!9?@`Q3|P`o zaI31rkCJi69$o(!{ovR^tC97&=85p5v~k$;+5RHWbJcXAJ`a5y=($%hf~ool6;KPB zHq>|=vSvGmrOdRP654bjMVL~{Kf$BXQWep-L^=xIOsL~i>vU#3eBfP^>W~lyjzid0 z)7HU|Q!~9`mQ$ipb<(+j_~f+=EMT&-DWZq~FruD`DO6D?UZGr>>Kg1FiJlqmJ$~|> zd;m;PEy%J#z>d(jLnlggmS}<7U_n|^W^+WTAivEV2zd7e2w7ugviu`s)@lH@|Y@D@)31!_Iue&R=fWArZ!dc4H7oeL z%J;j!^OfA@&Ro;3)uu1xo4!zJI&jz1bsxHQV18gRc-OP(6F(O?1gZA~b8f^TH)-m# zQc?`{*O zFveZ`l!KR|HH=2lCfW(d1ZdFc80SKcK^S~TwhE_GNH~URUo+2|Tw5fcr4)1M(P96uBN0864 z!&9Y{ED0xMJ(|9vnp|=Ot@2sCLUzsCJxvu~#7nhl9bmX#Ay?W(wt=Lb4gBSFLKk+H zSde4X!^(u4*~Dw@T%Xq4knuF&{|?85PUaC^bqn&(?* zdq1=XiVp8}&%9>=ZtHq{|LFMbu9bbgg_`5<@9SOe`Swc-153drTcNf!zppo6b3E@j z4r`cqeErl{Pp$IxdA@!zzC40lL7v}3)k9w$TIjiX^2W(UsSs>g%H)GPZ%xb&z0V*1 zpWj)z+7l*5{7-|=)z}`hhmAZzKSUdO0#L3!)vVQKt}JlX)LuDbAe!koq}*7_1VvCq z`H8$}9W@*%(bi>w&{c=De{M)eS!s!!=oDRD{1dUdMUUuZqF!B58AljpJ%-7=TJsFo zt*Jm!SB1K&jU%!-TCc;2zUr9h;L;CF92!3Yj|N>lZodi0zQYMrte zjDQNtjJU2)S!)_LatdW?%Ew)q<_b(k^p|E*kvphLuZ{snxsiE@7;)5;HDY}Ny|sGJ z&q1$Mht+={>r+>*1-<393c{=s%CpwL;6c@+rCBeRf6=lIEiL65{R=G(XxXlp%fDz@ zzt_mdfUc0udbcIgo1-*cI2wPC3I z{T2=CF%C9X$Bnr)jqcU4Rv$^94`4U!HP#6`$6OsgVk6@sVyAb?aV}n!PHJ;Lp}(DG zfqaR};1ay~idBC~rwB9Dg_(=E^ja7mFpyL3iXIyy?#Jc#G|mMwKqAg3jIl|__ShK4 zF8H&?a2R8BJvNWc6`XhI^0h6c=^Gy24q3O+RJqtNPwwhqb(EIkX=(?tyJ z*4q@_u*!NLyNE)1+#w^NfNNcdp=Xd?&$`N&YqI`uxD;1(`FDo0kFremgG@Pp)q|{56mYCnCdUp}bl|kYOJ-fEP0@nD zwuHhmPu3}1z01ghgyVU7P1386ULw74@Re@bkh!E~u;Tbq9Xf5QUlX;mV@yV~W^PC6 zi}V>1UOWbWSvAL|*cWrwaR-crvs6#zoiF(>s8NS9A$;UcO+I`q z@9dd>Dym`&ajCOvi+sQHL@7G3@Y3q|+m zURm6+B>v!~@4d7%aLZkE`&Qi>^X`o~*QUiw2xPQAlW%<{xBbxV7btUdoRC;;k6he_ zwq^c3@4Matf){+lY~Ou5qKX^v|DIzVq9Cu{cy(#)2baHh`Bv!Xmw$3Ozo&1-agy;W zwA}Ju_}%dG=G*3MsvRND~=1) z@tM29wrV{?_x%BIH|N^D0#DHGVKE!JU`HD|8V&JGgf;oL$t=FcGT+DqQG$mxXcA44 zU-_GOed?q$husQ9cNRi+=xy@1@P*nVR6&OZ>yd5DOiv}HQT;L@?!K3<6WT@MVG8*f6peZJ|bXh|$f#@o9t$adS{6t5lvL zMCP=DtAR6$T8{*I2XgF^)X}S5+#aC>xs_F^?ut5@Y-Cs{`qfZLk$J*L9xO!|GM8`* zSDP!v5E%EgLyw(-?Bt_UPG(0L9U(G#5h@RBAj9qzYIpQDQKjPf?f-*Wz6bIxCLuHSIYIj;FPT-(2M>+gH(W+|L*cK^58W2$?= z;q}mh;sYu^+SG3HKl-A{X$n4CUyIicpK0@>W}oTV-#4E$nY<6qnK|Awmn~Qti@g2o WeXsY;kGygA13vh`!tqUv#Qz7r^%T$m diff --git a/tools/__pycache__/release.cpython-313.pyc b/tools/__pycache__/release.cpython-313.pyc deleted file mode 100644 index c77c38ac237d6cd9ae7acbdbbf0f1634e3479b6a..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 7737 zcmc&(U2GdycAgQ3oZ&xdQNNb0v8-68Es~Zc$A6Wxu_Q~jEP0m`+CgNmGA2h7ZEA+O zLs}Ns-pJWqShCZV95*oCK3D|`R6!rA0tMQ)CflUVBU#EJGvNXO+7|77?DS!QUO|_a_nv$1nLGD>=R1ebolYA8<>-q~<8m!Q{3rgR7gMG2{0=leCOCp4 zhY6&8l898l*Pt5p(}Ya$Su;$XF(WfeaHeJgSvU%6nlnSqaF&bKi#9W8>Z#%AX5tEk z>>L9l4$ca-le0nX;_OhnFM2q~v9}3=X9(h=yNx(hF@z;1$w|`he1gA^enKktG)f>L zh2p|BAzBeqKkz!({O4;>eN2q0owy3u7)_(mMugzeenE0IoatzdNJNM*Wege(jyg6& z5V0nL7^C%0kOUJqmqVo1qs`#=!VJ4E!1?%MNLBgR+gk5^8`nh2Xh)MYK z)1!WqVw#Mn6e<>5ATqmr14i)rBvc5F5@dWluYSwExL$vb~OcdA4?i*#e2Zp^+}pm+RD?eh7*z4X_Y3cI>j z-z)4oQg-`tbten%zIm!-cQ19{JzjRW7l-Z){rQ=vCc@G1)J(J;eGZz54QC3!b@fb@ z;X&X!zy%m4L^$*!$(h6&I15{xtuQ``n+m*liZdU8w-2Yy^pbhT8LLi?nDnz#Lx|Lv zRUe7Kg*sGO8%E3#bJ#dgN}OVQ?y;c3hyIIvSN$esz z3FGMjWhQD^q7}5V#OPYDsg;-@Um@|wpOhHXB>c`%4K8}8QmLeTEEIw}D#{5yB?R%k zzbv51poAtvF=<9jNPH|brC;078xs64_ZaUs)uJ%FUk zQYru#lmHN!6aoi(le{boF+Zu$Qvx3okgU|qTvkYwX|JlUw)kW+B%^3(nvaWs5_0CX z>R7ck!ON)xFAAY)A;rh|6rb@`2dgcG@Ou;&I1@mhAQIk73vx=KjYe4S0SPGym`AC# zc?QK(f(m3({VMoKo%VsIf?w6S>`yIi|+ zJ$E#c3!cx}*qrSG$XMNrI~R5qt*v=$>#}Lh+P=d4WBcmOPh)EbhrhgDVmynk1=rH` z2RHBCy#M|h)A56e=srspt?l1i2%C4Y^-gPc#|q5rTqR4sx(8?e;>>)1skY%k@4en) z?f!i2{#<9cSj**Wxl#-JXyoBYv85;9(vv%MvDk7c-*V||Popu=m2c_F9US_ytCf)y>NOd_WN_+*@?PVSlbr(-bI+{+owM`JwN@JYWblKoAdbK@Cz{Mv9|`EG@-@d>yRZz1e* zG-&9L2@^cHc2)-VB~Ho}HW6d_iIjlYb!Uu?PbVdm>SSf``l%@pX5%tDgCud%Z$X&$ zAncbAb}Pt@E9_Pj+j+d^074S-lSnT;jk%`rg6rKJ^DeM~wxg&K^x$i$(jI6V1(QUT5S#_eFd-0sf@p3@ z1_&MCbztWM8!4tfjtOJfaQcQ720I1{LT=^wbr;WqGik?k#RPPP8Z*ei+daBVW7kt# z>QA@sI{IyphpPSH&ggDwgEJZe ztW8gMclCsLNlpef%tic6=rZ`Q=#=I!s=8{ioIH*pd>Wu}ItdH-83eC@P^csiFl~v; z7*Nq}+-1driMS}piaDB)WC7trfPzr@&5Cz}GIbW-e8sHI0P3u+g7Mr2OI22tF;=KZ zt(|Qz7K5`~wF?1BA+SmEEvV**Qe$_Hsw>mh+cURjZvXn$uXFVsMS5?Z-dlEfa^ALr zWA{8+@^~NE@7eD==glR?x=1h3A6iRp-@K#by)<{G9hpdCJy>P`3@YCt=DQ`MrM zJ)p^?NQMYZlWU1Va_A!%vAVLU9b3E<;AuqXs8i+3n^i9fi`c6Uj-=9iflIQ}Rb( zPv|Y5)vfCmnFyoV2>}HI!4*NlDH{R;FIpos=f+njVto~MVhNER0TPc%I_$7VeWF*K zPR^@lC_sYSCfd%qxLTOy!?RSYaep23*KqZsEmE@;E=Mdbf@_G-YHfqMvD-L7EZmWi zk1HWVlW4D^b;Qm!>pKPEl_TN^W2d>PtPdMD2XwCrBPNMT2(VoX0Tgm~#8yUw9_AF!&()^^r-pJ55*gzn_-gp9U4~8!(Ds_faLfRC1FT&46ytGGMNfd&q!f(G*A#LFAbkL6dj)J2VK-%t@ z>n}N6a|2~h9i%hdwTqdB%-tJmW@GV4Lo-o?v0160`|e=IzEP1+f;y$8JrYQ6t14iEVp2ufNMqyq^d+m zWk`Ck6+9(hm3jiah19Exw_6|?XR3MzV6GG}SFgTS)hlXgFR*q5ti>rO$YpOX2Cu;5 zb(=>4uMPCoeKOv&k^SI1EL$d|3?E+c$6F+fj5R5iI9Ib(0_g1?)sFx#9s}>A70wF2 z#C8au0w|F6-aKYsKlX~0PpE?Cj?v7%AEiPE{woBprewu_4RSO9x_uZJNfw>KqE15%0d^__ZLl zM=>WM3{ClIrD^~;0i;+l3&*--^g7n-HS8O&n6J>4hKk7OU67Ko=c&N52FPwL-ZUno z>mb2n(f@`D7`g9fb0cNDbMf%P;lH$ZSHj0v+aTHBU`rj zD~})Gnei5x#yr!Q^*w5O*z^^%8-U5X?_0`wd-T@m8s#fB?pkhp=qxq{@{NI&iPg)6 z#-l*`OD@mcAb?EH)xKtZZN>YIuTD#v9KUz`elIk^1H8ZRe)eY39Voa1rRJTF_C4IU z%>7kpWmmqrv)J67Z|*KMA6lZn_SAi6CF<%iGVOWlAeg2Cy}Qgf=cVk-3iFuh`Y)}W z+k50ObM%K=ka}!=?S+h!xqqnJ*T2p5uckdvf4m?!f=+n8Wy}w z>J$baZog?X6Vqcl2-Sj+;-5-iPD6Bse>V`s6r>%eL&HI<&z-{ImgO^!hr`XZ)bBt`CzJwXEwi5PV&qf!FK$ z41BKksRuzZtH1kXOc4J2ZCV^|0nQYii75>5Gc9#$j!t2uz`qix8B8_lqjDs5_to3^ z38C(@_O2+LCPu#j`J3=B4?+bb3Igca4`zS!t0KKUPjAl-7U}jp-CkyFi_9HnsqRB} znR5PixJ)%b)LOFt}(Bf-P1+IYzMYar6NmXvCG8(PjP&NNGADoIvx? zhk3a?4%Jf=Ns`Yk1Zn$*@P0!;?fN6J=NV%r8=h@*lRKX6F_Ap^)Zrq%*`8(7!=v95 v(0sOM2ifxMB)N?|_N>!HvdwV3h#InY zhO(vI7ET|Dyx2vQq+YAL0cvcF)<%QE$S$y1Y=O0t{@8zOSxTi&T({R^%-48e7Kc$`AOD19 zm@ACP5JqH0OCNj6LM#H-xjv3?^p__*{k0M+{k0Jr{k0Q2{PKN{Q)Q$~V21eMMzj0e zZ?Gm;H)|q04KK-HCoovmKD&la*dXo zhw4O6+c}1jsu%{^9$GoRk1?NFU`Dx7&R;eCdr?k|1T4c^>r>2^n&Tc6>Cg5oynj~-~l~iIX+#^WBh?G!eIITnzX*Hfy1Z_;xgbR`? zNKq{a1n2^%e%!aOd9P29m1r^+S4NNf!l5?;M|`iJa2})m zrsFV8MVgS0`^Mx%JUy|0-+|_Fd1^Q*kyvvA22q-O6?m}zOqW3L38lm+*vB`W+b7 zx|d$fK92hQH}R%KRGAd4KP+?U^~d*Ju>LFGfJWkzaVfFZE65wwSor)W&h)zeiSJ)+ z;-FD$O*|?o{Yfd-m(kF5Jb>}0*YmESZQk3adV)5Sr^vC%B$392F+hd`0!dz2QwQ@F z^p7#0k+>qSV@X!N(>#?1)tvs%bu0AhSaPmEu8bEb@PSnoVTA@4BS=z7Or2T#S#x8; zqvk!^31>9pzNlkC{|LwI<7TK_Va~CEa)wd3Fc-|YT-qul!WMAlmZKJd33E-1pBs2) zO=}~8BR5&yVZtncvhxA~TmZjwQS|rV36bR zuOdIV#4I~1FIQY@U*@gv_P^afTXXI3rT%$-_j0-GQqN;(!9-D*C9??ZJ}}gvoW2+B{k82p!Vt8@bM$jWLnYGhzyi& z%IF`tLL{z4qzHYK)_hZIexZB1pnLTyh)c|WyQ`PnJMX!7F1mN!7{2e>I2-)XG1GqE z<(=Jnw|d+4?=8Ca->=%VRMm8^s_ELukY=*7=6kTkVUkLvZ%J)0a=*tvm8r<8OBTYS*HxJA10_v(Dd~`qimL*U6`qOvUac z_pbkN?|P=9h5opssgvWr;GCUadui~P1vPz1;E@k6mK>Di41X{SD!~evd{J2eFW#aC za0n(_3M9Ckkpf7RzOO_)Sp+Vn?<)Z#~=YRy00SVi{Gbgof2`zit zs1IYEV2Jwwq5lD*vfo8&;fHKSk+VIQ*gJ5tzuPcAQim5IC}&7L3JiXAE2_Op(3z1f zD10D_Y^R>ufiwz8h32b9q$o+MDzz9@_pdaWRhpi`u73pL5_A7KmNGkZ{n(;wFSQz| zzqaUVd5+QiXedaR@Rnc!DjqpR7K&a?@1i`>$AJr@H9om%X z3nIYoB5}|#B|_sDy=A3#J@4AcEA9n z1qCK!^y)|iB4#ZfjYt|qn8Rr_y9haq4G@0l0+J0_L(vKUDgxd+%p*&gwTFGQ$zeVG z%|^}|Fq&hdDEwy&pgop#GSP}&Alsur$EQM+i$(X{A#zMOuwR%|A;FZMkO|;yOxS0TyXBx(7lz>(e(GkqlBJpgxKk2MblAQ@Ycz+z?62erwJ+ z$u@YM1VaGJQi4iMn6&%2G}1y_(ir`xj~s&-5TAI&=BVT+iL@;-V|`w7i%OwZGEd z#@%Uib~p;*9%CH}_v9t%ufpZ&WLA@57MYTZ*F;MIt}FanLk7T-RCUN3 zK$Rd2QuWs>4b2~cKwwOPU|^-1BeB%EWFl6q;adz1*Cc>#xE<5n5irl3c0(379|SYn zy=wb07)ZSUVs*{-RL@Ft_1C|@;5h)%Le-|L{tx^g?!Ly)S2fLa-gi~J*Yks(*|T%e zIp?CQF{9fbbuHEJy;r~YmiHH3zwG_A_vd}Lee(yp7V5hfU2m*Z_>QuUo!sX;ogIf( zQTUB7N#P;c!vrBIR9x0lYf#8+!k|>@px97Mg>1>H=C@Qy&QM9x3o|Z_zf>D3oi?x+ z=?6K5&AB3QRRLQNO%482Lp6=+Xjaow0L}XLQX00hT?#r=k?^dpSr~a{Ff+@9KME~1 z?7P>n@7C5|gnoJc)AK(cyxlr~uzR85jo(@pU42jOE5Nd@{U~?msI$Xvrwjy04mkqD zlA|cvP`rks8wHYl5=4RKTg+Ha;5ih?98?Uokg?dTVfK;7WvvyGD=_Vt5FCLW(PJMqUHP~fu z)$IBA8)iC|5%n+k{O}Zp!M@qVwLS=m@#=Tyy^Y3YNCV9UZta*kJ@0B;-q?6uyixV> zpWI0NTjy=duTS3H&^6NqCV#bWE;#SnGShW`L(SEr9~_+vefav*4Tgn7f+&+6YCqo5 zz@X5iLq>ReV)Ai7- z?YRmzu*#yfW)_W(0_-m(D>2w;$}S-UJn$wUahyxx>)$E4pzG-2hxqCY4g5n+1G^U7 z-7lhnm^P*6S^<``zGPLNx60cI!X3Ko4&IA;E7;1)#SW&|Du`RM>0Ov_rsUB#8c!(J zwLH$b?y$8W&H=BN^{klhC@P{=w25}~ii6n{Z>jwt_*lm=h~SeAD-=S!Wr zvw0Nu`A=9*Bc=?Lwq*NK>;RgWpDik3yXX?#=1P7`8e1_y;42`8`c~3|HDr!t3`N$t zl0U^m+(DeK369xRAiDD;gZ-Hl!d=L1fqn!qf$R;UggY z`YVb9xFPHaxx9htNR&w*ihdO6rVSeXU99~X3d~dk=R~{6`C&4Qhe%7v0E#GzaS$rf zghG$Jo7ys+c9v-)1V-=+BG~@P#B-1&iOKrW5=Q)Dx4w^~{|>}@{5u4H!*hE+aV>Zb zzCiF-%mD7z9lUKv3ea=kTYYu&2b-5YTdr@o7W}wo*;D^f^qTLdqsyMH*Y{u7{(0(W zQy+itR&ajD;f2N{5AA$?#bbuAs+{R~=w#g9%qCRun)hx~aM2|`Z72={I%_%)abFyA zcJVKtBxKs&6l5pb>Wbp~wb<)*S9t5M`G`Nm7Z_PTob8NPTQeeqf1a4opt z-uWV^jlnR7B;<6S>i~L{~i}SG5ns+uE9f`IUs~JV) z@2CIqM{dc@A#WeQIF5Ot43;AwXDP(yFx49fq*F0S?Pfz3N^$+pH6Rl4uLu)YCbTFV zA_x{Iv|gp)XAB$}&IDBv0$brR*jh;GdrIKls;&PC2ly1kdLYgSVXpTty7uLVFqw^S z$nah*`=D&@^-nr)S^nkZt)YM4f17-9c=?s)haCIL-p35<-M8Gj|B2P&KKRgA&6QBh2y zm`3rJC`u(c!gv7%<`dK%Al_k~Se(|zN3|BK@W??02Q_D2oOfLA8Cc*644ZCO}7zQOB01Q@Yn11NF|Hah)f#KeDyzO|m^6ko5_Wk|uYjZsx z{?$@VV7?}>P}8zhb9lby@Mr9NP20kT*A|&$Uo%@D+Uy?wlIvk@WhH;9{9zN1M^7q1 zm!7ybvw_(oOB}_Muva>x)8@J7G-1daQ hqleQh!@Fjt7r4#KyzAZ5e|`Gz&OGE8zK#Y|{|7SgEP?<4