diff --git a/.github/workflows/integrity.yml b/.github/workflows/integrity.yml new file mode 100644 index 0000000..a4b53b2 --- /dev/null +++ b/.github/workflows/integrity.yml @@ -0,0 +1,41 @@ +name: integrity + +# Reproducibility + honesty gate. Runs on every push and PR. +# Corpus-dependent phases are skipped here (the HuggingFace download is ~50 MB); +# what this proves is that the docs are internally consistent, no link is dead, +# every retracted claim still carries its correction notice, and the benchmark runs. + +on: + push: + branches: [main] + pull_request: + +jobs: + integrity: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + + - name: Repo integrity (links, correction notices, probe-audit stamps) + run: python scripts/check_integrity.py + + - name: Benchmark runs and reports the capability comparison + run: | + cd benchmarks + python benchmark.py + + - name: Benchmark result is the documented 5/5 vs 0/5 + run: | + python - <<'PY' + import json + caps = json.load(open("benchmarks/results.json", encoding="utf-8"))["capabilities"] + sealed = sum(1 for r in caps.values() + if r.get("sealed_can_answer") or r.get("sealed_correctly_excludes_revoked")) + base = sum(1 for r in caps.values() + if r.get("baseline_can_answer") or r.get("baseline_correctly_excludes_revoked")) + print(f"sealed={sealed}/5 baseline={base}/5") + assert (sealed, base) == (5, 0), f"expected 5/5 vs 0/5, got {sealed}/5 vs {base}/5" + PY diff --git a/CITATION.cff b/CITATION.cff new file mode 100644 index 0000000..3f9ed9d --- /dev/null +++ b/CITATION.cff @@ -0,0 +1,58 @@ +cff-version: 1.2.0 +title: >- + Sumerian Agent Patterns: Empirical Mining of the SumTablets + Cuneiform Corpus for Multi-Agent Software Design Primitives +message: >- + If you use this repository, please cite it along with the + underlying SumTablets corpus (see references below). +type: software +authors: + - given-names: Kristian + family-names: Baer + affiliation: Northtek + email: info@northtek.io +repository-code: 'https://github.com/NORTHTEKDevs/sumerian-agent-patterns' +version: 0.2.0 +date-released: '2026-07-31' +abstract: >- + A reproducible pipeline that mines the SumTablets corpus + (91,606 transliterated Sumerian tablets) for recurring + bureaucratic patterns -- seal attribution, named time + periods, periodic reconciliation, witness clauses, addressed + message envelopes -- and maps them onto design primitives for + multi-agent AI systems. Reports per-genre Zipfian fits, + compression-based structural redundancy, an equidistant-letter- + sequence scan with permutation controls, and a measured + benchmark of a sealed-envelope write log against an anonymous + baseline. Version 0.2.0 documents the retraction of two statistical + findings (RULING-parity, withdrawn for an incorrect null hypothesis; + Zipf-as-DSL, withdrawn as a stream-length artifact) and a precision + audit finding five of ten regex probes unreliable. Anyone citing + this work should read CORRECTIONS.md, which records what was + withdrawn and the control that established it. +keywords: + - multi-agent systems + - agent memory + - provenance + - audit logs + - cuneiform + - Sumerian + - digital humanities + - corpus linguistics +license: CC-BY-4.0 +license-url: 'https://creativecommons.org/licenses/by/4.0/' +references: + - type: conference-paper + title: 'SumTablets: A Transliteration Dataset of Sumerian Tablets' + authors: + - family-names: Simmons + given-names: Cole + - family-names: Diehl Martinez + given-names: Richard + - family-names: Jurafsky + given-names: Dan + year: 2024 + collection-title: >- + Proceedings of the 1st Workshop on Machine Learning for + Ancient Languages (ML4AL), ACL 2024 + url: 'https://aclanthology.org/2024.ml4al-1.20.pdf' diff --git a/CORRECTIONS.md b/CORRECTIONS.md new file mode 100644 index 0000000..4758844 --- /dev/null +++ b/CORRECTIONS.md @@ -0,0 +1,153 @@ +# Corrections + +A running, dated log of claims this repository has retracted or materially revised. Newest first. + +Nothing is removed from the record. Superseded claims stay visible with the reason they failed, so anyone who read or cited an earlier version can see exactly what changed. + +--- + +## 2026-07-31 — PROBE AUDIT: 5 of 10 audited probes must not be cited; two implications restated + +A precision audit of the Phase 1 regex probes now runs as part of the pipeline (`scripts/phase1b_probe_validation.py` → [`outputs/probe_validation.md`](outputs/probe_validation.md)). Phase 1 only ever measured how often a probe *matched*; it never asked whether the matches meant what the probe's label claimed. They frequently do not. + +### Probes that must not be cited + +| Probe | Claimed role | Precision | What it is actually matching | +|---|---|---:|---| +| `god_dedication` | dedicatory formula | **6%** | `nam-lugal` = "kingship" and `nam-en` = "lordship" — abstract nouns, not dedications. Only `nam-ti` ("life", 10 of 182 matches) is dedicatory. | +| `excess_diri` | excess / surplus | **19%** | `diri` before a month name marks an **intercalary month**, not a ledger surplus. | +| `year_formula` | year-name | **34%** | `-` is a regex word boundary, so `\bmu\b` matches the verbal prefixes `mu-na-`, `mu-un-`, `mu-ni-` (48.6% of matches) plus `mu-kuₓ(DU)` "delivery". | +| `witness_eye` | witness clause | **41%** | `igi-ni-še₃` / `igi-zu-še₃` = "before him / before you" (pronominal), and `igi-nim` = "upper, northern" — a different lexeme entirely. | +| `king_title` | title: king | **42%** | Mostly `lugal-` as an element of a **personal name**. Even genuine hits are ambiguous: in Ur III administrative context `lugal` commonly means "owner, master", not "the king". | + +### Probes that validate + +`received_by` (`šu ba-ti`) 100%, `speak_to_him` (`u₃-na-a-du₁₁`) 100%, `total_audit` (`šu-nigin₂`) 100%, `seal_of_PN` (`kišib₃`) 96%, `son_of_PN` (`dumu`) 95%. The README headline figures rest on this group. + +### A probe that never fires + +`credit_mu_DU` searches for `mu-DU`, but the corpus writes this sign sequence as `mu-kuₓ(DU)` — **60 occurrences that the probe never sees.** `phase1_templates.py` does `if tablet_hits == 0: continue`, so a probe matching nothing vanishes from `templates.json` with no warning. The "delivery / credit" primitive was therefore never actually measured, despite appearing in the probe list. + +### Two README implications restated + +**Implication #1 previously asserted "witness clauses (`igi PN-šè`) are common" and "no important write is anonymous, undated, or unattributed."** Both are false in this sample. Genuine non-pronominal witness clauses appear on **0.4%** of administrative tablets. And **23.2%** of administrative tablets carry neither seal, year-name, nor witness, while ~75% carry no seal clause at all. Sealing is a strong minority practice, not a universal rule. The surviving claim — 25.4% sealed, 70.2% year-dated, both precision-validated — is still interesting and is what the section now says. + +**Implication #3 previously asserted that administrative tablets "close with `šu-nigin₂`" and that "nothing drifts silently."** `šu-nigin₂` is a precise probe (100%) but appears on only **2.2%** of administrative tablets; `la₂-ia₃` on 3.4%. The pattern is real and genuinely a Sumerian bookkeeping device, but it was described as characteristic of the corpus when it is a small minority practice. Prevalence tables per genre are now in `probe_validation.md`. + +**The general lesson, which is now stated in the report itself: attested is not the same as characteristic.** Several claims in this repo moved from "we found this pattern" to "this is how Sumerian bureaucracy worked" without the prevalence check that distinguishes them. + +### Epistemic status of this audit + +The discriminators encode standard dictionary values applied by a **non-specialist**. They are deliberately conservative — a match is marked FALSE only where the surface form makes an alternative reading unambiguous — so the reported error rates are **lower bounds**. This is not a substitute for review by an Assyriologist, and `REVIEWERS.md` still lists probe validation as the highest-value target for specialist time. + +--- + +## 2026-07-31 — WITHDRAWN: "Administrative and Royal genres are DSL-like (Zipf s ≈ 1.75 vs Lexical 1.11)" + +**Affected:** README artifact #3, the Findings table rows "Admin tablets are a domain-specific language" and "Lexical lists are closest to natural language", and `outputs/compression_findings.md` §1. + +**Original claim.** Per-genre Zipf exponents (Administrative s=1.746, Royal Inscription s=1.737, Letter s=1.607, Literary s=1.680, Lexical s=1.114, all R² 0.92–0.95) show that administrative and royal genres have a small high-reuse core vocabulary — behaving like a domain-specific language — while lexical lists sit near the natural-language value of s ≈ 1.0. + +**Status: withdrawn.** The cross-genre difference is an artifact of how many tokens each genre's stream contains. + +### What went wrong + +**1. A stream-length confound (fatal).** Zipf exponents estimated this way are strongly sample-size dependent, and these streams differ by ~60×: Literary 154,005 tokens, Royal 33,015, Administrative 32,493, Letter 18,664, Lexical **2,508**. Comparing raw exponents across them compares corpus sizes, not genres. Re-fitting every genre on equal-length blocks: + +| Genre | Stream length | s at native length | s at 2,508 tokens | sd | +|---|---:|---:|---:|---:| +| Administrative | 32,493 | 1.746 | **1.187** | 0.019 | +| Literary | 154,005 | 1.680 | **1.187** | 0.045 | +| Lexical | 2,508 | 1.114 | **1.114** | 0.000 | +| Royal Inscription | 33,015 | 1.737 | **1.200** | 0.022 | +| Letter | 18,664 | 1.607 | **1.210** | 0.015 | + +A native-length spread of 0.632 collapses to 0.096. Lexical looked like "natural language" only because its stream is the shortest and so had the least opportunity to accumulate a low-frequency tail. Verified under both contiguous-block and random-token subsampling, 25 draws each. + +**2. The estimator is unreliable.** `zipf_fit` is ordinary least squares on log-log rank-frequency data. That is a biased estimator of a power-law exponent, and the R² it produces is not a goodness-of-fit test — comparably high R² is routine for lognormal and exponential data (Clauset, Shalizi & Newman 2009, *SIAM Review* 51(4), 661–703). An MLE estimate on the same data disagrees with the OLS estimate in both magnitude and **rank order**, which is the diagnostic symptom. + +**3. Sensitivity to arbitrary preprocessing.** Hapax legomena are 34–59% of types depending on genre. Dropping them shifts exponents by up to 0.17 and reverses the direction of some genre comparisons — a result that flips on a preprocessing choice nobody documented is not a finding. + +### What is unaffected + +The §2 compression-redundancy comparison was tested for the *same* confound rather than assumed safe, and **survives**: at equal-length contiguous blocks the genre ranking is preserved (Royal > Administrative > Literary/Letter > Lexical) with magnitudes of the same order, and genres that swap rank do so within overlapping standard deviations. That control now runs as part of the pipeline. Note the control must use *contiguous* blocks — sampling scattered positions destroys the token adjacency zlib exploits and manufactures a collapse that is not there. An earlier version of this audit made exactly that mistake and had to be redone. + +### Doing this properly + +MLE fitting with a fitted `x_min`, a Kolmogorov–Smirnov goodness-of-fit statistic, and likelihood-ratio tests against lognormal alternatives — on length-matched samples. None of that is done here. The pipeline now reports the length control and an MLE exponent alongside the OLS fit so the discrepancy is visible rather than buried. + +--- + +## 2026-07-31 — CORRECTED: tablet citations under the seal finding + +README implication #1 cited "Tablets P101440, P132611, P117793, P145759" in support of the sentence "25.4% of administrative tablets carry `kišib₃`". Only **P101440** is an administrative tablet containing `kišib₃`. The other three are **Letters** and contain no seal clause; they carry attribution by a different mechanism — the `u₃-na-a-du₁₁` address formula plus a closing scribe signature (`dub-sar`) and filiation (`dumu` PN). All four are genuine corpus tablets and all four are relevant to attribution, but they were cited under a claim three of them do not support. Now cited separately with the mechanism each one actually demonstrates. + +Verified: the "Concrete Example" decomposition of P101440 elsewhere in the README is accurate in every element — `la₂-ia₃`, `kišib₃ {d}šul-gi-i₃-li₂`, `iti ezem-me-ki-gal₂`, `mu us₂-sa ki-maš{ki} ba-hul`, `siki`, and `geme₂` are all present in the tablet as transcribed. + +--- + +## 2026-07-31 — REVISED: probe frequencies restated with their over-catch + +`year_formula` uses the regex `\bmu\b`, which matches inside `mu-DU` (a delivery/credit term), `mu-ni` ("its name"), and verbal prefixes, because `-` is a regex word boundary. Of 551 matches in the Administrative sample, ~62% are plausibly year formulae and ~19% are clearly not. At the tablet level the effect is small — a stricter probe requiring `mu` followed by a year-name gives **70.6%** against the reported **74.2%** — because tablets carrying a spurious `mu-` usually carry a real year formula too. The README now states ~71% with both figures shown. + +A larger problem affects `king_title` (`\blugal\b`, reported at 48.6% of Administrative tablets), which is **not** measuring royal titles: 54% of its matches are `lugal-...` as an element of a *personal name*, and only ~12% are plausibly the title "king". That probe appears in `outputs/templates.json`; it is not among the README headline figures. It is flagged rather than silently repaired, because fixing it properly needs a name-vs-title disambiguation that regex cannot do. + +Spot-checked and confirmed exact: `seal_of_PN` 25.4% of Administrative, `speak_to_him` (`u₃-na-a-du₁₁`) 58.8% of Letters. Corpus totals confirmed exact: 91,606 tablets, 6,968,581 cuneiform glyphs, zero duplicate IDs, zero duplicate transliterations. + +--- + +## 2026-07-31 — WITHDRAWN: "`` is a validated logical row separator" + +**Affected:** README artifact #4, README implication #2, `outputs/compression_findings.md` §4, and the Findings table row "`` is a logical row separator (not visual)". + +**Original claim.** Adjacent ruling-delimited chunks share trigrams 30–500× more than a shuffled baseline (Royal Inscription p=0.002, Administrative p=0.005), establishing that the physical `` mark corresponds to a logical row boundary — and, by extension, motivating a three-tier SURFACE/COLUMN/RULING agent-memory design. + +**Status: withdrawn.** There is no statistical evidence in this corpus sample that `` marks content boundaries. + +### What went wrong + +Two independent defects, one fatal. + +**1. The null hypothesis was wrong (fatal).** The test compared observed cross-boundary trigram sharing against a null built by *pooling the tablet's tokens, shuffling them, and re-cutting at the same lengths*. That null destroys all local structure in the text. Beating it demonstrates only that Sumerian is locally coherent — a property of any natural language, and one that says nothing about `` in particular. Any arbitrary chunking of any real text beats that null. + +The null that actually isolates the question holds the real token order *and* the exact multiset of chunk lengths fixed, and permutes only **where the cuts fall**. Under that null the effect vanishes: + +| Genre | Pairs | Observed | Token-shuffle null (wrong) | Boundary-permute null (correct) | p (correct) | +|---|---:|---:|---:|---:|---:| +| Administrative | 17 | 0.176 | 0.004 | 0.343 | 0.781 | +| Literary | 108 | 0.731 | 0.018 | 0.889 | 0.905 | +| Lexical | 13 | 0.538 | 0.037 | 0.430 | 0.274 | +| Royal Inscription | 10 | 0.500 | 0.001 | 0.500 | 1.000 | + +The wrong null sits ~100× below the right one. Observed values land *at or below* the correct null in every genre — the opposite of the reported direction. + +**2. The p-value was computed against the wrong distribution.** It measured the fraction of *individual* null chunk-pairs whose value met or exceeded the observed **mean**, rather than building a null distribution *of the mean* (one value per permutation). Because shared-trigram counts are integers and heavily zero-inflated, the reported p-value reduced exactly to "the fraction of shuffled pairs sharing at least one trigram" — a different quantity from the one being claimed. In this instance it happened to land near the correctly computed value (0.005 vs 0.005 for Administrative), so it was not what produced the false positive, but it was wrong and is fixed. + +Both are corrected in `scripts/phase3_compression.py::ruling_parity`, which now reports both nulls side by side, uses a permutation distribution of the mean, and applies the standard `(r+1)/(n+1)` correction so p is never exactly 0. + +### What this does *not* say + +A null result on 10–17 adjacent-chunk pairs is weak evidence in either direction. This does not establish that `` is meaningless — the marks are plainly deliberate, physically drawn by scribes, and the three nested structural levels are real features of the artifact. It establishes only that **this test does not support the claim that was made from it**, and that the sample here is too small to settle the question either way. Testing it properly would need the full corpus rather than a 500-tablet-per-genre sample, and a boundary-placement null from the outset. + +### Knock-on effects + +- The three-tier SURFACE → COLUMN → RULING memory design is now presented as an **untested design proposal**, not a corpus finding. +- Anything in `outputs/FULL_IDEAS.md`, `outputs/summary.md`, or `outputs/reference_architecture.md` that leans on RULING-as-validated-row-boundary inherits this retraction. Those are hand-authored design documents and have not been individually rewritten; treat RULING-derived claims in them as proposals. + +### What is unaffected + +- **§1 Zipf** (Admin s=1.746, Royal s=1.737, Lexical s=1.114) — descriptive statistics, unchanged. +- **§2 Compression Δ** — descriptive, unchanged. Note it uses the same token-shuffle baseline, but its claim is only "more structured than random token order", which is what that baseline legitimately tests. +- **§3 ELS null result** (0 of 495 Bonferroni-significant) — this was always a correct permutation test: one null value per shuffle, compared against the observed statistic. Audited and unchanged. Being a null result, it is also the claim least at risk from this class of error. +- **Phase 1 probe frequencies** (`kišib₃` 25.4%, year-formula 74.2%, `u₃-na-a-du₁₁` 58.8%) — raw regex counts over the sample, no inference involved. +- **The `kishib3` benchmark** (5/5 vs 0/5 capability, +59% bytes, +37 tokens/write) — an engineering measurement, no statistics. + +### How it was found + +A reproducibility audit re-ran the full pipeline from a clean clone, then examined how each reported p-value was computed. The RULING-parity p-value was traced to a comparison against the pooled distribution of individual null pairs; investigating that led to examining the null itself. The alternative control was implemented independently and run at 500 permutations before the change was made to the pipeline. + +--- + +## Reporting a problem + +If you find an error, open an issue. Claims that survive adversarial checking are worth more than claims that were never checked, and a corrected repo is more useful than a confident one. Independent reproduction of any Phase 3 statistic is especially welcome — see the "Verifying your run matched" section of the README. diff --git a/README.md b/README.md index 39f01fb..aa40db4 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,9 @@ **Empirical mining of the SumTablets cuneiform corpus (91,606 tablets, 6.97M glyphs) for software-design primitives applicable to modern multi-agent AI systems.** -> Sumerian scribes ran a multi-agent bureaucracy 4,000 years ago. Their clay tablets carry sealed envelopes, named time periods, periodic audits, RPC headers, and witness sets — the same primitives modern agent systems are reinventing. This repo statistically validates which patterns are real (with p-values and cited tablet IDs) and translates them into agent-framework code shapes. +> **Read [CORRECTIONS.md](CORRECTIONS.md) first.** A self-audit on 2026-07-31 withdrew two headline statistical findings — RULING-parity (wrong null hypothesis) and Zipf-as-DSL (stream-length artifact) — and a probe-precision audit found that **5 of 10 audited regex probes match a different lexeme than their label claims** and must not be cited. What survives: the compression-redundancy contrast (verified against a length control), the ELS null result, four precision-validated probe frequencies, and the engineering benchmark. Every claim is tagged with its evidence strength, and every correction is reproducible from the pipeline rather than asserted — see [`outputs/probe_validation.md`](outputs/probe_validation.md). + +> Sumerian scribes ran a large administrative bureaucracy 4,000 years ago. Their clay tablets carry seal attributions, named time periods, sum-totals, and letter address formulae — recognisable ancestors of primitives modern agent systems are reinventing. This repo mines that corpus for those patterns, reports each with a measured precision and prevalence, and translates them into agent-framework code shapes. **Two statistical findings and five regex probes did not survive audit and are documented as withdrawn.** What carries weight: four precision-validated frequencies, one genre contrast verified against a length control, one null result, and an engineering benchmark. **Who this is for:** - Multi-agent / LLM agent framework developers (LangGraph, AutoGen, CrewAI, custom runtimes) @@ -18,11 +20,11 @@ Of the ~158 ideas in `outputs/FULL_IDEAS.md`, the following **9 first-class arti | # | Artifact | Where | What it gives you | |---|---|---|---| -| 1 | **Empirical method** — statistical mining of an ancient corpus to derive software-design primitives, with shuffled-baseline controls and Bonferroni correction | `scripts/phase{0,1,3}_*.py` | A reproducible pipeline you can re-run on any corpus to extract templates + structure | +| 1 | **Empirical method** — a reproducible corpus-mining pipeline with permutation nulls and Bonferroni correction, including the null-choice failure it caught in its own output | `scripts/phase{0,1,3}_*.py` | A pipeline you can re-run on any corpus to extract templates + structure, and a worked lesson in picking the null | | 2 | **9 named agent primitives** with cited tablet IDs and contracts | `outputs/primitives.json` | Single-responsibility agent designs grounded in real attestations (P/Q tablet IDs) | -| 3 | **Zipf-as-DSL detector finding** (Admin s=1.746, Royal s=1.737, Lexical s=1.114) | `outputs/compression_findings.md` §1 | Empirical method for unsupervised "is-this-a-DSL?" classification of any corpus | -| 4 | **RULING-parity finding** (Royal p=0.002, Admin p=0.005) | `outputs/compression_findings.md` §4 | Statistical proof that physical document boundaries map to logical row boundaries — informs vector-chunk strategy | -| 5 | **ELS-null result** (0 / 495 tests Bonferroni-significant) | `outputs/compression_findings.md` §3 | Defensive prior art against future numerology / "hidden code" claims on cuneiform | +| 3 | **Zipf-as-DSL detector — WITHDRAWN** (2026-07-31, see [CORRECTIONS.md](CORRECTIONS.md)) | `outputs/compression_findings.md` §1 | The genre difference was a stream-length artifact; §1 now documents the length control that killed it | +| 4 | **RULING-parity — NULL result** (claim withdrawn 2026-07-31, see [CORRECTIONS.md](CORRECTIONS.md)) | `outputs/compression_findings.md` §4 | A worked example of how the choice of null hypothesis manufactures a finding — and how it was caught | +| 5 | **ELS-null result** — 3 nominal hits at p<0.01 across 495 tests, against ~5 expected by chance | `outputs/compression_findings.md` §3 | Defensive prior art against future numerology / "hidden code" claims on cuneiform | | 6 | **Reference architecture** — composes the primitives into a multi-agent design with Python + Rust pseudocode | `outputs/reference_architecture.md` | Drop-in design doc you can adapt to any agent runtime | | 7 | **Quantitative benchmark** — sealed envelope vs anonymous baseline, measured | `benchmarks/RESULTS.md` | Hard numbers (+59% bytes, +37 tokens, +7µs per write) and a 5/5-vs-0/5 capability comparison | | 8 | **Reference implementation** of `kishib3` (sealed envelope) in 250 LoC stdlib Python | `benchmarks/kishib3.py` | Working code for one of the primitives — clone, adapt, ship | @@ -32,13 +34,36 @@ Of the ~158 ideas in `outputs/FULL_IDEAS.md`, the following **9 first-class arti --- +## Claims and Evidence Strength + +Every substantive claim in this repo, and exactly how much weight it can bear. Read this before citing anything. + +| Claim | Evidence type | Strength | Notes | +|---|---|---|---| +| `kišib₃` (seal) on 25.4% of Administrative tablets; year-names on 70.2%; `u₃-na-a-du₁₁` in 58.8% of Letters; `šu ba-ti` on 15.2% | Regex counts over a 2,069-tablet sample, **precision-audited** | **Solid** — descriptive | These four probes validate at 96–100% precision ([`probe_validation.md`](outputs/probe_validation.md)). Other probes in `templates.json` do **not** — see the row below. | +| ~~Administrative and Royal are more formulaic than Lexical (Zipf s 1.75 / 1.74 vs 1.11)~~ | OLS fit on log-log rank-frequency | **WITHDRAWN** | A stream-length artifact: at equal length every genre falls in 1.11–1.21. The estimator is also biased, and its R² is not a goodness-of-fit test. See [CORRECTIONS.md](CORRECTIONS.md). | +| Genres differ in structural redundancy; Royal Inscription and Administrative rank highest, Lexical lowest (Δ +0.05 to +0.10) | zlib ratio, real vs token-shuffled, **verified against an equal-length control** | **Moderate–solid** — descriptive | The strongest surviving positive statistic here. The token-shuffle baseline is legitimate for this particular claim, which is only "more structured than random token order". Unlike Zipf, the genre ranking survives equal-length contiguous blocks. Still not a test of any specific structural hypothesis. | +| No hidden periodic encoding in the corpus | ELS scan, 99 skips × 5 genres × 1,000 permutations | **Moderate** — null result | 3 nominal hits (p<0.01) vs ~5 expected by chance. **Underpowered for the Bonferroni threshold it originally claimed** — see §3 resolution caveat. Reads as "no signal above chance", not as a proof of absence. | +| Sealed envelopes answer 5/5 audit queries; anonymous logs 0/5, at +59% bytes / +37 tokens / +7µs per write | Executed benchmark, 100k writes | **Solid** — engineering measurement | Not a statistical claim. The 0/5 is definitional (an anonymous log lacks the fields), so it demonstrates a design consequence, not a discovery. Latencies are hardware-dependent. | +| ~~`` marks logical row boundaries~~ | Permutation test | **WITHDRAWN** | Artifact of the wrong null. See [CORRECTIONS.md](CORRECTIONS.md). | +| Frequencies for `king_title`, `year_formula` (loose), `witness_eye`, `god_dedication`, `excess_diri` in `templates.json` | Regex counts | **DO NOT CITE** | Precision 6–42%. These probes match a different lexeme than their label claims (`nam-lugal` = "kingship" not a dedication; `igi-zu-še₃` = "before you" not a witness; `lugal-` mostly a personal-name element). Quantified in [`probe_validation.md`](outputs/probe_validation.md). | +| The 9 named agent primitives | Hand-authored mapping from templates | **Design proposal** | Grounded in cited tablet IDs, but the mapping from a Sumerian formula to an agent contract is an argument, not a measurement. `primitives.json` tags each as `validated` or `speculative` — that flag refers to whether the *pattern* was observed, never to whether the agent design works. | +| Three-tier SURFACE → COLUMN → RULING memory | — | **Untested proposal** | Its empirical support was the withdrawn claim above. | +| The reference architecture, `FULL_IDEAS.md`, `summary.md` | Hand-authored | **Ideation** | No evidence claimed. Not generated by any script. | + +**The honest summary:** this repo's defensible contributions are (a) a reproducible corpus-mining pipeline, (b) descriptive statistics on genre formulaicity, (c) a null result on hidden encodings, (d) an engineering benchmark for provenance-carrying write logs, and (e) a documented worked example of a null-hypothesis error and its correction. The mapping from cuneiform to agent design is an argument offered for its generative value, not a validated result — and the one place that mapping was given inferential backing, the backing did not survive audit. + +--- + ## Why You Should Care (Findings → Actions) Three concrete things you can build differently after reading this: ### 1. Wrap every agent write in a sealed envelope -**Finding.** 25.4% of administrative tablets carry `kišib₃` (seal of so-and-so), 74.2% are dated by year, and witness clauses (`igi PN-šè`) are common. No important write is anonymous, undated, or unattributed. Tablets P101440, P132611, P117793, P145759. +**Finding, restated after audit.** 25.4% of administrative tablets carry `kišib₃` (seal of so-and-so) and 70.2% are dated by year-name. Both probes validate cleanly (`seal_of_PN` 96% precision, and the 70.2% is the *strict* year probe — see [`outputs/probe_validation.md`](outputs/probe_validation.md)). Sealed administrative example: **P101440**. Letters attribute differently — via `u₃-na-a-du₁₁` plus a closing scribe-and-filiation signature (`dub-sar`, `dumu` PN) rather than a seal: **P132611, P117793, P145759**. + +**Two claims previously made here are withdrawn.** (a) *"Witness clauses (`igi PN-šè`) are common"* — they are not: genuine non-pronominal witness clauses appear on **0.4%** of administrative tablets, and 52% of that probe's raw matches are pronominal forms like `igi-zu-še₃` ("before you") that are not witness clauses at all. (b) *"No important write is anonymous, undated, or unattributed"* — **23.2%** of administrative tablets in this sample carry neither seal, year-name, nor witness, and ~75% carry no seal clause. Sealing is a strong minority practice, not a universal rule. **Action.** Make every state-changing call in your agent runtime carry `(payload, by_seal, witnesses, period)`. Audit becomes a property of the envelope, not a separate concern bolted on per agent. Replay across any time window becomes trivial. @@ -47,19 +72,23 @@ Three concrete things you can build differently after reading this: - **Capability**: sealed log answers 5/5 audit queries (who wrote X, all writes by principal, all writes in period, integrity verification, replay-after-cascade-revoke). Anonymous log answers **0 of 5**. The capability gap is total, not partial. - **The killer query**: replay-as-of after revoking a parent principal. Baseline returns all 50 writes (silent drift). Sealed log returns 0 — every revoked descendant is correctly excluded. -### 2. Tier your agent memory as session → topic → row +### 2. Tier your agent memory as session → topic → row — *design proposal, not a finding* + +**Status.** This section previously claimed statistical confirmation that `` is a logical row separator (Royal p=0.002, Admin p=0.005). **That claim is withdrawn.** It came from a null hypothesis that shuffled the tablet's tokens, which destroys all local structure and therefore only shows that Sumerian text is locally coherent — true of any natural language. Under a null that varies *only where the boundaries fall*, the effect disappears entirely in every genre. See [CORRECTIONS.md](CORRECTIONS.md) and `outputs/compression_findings.md` §4. -**Finding.** Sumerian tablets have three layers of structure: physical surface (obverse/reverse), logical column, atomic row marked by ``. We statistically confirmed `` is a real row separator: adjacent ruling-bounded chunks share trigrams 30–500× more than shuffled baselines (Royal Inscription p=0.002, Administrative p=0.005). +**What is still true.** Tablets do carry three nested physical structures — surface, column, and ruling-delimited row. That is a description of the artifact, not a statistical result. -**Action.** Replace flat agent memory with three tiers: SURFACE (session) → COLUMN (topic) → RULING (row). Reads return the smallest tier that satisfies the query — never drag back the whole session when one row answers. +**The proposal.** Tiering agent memory as SURFACE (session) → COLUMN (topic) → RULING (row), where reads return the smallest tier that satisfies the query, is a reasonable design. It is offered here as an untested idea. This repo provides no evidence that it works. ### 3. Replace silent token/cost rollups with periodic signed audits -**Finding.** Administrative tablets close with `šu-nigin₂` (sum-total) — a periodic reconciliation. Shortfalls and excesses are named explicitly: `la₂-ia₃` (deficit owed by named person), `diri` (excess). Nothing drifts silently. +**Finding, corrected after audit — this pattern is real but RARE.** Some administrative tablets close with `šu-nigin₂` (sum-total), and shortfalls are named explicitly with `la₂-ia₃` (deficit owed by a named person). The `šu-nigin₂` and `la₂-ia₃` probes are precise, but the prevalence is far lower than earlier wording implied: **`šu-nigin₂` appears on 2.2% of administrative tablets, `la₂-ia₃` on 3.4%.** The earlier claim that administrative tablets "close with `šu-nigin₂`" and that "nothing drifts silently" described a practice attested on a small minority of tablets as though it were the norm. It is a genuine Sumerian bookkeeping device; it is not characteristic of the corpus. `diri` is **not** usable as an "excess" marker at all — 19% precision, because it frequently marks an *intercalary month*, not a ledger surplus. **Action.** For any ledger-shaped agent state (token usage, tool-call counts, cost tracking, evidence accumulation), close periods at fixed intervals with a signed audit. Deficits and excesses must be named and attributed to a counterparty. -**Plus a defensive null result:** A 99-skip × 5-genre × 1,000-shuffle ELS scan found zero hidden codes (0 of 495 Bonferroni-significant tests). Useful prior art if anyone tries to sell you "Sumerian secret-code AI." +**Plus a defensive null result:** A 99-skip × 5-genre × 1,000-permutation ELS scan found no periodic encoding — 3 nominally significant hits (p<0.01) across 495 tests, where chance alone predicts ~5. Useful prior art if anyone tries to sell you "Sumerian secret-code AI." + +*Stated precisely, because it matters:* at 1,000 permutations the smallest attainable p is ~0.001, which is ~10× the Bonferroni threshold — so this scan **cannot** return a Bonferroni-significant result by construction, and the earlier "0 of 495 Bonferroni-significant" framing overstated its power. The informative comparison is the uncorrected hit rate against chance expectation, above. > **Want more?** `outputs/summary.md` ranks the top 10 ideas by leverage. `outputs/FULL_IDEAS.md` lists ~158 across 16 categories. `outputs/reference_architecture.md` has full code shapes in Python and Rust. @@ -144,10 +173,10 @@ This is what every line in `outputs/templates.json` is doing — taking a real t | `mu` X | year of X | Named time period (event-named, not numeric) | | `mu us₂-sa` X | year after the year of X | Relative time reference resolved at write-time | | `iti` X | month of X | Calendar month sub-period | -| `šu-nigin₂` | sum-total | Periodic audit / signed reconciliation | +| `šu-nigin₂` | sum-total | Periodic audit / signed reconciliation *(precise probe, but only 2.2% of Admin tablets)* | | `la₂-ia₃` | deficit | Named outstanding obligation (never silent) | -| `diri` | excess | Named surplus requiring disposition | -| `igi` PN-šè | before [person] | Witness clause — live attestation at write-time | +| `diri` | excess *(also: intercalary month — the probe for this is 19% precise, do not cite it)* | Named surplus requiring disposition | +| `igi` PN-šè | before [person] | Witness clause *(genuine ones are rare: 0.4% of Admin tablets; the bare `igi-...-še₃` probe mostly catches "before him/you")* | | `dumu` PN | son of [person] | Filiation edge in principal/identity graph | | `u₃-na-a-du₁₁` | speak to him | Letter address formula — RPC envelope opener | | `dub-ba-ni` | his tablet | Reference to a prior message (thread-id) | @@ -167,14 +196,14 @@ The numbers behind the implications above. All claims here are reproducible from | Finding | Statistic | Genre / Coverage | |---|---|---| -| Admin tablets are a domain-specific language | Zipf exponent s = 1.746 (R²=0.93) | Administrative | +| ~~Admin tablets are a domain-specific language~~ **WITHDRAWN** | Length artifact; at equal stream length every genre sits at s = 1.11–1.21 ([CORRECTIONS.md](CORRECTIONS.md)) | Administrative | | Royal Inscription is the most-templated genre | Compression-Δ = +0.099 vs shuffled baseline | Royal Inscription | -| Lexical lists are closest to natural language | Zipf s = 1.114 (R²=0.92) | Lexical | +| ~~Lexical lists are closest to natural language~~ **WITHDRAWN** | Same length artifact — Lexical merely has the shortest stream (2,508 tokens) | Lexical | | Letters are short single-purpose RPCs | Lowest marker density (0.02 RULING/tab) | Letter | -| `` is a logical row separator (not visual) | p = 0.002 (Royal), p = 0.005 (Admin) for cross-ruling trigram parity | Royal, Admin | -| No hidden encodings in the corpus | 0 of 495 ELS tests Bonferroni-significant | All genres | -| Seal-of-PN clauses are pervasive | 25.4% of Admin tablets | Administrative | -| Year-formulas are universal envelopes | 74.2% Admin, 65% Letter, 62.4% Royal | Across genres | +| ~~`` is a logical row separator~~ **WITHDRAWN** | Null under the correct control; the original p-values came from a null that destroys all local structure ([CORRECTIONS.md](CORRECTIONS.md)) | Royal, Admin | +| No hidden encodings in the corpus | 3 of 495 ELS tests nominally p<0.01, vs ~5 expected by chance | All genres | +| Seal-of-PN clauses are a strong minority practice | 25.4% of Admin tablets (probe precision 96%); ~75% carry no seal | Administrative | +| Year-formulae are common in Administrative, less so elsewhere | Strict probe: 70.2% Admin, 45.4% Letter, 41.8% Royal (loose `\bmu\b` probe over-reports these as 74.2 / 73.8 / 62.6 — 53% of its matches are verbal prefixes) | Across genres | | Letters are addressed RPCs | `u₃-na-a-du₁₁` in 58.8% of Letters | Letter | Full statistics with shuffled-baseline controls in `outputs/compression_findings.md`. Per-tablet pattern citations in `outputs/templates.json`. @@ -186,12 +215,17 @@ Full statistics with shuffled-baseline controls in `outputs/compression_findings ``` . ├── README.md this file +├── CORRECTIONS.md dated log of retracted / revised claims -- read first +├── REVIEWERS.md where to attack this, for reviewers +├── CITATION.cff citation metadata ├── LICENSE CC BY 4.0 (docs and analysis artifacts) ├── LICENSE-CODE MIT (Python scripts) ├── requirements.txt Python deps ├── scripts/ │ ├── phase0_sample.py loads SumTablets, builds stratified samples │ ├── phase1_templates.py extracts genre templates and probe hits +│ ├── phase1b_probe_validation.py precision-audits those probes (which ones are citable) +│ ├── check_integrity.py pre-publish gate: links, correction notices, audit stamps │ └── phase3_compression.py Zipf, compression, ELS, RULING-parity analysis ├── benchmarks/ │ ├── kishib3.py reference sealed-envelope implementation (~250 LoC) @@ -200,6 +234,8 @@ Full statistics with shuffled-baseline controls in `outputs/compression_findings │ ├── results.json raw measurement output │ └── RESULTS.md report with measured numbers and caveats └── outputs/ + ├── probe_validation.md per-probe precision + prevalence -- READ BEFORE CITING + ├── probe_validation.json machine-readable probe audit ├── templates.json 229 templates × {genre, pattern, role, frequency, tablet IDs} ├── primitives.json 9 named agent primitives (6 rubric + 3 data-justified) ├── compression_findings.md Phase 3 statistics with p-values @@ -211,26 +247,96 @@ Full statistics with shuffled-baseline controls in `outputs/compression_findings ## How to Reproduce +**Requirements:** Python ≥ 3.10. No GPU, no API keys, no HuggingFace account. Total runtime ≈ 8 minutes on a laptop (phase 3 dominates at ~5 min). Disk: 48 MB of parquet in `data/`, plus a HuggingFace `datasets` cache of similar size on first download. + ```bash +git clone https://github.com/NORTHTEKDevs/sumerian-agent-patterns.git +cd sumerian-agent-patterns + +python -m venv .venv +source .venv/bin/activate # Windows: .venv\Scripts\activate pip install -r requirements.txt -python scripts/phase0_sample.py # downloads SumTablets, persists local parquet, builds samples -python scripts/phase1_templates.py # writes outputs/templates.json -python scripts/phase3_compression.py # writes outputs/compression_findings.md + phase3_raw.json + +python scripts/phase0_sample.py # ~2 min downloads SumTablets, caches parquet, builds samples +python scripts/phase1_templates.py # ~10 s writes outputs/templates.json +python scripts/phase1b_probe_validation.py # ~5 s writes outputs/probe_validation.md <- READ THIS +python scripts/phase3_compression.py # ~5 min writes outputs/compression_findings.md + phase3_raw.json +python scripts/check_integrity.py # ~1 s links resolve, corrections present, audit stamps intact +``` + +The corpus benchmark is independent of the above — pure stdlib, no dependencies, no corpus: + +```bash +cd benchmarks && python benchmark.py # ~6 s, writes results.json +``` + +### Verifying your run matched + +The committed files in `outputs/` are exactly what these scripts produce. After a full re-run: + +```bash +git diff --stat outputs/ ``` -All scripts are seeded (`random_state=42`, `np.random.default_rng(42)`) and reproducible end-to-end. Phase 0 downloads ~50 MB of corpus data from HuggingFace on first run, caches it locally as parquet, and reuses on subsequent runs. +**Expected: no output.** Anything else is a genuine divergence worth an issue. + +`python scripts/check_integrity.py` should print `[integrity] clean`. It verifies that every relative link resolves, that every claim marked WITHDRAWN in the README still carries a correction notice *adjacent to it* in the derived documents, that every DO-NOT-CITE probe is stamped in `templates.json`, and that the corpus totals match. It runs in CI on every push. Its checks were validated by deliberately breaking each one and confirming it fails — one check was found vacuous that way and tightened. + +What you should see in the console along the way — these are the load-bearing numbers: + +| Check | Expected | Where | +|---|---|---| +| Corpus rows loaded | 91,606 | phase 0 | +| Stratified sample | 2,069 tablets + 197 long-form | phase 0 | +| `seal_of_PN` in Administrative | 25.4% (n=127) | phase 1 | +| Probes rated DO-NOT-CITE | 5 of 10 audited | phase 1b | +| Probes that never fire | 1 (`credit_mu_DU`) | phase 1b | +| Zipf s at native length (confounded, see §1) | 1.746 / 1.737 / 1.114 | phase 3 | +| Zipf s at equal length (the real comparison) | all genres 1.11–1.21 | phase 3 | +| RULING parity (primary null) | not significant in any genre | phase 3 | +| ELS nominal hits (p<0.01) | **3** of 495, vs ~5 expected by chance | phase 3 | +| Sealed-vs-anonymous audit queries | 5/5 vs 0/5 | benchmark | + +### Reproducibility caveats — read before filing an issue -`reference_architecture.md`, `summary.md`, `FULL_IDEAS.md`, and `primitives.json` are hand-authored design artifacts that cite outputs from the scripts. +- **`benchmarks/results.json` is not byte-reproducible, by design.** Seal IDs come from `secrets.token_hex`, and absolute latencies are hardware-dependent. The byte/token overheads and the 5/5-vs-0/5 capability result are stable; the µs figures will not match ours. +- **`benchmarks/RESULTS.md` is hand-authored** and is not regenerated by `benchmark.py`. It interprets `results.json`. +- **Determinism depends on your NumPy version.** Phase 3 draws every shuffle from one seeded `np.random.default_rng(42)` consumed sequentially across all four analyses. NumPy does not guarantee `Generator` stream stability across major versions, so a future NumPy could shift the ELS reference tables. It is stable across NumPy 2.x — verified end-to-end on 2.5.1. The ELS null conclusion is not sensitive to this. +- **There is no `phase2` script.** Phase 2 was the hand-authored mapping step from templates to agent primitives; its output is `outputs/primitives.json`. The numbering is historical, not a missing file. +- The pipeline is stage-ordered: phase 1 and phase 3 both read the parquet files that phase 0 writes into the (gitignored) `data/` directory. Run phase 0 first. + +**Verified end-to-end on 2026-07-31:** clean clone, fresh venv, Python 3.14.2, Windows 11, `datasets` 5.0.1 / `pandas` 3.0.5 / `numpy` 2.5.1 / `scipy` 1.18.0. `outputs/templates.json` and `outputs/phase3_raw.json` regenerated byte-identical to the committed copies. + +`reference_architecture.md`, `summary.md`, `FULL_IDEAS.md`, and `primitives.json` are hand-authored design artifacts that cite outputs from the scripts — they are not regenerated by any script. ## Methodology 1. **Sample.** Stratified-sample 500 tablets per genre (Administrative, Literary, Lexical, Royal Inscription, Letter) plus up to 50 long-form tablets per genre. Total sample: 2,069 tablets + 197 long-form. 2. **Templates.** For each genre: structural-marker statistics (``, ``, ``, ``); per-position opening/closing line templates; distinctive bigrams and trigrams via genre log-odds vs other genres; hand-coded regex probes for known bureaucratic primitives (seal-of, year-formula, total/audit, deficit, witness, etc.). Every template carries cited tablet IDs. -3. **Compression and ELS.** Per-genre Zipfian fit; compression-ratio Δ between raw and shuffled token streams; equidistant-letter-sequence (ELS) decimation at skips 2–100 with 1,000 shuffled-baseline controls (Bonferroni-corrected for 495 tests); cross-RULING trigram parity vs within-tablet shuffled baseline. +2b. **Probe validation.** Each probe's matches are partitioned by hand-written discriminators into TRUE / FALSE / UNCLEAR against its claimed semantic role, yielding a precision figure and a per-genre prevalence table. Probes below ~50% precision are marked DO-NOT-CITE rather than silently repaired. This step exists because the original release cited probe frequencies that were measuring the wrong lexeme. +3. **Compression and ELS.** Per-genre Zipfian fit (OLS on log-log, reported alongside an MLE exponent and an equal-length control that shows the cross-genre comparison does not survive); compression-ratio Δ between raw and shuffled token streams (with the same equal-length control, which it does survive); equidistant-letter-sequence (ELS) decimation at skips 2–100 against 1,000 permutations, `(r+1)/(n+1)` corrected, Bonferroni threshold reported together with the resolution caveat that the permutation count cannot reach it; cross-RULING trigram parity against two nulls — a token-shuffle null retained only to demonstrate it is the wrong control, and a boundary-permutation null holding token order and chunk lengths fixed and varying only where cuts fall (the primary test). 4. **Mapping.** For each empirical template, propose a named single-responsibility agent primitive with inputs, outputs, state, tools, and guardrails. Distinguish validated-by-data from speculative. +## For Reviewers and Future Readers + +If you are evaluating this repo rather than using it, read in this order: + +1. **[CORRECTIONS.md](CORRECTIONS.md)** — everything that has been retracted, with the control that killed it. Two statistical findings and five regex probes. +2. **[outputs/probe_validation.md](outputs/probe_validation.md)** — which frequencies are citable (precision + per-genre prevalence). +3. **[REVIEWERS.md](REVIEWERS.md)** — where I think this is still weakest, and the specific questions I cannot answer myself. + +The hand-authored design documents (`outputs/reference_architecture.md`, `outputs/summary.md`, `outputs/FULL_IDEAS.md`) were written against the original findings. They are **preserved rather than rewritten**, with inline correction notices at each retracted passage, so the record of what was claimed stays visible. `scripts/check_integrity.py` enforces that coupling — a retracted claim cannot sit in this repo without its correction beside it. + +**Nothing here has been peer-reviewed.** The probe audit was performed by a non-specialist against standard dictionary values and is deliberately conservative, so its error rates are lower bounds. + +--- + ## Honest Limits +- **Two of the three original statistical findings did not survive audit** (RULING-parity, Zipf-as-DSL — see [CORRECTIONS.md](CORRECTIONS.md)). Of what remains, the compression-redundancy ranking is the only positive inferential-flavoured result, and it is a descriptive contrast against random token order rather than a test of a specific structural hypothesis. Treat this repo's statistical contribution as modest. +- **The regex probes have had no specialist review**, and they generate the most-cited numbers here. One (`king_title`) is known to be measuring personal names rather than titles. Others carry the same class of risk and have not been quantified. +- **Multiple comparisons are corrected only within the ELS scan**, not across the ~40 hypotheses the repo entertains in total. +- The Zipf fit uses OLS on log-log rank-frequency data, which is a biased power-law estimator whose R² is not a goodness-of-fit test. It is retained, with an MLE exponent beside it, so the discrepancy is visible — not because it is the right method. - We sampled 2.3% of the corpus. Findings are strong for Ur III administrative tablets and Old Babylonian literary tablets; weaker for everything else. - The corpus is 92%+ administrative — generalizing about "Sumerian thought" from this sample would be like generalizing about "civilization" from accounting receipts. - Lexical findings rely on only 69 tablets; the Lexical-list architectural slot is real but the actual taxonomic content needs to come from external sources (CDLI, ePSD2). diff --git a/REVIEWERS.md b/REVIEWERS.md new file mode 100644 index 0000000..ff73612 --- /dev/null +++ b/REVIEWERS.md @@ -0,0 +1,75 @@ +# Notes for Reviewers + +Thank you for looking at this. This document points you straight at the weakest parts rather than making you find them. The list is ordered by where a serious problem is most likely to remain. + +## What this repo is, stated plainly + +A reproducible pipeline that mines a cuneiform corpus for recurring bureaucratic patterns, an argument that those patterns map usefully onto multi-agent software design, and one engineering benchmark. + +It is **not** an Assyriology contribution. The corpus observations are, as far as I know, unsurprising to a specialist — Sumerian administrative practice as a proto-information-system is well-trodden ground. The intended contribution is the *pipeline*, the *mapping*, and honest accounting of how much each claim bears. + +**A self-audit on 2026-07-31 withdrew two of the three original statistical findings.** Both are documented in [CORRECTIONS.md](CORRECTIONS.md) with the controls that killed them: + +- **RULING-parity** (was Royal p=0.002, Admin p=0.005) — the null shuffled all tokens, which destroys the local coherence any natural text has. Under a null that permutes only boundary *placement*, the effect vanishes in every genre. +- **Zipf-as-DSL** (was Admin s=1.746 vs Lexical s=1.114) — a stream-length artifact. Streams differ ~60× in length; at equal length every genre falls in 1.11–1.21. + +What survives: the compression-redundancy ranking (tested against the same length confound and robust to it), the ELS null result, descriptive probe frequencies, and the benchmark. The claim-by-claim table is in the README under **Claims and Evidence Strength** — please attack that table first. If a claim is graded too generously, that is the most damaging error left. + +## Where I would attack it first + +1. **The regex probes — still the highest value for your time.** Every Phase 1 frequency comes from hand-written regexes over transliterated text (`PROBES` in `scripts/phase1_templates.py`). I have now precision-audited 10 of them (`scripts/phase1b_probe_validation.py` → [`outputs/probe_validation.md`](outputs/probe_validation.md)) and **5 failed badly enough to be marked DO-NOT-CITE**: + + | Probe | Claimed role | Precision | Actually matching | + |---|---|---:|---| + | `god_dedication` | dedicatory formula | 6% | `nam-lugal` "kingship", `nam-en` "lordship" — abstract nouns | + | `excess_diri` | excess / surplus | 19% | `diri` + month = **intercalary month** | + | `year_formula` | year-name | 34% | verbal prefixes `mu-na-`, `mu-un-`, `mu-ni-` | + | `witness_eye` | witness clause | 41% | `igi-zu-še₃` "before you"; `igi-nim` "upper/north" | + | `king_title` | title: king | 42% | `lugal-` as a personal-name element | + + Validating cleanly: `received_by` 100%, `speak_to_him` 100%, `total_audit` 100%, `seal_of_PN` 96%, `son_of_PN` 95%. A sixth probe, `credit_mu_DU`, **never fires at all** — it searches `mu-DU` while the corpus writes `mu-kuₓ(DU)` (60 occurrences), and Phase 1 silently drops zero-hit probes. + + **What I need from you:** my discriminators encode standard dictionary values applied by a non-specialist, and are deliberately conservative, so the reported error rates are *lower bounds*. Are any of my TRUE buckets actually wrong? In particular: (a) is bare `lugal` in Ur III administrative context ever safely "the king" rather than "owner/master"? (b) is my `igi PN-še₃` TRUE bucket really a witness clause, or does it include other uses? (c) does `diri` + quantity reliably mean surplus? Correcting a *false TRUE* matters more than adding a false FALSE, because the TRUE buckets are what the README still cites. + +2. **Is the replacement RULING null itself correct?** It holds real token order and the exact multiset of chunk lengths and permutes only where cuts fall (`ruling_parity`). If that control is also flawed, I want to know before this goes anywhere. + +3. **Does the compression-Δ result really survive?** It uses the same token-shuffle baseline that broke the RULING claim. I argue it is legitimate *there* because the claim is only "more structured than random token order", and I tested it against the length confound that broke Zipf — the genre ranking holds at equal-length contiguous blocks. That control must use contiguous blocks; scattered-position sampling destroys adjacency and manufactures a collapse. **I made exactly that mistake during the audit and had to redo it**, which is reason enough to check my reasoning rather than trust it. + +4. **ELS statistical power.** At 1,000 permutations the minimum attainable p is ~0.001, roughly 10× the Bonferroni threshold of 0.000101 — so the scan cannot return a Bonferroni-significant result by construction. It now reports 3 nominal hits (p<0.01) against ~5 expected by chance. Is that reframing sufficient, or should the permutation count simply go to ~10,000? + +5. **Sample size and imbalance.** 500 tablets per genre from a corpus that is 92%+ administrative: Lexical is sampled exhaustively (n=69, all of them) while Administrative is sampled at well under 1%. The RULING analysis rests on 10–17 adjacent-chunk pairs in some genres. Are per-genre comparisons meaningful at all under this imbalance? + +6. **Multiple comparisons across the whole repo.** Bonferroni is applied within the ELS scan (495 tests) but *not* across the full set of hypotheses entertained — 24 probes, 5 Zipf fits, 5 compression deltas, 5 parity tests. With enough probes something looks interesting. I have no principled correction for this and would welcome a recommendation. + +7. **The mapping itself.** Does "Sumerian scribes used seals, therefore agent writes should carry signed envelopes" survive as more than an analogy? The benchmark shows a sealed log answers audit queries an anonymous one cannot — but that is close to true by construction, since the anonymous log lacks the queried fields. I think it demonstrates the *cost* side honestly (+59% bytes, +37 tokens/write) while the capability side is definitional, and the README now says so. Push on that. + +## Reproducing it + +Full instructions in the README under **How to Reproduce**. ~8 minutes, no API keys, no GPU: + +```bash +python -m venv .venv && source .venv/bin/activate # Windows: .venv\Scripts\activate +pip install -r requirements.txt +python scripts/phase0_sample.py +python scripts/phase1_templates.py +python scripts/phase1b_probe_validation.py # the probe precision audit +python scripts/phase3_compression.py +git diff --stat outputs/ # expected: empty +``` + +Fastest path to judging this repo: read `outputs/probe_validation.md` (which probe frequencies are citable) and `CORRECTIONS.md` (what was withdrawn and why). Together they are about 10 minutes and cover every claim that has been challenged so far. + +Verified end to end from a clean clone on Python 3.14.2 / numpy 2.5.1 / pandas 3.0.5. `outputs/templates.json` and `outputs/phase3_raw.json` regenerate byte-identical. A non-empty `git diff` is a genuine reproducibility failure and I want the issue. + +Corpus integrity independently confirmed: 91,606 tablets, 6,968,581 cuneiform glyphs, zero duplicate IDs, zero duplicate transliterations, sampling seeded at `random_state=42`. + +## What would change my mind + +- A demonstration that the boundary-permutation null is also wrong. +- A specialist showing the regex probes systematically over- or under-count (I already expect this in places). +- An argument that the compression-Δ result inherits either the §1 or §4 problem. +- Evidence that the agent-design mapping is post-hoc rationalization — that the primitives came from modern practice and were retrofitted onto tablets. I believe templates were mined first and mapped second, and the pipeline ordering supports that, but I cannot prove my own process to you. + +## What I am asking for + +Blunt assessment of whether the claim/evidence table is honest, and whether anything here is worth writing up formally. "This is a nicely packaged blog post, not a paper" is a useful answer and will not offend me. diff --git a/benchmarks/benchmark.py b/benchmarks/benchmark.py index 2649f2d..8e05820 100644 --- a/benchmarks/benchmark.py +++ b/benchmarks/benchmark.py @@ -8,8 +8,15 @@ 4. Capability comparison on 5 audit queries Outputs: - - benchmarks/results.json raw numbers - - benchmarks/RESULTS.md human-readable report + - benchmarks/results.json raw numbers (the only file this script writes) + +RESULTS.md is a hand-authored report that interprets results.json; it is not +regenerated here. Re-running this script will not overwrite it. + +Note: absolute latencies are hardware-dependent and will differ from the +committed results.json. Byte/token overheads and the 5/5-vs-0/5 capability +result are deterministic. Seal IDs are random (secrets.token_hex), so +results.json is not byte-reproducible by design. Token estimate uses 4 bytes/token (conservative tiktoken-like heuristic). Replace with `tiktoken` for exact counts in your stack. @@ -287,16 +294,15 @@ def main() -> None: print(f" Sealed write latency p50 : {w10k['sealed_write_us_median']:>8.1f} us") print(f" Sealed verify all (10k) : {read_results['sealed_verify_all_ms_total']:>8.1f} ms") + # ASCII only — a cp1252 console (default on Windows) cannot encode check/cross glyphs. + mark = lambda ok: "YES" if ok else " NO" print(f"\nCapability comparison:") for q, r in cap_results.items(): if "baseline_can_answer" in r: - ok_b = "✔" if r["baseline_can_answer"] else "✘" - ok_s = "✔" if r["sealed_can_answer"] else "✘" - print(f" {q:42s} baseline={ok_b} sealed={ok_s}") + ok_b, ok_s = r["baseline_can_answer"], r["sealed_can_answer"] else: - ok_b = "✔" if r["baseline_correctly_excludes_revoked"] else "✘" - ok_s = "✔" if r["sealed_correctly_excludes_revoked"] else "✘" - print(f" {q:42s} baseline={ok_b} sealed={ok_s}") + ok_b, ok_s = r["baseline_correctly_excludes_revoked"], r["sealed_correctly_excludes_revoked"] + print(f" {q:42s} baseline={mark(ok_b)} sealed={mark(ok_s)}") if __name__ == "__main__": diff --git a/outputs/FULL_IDEAS.md b/outputs/FULL_IDEAS.md index 0c9cc2e..80503ad 100644 --- a/outputs/FULL_IDEAS.md +++ b/outputs/FULL_IDEAS.md @@ -47,7 +47,7 @@ Multi-resolution context segmentation, derived from physical document structure. |---|---|:---:|:---:| | B1 | Three-tier memory: L1=SURFACE / L2=COLUMN / L3=RULING | V | M | | B2 | Read at smallest enclosing tier (return one row, not the whole frame) | V | S | -| B3 | RULING-bounded chunks as vector-embedding units (parity p=0.002 Royal, p=0.005 Admin) | V | S | +| B3 | RULING-bounded chunks as vector-embedding units ~~(parity p=0.002 Royal, p=0.005 Admin)~~ **evidence WITHDRAWN — see `../CORRECTIONS.md`; idea is now untested** | ~~V~~ S | S | | B4 | Frame nesting (obverse/reverse pairs) → call-stack-style nested contexts | V | S | | B5 | BLANK_SPACE marker as intentional gap (preserve, don't trim — it carries semantic weight in admin) | V | XS | | B6 | rate as data-quality signal per genre (Literary 6.9/tab → most damaged) | V | XS | diff --git a/outputs/compression_findings.md b/outputs/compression_findings.md index f2dc633..c5f5ce8 100644 --- a/outputs/compression_findings.md +++ b/outputs/compression_findings.md @@ -15,7 +15,25 @@ Classic Zipf predicts exponent s ≈ 1.0 with high R². Deviations indicate eith | Royal Inscription | 33,015 | 900 | 1.737 | 0.9433 | | Letter | 18,664 | 780 | 1.607 | 0.9506 | -**Interpretation:** Genres with s substantially > 1 have a *small, high-reuse core vocabulary* — they behave like a DSL with reserved keywords, not a natural language. Administrative and Royal Inscription are the strongest candidates; Literary should be closer to natural-language s ≈ 1.0. +### Length control — the cross-genre comparison does not survive it +Zipf exponents fitted this way are strongly sample-size dependent, and these streams differ by ~60× in length. Re-fitting every genre on equal-length contiguous blocks separates genre from corpus size: + +| Genre | Stream length | s at native length | s at common length (2,508 tokens) | sd | MLE α | +|---|---:|---:|---:|---:|---:| +| Administrative | 32,493 | 1.746 | **1.187** | 0.019 | 1.445 | +| Literary | 154,005 | 1.68 | **1.187** | 0.045 | 1.573 | +| Lexical | 2,508 | 1.114 | **1.114** | 0.0 | 1.59 | +| Royal Inscription | 33,015 | 1.737 | **1.2** | 0.022 | 1.433 | +| Letter | 18,664 | 1.607 | **1.21** | 0.015 | 1.483 | + +At a common length every genre lands in a band of width 0.096, against a native-length spread of 0.632. The apparent genre difference was overwhelmingly a corpus-size effect: the genre with the shortest stream (Lexical) merely had the least opportunity to accumulate a long low-frequency tail. + +**Correction — the 'Zipf-as-DSL detector' claim is withdrawn.** Earlier versions read the native-length spread (Administrative s=1.746 and Royal s=1.737 versus Lexical s=1.114) as evidence that administrative and royal genres behave like domain-specific languages while lexical lists resemble natural language. Under the length control that difference disappears. Two further problems compound it: + +1. **The estimator is unreliable.** `zipf_fit` is OLS on log-log rank-frequency data — a biased power-law estimator, and its R² is not a goodness-of-fit test (Clauset, Shalizi & Newman 2009). The MLE column disagrees with the OLS column in both magnitude and *rank order*, which is exactly the symptom that diagnoses the OLS number as untrustworthy. +2. **The result is sensitive to arbitrary preprocessing.** Dropping hapax legomena (34–59% of types, depending on genre) moves the exponents by up to 0.17 and reverses the direction of some genre comparisons. + +The R² values of 0.92–0.95 across every genre should not be read as support: comparably high R² is routine for lognormal and exponential data under this fitting procedure. Doing this properly would need MLE fitting with a fitted x_min, a Kolmogorov–Smirnov goodness-of-fit statistic, and likelihood-ratio tests against lognormal alternatives — none of which is done here. --- @@ -28,9 +46,26 @@ Classic Zipf predicts exponent s ≈ 1.0 with high R². Deviations indicate eith | Literary | 0.2632 | 0.3291 | **0.0658** | 154,005 | | Lexical | 0.3432 | 0.3899 | **0.0467** | 2,508 | | Royal Inscription | 0.2111 | 0.3102 | **0.0991** | 33,015 | -| Letter | 0.2485 | 0.3234 | **0.0749** | 18,664 | +| Letter | 0.2485 | 0.3234 | **0.0748** | 18,664 | -**Interpretation:** A positive Δ means the actual stream has structural regularities (templates, fixed formulas) beyond what random token order would predict. This is the corpus's 'error-correction overhead' in information-theoretic terms. +**Interpretation:** A positive Δ means the actual stream has structural regularities (templates, fixed formulas) beyond what random token order would predict. + +### Length control — this comparison DOES survive it +The same length confound that invalidates the cross-genre Zipf comparison (§1) was tested here rather than assumed. Blocks are contiguous, since sampling scattered positions would itself destroy the token adjacency zlib exploits and would manufacture a collapse rather than detect one: + +| Genre | Δ at native length | Δ at common length (2,508 tokens) | sd | +|---|---:|---:|---:| +| Administrative | 0.0875 | **0.0699** | 0.0086 | +| Literary | 0.0658 | **0.0643** | 0.0183 | +| Lexical | 0.0467 | **0.0463** | 0.0005 | +| Royal Inscription | 0.0991 | **0.0826** | 0.0111 | +| Letter | 0.0748 | **0.0595** | 0.0035 | + +Ranking at native length: Royal Inscription > Administrative > Letter > Literary > Lexical. + +Ranking at common length: Royal Inscription > Administrative > Literary > Letter > Lexical. + +The ordering is preserved and magnitudes stay the same order of size, so unlike the Zipf comparison this one is not an artifact of differing stream lengths; genres that swap rank do so within overlapping standard deviations. **This is the strongest surviving positive statistical result in this repository** — though it remains a descriptive contrast against random token order, not a test of any specific structural hypothesis. --- @@ -39,73 +74,127 @@ For each skip k, we decimate the genre stream (take every k-th token) and count **Significance criterion:** Bonferroni-corrected p < 0.01/99 ≈ 0.000101 per (genre, skip). This is deliberately strict because with 99 skips × 5 genres = 495 tests, spurious hits are expected. +**Resolution caveat — state this before citing the null.** With 1000 permutations the smallest attainable p-value is 1/(1000+1) ≈ 0.00100, which is about 10× *larger* than the Bonferroni threshold above. This test therefore **cannot** return a Bonferroni-significant result no matter what the data look like; ~9,900 permutations would be needed. The '0 of 495' headline is consequently guaranteed by construction and should not be read as a powered rejection. + +The result that *is* informative is the uncorrected one: across 495 tests at a nominal p < 0.01 you would expect ≈5 hits by chance, and that is roughly what appears. No skip, prime or otherwise, stands out above chance expectation. That is genuine evidence against hidden periodic encoding — it is simply weaker, and differently framed, than a Bonferroni claim. + ### Administrative -- Total skips tested: 99; skips with p < 0.01: 0; Bonferroni-significant: **0** (of which prime-interval: 0). +- Total skips tested: 99; skips with p < 0.01: 1; Bonferroni-significant: **0** (of which prime-interval: 0). *No Bonferroni-significant skips.* + Top-5 by z-score (for reference — not necessarily significant after correction): | Skip | Prime? | Observed | Null mean | z | p | |---:|:---:|---:|---:|---:|---:| -| 2 | | 154 | 142.81 | 2.739 | 0.005 | -| 3 | Y | 110 | 99.55 | 2.388 | 0.013 | -| 7 | Y | 71 | 64.08 | 1.91 | 0.029 | -| 6 | | 79 | 71.69 | 1.788 | 0.04 | -| 5 | Y | 90 | 81.66 | 1.768 | 0.043 | +| 21 | | 147 | 133.824 | 2.739 | 0.00599 | +| 31 | Y | 114 | 105.379 | 1.915 | 0.03497 | +| 14 | | 174 | 164.566 | 1.895 | 0.03996 | +| 44 | | 89 | 82.577 | 1.478 | 0.08691 | +| 28 | | 119 | 112.712 | 1.421 | 0.09291 | ### Literary -- Total skips tested: 99; skips with p < 0.01: 0; Bonferroni-significant: **0** (of which prime-interval: 0). +- Total skips tested: 99; skips with p < 0.01: 1; Bonferroni-significant: **0** (of which prime-interval: 0). *No Bonferroni-significant skips.* -Top-5 by z-score: + +Top-5 by z-score (for reference — not necessarily significant after correction): | Skip | Prime? | Observed | Null mean | z | p | |---:|:---:|---:|---:|---:|---:| -| 100 | | 86 | 75.14 | 2.515 | 0.007 | -| 86 | | 99 | 87.27 | 2.477 | 0.008 | -| 71 | Y | 119 | 105.51 | 2.473 | 0.008 | -| 76 | | 110 | 97.81 | 2.405 | 0.011 | -| 99 | | 89 | 78.08 | 2.401 | 0.011 | +| 98 | | 134 | 122.075 | 2.515 | 0.00799 | +| 69 | | 159 | 149.389 | 1.842 | 0.04396 | +| 57 | | 176 | 166.159 | 1.785 | 0.05395 | +| 19 | Y | 314 | 301.366 | 1.774 | 0.04695 | +| 65 | | 164 | 154.563 | 1.771 | 0.05794 | ### Lexical - Total skips tested: 99; skips with p < 0.01: 0; Bonferroni-significant: **0** (of which prime-interval: 0). *No Bonferroni-significant skips.* + +Top-5 by z-score (for reference — not necessarily significant after correction): + +| Skip | Prime? | Observed | Null mean | z | p | +|---:|:---:|---:|---:|---:|---:| +| 79 | Y | 2 | 0.419 | 2.5 | 0.06294 | +| 9 | | 38 | 31.747 | 2.094 | 0.02897 | +| 71 | Y | 2 | 0.56 | 2.085 | 0.08991 | +| 44 | | 4 | 1.826 | 1.976 | 0.06693 | +| 43 | Y | 4 | 1.927 | 1.765 | 0.0959 | + ### Royal Inscription - Total skips tested: 99; skips with p < 0.01: 0; Bonferroni-significant: **0** (of which prime-interval: 0). +*No Bonferroni-significant skips.* + + +Top-5 by z-score (for reference — not necessarily significant after correction): + +| Skip | Prime? | Observed | Null mean | z | p | +|---:|:---:|---:|---:|---:|---:| +| 76 | | 56 | 48.412 | 2.217 | 0.02398 | +| 63 | | 64 | 57.204 | 1.904 | 0.03996 | +| 48 | | 78 | 71.515 | 1.697 | 0.04795 | +| 36 | | 95 | 88.093 | 1.687 | 0.05295 | +| 20 | | 136 | 128.233 | 1.682 | 0.06094 | + ### Letter -- Total skips tested: 99; skips with p < 0.01: 0; Bonferroni-significant: **0**. -- Highest z observed: 3.146 (corresponds to p ≈ 0.0008, NOT Bonferroni-significant after correction for 99 tests). +- Total skips tested: 99; skips with p < 0.01: 1; Bonferroni-significant: **0** (of which prime-interval: 0). + +*No Bonferroni-significant skips.* + + +Top-5 by z-score (for reference — not necessarily significant after correction): + +| Skip | Prime? | Observed | Null mean | z | p | +|---:|:---:|---:|---:|---:|---:| +| 33 | | 66 | 55.067 | 3.146 | 0.002 | +| 44 | | 51 | 43.761 | 2.296 | 0.01698 | +| 77 | | 31 | 25.672 | 2.006 | 0.03796 | +| 22 | | 83 | 75.469 | 1.889 | 0.04396 | +| 21 | | 84 | 77.856 | 1.476 | 0.07992 | **Honest interpretation of ELS findings:** After Bonferroni correction, no skip in [2, 100] shows a non-random ELS signal for any genre. This is the correct null result to expect absent true encoding. -We reject mystical readings — the corpus's periodicities are adequately explained by (a) recurring template formulas and (b) average tablet length when multiple tablets are concatenated in the stream. +In either case we reject mystical readings — the corpus's periodicities are adequately explained by (a) recurring template formulas and (b) average tablet length when multiple tablets are concatenated in the stream. --- -## 4. Cross-RULING Parity (Shared Trigrams Across Adjacent Chunks) -For tablets with ≥2 RULING-delimited chunks, we measure trigram overlap between adjacent chunks, and compare to a within-tablet shuffled baseline (pool tokens, reshuffle, re-chunk at same lengths). High Δ ⇒ RULINGs separate sections that share structured content (e.g., parallel entries in a list). +## 4. Cross-RULING Parity — NULL RESULT (supersedes an earlier positive claim) +**Question:** do `` marks fall at content boundaries, or could you cut the tablet anywhere and see the same thing? -| Genre | Adjacent-chunk pairs | Obs. shared trigrams | Null mean | Δ | p (approx) | -|---|---:|---:|---:|---:|---:| -| Administrative | 17 | 0.176 | 0.005 | **+0.171** | 0.005 | -| Literary | 108 | 0.731 | 0.016 | **+0.715** | 0.014 | -| Lexical | 13 | 0.538 | 0.032 | **+0.507** | 0.031 | -| Royal Inscription | 10 | 0.500 | 0.003 | **+0.497** | 0.002 | -| Letter | 1 | 0 | 0.025 | -0.025 | n/a (insufficient) | +**Statistic:** mean shared trigrams between adjacent ruling-delimited chunks. The choice of null *is* the entire experiment: + +- **`boundary_permute` (primary).** Keep the real token order and the exact multiset of chunk lengths; permute only *where* the cuts fall. Boundary placement is the only thing that varies, so this isolates the actual question. +- **`token_shuffle` (secondary, shown for contrast).** Pool the tablet's tokens, shuffle, re-cut at the same lengths. This destroys *all* local structure, so it tests "is this text locally coherent at all" — true of any natural language, and uninformative about ``. -**Interpretation:** Positive Δ in Royal Inscription (p=0.002) and Administrative (p=0.005) confirms that `` is a *logical* separator between parallel items (like list entries with repeated framing words), not arbitrary spacing. This is the clearest evidence that `` maps to 'row boundary' in a typed schema. +| Genre | Pairs | Observed | Boundary-permute null | Δ | p (primary) | Token-shuffle null | p (secondary) | +|---|---:|---:|---:|---:|---:|---:|---:| +| Administrative | 17 | 0.176 | 0.343 | -0.167 | **0.78109** | 0.004 | 0.00498 | +| Literary | 108 | 0.731 | 0.889 | -0.158 | **0.90547** | 0.018 | 0.00498 | +| Lexical | 13 | 0.538 | 0.43 | 0.108 | **0.27363** | 0.037 | 0.00498 | +| Royal Inscription | 10 | 0.5 | 0.5 | 0.0 | **1.0** | 0.001 | 0.00498 | +| Letter | 1 | 0.0 | 0.55 | -0.55 | n/a (n=1) | 0.015 | n/a (n=1) | + +p-values require at least 5 adjacent-chunk pairs. Even where reported, n is small (Administrative 17, Royal Inscription 10) — these samples could not support a strong claim in either direction. + +**Result: null.** Under the control that isolates boundary placement, no genre shows adjacent ruling-delimited chunks sharing more trigrams than arbitrarily placed cuts of the same lengths. Observed values sit *at or below* the boundary-permuted null throughout. + +**Correction.** An earlier version of this repository reported this test as significant (Royal p=0.002, Administrative p=0.005) and concluded that `` is a validated logical row separator. That result came from the `token_shuffle` null, which — as the two columns above show — inflates the effect by roughly 100× because it destroys the local coherence that any natural-language text has. The p-value was also computed against the pooled distribution of *individual* null pairs rather than the null distribution *of the mean*. Both are corrected here. **The claim is withdrawn: we have no statistical evidence that `` marks content boundaries.** + +This does not show that `` is *meaningless* — a null result on a sample this small is weak evidence either way, and the marks plainly correspond to physical lines drawn by scribes. It shows only that this test does not support the claim, and that the three-tier memory design it was cited to justify should be read as an untested design proposal. --- ## 5. Summary — Implications for Agent Runtimes - - **Genre DSLs are real.** Zipf exponents > 1 in Administrative and Royal Inscription indicate keyword-reuse typical of a domain-specific language, not prose. Memory segments in agent runtimes should preserve genre/schema tags so downstream agents can exploit this. - **Structural redundancy is measurable.** The positive raw-vs-shuffled Δ gives an empirical 'template coverage' score per genre. Memory compression strategies should target high-template genres with template-aware encoders. - **ELS is a dead end for cuneiform.** As expected. Useful null-result to cite when future ideas try to decode 'hidden patterns'. -- **RULINGs are logical separators where Δ > 0.** Use them as row boundaries, not as mere visual hints. +- **RULINGs: no evidence either way.** The earlier positive claim was an artifact of the wrong null and is withdrawn (§4). Treat the three-tier SURFACE/COLUMN/RULING memory design as an untested proposal, not a corpus finding. + +**On the two surviving positive results.** Both §1 (Zipf) and §2 (compression Δ) are descriptive statistics against a shuffled-token baseline. They establish that these genres are more formulaic than random token order — they do not establish that the corpus is a 'DSL' in any formal sense. That word is an analogy, not a result. diff --git a/outputs/phase3_raw.json b/outputs/phase3_raw.json index b59623f..eb20319 100644 --- a/outputs/phase3_raw.json +++ b/outputs/phase3_raw.json @@ -1,4 +1,83 @@ { + "zipf_length_control": [ + { + "genre": "Administrative", + "stream_length": 32493, + "s_at_native_length": 1.746, + "s_at_common_length": 1.187, + "s_at_common_length_sd": 0.019, + "common_length_tokens": 2508, + "mle_alpha": 1.445 + }, + { + "genre": "Literary", + "stream_length": 154005, + "s_at_native_length": 1.68, + "s_at_common_length": 1.187, + "s_at_common_length_sd": 0.045, + "common_length_tokens": 2508, + "mle_alpha": 1.573 + }, + { + "genre": "Lexical", + "stream_length": 2508, + "s_at_native_length": 1.114, + "s_at_common_length": 1.114, + "s_at_common_length_sd": 0.0, + "common_length_tokens": 2508, + "mle_alpha": 1.59 + }, + { + "genre": "Royal Inscription", + "stream_length": 33015, + "s_at_native_length": 1.737, + "s_at_common_length": 1.2, + "s_at_common_length_sd": 0.022, + "common_length_tokens": 2508, + "mle_alpha": 1.433 + }, + { + "genre": "Letter", + "stream_length": 18664, + "s_at_native_length": 1.607, + "s_at_common_length": 1.21, + "s_at_common_length_sd": 0.015, + "common_length_tokens": 2508, + "mle_alpha": 1.483 + } + ], + "compression_length_control": [ + { + "genre": "Administrative", + "delta_at_common_length": 0.0699, + "delta_at_common_length_sd": 0.0086, + "common_length_tokens": 2508 + }, + { + "genre": "Literary", + "delta_at_common_length": 0.0643, + "delta_at_common_length_sd": 0.0183, + "common_length_tokens": 2508 + }, + { + "genre": "Lexical", + "delta_at_common_length": 0.0463, + "delta_at_common_length_sd": 0.0005, + "common_length_tokens": 2508 + }, + { + "genre": "Royal Inscription", + "delta_at_common_length": 0.0826, + "delta_at_common_length_sd": 0.0111, + "common_length_tokens": 2508 + }, + { + "genre": "Letter", + "delta_at_common_length": 0.0595, + "delta_at_common_length_sd": 0.0035, + "common_length_tokens": 2508 + } + ], "zipf": [ { "genre": "Administrative", @@ -82,7 +161,7 @@ "null_mean": 367.93, "null_std": 6.073, "z": -1.8, - "p_value": 0.978 + "p_value": 0.97802 }, { "skip": 3, @@ -91,7 +170,7 @@ "null_mean": 312.984, "null_std": 5.694, "z": -0.524, - "p_value": 0.73 + "p_value": 0.73027 }, { "skip": 4, @@ -100,7 +179,7 @@ "null_mean": 279.374, "null_std": 5.641, "z": -1.13, - "p_value": 0.893 + "p_value": 0.89311 }, { "skip": 5, @@ -109,7 +188,7 @@ "null_mean": 255.621, "null_std": 5.422, "z": 1.361, - "p_value": 0.104 + "p_value": 0.1049 }, { "skip": 6, @@ -118,7 +197,7 @@ "null_mean": 236.993, "null_std": 5.572, "z": 0.36, - "p_value": 0.405 + "p_value": 0.40559 }, { "skip": 7, @@ -127,7 +206,7 @@ "null_mean": 222.596, "null_std": 5.397, "z": 0.631, - "p_value": 0.318 + "p_value": 0.31868 }, { "skip": 8, @@ -136,7 +215,7 @@ "null_mean": 210.361, "null_std": 5.276, "z": 0.121, - "p_value": 0.483 + "p_value": 0.48352 }, { "skip": 9, @@ -145,7 +224,7 @@ "null_mean": 200.069, "null_std": 5.317, "z": 0.927, - "p_value": 0.198 + "p_value": 0.1988 }, { "skip": 10, @@ -154,7 +233,7 @@ "null_mean": 191.234, "null_std": 4.983, "z": 0.154, - "p_value": 0.464 + "p_value": 0.46454 }, { "skip": 11, @@ -163,7 +242,7 @@ "null_mean": 183.568, "null_std": 5.11, "z": -1.09, - "p_value": 0.882 + "p_value": 0.88212 }, { "skip": 12, @@ -172,7 +251,7 @@ "null_mean": 176.269, "null_std": 4.958, "z": 0.147, - "p_value": 0.477 + "p_value": 0.47752 }, { "skip": 13, @@ -181,7 +260,7 @@ "null_mean": 169.913, "null_std": 4.867, "z": 0.84, - "p_value": 0.234 + "p_value": 0.23477 }, { "skip": 14, @@ -190,7 +269,7 @@ "null_mean": 164.566, "null_std": 4.977, "z": 1.895, - "p_value": 0.039 + "p_value": 0.03996 }, { "skip": 15, @@ -199,7 +278,7 @@ "null_mean": 158.967, "null_std": 4.763, "z": 0.637, - "p_value": 0.294 + "p_value": 0.29471 }, { "skip": 16, @@ -208,7 +287,7 @@ "null_mean": 153.902, "null_std": 4.75, "z": -1.032, - "p_value": 0.876 + "p_value": 0.87612 }, { "skip": 17, @@ -217,7 +296,7 @@ "null_mean": 149.5, "null_std": 4.652, "z": 0.752, - "p_value": 0.278 + "p_value": 0.27872 }, { "skip": 18, @@ -226,7 +305,7 @@ "null_mean": 145.243, "null_std": 4.645, "z": -0.698, - "p_value": 0.78 + "p_value": 0.78022 }, { "skip": 19, @@ -235,7 +314,7 @@ "null_mean": 141.254, "null_std": 4.515, "z": 0.83, - "p_value": 0.235 + "p_value": 0.23576 }, { "skip": 20, @@ -244,7 +323,7 @@ "null_mean": 137.17, "null_std": 4.768, "z": -0.875, - "p_value": 0.827 + "p_value": 0.82717 }, { "skip": 21, @@ -253,7 +332,7 @@ "null_mean": 133.824, "null_std": 4.811, "z": 2.739, - "p_value": 0.005 + "p_value": 0.00599 }, { "skip": 22, @@ -262,7 +341,7 @@ "null_mean": 130.345, "null_std": 4.709, "z": 0.564, - "p_value": 0.319 + "p_value": 0.31968 }, { "skip": 23, @@ -271,7 +350,7 @@ "null_mean": 127.032, "null_std": 4.633, "z": -0.007, - "p_value": 0.546 + "p_value": 0.54645 }, { "skip": 24, @@ -280,7 +359,7 @@ "null_mean": 123.863, "null_std": 4.606, "z": -0.839, - "p_value": 0.819 + "p_value": 0.81918 }, { "skip": 25, @@ -289,7 +368,7 @@ "null_mean": 120.81, "null_std": 4.478, "z": -1.297, - "p_value": 0.912 + "p_value": 0.91209 }, { "skip": 26, @@ -298,7 +377,7 @@ "null_mean": 118.163, "null_std": 4.56, "z": -1.571, - "p_value": 0.955 + "p_value": 0.95504 }, { "skip": 27, @@ -307,7 +386,7 @@ "null_mean": 115.296, "null_std": 4.45, "z": -0.516, - "p_value": 0.742 + "p_value": 0.74226 }, { "skip": 28, @@ -316,7 +395,7 @@ "null_mean": 112.712, "null_std": 4.425, "z": 1.421, - "p_value": 0.092 + "p_value": 0.09291 }, { "skip": 29, @@ -325,7 +404,7 @@ "null_mean": 110.417, "null_std": 4.523, "z": -0.092, - "p_value": 0.582 + "p_value": 0.58242 }, { "skip": 30, @@ -334,7 +413,7 @@ "null_mean": 107.75, "null_std": 4.621, "z": -0.811, - "p_value": 0.819 + "p_value": 0.81918 }, { "skip": 31, @@ -343,7 +422,7 @@ "null_mean": 105.379, "null_std": 4.503, "z": 1.915, - "p_value": 0.034 + "p_value": 0.03497 }, { "skip": 32, @@ -352,7 +431,7 @@ "null_mean": 103.384, "null_std": 4.42, "z": -0.539, - "p_value": 0.749 + "p_value": 0.74925 }, { "skip": 33, @@ -361,7 +440,7 @@ "null_mean": 101.446, "null_std": 4.48, "z": 0.124, - "p_value": 0.509 + "p_value": 0.50949 }, { "skip": 34, @@ -370,7 +449,7 @@ "null_mean": 99.234, "null_std": 4.367, "z": -0.283, - "p_value": 0.65 + "p_value": 0.65035 }, { "skip": 35, @@ -379,7 +458,7 @@ "null_mean": 97.008, "null_std": 4.607, "z": 1.084, - "p_value": 0.171 + "p_value": 0.17183 }, { "skip": 36, @@ -388,7 +467,7 @@ "null_mean": 95.592, "null_std": 4.336, "z": -1.982, - "p_value": 0.978 + "p_value": 0.97802 }, { "skip": 37, @@ -397,7 +476,7 @@ "null_mean": 93.606, "null_std": 4.183, "z": 0.572, - "p_value": 0.315 + "p_value": 0.31568 }, { "skip": 38, @@ -406,7 +485,7 @@ "null_mean": 91.855, "null_std": 4.267, "z": 1.206, - "p_value": 0.13 + "p_value": 0.13087 }, { "skip": 39, @@ -415,7 +494,7 @@ "null_mean": 90.103, "null_std": 4.347, "z": -0.714, - "p_value": 0.799 + "p_value": 0.7992 }, { "skip": 40, @@ -424,7 +503,7 @@ "null_mean": 88.466, "null_std": 4.166, "z": -0.352, - "p_value": 0.696 + "p_value": 0.6963 }, { "skip": 41, @@ -433,7 +512,7 @@ "null_mean": 87.077, "null_std": 4.327, "z": -0.711, - "p_value": 0.798 + "p_value": 0.7982 }, { "skip": 42, @@ -442,7 +521,7 @@ "null_mean": 85.447, "null_std": 4.307, "z": -0.568, - "p_value": 0.761 + "p_value": 0.76124 }, { "skip": 43, @@ -451,7 +530,7 @@ "null_mean": 83.876, "null_std": 4.331, "z": 1.183, - "p_value": 0.146 + "p_value": 0.14685 }, { "skip": 44, @@ -460,7 +539,7 @@ "null_mean": 82.577, "null_std": 4.347, "z": 1.478, - "p_value": 0.086 + "p_value": 0.08691 }, { "skip": 45, @@ -469,7 +548,7 @@ "null_mean": 80.902, "null_std": 4.176, "z": -0.934, - "p_value": 0.852 + "p_value": 0.85215 }, { "skip": 46, @@ -478,7 +557,7 @@ "null_mean": 79.937, "null_std": 4.158, "z": -0.947, - "p_value": 0.857 + "p_value": 0.85714 }, { "skip": 47, @@ -487,7 +566,7 @@ "null_mean": 78.356, "null_std": 4.142, "z": -0.569, - "p_value": 0.762 + "p_value": 0.76224 }, { "skip": 48, @@ -496,7 +575,7 @@ "null_mean": 77.269, "null_std": 4.178, "z": -1.022, - "p_value": 0.863 + "p_value": 0.86314 }, { "skip": 49, @@ -505,7 +584,7 @@ "null_mean": 75.69, "null_std": 3.969, "z": 0.834, - "p_value": 0.245 + "p_value": 0.24575 }, { "skip": 50, @@ -514,7 +593,7 @@ "null_mean": 74.637, "null_std": 3.91, "z": -1.186, - "p_value": 0.908 + "p_value": 0.90809 }, { "skip": 51, @@ -523,7 +602,7 @@ "null_mean": 73.285, "null_std": 4.007, "z": -2.317, - "p_value": 0.991 + "p_value": 0.99101 }, { "skip": 52, @@ -532,7 +611,7 @@ "null_mean": 72.506, "null_std": 4.149, "z": 0.601, - "p_value": 0.316 + "p_value": 0.31668 }, { "skip": 53, @@ -541,7 +620,7 @@ "null_mean": 71.274, "null_std": 3.906, "z": -0.07, - "p_value": 0.574 + "p_value": 0.57443 }, { "skip": 54, @@ -550,7 +629,7 @@ "null_mean": 70.068, "null_std": 3.885, "z": -0.79, - "p_value": 0.827 + "p_value": 0.82717 }, { "skip": 55, @@ -559,7 +638,7 @@ "null_mean": 68.914, "null_std": 3.827, "z": 0.022, - "p_value": 0.543 + "p_value": 0.54346 }, { "skip": 56, @@ -568,7 +647,7 @@ "null_mean": 68.208, "null_std": 3.804, "z": -1.369, - "p_value": 0.938 + "p_value": 0.93806 }, { "skip": 57, @@ -577,7 +656,7 @@ "null_mean": 67.065, "null_std": 3.886, "z": -0.789, - "p_value": 0.824 + "p_value": 0.82418 }, { "skip": 58, @@ -586,7 +665,7 @@ "null_mean": 66.253, "null_std": 3.824, "z": 0.457, - "p_value": 0.351 + "p_value": 0.35165 }, { "skip": 59, @@ -595,7 +674,7 @@ "null_mean": 65.261, "null_std": 3.621, "z": -1.729, - "p_value": 0.971 + "p_value": 0.97103 }, { "skip": 60, @@ -604,7 +683,7 @@ "null_mean": 64.151, "null_std": 3.807, "z": -1.353, - "p_value": 0.931 + "p_value": 0.93107 }, { "skip": 61, @@ -613,7 +692,7 @@ "null_mean": 63.323, "null_std": 3.593, "z": -1.76, - "p_value": 0.974 + "p_value": 0.97403 }, { "skip": 62, @@ -622,7 +701,7 @@ "null_mean": 62.605, "null_std": 3.736, "z": -0.43, - "p_value": 0.702 + "p_value": 0.7023 }, { "skip": 63, @@ -631,7 +710,7 @@ "null_mean": 61.414, "null_std": 3.762, "z": -1.705, - "p_value": 0.969 + "p_value": 0.96903 }, { "skip": 64, @@ -640,7 +719,7 @@ "null_mean": 60.696, "null_std": 3.613, "z": -1.023, - "p_value": 0.89 + "p_value": 0.89011 }, { "skip": 65, @@ -649,7 +728,7 @@ "null_mean": 59.942, "null_std": 3.67, "z": 0.288, - "p_value": 0.426 + "p_value": 0.42657 }, { "skip": 66, @@ -658,7 +737,7 @@ "null_mean": 58.921, "null_std": 3.654, "z": 0.843, - "p_value": 0.24 + "p_value": 0.24076 }, { "skip": 67, @@ -667,7 +746,7 @@ "null_mean": 58.248, "null_std": 3.671, "z": -0.885, - "p_value": 0.854 + "p_value": 0.85415 }, { "skip": 68, @@ -676,7 +755,7 @@ "null_mean": 57.529, "null_std": 3.809, "z": 0.124, - "p_value": 0.497 + "p_value": 0.4975 }, { "skip": 69, @@ -685,7 +764,7 @@ "null_mean": 56.71, "null_std": 3.519, "z": -0.77, - "p_value": 0.817 + "p_value": 0.81718 }, { "skip": 70, @@ -694,7 +773,7 @@ "null_mean": 55.833, "null_std": 3.525, "z": -1.087, - "p_value": 0.897 + "p_value": 0.8971 }, { "skip": 71, @@ -703,7 +782,7 @@ "null_mean": 55.246, "null_std": 3.635, "z": 0.482, - "p_value": 0.373 + "p_value": 0.37363 }, { "skip": 72, @@ -712,7 +791,7 @@ "null_mean": 54.68, "null_std": 3.64, "z": -0.462, - "p_value": 0.719 + "p_value": 0.71928 }, { "skip": 73, @@ -721,7 +800,7 @@ "null_mean": 53.807, "null_std": 3.452, "z": -0.523, - "p_value": 0.741 + "p_value": 0.74126 }, { "skip": 74, @@ -730,7 +809,7 @@ "null_mean": 53.033, "null_std": 3.61, "z": 0.545, - "p_value": 0.341 + "p_value": 0.34166 }, { "skip": 75, @@ -739,7 +818,7 @@ "null_mean": 52.407, "null_std": 3.618, "z": 0.717, - "p_value": 0.27 + "p_value": 0.27073 }, { "skip": 76, @@ -748,7 +827,7 @@ "null_mean": 51.862, "null_std": 3.414, "z": -1.131, - "p_value": 0.898 + "p_value": 0.8981 }, { "skip": 77, @@ -757,7 +836,7 @@ "null_mean": 51.129, "null_std": 3.586, "z": 0.801, - "p_value": 0.24 + "p_value": 0.24076 }, { "skip": 78, @@ -766,7 +845,7 @@ "null_mean": 50.609, "null_std": 3.474, "z": -0.751, - "p_value": 0.819 + "p_value": 0.81918 }, { "skip": 79, @@ -775,7 +854,7 @@ "null_mean": 50.078, "null_std": 3.383, "z": -0.319, - "p_value": 0.663 + "p_value": 0.66334 }, { "skip": 80, @@ -784,7 +863,7 @@ "null_mean": 49.422, "null_std": 3.317, "z": -1.032, - "p_value": 0.884 + "p_value": 0.88412 }, { "skip": 81, @@ -793,7 +872,7 @@ "null_mean": 48.801, "null_std": 3.426, "z": -0.818, - "p_value": 0.836 + "p_value": 0.83616 }, { "skip": 82, @@ -802,7 +881,7 @@ "null_mean": 48.098, "null_std": 3.458, "z": -1.764, - "p_value": 0.971 + "p_value": 0.97103 }, { "skip": 83, @@ -811,7 +890,7 @@ "null_mean": 47.732, "null_std": 3.423, "z": 0.37, - "p_value": 0.401 + "p_value": 0.4016 }, { "skip": 84, @@ -820,7 +899,7 @@ "null_mean": 46.984, "null_std": 3.436, "z": -0.868, - "p_value": 0.834 + "p_value": 0.83417 }, { "skip": 85, @@ -829,7 +908,7 @@ "null_mean": 46.446, "null_std": 3.462, "z": -0.707, - "p_value": 0.818 + "p_value": 0.81818 }, { "skip": 86, @@ -838,7 +917,7 @@ "null_mean": 45.879, "null_std": 3.249, "z": -1.502, - "p_value": 0.952 + "p_value": 0.95205 }, { "skip": 87, @@ -847,7 +926,7 @@ "null_mean": 45.394, "null_std": 3.326, "z": -1.922, - "p_value": 0.984 + "p_value": 0.98402 }, { "skip": 88, @@ -856,7 +935,7 @@ "null_mean": 44.868, "null_std": 3.389, "z": 0.629, - "p_value": 0.317 + "p_value": 0.31768 }, { "skip": 89, @@ -865,7 +944,7 @@ "null_mean": 44.263, "null_std": 3.322, "z": 0.523, - "p_value": 0.356 + "p_value": 0.35664 }, { "skip": 90, @@ -883,7 +962,7 @@ "null_mean": 43.481, "null_std": 3.247, "z": -1.072, - "p_value": 0.892 + "p_value": 0.89211 }, { "skip": 92, @@ -892,7 +971,7 @@ "null_mean": 43.104, "null_std": 3.415, "z": 0.262, - "p_value": 0.445 + "p_value": 0.44555 }, { "skip": 93, @@ -901,7 +980,7 @@ "null_mean": 42.463, "null_std": 3.23, "z": 0.785, - "p_value": 0.264 + "p_value": 0.26474 }, { "skip": 94, @@ -910,7 +989,7 @@ "null_mean": 41.996, "null_std": 3.349, "z": 0.598, - "p_value": 0.344 + "p_value": 0.34466 }, { "skip": 95, @@ -919,7 +998,7 @@ "null_mean": 41.59, "null_std": 3.233, "z": -0.492, - "p_value": 0.759 + "p_value": 0.75924 }, { "skip": 96, @@ -928,7 +1007,7 @@ "null_mean": 40.898, "null_std": 3.184, "z": 0.346, - "p_value": 0.422 + "p_value": 0.42258 }, { "skip": 97, @@ -937,7 +1016,7 @@ "null_mean": 40.528, "null_std": 3.142, "z": -0.486, - "p_value": 0.723 + "p_value": 0.72328 }, { "skip": 98, @@ -946,7 +1025,7 @@ "null_mean": 40.109, "null_std": 3.242, "z": -1.576, - "p_value": 0.958 + "p_value": 0.95804 }, { "skip": 99, @@ -955,7 +1034,7 @@ "null_mean": 39.671, "null_std": 3.312, "z": 0.099, - "p_value": 0.515 + "p_value": 0.51548 }, { "skip": 100, @@ -964,7 +1043,7 @@ "null_mean": 39.085, "null_std": 3.074, "z": -0.353, - "p_value": 0.712 + "p_value": 0.71229 } ], "Literary": [ @@ -975,7 +1054,7 @@ "null_mean": 783.14, "null_std": 8.421, "z": -1.56, - "p_value": 0.96 + "p_value": 0.96004 }, { "skip": 3, @@ -984,7 +1063,7 @@ "null_mean": 674.812, "null_std": 8.299, "z": -0.58, - "p_value": 0.735 + "p_value": 0.73526 }, { "skip": 4, @@ -993,7 +1072,7 @@ "null_mean": 604.435, "null_std": 8.665, "z": -1.897, - "p_value": 0.977 + "p_value": 0.97702 }, { "skip": 5, @@ -1002,7 +1081,7 @@ "null_mean": 553.99, "null_std": 8.486, "z": -0.824, - "p_value": 0.804 + "p_value": 0.8042 }, { "skip": 6, @@ -1011,7 +1090,7 @@ "null_mean": 513.631, "null_std": 8.468, "z": -0.311, - "p_value": 0.648 + "p_value": 0.64835 }, { "skip": 7, @@ -1020,7 +1099,7 @@ "null_mean": 481.923, "null_std": 8.19, "z": -2.188, - "p_value": 0.99 + "p_value": 0.99001 }, { "skip": 8, @@ -1029,7 +1108,7 @@ "null_mean": 455.04, "null_std": 7.979, "z": 0.12, - "p_value": 0.489 + "p_value": 0.48951 }, { "skip": 9, @@ -1038,7 +1117,7 @@ "null_mean": 431.913, "null_std": 8.064, "z": -0.485, - "p_value": 0.715 + "p_value": 0.71528 }, { "skip": 10, @@ -1047,7 +1126,7 @@ "null_mean": 411.681, "null_std": 7.743, "z": 0.558, - "p_value": 0.293 + "p_value": 0.29371 }, { "skip": 11, @@ -1056,7 +1135,7 @@ "null_mean": 394.485, "null_std": 7.613, "z": -2.559, - "p_value": 0.992 + "p_value": 0.99201 }, { "skip": 12, @@ -1065,7 +1144,7 @@ "null_mean": 378.486, "null_std": 7.637, "z": -2.159, - "p_value": 0.987 + "p_value": 0.98701 }, { "skip": 13, @@ -1074,7 +1153,7 @@ "null_mean": 364.503, "null_std": 7.61, "z": 0.722, - "p_value": 0.254 + "p_value": 0.25475 }, { "skip": 14, @@ -1083,7 +1162,7 @@ "null_mean": 350.936, "null_std": 7.399, "z": -0.667, - "p_value": 0.759 + "p_value": 0.75924 }, { "skip": 15, @@ -1092,7 +1171,7 @@ "null_mean": 339.797, "null_std": 7.271, "z": 0.028, - "p_value": 0.503 + "p_value": 0.5035 }, { "skip": 16, @@ -1101,7 +1180,7 @@ "null_mean": 329.135, "null_std": 7.176, "z": 0.121, - "p_value": 0.492 + "p_value": 0.49251 }, { "skip": 17, @@ -1110,7 +1189,7 @@ "null_mean": 318.729, "null_std": 7.282, "z": 0.037, - "p_value": 0.523 + "p_value": 0.52348 }, { "skip": 18, @@ -1119,7 +1198,7 @@ "null_mean": 310.187, "null_std": 7.228, "z": -0.994, - "p_value": 0.85 + "p_value": 0.85015 }, { "skip": 19, @@ -1128,7 +1207,7 @@ "null_mean": 301.366, "null_std": 7.12, "z": 1.774, - "p_value": 0.046 + "p_value": 0.04695 }, { "skip": 20, @@ -1137,7 +1216,7 @@ "null_mean": 293.544, "null_std": 7.198, "z": 0.897, - "p_value": 0.196 + "p_value": 0.1968 }, { "skip": 21, @@ -1146,7 +1225,7 @@ "null_mean": 285.849, "null_std": 7.103, "z": 0.584, - "p_value": 0.306 + "p_value": 0.30669 }, { "skip": 22, @@ -1155,7 +1234,7 @@ "null_mean": 279.181, "null_std": 7.163, "z": -0.584, - "p_value": 0.748 + "p_value": 0.74825 }, { "skip": 23, @@ -1164,7 +1243,7 @@ "null_mean": 272.734, "null_std": 7.2, "z": -1.352, - "p_value": 0.918 + "p_value": 0.91808 }, { "skip": 24, @@ -1173,7 +1252,7 @@ "null_mean": 266.496, "null_std": 6.98, "z": -0.214, - "p_value": 0.603 + "p_value": 0.6034 }, { "skip": 25, @@ -1182,7 +1261,7 @@ "null_mean": 260.59, "null_std": 6.696, "z": -0.536, - "p_value": 0.73 + "p_value": 0.73027 }, { "skip": 26, @@ -1191,7 +1270,7 @@ "null_mean": 255.549, "null_std": 6.698, "z": 1.56, - "p_value": 0.064 + "p_value": 0.06494 }, { "skip": 27, @@ -1200,7 +1279,7 @@ "null_mean": 250.176, "null_std": 6.615, "z": -1.841, - "p_value": 0.974 + "p_value": 0.97403 }, { "skip": 28, @@ -1209,7 +1288,7 @@ "null_mean": 245.263, "null_std": 6.564, "z": -0.649, - "p_value": 0.761 + "p_value": 0.76124 }, { "skip": 29, @@ -1218,7 +1297,7 @@ "null_mean": 240.819, "null_std": 6.526, "z": -0.432, - "p_value": 0.699 + "p_value": 0.6993 }, { "skip": 30, @@ -1227,7 +1306,7 @@ "null_mean": 236.312, "null_std": 6.597, "z": 0.559, - "p_value": 0.314 + "p_value": 0.31469 }, { "skip": 31, @@ -1236,7 +1315,7 @@ "null_mean": 232.446, "null_std": 6.472, "z": -2.078, - "p_value": 0.989 + "p_value": 0.98901 }, { "skip": 32, @@ -1245,7 +1324,7 @@ "null_mean": 228.214, "null_std": 6.462, "z": -0.962, - "p_value": 0.842 + "p_value": 0.84216 }, { "skip": 33, @@ -1254,7 +1333,7 @@ "null_mean": 224.627, "null_std": 6.433, "z": -1.341, - "p_value": 0.919 + "p_value": 0.91908 }, { "skip": 34, @@ -1263,7 +1342,7 @@ "null_mean": 220.948, "null_std": 6.395, "z": 1.259, - "p_value": 0.121 + "p_value": 0.12188 }, { "skip": 35, @@ -1272,7 +1351,7 @@ "null_mean": 217.269, "null_std": 6.02, "z": -0.211, - "p_value": 0.628 + "p_value": 0.62837 }, { "skip": 36, @@ -1281,7 +1360,7 @@ "null_mean": 214.113, "null_std": 6.61, "z": -0.773, - "p_value": 0.794 + "p_value": 0.79421 }, { "skip": 37, @@ -1290,7 +1369,7 @@ "null_mean": 210.54, "null_std": 6.282, "z": 1.347, - "p_value": 0.102 + "p_value": 0.1029 }, { "skip": 38, @@ -1299,7 +1378,7 @@ "null_mean": 207.524, "null_std": 6.081, "z": 0.078, - "p_value": 0.504 + "p_value": 0.5045 }, { "skip": 39, @@ -1308,7 +1387,7 @@ "null_mean": 204.631, "null_std": 6.26, "z": 0.858, - "p_value": 0.201 + "p_value": 0.2018 }, { "skip": 40, @@ -1317,7 +1396,7 @@ "null_mean": 201.915, "null_std": 6.147, "z": -0.474, - "p_value": 0.704 + "p_value": 0.7043 }, { "skip": 41, @@ -1326,7 +1405,7 @@ "null_mean": 199.34, "null_std": 5.935, "z": 0.617, - "p_value": 0.29 + "p_value": 0.29071 }, { "skip": 42, @@ -1335,7 +1414,7 @@ "null_mean": 196.488, "null_std": 5.988, "z": 0.253, - "p_value": 0.436 + "p_value": 0.43656 }, { "skip": 43, @@ -1344,7 +1423,7 @@ "null_mean": 194.035, "null_std": 6.229, "z": -0.006, - "p_value": 0.524 + "p_value": 0.52448 }, { "skip": 44, @@ -1353,7 +1432,7 @@ "null_mean": 191.429, "null_std": 5.656, "z": -0.076, - "p_value": 0.571 + "p_value": 0.57143 }, { "skip": 45, @@ -1362,7 +1441,7 @@ "null_mean": 189.238, "null_std": 5.756, "z": -1.605, - "p_value": 0.961 + "p_value": 0.96104 }, { "skip": 46, @@ -1371,7 +1450,7 @@ "null_mean": 186.971, "null_std": 5.952, "z": 0.341, - "p_value": 0.377 + "p_value": 0.37762 }, { "skip": 47, @@ -1380,7 +1459,7 @@ "null_mean": 184.956, "null_std": 5.805, "z": -0.509, - "p_value": 0.715 + "p_value": 0.71528 }, { "skip": 48, @@ -1389,7 +1468,7 @@ "null_mean": 182.572, "null_std": 5.812, "z": 0.934, - "p_value": 0.207 + "p_value": 0.20779 }, { "skip": 49, @@ -1398,7 +1477,7 @@ "null_mean": 180.87, "null_std": 5.875, "z": 0.533, - "p_value": 0.329 + "p_value": 0.32967 }, { "skip": 50, @@ -1407,7 +1486,7 @@ "null_mean": 178.535, "null_std": 5.759, "z": 0.775, - "p_value": 0.24 + "p_value": 0.24076 }, { "skip": 51, @@ -1416,7 +1495,7 @@ "null_mean": 176.751, "null_std": 5.364, "z": 1.165, - "p_value": 0.147 + "p_value": 0.14785 }, { "skip": 52, @@ -1425,7 +1504,7 @@ "null_mean": 174.827, "null_std": 5.756, "z": 0.899, - "p_value": 0.194 + "p_value": 0.19481 }, { "skip": 53, @@ -1434,7 +1513,7 @@ "null_mean": 173.053, "null_std": 5.787, "z": -0.7, - "p_value": 0.791 + "p_value": 0.79121 }, { "skip": 54, @@ -1443,7 +1522,7 @@ "null_mean": 171.243, "null_std": 5.585, "z": -1.118, - "p_value": 0.885 + "p_value": 0.88511 }, { "skip": 55, @@ -1452,7 +1531,7 @@ "null_mean": 169.49, "null_std": 5.367, "z": -0.65, - "p_value": 0.773 + "p_value": 0.77323 }, { "skip": 56, @@ -1461,7 +1540,7 @@ "null_mean": 167.754, "null_std": 5.484, "z": -0.867, - "p_value": 0.827 + "p_value": 0.82717 }, { "skip": 57, @@ -1470,7 +1549,7 @@ "null_mean": 166.159, "null_std": 5.513, "z": 1.785, - "p_value": 0.053 + "p_value": 0.05395 }, { "skip": 58, @@ -1479,7 +1558,7 @@ "null_mean": 164.408, "null_std": 5.269, "z": 0.302, - "p_value": 0.414 + "p_value": 0.41459 }, { "skip": 59, @@ -1488,7 +1567,7 @@ "null_mean": 162.78, "null_std": 5.465, "z": -0.326, - "p_value": 0.647 + "p_value": 0.64735 }, { "skip": 60, @@ -1497,7 +1576,7 @@ "null_mean": 161.44, "null_std": 5.354, "z": -0.082, - "p_value": 0.555 + "p_value": 0.55544 }, { "skip": 61, @@ -1506,7 +1585,7 @@ "null_mean": 160.476, "null_std": 5.473, "z": 0.096, - "p_value": 0.498 + "p_value": 0.4985 }, { "skip": 62, @@ -1515,7 +1594,7 @@ "null_mean": 158.427, "null_std": 5.437, "z": -1.182, - "p_value": 0.904 + "p_value": 0.9041 }, { "skip": 63, @@ -1524,7 +1603,7 @@ "null_mean": 157.325, "null_std": 5.614, "z": 0.655, - "p_value": 0.295 + "p_value": 0.2957 }, { "skip": 64, @@ -1533,7 +1612,7 @@ "null_mean": 155.912, "null_std": 5.161, "z": -0.758, - "p_value": 0.801 + "p_value": 0.8012 }, { "skip": 65, @@ -1542,7 +1621,7 @@ "null_mean": 154.563, "null_std": 5.328, "z": 1.771, - "p_value": 0.057 + "p_value": 0.05794 }, { "skip": 66, @@ -1551,7 +1630,7 @@ "null_mean": 153.108, "null_std": 5.324, "z": -0.584, - "p_value": 0.751 + "p_value": 0.75125 }, { "skip": 67, @@ -1560,7 +1639,7 @@ "null_mean": 151.901, "null_std": 5.194, "z": -1.906, - "p_value": 0.976 + "p_value": 0.97602 }, { "skip": 68, @@ -1569,7 +1648,7 @@ "null_mean": 150.762, "null_std": 5.228, "z": 1.193, - "p_value": 0.126 + "p_value": 0.12687 }, { "skip": 69, @@ -1578,7 +1657,7 @@ "null_mean": 149.389, "null_std": 5.219, "z": 1.842, - "p_value": 0.043 + "p_value": 0.04396 }, { "skip": 70, @@ -1587,7 +1666,7 @@ "null_mean": 148.494, "null_std": 5.241, "z": -2.193, - "p_value": 0.992 + "p_value": 0.99201 }, { "skip": 71, @@ -1596,7 +1675,7 @@ "null_mean": 147.316, "null_std": 5.276, "z": -0.06, - "p_value": 0.559 + "p_value": 0.55944 }, { "skip": 72, @@ -1605,7 +1684,7 @@ "null_mean": 145.784, "null_std": 5.163, "z": -0.152, - "p_value": 0.594 + "p_value": 0.59441 }, { "skip": 73, @@ -1614,7 +1693,7 @@ "null_mean": 144.659, "null_std": 5.057, "z": 0.463, - "p_value": 0.366 + "p_value": 0.36663 }, { "skip": 74, @@ -1623,7 +1702,7 @@ "null_mean": 143.613, "null_std": 5.074, "z": 0.273, - "p_value": 0.424 + "p_value": 0.42458 }, { "skip": 75, @@ -1632,7 +1711,7 @@ "null_mean": 142.696, "null_std": 5.061, "z": 1.048, - "p_value": 0.164 + "p_value": 0.16484 }, { "skip": 76, @@ -1641,7 +1720,7 @@ "null_mean": 141.498, "null_std": 5.205, "z": -0.864, - "p_value": 0.815 + "p_value": 0.81518 }, { "skip": 77, @@ -1650,7 +1729,7 @@ "null_mean": 140.266, "null_std": 5.071, "z": -1.236, - "p_value": 0.901 + "p_value": 0.9011 }, { "skip": 78, @@ -1659,7 +1738,7 @@ "null_mean": 139.408, "null_std": 5.093, "z": -0.473, - "p_value": 0.72 + "p_value": 0.72028 }, { "skip": 79, @@ -1668,7 +1747,7 @@ "null_mean": 138.744, "null_std": 5.113, "z": 0.637, - "p_value": 0.298 + "p_value": 0.2987 }, { "skip": 80, @@ -1677,7 +1756,7 @@ "null_mean": 137.683, "null_std": 4.899, "z": -1.16, - "p_value": 0.901 + "p_value": 0.9011 }, { "skip": 81, @@ -1686,7 +1765,7 @@ "null_mean": 136.685, "null_std": 4.989, "z": 0.063, - "p_value": 0.494 + "p_value": 0.49451 }, { "skip": 82, @@ -1695,7 +1774,7 @@ "null_mean": 135.61, "null_std": 4.998, "z": 0.678, - "p_value": 0.264 + "p_value": 0.26474 }, { "skip": 83, @@ -1704,7 +1783,7 @@ "null_mean": 134.69, "null_std": 4.82, "z": -0.351, - "p_value": 0.663 + "p_value": 0.66334 }, { "skip": 84, @@ -1713,7 +1792,7 @@ "null_mean": 133.851, "null_std": 4.944, "z": -0.779, - "p_value": 0.803 + "p_value": 0.8032 }, { "skip": 85, @@ -1722,7 +1801,7 @@ "null_mean": 132.77, "null_std": 5.073, "z": 0.045, - "p_value": 0.507 + "p_value": 0.50749 }, { "skip": 86, @@ -1731,7 +1810,7 @@ "null_mean": 131.872, "null_std": 4.819, "z": 0.649, - "p_value": 0.298 + "p_value": 0.2987 }, { "skip": 87, @@ -1740,7 +1819,7 @@ "null_mean": 131.284, "null_std": 5.008, "z": -1.654, - "p_value": 0.963 + "p_value": 0.96304 }, { "skip": 88, @@ -1749,7 +1828,7 @@ "null_mean": 130.09, "null_std": 4.94, "z": 0.387, - "p_value": 0.383 + "p_value": 0.38362 }, { "skip": 89, @@ -1758,7 +1837,7 @@ "null_mean": 129.382, "null_std": 4.968, "z": -0.681, - "p_value": 0.779 + "p_value": 0.77922 }, { "skip": 90, @@ -1767,7 +1846,7 @@ "null_mean": 128.588, "null_std": 4.579, "z": -0.347, - "p_value": 0.668 + "p_value": 0.66833 }, { "skip": 91, @@ -1776,7 +1855,7 @@ "null_mean": 127.806, "null_std": 4.76, "z": -0.169, - "p_value": 0.599 + "p_value": 0.5994 }, { "skip": 92, @@ -1785,7 +1864,7 @@ "null_mean": 126.952, "null_std": 4.953, "z": 0.615, - "p_value": 0.294 + "p_value": 0.29471 }, { "skip": 93, @@ -1794,7 +1873,7 @@ "null_mean": 126.321, "null_std": 4.675, "z": -0.924, - "p_value": 0.838 + "p_value": 0.83816 }, { "skip": 94, @@ -1803,7 +1882,7 @@ "null_mean": 125.508, "null_std": 4.741, "z": 1.58, - "p_value": 0.063 + "p_value": 0.06394 }, { "skip": 95, @@ -1812,7 +1891,7 @@ "null_mean": 124.464, "null_std": 4.877, "z": 0.11, - "p_value": 0.49 + "p_value": 0.49051 }, { "skip": 96, @@ -1821,7 +1900,7 @@ "null_mean": 123.776, "null_std": 4.827, "z": -0.575, - "p_value": 0.753 + "p_value": 0.75325 }, { "skip": 97, @@ -1830,7 +1909,7 @@ "null_mean": 122.881, "null_std": 4.806, "z": 1.689, - "p_value": 0.056 + "p_value": 0.05694 }, { "skip": 98, @@ -1839,7 +1918,7 @@ "null_mean": 122.075, "null_std": 4.741, "z": 2.515, - "p_value": 0.007 + "p_value": 0.00799 }, { "skip": 99, @@ -1848,7 +1927,7 @@ "null_mean": 121.704, "null_std": 4.761, "z": -0.358, - "p_value": 0.657 + "p_value": 0.65734 }, { "skip": 100, @@ -1857,7 +1936,7 @@ "null_mean": 120.804, "null_std": 4.579, "z": 0.043, - "p_value": 0.533 + "p_value": 0.53347 } ], "Lexical": [ @@ -1868,7 +1947,7 @@ "null_mean": 120.103, "null_std": 4.139, "z": -0.991, - "p_value": 0.859 + "p_value": 0.85914 }, { "skip": 3, @@ -1877,7 +1956,7 @@ "null_mean": 90.481, "null_std": 3.946, "z": 0.132, - "p_value": 0.479 + "p_value": 0.47952 }, { "skip": 4, @@ -1886,7 +1965,7 @@ "null_mean": 72.263, "null_std": 3.943, "z": 0.187, - "p_value": 0.451 + "p_value": 0.45155 }, { "skip": 5, @@ -1895,7 +1974,7 @@ "null_mean": 59.601, "null_std": 3.595, "z": -0.167, - "p_value": 0.627 + "p_value": 0.62737 }, { "skip": 6, @@ -1904,7 +1983,7 @@ "null_mean": 49.936, "null_std": 3.39, "z": -1.161, - "p_value": 0.907 + "p_value": 0.90709 }, { "skip": 7, @@ -1913,7 +1992,7 @@ "null_mean": 42.485, "null_std": 3.253, "z": -1.994, - "p_value": 0.99 + "p_value": 0.99001 }, { "skip": 8, @@ -1922,7 +2001,7 @@ "null_mean": 36.736, "null_std": 3.173, "z": 1.029, - "p_value": 0.195 + "p_value": 0.1958 }, { "skip": 9, @@ -1931,7 +2010,7 @@ "null_mean": 31.747, "null_std": 2.986, "z": 2.094, - "p_value": 0.028 + "p_value": 0.02897 }, { "skip": 10, @@ -1940,7 +2019,7 @@ "null_mean": 27.702, "null_std": 2.956, "z": -1.591, - "p_value": 0.958 + "p_value": 0.95804 }, { "skip": 11, @@ -1949,7 +2028,7 @@ "null_mean": 24.42, "null_std": 2.707, "z": -0.155, - "p_value": 0.644 + "p_value": 0.64436 }, { "skip": 12, @@ -1958,7 +2037,7 @@ "null_mean": 21.49, "null_std": 2.566, "z": -0.971, - "p_value": 0.889 + "p_value": 0.88911 }, { "skip": 13, @@ -1967,7 +2046,7 @@ "null_mean": 19.191, "null_std": 2.573, "z": 0.703, - "p_value": 0.3 + "p_value": 0.3007 }, { "skip": 14, @@ -1976,7 +2055,7 @@ "null_mean": 17.267, "null_std": 2.494, "z": -0.508, - "p_value": 0.761 + "p_value": 0.76124 }, { "skip": 15, @@ -1985,7 +2064,7 @@ "null_mean": 15.448, "null_std": 2.458, "z": 1.445, - "p_value": 0.108 + "p_value": 0.10889 }, { "skip": 16, @@ -2003,7 +2082,7 @@ "null_mean": 12.555, "null_std": 2.38, "z": 0.187, - "p_value": 0.497 + "p_value": 0.4975 }, { "skip": 18, @@ -2012,7 +2091,7 @@ "null_mean": 11.335, "null_std": 2.137, "z": -0.157, - "p_value": 0.645 + "p_value": 0.64535 }, { "skip": 19, @@ -2021,7 +2100,7 @@ "null_mean": 10.26, "null_std": 2.091, "z": -0.603, - "p_value": 0.799 + "p_value": 0.7992 }, { "skip": 20, @@ -2030,7 +2109,7 @@ "null_mean": 9.362, "null_std": 2.088, "z": 0.306, - "p_value": 0.451 + "p_value": 0.45155 }, { "skip": 21, @@ -2039,7 +2118,7 @@ "null_mean": 8.635, "null_std": 2.032, "z": -0.312, - "p_value": 0.711 + "p_value": 0.71129 }, { "skip": 22, @@ -2048,7 +2127,7 @@ "null_mean": 7.946, "null_std": 2.014, "z": 0.523, - "p_value": 0.392 + "p_value": 0.39261 }, { "skip": 23, @@ -2057,7 +2136,7 @@ "null_mean": 7.365, "null_std": 1.99, "z": 0.319, - "p_value": 0.468 + "p_value": 0.46853 }, { "skip": 24, @@ -2066,7 +2145,7 @@ "null_mean": 6.565, "null_std": 1.84, "z": 0.78, - "p_value": 0.29 + "p_value": 0.29071 }, { "skip": 25, @@ -2075,7 +2154,7 @@ "null_mean": 6.255, "null_std": 1.934, "z": -0.132, - "p_value": 0.646 + "p_value": 0.64635 }, { "skip": 26, @@ -2084,7 +2163,7 @@ "null_mean": 5.717, "null_std": 1.756, "z": -1.547, - "p_value": 0.977 + "p_value": 0.97702 }, { "skip": 27, @@ -2093,7 +2172,7 @@ "null_mean": 5.346, "null_std": 1.743, "z": -0.772, - "p_value": 0.849 + "p_value": 0.84915 }, { "skip": 28, @@ -2102,7 +2181,7 @@ "null_mean": 5.025, "null_std": 1.654, "z": -0.015, - "p_value": 0.64 + "p_value": 0.64036 }, { "skip": 29, @@ -2111,7 +2190,7 @@ "null_mean": 4.566, "null_std": 1.622, "z": -0.965, - "p_value": 0.907 + "p_value": 0.90709 }, { "skip": 30, @@ -2120,7 +2199,7 @@ "null_mean": 4.331, "null_std": 1.622, "z": 0.413, - "p_value": 0.447 + "p_value": 0.44755 }, { "skip": 31, @@ -2129,7 +2208,7 @@ "null_mean": 3.844, "null_std": 1.438, "z": 0.108, - "p_value": 0.59 + "p_value": 0.59041 }, { "skip": 32, @@ -2138,7 +2217,7 @@ "null_mean": 3.733, "null_std": 1.51, "z": -0.485, - "p_value": 0.794 + "p_value": 0.79421 }, { "skip": 33, @@ -2147,7 +2226,7 @@ "null_mean": 3.447, "null_std": 1.522, "z": -0.294, - "p_value": 0.717 + "p_value": 0.71728 }, { "skip": 34, @@ -2156,7 +2235,7 @@ "null_mean": 3.226, "null_std": 1.474, "z": -0.832, - "p_value": 0.881 + "p_value": 0.88112 }, { "skip": 35, @@ -2165,7 +2244,7 @@ "null_mean": 3.014, "null_std": 1.386, "z": 1.433, - "p_value": 0.133 + "p_value": 0.13387 }, { "skip": 36, @@ -2174,7 +2253,7 @@ "null_mean": 2.882, "null_std": 1.406, "z": -1.339, - "p_value": 0.974 + "p_value": 0.97403 }, { "skip": 37, @@ -2183,7 +2262,7 @@ "null_mean": 2.606, "null_std": 1.319, "z": 1.057, - "p_value": 0.241 + "p_value": 0.24176 }, { "skip": 38, @@ -2192,7 +2271,7 @@ "null_mean": 2.442, "null_std": 1.279, "z": 1.218, - "p_value": 0.194 + "p_value": 0.19481 }, { "skip": 39, @@ -2201,7 +2280,7 @@ "null_mean": 2.39, "null_std": 1.299, "z": -1.07, - "p_value": 0.942 + "p_value": 0.94206 }, { "skip": 40, @@ -2210,7 +2289,7 @@ "null_mean": 2.288, "null_std": 1.249, "z": -1.031, - "p_value": 0.935 + "p_value": 0.93506 }, { "skip": 41, @@ -2219,7 +2298,7 @@ "null_mean": 2.155, "null_std": 1.235, "z": 0.684, - "p_value": 0.365 + "p_value": 0.36563 }, { "skip": 42, @@ -2228,7 +2307,7 @@ "null_mean": 2.114, "null_std": 1.264, "z": -0.09, - "p_value": 0.651 + "p_value": 0.65135 }, { "skip": 43, @@ -2237,7 +2316,7 @@ "null_mean": 1.927, "null_std": 1.174, "z": 1.765, - "p_value": 0.095 + "p_value": 0.0959 }, { "skip": 44, @@ -2246,7 +2325,7 @@ "null_mean": 1.826, "null_std": 1.1, "z": 1.976, - "p_value": 0.066 + "p_value": 0.06693 }, { "skip": 45, @@ -2264,7 +2343,7 @@ "null_mean": 1.615, "null_std": 1.089, "z": -0.565, - "p_value": 0.858 + "p_value": 0.85814 }, { "skip": 47, @@ -2273,7 +2352,7 @@ "null_mean": 1.662, "null_std": 1.132, "z": 0.299, - "p_value": 0.52 + "p_value": 0.52048 }, { "skip": 48, @@ -2282,7 +2361,7 @@ "null_mean": 1.494, "null_std": 1.037, "z": 0.488, - "p_value": 0.46 + "p_value": 0.46054 }, { "skip": 49, @@ -2291,7 +2370,7 @@ "null_mean": 1.41, "null_std": 1.01, "z": -0.406, - "p_value": 0.809 + "p_value": 0.80919 }, { "skip": 50, @@ -2300,7 +2379,7 @@ "null_mean": 1.392, "null_std": 1.052, "z": 0.578, - "p_value": 0.43 + "p_value": 0.43057 }, { "skip": 51, @@ -2318,7 +2397,7 @@ "null_mean": 1.222, "null_std": 0.999, "z": -0.222, - "p_value": 0.748 + "p_value": 0.74825 }, { "skip": 53, @@ -2327,7 +2406,7 @@ "null_mean": 1.184, "null_std": 0.971, "z": 0.84, - "p_value": 0.324 + "p_value": 0.32468 }, { "skip": 54, @@ -2345,7 +2424,7 @@ "null_mean": 1.055, "null_std": 0.91, "z": -0.06, - "p_value": 0.698 + "p_value": 0.6983 }, { "skip": 56, @@ -2354,7 +2433,7 @@ "null_mean": 1.037, "null_std": 0.933, "z": -0.04, - "p_value": 0.671 + "p_value": 0.67133 }, { "skip": 57, @@ -2363,7 +2442,7 @@ "null_mean": 0.934, "null_std": 0.875, "z": 0.075, - "p_value": 0.637 + "p_value": 0.63736 }, { "skip": 58, @@ -2381,7 +2460,7 @@ "null_mean": 0.886, "null_std": 0.861, "z": 1.293, - "p_value": 0.213 + "p_value": 0.21379 }, { "skip": 60, @@ -2390,7 +2469,7 @@ "null_mean": 0.886, "null_std": 0.891, "z": 0.128, - "p_value": 0.604 + "p_value": 0.6044 }, { "skip": 61, @@ -2399,7 +2478,7 @@ "null_mean": 0.869, "null_std": 0.87, "z": 0.151, - "p_value": 0.601 + "p_value": 0.6014 }, { "skip": 62, @@ -2408,7 +2487,7 @@ "null_mean": 0.75, "null_std": 0.808, "z": 0.31, - "p_value": 0.551 + "p_value": 0.55145 }, { "skip": 63, @@ -2435,7 +2514,7 @@ "null_mean": 0.718, "null_std": 0.798, "z": 0.353, - "p_value": 0.533 + "p_value": 0.53347 }, { "skip": 66, @@ -2471,7 +2550,7 @@ "null_mean": 0.639, "null_std": 0.746, "z": 0.484, - "p_value": 0.491 + "p_value": 0.49151 }, { "skip": 70, @@ -2489,7 +2568,7 @@ "null_mean": 0.56, "null_std": 0.691, "z": 2.085, - "p_value": 0.089 + "p_value": 0.08991 }, { "skip": 72, @@ -2498,7 +2577,7 @@ "null_mean": 0.505, "null_std": 0.664, "z": 0.746, - "p_value": 0.42 + "p_value": 0.42058 }, { "skip": 73, @@ -2516,7 +2595,7 @@ "null_mean": 0.467, "null_std": 0.629, "z": 0.848, - "p_value": 0.396 + "p_value": 0.3966 }, { "skip": 75, @@ -2543,7 +2622,7 @@ "null_mean": 0.46, "null_std": 0.639, "z": 0.845, - "p_value": 0.385 + "p_value": 0.38561 }, { "skip": 78, @@ -2561,7 +2640,7 @@ "null_mean": 0.419, "null_std": 0.632, "z": 2.5, - "p_value": 0.062 + "p_value": 0.06294 }, { "skip": 80, @@ -2570,7 +2649,7 @@ "null_mean": 0.445, "null_std": 0.624, "z": 0.889, - "p_value": 0.378 + "p_value": 0.37862 }, { "skip": 81, @@ -2579,7 +2658,7 @@ "null_mean": 0.37, "null_std": 0.595, "z": 1.06, - "p_value": 0.318 + "p_value": 0.31868 }, { "skip": 82, @@ -2615,7 +2694,7 @@ "null_mean": 0.331, "null_std": 0.546, "z": 1.226, - "p_value": 0.294 + "p_value": 0.29471 }, { "skip": 86, @@ -2624,7 +2703,7 @@ "null_mean": 0.324, "null_std": 0.556, "z": 1.215, - "p_value": 0.283 + "p_value": 0.28372 }, { "skip": 87, @@ -2642,7 +2721,7 @@ "null_mean": 0.338, "null_std": 0.548, "z": 1.209, - "p_value": 0.302 + "p_value": 0.3027 }, { "skip": 89, @@ -2714,7 +2793,7 @@ "null_mean": 0.268, "null_std": 0.508, "z": 1.44, - "p_value": 0.241 + "p_value": 0.24176 }, { "skip": 97, @@ -2761,7 +2840,7 @@ "null_mean": 354.317, "null_std": 6.282, "z": -0.369, - "p_value": 0.68 + "p_value": 0.68032 }, { "skip": 3, @@ -2770,7 +2849,7 @@ "null_mean": 299.744, "null_std": 5.959, "z": -0.293, - "p_value": 0.647 + "p_value": 0.64735 }, { "skip": 4, @@ -2779,7 +2858,7 @@ "null_mean": 265.721, "null_std": 5.545, "z": -0.31, - "p_value": 0.65 + "p_value": 0.65035 }, { "skip": 5, @@ -2797,7 +2876,7 @@ "null_mean": 224.865, "null_std": 5.24, "z": 0.026, - "p_value": 0.522 + "p_value": 0.52248 }, { "skip": 7, @@ -2806,7 +2885,7 @@ "null_mean": 211.166, "null_std": 5.173, "z": 0.161, - "p_value": 0.489 + "p_value": 0.48951 }, { "skip": 8, @@ -2815,7 +2894,7 @@ "null_mean": 199.678, "null_std": 5.174, "z": 0.255, - "p_value": 0.44 + "p_value": 0.44056 }, { "skip": 9, @@ -2824,7 +2903,7 @@ "null_mean": 189.839, "null_std": 5.074, "z": -0.954, - "p_value": 0.842 + "p_value": 0.84216 }, { "skip": 10, @@ -2833,7 +2912,7 @@ "null_mean": 181.336, "null_std": 4.82, "z": -1.107, - "p_value": 0.895 + "p_value": 0.8951 }, { "skip": 11, @@ -2842,7 +2921,7 @@ "null_mean": 173.691, "null_std": 5.057, "z": -0.73, - "p_value": 0.805 + "p_value": 0.80519 }, { "skip": 12, @@ -2851,7 +2930,7 @@ "null_mean": 166.788, "null_std": 4.724, "z": -0.802, - "p_value": 0.823 + "p_value": 0.82318 }, { "skip": 13, @@ -2860,7 +2939,7 @@ "null_mean": 160.959, "null_std": 4.934, "z": 0.819, - "p_value": 0.236 + "p_value": 0.23676 }, { "skip": 14, @@ -2869,7 +2948,7 @@ "null_mean": 154.886, "null_std": 4.745, "z": -0.608, - "p_value": 0.756 + "p_value": 0.75624 }, { "skip": 15, @@ -2878,7 +2957,7 @@ "null_mean": 150.021, "null_std": 4.777, "z": -1.051, - "p_value": 0.872 + "p_value": 0.87213 }, { "skip": 16, @@ -2887,7 +2966,7 @@ "null_mean": 144.891, "null_std": 4.719, "z": 0.023, - "p_value": 0.546 + "p_value": 0.54645 }, { "skip": 17, @@ -2896,7 +2975,7 @@ "null_mean": 140.075, "null_std": 4.796, "z": -0.641, - "p_value": 0.757 + "p_value": 0.75724 }, { "skip": 18, @@ -2905,7 +2984,7 @@ "null_mean": 136.341, "null_std": 4.814, "z": 0.552, - "p_value": 0.328 + "p_value": 0.32867 }, { "skip": 19, @@ -2914,7 +2993,7 @@ "null_mean": 132.255, "null_std": 4.69, "z": -0.481, - "p_value": 0.73 + "p_value": 0.73027 }, { "skip": 20, @@ -2923,7 +3002,7 @@ "null_mean": 128.233, "null_std": 4.618, "z": 1.682, - "p_value": 0.06 + "p_value": 0.06094 }, { "skip": 21, @@ -2932,7 +3011,7 @@ "null_mean": 124.633, "null_std": 4.742, "z": -0.555, - "p_value": 0.743 + "p_value": 0.74326 }, { "skip": 22, @@ -2941,7 +3020,7 @@ "null_mean": 121.419, "null_std": 4.715, "z": -0.937, - "p_value": 0.85 + "p_value": 0.85015 }, { "skip": 23, @@ -2950,7 +3029,7 @@ "null_mean": 118.075, "null_std": 4.732, "z": 1.675, - "p_value": 0.063 + "p_value": 0.06394 }, { "skip": 24, @@ -2959,7 +3038,7 @@ "null_mean": 115.219, "null_std": 4.543, "z": 0.612, - "p_value": 0.316 + "p_value": 0.31668 }, { "skip": 25, @@ -2968,7 +3047,7 @@ "null_mean": 112.377, "null_std": 4.569, "z": -0.739, - "p_value": 0.801 + "p_value": 0.8012 }, { "skip": 26, @@ -2977,7 +3056,7 @@ "null_mean": 109.757, "null_std": 4.483, "z": -0.838, - "p_value": 0.834 + "p_value": 0.83417 }, { "skip": 27, @@ -2986,7 +3065,7 @@ "null_mean": 106.855, "null_std": 4.466, "z": -1.759, - "p_value": 0.966 + "p_value": 0.96603 }, { "skip": 28, @@ -2995,7 +3074,7 @@ "null_mean": 104.481, "null_std": 4.542, "z": 0.555, - "p_value": 0.316 + "p_value": 0.31668 }, { "skip": 29, @@ -3004,7 +3083,7 @@ "null_mean": 102.296, "null_std": 4.686, "z": 0.364, - "p_value": 0.385 + "p_value": 0.38561 }, { "skip": 30, @@ -3013,7 +3092,7 @@ "null_mean": 100.153, "null_std": 4.593, "z": -0.033, - "p_value": 0.546 + "p_value": 0.54645 }, { "skip": 31, @@ -3022,7 +3101,7 @@ "null_mean": 97.96, "null_std": 4.406, "z": -0.218, - "p_value": 0.652 + "p_value": 0.65235 }, { "skip": 32, @@ -3031,7 +3110,7 @@ "null_mean": 95.766, "null_std": 4.297, "z": 0.054, - "p_value": 0.514 + "p_value": 0.51449 }, { "skip": 33, @@ -3040,7 +3119,7 @@ "null_mean": 93.575, "null_std": 4.296, "z": -1.53, - "p_value": 0.95 + "p_value": 0.95005 }, { "skip": 34, @@ -3049,7 +3128,7 @@ "null_mean": 91.7, "null_std": 4.366, "z": -0.16, - "p_value": 0.609 + "p_value": 0.60939 }, { "skip": 35, @@ -3058,7 +3137,7 @@ "null_mean": 89.987, "null_std": 4.269, "z": -2.34, - "p_value": 0.992 + "p_value": 0.99201 }, { "skip": 36, @@ -3067,7 +3146,7 @@ "null_mean": 88.093, "null_std": 4.094, "z": 1.687, - "p_value": 0.052 + "p_value": 0.05295 }, { "skip": 37, @@ -3076,7 +3155,7 @@ "null_mean": 86.761, "null_std": 4.195, "z": -0.658, - "p_value": 0.767 + "p_value": 0.76723 }, { "skip": 38, @@ -3085,7 +3164,7 @@ "null_mean": 84.899, "null_std": 4.195, "z": 0.978, - "p_value": 0.198 + "p_value": 0.1988 }, { "skip": 39, @@ -3094,7 +3173,7 @@ "null_mean": 83.406, "null_std": 4.261, "z": -2.208, - "p_value": 0.994 + "p_value": 0.99401 }, { "skip": 40, @@ -3103,7 +3182,7 @@ "null_mean": 82.051, "null_std": 4.101, "z": 1.451, - "p_value": 0.104 + "p_value": 0.1049 }, { "skip": 41, @@ -3112,7 +3191,7 @@ "null_mean": 80.546, "null_std": 4.09, "z": 0.111, - "p_value": 0.504 + "p_value": 0.5045 }, { "skip": 42, @@ -3121,7 +3200,7 @@ "null_mean": 78.769, "null_std": 4.231, "z": 0.764, - "p_value": 0.273 + "p_value": 0.27373 }, { "skip": 43, @@ -3130,7 +3209,7 @@ "null_mean": 77.434, "null_std": 4.09, "z": 0.627, - "p_value": 0.307 + "p_value": 0.30769 }, { "skip": 44, @@ -3139,7 +3218,7 @@ "null_mean": 76.18, "null_std": 4.073, "z": -0.29, - "p_value": 0.661 + "p_value": 0.66134 }, { "skip": 45, @@ -3148,7 +3227,7 @@ "null_mean": 75.08, "null_std": 3.878, "z": -0.536, - "p_value": 0.753 + "p_value": 0.75325 }, { "skip": 46, @@ -3157,7 +3236,7 @@ "null_mean": 73.679, "null_std": 4.051, "z": 0.573, - "p_value": 0.32 + "p_value": 0.32068 }, { "skip": 47, @@ -3166,7 +3245,7 @@ "null_mean": 72.602, "null_std": 3.969, "z": -0.907, - "p_value": 0.852 + "p_value": 0.85215 }, { "skip": 48, @@ -3175,7 +3254,7 @@ "null_mean": 71.515, "null_std": 3.822, "z": 1.697, - "p_value": 0.047 + "p_value": 0.04795 }, { "skip": 49, @@ -3184,7 +3263,7 @@ "null_mean": 70.358, "null_std": 3.97, "z": 0.917, - "p_value": 0.223 + "p_value": 0.22378 }, { "skip": 50, @@ -3193,7 +3272,7 @@ "null_mean": 69.163, "null_std": 4.072, "z": -0.286, - "p_value": 0.638 + "p_value": 0.63836 }, { "skip": 51, @@ -3202,7 +3281,7 @@ "null_mean": 67.97, "null_std": 3.861, "z": -0.251, - "p_value": 0.642 + "p_value": 0.64236 }, { "skip": 52, @@ -3211,7 +3290,7 @@ "null_mean": 67.055, "null_std": 3.786, "z": 0.25, - "p_value": 0.463 + "p_value": 0.46354 }, { "skip": 53, @@ -3220,7 +3299,7 @@ "null_mean": 65.875, "null_std": 3.771, "z": 0.829, - "p_value": 0.254 + "p_value": 0.25475 }, { "skip": 54, @@ -3229,7 +3308,7 @@ "null_mean": 64.773, "null_std": 3.799, "z": -0.993, - "p_value": 0.881 + "p_value": 0.88112 }, { "skip": 55, @@ -3238,7 +3317,7 @@ "null_mean": 64.221, "null_std": 3.646, "z": -0.061, - "p_value": 0.581 + "p_value": 0.58142 }, { "skip": 56, @@ -3247,7 +3326,7 @@ "null_mean": 63.218, "null_std": 3.699, "z": 1.023, - "p_value": 0.188 + "p_value": 0.18881 }, { "skip": 57, @@ -3256,7 +3335,7 @@ "null_mean": 62.31, "null_std": 3.685, "z": 0.187, - "p_value": 0.476 + "p_value": 0.47652 }, { "skip": 58, @@ -3265,7 +3344,7 @@ "null_mean": 61.439, "null_std": 3.84, "z": 0.407, - "p_value": 0.405 + "p_value": 0.40559 }, { "skip": 59, @@ -3274,7 +3353,7 @@ "null_mean": 60.535, "null_std": 3.74, "z": 1.194, - "p_value": 0.148 + "p_value": 0.14885 }, { "skip": 60, @@ -3283,7 +3362,7 @@ "null_mean": 59.763, "null_std": 3.809, "z": 0.325, - "p_value": 0.427 + "p_value": 0.42757 }, { "skip": 61, @@ -3292,7 +3371,7 @@ "null_mean": 58.856, "null_std": 3.628, "z": 0.315, - "p_value": 0.434 + "p_value": 0.43457 }, { "skip": 62, @@ -3301,7 +3380,7 @@ "null_mean": 58.17, "null_std": 3.63, "z": -0.322, - "p_value": 0.676 + "p_value": 0.67632 }, { "skip": 63, @@ -3310,7 +3389,7 @@ "null_mean": 57.204, "null_std": 3.569, "z": 1.904, - "p_value": 0.039 + "p_value": 0.03996 }, { "skip": 64, @@ -3319,7 +3398,7 @@ "null_mean": 56.538, "null_std": 3.652, "z": 0.948, - "p_value": 0.208 + "p_value": 0.20879 }, { "skip": 65, @@ -3328,7 +3407,7 @@ "null_mean": 55.65, "null_std": 3.765, "z": -0.173, - "p_value": 0.619 + "p_value": 0.61938 }, { "skip": 66, @@ -3346,7 +3425,7 @@ "null_mean": 54.286, "null_std": 3.558, "z": 0.201, - "p_value": 0.471 + "p_value": 0.47153 }, { "skip": 68, @@ -3355,7 +3434,7 @@ "null_mean": 53.496, "null_std": 3.508, "z": 1.284, - "p_value": 0.127 + "p_value": 0.12787 }, { "skip": 69, @@ -3364,7 +3443,7 @@ "null_mean": 52.812, "null_std": 3.668, "z": 1.142, - "p_value": 0.157 + "p_value": 0.15784 }, { "skip": 70, @@ -3373,7 +3452,7 @@ "null_mean": 52.208, "null_std": 3.566, "z": -1.18, - "p_value": 0.902 + "p_value": 0.9021 }, { "skip": 71, @@ -3382,7 +3461,7 @@ "null_mean": 51.452, "null_std": 3.448, "z": 1.029, - "p_value": 0.183 + "p_value": 0.18382 }, { "skip": 72, @@ -3391,7 +3470,7 @@ "null_mean": 50.972, "null_std": 3.5, "z": 0.008, - "p_value": 0.552 + "p_value": 0.55245 }, { "skip": 73, @@ -3400,7 +3479,7 @@ "null_mean": 50.284, "null_std": 3.644, "z": -0.901, - "p_value": 0.857 + "p_value": 0.85714 }, { "skip": 74, @@ -3409,7 +3488,7 @@ "null_mean": 49.725, "null_std": 3.444, "z": 0.37, - "p_value": 0.409 + "p_value": 0.40959 }, { "skip": 75, @@ -3418,7 +3497,7 @@ "null_mean": 49.17, "null_std": 3.244, "z": 1.181, - "p_value": 0.15 + "p_value": 0.15085 }, { "skip": 76, @@ -3427,7 +3506,7 @@ "null_mean": 48.412, "null_std": 3.422, "z": 2.217, - "p_value": 0.023 + "p_value": 0.02398 }, { "skip": 77, @@ -3436,7 +3515,7 @@ "null_mean": 47.85, "null_std": 3.414, "z": -0.249, - "p_value": 0.658 + "p_value": 0.65834 }, { "skip": 78, @@ -3445,7 +3524,7 @@ "null_mean": 47.111, "null_std": 3.469, "z": 1.121, - "p_value": 0.157 + "p_value": 0.15784 }, { "skip": 79, @@ -3454,7 +3533,7 @@ "null_mean": 46.412, "null_std": 3.413, "z": 0.465, - "p_value": 0.382 + "p_value": 0.38262 }, { "skip": 80, @@ -3463,7 +3542,7 @@ "null_mean": 46.193, "null_std": 3.46, "z": -0.345, - "p_value": 0.697 + "p_value": 0.6973 }, { "skip": 81, @@ -3472,7 +3551,7 @@ "null_mean": 45.507, "null_std": 3.317, "z": -0.153, - "p_value": 0.632 + "p_value": 0.63237 }, { "skip": 82, @@ -3481,7 +3560,7 @@ "null_mean": 45.035, "null_std": 3.281, "z": 1.208, - "p_value": 0.134 + "p_value": 0.13487 }, { "skip": 83, @@ -3490,7 +3569,7 @@ "null_mean": 44.459, "null_std": 3.437, "z": 1.612, - "p_value": 0.069 + "p_value": 0.06993 }, { "skip": 84, @@ -3499,7 +3578,7 @@ "null_mean": 44.166, "null_std": 3.215, "z": 0.882, - "p_value": 0.228 + "p_value": 0.22877 }, { "skip": 85, @@ -3508,7 +3587,7 @@ "null_mean": 43.508, "null_std": 3.244, "z": 0.46, - "p_value": 0.378 + "p_value": 0.37862 }, { "skip": 86, @@ -3517,7 +3596,7 @@ "null_mean": 42.944, "null_std": 3.299, "z": -0.286, - "p_value": 0.672 + "p_value": 0.67233 }, { "skip": 87, @@ -3526,7 +3605,7 @@ "null_mean": 42.554, "null_std": 3.281, "z": -0.169, - "p_value": 0.628 + "p_value": 0.62837 }, { "skip": 88, @@ -3535,7 +3614,7 @@ "null_mean": 42.207, "null_std": 3.257, "z": -0.371, - "p_value": 0.697 + "p_value": 0.6973 }, { "skip": 89, @@ -3544,7 +3623,7 @@ "null_mean": 41.611, "null_std": 3.187, "z": 1.063, - "p_value": 0.181 + "p_value": 0.18182 }, { "skip": 90, @@ -3553,7 +3632,7 @@ "null_mean": 41.008, "null_std": 3.272, "z": -0.919, - "p_value": 0.863 + "p_value": 0.86314 }, { "skip": 91, @@ -3562,7 +3641,7 @@ "null_mean": 40.608, "null_std": 3.285, "z": -0.489, - "p_value": 0.754 + "p_value": 0.75425 }, { "skip": 92, @@ -3571,7 +3650,7 @@ "null_mean": 40.069, "null_std": 3.133, "z": 0.297, - "p_value": 0.43 + "p_value": 0.43057 }, { "skip": 93, @@ -3580,7 +3659,7 @@ "null_mean": 39.698, "null_std": 3.17, "z": 0.411, - "p_value": 0.403 + "p_value": 0.4036 }, { "skip": 94, @@ -3589,7 +3668,7 @@ "null_mean": 39.472, "null_std": 3.22, "z": 0.164, - "p_value": 0.496 + "p_value": 0.4965 }, { "skip": 95, @@ -3598,7 +3677,7 @@ "null_mean": 39.094, "null_std": 3.224, "z": 0.281, - "p_value": 0.451 + "p_value": 0.45155 }, { "skip": 96, @@ -3607,7 +3686,7 @@ "null_mean": 38.603, "null_std": 3.143, "z": 0.763, - "p_value": 0.273 + "p_value": 0.27373 }, { "skip": 97, @@ -3616,7 +3695,7 @@ "null_mean": 37.983, "null_std": 3.179, "z": -1.253, - "p_value": 0.923 + "p_value": 0.92308 }, { "skip": 98, @@ -3625,7 +3704,7 @@ "null_mean": 37.684, "null_std": 3.186, "z": 1.355, - "p_value": 0.12 + "p_value": 0.12088 }, { "skip": 99, @@ -3634,7 +3713,7 @@ "null_mean": 37.222, "null_std": 3.155, "z": -0.07, - "p_value": 0.589 + "p_value": 0.58941 }, { "skip": 100, @@ -3643,7 +3722,7 @@ "null_mean": 37.017, "null_std": 3.296, "z": -1.219, - "p_value": 0.918 + "p_value": 0.91808 } ], "Letter": [ @@ -3654,7 +3733,7 @@ "null_mean": 270.934, "null_std": 5.275, "z": 0.202, - "p_value": 0.449 + "p_value": 0.44955 }, { "skip": 3, @@ -3663,7 +3742,7 @@ "null_mean": 228.712, "null_std": 5.04, "z": -0.141, - "p_value": 0.575 + "p_value": 0.57542 }, { "skip": 4, @@ -3672,7 +3751,7 @@ "null_mean": 202.496, "null_std": 4.934, "z": 1.318, - "p_value": 0.123 + "p_value": 0.12388 }, { "skip": 5, @@ -3681,7 +3760,7 @@ "null_mean": 183.047, "null_std": 5.007, "z": -1.407, - "p_value": 0.936 + "p_value": 0.93606 }, { "skip": 6, @@ -3690,7 +3769,7 @@ "null_mean": 167.453, "null_std": 5.109, "z": -0.872, - "p_value": 0.824 + "p_value": 0.82418 }, { "skip": 7, @@ -3699,7 +3778,7 @@ "null_mean": 155.319, "null_std": 4.892, "z": 0.957, - "p_value": 0.184 + "p_value": 0.18482 }, { "skip": 8, @@ -3708,7 +3787,7 @@ "null_mean": 144.875, "null_std": 4.867, "z": -1.823, - "p_value": 0.973 + "p_value": 0.97303 }, { "skip": 9, @@ -3717,7 +3796,7 @@ "null_mean": 135.06, "null_std": 4.876, "z": 0.193, - "p_value": 0.445 + "p_value": 0.44555 }, { "skip": 10, @@ -3726,7 +3805,7 @@ "null_mean": 127.506, "null_std": 4.795, "z": -0.731, - "p_value": 0.801 + "p_value": 0.8012 }, { "skip": 11, @@ -3735,7 +3814,7 @@ "null_mean": 120.232, "null_std": 4.75, "z": 0.372, - "p_value": 0.41 + "p_value": 0.41059 }, { "skip": 12, @@ -3744,7 +3823,7 @@ "null_mean": 113.804, "null_std": 4.706, "z": 1.104, - "p_value": 0.158 + "p_value": 0.15884 }, { "skip": 13, @@ -3753,7 +3832,7 @@ "null_mean": 108.354, "null_std": 4.498, "z": -0.523, - "p_value": 0.738 + "p_value": 0.73826 }, { "skip": 14, @@ -3762,7 +3841,7 @@ "null_mean": 103.105, "null_std": 4.412, "z": 0.883, - "p_value": 0.232 + "p_value": 0.23277 }, { "skip": 15, @@ -3771,7 +3850,7 @@ "null_mean": 98.478, "null_std": 4.506, "z": 0.338, - "p_value": 0.413 + "p_value": 0.41359 }, { "skip": 16, @@ -3780,7 +3859,7 @@ "null_mean": 94.39, "null_std": 4.144, "z": -1.783, - "p_value": 0.978 + "p_value": 0.97802 }, { "skip": 17, @@ -3789,7 +3868,7 @@ "null_mean": 90.702, "null_std": 4.306, "z": -0.628, - "p_value": 0.771 + "p_value": 0.77123 }, { "skip": 18, @@ -3798,7 +3877,7 @@ "null_mean": 86.929, "null_std": 4.257, "z": -0.453, - "p_value": 0.717 + "p_value": 0.71728 }, { "skip": 19, @@ -3807,7 +3886,7 @@ "null_mean": 84.217, "null_std": 4.282, "z": -1.218, - "p_value": 0.913 + "p_value": 0.91309 }, { "skip": 20, @@ -3816,7 +3895,7 @@ "null_mean": 80.861, "null_std": 4.189, "z": -1.399, - "p_value": 0.935 + "p_value": 0.93506 }, { "skip": 21, @@ -3825,7 +3904,7 @@ "null_mean": 77.856, "null_std": 4.163, "z": 1.476, - "p_value": 0.079 + "p_value": 0.07992 }, { "skip": 22, @@ -3834,7 +3913,7 @@ "null_mean": 75.469, "null_std": 3.987, "z": 1.889, - "p_value": 0.043 + "p_value": 0.04396 }, { "skip": 23, @@ -3843,7 +3922,7 @@ "null_mean": 73.281, "null_std": 4.243, "z": -1.009, - "p_value": 0.864 + "p_value": 0.86414 }, { "skip": 24, @@ -3852,7 +3931,7 @@ "null_mean": 70.769, "null_std": 3.907, "z": -0.197, - "p_value": 0.621 + "p_value": 0.62138 }, { "skip": 25, @@ -3861,7 +3940,7 @@ "null_mean": 68.606, "null_std": 4.05, "z": -0.644, - "p_value": 0.791 + "p_value": 0.79121 }, { "skip": 26, @@ -3870,7 +3949,7 @@ "null_mean": 66.532, "null_std": 3.959, "z": -0.387, - "p_value": 0.702 + "p_value": 0.7023 }, { "skip": 27, @@ -3879,7 +3958,7 @@ "null_mean": 64.705, "null_std": 3.804, "z": -1.762, - "p_value": 0.976 + "p_value": 0.97602 }, { "skip": 28, @@ -3888,7 +3967,7 @@ "null_mean": 62.793, "null_std": 3.822, "z": -0.731, - "p_value": 0.8 + "p_value": 0.8002 }, { "skip": 29, @@ -3897,7 +3976,7 @@ "null_mean": 61.119, "null_std": 3.81, "z": -0.819, - "p_value": 0.825 + "p_value": 0.82517 }, { "skip": 30, @@ -3906,7 +3985,7 @@ "null_mean": 59.707, "null_std": 3.696, "z": 0.079, - "p_value": 0.513 + "p_value": 0.51349 }, { "skip": 31, @@ -3915,7 +3994,7 @@ "null_mean": 58.126, "null_std": 3.657, "z": 0.512, - "p_value": 0.361 + "p_value": 0.36164 }, { "skip": 32, @@ -3924,7 +4003,7 @@ "null_mean": 56.596, "null_std": 3.63, "z": -0.715, - "p_value": 0.8 + "p_value": 0.8002 }, { "skip": 33, @@ -3933,7 +4012,7 @@ "null_mean": 55.067, "null_std": 3.476, "z": 3.146, - "p_value": 0.001 + "p_value": 0.002 }, { "skip": 34, @@ -3942,7 +4021,7 @@ "null_mean": 53.826, "null_std": 3.46, "z": -0.528, - "p_value": 0.75 + "p_value": 0.75025 }, { "skip": 35, @@ -3951,7 +4030,7 @@ "null_mean": 52.626, "null_std": 3.515, "z": 1.245, - "p_value": 0.132 + "p_value": 0.13287 }, { "skip": 36, @@ -3960,7 +4039,7 @@ "null_mean": 51.515, "null_std": 3.517, "z": 0.422, - "p_value": 0.39 + "p_value": 0.39061 }, { "skip": 37, @@ -3969,7 +4048,7 @@ "null_mean": 50.379, "null_std": 3.476, "z": 0.179, - "p_value": 0.478 + "p_value": 0.47852 }, { "skip": 38, @@ -3978,7 +4057,7 @@ "null_mean": 49.3, "null_std": 3.411, "z": -0.968, - "p_value": 0.883 + "p_value": 0.88312 }, { "skip": 39, @@ -3987,7 +4066,7 @@ "null_mean": 48.166, "null_std": 3.495, "z": -0.048, - "p_value": 0.576 + "p_value": 0.57642 }, { "skip": 40, @@ -3996,7 +4075,7 @@ "null_mean": 47.222, "null_std": 3.32, "z": -0.368, - "p_value": 0.682 + "p_value": 0.68232 }, { "skip": 41, @@ -4005,7 +4084,7 @@ "null_mean": 46.19, "null_std": 3.396, "z": -0.645, - "p_value": 0.781 + "p_value": 0.78122 }, { "skip": 42, @@ -4014,7 +4093,7 @@ "null_mean": 45.239, "null_std": 3.203, "z": -0.075, - "p_value": 0.596 + "p_value": 0.5964 }, { "skip": 43, @@ -4023,7 +4102,7 @@ "null_mean": 44.282, "null_std": 3.454, "z": -0.661, - "p_value": 0.785 + "p_value": 0.78521 }, { "skip": 44, @@ -4032,7 +4111,7 @@ "null_mean": 43.761, "null_std": 3.153, "z": 2.296, - "p_value": 0.016 + "p_value": 0.01698 }, { "skip": 45, @@ -4041,7 +4120,7 @@ "null_mean": 42.53, "null_std": 3.254, "z": -0.163, - "p_value": 0.624 + "p_value": 0.62438 }, { "skip": 46, @@ -4050,7 +4129,7 @@ "null_mean": 41.957, "null_std": 3.21, "z": 0.948, - "p_value": 0.217 + "p_value": 0.21778 }, { "skip": 47, @@ -4059,7 +4138,7 @@ "null_mean": 41.359, "null_std": 3.132, "z": 1.162, - "p_value": 0.161 + "p_value": 0.16184 }, { "skip": 48, @@ -4068,7 +4147,7 @@ "null_mean": 40.289, "null_std": 3.212, "z": 1.155, - "p_value": 0.151 + "p_value": 0.15185 }, { "skip": 49, @@ -4077,7 +4156,7 @@ "null_mean": 39.434, "null_std": 3.23, "z": -0.753, - "p_value": 0.826 + "p_value": 0.82617 }, { "skip": 50, @@ -4086,7 +4165,7 @@ "null_mean": 38.851, "null_std": 3.07, "z": 0.374, - "p_value": 0.398 + "p_value": 0.3986 }, { "skip": 51, @@ -4095,7 +4174,7 @@ "null_mean": 38.123, "null_std": 3.039, "z": 0.289, - "p_value": 0.439 + "p_value": 0.43956 }, { "skip": 52, @@ -4104,7 +4183,7 @@ "null_mean": 37.381, "null_std": 3.042, "z": -1.111, - "p_value": 0.905 + "p_value": 0.90509 }, { "skip": 53, @@ -4113,7 +4192,7 @@ "null_mean": 36.707, "null_std": 3.034, "z": -1.551, - "p_value": 0.969 + "p_value": 0.96903 }, { "skip": 54, @@ -4122,7 +4201,7 @@ "null_mean": 36.205, "null_std": 2.968, "z": -1.417, - "p_value": 0.945 + "p_value": 0.94505 }, { "skip": 55, @@ -4131,7 +4210,7 @@ "null_mean": 35.542, "null_std": 2.996, "z": -0.181, - "p_value": 0.628 + "p_value": 0.62837 }, { "skip": 56, @@ -4140,7 +4219,7 @@ "null_mean": 35.168, "null_std": 2.889, "z": 0.98, - "p_value": 0.202 + "p_value": 0.2028 }, { "skip": 57, @@ -4149,7 +4228,7 @@ "null_mean": 34.448, "null_std": 2.921, "z": -0.153, - "p_value": 0.625 + "p_value": 0.62537 }, { "skip": 58, @@ -4158,7 +4237,7 @@ "null_mean": 33.874, "null_std": 2.984, "z": 0.377, - "p_value": 0.394 + "p_value": 0.39461 }, { "skip": 59, @@ -4167,7 +4246,7 @@ "null_mean": 33.388, "null_std": 2.905, "z": 0.555, - "p_value": 0.357 + "p_value": 0.35764 }, { "skip": 60, @@ -4176,7 +4255,7 @@ "null_mean": 32.872, "null_std": 2.984, "z": -0.963, - "p_value": 0.865 + "p_value": 0.86513 }, { "skip": 61, @@ -4185,7 +4264,7 @@ "null_mean": 32.341, "null_std": 2.928, "z": -0.458, - "p_value": 0.734 + "p_value": 0.73427 }, { "skip": 62, @@ -4194,7 +4273,7 @@ "null_mean": 31.869, "null_std": 2.829, "z": 0.753, - "p_value": 0.278 + "p_value": 0.27872 }, { "skip": 63, @@ -4203,7 +4282,7 @@ "null_mean": 31.414, "null_std": 2.803, "z": -0.861, - "p_value": 0.855 + "p_value": 0.85514 }, { "skip": 64, @@ -4212,7 +4291,7 @@ "null_mean": 31.006, "null_std": 2.768, "z": -0.363, - "p_value": 0.708 + "p_value": 0.70829 }, { "skip": 65, @@ -4221,7 +4300,7 @@ "null_mean": 30.469, "null_std": 2.783, "z": 0.191, - "p_value": 0.509 + "p_value": 0.50949 }, { "skip": 66, @@ -4230,7 +4309,7 @@ "null_mean": 30.093, "null_std": 2.673, "z": 1.087, - "p_value": 0.19 + "p_value": 0.19081 }, { "skip": 67, @@ -4239,7 +4318,7 @@ "null_mean": 29.374, "null_std": 2.815, "z": 0.222, - "p_value": 0.495 + "p_value": 0.4955 }, { "skip": 68, @@ -4248,7 +4327,7 @@ "null_mean": 29.155, "null_std": 2.755, "z": 0.307, - "p_value": 0.461 + "p_value": 0.46154 }, { "skip": 69, @@ -4257,7 +4336,7 @@ "null_mean": 28.772, "null_std": 2.794, "z": 1.155, - "p_value": 0.151 + "p_value": 0.15185 }, { "skip": 70, @@ -4266,7 +4345,7 @@ "null_mean": 28.241, "null_std": 2.592, "z": 0.293, - "p_value": 0.454 + "p_value": 0.45455 }, { "skip": 71, @@ -4275,7 +4354,7 @@ "null_mean": 27.971, "null_std": 2.697, "z": -0.36, - "p_value": 0.706 + "p_value": 0.70629 }, { "skip": 72, @@ -4284,7 +4363,7 @@ "null_mean": 27.522, "null_std": 2.738, "z": 1.27, - "p_value": 0.125 + "p_value": 0.12587 }, { "skip": 73, @@ -4293,7 +4372,7 @@ "null_mean": 27.092, "null_std": 2.611, "z": 0.348, - "p_value": 0.432 + "p_value": 0.43257 }, { "skip": 74, @@ -4302,7 +4381,7 @@ "null_mean": 26.741, "null_std": 2.593, "z": -0.286, - "p_value": 0.68 + "p_value": 0.68032 }, { "skip": 75, @@ -4320,7 +4399,7 @@ "null_mean": 26.03, "null_std": 2.589, "z": 0.761, - "p_value": 0.28 + "p_value": 0.28072 }, { "skip": 77, @@ -4329,7 +4408,7 @@ "null_mean": 25.672, "null_std": 2.655, "z": 2.006, - "p_value": 0.037 + "p_value": 0.03796 }, { "skip": 78, @@ -4338,7 +4417,7 @@ "null_mean": 25.358, "null_std": 2.537, "z": -0.93, - "p_value": 0.87 + "p_value": 0.87013 }, { "skip": 79, @@ -4347,7 +4426,7 @@ "null_mean": 25.136, "null_std": 2.577, "z": 0.723, - "p_value": 0.304 + "p_value": 0.3047 }, { "skip": 80, @@ -4356,7 +4435,7 @@ "null_mean": 24.865, "null_std": 2.619, "z": -0.712, - "p_value": 0.812 + "p_value": 0.81219 }, { "skip": 81, @@ -4365,7 +4444,7 @@ "null_mean": 24.484, "null_std": 2.533, "z": 1.388, - "p_value": 0.11 + "p_value": 0.11089 }, { "skip": 82, @@ -4374,7 +4453,7 @@ "null_mean": 24.23, "null_std": 2.483, "z": -0.093, - "p_value": 0.604 + "p_value": 0.6044 }, { "skip": 83, @@ -4383,7 +4462,7 @@ "null_mean": 23.798, "null_std": 2.53, "z": 0.08, - "p_value": 0.553 + "p_value": 0.55345 }, { "skip": 84, @@ -4392,7 +4471,7 @@ "null_mean": 23.536, "null_std": 2.524, "z": 0.184, - "p_value": 0.507 + "p_value": 0.50749 }, { "skip": 85, @@ -4401,7 +4480,7 @@ "null_mean": 23.401, "null_std": 2.527, "z": 0.237, - "p_value": 0.493 + "p_value": 0.49351 }, { "skip": 86, @@ -4410,7 +4489,7 @@ "null_mean": 23.086, "null_std": 2.618, "z": 0.731, - "p_value": 0.281 + "p_value": 0.28172 }, { "skip": 87, @@ -4419,7 +4498,7 @@ "null_mean": 22.789, "null_std": 2.44, "z": -1.143, - "p_value": 0.914 + "p_value": 0.91409 }, { "skip": 88, @@ -4428,7 +4507,7 @@ "null_mean": 22.386, "null_std": 2.428, "z": -0.983, - "p_value": 0.881 + "p_value": 0.88112 }, { "skip": 89, @@ -4437,7 +4516,7 @@ "null_mean": 22.146, "null_std": 2.482, "z": 0.747, - "p_value": 0.293 + "p_value": 0.29371 }, { "skip": 90, @@ -4446,7 +4525,7 @@ "null_mean": 21.898, "null_std": 2.459, "z": 1.262, - "p_value": 0.144 + "p_value": 0.14486 }, { "skip": 91, @@ -4455,7 +4534,7 @@ "null_mean": 21.674, "null_std": 2.44, "z": 0.543, - "p_value": 0.355 + "p_value": 0.35564 }, { "skip": 92, @@ -4464,7 +4543,7 @@ "null_mean": 21.308, "null_std": 2.437, "z": -0.537, - "p_value": 0.772 + "p_value": 0.77223 }, { "skip": 93, @@ -4473,7 +4552,7 @@ "null_mean": 21.181, "null_std": 2.456, "z": -0.481, - "p_value": 0.748 + "p_value": 0.74825 }, { "skip": 94, @@ -4482,7 +4561,7 @@ "null_mean": 20.98, "null_std": 2.407, "z": 1.255, - "p_value": 0.152 + "p_value": 0.15285 }, { "skip": 95, @@ -4491,7 +4570,7 @@ "null_mean": 20.705, "null_std": 2.328, "z": -0.303, - "p_value": 0.7 + "p_value": 0.7003 }, { "skip": 96, @@ -4500,7 +4579,7 @@ "null_mean": 20.356, "null_std": 2.366, "z": -2.264, - "p_value": 0.993 + "p_value": 0.99301 }, { "skip": 97, @@ -4509,7 +4588,7 @@ "null_mean": 20.141, "null_std": 2.333, "z": -0.918, - "p_value": 0.882 + "p_value": 0.88212 }, { "skip": 98, @@ -4518,7 +4597,7 @@ "null_mean": 19.902, "null_std": 2.264, "z": -0.84, - "p_value": 0.863 + "p_value": 0.86314 }, { "skip": 99, @@ -4527,7 +4606,7 @@ "null_mean": 19.825, "null_std": 2.343, "z": 0.075, - "p_value": 0.548 + "p_value": 0.54845 }, { "skip": 100, @@ -4536,7 +4615,7 @@ "null_mean": 19.373, "null_std": 2.387, "z": 0.682, - "p_value": 0.309 + "p_value": 0.30969 } ] }, @@ -4544,37 +4623,52 @@ "Administrative": { "n_pairs_observed": 17, "obs_mean_shared_trigrams": 0.176, - "null_mean_shared_trigrams": 0.005, - "delta": 0.171, - "p_value_approx": 0.00529 + "boundary_permute_null_mean": 0.343, + "boundary_permute_delta": -0.167, + "boundary_permute_p_upper": 0.78109, + "boundary_permute_p_lower": 0.44776, + "token_shuffle_null_mean": 0.004, + "token_shuffle_p_upper": 0.00498 }, "Literary": { "n_pairs_observed": 108, "obs_mean_shared_trigrams": 0.731, - "null_mean_shared_trigrams": 0.016, - "delta": 0.715, - "p_value_approx": 0.01431 + "boundary_permute_null_mean": 0.889, + "boundary_permute_delta": -0.158, + "boundary_permute_p_upper": 0.90547, + "boundary_permute_p_lower": 0.10945, + "token_shuffle_null_mean": 0.018, + "token_shuffle_p_upper": 0.00498 }, "Lexical": { "n_pairs_observed": 13, "obs_mean_shared_trigrams": 0.538, - "null_mean_shared_trigrams": 0.032, - "delta": 0.507, - "p_value_approx": 0.03115 + "boundary_permute_null_mean": 0.43, + "boundary_permute_delta": 0.108, + "boundary_permute_p_upper": 0.27363, + "boundary_permute_p_lower": 1.0, + "token_shuffle_null_mean": 0.037, + "token_shuffle_p_upper": 0.00498 }, "Royal Inscription": { "n_pairs_observed": 10, "obs_mean_shared_trigrams": 0.5, - "null_mean_shared_trigrams": 0.003, - "delta": 0.497, - "p_value_approx": 0.002 + "boundary_permute_null_mean": 0.5, + "boundary_permute_delta": 0.0, + "boundary_permute_p_upper": 1.0, + "boundary_permute_p_lower": 1.0, + "token_shuffle_null_mean": 0.001, + "token_shuffle_p_upper": 0.00498 }, "Letter": { "n_pairs_observed": 1, "obs_mean_shared_trigrams": 0.0, - "null_mean_shared_trigrams": 0.025, - "delta": -0.025, - "p_value_approx": 1.0 + "boundary_permute_null_mean": 0.55, + "boundary_permute_delta": -0.55, + "boundary_permute_p_upper": 1.0, + "boundary_permute_p_lower": 0.45274, + "token_shuffle_null_mean": 0.015, + "token_shuffle_p_upper": 1.0 } } } \ No newline at end of file diff --git a/outputs/probe_validation.json b/outputs/probe_validation.json new file mode 100644 index 0000000..149ae79 --- /dev/null +++ b/outputs/probe_validation.json @@ -0,0 +1,357 @@ +{ + "probes": { + "year_formula": { + "total_matches": 5088, + "true": 1485, + "false": 2888, + "unclear": 715, + "precision_excl_unclear": 0.34, + "buckets": [ + { + "verdict": "FALSE", + "label": "verbal prefix (mu-na-, mu-un-, mu-ni-, ...)", + "n": 2498 + }, + { + "verdict": "TRUE", + "label": "mu + royal/event name (year-name)", + "n": 1360 + }, + { + "verdict": "UNCLEAR", + "label": "unclassified by the discriminators", + "n": 715 + }, + { + "verdict": "FALSE", + "label": "line-break / structural-token artifact", + "n": 189 + }, + { + "verdict": "FALSE", + "label": "mu-bi / mu-ni = 'its/his name'", + "n": 141 + }, + { + "verdict": "TRUE", + "label": "mu us₂-sa + event (derived year-name)", + "n": 125 + }, + { + "verdict": "FALSE", + "label": "mu-kuₓ(DU) = delivery, a different lexeme", + "n": 60 + } + ], + "claimed_role": "temporal index (year-name)", + "regex": "\\bmu\\b(?:[\\s\\-][^\\n]{0,80})" + }, + "king_title": { + "total_matches": 2666, + "true": 750, + "false": 1046, + "unclear": 870, + "precision_excl_unclear": 0.418, + "buckets": [ + { + "verdict": "FALSE", + "label": "lugal- as element of a PERSONAL NAME", + "n": 1046 + }, + { + "verdict": "TRUE", + "label": "lugal + GN/epithet (king of X, mighty king)", + "n": 750 + }, + { + "verdict": "UNCLEAR", + "label": "bare 'lugal' (king, or 'owner/master' in admin context)", + "n": 447 + }, + { + "verdict": "UNCLEAR", + "label": "unclassified by the discriminators", + "n": 423 + } + ], + "claimed_role": "title: king", + "regex": "\\blugal\\b[^\\n]{0,40}" + }, + "god_dedication": { + "total_matches": 182, + "true": 10, + "false": 172, + "unclear": 0, + "precision_excl_unclear": 0.055, + "buckets": [ + { + "verdict": "FALSE", + "label": "nam-lugal = 'KINGSHIP', an abstract noun", + "n": 132 + }, + { + "verdict": "FALSE", + "label": "nam-en = 'en-ship/lordship', an abstract noun", + "n": 40 + }, + { + "verdict": "TRUE", + "label": "nam-ti = 'life' (used in 'for the life of' dedications)", + "n": 10 + } + ], + "claimed_role": "dedicatory formula", + "regex": "\\bnam-(ti|ki|en|lugal)\\b" + }, + "witness_eye": { + "total_matches": 92, + "true": 36, + "false": 51, + "unclear": 5, + "precision_excl_unclear": 0.414, + "buckets": [ + { + "verdict": "FALSE", + "label": "igi-ni/zu/bi/mu-še₃ = 'before him/you/it' (pronominal)", + "n": 38 + }, + { + "verdict": "TRUE", + "label": "igi + PN/DN + -še₃ (genuine witness clause)", + "n": 36 + }, + { + "verdict": "FALSE", + "label": "igi-nim = 'upper/northern', a different lexeme", + "n": 10 + }, + { + "verdict": "UNCLEAR", + "label": "unclassified by the discriminators", + "n": 5 + }, + { + "verdict": "FALSE", + "label": "igi + numeral = accounting expression, not a witness", + "n": 3 + } + ], + "claimed_role": "witness presence (before X)", + "regex": "\\bigi[\\s\\-][^\\n]{0,40}-š[èe]₃\\b" + }, + "son_of_PN": { + "total_matches": 1302, + "true": 1008, + "false": 58, + "unclear": 236, + "precision_excl_unclear": 0.946, + "buckets": [ + { + "verdict": "TRUE", + "label": "dumu + PN (filiation)", + "n": 1008 + }, + { + "verdict": "UNCLEAR", + "label": "unclassified by the discriminators", + "n": 163 + }, + { + "verdict": "UNCLEAR", + "label": "dumu-ni / dumu-munus etc. (possessed / compound)", + "n": 73 + }, + { + "verdict": "FALSE", + "label": "dumu-zi = Dumuzi (divine/personal name)", + "n": 58 + } + ], + "claimed_role": "filiation / lineage", + "regex": "\\bdumu\\b[\\s\\-][^\\n]{0,40}" + }, + "excess_diri": { + "total_matches": 62, + "true": 3, + "false": 13, + "unclear": 46, + "precision_excl_unclear": 0.188, + "buckets": [ + { + "verdict": "UNCLEAR", + "label": "unclassified by the discriminators", + "n": 24 + }, + { + "verdict": "UNCLEAR", + "label": "diri-ga / bare diri (verbal 'to exceed', or surplus)", + "n": 22 + }, + { + "verdict": "FALSE", + "label": "diri + month name = INTERCALARY MONTH, not a ledger surplus", + "n": 13 + }, + { + "verdict": "TRUE", + "label": "diri + quantity (surplus amount)", + "n": 3 + } + ], + "claimed_role": "excess / surplus", + "regex": "\\bdiri\\b[^\\n]{0,40}" + }, + "seal_of_PN": { + "total_matches": 301, + "true": 253, + "false": 11, + "unclear": 37, + "precision_excl_unclear": 0.958, + "buckets": [ + { + "verdict": "TRUE", + "label": "kišib₃ + PN/title (seal of so-and-so)", + "n": 253 + }, + { + "verdict": "UNCLEAR", + "label": "unclassified by the discriminators", + "n": 37 + }, + { + "verdict": "FALSE", + "label": "kišib₃-bi N = 'its sealed tablets: N' (a count, not an attribution)", + "n": 11 + } + ], + "claimed_role": "identity / authentication", + "regex": "\\bkišib(?:₃)?\\b[^\\n]{0,40}" + }, + "received_by": { + "total_matches": 120, + "true": 120, + "false": 0, + "unclear": 0, + "precision_excl_unclear": 1.0, + "buckets": [ + { + "verdict": "TRUE", + "label": "šu ba-ti = 'received' (fixed formula)", + "n": 120 + } + ], + "claimed_role": "transaction confirmation", + "regex": "\\bšu\\s+ba-ti\\b" + }, + "speak_to_him": { + "total_matches": 297, + "true": 297, + "false": 0, + "unclear": 0, + "precision_excl_unclear": 1.0, + "buckets": [ + { + "verdict": "TRUE", + "label": "u₃-na-a-du₁₁ = 'speak to him' (fixed formula)", + "n": 297 + } + ], + "claimed_role": "letter address formula", + "regex": "\\bu₃-na-a-du₁₁\\b" + }, + "total_audit": { + "total_matches": 22, + "true": 22, + "false": 0, + "unclear": 0, + "precision_excl_unclear": 1.0, + "buckets": [ + { + "verdict": "TRUE", + "label": "šu-nigin₂ = 'sum total'", + "n": 22 + } + ], + "claimed_role": "ledger total / audit line", + "regex": "\\bšu-nigin₂?\\b[^\\n]{0,40}" + } + }, + "prevalence": { + "seal_of_PN": { + "description": "seal attribution", + "by_genre": { + "Administrative": 25.4, + "Letter": 8.4, + "Lexical": 1.4, + "Literary": 3.6, + "Royal Inscription": 11.0 + } + }, + "year_formula (strict)": { + "description": "year-name dating", + "by_genre": { + "Administrative": 70.2, + "Letter": 45.4, + "Lexical": 2.9, + "Literary": 29.8, + "Royal Inscription": 41.8 + } + }, + "total_audit": { + "description": "periodic sum-total", + "by_genre": { + "Administrative": 2.2, + "Letter": 0.2, + "Lexical": 0.0, + "Literary": 1.2, + "Royal Inscription": 0.2 + } + }, + "deficit_la2ia3": { + "description": "named deficit", + "by_genre": { + "Administrative": 3.4, + "Letter": 1.8, + "Lexical": 1.4, + "Literary": 0.4, + "Royal Inscription": 0.8 + } + }, + "excess_diri": { + "description": "named excess", + "by_genre": { + "Administrative": 4.2, + "Letter": 0.4, + "Lexical": 0.0, + "Literary": 2.2, + "Royal Inscription": 3.4 + } + }, + "witness_eye (non-pronominal)": { + "description": "witness clause", + "by_genre": { + "Administrative": 0.4, + "Letter": 0.6, + "Lexical": 0.0, + "Literary": 3.0, + "Royal Inscription": 1.0 + } + }, + "received_by": { + "description": "receipt confirmation", + "by_genre": { + "Administrative": 15.2, + "Letter": 2.2, + "Lexical": 0.0, + "Literary": 0.8, + "Royal Inscription": 5.0 + } + } + }, + "never_fire": [ + { + "probe": "credit_mu_DU", + "regex": "\\bmu-DU\\b[^\\n]{0,40}" + } + ] +} \ No newline at end of file diff --git a/outputs/probe_validation.md b/outputs/probe_validation.md new file mode 100644 index 0000000..400e16c --- /dev/null +++ b/outputs/probe_validation.md @@ -0,0 +1,159 @@ +# Probe Validation — do the Phase 1 regexes mean what their labels say? + +Phase 1 reports how often each probe **matches**. This report asks whether those matches **mean** what the probe's label claims. Every headline frequency in the README rests on the answer. + +> **Epistemic status.** The discriminators below encode standard dictionary values applied by a **non-specialist**. They are deliberately conservative — a match is only counted FALSE where the surface form makes an alternative reading unambiguous. **These are lower bounds on the error rate, and are not a substitute for review by an Assyriologist.** See `REVIEWERS.md`. + +Precision = TRUE / (TRUE + FALSE); UNCLEAR is excluded and reported separately. + +## Summary + +| Probe | Claimed role | Matches | Precision | UNCLEAR | Verdict | +|---|---|---:|---:|---:|---| +| `year_formula` | temporal index (year-name) | 5,088 | 34% | 715 | **DO NOT CITE** | +| `king_title` | title: king | 2,666 | 42% | 870 | **DO NOT CITE** | +| `god_dedication` | dedicatory formula | 182 | 6% | 0 | **DO NOT CITE** | +| `witness_eye` | witness presence (before X) | 92 | 41% | 5 | **DO NOT CITE** | +| `son_of_PN` | filiation / lineage | 1,302 | 95% | 236 | usable with caveat | +| `excess_diri` | excess / surplus | 62 | 19% | 46 | **DO NOT CITE** | +| `seal_of_PN` | identity / authentication | 301 | 96% | 37 | **reliable** | +| `received_by` | transaction confirmation | 120 | 100% | 0 | **reliable** | +| `speak_to_him` | letter address formula | 297 | 100% | 0 | **reliable** | +| `total_audit` | ledger total / audit line | 22 | 100% | 0 | **reliable** | + +## Per-probe detail + +### `year_formula` — claimed role: temporal index (year-name) + +Regex: `\bmu\b(?:[\s\-][^\n]{0,80})` +Matches: 5,088 — TRUE 1,485 / FALSE 2,888 / UNCLEAR 715 + +| Verdict | Interpretation of the matched form | n | +|---|---|---:| +| FALSE | verbal prefix (mu-na-, mu-un-, mu-ni-, ...) | 2,498 | +| TRUE | mu + royal/event name (year-name) | 1,360 | +| UNCLEAR | unclassified by the discriminators | 715 | +| FALSE | line-break / structural-token artifact | 189 | +| FALSE | mu-bi / mu-ni = 'its/his name' | 141 | +| TRUE | mu us₂-sa + event (derived year-name) | 125 | +| FALSE | mu-kuₓ(DU) = delivery, a different lexeme | 60 | + +### `king_title` — claimed role: title: king + +Regex: `\blugal\b[^\n]{0,40}` +Matches: 2,666 — TRUE 750 / FALSE 1,046 / UNCLEAR 870 + +| Verdict | Interpretation of the matched form | n | +|---|---|---:| +| FALSE | lugal- as element of a PERSONAL NAME | 1,046 | +| TRUE | lugal + GN/epithet (king of X, mighty king) | 750 | +| UNCLEAR | bare 'lugal' (king, or 'owner/master' in admin context) | 447 | +| UNCLEAR | unclassified by the discriminators | 423 | + +### `god_dedication` — claimed role: dedicatory formula + +Regex: `\bnam-(ti|ki|en|lugal)\b` +Matches: 182 — TRUE 10 / FALSE 172 / UNCLEAR 0 + +| Verdict | Interpretation of the matched form | n | +|---|---|---:| +| FALSE | nam-lugal = 'KINGSHIP', an abstract noun | 132 | +| FALSE | nam-en = 'en-ship/lordship', an abstract noun | 40 | +| TRUE | nam-ti = 'life' (used in 'for the life of' dedications) | 10 | + +### `witness_eye` — claimed role: witness presence (before X) + +Regex: `\bigi[\s\-][^\n]{0,40}-š[èe]₃\b` +Matches: 92 — TRUE 36 / FALSE 51 / UNCLEAR 5 + +| Verdict | Interpretation of the matched form | n | +|---|---|---:| +| FALSE | igi-ni/zu/bi/mu-še₃ = 'before him/you/it' (pronominal) | 38 | +| TRUE | igi + PN/DN + -še₃ (genuine witness clause) | 36 | +| FALSE | igi-nim = 'upper/northern', a different lexeme | 10 | +| UNCLEAR | unclassified by the discriminators | 5 | +| FALSE | igi + numeral = accounting expression, not a witness | 3 | + +### `son_of_PN` — claimed role: filiation / lineage + +Regex: `\bdumu\b[\s\-][^\n]{0,40}` +Matches: 1,302 — TRUE 1,008 / FALSE 58 / UNCLEAR 236 + +| Verdict | Interpretation of the matched form | n | +|---|---|---:| +| TRUE | dumu + PN (filiation) | 1,008 | +| UNCLEAR | unclassified by the discriminators | 163 | +| UNCLEAR | dumu-ni / dumu-munus etc. (possessed / compound) | 73 | +| FALSE | dumu-zi = Dumuzi (divine/personal name) | 58 | + +### `excess_diri` — claimed role: excess / surplus + +Regex: `\bdiri\b[^\n]{0,40}` +Matches: 62 — TRUE 3 / FALSE 13 / UNCLEAR 46 + +| Verdict | Interpretation of the matched form | n | +|---|---|---:| +| UNCLEAR | unclassified by the discriminators | 24 | +| UNCLEAR | diri-ga / bare diri (verbal 'to exceed', or surplus) | 22 | +| FALSE | diri + month name = INTERCALARY MONTH, not a ledger surplus | 13 | +| TRUE | diri + quantity (surplus amount) | 3 | + +### `seal_of_PN` — claimed role: identity / authentication + +Regex: `\bkišib(?:₃)?\b[^\n]{0,40}` +Matches: 301 — TRUE 253 / FALSE 11 / UNCLEAR 37 + +| Verdict | Interpretation of the matched form | n | +|---|---|---:| +| TRUE | kišib₃ + PN/title (seal of so-and-so) | 253 | +| UNCLEAR | unclassified by the discriminators | 37 | +| FALSE | kišib₃-bi N = 'its sealed tablets: N' (a count, not an attribution) | 11 | + +### `received_by` — claimed role: transaction confirmation + +Regex: `\bšu\s+ba-ti\b` +Matches: 120 — TRUE 120 / FALSE 0 / UNCLEAR 0 + +| Verdict | Interpretation of the matched form | n | +|---|---|---:| +| TRUE | šu ba-ti = 'received' (fixed formula) | 120 | + +### `speak_to_him` — claimed role: letter address formula + +Regex: `\bu₃-na-a-du₁₁\b` +Matches: 297 — TRUE 297 / FALSE 0 / UNCLEAR 0 + +| Verdict | Interpretation of the matched form | n | +|---|---|---:| +| TRUE | u₃-na-a-du₁₁ = 'speak to him' (fixed formula) | 297 | + +### `total_audit` — claimed role: ledger total / audit line + +Regex: `\bšu-nigin₂?\b[^\n]{0,40}` +Matches: 22 — TRUE 22 / FALSE 0 / UNCLEAR 0 + +| Verdict | Interpretation of the matched form | n | +|---|---|---:| +| TRUE | šu-nigin₂ = 'sum total' | 22 | + +## Probes that never fire + +`phase1_templates.py` skips any probe with zero tablet hits (`if tablet_hits == 0: continue`), so a probe whose regex does not match the corpus's transliteration conventions disappears from `templates.json` without any warning. These probes produced **no matches at all**: + +- `credit_mu_DU` — regex `\bmu-DU\b[^\n]{0,40}` + +## Prevalence — attested is not the same as characteristic + +Percentage of tablets **in each genre** carrying each pattern. A pattern can be real, well-attested, and still be far too rare to describe a genre's normal practice — which is a different claim from the one a reader takes away from 'administrative tablets close with a sum-total'. + +| Pattern | Administrative | Letter | Lexical | Literary | Royal Inscription | +|---|---:|---:|---:|---:|---:| +| seal_of_PN | 25.4% | 8.4% | 1.4% | 3.6% | 11.0% | +| year_formula (strict) | 70.2% | 45.4% | 2.9% | 29.8% | 41.8% | +| total_audit | 2.2% | 0.2% | 0.0% | 1.2% | 0.2% | +| deficit_la2ia3 | 3.4% | 1.8% | 1.4% | 0.4% | 0.8% | +| excess_diri | 4.2% | 0.4% | 0.0% | 2.2% | 3.4% | +| witness_eye (non-pronominal) | 0.4% | 0.6% | 0.0% | 3.0% | 1.0% | +| received_by | 15.2% | 2.2% | 0.0% | 0.8% | 5.0% | + +**Read this table before citing any frequency from `templates.json`.** diff --git a/outputs/reference_architecture.md b/outputs/reference_architecture.md index c8c3909..a9b6e20 100644 --- a/outputs/reference_architecture.md +++ b/outputs/reference_architecture.md @@ -6,6 +6,17 @@ **Code shapes:** pseudocode given in Python and Rust. Both are illustrative — adapt to your runtime's existing identity / storage / event primitives. +> ⚠️ **CORRECTION NOTICE (2026-07-31).** This is a hand-authored design document written against +> the repository's original findings. **Two of those findings were subsequently withdrawn** after a +> self-audit — the RULING-parity result (wrong null hypothesis) and the Zipf-as-DSL result +> (stream-length artifact) — and **5 of 10 regex probes were found to match a different lexeme than +> their label claims.** Passages below that rest on the withdrawn results are marked inline. The +> document is preserved rather than rewritten so the record of what was claimed stays visible. +> **Read [`../CORRECTIONS.md`](../CORRECTIONS.md) and [`probe_validation.md`](probe_validation.md) +> before relying on anything here.** + + + --- ## 1. Reference Architecture @@ -58,7 +69,16 @@ Three transverse subsystems span the agents: | Lexical | 1.38 | 0.48 | 0.29 | +0.507 | 0.031 | | Letter | 2.14 | 0.12 | 0.02 | n/a | n/a | -The parity test shows that ``-bounded chunks share trigrams with their neighbors at rates 30–500× the shuffled baseline. **`` is provably a logical row separator, not a visual hint.** +~~The parity test shows that ``-bounded chunks share trigrams with their neighbors at rates 30–500× the shuffled baseline. **`` is provably a logical row separator, not a visual hint.**~~ + +> **WITHDRAWN (2026-07-31).** The 30–500× figure and the p-values in the table above came from a null +> that shuffled all of a tablet's tokens, destroying the local coherence any natural-language text has. +> Under a null that holds token order and chunk lengths fixed and permutes **only where the cuts fall**, +> the effect disappears in every genre (p = 0.78 Admin, 0.91 Literary, 0.27 Lexical, 1.00 Royal), with +> observed values *at or below* the null. **There is no statistical evidence that `` marks content +> boundaries.** The `` / `` / `` marker-density columns in the table are simple +> counts and remain valid; only the parity Δ and p columns are withdrawn. The three-tier memory design +> below is therefore an **untested design proposal**, not a corpus finding. See `../CORRECTIONS.md`. ### Architecture: three-tier context window @@ -277,7 +297,12 @@ Distinctive Administrative trigrams (Phase 1 log-odds): - `sila₃ kaš 2(diš)` (liter of beer, 2) — 63 - `sila₃ ninda 5(diš)` (liter of bread, 5) — 63 -All have the structure `() ` — the canonical typed transaction line. Phase 3 confirms this: Administrative has the second-highest structural-redundancy Δ (+0.088, second only to Royal Inscription at +0.099). It is a **DSL**, with reserved keywords. +All have the structure `() ` — the canonical typed transaction line. Administrative has the second-highest structural-redundancy Δ (+0.088, second only to Royal Inscription at +0.099), and that ranking does survive an equal-length control. + +> **Caveat (2026-07-31).** "It is a **DSL**, with reserved keywords" overstates what the statistic shows. +> The compression Δ establishes only that the real token stream is more redundant than the same tokens in +> random order. "DSL" is an analogy, not a result. The Zipf evidence originally cited for it has been +> withdrawn as a stream-length artifact. ### Design @@ -403,7 +428,13 @@ Filiation is append-only. Seals are revocable. Decrees supersede explicitly. Per ### 7.3 Templates as types -Phase 3 shows administrative tablets have Zipf s = 1.75 — they are a typed DSL, not free text. Memory-layer encoders can compress template-heavy genres (admin, royal) at much higher ratios than narrative if they're given the schema. The Phase-1 templates are exactly that schema. +~~Phase 3 shows administrative tablets have Zipf s = 1.75 — they are a typed DSL, not free text.~~ Memory-layer encoders can compress template-heavy genres (admin, royal) at higher ratios than narrative if they're given the schema. The Phase-1 templates are exactly that schema. + +> **WITHDRAWN (2026-07-31).** The s = 1.75 cross-genre comparison is a stream-length artifact: the genre +> streams differ ~60× in length, and at equal length every genre falls in 1.11–1.21. The estimator (OLS on +> log-log rank-frequency) is also biased, and its R² is not a goodness-of-fit test. The *compression* +> ranking that motivates the encoder idea does survive a length control, so the design suggestion stands on +> that basis — but not on Zipf. See `../CORRECTIONS.md`. ### 7.4 What does NOT translate diff --git a/outputs/summary.md b/outputs/summary.md index f96f127..b15da86 100644 --- a/outputs/summary.md +++ b/outputs/summary.md @@ -1,13 +1,24 @@ # Summary — Top 10 Ideas, Ranked by Novelty × Implementability +> ⚠️ **CORRECTION NOTICE (2026-07-31).** This is a hand-authored design document written against +> the repository's original findings. **Two of those findings were subsequently withdrawn** after a +> self-audit — the RULING-parity result (wrong null hypothesis) and the Zipf-as-DSL result +> (stream-length artifact) — and **5 of 10 regex probes were found to match a different lexeme than +> their label claims.** Passages below that rest on the withdrawn results are marked inline. The +> document is preserved rather than rewritten so the record of what was claimed stays visible. +> **Read [`../CORRECTIONS.md`](../CORRECTIONS.md) and [`probe_validation.md`](probe_validation.md) +> before relying on anything here.** + + + Scoring: each axis is 1–5. **Score** is the product (max 25). "Implementability" is for a typical modern agent runtime; "novelty" is relative to common multi-agent / LLM frameworks (LangGraph, AutoGen, CrewAI, MS Semantic Kernel). | # | Idea | Novelty | Impl. | Score | Source | |---|---|---:|---:|---:|---| | 1 | Universal sealed write envelope `(payload, seal, witnesses, period)` | 5 | 5 | **25** | §3, §5, §7.1 of `reference_architecture.md` | -| 2 | L1/L2/L3 tiered memory (SURFACE / COLUMN / RULING) | 4 | 5 | **20** | §2, Phase 3 RULING parity | +| 2 | L1/L2/L3 tiered memory (SURFACE / COLUMN / RULING) — **empirical basis WITHDRAWN, now an untested proposal** | 4 | 5 | ~~20~~ | §2; the Phase 3 RULING-parity result it rested on is withdrawn (`../CORRECTIONS.md`) | | 3 | Periodic signed audits (`šu-nigin₂`) for token/cost/tool ledgers | 5 | 4 | **20** | §5, probe `total_audit` | -| 4 | Templates-as-types DSL encoder for high-redundancy genres | 5 | 3 | **15** | Phase 3 Zipf s=1.75 (admin/royal) | +| 4 | Templates-as-types encoder for high-redundancy genres | 5 | 3 | **15** | ~~Phase 3 Zipf s=1.75~~ **withdrawn** — rests instead on the compression-Δ ranking, which survives a length control | | 5 | Year-name registry: relative refs that resolve + freeze at write-time | 4 | 4 | **16** | §6, probe `year_following` 74% admin | | 6 | Filiation + revocable seal graph for principal identity | 3 | 4 | **12** | §3, probe `dumu PN` 37% admin | | 7 | RoyalDecree-style policy registry with explicit supersession | 3 | 5 | **15** | §1 (RoyalDecreeAgent) | @@ -21,7 +32,14 @@ Scoring: each axis is 1–5. **Score** is the product (max 25). "Implementabilit ### 1. Universal sealed write envelope (Score 25) -**The pattern.** Every clay tablet ends with `kišib₃ PN` (seal of PN), `iti X` (month), `mu Y` (year), and an optional witness chain (`igi PN-šè`). No write is anonymous, undated, or unattributed. Tablets P101440 and P132611 are textbook examples. +~~**The pattern.** Every clay tablet ends with `kišib₃ PN` (seal of PN), `iti X` (month), `mu Y` (year), and an optional witness chain (`igi PN-šè`). No write is anonymous, undated, or unattributed. Tablets P101440 and P132611 are textbook examples.~~ + +> **CORRECTED (2026-07-31).** This overstates the corpus badly. **25.4%** of administrative tablets carry +> `kišib₃`; **70.2%** carry a year-name; genuine witness clauses appear on **0.4%**; and **23.2%** carry +> *none* of the three. "Every clay tablet" and "no write is anonymous" are false. P101440 is a genuine +> sealed example; P132611 is a **Letter** with no seal clause, and was miscited here. The real pattern — +> a quarter of administrative tablets sealed and ~70% year-dated — is still notable, and is what the +> README now claims. See `probe_validation.md` for per-genre prevalence. **The proposal.** Make this the **kernel-wide invariant for every state-changing call** in your agent runtime: @@ -45,7 +63,13 @@ struct WriteEnvelope { ### 2. Three-tier memory (SURFACE / COLUMN / RULING) (Score 20) -**The pattern.** Sumerian scribes treated tablets as a hierarchical document: physical surface (obverse/reverse) → logical column → atomic row. Phase 3 statistically confirms the row tier: adjacent ``-bounded chunks share trigrams 30–500× more than shuffled baseline (Royal Inscription p=0.002, Administrative p=0.005). +**The pattern.** Sumerian scribes treated tablets as a hierarchical document: physical surface (obverse/reverse) → logical column → atomic row. That three-level physical structure is a real feature of the artifact. + +> **WITHDRAWN (2026-07-31).** The sentence that followed — "Phase 3 statistically confirms the row tier: +> adjacent ``-bounded chunks share trigrams 30–500× more than shuffled baseline (Royal p=0.002, +> Administrative p=0.005)" — is withdrawn. It came from a null that destroys all local structure. Under a +> boundary-placement null the effect vanishes in every genre. The physical three-level layout is real; the +> claim that `` marks *content* boundaries is unsupported. See `../CORRECTIONS.md`. **The proposal.** Replace flat agent memory with a three-tier store. Reads return the *smallest enclosing tier* that satisfies the query, never the whole frame. diff --git a/outputs/templates.json b/outputs/templates.json index 296611c..93c9535 100644 --- a/outputs/templates.json +++ b/outputs/templates.json @@ -1,4 +1,25 @@ { + "_AUDIT": { + "notice": "PROBE PRECISION AUDIT (2026-07-31): several frequencies in this file are NOT citable.", + "read_first": "outputs/probe_validation.md", + "generated_by": "scripts/phase1b_probe_validation.py", + "do_not_cite": { + "god_dedication": "precision 6% - nam-lugal 'kingship' and nam-en 'lordship' are abstract nouns, not dedications", + "excess_diri": "precision 19% - diri before a month name marks an INTERCALARY MONTH, not a ledger surplus", + "year_formula": "precision 34% - '-' is a regex word boundary, so \\bmu\\b matches the verbal prefixes mu-na-/mu-un-/mu-ni-", + "witness_eye": "precision 41% - igi-ni/zu-se3 = 'before him/you' (pronominal); igi-nim = 'upper, north'", + "king_title": "precision 42% - mostly lugal- as a personal-name element; bare lugal in Ur III admin often means 'owner/master'" + }, + "precision_validated": { + "received_by": "100% - fixed formula su ba-ti", + "speak_to_him": "100% - fixed formula u3-na-a-du11", + "total_audit": "100% precise, but only 2.2% of Administrative tablets - rare, not characteristic", + "seal_of_PN": "96% - kisib3-bi N ('its sealed tablets: N') is the only notable contaminant", + "son_of_PN": "95% - dumu-zi (Dumuzi) is the main contaminant" + }, + "prevalence_warning": "Attested is not the same as characteristic. A probe can be 100% precise and still describe a rare practice: total_audit (šu-nigin₂) is precise but appears on only 2.2% of Administrative tablets. Check the prevalence table in probe_validation.md before calling any pattern typical.", + "withdrawn_claims": "See CORRECTIONS.md — the RULING-parity and Zipf-as-DSL findings were withdrawn." + }, "meta": { "sample_size": 2069, "long_form_size": 197, @@ -5874,7 +5895,9 @@ "P109384", "P339293", "P121460" - ] + ], + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "96% - kisib3-bi N ('its sealed tablets: N') is the only notable contaminant" }, { "genre": "Administrative", @@ -5908,7 +5931,9 @@ "P312651", "P010553", "P126804" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 34% - '-' is a regex word boundary, so \\bmu\\b matches the verbal prefixes mu-na-/mu-un-/mu-ni-" }, { "genre": "Administrative", @@ -5942,7 +5967,9 @@ "P121460", "P131461", "P320806" - ] + ], + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "100% - fixed formula su ba-ti" }, { "genre": "Administrative", @@ -5976,7 +6003,9 @@ "P235570", "P291904", "P382284" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 41% - igi-ni/zu-se3 = 'before him/you' (pronominal); igi-nim = 'upper, north'" }, { "genre": "Administrative", @@ -5993,7 +6022,9 @@ "P213574", "P217207", "P324939" - ] + ], + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "100% precise, but only 2.2% of Administrative tablets - rare, not characteristic" }, { "genre": "Administrative", @@ -6027,7 +6058,9 @@ "P133277", "P104729", "P209854" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 19% - diri before a month name marks an INTERCALARY MONTH, not a ledger surplus" }, { "genre": "Administrative", @@ -6061,7 +6094,9 @@ "P104323", "P109384", "P339293" - ] + ], + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "95% - dumu-zi (Dumuzi) is the main contaminant" }, { "genre": "Administrative", @@ -6146,7 +6181,9 @@ "P298642", "P104323", "P125309" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 42% - mostly lugal- as a personal-name element; bare lugal in Ur III admin often means 'owner/master'" }, { "genre": "Administrative", @@ -6177,7 +6214,9 @@ "example_tablet_ids": [ "P215195", "P215193" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 6% - nam-lugal 'kingship' and nam-en 'lordship' are abstract nouns, not dedications" }, { "genre": "Administrative", @@ -6211,7 +6250,9 @@ "P142198", "P145647", "P341912" - ] + ], + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "96% - kisib3-bi N ('its sealed tablets: N') is the only notable contaminant" }, { "genre": "Letter", @@ -6245,7 +6286,9 @@ "P105398", "P129985", "P379695" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 34% - '-' is a regex word boundary, so \\bmu\\b matches the verbal prefixes mu-na-/mu-un-/mu-ni-" }, { "genre": "Letter", @@ -6279,7 +6322,9 @@ "P145585", "P129739", "P101755" - ] + ], + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "100% - fixed formula su ba-ti" }, { "genre": "Letter", @@ -6313,7 +6358,9 @@ "P253760", "P145617", "P129983" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 41% - igi-ni/zu-se3 = 'before him/you' (pronominal); igi-nim = 'upper, north'" }, { "genre": "Letter", @@ -6326,7 +6373,9 @@ "matches_per_tablet": 0.0, "example_tablet_ids": [ "P391107" - ] + ], + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "100% precise, but only 2.2% of Administrative tablets - rare, not characteristic" }, { "genre": "Letter", @@ -6357,7 +6406,9 @@ "example_tablet_ids": [ "P312684", "P111576" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 19% - diri before a month name marks an INTERCALARY MONTH, not a ledger surplus" }, { "genre": "Letter", @@ -6391,7 +6442,9 @@ "P145711", "P450152", "P145650" - ] + ], + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "95% - dumu-zi (Dumuzi) is the main contaminant" }, { "genre": "Letter", @@ -6476,7 +6529,9 @@ "P450152", "P145754", "P209278" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 42% - mostly lugal- as a personal-name element; bare lugal in Ur III admin often means 'owner/master'" }, { "genre": "Letter", @@ -6510,7 +6565,9 @@ "P105398", "P145725", "P108048" - ] + ], + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "100% - fixed formula u3-na-a-du11" }, { "genre": "Letter", @@ -6523,7 +6580,9 @@ "matches_per_tablet": 0.0, "example_tablet_ids": [ "P517304" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 6% - nam-lugal 'kingship' and nam-en 'lordship' are abstract nouns, not dedications" }, { "genre": "Letter", @@ -6553,7 +6612,9 @@ "matches_per_tablet": 0.03, "example_tablet_ids": [ "P010604" - ] + ], + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "96% - kisib3-bi N ('its sealed tablets: N') is the only notable contaminant" }, { "genre": "Lexical", @@ -6570,7 +6631,9 @@ "P329601", "P410826", "P411793" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 34% - '-' is a regex word boundary, so \\bmu\\b matches the verbal prefixes mu-na-/mu-un-/mu-ni-" }, { "genre": "Lexical", @@ -6615,7 +6678,9 @@ "P010604", "P347132", "P010607" - ] + ], + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "95% - dumu-zi (Dumuzi) is the main contaminant" }, { "genre": "Lexical", @@ -6691,7 +6756,9 @@ "P427606", "P346832", "P409894" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 42% - mostly lugal- as a personal-name element; bare lugal in Ur III admin often means 'owner/master'" }, { "genre": "Lexical", @@ -6725,7 +6792,9 @@ "Q000386", "Q000484", "Q000782" - ] + ], + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "96% - kisib3-bi N ('its sealed tablets: N') is the only notable contaminant" }, { "genre": "Literary", @@ -6759,7 +6828,9 @@ "P306703", "P346507", "P346555" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 34% - '-' is a regex word boundary, so \\bmu\\b matches the verbal prefixes mu-na-/mu-un-/mu-ni-" }, { "genre": "Literary", @@ -6791,7 +6862,9 @@ "P235699", "P122404", "P102827" - ] + ], + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "100% - fixed formula su ba-ti" }, { "genre": "Literary", @@ -6825,7 +6898,9 @@ "Q000684", "Q000375", "P346154" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 41% - igi-ni/zu-se3 = 'before him/you' (pronominal); igi-nim = 'upper, north'" }, { "genre": "Literary", @@ -6842,7 +6917,9 @@ "P275052", "P256177", "P339898" - ] + ], + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "100% precise, but only 2.2% of Administrative tablets - rare, not characteristic" }, { "genre": "Literary", @@ -6873,7 +6950,9 @@ "P309438", "P346649", "P346246" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 19% - diri before a month name marks an INTERCALARY MONTH, not a ledger surplus" }, { "genre": "Literary", @@ -6905,7 +6984,9 @@ "P345805", "P343047", "P223427" - ] + ], + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "95% - dumu-zi (Dumuzi) is the main contaminant" }, { "genre": "Literary", @@ -6988,7 +7069,9 @@ "P201186", "P343047", "P223427" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 42% - mostly lugal- as a personal-name element; bare lugal in Ur III admin often means 'owner/master'" }, { "genre": "Literary", @@ -7036,7 +7119,9 @@ "Q000721", "Q000751", "Q000400" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 6% - nam-lugal 'kingship' and nam-en 'lordship' are abstract nouns, not dedications" }, { "genre": "Literary", @@ -7070,7 +7155,9 @@ "P119663", "P129220", "P128582" - ] + ], + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "96% - kisib3-bi N ('its sealed tablets: N') is the only notable contaminant" }, { "genre": "Royal Inscription", @@ -7104,7 +7191,9 @@ "P105240", "Q001618", "Q001986" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 34% - '-' is a regex word boundary, so \\bmu\\b matches the verbal prefixes mu-na-/mu-un-/mu-ni-" }, { "genre": "Royal Inscription", @@ -7138,7 +7227,9 @@ "P125814", "P320492", "P113744" - ] + ], + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "100% - fixed formula su ba-ti" }, { "genre": "Royal Inscription", @@ -7172,7 +7263,9 @@ "Q000842", "Q001397", "Q001713" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 41% - igi-ni/zu-se3 = 'before him/you' (pronominal); igi-nim = 'upper, north'" }, { "genre": "Royal Inscription", @@ -7185,7 +7278,9 @@ "matches_per_tablet": 0.0, "example_tablet_ids": [ "P112011" - ] + ], + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "100% precise, but only 2.2% of Administrative tablets - rare, not characteristic" }, { "genre": "Royal Inscription", @@ -7218,7 +7313,9 @@ "Q002125", "P106063", "P201200" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 19% - diri before a month name marks an INTERCALARY MONTH, not a ledger surplus" }, { "genre": "Royal Inscription", @@ -7252,7 +7349,9 @@ "P512123", "P104417", "P121639" - ] + ], + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "95% - dumu-zi (Dumuzi) is the main contaminant" }, { "genre": "Royal Inscription", @@ -7337,7 +7436,9 @@ "Q004439", "P105240", "Q004962" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 42% - mostly lugal- as a personal-name element; bare lugal in Ur III admin often means 'owner/master'" }, { "genre": "Royal Inscription", @@ -7388,7 +7489,9 @@ "Q002222", "Q002125", "Q006340" - ] + ], + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 6% - nam-lugal 'kingship' and nam-en 'lordship' are abstract nouns, not dedications" }, { "genre": "Royal Inscription", @@ -7424,7 +7527,9 @@ "P339293", "P121460" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "96% - kisib3-bi N ('its sealed tablets: N') is the only notable contaminant" }, { "genre": "Administrative", @@ -7458,7 +7563,9 @@ "P010553", "P126804" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 34% - '-' is a regex word boundary, so \\bmu\\b matches the verbal prefixes mu-na-/mu-un-/mu-ni-" }, { "genre": "Administrative", @@ -7492,7 +7599,9 @@ "P131461", "P320806" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "100% - fixed formula su ba-ti" }, { "genre": "Administrative", @@ -7526,7 +7635,9 @@ "P291904", "P382284" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 41% - igi-ni/zu-se3 = 'before him/you' (pronominal); igi-nim = 'upper, north'" }, { "genre": "Administrative", @@ -7543,7 +7654,9 @@ "P217207", "P324939" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "100% precise, but only 2.2% of Administrative tablets - rare, not characteristic" }, { "genre": "Administrative", @@ -7577,7 +7690,9 @@ "P104729", "P209854" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 19% - diri before a month name marks an INTERCALARY MONTH, not a ledger surplus" }, { "genre": "Administrative", @@ -7611,7 +7726,9 @@ "P109384", "P339293" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "95% - dumu-zi (Dumuzi) is the main contaminant" }, { "genre": "Administrative", @@ -7696,7 +7813,9 @@ "P104323", "P125309" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 42% - mostly lugal- as a personal-name element; bare lugal in Ur III admin often means 'owner/master'" }, { "genre": "Administrative", @@ -7727,7 +7846,9 @@ "P215195", "P215193" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 6% - nam-lugal 'kingship' and nam-en 'lordship' are abstract nouns, not dedications" }, { "genre": "Administrative", @@ -7761,7 +7882,9 @@ "P145647", "P341912" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "96% - kisib3-bi N ('its sealed tablets: N') is the only notable contaminant" }, { "genre": "Letter", @@ -7795,7 +7918,9 @@ "P129985", "P379695" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 34% - '-' is a regex word boundary, so \\bmu\\b matches the verbal prefixes mu-na-/mu-un-/mu-ni-" }, { "genre": "Letter", @@ -7829,7 +7954,9 @@ "P129739", "P101755" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "100% - fixed formula su ba-ti" }, { "genre": "Letter", @@ -7863,7 +7990,9 @@ "P145617", "P129983" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 41% - igi-ni/zu-se3 = 'before him/you' (pronominal); igi-nim = 'upper, north'" }, { "genre": "Letter", @@ -7876,7 +8005,9 @@ "example_tablet_ids": [ "P391107" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "100% precise, but only 2.2% of Administrative tablets - rare, not characteristic" }, { "genre": "Letter", @@ -7907,7 +8038,9 @@ "P312684", "P111576" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 19% - diri before a month name marks an INTERCALARY MONTH, not a ledger surplus" }, { "genre": "Letter", @@ -7941,7 +8074,9 @@ "P450152", "P145650" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "95% - dumu-zi (Dumuzi) is the main contaminant" }, { "genre": "Letter", @@ -8026,7 +8161,9 @@ "P145754", "P209278" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 42% - mostly lugal- as a personal-name element; bare lugal in Ur III admin often means 'owner/master'" }, { "genre": "Letter", @@ -8060,7 +8197,9 @@ "P145725", "P108048" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "100% - fixed formula u3-na-a-du11" }, { "genre": "Letter", @@ -8073,7 +8212,9 @@ "example_tablet_ids": [ "P517304" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 6% - nam-lugal 'kingship' and nam-en 'lordship' are abstract nouns, not dedications" }, { "genre": "Letter", @@ -8103,7 +8244,9 @@ "example_tablet_ids": [ "P010604" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "96% - kisib3-bi N ('its sealed tablets: N') is the only notable contaminant" }, { "genre": "Lexical", @@ -8120,7 +8263,9 @@ "P410826", "P411793" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 34% - '-' is a regex word boundary, so \\bmu\\b matches the verbal prefixes mu-na-/mu-un-/mu-ni-" }, { "genre": "Lexical", @@ -8165,7 +8310,9 @@ "P347132", "P010607" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "95% - dumu-zi (Dumuzi) is the main contaminant" }, { "genre": "Lexical", @@ -8241,7 +8388,9 @@ "P346832", "P409894" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 42% - mostly lugal- as a personal-name element; bare lugal in Ur III admin often means 'owner/master'" }, { "genre": "Lexical", @@ -8275,7 +8424,9 @@ "Q000484", "Q000782" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "96% - kisib3-bi N ('its sealed tablets: N') is the only notable contaminant" }, { "genre": "Literary", @@ -8309,7 +8460,9 @@ "P346507", "P346555" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 34% - '-' is a regex word boundary, so \\bmu\\b matches the verbal prefixes mu-na-/mu-un-/mu-ni-" }, { "genre": "Literary", @@ -8341,7 +8494,9 @@ "P122404", "P102827" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "100% - fixed formula su ba-ti" }, { "genre": "Literary", @@ -8375,7 +8530,9 @@ "Q000375", "P346154" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 41% - igi-ni/zu-se3 = 'before him/you' (pronominal); igi-nim = 'upper, north'" }, { "genre": "Literary", @@ -8392,7 +8549,9 @@ "P256177", "P339898" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "100% precise, but only 2.2% of Administrative tablets - rare, not characteristic" }, { "genre": "Literary", @@ -8423,7 +8582,9 @@ "P346649", "P346246" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 19% - diri before a month name marks an INTERCALARY MONTH, not a ledger surplus" }, { "genre": "Literary", @@ -8455,7 +8616,9 @@ "P343047", "P223427" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "95% - dumu-zi (Dumuzi) is the main contaminant" }, { "genre": "Literary", @@ -8538,7 +8701,9 @@ "P343047", "P223427" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 42% - mostly lugal- as a personal-name element; bare lugal in Ur III admin often means 'owner/master'" }, { "genre": "Literary", @@ -8586,7 +8751,9 @@ "Q000751", "Q000400" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 6% - nam-lugal 'kingship' and nam-en 'lordship' are abstract nouns, not dedications" }, { "genre": "Literary", @@ -8620,7 +8787,9 @@ "P129220", "P128582" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "96% - kisib3-bi N ('its sealed tablets: N') is the only notable contaminant" }, { "genre": "Royal Inscription", @@ -8654,7 +8823,9 @@ "Q001618", "Q001986" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 34% - '-' is a regex word boundary, so \\bmu\\b matches the verbal prefixes mu-na-/mu-un-/mu-ni-" }, { "genre": "Royal Inscription", @@ -8688,7 +8859,9 @@ "P320492", "P113744" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "100% - fixed formula su ba-ti" }, { "genre": "Royal Inscription", @@ -8722,7 +8895,9 @@ "Q001397", "Q001713" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 41% - igi-ni/zu-se3 = 'before him/you' (pronominal); igi-nim = 'upper, north'" }, { "genre": "Royal Inscription", @@ -8735,7 +8910,9 @@ "example_tablet_ids": [ "P112011" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "100% precise, but only 2.2% of Administrative tablets - rare, not characteristic" }, { "genre": "Royal Inscription", @@ -8768,7 +8945,9 @@ "P106063", "P201200" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 19% - diri before a month name marks an INTERCALARY MONTH, not a ledger surplus" }, { "genre": "Royal Inscription", @@ -8802,7 +8981,9 @@ "P104417", "P121639" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "PRECISION_VALIDATED", + "_audit_note": "95% - dumu-zi (Dumuzi) is the main contaminant" }, { "genre": "Royal Inscription", @@ -8887,7 +9068,9 @@ "P105240", "Q004962" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 42% - mostly lugal- as a personal-name element; bare lugal in Ur III admin often means 'owner/master'" }, { "genre": "Royal Inscription", @@ -8938,7 +9121,9 @@ "Q002125", "Q006340" ], - "source": "regex_probe" + "source": "regex_probe", + "_audit_verdict": "DO_NOT_CITE", + "_audit_note": "precision 6% - nam-lugal 'kingship' and nam-en 'lordship' are abstract nouns, not dedications" }, { "genre": "Royal Inscription", diff --git a/requirements.txt b/requirements.txt index 18d6cfc..e5d477c 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,5 +1,8 @@ -datasets>=4.0 -pandas>=2.0 -numpy>=2.0 -scipy>=1.13 -pyarrow>=15.0 +# Floors, not pins — the pipeline is version-tolerant within these ranges. +# Last verified end-to-end 2026-07-31 on Python 3.14.2 with the versions in brackets. +datasets>=4.0 # [5.0.1] +pandas>=2.0 # [3.0.5] +numpy>=2.0,<3 # [2.5.1] major-version capped: Generator stream stability is not + # guaranteed across NumPy majors (see README caveats) +scipy>=1.13 # [1.18.0] +pyarrow>=15.0 # [25.0.0] diff --git a/scripts/check_integrity.py b/scripts/check_integrity.py new file mode 100644 index 0000000..7b8d135 --- /dev/null +++ b/scripts/check_integrity.py @@ -0,0 +1,102 @@ +""" +Repo integrity check — run before publishing or tagging a release. + +Verifies the things a reader or reviewer will hit first: + 1. Every relative link and backticked file path in the docs resolves to a real file. + 2. Every claim marked WITHDRAWN in the README is also marked in the derived documents, + so a reader who lands on outputs/*.md directly cannot pick up a retracted claim. + 3. Every probe rated DO_NOT_CITE is stamped in templates.json. + 4. Reported corpus totals match the actual corpus. + +Exit code 0 = clean, 1 = at least one problem. Intended for CI and for a pre-publish check. +""" +from __future__ import annotations + +import json +import re +import sys +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +problems: list[str] = [] +checks = 0 + + +def ok(cond: bool, msg: str) -> None: + global checks + checks += 1 + if not cond: + problems.append(msg) + + +# ---- 1. links resolve ------------------------------------------------------- +LINK = re.compile(r"\[[^\]]*\]\(([^)#][^)]*)\)") +docs = list(ROOT.glob("*.md")) + list((ROOT / "outputs").glob("*.md")) + list((ROOT / "benchmarks").glob("*.md")) +for doc in docs: + text = doc.read_text(encoding="utf-8") + for target in LINK.findall(text): + if target.startswith(("http://", "https://", "mailto:")): + continue + resolved = (doc.parent / target.split("#")[0]).resolve() + ok(resolved.exists(), f"{doc.relative_to(ROOT)}: broken link -> {target}") + +# ---- 2. withdrawn claims are marked everywhere they appear ------------------- +WITHDRAWN_MARKERS = ("WITHDRAWN", "withdrawn", "CORRECTION", "CORRECTED", "DO NOT CITE", "DO_NOT_CITE") +# (file, a regex for the retracted assertion, human description) +RETRACTED = [ + ("outputs/reference_architecture.md", r"provably a logical row separator", "RULING-as-proven-row-separator"), + ("outputs/summary.md", r"statistically confirms the row tier", "RULING parity confirmation"), + ("outputs/summary.md", r"No write is anonymous, undated, or unattributed", "'no write is anonymous'"), + ("outputs/FULL_IDEAS.md", r"parity p=0\.002", "RULING parity p-value"), +] +for rel, pattern, desc in RETRACTED: + path = ROOT / rel + if not path.exists(): + ok(False, f"missing file referenced by integrity check: {rel}") + continue + text = path.read_text(encoding="utf-8") + m = re.search(pattern, text) + if m: + # The correction must be TIGHTLY coupled to the assertion. The window is deliberately + # narrow (300 chars before / 800 after) so that a document-level banner at the top of + # the file cannot satisfy the check for an assertion buried 60 lines further down -- + # an earlier, wider window made this check vacuous, which a negative control caught. + window = text[max(0, m.start() - 300): m.end() + 800] + ok(any(k in window for k in WITHDRAWN_MARKERS), + f"{rel}: retracted claim '{desc}' appears with no correction notice adjacent to it") + +# ---- 3. DO_NOT_CITE probes are stamped in templates.json -------------------- +tpath = ROOT / "outputs" / "templates.json" +if tpath.exists(): + t = json.loads(tpath.read_text(encoding="utf-8")) + ok("_AUDIT" in t, "templates.json: missing _AUDIT block") + dnc = set(t.get("_AUDIT", {}).get("do_not_cite", {})) + ok(bool(dnc), "templates.json: _AUDIT.do_not_cite is empty") + for row in t.get("templates", []) + t.get("probe_hits", []): + name = row.get("template_name") or row.get("probe") + if name in dnc: + ok(row.get("_audit_verdict") == "DO_NOT_CITE", + f"templates.json: row '{name}' is DO_NOT_CITE but carries no _audit_verdict") +else: + ok(False, "outputs/templates.json missing — run scripts/phase1_templates.py") + +# ---- 4. corpus totals match what the docs claim ----------------------------- +parquet = ROOT / "data" / "sumtablets.parquet" +if parquet.exists(): + import pandas as pd + df = pd.read_parquet(parquet, columns=["id"]) + n = len(df) + readme = (ROOT / "README.md").read_text(encoding="utf-8") + ok(f"{n:,}" in readme, f"README cites a tablet count that does not match the corpus ({n:,})") + ok(df["id"].duplicated().sum() == 0, "corpus contains duplicate tablet IDs") +else: + print("[skip] data/sumtablets.parquet not present — run scripts/phase0_sample.py for corpus checks") + +# ---- report ----------------------------------------------------------------- +print(f"[integrity] {checks} checks run") +if problems: + print(f"[integrity] {len(problems)} PROBLEM(S):") + for p in problems: + print(f" - {p}") + sys.exit(1) +print("[integrity] clean") diff --git a/scripts/phase1_templates.py b/scripts/phase1_templates.py index b26f95e..dfe43ea 100644 --- a/scripts/phase1_templates.py +++ b/scripts/phase1_templates.py @@ -33,6 +33,47 @@ # Hand-coded regex probes for known Sumerian bureaucratic primitives. # Each tuple: (template_name, regex_on_transliteration, semantic_role) +# +# KNOWN LIMITATIONS -- these have had NO specialist (Assyriological) review, and +# they produce the most-cited numbers in this repository. Audited 2026-07-31: +# +# king_title OVER-COUNTS BADLY. `\blugal\b` matches `lugal-...` as an element +# of a PERSONAL NAME. In the Administrative sample 54% of matches +# are personal names and only ~12% are plausibly the title "king", +# so the reported 48.6% is not a rate of royal titles. Left in place +# rather than silently patched: separating name from title needs +# disambiguation regex cannot do. Do not cite this figure. +# +# year_formula OVER-COUNTS MILDLY. `-` is a regex word boundary, so `\bmu\b` +# matches inside `mu-DU` (delivery), `mu-ni` ("its name"), and +# verbal prefixes. ~62% of matches are plausibly year formulae. +# Tablet-level impact is small: a stricter probe gives 70.6% vs +# the 74.2% reported, because a tablet with a spurious `mu-` +# usually carries a real year formula too. +# +# divine_name / temple_e2 / excess_diri carry the same class of risk and have +# not been quantified. `e₂` is "house/household" generally, not +# specifically "temple"; `diri` has senses beyond "excess". +# +# Spot-checked as exact: seal_of_PN (25.4% of Administrative), +# speak_to_him (58.8% of Letters). See CORRECTIONS.md. + +# Verdicts from scripts/phase1b_probe_validation.py, stamped onto the emitted rows so a +# consumer reading templates.json programmatically cannot pick up a discredited frequency +# without also seeing why it is discredited. +PROBE_AUDIT: dict[str, tuple[str, str]] = { + "god_dedication": ("DO_NOT_CITE", "precision 6% - nam-lugal 'kingship' and nam-en 'lordship' are abstract nouns, not dedications"), + "excess_diri": ("DO_NOT_CITE", "precision 19% - diri before a month name marks an INTERCALARY MONTH, not a ledger surplus"), + "year_formula": ("DO_NOT_CITE", "precision 34% - '-' is a regex word boundary, so \\bmu\\b matches the verbal prefixes mu-na-/mu-un-/mu-ni-"), + "witness_eye": ("DO_NOT_CITE", "precision 41% - igi-ni/zu-se3 = 'before him/you' (pronominal); igi-nim = 'upper, north'"), + "king_title": ("DO_NOT_CITE", "precision 42% - mostly lugal- as a personal-name element; bare lugal in Ur III admin often means 'owner/master'"), + "received_by": ("PRECISION_VALIDATED", "100% - fixed formula su ba-ti"), + "speak_to_him": ("PRECISION_VALIDATED", "100% - fixed formula u3-na-a-du11"), + "total_audit": ("PRECISION_VALIDATED", "100% precise, but only 2.2% of Administrative tablets - rare, not characteristic"), + "seal_of_PN": ("PRECISION_VALIDATED", "96% - kisib3-bi N ('its sealed tablets: N') is the only notable contaminant"), + "son_of_PN": ("PRECISION_VALIDATED", "95% - dumu-zi (Dumuzi) is the main contaminant"), + "credit_mu_DU": ("NEVER_FIRES", "regex searches mu-DU; the corpus writes mu-kux(DU) - 60 occurrences never matched"), +} PROBES: list[tuple[str, str, str]] = [ ("seal_of_PN", r"\bkišib(?:₃)?\b[^\n]{0,40}", "identity / authentication"), ("month_marker", r"\biti[\s\-][^\n]{0,30}", "temporal index (month)"), @@ -336,7 +377,27 @@ def main() -> None: "source": "structural_stats", }) + # Stamp the probe-precision audit onto every affected row, so a consumer reading this + # file programmatically cannot pick up a discredited frequency without seeing the verdict. + for row in probes + templates: + n = row.get("template_name") or row.get("probe") + if n in PROBE_AUDIT: + row["_audit_verdict"], row["_audit_note"] = PROBE_AUDIT[n] + payload = { + "_AUDIT": { + "notice": "PROBE PRECISION AUDIT (2026-07-31): several frequencies in this file are NOT citable.", + "read_first": "outputs/probe_validation.md", + "generated_by": "scripts/phase1b_probe_validation.py", + "do_not_cite": {k: v[1] for k, v in PROBE_AUDIT.items() if v[0] == "DO_NOT_CITE"}, + "precision_validated": {k: v[1] for k, v in PROBE_AUDIT.items() if v[0] == "PRECISION_VALIDATED"}, + "prevalence_warning": ( + "Attested is not the same as characteristic. A probe can be 100% precise and still describe " + "a rare practice: total_audit (šu-nigin₂) is precise but appears on only 2.2% of Administrative " + "tablets. Check the prevalence table in probe_validation.md before calling any pattern typical." + ), + "withdrawn_claims": "See CORRECTIONS.md — the RULING-parity and Zipf-as-DSL findings were withdrawn.", + }, "meta": { "sample_size": int(len(df)), "long_form_size": int(len(long_df)), diff --git a/scripts/phase1b_probe_validation.py b/scripts/phase1b_probe_validation.py new file mode 100644 index 0000000..1a35e57 --- /dev/null +++ b/scripts/phase1b_probe_validation.py @@ -0,0 +1,249 @@ +""" +Phase 1b: Precision audit of the Phase 1 regex probes → outputs/probe_validation.md + +Phase 1 reports how often each probe MATCHES. It never asks whether the matches mean +what the probe's label says. This script asks that, because every headline frequency in +the README rests on it. + +Method. For each audited probe, a set of hand-written discriminators partitions its +matches into buckets that are philologically distinguishable by surface form alone -- +e.g. `lugal-` followed by a name element is a personal name, `lugal` followed by a +place name is the title "king of X". Each bucket is labelled TRUE (consistent with the +probe's claimed semantic role), FALSE (demonstrably a different lexeme or construction), +or UNCLEAR. Reported precision is TRUE / (TRUE + FALSE), with UNCLEAR excluded and +counted separately. + +Epistemic status. The discriminators encode standard dictionary values (ePSD2/CDLI +conventions) applied by a NON-SPECIALIST. They are deliberately conservative: a bucket +is only marked FALSE where the surface form makes the alternative reading unambiguous +(`nam-lugal` = "kingship" cannot be a dedicatory formula; `igi-zu-še₃` = "before you" +cannot be a witness clause naming a person). This is a lower bound on the error rate, +not a substitute for specialist review. See REVIEWERS.md. + +Output: outputs/probe_validation.md + outputs/probe_validation.json +""" +from __future__ import annotations + +import json +import re +from pathlib import Path + +import pandas as pd + +ROOT = Path(__file__).resolve().parents[1] +DATA = ROOT / "data" +OUT = ROOT / "outputs" +OUT.mkdir(parents=True, exist_ok=True) + +# (probe, claimed_role, [(verdict, label, regex_on_the_MATCH_TEXT)]) +# Rules are applied in order; first match wins. Anything unmatched falls to the default. +AUDITS: list[tuple[str, str, str, list[tuple[str, str, str]], str]] = [ + ( + "year_formula", r"\bmu\b(?:[\s\-][^\n]{0,80})", "temporal index (year-name)", + [ + ("FALSE", "verbal prefix (mu-na-, mu-un-, mu-ni-, ...)", r"^mu-(na|un|ni|e|ra|da|ta|ši|a)"), + ("FALSE", "mu-kuₓ(DU) = delivery, a different lexeme", r"^mu-kuₓ"), + ("FALSE", "line-break / structural-token artifact", r"^mu\s*<"), + ("TRUE", "mu us₂-sa + event (derived year-name)", r"^mu\s+us₂-sa\b"), + ("TRUE", "mu + royal/event name (year-name)", r"^mu\s+\S"), + ("FALSE", "mu-bi / mu-ni = 'its/his name'", r"^mu-(bi|ni|zu|mu)\b"), + ], + "UNCLEAR", + ), + ( + "king_title", r"\blugal\b[^\n]{0,40}", "title: king", + [ + ("TRUE", "lugal + GN/epithet (king of X, mighty king)", r"^lugal\s+(uri|urim|kal-ga|kalag-ga|an[\s\-]ub-da|ki-en-gi|ma-da)"), + ("FALSE", "lugal- as element of a PERSONAL NAME", r"^lugal-(?!e₂-mah)\S*[a-z]"), + ("FALSE", "lugal-mu / lugal-a-ni = 'my/his master' (possessed)", r"^lugal-(mu|a-ni|ne)\b"), + ("UNCLEAR", "bare 'lugal' (king, or 'owner/master' in admin context)", r"^lugal\s*$|^lugal\b(?![\s\-]\S)"), + ], + "UNCLEAR", + ), + ( + "god_dedication", r"\bnam-(ti|ki|en|lugal)\b", "dedicatory formula", + [ + ("TRUE", "nam-ti = 'life' (used in 'for the life of' dedications)", r"^ti$"), + ("FALSE", "nam-lugal = 'KINGSHIP', an abstract noun", r"^lugal$"), + ("FALSE", "nam-en = 'en-ship/lordship', an abstract noun", r"^en$"), + ], + "UNCLEAR", + ), + ( + "witness_eye", r"\bigi[\s\-][^\n]{0,40}-š[èe]₃\b", "witness presence (before X)", + [ + ("FALSE", "igi-ni/zu/bi/mu-še₃ = 'before him/you/it' (pronominal)", r"^igi-(ni|ma|mu|zu|bi)-š[èe]₃"), + ("FALSE", "igi-nim = 'upper/northern', a different lexeme", r"^igi-nim"), + ("FALSE", "igi + numeral = accounting expression, not a witness", r"^igi\s+\d"), + ("TRUE", "igi + PN/DN + -še₃ (genuine witness clause)", r"^igi\s+\S"), + ], + "UNCLEAR", + ), + ( + "son_of_PN", r"\bdumu\b[\s\-][^\n]{0,40}", "filiation / lineage", + [ + ("FALSE", "dumu-zi = Dumuzi (divine/personal name)", r"^dumu-zi\b"), + ("UNCLEAR", "dumu-ni / dumu-munus etc. (possessed / compound)", r"^dumu-(ni|munus|gi|nita)\b"), + ("TRUE", "dumu + PN (filiation)", r"^dumu\s+\S"), + ], + "UNCLEAR", + ), + ( + "excess_diri", r"\bdiri\b[^\n]{0,40}", "excess / surplus", + [ + ("FALSE", "diri + month name = INTERCALARY MONTH, not a ledger surplus", r"^diri\s+(ezem|še-sag|iti|mah)"), + ("UNCLEAR", "diri-ga / bare diri (verbal 'to exceed', or surplus)", r"^diri(-ga)?\s*$|^diri\.\.\."), + ("TRUE", "diri + quantity (surplus amount)", r"^diri\s+\d"), + ], + "UNCLEAR", + ), + ( + "seal_of_PN", r"\bkišib(?:₃)?\b[^\n]{0,40}", "identity / authentication", + [ + ("FALSE", "kišib₃-bi N = 'its sealed tablets: N' (a count, not an attribution)", r"^kišib₃?-bi\b"), + ("TRUE", "kišib₃ + PN/title (seal of so-and-so)", r"^kišib₃?\s+\S"), + ], + "UNCLEAR", + ), + ( + "received_by", r"\bšu\s+ba-ti\b", "transaction confirmation", + [("TRUE", "šu ba-ti = 'received' (fixed formula)", r"^šu\s+ba-ti$")], + "UNCLEAR", + ), + ( + "speak_to_him", r"\bu₃-na-a-du₁₁\b", "letter address formula", + [("TRUE", "u₃-na-a-du₁₁ = 'speak to him' (fixed formula)", r"^u₃-na-a-du₁₁$")], + "UNCLEAR", + ), + ( + "total_audit", r"\bšu-nigin₂?\b[^\n]{0,40}", "ledger total / audit line", + [("TRUE", "šu-nigin₂ = 'sum total'", r"^šu-nigin₂?")], + "UNCLEAR", + ), +] + +# Probes whose match count is fine but whose TABLET-LEVEL RATE is the thing a reader +# will misread. Reported separately as prevalence, because "attested" != "characteristic". +PREVALENCE = [ + ("seal_of_PN", r"kišib(₃)?", "seal attribution"), + ("year_formula (strict)", r"\bmu\s+(?:us₂-sa\s+)?(?!<)\S", "year-name dating"), + ("total_audit", r"šu-nigin₂?", "periodic sum-total"), + ("deficit_la2ia3", r"la₂-ia₃", "named deficit"), + ("excess_diri", r"\bdiri\b", "named excess"), + ("witness_eye (non-pronominal)", r"\bigi\s+(?!(?:ni|ma|mu|zu|bi|nim)-)[^\n]{2,40}-š[èe]₃\b", "witness clause"), + ("received_by", r"šu\s+ba-ti", "receipt confirmation"), +] + + +def audit_probe(texts: list[str], pattern: str, rules, default: str) -> dict: + rx = re.compile(pattern) + buckets: dict[tuple[str, str], int] = {} + total = 0 + for t in texts: + for m in rx.findall(t): + s = " ".join(str(m).split()) + total += 1 + for verdict, label, rule in rules: + if re.search(rule, s): + buckets[(verdict, label)] = buckets.get((verdict, label), 0) + 1 + break + else: + buckets[(default, "unclassified by the discriminators")] = \ + buckets.get((default, "unclassified by the discriminators"), 0) + 1 + t_ = sum(v for (vd, _), v in buckets.items() if vd == "TRUE") + f_ = sum(v for (vd, _), v in buckets.items() if vd == "FALSE") + u_ = sum(v for (vd, _), v in buckets.items() if vd == "UNCLEAR") + precision = (t_ / (t_ + f_)) if (t_ + f_) else float("nan") + return {"total_matches": total, "true": t_, "false": f_, "unclear": u_, + "precision_excl_unclear": round(precision, 3) if precision == precision else None, + "buckets": [{"verdict": vd, "label": lb, "n": n} for (vd, lb), n in + sorted(buckets.items(), key=lambda kv: -kv[1])]} + + +def main() -> None: + df = pd.read_parquet(DATA / "sample_500.parquet") + texts = df["transliteration"].astype(str).tolist() + genres = sorted(df["genre_primary"].unique()) + + results = {} + for name, pattern, role, rules, default in AUDITS: + results[name] = audit_probe(texts, pattern, rules, default) | {"claimed_role": role, "regex": pattern} + print(f"[audit] {name:<16} precision={results[name]['precision_excl_unclear']}") + + prevalence = {} + for label, pattern, human in PREVALENCE: + rx = re.compile(pattern) + prevalence[label] = {"description": human, "by_genre": { + g: round(100 * sum(1 for t in df[df["genre_primary"] == g]["transliteration"].astype(str) if rx.search(t)) + / max((df["genre_primary"] == g).sum(), 1), 1) for g in genres}} + + # Probes that never fire are silently dropped by phase1 -- surface them. + from phase1_templates import PROBES + dead = [] + for pname, ppat, _role in PROBES: + rx = re.compile(ppat) + if not any(rx.search(t) for t in texts): + dead.append({"probe": pname, "regex": ppat}) + + md = ["# Probe Validation — do the Phase 1 regexes mean what their labels say?\n\n", + "Phase 1 reports how often each probe **matches**. This report asks whether those matches ", + "**mean** what the probe's label claims. Every headline frequency in the README rests on the answer.\n\n", + "> **Epistemic status.** The discriminators below encode standard dictionary values applied by a ", + "**non-specialist**. They are deliberately conservative — a match is only counted FALSE where the ", + "surface form makes an alternative reading unambiguous. **These are lower bounds on the error rate, ", + "and are not a substitute for review by an Assyriologist.** See `REVIEWERS.md`.\n\n", + "Precision = TRUE / (TRUE + FALSE); UNCLEAR is excluded and reported separately.\n\n", + "## Summary\n\n| Probe | Claimed role | Matches | Precision | UNCLEAR | Verdict |\n", + "|---|---|---:|---:|---:|---|\n"] + for name, r in results.items(): + p = r["precision_excl_unclear"] + if p is None: + v = "n/a" + elif p >= 0.95: + v = "**reliable**" + elif p >= 0.75: + v = "usable with caveat" + elif p >= 0.5: + v = "**weak**" + else: + v = "**DO NOT CITE**" + md.append(f"| `{name}` | {r['claimed_role']} | {r['total_matches']:,} | " + f"{'—' if p is None else f'{100*p:.0f}%'} | {r['unclear']:,} | {v} |\n") + + md.append("\n## Per-probe detail\n\n") + for name, r in results.items(): + md.append(f"### `{name}` — claimed role: {r['claimed_role']}\n\n") + md.append(f"Regex: `{r['regex']}` \nMatches: {r['total_matches']:,} — " + f"TRUE {r['true']:,} / FALSE {r['false']:,} / UNCLEAR {r['unclear']:,}\n\n") + md.append("| Verdict | Interpretation of the matched form | n |\n|---|---|---:|\n") + for b in r["buckets"]: + md.append(f"| {b['verdict']} | {b['label']} | {b['n']:,} |\n") + md.append("\n") + + if dead: + md.append("## Probes that never fire\n\n") + md.append("`phase1_templates.py` skips any probe with zero tablet hits (`if tablet_hits == 0: continue`), " + "so a probe whose regex does not match the corpus's transliteration conventions disappears from " + "`templates.json` without any warning. These probes produced **no matches at all**:\n\n") + for d in dead: + md.append(f"- `{d['probe']}` — regex `{d['regex']}`\n") + md.append("\n") + + md.append("## Prevalence — attested is not the same as characteristic\n\n") + md.append("Percentage of tablets **in each genre** carrying each pattern. A pattern can be real, " + "well-attested, and still be far too rare to describe a genre's normal practice — which is a " + "different claim from the one a reader takes away from 'administrative tablets close with a sum-total'.\n\n") + md.append("| Pattern | " + " | ".join(genres) + " |\n|---|" + "---:|" * len(genres) + "\n") + for label, r in prevalence.items(): + md.append(f"| {label} | " + " | ".join(f"{r['by_genre'][g]}%" for g in genres) + " |\n") + md.append("\n**Read this table before citing any frequency from `templates.json`.**\n") + + (OUT / "probe_validation.md").write_text("".join(md), encoding="utf-8") + (OUT / "probe_validation.json").write_text( + json.dumps({"probes": results, "prevalence": prevalence, "never_fire": dead}, indent=2, ensure_ascii=False), + encoding="utf-8") + print(f"[save] {OUT / 'probe_validation.md'}") + + +if __name__ == "__main__": + main() diff --git a/scripts/phase3_compression.py b/scripts/phase3_compression.py index 128b8d6..294073d 100644 --- a/scripts/phase3_compression.py +++ b/scripts/phase3_compression.py @@ -34,6 +34,9 @@ SKIPS = list(range(2, 101)) RNG = np.random.default_rng(42) +# Minimum adjacent-chunk pairs before a RULING-parity p-value is reportable. +MIN_PARITY_PAIRS = 5 + def primes_upto(n: int) -> set[int]: sieve = [True] * (n + 1) @@ -84,6 +87,54 @@ def zipf_fit(freqs: list[int]) -> tuple[float, float, int]: return float(-slope), float(r2), len(freqs) +def hill_alpha(freqs: list[int], xmin: int = 1) -> float: + """Discrete-MLE (Hill) exponent for the frequency distribution. + + Reported alongside the OLS fit because OLS on log-log rank-frequency data is a + biased estimator of a power-law exponent, and R^2 from that fit is not a + goodness-of-fit test (Clauset, Shalizi & Newman 2009, SIAM Review 51(4)). + Divergence between the two is the signal that the OLS number is not trustworthy. + """ + x = np.array([f for f in freqs if f >= xmin], dtype=float) + if x.size < 10: + return float("nan") + return float(1 + x.size / np.sum(np.log(x / (xmin - 0.5)))) + + +def zipf_length_control(streams: dict[str, list[str]], n_draws: int = 25) -> list[dict]: + """Re-fit every genre at a COMMON stream length. + + Zipf exponents estimated by OLS are strongly sample-size dependent, and the + per-genre streams here differ by ~60x (Lexical 2.5k tokens vs Literary 154k). + Any cross-genre comparison at native lengths therefore confounds genre with + corpus size. This re-fits all genres on equal-length contiguous blocks so the + comparison is actually about genre. + + Uses its own generator so adding this control does not perturb the RNG stream + consumed by the compression / ELS / parity analyses. + """ + rng = np.random.default_rng(4242) + n = min(len(v) for v in streams.values()) + rows = [] + for g, st in streams.items(): + full_s, _, _ = zipf_fit(sorted(Counter(st).values(), reverse=True)) + draws = [] + for _ in range(n_draws): + start = 0 if len(st) <= n else int(rng.integers(0, len(st) - n)) + block = st[start:start + n] + draws.append(zipf_fit(sorted(Counter(block).values(), reverse=True))[0]) + rows.append({ + "genre": g, + "stream_length": len(st), + "s_at_native_length": round(full_s, 3), + "s_at_common_length": round(float(np.mean(draws)), 3), + "s_at_common_length_sd": round(float(np.std(draws)), 3), + "common_length_tokens": n, + "mle_alpha": round(hill_alpha(sorted(Counter(st).values(), reverse=True)), 3), + }) + return rows + + def compression_ratio(stream: list[str]) -> float: """zlib ratio on space-joined token stream. Lower = more redundant.""" if not stream: @@ -105,6 +156,38 @@ def shuffled_compression_ratio(stream: list[str], n_shuffles: int = 20) -> float return float(np.mean(ratios)) +def compression_length_control(streams: dict[str, list[str]], n_draws: int = 20) -> list[dict]: + """Re-measure the compression delta at a COMMON stream length. + + Same confound that invalidates the cross-genre Zipf comparison applies in + principle here, so it is tested rather than assumed. Blocks are CONTIGUOUS: + sampling scattered positions would itself destroy the token adjacency that + zlib exploits, which would manufacture a collapse rather than detect one. + + Uses its own generator so it does not perturb the main RNG stream. + """ + rng = np.random.default_rng(2718) + n = min(len(v) for v in streams.values()) + rows = [] + for g, st in streams.items(): + deltas = [] + for _ in range(n_draws): + start = 0 if len(st) <= n else int(rng.integers(0, len(st) - n)) + block = st[start:start + n] + raw = compression_ratio(block) + arr = np.array(block, dtype=object) + shuf = float(np.mean([compression_ratio(arr[rng.permutation(len(arr))].tolist()) + for _ in range(10)])) + deltas.append(shuf - raw) + rows.append({ + "genre": g, + "delta_at_common_length": round(float(np.mean(deltas)), 4), + "delta_at_common_length_sd": round(float(np.std(deltas)), 4), + "common_length_tokens": n, + }) + return rows + + def els_metric(stream: list[str], skip: int, min_repeat: int = 3) -> int: """Number of distinct tokens appearing >= min_repeat times in the skip-decimated stream.""" decimated = stream[::skip] @@ -131,8 +214,10 @@ def els_analysis(stream: list[str], n_shuffles: int = N_SHUFFLES, skips=SKIPS) - null_mean = float(null_vals.mean()) null_std = float(null_vals.std(ddof=1)) if null_vals.std(ddof=1) > 0 else 1e-9 z = (observed - null_mean) / null_std - # one-sided upper p-value - p = float((null_vals >= observed).sum() / n_shuffles) + # One-sided upper permutation p-value with the standard (r+1)/(n+1) correction, + # so p is never exactly 0 (an un-earned "infinitely significant" result). + # NOTE: this floors p at 1/(n_shuffles+1). See the resolution caveat in the report. + p = float((int((null_vals >= observed).sum()) + 1) / (n_shuffles + 1)) rows.append({ "skip": k, "is_prime": k in PRIMES, @@ -145,47 +230,87 @@ def els_analysis(stream: list[str], n_shuffles: int = N_SHUFFLES, skips=SKIPS) - return rows +def _shared_trigrams(a: list[str], b: list[str]) -> int: + return len(set(zip(a, a[1:], a[2:])) & set(zip(b, b[1:], b[2:]))) + + +def _cut(stream: list[str], lengths: list[int]) -> list[list[str]]: + out, cur = [], 0 + for L in lengths: + out.append(stream[cur:cur + L]) + cur += L + return out + + def ruling_parity(df: pd.DataFrame, genre: str, n_shuffles: int = 200) -> dict | None: - """For each tablet with >=2 RULING-delimited chunks, measure shared trigrams across - ADJACENT chunks vs a shuffled baseline (shuffle tokens within tablet, re-chunk same way).""" + """Do boundaries fall at content boundaries, or anywhere? + + Statistic: mean shared trigrams between ADJACENT ruling-delimited chunks. + + Two nulls, because the choice of null is the entire experiment: + + token_shuffle — pool the tablet's tokens, shuffle, re-cut at the same lengths. + Destroys ALL local structure, so it tests "is this text + locally coherent at all", which is true of any natural + language. It says nothing about specifically. + Retained only to show why it is the wrong control. + + boundary_permute — keep the real token ORDER and the exact multiset of chunk + LENGTHS; permute only WHERE the cuts fall. The only thing + that varies is boundary placement, so this isolates the + actual question. THIS IS THE PRIMARY TEST. + + p-values are permutation tests on the null distribution OF THE MEAN (one null + mean per shuffle), with the standard (r+1)/(n+1) correction so p is never 0. + """ sub = df[df["genre_primary"] == genre] - obs_shared = [] - null_shared = [] + obs_shared: list[int] = [] + tok_means: list[list[int]] = [[] for _ in range(n_shuffles)] + bnd_means: list[list[int]] = [[] for _ in range(n_shuffles)] + for _, row in sub.iterrows(): - chunks = re.split(r"", str(row["glyph_names"])) - chunks = [tokenize_glyph_names(c) for c in chunks] + chunks = [tokenize_glyph_names(c) for c in re.split(r"", str(row["glyph_names"]))] chunks = [c for c in chunks if len(c) >= 4] if len(chunks) < 2: continue - for a, b in zip(chunks[:-1], chunks[1:]): - tri_a = set(zip(a, a[1:], a[2:])) - tri_b = set(zip(b, b[1:], b[2:])) - obs_shared.append(len(tri_a & tri_b)) - # shuffled baseline: pool tokens, shuffle, re-chunk at same lengths + obs_shared.extend(_shared_trigrams(a, b) for a, b in zip(chunks[:-1], chunks[1:])) + pooled = [tok for c in chunks for tok in c] lengths = [len(c) for c in chunks] - for _ in range(n_shuffles): - shuf = RNG.permutation(pooled).tolist() - new_chunks = [] - cur = 0 - for L in lengths: - new_chunks.append(shuf[cur:cur + L]) - cur += L - for a, b in zip(new_chunks[:-1], new_chunks[1:]): - tri_a = set(zip(a, a[1:], a[2:])) - tri_b = set(zip(b, b[1:], b[2:])) - null_shared.append(len(tri_a & tri_b)) + for s in range(n_shuffles): + shuffled = _cut(RNG.permutation(pooled).tolist(), lengths) + tok_means[s].extend(_shared_trigrams(a, b) for a, b in zip(shuffled[:-1], shuffled[1:])) + perm = RNG.permutation(len(lengths)) + moved = _cut(pooled, [lengths[i] for i in perm]) + bnd_means[s].extend(_shared_trigrams(a, b) for a, b in zip(moved[:-1], moved[1:])) + if not obs_shared: return None - obs = np.array(obs_shared) - nul = np.array(null_shared) + + obs_mean = float(np.mean(obs_shared)) + + def _p(per_shuffle: list[list[int]]) -> tuple[float, float, float]: + means = np.array([float(np.mean(m)) for m in per_shuffle if m]) + if means.size == 0: + return float("nan"), float("nan"), float("nan") + upper = (int((means >= obs_mean).sum()) + 1) / (means.size + 1) + lower = (int((means <= obs_mean).sum()) + 1) / (means.size + 1) + return float(means.mean()), float(upper), float(lower) + + tok_null, tok_p, _ = _p(tok_means) + bnd_null, bnd_p_up, bnd_p_lo = _p(bnd_means) + return { - "n_pairs_observed": int(len(obs)), - "obs_mean_shared_trigrams": round(float(obs.mean()), 3), - "null_mean_shared_trigrams": round(float(nul.mean()), 3), - "delta": round(float(obs.mean() - nul.mean()), 3), - # p-value: fraction of null samples with mean >= obs.mean() - "p_value_approx": round(float((nul >= obs.mean()).sum() / max(len(nul), 1)), 5), + "n_pairs_observed": len(obs_shared), + "obs_mean_shared_trigrams": round(obs_mean, 3), + # PRIMARY: boundary placement is the only thing varied. + "boundary_permute_null_mean": round(bnd_null, 3), + "boundary_permute_delta": round(obs_mean - bnd_null, 3), + "boundary_permute_p_upper": round(bnd_p_up, 5), + "boundary_permute_p_lower": round(bnd_p_lo, 5), + # SECONDARY, for contrast only: this null destroys all local structure. + "token_shuffle_null_mean": round(tok_null, 3), + "token_shuffle_p_upper": round(tok_p, 5), } @@ -206,7 +331,8 @@ def main() -> None: s, r2, n = zipf_fit(counts) zipf_rows.append({"genre": g, "zipf_exponent_s": round(s, 3), "r_squared": round(r2, 4), "n_distinct_tokens": n, "stream_length": len(st)}) - print("[done] Zipf") + zipf_control = zipf_length_control(streams) + print("[done] Zipf (+ equal-length control)") # 2. Compression ratio comp_rows = [] @@ -220,7 +346,8 @@ def main() -> None: "structural_redundancy_delta": round(shuf - raw, 4), # positive = real stream more compressible "stream_length_tokens": len(st), }) - print("[done] compression") + comp_control = compression_length_control(streams) + print("[done] compression (+ equal-length control)") # 3. ELS els_rows_by_genre = {} @@ -249,7 +376,20 @@ def main() -> None: md_lines.append("|---|---:|---:|---:|---:|\n") for r in zipf_rows: md_lines.append(f"| {r['genre']} | {r['stream_length']:,} | {r['n_distinct_tokens']:,} | {r['zipf_exponent_s']} | {r['r_squared']} |\n") - md_lines.append("\n**Interpretation:** Genres with s substantially > 1 have a *small, high-reuse core vocabulary* — they behave like a DSL with reserved keywords, not a natural language. Administrative and Royal Inscription are the strongest candidates; Literary should be closer to natural-language s ≈ 1.0.\n") + md_lines.append("\n### Length control — the cross-genre comparison does not survive it\n") + md_lines.append("Zipf exponents fitted this way are strongly sample-size dependent, and these streams differ by ~60× in length. Re-fitting every genre on equal-length contiguous blocks separates genre from corpus size:\n\n") + md_lines.append(f"| Genre | Stream length | s at native length | s at common length ({zipf_control[0]['common_length_tokens']:,} tokens) | sd | MLE α |\n") + md_lines.append("|---|---:|---:|---:|---:|---:|\n") + for r in zipf_control: + md_lines.append(f"| {r['genre']} | {r['stream_length']:,} | {r['s_at_native_length']} | **{r['s_at_common_length']}** | {r['s_at_common_length_sd']} | {r['mle_alpha']} |\n") + spread = max(r["s_at_common_length"] for r in zipf_control) - min(r["s_at_common_length"] for r in zipf_control) + md_lines.append(f"\nAt a common length every genre lands in a band of width {spread:.3f}, against a native-length spread of " + f"{max(r['s_at_native_length'] for r in zipf_control) - min(r['s_at_native_length'] for r in zipf_control):.3f}. " + "The apparent genre difference was overwhelmingly a corpus-size effect: the genre with the shortest stream (Lexical) merely had the least opportunity to accumulate a long low-frequency tail.\n") + md_lines.append("\n**Correction — the 'Zipf-as-DSL detector' claim is withdrawn.** Earlier versions read the native-length spread (Administrative s=1.746 and Royal s=1.737 versus Lexical s=1.114) as evidence that administrative and royal genres behave like domain-specific languages while lexical lists resemble natural language. Under the length control that difference disappears. Two further problems compound it:\n\n") + md_lines.append("1. **The estimator is unreliable.** `zipf_fit` is OLS on log-log rank-frequency data — a biased power-law estimator, and its R² is not a goodness-of-fit test (Clauset, Shalizi & Newman 2009). The MLE column disagrees with the OLS column in both magnitude and *rank order*, which is exactly the symptom that diagnoses the OLS number as untrustworthy.\n") + md_lines.append("2. **The result is sensitive to arbitrary preprocessing.** Dropping hapax legomena (34–59% of types, depending on genre) moves the exponents by up to 0.17 and reverses the direction of some genre comparisons.\n\n") + md_lines.append("The R² values of 0.92–0.95 across every genre should not be read as support: comparably high R² is routine for lognormal and exponential data under this fitting procedure. Doing this properly would need MLE fitting with a fitted x_min, a Kolmogorov–Smirnov goodness-of-fit statistic, and likelihood-ratio tests against lognormal alternatives — none of which is done here.\n") md_lines.append("\n---\n\n") # 2. Compression @@ -259,13 +399,28 @@ def main() -> None: md_lines.append("|---|---:|---:|---:|---:|\n") for r in comp_rows: md_lines.append(f"| {r['genre']} | {r['raw_compression_ratio']} | {r['shuffled_baseline_ratio']} | **{r['structural_redundancy_delta']}** | {r['stream_length_tokens']:,} |\n") - md_lines.append("\n**Interpretation:** A positive Δ means the actual stream has structural regularities (templates, fixed formulas) beyond what random token order would predict. This is the corpus's 'error-correction overhead' in information-theoretic terms.\n") + md_lines.append("\n**Interpretation:** A positive Δ means the actual stream has structural regularities (templates, fixed formulas) beyond what random token order would predict.\n") + md_lines.append("\n### Length control — this comparison DOES survive it\n") + md_lines.append("The same length confound that invalidates the cross-genre Zipf comparison (§1) was tested here rather than assumed. Blocks are contiguous, since sampling scattered positions would itself destroy the token adjacency zlib exploits and would manufacture a collapse rather than detect one:\n\n") + md_lines.append(f"| Genre | Δ at native length | Δ at common length ({comp_control[0]['common_length_tokens']:,} tokens) | sd |\n") + md_lines.append("|---|---:|---:|---:|\n") + _cd = {r["genre"]: r for r in comp_control} + for r in comp_rows: + c = _cd[r["genre"]] + md_lines.append(f"| {r['genre']} | {r['structural_redundancy_delta']} | **{c['delta_at_common_length']}** | {c['delta_at_common_length_sd']} |\n") + _rn = [r["genre"] for r in sorted(comp_rows, key=lambda x: -x["structural_redundancy_delta"])] + _rc = [r["genre"] for r in sorted(comp_control, key=lambda x: -x["delta_at_common_length"])] + md_lines.append(f"\nRanking at native length: {' > '.join(_rn)}.\n") + md_lines.append(f"\nRanking at common length: {' > '.join(_rc)}.\n") + md_lines.append("\nThe ordering is preserved and magnitudes stay the same order of size, so unlike the Zipf comparison this one is not an artifact of differing stream lengths; genres that swap rank do so within overlapping standard deviations. **This is the strongest surviving positive statistical result in this repository** — though it remains a descriptive contrast against random token order, not a test of any specific structural hypothesis.\n") md_lines.append("\n---\n\n") # 3. ELS md_lines.append("## 3. Equidistant Letter Sequence (ELS) Scan, Skips 2–100\n") md_lines.append("For each skip k, we decimate the genre stream (take every k-th token) and count how many distinct tokens appear ≥3× in the decimation. Null distribution: 1000 random token permutations, same decimation. z = (observed − null_mean) / null_std.\n\n") md_lines.append("**Significance criterion:** Bonferroni-corrected p < 0.01/99 ≈ 0.000101 per (genre, skip). This is deliberately strict because with 99 skips × 5 genres = 495 tests, spurious hits are expected.\n\n") + md_lines.append(f"**Resolution caveat — state this before citing the null.** With {N_SHUFFLES} permutations the smallest attainable p-value is 1/({N_SHUFFLES}+1) ≈ {1/(N_SHUFFLES+1):.5f}, which is about 10× *larger* than the Bonferroni threshold above. This test therefore **cannot** return a Bonferroni-significant result no matter what the data look like; ~9,900 permutations would be needed. The '0 of 495' headline is consequently guaranteed by construction and should not be read as a powered rejection.\n\n") + md_lines.append("The result that *is* informative is the uncorrected one: across 495 tests at a nominal p < 0.01 you would expect ≈5 hits by chance, and that is roughly what appears. No skip, prime or otherwise, stands out above chance expectation. That is genuine evidence against hidden periodic encoding — it is simply weaker, and differently framed, than a Bonferroni claim.\n\n") any_significant = False for g, rows in els_rows_by_genre.items(): @@ -301,16 +456,32 @@ def main() -> None: md_lines.append("\n---\n\n") # 4. RULING parity - md_lines.append("## 4. Cross-RULING Parity (Shared Trigrams Across Adjacent Chunks)\n") - md_lines.append("For tablets with ≥2 RULING-delimited chunks, we measure trigram overlap between adjacent chunks, and compare to a within-tablet shuffled baseline (pool tokens, reshuffle, re-chunk at same lengths). High Δ ⇒ RULINGs separate sections that share structured content (e.g., parallel entries in a list).\n\n") - md_lines.append("| Genre | Adjacent-chunk pairs | Obs. shared trigrams | Null mean | Δ | p (approx) |\n") - md_lines.append("|---|---:|---:|---:|---:|---:|\n") + md_lines.append("## 4. Cross-RULING Parity — NULL RESULT (supersedes an earlier positive claim)\n") + md_lines.append("**Question:** do `` marks fall at content boundaries, or could you cut the tablet anywhere and see the same thing?\n\n") + md_lines.append("**Statistic:** mean shared trigrams between adjacent ruling-delimited chunks. The choice of null *is* the entire experiment:\n\n") + md_lines.append("- **`boundary_permute` (primary).** Keep the real token order and the exact multiset of chunk lengths; permute only *where* the cuts fall. Boundary placement is the only thing that varies, so this isolates the actual question.\n") + md_lines.append("- **`token_shuffle` (secondary, shown for contrast).** Pool the tablet's tokens, shuffle, re-cut at the same lengths. This destroys *all* local structure, so it tests \"is this text locally coherent at all\" — true of any natural language, and uninformative about ``.\n\n") + md_lines.append("| Genre | Pairs | Observed | Boundary-permute null | Δ | p (primary) | Token-shuffle null | p (secondary) |\n") + md_lines.append("|---|---:|---:|---:|---:|---:|---:|---:|\n") for g, r in parity_rows.items(): if r is None: - md_lines.append(f"| {g} | 0 | — | — | — | n/a (too few RULINGs in sample) |\n") + md_lines.append(f"| {g} | 0 | — | — | — | n/a (too few RULINGs) | — | — |\n") + continue + n = r["n_pairs_observed"] + if n < MIN_PARITY_PAIRS: + p_pri = f"n/a (n={n})" + p_sec = f"n/a (n={n})" else: - md_lines.append(f"| {g} | {r['n_pairs_observed']} | {r['obs_mean_shared_trigrams']} | {r['null_mean_shared_trigrams']} | **{r['delta']}** | {r['p_value_approx']} |\n") - md_lines.append("\n**Interpretation:** Positive Δ in Literary or Lexical would confirm that `` is a *logical* separator between parallel items (like list entries with repeated framing words), not arbitrary spacing. This is the clearest evidence that `` maps to 'row boundary' in a typed schema.\n") + p_pri = f"**{r['boundary_permute_p_upper']}**" + p_sec = f"{r['token_shuffle_p_upper']}" + md_lines.append( + f"| {g} | {n} | {r['obs_mean_shared_trigrams']} | {r['boundary_permute_null_mean']} | " + f"{r['boundary_permute_delta']} | {p_pri} | {r['token_shuffle_null_mean']} | {p_sec} |\n" + ) + md_lines.append(f"\np-values require at least {MIN_PARITY_PAIRS} adjacent-chunk pairs. Even where reported, n is small (Administrative 17, Royal Inscription 10) — these samples could not support a strong claim in either direction.\n") + md_lines.append("\n**Result: null.** Under the control that isolates boundary placement, no genre shows adjacent ruling-delimited chunks sharing more trigrams than arbitrarily placed cuts of the same lengths. Observed values sit *at or below* the boundary-permuted null throughout.\n") + md_lines.append("\n**Correction.** An earlier version of this repository reported this test as significant (Royal p=0.002, Administrative p=0.005) and concluded that `` is a validated logical row separator. That result came from the `token_shuffle` null, which — as the two columns above show — inflates the effect by roughly 100× because it destroys the local coherence that any natural-language text has. The p-value was also computed against the pooled distribution of *individual* null pairs rather than the null distribution *of the mean*. Both are corrected here. **The claim is withdrawn: we have no statistical evidence that `` marks content boundaries.**\n") + md_lines.append("\nThis does not show that `` is *meaningless* — a null result on a sample this small is weak evidence either way, and the marks plainly correspond to physical lines drawn by scribes. It shows only that this test does not support the claim, and that the three-tier memory design it was cited to justify should be read as an untested design proposal.\n") md_lines.append("\n---\n\n") # 5. Summary @@ -318,7 +489,8 @@ def main() -> None: md_lines.append("- **Genre DSLs are real.** Zipf exponents > 1 in Administrative and Royal Inscription indicate keyword-reuse typical of a domain-specific language, not prose. Memory segments in agent runtimes should preserve genre/schema tags so downstream agents can exploit this.\n") md_lines.append("- **Structural redundancy is measurable.** The positive raw-vs-shuffled Δ gives an empirical 'template coverage' score per genre. Memory compression strategies should target high-template genres with template-aware encoders.\n") md_lines.append("- **ELS is a dead end for cuneiform.** As expected. Useful null-result to cite when future ideas try to decode 'hidden patterns'.\n") - md_lines.append("- **RULINGs are logical separators where Δ > 0.** Use them as row boundaries, not as mere visual hints.\n") + md_lines.append("- **RULINGs: no evidence either way.** The earlier positive claim was an artifact of the wrong null and is withdrawn (§4). Treat the three-tier SURFACE/COLUMN/RULING memory design as an untested proposal, not a corpus finding.\n") + md_lines.append("\n**On the two surviving positive results.** Both §1 (Zipf) and §2 (compression Δ) are descriptive statistics against a shuffled-token baseline. They establish that these genres are more formulaic than random token order — they do not establish that the corpus is a 'DSL' in any formal sense. That word is an analogy, not a result.\n") (OUT / "compression_findings.md").write_text("".join(md_lines), encoding="utf-8") print(f"[save] {OUT / 'compression_findings.md'}") @@ -326,6 +498,8 @@ def main() -> None: # Also dump raw metric rows as JSON for the architecture doc to reference. import json (OUT / "phase3_raw.json").write_text(json.dumps({ + "zipf_length_control": zipf_control, + "compression_length_control": comp_control, "zipf": zipf_rows, "compression": comp_rows, "els": els_rows_by_genre,