feat(21.8): an index carries its own schema-cache TTL (Part D)

[fupelaqu]

Story 21.8 — Part D: per-index schema-cache TTL

Part D only. Parts A/B/C/E/F/G of story 21.8 are untouched and live on feature/21.8.

Why this is not a performance knob

Since #306 every executed statement reads the cached schema. A stale entry no longer means a stale
column list — it means Painless emitted for the previous mapping. Retype a column keyword →
long and, for the rest of the TTL, the client keeps emitting Long.parseLong(doc['x'].value)
against a field that is already numeric. It was a hardcoded five minutes, in three files.

What it does

D.1 — the default. elastic.schema-cache.ttl (ELASTIC_SCHEMA_CACHE_TTL, default 5m), a
HOCON duration.

D.2 — the per-index TTL. Stored in the index's own mapping metadata (_meta.schema_cache_ttl),
so every client against that cluster picks it up. Each cache entry is stamped with the TTL that
governed it and expires on its own clock (CachedSchema, with a single isExpired rule on a shared
CacheEntry).

Precedence: index metadata > elastic.schema-cache.ttl > the built-in 5 minutes.

Two spellings, one thing written. ALTER TABLE t SET SCHEMA CACHE TTL [=] '10m' and
DROP SCHEMA CACHE TTL are sugar: they desugar to the existing AlterTableMapping /
DropTableMapping on that _meta path — no new AST, no new merge / diff / render arm, one key
derivation. These three are identical:

ALTER TABLE orders SET SCHEMA CACHE TTL = '1h';
ALTER TABLE orders SET MAPPING _meta.schema_cache_ttl = '1h';
CREATE TABLE orders (id INT NOT NULL) OPTIONS (mappings = (_meta = (schema_cache_ttl = '1h')));

The duration is validated at parse time, by the same parse the client applies (err, never
throw — #250), so a misspelled TTL is refused before it reaches the cluster rather than silently
ignored for the life of the index.

The lead rulings this implements

• D.4 / OQ-2 — the shard-count cache follows the same value. ScrollApi.shardCountCacheTtlMs is
  deleted; the cache takes the shortest resolved TTL among the indices its key names. A schema
  cached for an hour next to a shard count re-probed every five minutes is incoherent, and two
  constants in two files drift silently — each half reading as correct in isolation.
• Both spellings (the _meta write and dedicated keywords), sharing one derivation of the key.
• Explicitly not added, per the lead: a per-index TTL map in configuration.

Deliberate asymmetries

• The 404 negative cache uses the default, and only downwards. It records a miss — there is no
  metadata to read a per-index TTL from (D.3.2). It is also capped at the built-in five minutes:
  nothing invalidates a miss and no lookup is attempted while one stands, so a lengthened one would
  leave a table created after a failed probe running with no schema attached — no conversions,
  no temporal-literal resolution — for that whole period.
• Aliases inherit their target's TTL, because the alias entry caches the target's schema.
• Changing a TTL is self-referential: another client notices only when its entry expires, so
  the old period governs. Bounded, and documented.
• The cache is now bounded — it never was. Past 256 entries a miss drops the expired ones; past
  1024 live entries it drops the cache. Values here are whole schemas, and expiry alone bounds
  nothing once an index can ask for a long TTL over ever-changing index names.

Verification

• Real Elasticsearch, all five clients (6.8 rest, 6.8 jest→pinned 6.7.2, 7.17, 8.18, 9.0): the
  TTL round-trips through _meta, survives an unrelated later ALTER (what Table.update()
  rebuilding its own _meta keys puts at risk), is replaceable, drops back to the default, and a
  bad duration never reaches the cluster.
• 931 sql + 944 core unit tests; 2.12 cross-compile (main + test); scalafmtCheckAll and
  headerCheck clean.
• 15 mutations, each RED as predicted — including min→max over an index set, >=→> on the
  expiry boundary, and the metadata key/path identity.
• An independent fresh-context review ran before commit; its two HIGH and three MEDIUM findings
  are fixed in this branch (see below).

Review findings fixed here

	finding	fix
HIGH	the purge could delete a live alias→target row (the row is written inside compute, before the entry is installed), resurrecting the #276/R4-21 stale-alias-mapping defect	the sweep removes alias rows only for entries it actually expired
HIGH	scalafmtCheck was vacuous — project.git = true, and every new file was untracked	staged, then formatted: three files were misformatted
MED	the operator WARN reported a different TTL than the entry was stamped with — D.4's own defect, in the half nobody would notice was lying	one shardCountTtlMs(indices) feeds both
MED	the negative cache became lengthenable without any invalidation hook	capped at the built-in five minutes
MED	"bounded" was false — expiry-only purging bounds nothing	hard stop added; doc wording corrected

Plus LOW fixes: the alias assertion (the case D.3.3 says must be decided) now asserts an observed
fetch, not just the resolver's return value; two integration tests no longer depend on the previous
test's state; a dead scaladoc link.

🔴 One fix is ungated and I would rather say so. The alias-row fix cannot be covered by a test:
the divergent state exists only between the alias put and compute installing the entry, and every
seam a test can override runs before that put. The structural cure is to carry the target on the
cache entry and delete the second map — recorded at the code site, deliberately not done here
because it rewrites #276's invalidation path.

Release notes

1. New setting elastic.schema-cache.ttl (ELASTIC_SCHEMA_CACHE_TTL, default 5m). It also
   governs the primary shard-count cache and (downwards only) the 404 negative cache, each of which
   had its own hardcoded five minutes. A deployment that sets nothing sees no behaviour change.
2. New DDL — ALTER TABLE … SET SCHEMA CACHE TTL [=] '<duration>' / DROP SCHEMA CACHE TTL.
3. ⚠️ ScrollApi.shardCountCacheTtlMs (protected) is REMOVED. A subclass overriding it no longer
   compiles — override schemaCacheTtlMs instead. No occurrence in extensions / jdbc / arrow.
4. ⚠️ Binary-incompatible for downstream ⇒ rebuild on the next core bump: ElasticConfig arity
   9→10 (new defaulted schemaCache, source-compatible), and IndicesApi / ScrollApi / SearchApi
   each gain SchemaCacheTtlApi as a parent (schemaCacheTtlMs moved there).
5. Three new keywords — SCHEMA, CACHE, TTL — are highlighted and completed in the REPL. They
   are not parser-reserved: SELECT schema FROM t, ADD COLUMN cache INT and
   SET MAPPING schema = … all still parse (pinned).
6. The schema cache is bounded for the first time.

Documentation

documentation/sql/ddl_statements.md, documentation/client/search.md (the Tuning section),
documentation/client/scroll.md, documentation/client/common_principles.md (HOCON reference), and
the REPL help corpus (help/commands/ddl/alter_table.json, guarded by HelpCorpusSpec).

Two stale claims were corrected while there: the schema lookup is no longer "every 5 minutes", and
dql_statements.md still said the resolution happens "only for statements whose WHERE compares a
string literal to a column" — false since #306 made the attach unconditional.

Every published SQL example was run through the real parser, and the distinctive ones are pinned so
they cannot rot. The docs-site twin is SOFTNETWORK-APP/softclient4es-web#54.

🤖 Generated with Claude Code