Skip to content

fix(sql-cache): measure the 5kb stored-SQL cap in UTF-8 bytes - #355

Open
OsamaAnsar wants to merge 1 commit into
tursodatabase:mainfrom
OsamaAnsar:fix/sql-cache-utf8-byte-limit
Open

fix(sql-cache): measure the 5kb stored-SQL cap in UTF-8 bytes#355
OsamaAnsar wants to merge 1 commit into
tursodatabase:mainfrom
OsamaAnsar:fix/sql-cache-utf8-byte-limit

Conversation

@OsamaAnsar

Copy link
Copy Markdown

Closes #353.

Problem

SqlCache.apply guards sqld's 5 KB stored-SQL cap with sqlText.length >= 5000, which counts UTF-16 code units. sqld enforces that cap in UTF-8 bytes (req.sql.len() in libsql-server/src/hrana/http/request.rs).

Any statement that falls between the two measures is cached and store_sql'd by the client, refused by the server, and then every later batch that references its sql_id fails with SQL text <id> not found. Pure-ASCII SQL is unaffected because the two measures coincide; the bug only shows up once statements carry multibyte text (CJK, accented Latin, emoji), which makes it look like a data-dependent protocol error rather than a size limit.

Fix

Measure the guard in the same units the server does:

if (new TextEncoder().encode(sqlText).length >= 5000) {
    continue;
}

TextEncoder is available on every runtime this package targets. The client's >= 5000 stays one byte stricter than the server's > 5000; that direction is safe (the client just sends a borderline statement inline instead of caching it), so this PR leaves it as-is.

Test

New packages/libsql-client/src/__tests__/sql_cache.test.ts with a fake SqlOwner:

  • a small statement is still cached
  • a statement that is under the cap by UTF-16 length but over it by UTF-8 bytes is not cached (the bug)
  • a statement over the cap by UTF-16 length is still not cached (unchanged behaviour)

The middle case fails on main without the fix and passes with it.

Verification

  • sql_cache, uri, config unit suites pass; tsc --noEmit and prettier --check clean
  • I couldn't run client.test.ts locally (it needs the Python hrana-test-server, and I'm on Windows without a working Python) — relying on CI for the integration matrix. Happy to add more coverage if you'd like.

Scope

Kept to the units fix. The issue mentions two optional follow-ups — surfacing the refused store_sql error to the caller instead of only into _setClosed, and documenting the cap's units in the guard comment — which I'd be glad to do in separate PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SqlCache measures SQLD's 5 KB stored-SQL cap in UTF-16 code units, but the server enforces it in UTF-8 bytes

1 participant