Skip to content

Commit f391def

Browse files
authored
fix(testing): reset Postgres tables with TRUNCATE so row triggers cannot refuse it (#1052) (#1059)
* fix(testing): reset Postgres tables with TRUNCATE so row triggers cannot refuse it (#1052) Claude-Session: https://claude.ai/code/session_01FXHA3Zj78w6GWaNKoovHq6 * refactor(optimize): duplication — drop postgres markers the module pytestmark already applies Claude-Session: https://claude.ai/code/session_01FXHA3Zj78w6GWaNKoovHq6 * fix: address code review findings (round 1, pass 1) Claude-Session: https://claude.ai/code/session_01FXHA3Zj78w6GWaNKoovHq6
1 parent 55979bb commit f391def

4 files changed

Lines changed: 138 additions & 10 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ All notable changes to this project are documented in this file. The format is b
2525
- SMTP preserves long `List-Unsubscribe` URLs within the 998-character line limit without RFC 2047 encoding, while retaining encoded non-ASCII headers and SMTPUTF8 support for internationalized mailboxes (#1053).
2626
- SMTP refolds long UTF-8 headers to respect the 998-byte wire-line limit (#1053).
2727
- Store OAuth access and refresh tokens as unbounded TEXT so long Microsoft Entra tokens can sign in on Postgres; migration `a1054e9c7b62` widens both columns (#1054).
28+
- **Behaviour change:** When reusing a schema, PostgreSQL test fixtures empty model tables with one `TRUNCATE ... RESTART IDENTITY RESTRICT`, so row-level `DELETE` triggers cannot block between-test resets. Foreign-key references from tables outside the reset list reject the reset, and lock waits fail after five seconds. Unowned public-schema sequences still restart at 1; statement-level `TRUNCATE` triggers still apply (#1052).
2829

2930
## [0.0.48] — 2026-11-19
3031

‎docs/testing/fixtures.md‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,20 @@ local `sm_test` database) and runs 4 workers by default. Under xdist each worker
3434

3535
Each test starts with empty tables. The schema is built once per process and
3636
rebuilt only when changed; Postgres fixtures do not create a separate database
37-
per test. Connections use `NullPool` because pytest-asyncio gives each test
37+
per test. When reusing a schema, all model tables are emptied in one
38+
`TRUNCATE <tables> RESTART IDENTITY RESTRICT` statement. This bypasses row-level
39+
`DELETE` triggers, including guards that reject deletion of immutable rows.
40+
Statement-level `TRUNCATE` triggers (including `BEFORE TRUNCATE`) still fire and
41+
can reject the reset. Public-schema sequences are also reset to 1, including
42+
sequences not owned by a table column, which `RESTART IDENTITY` does not reset.
43+
Foreign-key references from tables outside the model-table list reject the
44+
reset rather than implicitly emptying those tables. `TRUNCATE` takes exclusive
45+
table locks until the reset transaction completes; a transaction-local five-second
46+
lock timeout makes leaked read transactions fail setup rather than hang it.
47+
Keep using separate databases per xdist worker. The default in-memory SQLite
48+
fixtures copy a schema template into fresh databases instead of running this reset.
49+
50+
Connections use `NullPool` because pytest-asyncio gives each test
3851
its own event loop. Use `-p no:anyio` when invoking pytest directly on Postgres;
3952
the Make target already includes it.
4053

‎framework/testing/simple_module_test/database.py‎

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -152,8 +152,8 @@ async def prepare_schema_once(engine: Any, tables: list[Any], build: Any) -> Non
152152
per-process copy of the first build (``_sqlite_template``), a file one runs
153153
``build(engine)``.
154154
Postgres: when this process already built the schema and the catalog still
155-
matches it, delete every row and restart the sequences (~0.03 s) instead of
156-
dropping and rebuilding it (~0.25 s). Anything else (the first test, a
155+
matches it, empty the tables with one TRUNCATE and restart the sequences
156+
instead of dropping and rebuilding it. Anything else (the first test, a
157157
direct ``reset_schema`` call, a test's own DDL) resets and rebuilds.
158158
"""
159159
global _reset_for, _built_schema
@@ -181,13 +181,17 @@ async def _fingerprint(engine: Any) -> str:
181181

182182

183183
async def _empty_tables(engine: Any, tables: list[Any]) -> None:
184-
from sqlalchemy.schema import sort_tables
184+
"""Empty a reused Postgres schema; SQLite is handled before this call."""
185185

186186
async with engine.begin() as conn:
187-
quote = conn.dialect.identifier_preparer.format_table
188-
# Children first, so no foreign key refuses a parent's delete.
189-
for table in reversed(sort_tables(tables)):
190-
await conn.execute(text(f"DELETE FROM {quote(table)}"))
187+
# A leaked reader must fail setup instead of waiting forever for TRUNCATE.
188+
await conn.execute(text("SET LOCAL lock_timeout = '5s'"))
189+
if tables:
190+
quote = conn.dialect.identifier_preparer.format_table
191+
names = ", ".join(quote(table) for table in tables)
192+
# Refuse foreign-key dependants outside the caller's reset boundary.
193+
await conn.execute(text(f"TRUNCATE {names} RESTART IDENTITY RESTRICT"))
194+
# RESTART IDENTITY excludes unowned sequences; preserve their reset too.
191195
await conn.execute(_RESTART_SEQUENCES)
192196

193197

‎framework/testing/tests/test_schema_reuse_pg.py‎

Lines changed: 112 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,23 @@
22

33
from __future__ import annotations
44

5+
import asyncio
6+
57
import pytest
68
from simple_module_test import database
7-
from sqlalchemy import Column, Integer, MetaData, String, Table, func, select, text
9+
from sqlalchemy import (
10+
Column,
11+
ForeignKey,
12+
Identity,
13+
Integer,
14+
MetaData,
15+
String,
16+
Table,
17+
func,
18+
select,
19+
text,
20+
)
21+
from sqlalchemy.exc import DBAPIError
822
from sqlalchemy.ext.asyncio import create_async_engine
923

1024
pytestmark = pytest.mark.postgres
@@ -13,7 +27,7 @@
1327
_item = Table(
1428
"reuse_probe_item",
1529
_metadata,
16-
Column("id", Integer, primary_key=True),
30+
Column("id", Integer, Identity(), primary_key=True),
1731
Column("name", String(20)),
1832
)
1933

@@ -73,3 +87,99 @@ async def test_a_test_that_changes_the_schema_forces_a_rebuild(engine):
7387
assert builds == [1, 1]
7488
async with engine.connect() as conn:
7589
assert await conn.scalar(text("SELECT to_regclass('reuse_probe_extra')")) is None
90+
91+
92+
async def test_reset_bypasses_delete_row_triggers_and_restarts_identity(engine):
93+
await _next_test(engine, [])
94+
child = Table(
95+
"reset probe child",
96+
MetaData(),
97+
Column("item_id", Integer, ForeignKey(_item.c.id), primary_key=True),
98+
schema="public",
99+
)
100+
async with engine.begin() as conn:
101+
await conn.run_sync(child.create)
102+
await conn.execute(
103+
text("""
104+
CREATE FUNCTION refuse_probe_delete() RETURNS trigger
105+
LANGUAGE plpgsql AS $$
106+
BEGIN
107+
RAISE EXCEPTION 'probe rows are immutable';
108+
END;
109+
$$
110+
""")
111+
)
112+
await conn.execute(
113+
text("""
114+
CREATE TRIGGER refuse_probe_delete BEFORE DELETE ON reuse_probe_item
115+
FOR EACH ROW EXECUTE FUNCTION refuse_probe_delete()
116+
""")
117+
)
118+
assert [await _insert(engine), await _insert(engine)] == [1, 2]
119+
async with engine.begin() as conn:
120+
await conn.execute(child.insert().values(item_id=1))
121+
122+
# A quoted, schema-qualified child and parent-first input need no DELETE ordering.
123+
await database._empty_tables(engine, [_item, child])
124+
125+
async with engine.connect() as conn:
126+
assert await conn.scalar(select(func.count()).select_from(_item)) == 0
127+
assert await conn.scalar(select(func.count()).select_from(child)) == 0
128+
assert await _insert(engine) == 1
129+
130+
131+
async def test_reset_restarts_sequences_without_a_table_owner(engine):
132+
await _next_test(engine, [])
133+
async with engine.begin() as conn:
134+
await conn.execute(text("CREATE SEQUENCE reuse_probe_unowned"))
135+
assert await conn.scalar(text("SELECT nextval('reuse_probe_unowned')")) == 1
136+
assert await conn.scalar(text("SELECT nextval('reuse_probe_unowned')")) == 2
137+
138+
await database._empty_tables(engine, [_item])
139+
140+
async with engine.connect() as conn:
141+
assert await conn.scalar(text("SELECT nextval('reuse_probe_unowned')")) == 1
142+
143+
144+
async def test_reset_refuses_to_empty_tables_outside_the_passed_list(engine):
145+
await _next_test(engine, [])
146+
outside = Table(
147+
"outside_probe",
148+
MetaData(),
149+
Column("id", Integer, primary_key=True),
150+
Column("item_id", Integer, ForeignKey(_item.c.id)),
151+
)
152+
async with engine.begin() as conn:
153+
await conn.run_sync(outside.create)
154+
# DELETE on the empty parent would leave this unrelated row untouched.
155+
await conn.execute(outside.insert().values(id=1, item_id=None))
156+
157+
with pytest.raises(DBAPIError, match="foreign key"):
158+
await database._empty_tables(engine, [_item])
159+
160+
async with engine.connect() as conn:
161+
assert await conn.scalar(select(outside.c.id)) == 1
162+
163+
164+
async def test_reset_times_out_when_a_previous_reader_holds_a_lock(engine):
165+
await _next_test(engine, [])
166+
async with engine.connect() as reader:
167+
await reader.execute(select(_item))
168+
try:
169+
with pytest.raises(DBAPIError, match="lock timeout"):
170+
await asyncio.wait_for(database._empty_tables(engine, [_item]), timeout=10)
171+
finally:
172+
await reader.rollback()
173+
174+
175+
async def test_reset_with_no_tables_preserves_the_migration_stamp(engine):
176+
await _next_test(engine, [])
177+
async with engine.begin() as conn:
178+
await conn.execute(text("CREATE TABLE alembic_version (version_num text)"))
179+
await conn.execute(text("INSERT INTO alembic_version VALUES ('probe')"))
180+
181+
await database._empty_tables(engine, [])
182+
await database._empty_tables(engine, [_item])
183+
184+
async with engine.connect() as conn:
185+
assert await conn.scalar(text("SELECT version_num FROM alembic_version")) == "probe"

0 commit comments

Comments
 (0)