Repository navigation
Test that a replica keeps a row written after an expiring row in the same transaction #1410
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
b3efa36
Test that a replica keeps a row written after an expiring row in the β¦
DavidCockerill c103f8f
Bump core to HarperFast/harper#4391's head
DavidCockerill 2ee4bd5
Count a row as readable only when the result carries its id
DavidCockerill b87c394
Require the Exp rows on B before waiting for them to expire, and keepβ¦
DavidCockerill e80c968
Merge remote-tracking branch 'origin/main' into david/1399-replica-exβ¦
DavidCockerill File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Submodule core
updated
3 files
| +2 β0 | docs/reference/resources/resource-api.md | |
| +10 β4 | resources/Table.ts | |
| +149 β0 | unitTests/resources/sourceTxnExpiry.test.js |
5 changes: 5 additions & 0 deletions
5
integrationTests/cluster/fixture-replicated-expiry/config.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| rest: true | ||
| graphqlSchema: | ||
| files: 'schema.graphql' | ||
| jsResource: | ||
| files: 'resources.js' |
20 changes: 20 additions & 0 deletions
20
integrationTests/cluster/fixture-replicated-expiry/resources.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| // Each request is one transaction, so its two writes replicate as one origin transaction. | ||
| export class ExpiringThenPlain extends Resource { | ||
| static loadAsInstance = false; | ||
| async get(target) { | ||
| const id = String(target.id); | ||
| await tables.Exp.put(id, { id }); | ||
| await tables.Plain.put(id, { id }); | ||
| return { wrote: id }; | ||
| } | ||
| } | ||
|
|
||
| export class PlainThenExpiring extends Resource { | ||
| static loadAsInstance = false; | ||
| async get(target) { | ||
| const id = String(target.id); | ||
| await tables.Plain.put(id, { id }); | ||
| await tables.Exp.put(id, { id }); | ||
| return { wrote: id }; | ||
| } | ||
| } |
7 changes: 7 additions & 0 deletions
7
integrationTests/cluster/fixture-replicated-expiry/schema.graphql
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| type Exp @table(expiration: 5) @export { | ||
| id: ID @primaryKey | ||
| } | ||
|
|
||
| type Plain @table @export { | ||
| id: ID @primaryKey | ||
| } |
115 changes: 115 additions & 0 deletions
115
integrationTests/cluster/replicatedWriteOwnExpiry.test.mjs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,115 @@ | ||
| /** | ||
| * The receiver applies an origin transaction under its first event, so a `Plain` row written after an `Exp` row | ||
| * (table expiration) in one transaction must not take `Exp`'s expiry: the replica would hide it once that passes, | ||
| * while the origin keeps it. | ||
| */ | ||
| import { suite, test, before, after } from 'node:test'; | ||
| import { equal } from 'node:assert'; | ||
| import { join } from 'node:path'; | ||
| import { startHarper, getNextAvailableLoopbackAddress, targz } from '@harperfast/integration-testing'; | ||
| import { fetchWithRetry, sendOperation, stopAndTeardownNodes, waitForCondition } from './clusterShared.mjs'; | ||
|
|
||
| process.env.HARPER_INTEGRATION_TEST_INSTALL_SCRIPT = join(import.meta.dirname, '..', '..', 'dist', 'bin', 'harper.js'); | ||
|
|
||
| const DATABASE = 'data'; | ||
|
|
||
| const nodeConfig = (hostname) => ({ | ||
| config: { | ||
| analytics: { aggregatePeriod: -1 }, | ||
| logging: { colors: false, stdStreams: false, console: true }, | ||
| replication: { securePort: hostname + ':9933', databases: [DATABASE] }, | ||
| }, | ||
| env: { HARPER_NO_FLUSH_ON_EXIT: true }, | ||
| }); | ||
|
|
||
| function get(node, path, signal) { | ||
| return fetchWithRetry(`${node.httpURL}${path}`, { | ||
| headers: { | ||
| Authorization: 'Basic ' + Buffer.from(`${node.admin.username}:${node.admin.password}`).toString('base64'), | ||
| }, | ||
| retries: 0, | ||
| signal, | ||
| }); | ||
| } | ||
|
|
||
| // 1 or 0 rows with this id; -1 for a failed read or a per-record error entry, so neither passes as either. | ||
| function readable(node, table, id, signal) { | ||
| return sendOperation( | ||
| node, | ||
| { operation: 'search_by_id', database: DATABASE, table, ids: [id], get_attributes: ['id'] }, | ||
| { signal } | ||
| ).then( | ||
| (rows) => (rows.every((row) => row?.id === id) ? rows.length : -1), | ||
| () => -1 | ||
| ); | ||
| } | ||
|
|
||
| async function deploy(node) { | ||
| await sendOperation(node, { | ||
| operation: 'deploy_component', | ||
| project: 'replicated-expiry', | ||
| payload: await targz(join(import.meta.dirname, 'fixture-replicated-expiry')), | ||
| replicated: false, | ||
| restart: true, | ||
| }); | ||
| await waitForCondition(async (signal) => (await get(node, '/Plain/', signal).catch(() => null))?.status === 200, { | ||
| timeoutMs: 90_000, | ||
| description: `the fixture to be served on ${node.hostname}`, | ||
| }); | ||
| } | ||
|
|
||
| suite('a replicated write takes only its own expiry (harper-pro#1399)', { timeout: 300_000 }, (ctx) => { | ||
| before(async () => { | ||
| const [hostnameA, hostnameB] = await Promise.all([ | ||
| getNextAvailableLoopbackAddress(), | ||
| getNextAvailableLoopbackAddress(), | ||
| ]); | ||
| const nodeA = { name: ctx.name, harper: { hostname: hostnameA } }; | ||
| const nodeB = { name: ctx.name, harper: { hostname: hostnameB } }; | ||
| const starts = await Promise.allSettled([ | ||
| startHarper(nodeA, nodeConfig(hostnameA)), | ||
| startHarper(nodeB, nodeConfig(hostnameB)), | ||
| ]); | ||
| ctx.A = nodeA.harper; | ||
| ctx.B = nodeB.harper; | ||
| const failed = starts.find((start) => start.status === 'rejected'); | ||
| if (failed) throw failed.reason; | ||
| await Promise.all([deploy(ctx.A), deploy(ctx.B)]); | ||
| await sendOperation(ctx.B, { | ||
| operation: 'add_node', | ||
| hostname: ctx.A.hostname, | ||
| rejectUnauthorized: false, | ||
| isLeader: true, | ||
| authorization: ctx.A.admin, | ||
| }); | ||
| await sendOperation(ctx.A, { operation: 'upsert', database: DATABASE, table: 'Plain', records: [{ id: 'probe' }] }); | ||
| await waitForCondition((signal) => readable(ctx.B, 'Plain', 'probe', signal).then((count) => count === 1), { | ||
| timeoutMs: 60_000, | ||
| description: 'replication from A to B', | ||
| }); | ||
| }); | ||
|
|
||
| after(() => stopAndTeardownNodes([ctx.A, ctx.B])); | ||
|
|
||
| test('a row written after an expiring row in one transaction outlives that expiry on the replica', async () => { | ||
| equal((await get(ctx.A, '/ExpiringThenPlain/after')).status, 200); | ||
| equal((await get(ctx.A, '/PlainThenExpiring/before')).status, 200); | ||
| await waitForCondition( | ||
| async (signal) => { | ||
| for (const table of ['Exp', 'Plain']) | ||
| for (const id of ['after', 'before']) if ((await readable(ctx.B, table, id, signal)) !== 1) return false; | ||
| return true; | ||
| }, | ||
| { timeoutMs: 30_000, description: 'the Exp and Plain rows to reach B' } | ||
| ); | ||
| await waitForCondition( | ||
| async (signal) => | ||
| (await readable(ctx.B, 'Exp', 'after', signal)) === 0 && (await readable(ctx.B, 'Exp', 'before', signal)) === 0, | ||
| { timeoutMs: 60_000, description: 'the Exp rows to expire on B' } | ||
| ); | ||
| equal(await readable(ctx.B, 'Plain', 'after'), 1, 'B keeps the Plain row written after Exp'); | ||
| equal(await readable(ctx.B, 'Plain', 'before'), 1, 'B keeps the Plain row written before Exp'); | ||
| equal(await readable(ctx.A, 'Plain', 'after'), 1, 'A keeps the Plain row written after Exp'); | ||
| equal(await readable(ctx.A, 'Plain', 'before'), 1, 'A keeps the Plain row written before Exp'); | ||
| }); | ||
| }); | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please give all four final reads an explicit abort deadline.
readableforwards its optional signal tosendOperation, whose fetch and JSON-body read have no helper-level deadline (integrationTests/cluster/clusterShared.mjs:21-28). If a node accepts the request but stops completing its response, this assertion waits for the five-minute suite timeout instead of producing a focused failure. Pass a per-requestAbortSignal.timeout(...)here; the two fixture-triggering GETs atintegrationTests/cluster/replicatedWriteOwnExpiry.test.mjs:95-96need the same treatment.