Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion dataLayer/DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Set `audit = true` on the directive's `CreateTableObject`: `createTables()` audi
- A node that never runs the owner's code path never declares the rest, and its replicated rows are never removed.
- An install on a pre-release of the directive's version never runs it (the `compareVersions` caveat).

Repeating a local `table()` declaration repairs any of these in place, keeping every row, and is a no-op once the shape matches. Storage-layer limits: a thread that only loads the table scans daily, so a spent row can stay a day (reads skip it); under `threads: 0` nothing scans until the first expiring write or declaration after restart; and a dropped attribute's index store stays on disk, since `table()` looks it up by table name. The certificate verification tables follow the same pattern through `security/certificateVerification/verificationTables.ts`; their rows are covered in security/DESIGN.md.
Repeating a local `table()` declaration repairs any of these in place, keeping every row, and is a no-op once the shape matches. Storage-layer limit: a dropped attribute's index store stays on disk, since `table()` looks it up by table name. The certificate verification tables follow the same pattern through `security/certificateVerification/verificationTables.ts`; their rows are covered in security/DESIGN.md.

## RocksDB restore: the restore lock + marker protocol (`dataLayer/restoreMarker.ts`)

Expand Down
1 change: 1 addition & 0 deletions dataLayer/harperBridge/ResourceBridge.ts
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,7 @@ export class ResourceBridge extends BridgeMethods {
attributes,
schemaDefined,
expiration: tableCreateObj.expiration,
eviction: tableCreateObj.eviction,
audit: tableCreateObj.audit,
});
}
Expand Down
2 changes: 1 addition & 1 deletion dataLayer/schemaDescribe.ts
Original file line number Diff line number Diff line change
Expand Up @@ -222,7 +222,7 @@ async function descTable(describeTableObject: any, attrPerms?: any) {
db_size,
};
if (tableObj.replicate !== undefined) tableResult.replicate = tableObj.replicate;
if (tableObj.expirationMS !== undefined) tableResult.expiration = tableObj.expirationMS / 1000 + 's';
if (tableObj.expirationMS || tableObj.evictionMS) tableResult.expiration = tableObj.expirationMS / 1000 + 's';
if (tableObj.sealed !== undefined) tableResult.sealed = tableObj.sealed;
if (tableObj.cacheControl != null) tableResult.cacheControl = tableObj.cacheControl;
if ((tableObj as any).sources?.length > 0)
Expand Down
6 changes: 6 additions & 0 deletions docs/reference/database/schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,12 @@ type WeatherCache @table(expiration: 300, eviction: 3300, scanInterval: 600) {
}
```

**Changing or removing them** <VersionBadge type="changed" version="v5.4.0" />

`expiration` and `eviction` are saved with the table, and a restart applies what the schema last declared. `@table` (and `defineTable`) sets each of the two that it names and clears each one it omits, including a value `create_table` saved; setting one to `0` clears it too. Without `expiration`, records written afterwards no longer expire; without `eviction`, they are removed as soon as they expire. Records already written keep the expiry they were given. Before v5.4.0 an omitted value stayed in effect until the table was dropped. A TTL that application code sets at runtime (for example through `sourcedFrom` options) is not cleared by a schema that declares none or sets it to `0`; a TTL the schema itself set later is.

Each value must be a number of seconds, zero or more. A negative or non-numeric value fails the declaration with an error naming the table and the argument, and the table is left unchanged.

#### How `scanInterval` Determines the Eviction Cycle

`scanInterval` determines fixed clock-aligned times when eviction runs. Harper divides the clock into evenly spaced anchors based on the interval, calculated in the server's local timezone. As a result:
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/operations-api/operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ Drops a database and all its tables/records. **Replicates to all cluster nodes b

### `create_table`

Creates a new table. Optional fields: `database` (defaults to `data`), `attributes` (array defining schema), `expiration` (TTL in seconds).
Creates a new table. Optional fields: `database` (defaults to `data`), `attributes` (array defining schema), `expiration` (TTL in seconds), `eviction` (seconds after `expiration` before a record is removed; see [`@table`](../database/schema.md#table)). A negative or non-numeric `expiration` or `eviction` is rejected with a 400 error.

```json
{
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
section: 'Storage'
---

### Removing a Table's Expiration Clears It

`@table` (and `defineTable`) now clears an `expiration` or `eviction` it no longer names, or sets to `0`, after a restart and on threads that load the table without declaring it. Before, the old value stayed in effect, so a table kept expiring records after its schema stopped declaring a TTL. If you relied on that, declare the value again. Records already written keep the expiry they were given. A negative or non-numeric `expiration`, `eviction` or `scanInterval` now fails the declaration with an error naming the table and argument, instead of failing the table later; `create_table` accepts `eviction` and rejects a negative or non-numeric `expiration` or `eviction` with a 400. Expired records are now removed on schedule after a restart with `threads.count: 0`, and on a table that has only `eviction`, without waiting for a write. See [`@table`](/reference/v5/database/schema#table).
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
rest: true
graphqlSchema:
files: 'schema.graphql'
# Shortened from the 30s default so a run that times out instead of failing fast costs seconds.
timeout: 5000
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
type NegativeTtl @table(database: "ttl_config_bad", expiration: -1) @export {
id: ID @primaryKey
value: String
}
Loading
Loading