Create table, alter table - APIs and modals by simonw · Pull Request #2789 · simonw/datasette

simonw · 2026-06-17T16:24:04Z

Refs:

Video demo (before a few final cosmetic fixes):

CleanShot.2026-06-22.at.12.27.25.mp4

📚 Documentation preview 📚: https://datasette--2789.org.readthedocs.build/en/2789/

simonw · 2026-06-17T16:28:25Z

+dialog.table-create-dialog {
+    --ink: #0f0f0f;
+    --paper: #eef6ff;
+    --muted: #6b6b6b;
+    --rule: #d8e6f5;
+    --accent: #1a56db;
+    --card: #ffffff;
+    border: none;
+    border-radius: var(--modal-border-radius, 0.75rem);
+    padding: 0;
+    margin: auto;
+    width: min(760px, calc(100vw - 32px));
+    max-width: 95vw;
+    max-height: min(780px, calc(100vh - 32px));
+    box-shadow: var(--modal-shadow, 0 20px 25px -5px rgba(0, 0, 0, 0.1), 0 10px 10px -5px rgba(0, 0, 0, 0.04));
+    animation: datasette-modal-slide-in var(--modal-animation-duration, 0.2s) ease-out;
+    overflow: hidden;
+    font-family: system-ui, -apple-system, sans-serif;
+    background: var(--card);
+}
+
+dialog.table-create-dialog[open] {
+    display: flex;
+    flex-direction: column;
+}


This is consistent with our other dialogs but there's a whole lot of duplicate model code now. Filing an issue to clean that up later:

Refactor our modal dialogs to share a lot more code #2790

simonw · 2026-06-17T17:27:21Z

Pyodide tests are failing with:

ValueError: Couldn't find a pure Python 3 wheel for 'pydantic-core==2.46.4'. You can use micropip.install(..., keep_going=True) to get a list of all packages with missing wheels.

That shouldn't be an issue now, see: https://simonwillison.net/2026/Jun/13/publishing-wasm-wheels/

Might need to upgrade Pyodide (here and in Datasette Lite).

codecov · 2026-06-17T17:27:40Z

Codecov Report

❌ Patch coverage is 0% with 744 lines in your changes missing coverage. Please review.
✅ Project coverage is 0.00%. Comparing base (57e7bba) to head (b3b5c25).
⚠️ Report is 33 commits behind head on main.

Files with missing lines	Patch %	Lines
datasette/views/table_create_alter.py	0.00%	670 Missing ⚠️
datasette/views/table.py	0.00%	45 Missing ⚠️
datasette/default_table_actions.py	0.00%	12 Missing ⚠️
datasette/views/database.py	0.00%	7 Missing ⚠️
datasette/views/table_extras.py	0.00%	6 Missing ⚠️
datasette/app.py	0.00%	4 Missing ⚠️

Additional details and impacted files

@@          Coverage Diff           @@
##            main   #2789    +/-   ##
======================================
  Coverage   0.00%   0.00%            
======================================
  Files         70      72     +2     
  Lines      11183   11789   +606     
======================================
- Misses     11183   11789   +606

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Now that we depend on pydantic we need a more recent pyodide in order to load the emscripten build of pydantic-core. Refs #2789 (comment)

simonw · 2026-06-17T17:41:10Z

There's a feature missing from both create and alter: foreign key constraints. These are worth supporting, particularly since the edit/insert modals have neat support for them.

Related to that: setting the "label column" for a table. Might need a larger issue for that since we need to store label columns somewhere, probably in the same place as custom column types.

- Add fk_table and optional fk_column support to create-table columns. - Validate create-table requests with Pydantic while preserving existing errors. - Document the API and cover inferred primary-key and validation cases. Refs #2789 (comment)

Adds a permission-gated database action that opens a create table modal on database pages, backed by the existing create-table JSON API. The modal starts with an id integer primary key column plus a blank text column, supports SQLite type selection, and shows custom column type controls only when the actor can set column types. Selected custom column types are applied after table creation with follow-up set-column-type API calls. Includes styling plus HTML and Playwright coverage for the action payload and create-table flow.

- Add POST /<database>/<table>/-/alter with Pydantic validation and dry-run support. - Support add, rename, alter, drop, primary-key and reorder operations, including allow-listed default expressions. - Document the endpoint and cover schema changes, validation, permissions, events and dry runs. Refs #2788

- Register a built-in table action and expose alter-table metadata to table pages. - Build the client-side modal for editing columns, defaults, ordering, primary keys, and custom column types. - Add a review/apply confirmation flow with HTML and Playwright coverage. Refs #2788

- Use a per-process socket path for the UDS test fixture. - Clean up stale socket files before and after the fixture runs. - Close the HTTP client and wait for the Datasette subprocess to exit.

- Extract reusable helpers for database and table action permission preloading. - Precompute those permissions before building table-page HTML data. - Document the default table actions plugin.

Now that we depend on pydantic we need a more recent pyodide in order to load the emscripten build of pydantic-core. Refs #2789 (comment)

- Move create-table and alter-table API views into table_create_alter.py. - Keep create and alter schema-editing constants and helpers together. - Rename the create table modal context helper.

- Add fk_table and optional fk_column support to create-table columns. - Validate create-table requests with Pydantic while preserving existing errors. - Document the API and cover inferred primary-key and validation cases. Refs #2789 (comment)

- Add add_foreign_key, drop_foreign_key, and set_foreign_keys operations. - Validate flat fk_table and fk_column arguments with Pydantic. - Document the API and cover inferred primary-key and validation cases.

Improved version of the implementation datasette-edit-schema

Returns a list of tables with a single primary key, and for each one the name of that primary key column and its SQLite type affinity. This will be used by the create table UI to suggest foreign keys.

- Add foreignKeyTargetsPath to create table page data - Filter hidden tables from database-level foreign key target results - Update JSON API docs and tests for filtered targets

- Add create table advanced controls for foreign keys and first-column primary keys - Share schema dialog row helpers between create and alter dialogs - Move custom type into advanced options and add Add column icons

In the create table dialog a column can now have either a custom display type or a foreign key target, but not both - a foreign key column's type is determined by the referenced primary key, so a custom type doesn't apply. Setting one clears and disables the other, and the foreign key select stays disabled on the primary key column and when no targets exist. Also add "Controls how Datasette displays and edits this column" help text (with aria-describedby) under the custom type selector in both the create and alter dialogs, and style the alter dialog help text.

Refs https://github.com/simonw/datasette/pull/2789/changes#r3453964430

It works by doing conn.backup(memory_conn) which could use a lot of memory for a large database.

simonw · 2026-06-17T19:52:58Z

    sqlite_type: column_type
    for column_type, sqlite_type in CREATE_TABLE_SQLITE_TYPES.items()
 }
+TABLE_NAME_RE = re.compile(r"^(?!sqlite_)[^\n]+$")


Because SQLite preserves that prefix for internal use:

python3 << 'EOF' import sqlite3 conn = sqlite3.connect(":memory:") cur = conn.cursor() # Attempt 1: plain CREATE TABLE with the reserved-looking name try: cur.execute("CREATE TABLE sqlite_misx (id INTEGER PRIMARY KEY, name TEXT)") print("Attempt 1 (plain CREATE TABLE): SUCCESS") except sqlite3.Error as e: print(f"Attempt 1 (plain CREATE TABLE): FAILED -> {type(e).__name__}: {e}") conn.close() EOF

simonw · 2026-06-22T16:45:44Z

+function sqliteColumnTypeLabel(type) {
+  if (type === "float") {
+    return "floating point number";
+  }
+  if (type === "real") {
+    return "floating point number";
+  }
+  if (type === "blob") {
+    return "blob - binary data";
+  }
+  return type;
+}


I'm going to make this an object instead.

simonw · 2026-06-22T16:50:08Z

                "table_page_data": await _table_page_data(
                    self.ds,
                    request,
                    db,
                    database,
                    table,
                    not is_table,
                    None,
+                    None,
                ),


Hard to read

simonw · 2026-06-22T17:45:56Z

Alter table doesn't currently let you set foreign keys. Create table doesn't let you set a default expression.

I had Codex review them for other differences:

I compared the running forms and checked the shared JS.

Create Table

Creates a brand new table; has a Table name field.

Starts with id integer primary key plus one blank text column.

Per column: name, SQLite type, move/remove, advanced options.

Advanced options: custom Datasette column type, foreign key target, primary key.

Foreign keys are exposed in the UI, but only for non-first columns.

Primary key UI is effectively first-column-only.

Submits immediately with Create table; no review step.

No UI for not null, default value, or default expression.

Alter Table

Edits an existing table; no table name field.

Preloads every existing column.

Per column: rename, change SQLite type, reorder, drop/remove, advanced options.

Advanced options: custom Datasette column type, Not null, default value, default expression, primary key.

Can add new columns and change/drop/reorder existing ones.

Has Review changes then Apply changes, with a warning for dropped columns.

Includes Drop table in the same modal.

No foreign-key editing control in the form right now.

So the big asymmetry is: create has foreign-key selection, alter has defaults/not-null/review/drop-table. They share the column row mechanics, type/custom-type controls, move behavior, and primary-key ordering rules.

Need to unify the code and features before landing this.

simonw · 2026-06-22T17:55:31Z

Let's hide the default value stuff in a summary/details thing on both UIs - change text to "or default to a specific value" - and in the expression menu extend it to say things like "Current timestamp in uTC, e.g. 2026-05-01 13:34:00".

So we don't get test failures from reformatted SQL.

simonw · 2026-06-22T18:13:22Z

+DEFAULT_EXPR_SQL = {
+    "current_timestamp": "CURRENT_TIMESTAMP",
+    "current_date": "CURRENT_DATE",
+    "current_time": "CURRENT_TIME",
+}


Let's add unix timestamp options here as well.

current_unixtime - integer seconds since the epoch

current_unixtime_ms - integer ms since the epoch

simonw · 2026-06-22T18:47:13Z

Found this sqlite-utils bug while trying this out:

table.insert({}) fails with ZeroDivisionError sqlite-utils#759

I'm going to fix that in sqlite-utils rather than working around it here.

simonw · 2026-06-22T18:47:40Z

If you sort by a column, then rename that column using alter table, you get redirected back to a page with a 400 error saying Cannot sort table by old_name_of_column.

Instead it should remove that _sort= before the redirect.

simonw · 2026-06-22T18:49:09Z

The alter table form needs to let you rename a table.

simonw · 2026-06-22T19:08:37Z

In create table in the foreign key list we should show ALL foreign keys, not just the ones that match the selected SQLite column type - then if the user picks a foreign key that is of a different column type we should switch the column type in that select box to match the selected foreign key.

Refs #2789 (comment)

simonw · 2026-06-22T19:13:07Z

When selecting a foreign key column, if the column name has not yet been set, set the column name to table_id e.g. if the foreign key is to pypi_versions.id then the column name should be set to pypi_versions_id.

simonw · 2026-06-22T19:30:23Z

Here's a video of everything so far:

CleanShot.2026-06-22.at.12.27.25.mp4

simonw · 2026-06-22T19:31:30Z

The video shows that in macOS Safari the select boxes are not the same height as the input boxes.

Refs #2789 (comment)

simonw · 2026-06-22T19:35:59Z

That fixed the Safari select problem:

Include current foreign key metadata in the alter table page data and allow the foreign-key-targets endpoint to be read by actors with alter-table permission for a specific table. Add API and HTML data tests for the new alter-table foreign key support.

Share default value controls between the create and alter table dialogs and expose create-table default expressions to the frontend. Add create-table not-null/default handling and align the shared foreign key picker behavior across both dialogs.

Add a collapsed rename-table section to the alter table modal and include rename_table operations in the review/apply flow. Redirect to the renamed table URL after applying changes and cover the review text in Playwright.

Plus tweaked how alter table changing those works a bit.

Refs #2789

simonw commented Jun 17, 2026

View reviewed changes

simonw mentioned this pull request Jun 17, 2026

UI for creating a table #2787

Open

simonw added a commit that referenced this pull request Jun 17, 2026

Test against pyodide/v314.0.0

2c07d38

Now that we depend on pydantic we need a more recent pyodide in order to load the emscripten build of pydantic-core. Refs #2789 (comment)

simonw marked this pull request as ready for review June 17, 2026 18:37

simonw force-pushed the codex/alter-table-modal branch from 2930f7a to fbab41f Compare June 17, 2026 21:03

simonw added 21 commits June 22, 2026 10:11

Ran Prettier

15a3ac5

Isolate Unix domain socket test server paths

c9c79fd

- Use a per-process socket path for the UDS test fixture. - Clean up stale socket files before and after the fixture runs. - Close the HTTP client and wait for the Datasette subprocess to exit.

Precompute action permissions for table pages

4115213

- Extract reusable helpers for database and table action permission preloading. - Precompute those permissions before building table-page HTML data. - Document the default table actions plugin.

Test against pyodide/v314.0.0

8cec528

Now that we depend on pydantic we need a more recent pyodide in order to load the emscripten build of pydantic-core. Refs #2789 (comment)

Split table create and alter views

1972ba8

- Move create-table and alter-table API views into table_create_alter.py. - Keep create and alter schema-editing constants and helpers together. - Rename the create table modal context helper.

Add foreign keys to create table API

9766a9c

- Add fk_table and optional fk_column support to create-table columns. - Validate create-table requests with Pydantic while preserving existing errors. - Document the API and cover inferred primary-key and validation cases. Refs #2789 (comment)

Add foreign keys to alter table API

9d9a2d3

- Add add_foreign_key, drop_foreign_key, and set_foreign_keys operations. - Validate flat fk_table and fk_column arguments with Pydantic. - Document the API and cover inferred primary-key and validation cases.

/db/table/-/foreign-key-suggestions API

2900efb

Improved version of the implementation datasette-edit-schema

/<database>/-/foreign-key-targets API endpoint

a6ef65f

Returns a list of tables with a single primary key, and for each one the name of that primary key column and its SQLite type affinity. This will be used by the create table UI to suggest foreign keys.

Expose foreign key targets to create table UI

21c156d

- Add foreignKeyTargetsPath to create table page data - Filter hidden tables from database-level foreign key target results - Update JSON API docs and tests for filtered targets

Add foreign key controls to create table dialog

1f863de

- Add create table advanced controls for foreign keys and first-column primary keys - Share schema dialog row helpers between create and alter dialogs - Move custom type into advanced options and add Add column icons

More robust test_datasette_https_server.sh test

c77dad9

Drop table button in alter dialog

354780a

Object not chain of ifs

a87c4ac

Refs https://github.com/simonw/datasette/pull/2789/changes#r3453964430

Fix broken Playwright tests

b02999b

Keyword arguments for readability

17876cc

Ran prettier

c4aead6

simonw force-pushed the codex/alter-table-modal branch from d6a826e to c4aead6 Compare June 22, 2026 17:13

Removed the alter table dry run feature

084df1f

It works by doing conn.backup(memory_conn) which could use a lot of memory for a large database.

simonw commented Jun 22, 2026

View reviewed changes

simonw added 2 commits June 22, 2026 11:03

sqlite-utils>=3.30,<4.0

a2e7596

So we don't get test failures from reformatted SQL.

not_null, default and default_exr support for create table API columns

87354cf

simonw commented Jun 22, 2026

View reviewed changes

Alter table API can now rename tables, refs #2788

4b219be

Refs #2789 (comment)

Fix for Safari select box heights

dada4de

Refs #2789 (comment)

simonw added 5 commits June 22, 2026 12:47

Add rename table controls to alter table dialog

2ebae5e

Add a collapsed rename-table section to the alter table modal and include rename_table operations in the review/apply flow. Redirect to the renamed table URL after applying changes and cover the review text in Playwright.

current_unixtime and current_unixtime_ms default_expr options

b932d0d

Plus tweaked how alter table changing those works a bit.

Draft changelog for create/alter table UI, refs #2787, #2788

b3b5c25

simonw merged commit f0645c6 into main Jun 22, 2026
33 of 43 checks passed

simonw deleted the codex/alter-table-modal branch June 22, 2026 20:54

simonw added a commit that referenced this pull request Jun 22, 2026

Avoid SQLite RETURNING for compat with older SQLite

f831352

Refs #2789

Uh oh!

Conversation

simonw commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

simonw Jun 17, 2026

Choose a reason for hiding this comment

Uh oh!

simonw commented Jun 17, 2026

Uh oh!

codecov Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

simonw commented Jun 17, 2026

Uh oh!

simonw Jun 17, 2026

Choose a reason for hiding this comment

Uh oh!

simonw Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

simonw Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

simonw commented Jun 22, 2026

Uh oh!

simonw commented Jun 22, 2026

Uh oh!

simonw Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

simonw Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

simonw commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

simonw commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

simonw commented Jun 22, 2026

Uh oh!

simonw commented Jun 22, 2026

Uh oh!

simonw commented Jun 22, 2026

Uh oh!

simonw commented Jun 22, 2026

Uh oh!

simonw commented Jun 22, 2026

Uh oh!

simonw commented Jun 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

simonw commented Jun 17, 2026 •

edited

Loading

codecov Bot commented Jun 17, 2026 •

edited

Loading

simonw commented Jun 22, 2026 •

edited

Loading

simonw commented Jun 22, 2026 •

edited

Loading