Skip to content

feat(rows): add persistent UUIDv7 identifiers and UUID-based API access - #3229

Open
foolfitz wants to merge 7 commits into
nextcloud:mainfrom
foolfitz:feat/3093-row-uuid-squashed
Open

foolfitz wants to merge 7 commits into
nextcloud:mainfrom
foolfitz:feat/3093-row-uuid-squashed

Conversation

@foolfitz

@foolfitz foolfitz commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Related issue: #3093

Add a persistent UUIDv7 to each row so that it can still be identified after export and import through user migration, even if its integer ID changes. The existing integer primary keys and cell associations remain unchanged.

Main changes

  • Store UUIDv7 values in tables_row_sleeves and add a unique index covering all rows across the instance. Once assigned, a UUID cannot be replaced or cleared.
  • Assign UUIDs to new rows when they are inserted. Backfill existing rows through a resumable background job to avoid processing all historical rows synchronously during the schema migration. During backfill, UUIDs may be null, and rows remain accessible by numeric ID.
  • Preserve row UUIDs in user migration archives. On import, retain a UUID if it is unused on the destination. If a collision occurs, generate a new UUID for the imported copy without overwriting the existing row. Generate new UUIDs for older archives that lack them, and reject imports containing invalid UUIDs.
  • Add a single-row GET endpoint at /api/2/{tables|views}/{nodeId}/rows/{rowId} in API v2, and allow GET, PUT, and DELETE to accept either an integer ID or a UUID. UUID lookups retain the existing permissions, view filters, and column restrictions. Malformed identifiers return 400; rows not found within the authorized scope return 404. Add uuid to row responses and update the OpenAPI and TypeScript definitions together.

The UUID API provided by this PR can read, update, or delete rows on the instance receiving the request. For example, after importing data from Site A into Site B, UUID requests sent to Site B operate only on Site B's data and are not forwarded to Site A.

This defines the scope of this PR. Work left for later includes federated UUID forwarding, UUID addressing on public share token routes, cross-site synchronization, and full NOT NULL enforcement. UUID preservation applies to user migration archives; CSV import does not yet support migrating rows while retaining their source UUIDs.

Regarding the database migration: before merging, confirm with the maintainers which Tables release will include this change and align the migration's version identifier accordingly. This ensures that upgrading an existing installation automatically adds the UUID column and index and schedules background backfill for existing rows.

Testing

Manual testing: preserving UUIDs after archive migration

Using version 0e70ed8c, data with two text columns, Item and Status, was migrated through a User migration archive between two independent Nextcloud 33.0.8 / SQLite environments, with Site A as the source and Site B as the destination.

1. Create data on Site A and record the original ID and UUID

A row containing Transfer notebook / keep was created, and its identifiers were queried. The request returned HTTP 200, integer ID 4, and UUID 01a1ab40-6328-75f1-8976-3cd3e2a98dd8.

M2-source-id-uuid

2. Export a user migration archive from Site A

The user:export --types=tables command exported the Tables data, and the terminal reported a successful export. A separate inspection of the archive confirmed that it preserved the row's UUID and original content.

M3-export-completed

3. Import the same archive into Site B

The user:import command imported the archive into a dedicated empty account on Site B. The terminal reported that the Tables import completed, with the data written to Site B's own database.

M4-import-completed

4. Query the source row on Site A using the original UUID

A GET request using the original UUID was sent from Site A's browser session. It returned HTTP 200, with integer ID 4 and the same UUID as before export.

M5-source-live-get

5. Query the imported row on Site B using the same UUID

A GET request using the same UUID was sent from a separate browser session on Site B. It returned HTTP 200. The integer ID had been reassigned to 108, while the UUID still matched Site A's and the content remained Transfer notebook / keep. This confirmed that the original UUID could still locate the data after migration.

M5-destination-live-get

6. Update the content on Site B using the UUID

A PUT request using the UUID path changed Item to Notebook after transfer. It returned HTTP 200, with integer ID 108, an unchanged UUID, and Status still set to keep. The table in the screenshot had not yet been reloaded; the PUT response confirms the update.

M6-put-by-uuid

7. Query Site B again to confirm that the update was saved

A separate GET request using the UUID returned HTTP 200, with Item set to Notebook after transfer and both the integer ID and UUID unchanged. The reloaded table also displayed the new value.

M6-update-after-transfer

8. Query Site A again to confirm that the source data was unaffected

A final GET request to Site A using the original UUID returned HTTP 200, with integer ID 4, the same UUID, and the original content, Transfer notebook / keep. This confirmed that Site B's update had not synchronized back to Site A.

M6-source-unchanged

This manual test covers archive migration and local UUID GET/PUT operations. It does not cover DELETE, UUID collisions, historical backfill, view-sharing permissions, or federation operations.

Automated verification and upgrade measurements

Local verification records show that the full PHPUnit suite, MySQL/MariaDB/PostgreSQL database tests, Psalm, coding standards, PHP syntax checks, and TypeScript checks all passed. Regenerating the OpenAPI specification and types produced no differences.

An earlier SQLite benchmark with 1,000,014 rows recorded approximately 1.44 seconds for the schema/index migration and 41.48 seconds of total background worker time. All 165 sampled HTTP requests during backfill returned 200. This measurement predates the final cursor-write adjustment and has not been repeated; total worker time does not represent elapsed completion time under the normal cron schedule.

🏁 Checklist

  • ⛑️ Tests (unit and/or integration) are included or not needed
  • 🔙 Backport requests are created or not needed: /backport to stableX.X
  • 📅 Milestone is set
  • 🌸 PR title is meaningful (if it should be in the changelog: is it meaningful to users?)

🤖 AI (if applicable)

  • The content of this PR was partly or fully generated using AI

Codex assisted with developing the code and tests, and Claude Code assisted with their review. I performed the manual migration test and captured the screenshots; Codex assisted with drafting the PR description.

foolfitz and others added 5 commits November 5, 2026 10:31
Add a nullable, uniquely indexed uuid column to tables_row_sleeves.
Every sleeve insertion, including legacy transfers with explicit IDs,
assigns a UUIDv7 before writing. An assigned identity is normalized to
lowercase and cannot change or be cleared. Stored values load unchanged,
so rows created before this release keep NULL until they are backfilled.

User migration archives now carry each row's UUID. Import preserves an
unused UUID, generates one when the archive has none, and creates the
row with a fresh UUID when the identity is already taken. A unique
violation is retried only after a plain read confirms the UUID
collision, so other constraint failures surface unchanged. Invalid
archived identities fail before anything is written.

Refs: nextcloud#3093
Assisted-by: Codex:gpt-6
Assisted-by: Codex:gpt-6-astra
Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jiajun Xu <foolfitz@gmail.com>
Register a periodic job from the migration that assigns UUIDv7 values
to sleeves still holding NULL. A run processes batches of up to 500
rows for about five seconds and commits each batch. The update only
touches rows whose uuid is still NULL, so concurrent assignments and
deletions are respected and audit metadata stays unchanged. Parallel
runs are disabled.

Progress is a lazy app-config cursor written once per run. A lagging
cursor only repeats a cheap NULL scan, while writing it after every
batch would invalidate the shared app-config cache. When a pass finds
nothing left, the job rescans from the start for rows below the cursor
before it removes itself. On SQLite the job pauses briefly between
batches so requests can take the single writer.

Refs: nextcloud#3093
Assisted-by: Codex:gpt-6
Assisted-by: Codex:gpt-6-astra
Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jiajun Xu <foolfitz@gmail.com>
Row responses now include the persisted uuid next to the integer id,
including public-share rows and the row returned by POST. Rows created
before the backfill reaches them report null. Row responses forwarded
from a federated server keep the remote uuid, or report null when the
remote server does not send one.

Regenerate openapi.json and the TypeScript types for the new field.

Refs: nextcloud#3093
Assisted-by: Codex:gpt-6
Assisted-by: Codex:gpt-6-astra
Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jiajun Xu <foolfitz@gmail.com>
Add a RowIdentifier value object for positive integer IDs within the
signed 32-bit range of the id column and for UUIDv7 strings. The row
service resolves it inside the requested table or view, evaluating view
filters with a bounded single-row query and the same user placeholders
as listings, so knowing an identifier grants no access to other rows.

Rows that are missing, deleted, in another table or outside a view are
now reported as not found instead of as internal errors, including a
row removed between loading and updating it. A spreadsheet import
counts such a row as one failed line and continues. View projections
work on a copy instead of shortening a shared row object, and listings
of nodes without columns still return no rows. Row updates accept a
RowDataInput so that callers can validate cells first.

Refs: nextcloud#3093
Assisted-by: Codex:gpt-6
Assisted-by: Codex:gpt-6-astra
Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jiajun Xu <foolfitz@gmail.com>
Add GET /api/2/{nodeCollection}/{nodeId}/rows/{rowId} and let the
existing PUT and DELETE routes accept either a positive numeric ID or a
UUIDv7. Invalid identifiers and malformed cell lists return 400 before
any lookup, and rows outside the requested table or view return 404.
PUT and DELETE resolve only the row ID before mutating, and service
failures are no longer reported as client errors.

Federated nodes keep numeric PUT and DELETE forwarding. UUID addressing
and the new GET return 400 there, because the federation protocol has
no equivalent yet. The existing row routes stay without a rate limit,
since the web UI saves every cell and deletes each selected row through
them.

Regenerate openapi.json and the TypeScript types.

Refs: nextcloud#3093
Assisted-by: Codex:gpt-6
Assisted-by: Codex:gpt-6-astra
Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jiajun Xu <foolfitz@gmail.com>
@foolfitz
foolfitz marked this pull request as ready for review November 6, 2026 07:02
@foolfitz foolfitz changed the title feat(rows): add persistent UUIDv7 identifiers and UUID-based API access #3093 feat(rows): add persistent UUIDv7 identifiers and UUID-based API access Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant