feat(api): expand the public v2 tables surface - #6188
Conversation
Adds 16 operations so a v2 caller can do what the internal surface can: rename/move/lock a table, restore it, manage saved views, run enrichment columns, look up rows, and import/export with observable job control. Extracts lib/table/orchestration/import.ts (performTableCsvImport, performCreateTableFromCsv) and lib/table/export-stream.ts from the first-party routes, then repoints those routes at them, so v1 and v2 cannot drift on what an import or export actually does. events/stream, metadata and dispatches stay internal — they are editor state, not public API.
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
PR SummaryMedium Risk Overview Table resources in list/create examples and the OpenAPI adds paths for views, groups, Implementation alignment: the first-party Notable behavioral contracts spelled out in docs: non-atomic Reviewed by Cursor Bugbot for commit 0427ff7. Bugbot is set up for automated code reviews on this repo. Configure here. |
Greptile SummaryThe PR substantially expands the public v2 Tables API while extracting shared import/export orchestration for first-party and public routes.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains; both previously reported PATCH error-handling issues are fixed in the current implementation.
|
| Filename | Overview |
|---|---|
| apps/sim/app/api/v2/tables/[tableId]/route.ts | Adds table rename/move handling and correctly preserves schema signaling and applied-operation details across partial-write and final re-read failures. |
| apps/sim/lib/api/contracts/v2/tables.ts | Defines the expanded public v2 table, view, group, enrichment, and asynchronous job contracts. |
| apps/sim/lib/table/orchestration/import.ts | Extracts reusable table-import orchestration for both public and first-party routes. |
| apps/sim/lib/table/export-stream.ts | Centralizes export generation so first-party and public job paths share export behavior. |
| apps/sim/app/api/v2/tables/[tableId]/groups/route.ts | Adds workspace-scoped workflow and enrichment group management with coordinated column mutations. |
| apps/sim/app/api/v2/tables/[tableId]/import-async/route.ts | Adds asynchronous import startup using workspace-scoped uploaded file keys. |
| apps/sim/app/api/v2/tables/[tableId]/export-async/route.ts | Adds asynchronous export startup using the shared job and export infrastructure. |
| apps/docs/openapi-v2-tables.json | Documents the expanded v2 Tables API, canonical envelopes, resources, and asynchronous job lifecycle. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart LR
Client[Public API client] --> V2[Tables API v2]
V2 --> Tables[Table lifecycle]
V2 --> Rows[Rows and search]
V2 --> Views[Saved views]
V2 --> Groups[Workflow and enrichment groups]
V2 --> Jobs[Async import and export jobs]
Tables --> Orchestration[Shared table orchestration]
Rows --> Orchestration
Groups --> Orchestration
Jobs --> Orchestration
Internal[First-party table routes] --> Orchestration
Orchestration --> DB[(Table storage)]
Jobs --> Objects[(Workspace object storage)]
Reviews (9): Last reviewed commit: "improvement(api): make v2 table import a..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
…ry 423 Greptile P1: PATCH applied locks, rename and move as three sequential transactions, so a folder rejected mid-request left the earlier writes persisted while the response reported failure — and the schema-changed signal was skipped, leaving open clients on stale state. Every rejectable condition now runs before the first write, and the signal fires whenever anything did land. Cursor: v2TableLockError dropped the lock kind, so async import, column run, enrichment and table mutations returned a bare LOCKED. A table has four independent locks, so the caller could not tell which to clear.
|
@cursor review |
…n ones
The previous commit named the lock only where the rejection was thrown and
caught at the route boundary. Where it instead arrives as a classified
`errorCode: 'locked'` outcome — delete table, delete row, update column,
and the table mutations — the kind was dropped, so those 423s stayed
unactionable while their neighbours improved.
The orchestration results now carry `lock`, and a shared
`v2TableOrchestrationError` renders both arrival paths into the same
`{ code, message, details: { lock } }` body. `details` is omitted rather
than sent null when the kind is unknown, so a caller branching on it sees
absence instead of a phantom value.
|
@cursor review |
`POST /import-async` pointed callers at `GET /api/v2/tables/jobs` to track progress, but that endpoint filters to `type = 'export'` — imports are derived onto the table itself, one write job at a time, and exports get a separate list precisely because they are excluded from that derivation. The public Table shape omitted those derived fields, so an async import could be started and cancelled but never observed to completion, failure, or progress. That is the gap the import/export/job-control set was meant to close. Table now carries `job` — id, type, status, rowsProcessed, error, or null when idle — and the import-async docs point at the table rather than the export list.
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 83e04cb. Configure here.
Greptile held the PR at 4/5 on the residual non-atomicity and named two acceptable resolutions: make PATCH atomic, or have the contract adopt and expose partial-success explicitly. Atomicity would mean threading one transaction through renameTable, moveTableToFolder and updateTableLocks — three shared service functions with four non-test callers including the first-party route and two copilot tools — and deferring their per-operation audits to commit time. That is a refactor of shared write paths well outside this PR. So the contract states it instead. Every rejectable condition is already pre-validated, so a failure here is a genuine fault; when one follows a successful operation the error now carries `details.applied` listing what is live. Absent when nothing applied, so its presence always means "these changes took effect despite the error". Documented on the operation. `v2ErrorForOrchestration` gained the optional `details` this needs.
|
Addressing the 4/5 hold on non-atomic PATCH (b25c426) — the finding is fair, and I took the second of the two resolutions you named. Why not atomicity. It would require one transaction spanning What landed instead. The endpoint now adopts partial-success explicitly rather than implying atomicity it does not have:
Two tests pin both halves: a move failing after a successful rename asserts This required giving Worth flagging for the human reviewer: the first-party |
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit b25c426. Configure here.
…eat/v2-tables-coverage # Conflicts: # scripts/check-api-validation-contracts.ts
The new PATCH /api/v2/tables/[tableId] accepted a `locks` object, gated on workspace admin plus the table-locks feature. That still lets an API key clear the guard placed there to stop it: `write` is the floor for the endpoint, and admin keys are ordinary API keys, so a lock is no longer a boundary the key cannot cross. Locks stay readable on the table resource and enforcement is unchanged (a locked verb still returns 423). Changing one is now a first-party admin action only. The v2 body is declared here rather than reusing the first-party updateTableBodySchema, which keeps its `locks` field so the UI can still toggle them. It is .strict(), so a request carrying `locks` is rejected with a 400 naming the field instead of silently succeeding without applying it.
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 178a20d. Configure here.
The composite table PATCH promises that `error.details.applied` names the operations that are live despite an error, but `applied` was scoped inside the try. A rename or move that committed and was then followed by a throw in the final re-read — or a re-read finding the table archived — returned a bare 500/404 with no details, telling the caller nothing had landed. It would then retry into a duplicate-name conflict or repeat the move. `applied` is now function-scoped so every post-write exit carries it: the 404 on a missing re-read, a thrown lock error, a classified orchestration error, and the generic 500. `v2TableLockError` gains the same `extraDetails` parameter `v2TableOrchestrationError` already had.
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 2366a4e. Configure here.
v2 exposed GET /groups but none of the writes, so the public API could run an enrichment or workflow column and read its binding, but never create one. A caller could add a plain data column and trigger the machine; wiring the two together still required the UI. Adds POST/PATCH/DELETE on /api/v2/tables/[tableId]/groups. The group is the unit that fills columns — one group feeds several — so creating one creates its output columns in the same call, matching the first-party shape rather than inverting it onto the column endpoint. Four departures from the first-party body, all public-surface concerns: - group.id is optional and server-generated. The UI mints an id to render optimistically; a public caller has no such need and a client-chosen id is a collision waiting to happen. - outputColumns[].workflowGroupId is dropped from the body and stamped from the resolved group, so it cannot disagree with it. - autoRun defaults to false. First-party defaults true so a UI add fills cells immediately; here it would make one POST fan out a metered run across every existing row. - A group naming neither a workflowId (type manual) nor an enrichmentId (type enrichment) is a 400 rather than a half-specified group the route has to guess about. Also rejects an outputColumns entry no group output feeds — the two arrays are joined by column name, and the first-party client builds both from one picker so it cannot desync, but a public caller can. Workspace containment on workflowId is asserted before it is persisted, on create and on any update that re-points the group; without it a table becomes a way to invoke workflows the key cannot otherwise reach.
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 03e6ee9. Configure here.
Drops the three synchronous entry points: POST /tables/[tableId]/import, POST /tables/import-csv, and GET /tables/[tableId]/export. Sync import tied a write to the lifetime of an HTTP request. The body *was* the data, so it carried a 10 MB cap that Next silently truncates past — a partial import reporting success. It also had no job, so a timeout mid-write left rows in place with nothing to poll and nothing to cancel. The async path reads the file from storage instead: upload via POST /api/v2/files for a key, start with POST /import-async, watch GET /tables/[tableId] -> job, stop with POST /job/cancel. Sync export carried no such hazard, but one shape per operation beats two: with both removed the surface has exactly one way to move a table in or out, and the CLI wraps the extra calls. This also removes the last multipart handling in v2 tables. Those were the only routes bypassing parseRequest — form fields were parsed by hand against separate form schemas, outside the contract system every other v2 write goes through. Create-a-table-from-CSV is now two calls: POST /tables, then /import-async with createColumns. csvImportModeSchema is append|replace, so there is no single-call create. Route baseline 1064 -> 1061.
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit f6f5084. Configure here.
The docstring claimed there is no synchronous upload endpoint and so no request-body size cliff. Both are wrong: POST /api/v2/files is a synchronous multipart upload with a 100 MB cap, and it is the only v2 upload path (presigned is deliberately absent). What async-only actually bought: the cap went 10 MB -> 100 MB, it fails on an explicit size check and a bounded body read rather than a proxy cap that silently truncates, authorization completes before any body is buffered, and the table write is a job that can be watched and cancelled.
1 participant
Summary
/api/v2/tablesso a public caller can do what the internal surface can — previously it could read and write rows but not rename a table, restore one, manage views, define or run an enrichment column, or import/exportPATCH /tables/[tableId](rename/move — neither v1 nor v2 had it),POST /restore, saved views, workflow/enrichment groups (GET/POST/PATCH/DELETE /groups),POST /columns/run+POST /rows/[rowId]/enrichment/[groupId],POST /rows/findlib/table/orchestration/import.tsandlib/table/export-stream.tsout of the first-party routes and repoints those routes at them, so v1 and v2 can't drift on what an import or export doesevents/stream,metadataanddispatchesstay internal — editor state, not public APIAPI schemas
Every response is the canonical v2 envelope, with no exceptions:
{ data }for single/mutation,{ data, nextCursor }for lists,{ error: { code, message, details? } }for failures. Exports hand back a URL rather than a file body, so no route streams.Tables
dataGET /tables?workspaceId&folderId&search&sortBy&sortOrder&limit&cursorTable[]+nextCursorPOST /tables{ workspaceId, name, description?, folderId? }{ table }GET /tables/{id}?workspaceId{ table }PATCH /tables/{id}{ workspaceId, name?, folderId? }—.strict(), ≥1 of name/folderId{ table }DELETE /tables/{id}?workspaceId{ id }POST /tables/{id}/restore{ workspaceId }{ table }Table={ id, name, description, schema: { columns }, rowCount, maxRows, folderId, locks, job, createdAt, updatedAt }.locks={ schemaLocked, insertLocked, updateLocked, deleteLocked }— read-only on this API. See the note below.Columns
dataPOST /columns{ workspaceId, column: { name, type, required?, unique?, position?, options?, multiple?, currencyCode? } }{ columns }PATCH /columns{ workspaceId, columnName, updates: { name?, type?, required?, unique?, options?, multiple?, currencyCode? } }{ columns }DELETE /columns{ workspaceId, columnName }{ columns }Columns are addressed by name, so a rename changes the handle.
typeis one ofstring | number | currency | boolean | date | json | select.Rows
dataGET /rows?workspaceId&limit&cursor&…Row[]+nextCursorPOST /query{ workspaceId, filter?, sort?, limit?, cursor? }Row[]+nextCursorPOST /rows{ workspaceId, rows[] }{ rows, insertedCount }PUT /rows{ workspaceId, filter, updates }{ updatedCount, updatedRowIds }DELETE /rows{ workspaceId, rowIds? | filter? }{ deletedCount, deletedRowIds, requestedCount?, missingRowIds? }GET/PATCH/DELETE /rows/{rowId}?workspaceId/{ workspaceId, data }{ row }/{ deletedCount, deletedRowIds }POST /rows/upsert{ workspaceId, match, data }{ row, operation: 'insert' | 'update' }POST /rows/find{ workspaceId, query, filter?, sort?, … }Row={ id, data, createdAt, updatedAt }—datakeyed by column name, no storage internals. Filters speak only the typed predicate tree ({ all \| any: [{ field, op, value }] }); the v1$-operator dialect is not accepted.Views
GET /views→View[];POST /views,GET/PATCH/DELETE /views/{viewId}→{ view }/{ id }.Workflow & enrichment groups
A group is the unit that fills columns — one group can feed several. Creating one creates its output columns in the same call.
dataGET /groups?workspaceIdWorkflowGroup[]+nextCursor(alwaysnull— bounded per table)POST /groups{ workspaceId, group, outputColumns[], autoRun? }{ group, columns }(201)PATCH /groups{ workspaceId, groupId, workflowId?, name?, outputs?, newOutputColumns?, mappingUpdates?, inputMappings?, deploymentMode?, type?, autoRun? }{ group, columns }DELETE /groups{ workspaceId, groupId }{ id, deleted: true, columns }group={ id?, workflowId, enrichmentId?, name?, type: 'manual' \| 'enrichment', dependencies?, outputs[], inputMappings?, deploymentMode?, autoRun? }.outputColumns[]={ name, type, required?, unique? }.Both mutations return the group and the resulting column list, because one call changes both.
Async import / export
Every import and export is a background job — there is no synchronous variant. The two families differ only in how you observe them, because of what they touch.
POST /import-async{ workspaceId, fileKey, fileName, mode, mapping?, createColumns?, timezone? }→{ tableId, importId }POST /export-async{ workspaceId, format: 'csv' | 'json' }→{ tableId, jobId }GET /tables/{id}→.jobGET /tables/jobs?workspaceId&type=export→TableJobSummary[]GET /export/download?workspaceId&jobId→{ url, fileName }POST /job/cancel{ workspaceId, jobId }→{ jobId, canceled }Why observation is split. A table can have only one write job in flight, so the job is derived onto the table itself and
GET /tables/{id}is its status endpoint:Exports are read-only and run concurrently, so they can't be derived onto one table field and get the dedicated list instead:
{ "jobId", "tableId", "tableName", "status", "rowsProcessed", "format": "csv"|"json", "hasResult": true, "error": null }Cancellation is unified.
POST /job/cancelresolves the job's real type from its own row, so it stops exports as well as imports and deletes. It flips the status so the worker's next ownership check fails — work already committed is not rolled back, and cancelling a finished job returnscanceled: falserather than erroring, so a client racing the worker isn't a failure case.Getting a
fileKey.POST /api/v2/filesreturns{ key }; that key is whatimport-asynctakes — both sides use the workspace storage context. It is the general Files-module upload, not an import-specific one.Worth being precise about what async-only did and didn't buy: the upload step is still synchronous multipart, capped at 100 MB. The byte limit moved rather than vanished. What changed is that it went 10 MB → 100 MB, it fails on an explicit
file.sizecheck plus a bounded body read instead of a proxy cap that silently truncates, authorization completes before any body is buffered, and — the actual point — the table write is now a job with status and cancel. Presigned upload is deliberately absent (presign is advisory; a caller who presigns, PUTs, and never registers leaves unaccounted storage), so multipart is the only v2 upload path by design.Creating a table from a CSV is two calls:
POST /tablesthen/import-asyncwithcreateColumnsnaming the headers to add.csvImportModeSchemaisappend | replace, so there is no single-call create.Two rough edges I did not smooth over, since they're pre-existing shapes and renaming them is a breaking change:
importId(import-async),jobId(export-async, download, cancel),job.id(on the table). They're the same value.GET /tables/jobstakestype: 'export'as a literal — it can't list import jobs even though the summary shape would carry them. Import status is only observable via the table.Notes for review
TablegainsfolderIdandlocks(toApiTable,v2ApiTableSchema, OpenAPITable). Without themPATCHcould write a field the surface couldn't read back. Additive; existing response examples updated.locksonPATCH, gated on workspaceadmin+ thetable-locksfeature. That still let an API key clear the guard placed there to stop it —writeis the floor for the endpoint and admin keys are ordinary API keys, so a lock stopped being a boundary the key couldn't cross.locksis still returned on every read and still enforced (a locked verb returns 423); changing one is a first-party admin action. The v2 body is declared separately rather than reusing the first-partyupdateTableBodySchema, which keeps itslocksfield so the UI still works, and is.strict()so a request carryinglocksgets a 400 naming the field instead of silently succeeding.workflowIdbefore persisting it, on create and on any update that re-points the group — without it a table becomes a way to invoke workflows the key cannot otherwise reach.group.idis optional and server-generated (the UI mints one to render optimistically; a client-chosen id is a collision waiting to happen);outputColumns[].workflowGroupIdis dropped from the body and stamped from the resolved group;autoRundefaults to false (first-party defaults true so a UI add fills cells immediately — here it would fan out a metered run across every existing row); and a group naming neither aworkflowIdnor anenrichmentIdis a 400 rather than something the route guesses at. It also rejects anoutputColumnsentry that no group output feeds — the two arrays are joined by column name, and the first-party client builds both from one picker so it can't desync, but a public caller can.PATCH /tables/{id}is not atomic and says so. The two operations commit independently; everything that can be rejected is validated before the first write, so a rejected request changes nothing. If a later operation faults after an earlier one committed, the error carriesdetails.appliednaming what is live.appliedis function-scoped so every post-write exit reports it — including a throw in the final re-read and a re-read finding the table archived, both of which previously returned a bare 500/404 implying nothing had landed.runColumnBodySchemaandcancelTableRunsBodySchemawere split into un-refined base objects plus shared refine helpers (following the existinginsertTableRowBodyBaseSchema/rowAnchorMutexRefineprecedent) — Zod forbids.extend()on a refined schema and v2 narrowsfilterto predicate-only. Internal behavior unchanged.{ data }; there is nomode: 'stream'route left.parseRequest, parsing form fields by hand against separate form schemas, outside the contract system every other v2 write goes through.lockfield was being dropped. The orchestration now threads the lock kind through, so v1 renders{ error, lock }again (nodetails, which would make the client swallow the toast) and v2 surfaces it asdetails: { lock }.improvement/v2-endpoints, same as feat(api): complete the v2 workflows resource with versions and CRUD #6184 and feat(cli): Sim CLI with AWS-style profiles and a platform key exchange #6147.Known gap
There is no way to detach a column from its group while keeping its values. Both removal paths —
DELETE /columnson a bound column, andPATCH /groupswith a shortenedoutputs[]— delete the column and its data; the service comment is explicit that "both paths strip values out of every row". Documented in the PATCH description so integrators aren't surprised. Closing it is new table behavior rather than API surface, so it's deliberately not in this PR.Type of Change
Testing
bun run check:api-validation:strict— passes (route baseline 1046 → 1061)bun run check:openapi— passes, 114 operations / 105 contracts cross-checkedbun run type-check,bun run lint— passbunx vitest run app/api/v2 app/api/table lib/table— 105 files, 1477 tests pass, including the first-party group and table routes this PR refactorsroute.test.tsper new route: gate off → 404, invalid body → 400, access denied → 404 (masked), rate limited → 429, happy path asserting the exactdatashape and the lib call. Group writes additionally cover the server-generated id, theautoRundefault, foreign-workspace workflow rejection, the orphan-column guard, and both arms of the type refineChecklist