From b575531fdde711cdde823a45359eb6970b9e53b0 Mon Sep 17 00:00:00 2001
From: Necco Ceresani
Date: Wed, 26 Aug 2026 10:25:15 -0400
Subject: [PATCH 1/2] docs: rebuild README
Corrects the keybinding table, which documented a scroll-only feed while the
code has a cursor model, an errors-only toggle, and a detail overlay on Enter.
Adds the questions, agent-setup and without-a-TTY sections, a Contents block,
and three previously undocumented limits (blobs, database-file attribution,
the 512-byte text cap).
---
README.md | 371 +++++++++++++++++++++++++++++++++++++++---------------
1 file changed, 271 insertions(+), 100 deletions(-)
diff --git a/README.md b/README.md
index 789663c..449fcbf 100644
--- a/README.md
+++ b/README.md
@@ -2,172 +2,313 @@
# `sqlitefeed`
-> **`tail -f` for SQLite.** Every statement any process on the box runs against `libsqlite3` — the SQL (syntax-highlighted), the values bound to its `?` placeholders, the per-step latency and result code — decoded and streamed live to your terminal. No cooperation from the traced apps, no recompile, no `PRAGMA`.
+> **Every SQL statement your app really sent SQLite, including the values it bound to the `?`s.** No query log, no `PRAGMA`, no recompile, no cooperation from the process you're watching.
-
-
-
-
+
+
+
+
+
-
+
-**`sqlitefeed` uprobes the public `libsqlite3` API — `prepare` / `bind` / `step` / `exec` — and turns it into a live, newest-first feed of prepared statements: the SQL, the concrete bound values, rows returned, worst step latency (heat-colored), and the result code — for every process using the shared library at once.**
+**`sqlitefeed` is a live terminal SQLite statement monitor for Linux: it streams every statement any process on the box runs against `libsqlite3`, with the concrete bound values, per-step latency, and result code.**
-Because it attaches to the *library*, not to any one app, a single run watches every SQLite-backed process on the host — your app, a background job, and a `sqlite3` shell — with none of them aware they're being traced.
+## Quick start
+
+```sh
+curl -fsSL https://yeet.cx | sh # install yeet, once
+yeet run gh:yeet-src/sqlitefeed # clone, build and run in one step
+```
+
+SQLite has no query log. It is a library linked into your process, not a server you can ask, so the usual answer is to add logging inside the application: a tracing callback, an ORM `echo` flag, a `PRAGMA` nobody remembers the name of. That works for the one app you thought to instrument, and it is useless for the background job that starts tomorrow, a `sqlite3` one-liner someone runs by hand, or the vendored binary whose source you don't have.
+
+`sqlitefeed` attaches to the *library* instead of the app. One run watches every SQLite-backed process on the host at once, and none of them know they're being traced. Where you'd otherwise reach for `strace` and read `pread64` offsets, or bisect an ORM until it confesses what SQL it generated, you get the statement text and its bound parameters directly.
> [!TIP]
-> **You can't just add logging.** Query logging lives inside one application, compiled in or configured per-process; it can't see the job that starts tomorrow, and it certainly can't see a `sqlite3` one-liner. `sqlitefeed` hooks the shared library **in the kernel**, so it sees every caller uniformly — and it even recovers the SQL of statements that were prepared and cached *before* it attached, by reading `Vdbe.zSql` straight out of the statement struct (see [Recovering SQL it never saw prepared](#recovering-sql-it-never-saw-prepared)).
+> **The bound values are the point.** A query log that shows you `WHERE score > ?` has told you almost nothing; the bug is usually in the `?`. sqlitefeed hooks `sqlite3_bind_int`, `_int64`, `_text` and `_null` alongside `prepare` and `step`, so each row carries the actual values the application substituted, correlated by the `sqlite3_stmt*` pointer.
-## Quick start
+## Contents
+
+**Run it** — [Get started](#get-started) · [Have an agent set it up](#have-an-agent-set-it-up) · [Reading it without a TTY](#reading-it-without-a-tty)
+**Understand it** — [A 60-second primer on how SQLite runs a statement](#a-60-second-primer-on-how-sqlite-runs-a-statement) · [Questions this tool answers](#questions-this-tool-answers) · [What you're looking at](#what-youre-looking-at) · [Navigation](#navigation) · [How it works](#how-it-works)
+**Reference** — [Requirements](#requirements) · [What it can't see](#what-it-cant-see) · [FAQ](#faq)
+**Contribute** — [Building from source](#building-from-source) · [Testing across kernels](#testing-across-kernels) · [Try it without real traffic](#try-it-without-real-traffic)
+
+## Get started
```sh
-curl -fsSL https://yeet.cx | sh # install the yeet daemon (one time)
-make # build the BPF object + JS bundle
-yeet run . # run the dashboard (the daemon does the privileged BPF load)
+curl -fsSL https://yeet.cx | sh
+make # clang + bpftool → bin/sqlite.bpf.o ; esbuild → the JS bundle
+yeet run . # attach to libsqlite3.so.0 and stream every statement on the host
```
-[Manual install guide](https://yeet.cx/docs/manual-installation) | Linux only
+[Manual install guide](https://yeet.cx/docs/manual-installation?utm_source=github&utm_medium=readme&utm_campaign=sqlitefeed) | Linux only
-Nothing to configure — as soon as any process prepares or executes a statement, rows start landing at the top. No traffic handy? The bundled generators drive `libsqlite3` for you:
+There is nothing to configure and there are no flags. As soon as any process on the box prepares or executes a statement, rows start landing at the top of the feed. No SQLite traffic handy? [The bundled generators](#try-it-without-real-traffic) drive `libsqlite3` for you.
+
+It runs until you press `q` (or `Ctrl-C`), reflows when you resize the terminal, and needs a real TTY. Don't pipe or redirect it; for text output see [Reading it without a TTY](#reading-it-without-a-tty).
+
+## Have an agent set it up
+
+Paste this to a coding agent on the target Linux box:
-```sh
-demo/run.sh # a python workload + a trickle of sqlite3-CLI queries
-python3 demo/traffic.py # just the python workload (Ctrl-C to stop)
```
+Set up and verify github.com/yeet-src/sqlitefeed on this machine.
+
+1. Clone it (or `git pull` if it's already here) and read AGENTS.md.
+2. Install yeet if it isn't present: curl -fsSL https://yeet.cx | sh
+3. Run `make`. It fetches its own clang/bpftool/esbuild, so a missing system
+ toolchain is not an error.
+4. Confirm libsqlite3.so.0 is on the host: `ldconfig -p | grep libsqlite3`.
+ If it's missing there is nothing to attach to and the probe will fail.
+5. Start traffic in a second shell: `demo/run.sh`
+6. Verify from the headless probe, NOT the TUI:
+ `yeet run src/probes/sqlite.js`
+ Expect [PREPARE]/[BIND]/[STEP] lines within a few seconds. Ctrl-C to stop.
+7. Report the first three event lines verbatim.
+
+"It compiled" is not the same as "it works". Step 6 is the check that matters:
+if no events arrive, say so rather than reporting success.
+```
+
+Prefer to drive it yourself? [Get started](#get-started) is three lines.
+
+## A 60-second primer on how SQLite runs a statement
+
+SQLite isn't a server. It's a C library compiled into your process, so "the database" is a function call, and there is no daemon in between holding a log you could tail.
+
+Running one statement takes three steps, and each is a public C function that sqlitefeed hooks:
+
+- **`sqlite3_prepare_v2(db, "SELECT … WHERE score > ?", …)`** compiles the SQL into a bytecode program and hands back a `sqlite3_stmt*` pointer. The SQL text exists *here*, at compile time, and nowhere afterwards.
+- **`sqlite3_bind_int(stmt, 1, 500)`** substitutes a concrete value for each `?`. The SQL never changes; the values live beside it, which is why a query log alone can't tell you what actually ran.
+- **`sqlite3_step(stmt)`** executes the bytecode. It returns `SQLITE_ROW` once per row and `SQLITE_DONE` when it's finished, so one statement usually means many `step` calls, each with its own latency.
-## Controls
+That `sqlite3_stmt*` pointer is the thread tying it all together: prepare mints it, every bind and step names it. sqlitefeed keys on that pointer to reassemble one execution out of a dozen separate function calls.
-The feed follows the newest statement by default; scroll or pause and it holds still while data keeps flowing underneath.
+There's a fourth path, `sqlite3_exec(db, "CREATE TABLE …", …)`, a one-shot convenience that prepares, steps and finalizes internally. It's complete on arrival, so it shows as a single `exec` row.
-| key | action |
-| -------------- | ----------------------------------------------------------------- |
-| `↑`/`↓`, `j`/`k` | scroll (auto-holds position when you leave the top) |
-| `PgUp`/`PgDn` | scroll a page; mouse wheel also scrolls |
-| `g` | jump back to newest and resume following |
-| `/` | fuzzy filter — matches process, SQL, and bound values |
-| `p` | pause / resume the feed |
-| `q` / `Esc` | quit |
+The catch, and the reason for [one of the trickier parts of this tool](#recovering-sql-it-never-saw-prepared): applications prepare a statement **once** and reuse the handle for hours. Attach in the middle and you'll see thousands of `step` calls on pointers whose `prepare` happened long before you arrived.
+
+## Questions this tool answers
+
+**My ORM is generating some query that's slow and I can't tell what it actually sends to SQLite. How do I see the real SQL?**
+Run `yeet run .` and watch the feed. You get the compiled statement text as `sqlite3_prepare_v2` received it, after every layer of query-builder abstraction has had its say, plus the values bound to each placeholder. No ORM echo flag, and it works the same for Python, Go, Rust, or a binary you don't have source for.
+
+**How do I see which SQL a process is running right now, on a box where I can't install anything or add logging to the app?**
+That's the default mode. One `yeet run` attaches uprobes to the host's shared `libsqlite3`, so every process linked against it appears in the same feed, identified by the `comm/pid` gutter. Nothing is added to the traced application and nothing is restarted.
+
+**Can I trace SQLite queries without recompiling with SQLITE_ENABLE_SQLLOG or adding a tracing callback?**
+Yes, and that's the whole design. `sqlite3_trace_v2` and `SQLITE_ENABLE_SQLLOG` are compile- or app-level switches that only cover the process you configured. Uprobes hook the shared library in the kernel, so one attach covers every current and future caller.
+
+**One of my SQLite writes is failing with a constraint error and the app just logs "database error". How do I see which statement and which values?**
+Press `e` for the [errors-only view](#navigation). Failing statements render in red with their result code (`CONSTRAINT`, `UNIQUE`, `NOTNULL`, `BUSY`, …), and `sqlite3_exec` failures carry SQLite's own error text on a `✗` line. Press `Enter` on the row for the full SQL and every bound parameter, unclipped.
+
+**Which of my SQLite statements is actually the slow one, and is it slow every time or just occasionally?**
+The feed shows each execution's worst `sqlite3_step` latency, heat-colored, so the expensive ones are visually obvious as they scroll. Press `Enter` on one and the [detail overlay](#the-detail-overlay) aggregates p50/p95/p99 across every logged run of that exact SQL, with a sparkline of recent runs, which is what separates "always slow" from "slow when it contends".
+
+**My app is fast in tests and slow in production and I suspect it's doing far more queries than I think. How do I count them?**
+The title bar carries a running statement count plus live `steps/s` and `rows/s`. An N+1 pattern shows up immediately as the same SQL repeating with a different bound id on every row, which is a shape you can see in the feed long before you could infer it from a latency graph.
+
+**Is this a replacement for Datadog, Sentry, or my APM's database monitoring?**
+No. There's no retention, no query language, no alerting, and no fleet view; sqlitefeed keeps the most recent 2000 executions in memory on one host and forgets them when you quit. It's the live-debugging instrument you reach for when an APM has told you "the database is slow" and you need to see the actual statements and values. Use both.
+
+**When should I use this instead of `strace`, an ORM's echo flag, or SQLite's own `sqlite3_trace_v2`?**
+Reach for sqlitefeed when you want the SQL and its parameters from processes you didn't instrument, especially several at once. Reach for an ORM echo flag when you're in dev on one app and just want its queries in your own logs. Reach for `sqlite3_trace_v2` when you're building the app and want structured tracing shipped as a feature. `strace` sees `pread64` on a database file, which tells you SQLite did I/O but never what statement caused it. For traffic to a *networked* database, sqlitefeed is the wrong shape entirely; the wire is where you'd look, and [`redissnoop`](https://github.com/yeet-src/redissnoop) is the sibling for Redis.
## What you're looking at
```
- ● sqlite ▏ 1487 queries ▏ 12 steps/s ▏ 9 rows/s ▏ tracing
-python3/34412 SELECT id, username, score FROM users WHERE score > ? ORDER BY score DESC 3r 231µs DONE
-python3/34412 INSERT OR IGNORE INTO users (username, email, age, score) VALUES (?, ?, ?, ?) 0r 3.1ms DONE
+ ● sqlitefeed ▏ 1487 queries ▏ 12 steps/s ▏ 9 rows/s ▏ tracing
+python3/34412 SELECT id, username, score FROM users WHERE score > ? ORDER BY score DESC 3r 231µs DONE
+python3/34412 INSERT OR IGNORE INTO users (username, email, age, score) VALUES (?,?,?,?) 0r 3.1ms DONE
↳ ?1='alice5866' ?2='alice5866@example.com' ?3=27 ?4=«real»
-sqlite3/34530 SELECT count(*) FROM users a, users b WHERE a.score < b.score 0r 36ms DONE
-python3/34412 SELECT * FROM no_such_table WHERE oops = ? 0r 0 ERROR
+sqlite3/34530 SELECT count(*) FROM users a, users b WHERE a.score < b.score 0r 36ms DONE
+python3/34412 INSERT INTO users (username, email) VALUES (?, ?) 0r 412µs CONSTRAINT
+ ↳ ?1='alice5866' ?2='alice5866@example.com'
+python3/34412 CREATE TABLE IF NOT EXISTS sessions (id INTEGER PRIMARY KEY, …) exec 1.8ms DONE
```
-Each statement is one block: the left gutter carries the **process/pid**, the SQL flexes in the middle (ellipsized if it's wide, one terminal row per source line if it's multi-line), and three right-pinned columns give **rows returned**, **worst step latency** (heat-colored, cool → hot), and the **result code**. A dim `↳` line lists the bound parameters when the statement has any.
+Three regions. The **title bar** carries the running totals and the probe status. The **feed** fills the body, newest statement at the top. The **footer** shows the key hints, or the live filter prompt while you're typing one.
+
+Each statement is one block: the process and pid in the left gutter, the SQL flexing in the middle (one terminal row per source line, so multi-line SQL keeps its shape), and three right-pinned columns. A dim `↳` line lists the bound parameters when there are any, and a red `✗` line carries SQLite's error text when an `exec` fails.
+
+| column | meaning |
+| --- | --- |
+| `comm/pid` | the process that ran it, so several apps in one feed stay distinguishable. Blank on continuation lines of multi-line SQL |
+| SQL | the statement as `sqlite3_prepare_v2` received it, syntax-highlighted. `«unknown»` when the statement was prepared before sqlitefeed attached and [couldn't be recovered](#recovering-sql-it-never-saw-prepared) |
+| `↳` params | the concrete bound values, `?1`-indexed. Text is quoted and clipped to 24 chars; press `Enter` for the full values |
+| rows | rows this execution returned (`step` calls that came back `SQLITE_ROW`). `exec` instead of a count for a one-shot `sqlite3_exec` |
+| latency | the **worst** single `sqlite3_step` in this execution, not the total. Heat-colored on a log scale, roughly 10µs cool to 100ms white-hot |
+| result | the final result code. `OK`/`ROW`/`DONE` are the normal path in green; anything else turns the whole row red |
+
+Each row freezes the moment its execution completes and never mutates again, so a burst scrolls past as a stable log rather than a flickering aggregate. Re-running the same cached statement produces a *new* row rather than updating the old one, which is what makes an N+1 pattern visible as repetition.
+
+The SQL is colored by a small tokenizer ([`lib/sqlhl.js`](src/lib/sqlhl.js)) on the same 256-color palette as the rest of the UI: keywords in cornflower blue, identifiers near-white, string literals green, numbers gold, `?`/`:name` placeholders amber, comments and punctuation grey. An errored statement drops the highlighting and renders uniformly red, so it reads as one broken thing rather than a colorful one.
+
+## Navigation
+
+The feed follows the newest statement by default. Move the cursor off the top row and the view **holds**: it keeps showing the snapshot you're reading while statements keep arriving underneath, and the title bar shows `⏸ HOLD`. Press `g` to jump back to the newest and resume following.
+
+| key | action |
+| --- | --- |
+| `↑`/`↓`, `j`/`k` | move the cursor (holds the view once you leave the newest row) |
+| `PgUp`/`PgDn` | move ten rows; the mouse wheel moves three |
+| `Enter` | open the [detail overlay](#the-detail-overlay) for the selected statement |
+| `e` | errors-only view; press again for everything |
+| `/` | fuzzy filter, matching process, SQL, and bound values at once |
+| `p` | pause. Unlike `HOLD`, this survives jumping back to the top |
+| `g` | jump to newest and resume following |
+| `q` / `Esc` | quit (`Esc` closes the overlay or clears the filter first) |
+
+The filter is a subsequence match, so `stusr` finds `SELECT * FROM users`, and the matched characters are highlighted in place. Because the haystack includes bound values, `alice5866` finds every statement that touched that row regardless of which query it was.
+
+### The detail overlay
+
+`Enter` opens a modal view of one statement that the feed has to clip and it doesn't:
-Each row is frozen the moment its execution completes and never mutates again — so a burst scrolls past as a stable log, not a flickering aggregate.
+- **The full SQL**, wrapped rather than ellipsized, with the syntax highlighting intact.
+- **Every bound parameter** in full, with its type, unclipped.
+- **This execution**: rows, step count, selectivity (what fraction of steps returned a row), worst step, average step, and total time.
+- **Across every run of this SQL**: the processes that ran it, an error count, p50/p95/p99/max step latency, and a sparkline of the most recent 48 runs, oldest to newest.
-**The SQL is syntax-highlighted** by a small tokenizer (`lib/sqlhl.js`), on the same 256-color palette as the rest of the UI:
+The statement you opened is a frozen snapshot and never changes under you, but the cross-run panel reads the live log, so a hot query's percentiles keep moving while you watch. `Esc` or `Enter` returns to the feed.
-| token | color |
-|---|---|
-| keyword | cornflower blue |
-| identifier | near-white |
-| string literal | green |
-| number | gold |
-| comment | dim grey |
-| `?` / `:name` param | amber |
-| punctuation | mid grey |
+## Reading it without a TTY
-The result code is color-keyed: `OK` / `ROW` / `DONE` are the normal path (green); anything else — `ERROR`, `BUSY`, `CONSTRAINT`, `CORRUPT`, … — is an error, and the whole row goes red.
+A TUI is unreadable to an agent, a CI job, or an SSH session in a hurry. The data layer runs standalone and prints plain text:
-## Recovering SQL it never saw prepared
+```sh
+yeet run src/probes/sqlite.js
+```
+
+It attaches the same probes and prints one line per raw event until you `Ctrl-C` it:
-A long-running process prepares its statements once and reuses the cached handles for hours. Attach after that, and every `step` you see is for a statement whose `prepare` already happened — you'd have only the `sqlite3_stmt*` pointer and no SQL.
+```
+[sqlite] attached 11 probes on libsqlite3.so.0 — waiting…
+[PREPARE] python3/34412 stmt=0x7f2a1c0a4e28 rc=0 sql="SELECT id, username FROM users WHERE score > ?"
+[BIND] python3/34412 stmt=0x7f2a1c0a4e28 bind #1 = "500"
+[STEP] python3/34412 stmt=0x7f2a1c0a4e28 rc=100 latency=231000ns
+```
-`sqlitefeed` recovers it. `sqlite3_sql(stmt)` is essentially `return ((Vdbe*)stmt)->zSql`, which compiles to a single `mov OFFSET(%rdi),%rax`. At build time, `build/find-zsql-offset.sh` disassembles that one function in the target `libsqlite3` and reads the offset out of the instruction, baking it in as `-DZSQL_OFFSET=…`. On the first `step` or `bind` of an unknown statement, the probe reads `Vdbe.zSql` at that offset and emits a synthetic `PREPARE` — so a cached statement lights up with its real SQL, correlated identically to one we watched compile.
+This is the raw event stream, before the correlation the TUI does, so you see each `prepare`/`bind`/`step` separately rather than assembled into one row per execution. That makes it the right thing for verifying the probe works (it's step 6 of [the agent prompt](#have-an-agent-set-it-up)) and for piping somewhere, and the wrong thing for reading a busy system by eye.
-The `known` LRU map gates this to once per statement. Recovery is x86-64 only; elsewhere (or if the library can't be found) the offset is `0`, recovery is disabled, and unseen statements show as `«unknown»` until they're re-prepared.
+There is no `--json` mode. The `RingBuf.subscribe` callback in [`src/probes/sqlite.js`](src/probes/sqlite.js) holds every decoded record, so a JSON, HTTP, or Kafka sink is a branch there rather than a rewrite.
## How it works
-The core is [`src/bpf/sqlite.bpf.c`](src/bpf/sqlite.bpf.c) (kernel) and [`src/probes/sqlite.js`](src/probes/sqlite.js) (userspace).
+Three directories, one rule each: [`src/probes/`](src/probes/) is the only BPF-aware code, [`src/components/`](src/components/) is pure presentation, [`src/lib/`](src/lib/) is pure helpers. They're composed in `main.jsx` through the `@/` source alias.
+
+```
+src/
+├── main.jsx composition root: view state, keyboard + wheel input, mount
+├── probes/sqlite.js the only BPF-aware module — load, attach, fold events into a log
+├── components/
+│ ├── titlebar.jsx totals, steps/s, rows/s, probe status, hold/pause marker
+│ ├── statements.jsx the feed: highlighted, height-budgeted, variable-height rows
+│ ├── detail.jsx the Enter overlay: full SQL, all params, cross-run percentiles
+│ └── footer.jsx key hints and the live filter prompt
+└── lib/
+ ├── sqlhl.js SQL tokenizer → colored spans
+ ├── format.js rates, durations, latency heat ramp, result-code names
+ └── fuzzy.js subsequence match + matched-column positions
+```
### The BPF side
-A generic `SEC("uprobe")`/`SEC("uretprobe")` program carries no target; `probes/sqlite.js` binds each to a concrete `libsqlite3` symbol at `attach()` time. Everything is tied together by the `sqlite3_stmt*` pointer.
+[`src/bpf/sqlite.bpf.c`](src/bpf/sqlite.bpf.c) carries eleven programs. A generic `SEC("uprobe")`/`SEC("uretprobe")` names no target; `probes/sqlite.js` binds each to a concrete symbol in `libsqlite3.so.0` at `attach()` time, resolved by bare name through the linker cache.
-| Program | Attached to | What it captures |
-|---|---|---|
-| `prepare_entry`/`_return` | `sqlite3_prepare_v2` | the SQL text + the new `stmt` pointer (an out-param, known only on return) + compile `rc` |
-| `step_entry`/`_return` | `sqlite3_step` | per-call latency and result code (`ROW`/`DONE`/error); entry also triggers `zSql` recovery |
-| `exec_entry`/`_return` | `sqlite3_exec` | one-shot statements — SQL + total latency + `rc` |
-| `bind_{int,int64,text,null,double}` | `sqlite3_bind_*` | the concrete value bound to each `?` (entry-only, no pairing) |
+| program | attached to | what it captures |
+| --- | --- | --- |
+| `prepare_entry` / `_return` | `sqlite3_prepare_v2` | the SQL text, the compile result code, and the new `stmt` pointer (an out-param, known only on return) |
+| `step_entry` / `_return` | `sqlite3_step` | per-call latency and result code; entry also triggers `zSql` recovery |
+| `exec_entry` / `_return` | `sqlite3_exec` | one-shot statements: SQL, total latency, result code, and the error string on failure |
+| `bind_int`, `_int64`, `_text`, `_null`, `_double` | `sqlite3_bind_*` | the concrete value bound to each `?`. Entry-only, no return needed |
Five maps connect kernel to userspace:
-- `sql_events` — `RINGBUF`, one decoded `sqlite_event` per prepare/bind/step/exec.
-- `known` — `LRU_HASH` of statement pointers already emitted; gates `zSql` recovery to once each.
-- `prepare_scratch` / `step_scratch` / `exec_scratch` — `HASH` keyed by `pid_tgid`, a single per-thread slot that pairs each entry probe with its return (stashes args/timestamp at entry, reads and clears at return).
+- **`sql_events`** (`RINGBUF`, 512 KB) carries one `sqlite_event` per prepare, bind, step, or exec.
+- **`known`** (`LRU_HASH`, 65536) holds statement pointers whose SQL has already been emitted, gating recovery to once each. LRU so it self-bounds over a long session.
+- **`prepare_scratch`**, **`step_scratch`**, **`exec_scratch`** (`HASH`, 8192 each) pair each entry probe with its return: the entry stashes arguments and a timestamp keyed by `pid_tgid`, the return reads and deletes them.
-### The JS side
+Everything ties together through the `sqlite3_stmt*` pointer, which the kernel treats as an opaque per-statement id. The kernel deliberately does no correlation: it emits flat events and userspace assembles them.
-| file | responsibility |
-|---|---|
-| `probes/sqlite.js` | the only BPF-aware module: load the object, attach the probes, fold the event stream into an append-only log — exposes the `statements`, `stats`, and `status` signals |
-| `main.jsx` | composition root: view state (scroll / fuzzy filter / pause / freeze), all keyboard + wheel input, `mount` |
-| `components/titlebar.jsx` | status rail — queries tracked, steps/s, rows/s, and tracing/paused state |
-| `components/statements.jsx` | the statement list: syntax-highlighted, height-budgeted rows |
-| `components/footer.jsx` | key hints and the live filter prompt |
-| `lib/sqlhl.js` | SQL tokenizer → colored `` spans |
-| `lib/format.js` | pure formatters — rate, duration, latency-heat color, result-code names |
-| `lib/fuzzy.js` | subsequence fuzzy match over process + SQL + bound values |
+
+Why one scratch slot per thread rather than a nesting stack
-The model is an append-only **log of completed executions**, not a mutable per-statement aggregate. A statement is assembled in-flight (`prepare` → `bind*` → `step*`), then frozen into the log the instant it finishes — so a row already on screen never changes or jumps. A 250 ms window timer publishes one snapshot per frame, so a busy ring buffer costs one re-render, not thousands.
+Each of `prepare_scratch`, `step_scratch` and `exec_scratch` holds exactly one entry per thread, so a nested call on the same thread clobbers the outer one and that statement is missed, showing as `«unknown»` when it's later stepped. SQLite does make nested calls (reparsing `sqlite_master` mid-DDL, for instance), so this is a real failure mode, and it is the deliberate choice.
-### Why uprobes on `libsqlite3`, not a query log
+The apparently more correct alternative is a depth-counting stack. It's worse in practice. Uretprobes are silently dropped past the kernel's `maxactive` limit under rapid or nested calls, so pushes outnumber pops, the depth drifts, and every subsequent statement reads a stale frame. A missed pairing with one slot costs one statement and **self-corrects on the next top-level call**; a drifting stack corrupts everything after it, permanently.
-The public API is the seam where an application hands SQL to the engine, for *every* application, with no per-app setup. Uprobes hook it in the kernel: one attach covers every current and future process linked against the shared library, and pairing entry↔return probes is what yields per-call latency and the out-param `stmt` pointer that ties a statement's whole life together.
+
-## Testing across kernels
+### Recovering SQL it never saw prepared
+
+A long-running process prepares its statements once and reuses the cached handles for hours. Attach after that and every `step` you see belongs to a statement whose `prepare` already happened, leaving you a bare pointer and no SQL.
+
+sqlitefeed recovers it. `sqlite3_sql(stmt)` is essentially `return ((Vdbe*)stmt)->zSql`, which compiles to a single `mov OFFSET(%rdi),%rax`. At build time [`build/find-zsql-offset.sh`](build/find-zsql-offset.sh) disassembles that one function in the host's `libsqlite3`, reads the offset straight out of the instruction, and bakes it in as `-DZSQL_OFFSET=…`. No DWARF, no per-version offset table. On the first `step` or `bind` of an unknown statement the probe reads `Vdbe.zSql` at that offset and emits a synthetic `PREPARE`, so a cached statement lights up with its real SQL and correlates identically to one that was watched compiling.
+
+The `known` LRU map gates this to once per statement. Recovery is **x86-64 only**; on other architectures, or when the library can't be located, the offset is `0`, recovery is compiled out, and unseen statements show `«unknown»` until they re-prepare.
+
+### The JS side
+
+`probes/sqlite.js` folds the flat event stream into an append-only **log of completed executions**, not a mutable per-statement aggregate. A statement is assembled in flight (`prepare` → `bind*` → `step*`) in a map keyed by the `stmt` pointer, then frozen into the log the instant it finishes. That's why a row already on screen never changes or jumps, and why re-running a cached statement appends rather than updates.
+
+Getting that boundary right is most of the userspace logic. A `bind` arriving on a statement that has already stepped means a reset-and-rerun, so the previous execution is finalized and a new one starts against the same remembered SQL. A statement that stepped but never reached a terminal code, because it was superseded by a reset mid-fetch, is recorded as `DONE` rather than showing its stale initial `OK`.
+
+A 250 ms window timer publishes one snapshot per frame, so a busy ring buffer costs one re-render rather than thousands, and the log is capped at 2000 executions. Everything downstream is pure: the components read the `statements`, `stats` and `status` signals and nothing else.
-`make veristat` loads `bin/sqlite.bpf.o` with veristat on **your** kernel — a quick check that every program passes the verifier, plus per-program complexity (insns/states). Loading BPF needs privileges, so use `sudo`.
+### Why uprobes on `libsqlite3`, not a query log
+
+A query log lives inside one application. It's compiled in or configured per process, which means it covers the processes you thought about in advance and nothing else: not the cron job added next quarter, not the `sqlite3` shell someone runs by hand, not the vendored binary.
-A program that loads on your laptop can be rejected by an older kernel's verifier. [`.github/workflows/kernel-matrix.yml`](.github/workflows/kernel-matrix.yml) guards against that: for each kernel in its matrix it builds the object, boots that kernel in a VM ([cilium's little-vm-helper](https://github.com/cilium/little-vm-helper), images from `quay.io/lvh-images`), and runs the vendored static **veristat** against it — failing the job if the verifier rejects any program. The in-VM gate is `build/verify-kernel.sh`.
+The public C API is the seam where *every* application hands SQL to the engine, and it's stable across SQLite versions in a way internal symbols are not. Hooking it with uprobes puts the instrumentation in the kernel rather than the process, so one attach covers every current and future caller of the shared library with no per-app setup and no restarts. Pairing entry with return probes is what buys the two things a log can't easily give you: real per-call latency, and the out-param `stmt` pointer that ties a statement's whole life together.
-Run the same matrix locally (Linux + KVM) with `make veristat-matrix` — it boots the kernel images with `lvh` + QEMU and prints an `ok`/`FAIL` grid. Pick kernels with `make veristat-matrix KERNELS="6.6-main bpf-next-main"`.
+The cost of that seam is that it's a *library* boundary, not a file one. Anything that doesn't go through the shared `libsqlite3.so.0` is invisible, which is the first entry in [what it can't see](#what-it-cant-see).
## Requirements
> [!IMPORTANT]
-> - **A Linux kernel with BTF** (`CONFIG_DEBUG_INFO_BTF`) — `bpftool` generates `src/bpf/include/vmlinux.h` from it. Default on current Arch, Fedora, Ubuntu, and Debian.
-> - **The yeet daemon**, which performs the privileged BPF load. The BPF capabilities are delegated to a daemonized process, so `sqlitefeed` itself runs unprivileged. `curl -fsSL https://yeet.cx | sh` installs it.
-> - **`libsqlite3.so.0`** on the host — the uprobe target, resolved by bare name via the linker cache.
+> - **A Linux kernel with BTF** (`CONFIG_DEBUG_INFO_BTF=y`), which `bpftool` uses to generate `src/bpf/include/vmlinux.h`. Default on current Arch, Fedora, Ubuntu and Debian. Uprobes and ring buffers put the practical floor around 5.8.
+> - **`libsqlite3.so.0` on the host.** This is the uprobe target, resolved by bare name through the linker cache. Check with `ldconfig -p | grep libsqlite3`.
+> - **The yeet daemon**, which performs the privileged BPF load so `sqlitefeed` itself runs unprivileged. `curl -fsSL https://yeet.cx | sh` installs it. `yeet run` never needs `sudo`.
+> - **x86-64** for [SQL recovery](#recovering-sql-it-never-saw-prepared) specifically. Everything else works anywhere; recovery quietly disables itself elsewhere.
>
-> To build from source you also need `clang` and `bpftool`. No node/npm: esbuild is vendored by the toolchain and the project has no third-party deps.
+> Building from source additionally needs nothing you don't already have: clang, bpftool and esbuild are fetched as a static toolchain, and there's no node or npm involved.
-## Honest caveats
+## What it can't see
> [!NOTE]
-> `sqlitefeed` is observability, not enforcement. It shows you what ran; it does not block or alter anything.
+> sqlitefeed observes; it does not enforce. It shows you what ran, after it ran. It cannot block, delay, rewrite, or roll back a statement.
-- **`REAL` bind values are captured as a type, not a value.** A bound double arrives in an SSE register (`xmm0`) that isn't part of `pt_regs`, so a uprobe can't read it — the row shows `«real»`.
-- **Recovery is x86-64 only.** On other architectures, unseen cached statements show `«unknown»` until they re-prepare (see above).
-- **One per-thread scratch slot, not a nesting stack.** SQLite occasionally makes nested calls on one thread (e.g. reparsing `sqlite_master` mid-DDL); the inner call clobbers the slot and the outer statement is missed — shown as `«unknown»`. This is deliberate: it self-corrects on the next top-level call, whereas a depth-counting stack drifts permanently once a uretprobe is dropped past the kernel's `maxactive` limit.
-- **`exec`/`step` events carry the SQL text on every record** (bounded to 512 bytes); very high statement rates lean on the ring buffer, which drops under backpressure rather than blocking the traced app.
+- **Statically linked SQLite is invisible.** Plenty of programs bundle their own copy of SQLite rather than linking the shared library, including some language runtimes and most single-binary Go and Rust tools. There is no `libsqlite3.so.0` in the process to attach to, so those statements never appear. If your app is missing from the feed, check `ldd` on it first.
+- **`REAL` bind values arrive as a type, not a value.** A bound double is passed in an SSE register (`xmm0`) that isn't part of `pt_regs`, so a uprobe can't read it. The row shows `«real»`. Integers, text and NULLs are captured in full.
+- **Blobs aren't captured.** `sqlite3_bind_blob` isn't hooked, so a blob parameter simply doesn't appear on the `↳` line. That's a deliberate omission rather than a limitation: blob payloads are arbitrary binary of arbitrary size and don't belong in a scrolling terminal feed.
+- **SQL and text values are capped at 512 bytes**, NUL-terminated, so a very long statement or a large text parameter is truncated at the kernel boundary before it ever reaches userspace.
+- **Nested calls on one thread lose the outer statement**, shown as `«unknown»`. It self-corrects on the next top-level call, and the [reasoning for that trade](#the-bpf-side) is in the details block above.
+- **No retention and no aggregation across hosts.** The log holds the most recent 2000 executions in memory on one machine and is gone when you quit. There's no persistence, no query language, no alerting, and no fleet view.
+- **Rows are dropped rather than queued under extreme load.** Every `step` and `exec` event carries its SQL text, so a process running tens of thousands of statements a second can outrun the 512 KB ring buffer. It drops, which keeps the traced application at full speed rather than blocking it.
+- **Other databases need other tools.** This is SQLite specifically, hooked at a library boundary that doesn't generalize. For Redis reach for [`redissnoop`](https://github.com/yeet-src/redissnoop), and for a database reached over a socket the traffic is on the wire, where [`pktscope`](https://github.com/yeet-src/pktscope) reads it.
-## Community questions
+## FAQ
**Does it slow the traced application down?**
-No meaningful overhead. The probes are passive; the cost is a bounded ring-buffer write per call, and the buffer drops rather than blocks if userspace falls behind.
+Not measurably. The probes are passive and the cost is a bounded ring-buffer write per call, on a code path that was already doing a database operation. Under backpressure the buffer drops rather than blocking, so the traced app never waits on sqlitefeed.
**Will it show statements from a process that was already running when I start it?**
-Yes — that's exactly what the `zSql` recovery is for. Statements prepared and cached before you attached are recovered from the statement struct on their next `step` or `bind`.
+Yes, on x86-64. That's what [`zSql` recovery](#recovering-sql-it-never-saw-prepared) exists for: statements prepared and cached before you attached are recovered from the statement struct on their next `step` or `bind`.
+
+**Everything shows as `«unknown»`. What's wrong?**
+Recovery is disabled, which means either you're not on x86-64 or `find-zsql-offset.sh` couldn't locate `libsqlite3` at build time (it needs `objdump` and `nm`). Statements re-prepared after you attach still show their SQL, so a restart of the traced app gives you a full feed either way.
-**Does it work for any process, or just one app?**
-Any process linked against `libsqlite3.so.0`, all at once — the process/pid gutter tells them apart. Statically-linked SQLite (some CLIs bundle their own copy) isn't covered, since there's no shared library to hook.
+**Can it tell me which database file a statement ran against?**
+No. The probes key on the `sqlite3_stmt*` pointer, not the `sqlite3*` connection, and the filename lives on the connection. A process with several open databases shows all of them in one undifferentiated stream.
-**Can I export the feed?**
-Not built in. The `RingBuf.subscribe` callback in `probes/sqlite.js` holds every decoded record, so a JSON/HTTP/Kafka sink is a branch there. To set up a managed pipeline, [contact us](https://yeet.cx/).
+**Does it work inside containers?**
+Yes, from the host. The uprobe attaches to the host's `libsqlite3.so.0`, so it covers any container process that uses the host's shared library. A container carrying its own copy in its image isn't covered by that attach, because it's a different file on disk.
## Building from source
@@ -178,14 +319,44 @@ make bundle # just the JS bundle
make clean # remove build artifacts
```
-`make` runs two independent compilers: **clang + bpftool** compile `src/bpf/*.bpf.c` into the loadable object `bin/sqlite.bpf.o`; **esbuild** bundles `src/main.jsx` into `src/index.jsx`, resolving the `@/` (source root) and `#/` (project root) **bundle-time aliases** via tsconfig `paths` and leaving `yeet:*` builtins external. Both compilers come from a vendored static toolchain, so the build needs no system C/BPF toolchain and no node/npm. The generated `vmlinux.h`, `src/index.jsx`, and `bin/*.bpf.o` are build artifacts.
+`make` runs two independent compilers that know nothing about each other. **clang and bpftool** compile `src/bpf/sqlite.bpf.c` into the loadable object `bin/sqlite.bpf.o`, with the `zSql` offset discovered and baked in as a `-D` define. **esbuild** bundles `src/main.jsx` into `src/index.jsx`, resolving the `@/` (source root) and `#/` (project root) aliases through tsconfig `paths` and leaving `yeet:*` builtins external.
+
+Both come from a version-pinned static toolchain fetched into a per-machine cache, so the build needs no system clang, no bpftool, and no node or npm. The generated `vmlinux.h`, `src/index.jsx` and `bin/*.bpf.o` are build artifacts and are gitignored.
+
+One thing that surprises everyone once: those aliases are **bundle-time only**. Nothing rewrites them at runtime, which is why the probe locates its BPF object with `import.meta.dirname` and a relative path rather than an alias, and why that path differs depending on whether you run the bundle or the probe module directly.
+
+## Testing across kernels
+
+A BPF program that loads on your laptop can be rejected by an older kernel's verifier. That's the failure this guards against.
+
+`make veristat` loads `bin/sqlite.bpf.o` with veristat on **your** kernel, confirming every program passes the verifier and reporting per-program complexity (instructions and states). Loading BPF is privileged, so run it with `sudo`.
+
+For everything else, [`.github/workflows/kernel-matrix.yml`](.github/workflows/kernel-matrix.yml) builds the object once per kernel in its matrix, boots that kernel in a VM ([cilium's little-vm-helper](https://github.com/cilium/little-vm-helper), images from `quay.io/lvh-images`), and runs the vendored static veristat inside it, failing the job if any program is rejected. The in-VM gate is `build/verify-kernel.sh`.
+
+Run the same matrix locally on Linux with KVM using `make veristat-matrix`, which prints an `ok`/`FAIL` grid. Pick specific kernels with `make veristat-matrix KERNELS="6.6-main bpf-next-main"`.
+
+## Try it without real traffic
+
+An empty feed and a broken feed look identical, so verify with traffic you control. [`demo/run.sh`](demo/run.sh) owns the whole demo:
+
+```sh
+# terminal 1
+yeet run .
+
+# terminal 2
+demo/run.sh
+```
+
+It runs a Python workload in the foreground and, in the background, a trickle of `sqlite3`-CLI queries against the same database, so the process gutter shows both `python3` and `sqlite3`. Don't start `traffic.py` separately as well; `run.sh` already runs it.
+
+The workload is built to exercise every rendering path: `executescript` DDL for the `exec` row, parameterized inserts covering int, text, NULL and `«real»` binds, a multi-line `SELECT`, a deliberately slow self-join for the latency heat, and six distinct failures covering `ERROR`, `CONSTRAINT` and `MISMATCH` result codes. The database is a temp file (`/tmp/sqlitefeed_demo.db` by default, overridable as the first argument) in WAL mode; delete it any time to start fresh.
-Because the aliases are bundle-time only, the runtime locates the BPF object with `import.meta.dirname` rather than an alias.
+For just the Python side, `python3 demo/traffic.py` loops until `Ctrl-C`, `--once` runs a single round, and `--interval` and `--db` are available. See [`demo/README.md`](demo/README.md).
## License
-Dual BSD/GPL. The BPF program declares `char LICENSE[] SEC("license") = "Dual BSD/GPL"` in [`src/bpf/sqlite.bpf.c`](src/bpf/sqlite.bpf.c), which the kernel requires for the helpers it uses.
+Dual BSD/GPL.
---
-Built with [yeet](https://yeet.cx/docs/), a JS runtime for writing eBPF programs on Linux. Join us on [Discord](https://discord.gg/JxVseaAVAU).
+Built with [yeet](https://yeet.cx/docs/?utm_source=github&utm_medium=readme&utm_campaign=sqlitefeed&utm_content=footer), a JS runtime for writing eBPF programs on Linux machines. Join us on [discord](https://discord.gg/JxVseaAVAU).
From 7c61128ff6e5b5722ed61842d73cd4c7755de1f4 Mon Sep 17 00:00:00 2001
From: Necco Ceresani
Date: Wed, 26 Aug 2026 10:40:44 -0400
Subject: [PATCH 2/2] docs: update hero GIF
Recaptured against the current UI: the footer now shows the errors, details
and pause bindings the previous capture predated. Alt text updated to match.
---
README.md | 2 +-
assets/sqlitefeed.gif | Bin 4770503 -> 5226193 bytes
2 files changed, 1 insertion(+), 1 deletion(-)
diff --git a/README.md b/README.md
index 449fcbf..6a2ca3c 100644
--- a/README.md
+++ b/README.md
@@ -13,7 +13,7 @@
-
+
**`sqlitefeed` is a live terminal SQLite statement monitor for Linux: it streams every statement any process on the box runs against `libsqlite3`, with the concrete bound values, per-step latency, and result code.**
diff --git a/assets/sqlitefeed.gif b/assets/sqlitefeed.gif
index 2e44e9fc72450d30d93d324d6a8ef40f542e1403..405464c74aeacc9d1e1ff005b01eb784a7e6f5ae 100644
GIT binary patch
literal 5226193
zcmV(>K-j-WNk%w1Vc-Op0{0pKA|oFoBO@dw93&+lBqbvyCLks!Atxy%C@CQzAs!wd
z8zm+lA|o0nC?Fvs873zmDJv;2F)1!9B_JOcJV7`wFD@)DDpy4*M=d9EMlR>1B)o$Z
zF)uN>gBKec6;fY-Ha0LdI4~R>7B4U?QBg}+SygFjWhyKsGBqtWHZ(arH9$Z+(UTCq
zmrXo8IGLH1etvmCPeeXFJ@>B$n3$GDS4n?=dOIv7IW;LB9vE6$S35jCRa8+@Qcgug
zKxb!SaBypOc5z={Uu$Y*KtMi1LqtMCLQ6|YQd3ZTeR)=DWp8h6Q&Uc8XlH0>Wm8j8
zNJmF>bZ(@hpN(!dd15r9qoOM^Fnv8ZLOe2GUtL%-D?Ug*VpBDCc65e4J6Bj+ii(G7
zYHCJGQet9YNnl87h-p`ERbzl+H9b0GH!)jmVlX%~c9nCebr?88K3;lUL`6qxYG!b7
zZWtI5OG`&bR#8u8RfB_ni;IRb{{R30A^!_bMO0HmK~P09E-(WD0000X`2+<602%-+00000-~^Wf00{p8=QLPw
zV!#1|2n!+*u&^NmhY%S)lo+uh#fug*PSnV8qsNXQF@6*ovLwlqCR2`7$#SL3mM~Gi
zlo_)o&6_rJ&eU0Qf&iaD7X%&pz`?+xM~x~?%9QBRr%jPUohr2|)vHmlYR$@Zt5>UC
z!G0Ypwk+ARVbQ8h%XY0>vu)wNoh!F4-Mew|>dnh{uV0NS1oFg5kg(mphYKS%%y_Y5
z#gG$AZXB8N(0&F
zcJJT5g99JFTX=Ee$d4;e&fIwO=gy-;*9=1NL4^rBBOlCtyZ7$kwTJ&7FTOna^WV*<
z2kCV5=l7}O!=Ep|zI*%l@9WRcpZ