feat(deploy): proof-of-concept Maple Cloud self-host on celld - #689
feat(deploy): proof-of-concept Maple Cloud self-host on celld#689xesrevinu wants to merge 5 commits into
Conversation
Run the Cloud Workers (api, electric-sync, alerting) and the web UI on celld instead of Cloudflare, so a VPS install does not need a CF account. celld cannot TCP-dial Postgres, so Postgres goes through a Neon-style WebSocket proxy; AES-GCM uses Web Crypto because celld node:crypto createCipheriv is inert. This is an experiment toward simpler self-hosting, not a replacement for alchemy deploy or local-mode maple start.
ff5ec6b to
da18e48
Compare
|
Hey—I've just started trying maple recently, it's very interesting. Thank you for your continuous dedication; the daily experience is really great, and I will keep an eye on it. The one place I got stuck is running it on my own VPS. What I actually want is to keep the app on the machine and expose it through Cloudflare Tunnel, not to run it as Cloudflare Workers. Tunnel vs Workers is a different shape of “on Cloudflare,” and the current Cloud path is pretty tightly bound to the latter. So this draft is just a spike: same Worker source, celld as the runtime, origin sits on the VPS and can go out via a tunnel. The awkward bit is Postgres — celld doesn’t have Hyperdrive or working Totally a POC, not a “please merge this.” Figured I’d share the experiment in case the direction is interesting. |
Ship Postgres/ClickHouse/Electric/OTel alongside the Workers, and run first boot as drizzle-kit + clickhouse-cli + db:ensure-electric-publication instead of SQL in a ConfigMap. Environment pins (nodeSelector, ingress) go in a Kustomize overlay. Default RUST_LOG=warn so celld stops emitting a ship-loop line every second.
api/sync/alerting wait for a one-shot migrate service that calls drizzle-kit, clickhouse-cli, and db:ensure-electric-publication. The example k8s overlay uses strategic merge so re-apply is idempotent.
Self-hosted has no ClerkProvider. Settings → Ingestion used useAuth via useGuidedFramework, which threw. Read org id from useActiveOrgId instead. schemaDiff returns an empty diff when BYO ClickHouse is not configured instead of 400.
GET /internal/billing/customer is 500 BillingNotConfiguredError without AUTUMN_SECRET_KEY. Self-hosted has no Stripe/Autumn; settings nav still subscribed useMapleCustomer. Disable the atom in that mode.
|
One small note on product shape, since this is the Cloud stack rather than I know you’re building a business around hosted Maple, and I don’t want this to read as “here’s how to skip that.” Local-mode already covers self-host telemetry; this spike is just our VPS + tunnel case. If a recipe like this would get in the way of what you’re commercializing, totally fine to leave it as a draft or a fork — we’ll follow your lead ❤️. |
Proof of concept — not a production proposal
This is an experiment in running Maple Cloud (the Workers API + web UI + electric-sync + alerting) on celld so a VPS can sit behind something like Cloudflare Tunnel without putting the app on Cloudflare Workers, and without Wrangler / Miniflare / Alchemy.
It is not:
alchemy deploy)maple start(telemetry-only single binary)The goal is to keep one Worker source (
wrangler.celld.jsoncis a stripped subset of the existing Wrangler config) instead of rewriting the API as a second Node/Express server.Why there is a Postgres WebSocket proxy
On Cloudflare, Maple’s control-plane Postgres is Hyperdrive (
MAPLE_DB) plus workerd TCP (cloudflare:sockets/connect()/node:net) with postgres.js.celld v0.4.0 does not implement that surface:
hyperdriveis a forbidden key (deploy aborts)cloudflare:sockets/ TCPconnect()/node:netare an inert stub — the isolate cannot dialpostgres:5432So the Worker cannot open a normal Postgres connection. This POC puts a small Bun process in front of Postgres (
scripts/pg-ws-proxy.ts) that speaks the Neon serverless WebSocket protocol. Inside the Worker we use@neondatabase/serverlesswhenMAPLE_PG_WS_PROXYis set. Unset that env and the existing TCP/Hyperdrive path is unchanged (wrangler / alchemy).ClickHouse is already HTTP, so it did not need this detour.
If celld later grows real outbound TCP (or Hyperdrive-shaped pooling), this proxy should go away. The Worker would dial Postgres the same way Cloud does, and the self-host recipe shrinks to “celld + PG + CH + Caddy” without a WS sidecar. That is the main complexity we would rather not keep.
What this adds
bash scripts/celld-dev.sh— threecelld devprocesses (api / electric-sync / alerting) against docker Postgres + ClickHouse + Electric.deploy/celld-self-host/— Compose one-command stack, Caddy same-origin, Kustomize. Defaultdocker compose --env-file .env up --build→ http://127.0.0.1:8080. BYO Postgres/S3: copy the folder and delete those services.VITE_*URLs; Electric ShapeStream requires an absolute URL, so production resolveslocation.origin.node:cryptoHMAC works;createCipheriv("aes-256-gcm")does not. Stored{ciphertext,iv,tag}layout is unchanged.Other celld limits (v0.4.0)
wrangler keys
hyperdrive/ai/ratelimits/send_email/dev/workers_devabort deploy. Queues omitted (celld refuses a queue consumer on a Worker that also exportsfetch). Cron / DO / R2 are Partial.How to try it
Sign in at http://127.0.0.1:8080 with
MAPLE_ROOT_PASSWORD(docs examples usechange-me). Stop any localcelld-devstack first so 8080/4317 stay the default ports.First boot still needs Drizzle migrate + ClickHouse schema + Electric publication (same heal as
dev:celld).Relevant issues