Architecture

Guide for anyone (human or AI agent) developing fastdata. Read this first; it is the source of truth for architecture and conventions. Keep it current when you change something it describes.

What fastdata is#

An npm CLI (fastdata) that lets people ask questions about a database in plain language. A Claude agent (Claude Agent SDK) reads a knowledge base folder, writes SQL, runs it through a connector the user controls, and answers in Markdown. Two surfaces share one runtime: a local web chat (fastdata / fastdata serve) and a terminal command (fastdata ask). fastdata run is the third entry point: it runs the user's own SQL as written and, only when the database or the guard rejects it, asks Claude for a fix (src/agent/fix.ts). Successor to a Custom GPT prototype (see docs/concept.md, docs/customgpt/).

Two modes, chosen by config:

• execute: a connector is configured; the agent calls run_query and sees a row preview.
• manual: no connector; the agent calls propose_query, the user runs the SQL and pastes the results back (results field in the chat API / ask --results).

Architecture#

 browser (React, dist/web)          terminal
        │  POST /api/chat (SSE)         │ fastdata ask
        ▼                               ▼
 src/server/app.ts (Hono)        src/commands/ask.ts
        └──────────────┬────────────────┘
                       ▼
          src/agent/session.ts  runTurn(ctx, {prompt, sessionId}) → AsyncGenerator<FastdataEvent>
             │  Claude Agent SDK query(): tools = Read/Grep/Glob (confined to knowledge dir)
             │  + in-process MCP server "fastdata" (src/agent/tools.ts)
             ▼
   run_query / propose_query ── assertReadOnly (src/sql/guard.ts) ── Connector.execute()
                                                                      │
             src/connector/mcp.ts (stdio | streamable HTTP)  ◄────────┤
             src/connector/http.ts (plain JSON POST)          ◄───────┘
                    │
     user's MCP SQL server, or built-in: fastdata connector sqlite|postgres

• Event contract: src/events.ts (FastdataEvent). Shared by server, CLI and web (the web imports it directly, so keep it free of Node imports). Every turn ends with done. Changing it means updating web/src/state.ts and src/commands/ask.ts.
• Sessions: the Agent SDK persists transcripts; we pass its session_id back as resume. The CLI stores the last id in .fastdata/last-session next to fastdata.yaml; in a TTY, ask offers to continue it when it is under a day old, and ask --loop resumes the same session for every question.
• Results: full results live in ResultStore (in memory, per process); the model only gets preview_rows. UI fetches full rows / CSV from GET /api/results/:id.
• Connector output varies wildly; src/connector/normalize.ts turns JSON (arrays of objects, {columns, rows}, nested envelopes), CSV/TSV and Markdown tables into {columns, rows}.

Invariants (do not break)#

1. fastdata never holds database credentials. They live in the connector's environment or service. Config values use ${VAR} expansion from env / .env; never log expanded config. Exception by design: fastdata learn --url (or the URL prompt) writes the URL to .env as DATABASE_URL (mode 600, added to .gitignore) and the connector reads ${DATABASE_URL}; it is never written to fastdata.yaml or printed.
2. Read-only is enforced in code, three layers: assertReadOnly before every execution, read-only transactions in built-in executors, and a read-only DB role (documented, required). The guard is conservative: false positives are acceptable, false negatives are bugs. Unterminated comments/quotes and backslashes inside quotes must be rejected (dialects read them differently, which can hide a second statement). The same goes for every other place dialects disagree (nested /*, MySQL /*!, # and --x, $$ outside Postgres, [ ]): take the reading that exposes more SQL, or reject. test/guard-dialects.test.ts runs these cases on PGlite and SQLite; add new ones there. The Postgres executor also uses the extended protocol (one statement per call) and caps rows in SQL.
3. The agent is sandboxed: tools: ["Read","Grep","Glob"], settingSources: [], strictMcpConfig: true, permissionMode: "dontAsk", and a PreToolUse hook (outsideKnowledge) that denies paths and glob patterns reaching outside the knowledge folder. Never add Bash/Write/Edit/Web tools to the agent. Its environment comes from claudeEnv (src/agent/env.ts), which drops DATABASE_URL and every ${VAR} the connector block uses. The run fixer (claudeFixer) uses the same sandbox with no query tool: it sees the SQL and the error text, never rows, and every fixed query goes through assertReadOnly again.
4. Bulk data does not go through the model: preview only (limits.preview_rows).
5. Results and knowledge files are data, not instructions (stated in the system prompt; pasted results are wrapped in <results> tags by pastedResultsPrompt).
6. Local by default: server binds 127.0.0.1, rejects foreign Host and cross-origin requests. Every response carries a CSP that allows only this server, and the UI renders Markdown images as links: model output can be steered by data, and must not load remote URLs.
7. Claude only. The project deliberately uses the Claude Agent SDK (users already run Claude Code; its login works without an API key). Do not add other LLM providers.

Repository map#

Commands#

npm install
npm run dev -- ask "..."        # run the CLI from source (tsx); any subcommand works
npm run dev:web                  # Vite dev server on :5173, proxies /api to :8000 (run `npm run dev` too)
npm run build                    # web → dist/web, then CLI → dist/cli.js
npm run typecheck                # tsc for src+test and web
npm run lint                     # biome (npm run format to fix)
npm test                         # vitest, no network, no Claude calls
npm run example:ringly           # Ringly DB on :5433 via PGlite (no Docker)
npm run docs:build               # docs site → site/dist (rebuild after editing docs/*.md)
npm run docs:deploy              # build, then deploy site/ to Vercel production

Quick manual check without Claude: fastdata init /tmp/x --demo && cd /tmp/x && fastdata doctor. End-to-end with Claude (costs a few cents): fastdata ask "How many customers do we have?" in the demo folder.

Before finishing a change: npm run typecheck && npm run lint && npm test && npm run build.

Conventions#

• TypeScript strict, ESM, Node ≥ 22.13 (node:sqlite is used by the demo/SQLite executor).
• Imports use .js suffixes in src/ (NodeNext). The web uses .ts/.tsx suffixes (bundler).
• Errors users can act on: throw ConfigError / ConnectorError / SqlGuardError with a message that says what to do; the CLI prints these without a stack trace.
• CLI: results to stdout, progress and logs to stderr (so fastdata ask ... > answer.md works).
• Read SDK APIs from the installed .d.ts files (node_modules/@anthropic-ai/claude-agent-sdk/sdk.d.ts), not from memory; the SDK moves fast.
• tsup keeps node: prefixes (removeNodeProtocol: false) — node:sqlite has no bare form.
• Built-in executors are spawned via command: fastdata, which mcp.ts resolves to the running Node binary + entry script (works for global installs, npx and tsx).
• Default model claude-opus-5-5, effort medium (src/config.ts).

Known gaps / next steps#

• Auto-charts in the answer card (date+number → line, category+number → bar).
• fastdata learn: refresh mode that updates changed tables while keeping human edits.
• fastdata eval: replay examples/*.yaml, compare result sets, report accuracy.
• Feedback loop: thumbs up → pending example in examples/_pending/.
• Approval mode (confirm each query in the UI before it runs).
• Persist chat history in the UI across reloads; result store is in-memory per process.
• Token/latency tuning: the agent front-loads many file reads on broad questions.
• Auth for team deployments (currently local-only by design).