PRD Assistant

An AI assistant that turns user interviews into a product requirements document. Every claim in it comes with the interviewee's own words, and the app's code, not the AI, checks that those words were really said.

_{An illustration of the interface, with sample content. Click it for the full-quality video.}

Next.js · TypeScript · Vercel AI SDK · gpt-oss-120b on Cloudflare Workers AI · Supabase Postgres with pgvector · Playwright · GitHub Actions

Try the live demo → prd.oualidfaouzi.dev · No sign-up. Each visitor gets 10 messages a day. More below.

You're the product manager of a shared grocery list app, and a researcher has interviewed six fictional households. Click Draft the first section: the assistant searches the interviews and writes a PRD (product requirements document) beside the chat, one section at a time. Every statement names whose words back it, and a click on a name opens that person's answer with the quote highlighted. The PRD survives a reload and downloads as Markdown.

The catch. Language models paraphrase, and sometimes they make things up. So the app's own code, not the model, checks every quote word for word against what that person said, and shows any quote that doesn't match. A request no interview backs, such as adding grocery delivery, is marked Your decision, not from the interviews.

The check proves a quote is real, not that it's read fairly, which is why every quote is one click from its statement.

• Live: everything in the story, at prd.oualidfaouzi.dev (Vercel, Supabase in Frankfurt, Cloudflare Workers AI on its free plan), with a limit of 10 messages per visitor a day, so one visitor can't use up the free daily AI allowance. When that allowance runs out for the day, the demo says so and points to the recording above.
• Measured: the AI's behaviour, on fixed requests against a baseline (how).

The product description, its scope, the decisions and the build plan are all in the repository.

flowchart LR
  V[You, in the chat] --> R["/api/chat"]
  R --> M["gpt-oss-120b<br/>Cloudflare Workers AI"]
  M -- searchNotes --> S["Embed the question<br/>search_passages in pgvector"]
  S -- five closest passages --> M
  M -- saveSection --> K["Citation check<br/>the app's own code"]
  K -- counts and failures --> M
  K -- checked section --> P[The PRD beside the chat]

Before the assistant writes anything, it searches the interviews and works from what it finds, not from what the model happens to know. This is retrieval-augmented generation (RAG), in three parts:

1. Passages of one question and one answer. The notes loader cuts each interview into passages of one interviewer question and the interviewee's whole answer (parse-interview.ts). A passage never mixes two people, so a citation points at one person's words, and a quote is checked against that answer alone. The question stays attached because answers lean on it: "Every single item, yes" means little without "How would you want to know when Ineke adds something?"
2. Embeddings. bge-large-en-v1.5 on Cloudflare Workers AI turns each passage into 1,024 numbers, stored beside the text in Postgres with pgvector (transcripts.sql). Texts with similar meanings get vectors pointing in similar directions. The loader embeds only new or changed passages, and the model's name is part of each passage's hash, so a new model re-embeds everything instead of mixing two models' vectors. The model reads at most 512 tokens, so the loader refuses any passage that might be longer rather than let it be cut silently. No tokenizer is needed for that: this one never makes more tokens than a text has non-space characters, plus two (embedding-input.ts).
3. The search. The question is embedded with the same model, and one SQL function, search_passages, ranks every passage by cosine distance and returns the five closest, each with the interviewee's name and the question (search.ts). With tens of passages, comparing them all is exact and faster than an index; an approximate index (HNSW) pays off at thousands.

The app's read key may read the passages but not the vectors, so search_passages runs with its owner's rights, and only that key may call it. It refuses an empty question or one over 500 characters, and checks that a vector has exactly 1,024 numbers and isn't all zeros before it reaches the database.

The chat runs on the Vercel AI SDK: the browser posts the conversation to /api/chat, and the server streams the answer back (chat.ts).

• Two tools. OpenAI's open-weight gpt-oss-120b runs on Cloudflare Workers AI through its OpenAI-compatible endpoint, with searchNotes and saveSection. The model decides when to search and what for. After three searches the app stops offering the search; once a section is saved, it offers no tools, so the model replies.
• Whole replies. Streamed, Cloudflare sends gpt-oss's tool calls as reasoning text without the tool's name. So the app asks for whole replies and passes each one on in one piece (cloudflare.ts). While the model thinks, the chat says so and then shows how long it took.
• The instructions. The system prompt is the PRD prompt from my AI Product Development Toolkit, adapted and versioned in system-prompt.ts: nine sections, a check-in before each, draft first and ask only what the interviews don't say, never invent, show both sides of a contradiction. Changing the text needs a new version: a test compares its fingerprint with the one recorded for that version.
• Tokens. The server logs the tokens each answer used. The model's reasoning never goes back to it with later messages, and earlier searches go back as passage ids and names only, so a long conversation doesn't resend every passage it ever found.
• Failures. Whatever breaks, the provider or the connection, the answer ends with a plain error, the cause is logged, and "Try again" asks again. Once Cloudflare's free daily allowance is used up, the app stops retrying, hides the message box and says when it resets. When Cloudflare is busy, it retries twice, then asks you to try again in a minute.

The model never decides whether a quote is real. It saves each section with saveSection as a list of statements, each with its citations: a passage id such as henk-4, or chat for your own words, and a short quote. The app's code checks every quote before you see the section (citations.ts):

1. Against the right words. An interview quote is checked against the answer of the passage it cites, read from the database. The interviewer's question never counts, and neither does another answer, even by the same person. A quote of your words is checked against the messages you sent, never the assistant's.
2. Exact, within reason. A quote passes when it has the same words in the same order as its source, ignoring letter case, spacing, the style of quote marks and punctuation at its ends. It must match whole words: "ink" isn't in "drink". A changed word, a dropped comma or a gap in the middle fails.
3. Nothing hidden. Each statement ends with its sources by name, such as "Maya, Daan" or "You, in this chat". Whatever the check objects to is written out under the statement: an unverified quote with its reason, or "No quote backs this statement." A statement backed only by your own messages reads "Your decision, not from the interviews." (prd.ts).
4. The model hears the result: the counts, every unverified quote and every statement that cites nothing, so its reply can say what the check found.

The model answers differently every time, so one good answer proves little. npm run measure sends fixed requests to the real model, five runs each, for six answers: the first section, the disagreement about notifications, a gap in the interviews and your answer to it, and the grocery delivery and dark mode claims. Each run is scored on the product's rules: every quote passes, every statement has a source, the gap is asked about instead of invented, both sides of the disagreement show, and no statement about grocery delivery or dark mode cites an interviewee.

A change, such as a new prompt, records its expected effect before it runs and is compared with the baseline: a one-sided Fisher's exact test at 5% on its rule, and a flag on any other rule that drops by two runs or more. Every run's full record is kept, and the command stays within a budget it prints before sending anything, since the free plan allows 10,000 neurons a day.

You need Node.js 24 (see .nvmrc), Docker Desktop running, Git and the gitleaks secret scanner. npm run doctor checks them all.

npm ci
npm run db:start
npm run env:local
npm run dev

Then open http://localhost:3000. Everything runs on your machine, with no accounts or keys: the database in Docker, and fakes for the AI. The fake chat model searches with your own words and quotes the closest passage; the fake embedder counts shared words, so it knows no synonyms. Phrases in a message make the fake fail on purpose or save a flawed section, so the tests can show the error, the retry and the check catching bad quotes (fake-chat-model.ts).

The local database dashboard (Supabase Studio) is at http://127.0.0.1:57323. Before your first end-to-end test run, install the test browser with npx playwright install chromium.

npm run dev:replay shows the interface with the real model's answers, for free and without a token. It replays answers the measurement recorded on gpt-oss-120b (replays.json), each reply whole, after the time the real model took. The intro's button and the other measured requests replay, and the scenario tour plays the grocery delivery claim; any other message, the grocery delivery suggestion included, gets the scripted fake's answer, at the same pace. Searches and saves still run on your machine, so the search lines show local passages, while the words come from the recording. The header says "Replaying recorded answers".

A scenario tour in the corner of the page walks through every state the chat and the PRD can show, which recorded answers alone never reach: rich text and tables, folded and refused searches, a section with every kind of statement, a long wait to stop, each kind of failure and, last, the used-up daily allowance. Each step says what to watch for; "Play next" sends it. The scripted answers run through the same chat code, so their searches, saves and quote checks are real.

To refresh the copy after a new measurement, run npm run replays:copy -- --from docs/records/measurements/<name> where the records are.

To watch the real model live, with a Cloudflare token of your own:

1. Make a Workers AI token as in step 1 of the setup under Deploying your own copy (steps 2 to 4 there). A token of its own means you can revoke it later without touching staging. Copy the token and the Account ID.
2. Create .env.development.local in the project folder (that exact name, with no .txt ending, saved as UTF-8) with two lines: CLOUDFLARE_ACCOUNT_ID=<the account ID> and CLOUDFLARE_API_TOKEN=<the token>. Only next dev and the measurement read that file, and Git ignores it.
3. Run npm run dev:real instead of npm run dev.

The search still runs on the fake embedder, because the local passages carry its vectors, so the passages that come back differ from staging. Every message spends neurons from the free daily allowance that staging and production share: 10,000 a day. Every other command except the measurement keeps the fakes, and the environment guard refuses the token anywhere but that file.

• npm run measure -- --name <name> --purpose "<why>" runs a batch on the real model with the token above. Add --plan to only print what it would send and cost. npm run measure:fake runs the same tool on the fakes, for free, and npm run measure:report -- --name <name> prints a batch's scores and its comparison with the baseline, sending nothing.
• A batch fills only the runs that have no record yet and never repeats an answer by itself. Before it sends anything, it prints how many answers it will send and their expected cost, and it stops once it passes 1.5 times that. After each answer it prints the neurons used that day and month. One batch runs at a time.
• A "quota used up", "busy" or per-minute refusal from Cloudflare counts as "not run". The gap's answer and the two claims start from saved conversations, kept fixed for the measurement.
• Each answer's full record goes to docs/records/measurements/, which Git ignores: what was sent and what came back, the tokens, the timings and the setup, never the token.

Every dev, build, test and database command first runs an environment guard. It stops if anything points at a hosted database or API, picks a real AI provider, or holds a real provider key anywhere but .env.development.local.

The files in supabase/schemas/ are the single source of truth for the database.

1. Edit the schema files.
2. Run npm run db:migration -- <name> to generate supabase/migrations/<timestamp>_<name>.sql, and review it.
3. Run npm run db:reset to apply every migration to a fresh local database. It reloads the interviews.
4. Run npm run db:types to regenerate src/server/db/database.types.ts.
5. Add or update the SQL tests in supabase/tests/, then run npm run verify:db.
6. Commit the schema, the migration and the types together.

Every table and function starts closed to the public API and opens only through explicit grants. supabase/AGENTS.md has the rules.

The staging and production keys never sit on a development machine; a Workers AI token of your own may, in .env.development.local, for npm run dev:real and npm run measure only. Search engines may index only the production domain, SITE_INDEXABLE_HOST; every other address, staging included, sends noindex.

git push -u origin <your-branch>

Built with my AI App Starter.

MIT.