feat: generate_prd and generate_artifact tools (ENG-968, ENG-969) - #335
Open
StpMax wants to merge 45 commits into
Open
feat: generate_prd and generate_artifact tools (ENG-968, ENG-969)#335StpMax wants to merge 45 commits into
StpMax wants to merge 45 commits into
Conversation
* tool for artifacts generation * make api spec in json, not md * do not save api spec in file * fix prompt * prompts * del `data_refs` from `generate_artifact` * tool update * tests * fix issues and logging * make generate_artifact the primary path for html and fullstack artifacts
* feat: ArtifactStore.update() can change artifact type * feat: raise the shared per-turn question budget from 3 to 8 * feat: generate_prd sub-tool schemas and direct-elicit ask_user dispatch * feat: generate_prd phase 1 gathering loop * feat: generate_prd phase 2 steps (draft_brief, show_and_confirm, classify_feedback, write_prd) * feat: generate_prd orchestrator.run — full two-phase sequence * feat: register generate_prd as a tool * feat: point create_artifact's description at generate_prd for web artifacts * fix: draft_brief/write_prd prompts — lead-in sentence, no process meta-commentary, no technical detail in the short brief Live-testing feedback on ENG-969: the short brief landed on the user with no framing, echoed the tool's own regeneration framing into Goal, listed data sources not used, and leaked CSS/JS implementation detail (clamp(), hex colors) into a document meant for a non-technical reader. * fix: remove Cyrillic from source and docstrings (project convention: code in English) - prompt instruction/label text: 'Тип артефакта' -> 'Artifact type', 'Принять'/'Отменить' button labels -> 'Accept'/'Cancel' - docstrings referencing prd-design.md section titles translated to English * fix: allow multiple lines in the brief's Data model section 'One short line' was too strict for multi-table cases (e.g. a dashboard reading several DB tables) — the actual intent was 'no verbose negatives and no connection details', not a hard one-line cap. Multiple sources now each get their own short line. * feat: default-value support for choice questions — Enter accepts the PRD brief AskRequest gains default_value: the value of an option chosen when the user presses Enter with no input, instead of the channel's usual 'cancelled'. CLIElicitor._ask_choice forwards it to prompt_or_cancel (which already substitutes it on blank Enter and shows it in the prompt suffix) and now also resolves a directly-typed option value, not just its number. generate_prd's show_and_confirm sets default_value="accept" — a bare Enter now continues instead of cancelling, matching what users actually do most of the time. draft_brief's prompt also asks the model to close the brief with a short in-language line making that explicit. ask_user's own JSON schema is untouched — only orchestrator code that builds AskRequest directly (select_path, generate_prd) can set a default; the LLM never picks one on the human's behalf. * feat: compact rendering for self-explanatory choice prompts Live-testing feedback (ENG-969): the PRD brief ends with its own 'continue, or changes?' sentence, then repeats the same choice as a numbered Accept/Cancel list plus a descriptive input caption — pure noise once the sentence already explains the mechanic. AskRequest.compact (opt-in, off by default) tells StreamDisplay.show_question to skip the option list and CLIElicitor._ask_choice to drop the descriptive caption in favor of a bare input point; prompt_or_cancel's own default-value suffix (e.g. '(accept):') still carries the hint. Parsing is unaffected — numbers, typed values, and free text all still resolve the same way. generate_prd's show_and_confirm sets compact=True. Every other ask_user/select_path caller is untouched. * fix: restart the spinner after ask_user before the next direct LLM call elicit() stops the host spinner (phase="interactive") for ask_user/ show_and_confirm but nothing restarts it — the outer agent loop's own reasoning_start signal never fires for generate_prd's direct _llm.plan/code/generate_object calls, since they run outside that loop's tool-round machinery. Made the gap visible: after answering the brief confirmation with free-text feedback, the UI showed nothing for a couple seconds until the revised brief appeared. Add sub_tools.signal_thinking() and call it before every direct LLM call in phase 1's gathering loop and phase 2's draft_brief/ classify_feedback/write_prd. * fix: restart the Live spinner context on reasoning_start, not just update it phase="interactive" tears down the spinner's Live context entirely (_stop_spinner sets _live = None) so the user can type. The preceding fix (signal_thinking, previous commit) emits reasoning_start right after an ask_user/show_and_confirm answer, but that phase's handler only called _update_spinner(), a no-op once _live is None — so the event reached the display and still produced no visible spinner. Elsewhere reasoning_start already finds a running Live (a tool-result line printed in between implicitly restarts it), which is why this gap went unnoticed until generate_prd's direct LLM calls exposed it. * fix: constrain finish_gathering's artifact_type to the closed enum Nothing stopped the model from inventing an artifact_type string outside ArtifactType's closed set (html-app, document, dataset, image, mixed, fullstack-stateless-app, fullstack-stateful-app) — the tool schema had no enum and the gathering system prompt never listed the valid values. An invented type sailed through unvalidated until write_prd's ArtifactStore.update(type=...) call, which raises ValueError for anything outside the enum — crashing the whole generate_prd call with a message the outer agent sometimes misreads as a transient glitch worth silently retrying, producing an apparently-stuck "same brief shown again" loop. Add the enum to FINISH_GATHERING_SCHEMA, list the valid types in the gathering system prompt, and — since a schema enum is a hint, not an enforced constraint — fall back to the originally registered type in engine.py if the model returns one outside it anyway.
Mirrors generate_artifact/debug_trace.py's GenTrace/NullTrace shape and, deliberately, reads the SAME ANTON_DEBUG_ARTIFACT_GENERATE_TOOL env var: the two tools are the two steps of one end-to-end artifact-generation pipeline (PRD, then code), and both loggers append rather than truncate, so running them back to back writes one combined, chronologically ordered log — viewable as-is in artifact_trace_viewer.html. Instruments phase 1's gathering loop (every LLM call, ask_user, scratchpad/web_search/web_fetch dispatch, and how gathering ended) and phase 2's steps (draft_brief, show_and_confirm, classify_feedback, write_prd). generate_prd.generate() wraps orchestrator.run with run_start/run_result, including on a crash. Off by default (NullTrace) when the env var is unset — no behavior change for existing callers.
The brief's closing line asked the model to spell out that a bare Enter means "continue". That hint is CLI-specific text living in host-agnostic content: the same line is rendered in the terminal, where `prompt_or_cancel` already prints the `(accept)` default, and in a GUI, where the host draws Accept/Cancel buttons and there is no Enter to press. It was also LLM-generated, so its wording, language and very presence drifted between runs — an input affordance belongs to whatever code draws the input. The instruction already forbade inventing accept/cancel option labels for the same reason; the Enter hint just was not covered by it. Keep the closing question itself (added from ENG-969 live-testing feedback, and what makes `compact=True` legible) and forbid describing how to answer.
ENG-970. A fullstack generation runs for minutes and emitted nothing to the UI for the whole time, which reads as a hang. `handle_generate_artifact` becomes an async generator so it can use the streaming-tool protocol from ENG-763: `dispatch_tool_stream` forwards every yielded `ToolProgress` and takes the last non-marker item as the tool result, so all the existing return paths are unchanged apart from `return` becoming `yield`+`return`. That protocol was built and tested but had no production user until now. Yielding rather than calling `session.emit` directly (the way generate_prd's `signal_thinking` does) is what gets the marker its originating `tool_use` id: a handler never sees that id, only `dispatch_tool` does, and it stamps relayed markers with it. Without the id a marker is not the "first per id" the cloud wire never drops, so it falls under the rate limit, and a step UI has nothing to correlate it to. Progress travels up on an `asyncio.Queue` because the FSM cannot yield from the handler: steps start several frames down, including inside the `asyncio.gather` that generates backend and frontend at once. The channel's sentinel is pushed from a `finally`, so a crashed generation closes the drain loop instead of hanging it; the handler cancels the generation task from its own `finally`, so a cancelled turn does not leave the FSM writing files nobody will collect. `GenState.step_started` is deliberately separate from `record`, which fires when a node is already done — the two longest nodes would otherwise report only in hindsight. `progress.py` maps node names to plain-language lines, since graph vocabulary (`is_data_enough`) must never reach a user; an AST test over orchestrator.py fails if a call site has no label.
…rief `generate_prd` wrote prd.md and nothing ever read it. The artifact was built from whatever the calling agent chose to put in `context`, so the document the user actually reviewed and accepted had no effect on the result — the two tools were a pipeline only in the tool descriptions. The generator now reads prd.md from the artifact folder itself. That needs no new schema parameter: the handler already resolves the folder from `slug`, so the calling agent cannot forget to pass the PRD, mis-transcribe it, or paraphrase it. Every failure to load degrades to "no PRD" rather than stopping the run — an agent may legitimately skip the PRD step, artifacts created before ENG-969 have none, and a file that cannot be read must not cost a generation `context` alone can still complete. Which mode ran is recorded, so a wrong-looking artifact can be traced back to the requirements it was really built from. `context` is redefined as a supplement rather than a competing source of truth. Its `## Functional Requirements Specification` section is gone — requirements live in the PRD, and two copies invite disagreement — and `## Data` is narrowed to data that already exists in scratchpad cells, with `### Sample` rows only from cells actually run. Expected-but-unfetched sources are what the PRD describes; repeating them here is a guess competing with an accepted document. All three surfaces that state this contract are updated together, and the tool prompt now names generate_prd before generate_artifact — without that step the model has no reason to believe a PRD exists and every run would take the fallback. Pad matching now searches the PRD alongside the brief: generate_prd must cite the scratchpad and cell behind every source it describes, so on the normal path the PRD is where a pad name appears at all. Cells the PRD points at land in `data_notes`, which lets `is_data_enough` answer from what is already there instead of starting its own fetch loop. PRD_FILENAME is shared by the writer and the reader: a literal in both packages could drift apart silently, and the reader would then simply find nothing and build from a brief the user never confirmed.
`grep generate_prd anton/core/llm/prompts.py` came back empty: every always-on block sent the agent straight from `create_artifact` to `generate_artifact`, while `create_artifact`'s own description said to draft a PRD first. Which instruction won was luck. Since the generator now takes its requirements from the `prd.md` that `generate_prd` leaves in the artifact folder, this is no longer a missing step but a broken one: a run reached that way has no PRD at all and silently falls back to building from `context` — which, with the requirements section removed from it, carries considerably less than it used to. ARTIFACTS_PROMPT's workflow grows a PRD step ahead of generation, and states that the generator reads prd.md itself so the agent does not quote or paraphrase it back into `context`. The two visualization blocks and BACKEND_GENERATION_PROMPT get the same three-call normal path. A test locks the ordering across all four blocks. `generate_prd` has no `ToolDef.prompt` of its own, so these blocks and its tool description are the only places the model can learn the step exists — a new always-on artifact block would otherwise omit it again.
`_reconcile_files` re-derives `files[]` from disk and excluded only the store's own files, so `prd.md`, `spec.md` and `openapi.json` counted as artifact content: they inflated file_count, appeared in the rendered README, and invited the agent to hand the user a specification as the thing it built. They are what the generator built FROM. Publication was not affected, contrary to what the handoff assumed — no publish path reads `files[]`. An html-app publishes its primary file plus the siblings the HTML references, and `_zip_fullstack` bundles an explicit allowlist (backend.py, requirements.txt, static/**), so root-level inputs never entered a bundle. (`_zip_html`'s directory branch would take them, but no artifact path reaches it.) `backend.log` goes in too. The set is documented as mirroring cowork-server's artifacts service, and that service, `publish_access` and the publish bundle all excluded the launched backend's runtime log — the store was the one copy that did not, which is precisely the copy the agent and the UI read their file list from. Generation inputs stay in a set of their own rather than being folded into the housekeeping one: they are authored by the generation tools, not owned by the store, and merging them would quietly falsify the mirror claim. Exclusion is by relative path, so a `static/openapi.json` the artifact genuinely serves remains its own content. The three filenames now live together in `artifacts/internal_files.py` (renamed from prd.py), because spec.md and openapi.json were literals in the orchestrator — the exclusion set and the code writing the files would have drifted apart on the next rename.
ENG-1116. `make_tech_spec` and `make_api_spec` each produce a whole document in one call on the client's default 8192-token budget, and neither checked for truncation. The two then failed very differently: a cut API spec broke `json.loads` and was reported as "not valid JSON", blaming the model's syntax for an output-cap hit, while a cut tech spec was written to spec.md and handed to backend and frontend generation by `_spec_context` — half a spec building half a system with nothing anywhere reporting the loss. That silence is the defect, not the length. Both calls now run with an explicit budget and, when cut off, are re-asked once with more room plus an instruction to write compactly. The re-ask has to change the call: the main loop's own recovery measured three unchanged retries dying identically. If the second attempt is also cut, the node returns an error naming the output limit — nothing partial is written, and the API-spec path reports the cap rather than the JSON parse that follows from it. The budgets are measured, not chosen: against api.mindshub.ai's `opus` alias, 20480 answers normally and 24576 and above return HTTP 500. A 500 is classified as a transient provider error, so an over-large budget does not fail fast — it burns the retry ladder on every generation before dying. 16384/20480 keeps real headroom (reasoning models spend thinking from the same budget) while staying inside what the gateway serves; both constants carry the measurement and must be re-measured before a raise. Truncation is detected with the shared `looks_truncated`, which also honours stop_reason — the gateway has reported it correctly since 2026-08-03, and a cut that stops just under the cap is invisible to a token count alone.
…STATE store Align the generation pipeline with the STATE SDK model merged from staging (#259). Before this, the pipeline still carried the old stateful model (a local sqlite file in the artifact root), and a backend written to the new contract could not pass verification at all. - _STATEFUL_RULES rewritten to the STATE contract: module-level STATE = None slot, store built at point of use via get_store(), Collection-first API, no scan / no secondary indexes, atomic increment/update, no manual retry around mutations, never list anton_state in requirements.txt, and the flat state_manifest.json format (written by the generator sub-agent via write_file; validated by the verifier, so hand-written JSON is safe). - Stateful backend task is now three files; step injections chain backend.py -> state_manifest.json -> requirements.txt. - verify_backend runs its introspection subprocess with build_backend_env (anton_state on PYTHONPATH, same injection the launcher uses) — without it a correct stateful backend failed its own SDK import. The env builder went public in backend_launcher; the underscored name stays as an alias. - evaluate_backend gains artifact_type/state_manifest kwargs and six new contract errors: anton_state listed in requirements (any type); for stateful — missing STATE slot, missing/invalid manifest (validated via anton_state.schema.StateSchema), store built at import time (AST check); for stateless — anton_state imported at all. The requirements parser also drops anton_state before install, mirroring the launcher. - Spec prompts know about durable state: api-spec gets a stateful counterpart to the stateless constraint (key-per-access-pattern, one query per listing), tech-spec pins the STATE store in the fixed stack so spec.md stops inventing sqlite. - Type-selection surfaces rewritten to the new model: create_artifact's fullstack-stateful-app description, the ARTIFACTS_PROMPT paragraph (the staging merge had left it self-contradictory), and generate_prd gathering criteria; the PRD data-model sections now capture app-owned durable state. - Store/publish_access housekeeping sets exclude the local STATE driver's .anton_state.db(-wal/-shm) and the publisher's schema snapshot; state_manifest.json itself stays a visible deliverable. cowork-server's mirror copy still needs the same names. - Contract-lock RULES extended for the six new verifier messages; 19 new tests (pure checks, subprocess integration incl. a PYTHONPATH-injection proof, orchestrator wiring, store exclusions, prompt locks). Known limitation: a live end-to-end run is currently impossible — the api.mindshub.ai gateway WAF returns 403 on any request body containing "<script" (pre-auth), which breaks all artifact generation including the pre-existing html path. See claude_workspace docs/artifact-generation-tools/tracking-stateful.md (S-13). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…4 on long generations api.mindshub.ai sits behind Cloudflare, which kills a proxied connection after ~100s of silence (524). The FSM's spec/code-writing calls used the one-shot plan()/code(), which sends no bytes over the wire until the whole completion is ready — a large tech spec or a full HTML/JS file easily exceeds that window and the run dies with "the model provider returned 524", discarding an already-confirmed PRD. Swap _plan_whole_document and _run_loop onto the existing plan_stream(), plus a new code_stream() (mirrors plan_stream(), previously only code() had no streaming sibling). A small _drain_stream() helper consumes the token-level deltas and returns the terminal StreamComplete's response, so callers keep the exact same LLMResponse shape they had before. Bytes now flow continuously regardless of total generation length, so the proxy never observes silence. generate_prd has the identical plan()/code() call sites and is exposed to the same failure mode; left out of this change to keep it scoped to the reported incident.
…ay (ENG-1986) api.mindshub.ai sits behind a Cloudflare WAF rule that 403s any request body containing the literal `<script` or `</script` (case-insensitive, even with injected whitespace) before authentication. html-app generation necessarily asks the model to emit `<script>...</script>` tags, and once written the content is echoed back into the conversation history on every later round, so the block was unavoidable through prompt wording alone. OpenAIProvider.complete()/stream() now escape `<script`/`</script` to `<_script`/`<_/script` in the outgoing system prompt and messages, and reverse it on the incoming response content and tool-call input, so genuine `<script` never appears in a request body but callers still see real tags. The marker was verified against the live gateway to fall outside the WAF rule's match. Scoped to FLAVOR_MINDS_PASSTHROUGH only — no other provider flavor talks to this gateway.
…iversal !important rule
The `* { ... !important }` check fired on the standard accessibility reset
inside `@media (prefers-reduced-motion: reduce)` — a block models emit
reflexively and one that cannot override host styles, which is what the rule
exists to prevent. Live run 2026-08-27: an otherwise valid 14-slide
presentation was rejected on exactly this false positive.
Universal !important blocks are now allowed inside `prefers-reduced-motion`
and `print` media queries (block spans found by brace counting — nested rule
blocks must not truncate or extend the span). Everywhere else the rule and
its wording are unchanged, so the contract lock stays intact.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ry budgets One shared `range(GEN_VERIFY_MAX_RETRIES + 1)` counted a loop failure (round budget, no tool calls) and a verification failure as the same kind of attempt. Live run 2026-08-27: attempt 0 died on the round budget before any verifier ran, attempt 1 failed verification on a one-line CSS fix — and the run was terminal, because the loop failure had already burned the only retry meant for acting on verifier findings. `GEN_LOOP_MAX_RETRIES` and `GEN_VERIFY_MAX_RETRIES` now count independently in both `_gen_verify_frontend` and `_gen_verify_backend` (worst case three attempts instead of two). The terminal message names the budget that actually ran out: "verification failed after N attempt(s)" only when the verifier rejected, "generation failed" otherwise — the old wording claimed a verification retry that, on the loop-failure path, never happened. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…eneration loop Four changes to `_run_loop` and its messages, all from the 2026-08-27 live run, where a complete 48 KB page was deleted and regenerated because the loop ran out of rounds counting its own slides: - Round-budget exhaustion with files on disk now returns a normal result dict (`finished: False`) instead of an error, so the caller verifies what was actually written. A missing `finish` call is not evidence the files are bad. With no files written it stays an error. - Every tool-result message now carries a `[N round(s) left]` note, switching to an explicit "wrap up NOW" instruction on the last five rounds. The model had no way to see the budget it kept dying against. - Truncation detection honours `stop_reason` (`length`/`max_tokens`) alongside the token count, and reads the client's public `max_tokens` property instead of the private `_max_tokens` (I-08). The rejection messages name a concrete next-chunk size (4,000 chars) instead of "well under ~6 KB" — the vague form did not stop either live-run attempt from burning a full 8192-token reply on an oversized append. - `read_file` passes the new `full` flag through to the sub-tool. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e_file warns on oversized chunks `read_file` existed so the model could check what landed, but returning the whole content meant re-reading a 48 KB page into the context (and through the prompt-cache prefix) just to confirm it ends with `</html>` — measured 2026-08-27 at ~19k input tokens per check. It now returns the size plus the last 500 characters; `full=true` (advertised in the schema as expensive) returns everything, and files at or under the tail size come back whole. `write_file` still accepts an oversized chunk — the call that arrived fit under the output cap by definition — but its result now warns that the next one may not: in both live-run attempts the model followed a lucky oversized chunk with a bigger one and lost a full 8192-token reply to truncation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…del verification is code's job Three prompt changes, all measured against the 2026-08-27 live run (929 s, failed): - html-app with a confirmed `prd.md`: `make_tech_spec` is told the generator receives the PRD verbatim next to its document, so it must not restate it — only `## Insights` plus terse implementation notes. The full spec was near-pure duplication (190 s / 13k output tokens restating a 20 KB PRD as 35 KB of spec that then rode into every generation prompt). Fullstack types keep the full spec — it feeds the API design. - `_ROLE_WRITE` states that a deterministic verifier checks the output after `finish` and re-prompts with exact errors on failure: re-reading and re-counting one's own files is what burned nine of twenty rounds. - `_WRITE_DISCIPLINE` names a hard 6,000-character chunk limit that explicitly covers the first `mode="w"` chunk and the first append — the two calls that actually got cut — and `_ROLE_COMMON` forbids creating new scratchpad names (the once-per-turn challenge in `handle_scratchpad` makes the rejection non-deterministic across attempts). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 2026-08-27 live run retold one article three times on the way to the artifact: scratchpad pad → prd.md (20 KB) → spec.md (35 KB) → index.html. Only the last copy is necessary. `_WRITE_PRD_INSTRUCTION` now forbids copying long-form source content that already lives in scratchpad cells — the PRD describes the structure (e.g. a slide outline, one line per slide) and cites the pad and cell; the generator reads those cells directly. Short samples stay allowed, and the existing Data-model requirements (connection code, sample rows) are untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…iants The WAF escape is visible to the model: it reads `<_script`-style forms in its system prompt and in its own echoed chunks, and reproduces the underscore in positions the strict reverse pattern does not cover. Live run 2026-08-27: a generator closed its only script block with `</_script>` (underscore after the slash), the unescape left it untouched, and the mangled tag reached the file on disk — the page's JavaScript never ran, and the calling agent spent most of its token bill discovering and hand-fixing it. The unescape now normalises every underscore variant (`<_script`, `<_/script`, `</_script`, `<__script`, `<_/_script`) — none of these is meaningful anywhere else, so being wider than the escape is safe. The escape side is unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tags Two new frontend rules, both from the 2026-08-27 live run, where a page whose only script block was closed with `</_script>` passed verification and shipped with all of its JavaScript disabled: - any underscore variant of a script tag (`<_script`, `<_/script`, `</_script`) is an error — residue of the ENG-1986 WAF escape that the provider-level unescape may not have caught; - an opening `<script` with no closing `</script` anywhere in the document is an error — the general "JS never runs" class. Deliberately narrow (absence of ANY closer, not a count mismatch): a literal `"<script>"` inside an inline-JS string legitimately unbalances the raw counts. Both rules are announced in the HARD OUTPUT CONTRACT prompt block and added to the contract-lock table — a conscious rule addition, not a table refresh. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ccessful generation After a successful run the calling agent re-verified the pipeline's work by hand: the 2026-08-27 run spent 11 planning-model calls (each resending a 42k- char system prompt plus the whole conversation) re-reading and re-parsing an artifact the generator had already verified — the bulk of the run's token bill. The success payload now carries an explicit instruction: the files were statically verified, do not re-read or re-verify them, act only on problems the user actually reports. Same nudge one level down: `read_file`'s schema now says `full=true` is never for verifying finished work — the inner loop pulled the entire 40 KB page back into its context right after the size+tail check said it was complete. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One textual conflict, in tool_handlers.py: both branches inserted code right after `_artifact_store` — this branch the `_prd_generation_failed` + `handle_generate_prd` block, staging the artifact turn-tracking helpers (`_track_artifact` and friends, #399). Kept both. Plus one semantic gap git could not see: staging's turn tracking covers create/update/open_artifact and scratchpad-detected edits, but not this branch's `handle_generate_prd` / `handle_generate_artifact`, which write into the artifact folder directly. A PRD or regeneration happening in a later turn than `create_artifact` would leave that turn unattributed. Both handlers now call `_track_artifact` when they actually wrote (generate_prd: statuses that produced prd.md; generate_artifact: success and FSM failure alike — only the crash path skips). Tests after merge: 3076 passed, 30 skipped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
verify_frontend rejected any absolute `href`/`src` outside a `<script>` tag. Two consecutive live runs failed on it, both wrongly: a dashboard built from a web article links back to its source and shows that source's images, and the PRD the user had already accepted asked for exactly those. The retry then "fixed" it by moving the same URLs into JS strings rendered through innerHTML — identical DOM, one full regeneration burned (~50% of that run's tokens), and the model now knows the workaround. A check that a correct artifact fails and an incorrect one passes is worse than no check. Only fetch() keeps the rule: a hardcoded host there breaks the artifact the moment it is published, because the backend it names is the local one. No warning replaces the removed rule — of the remaining cases, a `<link>` to a font degrades appearance without the network, and an `<img>` from the source article IS the content. The contract changes, so the Rule is removed from RULES in the contract lock deliberately, and the bullet is removed from _VISUAL_RULES: that block is declared to the model as "a static verifier checks each of these", and leaving a rule there that no longer exists is a shipped contradiction. Restoring this needs a check that sees the rendered DOM, not the HTML text. Co-Authored-By: Claude <noreply@anthropic.com>
… dropped streams
The generation loop ran every round on the client default of 8192 output
tokens, a quarter of what the gateway accepts. A 41 570-character artifact
(16 063 tokens) therefore had to be written in ~8 chunks, and each chunk is
re-sent on every later round — 23% of the node's context was chunk bodies
being paid for again.
Measured 2026-08-28 against api.mindshub.ai:
- 20480 is accepted on BOTH aliases. The note claiming 16384 for the coding
model was wrong; its ceiling is the same 20480.
- One write_file call at that budget delivered 50 402 characters of Russian
HTML in 15 754 tokens.
Round 0 deliberately does NOT get the raised budget. It runs on the planning
model, ~2.3x slower per token, and at 20480 a write there runs long enough to
lose its connection — 4 failures out of 4 at 131-143s. At 8192 the same call is
merely truncated, which the loop recovers from.
Truncation is now judged against the budget the round actually ran on. Against
the client default instead, every reply over 8192 tokens would be called
truncated and its last tool call rejected — the exact failure the raised budget
exists to remove.
Dropped streams are retried once, halving the budget. A large tool-call
argument is NOT streamed incrementally: the connection carries nothing for the
whole generation and everything arrives in one burst (112s of silence for a
59 000-character argument). That profile is identical when talking straight to
api.anthropic.com, so it is not the gateway's doing and cannot be fixed here.
Whether such a call survives is a race against the proxy's idle timeout, so a
retry on the same budget would mostly reproduce the failure; half the budget is
half the silence. Nothing has executed when a drop happens, so the retry cannot
double-apply a write.
The chunk limit moves 6 000 -> 16 000 characters and is now derived from
duration, not from the token budget: ~170 tok/s and ~2.59 characters per token
on Cyrillic prose gives ~37s of silence, a ~3x margin against the shortest
observed drop. The number was written out in four places the model reads; all
four now read the one constant, as does the halved recovery size.
test_write_discipline_names_a_hard_chunk_limit was passing vacuously:
"6,000 characters" is a substring of "16,000 characters", so it would have
stayed green through this change while checking nothing. It now asserts against
the constant, and a new test pins the limit across all four surfaces.
httpx becomes a declared dependency: already installed as a hard requirement of
both SDKs, but now imported directly to catch transport errors by type.
Co-Authored-By: Claude <noreply@anthropic.com>
Three conflicts, all in the artifact housekeeping sets, all additive.
anton/core/artifacts/store.py
- Import: both sides added a name (ARTIFACT_TYPES here,
ARTIFACT_ID_SLUG_PREFIX_LEN on staging). Union.
- _HOUSEKEEPING_FILES: staging carried the base set plus a new
_HOUSEKEEPING_DIRS = {".revisions"}; this branch had grown the file set
(backend.log, the .anton_state.db* trio, .state_manifest.published.json)
and added _EXCLUDED_FROM_FILES for the generation inputs. Kept both: the
file set is matched whole-path, so a directory name cannot live in it.
- _reconcile_files: exclusion is now
`rel in _EXCLUDED_FROM_FILES or first component in _HOUSEKEEPING_DIRS`.
anton/publish_access.py
Same two additions to one set. Staging put ".revisions" straight into
_HOUSEKEEPING_FILES, which works there because matching is on the path's
first component — but it silently broke
test_housekeeping_set_matches_publish_access_copy, the lock that keeps
anton's two copies of that set identical. Resolved by mirroring the store's
split: _HOUSEKEEPING_FILES stays equal on both sides, ".revisions" moves to
its own _HOUSEKEEPING_DIRS, and the match site tests both.
Added test_housekeeping_dirs_match_publish_access_copy so the second set is
locked the same way. Without it the next merge repeats this exact drift, and
the existing lock only catches it by accident.
tests/test_artifacts.py
Disjoint tests on both sides (revision-journal exclusion from staging;
backend.log, generation inputs, state runtime files, nested-path handling
here). Kept both.
Checked for the semantic kind of conflict git cannot see: staging touched
llm/client.py, llm/provider.py and llm/openai.py, which the generation loop
now depends on. plan_stream/code_stream still take max_tokens and tools,
StreamComplete/LLMResponse are unchanged, and stream error handling was not
touched — so the output-budget and stream-retry work from 6cfdb71 still holds.
Tests: 3195 passed, 30 skipped.
Co-Authored-By: Claude <noreply@anthropic.com>
`uv lock --check` is the first step of run-tests and it failed the whole job in 12s: pyproject gained `httpx>=0.27` (needed to catch transport errors by type when a tool-call stream dies mid-generation) without a matching uv.lock update. Regenerated with `uv lock`. The diff is exactly the two lines that declare httpx as a direct dependency — no version churn elsewhere, since httpx was already in the graph as a transitive requirement of anthropic and openai. Co-Authored-By: Claude <noreply@anthropic.com>
…I-20) generate_artifact runs a whole pipeline inside one tool-use round, and the spend ceiling is only checked between rounds. A measured run crossed the threshold mid-call, spent ~1.25M tokens over 35 LLM calls unobserved, and only tripped the ceiling once the money was gone. SpendGuard gives the pipeline a place to look. It adds no accounting of its own — TurnCost already sees every internal call — and holds a single sticky latch. The closing-round budget deliberately lives in each write loop, not here: the fullstack path runs two loops concurrently and a shared counter would leave each with too few rounds to close its file. Co-Authored-By: Claude <noreply@anthropic.com>
The merged pipeline persists its phase boundary — fingerprints, stage, declared sources, brief, rendered notes — so a repeat call can resume without re-gathering. That file lands next to dashboard.html, so it goes in the one constant both ends read: the tools that write these files and the store that keeps them out of files[]. Co-Authored-By: Claude <noreply@anthropic.com>
…e-entry Where a repeat call resumes is decided from recorded state, not inferred from matching free text. Two independent checks: request_fingerprint answers "is this the same work" from user_request alone — the only input field the outer model does not re-type in its own words each call — and pipeline_stage answers "how far did the last call get". There is deliberately no `confirmed` flag: it is derivable from the stage, and two fields describing one fact drift apart silently. Normalization means whitespace noise cannot cost a full re-gather, questions to the user included. An unreadable or unknown-shaped file degrades to "start over" rather than raising. Co-Authored-By: Claude <noreply@anthropic.com>
… (I-06) The generator never had a channel for anything fetched from the web: phase A could web_fetch an article, and the only way that reached generation was as prose inside the PRD. render_web_notes closes that — code writes it, not a model, so a source URL cannot go missing in a summary. Capped well below a fetched page on purpose. The article body itself is read by make_tech_spec, the last node that sees it; what travels onward is the pointer plus enough text to recognise the source, because phase E re-sends its context on every write round. render_exec_notes moves here unchanged from the orchestrator: both notes are the same kind of thing — a deterministic record of what discovery found — and they now sit together. Co-Authored-By: Claude <noreply@anthropic.com>
…he trace Mechanical move, no behaviour change: the two tools still exist and still do exactly what they did. Kept separate from the semantic work on purpose — a red test after this commit means the move broke something, which is a question worth being able to answer on its own. The two step logs become one. They already read the same env var and appended to the same file so a back-to-back run produced one ordered log; with one package there is nothing left to keep in sync. `run_start` becomes open-ended rather than folding its inputs into a `brief` string: there is one run_start per run now, so the folding has nothing left to reconcile, and a debug log that reshapes its input cannot be compared against the call that produced it. Co-Authored-By: Claude <noreply@anthropic.com>
372 lines held phase B, phase C and the sequence they run in. The merge adds re-entry logic to that file, so the split happens before it grows, not after. Bodies are unchanged; the sequencer keeps only the order and the revise loop. Tests follow the code: the write_prd cases move to their own file, and the sequencer's tests now patch `brief.sub_tools` rather than reaching through the orchestrator for something it no longer owns. Co-Authored-By: Claude <noreply@anthropic.com>
The shared message list is the point of the merge, and it cannot live in a second dataclass without being threaded across a package boundary — which was the strongest argument against keeping two packages. GenState absorbs the discovery fields; discovery/state.py stays as an alias so the phase modules and their tests did not all have to move in this commit. Two fields stop being mandatory, and for different reasons. `brief` defaults to empty because that is now true for most of a run: the pipeline starts at gathering, not at a brief handed in by a caller. `is_fullstack` becomes derived from `artifact_type` unless passed, because two fields describing one fact drift apart silently — a state where they disagree would send an html-app down the fullstack branch with nothing reporting it. `brief_markdown` folds into `brief`: after the merge they were the same thing under two names. Co-Authored-By: Claude <noreply@anthropic.com>
… code System prompt, tool array and messages form the provider's cached prefix. Phases A-D now share one immutable system prompt and one fixed tool array, so the region becomes an append-only prefix and each call reads the previous ones from cache instead of paying for them again. Everything that changes mid-run — current artifact type, remaining question budget, the instruction for the step being executed — moves into a step message, because the prompt cannot be rewritten without discarding the cache it exists to keep. That costs the old way of making a tool unavailable. Dropping it from the array is now a prefix change, so availability moves into ALLOWED_TOOLS_BY_STEP and a disallowed call is refused in code without running. The refusal names the step and the expected action: a model left to guess spends another round, and on a spec step that round re-sends the whole shared history. `tool_rejected` records each miss with its node, because the trade — a stable prefix in exchange for misses costing a round rather than being impossible — is only defensible if the misses are countable. draft_brief and redraw_brief are separate rows over one tool. `finish_gathering` is denied on the first (a model reaches for it whenever a prompt says "you have finished gathering", which is why the old code removed it outright) and required on the second, where it is what re-declares the artifact type and data sources after a user correction. The four call fields move from the phase-1 system prompt into a kickoff message; without that the merged pipeline would gather for a request it cannot see. `restored_context` carries the same material on the cold path, where there is no conversation to continue. Co-Authored-By: Claude <noreply@anthropic.com>
The spec nodes are the last ones that see the material gathered earlier, so they now run on the shared message list instead of a fresh conversation with a restated summary. `tools` travels with it: a history carrying tool_use/tool_result blocks is rejected outright unless tools are declared. Both nodes branch on whether a history exists. Without one — the cold-start path, where the context was rebuilt from disk — the call keeps its original one-message shape, which is exactly what the pre-merge generator did. Two consequences worth naming. The tech-spec ask now states that this is the last step that can see the source material and that anything needed verbatim must be carried into spec.md; nothing else from the conversation reaches the code-writing steps. And a tool call on a spec step is refused once, then fails the node — availability is enforced in code now, so a spec node can be handed a call it must not run, and arguing costs a full re-send of the shared history per round. That refusal deliberately does not consume a rung of the truncation budget ladder, which exists for cut answers and would otherwise leave a genuinely truncated spec with no room to retry. Co-Authored-By: Claude <noreply@anthropic.com>
The pipeline now runs gather -> brief -> PRD -> spec -> code behind a single call. Phases A-D share one message list; the spec nodes are the last to see it, and it is dropped at that boundary so the code-writing rounds stay as small as they were. `spec.md` becomes the recoding point `prd.md` used to be, which lets the PRD go back to being what it is: the record of what the user agreed to. Where a repeat call resumes is read from `discovery.json`, not guessed. Only `user_request` decides whether this is the same work — the other three input fields are re-typed in the model's own words every call — and the recorded stage decides how far the last one got. `awaiting_confirmation` means the repeat call IS the confirmation: the contract already told the agent to get it, so asking twice would be asking twice. That also makes the flow converge in environments where the user cannot be reached at all, which the previous shape could not. Two nodes are gone. `inspect_scratchpads` rebuilt data notes by matching live pads against the brief text — a workaround for a generator that did not know what had been gathered; it does now. `is_data_enough` asked a second model to re-derive the verdict the gathering phase gave by calling `finish_gathering`. What survives is the fetch loop, entered from state rather than history: `unverified_sources` is tracked explicitly, because after a user correction the notes are full of the PREVIOUS gathering's cells and an "are the notes empty" check would read that as "everything is covered". The tool contract loses `context` and with it the three-surface markdown contract that had to be kept in sync by hand. Five typed fields replace it. Five statuses replace the old string-or-dict return, each carrying the instruction for what the agent should do next. Handlers now return ToolOutcome with an explicit verdict (I-03). A successful run's trace legitimately contains the word "failed" — "backend.py failed to import in venv" is what a successful retry looks like — and the legacy substring classifier counted that as an error, feeding the per-tool streak. Co-Authored-By: Claude <noreply@anthropic.com>
…ess stream The pipeline now asks the user from inside the task the tool handler is draining, which the two-tool shape never did. A progress marker emitted mid-question lands on top of a live prompt, and the CLI's Live context does not survive that — the same seam that already cost this branch a handful of spinner fixes. Both paths to `elicit` are wrapped, not just one. The `ask_user` sub-tool is the obvious one; the brief's confirmation reaches `elicit` on its own path and is the longest question of the run and the only one that always happens. They nest, so the drain counts depth rather than holding a flag — a flag would unmute on the inner close, while the brief is still on screen. Lines produced while a question is open are not dropped silently: the last one is emitted once the answer arrives, so the user still learns where the pipeline got to. Replaying them all would only describe steps that finished while they were reading. The label-coverage lock now walks the discovery phases too, and resolves `sub_tools.STEP_*` constants rather than only string literals — the phases pass constants, and a walk that understood only literals would wave exactly the new half of the pipeline through unchecked. Co-Authored-By: Claude <noreply@anthropic.com>
The checks were only ever going to be in the FSM, which is where the tool used to be one round of work. The pipeline is five phases now: gathering can spend twenty rounds with web_fetch, the revise loop up to ten iterations of two calls each, and neither is an FSM node or a `_run_loop`. Phases A-C wind down by converting what was spent into disk state and handing back: gathering stops, the PRD gets written, and the stage says `awaiting_confirmation` so a continuation resumes cold instead of re-gathering. The brief is deliberately not shown on that path — "confirm this" and "we stopped because this got expensive, continue?" are one question, and the outer agent is the one holding the conversation to ask it in. The write loop counts its own closing rounds rather than the guard holding one counter for the run. Fullstack runs two loops at once over one guard, and a shared budget would leave each with about one round — not enough to both emit a final chunk and call `finish`. `gather` still waits for both: killing the second mid-call would leave a half-written file, which is worse than one extra round. A loop that runs out of closing rounds returns a sentinel, not an error string. The cause is the forbidden retry, not unusable code, and the instruction the agent gets differs accordingly. Co-Authored-By: Claude <noreply@anthropic.com>
All four always-on blocks and the create_artifact description sent the agent through generate_prd first. That tool no longer exists, so the instruction is now a pointer at nothing. The workflow drops from seven numbered steps to six: registering and generating stay, the separate PRD step folds into the generator, and what replaces it is the part the agent still has to get right — follow the `instruction` the status carries, and put a correction in `agent_understanding` while leaving `user_request` alone. Invariant 13 is retired deliberately, not broken and patched. `TestEveryPathNamesThePrdStep` guarded the ORDER of two tools, and existed because `generate_prd` had no `ToolDef.prompt` of its own — these blocks were the only place the model could learn the step was there. With one tool the ordering has nothing to order, so the lock is replaced by its inverse: no block may name a tool that no longer exists, every block must still name the generator, and the artifacts block must say the tool agrees the requirements itself, so the agent neither pre-interviews the user nor writes a PRD by hand. Co-Authored-By: Claude <noreply@anthropic.com>
The PRD weighed 18-21KB because it was the only channel to the generator: it had to restate the data-access code, the environment variables and enough of the source content to build from. It is not that channel any more — the spec node carries the source material forward and `data_notes` holds the code that actually ran — so the document goes back to being what its name says. The earlier attempt at this (a "do not copy long content" line) left the size unchanged, because the demand for connection code was still in the same instruction. Removing the demand is what removes the bulk. `write_prd` also now updates `state.prd`, not just the file. `prd_section` renders that field and declares it the authoritative requirements source, so a PRD rewritten during this call — which is what every user correction produces — would otherwise never reach the spec node, and the correction would be lost inside a single run. Co-Authored-By: Claude <noreply@anthropic.com>
Three defects the merge shipped, all found by review of cb851ec..1e39c2a. 1. The web_notes/data_notes channel was dead (I-06 was a no-op). Phase A recorded every scratchpad exec and web call onto the state, and nothing ever rendered them: `render_web_notes` had no production call site, and `render_exec_notes` ran only over the emergency data loop's own result. So `web_notes` was written to `discovery.json` always empty and read back always empty. On the hot path the spec node still saw the raw material through the shared history, but phase E and every cold start got nothing — which is precisely the case the channel exists for, since a model summarising a page into spec.md is where a source URL goes missing. `_absorb_discovery_notes` now renders both channels once per call, before either checkpoint save. Additive rather than rebuilt from scratch: the recorded calls hold only this call's work while the notes may already carry an earlier call's, restored from disk. 2. `_run_and_verify_app`'s retry branch treated the OVER_BUDGET sentinel as an error string, so a budget stop during a backend rebuild reached the user as "generation failed: __over_budget__" and forbade the repeat call — the opposite of what a budget stop means. 3. The data phase ran outside the spend guard: `_fetch_data_sample` never passed `spend`, and nothing re-checked the ceiling between that phase and `make_tech_spec`, the most expensive call in the pipeline. Also: a fetched web page now verifies the source it came from. Requiring a scratchpad cell instead sent every web-sourced request through the emergency data loop on every run, to re-download an article already read. And `cancelled` no longer counts as "Generated artifact files" — it is the one outcome that writes nothing. Tests: the phase-boundary lock the plan called for but never got. It asserts the FILLING of the channels, not just their transport — a test handed a pre-filled state proves the renderer works and says nothing about whether anything calls it, which is how the channel shipped dead. Each fix was mutation-checked: reverting it fails its lock. Co-Authored-By: Claude <noreply@anthropic.com>
The package docstring still described `generate(..., context, slug)` and the brief's `## Data` section, neither of which survived the merge. It now names the real signature and the boundary that replaced them. `progress.py` named `make_api_spec` twice where it meant the two spec nodes. Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Combines two artifact-generation tools into the full two-step pipeline (PRD → code):
generate_artifact(ENG-968) — a deterministic FSM orchestrator that generates a finishedhtml-app/fullstack-stateless-app/fullstack-stateful-appartifact: data-sufficiency check → technical spec → (fullstack) API spec → backend & frontend generation with verification, running in parallel → app launch & health check. Backend/frontend verification checks routing under/api/*, health endpoint presence, secret handling, dependency manifests, and CSS/URL policy for the frontend. Replaces hand-written scratchpad generation as the primary path for these three artifact types.generate_prd(ENG-969) — a bounded two-phase tool that drafts a PRD and gets explicit user confirmation before any code is written: phase 1 gathers/verifies data and asks clarifying questions; phase 2 drafts a short brief, shows it for accept/cancel/revise, and on acceptance writes the fullprd.md.create_artifact's description now routes a web artifact through both tools in order: register →generate_prd(draft + confirm requirements) →generate_artifact(write and verify the code against that PRD). Non-web artifact types (document,dataset,image,mixed) skip both steps, as before.Resolves
Test plan
pytest— full project test suite: 2213 passed, 30 skipped (pre-existing), no failuresgenerate_artifact: 7 dedicated test files (FSM graph/prompts,_run_loop, state/models, orchestrator data loop and backend/frontend retries, backend/frontend verification, tool-handler error wrapping)generate_prd: 9 dedicated test files (state, prompts, sub-tools, gathering loop, phase-2 steps, fullrun()sequence, tool registration/handler,create_artifactdescription)generate_prd's confirm/revise flow (spinner behavior, artifact-type validation, brief formatting) — see individual fix commits on this branch🤖 Generated with Claude Code