Skip to content

fix(i18n): rescue unparseable translated MDX before attribute repair - #153

Open
hxy91819 wants to merge 1 commit into
openclaw:mainfrom
hxy91819:fix/i18n-mdx-syntax-repair
Open

fix(i18n): rescue unparseable translated MDX before attribute repair#153
hxy91819 wants to merge 1 commit into
openclaw:mainfrom
hxy91819:fix/i18n-mdx-syntax-repair

Conversation

@hxy91819

@hxy91819 hxy91819 commented Sep 1, 2026

Copy link
Copy Markdown
Member

Problem

Recent Translate Full runs kept failing at shard packaging even after the translation itself succeeded (e.g. run 33169524293: Docs MDX check passed (759 files) then mdx protected attribute repair failed). Root cause: the packaging-time repair chain parses every page with strict format: "mdx", while check-docs-mdx compiles pages by file extension (.md as plain Markdown). Translated pages with JSX-looking damage — fabricated <id> elements, stray closers, malformed attribute characters — therefore pass the workflow gate and only explode inside the repair chain, discarding otherwise complete (and expensive) translation shards.

Fix

Add a parser-guided syntax repair stage (.github/scripts/i18n/repair_mdx_syntax.mjs) that runs before the existing protected-attribute repair in package_artifact.py:

  • Acceptance equals the downstream chain's own tolerant parser (parseMdx), and diagnosis reuses the same tolerant masking with an offset map (parseMdxForOffsets, now exported), so valid Markdown constructs — HTML comments, prose less-than — are never diagnosed or rewritten.
  • Deterministic bounded loop (64 patches) keyed to the parser's own diagnostics: fabricated elements absent from the source are removed with prose kept (the diagnosed opener is located via a backward, comment/code-aware search — never the first same-name token); missing or stray closing tags are resynced; void elements are self-closed; unquoted attribute values are quoted (stopping before a /> delimiter); unterminated single-line comments are closed. Multi-line unterminated comments fail closed instead of guessing what stays hidden.
  • Both repair writers now validate --locale as a single safe path segment before any write (the Python-side manifest validation previously ran only after Node had already written).
  • Comment masking preserves UTF-16 code-unit length so astral characters cannot shift diagnostic offsets.
  • Artifacts record mdx_syntax_repair_outcome; failed shards keep the existing empty-artifact contract for the weekly reconciliation. Docs updated in docs/.i18n/translation-workflow.md.

Testing

  • 14-case parser-guided rescue matrix (fabricated/stray/void/attribute/comment/astral/mixed-prose scenarios) plus fail-closed cases, wiring test asserting syntax repair runs before protected-attribute repair, and locale-traversal rejection for both repair writers: python3 -m pytest .github/scripts/i18n/tests/test_i18n_scripts.py — 106 passed; 2 pre-existing failures unrelated to this change (test_translation_workflows_pin_latest_codex_and_tier_effort expects codex@0.146.0 while the workflow pins 0.146.1; a canary git-rebase test that also fails on pristine main in sandboxed environments), verified failing on unmodified HEAD.
  • Structured review (autoreview, Codex gpt-5.6-terra xhigh): clean, no accepted/actionable findings after resolving review findings (locale traversal guard, flow-level fabricated element handling, adjacent stray-closer offsets, astral-safe masking, prose-aware diagnosis, />-aware value quoting).
Full-translation shards keep failing at packaging when translated pages
contain JSX-looking damage (fabricated <id> elements, stray closers,
bad attribute characters) that the strict MDX parsers in the repair
chain reject while check-docs-mdx passes .md pages as plain Markdown.

Add a parser-guided syntax repair stage that runs before the existing
protected-attribute repair:

- acceptance equals the downstream chain's tolerant parser, so valid
  Markdown constructs (HTML comments, prose less-than) pass untouched
- fabricated elements absent from the source are removed with prose
  kept; missing or stray closing tags are resynced; void elements are
  self-closed; unquoted attribute values are quoted; unterminated
  comments are closed
- deterministic bounded loop (64 patches) keyed to the parser's own
  diagnostics; unresolvable damage still fails the shard fail-closed

Also validate --locale as a single safe path segment in both repair
writers (the Python-side manifest validation runs only after the Node
process has already written), and preserve UTF-16 length when masking
comments so astral characters cannot shift diagnostic offsets.

Artifacts record mdx_syntax_repair_outcome; failed shards keep the
existing empty-artifact contract for the weekly reconciliation.
@hxy91819
hxy91819 requested a review from a team as a code owner September 1, 2026 05:28
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant