Static analyzer for SKILL.md files. Validates frontmatter, body sizing, file references, and cross-agent compatibility against the agentskills.io specification. No network calls. No LLM API calls. No file mutations.
1058 tests cover all rule modules.
pip install skillcheckRequires Python 3.10 or later. For more accurate token estimates, install the optional extra:
pip install "skillcheck[tiktoken]"Token counts are estimates, and the sizing rules report them as such. The bands below are measured, not estimated: scripts/measure_token_error.py compares the offline heuristic against tiktoken cl100k_base across a corpus of 61 real SKILL.md files, for the three spans the rules actually size.
| Span | Rule | Median error | p95 error | Direction |
|---|---|---|---|---|
| Whole file | sizing.total-tokens |
23.0% | 30.7% | over-estimates 61/61 |
| Frontmatter | disclosure.metadata-budget |
25.9% | 35.3% | over-estimates 61/61 |
| Body | disclosure.body-budget |
22.7% | 30.7% | over-estimates 61/61 |
The bias is one-directional: the offline heuristic read high on every file in the corpus. For a budget check that is the safer direction, since it warns early rather than missing a skill that is genuinely over, but it means a warning close to a threshold is as likely to be the estimator as the file.
Install the tiktoken extra before trusting a diagnostic near a budget limit. With pip install "skillcheck[tiktoken]" the counts come from cl100k_base directly instead of the heuristic. tiktoken downloads its vocabulary on first use and caches it, so it is offline only once that cache is warm; the heuristic never touches the network.
Neither option reproduces Claude's own tokenizer, because Anthropic's vocabulary is not published, so the residual error against what Claude actually counts is unknown and not measured here. That is why token-based diagnostics are WARNING severity while line-based ones are not, and why messages carry the estimate and the threshold (got 612 tokens) so you can judge the margin yourself.
One measured result is worth flagging: the plain chars / 4 rule of thumb scored a 6.3% median error on the same corpus, against the word-run heuristic's 23.0%. The comment in tokenizer.py claimed the opposite. Changing the estimator would move every token diagnostic, so it has not been changed here; the number is recorded so the decision can be made deliberately.
skillcheck SKILL.md # validate one file
skillcheck skills/ # scan a directory for files named SKILL.md
skillcheck SKILL.md --format json
skillcheck --help # full flag referenceSample output:
✔ PASS skills/claude-api/SKILL.md
line 2 ⚠ warning frontmatter.name.reserved-word Name contains the term 'claude'.
line 4 · info frontmatter.field.ecosystem Field 'license' is ecosystem-common.
Checked 18 files: 18 passed, 0 failed, 29 warnings
- uses: moonrunnerkc/skillcheck@v1
with:
path: skills/Diagnostics appear as inline PR annotations. Inputs documented in action.yml.
repos:
- repo: https://github.com/moonrunnerkc/skillcheck
rev: v1.5.0
hooks:
- id: skillcheckThe hook passes --no-color by default so the captured pre-commit log stays clean. Override or extend with args: in your .pre-commit-config.yaml (for example, args: ["--no-color", "--strict"]).
- Frontmatter: required fields, types, name and description length limits, reserved-word collisions.
- Description quality: 0-100 score across action verbs, trigger phrases, keywords, specificity, and length.
- Sizing: line and token thresholds against the agentskills.io disclosure budgets. Token figures are estimates; see token estimation accuracy.
- References: broken links, escapes outside the skill directory, depth limits.
- Cross-agent compatibility: Claude Code, VS Code, Codex, Cursor.
- Capability graph (
--analyze-graph): orphaned capabilities, unused inputs, unproduced outputs, unreferenced tools. - History ledger (
--history): per-skill append-only JSON file tracking validation results across runs.
When the calling agent can run a prompt, skillcheck can ingest its response and merge findings into the report:
skillcheck SKILL.md --emit-critique-prompt > prompt.txt
# hand prompt.txt to the agent, then:
skillcheck SKILL.md --ingest-critique response.jsonThe same flow exists for capability graph extraction (--emit-graph-prompt / --ingest-graph). Prompt variants are tuned per agent via --critique-agent and --graph-agent (claude, codex, cursor).
An ingested response describes exactly one skill, so --ingest-critique and --ingest-graph require a single resolved SKILL.md. Pointing them at a directory that expands to more than one skill exits 2. Run the ingest once per skill.
| Code | Meaning |
|---|---|
0 |
No errors. Warnings alone exit 0 unless --strict is set. |
1 |
One or more errors. Also: warnings with --strict (the umbrella --strict-vscode / --strict-cursor only escalate their own diagnostics; the umbrella additionally escalates any warning-only run). Also: history.skill.regressed with --fail-on-regression. Also: any ingest parse failure. |
2 |
Input or argument error (missing path, conflicting flags, malformed input, an ingest flag pointed at more than one skill). |
3 |
Symbolic checks passed but an ingested critique reported semantic errors. |
When both 1 and 3 would apply, 1 wins so CI consumers see the higher-severity signal.
Defaults live in a skillcheck.toml discovered upward from the validated path. Override per invocation with --config PATH. Organization-specific frontmatter keys belong under [frontmatter] extension_fields. Override the name reserved-word list with [frontmatter] reserved_words = ["acme", "internal"] (an empty array reverts to the defaults).
--ignore PREFIX suppresses any diagnostic whose rule ID starts with PREFIX. The prefix is matched against the full dotted rule ID, so all three levels work: a top-level category (--ignore sizing), a category-and-field pair (--ignore frontmatter.name), or a fully-qualified rule (--ignore compat.unverified). The flag is repeatable.
CONTRIBUTING.md: testing, maintainer workflows, rule-authoring conventions.docs/case-study-v1-real-world-runs.md: runs against the Anthropic skills corpus.docs/case-study-silent-skill-failure.md: VS Code dirname-mismatch incident.skills/skillcheck/SKILL.md: a SKILL.md that passes every rule.
Pushing a version tag (v1.2.3) runs .github/workflows/release.yml, which builds the wheel and sdist, issues a SLSA build provenance attestation via actions/attest-build-provenance, and publishes to PyPI through trusted publishing. To verify a release artifact before installing:
gh attestation verify dist/skillcheck-*.whl --owner moonrunnerkcThis confirms the wheel was built by moonrunnerkc/skillcheck CI from the source at the tagged commit. Untagged builds (PR and main-branch CI) are not attested or published.
MIT. See LICENSE.