157 lines
10 KiB
Markdown
157 lines
10 KiB
Markdown
---
|
|
title: Documentation verification
|
|
description: The six documentation-contract checks in scripts/check-docs.mjs and how to prove they fail
|
|
---
|
|
|
|
# Documentation verification
|
|
|
|
`scripts/check-docs.mjs` is the drift detector for facts that are mirrored out of source into
|
|
Markdown, `POM.yml`, `package.json`, and `RELEASE.json`. Every fact POM documents has exactly one
|
|
canonical owner in `src/`, `scripts/`, or `package.json`; the checker fails when a mirror stops
|
|
matching its owner.
|
|
|
|
The checker is read-only, self-contained ESM, and imports only `node:fs`, `node:path`, and
|
|
`node:url`. It never writes files, never runs a build, and needs no `node_modules`, so it stays
|
|
runnable while the npm registry is unreachable.
|
|
|
|
## Running the checker
|
|
|
|
```bash
|
|
node scripts/check-docs.mjs
|
|
```
|
|
|
|
Each check prints one line, with actionable detail indented beneath a failure:
|
|
|
|
```text
|
|
PASS check 1/6 resource-package-parity
|
|
PASS check 2/6 command-parity
|
|
note: /pom resume is absent from src/command.ts HELP; allowlisted in KNOWN_COMMAND_DIVERGENCES
|
|
FAIL check 3/6 layout-contract
|
|
docs/INVENTORY.md:43 names "06_exports", which src/persistence.ts bootstrapProject never creates; the canonical names are 06_ledgers and 07_exports
|
|
POM docs verification FAIL (1/6: layout-contract)
|
|
```
|
|
|
|
The final line is `POM docs verification PASS` with exit code 0, or
|
|
`POM docs verification FAIL (<count>/6: <check names>)` with exit code 1. All six checks always
|
|
run; one failure never hides another.
|
|
|
|
## The six checks
|
|
|
|
| # | Name | Canonical owner | Mirrors checked |
|
|
|---|---|---|---|
|
|
| 1 | `resource-package-parity` | `src/prompts.ts`, `src/themes.ts`, `src/events.ts` | `package.json` `files` |
|
|
| 2 | `command-parity` | `src/command.ts` `HELP` | `README.md` command map, `POM.yml` `commands:` |
|
|
| 3 | `layout-contract` | `src/persistence.ts` `bootstrapProject` | `docs/INVENTORY.md`, `skills/pom/references/artifacts.md` |
|
|
| 4 | `prompt-order-contract` | `src/prompts.ts` `composePromptStack` and `renderStagePrompt` | `POM.yml` `prompt_precedence` |
|
|
| 5 | `link-and-orphan` | the `docs/` tree on disk | every `docs/*.md` page |
|
|
| 6 | `release-evidence` | `scripts/release-manifest.mjs` gate readers | `RELEASE.json` `verification` |
|
|
|
|
### 1. resource-package-parity
|
|
|
|
Defends the published tarball against missing runtime resources. The checker regex-matches
|
|
`new URL("../<root>", import.meta.url)` in the three resource-loading modules, then asserts every
|
|
discovered root is listed in `package.json` `files` and exists in the repository. The root list is
|
|
derived from source on every run, so a newly bundled resource directory cannot be added in code and
|
|
silently left out of the package.
|
|
|
|
Historical defect: `prompts` was loaded by `src/prompts.ts` and `src/events.ts` but omitted from
|
|
`package.json` `files`, so an installed copy of the extension shipped without any prompt fragments
|
|
while the repository checkout worked. A hardcoded root list would not have caught it, and would
|
|
rot the next time a root is added, so discovery failure is itself a failure: if the regex matches
|
|
nothing the check reports a broken checker rather than an empty pass.
|
|
|
|
### 2. command-parity
|
|
|
|
Defends the advertised command surface. The verbs are extracted from the `HELP` template literal in
|
|
`src/command.ts`, from the `## Command map` table in `README.md`, and from the `commands:` block in
|
|
`POM.yml`. A bare `/pom` normalizes to the verb `dashboard`, and only the first token after `/pom`
|
|
is significant, so `/pom theme install` contributes `theme`. The three sets must agree.
|
|
|
|
Historical defect: `/pom settings` and `/pom doctor` were implemented and advertised by `HELP` but
|
|
missing from both documented command maps, so operators had no written record of two shipped verbs.
|
|
|
|
### 3. layout-contract
|
|
|
|
Defends the generated project layout. `06_exports` and `06_logs` are forbidden in
|
|
`docs/INVENTORY.md` and `skills/pom/references/artifacts.md`, and the canonical `06_ledgers` and
|
|
`07_exports` must both appear in `docs/INVENTORY.md`. Failures cite the offending line numbers.
|
|
|
|
Historical defect: docs described the export directory as `06_exports` and invented a `06_logs`
|
|
directory. `bootstrapProject` creates neither, so operators looked for output in paths that never
|
|
exist and scripted against directory names the kernel never writes.
|
|
|
|
### 4. prompt-order-contract
|
|
|
|
Defends prompt provenance ordering. `POM.yml` `prompt_precedence` must open with the immutable POM
|
|
production law that `composePromptStack` prepends, must state that project-local fragments replace
|
|
bundled fragments by matching ID, and must close with the executable stage contract. The stage
|
|
contract must appear only as the final entry.
|
|
|
|
Historical defect: the manifest listed the executable stage contract mid-list, contradicting
|
|
`renderStagePrompt`, which appends it last. Anyone reasoning about prompt precedence from the
|
|
manifest inferred the wrong effective law, since a later fragment overrides an earlier one.
|
|
|
|
### 5. link-and-orphan
|
|
|
|
Defends wiki navigability in two independent directions, reported separately. Every relative
|
|
Markdown link in `docs/*.md` must resolve to a path that exists on disk, with fragments and query
|
|
strings stripped and absolute, anchor-only, and scheme-qualified links skipped. Separately, every
|
|
`docs/*.md` page except `index.md` must be linked from `docs/index.md` or `docs/navigation.md`.
|
|
|
|
Historical defect: navigation entries used repository-root paths such as `README.md` and
|
|
`skills/pom/references/commands.md` from inside `docs/`, where they resolve to `docs/README.md` and
|
|
`docs/skills/...` and render as dead links. New pages were also added without a navigation entry,
|
|
leaving them reachable only by direct URL. The reachability half applies to this page too: a new
|
|
`docs/*.md` file, including `VERIFICATION.md`, must be linked from `docs/index.md` or
|
|
`docs/navigation.md` or the check reports it as unreachable.
|
|
|
|
### 6. release-evidence
|
|
|
|
Defends release honesty. Every key under `RELEASE.json` `verification` must be an object carrying a
|
|
`status` string of `PASS`, `FAIL`, `NOT RUN`, or `BLOCKED`. A bare string, number, null, or array
|
|
value fails, and so does an unrecognized status. `scripts/release-manifest.mjs` produces these
|
|
objects from evidence files under `00_admin/validation/`; this check is the regression guard that
|
|
keeps them evidence-bound.
|
|
|
|
Historical defect: the manifest carried hard-coded `"PASS"` strings for gates that had never been
|
|
executed, so the release claimed verification it could not support.
|
|
|
|
## Known divergences
|
|
|
|
`KNOWN_COMMAND_DIVERGENCES` in `scripts/check-docs.mjs` is the only escape hatch, and it contains
|
|
exactly one entry: `resume`. `/pom resume [state-path]` is documented in the `README.md` command map
|
|
and in the `POM.yml` `commands:` block but is deliberately absent from `HELP`. `HELP` stays
|
|
canonical for what the built-in help advertises, the docs keep the verb, and check 2 prints the
|
|
divergence as a `note:` line so it stays visible while the check still passes deterministically. Any
|
|
divergence not in the allowlist fails.
|
|
|
|
## Mutation proofs
|
|
|
|
Each row is a specified procedure for proving that a check actually defends its contract: apply the
|
|
mutation, run `node scripts/check-docs.mjs`, confirm the named check reports `FAIL` with the
|
|
expected message, then revert the mutation and confirm the run returns to
|
|
`POM docs verification PASS`. Mutations are destructive edits to canonical files and must be
|
|
reverted before any commit or package step. This section specifies the procedure and the expected
|
|
output; it records no executed results, because the checker was not run when this page was written.
|
|
|
|
| Check | Mutation to apply | Expected failure |
|
|
|---|---|---|
|
|
| 1 `resource-package-parity` | Remove `"prompts"` from the `files` array in `package.json` | `FAIL check 1/6 resource-package-parity` — `package.json: files must contain "prompts" because src/prompts.ts and src/events.ts load it at runtime` |
|
|
| 2 `command-parity` | Delete the `/pom status [--json]` row from the `## Command map` table in `README.md` | `FAIL check 2/6 command-parity` — `/pom status is documented in src/command.ts HELP, POM.yml commands: but absent from README.md ## Command map` |
|
|
| 3 `layout-contract` | Rename `06_ledgers` to `06_logs` on the ledger line of `skills/pom/references/artifacts.md` | `FAIL check 3/6 layout-contract` — `skills/pom/references/artifacts.md:10 names "06_logs", which src/persistence.ts bootstrapProject never creates` |
|
|
| 4 `prompt-order-contract` | In `POM.yml` `prompt_precedence`, move the executable stage contract entry above the project-local fragment entry so it is no longer last | `FAIL check 4/6 prompt-order-contract` — `POM.yml: prompt_precedence must end with the executable stage contract appended by renderStagePrompt`, plus the misplaced-entry position |
|
|
| 5 `link-and-orphan` | In `docs/index.md`, change the Package inventory link target from `INVENTORY.md` to `INVENTORY-old.md` | `FAIL check 5/6 link-and-orphan` — `docs/index.md: broken relative link INVENTORY-old.md resolves to docs/INVENTORY-old.md, which does not exist on disk`, and `docs/INVENTORY.md: unreachable page` if `docs/navigation.md` no longer links it either |
|
|
| 6 `release-evidence` | Replace the `verification.runtimeSmoke` object in `RELEASE.json` with the bare string `"PASS"` | `FAIL check 6/6 release-evidence` — `RELEASE.json: verification.runtimeSmoke is the bare string "PASS"; every gate must be an evidence object carrying a status` |
|
|
|
|
Row 5 mutates a link rather than deleting a file, so the mutation is a one-token edit and the revert
|
|
is exact. Row 6 must be reverted by regenerating the manifest with `npm run release:manifest` rather
|
|
than by hand, so the recorded hashes stay consistent with the packaged files.
|
|
|
|
## Relationship to the other gates
|
|
|
|
`node scripts/check-docs.mjs` is a documentation-contract gate, not a replacement for the release
|
|
ladder in `package.json`. It complements `node scripts/verify.mjs`, which checks structure and
|
|
counts, and `npm run release:check`, which proves `RELEASE.json` matches the packaged tree. The
|
|
documentation checker is the only gate that reads prose, so it is the only one that can catch a doc
|
|
that quietly contradicts source.
|