Mdspec

mdspec Documentation Audit

The entire public documentation surface is a single page (/docs/api-reference) that the site treats as both quickstart and reference, while six advertised sub-paths 404 or silently rewrite. The rendered docs and the llms-full.txt contradict each other on how many required keys the config file has, and the homepage marketing copy contradicts the docs on when the agent layer runs.

1. Documentation is one page; six advertised sub-paths 404 or silently rewrite (critical)

Location: /docs, /docs/quickstart, /docs/integrations, /docs/errors, /docs/configuration, /docs/getting-started, /docs/cli

Problem: Every documentation URL a developer would naturally guess either 404s or silently rewrites to /docs/api-reference. /docs/quickstart, /docs/integrations, /docs/errors, /docs/configuration, and /docs/getting-started all return 404. /docs and /docs/cli rewrite to /docs/api-reference. There is no sitemap (returns the SPA shell) and no per-integration sub-pages for ClickUp, S3, Notion, or Confluence — even though the homepage and pricing page advertise four integrations and an "Agent layer."

Consequence: A developer evaluating mdspec cannot find documentation for any integration they are paying for. There is no error reference, no CLI sub-command reference, no per-target setup guide. Agents indexing the docs see one page total and assume that's the entire product surface, so retrieval over mdspec docs will always be shallow.

The fix: Build out the routes the page itself implies ("Docs — Getting started & API reference") — at minimum a per-integration page for each of ClickUp/Notion/Confluence/S3, a CLI reference, and an errors page. Return 404s for unbuilt routes, but don't silently rewrite /docs/cli to a getting-started page — that's a worse signal than a missing page.

2. llms-full.txt says the config has three top-level keys; the rendered docs say four (critical)

Location: https://mdspec.dev/llms-full.txt vs https://mdspec.dev/docs/api-reference

Problem: The rendered docs explicitly state the .mdspecmap file has four top-level keys: version (required, "Always 1"), mappings (required), default, and specs. The llms-full.txt artifact instead says: "Three top-level sections: mappings, default, specs" — version is dropped entirely. The two artifacts also disagree elsewhere: sync_all_on_first_run: false appears in the YAML example of both with a trailing "# see below" comment, but there is no "below" section in either artifact explaining it; frontmatter_map is referenced in llms-full.txt without a definition.

Consequence: llms-full.txt is the canonical entry point an AI coding agent reads. An agent following only llms-full.txt will generate .mdspecmap files without the required version: 1 key — a guaranteed schema-mismatch with whatever the API actually validates against. Two human-readable docs that disagree on whether a key is required is the worst possible failure mode: humans pick one and live, agents silently produce broken configs.

The fix: Generate llms-full.txt from the same source as the rendered docs so it cannot drift. Until then, fix the "three top-level sections" line in llms-full.txt to match the docs page, and either remove sync_all_on_first_run and frontmatter_map from examples or document them.

3. Agent layer timing contradicts itself between homepage and docs (significant)

Location: Homepage / ("Features" and "Under the hood" sections) vs /docs/api-reference and /privacy

Problem: The homepage describes the agent layer as post-publish: "Transform specs post-publish with built-in templates like task summaries and release notes." But the same homepage's "Under the hood" diagram places Agent Transform between the mdspec API and Destinations — i.e., before the spec lands in Notion/ClickUp. The privacy policy is unambiguous in the same direction: "sent to Anthropic's Claude API for transformation before publishing."

Consequence: A developer reading the "Agent layer" bullet thinks the spec is published verbatim and then a transformed copy is generated; a developer reading the diagram and privacy policy thinks the markdown is rewritten before it ever hits the destination. These are operationally very different — the post-publish reading suggests an audit trail of the original; the pre-publish reading means your spec never lands intact in the target tool. Nobody can build CI/CD with confidence around this.

The fix: Pick one. If the agent runs pre-publish (which the diagram and privacy policy both indicate), rewrite the "Agent layer" homepage bullet to drop "post-publish." If both modes exist, document them as separate features.

4. The npm typosquat warning lives only on marketing, not in the docs (critical)

Location: Homepage / ("⚠ The npm package is mdspeci (trailing i)…") vs /docs/api-reference

Problem: The homepage carries a prominent warning: "⚠ The npm package is mdspeci (trailing i) — not mdspec. Running npx mdspec installs an unrelated third-party package and exposes your CI secrets to it." The actual docs page uses npx mdspeci in code samples but never repeats this warning, never mentions the squatted name, and never explains the risk.

Consequence: This is a credential-exfiltration footgun, not a doc-cleanup issue. Agents and developers who land on the docs directly — a normal path for someone returning to set up CI, or for any agent that crawls /docs first — will copy npx mdspeci, mistype the trailing i, and ship MDSPEC_TOKEN (a production credential with publish rights to your team's docs targets) to an unrelated package owned by a third party. The warning belongs on the page where the install command lives.

The fix: Pin the same warning at the top of the docs page, immediately above the first npx mdspeci snippet. Better: publish a defensive mdspec package that errors with a redirect message, or file a typosquat report against the squatted name on npm.

5. Changelog, blog, security, and status pages are gated behind login (significant)

Location: /changelog, /blog, /security, /status

Problem: Anonymous users hitting /changelog, /blog, /security, or /status are redirected to /login?next=/changelog. The changelog is the page a developer checks before adopting a tool to gauge release cadence; the security page is what they check before piping MDSPEC_TOKEN through CI; the status page is what they check when their build breaks.

Consequence: Evaluating mdspec from outside requires creating an account just to read the release history or security posture. Agents auditing the product for adoption have no way to verify maintenance cadence. A "status page" behind a login is the opposite of a status page.

The fix: Make /changelog, /security, and /status public. Login-gating a status page is incompatible with what a status page is for.

6. "Alias system" is a paid feature with zero documentation (significant)

Location: /pricing (both Free and Pro tiers) vs /docs/api-reference

Problem: The pricing page lists "✓ Alias system" as a feature on both tiers, alongside CI/CD integration and Git-native change detection. The .mdspecmap example in the docs uses parent: alias:eng-docs with an inline comment "alias: prefix → dashboard alias" — and that comment is the entire explanation. No docs page explains how aliases are created, where they are scoped (per-project? per-organisation?), how they resolve, or what happens when an alias is missing.

Consequence: A feature explicitly sold on the pricing page cannot be configured by reading the docs. The example uses an alias prefix without any path for a developer to actually create or look up an eng-docs alias. Agents cannot generate a working .mdspecmap for any team that uses aliases, which the docs example implies is the recommended pattern.

The fix: Add an "Aliases" section to the docs page (or a sub-page) describing where aliases are managed in the dashboard, how the alias: prefix resolves, and what happens on a missing-alias failure.

7. sync_all_on_first_run and frontmatter_map appear in examples but are never defined (significant)

Location: /docs/api-reference and /llms-full.txt

Problem: The full .mdspecmap example includes sync_all_on_first_run: false with a trailing comment "# see below" — but no "below" section explains it. The llms-full.txt references frontmatter_map without a definition. The documented top-level keys are version, mappings, default, specs — yet sync_all_on_first_run sits at the top level of the example and frontmatter_map is mentioned without scope.

Consequence: Developers copy the example verbatim and either (a) leave a key they don't understand controlling first-run behaviour, or (b) drop it and hit different behaviour on first sync vs. subsequent syncs. Schema validators that agents auto-generate from the docs will reject these undocumented keys.

The fix: Either remove these keys from the example or add a "Configuration keys" reference table listing every top-level key — required/optional, type, default, description. The current four-key table omits keys that already appear in the canonical example.

8. Per-spec id: field format is undocumented (significant)

Location: /docs/api-reference (the specs: section of the example)

Problem: The .mdspecmap example shows id: CU-182 and id: CU-305 for two specs targeted at ClickUp. The CU- prefix strongly implies a ClickUp ID, but no docs page explains the format, where to find the ID in ClickUp, whether the prefix is required, what an id: looks like for Notion / Confluence / S3 targets, or whether id: is even valid when the destination isn't ClickUp.

Consequence: Per-spec id: is how an existing doc in the target tool gets reused on subsequent syncs (the alternative would be creating duplicates). Developers using Notion or Confluence cannot tell whether they should use a different ID format, omit the field, or whether the feature is ClickUp-only. Agents writing .mdspecmap files for non-ClickUp targets will guess.

The fix: Add a per-integration table documenting the id: field format for each destination — or state explicitly that id: is ClickUp-only and document the alternative for other targets.

9. MDSPEC_TOKEN auth contract is never explained (significant)

Location: Homepage CI snippet uses MDSPEC_TOKEN: ${{ secrets.MDSPEC_TOKEN }}; nothing further appears in /docs/api-reference or /llms-full.txt.

Problem: The entire CI integration pivots on a single secret named MDSPEC_TOKEN, but no docs page explains where the token is generated, how to rotate it, what scope it grants (per-project? per-organisation? read vs. publish?), how to revoke a compromised token, or what happens to in-flight syncs when a token is revoked. The privacy policy lists Supabase as the auth backend but doesn't surface a token UX.

Consequence: For a tool whose adoption model is "pipe this secret through CI on every push," the missing auth contract is a security and operations gap. A compromised CI runner means a compromised MDSPEC_TOKEN — and with no rotation procedure documented, the only response is "create a new account." This is the exact information a security review will demand before approving CI integration.

The fix: Add an "Authentication" section to the docs covering: where to create a token in the dashboard, scope and permissions, rotation procedure, revocation effects, and recommended GitHub Actions environment-secret setup.

10. No JSON Schema for .mdspecmap despite it being the entire product surface (significant)

Location: Entire docs surface

Problem: The .mdspecmap YAML file is the configuration interface for the product, yet there is no published JSON Schema, no OpenAPI spec for the mdspec API the CLI calls, and no machine-readable description of the config grammar. The rendered docs use a prose table with four rows; llms-full.txt is prose that disagrees with the rendered table on key count. Everything an agent needs to validate a .mdspecmap before commit is buried in one example block — an example that contains keys not listed in the table.

Consequence: An agent writing or editing .mdspecmap for a developer has no way to validate the result short of pushing to GitHub Actions and watching for failure. IDEs cannot offer schema-backed completion. Undocumented keys like frontmatter_map and sync_all_on_first_run cannot be discovered programmatically.

The fix: Publish a JSON Schema for .mdspecmap (link it from the docs and from llms-full.txt). Publish an OpenAPI document for the mdspec API the CLI calls. Both are small, mechanical artifacts that buy correctness for every agent and editor.

11. "Docs backup" feature description undercuts its own framing (minor)

Location: Homepage / ("Features" → "Docs backup")

Problem: The "Docs backup" feature reads: "Remove a file from the repo and it stays in the target tool. Published docs are not deleted automatically — manual cleanup in the target tool is required if you want to remove a published doc." This is framed as a feature ("backup") but the second sentence reveals it's a deletion-semantics limitation — the system cannot remove a doc once published.

Consequence: Developers expecting Git-native sync (Git is the source of truth → repo deletions propagate) will be surprised when git rm spec.md leaves stale docs in ClickUp/Notion forever. The marketing framing hides what is operationally a one-way sync.

The fix: Rename the feature to "Add-only sync" or "Append-only publishing" and document the manual cleanup requirement in the docs, not just on the marketing page. Better: add a delete_on_remove: true option and document it.

12. The page at /docs/api-reference is not an API reference (minor)

Location: /docs/api-reference

Problem: The URL path is /docs/api-reference, but the visible H1 is "Getting started," the meta description promises "Learn how to set up mdspec: create a .mdspecmap file, add the CI step…", and the HTML page title is "Docs — Getting started & API reference | mdspec" — the page is self-aware that it's two things stacked into one route. There is no documented HTTP API on this page — no endpoints, no request/response shapes, no authentication contract beyond MDSPEC_TOKEN being set as an env var, despite the privacy policy describing a hosted API that routes, transforms, and fans out content.

Consequence: Anyone linking to the "API reference" — including search results, deep-links from other tools, and agents — lands on getting-started content. There is no actual API reference for the mdspec API that the CLI calls. Deep-linking to specific reference sections is impossible because both halves share one route.

The fix: Split the route. /docs/getting-started for the quickstart, /docs/api-reference for the actual mdspec API (auth, routing, fan-out endpoints) the CLI uses against. The page already advertises both halves in its title — give each its own URL.

What they do well

• The homepage warning about the mdspeci (trailing-i) package name is a genuinely important security note — it just needs to also live in the docs.
• The privacy policy is unusually explicit about content flow ("processed transiently," "not durably stored," explicit Anthropic call-out for agent templates) — that level of clarity is rare and worth preserving.
• The pricing page is honest: both tiers include every feature, only limits differ. No artificial gating.

Top 3 recommendations

1. Fix the three-vs-four key contradiction immediately. llms-full.txt saying "three top-level sections" while the rendered docs require four (including version) is the single highest-leverage doc bug — every agent that follows the LLM-friendly artifact will generate invalid configs. Regenerate llms-full.txt from the same source as the rendered docs.
2. Move the mdspeci typosquat warning into the docs page, immediately above the first install snippet, and document MDSPEC_TOKEN auth (creation, scope, rotation, revocation). Both together close the credential-security gap.
3. Stop having one docs page pretend to be ten. Build out /docs/quickstart, /docs/cli, /docs/errors, an "Aliases" page, a per-integration page for each of ClickUp/Notion/Confluence/S3, and publish a JSON Schema for .mdspecmap. Either return 404 or build the page — silent rewrites to a getting-started page are worse than a 404.