Mastra Documentation Audit

The reference surface is genuinely deep — agent.generate() is one of the most thorough API pages I've seen for an agent framework — but the agent-readable layer Mastra markets most loudly is broken in several specific, demonstrable ways, and the human onboarding path has multiple 404s on URLs the docs themselves point at.

1. /llms-full.txt 404s where the convention expects an aggregate (significant)

Location: https://mastra.ai/llms-full.txt; framing context at https://mastra.ai/docs/getting-started/build-with-ai

Problem: /llms.txt resolves and is described in /docs/getting-started/build-with-ai as "a root file ... aggregating all available documentation pages." The widely-followed companion file /llms-full.txt — which agents commonly probe as the long-form aggregate — returns HTTP 404. The Build with AI page itself does not promise /llms-full.txt, so the issue is convention, not contradiction.

Consequence: Agents that follow the de-facto convention (probe llms.txt, then llms-full.txt) hit a hard 404 on the second hop and silently fall back to scraping HTML — exactly the failure mode the file is supposed to prevent. For a framework whose marketing emphasizes agent-readable docs, the missing companion file is a credibility gap.

The fix: Either generate and serve /llms-full.txt (the concatenated markdown of all docs pages) so agents have the aggregate they expect, or explicitly call out in /docs/getting-started/build-with-ai that there is no -full aggregate and direct agents to the per-page .md endpoints instead.

2. Core /docs/getting-started/* URLs 404 while sister /guides/... URLs serve the same content (critical)

Location: /docs/getting-started/installation, /docs/getting-started/quickstart

Problem: Both https://mastra.ai/docs/getting-started/installation and https://mastra.ai/docs/getting-started/quickstart return HTTP 404. The working quickstart lives at https://mastra.ai/guides/getting-started/quickstart. Meanwhile https://mastra.ai/docs/getting-started/project-structure and https://mastra.ai/docs/getting-started/build-with-ai both resolve — so this is selective rot, not a relocated section.

Consequence: "Getting started → installation" and "Getting started → quickstart" are the two URLs every developer (and every agent crawling for installation or quickstart) will guess first. Both 404. Search engines and existing inbound links almost certainly point at the /docs/... form. Anyone landing on those pages bounces or loses trust before reaching the working content.

The fix: 301 redirect /docs/getting-started/installation and /docs/getting-started/quickstart to the live /guides/getting-started/quickstart (or restore them as full pages). Audit internal links in the docs nav, llms.txt, and external docs (the README, the announce post) so they point at one canonical URL.

3. /docs/auth/overview 404s while auth providers are heavily documented in /reference/auth/... (critical)

Location: /docs/auth/overview (404), vs. /reference/auth/clerk, /reference/auth/auth0, etc.

Problem: The reference index lists eight auth integrations (Auth0, Better Auth, Clerk, Firebase, JWT, Okta, Supabase, WorkOS), and /docs/server/middleware references mapUserToResourceId and MASTRA_RESOURCE_ID_KEY as if explained elsewhere. But /docs/auth/overview — the conceptual page that should explain how Mastra auth actually plugs into a server, what gets injected into context, how mapUserToResourceId flows through to memory scoping, and how multi-tenant isolation is enforced — returns 404.

Consequence: Developers can find a Clerk constructor signature but no end-to-end story for "how does request → auth → resource scoping → memory isolation actually work." mapUserToResourceId is described in the middleware page as a way to "automatically scope threads and memory to authenticated users," and the security guarantee ("preventing users from accessing other users' resources even with manipulated IDs") rests on an architecture page that doesn't load. This is the exact area where misconfiguration leaks one user's threads to another.

The fix: Restore /docs/auth/overview with a concrete narrative: which provider gets called when, what MASTRA_RESOURCE_ID_KEY and MASTRA_THREAD_ID_KEY actually do, how authorizeUser interacts with mapUserToResourceId, and a worked multi-tenant example. Then make sure the middleware page links to it.

4. Workflow API has churned and a featured blog post still demos the old API with no migration note (significant)

Location: /blog/solving-the-hardest-problem-in-cs (Feb 14, 2025), vs. /docs/workflows/control-flow and /docs/workflows/overview

Problem: The "solving the hardest problem in CS" post — written by the CEO, still in the linked rotation, used as a marketing argument for Mastra's API design — showcases workflow methods including .step() and .after(). The current /docs/workflows/overview and /docs/workflows/control-flow pages document a different surface: workflows are now built with createStep() + createWorkflow(), composed with .then(), .parallel(), .branch(), .map(), .dountil(), .dowhile(), .foreach(), and finalized with .commit(). Neither .step() nor .after() is mentioned anywhere in the current overview/control-flow pages — .then() survives, but the surrounding shape doesn't.

Consequence: A developer reading the marketing content and then trying to build a workflow gets two different APIs. Worse, an agent ingesting all of mastra.ai (which the llms.txt explicitly invites) will train on conflicting workflow shapes and emit code that doesn't compile against current types. There is no banner on the post, no "this used the v1 API, see migration guide," nothing.

The fix: Add an "Update: this post predates the current workflow API; see /docs/workflows/control-flow" callout to the blog post (and any other v1-era posts), and ship a migration page mapping .step() / .after() → createStep() + createWorkflow() composition, calling out that .then() is preserved but its operands are now createStep results.

5. Same flagship OpenAI model has two different IDs across pages (significant)

Location: /models and /docs/voice/overview

Problem: /models lists "OpenAI (gpt-5.5, gpt-4o-mini)" as the example provider/model identifiers. The voice overview's example agent uses model: 'openai/gpt-5.4'. These are two different version slugs for what is presented as the same OpenAI flagship across the docs surface, with no explanation of why they differ. There is no banner clarifying that one of them is a placeholder or that one example is older than the other.

Consequence: A developer (or agent) reading both pages cannot tell which slug is current, which is stale, or whether they are deliberately different. The "Mastra exposes thousands of models through one interface" pitch is undercut the moment the documented examples for the same provider don't agree with each other. An agent reading the voice quickstart and writing scaffolding code may emit a slug that no longer exists in the gateway.

The fix: Pick one canonical slug for the OpenAI flagship in worked examples and use it consistently across /models, /docs/voice/overview, and any other example agent. If the slugs are intentionally illustrative placeholders, mark them as such ("<provider>/<model-id> — see /models for the live list") rather than embedding them in copy-paste-ready code.

6. README and /models disagree on how many providers Mastra supports (significant)

Location: GitHub README (mastra-ai/mastra) vs. /models

Problem: The README pitch line reads: "Model routing - Connect to 40+ providers through one standard interface." The /models page on the marketing site says: "Mastra provides access to 3897 models from 108 providers through a unified interface." Same product, same claim, two prominent surfaces — the headline number is off by ~2.7×.

Consequence: A developer evaluating Mastra against alternatives doesn't know which number to believe, and an agent summarizing "how many providers does Mastra support" will emit different answers depending on which surface it scraped. The discrepancy also makes the README look stale relative to the live site, which seeds doubt about whether other README claims are current.

The fix: Pick the number that is true today (the one on /models is the more specific and presumably the more recent), update the README to match, and add a note that the count refreshes from the model gateway so the docs and README don't drift again.

7. /changelog is linked from the homepage but 404s (significant)

Location: Homepage primary nav ("Resources → Changelogs"), /changelog

Problem: The homepage Resources menu lists "Changelogs" as a destination, but https://mastra.ai/changelog returns HTTP 404. The recent blog index shows shipping cadence is high (Channels, Background Tasks, ClickHouse observability, Remote Filesystems, Browser Support all landed inside two weeks in late April / early May), so the absence of a working changelog isn't because there's nothing to log.

Consequence: A framework that ships breaking-adjacent features weekly and has no live changelog forces every consumer — humans tracking upgrades, agents checking compatibility, customers building production systems — to crawl the blog index and infer what changed. The versions field on Mastra and versionOverrides in the constructor only make sense against a changelog that doesn't exist.

The fix: Either restore /changelog (with per-package version bump entries — @mastra/core, @mastra/memory, @mastra/observability, etc.) or remove the dead link from the homepage nav and point at /blog until the page is real.

8. Reference index points at scorer pages that 404, and the eval overview admits the inventory is incomplete (significant)

Location: /reference (lists scorers including Answer Relevancy, Bias, Faithfulness, Hallucination, Toxicity), /reference/scorers/answer-relevancy (404), /docs/evals/overview

Problem: The reference index at /reference advertises five named scorer reference pages — Answer Relevancy, Bias, Faithfulness, Hallucination, Toxicity. The first one I checked, /reference/scorers/answer-relevancy, returns 404. Compounding this, /docs/evals/overview describes three scorer categories (Textual, Classification, Prompt Engineering) and explicitly notes that "the documentation doesn't provide a comprehensive list of all built-in scorers — it references an external page for the complete inventory." So the reference index lists five, the overview implies more across three categories, and the per-scorer pages 404.

Consequence: Scorers are how you actually wire up live evaluation in Mastra (mastra_scorers table, sampling rates, isTaskComplete.scorers in agent.generate()), so the scorer reference is load-bearing for the eval product. A developer who needs to know what arguments AnswerRelevancyScorer takes hits 404 from the index page that listed it, and has no canonical inventory to fall back on.

The fix: Audit every link out of /reference — scorers, processors, vectors, storage, voice — and either ship the missing pages or remove the entries from the index until they exist. Then publish a single canonical scorer inventory and link /docs/evals/overview to it.

9. DATABASE_SSL=true is documented as a flag that "disables authorization checks" (significant)

Location: /reference/storage/postgresql

Problem: The Postgres storage reference notes the env var DATABASE_SSL and adds, parenthetically, "e.g., 'true' disables authorization checks." Whether the intent is "disables certificate authority verification" (i.e., rejectUnauthorized: false) or something else, the documented phrasing reads as an instruction to turn off security on a value developers will copy.

Consequence: A developer setting up Mastra against managed Postgres reads "set DATABASE_SSL=true" and ships a config that silently accepts any TLS certificate, which is a credential-interception footgun. The phrasing also makes it ambiguous whether DATABASE_SSL=true enables SSL (the obvious reading) or weakens it (what the parenthetical implies). This is a security-posture issue in an env-var description that production deployments will copy.

The fix: Rewrite the line to spell out exactly what DATABASE_SSL=true does at the connection level (e.g., "sets ssl: { rejectUnauthorized: false } — only use for local development; for production, pass an ssl object with a CA"). Document the production-grade form alongside.

10. Templates page lists 13 templates but never shows the install command (minor)

Location: /templates and /reference/cli/create-mastra

Problem: /reference/cli/create-mastra documents --template and even shows npx create-mastra@latest my-mastra-project -- --template coding-agent. The /templates page lists 13 templates by name and description but never shows what to type to actually scaffold one. There's no per-template npx create-mastra@latest --template <slug> command, no slug shown next to the template name, and no link to the API endpoint (https://mastra.ai/api/templates.json) that would expose those slugs.

Consequence: Discovery is split from installation. A developer browsing templates has to guess the slug, dig into the CLI reference, or fetch the JSON manifest manually. An agent building a "scaffold me a Mastra project that does X" tool can't read templates → emit a working command in one hop.

The fix: Add the exact npx create-mastra@latest my-app --template <slug> line under each template card, and surface the slug (or link to /api/templates.json) next to the template name.

11. /docs/agents/overview appears to be near-empty (minor)

Location: /docs/agents/overview

Problem: A scrape of the conceptual landing page for the framework's headline primitive returned only "Basic Setup:" as content — no body text, no example, no pointers to constructor configuration. Even allowing for the possibility that this is a scraping artifact rather than an empty page, the page should be substantive enough that any extraction tool reads more than a single heading.

Consequence: Agents (and search engines) summarizing "what is a Mastra agent" from this URL will emit a near-empty answer. New developers landing on the docs nav's most prominent agent page may bounce to the reference, skipping the conceptual framing.

The fix: Verify the page renders its full body to non-JS clients and to agent crawlers, and make sure the visible content lays out at minimum: what an agent is in Mastra's model, the minimum constructor shape, and links into Memory, Tools, and Workflows.

What they do well

• agent.generate() reference at /reference/agents/generate is unusually thorough — every option category (execution control, callbacks, structured output, tracing, telemetry) and every return field is enumerated, including operational details like reading anthropic-ratelimit-requests-remaining from result.response.headers.
• The agent-consumability story is unusually well-thought-out in principle: per-page "Copy markdown" buttons, .md URL extensions, embedded dist/docs SKILL.md files in published packages, and a Skills npm package that ships agent-ready knowledge. The execution gaps in this audit are all the more conspicuous because the framing is right.
• The CLI surface (mastra dev, mastra studio deploy, mastra server env, mastra api) is documented as an actual operational tool — env management, log streaming, --inspect-brk, MASTRA_DEV_NO_CACHE, telemetry opt-out — rather than a thin "run this command" page.

Top 3 recommendations

1. Make the agent layer match its marketing. Ship /llms-full.txt (or explicitly disclaim it), 301 the /docs/getting-started/{installation,quickstart} 404s to the live /guides/... URLs, and reconcile the README↔/models provider count and the gpt-5.5↔gpt-5.4 example slug. The framework is sold to AI-coding-agent authors; the parts of the site agents actually fetch shouldn't 404 or contradict each other.
2. Treat workflow-API churn as a documentation problem, not just a code problem. Add a .step() / .after() → createStep() + createWorkflow() migration page, banner old blog posts that demo the v1 API, and stop letting marketing posts ship contradictory APIs into the corpus that agents and humans both read.
3. Restore /docs/auth/overview and /changelog, audit every reference-index link, and rewrite the DATABASE_SSL=true line. These are the load-bearing pages and lines for production posture (security model, what changed last week, what scorers exist, how TLS to Postgres works) and they're either missing, broken, or quietly dangerous.