The Herm Agents API reference itself is tight and internally detailed, but the machine-readable entry point describes the wrong product, the marketing site sells an API surface the docs have never heard of, and ten homepage "Read docs" links land on 404s.
1. llms.txt describes a completely different product — and every docs page points agents at it (critical)
Problem: Every page's .md export begins: "Fetch the complete documentation index at: https://docs.herm.run/llms.txt — Use this file to discover all available pages before exploring further." The file it points to is titled "Prism Auth - Authentication-as-a-Service API" and documents an unrelated product: POST /login for browser-agent session cookies, Bearer pk_... authentication, and a dashboard at https://prismai.sh/workspace. The Herm API uses x-api-key auth against https://api.herm.run and has no /login endpoint. The file also lists zero docs.herm.run pages, so it is not an index of anything.
Consequence: AI coding agents follow that header literally. An agent asked to integrate Herm will fetch llms.txt, conclude the API is Bearer-token auth with a /login endpoint at prismai.sh, and generate confidently wrong code. This is worse than having no llms.txt: it is an authoritative pointer to the wrong product, on a site whose customers are, by definition, building AI agents.
The fix: Replace llms.txt with an actual index of the 14 Herm docs pages (the correct content already exists in llms-full.txt). Audit the Mintlify project for other Prism Auth leftovers.
2. Homepage sells an SDK and a deployments API that the docs say don't exist (critical)
Problem: The homepage hero shows import Herm from "herm"; herm.deployments.create({ customer_id, system_prompt, model: "anthropic/claude-fable-5", features: { memory: true, automations: true } }) with a preview URL of api.prismagents.com/v1/deployments/dep_7xK9s2, and the FAQ states "One POST /v1/deployments call provisions an isolated, container-sandboxed AI agent." The docs contradict every element: the quickstart says "This quickstart uses ; no SDK is required," the API base is , the resource is (not deployments), and fields are camelCase (, , ). This divergence is fresh churn, not ancient drift: a search-engine crawl of /introduction cached one day before this audit still showed SDK-based content ("The SDK wraps the REST API at ", , ), all of which is gone from the live page. Worse, the npm package that the hero imports is an unrelated third-party library ("UI components for herm.dev," published by a different author) — a developer who runs gets someone else's UI kit.
Code verification
Runtime snippet checks
Run the public examples from this audit in an isolated sandbox and attach the pass, fixed, skipped, and failed counts to the report.
Consequence: A developer (or agent) starting from the homepage will install the wrong npm package, call a nonexistent /v1/deployments endpoint on a different domain, and pass snake_case fields the real API rejects with 400 validation_error. Nothing on either surface acknowledges the other exists.
The fix: Rewrite the homepage code sample against the real API (POST /v1/agents at api.herm.run, curl or a real published SDK). If a TypeScript SDK is planned, publish it under a namespace you control and document it; until then, remove import Herm from "herm" from marketing.
3. Ten homepage "Read docs" links resolve to Page-Not-Found (critical)
Problem: The homepage's sixteen-feature grid links each capability to a docs page. Ten of those targets do not exist and render Mintlify's "Page not found — The page you are looking for does not exist" screen (served soft-404 after a 308 redirect): /agent-configuration/setup, /agent-configuration/mcp, /agent-configuration/skills, /agent-configuration/tools, /api-reference/files, /your-customers/automations, /your-customers/files, /your-customers/memory, /your-customers/secrets, /your-customers/sessions. The docs sitemap contains exactly 14 URLs and includes none of these paths — the entire agent-configuration/ and your-customers/ sections are missing.
Consequence: The main conversion path from marketing to documentation is broken for the majority of advertised features (Automations, Files, Memory, Secrets, MCP, Skills, Tools, Sessions). Because the pages return HTML with HTTP 200, crawlers and agents may not even register them as broken — they just index a 404 screen.
The fix: Either ship the agent-configuration/ and your-customers/ sections, or repoint every homepage link to the pages that exist (/api-reference/tools, /api-reference/skills, /api-reference/vaults, /api-reference/sessions, etc.) and add redirects for the dead paths.
4. server_error is 500, 502, and 503 depending on which page you read (significant)
Problem: The canonical error table on /errors defines exactly one server error: "500 server_error — An unexpected error occurred on the server," with no 502 or 503 rows. But the vaults page's error table lists "502server_error — The credential operation could not be completed," and the rate-limits page says "If the allowance cannot be verified, the API returns a retryable 503 server_error." Three pages, three different HTTP statuses for the same error code, and the page that presents itself as the machine-readable contract ("Treat the HTTP status and error as the machine-readable contract") omits two of them.
Consequence: Retry logic keyed off the documented contract will misclassify 502/503 responses. An agent generating an error handler from /errors will treat 503 during a rate-limit check as undocumented behavior instead of the explicitly retryable case the rate-limits page describes.
The fix: Add 502 and 503 rows to the /errors table (with retryability guidance), or normalize the API to one status and fix the outlier pages.
Problem: The Skills page states: "Agent updates currently require the complete agent configuration, including name, modelName, systemPrompt, skills, and tools" and that updating "replaces the entire skills array." The Agents page documents PATCH /v1/agents/{agentId}, which exists precisely to avoid that: "Collection entries are upserted instead of replacing the whole collection. This lets you update one skill or tool without fetching and resending every other entry."
Consequence: A developer managing skills from the Skills page will build the fetch-merge-resend workflow the PATCH endpoint was designed to eliminate — and may clobber concurrent changes to tools or roles in the process. An agent reading only the Skills page will assert that partial updates are unsupported.
The fix: Update the Skills page's "Version behavior" section to reference PATCH /v1/agents/{agentId} and its skills-upsert semantics, keeping the full-replace POST as the removal path.
6. Flagship features are advertised with docs links but have no documentation (significant)
Location:https://www.herm.run/ feature grid vs the entire docs surface (sitemap: 14 pages)
Problem: The homepage advertises Automations ("Scheduled background runs"), Webhooks ("React to workspace changes"), Files ("A persistent workspace"), Self-learning ("Improves between sessions"), Subagents ("Parallel isolated workers"), and browser control ("Web, browser, shell built in") — each with a "Read docs" link. None has a docs page. The tools reference documents web and terminal toolsets but no browser toolset. Subagents surface only as two stub event entries (agent.thread_message_sent / agent.thread_message_received) with no explanation of how threads are created or configured. The Skills page concedes learned-skill APIs are "coming soon."
Consequence: Developers evaluating Herm for its headline differentiators (automations, webhooks, self-learning) find zero integration guidance — not even a "not yet available via API" note. They cannot distinguish "undocumented but shipped" from "not built."
The fix: For each advertised capability, either publish a docs page or label it as roadmap on the homepage. At minimum, document how agent.thread_message_* events arise and what triggers automations.
7. The quickstart's first prerequisite has no acquisition path (significant)
Problem: The quickstart requires "A Herm workspace and API key," and the introduction says "Create and manage API keys from your workspace settings" — but no page in the docs links to a signup flow, dashboard, or console URL. The only dashboard URL anywhere on the docs domain is https://prismai.sh/workspace — inside the mismatched llms.txt, for the wrong product. (The docs site chrome links to herm.run/app, which redirects to a login page, but the docs text never mentions it.)
Consequence: Step zero of onboarding is unreachable from the documentation. A developer must guess that herm.run/app is the console; an agent following the docs literally cannot complete the prerequisite at all — or worse, follows llms.txt to prismai.sh.
The fix: Add an explicit "create a workspace at herm.run/app, then generate a key under Settings → API keys" step to the quickstart and the introduction's Authentication section.
8. Three brands in one product's plumbing: PRISM_API_KEY, prismvideos.com, prismagents.com (minor)
Problem: Every Herm code sample authenticates with an env var named PRISM_API_KEY, never explained. The self-hosting page routes contact to rajit@prismvideos.com and cal.com/team/prismai; the homepage preview pane shows api.prismagents.com. Nothing states the Herm/Prism relationship or which domains are canonical.
Consequence: Developers wiring secrets managers and egress allowlists can't tell which names are load-bearing (api.herm.run) and which are legacy branding. Fresh integrations copy a Prism-branded variable for a Herm-branded API and second-guess whether they hit the right service.
The fix: Rename the documented env var to HERM_API_KEY (or add one line explaining "Herm is built by Prism; the key variable is PRISM_API_KEY"), and consolidate contact/domains.
9. Rate-limit scope is "subject" everywhere except where it's "user" (minor)
Problem: The platform's identity concept is subjectId ("your stable identifier for the person, team, organization, or workspace"). The send-message page counts the same allowance "per stable user per fixed 60-second window," while the rate-limits page scopes it to "the authenticated organization, agent, and stable subjectId."
Consequence: Low-grade but real ambiguity for multi-tenant integrators: if a subject is an organization, "per user" reads as a different (smaller) scope than "per subject." Agents reconciling the two pages can infer two distinct limit dimensions.
The fix: Standardize on "per subject" everywhere and reserve "user" for the human in the chat UI.
Problem: The API is documented entirely as prose plus Markdown tables. There is no OpenAPI/Swagger document anywhere in the sitemap or page set, and the reference pages are hand-written rather than generated from a spec — which is plausibly how the 500/502/503 and skills/agents contradictions crept in.
Consequence: Agents and SDK generators cannot programmatically discover endpoints, schemas, or error contracts; every consumer re-derives the schema from prose, and drift between pages goes undetected.
The fix: Publish an OpenAPI 3 spec at a stable URL, link it from the introduction, and generate the endpoint tables from it.
What they do well
llms-full.txt is complete and current — all 14 pages, verbatim-matching the rendered site, with source URLs per section.
Failure-mode documentation is unusually honest: non-idempotent retries, SSE reconnect race conditions, and rate-limit counting edge cases are spelled out precisely (and the live API's 401 body matches the docs exactly).
Structured, parseable reference tables for fields, limits, headers, and event payloads on every endpoint page.
Top 3 recommendations
Replace llms.txt with a real Herm index — it is currently an authoritative pointer to a different product, cited in the header of every page.
Reconcile the marketing site with the API: fix the ten dead "Read docs" links and rewrite the hero code sample against POST /v1/agents (and stop importing the unrelated herm npm package).
Publish an OpenAPI spec and generate the error/status tables from it to eliminate the 500/502/503 and skills-vs-agents contradictions.