Befailproof Documentation Audit

The docs are well-structured at the Mintlify level (llms.txt and llms-full.txt both exist, sitemap is clean, 14 locales are wired up), but core surface-area facts contradict each other across nearly every page that mentions them — policy counts, CLI support, scope semantics, exit codes, and file-extension support all disagree. For a "reliability tool for autonomous agents," the docs themselves are not reliably consistent.

2. "Local" scope refers to two different files in two different places (critical)

Location: /configuration vs /cli/install-policies

Problem: The --scope local flag and the "Local" configuration scope are documented as different things:

• /configuration defines Local as .failproofai/policies-config.local.json — "Individual overrides (gitignored)"
• /cli/install-policies defines --scope local as "Claude only, uses .claude/settings.local.json"

These are two different files, in two different directories, with two different purposes (failproofai's own config vs. Claude's settings file).

Consequence: A developer running failproofai policies --install --scope local reasonably expects to write to their personal .failproofai/policies-config.local.json override. Instead the command writes to Claude Code's .claude/settings.local.json, which has very different semantics (it controls Claude itself, not failproofai's policy params). The override they thought they made silently goes to a different file and may have no effect.

The fix: Rename one of these. Either call the CLI flag --scope claude-local (since it's Claude-specific), or align the file path so --scope local writes to .failproofai/policies-config.local.json as the configuration page describes. Document the exact file path each --scope value writes to in a small table.

3. Exit code 2 means two different things in the hook protocol (critical)

Location: /cli/hook vs /testing

Problem: /cli/hook defines the exit-code contract as:

• Exit 0 (allow): Action proceeds
• Exit 1 (deny): Action blocked
• Exit 2 (instruct): Guidance injected into Claude's context

But /testing describes the same exit codes differently: "Stop directives exit with code 2, and allows produce empty stdout."

So exit code 2 is simultaneously "instruct" (per the hook spec) and "Stop" (per testing). Both are documented as authoritative.

Consequence: Anyone writing a custom hook integration, a test harness, or a wrapper script has to guess which interpretation Claude Code actually receives. If exit 2 is overloaded internally, an instruct decision could be misread as a Stop directive, silently terminating a session that should have continued with added context.

The fix: Clarify on /cli/hook whether exit code 2 carries different meanings based on event type (PreToolUse vs Stop), or pick one meaning. Add an explicit table mapping (event type, decision) → (exit code, stdout shape) so an integrator never has to infer.

4. The list of supported agent CLIs varies by page — five different claims (critical)

Location: / (landing), /examples, /for-agents, /getting-started, /dashboard, /cli/install-policies

Problem: The set of supported agent CLIs disagrees across at least five distinct claims:

• Landing hero: "Claude Code, OpenAI Codex, and the Agents SDK" (3 names; "Agents SDK" is not a CLI)
• /examples: "Claude Code and the Agents SDK" (2 names)
• /for-agents: "coding agents like Claude Code, Cursor, and Windsurf" (3 names — and Windsurf appears nowhere else)
• /getting-started, /dashboard, /llms-full.txt: "Claude Code, OpenAI Codex, GitHub Copilot, Cursor Agent, OpenCode, Pi, and Gemini CLI" (7 names)
• /cli/install-policies: --cli flag accepts only claude, codex, or copilot (3 names)

Consequence: A Cursor, Windsurf, OpenCode, Pi, or Gemini CLI user reads the dashboard or for-agents page, installs the tool, then can't actually wire it up — the install command rejects their CLI value. Windsurf in particular is named on /for-agents but is not an --cli option anywhere. A landing-page visitor sees only 3 CLIs and never learns Cursor or Gemini exist as targets. An agent fetching docs to answer "does this work with OpenCode?" will give a confidently wrong answer in either direction depending on which page it hit.

The fix: Maintain the canonical list in one place (e.g., the install command's --cli enum). Auto-generate the marketing, examples, for-agents, and getting-started lists from that source. If only 3 CLIs are truly installable today, fix every other claim; if 7 are supported, fix --cli to accept all 7 and add Windsurf (or remove it from /for-agents).

5. Custom-policy file-extension support contradicts itself (significant)

Location: /custom-policies and /llms-full.txt vs /getting-started

Problem: /custom-policies and /llms-full.txt say custom policies are auto-loaded from files matching *policies.{js,mjs,ts}. /getting-started says: "add policy files (.mjs format). These load automatically across the team without additional configuration steps once committed to version control."

Consequence: A team following the "Getting Started" path writes .ts policy files, commits them, and they fail to load — or vice versa. The mismatch quietly breaks team-shared policy enforcement, which is exactly the use case /getting-started is selling.

The fix: Update /getting-started to state all three supported extensions (.js, .mjs, .ts) and link to /custom-policies for loading-order rules. If .ts requires a build step or specific runtime, document it.

6. Custom policies advertise four event types; the hook handler ships 22 (significant)

Location: /custom-policies vs /cli/hook

Problem: /custom-policies lists eventType as one of "PreToolUse" | "PostToolUse" | "Notification" | "Stop". /cli/hook enumerates 22 supported events: PreToolUse, PostToolUse, PostToolUseFailure, PermissionRequest, PermissionDenied, SessionStart, SessionEnd, Stop, StopFailure, UserPromptSubmit, Notification, Elicitation, ElicitationResult, SubagentStart, SubagentStop, TaskCreated, TaskCompleted, TeammateIdle, InstructionsLoaded, ConfigChange, CwdChanged, FileChanged, WorktreeCreate, WorktreeRemove, PreCompact, PostCompact.

Consequence: A developer who wants to write a custom policy reacting to SessionStart or SubagentStop reads /custom-policies, doesn't see the event, and assumes it isn't possible — when in fact the underlying handler accepts it. Conversely, if those events are not deliverable to custom policies, the hook page advertises an API surface that custom-policy authors can't reach.

The fix: State explicitly on /custom-policies which subset of /cli/hook's events are deliverable to user code, and either widen the type union in the API reference or add a note explaining why custom policies see a narrower set than built-ins.

7. PORT env var is unprefixed; collides with every other Node service on a developer's machine (significant)

Location: /cli/environment-variables

Problem: Every other env var on the page is namespaced — FAILPROOFAI_DISABLE_PAGES, FAILPROOFAI_ALLOWED_DEV_ORIGINS, FAILPROOFAI_LOG_LEVEL, FAILPROOFAI_HOOK_LOG_FILE, FAILPROOFAI_TELEMETRY_DISABLED, FAILPROOFAI_LLM_*. Then: PORT: Server listening port (defaults to 8020).

Consequence: PORT is the most-collided variable name in the Node ecosystem (Next.js, Express, Nest, Heroku, Vercel local dev all use it). A developer running failproofai in a shell that already exports PORT=3000 for their Next app will silently bind failproofai to 3000, possibly conflicting with their dev server. The --port flag exists but isn't called out as the safer override.

The fix: Rename to FAILPROOFAI_PORT (keep PORT as a deprecated fallback for one release), or at minimum add a callout warning that PORT is read globally and recommend --port for project shells.

8. The llm config field exists but no page documents which policies need it (significant)

Location: /configuration and /cli/environment-variables

Problem: /configuration says: "llm (optional object) — Configures LLM client for policies requiring AI calls, including model selection and API credentials." /cli/environment-variables documents FAILPROOFAI_LLM_BASE_URL, FAILPROOFAI_LLM_API_KEY, FAILPROOFAI_LLM_MODEL (default gpt-4o-mini). No page anywhere — including /built-in-policies — identifies which policies trigger LLM calls, when they fire, what they cost, or what happens if the key is missing.

Consequence: A user turning on "all" policies (failproofai policies --install all) may unknowingly start making outbound OpenAI calls from a tool that elsewhere markets itself as "operates entirely locally" and "no external data transmission" (/dashboard, /introduction). That's a privacy and cost surprise tied directly to the product's headline claim of locality.

The fix: On /built-in-policies, tag the policies that require LLM calls with a clear icon/label. Add a section to /configuration listing those policy names and the call shape (provider, model, payload). Update the "stays on your machine" claims to qualify them: "except policies that explicitly opt into LLM evaluation."

9. "Operates entirely locally" trust claim is materially incorrect for LLM-backed policies (critical)

Location: /introduction, /dashboard, /llms-full.txt

Problem: Multiple pages emphasize local-only operation as a headline trust claim:

• /introduction: "all processing stays on your machine with no external data transmission"
• /dashboard: "operates entirely on your filesystem with no remote data transmission"
• /llms-full.txt: "nothing uploads to remote servers"

But /cli/environment-variables documents FAILPROOFAI_LLM_BASE_URL: API endpoint (defaults to https://api.openai.com/v1). The default is a remote OpenAI endpoint, and /configuration confirms an llm config field exists for "policies requiring AI calls."

Consequence: This is not a documentation nit — it's a defect in the product's primary trust claim. A privacy-conscious developer or a regulated team picks failproofai specifically for the "no external transmission" guarantee, then enables an LLM-backed policy and unknowingly streams agent tool inputs (which can include source code, paths, and sanitization-target output) to OpenAI. The default endpoint is OpenAI's hosted API, not a local model. For HIPAA, SOC2, or any data-residency context, this is the difference between a tool that's safe to deploy and one that triggers a security incident.

The fix: Soften and qualify every "stays local" claim. Add: "LLM-backed policies are opt-in and will send context to the configured FAILPROOFAI_LLM_BASE_URL (default: OpenAI). To stay fully local, leave the llm config unset or point it at a self-hosted endpoint." Also list which built-in policies are LLM-backed.

10. Activity-log scope disagrees: only non-allow vs all events (significant)

Location: /architecture vs /dashboard

Problem: /architecture says: "Each hook event generates one JSONL record to ~/.failproofai/hook-activity.jsonl, capturing timestamp, decision, tool name, and policy name. Only non-allow decisions are logged to minimize file size." /dashboard describes the Activity tab as: "Paginated history of all hook events with filtering by decision, event type, CLI, policy name, or session ID."

Consequence: A user investigating "why did my agent do X?" expects to find the full timeline on the Activity tab. If the underlying JSONL only captures non-allow decisions, the tab can never show the allow events the page promises to filter on. Conversely, if all events are logged, the architecture page's "minimize file size" claim is misleading and the log will grow much faster than expected.

The fix: Decide which is true. If only non-allow events are persisted, update /dashboard to remove the implication that allow events appear in Activity. If everything is persisted, drop the "non-allow only" line from /architecture and document the actual retention/rotation policy.

11. --scope options are asymmetric between install and uninstall (significant)

Location: /cli/install-policies vs /cli/remove-policies

Problem: The install command documents three scope values — user, project, local. The uninstall command documents four — user, project, local, and all ("removes from every configuration level simultaneously"). There is no corresponding --scope all on install.

Consequence: A user who runs failproofai policies --uninstall --scope all to clean up cannot symmetrically reinstall in one command — they have to invoke install three times, once per scope. Worse, if a user assumes the commands are symmetric and tries --install --scope all, the behavior is undocumented: either the flag is silently accepted (and does something), silently ignored, or errors. None of those is the expected "round-trip" behavior. The asymmetry also obscures whether install/uninstall manipulate the same set of files.

The fix: Either add --scope all to install (with the same semantics — write entries to every scope) or remove it from uninstall and require explicit scope iteration. Document the exact files each scope value reads from and writes to, in a single table shared by both pages.

12. Branding split between befailproof.ai and failproofai is never reconciled (minor)

Location: Sitewide

Problem: The docs domain is docs.befailproof.ai. The product name varies: "Failproof AI" (/introduction, /examples), "failproofai" (CLI, npm package). The npm package is failproofai — no be- prefix. The /package-aliases page lists typo-squat protections for failproof, failprof, failprof-ai, faliproof, faliproof-ai, but befailproof (the actual domain) is not among them. The npm aliases for failproof-ai and fail-proof-ai are listed as "pending npm approval."

Consequence: Developers Googling the product reach befailproof.ai but npm install befailproof fails (no such package). Agents asked "what's the package name?" can easily produce befailproof, failproof-ai, or failproofai depending on which page they parsed first. The typosquat list itself omits the most likely typo (the brand's own domain prefix).

The fix: Either rename the package to befailproofai to match the domain, or surface a clear "Package: failproofai, domain: befailproof.ai" callout on /introduction and /getting-started. Add befailproof, befailproofai, and be-failproof-ai to the typosquat registration list.

13. No machine-readable policy schema or registry (minor)

Location: /built-in-policies, /llms-full.txt

Problem: The 38–39 built-in policies are documented in prose with ad-hoc parameter mentions (allowPatterns, allowPaths, protectedBranches, thresholdKb, etc.). There is no JSON Schema, no OpenAPI-style spec, and no programmatically-fetchable policy registry. The customPolicies.add() signature is shown inline as TypeScript-ish prose.

Consequence: An agent building a config for a new user has to scrape prose to learn which policies exist and what parameters they accept. The /cli/list-policies page notes that the CLI flags unrecognized keys, meaning a typoed param fails late — there's no way to validate a policies-config.json against a schema before shipping it. Teams can't generate type-safe wrappers without reading every page.

The fix: Publish a JSON Schema for policies-config.json and a machine-readable index of built-in policies (name, description, parameter schema). Link them from /llms.txt so coding agents discover them automatically. Ship .d.ts for the customPolicies API.

14. FAILPROOFAI_HOOK_LOG_FILE "default location" is never defined (minor)

Location: /cli/environment-variables and /custom-policies

Problem: /cli/environment-variables documents FAILPROOFAI_HOOK_LOG_FILE: Custom logging destination or true for default location — but no page states what that default path actually is. /custom-policies mentions in passing that custom-policy errors can be viewed with tail -f ~/.failproofai/hook.log, but never connects that path to the env var or confirms it's the default.

Consequence: A developer debugging a hook integration sets FAILPROOFAI_HOOK_LOG_FILE=true, expecting logs to land somewhere predictable, then has to guess the path. They may grep ~/.failproofai/, search their cwd, or check /tmp. For an opt-in logging feature, the missing default destination defeats the point.

The fix: On /cli/environment-variables, replace "default location" with the actual path (presumably ~/.failproofai/hook.log based on /custom-policies). Cross-link the two pages so anyone hunting for hook errors lands in the same place.

What they do well

• Both llms.txt and llms-full.txt exist and are advertised in the navigation — uncommon and welcome.
• Eighteen pages localized into fourteen languages with a consistent sitemap.
• Strong proactive typosquatting story on /package-aliases (even if the brand's own prefix is missing from the list).

Top 3 recommendations

1. Reconcile the contradictions table. Pick the canonical value for: policy count, supported CLI list (across landing, examples, for-agents, getting-started, dashboard, install), custom-policy file extensions, activity-log scope, --scope options on install vs uninstall, and --scope local target file. Add CI that fails when the documented count or list drifts from the source of truth.
2. Fix the locality claim. This is a trust-claim defect, not a doc nit. Either qualify "stays on your machine" everywhere it appears, or make LLM-backed policies obvious and opt-in with a clear list of which built-ins call out and a default that isn't a remote OpenAI endpoint.
3. Publish a machine-readable policy registry. JSON Schema for policies-config.json + a structured index of built-in policies and their parameters, linked from llms.txt. This is the single highest-leverage change for agent-driven adoption of a product whose audience is, definitionally, AI coding agents.