Runcaptain Documentation Audit

The docs are well-structured for agents (llms.txt, page-level .md mirrors, MCP server, OpenAPI spec) but suffer from contradictions between the API reference and the feature guides, a Python SDK that's named two different things on two different pages, and a self-serve flow that quietly requires emailing support.

1. rerank default contradicts itself across pages (critical)

Location: /guides/get-started/multimodal-search and /api-reference/api-reference/query/collection-v-2

Problem: The multimodal search guide's request-fields table sets rerank default to true: "rerank | boolean | true | Enable reranking. Required when multimodal content is present". The Query API reference table sets the same parameter's default to false: "rerank | boolean | false | Improve relevance ordering". These are the two canonical reference tables for the same parameter on the same endpoint.

Consequence: A developer (or coding agent) reading the API reference assumes rerank is off and omits it from their request body, then gets surprise reranker billing or unexplained latency. A developer reading the multimodal guide assumes rerank is on by default and is shocked when their first text-only call returns un-reranked results. Agents can't infer which doc is canonical and will silently pick the wrong one.

The fix: Pick one default in code and update both tables to match. If multimodal requests truly require rerank=true, document that as a precondition rather than a default-flip per content type.

2. Python SDK has two different names across docs (critical)

Location: /guides/get-started/introduction and /api-reference/api-reference/indexing/parsing-scripts/validate-parsing-script-v-2

Problem: The introduction's Quick Reference says: "SDKs: Python (captain-sdk on PyPI), TypeScript (captain-sdk on npm)". The parsing-script validate page's Python example imports a different package and class: from runcaptain import Captain / client = Captain(key="YOUR_TOKEN_HERE"). Meanwhile the S3 TypeScript example uses import { CaptainClient } from "captain-sdk" — so the TS side at least matches the intro.

Consequence: A developer copy-pasting the validate-script example runs pip install captain-sdk (per the intro), then gets ModuleNotFoundError: No module named 'runcaptain'. Or they trust the example, run pip install runcaptain, and now have an SDK whose name doesn't match anything else in the docs. Agents using the docs as ground truth will emit broken installs either way.

The fix: Settle on one package name, audit every code sample for the right import, and add a one-line install command at the top of every code block (e.g. # pip install captain-sdk).

3. Self-serve flow requires emailing support for an External ID (critical)

Location: /guides/get-started/s-3-cross-account-iam

Problem: Step 1 of cross-account IAM setup is: "Email support@runcaptain.com with your Captain organization ID, or ask your account manager directly … We'll reply with a value of the form cap_org_<24-hex>, usually same-day. Save it; you'll paste it into your IAM trust policy in step 2." The External ID is "NOT customer-generated. Captain mints it server-side per organization."

Consequence: Cross-account IAM is the AWS-recommended way to grant access to a third party — and the docs gate it behind a same-day email round-trip. A developer evaluating Runcaptain on a Friday afternoon can't finish setup until Monday. Agents driving onboarding can't proceed without human intervention, which breaks every "from spec to running pipeline" use case Runcaptain pitches.

The fix: Expose External ID via the dashboard (Settings → Organization → IAM External ID) or via an API endpoint (GET /v2/organizations/me). Email remains a fallback, not the primary path.

4. Documented page /welcome returns 404 (significant)

Location: https://docs.runcaptain.com/welcome

Problem: Fetching /welcome returns "Page Not Found — This page does not exist." The llms-full.txt manifest and the navigation refer to "Welcome Aboard" as the docs landing page, but the canonical URL is /guides/get-started/introduction. There's no redirect from /welcome.

Consequence: The friendliest, most-guessable entry URL for a docs site dead-ends. Any external link, tweet, or onboarding email pointing at docs.runcaptain.com/welcome (a natural guess) breaks. Agents crawling guessable paths waste a request and may treat the 404 as "no docs at this site."

The fix: Redirect /welcome to /guides/get-started/introduction (301), or render the introduction at both paths.

5. OpenAPI spec lists Authorization header twice on validate-script endpoint (significant)

Location: /api-reference/api-reference/indexing/parsing-scripts/validate-parsing-script-v-2 (and its OpenAPI definition)

Problem: The parameters block for POST /v2/parsing-scripts/validate defines Authorization as an in: header parameter twice, with two different descriptions:

- name: Authorization
  in: header
  description: Bearer token authentication using API key
  required: true
...
- name: Authorization
  in: header
  description: Bearer token - your Captain API key.
  required: true

Consequence: OpenAPI 3.1 requires parameter name + in to be unique. Spec validators (Spectral, Redocly lint, openapi-generator) error on this, and code generators may produce broken clients that send the Authorization header twice or skip the operation. Anyone running the published spec through CI gets a failure.

The fix: Remove the duplicate. Define Authorization once in a shared securitySchemes block and reference it via security: so it can't drift again.

6. Response field for AI answers has two names (significant)

Location: /api-reference/api-reference/query/collection-v-2 vs /guides/get-started/multimodal-search

Problem: The Query API reference lists the AI answer under "summary / response: AI response (when inference=true)" — two field names for the same payload, with no explanation of which one clients should read. The multimodal search agent quick-reference lists per-result fields (score, content, document_id, filename, uri, modality, media_segment_start_sec, media_segment_end_sec, rerank_score, metadata) but never mentions either summary or response, leaving the answer-extraction path undefined for multimodal+inference queries.

Consequence: A developer writes result.summary and an agent's parser breaks the day the server returns result.response (or vice versa). For multimodal+inference calls — Runcaptain's headline feature — there's no documented field to read the LLM answer from.

The fix: Commit to one canonical field name in the response schema. If both names are returned for backwards compatibility, say so explicitly and mark one deprecated with a removal date.

7. Changelog advertises endpoints and protocols that aren't documented anywhere else (significant)

Location: /changelog/changelog (October and November 2025 entries)

Problem: The October 2025 changelog entry advertises "OpenAI SDK-compatible chat completions endpoint (/v1/chat/completions)" and "Vercel AI SDK support added." The November 2025 entry advertises "Tool calling support — OpenAI-compatible function calling with client-side execution." The introduction's endpoint inventory lists only /v2/... endpoints. There is no API reference page, no auth guide, no rate-limit doc, no model list, and no tool-schema reference for any of these features. They're mentioned in the changelog and nowhere else in the scraped corpus.

Consequence: Teams choosing Runcaptain for OpenAI-SDK drop-in compatibility or function calling — both explicitly advertised — have no documented schema, no auth example, no deviation list from the OpenAI spec, no error reference. They have to reverse-engineer behavior from OpenAI's docs and hope the server matches. For an agent-targeted product, undocumented agent surface area is the worst kind of gap.

The fix: Publish reference pages for /v1/chat/completions (request schema, supported models, deviations from OpenAI's spec, streaming behavior) and the tool-calling protocol (tool-schema shape, execution model, client-side vs server-side semantics). If either was rolled back, remove the claim from the changelog.

8. Scientific-medical page contradicts itself on client timeout (significant)

Location: /guides/odyssey/scientific/search-medical-papers

Problem: The Agent Quick Reference says "p50 ~8s, p95 ~12s. Set client timeouts ≥ 90s." The Limits section repeats "Set client timeouts ≥ 90s." But the body of the "Asking a Question" section says "First invocation of the day can hit a cold Lambda (~60s) + agent loop (~30-60s). set client timeouts to 180s and guard against non-JSON responses (504/502) so errors surface cleanly." 90s vs 180s on the same page, for the same endpoint, with the 180s figure explicitly justified by the documented worst-case cold-start.

Consequence: An agent that reads only the Quick Reference (which is what the docs train agents to do) sets a 90s timeout and gets a hard failure on the very cold-start path the docs warn about elsewhere. A human reader scrolls to the next section and gets a different answer with no indication which is canonical.

The fix: Pick one. If 180s is the safe upper bound, put 180s in the Quick Reference and Limits and explain that 90s works once Lambda is warm. If 90s is the right floor for the steady state, move the 180s cold-start guidance into a separate "First request" callout.

9. search_results is both a request parameter and a response field with different types (significant)

Location: /api-reference/api-reference/query/collection-v-2

Problem: On the same Query API reference page, search_results appears in the request parameters table as search_results | boolean | false | Include context chunks with inference response, and then again in the Response Structure block as search_results: Array of SearchResult objects. Same name, two different types (boolean in, array out), no note connecting the two.

Consequence: A developer or agent typing response.search_results against a TypeScript type generated from the request schema gets a boolean and a type error. A developer setting search_results: true to opt-in to chunks has no documented guarantee about what comes back when the value is omitted vs. false. Code generators may collapse the two into a single conflicting field.

The fix: Rename one of them — e.g. include_search_results: boolean for the request flag — or explicitly document that the request boolean toggles the response array, with a worked example of the response shape under both values.

10. Metadata filter operators table omits $and, $or, $not (significant)

Location: /guides/get-started/metadata-filtering

Problem: The "Filter Operators" table lists $eq, $ne, $gt, $gte, $lt, $lte, $in, $nin — but not $and, $or, or $not. Later sections demonstrate $or ("Use $or to match chunks that satisfy any of the conditions") and "Implicit AND," so the operators clearly exist, but they're not in the reference table.

Consequence: An agent generating filter expressions from the reference table won't emit $or because it doesn't appear in the canonical operator list. A developer scanning the table for "how do I OR" assumes it isn't supported and either drops the query or runs two queries client-side. Anyone Ctrl-F'ing the page for $or finds it only in a usage example, not in a contract.

The fix: Add $and, $or, $not rows to the operator table with example syntax, nesting rules, and depth limits.

11. TypeScript S3 IAM-role example is missing the auth block (significant)

Location: /api-reference/api-reference/indexing/s-3/index-s-3-bucket-v-2

Problem: The "Index S3 Bucket (IAM role)" TypeScript sample shows:

await client.indexing.indexS3BucketV2("my_documents", {
    bucketName: "my-documents-bucket",
    processingType: "advanced",
    bucketRegion: "us-east-1",
    skipExisting: true,
});

There is no field identifying which role ARN to assume, no auth / role_arn / external_id block, no reference to the cross-account setup from the companion guide. The label promises "IAM role" but the body looks identical to a public-bucket call.

Consequence: A developer copies this snippet expecting IAM-role auth, runs it against a private bucket, and gets a generic 403. They can't tell whether the SDK silently picked up ambient AWS credentials, whether a field is missing from the request, or whether IAM-role mode isn't actually wired up. Agents can't synthesize a working call from this example.

The fix: Show the full request body for IAM-role mode, including roleArn and externalId, and cross-link to /guides/get-started/s-3-cross-account-iam from the example.

12. Live Search advertises 10 sources but documents only one (significant)

Location: /guides/integrations/overview vs /guides/integrations/live-search-guides/*

Problem: The integrations overview lists Live Search sources as "Snowflake (SQL queries), Slack, Linear, Jira, Confluence, Asana, Gmail, Google Calendar, HubSpot, and Oracle NetSuite." Only one of those — Oracle NetSuite — has a dedicated guide in the scraped corpus. The other nine have no setup, auth, scope, or rate-limit documentation.

Consequence: A buyer comparing connectors thinks Runcaptain has nine more integrations than it has documentation for. Developers wiring up Slack or Jira get the connector name from marketing and then have nowhere to read about OAuth scopes, refresh behavior, or query semantics.

The fix: Ship a per-connector page for each Live Search source with the same structure as the NetSuite guide (authentication method, setup steps, supported query types, known limits). Until then, mark undocumented connectors as "Studio-only" or "private beta."

13. Cloud-storage setup has screenshots only for AWS (minor)

Location: /guides/get-started/connect-cloud-storage

Problem: The cloud-storage page hard-codes image URLs to Vercel Blob (https://y2i6auvwuaxlu05f.public.blob.vercel-storage.com/aws_image.png, aws_image2.png, google_image.png). AWS gets two screenshots; GCS gets one labeled "Service Accounts" with no follow-up; Azure and R2 — both listed as first-class integrations — get no visuals at all.

Consequence: AWS users get a guided console walkthrough; everyone else gets prose-only setup against unfamiliar console UIs. Azure's "generate a SAS token from the Azure Portal" instruction is hand-waved past the click-path that actually matters.

The fix: Add parity screenshots for GCS service-account key creation, Azure SAS generation, and R2 token creation. Move images to a stable CDN under the runcaptain.com domain so a Vercel Blob URL rotation doesn't silently 404 every screenshot.

14. Changelog claims monthly cadence and is on track to miss May 2026 (minor)

Location: /changelog/changelog

Problem: The changelog header states "Monthly updates to the Captain API — new features, improvements, and fixes." As of 2026-05-23, the most recent entry is April 2026. The month is not over yet — but with one week left and no entry, May is on track to be the first missing month under the stated cadence, and there's no signal to the reader about whether something is coming or whether the cadence promise has lapsed.

Consequence: Developers can't tell whether (a) nothing has shipped in May, (b) something shipped and the changelog is behind, or (c) the changelog is no longer maintained. For a product that bills its docs as agent-friendly, the absence of a "nothing this month" note is a silent staleness signal.

The fix: Either post a May entry before month-end (even a short "no user-facing changes" note) or rephrase the header to drop the monthly promise and describe the cadence honestly.

What they do well

• Strong agent-parsing surface: llms.txt, llms-full.txt, per-page .md mirrors, an MCP server at /_mcp/server, and "append .md to any page" — this is one of the more agent-aware docs sites in the category.
• Per-page Agent Quick Reference blocks front-load the key constraints (endpoint, required params, hard limits) so an agent doesn't have to read 800 words of prose to call the API.
• Honest limits sections: The parsers page lists sandbox restrictions (no require, no fetch, no setTimeout, 64 MB heap, 10s timeout) and the scientific dataset page documents the 8-call tool-call cap and p95 latency — both rare and useful.

Top 3 recommendations

1. Fix the contradictions first. rerank default (true vs false), Python SDK name (captain-sdk vs runcaptain), AI-response field name (summary vs response), and the 90s-vs-180s client timeout are all single-line changes that affect every agent-generated call.
2. Document /v1/chat/completions, the Vercel AI SDK surface, and tool calling — or retract the claims. OpenAI-SDK compatibility and function calling are serious selling points and currently exist only as changelog bullets.
3. Eliminate the email-for-External-ID handoff. Surface it in the dashboard or an API endpoint so AWS cross-account IAM setup is fully self-serve.