kapa.ai

Kapa.ai Documentation Audit

A RAG/AI-search vendor whose own docs are missing the things they sell to others: a machine-readable index, complete endpoint reference pages, and consistent terminology across surfaces. The product story and integration guides are strong; the developer reference layer underneath them is thin and contradictory.

1. No llms.txt or llms-full.txt published (critical)

Location: https://docs.kapa.ai/llms.txt and https://docs.kapa.ai/llms-full.txt

Problem: Both URLs return HTTP 404. Kapa sells a product whose explicit pitch includes "Give Context to AI Coding Agents" via MCP, and whose docs index talks about being machine-parseable, yet the company has not published the de-facto agent-discovery files for its own documentation.

Consequence: Coding agents (Claude Code, Cursor, Codex, Copilot) trying to programmatically discover Kapa's docs have to fall back to crawling HTML or web search, which is exactly the inefficiency Kapa's own product page argues against. It also undercuts every sales narrative about "documentation built for AI."

The fix: Publish llms.txt at the docs root with a curated map of high-value pages (API overview, quickstart, auth, every reference endpoint, SDK quickstarts), plus an llms-full.txt containing the full reference content. Mirror the structure Anthropic, Mintlify, and others already use.

2. API reference pages are placeholder shells (critical)

Location: /api/reference/chat, /api/reference/chat-streamed, /api/reference/query-v-1-projects-chat, /api/reference/query-v-1-projects-chat-chat-stream, /api/reference/retrieval, /api/reference/feedback

Problem: The scraped content for these pages contains only narrative descriptions ("Chat with your kapa.ai instance. It creates a new Thread object…"). Request bodies, response schemas, headers, error codes, and code samples are absent. The /api/reference/query-v-1-projects-chat page explicitly reads as "a documentation index rather than the full endpoint reference page." /api/reference/retrieval is a single paragraph pointing at itself.

Consequence: A developer (or agent) trying to call POST /query/v1/projects/{project_id}/chat/ cannot learn from the docs what fields the body takes, what comes back, what 4xx codes mean, or how to handle errors. They have to reverse-engineer from the SDK source or contact support — for the company's flagship endpoint.

The fix: Publish full OpenAPI-rendered reference pages with: request schema (with field types, required/optional, defaults), response schema, all status codes including the documented 429, example request in curl + Python + TypeScript, and example success/error response bodies. Link the underlying OpenAPI/JSON Schema spec from the page so agents can fetch it directly.

3. Inventory counts disagree across pages (significant)

Location: docs.kapa.ai/ (homepage), /overview/getting-started, /data-sources/overview, /data-sources/supported-data-sources, /integrations/website-widget/configuration/behavior, /integrations/zendesk-app

Problem: Connector count is given three ways: homepage says "20+ other sources," /overview/getting-started says "50+ data sources," /data-sources/overview says "22 different connectors," and /data-sources/supported-data-sources lists 22. Language support has its own mismatch: the widget behavior reference lists exactly 13 supported language codes (en, zh, cs, nl, fr, de, it, ja, ko, pt, ru, es — and the legacy widget page also references "13 UI languages"), while /integrations/zendesk-app advertises "20+ languages."

Consequence: A prospect comparing Kapa to a competitor will see a 2.5× discrepancy in connector claims depending on which page they read first, and a comparable gap in language support. An agent answering "how many sources/languages does Kapa support?" will hallucinate confidently from whichever page it grabbed.

The fix: Pick the actual counts (22 connectors per the supported-sources list; 13 widget languages per the behavior reference) and use them everywhere. If "50+" or "20+ languages" includes uncatalogued surfaces, reframe with a footnote pointing to the enumeration so the numbers reconcile.

4. Auth header conventions differ across products, with one quickstart silent (significant)

Location: /dev/agent/guides/authentication vs /integrations/mcp/overview vs /api/quickstart

Problem: The Agent SDK uses X-API-Key: your-api-key (explicit in /dev/agent/guides/authentication). MCP API-key access requires "bearer token in authorization headers" (per /integrations/mcp/overview). The HTTP API quickstart references "your API key" and instructs developers to call https://api.kapa.ai/query/v1/projects/{project_id}/chat/ but the scraped content does not state which header name to put the key in. Two confirmed conventions and a third quickstart that doesn't say.

Consequence: A developer who sets Authorization: Bearer <key> against the agent session endpoint will get 401s — that endpoint wants X-API-Key. A developer reading the HTTP API quickstart has to guess between the two. Agents copy-pasting an example from one page into a different endpoint will silently fail.

The fix: State the header name explicitly on every endpoint reference page and on the quickstart. Add a single "Authentication conventions" page that names which header each surface (Agent, MCP, HTTP API) expects, and link to it from every quickstart. Better: pick one convention and migrate the other.

5. Streamed chat uses an obscure Unicode delimiter (significant)

Location: /api/reference/query-v-1-projects-chat-chat-stream

Problem: The streamed chat response is split by \u241e (the Unicode "RECORD SEPARATOR" character) and clients are expected to split on it — but this is mentioned in passing and not paired with a worked client-side example. It is not server-sent events, not newline-delimited JSON, not the convention any standard SSE library handles.

Consequence: Developers reaching for EventSource or any standard streaming client library will get a single opaque chunk and have to write custom byte-level splitting after they figure out the delimiter is U+241E. Agents writing code from these docs will not invent this convention; they will produce broken streaming clients.

The fix: Either move to standard SSE/NDJSON, or — if the custom framing is intentional — give the delimiter its own labeled section with a copy-pasteable Python and JavaScript chunking example, and a note in every SDK that wraps this transport.

6. Rate limits don't reconcile across surfaces (significant)

Location: /api/overview vs /integrations/mcp/overview

Problem: The HTTP API page sets chat at 20 req/min and retrieval at 60 req/min. The MCP overview sets public/internal users at 300/day + 60/min and API-key access at 60/min per team. Both surfaces front the same underlying inference pipeline, but a developer reading both pages cannot tell whether MCP's 60/min is a separate budget from the API's 20/min, whether they share quota, or whether the MCP path is genuinely 3× faster than the chat API for the same questions.

Consequence: Capacity planning is impossible. Customers shipping a hybrid deployment (widget + MCP + API agent) cannot predict when they'll hit a 429, and the docs offer no answer beyond "contact support@kapa.ai."

The fix: Publish a single rate-limits table that lists every integration path (chat, retrieval, MCP public, MCP API-key, agent sessions) with per-minute, per-day, and per-team limits, and explicitly state whether quotas are shared or independent.

7. /account/authentication is silent on API keys (significant)

Location: /account/authentication

Problem: The page covers OTP, OAuth, SAML SSO, SCIM, and session expiry — but, per the scraped content, "does not contain information about API keys—only user authentication methods are covered." Meanwhile, API key creation is mentioned offhand in /api/quickstart ("requires team owner permissions"), in /dev/agent/guides/authentication ("Settings → API Keys"), and in /account/role-based-permissions ("Creating API keys" as a permission).

Consequence: A developer landing on the authentication page leaves thinking SSO + OTP is the whole auth story, then has to hunt across three other pages to learn how keys are minted, who can mint them, how they're rotated, what happens when an owner leaves, and whether keys expire. Critical questions like rotation policy and revocation flow have no documented answer.

The fix: Add an "API keys" section to /account/authentication covering creation permissions, scope, rotation, revocation, expiry behavior, and what gets invalidated when a team member is removed. Link to the quickstart for first-key creation, but make the authentication page the canonical home for key lifecycle.

8. Slack's 7-day refresh window is buried on the wrong page (significant)

Location: /data-sources/refreshes vs /data-sources/slack

Problem: /data-sources/refreshes reveals that Slack "scans back seven days, every six hours" — meaning any edit or new thread older than a week will not be picked up. The /data-sources/slack source page itself only mentions a default "Maximum thread age: 6 months" filter and doesn't surface the 7-day detection window.

Consequence: Customers configure Slack assuming changes are picked up the way Notion or Confluence is, then discover months later that backfilled history doesn't update incrementally and old-thread edits are silently dropped. This is a behavior gotcha that needs to live on the configuration page where someone will actually read it before deploying.

The fix: Add a "Refresh behavior" callout to /data-sources/slack quoting the 7-day / 6-hour window and explaining what types of changes will and won't propagate. Cross-link to /data-sources/refreshes for the broader schedule.

9. Custom-tool example demonstrates new Function() code eval (critical)

Location: /dev/agent/core/custom-tools

Problem: The headline "calculate" example uses new Function(\return (${expression})`)()to evaluate an LLM-supplied expression. This is arbitrary code execution against whatever the model decides to send, with no sandbox, no input validation, and no warning in the surrounding prose that this pattern is unsafe to copy verbatim. It sits as the canonical second-tool example next toget_current_time` in the page every developer learning the SDK will read.

Consequence: Because the agent is described as running "client-side using the user's credentials," a prompt-injected expression can pivot from "calculator" to "fetch and exfiltrate cookies / hit an authenticated backend / read localStorage" trivially. Agents and developers copy-paste this verbatim into production. This is a templated security incident waiting to ship in customer products.

The fix: Replace the example with something genuinely safe (e.g., a fixed-set unit converter, or mathjs.evaluate with a restricted scope and no symbol access). If eval-style intent is unavoidable, add a prominent "do not use in production" callout above the code block and explicitly walk through the prompt-injection failure mode.

10. Parallel "human" and "machine-slug" reference URLs in the sitemap, including deprecated endpoints (significant)

Location: sitemap.xml — /api/reference/chat and /api/reference/query-v-1-projects-chat both exist; same for chat-stream, retrieval, feedback. Also /api/reference/query-v-1-projects-chat-chat-custom and /api/reference/query-v-1-projects-search. And /integrations/website-widget/configuration/legacy.

Problem: The sitemap lists 328 URLs, and the API reference appears twice — once at readable slugs (/api/reference/chat) and once at generated slugs (/api/reference/query-v-1-projects-chat). On top of that, /api/overview warns that "Custom Chat and Search endpoints are marked for removal and should not be used in new implementations," yet both /api/reference/query-v-1-projects-chat-chat-custom and /api/reference/query-v-1-projects-search still appear in the sitemap with no deprecation banner on the pages themselves. The widget config has the same shape: a /configuration/legacy page that admits it's deprecated but is still indexable and copy-pasteable.

Consequence: Search ranks duplicate pages against each other; agents indexing the sitemap waste budget on near-duplicates and surface deprecated endpoints as if they were current; developers landing on a legacy widget config page may ship deprecated data-button-* attributes. The two versions of each endpoint can — and from the audit, do — drift in completeness.

The fix: Pick one canonical slug per endpoint, 301 the other, and emit <link rel="canonical"> from any remaining duplicates. Remove deprecated endpoints from the sitemap or add an unmissable deprecation banner at the top of each. Same for /integrations/website-widget/configuration/legacy.

11. Tutorial pins to a specific model name with no substitution guidance (minor)

Location: /integrations/mcp/guides/tutorial-build-an-in-product-agent

Problem: The tutorial's reference architecture hard-codes "a reasoning model (GPT-5.1)" as the orchestrator with no surrounding note that the choice is substitutable. A developer reading this on a team that uses Anthropic, Cohere, or an open model has no guidance for swapping it out.

Consequence: Readers reproduce the example with a model their org doesn't actually procure, or copy the string into a request that 404s at the provider for any team without GPT-5.1 access. Agents writing code from the doc will pin to that exact model rather than templating it.

The fix: Either generalize the example to "your preferred reasoning model" with a one-line list of tested options, or keep the specific model and add a sidebar explicitly noting it's substitutable and listing the minimum capabilities (tool calling, JSON output, context length) any replacement needs.

12. Placeholder values not marked as placeholders (minor)

Location: /integrations/website-widget/installation/general, /nextjs, /docusaurus, /mintlify, /dev/sdk/components/KapaProvider

Problem: Install snippets across the widget docs use bare placeholder strings — XXX-XXX-XXX, PROJECT_NAME, #HEX_COLOR_CODE, https://LINK_TO_LOGO.com/logo.png, YOUR_WEBSITE_ID — without a consistent convention (some pages use YOUR_…, others use XXX-XXX-XXX, others use ALL_CAPS_NAMES). Nothing in the snippet signals to an agent "this is a value to replace" vs. "this is a real string."

Consequence: An agent extracting an install snippet may ship data-website-id="XXX-XXX-XXX" to production. The widget will fail silently or with an unhelpful error. Even humans get tripped up when one page says YOUR_WEBSITE_ID and another says XXX-XXX-XXX for the same field.

The fix: Standardize on one placeholder convention (e.g., <YOUR_WEBSITE_ID> with angle brackets) and use it everywhere. Add a short legend at the top of the install section explaining the convention.

13. PII protection is off by default and not flagged on security pages (minor)

Location: /security/pii-protection

Problem: The page states plainly that "PII protection is not enabled by default" — true for both user-message and knowledge-source protections. The /security/certifications page does not link to or mention this default-off behavior.

Consequence: A security reviewer reading only the SOC 2 page assumes PII filtering is on; the actual default means customer PII (email, phone, names, IPs) can be persisted in conversation logs unless a customer opts in. This is the kind of default that should be discoverable from any security entry point.

The fix: Add a prominent callout to /security/certifications and the security overview noting PII protection is opt-in by default, with a one-click link to the enablement instructions.

14. Billing page contains no pricing information (minor)

Location: /account/billing

Problem: The entire billing page is a "contact finance@kapa.ai" stub. Per the scraped content, "the documentation provided does not contain specific pricing, subscription plan details, or subscription tier information." No tier names, no per-seat or per-question costs, no overage policy, no link to a pricing page.

Consequence: A prospect or existing customer trying to budget for a deployment — or an internal procurement reviewer — gets no answer from the docs and must enter a sales loop for basic questions. Agents asked "what does Kapa cost?" return nothing useful.

The fix: Either link prominently to a public pricing page from /account/billing, or document the tier shape (e.g., usage units, included questions, overage rates) even if exact dollar amounts require sales contact. At minimum, document the question/usage units customers will see on invoices.

15. Slack and Discord bots use inconsistent enablement command syntax (minor)

Location: /integrations/slack-bot vs /integrations/discord-bot

Problem: Slack enables a channel with /kapa-enable-channel (slash prefix, hyphen). Discord enables a channel with !kapa_enable_channel (bang prefix, underscore). Same conceptual command, three differences in spelling. The Slack workspace and Discord server variants follow the same divergence (/kapa-enable-workspace vs !kapa_enable_server).

Consequence: Teams operating both bots have to maintain two mental models; admins documenting internal runbooks duplicate near-identical commands. Agents asked to enable Kapa in a channel may emit the wrong syntax for the platform.

The fix: Live with platform-native conventions where required, but add a small "Command reference" comparison table on each bot page noting the per-platform spelling explicitly so the difference is documented, not folklore.

16. Typo in customer-facing FAQ (minor)

Location: /overview/faq

Problem: "The internal instance has access to the same public sources plus internal knwowledge bases…" — "knwowledge" appears in a top-level FAQ answer.

Consequence: Trivial in isolation, but it sits in the most-read page for prospects evaluating the product. Agents will reproduce the misspelling when summarizing.

The fix: Fix the typo. Add a spell-check pass to CI for docs.

What they do well

• The website-widget configuration reference (behavior parameters, JS API functions, event payloads) is genuinely thorough — every attribute is named with a default and type.
• The SDK split (@kapaai/agent-core + @kapaai/agent-react, @kapaai/react-sdk) is clean, and the session-token authentication flow in /dev/agent/guides/authentication is the most clearly explained auth pattern in the entire docs set.
• The /data-sources/refreshes page is unusually honest about per-connector limitations (Slack's 7-day window, the 30%-change human-review gate on web crawls).

Top 3 recommendations

1. Fix the API reference and the agent SDK security example. Every endpoint page should have a full schema, error codes, and runnable examples in at least curl + TypeScript + Python; publish the underlying OpenAPI spec. Replace the new Function() tool example with something that won't ship a copy-paste RCE.
2. Publish llms.txt and llms-full.txt. A vendor whose entire pitch is "make your docs work for AI agents" cannot ship without them.
3. Reconcile the contradictions: connector count (20+/22/50+), language support (13/20+), auth header (X-API-Key vs Bearer vs unstated), and the rate-limit numbers between API and MCP. Each one breaks an agent silently.