Sentrial

Sentrial Documentation Audit

One-line summary: A YC-backed AI observability product with a surprisingly broad SDK surface, but the docs ship two competing Python APIs side-by-side, a published OpenAPI that is literally Mintlify's sample Plant Store, and a homepage feature set (evaluations, A/B testing, Slack alerts) that has no documentation at all.

1. The published OpenAPI spec is Mintlify's sample Plant Store, not Sentrial's API (critical)

Location: https://www.sentrial.com/docs/api-reference/openapi.json (linked from llms.txt under "OpenAPI Specs")

Problem: The OpenAPI document that llms.txt points agents to contains:

"title": "OpenAPI Plant Store",
"description": "A sample API that uses a plant store as an example...",
"servers": [{ "url": "http://sandbox.mintlify.com" }],
"paths": { "/plants": ..., "/plants/{id}": ..., "/plant/webhook": ... }

This is the unmodified Mintlify starter template — wrong title, wrong server URL, wrong paths, wrong schemas. The actual Sentrial REST surface (POST /api/sdk/sessions, PATCH /api/sdk/sessions/:id, POST /api/sdk/events on https://api.sentrial.com) is documented only in prose on the API reference pages.

Consequence: Any AI coding agent that follows the llms.txt index — which is the documented agent entry point — will think Sentrial sells a plant store API hosted at sandbox.mintlify.com. Tools that auto-generate SDKs/types/clients from the OpenAPI will produce a Plant model and /plants endpoints. There is no machine-readable description of the real API at all.

The fix: Replace /docs/api-reference/openapi.json with a real generated spec covering POST /api/sdk/sessions, PATCH /api/sdk/sessions/:id, and POST /api/sdk/events, including the documented camelCase request/response schemas, the bearer-auth scheme, and every error code. Until that exists, remove the dead link from llms.txt so agents don't pick it up.

2. Two competing Python APIs documented as equally official, with no guidance on which to use (critical)

Location: /docs/quickstart, /docs/installation, /docs/concepts/sessions, /docs/sdk/python

Problem: Every primary surface presents two completely different code paths for the same task:

• "Simple API" — sentrial.configure(...) + sentrial.begin(...) → interaction.finish(...)
• "Full Control" — from sentrial import SentrialClient → client.create_session(...) → client.complete_session(...)

Within a single Quick Start page, Step 2 uses sentrial.begin/finish, and Step 3 silently switches to SentrialClient(api_key=...).create_session(...) for the same scenario without explaining why. The Sessions concept page labels them "Simple API (Recommended)" and "Full Control API" but never says when the simple one is insufficient, whether they can be mixed, or whether begin() and create_session() produce equivalent sessions on the backend.

Consequence: Developers (and agents) write hybrid code — sentrial.configure() then SentrialClient() then interaction = sentrial.begin(...) then client.track_tool_call(session_id=???). The session_id from begin() is never documented as being accessible, so users can't reach the client.track_* methods from an Interaction. The two APIs look interchangeable in examples but aren't.

The fix: Pick one default and demote the other to an "Advanced" section, or document the bridge explicitly: how to get a session_id from an Interaction, whether interaction.track_tool_call() and client.track_tool_call(session_id=...) are equivalent, and which one each integration page expects.

3. Homepage advertises three product surfaces that don't exist in the docs (critical)

Location: Homepage (sentrial.com) vs /docs/*

Problem: The marketing page prominently features:

• Evaluations / Flags & Assertions ("User Frustration", "Tool Schema Hallucination", "Goal Abandonment" — "Set up evaluations →")
• Prompt A/B Experiments ("Variant A / Variant B WINNER … 4,200 sessions … statistical rigor … Run an experiment →")
• Slack alerts on error spikes ("Configure alerts →")

None of these appear anywhere in the docs. The Issues & Diagnosis page uses entirely different vocabulary ("Task Failures, Tool Errors, Hallucinations, User Frustration, Confusion Loops, Slow Responses, High Cost") and no flags, assertions, experiments, variants, or alerts are mentioned. The llms.txt index has no pages on these features.

Consequence: A developer who books a demo expecting prompt A/B testing or Slack alerting can't self-serve any of it from the docs — there is no API, no SDK call, no dashboard walkthrough. Agents asked to "set up a prompt experiment in Sentrial" will hallucinate or fall back to generic advice.

The fix: Either ship docs for evaluations, experiments, and alerts (with concept pages and API/SDK surfaces), or remove those sections from the marketing page until they exist. Today the homepage promises a different product than the docs document.

4. Eventtype enum lists four values but only documents two (significant)

Location: /docs/api/events

Problem: The eventType ParamField says:

"tool_call", "llm_decision", "state_change", or "error".

The "Event Types" section below documents only tool_call and llm_decision. There is no schema, no example, and no description of what fields are required for state_change or error. The SDK reference has a client.track_error(session_id, error_message, error_type, tool_name, stack_trace, metadata) method but the REST equivalent — presumably eventType: "error" — is never spelled out (what fields does the REST API expect for an error event? does it accept stackTrace? errorType?).

Consequence: Anyone using the REST API directly (not the SDK) cannot record errors or state changes without guessing. Agents will hallucinate the payload shape.

The fix: Either remove state_change and error from the enum until they're documented, or add schema + example payloads for each. Map the SDK track_error fields explicitly to REST fields.

5. Sessions API documents create + update but no list, get, or delete (significant)

Location: /docs/api/sessions

Problem: The page documents POST /api/sdk/sessions and PATCH /api/sdk/sessions/:id and nothing else. There is no GET /api/sdk/sessions/:id, no list endpoint, no delete, no filtering, no pagination, no idempotency-key header, and no rate-limit information. The error table lists 404 Not Found — Session ID not found, implying a GET-by-ID endpoint exists, but it's not documented.

Consequence: Anyone building a backend integration that needs to reconcile session state, export sessions, or handle retries safely cannot do so against the documented API. Retries against POST /api/sdk/sessions will create duplicate sessions because there's no idempotency contract.

The fix: Document the full session resource (GET single, LIST with filters/pagination, DELETE if it exists), add an Idempotency-Key header, and publish a rate-limit policy. If list/delete genuinely don't exist, say so explicitly so developers stop looking.

6. Authentication page promises permission scoping but never documents it (significant)

Location: /docs/api/auth

Problem: The 403 error example is:

{ "error": "Forbidden", "message": "API key does not have access to this resource" }

This implies API keys have scopes or per-resource permissions, but the page only explains the key format (sentrial_live_xxx) and the bearer header. There is no documentation of:

• What scopes exist
• How to create a key with limited scope
• The difference (if any) between sentrial_live_ and other prefixes (the format note implies live_ means "production" — what are the alternatives? test_?)
• Rate limits
• Key expiry/rotation behavior beyond a "rotate regularly" platitude

Consequence: A developer who sees a 403 in production has no way to diagnose it from the docs. Anyone trying to follow "use separate keys per environment" can't, because non-live environments aren't described.

The fix: Add a Scopes section listing every permission keys can carry, document the full set of key prefixes (live_, test_, etc.), publish rate limits, and explain how scope mismatches surface as 403 vs how missing/invalid keys surface as 401.

7. Custom-integration tool tracking example records errors as successful tool calls (significant)

Location: /docs/integrations/custom, "Tracking Tool Calls" section

Problem: The recommended tracked_tool decorator pattern handles exceptions by calling track_tool_call again with tool_output={"error": str(e)}:

except Exception as e:
    client.track_tool_call(
        session_id=session_id,
        tool_name=func.__name__,
        tool_input=tool_input,
        tool_output={"error": str(e)}
    )
    raise

The SDK documents a separate dedicated error path (client.track_error(session_id, error_message, error_type, tool_name, stack_trace, metadata)) and the Issues & Diagnosis page lists "Tool Errors" as a distinct High-severity detected category. This decorator never calls track_error — it records every failure as an ordinary tool_call event with an error key buried in the output blob. Whether the platform's Tool-Error detector triggers on tool_output.error is undocumented; based on the surface APIs, the canonical failure path is track_error, so this pattern likely under-reports.

Consequence: Developers following the only documented "custom agent" pattern probably lose Tool Error signals — and at minimum end up with two inconsistent failure-recording mechanisms across the codebase (the Python SDK has track_error; the docs' decorator example doesn't use it). Tool-error grouping behavior is left to inference.

The fix: Rewrite the example to call client.track_error(...) (or whatever the canonical failure path is) in the except branch, and explicitly document what triggers Tool Error detection — is it track_error, tool_output.error, complete_session(success=False), or all three?

8. The Python SDK reference documents a large surface that the navigation and quickstart hide (significant)

Location: /docs/sdk/python vs every other page

Problem: The Python SDK reference documents wrap_openai, wrap_anthropic, wrap_google, wrap_llm, @tool and @session decorators, sync/async context managers, event batching (BatcherConfig, flush(), shutdown()), PII redaction with PiiConfig (custom regex patterns, built-in patterns), and cost-calculation helpers for OpenAI/Anthropic/Google.

None of these appear in:

• The Quick Start (only sentrial.begin/finish and SentrialClient are shown)
• The Installation page (no mention of batching, PII, or auto-wrappers)
• The llms.txt index (no separate page for batching, PII, or LLM auto-wrappers)
• The integration pages (LangChain, CrewAI, AutoGen do not mention PII redaction or batching despite both being SDK-wide concerns)

Consequence: A team evaluating Sentrial against an observability competitor will conclude PII redaction and event batching don't exist, because they're invisible unless you scroll the full Python reference. An agent asked "how do I enable PII redaction in Sentrial?" via SearchSentrial may not surface it depending on which page ranks.

The fix: Promote PII redaction, batching, and LLM auto-wrappers (wrap_openai, etc.) to first-class concept pages. Reference them from the Quick Start ("Production checklist: enable batching and PII before launch") and the integration pages.

9. Quickstart's event parameter is mis-quoted and collides with another core term (significant)

Location: /docs/quickstart, Step 2

Problem: The explanation text reads literally:

The "event parameter is how Sentrial groups sessions. Use consistent names like "support_agent" or "research_assistant"."

The opening quote is in the wrong place — it's around event instead of around the example values — leaving the sentence with mismatched quotes. More importantly, the same word "event" means two different things across the docs:

• On sentrial.begin(), event= is a session-grouping label (Quickstart and the Sessions concept page both use event="support_request" / event="chat_message" without defining what is being grouped).
• On POST /api/sdk/events, an "event" is a per-action record inside a session (a tool call or LLM decision).

Consequence: The single most foundational concept in the SDK collides with the term "event" used in the REST API. Developers and agents will conflate the two and pass tool-call values where a session-group label is expected (or vice versa).

The fix: Rename the event parameter on begin() to something distinct (e.g., session_type or flow), or at minimum fix the broken quotation, add a glossary entry, and stop calling tool-call/llm-decision records "events" if event already means something else at the session level.

10. Verification snippet uses the API that doesn't exist in the SDK examples that follow it (significant)

Location: /docs/installation, "Verify Installation" section

Problem: The verify snippet uses sentrial.begin(...).finish(...). Immediately above it, the Configuration section shows three configuration options, the third of which is from sentrial import SentrialClient; client = SentrialClient(...). A developer who picked Option 3 (the only one that names a class) is then handed a verification snippet that doesn't use that class at all. There's no client.create_session() verification example.

Consequence: First-time users who copy the Option 3 configuration and then the Verify snippet end up with code that imports SentrialClient but never uses it, while calling module-level sentrial.begin() that depends on the previously-set configure(). The result works only by accident, and it teaches developers a confused mental model.

The fix: Provide two verification snippets — one for sentrial.configure() + begin/finish, one for SentrialClient.create_session/complete_session — and tie each to the matching Configuration option above it.

11. Quickstart's "Complete Example" references undefined symbols (significant)

Location: /docs/quickstart, "Complete Example" at the bottom of the page

Problem: The final showcase example calls executor.invoke(...) but the variables prompt, search_kb, create_ticket, and send_email are referenced as tools and prompts without ever being defined in the snippet. The example is the last thing a reader sees before leaving the Quickstart — it is also exactly what an AI coding agent will copy into a new file.

Consequence: A developer or agent copy-pasting the "Complete Example" gets a NameError/ReferenceError on the first line they hit. The example doesn't demonstrate Sentrial — it demonstrates a non-runnable LangChain skeleton. Agents will hallucinate definitions for the missing tools rather than failing fast.

The fix: Make the Complete Example self-contained: define the ChatPromptTemplate, define each tool with a real (or trivially stubbed) @tool body, and ensure the snippet runs end-to-end against a free-tier OpenAI key.

12. Cost-helper model coverage is inconsistent between SDKs (significant)

Location: /docs/sdk/python vs /docs/sdk/typescript

Problem: The Python helper example uses model="gemini-2.0-flash". The TypeScript helper example uses model: 'gemini-2.5-flash'. The Python helper uses snake_case (input_tokens, output_tokens); the TS helper uses camelCase inside an options object (inputTokens, outputTokens). The TS page also claims support for "GPT-5, GPT-4.1, O-series, Claude Opus/Sonnet 4, Claude 3.5, Gemini 2.5/2.0, and more" but the Python page never lists supported models at all. The LangChain integration page meanwhile publishes a third, different model coverage table.

Consequence: Anyone using the SDKs cross-language (a TS frontend, a Python worker) can't tell whether calculate_*_cost will return the same number for the same model. Agents will pass gemini-2.0-flash in Python and gemini-2.5-flash in TS and get silently divergent cost numbers — or one will silently return None/0 for an unsupported model.

The fix: Publish a single canonical "Supported models and pricing" page referenced from both SDK pages and the LangChain page. Make Python and TS examples use the same model. Document what calculate_*_cost returns when the model is unknown.

13. Vercel AI SDK example uses a v3/v4-only response method while claiming v5/v6 support (significant)

Location: /docs/integrations/vercel-ai, "Next.js Chat Route" example

Problem: The page opens with "Supported Versions: Vercel AI SDK v3, v4, v5, and v6." The Next.js example ends with:

return result.toDataStreamResponse();

toDataStreamResponse() is the v3/v4 streaming-response method; AI SDK v5 renamed the streaming response to toUIMessageStreamResponse(). The page never branches by version or mentions the rename.

The same example calls wrapAISDK(ai) at module scope and then again inside the request handler as wrapAISDK(ai, { convoId: conversationId }) — without explaining whether this double-wrap is safe, idempotent, or required to thread per-request context.

Consequence: Users on AI SDK v5/v6 — explicitly listed as supported — will copy the example and get a runtime error from the missing method. Users who don't realize they're wrapping the SDK twice may end up with duplicate event emissions.

The fix: Branch the example by AI SDK major version (v3/v4: toDataStreamResponse(), v5/v6: toUIMessageStreamResponse()), or drop the version claim. Document explicitly whether wrapAISDK is idempotent and how to thread per-request context without double-wrapping.

14. TypeScript "Complete Example" uses error.message on an unknown catch binding (significant)

Location: /docs/sdk/typescript, "Complete Example" section

Problem: The example reads:

} catch (error) {
  await sentrial.completeSession({
    sessionId,
    success: false,
    failureReason: error.message
  });
  throw error;
}

Under TypeScript strict mode (the default since TS 4.4), the binding in catch (error) is typed as unknown, so error.message is a compile error ('error' is of type 'unknown'). The same pattern is repeated in the Custom Integration page's TypeScript example.

Consequence: Any TS user running tsc --strict (the modern default and a Mastra/Next.js norm) will see two-page-spanning compile errors copying the showcase example. Agents auto-generating TS code from this snippet will produce non-compiling files.

The fix: Use a type guard or narrow explicitly: failureReason: error instanceof Error ? error.message : String(error). Apply the same pattern everywhere the docs catch in TypeScript.

15. MCP install uses incompatible transports for the same URL with no warning (significant)

Location: /docs/use-with-ai

Problem: Both install paths point at https://www.sentrial.com/docs/mcp, but:

• Claude Code: claude mcp add --transport http Sentrial https://www.sentrial.com/docs/mcp
• Cursor: "Set the type to SSE and the URL to https://www.sentrial.com/docs/mcp" — the .cursor/mcp.json example uses "type": "sse".

These are different transport protocols (HTTP streaming vs SSE). The page never explains whether the endpoint speaks both, whether one is preferred, or what happens if a tool only supports one of the two.

Consequence: Cursor users on the latest version (which prefers streamable HTTP and is deprecating SSE) will get a confusing setup. Anyone integrating from a less common MCP client has no way to know which transport the endpoint actually supports.

The fix: State explicitly which transports the /docs/mcp endpoint serves (HTTP streamable, SSE, both?) and give one canonical config per client, with a note that the transport must match what the server actually serves.

16. AsyncSentrialClient and sentrial.claude_code are used in integration examples but absent from the SDK reference (significant)

Location: /docs/integrations/claude-code-python vs /docs/sdk/python

Problem: The Claude Code (Python) page opens its Quick Start with:

from sentrial import AsyncSentrialClient
from sentrial.claude_code import wrap_claude_agent

Neither AsyncSentrialClient nor the sentrial.claude_code submodule (including wrap_claude_agent / wrap_claude_client) appears anywhere on /docs/sdk/python. The SDK reference documents only SentrialClient (sync) and an async with async_client.begin(...) pattern without ever naming AsyncSentrialClient or explaining how it differs from the sync client.

Consequence: Developers troubleshooting an async setup (await sentrial.close(), lifecycle of AsyncSentrialClient in long-running services, whether wrap_claude_agent accepts sync or async clients) have no reference page. Agents asked "what's the API of AsyncSentrialClient?" via SearchSentrial will find an example but no signature.

The fix: Add AsyncSentrialClient to the Python SDK reference with full method signatures and lifecycle notes. Document sentrial.claude_code as a first-class submodule with wrap_claude_agent and wrap_claude_client signatures, default values, and version requirements.

17. Mastra ships a separate npm package the Installation page never lists (significant)

Location: /docs/installation vs /docs/integrations/mastra

Problem: The Installation page documents exactly one TypeScript install: npm install @sentrial/sdk. The Mastra integration page requires three packages: npm install @sentrial/mastra @sentrial/sdk @mastra/core, and the integration's main entry point (instrumentAgent) lives in @sentrial/mastra, not @sentrial/sdk. The Installation page never mentions that there is a second Sentrial npm package at all, and the TypeScript SDK reference only documents @sentrial/sdk.

Consequence: A developer who follows the Installation page first (the documented onboarding flow) and then tries to add Mastra hits Cannot find module '@sentrial/mastra' or doesn't discover the integration exists. Agents indexing the Installation page will not surface @sentrial/mastra even when prompted "how do I install Sentrial for Mastra?"

The fix: Add a "Framework packages" section to the Installation page listing every npm package the docs reference (@sentrial/sdk, @sentrial/mastra, and any future ones) with one-line descriptions and the integration page link.

18. AutoGen page documents two helper APIs that exist nowhere else (minor)

Location: /docs/integrations/autogen

Problem: The 0.4+ example calls handler.run_team(team, task="..."). The legacy pyautogen example calls handler.register_agents([assistant, user_proxy]). Neither method appears in the Python SDK reference, and there's no link to a SentrialAutogenHandler API page. Compared with the LangChain page's detailed handler.total_cost/handler.get_usage_summary() documentation, the AutoGen surface is opaque.

Consequence: Anyone wanting to customize AutoGen tracking (verbose mode, accessing token counts, switching between team types) has nothing to read. The handler looks like a black box.

The fix: Add a "SentrialAutogenHandler reference" section mirroring the LangChain handler reference: constructor params, methods (run_team, register_agents, finish), and accessible stats.

19. CrewAI page is conspicuously thin compared with its siblings (minor)

Location: /docs/integrations/crewai

Problem: The CrewAI page has no error-handling pattern, no model-coverage table, no PII/batching guidance, no "what you'll see in the dashboard" section, and uses ellipses in lieu of real examples (crew = Crew(agents=[...], tasks=[...])). Compare with the LangChain page (handler stats, supported models table, LangGraph migration notes) or the Vercel AI SDK page (Next.js route, dynamic IDs, tool tracking).

Consequence: CrewAI users get a strictly worse onboarding than LangChain or Vercel AI SDK users, and have to guess how multi-agent costs/tokens roll up into a single Sentrial session.

The fix: Bring the CrewAI page up to parity: complete examples, handler API table, error-handling pattern, and a "How multi-agent steps map to Sentrial events" section.

20. create_agent_with_sentrial() documented but absent from the SDK reference (minor)

Location: /docs/integrations/langchain vs /docs/sdk/python

Problem: The LangChain page introduces from sentrial.langchain import create_agent_with_sentrial with a "Quick Setup" section and a version-detection footnote ("auto-detects your LangChain version. On LangChain 1.0+ it uses LangGraph's create_react_agent. On older versions it uses AgentExecutor."). The Python SDK reference lists SentrialCallbackHandler indirectly but never documents create_agent_with_sentrial's full signature, return type, supported kwargs, or behavior when version auto-detection fails.

Consequence: Developers who hit a create_react_agent edge case (the page itself notes "never fires on_agent_finish") have no reference page to consult.

The fix: Add create_agent_with_sentrial to the Python SDK reference with full signature, kwargs, fallback behavior, and a "when not to use this" note pointing at the explicit SentrialCallbackHandler path.

21. SENTRIAL_API_URL is documented once and then forgotten (minor)

Location: /docs/installation vs /docs/api/auth, /docs/quickstart

Problem: The Installation page is the only place that mentions SENTRIAL_API_URL=https://api.sentrial.com ("Optional: custom API URL (for self-hosted)"). Neither the Authentication page nor the Quickstart nor any integration page mentions it again. The cURL examples on /docs/api/auth hardcode https://api.sentrial.com, and there's no documentation of whether the SDK reads SENTRIAL_API_URL automatically on every call path, whether it accepts a path prefix, or what happens when it disagrees with an explicit api_url= argument.

Consequence: Self-hosted or staging users have no reliable signal that the SDK respects SENTRIAL_API_URL. Anyone running the docs' cURL examples against a non-production environment has to rewrite every URL by hand.

The fix: Add a "Self-hosted / custom endpoints" section (or sidebar callout) to the Authentication page and Quickstart, explicitly stating SENTRIAL_API_URL is read by the Python and TypeScript SDKs, and parameterize cURL examples with ${SENTRIAL_API_URL:-https://api.sentrial.com}.

What they do well

• An llms.txt exists and is comprehensive — agent-discoverable index that names every doc page (modulo the broken OpenAPI link). Most products this size don't ship one.
• A dedicated /docs/use-with-ai page with a hosted MCP server — the SearchSentrial tool, no auth required, is a thoughtful agent-first touch.
• Strong per-framework integration coverage — Claude Code (Python and TS), Mastra, Vercel AI SDK, LangChain 1.0/LangGraph, AutoGen 0.4 and legacy pyautogen. Few competitors document this breadth.

Top 3 recommendations

1. Replace the Plant Store OpenAPI immediately. Today, the canonical machine-readable description of Sentrial's API tells agents you sell plants.
2. Pick one Python API as the documented default. sentrial.begin/finish vs SentrialClient.create_session/complete_session should not look interchangeable on every page. Document how to bridge them or demote one.
3. Either ship docs for Evaluations, Experiments, and Slack alerts, or take them off the homepage. The product the marketing page sells and the product the docs document are not the same product.