One-line state: Laminar's conceptual pages are strong and OpenTelemetry-native, but the surface a developer (or coding agent) copies from is riddled with contradictions — the CLI answers to two different names with two command grammars, the Anthropic client is imported three inconsistent ways (and the troubleshooting page's "fix" imports a package that doesn't exist), the SQL editor's queryable-table list collides with the examples other pages tell you to run, and the README's "list of all instruments" link 404s. The reference tables are genuinely good; the copy-paste path is where it breaks.
1. The CLI answers to two different names with two different command grammars (critical)
Problem: The authoritative CLI reference (/docs/platform/cli) states the CLI "is a standalone npm package (lmnr-cli)" and refers to dataset management under the lmnr-cli binary. But /docs/datasets/cli documents the same functionality as lmnr datasets / npx lmnr datasets — no -cli, and a pluraldatasets subcommand. So the same product is lmnr-cli on one page and lmnr on another, with dataset vs datasets grammar to match. (Per the page-level evidence notes, the evaluations pages additionally invoke lmnr eval/npx lmnr eval and the CLI reference's command list — setup, login, logout, project, sql, dataset, trace, debug — contains no eval; I flag those as drawn from the evidence annotations rather than re-quoted page text, but the lmnr-cli vs lmnr datasets contradiction is verbatim on both pages.)
Consequence: A developer who runs the command from whichever page they landed on can hit "command not found" or a subcommand spelled differently than documented. An AI coding agent — which /docs/getting-started positions as the primary setup driver — has no canonical answer to "what is this CLI called" and will guess.
The fix: Pick one binary name and one command grammar and regenerate every snippet from it. If lmnr-cli and lmnr are genuinely different binaries, say so explicitly on both pages and cross-link them. Confirm where eval invocation lives and document it in the CLI reference.
2. The Anthropic client is imported three inconsistent ways, and the troubleshooting page's "fix" imports a package that doesn't exist (critical)
Code verification
Runtime snippet checks
Run the public examples from this audit in an isolated sandbox and attach the pass, fixed, skipped, and failed counts to the report.
Problem: The same Anthropic SDK is imported three different ways across the integration docs, plus an inconsistent instrumentModules casing parallel on the OpenAI page:
Anthropic page: import Anthropic from '@anthropic-ai/sdk' with instrumentModules: { anthropic: Anthropic } (lowercase key)
Vercel AI SDK page: import * as anthropic from '@anthropic-ai/sdk' with new anthropic.Anthropic()
Troubleshooting page: import anthropic from 'anthropic' — anthropic is not the official package; the real one is @anthropic-ai/sdk
(Parallel inconsistency: the OpenAI page uses instrumentModules: { OpenAI: OpenAI } — capitalized key — and new OpenAI.OpenAI(), against the Anthropic page's lowercase anthropic key. The instrumentModules key must match what the instrumentation expects, so the casing divergence is not cosmetic.)
So the page whose entire purpose is fixing broken auto-instrumentation ships an import that won't resolve.
Consequence:import anthropic from 'anthropic' throws "module not found" — the troubleshooting fix doesn't even import. The instrumentModules casing inconsistency means a developer who copies the key from the wrong page can get silently un-instrumented calls — the exact failure the troubleshooting page is supposed to resolve. Agents copying any of these get a different, non-interoperable shape each time.
The fix: Standardize on one import form and one instrumentModules key casing, and apply it everywhere. Fix the troubleshooting page's from 'anthropic' to from '@anthropic-ai/sdk' immediately — it is currently copy-paste-broken.
3. The SQL editor's queryable-table list collides with the tables other pages tell you to query (critical)
Problem: The SQL editor reference is authoritative about which tables are queryable: its list includes evaluation_datapoints but no evaluations table and no evaluation_results table. The same page explicitly says under "Avoid joins": "ClickHouse isn't optimized for joins. Instead, run two queries and combine results in your application." Yet:
/docs/custom-dashboards/overview tells readers the schema reference lists tables "(traces, spans, signal_events, evaluation_results, etc.)" — but evaluation_results is not in the SQL editor's table list (it's evaluation_datapoints). This is verbatim on the dashboards page.
/docs/evaluations/comparing-runs says, for anything beyond CSV, "query the underlying table with SQL" — and per /docs/evaluations/concepts, the evaluations table lives in Postgres, not the ClickHouse SQL editor, so it is not reachable from the editor at all. (The comparing-runs SQL body was truncated in the scraped excerpt; the specific claim that its example JOINs against evaluations is inferred from the page's framing and the Postgres/ClickHouse split, not re-quoted query text.)
Consequence: The dashboards page sends developers hunting for an evaluation_results table that does not exist — schema autocomplete will never surface it. The comparing-runs export path points at evaluations, which is Postgres-only and not queryable in the ClickHouse SQL editor. The official examples don't run against the official tool.
The fix: Replace evaluation_results with evaluation_datapoints on the dashboards page. On comparing-runs, either make the example runnable in the SQL editor (use evaluation_datapoints) or state clearly that evaluations is Postgres-only and not reachable from the editor. Add a one-line "ClickHouse vs Postgres: what's queryable where" callout, since the storage split is the root cause.
4. The README's "list of all instruments" link 404s, and natural-guess integration slugs are dead (significant)
Problem: The README's "See list of all instruments here" links to https://laminar.sh/docs/installation, which returns HTTP 404 (confirmed). Two other natural-guess paths also 404: /docs/tracing/integrations/ai-sdk (the correct slug is vercel-ai-sdk) and /docs/tracing/structure/introduction (the correct path is /docs/tracing/structure).
Consequence: The README is the entry point for most developers; its primary "list of all integrations" link is dead. An agent or human who guesses ai-sdk (a very natural guess given the AI SDK is the flagship integration) hits a dead page rather than a redirect.
The fix: Point the README link to the live integrations overview (/docs/tracing/integrations/overview). Add redirects for the natural-guess slugs (ai-sdk → vercel-ai-sdk, tracing/structure/introduction → tracing/structure).
5. No machine-readable surfaces (llms.txt, OpenAPI) for a product that positions agents as the primary setup driver (significant)
Problem:/llms.txt, /llms-full.txt, and /docs/llms.txt all 404 — there is no machine-readable doc index. There is also no OpenAPI/Swagger spec referenced anywhere, despite documented HTTP endpoints: POST /v1/sql/query, /v1/traces, and /v1/mcp (all using Authorization: Bearer <PROJECT_API_KEY>).
Consequence:/docs/getting-started makes a coding agent the primary instrumentation driver, yet that agent has no llms.txt/llms-full.txt to ingest and no OpenAPI spec to discover the /v1/* API — it must scrape prose. This isn't production-breaking, but for an explicitly agent-first product it undercuts the headline workflow.
The fix: Add llms.txt and llms-full.txt (Mintlify can generate these — they're simply not being served at the probed paths). Publish an OpenAPI spec for the /v1/* HTTP endpoints so agents can discover them programmatically.
6. The README quickstart ships an empty-string API key that silently produces no auth (significant)
Location: GitHub README — Python quickstart
Problem: The Python quickstart shows Laminar.initialize(project_api_key="") — an empty string rather than a clearly-marked placeholder like "<YOUR_PROJECT_API_KEY>". An empty string is still a valid string, so it won't trigger an obvious "missing argument" error; it will simply fail to authenticate.
Consequence: A developer copies the snippet, runs it, sees no crash, and gets zero traces — with no clear signal that the cause is an unset key. This is the silent-failure mode that's hardest to debug, and an agent extracting the snippet has no way to know the empty string is a fill-in-the-blank rather than a working default.
The fix: Use an unmistakable placeholder (project_api_key="<YOUR_PROJECT_API_KEY>") or read from env (project_api_key=os.environ["LMNR_PROJECT_API_KEY"]), and have initialize raise a clear error on an empty/missing key.
7. The Anthropic page's model ID is not a valid Anthropic model string (significant)
Location: /docs/tracing/integrations/anthropic (and, by comparison, /docs/tracing/integrations/openai, /docs/tracing/integrations/litellm, /docs/evaluations/*, README)
Problem: The Anthropic integration page uses claude-3-7-sonnet, which is not a valid Anthropic model ID — real IDs carry a date suffix or -latest. Separately, the example OpenAI model string differs from page to page (gpt-4.1-mini, gpt-4o-mini/gpt-4o, gpt-4.1-nano, gpt-5-mini) and the debugger page reportedly uses claude-opus-4-5. (Provenance note: only claude-3-7-sonnet is confirmed as the model string shown on its page; the cross-page OpenAI/Claude variants above are drawn from the per-page evidence annotations rather than re-quoted snippet bodies, so treat the "every page differs" pattern as indicative, not exhaustively quoted.)
Consequence:claude-3-7-sonnet copied verbatim will be rejected by the Anthropic API with no hint that the model string is the problem. Where the OpenAI example strings do diverge, an agent has no canonical model to standardize on and may propagate a non-current one.
The fix: Fix claude-3-7-sonnet to a valid, current Anthropic model ID, and pick one canonical example model per provider so snippets are consistent. If the variety is intentional, add a one-line note that the model string is interchangeable and link a "supported models" reference.
8. Three different domains appear for the same product surfaces (significant)
Problem: The docs.json navbar "Dashboard" link points to https://www.lmnr.ai/projects, while the README and getting-started send users to https://laminar.sh/projects. The API base URL is a third host, https://api.lmnr.ai. So the product spans laminar.sh, www.lmnr.ai, and api.lmnr.ai with no page explaining the relationship.
Consequence: A developer can't tell whether laminar.sh/projects and www.lmnr.ai/projects are the same dashboard or two environments, or whether bookmarking one is safe. This complicates allowlist/CORS/firewall configuration where the exact host matters.
The fix: Standardize the dashboard URL (pick laminar.sh or www.lmnr.ai and use it everywhere), and add a short "Domains" note clarifying that api.lmnr.ai is the API host while the dashboard lives at the chosen domain.
9. The self-hosted port story is scattered across pages with no single reconciliation (significant)
Problem: The ports a self-hoster must reconcile are spread across pages: hosting-options uses 5667 (dashboard UI) plus 8000/8001; the OTel page's matrix gives self-hosted gRPC 8001 / HTTP 8000 (and cloud 8443 / 443); the self-hosted evals page defaults httpPort 443 / grpcPort 8443 (cloud values, on the self-hosted page); and the datasets CLI page documents only --port (HTTP, default 443, "use 8000 locally") and never mentions the gRPC port at all.
Consequence: A self-hosted user has to cross-reference four pages to learn that local traffic targets 8000 (HTTP) / 8001 (gRPC) while the UI is 5667 — and the self-hosted evals defaults (443/8443) point at cloud ports, so a copied eval config silently targets the wrong place. The datasets CLI page's omission of the gRPC port leaves trace export under-specified.
The fix: Promote the OTel page's port matrix to a single canonical "Ports" reference and link every SDK/CLI/hosting page to it. On the self-hosted evals page, show the self-hosted defaults (8000/8001), not the cloud ones. Document the gRPC port on the datasets CLI page.
10. The changelog removes "online evaluators" but no page reconciles what that leaves behind (significant)
Problem: The April 2026 changelog entry states flatly "Online evaluators are removed," with no explanation of what online evaluators were, what (if anything) replaced them, or any migration guidance. Meanwhile the evaluator model remains fully documented as current: /docs/evaluations/concepts defines the EVALUATION/EXECUTOR/EVALUATOR span structure (span_type = 'EVALUATOR'), and the SQL editor's span_type enum reference still documents evaluator span types (the enum is described on the page; the specific HUMAN_EVALUATOR value is noted in the page-level evidence rather than re-quoted here).
Consequence: A developer who reads "online evaluators are removed" can't tell whether the evaluator spans and span_type values they see elsewhere are the thing that was removed, a different (offline) concept, or stale references. Removing a feature without a one-line "here's what replaced it / here's what's unaffected" forces readers to guess whether they're building against something deprecated.
The fix: Add a short migration/clarification note to the changelog removal: what an "online evaluator" was, what replaces it, and an explicit statement that the offline EVALUATOR span model (and the span_type enum values) is unaffected. Audit the concepts and SQL editor pages to confirm no removed-feature references remain.
11. "Pro" and "Enterprise" plans are referenced but there is no pricing or plans page (minor)
Location: /docs/platform/pii-redaction, docs.json nav
Problem: The PII redaction page gates a feature: "PII redaction is available on Pro and Enterprise plans on Laminar Cloud." But docs.json has no pricing/plans entry, and no plan-tier documentation appears anywhere in the docs — the tier names are used without ever being defined.
Consequence: A developer reading the PII page (a compliance-sensitive audience) can't find out what Pro/Enterprise include, what they cost, or how to upgrade from within the docs. They can usually find pricing on the marketing site, so the impact is bounded, but the gating reference is a dead end in-docs.
The fix: Link the marketing pricing page from the docs nav (or add a plans page), and link every plan-gated feature mention to it. At minimum, define what Pro and Enterprise include and how to upgrade.
12. The LiteLLM install command is unquoted and breaks in the default macOS shell (minor)
Location: /docs/tracing/integrations/litellm
Problem: The LiteLLM page writes pip install -U lmnr[all] without quotes, while other pages quote it as 'lmnr[all]'. In zsh (the default macOS shell), unquoted square brackets are glob characters and the command fails with zsh: no matches found: lmnr[all].
Consequence: A Mac developer copy-pasting from the LiteLLM page hits a confusing glob error on the very first step, with no hint that quoting is the fix.
The fix: Quote it everywhere: pip install -U 'lmnr[all]'. While there, fix the Steps component on this page where step "3" is a deprecation note rather than an action (the run-through reads 1, 2, [note], 4).
13. The same eval concurrency setting has two different names — and two different words — in TS vs Python (minor)
Problem: In the evaluate parameter table, the "parallel executor invocations" knob is concurrencyLimit in TypeScript but batch_size in Python — not a casing/idiom difference but two semantically different words for the same setting (both default 5). The manual-evaluation page documents further TS/Python asymmetry (TS createEvaluation takes positional args and accepts traceId only on createDatapoint; Python create_evaluation takes keyword args and accepts trace_id on update_datapoint).
Consequence: A developer switching languages (or an agent generating cross-language code) reasonably assumes batch_size means batching, not concurrency, and mis-tunes throughput. The naming divergence makes the two SDKs read like different products.
The fix: Align the names where possible, or add an explicit note that concurrencyLimit (TS) and batch_size (Python) are the same concurrency control. The manual-evaluation page already flags its asymmetry well — apply that same explicitness to the parameter table.
14. The datasets page says "two fixed JSON objects," then lists three (minor)
Location: /docs/datasets/introduction
Problem: The Format section states "Every datapoint has two fixed JSON objects: data and target" and then immediately lists three fields: data, target, and metadata.
Consequence: A developer building a dataset can't tell whether metadata is a first-class field or an afterthought, and whether it's persisted/queryable like the other two.
The fix: Make the count match the list — "each datapoint has three JSON objects: data, target, and metadata" (with target noted as evaluation-only) — or explain why metadata is categorized separately.
15. Replay caching silently falls back to live calls for TS users on the most common integrations (minor)
Location: /docs/debugger/caching
Problem: The debugger caching page lists supported integrations as OpenAI (Python), Anthropic (Python), Google GenAI (Python), LiteLLM (Python), and AI SDK (TypeScript). It then states that in TypeScript "the AI SDK is the only caching integration today" and that with an unsupported integration "every LLM call on a replay run goes live rather than serving from the recorded trace." A TypeScript developer using the direct OpenAI or Anthropic integration is therefore unsupported for caching — but the OpenAI/Anthropic integration pages don't say so.
Consequence: A TS developer on the direct OpenAI/Anthropic integrations reasonably expects replay to serve from cache and instead silently incurs live LLM calls on every replay — real cost and non-deterministic debugging. The limitation is disclosed, but only on the caching page they may never open.
The fix: Add a one-line caching-support note (and link to the caching page) on the TS OpenAI/Anthropic integration pages, stating that replay caching requires the AI SDK + wrapLanguageModel and that direct-SDK replays run live.
16. No integration example shows error, verification, or failure-mode handling (minor)
Problem: Across the integration pages, every snippet is a happy-path "add one line to instrument" example. None shows what "working" looks like, what happens when initialization fails, the API key is wrong, the collector is unreachable, or instrumentation silently no-ops — and the troubleshooting page (see issue 2) is itself broken.
Consequence: Given that the most common real failure here is silent (un-instrumented calls produce no traces and no error — see issues 2 and 6), the absence of any "how do I know it's working / what does failure look like" guidance leaves developers debugging emptiness. The getting-started page's verification query (SELECT * FROM traces ORDER BY start_time DESC LIMIT 1) is the only smoke test offered, and it isn't surfaced on the integration pages.
The fix: Add a short "Verify it's working" + "Common failure modes" block to the integrations overview (and link it from each integration page), including the trace-count smoke-test query and the silent-no-op symptoms.
17. The changelog mixes three date-heading formats with month gaps (minor)
Location: /docs/changelog
Problem: Changelog entry headings are inconsistent: most are month-only ("June 2026", "April 2026", "January 2025"), one is a specific day ("December 17, 2025"), and one is a range ("April–May 2025"). There are also month gaps (e.g., no March 2026 between February and April 2026).
Consequence: A developer trying to pin "when did X ship / what version was current on date Y" can't reliably order or locate entries when the grain shifts between month, day, and range — minor, but it undercuts the changelog's value as a timeline.
The fix: Standardize on one date grain for changelog headings (month, or full date) and apply it consistently.
What they do well
Consistent auth pattern across HTTP surfaces — the SQL API (POST /v1/sql/query), OTLP (/v1/traces), and MCP (/v1/mcp) all use the same Authorization: Bearer <PROJECT_API_KEY>, which is easy to reason about and agent-friendly.
The OTel page's port matrix and the SQL editor's table reference are genuinely good single-source-of-truth tables — and the OTel page proactively documents real footguns (the lowercase-authorization gRPC requirement and the Python capital-A error). The problem is that other pages don't defer to these references.
The manual-evaluation page openly flags SDK asymmetry instead of hiding it, and the changelog documents removals, not just additions — that honesty is rare and valuable (the gap in issue 10 is follow-through, not candor).
Top 3 recommendations
Pick one CLI name and one command grammar (lmnr vs lmnr-cli, dataset vs datasets, and where eval lives) and regenerate every snippet from it — this is the single most confusing contradiction in the docs.
Make the official examples run against the official tools — fix the broken from 'anthropic' import, the dashboards page's non-existent evaluation_results table, the comparing-runs path that points at the Postgres-only evaluations table, the invalid claude-3-7-sonnet ID, and the empty-string API key in the README quickstart. These are copy-paste-broken today.
Add machine-readable surfaces and one source of truth for shared facts — ship llms.txt/llms-full.txt and an OpenAPI spec, fix the README's /docs/installation 404, and consolidate ports, domains, model IDs, and table names into canonical references that other pages link rather than restate.