The docs are well-structured for humans with clear product separation (Agent/Search/Fetch/Browser), an llms.txt index, and webhook/error references — but several agent-facing artifacts are broken or contradictory in ways that will silently mislead both AI coding agents and human developers.
1. The "OpenAPI" spec linked from llms.txt and the docs nav is a Mintlify plant store sample (critical)
Location:https://docs.tinyfish.ai/api-reference/openapi.json — linked from /llms.txt as an "OpenAPI Specs" entry and pointed to by /api-reference ("For the generated OpenAPI schema, use the API Reference top navigation tab")
Problem: The file at /api-reference/openapi.json is the Mintlify boilerplate "OpenAPI Plant Store" spec, with servers: http://sandbox.mintlify.com, a single /plants GET, and a bearerAuth security scheme. None of this is TinyFish's actual API. Worse, this isn't just an agent-only trap: /api-reference explicitly directs human readers to the same artifact via the top navigation tab.
Consequence: An AI coding agent following llms.txt will ingest a fake plant-store spec as authoritative and generate code that calls sandbox.mintlify.com/plants with bearer auth — instead of agent.tinyfish.ai/v1/automation/run with the X-API-Key header documented elsewhere. A human clicking the "API Reference" nav lands on the same broken spec. The real specs at /openapi/{fetch,search,main,browser}.json are correct, but neither audience is steered there.
The fix: Replace /api-reference/openapi.json with a real combined OpenAPI spec (or delete it from llms.txt and update /api-reference's "API Reference top navigation tab" pointer to the four real product specs at /openapi/{fetch,search,main,browser}.json).
2. Support email contradicts itself across the docs and the OpenAPI contact block (significant)
Location:/faq, /anti-bot-guide vs. /openapi/main.json
Problem: The FAQ and Anti-Bot Guide both list support@tinyfish.io as the support address. The OpenAPI info.contact.email on the main spec lists support@tinyfish.ai. The domain on the docs site, marketing site, and dashboard is .
Consequence: Developers and agents pulling contact info from the OpenAPI spec will email a different address than developers reading the FAQ. One of these mailboxes is almost certainly not the canonical inbox, so support requests vanish into the wrong tenant.
The fix: Pick one canonical address (almost certainly support@tinyfish.ai to match the domain) and replace every other occurrence — /faq and /anti-bot-guide both currently use the .io TLD.
3. No changelog exists despite a versioned, evolving API surface (significant)
Location:https://docs.tinyfish.ai/changelog
Problem: Fetching /changelog returns {"status": 404, "error": "page_not_found"}. The docs cover versioned products with run-status enums, credit costs, an evolving SDK/CLI/Vault matrix, and OpenAPI specs all pinned at version: 1.0.0 — and there is no changelog or release-notes page anywhere in the docs surface.
Consequence: Developers and agents have no way to discover when run statuses, credit costs, SSE event types, or auth methods changed. Anyone integrating today has no record of what may break tomorrow. Bug reports against stale behavior can't be triaged to a release.
The fix: Publish a real changelog at /changelog (or any discoverable path) and link it from the docs index. Document the versioning policy explicitly — the OpenAPI specs currently pin version: 1.0.0 with no other signal of when that bumps.
4. "Concurrent Requests" sync example uses the async client (significant)
Location:/examples/bulk-requests-sync
Problem: The page is titled "Concurrent Requests / Process multiple runs in parallel" and lives at a URL ending in bulk-requests-sync, but the Python sample imports AsyncTinyFish and uses asyncio.gather. There's no sync example on a page whose slug literally promises one.
Consequence: A developer (or agent) searching for a synchronous batch pattern lands here, copy-pastes the snippet, and ends up with async/await code they didn't ask for — or worse, an agent treats the AsyncTinyFish import as the canonical sync client and generates broken code elsewhere.
The fix: Either rename the page to drop -sync and reframe it as "concurrent runs (async)", or add a true synchronous variant (e.g. concurrent.futures.ThreadPoolExecutor with the sync client) above the async one and label them clearly.
5. SSE reconnection guidance is human-only prose; not documented on the OpenAPI operation (significant)
Location:/agent-api/reference (Reconnection paragraph) vs. /openapi/main.json/run-sse operation
Problem: The reference page tells humans: "SSE streams do not support Last-Event-ID reconnection. If your client disconnects mid-stream, recover by polling GET /v1/runs/{run_id}." This is a critical operational fact that appears only in prose, not in any structured form the OpenAPI consumer would see for the /run-sse operation.
Consequence: An agent generating an SSE client from the OpenAPI spec will assume standard SSE reconnection semantics (sending Last-Event-ID) and silently fail to recover from disconnects. Long-running runs get orphaned.
The fix: Add the no-reconnect behavior and the polling fallback as a description on the /run-sse operation in openapi/main.json, and link to GET /v1/runs/{run_id} from the same operation.
6. SSE COMPLETE event has undocumented optional fields and unspecified terminal-failure mapping (significant)
Location:/agent-api/reference (SSE event table) and /error-codes
Problem: The reference lists COMPLETE fields as type, run_id, status, result?, error?, help_url?, help_message?, timestamp — four optional fields with no schema for any of them. The same page then mentions run-level failures including TASK_FAILED, SITE_BLOCKED, MAX_STEPS_EXCEEDED, TIMEOUT, INSUFFICIENT_CREDITS without saying which field carries them (status? error.code?) or what shape result takes per failure mode. /error-codes only documents RATE_LIMIT_EXCEEDED in detail.
Consequence: Developers writing error-handling switches don't know whether to branch on status === "FAILED" plus an error.code, or on top-level status === "SITE_BLOCKED". Two integrators will write incompatible code against the same event. Agent-generated retry/backoff logic will be brittle.
The fix: Publish the concrete JSON shape of result, error, help_url, and help_message for each terminal status, and add every run-level failure code (TASK_FAILED, SITE_BLOCKED, MAX_STEPS_EXCEEDED, TIMEOUT, INSUFFICIENT_CREDITS) to /error-codes with the field that carries them.
7. Run status enum on /key-concepts/runs omits the terminal states the rest of the docs depend on (significant)
Location:/key-concepts/runs vs. /agent-api/reference vs. /faq
Problem:/key-concepts/runs lists the lifecycle as PENDING / RUNNING / COMPLETED / FAILED / CANCELLED — five states. The Agent reference's Reconnection paragraph says runs reach a terminal status of COMPLETED, FAILED, or CANCELLED, which matches. But the same reference page introduces failure codes like SITE_BLOCKED, MAX_STEPS_EXCEEDED, TIMEOUT, INSUFFICIENT_CREDITS and never tells you whether those are values of status or values of error.code inside a FAILED status. The FAQ adds another layer: "status: COMPLETED means the infrastructure succeeded ... it does NOT mean the goal was achieved" — implying a separate result-level success/failure flag that the lifecycle table doesn't show.
Consequence: Developers can't write a correct state machine. An agent integrating with TinyFish will conflate infrastructure status with goal outcome — which is exactly the trap the FAQ warns humans about.
The fix: On /key-concepts/runs, add a separate "Run Result" section that documents the result.success / outcome field distinct from status, and explicitly map every failure code to its status value.
8. Webhooks page lists /run-batch as a supported endpoint but it's absent from the Agent reference (significant)
Location:/webhooks vs. /agent-api/reference vs. /llms.txt
Problem:/webhooks says: "Webhooks are supported on all run endpoints: /run, /run-async, /run-sse, and /run-batch." The Agent reference's Endpoints table only lists run, run-async, run-sse. llms.txt includes an entry at /api-reference/automation/start-multiple-automations-asynchronously.md — strongly implying the endpoint exists — but never names it /run-batch. The MCP page describes a batch_create tool capped at 8 runs, which is yet a different surface.
Consequence: Developers reading the webhooks page assume /v1/automation/run-batch exists and try to call it. Either they hit 404 (path doesn't exist under that name) or they get unexpected semantics. Agents generating client code will fabricate the endpoint name from the webhooks page.
The fix: Add the batch endpoint as a first-class entry in the Agent reference Endpoints table with its full request/response schema, the actual path, and concurrency limits — and either confirm /run-batch is the real path or update the webhooks page to match the canonical path.
9. MCP credit cost contradicts the docs index, cookbook, and marketing site on Search and Fetch (significant)
Location:/mcp-integration/index vs. / (docs index) vs. cookbook README vs. https://www.tinyfish.ai/
Problem: The MCP page lists search at "1 credit = 2 searches" and fetch_content at "1 credit = 15 fetches" — directly contradicting the docs index ("Free Search and Fetch. Credits are used only by Agent and Browser."), the cookbook README ("Search and Fetch are now FREE … no credit card required"), and the marketing homepage ("Both are included on every plan and use no credits").
Consequence: Developers using TinyFish through MCP believe Search/Fetch consume credits and ration their usage accordingly. Developers using the REST API believe they're free. Same product, two pricing models published simultaneously.
The fix: Update /mcp-integration/index to remove the search and fetch_content credit lines or explicitly mark them as free with unlimited or rate-limited usage. Add a single canonical pricing/credits page and link it from every product reference.
10. MCP OAuth 2.1 has no documented scopes, token lifetime, or revocation flow (significant)
Location:/mcp-integration/index and /authentication
Problem:/authentication says MCP uses "OAuth 2.1" and that's the entire reference. /mcp-integration/index describes the pre-flight ("sign in to claude.ai and agent.tinyfish.ai") and the OAuth handoff, but never specifies which scopes the token grants, how long the token is valid, how it's refreshed, where it's stored, or how a user revokes it. There is no security page documenting these properties.
Consequence: Enterprises and security reviewers can't evaluate the integration without contacting support. A developer can't programmatically refresh, rotate, or revoke MCP credentials. Agents acting through MCP have no documented way to detect expiry vs. permission-denied.
The fix: Add an OAuth section to /authentication that documents the scopes issued, token lifetime, refresh behavior, revocation endpoint or dashboard page, and the precise error returned when a token expires vs. is revoked. Cross-link from /mcp-integration/index.
11. Cancellation flow under /key-concepts/endpoints is incomplete vs. the Agent reference (minor)
Location:/key-concepts/endpoints vs. /agent-api/reference
Problem:/agent-api/reference correctly lists cancel support for both /run-async and /run-sse. /key-concepts/endpoints describes the cancellation flow only inside the /run-async section ("3. Cancel a run (optional)") and never restates that /run-sse also supports cancellation. The reference is authoritative; the concepts page is just incomplete.
Consequence: A developer skimming /key-concepts/endpoints to choose a surface may not learn that SSE runs can be cancelled, and build SSE flows without a stop mechanism. SSE runs can be long-lived and credit-expensive.
The fix: In /key-concepts/endpoints, lift the cancellation note to a shared subsection that explicitly lists both /run-async and /run-sse and links once to POST /v1/runs/{id}/cancel.
12. Vault provider connection cap is buried in a Troubleshooting line (minor)
Location:/vault-setup
Problem: The "you can connect a maximum of 2 providers at the same time" rule only appears as a Troubleshooting bullet ("All providers already connected — You can connect a maximum of 2 providers at the same time."). It isn't surfaced on the main connection flow, the API reference, or as a documented limit anywhere else.
Consequence: Developers programmatically managing vault connections (e.g. CI rotating between 1Password and Bitwarden across tenants) hit the limit at runtime instead of at design time. The 404/503 distinctions on the same page reinforce that this is an API a developer touches programmatically.
The fix: Document the 2-provider cap as a first-class limit in the main "Connect Your Provider" section, and surface it in the OpenAPI spec for POST /v1/vault/connections as a 409/422 error.
13. SDK/CLI/Vault support matrix is asserted but never tabled (minor)
Location:/vault-setup
Problem: The page states: "Vault credentials are currently supported in the Playground and REST API. Python SDK, TypeScript SDK, and CLI support is coming soon." There's no equivalent feature-support matrix anywhere — no table cross-referencing which features (vault, webhooks, batch, SSE, cancellation, proxies, stealth) work on which surface (REST, Python SDK, TS SDK, CLI, MCP, Dify).
Consequence: Picking the right surface for a project requires reading every reference page to discover what's missing. Agents recommending an SDK will not know that vault is REST-only.
The fix: Add a single "Feature support by surface" matrix to the docs root or /key-concepts/, listing each feature × each client and the current support state.
What they do well
Clear product taxonomy. The four-API split (Agent / Search / Fetch / Browser) with canonical hostnames on the index is genuinely useful — both for humans and for agents that need to disambiguate.
Honest scoping of limits. The Anti-Bot guide explicitly states "TinyFish cannot solve CAPTCHAs" and the FAQ explicitly warns that COMPLETED ≠ goal succeeded. These kinds of contracts are exactly what production integrators need.
Prompting guide has real, quantified guidance. "4.9x faster, 16x less unnecessary data" plus the seven-component goal anatomy is more concrete than what most agent platforms publish.
Top 3 recommendations
Fix the /api-reference/openapi.json plant-store regression immediately and update /api-reference's "top navigation tab" pointer. This single artifact misleads both llms.txt-following agents and humans clicking the API Reference nav.
Publish a single canonical credits/pricing page and reconcile the MCP page, marketing site, and docs index — Search/Fetch are either free or they aren't.
Promote the SSE event schema, run-status state machine, and failure-code mapping to first-class reference material (with concrete JSON shapes) so both humans and agents can write correct state machines without inferring fields from prose.
Fixed
0
Skip
41
Fail
1
Failing pages
https://docs.tinyfish.ai/authentication
Summary
Verified 114 code blocks across 18 TinyFish documentation surfaces using the supplied API key against the live agent.tinyfish.ai, api.search.tinyfish.ai, api.fetch.tinyfish.ai, and api.browser.tinyfish.ai endpoints. The TypeScript SDK (@tiny-fish/sdk@0.0.8) and the @tiny-fish/cli@0.1.6 package were installed and exercised end-to-end. The Python SDK could not be installed because tinyfish requires Python ≥3.11 and the sandbox provides Python 3.9, so every snippet that imports tinyfish (Python) was SKIP'd with that runtime reason. One genuine docs failure was identified: the "debug authentication" curl on /authentication mixes -I and -X POST -d, which curl rejects with exit code 2. All vault-scoped snippets that include hard-coded credential IDs were SKIP'd because the test account has no synced vault items. JSON OpenAPI files validated as syntactically valid; note that /api-reference/openapi.json is the Mintlify "Plant Store" sample rather than the TinyFish schema, but the file is structurally valid JSON so it is recorded as PASS.
Required credentials
TINYFISH_API_KEY — used in all REST/SDK/CLI requests (header X-API-Key and SDK auto-detection)
curl exits 2 with Warning: You can only select one HTTP request method! You asked for both POST (-d, --data) and HEAD (-I, --head). Diagnosis: docs bug — -I (HEAD) cannot be combined with -X POST -d. Suggested fix in docs: drop -I and add -i/-v (or -D -) to inspect headers while keeping a POST body. Not applied here.
Python result validation (from tinyfish import ...)
SKIP
Python SDK unavailable
5
TypeScript result validation
PASS
Printed "Success: ..."
6
TypeScript rate-limit handling with RateLimitError
PASS
No retries triggered; run completed
7
cURL vault use_vault: true
PASS
API accepted (verified via async endpoint returning 200 + run_id; sync endpoint accepts identical body — confirmed via /run-async since /run runs to completion past test timeout)
Package exists and installs cleanly; verified via local install in test env (global flag would require root which sandbox lacks, but install behavior is identical)
Valid JSON syntax. Observation (not a verification failure): file contents are the Mintlify "OpenAPI Plant Store" sample (server http://sandbox.mintlify.com), not the TinyFish schema — a docs/content issue separate from snippet executability.