Zyte

Zyte Documentation Audit

Deep, mostly well-structured docs for Zyte API and Scrapy Cloud with real error references and multi-language examples — undermined by a broken agent-facing surface (dead llms.txt links, corrupted Markdown-export anchors), cross-page contradictions on rate limits, and a Search API reference that never names a valid value for its own required parameter.

1. llms.txt links to eight pages that 404 (critical)

Problem: https://docs.zyte.com/llms.txt — the index explicitly published for AI agents — lists ~133 relative .md links, and 8 of them return HTTP 404: smart-proxy-manager.md, smart-proxy-manager/errors.md, smart-proxy-manager/fetch-api.md, smart-proxy-manager/headless.md, smart-proxy-manager/old-fetch-api.md, smart-proxy-manager/residential.md, smart-proxy-manager/sessions.md, and smart-proxy-manager/stats.md. The .html versions of the same pages return 200 (e.g. smart-proxy-manager/errors.html), and sibling pages like smart-proxy-manager/integrations/python.md and smart-proxy-manager/sunset.md resolve fine — so the Markdown export simply doesn't exist for those eight pages while the index still advertises them.

Location: https://docs.zyte.com/llms.txt

Consequence: An AI agent crawling the advertised index hits a 6% dead-link rate concentrated in one product area. Agents don't retry with .html; they silently lose the entire Smart Proxy Manager error reference and Stats API, exactly the pages a migrating SPM customer's coding agent would need.

The fix: Generate the .md exports for the eight missing SPM pages, or remove/redirect their entries in llms.txt. Add a CI link-check over every URL in llms.txt.

2. Search API requires a domain but never names one valid value (critical)

Location: https://docs.zyte.com/zyte-api/usage/search/request.md (also quickstart.md, geo.md, index.md)

Problem: The request reference marks domain as required: "A supported search domain. Unsupported domains return a 400 error." But no page in the Search API section — index, quickstart, request parameters, or geo-targeting — lists the supported domains. Every single example, including the geo page's "Targeting a regional domain" section whose entire point is choosing a real domain, uses the placeholder "domain": "search.engine.com". Meanwhile the usage hub describes the feature as "Search Google with a typed interface."

Consequence: The quickstart is un-runnable as written: copy-pasting any example sends a placeholder domain and returns the documented 400. A developer (or agent) has to guess whether google.com, www.google.com, or google.co.uk is the accepted spelling, with no enumeration and no error-message guidance documented.

The fix: Publish the list of supported domain values (or link to a live endpoint that enumerates them) and use a real supported domain in every example.

3. Enterprise rate limit: pricing says 10000 RPM, rate-limit page says "Custom" (significant)

Location: https://docs.zyte.com/zyte-api/pricing.md vs https://docs.zyte.com/zyte-api/usage/rate-limit.md

Problem: The pricing plan table states the Enterprise rate limit is "10000 RPM". The dedicated rate-limits page states, under "API key rate limit": "Enterprise plans: Custom / Standard plans: 3000 RPM." Both pages cross-link each other, so a reader following the pricing table's own link lands on a page that contradicts it.

Consequence: Capacity planning for an Enterprise contract depends on this number — a team sizing concurrency with the documented formula (max concurrency ≈ RPM ÷ 60 × response time) gets materially different architectures from 10000 RPM vs "negotiate it". Agents reading both pages will report conflicting facts non-deterministically.

The fix: Pick one statement — e.g. "Enterprise: 10000 RPM by default, custom limits available" — and use it on both pages.

4. Markdown exports ship corrupted cross-page anchors (significant)

Location: https://docs.zyte.com/zyte-api/get-started.md (pattern visible across .md exports)

Problem: The .md export surface advertised via llms.txt fuses anchor ids with a stray .md suffix, producing anchors that exist nowhere: [Follow the tutorial](../web-scraping/tutorials/main/index.md#tutorial.md), [Usage](usage/index.md#zapi-usage.md), [Reference](usage/reference.md#zapi-reference.md), [Proxy mode](usage/proxy-mode.md#zapi-proxy.md), [Migrate](migration/index.md#zapi-migrate.md), [Zyte IDE](ide/index.md#zyte-ide.md). The HTML version of the same page links cleanly to usage/index.html, so the corruption is specific to the Markdown pipeline.

Consequence: The Markdown surface exists precisely so agents can navigate docs programmatically; anchors like #zapi-usage.md resolve to nothing, so anchor-following agents and link checkers report failures across the export, eroding trust in the very surface llms.txt promotes.

The fix: Fix the md-export transform so anchors render as #zapi-usage (or drop the fragment), and lint the generated .md files for #…​.md fragments.

5. No changelog anywhere in the docs (significant)

Location: https://docs.zyte.com (site-wide); probed /changelog, /zyte-api/changelog, /release-notes — all 404

Problem: llms.txt indexes ~133 pages spanning tutorials, references, migration guides, and FAQs, but no changelog or release-notes page exists. Yet the docs describe a platform that changes under users' feet: the API reference warns "In the future, however, the default value may depend on the target website," pricing notes "We review tier assignments quarterly," and SPM is being force-migrated "during Tuesday, December 9, 2025."

Consequence: Developers have no way to discover behavior changes (extraction defaults, tier reassignments, new Search API fields like the "coming soon" aiOverview) except by rereading every page. Breakage after a silent default change is undiagnosable from the docs alone.

The fix: Publish a dated changelog page, list it in llms.txt, and record default-behavior changes and deprecations there.

6. Proxy-mode endpoint written three different ways (minor)

Location: https://docs.zyte.com/zyte-api/usage/index.html vs https://docs.zyte.com/zyte-api/usage/proxy-mode.md

Problem: The usage hub says to use "https://api.zyte.com:8011 as your proxy endpoint." The proxy-mode page says "use the api.zyte.com:8011 endpoint," and its C# example constructs new WebProxy("http://api.zyte.com:8011", true). Three spellings — https://, scheme-less, and http:// — for the same endpoint across two pages.

Consequence: Proxy URL schemes are not cosmetic: many HTTP clients treat http:// and https:// proxy URLs differently (CONNECT vs TLS-to-proxy), and developers debugging the documented SSL/CA-certificate errors will reasonably suspect the scheme. An agent asked "what is the proxy endpoint?" gets a different literal answer depending on which page it read.

The fix: Standardize on one canonical form (e.g. http://api.zyte.com:8011) everywhere, and add a sentence on which scheme to use in proxy configuration.

7. End-of-life Smart Proxy Manager still occupies ~30 undated pages in the index (minor)

Location: https://docs.zyte.com/llms.txt (SPM section) and https://docs.zyte.com/smart-proxy-manager/sunset.md

Problem: The sunset FAQ is unambiguous: "SPM is end-of-life… SPM Platform will then be retired," with automigration completing December 9, 2025. Yet llms.txt still indexes roughly 30 SPM pages — full integration guides for Selenium, Puppeteer, PhantomJS, R, Objective-C, etc. — with only a parenthetical "(replaced by Zyte API)" on one entry and no EOL banner evident in the listings. The tooling pages are also broken at the top (see issue 1).

Consequence: A developer or agent landing on "Using Smart Proxy Manager with Playwright" gets a complete-looking guide for a product that is being retired within weeks, with no signpost to the Zyte API equivalent from the index.

The fix: Mark every SPM entry as deprecated in llms.txt (as done for the Fetch API entry), add an EOL banner linking to the sunset FAQ and the proxy-mode migration guide, and plan removal after the migration date.

What they do well

• A genuinely thorough error reference: concrete status codes (429/503/520/521) each paired with machine-readable type values like /download/temporary-error, plus explicit retry guidance and a documented 521→520 edge case.
• Real machine-readability investment: llms.txt and a 1.5 MB llms-full.txt, per-page .md exports, and a downloadable OpenAPI spec behind the Redocly reference.
• Code examples in many languages (C#, Java, curl, CLI, Python sync/async) with a dedicated requirements page explaining how to run them.

Top 3 recommendations

1. Add a link checker to the docs build that validates every llms.txt entry and every .md-export link/anchor; fix the eight 404ing SPM exports and the #…​.md anchor corruption.
2. Document the supported Search API domain values and replace search.engine.com with a real, runnable example.
3. Reconcile the Enterprise rate-limit figure between pricing.md and rate-limit.md, and publish a changelog so default-behavior changes (extraction defaults, tier reviews, EOL dates) are discoverable.