Modelence Documentation Audit

The docs are visually clean Mintlify pages with broad surface coverage, but they are riddled with cross-page contradictions, broken links, an outright dead OpenAPI spec, an incomplete llms.txt index, and a Store API reference that contradicts every Store example in the guides. For an LLM-targeted, agent-ready platform, these are platform-level problems.

1. OpenAPI spec advertised to LLMs is 404 (critical)

Location: https://docs.modelence.com/llms.txt → https://docs.modelence.com/api-reference/openapi.json

Problem: The site's own llms.txt lists [openapi](https://docs.modelence.com/api-reference/openapi.json) under an ## OpenAPI Specs section. Fetching that URL returns the literal body Asset not found (the SPA shell returns 200 but the asset is missing). There is no other machine-readable schema for the 25+ Data API endpoints.

Consequence: Every agent that follows llms.txt (the canonical entry point Modelence promotes for LLM consumers) hits a dead OpenAPI link. Coding agents cannot programmatically discover the Data API surface — they have to scrape the prose. A docs site whose primary AI affordance is broken is a hard credibility hit.

The fix: Either ship a real openapi.json at that path (generated from the same source as the Data API markdown) or remove the entry from llms.txt. If it's intentionally not provided, say so explicitly in llms.txt so agents don't waste a fetch.

2. Broken markdown fence corrupts the central env-vars reference (critical)

Location: /core-concepts/environment-and-setup

Problem: The page opens an env code block with four backticks ( env title=".modelence.env"`) and never closes it before continuing prose. As a result, the `## Modelence Cloud Configuration` H2, the bullet list, the `### Connecting to Modelence Cloud` H3, and the `npx modelence@latest setup --token` block all render *inside* the unterminated env code fence. The fence finally closes with a stray after the npx command.

Consequence: The single page meant to be the canonical environment-variable reference renders as a half-broken code block. Worse, the core variable list is incomplete: only MONGODB_URI, PORT, and MODELENCE_SITE_URL are listed, while the rest of the docs reference MODELENCE_AUTH_GITHUB_*, MODELENCE_AUTH_GOOGLE_*, MODELENCE_EMAIL_*, MODELENCE_LOG_LEVEL, MODELENCE_VOYAGEAI_API_KEY, DATA_API_KEY, and VOYAGE_API_KEY with no central anchor. Developers cannot reliably find the full set.

The fix: Close the fence after MODELENCE_SITE_URL=http://localhost, restore the headings to top-level markdown, and add a complete table of every MODELENCE_* variable used anywhere in the docs (auth, email, voyage, telemetry, data API) with type, default, and consuming module.

3. Store API reference contradicts every Store example in the guides (critical)

Location: /api-reference/modelence/server/classes/Store vs /stores, /tutorial, /core-concepts/mutations

Problem: The auto-generated Store reference says explicitly:

❌ TypeScript error - 'id' is not in schema: await store.findOne({ id: '123' })

but the /stores guide demonstrates await dbTodos.updateOne({ id: todoId }, { $set: { isCompleted: true } }), the /tutorial (Todo App) uses { id: todoId } style selectors, and several mutation examples follow suit. The reference says you must use _id or findById; the guides use id.

Consequence: Developers copy code from the guide, paste it into TypeScript, and get a compile error. Or worse, they cast and ship code that silently never matches a document. This is the #1 thing to get right in a TypeScript-first ORM.

The fix: Pick one. If id is a synthetic alias the Store layer accepts, document it on the API reference and update the type to allow it. If it's not, fix every guide example to use _id / findById / requireById.

4. Voyage AI tutorial stores an API key as type: 'string', contradicting the Configuration security model (critical)

Location: /voyage-ai (Step 4) vs /configuration

Problem: The Configuration page is unambiguous: 'secret' is for "a sensitive value like an API key or token. Masked in the Cloud dashboard" and string is described as "a short text value (single line)". The Voyage AI tutorial nevertheless registers the API key with apiKey: { type: 'string', isPublic: false, default: '' }.

Consequence: Anyone who copy-pastes the Voyage AI module ends up with their Voyage API key unmasked in the Modelence Cloud dashboard. This is a textbook security regression baked into a copy-paste tutorial.

The fix: Change the tutorial schema to apiKey: { type: 'secret', default: '' } (note: 'secret' cannot be isPublic: true, per the Configuration page's own warning) and add a one-line note explaining why secrets should never use 'string'.

5. Email page ships three dead /docs/* links in a high-traffic Next Steps block (critical)

Location: /email (Next Steps section)

Problem: The page closes with:

• [Authentication](/docs/authentication) → 404
• [User Management](/docs/user-management) → 404
• [API Reference](/docs/api-reference) → 404

The correct paths drop the /docs prefix (/authentication, /authentication/user-management, /api-reference).

Consequence: Every developer who finishes the Email guide and tries to navigate to the next logical doc gets three 404s in a row. Agents indexing internal links will mark these as broken and lose ~20% of the email page's outbound graph.

The fix: Remove the /docs prefix on all three links. Add a CI check (e.g., remark-validate-links or Mintlify's link checker) that fails the build on internal 404s.

6. Quick Start vs Setup vs Project Structure disagree on the env file name (critical)

Location: /quickstart vs /setup vs /core-concepts/project-structure vs /authentication/google-sign-in

Problem: Quick Start says the dev server reads from .env ("the port you specified in the .env file"). Google Sign-In tells developers to "configure three variables in your .env file." Setup, Project Structure, Environment & Setup, and the Voyage AI Quick Start all say .modelence.env. The Voyage AI tutorial itself uses both — .modelence.env in the Quick Start template section and .env in Step 6.

Consequence: Developers create the wrong file, the dev server starts without their config, and they spend time debugging missing env vars. With two filenames in circulation, agents produce inconsistent scaffolding. There is no documented relationship between .env and .modelence.env (e.g., "both are loaded; .modelence.env wins"), so it's not even clear if both are valid.

The fix: Standardize on .modelence.env everywhere. Edit /quickstart and /authentication/google-sign-in to match. If .env is also loaded as a fallback, document that explicitly with precedence.

7. Tutorial code is non-runnable: broken link, undefined handlers, and a method that can't survive serialization (critical)

Location: /tutorial

Problem: The Todo App tutorial — the docs' canonical end-to-end example — has four overlapping defects:

1. [Store API Reference](../../api-reference/store) 404s. Real path: /api-reference/modelence/server/classes/Store.
2. handleCreateTodo, handleDeleteTodo, and isCreating are referenced in JSX but never defined anywhere in the sample.
3. todo.isOverdue() is invoked on the result of useQuery(modelenceQuery('todo.getAll')). methods defined on a Store live on document instances; once the query handler returns and the data is JSON-serialized over the wire, instance methods are stripped. This will throw TypeError: todo.isOverdue is not a function at runtime.
4. The mutation's onSuccess invalidates queryKey: ['todo.getAll'] — a hand-rolled key — but modelenceQuery('todo.getAll') uses its own internal key shape, so cache invalidation likely silently no-ops.

Consequence: The tutorial is the front door for new developers. Following it verbatim produces broken code, broken cache invalidation, and a broken link. Agents using this tutorial as a few-shot example will reproduce all four defects.

The fix: Replace the broken Store link, define every referenced handler in the sample, remove todo.isOverdue() from the JSX (or add a server-side isOverdue boolean on the document) with a callout explaining that Store methods are server-side only, and use modelenceQuery('todo.getAll').queryKey (or the SDK's own invalidate helper) for invalidation.

8. llms.txt is sold as the canonical agent index but omits ~30+ public symbols (significant)

Location: /llms.txt vs the rest of the docs

Problem: The homepage explicitly steers agents to llms.txt as the documentation index. Yet the file omits many public symbols documented elsewhere on the site:

• Client API: verifyEmail, resendEmailVerification, sendResetPasswordToken, resetPassword (referenced in /authentication, /authentication/email-verification, /authentication/password-reset)
• Server API: LiveData, Store.watch(), Store.vectorIndex(), Store.vectorSearch(), Store.requireById(), Store.upsertOne(), setConfig, time, logDebug, logInfo, logError, startTransaction, captureError, disableUser, deleteUser, validateSignup, validateProfileUpdate, generateHandle, onAfterLogin, onAfterSignup, onAfterEmailVerification, onLoginError, onSignupError, onEmailVerificationError
• Error classes: AuthError, ValidationError, RateLimitError, AuthConfig, UserInfo
• WebSocket primitives: ServerChannel, ClientChannel, startWebsockets, joinChannel, leaveChannel, broadcast

Consequence: Agents that load llms.txt cannot resolve these symbols and either hallucinate signatures or refuse to use the API. A developer searching the site has no single page that lists what's actually exported.

The fix: Regenerate llms.txt from the actual TypeDoc output rather than a curated subset. If TypeDoc is the source of truth, ensure every guide-mentioned symbol exists in the TypeDoc output, and that llms.txt enumerates the full type-aliases / classes / variables / functions tree for each subpath (modelence/server, modelence/client, modelence).

9. callMethod signature contradicts every guide example that uses it (significant)

Location: /api-reference/modelence/client/functions/callMethod vs /core-concepts/queries, /core-concepts/mutations

Problem: The reference declares callMethod<T>(methodName, args, options): Promise<T> with no ? markers on args or options. Every guide example calls it with one argument, e.g. await callMethod('todo.getAll') in /core-concepts/queries. The reference page also has zero prose, no example, and no link to the CallMethodOptions type.

Consequence: Either the reference is wrong (parameters are actually optional) or every guide example fails TypeScript strict mode. Developers see a compile error from a copy-pasted snippet and have no narrative to disambiguate.

The fix: Add ? markers to args and options in the type signature if they are optional, then add a usage example to the reference page. If they are required, fix the guide examples to pass {} for args.

10. setConfig is in the Voyage AI tutorial but not in any reference (significant)

Location: /voyage-ai (Step 6, Option 2)

Problem: The tutorial imports setConfig from modelence/server and tells developers to call await setConfig('voyage.apiKey', 'your_api_key_here'). setConfig does not appear in llms.txt, the API Reference index, the Configuration page's Type Reference, or the Module class signature.

Consequence: Developers run import { setConfig } from 'modelence/server' and either get a TypeScript error or — if it does exist in the package — wonder whether it's a stable API, what its security implications are (it's writing a secret to MongoDB), or how it interacts with the precedence order (defaults < Cloud < env). Plus three different env var names for the same Voyage key (MODELENCE_VOYAGEAI_API_KEY, VOYAGE_API_KEY, the error message says "VOYAGE_API_KEY environment variable is not set") compounds the confusion.

The fix: Document setConfig as a first-class server function with parameter types, security notes, and precedence behavior — or remove it from the tutorial. Pick one env var name for the Voyage key and use it consistently across the tutorial, error messages, and example code.

11. Migration file layout contradicts itself across two pages (significant)

Location: /core-concepts/project-structure vs /core-concepts/migrations

Problem: Project Structure shows migrations as a single file: src/server/migrations.ts. The dedicated Migrations page tells developers to "Define each migration handler in its own file under a migrations/ directory" with a per-version handler file plus an index.ts. Two file-system layouts, no statement that both are accepted.

Consequence: Developers scaffold the wrong shape. Agents generating boilerplate pick whichever they saw last. Neither layout is endorsed as canonical.

The fix: Standardize on the directory layout (it scales) and update Project Structure to show migrations/ with at least an index.ts. While there, document the missing CLI for inspecting _modelenceMigrations, the manual cleanup procedure for failed migrations, and the rollback story — all currently absent.

12. Auth Overview omits GitHub Sign-In, the most discoverable provider page on the site (significant)

Location: /authentication

Problem: The Overview lists Google Sign-In as a provider but does not list GitHub Sign-In, despite /authentication/github-sign-in being a complete, well-documented page that's listed in llms.txt. The GitHub page is also more thorough than the Google one (it documents MODELENCE_AUTH_GITHUB_CLIENT_SCOPES; the Google page has no scopes equivalent).

Consequence: A developer auditing whether Modelence supports their auth requirements lands on the Overview, sees only Google, and may rule the platform out. Agents indexing the auth surface miss GitHub entirely.

The fix: Add a [GitHub Sign-In](/authentication/github-sign-in) bullet to the Overview's provider list. While editing, document the Google scopes env var and the account-merging behavior when an OAuth email matches an existing email/password account (currently unspecified for both providers).

13. WebSockets "API Reference" links are monorepo source paths, not docs URLs (significant)

Location: /websockets (API Reference section)

Problem: All six API Reference links use relative paths into the package source: [types.ts:5](../../packages/modelence/src/websocket/types.ts#L5), [serverChannel.ts:11](../../packages/modelence/src/websocket/serverChannel.ts#L11), etc. Rendered against https://docs.modelence.com/websockets, those resolve to nonexistent doc URLs (e.g., https://docs.modelence.com/packages/modelence/src/websocket/types.ts). None of the symbols (ServerChannel, ClientChannel, startWebsockets, joinChannel, leaveChannel, broadcast) are in llms.txt.

Consequence: Six dead links on a comprehensive page. There is no working path from the WebSockets guide to a type signature anywhere on the docs site.

The fix: Either point the links at GitHub permalinks (e.g., https://github.com/modelence/modelence/blob/main/packages/modelence/src/websocket/types.ts#L5) or — better — add the WebSocket primitives to the auto-generated TypeDoc output and link to /api-reference/modelence/server/classes/ServerChannel etc.

14. Auto-generated Store reference is missing methods that the guides actively use (significant)

Location: /api-reference/modelence/server/classes/Store

Problem: The reference enumerates ~14 methods (aggregate, bulkWrite, countDocuments, deleteMany, deleteOne, distinct, extend, fetch, findById, findOne, findOneAndDelete, findOneAndReplace, findOneAndUpdate, ...). The guides use at least five additional methods: Store.watch() (Live Queries — explicitly noted as introduced in 0.15.1), Store.vectorIndex() (Indexes page + Voyage AI), Store.vectorSearch() (Voyage AI), Store.requireById(), and Store.upsertOne() (Stores guide).

Consequence: Five publicly promoted methods have no type signature, no parameter documentation, no return type. Developers and agents have no canonical contract for arguably the most important Store features (live data, vector search).

The fix: Regenerate the TypeDoc with full coverage and verify against the public exports of modelence/server. Add a CI step that fails when a guide references a method missing from the reference.

15. Two pages titled "Rate Limiting" with no disambiguation (significant)

Location: /rate-limiting and /authentication/rate-limiting

Problem: Both pages have H1 = # Rate Limiting. One covers built-in auth defaults (concrete numbers like "50 attempts per 15 minutes"); the other covers custom rate-limit definitions. They link to each other, but neither title disambiguates.

Consequence: Search results show two identical titles. Developers click into one, don't find what they want, and don't realize a separate page exists. Agents indexing by title see a duplicate.

The fix: Rename one page (e.g., "Custom Rate Limiting" for /rate-limiting and "Auth Rate Limiting" for /authentication/rate-limiting) so titles match what each page actually documents.

16. Auth Hooks page documents nine hook names with zero code examples (significant)

Location: /authentication/hooks

Problem: The page describes validateSignup, validateProfileUpdate, generateHandle, onAfterLogin, onAfterSignup, onAfterEmailVerification, onLoginError, onSignupError, onEmailVerificationError — but none of these symbols appear in llms.txt or the API reference. The page also asserts hooks are configured "through the AuthConfig type" but no worked example anywhere in the docs shows registration. The visible examples (/authentication, /authentication/email-verification, /authentication/password-reset) all attach config under email: of startApp(), not an auth: key.

Consequence: Hooks are unusable from the docs alone. A developer wanting to run custom logic on signup has nine names, no signatures, and no example of how to register one.

The fix: Add a single concrete startApp({ auth: { validateSignup: async (data) => { ... } } }) example, link each hook name to its TypeDoc page, and surface AuthConfig in llms.txt.

17. Data API "sandbox" is served over HTTP, not HTTPS (significant)

Location: /data-api (Sandbox section)

Problem: The page invites developers to "Try the Data API live at data-api-demo.modelence.app to explore all endpoints and test operations." The link is http://, not https://. The Data API requires API keys (apiKey header or Bearer tokens) for any meaningful operation.

Consequence: A developer pasting a real API key into a try-it surface served over plaintext leaks it to anyone on the network path. Even if the demo accepts a sandbox key, the muscle memory is bad and the example is a security regression for a developer-tools product.

The fix: Serve the demo over HTTPS, update the docs link, and add a callout reminding users not to paste production API keys into the sandbox.

18. DATA_API_KEY rendered as DATA\_API\_KEY in prose (minor)

Location: /data-api (Authentication section)

Problem: The rendered text shows DATA\_API\_KEY environment variable — escaped underscores leaking through Mintlify's markdown processing. This is the exact env var name a developer needs to copy.

Consequence: Developers may paste DATA\_API\_KEY into their .modelence.env, which won't be picked up. Minor friction, real bug.

The fix: Wrap env var names in backticks (`DATA_API_KEY`) consistently so the underscore-escape doesn't trigger.

19. Voyage AI tutorial uses inconsistent heading levels (minor)

Location: /voyage-ai

Problem: Step 1 is ### Step 1: Install Dependencies (H3) while Steps 2–7 use ## Step N: (H2).

Consequence: The auto-generated table of contents skips Step 1 (or renders it as a sub-step of Quick Start). Visual flow of the tutorial is broken.

The fix: Change ### Step 1 to ## Step 1.

What they do well

• Live Queries page is genuinely strong — explicit error strings, version requirements, pitfalls section calling out the change-stream cleanup foot-gun, and a clear MongoDB replica-set caveat.
• Configuration page has an unambiguous type table and an explicit warning that 'secret' cannot be isPublic: true.
• Built-in auth rate limits are documented with concrete numbers (signup 20/15min, login 50/15min, verification 1/60s) — better than most platforms in this category.

Top 3 recommendations

1. Fix the LLM-facing surface first. Ship a real openapi.json (or remove it from llms.txt), regenerate llms.txt from actual TypeDoc output so it lists all 30+ public symbols, and add a CI link checker that fails on internal 404s.
2. Reconcile Store guides with the Store API reference. Either accept { id: ... } selectors and document them, or rewrite every guide example to use _id / findById. Then add the missing methods (watch, vectorIndex, vectorSearch, requireById, upsertOne) to the auto-generated reference.
3. Standardize on .modelence.env and fix the broken markdown fence in /core-concepts/environment-and-setup. That single page should become the canonical, complete env-var table — every MODELENCE_* variable used anywhere in the docs, with type, default, and consuming module.