Gumloop

Gumloop Documentation Audit

The docs cover a broad surface (workflows, agents, MCP, SDKs, OAuth) and ship an llms.txt, but the API reference has unschematized fields, placeholder SDK pages, and three different pagination contracts across sibling list endpoints — exactly the failure modes that break agent-driven integration.

1. Getting Started defaults to an insecure auth method without warning (significant)

Location: /api-reference/getting-started, /api-reference/authentication

Problem: getting-started.md states: "The default method is to include your API key as a query parameter in the URL." The dedicated authentication.md page does note both methods are equivalent ("Both grant the same permissions and work on every endpoint — pick whichever matches how your app gets the credential") and leads with the bearer-token approach. But there is no warning anywhere that query-string credentials end up in access logs, browser history, and Referer headers, and Getting Started — the page most developers actually copy from — pushes this as "the default method."

Consequence: A developer (or coding agent) following Getting Started will copy a query-string call and ship it. The API key then leaks into every request log along the path. The dedicated Authentication page solves this if you find it, but the first-hit page silently encourages the wrong choice.

The fix: Either retire the query-string method or, at minimum, lead Getting Started with bearer-token auth and add an explicit warning ("API keys passed as query parameters appear in logs and browser history — prefer the Authorization header"). Cross-link to authentication.md.

2. Create-agent tools array has no schema (critical)

Location: /api-reference/agents/create-agent

Problem: The tools field is documented as array of object with: "Each tool is an object whose shape depends on the tool type." No discriminator, no enumeration of valid tool types, no per-type schema, no example. The resources field has the same problem ("Resources attached to the agent. items: object").

Consequence: The single most important parameter for creating an agent is functionally undocumented. A developer cannot construct a valid request from this reference, and an LLM generating client code will hallucinate plausible-looking tool shapes that the API will reject.

The fix: Publish the discriminator (e.g. {type: "mcp" | "skill" | "code" | …}), one object schema per tool type, and at least one working JSON example per type in the page body. Same treatment for resources.

3. Run-history endpoint has a hard 10-row cap with no pagination (critical)

Location: /api-reference/getting-automation-details/retrieve-run-history

Problem: The endpoint description reads: "Returns the 10 most recent runs." The response shape is documented as "up to 10 runs per saved item." There is no cursor, no limit, no before/after parameter, and no documented alternative for retrieving older runs.

Consequence: Anyone building an admin dashboard, audit log, billing reconciliation, or failure-analysis tool against Gumloop has access to the last 10 runs and nothing more. There is no escape hatch documented. For an automation that runs hourly, that's less than half a day of history; for one that runs per webhook, it can be minutes. This is worse than the pagination drift in Finding 4 — there is no pagination contract at all.

The fix: Add cursor or offset pagination and document the actual retention window for run history. If 10 is truly the system cap, say so explicitly and direct developers to whatever the alternative is (webhook delivery into their own store, an export endpoint, etc.).

4. Pagination contract is inconsistent across list endpoints (significant)

Location: /api-reference/agents/list-agents vs /api-reference/skills/list-skills

Problem: list-skills offers a working cursor parameter, a page_size (1–100), a sort_order enum, and returns a real next_cursor. list-agents documents next_cursor as: "Reserved for future cursor-based pagination. Currently always null. example: null." Two sibling list endpoints in the same API have fundamentally different pagination semantics, with no shared note explaining why.

Consequence: A client that paginates skills correctly will silently fetch only the first page of agents and assume it has them all (because next_cursor is always null). For an org with more than the default page of agents, this is a data-completeness bug, not a docs nit.

The fix: Either ship cursor pagination on list-agents or document the actual cap (and how to fetch all agents — by team filter, by created_at, etc.). A shared "Pagination" page that both endpoints link to would prevent further drift.

5. OAuth gumloop_api scope silently fails for non-Pro users (significant)

Location: /api-reference/oauth

Problem: The scopes table notes: "The gumloop_api scope requires the authorizing user to be on the Pro plan or above. Token exchange will fail if the user's account does not meet this requirement." There is no example error response, no error code, no UX guidance for handling this case, and no mention of it on the authorization-step section above the scopes table.

Consequence: An OAuth app integrator builds and ships their integration, then watches free-tier users get an opaque token-exchange failure mid-flow. Without an error code, the integrator can't distinguish this failure from any other token error and can't show users a useful "upgrade required" message.

The fix: Document the exact error payload (HTTP status, error code, error description), surface it next to the authorization step (not only in the scopes glossary), and recommend a UX pattern for downgrading the scope set or prompting upgrade.

6. OAuth client registration is manual and email-gated (significant)

Location: /api-reference/oauth

Problem: "OAuth client registration is currently invite-only. Email support@gumloop.com with your app name, use case, redirect URI(s), and logo. We'll review and reach out with a client_id." This isn't surfaced on the API overview — a developer only discovers it after reading deep into the OAuth page.

Consequence: A developer evaluating Gumloop to add a "Sign in with Gumloop" button cannot complete a proof-of-concept on their own — every test app, every redirect URI rotation, and every staging client is blocked on a human email loop. This dramatically narrows OAuth adoption, and it isn't flagged anywhere a developer would see it before investing in the integration.

The fix: Add an "invite-only" badge to the OAuth entry in the API index. Document expected turnaround time, the review criteria, and ideally a self-serve sandbox client option for development. If invite-only is intentional long-term, say so; if it's temporary, give a target date.

7. Python and JavaScript SDK reference pages are placeholders (significant)

Location: /api-reference/sdk/python, /api-reference/sdk/javascript

Problem: Both SDK pages have effectively the same one-sentence intro ("For convenience, we have created a Gumloop … SDK to more easily perform operations like starting an automation and retrieving outputs.") followed only by an "## Installation" heading. No install command, no auth setup, no method reference, no example.

Consequence: A developer arriving from the "API Reference" tile on the docs home cannot install the SDK from these pages, let alone call it. Agents indexing the SDK pages get nothing to work with, so they fall back to raw HTTP — which is fine, except the SDK page advertises convenience and delivers a stub.

The fix: Fill in the install command, an authenticated client construction example, and at least one end-to-end call (start automation → poll → retrieve output). Or remove the pages and link directly to the package READMEs.

8. File upload/download pages have no inline content (significant)

Location: /api-reference/file-operations/upload-file, /api-reference/file-operations/download-file

Problem: Both pages render with only a title and "## OpenAPI" — no human-readable description, no size limit, no MIME-type list, no example request, no auth note. The Interfaces page mentions a 200 MB file-upload max for forms, but the API file-upload endpoint says nothing about its own limits.

Consequence: A developer integrating file flows over the API can't know the max upload size, allowed types, or response shape from these pages. Agents see two near-empty pages and have nothing to extract.

The fix: Render the OpenAPI section inline (request/response, parameters, content types) and add a prose note covering size limits, supported formats, and how the uploaded file_id is consumed downstream (workflows? agent resources? both?).

9. Custom MCP servers have no approval gate and the docs hand-wave the mitigation (significant)

Location: /nodes/mcp/custom_mcp_servers

Problem: The Security Considerations section states: "All tools exposed by your MCP server are immediately available to the AI. There are no approval prompts before tool execution. Use appropriate authorization scopes to limit access." That is the entire mitigation guidance for what is, in practice, an arbitrary-tool-execution surface. "Appropriate authorization scopes" is never defined, no example MCP server-side guard is given, and there's no link to MCP server-hardening guidance.

Consequence: A team enabling a custom MCP server gives the agent immediate, un-prompted access to every tool that server exposes. The docs acknowledge this exists but offload the safety work to the integrator with one vague sentence. Anyone wiring up an MCP server that touches production resources (database writes, GitHub PRs, payment APIs) is one bad prompt away from a real incident.

The fix: Spell out concrete patterns: per-tool API tokens with least-privilege scopes, read-only vs write-mode separate servers, server-side allow-lists, audit logging requirements. Link out to MCP server-hardening guidance if Gumloop won't host its own. Consider a UI-level warning when a user attaches a custom MCP server to an agent.

10. Anthropic MCP custom-header forwarding is a silent footgun (significant)

Location: /nodes/mcp/custom_mcp_servers

Problem: The "Header Handling by Model" table notes that for Anthropic native MCP, custom headers are "Not forwarded," and the prose says: "Anthropic models do not forward custom headers. If your MCP server relies on a custom header (e.g., X-API-Key), use the Access Token / API Key field with a Bearer token instead, or choose OpenAI, Gemini, or Groq." Good that it's documented — but there's no description of the failure mode the developer will actually see (silent 401 from their MCP server? tool calls just stop working? a Gumloop-side error?).

Consequence: A developer who configures X-API-Key and picks Claude Sonnet will see their MCP tools fail intermittently with no obvious cause. Because Anthropic is the marquee model, this is the default landing-spot for many users — and the failure surfaces as "MCP server doesn't work," not "headers were dropped."

The fix: Add the exact end-user error message and where it appears (agent run log, server-side 401, etc.). Consider a UI-level warning when a user pairs a custom-header MCP server with an Anthropic model. Provide an explicit migration snippet showing the same auth re-expressed via the Access Token field.

11. list-servers status enum is not fully enumerated (minor)

Location: /api-reference/mcp/list-servers

Problem: The status field describes "Connection state. connected means the server is ready to accept tool calls; other values (for example unauthenticated, blocked) indicate the user must complete OAuth or the server is otherwise unavailable." for example is doing a lot of work — the full set of values isn't listed.

Consequence: A client building UI or branching logic on status can't enumerate the cases exhaustively. New status values can ship silently and existing clients fall through to a default branch that may not be safe (e.g. treating an unknown state as "connected").

The fix: Replace the example with the actual enum and document each value's meaning. Add a "subject to extension" note and recommend a default-case strategy so clients can be forward-compatible.

12. Getting Started Introduction is a video-script transcript, not documentation (minor)

Location: /getting-started/introduction

Problem: The page reads: "In a more agentic... An AI agent... Agents... Agentic workflows... AI this, agents that. I know you feel like I do. It's all a little overwhelming." and ends "So join me in the next lesson where we build our first agent." This is verbatim narration from a Gumloop University video, not text-shaped documentation.

Consequence: It's the introduction page of Getting Started — the most-linked entry point — and it teaches a reader nothing concrete about Gumloop. Coding agents extracting this page get marketing prose with zero structured information about what a workflow is, what an agent is, or how to install anything.

The fix: Keep the video, but write a text-first intro that defines Workflows vs Agents vs Interfaces vs Hosted Pages, links to the four next steps, and lists prerequisites (account tier, credentials). Use the transcript as a transcript section if you want it preserved.

What they do well

• An llms.txt is published on the docs home and explicitly pitched as the entry point for agent indexing — most peer platforms still don't ship this.
• Interfaces docs are unusually explicit about a real footgun: "The workflow will use the interface user's credentials, not the workflow creator's" — exactly the kind of operational note that prevents support tickets.
• The MCP page enumerates per-model behavior in a clean table (transport, native vs backend connector, header handling), which is the kind of cross-cutting reference most MCP vendor docs lack.

Top 3 recommendations

1. Publish a concrete schema (or at least one full example per type) for the tools and resources arrays in create-agent. This is the single highest-impact reference bug.
2. Fix the pagination story end-to-end: ship cursor pagination on list-agents, add real pagination (or document the cap explicitly) on retrieve-run-history, and put a shared Pagination page in the API overview.
3. Replace the SDK and file-operation placeholder pages with real content, and move the OAuth invite-only friction (and free-tier scope failure) up to the API overview so developers see it before they build.