Trawl docs
Guides

Build with Trawl.

Five chapters from your first job to a production extraction pipeline with signed webhooks. Each chapter assumes you've finished the previous one but cross-references back, so jumping in mid-series works.

Looking for the full HTTP API instead? See API reference. In a hurry? The quickstart gets you to a first result in a few minutes.

Chapters

The series

01 · Foundations

Workspace boundary, enabling the trawl service, minting a scoped API key, the auth header, and the failure modes you hit first.
Read more

02 · Your first extraction

Create a job with a schema and a description, poll it to completion, read the result off the job row.
Read more

03 · Schema design

Top-level object rule, nullable fields, lists, enums, and writing descriptions the agent can act on.
Read more

04 · Webhooks

Create a webhook, verify HMAC signatures, understand the one-attempt delivery model and auto-disable.
Read more

05 · Operations

Timeouts and budgets, rate limits, what retries and what does not, job + delivery logs, cancel semantics, troubleshooting.
Read more

Mental model

Schema in → JSON out

Trawl is an asynchronous AI web-extraction service. You submit a job with a JSON Schema (the shape you want back) and a natural-language description (what to find); an agent driving a real headless browser searches, reads, and aggregates across pages, then returns a schema-validated JSON object. Seed URLs are optional — without them the agent finds sources itself. You read the result by polling GET /v1/workspaces/:workspace_id/jobs/:id, or have it pushed to an HMAC-signed webhook, or both.

What Trawl is not: a raw scraping proxy or an HTML-fetching API. You never see pages, selectors, or markup — only the structured data your schema asked for. And it's not for content behind a login: extraction covers the public web.

Examples in these guides are curl and plain fetch so they work from any stack. If you're on Node or TypeScript, @productcraft/trawl wraps the same endpoints in a typed client — new Trawl({ auth }).client.POST(...) with request and response types generated from the OpenAPI spec. Start with 01 · Foundations, or read Concepts first for the vocabulary.