WebCrawlerAPI provides crawling, page scraping, AI web agents, and monitored website feeds through a unified API.

Use it from your agent

One ClawLink account serves every agent you run. Pick the guide for yours, or connect WebCrawlerAPI once and call the same tools from all of them.

Setup

Connect WebCrawlerAPI through ClawLink's hosted setup.

Auth: Paste one API key. The credential stays on the ClawLink server; your agent holds only a ClawLink key you can revoke.
Where: the ClawLink dashboard, or ask your agent to connect it.

WebCrawlerAPI exposes 10 tools through ClawLink. The 9 most used are listed here; your agent sees all of them.

Tool	What it does	Kind
`webcrawlerapi_get_crawl_job`	Get a crawl job's status, configuration, per-page results, content URLs, errors, and observed	Read
`webcrawlerapi_get_feed`	Get one feed's configuration, lifecycle status, and recent run history, including per-run crawl	Read
`webcrawlerapi_get_organization_costs`	Return current spendable balance plus request count and USD usage for a date range	Read
`webcrawlerapi_list_feed_changes`	Return one page of detected feed changes as structured JSON, with an opaque continuation cursor	Read
`webcrawlerapi_list_feeds`	List active and paused feeds for the connected organization, newest first	Read
`webcrawlerapi_scrape_page`	Scrape one web page synchronously and return requested content or structured extraction	Read
`webcrawlerapi_create_crawl`	Start a metered asynchronous crawl over a site and return its job ID	Write
`webcrawlerapi_create_feed`	Create a recurring website-change feed	Write
`webcrawlerapi_set_feed_state`	Pause an active feed's future scheduled runs or resume a paused feed	Write