Your AI team, working in WebCrawlerAPI
WebCrawlerAPI provides crawling, page scraping, AI web agents, and monitored website feeds through a unified API.
Start your companyWhat your team can do in WebCrawlerAPI
10 tools to work with, connected with an API key.
Start a metered asynchronous crawl over a site and return its job ID. The crawl may visit and charge for up to items_limit pages; use GET_CRAWL_JOB to monitor it.
Create a recurring website-change feed. Creation immediately schedules an initial crawl that may consume metered credits, and future scheduled runs may also consume credits; delete the feed when it is no longer needed.
Permanently cancel a feed so it cannot be resumed. The feed is removed from feed listings, but retained history may remain directly readable for 30 days.
Get a crawl job's status, configuration, per-page results, content URLs, errors, and observed costs. Respect recommended_pull_delay_ms when polling.
Get one feed's configuration, lifecycle status, and recent run history, including per-run crawl counts and cost. Canceled feeds remain directly readable while retained by the provider.
Return current spendable balance plus request count and USD usage for a date range. The connected API key is used for both organization routes.
Return one page of detected feed changes as structured JSON, with an opaque continuation cursor for older pages.
List active and paused feeds for the connected organization, newest first. Logically canceled feeds are omitted.
Scrape one web page synchronously and return requested content or structured extraction. This consumes metered credits; supplying a prompt adds $0.002 per request.
Pause an active feed's future scheduled runs or resume a paused feed. Pausing does not stop a run that has already started, and resuming schedules future crawls that may consume metered credits.
Nothing matches that.
More in AI web scraping
See them all βZenRows is a web scraping API allowing developers to bypass CAPTCHAs and blocks, gather structured data from dynamic websites, and quickly integrateβ¦
14 toolsβ AAgentQL is a suite of tools designed to connect AI agents to the web, enabling web interaction and structured data extraction through a specializedβ¦
3 toolsβ BBotster automates web data collection through configurable bots, job lifecycle management, and structured run results.
11 toolsβ CCloro provides structured data from Google Search and leading AI assistants, with synchronous extraction and asynchronous task APIs.
14 toolsβ CCoresignal provides agentic search and real-time employee data from public web sources for research, recruiting, and sales workflows.
3 toolsβ CCrawl4AI Cloud provides web scraping and search, structured extraction, asynchronous crawl and recipe jobs, page monitoring, pricing and leadβ¦
51 toolsβ