Your AI team, working in Spider

Spider provides APIs for web crawling, scraping, search, screenshots, content transformation, AI extraction, and high-concurrency data collection.

Start your company

What your team can do in Spider

15 tools to work with, connected with an API key.

AI Browser

Drive a browser from a natural-language prompt and extract results. Requires AI Studio. Prompts can click, navigate, submit forms, purchase, message, or change accounts, so request those external effects only when intended and authorized.

AI Crawl WebsiteReads only

Crawl pages and use an AI prompt to extract structured data. Requires AI Studio. limit defaults to 1 to bound crawl and model costs; depth 0 removes the depth limit.

AI Discover LinksReads only

Collect links matching a natural-language prompt and return structured context. Requires AI Studio. limit defaults to 1 to bound crawl and model costs; depth 0 removes the depth limit.

AI Scrape PageReads only

Fetch exact pages and use an AI prompt to extract structured data. Requires AI Studio; batched or complex extraction increases model cost.

AI Search WebReads only

Run prompt-driven web search with optional page fetching and structured extraction. Requires AI Studio and defaults to one result without page fetching.

Capture ScreenshotReads only

Render one page and capture a PNG, JPEG, or WebP screenshot with a provider-side response-content cap. Full-page capture is the documented default; returns compact metadata by default, optional base64 content, or an offloaded file in binary mode.

Crawl WebsiteReads only

Crawl pages from one or more starting URLs. Defaults to at most 10 pages and depth 1; raising breadth increases cost and response size. Set billing_mode='unlimited' only for an active Unlimited plan with purchased concurrency.

Discover LinksReads only

Discover URLs from a site without returning full page content. Defaults to 25 pages and depth 1; larger traversal costs more. Set billing_mode='unlimited' only for an active Unlimited plan with purchased concurrency.

Fetch Configured Page

Fetch a domain and path with Spider's saved scraper configuration, discovering and storing one on first use. Defaults to one page; first-time discovery can take 30-120 seconds and incur AI cost.

Get Credit BalanceReads only

Get the connected Spider account's remaining credit balance and documented record metadata.

List Crawl LogsReads only

Return the connected account's crawl and scrape activity from the last 24 hours.

List Scraper DirectoryReads only

Browse public curated scraper configurations with domain, category, text, and page-number filters.

Scrape PagesReads only

Fetch and extract exact pages without following links. Use crawl only for traversal; set billing_mode='unlimited' only for an active Unlimited plan with purchased concurrency.

Search WebReads only

Search the web and optionally fetch result pages. Defaults to 10 results without page fetching or automatic multi-page expansion.

Transform ContentReads only

Convert supplied HTML to markdown, text, or cleaned output without fetching a URL; the total serialized JSON request must not exceed 10 MB.

More in Web scraping & browser

See them all β†’

Your company, run by an AI team

Start your company

BRAINER

Continue to B.R.A.I.N.E.R.

Continue with Google or your email.

or