Your AI team, working in Scrapfly

Scrapfly is a web scraping API that enables developers to extract data from websites efficiently, offering features like JavaScript rendering, anti-bot protection bypass, and proxy rotation.

Start your company

What your team can do in Scrapfly

12 tools to work with, connected with an API key.

Capture Screenshot Metadata (HEAD)

Tool to capture screenshot metadata without downloading the image body. Use this for async screenshot workflows where you need the URL to retrieve the image later. Returns the screenshot URL in response, saving bandwidth compared to full screenshot retrieval.

Capture Website Screenshot

Capture a full-page, viewport, or selected-element screenshot of a website, with options such as custom resolution and dark mode. Returns a downloadable screenshot image.

Create Scrapfly Crawler

Create a web crawler to recursively crawl pages from a starting URL, with configurable limits and content formats. Returns a crawler UUID for tracking progress.

Extract Structured Data

Tool to extract structured data from HTML or other content using AI models, LLM prompts, or custom templates. Use when you need to parse web pages or documents into structured JSON data. Supports predefined extraction models for common types (articles, products, events) or custom extraction via prompts/templates.

Get Crawler ArtifactReads only

Tool to download crawler artifact files in WARC or HAR format. Use when you need to retrieve the complete crawl results as an archive file. WARC format is recommended for large crawls as it includes gzip compression.

Get Crawler ContentsReads only

Tool to retrieve extracted content from crawled pages. Supports multiple output formats including markdown, text, HTML, and JSON. Use when you need to access the actual content extracted during a crawl, with optional filtering by URL and format selection.

Get Crawler StatusReads only

Tool to get the current status of a crawler including progress, pages crawled, and completion state. Use for polling workflow to monitor crawl progress.

Get Crawler URLsReads only

Retrieve discovered and crawled URLs from a crawler, optionally filtered by status. Supports pagination for large result sets.

Get Scrapfly Account InformationReads only

Retrieve Scrapfly account information, including subscription, usage, billing, and project settings. Use it to review account status and usage limits.

Scrape With PUT

Tool to scrape web pages using PUT method with body payload. Use when the target API requires PUT requests with data in the request body. Forwards PUT request with custom body to the target URL. If not specified, content-type defaults to application/x-www-form-urlencoded.

Scrapfly Scrape

Request a web scrape with options such as JavaScript rendering, proxy location, and extraction. The action requires a Bearer API key in metadata headers and reads results from the response's data wrapper.

Scrapfly Scrape POST

Tool to scrape web pages using POST method to send data in the request body. Use when you need to scrape endpoints that require POST requests, such as form submissions or APIs that expect data payload.

More in Web scraping & browser

See them all β†’

Your company, run by an AI team

Start your company

BRAINER

Continue to B.R.A.I.N.E.R.

Continue with Google or your email.

or