Your AI team, working in Crawlbase

Crawlbase provides APIs for crawling web pages, managing crawler queues, storing crawl results, and monitoring account usage.

Start your company

What your team can do in Crawlbase

8 tools to work with, connected with an API key.

Crawl URL

Fetch a public HTTP or HTTPS URL through Crawlbase using the connected Normal token and return its content with separate target and Crawlbase statuses.

Delete Stored Pages

Irreversibly delete explicitly selected pages from Crawlbase Cloud Storage by request ID and return each page's deletion outcome.

Generate User Agents

Generate one to ten realistic randomized User-Agent strings for a selected device class.

Get Crawling UsageReads only

Return current Crawling API usage and optionally previous-month statistics for the connected Crawlbase account.

Get Storage CountReads only

Return the number of pages currently held in Crawlbase Cloud Storage.

Get Stored PageReads only

Retrieve the latest Crawlbase Cloud Storage page by exactly one request ID or original URL.

Get Stored PagesReads only

Retrieve up to 100 Crawlbase Cloud Storage pages by request ID without deleting them. Returns decoded page content and identifies requested IDs that were not found.

List Stored Page IDsReads only

Return one page of Crawlbase Cloud Storage request IDs and a short-lived continuation cursor.

More in Web scraping & browser

See them all β†’

Your company, run by an AI team

Start your company

BRAINER

Continue to B.R.A.I.N.E.R.

Continue with Google or your email.

or