Scrapfly

About Scrapfly

Scrapfly provides a web scraping and browser automation platform accessed through a single API key. Its core offering is a Web Scraping API that returns rendered page content over HTTP with anti-bot bypass across more than 20 vendors (including Cloudflare, DataDome, Akamai, Kasada, F5, and AWS WAF), proxy rotation across 190+ countries, and JavaScript rendering. Additional products include a Cloud Browser API for driving remote stealth Chromium sessions over CDP with Playwright, Puppeteer, or Selenium; a Crawler API for full-site crawls with depth control and webhook streaming; a Data Extraction API that turns HTML into structured JSON using pre-trained models, CSS/XPath templates, or LLM prompts; and a Screenshot API for full-page or viewport captures with diff monitoring.

The platform also offers an AI Browser Agent for natural-language browser automation compatible with Browser Use, Stagehand, and Vibium, a hosted MCP Server exposing scrape, screenshot, extract, crawl, and account tools to MCP clients such as Claude, Cursor, Cline, and Windsurf, a command-line interface, a Proxy Saver tool for reducing bandwidth use with existing proxy providers, an Unblocker endpoint for anti-bot bypass, and a free Antibot Detector for identifying which anti-bot vendor a site uses. SDKs are available for Python, TypeScript, Go, Rust, and Scrapy, along with integrations for Zapier, Make, n8n, LlamaIndex, LangChain, and CrewAI.

Scrapfly is aimed at developers and businesses building data pipelines, AI agents, or automation that requires scraping websites protected by anti-bot systems, including use cases in e-commerce, real estate, social media, travel, SEO, and lead generation. The service offers 1,000 free credits with no credit card required, and bills on a pay-on-success basis where failed anti-bot bypass attempts are not charged.

Key features

• Anti-bot bypass across 20+ vendors including Cloudflare, Akamai, DataDome, and PerimeterX
• Cloud browser sessions with full Playwright/Puppeteer/Selenium compatibility over CDP
• Site-wide crawling with BFS/DFS depth control and webhook streaming
• AI-powered data extraction using pre-trained models, CSS/XPath templates, or LLM prompts
• Full-page screenshot capture with diff monitoring and layout-change alerting
• Hosted MCP server connecting Claude, Cursor, Cline, and other MCP clients to scraping tools

Frequently asked questions