Screaming Frog SEO Spider

About Screaming Frog SEO Spider

Screaming Frog SEO Spider is a website crawler for Windows, macOS, and Linux that audits sites for technical SEO and UX issues. It crawls a site to find broken links and server errors, audit redirects and redirect chains, analyze page titles and meta descriptions, discover duplicate content, review robots.txt and meta robots directives, generate XML sitemaps, and extract custom data using CSS Path, XPath, or regex. It can render JavaScript-heavy sites via an integrated Chromium engine, visualize site architecture, schedule recurring crawls, compare crawls over time, and integrate with Google Analytics, Search Console, and PageSpeed Insights APIs.

The tool is aimed at SEOs and agencies performing technical SEO site audits. A free version is available for crawls up to 500 URLs; a paid licence costing £199 per year removes the URL limit and adds features including near-duplicate content detection, structured data validation, accessibility auditing, forms-based authentication, AI crawling with OpenAI and Gemini, and free technical support. Licences last one year and require renewal.

Key features

• Crawls websites to identify over 300 SEO issues, warnings, and opportunities
• Finds broken links (404s) and server errors with bulk export options
• Audits redirects, redirect chains, and loops
• Analyses page titles and meta descriptions for length, missing, or duplicate issues
• Renders JavaScript-heavy sites via integrated Chromium to crawl frameworks like Angular, React, and Vue.js
• Extracts custom data using XPath, CSS Path, or regex

Frequently asked questions