SEO Spider

Crawl your own site on a regular basis to spot any SEO issues. Find broken links, errors or optimize your site and keep track of the performance.

No items found.


Pricing from
Average rating
(G2, GetApp, Trustradius & Capterra)
No rating available
Free version
Free trial
For businesses
Headquarter country
United Kingdom
Headquarter city
Employee size

+ Free version available
+ Great onsite SEO insights
+ Integrations with Googel products


- Onsite SEO only
- let us know


Get on-site SEO insights with SEO spider from Screamingfrog
No screenshots available.


No items found.

Tool FAQ

What is SEO spider?

How does Screaming Frog work?

Screaming Frog works by crawling and recolecting data from a website, it can crawl from 500 URLs and in its features offers finding broken links, errors or redirects that may affect the SEO performance of the site, as well as extract data, discovering duplicate content, analyze meta data, and more.

How do you activate Screaming Frog?

The first step to start using Screaming Frog is creating a free account to log in to. Then proceed to export the list of subdomains. After doing so, upload the same list to Screaming Frog using the List Mode. When the spider finishes crawling, the user will be able to see the data collected and make SEO decisions with it.

Is Screaming Frog free?

Screaming Frog is originally free but the options with this version are limited, an example of this is that the crawl limit is only 500 URLs, so the user can purchase a license for the price of £ 149.00 per year to have acces to more advanced features like unlimited crawling.

What is SEO crawling?

Crawling can be defined as the tracking or route that a crawler does to read and analyze the code and content of a web. A crawler does this by traveling from page to page following the links that finds trough the way. The most known example of these is the Google’s crawler.

Is Screaming Frog good?

Screaming Frog is all in one solution for spotting any SEO issues, it informs of the on-page and technical details, and also helps finding errors, broken links, keep track, and optimze the performance of a website. There is no doubt that SEO Spider is used by various of the biggest brands of the web.

SEO Spider Features

Screaming Frog’s SEO Spider is one of the industry’s most popular website crawlers and offers a free and paid version. When you utilize this tool, you'll be able to use your own powerful, flexible site crawler that allows you to see and analyze your site in real-time.

This will allow you to gather key information about on-site data that may be impacting your SEO and user experience. You'll be able to use this data to make informed decisions on different site changes that will impact your site's performance.

You can crawl 500 URLs with the free version, but the paid version will allow you to access unlimited URLs if you have a larger site or want more advanced features, such as scheduling.

While there are many features, it's best to break them down into three main groups:

Page Insights

As your site grows, you'll find that some of the pages that you created may no longer be working or may not have the best SEO measures in place. Once you go from a handful of pages to hundreds of pages, it can be very difficult to keep track of everything without using a site crawler.

You can gather great page insights using SEO Spider, such as:

You can also generate XML sitemaps that allow you to alert Google and other search engines about the pages on your site. This is a great option if your site is not being crawled, and it’s done automatically for you based on what the crawler finds.

Broken Link and Error Finder

One of the biggest issues that site owners have is that over time, the links that they have on their sites may break. If these are external links, the issue is not that major, but when it involves internal links, this can lead to negative impacts on your search engine optimization.

Using SEO Spider, you're able to find broken links and errors that would otherwise go missed by most site owners.

You can use this information to find redirect issues that may be occurring on your site. A lot of site owners use .htaccess files to create redirects, or they may use plugins to help control the redirects on a site.

Over time, if multiple people start updating the site and making changes, it's not uncommon for these redirects to start creating errors. All it takes is one person to create the wrong redirect or to create a redirect loop that continually tries to redirect to the same page for errors to pop up.

You will be able to find all of these issues and correct them, which can have a major impact on not only your search engine optimization efforts, but also your user experience.

Validation and Checks

Validation and checks are part of the entire program, but there are some features that are more important in this respect than others. The main validation and checks that are offered include:

SEO Spider incorporates a lot of additional features into their premium version of the platform. These features are great for larger site owners, and they also include the ability to schedule and customize the way your site is crawled.

You'll be able to integrate this data into your Google Analytics and Search Console. You'll also be able to integrate with PageSpeed Insights to better understand how the changes you made have impacted your site.

SEO Spider also gives you the ability to store and view the HTML that is rendered on your site, search for custom source codes and also create custom robot.txt for your site.

If JavaScript rendering does occur, the platform is able to render it and extract information from these pages. This is very important for dynamic pages that may change and for user generated content.

SEO Spider is a tool that is used by some of the largest brands in the world to make sense of their site’s performance and data.

Try out tool↗︎

Stay updated on the latest tools

Receive the latest tools and blogs via email. Your data will never be shared.

Thanks for subscribing! You'll receive a confirmation email soon.
Oops! Something went wrong while submitting the form.
new service

Want to get more organic traffic?

Let us review your website!