SEO Spider

Crawl your own site on a regular basis to spot any SEO issues. Find broken links, errors or optimize your site and keep track of the performance.

No items found.


Pricing from
Average rating
(G2, GetApp, Trustradius & Capterra)
No rating available
Free version
Free trial
For businesses
Headquarter country
United Kingdom
Headquarter city
Employee size

+ Free version available
+ Great onsite SEO insights
+ Integrations with Googel products


- Onsite SEO only
- let us know


Get on-site SEO insights with SEO spider from Screamingfrog
No screenshots available.


No items found.

Tool FAQ

No items found.

SEO Spider Features

Screaming Frog’s SEO Spider is one of the industry’s most popular website crawlers and offers a free and paid version. When you utilize this tool, you'll be able to use your own powerful, flexible site crawler that allows you to see and analyze your site in real-time.

This will allow you to gather key information about on-site data that may be impacting your SEO and user experience. You'll be able to use this data to make informed decisions on different site changes that will impact your site's performance.

You can crawl 500 URLs with the free version, but the paid version will allow you to access unlimited URLs if you have a larger site or want more advanced features, such as scheduling.

While there are many features, it's best to break them down into three main groups:

Page Insights

As your site grows, you'll find that some of the pages that you created may no longer be working or may not have the best SEO measures in place. Once you go from a handful of pages to hundreds of pages, it can be very difficult to keep track of everything without using a site crawler.

You can gather great page insights using SEO Spider, such as:

You can also generate XML sitemaps that allow you to alert Google and other search engines about the pages on your site. This is a great option if your site is not being crawled, and it’s done automatically for you based on what the crawler finds.

Broken Link and Error Finder

One of the biggest issues that site owners have is that over time, the links that they have on their sites may break. If these are external links, the issue is not that major, but when it involves internal links, this can lead to negative impacts on your search engine optimization.

Using SEO Spider, you're able to find broken links and errors that would otherwise go missed by most site owners.

You can use this information to find redirect issues that may be occurring on your site. A lot of site owners use .htaccess files to create redirects, or they may use plugins to help control the redirects on a site.

Over time, if multiple people start updating the site and making changes, it's not uncommon for these redirects to start creating errors. All it takes is one person to create the wrong redirect or to create a redirect loop that continually tries to redirect to the same page for errors to pop up.

You will be able to find all of these issues and correct them, which can have a major impact on not only your search engine optimization efforts, but also your user experience.

Validation and Checks

Validation and checks are part of the entire program, but there are some features that are more important in this respect than others. The main validation and checks that are offered include:

SEO Spider incorporates a lot of additional features into their premium version of the platform. These features are great for larger site owners, and they also include the ability to schedule and customize the way your site is crawled.

You'll be able to integrate this data into your Google Analytics and Search Console. You'll also be able to integrate with PageSpeed Insights to better understand how the changes you made have impacted your site.

SEO Spider also gives you the ability to store and view the HTML that is rendered on your site, search for custom source codes and also create custom robot.txt for your site.

If JavaScript rendering does occur, the platform is able to render it and extract information from these pages. This is very important for dynamic pages that may change and for user generated content.

SEO Spider is a tool that is used by some of the largest brands in the world to make sense of their site’s performance and data.

Try out tool↗︎

Stay updated on the latest tools

Receive the latest tools and blogs via email. Your data will never be shared.

Thanks for subscribing! You'll receive a confirmation email soon.
Oops! Something went wrong while submitting the form.
new service

Want to get more organic traffic?

Let us review your website!