Cookies disclaimer

I agree Our site saves small pieces of text information (cookies) on your device in order to deliver better content and for statistical purposes. You can disable the usage of cookies by changing the settings of your browser. By browsing our website without changing the browser settings you grant us permission to store that information on your device.



How to Crawl Websites

Our web crawler allows you to crawl websites in several ways, the guides in this section provide step by step instructions on how to crawl. Before you can crawl anything, you will need to create / add a project, so you may need to check out those guides first.

Crawling websites couldn’t be easier than with Raptor! The steps between signing up to Raptor and downloading a report are limited to a few clicks and some copying and pasting. We have made web crawling simple with our easy to use and intuitive interface.

If you do get stuck trying to crawl something, you can follow the relevant guide in this section. Within this section we describe how to…

  • Crawl a website
  • Crawl all your sites
  • Crawl all the sites in a project
  • Crawl all your competitors

As we add functionality to our software over time, we will add how to guides to this library.