Cookies disclaimer

I agree Our site saves small pieces of text information (cookies) on your device in order to deliver better content and for statistical purposes. You can disable the usage of cookies by changing the settings of your browser. By browsing our website without changing the browser settings you grant us permission to store that information on your device.

Usage & Analytics

Usage & Analytics

This section of the Web Crawler support documentation shows you how to understand both the usage limits and the project analytics pages.



Usage & Limits

Our web crawler subscriptions are primarily capped by the number if URLs that you can crawl per month. The URL usage is reset at the beginning of your billing cycle each month. If you hit your limit of the pricing plan you are on, you will be unable to crawl any more URLs until the first day of your next billing cycle.

The interface we provide within the software shows you how many URLs fall into each category:

  • Used
  • Remaining
  • Scheduled


Project Analytics

The project analytics pages can be accessed from within a project, this provides top level information about the crawls such as the status codes and file types. This shows you where you are spending your URLs, whether on 301’s, HTML files or images.

If you want to know what the data within the tables and charts of these pages means, or how we define it, you can look them up in the web crawler data section.