Cookies disclaimer

I agree Our site saves small pieces of text information (cookies) on your device in order to deliver better content and for statistical purposes. You can disable the usage of cookies by changing the settings of your browser. By browsing our website without changing the browser settings you grant us permission to store that information on your device.

Non-Indexable Data


Values Returned
Non-Indexable Resources
Examples of How Indexation Data is Used
- Example 1: Quickly Find Resources That Won’t Be Indexed by Google
- Example 2: Identify Follow Internal Links to Non-Indexable Resources
Related Content
Benefits of Our Data



HTML/Text pages that are either restricted by a robots.txt or meta robots tag from being indexed or  have a status code other than 200. Non-indexable, means a resource that Google will be unlikely to index and hence show in the SERPs (Search Engine Results Page).


Values Returned

Within the column for ‘non-Indexable’, the two options or the values we would return in the fields / cells are as follows:






Non-Indexable Resources

A resource is deemed to be non-indexable if any of the following are true:

  • the URL disallowed by the robots.txt
  • The URL contains the NOINDEX tag
  • The URL returns a status code of anything other than ‘200’
  • URL has at no follow internal links from and indexable page

This does not mean that a resource is not indexed by Google, as it may be indexed for any number of reasons. Rather, this is an indication that there may be a problem either now or in the future with the indexation of a specified URL.

If you want these pages or resources to be indexed, it is worth looking at the reason why they might not be.

There may be other reasons preventing a page from being indexed such as a penalty from Google, which we do not check for.


Examples of How Indexation Data is Used

There are various reasons why you would want to look at Indexation Data, we have set out some examples below.


Example 1: Quickly Find Resources That Won’t Be Indexed by Google

Filtering for ‘False’ within the ‘non-indexable’ column will show you are resources that are not indexable. This can often immediately highlight errors where they exist, for example; the home page or product pages should always be indexable.


Example 2: Identify Follow Internal Links to Non-Indexable Resources

Non-Indexable resource (URLs with the value of ‘False’ in the ‘non-indexable’ column) should not have ‘follow’ internal links pointing to them. You can see how many follow links a URL has in the column header ‘follow in link’, a non-indexable resource should have none.

This leaks authority to pages that do not require it, adding the ‘Nofollow’ tag to all internal links pointing to a resource will resolve this issue.


Related Content

The list of guides below might be useful if you are analysing this data and want to know more about it:

Related column headers in Raptor website crawler reports:


Benefits of Our Data

There are several benefits to analysing indexation data, such as those listed below:

  • Identify Indexation Issues for SEO
  • Identify all indexable Resources
  • Identify all non-indexable Resources
  • Resolve indexation issues & conflicts
  • Audit a site’s indexation profile
  • Scrape competitor indexation data


Sign up Today for a Free 30-Day Trial. Identify indexation Issues on your site!