Cookies disclaimer

I agree Our site saves small pieces of text information (cookies) on your device in order to deliver better content and for statistical purposes. You can disable the usage of cookies by changing the settings of your browser. By browsing our website without changing the browser settings you grant us permission to store that information on your device.

Step 1.10: Robots

Step 1.10: Robots

Core Concepts | Search Engines | Robots, Crawlers & Spiders


Also called bots, crawlers, web crawlers, & spiders.

These are programs that crawl web pages and collect data from them or about them. Robots, like Google’s ‘googlebot’ collect hundreds of pieces of information about every single web page that they visit.

It’s not only Google that use these, all search engines use crawlers or robots to collect data.


< Previous Step 1.9: Search Engines

Next Step 1.11: Indexation >
















30-Day Free Trial of our SEO Web Crawler Now Available, sign up with a valid email address and your name below to get instant access. No Credit Cards Required.