Robot detection

Abstract

Configure the robot detection component to filter out unwanted interactions from robots.

You can enable the Sitecore robot detection functionality to filter out unwanted interactions from automated browsers and robots. This ensures that only genuine contacts are registered in the xDB. Robot detection classifies a contact based on IP address, IP Geolocation information, user agent, or verified human behavior.

There are two main ways to identify robots:

  • Using the exclude robots configuration file to filter by IP address or user agent.

  • Using the robot detection functionality to identify human behavior.

Filtering contacts by IP address or user agent

The Sitecore.Analytics.ExcludeRobots.config file contains an exclude list of IP addresses and user agents. If a contact visits your website from one of the IP addresses or has a user agent in the exclude list, then the request to view the page is not tracked. You can add IP addresses and user agents to this list manually.

Identifying human contacts

The Sitecore robot detection component is enabled by default. It classifies all visitors on their first visit as robots, using the classification code 925.

Next, it performs a series of tests to determine whether a visitor is a human or a robot. For example, the VisitorIdentification.js JavaScript file tests if a visitor is human by detecting the movements of a computer mouse or the visitor touching the screen on a mobile phone or tablet. Human contacts are classified using the classification code 0.

Important

Ensure that you include the robot detection control in your main layout so that it renders with every page view.

Contact classification

The ContactClassification class contains constants and helper methods in the code that enable the robot detection component to set the following visitor classifications:

  • IsHuman

  • IsRobot

  • IsAutoDetectedRobot

The robot detection component

The Sitecore robot detection component consists of a pipeline processor, an event handler, a JavaScript file, and several robot-detection code classes.

The Sitecore.Analytics.Tracking.RobotDetection.config file contains several configuration settings in addition to the following pipeline and event handler:

  • Sitecore.Analytics.RobotDetection.Pipeline.InitializeTracker.Robots

  • Sitecore.Analytics.RobotDetection.Media.MediaRequestEventHandler

These components are invoked when a contact or robot interacts with your website, initiating a workflow that identifies robots and prevents them from accessing your website.