The search and indexing log files

The crawling log contains information about the indexing process. You set up the crawling log in the <log4net /> section of the App_Config/Sitecore.config file. The default logging level is INFO:

<logger name="Sitecore.Diagnostics.Crawling" additivity="false">
      <level value="INFO" />
      <appender-ref ref="CrawlingLogFileAppender" />
</logger>

You define the appender for this log in the same section. By default, it writes to a .txt file in the data/logs folder:

<appender name="CrawlingLogFileAppender" type=
          "log4net.Appender.SitecoreLogFileAppender, Sitecore.Logging">
      <file value="$(dataFolder)/logs/Crawling.log.{date}.txt" />
      <appendToFile value="true" />
      <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="%4t %d{ABSOLUTE} %-5p %m%n" />
      </layout>
</appender>

The implementation uses log4net. The log4net documentation describes how you can change the appender to write to a Windows Event Log, a database, or any other location.

This is an example of crawling log entries:

INFO  [Index=sitecore_core_index] Initializing IntervalAsynchronousUpdateStrategy 
              with interval '00:01:00'.
INFO  [Index=sitecore_core_index] Initializing LuceneDatabaseCrawler. 
              DB:core / Root:/sitecore
INFO  [Index=sitecore_master_index] Initializing SynchronousStrategy.
INFO  [Index=sitecore_master_index] Initializing LuceneDatabaseCrawler. 
              DB:master / Root:/sitecore
INFO  [Index=sitecore_web_index] Initializing OnPublishEndAsynchronousStrategy.
INFO  [Index=sitecore_web_index] Initializing LuceneDatabaseCrawler. 
              DB:web / Root:/sitecore
INFO  [Index=custom_master] Initializing IntervalAsynchronousUpdateStrategy 
              with interval '00:00:05'.
INFO  [Index=custom_master] Initializing LuceneDatabaseCrawler. DB:master / 
              Root:{D70CBEED-6DCF-483F-978F-6FC3C8049512}
INFO  [Index=custom_web] Initializing OnPublishEndAsynchronousStrategy.
INFO  [Index=custom_web] Initializing LuceneDatabaseCrawler. 
              DB:web / Root:{D70CBEED-6DCF-483F-978F-6FC3C8049512}
INFO  [Index=custom_web] Creating primary and secondary directories
INFO  [Index=custom_web] Resolving directories from index property 
              store for index 'custom_web'
INFO [Index=custom_master] IntervalAsynchronousUpdateStrategy executing.

The log begins with information about how indexes are configured and initialized. After that, an entry is written in the crawling log when an index update strategy is applied to an index or when a full rebuild is triggered.

Because the logging level in the example is INFO, there is a limited amount of entries in the log file.

For more detailed logs, for example, for troubleshooting purposes, you can change the logging level to DEBUG:

<logger name="Sitecore.Diagnostics.Crawling" additivity="false">
      <level value="DEBUG" />
      <appender-ref ref="CrawlingLogFileAppender" />
</logger>

With this configuration, and with the verbose logger enabled, Sitecore produces a very detailed item level indexing log.

You can enable verbose logging in addition to the standard logging. When you enable verbose logging, you can see more detail about the indexing.

You use verbose logging in situations such as:

You want to test how your index is updated across all servers in a multiserver environment

The VerboseLogger is instantiated when ContentSearch.VerboseLogging is true:

The default is false and the setting is not present in the configuration delivered by Sitecore. The Sitecore.ContentSearch.VerboseLogging.config.example file has an example that shows you how to enable verbose logging.

You use the verbose logging for troubleshooting. For example, if a specific item is not indexed, the VerboseLogger gives you more context and can help you solve the problem.

The VerboseLogger writes information about a large set of indexing events:

The search log contains information about the search queries that Sitecore executes.

You set up the search log in the <log4net /> section of the web.config file:

    <logger name="Sitecore.Diagnostics.Search" additivity="false">
      <level value="INFO" />
      <appender-ref ref="SearchLogFileAppender" />
    </logger>

You define the appender for this log in the same section. By default, it is set up to write to a .txt file in the data/logs folder:

<appender name="SearchLogFileAppender" 
           type="log4net.Appender.SitecoreLogFileAppender, Sitecore.Logging">
      <file value="$(dataFolder)/logs/Search.log.{date}.txt" />
      <appendToFile value="true" />
      <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="%4t %d{ABSOLUTE} %-5p %m%n" />
      </layout>
</appender>

The implementation is based on log4net. The log4net documentation describes how you change the appender to write to a Windows Event Log, a database, or any other location.

This is a sample of what the search log entries look like:

3212 19:31:56 INFO  ExecuteQueryAgainstLucene :
         +_datasource:sitecore +(+(+_path:11111111111111111111111111111111 +_latestversion:1)
         +mileagehwy:[1 TO 4mileagecity]) - Filter : 3212 19:31:56 
       INFO  Results from web database :8818

You can enable a full level debug of content searches by setting ContentSearch.EnableSearchDebug = true in the Sitecore.ContentSearch.config file and changing the logging level for the search logger to DEBUG:

<logger name="Sitecore.Diagnostics.Search" additivity="false">
      <level value="DEBUG" />
      <appender-ref ref="SearchLogFileAppender" />
</logger>

The entries in the search are useful for: