+

Search Tips   |   Advanced Search

Configure verbose logging

Verbose logging for crawling and indexing goes to SystemOut.log. Verbose logging is enabled by default.

To configure verbose logging...

  1. Check out search-config.xml:

    where:

    To determine cellname:

    print AdminControl.getCell()

  2. Run:

    SearchCellConfig.enableVerboseLogging()

    Enable more detailed status reporting during crawling and indexing in the form of more verbose logging to SystemOut.log. Verbose logging is automatically enabled when Connections is installed. This command does not take any parameters.

    We can use the following commands to tune the frequency with which status information is logged to SystemOut.log during different stages of the crawling and indexing process:

    For more information about each of these commands, refer to the command descriptions that follow.

    SearchCellConfig.disableVerboseLogging()

    Disable verbose logging. This command does not take any parameters.

    Verbose logging fills SystemOut.log with detailed output that can occupy an increasing amount of disk space, unless you have configured the deployment to retain only a limited number of the most recent log files. A high turnover of logs might be a problem when we are trying to track down the cause of an issue if the log file that we are interested in has been deleted. For this reason, you might want to disable verbose logging. The performance impact of having verbose logging enabled is negligible.

    SearchCellConfig.setVerboseInitialLoggingInterval(int initialInterval)

    Control the frequency with which initial index creation progress is logged to SystemOut.log.

    Parameter:

    initialInterval

    Number of seedlist entries. A seedlist entry is an indexing instruction that specifies an action, such as the creation, deletion, or update of a specified document in the Search index. For example, if an interval of 500 is specified, then for every 500 entries processed, the number of seedlist entries indexed so far for an application by the current indexing job is logged. The initialInterval parameter is set to 250 by default.

    We can find additional logging information about initial index creation in SystemOut.log by searching for occurrences of the CLFRW0581I logging message. For example: 

    CLFRW0581I: Search is continuing to build the index 
for activities: 3500 seedlist entries indexed.

    For example:

      SearchCellConfig.setVerboseInitialLoggingInterval(500)

    SearchCellConfig.setVerboseSeedlistRequestLoggingInterval(int seedlistRequestInterval)

    Control the frequency with which seedlist crawling progress is logged to SystemOut.log.

    Parameter:

    seedlistRequestInterval

    Number of seedlist page requests. A seedlist crawl is a sequence of seedlist page requests, which are HTTP GET operations that fetch seedlist pages. A seedlist page can contain zero or more seedlist entries up to a specified maximum. For example, if an interval of 1 is specified, then after every seedlist request, the crawling progress of the application being currently crawled is logged. The seedlistRequestInterval parameter is set to 1 by default.

    We can find additional logging information about seedlist crawling in SystemOut.log by searching for occurrences of the CLFRW0604 logging message. For example: 

    CLFRW0604 : Current seedlist state: Finish Date: Thu May 12 10:14:58 
IST 2011; Start Date: Thu Jan 01 01:00:00 GMT 1970; Type: 1; 
Last Modified: Thu Jan 01 01:00:00 GMT 1970; Finished: false; 
Started: true; ACL Start: 0; Offset: 0;

    For example:

      SearchCellConfig.setVerboseSeedlistRequestLoggingInterval(1)

    SearchCellConfig.setVerboseIncrementalCrawlingLoggingInterval(int incrementalCrawlingInterval)

    Control the frequency with which seedlist update crawling progress is logged to SystemOut.log. An update crawl of an application fetches data that was created, updated, or deleted since the previous crawl of that application began.

    Parameter:

    incrementalCrawlingInterval

    Number of seedlist entries. For example, if an interval of 100 is specified, then, for every 100 entries that have been crawled, the number of entries that have been crawled for a particular application during the current indexing job is logged. The incrementalCrawlingInterval parameter is set to 100 by default.

    We can find additional logging information about initial index creation in SystemOut.log by searching for occurrences of the CLFRW0589I logging message. For example: 

    CLFRW0589I: Search is continuing to build the index for 
profiles: 1,600 seedlist entries indexed.

    For example:

      SearchCellConfig.setVerboseIncrementalCrawlingLoggingInterval(100)

    SearchCellConfig.setVerboseIncrementalBuildingLoggingInterval(int incrementalBuildingInterval)

    Control the frequency with which update indexing progress is logged to SystemOut.log. Update indexing of a Connections application or set of applications, is an indexing job that updates an index that already has content from all applications that are to be indexed as part of the current indexing job.

    Parameter:

    incrementalBuildingInterval

    Number of documents. For example, if an interval of 20 is specified, then for every 20 documents that have been indexed, the number of documents indexed when indexing a particular application during the current indexing job is logged. The incrementalBuildingInterval parameter is set to 100 by default.

    We can find additional logging information about update indexing progress in SystemOut.log by searching for occurrences of the CLFRW0600I logging message. For example:

      CLFRW0600I: Search is continuing to build the index for blogs: 40 documents indexed.

    For example:

      SearchCellConfig.setVerboseIncrementalBuildingLoggingInterval(100)

    SearchCellConfig.setVerboseLogging(int initialInterval, int seedlistRequestInterval, int incrementalCrawlingInterval, int incrementalBuildingInterval)

    Enable verbose logging with the specified initial interval, seedlist request interval, crawling interval, and incremental building interval.

    Run this command has the same net effect as calling the following commands in sequence:

  3. Check in search-config.xml:

    SearchCellConfig.checkInConfig()

  4. To exit the wsadmin client, type exit at the prompt.

  5. Stop the server or servers hosting the Search application, and then restart the Search servers. The next time the scheduled task runs, persisted seedlists are retained after indexing finishes.


Parent topic:
Verify Search


Related:

Verify Search index creation

Verify that Search is crawling regularly

Verify the index is being built incrementally