Portal, Express Beta Version 6.1 Operating systems: i5/OS, Linux,Windows

Configuring a crawler to search your local portal site

Configure and run a search crawler on your local portal site to gather information and create a search collection that enables your users to search your portal site.

1. Optional: Set the crawler user ID. If you want to use a dedicated crawler user ID for crawling the portal site content source, define and set that crawler user ID:

   1. Define the crawler user ID by using the Manage Users and Groups portlet.
   2. Set the preferred language of the portal site crawler user ID to match the language of the portal site search collection that it crawls. (If you do this after you started a crawl on the portal site search collection, you need to reset the portal site collection. Refer to Creating or resetting the portal site collection.)
   3. Edit the portal site collection content source and fill in the crawler user ID and its password. To do this, proceed as follows:

      1. Click Launch > Administration > Manage Search > Search Collections.
      2. In the search collection list, click the Portal Content search collection.
      3. Click the Edit icon next to the Portal Content Source collection name.
      4. Under the General Parameters tab, type the crawler user ID into the appropriate field.
      5. Under the Security tab, type the crawler password into the appropriate field.
      6. Click Save.
   4. Optional: Configure the crawler to follow external links. If you want the crawler to follow external links from inside the Portal, you can modify the value in the Levels of links to follow field under the General Parameters tab. Set the level to a value higher than 1. In addition, you can configure filters for those external links from the Filters tab. The default filter suppresses any links that point back to Portal pages. The default filter is displayed only after saving the configuration of the content source.
2. Start the initial crawl. Start the initial crawl on the portal site content source:

   1. Click Launch > Administration > Manage Search > Search Collections.
   2. In the search collection list, click the Portal Content search collection.
   3. Click the Start Crawler icon (right-pointing arrow) next to the Portal content source name.
3. Configure regular crawls. If you want regular crawls on the portal site content source, perform either of the following tasks:

   • Enable the default scheduler. To do this, proceed as follows:

     1. Click the View Content Source Schedulers icon next to the collection name.
     2. In the Manage Schedulers page, click Disabled. This changes the status of the scheduler to Enabled and displays a confirmation message.
   • Set up your own scheduler. To do this, proceed as follows:

     1. Click the Edit icon for the content source.
     2. Select the Schedulers tab.
     3. Configure your own scheduler as required. For more details about how to do this, refer to the Manage Search portlet help.

1. Only the main panels of the portlets on the portal pages are indexed and can be searched. The crawler does not follow links that are specified within a portlet.
2. By default, items in the result lists from portal site searches provide no summary information. If end users are using the Search and Browse portlet they can refer to the information given under Description: for information about the search result list item. If you want to have the summary information added, configure the portlet with the summary parameter enabled as follows: PortalCollectionSummarizer=on.
3. When you crawl a portal site, be aware of the Memory required for crawls and the Time required for crawls and imports and availability of documents.
4. Set the preferred language of the crawler user ID to match the language of the search collection that it crawls.
5. The portal site search collection is created when an administrator navigates to the Manage Search portlet. However, start the crawl for users to be able to search the portal site. Depending on your portal configuration and environment and possible customization, you might need to reset the portal site search collection that was created. For details about such scenarios and the necessary tasks to perform refer to Creating or resetting the portal site collection.
6. If your users search the portal site search collection on a secured portal site, refer to the additional information under Enabling search on a secured portal site with the default configuration.

• Public or anonymous portal pages. These are pages that users can view without authentication by user ID and password. The crawler can crawl public pages on the portal site on which it resides, or on a remote portal.

  If you want anonymous users to be able to search the public pages of your portal site, refer to Enabling anonymous users to search public pages of your portal.

• Secured portal pages. These are pages that users can only view if they authenticate themselves to the portal by logging in to the portal with a user ID and password. Refer to Configuring search on a secured portal site. Note: You can crawl, index, and search secured portal pages only on your local portal installation. For security reasons, you cannot crawl secured pages of one portal site from another portal site.

If you customize search on your portal site, you might find useful information under Configuring the default location for search collections and Creating or resetting the portal site collection.

If your portal site is multilingual and your users use different languages to search your portal, refer to Crawling a multilingual portal site.