Configure a crawler to search the local portal site

Configure and run a search crawler on the local portal site to gather information and create a search collection that enables your users to search the portal site.

Portal Search provides a default portal site search collection that enables the users to search the portal site. Before the users can search the portal site collection, perform the following tasks.

1. Set the crawler user ID. Set a dedicated crawler user ID for crawling the portal site content source. Proceed as follows:

   1. Define the crawler user ID using the Manage Users and Groups portlet. Proceed as follows:

      It is of benefit to define a dedicated crawler user ID. The pre-configured default portal site search uses the default administrator user ID wpsadmin with the default password on that user ID for the crawler. If you changed the default administrator user ID during the portal installation, the crawler user that default user ID. If you have changed the password for the wpsadmin or other admin user ID, or if you changed the default administrator user ID to an ID other than wpsadmin, or to use a separate user ID, set the crawler user ID.

   2. Set the preferred language of the portal site crawler user ID to match the language of the portal site search collection that it crawls. If you do this after you started a crawl on the portal site search collection, you need to reset the portal site collection.

   3. Edit the portal site collection content source and fill in the crawler user ID and its password:

      1. Click...
         Administration | Manage Search | Search Collections | Default Search Collection

      2. In the Content Source Name list of the search collection, click the Portal Content Source search collection.

      3. Click the Edit icon next to the Portal Content Source collection name.

      4. Select the Security tab.

      5. Click the Edit icon next to the security realm to modify.

      6. Type the crawler user ID and password into the appropriate fields.

      7. Click Update.

      8. Click Save to save the changes.

   4. Optional: For content sources of type Web Site, we can configure the crawler to follow external links from inside the portal. To do this, modify the value in the field Levels of links to follow under the tab General Parameters. Set the level to a value higher than 1. In addition, we can configure filters for those external links from the Filters tab. The default filter suppresses all links that point back to portal pages. The default filter is displayed only after saving the configuration of the content source.

2. Start the initial crawl. Start the initial crawl on the portal site content source:

   1. Click...
      Administration | Manage Search | Search Collections | Default Search Collection

   2. Click the Start Crawler icon (right-pointing arrow) next to the Portal content source name.

3. Configure regular crawls. If you want regular crawls on the portal site content source, perform either of the following tasks:

   • Enable the default scheduler:

     1. Click the View Content Source Schedulers icon next to the collection name.

     2. In the Manage Schedulers page, click Disabled.

        This changes the status of the scheduler to Enabled and displays a confirmation message.

   • Set up our own scheduler.

     1. Click the Edit icon for the content source.

        We can have only one schedule at a time. Therefore, to create our own schedule, you first have to delete the existing schedule.

     2. Select the Schedulers tab.

     3. Configure our own scheduler as required. For more details about how to do this, refer to the Manage Search portlet help.

     4. Click Save to save the changes.

For more detailed information about how to work with content sources refer to Manage the content sources of a search collection and to the Manage Search portlet help.

1. The local portal site is exposed through a service that requires SSL. If the portal is configured with a Web server and you configure the content source root URL through the Web server, configure the Web server for SSL.

2. By default, items in the result lists from portal site searches provide no summary information. To have the summary information added, configure the portlet with the summary parameter enabled as follows: PortalCollectionSummarizer=on.

3. When you crawl a portal site, be aware of the Memory required for crawls and the Time required for crawls and imports and availability of documents.

4. Set the preferred language of the crawler user ID to match the language of the search collection that it crawls.

5. The portal site search collection is created when an administrator navigates to the Manage Search portlet. However, you must start the crawl for users to be able to search the portal site. Depending on the portal configuration and environment and possible customization, we might need to reset the portal site search collection that was created. For details about such scenarios and the necessary tasks to perform refer to Reset the default search collection.

6. If the users search the portal site search collection on a secured portal site, refer to the additional information under Enable search on a secured portal site with the default configuration.

7. The portal search crawler indexes static content pages and all pages that include portlets.

When users search a portal site, they can access portal pages of two types:

• Public or anonymous portal pages.

  These are pages that users can view without authentication by user ID and password. The crawler can crawl public pages on the portal site on which it resides, or on a remote portal.

  If you want anonymous users to be able to search the public pages of the portal site, refer to Enable anonymous users to search public pages of the portal.

• Secured portal pages.

  These are pages that users can only view if they authenticate themselves to the portal by logging in to the portal with a user ID and password. Refer to Configure search on a secured portal site.

  We can crawl, index, and search secured portal pages only on the local portal installation. For security reasons, we cannot crawl secured pages of one portal site from another portal site.

If you customize search on the portal site, we might find useful information under Configure the default location for search collections and Reset the default search collection.

If the portal site is multilingual and the users use different languages to search the portal, refer to Crawl a multilingual portal site.

Parent: Search the local portal
Related: Crawl a multilingual portal site
Configure search on a secured portal site
Reset the default search collection
Manage the content sources of a search collection
Enable search on a secured portal site with the default configuration
Enable anonymous users to search public pages of the portal
Configure the default location for search collections
Tips for using Portal Search

Configure the Web server plug-in for Secure Sockets Layer (WAS information center)