Configuring and Indexing a Lithium Source

A source defines a set of configuration parameters for a specific Lithium community. When you want to index more than one Lithium community, configure one source for each community.

To configure and index a source with the Lithium connector

1. On the Coveo server, access the Administration Tool (see Opening the Administration Tool).

2. Select Index > Sources and Collections.

3. In the Collections section:

   1. Select an existing collection in which you want to add the new source.

      OR

   2. Click Add to create a new collection (see Adding a Collection).

4. In the Sources section, click Add.

5. In the General Settings section of the Add Source page:

   

   1. Enter the appropriate value for the following required parameters:

      Name

      Enter a descriptive name of your choice for the connector source.

      Example: Lithium Community

      Source Type

      The connector used by this source. In this case, select Lithium.

      Addresses

      Enter the URL of the Lithium community to index. Enter only one Lithium community address. Create other sources for other communities.

      Examples: The Lithium community addresses can be:

      • http://companymname.lithium.com/

      • https://community.companyname.com/

      Notes:

      • You can also specify to crawl only a subset of a Lithium community by simply copying and pasting the address of a Lithium category, board, or thread. The URL is in the form: 

        http://community.company.com/[...]/lithiumobjectlabel/objectid

        where the possible values for lithiumobjectlabel are given in the Label column in the following table.

        Label	Lithium object
        ct-p	Category
        tkbc-p	Knowledge base category
        bd-p	Forum
        tkb-p	Knowledge base
        con-p	Contest
        idb-p	Idea board
        qa-p	Q&A board
        bg-p	Blog
        gp-p	Group
        m-p	CES 7.0.7599+ (April 2015) Forum thread
        td-p	Forum thread
        ta-p	Knowledge base article
        cns-p	Contest submission
        idi-p	Idea
        qaq-p	Q&A Question
        ba-p	Blog article
        gpm-p	Group article

        Example: On the Lenovo Lithium community the following address is for the CES2014 forum:

        http://forums.lenovo.com/t5/Lenovo-CES-2014/bd-p/CES2014

      • If you use a reverse proxy, your REST API might not reside at your community root URL. You must therefore add the RestApiUrl hidden parameter to specify the URL address at which your REST API resides (see Modifying Hidden Lithium Source Parameters). Be aware, however, that Lithium does not recommend using a reverse proxy (see Reverse proxy and Lithium communities while logged in with a Lithium account).

      Fields

      If you created a custom Lithium field set, select it (see Lithium Connector Deployment Overview). Otherwise, leave the Default Scheme.

      Refresh Schedule

      Select the time interval at which the source is automatically refreshed to keep the index content up-to-date. By default, the Every day option instructs CES to refresh the source everyday at 12 AM. Because the incremental refresh takes care of maintaining the source up-to-date, you can select a longer interval such as Every Sunday (see What Should Be the Frequency of Source Refresh Schedules?).

      Notes:

      • You must enable incremental refresh on your source to continuously maintain the index up-to-date with your source between full refreshes (see Scheduling a Source Incremental Refresh.

      • CES 7.0.8996– (June 2017) The Lithium connector does not support incremental refresh. Consequently, the full refresh schedule is the only process that keeps your source up-to-date.

   2. Review the value for the following parameters that often do not need to be modified:

      Rating

      Change this value only when you want to globally change the rating associated with all items in this source relative to the rating to other sources (see Understanding Search Results Ranking).

      Example: If this source was for a legacy Lithium community, you may want to set this parameter to Low, so that in the search interface, results from this source appear later in the list compared to those from other sources.

      Document Types

      If you defined custom document type sets, ensure to select the most appropriate for this source (see What Are Document Type Sets?).

      Active Languages

      If you defined custom language sets, ensure to select the most appropriate for this source (see Adding and Configuring a Language Set).

6. In the Specific Connector Parameters & Options section of the Add Source page:

   

   1. Review if you need to change the default values for the following parameters:

      Number of Refresh Threads

      Determines the number of simultaneous downloads handled by the connector for this source. The default value is 2.

      Mapping File

      The path to the mapping file. Leave the default value to use the default mapping file that comes with the connector (Coveo.CES.CustomCrawlers.Lithium.MappingFile.xml). If you create a custom mapping file, enter the full path to your custom mapping file. Contact Coveo Support for assistance if you need to customize the mapping file.

   2. CES 7.0.9093+ (September 2017) Select the following checkboxes if you want to index the corresponding content.

      • Index attachments

        Select this option to index message attachments, such as images, videos, and PDF files.

        Notes:

        • For attachments to be indexed, the hidden parameter ItemTypesToIndex value must be empty or contain attachment as well (see Modifying Hidden Lithium Source Parameters).

        • Selecting this option requires one more API call per message.

        • CES 7.0.8996– (June 2017) For attachments to be indexed as well, you must add the FetchMessageAttachments hidden parameter (see FetchMessageAttachments and To modify hidden Lithium source parameters).

      • Index users

        Select this option to index data regarding users of a community.

        (missing or bad snippet)

        • For users to be indexed, the hidden parameter ItemTypesToIndex value must be empty or contain user as well (see Modifying Hidden Lithium Source Parameters).

        • You can index users only when indexing an entire community. If you only index a community subset such as a category or a board, you cannot index users (see Note: Crawling a community subset).

        • Indexing users of a large Lithium community can have a significant impact on the indexing process performance.

        • CES 7.0.8996– (June 2017) For users to be indexed as well, you must add the FetchUsers hidden parameter (see FetchUsers and To modify hidden Lithium source parameters).

   3. In the Parameters section, click Add Parameter to be able to change the default value of hidden parameters (see Modifying Hidden Lithium Source Parameters).

      Note: When you want to index the content of a Lithium community that is secured using form authentication, add the following hidden parameters (see Modifying Hidden Lithium Source Parameters).

      Hidden parameter	Value
      AuthenticationFormId	if not the default (form)
      LoginPageUrl	The login page URL
      LoginFieldName	if not the default (login)
      PasswordFieldName	if not the default (password)
      UseFormAuthenticationToFetchAttachments	true

   4. In the Option section, review the default value of the following check boxes:

      Index Subfolders

      Check to index all subfolders below the specified Lithium server address. Selected by default.

      Index the document's metadata

      When selected, CES indexes all the document metadata, even metadata that are not associated with a field. The orphan metadata are added to the body of the document so that they can be searched using free text queries.

      When cleared (default), only the values of system and custom fields that have the Free Text Queries attribute selected will be searchable without using a field query (see Adding a Field to Search On and What Are Field Queries and Free Text Queries?).

      Example: A document has two metadata:

      • LastEditedBy containing the value Hector Smith

      • Department containing the value RH

      In CES, the custom field CorpDepartment is bound to the metadata Department and its Free Text Queries attribute is selected.

      When the Index the document's metadata option is cleared, searching for RH returns the document because a field is indexing this value. Searching for hector does not return the document because no field is indexing this value.

      When the Index the document's metadata option is selected, searching for hector also returns the document because CES indexed orphan metadata.

      Generate a cached HTML version of indexed documents

      When you select this check box (recommended), at indexing time, CES creates HTML versions of indexed documents. In the search interfaces, users can then more rapidly review the content by clicking the Quick View link rather than opening the original document with the original application. Consider clearing this check box only if you do not want to use Quick View links or to save resources when building the source.

      Open results with cached version

      Leave this check box cleared (recommended) so that in the search interfaces, the main search result link opens the original document with the original application. Consider selecting this check box only when you do not want users to be able to open the original document but only see the HTML version of the document as a Quick View. In this case, you must also select Generate a cached HTML version of indexed documents.

7. In the Security section of the Add Source page:

   

   1. In the Authentication drop-down list:

      • When you want to index secured Lithium content, select the user identity that you created for the Lithium community to index.

        Note: The user identity you select must be a native Lithium user. An SSO user cannot be used to retrieve secured Lithium content.

        OR

      • When you index a public Lithium community, select (none).

   2. Click Save to save the source configuration.

8. Because the Lithium connector does not support indexing Lithium item permissions, you must rather set the permissions globally on the source:

   Note: You get the following error message in the CES Console when you try to index the permissions for a Lithium community source.

   Error with ID 'LITHIUM_INVALID_CONFIGURATION': Permissions indexing is not provided by the Lithium connector

   1. In the navigation panel on the left, select Permissions.

   2. In the Permissions page:

      

      1. Select the Specifies the security permissions to index option.

      2. Optionally, in the Allowed Users list, add or remove users or groups to precisely specify who has access to the content from this source. By default, the Active Directory everyone group specifies that any AD user can see all the content from this source.

      3. Optionally, in the Denied Users list, add users or groups to specify who has not access to the content from this source.

      4. Click Apply Changes.

9. On the toolbar, click Start/Rebuild to start indexing your source.

10. Validate that the source building process is executed without errors:

    • In the navigation panel on the left, click Status, and then validate that the indexing proceeds without errors.

      OR

    • Open the CES Console to monitor the source building activities (see Using the CES Console).