To configure Near Real-Time Indexing

1. Ensure that your Coveo Master and Mirror (if applicable) servers meet the requirements for the size of your index.

   Consider following the recommendation to add a NRTI dedicated disk on each server serving queries (see Coveo Platform Hardware and Software Requirements).

2. On the Coveo server, access the Administration Tool (see Opening the Administration Tool).

3. Before enabling NRTI on a Mirror server:

   1. Ensure that the Mirror is Online and In Sync (see Determining the Status of a Mirror or Slice).

   2. Synchronize the Mirror when it is not the case (see Applying an Action to a Mirror).

4. On the Administration Tool menu, select Index > Mirrors.

5. In the Mirrors page, in the Name column table, click the mirror for which you want to enable NRTI.

   Repeat the following steps for each mirror for which you want to enable NRTI.

6. In the Mirror: [name] General page, in the navigation panel on the left, click Near Real-Time Indexing.

7. In the Mirror: [name] Near Real-Time Indexing page:

   

   1. If not already the case, select the Enabled check box.

      For CES 7.0.7022+, NRTI is enabled by default for both new and existing indexes.

   2. In the Index Folder box, enter the full path where you want to save the NRTI index files.

      Note: The default folder for new or existing indexes like when the box is empty is [Index_Path]\Index\Default\RealtimeIndexing so by default, the NRTI files are stored on the same disk as the normal index slices.

      Sharing the same disk will work, but because NRTI is an I/O intensive process, to achieve optimal performances, it is recommended to add one dedicated disk for the NRTI files on each Coveo server (Master or Mirror) that are serving queries, and point to this disk with the Index Folder parameter. The dedicated disk specifications depend on the size of your index (see Coveo Platform Hardware and Software Requirements).

   3. In the Temporary Files Folder box, enter the path where the NRTI temporary files are saved.

      For an existing index, the default folder that applies when the box is empty is [Index_Path]\Temp.

   4. In the Maximum Number of Documents per Slice box, leave the default value (200000) unless Coveo Support instructs you to change it.

      This NRTI temporary slice size optimally minimizes the indexing pipeline processing time and the required NRTI resources.

   5. CES 7.0.7022+ (September 2014) In the Minimum Number of Documents to Activate box, enter the index size specified in number of documents, above which the enabled NRTI will start to operate. The default and recommended value is 1000000. The NRTI process starts to improve performances for indexes of about this size. Consult Coveo Support before changing this value.

   Note: The documents that are in the index pipeline at the time NRTI is enabled must be indexed before NRTI becomes effective. In a healthy index, this delay will be relatively short, such as 30 minutes for a 40-million document index. In a bad case scenario where the Coveo server is lacking resources and cannot manage to close pending transactions, this delay could be significantly longer.

8. Estimate the index pipeline processing time.

   The following steps give a method to estimate how long it takes to complete the index pipeline process.

   1. On the Coveo server, using Windows Explorer, open the folder of the regular slice. If there is more than one slice, open the folder of each slice.

      Example: By default, the first slice is called Default and its folder is: [CES_Path]\Index\Default\Default

   2. In the slice folder(s), look for pre-transaction and transaction files that are named respectively using the following patterns: 

      • Index-nnnnnnnnnn.ptn

      • Index-nnnnnnnnnn.trn

      where nnnnnnnnnn is an incremental transaction ID number.

      Note: When no documents are being indexed, no pre-transaction and transaction files will be present in the slice folder(s). Either force a source refresh/rebuild or wait for some source to get refreshed to see new transaction files appear.

   3. At a given time, note the highest transaction ID number (without the leading zeroes) from the slice folder(s), and also note the current time.

   4. Using an account and a Coveo search interface having access to all the indexed content, in the search box, repeatedly submit the following query until you start getting results: 

      @systransactionid >= [highest_transaction_id]

      Example: At 14:05, the pre-transaction or transaction file in the slice folders with the highest transaction number is Index-0000005044.trn.

      In the search interface, you type the following query:

      @systransactionid >= 5044

   5. Note the time at which you start getting results, and then subtract the time at which you noted the highest transaction ID to get an estimate of the index pipeline process time.

What's Next?

Ensure that all CES aspects are optimized to minimize the searchability delay (see Minimizing the Searchability Delay).