Adding or Modifying Source Filters

The starting address of a source determines the content that is crawled while source filters control what is indexed. You can use source filters to more specifically control subsections to index or not under a source starting address. Inclusion/exclusion rules use either wildcard or regular expressions to define filtering patterns.

Exclusion filters are commonly used for any source type to prevent indexing of one or more subsections under the stating address.

Inclusion filters are typically used for Web connector sources to include referred web pages outside of the starting address.

To add or modify a source filter

1. On the Coveo server, access the Administration Tool (see Opening the Administration Tool).

2. In the Administration Tool, select Index > Sources and Collections.

3. In the Sources and Collections page, select the collection containing the source for which you want to add or modify source filters.

4. In the Sources section, select the source.

5. In the navigation panel on the left, select Filters.

6. In the Filters page:

   

   • To add or modify an exclusion filter:

     1. Click Add an Exclusion Filter to create a new filter or click an existing exclusion filter to modify it.

     2. In the Edit an Exclusion Filter page, in the Excluded Patterns box, enter one or more address patterns (one pattern per line) to exclude using either wildcard characters ( * or ?) or a regular expression.

        Patterns must be subsections of existing inclusion filters otherwise they will be useless.

     3. In the Type box, select the search pattern option used in the patterns you entered in Excluded Patterns.

     4. Click Save/Apply Changes.

   • To add or modify an inclusion filter:

     Note: Inclusion filters already listed correspond to entries in the Addresses box of the General page. In this box, using a slash or not, at the end of the starting address path for a file system folder (as opposed to a file) as no effect on what is crawled.

     It however affects what is indexed with the inclusion filter automatically created for each starting address. When the trailing slash is omitted, the last folder of the starting address is truncated from the inclusion filter.

     Example: The starting address file:///C:/temp/ creates the inclusion filter file:///C:/temp/*, while the starting address file:///C:/temp creates the inclusion filter file:///C:/*.

     1. Click Add an Inclusion Filter to create a new filter or click an existing inclusion filter to modify it.

     2. In the Edit an Inclusion Filter page, in the Allowed Patterns box, enter one or more address patterns (one pattern per line) to include using either wildcard characters ( * or ?) or a regular expression.

        Note: A filter pattern can include a file extension to filter by file type but it is a better practice to define an appropriate document type set for this source (see Creating a Document Type Set).

     3. In the Type box, select the search pattern option used in Allowed Patterns.

     4. Click Save/Apply Changes.

     5. If you are using inclusion filters to index subsections under the starting address: 

        Note: Using inclusion filters like this can have a significant performance cost. The best practices is to rather create one source per subsection to include, or when the source type allows it, enter more than one starting address for one source.

        1. If the default wildcard * inclusion filter is present, select it, and then click Delete to remove it and prevent everything below the starting address to be indexed.

        2. For a File connector source, in the General page, ensure that the Expand Before Filtering option is selected otherwise nothing will be crawled.

        3. For a SharePoint connector source, in the Advanced page, ensure that the Expand sites and lists before applying filter option is selected otherwise nothing will be crawled.

7. Refresh the source for the modifications to take effect.