Adding and Indexing Custom Metadata in an XML Sitemap

CES 7.0.7814+ (August 2015)

The Sitemap connector now supports to index Coveo specific custom metadata that a developer adds to an XML sitemap file. When a developer can generate or modify the sitemap XML file of a repository to index, he can also include a Coveo namespace (coveo:metatada) and metadata to provide information on documents that is not found in the out-of-the-box fields (Sitemap field set and Coveo System fields).

The added Coveo metadata will only be read by the Coveo crawler and connector but ignored by all other processes, but respect the Sitemap protocol (see Sitemap protocol).

The following procedure requires a user that has the permissions and skills to modify or create an XML sitemap file and access to the CES Administration Tool.

To add and index custom metadata in an XML Sitemap

1. You or a developer must code a third-party process to modify or create an XML sitemap file as follows:

   Note: Contact Coveo Professional Services if you need assistance.

   1. In the urlset XML element start tag (<urlset>), extend the Sitemap protocol using the Coveo namespace by adding the following line:

      xmlns:coveo="http://www.coveo.com/schemas/metadata"

      Example:

      <?xml version="1.0" encoding="UTF-8"?>
      <urlset
            xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd"
            xmlns:coveo="http://www.coveo.com/schemas/metadata">  

   2. For each url elements (<url></url>) in the sitemap, create a new XML element named coveo:metadata (<coveo:metadata></coveo:metadata>).

      Example:

      <url>
        <loc>http://example.com/about/</loc>
        <lastmod>2015-02-10T13:47:23+00:00</lastmod>
        <changefreq>weekly</changefreq>
        <priority>1.00</priority>
        <coveo:metadata>
        </coveo:metadata>
      </url>  

   3. Within the coveo:metadata elements, add your custom metadata (name and value).

      Notes: CES 7.0.8850+ (March 2017)

      • Character Data (CDATA) is supported when you place the CDATA tag (![CDATA[) at the beginning of the node (see Character Data and Markup).

        Example:

        <coveo:metadata>
        	<casenumber>18467</casenumber>
        	<companyname>
        		<![CDATA[
        		Company XYZ Inc. <USA>
        		]]>
        	</companyname>
        </coveo:metadata>  

      • The connector ignores the CDATA tag and indexes the rest of the node content such as special characters (e.g., &, %, $, and ~) and <xml> tags as text.

      Example: You want to add the name of the author, the last date of modification and the document tags (if any) so you add the following XML elements:

      <coveo:metadata>
      	<modificationdate>2015-02-10T13:47:23+00:00</modificationdate>
          <authorname>John Smith</authorname>
          <tags />
        </coveo:metadata>  

      Once done, the sitemap could look like the following:

      <?xml version="1.0" encoding="UTF-8"?>
      <urlset
            xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd"
            xmlns:coveo="http://www.coveo.com/schemas/metadata">
      <url>
        <loc>http://example.com/about/</loc>
        <lastmod>2015-02-10T13:47:23+00:00</lastmod>
        <changefreq>weekly</changefreq>
        <priority>1.00</priority>
        <coveo:metadata>
          <modificationdate>2015-02-10T13:47:23+00:00</modificationdate>
          <authorname>John Smith</authorname>
          <tags />
        </coveo:metadata>
      </url>
      </urlset>  

2. In the Administration Tool, for the custom metadata you want to see in your document details, add the corresponding custom fields with the sitemap prefix in the sitemap field set (see Adding or Modifying Custom Fields).

   Note: It is not mandatory to add every custom metadata, but when you do so the Name must match the XML element name.

   Important: XML element names are case-sensitive.

   Example: You want to have the author name in the results metadata so you add the sitemapauthorname field.

   

3. Build or rebuild your Sitemap source.

4. In the Index Browser, verify that the new metadata are available on your Sitemap source documents (see Reviewing Document Details from the Index Browser) .