Overview
It is possible to integrate Fluid Topics with the Coveo search engine. Once the integration is set up, the Coveo search engine uses its generic REST API source to perform the following actions:
- Crawl content in the Knowledge Hub.
- Index the crawled content on Coveo servers.
- Each API crawl counts for one call to list maps, and one additional call per topic for each map.
- For more information about how crawling works, refer to Introduction to crawling.
Procedure
Users with the ADMIN
role can follow the steps in this section to configure a Coveo instance for integration with Fluid Topics.
Prerequisites
Before configuring the Coveo instance, it is necessary to meet the following prerequisites:
- It is necessary to have a Coveo license before starting this procedure.
- Make sure that none of the content access rules configured for the portal prevent bots from crawling content that should be indexed.
- Create an API key with the
KHUB_ADMIN
role.
Create a Generic REST API source
Authorized administrators can create a generic REST API source for a Coveo instance as follows:
-
In the Coveo instance, go to the Content > Sources menu.
-
Select the Add source button.
-
Select the Generic REST API source type.
-
In the Authentication section, paste the API key (see Prerequisites) in the API key authentication field.
-
Download the content of the example configuration file provided as attachment in the Attachments tab of the Reader page sidebar.
-
Paste the content of the example configuration file in the JSON Configuration window .
-
Select Save (and not Save and build).
Create Coveo fields
Coveo fields are necessary to retrieve metadata from the portal. Authorized administrators can configure Coveo fields as follows:
-
Go to Content > Fields.
-
Select Add field.
-
Define a Field name. This can be a title, a date, the value of an
ft:originId
metadata, etc. -
Define a Type.
-
Select Add field.
Create Coveo mappings
Coveo mappings are necessary to retrieve metadata from the portal. Authorized administrators can configure Coveo mappings as follows:
-
Go to Content > Sources.
-
Select the
Generic REST API
source. -
In the More menu, select Manage mappings.
-
In the Common tab, select the Add button, then select Mapping.
-
In the window that appears, choose Select an option and define the target field to map.
-
In the Rules field, define a metadata key for Fluid Topics to return as shown in the following example:
%[my_metadata_key]
-
Select Add mapping.
-
Select Save and rebuild source.
Example
A Coveo source configuration file in JSON
format is available in the Attachments tab of the Reader page sidebar.
The file describes the API that crawls content and defines the following information:
- The order of calls to make
- How to handle paginated queries
- Authorization
- How to interpret the JSON responses
The following lines show an example of a file that indexes all maps, topics and unstructured documents:
{
"Services": [
{
"Url": "https://my.fluidtopics.net/coveo",
"Paging": {
"PageSize": 25,
"OffsetStart": 1,
"OffsetType": "page",
"Parameters": {
"Limit": "per_page",
"Offset": "page"
}
},
"Endpoints": [
{
"Path": "/api/khub/maps/search",
"Method": "POST",
"Headers": {
"Authorization": "Bearer @ApiKey",
"Ft-Calling-App": "coveo/index"
},
"PayloadJsonContent": "{}",
"RefreshEndpoints": [
{
"Method": "POST",
"DateFormat": "yyyy-MM-dd",
"QueryParameters": {
"from": "@RefreshDate"
}
}
],
"ItemType": "Structured",
"ItemPath": "results",
"Uri": "%[mapUrl]",
"ClickableUri": "%[readerUrl]",
"ModifiedDate": "%[metadata[?(@.key=='ft:lastEdition')].values[0]]",
"Title": "%[title]",
"SubQueries": [
{
"Path": "/api/khub/maps/%[coveo_parent.raw.mapId]",
"Method": "GET",
"Headers": {
"Authorization": "Bearer @ApiKey",
"Ft-Calling-App": "coveo/index"
},
"Body": "%[description]",
"Metadata": {
"id": "%[id]",
"language": "%[lang]",
"lastEdition": "%[lastEdition]",
"lastPublication": "%[lastPublication]",
"filetype": "html"
}
},
{
"Path": "/api/khub/maps/%[coveo_parent.raw.mapId]/rights",
"Method": "GET",
"Headers": {
"Authorization": "Bearer @ApiKey",
"Ft-Calling-App": "coveo/index"
},
"Metadata": {
"accessLevel": "%[accessLevel]",
"groups": "%[groups]"
}
}
],
"SubItems": [
{
"Path": "/api/khub/maps/%[coveo_parent.id]/topics",
"Method": "GET",
"Headers": {
"Authorization": "Bearer @ApiKey",
"Ft-Calling-App": "coveo/index"
},
"ItemType": "Topic",
"Uri": "%[coveo_url]/api/khub/maps/%[coveo_parent.id]/topics/%[id]",
"ClickableUri": "%[coveo_url]%[readerUrl]",
"Title": "%[title]",
"ModifiedDate": "%[coveo_parent.lastEdition]",
"Metadata": {
"id": "%[id]",
"language": "%[coveo_parent.language]",
"lastEdition": "%[coveo_parent.lastEdition]",
"lastPublication": "%[coveo_parent.lastPublication]"
},
"SubQueries": [
{
"Path": "%[coveo_parent.raw.contentApiEndpoint]",
"Method": "GET",
"Headers": {
"Authorization": "Bearer @ApiKey",
"Ft-Calling-App": "coveo/index"
},
"IsBinaryBody": true
}
]
}
]
},
{
"Path": "/api/khub/documents/search",
"Method": "POST",
"Headers": {
"Authorization": "Bearer @ApiKey",
"Ft-Calling-App": "coveo/index"
},
"PayloadJsonContent": "{}",
"RefreshEndpoints": [
{
"Method": "POST",
"DateFormat": "yyyy-MM-dd",
"QueryParameters": {
"from": "@RefreshDate"
}
}
],
"ItemType": "Unstructured",
"ItemPath": "results",
"Uri": "%[documentUrl]",
"ClickableUri": "%[viewerUrl]",
"ModifiedDate": "%[metadata[?(@.key=='ft:lastEdition')].values[0]]",
"Title": "%[title]",
"SubQueries": [
{
"Path": "/api/khub/documents/%[coveo_parent.raw.documentId]",
"Method": "GET",
"Headers": {
"Authorization": "Bearer @ApiKey",
"Ft-Calling-App": "coveo/index"
},
"Body": "%[description]",
"Metadata": {
"id": "%[id]",
"language": "%[lang]",
"lastEdition": "%[lastEdition]",
"lastPublication": "%[lastPublication]",
"filetype": "%[mimeType]",
"filename": "%[filename]"
}
},
{
"Path": "/api/khub/documents/%[coveo_parent.raw.documentId]/rights",
"Method": "GET",
"Headers": {
"Authorization": "Bearer @ApiKey",
"Ft-Calling-App": "coveo/index"
},
"Metadata": {
"accessLevel": "%[accessLevel]",
"groups": "%[groups]"
}
},
{
"Path": "/api/khub/documents/%[coveo_parent.raw.documentId]/content",
"Method": "GET",
"Headers": {
"Authorization": "Bearer @ApiKey",
"Ft-Calling-App": "coveo/index"
},
"IsBinaryBody": true
}
]
}
]
}
]
}
Certain fields in this file are formatted as follows:
html
"foo": "‰[bar]"
This is an internal Coveo syntax that refers to an attribute of the object that Coveo's Generic Rest API connector creates.
Known limitations
Items deleted from Fluid Topics will not be deleted from Coveo. It is necessary to regularly schedule a rescan in Coveo to remove deleted items.