Robots.txt - Fluid Topics - 3.11

Fluid Topics Configuration and Administration Guide

Category
Reference Guides
Audience
public
Version
3.11

If an administrator enables the Crawl feature for a tenant, the robots.txt file gives crawlers instructions about how to index the following parts of the portal:

  • The Homepage
  • The Reader page
  • The Viewer page

The robots.txt file also provides the path to a sitemap which allows bots to retrieve URLs to the Reader page and the Viewer page.

It is possible to replace the robots.txt file with a custom file. This custom file must be named robots.txt and located in the public tenant configuration directory.

When a custom file is deployed, the Fluid Topics server no longer generates or updates the default version of the file.

Antidot cannot be held responsible for any issue caused by the use of a custom robots.txt file.