diff options
Diffstat (limited to 'docs/admin/robots.md')
| -rw-r--r-- | docs/admin/robots.md | 4 |
1 files changed, 4 insertions, 0 deletions
diff --git a/docs/admin/robots.md b/docs/admin/robots.md index b9e0468ce..3de4fe079 100644 --- a/docs/admin/robots.md +++ b/docs/admin/robots.md @@ -2,6 +2,10 @@ GoToSocial serves a `robots.txt` file on the host domain. This file contains rules that attempt to block known AI scrapers, as well as some other indexers. It also includes some rules to ensure things like API endpoints aren't indexed by search engines since there really isn't any point to them. +## Allow/disallow stats collection + +You can allow or disallow crawlers from collecting stats about your instance from the `/nodeinfo/2.0` and `/nodeinfo/2.1` endpoints by changing the setting `instance-stats-mode`, which modifies the `robots.txt` file. See [instance configuration](../configuration/instance.md) for more details. + ## AI scrapers The AI scrapers come from a [community maintained repository][airobots]. It's manually kept in sync for the time being. If you know of any missing robots, please send them a PR! |