From 07d27709957248008c61d6b8d553e3d2eb14d154 Mon Sep 17 00:00:00 2001 From: tobi <31960611+tsmethurst@users.noreply.github.com> Date: Tue, 4 Feb 2025 16:52:42 +0100 Subject: [feature] Change `instance-stats-randomize` to `instance-stats-mode` with multiple options; implement nodeinfo 2.1 (#3734) * [feature] Change `instance-stats-randomize` to `instance-stats-mode` with multiple options; implement nodeinfo 2.1 * swaggalaggadingdong --- docs/admin/robots.md | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'docs/admin/robots.md') diff --git a/docs/admin/robots.md b/docs/admin/robots.md index b9e0468ce..3de4fe079 100644 --- a/docs/admin/robots.md +++ b/docs/admin/robots.md @@ -2,6 +2,10 @@ GoToSocial serves a `robots.txt` file on the host domain. This file contains rules that attempt to block known AI scrapers, as well as some other indexers. It also includes some rules to ensure things like API endpoints aren't indexed by search engines since there really isn't any point to them. +## Allow/disallow stats collection + +You can allow or disallow crawlers from collecting stats about your instance from the `/nodeinfo/2.0` and `/nodeinfo/2.1` endpoints by changing the setting `instance-stats-mode`, which modifies the `robots.txt` file. See [instance configuration](../configuration/instance.md) for more details. + ## AI scrapers The AI scrapers come from a [community maintained repository][airobots]. It's manually kept in sync for the time being. If you know of any missing robots, please send them a PR! -- cgit v1.2.3