summaryrefslogtreecommitdiff
path: root/internal/api/util/robots.go
diff options
context:
space:
mode:
authorLibravatar tobi <31960611+tsmethurst@users.noreply.github.com>2025-02-24 11:17:18 +0100
committerLibravatar GitHub <noreply@github.com>2025-02-24 11:17:18 +0100
commitfd670c6a279e2aa54822546536dbf88b45a93051 (patch)
tree108c9b8ff3f01bc845a33f628ff302accde00211 /internal/api/util/robots.go
parent[docs] Update `interactionPolicy` (#3703) (diff)
downloadgotosocial-fd670c6a279e2aa54822546536dbf88b45a93051.tar.xz
[feature] Use ETag for robots.txt to prevent mishaps (#3829)v0.18.1
* [feature] Use ETag for robots.txt to prevent mishaps * check incoming if-none-match header
Diffstat (limited to 'internal/api/util/robots.go')
-rw-r--r--internal/api/util/robots.go5
1 files changed, 5 insertions, 0 deletions
diff --git a/internal/api/util/robots.go b/internal/api/util/robots.go
index 49fb04561..32d510edd 100644
--- a/internal/api/util/robots.go
+++ b/internal/api/util/robots.go
@@ -130,4 +130,9 @@ Disallow: /.well-known/webfinger
Disallow: /.well-known/nodeinfo
Disallow: /nodeinfo/
`
+
+ // MD5 hash of basic robots.txt.
+ RobotsTxtETag = `ce6729aacbb16fae3628210c04b462b7`
+ // MD5 hash of robots.txt with NodeInfo disallowed.
+ RobotsTxtDisallowNodeInfoETag = `a1e4ce6342978bc8d6c3e3dfab07cab4`
)