summaryrefslogtreecommitdiff
path: root/docs/admin/robots.md
AgeCommit message (Collapse)AuthorFiles
2024-04-22[chore] Update robots.txt (#2856)Libravatar Daenney1
This updates the robots.txt based on the list of the ai.robots.txt repository. We can look at automating that at some point. It's worth pointing out that some robots, namely the ones by Bytedance, are known to ignore robots.txt entirely.