| Age | Commit message (Expand) | Author | Files |
|---|---|---|---|
| 2026-01-22 | [chore] Update robots.txt md5 hashes for etags (#4623) | 1 | |
| 2026-01-22 | [chore] Update robots.txt with latest list of "ai" bullshit user-agents (#4622) | 1 | |
| 2025-04-19 | [chore] Update robots.txt with latest ai bullshit (#4024) | 1 | |
| 2025-02-24 | [feature] Use ETag for robots.txt to prevent mishaps (#3829)v0.18.1 | 1 | |
| 2025-02-05 | [feature] Use `X-Robots-Tag` headers to instruct scrapers/crawlers (#3737) | 1 |
