Timeline manager (#40)

* start messing about with timeline manager

* i have no idea what i'm doing

* i continue to not know what i'm doing

* it's coming along

* bit more progress

* update timeline with new posts as they come in

* lint and fmt

* Select accounts where empty string

* restructure a bunch, get unfaves working

* moving stuff around

* federate status deletes properly

* mention regex better but not 100% there

* fix regex

* some more hacking away at the timeline code phew

* fix up some little things

* i can't even

* more timeline stuff

* move to ulid

* fiddley

* some lil fixes for kibou compatibility

* timelines working pretty alright!

* tidy + lint

1 files changed, 17 insertions, 0 deletions

diff --git a/internal/api/security/robots.go b/internal/api/security/robots.go
new file mode 100644
index 000000000..65056072a
--- /dev/null
+++ b/internal/api/security/robots.go
@@ -0,0 +1,17 @@
+package security
+
+import (
+	"net/http"
+
+	"github.com/gin-gonic/gin"
+)
+
+const robotsString = `User-agent: *
+Disallow: /
+`
+
+// RobotsGETHandler returns the most restrictive possible robots.txt file in response to a call to /robots.txt.
+// The response instructs bots with *any* user agent not to index the instance at all.
+func (m *Module) RobotsGETHandler(c *gin.Context) {
+	c.String(http.StatusOK, robotsString)
+}