[feature] Allow exposing allows, implement `/api/v1/domain_blocks` and `/api/v1/domain_allows` (#4169)

- adds config flags `instance-expose-allowlist` and `instance-expose-allowlist-web` to allow instance admins to expose their allowlist via the web + api.
- renames `instance-expose-suspended` and `instance-expose-suspended-web` to  `instance-expose-blocklist` and `instance-expose-blocklist-web`.
- deprecates the `suspended` filter on `/api/v1/instance/peers` endpoint and adds `blocked` and `allowed` filters
- adds the `flat` query param to `/api/v1/instance/peers` to allow forcing return of a flat list of domains
- implements `/api/v1/instance/domain_blocks` and `/api/v1/instance/domain_allows` endpoints with or without auth depending on config
- rejigs the instance about page to include a general section on domain permissions, with block and allow subsections (and appropriate links)

Closes https://codeberg.org/superseriousbusiness/gotosocial/issues/3847
Closes https://codeberg.org/superseriousbusiness/gotosocial/issues/4150

Prerequisite to https://codeberg.org/superseriousbusiness/gotosocial/issues/3711

Reviewed-on: https://codeberg.org/superseriousbusiness/gotosocial/pulls/4169
Co-authored-by: tobi <tobi.smethurst@protonmail.com>
Co-committed-by: tobi <tobi.smethurst@protonmail.com>

3 files changed, 141 insertions, 28 deletions

diff --git a/docs/admin/domain_blocks.md b/docs/admin/domain_blocks.md
index f89c84a10..98d532481 100644
--- a/docs/admin/domain_blocks.md
+++ b/docs/admin/domain_blocks.md
@@ -41,9 +41,9 @@ With this in mind, you should only ever treat domain blocking as *one layer* of
 
 Unfortunately, the Fediverse has its share of trolls, many of whom see domain blocking as an adversary to be defeated. To achieve this, they often target instances which use domain blocks to protect users.
 
-As such, there are bots on the Fediverse which scrape instance domain blocks and announce any discovered blocks to the followers of the bot, opening the admin of the blocking instance up to harassment. These bots use the `api/v1/instance/peers?filter=suspended` endpoint of GoToSocial instances to gather domain block information.
+As such, there are bots on the Fediverse which scrape instance domain blocks and announce any discovered blocks to the followers of the bot, opening the admin of the blocking instance up to harassment. These bots use the `api/v1/instance/peers?filter=suspended`, `api/v1/instance/peers?filter=blocked`, and/or `api/v1/instance/domain_blocks` endpoints of GoToSocial instances to gather domain block information.
 
-By default, GoToSocial does not expose this endpoint publicly, so your instance will be safe from such scraping. However, if you set `instance-expose-suspended` to `true` in your config.yaml file, you may find that this endpoint gets scraped occasionally, and you may see your blocks being announced by troll bots.
+By default, GoToSocial does not expose these endpoints publicly, so your instance will be safe from such scraping. However, if you set `instance-expose-blocklist` to `true` in your config.yaml file, you may find that these endpoints gets scraped occasionally, and you may see your blocks being announced by troll bots.
 
 ## What are the side effects of creating a domain block
 
diff --git a/docs/api/swagger.yaml b/docs/api/swagger.yaml
index 0eca007ea..97741b24e 100644
--- a/docs/api/swagger.yaml
+++ b/docs/api/swagger.yaml
@@ -1101,7 +1101,7 @@ definitions:
         properties:
             comment:
                 description: |-
-                    If the domain is blocked, what's the publicly-stated reason for the block.
+                    If the domain is blocked or allowed, what's the publicly-stated reason (if any).
                     Alternative to `public_comment` to be used when serializing/deserializing via /api/v1/instance.
                 example: they smell
                 type: string
@@ -1113,11 +1113,17 @@ definitions:
                 x-go-name: Domain
             public_comment:
                 description: |-
-                    If the domain is blocked, what's the publicly-stated reason for the block.
+                    If the domain is blocked or allowed, what's the publicly-stated reason (if any).
                     Alternative to `comment` to be used when serializing/deserializing NOT via /api/v1/instance.
                 example: they smell
                 type: string
                 x-go-name: PublicComment
+            severity:
+                description: |-
+                    Severity of this entry.
+                    Only ever set for domain blocks, and if set, always="suspend".
+                type: string
+                x-go-name: Severity
             silenced_at:
                 description: Time at which this domain was silenced. Key will not be present on open domains.
                 example: "2021-07-30T09:20:25+00:00"
@@ -1135,7 +1141,7 @@ definitions:
         properties:
             comment:
                 description: |-
-                    If the domain is blocked, what's the publicly-stated reason for the block.
+                    If the domain is blocked or allowed, what's the publicly-stated reason (if any).
                     Alternative to `public_comment` to be used when serializing/deserializing via /api/v1/instance.
                 example: they smell
                 type: string
@@ -1179,11 +1185,17 @@ definitions:
                 x-go-name: PrivateComment
             public_comment:
                 description: |-
-                    If the domain is blocked, what's the publicly-stated reason for the block.
+                    If the domain is blocked or allowed, what's the publicly-stated reason (if any).
                     Alternative to `comment` to be used when serializing/deserializing NOT via /api/v1/instance.
                 example: they smell
                 type: string
                 x-go-name: PublicComment
+            severity:
+                description: |-
+                    Severity of this entry.
+                    Only ever set for domain blocks, and if set, always="suspend".
+                type: string
+                x-go-name: Severity
             silenced_at:
                 description: Time at which this domain was silenced. Key will not be present on open domains.
                 example: "2021-07-30T09:20:25+00:00"
@@ -8928,37 +8940,105 @@ paths:
             summary: Update your instance information and/or upload a new avatar/header for the instance.
             tags:
                 - instance
+    /api/v1/instance/domain_allows:
+        get:
+            description: OAuth token may need to be provided depending on setting `instance-expose-allowlist`.
+            operationId: instanceDomainAllowsGet
+            produces:
+                - application/json
+            responses:
+                "200":
+                    description: List of explicitly allowed domains.
+                    schema:
+                        items:
+                            $ref: '#/definitions/domain'
+                        type: array
+                "400":
+                    description: bad request
+                "401":
+                    description: unauthorized
+                "403":
+                    description: forbidden
+                "404":
+                    description: not found
+                "406":
+                    description: not acceptable
+                "500":
+                    description: internal server error
+            security:
+                - OAuth2 Bearer: []
+            summary: List explicitly allowed domains.
+            tags:
+                - instance
+    /api/v1/instance/domain_blocks:
+        get:
+            description: OAuth token may need to be provided depending on setting `instance-expose-blocklist`.
+            operationId: instanceDomainBlocksGet
+            produces:
+                - application/json
+            responses:
+                "200":
+                    description: List of blocked domains.
+                    schema:
+                        items:
+                            $ref: '#/definitions/domain'
+                        type: array
+                "400":
+                    description: bad request
+                "401":
+                    description: unauthorized
+                "403":
+                    description: forbidden
+                "404":
+                    description: not found
+                "406":
+                    description: not acceptable
+                "500":
+                    description: internal server error
+            security:
+                - OAuth2 Bearer: []
+            summary: List blocked domains.
+            tags:
+                - instance
     /api/v1/instance/peers:
         get:
             operationId: instancePeersGet
             parameters:
-                - default: open
+                - default: flat
                   description: |-
                     Comma-separated list of filters to apply to results. Recognized filters are:
-                      - `open` -- include peers that are not suspended or silenced
-                      - `suspended` -- include peers that have been suspended.
+                      - `open` -- include known domains that are not in the domain blocklist
+                      - `allowed` -- include domains that are in the domain allowlist
+                      - `blocked` -- include domains that are in the domain blocklist
+                      - `suspended` -- DEPRECATED! Use `blocked` instead. Same as `blocked`: include domains that are in the domain blocklist;
+
+                    If filter is `open`, only domains that aren't in the blocklist will be shown.
 
-                    If filter is `open`, only instances that haven't been suspended or silenced will be returned.
+                    If filter is `blocked`, only domains that *are* in the blocklist will be shown.
 
-                    If filter is `suspended`, only suspended instances will be shown.
+                    If filter is `allowed`, only domains that are in the allowlist will be shown.
 
-                    If filter is `open,suspended`, then all known instances will be returned.
+                    If filter is `open,blocked`, then blocked domains and known domains not on the blocklist will be shown.
+
+                    If filter is `open,allowed`, then allowed domains and known domains not on the blocklist will be shown.
 
                     If filter is an empty string or not set, then `open` will be assumed as the default.
                   in: query
                   name: filter
                   type: string
+                - default: false
+                  description: If true, a "flat" array of strings will be returned corresponding to just domain names.
+                  in: query
+                  name: flat
+                  type: boolean
             produces:
                 - application/json
             responses:
                 "200":
                     description: |-
-                        If no filter parameter is provided, or filter is empty, then a legacy, Mastodon-API compatible response will be returned. This will consist of just a 'flat' array of strings like `["example.com", "example.org"]`, which corresponds to domains this instance peers with.
-
-                        If a filter parameter is provided, then an array of objects with at least a `domain` key set on each object will be returned.
-
+                        If no filter parameter is provided, or filter is empty, then a legacy, Mastodon-API compatible response will be returned. This will consist of just a 'flat' array of strings like `["example.com", "example.org"]`, which corresponds to setting a filter of `open` and flat=true.
+                        If a filter parameter is provided and flat is not true, then an array of objects with at least a `domain` key set on each object will be returned.
                         Domains that are silenced or suspended will also have a key `suspended_at` or `silenced_at` that contains an iso8601 date string. If one of these keys is not present on the domain object, it is open. Suspended instances may in some cases be obfuscated, which means they will have some letters replaced by `*` to make it more difficult for bad actors to target instances with harassment.
-
                         Whether a flat response or a more detailed response is returned, domains will be sorted alphabetically by hostname.
                     schema:
                         items:
@@ -8978,6 +9058,7 @@ paths:
                     description: internal server error
             security:
                 - OAuth2 Bearer: []
+            summary: List peer domains.
             tags:
                 - instance
     /api/v1/instance/rules:
diff --git a/docs/configuration/instance.md b/docs/configuration/instance.md
index 2a945eed2..1b8282a27 100644
--- a/docs/configuration/instance.md
+++ b/docs/configuration/instance.md
@@ -76,29 +76,61 @@ instance-federation-mode: "blocklist"
 # Default: false
 instance-federation-spam-filter: false
 
-# Bool. Allow unauthenticated users to make queries to /api/v1/instance/peers?filter=open in order
-# to see a list of instances that this instance 'peers' with. Even if set to 'false', then authenticated
-# users (members of the instance) will still be able to query the endpoint.
+# Bool. Allow unauthenticated users to make queries to /api/v1/instance/peers?filter=open
+# in order to see a list of domains that this instance 'peers' with.
+#
+# Even if set to 'false', then authenticated users (members of the instance)
+# will still be able to query these endpoints using an OAuth token.
+#
 # Options: [true, false]
 # Default: false
 instance-expose-peers: false
 
-# Bool. Allow unauthenticated users to make queries to /api/v1/instance/peers?filter=suspended in order
-# to see a list of instances that this instance blocks/suspends. Even if set to 'false', then authenticated
-# users (members of the instance) will still be able to query the endpoint.
+# Bool. Allow unauthenticated users to make queries to the following instance API
+# endpoints in order to see a list of domains that this instance explicitly blocks,
+# including the public reason for each block:
 #
-# WARNING: Setting this variable to 'true' may result in your instance being scraped by blocklist scrapers.
+# - /api/v1/instance/peers?filter=blocklist
+# - /api/v1/instance/peers?filter=suspended (deprecated)
+# - /api/v1/instance/domain_blocks
+#
+# Even if set to 'false', then authenticated users (members of the instance)
+# will still be able to query these endpoints using an OAuth token.
+#
+# WARNING: Setting to 'true' may result in your instance being targeted by blocklist scrapers.
 # See: https://docs.gotosocial.org/en/latest/admin/domain_blocks/#block-announce-bots
 #
 # Options: [true, false]
 # Default: false
-instance-expose-suspended: false
+instance-expose-blocklist: false
+
+# Bool. Allow unauthenticated users to view /about/domain_blocks,
+# which shows an HTML-rendered list of domains that this instance blocks,
+# including the public reason for each block.
+# Options: [true, false]
+# Default: false
+instance-expose-blocklist-web: false
+
+# Bool. Allow unauthenticated users to make queries to the following instance API
+# endpoints in order to see a list of domains that this instance explicitly allows
+# including the public reason for each allow:
+#
+# - /api/v1/instance/peers?filter=allowlist
+# - /api/v1/instance/domain_allows
+#
+# Even if set to 'false', then authenticated users (members of the instance)
+# will still be able to query these endpoints using an OAuth token.
+#
+# Options: [true, false]
+# Default: false
+instance-expose-allowlist: false
 
-# Bool. Allow unauthenticated users to view /about/suspended,
-# showing the HTML rendered list of instances that this instance blocks/suspends.
+# Bool. Allow unauthenticated users to view /about/domain_allows,
+# which shows an HTML-rendered list of domains that this instance allows,
+# including the public reason for each allow.
 # Options: [true, false]
 # Default: false
-instance-expose-suspended-web: false
+instance-expose-blocklist-web: false
 
 # Bool. Allow unauthenticated users to make queries to /api/v1/timelines/public in order
 # to see a list of public posts on this server. Even if set to 'false', then authenticated