summaryrefslogtreecommitdiff
path: root/internal/db/db.go
AgeCommit message (Collapse)AuthorFiles
2025-01-23[feature] Push notifications (#3587)Libravatar Vyr Cossont1
* Update push subscription API model to be Mastodon 4.0 compatible * Add webpush-go dependency # Conflicts: # go.sum * Single-row table for storing instance's VAPID key pair * Generate VAPID key pair during startup * Add VAPID public key to instance info API * Return VAPID public key when registering an app * Store Web Push subscriptions in DB * Add Web Push sender (similar to email sender) * Add no-op push senders to most processor tests * Test Web Push notifications from workers * Delete Web Push subscriptions when account is deleted * Implement push subscription API * Linter fixes * Update Swagger * Fix enum to int migration * Fix GetVAPIDKeyPair * Create web push subscriptions table with indexes * Log Web Push server error messages * Send instance URL as Web Push JWT subject * Accept any 2xx code as a success * Fix malformed VAPID sub claim * Use packed notification flags * Remove unused date columns * Add notification type for update notifications Not used yet * Make GetVAPIDKeyPair idempotent and remove PutVAPIDKeyPair * Post-rebase fixes * go mod tidy * Special-case 400 errors other than 408/429 Most client errors should remove the subscription. * Improve titles, trim body to reasonable length * Disallow cleartext HTTP for Web Push servers * Fix lint * Remove redundant index on unique column Also removes redundant unique and notnull tags on ID column since these are implied by pk * Make realsender.go more readable * Use Tobi's style for wrapping errors * Restore treating all 5xx codes as temporary problems * Always load target account settings * Stub `policy` and `standard` * webpush.Sender: take type converter as ctor param * Move webpush.MockSender and noopSender into testrig
2024-12-05[feature] add support for receiving federated status edits (#3597)Libravatar kim1
* add support for extracting Updated field from Statusable implementers * add support for status edits in the database, and update status dereferencer to handle them * remove unused AdditionalInfo{}.CreatedAt * remove unused AdditionalEmojiInfo{}.CreatedAt * update new mention creation to use status.UpdatedAt * remove mention.UpdatedAt, fixes related to NewULIDFromTime() change * add migration to remove Mention{}.UpdatedAt field * add migration to add the StatusEdit{} table * start adding tests, add delete function for status edits * add more of status edit migrations, fill in more of the necessary edit delete functionality * remove unused function * allow generating gotosocial compatible ulid via CLI with `go run ./cmd/gen-ulid` * add StatusEdit{} test models * fix new statusedits sql * use model instead of table name * actually remove the Mention.UpdatedAt field... * fix tests now new models are added, add more status edit DB tests * fix panic wording * add test for deleting status edits * don't automatically set `updated_at` field on updated statuses * flesh out more of the dereferencer status edit tests, ensure updated at field set on outgoing AS statuses * remove media_attachments.updated_at column * fix up more tests, further complete the dereferencer status edit tests * update more status serialization tests not expecting 'updated' AS property * gah!! json serialization tests!! * undo some gtscontext wrapping changes * more serialization test fixing :smiling_face_with_tear: * more test fixing, ensure the edit.status_id field is actually set :facepalm: * fix status edit test * grrr linter * add edited_at field to apimodel status * remove the choice of paging on the timeline public filtered test (otherwise it needs updating every time you add statuses ...) * ensure that status.updated_at always fits chronologically * fix more serialization tests ... * add more code comments * fix envparsing * update swagger file * properly handle media description changes during status edits * slight formatting tweak * code comment
2024-09-10[feature] Process `Reject` of interaction via fedi API, put rejected ↵Libravatar tobi1
statuses in the "sin bin" 😈 (#3271) * [feature] Process `Reject` of interaction via fedi API, put rejected statuses in the "sin bin" * update test * move nil check back to `rejectStatusIRI`
2024-07-30[feature] persist worker queues to db (#3042)Libravatar kim1
* persist queued worker tasks to database on shutdown, fill worker queues from database on startup * ensure the tasks are sorted by creation time before pushing them * add migration to insert WorkerTask{} into database, add test for worker task persistence * add test for recovering worker queues from database * quick tweak * whoops we ended up with double cleaner job scheduling * insert each task separately, because bun is throwing some reflection error?? * add specific checking of cancelled worker contexts * add http request signing to deliveries recovered from database * add test for outgoing public key ID being correctly set on delivery * replace select with Queue.PopCtx() * get rid of loop now we don't use it * remove field now we don't use it * ensure that signing func is set * header values weren't being copied over :facepalm: * use ptr for httpclient.Request in delivery * move worker queue filling to later in server init process * fix rebase issues * make logging less shouty * use slices.Delete() instead of copying / reslicing * have database return tasks in ascending order instead of sorting them * add a 1 minute timeout to persisting worker queues
2024-07-23[feature] Conversations API (#3013)Libravatar Vyr Cossont1
* Implement conversations API * Sort and page conversations by last status ID * Appease linter * Fix deleting conversations and statuses * Refactor to make migrations automatic * Lint * Update tests post-merge * Fixes from live-fire testing * Linter caught a format problem * Refactor tests, fix cache * Negative test for non-DMs * Run conversations advanced migration on testrig startup as well as regular server startup * Document (lack of) side effects of API method for deleting a conversation * Make not-found check less nested for readability * Rename PutConversation to UpsertConversation * Use util.Ptr instead of IIFE * Reduce cache used by conversations * Remove unnecessary TableExpr/ColumnExpr * Use struct tags for both unique constraints on Conversation * Make it clear how paging with GetDirectStatusIDsBatch should be used * Let conversation paging skip conversations it can't render * Use Bun NewDropTable * Convert delete raw query to Bun * Convert update raw query to Bun * Convert latestConversationStatusesTempTable raw query partially to Bun * Convert conversationStatusesTempTable raw query partially to Bun * Rename field used to store result of MaxDirectStatusID * Move advanced migrations to their own tiny processor * Catch up util function name with main * Remove json.… wrappers * Remove redundant check * Combine error checks * Replace map with slice of structs * Address processor/type converter comments - Add context info for errors - Extract some common processor code into shared methods - Move conversation eligibility check ahead of populating conversation * Add error context when dropping temp tables
2024-07-11[chore] Add interaction policy gtsmodels (#3075)Libravatar tobi1
* [chore] introduce interaction policy gts models * update migration a smidge * fix copy paste typo * update migration * use int for InteractionType
2024-03-06[feature/chore] Add Move database functions + cache (#2647)Libravatar tobi1
* [feature/chore] Add Move database functions + cache * add move mem ratio to envparsing.sh * update comment
2024-03-06[feature] Filters v1 (#2594)Libravatar Vyr Cossont1
* Implement client-side v1 filters * Exclude linter false positives * Update test/envparsing.sh * Fix minor Swagger, style, and Bun usage issues * Regenerate Swagger * De-generify filter keywords * Remove updating filter statuses This is an operation that the Mastodon v2 filter API doesn't actually have, because filter statuses, unlike keywords, don't have options: the only info they contain is the status ID to be filtered. * Add a test for filter statuses specifically * De-generify filter statuses * Inline FilterEntry * Use vertical style for Bun operations consistently * Add comment on Filter DB interface * Remove GoLand linter control comments Our existing linters should catch these, or they don't matter very much * Reduce memory ratio for filters
2023-12-18[feature] request blocking by http headers (#2409)Libravatar kim1
2023-11-08[feature] add support for polls + receiving federated status edits (#2330)Libravatar kim1
2023-10-25[feature] Status thread mute/unmute functionality (#2278)Libravatar tobi1
* add db models + functions for keeping track of threads * give em the old linty testy * create, remove, check mutes * swagger * testerino * test mute/unmute via api * add info log about new index creation * thread + allow muting of any remote statuses that mention a local account * IsStatusThreadMutedBy -> IsThreadMutedByAccount * use common processing functions in status processor * set = NULL * favee! * get rekt darlings, darlings get rekt * testrig please, have mercy muy liege
2023-08-19[feature] Instance rules (#2125)Libravatar f0x521
* init instance rules database model, admin api * expose instance rules in public instance api * public /api/v1/instance/rules route * GET ruleById * createRule route * createRule auth check * updateRule * deleteRule * list rules on about page * ruleGet auth * add about page ids for anchors * process and store adding violated rules to reports * admin api models for instance rules * instance rule edit frontend * change rule inputs to textareas * database fixes after rebase (#2124) * remove unused imports * fix db migration column name * fix tests * fix more tests * fix postgres error with wrongly used Ident * add some tests, fiddle with rule model a bit, fix postgres migration * swagger docs --------- Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-08-10[performance] remove last of relational queries to instead rely on caches ↵Libravatar kim1
(#2091)
2023-07-31[feature] Hashtag federation (in/out), hashtag client API endpoints (#2032)Libravatar tobi1
* update go-fed * do the things * remove unused columns from tags * update to latest lingo from main * further tag shenanigans * serve stub page at tag endpoint * we did it lads * tests, oh tests, ohhh tests, oh tests (doo doo doo doo) * swagger docs * document hashtag usage + federation * instanceGet * don't bother parsing tag href * rename whereStartsWith -> whereStartsLike * remove GetOrCreateTag * dont cache status tag timelineability
2023-07-29[feature] Implement markers API (#1989)Libravatar Vyr Cossont1
* Implement markers API Fixes #1856 * Correct import grouping in markers files * Regenerate Swagger for markers API * Shorten names for readability * Cache markers for 6 hours * Update DB ref * Update envparsing.sh
2023-06-21[feature] Add partial text search for accounts + statuses (#1836)Libravatar tobi1
2023-05-25[feature] Add List functionality (#1802)Libravatar tobi1
* start working on lists * further list work * test list db functions nicely * more work on lists * peepoopeepoo * poke * start list timeline func * we're getting there lads * couldn't be me working on stuff... could it? * hook up handlers * fiddling * weeee * woah * screaming, pissing * fix streaming being a whiny baby * lint, small test fix, swagger * tidying up, testing * fucked! by the linter * move timelines to state like a boss * add timeline start to tests using state * invalidate lists
2023-03-20[chore] Refactor account deleting/block logic, tidy up some other processing ↵Libravatar tobi1
things (#1599) * start refactoring account deletion * update to use state.DB * further messing about * some more tidying up * more tidying, cleaning, nice-making * further adventures in refactoring and the woes of technical debt * update fr accept/reject * poking + prodding * fix up deleting * create fave uri * don't log using requestingAccount.ID because it might be nil * move getBookmarks function * use exists query to check for status bookmark * use deletenotifications func * fiddle * delete follow request notif * split up some db functions * Fix possible nil pointer panic * fix more possible nil pointers * fix license headers * warn when follow missing (target) account * return wrapped err when bookmark/fave models can't be retrieved * simplify self account delete * warn log likely race condition * de-sillify status delete loop * move error check due north * warn when unfollowSideEffects has no target account * warn when no boost account is found * warn + dump follow when no account * more warnings * warn on fave account not set * move for loop inside anonymous function * fix funky logic * don't remove mutual account items on block; do make sure unfollow occurs in both directions!
2023-03-12[chore] Improve copyright header handling (#1608)Libravatar Daenney1
* [chore] Remove years from all license headers Years or year ranges aren't required in license headers. Many projects have removed them in recent years and it avoids a bit of yearly toil. In many cases our copyright claim was also a bit dodgy since we added the 2021-2023 header to files created after 2021 but you can't claim copyright into the past that way. * [chore] Add license header check This ensures a license header is always added to any new file. This avoids maintainers/reviewers needing to remember to check for and ask for it in case a contribution doesn't include it. * [chore] Add missing license headers * [chore] Further updates to license header * Use the more common // indentend comment format * Remove the hack we had for the linter now that we use the // format * Add SPDX license identifier
2023-02-03[chore] Text formatting overhaul (#1406)Libravatar Autumn!1
* Implement goldmark debug print for hashtags and mentions * Minify HTML in FromPlain * Convert plaintext status parser to goldmark * Move mention/tag/emoji finding logic into formatter * Combine mention and hashtag boundary characters * Normalize unicode when rendering hashtags
2023-01-10[feature] Implement Report database model and utility functions (#1310)Libravatar tobi1
* implement report database model * implement report cache + config changes * implement report database functions * report uri / regex functions * update envparsing test * remove unnecessary uri index * remove unused function + cache lookup * process error when storing report
2023-01-05[chore] Update/add license headers for 2023 (#1304)Libravatar tobi1
2023-01-02[chore] The Big Middleware and API Refactor (tm) (#1250)Libravatar tobi1
* interim commit: start refactoring middlewares into package under router * another interim commit, this is becoming a big job * another fucking massive interim commit * refactor bookmarks to new style * ambassador, wiz zeze commits you are spoiling uz * she compiles, we're getting there * we're just normal men; we're just innocent men * apiutil * whoopsie * i'm glad noone reads commit msgs haha :blob_sweat: * use that weirdo go-bytesize library for maxMultipartMemory * fix media module paths
2022-11-11[feature] Read + Write tombstones for deleted Actors (#1005)Libravatar tobi1
* [feature] Read + Write tombstones for deleted Actors * copyTombstone * update to use resultcache instead of old ttl cache Signed-off-by: kim <grufwub@gmail.com> * update go-cache library to fix result cache capacity / ordering bugs Signed-off-by: kim <grufwub@gmail.com> * bump go-cache/v3 to v3.1.6 to fix bugs Signed-off-by: kim <grufwub@gmail.com> * switch on status code * better explain ErrGone reasoning Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: kim <grufwub@gmail.com>
2022-10-03[performance] add user cache and database (#879)Libravatar tobi1
* go fmt * add + use user cache and database * fix import * update tests * remove unused relation
2022-09-06[feature] Custom emoji updates (serve emoji via s2s api, tune db models) (#805)Libravatar tobi1
* migrate emojis * add get emoji to s2s (federation) API * add new emoji db + cache functions * add shortcodeDomain lookup for emojis * check existing emojis w/cache, not w/constraints * go fmt * add putEmoji func * use new db emoji funcs instead of where * remove emojistringstotags func * add unique constraint back in * fix up broken migration * update index
2022-05-20[feature] implement custom_emojis endpoint (#563)Libravatar Adelie Paull1
* implement custom_emojis api endpoint * add tests for getting custom emoji out of the database and converting to api emoji * change sort direction of emoji query * change logging level and initialize array with known length as per kim's suggestions * add continue to lessen risk of making a malformed struct during conversion from db to api emojis
2022-03-29[feature] Dereference remote mentions when the account is not already known ↵v0.2.2Libravatar tobi1
(#442) * remove mention util function from db * add ParseMentionFunc to gtsmodel * add parseMentionFunc to processor * refactor search to simplify it a bit * add parseMentionFunc to account * add parseMentionFunc to status * some renaming for clarity * test dereference of unknown mentioned account
2021-12-20Extend license notices to 2022 (#354)Libravatar tobi1
2021-09-11kim is a reply guy (#208)Libravatar tobi1
* bun debug * bun trace logging hooks * more tests * fix up some stuffffff * drop the frontend cache until a proper fix is made * go fmt
2021-08-25Pg to bun (#148)Libravatar tobi1
* start moving to bun * changing more stuff * more * and yet more * tests passing * seems stable now * more big changes * small fix * little fixes
2021-08-20Database updates (#144)Libravatar tobi1
* start moving some database stuff around * continue moving db stuff around * more fiddling * more updates * and some more * and yet more * i broke SOMETHING but what, it's a mystery * tidy up * vendor ttlcache * use ttlcache * fix up some tests * rename some stuff * little reminder * some more updates
2021-08-10Dereference remote replies (#132)Libravatar Tobi Smethurst1
* decided where to put reply dereferencing * fiddling with dereferencing threads * further adventures * tidy up some stuff * move dereferencing functionality * a bunch of refactoring * go fmt * more refactoring * bleep bloop * docs and linting * start implementing replies collection on gts side * fiddling around * allow dereferencing our replies * lint, fmt
2021-07-23Oidc (#109)Libravatar Tobi Smethurst1
* add oidc config * inching forward with oidc idp * lil webfingy fix * bit more progress * further oidc * oidc now working * document dex config * replace broken images * add additional credits * tiny doc update * update * add oidc config * inching forward with oidc idp * bit more progress * further oidc * oidc now working * document dex config * replace broken images * add additional credits * tiny doc update * update * document * docs + comments
2021-07-11Block/unblock (#96)Libravatar Tobi Smethurst1
* remote + local block logic, incl. federation * improve blocking stuff * fiddle with display of blocked profiles * go fmt
2021-07-09favourites GET implementation (#95)Libravatar Tobi Smethurst1
2021-07-06Blocklist import (#77)Libravatar Tobi Smethurst1
* first steps on importing blocklists * unblock domains properly
2021-07-05Domain block (#76)Libravatar Tobi Smethurst1
* start work on admin domain blocking * move stuff around + further work on domain blocks * move + restructure processor * prep work for deleting account * tidy * go fmt * formatting * domain blocking more work * check domain blocks way earlier on * progress on delete account * delete more stuff when an account is gone * and more... * domain blocky block block * get individual domain block, delete a block
2021-06-23Instance settings updates (#59)Libravatar Tobi Smethurst1
Allow admins to set instance settings through a PATCH to /api/v1/instance Update templates to reflect some of the new fields
2021-06-17Timeline improvements (#41)Libravatar Tobi Smethurst1
Tidying up. Parent/child statuses now display correctly in status/id/context.
2021-06-13Timeline manager (#40)Libravatar Tobi Smethurst1
* start messing about with timeline manager * i have no idea what i'm doing * i continue to not know what i'm doing * it's coming along * bit more progress * update timeline with new posts as they come in * lint and fmt * Select accounts where empty string * restructure a bunch, get unfaves working * moving stuff around * federate status deletes properly * mention regex better but not 100% there * fix regex * some more hacking away at the timeline code phew * fix up some little things * i can't even * more timeline stuff * move to ulid * fiddley * some lil fixes for kibou compatibility * timelines working pretty alright! * tidy + lint
2021-05-31Tidy + timeline embetterment (#38)Libravatar Tobi Smethurst1
* tidy up timelines a bit + stub out some endpoints * who's faved and who's boosted, reblog notifs * linting * Update progress with new endpoints
2021-05-30Move a lot of stuff + tidy stuff (#37)Libravatar Tobi Smethurst1
Lots of renaming and moving stuff, some bug fixes, more lenient parsing of notifications and home timeline.
2021-05-29first implementation of search featureLibravatar tsmethurst1
2021-05-27Notifications (#34)Libravatar Tobi Smethurst1
Notifications working for: * Mentions * Faves * New follow requests * New followers
2021-05-24Faves (#31)Libravatar Tobi Smethurst1
* start on federating faves * outbound federation of likes working
2021-05-21Home timeline (#28)Libravatar Tobi Smethurst1
* v. basic implementation of home timeline * Go fmt ./...
2021-05-21Follows and relationships (#27)Libravatar Tobi Smethurst1
* Follows -- create and undo, both remote and local * Statuses -- federate new posts, including media, attachments, CWs and image descriptions.
2021-05-17Refine statuses (#26)Libravatar Tobi Smethurst1
Remote media is now dereferenced and attached properly to incoming federated statuses. Mentions are now dereferenced and attached properly to incoming federated statuses. Small fixes to status visibility. Allow URL params for filtering statuses: // ExcludeRepliesKey is for specifying whether to exclude replies in a list of returned statuses by an account. // PinnedKey is for specifying whether to include pinned statuses in a list of returned statuses by an account. // MaxIDKey is for specifying the maximum ID of the status to retrieve. // MediaOnlyKey is for specifying that only statuses with media should be returned in a list of returned statuses by an account. Add endpoint for fetching an account's statuses.
2021-05-15Inbox post (#22)Libravatar Tobi Smethurst1
Inbox POST from federated servers now working for statuses and follow requests. Follow request client API added. Start work on federating outgoing messages. Other fixes and changes/tidying up.