summaryrefslogtreecommitdiff
path: root/internal/processing
AgeCommit message (Collapse)AuthorFiles
2024-05-22[performance] update storage backend and make use of seek syscall when ↵Libravatar kim1
available (#2924) * update to use go-storage/ instead of go-store/v2/storage/ * pull in latest version from codeberg * remove test output :innocent: * add code comments * set the exclusive bit when creating new files in disk config * bump to actual release version * bump to v0.1.1 (tis a simple no-logic change) * update readme * only use a temporary read seeker when decoding video if required (should only be S3 now) * use fastcopy library to use memory pooled buffers when calling TempFileSeeker() * update to use seek call in serveFileRange()
2024-05-06feature: filters v2 server-side warning/hiding (#2793)Libravatar Vyr Cossont15
* Remove dead code * Filter statuses when converting to frontend representation * status.filtered is an array * Make matching case-insensitive * Remove TODOs that don't need to be done now * Add missing filter check for notification * lint: rename ErrHideStatus * APIFilterActionToFilterAction not used yet * swaggerino docseroni * Address review comments * Add apimodel.FilterActionNone --------- Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com> Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-05-02[bugfix] Lock when checking/creating notifs to avoid race (#2890)Libravatar tobi14
* [bugfix] Lock when checking/creating notifs to avoid race * test notif spam
2024-05-01[feature] Page through accounts as moderator (#2881)Libravatar tobi1
* [feature] Page through accounts as moderator * aaaaa * use COLLATE "C" for Postgres to ensure same ordering as SQLite * fix typo, test paging up * don't show moderation / info for our instance acct
2024-04-29[chore] Fix conflict in workers tests (#2880)Libravatar tobi3
* [chore] Fix conflict in workers tests * commenty-wenty
2024-04-26[bugfix] Fix error string typo (#2873)Libravatar kim1
2024-04-26[performance] update remaining worker pools to use queues (#2865)Libravatar kim34
* start replacing client + federator + media workers with new worker + queue types * refactor federatingDB.Delete(), drop queued messages when deleting account / status * move all queue purging to the processor workers * undo toolchain updates * code comments, ensure dereferencer worker pool gets started * update gruf libraries in readme * start the job scheduler separately to the worker pools * reshuffle ordering or server.go + remove duplicate worker start / stop * update go-list version * fix vendoring * move queue invalidation to before wipeing / deletion, to ensure queued work not dropped * add logging to worker processing functions in testrig, don't start workers in unexpected places * update go-structr to add (+then rely on) QueueCtx{} type * ensure more worker pools get started properly in tests * fix remaining broken tests relying on worker queue logic * fix account test suite queue popping logic, ensure noop workers do not pull from queue * move back accidentally shuffled account deletion order * ensure error (non nil!!) gets passed in refactored federatingDB{}.Delete() * silently drop deletes from accounts not permitted to * don't warn log on forwarded deletes * make if else clauses easier to parse * use getFederatorMsg() * improved code comment * improved code comment re: requesting account delete checks * remove boolean result from worker start / stop since false = already running or already stopped * remove optional passed-in http.client * remove worker starting from the admin CLI commands (we don't need to handle side-effects) * update prune cli to start scheduler but not all of the workers * fix rebase issues * remove redundant return statements * i'm sorry sir linter
2024-04-17[feature] Status source endpoint (#2848)Libravatar tobi1
* [feature] statusSource endpoint * finish up
2024-04-17[feature] Stub status history endpoint (#2847)Libravatar tobi1
2024-04-16[feature/performance] Store account stats in separate table (#2831)Libravatar tobi13
* [feature/performance] Store account stats in separate table, get stats from remote * test account stats * add some missing increment / decrement calls * change stats function signatures * rejig logging a bit * use lock when updating stats
2024-04-15[performance] cached oauth database types (#2838)Libravatar kim1
* update token + client code to use struct caches * add code comments * slight tweak to default mem ratios * fix envparsing * add appropriate invalidate hooks * update the tokenstore sweeping function to rely on caches * update to use PutClient() * add ClientID to list of token struct indices
2024-04-13[feature] Admin accounts endpoints; approve/reject sign-ups (#2826)Libravatar tobi9
* update settings panels, add pending overview + approve/deny functions * add admin accounts get, approve, reject * send approved/rejected emails * use signup URL * docs! * email * swagger * web linting * fix email tests * wee lil fixerinos * use new paging logic for GetAccounts() series of admin endpoints, small changes to query building * shuffle useAccountIDIn check *before* adding to query * fix parse from toot react error * use `netip.Addr` * put valid slices in globals * optimistic updates for account state --------- Co-authored-by: kim <grufwub@gmail.com>
2024-04-11[feature] New user sign-up via web page (#2796)Libravatar tobi10
* [feature] User sign-up form and admin notifs * add chosen + filtered languages to migration * remove stray comment * chosen languages schmosen schmanguages * proper error on local account missing
2024-04-11[performance] massively improved ActivityPub delivery worker efficiency (#2812)Libravatar kim4
* add delivery worker type that pulls from queue to httpclient package * finish up some code commenting, bodge a vendored activity library change, integrate the deliverypool changes into transportcontroller * hook up queue deletion logic * support deleting queued http requests by target ID * don't index APRequest by hostname in the queue * use gorun * use the original context's values when wrapping msg type as delivery{} * actually log in the AP delivery worker ... * add uncommitted changes * use errors.AsV2() * use errorsv2.AsV2() * finish adding some code comments, add bad host handling to delivery workers * slightly tweak deliveryworkerpool API, use advanced sender multiplier * remove PopCtx() method, let others instead rely on Wait() * shuffle things around to move delivery stuff into transport/ subpkg * remove dead code * formatting * validate request before queueing for delivery * finish adding code comments, fix up backoff code * finish adding more code comments * clamp minimum no. senders to 1 * add start/stop logging to delivery worker, some slight changes * remove double logging * use worker ptrs * expose the embedded log fields in httpclient.Request{} * ensure request context values are preserved when updating ctx * add delivery worker tests * fix linter issues * ensure delivery worker gets inited in testrig * fix tests to delivering messages to check worker delivery queue * update error type to use ptr instead of value receiver * fix test calling Workers{}.Start() instead of testrig.StartWorkers() * update docs for advanced-sender-multiplier * update to the latest activity library version * add comment about not using httptest.Server{}
2024-04-08[bugfix] Ensure side effects for local -> local follows get processed (#2820)v0.15.0-rc2Libravatar tobi2
2024-04-02[bugfix] Set the `Host` header within the signing transport (#2799)Libravatar kim1
2024-04-02[feature] Option to hide followers/following (#2788)Libravatar tobi4
2024-03-28[bugfix] Serve correct URI for AP following collection (#2787)Libravatar tobi1
2024-03-25[feature] User-selectable preset CSS themes for accounts (#2777)Libravatar tobi4
* [feature] User-selectable preset themes * docs, more theme stuff * lint, tests * fix css name * correct some little issues * add another theme * fix poll background * okay last theme i swear * make retrieval of apimodel themes more conventional * preallocate stylesheet slices
2024-03-25[bugfix] Avoid empty public/local timeline queries (#2784)Libravatar tobi3
2024-03-22[chore] Move local account settings to separate db table (#2770)Libravatar tobi8
* [chore] Move local account settings to separate database model * don't use separate settings_id
2024-03-14[bugfix] Don't return 500 when searching for unpermitted status (#2753)Libravatar tobi1
2024-03-13[chore] Expose move endpoint again, small settings panel fixes (#2752)Libravatar tobi1
2024-03-13[feature] Process outgoing Move from clientAPI (#2750)Libravatar tobi9
* prevent moved accounts from taking create-type actions * update move logic * federate move out * indicate on web profile when an account has moved * [docs] Add migration docs section * lock while checking + setting move state * use redirectFollowers func for clientAPI as well * comment typo * linter? i barely know 'er! * Update internal/uris/uri.go Co-authored-by: Daenney <daenney@users.noreply.github.com> * add a couple tests for move * fix little mistake exposed by tests (thanks tests) * ensure Move marked as successful * attach shared util funcs to struct * lock whole account when doing move * move moving check to after error check * replace repeated text with error func * linterrrrrr!!!! * catch self follow case --------- Co-authored-by: Daenney <daenney@users.noreply.github.com>
2024-03-12[feature] Process incoming `Move` activity (#2724)Libravatar tobi3
* [feature] Process incoming account Move activity * fix targetAcct typo * put move origin account on fMsg * shift more move functionality back to the worker fn * simplify error logic
2024-03-06[feature] Filters v1 (#2594)Libravatar Vyr Cossont8
* Implement client-side v1 filters * Exclude linter false positives * Update test/envparsing.sh * Fix minor Swagger, style, and Bun usage issues * Regenerate Swagger * De-generify filter keywords * Remove updating filter statuses This is an operation that the Mastodon v2 filter API doesn't actually have, because filter statuses, unlike keywords, don't have options: the only info they contain is the status ID to be filtered. * Add a test for filter statuses specifically * De-generify filter statuses * Inline FilterEntry * Use vertical style for Bun operations consistently * Add comment on Filter DB interface * Remove GoLand linter control comments Our existing linters should catch these, or they don't matter very much * Reduce memory ratio for filters
2024-03-04[bugfix] check remote status permissibility (#2703)Libravatar kim1
* add more stringent checks for remote status permissibility * add check for inreplyto of a remote status being a boost * do not permit inReplyTo boost wrapper statuses * change comment wording * fix calls to NewFederator() * add code comments for NotPermitted() and SetNotPermitted() * improve comment * check that existing != nil before attempting delete * ensure replying account isn't suspended * use a debug log instead of info. check for boost using ID * shorten log string length. make info level * add note that replying to boost wrapper status shouldn't be able to happen anyways * update to use onFail() function
2024-02-29[bugfix] unwrap boosts when checking in-reply-to status (#2702)Libravatar kim1
* add stronger checks on status being replied to * update error code test is expecting
2024-02-27[feature] Add experimental `instance-federation-spam-filter` option (#2685)Libravatar tobi17
* [chore] Move `visibility` to `filter/visibility` * [feature] Add experimental instance-federation-spam-filter option
2024-02-20[bugfix] fix possible mutex lockup during streaming code (#2633)Libravatar kim12
* rewrite Stream{} to use much less mutex locking, update related code * use new context for the stream context * ensure stream gets closed on return of writeTo / readFrom WSConn() * ensure stream write timeout gets cancelled * remove embedded context type from Stream{}, reformat log messages for consistency * use c.Request.Context() for context passed into Stream().Open() * only return 1 boolean, fix tests to expect multiple stream types in messages * changes to ping logic * further improved ping logic * don't export unused function types, update message sending to only include relevant stream type * ensure stream gets closed :facepalm: * update to error log on failed json marshal (instead of panic) * inverse websocket read error checking to _ignore_ expected close errors
2024-02-19[bugfix] Ensure local statuses always get a threadID so they can be muted ↵Libravatar tobi2
(#2665) * [chore/bugfix] Ensure threadID always set on local statuses * test
2024-02-19[bugfix] Refactor parse mention, fix local mention bug (#2657)Libravatar tobi6
* [bugfix] Refactor parse mention, fix local mention bug * originAccount -> originAcct
2024-02-09use pointer for freshness window (#2614)Libravatar tobi11
2024-02-01stop paged endpoints returning null for empty items (#2597)Libravatar kim2
2024-01-31Improve context descendant sorting (#2579)Libravatar Vyr Cossont2
* Improve context descendant sorting Topologically sort replies, then move self-replies to top of list * Unify descendant sort passes * Correct test package name * Preallocate maps
2024-01-29[bugfix] Fix Postgres emoji delete, emoji category change (#2570)Libravatar tobi4
* [bugfix] Fix Postgres emoji delete, emoji category change * revert trace logging * caching issue * update tests
2024-01-26[bugfix] Don't return Account or Status if new and dereferencing failed, ↵Libravatar tobi1
other small fixes (#2563) * tidy up account, status, webfingering logic a wee bit * go fmt * invert published check * alter resp initialization * get Published from account in typeutils * don't instantiate error for no darn good reason * shadow err * don't repeat error codes in wrapped errors * don't wrap error unnecessarily
2024-01-20[bugfix] Prevent URL + URI for same account being used as alias target (#2545)Libravatar tobi2
* [bugfix] Ensure URL and URI for same account can't both be provided as alias * test whoopsie from previous PR
2024-01-19[bugfix] Ensure domain block side effects skipped if allow in place ↵Libravatar tobi2
(blocklist mode) (#2542)
2024-01-19[performance] overhaul struct (+ result) caching library for simplicity, ↵Libravatar kim1
performance and multiple-result lookups (#2535) * rewrite cache library as codeberg.org/gruf/go-structr, implement in gotosocial * use actual go-structr release version (not just commit hash) * revert go toolchain changes (damn you go for auto changing this) * fix go mod woes * ensure %w is used in calls to errs.Appendf() * fix error checking * fix possible panic * remove unnecessary start/stop functions, move to main Cache{} struct, add note regarding which caches require start/stop * fix copy-paste artifact... :innocent: * fix all comment copy-paste artifacts * remove dropID() function, now we can just use slices.DeleteFunc() * use util.Deduplicate() instead of collate(), move collate to util * move orderByIDs() to util package and "generify" * add a util.DeleteIf() function, use this to delete entries on failed population * use slices.DeleteFunc() instead of util.DeleteIf() (i had the logic mixed up in my head somehow lol) * add note about how collate differs from deduplicate
2024-01-17[chore] update viper version (#2539)Libravatar kim1
* update viper version * removes our last uses of the slice package * fix tests
2024-01-16[feature] Account alias / move API + db models (#2518)Libravatar tobi6
* [feature] Account alias / move API + db models * go fmt * fix little cherry-pick issues * update error checking, formatting * add and use new util functions to simplify alias logic
2024-01-05[feature] Parse instance descriptors as markdown, show T&C on /about (#2481)Libravatar tobi2
* [feature] Parse instance descriptors as markdown, show T&C on /about * lint * remove unnecessary nullzero tags
2024-01-03[bugfix] fix check for closed poll to account for non-zero closed time but ↵Libravatar kim1
in the future (#2486)
2023-12-27[chore] Refactor HTML templates and CSS (#2480)Libravatar tobi1
* [chore] Refactor HTML templates and CSS * eslint * ignore "Local" * rss tests * fiddle with OG just a tiny bit * dick around with polls a bit more so SR stops saying "clickable" * remove break * oh lord * don't lazy load avatar * fix ogmeta tests * clean up some cruft * catch remaining calls to c.HTML * fix error rendering + stack overflow in tag * allow templating attributes * fix indent * set aria-hidden on status complementary content, since it's already present in the label anyway * tidy up templating calls a little * try to make styling a bit more consistent + readable * fix up some remaining CSS issues * fix up reports
2023-12-18[feature] request blocking by http headers (#2409)Libravatar kim1
2023-12-16[feature] Push status edit messages into open streams (#2418)Libravatar Sam Lade6
* push status edit messages into open streams * fix a few comments * test++ * commented out code? moi?
2023-12-15[bugfix] use a much shorter refresh limit for statuses with polls (#2453)v0.13.0Libravatar kim9
* specifically use a much shorter refresh limit for statuses with polls * allow specifying whether status must be upToDate in calls to Get(Visible)?TargetStatusBy_(), limit force refresh to 5 minute cooldown * remove the PollID check from statusUpToDate() * remove unnecessary force flag checks * remove unused field * check refresh status error * use argument name 'refresh' instead of 'upToDate' to better fit with the codebase * add statuses_poll_id_idx * remove the definitely-not copy-pasted comment i accidentally typed out in full * only synchronously refresh if the refresh flag is provided, otherwise do async * fix wrong force value being provided for async --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2023-12-10[bugfix] Narrow search scope for accounts starting with '@'; don't LOWER ↵Libravatar tobi2
SQLite text searches (#2435)
2023-12-10[bugfix] Ensure `pre` renders as expected, fix orderedCollectionPage (#2434)Libravatar tobi1