summaryrefslogtreecommitdiff
path: root/internal/processing
AgeCommit message (Collapse)AuthorFiles
2024-07-17[feature] Allow users to set default interaction policies per status ↵Libravatar tobi3
visibility (#3108) * [feature] Allow users to set default interaction policies * use vars for default policies * avoid some code repetition * unfuck form binding * avoid bonkers loop * beep boop * put policyValsToAPIPolicyVals in separate function * don't bother with slices.Grow * oops
2024-07-15[feature/frontend] Add player for audio files; use thumbnail for `poster` ↵Libravatar tobi1
(#3099) * [feature/frontend] Audio player for audio media types * use video preview images for previews instead of video itself * don't preload * update tests for new zork status * collapse media gallery into single row when small
2024-07-13[chore/frontend] Tweak threading a bit, inform about hidden replies (#3097)Libravatar tobi1
* [chore/frontend] Tweak threading a bit, inform about hidden replies * whoops * round off bottom of replies col-header if no replies visible
2024-07-12[feature/frontend] Better visual separation between "main" thread and ↵Libravatar tobi4
"replies" (#3093) * [feature/frontend] Better web threading model * fix test * bwap * tweaks * more tweaks to wording * typo * indenting * adjust wording * aaa
2024-07-12[feature] support processing of (many) more media types (#3090)Libravatar kim5
* initial work replacing our media decoding / encoding pipeline with ffprobe + ffmpeg * specify the video codec to use when generating static image from emoji * update go-storage library (fixes incompatibility after updating go-iotools) * maintain image aspect ratio when generating a thumbnail for it * update readme to show go-ffmpreg * fix a bunch of media tests, move filesize checking to callers of media manager for more flexibility * remove extra debug from error message * fix up incorrect function signatures * update PutFile to just use regular file copy, as changes are file is on separate partition * fix remaining tests, remove some unneeded tests now we're working with ffmpeg/ffprobe * update more tests, add more code comments * add utilities to generate processed emoji / media outputs * fix remaining tests * add test for opus media file, add license header to utility cmds * limit the number of concurrently available ffmpeg / ffprobe instances * reduce number of instances * further reduce number of instances * fix envparsing test with configuration variables * update docs and configuration with new media-{local,remote}-max-size variables
2024-07-11[chore] Add interaction policy gtsmodels (#3075)Libravatar tobi5
* [chore] introduce interaction policy gts models * update migration a smidge * fix copy paste typo * update migration * use int for InteractionType
2024-07-08[feature/frontend] Allow setting alt-text for avatar + header (#3086)Libravatar tobi2
2024-07-04[bugfix] Handle ErrHideStatus when preparing timeline statuses (#3071)Libravatar Vyr Cossont3
2024-07-03[chore] Allow gtsmodel to depend on util (#3068)Libravatar Vyr Cossont1
Convert one free function into a gtsmodel.Emoji method so that util does not depend on gtsmodel and can be used from it in the future
2024-06-30[bugfix] update SignupURL routing in email template. (#3055)Libravatar CDN1
2024-06-26[chore] media and emoji refactoring (#3000)Libravatar kim17
* start updating media manager interface ready for storing attachments / emoji right away * store emoji and media as uncached immediately, then (re-)cache on Processing{}.Load() * remove now unused media workers * fix tests and issues * fix another test! * fix emoji activitypub uri setting behaviour, fix remainder of test compilation issues * fix more tests * fix (most of) remaining tests, add debouncing to repeatedly failing media / emojis * whoops, rebase issue * remove kim's whacky experiments * do some reshuffling, ensure emoji uri gets set * ensure marked as not cached on cleanup * tweaks to media / emoji processing to handle context canceled better * ensure newly fetched emojis actually get set in returned slice * use different varnames to be a bit more obvious * move emoji refresh rate limiting to dereferencer * add exported dereferencer functions for remote media, use these for recaching in processor * add check for nil attachment in updateAttachment() * remove unused emoji and media fields + columns * see previous commit * fix old migrations expecting image_updated_at to exists (from copies of old models) * remove freshness checking code (seems to be broken...) * fix error arg causing nil ptr exception * finish documentating functions with comments, slight tweaks to media / emoji deref error logic * remove some extra unneeded boolean checking * finish writing documentation (code comments) for exported media manager methods * undo changes to migration snapshot gtsmodels, updated failing migration to have its own snapshot * move doesColumnExist() to util.go in migrations package
2024-06-24[chore]: Bump github.com/gorilla/feeds from 1.1.2 to 1.2.0 (#3035)Libravatar dependabot[bot]1
2024-06-18[bugfix] rename `include_types[]` to `types[]` (#3023)Libravatar tobi1
2024-06-18[feature/frontend] Reports frontend v2 (#3022)Libravatar tobi2
* use apiutil + paging in admin processor+handlers * we're making it happen * fix little whoopsie * styling for report list * don't youuuu forget about meee don't don't don't don't * last bits * sanitize content before showing in report statuses * update report docs
2024-06-17[feature] Implement types[] param for notifications (#3009)Libravatar Vyr Cossont2
Counterpart of exclude_types[]. Also updates Swagger spec for types[] to use the correct param name and enumerate possible values. Fixes #3003
2024-06-14[feature] filter API v2: Restore keywords_attributes and statuses_attributes ↵Libravatar Vyr Cossont2
(#2995) These filter API v2 features were cut late in development because the form encoding version is hard to implement correctly and because I thought no clients actually used `keywords_attributes`. Unfortunately, Phanpy does use `keywords_attributes`.
2024-06-13[chore] Update move test (#3005)Libravatar tobi1
2024-06-12[bugfix] Ensure side effects handled for local unlocked follows (#2998)Libravatar tobi2
2024-06-11[bugfix] Deref stats async, serve stub collections if handshaking (#2990)v0.16.0-rc2Libravatar tobi3
* [bugfix] Deref stats async, allow peek if handshaking * don't return totalItems when handshaking or hiding collections * use GetLimit() * use StubAccountStats
2024-06-10[bugfix] boost and account recursion (#2982)Libravatar kim3
* fix possible infinite recursion if moved accounts are self-referential * adds a defensive check for a boost being a boost of a boost wrapper * add checks on input for a boost of a boost * remove unnecessary check * add protections on account move to prevent move recursion loops * separate status conversion without boost logic into separate function to remove risk of recursion * move boost check to boost function itself * formatting * use error 422 instead of 500 * use gtserror not standard errors package for error creation
2024-06-07[feature] Implement filters_changed stream event (#2972)Libravatar Vyr Cossont15
2024-06-06[feature] User muting (#2960)Libravatar Vyr Cossont16
* User muting * Address review feedback * Rename uniqueness constraint on user_mutes to match convention * Remove unused account_id from where clause * Add UserMute to NewTestDB * Update test/envparsing.sh with new and fixed cache stuff * Address tobi's review comments * Make compiledUserMuteListEntry.expired consistent with UserMute.Expired * Make sure mute_expires_at is serialized as an explicit null for indefinite mutes --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-06-06[feature] Self-serve email change for users (#2957)Libravatar tobi22
* [feature] Email change * frontend stuff for changing email * docs * tests etc * differentiate more clearly between local user+account and account * populate user
2024-06-06[feature] do not uncache status / emoji media if attached status is ↵Libravatar kim1
bookmarked (#2956) * do not uncache status / emoji media if attached status is bookmarked * add status bookmark and bookmark IDs caches * update status bookmark tests * move IsStatusBookmarkedBy() to StatusBookmark{} interface, rely on cache * fix envparsing.sh test
2024-05-31[feature] Add from: search operator and account_id query param (#2943)Libravatar Vyr Cossont1
* Add from: search operator * Fix whitespace in Swagger YAML comment * Move query parsing into its own method * Document search * Clarify post search scope
2024-05-31[feature] Implement Filter API v2 (#2936)Libravatar Vyr Cossont16
* Use correct entity name * We support server-side filters now * Document filter v1 methods that can throw a 409 * Validate v1 filter phrase as filter title * Always check v1 filter API status codes in tests * Document keyword minimum requirement on filter API v1 * Make it possible to specify filter keyword update columns per filter keyword * Implement v2 filter API * Fix lint and tests * Update Swagger spec * Fix filter update test * Update Swagger spec *correctly* * Update actual files Swagger spec was generated from * Remove keywords_attributes and statuses_attributes * Add test for serialization of empty filter * More helpful messages when object is owned by wrong account
2024-05-29[feature] Implement profile API (#2926)Libravatar Vyr Cossont1
* Implement profile API This Mastodon 4.2 extension provides capabilities missing from the existing Mastodon account update API: deleting an account's avatar or header. See: https://docs.joinmastodon.org/methods/profile/ * Move profile media methods to media processor * Remove check for moved account
2024-05-27[feature/frontend] Let admins send test email to validate SMTP config (#2934)Libravatar tobi1
* [feature/frontend] Let admins send test email to validate SMTP config * wee
2024-05-27[chore] Fix report email link (#2932)Libravatar tobi1
2024-05-22[performance] update storage backend and make use of seek syscall when ↵Libravatar kim1
available (#2924) * update to use go-storage/ instead of go-store/v2/storage/ * pull in latest version from codeberg * remove test output :innocent: * add code comments * set the exclusive bit when creating new files in disk config * bump to actual release version * bump to v0.1.1 (tis a simple no-logic change) * update readme * only use a temporary read seeker when decoding video if required (should only be S3 now) * use fastcopy library to use memory pooled buffers when calling TempFileSeeker() * update to use seek call in serveFileRange()
2024-05-06feature: filters v2 server-side warning/hiding (#2793)Libravatar Vyr Cossont15
* Remove dead code * Filter statuses when converting to frontend representation * status.filtered is an array * Make matching case-insensitive * Remove TODOs that don't need to be done now * Add missing filter check for notification * lint: rename ErrHideStatus * APIFilterActionToFilterAction not used yet * swaggerino docseroni * Address review comments * Add apimodel.FilterActionNone --------- Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com> Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-05-02[bugfix] Lock when checking/creating notifs to avoid race (#2890)Libravatar tobi14
* [bugfix] Lock when checking/creating notifs to avoid race * test notif spam
2024-05-01[feature] Page through accounts as moderator (#2881)Libravatar tobi1
* [feature] Page through accounts as moderator * aaaaa * use COLLATE "C" for Postgres to ensure same ordering as SQLite * fix typo, test paging up * don't show moderation / info for our instance acct
2024-04-29[chore] Fix conflict in workers tests (#2880)Libravatar tobi3
* [chore] Fix conflict in workers tests * commenty-wenty
2024-04-26[bugfix] Fix error string typo (#2873)Libravatar kim1
2024-04-26[performance] update remaining worker pools to use queues (#2865)Libravatar kim34
* start replacing client + federator + media workers with new worker + queue types * refactor federatingDB.Delete(), drop queued messages when deleting account / status * move all queue purging to the processor workers * undo toolchain updates * code comments, ensure dereferencer worker pool gets started * update gruf libraries in readme * start the job scheduler separately to the worker pools * reshuffle ordering or server.go + remove duplicate worker start / stop * update go-list version * fix vendoring * move queue invalidation to before wipeing / deletion, to ensure queued work not dropped * add logging to worker processing functions in testrig, don't start workers in unexpected places * update go-structr to add (+then rely on) QueueCtx{} type * ensure more worker pools get started properly in tests * fix remaining broken tests relying on worker queue logic * fix account test suite queue popping logic, ensure noop workers do not pull from queue * move back accidentally shuffled account deletion order * ensure error (non nil!!) gets passed in refactored federatingDB{}.Delete() * silently drop deletes from accounts not permitted to * don't warn log on forwarded deletes * make if else clauses easier to parse * use getFederatorMsg() * improved code comment * improved code comment re: requesting account delete checks * remove boolean result from worker start / stop since false = already running or already stopped * remove optional passed-in http.client * remove worker starting from the admin CLI commands (we don't need to handle side-effects) * update prune cli to start scheduler but not all of the workers * fix rebase issues * remove redundant return statements * i'm sorry sir linter
2024-04-17[feature] Status source endpoint (#2848)Libravatar tobi1
* [feature] statusSource endpoint * finish up
2024-04-17[feature] Stub status history endpoint (#2847)Libravatar tobi1
2024-04-16[feature/performance] Store account stats in separate table (#2831)Libravatar tobi13
* [feature/performance] Store account stats in separate table, get stats from remote * test account stats * add some missing increment / decrement calls * change stats function signatures * rejig logging a bit * use lock when updating stats
2024-04-15[performance] cached oauth database types (#2838)Libravatar kim1
* update token + client code to use struct caches * add code comments * slight tweak to default mem ratios * fix envparsing * add appropriate invalidate hooks * update the tokenstore sweeping function to rely on caches * update to use PutClient() * add ClientID to list of token struct indices
2024-04-13[feature] Admin accounts endpoints; approve/reject sign-ups (#2826)Libravatar tobi9
* update settings panels, add pending overview + approve/deny functions * add admin accounts get, approve, reject * send approved/rejected emails * use signup URL * docs! * email * swagger * web linting * fix email tests * wee lil fixerinos * use new paging logic for GetAccounts() series of admin endpoints, small changes to query building * shuffle useAccountIDIn check *before* adding to query * fix parse from toot react error * use `netip.Addr` * put valid slices in globals * optimistic updates for account state --------- Co-authored-by: kim <grufwub@gmail.com>
2024-04-11[feature] New user sign-up via web page (#2796)Libravatar tobi10
* [feature] User sign-up form and admin notifs * add chosen + filtered languages to migration * remove stray comment * chosen languages schmosen schmanguages * proper error on local account missing
2024-04-11[performance] massively improved ActivityPub delivery worker efficiency (#2812)Libravatar kim4
* add delivery worker type that pulls from queue to httpclient package * finish up some code commenting, bodge a vendored activity library change, integrate the deliverypool changes into transportcontroller * hook up queue deletion logic * support deleting queued http requests by target ID * don't index APRequest by hostname in the queue * use gorun * use the original context's values when wrapping msg type as delivery{} * actually log in the AP delivery worker ... * add uncommitted changes * use errors.AsV2() * use errorsv2.AsV2() * finish adding some code comments, add bad host handling to delivery workers * slightly tweak deliveryworkerpool API, use advanced sender multiplier * remove PopCtx() method, let others instead rely on Wait() * shuffle things around to move delivery stuff into transport/ subpkg * remove dead code * formatting * validate request before queueing for delivery * finish adding code comments, fix up backoff code * finish adding more code comments * clamp minimum no. senders to 1 * add start/stop logging to delivery worker, some slight changes * remove double logging * use worker ptrs * expose the embedded log fields in httpclient.Request{} * ensure request context values are preserved when updating ctx * add delivery worker tests * fix linter issues * ensure delivery worker gets inited in testrig * fix tests to delivering messages to check worker delivery queue * update error type to use ptr instead of value receiver * fix test calling Workers{}.Start() instead of testrig.StartWorkers() * update docs for advanced-sender-multiplier * update to the latest activity library version * add comment about not using httptest.Server{}
2024-04-08[bugfix] Ensure side effects for local -> local follows get processed (#2820)v0.15.0-rc2Libravatar tobi2
2024-04-02[bugfix] Set the `Host` header within the signing transport (#2799)Libravatar kim1
2024-04-02[feature] Option to hide followers/following (#2788)Libravatar tobi4
2024-03-28[bugfix] Serve correct URI for AP following collection (#2787)Libravatar tobi1
2024-03-25[feature] User-selectable preset CSS themes for accounts (#2777)Libravatar tobi4
* [feature] User-selectable preset themes * docs, more theme stuff * lint, tests * fix css name * correct some little issues * add another theme * fix poll background * okay last theme i swear * make retrieval of apimodel themes more conventional * preallocate stylesheet slices
2024-03-25[bugfix] Avoid empty public/local timeline queries (#2784)Libravatar tobi3
2024-03-22[chore] Move local account settings to separate db table (#2770)Libravatar tobi8
* [chore] Move local account settings to separate database model * don't use separate settings_id