summaryrefslogtreecommitdiff
path: root/internal
AgeCommit message (Collapse)AuthorFiles
2024-08-25[chore] Remove unused "env" module (#3235)Libravatar tobi1
2024-08-25[chore] Tiny tweak to ApprovedByURI (#3234)Libravatar tobi1
2024-08-25[bugfix] Let prometheus client do its own compression handling (#3232)Libravatar tobi1
2024-08-25[bugfix] Carry-over `ApprovedByURI` to avoid marking already-approved remote ↵Libravatar tobi1
statuses as pending approval (#3231) * [bugfix] Carry-over pendingApproval + approvedByURI to avoid pending already-approved remote statuses * don't carry over pending_approval, not necessary * comment
2024-08-24[chore/bugfix] Fix missing insertion of preapproved interaction requests (#3228)Libravatar tobi1
2024-08-24[feature] Interaction requests client api + settings panel (#3215)Libravatar tobi52
* [feature] Interaction requests client api + settings panel * test accept / reject * fmt * don't pin rejected interaction * use single db model for interaction accept, reject, and request * swaggor * env sharting * append errors * remove ErrNoEntries checks * change intReqID to reqID * rename "pend" to "request" * markIntsPending -> mark interactionsPending * use log instead of returning error when rejecting interaction * empty migration * jolly renaming * make interactionURI unique again * swag grr * remove unnecessary locks * invalidate as last step
2024-08-23[performance] ffmpeg ffprobe wrapper improvements (#3225)Libravatar kim7
* use a single instance of wazero runtime and compiled modules * remove test output :facepalm: * undo process-{media,emoji} changes * update test runner to include wazero compilation cache * sign drone.yml --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-08-22[feature] Use `local_only` field, deprecate `federated` field (#3222)Libravatar tobi11
* [feature] Use `local_only` field, deprecate `federated` field * use `deprecated` comment for form.Federated * nolint
2024-08-19[performance] use native Go code to probe JPEGs (#3206)Libravatar kim4
* use native Go code to probe JPEGs * add note about copying from github.com/disintegration/imaging * add more code comments
2024-08-19[chore] update default http client timeout to 30s (#3214)Libravatar kim1
2024-08-16[bugfix] permit unspecified orientation data (#3205)Libravatar kim1
2024-08-13[security] harden account update logic (#3198)Libravatar kim4
* on account update, ensure that public key has not changed * change expected error message * also support the case of changing account keys when expired (not waiting for handshake) * tweak account update hardening logic, add tests for updating account with pubkey expired * add check for whether incoming data was via federator, accepting keys if so * use freshest window for federated account updates + comment about it
2024-08-13[bugfix] relax missing preferred_username, instead using webfingered ↵Libravatar kim7
username (#3189) * support no preferred_username, instead using webfingered username * add tests for the new preferred_username behaviour
2024-08-12[bugfix] incorrect AP serialize function used serializing worker data (#3196)Libravatar kim1
2024-08-11[bugfix] updated pinned counts on status delete (#3188)Libravatar kim4
* include pinned status when incrementing / decrementing status counts * remove the pinned increment on status creation * code comments * microoptimize decr
2024-08-08[performance] move thumbnail generation to go code where possible (#3183)Libravatar kim19
* wrap thumbnailing code to handle generation natively where possible * more code comments! * add even more code comments! * add code comments about blurhash generation * maintain image rotation if contained in exif data * move rotation before resizing * ensure pix_fmt actually selected by ffprobe, check for alpha layer with gifs * use linear instead of nearest-neighbour for resizing * work with image "orientation" instead of "rotation". use default 75% quality for both webp and jpeg generation * add header to new file * use thumb extension when getting thumb mime type * update test models and tests with new media processing * add suggested code comments * add note about thumbnail filter count reducing memory usage
2024-08-08[feature] Add `db-postgres-connection-string` option (#3178)Libravatar John Winston3
* handle db-url * lint and add doc * add more doc * fix config test * return error * change name from db-url to db-postgres-connection-string
2024-08-08[feature] Use gifv type for short soundless mp4 videos (#3182)Libravatar tobi3
2024-08-06[bugfix] send back Sec-Websocket-Protocol header for streaming WebSocket (#3169)Libravatar Leah Neukirchen2
* [bugfix] send back Sec-Websocket-Protocol header for streaming WebSocket Chrome expects the selected Sec-Websocket-Protocol to be sent back on the WebSocket upgrade request (RFC6455 1.9). * fiddle a bit to avoid getting headers multiple times * add some explanatory notes --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-08-03[bugfix] fix emoji recaching operations (#3167)Libravatar kim9
* add test for emoji update image * update emoji recache to set the instance account id * don't refresh emoji if only not cached. in that case literally just recache * code comment * rename + move a few things * add some more code comments, and rename some functions to make logic a bit clearer * remove unnecessary nil check (the value can be nil) * comment wording * remove test data output * handle the case of caching an emoji which has been refreshed then uncached * allow overwriting on testrig storage as we do now on regular storage * fix emoji category ID not getting updated --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-08-03[chore] Add `media-ffmpeg-pool-size` config var (#3164)Libravatar tobi3
2024-08-02[feature] Beef up our AI opt-outs (#3165)Libravatar Daenney2
* [chore] Synchronise our robots.txt with upstream * [feature] Add headers to escape AI crawlers This adds 2 headers that a number of AI crawlers respect to signal that content should not be included in their datasets.
2024-08-02[bugfix] close files before error return (#3163)Libravatar kim2
* close files before error return * use defer statements * shuffle around some defers
2024-08-02[chore] move PopulateAccountStats() nil check often performed into function ↵Libravatar kim8
itself (#3158) * move PopulateAccountStats() nil check often performed into function itself * fix test to take in mind we don't repopulate account stats if not-nil
2024-08-02[chore] add back exif-terminator and use only for jpeg,png,webp (#3161)Libravatar kim6
* add back exif-terminator and use only for jpeg,png,webp * fix arguments passed to terminateExif() * pull in latest exif-terminator * fix test * update processed img --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-08-02[feature] Allow import of following and blocks via CSV (#3150)Libravatar tobi17
* [feature] Import follows + blocks via settings panel * test import follows
2024-07-31[chore] Take account of rotation data when calculating full size image ↵Libravatar tobi3
dimensions (#3159) * [chore] Take account of rotation data when calculating full size image dimensions * boobies
2024-07-31[feature] Implement Mastodon-compatible roles (#3136)Libravatar Vyr Cossont10
* Implement Mastodon-compatible roles - `Account.role` should only be available through verify_credentials for checking current user's permissions - `Account.role` now carries a Mastodon-compatible permissions bitmap and a marker for whether it should be shown to the public - `Account.roles` added for *public* display roles (undocumented but stable since Mastodon 4.1) - Web template now uses only public display roles (no user-visible change here, we already special-cased the `user` role) * Handle verify_credentials case for default role * Update JSON exact-match tests * Address review comments * Add blocks bit to admin permissions bitmap
2024-07-31[feature] Allow users to export data via the settings panel (#3140)Libravatar tobi22
* [feature] Allow users to export data via the settings panel * rename/move some stuff
2024-07-31[feature] Object store custom URL (S3) (#3046)Libravatar CDN5
* tweaks * boobs * fix variable name + typo --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-07-30[performance] limit ffprobe read time to at most 1s after start of file (#3155)Libravatar kim1
2024-07-30[feature] persist worker queues to db (#3042)Libravatar kim18
* persist queued worker tasks to database on shutdown, fill worker queues from database on startup * ensure the tasks are sorted by creation time before pushing them * add migration to insert WorkerTask{} into database, add test for worker task persistence * add test for recovering worker queues from database * quick tweak * whoops we ended up with double cleaner job scheduling * insert each task separately, because bun is throwing some reflection error?? * add specific checking of cancelled worker contexts * add http request signing to deliveries recovered from database * add test for outgoing public key ID being correctly set on delivery * replace select with Queue.PopCtx() * get rid of loop now we don't use it * remove field now we don't use it * ensure that signing func is set * header values weren't being copied over :facepalm: * use ptr for httpclient.Request in delivery * move worker queue filling to later in server init process * fix rebase issues * make logging less shouty * use slices.Delete() instead of copying / reslicing * have database return tasks in ascending order instead of sorting them * add a 1 minute timeout to persisting worker queues
2024-07-30[chore] replace UniqueStrings with Deduplicate (#3154)Libravatar tobi1
2024-07-30[chore] add some more slice related utility functions + remove duplicated ↵Libravatar kim7
functions (#3149)
2024-07-29[feature] Implement following hashtags (#3141)Libravatar Vyr Cossont35
* Implement followed tags API * Insert statuses with followed tags into home timelines * Test following and unfollowing tags * Correct Swagger path params * Trim conversation caches * Migration for followed_tags table * Followed tag caches and DB implementation * Lint and tests * Add missing tag info endpoint, reorganize tag API * Unwrap boosts when timelining based on tags * Apply visibility filters to tag followers * Address review comments
2024-07-28[bugfix] take into account rotation when generating thumbnail (#3147)Libravatar kim4
* take into account rotation when generating thumbnail, simplify ffprobe output to show only fields we need * only show rotation side data * remove unnecessary comment * fix code comments * remove debug logging
2024-07-28[bugfix] moves file rename to earlier in media pipeline so ffmpeg calls ↵Libravatar kim3
ALWAYS have extension (#3146)
2024-07-27[chore] log delivery worker stop/start at debug level (#3145)Libravatar tobi1
2024-07-27[chore] replaces nested ifs with switch cases, removes defer 'onFail()' ↵Libravatar kim1
being passed to funcs (#3143)
2024-07-26ensure ffmpeg muxer gets set correctly by setting input file extension ↵Libravatar kim1
before clean (#3142)
2024-07-26[bugfix] Use punycode for `host` part of `resource` query param when doing ↵Libravatar tobi6
webfinger requests (#3133) * [bugfix] use punycode when webfingering * account for punycode when checking if final URI matches expected * hmm * fix test
2024-07-26[feature] Federate interaction policies + Accepts; enforce policies (#3138)Libravatar tobi35
* [feature] Federate interaction policies + Accepts; enforce policies * use Acceptable type * fix index * remove appendIRIStrs * add GetAccept federatingdb function * lock on object IRI
2024-07-24[chore] Add interaction filter to complement existing visibility filter (#3111)Libravatar tobi56
* [chore] Add interaction filter to complement existing visibility filter * pass in ptr to visibility and interaction filters to Processor{} to ensure shared * use int constants for for match type, cache db calls in filterctx * function name typo :innocent: --------- Co-authored-by: kim <grufwub@gmail.com>
2024-07-24[chore] renames the `GTS` caches to `DB` caches (#3127)Libravatar kim39
* renames the `GTS` caches to `DB` caches, as it better references what they are * change remaining Caches.GTS uses to Caches.DB
2024-07-24[bugfix] Serialize empty conversation account list as empty list, not null ↵Libravatar Vyr Cossont1
(#3137)
2024-07-23[chore] Add some log lines to recent migrations warning not to interrupt (#3134)Libravatar tobi4
* [chore] Add some log lines to recent migrations warning not to interrupt * arse
2024-07-23Remove content and related fields from boosts (#3131)Libravatar Vyr Cossont4
These duplicate the content of the target and aren't necessary for anything. - Stops copying some fields from target when boosting or processing a remote boost - Adds a migration to null out existing duplicate data - Updates tests
2024-07-23Explicitly propagate filter results from statuses to their boosts in API ↵Libravatar Vyr Cossont2
responses (#3130) Related to #3128
2024-07-23[feature] Conversations API (#3013)Libravatar Vyr Cossont51
* Implement conversations API * Sort and page conversations by last status ID * Appease linter * Fix deleting conversations and statuses * Refactor to make migrations automatic * Lint * Update tests post-merge * Fixes from live-fire testing * Linter caught a format problem * Refactor tests, fix cache * Negative test for non-DMs * Run conversations advanced migration on testrig startup as well as regular server startup * Document (lack of) side effects of API method for deleting a conversation * Make not-found check less nested for readability * Rename PutConversation to UpsertConversation * Use util.Ptr instead of IIFE * Reduce cache used by conversations * Remove unnecessary TableExpr/ColumnExpr * Use struct tags for both unique constraints on Conversation * Make it clear how paging with GetDirectStatusIDsBatch should be used * Let conversation paging skip conversations it can't render * Use Bun NewDropTable * Convert delete raw query to Bun * Convert update raw query to Bun * Convert latestConversationStatusesTempTable raw query partially to Bun * Convert conversationStatusesTempTable raw query partially to Bun * Rename field used to store result of MaxDirectStatusID * Move advanced migrations to their own tiny processor * Catch up util function name with main * Remove json.… wrappers * Remove redundant check * Combine error checks * Replace map with slice of structs * Address processor/type converter comments - Add context info for errors - Extract some common processor code into shared methods - Move conversation eligibility check ahead of populating conversation * Add error context when dropping temp tables
2024-07-22[bugfix] media.Processor{}.GetFile() returning 404s on first call, correctly ↵Libravatar kim9
loading on 2nd (#3129) * refactor file handling a tiny bit * whoops * make processing media / emoji defers a bit clear to see that it's the "on finished processing" path * some wording * add some debug logging * add mutex locks for processing remote media * try removing freshness check * fix derefMedia not being allocated * fix log format string * handle case of empty file paths (i.e. not stored) * remove media / emoji once finished processing from dereferencer maps * whoops, fix the cached / force checks * move url parsing outside of 'process___Safely()' funcs to prevalidate url * use emoji.ShortcodeDomain() * update RefreshEmoji() to also match RefreshMedia() changes --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>