summaryrefslogtreecommitdiff
path: root/internal
AgeCommit message (Collapse)AuthorFiles
2024-09-16[performance] cache more database calls, reduce required database calls ↵Libravatar kim61
overall (#3290) * improvements to caching for lists and relationship to accounts / follows * fix nil panic in AddToList() * ensure list related caches are correctly invalidated * ensure returned ID lists are ordered correctly * bump go-structr to v0.8.9 (returns early if zero uncached keys to be loaded) * remove zero checks in uncached key load functions (go-structr now handles this) * fix issues after rebase on upstream/main * update the expected return order of CSV exports (since list entries are now down by entry creation date) * rename some funcs, allow deleting list entries for multiple follow IDs at a time, fix up more tests * use returning statements on delete to get cache invalidation info * fixes to recent database delete changes * fix broken list entries delete sql * remove unused db function * update remainder of delete functions to behave in similar way, some other small tweaks * fix delete user sql, allow returning on err no entries * uncomment + fix list database tests * update remaining list tests * update envparsing test * add comments to each specific key being invalidated * add more cache invalidation explanatory comments * whoops; actually delete poll votes from database in the DeletePollByID() func * remove added but-commented-out field * improved comment regarding paging being disabled * make cache invalidation comments match what's actually happening * fix up delete query comments to match what is happening * rename function to read a bit better * don't use ErrNoEntries on delete when not needed (it's only needed for a RETURNING call) * update function name in test * move list exclusivity check to AFTER eligibility check. use log.Panic() instead of panic() * use the poll_id column in poll_votes for selecting votes in poll ID * fix function name
2024-09-16[chore] Refactor federatingDB.Undo, avoid 500 errors on Undo Like (#3310)Libravatar tobi3
2024-09-16[chore] Reject replies to rejected replies (#3291)Libravatar tobi2
* [chore] Reject replies to rejected replies * tweak * don't set URI for implicit Rejects
2024-09-16[bugfix] Use better plaintext representation of status for filtering (#3301)Libravatar tobi6
* [bugfix] Use better plaintext representation of status for filtering * add new deps to readme * lint * update tests * update regexes * address review comments * remove now unused xxhash * whoops, wrong logger * Merge branch 'main' into status_filtering_bugfix * put cache in caches struct * pain
2024-09-16[bugfix/chore] Always set the status sensitive if media + content-warning ↵Libravatar kim1
present (#3308) * always set the status sensitive flag to true if it has a content-warning with media * whoops use attachment ids instead of mention ids ... :facepalm:
2024-09-15[bugfix] Hoist filterable text field extraction out of loop (#3297)Libravatar Vyr Cossont1
This doesn't change within the loop, so there's no point in repeating it.
2024-09-14[chore] Update go-sqlite3 to 0.18.3 (#3295)Libravatar Daenney1
* [chore] Update go-sqlite3 to 0.18.3 * [chore] Fix getting the sqlite3.Conn
2024-09-11[bugfix] Update home timeline query to ignore exclusive list entries (#3289)Libravatar tobi2
* [bugfix] Update home timeline query to ignore exclusive list entries * a
2024-09-10[feature] Process `Reject` of interaction via fedi API, put rejected ↵Libravatar tobi20
statuses in the "sin bin" 😈 (#3271) * [feature] Process `Reject` of interaction via fedi API, put rejected statuses in the "sin bin" * update test * move nil check back to `rejectStatusIRI`
2024-09-10[chore] status dereferencing improvements (#3255)Libravatar kim4
* search for mentions also by username,domain in status deref, handle deleted statuses in enrichStatusSafely() * return d.enrichStatusSafely() directly
2024-09-09[feature] Implement exclusive lists (#3280)Libravatar Vyr Cossont13
Fixes #2616
2024-09-09[feature/frontend] Add options to include Unlisted posts or hide all posts ↵Libravatar tobi20
(#3272) * [feature/frontend] Add options to include Unlisted posts or hide all posts * finish up * swagger * move invalidate call into bundb package, avoid invalidating if not necessary * rename show_web_statuses => web_visibility * don't use ptr for webvisibility * last bits
2024-09-09[bugfix] Swagger: add missing filter_action param for v2 filter PUT (#3281)Libravatar Vyr Cossont1
2024-09-08[bugfix] Fix temp table deletion causing runaway allocations (#3278)Libravatar tobi2
* [bugfix] Fix temp table deletion causing runaway allocations * move some vars around * small fixes * rely on conn max age to recycle temp tables * fackin' ell m8
2024-09-07[chore] More AI blocking (#3273)Libravatar Daenney1
2024-09-03fix new processing media / emoji not being added to dereferencer maps (#3269)Libravatar kim2
2024-09-02[chore/performance] Further reduce nil uncached queries (#3267)Libravatar tobi8
* [chore/performance] Further reduce nil uncached queries * more checks
2024-09-02[chore] Don't try to select zero uncached filters (#3266)Libravatar tobi1
2024-09-02[chore/performance] Avoid unnecessary "uncached" queries (#3265)Libravatar tobi17
* [chore/performance] Avoid unnecessary "uncached" queries * go fmt
2024-08-31[performance] use single-threaded image transforms (#3252)Libravatar kim5
* use single-threaded image resizing in native code so we have more control over goroutines * implement parallel-free versions of image transform functions also * remove debug code
2024-08-30[chore] Close copied request body in SignDelivery (#3254)Libravatar tobi1
2024-08-30[security] Implement `allowFiles` fs for better isolation of ffmpeg / ↵Libravatar tobi2
ffprobe (#3251) * [chore] Implement readOneFile fs * further isolation * remove fmt call * tweaks
2024-08-29[chore] Only call `imaging.Resize when necessary`, use even tinier ↵Libravatar tobi5
blurhashes (#3247) * [chore] Use `imaging.Fit`, use even tinier blurhashes * avoid calling resize if not necessary * update blurhashes + thumb
2024-08-29[chore] Update robots.txt with more AI scrapers (#3248)Libravatar Daenney1
2024-08-26[bugfix] Fix incorrect json-ld `@context` serialization (#3243)Libravatar tobi2
2024-08-25[chore] Remove unused "env" module (#3235)Libravatar tobi1
2024-08-25[chore] Tiny tweak to ApprovedByURI (#3234)Libravatar tobi1
2024-08-25[bugfix] Let prometheus client do its own compression handling (#3232)Libravatar tobi1
2024-08-25[bugfix] Carry-over `ApprovedByURI` to avoid marking already-approved remote ↵Libravatar tobi1
statuses as pending approval (#3231) * [bugfix] Carry-over pendingApproval + approvedByURI to avoid pending already-approved remote statuses * don't carry over pending_approval, not necessary * comment
2024-08-24[chore/bugfix] Fix missing insertion of preapproved interaction requests (#3228)Libravatar tobi1
2024-08-24[feature] Interaction requests client api + settings panel (#3215)Libravatar tobi52
* [feature] Interaction requests client api + settings panel * test accept / reject * fmt * don't pin rejected interaction * use single db model for interaction accept, reject, and request * swaggor * env sharting * append errors * remove ErrNoEntries checks * change intReqID to reqID * rename "pend" to "request" * markIntsPending -> mark interactionsPending * use log instead of returning error when rejecting interaction * empty migration * jolly renaming * make interactionURI unique again * swag grr * remove unnecessary locks * invalidate as last step
2024-08-23[performance] ffmpeg ffprobe wrapper improvements (#3225)Libravatar kim7
* use a single instance of wazero runtime and compiled modules * remove test output :facepalm: * undo process-{media,emoji} changes * update test runner to include wazero compilation cache * sign drone.yml --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-08-22[feature] Use `local_only` field, deprecate `federated` field (#3222)Libravatar tobi11
* [feature] Use `local_only` field, deprecate `federated` field * use `deprecated` comment for form.Federated * nolint
2024-08-19[performance] use native Go code to probe JPEGs (#3206)Libravatar kim4
* use native Go code to probe JPEGs * add note about copying from github.com/disintegration/imaging * add more code comments
2024-08-19[chore] update default http client timeout to 30s (#3214)Libravatar kim1
2024-08-16[bugfix] permit unspecified orientation data (#3205)Libravatar kim1
2024-08-13[security] harden account update logic (#3198)Libravatar kim4
* on account update, ensure that public key has not changed * change expected error message * also support the case of changing account keys when expired (not waiting for handshake) * tweak account update hardening logic, add tests for updating account with pubkey expired * add check for whether incoming data was via federator, accepting keys if so * use freshest window for federated account updates + comment about it
2024-08-13[bugfix] relax missing preferred_username, instead using webfingered ↵Libravatar kim7
username (#3189) * support no preferred_username, instead using webfingered username * add tests for the new preferred_username behaviour
2024-08-12[bugfix] incorrect AP serialize function used serializing worker data (#3196)Libravatar kim1
2024-08-11[bugfix] updated pinned counts on status delete (#3188)Libravatar kim4
* include pinned status when incrementing / decrementing status counts * remove the pinned increment on status creation * code comments * microoptimize decr
2024-08-08[performance] move thumbnail generation to go code where possible (#3183)Libravatar kim19
* wrap thumbnailing code to handle generation natively where possible * more code comments! * add even more code comments! * add code comments about blurhash generation * maintain image rotation if contained in exif data * move rotation before resizing * ensure pix_fmt actually selected by ffprobe, check for alpha layer with gifs * use linear instead of nearest-neighbour for resizing * work with image "orientation" instead of "rotation". use default 75% quality for both webp and jpeg generation * add header to new file * use thumb extension when getting thumb mime type * update test models and tests with new media processing * add suggested code comments * add note about thumbnail filter count reducing memory usage
2024-08-08[feature] Add `db-postgres-connection-string` option (#3178)Libravatar John Winston3
* handle db-url * lint and add doc * add more doc * fix config test * return error * change name from db-url to db-postgres-connection-string
2024-08-08[feature] Use gifv type for short soundless mp4 videos (#3182)Libravatar tobi3
2024-08-06[bugfix] send back Sec-Websocket-Protocol header for streaming WebSocket (#3169)Libravatar Leah Neukirchen2
* [bugfix] send back Sec-Websocket-Protocol header for streaming WebSocket Chrome expects the selected Sec-Websocket-Protocol to be sent back on the WebSocket upgrade request (RFC6455 1.9). * fiddle a bit to avoid getting headers multiple times * add some explanatory notes --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-08-03[bugfix] fix emoji recaching operations (#3167)Libravatar kim9
* add test for emoji update image * update emoji recache to set the instance account id * don't refresh emoji if only not cached. in that case literally just recache * code comment * rename + move a few things * add some more code comments, and rename some functions to make logic a bit clearer * remove unnecessary nil check (the value can be nil) * comment wording * remove test data output * handle the case of caching an emoji which has been refreshed then uncached * allow overwriting on testrig storage as we do now on regular storage * fix emoji category ID not getting updated --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-08-03[chore] Add `media-ffmpeg-pool-size` config var (#3164)Libravatar tobi3
2024-08-02[feature] Beef up our AI opt-outs (#3165)Libravatar Daenney2
* [chore] Synchronise our robots.txt with upstream * [feature] Add headers to escape AI crawlers This adds 2 headers that a number of AI crawlers respect to signal that content should not be included in their datasets.
2024-08-02[bugfix] close files before error return (#3163)Libravatar kim2
* close files before error return * use defer statements * shuffle around some defers
2024-08-02[chore] move PopulateAccountStats() nil check often performed into function ↵Libravatar kim8
itself (#3158) * move PopulateAccountStats() nil check often performed into function itself * fix test to take in mind we don't repopulate account stats if not-nil
2024-08-02[chore] add back exif-terminator and use only for jpeg,png,webp (#3161)Libravatar kim6
* add back exif-terminator and use only for jpeg,png,webp * fix arguments passed to terminateExif() * pull in latest exif-terminator * fix test * update processed img --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>