summaryrefslogtreecommitdiff
path: root/internal
AgeCommit message (Collapse)AuthorFiles
2024-07-31[feature] Allow users to export data via the settings panel (#3140)Libravatar tobi22
* [feature] Allow users to export data via the settings panel * rename/move some stuff
2024-07-31[feature] Object store custom URL (S3) (#3046)Libravatar CDN5
* tweaks * boobs * fix variable name + typo --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-07-30[performance] limit ffprobe read time to at most 1s after start of file (#3155)Libravatar kim1
2024-07-30[feature] persist worker queues to db (#3042)Libravatar kim18
* persist queued worker tasks to database on shutdown, fill worker queues from database on startup * ensure the tasks are sorted by creation time before pushing them * add migration to insert WorkerTask{} into database, add test for worker task persistence * add test for recovering worker queues from database * quick tweak * whoops we ended up with double cleaner job scheduling * insert each task separately, because bun is throwing some reflection error?? * add specific checking of cancelled worker contexts * add http request signing to deliveries recovered from database * add test for outgoing public key ID being correctly set on delivery * replace select with Queue.PopCtx() * get rid of loop now we don't use it * remove field now we don't use it * ensure that signing func is set * header values weren't being copied over :facepalm: * use ptr for httpclient.Request in delivery * move worker queue filling to later in server init process * fix rebase issues * make logging less shouty * use slices.Delete() instead of copying / reslicing * have database return tasks in ascending order instead of sorting them * add a 1 minute timeout to persisting worker queues
2024-07-30[chore] replace UniqueStrings with Deduplicate (#3154)Libravatar tobi1
2024-07-30[chore] add some more slice related utility functions + remove duplicated ↵Libravatar kim7
functions (#3149)
2024-07-29[feature] Implement following hashtags (#3141)Libravatar Vyr Cossont35
* Implement followed tags API * Insert statuses with followed tags into home timelines * Test following and unfollowing tags * Correct Swagger path params * Trim conversation caches * Migration for followed_tags table * Followed tag caches and DB implementation * Lint and tests * Add missing tag info endpoint, reorganize tag API * Unwrap boosts when timelining based on tags * Apply visibility filters to tag followers * Address review comments
2024-07-28[bugfix] take into account rotation when generating thumbnail (#3147)Libravatar kim4
* take into account rotation when generating thumbnail, simplify ffprobe output to show only fields we need * only show rotation side data * remove unnecessary comment * fix code comments * remove debug logging
2024-07-28[bugfix] moves file rename to earlier in media pipeline so ffmpeg calls ↵Libravatar kim3
ALWAYS have extension (#3146)
2024-07-27[chore] log delivery worker stop/start at debug level (#3145)Libravatar tobi1
2024-07-27[chore] replaces nested ifs with switch cases, removes defer 'onFail()' ↵Libravatar kim1
being passed to funcs (#3143)
2024-07-26ensure ffmpeg muxer gets set correctly by setting input file extension ↵Libravatar kim1
before clean (#3142)
2024-07-26[bugfix] Use punycode for `host` part of `resource` query param when doing ↵Libravatar tobi6
webfinger requests (#3133) * [bugfix] use punycode when webfingering * account for punycode when checking if final URI matches expected * hmm * fix test
2024-07-26[feature] Federate interaction policies + Accepts; enforce policies (#3138)Libravatar tobi35
* [feature] Federate interaction policies + Accepts; enforce policies * use Acceptable type * fix index * remove appendIRIStrs * add GetAccept federatingdb function * lock on object IRI
2024-07-24[chore] Add interaction filter to complement existing visibility filter (#3111)Libravatar tobi56
* [chore] Add interaction filter to complement existing visibility filter * pass in ptr to visibility and interaction filters to Processor{} to ensure shared * use int constants for for match type, cache db calls in filterctx * function name typo :innocent: --------- Co-authored-by: kim <grufwub@gmail.com>
2024-07-24[chore] renames the `GTS` caches to `DB` caches (#3127)Libravatar kim39
* renames the `GTS` caches to `DB` caches, as it better references what they are * change remaining Caches.GTS uses to Caches.DB
2024-07-24[bugfix] Serialize empty conversation account list as empty list, not null ↵Libravatar Vyr Cossont1
(#3137)
2024-07-23[chore] Add some log lines to recent migrations warning not to interrupt (#3134)Libravatar tobi4
* [chore] Add some log lines to recent migrations warning not to interrupt * arse
2024-07-23Remove content and related fields from boosts (#3131)Libravatar Vyr Cossont4
These duplicate the content of the target and aren't necessary for anything. - Stops copying some fields from target when boosting or processing a remote boost - Adds a migration to null out existing duplicate data - Updates tests
2024-07-23Explicitly propagate filter results from statuses to their boosts in API ↵Libravatar Vyr Cossont2
responses (#3130) Related to #3128
2024-07-23[feature] Conversations API (#3013)Libravatar Vyr Cossont51
* Implement conversations API * Sort and page conversations by last status ID * Appease linter * Fix deleting conversations and statuses * Refactor to make migrations automatic * Lint * Update tests post-merge * Fixes from live-fire testing * Linter caught a format problem * Refactor tests, fix cache * Negative test for non-DMs * Run conversations advanced migration on testrig startup as well as regular server startup * Document (lack of) side effects of API method for deleting a conversation * Make not-found check less nested for readability * Rename PutConversation to UpsertConversation * Use util.Ptr instead of IIFE * Reduce cache used by conversations * Remove unnecessary TableExpr/ColumnExpr * Use struct tags for both unique constraints on Conversation * Make it clear how paging with GetDirectStatusIDsBatch should be used * Let conversation paging skip conversations it can't render * Use Bun NewDropTable * Convert delete raw query to Bun * Convert update raw query to Bun * Convert latestConversationStatusesTempTable raw query partially to Bun * Convert conversationStatusesTempTable raw query partially to Bun * Rename field used to store result of MaxDirectStatusID * Move advanced migrations to their own tiny processor * Catch up util function name with main * Remove json.… wrappers * Remove redundant check * Combine error checks * Replace map with slice of structs * Address processor/type converter comments - Add context info for errors - Extract some common processor code into shared methods - Move conversation eligibility check ahead of populating conversation * Add error context when dropping temp tables
2024-07-22[bugfix] media.Processor{}.GetFile() returning 404s on first call, correctly ↵Libravatar kim9
loading on 2nd (#3129) * refactor file handling a tiny bit * whoops * make processing media / emoji defers a bit clear to see that it's the "on finished processing" path * some wording * add some debug logging * add mutex locks for processing remote media * try removing freshness check * fix derefMedia not being allocated * fix log format string * handle case of empty file paths (i.e. not stored) * remove media / emoji once finished processing from dereferencer maps * whoops, fix the cached / force checks * move url parsing outside of 'process___Safely()' funcs to prevalidate url * use emoji.ShortcodeDomain() * update RefreshEmoji() to also match RefreshMedia() changes --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-07-21[feature] add flac support (#3121)Libravatar kim4
* add flac support to the ffprobe format/stream -> filetype parser * also add audio/flac for flac (not just x-flac) * update tests --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-07-21[feature/frontend] Respect `prefers-reduced-motion` for avatars, headers, ↵Libravatar tobi15
and emojis (#3118) * [feature/frontend] Respect `prefers-reduced-motion` for avatars, headers, and emojis * go fmt * fix tests * use static version of instance thumbnail when appropriate * use prefers-reduced-motion * simplify account conversion a bit * fix c&p error
2024-07-21[bugfix] update common get target account / status doing refresh async (#3124)Libravatar kim2
2024-07-21[chore] Remove duplicate `<hr>` (#3123)Libravatar tobi2
* [chore] Remove duplicate `<hr>` * update test
2024-07-20update go-ffmpreg to v0.2.3 (#3120)Libravatar kim2
2024-07-20[chore/frontend] Use smaller webp images for logo and default avatars (#3119)Libravatar tobi13
2024-07-19[feature] use webp for thumbnails (#3116)Libravatar kim35
* update to use webp for thumbnails * bump webp quality up to 40% from 12% (it's a bit different to jpeg quality setting) * update to use yuva colorspace, and use thumbnail=n=10 to select frame * fix missing comma in ffmpeg args * add links to appropriate ffmpeg docs * update tests * add file size tests for thumbnails --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-07-17[chore] Fix typo (#3113)Libravatar tobi1
2024-07-17[chore] media pipeline improvements (#3110)Libravatar kim26
* don't set emoji / media image paths on failed download, migrate FileType from string to integer * fix incorrect uses of util.PtrOr, fix returned frontend media * fix migration not setting arguments correctly in where clause * fix not providing default with not null column * whoops * ensure a default gets set for media attachment file type * remove the exclusive flag from writing files in disk storage * rename PtrOr -> PtrOrZero, and rename PtrValueOr -> PtrOrValue to match * slight wording changes * use singular / plural word forms (no parentheses), is better for screen readers * update testmodels with unknown media type to have unset file details, update attachment focus handling converting to frontend, update tests * store first instance in ffmpeg wasm pool, fill remaining with closed instances
2024-07-17[feature] Allow users to set default interaction policies per status ↵Libravatar tobi15
visibility (#3108) * [feature] Allow users to set default interaction policies * use vars for default policies * avoid some code repetition * unfuck form binding * avoid bonkers loop * beep boop * put policyValsToAPIPolicyVals in separate function * don't bother with slices.Grow * oops
2024-07-16give read-only access to /dev for ffmpeg to access /dev/urandom (#3109)Libravatar kim1
2024-07-15[feature] more filetype support! (#3107)Libravatar kim10
* add more supported file types to our media processor that ffmpeg supports, update supported mime type lists * add code comments to the supported mime types slice * don't check for zero value string, just parse * remove some unneeded consts which make the code a bit harder to read * fix test expected instance media mime types, use compact ffprobe json, simple media processing by type * final tweaks to media processing code * don't use safe divide where we don't need to
2024-07-15[feature/frontend] Add player for audio files; use thumbnail for `poster` ↵Libravatar tobi21
(#3099) * [feature/frontend] Audio player for audio media types * use video preview images for previews instead of video itself * don't preload * update tests for new zork status * collapse media gallery into single row when small
2024-07-15[chore] add step-by-step logging to long migration (#3102)Libravatar tobi1
2024-07-13[chore/frontend] Tweak threading a bit, inform about hidden replies (#3097)Libravatar tobi4
* [chore/frontend] Tweak threading a bit, inform about hidden replies * whoops * round off bottom of replies col-header if no replies visible
2024-07-13[bugfix] Fix panic in `isStatusHomeTimelineable` (#3096)Libravatar tobi1
2024-07-12[feature/frontend] Better visual separation between "main" thread and ↵Libravatar tobi12
"replies" (#3093) * [feature/frontend] Better web threading model * fix test * bwap * tweaks * more tweaks to wording * typo * indenting * adjust wording * aaa
2024-07-12[feature] support processing of (many) more media types (#3090)Libravatar kim58
* initial work replacing our media decoding / encoding pipeline with ffprobe + ffmpeg * specify the video codec to use when generating static image from emoji * update go-storage library (fixes incompatibility after updating go-iotools) * maintain image aspect ratio when generating a thumbnail for it * update readme to show go-ffmpreg * fix a bunch of media tests, move filesize checking to callers of media manager for more flexibility * remove extra debug from error message * fix up incorrect function signatures * update PutFile to just use regular file copy, as changes are file is on separate partition * fix remaining tests, remove some unneeded tests now we're working with ffmpeg/ffprobe * update more tests, add more code comments * add utilities to generate processed emoji / media outputs * fix remaining tests * add test for opus media file, add license header to utility cmds * limit the number of concurrently available ffmpeg / ffprobe instances * reduce number of instances * further reduce number of instances * fix envparsing test with configuration variables * update docs and configuration with new media-{local,remote}-max-size variables
2024-07-11[chore] Add interaction policy gtsmodels (#3075)Libravatar tobi43
* [chore] introduce interaction policy gts models * update migration a smidge * fix copy paste typo * update migration * use int for InteractionType
2024-07-10[bugfix] Don't throw error when parent statuses are missing (#2011) (#3088)Libravatar snow1
* [bugfix] Don't throw error when parent statuses are missing (#2011) * Split missing parent status case from error check
2024-07-10[choore] Update robots.txt (#3092)Libravatar Daenney1
Recategorises a pair of scrapers according to their use.
2024-07-08[feature/frontend] Allow setting alt-text for avatar + header (#3086)Libravatar tobi11
2024-07-04[bugfix] Handle ErrHideStatus when preparing timeline statuses (#3071)Libravatar Vyr Cossont4
2024-07-04[feature] Set some security related headers (#3065)Libravatar Daenney2
* Set frame-ancestors in the CSP This ensures we can't be loaded/embedded in an iframe. It also sets the older X-Frame-Options for fallback. * Disable MIME type sniffing * Set Referrer-Policy This sets the policy such that browsers will never send the Referer header along with a request, unless it's a request to the same protocol, host/domain and port. Basically, only send it when navigating through our own UI, but not anything external. The default is strict-origin-when-cross-origin when unset, which sends the Referer header for requests unless it's going from HTTPS to HTTP (i.e a security downgrade, hence the 'strict').
2024-07-03[chore] Allow gtsmodel to depend on util (#3068)Libravatar Vyr Cossont5
Convert one free function into a gtsmodel.Emoji method so that util does not depend on gtsmodel and can be used from it in the future
2024-06-30[bugfix] update SignupURL routing in email template. (#3055)Libravatar CDN1
2024-06-29[feature] Default to WASM-based SQLite driver (#3053)Libravatar Daenney4
* [feature] Default to WASM-based SQLite driver With 0.16 out this switches our default SQLite driver to the WASM-based solution instead. So far the driver seems to perform just as well. Switching our default should result in it getting a bit more testing during the 0.17 development cycle. * add the ol' john hancock --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-06-28[bugfix] Fix typo in instance cache copyF (#3052)Libravatar tobi1