summaryrefslogtreecommitdiff
path: root/internal/media/media_test.go
AgeCommit message (Collapse)AuthorFiles
2025-01-08[feature] Fetch + create domain permissions from subscriptions nightly (#3635)Libravatar tobi1
* peepeepoopoo * test domain perm subs * swagger * envparsing * dries your wets * start on docs * finish up docs * copy paste errors * rename actions package * rename force -> skipCache * move obfuscate parse nearer to where err is checked * make higherPrios a simple slice * don't use receiver for permsFrom funcs * add more context to error logs * defer finished log * use switch for permType instead of if/else * thanks linter, love you <3 * validate csv headers before full read * use bufio scanner
2024-02-27[feature] Add experimental `instance-federation-spam-filter` option (#2685)Libravatar tobi1
* [chore] Move `visibility` to `filter/visibility` * [feature] Add experimental instance-federation-spam-filter option
2023-11-14[bugfix] process account delete side effects in serial, not in parallel (#2360)Libravatar tobi1
* [bugfix] process account delete side effects in serial, not in parallel * StartWorkers / StartNoopWorkers for tests * undo testrig trace logging * log errors instead of immediately returning
2023-09-23[chore] deinterface the typeutils.Converter and update to use state ↵Libravatar kim1
structure (#2217) * update typeconverter to use state structure * deinterface the typeutils.TypeConverter -> typeutils.Converter * finish copying over old type converter code comments * fix cherry-pick merge issues, fix tests pointing to old typeutils interface type still
2023-05-28[chore] tidy up media manager, add calling func to errors, build-script ↵Libravatar kim1
improvements (#1835) * media manager tidy-up: de-interface and remove unused PostDataFunc Signed-off-by: kim <grufwub@gmail.com> * remove last traces of media.Manager being an interface Signed-off-by: kim <grufwub@gmail.com> * update error to provide caller, allow tuneable via build tags Signed-off-by: kim <grufwub@gmail.com> * remove kim-specific build script changes Signed-off-by: kim <grufwub@gmail.com> * fix merge conflicts Signed-off-by: kim <grufwub@gmail.com> * update build-script to support externally setting build variables Signed-off-by: kim <grufwub@gmail.com> --------- Signed-off-by: kim <grufwub@gmail.com>
2023-05-25[feature] Add List functionality (#1802)Libravatar tobi1
* start working on lists * further list work * test list db functions nicely * more work on lists * peepoopeepoo * poke * start list timeline func * we're getting there lads * couldn't be me working on stuff... could it? * hook up handlers * fiddling * weeee * woah * screaming, pissing * fix streaming being a whiny baby * lint, small test fix, swagger * tidying up, testing * fucked! by the linter * move timelines to state like a boss * add timeline start to tests using state * invalidate lists
2023-03-12[chore] Improve copyright header handling (#1608)Libravatar Daenney1
* [chore] Remove years from all license headers Years or year ranges aren't required in license headers. Many projects have removed them in recent years and it avoids a bit of yearly toil. In many cases our copyright claim was also a bit dodgy since we added the 2021-2023 header to files created after 2021 but you can't claim copyright into the past that way. * [chore] Add license header check This ensures a license header is always added to any new file. This avoids maintainers/reviewers needing to remember to check for and ask for it in case a contribution doesn't include it. * [chore] Add missing license headers * [chore] Further updates to license header * Use the more common // indentend comment format * Remove the hack we had for the linter now that we use the // format * Add SPDX license identifier
2023-03-01[chore] move client/federator workerpools to Workers{} (#1575)Libravatar kim1
* replace concurrency worker pools with base models in State.Workers, update code and tests accordingly * improve code comment * change back testrig default log level * un-comment-out TestAnnounceTwice() and fix --------- Signed-off-by: kim <grufwub@gmail.com> Reviewed-by: tobi
2023-01-05[chore] Update/add license headers for 2023 (#1304)Libravatar tobi1
2022-12-10[feature] Start implementing refetch of lost media files via ↵Libravatar tobi1
`/api/v1/admin/media_refetch` (#1221) * [chore] Move ShortcodeDomain to its own little util func * [feature] Add RefetchEmojis function to media manager * [feature] Expose admin media refresh via admin API * update following review feedback - change/fix log levels - make sure not to try to refetch local emojis - small style refactoring + comments * log on emoji refetch start Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: kim <grufwub@gmail.com>
2022-11-24[chore] cleanup storage implementation, no need for multiple interface types ↵Libravatar kim1
(#1131) Signed-off-by: kim <grufwub@gmail.com> Signed-off-by: kim <grufwub@gmail.com>
2022-10-13[feature] Refetch emojis when they change on remote instances (#905)Libravatar tobi1
* select emoji using image_static_url * use updated on AP emojis * allow refetch of updated emojis * cheeky workaround for test * clean up old files for refreshed emoji * check error for originalPostData * shorten GetEmojiByStaticImageURL * delete kirby (sorry nintendo)
2022-07-03[feature] S3 support (#674)Libravatar Dominik Süß1
* feat: vendor minio client * feat: introduce storage package with s3 support * feat: serve s3 files directly this saves a lot of bandwith as the files are fetched from the object store directly * fix: use explicit local storage in tests * feat: integrate s3 storage with the main server * fix: add s3 config to cli tests * docs: explicitly set values in example config also adds license header to the storage package * fix: use better http status code on s3 redirect HTTP 302 Found is the best fit, as it signifies that the resource requested was found but not under its presumed URL 307/TemporaryRedirect would mean that this resource is usually located here, not in this case 303/SeeOther indicates that the redirection does not link to the requested resource but to another page * refactor: use context in storage driver interface
2022-05-15[feature] Unused avatar and header cleanup (#574)Libravatar tobi1
* rename + tidy up remote pruning * fix media attachment account join see https://bun.uptrace.dev/guide/golang-orm.html#table-relationships * update logging to new function name * add get avatars and headers to bun * add pruneallmeta function * don't set uncached since we're deleting anyway * fix totalPruned being in wrong place * test pruning meta * go fmt ./... * rename mediaprune * add meta pruning to routine mediaprune * tidy up cleanup job scheduling * rename adminmediaremoteprune * update mediacleanup to use renamed prune func * update swagger docs a little bit * reuse cancel + context
2022-03-07[feature] Clean up/uncache remote media (#407)Libravatar tobi1
* Add whereNotEmptyAndNotNull * Add GetRemoteOlderThanDays * Add GetRemoteOlderThanDays * Add PruneRemote to Manager interface * Start implementing PruneRemote * add new attachment + status to tests * fix up and test GetRemoteOlderThan * fix bad import * PruneRemote: return number pruned * add Cached column to mediaattachment * update + test pruneRemote * update mediaTest * use Cached column * upstep bun to latest version * embed structs in mediaAttachment * migrate mediaAttachment to new format * don't default cached to true * select only remote media * update db dependencies * step bun back to last working version * update pruneRemote to use Cached field * fix storage path of test attachments * add recache logic to manager * fix trimmed aspect ratio * test prune and recache * return errwithcode * tidy up different paths for emoji vs attachment * fix incorrect thumbnail type being stored * expose TransportController to media processor * implement tee-ing recached content * add thoughts of dog to test fedi attachments * test get remote files * add comment on PruneRemote * add postData cleanup to recache * test thumbnail fetching * add incredible diagram * go mod tidy * buffer pipes for recache streaming * test for client stops reading after 1kb * add media-remote-cache-days to config * add cron package * wrap logrus so it's available to cron * start and stop cron jobs gracefully
2022-01-10test the media manager a bit, add shutdown logicLibravatar tsmethurst1
2022-01-09tests are passing, but there's still much to be doneLibravatar tsmethurst1
2022-01-08compiling nowLibravatar tsmethurst1
2021-05-17Refine statuses (#26)Libravatar Tobi Smethurst1
Remote media is now dereferenced and attached properly to incoming federated statuses. Mentions are now dereferenced and attached properly to incoming federated statuses. Small fixes to status visibility. Allow URL params for filtering statuses: // ExcludeRepliesKey is for specifying whether to exclude replies in a list of returned statuses by an account. // PinnedKey is for specifying whether to include pinned statuses in a list of returned statuses by an account. // MaxIDKey is for specifying the maximum ID of the status to retrieve. // MediaOnlyKey is for specifying that only statuses with media should be returned in a list of returned statuses by an account. Add endpoint for fetching an account's statuses.
2021-05-15Inbox post (#22)Libravatar Tobi Smethurst1
Inbox POST from federated servers now working for statuses and follow requests. Follow request client API added. Start work on federating outgoing messages. Other fixes and changes/tidying up.
2021-05-08Ap (#14)Libravatar Tobi Smethurst1
Big restructuring and initial work on activitypub
2021-04-19Api/v1/statuses (#11)Libravatar Tobi Smethurst1
This PR adds: Statuses New status creation. View existing status Delete a status Fave a status Unfave a status See who's faved a status Media Upload media attachment and store/retrieve it Upload custom emoji and store/retrieve it Fileserver Serve files from storage Testing Test models, testrig -- run a GTS test instance and play around with it.
2021-04-01Api/v1/accounts (#8)Libravatar Tobi Smethurst1
* start work on accounts module * plodding away on the accounts endpoint * groundwork for other account routes * add password validator * validation utils * require account approval flags * comments * comments * go fmt * comments * add distributor stub * rename api to federator * tidy a bit * validate new account requests * rename r router * comments * add domain blocks * add some more shortcuts * add some more shortcuts * check email + username availability * email block checking for signups * chunking away at it * tick off a few more things * some fiddling with tests * add mock package * relocate repo * move mocks around * set app id on new signups * initialize oauth server properly * rename oauth server * proper mocking tests * go fmt ./... * add required fields * change name of func * move validation to account.go * more tests! * add some file utility tools * add mediaconfig * new shortcut * add some more fields * add followrequest model * add notify * update mastotypes * mock out storage interface * start building media interface * start on update credentials * mess about with media a bit more * test image manipulation * media more or less working * account update nearly working * rearranging my package ;) ;) ;) * phew big stuff!!!! * fix type checking * *fiddles* * Add CreateTables func * account registration flow working * tidy * script to step through auth flow * add a lil helper for generating user uris * fiddling with federation a bit * update progress * Tidying and linting