summaryrefslogtreecommitdiff
path: root/testrig
AgeCommit message (Collapse)AuthorFiles
2024-05-05[feature] Add HTTP header permission section to frontend (#2893)Libravatar tobi1
* [feature] Add HTTP header filter section to frontend * tweak naming a bit
2024-04-26[performance] update remaining worker pools to use queues (#2865)Libravatar kim3
* start replacing client + federator + media workers with new worker + queue types * refactor federatingDB.Delete(), drop queued messages when deleting account / status * move all queue purging to the processor workers * undo toolchain updates * code comments, ensure dereferencer worker pool gets started * update gruf libraries in readme * start the job scheduler separately to the worker pools * reshuffle ordering or server.go + remove duplicate worker start / stop * update go-list version * fix vendoring * move queue invalidation to before wipeing / deletion, to ensure queued work not dropped * add logging to worker processing functions in testrig, don't start workers in unexpected places * update go-structr to add (+then rely on) QueueCtx{} type * ensure more worker pools get started properly in tests * fix remaining broken tests relying on worker queue logic * fix account test suite queue popping logic, ensure noop workers do not pull from queue * move back accidentally shuffled account deletion order * ensure error (non nil!!) gets passed in refactored federatingDB{}.Delete() * silently drop deletes from accounts not permitted to * don't warn log on forwarded deletes * make if else clauses easier to parse * use getFederatorMsg() * improved code comment * improved code comment re: requesting account delete checks * remove boolean result from worker start / stop since false = already running or already stopped * remove optional passed-in http.client * remove worker starting from the admin CLI commands (we don't need to handle side-effects) * update prune cli to start scheduler but not all of the workers * fix rebase issues * remove redundant return statements * i'm sorry sir linter
2024-04-26[chore] Update setting testrig loglevel (#2870)Libravatar Daenney1
cmp.Or was introduced in Go 1.22 and picks the first value that's not the zero value for the type. For a string, the zero value is the empty string, which is what os.Getenv will return if the environment variable is not set. That then results in "error" being returned instead. This allows loading an environment variable with a default without having to do the check and write out the conditional.
2024-04-11[feature] New user sign-up via web page (#2796)Libravatar tobi3
* [feature] User sign-up form and admin notifs * add chosen + filtered languages to migration * remove stray comment * chosen languages schmosen schmanguages * proper error on local account missing
2024-04-11[performance] massively improved ActivityPub delivery worker efficiency (#2812)Libravatar kim1
* add delivery worker type that pulls from queue to httpclient package * finish up some code commenting, bodge a vendored activity library change, integrate the deliverypool changes into transportcontroller * hook up queue deletion logic * support deleting queued http requests by target ID * don't index APRequest by hostname in the queue * use gorun * use the original context's values when wrapping msg type as delivery{} * actually log in the AP delivery worker ... * add uncommitted changes * use errors.AsV2() * use errorsv2.AsV2() * finish adding some code comments, add bad host handling to delivery workers * slightly tweak deliveryworkerpool API, use advanced sender multiplier * remove PopCtx() method, let others instead rely on Wait() * shuffle things around to move delivery stuff into transport/ subpkg * remove dead code * formatting * validate request before queueing for delivery * finish adding code comments, fix up backoff code * finish adding more code comments * clamp minimum no. senders to 1 * add start/stop logging to delivery worker, some slight changes * remove double logging * use worker ptrs * expose the embedded log fields in httpclient.Request{} * ensure request context values are preserved when updating ctx * add delivery worker tests * fix linter issues * ensure delivery worker gets inited in testrig * fix tests to delivering messages to check worker delivery queue * update error type to use ptr instead of value receiver * fix test calling Workers{}.Start() instead of testrig.StartWorkers() * update docs for advanced-sender-multiplier * update to the latest activity library version * add comment about not using httptest.Server{}
2024-04-04[chore] Log less output on failed test (#2804)Libravatar Daenney1
* [chore] Log less output on failed test This changes the testrig log level to be error by default instead of info. This makes test failures a lot easier to read, as we don't have the parade of info logs for each failure to scroll through. It speeds up the test suite by a couple of seconds since we need to buffer and flush a lot less messages. On a clean run, so no test failures, it's about a 3s difference on my machine. Depending on the amount of test failures, total time saved can vary. This also introduces a GTS_TESTRIG_LOG_LEVEL environment variable that we explicitly check for, making it easy to override the log level should we have a need for it. This would be primarily for running locally, and not so much as part of go test. Lastly, it updates the syslog tests to use log.Error because if the log level is set to error but we call log.Info no message is emitted and we hang indefinitely on the channel read. * [chore] Rename the testrig log level env var
2024-04-02[bugfix] httpclient not signing subsequent redirect requests (#2798)Libravatar kim1
* move http request signing to transport * actually hook up the http roundtripper ... * add code comments for the new gtscontext functions
2024-04-02[feature] Option to hide followers/following (#2788)Libravatar tobi1
2024-03-22[chore] Move local account settings to separate db table (#2770)Libravatar tobi2
* [chore] Move local account settings to separate database model * don't use separate settings_id
2024-03-06[feature] Filters v1 (#2594)Libravatar Vyr Cossont2
* Implement client-side v1 filters * Exclude linter false positives * Update test/envparsing.sh * Fix minor Swagger, style, and Bun usage issues * Regenerate Swagger * De-generify filter keywords * Remove updating filter statuses This is an operation that the Mastodon v2 filter API doesn't actually have, because filter statuses, unlike keywords, don't have options: the only info they contain is the status ID to be filtered. * Add a test for filter statuses specifically * De-generify filter statuses * Inline FilterEntry * Use vertical style for Bun operations consistently * Add comment on Filter DB interface * Remove GoLand linter control comments Our existing linters should catch these, or they don't matter very much * Reduce memory ratio for filters
2024-03-04[bugfix] check remote status permissibility (#2703)Libravatar kim1
* add more stringent checks for remote status permissibility * add check for inreplyto of a remote status being a boost * do not permit inReplyTo boost wrapper statuses * change comment wording * fix calls to NewFederator() * add code comments for NotPermitted() and SetNotPermitted() * improve comment * check that existing != nil before attempting delete * ensure replying account isn't suspended * use a debug log instead of info. check for boost using ID * shorten log string length. make info level * add note that replying to boost wrapper status shouldn't be able to happen anyways * update to use onFail() function
2024-03-01[docs] Use human sizes for size limitations (#2706)Libravatar Daenney1
This switches the documentation to use human sizes, so 50MiB instead of an integer number of bytes. This makes it much easier to understand what values are set, and less likely to accidentally set the wrong value.
2024-02-27[feature/oidc] Add support for very basic RBAC (#2642)Libravatar 9p41
* Add support for very basic RBAC * Add some small tests for allowedGroup and adminGroup * Switch to table-driven tests
2024-02-27[feature] Add experimental `instance-federation-spam-filter` option (#2685)Libravatar tobi3
* [chore] Move `visibility` to `filter/visibility` * [feature] Add experimental instance-federation-spam-filter option
2024-02-23[chore] Increase default max image description to 1500 chars, collapse cw ↵Libravatar tobi1
char count into status (#2682) * [chore] Make default max image description 1500 chars, collapse cw char count into status * oops * tests
2024-02-14[bugfix] add stricter checks during all stages of dereferencing remote AS ↵Libravatar kim1
objects (#2639) * add stricter checks during all stages of dereferencing remote AS objects * a comment
2024-02-06[feature] serdes for moved/also_known_as (#2600)Libravatar tobi1
* [feature] serdes for moved/also_known_as * document `alsoKnownAs` and `movedTo` properties * only implicitly populate AKA uris from DB for local accounts * don't let remotes store more than 20 AKA uris to avoid shenanigans
2024-01-19[performance] overhaul struct (+ result) caching library for simplicity, ↵Libravatar kim1
performance and multiple-result lookups (#2535) * rewrite cache library as codeberg.org/gruf/go-structr, implement in gotosocial * use actual go-structr release version (not just commit hash) * revert go toolchain changes (damn you go for auto changing this) * fix go mod woes * ensure %w is used in calls to errs.Appendf() * fix error checking * fix possible panic * remove unnecessary start/stop functions, move to main Cache{} struct, add note regarding which caches require start/stop * fix copy-paste artifact... :innocent: * fix all comment copy-paste artifacts * remove dropID() function, now we can just use slices.DeleteFunc() * use util.Deduplicate() instead of collate(), move collate to util * move orderByIDs() to util package and "generify" * add a util.DeleteIf() function, use this to delete entries on failed population * use slices.DeleteFunc() instead of util.DeleteIf() (i had the logic mixed up in my head somehow lol) * add note about how collate differs from deduplicate
2024-01-16[feature] Account alias / move API + db models (#2518)Libravatar tobi1
* [feature] Account alias / move API + db models * go fmt * fix little cherry-pick issues * update error checking, formatting * add and use new util functions to simplify alias logic
2024-01-05[feature] Parse instance descriptors as markdown, show T&C on /about (#2481)Libravatar tobi1
* [feature] Parse instance descriptors as markdown, show T&C on /about * lint * remove unnecessary nullzero tags
2023-12-27[chore] Refactor HTML templates and CSS (#2480)Libravatar tobi2
* [chore] Refactor HTML templates and CSS * eslint * ignore "Local" * rss tests * fiddle with OG just a tiny bit * dick around with polls a bit more so SR stops saying "clickable" * remove break * oh lord * don't lazy load avatar * fix ogmeta tests * clean up some cruft * catch remaining calls to c.HTML * fix error rendering + stack overflow in tag * allow templating attributes * fix indent * set aria-hidden on status complementary content, since it's already present in the label anyway * tidy up templating calls a little * try to make styling a bit more consistent + readable * fix up some remaining CSS issues * fix up reports
2023-12-10[bugfix] Ensure `pre` renders as expected, fix orderedCollectionPage (#2434)Libravatar tobi1
2023-12-09[bugfix] Fix web media not showing as sensitive (#2433)Libravatar tobi1
* [bugfix] Fix web media not showing as sensitive * test * go fmt
2023-11-22[feature] Poll web view (#2377)Libravatar tobi1
* [feature] Render polls nicely on the web view * use figure for poll, other small tweaks * reverse share + count (lines up better) * poll options list entries * fix up some remaining things
2023-11-20[feature] Initial Prometheus metrics implementation (#2334)Libravatar Tsuribori1
* feat: Initial OTEL metrics * docs: add metrics documentation * fix: metrics endpoint conditional check * feat: metrics endpoint basic auth * fix: make metrics-auth-enabled default false * fix: go fmt helpers.gen.go * fix: add metric-related env vars to envparsing.sh * fix: metrics docs * fix: metrics related stuff in envparsing.sh * fix: metrics docs * chore: metrics docs wording * fix: metrics stuff in envparsing? * bump otel versions --------- Co-authored-by: Tsuribori <user@acertaindebian> Co-authored-by: Tsuribori <none@example.org> Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-11-20[bugfix] self-referencing collection pages for status replies (#2364)Libravatar kim1
2023-11-17[feature] Set/show instance language(s); show post language on frontend (#2362)Libravatar tobi1
* update go text, include text/display * [feature] Set instance langs, show post lang on frontend * go fmt * WebGet * set language for whole article, don't use FA icon * mention instance languages + other optional config vars * little tweak * put languages in config properly * warn log language parse * change some naming around * tidy up validate a bit * lint * rename LanguageTmpl in template
2023-11-14[bugfix] process account delete side effects in serial, not in parallel (#2360)Libravatar tobi3
* [bugfix] process account delete side effects in serial, not in parallel * StartWorkers / StartNoopWorkers for tests * undo testrig trace logging * log errors instead of immediately returning
2023-11-13[feature/performance] Wrap incoming HTTP requests in timeout handler (#2353)Libravatar tobi1
* deinterface router, start messing about with deadlines * weeeee * thanks linter (thinter) * write Connection: close when timing out requests * update wording * don't replace req * don't bother with fancy Cause functions (I'll use them one day...)
2023-11-11[bugfix] support endless polls, and misskey's' method of inferring expiry in ↵Libravatar kim1
closed polls (#2349)
2023-11-10[feature] Media attachment placeholders (#2331)Libravatar tobi3
* [feature] Use placeholders for unknown media types * fix read of underreported small files * switch to reduce nesting * simplify cleanup
2023-11-10[chore/bugfix/horror] Allow `expires_in` and poll choices to be parsed from ↵Libravatar tobi1
strings (#2346)
2023-11-08[feature] add support for polls + receiving federated status edits (#2330)Libravatar kim2
2023-11-04[feature] support canceling scheduled tasks, some federation API performance ↵Libravatar kim2
improvements (#2329)
2023-10-30[feature] Customizable media cleaner schedule (#2304)Libravatar tobi2
2023-10-25[feature] Status thread mute/unmute functionality (#2278)Libravatar tobi2
* add db models + functions for keeping track of threads * give em the old linty testy * create, remove, check mutes * swagger * testerino * test mute/unmute via api * add info log about new index creation * thread + allow muting of any remote statuses that mention a local account * IsStatusThreadMutedBy -> IsThreadMutedByAccount * use common processing functions in status processor * set = NULL * favee! * get rekt darlings, darlings get rekt * testrig please, have mercy muy liege
2023-10-23[chore] de-interface{} the federator and dereferencer structs (#2285)Libravatar kim2
* de-interface{} the federator and dereferencer structs * fix broken type signatures
2023-10-21[bugfix/frontend] fix typo and other oddness in patchRemoteEmojis (#2281)v0.12.0Libravatar tobi1
* fix emoji test model * found the bug! * remove unused 'current' import * comment useChecklistReducer * wah * lint * fix cleaner tests
2023-09-23[chore] deinterface the typeutils.Converter and update to use state ↵Libravatar kim5
structure (#2217) * update typeconverter to use state structure * deinterface the typeutils.TypeConverter -> typeutils.Converter * finish copying over old type converter code comments * fix cherry-pick merge issues, fix tests pointing to old typeutils interface type still
2023-09-21[feature] Implement explicit domain allows + allowlist federation mode (#2200)Libravatar tobi1
* love like winter! wohoah, wohoah * domain allow side effects * tests! logging! unallow! * document federation modes * linty linterson * test * further adventures in documentation * finish up domain block documentation (i think) * change wording a wee little bit * docs, example * consolidate shared domainPermission code * call mode once * fetch federation mode within domain blocked func * read domain perm import in streaming manner * don't use pointer to slice for domain perms * don't bother copying blocks + allows before deleting * admonish! * change wording just a scooch * update docs
2023-09-12[feature] add paging to account follows, followers and follow requests ↵Libravatar kim1
endpoints (#2186)
2023-09-12[feature] Allow admins to expire remote public keys; refetch expired keys on ↵Libravatar tobi1
demand (#2183)
2023-09-04[feature] Store admin actions in the db, prevent conflicting actions (#2167)Libravatar tobi1
2023-08-21[feature] Make log format configurable (#2130)Libravatar Daenney2
* [feature] Don't emit timestamp in log lines When running gotosocial with a service manager like systemd, or a container runtime, the associated log driver usually emits timestamps itself. In those cases, having the extra timestamp from our own log lines ends up being a bit noisy and when centrally ingesting logs is duplicate information. This introduces a configuration flag that allows disabling emitting the timestamp. It's only wired up for "daemonised" processes, meaning server and testrig. * [chore] Add docs for log-timestamp * [feature] Simplify timestamp handling Co-Authored-By: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com> * [chore] Less escaped double-quotes * [chore] Fix help string --------- Co-authored-by: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>
2023-08-19[feature] Instance rules (#2125)Libravatar f0x522
* init instance rules database model, admin api * expose instance rules in public instance api * public /api/v1/instance/rules route * GET ruleById * createRule route * createRule auth check * updateRule * deleteRule * list rules on about page * ruleGet auth * add about page ids for anchors * process and store adding violated rules to reports * admin api models for instance rules * instance rule edit frontend * change rule inputs to textareas * database fixes after rebase (#2124) * remove unused imports * fix db migration column name * fix tests * fix more tests * fix postgres error with wrongly used Ident * add some tests, fiddle with rule model a bit, fix postgres migration * swagger docs --------- Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-08-09[feature] List replies policy, refactor async workers (#2087)Libravatar tobi2
* Add/update some DB functions. * move async workers into subprocessor * rename FromFederator -> FromFediAPI * update home timeline check to include check for current status first before moving to parent status * change streamMap to pointer to mollify linter * update followtoas func signature * fix merge * remove errant debug log * don't use separate errs.Combine() check to wrap errs * wrap parts of workers functionality in sub-structs * populate report using new db funcs * embed federator (tiny bit tidier) * flesh out error msg, add continue(!) * fix other error messages to be more specific * better, nicer * give parseURI util function a bit more util * missing headers * use pointers for subprocessors
2023-08-08[bugfix] don't accept unrelated statuses (#2078)Libravatar kim1
Co-authored-by: Daenney <daenney@users.noreply.github.com> Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-08-07[chore] Use generic pointer function (#2080)Libravatar Daenney1
This replaces the different $TypePtr functions with a generic implementation.
2023-07-31[feature] Hashtag federation (in/out), hashtag client API endpoints (#2032)Libravatar tobi1
* update go-fed * do the things * remove unused columns from tags * update to latest lingo from main * further tag shenanigans * serve stub page at tag endpoint * we did it lads * tests, oh tests, ohhh tests, oh tests (doo doo doo doo) * swagger docs * document hashtag usage + federation * instanceGet * don't bother parsing tag href * rename whereStartsWith -> whereStartsLike * remove GetOrCreateTag * dont cache status tag timelineability
2023-07-29[feature] Implement markers API (#1989)Libravatar Vyr Cossont2
* Implement markers API Fixes #1856 * Correct import grouping in markers files * Regenerate Swagger for markers API * Shorten names for readability * Cache markers for 6 hours * Update DB ref * Update envparsing.sh