summaryrefslogtreecommitdiff
path: root/internal/processing
AgeCommit message (Collapse)AuthorFiles
2025-05-05[bugfix] Prevent multiple mentions being created for one mention (#4127)Libravatar tobi2
# Description > If this is a code change, please include a summary of what you've coded, and link to the issue(s) it closes/implements. > > If this is a documentation change, please briefly describe what you've changed and why. This pull request fixes an issue where multiple mentions were being created for one statuses, when delivered to multiple inboxes on an instance, but only the final mention ended up actually being used. Also adds tests to make sure that a notification is received when a status is edited to include a mention. ## Checklist Please put an x inside each checkbox to indicate that you've read and followed it: `[ ]` -> `[x]` If this is a documentation change, only the first checkbox must be filled (you can delete the others if you want). - [x] I/we have read the [GoToSocial contribution guidelines](https://codeberg.org/superseriousbusiness/gotosocial/src/branch/main/CONTRIBUTING.md). - [x] I/we have discussed the proposed changes already, either in an issue on the repository, or in the Matrix chat. - [x] I/we have not leveraged AI to create the proposed changes. - [x] I/we have performed a self-review of added code. - [x] I/we have written code that is legible and maintainable by others. - [x] I/we have commented the added code, particularly in hard-to-understand areas. - [ ] I/we have made any necessary changes to documentation. - [x] I/we have added tests that cover new code. - [x] I/we have run tests and they pass locally with the changes. - [x] I/we have run `go fmt ./...` and `golangci-lint run`. Reviewed-on: https://codeberg.org/superseriousbusiness/gotosocial/pulls/4127 Co-authored-by: tobi <tobi.smethurst@protonmail.com> Co-committed-by: tobi <tobi.smethurst@protonmail.com>
2025-05-04[bugfix] Fix no notification if mention edited into status (#4102)Libravatar tobi5
This pull request adds mention notifications if a mention was edited into a status after its initial publication. Closes https://codeberg.org/superseriousbusiness/gotosocial/issues/3869 Reviewed-on: https://codeberg.org/superseriousbusiness/gotosocial/pulls/4102 Co-authored-by: tobi <tobi.smethurst@protonmail.com> Co-committed-by: tobi <tobi.smethurst@protonmail.com>
2025-05-03[bugfix] Ensure Account and TargetAccount set when doing UndoFollow (#4118)Libravatar tobi1
Closes https://codeberg.org/superseriousbusiness/gotosocial/issues/4112 by ensuring that Account and TargetAccount are properly set on the follow passed through to UndoFollow. Reviewed-on: https://codeberg.org/superseriousbusiness/gotosocial/pulls/4118 Co-authored-by: tobi <tobi.smethurst@protonmail.com> Co-committed-by: tobi <tobi.smethurst@protonmail.com>
2025-05-02[bugfix/chore] Invalidate prepared statuses when account representation ↵Libravatar tobi3
changes (#4107) This pull requests adds logic to invalidate prepared statuses when an account's representation changes due to an update. Closes https://codeberg.org/superseriousbusiness/gotosocial/issues/4104 Reviewed-on: https://codeberg.org/superseriousbusiness/gotosocial/pulls/4107 Co-authored-by: tobi <tobi.smethurst@protonmail.com> Co-committed-by: tobi <tobi.smethurst@protonmail.com>
2025-04-27[chore] Rewrite all remaining Github linksLibravatar Daenney1
2025-04-26[feature] Move to code.superseriousbusiness.orgLibravatar Daenney227
2025-04-26[performance] rewrite timelines to rely on new timeline cache type (#3941)Libravatar kim20
* start work rewriting timeline cache type * further work rewriting timeline caching * more work integration new timeline code * remove old code * add local timeline, fix up merge conflicts * remove old use of go-bytes * implement new timeline code into more areas of codebase, pull in latest go-mangler, go-mutexes, go-structr * remove old timeline package, add local timeline cache * remove references to old timeline types that needed starting up in tests * start adding page validation * fix test-identified timeline cache package issues * fix up more tests, fix missing required changes, etc * add exclusion for test.out in gitignore * clarify some things better in code comments * tweak cache size limits * fix list timeline cache fetching * further list timeline fixes * linter, ssssssssshhhhhhhhhhhh please * fix linter hints * reslice the output if it's beyond length of 'lim' * remove old timeline initialization code, bump go-structr to v0.9.4 * continued from previous commit * improved code comments * don't allow multiple entries for BoostOfID values to prevent repeated boosts of same boosts * finish writing more code comments * some variable renaming, for ease of following * change the way we update lo,hi paging values during timeline load * improved code comments for updated / returned lo , hi paging values * finish writing code comments for the StatusTimeline{} type itself * fill in more code comments * update go-structr version to latest with changed timeline unique indexing logic * have a local and public timeline *per user* * rewrite calls to public / local timeline calls * remove the zero length check, as lo, hi values might still be set * simplify timeline cache loading, fix lo/hi returns, fix timeline invalidation side-effects missing for some federated actions * swap the lo, hi values :facepalm: * add (now) missing slice reverse of tag timeline statuses when paging ASC * remove local / public caches (is out of scope for this work), share more timeline code * remove unnecessary change * again, remove more unused code * remove unused function to appease the linter * move boost checking to prepare function * fix use of timeline.lastOrder, fix incorrect range functions used * remove comments for repeat code * remove the boost logic from prepare function * do a maximum of 5 loads, not 10 * add repeat boost filtering logic, update go-structr, general improvements * more code comments * add important note * fix timeline tests now that timelines are returned in page order * remove unused field * add StatusTimeline{} tests * add more status timeline tests * start adding preloading support * ensure repeat boosts are marked in preloaded entries * share a bunch of the database load code in timeline cache, don't clear timelines on relationship change * add logic to allow dynamic clear / preloading of timelines * comment-out unused functions, but leave in place as we might end-up using them * fix timeline preload state check * much improved status timeline code comments * more code comments, don't bother inserting statuses if timeline not preloaded * shift around some logic to make sure things aren't accidentally left set * finish writing code comments * remove trim-after-insert behaviour * fix-up some comments referring to old logic * remove unsetting of lo, hi * fix preload repeatBoost checking logic * don't return on status filter errors, these are usually transient * better concurrency safety in Clear() and Done() * fix test broken due to addition of preloader * fix repeatBoost logic that doesn't account for already-hidden repeatBoosts * ensure edit submodels are dropped on cache insertion * update code-comment to expand CAS accronym * use a plus1hULID() instead of 24h * remove unused functions * add note that public / local timeline requester can be nil * fix incorrect visibility filtering of tag timeline statuses * ensure we filter home timeline statuses on local only * some small re-orderings to confirm query params in correct places * fix the local only home timeline filter func
2025-04-25[chore] Move deps to code.superseriousbusiness.org (#4054)Libravatar tobi5
2025-04-22[bugfix] Use util.IsNil for checking DomainPermission (#4040)Libravatar tobi1
2025-04-14ensure 'none' gets included in serializable visibilities (#4007)Libravatar kim1
2025-04-11[bugfix] Fix setting bot on/off (#3986)Libravatar tobi2
* [bugfix] Fix setting bot on/off * read client messages in tests * test fix
2025-04-10[feature] Implement /oauth/revoke for token revocation (#3983)Libravatar tobi1
2025-04-07[feature] add TOTP two-factor authentication (2FA) (#3960)Libravatar tobi5
* [feature] add TOTP two-factor authentication (2FA) * use byteutil.S2B to avoid allocations when comparing + generating password hashes * don't bother with string conversion for consts * use io.ReadFull * use MustGenerateSecret for backup codes * rename util functions
2025-04-06[chore] Migrate accounts to new table, relax uniqueness constraint of actor ↵Libravatar tobi8
`url` and collections (#3928) * [chore] Migrate accounts to new table, relax uniqueness constraint of actor url and collections * fiddle with it! (that's what she said) * remove unused cache fields * sillyness * fix tiny whoopsie
2025-04-04[feature] Allow editing domain blocks/allows, fix comment import (#3967)Libravatar tobi4
* start implementing editing of existing domain permissions * [feature] Allow editing domain blocks/allows, fix comment import * [bugfix] Use "comment" via /api/v1/instance * fix the stuff
2025-03-31[feature] Use blurhashes in frontend, tidy up gallery view a bit (#3948)Libravatar tobi1
* [feature] Use blurhashes in frontend, tidy up gallery view a bit * weeeeeeeeeeeeeeeee * beep boop
2025-03-26[feature] Allow user to choose "gallery" style layout for web view of ↵Libravatar tobi3
profile (#3917) * [feature] Allow user to choose "gallery" style web layout * find a bug and squish it up and all day long you'll have good luck * just a sec * [performance] reindex public timeline + tinker with query a bit * fiddling * should be good now * last bit of finagling, i'm done now i prommy * panic normally
2025-03-17[feature] Application creation + management via API + settings panel (#3906)Libravatar tobi6
* [feature] Application creation + management via API + settings panel * fix docs links * add errnorows test * use known application as shorter * add comment about side effects
2025-03-07[feature] Parse content warning to HTML, serialize via client API as ↵Libravatar tobi14
plaintext (#3876) * [feature] Parse content warning as HTML, serialize via API to plaintext * tidy up some cruft * whoops * oops * i'm da joker baybee * clemency muy lorde * rename some of the text functions for clarity * jiggle the opts * fiddle de deee * hopefully the last test fix i ever have to do in my beautiful life
2025-03-06[bugfix] Store and expose status content type (#3870)Libravatar ewwwin6
* Add ContentType to internal models * Add ContentType to API models StatusSource and StatusEdit * Add helpers to convert between API/internal StatusContentType * Write status content type on create/edit * Add migration * Update API docs go run github.com/go-swagger/go-swagger/cmd/swagger generate spec --scan-models --exclude-deps --output docs/api/swagger.yaml * ensure ContentType is updated anywhere Text is * Update docs, take care of TODOs * Set ContentType in more places where Text is set * We don't actually use ContentType on the API status model * Update StatusSource test * Remove unused helper function I copied * Revert change to StatusContentType swagger annotation I'm going to include this in a follow-on PR instead. * Add test for updating content type in edits * Return a value from processContentType instead of modifying the existing status Fixes an issue that was caught by the test I just added - the recorded edit would be marked with the *new* content type instead of the old one, which is obviously bad * Add test for handling of statuses with no stored content type * repurpose an existing test status instead of adding a new one to avoid breaking other tests * Add test to ensure newly created statuses always have content type saved * Do include content type on status API model actually This is mostly important when deleting and redrafting. The comment on `apimodel.Status.Text` implies that it's not sent except in response to status deletion, but actually this doesn't seem to be the case; it also appears to be present in responses to creations and normal fetches and stuff. So I'm treating `ContentType` the same here. * Update new tests to check content type on API statuses * Check content type of API statuses in all tests where text is checked * update other api tests with status content type field * Add test ensuring text and content type are returned when deleting a status * Convert processContentType to free function and remove unused parameter * check for the correct value in the deletion test * Be explicit about this test status having an empty content type * Use omitempty consistently on API models * clean up the final diff a bit * one more swagger regen for the road * Handle nil statuses in processContentType * Don't pass processContentType the entire edit form, it doesn't need it * Move processContentType to common.go and use for creation as well * Remove unused parameters to ContentTypeToAPIContentType
2025-03-04[feature] Add token review / delete to backend + settings panel (#3845)Libravatar tobi1
2025-03-03[feature] Refactor tokens, allow multiple app redirect_uris (#3849)Libravatar tobi11
* [feature] Refactor tokens, allow multiple app redirect_uris * move + tweak handlers a bit * return error for unset oauth2.ClientStore funcs * wrap UpdateToken with cache * panic handling * cheeky little time optimization * unlock on error
2025-03-02[chore] migrate oauth2 -> codeberg (#3857)Libravatar tobi2
2025-03-02[chore] github.com/superseriousbusiness/activity -> ↵Libravatar tobi3
codeberg.org/superseriousbusiness/activity (#3853)
2025-03-01[feature] Implement CSV import for mutes (#3696)Libravatar Xavier Vello1
* Implement CSV import for mutes * update swagger.yaml * update documentation * add ImportTestSuite.TestImportMutes * fix comment typo
2025-02-26[feature] Enforce OAuth token scopes (#3835)Libravatar tobi11
* move tokenauth to apiutil * enforce scopes * docs * update test models, remove deprecated "follow" * file header * tests * tweak scope matcher * simplify... * fix tests * log user out of settings panel in case of oauth error
2025-02-20[bugfix] update fedi api to support multiple separate votes in same multiple ↵Libravatar kim2
choice poll (#3809)
2025-02-19[feature] Forward-compatibility with Approval objects (#3807)Libravatar tobi1
* vendor * [feature] Forward-compatibility with Approval objects * vendor the thing * fix leetle bug * lil syntax tweak for beloved kimb
2025-02-17adds more code comments and some small code formatting tweaks (#3799)Libravatar kim2
2025-02-13[bugfix] Return 404 when web context target status hidden (#3792)Libravatar tobi1
2025-02-12[feature] Implement backfilling statuses thru scheduled_at (#3685)Libravatar Vyr Cossont4
* Implement backfilling statuses thru scheduled_at * Forbid mentioning others in backfills * Update error messages & codes * Add new tests for backfilled statuses * Test that backfilling doesn't timeline or notify * Fix check for absence of notification * Test that backfills do not cause federation * Fix type of apimodel.StatusCreateRequest.ScheduledAt in tests * Add config file switch and min date check
2025-02-10[feature] make account sign-up / backlog limits configurable (#3768)Libravatar tobi1
2025-02-09[bugfix] Fix POST to create account endpoint (#3767)Libravatar tobi3
2025-02-08[bug] Add missing `group` property to the Account model (#3746)Libravatar pnwmatt2
* Stub out account's Group parameter * Fix typo in Contributing documentation * Update swagger and match style * Update the swagger command to mimic test/swagger.sh * Fix tests for new `group` param in Account model * More test changes for new `group` param in Account * Continuing test changes for `group` param stubbing * Another round of `group` stubbing for tests
2025-02-04[bugfix] wrong nodeinfo version (tobi is a boob) (#3735)Libravatar tobi1
2025-02-04[feature] Change `instance-stats-randomize` to `instance-stats-mode` with ↵Libravatar tobi1
multiple options; implement nodeinfo 2.1 (#3734) * [feature] Change `instance-stats-randomize` to `instance-stats-mode` with multiple options; implement nodeinfo 2.1 * swaggalaggadingdong
2025-02-03[feature] Implement Web Push notification policy (#3721)Libravatar Vyr Cossont2
* Web Push: add policy column to subscriptions * Web Push: add policy to API * Web Push: test notification policy * go-fmt unrelated file (how did this get thru?)
2025-01-31[feature] Add `instance-stats-randomize` config option (#3718)Libravatar tobi1
* [feature] Add `instance-stats-randomize` config option * don't use cache (overkill)
2025-01-24[feature] Process incoming Undo Announce properly (#3676)Libravatar tobi2
* [feature] Process incoming Undo Announce properly * test undo announce
2025-01-23[feature] Push notifications (#3587)Libravatar Vyr Cossont16
* Update push subscription API model to be Mastodon 4.0 compatible * Add webpush-go dependency # Conflicts: # go.sum * Single-row table for storing instance's VAPID key pair * Generate VAPID key pair during startup * Add VAPID public key to instance info API * Return VAPID public key when registering an app * Store Web Push subscriptions in DB * Add Web Push sender (similar to email sender) * Add no-op push senders to most processor tests * Test Web Push notifications from workers * Delete Web Push subscriptions when account is deleted * Implement push subscription API * Linter fixes * Update Swagger * Fix enum to int migration * Fix GetVAPIDKeyPair * Create web push subscriptions table with indexes * Log Web Push server error messages * Send instance URL as Web Push JWT subject * Accept any 2xx code as a success * Fix malformed VAPID sub claim * Use packed notification flags * Remove unused date columns * Add notification type for update notifications Not used yet * Make GetVAPIDKeyPair idempotent and remove PutVAPIDKeyPair * Post-rebase fixes * go mod tidy * Special-case 400 errors other than 408/429 Most client errors should remove the subscription. * Improve titles, trim body to reasonable length * Disallow cleartext HTTP for Web Push servers * Fix lint * Remove redundant index on unique column Also removes redundant unique and notnull tags on ID column since these are implied by pk * Make realsender.go more readable * Use Tobi's style for wrapping errors * Restore treating all 5xx codes as temporary problems * Always load target account settings * Stub `policy` and `standard` * webpush.Sender: take type converter as ctor param * Move webpush.MockSender and noopSender into testrig
2025-01-23[feature] Serve bot accounts over AP as Service instead of Person (#3672)Libravatar tobi2
* pepis * oopsie doopsie * bollocks
2025-01-08[bugfix] More permissive CSV parsing for perm subs, text parse fix (#3638)Libravatar tobi1
* [bugfix] More permissive CSV parsing for perm subs, text parse fix * wee * change the way dry works, slightly * me oh my, i'm just a little guy * we're just normal men
2025-01-08[feature] Fetch + create domain permissions from subscriptions nightly (#3635)Libravatar tobi21
* peepeepoopoo * test domain perm subs * swagger * envparsing * dries your wets * start on docs * finish up docs * copy paste errors * rename actions package * rename force -> skipCache * move obfuscate parse nearer to where err is checked * make higherPrios a simple slice * don't use receiver for permsFrom funcs * add more context to error logs * defer finished log * use switch for permType instead of if/else * thanks linter, love you <3 * validate csv headers before full read * use bufio scanner
2025-01-08[chore] replace statuses.updated_at column with statuses.edited_at (#3636)Libravatar kim4
* update statuses table to replace updated_at column with edited_at * code comment * better code comments, fix setting of status + edit + mention + poll database times * fix log to logf call * fix status.EditIDs not being carried over in dereferencer.encrichStatus() * move status.EditID setting into handleStatusEdit()
2025-01-05[feature] Create/update/remove domain permission subscriptions (#3623)Libravatar tobi2
* [feature] Create/update/remove domain permission subscriptions * lint * envparsing * remove errant fmt.Println * create drafts, subs, exclude, from snapshot models * name etag column correctly * remove count column * lint
2024-12-23[feature] add support for clients editing statuses and fetching status ↵Libravatar kim13
revision history (#3628) * start adding client support for making status edits and viewing history * modify 'freshest' freshness window to be 5s, add typeutils test for status -> api edits * only populate the status edits when specifically requested * start adding some simple processor status edit tests * add test editing status but adding a poll * test edits appropriately adding poll expiry handlers * finish adding status edit tests * store both new and old revision emojis in status * add code comment * ensure the requester's account is populated before status edits * add code comments for status edit tests * update status edit form swagger comments * remove unused function * fix status source test * add more code comments, move media description check back to media process in status create * fix tests, add necessary form struct tag
2024-12-05[feature] add support for receiving federated status edits (#3597)Libravatar kim8
* add support for extracting Updated field from Statusable implementers * add support for status edits in the database, and update status dereferencer to handle them * remove unused AdditionalInfo{}.CreatedAt * remove unused AdditionalEmojiInfo{}.CreatedAt * update new mention creation to use status.UpdatedAt * remove mention.UpdatedAt, fixes related to NewULIDFromTime() change * add migration to remove Mention{}.UpdatedAt field * add migration to add the StatusEdit{} table * start adding tests, add delete function for status edits * add more of status edit migrations, fill in more of the necessary edit delete functionality * remove unused function * allow generating gotosocial compatible ulid via CLI with `go run ./cmd/gen-ulid` * add StatusEdit{} test models * fix new statusedits sql * use model instead of table name * actually remove the Mention.UpdatedAt field... * fix tests now new models are added, add more status edit DB tests * fix panic wording * add test for deleting status edits * don't automatically set `updated_at` field on updated statuses * flesh out more of the dereferencer status edit tests, ensure updated at field set on outgoing AS statuses * remove media_attachments.updated_at column * fix up more tests, further complete the dereferencer status edit tests * update more status serialization tests not expecting 'updated' AS property * gah!! json serialization tests!! * undo some gtscontext wrapping changes * more serialization test fixing :smiling_face_with_tear: * more test fixing, ensure the edit.status_id field is actually set :facepalm: * fix status edit test * grrr linter * add edited_at field to apimodel status * remove the choice of paging on the timeline public filtered test (otherwise it needs updating every time you add statuses ...) * ensure that status.updated_at always fits chronologically * fix more serialization tests ... * add more code comments * fix envparsing * update swagger file * properly handle media description changes during status edits * slight formatting tweak * code comment
2024-12-04[feature] unending polls (#3592)Libravatar kim1
* adds support for unending polls to be created locally * remove unused argument
2024-12-02[feature] Add global instance CSS customization setting (#3352)Libravatar Victor Dyotte1
Allow instance admins to add custom CSS that will affect every page of their instance. This is done with a new CustomCSS instance setting that works pretty much exactly like the Users CustomCSS property. This custom CSS is then requested for every page load. User styles/themes take precedence over this CSS. Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-11-28send out poll votes as separate create activities given that no other AP ↵Libravatar kim1
servers support multiple objects in a single activity (#3582)