summaryrefslogtreecommitdiff
path: root/internal/processing
AgeCommit message (Collapse)AuthorFiles
2023-01-08[bugfix] fix panic during status delete loop by breaking out early on ↵Libravatar kim1
len(statuses) == 0 (#1317) Signed-off-by: kim <grufwub@gmail.com> Signed-off-by: kim <grufwub@gmail.com>
2023-01-05[chore] Update/add license headers for 2023 (#1304)Libravatar tobi120
2023-01-02[chore] The Big Middleware and API Refactor (tm) (#1250)Libravatar tobi10
* interim commit: start refactoring middlewares into package under router * another interim commit, this is becoming a big job * another fucking massive interim commit * refactor bookmarks to new style * ambassador, wiz zeze commits you are spoiling uz * she compiles, we're getting there * we're just normal men; we're just innocent men * apiutil * whoopsie * i'm glad noone reads commit msgs haha :blob_sweat: * use that weirdo go-bytesize library for maxMultipartMemory * fix media module paths
2022-12-21[bugfix] Close reader gracefully when streaming recache of remote media to ↵Libravatar tobi3
fileserver api caller (#1281) * close pipereader on failed data function * gently slurp the bytes * readability updates * go fmt * tidy up file server tests + add more cases * start moving io wrappers to separate iotools package. Remove use of buffering while piping recache stream Signed-off-by: kim <grufwub@gmail.com> * add license text Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: kim <grufwub@gmail.com>
2022-12-10[feature] Start implementing refetch of lost media files via ↵Libravatar tobi5
`/api/v1/admin/media_refetch` (#1221) * [chore] Move ShortcodeDomain to its own little util func * [feature] Add RefetchEmojis function to media manager * [feature] Expose admin media refresh via admin API * update following review feedback - change/fix log levels - make sure not to try to refetch local emojis - small style refactoring + comments * log on emoji refetch start Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: kim <grufwub@gmail.com>
2022-12-09[feature] Allow users to create + delete bookbarks, and view bookmarked ↵Libravatar Matthew Phillips10
statuses (#1168) * Implement Bookmarks * Update based on review comments * Update swagger doc * Fix argument passing to status.Bookmark * Update changed test * Updates based on latest PR review
2022-12-06[feature] overhaul the oidc system (#961)Libravatar Dominik Süß1
* [feature] overhaul the oidc system this allows for more flexible username handling and prevents account takeover using old email addresses * [feature] add migration path for old OIDC users * [feature] nicer error reporting for users * [docs] document the new OIDC flow * [fix] return early on oidc error * [docs]: add comments on the finalization logic
2022-11-30[bugfix] Don't call `strings.ToLower()` on usernames when selecting account ↵Libravatar tobi2
by domain+username (#1190) * don't lowercase account username when doing a select * test getting remote user with uppercase username
2022-11-30[bugfix] Avoid accidentally marking changed emojis as orphaned + pruning ↵Libravatar tobi1
them (#1188) * add predictable instance account to tests, use it in emoji urls + paths * use static image url to select emojis when pruning orphaned
2022-11-29[chore] Tidy up some of the search logic (#1082)v0.6.0-rc1Libravatar tobi10
* start refactoring some of the search + deref logic * add tests for search api * rename GetRemoteAccount + GetRemoteStatus * make search function a bit simpler + clearer * fix little fucky wucky uwu owo i'm just a little guy * update faulty switch statements * update test to use storage struct * redo switches for clarity * reduce repeated logic in search tests * fastfail getstatus by uri * debug log + trace log better * add implementation note * return early if no result for namestring search * return + check on dereferencing error types * errors hah what errors * remove unneeded error type alias, add custom error text during stringification itself * fix a woops recursion :see_no_evil: Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: kim <grufwub@gmail.com>
2022-11-24[feature] `PATCH /api/v1/admin/custom_emojis/{id}` endpoint (#1061)Libravatar tobi4
* start adding admin emoji PATCH stuff * updating works OK, now how about copying * allow emojis to be copied * update swagger docs * update admin processer to use non-interface storage driver * remove shortcode updating for local emojis * go fmt Co-authored-by: f0x52 <f0x@cthu.lu>
2022-11-24[bugfix] Fix status boosts giving 404 (#1137)Libravatar tobi1
We broke this at some point recently, and i'm not sure when. In any case, i updated some of the logic in there + added a test for it.
2022-11-24[chore] cleanup storage implementation, no need for multiple interface types ↵Libravatar kim6
(#1131) Signed-off-by: kim <grufwub@gmail.com> Signed-off-by: kim <grufwub@gmail.com>
2022-11-23[feature/performance] Fail fast when doing remote transport calls inside ↵Libravatar tobi10
incoming request contexts (#1119) * [feature/performance] Fail fast when doing remote transport calls inside incoming request contexts * [chore] Reduce outgoing request timeout to 15s * log error messages when fastfailing * use context.Value() instead of wrapped context, wrap error with fastfail instead of extra log entry * add fast-fail context key test Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: kim <grufwub@gmail.com>
2022-11-22[feature] Prune timelines once per hour to plug memory leak (#1117)Libravatar tobi1
* export highest/lowest ULIDs as proper const * add stop + start to timeline manager, other small fixes * unexport unused interface funcs + tidy up * add LastGot func * add timeline Prune function * test prune * update lastGot
2022-11-20[performance] add account block DB cache and remove block query joins (#1085)Libravatar kim3
* add account block DB cache and remove reliance on relational joins * actually include cache key arguments... * add a PutBlock() method which also updates the block cache, update tests accordingly * use `PutBlock` instead of `Put(ctx, block)` * add + use functions for deleting + invalidating blocks Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-11-20[bugfix] fix possible infinite loop on federated AP profile delete (#1091)Libravatar kim1
* refactor federator account statuses delete to better catch errors, ensure next maxID is always set Signed-off-by: kim <grufwub@gmail.com> * fix error statement missing 2nd format operator Signed-off-by: kim <grufwub@gmail.com> Signed-off-by: kim <grufwub@gmail.com>
2022-11-19[feature] Support markdown format for Account bio/note (#1037)Libravatar Michael Manfre2
* [feature] Status format also controls bio format * test
2022-11-15[chore] update database caching library (#1040)Libravatar kim4
* convert most of the caches to use result.Cache{} * add caching of emojis * fix issues causing failing tests * update go-cache/v2 instances with v3 * fix getnotification * add a note about the left-in StatusCreate comment * update EmojiCategory db access to use new result.Cache{} * fix possible panic in getstatusparents * further proof that kim is not stinky
2022-11-14[feature] Allow newly uploaded emojis to be placed in categories (#939)Libravatar tobi6
* [feature] Add emoji categories GET Serialize emojis in appropriate categories; make it possible to get categories via the admin API * [feature] Create (or use existing) category for new emoji uploads * fix lint issue * update misleading line in swagger docs
2022-11-14[feature] add instance-expose-public-timeline flag (#1039)Libravatar Rob Sargant1
* Add instance-expose-public-timeline flag Adds a config flag that allows unauthenticated access to /api/v1/timelines/public. Defaults to false to replicate existing behaviour. * Update structure following review * Add comment * Fix linting
2022-11-08[feature] Make instance thumbnail configurable via admin panel (#973)Libravatar tobi3
* [feature] Make instance thumbnail configurable via admin panel * log db errors in InstanceToAPIInstance * only update instance in db if necessary * start adding tests * finish test
2022-11-05[bugfix] Check `media-description-min-chars` on submission of new status (#960)Libravatar tobi4
2022-11-03[bugfix] Wrap media in read closer (#941)Libravatar tobi5
* use readcloser for content.Content * call media postdata function no matter what * return a readcloser from data func * tidy of logic of readertostore * fix whoopsie
2022-10-14[feature] Add emoji DELETE handler at `/api/v1/admin/custom_emojis` (#913)Libravatar tobi4
* add emoji DELETE handler * no need to process error (thanks kim) * don't double check if user is admin * add missing security annotation
2022-10-13[feature] Add `/api/v1/admin/custom_emojis/{id}` endpoint for single emoji ↵Libravatar tobi4
GET (#910) * fix error in prev swagger docs * add GET for single admin emoji
2022-10-13[feature] Refetch emojis when they change on remote instances (#905)Libravatar tobi2
* select emoji using image_static_url * use updated on AP emojis * allow refetch of updated emojis * cheeky workaround for test * clean up old files for refreshed emoji * check error for originalPostData * shorten GetEmojiByStaticImageURL * delete kirby (sorry nintendo)
2022-10-12[feature] Add `/api/v1/admin/custom_emojis` endpoint (#902)Libravatar tobi6
* add admin emojis get path + model + docs * stub admin emojis get processor function * add id + disabled fields to admin emoji * add emoji -> api admin emoji converter * tidy up a bit * add GetEmojis function * finish up get emojis function * order by shortcodedomain * ASC * tidy up + explain * update to allow paging * make admin emojis pageable * fix mixed case paging * normalize emoji queries a bit better * test emoji get paging * make limit optional * fix incorrect path in media cleanup tests * i have bad coder syndrome * don't trimspace * rename -> GetUseableEmojis * wrap emoji query in subquery avoid selecting more than we need * fix a bit of sillyness teehee * fix subquery postgres woes
2022-10-10[chore] Make paging logic more generic (#901)Libravatar tobi6
* make paging logic more generic not just for timelines! * linty linterson
2022-10-08[feature] Add opt-in RSS feed for account's latest Public posts (#897)Libravatar tobi6
* start adding rss functionality * add gorilla/feeds dependency * first bash at building rss feed still needs work, this is an interim commit * tidy up a bit * add publicOnly option to GetAccountLastPosted * implement rss endpoint * fix test * add initial user docs for rss * update rss logo * docs update * add rssFeed to frontend * feed -> feed.rss * enableRSS * increase rss logo size a lil bit * add rss toggle * move emojify to text package * fiddle with rss feed formatting * add Text field to test statuses * move status to rss item to typeconverter * update bun schema for enablerss * simplify 304 checking * assume account not rss * update tests * update swagger docs * allow more characters in title, trim nicer * update last posted to be more consistent
2022-10-08[chore] Standardize database queries, use `bun.Ident()` properly (#886)Libravatar tobi10
* use bun.Ident for user queries * use bun.Ident for account queries * use bun.Ident for media queries * add DeleteAccount func * remove CaseInsensitive in Where+use Ident ipv Safe * update admin db * update domain, use ident * update emoji, use ident * update instance queries, use bun.Ident * fix media * update mentions, use bun ident * update relationship + tests * use tableexpr * add test follows to bun db test suite * update notifications * updatebyprimarykey => updatebyid * fix session * prefer explicit ID to pk * fix little fucky wucky * remove workaround * use proper db func for attachment selection * update status db * add m2m entries in test rig * fix up timeline * go fmt * fix status put issue * update GetAccountStatuses
2022-10-08[feature] `oob` oauth token support (#889)Libravatar tobi2
* move helpful advice into oauth server * rewrite HandleAuthorizeRequest to allow oob
2022-10-06[bugfix] Fix new domain block date (#893)Libravatar tobi1
2022-10-04[bugfix] Use background context instead of request context for async ↵Libravatar tobi2
processing (#888) Fixes an issue where async processing was not completing correctly. In particular this applies to side effects of domain blocks: while the domain block was being entered and enforced correctly, side effects like deleting accounts and updating the instance entry for the blocked instance were not. This fixes that :)
2022-10-03[performance] add user cache and database (#879)Libravatar tobi6
* go fmt * add + use user cache and database * fix import * update tests * remove unused relation
2022-10-02fix emojis not showing in spoiler text (#878)Libravatar tobi2
2022-09-29[feature] update config types to use bytesize.Size (#828)Libravatar kim4
* update config size types to use bytesize.Size * submit unchecked-out file ... :facepalm: * fix bytesize config var decoding * bump bytesize version * update kim's libraries in readme * update envparse.sh to output more useful errors * improve envparse.sh * remove reliance on jq * instead, use uint64 for bytesize flag types * remove redundant type * fix viper unmarshaling * Update envparsing.sh * fix envparsing test Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
2022-09-28[chore] update dependencies, bump to Go 1.19.1 (#826)Libravatar kim1
* update dependencies, bump Go version to 1.19 * bump test image Go version * update golangci-lint * update gotosocial-drone-build * sign * linting, go fmt * update swagger docs * update swagger docs * whitespace * update contributing.md * fuckin whoopsie doopsie * linterino, linteroni * fix followrequest test not starting processor * fix other api/client tests not starting processor * fix remaining tests where processor not started * bump go-runners version * don't check last-webfingered-at, processor may have updated this * update swagger command * update bun to latest version * fix embed to work the same as before with new bun Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-09-27[bugfix] Fix emphasis being added to emoji shortcodes with markdown parsing ↵v0.5.0Libravatar tobi2
(#856) * fix underscored emoji shortcodes being emphasized * remove footnote parsing from md
2022-09-26[feature] Show + federate emojis in accounts (#837)Libravatar tobi3
* Start adding account emoji * get emojis serialized + deserialized nicely * update tests * set / retrieve emojis on accounts * show account emojis in web view * fetch emojis from db based on ids * fix typo in test * lint * fix pg migration * update tests * update emoji checking logic * update comment * clarify comments + add some spacing * tidy up loops a lil (thanks kim)
2022-09-25[bugfix] update thread iterators to not use recursion (#851)Libravatar kim1
* update thread iterators to not use recursion, rewrote both Signed-off-by: kim <grufwub@gmail.com> * fix endless descendant deref, don't error if fetching existing status Signed-off-by: kim <grufwub@gmail.com> * don't refetch remote ancestor statuses, improve descendant iter commenting Signed-off-by: kim <grufwub@gmail.com> * move collection page next logic so we capture first page of entities Signed-off-by: kim <grufwub@gmail.com> * improve log format argument quoting Signed-off-by: kim <grufwub@gmail.com> * improve code commenting of collection paging Signed-off-by: kim <grufwub@gmail.com> * only dereference announce's originating status if _not_ local. update DereferenceThread() signature. cleanup searchStatusByURI() Signed-off-by: kim <grufwub@gmail.com> Signed-off-by: kim <grufwub@gmail.com>
2022-09-23[feature] Allow delivery to sharedInboxes where possible (#847)Libravatar tobi2
* update Activity * add instance-deliver-to-shared-inboxes setting * update activity version again * add SharedInboxURI field to accounts * serdes for endpoints/sharedInbox * deliver to sharedInbox if one is available * update tests * only assign shared inbox if shared domain * look for shared inbox if currently nil * go fmt * finger to get params.RemoteAccountID if necessary * make comments clearer * compare dns more consistently
2022-09-21[chore] Tidy up status deletion, remove from cache too (#845)Libravatar tobi6
* add func for deleting status from db + cache * move deletes entirely back to processor and also only do a delete if the requesting account owns the item being deleted * tidy up unboost processing * delete status more efficiently * fix wrong account id on remote test attachments * fix federator test
2022-09-12[feature] Allow users to set custom css for their profiles + threads (#808)Libravatar tobi5
* add custom css account property + db func to fetch * allow account to get/set custom css * serve custom css for an account * go fmt * use monospace for customcss, add link * add custom css to account cache * fix broken field * add custom css docs to user guide * add `accounts-allow-custom-css` config flag * add allow custom css to /api/v1/instance response * only show/set custom css if allowed to do so * only set/serve custom account css if enabled * update swagger docs * chain promise * make bool a bit clearer * use cache for GetAccountCustomCSSByUsername
2022-09-12[feature] Fetch + display custom emoji in statuses from remote instances (#807)Libravatar tobi1
* start implementing remote emoji fetcher * update status where pk * aaa * tidy up a little * check size limits for emojis * thank you linter, i love you <3 * update swagger docs * add emoji dereference test * make emoji max sizes configurable * normalize db.ErrAlreadyExists
2022-09-06[feature] Custom emoji updates (serve emoji via s2s api, tune db models) (#805)Libravatar tobi7
* migrate emojis * add get emoji to s2s (federation) API * add new emoji db + cache functions * add shortcodeDomain lookup for emojis * check existing emojis w/cache, not w/constraints * go fmt * add putEmoji func * use new db emoji funcs instead of where * remove emojistringstotags func * add unique constraint back in * fix up broken migration * update index
2022-09-04[feature] Set default header and avatar for API accounts to GtS ones (#799)Libravatar tobi1
* validate web-asset-base-dir * move default icons into converter * always ensure avatar + header on api accounts * update tests * add default header * don't return error from web module creation anymore * tidy a bit * use pngs for default avatars rather than svgs
2022-09-02[performance] cache recently allowed/denied domains to cut down on db calls ↵Libravatar kim2
(#794) * fetch creation and fetching domain blocks from db Signed-off-by: kim <grufwub@gmail.com> * add separate domainblock cache type, handle removing block from cache on delete Signed-off-by: kim <grufwub@gmail.com> * fix sentinel nil values being passed into cache Signed-off-by: kim <grufwub@gmail.com> Signed-off-by: kim <grufwub@gmail.com>
2022-09-02[performance] use GetAccountByUsernameDomain() for local account lookups to ↵Libravatar kim11
rely on cache (#793) Signed-off-by: kim <grufwub@gmail.com> Signed-off-by: kim <grufwub@gmail.com>
2022-08-31[feature] Add support for the exclude_types[] parameter on the notifications ↵Libravatar Blackle Morisanchetto3
endpoint (#784) * Add support for the exclude_types[] parameter on the notifications endpoint * Add swagger docs to notifications
generated by cgit v1.2.3 (git 2.46.0) at 2025-07-14 11:53:49 +0000