summaryrefslogtreecommitdiff
path: root/internal/processing/status
AgeCommit message (Collapse)AuthorFiles
2023-11-08[feature] add support for polls + receiving federated status edits (#2330)Libravatar kim3
2023-10-25[feature] Status thread mute/unmute functionality (#2278)Libravatar tobi11
* add db models + functions for keeping track of threads * give em the old linty testy * create, remove, check mutes * swagger * testerino * test mute/unmute via api * add info log about new index creation * thread + allow muting of any remote statuses that mention a local account * IsStatusThreadMutedBy -> IsThreadMutedByAccount * use common processing functions in status processor * set = NULL * favee! * get rekt darlings, darlings get rekt * testrig please, have mercy muy liege
2023-10-23[chore] de-interface{} the federator and dereferencer structs (#2285)Libravatar kim2
* de-interface{} the federator and dereferencer structs * fix broken type signatures
2023-10-04[feature] tentatively start adding polls support (#2249)Libravatar kim2
2023-09-29[chore/bugfix] Deinterface text.Formatter, allow underscores in hashtags (#2233)Libravatar tobi2
2023-09-23[chore] deinterface the typeutils.Converter and update to use state ↵Libravatar kim6
structure (#2217) * update typeconverter to use state structure * deinterface the typeutils.TypeConverter -> typeutils.Converter * finish copying over old type converter code comments * fix cherry-pick merge issues, fix tests pointing to old typeutils interface type still
2023-08-11[bugfix] Use custom bluemonday policy to disallow inline img tags (#2100)Libravatar tobi1
2023-08-07[feature] Allow full BCP 47 in language inputs (#2067)Libravatar Vyr Cossont2
* Allow full BCP 47 in language inputs Fixes #2066 * Fuse validation and normalization for languages * Remove outdated comment line * Move post language canonicalization test
2023-08-04[performance] add caching of status fave, boost of, in reply to ID lists (#2060)Libravatar kim2
2023-06-11[bugfix] Invalidate timeline entries for status when stats change (#1879)Libravatar tobi3
2023-05-28[chore] tidy up media manager, add calling func to errors, build-script ↵Libravatar kim1
improvements (#1835) * media manager tidy-up: de-interface and remove unused PostDataFunc Signed-off-by: kim <grufwub@gmail.com> * remove last traces of media.Manager being an interface Signed-off-by: kim <grufwub@gmail.com> * update error to provide caller, allow tuneable via build tags Signed-off-by: kim <grufwub@gmail.com> * remove kim-specific build script changes Signed-off-by: kim <grufwub@gmail.com> * fix merge conflicts Signed-off-by: kim <grufwub@gmail.com> * update build-script to support externally setting build variables Signed-off-by: kim <grufwub@gmail.com> --------- Signed-off-by: kim <grufwub@gmail.com>
2023-05-25[feature] Add List functionality (#1802)Libravatar tobi1
* start working on lists * further list work * test list db functions nicely * more work on lists * peepoopeepoo * poke * start list timeline func * we're getting there lads * couldn't be me working on stuff... could it? * hook up handlers * fiddling * weeee * woah * screaming, pissing * fix streaming being a whiny baby * lint, small test fix, swagger * tidying up, testing * fucked! by the linter * move timelines to state like a boss * add timeline start to tests using state * invalidate lists
2023-05-12[feature] status refetch support (#1690)Libravatar kim3
* revamp http client to not limit requests, instead use sender worker Signed-off-by: kim <grufwub@gmail.com> * remove separate sender worker pool, spawn 2*GOMAXPROCS batch senders each time, no need for transport cache sweeping Signed-off-by: kim <grufwub@gmail.com> * improve batch senders to keep popping recipients until remote URL found Signed-off-by: kim <grufwub@gmail.com> * fix recipient looping issue Signed-off-by: kim <grufwub@gmail.com> * move request id ctx key to gtscontext, finish filling out more code comments, add basic support for not logging client IP Signed-off-by: kim <grufwub@gmail.com> * first draft of status refetching logic Signed-off-by: kim <grufwub@gmail.com> * fix testrig to use new federation alloc func signature Signed-off-by: kim <grufwub@gmail.com> * fix log format directive Signed-off-by: kim <grufwub@gmail.com> * add status fetched_at migration Signed-off-by: kim <grufwub@gmail.com> * remove unused / unchecked for error types Signed-off-by: kim <grufwub@gmail.com> * add back the used type... Signed-off-by: kim <grufwub@gmail.com> * add separate internal getStatus() function for derefThread() that doesn't recurse Signed-off-by: kim <grufwub@gmail.com> * improved mention and media attachment error handling Signed-off-by: kim <grufwub@gmail.com> * fix log and error format directives Signed-off-by: kim <grufwub@gmail.com> * update account deref to match status deref changes Signed-off-by: kim <grufwub@gmail.com> * very small code formatting change to make things clearer Signed-off-by: kim <grufwub@gmail.com> * add more code comments Signed-off-by: kim <grufwub@gmail.com> * improved code commenting Signed-off-by: kim <grufwub@gmail.com> * only check for required further derefs if needed Signed-off-by: kim <grufwub@gmail.com> * improved cache invalidation Signed-off-by: kim <grufwub@gmail.com> * tweak cache restarting to use a (very small) backoff Signed-off-by: kim <grufwub@gmail.com> * small readability changes and fixes Signed-off-by: kim <grufwub@gmail.com> * fix account sync issues Signed-off-by: kim <grufwub@gmail.com> * fix merge conflicts + update account enrichment to accept already-passed accountable Signed-off-by: kim <grufwub@gmail.com> * remove secondary function declaration Signed-off-by: kim <grufwub@gmail.com> * normalise dereferencer get status / account behaviour, fix remaining tests Signed-off-by: kim <grufwub@gmail.com> * fix remaining rebase conflicts, finish commenting code Signed-off-by: kim <grufwub@gmail.com> * appease the linter Signed-off-by: kim <grufwub@gmail.com> * add source file header Signed-off-by: kim <grufwub@gmail.com> * update to use TIMESTAMPTZ column type instead of just TIMESTAMP Signed-off-by: kim <grufwub@gmail.com> * don't pass in 'updated_at' to UpdateEmoji() Signed-off-by: kim <grufwub@gmail.com> * use new ap.Resolve{Account,Status}able() functions Signed-off-by: kim <grufwub@gmail.com> * remove the somewhat confusing rescoping of the same variable names Signed-off-by: kim <grufwub@gmail.com> * update migration file name, improved database delete error returns Signed-off-by: kim <grufwub@gmail.com> * formatting Signed-off-by: kim <grufwub@gmail.com> * improved multi-delete database functions to minimise DB calls Signed-off-by: kim <grufwub@gmail.com> * remove unused type Signed-off-by: kim <grufwub@gmail.com> * fix delete statements Signed-off-by: kim <grufwub@gmail.com> --------- Signed-off-by: kim <grufwub@gmail.com>
2023-03-28[performance] refactoring + add fave / follow / request / visibility caching ↵Libravatar kim7
(#1607) * refactor visibility checking, add caching for visibility * invalidate visibility cache items on account / status deletes * fix requester ID passed to visibility cache nil ptr * de-interface caches, fix home / public timeline caching + visibility * finish adding code comments for visibility filter * fix angry goconst linter warnings * actually finish adding filter visibility code comments for timeline functions * move home timeline status author check to after visibility * remove now-unused code * add more code comments * add TODO code comment, update printed cache start names * update printed cache names on stop * start adding separate follow(request) delete db functions, add specific visibility cache tests * add relationship type caching * fix getting local account follows / followed-bys, other small codebase improvements * simplify invalidation using cache hooks, add more GetAccountBy___() functions * fix boosting to return 404 if not boostable but no error (to not leak status ID) * remove dead code * improved placement of cache invalidation * update license headers * add example follow, follow-request config entries * add example visibility cache configuration to config file * use specific PutFollowRequest() instead of just Put() * add tests for all GetAccountBy() * add GetBlockBy() tests * update block to check primitive fields * update and finish adding Get{Account,Block,Follow,FollowRequest}By() tests * fix copy-pasted code * update envparsing test * whitespace * fix bun struct tag * add license header to gtscontext * fix old license header * improved error creation to not use fmt.Errorf() when not needed * fix various rebase conflicts, fix account test * remove commented-out code, fix-up mention caching * fix mention select bun statement * ensure mention target account populated, pass in context to customrenderer logging * remove more uncommented code, fix typeutil test * add statusfave database model caching * add status fave cache configuration * add status fave cache example config * woops, catch missed error. nice catch linter! * add back testrig panic on nil db * update example configuration to match defaults, slight tweak to cache configuration defaults * update envparsing test with new defaults * fetch followingget to use the follow target account * use accounnt.IsLocal() instead of empty domain check * use constants for the cache visibility type check * use bun.In() for notification type restriction in db query * include replies when fetching PublicTimeline() (to account for single-author threads in Visibility{}.StatusPublicTimelineable()) * use bun query building for nested select statements to ensure working with postgres * update public timeline future status checks to match visibility filter * same as previous, for home timeline * update public timeline tests to dynamically check for appropriate statuses * migrate accounts to allow unique constraint on public_key * provide minimal account with publicKey --------- Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-03-20[chore] Refactor account deleting/block logic, tidy up some other processing ↵Libravatar tobi8
things (#1599) * start refactoring account deletion * update to use state.DB * further messing about * some more tidying up * more tidying, cleaning, nice-making * further adventures in refactoring and the woes of technical debt * update fr accept/reject * poking + prodding * fix up deleting * create fave uri * don't log using requestingAccount.ID because it might be nil * move getBookmarks function * use exists query to check for status bookmark * use deletenotifications func * fiddle * delete follow request notif * split up some db functions * Fix possible nil pointer panic * fix more possible nil pointers * fix license headers * warn when follow missing (target) account * return wrapped err when bookmark/fave models can't be retrieved * simplify self account delete * warn log likely race condition * de-sillify status delete loop * move error check due north * warn when unfollowSideEffects has no target account * warn when no boost account is found * warn + dump follow when no account * more warnings * warn on fave account not set * move for loop inside anonymous function * fix funky logic * don't remove mutual account items on block; do make sure unfollow occurs in both directions!
2023-03-12[chore] Improve copyright header handling (#1608)Libravatar Daenney12
* [chore] Remove years from all license headers Years or year ranges aren't required in license headers. Many projects have removed them in recent years and it avoids a bit of yearly toil. In many cases our copyright claim was also a bit dodgy since we added the 2021-2023 header to files created after 2021 but you can't claim copyright into the past that way. * [chore] Add license header check This ensures a license header is always added to any new file. This avoids maintainers/reviewers needing to remember to check for and ask for it in case a contribution doesn't include it. * [chore] Add missing license headers * [chore] Further updates to license header * Use the more common // indentend comment format * Remove the hack we had for the linter now that we use the // format * Add SPDX license identifier
2023-03-03[bugfix] Fix unpinning statuses not working (#1582)Libravatar tobi1
And also fix unpinning/pinning potentially leaking the ID of followers-only statuses through returning 422 instead of 404. Also tests!
2023-03-02[feature] Advertise rich text formats, support content_type field (#1370)Libravatar Vyr Cossont2
* Advertise rich text formats, support content_type field * Update JSON in instance patch tests * Replace format with content_type everywhere * update migration to work with both pg and sqlite * regenerate swagger docs * update instance serialization + tests * fix up * learn to code tobi please, i'm begging you --------- Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-03-01[chore] move client/federator workerpools to Workers{} (#1575)Libravatar kim9
* replace concurrency worker pools with base models in State.Workers, update code and tests accordingly * improve code comment * change back testrig default log level * un-comment-out TestAnnounceTwice() and fix --------- Signed-off-by: kim <grufwub@gmail.com> Reviewed-by: tobi
2023-03-01[feature] Federate pinned posts (aka `featuredCollection`) in and out (#1560)Libravatar tobi1
* start fiddling * the ol' fiddle + update * start working on fetching statuses * poopy doopy doo where r u uwu * further adventures in featuring statuses * finishing up * fmt * simply status unpin loop * move empty featured check back to caller function * remove unnecessary log.WithContext calls * remove unnecessary IsIRI() checks * add explanatory comment about status URIs * change log level to error * better test names
2023-02-25[feature] Client API endpoints + v. basic web view for pinned posts (#1547)Libravatar tobi1
* implement status pin client api + web handler * make test names + comments more descriptive * don't use separate table for status pins * remove unused add + remove checking * tidy up + add some more tests
2023-02-22[chore] Deinterface processor and subprocessors (#1501)Libravatar tobi18
* [chore] Deinterface processor and subprocessors * expose subprocessors via function calls * missing license header
2023-02-03[chore] reformat GetAccount() functionality, support updating accounts based ↵Libravatar kim3
on last_fetch (#1411) * reformat GetAccount() functionality, and add UpdateAccount() function. * use fetched_at instead of last_webfingered_at * catch local "not found" errors. small formatting / error string changes * remove now unused error type * return nil when wrapping nil error * update expected error messages * return correct url for foss satan webfinger * add AP model for Some_User * normalize local domain * return notretrievable where appropriate * expose NewErrNotRetrievable * ensure webfinger for new accounts searched by uri * update local account short circuit * allow enrich to fail for already-known accounts * remove unused LastWebfingeredAt * expose test maps on mock http client * update Update test * reformat GetAccount() functionality, and add UpdateAccount() function. * use fetched_at instead of last_webfingered_at * catch local "not found" errors. small formatting / error string changes * remove nil error checks (we shouldn't be passing nil errors to newError() initializers) * remove mutex unlock on transport init fail (it hasn't yet been locked!) * woops add back the error wrapping to use ErrNotRetrievable * caches were never being started... :see_no_evil: --------- Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-02-03[chore] Text formatting overhaul (#1406)Libravatar Autumn!4
* Implement goldmark debug print for hashtags and mentions * Minify HTML in FromPlain * Convert plaintext status parser to goldmark * Move mention/tag/emoji finding logic into formatter * Combine mention and hashtag boundary characters * Normalize unicode when rendering hashtags
2023-01-05[chore] Update/add license headers for 2023 (#1304)Libravatar tobi20
2023-01-02[chore] The Big Middleware and API Refactor (tm) (#1250)Libravatar tobi3
* interim commit: start refactoring middlewares into package under router * another interim commit, this is becoming a big job * another fucking massive interim commit * refactor bookmarks to new style * ambassador, wiz zeze commits you are spoiling uz * she compiles, we're getting there * we're just normal men; we're just innocent men * apiutil * whoopsie * i'm glad noone reads commit msgs haha :blob_sweat: * use that weirdo go-bytesize library for maxMultipartMemory * fix media module paths
2022-12-09[feature] Allow users to create + delete bookbarks, and view bookmarked ↵Libravatar Matthew Phillips5
statuses (#1168) * Implement Bookmarks * Update based on review comments * Update swagger doc * Fix argument passing to status.Bookmark * Update changed test * Updates based on latest PR review
2022-11-24[bugfix] Fix status boosts giving 404 (#1137)Libravatar tobi1
We broke this at some point recently, and i'm not sure when. In any case, i updated some of the logic in there + added a test for it.
2022-11-24[chore] cleanup storage implementation, no need for multiple interface types ↵Libravatar kim1
(#1131) Signed-off-by: kim <grufwub@gmail.com> Signed-off-by: kim <grufwub@gmail.com>
2022-11-05[bugfix] Check `media-description-min-chars` on submission of new status (#960)Libravatar tobi4
2022-10-08[chore] Standardize database queries, use `bun.Ident()` properly (#886)Libravatar tobi1
* use bun.Ident for user queries * use bun.Ident for account queries * use bun.Ident for media queries * add DeleteAccount func * remove CaseInsensitive in Where+use Ident ipv Safe * update admin db * update domain, use ident * update emoji, use ident * update instance queries, use bun.Ident * fix media * update mentions, use bun ident * update relationship + tests * use tableexpr * add test follows to bun db test suite * update notifications * updatebyprimarykey => updatebyid * fix session * prefer explicit ID to pk * fix little fucky wucky * remove workaround * use proper db func for attachment selection * update status db * add m2m entries in test rig * fix up timeline * go fmt * fix status put issue * update GetAccountStatuses
2022-10-02fix emojis not showing in spoiler text (#878)Libravatar tobi2
2022-09-28[chore] update dependencies, bump to Go 1.19.1 (#826)Libravatar kim1
* update dependencies, bump Go version to 1.19 * bump test image Go version * update golangci-lint * update gotosocial-drone-build * sign * linting, go fmt * update swagger docs * update swagger docs * whitespace * update contributing.md * fuckin whoopsie doopsie * linterino, linteroni * fix followrequest test not starting processor * fix other api/client tests not starting processor * fix remaining tests where processor not started * bump go-runners version * don't check last-webfingered-at, processor may have updated this * update swagger command * update bun to latest version * fix embed to work the same as before with new bun Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-09-27[bugfix] Fix emphasis being added to emoji shortcodes with markdown parsing ↵v0.5.0Libravatar tobi2
(#856) * fix underscored emoji shortcodes being emphasized * remove footnote parsing from md
2022-09-21[chore] Tidy up status deletion, remove from cache too (#845)Libravatar tobi2
* add func for deleting status from db + cache * move deletes entirely back to processor and also only do a delete if the requesting account owns the item being deleted * tidy up unboost processing * delete status more efficiently * fix wrong account id on remote test attachments * fix federator test
2022-09-12[feature] Fetch + display custom emoji in statuses from remote instances (#807)Libravatar tobi1
* start implementing remote emoji fetcher * update status where pk * aaa * tidy up a little * check size limits for emojis * thank you linter, i love you <3 * update swagger docs * add emoji dereference test * make emoji max sizes configurable * normalize db.ErrAlreadyExists
2022-09-06[feature] Custom emoji updates (serve emoji via s2s api, tune db models) (#805)Libravatar tobi1
* migrate emojis * add get emoji to s2s (federation) API * add new emoji db + cache functions * add shortcodeDomain lookup for emojis * check existing emojis w/cache, not w/constraints * go fmt * add putEmoji func * use new db emoji funcs instead of where * remove emojistringstotags func * add unique constraint back in * fix up broken migration * update index
2022-08-27[bugfix] Status visibility + `excludeReplies` fixes (#769)Libravatar Blackle Morisanchetto1
* Fix some bugs when viewing a user's posts: include their self-replies (threads) even when excludeReplies is set, and use in_reply_to_uri instead of in_reply_to_id to filter out replies * Assign values to InReplyToURI when creating statuses. Add index and update old statuses with a migration
2022-08-23[bugfix] Fix boost of boost issue (#764)Libravatar tobi2
2022-08-15[chore] Update bun / sqlite versions; update gtsmodels (#754)Libravatar tobi3
* upstep bun and sqlite versions * allow specific columns to be updated in the db * only update necessary columns for user * bit tidier * only update necessary fields of media_attachment * only update relevant instance fields * update tests * update only specific account columns * use bool pointers on gtsmodels includes attachment, status, account, user * update columns more selectively * test all default fields on new account insert * updating remaining bools on gtsmodels * initialize pointer fields when extracting AP emoji * copy bools properly * add copyBoolPtr convenience function + test it * initialize false bool ptrs a bit more neatly
2022-08-06[feature] Let accounts set default status format, and use this when ↵Libravatar tobi1
processing new statuses (#739) * add post_format to acct & use it when making post * update swagger docs * add status_format updating to frontend * fix up tests * post_format => status_format * add status_format to account validation
2022-07-19[bugfix] html escape special characters in text instead of totally removing ↵Libravatar tobi1
them (#719) * remove minify dependency * tidy up some tests * remove pre + postformat funcs * rework sanitization + formatting * update tests * add some more markdown tests
2022-07-19[chore] use our own logging implementation (#716)Libravatar kim2
* first commit Signed-off-by: kim <grufwub@gmail.com> * replace logging with our own log library Signed-off-by: kim <grufwub@gmail.com> * fix imports Signed-off-by: kim <grufwub@gmail.com> * fix log imports Signed-off-by: kim <grufwub@gmail.com> * add license text Signed-off-by: kim <grufwub@gmail.com> * fix package import cycle between config and log package Signed-off-by: kim <grufwub@gmail.com> * fix empty kv.Fields{} being passed to WithFields() Signed-off-by: kim <grufwub@gmail.com> * fix uses of log.WithFields() with whitespace issues and empty slices Signed-off-by: kim <grufwub@gmail.com> * *linter related grumbling* Signed-off-by: kim <grufwub@gmail.com> * gofmt the codebase! also fix more log.WithFields() formatting issues Signed-off-by: kim <grufwub@gmail.com> * update testrig code to match new changes Signed-off-by: kim <grufwub@gmail.com> * fix error wrapping in non fmt.Errorf function Signed-off-by: kim <grufwub@gmail.com> * add benchmarking of log.Caller() vs non-cached Signed-off-by: kim <grufwub@gmail.com> * fix syslog tests, add standard build tags to test runner to ensure consistency Signed-off-by: kim <grufwub@gmail.com> * make syslog tests more robust Signed-off-by: kim <grufwub@gmail.com> * fix caller depth arithmatic (is that how you spell it?) Signed-off-by: kim <grufwub@gmail.com> * update to use unkeyed fields in kv.Field{} instances Signed-off-by: kim <grufwub@gmail.com> * update go-kv library Signed-off-by: kim <grufwub@gmail.com> * update libraries list Signed-off-by: kim <grufwub@gmail.com> * fuck you linter get nerfed Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
2022-07-03[feature] S3 support (#674)Libravatar Dominik Süß1
* feat: vendor minio client * feat: introduce storage package with s3 support * feat: serve s3 files directly this saves a lot of bandwith as the files are fetched from the object store directly * fix: use explicit local storage in tests * feat: integrate s3 storage with the main server * fix: add s3 config to cli tests * docs: explicitly set values in example config also adds license header to the storage package * fix: use better http status code on s3 redirect HTTP 302 Found is the best fit, as it signifies that the resource requested was found but not under its presumed URL 307/TemporaryRedirect would mean that this resource is usually located here, not in this case 303/SeeOther indicates that the redirection does not link to the requested resource but to another page * refactor: use context in storage driver interface
2022-06-11[chore] Webfinger rework (#627)Libravatar tobi1
* move finger to dereferencer * totally break GetRemoteAccount * start reworking finger func a bit * start reworking getRemoteAccount a bit * move mention parts to namestring * rework webfingerget * use util function to extract webfinger parts * use accountDomain * rework finger again, final form * just a real nasty commit, the worst * remove refresh from account * use new ASRepToAccount signature * fix incorrect debug call * fix for new getRemoteAccount * rework GetRemoteAccount * start updating tests to remove repetition * break a lot of tests Move shared test logic into the testrig, rather than having it scattered all over the place. This allows us to just mock the transport controller once, and have all tests use it (unless they need not to for some other reason). * fix up tests to use main mock httpclient * webfinger only if necessary * cheeky linting with the lads * update mentionName regex recognize instance accounts * don't finger instance accounts * test webfinger part extraction * increase default worker count to 4 per cpu * don't repeat regex parsing * final search for discovered accountDomain * be more permissive in namestring lookup * add more extraction tests * simplify GetParseMentionFunc * skip long search if local account * fix broken test
2022-06-08[feature] More consistent API error handling (#637)Libravatar tobi3
* update templates * start reworking api error handling * update template * return AP status at web endpoint if negotiated * start making api error handling much more consistent * update account endpoints to new error handling * use new api error handling in admin endpoints * go fmt ./... * use api error logic in app * use generic error handling in auth * don't export generic error handler * don't defer clearing session * user nicer error handling on oidc callback handler * tidy up the sign in handler * tidy up the token handler * use nicer error handling in blocksget * auth emojis endpoint * fix up remaining api endpoints * fix whoopsie during login flow * regenerate swagger docs * change http error logging to debug
2022-05-26[bugfix] Fix HTML escaping in instance title (#607)Libravatar tobi2
* move caption sanitization -> sanitize.go * use sanitizeplaintext rather than removehtml * rename sanitizecaption to sanitizeplaintext * avoid removing html twice from statuses * unexport remoteHTML it's no longer used outside the text package so this makes it less confusing * test instance PATCH
2022-05-15[security] transport.Controller{} and transport.Transport{} security and ↵Libravatar kim2
performance improvements (#564) * cache transports in controller by privkey-generated pubkey, add retry logic to transport requests Signed-off-by: kim <grufwub@gmail.com> * update code comments, defer mutex unlocks Signed-off-by: kim <grufwub@gmail.com> * add count to 'performing request' log message Signed-off-by: kim <grufwub@gmail.com> * reduce repeated conversions of same url.URL object Signed-off-by: kim <grufwub@gmail.com> * move worker.Worker to concurrency subpackage, add WorkQueue type, limit transport http client use by WorkQueue Signed-off-by: kim <grufwub@gmail.com> * fix security advisories regarding max outgoing conns, max rsp body size - implemented by a new httpclient.Client{} that wraps an underlying client with a queue to limit connections, and limit reader wrapping a response body with a configured maximum size - update pub.HttpClient args passed around to be this new httpclient.Client{} Signed-off-by: kim <grufwub@gmail.com> * add httpclient tests, move ip validation to separate package + change mechanism Signed-off-by: kim <grufwub@gmail.com> * fix merge conflicts Signed-off-by: kim <grufwub@gmail.com> * use singular mutex in transport rather than separate signer mus Signed-off-by: kim <grufwub@gmail.com> * improved useragent string Signed-off-by: kim <grufwub@gmail.com> * add note regarding missing test Signed-off-by: kim <grufwub@gmail.com> * remove useragent field from transport (instead store in controller) Signed-off-by: kim <grufwub@gmail.com> * shutup linter Signed-off-by: kim <grufwub@gmail.com> * reset other signing headers on each loop iteration Signed-off-by: kim <grufwub@gmail.com> * respect request ctx during retry-backoff sleep period Signed-off-by: kim <grufwub@gmail.com> * use external pkg with docs explaining performance "hack" Signed-off-by: kim <grufwub@gmail.com> * use http package constants instead of string method literals Signed-off-by: kim <grufwub@gmail.com> * add license file headers Signed-off-by: kim <grufwub@gmail.com> * update code comment to match new func names Signed-off-by: kim <grufwub@gmail.com> * updates to user-agent string Signed-off-by: kim <grufwub@gmail.com> * update signed testrig models to fit with new transport logic (instead uses separate signer now) Signed-off-by: kim <grufwub@gmail.com> * fuck you linter Signed-off-by: kim <grufwub@gmail.com>
2022-05-07[bugfix] Fix existing bio text showing as HTML (#531)Libravatar kim2
* fix existing bio text showing as HTML - updated replaced mentions to include instance - strips HTML from account source note in Verify handler - update text formatter to use buffers for string writes Signed-off-by: kim <grufwub@gmail.com> * go away linter Signed-off-by: kim <grufwub@gmail.com> * change buf reset location, change html mention tags Signed-off-by: kim <grufwub@gmail.com> * reduce FindLinks code complexity Signed-off-by: kim <grufwub@gmail.com> * fix HTML to text conversion Signed-off-by: kim <grufwub@gmail.com> * Update internal/regexes/regexes.go Co-authored-by: Mina Galić <mina.galic@puppet.com> * use improved html2text lib with more options Signed-off-by: kim <grufwub@gmail.com> * fix to produce actual plaintext from html Signed-off-by: kim <grufwub@gmail.com> * fix span tags instead written as space Signed-off-by: kim <grufwub@gmail.com> * performance improvements to regex replacements, fix link replace logic for un-html-ing in the future Signed-off-by: kim <grufwub@gmail.com> * fix tag/mention replacements to use input string, fix link replace to not include scheme Signed-off-by: kim <grufwub@gmail.com> * use matched input string for link replace href text Signed-off-by: kim <grufwub@gmail.com> * remove unused code (to appease linter :sobs:) Signed-off-by: kim <grufwub@gmail.com> * improve hashtagFinger regex to be more compliant Signed-off-by: kim <grufwub@gmail.com> * update breakReplacer to include both unix and windows line endings Signed-off-by: kim <grufwub@gmail.com> * add NoteRaw field to Account to store plaintext account bio, add migration for this, set for sensitive accounts Signed-off-by: kim <grufwub@gmail.com> * drop unnecessary code Signed-off-by: kim <grufwub@gmail.com> * update text package tests to fix logic changes Signed-off-by: kim <grufwub@gmail.com> * add raw note content testing to account update and account verify Signed-off-by: kim <grufwub@gmail.com> * remove unused modules Signed-off-by: kim <grufwub@gmail.com> * fix emoji regex Signed-off-by: kim <grufwub@gmail.com> * fix replacement of hashtags Signed-off-by: kim <grufwub@gmail.com> * update code comment Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: Mina Galić <mina.galic@puppet.com>
2022-05-02[bugfix] Allow self-boosting for any visibility but direct (#510)Libravatar Adelie Paull1
* create visibility filter for boostability and allow self-boosting for any visbility but direct messages * add a followers-only status to local_account_2 * fix typo in comment * add license header, unwrap errors, be explicit about non-boostable visibility settings to avoid rogue boosting from miscoded clients, use ID compare for checking if self-boosting * add tests for statusboostable filter * fix tests that were affected by adding a new status to the test data * fix the rest of tests affected by adding a status to the textrig data