summaryrefslogtreecommitdiff
path: root/internal/httpclient/client.go
AgeCommit message (Collapse)AuthorFiles
2022-11-26Additional IP range validations (#1152)Libravatar Daniele Sluijters1
* [bugfix] Ensure requests happen over TCP It's possible for the network to be udp4 or udp6. This is rather unlikely to occur, but since we're given the network anyway as part of the Sanitize function getting called we might as well check for it. * [chore] Align reserved v6 blocks to IANA registry * [chore] Add test for ValidateIP The net and netip packages diverge in that net.ParseIP will consider an IPv4-mapped address to be an IPv4 address and as such it would get caught by the IPv4Reserved list. However, netip considers it an IPv6 address, so we need to ensure the mapped range is in IPv6Reserved. * [chore] Align reserved v4 blocks to IANA registry This includes a number of tests for /32's explicitly called out in the registry to ensure we always consider those invalid.
2022-11-23[feature/performance] Fail fast when doing remote transport calls inside ↵Libravatar tobi1
incoming request contexts (#1119) * [feature/performance] Fail fast when doing remote transport calls inside incoming request contexts * [chore] Reduce outgoing request timeout to 15s * log error messages when fastfailing * use context.Value() instead of wrapped context, wrap error with fastfail instead of extra log entry * add fast-fail context key test Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: kim <grufwub@gmail.com>
2022-11-08[feature] various worker / request queue improvements (#995)Libravatar kim1
* greatly simplify httpclient request queuing Signed-off-by: kim <grufwub@gmail.com> * improved request queue mutex logic Signed-off-by: kim <grufwub@gmail.com> * use improved hashmap library Signed-off-by: kim <grufwub@gmail.com> * add warn logging when request queues are full Signed-off-by: kim <grufwub@gmail.com> * improve worker pool prefix var naming Signed-off-by: kim <grufwub@gmail.com> * improved worker pool error logging Signed-off-by: kim <grufwub@gmail.com> * move error message into separate field Signed-off-by: kim <grufwub@gmail.com> * remove old log statement Signed-off-by: kim <grufwub@gmail.com> * don't export worker message, it gets very spammy :') Signed-off-by: kim <grufwub@gmail.com> Signed-off-by: kim <grufwub@gmail.com>
2022-09-28[chore] update dependencies, bump to Go 1.19.1 (#826)Libravatar kim1
* update dependencies, bump Go version to 1.19 * bump test image Go version * update golangci-lint * update gotosocial-drone-build * sign * linting, go fmt * update swagger docs * update swagger docs * whitespace * update contributing.md * fuckin whoopsie doopsie * linterino, linteroni * fix followrequest test not starting processor * fix other api/client tests not starting processor * fix remaining tests where processor not started * bump go-runners version * don't check last-webfingered-at, processor may have updated this * update swagger command * update bun to latest version * fix embed to work the same as before with new bun Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-08-27[bugfix] Relax outgoing http request queue (#760)v0.4.0Libravatar tobi1
* add request queue trace logging * fix misleading wording * implement request slots per host per method * undo formatting change (?) * remove gratuitous trace logging * rename done -> release avoids confusion with ctx.Done
2022-07-18[chore] move dialer inside new (#715)Libravatar tobi1
* move dialer inside new, use default resolver * instantiate resolver
2022-05-26[performance] Don't retry/backoff invalid http requests that will never ↵v0.3.4Libravatar tobi1
succeed (#609) * add httpguts (ew) * add ValidateRequest err wrapping logic * don't retry on unrecoverable errors * i am very clever
2022-05-15[security] transport.Controller{} and transport.Transport{} security and ↵Libravatar kim1
performance improvements (#564) * cache transports in controller by privkey-generated pubkey, add retry logic to transport requests Signed-off-by: kim <grufwub@gmail.com> * update code comments, defer mutex unlocks Signed-off-by: kim <grufwub@gmail.com> * add count to 'performing request' log message Signed-off-by: kim <grufwub@gmail.com> * reduce repeated conversions of same url.URL object Signed-off-by: kim <grufwub@gmail.com> * move worker.Worker to concurrency subpackage, add WorkQueue type, limit transport http client use by WorkQueue Signed-off-by: kim <grufwub@gmail.com> * fix security advisories regarding max outgoing conns, max rsp body size - implemented by a new httpclient.Client{} that wraps an underlying client with a queue to limit connections, and limit reader wrapping a response body with a configured maximum size - update pub.HttpClient args passed around to be this new httpclient.Client{} Signed-off-by: kim <grufwub@gmail.com> * add httpclient tests, move ip validation to separate package + change mechanism Signed-off-by: kim <grufwub@gmail.com> * fix merge conflicts Signed-off-by: kim <grufwub@gmail.com> * use singular mutex in transport rather than separate signer mus Signed-off-by: kim <grufwub@gmail.com> * improved useragent string Signed-off-by: kim <grufwub@gmail.com> * add note regarding missing test Signed-off-by: kim <grufwub@gmail.com> * remove useragent field from transport (instead store in controller) Signed-off-by: kim <grufwub@gmail.com> * shutup linter Signed-off-by: kim <grufwub@gmail.com> * reset other signing headers on each loop iteration Signed-off-by: kim <grufwub@gmail.com> * respect request ctx during retry-backoff sleep period Signed-off-by: kim <grufwub@gmail.com> * use external pkg with docs explaining performance "hack" Signed-off-by: kim <grufwub@gmail.com> * use http package constants instead of string method literals Signed-off-by: kim <grufwub@gmail.com> * add license file headers Signed-off-by: kim <grufwub@gmail.com> * update code comment to match new func names Signed-off-by: kim <grufwub@gmail.com> * updates to user-agent string Signed-off-by: kim <grufwub@gmail.com> * update signed testrig models to fit with new transport logic (instead uses separate signer now) Signed-off-by: kim <grufwub@gmail.com> * fuck you linter Signed-off-by: kim <grufwub@gmail.com>