summaryrefslogtreecommitdiff
path: root/internal
AgeCommit message (Collapse)AuthorFiles
2023-09-07[chore] much improved paging package (#2182)Libravatar kim15
2023-09-07[feature] Support OTLP HTTP, drop Jaeger (#2184)Libravatar Daenney3
* [feature] Add http trace exporter, drop Jaeger Jaeger supports ingesting traces using the OpenTelemetry gRPC or HTTP methods. The Jaeger project has deprecated the old jaeger transport. * Add support for submitting traces over HTTP * Drop support for the old Jaeger protocol * Upgrade the trace libraries to v1.17 Fixes: #2176 Fixes: #2179
2023-09-05[bugfix] fix checks for deref the same status descendants / ascendants (#2181)Libravatar kim1
2023-09-04[feature] Don't trace non-existing routes (#2172)Libravatar Daenney1
c.FullPath() is the empty string if a request doesn't match any route on our mux. In those cases, there's no value in emitting a trace. The trace will be empty, containing no other information beyond the fact that we didn't match a route. Since Gin breaks off the processing early we don't need to trace this request as it won't do anything and consumes no further resources. The 404 will still be emitted by our logs and will be visible from a reverse proxy too.
2023-09-04[feature] Store admin actions in the db, prevent conflicting actions (#2167)Libravatar tobi20
2023-09-01[bugfix] move SQLite pragmas into connection string (#2171)Libravatar kim1
* move SQLite pragmas into connection string Signed-off-by: kim <grufwub@gmail.com> * use url.Values type for SQLite connection preferences Signed-off-by: kim <grufwub@gmail.com> * set SQLite URI prefs properly using _pragma query key Signed-off-by: kim <grufwub@gmail.com> * add notes on SQLite connection preferences Signed-off-by: kim <grufwub@gmail.com> * fix typo Signed-off-by: kim <grufwub@gmail.com> * add one extra line regarding connection pooling Signed-off-by: kim <grufwub@gmail.com> --------- Signed-off-by: kim <grufwub@gmail.com>
2023-08-31[bugfix] wrap bun.Tx to add our own error processing (#2169)Libravatar kim7
* wrap bun.Tx to add our own error processing Signed-off-by: kim <grufwub@gmail.com> * add compile-time check for updateRowError() compatibility with sql.Row, fix wrapTx() not being used properly Signed-off-by: kim <grufwub@gmail.com> --------- Signed-off-by: kim <grufwub@gmail.com>
2023-08-23[feature] list commands for both attachment and emojis (#2121)Libravatar rdelaage3
* [feature] list commands for both attachment and emojis * use fewer commands, provide `local-only` and `remote-only` as filters * envparsing --------- Co-authored-by: Romain de Laage <romain.delaage@rdelaage.ovh> Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-08-23[feature] Add rate limit exceptions option, use ISO8601 for rate limit reset ↵Libravatar tobi7
(#2151) * start updating rate limiting, add exceptions * tests, comments, tidying up * add rate limiting exceptions to example config * envparsing * nolint * apply kimbediff * add examples
2023-08-22[performance] Rework home timeline query to use cache more (#2148)Libravatar tobi2
2023-08-21[feature] Make log format configurable (#2130)Libravatar Daenney6
* [feature] Don't emit timestamp in log lines When running gotosocial with a service manager like systemd, or a container runtime, the associated log driver usually emits timestamps itself. In those cases, having the extra timestamp from our own log lines ends up being a bit noisy and when centrally ingesting logs is duplicate information. This introduces a configuration flag that allows disabling emitting the timestamp. It's only wired up for "daemonised" processes, meaning server and testrig. * [chore] Add docs for log-timestamp * [feature] Simplify timestamp handling Co-Authored-By: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com> * [chore] Less escaped double-quotes * [chore] Fix help string --------- Co-authored-by: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>
2023-08-21[performance] Tweak media attachment cleanup; replace stale index (#2143)Libravatar tobi5
2023-08-20[feature/bugfix] Probe S3 storage for CSP uri, add config flag for extra ↵Libravatar tobi8
URIs (#2134) * [feature/bugfix] Probe S3 storage for CSP uri, add config flag for extra URIs * env parsing tests, my coy mistress
2023-08-19[feature] Instance rules (#2125)Libravatar f0x5237
* init instance rules database model, admin api * expose instance rules in public instance api * public /api/v1/instance/rules route * GET ruleById * createRule route * createRule auth check * updateRule * deleteRule * list rules on about page * ruleGet auth * add about page ids for anchors * process and store adding violated rules to reports * admin api models for instance rules * instance rule edit frontend * change rule inputs to textareas * database fixes after rebase (#2124) * remove unused imports * fix db migration column name * fix tests * fix more tests * fix postgres error with wrongly used Ident * add some tests, fiddle with rule model a bit, fix postgres migration * swagger docs --------- Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-08-17[bugfix] fix double firing bun.DB query hooks (#2124)Libravatar kim36
* improve bun.DB wrapping readability + comments, fix double-firing query hooks * fix incorrect code comment placement * fix linter issues * Update internal/db/basic.go * do as the linter commmands ... --------- Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: Daenney <daenney@users.noreply.github.com>
2023-08-15[chore] ensure worker contexts have request ID (#2120)Libravatar kim4
2023-08-14[bugfix] fix inconsistent calculated cache sizes (#2115)v0.11.0Libravatar kim3
* use calculated exampleTime instead of `time.Now()` to ensure no locale data, retweak cache ratios * update envparsing test * update default cache memory to 100MiB * fix envparsing with latest cache target default --------- Signed-off-by: kim <grufwub@gmail.com>
2023-08-14[fix] Update CSP header for blob images (upload preview) and dev livereload ↵v0.11.0-rc3Libravatar f0x522
(#2109) * update CSP header for blob images (upload preview) and dev livereload websocket * update csp for s3, update csp tests
2023-08-12[bugfix] CSP policy fixes for S3/object storage (#2104)Libravatar Daenney2
* [bugfix] CSP policy fixes for S3 in non-proxied mode * It should be img-src * In both img-src and media-src we still need to include 'self'
2023-08-11[bugfix] Add s3 endpoint as image-src and media-src for CSP (#2103)v0.11.0-rc2Libravatar tobi2
* [bugfix] Add s3 endpoint as image-src and media-src for CSP * use https if secure * reorder comment
2023-08-11[bugfix] Suppress 'errNoEntries' warnings from InboxForwarding function call ↵v0.11.0-rc1Libravatar tobi1
(#2102)
2023-08-11[bugfix] Fix using wrong key for clientID during oauth callback (#2101)Libravatar tobi1
2023-08-11[bugfix] Use custom bluemonday policy to disallow inline img tags (#2100)Libravatar tobi17
2023-08-11[feature] Set Content-Security-Policy header (#2095)Libravatar Daenney1
This adds the CSP header with a policy of only loading from the same domain. We don't make use of external media, CSS, JS, fonts, so we don't ever need external data loaded in our context. When building a DEBUG build, the policy gets extended to include localhost:*, i.e localhost on any port. This keeps the live-reloading flow for JS development working. localhost and 127.0.0.1 are considered to be the same so mixing and matching those doesn't result in a CSP violation.
2023-08-11[bugfix] Populate followReq before accessing targetaccount pointer (#2099)Libravatar tobi1
2023-08-10[bugfix] Use length in runes when trimming for RSS (#2094)Libravatar tobi2
2023-08-10[chore] Add test to ensure `show_reblogs` on follow works as expected (#2093)Libravatar tobi1
2023-08-10[performance] remove last of relational queries to instead rely on caches ↵Libravatar kim18
(#2091)
2023-08-09[feature] List replies policy, refactor async workers (#2087)Libravatar tobi46
* Add/update some DB functions. * move async workers into subprocessor * rename FromFederator -> FromFediAPI * update home timeline check to include check for current status first before moving to parent status * change streamMap to pointer to mollify linter * update followtoas func signature * fix merge * remove errant debug log * don't use separate errs.Combine() check to wrap errs * wrap parts of workers functionality in sub-structs * populate report using new db funcs * embed federator (tiny bit tidier) * flesh out error msg, add continue(!) * fix other error messages to be more specific * better, nicer * give parseURI util function a bit more util * missing headers * use pointers for subprocessors
2023-08-09[bugfix] Fix incorrect per-loop variable capture (#2092)Libravatar Daenney2
These should be per iteration, not per loop. This was caught by running a build with the loopvar experiment: go build -gcflags=-d=loopvar=2.
2023-08-08[bugfix] only set content-length AFTER rewinding body bytes (#2086)Libravatar kim1
2023-08-08[bugfix] don't accept unrelated statuses (#2078)Libravatar kim4
Co-authored-by: Daenney <daenney@users.noreply.github.com> Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-08-08[chore] Update robots.txt, give chatgpt the middle finger (#2085)Libravatar tobi1
2023-08-07[chore] Use generic pointer function (#2080)Libravatar Daenney21
This replaces the different $TypePtr functions with a generic implementation.
2023-08-07[feature] Allow full BCP 47 in language inputs (#2067)Libravatar Vyr Cossont9
* Allow full BCP 47 in language inputs Fixes #2066 * Fuse validation and normalization for languages * Remove outdated comment line * Move post language canonicalization test
2023-08-06[chore] Remove go-playground/validator (#2069)Libravatar Daenney50
* [chore] Remove go-playground/validator It turns out we're not actually using the validator code. This is a remnant from when we intended to use it, but the presence of it and its struct tags creates the illusion we're validating a lot of things we're not. It resulted in some confusion when we were trying to figure out language valdiation. Remove all this code, so that only the validation functions from the validate package we actually use remain. I'm not touching the struct tags in the migrations in order to avoid things potentially thinking migrations need to be re-run. * [chore] Bring back a struct tag on api The validate on internal/api is Gin doing form validation, not the validator from go-playground/validator.
2023-08-06[bugfix] update cache library with nil ptr fix (#2070)Libravatar kim1
Signed-off-by: kim <grufwub@gmail.com>
2023-08-04[performance] add caching of status fave, boost of, in reply to ID lists (#2060)Libravatar kim19
2023-08-03[feature] simpler cache size configuration (#2051)Libravatar kim12
* add automatic cache max size generation based on ratios of a singular fixed memory target Signed-off-by: kim <grufwub@gmail.com> * remove now-unused cache max-size config variables Signed-off-by: kim <grufwub@gmail.com> * slight ratio tweak Signed-off-by: kim <grufwub@gmail.com> * remove unused visibility config var Signed-off-by: kim <grufwub@gmail.com> * add secret little ratio config trick Signed-off-by: kim <grufwub@gmail.com> * fixed a word Signed-off-by: kim <grufwub@gmail.com> * update cache library to remove use of TTL in result caches + slice cache Signed-off-by: kim <grufwub@gmail.com> * update other cache usages to use correct interface Signed-off-by: kim <grufwub@gmail.com> * update example config to explain the cache memory target Signed-off-by: kim <grufwub@gmail.com> * update env parsing test with new config values Signed-off-by: kim <grufwub@gmail.com> * do some ratio twiddling Signed-off-by: kim <grufwub@gmail.com> * add missing header * update envparsing with latest defaults Signed-off-by: kim <grufwub@gmail.com> * update size calculations to take into account result cache, simple cache and extra map overheads Signed-off-by: kim <grufwub@gmail.com> * tweak the ratios some more Signed-off-by: kim <grufwub@gmail.com> * more nan rampaging Signed-off-by: kim <grufwub@gmail.com> * fix envparsing script Signed-off-by: kim <grufwub@gmail.com> * update cache library, add sweep function to keep caches trim Signed-off-by: kim <grufwub@gmail.com> * sweep caches once a minute Signed-off-by: kim <grufwub@gmail.com> * add a regular job to sweep caches and keep under 80% utilisation Signed-off-by: kim <grufwub@gmail.com> * remove dead code Signed-off-by: kim <grufwub@gmail.com> * add new size library used to libraries section of readme Signed-off-by: kim <grufwub@gmail.com> * add better explanations for the mem-ratio numbers Signed-off-by: kim <grufwub@gmail.com> * update go-cache Signed-off-by: kim <grufwub@gmail.com> * library version bump Signed-off-by: kim <grufwub@gmail.com> * update cache.result{} size model estimation Signed-off-by: kim <grufwub@gmail.com> --------- Signed-off-by: kim <grufwub@gmail.com>
2023-08-02[bugfix] Rework MultiError to wrap + unwrap errors properly (#2057)Libravatar tobi23
* rework multierror a bit * test multierror
2023-08-02[bugfix] fix slow accounts / statuses using emojis lookups (#2056)Libravatar kim3
* update DeleteEmoji to use faster relational tables for status / account finding Signed-off-by: kim <grufwub@gmail.com> * update Get{Accounts,Statuses}UsingEmoji() to also use relational tables Signed-off-by: kim <grufwub@gmail.com> * remove the now unneeded tags relation from newStatusQ() Signed-off-by: kim <grufwub@gmail.com> * fix table names Signed-off-by: kim <grufwub@gmail.com> * fix account and status selects using emojis Signed-off-by: kim <grufwub@gmail.com> --------- Signed-off-by: kim <grufwub@gmail.com>
2023-08-02[bugfix] handle HEAD requests more elegantly (#2055)Libravatar kim1
Signed-off-by: kim <grufwub@gmail.com>
2023-08-02[bugfix] Allow instance accounts to be shown in search results in certain ↵Libravatar tobi6
circumstances (#2053)
2023-08-01[feature] Allow users to skip http client tls verification for testing ↵Libravatar tobi5
purposes (with appropriately loud warnings) (#2052)
2023-07-31[feature] Hashtag federation (in/out), hashtag client API endpoints (#2032)Libravatar tobi61
* update go-fed * do the things * remove unused columns from tags * update to latest lingo from main * further tag shenanigans * serve stub page at tag endpoint * we did it lads * tests, oh tests, ohhh tests, oh tests (doo doo doo doo) * swagger docs * document hashtag usage + federation * instanceGet * don't bother parsing tag href * rename whereStartsWith -> whereStartsLike * remove GetOrCreateTag * dont cache status tag timelineability
2023-07-31[performance] cache follow, follow request and block ID lists (#2027)Libravatar kim24
2023-07-29[feature] Implement markers API (#1989)Libravatar Vyr Cossont25
* Implement markers API Fixes #1856 * Correct import grouping in markers files * Regenerate Swagger for markers API * Shorten names for readability * Cache markers for 6 hours * Update DB ref * Update envparsing.sh
2023-07-27[chore] fix merge issue (#2029)Libravatar tobi1
2023-07-27[feature] Support setting private notes on accounts (#1982)Libravatar Vyr Cossont15
* Support setting private notes on accounts * Reformat comment whitespace * Add missing license headers * Use apiutil.ParseID * Rename Note model and cache to AccountNote * Update golden cache config in test/envparsing.sh * Rename gtsmodel/note.go to gtsmodel/accountnote.go * Update AccountNote uniqueness constraint name Now has same prefix as other indexes on this table. --------- Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
2023-07-25[performance] retry db queries on busy errors (#2025)Libravatar kim52
* catch SQLITE_BUSY errors, wrap bun.DB to use our own busy retrier, remove unnecessary db.Error type Signed-off-by: kim <grufwub@gmail.com> * remove dead code Signed-off-by: kim <grufwub@gmail.com> * remove more dead code, add missing error arguments Signed-off-by: kim <grufwub@gmail.com> * update sqlite to use maxOpenConns() Signed-off-by: kim <grufwub@gmail.com> * add uncommitted changes Signed-off-by: kim <grufwub@gmail.com> * use direct calls-through for the ConnIface to make sure we don't double query hook Signed-off-by: kim <grufwub@gmail.com> * expose underlying bun.DB better Signed-off-by: kim <grufwub@gmail.com> * retry on the correct busy error Signed-off-by: kim <grufwub@gmail.com> * use longer possible maxRetries for db retry-backoff Signed-off-by: kim <grufwub@gmail.com> * remove the note regarding max-open-conns only applying to postgres Signed-off-by: kim <grufwub@gmail.com> * improved code commenting Signed-off-by: kim <grufwub@gmail.com> * remove unnecessary infof call (just use info) Signed-off-by: kim <grufwub@gmail.com> * rename DBConn to WrappedDB to better follow sql package name conventions Signed-off-by: kim <grufwub@gmail.com> * update test error string checks Signed-off-by: kim <grufwub@gmail.com> * shush linter Signed-off-by: kim <grufwub@gmail.com> * update backoff logic to be more transparent Signed-off-by: kim <grufwub@gmail.com> --------- Signed-off-by: kim <grufwub@gmail.com>
generated by cgit v1.2.3 (git 2.46.0) at 2025-08-03 13:52:20 +0000