summaryrefslogtreecommitdiff
path: root/internal/db/bundb
AgeCommit message (Collapse)AuthorFiles
2023-03-12[chore] Improve copyright header handling (#1608)Libravatar Daenney95
* [chore] Remove years from all license headers Years or year ranges aren't required in license headers. Many projects have removed them in recent years and it avoids a bit of yearly toil. In many cases our copyright claim was also a bit dodgy since we added the 2021-2023 header to files created after 2021 but you can't claim copyright into the past that way. * [chore] Add license header check This ensures a license header is always added to any new file. This avoids maintainers/reviewers needing to remember to check for and ask for it in case a contribution doesn't include it. * [chore] Add missing license headers * [chore] Further updates to license header * Use the more common // indentend comment format * Remove the hack we had for the linter now that we use the // format * Add SPDX license identifier
2023-03-03[performance] cache media attachments (#1525)Libravatar kim2
* replace concurrency worker pools with base models in State.Workers, update code and tests accordingly * add media attachment caching, slightly tweak default cache config * further tweak default cache config values * replace other media attachment db calls to go through cache * update envparsing test * fix delete media attachment sql * fix media sql query * invalidate cached media entries during status create / update * fix envparsing test * fix typo in panic log message... * add 'updated_at' column during UpdateAttachment * remove unused func --------- Signed-off-by: kim <grufwub@gmail.com>
2023-03-02[bugfix] add ON CONFLICT statements to status updates (#1580)Libravatar tobi2
2023-03-02[feature] Advertise rich text formats, support content_type field (#1370)Libravatar Vyr Cossont2
* Advertise rich text formats, support content_type field * Update JSON in instance patch tests * Replace format with content_type everywhere * update migration to work with both pg and sqlite * regenerate swagger docs * update instance serialization + tests * fix up * learn to code tobi please, i'm begging you --------- Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-03-01[chore] move client/federator workerpools to Workers{} (#1575)Libravatar kim2
* replace concurrency worker pools with base models in State.Workers, update code and tests accordingly * improve code comment * change back testrig default log level * un-comment-out TestAnnounceTwice() and fix --------- Signed-off-by: kim <grufwub@gmail.com> Reviewed-by: tobi
2023-03-01[feature] Federate pinned posts (aka `featuredCollection`) in and out (#1560)Libravatar tobi2
* start fiddling * the ol' fiddle + update * start working on fetching statuses * poopy doopy doo where r u uwu * further adventures in featuring statuses * finishing up * fmt * simply status unpin loop * move empty featured check back to caller function * remove unnecessary log.WithContext calls * remove unnecessary IsIRI() checks * add explanatory comment about status URIs * change log level to error * better test names
2023-02-25[feature] Client API endpoints + v. basic web view for pinned posts (#1547)Libravatar tobi4
* implement status pin client api + web handler * make test names + comments more descriptive * don't use separate table for status pins * remove unused add + remove checking * tidy up + add some more tests
2023-02-18[bugfix] In Postgres, drop shortcodedomain constraint before creating new ↵Libravatar tobi1
emoji table (#1528)
2023-02-17[feature] Add a request ID and include it in logs (#1476)Libravatar Daenney13
This adds a lightweight form of tracing to GTS. Each incoming request is assigned a Request ID which we then pass on and log in all our log lines. Any function that gets called downstream from an HTTP handler should now emit a requestID=value pair whenever it logs something. Co-authored-by: kim <grufwub@gmail.com>
2023-02-13[bugfix] Fix up `error getting account avatar/header` errors, other small ↵Libravatar tobi3
fixes (#1496) * start fiddling with media + account queries a little * initialize state when pruning * allow for unsetting remote media make sure to wait til media loaded fix silly tiny bug * move comment a bit for readability * slight reformat of fetchRemoteAccount{Avatar,Header} * fix issue after rebase * slightly neaten up logic of avatar/header media handling * remove log prefix (callername log field handles this) --------- Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: kim <grufwub@gmail.com>
2023-02-11[chore/performance] Update media prune logic, add extra CLI command (#1474)v0.7.0-rc2Libravatar tobi2
* start updating media prune stuff a wee bit * continue prune / uncache work * more tidying + consistency stuff * add prune CLI command * docs * arg
2023-02-03[chore] reformat GetAccount() functionality, support updating accounts based ↵Libravatar kim5
on last_fetch (#1411) * reformat GetAccount() functionality, and add UpdateAccount() function. * use fetched_at instead of last_webfingered_at * catch local "not found" errors. small formatting / error string changes * remove now unused error type * return nil when wrapping nil error * update expected error messages * return correct url for foss satan webfinger * add AP model for Some_User * normalize local domain * return notretrievable where appropriate * expose NewErrNotRetrievable * ensure webfinger for new accounts searched by uri * update local account short circuit * allow enrich to fail for already-known accounts * remove unused LastWebfingeredAt * expose test maps on mock http client * update Update test * reformat GetAccount() functionality, and add UpdateAccount() function. * use fetched_at instead of last_webfingered_at * catch local "not found" errors. small formatting / error string changes * remove nil error checks (we shouldn't be passing nil errors to newError() initializers) * remove mutex unlock on transport init fail (it hasn't yet been locked!) * woops add back the error wrapping to use ErrNotRetrievable * caches were never being started... :see_no_evil: --------- Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-02-03[chore] Text formatting overhaul (#1406)Libravatar Autumn!1
* Implement goldmark debug print for hashtags and mentions * Minify HTML in FromPlain * Convert plaintext status parser to goldmark * Move mention/tag/emoji finding logic into formatter * Combine mention and hashtag boundary characters * Normalize unicode when rendering hashtags
2023-02-01[chore/performance] use only 1 sqlite db connection regardless of multiplier ↵Libravatar tobi1
(#1408)
2023-01-31[chore] Use 'immediate' lock for sqlite transactions (#1404)Libravatar tobi1
* [chore] Use 'immediate' lock for sqlite transactions * allow 1 connection regardless of cpu amount
2023-01-26[chore] set max open / idle conns + conn max lifetime for both postgres and ↵Libravatar tobi1
sqlite (#1369) * [chore] set max open / idle conns + conn max lifetime for both postgres and sqlite * reduce cache size default to 8MiB, reduce connections to 2 * cpu * introduce max open conns multiplier, tune sqlite and pg separately * go fmt
2023-01-23[chore] remove funky duplicate attachment in testrig (#1379)Libravatar tobi1
* [chore] remove funky duplicate attachment in testrig * whoops
2023-01-23[feature] Implement `/api/v1/reports` endpoints on client API (#1330)Libravatar tobi2
* start adding report client api * route + test reports get * start report create endpoint * you can create reports now babyy * stub account report processor * add single reportGet endpoint * fix test * add more filtering params to /api/v1/reports GET * update swagger * use marshalIndent in tests * add + test missing Link info
2023-01-17[feature] Tune sqlite pragmas (#1349)Libravatar tobi1
* sqlite pragma tuning * use formatuint * add sqlite busy timeout * fix incorrect cache size format * update envparsing test * add sqlite tuning flags to cli * set sqlite timeout to 30s default
2023-01-10[feature] Implement Report database model and utility functions (#1310)Libravatar tobi7
* implement report database model * implement report cache + config changes * implement report database functions * report uri / regex functions * update envparsing test * remove unnecessary uri index * remove unused function + cache lookup * process error when storing report
2023-01-05[chore] Update/add license headers for 2023 (#1304)Libravatar tobi90
2023-01-05[chore] Fix emoji notnull constraint on initial gtsmodel (#1303)Libravatar tobi1
2022-12-22[feature] For video attachments, store + return fps, bitrate, duration (#1282)Libravatar tobi1
* start messing about with different mp4 metadata extraction * heyyooo it works * add test cow * move useful multierror to gtserror package * error out if video doesn't seem to be a real mp4 * test parsing mkv in disguise as mp4 * tidy up error handling * remove extraneous line * update framerate formatting * use float32 for aspect * fixy mctesterson
2022-12-14[feature] domain block wildcarding (#1178)Libravatar kim2
* for domain block lookups, lookup along subdomain parts Signed-off-by: kim <grufwub@gmail.com> * only lookup up to a max of 5 domain parts to prevent DOS, limit inserted domains to max of 5 subdomains Signed-off-by: kim <grufwub@gmail.com> * add test for domain block wildcarding Signed-off-by: kim <grufwub@gmail.com> * check cached status first, increase cached domain time Signed-off-by: kim <grufwub@gmail.com> * fix domain wildcard part building logic Signed-off-by: kim <grufwub@gmail.com> * create separate domain.BlockCache{} type to hold all domain blocks in memory Signed-off-by: kim <grufwub@gmail.com> * remove unused variable Signed-off-by: kim <grufwub@gmail.com> * add docs and test to domain block cache, check for domain == host in domain block getter funcs Signed-off-by: kim <grufwub@gmail.com> * add license text Signed-off-by: kim <grufwub@gmail.com> * check order in which we check primary cache Signed-off-by: kim <grufwub@gmail.com> * add better documentation of how domain block checking is performed Signed-off-by: kim <grufwub@gmail.com> * change Signed-off-by: kim <grufwub@gmail.com> Signed-off-by: kim <grufwub@gmail.com>
2022-12-13[bugfix] fix unordered favorites (#1245)Libravatar Christoph Voigt1
* [bugfix] fix unordered favorites * add test for favouritesget * add license to new test files
2022-12-12[feature] allow uncaching of other media types (#1234)Libravatar kim2
* simplify pruneRemote, remove unncecessary media trace logging, update RemoteOlderThan() to include headers/avis Signed-off-by: kim <grufwub@gmail.com> * cleanup pruneallmeta, add remote header to pruneremote tests Signed-off-by: kim <grufwub@gmail.com> * fix olderthan duration additions Signed-off-by: kim <grufwub@gmail.com> * fix broken test now that test model header changed Signed-off-by: kim <grufwub@gmail.com> * instead use new remote test account for new header model Signed-off-by: kim <grufwub@gmail.com> * use newer generated ULID for remote_account_3 to ensure it is sorted last Signed-off-by: kim <grufwub@gmail.com> * reorganize serialized keys to match expected test account model order Signed-off-by: kim <grufwub@gmail.com> Signed-off-by: kim <grufwub@gmail.com>
2022-12-09[bugfix] fix unordered favorites (#1236)Libravatar Christoph Voigt1
2022-12-09[feature] Allow users to create + delete bookbarks, and view bookmarked ↵Libravatar Matthew Phillips2
statuses (#1168) * Implement Bookmarks * Update based on review comments * Update swagger doc * Fix argument passing to status.Bookmark * Update changed test * Updates based on latest PR review
2022-12-08[chore] move caches to a separate State{} structure (#1078)Libravatar kim13
* move caches to a separate State{} structure Signed-off-by: kim <grufwub@gmail.com> * fix call to log.Panic not using formatted call Signed-off-by: kim <grufwub@gmail.com> * move caches to use interfaces, to make switchouts easier in future Signed-off-by: kim <grufwub@gmail.com> * fix rebase issue Signed-off-by: kim <grufwub@gmail.com> * improve code comment Signed-off-by: kim <grufwub@gmail.com> * fix further issues after rebase Signed-off-by: kim <grufwub@gmail.com> * heh Signed-off-by: kim <grufwub@gmail.com> * add missing license text Signed-off-by: kim <grufwub@gmail.com> Signed-off-by: kim <grufwub@gmail.com>
2022-12-06[feature] overhaul the oidc system (#961)Libravatar Dominik Süß3
* [feature] overhaul the oidc system this allows for more flexible username handling and prevents account takeover using old email addresses * [feature] add migration path for old OIDC users * [feature] nicer error reporting for users * [docs] document the new OIDC flow * [fix] return early on oidc error * [docs]: add comments on the finalization logic
2022-12-02[bugfix] Fix `admin account confirm` on pre-confirmed account setting email ↵Libravatar tobi1
address to empty string (#1203)
2022-12-02[bugfix] Add missing continues in emoji get funcs (#1200)Libravatar tobi1
2022-12-01[bugfix] Use case-insensitive selects when getting remote accounts by ↵v0.6.0-rc2Libravatar tobi2
username/domain (#1191) * [bugfix] Case-insensitive account selection * don't lowercase cache key
2022-11-30[bugfix] Don't call `strings.ToLower()` on usernames when selecting account ↵Libravatar tobi1
by domain+username (#1190) * don't lowercase account username when doing a select * test getting remote user with uppercase username
2022-11-30[bugfix] Avoid accidentally marking changed emojis as orphaned + pruning ↵Libravatar tobi1
them (#1188) * add predictable instance account to tests, use it in emoji urls + paths * use static image url to select emojis when pruning orphaned
2022-11-27fix missing lookup cache key for invalid domain block (#1158)Libravatar kim1
2022-11-22[feature] Prune timelines once per hour to plug memory leak (#1117)Libravatar tobi1
* export highest/lowest ULIDs as proper const * add stop + start to timeline manager, other small fixes * unexport unused interface funcs + tidy up * add LastGot func * add timeline Prune function * test prune * update lastGot
2022-11-20[chore] improve errors returned fetching account/statues from DB (#1084)Libravatar kim2
Signed-off-by: kim <grufwub@gmail.com> Signed-off-by: kim <grufwub@gmail.com>
2022-11-20[performance] add account block DB cache and remove block query joins (#1085)Libravatar kim3
* add account block DB cache and remove reliance on relational joins * actually include cache key arguments... * add a PutBlock() method which also updates the block cache, update tests accordingly * use `PutBlock` instead of `Put(ctx, block)` * add + use functions for deleting + invalidating blocks Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-11-18[performance] replace account emojis relational query with separate calls to ↵Libravatar kim2
emojiDB to rely on cache (#1074) Signed-off-by: kim <grufwub@gmail.com> Signed-off-by: kim <grufwub@gmail.com>
2022-11-18[performance] replace status query relationals with separate calls in order ↵Libravatar kim2
to rely on caches more (#1073) Signed-off-by: kim <grufwub@gmail.com> Signed-off-by: kim <grufwub@gmail.com>
2022-11-18[performance] don't use relations to select notification structs, use caches ↵Libravatar tobi2
instead (#1072)
2022-11-16[chore] reversion: use specific columns for updating user again (#1059)Libravatar tobi1
2022-11-15[chore] update database caching library (#1040)Libravatar kim14
* convert most of the caches to use result.Cache{} * add caching of emojis * fix issues causing failing tests * update go-cache/v2 instances with v3 * fix getnotification * add a note about the left-in StatusCreate comment * update EmojiCategory db access to use new result.Cache{} * fix possible panic in getstatusparents * further proof that kim is not stinky
2022-11-14[feature] Allow newly uploaded emojis to be placed in categories (#939)Libravatar tobi4
* [feature] Add emoji categories GET Serialize emojis in appropriate categories; make it possible to get categories via the admin API * [feature] Create (or use existing) category for new emoji uploads * fix lint issue * update misleading line in swagger docs
2022-11-14[feature] add instance-expose-public-timeline flag (#1039)Libravatar Rob Sargant2
* Add instance-expose-public-timeline flag Adds a config flag that allows unauthenticated access to /api/v1/timelines/public. Defaults to false to replicate existing behaviour. * Update structure following review * Add comment * Fix linting
2022-11-13[chore] bump go-cache to v3.1.7 to fix possible issues with zero value keys ↵Libravatar kim1
(#1038) Signed-off-by: kim <grufwub@gmail.com> Signed-off-by: kim <grufwub@gmail.com>
2022-11-11[feature] Read + Write tombstones for deleted Actors (#1005)Libravatar tobi3
* [feature] Read + Write tombstones for deleted Actors * copyTombstone * update to use resultcache instead of old ttl cache Signed-off-by: kim <grufwub@gmail.com> * update go-cache library to fix result cache capacity / ordering bugs Signed-off-by: kim <grufwub@gmail.com> * bump go-cache/v3 to v3.1.6 to fix bugs Signed-off-by: kim <grufwub@gmail.com> * switch on status code * better explain ErrGone reasoning Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: kim <grufwub@gmail.com>
2022-11-05[chore] bump gruf/go-store to v2 (#953)Libravatar tobi1
* [chore] bump gruf/go-store to v2 * no more boobs
2022-11-01[bugfix] create admin_account_actions table in tx (#940)Libravatar Terin Stock1
The migration that adds the `admin_account_actions` table did so at the same time as adding indexes onto the new table. This code was ran inside a `RunInTx` function, but the table creation did not use the transaction reference, while the creation of the indexes did. This could cause a race between the table and index creations, depending on the scheduling order. If the table creation did not win the race, then the migration would fail. This changeset corrects the table creation to also be done inside the same transaction as the index creation. Signed-off-by: Terin Stock <terinjokes@gmail.com> Signed-off-by: Terin Stock <terinjokes@gmail.com>