summaryrefslogtreecommitdiff
path: root/internal/db
AgeCommit message (Collapse)AuthorFiles
2024-09-28[bugfix] visibility after implicit approval not getting invalidated (#3370)Libravatar kim2
* replicate issue * update go-structr to v0.8.10 with internal linked-list fix, small tweaks to caching of interaction requests * remove debug function --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-09-23[chore] local instance count query caching, improved status context endpoint ↵Libravatar kim1
logging, don't log ErrHideStatus when timelining (#3330) * ensure that errors checking status visibility / converting aren't dropped * add some more context to error messages * include calling function name in log entries * don't error on timelining hidden status * further code to ignore statusfilter.ErrHideStatus type errors * remove unused error type * add local instance status / domain / user counts * add checks for localhost * rename from InstanceCounts to LocalInstance * improved code comment
2024-09-20[chore] ensure consistent caller name fetching regardless of compiler ↵Libravatar kim1
inlining (#3323) * move logging levels into log package itself * ensure inconsistent inlining doesn't mess with log calling function name * remove unused global variable * fix log level
2024-09-16[performance] cache more database calls, reduce required database calls ↵Libravatar kim36
overall (#3290) * improvements to caching for lists and relationship to accounts / follows * fix nil panic in AddToList() * ensure list related caches are correctly invalidated * ensure returned ID lists are ordered correctly * bump go-structr to v0.8.9 (returns early if zero uncached keys to be loaded) * remove zero checks in uncached key load functions (go-structr now handles this) * fix issues after rebase on upstream/main * update the expected return order of CSV exports (since list entries are now down by entry creation date) * rename some funcs, allow deleting list entries for multiple follow IDs at a time, fix up more tests * use returning statements on delete to get cache invalidation info * fixes to recent database delete changes * fix broken list entries delete sql * remove unused db function * update remainder of delete functions to behave in similar way, some other small tweaks * fix delete user sql, allow returning on err no entries * uncomment + fix list database tests * update remaining list tests * update envparsing test * add comments to each specific key being invalidated * add more cache invalidation explanatory comments * whoops; actually delete poll votes from database in the DeletePollByID() func * remove added but-commented-out field * improved comment regarding paging being disabled * make cache invalidation comments match what's actually happening * fix up delete query comments to match what is happening * rename function to read a bit better * don't use ErrNoEntries on delete when not needed (it's only needed for a RETURNING call) * update function name in test * move list exclusivity check to AFTER eligibility check. use log.Panic() instead of panic() * use the poll_id column in poll_votes for selecting votes in poll ID * fix function name
2024-09-14[chore] Update go-sqlite3 to 0.18.3 (#3295)Libravatar Daenney1
* [chore] Update go-sqlite3 to 0.18.3 * [chore] Fix getting the sqlite3.Conn
2024-09-11[bugfix] Update home timeline query to ignore exclusive list entries (#3289)Libravatar tobi2
* [bugfix] Update home timeline query to ignore exclusive list entries * a
2024-09-10[feature] Process `Reject` of interaction via fedi API, put rejected ↵Libravatar tobi5
statuses in the "sin bin" 😈 (#3271) * [feature] Process `Reject` of interaction via fedi API, put rejected statuses in the "sin bin" * update test * move nil check back to `rejectStatusIRI`
2024-09-09[feature] Implement exclusive lists (#3280)Libravatar Vyr Cossont1
Fixes #2616
2024-09-09[feature/frontend] Add options to include Unlisted posts or hide all posts ↵Libravatar tobi3
(#3272) * [feature/frontend] Add options to include Unlisted posts or hide all posts * finish up * swagger * move invalidate call into bundb package, avoid invalidating if not necessary * rename show_web_statuses => web_visibility * don't use ptr for webvisibility * last bits
2024-09-08[bugfix] Fix temp table deletion causing runaway allocations (#3278)Libravatar tobi2
* [bugfix] Fix temp table deletion causing runaway allocations * move some vars around * small fixes * rely on conn max age to recycle temp tables * fackin' ell m8
2024-09-02[chore/performance] Further reduce nil uncached queries (#3267)Libravatar tobi8
* [chore/performance] Further reduce nil uncached queries * more checks
2024-09-02[chore] Don't try to select zero uncached filters (#3266)Libravatar tobi1
2024-09-02[chore/performance] Avoid unnecessary "uncached" queries (#3265)Libravatar tobi17
* [chore/performance] Avoid unnecessary "uncached" queries * go fmt
2024-08-24[feature] Interaction requests client api + settings panel (#3215)Libravatar tobi10
* [feature] Interaction requests client api + settings panel * test accept / reject * fmt * don't pin rejected interaction * use single db model for interaction accept, reject, and request * swaggor * env sharting * append errors * remove ErrNoEntries checks * change intReqID to reqID * rename "pend" to "request" * markIntsPending -> mark interactionsPending * use log instead of returning error when rejecting interaction * empty migration * jolly renaming * make interactionURI unique again * swag grr * remove unnecessary locks * invalidate as last step
2024-08-22[feature] Use `local_only` field, deprecate `federated` field (#3222)Libravatar tobi1
* [feature] Use `local_only` field, deprecate `federated` field * use `deprecated` comment for form.Federated * nolint
2024-08-08[feature] Add `db-postgres-connection-string` option (#3178)Libravatar John Winston1
* handle db-url * lint and add doc * add more doc * fix config test * return error * change name from db-url to db-postgres-connection-string
2024-08-02[chore] move PopulateAccountStats() nil check often performed into function ↵Libravatar kim2
itself (#3158) * move PopulateAccountStats() nil check often performed into function itself * fix test to take in mind we don't repopulate account stats if not-nil
2024-07-31[feature] Allow users to export data via the settings panel (#3140)Libravatar tobi5
* [feature] Allow users to export data via the settings panel * rename/move some stuff
2024-07-30[feature] persist worker queues to db (#3042)Libravatar kim5
* persist queued worker tasks to database on shutdown, fill worker queues from database on startup * ensure the tasks are sorted by creation time before pushing them * add migration to insert WorkerTask{} into database, add test for worker task persistence * add test for recovering worker queues from database * quick tweak * whoops we ended up with double cleaner job scheduling * insert each task separately, because bun is throwing some reflection error?? * add specific checking of cancelled worker contexts * add http request signing to deliveries recovered from database * add test for outgoing public key ID being correctly set on delivery * replace select with Queue.PopCtx() * get rid of loop now we don't use it * remove field now we don't use it * ensure that signing func is set * header values weren't being copied over :facepalm: * use ptr for httpclient.Request in delivery * move worker queue filling to later in server init process * fix rebase issues * make logging less shouty * use slices.Delete() instead of copying / reslicing * have database return tasks in ascending order instead of sorting them * add a 1 minute timeout to persisting worker queues
2024-07-30[chore] replace UniqueStrings with Deduplicate (#3154)Libravatar tobi1
2024-07-29[feature] Implement following hashtags (#3141)Libravatar Vyr Cossont3
* Implement followed tags API * Insert statuses with followed tags into home timelines * Test following and unfollowing tags * Correct Swagger path params * Trim conversation caches * Migration for followed_tags table * Followed tag caches and DB implementation * Lint and tests * Add missing tag info endpoint, reorganize tag API * Unwrap boosts when timelining based on tags * Apply visibility filters to tag followers * Address review comments
2024-07-26[feature] Federate interaction policies + Accepts; enforce policies (#3138)Libravatar tobi1
* [feature] Federate interaction policies + Accepts; enforce policies * use Acceptable type * fix index * remove appendIRIStrs * add GetAccept federatingdb function * lock on object IRI
2024-07-24[chore] renames the `GTS` caches to `DB` caches (#3127)Libravatar kim32
* renames the `GTS` caches to `DB` caches, as it better references what they are * change remaining Caches.GTS uses to Caches.DB
2024-07-23[chore] Add some log lines to recent migrations warning not to interrupt (#3134)Libravatar tobi3
* [chore] Add some log lines to recent migrations warning not to interrupt * arse
2024-07-23Remove content and related fields from boosts (#3131)Libravatar Vyr Cossont1
These duplicate the content of the target and aren't necessary for anything. - Stops copying some fields from target when boosting or processing a remote boost - Adds a migration to null out existing duplicate data - Updates tests
2024-07-23[feature] Conversations API (#3013)Libravatar Vyr Cossont12
* Implement conversations API * Sort and page conversations by last status ID * Appease linter * Fix deleting conversations and statuses * Refactor to make migrations automatic * Lint * Update tests post-merge * Fixes from live-fire testing * Linter caught a format problem * Refactor tests, fix cache * Negative test for non-DMs * Run conversations advanced migration on testrig startup as well as regular server startup * Document (lack of) side effects of API method for deleting a conversation * Make not-found check less nested for readability * Rename PutConversation to UpsertConversation * Use util.Ptr instead of IIFE * Reduce cache used by conversations * Remove unnecessary TableExpr/ColumnExpr * Use struct tags for both unique constraints on Conversation * Make it clear how paging with GetDirectStatusIDsBatch should be used * Let conversation paging skip conversations it can't render * Use Bun NewDropTable * Convert delete raw query to Bun * Convert update raw query to Bun * Convert latestConversationStatusesTempTable raw query partially to Bun * Convert conversationStatusesTempTable raw query partially to Bun * Rename field used to store result of MaxDirectStatusID * Move advanced migrations to their own tiny processor * Catch up util function name with main * Remove json.… wrappers * Remove redundant check * Combine error checks * Replace map with slice of structs * Address processor/type converter comments - Add context info for errors - Extract some common processor code into shared methods - Move conversation eligibility check ahead of populating conversation * Add error context when dropping temp tables
2024-07-17[chore] media pipeline improvements (#3110)Libravatar kim4
* don't set emoji / media image paths on failed download, migrate FileType from string to integer * fix incorrect uses of util.PtrOr, fix returned frontend media * fix migration not setting arguments correctly in where clause * fix not providing default with not null column * whoops * ensure a default gets set for media attachment file type * remove the exclusive flag from writing files in disk storage * rename PtrOr -> PtrOrZero, and rename PtrValueOr -> PtrOrValue to match * slight wording changes * use singular / plural word forms (no parentheses), is better for screen readers * update testmodels with unknown media type to have unset file details, update attachment focus handling converting to frontend, update tests * store first instance in ffmpeg wasm pool, fill remaining with closed instances
2024-07-15[feature/frontend] Add player for audio files; use thumbnail for `poster` ↵Libravatar tobi5
(#3099) * [feature/frontend] Audio player for audio media types * use video preview images for previews instead of video itself * don't preload * update tests for new zork status * collapse media gallery into single row when small
2024-07-15[chore] add step-by-step logging to long migration (#3102)Libravatar tobi1
2024-07-11[chore] Add interaction policy gtsmodels (#3075)Libravatar tobi11
* [chore] introduce interaction policy gts models * update migration a smidge * fix copy paste typo * update migration * use int for InteractionType
2024-07-10[bugfix] Don't throw error when parent statuses are missing (#2011) (#3088)Libravatar snow1
* [bugfix] Don't throw error when parent statuses are missing (#2011) * Split missing parent status case from error check
2024-06-29[feature] Default to WASM-based SQLite driver (#3053)Libravatar Daenney4
* [feature] Default to WASM-based SQLite driver With 0.16 out this switches our default SQLite driver to the WASM-based solution instead. So far the driver seems to perform just as well. Switching our default should result in it getting a bit more testing during the 0.17 development cycle. * add the ol' john hancock --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-06-26[chore] media and emoji refactoring (#3000)Libravatar kim4
* start updating media manager interface ready for storing attachments / emoji right away * store emoji and media as uncached immediately, then (re-)cache on Processing{}.Load() * remove now unused media workers * fix tests and issues * fix another test! * fix emoji activitypub uri setting behaviour, fix remainder of test compilation issues * fix more tests * fix (most of) remaining tests, add debouncing to repeatedly failing media / emojis * whoops, rebase issue * remove kim's whacky experiments * do some reshuffling, ensure emoji uri gets set * ensure marked as not cached on cleanup * tweaks to media / emoji processing to handle context canceled better * ensure newly fetched emojis actually get set in returned slice * use different varnames to be a bit more obvious * move emoji refresh rate limiting to dereferencer * add exported dereferencer functions for remote media, use these for recaching in processor * add check for nil attachment in updateAttachment() * remove unused emoji and media fields + columns * see previous commit * fix old migrations expecting image_updated_at to exists (from copies of old models) * remove freshness checking code (seems to be broken...) * fix error arg causing nil ptr exception * finish documentating functions with comments, slight tweaks to media / emoji deref error logic * remove some extra unneeded boolean checking * finish writing documentation (code comments) for exported media manager methods * undo changes to migration snapshot gtsmodels, updated failing migration to have its own snapshot * move doesColumnExist() to util.go in migrations package
2024-06-18[bugfix] rename `include_types[]` to `types[]` (#3023)Libravatar tobi2
2024-06-18[feature/frontend] Reports frontend v2 (#3022)Libravatar tobi3
* use apiutil + paging in admin processor+handlers * we're making it happen * fix little whoopsie * styling for report list * don't youuuu forget about meee don't don't don't don't * last bits * sanitize content before showing in report statuses * update report docs
2024-06-17[feature] Implement types[] param for notifications (#3009)Libravatar Vyr Cossont3
Counterpart of exclude_types[]. Also updates Swagger spec for types[] to use the correct param name and enumerate possible values. Fixes #3003
2024-06-14[bugfix] avoid v. long notification clear query (#3007)v0.16.0-rc3Libravatar tobi2
2024-06-11[bugfix] Deref stats async, serve stub collections if handshaking (#2990)v0.16.0-rc2Libravatar tobi3
* [bugfix] Deref stats async, allow peek if handshaking * don't return totalItems when handshaking or hiding collections * use GetLimit() * use StubAccountStats
2024-06-10[bugfix] boost and account recursion (#2982)Libravatar kim2
* fix possible infinite recursion if moved accounts are self-referential * adds a defensive check for a boost being a boost of a boost wrapper * add checks on input for a boost of a boost * remove unnecessary check * add protections on account move to prevent move recursion loops * separate status conversion without boost logic into separate function to remove risk of recursion * move boost check to boost function itself * formatting * use error 422 instead of 500 * use gtserror not standard errors package for error creation
2024-06-06[feature] User muting (#2960)Libravatar Vyr Cossont5
* User muting * Address review feedback * Rename uniqueness constraint on user_mutes to match convention * Remove unused account_id from where clause * Add UserMute to NewTestDB * Update test/envparsing.sh with new and fixed cache stuff * Address tobi's review comments * Make compiledUserMuteListEntry.expired consistent with UserMute.Expired * Make sure mute_expires_at is serialized as an explicit null for indefinite mutes --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-06-06[feature] do not uncache status / emoji media if attached status is ↵Libravatar kim5
bookmarked (#2956) * do not uncache status / emoji media if attached status is bookmarked * add status bookmark and bookmark IDs caches * update status bookmark tests * move IsStatusBookmarkedBy() to StatusBookmark{} interface, rely on cache * fix envparsing.sh test
2024-06-05[chore] Fiddle with CI tests; use wasmsqlite3 for CI tests (#2966)Libravatar tobi1
* [chore] Update CI test stuff * don't bother with ldflags * use wasmsqlite3
2024-06-04Compile filter keyword regexps when touched through PutFilter or ↵Libravatar Vyr Cossont1
UpdateFilter (#2951) Followup to #2903
2024-05-31[feature] Add from: search operator and account_id query param (#2943)Libravatar Vyr Cossont3
* Add from: search operator * Fix whitespace in Swagger YAML comment * Move query parsing into its own method * Document search * Clarify post search scope
2024-05-31[feature] Implement Filter API v2 (#2936)Libravatar Vyr Cossont3
* Use correct entity name * We support server-side filters now * Document filter v1 methods that can throw a 409 * Validate v1 filter phrase as filter title * Always check v1 filter API status codes in tests * Document keyword minimum requirement on filter API v1 * Make it possible to specify filter keyword update columns per filter keyword * Implement v2 filter API * Fix lint and tests * Update Swagger spec * Fix filter update test * Update Swagger spec *correctly* * Update actual files Swagger spec was generated from * Remove keywords_attributes and statuses_attributes * Add test for serialization of empty filter * More helpful messages when object is owned by wrong account
2024-05-27[experiment] add alternative wasm sqlite3 implementation available via ↵Libravatar kim13
build-tag (#2863) This allows for building GoToSocial with [SQLite transpiled to WASM](https://github.com/ncruces/go-sqlite3) and accessed through [Wazero](https://wazero.io/).
2024-05-22[performance] update storage backend and make use of seek syscall when ↵Libravatar kim1
available (#2924) * update to use go-storage/ instead of go-store/v2/storage/ * pull in latest version from codeberg * remove test output :innocent: * add code comments * set the exclusive bit when creating new files in disk config * bump to actual release version * bump to v0.1.1 (tis a simple no-logic change) * update readme * only use a temporary read seeker when decoding video if required (should only be S3 now) * use fastcopy library to use memory pooled buffers when calling TempFileSeeker() * update to use seek call in serveFileRange()
2024-05-21[performance] cache v2 filter keyword regular expressions (#2903)Libravatar kim1
* add caching of filterkeyword regular expressions * formatting * fix WholeWord nil check
2024-05-01[feature] Page through accounts as moderator (#2881)Libravatar tobi3
* [feature] Page through accounts as moderator * aaaaa * use COLLATE "C" for Postgres to ensure same ordering as SQLite * fix typo, test paging up * don't show moderation / info for our instance acct
2024-04-16[bugfix] fix get all tokens (#2841)Libravatar tobi2