summaryrefslogtreecommitdiff
path: root/internal/db
AgeCommit message (Collapse)AuthorFiles
2025-03-10[feat] utilize system's libsqlite3HEADtrunknextLibravatar Terin Stock4
Add support for using the system's libsqlite3 library by adding a driver using "github.com/mattn/go-sqlite3".
2025-02-20[bugfix] update fedi api to support multiple separate votes in same multiple ↵Libravatar kim2
choice poll (#3809)
2025-02-15[chore/performance] Batch migration queries (#3798)Libravatar tobi2
* separate enum migrations into their own individual transactions * pee poo * some performance tweaks and adding more comments * batch --------- Co-authored-by: kim <grufwub@gmail.com>
2025-02-14[bugfix] Drop status indices AFTER updating visibility (#3795)Libravatar tobi2
* [bugfix] Drop status indices AFTER updating visibility * rename to status vis indices just to indicate they're only used in the statuses hook func --------- Co-authored-by: kim <grufwub@gmail.com>
2025-02-11[performance] improved enum migrations (#3782)Libravatar kim2
* updates the enum migration to perform a singular update for all values, using an SQL case statement * fix logging * fix code comment * well i guess we'll get rid of the useful but unused function then, linter. fine, i see how it is! * append to byte buffer instead of WriteString() to shut the linter up (i know you're reading this, linter)
2025-02-06[chore] update otel libraries (#3740)Libravatar Dominik Süß1
* chore: update otel dependencies * refactor: combine tracing & metrics in observability package * chore: update example tracing compose file
2025-02-03[feature] Implement Web Push notification policy (#3721)Libravatar Vyr Cossont1
* Web Push: add policy column to subscriptions * Web Push: add policy to API * Web Push: test notification policy * go-fmt unrelated file (how did this get thru?)
2025-01-27[chore] some tidy ups (#3677)Libravatar kim1
* small formatting changes (no logic) * improve code comments * fix import cycle * shutup stinky linter
2025-01-27[bugfix] Fix top-level posts with a mention being counted as replies when ↵Libravatar tobi2
doing `exclude_replies` (#3689) * [bugfix] Fix top-level posts with a mention being counted as replies * add index for new reply exclusion query
2025-01-24[bugfix] Rename domain perm sub migration to unique date (#3679)Libravatar tobi2
* [bugfix] Rename domain perm sub migration to unique date * add repeat migration
2025-01-23[feature] Push notifications (#3587)Libravatar Vyr Cossont11
* Update push subscription API model to be Mastodon 4.0 compatible * Add webpush-go dependency # Conflicts: # go.sum * Single-row table for storing instance's VAPID key pair * Generate VAPID key pair during startup * Add VAPID public key to instance info API * Return VAPID public key when registering an app * Store Web Push subscriptions in DB * Add Web Push sender (similar to email sender) * Add no-op push senders to most processor tests * Test Web Push notifications from workers * Delete Web Push subscriptions when account is deleted * Implement push subscription API * Linter fixes * Update Swagger * Fix enum to int migration * Fix GetVAPIDKeyPair * Create web push subscriptions table with indexes * Log Web Push server error messages * Send instance URL as Web Push JWT subject * Accept any 2xx code as a success * Fix malformed VAPID sub claim * Use packed notification flags * Remove unused date columns * Add notification type for update notifications Not used yet * Make GetVAPIDKeyPair idempotent and remove PutVAPIDKeyPair * Post-rebase fixes * go mod tidy * Special-case 400 errors other than 408/429 Most client errors should remove the subscription. * Improve titles, trim body to reasonable length * Disallow cleartext HTTP for Web Push servers * Fix lint * Remove redundant index on unique column Also removes redundant unique and notnull tags on ID column since these are implied by pk * Make realsender.go more readable * Use Tobi's style for wrapping errors * Restore treating all 5xx codes as temporary problems * Always load target account settings * Stub `policy` and `standard` * webpush.Sender: take type converter as ctor param * Move webpush.MockSender and noopSender into testrig
2025-01-20[bugfix] Store LastModified for domain perm subs + send as If-Modified-Since ↵Libravatar tobi2
(#3655)
2025-01-14[chore] better dns validation (#3644)Libravatar kim5
* add seperate PunifyValidate() function for properly validating domain names when converting to punycode * rename function, strip port from domain validation
2025-01-09[bugfix] migration to cleanup dropped status edits (#3637)Libravatar kim1
* add migration to cleanup dropped status edits from previous bug, relinking with statuses * don't use bun.Ident where not needed * fix join statement * fix query to work for both postgres and sqlite * actually update by the currently set status ID field :facepalm:
2025-01-08[chore] replace statuses.updated_at column with statuses.edited_at (#3636)Libravatar kim6
* update statuses table to replace updated_at column with edited_at * code comment * better code comments, fix setting of status + edit + mention + poll database times * fix log to logf call * fix status.EditIDs not being carried over in dereferencer.encrichStatus() * move status.EditID setting into handleStatusEdit()
2025-01-05[feature] Create/update/remove domain permission subscriptions (#3623)Libravatar tobi8
* [feature] Create/update/remove domain permission subscriptions * lint * envparsing * remove errant fmt.Println * create drafts, subs, exclude, from snapshot models * name etag column correctly * remove count column * lint
2024-12-23[feature] add support for clients editing statuses and fetching status ↵Libravatar kim2
revision history (#3628) * start adding client support for making status edits and viewing history * modify 'freshest' freshness window to be 5s, add typeutils test for status -> api edits * only populate the status edits when specifically requested * start adding some simple processor status edit tests * add test editing status but adding a poll * test edits appropriately adding poll expiry handlers * finish adding status edit tests * store both new and old revision emojis in status * add code comment * ensure the requester's account is populated before status edits * add code comments for status edit tests * update status edit form swagger comments * remove unused function * fix status source test * add more code comments, move media description check back to media process in status create * fix tests, add necessary form struct tag
2024-12-09[chore] Log during potentially long migrations (#3613)Libravatar tobi4
* [chore] Log during potentially long migrations * remove unneeded log lines
2024-12-05[feature] add support for receiving federated status edits (#3597)Libravatar kim22
* add support for extracting Updated field from Statusable implementers * add support for status edits in the database, and update status dereferencer to handle them * remove unused AdditionalInfo{}.CreatedAt * remove unused AdditionalEmojiInfo{}.CreatedAt * update new mention creation to use status.UpdatedAt * remove mention.UpdatedAt, fixes related to NewULIDFromTime() change * add migration to remove Mention{}.UpdatedAt field * add migration to add the StatusEdit{} table * start adding tests, add delete function for status edits * add more of status edit migrations, fill in more of the necessary edit delete functionality * remove unused function * allow generating gotosocial compatible ulid via CLI with `go run ./cmd/gen-ulid` * add StatusEdit{} test models * fix new statusedits sql * use model instead of table name * actually remove the Mention.UpdatedAt field... * fix tests now new models are added, add more status edit DB tests * fix panic wording * add test for deleting status edits * don't automatically set `updated_at` field on updated statuses * flesh out more of the dereferencer status edit tests, ensure updated at field set on outgoing AS statuses * remove media_attachments.updated_at column * fix up more tests, further complete the dereferencer status edit tests * update more status serialization tests not expecting 'updated' AS property * gah!! json serialization tests!! * undo some gtscontext wrapping changes * more serialization test fixing :smiling_face_with_tear: * more test fixing, ensure the edit.status_id field is actually set :facepalm: * fix status edit test * grrr linter * add edited_at field to apimodel status * remove the choice of paging on the timeline public filtered test (otherwise it needs updating every time you add statuses ...) * ensure that status.updated_at always fits chronologically * fix more serialization tests ... * add more code comments * fix envparsing * update swagger file * properly handle media description changes during status edits * slight formatting tweak * code comment
2024-12-04[feature] unending polls (#3592)Libravatar kim1
* adds support for unending polls to be created locally * remove unused argument
2024-12-02[feature] Add global instance CSS customization setting (#3352)Libravatar Victor Dyotte1
Allow instance admins to add custom CSS that will affect every page of their instance. This is done with a new CustomCSS instance setting that works pretty much exactly like the Users CustomCSS property. This custom CSS is then requested for every page load. User styles/themes take precedence over this CSS. Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-11-29[performance] use new instance of bun.DB *after* migrations to reduce number ↵Libravatar kim1
of in-memory model schema (#3578) * use new instance of bun.DB *after* migrations to reduce number of model schema in-memory * update sqlite address comment
2024-11-28[chore] rename New___(string) int signature functions to Parse___(string) ↵Libravatar kim1
int (#3580) * rename New___(string) int {} signature functions to Parse___(string) int {} * remove test output
2024-11-25[bugfix] notification types missing from link header (#3571)Libravatar kim3
* ensure notification types get included in link header query for notifications * fix type query keys
2024-11-25[chore] Bump tooling versions, bump go -> v1.23.0 (#3258)Libravatar tobi1
* [chore] Bump tooling versions, bump go -> v1.23.0 * undo silly change * sign * bump go version in go.mod * allow overflow in imaging * goreleaser deprecation notices * bump versions * undo accidental rebase change * update container versions to just use latest major version * update swagger to our release with go1.23 fix * update goreleaser to use our vendored swagger version --------- Co-authored-by: kim <grufwub@gmail.com>
2024-11-25[performance] convert enum strings to ints (#3558)Libravatar kim17
* convert statuses.visibility and notifications.notification_type columns from type string -> int for performance / space savings * fix test trying to compare string to int * fix instance count query using string literal instead of gtsmodel const type * ensure a default value is always set * also migrate the account settings and sin bin status tables * initialize maps outside loops and place into singular enum mapping creation func * use int16 for enum types * update sinbinstatus creation to be from a snapshot at initial creation * add snapshot of poll type at creation time
2024-11-21[feature] Add domain permission drafts and excludes (#3547)Libravatar tobi7
* [feature] Add domain permission drafts and excludes * fix typescript complaining * lint * make filenames more consistent * test own domain excluded
2024-11-21[bugfix] post counters should not include direct messages (#3554)Libravatar Jannis2
* [bugfix] post counters should not include direct messages #3504 The fix is relativly simple, it just adds a line to the relevant function which excludes all private posts. * Formating fix * mb
2024-11-11[performance] minimise log field allocations (#3529)Libravatar kim21
* when appending log field only do so by minimal amount * move slice utils to separate package to fix import cycle, add GrowJust() and AppendJust() functions * fix GrowJust() not returning slice of same length * improved xslices tests * make AppendJust() test check for slice contents, fix AppendJust() final copying behaviour * add a +1 with field growth to try minimise allocation for log 'msg' field
2024-10-19[bugfix] Fix filter title unique constraint (#3458)Libravatar tobi4
2024-10-16[chore] Upgrade golangci-lint, ignore existing int overflow warnings (#3420)Libravatar Markus Unterwaditzer1
* [chore] Bump tooling versions, bump go -> v1.23.0 * undo silly change * sign * bump go version in go.mod * allow overflow in imaging * goreleaser deprecation notices * [chore] Upgrade golangci-lint, ignore existing int overflow warnings There is a new lint for unchecked int casts. Integer overflows are bad, but the old code that triggers this lint seems to be perfectly fine. Instead of disabling the lint entirely for new code as well, grandfather in existing code. * fix golangci-lint documentation link * revert unrelated changes * revert another unrelated change * get rid of remaining nolint:gosec * swagger updates * apply review feedback * fix wrong formatting specifier thing * fix the linter for real --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-10-14updates exported interface types to match ncruces/go-sqlite3/driver methods ↵v0.17.0Libravatar kim1
(#3440)
2024-10-14[feature/OFFICIALLY UNSUPPORTED] add nowasm build tag to disable building ↵Libravatar kim4
with WebAssembly (#3429) * add experimental build-tag 'nowasm' which uses local ffmpeg / ffprobe * updated experimental support message * add comment to build script explaining build tag * add nowasm build tags to moderncsqlite files
2024-10-14[feature] for an sqlite database with journal mode != WAL, use maximum of 1 ↵Libravatar kim1
open conn (#3428)
2024-10-11[bugfix] Ensure `pending_approval` set on statuses + status faves (#3415)Libravatar tobi2
* [bugfix] Ensure pending_approval set on statuses + status faves * set PendingApproval on boosts * assume not pending approval
2024-10-05[bugfix] Update select of pending interaction requests to account for ↵Libravatar tobi2
potential nil URI (#3392)
2024-10-04[performance] remove the pragma optimize analysis limit on connection close ↵Libravatar kim1
(#3386)
2024-09-28[bugfix] visibility after implicit approval not getting invalidated (#3370)Libravatar kim2
* replicate issue * update go-structr to v0.8.10 with internal linked-list fix, small tweaks to caching of interaction requests * remove debug function --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-09-23[chore] local instance count query caching, improved status context endpoint ↵Libravatar kim1
logging, don't log ErrHideStatus when timelining (#3330) * ensure that errors checking status visibility / converting aren't dropped * add some more context to error messages * include calling function name in log entries * don't error on timelining hidden status * further code to ignore statusfilter.ErrHideStatus type errors * remove unused error type * add local instance status / domain / user counts * add checks for localhost * rename from InstanceCounts to LocalInstance * improved code comment
2024-09-20[chore] ensure consistent caller name fetching regardless of compiler ↵Libravatar kim1
inlining (#3323) * move logging levels into log package itself * ensure inconsistent inlining doesn't mess with log calling function name * remove unused global variable * fix log level
2024-09-16[performance] cache more database calls, reduce required database calls ↵Libravatar kim36
overall (#3290) * improvements to caching for lists and relationship to accounts / follows * fix nil panic in AddToList() * ensure list related caches are correctly invalidated * ensure returned ID lists are ordered correctly * bump go-structr to v0.8.9 (returns early if zero uncached keys to be loaded) * remove zero checks in uncached key load functions (go-structr now handles this) * fix issues after rebase on upstream/main * update the expected return order of CSV exports (since list entries are now down by entry creation date) * rename some funcs, allow deleting list entries for multiple follow IDs at a time, fix up more tests * use returning statements on delete to get cache invalidation info * fixes to recent database delete changes * fix broken list entries delete sql * remove unused db function * update remainder of delete functions to behave in similar way, some other small tweaks * fix delete user sql, allow returning on err no entries * uncomment + fix list database tests * update remaining list tests * update envparsing test * add comments to each specific key being invalidated * add more cache invalidation explanatory comments * whoops; actually delete poll votes from database in the DeletePollByID() func * remove added but-commented-out field * improved comment regarding paging being disabled * make cache invalidation comments match what's actually happening * fix up delete query comments to match what is happening * rename function to read a bit better * don't use ErrNoEntries on delete when not needed (it's only needed for a RETURNING call) * update function name in test * move list exclusivity check to AFTER eligibility check. use log.Panic() instead of panic() * use the poll_id column in poll_votes for selecting votes in poll ID * fix function name
2024-09-14[chore] Update go-sqlite3 to 0.18.3 (#3295)Libravatar Daenney1
* [chore] Update go-sqlite3 to 0.18.3 * [chore] Fix getting the sqlite3.Conn
2024-09-11[bugfix] Update home timeline query to ignore exclusive list entries (#3289)Libravatar tobi2
* [bugfix] Update home timeline query to ignore exclusive list entries * a
2024-09-10[feature] Process `Reject` of interaction via fedi API, put rejected ↵Libravatar tobi5
statuses in the "sin bin" 😈 (#3271) * [feature] Process `Reject` of interaction via fedi API, put rejected statuses in the "sin bin" * update test * move nil check back to `rejectStatusIRI`
2024-09-09[feature] Implement exclusive lists (#3280)Libravatar Vyr Cossont1
Fixes #2616
2024-09-09[feature/frontend] Add options to include Unlisted posts or hide all posts ↵Libravatar tobi3
(#3272) * [feature/frontend] Add options to include Unlisted posts or hide all posts * finish up * swagger * move invalidate call into bundb package, avoid invalidating if not necessary * rename show_web_statuses => web_visibility * don't use ptr for webvisibility * last bits
2024-09-08[bugfix] Fix temp table deletion causing runaway allocations (#3278)Libravatar tobi2
* [bugfix] Fix temp table deletion causing runaway allocations * move some vars around * small fixes * rely on conn max age to recycle temp tables * fackin' ell m8
2024-09-02[chore/performance] Further reduce nil uncached queries (#3267)Libravatar tobi8
* [chore/performance] Further reduce nil uncached queries * more checks
2024-09-02[chore] Don't try to select zero uncached filters (#3266)Libravatar tobi1
2024-09-02[chore/performance] Avoid unnecessary "uncached" queries (#3265)Libravatar tobi17
* [chore/performance] Avoid unnecessary "uncached" queries * go fmt