summaryrefslogtreecommitdiff
path: root/internal/db/bundb/migrations
AgeCommit message (Collapse)AuthorFiles
2025-02-15[chore/performance] Batch migration queries (#3798)Libravatar tobi2
* separate enum migrations into their own individual transactions * pee poo * some performance tweaks and adding more comments * batch --------- Co-authored-by: kim <grufwub@gmail.com>
2025-02-14[bugfix] Drop status indices AFTER updating visibility (#3795)Libravatar tobi2
* [bugfix] Drop status indices AFTER updating visibility * rename to status vis indices just to indicate they're only used in the statuses hook func --------- Co-authored-by: kim <grufwub@gmail.com>
2025-02-11[performance] improved enum migrations (#3782)Libravatar kim2
* updates the enum migration to perform a singular update for all values, using an SQL case statement * fix logging * fix code comment * well i guess we'll get rid of the useful but unused function then, linter. fine, i see how it is! * append to byte buffer instead of WriteString() to shut the linter up (i know you're reading this, linter)
2025-02-03[feature] Implement Web Push notification policy (#3721)Libravatar Vyr Cossont1
* Web Push: add policy column to subscriptions * Web Push: add policy to API * Web Push: test notification policy * go-fmt unrelated file (how did this get thru?)
2025-01-27[bugfix] Fix top-level posts with a mention being counted as replies when ↵Libravatar tobi1
doing `exclude_replies` (#3689) * [bugfix] Fix top-level posts with a mention being counted as replies * add index for new reply exclusion query
2025-01-24[bugfix] Rename domain perm sub migration to unique date (#3679)Libravatar tobi2
* [bugfix] Rename domain perm sub migration to unique date * add repeat migration
2025-01-23[feature] Push notifications (#3587)Libravatar Vyr Cossont3
* Update push subscription API model to be Mastodon 4.0 compatible * Add webpush-go dependency # Conflicts: # go.sum * Single-row table for storing instance's VAPID key pair * Generate VAPID key pair during startup * Add VAPID public key to instance info API * Return VAPID public key when registering an app * Store Web Push subscriptions in DB * Add Web Push sender (similar to email sender) * Add no-op push senders to most processor tests * Test Web Push notifications from workers * Delete Web Push subscriptions when account is deleted * Implement push subscription API * Linter fixes * Update Swagger * Fix enum to int migration * Fix GetVAPIDKeyPair * Create web push subscriptions table with indexes * Log Web Push server error messages * Send instance URL as Web Push JWT subject * Accept any 2xx code as a success * Fix malformed VAPID sub claim * Use packed notification flags * Remove unused date columns * Add notification type for update notifications Not used yet * Make GetVAPIDKeyPair idempotent and remove PutVAPIDKeyPair * Post-rebase fixes * go mod tidy * Special-case 400 errors other than 408/429 Most client errors should remove the subscription. * Improve titles, trim body to reasonable length * Disallow cleartext HTTP for Web Push servers * Fix lint * Remove redundant index on unique column Also removes redundant unique and notnull tags on ID column since these are implied by pk * Make realsender.go more readable * Use Tobi's style for wrapping errors * Restore treating all 5xx codes as temporary problems * Always load target account settings * Stub `policy` and `standard` * webpush.Sender: take type converter as ctor param * Move webpush.MockSender and noopSender into testrig
2025-01-20[bugfix] Store LastModified for domain perm subs + send as If-Modified-Since ↵Libravatar tobi2
(#3655)
2025-01-09[bugfix] migration to cleanup dropped status edits (#3637)Libravatar kim1
* add migration to cleanup dropped status edits from previous bug, relinking with statuses * don't use bun.Ident where not needed * fix join statement * fix query to work for both postgres and sqlite * actually update by the currently set status ID field :facepalm:
2025-01-08[chore] replace statuses.updated_at column with statuses.edited_at (#3636)Libravatar kim3
* update statuses table to replace updated_at column with edited_at * code comment * better code comments, fix setting of status + edit + mention + poll database times * fix log to logf call * fix status.EditIDs not being carried over in dereferencer.encrichStatus() * move status.EditID setting into handleStatusEdit()
2025-01-05[feature] Create/update/remove domain permission subscriptions (#3623)Libravatar tobi5
* [feature] Create/update/remove domain permission subscriptions * lint * envparsing * remove errant fmt.Println * create drafts, subs, exclude, from snapshot models * name etag column correctly * remove count column * lint
2024-12-09[chore] Log during potentially long migrations (#3613)Libravatar tobi4
* [chore] Log during potentially long migrations * remove unneeded log lines
2024-12-05[feature] add support for receiving federated status edits (#3597)Libravatar kim8
* add support for extracting Updated field from Statusable implementers * add support for status edits in the database, and update status dereferencer to handle them * remove unused AdditionalInfo{}.CreatedAt * remove unused AdditionalEmojiInfo{}.CreatedAt * update new mention creation to use status.UpdatedAt * remove mention.UpdatedAt, fixes related to NewULIDFromTime() change * add migration to remove Mention{}.UpdatedAt field * add migration to add the StatusEdit{} table * start adding tests, add delete function for status edits * add more of status edit migrations, fill in more of the necessary edit delete functionality * remove unused function * allow generating gotosocial compatible ulid via CLI with `go run ./cmd/gen-ulid` * add StatusEdit{} test models * fix new statusedits sql * use model instead of table name * actually remove the Mention.UpdatedAt field... * fix tests now new models are added, add more status edit DB tests * fix panic wording * add test for deleting status edits * don't automatically set `updated_at` field on updated statuses * flesh out more of the dereferencer status edit tests, ensure updated at field set on outgoing AS statuses * remove media_attachments.updated_at column * fix up more tests, further complete the dereferencer status edit tests * update more status serialization tests not expecting 'updated' AS property * gah!! json serialization tests!! * undo some gtscontext wrapping changes * more serialization test fixing :smiling_face_with_tear: * more test fixing, ensure the edit.status_id field is actually set :facepalm: * fix status edit test * grrr linter * add edited_at field to apimodel status * remove the choice of paging on the timeline public filtered test (otherwise it needs updating every time you add statuses ...) * ensure that status.updated_at always fits chronologically * fix more serialization tests ... * add more code comments * fix envparsing * update swagger file * properly handle media description changes during status edits * slight formatting tweak * code comment
2024-12-02[feature] Add global instance CSS customization setting (#3352)Libravatar Victor Dyotte1
Allow instance admins to add custom CSS that will affect every page of their instance. This is done with a new CustomCSS instance setting that works pretty much exactly like the Users CustomCSS property. This custom CSS is then requested for every page load. User styles/themes take precedence over this CSS. Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-11-28[chore] rename New___(string) int signature functions to Parse___(string) ↵Libravatar kim1
int (#3580) * rename New___(string) int {} signature functions to Parse___(string) int {} * remove test output
2024-11-25[performance] convert enum strings to ints (#3558)Libravatar kim12
* convert statuses.visibility and notifications.notification_type columns from type string -> int for performance / space savings * fix test trying to compare string to int * fix instance count query using string literal instead of gtsmodel const type * ensure a default value is always set * also migrate the account settings and sin bin status tables * initialize maps outside loops and place into singular enum mapping creation func * use int16 for enum types * update sinbinstatus creation to be from a snapshot at initial creation * add snapshot of poll type at creation time
2024-11-21[feature] Add domain permission drafts and excludes (#3547)Libravatar tobi1
* [feature] Add domain permission drafts and excludes * fix typescript complaining * lint * make filenames more consistent * test own domain excluded
2024-10-19[bugfix] Fix filter title unique constraint (#3458)Libravatar tobi3
2024-10-11[bugfix] Ensure `pending_approval` set on statuses + status faves (#3415)Libravatar tobi2
* [bugfix] Ensure pending_approval set on statuses + status faves * set PendingApproval on boosts * assume not pending approval
2024-10-05[bugfix] Update select of pending interaction requests to account for ↵Libravatar tobi1
potential nil URI (#3392)
2024-09-10[feature] Process `Reject` of interaction via fedi API, put rejected ↵Libravatar tobi1
statuses in the "sin bin" 😈 (#3271) * [feature] Process `Reject` of interaction via fedi API, put rejected statuses in the "sin bin" * update test * move nil check back to `rejectStatusIRI`
2024-09-09[feature] Implement exclusive lists (#3280)Libravatar Vyr Cossont1
Fixes #2616
2024-09-09[feature/frontend] Add options to include Unlisted posts or hide all posts ↵Libravatar tobi1
(#3272) * [feature/frontend] Add options to include Unlisted posts or hide all posts * finish up * swagger * move invalidate call into bundb package, avoid invalidating if not necessary * rename show_web_statuses => web_visibility * don't use ptr for webvisibility * last bits
2024-08-24[feature] Interaction requests client api + settings panel (#3215)Libravatar tobi2
* [feature] Interaction requests client api + settings panel * test accept / reject * fmt * don't pin rejected interaction * use single db model for interaction accept, reject, and request * swaggor * env sharting * append errors * remove ErrNoEntries checks * change intReqID to reqID * rename "pend" to "request" * markIntsPending -> mark interactionsPending * use log instead of returning error when rejecting interaction * empty migration * jolly renaming * make interactionURI unique again * swag grr * remove unnecessary locks * invalidate as last step
2024-07-30[feature] persist worker queues to db (#3042)Libravatar kim1
* persist queued worker tasks to database on shutdown, fill worker queues from database on startup * ensure the tasks are sorted by creation time before pushing them * add migration to insert WorkerTask{} into database, add test for worker task persistence * add test for recovering worker queues from database * quick tweak * whoops we ended up with double cleaner job scheduling * insert each task separately, because bun is throwing some reflection error?? * add specific checking of cancelled worker contexts * add http request signing to deliveries recovered from database * add test for outgoing public key ID being correctly set on delivery * replace select with Queue.PopCtx() * get rid of loop now we don't use it * remove field now we don't use it * ensure that signing func is set * header values weren't being copied over :facepalm: * use ptr for httpclient.Request in delivery * move worker queue filling to later in server init process * fix rebase issues * make logging less shouty * use slices.Delete() instead of copying / reslicing * have database return tasks in ascending order instead of sorting them * add a 1 minute timeout to persisting worker queues
2024-07-29[feature] Implement following hashtags (#3141)Libravatar Vyr Cossont1
* Implement followed tags API * Insert statuses with followed tags into home timelines * Test following and unfollowing tags * Correct Swagger path params * Trim conversation caches * Migration for followed_tags table * Followed tag caches and DB implementation * Lint and tests * Add missing tag info endpoint, reorganize tag API * Unwrap boosts when timelining based on tags * Apply visibility filters to tag followers * Address review comments
2024-07-26[feature] Federate interaction policies + Accepts; enforce policies (#3138)Libravatar tobi1
* [feature] Federate interaction policies + Accepts; enforce policies * use Acceptable type * fix index * remove appendIRIStrs * add GetAccept federatingdb function * lock on object IRI
2024-07-23[chore] Add some log lines to recent migrations warning not to interrupt (#3134)Libravatar tobi3
* [chore] Add some log lines to recent migrations warning not to interrupt * arse
2024-07-23Remove content and related fields from boosts (#3131)Libravatar Vyr Cossont1
These duplicate the content of the target and aren't necessary for anything. - Stops copying some fields from target when boosting or processing a remote boost - Adds a migration to null out existing duplicate data - Updates tests
2024-07-23[feature] Conversations API (#3013)Libravatar Vyr Cossont2
* Implement conversations API * Sort and page conversations by last status ID * Appease linter * Fix deleting conversations and statuses * Refactor to make migrations automatic * Lint * Update tests post-merge * Fixes from live-fire testing * Linter caught a format problem * Refactor tests, fix cache * Negative test for non-DMs * Run conversations advanced migration on testrig startup as well as regular server startup * Document (lack of) side effects of API method for deleting a conversation * Make not-found check less nested for readability * Rename PutConversation to UpsertConversation * Use util.Ptr instead of IIFE * Reduce cache used by conversations * Remove unnecessary TableExpr/ColumnExpr * Use struct tags for both unique constraints on Conversation * Make it clear how paging with GetDirectStatusIDsBatch should be used * Let conversation paging skip conversations it can't render * Use Bun NewDropTable * Convert delete raw query to Bun * Convert update raw query to Bun * Convert latestConversationStatusesTempTable raw query partially to Bun * Convert conversationStatusesTempTable raw query partially to Bun * Rename field used to store result of MaxDirectStatusID * Move advanced migrations to their own tiny processor * Catch up util function name with main * Remove json.… wrappers * Remove redundant check * Combine error checks * Replace map with slice of structs * Address processor/type converter comments - Add context info for errors - Extract some common processor code into shared methods - Move conversation eligibility check ahead of populating conversation * Add error context when dropping temp tables
2024-07-17[chore] media pipeline improvements (#3110)Libravatar kim4
* don't set emoji / media image paths on failed download, migrate FileType from string to integer * fix incorrect uses of util.PtrOr, fix returned frontend media * fix migration not setting arguments correctly in where clause * fix not providing default with not null column * whoops * ensure a default gets set for media attachment file type * remove the exclusive flag from writing files in disk storage * rename PtrOr -> PtrOrZero, and rename PtrValueOr -> PtrOrValue to match * slight wording changes * use singular / plural word forms (no parentheses), is better for screen readers * update testmodels with unknown media type to have unset file details, update attachment focus handling converting to frontend, update tests * store first instance in ffmpeg wasm pool, fill remaining with closed instances
2024-07-15[chore] add step-by-step logging to long migration (#3102)Libravatar tobi1
2024-07-11[chore] Add interaction policy gtsmodels (#3075)Libravatar tobi2
* [chore] introduce interaction policy gts models * update migration a smidge * fix copy paste typo * update migration * use int for InteractionType
2024-06-26[chore] media and emoji refactoring (#3000)Libravatar kim4
* start updating media manager interface ready for storing attachments / emoji right away * store emoji and media as uncached immediately, then (re-)cache on Processing{}.Load() * remove now unused media workers * fix tests and issues * fix another test! * fix emoji activitypub uri setting behaviour, fix remainder of test compilation issues * fix more tests * fix (most of) remaining tests, add debouncing to repeatedly failing media / emojis * whoops, rebase issue * remove kim's whacky experiments * do some reshuffling, ensure emoji uri gets set * ensure marked as not cached on cleanup * tweaks to media / emoji processing to handle context canceled better * ensure newly fetched emojis actually get set in returned slice * use different varnames to be a bit more obvious * move emoji refresh rate limiting to dereferencer * add exported dereferencer functions for remote media, use these for recaching in processor * add check for nil attachment in updateAttachment() * remove unused emoji and media fields + columns * see previous commit * fix old migrations expecting image_updated_at to exists (from copies of old models) * remove freshness checking code (seems to be broken...) * fix error arg causing nil ptr exception * finish documentating functions with comments, slight tweaks to media / emoji deref error logic * remove some extra unneeded boolean checking * finish writing documentation (code comments) for exported media manager methods * undo changes to migration snapshot gtsmodels, updated failing migration to have its own snapshot * move doesColumnExist() to util.go in migrations package
2024-06-06[feature] User muting (#2960)Libravatar Vyr Cossont1
* User muting * Address review feedback * Rename uniqueness constraint on user_mutes to match convention * Remove unused account_id from where clause * Add UserMute to NewTestDB * Update test/envparsing.sh with new and fixed cache stuff * Address tobi's review comments * Make compiledUserMuteListEntry.expired consistent with UserMute.Expired * Make sure mute_expires_at is serialized as an explicit null for indefinite mutes --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-05-22[performance] update storage backend and make use of seek syscall when ↵Libravatar kim1
available (#2924) * update to use go-storage/ instead of go-store/v2/storage/ * pull in latest version from codeberg * remove test output :innocent: * add code comments * set the exclusive bit when creating new files in disk config * bump to actual release version * bump to v0.1.1 (tis a simple no-logic change) * update readme * only use a temporary read seeker when decoding video if required (should only be S3 now) * use fastcopy library to use memory pooled buffers when calling TempFileSeeker() * update to use seek call in serveFileRange()
2024-05-01[feature] Page through accounts as moderator (#2881)Libravatar tobi1
* [feature] Page through accounts as moderator * aaaaa * use COLLATE "C" for Postgres to ensure same ordering as SQLite * fix typo, test paging up * don't show moderation / info for our instance acct
2024-04-16[feature/performance] Store account stats in separate table (#2831)Libravatar tobi1
* [feature/performance] Store account stats in separate table, get stats from remote * test account stats * add some missing increment / decrement calls * change stats function signatures * rejig logging a bit * use lock when updating stats
2024-04-11[feature] New user sign-up via web page (#2796)Libravatar tobi3
* [feature] User sign-up form and admin notifs * add chosen + filtered languages to migration * remove stray comment * chosen languages schmosen schmanguages * proper error on local account missing
2024-04-02[bugfix] Set domain for empty-domain Friendica accounts (#2800)v0.15.0-rc1Libravatar tobi1
2024-03-25[feature] User-selectable preset CSS themes for accounts (#2777)Libravatar tobi1
* [feature] User-selectable preset themes * docs, more theme stuff * lint, tests * fix css name * correct some little issues * add another theme * fix poll background * okay last theme i swear * make retrieval of apimodel themes more conventional * preallocate stylesheet slices
2024-03-22[chore] Move local account settings to separate db table (#2770)Libravatar tobi1
* [chore] Move local account settings to separate database model * don't use separate settings_id
2024-03-10[bugfix] Fix whitespace move_id issue (#2742)Libravatar tobi1
2024-03-06[feature/chore] Add Move database functions + cache (#2647)Libravatar tobi1
* [feature/chore] Add Move database functions + cache * add move mem ratio to envparsing.sh * update comment
2024-03-06[feature] Filters v1 (#2594)Libravatar Vyr Cossont1
* Implement client-side v1 filters * Exclude linter false positives * Update test/envparsing.sh * Fix minor Swagger, style, and Bun usage issues * Regenerate Swagger * De-generify filter keywords * Remove updating filter statuses This is an operation that the Mastodon v2 filter API doesn't actually have, because filter statuses, unlike keywords, don't have options: the only info they contain is the status ID to be filtered. * Add a test for filter statuses specifically * De-generify filter statuses * Inline FilterEntry * Use vertical style for Bun operations consistently * Add comment on Filter DB interface * Remove GoLand linter control comments Our existing linters should catch these, or they don't matter very much * Reduce memory ratio for filters
2024-02-28[chore] re-add `statuses_account_id_id_idx` if removed (#2699)v0.14.0-rc2Libravatar tobi2
* [chore] re-add `statuses_account_id_id_idx` if removed * if not exists
2024-02-27[chore] add log line about lengthy reindex migration (#2695)v0.14.0-rc1Libravatar tobi1
2024-02-27[bugfix] Account timeline: exclude self-replies that mention other accounts ↵Libravatar Vyr Cossont1
(#2670) * Account timeline: exclude self-replies that mention other accounts * Add index for querying unmentioned statuses * remove now unused statuses_account_id_id_idx --------- Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-01-16[feature] Account alias / move API + db models (#2518)Libravatar tobi3
* [feature] Account alias / move API + db models * go fmt * fix little cherry-pick issues * update error checking, formatting * add and use new util functions to simplify alias logic
2024-01-15[bugfix] Replace named unique constraint on header filter header with ↵Libravatar tobi1
generic unique directive (#2525) * [bugfix] Replace named unique constraint on header filter header with generic unique directive * add migration retry * the old fixie uppie * fix constraint name * my goodness