diff options
| author | 2025-10-03 12:28:55 +0200 | |
|---|---|---|
| committer | 2025-10-03 12:28:55 +0200 | |
| commit | e7cd8bb43ef3a17b0a856f45b64bdc2c8336ba44 (patch) | |
| tree | 3e86c27bb31e029b410fd2c89b5610eee5e93073 /internal/db/bundb/migrations/20250415111056_thread_all_statuses/old/status.go | |
| parent | [bugfix/frontend] restore blockquote “block” margin (#4465) (diff) | |
| download | gotosocial-e7cd8bb43ef3a17b0a856f45b64bdc2c8336ba44.tar.xz | |
[chore] Use bulk updates + fewer loops in status rethreading migration (#4459)
This pull request tries to optimize our status rethreading migration by using bulk updates + avoiding unnecessary writes, and doing the migration in one top-level loop and one stragglers loop, without the extra loop to copy thread_id over.
On my machine it runs at about 2400 rows per second on Postgres, now, and about 9000 rows per second on SQLite.
Tried *many* different ways of doing this, with and without temporary indexes, with different batch and transaction sizes, etc., and this seems to be just about the most performant way of getting stuff done.
With the changes, a few minutes have been shaved off migration time testing on my development machine. *Hopefully* this will translate to more time shaved off when running on a vps with slower read/write speed and less processor power.
SQLite before:
```
real 20m58,446s
user 16m26,635s
sys 5m53,648s
```
SQLite after:
```
real 14m25,435s
user 12m47,449s
sys 2m27,898s
```
Postgres before:
```
real 28m25,307s
user 3m40,005s
sys 4m45,018s
```
Postgres after:
```
real 22m31,999s
user 3m46,674s
sys 4m39,592s
```
Reviewed-on: https://codeberg.org/superseriousbusiness/gotosocial/pulls/4459
Co-authored-by: tobi <tobi.smethurst@protonmail.com>
Co-committed-by: tobi <tobi.smethurst@protonmail.com>
Diffstat (limited to 'internal/db/bundb/migrations/20250415111056_thread_all_statuses/old/status.go')
| -rw-r--r-- | internal/db/bundb/migrations/20250415111056_thread_all_statuses/old/status.go | 8 |
1 files changed, 7 insertions, 1 deletions
diff --git a/internal/db/bundb/migrations/20250415111056_thread_all_statuses/old/status.go b/internal/db/bundb/migrations/20250415111056_thread_all_statuses/old/status.go index f33a2b29e..8cfce2e6b 100644 --- a/internal/db/bundb/migrations/20250415111056_thread_all_statuses/old/status.go +++ b/internal/db/bundb/migrations/20250415111056_thread_all_statuses/old/status.go @@ -21,7 +21,10 @@ import ( "time" ) -// Status represents a user-created 'post' or 'status' in the database, either remote or local +// Status represents a user-created 'post' or 'status' in the database, either remote or local. +// +// Note: this model differs from an exact representation of the old model at the time of migration, +// as it includes the intermediate field "ThreadIDNew", which is only used during the migration. type Status struct { ID string `bun:"type:CHAR(26),pk,nullzero,notnull,unique"` // id of this item in the database CreatedAt time.Time `bun:"type:timestamptz,nullzero,notnull,default:current_timestamp"` // when was item created @@ -60,6 +63,9 @@ type Status struct { PendingApproval *bool `bun:",nullzero,notnull,default:false"` // If true then status is a reply or boost wrapper that must be Approved by the reply-ee or boost-ee before being fully distributed. PreApproved bool `bun:"-"` // If true, then status is a reply to or boost wrapper of a status on our instance, has permission to do the interaction, and an Accept should be sent out for it immediately. Field not stored in the DB. ApprovedByURI string `bun:",nullzero"` // URI of an Accept Activity that approves the Announce or Create Activity that this status was/will be attached to. + + // This field is *only* used during the migration, it was not on the original status model. + ThreadIDNew string `bun:"type:CHAR(26),nullzero,notnull,default:'00000000000000000000000000'"` } // enumType is the type we (at least, should) use |
