summaryrefslogtreecommitdiff
path: root/cmd
AgeCommit message (Collapse)AuthorFiles
2022-09-04[feature] Set default header and avatar for API accounts to GtS ones (#799)Libravatar tobi2
* validate web-asset-base-dir * move default icons into converter * always ensure avatar + header on api accounts * update tests * add default header * don't return error from web module creation anymore * tidy a bit * use pngs for default avatars rather than svgs
2022-09-02[performance] use GetAccountByUsernameDomain() for local account lookups to ↵Libravatar kim1
rely on cache (#793) Signed-off-by: kim <grufwub@gmail.com> Signed-off-by: kim <grufwub@gmail.com>
2022-08-15[chore] Update bun / sqlite versions; update gtsmodels (#754)Libravatar tobi1
* upstep bun and sqlite versions * allow specific columns to be updated in the db * only update necessary columns for user * bit tidier * only update necessary fields of media_attachment * only update relevant instance fields * update tests * update only specific account columns * use bool pointers on gtsmodels includes attachment, status, account, user * update columns more selectively * test all default fields on new account insert * updating remaining bools on gtsmodels * initialize pointer fields when extracting AP emoji * copy bools properly * add copyBoolPtr convenience function + test it * initialize false bool ptrs a bit more neatly
2022-07-19[chore] use our own logging implementation (#716)Libravatar kim4
* first commit Signed-off-by: kim <grufwub@gmail.com> * replace logging with our own log library Signed-off-by: kim <grufwub@gmail.com> * fix imports Signed-off-by: kim <grufwub@gmail.com> * fix log imports Signed-off-by: kim <grufwub@gmail.com> * add license text Signed-off-by: kim <grufwub@gmail.com> * fix package import cycle between config and log package Signed-off-by: kim <grufwub@gmail.com> * fix empty kv.Fields{} being passed to WithFields() Signed-off-by: kim <grufwub@gmail.com> * fix uses of log.WithFields() with whitespace issues and empty slices Signed-off-by: kim <grufwub@gmail.com> * *linter related grumbling* Signed-off-by: kim <grufwub@gmail.com> * gofmt the codebase! also fix more log.WithFields() formatting issues Signed-off-by: kim <grufwub@gmail.com> * update testrig code to match new changes Signed-off-by: kim <grufwub@gmail.com> * fix error wrapping in non fmt.Errorf function Signed-off-by: kim <grufwub@gmail.com> * add benchmarking of log.Caller() vs non-cached Signed-off-by: kim <grufwub@gmail.com> * fix syslog tests, add standard build tags to test runner to ensure consistency Signed-off-by: kim <grufwub@gmail.com> * make syslog tests more robust Signed-off-by: kim <grufwub@gmail.com> * fix caller depth arithmatic (is that how you spell it?) Signed-off-by: kim <grufwub@gmail.com> * update to use unkeyed fields in kv.Field{} instances Signed-off-by: kim <grufwub@gmail.com> * update go-kv library Signed-off-by: kim <grufwub@gmail.com> * update libraries list Signed-off-by: kim <grufwub@gmail.com> * fuck you linter get nerfed Signed-off-by: kim <grufwub@gmail.com> Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
2022-07-03[feature] S3 support (#674)Libravatar Dominik Süß2
* feat: vendor minio client * feat: introduce storage package with s3 support * feat: serve s3 files directly this saves a lot of bandwith as the files are fetched from the object store directly * fix: use explicit local storage in tests * feat: integrate s3 storage with the main server * fix: add s3 config to cli tests * docs: explicitly set values in example config also adds license header to the storage package * fix: use better http status code on s3 redirect HTTP 302 Found is the best fit, as it signifies that the resource requested was found but not under its presumed URL 307/TemporaryRedirect would mean that this resource is usually located here, not in this case 303/SeeOther indicates that the redirection does not link to the requested resource but to another page * refactor: use context in storage driver interface
2022-06-11[chore] Webfinger rework (#627)Libravatar tobi1
* move finger to dereferencer * totally break GetRemoteAccount * start reworking finger func a bit * start reworking getRemoteAccount a bit * move mention parts to namestring * rework webfingerget * use util function to extract webfinger parts * use accountDomain * rework finger again, final form * just a real nasty commit, the worst * remove refresh from account * use new ASRepToAccount signature * fix incorrect debug call * fix for new getRemoteAccount * rework GetRemoteAccount * start updating tests to remove repetition * break a lot of tests Move shared test logic into the testrig, rather than having it scattered all over the place. This allows us to just mock the transport controller once, and have all tests use it (unless they need not to for some other reason). * fix up tests to use main mock httpclient * webfinger only if necessary * cheeky linting with the lads * update mentionName regex recognize instance accounts * don't finger instance accounts * test webfinger part extraction * increase default worker count to 4 per cpu * don't repeat regex parsing * final search for discovered accountDomain * be more permissive in namestring lookup * add more extraction tests * simplify GetParseMentionFunc * skip long search if local account * fix broken test
2022-06-11[feature] Add `created_at` and `error_description` to `/oauth/token` ↵Libravatar tobi2
endpoint (#645) * start fiddling about with oauth server * start returning more helpful errors from oauth * test helpful(ish) token errors * add missing license header
2022-06-08[feature] More consistent API error handling (#637)Libravatar tobi2
* update templates * start reworking api error handling * update template * return AP status at web endpoint if negotiated * start making api error handling much more consistent * update account endpoints to new error handling * use new api error handling in admin endpoints * go fmt ./... * use api error logic in app * use generic error handling in auth * don't export generic error handler * don't defer clearing session * user nicer error handling on oidc callback handler * tidy up the sign in handler * tidy up the token handler * use nicer error handling in blocksget * auth emojis endpoint * fix up remaining api endpoints * fix whoopsie during login flow * regenerate swagger docs * change http error logging to debug
2022-05-30[chore] Global server configuration overhaul (#575)Libravatar kim14
* move config flag names and usage to config package, rewrite config package to use global Configuration{} struct Signed-off-by: kim <grufwub@gmail.com> * improved code comment Signed-off-by: kim <grufwub@gmail.com> * linter Signed-off-by: kim <grufwub@gmail.com> * fix unmarshaling Signed-off-by: kim <grufwub@gmail.com> * remove kim's custom go compiler changes Signed-off-by: kim <grufwub@gmail.com> * generate setter and flag-name functions, implement these in codebase Signed-off-by: kim <grufwub@gmail.com> * update deps Signed-off-by: kim <grufwub@gmail.com> * small change Signed-off-by: kim <grufwub@gmail.com> * appease the linter... Signed-off-by: kim <grufwub@gmail.com> * move configuration into ConfigState structure, ensure reloading to/from viper settings to keep in sync Signed-off-by: kim <grufwub@gmail.com> * lint Signed-off-by: kim <grufwub@gmail.com> * update code comments Signed-off-by: kim <grufwub@gmail.com> * fix merge issue Signed-off-by: kim <grufwub@gmail.com> * fix merge issue Signed-off-by: kim <grufwub@gmail.com> * improved version string (removes time + go version) Signed-off-by: kim <grufwub@gmail.com> * fix version string build to pass test script + consolidate logic in func Signed-off-by: kim <grufwub@gmail.com> * add license text, update config.Defaults comment Signed-off-by: kim <grufwub@gmail.com> * add license text to generated config helpers file Signed-off-by: kim <grufwub@gmail.com> * defer unlock on config.Set___(), to ensure unlocked on panic Signed-off-by: kim <grufwub@gmail.com> * make it more obvious which cmd flags are being attached Signed-off-by: kim <grufwub@gmail.com>
2022-05-16[feature] Basic config validation (#562)Libravatar tobi5
* add optional config validation * clarify that host and protocol are required * add validation for host and protocol * pass prerunArgs as a struct (validate by default)
2022-05-15[feature] Unused avatar and header cleanup (#574)Libravatar tobi1
* rename + tidy up remote pruning * fix media attachment account join see https://bun.uptrace.dev/guide/golang-orm.html#table-relationships * update logging to new function name * add get avatars and headers to bun * add pruneallmeta function * don't set uncached since we're deleting anyway * fix totalPruned being in wrong place * test pruning meta * go fmt ./... * rename mediaprune * add meta pruning to routine mediaprune * tidy up cleanup job scheduling * rename adminmediaremoteprune * update mediacleanup to use renamed prune func * update swagger docs a little bit * reuse cancel + context
2022-05-15[security] transport.Controller{} and transport.Transport{} security and ↵Libravatar kim2
performance improvements (#564) * cache transports in controller by privkey-generated pubkey, add retry logic to transport requests Signed-off-by: kim <grufwub@gmail.com> * update code comments, defer mutex unlocks Signed-off-by: kim <grufwub@gmail.com> * add count to 'performing request' log message Signed-off-by: kim <grufwub@gmail.com> * reduce repeated conversions of same url.URL object Signed-off-by: kim <grufwub@gmail.com> * move worker.Worker to concurrency subpackage, add WorkQueue type, limit transport http client use by WorkQueue Signed-off-by: kim <grufwub@gmail.com> * fix security advisories regarding max outgoing conns, max rsp body size - implemented by a new httpclient.Client{} that wraps an underlying client with a queue to limit connections, and limit reader wrapping a response body with a configured maximum size - update pub.HttpClient args passed around to be this new httpclient.Client{} Signed-off-by: kim <grufwub@gmail.com> * add httpclient tests, move ip validation to separate package + change mechanism Signed-off-by: kim <grufwub@gmail.com> * fix merge conflicts Signed-off-by: kim <grufwub@gmail.com> * use singular mutex in transport rather than separate signer mus Signed-off-by: kim <grufwub@gmail.com> * improved useragent string Signed-off-by: kim <grufwub@gmail.com> * add note regarding missing test Signed-off-by: kim <grufwub@gmail.com> * remove useragent field from transport (instead store in controller) Signed-off-by: kim <grufwub@gmail.com> * shutup linter Signed-off-by: kim <grufwub@gmail.com> * reset other signing headers on each loop iteration Signed-off-by: kim <grufwub@gmail.com> * respect request ctx during retry-backoff sleep period Signed-off-by: kim <grufwub@gmail.com> * use external pkg with docs explaining performance "hack" Signed-off-by: kim <grufwub@gmail.com> * use http package constants instead of string method literals Signed-off-by: kim <grufwub@gmail.com> * add license file headers Signed-off-by: kim <grufwub@gmail.com> * update code comment to match new func names Signed-off-by: kim <grufwub@gmail.com> * updates to user-agent string Signed-off-by: kim <grufwub@gmail.com> * update signed testrig models to fit with new transport logic (instead uses separate signer now) Signed-off-by: kim <grufwub@gmail.com> * fuck you linter Signed-off-by: kim <grufwub@gmail.com>
2022-05-03[chore] Move initial MediaRemotePrune to cmd.server function (#536)Libravatar tobi1
* move initial prune up to the server start function * don't do initial prune when starting media manager * add admin/mediaremoteprune in processor
2022-04-28replace async client API / federator msg processing with worker pools (#497)Libravatar kim2
* replace async client API / federator msg processing with worker pools * appease our lord-and-saviour, the linter
2022-04-18[feature] Add log-db-queries config option (#465)Libravatar tobi2
2022-04-15[feature] Web profile pages for accounts (#449)Libravatar tobi2
* add default avatars * allow webModule to error * return errWithCode from account get * add AccountGetLocalByUsername * check nil requesting account * add timestampShort function for just month/year * move loading logic to New + add default avatars * add profile page view * update swagger docs * add excludeReblogs to GetAccountStatuses * ignore casing when selecting local account by username * appropriate redirects * css fiddling * add 'about' heading * adjust thread page to work with routing * return AP representation if requested + authorized * simplify auth check * go fmt * golangci-lint ignore math/rand
2022-04-02[chore] Update Go version to 1.18 (#444)Libravatar tobi1
* linting with new golangci-lint version * update go to 1.18 * bump versions in drone.yml * use new runtime/debug package for version info * remove Commit build flag from goreleaser * remove mock commit + version from build script * go fmt * add dummy version env flag to test container * install git in golang container for testing * only set versionString if Version is defined
2022-03-15[performance] Add dereference shortcuts to avoid making http calls to self ↵Libravatar tobi1
(#430) * update transport (controller) to allow shortcuts * go fmt * expose underlying sig transport to allow test sigs
2022-03-07[feature] Clean up/uncache remote media (#407)Libravatar tobi2
* Add whereNotEmptyAndNotNull * Add GetRemoteOlderThanDays * Add GetRemoteOlderThanDays * Add PruneRemote to Manager interface * Start implementing PruneRemote * add new attachment + status to tests * fix up and test GetRemoteOlderThan * fix bad import * PruneRemote: return number pruned * add Cached column to mediaattachment * update + test pruneRemote * update mediaTest * use Cached column * upstep bun to latest version * embed structs in mediaAttachment * migrate mediaAttachment to new format * don't default cached to true * select only remote media * update db dependencies * step bun back to last working version * update pruneRemote to use Cached field * fix storage path of test attachments * add recache logic to manager * fix trimmed aspect ratio * test prune and recache * return errwithcode * tidy up different paths for emoji vs attachment * fix incorrect thumbnail type being stored * expose TransportController to media processor * implement tee-ing recached content * add thoughts of dog to test fedi attachments * test get remote files * add comment on PruneRemote * add postData cleanup to recache * test thumbnail fetching * add incredible diagram * go mod tidy * buffer pipes for recache streaming * test for client stops reading after 1kb * add media-remote-cache-days to config * add cron package * wrap logrus so it's available to cron * start and stop cron jobs gracefully
2022-02-08go fmtLibravatar tsmethurst1
2022-02-08merge fixupLibravatar tsmethurst1
2022-02-08Merge branch 'main' into media_refactorLibravatar tsmethurst1
2022-02-08add comment explaining store.lockLibravatar tsmethurst1
2022-02-05[feature] Rework timeline code to make it useful for more than just statuses ↵Libravatar tobi1
(#373) * add preparable and timelineable interfaces * initialize timeline manager within the processor * generic renaming * move status-specific timeline logic into the processor * refactor timeline to make it useful for more than statuses
2022-01-29put store lock file in base of storageLibravatar tsmethurst1
2022-01-10test the media manager a bit, add shutdown logicLibravatar tsmethurst2
2022-01-09tests are passing, but there's still much to be doneLibravatar tsmethurst1
2021-12-28more refactoring, media handler => managerLibravatar tsmethurst1
2021-12-20Extend license notices to 2022 (#354)Libravatar tobi17
2021-12-20Remove unnecessary storage config variables (#344)Libravatar tobi3
* rewire config to not use extraneous serve vars * rename 'file' to 'local' for consistency * use Type and Size again
2021-12-12Add optional syslog logrus hook (#343)Libravatar tobi3
* add optional syslog logrus hook * document syslog
2021-12-07Implement Cobra CLI tooling, Viper config tooling (#336)Libravatar tobi28
* start pulling out + replacing urfave and config * replace many many instances of config * move more stuff => viper * properly remove urfave * move some flags to root command * add testrig commands to root * alias config file keys * start adding cli parsing tests * reorder viper init * remove config path alias * fmt * change config file keys to non-nested * we're more or less in business now * tidy up the common func * go fmt * get tests passing again * add note about the cliparsing tests * reorganize * update docs with changes * structure cmd dir better * rename + move some files around * fix dangling comma
2021-11-24Fix broken defaults and broken flags (#314)v0.1.1Libravatar Forest Johnson6
* start with a default config, not an empty config. * some data structures were present on Empty config but not Default config * the monkey patched CLIContext is working * remove print debugging log * make the behaviour of the flags consistent across all data types Conflicts: internal/config/config.go * try to fix accidentally broken test
2021-11-22Use IPv6 doc prefix for docs (#324)Libravatar Mara Sophie Grosch1
IPv6 has a dedicated prefix for documentation and examples, this commit makes use of it for BindAddress config flag docs and examples.
2021-11-22add bindAddress configuration option (#320)Libravatar tobi1
* add bindAddress configuration option * clarify that bindAddress can be a hostname
2021-11-22Enable stricter linting with golangci-lint (#316)Libravatar tobi1
* update golangci-lint * add golangci config file w/ more linters * correct issues flagged by stricter linters * add more generous timeout for golangci-lint * add some style + formatting guidelines * move timeout to config file * go fmt
2021-10-31smtp + email confirmation (#285)Libravatar tobi2
* add smtp configuration * add email confirm + reset templates * add email sender to testrig * flesh out the email sender interface * go fmt * golint * update from field with more clarity * tidy up the email formatting * fix tests * add email sender to processor * tidy client api processing a bit * further tidying in fromClientAPI * pin new account to user * send msg to processor on new account creation * generate confirm email uri * remove emailer from account processor again * add processCreateAccountFromClientAPI * move emailer accountprocessor => userprocessor * add email sender to user processor * SendConfirmEmail function * add noop email sender * use noop email sender in tests * only assemble message if callback is not nil * use noop email sender if no smtp host is defined * minify email html before sending * fix wrong email address * email confirm test * fmt * serve web hndler * add email confirm handler * init test log properly on testrig * log emails that *would* have been sent * go fmt ./... * unexport confirm email handler * updatedAt * test confirm email function * don't allow tokens older than 7 days * change error message a bit * add basic smtp docs * add a few more snippets * typo * add email sender to outbox tests * don't use dutch wikipedia link * don't minify email html
2021-10-11reference global logrus (#274)Libravatar R. Aidan Campbell1
* reference logrus' global logger instead of passing and storing a logger reference everywhere * always directly use global logrus logger instead of referencing an instance * test suites should also directly use the global logrus logger * rename gin logging function to clarify that it's middleware * correct comments which erroneously referenced removed logger parameter * setting log level for tests now uses logrus' exported type instead of the string value, to guarantee error isn't possible
2021-09-30Golint (#255)Libravatar tobi1
2021-09-09Import export (#194)Libravatar tobi1
* start with export/import code * messing about with decoding/encoding * some more fiddling * stuff is WORKING * working pretty alright! * go fmt * fix up tests, add docs * start backup/restore doc * tweaks * credits * update advancedVisibility settings * update bun library -> v1.0.4 Signed-off-by: kim (grufwub) <grufwub@gmail.com> * update oauth library -> v4.3.1-SSB Signed-off-by: kim (grufwub) <grufwub@gmail.com> * handle oauth token scope, fix user.SigninCount + token.UserID Signed-off-by: kim (grufwub) <grufwub@gmail.com> * update oauth library --> v4.3.2-SSB Signed-off-by: kim (grufwub) <grufwub@gmail.com> * update sqlite library -> v1.13.0 Signed-off-by: kim (grufwub) <grufwub@gmail.com> * review changes * start with export/import code * messing about with decoding/encoding * some more fiddling * stuff is WORKING * working pretty alright! * go fmt * fix up tests, add docs * start backup/restore doc * tweaks * credits * update advancedVisibility settings * review changes Co-authored-by: kim (grufwub) <grufwub@gmail.com> Co-authored-by: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>
2021-07-31Swagger (#124)Libravatar Tobi Smethurst1
* start experimenting with swagger documentation * further adventures in swagger * do a few more api paths * account paths documented * go fmt * fix up some models * bit o lintin'
2021-07-31Password change (#123)Libravatar Tobi Smethurst1
* add password change command * document cli commands * go fmt
2021-07-26add trusted proxy for parsing client IPs (#115)Libravatar Tobi Smethurst1
2021-07-24allow custom ports for webserver and leLibravatar tsmethurst2
2021-07-23Oidc (#109)Libravatar Tobi Smethurst16
* add oidc config * inching forward with oidc idp * lil webfingy fix * bit more progress * further oidc * oidc now working * document dex config * replace broken images * add additional credits * tiny doc update * update * add oidc config * inching forward with oidc idp * bit more progress * further oidc * oidc now working * document dex config * replace broken images * add additional credits * tiny doc update * update * document * docs + comments
2021-07-19allow different host + accountDomain (#103)Libravatar Tobi Smethurst1
* allow different host + accountDomain * use accountDomain in tags
2021-07-19Db tls (#102)Libravatar Tobi Smethurst1
* go mod tidy * complete example config * add tls support for db connection * add certpool to tlsConfig * add some lil docker scripts
2021-06-28put version in binary properly (#73)Libravatar Tobi Smethurst1
Addresses #71 : Set version on the CLI framework. Add a build.sh script that injects variables into the build tooling using git and a version file. Set version in config.
2021-06-21new styling for frontpage, update login and authorize templates (#46)Libravatar f0x521
* new styling for frontpage, update login and authorize templates * run go fmt * add AssetBaseDir to command flag parsing * untested: move landing page to it's own router * go fmt, fix typo * fix package, adapt to proper Route structure
2021-06-21Testrig fixes (#50)Libravatar Tobi Smethurst1
* testrig is runnable again * little fixes, add some more test models * address https://github.com/superseriousbusiness/gotosocial/issues/44