Commit Graph

93 Commits

Author SHA1 Message Date
coffeegoddd☕️✨
dcbda3b773 /go/performance/utils: add error info 2026-01-13 10:28:35 -08:00
efcking
1d8045c256 refactor: remove redundant variable declarations in for loops
Signed-off-by: efcking <efcking@outlook.com>
2025-11-19 12:34:23 +08:00
James Cor
01775bc5cb disable only_full_group_by in sysbench (#9721) 2025-08-21 14:26:45 -07:00
Neil Macneale IV
337c90246c Remove broken symlink (#9577) 2025-07-24 17:36:35 -07:00
Maximilian Hoffman
b8b2ff1c99 [stats] Rewrite stat management to use single threaded event loop (#8815)
* [stats] event loop

* more progress

* basic scheduler test working

* analyze

* add/drop hooks

* gc

* delete an alter

* drop index and table

* fix other tests

* branch management

* starter for kv

* gc and refactor maintanance

* fix bucket doubling

* delete log

* better bucket counting

* test for disk round trip

* more prolly stats gc tests

* rotate backing stats db

* progress towards swapping old for new, deleting old code

* fix gc bucket overflow

* test for gc overflow

* org and closers

* save progress update

* finally get first two bats running

* startup bound hash issue

* rewrite GC to be synchronous, fix more bugs

* fix session freshness

* fix branch gc

* cache writes and gc are serialized

* fix gc/branch update dropped hashes

* fix gc race, doubling race, jobs race

* fix more races

* docs

* convert bats to script tests

* more tests, purge/stop

* validate

* docs

* some PR cleanup

* more cleanup

* stash for pull

* fix bucket hash conflicts

* Fix more collection bugs.

* bump, timer proc

* more test fixes

* cache bats changes

* Another deadlock

* delete comment

* fmt

* no read replica stats

* fix plan tests

* branch qualified analyze fix

* [no-release-notes] go: statspro/jobqueue: Create a SerialQueue, which can perform asynchronous work on a worker thread.

* go: statspro/jobqueue: A bit of cleanup, fix a flakey test.

* rewrite with GDQ

* prog

* tests run

* fix info and storage

* outline for gc impl

* fix tests and races

* bump

* better error and panic management

* better start/stop/wait

* Add rate limiting

* gc ticker

* docs

* doc

* test prog

* fix more tests

* finish up listener tests

* add comments

* gc concurrency

* enginetests and statspro tests passing

* simplify listeners

* bats progress

* small edits

* tests progress

* bats are running

* fmt

* build

* edits

* fix interface

* fix build

* stats alternate index types

* fix mem test

* build

* fix more tests

* fmt

* more fmt

* copyright

* license

* fix races

* syntax error

* fix windows path

* nil mcv panic

* fix test races

* bump def job interval to 30ms

* deterministic tests

* more tests

* TEMP COMMIT: valctx plus some other stuff...

* shorter concurrency tests

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

* nondeterministic test

* try to make queue tests less racy

* missed one start

* stats granular session locks

* simplify a little

* try to avoid serialq test deadlock

* try to fix flakes

* more races

* bump

* another race

* cleanup

* more cleanup

* revert ctx validation

* most zach comments

* more comments

* more race

* bump

* more race

* bump

* schemas

* skip windows racees

* standardize server config init, use background threads management

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

* default stats noop

* threads management improvements

* undo change

* move stats initialization back to engine

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

---------

Co-authored-by: Aaron Son <aaron@dolthub.com>
Co-authored-by: max-hoffman <max-hoffman@users.noreply.github.com>
2025-03-20 15:56:48 -07:00
coffeegoddd
a814768ee1 /go/performance/utils/benchmark_runner: use different connection strings for postgres and doltgres 2024-11-18 11:11:42 -08:00
coffeegoddd
849e8d4287 /go/performance/utils/benchmark_runner/doltgres.go: remove commentted out code 2024-11-18 10:45:57 -08:00
coffeegoddd
a280d6f012 /go/performance/utils/benchmark_runner: fix for latest doltgres 2024-11-18 10:13:23 -08:00
coffeegoddd
eb4bc615f1 /go/performance/utils/benchmark_runner/doltgres.go: skip ping call in doltgres 2024-10-23 15:07:45 -07:00
Neil Macneale IV
49e0cb67e7 Reverts to some changes in PR 2024-08-20 15:49:23 -07:00
Dylan Varga
fed30c181d Fixed grammar typos re issue #8221 2024-08-09 22:24:19 -07:00
Zach Musgrave
f849cbead4 Re-enable doltgres sysbench scripts 2024-06-26 17:28:54 -07:00
Maximilian Hoffman
7aef8fcde1 [no-release-notes] tracing infra (#7783)
* tracing starter

* bump

* trace reads

* bump

* undo some unnecessary changes

* fix build

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

---------

Co-authored-by: max-hoffman <max-hoffman@users.noreply.github.com>
2024-06-26 11:59:46 -07:00
Maximilian Hoffman
7059d8abf9 [no-release-notes] need stats to run on dolt servers (#8055) 2024-06-24 13:40:36 -07:00
Maximilian Hoffman
0c4df2d519 [no-release-notes] Initialize sysbench statistics, for perf (#8015)
* [no-release-notes] Initialize sysbench statistics, for perf

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

* don't match doltgres

* tpcc stats exlcudes doltgres also, by default

---------

Co-authored-by: max-hoffman <max-hoffman@users.noreply.github.com>
2024-06-20 15:45:47 -07:00
coffeegoddd
dafa8b855e /go/performance/utils/dolt_builder/run.go: log build error 2024-06-10 12:49:45 -07:00
coffeegoddd
8f00fd0988 /{.github,go}: try fixing doltgres benchmark runner tests 2024-05-29 12:54:45 -07:00
coffeegoddd
91d00f2e1e /go/performance/utils/benchmark_runner: fix doltgres version command 2024-05-29 10:48:21 -07:00
Maximilian Hoffman
34c36139d1 [no-release-notes] redo how to set tpc-c testing statistics (#7806)
* [no-release-notes] redo how to set tpc-c testing statistics

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

* Remove old code

* don't collect stats for mysql

* tpcc uses non-standard port

---------

Co-authored-by: max-hoffman <max-hoffman@users.noreply.github.com>
2024-05-03 13:24:27 -07:00
luchenhan
1ecced743a chore: fix some function names
Signed-off-by: luchenhan <hanluchen@aliyun.com>
2024-04-29 17:02:11 +08:00
Maximilian Hoffman
b6b0261a47 [no-release-notes] TPC-C collect stats once before run (#7774) 2024-04-24 11:32:34 -07:00
jennifersp
8f7caa2e98 [no-release-notes] fix sysbench test cleanup (#7733) 2024-04-11 13:20:13 -07:00
jennifersp
498af25e3d [no-release-notes] skip failed test files for Doltgres sysbench benchmark (#7703) 2024-04-11 09:43:36 -07:00
Maximilian Hoffman
01bf4ad446 [no-release-notes] bump TPC-C stats interval (#7686) 2024-04-02 18:42:01 -07:00
Max Hoffman
48b0e0f574 [no-release-notes] Double TPC-C time again 2024-04-02 15:01:10 -07:00
Maximilian Hoffman
c59b019d11 [no-release-notes] more TPC-C, reduce variance (#7570) 2024-03-06 10:10:13 -08:00
Maximilian Hoffman
89271d78c2 [no-release-notes] TPC-C CI runs longer (#7567) 2024-03-04 16:27:38 -08:00
coffeegoddd
3263cbe920 /go/performance/utils/benchmark_runner/run_test.go: reduce testing time 2024-02-15 17:03:46 -08:00
coffeegoddd
3d0c9df515 /{.github,go}: fix benchmark_runner tests so that they test all use cases 2024-02-15 16:32:59 -08:00
coffeegoddd
20ab5d0e45 /go/performance/utils/dolt_builder: allow profile builds 2024-02-15 14:03:46 -08:00
coffeegoddd
c0e9d077b9 /go/performance/continuous_integration: update dockerfile to install postgres and doltgres for testing the benchmark runner 2024-02-14 16:01:23 -08:00
coffeegoddd
2275eba91f /go/performance/utils/benchmark_runner/{sysbench,tpcc}.go: fix output already set error 2024-02-14 15:18:25 -08:00
Maximilian Hoffman
ded3fdbb1b [no-release-notes] Stats on for TPC-C (#7489)
* [no-release-notes] Stats on for TPC-C

* normalize tpcc to one client

* tpcc tps multiplier

* add multiplier
2024-02-14 14:13:23 -08:00
coffeegoddd
f75b598c31 /go/performance/utils/benchmark_runner: fix tests 2024-02-14 10:48:24 -08:00
coffeegoddd
d70b0470a7 /go/performance/utils/benchmark_runner: done refactoring 2024-02-14 10:17:31 -08:00
coffeegoddd
ce21c0b610 /go/performance/utils/sysbench_runner: wip, all seems to work, need to fix json unmarshalling 2024-02-13 17:24:47 -08:00
coffeegoddd
aabdce4be9 /go/performance/utils/sysbench_runner: wip, tests working, runner and profile working, need tpcc now, then unmarshal stuff 2024-02-13 16:42:49 -08:00
coffeegoddd
8049429847 /go/performance/utils/sysbench_runner/results_test.go: get tests passing again 2024-02-13 15:10:48 -08:00
coffeegoddd
46bc6bacab /go/performance/utils/sysbench_runner: wip, need to test now 2024-02-13 14:57:40 -08:00
coffeegoddd
05bf46fb74 /go/performance/utils/sysbench_runner: wip, need to fix tests, then test it all 2024-02-13 14:39:12 -08:00
coffeegoddd
6b22c8da86 /go/performance/utils/sysbench_runner: wip, still grinding 2024-02-13 12:40:03 -08:00
coffeegoddd
53bdaeecad /go/performance/utils/sysbench_runner: wip, still jamming 2024-02-13 12:26:16 -08:00
coffeegoddd
ce120e9fd9 /go/performance/utils/sysbench_runner: wip, still grinding on refactor 2024-02-13 11:23:08 -08:00
coffeegoddd
25c78c33c0 /go/performance/utils/sysbench_runner: wip, working through config refactor 2024-02-12 17:23:35 -08:00
coffeegoddd
ffecc7b7cf /go/performance/utils/sysbench_runner: wip, remove old benchmark code 2024-02-12 15:38:05 -08:00
coffeegoddd
afcfdd56ed /go/performance/utils/sysbench_runner: wip, remove tpcc code 2024-02-12 15:36:13 -08:00
coffeegoddd
6065eab29c /go/performance/utils/{sysbench_runner,tpcc_runner}: wip, tpcc seems to work still 2024-02-12 15:30:45 -08:00
coffeegoddd
985238b423 /go/performance/utils/sysbench_runner: wip, refactoring tpcc, needs testing 2024-02-12 15:08:18 -08:00
coffeegoddd
5e066a5c93 /go/performance/utils/{sysbench_runner,tpcc_runner}: wip, refactoring tpcc stuff 2024-02-12 14:48:35 -08:00
coffeegoddd
1162d2c4c5 /go/performance/utils/sysbench_runner: wip, profiling seems to work 2024-02-12 13:29:07 -08:00