Commit Graph

266 Commits

Author SHA1 Message Date
Aaron Son dbd015fcc3 Fix some Dockerfiles used in CI to include libicu-dev. 2025-09-16 01:13:35 -07:00
Aaron Son 544f298739 Merge pull request #9726 from dolthub/aaron/auto-gc-for-dolt-sql
go: cmd/dolt: sql.go: Enable Auto GC when running `dolt sql`.
2025-09-01 15:48:34 +02:00
James Cor 5fa69b5a30 Optimize commit iterator (#9751) 2025-08-29 17:13:57 -07:00
Aaron Son 0510550e58 go/cmd/dolt: commands/sql: Enable Auto GC for dolt sql.
This allows things like bulk imports of mysqldumps to take advantage of auto-gc.

In this change, this setting is not currently configurable. It may represent a
performance regression for some workloads.
2025-08-24 20:50:32 -07:00
James Cor 01775bc5cb disable only_full_group_by in sysbench (#9721) 2025-08-21 14:26:45 -07:00
Aaron Son 07ac9e016a Merge pull request #9702 from dolthub/aaron/command-lifecycle
go: cmd/dolt: cli_context: Slightly rework CliContext and LateBinderyQueryist lifecycle.
2025-08-21 05:49:07 -07:00
Aaron Son 27218fee46 go: cmd/dolt: Move to closing the CliContext, instead of closing the response from Queryist QueryEngine().
This makes the lifecycle of the LateBinder resources more correct, since they
should typically live until the end of the process. In test contexts, in
particular, it was common to have one command see the closer != nil, close it,
and still receive the cached Queryist from the CliContext on later Command.Exec
invocations.
2025-08-19 07:02:12 -07:00
Aaron Son b871bd5b80 go: SysbenchDockerfile: Mirror the download tarball for sqlite.
Fixes broken Github actions based on recently implemented robot checks on the
sqlite server.
2025-08-19 01:32:12 -07:00
James Cor faeb388aad fix local sysbench scripts for ssl (#9650)
Co-authored-by: James Cor <james@dolthub.com>
2025-08-08 15:15:49 -07:00
Nick Tobey 6ae35a6bd8 Merge remote-tracking branch 'origin/aaron/clean-up-autoincrement-init-on-shutdown' into nicktobey/mmap 2025-07-29 14:24:49 -04:00
Nick Tobey b4403f0191 Update tests. 2025-07-25 10:32:28 -07:00
Neil Macneale IV 337c90246c Remove broken symlink (#9577) 2025-07-24 17:36:35 -07:00
Nick Tobey 93f980f9ed Update local benchamark scripts to match current server config. 2025-04-07 13:33:08 -07:00
Maximilian Hoffman b8b2ff1c99 [stats] Rewrite stat management to use single threaded event loop (#8815)
* [stats] event loop

* more progress

* basic scheduler test working

* analyze

* add/drop hooks

* gc

* delete an alter

* drop index and table

* fix other tests

* branch management

* starter for kv

* gc and refactor maintanance

* fix bucket doubling

* delete log

* better bucket counting

* test for disk round trip

* more prolly stats gc tests

* rotate backing stats db

* progress towards swapping old for new, deleting old code

* fix gc bucket overflow

* test for gc overflow

* org and closers

* save progress update

* finally get first two bats running

* startup bound hash issue

* rewrite GC to be synchronous, fix more bugs

* fix session freshness

* fix branch gc

* cache writes and gc are serialized

* fix gc/branch update dropped hashes

* fix gc race, doubling race, jobs race

* fix more races

* docs

* convert bats to script tests

* more tests, purge/stop

* validate

* docs

* some PR cleanup

* more cleanup

* stash for pull

* fix bucket hash conflicts

* Fix more collection bugs.

* bump, timer proc

* more test fixes

* cache bats changes

* Another deadlock

* delete comment

* fmt

* no read replica stats

* fix plan tests

* branch qualified analyze fix

* [no-release-notes] go: statspro/jobqueue: Create a SerialQueue, which can perform asynchronous work on a worker thread.

* go: statspro/jobqueue: A bit of cleanup, fix a flakey test.

* rewrite with GDQ

* prog

* tests run

* fix info and storage

* outline for gc impl

* fix tests and races

* bump

* better error and panic management

* better start/stop/wait

* Add rate limiting

* gc ticker

* docs

* doc

* test prog

* fix more tests

* finish up listener tests

* add comments

* gc concurrency

* enginetests and statspro tests passing

* simplify listeners

* bats progress

* small edits

* tests progress

* bats are running

* fmt

* build

* edits

* fix interface

* fix build

* stats alternate index types

* fix mem test

* build

* fix more tests

* fmt

* more fmt

* copyright

* license

* fix races

* syntax error

* fix windows path

* nil mcv panic

* fix test races

* bump def job interval to 30ms

* deterministic tests

* more tests

* TEMP COMMIT: valctx plus some other stuff...

* shorter concurrency tests

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

* nondeterministic test

* try to make queue tests less racy

* missed one start

* stats granular session locks

* simplify a little

* try to avoid serialq test deadlock

* try to fix flakes

* more races

* bump

* another race

* cleanup

* more cleanup

* revert ctx validation

* most zach comments

* more comments

* more race

* bump

* more race

* bump

* schemas

* skip windows racees

* standardize server config init, use background threads management

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

* default stats noop

* threads management improvements

* undo change

* move stats initialization back to engine

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

---------

Co-authored-by: Aaron Son <aaron@dolthub.com>
Co-authored-by: max-hoffman <max-hoffman@users.noreply.github.com>
2025-03-20 15:56:48 -07:00
Aaron Son 60cbf4726f go: cmd/dolt: sqlserver: Create a Config struct to encapsulate arguments that control sqlserver.ConfigureServices behavior.
Allow configuring the ProtocolListenerFactory through ConfigureServices.
2025-03-05 15:44:17 -08:00
Aaron Son 17ef5fb1c3 go: Bump to go 1.24. Fix some vet errors related to non-constant format strings. 2025-02-26 15:01:26 -08:00
Jason Fulghum f14376d773 pr feedback: removing SkipRootUserInitialization from ServerConfig interface and from YAML config 2025-01-14 17:52:32 -08:00
Neil Macneale IV da3c002682 Add the cwd to cliCtx and use it in server config resolution 2025-01-02 09:22:45 -08:00
Eng Zer Jun 745d520876 Replace cespare/xxhash with cespare/xxhash/v2 (#8698)
`github.com/cespare/xxhash/v2` is the latest version with bug fixes and
improvements.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2024-12-26 15:33:32 -06:00
Maximilian Hoffman 049dea8be1 [no-release-notes] write buf reuse bump (#8693)
* [no-release-notes] bump for cache charset id

* add sysbench tests

* fix build

* [no-release-notes] prototype for reusing write buffer

* bump

* bump

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

* bump

* bump

* bump

* bump

* bump

* bump

* bump

* bump

---------

Co-authored-by: max-hoffman <max-hoffman@users.noreply.github.com>
2024-12-20 17:13:08 -08:00
Maximilian Hoffman e71262c677 cache charset bump (#8691)
* [no-release-notes] bump for cache charset id

* add sysbench tests

* fix build

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

* bump

---------

Co-authored-by: max-hoffman <max-hoffman@users.noreply.github.com>
2024-12-19 11:18:08 -08:00
coffeegoddd 00732cb5e0 /{.github,go,integration-tests}: bump go 2024-12-02 17:05:37 -08:00
coffeegoddd a814768ee1 /go/performance/utils/benchmark_runner: use different connection strings for postgres and doltgres 2024-11-18 11:11:42 -08:00
coffeegoddd 849e8d4287 /go/performance/utils/benchmark_runner/doltgres.go: remove commentted out code 2024-11-18 10:45:57 -08:00
coffeegoddd a280d6f012 /go/performance/utils/benchmark_runner: fix for latest doltgres 2024-11-18 10:13:23 -08:00
coffeegoddd eb4bc615f1 /go/performance/utils/benchmark_runner/doltgres.go: skip ping call in doltgres 2024-10-23 15:07:45 -07:00
coffeegoddd 814453aeaf /go/performance/continuous_integration/SysbenchDockerfile: bump doltgres version tested in sysbench tester 2024-10-23 14:22:31 -07:00
Neil Macneale IV 34bd5c09b4 Merge remote-tracking branch 'origin/main' into spelling-3p 2024-08-20 17:34:15 -07:00
Neil Macneale IV 49e0cb67e7 Reverts to some changes in PR 2024-08-20 15:49:23 -07:00
Maximilian Hoffman adf405749c [no-release-notes] range select perf bump (#8241)
* [no-release-notes] range select perf bump

* bump, catalog function interface simplification

* bump

* bump
2024-08-13 17:21:12 -07:00
Dylan Varga fed30c181d Fixed grammar typos re issue #8221 2024-08-09 22:24:19 -07:00
Maximilian Hoffman 594fd2d2d7 [no-release-notes] another max1row iter (#8210)
* [no-release-notes] bump for another max1iter

* bug fix

* bump

* fix test build

* more test build

* more test build

* more test build

* bump

* bump
2024-08-06 13:05:13 -07:00
Zach Musgrave f849cbead4 Re-enable doltgres sysbench scripts 2024-06-26 17:28:54 -07:00
Maximilian Hoffman 7aef8fcde1 [no-release-notes] tracing infra (#7783)
* tracing starter

* bump

* trace reads

* bump

* undo some unnecessary changes

* fix build

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

---------

Co-authored-by: max-hoffman <max-hoffman@users.noreply.github.com>
2024-06-26 11:59:46 -07:00
Maximilian Hoffman 7059d8abf9 [no-release-notes] need stats to run on dolt servers (#8055) 2024-06-24 13:40:36 -07:00
Maximilian Hoffman 0c4df2d519 [no-release-notes] Initialize sysbench statistics, for perf (#8015)
* [no-release-notes] Initialize sysbench statistics, for perf

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

* don't match doltgres

* tpcc stats exlcudes doltgres also, by default

---------

Co-authored-by: max-hoffman <max-hoffman@users.noreply.github.com>
2024-06-20 15:45:47 -07:00
coffeegoddd dafa8b855e /go/performance/utils/dolt_builder/run.go: log build error 2024-06-10 12:49:45 -07:00
coffeegoddd ddf07da141 /go/performance/continuous_integration/SysbenchDockerfile: fix heredoc 2024-05-29 12:54:45 -07:00
coffeegoddd 8f00fd0988 /{.github,go}: try fixing doltgres benchmark runner tests 2024-05-29 12:54:45 -07:00
coffeegoddd 91d00f2e1e /go/performance/utils/benchmark_runner: fix doltgres version command 2024-05-29 10:48:21 -07:00
coffeegoddd 6a4e82f9a0 /go/performance/continuous_integration/SysbenchDockerfile: fix version command 2024-05-29 10:25:35 -07:00
coffeegoddd 3812b1de24 /go/performance/continuous_integration/SysbenchDockerfile: fix ci 2024-05-29 10:08:49 -07:00
Brian Hendriks 2e6b5a5452 formatting 2024-05-15 10:33:40 -07:00
Brian Hendriks b6cd726af3 move config 2024-05-15 10:27:16 -07:00
Maximilian Hoffman 34c36139d1 [no-release-notes] redo how to set tpc-c testing statistics (#7806)
* [no-release-notes] redo how to set tpc-c testing statistics

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

* Remove old code

* don't collect stats for mysql

* tpcc uses non-standard port

---------

Co-authored-by: max-hoffman <max-hoffman@users.noreply.github.com>
2024-05-03 13:24:27 -07:00
luchenhan 1ecced743a chore: fix some function names
Signed-off-by: luchenhan <hanluchen@aliyun.com>
2024-04-29 17:02:11 +08:00
Maximilian Hoffman b6b0261a47 [no-release-notes] TPC-C collect stats once before run (#7774) 2024-04-24 11:32:34 -07:00
momantech 73ac859452 chore: fix some comments
Signed-off-by: momantech <cuimoman@qq.com>
2024-04-17 17:26:23 +08:00
jennifersp 8f7caa2e98 [no-release-notes] fix sysbench test cleanup (#7733) 2024-04-11 13:20:13 -07:00
jennifersp 498af25e3d [no-release-notes] skip failed test files for Doltgres sysbench benchmark (#7703) 2024-04-11 09:43:36 -07:00