Commit Graph

3356 Commits

Author SHA1 Message Date
Neil Macneale IV
f739f372e4 Merge pull request #9118 from dolthub/macneale4/auto-gc-archive
Enable the GC process to write to archive files
2025-04-22 15:16:39 -07:00
Neil Macneale IV
3c026b869c go mod tidy 2025-04-18 14:55:08 -07:00
Neil Macneale IV
4eddee6530 remove extra brackets 2025-04-16 13:42:34 -07:00
Neil Macneale IV
8824120bba Add the --archive-level option to the cli 2025-04-16 12:30:42 -07:00
Jason Fulghum
a7dcd7d311 PR Feedback 2025-04-10 11:26:41 -07:00
Jason Fulghum
0ecf245ac3 Adding new copy-tags command 2025-04-10 11:26:31 -07:00
Neil Macneale IV
728b7c17b9 Merge pull request #9095 from dolthub/macneale4/archive-purge
Add the archive --purge flag to clean up table files
2025-04-10 10:18:35 -07:00
Neil Macneale IV
cff2fd3957 find is different on mac and linux 2025-04-08 16:35:55 -07:00
Neil Macneale IV
838a3327bf Merge pull request #9075 from dolthub/macneale4/amend
Don't alter branch head outside of transactions when amending a commit
2025-04-08 15:49:15 -07:00
Neil Macneale IV
8d2ffa595f Add the archive --purge flag to clean up table files 2025-04-08 15:29:06 -07:00
Neil Macneale IV
dc16683143 A more non-deterministic test for --amend 2025-04-07 15:43:45 -07:00
Neil Macneale IV
297f42cdb5 Add basic interleaved test for --amend 2025-04-07 15:25:32 -07:00
Nick Tobey
baac6bed5c Merge remote-tracking branch 'origin/main' into nicktobey/toast 2025-04-02 14:07:05 -07:00
Aaron Son
a20f97458a go: sqle/dprocedures: Implement operational inspection hook, dolt_thread_dump stored procedure. 2025-04-01 14:22:23 -07:00
Nick Tobey
0577cf21e1 GMS Bump 2025-03-31 23:04:31 -07:00
Nick Tobey
f7f530a401 Update auto_increment.bats tests to account for the fact that setup now commits the initial empty table. 2025-03-31 18:32:05 -07:00
Nick Tobey
c713343cec Normalize AUTO_INCREMENT values in tables.
When the AUTO_INCREMENT field is unset, the value of the next AUTO_INCREMENT column defaults to 1. Thus, setting it to 0 and 1 produce identical behavior.

By normalizing the value as its written, we ensure that both operations produce the same table hash as if the value was unset.
2025-03-31 16:56:28 -07:00
Aaron Son
906ff580b0 go: cmd/dolt: commands/sql: Fix the displayed query time in interactive mode after a query completes.
The migration to subcontexts of the Queryist sql.Context meant that
we lost the default setting of the query start time. Restore it
in the sql command implementation itself.
2025-03-31 11:50:25 -07:00
Aaron Son
ac439ac8f2 go/.../dprocedures,integration-tests/{bats,go-sql-server-driver}: Add simple smoke tests to assert that valctx is enabled for these test suites. 2025-03-30 11:48:14 -07:00
Aaron Son
5ebc57e0ab go,integration-tests/{bats,go-sql-server-driver}: Implement GC sesssion lifecycle validation at the storage layer. Enable checks for bats and go sql server integration tests.
This makes *NomsBlockStore check the incoming Context object to ensure that it
itself has been invovled in the appropriate GC lifecycle callbacks.

It fixes a problem with statspro.AnalyzeTable, where the GC lifecycle callbacks
happened more than once for a single session.

It fixes some callsites to appropriately make the GC lifecycle callbacks,
including LateBindingQueryist opening one session command for the whole
lifetime of the returned sql.Context.
2025-03-28 18:06:27 -07:00
Nick Tobey
9b9ba21e69 Restore accidentally removed bats tests. 2025-03-28 14:29:03 -07:00
Nick Tobey
1b20b2c343 Merge remote-tracking branch 'origin/main' into nicktobey/toast 2025-03-27 16:31:29 -07:00
Aaron Son
2d67459aee Update integration-tests/go-sql-server-driver/sql_server_max_conns_test.go
Co-authored-by: Neil Macneale IV <46170177+macneale4@users.noreply.github.com>
2025-03-27 14:28:03 -07:00
Aaron Son
9773be16d1 Merge remote-tracking branch 'origin/main' into aaron/max-connections-tests-go-sql-server-driver 2025-03-25 17:14:30 -07:00
Nick Tobey
4442d304aa Merge remote-tracking branch 'origin/main' into nicktobey/toast 2025-03-25 17:03:27 -07:00
Neil Macneale IV
f1135ce1e0 Merge pull request #9028 from dolthub/macneale4-a127ac54
Test new connection configuration in vitess
2025-03-25 16:28:29 -07:00
Neil Macneale IV
9e0e0a2f3f PR Feedback 2025-03-25 15:33:42 -07:00
Aaron Son
bcfe25718a integration-tests/go-sql-server-driver: Run max conns tests in parallel. 2025-03-25 14:35:23 -07:00
Aaron Son
decdc9ee4c integration-tests/bats: sql-server.bats: Remove redundant max-connections behavior tests. 2025-03-25 09:56:08 -07:00
Aaron Son
2ff00c2afc integration-tests/go-sql-server-driver: Implement max-connections testing to go-sql-server-driver. 2025-03-25 09:56:05 -07:00
Aaron Son
89b1513ec2 Merge pull request #8997 from dolthub/aaron/go-integration-tests-parallel
[no-release-notes] integration-tests/go-sql-server-driver: Add dynamic port allocation and run these tests in parallel.
2025-03-25 09:43:53 -07:00
Nick Tobey
5574e8ba9a Prefer sql.Unwrap over explicit casts. 2025-03-24 22:03:14 -07:00
Neil Macneale IV
24c4e7ca20 Make the last timeout test more reliable 2025-03-24 21:02:14 -07:00
Neil Macneale IV
aacde670b0 Test the --max-connections-timeout has an effect 2025-03-24 21:02:14 -07:00
Neil Macneale IV
ae18118f31 Get max timeout for waiting connections all the way through 2025-03-24 21:02:14 -07:00
Neil Macneale IV
27ff3beaac Add max connections timeout 2025-03-24 21:02:14 -07:00
Neil Macneale IV
11dcccd1cc Add more tests for max cons and back log 2025-03-24 21:02:14 -07:00
Neil Macneale IV
66180b5623 Reliable first test for max-conns and back-log 2025-03-24 21:02:14 -07:00
Neil Macneale IV
309ba1d3d4 Test the --back-log value gets written to config.yaml 2025-03-24 21:02:14 -07:00
Aaron Son
fdb00a3853 integration-tests/go-sql-server-driver: PR feedback. 2025-03-24 17:52:53 -07:00
Aaron Son
7bde4e7198 integration-tests/go-sql-server-driver: Improve the robustness of TestConcurrentGC a bit. 2025-03-24 14:27:46 -07:00
Neil Macneale IV
f5cae4ca38 Merge pull request #8995 from dolthub/macneale4/archive-hot-convert
[no-release-notes] archive hot convert from snappy to zStd compressed chunks
2025-03-21 09:48:23 -07:00
Neil Macneale IV
1d8aacd782 Revert "Randomly disable archive STREAMER for each test"
This reverts commit ddf9cf4712.
2025-03-20 17:38:59 -07:00
Maximilian Hoffman
b8b2ff1c99 [stats] Rewrite stat management to use single threaded event loop (#8815)
* [stats] event loop

* more progress

* basic scheduler test working

* analyze

* add/drop hooks

* gc

* delete an alter

* drop index and table

* fix other tests

* branch management

* starter for kv

* gc and refactor maintanance

* fix bucket doubling

* delete log

* better bucket counting

* test for disk round trip

* more prolly stats gc tests

* rotate backing stats db

* progress towards swapping old for new, deleting old code

* fix gc bucket overflow

* test for gc overflow

* org and closers

* save progress update

* finally get first two bats running

* startup bound hash issue

* rewrite GC to be synchronous, fix more bugs

* fix session freshness

* fix branch gc

* cache writes and gc are serialized

* fix gc/branch update dropped hashes

* fix gc race, doubling race, jobs race

* fix more races

* docs

* convert bats to script tests

* more tests, purge/stop

* validate

* docs

* some PR cleanup

* more cleanup

* stash for pull

* fix bucket hash conflicts

* Fix more collection bugs.

* bump, timer proc

* more test fixes

* cache bats changes

* Another deadlock

* delete comment

* fmt

* no read replica stats

* fix plan tests

* branch qualified analyze fix

* [no-release-notes] go: statspro/jobqueue: Create a SerialQueue, which can perform asynchronous work on a worker thread.

* go: statspro/jobqueue: A bit of cleanup, fix a flakey test.

* rewrite with GDQ

* prog

* tests run

* fix info and storage

* outline for gc impl

* fix tests and races

* bump

* better error and panic management

* better start/stop/wait

* Add rate limiting

* gc ticker

* docs

* doc

* test prog

* fix more tests

* finish up listener tests

* add comments

* gc concurrency

* enginetests and statspro tests passing

* simplify listeners

* bats progress

* small edits

* tests progress

* bats are running

* fmt

* build

* edits

* fix interface

* fix build

* stats alternate index types

* fix mem test

* build

* fix more tests

* fmt

* more fmt

* copyright

* license

* fix races

* syntax error

* fix windows path

* nil mcv panic

* fix test races

* bump def job interval to 30ms

* deterministic tests

* more tests

* TEMP COMMIT: valctx plus some other stuff...

* shorter concurrency tests

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

* nondeterministic test

* try to make queue tests less racy

* missed one start

* stats granular session locks

* simplify a little

* try to avoid serialq test deadlock

* try to fix flakes

* more races

* bump

* another race

* cleanup

* more cleanup

* revert ctx validation

* most zach comments

* more comments

* more race

* bump

* more race

* bump

* schemas

* skip windows racees

* standardize server config init, use background threads management

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

* default stats noop

* threads management improvements

* undo change

* move stats initialization back to engine

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

---------

Co-authored-by: Aaron Son <aaron@dolthub.com>
Co-authored-by: max-hoffman <max-hoffman@users.noreply.github.com>
2025-03-20 15:56:48 -07:00
John Bampton
977064fc0d Fix spelling 2025-03-19 03:48:45 +10:00
Aaron Son
da6ddc205f integration-tests/go-sql-server-driver: Give some replication tests a bit longer grace period to replicate synchronously. 2025-03-17 18:08:23 -07:00
Aaron Son
14f5157b03 [no-release-notes] integration-tests/go-sql-server-driver: Add dynamic port allocation and run these tests in parallel.
Takes local runtime from 500+s to ~100s.
2025-03-17 17:48:33 -07:00
Neil Macneale IV
badd8cb180 missed changes to the archive-aws tests now fixed 2025-03-17 16:18:23 -07:00
Neil Macneale IV
d12f777e5c Tests for mixed format pushes and fetches 2025-03-17 15:27:02 -07:00
Neil Macneale IV
6e3fe5ff47 Move the archive test data directory 2025-03-17 15:27:02 -07:00