Commit Graph

2927 Commits

Author SHA1 Message Date
Neil Macneale IV 4eddee6530 remove extra brackets 2025-04-16 13:42:34 -07:00
Neil Macneale IV 8824120bba Add the --archive-level option to the cli 2025-04-16 12:30:42 -07:00
Jason Fulghum a7dcd7d311 PR Feedback 2025-04-10 11:26:41 -07:00
Jason Fulghum 0ecf245ac3 Adding new copy-tags command 2025-04-10 11:26:31 -07:00
Neil Macneale IV cff2fd3957 find is different on mac and linux 2025-04-08 16:35:55 -07:00
Neil Macneale IV 8d2ffa595f Add the archive --purge flag to clean up table files 2025-04-08 15:29:06 -07:00
Nick Tobey baac6bed5c Merge remote-tracking branch 'origin/main' into nicktobey/toast 2025-04-02 14:07:05 -07:00
Aaron Son a20f97458a go: sqle/dprocedures: Implement operational inspection hook, dolt_thread_dump stored procedure. 2025-04-01 14:22:23 -07:00
Nick Tobey 0577cf21e1 GMS Bump 2025-03-31 23:04:31 -07:00
Nick Tobey f7f530a401 Update auto_increment.bats tests to account for the fact that setup now commits the initial empty table. 2025-03-31 18:32:05 -07:00
Nick Tobey c713343cec Normalize AUTO_INCREMENT values in tables.
When the AUTO_INCREMENT field is unset, the value of the next AUTO_INCREMENT column defaults to 1. Thus, setting it to 0 and 1 produce identical behavior.

By normalizing the value as its written, we ensure that both operations produce the same table hash as if the value was unset.
2025-03-31 16:56:28 -07:00
Aaron Son 906ff580b0 go: cmd/dolt: commands/sql: Fix the displayed query time in interactive mode after a query completes.
The migration to subcontexts of the Queryist sql.Context meant that
we lost the default setting of the query start time. Restore it
in the sql command implementation itself.
2025-03-31 11:50:25 -07:00
Aaron Son ac439ac8f2 go/.../dprocedures,integration-tests/{bats,go-sql-server-driver}: Add simple smoke tests to assert that valctx is enabled for these test suites. 2025-03-30 11:48:14 -07:00
Aaron Son 5ebc57e0ab go,integration-tests/{bats,go-sql-server-driver}: Implement GC sesssion lifecycle validation at the storage layer. Enable checks for bats and go sql server integration tests.
This makes *NomsBlockStore check the incoming Context object to ensure that it
itself has been invovled in the appropriate GC lifecycle callbacks.

It fixes a problem with statspro.AnalyzeTable, where the GC lifecycle callbacks
happened more than once for a single session.

It fixes some callsites to appropriately make the GC lifecycle callbacks,
including LateBindingQueryist opening one session command for the whole
lifetime of the returned sql.Context.
2025-03-28 18:06:27 -07:00
Nick Tobey 9b9ba21e69 Restore accidentally removed bats tests. 2025-03-28 14:29:03 -07:00
Nick Tobey 1b20b2c343 Merge remote-tracking branch 'origin/main' into nicktobey/toast 2025-03-27 16:31:29 -07:00
Nick Tobey 4442d304aa Merge remote-tracking branch 'origin/main' into nicktobey/toast 2025-03-25 17:03:27 -07:00
Neil Macneale IV 9e0e0a2f3f PR Feedback 2025-03-25 15:33:42 -07:00
Aaron Son decdc9ee4c integration-tests/bats: sql-server.bats: Remove redundant max-connections behavior tests. 2025-03-25 09:56:08 -07:00
Nick Tobey 5574e8ba9a Prefer sql.Unwrap over explicit casts. 2025-03-24 22:03:14 -07:00
Neil Macneale IV 24c4e7ca20 Make the last timeout test more reliable 2025-03-24 21:02:14 -07:00
Neil Macneale IV aacde670b0 Test the --max-connections-timeout has an effect 2025-03-24 21:02:14 -07:00
Neil Macneale IV ae18118f31 Get max timeout for waiting connections all the way through 2025-03-24 21:02:14 -07:00
Neil Macneale IV 27ff3beaac Add max connections timeout 2025-03-24 21:02:14 -07:00
Neil Macneale IV 11dcccd1cc Add more tests for max cons and back log 2025-03-24 21:02:14 -07:00
Neil Macneale IV 66180b5623 Reliable first test for max-conns and back-log 2025-03-24 21:02:14 -07:00
Neil Macneale IV 309ba1d3d4 Test the --back-log value gets written to config.yaml 2025-03-24 21:02:14 -07:00
Neil Macneale IV f5cae4ca38 Merge pull request #8995 from dolthub/macneale4/archive-hot-convert
[no-release-notes] archive hot convert from snappy to zStd compressed chunks
2025-03-21 09:48:23 -07:00
Neil Macneale IV 1d8aacd782 Revert "Randomly disable archive STREAMER for each test"
This reverts commit ddf9cf4712.
2025-03-20 17:38:59 -07:00
Maximilian Hoffman b8b2ff1c99 [stats] Rewrite stat management to use single threaded event loop (#8815)
* [stats] event loop

* more progress

* basic scheduler test working

* analyze

* add/drop hooks

* gc

* delete an alter

* drop index and table

* fix other tests

* branch management

* starter for kv

* gc and refactor maintanance

* fix bucket doubling

* delete log

* better bucket counting

* test for disk round trip

* more prolly stats gc tests

* rotate backing stats db

* progress towards swapping old for new, deleting old code

* fix gc bucket overflow

* test for gc overflow

* org and closers

* save progress update

* finally get first two bats running

* startup bound hash issue

* rewrite GC to be synchronous, fix more bugs

* fix session freshness

* fix branch gc

* cache writes and gc are serialized

* fix gc/branch update dropped hashes

* fix gc race, doubling race, jobs race

* fix more races

* docs

* convert bats to script tests

* more tests, purge/stop

* validate

* docs

* some PR cleanup

* more cleanup

* stash for pull

* fix bucket hash conflicts

* Fix more collection bugs.

* bump, timer proc

* more test fixes

* cache bats changes

* Another deadlock

* delete comment

* fmt

* no read replica stats

* fix plan tests

* branch qualified analyze fix

* [no-release-notes] go: statspro/jobqueue: Create a SerialQueue, which can perform asynchronous work on a worker thread.

* go: statspro/jobqueue: A bit of cleanup, fix a flakey test.

* rewrite with GDQ

* prog

* tests run

* fix info and storage

* outline for gc impl

* fix tests and races

* bump

* better error and panic management

* better start/stop/wait

* Add rate limiting

* gc ticker

* docs

* doc

* test prog

* fix more tests

* finish up listener tests

* add comments

* gc concurrency

* enginetests and statspro tests passing

* simplify listeners

* bats progress

* small edits

* tests progress

* bats are running

* fmt

* build

* edits

* fix interface

* fix build

* stats alternate index types

* fix mem test

* build

* fix more tests

* fmt

* more fmt

* copyright

* license

* fix races

* syntax error

* fix windows path

* nil mcv panic

* fix test races

* bump def job interval to 30ms

* deterministic tests

* more tests

* TEMP COMMIT: valctx plus some other stuff...

* shorter concurrency tests

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

* nondeterministic test

* try to make queue tests less racy

* missed one start

* stats granular session locks

* simplify a little

* try to avoid serialq test deadlock

* try to fix flakes

* more races

* bump

* another race

* cleanup

* more cleanup

* revert ctx validation

* most zach comments

* more comments

* more race

* bump

* more race

* bump

* schemas

* skip windows racees

* standardize server config init, use background threads management

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

* default stats noop

* threads management improvements

* undo change

* move stats initialization back to engine

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

---------

Co-authored-by: Aaron Son <aaron@dolthub.com>
Co-authored-by: max-hoffman <max-hoffman@users.noreply.github.com>
2025-03-20 15:56:48 -07:00
John Bampton 977064fc0d Fix spelling 2025-03-19 03:48:45 +10:00
Neil Macneale IV badd8cb180 missed changes to the archive-aws tests now fixed 2025-03-17 16:18:23 -07:00
Neil Macneale IV d12f777e5c Tests for mixed format pushes and fetches 2025-03-17 15:27:02 -07:00
Neil Macneale IV 6e3fe5ff47 Move the archive test data directory 2025-03-17 15:27:02 -07:00
Neil Macneale IV ddf9cf4712 Randomly disable archive STREAMER for each test
In a regular test workflow, the DOLT_ARCHIVE_PULL_STREAMER env var is not
set. It is in place currently so devs can run lambdabats with
it for more testing.

localbs tests are don't play well with this env var, so disabling it
in those tests. TBD on if they will be updated.
2025-03-17 15:27:02 -07:00
Neil Macneale IV 5a405d8e50 Add archive test which would trip on broken dictionaries 2025-03-12 15:40:30 -07:00
Maximilian Hoffman 51eb05981d Sql debug cmd (#8920)
* debug command

* basic debugging working

* type

* bump

* basic bats

* fmt

* license

* fmt
2025-03-07 12:12:24 -08:00
Jason Fulghum 5d2c2bc547 Merge pull request #8890 from damanV5/feat/logging
feat:(logging) Added Log Format provision
2025-03-04 19:00:22 -08:00
Damandeep Singh 7ebe0288d7 chore: log_foramt functionality added as separate test case 2025-03-04 08:36:31 +05:30
Nick Tobey e8b0967aaa Report auto increment changes in dolt diff and dolt_diff_summary() 2025-02-28 16:01:37 -08:00
Aaron Son defa0bb155 [no-release-notes] integration-tests/bats: garbage_collection.bats: Fix flakiness stemming from times appear in ls output that we use for cmp.
These bats tests perform some GCs and inspect filesystem state to assert that
either nothing changed for a noop GC or that things actually did change for a
non-noop GC. When the inspect the filesystem state, the `ls` output includes
mtimes for the files and directories. `.dolt/stats` actually does, or at least
can, change across these runs, and so it is explicitly removed before the
comparison is done. That means the mtime on the `.dolt` directory itself
changes. These timestamps are at minute resolution, and so the tests normally
pass, but if the interactions happen in two separate minutes, they fail.

This change removes the directories themselves from the listing, leaving the
files all in place. This asserts that the files' sizes and mtimes haven't (or
have) changed across the `dolt gc` runs.
2025-02-28 10:24:48 -08:00
Damandeep Singh 2ebec3f2d3 simplified test 2025-02-28 11:47:19 +05:30
Damandeep Singh f7484a032d added test for output verification 2025-02-28 11:35:17 +05:30
Zach Musgrave 1f983d7b40 Added a sanity check test to make sure we aren't accidentally emitting server heartbeats early 2025-02-27 12:16:12 -08:00
Damandeep Singh 5ce7ea7aa0 chore: added test case for log_format 2025-02-27 21:23:51 +05:30
Neil Macneale IV 9c9cb8be28 More cleanup and tests for AWS support of archives 2025-02-25 16:09:50 -08:00
Neil Macneale IV e890d96d59 Add AWS+Archive bats tests 2025-02-25 16:09:50 -08:00
Neil Macneale IV ab68948db5 Local file backup and restore working 2025-02-25 11:02:39 -08:00
Nick Tobey 1d253fbf6e Merge pull request #8865 from dolthub/nicktobey/enum
Don't rewrite table data when changing enum/varchar/varbinary type to make it more permissive.
2025-02-25 10:29:56 -08:00
Nick Tobey 4c3748c4e4 Apply suggestions from code review
Co-authored-by: Jason Fulghum <jason@dolthub.com>
2025-02-24 19:31:51 -08:00