b8b2ff1c99
[stats] Rewrite stat management to use single threaded event loop ( #8815 )
...
* [stats] event loop
* more progress
* basic scheduler test working
* analyze
* add/drop hooks
* gc
* delete an alter
* drop index and table
* fix other tests
* branch management
* starter for kv
* gc and refactor maintanance
* fix bucket doubling
* delete log
* better bucket counting
* test for disk round trip
* more prolly stats gc tests
* rotate backing stats db
* progress towards swapping old for new, deleting old code
* fix gc bucket overflow
* test for gc overflow
* org and closers
* save progress update
* finally get first two bats running
* startup bound hash issue
* rewrite GC to be synchronous, fix more bugs
* fix session freshness
* fix branch gc
* cache writes and gc are serialized
* fix gc/branch update dropped hashes
* fix gc race, doubling race, jobs race
* fix more races
* docs
* convert bats to script tests
* more tests, purge/stop
* validate
* docs
* some PR cleanup
* more cleanup
* stash for pull
* fix bucket hash conflicts
* Fix more collection bugs.
* bump, timer proc
* more test fixes
* cache bats changes
* Another deadlock
* delete comment
* fmt
* no read replica stats
* fix plan tests
* branch qualified analyze fix
* [no-release-notes] go: statspro/jobqueue: Create a SerialQueue, which can perform asynchronous work on a worker thread.
* go: statspro/jobqueue: A bit of cleanup, fix a flakey test.
* rewrite with GDQ
* prog
* tests run
* fix info and storage
* outline for gc impl
* fix tests and races
* bump
* better error and panic management
* better start/stop/wait
* Add rate limiting
* gc ticker
* docs
* doc
* test prog
* fix more tests
* finish up listener tests
* add comments
* gc concurrency
* enginetests and statspro tests passing
* simplify listeners
* bats progress
* small edits
* tests progress
* bats are running
* fmt
* build
* edits
* fix interface
* fix build
* stats alternate index types
* fix mem test
* build
* fix more tests
* fmt
* more fmt
* copyright
* license
* fix races
* syntax error
* fix windows path
* nil mcv panic
* fix test races
* bump def job interval to 30ms
* deterministic tests
* more tests
* TEMP COMMIT: valctx plus some other stuff...
* shorter concurrency tests
* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh
* nondeterministic test
* try to make queue tests less racy
* missed one start
* stats granular session locks
* simplify a little
* try to avoid serialq test deadlock
* try to fix flakes
* more races
* bump
* another race
* cleanup
* more cleanup
* revert ctx validation
* most zach comments
* more comments
* more race
* bump
* more race
* bump
* schemas
* skip windows racees
* standardize server config init, use background threads management
* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh
* default stats noop
* threads management improvements
* undo change
* move stats initialization back to engine
* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh
---------
Co-authored-by: Aaron Son <aaron@dolthub.com >
Co-authored-by: max-hoffman <max-hoffman@users.noreply.github.com >
2025-03-20 15:56:48 -07:00
977064fc0d
Fix spelling
2025-03-19 03:48:45 +10:00
e7f6199e10
Merge pull request #8993 from dolthub/macneale4/dependabot
...
Bump golang.org/x/net in /go/gen/proto/dolt/services/eventsapi
2025-03-17 11:52:47 -07:00
3c6be04765
I got a fever, and I need more cowbell... or go mod tidy
2025-03-17 10:45:27 -07:00
5a405d8e50
Add archive test which would trip on broken dictionaries
2025-03-12 15:40:30 -07:00
68175f00d3
go: sqle/dsess: session.go: Fix transaction savepoint creation when cluster replication is enabled.
...
Cluster replication creates a system database, `dolt_cluster`, which does not
support transactions. There are also UserSpaceDatabases which do not support
CreateTransaction. CreateTransaction has special code to avoid visiting them
when recording the start transaction state. CreateSavepoint needed to be
updated with that logic as well.
2025-03-12 10:54:33 -07:00
51eb05981d
Sql debug cmd ( #8920 )
...
* debug command
* basic debugging working
* type
* bump
* basic bats
* fmt
* license
* fmt
2025-03-07 12:12:24 -08:00
73418533f8
Merge remote-tracking branch 'origin/main' into jycor-f9c30b3b
2025-03-04 21:52:55 -08:00
5d2c2bc547
Merge pull request #8890 from damanV5/feat/logging
...
feat:(logging) Added Log Format provision
2025-03-04 19:00:22 -08:00
2c9a3a06c8
integration-tests/mysql-client-tests: Another pass...
2025-03-04 14:23:51 -08:00
89c2542974
/integration-tests/MySQLDockerfile: run the rustup install shit
2025-03-04 13:14:00 -08:00
7ebe0288d7
chore: log_foramt functionality added as separate test case
2025-03-04 08:36:31 +05:30
e8b0967aaa
Report auto increment changes in dolt diff and dolt_diff_summary()
2025-02-28 16:01:37 -08:00
defa0bb155
[no-release-notes] integration-tests/bats: garbage_collection.bats: Fix flakiness stemming from times appear in ls output that we use for cmp.
...
These bats tests perform some GCs and inspect filesystem state to assert that
either nothing changed for a noop GC or that things actually did change for a
non-noop GC. When the inspect the filesystem state, the `ls` output includes
mtimes for the files and directories. `.dolt/stats` actually does, or at least
can, change across these runs, and so it is explicitly removed before the
comparison is done. That means the mtime on the `.dolt` directory itself
changes. These timestamps are at minute resolution, and so the tests normally
pass, but if the interactions happen in two separate minutes, they fail.
This change removes the directories themselves from the listing, leaving the
files all in place. This asserts that the files' sizes and mtimes haven't (or
have) changed across the `dolt gc` runs.
2025-02-28 10:24:48 -08:00
2ebec3f2d3
simplified test
2025-02-28 11:47:19 +05:30
f7484a032d
added test for output verification
2025-02-28 11:35:17 +05:30
1b0f660cfa
Merge pull request #8897 from dolthub/aaron/go-1.24
...
go: Bump to go 1.24. Fix some `vet` errors related to non-constant format strings.
2025-02-27 13:27:25 -08:00
1f983d7b40
Added a sanity check test to make sure we aren't accidentally emitting server heartbeats early
2025-02-27 12:16:12 -08:00
5ce7ea7aa0
chore: added test case for log_format
2025-02-27 21:23:51 +05:30
03510f5510
go/Godeps,integration-tests/go-sql-server-driver: Hygiene for the Golang version bump.
2025-02-26 15:35:24 -08:00
9c9cb8be28
More cleanup and tests for AWS support of archives
2025-02-25 16:09:50 -08:00
e890d96d59
Add AWS+Archive bats tests
2025-02-25 16:09:50 -08:00
ab68948db5
Local file backup and restore working
2025-02-25 11:02:39 -08:00
1d253fbf6e
Merge pull request #8865 from dolthub/nicktobey/enum
...
Don't rewrite table data when changing enum/varchar/varbinary type to make it more permissive.
2025-02-25 10:29:56 -08:00
4c3748c4e4
Apply suggestions from code review
...
Co-authored-by: Jason Fulghum <jason@dolthub.com >
2025-02-24 19:31:51 -08:00
837ae43d35
integration-tests/go-sql-server-driver: auto_gc_test.go: Bring down amount of garbage produced by replication test to improve speed.
2025-02-22 06:34:22 -08:00
4ce49e6a24
go: store/nbs: store.go: Fix GetManyCompressed to not redeliver chunks on waitForGC retry.
2025-02-21 17:14:53 -08:00
3c2827704f
Add comment to bats test.
2025-02-21 12:49:36 -08:00
d5c0f5abb2
Fix typo in env var name.
2025-02-21 12:47:44 -08:00
97fd0f8112
Merge remote-tracking branch 'origin/main' into aaron/autogc
2025-02-21 08:41:59 -08:00
efb5edca30
go: sqle: auto_gc: Move to a background thread which periodically checks, plus gets triggered by a commit hook.
...
This helps handle the stanbdy replica case, where commits come in through
remotesrv directly into the ChunkStore, and not through the datas.Database.
2025-02-21 08:41:03 -08:00
143893303b
integration-tests: go-sql-server-driver: auto_gc_test: Add a skipped test for auto-gc occurring on a standby replica.
2025-02-20 16:15:21 -08:00
7a04165354
Test updates for new system table
2025-02-20 09:30:10 -08:00
10e558dbbc
Add --DOLT_GC_SAFEPOINT_CONTROLLER_CHOICE environment variable and tests.
2025-02-19 19:20:37 -08:00
1e20b67c42
Merge remote-tracking branch 'origin/main' into aaron/autogc
2025-02-14 13:46:58 -08:00
eb91821ce1
Merge pull request #8857 from dolthub/macneale4/archive-clone-fetch
...
[no-release-notes] archive clone/fetch
2025-02-13 17:55:34 -08:00
7c9c808bf3
Merge remote-tracking branch 'origin/main' into aaron/autogc
2025-02-13 17:32:18 -08:00
ced4653888
go: store: nbs: Improve goroutine cleanup on gc_copier, some robustness improvements on KillConnection safepoint controller.
2025-02-13 16:36:43 -08:00
4a3d7f8177
go: remotestorage: Rework how dictionary fetching and dictionary cache is populated.
2025-02-12 18:34:20 -08:00
f8330ff852
Make metadata code aware of newgen archives
2025-02-12 13:47:43 -08:00
6c270b1d2c
First steel thread for archive fetch.
2025-02-12 13:47:43 -08:00
6c0a0396b6
Add test for cloning archived repository.
2025-02-12 13:47:43 -08:00
62e50323df
integration-tests/go-sql-server-driver: auto_gc_test.go: Tweak asserts to be more reliable.
2025-02-10 21:55:32 -08:00
dc4b94d68d
sql-server: Add behavior: auto_gc_behavior: enable.
...
When Auto GC is enabled, the running sql-server will periodically
collect a Dolt database that is growing in size. This behavior
is currently experimental. Tuning the behavior around how often to collect is
ongoing work.
2025-02-10 19:54:44 -08:00
2d8f6d4ad0
move status messages from parquet to stderr ( #8842 )
2025-02-10 15:50:05 -08:00
5a3735c558
allow dolt init on dolt repo with just config.json ( #8838 )
2025-02-10 11:37:25 -08:00
c6d7f65d2d
Re-enable disabled json test.
2025-02-07 11:31:10 -08:00
34f9158b52
Revert PR #8723 which disabled the smart JSON chunker when the document contained large strings.
...
This is now safe.
2025-02-06 22:58:05 -08:00
d7662062a8
Merge pull request #8800 from dolthub/fulghum/rm-sql-server-user
...
Remove `user` and `password` options for sql-server
2025-02-05 09:09:38 -08:00
9dd401c286
Deprecate user and pass options for sql-server
2025-02-03 13:21:07 -08:00
Maximilian Hoffman