Commit Graph

17672 Commits

Author SHA1 Message Date
Aaron Son df991698ec go: sqle/auto_gc: Add some simple tests for controller and hook. 2025-02-23 11:46:12 -08:00
Aaron Son 36b39dfc25 Merge remote-tracking branch 'origin/main' into aaron/autogc 2025-02-22 15:27:49 -08:00
Aaron Son 4ce49e6a24 go: store/nbs: store.go: Fix GetManyCompressed to not redeliver chunks on waitForGC retry. 2025-02-21 17:14:53 -08:00
Aaron Son fd79008176 go: publishrelease: buildpgobinaries.sh: Manually specify the SDK version of our toolchain so it shows up in our release binaries. 2025-02-21 12:37:06 -08:00
Aaron Son e4a4b5dadb go: store,remotesrv: Improve logging. Improve gRPC error statuses for certain errros that arise after a GC. 2025-02-21 11:31:23 -08:00
coffeegoddd cd1ed7276b [ga-bump-release] Update Dolt version to 1.49.3 and release v1.49.3 2025-02-21 19:07:30 +00:00
Dustin Brown 5fad9beeb6 [ga-bump-dep] Bump dependency in Dolt by max-hoffman (#8877)
Co-authored-by: max-hoffman <maximilian.wolfgang1@gmail.com>
2025-02-21 10:33:25 -08:00
Aaron Son 97fd0f8112 Merge remote-tracking branch 'origin/main' into aaron/autogc 2025-02-21 08:41:59 -08:00
Aaron Son 89fce6e919 Merge pull request #8861 from dolthub/aaron/remotestorage-cache-cleanup
[no-release-notes] go: remotestorage: chunk_store.go: Clean up ChunkCache.
2025-02-21 08:41:48 -08:00
Aaron Son efb5edca30 go: sqle: auto_gc: Move to a background thread which periodically checks, plus gets triggered by a commit hook.
This helps handle the stanbdy replica case, where commits come in through
remotesrv directly into the ChunkStore, and not through the datas.Database.
2025-02-21 08:41:03 -08:00
Aaron Son d586d2a84c go: remotestorage: chunk cache: PR feedback. 2025-02-20 10:13:15 -08:00
Neil Macneale IV 389f4a95c3 Make tests resilient to windows file paths 2025-02-20 10:02:19 -08:00
Neil Macneale IV 7a04165354 Test updates for new system table 2025-02-20 09:30:10 -08:00
macneale4 0cc8177cef [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh 2025-02-20 00:30:29 +00:00
Aaron Son 7dadc7ceed go: remotestorage: PR feedback on ChunkCache. 2025-02-19 15:13:26 -08:00
Aaron Son 69630e1007 go: auto_gc: Add a heuristic which allows followers in cluster replication to also auto-gc. 2025-02-19 15:07:07 -08:00
Aaron Son 20f38cae6f Merge remote-tracking branch 'origin/main' into aaron/autogc 2025-02-19 13:30:33 -08:00
Neil Macneale IV 0401281800 Add the backups_table
Read only table which only lists the name and url of backups created
by the dolt_backups procedure
2025-02-19 11:23:37 -08:00
coffeegoddd f9a5d44353 [ga-bump-release] Update Dolt version to 1.49.2 and release v1.49.2 2025-02-17 20:37:56 +00:00
Dustin Brown a4a5a75f1d [ga-bump-dep] Bump dependency in Dolt by max-hoffman (#8867)
Co-authored-by: max-hoffman <maximilian.wolfgang1@gmail.com>
2025-02-17 11:52:57 -08:00
Taylor Bantle 872b69e2af Merge pull request #8860 from dolthub/taylor/restore-perms
Add pointer for checking super user perms for backup restore for doltgres
2025-02-17 10:07:30 -06:00
fulghum 6e19bfaf45 [ga-bump-dep] Bump dependency in Dolt by fulghum 2025-02-14 23:10:11 +00:00
Aaron Son 1e20b67c42 Merge remote-tracking branch 'origin/main' into aaron/autogc 2025-02-14 13:46:58 -08:00
Aaron Son fc100694f6 Merge remote-tracking branch 'origin/main' into aaron/remotestorage-cache-cleanup 2025-02-14 13:33:45 -08:00
Aaron Son 449318ac97 go: remotestorage: Fix data race in writebuffer_test.go. 2025-02-14 12:43:43 -08:00
reltuk c9133db51f [ga-bump-dep] Bump dependency in Dolt by reltuk 2025-02-14 20:43:09 +00:00
Aaron Son 5d4d380f72 go: remotestorage: Make write buffer machinery more robust. Add some testing. 2025-02-14 12:32:59 -08:00
Aaron Son 9b267b47c5 go: remotestorage: writebuffer: Fix write buffer chunk caching on failed commits. 2025-02-14 11:22:34 -08:00
Aaron Son bfa2739516 Merge remote-tracking branch 'origin/main' into aaron/remotestorage-cache-cleanup 2025-02-14 10:47:15 -08:00
Neil Macneale IV eb91821ce1 Merge pull request #8857 from dolthub/macneale4/archive-clone-fetch
[no-release-notes] archive clone/fetch
2025-02-13 17:55:34 -08:00
Aaron Son 7c9c808bf3 Merge remote-tracking branch 'origin/main' into aaron/autogc 2025-02-13 17:32:18 -08:00
Aaron Son 2a47d402b2 go: sqle: auto_gc: Responding to PR feedback. Comments and cleanup. 2025-02-13 17:24:03 -08:00
Aaron Son d4a62c13b7 Merge pull request #8848 from dolthub/aaron/async-push-hook-session-lifecycle
[no-release-notes] go: sqle: Change how we run background threads for replication hooks. Add session lifecycle callbacks for replication hooks.
2025-02-13 17:14:11 -08:00
Aaron Son f662a63b1e Merge pull request #8858 from dolthub/aaron/gc-clear-caches
[no-release-notes] go: sqle: dolt_gc: Purge read caches associated with the DoltDB as we begin GC.
2025-02-13 17:12:50 -08:00
Neil Macneale IV 2cab73a144 Archive test updates to account for default dictionaries 2025-02-13 16:47:01 -08:00
Aaron Son cd8372b9c8 go: prolly/tree: node_store.go: Doc comment on PurgeCaches. 2025-02-13 16:38:32 -08:00
Aaron Son ced4653888 go: store: nbs: Improve goroutine cleanup on gc_copier, some robustness improvements on KillConnection safepoint controller. 2025-02-13 16:36:43 -08:00
Neil Macneale IV b086492018 Account for empty chunks in archive.getManyCompressed 2025-02-13 16:05:48 -08:00
Neil Macneale IV 77a4e59fc2 More PR Feedback 2025-02-13 15:44:57 -08:00
Neil Macneale IV 2434b63eae PR Feedback 2025-02-13 15:39:06 -08:00
Neil Macneale IV 63e29a790d rollback the change to the getMany interface 2025-02-13 15:24:08 -08:00
Aaron Son d459962770 Merge pull request #8851 from dolthub/aaron/valuestore-gc-states-unit-tests
[no-release-notes] go/store/types: value_store: Add some unit tests for gc states and gcAddChunk behavior.
2025-02-13 14:48:20 -08:00
Aaron Son ea5615dae2 go: store/types: value_store_test.go: PR feedback, some comments. 2025-02-13 14:47:39 -08:00
Aaron Son 64cef607e3 Merge pull request #8854 from dolthub/aaron/statspro-session-lifecycle
[no-release-notes] go: statspro: Add session lifecycle callbacks to integrate with session-aware GC safepoint controller.
2025-02-13 14:40:07 -08:00
Aaron Son 5eea56b79d go: remotestorage: chunk_store.go: Clean up ChunkCache.
When DoltChunkStore was implemented, it needed to do write buffering in order
to read its own writes and to flush table files to the upstream. At some point,
read caching and caching of has many results was added onto the write buffer,
creating confusion and despair. This change separates back out the use cases.
2025-02-13 13:01:59 -08:00
Neil Macneale IV ecac26bf2b Remove the FullCompressedChunkLen function from ToChunker interface 2025-02-13 12:23:47 -08:00
Taylor Bantle 54b504f4b0 Add pointer for checking super user perms for backup restore for doltgres 2025-02-13 11:37:56 -08:00
Neil Macneale IV c76ab7e7d9 more cleanup 2025-02-13 10:55:01 -08:00
reltuk caa1e47e8a [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh 2025-02-13 18:32:20 +00:00
Aaron Son c8353bb053 go: Fix some tests for new ValueStore cache purging behavior where we rely on a safepoint controller to do it. 2025-02-13 10:24:23 -08:00