Commit Graph

3163 Commits

Author SHA1 Message Date
James Cor 0b7a78bb24 Implement sql.ValueRow (#9906) 2025-11-03 13:49:17 -08:00
James Cor 8d2b8883f3 use unsafe pointer for textstorage (#9994)
Co-authored-by: James Cor <james@dolthub.com>
2025-10-31 13:58:03 -07:00
Aaron Son 947f31cb8b Merge pull request #10014 from dolthub/aaron/conjoin-small-fixes
[no-release-notes] go/store/nbs: Small cleanups to archive and blobstore conjoin.
2025-10-30 18:19:11 -07:00
Aaron Son 79b851bf33 go/store/nbs: conjoiner_test.go: Close ReadCloser on archive-in-blobstore conjoin tests. 2025-10-30 15:33:57 -07:00
reltuk fa32792591 [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh 2025-10-30 21:51:29 +00:00
Aaron Son 8a06a76dbb go/store/nbs: Add some conjoin testing which excerises archive path with blobstores. 2025-10-30 14:42:54 -07:00
Aaron Son 3552270ded go/store/nbs: table_persister.go: PR feedback: Remove unused conjoinModeUnknown. 2025-10-30 13:47:39 -07:00
Aaron Son 3599845647 Update go/store/nbs/archive_writer.go
Co-authored-by: Neil Macneale IV <46170177+macneale4@users.noreply.github.com>
2025-10-30 13:31:23 -07:00
Aaron Son 55f352f448 Merge remote-tracking branch 'origin/main' into aaron/conjoin-small-fixes 2025-10-30 12:30:35 -07:00
Aaron Son 7918c7860e [no-release-notes] go/store/nbs: Small cleanups to archive and blobstore conjoin. 2025-10-30 12:27:18 -07:00
Neil Macneale IV 4a283e530b correctly initialize seed in archive test 2025-10-30 12:21:02 -07:00
Neil Macneale IV 44a708c1c6 remove dead function 2025-10-29 16:39:51 -07:00
macneale4 26209cde90 [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh 2025-10-29 22:32:59 +00:00
Neil Macneale IV 771d6aea23 Update iterateAllChunks docs to mention duplicates 2025-10-29 15:21:12 -07:00
Aaron Son bdadede2d3 go/store/nbs: Use bufio in iterateAll implementations. (#9515) 2025-10-29 15:21:08 -07:00
Neil Macneale IV 59943d70a8 One more meaningless comment removed 2025-10-29 14:57:22 -07:00
macneale4 abffd35941 [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh 2025-10-29 14:57:22 -07:00
Neil Macneale IV bb41cbdb44 Use 4Mb window. And clean up so many comments 2025-10-29 14:57:17 -07:00
macneale4 dcf6d4ad99 [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh 2025-10-29 14:53:56 -07:00
Neil Macneale IV c2ec592a7c Remove unnecessary alloc 2025-10-29 14:53:56 -07:00
macneale4 a4020acce6 Iterating over TableFile chunks faster 2025-10-29 14:53:46 -07:00
Neil Macneale IV deee7620d3 First pass on faster chunk iteration 2025-10-29 14:49:46 -07:00
Aaron Son c9e611e6cc go/store/nbs: tableReader: Account for allocated table index prefixes in a tableReader's quota usage. 2025-10-23 16:19:23 -07:00
Aaron Son 38a1ed32ff Merge pull request #9990 from dolthub/aaron/gc-full-oldgen-fix
go/store/nbs: store.go: Fix a bug which caused dolt gc --full to not collect new data sometimes.
2025-10-23 11:40:25 -07:00
Aaron Son f3adba56cc go/store/nbs: store.go: Fix a bug which caused dolt gc --full to not collect new data sometimes.
If `dolt gc --full` was run, and then branches were changed and `dolt gc` was run, possibly multiple times, and then all of the state was returned to the state of the database during the initial `dolt gc --full`, then running `dolt gc --full` anew would falsely conclude that there was nothing to collect.

After the GC work is done, and we are inside swapTables, there is no need to check for whether we should swap the tables. We already have done the work to compute the new files and we should apply them regardless.
2025-10-23 10:54:38 -07:00
Aaron Son db127eb1b8 Apply suggestions from code review
Co-authored-by: Neil Macneale IV <46170177+macneale4@users.noreply.github.com>
2025-10-22 16:51:49 -07:00
Aaron Son 52e673e445 go/store/nbs: archive_writer.go: Fix problem with archive stream writer where we did not remove temptf files when we were pushing changes. 2025-10-22 16:23:08 -07:00
Nick Tobey a7006502d7 Respond to PR feedback. 2025-10-21 13:04:06 -07:00
nicktobey 92519382c4 [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh 2025-10-21 03:49:26 +00:00
Nick Tobey be9faf4e5a Replace uses of io.EOF in merge logic with an explicit return value. 2025-10-20 20:40:40 -07:00
Neil Macneale IV ed4e0449ea Merge pull request #9964 from dolthub/macneale4/dolt-1-75
Enable AutoGC and Archive format by default
2025-10-16 11:32:59 -07:00
Aaron Son 87272d6951 go/store/nbs: Fix merge conflict on interface change for bs_persister.go. 2025-10-16 12:01:54 +02:00
Aaron Son 209c11dc7b Merge pull request #9958 from dolthub/aaron/archive-reader-uses-quota
[no-release-notes] go/store/nbs: Add memory quota usage for in-memory archive index reader.
2025-10-16 11:50:05 +02:00
Neil Macneale IV 1d6f2dd0e3 return approximate size from GenericTableWriter.AddChunk 2025-10-15 11:54:12 -07:00
Neil Macneale IV d89b5f92bb AutoGC and Archive as defaults. Untested 2025-10-15 11:39:45 -07:00
Neil Macneale IV 40fef904c4 Merge pull request #9939 from dolthub/macneale4/blobstore-conjoin
blobstore push/pull and conjoin
2025-10-15 10:49:01 -07:00
Aaron Son cabc3051b3 go/store/nbs: archive reader quota: PR feedback. 2025-10-15 19:32:16 +02:00
Aaron Son 8922baf7f4 Merge pull request #9940 from dolthub/aaron/visit-gc-roots-map-editors
go: sqle: Add support for visting GC roots in current table editors.
2025-10-15 19:12:20 +02:00
Aaron Son 2c59cc078a go/store/nbs: archive_reader_test.go: Slightly more robust testing around clone and close behavior and interaction with quota. 2025-10-15 11:00:50 +02:00
Aaron Son 4956787878 go/store/nbs: archive_reader_test.go: Make sure to close tableReaderAts when there is an error constructing the archive reader. 2025-10-15 10:48:49 +02:00
Aaron Son 6c30d14899 [no-release-notes] go/store/nbs: Add memory quota usage for in-memory archive index reader. 2025-10-14 19:18:19 +02:00
Aaron Son 1ab4d9b373 go: utils/earl,store/spec: Fix AWS parsing for spec as well. 2025-10-14 13:07:02 +02:00
Aaron Son 06a1e9bb3a go/libraries/utils/earl: Add special handling for AWS URLs with non-compliant host components so that we can bump the Golang version. 2025-10-14 11:56:26 +02:00
Neil Macneale IV d201688929 mem table WriteChunks to return splitOffset 2025-10-10 11:50:51 -07:00
Neil Macneale IV b71dc7561e Cope with splitOffset being unset 2025-10-10 11:28:46 -07:00
Neil Macneale IV cb605d3ecc Update go/store/chunks/tablefilestore.go
Co-authored-by: Aaron Son <aaron@dolthub.com>
2025-10-10 10:24:40 -07:00
Aaron Son bb69179164 Merge remote-tracking branch 'origin/main' into aaron/auto-gc-periodic-check 2025-10-09 05:22:15 -07:00
macneale4 1ca2727ba5 [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh 2025-10-09 00:19:04 +00:00
Neil Macneale IV f84a15c515 Create the *.darc.records file when it doesn't exist 2025-10-08 17:08:10 -07:00
Neil Macneale IV 405946676f more fixes to blobstore file resolution 2025-10-08 17:08:10 -07:00