Commit Graph

10677 Commits

Author SHA1 Message Date
jennifersp 463e93169b fix panic for unexpected json file format (#5302) 2023-02-06 17:55:14 -08:00
Zach Musgrave 5dc703d8a5 Merge pull request #5306 from dolthub/zachmu/all-branches
dolt_remote_branches table
2023-02-06 15:56:21 -08:00
jennifersp f3f68b35c2 add --track flag for dolt_branch() procedure (#5312) 2023-02-06 14:37:25 -08:00
Zach Musgrave b2323008eb PR feedback, took this in a different direction 2023-02-06 14:17:37 -08:00
stephanie 2d51348360 Merge pull request #5311 from dolthub/steph/dolt-4943
makes -f default behavior for dolt fetch
2023-02-06 11:17:12 -08:00
Aaron Son a01c6717c3 Merge branch 'fix-int-overflow' 2023-02-06 11:04:12 -08:00
Aaron Son a4a48e0ed7 go/store/skip/list.go: PR feedback. 2023-02-06 11:04:05 -08:00
Stephanie You ee7d6afe1a update dolt_fetch stored procedure to make --force default 2023-02-06 09:52:17 -08:00
James Cor add5284ac2 Merge pull request #5279 from dolthub/james/fk-stuff
reuse cursor when verifying fks using secondary indexes
2023-02-03 20:33:00 -08:00
reltuk acbc3976cf [ga-bump-release] Update Dolt version to 0.52.20 and release v0.52.20 2023-02-04 02:51:15 +00:00
JCOR11599 b47151aa66 [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh 2023-02-04 01:22:53 +00:00
James Cor 82eb5129b7 cheaper compare 2023-02-03 16:42:25 -08:00
Stephanie You fd0e4d6faa makes -f default behavior for dolt fetch 2023-02-03 16:07:19 -08:00
James Cor 9d25a10e79 adding tests 2023-02-03 16:03:36 -08:00
Aaron Son 508f67554d go/store/nbs: Make a chunkSource able to return a real io.Reader.
With TableFileStore.WriteTableFile() being used on the clone path, we want to
turn chunk sources into an io.ReadCloser. In the past, store/nbs always did
this by using the ReaderAt to emulate an io.Reader. For files this maybe has OK
performance, but for AWS/Blobstore remotes it causes a serial sequence of lots
of small reads against blob storage as the io.Copy() implementation copies a
few dozen KBs at a time.

This change makes it so a chunkSource can return an actual Reader.
file_table_reader leaves the reader at emulation in place for now. One reason
to potentially get rid of even that layer is to enable something like
.WriteTo() -> sendfile() translations. Those are not currently directly in
place because we currently instrument the returned Reader with byte counting,
for example, so we can report on Clone progress. One reason to leave the
ReaderAt implementation in place, on the other hand, is interactions with file
descriptors and the file descriptor cache. This is an investigation for later;
it seemed most prudent to leave it for now since it's not currently causing
pain.
2023-02-03 14:56:15 -08:00
zachmu caf162eeb1 [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh 2023-02-03 22:13:17 +00:00
Zach Musgrave 09c52bd41e Tests and bug fix 2023-02-03 14:05:02 -08:00
James Cor 8745744f22 adding comments 2023-02-03 13:33:02 -08:00
Zach Musgrave fe3e12c957 dolt_all_branches table 2023-02-03 13:12:05 -08:00
AndyA 43907f3b0b Merge pull request #5300 from dolthub/andy/catch-gc-test-panic
go/doltcore/doltdb: catch panic in gc concurrency tests
2023-02-03 11:49:29 -08:00
Zach Musgrave 11325bb4b6 Merge pull request #5297 from dolthub/zachmu/enginetest-cleanup
[no-release-notes] engine tests changes from GMS cleanup
2023-02-03 11:01:12 -08:00
Zach Musgrave 21a8025ee7 Merge pull request #5258 from dolthub/zachmu/gc-panic
[no-release-notes] noms command to find dangling references
2023-02-03 10:42:27 -08:00
Zach Musgrave b954c2593e Merge branch 'main' into zachmu/enginetest-cleanup 2023-02-03 10:26:53 -08:00
Zach Musgrave 6f199b4400 Upgrade gms 2023-02-03 10:25:39 -08:00
Aaron Son c208615518 Merge pull request #5288 from dolthub/aaron/nbs-store-ioParallelism
go/store/nbs: tableReader, *NomsBlockStore: Use errgroup.SetLimit to enforce limit on io parallelism across the GetMany call, instead of just at the chunk source.
2023-02-03 10:17:40 -08:00
jennifersp 257b6b25fc allow different format options for dolt branch with track flag (#5286) 2023-02-03 09:34:40 -08:00
stephanie a3ba461d1c Merge pull request #5291 from dolthub/steph/dolt-5241
update query timing to be accurate when using dolt sql-client
2023-02-03 09:25:46 -08:00
Andy Arthur 4bb30885d4 go/doltcore/doltdb: catch panic in gc concurrency tests 2023-02-03 09:11:42 -08:00
JCOR11599 6fe7161ba5 [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh 2023-02-03 05:49:43 +00:00
James Cor 213433cc35 dolt1 only 2023-02-02 21:46:15 -08:00
James Cor 52c012fa96 adding test 2023-02-02 21:23:42 -08:00
AndyA 02167c3cd8 Merge pull request #5290 from dolthub/andy/concurrent-gc-tests
[no-release-notes] go/dolcore/doltdb: unskip concurrent GC integration tests
2023-02-02 19:34:07 -08:00
James Cor 0156e007aa cleaning up 2023-02-02 19:28:46 -08:00
James Cor 3c29983a57 Merge branch 'james/fk-stuff' of https://github.com/dolthub/dolt into james/fk-stuff 2023-02-02 17:08:49 -08:00
James Cor 9ed59ce529 skip nulls 2023-02-02 17:08:40 -08:00
James Cor 879c70f1a9 using too many tags 2023-02-02 16:40:29 -08:00
Andy Arthur 76b457e566 go/doltcore/doltdb: fix GC race in value_store 2023-02-02 16:17:27 -08:00
Aaron Son acea65ccbd Merge pull request #5274 from dolthub/aaron/nbs-store-memtable-sanity-check
go/store/nbs: store.go: Change memtable reference sanity check to happen at table_set.append() instead of trying to check novel chunk sources.
2023-02-02 15:56:11 -08:00
Andy Arthur 64deebfa4e go/doltcore/doltdb: fix go vet 2023-02-02 15:01:23 -08:00
JCOR11599 0eac278a6c [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh 2023-02-02 23:00:12 +00:00
James Cor 9128eb96ba I type parent instead of child 2023-02-02 14:57:19 -08:00
stephkyou 65298cd11a [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh 2023-02-02 22:21:29 +00:00
stephanie d0daacddbe Merge branch 'main' into steph/dolt-5241 2023-02-02 14:20:14 -08:00
Andy Arthur ed98180819 go/dolcore/doltdb: unskip concurrent GC integration tests 2023-02-02 14:13:03 -08:00
Stephanie You 01f29edd81 update query timing to be accurate when using dolt sql-client 2023-02-02 14:12:39 -08:00
AndyA c113089de2 Merge pull request #5282 from dolthub/andy/batch-stack
go/store/datas/pull: try to avoid unbounded growth of outstanding abs…
2023-02-02 13:00:12 -08:00
Aaron Son 46e87a7e2b go/store/nbs: tableReader, *NomsBlockStore: Use errgroup.SetLimit to enforce limit on io parallelism across the GetMany call, instead of just at the chunk source. 2023-02-02 12:37:53 -08:00
Aaron Son 98a10e646c Merge remote-tracking branch 'origin/main' into aaron/nbs-store-memtable-sanity-check 2023-02-02 12:30:16 -08:00
Aaron Son 2cd53008da Merge pull request #5273 from dolthub/aaron/value-store-gc-locking-deadlock
go/store/types: ValueStore: Fix deadlocks for concurrent writes, bufferMu and NBSStore.
2023-02-02 12:23:15 -08:00
Aaron Son f41cd010aa Merge pull request #5270 from dolthub/aaron/online-gc-prune-table-files-change
go/store/types: value_store.go: Change GC implementation to call TableFileStore.PruneTableFiles after the copy is complete.
2023-02-02 12:19:05 -08:00