98a10e646c
Merge remote-tracking branch 'origin/main' into aaron/nbs-store-memtable-sanity-check
2023-02-02 12:30:16 -08:00
2cd53008da
Merge pull request #5273 from dolthub/aaron/value-store-gc-locking-deadlock
...
go/store/types: ValueStore: Fix deadlocks for concurrent writes, bufferMu and NBSStore.
2023-02-02 12:23:15 -08:00
f41cd010aa
Merge pull request #5270 from dolthub/aaron/online-gc-prune-table-files-change
...
go/store/types: value_store.go: Change GC implementation to call TableFileStore.PruneTableFiles after the copy is complete.
2023-02-02 12:19:05 -08:00
0a414c0d5c
Merge pull request #5269 from dolthub/aaron/nbs-store-lock-ordering
...
go/store/nbs: store.go: Fix some lock ordering issues with GC and mm.LockByName. Change swapTables to not drop tables.novel or nbs.mt.
2023-02-02 12:18:49 -08:00
8f9ab7ea75
no way this worked
2023-02-02 12:11:34 -08:00
c4d82faed0
Merge branch 'main' into james/fk-stuff
2023-02-02 10:47:53 -08:00
7ae021dd6a
go/store/datas/pull: refactored puller code again
2023-02-02 10:22:29 -08:00
f6d682c308
[ga-bump-release] Update Dolt version to 0.52.19 and release v0.52.19
2023-02-02 05:13:54 +00:00
4a6eed37d7
[ga-bump-dep] Bump dependency in Dolt by max-hoffman ( #5284 )
...
Co-authored-by: max-hoffman <maximilian.wolfgang1@gmail.com >
2023-02-01 20:58:24 -08:00
b703a5673a
Refactoring changes from newest GMS
2023-02-01 17:49:07 -08:00
002641e5ab
go/store/datas/pull: fix bug in visited set update
2023-02-01 17:36:27 -08:00
0bd07d8252
go/store/datas/pull: unref visited set in pull earlier
2023-02-01 16:50:21 -08:00
83c369e47b
go/store/datas/pull: some renaming in puller
2023-02-01 16:47:30 -08:00
d7b58abc8b
go/store/datas/pull: try to avoid unbounded growth of outstanding absent set
2023-02-01 16:45:07 -08:00
eaa0877902
Merge pull request #5280 from dolthub/andy/slimmer-hash-sets
...
go/store/datas/pull: trim memory footprint for puller hash sets
2023-02-01 16:41:09 -08:00
c782af7a4a
go/store/datas/pull: also reduce nextAbsent footprint
2023-02-01 15:29:16 -08:00
1c39fea3c1
go/store/datas/pull: trim memory footprint for puller hash sets
2023-02-01 15:13:01 -08:00
eefec86552
oops
2023-02-01 14:21:17 -08:00
9cbe78cd17
works, but I think I changed nothing
2023-02-01 14:19:27 -08:00
bb9228ccd8
Merge pull request #5275 from dolthub/james/fk-stuff
...
verify fk with secondary index even when empty
2023-02-01 13:13:30 -08:00
3195ab9366
condense
2023-02-01 11:49:59 -08:00
c0fa0b7bad
[ga-bump-release] Update Dolt version to 0.52.18 and release v0.52.18
2023-02-01 19:16:45 +00:00
a50b3f5df8
no idx var needed
2023-02-01 11:04:58 -08:00
e5e41be9cb
condense
2023-02-01 11:03:19 -08:00
ecdadb9f57
Merge pull request #5271 from dolthub/bh/more-batching-three-exclamation-points
...
add another layer of batching
2023-02-01 10:48:03 -08:00
b5f43141e8
[ga-bump-release] Update Dolt version to 0.52.17 and release v0.52.17
2023-02-01 16:37:39 +00:00
b010a2106d
Added document explaining how branch control works
2023-02-01 02:13:29 -08:00
4c3e3ac557
go mod
2023-01-31 23:40:08 -08:00
359758b85b
uncomment tests
2023-01-31 23:39:17 -08:00
d4032a572b
maybe?
2023-01-31 23:38:23 -08:00
26ef0a7fd7
compare secondary even if empty
2023-01-31 20:00:38 -08:00
a8bf4f6399
go/store/nbs: store.go: Ensure we clear nbs.mt when we see a dangling ref.
2023-01-31 18:43:06 -08:00
3391a40422
[ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh
2023-02-01 02:14:07 +00:00
ff9e278b85
Reverting nil check
2023-01-31 17:32:57 -08:00
454f2d596d
Rename
2023-01-31 17:31:24 -08:00
c383aef862
Huge efficiency gain and logging for progress
2023-01-31 17:30:37 -08:00
5253e3c85a
go/store/nbs: store.go: Change memtable reference sanity check to happen at table_set.append() instead of trying to check novel chunk sources.
...
Removes enforce completeness functionality from ValueStore, which becomes
superfluous.
2023-01-31 17:02:58 -08:00
9639786b13
Merge pull request #5267 from dolthub/aaron/sqle-dprocedures-disable-dolt_gc
...
go/libraries/doltcore/sqle/dprocedures: Do not register dolt_gc.
2023-01-31 16:21:49 -08:00
9488644cf3
go/store/types: ValueStore: Fix deadlocks for concurrent writes, bufferMu and NBSStore.
...
At times ValueStore calls into ChunkStore.PutChunk, for example, with bufferMu
held. With online-GC, that call can block until a GC process is finished. So a
calling goroutine would block with bufferMu held. However, in order for the GC
process to finish, ValueStore.GC needs to grab bufferMu in order to finalize
its bookkeeping state after ChunkStore GC is successful.
This change allows reads to proceed against ValueStore while it waits for GC to
complete by adding a condition variable which we toggle before we start GC and
broadcast on after we're done.
2023-01-31 16:17:25 -08:00
e7bae47cc3
[ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh
2023-01-31 23:55:44 +00:00
e23e7ee84a
[ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh
2023-01-31 23:51:44 +00:00
56706c0826
add another layer of batching
2023-01-31 15:51:11 -08:00
ff63732b49
go/store/types: value_store.go: Change GC implementation to call TableFileStore.PruneTableFiles after the copy is complete.
2023-01-31 15:48:39 -08:00
41ee59f90d
Second pass, made arg optional
2023-01-31 15:16:34 -08:00
73187a3ec6
go/store/nbs: store.go: Fix some lock ordering issues with GC and mm.LockByName. Change swapTables to not drop tables.novel or nbs.mt.
2023-01-31 15:13:00 -08:00
754a84ca03
go/store/nbs: mem_table: getMany,getManyCompressed: Make certain to set getRecord.found = true.
...
This keeps NBSStore from returning the same chunk twice if it appears in both
memtable and a chunkSourceSet.
2023-01-31 14:42:22 -08:00
c690a702f2
[ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh
2023-01-31 22:41:48 +00:00
de9892237e
go/libraries/doltcore/sqle/dprocedures: Do not register dolt_gc.
2023-01-31 14:23:40 -08:00
9c5af6c12c
First draft at a walk command
2023-01-31 14:07:03 -08:00
6f63d326f9
[ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh
2023-01-31 21:33:52 +00:00
Aaron Son