mirror of
https://github.com/dolthub/dolt.git
synced 2026-04-30 11:31:37 -05:00
cmasone-attic
c9c1bb9ff5
There are two places where ValidatingBatchingSink could be more concurrent: Prepare(), where it's reading in hints, and Enqueue(). Making Prepare() handle many hints concurrently is easy because the hints don't depend on one another, so that method now just spins up a number of goroutines and runs them all at once. Enqueue() is more complex, because while Chunk decoding and validation of its hash can proceed concurrently, validating that a given Chunk is 'ref-complete' requires that the chunks in the writeValue payload all be processed in order. So, this patch uses orderedparallel to run the new Decode() method on chunks in parallel, but then return to serial operation before calling the modified Enqueue() method. Fixes #1935
Example
cd $GOPATH/src/github.com/attic-labs/noms/samples/go/counter
go build
./counter /tmp/nomsdb::counter
./counter /tmp/nomsdb::counter
./counter /tmp/nomsdb::counter
noms serve /tmp/nomsdb
Then, in a separate shell:
# This starts where the previous count left off because we're serving the same database
./counter http://localhost:8000::counter
# Display the datasets at this server
noms ds http://localhost:8000
# Print the history of the counter dataset
noms log http://localhost:8000::counter