mirror of
https://github.com/dolthub/dolt.git
synced 2026-01-24 10:30:48 -06:00
There are two places where ValidatingBatchingSink could be more concurrent: Prepare(), where it's reading in hints, and Enqueue(). Making Prepare() handle many hints concurrently is easy because the hints don't depend on one another, so that method now just spins up a number of goroutines and runs them all at once. Enqueue() is more complex, because while Chunk decoding and validation of its hash can proceed concurrently, validating that a given Chunk is 'ref-complete' requires that the chunks in the writeValue payload all be processed in order. So, this patch uses orderedparallel to run the new Decode() method on chunks in parallel, but then return to serial operation before calling the modified Enqueue() method. Fixes #1935