Eric Halpern 242b782748 Improve sequence read performance using read-ahead (#2711) ...

* Implement read-ahead in sequence_cursor
For each meta-sequence that contains leaf sequences, start reading ahead in
parallel and deliver in order to a buffered channel. Each advance of the cursor gets
the next sequence in the read-ahead channel.

toward: #2079
-

* Address code review comments:
- Use // for all comments
- Fix label format
- Increase channel read timeout

* Rework read-ahead to use map[int]channel sequence instead of a channel of sequences

* Rework sequence cursor read-ahead for better throughput

- Guts of read-ahead now encapsulted in sequenceReadAhead
- New implemention uses a cursor to iterate across the leaves ahead
  of the current cursor
  - It reads ahead using short-lived go routines that place each read-ahead
    sequence in a channel that is then stored by hash in a map
  - When the sequence is needed, the cursor first looks in the map. If found,
    it reads the sequence from the channel stored in the map. If not, it reads
    it normally.
  - This approach allows for reading ahead in parallel without requiring a long
    running pool of goroutines
- Introduce sequenceIterator to encapulate read-ahead behind an abstraction that
  always reads forward. This is currently used narrowly but could be used more
  widely as the the core implementation for all sequence iterators

* Address review comments

2016-11-12 11:51:26 -08:00

..

minmax.go

Improve sequence read performance using read-ahead (#2711)

2016-11-12 11:51:26 -08:00