Previously we had flatSet vs chunkedSet, each implementing the Set
interface, and SetBuilder dealing with this Set interface.
Now we have a chunkedSet interface with setValueChunk and setIndexChunk
implementations, SetBuilder dealing with the chunkedSet interface, and a
Set class which implements Set primitives on top of a chunkedSet.
Of particular note is the introduction of the setChunkStore, an
in-memory cache of setChunks keyed by their refs. This is needed right
now so that we only have a single SetBuilder implementation which chunks
refs, as opposed to 2 SetBuilder implementations, one which chunks refs
and the other chunkedSet instances. Something like it will be needed in
the longer term to implement lazily loading set chunks, however there
will be issues like memory pressure to deal with as well.
Follow-ups to this patch include renaming files to reflect their class
names better, and doing a bit of function/member renaming to better
reflect semantics. I've tried to keep the diff as small as possible.
So far:
- It does not support Put. It can only be constructed with a SetBuilder,
then queried e.g. Has.
- It only supports ref.Refs as values.
- I'm not convinced about the naming. Perhaps SetChunker rather than
SetBuilder, FlatLevel rather that FlatSet, etc. I expect they will change.
- The test is fairly simplistic but there isn't much point testing this
exhaustively, since once we support mutation, the testing abstraction
will probably be with an Iterator.
