* Fixes a long standing bug in which the RemoteBatchStore is accidentally caching all chunks
* ValueStore's value cache now stores `Promise<?Value>` so that concurrent `readValues` of the same value can share a single decoding
* Removes the debug-only chunk hash check which keeps tripping up perf investigations
Since the head of each dataset can have an arbitrarily complex
type, type accretion leads the Datasets map at the root of the
DB to become very large. This type info isn't really very useful
at that level either. So, get rid of it by making this map be
from String -> Ref<Value>.
Fixes#2869
We are using babel-plugin-transform-inline-environment-variables which
replaces `process.env.FOO` with the value of the `FOO` environment
variable at compile time.
However, due to our pipeline we end with something like:
```js
var NAME = 'NOMS_VERSION_NEXT';
process.env[NAME]
```
which does not get replaced in development mode. If it was a const the
transformer could replace it but var bindings can change.
The old strategy for writing values was to recursively encode them,
putting the resulting chunks into a BatchStore from the bottom up as
they were generated.
The new strategy tries to keep chunks from the same 'level' of a
graph together by caching chunks as they're encoded and only writing
them once they're referenced by some other value. When a collection
is written, the graph representing it is encoded recursively, and
chunks are generated bottom-up. The new strategy should, in practice,
mean that the children of a given parent node in this graph will be
cached until that parent gets written, and then they'll get written
all at once.
Prior to this patch, ValueStore only kept a record of where referenced Refs originally reside in from chunks read from the server. This ignored the case of a client doing subsequent commits with the same ValueStore (for example, writing multiple states of a map). This was resulting in the server being forced to load a ton of chunks to validate.
Adds the ability for SequenceCursors to eagerly load all child sequences when the first child is requested.
The effect is for uses where noms tends to forward scan, it will read in batches of ~150 chunks (e.g. ~512k) rather than one chunk at a time.
On my MBP, this improves the raw blob read perf over http from ~60K/s to ~5MB/s.
When we read a chunk and create structs we were validating that the
struct was of the type it claimed to be.
This now, no longer does that validation, which matches Go.
Rafael Weinstein