Earlier I'd used 2 channels and selected over them, but as I found
elsewhere, this doesn't scale very well. It's simpler to just use a
close channel with a buffer size of 1.
sequence.getOffset was problematic and didn't have a clear meaning. In addition it was causing a bunch of +1 code at call sites. This patch replaces it with cumulativeNumLeaves which has a clearer meaning.
NewSerializer spun up a goroutine within itself. We've decided
this is an anti-pattern. Furthermore, we were using this inside
our remote database handler code, and a panic inside that goroutine
could take down the server. The callsites now use Serialize() directly.
Fixes#2169
Shows number of changes between two top level values.
```
noms diff -summarize $l::#t5p4im6uug7n5m72frr0dnjnkm04e4ph.value $l::#ueo0utduuqsf9vrntrhn25lnc19m848l.value
```
Prints:
```
13,636 insertions, 0 deletions, 107 changes, (1,919,894 values vs 1,933,530 values)
```
Where the numbers are updated as more data is computed from the diff.
Towards #2031
Leaving them running will cause a few database reads after
orderedSequenceDiffBest returns, and it looks like this is causing a
deadlock (see https://github.com/attic-labs/noms/issues/2165). I've also
seen a log related crash which goes away after this patch.
- Too have same API as all the other diff methods
- Send changes to channel without intermediary slices and without the
need to union and sort the fields
Previously a header was only printed once per entire map. This led to a
confusing diff where if a map like:
```
{
"a": {
"b": 1
}
}
```
changed to:
```
{
"a": {
"b": 2
},
"c": {
"d": 3
}
}
```
then the diff would be:
```
/["a"]
- "b": 1
+ "b": 2
+ "c": {
+ "d": 3
+ }
```
which makes it look like the "c" change is part of the ["a"]["b"] one.
After this change, the diff will be:
```
/["a"]
- "b": 1
+ "b": 2
/
+ "c": {
+ "d": 3
+ }
```
I also fixed up a bit of the diff formatting in this change.
First: diff wasn't checking whether it had stopped before sending the
final set of changes. If the caller had stopped - and therefore no
longer reading from the changes channel - diff would hang.
Second: the same close channel was being used on 2 threads and it was
possible for both to read the close signal - causing the other to hang.
The convenience constructor changed in this patch takes in a aws.Config
object directly. This allows any implementation of the mentioned interface
to be passed in to Noms's Dynamo store -- giving flexibility for client
code to add their own credential acquisition mechanisms, for instance.
[fixes#2151]
We need to also pull over the meta field. The references in meta are
already getting pulled over by `Chunks()`. If we do not pull over the
meta field we get a different commit value, with a different hash
which leads to newer pulls not being able to merge cleanly.
Fixes#2112
* Default empty csv cells to reasonable values.
It would probably be better to have more control over this, or even
better, some kind of 'null' value. But I think this is perfectly
reasonable initial behavior, and much more compatible with other
code.
* Support more spellings of "true" and "false"
* csv-import: provide more helpful value parse errors
This changes so that all commit struct types have a meta field
(which might be an empty struct).
Increment the serialization version since the old data does not
necessarily have the meta field.
Fixes ##2112
The left-right diff is expected to return results earlier, whereas the
top-down approach is faster overall. This new diff algorithm runs both:
- early results are returned from the left-right diff.
- if/when top-down catches up, left-right is stopped and the rest of the
changes are returned from top-down.
* changes here change Path index encoding from exponential to straight integer numeric format
* added flag to hrsWriter to support different float encoding formats per suggestion
* changes here consolidate encoded value logic and leave it up to the producers of hrsWriter to specify the floatFormat
Ben Kalman