We used to use a third party module that allocated new buffers all
the time. For Noms, we already have a buffer that we want to write
to.
For the rolling hash we use a scratch buffer that we reuse between
calls.
This also removes the DataView objects. Our usage is so basic that
having a DataView object wrapper is just overkill.
Fixes#2304
* Use sampling for a better bytes-written estimate for noms sync
* Confirmed that remaining overestimate of data written is consistent with leveldb stats and opened #2567 to track
These were previously intertwined into one writer that was
embedded in and only usable by the 'noms' command.
This commit separates them into to separate writers that
can be used independently or combined. I also moved them
into go/utils/writers so they can be used by other code.
The main impetus to do this was to fix Bug #2593.
DashCast makes the background color black, which makes sense for
displaying on a TV screen anyway - easier on the eyes, less distracting,
probably less power usage. Looks fine black on a computer screen too.
This makes a number of changes to simplify code:
1. metaSequence is now a struct, not an interface
2. orderedMetaSequence and indexedMetaSequence is gone
3. metaSequenceObject is gone
4. indexedSequence is gone
5. add leafSequence struct for leaf sequences to embed
Everything but change 5 was done by rafael@atticlabs.io.
This changes effectively:
> go test ./go/types/perf ./samples/csv/csv-import <etc>
to:
> go test ./go/types/perf <etc>
> go test ./samples/csv/csv-import <etc>
The perf results started going crazy when I started running a second
test package (./go/types/perf), this is one reason why it may have
happened. It's hard to say for sure what go test is doing.
Once we integrate noms-merge into the `noms commit` command, this
function will allow us to stop requiring users to pass in the common
ancestor to be used when merging. The code can just find it and merge
away.
Toward #2535
The 'ds' parameter can now have multiple comma separated datasets, and
every time the page refreshes (which now happens programmatically, not
via location.reload) it shows the next.
It exploits the chunked structure of Lists to allow concatenating
arbitrarily large Lists.
I've added both functional and perf tests for Concat, and the perf tests
only made sense with tests for building and reading Lists, so now we
have those too.
Prior to this change, concurrent modification of two different
Datasets in a given Database would result in an
ErrOptimisticLockFailed for whichever process lost the race. As of
this patch, concurrent changes to the _same_ Dataset will result in an
ErrMergeNeeded for the loser, but concurrent changes to different
Datasets will succeed.
Fixes#2524
* Add "noms commit" command
* Updated csv-import, json-import, xml-import and url-fetch to (optionally) not commit results
* Added helpers for creating commit meta-data struct through command line or function calls
This add two keyword arguments to the GlobCopier:
- rename - if True, will rename the files to include their hash
- index_file - if present, then the paths in this file here will be
updated to the renamed paths.
This patch modifies merge.ThreeWay() to take a callback that allows
for custom conflict resolution. The noms-merge command-line tool uses
this to inject a callback that accepts input from the console
dictating whether to accept the value from the 'left' or 'right' merge
candidates.
Toward #2445
When merge.ThreeWay() merges list splices, it frequently needs to
extract a slice of a List or check whether a given range of values is
exactly equal in two different Lists. Repeatedly Getting elements from
a List is expensive, because it creates a new cursor internally every
time. Adding IteratorAt(idx uint64) allows code to iterate over a
range of a List while only creating a cursor once. This allows a slice
of Values to be extracted or used in comparisons efficiently.
Toward #2445
For compound types (List, Set, Map, Ref) the concrete type may be a
union. If that is the case all the types in the union must be a
subtype of the concrete type's element type.
`C<T | V>` is a subtype of `C<S>` iff T is a subtype of S and V is a
subtype of S.
Fixes are based on Go report card output:
- `gofmt -s` eliminates some duplication in struct/slice initialization
- `golint` found some issues like: `warning: should drop = nil from declaration of var XXX; it is the zero value`
- `golint` found some issues like: `warning: receiver name XXX should be consistent with previous receiver name YYY for ZZZ`
- `golint` says not to use underscores for function/variable names
- `golint` found several issues like: `warning: if block ends with a return statement, so drop this else and outdent its block`
No functional changes are included - just source code quality improvements.
