Also, factor out a separate NewDataStoreWithRootTracker() since
the common case is to use the same value for both the ChunkStore
and the RootTracker.
Fixes#134
In addition to putting in the 'pull' tool that I forgot to add in my initial PR,
I added an extra unit test to cover a case that we found to be buggy, as well
as addressing some comments by aa and arv.
1) Switched to io.Copy in CopyChunks
2) Added NewFlagsWithPrefix()
3) Cleaned up some error reporting
We've been keeping some special-case code in Datastore to handle the
situation where the Root of the store is nonexistent. There was some
checking for the empty Ref and creating a SetOfCommit out of whole
cloth. This meant that if you asked an empty Datastore (or Dataset)
what its Heads() were, it would give you back a Value that wasn't
backed by a Chunk in its underlying ChunkStore. This caused some
issues with pull code, so we decided to change things such that a
DataStore is primed with an initial empty SetOfCommit upon creation.
This means creating a Dataset in a DataStore now shows up in DataStore
history, which it did not before. Essentially, every Dataset now has
an "initial commit" of an empty SetOfCommit when it's created. I think
that having these show up as part of the DataStore history makes sense,
but the model may evolve over time.
This initial implementation requires that both the "remote" and local
ChunkStores be accessible by the machine running the pull command.
I took an initial pass at splitting up the functions so that, e.g.,
calculating which refs are needed could be done on an actual remote
machine, and we can add a chunk copying routine that gets data from
the network or something.
Towards issue #81
The test for this does the following:
1. The generated code is checked in
2. Running the test regenerates the generated code
3. Then the test.go is compiled and run
This also adds a smoke test for the codegen which just makes sure
that we do not fail when we try to generate code.
Fixes#90
It turns out these functions aren't actually generally useful,
because they deal in Values and Refs, while what we actually
want is to deal in chunks. So...
Revert "Settle big/small vs to/from ONCE AND FOR ALL"
This reverts commit f51c575cd2.
Revert "Add a naive Diff() function between two Values."
This reverts commit 662bc5e3c4.
Given two refs, we need to be able to determine the set of chunks
reachable from one that cannot be reached from the other. This patch
adds a naive implementation of this functionality that essentially walks
the value tree down from the values pointed to by two refs and compares them.
It also has a little command-line driver, but this may be removed.
Fixes issue #82
Also change ChunkStore from being embedded in server to being a field of it. It is equivalent, but this feels more semantically correct and it's less code anyway.
Introduce the 'NopStore', which is a sorta-ChunkStore that throws away
all written data. Get() can't really be called on it, as a result,
but it's useful for taking IO totally out of the equation when testing
our code for performance.
Towards issue #67
Aaron Boodman