Fixes#145.
I looked for other occurences via `git grep Bytes()`. Only other
suspicious cases I found were related to types.Blob, which doesn't
deal in io.Reader yet, and memoryChunkWriter. In memoryChunkWriter
we're copying from a buffer, so I don't think there's any win, so
left that one.
In addition to putting in the 'pull' tool that I forgot to add in my initial PR,
I added an extra unit test to cover a case that we found to be buggy, as well
as addressing some comments by aa and arv.
1) Switched to io.Copy in CopyChunks
2) Added NewFlagsWithPrefix()
3) Cleaned up some error reporting
This initial implementation requires that both the "remote" and local
ChunkStores be accessible by the machine running the pull command.
I took an initial pass at splitting up the functions so that, e.g.,
calculating which refs are needed could be done on an actual remote
machine, and we can add a chunk copying routine that gets data from
the network or something.
Towards issue #81
Introduce the 'NopStore', which is a sorta-ChunkStore that throws away
all written data. Get() can't really be called on it, as a result,
but it's useful for taking IO totally out of the equation when testing
our code for performance.
Towards issue #67
1) Get rid of temp file usage in FileStore. Instead, write to a buffer and then
dump to disk on Close().
2) Chk.Equals() uses reflection even if you call it on primitive types, which
can be surprisingly costly. Switch to a Chk.True() in a couple of hot paths.
I incorrectly said this was already true in a previous patch, so
make it true now!
Also, updated file_store_test.go and s3_store_test.go to use stretchr
testify suites so they can share setup and teardown code.
Fixes issue #5 as well
In both FileStore and S3Store, the behavior of calling Ref() twice, or of
calling Ref() after Close() was undefined and probably crashy. After this,
the semantics are as follows:
Once either Ref() or Close() is called, you can't Write() any more.
Ref() can be called any number of times
Close() can be called any number of times
Ref() and Close() can be called in any order.
Addresses issue #6
This is in preparation for renaming the 'Commit' abstraction to
DataStore. So we will essentially have a 'chunkstore' abstraction
and a 'datastore' abstraction.
Rafael Weinstein