Commit Graph

23 Commits

Author SHA1 Message Date
Dan Willhite 2018863432 Add hasCache to datastore.
Fixes #693.
2015-12-08 11:22:50 -08:00
Rafael Weinstein adabda61a4 Make DataStore an interface 2015-09-22 17:30:14 -07:00
Rafael Weinstein 591026bcfa Chunk 2015-09-21 13:24:41 -07:00
Rafael Weinstein 8d56b8c968 Reland: ChunkSource.Get() now returns a []byte. chunks.Serialize takes a stream of Chunks 2015-09-17 12:31:25 -07:00
Rafael Weinstein a62cf1bbb8 Revert "ChunkSouce.Get() now returns []byte"
This reverts commit 2be269acf4.
2015-09-16 20:47:36 -07:00
Rafael Weinstein 2be269acf4 ChunkSouce.Get() now returns []byte 2015-09-16 16:42:36 -07:00
Rafael Weinstein 9c6effe19f split chunks/http_store into http/http_client & http/http_server 2015-09-16 10:56:43 -07:00
Rafael Weinstein a877c6fac9 use MemoryStore rather than ChunkBuffer 2015-09-10 12:48:27 -07:00
Rafael Weinstein 9bac46b7f3 better read buffering 2015-09-09 10:58:29 -07:00
Rafael Weinstein 55c9f7a6f2 chunk_writer shouldnt decide to check whether a chunk exists before writing it 2015-08-27 13:07:00 -07:00
Rafael Weinstein 06c5bc6c1b Abstract ChunkStoreWriter 2015-08-20 10:58:41 -07:00
Rafael Weinstein 0e7d61efc6 Remove errors from ChunkStore and Ref 2015-08-18 16:24:26 -07:00
Aaron Boodman 214b37eccf Remove global imports of dbg package
Fixes #179
2015-08-08 23:57:37 -07:00
Aaron Boodman 4488d6b4c6 Abbreviate chunkstore related flags
We type them a lot and the long names are annoying.
2015-07-29 15:01:25 -07:00
Chris Masone f47d117bf2 Add pull command line tool, test for pulling an initial commit, address comments
In addition to putting in the 'pull' tool that I forgot to add in my initial PR,
I added an extra unit test to cover a case that we found to be buggy, as well
as addressing some comments by aa and arv.

1) Switched to io.Copy in CopyChunks
2) Added NewFlagsWithPrefix()
3) Cleaned up some error reporting
2015-07-28 11:21:45 -07:00
Chris Masone 0d830e3c1d Pull from one ChunkStore to another
This initial implementation requires that both the "remote" and local
ChunkStores be accessible by the machine running the pull command.
I took an initial pass at splitting up the functions so that, e.g.,
calculating which refs are needed could be done on an actual remote
machine, and we can add a chunk copying routine that gets data from
the network or something.

Towards issue #81
2015-07-27 11:41:18 -07:00
Erik Arvidsson 8b566139d0 Fix MemoryStore to get the ref in close 2015-07-21 09:37:07 -07:00
Chris Masone a22b9de144 Changes to xml_importer.go to allow for naive benchmarking
Introduce the 'NopStore', which is a sorta-ChunkStore that throws away
all written data. Get() can't really be called on it, as a result,
but it's useful for taking IO totally out of the equation when testing
our code for performance.

Towards issue #67
2015-07-20 10:08:00 -07:00
Aaron Boodman c364850618 Fix some bone-headed-ness in s3 flag handling. 2015-07-14 18:05:58 -07:00
Chris Masone 6c3047cf03 Make all ChunkStore impls crash on Write() after Close/Ref()
I incorrectly said this was already true in a previous patch, so
make it true now!

Also, updated file_store_test.go and s3_store_test.go to use stretchr
testify suites so they can share setup and teardown code.

Fixes issue #5 as well
2015-07-10 11:31:11 -07:00
Aaron Boodman fdebc00dc9 Add first app: clients/counter! 2015-06-19 15:23:02 -07:00
Aaron Boodman b92551d711 Move the TODOs into bugs ... it is easier that way to refer to them from within the code 2015-06-18 18:11:52 -07:00
Aaron Boodman b33cab2b34 Rename package 'store' to 'chunks'.
This is in preparation for renaming the 'Commit' abstraction to
DataStore. So we will essentially have a 'chunkstore' abstraction
and a 'datastore' abstraction.
2015-06-16 16:19:16 -07:00