Commit Graph

19 Commits

Author SHA1 Message Date
Erik Arvidsson f08e7f0f5e Make sure that FileStore returns no error for missing refs 2015-07-21 13:13:46 -07:00
Erik Arvidsson 8b566139d0 Fix MemoryStore to get the ref in close 2015-07-21 09:37:07 -07:00
Chris Masone bd3a5ee878 Fix comments about hot paths to refer to a bug instead 2015-07-20 15:23:25 -07:00
Chris Masone 0c58895cde Add a few reminder comments about hot-paths 2015-07-20 10:49:09 -07:00
Chris Masone a22b9de144 Changes to xml_importer.go to allow for naive benchmarking
Introduce the 'NopStore', which is a sorta-ChunkStore that throws away
all written data. Get() can't really be called on it, as a result,
but it's useful for taking IO totally out of the equation when testing
our code for performance.

Towards issue #67
2015-07-20 10:08:00 -07:00
Chris Masone 4632a73bc2 Performance improvements, given the xml_importer workload
1) Get rid of temp file usage in FileStore. Instead, write to a buffer and then
   dump to disk on Close().
2) Chk.Equals() uses reflection even if you call it on primitive types, which
   can be surprisingly costly. Switch to a Chk.True() in a couple of hot paths.
2015-07-17 14:14:42 -07:00
Aaron Boodman 8f3a0c4b9d Make AWSStore not put duplicate objects.
Fixes #16.
2015-07-16 12:59:54 -07:00
Erik Arvidsson e37b50f7a2 Rename s3Store to awsStore
Fixes #38
2015-07-15 18:00:14 -07:00
Aaron Boodman c364850618 Fix some bone-headed-ness in s3 flag handling. 2015-07-14 18:05:58 -07:00
Chris Masone 3dcc16dd4c Fix some nits in file_store_test.go and s3_store_test.go
1) Members of the 'suite' struct can be private
2) file is not a reserved word, so no need for fyle
3) no 'new'
2015-07-10 11:46:54 -07:00
Chris Masone 6c3047cf03 Make all ChunkStore impls crash on Write() after Close/Ref()
I incorrectly said this was already true in a previous patch, so
make it true now!

Also, updated file_store_test.go and s3_store_test.go to use stretchr
testify suites so they can share setup and teardown code.

Fixes issue #5 as well
2015-07-10 11:31:11 -07:00
Aaron Boodman 4b4a1b1f47 FileStore should not write duplicate files (issue 16) 2015-07-08 11:40:07 -07:00
Chris Masone 7a84f6313e Cleanup Close() and Ref() semantics in ChunkStore impls
In both FileStore and S3Store, the behavior of calling Ref() twice, or of
calling Ref() after Close() was undefined and probably crashy. After this,
the semantics are as follows:

Once either Ref() or Close() is called, you can't Write() any more.
Ref() can be called any number of times
Close() can be called any number of times
Ref() and Close() can be called in any order.

Addresses issue #6
2015-07-07 15:54:20 -07:00
Aaron Boodman cb1fb984b3 Remove distinction between RootTracker and ChunkStore in DataStore 2015-07-05 23:05:51 -07:00
Rafael Weinstein 153d83600e remove named TODOs 2015-06-25 09:47:54 -07:00
Aaron Boodman fdebc00dc9 Add first app: clients/counter! 2015-06-19 15:23:02 -07:00
Aaron Boodman b92551d711 Move the TODOs into bugs ... it is easier that way to refer to them from within the code 2015-06-18 18:11:52 -07:00
Aaron Boodman 4815998aa9 Add AppRootTracker and test 2015-06-18 17:59:48 -07:00
Aaron Boodman b33cab2b34 Rename package 'store' to 'chunks'.
This is in preparation for renaming the 'Commit' abstraction to
DataStore. So we will essentially have a 'chunkstore' abstraction
and a 'datastore' abstraction.
2015-06-16 16:19:16 -07:00