6c5b2eb6f4
Avoid Excess Ref Computation
2015-09-26 11:33:04 -07:00
4ba824739a
Protect readBatch map with a rateLimiter
2015-09-25 14:35:37 -07:00
ec8461e8fd
RemoteDataStore / CopyReachableChunksP
2015-09-24 11:06:50 -07:00
691f71ec08
Add LevelDBStore.dumpStats
2015-09-23 14:10:27 -07:00
ee47fb0bf2
Reland: limit write concurrency of leveldb to max open file handles limit
2015-09-23 11:54:21 -07:00
b661beceae
Revert "limit write concurrency of leveldb to max open file handles limit"
...
This reverts commit 8d2d380676
2015-09-23 11:49:00 -07:00
8d2d380676
limit write concurrency of leveldb to max open file handles limit
2015-09-23 11:32:02 -07:00
adabda61a4
Make DataStore an interface
2015-09-22 17:30:14 -07:00
591026bcfa
Chunk
2015-09-21 13:24:41 -07:00
8d56b8c968
Reland: ChunkSource.Get() now returns a []byte. chunks.Serialize takes a stream of Chunks
2015-09-17 12:31:25 -07:00
a62cf1bbb8
Revert "ChunkSouce.Get() now returns []byte"
...
This reverts commit 2be269acf4
2015-09-16 20:47:36 -07:00
2be269acf4
ChunkSouce.Get() now returns []byte
2015-09-16 16:42:36 -07:00
0df9f46a2e
Remove http_store (someone got lost in recent patch)
2015-09-16 12:55:55 -07:00
9c6effe19f
split chunks/http_store into http/http_client & http/http_server
2015-09-16 10:56:43 -07:00
10350dbd7e
switch js to use buffer reads
2015-09-11 14:41:51 -07:00
ae4e35d9d1
possible fix for continuous build
2015-09-11 12:18:58 -07:00
a877c6fac9
use MemoryStore rather than ChunkBuffer
2015-09-10 12:48:27 -07:00
cdcd60c8ea
Short-term fix for parallel writes
2015-09-09 18:11:22 -07:00
9bac46b7f3
better read buffering
2015-09-09 10:58:29 -07:00
89fa402ea1
buffer pending reads
2015-09-08 17:43:32 -07:00
326dfb7953
ldbstore: Reduce the max number of open file handles, but make configurable.
2015-09-03 13:21:46 -07:00
68ce9c1391
Merge pull request #253 from rafael-atticlabs/fixHttpStoreTest
...
fix http_store test
2015-09-03 05:39:56 -07:00
b58e20eb8c
fix http_store test
2015-09-02 13:20:34 -07:00
45c6bc586b
limit number of open files for leveldb store
2015-09-02 13:13:14 -07:00
d7e73f8ea8
Put back CORS directive (broke in change to ServeMux
2015-08-30 20:48:57 -07:00
ae7dee8eaa
buffer http puts
2015-08-30 10:48:08 -07:00
925d268f0b
Use built-in ServeMux
2015-08-28 11:30:06 -07:00
834445954a
Fully shutdown server
2015-08-28 10:51:59 -07:00
29af6c47a8
Fix chunkstore breakage after removing AWS/FileStore
2015-08-27 13:51:47 -07:00
3deeae09df
Remove AWSStore, FileStore, StdoutWriter and deps
2015-08-27 13:44:22 -07:00
55c9f7a6f2
chunk_writer shouldnt decide to check whether a chunk exists before writing it
2015-08-27 13:07:00 -07:00
a21d121d71
Dont bind directly to loopback addr; TBR=aa
2015-08-27 10:27:44 -07:00
ab34143ba5
Pin dependencies using godep tool. Rewrite dep urls.
2015-08-26 14:05:40 -07:00
abc7102ad6
Clean following HttpStore implementation
2015-08-24 10:57:50 -07:00
0b57427496
Remove default hostname for httpstore
2015-08-21 14:34:23 -07:00
f6490cbbe5
Add HttpStore implementation
2015-08-21 14:22:45 -07:00
b6ebb67e91
Fix AWS update breakage
2015-08-21 14:21:04 -07:00
06c5bc6c1b
Abstract ChunkStoreWriter
2015-08-20 10:58:41 -07:00
94e7c3b73a
Abstract a bunch of (haphazzardly) duplicated tests into chunk_store_test
2015-08-19 18:23:09 -07:00
0e7d61efc6
Remove errors from ChunkStore and Ref
2015-08-18 16:24:26 -07:00
0222b95ff2
LevelDBStore
2015-08-18 11:20:17 -07:00
214b37eccf
Remove global imports of dbg package
...
Fixes #179
2015-08-08 23:57:37 -07:00
ee9a9ae03d
Fix compile breakage as a result of updating aws import
2015-08-04 15:44:40 -07:00
82523740fb
Merge pull request #151 from aboodman/iocopy
...
Use io.Copy in fileChunkWriter rather than Bytes()
2015-07-29 15:26:33 -07:00
3e472d447b
Use io.Copy in fileChunkWriter rather than Bytes()
...
Fixes #145 .
I looked for other occurences via `git grep Bytes()`. Only other
suspicious cases I found were related to types.Blob, which doesn't
deal in io.Reader yet, and memoryChunkWriter. In memoryChunkWriter
we're copying from a buffer, so I don't think there's any win, so
left that one.
2015-07-29 15:21:17 -07:00
4488d6b4c6
Abbreviate chunkstore related flags
...
We type them a lot and the long names are annoying.
2015-07-29 15:01:25 -07:00
d1e1fb4f01
Revise some comments per feedback
2015-07-29 09:22:56 -07:00
5a5ba47fb4
Add comments to DataStore and ChunkStore
...
Towards issue #136
2015-07-28 16:01:18 -07:00
f47d117bf2
Add pull command line tool, test for pulling an initial commit, address comments
...
In addition to putting in the 'pull' tool that I forgot to add in my initial PR,
I added an extra unit test to cover a case that we found to be buggy, as well
as addressing some comments by aa and arv.
1) Switched to io.Copy in CopyChunks
2) Added NewFlagsWithPrefix()
3) Cleaned up some error reporting
2015-07-28 11:21:45 -07:00
0d830e3c1d
Pull from one ChunkStore to another
...
This initial implementation requires that both the "remote" and local
ChunkStores be accessible by the machine running the pull command.
I took an initial pass at splitting up the functions so that, e.g.,
calculating which refs are needed could be done on an actual remote
machine, and we can add a chunk copying routine that gets data from
the network or something.
Towards issue #81
2015-07-27 11:41:18 -07:00
Rafael Weinstein