mirror of https://github.com/dolthub/dolt.git synced 2026-04-28 20:49:43 -05:00

Files

T

cmasone-attic d129580007 Add frag tool to measure nbs fragmentation (#2963 )

Before we can defragment NBS stores, we need to understand how
fragmented they are. This tool provides a measure of fragmentation in
which optimal chunk-graph layout implies that ALL children of a given
parent can be read in one storage-layer operation (e.g. disk read, S3
transaction, etc).

2016-12-20 17:01:18 -08:00

benchmarks

Add AWS backend for NBS (#2914 )

2016-12-06 15:12:28 -08:00

frag

Add frag tool to measure nbs fragmentation (#2963 )

2016-12-20 17:01:18 -08:00

test

Break NomsBlockStore dependency on disk storage (#2905 )

2016-12-05 09:05:40 -08:00

block_store_test.go

Break NomsBlockStore dependency on disk storage (#2905 )

2016-12-05 09:05:40 -08:00

compacting_chunk_source_test.go

tablePersister.Compact returns a chunkSource (#2939 )

2016-12-13 06:20:33 -08:00

compacting_chunk_source.go

Add frag tool to measure nbs fragmentation (#2963 )

2016-12-20 17:01:18 -08:00

dynamo_manifest_test.go

ChunkSink.Flush() (#2937 )

2016-12-12 15:39:13 -08:00

dynamo_manifest.go

ChunkSink.Flush() (#2937 )

2016-12-12 15:39:13 -08:00

file_manifest_test.go

Break NomsBlockStore dependency on disk storage (#2905 )

2016-12-05 09:05:40 -08:00

file_manifest.go

Add AWS backend for NBS (#2914 )

2016-12-06 15:12:28 -08:00

file_table_persister.go

Move concrete impls out of table_persister.go (#2952 )

2016-12-14 12:48:05 -08:00

manifest.go

Add AWS backend for NBS (#2914 )

2016-12-06 15:12:28 -08:00

mem_table_test.go

Add frag tool to measure nbs fragmentation (#2963 )

2016-12-20 17:01:18 -08:00

mem_table.go

Add AWS backend for NBS (#2914 )

2016-12-06 15:12:28 -08:00

mmap_table_reader_test.go

uint32ChunkCount (#2894 )

2016-12-02 10:12:40 -08:00

mmap_table_reader.go

Add frag tool to measure nbs fragmentation (#2963 )

2016-12-20 17:01:18 -08:00

README.md

…

root_tracker_test.go

Add frag tool to measure nbs fragmentation (#2963 )

2016-12-20 17:01:18 -08:00

s3_fake_test.go

Add frag tool to measure nbs fragmentation (#2963 )

2016-12-20 17:01:18 -08:00

s3_table_persister.go

Add frag tool to measure nbs fragmentation (#2963 )

2016-12-20 17:01:18 -08:00

s3_table_reader_test.go

Add s3 tableIndex cache (#2953 )

2016-12-14 15:06:31 -08:00

s3_table_reader.go

Add frag tool to measure nbs fragmentation (#2963 )

2016-12-20 17:01:18 -08:00

store.go

Add frag tool to measure nbs fragmentation (#2963 )

2016-12-20 17:01:18 -08:00

table_persister.go

Move concrete impls out of table_persister.go (#2952 )

2016-12-14 12:48:05 -08:00

table_reader.go

Add frag tool to measure nbs fragmentation (#2963 )

2016-12-20 17:01:18 -08:00

table_set_test.go

Add frag tool to measure nbs fragmentation (#2963 )

2016-12-20 17:01:18 -08:00

table_set.go

Add frag tool to measure nbs fragmentation (#2963 )

2016-12-20 17:01:18 -08:00

table_test.go

Add frag tool to measure nbs fragmentation (#2963 )

2016-12-20 17:01:18 -08:00

table_writer.go

Verify chunks using suffix index not computing address from data. (#2907 )

2016-12-05 11:44:43 -08:00

table.go

Add frag tool to measure nbs fragmentation (#2963 )

2016-12-20 17:01:18 -08:00

README.md

Noms Block Store

An experimental storage layer for noms.

Provides storage for a content-addressed DAG of nodes (with exactly one root), where each node is encoded as a sequence of bytes and addressed by a 20-byte hash of the byte-sequence.
There is no update or delete, only insert, update root and garbage collect.
Insertion of any novel byte-sequence is durable only upon updating the root.
File-level multiprocess concurrency is supported, with optimistic locking for multiple writers.
Writers need not worry about re-writing duplicate chunks. NBS will efficiently detect and drop (most) duplicates.