2dec53453e
NomDL: Add new serialization format
...
The new serialization format use "t " as in typed. The rest of the
message is a JSON array describing the typed data. The type is
described by types.TypeRef
Fixes #384
Issues #281 , #304
2015-10-06 15:56:10 -07:00
6c5b2eb6f4
Avoid Excess Ref Computation
2015-09-26 11:33:04 -07:00
cdb22e131b
Address comments
2015-09-23 11:49:14 -07:00
63c956a5c5
Add types.TypeRef
...
We want to explore encoding type information about Noms data in
the Noms database. So, we need some way to describe types. This
takes the shortest path to making a Noms type called "TypeRef" that
is a peer of Set, Map et al and can describe all the types we currently
use.
2015-09-22 16:19:38 -07:00
591026bcfa
Chunk
2015-09-21 13:24:41 -07:00
b6197aadc4
Add support for compound lists.
...
Lists are now either a leafList or a compoundList. The compoundList
consists of sublists that are the chunks of the whole list.
2015-08-28 15:41:51 -04:00
2d96186090
Add encoding/decoding of compound lists
2015-08-28 15:27:35 -04:00
ab34143ba5
Pin dependencies using godep tool. Rewrite dep urls.
2015-08-26 14:05:40 -07:00
1d152410db
Tighten up compound blob encoding
...
Before:
```json
{"cb":[{"ref":"sha1-x"},lengthX,{"ref":"sha1-y"},lengthY]}
```
After:
```json
{"cb":["sha1-x",lengthX,"sha1-y",lengthY]}
```
Fixes #170
2015-08-25 12:16:43 -04:00
06c5bc6c1b
Abstract ChunkStoreWriter
2015-08-20 10:58:41 -07:00
0555d7a3c1
Remove errors from read/write/encode/decode
2015-08-18 16:37:04 -07:00
0e7d61efc6
Remove errors from ChunkStore and Ref
2015-08-18 16:24:26 -07:00
1bd5af910a
Add support for int8/uint8
...
It turns out that having these makes marshaling native go to and from noms
cleaner.
Towards issue #160
2015-08-17 13:44:50 -07:00
214b37eccf
Remove global imports of dbg package
...
Fixes #179
2015-08-08 23:57:37 -07:00
0a6f1a1a68
Remove some unused functions in json_encode.go
...
I intended to delete these before landing my last encode patch,
but failed to save the file.
2015-08-07 13:51:11 -07:00
ddebdcaefd
Slight modification to compound blob encoding
...
The json serialization now only contains the length of each individual
blob child.
The go representation of this still uses offsets but the offsets are
for the end delimiter.
For "hi" "bye" we get
{"cb", [{"ref": "sha1-hi"}, 2, {"ref": "sha1-bye"}, 3]}
compoundBlob{[2, 5], [sha1-hi, ,sha1-bye]}
Keeping the length in the serialization leads to smaller serializations
Using the end offset leads to simpler binary search and allows us to
use the last entry as the length.
Issue #17
2015-08-07 11:24:27 -04:00
f4ef0d5cbc
Address comments
...
Ensure reader gets closed in all cases in ReadValue, clean up BUG references,
delete NewCompoundBlob, and switch an io.Copy -> ioutil.ReadAll
2015-08-06 16:46:00 -07:00
bec1b344be
Migrate the types package to JSON decode using the enc package
...
This change removes the json decoding code from the types package and ports
it onto the enc package's encoding API.
Fixes issue #159
2015-08-06 16:09:35 -07:00
07046ce567
Address aa comments
2015-08-06 12:48:39 -07:00
1a3c3e2c41
Migrate the types package to JSON encode using the enc package
...
This change removes the json encoding code from the types package and ports
it onto the enc package's encoding API.
Towards issue #159
2015-08-06 10:30:16 -07:00
7ebe98f49c
Address final comments
...
Primarily removing actual ref caclulations from unit tests, since
those aren't this package's responsiblility
2015-08-05 13:00:49 -07:00
3c360b7037
Adapt to new compound blob encoding
2015-08-05 11:56:49 -07:00
e562847b5a
Address Aaron's comments
...
1) Clean up unit tests to check against serialization directly
instead of checking refs.
2) Switch argument order of Encode functions to be dst, src
3) Misc other stuff
2015-08-05 11:56:49 -07:00
7da05a4be7
First cut of Very Low-Level JSON Encoding Package
...
VLLJEP is go! This first pass handles primitives, strings, refs, lists
and sets of encodeable things, maps of encodeable -> encodeable,
blobs, and compound-blobs. The difference between this and the
encoding stuff in types/ is that it doesn't recurse. The caller is
responsible for going down its value tree until it finds leaves,
encoding them, and then popping back up a level to replace those
values with Refs.
Towards issue #159
2015-08-05 11:56:49 -07:00
2aa809e167
Move enc/* into types/*.
2015-07-09 17:30:43 -07:00
8ec835a134
Ensure that chunks opened for reading get closed
2015-07-09 13:30:41 -07:00
8590bd4249
Finish first crack at generating strong Go types for noms data
2015-06-26 16:35:08 -07:00
b33cab2b34
Rename package 'store' to 'chunks'.
...
This is in preparation for renaming the 'Commit' abstraction to
DataStore. So we will essentially have a 'chunkstore' abstraction
and a 'datastore' abstraction.
2015-06-16 16:19:16 -07:00
8b28d4fbba
Added Reachable and correct calculation of current roots
2015-06-16 15:10:12 -07:00
129798c006
Support any type of Value for keys of maps
2015-06-15 15:57:33 -07:00
08a53bb48f
add types.Set
2015-06-13 23:25:48 -07:00
7aaa583441
Moved Root()/UpdateRoot() to new interface (RootStore), Fixed other store impls
2015-06-12 16:10:33 -07:00
53003f23f2
Add Value::Ref()
2015-06-12 15:22:27 -07:00
22eb927c4e
Blob::Read()->Blob::Reader()
2015-06-12 08:45:29 -07:00
edbc1c87cf
(en|de)codeBlob->blob(En|De)code
2015-06-10 18:51:58 -07:00
1a1832bb35
Add WriteValue and remove support for blobs from top level in json codec
2015-06-10 18:45:54 -07:00
b6291f88e0
Add jsonDecode and top-level enc.ReadValue function
2015-06-10 13:35:06 -07:00
c8f9084ade
Forgot tests for floats
2015-06-10 06:56:18 -07:00
5884e93e3d
Use raw strings instead of normal strings
2015-06-10 06:53:09 -07:00
c63c3ee422
Forgot: don't need to test actual contents
2015-06-09 23:34:41 -07:00
7bed29bcde
Rest of tests for json_encode.go
2015-06-09 23:32:15 -07:00
5640511f7c
fix broken imports
2015-06-09 16:23:08 -07:00
554f7b6fed
wip: added some testing for json_encode.go
2015-06-09 16:21:37 -07:00
9cef1611ce
Added json_encode.go -> untested so far
2015-06-05 01:05:05 -07:00
0d5e3a88a1
Revert "add string_codec.go"
...
This reverts commit 681d48fdb7
2015-06-04 23:42:07 -07:00
681d48fdb7
add string_codec.go
2015-06-04 23:39:50 -07:00
98d6999e63
Rename blob_chunk_codec.go -> blob_codec.go. The "chunk" is not really accurate.
2015-06-04 23:26:10 -07:00
0018b02453
Add enc/blob_chunk_codec.go
2015-06-04 23:23:51 -07:00
Erik Arvidsson