Commit Graph

295 Commits

Author SHA1 Message Date
Erik Arvidsson 84b4aba5a6 HRS: Prefix map/set values with map/set (#3655)
We now print map and set values as:

```
map {
  "string": 42,
  "set": set {
    true,
    false,
  },
}
```

Towards #1466
2017-08-31 13:31:21 -07:00
Erik Arvidsson e5bcde644a HRS: Write blob values as blob { ... } (#3654)
This prints blob values as:

```
blob {  // 17 B
  00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f
  10
}
```

Towards #1466
2017-08-30 15:16:22 -07:00
Erik Arvidsson 864f1a8fae HRS: Struct printing cleanup (#3653)
We always write the struct name now so no point in passing the
param.
2017-08-30 11:05:42 -07:00
Erik Arvidsson a93777d93a HRS: Cleanup struct head (#3647)
For struct values we always prefix with struct.

For struct types we always prefix with Struct.

Towards #1466
2017-08-29 17:47:54 -07:00
Erik Arvidsson 9cd2aae786 Print Refs with # (#3644)
Print Ref values as #123 instead of 123

Since our hashes are SHA-512 and we write them using Base32 there are a lot of overlaps with other parts of NomDL. This makes them unambiguous.

Towards #1466
2017-08-29 17:43:29 -07:00
Erik Arvidsson 9200aabaa7 Remove HRS tagged versions (#3643)
This removes the type tagged version of the human readable encoding.

Motivation: Simplify this in preparation to make the HRS unambiguous so that we can write a parser.

Towards #1466
2017-08-29 17:38:46 -07:00
Rafael Weinstein c3f98d1631 Remove in mem graphs (#3635)
This patch removes the ability to keep alive uncommitted prolly-tree sequences.
2017-08-29 13:12:10 -07:00
Rafael Weinstein 61f3d87dcf Introduce Sloppy (#3631)
Introduce Sloppy - an estimating compression function for snappy - which allows for the rolling hash to better produce a given target chunk size after compression.
2017-08-28 13:23:00 -07:00
Ben Kalman f23cbe5344 Make field and index paths work for types (#3639)
Struct type field support field paths, like `.fieldName`
Compound types (List/Set/Map/Union) support index paths, like `[0]`

Fixes https://github.com/attic-labs/noms/issues/3622
2017-08-25 12:23:59 -07:00
Rafael Weinstein a95b2ba9ff Collection test (#3624)
* loadLeafSeqs => loadLeafCollections

* Collection tests examine leaf node counts
2017-08-18 13:40:54 -07:00
Dan Willhite 0c14fbad05 Ensure NewStreamingMap and NewStreamingSet panic when input is not ordered (#3570)
Fixes #3560
2017-08-16 11:37:40 -07:00
Rafael Weinstein f290a711c2 BlobEditor (#3599) 2017-07-25 13:40:49 -07:00
Rafael Weinstein cd0c27a565 minor test cleanup (#3600) 2017-07-24 16:01:36 -07:00
Rafael Weinstein 44941ee44c Refactor Blob reading (#3593) 2017-07-19 15:01:44 -07:00
Rafael Weinstein 22e5b323e4 List Editor (#3586) 2017-07-13 15:37:48 -07:00
Rafael Weinstein fe0fc3ad86 Introduce SetEditor, Nestable Editors (#3557) 2017-06-21 15:22:04 -07:00
Rafael Weinstein 763a87aa60 Reland advance to fix (#3555) 2017-06-20 12:45:51 -07:00
Rafael Weinstein 0c3af1c1c5 Revert "sequenceChunker.advanceTo avoid resuming over items it already consumed (#3553)" (#3554) 2017-06-20 12:39:58 -07:00
Rafael Weinstein d2cc508729 sequenceChunker.advanceTo avoid resuming over items it already consumed (#3553) 2017-06-20 12:34:47 -07:00
Rafael Weinstein 2035657441 remove dead code (#3549) 2017-06-16 15:54:08 -07:00
Rafael Weinstein 70d9564b18 SequenceCursor.advanceTo shouldnt step out of parent sequence (#3548) 2017-06-16 09:56:47 -07:00
Rafael Weinstein e0f68b08bd Minor fixes to MapEditor (#3547) 2017-06-15 17:28:49 -07:00
Rafael Weinstein 1796d2f865 Streaming p tree updates + Map Editor (#3545) 2017-06-15 15:49:04 -07:00
Rafael Weinstein 7251afe285 dont clone cursors in sequence chunker (#3541) 2017-06-14 11:59:18 -07:00
Rafael Weinstein 70f145e3de trivial sequence chunking refactor (#3540) 2017-06-13 19:21:00 -07:00
Rafael Weinstein d675e4d8f4 Chunking V2 (#3521) 2017-06-13 10:48:03 -07:00
Rafael Weinstein 3ff92950d8 Revert removal of |last| from Commit() (#3531) 2017-06-09 11:20:45 -07:00
Rafael Weinstein 214054986b Enforce clearer concurrency semantics of ValueStore (#3527) 2017-06-08 11:40:22 -07:00
Rafael Weinstein 3863e91c17 Remove WalkDifferentStructs (#3528) 2017-06-07 18:56:14 -07:00
Rafael Weinstein 81769cd7b3 move flushing of uncommitted ptree nodes out of EncodeValue (#3524) 2017-06-07 11:21:56 -07:00
Rafael Weinstein 3df9e116d6 narrow semantics of value_store value_cache (#3523) 2017-06-06 15:05:54 -07:00
Dan Willhite 323edf618a Add hasExtra return value to IsValueSubtypeOf() (#3483)
* Add hasExtra return value to IsValueSubtypeOf()

* cr changes

* cr changes
2017-05-19 16:06:03 -07:00
Dan Willhite dafd868471 Fix intermittent test bug TestStreamingSet2 and TestStreamingMap2 (#3487) 2017-05-18 16:58:27 -07:00
Rafael Weinstein 6eac51c708 map mutation test (#3488) 2017-05-18 16:37:23 -07:00
Rafael Weinstein 0570d0c565 add noms show --stats (#3486) 2017-05-18 15:24:22 -07:00
Dan Willhite f196e50e03 Remove vestigial 'verbose' argument to NewGraphBuilder (#3482) 2017-05-17 15:57:49 -07:00
Dan Willhite 0aac4be0bd Stop using graphbuilder in NewStreamingMap and NewStreamingSet (#3481) 2017-05-17 15:27:52 -07:00
Dan Willhite b41e694eda Buffer out-channel of streaming collections to prevent blocking (#3469) 2017-05-10 08:47:26 -07:00
Rafael Weinstein 72b29042c9 EmptySequenceDiff (#3470)
* try again

* cr changes
2017-05-09 18:17:34 -07:00
Rafael Weinstein 9654d372a2 OrderedSequenceDiff ValueChanged adds NewValue/OldValue (#3468) 2017-05-09 16:43:04 -07:00
Rafael Weinstein 78930b2487 expose isLeaf() as a more direct signal on sequence (#3465) 2017-05-09 13:23:55 -07:00
Rafael Weinstein fe13b58a8b Is subtype disallow extra struct fields (#3445) 2017-05-02 15:35:39 -07:00
Erik Arvidsson 9a7ba92550 Nil is not a valid Value (#3442)
Add checks for nil in Struct and add tests for the collections too.

Fixes #3439
2017-05-01 10:26:23 -07:00
Erik Arvidsson 3c01006990 Subtype: Make sure we continue to test after optional (#3441)
In the case where we have an optional field that is not present we still
need to check the remaining fields.
2017-04-28 16:55:33 -07:00
Dan Willhite 4b47592a53 Use one ldb instance per graph builder (#3435)
Remove unused opCache method from virtual store.
Also remove unused setMutator and mapMutator classes.

Fixes https://github.com/attic-labs/attic/issues/1562
2017-04-28 15:50:01 -07:00
cmasone-attic 98e408a0d0 Add Database.Rebase() (#3420)
All this really does is tell the underlying ChunkStore
to go fetch the current root from persistent storage
and drops the now-out-of-date Dataset map on the floor.

Fixes https://github.com/attic-labs/attic/issues/1157
2017-04-24 14:17:08 -07:00
cmasone-attic fef871c1a7 ValueStore.Flush() no longer persists Chunks (#3416)
ValueStore.Flush() now Puts all Chunks buffered in the ValueStore
layer into the underlying ChunkStore. The Chunks are not persistent
at this point, not until and unless the caller calls Commit() on
the ChunkStore.

This patch also removes ChunkStore.Flush(). The same effect can be
achieved by calling ChunkStore.Commit() with the current Root for both
last and current.

NB: newTestValueStore is now private to the types package.
The logic is that, now, outside the types package, callers
need to hold onto the underlying ChunkStore if they want to
persist Chunks.

Toward #3404
2017-04-21 17:30:56 -07:00
Erik Arvidsson fff2d75481 Make Marshal use StructTemplate (#3413) 2017-04-21 16:27:37 -07:00
cmasone-attic 16ef8884a7 Make MemoryStore come correct (#3406)
It's important that MemoryStore (and, by extension TestStore)
correctly implement the new ChunkStore semantics before we go
shifting around the Flush semantics like we want to do in #3404

In order to make this a reality, I introduced a "persistence"
layer for MemoryStore called MemoryStorage, which can vend
MemoryStoreView objects that represent a snapshot of the
persistent storage and implement the ChunkStore contract.

Fixes #3400

Removed Rebase() in HandleRootGet, and added ChunkStore
tests to validate the new Put behavior more fully
2017-04-21 14:13:52 -07:00
Rafael Weinstein 5d6032a9aa Add types.MakeStructTemplate (#3412)
Add types.MakeStructTemplate
2017-04-20 14:41:57 -07:00