Commit Graph

89 Commits

Author SHA1 Message Date
Erik Arvidsson eeaac87d5f Make Chunks return all reachable Refs from the current Value
We now do a recursive call which bottoms out with a ref.Ref for RefKind
Values. This means that we traverse into nested structures consistently.

The effect of this is that we get all the refs that the current chunk
references.
2015-11-03 17:13:20 -05:00
Erik Arvidsson 30cc7d518f Make Equals compare by Ref (after comparing TypeRef)
This changes equal to compare by ref. Since computing the ref can be
expensive we first check that the type refs are equal.

Fixes #532
2015-10-30 16:50:49 -04:00
Dan Willhite 80be862fda Regroup common code. 2015-10-30 10:36:34 -07:00
Aaron Boodman a9bc416a2f Change Chunks() to return []ref.Ref 2015-10-30 09:24:58 -07:00
Dan Willhite 84b99fbc37 Check in generated files with new names and delete old ones. 2015-10-28 14:28:20 -07:00
Erik Arvidsson 81f80d5ce6 NomDL: Encode compound blobs as structs
Before encoding a compoundBlob we create a compoundBlobStruct and
encode that instead.

After decoding we check if the value is a compoundBlobStruct and if so
we convert it to a compoundBlob.

Fixes #455, #165
2015-10-27 17:53:16 -04:00
Erik Arvidsson 11799dc957 Merge pull request #481 from arv/counter
Don't read the current commit from the datastore
2015-10-26 16:38:32 -04:00
Erik Arvidsson 8702743875 Cleanup based on code review 2015-10-26 16:22:57 -04:00
Erik Arvidsson cdcf952270 Update TypeRef comment and get rid of useless param
The name param of MakeCompoundTypeRef is always the empty string.

I didn't change the underlying storage or serialization.

Fixes #436, #477
2015-10-26 15:35:45 -04:00
Erik Arvidsson 33c88381ee Don't read the current commit from the datastore
We tried to read the current commit from that datastore almost
directly after it was written. Instead of reading it we pass along the
Commit.
2015-10-26 15:03:32 -04:00
Erik Arvidsson 1d13a878c4 Fix types.Ref and RefKind objects
Ref Values now have a TargetRef() method that returns the ref.Ref of
the target the Value is referencing.

Note: This is a breaking change. In old code the Ref() of the Value was
the Ref of the underlying target.

Fixes #464
2015-10-26 11:18:02 -04:00
Erik Arvidsson a60de44b13 Generated code 2015-10-23 11:04:41 -04:00
Erik Arvidsson 71197cb4e7 Generated code 2015-10-22 12:33:39 -04:00
Erik Arvidsson 64217a8548 Generated files 2015-10-22 10:49:06 -04:00
Erik Arvidsson 0b80508047 Fix Commit to use Set(Ref(Commit)) 2015-10-21 19:29:11 -04:00
Erik Arvidsson afeef44020 Updated generated code 2015-10-21 19:04:22 -04:00
Erik Arvidsson 16353f38f8 NomDL: Make the new serialization default
This makes the new typed serialization the default (the old
serialization is not used but the code has not been cleaned up yet).

Some things are no working in the new world:

Chunking -  The compound list is not working correctly any more. The
Chunks method is having issues because it assumed things based on the
old implicit chunking.

Commit - uses a `Set(Commit)` which means that the parent commit is
embedded. We need to change that to be `Set(Ref(Commit))` so that the
parent commit is referenced instead.
2015-10-21 19:04:22 -04:00
Dan Willhite bb2f983446 Add IsEmpty() function on ref.Ref.
Fixes #337.
2015-10-21 12:07:34 -07:00
Erik Arvidsson 91f720be81 NomDL: Need to include the deps in the package 2015-10-21 09:17:35 -04:00
Aaron Boodman e2e51a3ae7 Merge pull request #438 from aboodman/server-port
server: print port listening on at startup
2015-10-20 10:54:33 -07:00
Aaron Boodman 9118530e53 server: print port listening on at startup 2015-10-20 10:28:31 -07:00
Erik Arvidsson d6869cef35 NomDL: Make Package a primitive type
This solves the bootstrapping problems we are encountering.

Previously Package was a Struct but structs depend on Package.
2015-10-20 10:26:00 -04:00
Erik Arvidsson e9f5b1e2b5 NomDL codegen: Clean up write and improve ordering
Now we write the named types first, then the using types and finally
the inline type defs in the order that they were defined.
2015-10-19 11:39:58 -04:00
Erik Arvidsson 36d9362628 NomDL: Use ordinal for unresolved type refs
Instead of using the name of the type we now use the index of the type
in the Package that defines it.
2015-10-16 17:46:15 -04:00
Chris Masone 30422eaa33 Fix a few issues with switching from NamedTypes -> Types and re-run go generate
grammar.peg didn't get updated along with grammar.peg.go in arv's last patch,
so that needed to be fixed. Also, pkg.Parsed had its own field named Types, which
shadowed the one it got by embedded types.Package. This only came into play when
generating code for packages pulled out of a dataset. Since arv had to manually
patch up all generated code in his last patch, he never hit this issue and I
missed it in review.

Now, go generate passes once more. Yay
2015-10-14 14:31:06 -07:00
Erik Arvidsson 6185ea1ddb NomDL: Use a list for the types in the package
This does not yet update the serialization to use the ordinal.
2015-10-14 14:49:04 -04:00
Dan Willhite 43d9a6f4fb Remove commit lineage from datastore head.
Replace datastore head with a map of datasetID's to commits. Each commit in the map represents that dataset's head. Fixes #402. Fixes #60. Filed #404 about small window of potential conflict with updating root that needs to be resolved at some point. # Please enter the commit message for your changes. Lines starting
2015-10-14 11:04:13 -07:00
Erik Arvidsson 8565f175ee Make sure Chunks includes type.Refs and the TypeRef
Chunks shold return the futures for types.Ref values.

For typed values that have a TypeRef which has a package ref, also
include that.
2015-10-13 11:00:51 -04:00
Dan Willhite d6b221444a Simplify logic in doCommit as per Raf's suggestions. Define ref.EmptyRef variable. 2015-10-12 10:53:12 -07:00
Dan Willhite 4852067aad Fix concurrency overwrite on dataset.
When there are two dataset clients working on the same empty dataset. The second one to commit would override the first.
2015-10-12 10:50:28 -07:00
Erik Arvidsson 37336f41be NomDL serializing
This now handles serializing TypeRef values
2015-10-09 14:09:49 -07:00
Erik Arvidsson 9cb7596409 NomDL: Make NomsValue a Value
This means that when we ReadValue we can now return a NomsValue

Towards #281
2015-10-06 16:38:11 -07:00
Erik Arvidsson 2dec53453e NomDL: Add new serialization format
The new serialization format use "t " as in typed. The rest of the
message is a JSON array describing the typed data. The type is
described by types.TypeRef

Fixes #384
Issues #281, #304
2015-10-06 15:56:10 -07:00
Erik Arvidsson 72f4cd3a7a NomDL Codegen: Make the TypeRef return a package ref
The TypeRef function for a Noms Struct should be the (Name, PkgRef) and
not the description of the struct fields. This is important because
when serializing we need to write the package ref.

Towards #281 #304
2015-10-05 14:24:25 -07:00
Dan Willhite 42688454f6 Use channel to protect access to datastore_server's connection map. 2015-10-01 09:17:04 -07:00
Erik Arvidsson 8c2caa3b27 NomDL Codegen: Add TypeRef method to typed objects 2015-09-30 17:48:32 -04:00
Chris Masone 9168a902f9 Modify codegen to include pacakge refs for unresolved types
Also, switch to using a ref.Ref when getting/setting the package
ref in a TypeRef. Using a types.Ref just led to lots of manual
boxing and unboxing every time I wanted to use the reference.

Toward issue #294
2015-09-29 12:58:56 -07:00
Chris Masone 5ce93dad2e Beginning of import support in NomDL
This patch mostly merges parse.Package and types.Package, though it
can't quite go all the way. A types.Package doesn't have 'using'
declarations, while the parsed representation of a .noms file needs to
have that information. Hence, the parse package is moved to the 'pkg'
package, and pkg.Parsed is introduced. This type embeds types.Package
and adds the necessary additional information.

To make inroads on handling imports, I enhanced ParsePackage() (now
called ParseNomDL()) to actually process the 'alias' and 'import'
statements in the input and go replace namespaced type names in the
package with refs of imported packages. For example, the TypeRef for
'Bar' generated in the following package

alias Foo = import "sha1-ffffffff"

struct Bar {
  f: Foo.RockinStruct
}

will actually return types.Ref{sha1-ffffffff} when you call PackageRef()
on it.

In addition, I've added a function to the new 'pkg' package,
which allows the caller to get the dependencies of a type package
from a chunk store.

Fixes issue #353, towards issue #294
2015-09-28 16:08:22 -07:00
Rafael Weinstein 6c5b2eb6f4 Avoid Excess Ref Computation 2015-09-26 11:33:04 -07:00
Chris Masone c80a1c55b3 Merge parse.TypeRef and types.TypeRef
These were two representations of, essentially, the same information.
They were separate because they provided different APIs to similar
information, but the APIs became more similar once we started using
native types (as opposed to Noms types) for the various Make*TypeRef()
functions.

Unifying these is a big step to unifying parse.Package and types.Package,
which is pretty necessary for dealing with imported packages.

Fixes issue #338
2015-09-25 15:17:49 -07:00
Erik Arvidsson 2b9ec7e844 Merge pull request #350 from arv/nomdl-optional-codegen
Nomdl optional codegen
2015-09-25 18:11:44 -04:00
Rafael Weinstein 4ba824739a Protect readBatch map with a rateLimiter 2015-09-25 14:35:37 -07:00
Erik Arvidsson bf095dab3c Merge branch 'master' into nomdl-optional-codegen 2015-09-25 17:20:23 -04:00
Erik Arvidsson d17dc67a73 NomDL CodeGen: Rename self 2015-09-25 17:17:12 -04:00
Erik Arvidsson 5b3f611104 NomDL Codegen: Add optional flag to Field 2015-09-25 14:38:41 -04:00
Chris Masone ace13f2c01 Regenerate datas/types.go to fix test failure 2015-09-24 14:10:43 -07:00
Rafael Weinstein ec8461e8fd RemoteDataStore / CopyReachableChunksP 2015-09-24 11:06:50 -07:00
Erik Arvidsson b67ab2a9e1 Convert datas/ to nomdl/codegen
This make Commit a typed struct with a Set(Commit).

This also fixes a case where the recursive detection for determining
if a Def can be created was not working.
2015-09-24 12:56:58 -04:00
Rafael Weinstein d884cd263b make dataStoreCommon private 2015-09-22 17:46:53 -07:00
Rafael Weinstein 82c4a54bd7 move comments 2015-09-22 17:31:56 -07:00