Commit Graph

3377 Commits

Author SHA1 Message Date
Erik Arvidsson
cc77359e65 Add Set.IteratorAt and Set.IteratorFrom (#3228)
These two were missing for Set
2017-02-24 16:57:54 -08:00
Erik Arvidsson
046d842181 Update to Jest 19 (#3226) 2017-02-24 12:46:17 -08:00
Eric Halpern
a355d619fc Implement TypesIntersect(a, b *types.Type) bool (#3223)
* Implement `TypesIntersect(a, b *types.Type) bool`

fixes: #3203

* Respond to review:
- Require at least one key type to match for map intersction
- Do parallel scan of field when comparing struct
- Add more tests for unions and nested collections

* Rename to ContainCommonSupertype
2017-02-24 11:23:47 -08:00
Aaron Boodman
1f25ceced8 Introduce Map.Iterator() and friends (#3225)
Introduce Map.Iterator() and friends.
2017-02-24 11:10:12 -08:00
Aaron Boodman
dc41306cf2 Update README.md 2017-02-23 22:45:22 -08:00
Aaron Boodman
ace7527396 Update README.md 2017-02-23 22:44:51 -08:00
Aaron Boodman
e8b93dea34 Update nbs/README.md (#3224) 2017-02-23 22:37:37 -08:00
Aaron Boodman
d1244c16d2 Update CONTRIBUTING.md 2017-02-23 14:15:37 -08:00
Dan Willhite
2dadf814d2 Add marshal.MustUnmarshal() function (#3220) 2017-02-23 14:11:00 -08:00
Eric Halpern
7ae6264782 Implement unionTypes(a, b *types.Type) *types.Type (#3204) (#3214)
* Implement `unionTypes(a, b *types.Type) *types.Type`

Implements type merging  building off of makeSimplifiedUnion.
The difference between siplified unions and merged types  are
in how structs are handled. In the former, we produce the intersection
of input structs to ensure the simplified type is a supertype of the input
types. In the later, we produce the union of the input structs which leads
to a type that is a subtype of all inputs.

Response to review and discussion
- Rename to MakeFullUnion to MakeMergedType
- Revert to old file naming
- Rename makeSimplifiedType to makeSupertype
- MakeUnion delegates to makeSupertype. Should prabably be be renamed, but
  this touches a lot of files, so keeping same for now.

fixes: #3204

* Rename to makeSimplifiedType and makeSimplifiedType2
2017-02-23 13:17:59 -08:00
Erik Arvidsson
310742211c Marshal to type (#3221)
This takes a Go value and reads the Go type from that and returns a Noms
type for that.
2017-02-23 13:09:37 -08:00
Ben Kalman
c1afbb9013 Fix ngql test (#3217) 2017-02-22 15:34:57 -08:00
Ben Kalman
449b6d6b8f Use attic-labs/graphql (#3216) 2017-02-22 15:04:33 -08:00
cmasone-attic
b1e918d1d4 Share s3, dynamodb clients (#3212)
These objects manage their own pools of HTTP connections and
other resources, so it's generally best to share them
process-wide if you can.

Fixes #3027
2017-02-22 13:41:23 -08:00
Eric Halpern
5b63cf6873 Ignore .nomsconfig 2017-02-22 12:40:56 -08:00
Aaron Boodman
4c55f2d64e noms log: add ability to log any path, not just dataset (#3202) 2017-02-21 23:12:11 -08:00
Ben Kalman
88c24324e4 Upgrade all extra JS and samples to Flow 0.39 (#3211) 2017-02-21 17:14:22 -08:00
Aaron Boodman
2a61225de3 Update README.md 2017-02-21 15:28:46 -08:00
Aaron Boodman
06bbabd29d Update README.md 2017-02-21 15:28:15 -08:00
Aaron Boodman
8a353cda95 Update README.md 2017-02-21 15:27:38 -08:00
Aaron Boodman
0f2291477c Update README.md 2017-02-21 15:27:07 -08:00
Aaron Boodman
3ef9dc2a75 Update README.md 2017-02-21 15:26:26 -08:00
Aaron Boodman
a496c53897 Update README.md 2017-02-21 15:25:24 -08:00
Ben Kalman
9617c5c12a Make AsyncIterator a type, upgrade flow to 0.39 (#3209) 2017-02-21 14:03:26 -08:00
Aaron Boodman
4ea3981ba6 Update README.md 2017-02-21 13:13:44 -08:00
Aaron Boodman
784ff847ff Update README.md 2017-02-21 13:12:02 -08:00
Aaron Boodman
6a5dc86714 Update README.md 2017-02-21 13:08:47 -08:00
Aaron Boodman
ba90efec1b Update README.md 2017-02-21 13:04:20 -08:00
Aaron Boodman
c1487cca25 Refresh of the Noms homepage (#3208) 2017-02-21 13:01:06 -08:00
Aaron Boodman
4eaa0ba353 Remove Spec.DatasetName and Spec.Spec (#3102)
Remove Spec.DatasetName and Spec.Spec.

They were duplicating state that already existed elsewhere and easily
got out of sync.
2017-02-21 11:26:12 -08:00
Erik Arvidsson
edb81e4d15 Fix NPE in GraphQL (#3201)
When we have a cyclic type we ended up getting the non complete type out
of the map.

Use graphql.FieldsThunk which is designed to allow recursive types.
2017-02-17 14:01:41 -08:00
Ben Kalman
a020555eaf Propagate not found as GQL errors (#3200) 2017-02-17 09:32:31 -08:00
cmasone-attic
8b284a7c1e Revert "Log when s3TableReader hits the rate limiter" (#3199) 2017-02-16 15:48:45 -08:00
cmasone-attic
3391a5549c Roll aws-sdk-go (#3198)
Pull in https://github.com/aws/aws-sdk-go at 2d3b3bc3aae6a09a9b194aa6eb71326fcbe2e918
2017-02-16 15:00:27 -08:00
cmasone-attic
0cbd4b3809 Revert "Drop s3 read rate limit to 768" (#3197) 2017-02-16 11:51:16 -08:00
cmasone-attic
4069dc2109 Drop s3 read rate limit to 768 (#3196) 2017-02-16 11:44:55 -08:00
cmasone-attic
9798d6aab5 Log when s3TableReader hits the rate limiter (#3195)
We're getting some TCP hangups when trying to talk to S3.
We have an S3 reading rate limiter that's supposed to
prevent issues like this, so the question is whether that's
set too high. Rather than just turning the knob and seeing
if things are OK, this patch adds some logging to verify
that we're actually hitting the rate limiter.
2017-02-16 10:32:13 -08:00
Erik Arvidsson
cb520e76fa Fix GraphQL to use unique type names (#3194)
The names of GraphQL types needs to be globally unique. We therefore
name struct types as NAME_HASH (where hash is the first 6 chars of
the noms hash).

Also, make sure that we always map the noms type to the same GraphQL
type, even if the boxedIfScalar is true.

Fixes #3161
2017-02-15 17:24:38 -08:00
Rafael Weinstein
83b657fe62 Remove LevelDBStore (#3193)
Remove LevelDBStore
2017-02-14 21:52:25 -08:00
Rafael Weinstein
9527907674 Nbs local store factory (#3191)
Add NBS LocalStoreFactory
2017-02-14 20:52:30 -08:00
cmasone-attic
5025a45b0b Remove usages of LevelDBStore (#3190)
NBS is stable enough that we've made it the default store for command
line tools, and the go-to store for tests that require temporary, but
persistent, storage.

We intend to remove support for LevelDB-backed chunk storage
completely ASAP. This patch removes all usage of LevelDBStore from
noms.git, but doesn't remove LevelDBStore _just_ yet as there are
still some dependencies on it elsewhere.

Toward #3127
2017-02-14 19:49:23 -08:00
cmasone-attic
53d343a546 NBS: tableSet.Flatten() must exclude empty tables (#3189)
I missed this in the compaction patch :-/ I caught it in another
test when the code panic'd while trying to write a manifest with
an empty table in it. So at least it got caught there?
2017-02-14 10:46:43 -08:00
zcstarr
1898f2a588 Fix graphql css reference (#3188) 2017-02-13 11:31:17 -08:00
Sungguk Lim
3e0056a5fa Fix nitty typo from counter sample README. (#3187)
Delimiter should be `::` instead of `:`.
2017-02-13 10:05:23 -08:00
zcstarr
3f973f593d Fix staging folder to correspond to build folder (#3185) 2017-02-13 09:33:57 -08:00
Rafael Weinstein
abfaf36ae7 fix typo (#3171) 2017-02-10 13:41:16 -08:00
cmasone-attic
e836e003c5 NBS: Store total uncompressed data size in NBS tables (#3170)
BUG 3156 is caused by the compaction code trying to estimate the
maximum possible table size for chunk data pulled from a bunch of
existing tables. The problem was that we only had _compressed_ data
lengths for the chunks in existing tables, so we were drastically
underestimating the worst-case space that we might need during
compaction.

The fix is to have tables store the total number of _uncompressed_
bytes that were inserted, so that the compaction code can use this to
get the right estimate when putting together a bunch of tables.

Fixes #3156
2017-02-10 12:12:38 -08:00
Erik Arvidsson
651c140472 Add missing content type header to writeValue (#3169)
When we do a writeValue we do a POST with a request body of some binary
data. Some clients (RN) do not set the content-type header which then
leads to failure on our server.
2017-02-10 11:29:47 -08:00
cmasone-attic
7289c851f3 Work around issue #3156 (#3168)
While I dig back in to figure out how to get the debugging
info we need, work around this problem to unblock others.
2017-02-10 09:10:47 -08:00
Rafael Weinstein
140de8081c Add config prompt to graphql ui (#3167) 2017-02-09 20:51:24 -08:00