Commit Graph

2820 Commits

Author SHA1 Message Date
Mike Gray
4e54c44d56 no functional changes, improving code quality (#2410)
fix misspellings; fix code that was not gofmt'd - plus take advantage of gofmt -s too; couple of unreachable golint reported fixes; reference go report card results and tests
2016-08-23 13:51:38 -04:00
Dan Willhite
7b3adfe657 Add support to nomdex to query against multiple indexes. (#2396)
Fixes #2405
Fixes #2405
Fixes #2114 (Implement nomdex)
2016-08-23 10:26:11 -07:00
Erik Arvidsson
99eb3306b6 Go Marshal: Title case struct names (#2408)
Make the name of the Noms struct start with a capital letter.

When unmarshalling a struct we now ignore the case.

This is a breaking change. If you previously marshalled a Go struct with a lower case name (ie "abc") the Noms struct we generate now has the name "Abc" (previously the name was "abc")

Towards #2376
2016-08-23 10:05:50 -07:00
Erik Arvidsson
0d6e9a56f7 Marshal and Unmarshal handling of interface{} (#2391)
When unmarshalling onto interface{}` the following rules are used:
 - `types.Bool` -> `bool`
 - `types.List` -> `[]T`, where `T` is determined recursively using the same rules.
 - `types.Map` -> `map[T]V`, where `T` and `V` is determined recursively using the same rules.
 - `types.Number` -> `float64`
 - `types.String` -> `string`
 - `types.Union` -> `interface`
 - Everything else an error

Towards #2376
2016-08-22 16:52:01 -07:00
Ben Kalman
892f98050e Make perf suite testdata path configurable by -perf.testdata flag (#2406)
Needed by Jenkins shared workspaces.
2016-08-22 16:40:27 -07:00
Dan Willhite
d162173d90 Change SetIterator function name from NextFrom() to SkipTo() (#2401) 2016-08-22 16:06:47 -07:00
Erik Arvidsson
c6ad845129 Update README.md 2016-08-22 15:22:34 -07:00
Erik Arvidsson
d0d1d31efb JS: Fix OrderedPutCache for browser (#2398)
Fixes #2395
2016-08-22 14:45:23 -07:00
Erik Arvidsson
0d2c4fd56d Revert "Move noms/samples/js/perf to noms/js/perf" (#2400) 2016-08-22 13:57:41 -07:00
Mike Gray
22bc81e355 adding godoc synopsis for several top level packages (#2394) 2016-08-22 13:50:31 -04:00
Erik Arvidsson
ecbdd2412b Use Go marshal API in the HR sample 2016-08-22 10:38:20 -07:00
Erik Arvidsson
f41e413fe2 Merge pull request #2397 from sunglim/move_perf_js
Move noms/samples/js/perf to noms/js/perf
2016-08-22 10:28:31 -07:00
Ben Kalman
b350b13303 Use new Go-to-Noms map marshalling in perf suite (#2392) 2016-08-20 18:43:37 -07:00
LIMSUNG GUK
0c99f194d2 Add config files for js/perf/ 2016-08-20 20:56:47 +09:00
Dan Willhite
9edb24905b Set, Union, & Intersection iterators. (#2383)
Implements #2294. (Set Union Iterator)
Implements #1851. (Set Itersection Iterator)
Towards #2114 (Nomdex with multiple indexes)
2016-08-19 15:47:10 -07:00
cmasone-attic
eaf3240438 Merge pull request #2388 from cmasone-attic/issue148
Go: Implement best-effort three-way merge for Maps
2016-08-19 15:08:44 -07:00
Chris Masone
ad0da144c5 Go: Implement best-effort three-way merge for Maps
Add the Go 'merge' package, which exports one function called ThreeWay().
ThreeWay() attempts a three-way merge between two candidates and a common
ancestor. It considers the three of them recursively, applying some simple rules:

- If any of the three nodes are different kinds: conflict
- If we are dealing with a map:
  - If the same key is both removed and inserted wrt parent: conflict
  - If the same key is inserted wrt parent, but with different values: conflict
- If we are dealing with a struct:
  - same as map
- If we are dealing with a list:
  - Same as map but substitute "index" for "key"
- If we are dealing with a set:
  - If the same object is both removed and inserted wrt parent: conflict
... otherwise, the concurrent modification is allowed.

Currently, ThreeWay() only works on types.Map.

Towards #148
2016-08-19 14:57:56 -07:00
Mike Gray
1197b4127f csv-analyze to detect column types and primary keys (#2366)
create csv-analyze; add detection of PKs; move detect-columns from csv-import to csv-analyze
2016-08-19 14:24:32 -04:00
Erik Arvidsson
1728070df1 Map marshal - cleanup test and commment (#2390) 2016-08-18 18:02:53 -07:00
Erik Arvidsson
78bb271d3f Go marshal maps (#2389)
Go maps are marshaled to Noms Map.

 Towards #2376
2016-08-18 17:54:16 -07:00
Ben Kalman
064c398dec Add perf test suite infrastructure, and a perf test for csv-import (#2384)
You can run these yourself using the -perf flag, e.g.

> noms serve &
> go test -v -perf http://localhost:8000 ./samples/go/csv/csv-import
> noms ds http://localhost:8000

Though you'll need to go-get github.com/attic-labs/testdata.

Note that all of this only records test results, it doesn't have any
concept of failing perf (unless test assertsions themselves fail). It
will be the job of some other Noms client (work in progress) to do that.

I will be setting this up to run continuously momentarily.
2016-08-18 15:49:21 -07:00
LIMSUNG GUK
89fb45553b Move sample/js/*-perf-rig to js/perf/ 2016-08-18 18:33:19 +09:00
Erik Arvidsson
15d80f9afd Go marshal slices and arrays (#2381)
* Go marshal slices and arrays

Slices and arrays are marshaled to Noms List. For arrays we also enforce
that the length is the same.

Towards #2376
2016-08-17 14:46:50 -07:00
Ben Kalman
8a92b75995 Add missing ==nil check to list diff (#2362)
This isn't triggering any problem in particular, I just noticed it. All
it meant was that noms log might take unnecessarily long with list diff
when there are a lot of changes.
2016-08-17 14:44:21 -07:00
Dan Willhite
218c98f209 Nomdex - for indexing and querying. (#2357)
This first version builds indexes and can query against a single one.
2016-08-17 14:42:34 -07:00
Adam Leventhal
b42043118a add Map.Last() (#2382) 2016-08-16 23:33:52 -07:00
Ben Kalman
a28f5ddaf9 Render diff struct field paths .foo not ["foo"] (#2379) 2016-08-16 15:28:47 -07:00
Erik Arvidsson
f5433ec1b7 JS: Do not shadow struct properties (#2378)
When we create JS struct classes we no longer creates properties for
`chunks`, `hash` or `type` (nor for `toString`, `hashOwnProperty` etc)

If you are using structs with field names that clashes with these then
use a `StructMirror`.

Fixes #2332
2016-08-16 15:12:34 -07:00
Erik Arvidsson
3320518f88 Fix shell script flags (#2377) 2016-08-16 11:58:57 -07:00
Erik Arvidsson
c013f1ea1c Go marshal (#2374)
Basic support for creating Noms value from Go values as well as reading
Noms values onto Go values.

Typical usage:

```
// Go to Noms
type T struct {
  S string
  N int32
}
nomsVal, err := marshal.Marshal(T{"Hi", 42})

// Noms to Go
var goVal T
err = marshal.Unmarshal(types.NewStruct("T", types.StructData{
  "S": types.String("Bye"),
  "N": types.Number(555),
}, &goVal)
```

Fixes #1591
2016-08-16 11:47:32 -07:00
Erik Arvidsson
4532c954c0 Update CONTRIBUTING.md 2016-08-16 11:45:02 -07:00
Aaron Boodman
a100b28076 Update CONTRIBUTING.md 2016-08-16 11:44:15 -07:00
Sungguk Lim
ed549000f1 Updating CONTRIBUTING.md (#2355)
* Describe how to test in CONTRIBUTING.mD

* Adding a way to run js test in CONTRIBUTING.md

* Move 'hacking on noms' contents to CONTRIBUTING.md
2016-08-16 11:42:19 -07:00
Ben Kalman
9079b83927 Use best set/map diff in noms diff (#2372)
In 7c103344 I changed noms log to use left-right diff so that it
completes faster, but it changed noms diff to use left-right as well.
Noms diff is supposed to use best diff.
2016-08-15 17:15:12 -07:00
cmasone-attic
2c63b7cf69 Merge pull request #2371 from cmasone-attic/issue148
Update FAQ to address conflict resolution
2016-08-15 11:03:18 -07:00
Aaron Boodman
c8b0d4c680 Add better error reporting to HTTPBatchStore::UpdateRoot() (#2369)
* Add better error reporting to HTTPBatchStore::UpdateRoot()

* Add logging at places where we return http.BadRequest
2016-08-15 11:02:28 -07:00
Chris Masone
b5c4d784bb Update FAQ to address conflict resolution
Towards #148
2016-08-15 10:56:37 -07:00
Jang-Soo "Bruce" Lee
7f17329ed9 Add a friendly message to the base path of the noms server, as opposed to giving an unfriendly 404 message (#2365) 2016-08-13 15:22:07 -07:00
Jang-Soo "Bruce" Lee
f3dc44a3b1 fix error message (#2364) 2016-08-12 18:08:32 -07:00
Ben Kalman
1e511a61c6 Let Splore open a noms path (#2356)
Currently Splore can only open a database (db=) or a hash (hash=). This
removes the hash and replaces it with the much more general path (p=).

For example, what used to be:
  ?db=http://demo.noms.io&hash=abcd
is now:
  ?db=http://demo.noms.io&p=#abcd

and of course more complex paths work, like:
  ?db=http://demo.noms.io&p=#abcd.field[42].name

I've exposed getHashOfValue in noms.js.
2016-08-12 16:47:26 -07:00
Ben Kalman
dac857724d README.md: mention that $GOPATH/bin needs to be in your $PATH (#2359) 2016-08-12 15:52:18 -07:00
Ben Kalman
da62bb2b97 Remove most references to ldb: prefix in the .md docs (#2360) 2016-08-12 15:51:25 -07:00
Dan Willhite
e975427c02 Merge pull request #2346 from willhite/iter-range
Add IterFrom() function to Map.
2016-08-11 16:41:39 -07:00
Dan Willhite
c21d67cbc1 Add IterFrom() function to Map.
Towards #2114 (nomdex)
2016-08-11 16:29:53 -07:00
Eric Halpern
c5ccae3852 Fix noms_sync status message to indicate when new dataset is created (#2276)
* Fix noms_sync status message to indicate when new dataset is created

Originally, syncing a commit to a non-existent dataset and an already sync'ed dataset resulted in the same message: "<src> is up to date"

This fix distinguishes 2 cases:

  - Syncing to a new dataset prints "<dest> created"

  - Syncing to an identical dataset prints "<dest> already up to date"

Resolves: #2053

* Make message when noms sync creates new dataset more fun

Addresses: #2053
2016-08-11 16:17:15 -07:00
Ben Kalman
0d6bd01810 Port PathSpec implementation to JS (#2335)
This is a breaking change to the JS API. HashSpec and parseObjectSpec
have been removed in favour of PathSpec, and parse errors now throw
SyntaxError instead of silently failing (this matches JSON.parse).
2016-08-11 15:29:27 -07:00
Dan Willhite
695bc5bcba Merge pull request #2289 from willhite/sets
Add streaming set and improve streaming collection performance.
2016-08-11 15:15:15 -07:00
Dan Willhite
0dce4fd468 Add streaming set and improve streaming collection performance.
Streaming collections now share one ldb instance rather than having 1 ldb instance per collection.
Towards: #2114 (nomsdex)
2016-08-11 14:46:29 -07:00
Mike Gray
865e977320 add csv-import --detect-column-types option (#2329) 2016-08-11 17:02:05 -04:00
Mike Gray
b284c7ce03 removing shell script (#2347) 2016-08-11 16:49:00 -04:00