Commit Graph

2892 Commits

Author SHA1 Message Date
zcstarr
3cdebb7e77 Add run safe method that reads stderr and stdout regardless of panic (#2475)
Run method will now always return stdout,stderr, and a recoveredErr
on Exit or Panic. MustRun will Panic with recoveredErr.
2016-09-06 11:30:57 -07:00
Erik Arvidsson
fd4c52acef stage.py: import os too (#2514) 2016-09-02 14:00:15 -07:00
Erik Arvidsson
893d0fa360 stage.py needs to import sys before using it (#2513)
Also changes the tabs to spaces. Thanks pylint.
2016-09-02 13:38:43 -07:00
Erik Arvidsson
f465b0bd3c Fix import of noms.staging in stage.py (#2510)
Also fix unit test that was not updated when the functions were
renamed.
2016-09-02 13:09:07 -07:00
Eric Halpern
48854dc999 Move datas.descendsFrom and prune search using Ref.Height (#2480)
- Move to commit.CommitDescendsFrom
 - When searching for an ancestor in the commit history, prune
   whenever the commit.Height() < ancestor.Height()
 - Add new test TestCommitDescendsFrom to verify correctness and pruning.
2016-09-02 11:44:38 -07:00
Dan Willhite
57f653dd39 Disable perf suite check for nomsRevision (#2508)
This needs to be disabled so that test cases succeed when
building demo-server. Currently this fails because we build
with CodePipeline so the source code is not in a git repo
when getting built.
2016-09-02 10:53:18 -07:00
Erik Arvidsson
017d599b1d Fix subtype issue with unions as elem types (#2506)
For compound types (List, Set, Map, Ref) the concrete type may be a
union. If that is the case all the types in the union must be a
subtype of the concrete type's element type.

`C<T | V>` is a subtype of `C<S>` iff T is a subtype of S and V is a
subtype of S.
2016-09-01 18:40:51 -07:00
cmasone-attic
e5fcfd6ebf Make poke hang on to parent's Commit metadata (#2504)
Before this, poke would drop any commit metadata from the dataset being modified. Now, it just pulls it forward.
2016-09-01 17:21:31 -07:00
Ben Kalman
ace91a323d Update CONTRIBUTING.md (#2468)
Add a bit more info on squash merges and perf tests.
2016-09-01 16:22:46 -07:00
Ben Kalman
0f6cc2023c Print skipped perf tests in verbose mode (#2497) 2016-09-01 16:22:23 -07:00
Erik Arvidsson
b9f15526ba JS Docs: Redirect to versioned sub directory (#2503) 2016-09-01 16:10:38 -07:00
Ben Kalman
8eba76a2ec Check before calling mkdirs on staging directory (#2502)
Also run pylint on staging.py.
2016-09-01 15:19:13 -07:00
Ben Kalman
6393e7c6e7 Fix "and and" comment from last patch (#2500) 2016-09-01 13:39:08 -07:00
Ben Kalman
b734a80cb1 Improvements to the perf viewer (#2499)
* Increase number of results to 20.
* See newer results at the end of the graph, not start.
* Make point radius show standard deviation to scale.
* Try not to draw datasets over each other.
* Invert light colours to make them darker.
2016-09-01 13:36:53 -07:00
cmasone-attic
9f080a2fa7 Fix test assertion in noms-merge::TestLose() (#2498)
This was caused by me changing an error string and failing
to update the test.
2016-09-01 11:34:04 -07:00
cmasone-attic
771eb092da Add failure tests for noms-merge (#2484)
Tests command line validation for noms-merge

Toward #2445
2016-09-01 11:31:29 -07:00
Erik Arvidsson
456f403b8a Build js docs correctly (#2495)
There was a typo (missing comma) in the command.
2016-09-01 10:38:29 -07:00
Ben Kalman
d1c90d11cd Add prepublish step to perf/viewer so that out.js gets built (#2496) 2016-09-01 10:33:03 -07:00
Aaron Boodman
42fd80be2e Add a super quick indeterminite progress meter to noms-merge (#2488) 2016-09-01 10:24:30 -07:00
Daniel Krech
01bdeab025 Add progress reporting to json_importer (#2494)
Fixes #2494
2016-09-01 00:04:26 -07:00
Aaron Boodman
63f4127aac Update CONTRIBUTING.md 2016-08-31 23:43:22 -07:00
Aaron Boodman
25fedb51fc Update cli-tour.md 2016-08-31 21:37:56 -07:00
Erik Arvidsson
eb42d99950 JS Doc builder (#2493)
Trying to work around issue where `npm run build-docs` fails
2016-08-31 18:27:58 -07:00
Erik Arvidsson
9452898ff7 Revert "Work around python popen bug for js docs" (#2492) 2016-08-31 18:18:16 -07:00
Erik Arvidsson
0a53ee7e3d Work around python popen bug for js docs (#2491) 2016-08-31 18:10:10 -07:00
Erik Arvidsson
ad66582f44 npm run did not work (#2489) 2016-08-31 17:56:12 -07:00
Ben Kalman
9c694f024b Add a perf test for CSV map import (#2461)
Currently we only have a perf test for CSV list import, which uses the
sf-crime dataset. This test uses the 43MB sf-registered-businesses
dataset instead, since sf-crime is too slow. Which is ironic, since we
normally parse sf-crime into a map.

I've also tightened up some of the other perf tests.
- Fixed a bug where Database was shared between runs.
- Make the pure CSV parsing test use a smaller dataset, it doesn't need
  to use something as large as ny-vehicle-registrations.
2016-08-31 17:05:00 -07:00
Erik Arvidsson
b857b95bb4 Drive js docs from npm script (#2486)
Turns out that documentationjs reads the babelrc file from js/noms
which leads to us having to do npm install in js/noms. Therefore
putting the dev deps in there and running the script from there
makes things cleaner.

It also allows doing:

  npm run build-docs

Towards #1471
2016-08-31 16:57:31 -07:00
Ben Kalman
815ca1586f Add perf build.py and util (#2487) 2016-08-31 16:50:34 -07:00
Ben Kalman
1e8ae3f214 Make perf viewer point radius show standard deviation (#2485) 2016-08-31 16:42:59 -07:00
Mike Gray
733a1fd97a Update spelling.md with more syntax (#2450) 2016-08-31 19:25:08 -04:00
Erik Arvidsson
7c47547110 Add node to the PATH for documentation build (#2482) 2016-08-31 15:43:43 -07:00
Erik Arvidsson
adff0da9c3 Build script for building JS SDK Docs (#2479)
This script npm installs documentationjs and runs it on noms.js and
puts the output in $WORKSPACE/build

It is intended to be used with Jenkins.
2016-08-31 14:51:23 -07:00
Ben Kalman
70c43f4e28 Add stage.py for perf viewer (#2478) 2016-08-31 14:29:17 -07:00
cmasone-attic
49ea5ec3c0 Introduce noms-merge, a standalone noms tool for merging datasets (#2470)
This is a first pass at exposing the new merge package to users.  The
tool is very basic, and currently only works on datasets in the same
database. It requires the 'parent' (i.e. a common ancestor of the two
datasets being merged) to be provided by a commandline option; a
follow-on patch will make the code discover this ancestor automatically.

Toward #2445
2016-08-31 14:20:21 -07:00
Ben Kalman
aaad3601db Tweak number of reps and timeout of perf bot script (#2462) 2016-08-31 11:27:44 -07:00
Aaron Boodman
b279f08057 Update CONTRIBUTING.md 2016-08-31 11:15:24 -07:00
Aaron Boodman
10944bf23b Update CONTRIBUTING.md 2016-08-31 11:05:51 -07:00
Aaron Boodman
4227aae20c Update CONTRIBUTING.md 2016-08-31 11:04:10 -07:00
Erik Arvidsson
f5ce7e056b Fix path in splore build script (#2471)
The path to the noms sdk in the build script was not updated when the
js sdk was moved.
2016-08-31 12:01:10 -04:00
Ben Kalman
572f2dcc0d Make the Go doc for go/perf/suite look idiomatic (#2469)
I hope.

We should stop using Markdown syntax in our public Go docs. godoc
doesn't recognise it (https://godoc.org/github.com/fluhus/godoc-tricks
is a reasonable summary of what *is* supported).

This patch:
 - Wraps comments at 100 characters, because godoc doesn't do wrapping.
 - Replaces triple-backtick blocks with indented blocks.
 - Removes single-backtick lines entirely.
 - Lines things up better.
 - An unfortunate side-effect is bullet points are monospaced in
   godoc.org, but there isn't a way around this.
2016-08-30 17:12:38 -07:00
Ben Kalman
b5098227b8 Add a perf viewer (#2434)
This is a web page which renders a graph of the data generated by perf
suite tests.
2016-08-30 16:11:21 -07:00
zcstarr
aeb5c42bcc Add special encoding to csv imported struct fields (#2441)
CSV importing is changed to strip invalid characters from csv fields 
and camel case spaces. i.e. ca-mel case is translated to camelCase.
2016-08-30 14:59:10 -07:00
cmasone-attic
cba7a4f118 Merge pull request #2467 from cmasone-attic/issue148
In merge.ThreeWay(), use Diff() across the board
2016-08-30 13:48:50 -07:00
Chris Masone
2d6cfbac20 In merge.ThreeWay(), use Diff() across the board
Initially, this code used DiffLeftRight() for some kinds
of collections. Standardize on Diff().

Towards #148
2016-08-30 13:10:46 -07:00
cmasone-attic
3830ed5040 Clean up test code in walk package (#2465)
Remove some unneeded helper functions
2016-08-30 12:26:38 -07:00
Aaron Boodman
24c99ae3b5 Introduce poke: a simple tool for modifying Noms data (#2449)
Introduce poke: a simple tool for modifying Noms data
2016-08-30 00:24:38 -07:00
cmasone-attic
36bfaa77f3 Add types.List support merge.ThreeWay() (#2448)
While the code in threeWayListMerge() starts similarly to that
in threeWayOrderedSequenceMerge(), the types of the channels
are different and trying to shoehorn them together would
remove a lot of clarity just for the sake of sharing maybe
15 lines of code.

Toward #148
2016-08-29 21:20:29 -07:00
Aaron Boodman
322ac4344d Update CONTRIBUTING.md 2016-08-29 16:53:17 -07:00
Aaron Boodman
4cb659f9c3 Update CONTRIBUTING.md 2016-08-29 16:11:28 -07:00