Commit Graph

2912 Commits

Author SHA1 Message Date
Erik Arvidsson 4082fb14c4 Add a bare bones landing page to docs.noms.io (#2547)
Towards #2532
2016-09-12 14:08:45 -07:00
Erik Arvidsson d9ebf6ac90 flickr/slurp: Treat fail as an error (#2544)
Reject the promise when we get a fail status.
2016-09-12 13:57:33 -07:00
Erik Arvidsson 0b2eb226bc Generate js docs in /js/ (#2543)
Also keep the hash when redirecting

Towards #2532
2016-09-12 13:11:57 -07:00
Adam Leventhal df677ad0b4 noms diff should exit after the pager exits (#2526) 2016-09-09 21:46:00 -07:00
Erik Arvidsson d3198b9ba3 Update to latest documentation.js (#2540)
This picks up my commits to discover all exported bindings as well as
sorting the docs alphabetically.

Towards #1471
2016-09-09 14:01:27 -07:00
Erik Arvidsson c1ed55e009 Fix JSDoc (#2539) 2016-09-09 11:32:24 -07:00
Mike Gray 1996e0a3d8 Add Noms commit command (#2474)
* Add "noms commit" command
* Updated csv-import, json-import, xml-import and url-fetch to (optionally) not commit results
* Added helpers for creating commit meta-data struct through command line or function calls
2016-09-09 12:42:27 -04:00
Erik Arvidsson e070500f20 Fix lint error (#2537)
The line length exceeded 100 cols
2016-09-09 09:05:42 -07:00
Erik Arvidsson 59b17762fe Update JSDoc 2016-09-08 18:45:38 -07:00
Erik Arvidsson 70004e1699 Let splore use versioned URLs too (#2530) 2016-09-07 18:05:14 -07:00
Erik Arvidsson 3d83e4e72f Allow stage.py to create the staging dir (#2528)
Previously, it was required to already exist
2016-09-07 14:24:16 -07:00
Erik Arvidsson dbe0556c63 Rename files to their hashes when staging (#2515)
This add two keyword arguments to the GlobCopier:
- rename - if True, will rename the files to include their hash
- index_file - if present, then the paths in this file here will be
  updated to the renamed paths.
2016-09-07 13:34:14 -07:00
cmasone-attic 1c69c6b891 Update merge.ThreeWay() to allow very basic custom conflict resolution (#2505)
This patch modifies merge.ThreeWay() to take a callback that allows
for custom conflict resolution. The noms-merge command-line tool uses
this to inject a callback that accepts input from the console
dictating whether to accept the value from the 'left' or 'right' merge
candidates.

Toward #2445
2016-09-07 13:21:22 -07:00
zcstarr ef817db179 Adds error for invalid skip records argument and exits csv importer (#2522) 2016-09-06 17:37:05 -07:00
zcstarr b1c0aeb9c5 Adds checks for bad column-type and header csv-import flag values (#2525) 2016-09-06 17:12:21 -07:00
zcstarr ef11062cab Fixes bug where delimiter applied to header arguments (#2523) 2016-09-06 16:50:05 -07:00
zcstarr 65131432c7 Fix cmdline help message color argument (#2520) 2016-09-06 14:59:01 -07:00
cmasone-attic 652613d09e Go: Add List.IteratorAt() and use in list merging (#2512)
When merge.ThreeWay() merges list splices, it frequently needs to
extract a slice of a List or check whether a given range of values is
exactly equal in two different Lists. Repeatedly Getting elements from
a List is expensive, because it creates a new cursor internally every
time. Adding IteratorAt(idx uint64) allows code to iterate over a
range of a List while only creating a cursor once. This allows a slice
of Values to be extracted or used in comparisons efficiently.

Toward #2445
2016-09-06 13:50:08 -07:00
Erik Arvidsson 061f7694ad JS: Fix subtype issue with unions as elem types (#2518)
For compound types (List, Set, Map, Ref) the concrete type may be a
union. If that is the case all the types in the union must be a
subtype of the concrete type's element type.

`C<T | V>` is a subtype of `C<S>` iff T is a subtype of S and V is a
subtype of S.
2016-09-06 13:38:51 -07:00
Mike Gray 47565f39d1 Improve code based on tool analysis feeback (#2521)
Fixes are based on Go report card output:
- `gofmt -s` eliminates some duplication in struct/slice initialization
- `golint` found some issues like: `warning: should drop = nil from declaration of var XXX; it is the zero value`
- `golint` found some issues like: `warning: receiver name XXX should be consistent with previous receiver name YYY for ZZZ`
- `golint` says not to use underscores for function/variable names
- `golint` found several issues like: `warning: if block ends with a return statement, so drop this else and outdent its block`

No functional changes are included - just source code quality improvements.
2016-09-06 16:35:25 -04:00
zcstarr 3cdebb7e77 Add run safe method that reads stderr and stdout regardless of panic (#2475)
Run method will now always return stdout,stderr, and a recoveredErr
on Exit or Panic. MustRun will Panic with recoveredErr.
2016-09-06 11:30:57 -07:00
Erik Arvidsson fd4c52acef stage.py: import os too (#2514) 2016-09-02 14:00:15 -07:00
Erik Arvidsson 893d0fa360 stage.py needs to import sys before using it (#2513)
Also changes the tabs to spaces. Thanks pylint.
2016-09-02 13:38:43 -07:00
Erik Arvidsson f465b0bd3c Fix import of noms.staging in stage.py (#2510)
Also fix unit test that was not updated when the functions were
renamed.
2016-09-02 13:09:07 -07:00
Eric Halpern 48854dc999 Move datas.descendsFrom and prune search using Ref.Height (#2480)
- Move to commit.CommitDescendsFrom
 - When searching for an ancestor in the commit history, prune
   whenever the commit.Height() < ancestor.Height()
 - Add new test TestCommitDescendsFrom to verify correctness and pruning.
2016-09-02 11:44:38 -07:00
Dan Willhite 57f653dd39 Disable perf suite check for nomsRevision (#2508)
This needs to be disabled so that test cases succeed when
building demo-server. Currently this fails because we build
with CodePipeline so the source code is not in a git repo
when getting built.
2016-09-02 10:53:18 -07:00
Erik Arvidsson 017d599b1d Fix subtype issue with unions as elem types (#2506)
For compound types (List, Set, Map, Ref) the concrete type may be a
union. If that is the case all the types in the union must be a
subtype of the concrete type's element type.

`C<T | V>` is a subtype of `C<S>` iff T is a subtype of S and V is a
subtype of S.
2016-09-01 18:40:51 -07:00
cmasone-attic e5fcfd6ebf Make poke hang on to parent's Commit metadata (#2504)
Before this, poke would drop any commit metadata from the dataset being modified. Now, it just pulls it forward.
2016-09-01 17:21:31 -07:00
Ben Kalman ace91a323d Update CONTRIBUTING.md (#2468)
Add a bit more info on squash merges and perf tests.
2016-09-01 16:22:46 -07:00
Ben Kalman 0f6cc2023c Print skipped perf tests in verbose mode (#2497) 2016-09-01 16:22:23 -07:00
Erik Arvidsson b9f15526ba JS Docs: Redirect to versioned sub directory (#2503) 2016-09-01 16:10:38 -07:00
Ben Kalman 8eba76a2ec Check before calling mkdirs on staging directory (#2502)
Also run pylint on staging.py.
2016-09-01 15:19:13 -07:00
Ben Kalman 6393e7c6e7 Fix "and and" comment from last patch (#2500) 2016-09-01 13:39:08 -07:00
Ben Kalman b734a80cb1 Improvements to the perf viewer (#2499)
* Increase number of results to 20.
* See newer results at the end of the graph, not start.
* Make point radius show standard deviation to scale.
* Try not to draw datasets over each other.
* Invert light colours to make them darker.
2016-09-01 13:36:53 -07:00
cmasone-attic 9f080a2fa7 Fix test assertion in noms-merge::TestLose() (#2498)
This was caused by me changing an error string and failing
to update the test.
2016-09-01 11:34:04 -07:00
cmasone-attic 771eb092da Add failure tests for noms-merge (#2484)
Tests command line validation for noms-merge

Toward #2445
2016-09-01 11:31:29 -07:00
Erik Arvidsson 456f403b8a Build js docs correctly (#2495)
There was a typo (missing comma) in the command.
2016-09-01 10:38:29 -07:00
Ben Kalman d1c90d11cd Add prepublish step to perf/viewer so that out.js gets built (#2496) 2016-09-01 10:33:03 -07:00
Aaron Boodman 42fd80be2e Add a super quick indeterminite progress meter to noms-merge (#2488) 2016-09-01 10:24:30 -07:00
Daniel Krech 01bdeab025 Add progress reporting to json_importer (#2494)
Fixes #2494
2016-09-01 00:04:26 -07:00
Aaron Boodman 63f4127aac Update CONTRIBUTING.md 2016-08-31 23:43:22 -07:00
Aaron Boodman 25fedb51fc Update cli-tour.md 2016-08-31 21:37:56 -07:00
Erik Arvidsson eb42d99950 JS Doc builder (#2493)
Trying to work around issue where `npm run build-docs` fails
2016-08-31 18:27:58 -07:00
Erik Arvidsson 9452898ff7 Revert "Work around python popen bug for js docs" (#2492) 2016-08-31 18:18:16 -07:00
Erik Arvidsson 0a53ee7e3d Work around python popen bug for js docs (#2491) 2016-08-31 18:10:10 -07:00
Erik Arvidsson ad66582f44 npm run did not work (#2489) 2016-08-31 17:56:12 -07:00
Ben Kalman 9c694f024b Add a perf test for CSV map import (#2461)
Currently we only have a perf test for CSV list import, which uses the
sf-crime dataset. This test uses the 43MB sf-registered-businesses
dataset instead, since sf-crime is too slow. Which is ironic, since we
normally parse sf-crime into a map.

I've also tightened up some of the other perf tests.
- Fixed a bug where Database was shared between runs.
- Make the pure CSV parsing test use a smaller dataset, it doesn't need
  to use something as large as ny-vehicle-registrations.
2016-08-31 17:05:00 -07:00
Erik Arvidsson b857b95bb4 Drive js docs from npm script (#2486)
Turns out that documentationjs reads the babelrc file from js/noms
which leads to us having to do npm install in js/noms. Therefore
putting the dev deps in there and running the script from there
makes things cleaner.

It also allows doing:

  npm run build-docs

Towards #1471
2016-08-31 16:57:31 -07:00
Ben Kalman 815ca1586f Add perf build.py and util (#2487) 2016-08-31 16:50:34 -07:00
Ben Kalman 1e8ae3f214 Make perf viewer point radius show standard deviation (#2485) 2016-08-31 16:42:59 -07:00