Commit Graph

84 Commits

Author SHA1 Message Date
Mike Gray
4e54c44d56 no functional changes, improving code quality (#2410)
fix misspellings; fix code that was not gofmt'd - plus take advantage of gofmt -s too; couple of unreachable golint reported fixes; reference go report card results and tests
2016-08-23 13:51:38 -04:00
Dan Willhite
7b3adfe657 Add support to nomdex to query against multiple indexes. (#2396)
Fixes #2405
Fixes #2405
Fixes #2114 (Implement nomdex)
2016-08-23 10:26:11 -07:00
Ben Kalman
892f98050e Make perf suite testdata path configurable by -perf.testdata flag (#2406)
Needed by Jenkins shared workspaces.
2016-08-22 16:40:27 -07:00
Erik Arvidsson
ecbdd2412b Use Go marshal API in the HR sample 2016-08-22 10:38:20 -07:00
Mike Gray
1197b4127f csv-analyze to detect column types and primary keys (#2366)
create csv-analyze; add detection of PKs; move detect-columns from csv-import to csv-analyze
2016-08-19 14:24:32 -04:00
Ben Kalman
064c398dec Add perf test suite infrastructure, and a perf test for csv-import (#2384)
You can run these yourself using the -perf flag, e.g.

> noms serve &
> go test -v -perf http://localhost:8000 ./samples/go/csv/csv-import
> noms ds http://localhost:8000

Though you'll need to go-get github.com/attic-labs/testdata.

Note that all of this only records test results, it doesn't have any
concept of failing perf (unless test assertsions themselves fail). It
will be the job of some other Noms client (work in progress) to do that.

I will be setting this up to run continuously momentarily.
2016-08-18 15:49:21 -07:00
Dan Willhite
218c98f209 Nomdex - for indexing and querying. (#2357)
This first version builds indexes and can query against a single one.
2016-08-17 14:42:34 -07:00
Erik Arvidsson
3320518f88 Fix shell script flags (#2377) 2016-08-16 11:58:57 -07:00
Mike Gray
865e977320 add csv-import --detect-column-types option (#2329) 2016-08-11 17:02:05 -04:00
Sungguk Lim
6697c2e6fc Replace github.com/tsuru/gnuflag with github.com/juju/gnuflag (#2340)
Replace vendor folder and where it is used.
2016-08-11 10:29:57 -07:00
Tegan Snyder
25124a12b0 Update csv README.md (#2328) 2016-08-10 12:13:19 -04:00
Mike Gray
a812034182 changing csv-import of map to include pk in struct (#2316) 2016-08-09 19:57:11 -04:00
Mike Gray
75ef96bdc7 implement csv-export writing maps of structs (#2322) 2016-08-09 18:31:54 -04:00
Adam Leventhal
e1fd70c2e3 nomsfs: small fix up for Finder regression (#2318) 2016-08-08 21:40:42 -07:00
Adam Leventhal
8fb43f1ef4 nomsfs: general cleanup (#2309) 2016-08-08 18:08:32 -07:00
Mike Gray
8c5de64bda adding readme for nomsfs (#2280) 2016-08-05 10:55:42 -07:00
Surya Gaddipati
3f9d85359a Fix command line flag (#2266) 2016-08-03 18:32:33 -07:00
Adam Leventhal
c6d99f2fab noms-based fuse implementation (#2256) 2016-08-02 17:50:39 -07:00
Ben Kalman
07be381117 Support reading from stdin in url-fetch
I need this in the short term because CSV raw files are too large to
check into github, so they need to be split up. It's easier to just do
"cat * | url-fetch -stdin" than recombining them on the file system.
2016-08-02 17:29:10 -07:00
Chris Masone
4f073aee3d Validate type of new Root in root/ POST handler
In the server side of the Remote Databse, the handler for
UpdateRoot now verifies that the new proposed Root is of a
legal type: empty map OR Map<String, Ref<Commit-like>>

Fixes #2116
2016-08-01 16:05:37 -07:00
cmasone-attic
55025ee801 Add caching layer to demo-server (#2228)
This patch creates a new kind of chunks.Factory that demo-server
uses to vend ChunkStore instances that all share the same
MemoryStore-based Chunk cache. This cache _will_ grow without bound,
but the current RAM/data ratio on demo.noms.io means that, in practice,
we will be fine for a bit.

This will need to be removed in favor of a real solution in Issue #2227

Fixes #2009
2016-08-01 11:55:16 -07:00
Mike Gray
b42733e982 changing order of args to match JS and be more logical (#2223) 2016-08-01 11:17:59 -07:00
Mike Gray
93916c5139 exposing StructData, updating tour (#2214) 2016-08-01 12:09:02 -04:00
Adam Leventhal
e920c12c1c Fix cyclic type nondeterminism (#2147) (#2148) 2016-07-29 21:35:17 -07:00
Erik Arvidsson
ed0364cc19 Switch to gnuflag (#2206)
This is to support:
- shorthands
- Putting commands anywhere (after positional arguments too)
2016-07-29 18:08:23 -07:00
Ben Kalman
467bdc9bc1 Let csv-importer take the metadata date as a flag (#2196) 2016-07-29 10:44:08 -07:00
Chris Masone
b0112ba52b Remove NewSerializer
NewSerializer spun up a goroutine within itself. We've decided
this is an anti-pattern. Furthermore, we were using this inside
our remote database handler code, and a panic inside that goroutine
could take down the server. The callsites now use Serialize() directly.

Fixes #2169
2016-07-28 16:05:03 -07:00
Erik Arvidsson
97093afa8c Rename csv StringToType to StringToValue (#2154) 2016-07-25 14:46:11 -07:00
Rafael Weinstein
b57377c1ed Chunk over value (non-type) serialization bytes (#2130)
Chunk over value (non-type) serialization bytes
2016-07-25 11:02:26 -07:00
Aaron Boodman
a23052a1dd csv-import: add flexibility to handling of numbers and booleans (#2141)
* Default empty csv cells to reasonable values.

It would probably be better to have more control over this, or even
better, some kind of 'null' value. But I think this is perfectly
reasonable initial behavior, and much more compatible with other
code.

* Support more spellings of "true" and "false"

* csv-import: provide more helpful value parse errors
2016-07-23 23:01:38 -07:00
Ben Kalman
270ba908cd Introduce samples/go/blob-get, utility which downloads a noms blob (#2127)
... from a noms database. Right now it's very slow (< 100kb/s).
2016-07-22 17:50:33 -07:00
Erik Arvidsson
cb8d44b3f9 Merge pull request #2120 from arv/commit-no-meta
Commit type should always have a meta field
2016-07-21 18:29:45 -07:00
Erik Arvidsson
68e92092e5 Commit type: Inner parents struct should also have meta
This changes so that all commit struct types have a meta field
(which might be an empty struct).

Increment the serialization version since the old data does not
necessarily have the meta field.

Fixes ##2112
2016-07-21 18:25:17 -07:00
Aaron Boodman
521e120c74 Fix double-escaping in csv.Read. (#2122)
Fixes #2109
2016-07-21 15:33:35 -07:00
Dan Willhite
137e39d683 Add meta information to commits in cvs-import and url-fetch.
Add "meta" field to commit.
Change noms_log to print meta information when it exists.
Fixes #2012.
2016-07-20 17:18:21 -07:00
Erik Arvidsson
984fef226f Change dataset CommitWithParents to Commit with options (#2095)
The new API is `ds.Commit(value, CommitOptions{Parents: p})`

Related to #2012
2016-07-19 14:30:59 -07:00
Aaron Boodman
b968b27b93 Make clear samples/go/url-fetch supports local files (#2055) 2016-07-15 14:43:27 -07:00
Aaron Boodman
3d39d2b713 Add -skip-lines flag to csv-import (#2050) 2016-07-13 17:18:07 -07:00
Ben Kalman
d992b1c352 More informative url-fetch error checking (#2049) 2016-07-13 16:36:04 -07:00
Dan Willhite
ff5876349a Add CORS code to demo-server and noms-serve.
This is necessary to handle x-noms-vers header from browsers.
Adds x-noms-vers header to all responses from demo-server and 'noms serve'.
Fixes #1984. Fixes #1991.
2016-07-12 16:25:21 -07:00
Erik Arvidsson
1507b8dd8f Go: Change hash function to sha512 2016-07-12 13:59:08 -07:00
Mike Gray
7c9ad898de use column-type from original csv file column order (#2013)
* use column-type from original csv file column order, not the one after map/struct field name reordering
* use column-type from original csv file column order (part 2)
2016-07-12 11:33:01 -07:00
Mike Gray
b6f19dfea5 number encoding (#1845)
Moved to encoding Values of type Number as 2 varints.
2016-07-11 12:08:56 -07:00
Rafael Weinstein
ee5c0f3a72 MakeStructType and NewStructWithType now takes slices rather than maps (#1945)
NewStructType and NewStructWithType now takes slices rather than maps
2016-07-07 16:36:48 -07:00
Ben Kalman
eb9b67d571 csv-importer can accept a noms path as a source (#1981) 2016-07-07 15:10:13 -07:00
Mike Gray
a7f29a716d noms as one command line application, with version and help (#1874) 2016-07-06 15:38:25 -04:00
Aaron Boodman
1d752e3101 Update license: Noms will be copyright Attic Labs. (#1976) 2016-07-05 22:07:42 -07:00
Mike Gray
b2a4aa62b7 removing runtime.GOMAXPROCS calls (#1947) 2016-07-01 17:42:46 -04:00
Aaron Boodman
2921ca343c Part 2: Move things that are not samples out of samples/go (#1944) 2016-07-01 11:58:26 -07:00
Aaron Boodman
2386afc7fb Move stuff that is not samples out of samples directory (#1943)
* Move samples/go/*perf-reg to go/perf/

They aren't really samples.

* Move samples/go/util/check_error.go to go/d

* Remove samples/go/util/client_flags.go - unused

* remove httpcache.go - unused

* Remove samples/go/util/rungen.go - no generated code here anymore

* move NomsValueFromJSON to go/util/jsontonoms
2016-07-01 11:04:17 -07:00