Commit Graph

67 Commits

Author SHA1 Message Date
Erik Arvidsson
1507b8dd8f Go: Change hash function to sha512 2016-07-12 13:59:08 -07:00
Mike Gray
7c9ad898de use column-type from original csv file column order (#2013)
* use column-type from original csv file column order, not the one after map/struct field name reordering
* use column-type from original csv file column order (part 2)
2016-07-12 11:33:01 -07:00
Rafael Weinstein
94a27139c7 bump noms npm version & update codec-perf-rig numbers (#2016) 2016-07-11 16:54:12 -07:00
Mike Gray
b6f19dfea5 number encoding (#1845)
Moved to encoding Values of type Number as 2 varints.
2016-07-11 12:08:56 -07:00
Erik Arvidsson
abff0f247f Make go test ./... and go build ./... work with integration tests (#2005)
For go build we need a dummy go file or go build fails

For go test we now make sure we run npm install so that the test
passes
2016-07-11 11:01:51 -07:00
Erik Arvidsson
410a1664ba Create symlinks from npm i (#1996) 2016-07-08 13:47:00 -07:00
Erik Arvidsson
899ed35e55 Symlinks needed by other heaps of peeps (#1995) 2016-07-08 13:40:19 -07:00
Erik Arvidsson
96ed40bbb5 Setup symlinks again (#1994) 2016-07-08 13:33:01 -07:00
Erik Arvidsson
88fd487f00 Splore deployment (#1992)
Make sure we run npm install in js first
2016-07-08 11:46:10 -07:00
Ben Kalman
68e16f338e Fix noms browser fetch and splore layout (#1989) 2016-07-07 18:19:18 -07:00
Rafael Weinstein
ee5c0f3a72 MakeStructType and NewStructWithType now takes slices rather than maps (#1945)
NewStructType and NewStructWithType now takes slices rather than maps
2016-07-07 16:36:48 -07:00
Rafael Weinstein
c38c884810 Checking last codec perf numbers (#1986) 2016-07-07 15:18:28 -07:00
Ben Kalman
eb9b67d571 csv-importer can accept a noms path as a source (#1981) 2016-07-07 15:10:13 -07:00
Erik Arvidsson
fb4c8624a0 JS integration tests for samples/js/{counter,fs} (#1951)
This adds some a basic integration tests for sample/js/{counter,fs}.

It work pretty much like this:

- Run `npm install`
- Do setup (good place to initialize the database) (optional).
- Start a http store from go.
- Run `node . <args...>`. The IntegrationTestSuite has convenience
  methods to get the spec.
- Do teardown, which is a good time to check the output and the current state of the db (optional).

Towards #1888
2016-07-07 14:00:32 -07:00
Erik Arvidsson
4e4cbd323c Update samples/js/**/README (#1979) 2016-07-06 14:20:05 -07:00
Erik Arvidsson
c0bd2c90d4 More tweaks to tools/run-all-js-tests.py (#1974)
* Update Splore build.py

* path

* Setup the bin symlinks in preinstall

* Link entire node_modules instead
2016-07-06 13:44:11 -07:00
Mike Gray
a7f29a716d noms as one command line application, with version and help (#1874) 2016-07-06 15:38:25 -04:00
Aaron Boodman
1d752e3101 Update license: Noms will be copyright Attic Labs. (#1976) 2016-07-05 22:07:42 -07:00
Erik Arvidsson
983fa59090 Samples js shared 2 (#1973)
* Revert "Revert "Share node_modules for samples/js" (#1967)"

This reverts commit 7bb1623e99.

* Add parent directory to the PATH

* Change run-all-js-tests to run samples/js before samples/js/**

* Fix a temporary comment
2016-07-05 16:25:21 -07:00
Erik Arvidsson
7bb1623e99 Revert "Share node_modules for samples/js" (#1967) 2016-07-05 14:44:21 -07:00
Erik Arvidsson
b0ab909e4f Share node_modules between samples/js (#1955)
This is so that we do not need to do npm install 10 times in the
samples directory. Instead we do it once inside samples/js and let
all the sub directories have no dependencies. This works because
nodejs searches the parent directories when looking for a module.

Towards #1888
2016-07-05 13:55:36 -07:00
Mike Gray
b2a4aa62b7 removing runtime.GOMAXPROCS calls (#1947) 2016-07-01 17:42:46 -04:00
Aaron Boodman
2921ca343c Part 2: Move things that are not samples out of samples/go (#1944) 2016-07-01 11:58:26 -07:00
Aaron Boodman
2386afc7fb Move stuff that is not samples out of samples directory (#1943)
* Move samples/go/*perf-reg to go/perf/

They aren't really samples.

* Move samples/go/util/check_error.go to go/d

* Remove samples/go/util/client_flags.go - unused

* remove httpcache.go - unused

* Remove samples/go/util/rungen.go - no generated code here anymore

* move NomsValueFromJSON to go/util/jsontonoms
2016-07-01 11:04:17 -07:00
Aaron Boodman
fcca02d6b0 Fix crash in hr when dataset spec invalid (#1939)
Also remove test util code from the two simpler Go samples. I think
we will want to distinguish between "samples" and "utilities". The
former should be as easy to understand and lightweight as possible,
while the latter will tend to be more a real part of the project
that is maintained, tuned, relied upon, etc.

The former should only rely on the Noms SDK and the Go stdlib. The
latter should follow normal Noms internal coding conventions.
2016-07-01 10:27:44 -07:00
Erik Arvidsson
0a502c3ee4 Fix some js tests (#1932) 2016-06-29 15:57:53 -07:00
Rafael Weinstein
f464a2d94f Map.MX() -> NewStreamingMap (#1918) 2016-06-27 16:06:40 -07:00
Rafael Weinstein
aacfe751fa Add (temporary) custom leaf-chunker for blobs (#1911)
Add (temporary) custom leaf-chunker for blobs
2016-06-27 15:16:50 -07:00
cmasone-attic
d1f46401d1 Go: Add Mx() for Noms Maps (#1876)
As one step towards #1819, we've created MapMutator, which can take a
bunch of (what would normally be) Map.Set() calls and batch them up to
be applied all at once. The keys and values are held in a LevelDB cache
until everything's done. Usage looks like this:

 m := types.NewMap()
 mx := m.Mx()
 mx = mx.Set(String("foo"), String("bar")).Set(String("baz"), Number(42))
 m = mx.Finish()

We intend to make this the only way to modify collections, but at first
this will only work on an empty NomsMap.
2016-06-27 13:52:52 -07:00
Rafael Weinstein
2a04a0abaf Add sizecache to ValueStore (#1909)
Add sizecache to ValueStore.
2016-06-25 07:09:08 -07:00
Ben Kalman
0f4ed16641 Meta nodes encode a fake simple ref of non-ordered values (#1893)
Previously when we had an ordered (set/map) prolly tree containing
non-ordered values (blobs/refs/etc), we'd put the ref of the largest
value in each meta node, complete with its full type info and height.

This is wasteful, all we really need is the hash of the largest item for
searching the tree. In lieu of encoding just the hash - which isn't a
value - this patch creates a fake Ref<Boolean> with height 0.
2016-06-23 15:59:26 -07:00
Aaron Boodman
870183f22b samples/js/fs: remove extraneous ref links (#1886) 2016-06-23 05:26:01 -07:00
Erik Arvidsson
ebd9a88d23 JS: Fix http batch store (#1885)
There were a few issues here:

1. The version header was not always passed to fetch
2. HTTP headers are case insensitive and Node an Fetch uses lower case.
3. The old code used a Map instead of an object as map. Node and Fetch
   uses an object as map. Now we just pass along the response headers.

Fixes #1881
Closes #1880 (which this is partially based in)
2016-06-23 17:41:56 +08:00
Erik Arvidsson
123544c4f7 Fix samples/js/fs (#1884)
The values did not match the type declaration
2016-06-23 16:34:47 +08:00
Dan Willhite
095d49fa84 New error handling. 2016-06-22 12:11:31 -07:00
Aaron Boodman
ed3f635979 Fix bonehead mistake in hr test -- test function not being run (#1869) 2016-06-22 10:18:08 -07:00
Aaron Boodman
192e7c471b Remove forked version of buzhash from vendor dir (#1868)
Replace with upstream

Fixes #1864
2016-06-22 09:54:15 -07:00
Chris Masone
15e1ceef03 Update hr sample canned database 2016-06-22 09:10:06 -07:00
cmasone-attic
4d5eb3b334 Version Noms SDK and storage (#1841)
ChunkStores provide a Version() method so that anyone directly
using a ChunkStore (e.g. BatchStoreAdaptor) can retrieve and
check the version of the underlying store.

remoteDatabaseServer checks the version of the ChunkStore it's
backed by at startup, and then provides that version as an HTTP
header to all clients. In Go, httpBatchStore checks this header
anytime it gets a response from the server and bails if there's
version skew.

In Go, the responsibility for checking whether the running code and
the data being accessed lies with the BatchStore layer. In JS, there
is code in fetch.js that checks the header mentioned above.

Towards #1561
2016-06-21 20:55:07 -07:00
Dan Willhite
ad9c383744 Fix description of authkey argument. 2016-06-21 09:24:36 -07:00
Dan Willhite
7c6048cc67 Set noms version of splore to latest version. 2016-06-16 16:58:00 -07:00
Aaron Boodman
d58d558548 Make types.String not cache ref (#1836)
Fixes https://github.com/attic-labs/noms/issues/1542
2016-06-16 07:04:28 -07:00
Dan Willhite
8d112992c6 Move demo-server from it's own project into noms. 2016-06-15 14:06:23 -07:00
Rafael Weinstein
9d33933a4f close db in url fetch (#1826) 2016-06-15 12:16:52 -07:00
Dan Willhite
a3a78d695d Add HeadValue() and MaybeHeadValue() to dataset.go and js equivalents
Fixes #1801.
2016-06-15 11:16:25 -07:00
cmasone-attic
9c5d3289fb Merge pull request #1815 from cmasone-attic/export-prof
Add profiling to csv-export
2016-06-14 14:42:49 -07:00
Chris Masone
29e391c6af Add profiling to csv-export 2016-06-14 14:24:17 -07:00
Aaron Boodman
01be9f1705 Introduce spec.Get(Database|Dataset|Path) conveniences (#1813)
Fixes: https://github.com/attic-labs/noms/issues/1800
2016-06-14 14:06:22 -07:00
Aaron Boodman
2d3dd6d925 Update README.md 2016-06-13 16:12:57 -07:00
Aaron Boodman
16372d7376 Update README.md 2016-06-13 11:03:39 -07:00