Big docs update

README.md

@@ -1,26 +1,20 @@
 # Noms
 
-Noms is a content-addressable, immutable, peer-to-peer datastore for structured data.
+Noms is a content-addressable, append-only, peer-to-peer, structured data store.
 
 In other words, *noms is git for data*.
 
-This repository will contain the reference implementation of the noms protocol, and will eventually be open sourced. 
-
-This includes:
-
-* Go wrappers for all the core noms types
-* Support for generating Go types from schema definitions (aka 'nomdl')
-* Chunking and dechunking
-* Serialization and deserialization
-* Chunkstore interface as well as several sample implementations
-* Search support
-* Sample applications
+This repository contains two reference implementations of the noms protocol - one in Go, and one in JavaScript. It also includes a number of tools and sample applications.
 
 # Prerequisites
 
 * [Go 1.4+](https://golang.org/dl/)
+* [Python 2.7+](https://www.python.org/downloads/) (Note: Python 2.x only, not Python 3.x)
+* [Node.js 5.3+](https://nodejs.org/download/)
 
-# Get
+# Get the code
+
+First, ensure `$GOPATH` is [set correctly](https://golang.org/doc/code.html#GOPATH). Then:
 
 ```
 go get -u -t github.com/attic-labs/noms/...
@@ -43,17 +37,10 @@ go build
 ./counter -ldb=/tmp/foo -ds=foo
 ```
 
-# Rejoice!
+# What next?
 
-You can see the raw data:
-
-```
-ls /tmp/foo
-cat /tmp/foo/*.log | strings
-```
-
-You can also explore the data visually. Follow the instructions in `clients/splore`.
-
-There are lots of other sample programs in `clients/` and they usually have `README`s. Have fun...
-
-TODO: There needs to be more of a big-picture introduction.
+* Learn the core tools: [`server`](clients/server/README.md), [`splore`](clients/splore/README.md), [`shove`](clients/shove/README.md), [`csv_importer`](clients/csv_importer/README.md), [`json_importer`](clients/json_importer), [`xml_importer`](clients/xml_importer)
+* Run sample apps: [`sfcrime`](clients/sfcrime/README.md), [`tagshow` photo viewer](clients/tagshow/README.md)
+* NomDL reference
+* Go SDK
+* JavaScript SDK

clients/csv_importer/README.md

@@ -1,7 +1,6 @@
 # CSV Importer
 
-Imports a CSV file as a List of Maps where the first row of the CSV file
-describes the keys of the Map.
+Imports a CSV file as List<T> where T is generated from the header row of the CSV (this can also be overridden with the `-header` flag).
 
 ## Usage
 

clients/server/README.md

@@ -1,16 +1,30 @@
 # Server
 
-This is an http server for noms. Currently it is read-only.
+Server implements a noms datastore over HTTP.
 
-## Build
+## Example
 
 ```
-cd <noms>/clients/server
+cd $GOPATH/src/github.com/attic-labs/noms/clients/counter
 go build
+./counter -ldb="/tmp/servertest" -ds="counter"
+./counter -ldb="/tmp/servertest" -ds="counter"
+./counter -ldb="/tmp/servertest" -ds="counter"
+
+cd ../server
+go build
+./server -ldb="/tmp/servertest" &
 ```
 
-## Run
+Then navigate a web browser to [http://localhost:8000/root]. You should see a string starting with `sha1-...`. This _ref_ is the unique identifier for the current state of the datastore. You an explore it further by fetching URLs like [http://localhost:8000/ref/sha1-...].
+
+## About
+
+Server is not commonly used directly by users, but is a building block used by other tools. For example, you can connect the counter application to your running server like so:
+
 ```
-# See -h for more options
-./server -ldb="/tmp/foo"
+./counter -h="http://localhost:8000" -ds="counter"
+./counter -h="http://localhost:8000" -ds="counter"
 ```
+
+Most noms clients accept this `-h` flag to connect to an http datastore.

clients/shove/README.md

@@ -0,0 +1,28 @@
+# Shove
+
+Shove syncs between datastores and datasets. It is the noms equivalent of Git's `push` and `pull` commands.
+
+## Howto
+
+```
+cd $GOPATH/src/github.com/attic-labs/noms/clients/counter
+go build
+./counter -ldb="/tmp/shovetest1" -ds="counter"
+./counter -ldb="/tmp/shovetest1" -ds="counter"
+./counter -ldb="/tmp/shovetest1" -ds="counter"
+
+cd ../shove
+go build
+./shove -source-ldb="/tmp/shovetest1" -source="counter" -sink-ldb="/tmp/shovetest2" -sink-ds="counter2"
+../counter/counter -ldb="/tmp/shovetest2" -ds="counter2"
+
+# Shove can also connect to http datastores
+cd ../server
+go build
+./server -ldb="/tmp/shovetest2" &
+
+../shove/shove -source-h="http://localhost:8000" -source="counter2" -sink-ldb="/tmp/shovetest3" -sink-ds="counter3"
+../counter/counter -ldb="/tmp/shovetest3" -ds="counter3"
+```
+
+There are currently a small collection of datasets you can sync available at `-h="ds.noms.io"`. You can browse them at [http://apps.noms.io/splore](http://apps.noms.io/splore) (username: attic, password: labs).

clients/splore/README.md

@@ -1,20 +1,29 @@
 # Splore
 
-This is a generic noms data explorer.
+Splore is a general-purpose debug UI for exploring noms data.
 
-## Requirements
+![splore and counter](screenshot.png)
 
-* Node.js: https://nodejs.org/download/
-* You probably want to configure npm to [use a global module path that your user owns](https://docs.npmjs.com/getting-started/fixing-npm-permissions)
+## Howto
 
-## Build
+```
+cd $GOPATH/src/github.com/attic-labs/noms/clients/counter
+go build
+./counter -ldb="/tmp/sploretest" -ds="counter"
+./counter -ldb="/tmp/sploretest" -ds="counter"
 
-* `NOMS_SERVER=http://localhost:8000 python build.py`
+# Splore requires server to be running
+cd ../server
+go build
+./server -ldb="/tmp/sploretest" &
 
-## Run
+cd ../splore
+PYTHONPATH=$GOPATH/src/github.com/attic-labs/noms/tools NOMS_SERVER=http://localhost:8000 python build.py
+./node_modules/.bin/http-server
+```
+
+Then, navigate to [http://localhost:8080].
 
-* `python -m SimpleHTTPServer 8082` (expects ../server to run on same host, port 8000)
-* navigate to http://localhost:8082/
 
 ## Develop
 

clients/splore/package.json

@@ -28,6 +28,7 @@
     "eslint-plugin-react": "^3.8.0",
     "flow-bin": "^0.19.1",
     "grunt": "^0.4.5",
+    "http-server": "^0.8.5",
     "mocha": "^2.3.0",
     "uglify-js": "^2.6.1",
     "watchify": "^3.6"