* Modifications to ipfs-chat and ipfs chunkstore
* Change ipfs paths to include directory where ipfs repo is stored.
* Rework ipfs-chat to create ipfs chunkstores manually rather than
relying on Spec.ForDataset. This enables creating two chunkstores
(one local and one network) using the same IpfsNode (ipfs repo).
* Create separate replicate function for daemon and mergeMessage
function for client to experiment with slightly different behaviors
for each.
* Re-organization of code to remove duplication.
The main points are:
* added event loop to process events synchronously
* more agressive about not re-processing msgs from other nodes
that we've already processed
* fixed bug in ipfs chunkstore HasMany()
* Add go-base58 library
stretchr has fixed a bug with the -count flag. I could merge these
changes into attic-labs, but it's easier to just use strechr.
We forked stretchr a long time ago so that we didn't link in the HTTP
testing libraries into the noms binaries (because we were using d.Chk in
production code). The HTTP issue doesn't seem to happen anymore, even
though we're still using d.Chk.
A lot of the JS code is taken from the old splore sample, but in
particular main.js is completely different - much simpler, because the
architecture of noms-splore uses a specialised {path => node} HTTP API,
implemented in Go, which does the noms graph traversal.
noms-splore also improves on the old splore sample by making it more
obvious what the key/value pairs are for maps and structs, but regresses
slightly in what it can say about prollytrees.
* use kingpin for help and new commands, set up dummy command for noms blob
* document existing commands using kingpin
* remove noms-get and noms-set in favor of new noms blob command
* normalize bool flags in tests, remove redundant cases that kingpin now handles
* add kingpin to vendor files
* make profile flags global
* move --verbose and --quiet to global flags
* Add jobs for grouping similar photos in PhotoGroups
Outline:
- The first photo-dhash job adds a dhash field to each photo. The dhash is a 128 bit
downsampled representation of the photo that works well for visual similarity comparisions.
- The second photo-dedup job groups photos that have similar dhash's into PhotoGroups.
fixes: #2787
* Implement noms cli configuration support
- Introduce .nomsconfig
- Supports a default db to use when no explicit db is given
- Supports defining db aliases to use as short hand for db urls
- See samples/cli/nomsconfig for more info
fix: #2131
You can run these yourself using the -perf flag, e.g.
> noms serve &
> go test -v -perf http://localhost:8000 ./samples/go/csv/csv-import
> noms ds http://localhost:8000
Though you'll need to go-get github.com/attic-labs/testdata.
Note that all of this only records test results, it doesn't have any
concept of failing perf (unless test assertsions themselves fail). It
will be the job of some other Noms client (work in progress) to do that.
I will be setting this up to run continuously momentarily.
This adds a simple client which fetches a URL into a noms blob. In a separate change, I will change csv_importer to link up to this rather than reading from a file.
Note: I was sad to see that chunking performance here is quite slow. We can chunk at the rate of about 150kB second on a single core, and it's cpu-bound.
