NBS is stable enough that we've made it the default store for command
line tools, and the go-to store for tests that require temporary, but
persistent, storage.
We intend to remove support for LevelDB-backed chunk storage
completely ASAP. This patch removes all usage of LevelDBStore from
noms.git, but doesn't remove LevelDBStore _just_ yet as there are
still some dependencies on it elsewhere.
Toward #3127
The subsequent runs of url-fetch on jenkins are way faster, and this
appears to be because commiting is much faster on subsequent runs. The
perf tests now use a new database each time.
Noms SDK users frequently shoot themselves in the foot because they're
holding onto an "old" Database object. That is, they have a Database
tucked away in some internal state, they call Commit() on it, and
don't replace the object in their internal state with the new Database
returned from Commit.
This PR changes the Database and Dataset Go API to be in line with the
proposal in Issue #2589. JS follows in a separate patch.
It exploits the chunked structure of Lists to allow concatenating
arbitrarily large Lists.
I've added both functional and perf tests for Concat, and the perf tests
only made sense with tests for building and reading Lists, so now we
have those too.
This needs to be disabled so that test cases succeed when
building demo-server. Currently this fails because we build
with CodePipeline so the source code is not in a git repo
when getting built.
Currently we only have a perf test for CSV list import, which uses the
sf-crime dataset. This test uses the 43MB sf-registered-businesses
dataset instead, since sf-crime is too slow. Which is ironic, since we
normally parse sf-crime into a map.
I've also tightened up some of the other perf tests.
- Fixed a bug where Database was shared between runs.
- Make the pure CSV parsing test use a smaller dataset, it doesn't need
to use something as large as ny-vehicle-registrations.
I hope.
We should stop using Markdown syntax in our public Go docs. godoc
doesn't recognise it (https://godoc.org/github.com/fluhus/godoc-tricks
is a reasonable summary of what *is* supported).
This patch:
- Wraps comments at 100 characters, because godoc doesn't do wrapping.
- Replaces triple-backtick blocks with indented blocks.
- Removes single-backtick lines entirely.
- Lines things up better.
- An unfortunate side-effect is bullet points are monospaced in
godoc.org, but there isn't a way around this.
This prepends a prefix to every dataset ID written by perf tests. It
will be used for namespacing the test results for different PRs.
Currently they're written to different databases, which isn't as good.
fix misspellings; fix code that was not gofmt'd - plus take advantage of gofmt -s too; couple of unreachable golint reported fixes; reference go report card results and tests
You can run these yourself using the -perf flag, e.g.
> noms serve &
> go test -v -perf http://localhost:8000 ./samples/go/csv/csv-import
> noms ds http://localhost:8000
Though you'll need to go-get github.com/attic-labs/testdata.
Note that all of this only records test results, it doesn't have any
concept of failing perf (unless test assertsions themselves fail). It
will be the job of some other Noms client (work in progress) to do that.
I will be setting this up to run continuously momentarily.
cmasone-attic