de76d37f09
* Add HasMany() to the ChunkStore interface We'll need this as a part of #3180 * Rip out hinting The hinting mechanism used to assist in server-side validation of values served us well, but now it's in the way of building a more suitable validation strategy. Tear it out and go without validation for a hot minute until #3180 gets done. Fixes #3178 * Implement server-side lazy ref validation The server, when handling writeValue, now just keeps track of all the refs it sees in the novel chunks coming from the client. Once it's processed all the incoming chunks, it just does a big bulk HasMany to determine if any of them aren't present in the storage backend. Fixes #3180 * Remove chunk-write-order requirements With our old validation strategy, it was critical that chunk graphs be written bottom-up, during both novel value creation and sync. With the strategy implemented in #3180, this is no longer required, which lets us get rid of a bunch of machinery: 1) The reverse-order hack in httpBatchStore 2) the EnumerationOrder stuff in NomsBlockCache 3) the orderedPutCache in datas/ 4) the refHeight arg on SchedulePut() Fixes #2982
Command-Line Tour | Go SDK Tour | Documentation | Project Status | Download
Noms is a decentralized database philosophically descendant from the Git version control system.
Like Git, Noms is:
- Versioned: By default, all previous versions of the database are retained. You can trivially track how the database evolved to its current state, easily and efficiently compare any two versions, or even rewind and branch from any previous version.
- Synchronizable: Instances of a single Noms database can be disconnected from each other for any amount of time, then later reconcile their changes efficiently and correctly.
Unlike Git, Noms is a database, so it also:
- Primarily stores structured data, not files and directories (see: the Noms type system)
- Scales well to large amounts of data and concurrent clients (TODO: benchmarks)
- Supports atomic transactions (a single instance of Noms is CP, but Noms is typically run in production backed by S3, in which case it is "effectively CA")
- Supports efficient indexes (see: Noms prolly-trees)
- Features a flexible query model (see: GraphQL)
Finally, because Noms is content-addressed, it yields a very pleasant programming model.
Working with Noms is declarative. You don't INSERT new data, UPDATE existing data, or DELETE old data. You simply declare what the data ought to be right now. If you commit the same data twice, it will be deduplicated because of content-addressing. If you commit almost the same data, only the part that is different will be written.
Install Noms
Noms is supported on Mac OS X and Linux.
The build contains the Noms command-line and some utility tools. You can use tar -ztvf noms-*.tar.gz to view the contents of the tar.
- Extract the Noms commands.
tar -xzf noms-*.tar.gz
Get started
-
Use the
noms dscommand to connect to thecli-tourdatabase../noms ds http://demo.noms.io/cli-tour chi-building-violations chi-building-violations/backup chi-building-violations/raw ...<snip>.. sf-film-locations sf-film-locations/raw sf-fire-inspections sf-fire-inspections/raw sf-registered-business sf-registered-business/raw -
View the history for the
sf-film-locationsdataset../noms log http://demo.noms.io/cli-tour::sf-film-locations
Learn Noms
Learn the basics: Introduction to Noms
Tour the CLI: Command-Line Interface Tour
Tour the Go API: Go SDK Tour
Status / Roadmap
Noms is currently under heavy development. Attic Labs plans to use it as an integral piece of an upcoming consumer product.
Data Format
We are fairly confident in the core data format, and plan to support Noms database version 7 and forward. If you create a database with Noms today, future versions will have migration tools to pull your databases forward.
Roadmap
We plan to implement the following for Noms version 8. Beyond that unknown.
- Horizontal scalability (Done! See: nbs)
- Automatic merge (Done! See: CommitOptions.Policy and the
noms mergesubcommand). - Query language (In progress: See ngql)
- Garbage Collection (https://github.com/attic-labs/noms/issues/3192)
- Optional fields (https://github.com/attic-labs/noms/issues/2327)
- Fix sync performance with long commit chains (https://github.com/attic-labs/noms/issues/2233)
- Various other smaller bugs and improvements
API
The Public API will continue to evolve. Pull requests which represent breaking API changes should be marked with APIChange and sent to the slack channel and mailing list below for advance warning and feedback.
Talk
If you'd like to use Noms for something, we'd love to hear. Contact us: