Commit Graph

133 Commits

Author SHA1 Message Date
Mike Gray 4e54c44d56 no functional changes, improving code quality (#2410)
fix misspellings; fix code that was not gofmt'd - plus take advantage of gofmt -s too; couple of unreachable golint reported fixes; reference go report card results and tests
2016-08-23 13:51:38 -04:00
Ben Kalman 8a92b75995 Add missing ==nil check to list diff (#2362)
This isn't triggering any problem in particular, I just noticed it. All
it meant was that noms log might take unnecessarily long with list diff
when there are a lot of changes.
2016-08-17 14:44:21 -07:00
Ben Kalman a28f5ddaf9 Render diff struct field paths .foo not ["foo"] (#2379) 2016-08-16 15:28:47 -07:00
Ben Kalman 9079b83927 Use best set/map diff in noms diff (#2372)
In 7c103344 I changed noms log to use left-right diff so that it
completes faster, but it changed noms diff to use left-right as well.
Noms diff is supposed to use best diff.
2016-08-15 17:15:12 -07:00
Ben Kalman da62bb2b97 Remove most references to ldb: prefix in the .md docs (#2360) 2016-08-12 15:51:25 -07:00
Eric Halpern c5ccae3852 Fix noms_sync status message to indicate when new dataset is created (#2276)
* Fix noms_sync status message to indicate when new dataset is created

Originally, syncing a commit to a non-existent dataset and an already sync'ed dataset resulted in the same message: "<src> is up to date"

This fix distinguishes 2 cases:

  - Syncing to a new dataset prints "<dest> created"

  - Syncing to an identical dataset prints "<dest> already up to date"

Resolves: #2053

* Make message when noms sync creates new dataset more fun

Addresses: #2053
2016-08-11 16:17:15 -07:00
Sungguk Lim 6697c2e6fc Replace github.com/tsuru/gnuflag with github.com/juju/gnuflag (#2340)
Replace vendor folder and where it is used.
2016-08-11 10:29:57 -07:00
Dan Willhite deb4d2ac0c Factor out noms Command and Help so they are reusable. 2016-08-08 14:08:29 -07:00
cmasone-attic 65edbaabe3 Allow Dataset HEAD to be set to non-fastforward commits (#2271)
The Dataset.Commit() code pathway still enforces fast-forward-only
behavior, but a new SetHead() method allows the HEAD of a Dataset to
be forced to any other Commit.

noms sync detects the case where the source Commit is not a descendent
of the provided sink Dataset's HEAD and uses the new API to force the
sink to the desired new Commit, printing out the now-abandoned old
HEAD.

Fixes #2240
2016-08-05 15:38:41 -07:00
Ben Kalman fd034f8c81 Simplify Path construction in Go (#2262)
Previously there were 3 ways to construct a Path: via methods like
AddIndex, via parsing/AddPath, and simply via using append() (since Path
is a slice).

This patch deletes the former and leaves only ParsePath and append(),
with the ability to manually create the various path parts (I've made
them public). I also added documentation for the public types/fields.
2016-08-05 15:16:04 -07:00
Ben Kalman c386725d4e Fix struct diff (#2281)
It inverted the added/removed order.
Fixes https://github.com/attic-labs/noms/issues/2278.
2016-08-04 16:11:11 -07:00
Ben Kalman 02f7e9556c Tweak "Deleted dataset..." message (#2255)
It (a) had a spurious blank line, (b) should print the dataset path not
just its ID, (c) should include the # for the hash. Now it says:
```
> noms ds -d path/to/db::my-dataset
Deleted path/to/db::my-dataset (was #lvnbmpj94r9h3nuaj1t0m4jeqp25o1ve)
>
```
2016-08-02 18:22:20 -07:00
Ben Kalman 7c10334472 Make sure that set/map diff has completed before returning from noms log (#2249)
This fixes https://github.com/attic-labs/noms/issues/2235 where set/map
diff was still running when noms log has finished, and already closed
its database. Commonly this would crash.

I've removed panic-based control flow to make error handling explicit,
and added a "DiffLeftRight" method to set/map so that noms log completes
ASAP. See the issue for details.
2016-08-02 18:14:09 -07:00
Dan Willhite e4266a0d3b Add test 2016-08-01 15:30:25 -07:00
Dan Willhite 8d09fd770b Fix noms log output when --online is used. Fixes #2150 2016-08-01 14:19:09 -07:00
Erik Arvidsson 93e56eb4c2 Handle Blob and Ref in noms diff (#2216)
For Blobs we print:

```
-   Blob (42 kB)
+   Blob (1 B)
```

For Refs we just print the hashes:

```
-   abcdeabcdeabcdeabcde
+   defghdefghdefghdefgh
```

Fixes #2213
2016-08-01 09:34:39 -07:00
Mike Gray 93916c5139 exposing StructData, updating tour (#2214) 2016-08-01 12:09:02 -04:00
Adam Leventhal e920c12c1c Fix cyclic type nondeterminism (#2147) (#2148) 2016-07-29 21:35:17 -07:00
Erik Arvidsson ed0364cc19 Switch to gnuflag (#2206)
This is to support:
- shorthands
- Putting commands anywhere (after positional arguments too)
2016-07-29 18:08:23 -07:00
Ben Kalman 1db33d33f8 Use correct first arg when printing "up to date" in noms sync (#2200) 2016-07-29 11:39:31 -07:00
Erik Arvidsson dba9e5584b Print percentages in diff summary (#2195)
For example:

```
0 insertions (0.00%), 5,405 deletions (0.28%), 45 changes (0.00%), (1,938,935 entries vs 1,933,530 entries)
```

Fixes #2194
Towards #2031
2016-07-28 18:04:33 -07:00
Ben Kalman b2096234fd Make the output pager close the pipes it creates (#2187)
This makes less exit properly - before, it wasn't properly restoring the
terminal sometimes, and keyboard input stopped working.
Fixes https://github.com/attic-labs/noms/issues/2180.
2016-07-28 17:55:18 -07:00
Ben Kalman 6d2710a04e Simplify the channel logic in diff.go/summary.go (#2190)
Earlier I'd used 2 channels and selected over them, but as I found
elsewhere, this doesn't scale very well. It's simpler to just use a
close channel with a buffer size of 1.
2016-07-28 17:44:20 -07:00
Erik Arvidsson 26b94cf816 Minor cleanup of noms diff code (#2191) 2016-07-28 16:55:37 -07:00
Erik Arvidsson b20bec9b23 noms diff --summarize (#2183)
Shows number of changes between two top level values.

```
noms diff -summarize $l::#t5p4im6uug7n5m72frr0dnjnkm04e4ph.value $l::#ueo0utduuqsf9vrntrhn25lnc19m848l.value
```

Prints:

```
13,636 insertions, 0 deletions, 107 changes, (1,919,894 values vs 1,933,530 values)
```

Where the numbers are updated as more data is computed from the diff.

Towards #2031
2016-07-28 15:01:27 -07:00
Ben Kalman 8330f3b7f3 Print closing brace for struct diff (#2178) 2016-07-27 17:36:14 -07:00
Ben Kalman 7998f302be Don't quote struct field names in diff (#2177) 2016-07-27 16:14:35 -07:00
Erik Arvidsson b0e77c250c Cleanup Struct Diff (#2160)
- Too have same API as all the other diff methods
- Send changes to channel without intermediary slices and without the
need to union and sort the fields
2016-07-27 14:56:22 -07:00
Ben Kalman 57bd3d2540 Make map diff print headers for each block of additions/deletions (#2166)
Previously a header was only printed once per entire map. This led to a
confusing diff where if a map like:

```
{
  "a": {
    "b": 1
  }
}
```

changed to:

```
{
  "a": {
    "b": 2
  },
  "c": {
    "d": 3
  }
}
```

then the diff would be:

```
/["a"]
- "b": 1
+ "b": 2
+ "c": {
+   "d": 3
+ }
```

which makes it look like the "c" change is part of the ["a"]["b"] one.

After this change, the diff will be:

```
/["a"]
- "b": 1
+ "b": 2
/
+ "c": {
+   "d": 3
+ }
```

I also fixed up a bit of the diff formatting in this change.
2016-07-26 16:48:33 -07:00
Rafael Weinstein b57377c1ed Chunk over value (non-type) serialization bytes (#2130)
Chunk over value (non-type) serialization bytes
2016-07-25 11:02:26 -07:00
Dan Willhite c12402668a Treat diffing of structs of different types, like maps. (#2062) 2016-07-22 14:49:51 -07:00
Ben Kalman b68f8eb309 Make diff depth first, fix error handling (#2125)
This fixes several diff-is-slow and log-is-slow bugs, and a bug where
diff was hanging in some error conditions.
2016-07-22 12:55:59 -07:00
Erik Arvidsson 68e92092e5 Commit type: Inner parents struct should also have meta
This changes so that all commit struct types have a meta field
(which might be an empty struct).

Increment the serialization version since the old data does not
necessarily have the meta field.

Fixes ##2112
2016-07-21 18:25:17 -07:00
Dan Willhite 4e1bbfcfa5 Simplify basic commit type for validation check 2016-07-21 12:30:36 -07:00
Dan Willhite 137e39d683 Add meta information to commits in cvs-import and url-fetch.
Add "meta" field to commit.
Change noms_log to print meta information when it exists.
Fixes #2012.
2016-07-20 17:18:21 -07:00
Erik Arvidsson 00105d649e Diff: We can never get nil as a Value (#2108)
...so remove the handling of that and panic.
2016-07-20 14:53:09 -07:00
Ben Kalman 74200673d4 Change sequence diff functions to be synchronous (#2098)
Currently they return immediately, but run internal goroutines which
stream results back on a channel. Now they must be run on their own
goroutine from outside the function.

This makes the code easier to reason about, and a future patch to write
a top-down and left-right multiplexing diff easier to write.
2016-07-20 13:20:17 -07:00
Dan Willhite 9931528c1b Merge pull request #2096 from willhite/diff-flush
Flush() output from noms diff to ensure results are printed.
2016-07-19 16:29:00 -07:00
Dan Willhite ac4472eed7 Flush() output from noms diff to ensure results are printed. 2016-07-19 16:27:50 -07:00
Erik Arvidsson 984fef226f Change dataset CommitWithParents to Commit with options (#2095)
The new API is `ds.Commit(value, CommitOptions{Parents: p})`

Related to #2012
2016-07-19 14:30:59 -07:00
Erik Arvidsson e2f261b142 Go: Compute commit type based on value and parents
We now compute the commit type based on the type of the value and
the type of the parents.

For the first commit we get:

```
struct Commit {
  parents: Set<Ref<Cycle<0>>>,
  value: T,
}
```

As long as we continue to commit values with type T that type stays
the same.

When we later commits a value of type U we get:

```
struct Commit {
  parents: Set<Ref<struct Commit {
    parents: Set<Ref<Cycle<0>>>,
    value: T | U
  }>>,
  value: U,
}
```

The new type gets combined as a union type for the value of the inner
commit struct.

Fixes #1495
2016-07-18 14:28:56 -07:00
Erik Arvidsson 229c1708a2 Noms is a not a tool for *iterating* with Noms data (#2078)
Fixes #2051
2016-07-18 10:18:36 -07:00
mgedigian e3a891bed7 Prevent sync crash when source is invalid (#2002)
* When sync source is not a valid object, report error without crashing.

Fixes #1983
2016-07-17 17:31:25 -07:00
Ben Kalman 534faa6b6d exit noms as soon as less quits, not when stdout closes (#2068) 2016-07-15 10:10:57 -07:00
Ben Kalman be463555f6 Add noms log -oneline option (#2067) 2016-07-14 16:58:16 -07:00
Mike Gray 4dd9415a1a orderedSequenceDiff more like indexedSequenceDiff - parallel (#1961)
orderedSequenceDiff more like indexedSequenceDiff - parallel and working top-down rather than sequential
2016-07-12 17:57:26 -07:00
Erik Arvidsson 1507b8dd8f Go: Change hash function to sha512 2016-07-12 13:59:08 -07:00
Mike Gray b6f19dfea5 number encoding (#1845)
Moved to encoding Values of type Number as 2 varints.
2016-07-11 12:08:56 -07:00
Ben Kalman c80be5c1ec Formatting nits for noms sync progress (#1985) 2016-07-07 15:03:25 -07:00
Ben Kalman f57353ea89 Include MB and MB/s in sync progress (#1980) 2016-07-06 16:26:13 -07:00