This is in preparation with a deeper refactor of reading from Go
in general. The big difference that we have now which we did not
have before is that we now have CRCs for every single page.
In case of full shutdown/startup and usage of dynamic ports it is
possible replicas get out of date address information.
Since they think there is enough replicas to form a quorum they will
not try to refetch address information immediately but after some delay.
This flag -using-dynamic-ports informs registry to wipe ports on startup
as empty ports are not considered valid addresses replicas will refetch
information quickly.
Note that in case of production rolling restart this is not a problem
as replica information changes slowly.
Specifically:
* Extend tables so that they won't wrap around wrongly for big sizes
(we would never hit this since our blocks are at most a few MBs
big).
* Use CRC instructions to compute remainders.
The FUSE driver, up to now, had no way to know when the user had
"explicitly" closed a file. Instead it linked the TernFS file on
flush, which could cause nasty situation. The classic example
is a fork causing the FD to a TernFS file being present in the forked
process, and then the process dying causing a spurious flush.
This commit adds a way to detect when a flush is due to a close(),
which allows us to link the file only in the cases where that happened,
which is a much better heuristic and close to what we do in the kernel
module.
This commit also contains various other improvements to make all tests
pass under FUSE. The big remaining item is changing how files are read
(they're currently read all upfront and then kept in memory).
Unfortunately Lemire's wyhash code was licensed as Apache, we'd rather
have something compatible with GPLv2. This probably a more appropriate
choice anyway.
Things not done because probably disruptive:
* kmod filesystem string
* sysctl/debugfs/trace
* metrics names
* xmon instance names
Some of these might be renamed too, but starting with a relatively
safe set.
We manually control flush of the WAL.
We persist the changes outside the loop by calling flush() with sync.
Since we only send responses after flush we will never send information
about anything that was not persisted.
