The system variable is read-only. The only way to enable Auto GC remains setting:
behavior:
auto_gc_behavior:
enable: true
in the config.yaml file which is given to `dolt sql-server` in its `--config` parameter.
A bug in the usage of dynassert meant that clone() was always opening a new table file, instead of using the existing open file.
There were a few known consequences:
* the window of opportunity when unlocked reads could try to read a closed chunk source and receive an error was much larger than typical. As a result, auto_gc on a standby replica in a cluster was much more likely to fail spuriously.
* the number of file descriptors a running server would churn was higher than typical, and some operations were a bit slower / more prone to context switches than they needed to be.
This PR also updates some errors to be slightly more verbose about their origin.
This PR also updates TestAutoGC in go-sql-server-tests to run less iterations in CI and in -test.short mode. The iterations should be able to be lower since auto GC will fail less often now.
Previously GC, even with the session-aware safepoint controller, such as is used with Auto GC, would cancel any ongoing stats work and would restart it at the end of the run. Because Auto GC can run quite frequently and because stats can take a while to run to completion, this meant that on some workloads stats would never successfully populate.
This PR changes stats so that it more correctly integrates with the session-safe GC safepoint controller. That allows GC to ensure that the stuff stats is currently working on gets carried over to the collected view of database, and stats itself has a chance to run to completion regardless of what GC work is going on.
This PR leaves the cancel-stats-on-collect behavior enabled for the kill-connections safepoint controller, where that behavior is still the correct one.
A push to a remote works by uploading the missing content and then adding
references to it in the remote datastore. If the remote is running a GC during
the push, it is possible for the newly added data to be collected and no longer
be available when the references are added.
This should cause a transient failure which is safe is retry. There were a
couple bugs which could instead cause a panic. This makes some changes to
safeguard against those case.
For sql-server instances with many databases, many branches or many tables
indexes, flushing stats could cause queries to block until stats were finished
flushing. StatsController attempted to rate limited how much I/O bandwidth
would be used to flush stats, but this is currently counter-productive because
all queries are blocked from analyzing successfully until stats are flushed.
This changes it back to flush stats to disk as quickly as possible when we need
to do it. Computing stats is already rate limited, so for large flushes this
should not happen to often.
We will continue to improve things here so that stats writes have less impact
on server resource utilization and do not block user-facing work as they
proceed.
This makes *NomsBlockStore check the incoming Context object to ensure that it
itself has been invovled in the appropriate GC lifecycle callbacks.
It fixes a problem with statspro.AnalyzeTable, where the GC lifecycle callbacks
happened more than once for a single session.
It fixes some callsites to appropriately make the GC lifecycle callbacks,
including LateBindingQueryist opening one session command for the whole
lifetime of the returned sql.Context.
