Commit Graph

159 Commits

Author SHA1 Message Date
Dhruv Sringari c0c54db5d2 Fix bad upload table file retries 2022-03-25 14:09:46 -07:00
James Cor 0cadc528e5 renamed WriteNoNewLine to WriteWithoutNewLine 2022-03-02 12:16:56 -08:00
James Cor 88b6a6f197 working for sql format; not sure if i need any other formats 2022-03-01 15:42:33 -08:00
Lars Lehtonen bf727113cc libraries/utils/editor: fix dropped error 2022-02-21 18:54:48 -08:00
Dhruv Sringari 009bb85d8d fix bats 2022-02-17 14:22:46 -08:00
Dhruv Sringari c662e2fa8c self-review 2022-02-16 16:29:00 -08:00
Dhruv Sringari a18bfc6c27 wip 2022-02-16 14:52:34 -08:00
Zach Musgrave 576753d38c Added failsafe values to dolt CLI config, only in sql context for now 2021-11-08 12:26:52 -08:00
Zach Musgrave c76f4bde26 Fixed bug in config hierarchy construction -- it was being constructed backwards, so globals were being returned instead of locals 2021-11-08 11:43:13 -08:00
Zach Musgrave 37c2137557 merge main 2021-11-04 14:12:38 -07:00
Zach Musgrave 4a236171e7 More clean up of session setup 2021-11-04 09:51:30 -07:00
jennifersp efef83b167 dolt dump: Add dolt dump command and bats tests. 2021-11-03 15:21:46 -07:00
Zach Musgrave 92f175a37a Merge branch 'main' into zachmu/create-db 2021-11-02 10:46:50 -07:00
Zach Musgrave 037f249e91 First pass at getting something working for create db 2021-11-02 10:46:12 -07:00
Maximilian Hoffman d11f5ec41b default session implements persistable interface, can save SQL variables (#2270)
* save version with defaultConfig in session -- todo remove and make PersistedSession

* add PersistedSession, and DoltSession interface to switch

* fmt

* fixup test for persistedSession

* PersistedSession progress, interfaces and enginetests

* All DsessFromSess references fixed

* delete unnuecessary lines

* Comments for dolt session

* Dolt session tests

* save bats progress

* formatting

* GMS version

* Bump GMS version

* add license header

* fix multi-db errors

* try copyright again

* make test deterministic

* higher connection timeout default

* sever config ordering

* PR fixes

* PR fixes

* config rewrite progress

* server config refactor

* PR cleanup

* delete lines

* add go tests, refactor persistent global initialization

* PR fixes

* delete more lines

* small PR cleanup

* try to fix data race

* better constructor names

* zach don't like --server flag

* edge cases around disabling replication

* missed line in merge

* Skip two tests i fixed in follow-on PR

* merge two config prs

* fix nondeterministic bats

* zach's comments

* bump bats sql-shell expect timeout

* GMS and integrator race to access system vars; need to decide whether to force integrator to initialize, or make GMS init private; leaving in GMS for now
2021-11-01 16:49:00 -07:00
Aaron Son 86307b6f8e Merge pull request #2230 from abmyii/arm-dolt-push
Fixes for dolt on arm devices
2021-10-07 09:05:55 -07:00
Aaron Son 97ea3b8749 go/libraries/utils/iohelp: ReaderWithStats: Support > 4GB read stats. 2021-10-06 14:39:47 -07:00
Abdurrahmaan Iqbal 7e760c4e31 Fix for dolt push on arm devices 2021-10-06 21:04:50 +01:00
Zach Musgrave 23df5c88fe Merge pull request #2211 from Juneezee/deprecate-ioutil
refactor: move from io/ioutil to io and os package
2021-10-04 14:10:04 -07:00
Maximilian Hoffman 9a7a49f516 initial dolt backup cli (#2208)
* prototype

* backup pushes noms root

* backup pushes root value, basic restore still needs origin drop

* bats tests, no origin on restore, only head ref on restore

* cleanup code, up to date bats test

* make backup restore a root sync also, to catch branches and remote refs

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

* couple of tim's comments

couple of tim's comments

* zach's comments

* loosen restrictions for conflicting remote addresses

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

Co-authored-by: max-hoffman <max-hoffman@users.noreply.github.com>
2021-10-04 11:53:05 -07:00
Eng Zer Jun 8164a625c7 refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-10-05 01:40:59 +08:00
Maximilian Hoffman 8a5f3f54be config, clone, backups refactor (#2196)
* shittiest read replica imaginable is kind of working

* import cycle progress

* delete unecessary files and fix  db type switch bug

* Add bats test

* delete comments

* fix working set updates for cli

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

* clean comments

* comment racy server test

* move env variables to local dolt config

* refactor clone, fix multienv test

* cleanup comments

* missing copyright

* brian's comments, add compile time checks for config interfaces

* format

* fix windows filepaths issue

* file:/// with three slashes should work on windows

* more windows problems

* three slashes didn't work for clone, do chdir to ref local dolt db

Co-authored-by: max-hoffman <max-hoffman@users.noreply.github.com>
2021-09-29 19:10:37 -07:00
Brian Hendriks 3070d3af9f merge rows on dolt table import -u when import data only represents a subset of the columns (#2105) 2021-09-16 14:06:45 -07:00
Brian Hendriks 775ac0af51 Import Performance and Garbage Generation improvements. 2021-09-08 14:52:42 -07:00
Maximilian Hoffman f5d63f4e3d move common merge/push/pull code into doltcore/env (#2063)
* Start moving code out of cmd

* hacky cmd -> action/evn refactor

* progress

* String verbose error from lib, mostly not reconstructed on cli side

* Missing verbose errors in lib, still not handled in cli. Refactor vinai's doc diffs to not depend on merge

* Small progress commit on other laptop

* Merge bats passing

* format

* progress

* Remotes bats tests working

* More error fixes

* Fix cyclic foreign keys bats

* Fix import cycle

* Start cleaning up code

* Fix a couple bats

* Clean up comments and println

* Brian's comments
2021-08-31 15:23:23 -07:00
Brian Hendriks 0a0ce2181c Bh/gen cs (#1938) 2021-08-05 18:20:19 -07:00
Daylon Wilkins 1d33bde91b Reworked verify-constraints and renamed to constraints verify 2021-07-20 18:16:38 -07:00
Daylon Wilkins 8d971dcd35 Added fast forward handling and fleshed out constraint violation messages 2021-07-15 17:16:16 -07:00
Daylon Wilkins 6b2b68923b Further attempts to fix ACCESS_DENIED error on Windows 2021-07-15 13:26:59 -07:00
Zach Musgrave 475a6c4787 Merge pull request #1890 from dolthub/zachmu/tx-enabled
Enable SQL transactions by default, and begin writing working set to database instead of repo_state.json file
2021-07-06 18:12:47 -07:00
Brian Hendriks 88fd93e7cc Bh/push pull temptf (#1885) 2021-07-06 17:09:16 -07:00
Zach Musgrave e78ed3b263 Merge master 2021-07-04 09:30:24 -07:00
Brian Hendriks 86bd20a4cd dolt roots command (#1891) 2021-07-04 07:51:07 -07:00
Zach Musgrave d8dfd68383 Migrating workingset from existing repo state file on load 2021-06-23 17:33:40 -07:00
Aaron Son 9b763af5a6 libraries/utils/async/ring_buffer.go: PR feedback. Return error on epoch mismatch. 2021-03-24 15:45:58 -07:00
Aaron Son b77f2bad8c go/libraries/doltcore/sqle/async_indexed_lookups.go: Fix leaking of results from one index lookup to another.
When a query with an index lookup as a LIMIT, it's possible for the result
iterator to be closed before all the index results have been streamed. When
that happens, we can end up with queueRows goroutine which is enqueuing rows
into a ring buffer that is going to be used by a different query later.

This PR adds two mechanisms, one to stop the wrong results from being seen and
to cancel the unnecessary work.

1. RingBuffer gets an epoch, which is incremented on each Reset(). If a given
   Push does not match the current epoch, then nothing is added to the ring
   buffer.

2. queueRows is run with a cancelable (sub-)Context, and that Context is
   canceled when the corresponding iterator is Closed().
2021-03-24 14:11:09 -07:00
Brian Hendriks c8264ef610 add file copy helper method to iohelp package (#1403) 2021-03-09 16:46:50 -08:00
Brian Hendriks 04d189937e chunk store in a file no manifest needed (#1401) 2021-03-09 16:46:08 -08:00
Brian Hendriks 69de83a757 Fixed worker pool used for index lookups (#1304) 2021-02-16 17:28:20 -08:00
Aaron Son 2ed61f501f go/libraries/utils/tracing: Add pluggable context-aware Tracing function. 2021-01-07 11:59:18 -08:00
Aaron Son a1bf8b62fd repofmt: Hygiene. 2021-01-06 15:02:24 -08:00
Aaron Son a654f02e14 go/cmd/dolt: Add debugging flag to run with Jaeger span reporting to localhost.
This adds a --jaeger flag to dolt CLI which installs a Jaeger Tracer as the
global opentracing Tracer. The Tracer is configured to report to an HTTP
collector running on http://localhost:14268, which is the port that docker
image `jaegertracing/all-in-one` listens on.

Also adds some parameters in places where Dolt constructs sql.Contexts to pass
the correct Tracer through.

Also adds a few new Span points, in things like nbs.Get and
metaSequenceImpl.getChildSequence.
2021-01-06 14:50:06 -08:00
Andy Arthur b635b72e7b match keyless tables by column intersection, ddl diff bats 2020-12-17 11:54:00 -08:00
Brian Hendriks 602f0ae2c0 partition ranges, covering indexes, smarter iterators (#1116) 2020-12-15 18:44:57 -08:00
Brian Hendriks ffab301d8c PR feedback 2020-11-24 15:39:54 -08:00
Brian Hendriks d26d82065b Merge remote-tracking branch 'remotes/origin/master' into bh/pipelinev2 2020-11-05 11:43:18 -08:00
Brian Hendriks ea852be8b2 test fixes 2020-11-04 17:09:35 -08:00
Brian Hendriks 493713eec5 Merge remote-tracking branch 'remotes/origin/master' into bh/pipelinev2 2020-11-02 16:02:41 -08:00
Aaron Son 8f2414ee1e go/libraries/util/async/errgroup{,_test}.go: Update copyright headers for Dolthub. 2020-11-02 11:09:44 -08:00
Aaron Son eb96988abb Merge remote-tracking branch 'origin/master' into aaron/diff-summary-error-handling 2020-11-02 11:07:55 -08:00