Commit Graph

2234 Commits

Author SHA1 Message Date
Aaron Son 048589b6cd Merge remote-tracking branch 'origin/master' into aaron/create-view-bats-tests 2019-12-09 11:24:33 -08:00
Aaron Son f8aacfcb63 go/go.mod: Pick up view changes in go-mysql-server. 2019-12-09 11:22:18 -08:00
Dustin Brown 531b2f4816 Merge pull request #237 from liquidata-inc/db/change-benchmark-results
go/performance/benchmarks: remove id from results
2019-12-06 16:14:54 -08:00
Dustin Brown 58fa8d69da go/performance/benchmarks: remove id from results 2019-12-06 15:41:48 -08:00
Brian Hendriks 860e690aff add date to commit and init
* add --date param support to commit
* add --date param support to init
2019-12-06 15:01:46 -08:00
Zach Musgrave d2ebd8736b Merge pull request #233 from liquidata-inc/zachmu/sql-batch
Implemented batch insert semantics for the new SQL engine.
2019-12-05 14:06:29 -08:00
Zach Musgrave 24d0956b77 Removed obsolete test
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-05 13:52:52 -08:00
Zach Musgrave 2016326686 Added logic to silently skip empty statements when parsing
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-05 13:16:20 -08:00
Zach Musgrave 0a3a8cd438 Updated DropTable to un-cache the table.
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-05 13:14:11 -08:00
Zach Musgrave 94aa7f54b9 Bug fix in flush: forget tables when flushing
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-05 13:12:24 -08:00
Zach Musgrave fdf6ac4ac7 Couple performance fixes: 1) cache the SQL schema for a table, and 2) only update output every 500 lines. Together these give us a 60% speedup on a large import.
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-05 12:42:03 -08:00
Zach Musgrave b574e6c20c PR feedback
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-05 11:38:45 -08:00
Zach Musgrave c3cae470fe Better error messages for duplicate primary keys on insert
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-05 11:29:28 -08:00
Zach Musgrave 9764ed3f7d Updated go-mysql-server dependency version, which contains a bug fix needed for one of the tests to run correctly.
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 16:49:37 -08:00
Zach Musgrave c9557a9805 Formatting
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 15:57:49 -08:00
Zach Musgrave 4c0e6087f9 Killed unused sqlinsert.go and tests. Removed remaining reference to deleted batcher in test util code.
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 15:57:04 -08:00
Andy Arthur 1c7b9c6c65 repo fmt 2019-12-04 15:49:48 -08:00
Andy Arthur e08f1848fd created library methods for schema modification sql statements 2019-12-04 15:46:38 -08:00
Zach Musgrave 3795238130 Flush batch insert cache every 50k rows
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 15:38:45 -08:00
Zach Musgrave 497989ed15 Deleted now unused sqlbatcher.go, formatted other files
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 15:31:24 -08:00
Zach Musgrave 6e7676a571 Fixed the rest of the tests for batch inserts
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 15:23:04 -08:00
Zach Musgrave c7bed47c4f Fixed a bug in batch mode: re-use the same tables for every query rather than creating them each time. Ported part of the sql batcher tests to use the new batch mechanism.
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 15:03:22 -08:00
Andy Arthur e29e9cf900 grouped sql statment generation into a file, moved tests from sqlExportWriter to sql pkg 2019-12-04 14:36:03 -08:00
Zach Musgrave b0c2551dd4 Wired new batching method into the sql command, no tests yet
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 14:23:28 -08:00
Andy Arthur 85225b9961 moved sql generation from SqlExportWriter to sql pkg 2019-12-04 13:43:37 -08:00
Zach Musgrave f28ef1611f First pass at batched database interface
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 11:29:46 -08:00
Zach Musgrave ac85b3806d Added a caveat to the pile of sugar-coated broken glass I just committed
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 11:19:33 -08:00
Zach Musgrave 4faea631ea Fixed bug in table editor for inserts combined with updates. Still kind of fragile and broken in other ways, probably.
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 11:16:36 -08:00
Zach Musgrave f3fe71ec31 Added batch awareness to table editor / database, needs testing. Reverted change to table editor to disallow changing primary keys, since that is required for some updates statements. Have a bug to fix in the table editor as a result.
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 10:24:40 -08:00
Zach Musgrave 000657175f Tests of insert, update, delete on a single table_editor
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-03 16:59:11 -08:00
Zach Musgrave dbdb5acc18 More tests and a bug fix for table editor
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-03 16:41:37 -08:00
Aaron Son d4ca034c94 Merge pull request #230 from liquidata-inc/aaron/store-table-set-Rebase-reuse-open-table-files
go/store/nbs: table_set.go: Rebase: Reuse upstream table file instances when supplied table specs correspond to them.
2019-12-03 15:46:22 -08:00
Aaron Son 1b9af21ce9 go/store/nbs: table_set.go: Rebase: Reuse upstream table file instances when supplied table specs correspond to them. 2019-12-03 15:14:54 -08:00
Dustin Brown 30e2a72278 Merge pull request #229 from liquidata-inc/db/fix-schema-diff
fix schema diff primary key changes
2019-12-03 12:35:35 -08:00
Zach Musgrave 7f1d268dac Factored table editor functions into their own file
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-03 12:22:57 -08:00
Zach Musgrave 3de3580714 Fixed a bug in inserting multiple values, got table tests working
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-03 12:10:20 -08:00
Dustin Brown 7377914904 rm unecessary checks in bats test, use sql.QuoteIdentifier func 2019-12-03 11:00:30 -08:00
Dustin Brown 9e25330597 fix dolt diff schema primary key changes 2019-12-02 17:54:53 -08:00
Dustin Brown 38b1905cd0 wip, fix dolt diff schema bug 2019-12-02 17:11:17 -08:00
Zach Musgrave ce1485d214 First pass at combining all table editor interfaces into one struct, and a test. Not working yet.
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-02 17:01:17 -08:00
Aaron Son 5ed24cdbe1 go/go.mod: Pick up view changes in go-mysql-server. 2019-12-02 10:51:33 -08:00
Zach Musgrave 08e1cbd2af Fixed formatting error
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-11-27 14:47:03 -08:00
Zach Musgrave bd8c323f0f Updated dependencies for vitess, go-mysql-server, sqllogictest. Ran go mod tidy.
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-11-27 14:32:15 -08:00
Zach Musgrave 78356104bd Bug fixes for sqllogictest dolt harness:
* More inclusive types
* Better error handling for panics
* Cheat on tables without primary keys to allow more tests (~40%) to succeed.

Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-11-27 14:16:54 -08:00
Dustin Brown c0d8c12bb1 disable benchmarking dolt sql imports 2019-11-26 15:08:02 -08:00
Dustin Brown 531218ba6d Add dolt performance benchmarks 2019-11-26 13:12:45 -08:00
Dustin Brown ba7e8f34ce WIP add dolt performance benchmarks 2019-11-26 10:27:14 -08:00
Aaron Son a5ed797530 go/cmd/dolt: commands/sql: Fix up comments for new function returns. 2019-11-25 12:01:05 -08:00
Aaron Son 1aff3ace98 go/cmd/dolt: commands/sql: Keep the sql engine around throughout the lifetime of the shell / batch import. 2019-11-25 11:53:08 -08:00
Brian Hendriks 549e8ea54f improved super schema names (#221) 2019-11-21 15:09:05 -08:00