Commit Graph

5825 Commits

Author SHA1 Message Date
Dustin Brown
531b2f4816 Merge pull request #237 from liquidata-inc/db/change-benchmark-results
go/performance/benchmarks: remove id from results
2019-12-06 16:14:54 -08:00
Dustin Brown
58fa8d69da go/performance/benchmarks: remove id from results 2019-12-06 15:41:48 -08:00
Brian Hendriks
860e690aff add date to commit and init
* add --date param support to commit
* add --date param support to init
2019-12-06 15:01:46 -08:00
Tim Sehn
bc6347917b Fixed small typo in Dolt README 2019-12-06 13:57:15 -08:00
Tim Sehn
1222b0f459 Merge pull request #236 from liquidata-inc/tim/unskip-bats-test
Noticed an alter table test that now works was skipped. Unskipped.
2019-12-06 13:11:24 -08:00
Timothy Sehn
713c5f050c Noticed an later table test that now works was skipped. Unskipped. 2019-12-06 12:07:06 -08:00
Nam D. Nguyen
39cf275082 Fix typo in README for table import (#235) 2019-12-06 11:07:19 -08:00
Zach Musgrave
d2ebd8736b Merge pull request #233 from liquidata-inc/zachmu/sql-batch
Implemented batch insert semantics for the new SQL engine.
2019-12-05 14:06:29 -08:00
Zach Musgrave
24d0956b77 Removed obsolete test
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-05 13:52:52 -08:00
Zach Musgrave
2016326686 Added logic to silently skip empty statements when parsing
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-05 13:16:20 -08:00
Zach Musgrave
0a3a8cd438 Updated DropTable to un-cache the table.
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-05 13:14:11 -08:00
Zach Musgrave
94aa7f54b9 Bug fix in flush: forget tables when flushing
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-05 13:12:24 -08:00
Zach Musgrave
fdf6ac4ac7 Couple performance fixes: 1) cache the SQL schema for a table, and 2) only update output every 500 lines. Together these give us a 60% speedup on a large import.
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-05 12:42:03 -08:00
Zach Musgrave
b574e6c20c PR feedback
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-05 11:38:45 -08:00
Zach Musgrave
c3cae470fe Better error messages for duplicate primary keys on insert
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-05 11:29:28 -08:00
AndyA
7bece1a865 Merge pull request #232 from liquidata-inc/andy/sqldiffrefactor
Andy/sqldiffrefactor
2019-12-04 20:49:13 -08:00
Zach Musgrave
9764ed3f7d Updated go-mysql-server dependency version, which contains a bug fix needed for one of the tests to run correctly.
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 16:49:37 -08:00
Zach Musgrave
c9557a9805 Formatting
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 15:57:49 -08:00
Zach Musgrave
4c0e6087f9 Killed unused sqlinsert.go and tests. Removed remaining reference to deleted batcher in test util code.
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 15:57:04 -08:00
Andy Arthur
1c7b9c6c65 repo fmt 2019-12-04 15:49:48 -08:00
Andy Arthur
e08f1848fd created library methods for schema modification sql statements 2019-12-04 15:46:38 -08:00
Zach Musgrave
3795238130 Flush batch insert cache every 50k rows
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 15:38:45 -08:00
Zach Musgrave
497989ed15 Deleted now unused sqlbatcher.go, formatted other files
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 15:31:24 -08:00
Zach Musgrave
6e7676a571 Fixed the rest of the tests for batch inserts
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 15:23:04 -08:00
Zach Musgrave
c7bed47c4f Fixed a bug in batch mode: re-use the same tables for every query rather than creating them each time. Ported part of the sql batcher tests to use the new batch mechanism.
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 15:03:22 -08:00
Andy Arthur
e29e9cf900 grouped sql statment generation into a file, moved tests from sqlExportWriter to sql pkg 2019-12-04 14:36:03 -08:00
Zach Musgrave
b0c2551dd4 Wired new batching method into the sql command, no tests yet
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 14:23:28 -08:00
Andy Arthur
85225b9961 moved sql generation from SqlExportWriter to sql pkg 2019-12-04 13:43:37 -08:00
Zach Musgrave
f28ef1611f First pass at batched database interface
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 11:29:46 -08:00
Zach Musgrave
ac85b3806d Added a caveat to the pile of sugar-coated broken glass I just committed
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 11:19:33 -08:00
Zach Musgrave
4faea631ea Fixed bug in table editor for inserts combined with updates. Still kind of fragile and broken in other ways, probably.
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 11:16:36 -08:00
Zach Musgrave
f3fe71ec31 Added batch awareness to table editor / database, needs testing. Reverted change to table editor to disallow changing primary keys, since that is required for some updates statements. Have a bug to fix in the table editor as a result.
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-04 10:24:40 -08:00
Zach Musgrave
000657175f Tests of insert, update, delete on a single table_editor
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-03 16:59:11 -08:00
Zach Musgrave
dbdb5acc18 More tests and a bug fix for table editor
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-03 16:41:37 -08:00
Aaron Son
d4ca034c94 Merge pull request #230 from liquidata-inc/aaron/store-table-set-Rebase-reuse-open-table-files
go/store/nbs: table_set.go: Rebase: Reuse upstream table file instances when supplied table specs correspond to them.
2019-12-03 15:46:22 -08:00
Aaron Son
1b9af21ce9 go/store/nbs: table_set.go: Rebase: Reuse upstream table file instances when supplied table specs correspond to them. 2019-12-03 15:14:54 -08:00
Dustin Brown
30e2a72278 Merge pull request #229 from liquidata-inc/db/fix-schema-diff
fix schema diff primary key changes
2019-12-03 12:35:35 -08:00
Zach Musgrave
7f1d268dac Factored table editor functions into their own file
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-03 12:22:57 -08:00
Dustin Brown
7ee1771593 add newline to test file 2019-12-03 12:20:42 -08:00
Zach Musgrave
3de3580714 Fixed a bug in inserting multiple values, got table tests working
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-03 12:10:20 -08:00
Dustin Brown
7377914904 rm unecessary checks in bats test, use sql.QuoteIdentifier func 2019-12-03 11:00:30 -08:00
Dustin Brown
9e25330597 fix dolt diff schema primary key changes 2019-12-02 17:54:53 -08:00
Dustin Brown
38b1905cd0 wip, fix dolt diff schema bug 2019-12-02 17:11:17 -08:00
Zach Musgrave
ce1485d214 First pass at combining all table editor interfaces into one struct, and a test. Not working yet.
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-12-02 17:01:17 -08:00
Zach Musgrave
b90a8171a9 Merge pull request #227 from liquidata-inc/zachmu/sqllogictest
Bug fixes for sqllogictest dolt harness:
* More inclusive types
* Better error handling for panics
* Cheat on tables without primary keys to allow more tests (~40%) to succeed.
2019-11-27 15:15:49 -08:00
Zach Musgrave
08e1cbd2af Fixed formatting error
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-11-27 14:47:03 -08:00
Zach Musgrave
bd8c323f0f Updated dependencies for vitess, go-mysql-server, sqllogictest. Ran go mod tidy.
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-11-27 14:32:15 -08:00
Zach Musgrave
78356104bd Bug fixes for sqllogictest dolt harness:
* More inclusive types
* Better error handling for panics
* Cheat on tables without primary keys to allow more tests (~40%) to succeed.

Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-11-27 14:16:54 -08:00
Dustin Brown
7444e33fb0 Merge pull request #225 from liquidata-inc/db/disable-sql-benchmarks
disable benchmarking dolt sql imports
2019-11-26 15:21:24 -08:00
Dustin Brown
c0d8c12bb1 disable benchmarking dolt sql imports 2019-11-26 15:08:02 -08:00