Commit Graph

5262 Commits

Author SHA1 Message Date
Zach Musgrave
9b9ec6dc3e Fixed a bug on windows when redirecting STDIN for SQL import, e.g. dolt sql < dump.sql. Also fixed up ip2nation sample so that it successfully imports
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-08-09 12:48:19 -07:00
Aaron Son
7e969ffb51 Merge pull request #18 from liquidata-inc/aaron/repofmt-updates
go/utils/repofmt/check_fmt.sh: Add a script to format the repo appropriately.
2019-08-07 13:15:07 -07:00
Aaron Son
5427c2c83f Merge pull request #19 from liquidata-inc/aaron/3pdeps-handle-brew-go
utils/3pdeps/main.go: Handle installation of brew go into libexec.
2019-08-07 13:14:59 -07:00
Aaron Son
a797ceeb20 utils/3pdeps/main.go: Handle installation of brew go into libexec. 2019-08-07 12:36:31 -07:00
Aaron Son
83b776af46 go/utils/repofmt/check_fmt.sh: Add a script to format the repo appropriately. 2019-08-07 12:20:25 -07:00
Zach Musgrave
6a35448c5a Added "why dolt" section to README about the name of the product 2019-08-07 09:07:55 -07:00
Zach Musgrave
b41a4f022d Update README.md
Fixed extra w. It's the world wide web, not the world wide whatever web
2019-08-06 16:03:22 -07:00
Aaron Son
84d9eded51 Merge pull request #15 from liquidata-inc/aaron/make-round-trip-floats-default-format
go/store/constants/version: Make floats round trip successfully in the default format.
2019-08-06 15:19:58 -07:00
Aaron Son
e8e81cc410 Merge pull request #16 from liquidata-inc/aaron/make-beta-dolthub-default-login-url
go/libraries/doltcore/env: Fix default login and remote api endpoints for beta.dolthub.com.
2019-08-06 15:15:20 -07:00
Aaron Son
e1fb528bf9 Merge remote-tracking branch 'origin/master' into aaron/make-round-trip-floats-default-format 2019-08-06 15:06:25 -07:00
Aaron Son
eeaf7ea71c go/libraries/doltcore/env: Fix default login and remote api endpoints for beta.dolthub.com. 2019-08-06 15:01:26 -07:00
Brian Hendriks
55c2412db3 error changes (#7)
The existing noms codebase uses panics for signaling errors in most cases.  This is not idiomatic go, and it is not the optimal behavior when building cloud services, or products on top of noms as a panic in any go routine causes the entire process to exit.

This is a large step in removing panics, and bubbling errors up through the code.
2019-08-06 15:00:02 -07:00
Aaron Son
b708eaa71e go/store: Minor improvements to test fixes. 2019-08-06 14:49:14 -07:00
Aaron Son
1ae5b494c5 go/store: Fix tests for default format LD1. 2019-08-06 14:33:31 -07:00
Aaron Son
6354d4b36f go/store/constants/version: Make floats round trip successfully in the default format. 2019-08-06 14:33:16 -07:00
Daylon Wilkins
8511bcff2d Fixed test issues 2019-08-05 16:06:33 -07:00
Daylon Wilkins
0f47192214 Changed sql-server params to make more sense 2019-08-05 11:56:00 -07:00
Zach Musgrave
b113cdb4a4 Merge pull request #11 from liquidata-inc/zachmu/sql-fast-inserts
Batch inserts for SQL import. This results in a 60x speed increase and much less disk usage.

Also in this change:

Changed how triggering memory / CPU profiling works -- now a command line flag rather than a source change
Better integration test for joins (stock market data)
Better reporting during SQL import
2019-08-05 11:11:54 -07:00
Zach Musgrave
db24a7dc9f Fixed yet more import format errors, I guess this is my life now
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-08-05 11:03:54 -07:00
Zach Musgrave
04c0f09393 Fixed import errors
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-08-05 10:53:25 -07:00
Zach Musgrave
2633a1e73c Deleted interactive SQL shell tests -- most recent change detects TTY and runs in batch mode, so these no long make sense in this context
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-08-05 10:26:59 -07:00
timsehn
e41d326b68 Update README.md 2019-08-05 10:11:05 -07:00
Zach Musgrave
2a6f288ec2 Changed SQL import to continuously output and delete stats similar to CSV import
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-08-02 17:19:37 -07:00
Zach Musgrave
16245edf73 Added more tests for batch insert, removed unused Update method
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-08-02 16:58:11 -07:00
Zach Musgrave
ee7f8f72a6 Fixed broken integratoin test (caused by faulty data), and applied batch processing to the SQL setup.
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-08-02 16:31:25 -07:00
Zach Musgrave
b913614c87 Peformance tuning for batch SQL insert for the SQL import use case. Now as fast as CSV for single core, but doesn't multiplex so is ~4x worse on my 4-core machine for large data sets. Added profiling command line flags to dolt command (instead of enabling profiling via code changes).
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-08-01 13:32:01 -07:00
Zach Musgrave
17ae881db1 Batch SQL inserts for the SQL command. Still 10x slower than CSV import.
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-07-31 16:31:07 -07:00
Zach Musgrave
5978dec627 Tests for batch inserts
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-07-31 11:59:44 -07:00
Zach Musgrave
88f9b30578 Fully working batch for a batch size of 1
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-07-30 13:37:55 -07:00
Zach Musgrave
6d534d79c1 First pass at batch insert
Signed-off-by: Zach Musgrave <zach@liquidata.co>
2019-07-30 12:18:28 -07:00
Daylon Wilkins
38ee17268b Jenkinsfile: Iterate. 2019-07-25 18:31:24 -07:00
Daylon Wilkins
abe09dcf67 Jenkinsfile: Iterate. 2019-07-25 18:31:24 -07:00
Daylon Wilkins
9c240e30b5 Jenkinsfile: Iterate. 2019-07-25 18:31:24 -07:00
Daylon Wilkins
97240b15e6 Jenkinsfile: Iterate. 2019-07-25 18:31:24 -07:00
Daylon Wilkins
db35f6f3eb Jenkinsfile: Iterate. 2019-07-25 18:31:24 -07:00
Daylon Wilkins
ded5752dc8 Jenkinsfile: Iterate. 2019-07-25 18:31:24 -07:00
Daylon Wilkins
ebb6e01216 Jenkinsfile: Iterate. 2019-07-25 18:31:24 -07:00
Daylon Wilkins
9c9d5c72aa Jenkinsfile: Iterate. 2019-07-25 18:31:24 -07:00
Daylon Wilkins
1ca507527d Jenkinsfile: Iterate. 2019-07-25 18:31:24 -07:00
Daylon Wilkins
fb69985a59 Jenkinsfile: Iterate. 2019-07-25 18:31:24 -07:00
Daylon Wilkins
9403075d50 Jenkinsfile: Iterate. 2019-07-25 18:31:24 -07:00
Daylon Wilkins
17210014fa Jenkinsfile: Iterate. 2019-07-25 18:31:24 -07:00
Daylon Wilkins
5c5fe2e017 Moved Windows test changes to new repo 2019-07-25 18:31:24 -07:00
Aaron Son
6bb6e684ac Merge pull request #8 from liquidata-inc/aaron/add-failed-import-bad-filename-test
Added failing bats test for a badly named file
2019-07-25 11:34:16 -07:00
Aaron Son
f9b3a559b5 Merge pull request #4 from liquidata-inc/aaron/assert-commit-authors-in-jenkins
Jenkinsfile: Iterate on asserting commit authors in Jenkins.
2019-07-25 11:33:39 -07:00
Aaron Son
b6295c700a go/utils/checkcommitters: Another pass at check_pr logic. 2019-07-25 11:24:53 -07:00
Aaron Son
095eb81591 Added failing bats test for a badly named file 2019-07-25 11:13:02 -07:00
Aaron Son
f9421eb8ba go/utils/checkcommitters: Add Taylor. 2019-07-25 10:36:26 -07:00
Matt Jesuele
3a643624a5 bats/git-dolt.bats: Use cp instead of ln to fix test on windows 2019-07-25 09:45:53 -07:00
Aaron Son
4ca565249f go/utils/checkcommitters/check_pr.sh: Fix cwd for run. 2019-07-24 20:36:36 -07:00