Commit Graph

21725 Commits

Author SHA1 Message Date
Zach Musgrave ca43a45013 Fixed some bugs in serialization / deserialization, getting started on converting virtual columns on store / read 2023-10-27 16:48:04 -07:00
Zach Musgrave dabe6b9728 Bug fix for serialization, added generated tests to dolt_engine_tests 2023-10-26 13:01:20 -07:00
Zach Musgrave c084c31887 Fixed deserialization 2023-10-26 12:54:53 -07:00
Zach Musgrave 0e0a5ee137 Added generated and virtual markers to schema 2023-10-26 12:50:41 -07:00
Zach Musgrave 94541c890e Added new fields to schema.Column. Killed all un-named column struct constructions 2023-10-26 10:19:04 -07:00
Aaron Son 9c6c9761c9 Merge pull request #6883 from dolthub/aaron/aws-table-persister-uses-s3manager
Remotes: AWS: Fix a bug where uploading tables files to S3 could have unbounded memory usage.
2023-10-26 08:27:39 -07:00
Aaron Son c3af0c6b84 go/store/nbs: aws_table_persister.go: Convert to using s3manager.Uploader instead of hand rolled multipart upload.
s3manager.Uploader gets us pooled byte slices, sane sizing and concurrency
limits, optimizations to use io.ReadSeeker when it's available, etc.

Fixes a bug where aws_table_persister could have unbounded memory usage,
proportional to the size of the file, when uploading large table files to S3.
2023-10-25 16:27:58 -07:00
Aaron Son 21c1a1329c go/store/nbs: Convert to use s3iface.S3API as the minimally mockable interface for our S3 client. 2023-10-25 15:19:17 -07:00
Aaron Son 6995bad390 go/store/nbs: aws_table_persister.go: Thread the io.Reader to uploadParts. 2023-10-25 15:09:59 -07:00
Tim Sehn ce375e4300 Merge pull request #6871 from dolthub/tim/unskip-bats
Remove skip on now passing test for multi-db joins on tables with the same name
2023-10-25 14:10:50 -07:00
Dustin Brown 4dbf6c4674 [auto-bump] [no-release-notes] dependency by max-hoffman (#6877)
* [ga-bump-dep] Bump dependency in Dolt by max-hoffman

* Refactor to account for GMS interface change

---------

Co-authored-by: max-hoffman <maximilian.wolfgang1@gmail.com>
Co-authored-by: Neil Macneale IV <neil@dolthub.com>
2023-10-25 12:58:40 -07:00
Tim Sehn 0979ffff47 Unskipped a bats test I added for a join across multiple databases on tables of the same name that is now fixed. 2023-10-24 16:03:22 -07:00
fulghum 17937a905a [ga-bump-release] Update Dolt version to 1.21.4 and release v1.21.4 v1.21.4 2023-10-24 22:40:30 +00:00
Jason Fulghum 6d4dbfdda5 Merge pull request #6870 from dolthub/fulghum/revert
Revert "Change parent order processing in commit iterator …"
2023-10-24 15:30:02 -07:00
Jason Fulghum fbc63d3742 Revert "Change parent order processing in commit iterator so that we always traverse first the parent of a merge commit that supplied the commits merged in that commit."
This reverts commit 2e5e6aa968.
2023-10-24 14:03:19 -07:00
Nick Tobey b3a93c1242 Merge pull request #6866 from dolthub/nicktobey/multidb
Store source database info in tables and columns.
2023-10-24 13:02:19 -07:00
Nick Tobey f2f5bbbbe2 Replace BATS test with engine test for joining tables from different branches. 2023-10-24 11:39:17 -07:00
fulghum 4d8fe2a875 [ga-bump-release] Update Dolt version to 1.21.3 and release v1.21.3 v1.21.3 2023-10-24 17:43:04 +00:00
Jason Fulghum e9f89a12da Merge pull request #6862 from dolthub/fulghum/dolt-6861
Ensure `dolt_diff_<tablename>` reports most-specific commit that changed a row
2023-10-24 10:18:03 -07:00
nicktobey c2a4ed630e [ga-bump-dep] Bump dependency in Dolt by nicktobey 2023-10-24 09:37:30 -07:00
reltuk a8ae15ef42 [ga-bump-release] Update Dolt version to 1.21.2 and release v1.21.2 v1.21.2 2023-10-24 01:39:38 +00:00
Aaron Son 7f11dcbfc5 Merge pull request #6864 from dolthub/aaron/dbfactory-preiodically-refresh-aws-file-credentials
dbfactory: aws: For AWS remotes using an explicit file credentials, periodically refresh the credentials the AWS client uses from the file contents.
2023-10-23 18:30:20 -07:00
Nick Tobey ca85b446c3 Use database name for system tables and table functions. 2023-10-23 17:54:24 -07:00
Nick Tobey b6b71031de Change expected error in ambiguous select. 2023-10-23 16:55:08 -07:00
Aaron Son b083c24d69 dbfactory: aws: For AWS remotes using an explicit file credentials, periodically refresh the credentials the AWS client uses from the file contents.
Some use cases put attenuated, expiring credentials into files. It's nice to
pick up the new credentials without needing to recreate the client.
2023-10-23 16:54:55 -07:00
Nick Tobey 3af5f45273 Track database in system tables. 2023-10-23 16:45:12 -07:00
Jason Fulghum 2e5e6aa968 Change parent order processing in commit iterator so that we always traverse first the parent of a merge commit that supplied the commits merged in that commit.
Fixes: https://github.com/dolthub/dolt/issues/6861
2023-10-23 14:40:27 -07:00
stephanie 8084c49ffe Merge pull request #6858 from dolthub/steph/version-err
Allow `dolt version` in folders without full permissions
2023-10-23 14:35:34 -07:00
Nick Tobey f8af142404 Remove unused sqlSchema parameter. 2023-10-23 14:12:01 -07:00
stephanie 41a47d9b8e Merge pull request #6857 from dolthub/steph/import-error
Improve import table error message
2023-10-21 00:54:46 -07:00
Stephanie You a02ee53936 fix panic for mismatched column schema 2023-10-20 16:37:52 -07:00
Stephanie You 9ab9eb2a79 skip write permission check for dolt version 2023-10-20 16:21:19 -07:00
Stephanie You 93c44b6dac add file line number to import table error message 2023-10-20 15:23:19 -07:00
fulghum 029fa6c3e2 [ga-bump-release] Update Dolt version to 1.21.1 and release v1.21.1 v1.21.1 2023-10-20 21:43:19 +00:00
Stephanie You e2e1ca0279 add table name to import table error 2023-10-20 14:39:44 -07:00
Stephanie You 0c2c6270de add column names to import table error message 2023-10-20 14:33:44 -07:00
Jason Fulghum 8d4cea06b5 Merge pull request #6856 from dolthub/max-hoffman-724edc96
[auto-bump] [no-release-notes] dependency by max-hoffman
2023-10-20 14:33:13 -07:00
max-hoffman 635a5c5c04 [ga-bump-dep] Bump dependency in Dolt by max-hoffman 2023-10-20 20:02:13 +00:00
Maximilian Hoffman cb49ae6f9c has_ancestor() function (#6847)
* has_ancestor() function

* license

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

* zach's comments

* one more test

---------

Co-authored-by: max-hoffman <max-hoffman@users.noreply.github.com>
2023-10-19 20:21:07 -07:00
Dustin Brown acf1f40aca [ga-bump-dep] Bump dependency in Dolt by JCOR11599 (#6851)
Co-authored-by: JCOR11599 <jcor@ucsd.edu>
2023-10-19 14:57:54 -07:00
stephanie 84d2b376a1 Merge pull request #6845 from dolthub/steph/silent-push-pull
add --silent option for push, pull, fetch
2023-10-19 09:50:59 -07:00
stephanie 3688b8325e Merge pull request #6834 from dolthub/steph/ls
migrate dolt ls to use sql queries
2023-10-19 09:50:47 -07:00
Dustin Brown 9033421111 [ga-bump-dep] Bump dependency in Dolt by jennifersp (#6846) 2023-10-19 08:49:05 -07:00
Maximilian Hoffman 5364ef3786 JSON and stats interface refactor (#6833)
* JSON and stats interface refactor

* Fix some tests

* fix more tests

* [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh

* bump

* bump

* cleanup

* json keys are ordered

* bump

* keep ordered json string for CV types

---------

Co-authored-by: max-hoffman <max-hoffman@users.noreply.github.com>
2023-10-18 20:29:31 -07:00
JCOR11599 0bf4975025 [ga-bump-release] Update Dolt version to 1.21.0 and release v1.21.0 v1.21.0 2023-10-19 00:23:36 +00:00
Dustin Brown 55fee23044 [ga-bump-dep] Bump dependency in Dolt by JCOR11599 (#6842)
Co-authored-by: JCOR11599 <jcor@ucsd.edu>
2023-10-18 17:14:54 -07:00
stephanie 973e676a0e Merge branch 'main' into steph/ls 2023-10-18 15:52:26 -07:00
Stephanie You ab4e12c023 add --silent option for push, pull, fetch 2023-10-18 15:48:30 -07:00
stephanie 48f565111c Merge pull request #6840 from dolthub/steph/merge-base
migrate dolt merge-base to use sql queries
2023-10-18 15:06:50 -07:00
Stephanie You ba830df2ed typo fix 2023-10-18 15:03:57 -07:00