Commit Graph

28722 Commits

Author SHA1 Message Date
Zach Musgrave
28b37aa37e new gms 2025-12-10 16:20:23 -08:00
Zach Musgrave
b4f039a8fd new gms 2025-12-10 15:56:52 -08:00
Zach Musgrave
f0258044e4 new gms 2025-12-10 15:43:07 -08:00
zachmu
839163e887 [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh 2025-12-10 23:01:54 +00:00
Zach Musgrave
3f2bd78e71 new gms 2025-12-10 14:53:00 -08:00
Zach Musgrave
f6cc99c328 merge main 2025-12-10 14:52:42 -08:00
Zach Musgrave
21d10b4a19 missing copyright header 2025-12-10 14:11:22 -08:00
Dustin Brown
839a62d04f Merge pull request #10193 from dolthub/coffeegoddd-73e40081
[auto-bump] [no-release-notes] dependency by coffeegoddd
2025-12-10 13:56:36 -08:00
Zach Musgrave
1874c65027 wrong gms 2025-12-10 13:36:15 -08:00
Zach Musgrave
be61fb4ef9 new gms 2025-12-10 13:20:37 -08:00
Zach Musgrave
ca6be820b9 Merge branch 'main' into zachmu/row-exec-2 2025-12-10 13:19:23 -08:00
coffeegoddd
0b3bfa1ad2 [ga-bump-dep] Bump dependency in Dolt by coffeegoddd 2025-12-10 20:39:20 +00:00
Elian
a1d078810e Merge pull request #10159 from dolthub/elian/7628c
Add `adapters.TableAdapter` to handle `dolt_status` and other table conversions for integrators (a.k.a. Doltgres)
2025-12-10 11:57:31 -08:00
elianddb
a354d667a0 add copyright in adapters/table.go 2025-12-10 10:16:38 -08:00
Zach Musgrave
6fa920d4d4 new gms 2025-12-09 16:05:54 -08:00
Zach Musgrave
210366c1bb Merge branch 'zachmu/row-exec-2' of github.com:dolthub/dolt into zachmu/row-exec-2 2025-12-09 16:03:32 -08:00
Elian
7cff94757a Merge pull request #10097 from dolthub/codeaucafe/1430
dolthub/dolt#10030: `--filter` contribution for `dolt diff`
2025-12-09 15:14:32 -08:00
Zach Musgrave
7125ee529d Safer type checking for equality 2025-12-09 14:56:58 -08:00
elianddb
904f8e281c [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh 2025-12-09 22:30:28 +00:00
zachmu
ca691eaa10 [ga-format-pr] Run go/utils/repofmt/format_repo.sh and go/Godeps/update.sh 2025-12-09 20:58:43 +00:00
Zach Musgrave
3b71615c4a bug fix for kvexec 2025-12-09 12:50:08 -08:00
Zach Musgrave
05adb0634b compile error 2025-12-09 12:29:15 -08:00
Zach Musgrave
53610b1068 merge main 2025-12-09 12:20:54 -08:00
Zach Musgrave
730249b661 merge main 2025-12-09 12:20:42 -08:00
Zach Musgrave
4b5ff0eff5 new gms 2025-12-09 12:19:59 -08:00
elianddb
d956afb08a rm privs check in dolt_purge_dropped_databases as it is already handled 2025-12-09 03:30:19 -08:00
coffeegoddd
4fd223dfda [ga-bump-release] Update Dolt version to 1.78.8 and release v1.78.8 v1.78.8 2025-12-09 04:40:11 +00:00
Aaron Son
7dcacba0f4 Merge pull request #10183 from dolthub/aaron/backup-sync-memory-opt
go: store/datas/pull: pull_chunk_tracker.go: Optimize memory use when backing up to an AWS remote.
2025-12-08 20:06:51 -08:00
Aaron Son
e63f912e55 go: store/datas/pull: pull_chunk_tracker.go: PR feedback. 2025-12-08 19:28:48 -08:00
Aaron Son
43da391c52 go: store/datas/pull: pull_chunk_tracker.go: Optimize memory use when backing up to an AWS remote.
PullChunkTracker is responsible for making the HasMany calls against the
destination and batching up absent hashes into HashSets which will be delivered
to GetManyCompressed and eventually written into table files which are
uploaded. This code is used for both pull and push, when the destination is the
"local" database or when destination is the remote database respectively. It is
used when the remote is both doltremoteapi, thus every HasMany call is an RPC,
and when the remote is something like file:// or aws://, thus the table file
indexes for the remote are in memory and HasMany calls are very quick.

Different operational characteristics of the various dependencies mean that
sometimes a Pull is prone to build up large sets of hashes waiting for HasMany
calls, whereas other times it is prone to build up large sets of absent hashes
which are waiting for the fetcher thread(s) to take them.

Previously, PullChunkTracker was structured to accumulate HasMany responses and
wait to batch them into appropriately-sized batches for GetManyCompressed until
the fetcher threads asked for them. This meant that if HasMany batches were
very small, because HasMany was very fast, we would accumulate a large number
of very small HashSets. These HashSets would take up large amounts of memory.
Accumulating the batches as the HasMany responses come in is more memory
efficient and should be no slower - we will always accumulate the full batches,
and in basically the same order.

Tested by pushing a large database to an AWS remote and memory profiling the
result.
2025-12-08 19:26:44 -08:00
Zach Musgrave
5583bca12d bug fix 2025-12-08 17:37:34 -08:00
Tim Sehn
172098be14 Merge pull request #10182 from dolthub/timsehn-46280834
[auto-bump] [no-release-notes] dependency by timsehn
2025-12-08 17:06:59 -08:00
timsehn
243d5e1190 [ga-bump-dep] Bump dependency in Dolt by timsehn 2025-12-09 00:36:40 +00:00
elianddb
16069d8889 rename table_adapter.go -> table.go 2025-12-08 16:22:04 -08:00
Zach Musgrave
85884d4159 Merge branch 'main' into zachmu/row-exec-2 2025-12-08 14:28:36 -08:00
Elian
a4087f5709 Merge pull request #10164 from dolthub/elian/10136
dolthub/dolt#10136: Fix `dolt_backup` to work in non-Dolt directories
2025-12-08 13:17:49 -08:00
elianddb
a17a3564b3 fix 'NewStatusTable' func doc 2025-12-08 12:42:04 -08:00
elianddb
f9ff5ed52f add test for Dolt table adapter registry 2025-12-08 12:01:03 -08:00
elianddb
6990d48af4 fix generated system table name set to use adapter names conditionally 2025-12-08 11:50:38 -08:00
elianddb
921d8a8909 amend func docs 2025-12-08 04:55:10 -08:00
elianddb
cd084b0cf1 rm extra default adapter and normalize internal alias instead 2025-12-08 04:49:38 -08:00
elianddb
752db82b84 add dolt table specific adapter registry for integrators 2025-12-05 16:32:41 -08:00
coffeegoddd
628b9dcce5 [ga-bump-release] Update Dolt version to 1.78.7 and release v1.78.7 v1.78.7 2025-12-05 23:21:23 +00:00
Nick Tobey
ef273f60d6 Merge pull request #10178 from dolthub/nicktobey/json-null
Report merge conflict when merging a JSON document that is NULL in the common ancestor.
2025-12-05 14:39:12 -08:00
Nick Tobey
a896b44cdc Merge pull request #10179 from dolthub/nicktobey-d4cc0dab
[auto-bump] [no-release-notes] dependency by nicktobey
2025-12-05 14:26:11 -08:00
elianddb
d7cf4b6d3b add dolt adapter to standardize table overwrites by integrators 2025-12-05 13:27:00 -08:00
nicktobey
9912094ce2 [ga-bump-dep] Bump dependency in Dolt by nicktobey 2025-12-05 21:13:48 +00:00
Nick Tobey
984eeba88d Report conflicts in JSON merge when one value is NULL 2025-12-05 13:03:03 -08:00
Nick Tobey
09b5037afe Add json merge tests with NULLs 2025-12-05 13:02:38 -08:00
Dustin Brown
f5562654c9 Merge pull request #10172 from dolthub/db/fix-workflow
[no-release-notes]: don't close failing auto bumps
2025-12-05 09:49:33 -08:00