mirror/dolt
1
0
Fork 0
mirror of https://github.com/dolthub/dolt.git synced 2026-02-28 10:19:56 -06:00
Code Issues Packages Projects Releases 139 Wiki Activity
21,426 Commits 133 Branches 609 Tags
e4aa6ffad86dcb2aade0baeb85712e01718b1ca4
Commit Graph

49 Commits

Author SHA1 Message Date
Aaron Son
e4aa6ffad8 go/libraries/doltcore/sqle/cluster: First pass at replicating drop database. 2023-09-29 14:14:09 -07:00
Aaron Son
3b0ca8bb21 go/libraries/doltcore/sqle: cluster: When a database is dropped, shutdown the replication goroutines and stop returning its status in the dolt_cluster_status table. 2023-09-20 15:47:46 -07:00
Aaron Son
1002a5d542 go: sqle: cluster: First pass at replicating branch control permissions. 2023-09-14 13:37:35 -07:00
Aaron Son
8021bc5f02 sqle: cluster: Set the engine to read-only when a replica is in standby mode. Set it back to read-write when it becomes primary. 
This prevents standby replicas from running some DDL which they were previously
erroneously allowed to run, including CREATE USER, GRANT, CREATE DATABASE and
DROP DATABASE.
 2023-09-01 10:47:23 -07:00
Aaron Son
48cf308d05 Merge pull request #6454 from dolthub/aaron/cluster-replication-users-and-grants 
go/libraries/doltcore/sqle/cluster: Add replication of user and grant changes from primary to standby replicas.
 2023-08-22 13:28:35 -07:00
Aaron Son
6d7c73575d dolt sql-server: Add a system_variables: key to config.yaml, which allows setting global system variables. 
For example, this can be used to set `secure_file_priv` when the server starts
up. That variable is non-dynamic, so it cannot be set with `SET @@GLOBAL...`
 2023-08-15 15:53:18 -07:00
Aaron Son
08f91f74a0 go/libraries/doltcore/sqle/cluster: Add replication of user and grant changes from primary to standby replicas. 2023-08-04 11:33:57 -07:00
Aaron Son
c0b0cc42d1 dolt clone: Fix dolt clone run against a sql-server where the database has been GCd. 
A long-standing bug in the remotesapi which the sql-server exposes could cause
a `dolt clone` to fail when running against a database which had been garbage
collected. This change fixes the bug in the server. It also patches the client
behavior so that it will tolerate responses from older versions of dolt.
 2023-06-27 16:41:22 -07:00
Aaron Son
6314e27de9 Merge pull request #6204 from dolthub/aaron/cluster-assume-role-standby-changes 
sqle: cluster: Add dolt_cluster_transition_to_standby stored procedure.
 2023-06-22 14:03:13 -07:00
Aaron Son
155884711e sqle: cluster: dolt_cluster_transition_to_standby: Add logic to respect and enforce minCaughtUpReplicas. 2023-06-21 16:29:45 -07:00
Aaron Son
0a7cd7e033 sqle: cluster: dolt_cluster_transition_to_standby: Register the implementation (without transition transition thresholds) and test it. 2023-06-21 16:03:59 -07:00
Zach Musgrave
0b52f6c424 Revert "Revert "Merge pull request #5968 from dolthub/zachmu/tx-6"" 
This reverts commit bb03b8cb25.
 2023-06-08 11:09:54 -07:00
Zach Musgrave
bb03b8cb25 Revert "Merge pull request #5968 from dolthub/zachmu/tx-6" 
This reverts commit 51c2129e51, reversing
changes made to 9692f6ef92.
 2023-06-07 17:27:06 -07:00
Zach Musgrave
e672c304ab Merge main 2023-06-06 15:17:07 -07:00
Taylor Bantle
ea2e87eac8 Fix go-sql-server-driver tests 2023-06-05 13:39:58 -07:00
Taylor Bantle
ed8f23481c Integration tests for load data ignore/replace 2023-06-05 13:23:35 -07:00
Zach Musgrave
9b2cd87c57 Added a README 2023-05-22 13:49:32 -07:00
Aaron Son
6ea915ca4d Merge remote-tracking branch 'origin/main' into aaron/block-on-cluster-replication 2023-05-16 16:57:09 -07:00
Aaron Son
6ccae75a8d integration-tests/go-sql-server-driver: Add TestCluster/last_updated_heartbeats test which asserts that last_updated does not grow while there are no writes to replicate. 2023-05-16 14:21:41 -07:00
Aaron Son
535e59141d integration-tests/go-sql-server-driver: Cluster, dolt_cluster_ack_writes_timeout_secs, add a failing test to assert that dolt_branch modifications are waited on. 2023-05-15 14:00:04 -07:00
Aaron Son
dfdb4a4b2a go: sqle: cluster: Implement a first pass at dolt_cluster_ack_writes_timeout_secs. 
Setting this system variable to a non-zero value on a primary replica in a
sql-server cluster will cause dolt to block a SQL client performing a commit
until that client's commit is fully replicated to the replicas.

If there is a timeout, currently a warning is logged in the logs.
 2023-05-11 15:50:43 -07:00
Aaron Son
2674c3cf2a sql-server: Allow dolt_log_level system variable to receive writes and dynamically update the log level. 2023-05-10 14:46:45 -07:00
Aaron Son
663d2e3181 sql-server: Expose the current server log level through the dolt_log_level system variable. 2023-05-10 14:25:41 -07:00
Aaron Son
6e6ed1b77b Fix result ordering in integration test. 2023-05-09 14:08:19 -07:00
Aaron Son
acf99f1edf go: sqle: dolt_gc.go: Disable CALL dolt_gc() on a standby replica, where it is not currently safe. Shallow GC is still available. 2023-05-09 13:41:16 -07:00
Zach Musgrave
75b1395510 New integration test ala aaron 2023-05-05 07:42:32 -07:00
Aaron Son
1d0a6001ac go/libraries/doltcore/sqle: database_provider.go: When cloning a remote into a sql-server, always run the InitDatabaseHook on it. 
Failing to run the hook means that the cloned database is not appropriately
configured for replication, for example.

Fixes: #5850
 2023-05-01 12:42:02 -07:00
Aaron Son
d1bb4f6807 integration-tests/go-sql-server-driver: Add a test which fails when a non-chunk journal repo first switches to chunk journal and runs call dolt_gc. 2023-03-24 09:43:54 -07:00
Aaron Son
f7c69bf881 Merge pull request #4749 from dolthub/aaron/cluster-jwks-auth 
go/libraries/doltcore/sqle/cluster: Add JWT authentication for peer communication to cluster replicas.
 2022-11-15 12:21:55 -08:00
Aaron Son
6d7a569e61 integration-tests/go-sql-server-driver: Use 385{1,2} instead of 5005{1,2} for cluster remotesapi ports. 2022-11-10 16:34:49 -08:00
Aaron Son
b22fbf11f2 go/libraries/doltcore/sqle/cluster: Add JWT authentication for peer communication to cluster replicas. 
A remotesapi server running on a cluster replica publishes a JWKS.

Every outbound GRPC call the cluster replica makes includes a JWT signed with a
private key.

remotesapi servers running on cluster replicas require and validate incoming
JWTs for cluster traffic. The set of valid signing keys is taken from the
JWKSes which are published at /.well-known/jwks.json on the standby replica
hosts.

It is possible to configure tls_ca on cluster remotesapi to configure the set
of trusted roots for outbound TLS connections. Because the JWKSes are served
over the same connection, and becuase signed JWTs are not replay resistent, TLS
is recommended for all deployment topologies.
 2022-11-10 09:26:38 -08:00
Aaron Son
57523a9323 go/libraries/doltcore/sqle/cluster: Add the ability to generate a keypair, serve a JWKS, and sign outbound RPCs with a JWT. 2022-11-02 09:54:09 -07:00
Aaron Son
363248538e Merge pull request #4659 from dolthub/aaron/remotesrv-tls-san 
go/libraries/doltcore/sqle/cluster: Add the ability to configure SAN assertions on URIs and DNS names in the server certificate when configuring TLS.
 2022-10-28 16:29:39 -07:00
Aaron Son
d85ad51b4b Merge remote-tracking branch 'origin/main' into aaron/bats-move-sql-server-temporary-table-test 2022-10-28 13:33:00 -07:00
Aaron Son
248be8b21a go/libraries/doltcore/sqle/cluster: Add the ability to configure SAN assertions on URIs and DNS names in the server certificate when configuring TLS. 2022-10-28 10:31:14 -07:00
Aaron Son
4d93b69ef3 integration-tests/bats/sql-server.bats: Move the temporary table session specific test to go-sql-server-driver. 2022-10-27 09:43:29 -07:00
Aaron Son
27a528e498 go/libraries/doltcore/sqle/cluster: Add support for configured tls_{cert,key,ca} on the cluster.remotesapi. 
For now, this is server-side TLS, not mTLS.

If a tls_ca is configured, the certificates in that file are PEM encoded. They
are the only trusted roots for server certificate verification on both gRPC and
HTTP connections. If tls_ca is set, no server name verification is done for
now.
 2022-10-26 16:13:51 -07:00
Aaron Son
995e3818cd go: sqle: remotesrv: Successfully replicate CREATE DATABASE commands by instantiating new databases on the standby on demand. 2022-10-05 12:37:39 -07:00
Aaron Son
cc82355269 integration-tests/go-sql-server-driver: Fix JSON test for LD_1. 2022-10-04 11:49:11 -07:00
Aaron Son
c56b0630a9 integration-tests/go-sql-server-driver: Finish transitioning the rest of the sql-server-cluster tests. 2022-10-04 10:22:38 -07:00
Aaron Son
fa7db054ad integration-tests/go-sql-server-driver: Convert some more sql-server-cluster.bats tests. 2022-10-03 17:23:53 -07:00
Aaron Son
dd7b5b5bec integration-tests/go-sql-server-driver: Move over some more tests. 2022-10-03 16:37:24 -07:00
Aaron Son
364024f7a2 integration-tests/go-sql-server-driver: Make a new connection pool per connection assertion, instead of trying to reuse them. 
Simpler.

Much better structure for lots of tests which test things like connection
strings, users and passwords provided at connection time, etc.
 2022-10-03 15:26:48 -07:00
Aaron Son
bf7480eb1b integration-tests/go-sql-server-driver: sql-server-cluster.yaml: Fix dolt_assume_cluster_role tests for new behavior of not breaking connection when role does not change. 2022-10-03 15:02:40 -07:00
Aaron Son
153ea002e4 integration-tests/go-sql-server-driver: Start migrating over some simple tests from sql-server.bats. 2022-10-03 14:00:56 -07:00
Aaron Son
7ebc1bb45f integration-tests/go-sql-server-driver: Move Server definitions to TestRepo/MultiRepo, instead of a separate top-level field. 2022-10-03 12:04:35 -07:00
Aaron Son
87de2bb7d9 integration-tests/go-sql-server-driver: Add retry_attempts functionality, drop sleep_millis. Add some documentation. 2022-10-03 11:38:01 -07:00
Aaron Son
cf639c3585 integration-tests/go-sql-server-driver: Start converting some sql-server bats tests. 2022-10-03 10:01:47 -07:00
Aaron Son
1857806d2d integration-tests/go-sql-server-driver: Proof of concept a declarative approach to integration tests for sql-server interactions. 
Converts bats/sql-server-config.bats to use a new approach.
 2022-10-01 15:42:22 -07:00