Commit Graph

387 Commits

Author SHA1 Message Date
Francesco Mazzoli d92265d1ce Better assertion still 2024-08-15 13:55:28 +00:00
Francesco Mazzoli 2d79d7156f More informative assertion 2024-08-15 13:44:37 +00:00
Miroslav Crnic 5acdd2c6dc eggsdbtools: remove fix-name-hash-mismatch as fix deployed 2024-07-29 15:12:44 +00:00
Miroslav Crnic 95a2681121 eggsdbtools: fix nameHash mismatch in edge keys for shard 0 2024-07-29 16:08:28 +01:00
Miroslav Crnic cf40e318ec shuckle: support BlockServicesWithFlagChangeReq 2024-07-24 10:08:01 +01:00
Miroslav Crnic a41a4b7482 shuckle: drop BlockServiceInfoWithoutFlagsLastChanged 2024-07-23 15:40:44 +01:00
Miroslav Crnic 1b258dc422 all: migrate to new BlockServiceInfo 2024-07-23 13:53:07 +01:00
Miroslav Crnic 49723653f8 shuckle: BlockServiceInfo backward compatibility
* shuckle: rename BlockServiceInfo to BlockServiceInfoWithoutFlagsLastChanged

* shuckle: handle AllBlockServices
2024-07-23 13:10:57 +01:00
Miroslav Crnic 18ddaaf5ea shard: stop alerting on possible two migrations at once 2024-07-15 09:39:00 +00:00
Miroslav Crnic e2bfb15c5f blockservice: add BlockFetchWithCrc 2024-07-12 14:24:37 +01:00
Miroslav Crnic 3195d39d9d stats: fully remove everywhere 2024-07-09 15:22:10 +00:00
Miroslav Crnic c7b6a1cbeb stats: stop producing them 2024-07-09 15:57:01 +01:00
Francesco Mazzoli 285ffd66c7 Comment 2024-07-02 15:44:22 +00:00
Miroslav Crnic f3b7ef4d94 eggsgc: destroy decommissioned blocks through shuckle 2024-07-02 09:52:20 +00:00
Miroslav Crnic 99aa882575 gcmigrate: support sharding 2024-06-20 15:30:51 +00:00
Miroslav Crnic 6cf8a01272 core: UDPSocketPair send in a loop to avoid sendmmsg UIO_MAXIOV limit 2024-06-19 17:23:10 +01:00
Miroslav Crnic 78baed62a5 cdc: request checkpoints from shard and push through log 2024-06-13 16:24:22 +01:00
Miroslav Crnic 9e13d6b56e shard: support checkpointed responses 2024-06-13 15:39:37 +01:00
Miroslav Crnic 9d06deeedc cdc: error part of shard response 2024-06-13 13:00:43 +01:00
Miroslav Crnic 2cd15fc0be core: various protocol changes 2024-06-13 09:13:11 +01:00
Miroslav Crnic 7aac745457 shard/cdc: fetch all replicas quickly unless in do not replicate mode 2024-06-12 16:08:29 +00:00
Miroslav Crnic 6eaed4ff0e core: remove Stopped (unused) 2024-06-12 15:10:58 +00:00
Miroslav Crnic 71ee6568c5 cdc: correctly name logsdb stats 2024-06-10 15:49:44 +00:00
Miroslav Crnic 170f2fbc61 logsdb: add stats and expose in shard/cdc 2024-06-10 16:24:49 +01:00
Miroslav Crnic ffb5989692 cdc: increase amount of message received to account for LogsDB msgs 2024-06-02 11:22:20 +00:00
Miroslav Crnic bb47c8f38d cdc: remove adaptive msg receive 2024-06-02 07:59:16 +00:00
Miroslav Crnic 98d0376fc9 cdc: dont forget requests until they are handed over to LogsDB 2024-05-24 10:16:54 +01:00
Miroslav Crnic 1f145c030e shard/cdc: support snapshoting 2024-05-23 10:17:59 +01:00
Miroslav Crnic 1446e4d0d2 cdc: force log cleanup after crash
Transactional db that CDC uses has a slightly
annoying property that it flushes WAL on transaction
start. As a result release point can get moved and
log records persisted even if we crash.
We want to remove them automatically for now.
2024-05-22 16:58:18 +00:00
Miroslav Crnic a377536b40 cdc: correctly rewind assumed logIndex 2024-05-22 15:28:15 +00:00
Miroslav Crnic f11b675807 shuckle: add cdc replicas to page 2024-05-22 11:57:34 +00:00
Miroslav Crnic 4e574374ca shard/cdc: cleanup logsdb options, hostmon name match service name 2024-05-22 10:21:41 +00:00
Miroslav Crnic 25e8264517 cdc: rewind expected LogIdx on append window full 2024-05-22 09:14:16 +00:00
Miroslav Crnic b524748210 cdc: use DEFAULT_UDP_MTU for serialization of entries 2024-05-21 14:23:27 +00:00
Miroslav Crnic 121340f1b2 cdc: log line fixes and handle interupt 2024-05-21 12:56:26 +00:00
Miroslav Crnic ab4c25e5e3 cdc: use normal buffer size in cdc sockets 2024-05-21 12:55:47 +00:00
Miroslav Crnic 8be746de5b cdc: differentiate replicas in xmon and in metrics 2024-05-21 09:51:37 +00:00
Miroslav Crnic 5d453179ad cdc: dont alert on missing replicas if replication is off 2024-05-20 13:10:31 +00:00
Francesco Mazzoli 6faa917c18 Add endpoint and cli util to resurrect files
Only works in the same shard, for now.
2024-05-20 12:06:15 +00:00
Miroslav Crnic 0b3348b458 SharedRocksDB: (ref) paths in constructor 2024-05-17 14:23:25 +00:00
Miroslav Crnic ab337068ad httplib: use poll instead of select 2024-05-17 14:23:01 +00:00
Miroslav Crnic f5e17dace5 cdc: add LogsDB
* cdc: pack req/resp into log entries and apply

* shard: drop support for unused incomming packet drop

* cdc: add logsdb
2024-05-14 12:50:17 +01:00
Miroslav Crnic 91d462ab0e UDPSocketPair: (fix) dont look at unused fd-s 2024-05-14 08:51:29 +00:00
Miroslav Crnic aa8925adf9 shard: fix stats 2024-05-04 09:17:45 +01:00
Miroslav Crnic 8a0ea10cde core: UDPSocketPair and use IpPort AddrsInfo everywhere
* core: UDPSocketPair and use IpPort AddrsInfo everywhere

* Refactor UDPSocketPair a bit

* ci: kmod always delete img before create

* shuckle: fix scripts/json marshal

---------

Co-authored-by: Francesco Mazzoli <francesco.mazzoli@xtxmarkets.com>
2024-05-03 11:32:07 +01:00
Francesco Mazzoli ded0787c18 Fix no-return function 2024-05-01 08:53:38 +00:00
Francesco Mazzoli cd8e52f8f7 Remove assertions in ShardDB
We got a crash because of it (presumably can happen if defrag
conflicts with migrate or something like that)
2024-05-01 08:13:19 +00:00
Miroslav Crnic bbe201964d shard: bump quiet time for metric inserter alert to 5 min 2024-04-30 17:54:13 +01:00
Francesco Mazzoli 40ed10b2c6 Bump quiet period for metrics alerts
VictoraMetrics is often acting up
2024-04-29 11:39:54 +00:00
Francesco Mazzoli d3be7bf53a Remove old-style register block service request 2024-04-22 19:20:04 +00:00