Commit Graph

36 Commits

Author SHA1 Message Date
Francesco Mazzoli a512f07da1 Use DNS to reach shuckle, kill shuckle beacon 2025-05-29 13:40:21 +00:00
Miroslav Crnic 760859c041 fuse: file/dir attr cache and more efficient file fetch 2025-03-14 13:49:18 +00:00
Miroslav Crnic 452ff75ea0 shard: allow initiating spans at non default locations 2024-12-03 13:08:22 +00:00
Miroslav Crnic e1289d637f blocks: reports location 2024-11-21 00:06:06 +00:00
Miroslav Crnic d510cfd1c8 handle 1 nic down in all services 2024-11-20 23:18:26 +00:00
Miroslav Crnic 1a47089b3d shard: proxy read/write 2024-11-17 16:38:43 +00:00
Miroslav Crnic 48c3aa7d4a logsdb: enable partial leader election 2024-10-11 09:52:18 +01:00
Miroslav Crnic d026c21831 fix tests 2024-09-20 16:05:48 +00:00
Miroslav Crnic d5522678ef eggstests: use old/new fetch block api at random to test both 2024-09-03 18:14:43 +01:00
Miroslav Crnic f5e17dace5 cdc: add LogsDB
* cdc: pack req/resp into log entries and apply

* shard: drop support for unused incomming packet drop

* cdc: add logsdb
2024-05-14 12:50:17 +01:00
Francesco Mazzoli 13bf9a005a Run fsck at the end of tests 2024-04-09 07:53:10 +01:00
Miroslav Crnic 27faaa45ae ci: add ability to run with LogsDB, shard: add handling of LogsDB messages 2024-03-15 16:49:39 +00:00
Francesco Mazzoli 32cf90cd09 Add symbolizer path 2023-11-21 15:55:17 +00:00
Francesco Mazzoli 248abb2681 Fix memory leak in shards 2023-11-15 12:20:16 +00:00
Francesco Mazzoli 95a3406ff6 Add shuckle beacon and test it in the kmod
See #112.
2023-11-14 15:01:15 +00:00
Francesco Mazzoli 6c26349953 Store kmod on failure, so that I can investigate backtraces 2023-11-14 13:49:36 +00:00
Francesco Mazzoli 340e7f2f37 Harmonize addr-passing, add shuckle beacon and test it in kmod 2023-11-14 13:49:36 +00:00
Francesco Mazzoli f483a9c59f Lower initial timeouts in tests to speed them up...
...when we run with simulated packet drop.
2023-11-14 13:49:36 +00:00
Saulius Grusnys 2ce5586eb9 Periodically refresh metadata info in kmod, use two IPs for shuckle
Fixes #112.

Co-authored-by: Francesco Mazzoli <francesco.mazzoli@xtxmarkets.com>
2023-11-14 13:49:36 +00:00
Francesco Mazzoli 48985a0394 Add pagination to shuckle index table 2023-10-13 14:24:12 +00:00
Francesco Mazzoli 19fbbefec8 Implement xmon quiet time in Go, see #72 2023-10-02 20:28:26 +00:00
Saulius Grusnys 113efd02a2 [shuckle] add minimum free bytes threshold to mark blockservice as NO_WRITE 2023-09-26 10:22:32 +01:00
Francesco Mazzoli 9405b64a76 Remove ExpireTransientFile, make future cutoff tunable
Fixes #48. Also, reorganize error handling in `eggsblocks` requests,
especially around write requests, which might help with #45.
2023-08-15 12:43:49 +01:00
Francesco Mazzoli 2713ba9ae1 Better logging when waiting for shuckle 2023-08-10 10:35:11 +01:00
Francesco Mazzoli 76f581f194 Rationalize timeout handling 2023-07-28 15:01:55 +00:00
Francesco Mazzoli a35ea2dacf Minor shuckle tweaks 2023-07-26 07:58:17 +00:00
Francesco Mazzoli 1a4301a499 Simplify go span read/write code, make it work with broken block services
And some other assorted changes.
2023-07-04 08:05:42 +00:00
Francesco Mazzoli 4e0e6fe8a8 Configurable CDC shard timeout
Running in valgrind seems to just not be able to process a small
FullReadDirReq in 100ms, which is a bit concerning, but I'll let
it slide for now.
2023-07-04 08:05:42 +00:00
Francesco Mazzoli 42268ccaeb Have many block services per process in tests like in production 2023-07-04 08:05:29 +00:00
Francesco Mazzoli 40bfd1ca3d Add ways to periodically kill block services
To test them going down.
2023-06-26 13:42:18 +00:00
Francesco Mazzoli e26eeaede1 Add "mtu" field to requests that benefit from it
Not used right now, but this way we can easily start stuffing more
data in responses.

I also split off some arguments in `NewClient`, unrelated change
(I wanted to pair the MTU with a single client, but I then realized
that it's enough to have it as some global property for now).
2023-06-15 11:57:05 +00:00
Francesco Mazzoli b041d14860 Add second ip/addr for CDC/shards too
This is one of the two data model/protocol changes I want to perform
before going into production, the other being file atime.

Right now the kernel module does not take advantage of this, but
it's OK since I tested the rest of the code reasonably and the goal
here is to perform the protocol/data changes.
2023-06-05 12:14:14 +00:00
Saulius Grusnys 3b503861e9 Switch shuckle to store data in sqlite db, add block services flags
Co-authored-by: Francesco Mazzoli <francesco.mazzoli@xtxmarkets.com>
2023-06-04 16:10:14 +00:00
Francesco Mazzoli b771c12763 Static go builds 2023-03-10 16:23:30 +00:00
Francesco Mazzoli 600ec5c7d0 FUSE driver reborn 2023-03-10 16:23:28 +00:00
Francesco Mazzoli 5bff9b8fae Many, many changes -- tests pass, but FUSE is currently not present
The main thing that's added is full RS support, but a lot of things
were rejigged along the way. The tests are still a bit lacking,
and will be augmented in future commits.
2023-03-03 16:42:22 +00:00