Commit Graph

31 Commits

Author SHA1 Message Date
Francesco Mazzoli
df9efa481d Keep pinging shuckle 2023-01-30 12:18:46 +00:00
Francesco Mazzoli
85889266b1 Various housekeeping while I get ready to deploy...
...most notably we now produce fully static binaries in an alpine
image.

A few assorted thoughts:

* I really like static binaries, ideally I'd like to run EggsFS
    deployments with just systemd scripts and a few binaries.

* Go already does this, which is great.

* C++ does not, which is less great.

* Linking statically against `glibc` works, but is unsupported.
    Not only stuff like NSS (which `gethostbyname` requires)
    straight up does not work, unless you build `glibc` with
    unsupported and currently apparently broken flags
    (`--enable-static-nss`), but also other stuff is subtly
    broken (I couldn't remember exactly what was broken,
    but see comments such as
    <https://github.com/haskell/haskell-language-server/issues/2431#issuecomment-985880838>).

* So we're left with alternative libcs -- the most popular being
    musl.

* The simplest way to build a C++ application using musl is to just
    build on a system where musl is already the default libc -- such
    as alpine linux.

The backtrace support is in a bit of a bad state. Exception stacktraces
work on musl, but DWARF seems to be broken on the normal release build.

Moreover, libunwind doesn't play well with musl's signal handler:
<https://maskray.me/blog/2022-04-10-unwinding-through-signal-handler>.

Keeping it working seems to be a bit of a chore, and I'm going to revisit
it later.

In the meantime, gdb stack traces do work fine.
2023-01-29 21:41:40 +00:00
Francesco Mazzoli
9adca070ba Convert build system to cmake
Also, produce fully static binaries. This means that `gethostname`
does not work (doesn't work with static glibc unless you build it
with `--enable-static-nss`, which no distro builds glibc with).
2023-01-26 23:20:58 +00:00
Francesco Mazzoli
aac9e275d7 Go blockservice, and a bunch of shuckle improvements 2023-01-24 11:44:46 +00:00
Francesco Mazzoli
adfa282dbd Test inline bodies in integration test 2023-01-23 18:22:18 +00:00
Francesco Mazzoli
51d0769cb3 Test block migration 2023-01-19 14:25:47 +00:00
Francesco Mazzoli
5acefed1a7 Implement loop check in rename directory 2023-01-18 10:46:11 +00:00
Francesco Mazzoli
ac99f10f94 Add artificial packet drop to integration tests...
...and fixup many places in the code to allow for such drops to
happen somewhat smoothly.
2023-01-16 22:54:51 +00:00
Francesco Mazzoli
4d03035e00 Forgot to close the door 2023-01-11 14:04:11 +00:00
Francesco Mazzoli
89e640d7dd Remove OpenSSL dependency 2023-01-11 13:53:30 +00:00
Francesco Mazzoli
e9ab9d58bd Clean up various things including README, preparing for merge 2023-01-11 11:16:13 +00:00
Francesco Mazzoli
781bf9f228 Add a simple FS test involving many directories
Fix some bugs in the process
2023-01-10 16:56:30 +00:00
Francesco Mazzoli
9c1b44111b As always, my laziness immediately bites me 2023-01-04 09:42:44 +00:00
Francesco Mazzoli
b91b9c3e74 C++ CDC, happy path somewhat tested, but a lot to go 2023-01-03 17:41:33 +00:00
Francesco Mazzoli
8e59a68708 Begin C++ CDC 2022-12-15 18:22:35 +00:00
Francesco Mazzoli
c66fc4f4a5 Seemingly functional span adding/removal...
...much more testing needed.
2022-12-15 13:36:15 +00:00
Francesco Mazzoli
c1434ae040 Read/write spans 2022-12-13 22:48:35 +00:00
Francesco Mazzoli
ba0f8a7a91 Move things around a bit in preparation for cpp CDC
Also, make log preparation in shard read-only, in preparation for
distributed consensus.
2022-12-11 20:08:31 +00:00
Francesco Mazzoli
f94c911f14 Push the log-handling logic outside ShardDB
As I'm thinking what to do of the CDC for now, I got thinking
a bit more about how to handle the log entry/state persisting split.

I think this makes more sense, it'll allow the log consensus module
to be bolted on top fairly cleanly.
2022-12-10 12:36:03 +00:00
Francesco Mazzoli
fc0cee0e07 C++ shard
Most operations apart from spans-related ones work. Using this as
a checkpoint -- the Python code is currently not really working,
I'm working to migrate to pretty much a full C++/go world.
2022-12-10 10:55:25 +00:00
Francesco Mazzoli
adba0ff39d Bulk migration 2022-11-05 22:34:25 +00:00
Francesco Mazzoli
49036030ce wip 2022-11-03 18:11:45 +00:00
Francesco Mazzoli
5d94c8129b Refactoring block-related stuff
Mostly, we now have ext block ids everywhere, and block service
requests take the block id.

Also, gearing up to explicit blacklists for block services,
in case a client detects failures.
2022-11-03 14:46:18 +00:00
Francesco Mazzoli
ea8947fa2e Implement per-directory policies 2022-10-27 14:07:09 +00:00
Francesco Mazzoli
c1c2619f8d GC basically done, although not thoroughly tested 2022-10-25 14:20:54 +00:00
Francesco Mazzoli
2652e558c3 More GC work 2022-10-24 10:45:04 +00:00
Francesco Mazzoli
5750727b9c Return DIRECTORY_NOT_FOUND when current directory is not present 2022-10-22 10:25:46 +00:00
Francesco Mazzoli
80ff4153c9 Various fixes/improvements leading up to GC 2022-10-21 15:06:59 +00:00
Francesco Mazzoli
e0535b83b5 Prevent creation of current edges in dead directories 2022-10-21 09:54:50 +00:00
Francesco Mazzoli
db668c072e goify CDC messages, too 2022-10-20 21:16:03 +00:00
Francesco Mazzoli
be20606781 Various changes gearing up to GC
Most notably we generate messages automatically from Go.
2022-10-20 16:59:22 +00:00