Commit Graph

1664 Commits

Author SHA1 Message Date
Miroslav Crnic
d5c40f7e74 span.go write blocks in parallel 2025-11-17 10:43:28 +00:00
Copilot
045e9adb8a cdc: Fix various RenameDirectory issues
RenameDirectory state machine was not handling target not found correctly.
This would have caused asserts (which result in crashes in production builds)
There was also a bug in the rollback logic which would have caused a lingering
lock on the source link. While breaking assumptions this was a benign bug as
any operation on that directory would try and succeed acquiring this lock again.
It would succeed as lock requests are idempotent.
2025-11-13 15:09:34 +00:00
Miroslav Crnic
b110a7cb38 cdc: ignore unknown tags 2025-11-12 13:18:53 +00:00
Miroslav Crnic
2d7abe35b4 shard: support wait for state applied req 2025-11-12 09:12:07 +00:00
Miroslav Crnic
01cee15980 kmod: fix unsafe span rb tree erase 2025-11-10 13:41:36 +00:00
Miroslav Crnic
5b1a1351e2 kmod: cache inline spans indefinitely 2025-11-07 13:37:47 +00:00
Miroslav Crnic
844bc9adcc client: fix deadlock in fetchRsSpan 2025-11-07 12:26:40 +00:00
Miroslav Crnic
0436fe878c kmod: configurable span cache retention 2025-11-07 10:26:10 +00:00
Miroslav Crnic
9faa523871 ternclient: fix fetching mirrored span 2025-10-31 11:22:35 +00:00
Miroslav Crnic
2ed670a907 udpSocketPair: spread across sockets better
We were using last 2 bit of time but clock could be low precission.
Hashing the time is better here.
2025-10-27 15:30:16 +00:00
Miroslav Crnic
398af8d3cd kmod: memcmp ternfs_block_service 2025-10-27 10:50:10 +00:00
Francesco Mazzoli
02891b6863 Use mimalloc in release and alpine builds
This should make the alpine build usable in production.
2025-10-27 10:20:15 +00:00
Miroslav Crnic
b744242b5a kmod: dont compare block service padding when upserting 2025-10-27 10:19:37 +00:00
Miroslav Crnic
f95775e614 shard: metrics as simple counters 2025-10-24 16:57:23 +01:00
Miroslav Crnic
3e4652eec3 migrate: num-migrations-per-shard -> num-file-migrators 2025-10-24 16:06:58 +01:00
Miroslav Crnic
d4cb2d50cb kmod: need to check if ok to splice page 2025-10-21 22:53:59 +01:00
Miroslav Crnic
7758eb8938 kmod: synchronously fetch policy on dir inode lookup 2025-10-19 12:14:46 +01:00
Miroslav Crnic
d96abd3083 minor fixes 2025-10-16 17:12:37 +00:00
Miroslav Crnic
6bfd89dec7 options: parse -syslog 2025-10-16 14:32:37 +00:00
Francesco Mazzoli
c1e3fa9807 Add way to specify ubuntu build image 2025-10-16 14:01:06 +00:00
Miroslav Crnic
924e75674f shard: support multiple reader threads 2025-10-16 12:39:11 +01:00
Miroslav Crnic
8cec8bcf6b kmod: delete files immediately if policy allows 2025-10-15 22:58:58 +01:00
Francesco Mazzoli
7d92031472 Ignore errors when deleting files immediately (see comment) 2025-10-15 09:41:52 +01:00
Francesco Mazzoli
ee672c0d17 Delete files immediately when policy allows it
We shoud do the same on edge overwrite, but it's a bit more annoying
2025-10-15 09:41:52 +01:00
Miroslav Crnic
ffe3416f16 kmod: minor write path fixes
* kmod: minor write path fixes

We didn't actually see these happen in production.

Fix 1:
From kernel code it looks like copy_page_from_iter can not return 0 in
normal cases but our code should still cover the case if this changes in
the future.

Fix 2:
-ENOMEM was other error where we could write things partially in which
case we would not return written and we would end up at wrong offset.
It's simpler to just return written if we managed to write anything
and surface the error on subsequent call in which we will fail early.

* kmod: add BUG_ON for unexpected span pages
2025-10-10 16:03:43 +01:00
Francesco Mazzoli
29af79e8a8 Fix FUSE options 2025-10-09 22:53:25 +01:00
Francesco Mazzoli
e724223228 Do not ignore changes to the workflow file itself when doing CI 2025-10-09 21:05:28 +01:00
Francesco Mazzoli
641cc89d12 bpf building, take 2 2025-10-09 21:04:33 +01:00
Francesco Mazzoli
3a11d19b4e Allow to direct mount in FUSE 2025-10-09 10:26:51 +00:00
Francesco Mazzoli
d92f18a445 Do not print quiet alerts in FUSE by default 2025-10-09 10:11:07 +00:00
Francesco Mazzoli
e0e7cadfe9 Pin BPF source 2025-10-09 09:59:42 +00:00
Francesco Mazzoli
e24cdbebfe Make default attr cache in FUSE a bit less extreme 2025-10-09 09:46:48 +00:00
Francesco Mazzoli
e5423105f6 Allow to pass arbitrary options to ternfuse 2025-10-09 09:46:19 +00:00
Miroslav Crnic
bad26576a8 kmod: dont revalidate past fs boundary 2025-10-05 19:19:05 +01:00
Francesco Mazzoli
3aff31a026 Unbreak old kernels 2025-10-03 16:36:19 +00:00
Miroslav Crnic
27308ea4ef client: set deadline for blocks requests 2025-10-03 12:09:18 +00:00
Miroslav Crnic
9cba2856ba SPSC: another MultiWaiter fix
_queuesWithWork can go negative.
Consider the following scenario
there are 2 queues first one has work and
consumer is active.
1. producer adds work to queue 1 and sees it needs
  to update _queuesWithWork but is preempted
2. consumer consumes work from queue 0 and
   decrements _queuesWithWork to 0
3. consumer continues consuming and consumes all
   work from queue 1
4. consumer decrements _queuesWithWork to -1
5. only now producer updates _queuesWithWork to 0

It could be also solved by removeWork returning new value
and then consumer stopping consumption if it is 0.
2025-10-03 10:57:16 +00:00
Francesco Mazzoli
a49f29219b Do not start FUSE if we're running tests with kmod mount 2025-10-03 11:08:45 +01:00
Francesco Mazzoli
75b3808a0e Remove sysctl sentinels
You shouldn't put it from 6.11 onwards and in fact they mess things
up with CONFIG_RANDSTRUCT.

See <https://github.com/openzfs/zfs/issues/16620>
2025-10-03 11:08:45 +01:00
Francesco Mazzoli
6e46e9bf1c Start logging stuff before inserting module 2025-10-03 11:08:45 +01:00
Francesco Mazzoli
03a2a08ec9 Linux 6.12 2025-10-03 11:08:45 +01:00
Francesco Mazzoli
2b0444e218 Update README.md 2025-10-01 16:57:21 +01:00
Francesco Mazzoli
9ac86d31a3 Update README.md 2025-10-01 16:54:43 +01:00
Francesco Mazzoli
116c7495c9 Test that reads are precise when there are no failures 2025-10-01 14:09:17 +00:00
Francesco Mazzoli
7e24623dc5 Implement readdirplus in FUSE
This should greatly speed listings up.
2025-10-01 14:14:44 +01:00
Francesco Mazzoli
3737d9d629 Allow to not start GC in ternrun 2025-10-01 14:14:44 +01:00
Francesco Mazzoli
4faefcde73 Write out to cache, and enable page cache read/write in FUSE 2025-10-01 14:14:44 +01:00
Francesco Mazzoli
3562c1cbc7 Use clear when appropriate 2025-10-01 14:14:44 +01:00
Francesco Mazzoli
5d6a4386a7 Add page cache read functionality to span reader
Don't use it for now in FUSE, but test it in tests.
2025-10-01 14:14:44 +01:00
Francesco Mazzoli
b5f609e88e Precise Go RS span reads
The new code reads only what's needed in the happy path, and does not
fall back to reading the entire span when doing RS recovery (but it
might read slightly more than necessary in that case).

There's no lazy CRC check or page cache reading yet.
2025-10-01 14:14:44 +01:00