6b58a6b531
Fix error handling of Connection::execute_batch.
2025-05-06 10:22:41 +02:00
69b4f69d7c
Periodically yield back to the JS event loop while waiting for a DB write lock to allow it to make progress even while the lock is held.
2025-05-05 16:14:32 +02:00
c06b3ad1ef
Close sqlite connections locally rather than on the worker loops.
2025-05-05 10:30:08 +02:00
5f1b5614b4
Add an experimental API to get a write-lock to the underlying rusqlite::Connection.
2025-05-03 12:10:19 +02:00
c4a4dfd575
Update rusqlite to v0.35, which has more strict query validation \o/. Also update other Rust and JS deps.
2025-05-03 11:42:59 +02:00
122dd1ebb0
Allow applying recorded transaction as migration or plain transaction. Add tests.
2025-05-02 21:31:34 +02:00
e897169aac
Move JS runtime into a separate crate.
2025-04-29 13:47:10 +02:00
9ca34f46f3
Further tune SQLite execution model.
...
Previously reader and writer channels were operating entirely
independently, which yields the lowest overhead for pure read and pure
write benchmarks.
In mixed workloads like prod, however, this is not ideal since we're
running into SQLite's own file-locking, which has more overhead than
in-process locking.
With this change, we use an RwLock to reduce congestion while still
allowing parallel reads.
Together with a low-overhead kanal channel, we see little to no
regression for pure read and pure write benchmarks, while seeing about a
3x improvement for mixed workloads.
We also move ACL checks for Create/Update queries to the writer SQLite
connection, to reduce thrashing.
2025-04-28 14:11:44 +02:00
706870c50a
Minor: use const reference internally for SQLite read queue operations.
2025-04-27 12:07:58 +02:00
fe2e8cfda4
Clean up Sqlite connection setup, no-parallel reads for in-memory dbs, and add a placeholder storage for persisting apalis queues.
2025-04-23 11:56:36 +02:00
d6b199587d
Update to Rust edition 2024.
2025-04-20 07:36:26 +02:00
e0ad4a898c
Extend trailbase-sqlites execution model to allow for parallel reads. This reduces the latency long-tail for slow reads.
...
Note that this complicates the APIs, since it pushes the responsibility
of declaring a query a read or write to the user to then be scheduled
appropriately.
Add `.(read_|)query_row_f` APIs similar to rusqlites
`conn.query_row` accepting a `|row| -> Result<T>` to reduce the use of
`Row` and `Rows`.
Make benchmarks more isolated by not sharing a DB across runs
accumulating writes.
2025-04-15 12:15:40 +02:00
eecab32d39
Minor: consistent tokio and env_logger dependencies.
2025-04-14 09:25:44 +02:00
9e040d34f6
Make chat-room benchmark more realistic: hammer server in parallel.
2025-04-13 12:28:20 +02:00
914a9b3f03
Make query mix benchmark's slow queries slower and group benchmarks with throughput as requested.
2025-04-13 11:12:08 +02:00
844ccfa80c
Add a mixed query benchmark to trailbase-sqlite and make all benchmarks concurrently hammer the connection.
2025-04-12 15:23:17 +02:00
faa1712601
Add read benchmarks to trailbase-sqlite.
2025-04-12 13:09:22 +02:00
d0d60857eb
Add a shared and TL/pool SQLite connection implementation to the benchmarks as a mere reference.
...
These implementations are dangerous since despite being behind an async
API, they block on I/O and can thus clog up the tokio runtime. Which is
fine if SQLite is all you do but really bad if other, independent work
gets stalled.
2025-04-12 11:56:47 +02:00
458adcf9ec
Add a benchmark scaffold to sqlite.
2025-04-12 10:35:55 +02:00
c0c7681f4a
Restructure crates: make the sqlite execution model more standalone, i.e. move more extension stuff back to extension crate and schema stuff to a new schema crate.
2025-04-08 12:25:10 +02:00
503d7cae9b
Pass connection constructor to prepare for broader concurrency models.
2025-04-08 10:05:56 +02:00
f7cce2dc44
Fix logging: deterministically and separately initialize log and tracing.
2025-04-04 13:23:26 +02:00
31d7dc4158
Use statement cache in more places.
2025-04-03 21:59:23 +02:00
edfbd4c4f6
Manage file deletions through an auxiliary table: "_file_deletions" and triggers.
...
This allows deleting files eagerly w/o a transaction (i.e. pre-op query to get
old values), allows for defered cleanups, and retries.
2025-04-02 00:15:05 +02:00
d9525cd754
Minor: remove log dep from library crates.
2025-03-31 11:26:03 +02:00
3fe9dc8494
Update UUID sqlite extensions, fix example blog and clean it up.
2025-03-24 15:34:01 +01:00
2f07a86030
Update rusqlite: v0.33 -> v0.34.
2025-03-10 14:10:18 +01:00
c63f3f4089
Cleanup: remove legacy query_one_row utility.
2025-03-10 11:09:31 +01:00
68824465a0
Publish updated crates: trailbase(|-sqlite|-extension).
2025-03-09 12:08:46 +01:00
5705b99b6c
Audit and remove remaining unwraps.
2025-03-09 00:04:09 +01:00
527974284f
Update Rust and JS deps.
2025-02-27 00:59:20 +01:00
c06ceee31b
Control expansion of FKs in RecordApi.(read|list) via query parameters.
2025-02-13 13:47:10 +01:00
fd318cf328
Expand/inline foreign records in RecordApi list results.
2025-02-13 13:47:10 +01:00
757f0c370d
Expand/inline foreign records in RecordApi read.
2025-02-13 13:47:03 +01:00
eec17e1343
Update Rust dependencies.
2025-02-12 22:37:55 +01:00
031e2a5032
Update Rust and JS deps.
2025-02-02 12:26:02 +01:00
31e8421451
Updating and cleaning up some Rust deps.
2025-01-26 21:29:44 +01:00
55c26afb0f
Prepare release v0.5.1 and publish new crates.
2025-01-20 15:27:13 +01:00
5144dabc68
Major version update: SQLite 3.46.1 to 3.48.0.
2025-01-19 17:39:30 +01:00
2fb294f17c
Breaking: nest records in output of RecordAPI.list to contain cursor now and potentially more in the future.
...
Also update all the client libraries to accept the new format.
2025-01-19 15:40:09 +01:00
1828ebad5a
Optimization: reduce allocations, cloning and re-encoding.
2025-01-13 22:40:27 +01:00
30f295e6fd
Add "realtime" subscriptions for a specific record, i.e. updates and deletion.
2025-01-13 22:38:14 +01:00
c9199edd22
Prepare trailbase-sqlite for publishing.
2024-12-30 11:52:02 +01:00
e2c43a4c41
Prepare vendored sqlean to be published as a crate.
2024-12-30 11:36:07 +01:00
7e43e3b149
Update all dependencies.
2024-12-30 10:01:57 +01:00
a464103b17
Update Rust dependencies.
2024-12-24 10:16:22 +01:00
312d161048
Reduce number of allocations.
2024-12-17 13:11:01 +01:00
18d1d87710
Add a hooks API to trailbase_sqlite::Connection.
2024-12-16 17:07:17 +01:00
746f0c1108
Make connection constructor synchronous.
2024-12-16 17:00:51 +01:00
4894cc0bb0
Minor: move Row/Rows into a separate file.
2024-12-16 17:00:47 +01:00
Sebastian Jeltsch