Gabe Ruttner
fca52697a8
refactor: ts in sync with python ( #3199 )
...
* refactor: ts in sync with python
* fix: waits respect abort signal
* chore: gen, lint
* fix: update key extraction logic in durable workflow example
* fix: flake
* fix: racey test
* chore: gen
2026-03-07 05:42:42 -08:00
Gabe Ruttner
f177f11290
Feat ts durable execution ( #3134 )
...
* chore: generate
* feat: durable execution in typescript
* fix: cancellation logs
* chore: lint
* chore: generate
* chore: address copilot review
* chore: update docstring
* chore: lint
* refactor: how memos are constructed
* chore: generate ts
* feat: bulk spawn
* feat: compat
* chore: gen
* chore: cleanup and lint
* fix: durable spawn types
* fix: build
* fix: graceful termination
* flyby: namespaces
* fix: graceful term
* chore: lint
* fix: compat
* chore: lint
* fix: legacy race
* fix: last tests
* chore: lint
* fix: bulk replay e2e test
* fix: relax assertion
* fix: bulk
2026-03-06 10:00:31 -08:00
Gabe Ruttner
12bf7514c5
feat: durable-bulk-spawn ( #3173 )
...
* feat: durable bulk spawn
* feat: bulk db operations
* chore: lint
* chore: generate
* fix: sqlc hack
* cleanup
* chore: lint
* refactor: remove duplicated code
* refactor: simplify input
* ops: rules
* fix: add missing workflows to worker
* chore: add todo
* refactor: single execution path
* comment
* refactor: remove extra class
* chore: lint
* chore: lint
* refactor: batch ack
* fix: handle empty refs
* chore: feedback
* fix: durable output
* chore: lint
* chore: lint
* chore: test fix and gen
* revert: actually use bulk
* refactor: simplify path
* chore: empty commit
Made-with: Cursor
2026-03-05 16:08:13 -08:00
matt
77f769bd66
Feat: Durable Memoization ( #3112 )
...
* feat: initial engine / db work
* chore: gen python
* feat: initial python work
* feat: wiring
* feat: initial test
* fix: scope memo key to the run id (we might not even need this)
* chore: gen, docstring
* fix: mandatory result type
* fix: docs
* fix: log a warning and don't cache
* fix: add test for non-unique keys, fix some bugs
* chore: lint
* chore: naming
* fix: more naming
* fix: comment
* chore: gen
* fix: docs
* fix: naming, ugh
* chore: gen
* chore: gen
* fix: always send event log entry and get ack
* fix: initial rework of memo to only use stream
* fix: start reworking python side
* fix: finish wiring up memo put
* chore: fmt
* chore: comment for monday
* fix: start reworking signature
* fix: docstring
* fix: union type for send event to improve typing a bunch
* fix: memo key
* chore: gen
* fix: rm unused query
* fix: comments
2026-03-01 16:27:56 -08:00
Gabe Ruttner
daff28dbfe
Feat: durable eviction take 2 ( #3075 )
...
* feat: simplified eviction feature
* fix: assign new worker id
* test: shorter sleep
* fix: completion race on same worker
* chore: address todo
* chore: lint
* chore: generate
* fix: n+1 queries
* refactor: WasEvicted bool
* feat: evicted state
* chore: generate
* fix: map status
* fix: update PendingCallback structure to include InvocationCount
* revert: comment
* feat: add support for EVICTED status in waterfall component and metrics display
* fix: implicit eviction
* chore: readable cte
* refactor: queued bool
* refactor: rename eviction_policy
* fix: aio only
* chore: example return type
* fix: map
* feat: eviction error cases
* refactor: change external ID maps to use UUID type
* chore: feedback, cleanup
* tests: additional cases
* chore: generate
* chore: lint
* chore: lint generate
* chore: clean up comments to make matt happy
* refactor: more feedback
* chore: add TODO for worker state reconciliation and clean up comments in eviction policy
* tests: fix
* chore: gen
* test: increase ruby timeout...
* fix: invocation count
* fix: test cases
* fix: stale log entry
* chore: lint
* revert: durable tests to use time.time
* chore: lint
2026-02-27 09:25:50 -08:00
matt
6f3f6e08ac
Feat: Replay as new (or from a node) ( #3055 )
...
* feat: new messages for reset
* chore: gen python
* feat: reset scaffolding
* feat: initial work
* feat: initial e2e wiring of resetting from a specific node
* fix: add branch to pk
* fix: wire up branches
* fix: add branch to awaited entry
* feat: start wiring up reset api
* fix: colname
* fix: add branch id more places
* fix: some bugs
* fix: replay
* fix: replay, simplify
* feat: add parent branch id
* fix: start reworking parent nodes and branches
* fix: parent branch wiring
* fix: start fixing some bugs
* fix: parent branch bug
* fix: advisory lock for locking the log file to prevent concurrent modification
* fix: move claude.md ignore path
* fix: remove eager replays of events
* fix: rm cruft
* fix: cleanup more params and such
* fix: return type
* fix: comment
* fix: comments
* fix: comment
* chore: gen
* chore: gen
* fix: decrease sleep time
* chore: gen again
* fix: add invocation count on event log entries, make it int32, fix toInt
* fix: more wiring
* chore: gen, simplify
* fix: lint
* fix: more zero values, I hate Go
* feat: add `is_durable` to v1_task
* feat: initial work wiring up dispatcher to increment log entry invocation counts
* feat: wire up assigned action
* fix: property
* fix: send is durable through to the engine
* fix: more invoc count wiring
* fix: node resetting
* fix:revert
* fix: import
* chore: gen
* fix: reset -> fork
* fix: rm a bunch of dead code
* fix: api
* fix: repo method
* fix: log file locking using `FOR UPDATE` + atomic compare-and-set update
* fix: move to shared repo
* feat: increment invocation count on the scheduler
* fix: naming
* fix: make test more reliable
* fix: props
* fix: node id reset
2026-02-20 13:01:46 -05:00
matt
7e3e3b8fc0
Feat: Non-determinism errors ( #3041 )
...
* fix: retrieve payloads in bulk
* fix: hash -> idempotency key
* feat: initial hashing work
* feat: check idempotency key if entry exists
* fix: panic
* feat: initial work on custom error for non-determinism
* fix: handle nondeterminism error properly
* feat: add error response, pub message to task controller
* chore: lint
* feat: add node id field to error proto
* chore: rm a bunch of unhelpful cancellation logs
* fix: conflict issues
* fix: rm another log
* fix: send node id properly
* fix: improve what we hash
* fix: improve error handling
* fix: python issues
* fix: don't hash or group id
* fix: rm print
* feat: add python test
* fix: add timeout
* fix: improve handling of non determinism error
* fix: propagate node id through
* fix: types, test
* fix: make serializable
* fix: no need to cancel internally anymore
* fix: hide another internal log
* fix: add link to docs
* fix: copilot
* fix: use sha256
* fix: test cleanup
* fix: add error type enum
* fix: handle exceptions on the worker
* fix: clean up a bunch of cursor imports
* fix: cursor docstring formatting
* fix: simplify idempotency key func
* fix: add back cancellation logs
* feat: tests for idempotency keys
* fix: add a couple more for priority and metadata
* chore: gen
* fix: python reconnect
* fix: noisy error
* fix: improve log
* fix: don't run durable listener if no durable tasks are registered
* fix: non-null idempotency keys
2026-02-18 11:27:02 -05:00
mrkaye97
7ad5bfdf89
Merge branch 'main' into feat-durable-execution
2026-02-17 08:45:41 -05:00
Gabe Ruttner
2fdc47a6af
feat: multiple slot types ( #2927 )
...
* feat: adds support for multiple slot types, primarily motivated by durable slots
---------
Co-authored-by: mrkaye97 <mrkaye97@gmail.com >
2026-02-17 05:43:47 -08:00
matt
05399ebf39
Feat: Durable event log wiring ( #2956 )
...
* feat: initial protos
* chore: lint
* fix: work on improving naming
* chore: rename session id to invocation count
* feat: scaffold implementation of durabletask rpc
* fix: one more session rename
* feat: initial work on the server scaffolding
* chore: gen protos for python
* feat: initial durable task client
* feat: initial durable context work for python
* fix: pass client through to runner
* fix: clean up type checking errors
* fix: cruft
* feat: initial work wiring up durable events
* fix: get -> getorcreate
* feat: query + wiring for updating latest node id
* fix: simplify, bump latest node ids in the same query
* chore: note
* feat: wire up sleeps with internal signal matches
* chore: gen
* fix: callback data writes
* feat: cache previous events
* fix: wire up external id writes
* feat: got sleeps sorta working!
* fix: tenant and external id wiring
* chore: comments
* fix: clean up some types a bit
* feat: add run triggering params to proto to allow for spawning children
* feat: first pass at child spawning
* feat: start wiring up child spawning
* fix: use `triggerWriter` for spawn
* feat: update trigger proto def
* chore: regen python
* feat: start wiring up spawning correctly with all opts
* refactor: share trigger code
* chore: remove log lines, lint
* fix: add triggered run external id
* feat: start wiring up child key storage better
* chore: gen again
* fix: gen, colname
* fix: trigger opts panicking
* hack: get things working for now
* feat: shared rpc message
* chore: fix imports
* feat: add tenant id to tables
* fix: improve ingest logic
* refactor: shared trigger opt type
* fix: send tenant id through everywhere
* chore: fix log file insert on conflict
* fix: repo
* fix: generate external id upstream
* feat: add columns to the match
* feat: first pass at durable waits on the controllers instead of the dispatcher
* fix: types
* feat: wire up callbacks
* fix: invoc counts
* fix: typing, lint
* driveby: more constants for message ids
* refactor: struct for callback keys everywhere
* fix: bugs, passing tests
* fix: return errnorows
* fix: schema
* fix: remove current callback flow
* feat: new message types
* fix: remove key from callback model
* fix: rm unused queries
* refactor: start reworking flow
* fix: start working on feedback
* fix: query
* fix: wire up external ids
* revert: drive by
* refactor: rm extra interface
* chore: move listener, lint
* refactor: remove old listener, rename
* refactor: consolidate migrations
* fix: immediately send already-satisfied callbacks
* fix: union
* chore: rm unused queries
* fix: check if entry already exists before re-spawning / signaling
* fix: node id incrementation
* fix: rm json dump
* fix: don't pass node id
* fix: store latest invocation, update query
* fix: upsert logic
* Revert "fix: upsert logic"
This reverts commit cf7c609c1d26e6c82797
2026-02-16 12:23:58 -05:00
matt
c8d5144ed4
[Python] Feat: Dependency Injection, Improved error handling ( #2067 )
...
* feat: first pass at dependency injection
* feat: add DI example + tests
* feat: finish up tests
* feat: docs
* chore: gen
* chore: lint
* chore: changelog + ver
* fix: split up paragraphs
* refactor: improve impl
* chore: gen
* feat: inject input + ctx into deps
* chore: gen
* [Python] Feat: More use of `logger.exception` (#2069 )
* feat: add more instances of `logger.exception`
* chore: ver
* chore: changelog
* fix: one more error log
* chore: gen
* chore: gen
* chore: lint
* fix: improve shutdown
* chore: changelog
* unwind: exit handler
* feat: task run error
* feat: improve error serde with more context
* chore: changelog
* fix: changelog
* chore: gen
* fix: rm celpy + lark dep, casing issues
* chore: changelog
* fix: respect log levels over the API
* fix: changelog
* refactor: rename log forwarder
* fix: circular import
2025-08-11 23:10:44 -04:00
matt
cca0999eea
[Python]: Fixing logging bugs, fixing duped sleep key bug ( #2040 )
...
* feat: add flag to disable log capture
* fix: sleep bug with duped key
* fix: allow formatters to be passed through
* feat: support filters too
* fix: cruft
* chore: gen
* feat: changelog
* fix: lint
* [Python] Fix: Don't retry gRPC requests on 4xx (#2024 )
* fix: dont retry on 4xx
* chore: ver
* fix: sleep conditions with index
* fix: bug in sleep conditions
* chore: gen
2025-07-23 15:15:31 -04:00
Matt Kaye
7679732b15
Fix: Skipping conditions with multiple parents ( #1948 )
...
* fix: skipping bug
* fix: move `waits` -> `conditions`
* fix: refs
* chore: ver
* feat: add skipped task to test
* feat: start implementing or groups in wait for
* feat: test of or groups on durable context
* fix: lint
* chore: gen
* fix: lint
* fix: branching hell
2025-07-03 16:50:57 -04:00
Matt Kaye
2f33dd4dbd
Feat: Misc. Python improvements + Streaming Improvements ( #1846 )
...
* fix: contextvars explicit copy
* feat: fix a ton of ruff errors
* fix: couple more ruff rules
* fix: ignore unhelpful rule
* fix: exception group in newer Python versions for improved handling
* fix: workflow docs
* feat: context docs
* feat: simple task counter
* feat: config for setting max tasks
* feat: graceful exit once worker exceeds max tasks
* fix: optional
* fix: docs
* fix: events docs + gen
* chore: gen
* fix: one more dangling task
* feat: add xdist in ci
* fix: CI
* fix: xdist fails me once again
* fix: fix + extend some tests
* fix: test cleanup
* fix: exception group
* fix: ugh
* feat: changelog
* Add Ruff linter callout to post
* refactor: clean up runner error handling
* feat: improved errors
* fix: lint
* feat: hacky serde impl
* fix: improve serde + formatting
* fix: logging
* fix: lint
* fix: unexpected errors
* fix: naming, ruff
* fix: rm cruft
* Fix: Attempt to fix namespacing issue in event waits (#1885 )
* feat: add xdist in ci
* fix: attempt to fix namespacing issue in event waits
* fix: namespaced worker names
* fix: applied namespace to the wrong thing
* fix: rm hack
* drive by: namespacing improvement
* fix: delay
* fix: changelog
* fix: initial log work
* fix: more logging work
* fix: rm print cruft
* feat: use a queue to send logs
* fix: sentinel value to stop the loop
* fix: use the log sender everywhere
* fix: make streaming blocking, remove more thread pools
* feat: changelog
* fix: linting issues
* fix: broken test
* chore: bunch more generated stuff
* fix: changelog
* fix: one more
* fix: mypy
* chore: gen
* Feat: Streaming Improvements (#1886 )
* Fix: Filter list improvements (#1899 )
* fix: uuid validation
* fix: improve filter filtering
* fix: inner join
* fix: bug in workflow cached prop
* chore: bump
* fix: lint
* chore: changelog
* fix: separate filter queries
* feat: improve filter filtering
* fix: queries and the like
* feat: add xdist in ci
* feat: streaming test + gen
* feat: add index to stream event
* fix: rm langfuse dep
* fix: lf
* chore: gen
* feat: impl index for stream on context
* feat: tweak protos
* feat: extend test
* feat: send event index through queue
* feat: first pass + debug logging
* debug: fixes
* debug: more possible issues
* feat: generate new stream event protos
* feat: first pass at using an alternate exchange for replaying incoming stream events
* fix: exchange create timing
* fix: rm unused protos
* chore: gen
* feat: python cleanup
* fix: revert rabbit changes
* fix: unwind a bunch of cruft
* fix: optional index
* chore: gen python
* fix: event index nil handling
* feat: improve test
* fix: stream impl in sdk
* fix: make test faster
* chore: gen a ton more stuff
* fix: test
* fix: sorting helper
* fix: bug
* fix: one more ordering bug
* feat: add some tests for buffering logic
* feat: hangup test
* feat: test no buffering if no index sent
* fix: regular mutex
* fix: pr feedback
* fix: conflicts
2025-06-25 10:11:01 -04:00
Gabe Ruttner
8e80faf2d6
Fe overhaul docs ( #1640 )
...
* api changes
* doc changes
* move docs
* generated
* generate
* pkg
* backmerge main
* revert to main
* revert main
* race?
* remove go tests
2025-04-30 14:10:09 -07:00
