Gabe Ruttner
12bf7514c5
feat: durable-bulk-spawn ( #3173 )
...
* feat: durable bulk spawn
* feat: bulk db operations
* chore: lint
* chore: generate
* fix: sqlc hack
* cleanup
* chore: lint
* refactor: remove duplicated code
* refactor: simplify input
* ops: rules
* fix: add missing workflows to worker
* chore: add todo
* refactor: single execution path
* comment
* refactor: remove extra class
* chore: lint
* chore: lint
* refactor: batch ack
* fix: handle empty refs
* chore: feedback
* fix: durable output
* chore: lint
* chore: lint
* chore: test fix and gen
* revert: actually use bulk
* refactor: simplify path
* chore: empty commit
Made-with: Cursor
2026-03-05 16:08:13 -08:00
matt
77f769bd66
Feat: Durable Memoization ( #3112 )
...
* feat: initial engine / db work
* chore: gen python
* feat: initial python work
* feat: wiring
* feat: initial test
* fix: scope memo key to the run id (we might not even need this)
* chore: gen, docstring
* fix: mandatory result type
* fix: docs
* fix: log a warning and don't cache
* fix: add test for non-unique keys, fix some bugs
* chore: lint
* chore: naming
* fix: more naming
* fix: comment
* chore: gen
* fix: docs
* fix: naming, ugh
* chore: gen
* chore: gen
* fix: always send event log entry and get ack
* fix: initial rework of memo to only use stream
* fix: start reworking python side
* fix: finish wiring up memo put
* chore: fmt
* chore: comment for monday
* fix: start reworking signature
* fix: docstring
* fix: union type for send event to improve typing a bunch
* fix: memo key
* chore: gen
* fix: rm unused query
* fix: comments
2026-03-01 16:27:56 -08:00
Gabe Ruttner
daff28dbfe
Feat: durable eviction take 2 ( #3075 )
...
* feat: simplified eviction feature
* fix: assign new worker id
* test: shorter sleep
* fix: completion race on same worker
* chore: address todo
* chore: lint
* chore: generate
* fix: n+1 queries
* refactor: WasEvicted bool
* feat: evicted state
* chore: generate
* fix: map status
* fix: update PendingCallback structure to include InvocationCount
* revert: comment
* feat: add support for EVICTED status in waterfall component and metrics display
* fix: implicit eviction
* chore: readable cte
* refactor: queued bool
* refactor: rename eviction_policy
* fix: aio only
* chore: example return type
* fix: map
* feat: eviction error cases
* refactor: change external ID maps to use UUID type
* chore: feedback, cleanup
* tests: additional cases
* chore: generate
* chore: lint
* chore: lint generate
* chore: clean up comments to make matt happy
* refactor: more feedback
* chore: add TODO for worker state reconciliation and clean up comments in eviction policy
* tests: fix
* chore: gen
* test: increase ruby timeout...
* fix: invocation count
* fix: test cases
* fix: stale log entry
* chore: lint
* revert: durable tests to use time.time
* chore: lint
2026-02-27 09:25:50 -08:00
mrkaye97
392391b4a7
Merge branch 'main' into feat-durable-execution
2026-02-26 14:21:53 -05:00
github-actions[bot]
c5afa138c1
chore: regenerate examples ( #3098 )
...
Co-authored-by: GitHub Action <action@github.com >
2026-02-25 08:55:23 -05:00
matt
6f3f6e08ac
Feat: Replay as new (or from a node) ( #3055 )
...
* feat: new messages for reset
* chore: gen python
* feat: reset scaffolding
* feat: initial work
* feat: initial e2e wiring of resetting from a specific node
* fix: add branch to pk
* fix: wire up branches
* fix: add branch to awaited entry
* feat: start wiring up reset api
* fix: colname
* fix: add branch id more places
* fix: some bugs
* fix: replay
* fix: replay, simplify
* feat: add parent branch id
* fix: start reworking parent nodes and branches
* fix: parent branch wiring
* fix: start fixing some bugs
* fix: parent branch bug
* fix: advisory lock for locking the log file to prevent concurrent modification
* fix: move claude.md ignore path
* fix: remove eager replays of events
* fix: rm cruft
* fix: cleanup more params and such
* fix: return type
* fix: comment
* fix: comments
* fix: comment
* chore: gen
* chore: gen
* fix: decrease sleep time
* chore: gen again
* fix: add invocation count on event log entries, make it int32, fix toInt
* fix: more wiring
* chore: gen, simplify
* fix: lint
* fix: more zero values, I hate Go
* feat: add `is_durable` to v1_task
* feat: initial work wiring up dispatcher to increment log entry invocation counts
* feat: wire up assigned action
* fix: property
* fix: send is durable through to the engine
* fix: more invoc count wiring
* fix: node resetting
* fix:revert
* fix: import
* chore: gen
* fix: reset -> fork
* fix: rm a bunch of dead code
* fix: api
* fix: repo method
* fix: log file locking using `FOR UPDATE` + atomic compare-and-set update
* fix: move to shared repo
* feat: increment invocation count on the scheduler
* fix: naming
* fix: make test more reliable
* fix: props
* fix: node id reset
2026-02-20 13:01:46 -05:00
matt
7e3e3b8fc0
Feat: Non-determinism errors ( #3041 )
...
* fix: retrieve payloads in bulk
* fix: hash -> idempotency key
* feat: initial hashing work
* feat: check idempotency key if entry exists
* fix: panic
* feat: initial work on custom error for non-determinism
* fix: handle nondeterminism error properly
* feat: add error response, pub message to task controller
* chore: lint
* feat: add node id field to error proto
* chore: rm a bunch of unhelpful cancellation logs
* fix: conflict issues
* fix: rm another log
* fix: send node id properly
* fix: improve what we hash
* fix: improve error handling
* fix: python issues
* fix: don't hash or group id
* fix: rm print
* feat: add python test
* fix: add timeout
* fix: improve handling of non determinism error
* fix: propagate node id through
* fix: types, test
* fix: make serializable
* fix: no need to cancel internally anymore
* fix: hide another internal log
* fix: add link to docs
* fix: copilot
* fix: use sha256
* fix: test cleanup
* fix: add error type enum
* fix: handle exceptions on the worker
* fix: clean up a bunch of cursor imports
* fix: cursor docstring formatting
* fix: simplify idempotency key func
* fix: add back cancellation logs
* feat: tests for idempotency keys
* fix: add a couple more for priority and metadata
* chore: gen
* fix: python reconnect
* fix: noisy error
* fix: improve log
* fix: don't run durable listener if no durable tasks are registered
* fix: non-null idempotency keys
2026-02-18 11:27:02 -05:00
matt
05399ebf39
Feat: Durable event log wiring ( #2956 )
...
* feat: initial protos
* chore: lint
* fix: work on improving naming
* chore: rename session id to invocation count
* feat: scaffold implementation of durabletask rpc
* fix: one more session rename
* feat: initial work on the server scaffolding
* chore: gen protos for python
* feat: initial durable task client
* feat: initial durable context work for python
* fix: pass client through to runner
* fix: clean up type checking errors
* fix: cruft
* feat: initial work wiring up durable events
* fix: get -> getorcreate
* feat: query + wiring for updating latest node id
* fix: simplify, bump latest node ids in the same query
* chore: note
* feat: wire up sleeps with internal signal matches
* chore: gen
* fix: callback data writes
* feat: cache previous events
* fix: wire up external id writes
* feat: got sleeps sorta working!
* fix: tenant and external id wiring
* chore: comments
* fix: clean up some types a bit
* feat: add run triggering params to proto to allow for spawning children
* feat: first pass at child spawning
* feat: start wiring up child spawning
* fix: use `triggerWriter` for spawn
* feat: update trigger proto def
* chore: regen python
* feat: start wiring up spawning correctly with all opts
* refactor: share trigger code
* chore: remove log lines, lint
* fix: add triggered run external id
* feat: start wiring up child key storage better
* chore: gen again
* fix: gen, colname
* fix: trigger opts panicking
* hack: get things working for now
* feat: shared rpc message
* chore: fix imports
* feat: add tenant id to tables
* fix: improve ingest logic
* refactor: shared trigger opt type
* fix: send tenant id through everywhere
* chore: fix log file insert on conflict
* fix: repo
* fix: generate external id upstream
* feat: add columns to the match
* feat: first pass at durable waits on the controllers instead of the dispatcher
* fix: types
* feat: wire up callbacks
* fix: invoc counts
* fix: typing, lint
* driveby: more constants for message ids
* refactor: struct for callback keys everywhere
* fix: bugs, passing tests
* fix: return errnorows
* fix: schema
* fix: remove current callback flow
* feat: new message types
* fix: remove key from callback model
* fix: rm unused queries
* refactor: start reworking flow
* fix: start working on feedback
* fix: query
* fix: wire up external ids
* revert: drive by
* refactor: rm extra interface
* chore: move listener, lint
* refactor: remove old listener, rename
* refactor: consolidate migrations
* fix: immediately send already-satisfied callbacks
* fix: union
* chore: rm unused queries
* fix: check if entry already exists before re-spawning / signaling
* fix: node id incrementation
* fix: rm json dump
* fix: don't pass node id
* fix: store latest invocation, update query
* fix: upsert logic
* Revert "fix: upsert logic"
This reverts commit cf7c609c1d26e6c82797
2026-02-16 12:23:58 -05:00
github-actions[bot]
c922830c77
chore: regenerate examples ( #3018 )
...
Co-authored-by: GitHub Action <action@github.com >
2026-02-13 11:24:46 -05:00
github-actions[bot]
d4867568ff
chore: regenerate examples ( #2939 )
...
Co-authored-by: GitHub Action <action@github.com >
2026-02-04 12:46:26 -05:00
github-actions[bot]
7cbd49056a
chore: regenerate examples ( #2820 )
...
Co-authored-by: GitHub Action <action@github.com >
2026-01-19 07:44:26 -05:00
github-actions[bot]
ab70b77b0c
chore: regenerate examples ( #2764 )
...
Co-authored-by: GitHub Action <action@github.com >
2026-01-08 12:49:07 -05:00
github-actions[bot]
1301544a50
chore: regenerate examples ( #2717 )
...
Co-authored-by: GitHub Action <action@github.com >
2025-12-26 09:38:13 -07:00
github-actions[bot]
f12c3e96ce
chore: regenerate examples ( #2477 )
...
Co-authored-by: GitHub Action <action@github.com >
2025-11-01 00:28:24 +01:00
github-actions[bot]
29819ebf08
chore: regenerate examples ( #2318 )
...
Co-authored-by: GitHub Action <action@github.com >
2025-09-18 20:43:30 -04:00
matt
57bb24aef1
Fix: Auto-generate docs snippets and examples ( #2139 )
...
* fix: gitignore all the generated stuff
* debug: try fixing build
* debug: build error part ii
* debug: move more deps out of dev
* fix: lock
* debug: lockfile
* fix: make dir
* fix: ci
* fix: dir
* debug: sed
* fix: sed
* debug: allow skipping
* Revert "debug: allow skipping"
This reverts commit 88e0ff870daction@github.com >
2025-08-14 18:17:29 -05:00
matt
c8d5144ed4
[Python] Feat: Dependency Injection, Improved error handling ( #2067 )
...
* feat: first pass at dependency injection
* feat: add DI example + tests
* feat: finish up tests
* feat: docs
* chore: gen
* chore: lint
* chore: changelog + ver
* fix: split up paragraphs
* refactor: improve impl
* chore: gen
* feat: inject input + ctx into deps
* chore: gen
* [Python] Feat: More use of `logger.exception` (#2069 )
* feat: add more instances of `logger.exception`
* chore: ver
* chore: changelog
* fix: one more error log
* chore: gen
* chore: gen
* chore: lint
* fix: improve shutdown
* chore: changelog
* unwind: exit handler
* feat: task run error
* feat: improve error serde with more context
* chore: changelog
* fix: changelog
* chore: gen
* fix: rm celpy + lark dep, casing issues
* chore: changelog
* fix: respect log levels over the API
* fix: changelog
* refactor: rename log forwarder
* fix: circular import
2025-08-11 23:10:44 -04:00
matt
d6f8be2c0f
Feat: OLAP Table for CEL Eval Failures ( #2012 )
...
* feat: add table, wire up partitioning
* feat: wire failures into the OLAP db from rabbit
* feat: bubble failures up to controller
* fix: naming
* fix: hack around enum type
* fix: typo
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: typos
* fix: migration name
* feat: log debug failure
* feat: pub message from debug endpoint to log failure
* fix: error handling
* fix: use ingestor
* fix: olap suffix
* fix: pass source through
* fix: dont log ingest failure
* fix: rm debug as enum opt
* chore: gen
* Feat: Webhooks (#1978 )
* feat: migration + go gen
* feat: non unique source name
* feat: api types
* fix: rm cruft
* feat: initial api for webhooks
* feat: handle encryption of incoming keys
* fix: nil pointer errors
* fix: import
* feat: add endpoint for incoming webhooks
* fix: naming
* feat: start wiring up basic auth
* feat: wire up cel event parsing
* feat: implement authentication
* fix: hack for plain text content
* feat: add source to enum
* feat: add source name enum
* feat: db source name enum fix
* fix: use source name enums
* feat: nest sources
* feat: first pass at stripe
* fix: clean up source name passing
* fix: use unique name for webhook
* feat: populator test
* fix: null values
* fix: ordering
* fix: rm unnecessary index
* fix: validation
* feat: validation on create
* fix: lint
* fix: naming
* feat: wire triggering webhook name through to events table
* feat: cleanup + python gen + e2e test for basic auth
* feat: query to insert webhook validation errors
* refactor: auth handler
* fix: naming
* refactor: validation errors, part II
* feat: wire up writes through olap
* fix: linting, fallthrough case
* fix: validation
* feat: tests for failure cases for basic auth
* feat: expand tests
* fix: correctly return 404 out of task getter
* chore: generated stuff
* fix: rm cruft
* fix: longer sleep
* debug: print name + events to logs
* feat: limit to N
* feat: add limit env var
* debug: ci test
* fix: apply namespaces to keys
* fix: namespacing, part ii
* fix: sdk config
* fix: handle prefixing
* feat: handle partitioning logic
* chore: gen
* feat: add webhook limit
* feat: wire up limits
* fix: gen
* fix: reverse order of generic fallthrough
* fix: comment for potential unexpected behavior
* fix: add check constraints, improve error handling
* chore: gen
* chore: gen
* fix: improve naming
* feat: scaffold webhooks page
* feat: sidebar
* feat: first pass at page
* feat: improve feedback on UI
* feat: initial work on create modal
* feat: change default to basic
* fix: openapi spec discriminated union
* fix: go side
* feat: start wiring up placeholders for stripe and github
* feat: pre-populated fields for Stripe + Github
* feat: add name section
* feat: copy improvements, show URL
* feat: UI cleanup
* fix: check if tenant populator errors
* feat: add comments
* chore: gen again
* fix: default name
* fix: styling
* fix: improve stripe header processing
* feat: docs, part 1
* fix: lint
* fix: migration order
* feat: implement rate limit per-webhook
* feat: comment
* feat: clean up docs
* chore: gen
* fix: migration versions
* fix: olap naming
* fix: partitions
* chore: gen
* feat: store webhook cel eval failures properly
* fix: pk order
* fix: auth tweaks, move fetches out of populator
* fix: pgtype.Text instead of string pointer
* chore: gen
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-07-30 13:27:38 -04:00
matt
cca0999eea
[Python]: Fixing logging bugs, fixing duped sleep key bug ( #2040 )
...
* feat: add flag to disable log capture
* fix: sleep bug with duped key
* fix: allow formatters to be passed through
* feat: support filters too
* fix: cruft
* chore: gen
* feat: changelog
* fix: lint
* [Python] Fix: Don't retry gRPC requests on 4xx (#2024 )
* fix: dont retry on 4xx
* chore: ver
* fix: sleep conditions with index
* fix: bug in sleep conditions
* chore: gen
2025-07-23 15:15:31 -04:00
Matt Kaye
f1f276f6dc
Feat: Python task unit tests ( #1990 )
...
* feat: add mock run methods for tasks
* feat: docs
* feat: first pass at unit tests
* cleanup: split out tests
* feat: pass lifespan through
* fix: rm comment
* drive by: retry on 404 to help with races
* chore: changelog
* chore: ver
* feat: improve logging everywhere
* chore: changelog
* fix: rm print cruft
* feat: print statement linter
* feat: helper for getting result of a standalone
* feat: docs for mock run
* feat: add task run getter
* feat: propagate additional metadata properly
* chore: gen
* fix: date
* chore: gen
* feat: return exceptions
* chore: gen
* chore: changelog
* feat: tests + gen again
* fix: rm print cruft
2025-07-17 13:54:40 -04:00
Matt Kaye
7679732b15
Fix: Skipping conditions with multiple parents ( #1948 )
...
* fix: skipping bug
* fix: move `waits` -> `conditions`
* fix: refs
* chore: ver
* feat: add skipped task to test
* feat: start implementing or groups in wait for
* feat: test of or groups on durable context
* fix: lint
* chore: gen
* fix: lint
* fix: branching hell
2025-07-03 16:50:57 -04:00
Matt Kaye
a73e34cd92
Feat: Batched replays ( #1860 )
...
* feat: batched replays
* fix: add some comments
* feat: e2e test for bulk replays
* chore: gen
* fix: improving tests a bit
* fix: copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* chore: gen
* fix: tag
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-06-13 13:47:50 -04:00
Matt Kaye
f9d3d508ca
Blocked event loop blog post ( #1775 )
...
* feat: snippets
* feat: first couple paragraphs
* feat: flesh out hatchet examples
* fix: one more
* feat: log view
* fix: cleanup
* feat: another section
* fix: fmt
* feat: debugging section
* fix: proofread
* fix: lint
* chore: gen
* fix: copilot
* fix: copilot
* feat: add blog post link
* fix: feedback
* chore: sdk ver
* chore: gen
* fix: ugh
2025-05-27 11:07:34 -07:00
Gabe Ruttner
3831fa3ede
feat(ts): improved custom logger ( #1652 )
...
* wip improved signatures
* feat: retry count and improved logger
* task pre
* update example
2025-05-12 09:49:18 -07:00
Gabe Ruttner
8e80faf2d6
Fe overhaul docs ( #1640 )
...
* api changes
* doc changes
* move docs
* generated
* generate
* pkg
* backmerge main
* revert to main
* revert main
* race?
* remove go tests
2025-04-30 14:10:09 -07:00
