mirror/hatchet
1
0
Fork 0
mirror of https://github.com/hatchet-dev/hatchet.git synced 2026-03-17 10:15:49 -05:00
Code Issues Packages Projects Releases Wiki Activity
2,594 Commits 242 Branches 926 Tags
feat/durable-complex-tests
Commit Graph

292 Commits

Author SHA1 Message Date
Gabe Ruttner
6c3a954553 Merge branch 'feat-durable-execution' into feat/durable-complex-tests 2026-03-13 11:20:29 -07:00
matt
6fb50ce004 Feat: Wait wrapper signature improvements (#3241) 
* fix: make memo private for now

* feat: first pass at returning structured data back

* fix: json marshalling, types

* feat: wiring

* fix: sleep for return type

* chore: add comments

* fix: tests for `expr_to_timedelta`

* fix: change examples

* feat: improve asserts

* fix: pass payloads through

* fix: add hacky fallbacks to avoid breaking changes

* feat: add memo now

* feat: add test for memo now

* fix: hack for old engine test

* fix: rm restore button

* fix: one more test

* fix: ts impls

* revert: unwind engine changes (woohoo!)

* fix: payload validation

* fix: ts

* chore: gen

* feat: sleep until

* fix: return wait result on replay

* fix: data get, pydantic warning

* fix: test assert

* fix: ts memo key, make memo private

* chore: rm comment, gen again

* chore: merge and gen
 2026-03-12 12:53:03 -04:00
mrkaye97
c0204fbd8b Merge branch 'main' into feat-durable-execution 2026-03-12 12:24:32 -04:00
github-actions[bot]
18f8fc45f7 chore: regenerate examples (#3242) 
Co-authored-by: GitHub Action <action@github.com>
 2026-03-11 17:34:28 -04:00
Gabe Ruttner
4f37653776 chore: lint 2026-03-10 09:11:51 -04:00
Gabe Ruttner
98212cb2de Merge branch 'main' into feat-durable-execution 2026-03-10 09:01:01 -04:00
Mohammed Nafees
ab6cae4420 Test PgBouncer against our code-level partitioning logic (#3178) 
* test partition drops with pgbouncer

* dedicated DDLPool to bypass pgbouncer

* fix generate

* add env var

* min env var

* update documentation

* no go mod redundancy
 2026-03-10 12:24:07 +01:00
Gabe Ruttner
468de0c584 feat: handle server initiated eviction on workers (#3200) 
* refactor: ts in sync with python

* fix: waits respect abort signal

* chore: gen, lint

* fix: update key extraction logic in durable workflow example

* fix: flake

* fix: racey test

* chore: gen

* feat: send eviction to engine

* feat: handle eviction on workers

* fix: tests and invocation scoped eviction

* fix: cleanup scope

* refactor: cleanup how we do action keys in ts

* chore: gen, lint

* fix: test

* chore: lint

* fix: mock

* fix: cleanup

* chore: lint

* chore: copilot feedback

* fix: reconnect durable listener

* chore: feedback

* fix: bump worker status on wait and reconnect

* refactor: rename eviction notice

* fix: return error on query failure

* chore: lint

* chore: gen

* refactor: remove early completion

* Revert "refactor: remove early completion"

This reverts commit f756ce37b0.

* chore: remove unused param
 2026-03-09 05:16:46 -07:00
github-actions[bot]
af0f04b3f1 chore: regenerate examples (#3203) 
Co-authored-by: GitHub Action <action@github.com>
 2026-03-09 13:35:49 +02:00
dependabot[bot]
b0c99b2028 chore(deps): bump hatchet-sdk in /examples/python/quickstart (#3212) 
Bumps hatchet-sdk from 1.28.0 to 1.28.1.

---
updated-dependencies:
- dependency-name: hatchet-sdk
  dependency-version: 1.28.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-03-09 04:29:21 +00:00
Gabe Ruttner
7e01f7ba94 Merge branch 'main' into feat-durable-execution 2026-03-08 17:50:42 -04:00
Gabe Ruttner
2a98243cd4 chore: update eslint and other packages (#3202) 
* fix: eslint for local dev

* chore: eslint v8 -> 10

* fix: flyby bug

* chore: remove many anys

* chore: update dependencies

* chore: lint
 2026-03-08 13:11:51 -07:00
Gabe Ruttner
fca52697a8 refactor: ts in sync with python (#3199) 
* refactor: ts in sync with python

* fix: waits respect abort signal

* chore: gen, lint

* fix: update key extraction logic in durable workflow example

* fix: flake

* fix: racey test

* chore: gen
 2026-03-07 05:42:42 -08:00
Gabe Ruttner
f177f11290 Feat ts durable execution (#3134) 
* chore: generate

* feat: durable execution in typescript

* fix: cancellation logs

* chore: lint

* chore: generate

* chore: address copilot review

* chore: update docstring

* chore: lint

* refactor: how memos are constructed

* chore: generate ts

* feat: bulk spawn

* feat: compat

* chore: gen

* chore: cleanup and lint

* fix: durable spawn types

* fix: build

* fix: graceful termination

* flyby: namespaces

* fix: graceful term

* chore: lint

* fix: compat

* chore: lint

* fix: legacy race

* fix: last tests

* chore: lint

* fix: bulk replay e2e test

* fix: relax assertion

* fix: bulk
 2026-03-06 10:00:31 -08:00
github-actions[bot]
061787fb48 chore: regenerate examples (#3189) 
Co-authored-by: GitHub Action <action@github.com>
 2026-03-06 12:59:43 -05:00
github-actions[bot]
b5107b052c chore: regenerate examples (#3184) 
Co-authored-by: GitHub Action <action@github.com>
 2026-03-06 09:59:21 -05:00
Mohammed Nafees
00b53529bd Add missing Go SDK examples for worker affinity and manual slot release (#3179) 
* add missing Go SDK examples for affinity and manual slot release

* fix gen
 2026-03-06 01:51:27 +01:00
Gabe Ruttner
12bf7514c5 feat: durable-bulk-spawn (#3173) 
* feat: durable bulk spawn

* feat: bulk db operations

* chore: lint

* chore: generate

* fix: sqlc hack

* cleanup

* chore: lint

* refactor: remove duplicated code

* refactor: simplify input

* ops: rules

* fix: add missing workflows to worker

* chore: add todo

* refactor: single execution path

* comment

* refactor: remove extra class

* chore: lint

* chore: lint

* refactor: batch ack

* fix: handle empty refs

* chore: feedback

* fix: durable output

* chore: lint

* chore: lint

* chore: test fix and gen

* revert: actually use bulk

* refactor: simplify path

* chore: empty commit

Made-with: Cursor
 2026-03-05 16:08:13 -08:00
Gabe Ruttner
217aa2bec2 fix: register the examples on the worker 2026-03-05 08:04:42 -05:00
Gabe Ruttner
d4df0d91b7 feat: add durable complex test cases 
Add comprehensive test suite for durable complex workflows including:
- DAG workflows with mixed ephemeral/durable tasks
- Concurrency control and slot retention
- Execution timeout handling
- On-failure callback tests
- Rate limiting
- Retry behavior

Made-with: Cursor
 2026-03-05 07:45:09 -05:00
dependabot[bot]
318575f4cf chore(deps): bump hatchet-sdk in /examples/python/quickstart (#3171) 
Bumps hatchet-sdk from 1.27.2 to 1.28.0.

---
updated-dependencies:
- dependency-name: hatchet-sdk
  dependency-version: 1.28.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-03-05 04:25:04 +00:00
github-actions[bot]
a78ec7bb06 chore: regenerate examples (#3163) 
Co-authored-by: GitHub Action <action@github.com>
 2026-03-04 17:47:47 -05:00
Gabe Ruttner
5babd09679 docs: chunky overhaul (#3129) 
* improve overall docs structure and things
 2026-03-04 14:33:15 -08:00
mrkaye97
e29459f58a chore: merge main 2026-03-04 13:38:37 -05:00
matt
6c29e48204 Feat: Dynamic worker label assign (#3137) 
* feat: initial wiring work on desired labels

* feat: initial wiring

* chore: gen python

* fix: use the whole desired label thing instead

* fix: more wiring, improve types

* fix: sql type

* fix: len check

* chore: gen python

* fix: initial plural label work

* fix: store the labels properly on the task

* fix: skip cache on override

* fix: bug

* fix: scoping bug whoops

* chore: lint

* fix: send labels back over the api correctly

* feat: python test

* fix: lint

* fix: comment

* fix: override

* fix: namespaces, ugh

* fix: no need for error here

* chore: version

* feat: ruby, go, ts

* feat: versions

* fix: appease the rubocop

* chore: lint

* chore: bundle install

* fix: tests

* chore: lint

* chore: lint more

* fix: ts test

* fix: rb

* chore: gen

* chore: reset gemfile

* chore: reset changelog

* fix: pgroup

* fix: tests, part i

* Revert "chore: reset changelog"

This reverts commit b63bf7d3e5.

* Revert "chore: reset gemfile"

This reverts commit bb848bb6f0.

* fix: go -> golang mapping hack

* fix: go enums

* fix: appease the cop

* fix: namespace

* chore: gen
 2026-03-04 11:03:58 -05:00
mrkaye97
a4f65454f3 Merge branch 'main' into feat-durable-execution 2026-03-03 17:24:23 -05:00
dependabot[bot]
d46c873e2e chore(deps): bump hatchet-sdk in /examples/python/quickstart (#3133) 
Bumps hatchet-sdk from 1.26.2 to 1.27.2.

---
updated-dependencies:
- dependency-name: hatchet-sdk
  dependency-version: 1.27.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-03-02 04:30:47 +00:00
matt
77f769bd66 Feat: Durable Memoization (#3112) 
* feat: initial engine / db work

* chore: gen python

* feat: initial python work

* feat: wiring

* feat: initial test

* fix: scope memo key to the run id (we might not even need this)

* chore: gen, docstring

* fix: mandatory result type

* fix: docs

* fix: log a warning and don't cache

* fix: add test for non-unique keys, fix some bugs

* chore: lint

* chore: naming

* fix: more naming

* fix: comment

* chore: gen

* fix: docs

* fix: naming, ugh

* chore: gen

* chore: gen

* fix: always send event log entry and get ack

* fix: initial rework of memo to only use stream

* fix: start reworking python side

* fix: finish wiring up memo put

* chore: fmt

* chore: comment for monday

* fix: start reworking signature

* fix: docstring

* fix: union type for send event to improve typing a bunch

* fix: memo key

* chore: gen

* fix: rm unused query

* fix: comments
 2026-03-01 16:27:56 -08:00
Gabe Ruttner
e2eb3f05e9 Merge branch 'main' into feat-durable-execution 2026-03-01 19:11:13 -05:00
Gabe Ruttner
5a79618824 chore: deprecate ts v0 (#2960) 
* rip ts v0: keep only TypeScript SDK changes

Cherry-pick TypeScript changes from rip--ts-v0 onto main,
discarding all non-TS changes (Go, Python, proto, SQL, etc.)

Made-with: Cursor

* merge: main into branch

* chore: rm redundant examples

* fix: tests

* fix: tests

* tests: speed em up

* tests: unskip

* tests: add missing p1s

* version: bump to 1.14.0

* chore: generate

* fix: release version

* chore: generate

* chore: rm legacy examples

* tests: timeouts...

* chore: remove unused config

* refactor: minimize dependency on legacy client

* refactor: rename V1Worker to InternalWorker and update references

* chore: lint

* tests: one big worker

* chore: lint

* chore: generate and test fixture tweaks

* tests: debug logs

* tests: stable order

* fix: abortable sleep

* tests: log artifacts on CI failure

* tmp: tmate

* chore: lint

* test: remove tmate

* chore: generate

* tmate

* tmate alwasy

* fix: log level

* fix: casing

* chore: remove tmate

* chore: consistent naming

* fix: test

* test: skip flaky test

* fix: casing

* chore: gen

* chore: lint

* revert: dependency version changes
 2026-03-01 16:06:56 -08:00
mrkaye97
076f5b5f4c Merge branch 'main' into feat-durable-execution 2026-03-01 13:43:51 -05:00
Gabe Ruttner
8796d0ab7d chore: update dependencies for dependabot alerts (#3119) 
* chore: update dependencies for dependabot alerts

* chore: bump main deps instead

* release: bump versions

* chore: bump gem

* chore: update hatchet-sdk to version 0.1.1 in Gemfile.lock
 2026-02-28 06:48:15 -08:00
Gabe Ruttner
daff28dbfe Feat: durable eviction take 2 (#3075) 
* feat: simplified eviction feature

* fix: assign new worker id

* test: shorter sleep

* fix: completion race on same worker

* chore: address todo

* chore: lint

* chore: generate

* fix: n+1 queries

* refactor: WasEvicted bool

* feat: evicted state

* chore: generate

* fix: map status

* fix: update PendingCallback structure to include InvocationCount

* revert: comment

* feat: add support for EVICTED status in waterfall component and metrics display

* fix: implicit eviction

* chore: readable cte

* refactor: queued bool

* refactor: rename eviction_policy

* fix: aio only

* chore: example return type

* fix: map

* feat: eviction error cases

* refactor: change external ID maps to use UUID type

* chore: feedback, cleanup

* tests: additional cases

* chore: generate

* chore: lint

* chore: lint generate

* chore: clean up comments to make matt happy

* refactor: more feedback

* chore: add TODO for worker state reconciliation and clean up comments in eviction policy

* tests: fix

* chore: gen

* test: increase ruby timeout...

* fix: invocation count

* fix: test cases

* fix: stale log entry

* chore: lint

* revert: durable tests to use time.time

* chore: lint
 2026-02-27 09:25:50 -08:00
mrkaye97
6a7f5578c3 Merge branch 'main' into feat-durable-execution 2026-02-27 11:44:17 -05:00
dependabot[bot]
13503f3dae chore(deps): bump hatchet-sdk in /examples/python/quickstart (#3118) 
Bumps hatchet-sdk from 1.26.0 to 1.26.2.

---
updated-dependencies:
- dependency-name: hatchet-sdk
  dependency-version: 1.26.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-02-27 04:24:41 +00:00
mrkaye97
392391b4a7 Merge branch 'main' into feat-durable-execution 2026-02-26 14:21:53 -05:00
Mohammed Nafees
ef2c01a499 Add ctx.WasSkipped helper to the Go SDK (#3094) 
* introduce ctx.WasSkipped just like Py SDK

* fix lint and example
 2026-02-26 17:56:45 +01:00
dependabot[bot]
204cc36103 chore(deps): bump hatchet-sdk in /examples/python/quickstart (#3109) 
Bumps hatchet-sdk from 1.25.2 to 1.26.0.

---
updated-dependencies:
- dependency-name: hatchet-sdk
  dependency-version: 1.26.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-02-26 04:24:16 +00:00
github-actions[bot]
c5afa138c1 chore: regenerate examples (#3098) 
Co-authored-by: GitHub Action <action@github.com>
 2026-02-25 08:55:23 -05:00
mrkaye97
736ecaa3c0 Merge branch 'main' into feat-durable-execution 2026-02-24 13:26:48 -08:00
github-actions[bot]
47f34f2587 chore: regenerate examples (#3084) 
Co-authored-by: GitHub Action <action@github.com>
 2026-02-23 14:48:09 +01:00
Gabe Ruttner
de68e1375a Feat: typescript middleware (#3066) 
* feat: typed middleware

* feat: chaining

* feat: typed global input

* feat: typed global output

* feat: inferred types from middleware

* feat: with chaining

* docs: initial pass

* feat: implicit chaining

* fix: implicit spread

* docs: separate examples

* refactor: rename middleware hooks from `pre`/`post` to `before`/`after` for consistency

* fix: search

* chore: lint

* fix: tests

* Update frontend/docs/pages/home/middleware.mdx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* release: 1.13.0

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-02-23 04:16:00 -08:00
mrkaye97
201a4c74e5 Merge branch 'main' into feat-durable-execution 2026-02-20 13:05:56 -05:00
matt
6f3f6e08ac Feat: Replay as new (or from a node) (#3055) 
* feat: new messages for reset

* chore: gen python

* feat: reset scaffolding

* feat: initial work

* feat: initial e2e wiring of resetting from a specific node

* fix: add branch to pk

* fix: wire up branches

* fix: add branch to awaited entry

* feat: start wiring up reset api

* fix: colname

* fix: add branch id more places

* fix: some bugs

* fix: replay

* fix: replay, simplify

* feat: add parent branch id

* fix: start reworking parent nodes and branches

* fix: parent branch wiring

* fix: start fixing some bugs

* fix: parent branch bug

* fix: advisory lock for locking the log file to prevent concurrent modification

* fix: move claude.md ignore path

* fix: remove eager replays of events

* fix: rm cruft

* fix: cleanup more params and such

* fix: return type

* fix: comment

* fix: comments

* fix: comment

* chore: gen

* chore: gen

* fix: decrease sleep time

* chore: gen again

* fix: add invocation count on event log entries, make it int32, fix toInt

* fix: more wiring

* chore: gen, simplify

* fix: lint

* fix: more zero values, I hate Go

* feat: add `is_durable` to v1_task

* feat: initial work wiring up dispatcher to increment log entry invocation counts

* feat: wire up assigned action

* fix: property

* fix: send is durable through to the engine

* fix: more invoc count wiring

* fix: node resetting

* fix:revert

* fix: import

* chore: gen

* fix: reset -> fork

* fix: rm a bunch of dead code

* fix: api

* fix: repo method

* fix: log file locking using `FOR UPDATE` + atomic compare-and-set update

* fix: move to shared repo

* feat: increment invocation count on the scheduler

* fix: naming

* fix: make test more reliable

* fix: props

* fix: node id reset
 2026-02-20 13:01:46 -05:00
dependabot[bot]
9c13ce9e36 chore(deps): bump hatchet-sdk in /examples/python/quickstart (#3063) 
Bumps hatchet-sdk from 1.25.0 to 1.25.2.

---
updated-dependencies:
- dependency-name: hatchet-sdk
  dependency-version: 1.25.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-02-20 04:24:35 +00:00
mrkaye97
f8e787cd89 Merge branch 'main' into feat-durable-execution 2026-02-19 19:35:40 -05:00
matt
0cce1cfc04 Revert: Cancellation token Python changes (#3061) 
* revert: cancellation token changes

* fix: changelog

* fix: add note on yank
 2026-02-19 12:41:43 -05:00
matt
7e3e3b8fc0 Feat: Non-determinism errors (#3041) 
* fix: retrieve payloads in bulk

* fix: hash -> idempotency key

* feat: initial hashing work

* feat: check idempotency key if entry exists

* fix: panic

* feat: initial work on custom error for non-determinism

* fix: handle nondeterminism error properly

* feat: add error response, pub message to task controller

* chore: lint

* feat: add node id field to error proto

* chore: rm a bunch of unhelpful cancellation logs

* fix: conflict issues

* fix: rm another log

* fix: send node id properly

* fix: improve what we hash

* fix: improve error handling

* fix: python issues

* fix: don't hash or group id

* fix: rm print

* feat: add python test

* fix: add timeout

* fix: improve handling of non determinism error

* fix: propagate node id through

* fix: types, test

* fix: make serializable

* fix: no need to cancel internally anymore

* fix: hide another internal log

* fix: add link to docs

* fix: copilot

* fix: use sha256

* fix: test cleanup

* fix: add error type enum

* fix: handle exceptions on the worker

* fix: clean up a bunch of cursor imports

* fix: cursor docstring formatting

* fix: simplify idempotency key func

* fix: add back cancellation logs

* feat: tests for idempotency keys

* fix: add a couple more for priority and metadata

* chore: gen

* fix: python reconnect

* fix: noisy error

* fix: improve log

* fix: don't run durable listener if no durable tasks are registered

* fix: non-null idempotency keys
 2026-02-18 11:27:02 -05:00
dependabot[bot]
e13721b24a chore(deps): bump hatchet-sdk in /examples/python/quickstart (#3051) 
Bumps hatchet-sdk from 1.24.0 to 1.25.0.

---
updated-dependencies:
- dependency-name: hatchet-sdk
  dependency-version: 1.25.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-02-18 04:24:41 +00:00
github-actions[bot]
6f41c80fca chore: regenerate examples (#3043) 
Co-authored-by: GitHub Action <action@github.com>
 2026-02-17 15:25:45 -05:00