Commit Graph

570 Commits

Author SHA1 Message Date
Gabe Ruttner
583bb6f4ba Feat--unify-running-and-evicted (#3248)
* feat: unify running and evicted counts with filter

* chore: gen

* chore: gen lint

* fix: opsies...

* chore: gen

* chore: feedback
2026-03-13 05:57:07 -07:00
matt
6fb50ce004 Feat: Wait wrapper signature improvements (#3241)
* fix: make memo private for now

* feat: first pass at returning structured data back

* fix: json marshalling, types

* feat: wiring

* fix: sleep for return type

* chore: add comments

* fix: tests for `expr_to_timedelta`

* fix: change examples

* feat: improve asserts

* fix: pass payloads through

* fix: add hacky fallbacks to avoid breaking changes

* feat: add memo now

* feat: add test for memo now

* fix: hack for old engine test

* fix: rm restore button

* fix: one more test

* fix: ts impls

* revert: unwind engine changes (woohoo!)

* fix: payload validation

* fix: ts

* chore: gen

* feat: sleep until

* fix: return wait result on replay

* fix: data get, pydantic warning

* fix: test assert

* fix: ts memo key, make memo private

* chore: rm comment, gen again

* chore: merge and gen
2026-03-12 12:53:03 -04:00
mrkaye97
c0204fbd8b Merge branch 'main' into feat-durable-execution 2026-03-12 12:24:32 -04:00
Mohammed Nafees
9e002a6c4c Pool gzip writers to reduce RabbitMQ message compression allocations (#3103)
* use sync.Pool for gzip writers to avoid memory hogging

* comments

* fix deadlock failure

* copilot suggestion

* fix formatting

* use asserts

* PR comments
2026-03-12 13:09:11 +01:00
matt
b942092ba5 Feat: env var for stream event buffer timeout (#3223)
* feat: env var for stream event buffer timeout

* chore: gen
2026-03-11 11:41:18 -04:00
Gabe Ruttner
468de0c584 feat: handle server initiated eviction on workers (#3200)
* refactor: ts in sync with python

* fix: waits respect abort signal

* chore: gen, lint

* fix: update key extraction logic in durable workflow example

* fix: flake

* fix: racey test

* chore: gen

* feat: send eviction to engine

* feat: handle eviction on workers

* fix: tests and invocation scoped eviction

* fix: cleanup scope

* refactor: cleanup how we do action keys in ts

* chore: gen, lint

* fix: test

* chore: lint

* fix: mock

* fix: cleanup

* chore: lint

* chore: copilot feedback

* fix: reconnect durable listener

* chore: feedback

* fix: bump worker status on wait and reconnect

* refactor: rename eviction notice

* fix: return error on query failure

* chore: lint

* chore: gen

* refactor: remove early completion

* Revert "refactor: remove early completion"

This reverts commit f756ce37b0.

* chore: remove unused param
2026-03-09 05:16:46 -07:00
Gabe Ruttner
1cd9660835 rip: remove unneeded durable event log update (#3186)
* rip: update

* refactor: start cleaning up proto defs

* refactor: finish cleaning up proto definitions

* fix: rm the kind

* refactor: rewire the server to have different methods for the different paths

* refactor: more intermediate work

* fix: variables

* fix: wire the kind through

* refactor: get it to compile

* chore: start fixing python

* fix: first pass at fixing python

* chore: rm unused sql

* fix: rm invocation count, rework some logic

* fix: alias (why does sqlc not catch this)

* fix: panics

* fix: add faster timeout to durable spawn test

* fix: task id bug

* refactor: more cleanup of types

* refactor: rm stale entries logic

* fix: rework getOrCreate logic

* fix: clean up a bunch more unneeded stuff

* fix: bug

* fix: python code

* fix: dag matches bug

* fix: add parent to dag to make it more broken, add timeout

* fix: more involved tests

* fix: dag waits

* fix: tests

---------

Co-authored-by: mrkaye97 <mrkaye97@gmail.com>
2026-03-07 04:02:48 -08:00
Gabe Ruttner
12bf7514c5 feat: durable-bulk-spawn (#3173)
* feat: durable bulk spawn

* feat: bulk db operations

* chore: lint

* chore: generate

* fix: sqlc hack

* cleanup

* chore: lint

* refactor: remove duplicated code

* refactor: simplify input

* ops: rules

* fix: add missing workflows to worker

* chore: add todo

* refactor: single execution path

* comment

* refactor: remove extra class

* chore: lint

* chore: lint

* refactor: batch ack

* fix: handle empty refs

* chore: feedback

* fix: durable output

* chore: lint

* chore: lint

* chore: test fix and gen

* revert: actually use bulk

* refactor: simplify path

* chore: empty commit

Made-with: Cursor
2026-03-05 16:08:13 -08:00
matt
1bffb66bb3 Feat: Branching off branches (#3150)
* feat: add branch point table

* chore: gen

* feat: id for ordering

* feat: check for `isBranchPoint` and handle branching

* feat: wire up branching over the api

* fix: api, gen

* fix: gen

* fix: branch

* feat: remove duped parent node and branch ids

* fix: rm branch count, dupe of latest id

* refactor: resolve naming

* fix: base case

* fix: test, + it's literally always a caching issue omg

* fix: docs

* chore: lint

* refactor: make branch resolution more efficient

* feat: stable sort, add a bunch of tests

* fix: confusing naming

* fix: naming

* chore: gen

* fix: update

* fix: failing test

---------

Co-authored-by: Gabe Ruttner <gabriel.ruttner@gmail.com>
2026-03-05 16:31:01 -05:00
Gabe Ruttner
a6f5fb6871 chore: rename durable fork to branch (#3174) 2026-03-05 09:56:46 -08:00
Gabe Ruttner
5b58eed1ce tests: compat testing (#3144)
* tests: compat testing

* fix: new engine old sdk

* feat: ruby ts

* fix: namespaced tests, conditions

* chore: lint ts

* fix: python compat

* chore: lint

* fix: warn on version mismatch

* chore: lint

* fix: child spawn

* refactor: address review

* chore: lint

* chore: lint

* chore: lint

* refactor: address reviews

* chore: lint

* fix: typeguard

* tests: docker-compose

* fix: docker compose

* fix: labels?

* chore: skip new tests

* fix: backwards compat

* fix: grpc proxy
2026-03-04 18:12:45 -08:00
mrkaye97
5d735b4c41 fix: linting issues 2026-03-04 13:59:05 -05:00
mrkaye97
e29459f58a chore: merge main 2026-03-04 13:38:37 -05:00
Julius Park
d94a3b4718 Add queue to update scheduled cron triggers on-demand (#3149)
Adds a queue that gets triggered whenever a cron is created, updated, or deleted that will automatically update the list of crons running in the ticker.
2026-03-04 11:34:43 -05:00
matt
6c29e48204 Feat: Dynamic worker label assign (#3137)
* feat: initial wiring work on desired labels

* feat: initial wiring

* chore: gen python

* fix: use the whole desired label thing instead

* fix: more wiring, improve types

* fix: sql type

* fix: len check

* chore: gen python

* fix: initial plural label work

* fix: store the labels properly on the task

* fix: skip cache on override

* fix: bug

* fix: scoping bug whoops

* chore: lint

* fix: send labels back over the api correctly

* feat: python test

* fix: lint

* fix: comment

* fix: override

* fix: namespaces, ugh

* fix: no need for error here

* chore: version

* feat: ruby, go, ts

* feat: versions

* fix: appease the rubocop

* chore: lint

* chore: bundle install

* fix: tests

* chore: lint

* chore: lint more

* fix: ts test

* fix: rb

* chore: gen

* chore: reset gemfile

* chore: reset changelog

* fix: pgroup

* fix: tests, part i

* Revert "chore: reset changelog"

This reverts commit b63bf7d3e5.

* Revert "chore: reset gemfile"

This reverts commit bb848bb6f0.

* fix: go -> golang mapping hack

* fix: go enums

* fix: appease the cop

* fix: namespace

* chore: gen
2026-03-04 11:03:58 -05:00
mrkaye97
a4f65454f3 Merge branch 'main' into feat-durable-execution 2026-03-03 17:24:23 -05:00
Julius Park
5e4bb9d151 Add seconds granularity to cron jobs (#3136) 2026-03-02 18:07:15 -05:00
matt
77f769bd66 Feat: Durable Memoization (#3112)
* feat: initial engine / db work

* chore: gen python

* feat: initial python work

* feat: wiring

* feat: initial test

* fix: scope memo key to the run id (we might not even need this)

* chore: gen, docstring

* fix: mandatory result type

* fix: docs

* fix: log a warning and don't cache

* fix: add test for non-unique keys, fix some bugs

* chore: lint

* chore: naming

* fix: more naming

* fix: comment

* chore: gen

* fix: docs

* fix: naming, ugh

* chore: gen

* chore: gen

* fix: always send event log entry and get ack

* fix: initial rework of memo to only use stream

* fix: start reworking python side

* fix: finish wiring up memo put

* chore: fmt

* chore: comment for monday

* fix: start reworking signature

* fix: docstring

* fix: union type for send event to improve typing a bunch

* fix: memo key

* chore: gen

* fix: rm unused query

* fix: comments
2026-03-01 16:27:56 -08:00
Gabe Ruttner
2d57b6793a Feat durable olap refactor (#3115)
* chore: lint

* feat: counting and partitioning

* feat: add reason field to DurableTaskEvictInvocationRequest and update eviction handling

* fix: eviction durable execution race

* chore: generate

* refactor: simplified migration

* refactor: address review

* refactor: analyze parent tables in migraiton

* fix: migration

* fix: remove no txn

* fix: one statement

* fix: we do infact need no transaction

* add down/up/down to the online migration test

* fix: or multiple statements

* fix: two migrations...

* chore: rm old migraiton

* chore: generate

* chore: feedback

* fix: idempotent migration

* refactor: update assertions in durable tests and clean up imports in cache.py

* revert: migraiton

* chore: wrap down
2026-02-27 10:58:01 -08:00
Gabe Ruttner
daff28dbfe Feat: durable eviction take 2 (#3075)
* feat: simplified eviction feature

* fix: assign new worker id

* test: shorter sleep

* fix: completion race on same worker

* chore: address todo

* chore: lint

* chore: generate

* fix: n+1 queries

* refactor: WasEvicted bool

* feat: evicted state

* chore: generate

* fix: map status

* fix: update PendingCallback structure to include InvocationCount

* revert: comment

* feat: add support for EVICTED status in waterfall component and metrics display

* fix: implicit eviction

* chore: readable cte

* refactor: queued bool

* refactor: rename eviction_policy

* fix: aio only

* chore: example return type

* fix: map

* feat: eviction error cases

* refactor: change external ID maps to use UUID type

* chore: feedback, cleanup

* tests: additional cases

* chore: generate

* chore: lint

* chore: lint generate

* chore: clean up comments to make matt happy

* refactor: more feedback

* chore: add TODO for worker state reconciliation and clean up comments in eviction policy

* tests: fix

* chore: gen

* test: increase ruby timeout...

* fix: invocation count

* fix: test cases

* fix: stale log entry

* chore: lint

* revert: durable tests to use time.time

* chore: lint
2026-02-27 09:25:50 -08:00
matt
6f3f6e08ac Feat: Replay as new (or from a node) (#3055)
* feat: new messages for reset

* chore: gen python

* feat: reset scaffolding

* feat: initial work

* feat: initial e2e wiring of resetting from a specific node

* fix: add branch to pk

* fix: wire up branches

* fix: add branch to awaited entry

* feat: start wiring up reset api

* fix: colname

* fix: add branch id more places

* fix: some bugs

* fix: replay

* fix: replay, simplify

* feat: add parent branch id

* fix: start reworking parent nodes and branches

* fix: parent branch wiring

* fix: start fixing some bugs

* fix: parent branch bug

* fix: advisory lock for locking the log file to prevent concurrent modification

* fix: move claude.md ignore path

* fix: remove eager replays of events

* fix: rm cruft

* fix: cleanup more params and such

* fix: return type

* fix: comment

* fix: comments

* fix: comment

* chore: gen

* chore: gen

* fix: decrease sleep time

* chore: gen again

* fix: add invocation count on event log entries, make it int32, fix toInt

* fix: more wiring

* chore: gen, simplify

* fix: lint

* fix: more zero values, I hate Go

* feat: add `is_durable` to v1_task

* feat: initial work wiring up dispatcher to increment log entry invocation counts

* feat: wire up assigned action

* fix: property

* fix: send is durable through to the engine

* fix: more invoc count wiring

* fix: node resetting

* fix:revert

* fix: import

* chore: gen

* fix: reset -> fork

* fix: rm a bunch of dead code

* fix: api

* fix: repo method

* fix: log file locking using `FOR UPDATE` + atomic compare-and-set update

* fix: move to shared repo

* feat: increment invocation count on the scheduler

* fix: naming

* fix: make test more reliable

* fix: props

* fix: node id reset
2026-02-20 13:01:46 -05:00
matt
7e3e3b8fc0 Feat: Non-determinism errors (#3041)
* fix: retrieve payloads in bulk

* fix: hash -> idempotency key

* feat: initial hashing work

* feat: check idempotency key if entry exists

* fix: panic

* feat: initial work on custom error for non-determinism

* fix: handle nondeterminism error properly

* feat: add error response, pub message to task controller

* chore: lint

* feat: add node id field to error proto

* chore: rm a bunch of unhelpful cancellation logs

* fix: conflict issues

* fix: rm another log

* fix: send node id properly

* fix: improve what we hash

* fix: improve error handling

* fix: python issues

* fix: don't hash or group id

* fix: rm print

* feat: add python test

* fix: add timeout

* fix: improve handling of non determinism error

* fix: propagate node id through

* fix: types, test

* fix: make serializable

* fix: no need to cancel internally anymore

* fix: hide another internal log

* fix: add link to docs

* fix: copilot

* fix: use sha256

* fix: test cleanup

* fix: add error type enum

* fix: handle exceptions on the worker

* fix: clean up a bunch of cursor imports

* fix: cursor docstring formatting

* fix: simplify idempotency key func

* fix: add back cancellation logs

* feat: tests for idempotency keys

* fix: add a couple more for priority and metadata

* chore: gen

* fix: python reconnect

* fix: noisy error

* fix: improve log

* fix: don't run durable listener if no durable tasks are registered

* fix: non-null idempotency keys
2026-02-18 11:27:02 -05:00
matt
eaf6bba824 Refactor: Remove separate callback table (#3045)
* fix: remove callback table

* fix: type

* fix: type

* fix: wiring everything up

* fix: result payload for replays

* chore: lint

* feat: set fillfactor

* chore: gen

* fix: simplify v1 match changes

* fix: simplify v1 match wiring

* fix: rm print line

* fix: some more wiring

* fix: wiring

* chore: comments

* chore: gen, proto naming

* chore: comments

* fix: rm comment

* fix: broken listener
2026-02-17 13:25:08 -05:00
mrkaye97
7ad5bfdf89 Merge branch 'main' into feat-durable-execution 2026-02-17 08:45:41 -05:00
Gabe Ruttner
2fdc47a6af feat: multiple slot types (#2927)
* feat: adds support for multiple slot types, primarily motivated by durable slots

---------

Co-authored-by: mrkaye97 <mrkaye97@gmail.com>
2026-02-17 05:43:47 -08:00
mrkaye97
b507dac7e3 fix: type 2026-02-16 13:32:15 -05:00
matt
05399ebf39 Feat: Durable event log wiring (#2956)
* feat: initial protos

* chore: lint

* fix: work on improving naming

* chore: rename session id to invocation count

* feat: scaffold implementation of durabletask rpc

* fix: one more session rename

* feat: initial work on the server scaffolding

* chore: gen protos for python

* feat: initial durable task client

* feat: initial durable context work for python

* fix: pass client through to runner

* fix: clean up type checking errors

* fix: cruft

* feat: initial work wiring up durable events

* fix: get -> getorcreate

* feat: query + wiring for updating latest node id

* fix: simplify, bump latest node ids in the same query

* chore: note

* feat: wire up sleeps with internal signal matches

* chore: gen

* fix: callback data writes

* feat: cache previous events

* fix: wire up external id writes

* feat: got sleeps sorta working!

* fix: tenant and external id wiring

* chore: comments

* fix: clean up some types a bit

* feat: add run triggering params to proto to allow for spawning children

* feat: first pass at child spawning

* feat: start wiring up child spawning

* fix: use `triggerWriter` for spawn

* feat: update trigger proto def

* chore: regen python

* feat: start wiring up spawning correctly with all opts

* refactor: share trigger code

* chore: remove log lines, lint

* fix: add triggered run external id

* feat: start wiring up child key storage better

* chore: gen again

* fix: gen, colname

* fix: trigger opts panicking

* hack: get things working for now

* feat: shared rpc message

* chore: fix imports

* feat: add tenant id to tables

* fix: improve ingest logic

* refactor: shared trigger opt type

* fix: send tenant id through everywhere

* chore: fix log file insert on conflict

* fix: repo

* fix: generate external id upstream

* feat: add columns to the match

* feat: first pass at durable waits on the controllers instead of the dispatcher

* fix: types

* feat: wire up callbacks

* fix: invoc counts

* fix: typing, lint

* driveby: more constants for message ids

* refactor: struct for callback keys everywhere

* fix: bugs, passing tests

* fix: return errnorows

* fix: schema

* fix: remove current callback flow

* feat: new message types

* fix: remove key from callback model

* fix: rm unused queries

* refactor: start reworking flow

* fix: start working on feedback

* fix: query

* fix: wire up external ids

* revert: drive by

* refactor: rm extra interface

* chore: move listener, lint

* refactor: remove old listener, rename

* refactor: consolidate migrations

* fix: immediately send already-satisfied callbacks

* fix: union

* chore: rm unused queries

* fix: check if entry already exists before re-spawning / signaling

* fix: node id incrementation

* fix: rm json dump

* fix: don't pass node id

* fix: store latest invocation, update query

* fix: upsert logic

* Revert "fix: upsert logic"

This reverts commit cf7c609c1d.

* fix: change logic slightly

* fix: split up get and create queries

* fix: err

* fix: pass node ids around properly

* fix: invocation handling

* fix: callback bug

* fix: naming

* fix: rm cruft method, dynamic kind

* fix: wire up memo payload and kind stuff

* fix: propagate trigger opts

* fix: child spawn signaling + olap wiring

* fix: extract output method

* feat: improve test coverage a bit

* fix: child spawning

* feat: another test

* fix: query fixes, overwrite

* fix: match bug

* fix: proto indexes, regen

* fix: eviction comment

* fix: warning for non-async durable tasks

* fix: rm contracts import

* fix: basic locking, rm sync durable tasks

* fix: invocation counts, etc.

* chore: add fixme

* fix: rm unused invocation count param from callback response

* fix: rm dispatcher id from the callback

* fix: di test

* Revert "fix: rm dispatcher id from the callback"

This reverts commit 26e6c82797.

* fix: migration

* fix: use optimistictx

* fix: lift grpc codes out of trigger repo

* fix: span names

* fix: rm comment

* fix: consolidate kind types, batching, not-null kinds

* fix: null bug

* fix: satisfied claim bug, simplify queries

* fix: add back payload storage

* fix: match bug, simplification

* fix: factor out trigger opts to the dispatcher level

* fix: factor out conditions

* fix: rm unused structs

* fix: rm dupes

* fix: migration

* refactor: switch case helpers

* fix: panic

* fix: couple warnings

* fix: lint

* fix: generate external ids properly

* refactor: return trigger task data from helper

* fix: handle matches correctly for dag spawns

* fix: add validators, one more uuid type

* chore: gen

* chore: bump pytest-asyncio to latest

* fix: store the worker instead of the dispatcher, then look up the dispatcher

* fix: store dispatcher id on the worker

* chore: lint
2026-02-16 12:23:58 -05:00
Gabe Ruttner
7875d78057 Feat: Official Ruby SDK (#3004)
* feat: initial ruby sdk

* fix: run listener

* fix: scope

* feat: rest feature clients

* fix: bugs

* fix: concurrent register

* fix: tests and ergonomics

* docs: all of them

* chore: lint

* feat: add RBS

* feat: add GitHub Actions workflow for Ruby SDK with linting, testing, and publishing steps

* chore: lint

* refactor: simplify load path setup for Hatchet REST client and remove symlink creation

* fix: cert path

* fix: test

* fix: blocking

* fix: ensure Hatchet client is only initialized once across examples

* fix: tests

* remove: unused example

* fix: bubble up errors

* test: skip flaky for now

* remove: lifespans

* fix: durable context bugs

* fix: bulk replay

* fix: tests

* cleanup: generate tooling

* fix: integration test

* chore: lint

* release: 0.1.0

* chore: remove python comments

* refactor: remove OpenTelemetry configuration and related unused options

* fix: default no healthcheck

* chore: lockfile

* feat: register as ruby

* chore: lint

* chore: update py/ts apis to include ruby

* chore: docs pass

* chore: lint

* chore: generate

* chore: cleanup

* chore: generate examples

* tests: add e2e tests

* tests: cache examples dependencies

* fix: namespace

* fix: namespace

* fix: namespaces

* chore:lint

* fix: improve cancellation workflow polling logic and add error handling

* revert: py/ts versions
2026-02-15 14:32:15 -08:00
matt
ac98a33992 Fix: Remove null bytes from error message to prevent db crash (#3010)
* fix: replace null byte in error message

* fix: rm space
2026-02-12 18:07:28 -05:00
abelanger5
851fbaf214 feat: reduced cold starts for new workers and queues (#2969)
* feat: reduced cold starts for new workers and queues

* address changes from pr review

* fix: data race

* set logs to debug on the harness

* debug for queue level as well

* debug lines for queuer

* fix: add queue notifier to v0 workflow registration

* revert: lease manager interval

* revert log level changes

* add more debug, revert reverts

* more debug

* add debug to lease manager

* do it, try it

* fix: call upsertQueue as part of workflow version put

* change log level to error again

* pr review changes
2026-02-11 13:12:10 -08:00
Greg Furman
80dc9786fd chore: run go-fmt (#2984) 2026-02-10 17:34:36 -05:00
Mohammed Nafees
5db655e9aa Do not replay invalid tasks (#2976)
* filter valid tasks when replaying

* renamings

* query optim

* slice len

* fix method signature

* PR comments

* rename var

* PR comments
2026-02-10 13:08:38 -05:00
Gabe Ruttner
4054d170d8 chore: proto naming conventions (#2913)
* fix: add type override in sqlc.yaml

* chore: gen sqlc

* chore: big find and replace

* chore: more

* fix: clean up bunch of outdated `.Valid` refs

* refactor: remove `sqlchelpers.uuidFromStr()` in favor of `uuid.MustParse()`

* refactor: remove uuidToStr

* fix: lint

* fix: use pointers for null uuids

* chore: clean up more null pointers

* chore: clean up a bunch more

* fix: couple more

* fix: some types on the api

* fix: incorrectly non-null param

* fix: more nullable params

* fix: more refs

* refactor: start replacing tenant id strings with uuids

* refactor: more tenant id uuid casting

* refactor: fix a bunch more

* refactor: more

* refactor: more

* refactor: is that all of them?!

* fix: panic

* fix: rm scans

* fix: unwind some broken things

* chore: tests

* fix: rebase issues

* fix: more tests

* fix: nil checks

* Refactor: Make all UUIDs into `uuid.UUID` (#2897)

* refactor: remove a bunch more string uuids

* refactor: pointers and lists

* refactor: fix all the refs

* refactor: fix a few more

* fix: config loader

* fix: revert some changes

* fix: tests

* fix: test

* chore: proto

* fix: durable listener

* fix: some more string types

* fix: python health worker sleep

* fix: remove a bunch of `MustParse`s from the various gRPC servers

* fix: rm more uuid.MustParse calls

* fix: rm mustparse from api

* fix: test

* fix: merge issues

* fix: handle a bunch more uses of `MustParse` everywhere

* fix: nil id for worker label

* fix: more casting in the oss

* fix: more id parsing

* fix: stringify jwt opt

* fix: couple more bugs in untyped calls

* fix: more types

* fix: broken test

* refactor: implement `GetKeyUuid`

* cleanup

* gen

* missed name

* slot naming consistency

* snake_case

* lint

* regenerate python

* typescript sdk

* deprecated getters

* lint

* fix tests

* version bumps

* chore: regen sqlc

* chore: replace pgtype.UUID again

* fix: bunch more type errors

* no compat

* fix: remove isort

* note

* Update sdks/python/CHANGELOG.md

Co-authored-by: matt <mrkaye97@gmail.com>

* feedback

* fix: no isort in ci

* tui

* ts lint

* lint

* weird undefined

* fix test

* last changes

* lint

* fix: ts child spawning

* consistent naming

* map fields

---------

Co-authored-by: mrkaye97 <mrkaye97@gmail.com>
2026-02-05 10:35:13 -08:00
matt
9781e06c55 Fix: Check uuid.Nil when creating or groups too (#2958)
* fix: also check uuid.Nil 🤦

* fix: weird diff
2026-02-05 12:05:00 -05:00
matt
91c280295e Fix: Always generate UUID for OrGroup (#2955)
* fix: null uuid for or group

* fix: engine side

* fix: two more
2026-02-05 10:50:03 -05:00
matt
961682d704 Hotfix: More panics (#2945)
* fix: check if dispatcher id is nil

* explicit nil check

* proposal: extra return out of `GetDispatcherIdsForWorkers`
2026-02-04 19:08:17 -05:00
matt
a782d9fd01 Hotfix: UUID Panics (#2944)
* fix: panic

* fix: more panic risks

* fix: two more possible panics
2026-02-04 15:11:08 -05:00
matt
a52a2b4b90 Fix: Resource ID type (#2929)
* fix: resource id

* fix: userId

* fix: make resourceId string

* chore: minor cleanup

* chore: naming

* fix: user id type casts

* fix: debug message

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-04 10:09:33 -05:00
abelanger5
2ddcbd2672 refactor: use typed maps (#2928)
* refactor: use typed maps

* self-review comments
2026-02-03 19:35:09 -05:00
abelanger5
53097b054a fix: startup for optimistic scheduler (#2924) 2026-02-03 14:30:49 -05:00
matt
058968c06b Refactor: Attempt II at removing pgtype.UUID everywhere + convert string UUIDs into uuid.UUID (#2894)
* fix: add type override in sqlc.yaml

* chore: gen sqlc

* chore: big find and replace

* chore: more

* fix: clean up bunch of outdated `.Valid` refs

* refactor: remove `sqlchelpers.uuidFromStr()` in favor of `uuid.MustParse()`

* refactor: remove uuidToStr

* fix: lint

* fix: use pointers for null uuids

* chore: clean up more null pointers

* chore: clean up a bunch more

* fix: couple more

* fix: some types on the api

* fix: incorrectly non-null param

* fix: more nullable params

* fix: more refs

* refactor: start replacing tenant id strings with uuids

* refactor: more tenant id uuid casting

* refactor: fix a bunch more

* refactor: more

* refactor: more

* refactor: is that all of them?!

* fix: panic

* fix: rm scans

* fix: unwind some broken things

* chore: tests

* fix: rebase issues

* fix: more tests

* fix: nil checks

* Refactor: Make all UUIDs into `uuid.UUID` (#2897)

* refactor: remove a bunch more string uuids

* refactor: pointers and lists

* refactor: fix all the refs

* refactor: fix a few more

* fix: config loader

* fix: revert some changes

* fix: tests

* fix: test

* chore: proto

* fix: durable listener

* fix: some more string types

* fix: python health worker sleep

* fix: remove a bunch of `MustParse`s from the various gRPC servers

* fix: rm more uuid.MustParse calls

* fix: rm mustparse from api

* fix: test

* fix: merge issues

* fix: handle a bunch more uses of `MustParse` everywhere

* fix: nil id for worker label

* fix: more casting in the oss

* fix: more id parsing

* fix: stringify jwt opt

* fix: couple more bugs in untyped calls

* fix: more types

* fix: broken test

* refactor: implement `GetKeyUuid`

* chore: regen sqlc

* chore: replace pgtype.UUID again

* fix: bunch more type errors

* fix: panic
2026-02-03 11:02:59 -05:00
abelanger5
d56dee4266 feat: durable user event log (#2861)
* placeholder

* feat: db tables for user events (#2862)

* feat: db tables for user events

* move event payloads to payloads table, fix env var loading

* fix: address pr review comments

* missed save

* feat: optimistic scheduling (#2867)

* feat: db tables for user events

* move event payloads to payloads table, fix env var loading

* refactor: small changes to prepare optimistic txs

* feat: optimistic scheduling

* address pr review comments

* rm comments

* fix: rampup test race condition

* fix: goleak

* feat: grpc-side triggers

* fix: config and sem logic

* fix: respect optimistic scheduling env var

* add optimistic to testing matrix, remove pg-only mode

* fix cleanup of pubbuffers

* merge migrations

* last testing fixes
2026-02-02 18:04:02 -05:00
Gabe Ruttner
8186db18a4 fix: only fetch finalized workflow runs (#2896)
* only fetch finalized

* further optimization

* configurable buffer
2026-01-31 16:32:47 -05:00
abelanger5
04953129a4 fix: compute payload size correctly for pg_notify (#2873) 2026-01-31 16:31:49 -05:00
matt
a3fe89ef03 Feat: Workflow input JSON schema in trigger preview (#2851)
* feat: add input json schema to workflow version

* feat: add json schema to putworkflow proto

* feat: wire up writes of the json schema

* chore: gen python

* feat: send json schema from the python code

* feat: wiring

* feat: pass json schema into code editor

* feat: pass prop

* fix: clean up validation stuff

* feat: allow zod `input` as optional ts field

* fix: try except logic hack

* fix: rename input -> inputValidator for consistency

* chore: gen api

* fix: improve hack slightly

* chore: changelogs, versions

* feat: zod example

* chore: rework api a bit

* fix: tsc, allow schema to update

* fix: improve loading state

* fix: api cleanup, sqlc cleanup

* fix: initial mount

* chore: lint

* chore: lint

* chore: tsc

* fix: lint

* fix: unwind unneeded change

* [Python] Feat: Default additional metadata (#2876)

* Add doc about autoscaling workers (#2864)

* add doc for autoscaling workers

* oldest also in running stats

* chore(deps): bump google.golang.org/api from 0.262.0 to 0.263.0 (#2869)

Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.262.0 to 0.263.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.262.0...v0.263.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-version: 0.263.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/getsentry/sentry-go from 0.41.0 to 0.42.0 (#2870)

Bumps [github.com/getsentry/sentry-go](https://github.com/getsentry/sentry-go) from 0.41.0 to 0.42.0.
- [Release notes](https://github.com/getsentry/sentry-go/releases)
- [Changelog](https://github.com/getsentry/sentry-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/getsentry/sentry-go/compare/v0.41.0...v0.42.0)

---
updated-dependencies:
- dependency-name: github.com/getsentry/sentry-go
  dependency-version: 0.42.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump hatchet-sdk in /examples/python/quickstart (#2871)

Bumps hatchet-sdk from 1.22.10 to 1.22.11.

---
updated-dependencies:
- dependency-name: hatchet-sdk
  dependency-version: 1.22.11
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* feat: default additional meta

* feat: wiring

* chore: changelog, version

* fix: copy

* feat: add default meta to stubs

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Mohammed Nafees <hello@mnafees.me>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: migration ver

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Mohammed Nafees <hello@mnafees.me>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-29 11:38:25 -05:00
Gabe Ruttner
a8afa07dcf fix: validate json at edges and dont retry on invalid (#2882)
* drop and validate at edges

* rm submod

* use enum

* lint
2026-01-29 08:04:55 -08:00
matt
3c416ee949 Feat: OTel Collector (#2863)
* feat: initial otel collector work

* chore: vendor otel protos

* feat: add rate limiter for otelcol

* fix: clean up naming

* feat: shell implementation

* feat: add buf + generate protos

* fix: naming

* fix: naming

* chore: simplify a bit

* fix: rename logger

* fix: naming cleanup

* fix: rm unused var

* fix: rm unused struct

* chore: rm vendored stuff, don't need it apparently
2026-01-28 14:10:15 -05:00
abelanger5
a6e88074de refactor: have log line lookups use external id (#2822)
* fix: modify log line query to use external id

* pass in task external id
2026-01-20 11:17:06 -05:00
Mohammed Nafees
6eba6fa91f Billing changes (#2643)
* make changes for billing

* progress around redesign

* meter callback

* modify limits

* upcoming subscription

* fix lint

* fix payment methods

* fix build

* PR comments

* address PR comments

* update cloud contracts

* fix migration name

* fix json serialization error

* loader and fixed for managed compute

* PR comments

* upgrade Go version

* fix migration name

* fix CI

* fix lint CI

* golangci-lint fix

* dedicated subscription
2026-01-19 12:15:11 +01:00
matt
ebe2f9acfd Feat: Add additional meta to the run detail getter (#2770)
* feat: add additional meta to proto

* feat: propagate metadata

* feat: expand test

* fix: wiring

* chore: version

* chore: lint
2026-01-12 13:46:16 -05:00