Commit Graph

81 Commits

Author SHA1 Message Date
Mohammed Nafees
c86a65bb0f Add new streaming support to Go SDK (#1955)
* add Go SDK streaming support

* make docs changes for go sdk streaming

* fix git lfs warning

* streaming go example

* fix lint

* fix auto generated snip

* revert poetry lock changes

* some cleanup
2025-07-11 18:00:30 +02:00
Matt Kaye
752e6bd5cf Feat: Add event key to CEL env for filtering (#1748)
* feat: add event key to env

* chore: generate a bunch of stuff again

* feat: add a test for filtering by event key

* refactor: duped code

* fix: skip flaky test in ci

* Fix: Add filter payloads to context (#1743)

* feat: add payload to ctx

* chore: ver

* fix: add filter payload to go

* fix: tests + lint

* fix: lint

* fix: naming
2025-05-19 12:52:41 -04:00
Matt Kaye
9a5e4d7642 Fix: Add workflowId and workflowVersionId to AssignedAction (#1726)
* feat: send workflow id + version id back on assigned action

* feat: wire up python sdk

* fix: patch ver

* feat: wire ts up

* feat: wire up Go

* fix: lint + tests

* Feat: Cancel from context in TS (#1722)

---------

Co-authored-by: Gabe Ruttner <gabriel.ruttner@gmail.com>
2025-05-16 11:46:57 -04:00
Matt Kaye
3e7a73a81c Hotfix: Go SDK scheduled workflows issues (#1727)
* fix: scheduled workflow namespace + input

* fix: send namespace through to schedules client

* tweak: do nothing with invalid json

* fix: helper
2025-05-16 08:11:33 -04:00
abelanger5
5c5c1aa5a1 feat: more features in the load testing harness (#1691)
* fix: make stripped payload size configurable

* feat: more load test features

* Update cmd/hatchet-loadtest/do.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: try to fix load tests

* increase timeout, update goleak ignores

* fix: data race in scheduler with snapshot input

* fix: logger improvements

* add one more goleak ignore

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-07 21:39:30 -04:00
abelanger5
ffbeafc204 revert: add back testing harness (#1659)
* re-add new testing harness

* add healthcheck port and pick random grpc port to listen on

* feat: parallel load tests and faster tests

* make parallelism = 5

* fix: lint

* add linter to pre

* fix: add back rampup fixes

* reduce matrix on PR, add matrix to pre-release step

* make load tests less likely to block

* make limit strategy group round robin

* uncomment lines
2025-05-01 15:22:30 -04:00
abelanger5
e6d35e0737 fix(go-sdk): actionId casing (#1637) 2025-04-29 09:58:44 -04:00
Matt Kaye
2df1b32932 fix: add ToPB implementation for parent condition (#1601) 2025-04-23 15:05:04 -04:00
Matt Kaye
80137736af Feat: Priority (#1513)
* feat: initial work wiring up priorities

* fix: add default to default prio in the db

* feat: wire priority through api on wf creation

* feat: extend python test

* feat: priority for scheduled workflows

* feat: wire priority through python api

* feat: more wiring priority through the api

* feat: I think it works?

* feat: e2e test for priority

* it works!

* feat: expand tests for default priorities

* feat: e2e scheduling test

* fix: skip broken test for now

* fix: lint

* feat: add priority columns to cron and schedule ref  tables

* feat: update inserts to include prio

* feat: wire up more apis

* feat: more wiring

* feat: wire up more rest api fields

* chore: cruft

* fix: more wiring

* fix: lint

* chore: gen + wire up priorities

* fix: retries

* fix: try changing fixture scope

* chore: bump version again

* feat: send priority with action payload

* fix: generate script

* Feat  priority ts (#1518)

* feat: initial work wiring up priorities

* fix: add default to default prio in the db

* feat: wire priority through api on wf creation

* feat: extend python test

* feat: priority for scheduled workflows

* feat: wire priority through python api

* feat: more wiring priority through the api

* feat: I think it works?

* feat: e2e test for priority

* it works!

* feat: expand tests for default priorities

* feat: e2e scheduling test

* chore: minor version for priority

* fix: skip broken test for now

* fix: lint

* feat: add priority columns to cron and schedule ref  tables

* feat: update inserts to include prio

* feat: wire up more apis

* feat: more wiring

* feat: wire up more rest api fields

* chore: cruft

* fix: more wiring

* fix: lint

* chore: gen + wire up priorities

* fix: increase timeout

* fix: retries

* fix: try changing fixture scope

* chore: generate

* fix: set schedule priority

* feat: priority

* fix: move priority to wf

* release: 1.2.0

* rm log

* fix: import

* fix: add priority to step

---------

Co-authored-by: mrkaye97 <mrkaye97@gmail.com>

* fix: add dummy runs to priority test to prevent race conditions

* fix: non-breaking field

* fix: gen

* feat: initial pass at docs

* feat: priority in go sdk

* feat: initial work on go example

* fix: doc examples

* fix: proofread

* chore: version

* feat: go sdk

* fix: lint

* fix: declarations and add back RunAsChild

* fix: child workflows

* fix: namespace

* fix: faster child workflows

* fix: sticky

* add back run as child

---------

Co-authored-by: Gabe Ruttner <gabriel.ruttner@gmail.com>
Co-authored-by: Alexander Belanger <alexander@hatchet.run>
2025-04-14 16:22:00 -04:00
Matt Kaye
58d54703b2 Feat: Non-Retryable Exceptions (#1456)
* feat: add boolean flag to proto

* feat: initial wiring up priorities and non-retryables

* fix: query

* fix: cruft comment

* fix: rm priority changes

* feat: python side

* feat: tests for non-retrying workflows

* feat: expand tests

* chore: generate ts

* feat: add name prop to wf

* feat(go-sdk): non retryable error

* feat: start implementing ts

* cleanup: simplify to raising a specific error

* fix: simplify ts

* feat: ts examples

* feat: ver

* feat: docs

* fix: tests + linters

---------

Co-authored-by: Alexander Belanger <alexander@hatchet.run>
2025-04-01 15:34:43 -04:00
abelanger5
c71ae728ed feat(go-sdk): v1 durable event listener, task factory, DAG triggers (#1427)
* feat: durable event listener

* Feat  go task wrapper (#1429)

* feat: improved typing and single task

* add back durable

* conditions example

* on failure

* fix

* cleanup

* fix order

* add durable event

* rewrite durable event wkflow

* fix conditions and move shared listener

* Update pkg/client/durable_listener.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Gabe Ruttner <gabriel.ruttner@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-03-27 15:21:30 -07:00
Matt Kaye
5062bf1e3e V1 SDKs and Docs (#1361)
New SDKs and docs for the v1 release.
2025-03-25 15:45:07 -07:00
abelanger5
1f2096313d feat: v1 engine (#1318) 2025-03-11 14:57:13 -04:00
Matt Kaye
9efd56c7de Feat: Propagate Error Through Context (#1193)
* feat: add query to fetch upstream errors from db

* fix: return many

* feat: propagate errors through `input`

* fix: implement the method to get the errors out

* fix: query cleanup

* feat: rename errors

* fix: col names

* fix: key name in the json

* feat: add method to context to get failed step errors

* fix: add 👀

Co-authored-by: abelanger5 <belanger@sas.upenn.edu>

* feat: add error log if not errors

* fix: logger

* fix: simplify query

---------

Co-authored-by: abelanger5 <belanger@sas.upenn.edu>
2025-01-17 21:49:13 -05:00
Gabe Ruttner
cc0a8db4fd feat: expose stepId on context (#1169)
* feat: expose stepId on context

* fix: test context
2025-01-08 16:12:25 -05:00
abelanger5
75cd8e6795 fix: memory leak in go-sdk (#1168) 2025-01-08 14:41:17 -05:00
Gabe Ruttner
e92146816f fix: webhook workers on rebalance (#1162)
* fix: log ui

* fix: partition handling and unregister

* fix: concurrent cleanup

* feat: op pool

* fix: run or continue partition id

* fix: return false out of check
2025-01-07 10:54:15 -08:00
abelanger5
94d14336aa feat(go-sdk): blocking worker (#1106) 2024-12-12 20:42:13 -05:00
abelanger5
3d616c42b1 feat(go-sdk): expose user data (#1107) 2024-12-12 20:41:37 -05:00
abelanger5
197bdd1f88 feat: exponential backoff (#1062)
* initial migration

* feat: exp backoff, fix linting

* fix utc issue and cleanup
2024-11-21 13:39:02 -05:00
Gabe Ruttner
3850964a98 feat: initial doc pages (#1020)
* generate initial cloud client

* feat: initial doc pages

* feat: cloud register id, action filtering

* feat:cloud register

* fix: env var

* chore:lint

---------

Co-authored-by: Alexander Belanger <alexander@hatchet.run>
2024-11-08 07:46:43 -08:00
abelanger5
19e151e29a fix: RunWorkflow and SpawnWorkflow should respond with consistent APIs (#965) 2024-10-15 11:09:58 -04:00
Sean Reilly
29721cd1f0 Feat bulk workflows (#940)
Adds support for inserting workflows in bulk via the API and an optional buffered insert on the engine.
2024-10-14 15:35:29 -04:00
Gabe Ruttner
e046566db8 fix: go sdk improvements (#952)
* fix: default service

* fix: lowercase

* fix: runId in alert

* fix: correct file
2024-10-08 13:47:03 -04:00
Gabe Ruttner
5ee9560c56 feat: expose addl meta on ctx (#938) 2024-10-03 11:02:15 -07:00
abelanger5
a1a10b4073 feat: dynamic rate limits (#904)
* wip: step run expressions on rate limits

* feat: dynamic rate limits

* chore: v0.47.0

* chore: address changes from PR review

* fix: improved error handling

* address pr review

* better error messages for step run cels, remove debug logs

* fix: hash

---------

Co-authored-by: gabriel ruttner <gabriel.ruttner@gmail.com>
2024-09-26 22:00:34 +00:00
abelanger5
d23e5d9963 feat: expression-based concurrency keys (#889)
* feat: expression-based concurrency keys

* fix: build

* fix: typos

* fix: gen

* fix: migration

* fix: remove print statements

* fix: reassignment bugs, retries on closed transport, pr review
2024-09-19 10:32:22 -04:00
Steinway Wu
44d03af852 fix: propagating additional metadata for child workflows (#882)
* fix: propagating additional metadata for child workflows

* add unit test
2024-09-19 13:28:46 +00:00
Gabe Ruttner
53be615d5f Enhancement webhook usability (#807)
* feat: secret copier

* feat: improved form

* fix: quotes

* wip: improved flow

* feat: health check logging

* fix: page design

* fix: hard delete, no upsert

* fix: reset modal state

* fix: empty text

* fix: worker state

* fix: update only token

* fix: dont delete name

* fix: logs component

* fix: sort order

* chore: build

* fix: webhook worker cleanup

* chore: squash migrations

* Update api-contracts/openapi/paths/webhook-worker/webhook-worker.yaml

Co-authored-by: abelanger5 <belanger@sas.upenn.edu>

* chore: rename

* fix: wrong query

---------

Co-authored-by: abelanger5 <belanger@sas.upenn.edu>
2024-08-23 10:09:09 -04:00
Gabe Ruttner
9bea55438a Fix webhook healthcheck race (#797)
* fix: race

* fix: partition no rows

* chore: move to workers tab

* feat: redirect empty worker path to all

* chore: add worker type and webhook id

* fix: upsert webhook worker

* fix: update by webhookId

* fix: only stub on create

* feat: url on worker

* chore: migration version

* fix: move

* fix: upsert

* fix: upert

* chore: fix migration

* fix: migrations

* chore: generate
2024-08-21 19:23:24 +00:00
Viktor Szépe
0948598749 Fix typos (#775) 2024-08-10 10:58:33 +00:00
abelanger5
9efd9368fd feat: deduplicated enqueue error and additional context methods (#747)
* feat: additional context fields and dedupe error

* fix: case on error properly
2024-07-26 18:32:56 +00:00
Gabe Ruttner
fd947cb5bc feat: go worker assignment (#741)
* feat: create worker with label

* feat: worker context

* feat: dynamic labels

* feat: affinity

* fix: ptr

* fix: nil labels

* feat: sticky dag

* feat: sticky docs

* feat: sticky children

* chore: lint

* fix: tests

* fix: possibly nil workerId

* chore: cleanup unneeded pointers
2024-07-26 10:19:11 -07:00
Luca Steeb
62e02495c9 fix(webhooks): remove unused workflows field (#665) 2024-07-02 15:24:23 +01:00
abelanger5
8a991c1cb0 fix(go-sdk): add schedule timeout (#673)
* fix(go-sdk): add schedule timeout

* chore: lint
2024-06-30 11:38:39 -04:00
Luca Steeb
b88989d825 fix(webhooks): send ActionPayload as string (#657) 2024-06-26 23:20:50 +00:00
Luca Steeb
a51681ddc6 fix(webhooks): use PUT for healthcheck (#644) 2024-06-26 10:39:11 -04:00
Luca Steeb
5ffa3d1839 fix(worker): deprecate On in favor of RegisterWorkflow (#627) 2024-06-25 22:26:04 +00:00
Luca Steeb
1490d88954 feat: webhook workers (#542)
Adds serverless support via the concept of webhook workers. Allows any webhook to be registered as a serverless endpoint for executing a step.
2024-06-25 17:06:43 -04:00
abelanger5
7c3ddfca32 feat: api server extensions (#614)
* feat: allow extending the api server

* chore: remove internal packages to pkg

* chore: update db_gen.go

* fix: expose auth

* fix: move logger to pkg

* fix: don't generate gitignore for prisma client

* fix: allow extensions to register their own api spec

* feat: expose pool on server config

* fix: nil pointer exception on empty opts

* fix: run.go file
2024-06-19 09:36:13 -04:00
Gabe Ruttner
e09ee98df5 feat: expose retry count (#524)
* feat: expose retry count

* feat: expose retry count go

* docs: accessing retry count

* fix: import

* fix: tests

* fix: docs formatting

---------

Co-authored-by: Alexander Belanger <alexander@hatchet.run>
Co-authored-by: abelanger5 <belanger@sas.upenn.edu>
2024-06-07 15:27:38 +00:00
abelanger5
5f93a41e8b fix(go-sdk): propagate namespace to child workflows (#562) 2024-06-06 12:36:50 -04:00
Gabe Ruttner
e6617e1381 feat: refresh timeout (#495)
* feat: show timeout on step run

* feat: refresh timeout

* fix: increment timeout from last time

* feat: add step run event for timeout refresh

* chore: generate

* feat: add refreshing timeouts docs section

* chore: linting

* feat: go client

* chore: generate

* chore: generate

* fix: test context

* chore: generate

* fix: rm txn

* fix: validator

* chore: lint

---------

Co-authored-by: Alexander Belanger <alexander@hatchet.run>
2024-05-16 12:23:36 -04:00
Gabe Ruttner
48d06b931a feat: client releasable slots (#476)
* feat: add release slot proto

* feat: add semaphore release state and methods

* feat: go sdk and example

* docs: manual slot release

* chore: linting

* fix: broken test

* fix: unlink step run on manual release

* feat: release slot event

* fix: test

* fix: revert e2e test changes

* chore: remove debug line

* fix: place step run query in same tx

* fix: change migration release version

---------

Co-authored-by: Alexander Belanger <belanger@sas.upenn.edu>
2024-05-14 15:22:46 -04:00
abelanger5
7543a0c2a5 add jobs which always run on failure (#445)
* (wip) prisma schema

* feat: on-failure steps

* chore: address changes from PR review

* chore: bump migration number
2024-05-06 15:39:22 -04:00
abelanger5
671a7e1474 feat(go-sdk): capture panics and send to alerter (#403) 2024-04-22 05:35:40 -04:00
abelanger5
4ce1dd8632 feat: multi-workflow runs listener on a single endpoint
* new api-contract for workflow run events

* feat: initial implementation for new subscribe listener

* fix: sync issues and send workflow runs immediately

* refactor: add context to all engine db queries, fix deadlocking query

* fix: use new ctx for deleting dispatcher and ticker

* add cancellation reasons

* fix: docs linting

---------

Co-authored-by: gabriel ruttner <gabriel.ruttner@gmail.com>
2024-04-18 20:55:11 -04:00
Gabe Ruttner
f43f32283c feat(py/go): namespaces (#354)
* feat: namespaced python

* wip: namespaced go

* fix: service name

* fix: tests

* feat: client WithNamespace

* feat: namespace example

* feat: namespaced event triggers

* docs: namespace docs

* chore: linting

---------

Co-authored-by: gabriel ruttner <gabe@hatchet.run>
2024-04-15 11:26:05 -07:00
Luca Steeb
3c112d9df8 fix(workflow): remove workflow timeout in favor of step timeout (#366) 2024-04-10 23:53:00 +07:00
abelanger5
066b3c5b71 feat(engine): initial rate-limiting engine implementation (#324)
* feat(engine): initial rate-limiting engine implementation

* fixes and implement go sdk rate limiting
2024-04-02 10:53:03 -04:00