Commit Graph

38 Commits

Author SHA1 Message Date
Matt Kaye
3e7a73a81c Hotfix: Go SDK scheduled workflows issues (#1727)
* fix: scheduled workflow namespace + input

* fix: send namespace through to schedules client

* tweak: do nothing with invalid json

* fix: helper
2025-05-16 08:11:33 -04:00
abelanger5
1dcb50c170 hotfix: priority on schedule workflow (#1556)
* hotfix: priority on schedule workflow

* fix: build
2025-04-15 13:10:40 -04:00
Matt Kaye
80137736af Feat: Priority (#1513)
* feat: initial work wiring up priorities

* fix: add default to default prio in the db

* feat: wire priority through api on wf creation

* feat: extend python test

* feat: priority for scheduled workflows

* feat: wire priority through python api

* feat: more wiring priority through the api

* feat: I think it works?

* feat: e2e test for priority

* it works!

* feat: expand tests for default priorities

* feat: e2e scheduling test

* fix: skip broken test for now

* fix: lint

* feat: add priority columns to cron and schedule ref  tables

* feat: update inserts to include prio

* feat: wire up more apis

* feat: more wiring

* feat: wire up more rest api fields

* chore: cruft

* fix: more wiring

* fix: lint

* chore: gen + wire up priorities

* fix: retries

* fix: try changing fixture scope

* chore: bump version again

* feat: send priority with action payload

* fix: generate script

* Feat  priority ts (#1518)

* feat: initial work wiring up priorities

* fix: add default to default prio in the db

* feat: wire priority through api on wf creation

* feat: extend python test

* feat: priority for scheduled workflows

* feat: wire priority through python api

* feat: more wiring priority through the api

* feat: I think it works?

* feat: e2e test for priority

* it works!

* feat: expand tests for default priorities

* feat: e2e scheduling test

* chore: minor version for priority

* fix: skip broken test for now

* fix: lint

* feat: add priority columns to cron and schedule ref  tables

* feat: update inserts to include prio

* feat: wire up more apis

* feat: more wiring

* feat: wire up more rest api fields

* chore: cruft

* fix: more wiring

* fix: lint

* chore: gen + wire up priorities

* fix: increase timeout

* fix: retries

* fix: try changing fixture scope

* chore: generate

* fix: set schedule priority

* feat: priority

* fix: move priority to wf

* release: 1.2.0

* rm log

* fix: import

* fix: add priority to step

---------

Co-authored-by: mrkaye97 <mrkaye97@gmail.com>

* fix: add dummy runs to priority test to prevent race conditions

* fix: non-breaking field

* fix: gen

* feat: initial pass at docs

* feat: priority in go sdk

* feat: initial work on go example

* fix: doc examples

* fix: proofread

* chore: version

* feat: go sdk

* fix: lint

* fix: declarations and add back RunAsChild

* fix: child workflows

* fix: namespace

* fix: faster child workflows

* fix: sticky

* add back run as child

---------

Co-authored-by: Gabe Ruttner <gabriel.ruttner@gmail.com>
Co-authored-by: Alexander Belanger <alexander@hatchet.run>
2025-04-14 16:22:00 -04:00
Matt Kaye
5062bf1e3e V1 SDKs and Docs (#1361)
New SDKs and docs for the v1 release.
2025-03-25 15:45:07 -07:00
abelanger5
1f2096313d feat: v1 engine (#1318) 2025-03-11 14:57:13 -04:00
abelanger5
e12e700980 feat: CANCEL_NEWEST strategy and make cancel in progress more reliable (#1127) 2024-12-18 01:40:14 +00:00
abelanger5
6a0d8c762d feat: shared meta (#1102) 2024-12-12 20:42:01 -05:00
abelanger5
3d616c42b1 feat(go-sdk): expose user data (#1107) 2024-12-12 20:41:37 -05:00
abelanger5
197bdd1f88 feat: exponential backoff (#1062)
* initial migration

* feat: exp backoff, fix linting

* fix utc issue and cleanup
2024-11-21 13:39:02 -05:00
abelanger5
19e151e29a fix: RunWorkflow and SpawnWorkflow should respond with consistent APIs (#965) 2024-10-15 11:09:58 -04:00
Sean Reilly
29721cd1f0 Feat bulk workflows (#940)
Adds support for inserting workflows in bulk via the API and an optional buffered insert on the engine.
2024-10-14 15:35:29 -04:00
abelanger5
a1a10b4073 feat: dynamic rate limits (#904)
* wip: step run expressions on rate limits

* feat: dynamic rate limits

* chore: v0.47.0

* chore: address changes from PR review

* fix: improved error handling

* address pr review

* better error messages for step run cels, remove debug logs

* fix: hash

---------

Co-authored-by: gabriel ruttner <gabriel.ruttner@gmail.com>
2024-09-26 22:00:34 +00:00
abelanger5
d23e5d9963 feat: expression-based concurrency keys (#889)
* feat: expression-based concurrency keys

* fix: build

* fix: typos

* fix: gen

* fix: migration

* fix: remove print statements

* fix: reassignment bugs, retries on closed transport, pr review
2024-09-19 10:32:22 -04:00
Steinway Wu
44d03af852 fix: propagating additional metadata for child workflows (#882)
* fix: propagating additional metadata for child workflows

* add unit test
2024-09-19 13:28:46 +00:00
abelanger5
9efd9368fd feat: deduplicated enqueue error and additional context methods (#747)
* feat: additional context fields and dedupe error

* fix: case on error properly
2024-07-26 18:32:56 +00:00
Gabe Ruttner
fd947cb5bc feat: go worker assignment (#741)
* feat: create worker with label

* feat: worker context

* feat: dynamic labels

* feat: affinity

* fix: ptr

* fix: nil labels

* feat: sticky dag

* feat: sticky docs

* feat: sticky children

* chore: lint

* fix: tests

* fix: possibly nil workerId

* chore: cleanup unneeded pointers
2024-07-26 10:19:11 -07:00
abelanger5
8a991c1cb0 fix(go-sdk): add schedule timeout (#673)
* fix(go-sdk): add schedule timeout

* chore: lint
2024-06-30 11:38:39 -04:00
abelanger5
7c3ddfca32 feat: api server extensions (#614)
* feat: allow extending the api server

* chore: remove internal packages to pkg

* chore: update db_gen.go

* fix: expose auth

* fix: move logger to pkg

* fix: don't generate gitignore for prisma client

* fix: allow extensions to register their own api spec

* feat: expose pool on server config

* fix: nil pointer exception on empty opts

* fix: run.go file
2024-06-19 09:36:13 -04:00
abelanger5
5f93a41e8b fix(go-sdk): propagate namespace to child workflows (#562) 2024-06-06 12:36:50 -04:00
Gabe Ruttner
c8b59d1cc1 Feat add additional meta to trigger (#458)
* feat: add additional metadata to sdk

* docs: metadata

* fix: build

* feat: filter gif
2024-05-08 10:07:28 -04:00
abelanger5
7543a0c2a5 add jobs which always run on failure (#445)
* (wip) prisma schema

* feat: on-failure steps

* chore: address changes from PR review

* chore: bump migration number
2024-05-06 15:39:22 -04:00
abelanger5
fcdbfbf0ab fix: pass external error on PutWorkflow (#401)
* fix: pass external error on PutWorkflow

* fix: don't set duration as pointer to empty string
2024-04-22 05:35:17 -04:00
Luca Steeb
3c112d9df8 fix(workflow): remove workflow timeout in favor of step timeout (#366) 2024-04-10 23:53:00 +07:00
abelanger5
066b3c5b71 feat(engine): initial rate-limiting engine implementation (#324)
* feat(engine): initial rate-limiting engine implementation

* fixes and implement go sdk rate limiting
2024-04-02 10:53:03 -04:00
abelanger5
77e5d2b77c feat(go-sdk): spawnWorkflow method and get up to speed with other sdks (#297)
* feat(go-sdk): spawnWorkflow method and get up to speed with other sdks

* fix: manual trigger example

* fix: linting errors

* fix: double serialization from go sdk

* fix: spawn workflow logic and procedural example

* test(e2e): add procedural test

* fix: panic in e2e test

* fix: e2e test preparation

* fix: api server url in test.yml

* fix: load test server url

* chore: make num children configurable

* address pr review
2024-03-29 14:07:39 -07:00
abelanger5
092f54c64f refactor: separate api and engine repositories, change ticker logic (#281)
* refactor: separate api and engine repositories, change ticker logic

* fix: nil error blocks

* fix: run migration on load test

* fix: generate db package in load test

* fix: test.yml

* fix: add pnpm to load test

* fix: don't lock CTEs with columns that don't get updated

* fix: update heartbeat for worker every 4 seconds, not 5

* chore: remove dead code

* chore: update python sdk

* chore: add back telemetry attributes
2024-03-21 14:10:34 -04:00
abelanger5
2d625fec81 feat: round robin queueing (#194) 2024-02-26 00:16:40 -05:00
abelanger5
df3f540748 feat: add retries to the engine and SDKs (#171)
This PR adds support for retrying failed step runs against the engine and SDKs. This was tested up to 30 retries per step run, with both failure and success at the 30th step run. Each SDK now has a `retries` configurable param for steps when declaring a workflow.
2024-02-16 13:00:22 -05:00
abelanger5
82d7995343 feat: manual triggers and give clients a hook into step run events (#141)
* feat: pubsub for clients, more qol stuff

* fix: generate sqlc files

* chore: linting and comments
2024-02-02 12:52:34 -05:00
abelanger5
d63b66a837 feat: concurrency groups (#135)
* first pass at moving controllers around

* feat: concurrency limits for strategy CANCEL_IN_PROGRESS

* fix: linting

* chore: bump python sdk version
2024-01-30 00:00:28 -05:00
abelanger5
14a971bfd7 feat: improve versioning to use checksums (#129)
* feat: improve versioning to use checksums

* fix: remove json ignore for scheduled triggers

* update migration with random strings for checksum
2024-01-26 20:47:15 -05:00
abelanger5
78685d0098 feat(security): multiple encryption options, API tokens, easier setup (#125)
* (wip) encryption

* feat: api tokens

* chore: add api token generation command

* fix: e2e tests

* chore: set timeout for e2e job

* fix: e2e tests, remove client-side certs

* chore: address PR review comments

* fix: token tests

* chore: address review comments and fix tests
2024-01-26 15:38:36 -05:00
abelanger5
52fde1e704 feat: dag-style execution (#108)
* feat: dag-style execution

* docs: update to reflect new context

* ensure no cycles

* remove example cycle

* linting

* lint and small fixes

* update deferred rollback

* last rollback handling

* unset max issues

* fix requeue edge case
2024-01-16 11:31:24 -05:00
Luca Steeb
0ad77d50cb chore(lint): run golangci-lint --fix (#105) 2024-01-16 01:34:07 +07:00
abelanger5
752d5b0ab7 feat: support passing inputs to scheduled workflows (#104)
* fix: usage of RegisterAction

* make registered actions callable

* chore: update yaml example

* docs: register action documented

* feat: support input to scheduled workflows

* add worker

* add client

* docs: add input to schedule workflow

* chore: generate with updated protoc

* chore: sqlc generate
2024-01-12 00:27:34 -05:00
abelanger5
62445dc37f feat: support one-time scheduled workflows (#84)
* feat: support one-time scheduled workflows

* refactor: move schedule out of workflow trigger def

* docs: add scheduling workflows section

* docs: update creating workflow

* only cancel schedules that are in the future
2024-01-08 10:03:32 -05:00
abelanger5
bf16ba3d30 feat: better support for workflow versions (#55) 2024-01-02 16:15:58 -05:00
Alexander Belanger
366c79441d first commit 2023-12-15 13:08:04 -05:00