1ea4dfc5de
feat: deduplicated enqueue ( #735 )
...
* wip
* wip: functional query
* feat: expose affinity config
* feat: add weight to proto
* feat: upsert affinity state on worker start
* fix: linting
* feat: add upsert proto
* feat: upsert handler
* feat: revise model
* fix: labels
* feat: functional desired worker
* wip: ui
* feat: add state to step run events
* fix: filter empty keys
* fix: labels as badges
* feat: empty state and descriptive text
* chore: add todo
* chore: whitespace
* chore: cleanup
* chore: cleanup
* chore: fix hash
* chore: squash migrations
* fix: fair worker assignment
* fix: remaining slots on valid desired workers
* wip: sticky
* fix: count slots
* chore: rm log line
* feat: expose sticky config
* wip: sticky dag
* feat: expose desired worker id to trigger
* feat: trigger on desired worker
* feat: typescript docs
* feat: sticky python
* feat: py sticky children
* wip: py affinity
* serverless note
* feat: complete python examples
* linting
* feat: deduplicated enqueue
* fix: address changes from PR review
* chore: generate
---------
Co-authored-by: gabriel ruttner <gabriel.ruttner@gmail.com >
2024-07-26 16:47:46 +00:00
ee68786d69
feat: sticky workers ( #695 )
...
* wip
* wip: functional query
* feat: expose affinity config
* feat: add weight to proto
* feat: upsert affinity state on worker start
* fix: linting
* feat: add upsert proto
* feat: upsert handler
* feat: revise model
* fix: labels
* feat: functional desired worker
* wip: ui
* feat: add state to step run events
* fix: filter empty keys
* fix: labels as badges
* feat: empty state and descriptive text
* chore: add todo
* chore: whitespace
* chore: cleanup
* chore: cleanup
* chore: fix hash
* chore: squash migrations
* fix: fair worker assignment
* fix: remaining slots on valid desired workers
* wip: sticky
* fix: count slots
* chore: rm log line
* feat: expose sticky config
* wip: sticky dag
* feat: expose desired worker id to trigger
* feat: trigger on desired worker
* feat: typescript docs
* feat: sticky python
* feat: py sticky children
* wip: py affinity
* serverless note
* feat: complete python examples
* linting
* fix: doc link
* chore: rm debug log
* fix: simplify list labels
* fix: typo
2024-07-22 17:20:23 -04:00
7c3ddfca32
feat: api server extensions ( #614 )
...
* feat: allow extending the api server
* chore: remove internal packages to pkg
* chore: update db_gen.go
* fix: expose auth
* fix: move logger to pkg
* fix: don't generate gitignore for prisma client
* fix: allow extensions to register their own api spec
* feat: expose pool on server config
* fix: nil pointer exception on empty opts
* fix: run.go file
2024-06-19 09:36:13 -04:00
bbc4e58dd9
feat: limits ( #559 )
...
* feat: workflow run limits
* fix: resource exhausted 429
* feat: event limit
* feat: worker limit
* fix: sensible error
* fix: pb
* feat: expose limits api
* feat: default limits
* feat: add enable alert option
* feat: slack and email alerts
* fix: cron interval
* feat: make metered util
* wip: schedules and crons
* chore: squash migration
* fix: select or insert
* fix: remove unfinished meter
* chore: atlas migration
* fix: template format
* fix: shared ErrResourceExhausted
* feat: cache
* fix: limit can be nil
* fix: clarification
* fix: close meter ticker
* fix: friendly error for child workflows
2024-06-07 10:57:57 -07:00
fa07400159
feat: event and workflow run metadata ( #446 )
...
Adds additional user-defined metadata to events and workflow runs.
2024-05-06 17:10:33 -04:00
7543a0c2a5
add jobs which always run on failure ( #445 )
...
* (wip) prisma schema
* feat: on-failure steps
* chore: address changes from PR review
* chore: bump migration number
2024-05-06 15:39:22 -04:00
fcdbfbf0ab
fix: pass external error on PutWorkflow ( #401 )
...
* fix: pass external error on PutWorkflow
* fix: don't set duration as pointer to empty string
2024-04-22 05:35:17 -04:00
4ce1dd8632
feat: multi-workflow runs listener on a single endpoint
...
* new api-contract for workflow run events
* feat: initial implementation for new subscribe listener
* fix: sync issues and send workflow runs immediately
* refactor: add context to all engine db queries, fix deadlocking query
* fix: use new ctx for deleting dispatcher and ticker
* add cancellation reasons
* fix: docs linting
---------
Co-authored-by: gabriel ruttner <gabriel.ruttner@gmail.com >
2024-04-18 20:55:11 -04:00
3c112d9df8
fix(workflow): remove workflow timeout in favor of step timeout ( #366 )
2024-04-10 23:53:00 +07:00
d01736c15a
feat(py-sdk): add support for put_workflow extending from base workflow ( #329 )
...
* feat(py-sdk): add put_workflow method and extend from base class
* feat: add cron_input to create workflow API
2024-04-02 13:29:16 -04:00
066b3c5b71
feat(engine): initial rate-limiting engine implementation ( #324 )
...
* feat(engine): initial rate-limiting engine implementation
* fixes and implement go sdk rate limiting
2024-04-02 10:53:03 -04:00
ad9e9aedbf
feat(engine/api): implementation of child/parent workflow runs ( #292 )
2024-03-29 14:07:39 -07:00
092f54c64f
refactor: separate api and engine repositories, change ticker logic ( #281 )
...
* refactor: separate api and engine repositories, change ticker logic
* fix: nil error blocks
* fix: run migration on load test
* fix: generate db package in load test
* fix: test.yml
* fix: add pnpm to load test
* fix: don't lock CTEs with columns that don't get updated
* fix: update heartbeat for worker every 4 seconds, not 5
* chore: remove dead code
* chore: update python sdk
* chore: add back telemetry attributes
2024-03-21 14:10:34 -04:00
d91a8d7bc3
fix: handle inefficient assignments ( #263 )
2024-03-16 17:09:35 +07:00
c66f97c856
fix: deadlocks on workers and tickers ( #241 )
...
* chore: add sentry support to engine
* fix: deadlocks on workers and tickers
* refactor: reduce prisma calls in engine
* trigger
* fix: remove some tenant lookups
* feat: dlx and renamed taskqueue -> msgqueue
* refactor: get group key run logic
* fix: retry counts on messages and concurrency edge cases
* fix: rabbitmq integration tests
* feat: add consumer timeouts
---------
Co-authored-by: Luca Steeb <contact@luca-steeb.com >
2024-03-12 00:45:18 -04:00
46967db3ad
feat: configurable scheduling timeouts ( #204 )
...
* feat: configurable scheduling timeouts
* fix: remove validator for now
2024-02-28 01:02:13 -05:00
df3f540748
feat: add retries to the engine and SDKs ( #171 )
...
This PR adds support for retrying failed step runs against the engine and SDKs. This was tested up to 30 retries per step run, with both failure and success at the 30th step run. Each SDK now has a `retries` configurable param for steps when declaring a workflow.
2024-02-16 13:00:22 -05:00
d5f991d15b
feat: add custom user data to each step ( #156 )
...
* feat: (wip) bundle rest API in typescript client
* chore: add generated files
* chore: pnpm lock file
* extract tenant id from token subject
* feat: add custom user data to context
* docs: generated with typedoc
2024-02-09 09:43:33 -05:00
82d7995343
feat: manual triggers and give clients a hook into step run events ( #141 )
...
* feat: pubsub for clients, more qol stuff
* fix: generate sqlc files
* chore: linting and comments
2024-02-02 12:52:34 -05:00
d63b66a837
feat: concurrency groups ( #135 )
...
* first pass at moving controllers around
* feat: concurrency limits for strategy CANCEL_IN_PROGRESS
* fix: linting
* chore: bump python sdk version
2024-01-30 00:00:28 -05:00
14a971bfd7
feat: improve versioning to use checksums ( #129 )
...
* feat: improve versioning to use checksums
* fix: remove json ignore for scheduled triggers
* update migration with random strings for checksum
2024-01-26 20:47:15 -05:00
78685d0098
feat(security): multiple encryption options, API tokens, easier setup ( #125 )
...
* (wip) encryption
* feat: api tokens
* chore: add api token generation command
* fix: e2e tests
* chore: set timeout for e2e job
* fix: e2e tests, remove client-side certs
* chore: address PR review comments
* fix: token tests
* chore: address review comments and fix tests
2024-01-26 15:38:36 -05:00
0c94f0d933
feat: python client ( #109 )
...
* (wip) python SDK
* feat: python client, initial version finished
* fix: add curl to migration dockerfile
* add insecure option for grpc
* create docs and publishing workflow
* delete old hatchet folder
2024-01-20 09:18:25 -05:00
52fde1e704
feat: dag-style execution ( #108 )
...
* feat: dag-style execution
* docs: update to reflect new context
* ensure no cycles
* remove example cycle
* linting
* lint and small fixes
* update deferred rollback
* last rollback handling
* unset max issues
* fix requeue edge case
2024-01-16 11:31:24 -05:00
752d5b0ab7
feat: support passing inputs to scheduled workflows ( #104 )
...
* fix: usage of RegisterAction
* make registered actions callable
* chore: update yaml example
* docs: register action documented
* feat: support input to scheduled workflows
* add worker
* add client
* docs: add input to schedule workflow
* chore: generate with updated protoc
* chore: sqlc generate
2024-01-12 00:27:34 -05:00
ac0c4e934a
fix: rabbitmq concurrent processing ( #92 )
2024-01-09 21:15:19 -05:00
fe76c724d1
feat: add ticker reassignment to engine ( #86 )
2024-01-08 14:11:30 -05:00
62445dc37f
feat: support one-time scheduled workflows ( #84 )
...
* feat: support one-time scheduled workflows
* refactor: move schedule out of workflow trigger def
* docs: add scheduling workflows section
* docs: update creating workflow
* only cancel schedules that are in the future
2024-01-08 10:03:32 -05:00
4ca785ba61
feat: add scheduling timeout (non-configurable) ( #24 )
2023-12-27 14:40:24 -05:00
366c79441d
first commit
2023-12-15 13:08:04 -05:00
abelanger5