Commit Graph

31 Commits

Author SHA1 Message Date
Luca Steeb
713b8c95c6 fix: eliminate remaining race conditions (#220) 2024-03-02 23:47:50 +07:00
Luca Steeb
9b68115fb5 refactor: cleanup functions in api + worker (#192) 2024-03-02 00:37:02 +07:00
Luca Steeb
ae4841031b feat(engine): standalone tests and engine teardown (#172) 2024-02-28 00:15:25 +07:00
abelanger5
6ea38a99f2 feat: support maxRuns parameter on workers (#195)
* feat: round robin queueing

* feat: configurable max runs per worker

* fix: address PR review

* docs for max runs and group round robin
2024-02-26 00:48:46 -05:00
abelanger5
2d625fec81 feat: round robin queueing (#194) 2024-02-26 00:16:40 -05:00
abelanger5
df3f540748 feat: add retries to the engine and SDKs (#171)
This PR adds support for retrying failed step runs against the engine and SDKs. This was tested up to 30 retries per step run, with both failure and success at the 30th step run. Each SDK now has a `retries` configurable param for steps when declaring a workflow.
2024-02-16 13:00:22 -05:00
Luca Steeb
00111d823c test(load): add load tests CLI & e2e tests (#157) 2024-02-16 23:47:34 +07:00
abelanger5
c2ea09f375 feat: step reruns from the dashboard (#143) 2024-02-03 01:26:09 -05:00
abelanger5
82d7995343 feat: manual triggers and give clients a hook into step run events (#141)
* feat: pubsub for clients, more qol stuff

* fix: generate sqlc files

* chore: linting and comments
2024-02-02 12:52:34 -05:00
abelanger5
aed11c3958 feat: workflow visualization and qol improvements (#140)
* feat: workflow visualization and qol improvements

* fix: npm build
2024-02-02 01:35:05 -05:00
abelanger5
d63b66a837 feat: concurrency groups (#135)
* first pass at moving controllers around

* feat: concurrency limits for strategy CANCEL_IN_PROGRESS

* fix: linting

* chore: bump python sdk version
2024-01-30 00:00:28 -05:00
abelanger5
78685d0098 feat(security): multiple encryption options, API tokens, easier setup (#125)
* (wip) encryption

* feat: api tokens

* chore: add api token generation command

* fix: e2e tests

* chore: set timeout for e2e job

* fix: e2e tests, remove client-side certs

* chore: address PR review comments

* fix: token tests

* chore: address review comments and fix tests
2024-01-26 15:38:36 -05:00
Luca Steeb
8b379ee9d1 feat(events): add workflow filter (#114)
* feat(events): add workflow filter

* cast to uuid

---------

Co-authored-by: Alexander Belanger <belanger@sas.upenn.edu>
2024-01-21 22:33:58 -05:00
abelanger5
52fde1e704 feat: dag-style execution (#108)
* feat: dag-style execution

* docs: update to reflect new context

* ensure no cycles

* remove example cycle

* linting

* lint and small fixes

* update deferred rollback

* last rollback handling

* unset max issues

* fix requeue edge case
2024-01-16 11:31:24 -05:00
abelanger5
752d5b0ab7 feat: support passing inputs to scheduled workflows (#104)
* fix: usage of RegisterAction

* make registered actions callable

* chore: update yaml example

* docs: register action documented

* feat: support input to scheduled workflows

* add worker

* add client

* docs: add input to schedule workflow

* chore: generate with updated protoc

* chore: sqlc generate
2024-01-12 00:27:34 -05:00
abelanger5
1f7baacb94 Fix usage of RegisterAction and update docs (#103)
* fix: usage of RegisterAction

* make registered actions callable

* chore: update yaml example

* docs: register action documented
2024-01-11 18:42:20 -05:00
Luca Steeb
6d8c7ab073 feat(test): introduce basic e2e tests (#97) 2024-01-11 13:36:15 -05:00
abelanger5
7011416cb7 fix: simple example (#95) 2024-01-10 11:05:54 -05:00
abelanger5
ac0c4e934a fix: rabbitmq concurrent processing (#92) 2024-01-09 21:15:19 -05:00
abelanger5
fe76c724d1 feat: add ticker reassignment to engine (#86) 2024-01-08 14:11:30 -05:00
abelanger5
62445dc37f feat: support one-time scheduled workflows (#84)
* feat: support one-time scheduled workflows

* refactor: move schedule out of workflow trigger def

* docs: add scheduling workflows section

* docs: update creating workflow

* only cancel schedules that are in the future
2024-01-08 10:03:32 -05:00
abelanger5
7d1cf5400f feat: add middleware to worker and services (#82)
* feat: add middleware support

* docs: add middleware to docs

* chore: remove commented code

* address review comments
2024-01-05 08:35:19 -05:00
abelanger5
65c165caf6 fix: slack example and rendering bug (#79) 2024-01-04 08:28:49 -05:00
abelanger5
ff30027ae7 feat: update go-sdk workflow syntax to make it less verbose (#78)
* feat: update go-sdk workflow syntax to make it less verbose

* docs: make triggers and definitions more clear
2024-01-03 17:14:12 -05:00
abelanger5
76d38d1af9 fix: allow rendering step runs with - in name (fixes #69) (#74)
* fix: allow rendering step runs with - in name (fixes #69)

* remove debug lines

* include worker fix
2024-01-03 14:48:18 -05:00
abelanger5
49693a5ad6 fix: duplicate workflows showing up in workflow list (#59)
* fix: duplicate workflows showing up in workflow list

* update formatting

* fix: action map should have proper action id for fn
2024-01-03 09:20:21 -05:00
abelanger5
373b9f4456 feat: add sentry integration to workers (#46)
* feat: add sentry integration to worker

* docs: add sentry alerter docs for worker
2024-01-02 09:56:15 -05:00
abelanger5
5937b9fd98 feat(go-sdk): add ability to create workflows from the Go SDK more easily, quickstart improvements (#30)
* feat: add initial docs site

* feat: allow workflows to be defined from go sdk

* fix release action

* chore: remove server dependencies from client

* fix: use correct certificate for server

* chore: add port and bind address to grpc config

* fix: add env for grpc config

* fix: nil pointer when output is null

* chore: support variation in output args

* fix unresolve merge conflict

* fix: quickstart improvements

* temp remove database url

* fix: action id not required for event

* fix: actionid validation for events

* Remove deleted files
2024-01-02 09:02:53 -05:00
abelanger5
4ca785ba61 feat: add scheduling timeout (non-configurable) (#24) 2023-12-27 14:40:24 -05:00
abelanger5
cb07d1211a feat: improve debugging experience of workflow runs (#5)
* feat: improve debugging experience of workflow runs

Makes steps in a workflow run expandable to view input, output and error information.

Also adds the triggering cron or event to the top of the workflow run.

* chore: remove commented, unused code

* chore: address PR comments
2023-12-19 08:23:21 -05:00
Alexander Belanger
366c79441d first commit 2023-12-15 13:08:04 -05:00