* user provided priorities can only be 1,2,3
* sanitize
* check for retry counts
* update partition functions to include constraints
* do SQL migration afterwards
* revert sql changes
* feat: initial work wiring up priorities
* fix: add default to default prio in the db
* feat: wire priority through api on wf creation
* feat: extend python test
* feat: priority for scheduled workflows
* feat: wire priority through python api
* feat: more wiring priority through the api
* feat: I think it works?
* feat: e2e test for priority
* it works!
* feat: expand tests for default priorities
* feat: e2e scheduling test
* fix: skip broken test for now
* fix: lint
* feat: add priority columns to cron and schedule ref tables
* feat: update inserts to include prio
* feat: wire up more apis
* feat: more wiring
* feat: wire up more rest api fields
* chore: cruft
* fix: more wiring
* fix: lint
* chore: gen + wire up priorities
* fix: retries
* fix: try changing fixture scope
* chore: bump version again
* feat: send priority with action payload
* fix: generate script
* Feat priority ts (#1518)
* feat: initial work wiring up priorities
* fix: add default to default prio in the db
* feat: wire priority through api on wf creation
* feat: extend python test
* feat: priority for scheduled workflows
* feat: wire priority through python api
* feat: more wiring priority through the api
* feat: I think it works?
* feat: e2e test for priority
* it works!
* feat: expand tests for default priorities
* feat: e2e scheduling test
* chore: minor version for priority
* fix: skip broken test for now
* fix: lint
* feat: add priority columns to cron and schedule ref tables
* feat: update inserts to include prio
* feat: wire up more apis
* feat: more wiring
* feat: wire up more rest api fields
* chore: cruft
* fix: more wiring
* fix: lint
* chore: gen + wire up priorities
* fix: increase timeout
* fix: retries
* fix: try changing fixture scope
* chore: generate
* fix: set schedule priority
* feat: priority
* fix: move priority to wf
* release: 1.2.0
* rm log
* fix: import
* fix: add priority to step
---------
Co-authored-by: mrkaye97 <mrkaye97@gmail.com>
* fix: add dummy runs to priority test to prevent race conditions
* fix: non-breaking field
* fix: gen
* feat: initial pass at docs
* feat: priority in go sdk
* feat: initial work on go example
* fix: doc examples
* fix: proofread
* chore: version
* feat: go sdk
* fix: lint
* fix: declarations and add back RunAsChild
* fix: child workflows
* fix: namespace
* fix: faster child workflows
* fix: sticky
* add back run as child
---------
Co-authored-by: Gabe Ruttner <gabriel.ruttner@gmail.com>
Co-authored-by: Alexander Belanger <alexander@hatchet.run>
- Simplifies architecture for splitting engine services into different components. The three supported services are now `grpc-api`, `scheduler`, and `controllers`. The `grpc-api` service is the only one which needs to be exposed for workers. The other two can run as unexposed services.
- Fixes a set of bugs and race conditions in the `v2` scheduler
- Adds a `lastActive` time to the `Queue` table and includes a migration which sets this `lastActive` time for the most recent 24 hours of queues. Effectively this means that the max scheduling time in a queue is 24 hours.
- Rewrites the `ListWorkflowsForEvent` query to improve performance and select far fewer rows.
Refactors the queueing logic to be fairly balanced between actions, with each action backed as a separate FIFO queue. Also adds support for priority queueing and custom queues, though those aren't exposed on the API layer yet. Improves throughput to be > 5000 tasks/second on a single queue.
---------
Co-authored-by: Alexander Belanger <alexander@hatchet.run>
* feat: allow extending the api server
* chore: remove internal packages to pkg
* chore: update db_gen.go
* fix: expose auth
* fix: move logger to pkg
* fix: don't generate gitignore for prisma client
* fix: allow extensions to register their own api spec
* feat: expose pool on server config
* fix: nil pointer exception on empty opts
* fix: run.go file
* new api-contract for workflow run events
* feat: initial implementation for new subscribe listener
* fix: sync issues and send workflow runs immediately
* refactor: add context to all engine db queries, fix deadlocking query
* fix: use new ctx for deleting dispatcher and ticker
* add cancellation reasons
* fix: docs linting
---------
Co-authored-by: gabriel ruttner <gabriel.ruttner@gmail.com>
* refactor: separate api and engine repositories, change ticker logic
* fix: nil error blocks
* fix: run migration on load test
* fix: generate db package in load test
* fix: test.yml
* fix: add pnpm to load test
* fix: don't lock CTEs with columns that don't get updated
* fix: update heartbeat for worker every 4 seconds, not 5
* chore: remove dead code
* chore: update python sdk
* chore: add back telemetry attributes
This PR adds support for retrying failed step runs against the engine and SDKs. This was tested up to 30 retries per step run, with both failure and success at the 30th step run. Each SDK now has a `retries` configurable param for steps when declaring a workflow.
* feat: (wip) bundle rest API in typescript client
* chore: add generated files
* chore: pnpm lock file
* extract tenant id from token subject
* feat: add custom user data to context
* docs: generated with typedoc
* feat: dag-style execution
* docs: update to reflect new context
* ensure no cycles
* remove example cycle
* linting
* lint and small fixes
* update deferred rollback
* last rollback handling
* unset max issues
* fix requeue edge case
Mohammed Nafees