abelanger5
baf13bd577
fix: duration int -> bigint ( #902 )
2024-09-23 08:30:16 -07:00
abelanger5
d23e5d9963
feat: expression-based concurrency keys ( #889 )
...
* feat: expression-based concurrency keys
* fix: build
* fix: typos
* fix: gen
* fix: migration
* fix: remove print statements
* fix: reassignment bugs, retries on closed transport, pr review
2024-09-19 10:32:22 -04:00
Gabe Ruttner
c64c62f66a
feat: improved workflow run details page ( #821 )
...
* wip: rip prisma
* wip
* wip
* fix: lint
* wip
* wip
* gen
* wip
* wip
* fix trigger
* hide overview
* revert db changes
* feat: wrap up frontend changes and perf
* chore: generate
* chore: frontend build
* fix: workflow transformer
* fix: avoid race conditions on simultaneous parent completions
* fix: 2025 started
* feat: toast for replay/cancel
* fix: toast
---------
Co-authored-by: Alexander Belanger <alexander@hatchet.run >
2024-09-16 15:39:49 +00:00
abelanger5
bed2cb559a
fix: add back sem slots, without row contention ( #868 )
...
* fix: add back sem slots, without row contention
* fix: serialize queue step runs to prevent dirty reads
* remove serializable for now
* statement timeouts on create workflow run
* statement timeout for reassign
* proper migration + cleanup
* remove old tables and code
* fix: worker slot state
* remove last unused table from workers
2024-09-11 20:47:49 +00:00
abelanger5
f4c5cd973e
feat: more efficient step run timeouts ( #863 )
2024-09-10 18:23:11 -04:00
abelanger5
478c897035
fix: proper migration to counts, startable step runs query ( #850 )
2024-09-08 12:48:51 -04:00
abelanger5
891514b461
feat: queue v4 ( #842 )
...
* wip: v4 of queue
* fix: correct query for updating counts
* tmp: save migration files
* feat: wrap up initial queue
* fix compilation
* fix: reassigns
2024-09-06 16:12:22 -04:00
abelanger5
b5014f6b3d
chore: more visibility and debug lines for queues ( #836 )
...
* chore: more visibility and debug options for queues
* better debug lines on queue repo
* don't log so much in load test
2024-08-29 14:49:24 -04:00
Gabe Ruttner
526e7ef308
feat: expose priority queue ( #814 )
...
* feat: workflow default priority
* feat: write priority on run
* feat: propagate to queue
* chore: squash migrations
* chore: generate
2024-08-26 14:11:28 -04:00
Gabe Ruttner
53be615d5f
Enhancement webhook usability ( #807 )
...
* feat: secret copier
* feat: improved form
* fix: quotes
* wip: improved flow
* feat: health check logging
* fix: page design
* fix: hard delete, no upsert
* fix: reset modal state
* fix: empty text
* fix: worker state
* fix: update only token
* fix: dont delete name
* fix: logs component
* fix: sort order
* chore: build
* fix: webhook worker cleanup
* chore: squash migrations
* Update api-contracts/openapi/paths/webhook-worker/webhook-worker.yaml
Co-authored-by: abelanger5 <belanger@sas.upenn.edu >
* chore: rename
* fix: wrong query
---------
Co-authored-by: abelanger5 <belanger@sas.upenn.edu >
2024-08-23 10:09:09 -04:00
Gabe Ruttner
9bea55438a
Fix webhook healthcheck race ( #797 )
...
* fix: race
* fix: partition no rows
* chore: move to workers tab
* feat: redirect empty worker path to all
* chore: add worker type and webhook id
* fix: upsert webhook worker
* fix: update by webhookId
* fix: only stub on create
* feat: url on worker
* chore: migration version
* fix: move
* fix: upsert
* fix: upert
* chore: fix migration
* fix: migrations
* chore: generate
2024-08-21 19:23:24 +00:00
Gabe Ruttner
651be542c3
fix: migration state ( #800 )
...
* fix: migration state
* almost there...
* fix: hack for constraints
* chore: lint
2024-08-21 17:07:00 +00:00
abelanger5
67357cfa64
fix: add correct indexes for get group key runs, improve queries ( #786 )
...
* fix: add correct indexes for get group key runs
* chore: generate
* fix: hash
2024-08-20 09:31:30 -04:00
abelanger5
d4d3512a28
fix: add constraint to priority ( #777 )
...
* fix: add constraint to priority
* fix: new index
* fix: drop old index
* fix: typo
2024-08-12 17:29:57 +00:00
Gabe Ruttner
4ea4712d4d
refactor: performance and throughput ( #756 )
...
Refactors the queueing logic to be fairly balanced between actions, with each action backed as a separate FIFO queue. Also adds support for priority queueing and custom queues, though those aren't exposed on the API layer yet. Improves throughput to be > 5000 tasks/second on a single queue.
---------
Co-authored-by: Alexander Belanger <alexander@hatchet.run >
2024-08-12 14:38:47 +00:00
abelanger5
a245151d91
feat: add workflow kind to workflow versions ( #750 )
...
* feat: support workflow kinds
* chore: generate
2024-07-29 12:07:34 -07:00
abelanger5
1ea4dfc5de
feat: deduplicated enqueue ( #735 )
...
* wip
* wip: functional query
* feat: expose affinity config
* feat: add weight to proto
* feat: upsert affinity state on worker start
* fix: linting
* feat: add upsert proto
* feat: upsert handler
* feat: revise model
* fix: labels
* feat: functional desired worker
* wip: ui
* feat: add state to step run events
* fix: filter empty keys
* fix: labels as badges
* feat: empty state and descriptive text
* chore: add todo
* chore: whitespace
* chore: cleanup
* chore: cleanup
* chore: fix hash
* chore: squash migrations
* fix: fair worker assignment
* fix: remaining slots on valid desired workers
* wip: sticky
* fix: count slots
* chore: rm log line
* feat: expose sticky config
* wip: sticky dag
* feat: expose desired worker id to trigger
* feat: trigger on desired worker
* feat: typescript docs
* feat: sticky python
* feat: py sticky children
* wip: py affinity
* serverless note
* feat: complete python examples
* linting
* feat: deduplicated enqueue
* fix: address changes from PR review
* chore: generate
---------
Co-authored-by: gabriel ruttner <gabriel.ruttner@gmail.com >
2024-07-26 16:47:46 +00:00
Gabe Ruttner
ee68786d69
feat: sticky workers ( #695 )
...
* wip
* wip: functional query
* feat: expose affinity config
* feat: add weight to proto
* feat: upsert affinity state on worker start
* fix: linting
* feat: add upsert proto
* feat: upsert handler
* feat: revise model
* fix: labels
* feat: functional desired worker
* wip: ui
* feat: add state to step run events
* fix: filter empty keys
* fix: labels as badges
* feat: empty state and descriptive text
* chore: add todo
* chore: whitespace
* chore: cleanup
* chore: cleanup
* chore: fix hash
* chore: squash migrations
* fix: fair worker assignment
* fix: remaining slots on valid desired workers
* wip: sticky
* fix: count slots
* chore: rm log line
* feat: expose sticky config
* wip: sticky dag
* feat: expose desired worker id to trigger
* feat: trigger on desired worker
* feat: typescript docs
* feat: sticky python
* feat: py sticky children
* wip: py affinity
* serverless note
* feat: complete python examples
* linting
* fix: doc link
* chore: rm debug log
* fix: simplify list labels
* fix: typo
2024-07-22 17:20:23 -04:00
Gabe Ruttner
b7cec9ec53
feat: soft delete ( #717 )
...
* feat: soft delete workflows and versions
* feat: filter soft deletes wf and wfr
* feat: filter events and step runs
* fix: query
* fix: query
* chore: generate
* wip
* chore: squash migrations
* chore: separate retention into new service
* feat: regularly clean up
* chore: migrations
* fix: tests
* fix: queries
* fix: ambiguous
* fix: refs
* fix: ambiguous id
* fix: remove update from
* fix: soft delete
* fix: cleanup retention scheduler
* fix: has more query
* chore: gen
* fix: query
* fix: table
2024-07-18 09:06:05 -04:00
abelanger5
5d87f380ef
feat: managed worker pools ( #725 )
...
* change api extension spec to register custom populators
* fix: support only bearer auth
* fix: correct authn logic
* fix: indexes on workflow runs, events
* feat: managed worker pools
* chore: lint fix
* hide workers view when not enabled
* support internal api tokens, minor improvements
* fix: actually write internal
* fix breaking changes
* don't allow revoking internal tokens
* fix: linting and remove metrics view
* fix: token
* address review and add feat flags
2024-07-16 13:33:46 +00:00
Gabe Ruttner
38a4ce647d
feat: improved workflow runs list ( #723 )
...
* feat: add duration to wfr
* chore: generate
* feat: sort order
* feat: add column visibility toggle
* chore: gen
2024-07-15 18:05:57 -04:00
abelanger5
8f8f3ad287
fix: reduce max throughput of requeue ( #713 )
...
* fix: reduce max throughput of requeue
* fix: reassign query
* fix: move step run timeout to partition model
* fix: partitioning queries and index
* better logs on requeue
* fix: inactive rebalance and get step run for engine query
* fix: correct inactive queries
2024-07-12 14:03:55 -04:00
abelanger5
a96738539e
fix: simplify reassign query ( #712 )
...
* fix: simplify reassign query
* chore: lint and rehash
2024-07-10 19:38:02 -04:00
abelanger5
fcb3a88078
fix: improved performance of reassign/requeue ( #711 )
...
* fix: improved performance of reassign/requeue
* chore: generate
2024-07-10 17:04:16 -04:00
Gabe Ruttner
1e20bf946a
fix: improved assign, reassign, and requeue ( #702 )
...
* fix: improved queries
* fix: 1s timeout
* fix: indexes
* fix: increase timeout to 4s
* fix: migration
* merge in db changes
* chore: squash migrations
* chore: re-hash
* chore: remove comment
* chore: rm unused query
* fix: state
* fix: check valid workers before commit
* fix: query
* chore: gen
2024-07-10 12:45:08 -04:00
Gabe Ruttner
16d52277f1
fix: steps to timeout ( #707 )
...
* fix: sync prisma state to atlas state
* chore: gen
* chore: gen again
* fix: rm ticker indexes
* chore: gen
* fix: improved query and indexes
2024-07-10 11:12:51 -04:00
Gabe Ruttner
e420f4960f
fix: sync prisma state to atlas state ( #706 )
...
* fix: sync prisma state to atlas state
* chore: gen
* chore: gen again
* fix: rm ticker indexes
* chore: gen
2024-07-10 07:55:56 -07:00
Gabe Ruttner
f5bdc409d4
fix: reverse migration ( #703 )
...
* fix: reverse migration
* chore: hash
2024-07-10 04:03:25 -07:00
Gabe Ruttner
ccd38f0506
fix: step run queries ( #696 )
...
* fix: rm unused count
* fix: indexes
* fix: limit PollStepRuns limit
* fix: create concurrently if not exists
* chore: gen
* fix: txmode
* fix: dont run expired crons
* revert: cron
* chore: rename migration
* chore: gen
2024-07-06 20:12:28 -04:00
abelanger5
f36e66cd28
feat: configurable data retention period ( #693 )
...
* feat: data retention for tenants
* chore: generate and docs
* chore: lint
2024-07-06 14:31:12 +00:00
abelanger5
0a2a26f59a
fix: indexes on workflow runs, events ( #688 )
2024-07-03 18:15:18 -04:00
Gabe Ruttner
461eda194a
feat: worker paused state ( #677 )
...
* feat: worker paused state
* feat: ui
* fix: comment
2024-07-01 18:44:12 +00:00
abelanger5
f2c6bc1f44
feat: tenant partitioning ( #649 )
...
* feat: tenant partitioning
* fix: rebalance inactive partitions, split into separate partitioner
* fix: shutdown partitioner scheduler properly
* update config options
* fix: config options linting
2024-06-26 21:06:51 +00:00
Gabe Ruttner
a8d42819ea
feat: check security service ( #639 )
...
* feat: check security service
* feat: propegate version
* feat: with ident
* fix: lint
* chore: generate
* fix: change domain
* fix: panic recover
* fix: migrations
* fix: hash
* fix: dont check in tests
2024-06-26 16:26:29 -04:00
Luca Steeb
1490d88954
feat: webhook workers ( #542 )
...
Adds serverless support via the concept of webhook workers. Allows any webhook to be registered as a serverless endpoint for executing a step.
2024-06-25 17:06:43 -04:00
Gabe Ruttner
697757879f
feat: billing ( #624 )
...
* feat: init lago client
* feat: billable meter
* feat: db persistence
* wip: expose sub
* feat: rename page
* wip: billing section
* wip: lago integration
* feat: separate plan and period
* wip: webhook
* feat: improve empty state
* feat: update limits on plan changes
* feat: can change plans
* feat: change plan loading state
* feat: yearly filter
* feat: billing clarification
* fix: treatment
* feat: filter plans
* feat: prevent non-owner from changing plan
* fix: loading state
* fix: jit portal link
* fix: rm import
* fix: build errors
* fix: default to free
* fix: wrong files
* fix: select or insert customer
* fix: note
* feat: upgrade dependent on payment method state
* fix: dedupe
* chore: remove github-app from core
* chore: port to cloud
* chore: port to cloud
* chore: port to cloud
* chore: port to cloud
* chore: port to cloud
* add new components, repository callbacks
* chore: rm unused packages
* chore: fix generation
* chore: gen
* fix: cloud api references
* debug
* debug
* fix: actually set plans
* chore: rm debug
* fix: build
* feat: callbacks
* fix: add generated code
* chore: group cloud components
* chore: group by feature
* feat: alert change
* feat: confirm
* fix: confirm modal
* fix: ui
* fix: remove arrears
* fix: open in same tab
* fix: wan alert
* fix: call the callback
* fix: callback obj
* fix: disable if no cloud meta
---------
Co-authored-by: Alexander Belanger <alexander@hatchet.run >
2024-06-25 13:57:16 -04:00
Gabe Ruttner
bbc4e58dd9
feat: limits ( #559 )
...
* feat: workflow run limits
* fix: resource exhausted 429
* feat: event limit
* feat: worker limit
* fix: sensible error
* fix: pb
* feat: expose limits api
* feat: default limits
* feat: add enable alert option
* feat: slack and email alerts
* fix: cron interval
* feat: make metered util
* wip: schedules and crons
* chore: squash migration
* fix: select or insert
* fix: remove unfinished meter
* chore: atlas migration
* fix: template format
* fix: shared ErrResourceExhausted
* feat: cache
* fix: limit can be nil
* fix: clarification
* fix: close meter ticker
* fix: friendly error for child workflows
2024-06-07 10:57:57 -07:00
abelanger5
b0b2e26952
feat: hatchet-lite ( #560 )
...
* feat: hatchet-lite mvp
* fix: init shadow db
* fix: install atlas
* fix: correct env
* fix: wait for db ready
* fix: remove name flag
* fix: add hatchet-lite to build
2024-06-06 14:03:53 -04:00
