Commit Graph

167 Commits

Author SHA1 Message Date
Matt Kaye 7781200123 Hotfix: Pydantic validation of actionId (#1525)
* fix: pydantic validation of action

* chore: version
2025-04-10 12:25:39 -04:00
Matt Kaye 771b733e10 Feat: Add Parent Task External ID to workflow run details, Fix input type marshalling to null (#1505)
* feat: add parent task external id to api for workflow run

* feat: wire up parent ids

* feat: custom go type for non-null json

* feat: gen

* feat: send action ids back over the api

* fix: lint
2025-04-07 23:40:13 -04:00
abelanger5 b6d077f96d feat: show concurrency queue counts in the UI (#1495)
* feat: show concurrency queue counts in the UI

* fix: parent concurrency queues
2025-04-04 12:22:14 -04:00
abelanger5 113fa98b55 fix: UI small improvements (#1430)
* fix: UI small improvements

* fix: build
2025-03-27 15:21:48 -07:00
Matt Kaye 5c6e8ee9a7 fix: adding a couple missing fields for the API (#1383) 2025-03-24 10:37:39 -04:00
abelanger5 c54bf9266c feat(v1): tenant limits (#1388)
* feat(v1): tenant limits

* fix: migration

* fix: kill metered cache
2025-03-23 19:03:55 -07:00
abelanger5 00c4bbff09 feat(v1): new gRPC API endpoints (#1367)
* wip: api contracts

* feat: implement put workflow version endpoint

* add support for match existing data, get scaffolding in place for additional triggers

* create additional matches

* feat: durable sleep, user event matching

* update protos

* fix: working poc of user events, durable sleep

* add migration

* fix: migration column

* feat: durable event listener

* fix: skip overrides

* fix: input -> output
2025-03-23 18:58:20 -07:00
Gabe Ruttner 3670b94fc4 Feat v1 UI tweaks (#1344)
* fix: drop uncached loader

* feat: upgrade modal

* add beta

* hacky feature flag

* fix: build

* refetch interval

* 5s

* stop flashing on load

* lint

* fix: map

* fix: last redir

* nil check

* small styling and wording things, change default canUpgrade -> true

* switch link to github discussion

---------

Co-authored-by: Alexander Belanger <alexander@hatchet.run>
2025-03-15 09:23:32 -04:00
abelanger5 afd853e223 v1 hotfixes (#1320)
* fix: when grpcInsecure is set to true with no internal client overrides, use TLS strategy=none

* fix: invites
2025-03-11 16:18:07 -04:00
abelanger5 1f2096313d feat: v1 engine (#1318) 2025-03-11 14:57:13 -04:00
Gabe Ruttner 30c1a979ac Fix: backoff bugs (#1248)
* fix: cancel backoff state

* fix: two paths for retry

* lint
2025-02-05 06:57:05 -08:00
Gabe Ruttner 0e91542d87 wip: backoff state (#1225)
* wip: backoff state

* fix: retry state and step run start condition

* fix: missing key

* fix: gen

* chore: squash migration

* chore: rm todos

* ops: upgrade proto
2025-01-28 19:16:12 +00:00
Gabe Ruttner 13024c09bd Feat canceled state (#1228)
* feat: add cancel state to event list

* ops: db conns

* feat: add cancelled status to wfr

* feat: mark cancelled workflow runs
2025-01-28 10:31:04 -08:00
Matt Kaye 52664d66fd fix: workflow run status is an empty string (#1208) 2025-01-23 12:49:31 -05:00
Sean Reilly c2248c08ab Fix security headers and emails (#1181)
* add a bunch of default headers

* add a check on the emails so we don't resend if we have a valid invite in future

* lets people invite for a new role

* add in some logging so we have more visibility on what is hapening here

* Add a limit to the number of pending invites a user can have. Add comments for the various headers
2025-01-17 15:06:26 -08:00
Sean Reilly 9e961ac196 Feature add version info (#1154)
* adding a /version endpoint for the engine and a /api/v1/version endpoint for the API

* make the security optional so we don't get redirected for having auth

* lint

* upgrade protoc to the latest available version on brew

* use useQuery and clean up html
2025-01-06 10:50:04 -08:00
abelanger5 dcb67a1dac feat: postgres-backed message queue (#1119) 2024-12-18 09:00:54 -05:00
Sean Reilly 53d5347ef4 Sean monitioring manual probes (#1128)
* add a monitoring probe

* add in untracked files

* don't need the security thing

* revert this change

* go generate

* remove the namespace stuff

* cleanup

* clean up workflow afterwards, turn off logs for client and worker wait for tasks to finish so we don't cause errors

* clean up workflow afterwards, turn off logs for client and worker wait for tasks to finish so we don't cause errors

* extend the delay a little

* use the tenant from the authed token

* remove static field

* PR feedback add polling for workflow state before deletion, check the step runs finish in the right order

* PR review comments

* tweak the timeout so we don't see errors in dev and fix description

* fix the open api build

* modify return type, return if we delete or do it 10 times, 10 second timeout for deletion attempts

---------

Co-authored-by: Sean Reilly <sean@hatchet.run>
2024-12-17 14:37:30 -08:00
Sean Reilly cbc2526c0b add a monitoring probe (#1108)
* add a monitoring probe

---------

Co-authored-by: Sean Reilly <sean@hatchet.run>
2024-12-17 15:55:50 -05:00
Sean Reilly 279f68c3a4 add pagination for workflows (#1109)
* add some more detail to the list worklflow API endpoint and add pagination and a search function to the frontend that searches the workflow names

* switch

* remove the column filter stuff

* cleanup

* lint and clean up query

* fix linter warning

---------

Co-authored-by: Sean Reilly <sean@hatchet.run>
2024-12-12 19:33:43 +00:00
abelanger5 b0c6c7cd46 feat(go-sdk): cron and schedules API, minor fixes (#1083)
* feat(go-sdk): cron and schedules API, minor fixes

* try to improve code block and docs

* revert pre-commit

* fix: generate

* fix: put overflow in right place

* remove branch specs
2024-12-04 21:18:05 +00:00
abelanger5 f821379997 fix: remove deprecated swagger-cli (#1080)
* fix: remove deprecated swagger-cli

* fix: use node:18-alpine
2024-12-03 16:01:46 +00:00
Gabe Ruttner 23201665a6 Feat: dynamic crons fixes (#1073) 2024-11-26 10:47:30 -05:00
Gabe Ruttner 574eb0b67e feat: dynamic crons (#1000)
* wip: stub schedule page

* wip: stub list

* fix: 2025 bug...

* feat: wip cron list

* feat: addl meta

* feat: expose metadata column

* feat: sort and created at

* cron to recurring

* scheduled: with statuses

* fix: links

* feat: expose schedule ids

* feat: delete run

* fix: remove search

* feat: filterable scheduled

* fix: remove broken features

* chore: lint

* rm metadata for now

* chore: lint

* chore: recurring to cron job

* fix: review comments

* fix: populator

* wip cron changes

* fix: ids are helpful

* fix: populator

* wip

* wip: create crons, stub scheduled

* wip: create schedule

* wip add trigger buttons to all the pages

* wip: reusable trigger form

* fix: hash

* fixes: cron bugs

* fixes: cron sort

* fix: out of order migrations

* fix: add internalRetryCount

* feat: api things survive version transitions

* feat: table things

* feat: delete disabled for non api

* feat: prevent delete non api

* feat: filters

* require cron name for api

* default name

* fix: migrations

* frontend improvements and migrations

* fix: pagination

---------

Co-authored-by: Alexander Belanger <alexander@hatchet.run>
2024-11-21 16:18:24 -05:00
abelanger5 197bdd1f88 feat: exponential backoff (#1062)
* initial migration

* feat: exp backoff, fix linting

* fix utc issue and cleanup
2024-11-21 13:39:02 -05:00
Alexander Belanger 5b59af076e fix: cancellation status propagation and minimap view 2024-11-07 11:13:14 -05:00
Gabe Ruttner abdd81c1eb fix: orderby (#1008) 2024-11-01 08:48:09 -04:00
Sean Reilly b456382429 add multiple rate limiter in grpc using a token bucket (#984)
* add multiple rate limiter in grpc using a token bucket

* PR feedback

* add in client retry for go client

* update test files

* remove log line only retry on ResourceExhausted and Unavailable

* add some concurrency limits so we don't swamp ourselves

* add some logging for when we are getting backed up

* lets not queue up when we are too full to prevent OOM problems

* fix spelling

* add config options for maximum concurrent and how long to wait for flush , let the wait for flush setting be used as back pressure and a signal to writers that we are slowing up

* lots of changes to buffering

* fix data race

* add some comments explaing how this works, change errors to be ResourceExhausted now that we have client retry and limit how many gofuncs we can create on cleanup and wait for them to finish before we exit

* hooking up the config values so they go to the right place

* Update config.go to default to 1 ms waitForFlush

* disable grpc_retry for client streams

* explicitly set the limit if it is 0

* weirdness because we were using an older version of the lib

---------

Co-authored-by: Sean Reilly <sean@hatchet.run>
Co-authored-by: Alexander Belanger <alexander@hatchet.run>
2024-11-01 11:48:23 +00:00
Gabe Ruttner 44addbb47e Feat scheduled improvements (#992)
* wip: stub schedule page

* wip: stub list

* fix: 2025 bug...

* feat: wip cron list

* feat: addl meta

* feat: expose metadata column

* feat: sort and created at

* cron to recurring

* scheduled: with statuses

* fix: links

* feat: expose schedule ids

* feat: delete run

* fix: remove search

* feat: filterable scheduled

* fix: remove broken features

* chore: lint

* rm metadata for now

* chore: lint

* chore: recurring to cron job

* fix: review comments

* fix: populator
2024-11-01 07:16:20 -04:00
Gabe Ruttner 2c89abbb31 feat:add addl meta to run details (#996)
* feat:add addl meta to run details

* chore: debug

* feat: filter on click

* fix: build

* fix: build
2024-10-29 14:19:38 +00:00
Gabe Ruttner 4932e7f863 Feat sdk runtime (#942)
* feat: runtime signature

* feat: add sdk runtime to worker model

* feat: post runtime

* feat: expose sdk version on worker

* feat: go inf

* chore: gen

* chore: migrations and generation

* fix: simpler runtime

* feat: hatchet sdk ver

* fix: rm debug line
2024-10-28 13:47:12 -07:00
Gabe Ruttner 7cd08077d5 feat: improved sdk ack (#931)
* feat: add step run event reasons

* feat: ack

* fix: remove rejected reason

* fix: merge

* fix: correct buffer

* fix: consistent message

* chore: rm todo
2024-10-15 15:52:42 +00:00
Sean Reilly 29721cd1f0 Feat bulk workflows (#940)
Adds support for inserting workflows in bulk via the API and an optional buffered insert on the engine.
2024-10-14 15:35:29 -04:00
abelanger5 b4c861d7a1 patch: release semaphore slots before jobs controller (#927)
* fix: don't need acks on queue checks

* patch: release semaphores early

* proper list on high queue depth

* fix: don't release on started
2024-10-02 11:36:05 -04:00
Gabe Ruttner 7d7e43d4e1 feat: pauseable workflows (#879)
* feat: pause workflow state

* feat: dont run paused workflows

* feat: skipped paused

* implement unpaused behavior for workflow runs

* fix: frontend

* fix: more frontend

* fix: imports

---------

Co-authored-by: Alexander Belanger <alexander@hatchet.run>
2024-09-29 10:58:10 -04:00
abelanger5 925b2654c8 feat: workflow run metrics view (#912)
* feat: add callbacks for workflow run completed

* add tenant id to resolve row

* add finishedBefore, finishedAfter to workflow runs query

* add more callbacks

* feat: tenant ids and loggers in callback

* feat: workflow run metrics frontend

* fix: frontend build
2024-09-27 07:38:15 -04:00
abelanger5 c24d65b76e add timeout and context to queue metrics (#914) 2024-09-27 07:10:25 -04:00
abelanger5 a1a10b4073 feat: dynamic rate limits (#904)
* wip: step run expressions on rate limits

* feat: dynamic rate limits

* chore: v0.47.0

* chore: address changes from PR review

* fix: improved error handling

* address pr review

* better error messages for step run cels, remove debug logs

* fix: hash

---------

Co-authored-by: gabriel ruttner <gabriel.ruttner@gmail.com>
2024-09-26 22:00:34 +00:00
abelanger5 840e590312 fix: frontend improvements (#905)
* fix: set time range properly on reload

* fix: small button to show queue counts for now
2024-09-24 19:08:41 -04:00
Gabe Ruttner f98d3277b7 fix: trunc large payloads (#903)
* fix: trunc large payloads

* lets send the stepRuns and steps with output back on the WorkflowRunGet

* fix: times

* fix: rm unsafe

* rename to GetStepRunsForJobRunsWithOutput so we know we might potentially be getting a very large result set

---------

Co-authored-by: Sean Reilly <sean@hatchet.run>
2024-09-24 22:52:00 +00:00
Sean Reilly 5811929928 feat: bulk inserts of events (#887)
* progress commit of bulk inserts

* in_flight: Add changes to metering finish the bulk insert

* remove an attempt to overide enforce limits

* merge in PR fixes

* update docs to add in an additional section in the User guide to describe pushing single events and pushing multiple events

* run lint fix

---------

Co-authored-by: Sean Reilly <sean@hatchet.run>
2024-09-23 09:19:39 -07:00
abelanger5 baf13bd577 fix: duration int -> bigint (#902) 2024-09-23 08:30:16 -07:00
Sean Reilly 15c50f46b5 Partial PR - need to generate SDK - Add endpoint to get the total free worker slots for a worker and the … (#857)
* Add endpoint to get the total free worker slots for a worker and the max runs

* update to use WorkerSempahoreCount instead of checking stepRunId

* modify the query for the new table and change the interface

* bump golangci-lint make changes to name of returned data

* revert the simple example

---------

Co-authored-by: Sean Reilly <sean@hatchet.run>
2024-09-19 10:11:16 -07:00
abelanger5 d23e5d9963 feat: expression-based concurrency keys (#889)
* feat: expression-based concurrency keys

* fix: build

* fix: typos

* fix: gen

* fix: migration

* fix: remove print statements

* fix: reassignment bugs, retries on closed transport, pr review
2024-09-19 10:32:22 -04:00
Gabe Ruttner af9ed49f1e fix: events list view (#878)
* fix: filter by event id

* fix: run count

* feat: filter by id api

* feat: filter by Event Id

* chore: default page is runs

* feat: cancel event runs

---------

Co-authored-by: Alexander Belanger <alexander@hatchet.run>
2024-09-16 16:46:31 +00:00
Gabe Ruttner c64c62f66a feat: improved workflow run details page (#821)
* wip: rip prisma

* wip

* wip

* fix: lint

* wip

* wip

* gen

* wip

* wip

* fix trigger

* hide overview

* revert db changes

* feat: wrap up frontend changes and perf

* chore: generate

* chore: frontend build

* fix: workflow transformer

* fix: avoid race conditions on simultaneous parent completions

* fix: 2025 started

* feat: toast for replay/cancel

* fix: toast

---------

Co-authored-by: Alexander Belanger <alexander@hatchet.run>
2024-09-16 15:39:49 +00:00
abelanger5 bed2cb559a fix: add back sem slots, without row contention (#868)
* fix: add back sem slots, without row contention

* fix: serialize queue step runs to prevent dirty reads

* remove serializable for now

* statement timeouts on create workflow run

* statement timeout for reassign

* proper migration + cleanup

* remove old tables and code

* fix: worker slot state

* remove last unused table from workers
2024-09-11 20:47:49 +00:00
Gabe Ruttner 460438ab22 fix: tag (#862) 2024-09-10 09:31:39 -07:00
abelanger5 9efcebe6af fix: better logic for multiple restricted domains (#860) 2024-09-10 12:07:55 -04:00
abelanger5 e59ce100e3 fix: cleanup worker assign events + limit semaphore query (#854) 2024-09-09 11:11:44 -04:00