mirror/hatchet
1
0
Fork 0
mirror of https://github.com/hatchet-dev/hatchet.git synced 2026-02-20 07:09:13 -06:00
Code Issues Packages Projects Releases Wiki Activity
2,065 Commits 213 Branches 872 Tags
2dfdd9dd0cf1343a88f104f8a11af9608e8ddf8f
Commit Graph

83 Commits

Author SHA1 Message Date
Mohammed Nafees
57ad1af68d fix: deadlocks on trigger, olap prometheus background worker, otel improvements (#2475) 
* print error log temporarily

* casing

* only for create-monitoring-event

* rate limit iterator

* add a debugger

* remove rate limiter

* improve otel on trigger

* cache probability stuff

* track misses

* move down one ln

* default

* Fix: Pass tx down into payload retrieve (#2483)

* [Python] Feat: Dataclass Support (#2476)

* fix: prevent lifespan error from hanging worker

* fix: handle cleanup

* feat: dataclass outputs

* feat: dataclasses

* feat: incremental dataclass work

* feat: dataclass tests

* fix: lint

* fix: register wf

* fix: ugh

* chore: changelog

* fix: validation issue

* fix: none check

* fix: lint

* fix: error type

* chore: regenerate examples (#2477)

Co-authored-by: GitHub Action <action@github.com>

* feat: add health and metrics api on typescript sdk worker (#2457)

* feat: add health and metrics api on typescript sdk worker

add: prom-client to fetch metrics data
add: track health status of worker across different states

* refactor: keep prom-client as optional dependency

* refactor: remove async import of prom-client

* chore: update package version for ts sdk

* fix: lint

* fix: lint, const enum

---------

Co-authored-by: mrkaye97 <mrkaye97@gmail.com>

* Update frontend onboarding steps (#2478)

* Update frontend onboarding steps

* Update sidebar as well

* Fix Go SDK cron inputs (#2481)

* cron input in Go SDK

* add example

* fix: pass tx down to retrieve

* fix: attempt 2, another pool use

* fix: spans and debugging for task statuses

* attempted hotfix on olap statuses

* process tenants in parallel in prom worker

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Jishnu <jishnun789@gmail.com>
Co-authored-by: Sid Premkumar <sid.premkumar@gmail.com>
Co-authored-by: Mohammed Nafees <hello@mnafees.me>
Co-authored-by: Alexander Belanger <alexander@hatchet.run>

* move debugger package, clean up init

* remove probability factor logic

* remove debug

* fix: debugger instantiation

---------

Co-authored-by: Alexander Belanger <alexander@hatchet.run>
Co-authored-by: gabriel ruttner <gabriel.ruttner@gmail.com>
Co-authored-by: mrkaye97 <mrkaye97@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Jishnu <jishnun789@gmail.com>
Co-authored-by: Sid Premkumar <sid.premkumar@gmail.com>
 2025-11-04 09:05:44 +01:00
matt
d2cab4924a Fix: use SplitN instead of Split (#2336) 2025-09-24 15:26:38 -04:00
Gabe Ruttner
f59ebd6c47 feat: analytics events (#2171) 
* feat: analytics events

* review comments
 2025-08-22 05:41:17 -07:00
Mohammed Nafees
8a0e88ac48 [HAT-432] Enforce task priorities to be between 1 and 3 (#2110) 
* user provided priorities can only be 1,2,3

* sanitize

* check for retry counts

* update partition functions to include constraints

* do SQL migration afterwards

* revert sql changes
 2025-08-11 21:50:34 +02:00
Mohammed Nafees
34074affd8 Add contextual data for trigger via events (#2092) 
* add contextual data for trigger via events

* fix corrId

* string needed
 2025-08-06 16:52:06 -04:00
Mohammed Nafees
0b646316f1 Add GRPC callback interceptor and correlation IDs to respective API and GRPC handlers (#2073) 
* chore(deps): bump hatchet-sdk in /examples/python/quickstart (#2070)

Bumps hatchet-sdk from 1.16.3 to 1.16.4.

---
updated-dependencies:
- dependency-name: hatchet-sdk
  dependency-version: 1.16.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump google.golang.org/api from 0.243.0 to 0.244.0 (#2071)

Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.243.0 to 0.244.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.243.0...v0.244.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-version: 0.244.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* add grpc callback interceptor

* add correlation id to more endpoints

* fix string interpolation payment methods (#2072)

* hotfix: empty scope in OLAP replication (#2068)

* fix lint

* update comment

* feat: activity detection (#2055)

* feat: activity detection

* address comments

* chore(deps): bump github.com/prometheus/client_golang (#2074)

Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.22.0 to 1.23.0.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/v1.23.0/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.22.0...v1.23.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-version: 1.23.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/getsentry/sentry-go from 0.34.1 to 0.35.0 (#2075)

Bumps [github.com/getsentry/sentry-go](https://github.com/getsentry/sentry-go) from 0.34.1 to 0.35.0.
- [Release notes](https://github.com/getsentry/sentry-go/releases)
- [Changelog](https://github.com/getsentry/sentry-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/getsentry/sentry-go/compare/v0.34.1...v0.35.0)

---
updated-dependencies:
- dependency-name: github.com/getsentry/sentry-go
  dependency-version: 0.35.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* add resource id and type

* update grpc callback middleware

* fix v0 trigger

* use constants

* fix values

* use constants

* use string declared method

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: matt <mrkaye97@gmail.com>
Co-authored-by: Gabe Ruttner <gabriel.ruttner@gmail.com>
 2025-08-04 12:29:01 -04:00
matt
5bf9f97720 Fix: Validate payloads + metadata and error on illegal unicode (#2023) 
* feat: add helper method to repository

* feat: 400 on event pushes with invalid payloads

* fix: pointer

* feat: add to trigger

* feat: error on bulk trigger

* feat: error on schedule

* fix: validate log lines

* feat: validate crons

* feat: fail the task

* fix: rm debug line
 2025-07-20 22:44:28 -04:00
Matt Kaye
02601fa0ef Fix: Replay bugs (#2001) 
* fix: dedupe tasks before replaying

* fix: two toasts

* fix: send workflow run external id through

* fix: send messages to queue immediately

* fix: clean up types

* fix: dedupe

* fix: return task ids instead of workflow ones
 2025-07-16 11:42:36 -04:00
Matt Kaye
a73e34cd92 Feat: Batched replays (#1860) 
* feat: batched replays

* fix: add some comments

* feat: e2e test for bulk replays

* chore: gen

* fix: improving tests a bit

* fix: copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* chore: gen

* fix: tag

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-06-13 13:47:50 -04:00
Matt Kaye
92e86dc163 Feat: Next UI improvements, filters improvements, Langfuse docs, tenant getter, workflow status getter (#1801) 
* fix: rm method from docs button

* feat: close side panel on navigate

* feat: migration to fix pk

* fix: create index at the end

* fix: properly compute pagination response for events

* feat: add event count to query

* feat: queries

* wire up since and until queries

* fix: fe

* fix: ordering

* chore: gen

* fix: pesky zero values

* fix: rm react-table side pagination

* fix: bug

* fix: start working on pagination

* fix: refactor a bit

* fix: pagination

* feat: pagination on workflows

* fix: callbacks

* fix: key

* fix: calc

* feat: migration attempt 2

* fix: lint

* chore: gen

* fix: recreate trigger in migration

* fix: test improvement

* fix: lint

* fix: order in schema

* fix: rename indexes in partitions too

* Feat: FE Burndown, Part V (#1814)

* feat: extend eslint config to make a lot of rules much stricter

* fix: auto-fix a bunch of linting failures

* feat: start fixing a bunch of linting errors

* fix: more

* fix: run knip to remove a bunch of unused stuff

* fix: bunch more errors

* fix: bunch more

* fix: more

* fix: checkout old file

* fix: more

* fix: couple more callbacks

* fix: remaining issues

* fix: tsc

* fix: copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: copilot pushing stuff that doesn't work, as usual

* fix: more ignores

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Feat: Langfuse Integration (#1822)

* Fix: Internal blocking calls in admin client (#1818)

* fix: admin client blockages

* chore: ver

* chore: changelog

* Revert "fix: rm langfuse for now"

This reverts commit 493566a307.

* Revert "fix: rm langfuse trace pic"

This reverts commit fb689f4c50.

* fix: pre

* chore(deps): bump golang.org/x/crypto from 0.38.0 to 0.39.0 (#1827)

Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.38.0 to 0.39.0.
- [Commits](https://github.com/golang/crypto/compare/v0.38.0...v0.39.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.39.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump google.golang.org/grpc from 1.72.2 to 1.73.0 (#1828)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.72.2 to 1.73.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.72.2...v1.73.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.73.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump golang.org/x/text from 0.25.0 to 0.26.0 (#1829)

Bumps [golang.org/x/text](https://github.com/golang/text) from 0.25.0 to 0.26.0.
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.25.0...v0.26.0)

---
updated-dependencies:
- dependency-name: golang.org/x/text
  dependency-version: 0.26.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump golang.org/x/sync from 0.14.0 to 0.15.0 (#1830)

Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.14.0 to 0.15.0.
- [Commits](https://github.com/golang/sync/compare/v0.14.0...v0.15.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sync
  dependency-version: 0.15.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump golang.org/x/time from 0.11.0 to 0.12.0 (#1831)

Bumps [golang.org/x/time](https://github.com/golang/time) from 0.11.0 to 0.12.0.
- [Commits](https://github.com/golang/time/compare/v0.11.0...v0.12.0)

---
updated-dependencies:
- dependency-name: golang.org/x/time
  dependency-version: 0.12.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump hatchet-sdk in /examples/python/quickstart (#1832)

Bumps hatchet-sdk from 1.11.0 to 1.11.1.

---
updated-dependencies:
- dependency-name: hatchet-sdk
  dependency-version: 1.11.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix: update langfuse docs / examples to agree with their recommendations

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Feat: Filter + Events Improvements (#1806)

* feat: query for listing filters on a workflow

* feat: first pass at new filtering logic

* feat: event key wildcards

* fix: typo

* fix: write wildcard on event ref insert

* feat: tests for wildcard

* chore: gen

* feat: working wildcards

* fix: test cruft

* fix: tests

* fix: tests

* fix: tests

* feat: improve wildcard handling

* fix: missed a payload include spot

* feat: extend tests more

* feat: extend test more

* fix: flakiness

* feat: add scope col to events

* feat: write scopes into db with events

* fix: god I hate zero values

* chore: gen, lint, etc.

* fix: try wrapping yield in try / finally for robustness

* fix: typing

* fix: simplify

* fix: migration ver

* Feat: Tenant getter + corresponding SDK warnings (#1809)

* feat: tenant get endpoint

* feat: impl for tenant

* chore: gen

* feat: Python impl

* feat: scary warning

* chore: lint

* fix: try / except

* feat: ts client

* feat: go

* chore: versions

* Update sdks/python/hatchet_sdk/hatchet.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update sdks/typescript/src/v1/client/client.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: fmt

* fix: description

* fix: review

* fix: changelog

* chore: gen

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Feat: Workflow run status getter on the API (#1808)

* feat: api for status getter

* feat: api

* feat: sdk

* chore: gen python

* chore: gen ts

* fix: simplify api

* chore: gen

* chore: rm unused python

* chore: vers

* fix: pr feedback

* chore: gen

* chore: gen

* chore: gen

* Feat: Filter Updates (#1840)

* feat: api

* feat: impl

* fix: patch

* chore: gen

* feat: python

* chore: changelog

* feat: ts

* feat: go

* fix: copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Feat: Declaratively create filters on worker start (#1825)

* feat: dynamically create filters when workflow is registered

* fix: python utc timestamps everywhere

* refactor: fix up runs list a bit

* refactor: move more methods under shared BaseWorkflow

* feat: register default filters

* fix: docs

* chore: gen, docs

* chore: lint

* fix: v1

* fix: add filter to proto

* feat: implement default filters on put workflow

* feat: fix go impl

* chore: gen py

* feat: wire up Python

* fix: rm cruft

* fix: ts

* fix: bugs

* chore: gen, versions

* feat: changelog

* chore: lock

* fix: go

* fix: rm cruft

* fix: lockfile

* feat: add is_declarative flag to filters

* feat: extend filter insert queries

* feat: bulk upsert filters on workflow version create

* fix: wire up declarative stuff

* fix: mutexing issue

* feat: wire up declarative filters

* feat: naming

* chore: gen

* fix: nullable payloads

* fix: check json validity

* fix: rm on conflict

* fix: query naming + declarative-ness handling

* fix: rm payload from error

* fix: versions and such

* chore: gen

* Feat: Filtering + Events Docs Revamp + SDK Tweaks (#1843)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-06-10 11:48:31 -04:00
Matt Kaye
3878bc3ca3 Feat: Remove inputs and outputs from workflow run list (#1720) 
* feat: add new route

* feat: thread param through for i/o

* fix: rm inputs

* fix: children

* refactor: cleanup

* fix: rebase

* fix: nil check

* fix: use query param instead

* fix: lint

* fix: lint

* fix: wire param through

* chore: gen yet again

* fix: comma

* chore: gen
 2025-05-20 16:45:09 -04:00
Gabe Ruttner
3cbbc9b9f1 fix: concurrent workflow version on put (#1570) 
* fix: lock previous version before list

* patch migration

* transactional

* lint

* fix nil case
 2025-04-18 10:38:12 -04:00
abelanger5
1dcb50c170 hotfix: priority on schedule workflow (#1556) 
* hotfix: priority on schedule workflow

* fix: build
 2025-04-15 13:10:40 -04:00
abelanger5
c6abd6b9d2 feat: multiple workflow concurrency keys (#1511) 
* feat: multiple workflow concurrency keys

* [Python]: Allow multiple workflow-level concurrency keys (#1512)

* chore: generate

* feat: multi concurrency

* chore: version

* feat: example + test

* fix: expand tests

* Feat  ts multiple wf concurrency (#1522)

* feat: multiple concurrency

* release: 1.2.0

* fix: merge

* fix: concurrency defn

* fix: ts multiple concurrency backwards compat (#1531)

* fix

* gen

* chore: lint

---------

Co-authored-by: mrkaye97 <mrkaye97@gmail.com>

---------

Co-authored-by: Gabe Ruttner <gabriel.ruttner@gmail.com>

* chore: ver

* chore: gen

* chore: versions

* fix: manually rename migration

* fix: patch ver

---------

Co-authored-by: mrkaye97 <mrkaye97@gmail.com>
Co-authored-by: Gabe Ruttner <gabriel.ruttner@gmail.com>
 2025-04-14 17:29:17 -04:00
Matt Kaye
80137736af Feat: Priority (#1513) 
* feat: initial work wiring up priorities

* fix: add default to default prio in the db

* feat: wire priority through api on wf creation

* feat: extend python test

* feat: priority for scheduled workflows

* feat: wire priority through python api

* feat: more wiring priority through the api

* feat: I think it works?

* feat: e2e test for priority

* it works!

* feat: expand tests for default priorities

* feat: e2e scheduling test

* fix: skip broken test for now

* fix: lint

* feat: add priority columns to cron and schedule ref  tables

* feat: update inserts to include prio

* feat: wire up more apis

* feat: more wiring

* feat: wire up more rest api fields

* chore: cruft

* fix: more wiring

* fix: lint

* chore: gen + wire up priorities

* fix: retries

* fix: try changing fixture scope

* chore: bump version again

* feat: send priority with action payload

* fix: generate script

* Feat  priority ts (#1518)

* feat: initial work wiring up priorities

* fix: add default to default prio in the db

* feat: wire priority through api on wf creation

* feat: extend python test

* feat: priority for scheduled workflows

* feat: wire priority through python api

* feat: more wiring priority through the api

* feat: I think it works?

* feat: e2e test for priority

* it works!

* feat: expand tests for default priorities

* feat: e2e scheduling test

* chore: minor version for priority

* fix: skip broken test for now

* fix: lint

* feat: add priority columns to cron and schedule ref  tables

* feat: update inserts to include prio

* feat: wire up more apis

* feat: more wiring

* feat: wire up more rest api fields

* chore: cruft

* fix: more wiring

* fix: lint

* chore: gen + wire up priorities

* fix: increase timeout

* fix: retries

* fix: try changing fixture scope

* chore: generate

* fix: set schedule priority

* feat: priority

* fix: move priority to wf

* release: 1.2.0

* rm log

* fix: import

* fix: add priority to step

---------

Co-authored-by: mrkaye97 <mrkaye97@gmail.com>

* fix: add dummy runs to priority test to prevent race conditions

* fix: non-breaking field

* fix: gen

* feat: initial pass at docs

* feat: priority in go sdk

* feat: initial work on go example

* fix: doc examples

* fix: proofread

* chore: version

* feat: go sdk

* fix: lint

* fix: declarations and add back RunAsChild

* fix: child workflows

* fix: namespace

* fix: faster child workflows

* fix: sticky

* add back run as child

---------

Co-authored-by: Gabe Ruttner <gabriel.ruttner@gmail.com>
Co-authored-by: Alexander Belanger <alexander@hatchet.run>
 2025-04-14 16:22:00 -04:00
abelanger5
29a7258e5c fix: match condition writes and retry counts on failure (#1507) 2025-04-08 13:34:33 -04:00
Gabe Ruttner
dc757a36b5 fix: filter scope for bulk ops (#1503) 
* fix: scope

* fix: default filter and both filter and ids

* release: 1.1.5
 2025-04-07 15:48:54 -07:00
Gabe Ruttner
bc72465b65 fix: list and cancel (#1502) 
* fix: list

* fix: optional since

* fix: scope

* fix: keep nil ptr

* release: 1.1.3

* Update sdks/typescript/src/v1/client/features/runs.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-04-07 09:03:32 -04:00
abelanger5
5c985c3f49 fix: set schedule timeout on task level (#1492) 2025-04-03 22:46:33 -04:00
abelanger5
cff2b37a6a fix: checksums on workflow versions (#1410) 2025-03-26 08:00:39 -07:00
Matt Kaye
5062bf1e3e V1 SDKs and Docs (#1361) 
New SDKs and docs for the v1 release.
 2025-03-25 15:45:07 -07:00
abelanger5
a20ab2de65 fix(v1): add exponential backoff for internal retries (#1399) 2025-03-25 09:14:15 -07:00
abelanger5
c54bf9266c feat(v1): tenant limits (#1388) 
* feat(v1): tenant limits

* fix: migration

* fix: kill metered cache
 2025-03-23 19:03:55 -07:00
abelanger5
00c4bbff09 feat(v1): new gRPC API endpoints (#1367) 
* wip: api contracts

* feat: implement put workflow version endpoint

* add support for match existing data, get scaffolding in place for additional triggers

* create additional matches

* feat: durable sleep, user event matching

* update protos

* fix: working poc of user events, durable sleep

* add migration

* fix: migration column

* feat: durable event listener

* fix: skip overrides

* fix: input -> output
 2025-03-23 18:58:20 -07:00
abelanger5
1f2096313d feat: v1 engine (#1318) 2025-03-11 14:57:13 -04:00
Gabe Ruttner
0e91542d87 wip: backoff state (#1225) 
* wip: backoff state

* fix: retry state and step run start condition

* fix: missing key

* fix: gen

* chore: squash migration

* chore: rm todos

* ops: upgrade proto
 2025-01-28 19:16:12 +00:00
Sean Reilly
9e961ac196 Feature add version info (#1154) 
* adding a /version endpoint for the engine and a /api/v1/version endpoint for the API

* make the security optional so we don't get redirected for having auth

* lint

* upgrade protoc to the latest available version on brew

* use useQuery and clean up html
 2025-01-06 10:50:04 -08:00
abelanger5
e12e700980 feat: CANCEL_NEWEST strategy and make cancel in progress more reliable (#1127) 2024-12-18 01:40:14 +00:00
Gabe Ruttner
574eb0b67e feat: dynamic crons (#1000) 
* wip: stub schedule page

* wip: stub list

* fix: 2025 bug...

* feat: wip cron list

* feat: addl meta

* feat: expose metadata column

* feat: sort and created at

* cron to recurring

* scheduled: with statuses

* fix: links

* feat: expose schedule ids

* feat: delete run

* fix: remove search

* feat: filterable scheduled

* fix: remove broken features

* chore: lint

* rm metadata for now

* chore: lint

* chore: recurring to cron job

* fix: review comments

* fix: populator

* wip cron changes

* fix: ids are helpful

* fix: populator

* wip

* wip: create crons, stub scheduled

* wip: create schedule

* wip add trigger buttons to all the pages

* wip: reusable trigger form

* fix: hash

* fixes: cron bugs

* fixes: cron sort

* fix: out of order migrations

* fix: add internalRetryCount

* feat: api things survive version transitions

* feat: table things

* feat: delete disabled for non api

* feat: prevent delete non api

* feat: filters

* require cron name for api

* default name

* fix: migrations

* frontend improvements and migrations

* fix: pagination

---------

Co-authored-by: Alexander Belanger <alexander@hatchet.run>
 2024-11-21 16:18:24 -05:00
abelanger5
197bdd1f88 feat: exponential backoff (#1062) 
* initial migration

* feat: exp backoff, fix linting

* fix utc issue and cleanup
 2024-11-21 13:39:02 -05:00
Gabe Ruttner
44addbb47e Feat scheduled improvements (#992) 
* wip: stub schedule page

* wip: stub list

* fix: 2025 bug...

* feat: wip cron list

* feat: addl meta

* feat: expose metadata column

* feat: sort and created at

* cron to recurring

* scheduled: with statuses

* fix: links

* feat: expose schedule ids

* feat: delete run

* fix: remove search

* feat: filterable scheduled

* fix: remove broken features

* chore: lint

* rm metadata for now

* chore: lint

* chore: recurring to cron job

* fix: review comments

* fix: populator
 2024-11-01 07:16:20 -04:00
abelanger5
2cdee59aea refactor: optimize v0.50.0 release (#975) 
- Simplifies architecture for splitting engine services into different components. The three supported services are now `grpc-api`, `scheduler`, and `controllers`. The `grpc-api` service is the only one which needs to be exposed for workers. The other two can run as unexposed services.
- Fixes a set of bugs and race conditions in the `v2` scheduler
- Adds a `lastActive` time to the `Queue` table and includes a migration which sets this `lastActive` time for the most recent 24 hours of queues. Effectively this means that the max scheduling time in a queue is 24 hours. 
- Rewrites the `ListWorkflowsForEvent` query to improve performance and select far fewer rows.
 2024-10-23 12:05:16 +00:00
Sean Reilly
29721cd1f0 Feat bulk workflows (#940) 
Adds support for inserting workflows in bulk via the API and an optional buffered insert on the engine.
 2024-10-14 15:35:29 -04:00
Gabe Ruttner
f5add0d15c fix: write duration (#936) 2024-10-03 09:37:53 -04:00
Gabe Ruttner
5fcf5eff6a fix: separate context (#929) 
* fix: separate context

* chore: comments

* chore: generate

* chore: gen

* chore: update protoc 28.2
 2024-10-02 10:53:51 -04:00
Gabe Ruttner
3ff00a1866 feat: improved dag validation (#915) 2024-09-27 14:23:06 -04:00
abelanger5
a1a10b4073 feat: dynamic rate limits (#904) 
* wip: step run expressions on rate limits

* feat: dynamic rate limits

* chore: v0.47.0

* chore: address changes from PR review

* fix: improved error handling

* address pr review

* better error messages for step run cels, remove debug logs

* fix: hash

---------

Co-authored-by: gabriel ruttner <gabriel.ruttner@gmail.com>
 2024-09-26 22:00:34 +00:00
abelanger5
7e2923d402 fix: nil concurrency limit strategy (#893) 
* fix: queue properly on internal retry

* fix: panic on nil limit strategy
 2024-09-19 16:52:13 -04:00
abelanger5
d23e5d9963 feat: expression-based concurrency keys (#889) 
* feat: expression-based concurrency keys

* fix: build

* fix: typos

* fix: gen

* fix: migration

* fix: remove print statements

* fix: reassignment bugs, retries on closed transport, pr review
 2024-09-19 10:32:22 -04:00
Sean Reilly
421fd08f78 bump protobuf to latest stable (#886) 
* bump protobuf to latest stable

* regenerate the proto

---------

Co-authored-by: Sean Reilly <sean@hatchet.run>
 2024-09-17 18:09:03 -07:00
Gabe Ruttner
526e7ef308 feat: expose priority queue (#814) 
* feat: workflow default priority

* feat: write priority on run

* feat: propagate to queue

* chore: squash migrations

* chore: generate
 2024-08-26 14:11:28 -04:00
Gabe Ruttner
4ea4712d4d refactor: performance and throughput (#756) 
Refactors the queueing logic to be fairly balanced between actions, with each action backed as a separate FIFO queue. Also adds support for priority queueing and custom queues, though those aren't exposed on the API layer yet. Improves throughput to be > 5000 tasks/second on a single queue. 

---------

Co-authored-by: Alexander Belanger <alexander@hatchet.run>
 2024-08-12 14:38:47 +00:00
Gabe Ruttner
b802f9f45f feat: stream by addl meta (#751) 
* feat: prop schedule and run

* wip

* fix: filter wfrid

* feat: hangup

* chore: rm debug log

* chore: func name

* fix: cancelled payload

* fix: load

* fix: cleanup the cahce

* fix: single proto

* fix: key -> val

* chore: case

* chore: rm dead code

* chore: rm dead code

* feat: go and docs

* fix: docs
 2024-07-29 19:09:51 +00:00
abelanger5
a245151d91 feat: add workflow kind to workflow versions (#750) 
* feat: support workflow kinds

* chore: generate
 2024-07-29 12:07:34 -07:00
Gabe Ruttner
5c5f1c5b7b feat: prop schedule and run (#749) 2024-07-29 06:38:53 -07:00
Gabe Ruttner
fd947cb5bc feat: go worker assignment (#741) 
* feat: create worker with label

* feat: worker context

* feat: dynamic labels

* feat: affinity

* fix: ptr

* fix: nil labels

* feat: sticky dag

* feat: sticky docs

* feat: sticky children

* chore: lint

* fix: tests

* fix: possibly nil workerId

* chore: cleanup unneeded pointers
 2024-07-26 10:19:11 -07:00
abelanger5
1ea4dfc5de feat: deduplicated enqueue (#735) 
* wip

* wip: functional query

* feat: expose affinity config

* feat: add weight to proto

* feat: upsert affinity state on worker start

* fix: linting

* feat: add upsert proto

* feat: upsert handler

* feat: revise model

* fix: labels

* feat: functional desired worker

* wip: ui

* feat: add state to step run events

* fix: filter empty keys

* fix: labels as badges

* feat: empty state and descriptive text

* chore: add todo

* chore: whitespace

* chore: cleanup

* chore: cleanup

* chore: fix hash

* chore: squash migrations

* fix: fair worker assignment

* fix: remaining slots on valid desired workers

* wip: sticky

* fix: count slots

* chore: rm log line

* feat: expose sticky config

* wip: sticky dag

* feat: expose desired worker id to trigger

* feat: trigger on desired worker

* feat: typescript docs

* feat: sticky python

* feat: py sticky children

* wip: py affinity

* serverless note

* feat: complete python examples

* linting

* feat: deduplicated enqueue

* fix: address changes from PR review

* chore: generate

---------

Co-authored-by: gabriel ruttner <gabriel.ruttner@gmail.com>
 2024-07-26 16:47:46 +00:00
Gabe Ruttner
ee68786d69 feat: sticky workers (#695) 
* wip

* wip: functional query

* feat: expose affinity config

* feat: add weight to proto

* feat: upsert affinity state on worker start

* fix: linting

* feat: add upsert proto

* feat: upsert handler

* feat: revise model

* fix: labels

* feat: functional desired worker

* wip: ui

* feat: add state to step run events

* fix: filter empty keys

* fix: labels as badges

* feat: empty state and descriptive text

* chore: add todo

* chore: whitespace

* chore: cleanup

* chore: cleanup

* chore: fix hash

* chore: squash migrations

* fix: fair worker assignment

* fix: remaining slots on valid desired workers

* wip: sticky

* fix: count slots

* chore: rm log line

* feat: expose sticky config

* wip: sticky dag

* feat: expose desired worker id to trigger

* feat: trigger on desired worker

* feat: typescript docs

* feat: sticky python

* feat: py sticky children

* wip: py affinity

* serverless note

* feat: complete python examples

* linting

* fix: doc link

* chore: rm debug log

* fix: simplify list labels

* fix: typo
 2024-07-22 17:20:23 -04:00
abelanger5
7c3ddfca32 feat: api server extensions (#614) 
* feat: allow extending the api server

* chore: remove internal packages to pkg

* chore: update db_gen.go

* fix: expose auth

* fix: move logger to pkg

* fix: don't generate gitignore for prisma client

* fix: allow extensions to register their own api spec

* feat: expose pool on server config

* fix: nil pointer exception on empty opts

* fix: run.go file
 2024-06-19 09:36:13 -04:00
Gabe Ruttner
bbc4e58dd9 feat: limits (#559) 
* feat: workflow run limits

* fix: resource exhausted 429

* feat: event limit

* feat: worker limit

* fix: sensible error

* fix: pb

* feat: expose limits api

* feat: default limits

* feat: add enable alert option

* feat: slack and email alerts

* fix: cron interval

* feat: make metered util

* wip: schedules and crons

* chore: squash migration

* fix: select or insert

* fix: remove unfinished meter

* chore: atlas migration

* fix: template format

* fix: shared ErrResourceExhausted

* feat: cache

* fix: limit can be nil

* fix: clarification

* fix: close meter ticker

* fix: friendly error for child workflows
 2024-06-07 10:57:57 -07:00