Commit Graph

426 Commits

Author SHA1 Message Date
Matt Kaye
8f098360ef Feat: Tenant-in-path (#1756)
* feat: put tenant in path everywhere

* fix: no need for a tenant provider

* feat: rm tenant from local storage + query params

* fix: use `tenantId` everywhere we access the id

* fix: couple more

* feat: separate hook for getting just the tenant id

* fix: fix a couple routes

* chore: tons more generated stuff

* fix: login page

* fix: some paths

* fix: onboarding create tenant page

* feat: sent ui version when creating tenant

* fix: new tenant creation on signup

* fix: naming

* feat: redirect to v1 if the tenant has a v1 ui ver

* hack: redirect button if we get stuck

* fix: cruft

* fix: login

* chore: lint

* fix: redirect to signup

* fix: null pointer

* fix: redirects

* fix: redirect to the right place on login

* fix: make `/learn` tenanted

* fix: check tenant exists

* fix: broken lesson styling

* chore: lint

* Feat: Worker Detail (#1765)

* fix: nuke breadcrumbs

* fix: rip out query param state thing for now

* fix: rm more query state

* fix: eslint warnings

* debug: infinite loop

* fix: infinite loop

* fix: worker detail

* feat: initial work on worker filter

* fix: docs

* fix: rename worker services to pools

* fix: copy

* fix: rename

* fix: bunch more renaming

* fix: tsc

* fix: one more

* fix: more "service" refs

* fix: whoooole bunch more naming

* fix: last few

* fix: id -> name

* fix: list task runs on worker

* fix: filter state updates

* feat: worker detail query

* fix: separate components for the two pages

* fix: tsc

* feat: start sending actions over the API for worker list

* feat: actions view

* fix: styling

* fix: group pools by actions

* fix: ordering

* fix: improve sorting

* fix: copilot

* fix: don't set `replace`
2025-05-23 15:29:27 -04:00
Gabe Ruttner
59edb02a8b Fix olap txs (#1763)
* jitter

* times

* configurable olap jitter and interval

* fix--olap-tx

* unlock mu

* err
2025-05-21 13:17:39 -04:00
Gabe Ruttner
d32bb66233 log prefixes (#1762)
* log prefixes

* include strategy
2025-05-21 11:01:09 -04:00
Gabe Ruttner
1421c826ad Feat configurable olap jitter (#1759)
* jitter

* times

* configurable olap jitter and interval
2025-05-21 11:01:00 -04:00
Matt Kaye
3878bc3ca3 Feat: Remove inputs and outputs from workflow run list (#1720)
* feat: add new route

* feat: thread param through for i/o

* fix: rm inputs

* fix: children

* refactor: cleanup

* fix: rebase

* fix: nil check

* fix: use query param instead

* fix: lint

* fix: lint

* fix: wire param through

* chore: gen yet again

* fix: comma

* chore: gen
2025-05-20 16:45:09 -04:00
Matt Kaye
495fb756b5 Feat: UI Upgrade Flow (#1750)
* refactor: hook

* feat: add ui version to the tenant

* feat: add ui version to the tenant

* chore: bunch of generated stuff

* feat: return ui ver from db

* revert: flags change

* feat: wire up tenant update

* fix: type

* feat: upgrade / downgrade UI

* chore: tons more generated stuff 😬

* fix: some redirect logic

* fix: copy

* fix: non-required

* chore: gen

* fix: transformers

* fix: rm from v0 tenants

* fix: redirect
2025-05-20 16:28:50 -04:00
Matt Kaye
76ecbcc1fc Feat: Event filtering docs (#1753)
* chore: gen yet again

* feat: filters client + docs

* fix: lint

* fix: revert cruft

* fix: trigger

* fix: lint

* fix: gen

* Update frontend/docs/pages/home/run-on-event.mdx

Co-authored-by: abelanger5 <belanger@sas.upenn.edu>

* Update frontend/docs/pages/home/run-on-event.mdx

Co-authored-by: abelanger5 <belanger@sas.upenn.edu>

---------

Co-authored-by: abelanger5 <belanger@sas.upenn.edu>
2025-05-19 18:56:42 -04:00
Matt Kaye
752e6bd5cf Feat: Add event key to CEL env for filtering (#1748)
* feat: add event key to env

* chore: generate a bunch of stuff again

* feat: add a test for filtering by event key

* refactor: duped code

* fix: skip flaky test in ci

* Fix: Add filter payloads to context (#1743)

* feat: add payload to ctx

* chore: ver

* fix: add filter payload to go

* fix: tests + lint

* fix: lint

* fix: naming
2025-05-19 12:52:41 -04:00
Matt Kaye
0061ab1baf Fix: scheduled client namespacing, attempt 2 (#1740) 2025-05-16 17:06:34 -04:00
Matt Kaye
08bd27a869 Feat: Dynamic (Event) Filters (#1704)
* feat: add events tables

* fix: tweak PK

* feat: migration

* feat: gen models

* fix: add external id col + index

* fix: uuid pk

* fix: types

* chore: gen

* feat: add index

* Feat: Write events into OLAP tables (#1634)

* feat: query for event creation

* feat: olap impl

* feat: wire up the olap event write

* feat: goroutine?

* feat: start wiring up inserts to triggers

* fix: no `RETURNING`

* fix: hack

* fix: inner join

* feat: attempt 2

* fix: return errors

* chore: lint

* fix: diff

* feat: add new partitions

* fix: eof

* fix: write external ids into table

* chore: gen

* fix: wiring

* fix: event deduping

* fix: insert in bulk

* fix: bug

* refactor: return type of trigger

* fix: unnest ids

* fix: unnest tenant ids

* fix: run ids in bulk insert

* feat: two bulk inserts, one tx

* fix: cruft

* fix: bug

* Update pkg/repository/v1/olap.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: rework to avoid n^2 loop

* fix: transaction timeout

* fix: lint

* fix: use error

* fix: rm penultimate version

* fix: rm penultimate test part ii

* Feat: CEL-based filtering of events (#1676)

* feat: add optional expression to workflow trigger event ref

* feat: proto field for expression

* feat: write and parse the expression

* feat: wire up through put workflow ver request

* feat: query

* fix: naming

* fix: cleanup

* fix: rebase

* Update pkg/repository/v1/trigger.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: skip workflow on cel eval failure

* fix: zero value

* fix: cel evaluator

* fix: usage

* fix: naming + type

* fix: rm event filter from v0 defn

* feat: tests + fix typing

* fix: usage

* fix: construct input

* feat: always write events

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: select existing partitions

* feat: add prio to push event request

* feat: priority from pushed events

* fix: missed a spot

* fix: write events even if they're not tied to any workflows

* fix: revert cel event filtering

* fix: couple more

* fix: simplify

* feat: filters api

* feat: table for storing filters

* feat: migration

* fix: pk ordering

* feat: wiring up initial api

* feat: impl filter list

* feat: wire up inserts of filters

* feat: add resource hint to push event

* feat: multi-select filters

* feat: wire up resource hint on event push

* feat: filtering

* fix: small bug

* fix: rm version id

* fix: query

* fix: panic

* fix: schema

* fix: naming

* fix: rm python changes

* chore: lint

* fix: uuid hint

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: use overwrite for listing filters b/c of nullable arg

* fix: naming

* feat: add events tables

* fix: tweak PK

* feat: migration

* feat: gen models

* fix: add external id col + index

* fix: uuid pk

* fix: types

* chore: gen

* feat: add index

* Feat: Write events into OLAP tables (#1634)

* feat: query for event creation

* feat: olap impl

* feat: wire up the olap event write

* feat: goroutine?

* feat: start wiring up inserts to triggers

* fix: no `RETURNING`

* fix: hack

* fix: inner join

* feat: attempt 2

* fix: return errors

* chore: lint

* fix: diff

* feat: add new partitions

* fix: eof

* fix: write external ids into table

* chore: gen

* fix: wiring

* fix: event deduping

* fix: insert in bulk

* fix: bug

* refactor: return type of trigger

* fix: unnest ids

* fix: unnest tenant ids

* fix: run ids in bulk insert

* feat: two bulk inserts, one tx

* fix: cruft

* fix: bug

* Update pkg/repository/v1/olap.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: rework to avoid n^2 loop

* fix: transaction timeout

* fix: lint

* fix: use error

* fix: rm penultimate version

* fix: rm penultimate test part ii

* Feat: CEL-based filtering of events (#1676)

* feat: add optional expression to workflow trigger event ref

* feat: proto field for expression

* feat: write and parse the expression

* feat: wire up through put workflow ver request

* feat: query

* fix: naming

* fix: cleanup

* fix: rebase

* Update pkg/repository/v1/trigger.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: skip workflow on cel eval failure

* fix: zero value

* fix: cel evaluator

* fix: usage

* fix: naming + type

* fix: rm event filter from v0 defn

* feat: tests + fix typing

* fix: usage

* fix: construct input

* feat: always write events

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: select existing partitions

* feat: add prio to push event request

* feat: priority from pushed events

* fix: missed a spot

* fix: write events even if they're not tied to any workflows

* fix: revert cel event filtering

* fix: couple more

* fix: simplify

* fix: gen api

* fix: merge issues

* feat: filter delete endpoint

* fix: overwrite

* fix: delete filter api wiring

* fix: migration patch

* chore: gen

* fix: merge hell

* Revert "Revert "Feat: Events in the OLAP Repo (#1633)" (#1706)"

This reverts commit bf29269a27.

* Feat: Events Frontend (#1678)

* feat: add events tables

* fix: tweak PK

* feat: migration

* feat: gen models

* fix: add external id col + index

* fix: uuid pk

* fix: types

* chore: gen

* feat: add index

* Feat: Write events into OLAP tables (#1634)

* feat: query for event creation

* feat: olap impl

* feat: wire up the olap event write

* feat: goroutine?

* feat: start wiring up inserts to triggers

* fix: no `RETURNING`

* fix: hack

* fix: inner join

* feat: attempt 2

* fix: return errors

* chore: lint

* fix: diff

* feat: add new partitions

* fix: eof

* fix: write external ids into table

* chore: gen

* fix: wiring

* fix: event deduping

* fix: insert in bulk

* fix: bug

* refactor: return type of trigger

* fix: unnest ids

* fix: unnest tenant ids

* fix: run ids in bulk insert

* feat: two bulk inserts, one tx

* fix: cruft

* fix: bug

* Update pkg/repository/v1/olap.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: rework to avoid n^2 loop

* fix: transaction timeout

* fix: lint

* fix: use error

* fix: rm penultimate version

* fix: rm penultimate test part ii

* Feat: CEL-based filtering of events (#1676)

* feat: add optional expression to workflow trigger event ref

* feat: proto field for expression

* feat: write and parse the expression

* feat: wire up through put workflow ver request

* feat: query

* fix: naming

* fix: cleanup

* fix: rebase

* Update pkg/repository/v1/trigger.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: skip workflow on cel eval failure

* fix: zero value

* fix: cel evaluator

* fix: usage

* fix: naming + type

* fix: rm event filter from v0 defn

* feat: tests + fix typing

* fix: usage

* fix: construct input

* feat: always write events

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: select existing partitions

* feat: add prio to push event request

* feat: priority from pushed events

* fix: missed a spot

* fix: write events even if they're not tied to any workflows

* fix: revert cel event filtering

* fix: couple more

* fix: simplify

* feat: initial API work

* chore: gen ts

* feat: fe skeleton

* feat: wiring up skeleton data

* feat: hook

* fix: bugs

* fix: lint on gen

* fix: couple more

* feat: wire up counts

* feat: initial events cols + styling

* feat: layout

* feat: styling

* fix: cleanup

* feat: use external ids on the FE

* fix: separate openapi spec for new events route

* fix: required param

* fix: update queries and api

* feat: event detail

* fix: page

* fix: rebase

* tweak: table

* feat: add events page to sidebar

* feat: modify queries to allow fetching by triggering event

* feat: add triggering event id to api

* chore: lint

* feat: wire up events api

* fix: rm log

* fix: gen

* feat: wire up status counts

* fix: rm time series

* fix: rm state

* fix: lint

* fix: eof

* chore: lint

* feat: wire up filters

* fix: lint

* chore: api gen

* feat: add events tables

* fix: tweak PK

* feat: migration

* feat: gen models

* fix: add external id col + index

* fix: uuid pk

* fix: types

* chore: gen

* feat: add index

* Feat: Write events into OLAP tables (#1634)

* feat: query for event creation

* feat: olap impl

* feat: wire up the olap event write

* feat: goroutine?

* feat: start wiring up inserts to triggers

* fix: no `RETURNING`

* fix: hack

* fix: inner join

* feat: attempt 2

* fix: return errors

* chore: lint

* fix: diff

* feat: add new partitions

* fix: eof

* fix: write external ids into table

* chore: gen

* fix: wiring

* fix: event deduping

* fix: insert in bulk

* fix: bug

* refactor: return type of trigger

* fix: unnest ids

* fix: unnest tenant ids

* fix: run ids in bulk insert

* feat: two bulk inserts, one tx

* fix: cruft

* fix: bug

* Update pkg/repository/v1/olap.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: rework to avoid n^2 loop

* fix: transaction timeout

* fix: lint

* fix: use error

* fix: rm penultimate version

* fix: rm penultimate test part ii

* Feat: CEL-based filtering of events (#1676)

* feat: add optional expression to workflow trigger event ref

* feat: proto field for expression

* feat: write and parse the expression

* feat: wire up through put workflow ver request

* feat: query

* fix: naming

* fix: cleanup

* fix: rebase

* Update pkg/repository/v1/trigger.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: skip workflow on cel eval failure

* fix: zero value

* fix: cel evaluator

* fix: usage

* fix: naming + type

* fix: rm event filter from v0 defn

* feat: tests + fix typing

* fix: usage

* fix: construct input

* feat: always write events

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: select existing partitions

* feat: add prio to push event request

* feat: priority from pushed events

* fix: missed a spot

* fix: write events even if they're not tied to any workflows

* fix: revert cel event filtering

* fix: couple more

* fix: simplify

* fix: gen api

* fix: gen

* fix: more merge issues

* chore: gen

* fix: lockfile

* fix: merge issues

* chore: gen again

* fix: rm unused fields from openapi spec

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: migration ver

* fix: insert trigger, event types

* fix: bunch o refs

* fix: migration

* fix: queries

* fix: finish wiring up inserts

* fix: misc bugs

* fix: fe filtering

* chore: gen

* fix: migration ver

* chore: lint

* fix: missed a couple

* fix: whitespace

* fix: formatting, gen

* fix: uuid id for filters

* feat: rest api for filters

* fix: tag

* feat: add event id, payload, and metadata to cel env

* fix: rename resource hint to scope

* chore: gen

* fix: same conflict, different day

* feat: send filter payload with input

* fix: lint

* fix: invert filter to "positive" case

* fix: naming

* feat: send payloads back with trigger data

* fix: add case to check if no filters were found

* fix: send additional meta back on bulk push

* fix: cleanup from self review

* fix: more small cleanup

* feat: initial pr feedback

* feat: validation

* fix: populator

* Feat: SDK changes for event filtering (#1677)

* feat: query for event creation

* feat: wire up the olap event write

* feat: goroutine?

* feat: start wiring up inserts to triggers

* fix: no `RETURNING`

* fix: hack

* fix: inner join

* feat: attempt 2

* fix: return errors

* fix: diff

* feat: add new partitions

* fix: write external ids into table

* chore: gen

* fix: wiring

* fix: event deduping

* fix: insert in bulk

* fix: unnest ids

* fix: run ids in bulk insert

* feat: two bulk inserts, one tx

* fix: rework to avoid n^2 loop

* feat: proto field for expression

* fix: rm event filter from v0 defn

* chore: gen python

* Revert "fix: select existing partitions"

This reverts commit fefbdd5122b85c5d807c3dce3aed7d974f01a7d8.

* fix: rebase hell

* feat: prio

* chore: docs

* feat: gen ts

* feat: ts wiring

* feat: go

* fix: prio test bug

* chore: gen

* fix: validation bug

* feat: extend events client

* feat: e2e test

* chore: docs

* fix: test

* fix: unwind event filter

* fix: rm should skip

* chore: gen

* chore: gen

* chore: gen

* feat: resource hints + more e2e tests

* fix: use `cached_property` for id

* fix: raises

* fix: rm print cruft

* feat: wiring + e2e test

* chore: gen

* feat: wire up python sdk

* feat: static payload in test

* fix: use test run id in payload

* fix: longer sleeps

* feat: more tests

* feat: intermediate work

* feat: add validator for filter payload

* fix: rm cruft

* fix: event example

* fix: event example

* fix: e2e tests

* fix: finish cleaning up tests

* fix: __hash__ method

* fix: copilot comments!

* fix: apply namespaces to workflow names in a couple of the feature clients

* fix: handle case where namespace is falsey

* refactor: factor out

* fix: all the other refs to namespaced things

* fix: put `apply_namespace` on the client config

* fix: namespace overrides

* fix: implicitly use function name for hatchet task

* fix: name

* fix: refs

* chore: ver

* fix: durable tests

* feat: add a changelog!

* fix: changelog format

* feat: start wiring up filters in ts

* feat: scopes on event push

* feat: wire up triggering event id filter

* feat: initial ts e2e test work

* fix: run the test

* fix: drive-by bug in AI-gen slop

* fix: test naming

* feat: more test cleanup

* fix: api

* fix: ns override

* fix: test + API

* fix: lint

* fix: cruft

* chore: gen

* fix: dont run in ci?

* fix: wire up Go SDK

* fix: compiler

* fix: examples

* fix: event snippet, I think

* chore: docs

* Revert "fix: examples"

This reverts commit cbf33d6299.

* Revert "fix: compiler"

This reverts commit 52336abeb2.

* fix: add push opt funcs for prio and scope

* chore: ver

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-16 15:44:28 -04:00
Matt Kaye
9a5e4d7642 Fix: Add workflowId and workflowVersionId to AssignedAction (#1726)
* feat: send workflow id + version id back on assigned action

* feat: wire up python sdk

* fix: patch ver

* feat: wire ts up

* feat: wire up Go

* fix: lint + tests

* Feat: Cancel from context in TS (#1722)

---------

Co-authored-by: Gabe Ruttner <gabriel.ruttner@gmail.com>
2025-05-16 11:46:57 -04:00
Matt Kaye
3e7a73a81c Hotfix: Go SDK scheduled workflows issues (#1727)
* fix: scheduled workflow namespace + input

* fix: send namespace through to schedules client

* tweak: do nothing with invalid json

* fix: helper
2025-05-16 08:11:33 -04:00
abelanger5
02cf65499c fix: topological sort should be stable (#1723) 2025-05-14 17:31:40 -04:00
Gabe Ruttner
d00112746f second cast (#1721) 2025-05-14 13:35:08 -04:00
Gabe Ruttner
059293d541 non-null (#1719) 2025-05-14 11:21:28 -04:00
Matt Kaye
8c1b93b4f8 Fix: Multiple Queries for Event Partitions (#1714)
* fix: separate queries

* fix: name

* fix: dupe the code
2025-05-14 08:15:52 -04:00
Matt Kaye
59af4c96c4 Feat: Conditional Table Partitioning (#1712)
* fix: conditional partitioning of tables

* fix: invert default

* fix: refs

* fix: whoops

* hack: case when handling of partitioning

* fix: revert to true

* fix: copy paste
2025-05-13 18:39:28 -04:00
Matt Kaye
37482a354b Feat: Events in the OLAP Repo (#1707)
* Revert "Revert "Feat: Events in the OLAP Repo (#1633)" (#1706)"

This reverts commit bf29269a27.

* Feat: Events Frontend (#1678)

* feat: add events tables

* fix: tweak PK

* feat: migration

* feat: gen models

* fix: add external id col + index

* fix: uuid pk

* fix: types

* chore: gen

* feat: add index

* Feat: Write events into OLAP tables (#1634)

* feat: query for event creation

* feat: olap impl

* feat: wire up the olap event write

* feat: goroutine?

* feat: start wiring up inserts to triggers

* fix: no `RETURNING`

* fix: hack

* fix: inner join

* feat: attempt 2

* fix: return errors

* chore: lint

* fix: diff

* feat: add new partitions

* fix: eof

* fix: write external ids into table

* chore: gen

* fix: wiring

* fix: event deduping

* fix: insert in bulk

* fix: bug

* refactor: return type of trigger

* fix: unnest ids

* fix: unnest tenant ids

* fix: run ids in bulk insert

* feat: two bulk inserts, one tx

* fix: cruft

* fix: bug

* Update pkg/repository/v1/olap.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: rework to avoid n^2 loop

* fix: transaction timeout

* fix: lint

* fix: use error

* fix: rm penultimate version

* fix: rm penultimate test part ii

* Feat: CEL-based filtering of events (#1676)

* feat: add optional expression to workflow trigger event ref

* feat: proto field for expression

* feat: write and parse the expression

* feat: wire up through put workflow ver request

* feat: query

* fix: naming

* fix: cleanup

* fix: rebase

* Update pkg/repository/v1/trigger.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: skip workflow on cel eval failure

* fix: zero value

* fix: cel evaluator

* fix: usage

* fix: naming + type

* fix: rm event filter from v0 defn

* feat: tests + fix typing

* fix: usage

* fix: construct input

* feat: always write events

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: select existing partitions

* feat: add prio to push event request

* feat: priority from pushed events

* fix: missed a spot

* fix: write events even if they're not tied to any workflows

* fix: revert cel event filtering

* fix: couple more

* fix: simplify

* feat: initial API work

* chore: gen ts

* feat: fe skeleton

* feat: wiring up skeleton data

* feat: hook

* fix: bugs

* fix: lint on gen

* fix: couple more

* feat: wire up counts

* feat: initial events cols + styling

* feat: layout

* feat: styling

* fix: cleanup

* feat: use external ids on the FE

* fix: separate openapi spec for new events route

* fix: required param

* fix: update queries and api

* feat: event detail

* fix: page

* fix: rebase

* tweak: table

* feat: add events page to sidebar

* feat: modify queries to allow fetching by triggering event

* feat: add triggering event id to api

* chore: lint

* feat: wire up events api

* fix: rm log

* fix: gen

* feat: wire up status counts

* fix: rm time series

* fix: rm state

* fix: lint

* fix: eof

* chore: lint

* feat: wire up filters

* fix: lint

* chore: api gen

* feat: add events tables

* fix: tweak PK

* feat: migration

* feat: gen models

* fix: add external id col + index

* fix: uuid pk

* fix: types

* chore: gen

* feat: add index

* Feat: Write events into OLAP tables (#1634)

* feat: query for event creation

* feat: olap impl

* feat: wire up the olap event write

* feat: goroutine?

* feat: start wiring up inserts to triggers

* fix: no `RETURNING`

* fix: hack

* fix: inner join

* feat: attempt 2

* fix: return errors

* chore: lint

* fix: diff

* feat: add new partitions

* fix: eof

* fix: write external ids into table

* chore: gen

* fix: wiring

* fix: event deduping

* fix: insert in bulk

* fix: bug

* refactor: return type of trigger

* fix: unnest ids

* fix: unnest tenant ids

* fix: run ids in bulk insert

* feat: two bulk inserts, one tx

* fix: cruft

* fix: bug

* Update pkg/repository/v1/olap.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: rework to avoid n^2 loop

* fix: transaction timeout

* fix: lint

* fix: use error

* fix: rm penultimate version

* fix: rm penultimate test part ii

* Feat: CEL-based filtering of events (#1676)

* feat: add optional expression to workflow trigger event ref

* feat: proto field for expression

* feat: write and parse the expression

* feat: wire up through put workflow ver request

* feat: query

* fix: naming

* fix: cleanup

* fix: rebase

* Update pkg/repository/v1/trigger.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: skip workflow on cel eval failure

* fix: zero value

* fix: cel evaluator

* fix: usage

* fix: naming + type

* fix: rm event filter from v0 defn

* feat: tests + fix typing

* fix: usage

* fix: construct input

* feat: always write events

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: select existing partitions

* feat: add prio to push event request

* feat: priority from pushed events

* fix: missed a spot

* fix: write events even if they're not tied to any workflows

* fix: revert cel event filtering

* fix: couple more

* fix: simplify

* fix: gen api

* fix: gen

* fix: more merge issues

* chore: gen

* fix: lockfile

* fix: merge issues

* chore: gen again

* fix: rm unused fields from openapi spec

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: migration ver

* fix: insert trigger, event types

* fix: bunch o refs

* fix: migration

* fix: queries

* fix: finish wiring up inserts

* fix: misc bugs

* fix: fe filtering

* chore: lint

* fix: formatting, gen

* fix: current_date

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-13 14:11:12 -04:00
abelanger5
41d48d8f71 fix: attempt to optimize match condition query (#1710)
* fix: attempt to optimize match condition query

* attempt where instead of join condition

* different join strategy

* split query into two

* don't send unnecessary requests

* remove query from sql file
2025-05-13 13:43:02 -04:00
Matt Kaye
bf29269a27 Revert "Feat: Events in the OLAP Repo (#1633)" (#1706)
This reverts commit d6c491badf.
2025-05-12 17:17:26 -04:00
Matt Kaye
d6c491badf Feat: Events in the OLAP Repo (#1633)
* feat: add events tables

* fix: tweak PK

* feat: migration

* feat: gen models

* fix: add external id col + index

* fix: uuid pk

* fix: types

* chore: gen

* feat: add index

* Feat: Write events into OLAP tables (#1634)

* feat: query for event creation

* feat: olap impl

* feat: wire up the olap event write

* feat: goroutine?

* feat: start wiring up inserts to triggers

* fix: no `RETURNING`

* fix: hack

* fix: inner join

* feat: attempt 2

* fix: return errors

* chore: lint

* fix: diff

* feat: add new partitions

* fix: eof

* fix: write external ids into table

* chore: gen

* fix: wiring

* fix: event deduping

* fix: insert in bulk

* fix: bug

* refactor: return type of trigger

* fix: unnest ids

* fix: unnest tenant ids

* fix: run ids in bulk insert

* feat: two bulk inserts, one tx

* fix: cruft

* fix: bug

* Update pkg/repository/v1/olap.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: rework to avoid n^2 loop

* fix: transaction timeout

* fix: lint

* fix: use error

* fix: rm penultimate version

* fix: rm penultimate test part ii

* Feat: CEL-based filtering of events (#1676)

* feat: add optional expression to workflow trigger event ref

* feat: proto field for expression

* feat: write and parse the expression

* feat: wire up through put workflow ver request

* feat: query

* fix: naming

* fix: cleanup

* fix: rebase

* Update pkg/repository/v1/trigger.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: skip workflow on cel eval failure

* fix: zero value

* fix: cel evaluator

* fix: usage

* fix: naming + type

* fix: rm event filter from v0 defn

* feat: tests + fix typing

* fix: usage

* fix: construct input

* feat: always write events

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: select existing partitions

* feat: add prio to push event request

* feat: priority from pushed events

* fix: missed a spot

* fix: write events even if they're not tied to any workflows

* fix: revert cel event filtering

* fix: couple more

* fix: simplify

* fix: gen api

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-12 16:17:29 -04:00
Gabe Ruttner
00510869d2 Fix: more UI bugs -- focus on run detail view (#1703)
* wip improved signatures

* gen

* api tokens page

* codeblock

* no bg on button

* alerter empty state

* layout consistency, sidebar fixes

* sidebar tool tips

* layout improvements

* spacing

* not active top level settings

* no analytics opt out oss

* layout consistency

* pagination consistency

* only show chat if enabled

* refresh list

* fundamental responsive fixes

* navbar close on interaction

* update

* lint

* fix browser navigation

* lint

* layout

* clean console

* simplified refetch

* finally bash breadcrumbs hopefully

* lint

* no sync

* build

* invalidate

* add workflow run id to task timing

* fix parent breadcrumb

* add fallback runid

* waterfall fixes

* lint

* revert

* optional workflow run id

* remove unused prop

* lint

* fix build

* fix: dont render trigger unless needed

* variable size sheet

* minor tweaks

* global sidebar

* sheet scrolling

* header consistency

* expose classname

* selection state

* revert layout

* tmp pulse indicator

* fix: minimap

* clear breadcrumbs on exit

* error data might not be json

* interactive dag

* expand row on click

* bug fixes

* add retry counts to task summary

* wip

* feat: retry count and improved logger

* task pre

* update example

* gen

* add retry count to task event

* sheet with attempt state

* wip with attempt payloads

* event log navigation

* wip dag sidebar

* rm debug lines

* fix

* fix 404

* fix: latest default

* optimizations

* fix dag nav

* layout

* lint

* fix

* address reveiw

* use shad button

* add fixme

* review

* lint

* fix panic
2025-05-12 10:49:52 -07:00
Gabe Ruttner
3831fa3ede feat(ts): improved custom logger (#1652)
* wip improved signatures

* feat: retry count and improved logger

* task pre

* update example
2025-05-12 09:49:18 -07:00
abelanger5
c1a0783fd8 fix: one more issue in dag populator (#1698)
* fix: one more issue in dag populator

* fix: order by
2025-05-08 20:26:30 -04:00
abelanger5
3d31d206e8 fix: select the max retry count from the DAG data (#1697) 2025-05-08 13:53:15 -04:00
abelanger5
5c5c1aa5a1 feat: more features in the load testing harness (#1691)
* fix: make stripped payload size configurable

* feat: more load test features

* Update cmd/hatchet-loadtest/do.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: try to fix load tests

* increase timeout, update goleak ignores

* fix: data race in scheduler with snapshot input

* fix: logger improvements

* add one more goleak ignore

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-07 21:39:30 -04:00
Gabe Ruttner
d56da1aa4c fix: UI bugs (#1690)
* gen

* api tokens page

* codeblock

* no bg on button

* alerter empty state

* layout consistency, sidebar fixes

* sidebar tool tips

* layout improvements

* spacing

* not active top level settings

* no analytics opt out oss

* layout consistency

* pagination consistency

* only show chat if enabled

* refresh list

* fundamental responsive fixes

* navbar close on interaction

* update

* lint

* fix browser navigation

* lint

* layout

* clean console

* simplified refetch

* finally bash breadcrumbs hopefully

* lint

* no sync

* build

* invalidate

* add workflow run id to task timing

* fix parent breadcrumb

* add fallback runid

* waterfall fixes

* lint

* revert

* optional workflow run id

* remove unused prop

* lint

* fix build
2025-05-07 12:53:32 -07:00
Gabe Ruttner
3f4424b0fc fix: check candidate slots in bounds (#1684)
* fix: check candidate slots in bounds

* fix
2025-05-06 15:34:52 -04:00
Gabe Ruttner
09f8bd6118 feat--frontend-overhaul (#1537)
* reintegration

* fix: relative routes

* dynamic routes

* up/downgrade ui

* memoize

* fix breadcrumbs

* cleanup

* titles

* title format

* additional meta

* cli upgrade

* button

* lint

* Update frontend/app/src/next/components/runs/run-id.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* build errors

* unbind on return

* lint

* Fe overhaul  run form (#1547)

* simple trigger

* populated

* feat:trigger modal

* clear

* Update frontend/app/src/next/hooks/use-runs.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fe overhaul  details (#1552)

* simple trigger

* populated

* feat:trigger modal

* clear

* Update frontend/app/src/next/hooks/use-runs.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* wip activity log

* merged logs

* wip

* wip

* search box

* fudge sort

* wip improved worker sheet

* wip

* chore: improve error on dispatcher (#1538)

* fix: empty billing context (#1553)

* fix: empty

* precommit

* hotfix: priority nil pointer (#1555)

* hotfix: priority on schedule workflow (#1556)

* hotfix: priority on schedule workflow

* fix: build

* build

* lint

* build

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: abelanger5 <belanger@sas.upenn.edu>

* Fe  overhaul  burndown 1 (#1563)

* chore: improve error on dispatcher (#1538)

* fix: empty billing context (#1553)

* fix: empty

* precommit

* hotfix: priority nil pointer (#1555)

* hotfix: priority on schedule workflow (#1556)

* hotfix: priority on schedule workflow

* fix: build

* Hotfix: Handle EOF Properly (#1557)

* fix: handle EOF properly

* chore: version

* fix: debug logs

* fix: rm eof type

* hotfix: priority on cron workflow for v0 (#1558)

* fix: one more possible null deref (#1560)

* Hatchet Python Blog Post (#1526)

* feat: initial pass at first parts of blog post

* feat: initial mkdocs setup

* feat: first pass at embedding mkdocs

* fix: config

* debug: paths

* fix: unwind docs hack

* feat: start working on mkdocs theme

* fix: paths

* feat: wrap up post

* fix: proof

* fix: doc links

* fix: rm docs

* fix: lint

* fix: lint

* fix: typos + tweak

* fix: tweaks

* fix: typo

* fix: cleanup

* fix: go signature and docs (#1561)

* fix: go signature and docs

* Update examples/v1/workflows/concurrency-rr.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* feat: toggle doc sheet

* docs: concurrency cleanup (#1562)

* feat: storage adapter

* docs--worker-config-options (#1535)

* docs--worker-config-options

* Update frontend/docs/pages/home/workers.mdx

Co-authored-by: abelanger5 <belanger@sas.upenn.edu>

* Update worker-configuration-options.mdx

* lint

---------

Co-authored-by: abelanger5 <belanger@sas.upenn.edu>

---------

Co-authored-by: abelanger5 <belanger@sas.upenn.edu>
Co-authored-by: Matt Kaye <mrkaye97@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: bit of spacing

* single generated api

* feat: initial styling pass on runs views (#1586)

* Feat: Workflows pages (#1577)

* feat: simple workflows list page

* feat: refactor + add route for individual workflow

* feat: start wiring up workflow detail page

---------

Co-authored-by: Gabe Ruttner <gabriel.ruttner@gmail.com>

* fix: tsc

* fix: compiler

* fe  overhaul runs bash (#1599)

* wip sidebar

* wip: navigation

* fix nits

* tweaks

* clear filter button

* filter on click

* always scroll y

* wip time filters

* state changes

* wip

* wip

* functional, no style

* fix default

* wip

* queue metrics

* multi-select

* padding

* counts

* fix always num

* actions

* fix maxheight

* wip

* spacing

* fix: tenant state

* fix: tenant

* lint

* github page

* tenant alerting

* ingestors

* fix: tenant state

* build errrors

* empty state alerter

* dropdown only on mobile

* billing and limits

* fix form

* fix form

* wip

* wip

* env vars

* wip

* what a component

* wip

* fix inf render

* cleanup

* persistent tabs

* wip

* common actions

* deploy modal

* update

* delete

* you're a wizard harry

* Feat: Subrows in Runs View (#1595)

* fix: type

* feat: start wiring up subrows

* fix: tsc

* fix: hook order

* fix: hack for tsc

* fix: add some margin

* fix: empty state

* fix: empty state centering

* fix: task detail links

* fix: handle error, tweak child run styling

* fix: lint

* fix: edge case

* all mw config

* wip

* fix breadcrumb hook

* fixes

* upgrade surfaces

---------

Co-authored-by: Matt Kaye <mrkaye97@gmail.com>

* bring back error toast

* toast improvements

* lint

* fix: cloud only surface

* cloud surface

* fix breadcrumbs

* feat: config view on workflow run page (#1607)

* feat: config

* fix: rm schedule timeout

* Feat: Rework runs sidebar (#1612)

* feat: rq devtools

* fix: badge hover

* feat: improve sidebar

* refactor: hook

* fix: input undefined

* feat: waterfall diagram and cleanup runs view (#1606)

* merge

* merge

* fix merge issue

* rm debug lines

* case on count

* fix: display names

* colors and handle on click

---------

Co-authored-by: mrkaye97 <mrkaye97@gmail.com>

* fix: remove a ton of dead code (#1618)

* fix: remove n+1 query (#1619)

* Fe  overhaul managed compute and onboarding (#1614)

* fix tab state on state changes

* fix collapsed children

* setup

* layout

* layout

* move feature

* wip

* chore(deps): bump k8s.io/client-go from 0.32.3 to 0.33.0 (#1608)

Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.32.3 to 0.33.0.
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kubernetes/client-go/compare/v0.32.3...v0.33.0)

---
updated-dependencies:
- dependency-name: k8s.io/client-go
  dependency-version: 0.33.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* wip

* universal install

* initial migration guide

* fakefake tokens

* fake secret

* fixes

* lint

* lint

* lint

* tidy

* chore: expose clean docs on top level

* drop reo

* wip

* static

* functional

* bump go 1.23 -> 1.24

* fix: whitespace lint

* bump golangci-lint version

* wip

* try to set up go before pre commit runs

* lint

* names

* lint

* fix: session store

* wip

* normalized snips

* fix links

* blog: mergent migration (#1611)

* universal install

* initial migration guide

* fakefake tokens

* fake secret

* fixes

* lint

* lint

* lint

* tidy

* bump go 1.23 -> 1.24

* fix: whitespace lint

* bump golangci-lint version

* try to set up go before pre commit runs

* lint

* names

* lint

* fix: session store

* fix links

---------

Co-authored-by: Alexander Belanger <alexander@hatchet.run>

* reusable state

* fix: rm unimplemented pages for now (#1615)

* feature dir

* fixes (#1616)

* with public auth

* completed typescript flow

* go blocks

* wip

* layout

* wip

* feat: read replica support and docs (#1617)

* feat: read replica support and docs

* fix: load logic

* wip

* wip

* remove provider

* migrate to static

* fix block

* lint

* fix: loadtest

* add task for linting

* cleanup

* fix meta sync

* clean examples with highlights

* get snips into app

* sync

* remove fake highlights

* always gen before build

* ignore generated

* ignore

* ignore generated

* cleanup

* always start the dev server

* examples dependabot

* app examples built off sdk examples

* auto sync on main changes

* sync the quickstarts with updating dependencies

* examples and quickstart dependabots

* only examples

* functional onboarding

* fix worker detail page

* fix detail page

* lint

* init

* tested structure copy

* prepend

* to >, ‼️ to !!

* normalize

* separate example source

* multi source

* with index

* wip

* clean generation

* migrated

* cleanup

* should build

* sync before build

* fix steps

* inline

* mkdirs

* revert build

* gen docs

* static

* rm

* update examples

* lint

* fix: ts

* fix remove lines on match

* fix: client

* add snips

* dont lint examples

* fix source

* dont test examples

* exclude quickstart

* lint

* dont break examples

* cleanup

* cleanup

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Alexander Belanger <alexander@hatchet.run>
Co-authored-by: Matt Kaye <mrkaye97@gmail.com>
Co-authored-by: abelanger5 <belanger@sas.upenn.edu>

* Merge main

* lint

* fix

* Update api-server-setup.mdx

* cleanup

* github state

* review and build

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: abelanger5 <belanger@sas.upenn.edu>
Co-authored-by: Matt Kaye <mrkaye97@gmail.com>
Co-authored-by: Alexander Belanger <alexander@hatchet.run>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-05 08:17:29 -07:00
Flo
a9d635a532 fix: validate cron expression without regex (#1669) 2025-05-03 18:20:07 -04:00
abelanger5
d4ba9c761d feat: pause internal controllers (#1670)
* feat: pause internal controllers

* improve controller active logic
2025-05-03 18:19:34 -04:00
abelanger5
ffbeafc204 revert: add back testing harness (#1659)
* re-add new testing harness

* add healthcheck port and pick random grpc port to listen on

* feat: parallel load tests and faster tests

* make parallelism = 5

* fix: lint

* add linter to pre

* fix: add back rampup fixes

* reduce matrix on PR, add matrix to pre-release step

* make load tests less likely to block

* make limit strategy group round robin

* uncomment lines
2025-05-01 15:22:30 -04:00
Gabe Ruttner
8e80faf2d6 Fe overhaul docs (#1640)
* api changes

* doc changes

* move docs

* generated

* generate

* pkg

* backmerge main

* revert to main

* revert main

* race?

* remove go tests
2025-04-30 14:10:09 -07:00
Matt Kaye
799b5d0dcf Feat: Add cron filters to the API (#1641)
* feat: add params to api

* feat: wire up API to accept params

* fix: queries, gen python

* chore: python version

* feat: gen ts

* chore: ver

* chore: lint

* [Python]: Feat: Workflow runs API wrapper, slightly improved typing, cruft (#1636)

* feat: add runs list wrapper around API

* chore: version

* feat: add to standalone

* chore: docs

* fix: clean up workflow registration

* fix: add additional opts to trigger opts

* fix: warning

* fix: defaults

* fix: docs

* docs: fix cron docs

* fix: gen
2025-04-30 16:41:33 -04:00
abelanger5
fa7ab2bd75 fix: random ticker stops working when no receiver (#1653) 2025-04-30 15:44:22 -04:00
Matt Kaye
f474c34a19 fix: slow query (#1651) 2025-04-30 12:55:56 -04:00
Gabe Ruttner
ccfc0c8d60 feat: fe overhaul api changes (#1639)
* api changes

* generate

* pkg

* backmerge main

* revert to main

* revert main
2025-04-30 08:09:34 -07:00
abelanger5
d047813fd8 fix: randomize concurrency loop (#1644) 2025-04-30 07:38:34 -04:00
Matt Kaye
91c019fa1f Fix: Slow query (#1643)
* fix: slow query

* feat: instrument controller

* fix: ctx naming
2025-04-29 14:31:37 -04:00
abelanger5
2c1f1f4808 test: improve Go testing harness (#1631)
* test: improves testing harness for engine

* update CI test

* fix: race condition in test

* make tests more stable

* cleanup pub and sub buffers

* fix: goleak on rampup test

* feat: matrix tests for engine
2025-04-29 10:55:16 -04:00
abelanger5
e6d35e0737 fix(go-sdk): actionId casing (#1637) 2025-04-29 09:58:44 -04:00
Matt Kaye
19fc908f0c Debug: Add some tracing to OLAP workflow run list (#1630)
* feat: add spans for workflow run list

* feat: add more spans
2025-04-28 12:41:29 -04:00
abelanger5
f91dcb029d fix: cancel if behavior (#1625)
* fix: cancel if behavior

* add more cancellation or groups

* one last try
2025-04-25 22:52:26 -04:00
Matt Kaye
1eeb8e915d Fix: Queue blocking on many concurrency keys + failures (#1622)
* fix: move around case ordering

* feat: move worker fixture around

* fix: clean up fixtures

* feat: expand test, try to repro

* debug: more minimal repro

* fix: bug bashing

* fix: factor out concurrency queries into overwrite

* feat: improve test

* fix: improve test

* fix: lint

* feat: migration for trigger

* Fix: Retry + Cancel Bugs (#1620)

* fix: send original retry count to cancel

* fix: key threads, contexts, and tasks on retry count

* fix: store the key on the action and use it everywhere

* refactor: signature consistency

* fix: instrumentor types

* chore: version

* feat: comment

* fix: thank you mypy

* fix: simplify callback

* fix: ts implementation

* chore: lint

* fix: rework how retries are passed

* Fix: Add Retries to Log Pushes (#1594)

* fix: retry log pushes

* chore: version
2025-04-25 21:49:30 -04:00
abelanger5
5084934b40 fix: critical deadlock bug in scheduler (#1621) 2025-04-25 21:28:15 -04:00
abelanger5
d702d5e824 feat: read replica support and docs (#1617)
* feat: read replica support and docs

* fix: load logic
2025-04-24 20:42:56 -04:00
Gabe Ruttner
5655f23e07 blog: mergent migration (#1611)
* universal install

* initial migration guide

* fakefake tokens

* fake secret

* fixes

* lint

* lint

* lint

* tidy

* bump go 1.23 -> 1.24

* fix: whitespace lint

* bump golangci-lint version

* try to set up go before pre commit runs

* lint

* names

* lint

* fix: session store

* fix links

---------

Co-authored-by: Alexander Belanger <alexander@hatchet.run>
2025-04-24 12:09:09 -07:00
abelanger5
dacf48180b feat: sampling (#1592)
* feat: sampling

* Update internal/services/controllers/v1/olap/controller.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* docs: sampling

* sampling -> trace sampling

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-04-23 17:28:58 -04:00
Matt Kaye
2df1b32932 fix: add ToPB implementation for parent condition (#1601) 2025-04-23 15:05:04 -04:00
abelanger5
2b790b96e2 fix: additional metadata for event-triggered workflows, some API fixes (#1589)
* fix: metadata for event trigger

* fix: workflow versions and timestamps in api

* fix: truncate step nodes with max width
2025-04-22 09:15:34 -04:00