mirror/hatchet
1
0
Fork 0
mirror of https://github.com/hatchet-dev/hatchet.git synced 2025-12-29 12:49:45 -06:00
Code Issues Packages Projects Releases Wiki Activity
2,116 Commits 171 Branches 751 Tags
23db2a4fac1bf31cded684e55801fc40b7b04c81
Commit Graph

54 Commits

Author SHA1 Message Date
matt
23db2a4fac Fix: Pagination by bounds (#2654) 
* fix: pagination missing rows

* fix: separate functions

* fix: return both bounds from query

* fix: wiring

* fix: func

* fix: order col

* fix: bug

* fix: math is hard

* fix: more math

* fix: math and math and math

* fix: slightly more math

* fix: placeholders 🤦

* fix: where clause

* fix: math!

* fix: schema

* refactor: try with `CEIL`

* fix: mathin up a storm

* fix: I was actually a math major in college, who knew

* fix: copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-12-15 13:07:51 -05:00
matt
0a947924fa Feat: Parallelize replication from PG -> External (#2637) 
* feat: chunking query

* feat: first pass at range chunking

* fix: bug bashing

* fix: function geq

* fix: use maps.Copy

* fix: olap func

* feat: olap side

* refactor: external id

* fix: order by

* feat: wire up env vars

* fix: pass var through

* fix: naming

* fix: append to returnErr properly

* fix: use eg.Go
 2025-12-10 17:11:03 -05:00
matt
fe1bf7d356 Fix: Global Lease for OLAP (#2635) 
* feat: global lease

* fix: null handling

* fix: core db

* feat: add some more spans
 2025-12-10 13:01:58 -05:00
matt
3ff672ebe4 Fix: Don't reset offset if a new process acquires lease (#2628) 
* fix: don't reset offset if a new process acquires the lease

* fix: copy paste

* feat: migration, fix queries

* fix: more queries

* fix: down migration

* fix: comment

* feat: finish wiring up everything else

* fix: placeholder initial type

* fix: zero values everywhere

* fix: param ordering

* fix: handle no rows

* fix: zero values

* fix: limit

* fix: simplify

* fix: better defaults
 2025-12-09 19:01:51 -05:00
matt
9e14814acb Feat: OLAP Payload Cutover Job (#2618) 
* feat: migration

* feat: queries

* feat: overwrite queries

* fix: bug

* feat: first pass

* fix: more olap job wiring

* fix: signature

* fix: refs to a bunch of funcs

* feat: job

* fix: table names

* fix: span name

* chore: lint

* fix: redundant error check

* fix: naming

* fix: handle nil external id

* fix: order payload partitions descending

* fix: param for limiting which partitions get processed

* fix: olap
 2025-12-09 12:33:07 -05:00
Gabe Ruttner
c920d54519 analyze v1 lookup table (#2568) 
Co-authored-by: matt <mrkaye97@gmail.com>
 2025-11-25 17:25:40 -05:00
matt
62a163d835 Fix: Revert n+1 queries on the list API (#2531) 
* feat: revert query

* feat: revert n+1 query

* feat: revert another n+1 query

* fix: payloads
 2025-11-17 10:54:05 -05:00
matt
2824646ad7 Immediate Payload Offloads OLAP Wiring (#2492) 
* feat: payload store updates for immediate offloads

* feat: handle immediate offloads

* feat: start wiring up immediate offloads

* fix: get rid of payload store return

* feat: start immediate offloads work

* fix: event trigger put call

* fix: dynamic payload put depending on if offload worked

* fix: rm put

* fix: write event payload from the right place

* fix: dummy id for task events to prevent duplication issues with the tasks themselves

* fix: rm comments

* fix: rm unused struct

* fix: enabled wal

* fix: rm `RETURNING`

* fix: small cleanup

* fix: wal issue
 2025-11-07 17:38:10 +01:00
matt
c33091e815 fix: include payload partitions in olap partitions to drop (#2472) 2025-10-31 10:39:39 +01:00
abelanger5
e1fdeeaf1c fix: payload performance (#2441) 
* change some olap flush settings

* increase timeouts for payload wal

* fix: improve performance of payload wal metrics

* slight updates

* more small tweaks

* undo some olap changes, don't offload some payloads

* remove double reads

* try reducing wal poll limit

* analyze v1_dag

* move partition method
 2025-10-23 17:45:49 -04:00
matt
c6e154fd03 Feat: OLAP Payloads (#2410) 
* feat: olap payloads table

* feat: olap queue messages for payload puts

* feat: wire up writes on task write

* driveby: add + ignore psql-connect

* fix: down migration

* fix: use external id for pk

* fix: insert query

* fix: more external ids

* fix: bit more cleanup

* feat: dags

* fix: the rest of the refs

* fix: placeholder uuid

* fix: write external ids

* feat: wire up messages over the queue

* fix: panic

* Revert "fix: panic"

This reverts commit c0adccf2ea.

* Revert "feat: wire up messages over the queue"

This reverts commit 36f425f3c1.

* fix: rm unused method

* fix: rm more

* fix: rm cruft

* feat: wire up failures

* feat: start wiring up completed events

* fix: more wiring

* fix: finish wiring up completed event payloads

* fix: lint

* feat: start wiring up external ids in the core

* feat: olap pub

* fix: add returning

* fix: wiring

* debug: log lines for pubs

* fix: external id writes

* Revert "debug: log lines for pubs"

This reverts commit fe430840bd.

* fix: rm sample

* debug: rm pub buffer param

* Revert "debug: rm pub buffer param"

This reverts commit b42a5cacbb.

* debug: stuck queries

* debug: more logs

* debug: yet more logs

* fix: rename BulkRetrieve -> Retrieve

* chore: lint

* fix: naming

* fix: conn leak in putpayloads

* fix: revert debug

* Revert "debug: more logs"

This reverts commit 95da7de64f.

* Revert "debug: stuck queries"

This reverts commit 8fda64adc4.

* feat: improve getters, olap getter

* fix: key type

* feat: first pass at pulling olap payloads from the payload store

* fix: start fixing bugs

* fix: start reworking `includePayloads` param

* fix: include payloads wiring

* feat: analyze for payloads

* fix: simplify writes more + write event payloads

* feat: read out event payloads

* feat: env vars for dual writes

* refactor: clean up task prop drilling a bit

* feat: add include payloads params to python for tests

* fix: tx commit

* fix: dual writes

* fix: not null constraint

* fix: one more

* debug: logging

* fix: more debugging, tweak function sig

* fix: function sig

* fix: refs

* debug: more logging

* debug: more logging

* debug: fix condition

* debug: overwrite properly

* fix: revert debug

* fix: rm more drilling

* fix: comments

* fix: partitioning jobs

* chore: ver

* fix: bug, docs

* hack: dummy id and inserted at for payload offloads

* fix: bug

* fix: no need to handle offloads for task event data

* hack: jitter + current ts

* fix: short circuit

* fix: offload payloads in a tx

* fix: uncomment sampling

* fix: don't offload if external store is disabled

* chore: gen sqlc

* fix: migration

* fix: start reworking types

* fix: couple more

* fix: rm unused code

* fix: drill includePayloads down again

* fix: silence annoying error in some cases

* fix: always store payloads

* debug: use workflow run id for input

* fix: improve logging

* debug: logging on retrieve

* debug: task input

* fix: use correct field

* debug: write even null payloads to limit errors

* debug: hide error lines

* fix: quieting more errors

* fix: duplicate example names, remove print lines

* debug: add logging for olap event writes

* hack: immediate event offloads and cutovers

* fix: rm log line

* fix: import

* fix: short circuit events

* fix: duped names
 2025-10-20 09:09:49 -04:00
matt
5b5adcb8ed Feat: Scheduled run detail view, bulk cancel / replay with pagination helper (#2416) 
* feat: endpoint for listing external ids

* feat: wire up external id list

* chore: regen api

* feat: py sdk wrapper

* fix: since type

* fix: log

* fix: improve defaults for statuses

* feat: docs

* feat: docs

* fix: rm extra file

* feat: add id column to scheduled runs

* feat: side panel for scheduled runs

* fix: side panel header pinned

* fix: border + padding

* chore: gen

* chore: lint

* chore: changelog, version

* fix: spacing of cols

* fix: empty webhook resource limit

* fix: tsc

* fix: sort organizations and tenants alphabetically
 2025-10-15 11:36:45 -04:00
abelanger5
3fe1cdde37 fix: improve DAG status updates (#2343) 
* fix: improve DAG status updates

* fix: error msg
 2025-09-25 12:22:35 -04:00
matt
ee17433ac9 Fix: Event getter backwards compat (#2337) 
* feat: query for getting event in v1

* feat: extend populator to fetch v1 event

* fix: rm debug

* fix: simplify join

* fix: ctx
 2025-09-24 17:10:20 -04:00
matt
df1dab945b Hotfix: Stable ordering for flattened tasks + child tasks (#2334) 
* fix: stable ordering with flattened tasks / child tasks

* fix: `Equal` method
 2025-09-24 12:46:27 -04:00
matt
80fb7657ed Fix: Child runs not rendering after one day, empty worker ids, additional meta filters not being applied to counts (#2196) 
* fix: child runs not rendering b/c they've timed out of the lookback window

* fix: migration version

* fix: dead links

* fix: additional meta filters for status counts

* chore: lint
 2025-08-25 18:20:08 -04:00
matt
5eab4b74e7 Feat: Run ANALYZE on a few tables once a day (#2163) 
* feat: add analyze for a few tables

* feat: run at 5am utc

* fix: add tx, timeout

* fix: 30m timeout
 2025-08-19 13:43:27 -04:00
matt
c4ad23d92c Fix: Populate DAG Metadata Sequentially (#2156) 
* feat: add n+1 query

* feat: finish wiring up n+1 query

* fix: type hack

* fix: comment + partition pruning

* fix: copy paste

* fix: return error

* fix: slight correctness improvement

* fix: handle no rows error
 2025-08-18 11:31:07 -04:00
matt
ed65e41ff2 Fix: Optimize DAG timing query for Prom (#2102) 
* feat: improve dag duration query

* fix: naming

* fix: wiring

* feat: add trace

* fix: add timeouts

* fix: inserted at

* fix: correctness tweak

* fix: try upgrading pino
 2025-08-12 08:01:00 -04:00
matt
d2b60917ee Fix: Waterfall panic + query simplification (#2116) 
* fix: simplify query a bunch

* fix: simplify more

* fix: simplify a whole bunch more

* fix: wire up

* fix: query

* fix: the actual bug
 2025-08-12 07:56:13 -04:00
matt
285f1728d5 Fix: Call PopulateTaskRunData sequentially (#2097) 
* fix: call task data lookup query sequentially

* fix: error fmt

* feat: add more span attrs

* fix: end + ctx handling

* fix: int type

* fix: handle dupes, factor out into helper

* fix: naming

* fix: unwind naming change

* fix: naming
 2025-08-06 18:40:02 -04:00
matt
8480228d79 Fix: Allow bypassing partitioning for events lookup table (#2054) 
* fix: allow olap events lt to not be partitioned manually

* chore: gen

* chore: gen
 2025-07-31 18:18:49 -04:00
matt
392483c5d8 Fix: Weekly partition dropping (#2066) 
* fix: check weekly partitions older than a week old

* fix: logic

* chore: gen
 2025-07-30 16:28:08 -04:00
matt
d6f8be2c0f Feat: OLAP Table for CEL Eval Failures (#2012) 
* feat: add table, wire up partitioning

* feat: wire failures into the OLAP db from rabbit

* feat: bubble failures up to controller

* fix: naming

* fix: hack around enum type

* fix: typo

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: typos

* fix: migration name

* feat: log debug failure

* feat: pub message from debug endpoint to log failure

* fix: error handling

* fix: use ingestor

* fix: olap suffix

* fix: pass source through

* fix: dont log ingest failure

* fix: rm debug as enum opt

* chore: gen

* Feat: Webhooks (#1978)

* feat: migration + go gen

* feat: non unique source name

* feat: api types

* fix: rm cruft

* feat: initial api for webhooks

* feat: handle encryption of incoming keys

* fix: nil pointer errors

* fix: import

* feat: add endpoint for incoming webhooks

* fix: naming

* feat: start wiring up basic auth

* feat: wire up cel event parsing

* feat: implement authentication

* fix: hack for plain text content

* feat: add source to enum

* feat: add source name enum

* feat: db source name enum fix

* fix: use source name enums

* feat: nest sources

* feat: first pass at stripe

* fix: clean up source name passing

* fix: use unique name for webhook

* feat: populator test

* fix: null values

* fix: ordering

* fix: rm unnecessary index

* fix: validation

* feat: validation on create

* fix: lint

* fix: naming

* feat: wire triggering webhook name through to events table

* feat: cleanup + python gen + e2e test for basic auth

* feat: query to insert webhook validation errors

* refactor: auth handler

* fix: naming

* refactor: validation errors, part II

* feat: wire up writes through olap

* fix: linting, fallthrough case

* fix: validation

* feat: tests for failure cases for basic auth

* feat: expand tests

* fix: correctly return 404 out of task getter

* chore: generated stuff

* fix: rm cruft

* fix: longer sleep

* debug: print name + events to logs

* feat: limit to N

* feat: add limit env var

* debug: ci test

* fix: apply namespaces to keys

* fix: namespacing, part ii

* fix: sdk config

* fix: handle prefixing

* feat: handle partitioning logic

* chore: gen

* feat: add webhook limit

* feat: wire up limits

* fix: gen

* fix: reverse order of generic fallthrough

* fix: comment for potential unexpected behavior

* fix: add check constraints, improve error handling

* chore: gen

* chore: gen

* fix: improve naming

* feat: scaffold webhooks page

* feat: sidebar

* feat: first pass at page

* feat: improve feedback on UI

* feat: initial work on create modal

* feat: change default to basic

* fix: openapi spec discriminated union

* fix: go side

* feat: start wiring up placeholders for stripe and github

* feat: pre-populated fields for Stripe + Github

* feat: add name section

* feat: copy improvements, show URL

* feat: UI cleanup

* fix: check if tenant populator errors

* feat: add comments

* chore: gen again

* fix: default name

* fix: styling

* fix: improve stripe header processing

* feat: docs, part 1

* fix: lint

* fix: migration order

* feat: implement rate limit per-webhook

* feat: comment

* feat: clean up docs

* chore: gen

* fix: migration versions

* fix: olap naming

* fix: partitions

* chore: gen

* feat: store webhook cel eval failures properly

* fix: pk order

* fix: auth tweaks, move fetches out of populator

* fix: pgtype.Text instead of string pointer

* chore: gen

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-07-30 13:27:38 -04:00
Mohammed Nafees
c26ff03dc0 [hotfix] Fix duration calculation of DAGs and single tasks (#2035) 
* fix duration for multi tenant

* use external ids

* fix lint
 2025-07-21 20:41:23 +02:00
Matt Kaye
7388c6df73 Fix: Improve UpdateDAGStatuses and UpdateTaskStatuses (#2020) 
* fix: start improving query

* feat: add helper query for partition pruning

* feat: use helper query

* feat: similar optimizations for tasks query
 2025-07-18 08:29:44 -04:00
Matt Kaye
48734c8cb8 Fix: Multiple tenants for task & dag status updates (#2019) 
* feat: add function to fetch tenants in partition

* feat: update updatedagstatuses query to take list of tenants

* feat: wire tenant id through

* feat: hack string delim to wire writes through

* fix: unnest result of first func

* feat: task updates

* fix: error handling

* fix: one more func + migration

* fix: gen

* fix: concurrent tenant prom metrics and remove tenant operations

---------

Co-authored-by: Alexander Belanger <alexander@hatchet.run>
 2025-07-17 14:59:45 -04:00
Matt Kaye
c805a52e38 Fix: Events query performance improvements (#1930) 
* fix: split up event queries for perf

* fix: refs

* fix: event join
 2025-07-01 11:58:15 -04:00
abelanger5
b6d5a38c0f refactor: small updates to how task and dag statuses are handled (#1922) 
* fix: lengths of update rows

* better fix for task and dag updates
 2025-06-30 15:43:31 -04:00
Mohammed Nafees
ef498a6235 Introduce tenant Prometheus metrics (#1875) 
* introduce tenant workflow completed metric

* expose tenant prom metrics via handler

* fix workflow and worker id in metrics

* correctly add workflow metrics from workflow controller

* use olap DB to gather information for workflow completion

* fix prom metrics endpoint for tenant

* workflow name from external id

* simplify tenant registry based metrics

* add docs for prometheus metrics

* fix docs lint

* run prettier fix

* WIP metrics work

* use federate prom server URL to proxy metrics

* implement workflow duration histogram metric

* separate prom stack docker compose

* fix duration metrics calls

* move scheduler metrics to prom tenant specific file

* update docs for prom metrics

* fix lint

* use proper indices to query for durations

* reorg tenant metrics

* fix lint for doc

* update docs with promql examples and casing around prom metrics enabled

* update prom server url

* fix lint

* enabled prom metrics for v1 only from controller
 2025-06-27 11:46:31 -04:00
Matt Kaye
4348dc99e5 Feat: Events Frontend, Redux (#1859) 
* feat: add params to events api

* feat: add more filters to event query

* fix: count query

* feat: start wiring up events queries

* feat: overwrites for queries

* feat: query for event keys

* feat: wire up event key api

* feat: wire up two queries on fe

* feat: fix more wiring

* feat: display event data

* fix: styling

* feat: sidebar

* fix: types + default viz

* feat: scope filter

* feat: index

* fix: queries

* fix: queries, deduping, popover

* fix: lint

* feat: add filter id col to the event to run table

* feat: add filter id to copy

* feat: wire filter through to the db

* feat: send back triggered run ids over the api

* fix: query

* feat: json obj for triggered runs on the event

* fix: cast trick to rm overwrite

* fix: migration

* fix: rm search

* fix: don't show counts on runs modal

* fix: key

* chore: gen ts

* chore: python

* fix: changelog
 2025-06-13 14:16:08 -04:00
Matt Kaye
92e86dc163 Feat: Next UI improvements, filters improvements, Langfuse docs, tenant getter, workflow status getter (#1801) 
* fix: rm method from docs button

* feat: close side panel on navigate

* feat: migration to fix pk

* fix: create index at the end

* fix: properly compute pagination response for events

* feat: add event count to query

* feat: queries

* wire up since and until queries

* fix: fe

* fix: ordering

* chore: gen

* fix: pesky zero values

* fix: rm react-table side pagination

* fix: bug

* fix: start working on pagination

* fix: refactor a bit

* fix: pagination

* feat: pagination on workflows

* fix: callbacks

* fix: key

* fix: calc

* feat: migration attempt 2

* fix: lint

* chore: gen

* fix: recreate trigger in migration

* fix: test improvement

* fix: lint

* fix: order in schema

* fix: rename indexes in partitions too

* Feat: FE Burndown, Part V (#1814)

* feat: extend eslint config to make a lot of rules much stricter

* fix: auto-fix a bunch of linting failures

* feat: start fixing a bunch of linting errors

* fix: more

* fix: run knip to remove a bunch of unused stuff

* fix: bunch more errors

* fix: bunch more

* fix: more

* fix: checkout old file

* fix: more

* fix: couple more callbacks

* fix: remaining issues

* fix: tsc

* fix: copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: copilot pushing stuff that doesn't work, as usual

* fix: more ignores

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Feat: Langfuse Integration (#1822)

* Fix: Internal blocking calls in admin client (#1818)

* fix: admin client blockages

* chore: ver

* chore: changelog

* Revert "fix: rm langfuse for now"

This reverts commit 493566a307.

* Revert "fix: rm langfuse trace pic"

This reverts commit fb689f4c50.

* fix: pre

* chore(deps): bump golang.org/x/crypto from 0.38.0 to 0.39.0 (#1827)

Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.38.0 to 0.39.0.
- [Commits](https://github.com/golang/crypto/compare/v0.38.0...v0.39.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.39.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump google.golang.org/grpc from 1.72.2 to 1.73.0 (#1828)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.72.2 to 1.73.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.72.2...v1.73.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.73.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump golang.org/x/text from 0.25.0 to 0.26.0 (#1829)

Bumps [golang.org/x/text](https://github.com/golang/text) from 0.25.0 to 0.26.0.
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.25.0...v0.26.0)

---
updated-dependencies:
- dependency-name: golang.org/x/text
  dependency-version: 0.26.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump golang.org/x/sync from 0.14.0 to 0.15.0 (#1830)

Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.14.0 to 0.15.0.
- [Commits](https://github.com/golang/sync/compare/v0.14.0...v0.15.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sync
  dependency-version: 0.15.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump golang.org/x/time from 0.11.0 to 0.12.0 (#1831)

Bumps [golang.org/x/time](https://github.com/golang/time) from 0.11.0 to 0.12.0.
- [Commits](https://github.com/golang/time/compare/v0.11.0...v0.12.0)

---
updated-dependencies:
- dependency-name: golang.org/x/time
  dependency-version: 0.12.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump hatchet-sdk in /examples/python/quickstart (#1832)

Bumps hatchet-sdk from 1.11.0 to 1.11.1.

---
updated-dependencies:
- dependency-name: hatchet-sdk
  dependency-version: 1.11.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix: update langfuse docs / examples to agree with their recommendations

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Feat: Filter + Events Improvements (#1806)

* feat: query for listing filters on a workflow

* feat: first pass at new filtering logic

* feat: event key wildcards

* fix: typo

* fix: write wildcard on event ref insert

* feat: tests for wildcard

* chore: gen

* feat: working wildcards

* fix: test cruft

* fix: tests

* fix: tests

* fix: tests

* feat: improve wildcard handling

* fix: missed a payload include spot

* feat: extend tests more

* feat: extend test more

* fix: flakiness

* feat: add scope col to events

* feat: write scopes into db with events

* fix: god I hate zero values

* chore: gen, lint, etc.

* fix: try wrapping yield in try / finally for robustness

* fix: typing

* fix: simplify

* fix: migration ver

* Feat: Tenant getter + corresponding SDK warnings (#1809)

* feat: tenant get endpoint

* feat: impl for tenant

* chore: gen

* feat: Python impl

* feat: scary warning

* chore: lint

* fix: try / except

* feat: ts client

* feat: go

* chore: versions

* Update sdks/python/hatchet_sdk/hatchet.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update sdks/typescript/src/v1/client/client.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: fmt

* fix: description

* fix: review

* fix: changelog

* chore: gen

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Feat: Workflow run status getter on the API (#1808)

* feat: api for status getter

* feat: api

* feat: sdk

* chore: gen python

* chore: gen ts

* fix: simplify api

* chore: gen

* chore: rm unused python

* chore: vers

* fix: pr feedback

* chore: gen

* chore: gen

* chore: gen

* Feat: Filter Updates (#1840)

* feat: api

* feat: impl

* fix: patch

* chore: gen

* feat: python

* chore: changelog

* feat: ts

* feat: go

* fix: copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Feat: Declaratively create filters on worker start (#1825)

* feat: dynamically create filters when workflow is registered

* fix: python utc timestamps everywhere

* refactor: fix up runs list a bit

* refactor: move more methods under shared BaseWorkflow

* feat: register default filters

* fix: docs

* chore: gen, docs

* chore: lint

* fix: v1

* fix: add filter to proto

* feat: implement default filters on put workflow

* feat: fix go impl

* chore: gen py

* feat: wire up Python

* fix: rm cruft

* fix: ts

* fix: bugs

* chore: gen, versions

* feat: changelog

* chore: lock

* fix: go

* fix: rm cruft

* fix: lockfile

* feat: add is_declarative flag to filters

* feat: extend filter insert queries

* feat: bulk upsert filters on workflow version create

* fix: wire up declarative stuff

* fix: mutexing issue

* feat: wire up declarative filters

* feat: naming

* chore: gen

* fix: nullable payloads

* fix: check json validity

* fix: rm on conflict

* fix: query naming + declarative-ness handling

* fix: rm payload from error

* fix: versions and such

* chore: gen

* Feat: Filtering + Events Docs Revamp + SDK Tweaks (#1843)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-06-10 11:48:31 -04:00
Matt Kaye
521c5f430f Fix: UI Bug Burndown, Part I (#1774) 
* fix: horrific recursive query performance

* feat: start refactoring waterfall a bit

* fix: more waterfall cleanup

* fix: overflow

* fix: recenter button for dag view

* fix: button order
 2025-05-27 14:07:07 -04:00
Matt Kaye
3878bc3ca3 Feat: Remove inputs and outputs from workflow run list (#1720) 
* feat: add new route

* feat: thread param through for i/o

* fix: rm inputs

* fix: children

* refactor: cleanup

* fix: rebase

* fix: nil check

* fix: use query param instead

* fix: lint

* fix: lint

* fix: wire param through

* chore: gen yet again

* fix: comma

* chore: gen
 2025-05-20 16:45:09 -04:00
Gabe Ruttner
d00112746f second cast (#1721) 2025-05-14 13:35:08 -04:00
Gabe Ruttner
059293d541 non-null (#1719) 2025-05-14 11:21:28 -04:00
Matt Kaye
8c1b93b4f8 Fix: Multiple Queries for Event Partitions (#1714) 
* fix: separate queries

* fix: name

* fix: dupe the code
 2025-05-14 08:15:52 -04:00
Matt Kaye
59af4c96c4 Feat: Conditional Table Partitioning (#1712) 
* fix: conditional partitioning of tables

* fix: invert default

* fix: refs

* fix: whoops

* hack: case when handling of partitioning

* fix: revert to true

* fix: copy paste
 2025-05-13 18:39:28 -04:00
Matt Kaye
37482a354b Feat: Events in the OLAP Repo (#1707) 
* Revert "Revert "Feat: Events in the OLAP Repo (#1633)" (#1706)"

This reverts commit bf29269a27.

* Feat: Events Frontend (#1678)

* feat: add events tables

* fix: tweak PK

* feat: migration

* feat: gen models

* fix: add external id col + index

* fix: uuid pk

* fix: types

* chore: gen

* feat: add index

* Feat: Write events into OLAP tables (#1634)

* feat: query for event creation

* feat: olap impl

* feat: wire up the olap event write

* feat: goroutine?

* feat: start wiring up inserts to triggers

* fix: no `RETURNING`

* fix: hack

* fix: inner join

* feat: attempt 2

* fix: return errors

* chore: lint

* fix: diff

* feat: add new partitions

* fix: eof

* fix: write external ids into table

* chore: gen

* fix: wiring

* fix: event deduping

* fix: insert in bulk

* fix: bug

* refactor: return type of trigger

* fix: unnest ids

* fix: unnest tenant ids

* fix: run ids in bulk insert

* feat: two bulk inserts, one tx

* fix: cruft

* fix: bug

* Update pkg/repository/v1/olap.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: rework to avoid n^2 loop

* fix: transaction timeout

* fix: lint

* fix: use error

* fix: rm penultimate version

* fix: rm penultimate test part ii

* Feat: CEL-based filtering of events (#1676)

* feat: add optional expression to workflow trigger event ref

* feat: proto field for expression

* feat: write and parse the expression

* feat: wire up through put workflow ver request

* feat: query

* fix: naming

* fix: cleanup

* fix: rebase

* Update pkg/repository/v1/trigger.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: skip workflow on cel eval failure

* fix: zero value

* fix: cel evaluator

* fix: usage

* fix: naming + type

* fix: rm event filter from v0 defn

* feat: tests + fix typing

* fix: usage

* fix: construct input

* feat: always write events

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: select existing partitions

* feat: add prio to push event request

* feat: priority from pushed events

* fix: missed a spot

* fix: write events even if they're not tied to any workflows

* fix: revert cel event filtering

* fix: couple more

* fix: simplify

* feat: initial API work

* chore: gen ts

* feat: fe skeleton

* feat: wiring up skeleton data

* feat: hook

* fix: bugs

* fix: lint on gen

* fix: couple more

* feat: wire up counts

* feat: initial events cols + styling

* feat: layout

* feat: styling

* fix: cleanup

* feat: use external ids on the FE

* fix: separate openapi spec for new events route

* fix: required param

* fix: update queries and api

* feat: event detail

* fix: page

* fix: rebase

* tweak: table

* feat: add events page to sidebar

* feat: modify queries to allow fetching by triggering event

* feat: add triggering event id to api

* chore: lint

* feat: wire up events api

* fix: rm log

* fix: gen

* feat: wire up status counts

* fix: rm time series

* fix: rm state

* fix: lint

* fix: eof

* chore: lint

* feat: wire up filters

* fix: lint

* chore: api gen

* feat: add events tables

* fix: tweak PK

* feat: migration

* feat: gen models

* fix: add external id col + index

* fix: uuid pk

* fix: types

* chore: gen

* feat: add index

* Feat: Write events into OLAP tables (#1634)

* feat: query for event creation

* feat: olap impl

* feat: wire up the olap event write

* feat: goroutine?

* feat: start wiring up inserts to triggers

* fix: no `RETURNING`

* fix: hack

* fix: inner join

* feat: attempt 2

* fix: return errors

* chore: lint

* fix: diff

* feat: add new partitions

* fix: eof

* fix: write external ids into table

* chore: gen

* fix: wiring

* fix: event deduping

* fix: insert in bulk

* fix: bug

* refactor: return type of trigger

* fix: unnest ids

* fix: unnest tenant ids

* fix: run ids in bulk insert

* feat: two bulk inserts, one tx

* fix: cruft

* fix: bug

* Update pkg/repository/v1/olap.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: rework to avoid n^2 loop

* fix: transaction timeout

* fix: lint

* fix: use error

* fix: rm penultimate version

* fix: rm penultimate test part ii

* Feat: CEL-based filtering of events (#1676)

* feat: add optional expression to workflow trigger event ref

* feat: proto field for expression

* feat: write and parse the expression

* feat: wire up through put workflow ver request

* feat: query

* fix: naming

* fix: cleanup

* fix: rebase

* Update pkg/repository/v1/trigger.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: skip workflow on cel eval failure

* fix: zero value

* fix: cel evaluator

* fix: usage

* fix: naming + type

* fix: rm event filter from v0 defn

* feat: tests + fix typing

* fix: usage

* fix: construct input

* feat: always write events

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: select existing partitions

* feat: add prio to push event request

* feat: priority from pushed events

* fix: missed a spot

* fix: write events even if they're not tied to any workflows

* fix: revert cel event filtering

* fix: couple more

* fix: simplify

* fix: gen api

* fix: gen

* fix: more merge issues

* chore: gen

* fix: lockfile

* fix: merge issues

* chore: gen again

* fix: rm unused fields from openapi spec

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: migration ver

* fix: insert trigger, event types

* fix: bunch o refs

* fix: migration

* fix: queries

* fix: finish wiring up inserts

* fix: misc bugs

* fix: fe filtering

* chore: lint

* fix: formatting, gen

* fix: current_date

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-05-13 14:11:12 -04:00
Matt Kaye
bf29269a27 Revert "Feat: Events in the OLAP Repo (#1633)" (#1706) 
This reverts commit d6c491badf.
 2025-05-12 17:17:26 -04:00
Matt Kaye
d6c491badf Feat: Events in the OLAP Repo (#1633) 
* feat: add events tables

* fix: tweak PK

* feat: migration

* feat: gen models

* fix: add external id col + index

* fix: uuid pk

* fix: types

* chore: gen

* feat: add index

* Feat: Write events into OLAP tables (#1634)

* feat: query for event creation

* feat: olap impl

* feat: wire up the olap event write

* feat: goroutine?

* feat: start wiring up inserts to triggers

* fix: no `RETURNING`

* fix: hack

* fix: inner join

* feat: attempt 2

* fix: return errors

* chore: lint

* fix: diff

* feat: add new partitions

* fix: eof

* fix: write external ids into table

* chore: gen

* fix: wiring

* fix: event deduping

* fix: insert in bulk

* fix: bug

* refactor: return type of trigger

* fix: unnest ids

* fix: unnest tenant ids

* fix: run ids in bulk insert

* feat: two bulk inserts, one tx

* fix: cruft

* fix: bug

* Update pkg/repository/v1/olap.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: rework to avoid n^2 loop

* fix: transaction timeout

* fix: lint

* fix: use error

* fix: rm penultimate version

* fix: rm penultimate test part ii

* Feat: CEL-based filtering of events (#1676)

* feat: add optional expression to workflow trigger event ref

* feat: proto field for expression

* feat: write and parse the expression

* feat: wire up through put workflow ver request

* feat: query

* fix: naming

* fix: cleanup

* fix: rebase

* Update pkg/repository/v1/trigger.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: skip workflow on cel eval failure

* fix: zero value

* fix: cel evaluator

* fix: usage

* fix: naming + type

* fix: rm event filter from v0 defn

* feat: tests + fix typing

* fix: usage

* fix: construct input

* feat: always write events

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: select existing partitions

* feat: add prio to push event request

* feat: priority from pushed events

* fix: missed a spot

* fix: write events even if they're not tied to any workflows

* fix: revert cel event filtering

* fix: couple more

* fix: simplify

* fix: gen api

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-05-12 16:17:29 -04:00
Gabe Ruttner
00510869d2 Fix: more UI bugs -- focus on run detail view (#1703) 
* wip improved signatures

* gen

* api tokens page

* codeblock

* no bg on button

* alerter empty state

* layout consistency, sidebar fixes

* sidebar tool tips

* layout improvements

* spacing

* not active top level settings

* no analytics opt out oss

* layout consistency

* pagination consistency

* only show chat if enabled

* refresh list

* fundamental responsive fixes

* navbar close on interaction

* update

* lint

* fix browser navigation

* lint

* layout

* clean console

* simplified refetch

* finally bash breadcrumbs hopefully

* lint

* no sync

* build

* invalidate

* add workflow run id to task timing

* fix parent breadcrumb

* add fallback runid

* waterfall fixes

* lint

* revert

* optional workflow run id

* remove unused prop

* lint

* fix build

* fix: dont render trigger unless needed

* variable size sheet

* minor tweaks

* global sidebar

* sheet scrolling

* header consistency

* expose classname

* selection state

* revert layout

* tmp pulse indicator

* fix: minimap

* clear breadcrumbs on exit

* error data might not be json

* interactive dag

* expand row on click

* bug fixes

* add retry counts to task summary

* wip

* feat: retry count and improved logger

* task pre

* update example

* gen

* add retry count to task event

* sheet with attempt state

* wip with attempt payloads

* event log navigation

* wip dag sidebar

* rm debug lines

* fix

* fix 404

* fix: latest default

* optimizations

* fix dag nav

* layout

* lint

* fix

* address reveiw

* use shad button

* add fixme

* review

* lint

* fix panic
 2025-05-12 10:49:52 -07:00
Matt Kaye
f474c34a19 fix: slow query (#1651) 2025-04-30 12:55:56 -04:00
Gabe Ruttner
ccfc0c8d60 feat: fe overhaul api changes (#1639) 
* api changes

* generate

* pkg

* backmerge main

* revert to main

* revert main
 2025-04-30 08:09:34 -07:00
Matt Kaye
91c019fa1f Fix: Slow query (#1643) 
* fix: slow query

* feat: instrument controller

* fix: ctx naming
 2025-04-29 14:31:37 -04:00
abelanger5
2b790b96e2 fix: additional metadata for event-triggered workflows, some API fixes (#1589) 
* fix: metadata for event trigger

* fix: workflow versions and timestamps in api

* fix: truncate step nodes with max width
 2025-04-22 09:15:34 -04:00
Matt Kaye
771b733e10 Feat: Add Parent Task External ID to workflow run details, Fix input type marshalling to null (#1505) 
* feat: add parent task external id to api for workflow run

* feat: wire up parent ids

* feat: custom go type for non-null json

* feat: gen

* feat: send action ids back over the api

* fix: lint
 2025-04-07 23:40:13 -04:00
abelanger5
1bad2de35f fix: dag statuses should wait for all tasks to be created (#1428) 2025-03-27 15:22:01 -07:00
abelanger5
d7812e6847 fix(v1): use inserted_at to join dags, tasks in olap queries (#1358) 
* fix: use inserted_at to join dags, tasks in olap queries

* proper olap query
 2025-03-17 16:59:45 -04:00
abelanger5
436f0b8699 remove order by in olap query (#1342) 2025-03-14 15:21:18 +00:00