mirror/hatchet
1
0
Fork 0
mirror of https://github.com/hatchet-dev/hatchet.git synced 2026-04-29 14:09:54 -05:00
Code Issues Packages Projects Releases Wiki Activity

Feat: Events in the OLAP Repo (#1633)

Browse Source 
* feat: add events tables

* fix: tweak PK

* feat: migration

* feat: gen models

* fix: add external id col + index

* fix: uuid pk

* fix: types

* chore: gen

* feat: add index

* Feat: Write events into OLAP tables (#1634)

* feat: query for event creation

* feat: olap impl

* feat: wire up the olap event write

* feat: goroutine?

* feat: start wiring up inserts to triggers

* fix: no `RETURNING`

* fix: hack

* fix: inner join

* feat: attempt 2

* fix: return errors

* chore: lint

* fix: diff

* feat: add new partitions

* fix: eof

* fix: write external ids into table

* chore: gen

* fix: wiring

* fix: event deduping

* fix: insert in bulk

* fix: bug

* refactor: return type of trigger

* fix: unnest ids

* fix: unnest tenant ids

* fix: run ids in bulk insert

* feat: two bulk inserts, one tx

* fix: cruft

* fix: bug

* Update pkg/repository/v1/olap.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: rework to avoid n^2 loop

* fix: transaction timeout

* fix: lint

* fix: use error

* fix: rm penultimate version

* fix: rm penultimate test part ii

* Feat: CEL-based filtering of events (#1676)

* feat: add optional expression to workflow trigger event ref

* feat: proto field for expression

* feat: write and parse the expression

* feat: wire up through put workflow ver request

* feat: query

* fix: naming

* fix: cleanup

* fix: rebase

* Update pkg/repository/v1/trigger.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: skip workflow on cel eval failure

* fix: zero value

* fix: cel evaluator

* fix: usage

* fix: naming + type

* fix: rm event filter from v0 defn

* feat: tests + fix typing

* fix: usage

* fix: construct input

* feat: always write events

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: select existing partitions

* feat: add prio to push event request

* feat: priority from pushed events

* fix: missed a spot

* fix: write events even if they're not tied to any workflows

* fix: revert cel event filtering

* fix: couple more

* fix: simplify

* fix: gen api

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
Matt Kaye
2025-05-12 16:17:29 -04:00
committed by GitHub
parent 219e43741a
commit d6c491badf
26 changed files with 766 additions and 277 deletions
Download Patch File Download Diff File Expand all files Collapse all files
sql/schema/v1-olap.sql
+24
View File
@@ -373,6 +373,30 @@ BEGIN
END;
$$;
-- Events tables
CREATE TABLE v1_events_olap (
tenant_id UUID NOT NULL,
id UUID NOT NULL,
seen_at TIMESTAMPTZ NOT NULL,
key TEXT NOT NULL,
payload JSONB NOT NULL,
additional_metadata JSONB,
PRIMARY KEY (tenant_id, id, seen_at)
) PARTITION BY RANGE(seen_at);
CREATE INDEX v1_events_olap_key_idx ON v1_events_olap (tenant_id, key);
CREATE TABLE v1_event_to_run_olap (
run_id BIGINT NOT NULL,
run_inserted_at TIMESTAMPTZ NOT NULL,
event_id UUID NOT NULL,
event_seen_at TIMESTAMPTZ NOT NULL,
PRIMARY KEY (event_id, event_seen_at, run_id, run_inserted_at)
) PARTITION BY RANGE(event_seen_at);
-- TRIGGERS TO LINK TASKS, DAGS AND EVENTS --
CREATE OR REPLACE FUNCTION v1_tasks_olap_insert_function()
RETURNS TRIGGER AS