Gabe Ruttner
1261509755
configurable task ops jitter ( #1800 )
...
* configurable task ops jitter
* single config, configurable poll
* revert timeout
* fix correct param
2025-06-02 16:02:01 -04:00
Gabe Ruttner
1421c826ad
Feat configurable olap jitter ( #1759 )
...
* jitter
* times
* configurable olap jitter and interval
2025-05-21 11:01:00 -04:00
Matt Kaye
495fb756b5
Feat: UI Upgrade Flow ( #1750 )
...
* refactor: hook
* feat: add ui version to the tenant
* feat: add ui version to the tenant
* chore: bunch of generated stuff
* feat: return ui ver from db
* revert: flags change
* feat: wire up tenant update
* fix: type
* feat: upgrade / downgrade UI
* chore: tons more generated stuff 😬
* fix: some redirect logic
* fix: copy
* fix: non-required
* chore: gen
* fix: transformers
* fix: rm from v0 tenants
* fix: redirect
2025-05-20 16:28:50 -04:00
Matt Kaye
08bd27a869
Feat: Dynamic (Event) Filters ( #1704 )
...
* feat: add events tables
* fix: tweak PK
* feat: migration
* feat: gen models
* fix: add external id col + index
* fix: uuid pk
* fix: types
* chore: gen
* feat: add index
* Feat: Write events into OLAP tables (#1634 )
* feat: query for event creation
* feat: olap impl
* feat: wire up the olap event write
* feat: goroutine?
* feat: start wiring up inserts to triggers
* fix: no `RETURNING`
* fix: hack
* fix: inner join
* feat: attempt 2
* fix: return errors
* chore: lint
* fix: diff
* feat: add new partitions
* fix: eof
* fix: write external ids into table
* chore: gen
* fix: wiring
* fix: event deduping
* fix: insert in bulk
* fix: bug
* refactor: return type of trigger
* fix: unnest ids
* fix: unnest tenant ids
* fix: run ids in bulk insert
* feat: two bulk inserts, one tx
* fix: cruft
* fix: bug
* Update pkg/repository/v1/olap.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: rework to avoid n^2 loop
* fix: transaction timeout
* fix: lint
* fix: use error
* fix: rm penultimate version
* fix: rm penultimate test part ii
* Feat: CEL-based filtering of events (#1676 )
* feat: add optional expression to workflow trigger event ref
* feat: proto field for expression
* feat: write and parse the expression
* feat: wire up through put workflow ver request
* feat: query
* fix: naming
* fix: cleanup
* fix: rebase
* Update pkg/repository/v1/trigger.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: skip workflow on cel eval failure
* fix: zero value
* fix: cel evaluator
* fix: usage
* fix: naming + type
* fix: rm event filter from v0 defn
* feat: tests + fix typing
* fix: usage
* fix: construct input
* feat: always write events
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: select existing partitions
* feat: add prio to push event request
* feat: priority from pushed events
* fix: missed a spot
* fix: write events even if they're not tied to any workflows
* fix: revert cel event filtering
* fix: couple more
* fix: simplify
* feat: filters api
* feat: table for storing filters
* feat: migration
* fix: pk ordering
* feat: wiring up initial api
* feat: impl filter list
* feat: wire up inserts of filters
* feat: add resource hint to push event
* feat: multi-select filters
* feat: wire up resource hint on event push
* feat: filtering
* fix: small bug
* fix: rm version id
* fix: query
* fix: panic
* fix: schema
* fix: naming
* fix: rm python changes
* chore: lint
* fix: uuid hint
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: use overwrite for listing filters b/c of nullable arg
* fix: naming
* feat: add events tables
* fix: tweak PK
* feat: migration
* feat: gen models
* fix: add external id col + index
* fix: uuid pk
* fix: types
* chore: gen
* feat: add index
* Feat: Write events into OLAP tables (#1634 )
* feat: query for event creation
* feat: olap impl
* feat: wire up the olap event write
* feat: goroutine?
* feat: start wiring up inserts to triggers
* fix: no `RETURNING`
* fix: hack
* fix: inner join
* feat: attempt 2
* fix: return errors
* chore: lint
* fix: diff
* feat: add new partitions
* fix: eof
* fix: write external ids into table
* chore: gen
* fix: wiring
* fix: event deduping
* fix: insert in bulk
* fix: bug
* refactor: return type of trigger
* fix: unnest ids
* fix: unnest tenant ids
* fix: run ids in bulk insert
* feat: two bulk inserts, one tx
* fix: cruft
* fix: bug
* Update pkg/repository/v1/olap.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: rework to avoid n^2 loop
* fix: transaction timeout
* fix: lint
* fix: use error
* fix: rm penultimate version
* fix: rm penultimate test part ii
* Feat: CEL-based filtering of events (#1676 )
* feat: add optional expression to workflow trigger event ref
* feat: proto field for expression
* feat: write and parse the expression
* feat: wire up through put workflow ver request
* feat: query
* fix: naming
* fix: cleanup
* fix: rebase
* Update pkg/repository/v1/trigger.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: skip workflow on cel eval failure
* fix: zero value
* fix: cel evaluator
* fix: usage
* fix: naming + type
* fix: rm event filter from v0 defn
* feat: tests + fix typing
* fix: usage
* fix: construct input
* feat: always write events
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: select existing partitions
* feat: add prio to push event request
* feat: priority from pushed events
* fix: missed a spot
* fix: write events even if they're not tied to any workflows
* fix: revert cel event filtering
* fix: couple more
* fix: simplify
* fix: gen api
* fix: merge issues
* feat: filter delete endpoint
* fix: overwrite
* fix: delete filter api wiring
* fix: migration patch
* chore: gen
* fix: merge hell
* Revert "Revert "Feat: Events in the OLAP Repo (#1633 )" (#1706 )"
This reverts commit bf29269a27#1678 )
* feat: add events tables
* fix: tweak PK
* feat: migration
* feat: gen models
* fix: add external id col + index
* fix: uuid pk
* fix: types
* chore: gen
* feat: add index
* Feat: Write events into OLAP tables (#1634 )
* feat: query for event creation
* feat: olap impl
* feat: wire up the olap event write
* feat: goroutine?
* feat: start wiring up inserts to triggers
* fix: no `RETURNING`
* fix: hack
* fix: inner join
* feat: attempt 2
* fix: return errors
* chore: lint
* fix: diff
* feat: add new partitions
* fix: eof
* fix: write external ids into table
* chore: gen
* fix: wiring
* fix: event deduping
* fix: insert in bulk
* fix: bug
* refactor: return type of trigger
* fix: unnest ids
* fix: unnest tenant ids
* fix: run ids in bulk insert
* feat: two bulk inserts, one tx
* fix: cruft
* fix: bug
* Update pkg/repository/v1/olap.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: rework to avoid n^2 loop
* fix: transaction timeout
* fix: lint
* fix: use error
* fix: rm penultimate version
* fix: rm penultimate test part ii
* Feat: CEL-based filtering of events (#1676 )
* feat: add optional expression to workflow trigger event ref
* feat: proto field for expression
* feat: write and parse the expression
* feat: wire up through put workflow ver request
* feat: query
* fix: naming
* fix: cleanup
* fix: rebase
* Update pkg/repository/v1/trigger.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: skip workflow on cel eval failure
* fix: zero value
* fix: cel evaluator
* fix: usage
* fix: naming + type
* fix: rm event filter from v0 defn
* feat: tests + fix typing
* fix: usage
* fix: construct input
* feat: always write events
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: select existing partitions
* feat: add prio to push event request
* feat: priority from pushed events
* fix: missed a spot
* fix: write events even if they're not tied to any workflows
* fix: revert cel event filtering
* fix: couple more
* fix: simplify
* feat: initial API work
* chore: gen ts
* feat: fe skeleton
* feat: wiring up skeleton data
* feat: hook
* fix: bugs
* fix: lint on gen
* fix: couple more
* feat: wire up counts
* feat: initial events cols + styling
* feat: layout
* feat: styling
* fix: cleanup
* feat: use external ids on the FE
* fix: separate openapi spec for new events route
* fix: required param
* fix: update queries and api
* feat: event detail
* fix: page
* fix: rebase
* tweak: table
* feat: add events page to sidebar
* feat: modify queries to allow fetching by triggering event
* feat: add triggering event id to api
* chore: lint
* feat: wire up events api
* fix: rm log
* fix: gen
* feat: wire up status counts
* fix: rm time series
* fix: rm state
* fix: lint
* fix: eof
* chore: lint
* feat: wire up filters
* fix: lint
* chore: api gen
* feat: add events tables
* fix: tweak PK
* feat: migration
* feat: gen models
* fix: add external id col + index
* fix: uuid pk
* fix: types
* chore: gen
* feat: add index
* Feat: Write events into OLAP tables (#1634 )
* feat: query for event creation
* feat: olap impl
* feat: wire up the olap event write
* feat: goroutine?
* feat: start wiring up inserts to triggers
* fix: no `RETURNING`
* fix: hack
* fix: inner join
* feat: attempt 2
* fix: return errors
* chore: lint
* fix: diff
* feat: add new partitions
* fix: eof
* fix: write external ids into table
* chore: gen
* fix: wiring
* fix: event deduping
* fix: insert in bulk
* fix: bug
* refactor: return type of trigger
* fix: unnest ids
* fix: unnest tenant ids
* fix: run ids in bulk insert
* feat: two bulk inserts, one tx
* fix: cruft
* fix: bug
* Update pkg/repository/v1/olap.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: rework to avoid n^2 loop
* fix: transaction timeout
* fix: lint
* fix: use error
* fix: rm penultimate version
* fix: rm penultimate test part ii
* Feat: CEL-based filtering of events (#1676 )
* feat: add optional expression to workflow trigger event ref
* feat: proto field for expression
* feat: write and parse the expression
* feat: wire up through put workflow ver request
* feat: query
* fix: naming
* fix: cleanup
* fix: rebase
* Update pkg/repository/v1/trigger.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: skip workflow on cel eval failure
* fix: zero value
* fix: cel evaluator
* fix: usage
* fix: naming + type
* fix: rm event filter from v0 defn
* feat: tests + fix typing
* fix: usage
* fix: construct input
* feat: always write events
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: select existing partitions
* feat: add prio to push event request
* feat: priority from pushed events
* fix: missed a spot
* fix: write events even if they're not tied to any workflows
* fix: revert cel event filtering
* fix: couple more
* fix: simplify
* fix: gen api
* fix: gen
* fix: more merge issues
* chore: gen
* fix: lockfile
* fix: merge issues
* chore: gen again
* fix: rm unused fields from openapi spec
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: migration ver
* fix: insert trigger, event types
* fix: bunch o refs
* fix: migration
* fix: queries
* fix: finish wiring up inserts
* fix: misc bugs
* fix: fe filtering
* chore: gen
* fix: migration ver
* chore: lint
* fix: missed a couple
* fix: whitespace
* fix: formatting, gen
* fix: uuid id for filters
* feat: rest api for filters
* fix: tag
* feat: add event id, payload, and metadata to cel env
* fix: rename resource hint to scope
* chore: gen
* fix: same conflict, different day
* feat: send filter payload with input
* fix: lint
* fix: invert filter to "positive" case
* fix: naming
* feat: send payloads back with trigger data
* fix: add case to check if no filters were found
* fix: send additional meta back on bulk push
* fix: cleanup from self review
* fix: more small cleanup
* feat: initial pr feedback
* feat: validation
* fix: populator
* Feat: SDK changes for event filtering (#1677 )
* feat: query for event creation
* feat: wire up the olap event write
* feat: goroutine?
* feat: start wiring up inserts to triggers
* fix: no `RETURNING`
* fix: hack
* fix: inner join
* feat: attempt 2
* fix: return errors
* fix: diff
* feat: add new partitions
* fix: write external ids into table
* chore: gen
* fix: wiring
* fix: event deduping
* fix: insert in bulk
* fix: unnest ids
* fix: run ids in bulk insert
* feat: two bulk inserts, one tx
* fix: rework to avoid n^2 loop
* feat: proto field for expression
* fix: rm event filter from v0 defn
* chore: gen python
* Revert "fix: select existing partitions"
This reverts commit fefbdd5122b85c5d807c3dce3aed7d974f01a7d8.
* fix: rebase hell
* feat: prio
* chore: docs
* feat: gen ts
* feat: ts wiring
* feat: go
* fix: prio test bug
* chore: gen
* fix: validation bug
* feat: extend events client
* feat: e2e test
* chore: docs
* fix: test
* fix: unwind event filter
* fix: rm should skip
* chore: gen
* chore: gen
* chore: gen
* feat: resource hints + more e2e tests
* fix: use `cached_property` for id
* fix: raises
* fix: rm print cruft
* feat: wiring + e2e test
* chore: gen
* feat: wire up python sdk
* feat: static payload in test
* fix: use test run id in payload
* fix: longer sleeps
* feat: more tests
* feat: intermediate work
* feat: add validator for filter payload
* fix: rm cruft
* fix: event example
* fix: event example
* fix: e2e tests
* fix: finish cleaning up tests
* fix: __hash__ method
* fix: copilot comments!
* fix: apply namespaces to workflow names in a couple of the feature clients
* fix: handle case where namespace is falsey
* refactor: factor out
* fix: all the other refs to namespaced things
* fix: put `apply_namespace` on the client config
* fix: namespace overrides
* fix: implicitly use function name for hatchet task
* fix: name
* fix: refs
* chore: ver
* fix: durable tests
* feat: add a changelog!
* fix: changelog format
* feat: start wiring up filters in ts
* feat: scopes on event push
* feat: wire up triggering event id filter
* feat: initial ts e2e test work
* fix: run the test
* fix: drive-by bug in AI-gen slop
* fix: test naming
* feat: more test cleanup
* fix: api
* fix: ns override
* fix: test + API
* fix: lint
* fix: cruft
* chore: gen
* fix: dont run in ci?
* fix: wire up Go SDK
* fix: compiler
* fix: examples
* fix: event snippet, I think
* chore: docs
* Revert "fix: examples"
This reverts commit cbf33d629952336abeb2175728472+Copilot@users.noreply.github.com >
2025-05-16 15:44:28 -04:00
Matt Kaye
37482a354b
Feat: Events in the OLAP Repo ( #1707 )
...
* Revert "Revert "Feat: Events in the OLAP Repo (#1633 )" (#1706 )"
This reverts commit bf29269a27#1678 )
* feat: add events tables
* fix: tweak PK
* feat: migration
* feat: gen models
* fix: add external id col + index
* fix: uuid pk
* fix: types
* chore: gen
* feat: add index
* Feat: Write events into OLAP tables (#1634 )
* feat: query for event creation
* feat: olap impl
* feat: wire up the olap event write
* feat: goroutine?
* feat: start wiring up inserts to triggers
* fix: no `RETURNING`
* fix: hack
* fix: inner join
* feat: attempt 2
* fix: return errors
* chore: lint
* fix: diff
* feat: add new partitions
* fix: eof
* fix: write external ids into table
* chore: gen
* fix: wiring
* fix: event deduping
* fix: insert in bulk
* fix: bug
* refactor: return type of trigger
* fix: unnest ids
* fix: unnest tenant ids
* fix: run ids in bulk insert
* feat: two bulk inserts, one tx
* fix: cruft
* fix: bug
* Update pkg/repository/v1/olap.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: rework to avoid n^2 loop
* fix: transaction timeout
* fix: lint
* fix: use error
* fix: rm penultimate version
* fix: rm penultimate test part ii
* Feat: CEL-based filtering of events (#1676 )
* feat: add optional expression to workflow trigger event ref
* feat: proto field for expression
* feat: write and parse the expression
* feat: wire up through put workflow ver request
* feat: query
* fix: naming
* fix: cleanup
* fix: rebase
* Update pkg/repository/v1/trigger.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: skip workflow on cel eval failure
* fix: zero value
* fix: cel evaluator
* fix: usage
* fix: naming + type
* fix: rm event filter from v0 defn
* feat: tests + fix typing
* fix: usage
* fix: construct input
* feat: always write events
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: select existing partitions
* feat: add prio to push event request
* feat: priority from pushed events
* fix: missed a spot
* fix: write events even if they're not tied to any workflows
* fix: revert cel event filtering
* fix: couple more
* fix: simplify
* feat: initial API work
* chore: gen ts
* feat: fe skeleton
* feat: wiring up skeleton data
* feat: hook
* fix: bugs
* fix: lint on gen
* fix: couple more
* feat: wire up counts
* feat: initial events cols + styling
* feat: layout
* feat: styling
* fix: cleanup
* feat: use external ids on the FE
* fix: separate openapi spec for new events route
* fix: required param
* fix: update queries and api
* feat: event detail
* fix: page
* fix: rebase
* tweak: table
* feat: add events page to sidebar
* feat: modify queries to allow fetching by triggering event
* feat: add triggering event id to api
* chore: lint
* feat: wire up events api
* fix: rm log
* fix: gen
* feat: wire up status counts
* fix: rm time series
* fix: rm state
* fix: lint
* fix: eof
* chore: lint
* feat: wire up filters
* fix: lint
* chore: api gen
* feat: add events tables
* fix: tweak PK
* feat: migration
* feat: gen models
* fix: add external id col + index
* fix: uuid pk
* fix: types
* chore: gen
* feat: add index
* Feat: Write events into OLAP tables (#1634 )
* feat: query for event creation
* feat: olap impl
* feat: wire up the olap event write
* feat: goroutine?
* feat: start wiring up inserts to triggers
* fix: no `RETURNING`
* fix: hack
* fix: inner join
* feat: attempt 2
* fix: return errors
* chore: lint
* fix: diff
* feat: add new partitions
* fix: eof
* fix: write external ids into table
* chore: gen
* fix: wiring
* fix: event deduping
* fix: insert in bulk
* fix: bug
* refactor: return type of trigger
* fix: unnest ids
* fix: unnest tenant ids
* fix: run ids in bulk insert
* feat: two bulk inserts, one tx
* fix: cruft
* fix: bug
* Update pkg/repository/v1/olap.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: rework to avoid n^2 loop
* fix: transaction timeout
* fix: lint
* fix: use error
* fix: rm penultimate version
* fix: rm penultimate test part ii
* Feat: CEL-based filtering of events (#1676 )
* feat: add optional expression to workflow trigger event ref
* feat: proto field for expression
* feat: write and parse the expression
* feat: wire up through put workflow ver request
* feat: query
* fix: naming
* fix: cleanup
* fix: rebase
* Update pkg/repository/v1/trigger.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: skip workflow on cel eval failure
* fix: zero value
* fix: cel evaluator
* fix: usage
* fix: naming + type
* fix: rm event filter from v0 defn
* feat: tests + fix typing
* fix: usage
* fix: construct input
* feat: always write events
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: select existing partitions
* feat: add prio to push event request
* feat: priority from pushed events
* fix: missed a spot
* fix: write events even if they're not tied to any workflows
* fix: revert cel event filtering
* fix: couple more
* fix: simplify
* fix: gen api
* fix: gen
* fix: more merge issues
* chore: gen
* fix: lockfile
* fix: merge issues
* chore: gen again
* fix: rm unused fields from openapi spec
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: migration ver
* fix: insert trigger, event types
* fix: bunch o refs
* fix: migration
* fix: queries
* fix: finish wiring up inserts
* fix: misc bugs
* fix: fe filtering
* chore: lint
* fix: formatting, gen
* fix: current_date
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-05-13 14:11:12 -04:00
Matt Kaye
bf29269a27
Revert "Feat: Events in the OLAP Repo ( #1633 )" ( #1706 )
...
This reverts commit d6c491badf
2025-05-12 17:17:26 -04:00
Matt Kaye
d6c491badf
Feat: Events in the OLAP Repo ( #1633 )
...
* feat: add events tables
* fix: tweak PK
* feat: migration
* feat: gen models
* fix: add external id col + index
* fix: uuid pk
* fix: types
* chore: gen
* feat: add index
* Feat: Write events into OLAP tables (#1634 )
* feat: query for event creation
* feat: olap impl
* feat: wire up the olap event write
* feat: goroutine?
* feat: start wiring up inserts to triggers
* fix: no `RETURNING`
* fix: hack
* fix: inner join
* feat: attempt 2
* fix: return errors
* chore: lint
* fix: diff
* feat: add new partitions
* fix: eof
* fix: write external ids into table
* chore: gen
* fix: wiring
* fix: event deduping
* fix: insert in bulk
* fix: bug
* refactor: return type of trigger
* fix: unnest ids
* fix: unnest tenant ids
* fix: run ids in bulk insert
* feat: two bulk inserts, one tx
* fix: cruft
* fix: bug
* Update pkg/repository/v1/olap.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: rework to avoid n^2 loop
* fix: transaction timeout
* fix: lint
* fix: use error
* fix: rm penultimate version
* fix: rm penultimate test part ii
* Feat: CEL-based filtering of events (#1676 )
* feat: add optional expression to workflow trigger event ref
* feat: proto field for expression
* feat: write and parse the expression
* feat: wire up through put workflow ver request
* feat: query
* fix: naming
* fix: cleanup
* fix: rebase
* Update pkg/repository/v1/trigger.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: skip workflow on cel eval failure
* fix: zero value
* fix: cel evaluator
* fix: usage
* fix: naming + type
* fix: rm event filter from v0 defn
* feat: tests + fix typing
* fix: usage
* fix: construct input
* feat: always write events
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: select existing partitions
* feat: add prio to push event request
* feat: priority from pushed events
* fix: missed a spot
* fix: write events even if they're not tied to any workflows
* fix: revert cel event filtering
* fix: couple more
* fix: simplify
* fix: gen api
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-05-12 16:17:29 -04:00
Gabe Ruttner
3831fa3ede
feat(ts): improved custom logger ( #1652 )
...
* wip improved signatures
* feat: retry count and improved logger
* task pre
* update example
2025-05-12 09:49:18 -07:00
abelanger5
5c5c1aa5a1
feat: more features in the load testing harness ( #1691 )
...
* fix: make stripped payload size configurable
* feat: more load test features
* Update cmd/hatchet-loadtest/do.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: try to fix load tests
* increase timeout, update goleak ignores
* fix: data race in scheduler with snapshot input
* fix: logger improvements
* add one more goleak ignore
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-05-07 21:39:30 -04:00
abelanger5
858f4d9b82
fix: reset scheduling timeouts on replay ( #1693 )
2025-05-07 20:38:51 -04:00
abelanger5
8f9ae4ecf2
fix: make stripped payload size configurable ( #1685 )
2025-05-07 09:13:07 -04:00
abelanger5
d4ba9c761d
feat: pause internal controllers ( #1670 )
...
* feat: pause internal controllers
* improve controller active logic
2025-05-03 18:19:34 -04:00
abelanger5
ffbeafc204
revert: add back testing harness ( #1659 )
...
* re-add new testing harness
* add healthcheck port and pick random grpc port to listen on
* feat: parallel load tests and faster tests
* make parallelism = 5
* fix: lint
* add linter to pre
* fix: add back rampup fixes
* reduce matrix on PR, add matrix to pre-release step
* make load tests less likely to block
* make limit strategy group round robin
* uncomment lines
2025-05-01 15:22:30 -04:00
abelanger5
a2e04d34b4
fix: initialize otel in api runner ( #1648 )
...
Co-authored-by: mrkaye97 <mrkaye97@gmail.com >
2025-04-30 11:38:19 -04:00
abelanger5
2c1f1f4808
test: improve Go testing harness ( #1631 )
...
* test: improves testing harness for engine
* update CI test
* fix: race condition in test
* make tests more stable
* cleanup pub and sub buffers
* fix: goleak on rampup test
* feat: matrix tests for engine
2025-04-29 10:55:16 -04:00
Matt Kaye
1eeb8e915d
Fix: Queue blocking on many concurrency keys + failures ( #1622 )
...
* fix: move around case ordering
* feat: move worker fixture around
* fix: clean up fixtures
* feat: expand test, try to repro
* debug: more minimal repro
* fix: bug bashing
* fix: factor out concurrency queries into overwrite
* feat: improve test
* fix: improve test
* fix: lint
* feat: migration for trigger
* Fix: Retry + Cancel Bugs (#1620 )
* fix: send original retry count to cancel
* fix: key threads, contexts, and tasks on retry count
* fix: store the key on the action and use it everywhere
* refactor: signature consistency
* fix: instrumentor types
* chore: version
* feat: comment
* fix: thank you mypy
* fix: simplify callback
* fix: ts implementation
* chore: lint
* fix: rework how retries are passed
* Fix: Add Retries to Log Pushes (#1594 )
* fix: retry log pushes
* chore: version
2025-04-25 21:49:30 -04:00
abelanger5
dacf48180b
feat: sampling ( #1592 )
...
* feat: sampling
* Update internal/services/controllers/v1/olap/controller.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* docs: sampling
* sampling -> trace sampling
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-04-23 17:28:58 -04:00
Gabe Ruttner
0ec8f830b9
optimization: tenant metrics queries ( #1571 )
...
* exhaustive indexes
* fix migrations
* remove statemtn
* lint
* prune
* fix: indexes
* concurrent
* concurrent
* revert
* lint
2025-04-18 10:46:37 -04:00
Gabe Ruttner
3cbbc9b9f1
fix: concurrent workflow version on put ( #1570 )
...
* fix: lock previous version before list
* patch migration
* transactional
* lint
* fix nil case
2025-04-18 10:38:12 -04:00
abelanger5
9aead7ab68
feat: global prometheus metrics ( #1568 )
...
* feat: global prometheus metrics
* configure prom with env vars, clean up metrics
* add histogram and docs
* update port
2025-04-17 15:11:38 -04:00
abelanger5
c6abd6b9d2
feat: multiple workflow concurrency keys ( #1511 )
...
* feat: multiple workflow concurrency keys
* [Python]: Allow multiple workflow-level concurrency keys (#1512 )
* chore: generate
* feat: multi concurrency
* chore: version
* feat: example + test
* fix: expand tests
* Feat ts multiple wf concurrency (#1522 )
* feat: multiple concurrency
* release: 1.2.0
* fix: merge
* fix: concurrency defn
* fix: ts multiple concurrency backwards compat (#1531 )
* fix
* gen
* chore: lint
---------
Co-authored-by: mrkaye97 <mrkaye97@gmail.com >
---------
Co-authored-by: Gabe Ruttner <gabriel.ruttner@gmail.com >
* chore: ver
* chore: gen
* chore: versions
* fix: manually rename migration
* fix: patch ver
---------
Co-authored-by: mrkaye97 <mrkaye97@gmail.com >
Co-authored-by: Gabe Ruttner <gabriel.ruttner@gmail.com >
2025-04-14 17:29:17 -04:00
Matt Kaye
80137736af
Feat: Priority ( #1513 )
...
* feat: initial work wiring up priorities
* fix: add default to default prio in the db
* feat: wire priority through api on wf creation
* feat: extend python test
* feat: priority for scheduled workflows
* feat: wire priority through python api
* feat: more wiring priority through the api
* feat: I think it works?
* feat: e2e test for priority
* it works!
* feat: expand tests for default priorities
* feat: e2e scheduling test
* fix: skip broken test for now
* fix: lint
* feat: add priority columns to cron and schedule ref tables
* feat: update inserts to include prio
* feat: wire up more apis
* feat: more wiring
* feat: wire up more rest api fields
* chore: cruft
* fix: more wiring
* fix: lint
* chore: gen + wire up priorities
* fix: retries
* fix: try changing fixture scope
* chore: bump version again
* feat: send priority with action payload
* fix: generate script
* Feat priority ts (#1518 )
* feat: initial work wiring up priorities
* fix: add default to default prio in the db
* feat: wire priority through api on wf creation
* feat: extend python test
* feat: priority for scheduled workflows
* feat: wire priority through python api
* feat: more wiring priority through the api
* feat: I think it works?
* feat: e2e test for priority
* it works!
* feat: expand tests for default priorities
* feat: e2e scheduling test
* chore: minor version for priority
* fix: skip broken test for now
* fix: lint
* feat: add priority columns to cron and schedule ref tables
* feat: update inserts to include prio
* feat: wire up more apis
* feat: more wiring
* feat: wire up more rest api fields
* chore: cruft
* fix: more wiring
* fix: lint
* chore: gen + wire up priorities
* fix: increase timeout
* fix: retries
* fix: try changing fixture scope
* chore: generate
* fix: set schedule priority
* feat: priority
* fix: move priority to wf
* release: 1.2.0
* rm log
* fix: import
* fix: add priority to step
---------
Co-authored-by: mrkaye97 <mrkaye97@gmail.com >
* fix: add dummy runs to priority test to prevent race conditions
* fix: non-breaking field
* fix: gen
* feat: initial pass at docs
* feat: priority in go sdk
* feat: initial work on go example
* fix: doc examples
* fix: proofread
* chore: version
* feat: go sdk
* fix: lint
* fix: declarations and add back RunAsChild
* fix: child workflows
* fix: namespace
* fix: faster child workflows
* fix: sticky
* add back run as child
---------
Co-authored-by: Gabe Ruttner <gabriel.ruttner@gmail.com >
Co-authored-by: Alexander Belanger <alexander@hatchet.run >
2025-04-14 16:22:00 -04:00
abelanger5
7190078b8d
fix: remove fk constraint on crons ( #1543 )
2025-04-14 09:59:27 -04:00
abelanger5
d4e489996c
fix: v1 edge cases on concurrency, go SDK, parent outputs ( #1497 )
...
* fix: v1 edge cases on concurrency, go SDK, parent outputs
* fix: overflow on queue metrics
* revert changes to DAG
* fix: remove prefix on error for Result method
* cleanup schema, fix migrations
* fix panic edge case
2025-04-07 08:19:13 -04:00
abelanger5
b6d077f96d
feat: show concurrency queue counts in the UI ( #1495 )
...
* feat: show concurrency queue counts in the UI
* fix: parent concurrency queues
2025-04-04 12:22:14 -04:00
abelanger5
1bad2de35f
fix: dag statuses should wait for all tasks to be created ( #1428 )
2025-03-27 15:22:01 -07:00
abelanger5
bc790dbdbb
fix: migration ( #1395 )
2025-03-24 12:52:11 -07:00
abelanger5
c54bf9266c
feat(v1): tenant limits ( #1388 )
...
* feat(v1): tenant limits
* fix: migration
* fix: kill metered cache
2025-03-23 19:03:55 -07:00
abelanger5
00c4bbff09
feat(v1): new gRPC API endpoints ( #1367 )
...
* wip: api contracts
* feat: implement put workflow version endpoint
* add support for match existing data, get scaffolding in place for additional triggers
* create additional matches
* feat: durable sleep, user event matching
* update protos
* fix: working poc of user events, durable sleep
* add migration
* fix: migration column
* feat: durable event listener
* fix: skip overrides
* fix: input -> output
2025-03-23 18:58:20 -07:00
abelanger5
aebcf0bb0c
fix: boundary conditions on 1-second rate limiters ( #1379 )
2025-03-20 21:44:08 +00:00
abelanger5
e91047d7b3
feat: add back tenant alerting to v1 ( #1372 )
2025-03-19 17:50:42 -04:00
abelanger5
4164b80f13
fix: race conditions on retries w/out backoff and concurrency keys ( #1368 )
2025-03-19 12:36:36 -04:00
Gabe Ruttner
3670b94fc4
Feat v1 UI tweaks ( #1344 )
...
* fix: drop uncached loader
* feat: upgrade modal
* add beta
* hacky feature flag
* fix: build
* refetch interval
* 5s
* stop flashing on load
* lint
* fix: map
* fix: last redir
* nil check
* small styling and wording things, change default canUpgrade -> true
* switch link to github discussion
---------
Co-authored-by: Alexander Belanger <alexander@hatchet.run >
2025-03-15 09:23:32 -04:00
abelanger5
4cbde4405a
fix: more v1 bug bashing ( #1334 )
2025-03-13 17:13:04 -04:00
abelanger5
d676ef30dc
switches the default msgqueue kind to postgres in hatchet-lite ( #1325 )
2025-03-12 16:30:46 -04:00
abelanger5
ac968e94b8
fix: concurrency issues and a few small improvements ( #1324 )
2025-03-12 16:30:34 -04:00
abelanger5
1f2096313d
feat: v1 engine ( #1318 )
2025-03-11 14:57:13 -04:00
Sean Reilly
a8dd33c61f
Feature - configurable logging backend ( #1188 )
...
* allow us to configure different repos
* make the struct contents public
* pass in config values to new log repo
* rename functions - possibly breaking changes so lets discuss
* make the logging backend configurable
* fix tests
* don't allow calls to WithAdditionalConfig
* cleanup
* replace sc with server
Co-authored-by: abelanger5 <belanger@sas.upenn.edu >
* rename sc to server
* add a LRU cache for the step run lookup
* lets not use an expirable cache and just use the regular one - we cannot close the go func in exirable
---------
Co-authored-by: abelanger5 <belanger@sas.upenn.edu >
2025-01-17 15:34:10 -08:00
Sean Reilly
9e961ac196
Feature add version info ( #1154 )
...
* adding a /version endpoint for the engine and a /api/v1/version endpoint for the API
* make the security optional so we don't get redirected for having auth
* lint
* upgrade protoc to the latest available version on brew
* use useQuery and clean up html
2025-01-06 10:50:04 -08:00
abelanger5
a9936ef687
fix: set otel insecure flag for all telemetry instantiations ( #999 )
2024-10-30 17:34:36 -04:00
abelanger5
7ece86dfff
fix: start scheduler if old config is used ( #989 )
2024-10-24 10:52:57 -04:00
abelanger5
2cdee59aea
refactor: optimize v0.50.0 release ( #975 )
...
- Simplifies architecture for splitting engine services into different components. The three supported services are now `grpc-api`, `scheduler`, and `controllers`. The `grpc-api` service is the only one which needs to be exposed for workers. The other two can run as unexposed services.
- Fixes a set of bugs and race conditions in the `v2` scheduler
- Adds a `lastActive` time to the `Queue` table and includes a migration which sets this `lastActive` time for the most recent 24 hours of queues. Effectively this means that the max scheduling time in a queue is 24 hours.
- Rewrites the `ListWorkflowsForEvent` query to improve performance and select far fewer rows.
2024-10-23 12:05:16 +00:00
abelanger5
0ec434d62e
feat: allow insecure option for otel collector address ( #971 )
...
* feat: allow insecure option for otel collector address
* cast to lower
2024-10-16 20:16:22 +00:00
abelanger5
67a96d7166
feat(throughput): single process per queue ( #956 )
...
* feat(throughput): single process per queue
* fix data race
* fix: golint and data race on load test
* wrap up initial v2 scheduler
* fix: more debug logs and tighten channel logic/blocking sends
* improved casing on dispatcher and lease manager
* fix: data race on min id
* increase wait on load test, fix data race
* fix: trylock -> lock
* clean up queue when no longer in set
* fix: clean up cache on exit
* ensure cleanup is only called once
* address review comments
2024-10-15 11:05:19 -04:00
Sean Reilly
27736fa30f
bulk insert buffering ( #913 )
...
Adds bulk inserts to event writes, and adds a generic buffer which can be used by future batch implementations.
2024-10-03 16:26:12 -04:00
abelanger5
bfb11cac51
fix: always use retention on queues, optional data/worker ( #916 )
2024-09-27 14:23:14 -04:00
abelanger5
d23e5d9963
feat: expression-based concurrency keys ( #889 )
...
* feat: expression-based concurrency keys
* fix: build
* fix: typos
* fix: gen
* fix: migration
* fix: remove print statements
* fix: reassignment bugs, retries on closed transport, pr review
2024-09-19 10:32:22 -04:00
abelanger5
b5014f6b3d
chore: more visibility and debug lines for queues ( #836 )
...
* chore: more visibility and debug options for queues
* better debug lines on queue repo
* don't log so much in load test
2024-08-29 14:49:24 -04:00
abelanger5
263eaf069b
feat: pass otel through msgqueue ( #802 )
...
* feat: pass otel through msgqueue
* feat: more spans on scheduling
* otel increase batch size
2024-08-28 14:45:02 +00:00
abelanger5
6317f86793
refactor: consolidate partition logic ( #826 )
...
* refactor: consolidate partition logic
* fix: race on scheduler
* fix: move partition uuid to db query
* fix: generate
2024-08-27 15:28:53 -04:00
