44addbb47e
Feat scheduled improvements ( #992 )
...
* wip: stub schedule page
* wip: stub list
* fix: 2025 bug...
* feat: wip cron list
* feat: addl meta
* feat: expose metadata column
* feat: sort and created at
* cron to recurring
* scheduled: with statuses
* fix: links
* feat: expose schedule ids
* feat: delete run
* fix: remove search
* feat: filterable scheduled
* fix: remove broken features
* chore: lint
* rm metadata for now
* chore: lint
* chore: recurring to cron job
* fix: review comments
* fix: populator
2024-11-01 07:16:20 -04:00
831c6cfd9f
feat: add back DAG view ( #1004 )
...
* feat: add back DAG view
* fix: default to minimap if no child steps
2024-11-01 07:14:40 -04:00
7d5b41b082
add an essential pool for heatbeats ( #1003 )
...
* add an essential pool for heatbeats
* add some telemetry spans to heartbeat and capture any errors
---------
Co-authored-by: Sean Reilly <sean@hatchet.run >
2024-11-01 07:09:45 -04:00
80c3090ee0
chore(deps): bump github.com/go-co-op/gocron/v2 from 2.12.1 to 2.12.3 ( #1006 )
...
Bumps [github.com/go-co-op/gocron/v2](https://github.com/go-co-op/gocron ) from 2.12.1 to 2.12.3.
- [Release notes](https://github.com/go-co-op/gocron/releases )
- [Commits](https://github.com/go-co-op/gocron/compare/v2.12.1...v2.12.3 )
---
updated-dependencies:
- dependency-name: github.com/go-co-op/gocron/v2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-01 05:11:42 +00:00
375ac36a25
chore(deps): bump google.golang.org/api from 0.203.0 to 0.204.0 ( #1005 )
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.203.0 to 0.204.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.203.0...v0.204.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-01 05:11:24 +00:00
ea682f5c6b
Feat concurrency limit for flush ( #991 )
...
* add some concurrency limits so we don't swamp ourselves
* lets not queue up when we are too full to prevent OOM problems
* add config options for maximum concurrent and how long to wait for flush , let the wait for flush setting be used as back pressure and a signal to writers that we are slowing up
---------
Co-authored-by: Sean Reilly <sean@hatchet.run >
2024-10-31 09:43:21 -07:00
a9936ef687
fix: set otel insecure flag for all telemetry instantiations ( #999 )
2024-10-30 17:34:36 -04:00
982e4a290b
add some example values for the env vars ( #998 )
...
* add some example values for the env vars
* spelling
* space in the lint check
---------
Co-authored-by: Sean Reilly <sean@hatchet.run >
2024-10-30 11:06:38 -04:00
6158aa2a4c
feat: docs for performance ( #997 )
...
* feat: docs for performance
* wrap up perf doc
* address review comments
2024-10-29 18:29:03 -04:00
2c89abbb31
feat:add addl meta to run details ( #996 )
...
* feat:add addl meta to run details
* chore: debug
* feat: filter on click
* fix: build
* fix: build
2024-10-29 14:19:38 +00:00
4932e7f863
Feat sdk runtime ( #942 )
...
* feat: runtime signature
* feat: add sdk runtime to worker model
* feat: post runtime
* feat: expose sdk version on worker
* feat: go inf
* chore: gen
* chore: migrations and generation
* fix: simpler runtime
* feat: hatchet sdk ver
* fix: rm debug line
2024-10-28 13:47:12 -07:00
3e0f15c0d8
fix: divide by zero panic ( #995 )
...
* fix: divide by zero panic
* fix: add continue
2024-10-25 19:57:55 -04:00
8a3c8b573c
fix: update docs with process main guard ( #994 )
2024-10-25 16:45:07 -04:00
9f4b63817d
add a serial write for step run events ( #990 )
...
* add a serial write for step run events
* update other problematic queries
* tmp: don't upsert queue
* add SerialBuffer to the config
* revert the change to config
* fix: add back queue upsert
* add statement timeout to upsert queue
---------
Co-authored-by: Sean Reilly <sean@hatchet.run >
Co-authored-by: Alexander Belanger <alexander@hatchet.run >
2024-10-25 16:56:38 +00:00
509542b804
fix: duplicate assignments in queuer ( #993 )
...
* wip: individual mutexes for actions
* tmp: debug panic
* remove debug code
* remove deadlocks package and don't write unassigned events
* fix: race condition in scheduler and add internal retries
* fix: data race
2024-10-25 16:52:43 +00:00
7ece86dfff
fix: start scheduler if old config is used ( #989 )
2024-10-24 10:52:57 -04:00
706c0d0c60
chore(deps): bump k8s.io/client-go from 0.31.1 to 0.31.2 ( #988 )
...
Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go ) from 0.31.1 to 0.31.2.
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md )
- [Commits](https://github.com/kubernetes/client-go/compare/v0.31.1...v0.31.2 )
---
updated-dependencies:
- dependency-name: k8s.io/client-go
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-24 05:46:27 +00:00
6824de8306
chore(deps): bump k8s.io/api from 0.31.1 to 0.31.2 ( #985 )
...
Bumps [k8s.io/api](https://github.com/kubernetes/api ) from 0.31.1 to 0.31.2.
- [Commits](https://github.com/kubernetes/api/compare/v0.31.1...v0.31.2 )
---
updated-dependencies:
- dependency-name: k8s.io/api
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-24 05:22:13 +00:00
b2980500e5
chore(deps): bump google.golang.org/api from 0.202.0 to 0.203.0 ( #986 )
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.202.0 to 0.203.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.202.0...v0.203.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-24 05:22:07 +00:00
718d8f59c9
fix: rewrite queries for checking child workflows ( #983 )
...
* rewrite queries for child workflows
* add index
* fix: remove tenant id where it's not needed
2024-10-23 19:18:26 -04:00
dd5bc90497
fix: more efficient step run events, reduce caching on queue ( #981 )
2024-10-23 16:23:59 -04:00
35b115cb4f
don't need to filter on tenant id for step runs & some debug for buffers ( #980 )
...
Co-authored-by: Sean Reilly <sean@hatchet.run >
2024-10-23 15:04:11 -04:00
1a3d68aa0a
Docs: document improvement ( #977 )
...
* docs: fix invalid codeblock
* docs: fix go example
* Update cron-trigger.mdx
---------
Co-authored-by: abelanger5 <alexander@hatchet.run >
2024-10-23 15:03:18 -04:00
2cdee59aea
refactor: optimize v0.50.0 release ( #975 )
...
- Simplifies architecture for splitting engine services into different components. The three supported services are now `grpc-api`, `scheduler`, and `controllers`. The `grpc-api` service is the only one which needs to be exposed for workers. The other two can run as unexposed services.
- Fixes a set of bugs and race conditions in the `v2` scheduler
- Adds a `lastActive` time to the `Queue` table and includes a migration which sets this `lastActive` time for the most recent 24 hours of queues. Effectively this means that the max scheduling time in a queue is 24 hours.
- Rewrites the `ListWorkflowsForEvent` query to improve performance and select far fewer rows.
2024-10-23 12:05:16 +00:00
b3c0b268af
chore(deps): bump google.golang.org/api from 0.201.0 to 0.202.0 ( #979 )
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.201.0 to 0.202.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.201.0...v0.202.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-23 05:34:48 +00:00
29e874e71b
chore(deps): bump github.com/fatih/color from 1.17.0 to 1.18.0 ( #978 )
...
Bumps [github.com/fatih/color](https://github.com/fatih/color ) from 1.17.0 to 1.18.0.
- [Release notes](https://github.com/fatih/color/releases )
- [Commits](https://github.com/fatih/color/compare/v1.17.0...v1.18.0 )
---
updated-dependencies:
- dependency-name: github.com/fatih/color
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-23 05:34:37 +00:00
7b701ed209
fix: proper deletion of tenants from the scheduling pool ( #974 )
...
* fix: proper deletion of tenants from the scheduling pool
* adds some assignment spans
* feat: caching for rankings
* remove cache
2024-10-17 15:47:15 -04:00
ecb9ce1e1e
rejig the query for creating multiple sticky states ( #973 )
...
* rejig the query for creating multiple sticky states
* fix: sticky strategy of soft and improve query
* fix: sort method was using indexes that didn't necessarilly correspond to original indexes, leading to inconsistent behavior
---------
Co-authored-by: Sean Reilly <sean@hatchet.run >
Co-authored-by: Alexander Belanger <alexander@hatchet.run >
2024-10-17 13:29:19 +00:00
17dc80cad8
fix: don't append invalid slots with a hard sticky strategy ( #972 )
2024-10-16 20:21:39 +00:00
0ec434d62e
feat: allow insecure option for otel collector address ( #971 )
...
* feat: allow insecure option for otel collector address
* cast to lower
2024-10-16 20:16:22 +00:00
c86a50711b
fix: don't reset input for concurrency keys on replay ( #970 )
2024-10-16 15:55:28 -04:00
e4af494f69
fix: add slot expiry and delete actions from scheduler properly ( #969 )
...
* fix: add back slot expiry
* fix: remove action if all slots are inactive
2024-10-16 15:55:18 -04:00
cb39c938b3
fix: ack rate limits properly ( #968 )
2024-10-16 13:32:10 -04:00
7e526de381
fix: deadlocks on events and incorrect step run ordering query ( #966 )
...
* make it so the bulk example succeeds
* make the bulk workflows work a little harder
* add some ordering to mitigate deadlocks
* fix: link step run parents bad query, improvements to locking
* add timed mutex and telemetry
* remove for update on cancel
---------
Co-authored-by: Sean Reilly <sean@hatchet.run >
Co-authored-by: Alexander Belanger <alexander@hatchet.run >
2024-10-16 10:28:33 -04:00
d8e5055f5e
chore(deps): bump google.golang.org/api from 0.200.0 to 0.201.0 ( #967 )
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.200.0 to 0.201.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.200.0...v0.201.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-16 05:09:37 +00:00
7cd08077d5
feat: improved sdk ack ( #931 )
...
* feat: add step run event reasons
* feat: ack
* fix: remove rejected reason
* fix: merge
* fix: correct buffer
* fix: consistent message
* chore: rm todo
2024-10-15 15:52:42 +00:00
19e151e29a
fix: RunWorkflow and SpawnWorkflow should respond with consistent APIs ( #965 )
2024-10-15 11:09:58 -04:00
67a96d7166
feat(throughput): single process per queue ( #956 )
...
* feat(throughput): single process per queue
* fix data race
* fix: golint and data race on load test
* wrap up initial v2 scheduler
* fix: more debug logs and tighten channel logic/blocking sends
* improved casing on dispatcher and lease manager
* fix: data race on min id
* increase wait on load test, fix data race
* fix: trylock -> lock
* clean up queue when no longer in set
* fix: clean up cache on exit
* ensure cleanup is only called once
* address review comments
2024-10-15 11:05:19 -04:00
49dd3f4bb2
chore(deps): bump github.com/slack-go/slack from 0.14.0 to 0.15.0 ( #964 )
...
Bumps [github.com/slack-go/slack](https://github.com/slack-go/slack ) from 0.14.0 to 0.15.0.
- [Release notes](https://github.com/slack-go/slack/releases )
- [Changelog](https://github.com/slack-go/slack/blob/master/CHANGELOG.md )
- [Commits](https://github.com/slack-go/slack/compare/v0.14.0...v0.15.0 )
---
updated-dependencies:
- dependency-name: github.com/slack-go/slack
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-15 04:39:30 +00:00
020ebb8465
chore(deps): bump github.com/getsentry/sentry-go from 0.29.0 to 0.29.1 ( #963 )
...
Bumps [github.com/getsentry/sentry-go](https://github.com/getsentry/sentry-go ) from 0.29.0 to 0.29.1.
- [Release notes](https://github.com/getsentry/sentry-go/releases )
- [Changelog](https://github.com/getsentry/sentry-go/blob/master/CHANGELOG.md )
- [Commits](https://github.com/getsentry/sentry-go/compare/v0.29.0...v0.29.1 )
---
updated-dependencies:
- dependency-name: github.com/getsentry/sentry-go
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-15 04:39:20 +00:00
29721cd1f0
Feat bulk workflows ( #940 )
...
Adds support for inserting workflows in bulk via the API and an optional buffered insert on the engine.
2024-10-14 15:35:29 -04:00
2519b71b9e
fix: concurrency ( #962 )
...
* fix: concurrency
* fix: counts
* fix: try lock
2024-10-14 18:44:53 +00:00
f7329110c1
chore(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc ( #961 )
...
Bumps [go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc](https://github.com/open-telemetry/opentelemetry-go ) from 1.30.0 to 1.31.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases )
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md )
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.30.0...v1.31.0 )
---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-14 06:12:43 +00:00
90cf6ed8f5
chore(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlptrace ( #959 )
...
Bumps [go.opentelemetry.io/otel/exporters/otlp/otlptrace](https://github.com/open-telemetry/opentelemetry-go ) from 1.30.0 to 1.31.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases )
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md )
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.30.0...v1.31.0 )
---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/exporters/otlp/otlptrace
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-14 05:51:09 +00:00
5f912d20a1
chore(deps): bump go.opentelemetry.io/otel/sdk from 1.30.0 to 1.31.0 ( #958 )
...
Bumps [go.opentelemetry.io/otel/sdk](https://github.com/open-telemetry/opentelemetry-go ) from 1.30.0 to 1.31.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases )
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md )
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.30.0...v1.31.0 )
---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/sdk
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-14 05:29:26 +00:00
c8711f7f83
fix: id constraint ( #957 )
...
* fix: id constraint
* chore: gen
2024-10-11 18:00:12 -04:00
6af75638f2
feat: add helpful context to alert email ( #954 )
2024-10-11 09:53:28 -04:00
3340ec8626
fix: event keys ( #951 )
...
* feat: insert unique event keys
* fix: list query
* feat: bulk
* chore: gen
2024-10-10 08:54:52 -04:00
95558138a4
chore: improve throughput, remove deadlocks ( #949 )
...
* add otel to pub
* temporarily remove tenant id exchange
* fix: increase internal queue throughput
* fix: remove potential deadlocking
* rollback hash factor multiplier
* fix: batch update issues
* fix: rm unneeded locks
* move disable tenant pubsub to an env var
---------
Co-authored-by: gabriel ruttner <gabriel.ruttner@gmail.com >
2024-10-10 08:54:34 -04:00
089b2b5e33
chore(deps): bump google.golang.org/api from 0.199.0 to 0.200.0 ( #955 )
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.199.0 to 0.200.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.199.0...v0.200.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-10 04:54:21 +00:00
Gabe Ruttner