Commit Graph

654 Commits

Author SHA1 Message Date
Gabe Ruttner
b4670af138 Fix qos otel config (#754)
* feat: otel trace id ratio

* feat: rabbitmq qos

* feat: requeue limit

* fix: tests
v0.41.1
2024-07-30 18:11:10 -04:00
Gabe Ruttner
b802f9f45f feat: stream by addl meta (#751)
* feat: prop schedule and run

* wip

* fix: filter wfrid

* feat: hangup

* chore: rm debug log

* chore: func name

* fix: cancelled payload

* fix: load

* fix: cleanup the cahce

* fix: single proto

* fix: key -> val

* chore: case

* chore: rm dead code

* chore: rm dead code

* feat: go and docs

* fix: docs
v0.41.0
2024-07-29 19:09:51 +00:00
abelanger5
a245151d91 feat: add workflow kind to workflow versions (#750)
* feat: support workflow kinds

* chore: generate
2024-07-29 12:07:34 -07:00
Gabe Ruttner
5c5f1c5b7b feat: prop schedule and run (#749) 2024-07-29 06:38:53 -07:00
Jürgen Brandstetter
01a66dcf05 The tab order for python/typescript/go was wrong (#753) 2024-07-29 12:51:28 +00:00
abelanger5
9efd9368fd feat: deduplicated enqueue error and additional context methods (#747)
* feat: additional context fields and dedupe error

* fix: case on error properly
v0.40.0
2024-07-26 18:32:56 +00:00
abelanger5
16827223da fix: allow setting 0 replicas (#746) 2024-07-26 10:58:25 -07:00
abelanger5
aafdd278db make max msg size configurable (#745) 2024-07-26 10:58:16 -07:00
Gabe Ruttner
fd947cb5bc feat: go worker assignment (#741)
* feat: create worker with label

* feat: worker context

* feat: dynamic labels

* feat: affinity

* fix: ptr

* fix: nil labels

* feat: sticky dag

* feat: sticky docs

* feat: sticky children

* chore: lint

* fix: tests

* fix: possibly nil workerId

* chore: cleanup unneeded pointers
2024-07-26 10:19:11 -07:00
abelanger5
1ea4dfc5de feat: deduplicated enqueue (#735)
* wip

* wip: functional query

* feat: expose affinity config

* feat: add weight to proto

* feat: upsert affinity state on worker start

* fix: linting

* feat: add upsert proto

* feat: upsert handler

* feat: revise model

* fix: labels

* feat: functional desired worker

* wip: ui

* feat: add state to step run events

* fix: filter empty keys

* fix: labels as badges

* feat: empty state and descriptive text

* chore: add todo

* chore: whitespace

* chore: cleanup

* chore: cleanup

* chore: fix hash

* chore: squash migrations

* fix: fair worker assignment

* fix: remaining slots on valid desired workers

* wip: sticky

* fix: count slots

* chore: rm log line

* feat: expose sticky config

* wip: sticky dag

* feat: expose desired worker id to trigger

* feat: trigger on desired worker

* feat: typescript docs

* feat: sticky python

* feat: py sticky children

* wip: py affinity

* serverless note

* feat: complete python examples

* linting

* feat: deduplicated enqueue

* fix: address changes from PR review

* chore: generate

---------

Co-authored-by: gabriel ruttner <gabriel.ruttner@gmail.com>
2024-07-26 16:47:46 +00:00
Gabe Ruttner
2711fb84cb fix: small resolver (#742) 2024-07-26 09:31:09 -07:00
abelanger5
77193df928 feat: expose additional fields on assigned action (#738) 2024-07-26 09:17:05 -07:00
abelanger5
c0b01f1b9b fix: workflow runs replays and show workflow run input (#744) 2024-07-25 17:35:10 +00:00
dependabot[bot]
77ac85b35b chore(deps): bump google.golang.org/api from 0.188.0 to 0.189.0 (#739)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.188.0 to 0.189.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.188.0...v0.189.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-23 05:25:42 +00:00
Gabe Ruttner
ee68786d69 feat: sticky workers (#695)
* wip

* wip: functional query

* feat: expose affinity config

* feat: add weight to proto

* feat: upsert affinity state on worker start

* fix: linting

* feat: add upsert proto

* feat: upsert handler

* feat: revise model

* fix: labels

* feat: functional desired worker

* wip: ui

* feat: add state to step run events

* fix: filter empty keys

* fix: labels as badges

* feat: empty state and descriptive text

* chore: add todo

* chore: whitespace

* chore: cleanup

* chore: cleanup

* chore: fix hash

* chore: squash migrations

* fix: fair worker assignment

* fix: remaining slots on valid desired workers

* wip: sticky

* fix: count slots

* chore: rm log line

* feat: expose sticky config

* wip: sticky dag

* feat: expose desired worker id to trigger

* feat: trigger on desired worker

* feat: typescript docs

* feat: sticky python

* feat: py sticky children

* wip: py affinity

* serverless note

* feat: complete python examples

* linting

* fix: doc link

* chore: rm debug log

* fix: simplify list labels

* fix: typo
2024-07-22 17:20:23 -04:00
abelanger5
f8981809f7 fix: soft delete workflows causes collision on new workflow inserts (#737) v0.39.1 2024-07-22 12:06:57 -04:00
dependabot[bot]
89ddce5ebf chore(deps): bump github.com/go-co-op/gocron/v2 from 2.10.1 to 2.11.0 (#736)
Bumps [github.com/go-co-op/gocron/v2](https://github.com/go-co-op/gocron) from 2.10.1 to 2.11.0.
- [Release notes](https://github.com/go-co-op/gocron/releases)
- [Commits](https://github.com/go-co-op/gocron/compare/v2.10.1...v2.11.0)

---
updated-dependencies:
- dependency-name: github.com/go-co-op/gocron/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-22 04:48:10 +00:00
Gabe Ruttner
ad29edb44f fix: partitioned semaphore resolver (#731)
* fix: partition and improve query

* feat: paginate until done

* chore: address comments

* fix: write partitions
v0.39.0
2024-07-18 11:06:25 -04:00
abelanger5
a5724fc4f9 feat: replay workflow runs (#732)
* feat: replay workflow runs

* address pr review changes
2024-07-18 08:05:15 -07:00
Gabe Ruttner
b7cec9ec53 feat: soft delete (#717)
* feat: soft delete workflows and versions

* feat: filter soft deletes wf and wfr

* feat: filter events and step runs

* fix: query

* fix: query

* chore: generate

* wip

* chore: squash migrations

* chore: separate retention into new service

* feat: regularly clean up

* chore: migrations

* fix: tests

* fix: queries

* fix: ambiguous

* fix: refs

* fix: ambiguous id

* fix: remove update from

* fix: soft delete

* fix: cleanup retention scheduler

* fix: has more query

* chore: gen

* fix: query

* fix: table
2024-07-18 09:06:05 -04:00
dependabot[bot]
0646b46969 chore(deps): bump github.com/go-co-op/gocron/v2 from 2.10.0 to 2.10.1 (#730)
Bumps [github.com/go-co-op/gocron/v2](https://github.com/go-co-op/gocron) from 2.10.0 to 2.10.1.
- [Release notes](https://github.com/go-co-op/gocron/releases)
- [Commits](https://github.com/go-co-op/gocron/compare/v2.10.0...v2.10.1)

---
updated-dependencies:
- dependency-name: github.com/go-co-op/gocron/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-18 04:36:08 +00:00
dependabot[bot]
0d8633d1cf chore(deps): bump github.com/slack-go/slack from 0.13.0 to 0.13.1 (#727)
Bumps [github.com/slack-go/slack](https://github.com/slack-go/slack) from 0.13.0 to 0.13.1.
- [Release notes](https://github.com/slack-go/slack/releases)
- [Changelog](https://github.com/slack-go/slack/blob/master/CHANGELOG.md)
- [Commits](https://github.com/slack-go/slack/compare/v0.13.0...v0.13.1)

---
updated-dependencies:
- dependency-name: github.com/slack-go/slack
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-17 04:40:37 +00:00
Gabe Ruttner
2427707098 Pr 629 (#705)
* docs: added concurrency examples

* docs: added more examples in user guide

* lint: fixed linting and build issues

* docs: added more examples for user guide features

* lint: fixed pre-commit lint issues

* lint: prettier issues fixed

* lint: fixed linting and formatting issues

* lint: formatting fix

* docs: replace deprecated on method in user guide with register workflow and added tabs in cancellation mechanisms

* docs: fixed go sdk registerworkflow params in user guide

* lint-fix: Removed four backticks and random whitespaces in documentation (#715)

---------

Co-authored-by: Akshat Sharma <akshatsharma2602@gmail.com>
Co-authored-by: Akshat Sharma <35724794+akshat2602@users.noreply.github.com>
2024-07-16 14:56:28 +00:00
abelanger5
5d87f380ef feat: managed worker pools (#725)
* change api extension spec to register custom populators

* fix: support only bearer auth

* fix: correct authn logic

* fix: indexes on workflow runs, events

* feat: managed worker pools

* chore: lint fix

* hide workers view when not enabled

* support internal api tokens, minor improvements

* fix: actually write internal

* fix breaking changes

* don't allow revoking internal tokens

* fix: linting and remove metrics view

* fix: token

* address review and add feat flags
v0.38.0
2024-07-16 13:33:46 +00:00
Gabe Ruttner
38a4ce647d feat: improved workflow runs list (#723)
* feat: add duration to wfr

* chore: generate

* feat: sort order

* feat: add column visibility toggle

* chore: gen
v0.37.0
2024-07-15 18:05:57 -04:00
Gabe Ruttner
c935a38708 fix: revert failed assigned grpc stream and write event (#726)
* fix: revert and write event

* fix: write critical event

* fix: unassign repository method

* fix: remove unneeded event
2024-07-15 18:05:49 -04:00
Gabe Ruttner
11cd330fc3 fix: timeout assigned (#724)
* fix: timeout assigned

* fix: any instead of or
v0.36.11
2024-07-15 10:34:54 -04:00
Gabe Ruttner
d9ce67bf89 fix: remove extra ports (#721) v0.36.10 2024-07-15 09:17:50 -04:00
Gabe Ruttner
d866f8167d fix: add job run lookup data to data query (#722) 2024-07-15 09:17:42 -04:00
dependabot[bot]
2d51d95341 chore(deps): bump github.com/go-co-op/gocron/v2 from 2.9.0 to 2.10.0 (#720)
Bumps [github.com/go-co-op/gocron/v2](https://github.com/go-co-op/gocron) from 2.9.0 to 2.10.0.
- [Release notes](https://github.com/go-co-op/gocron/releases)
- [Commits](https://github.com/go-co-op/gocron/compare/v2.9.0...v2.10.0)

---
updated-dependencies:
- dependency-name: github.com/go-co-op/gocron/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-15 05:06:56 +00:00
Gabe Ruttner
f4f752b5d2 fix: large step run payloads (#719)
* wip: just needs working transformation

* fix: refactor data into separate query
v0.36.9
2024-07-14 10:27:25 -04:00
abelanger5
fee4c63319 fix: infinite meta loading loop (#718)
* fix: infinite meta loading loop

* fix: workflow run id parsing
v0.36.8
2024-07-12 14:39:01 -04:00
Gabe Ruttner
af705a0aea fix: resolve unresolved failed steps (#700)
* feat: resolve unresolved failed steps

* fix: poll active queues

* chore: comment temp unused
v0.36.7
2024-07-12 14:14:29 -04:00
abelanger5
8f8f3ad287 fix: reduce max throughput of requeue (#713)
* fix: reduce max throughput of requeue

* fix: reassign query

* fix: move step run timeout to partition model

* fix: partitioning queries and index

* better logs on requeue

* fix: inactive rebalance and get step run for engine query

* fix: correct inactive queries
2024-07-12 14:03:55 -04:00
dependabot[bot]
12d895cc84 chore(deps): bump github.com/steebchen/prisma-client-go (#668)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: steebchen <contact@luca-steeb.com>
2024-07-11 11:31:11 +00:00
dependabot[bot]
5d7797f60d chore(deps): bump github.com/go-co-op/gocron/v2 from 2.8.0 to 2.9.0 (#714)
Bumps [github.com/go-co-op/gocron/v2](https://github.com/go-co-op/gocron) from 2.8.0 to 2.9.0.
- [Release notes](https://github.com/go-co-op/gocron/releases)
- [Commits](https://github.com/go-co-op/gocron/compare/v2.8.0...v2.9.0)

---
updated-dependencies:
- dependency-name: github.com/go-co-op/gocron/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-11 05:03:37 +00:00
abelanger5
a96738539e fix: simplify reassign query (#712)
* fix: simplify reassign query

* chore: lint and rehash
v0.36.6
2024-07-10 19:38:02 -04:00
abelanger5
fcb3a88078 fix: improved performance of reassign/requeue (#711)
* fix: improved performance of reassign/requeue

* chore: generate
v0.36.5
2024-07-10 17:04:16 -04:00
Gabe Ruttner
521cf3e2aa feat: filter chips (#709) 2024-07-10 10:23:41 -07:00
Gabe Ruttner
1e20bf946a fix: improved assign, reassign, and requeue (#702)
* fix: improved queries

* fix: 1s timeout

* fix: indexes

* fix: increase timeout to 4s

* fix: migration

* merge in db changes

* chore: squash migrations

* chore: re-hash

* chore: remove comment

* chore: rm unused query

* fix: state

* fix: check valid workers before commit

* fix: query

* chore: gen
v0.36.4
2024-07-10 12:45:08 -04:00
abelanger5
560ac1ba5c fix: patch list queries to only count 10k rows (#708)
* fix: patch list queries to only count 10k rows

* fix: from

---------

Co-authored-by: gabriel ruttner <gabriel.ruttner@gmail.com>
v0.36.3
2024-07-10 11:13:13 -04:00
Gabe Ruttner
16d52277f1 fix: steps to timeout (#707)
* fix: sync prisma state to atlas state

* chore: gen

* chore: gen again

* fix: rm ticker indexes

* chore: gen

* fix: improved query and indexes
2024-07-10 11:12:51 -04:00
Gabe Ruttner
7dc274bbd6 fix: remove expired crons (#697)
Co-authored-by: abelanger5 <belanger@sas.upenn.edu>
2024-07-10 11:01:40 -04:00
Gabe Ruttner
e420f4960f fix: sync prisma state to atlas state (#706)
* fix: sync prisma state to atlas state

* chore: gen

* chore: gen again

* fix: rm ticker indexes

* chore: gen
2024-07-10 07:55:56 -07:00
Gabe Ruttner
f5bdc409d4 fix: reverse migration (#703)
* fix: reverse migration

* chore: hash
2024-07-10 04:03:25 -07:00
dependabot[bot]
37b00a730a chore(deps): bump google.golang.org/api from 0.187.0 to 0.188.0 (#704)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.187.0 to 0.188.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.187.0...v0.188.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-10 04:43:40 +00:00
dependabot[bot]
d6f723613e chore(deps): bump golang.org/x/crypto from 0.24.0 to 0.25.0 (#698)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.24.0 to 0.25.0.
- [Commits](https://github.com/golang/crypto/compare/v0.24.0...v0.25.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-08 05:06:11 +00:00
dependabot[bot]
4f8b84cde9 chore(deps): bump github.com/getkin/kin-openapi from 0.125.0 to 0.126.0 (#699)
Bumps [github.com/getkin/kin-openapi](https://github.com/getkin/kin-openapi) from 0.125.0 to 0.126.0.
- [Release notes](https://github.com/getkin/kin-openapi/releases)
- [Commits](https://github.com/getkin/kin-openapi/compare/v0.125.0...v0.126.0)

---
updated-dependencies:
- dependency-name: github.com/getkin/kin-openapi
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-08 05:05:53 +00:00
Gabe Ruttner
ccd38f0506 fix: step run queries (#696)
* fix: rm unused count

* fix: indexes

* fix: limit PollStepRuns limit

* fix: create concurrently if not exists

* chore: gen

* fix: txmode

* fix: dont run expired crons

* revert: cron

* chore: rename migration

* chore: gen
v0.36.1
2024-07-06 20:12:28 -04:00
abelanger5
f36e66cd28 feat: configurable data retention period (#693)
* feat: data retention for tenants

* chore: generate and docs

* chore: lint
v0.36.0
2024-07-06 14:31:12 +00:00