Commit Graph

733 Commits

Author SHA1 Message Date
Gabe Ruttner 460438ab22 fix: tag (#862) 2024-09-10 09:31:39 -07:00
abelanger5 dda47fb24f fix: dispatcher and teardown errors (#856) 2024-09-10 12:08:08 -04:00
abelanger5 9efcebe6af fix: better logic for multiple restricted domains (#860) 2024-09-10 12:07:55 -04:00
abelanger5 77ab9460d3 fix: unset statement timeouts for session, print more debug info (#861) 2024-09-10 12:07:47 -04:00
dependabot[bot] 91e68f978c chore(deps): bump google.golang.org/grpc from 1.66.0 to 1.66.1 (#859)
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.66.0 to 1.66.1.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.66.0...v1.66.1)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-10 05:09:13 +00:00
abelanger5 a1324d43db fix: improve status updates and event writes (#855)
* fix: improve status updates and event writes

* chore: better debug line
v0.44.4
2024-09-09 18:09:44 +00:00
abelanger5 e59ce100e3 fix: cleanup worker assign events + limit semaphore query (#854) v0.44.3 2024-09-09 11:11:44 -04:00
dependabot[bot] 8f511c806f chore(deps): bump github.com/go-playground/validator/v10 (#853)
Bumps [github.com/go-playground/validator/v10](https://github.com/go-playground/validator) from 10.22.0 to 10.22.1.
- [Release notes](https://github.com/go-playground/validator/releases)
- [Commits](https://github.com/go-playground/validator/compare/v10.22.0...v10.22.1)

---
updated-dependencies:
- dependency-name: github.com/go-playground/validator/v10
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-09 04:48:06 +00:00
abelanger5 23daa89ba0 fix: list initial step runs query + better serialized queue operations (#851)
* fix: list initial step runs

* fix: better tenant operation logic
v0.44.2
2024-09-08 19:29:11 -04:00
abelanger5 478c897035 fix: proper migration to counts, startable step runs query (#850) v0.44.1 2024-09-08 12:48:51 -04:00
abelanger5 70855ecc6b refactor: bulk step run updates (#847)
* wip: v4 of queue

* fix: correct query for updating counts

* tmp: save migration files

* feat: wrap up initial queue

* fix compilation

* fix: reassigns

* temp: refactor on step runs

* feat: bulk step run updates and processing

* fix: cancellation

* address review comments
v0.44.0
2024-09-06 17:11:19 -04:00
abelanger5 bb459d1f7f fix: rendering additional metadata (#846) 2024-09-06 17:11:10 -04:00
abelanger5 891514b461 feat: queue v4 (#842)
* wip: v4 of queue

* fix: correct query for updating counts

* tmp: save migration files

* feat: wrap up initial queue

* fix compilation

* fix: reassigns
2024-09-06 16:12:22 -04:00
dependabot[bot] c720311aba chore(deps): bump golang.org/x/crypto from 0.26.0 to 0.27.0 (#845)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.26.0 to 0.27.0.
- [Commits](https://github.com/golang/crypto/compare/v0.26.0...v0.27.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-06 05:10:06 +00:00
dependabot[bot] 33d7ed471e chore(deps): bump golang.org/x/oauth2 from 0.22.0 to 0.23.0 (#844)
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.22.0 to 0.23.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.22.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-05 05:24:39 +00:00
dependabot[bot] ad2f653f70 chore(deps): bump golang.org/x/text from 0.17.0 to 0.18.0 (#843)
Bumps [golang.org/x/text](https://github.com/golang/text) from 0.17.0 to 0.18.0.
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.17.0...v0.18.0)

---
updated-dependencies:
- dependency-name: golang.org/x/text
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-05 05:24:27 +00:00
dependabot[bot] 191cbf310f chore(deps): bump google.golang.org/api from 0.195.0 to 0.196.0 (#841)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.195.0 to 0.196.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.195.0...v0.196.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-04 05:20:44 +00:00
abelanger5 7308876776 fix: use separate database pool for queueing, statement timeouts on tx (#839)
* fix: different queue pool and statement timeouts on step runs

* fix: implement prepareTx

* fix: defer rollback properly

* fix: race condition
2024-09-03 21:07:26 +00:00
dependabot[bot] 610edf76c3 chore(deps): bump github.com/steebchen/prisma-client-go (#838)
Bumps [github.com/steebchen/prisma-client-go](https://github.com/steebchen/prisma-client-go) from 0.39.0 to 0.40.1.
- [Release notes](https://github.com/steebchen/prisma-client-go/releases)
- [Commits](https://github.com/steebchen/prisma-client-go/compare/v0.39.0...v0.40.1)

---
updated-dependencies:
- dependency-name: github.com/steebchen/prisma-client-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-03 05:00:21 +00:00
dependabot[bot] 026c5d0dcc chore(deps): bump github.com/posthog/posthog-go from 1.2.20 to 1.2.21 (#837)
Bumps [github.com/posthog/posthog-go](https://github.com/posthog/posthog-go) from 1.2.20 to 1.2.21.
- [Release notes](https://github.com/posthog/posthog-go/releases)
- [Changelog](https://github.com/PostHog/posthog-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/posthog/posthog-go/compare/v1.2.20...v1.2.21)

---
updated-dependencies:
- dependency-name: github.com/posthog/posthog-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-30 04:49:28 +00:00
abelanger5 b5014f6b3d chore: more visibility and debug lines for queues (#836)
* chore: more visibility and debug options for queues

* better debug lines on queue repo

* don't log so much in load test
v0.43.2
2024-08-29 14:49:24 -04:00
dependabot[bot] 96c040ebb7 chore(deps): bump google.golang.org/api from 0.194.0 to 0.195.0 (#832)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.194.0 to 0.195.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.194.0...v0.195.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-29 05:55:26 +00:00
dependabot[bot] d1162fa99f chore(deps): bump google.golang.org/grpc from 1.65.0 to 1.66.0 (#833)
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.65.0 to 1.66.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.65.0...v1.66.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-29 05:32:51 +00:00
abelanger5 17b7e84876 fix: delete queue items when no longer used (#831) 2024-08-28 17:12:31 -04:00
abelanger5 263eaf069b feat: pass otel through msgqueue (#802)
* feat: pass otel through msgqueue

* feat: more spans on scheduling

* otel increase batch size
v0.43.1
2024-08-28 14:45:02 +00:00
dependabot[bot] f6e937f23b chore(deps): bump github.com/steebchen/prisma-client-go (#829)
Bumps [github.com/steebchen/prisma-client-go](https://github.com/steebchen/prisma-client-go) from 0.38.0 to 0.39.0.
- [Release notes](https://github.com/steebchen/prisma-client-go/releases)
- [Commits](https://github.com/steebchen/prisma-client-go/compare/v0.38.0...v0.39.0)

---
updated-dependencies:
- dependency-name: github.com/steebchen/prisma-client-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-28 04:47:32 +00:00
dependabot[bot] 69ab29a536 chore(deps): bump github.com/Masterminds/semver/v3 from 3.2.1 to 3.3.0 (#828)
Bumps [github.com/Masterminds/semver/v3](https://github.com/Masterminds/semver) from 3.2.1 to 3.3.0.
- [Release notes](https://github.com/Masterminds/semver/releases)
- [Changelog](https://github.com/Masterminds/semver/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Masterminds/semver/compare/v3.2.1...v3.3.0)

---
updated-dependencies:
- dependency-name: github.com/Masterminds/semver/v3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-28 04:47:14 +00:00
abelanger5 6317f86793 refactor: consolidate partition logic (#826)
* refactor: consolidate partition logic

* fix: race on scheduler

* fix: move partition uuid to db query

* fix: generate
2024-08-27 15:28:53 -04:00
dependabot[bot] cf4c07daa4 chore(deps): bump github.com/posthog/posthog-go from 1.2.19 to 1.2.20 (#825)
Bumps [github.com/posthog/posthog-go](https://github.com/posthog/posthog-go) from 1.2.19 to 1.2.20.
- [Release notes](https://github.com/posthog/posthog-go/releases)
- [Changelog](https://github.com/PostHog/posthog-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/posthog/posthog-go/compare/v1.2.19...v1.2.20)

---
updated-dependencies:
- dependency-name: github.com/posthog/posthog-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-27 05:38:04 +00:00
Gabe Ruttner 526e7ef308 feat: expose priority queue (#814)
* feat: workflow default priority

* feat: write priority on run

* feat: propagate to queue

* chore: squash migrations

* chore: generate
v0.43.0
2024-08-26 14:11:28 -04:00
abelanger5 2d36507dfd feat: add managed worker regions (#819)
* feat: add managed worker regions

* fix: enum/validation on regions
2024-08-26 14:03:38 -04:00
abelanger5 c37c5b4664 fix: stream by additional metadata terminating early (#820) 2024-08-26 11:59:49 -04:00
dependabot[bot] 07bf3efeaf chore(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc (#817)
Bumps [go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc](https://github.com/open-telemetry/opentelemetry-go) from 1.28.0 to 1.29.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.28.0...v1.29.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-26 05:15:01 +00:00
dependabot[bot] c4ef92d1b6 chore(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlptrace (#815)
Bumps [go.opentelemetry.io/otel/exporters/otlp/otlptrace](https://github.com/open-telemetry/opentelemetry-go) from 1.28.0 to 1.29.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.28.0...v1.29.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/exporters/otlp/otlptrace
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-26 04:53:27 +00:00
Gabe Ruttner ee5d86796f fix: required affinity (#812)
* fix: required affinity

* chore: rm dead code
v0.42.14
2024-08-23 15:19:29 -04:00
Gabe Ruttner 53be615d5f Enhancement webhook usability (#807)
* feat: secret copier

* feat: improved form

* fix: quotes

* wip: improved flow

* feat: health check logging

* fix: page design

* fix: hard delete, no upsert

* fix: reset modal state

* fix: empty text

* fix: worker state

* fix: update only token

* fix: dont delete name

* fix: logs component

* fix: sort order

* chore: build

* fix: webhook worker cleanup

* chore: squash migrations

* Update api-contracts/openapi/paths/webhook-worker/webhook-worker.yaml

Co-authored-by: abelanger5 <belanger@sas.upenn.edu>

* chore: rename

* fix: wrong query

---------

Co-authored-by: abelanger5 <belanger@sas.upenn.edu>
v0.42.13
2024-08-23 10:09:09 -04:00
abelanger5 2b9121d295 fix: long token expiry for k8s quickstart (#808)
* debug overwrite

* fix: k8s 100 year expiry
2024-08-23 07:43:02 -04:00
abelanger5 dd8a4144cb fix: hard sticky assignment to workers when no desired worker id (#809) 2024-08-23 07:42:52 -04:00
dependabot[bot] 61b8b84f4c chore(deps): bump github.com/posthog/posthog-go from 1.2.18 to 1.2.19 (#811)
Bumps [github.com/posthog/posthog-go](https://github.com/posthog/posthog-go) from 1.2.18 to 1.2.19.
- [Release notes](https://github.com/posthog/posthog-go/releases)
- [Changelog](https://github.com/PostHog/posthog-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/posthog/posthog-go/compare/v1.2.18...v1.2.19)

---
updated-dependencies:
- dependency-name: github.com/posthog/posthog-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-23 04:40:57 +00:00
dependabot[bot] fdb8884813 chore(deps): bump google.golang.org/api from 0.193.0 to 0.194.0 (#810)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.193.0 to 0.194.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.193.0...v0.194.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-23 04:40:15 +00:00
abelanger5 93438ce09d feat: adds a k8s helper for easy k8s installation (#806)
* feat: k8s helper script for generating env

* chore: bump 1.21 -> 1.22

* feat: support both secret, configmap and generate api token

* fix: better errors

* fix: upsert logic

* use default seed tenant for token generation
v0.42.12
2024-08-22 21:24:16 +00:00
abelanger5 7a3c06884f fix: don't queue cancelled step runs (#805)
* fix: cancelled step runs should not be assigned

* check cancellations before planning

* remove reassign logic from controller
v0.42.11
2024-08-22 11:21:49 -04:00
abelanger5 e40d77d9d7 fix: reassignment should reset scheduling timeout (#804) 2024-08-22 10:07:10 -04:00
Gabe Ruttner 9bea55438a Fix webhook healthcheck race (#797)
* fix: race

* fix: partition no rows

* chore: move to workers tab

* feat: redirect empty worker path to all

* chore: add worker type and webhook id

* fix: upsert webhook worker

* fix: update by webhookId

* fix: only stub on create

* feat: url on worker

* chore: migration version

* fix: move

* fix: upsert

* fix: upert

* chore: fix migration

* fix: migrations

* chore: generate
v0.42.10
2024-08-21 19:23:24 +00:00
Gabe Ruttner 46956a4153 fix: sort (#803) v0.42.9 2024-08-21 17:55:15 +00:00
Gabe Ruttner 651be542c3 fix: migration state (#800)
* fix: migration state

* almost there...

* fix: hack for constraints

* chore: lint
2024-08-21 17:07:00 +00:00
abelanger5 47d2c4edb4 fix: concurrent rw on map (#796)
* fix: concurrent rw on map

* write correct map
2024-08-21 12:58:06 -04:00
abelanger5 6b7e6de4c4 fix: use single queue limit (#801) 2024-08-21 12:46:05 -04:00
dependabot[bot] 2cca30bab7 chore(deps): bump google.golang.org/api from 0.192.0 to 0.193.0 (#798)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.192.0 to 0.193.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.192.0...v0.193.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-21 04:33:36 +00:00
abelanger5 84f7334a06 feat: add windows for metrics and selector (#794)
* feat: add windows for metrics and selector

* better placeholder
v0.42.6
2024-08-20 15:29:32 +00:00