Commit Graph

920 Commits

Author SHA1 Message Date
Sean Reilly 42afe083cf Partition Step Run and Remove Prisma (#982)
* add in the migration for now

* Update step_runs.sql

remove TODO

* change the schema so we don't undo it

* add the migration for step run partition. remove prisma. add a helper task for recreating the db

* do a manual merge of the schema.sql

* add in the serial

* update docs

* PR feedback

* add Identity to all tables that don't have a Bigserial

* do the atlas hash with the new migration

* squash the migrations

---------

Co-authored-by: Sean Reilly <sean@hatchet.run>
2024-11-20 15:20:36 -08:00
Sean Reilly b5de6e26ff Add a dynamic strategy for flushing as a function of currently flushing (#1055)
* add a dynamic strategy for flushing where we make the trigger for flush a funciton of the depth of the concurrency

* default value for tests and New for FlushStrategy

* clean up the currently flushing locking and add deadlock.Mutex

* don't wait as long for the buffer

* lets see if this 2ms thing is what is causing things to break

* lets error for this to see if we are actually hitting these limits

* put a really short deadline on the lock timeout to see if github actions will blow up

* lets use RW mutexs se we don't block as much

* lets extend this out to 100ms

* lets just do fewer locks

* add a lock to prevent a queue behind the semaphore

* deal with potential data races

* a simpler loop fib and now locks

* lets get rid of the wait for flush

* remove the deadlock stuff

* mod tidy

---------

Co-authored-by: Sean Reilly <sean@hatchet.run>
2024-11-20 19:49:30 +00:00
abelanger5 2d54fa0dd1 Update postgres-events-table.mdx 2024-11-20 12:10:40 -05:00
abelanger5 980649e673 blog: events table in postgres (#1059)
* blog: events table in postgres

* flip index order

* address feedback
2024-11-20 11:32:19 -05:00
gabriel ruttner 1d22ee312b docs: fix webhook navigation 2024-11-20 09:34:06 -05:00
dependabot[bot] 7e3f41a7bf chore(deps): bump github.com/Masterminds/semver/v3 from 3.3.0 to 3.3.1 (#1058)
Bumps [github.com/Masterminds/semver/v3](https://github.com/Masterminds/semver) from 3.3.0 to 3.3.1.
- [Release notes](https://github.com/Masterminds/semver/releases)
- [Changelog](https://github.com/Masterminds/semver/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Masterminds/semver/compare/v3.3.0...v3.3.1)

---
updated-dependencies:
- dependency-name: github.com/Masterminds/semver/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-20 05:12:40 +00:00
dependabot[bot] bbf0abe373 chore(deps): bump google.golang.org/api from 0.206.0 to 0.207.0 (#1057)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.206.0 to 0.207.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.206.0...v0.207.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-20 05:12:17 +00:00
Gabe Ruttner c88a18180a docs: git driven snips -- on failure page (#1043)
* feat: universal tabs

* fix: consistent tab naming

* feat: paths

* wip: chunk parser

* wip: dynamic snips

* wip: components

* docs: annotated go

* docs: improved annotations

* chore: fix

* feat: on-failure

* todo: switch branch to main

* chore: lint

* chore: lint

* fix: resolved theme

* cache

* fix: theme again...

* todo list

* docs: shadcn

* docs: code example uses shadcn

* fix: build

* python note

* chore: lint

* chore: swap main

* feat: local story

* fix: is dev

* revert the change that broke vercel

* fix: redirect?

* feat: seo title
2024-11-18 15:50:51 -08:00
dependabot[bot] 6828f5c818 chore(deps): bump github.com/go-playground/validator/v10 (#1054)
Bumps [github.com/go-playground/validator/v10](https://github.com/go-playground/validator) from 10.22.1 to 10.23.0.
- [Release notes](https://github.com/go-playground/validator/releases)
- [Commits](https://github.com/go-playground/validator/compare/v10.22.1...v10.23.0)

---
updated-dependencies:
- dependency-name: github.com/go-playground/validator/v10
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-18 05:06:57 +00:00
dependabot[bot] 03e92ebe97 chore(deps): bump github.com/exaring/otelpgx from 0.6.2 to 0.7.0 (#1053)
Bumps [github.com/exaring/otelpgx](https://github.com/exaring/otelpgx) from 0.6.2 to 0.7.0.
- [Release notes](https://github.com/exaring/otelpgx/releases)
- [Commits](https://github.com/exaring/otelpgx/compare/v0.6.2...v0.7.0)

---
updated-dependencies:
- dependency-name: github.com/exaring/otelpgx
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-18 05:06:51 +00:00
abelanger5 ae5df5b88d fix: make race condition on reassignment more rare (#1052)
* fix: make race condition on reassignment more rare

* fix: proper concurrency on bulk dispatch

* prevent concurrent err assignments
v0.51.5
2024-11-15 14:17:51 -05:00
Gabe Ruttner 961e8840c0 feat: universal doc tabs (#1041)
* feat: universal tabs

* fix: consistent tab naming

* feat: paths
2024-11-15 11:16:51 -08:00
abelanger5 faff6001a8 fix: propagate schedule timeouts to children (#1051) 2024-11-15 10:07:33 -05:00
Sean Reilly d7d80393c3 add some logging so it is easier to see what grpc rate limits are set (#1045)
Co-authored-by: Sean Reilly <sean@hatchet.run>
v0.51.4
2024-11-15 09:20:15 -05:00
abelanger5 11ec54a18d fix: typescript typo on onboarding (#1042) 2024-11-15 09:19:45 -05:00
abelanger5 c40b9154d8 fix: tenant race conditions, cleanup logic, old workers getting assigned (#1050) 2024-11-15 09:19:36 -05:00
Gabe Ruttner 4eaa9e7fd9 feat: configurable internal retry (#1049)
* feat: configurable internal retry

* fix: bump default to 3
2024-11-15 09:19:24 -05:00
abelanger5 f22c49652c Update migrating-off-prisma.mdx 2024-11-15 06:32:39 -05:00
dependabot[bot] 78c333a734 chore(deps): bump google.golang.org/api from 0.205.0 to 0.206.0 (#1048)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.205.0 to 0.206.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.205.0...v0.206.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-15 04:45:07 +00:00
dependabot[bot] 00337dd639 chore(deps): bump google.golang.org/protobuf from 1.35.1 to 1.35.2 (#1047)
Bumps google.golang.org/protobuf from 1.35.1 to 1.35.2.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-15 04:44:57 +00:00
Sean Reilly 9a5acc5179 modify the Event created at to be a clock_timestamp instead of a transaction timestamp so we maintain ordering of inserted events - also extend the length of the timestamp so we have enough significant bits (#1044)
* add the migration for the timestamp and clock

* regenerate

---------

Co-authored-by: Sean Reilly <sean@hatchet.run>
2024-11-14 11:15:45 -08:00
Alexander Belanger a1e2a36330 filename 2024-11-12 11:01:35 -05:00
Alexander Belanger 99a8675727 url 2024-11-12 10:47:22 -05:00
dependabot[bot] f965bb2f5d chore(deps): bump github.com/steebchen/prisma-client-go (#1040)
Bumps [github.com/steebchen/prisma-client-go](https://github.com/steebchen/prisma-client-go) from 0.42.0 to 0.43.0.
- [Release notes](https://github.com/steebchen/prisma-client-go/releases)
- [Commits](https://github.com/steebchen/prisma-client-go/compare/v0.42.0...v0.43.0)

---
updated-dependencies:
- dependency-name: github.com/steebchen/prisma-client-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-12 04:39:45 +00:00
abelanger5 7e7e4ee406 blog: moving off prisma (#1034)
* blog: moving off prisma

* footnote improvement

* add link to prisma

* add sqlc link

* footnote spacing

* minor nits
2024-11-11 09:30:37 -05:00
dependabot[bot] 347c75cc10 chore(deps): bump golang.org/x/crypto from 0.28.0 to 0.29.0 (#1037)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.28.0 to 0.29.0.
- [Commits](https://github.com/golang/crypto/compare/v0.28.0...v0.29.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-11 05:13:05 +00:00
dependabot[bot] a18aefa368 chore(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc (#1036)
Bumps [go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc](https://github.com/open-telemetry/opentelemetry-go) from 1.31.0 to 1.32.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.31.0...v1.32.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-11 04:48:37 +00:00
gabriel ruttner 357e8d2c9c fix: link 2024-11-08 11:10:11 -05:00
gabriel ruttner d80140dc5f fix: path 2024-11-08 11:04:08 -05:00
gabriel ruttner 1f4715a9b3 fix: doc links 2024-11-08 10:51:52 -05:00
Gabe Ruttner 3850964a98 feat: initial doc pages (#1020)
* generate initial cloud client

* feat: initial doc pages

* feat: cloud register id, action filtering

* feat:cloud register

* fix: env var

* chore:lint

---------

Co-authored-by: Alexander Belanger <alexander@hatchet.run>
2024-11-08 07:46:43 -08:00
dependabot[bot] 07c8b7cc8d chore(deps): bump golang.org/x/time from 0.7.0 to 0.8.0 (#1033)
Bumps [golang.org/x/time](https://github.com/golang/time) from 0.7.0 to 0.8.0.
- [Commits](https://github.com/golang/time/compare/v0.7.0...v0.8.0)

---
updated-dependencies:
- dependency-name: golang.org/x/time
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
v0.51.3
2024-11-08 05:42:24 +00:00
dependabot[bot] ef0889ca5d chore(deps): bump golang.org/x/oauth2 from 0.23.0 to 0.24.0 (#1030)
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.23.0 to 0.24.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.23.0...v0.24.0)

---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-08 05:42:16 +00:00
dependabot[bot] 293913c86a chore(deps): bump golang.org/x/sync from 0.8.0 to 0.9.0 (#1032)
Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.8.0 to 0.9.0.
- [Commits](https://github.com/golang/sync/compare/v0.8.0...v0.9.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sync
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-08 05:17:51 +00:00
dependabot[bot] 80d3e72d79 chore(deps): bump golang.org/x/text from 0.19.0 to 0.20.0 (#1031)
Bumps [golang.org/x/text](https://github.com/golang/text) from 0.19.0 to 0.20.0.
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.19.0...v0.20.0)

---
updated-dependencies:
- dependency-name: golang.org/x/text
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-08 05:17:33 +00:00
abelanger5 48aadc6ace fix: avoid panics in lease manager (#1029) 2024-11-07 16:07:01 -05:00
abelanger5 780496e7fb fix: prevent infinite reassign loop (#1028) v0.51.2 2024-11-07 17:28:12 +00:00
Gabe Ruttner c531c36870 fix: filter-cancel-cases (#1027)
* fix: filter-cancel-cases

* fix: case CANCELLED_BY_CONCURRENCY_LIMIT
2024-11-07 11:18:50 -05:00
Alexander Belanger 5b59af076e fix: cancellation status propagation and minimap view 2024-11-07 11:13:14 -05:00
dependabot[bot] 0eb2bdf757 chore(deps): bump google.golang.org/api from 0.204.0 to 0.205.0 (#1026)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.204.0 to 0.205.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.204.0...v0.205.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-07 04:59:45 +00:00
dependabot[bot] c03a9f49fd chore(deps): bump github.com/google/cel-go from 0.21.0 to 0.22.0 (#1025)
Bumps [github.com/google/cel-go](https://github.com/google/cel-go) from 0.21.0 to 0.22.0.
- [Release notes](https://github.com/google/cel-go/releases)
- [Commits](https://github.com/google/cel-go/compare/v0.21.0...v0.22.0)

---
updated-dependencies:
- dependency-name: github.com/google/cel-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-07 04:35:31 +00:00
Gabe Ruttner 3871df01ee fix: dont bump deleted (#1024) 2024-11-06 16:11:36 -05:00
Gabe Ruttner 5759311574 fix: ratelimit and invalid output blocking queue (#1023)
* fix: rm unused offending code, handle unacked

* fix: handle invalid outputs

* fix: dont reset failed

* fix: case on json err

* fix: completed step run ids

* fix: scope
v0.51.1
2024-11-06 18:21:22 +00:00
abelanger5 71e01b3b5a fix: compute wording and add user callback (#1018)
* user callbacks and move location of managed workers

* rename pools to compute

* move managed workers to right fs location, remove prefix on /workers
v0.51.0
2024-11-05 20:14:57 +00:00
abelanger5 ddc99b0468 feat: infra-as-code (#953)
* feat: infra-as-code

* wrap up new metrics view

* add support for multi-region

* remove hardcoded colors

* fix: chart improvements

* fix: color

* fix: lint

* delete runtimeConfig if this is iac

* feat: add stacked area chart

* fix: proper area chart

* switch back to line charts

* feat: add iac debug info

* fix: back button on iac deploy
v0.51.0-beta.2
2024-11-05 11:39:31 -05:00
abelanger5 9d133bc15c fix: catch all nack cases for rate limits (#1015)
* fix: properly nack rate limit when failing to schedule

* more nack cases
2024-11-05 11:37:47 -05:00
abelanger5 68bc5a0197 fix: unacked messages in the queuer (#1014)
* fix: when scheduling fails with schedule timeouts, we never ack the queue item

* add error line if we don't process everything we pass into the scheduler
2024-11-05 10:27:53 -05:00
ulascanzorer 85e285df01 Fixed problem in the Python version of the example worker. (#1011)
If we run the worker without (if __name__ == __main__), I have observed that we could get the following error:



failed to start action listener: 
        An attempt has been made to start a new process before the current process has finished its bootstrapping phase. This probably means that you are not using fork to start your child processes and you have forgotten to use the proper idiomin the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

The "freeze_support()" line can be omitted if the program is not going to be frozen to produce an executable. To fix this issue, refer to the "Safe importing of main module" section in https://docs.python.org/3/library/multiprocessing.html.



Adding (if __name__ == "__main__") fixes the issue, and wouldn't break already working setups.
2024-11-04 16:31:35 -05:00
Gabe Ruttner c227960453 fix: drop e in Requeuing (#1013) 2024-11-04 16:30:38 -05:00
abelanger5 3f671ed51c fix: onClick for frontend (#1009) 2024-11-02 11:52:53 -04:00