Commit Graph

21 Commits

Author SHA1 Message Date
abelanger5
1f2096313d feat: v1 engine (#1318) 2025-03-11 14:57:13 -04:00
Sean Reilly
a8dd33c61f Feature - configurable logging backend (#1188)
* allow us to configure different repos

* make the struct contents public

* pass in config values to new log repo

* rename functions - possibly breaking changes so lets discuss

* make the logging backend configurable

* fix tests

* don't allow calls to WithAdditionalConfig

* cleanup

* replace sc with server

Co-authored-by: abelanger5 <belanger@sas.upenn.edu>

* rename sc to server

* add a LRU cache for the step run lookup

* lets not use an expirable cache and just use the regular one - we cannot close the go func in exirable

---------

Co-authored-by: abelanger5 <belanger@sas.upenn.edu>
2025-01-17 15:34:10 -08:00
abelanger5
b5014f6b3d chore: more visibility and debug lines for queues (#836)
* chore: more visibility and debug options for queues

* better debug lines on queue repo

* don't log so much in load test
2024-08-29 14:49:24 -04:00
abelanger5
5d87f380ef feat: managed worker pools (#725)
* change api extension spec to register custom populators

* fix: support only bearer auth

* fix: correct authn logic

* fix: indexes on workflow runs, events

* feat: managed worker pools

* chore: lint fix

* hide workers view when not enabled

* support internal api tokens, minor improvements

* fix: actually write internal

* fix breaking changes

* don't allow revoking internal tokens

* fix: linting and remove metrics view

* fix: token

* address review and add feat flags
2024-07-16 13:33:46 +00:00
Gabe Ruttner
8a8a033af6 feat: variable token expiration (#670)
* feat: variable token expiration

* fix: typo

* fix: long tokens for webhook workers

* fix: format
2024-07-01 15:47:32 +00:00
Gabe Ruttner
a8d42819ea feat: check security service (#639)
* feat: check security service

* feat: propegate version

* feat: with ident

* fix: lint

* chore: generate

* fix: change domain

* fix: panic recover

* fix: migrations

* fix: hash

* fix: dont check in tests
2024-06-26 16:26:29 -04:00
Luca Steeb
1490d88954 feat: webhook workers (#542)
Adds serverless support via the concept of webhook workers. Allows any webhook to be registered as a serverless endpoint for executing a step.
2024-06-25 17:06:43 -04:00
abelanger5
7c3ddfca32 feat: api server extensions (#614)
* feat: allow extending the api server

* chore: remove internal packages to pkg

* chore: update db_gen.go

* fix: expose auth

* fix: move logger to pkg

* fix: don't generate gitignore for prisma client

* fix: allow extensions to register their own api spec

* feat: expose pool on server config

* fix: nil pointer exception on empty opts

* fix: run.go file
2024-06-19 09:36:13 -04:00
abelanger5
4ce1dd8632 feat: multi-workflow runs listener on a single endpoint
* new api-contract for workflow run events

* feat: initial implementation for new subscribe listener

* fix: sync issues and send workflow runs immediately

* refactor: add context to all engine db queries, fix deadlocking query

* fix: use new ctx for deleting dispatcher and ticker

* add cancellation reasons

* fix: docs linting

---------

Co-authored-by: gabriel ruttner <gabriel.ruttner@gmail.com>
2024-04-18 20:55:11 -04:00
Luca Steeb
8183dd509a test(rampup): add load ramp up test (#273)
* test(rampup): add load ramp up test

* disable debug logging

* actual implementation

* refactor

* max acceptable schedule

* check for non-executed events

* fixes

* chore: set log level to error in engine tests

---------

Co-authored-by: abelanger5 <belanger@sas.upenn.edu>
2024-03-31 19:14:30 -04:00
abelanger5
77e5d2b77c feat(go-sdk): spawnWorkflow method and get up to speed with other sdks (#297)
* feat(go-sdk): spawnWorkflow method and get up to speed with other sdks

* fix: manual trigger example

* fix: linting errors

* fix: double serialization from go sdk

* fix: spawn workflow logic and procedural example

* test(e2e): add procedural test

* fix: panic in e2e test

* fix: e2e test preparation

* fix: api server url in test.yml

* fix: load test server url

* chore: make num children configurable

* address pr review
2024-03-29 14:07:39 -07:00
abelanger5
092f54c64f refactor: separate api and engine repositories, change ticker logic (#281)
* refactor: separate api and engine repositories, change ticker logic

* fix: nil error blocks

* fix: run migration on load test

* fix: generate db package in load test

* fix: test.yml

* fix: add pnpm to load test

* fix: don't lock CTEs with columns that don't get updated

* fix: update heartbeat for worker every 4 seconds, not 5

* chore: remove dead code

* chore: update python sdk

* chore: add back telemetry attributes
2024-03-21 14:10:34 -04:00
Luca Steeb
d91a8d7bc3 fix: handle inefficient assignments (#263) 2024-03-16 17:09:35 +07:00
Luca Steeb
4631747fe4 test(e2e): set up env programmatically (#218) 2024-03-02 00:38:20 +07:00
Luca Steeb
ae4841031b feat(engine): standalone tests and engine teardown (#172) 2024-02-28 00:15:25 +07:00
Luca Steeb
00111d823c test(load): add load tests CLI & e2e tests (#157) 2024-02-16 23:47:34 +07:00
abelanger5
78685d0098 feat(security): multiple encryption options, API tokens, easier setup (#125)
* (wip) encryption

* feat: api tokens

* chore: add api token generation command

* fix: e2e tests

* chore: set timeout for e2e job

* fix: e2e tests, remove client-side certs

* chore: address PR review comments

* fix: token tests

* chore: address review comments and fix tests
2024-01-26 15:38:36 -05:00
Luca Steeb
c8f87599c7 chore: ignore or fix linter errors (#116)
* chore: ignore or fix linter errors

* chore: wrap up lint errors

* chore: sqlc generation

---------

Co-authored-by: Alexander Belanger <belanger@sas.upenn.edu>
2024-01-21 22:01:20 -05:00
Luca Steeb
6d8c7ab073 feat(test): introduce basic e2e tests (#97) 2024-01-11 13:36:15 -05:00
Luca Steeb
14e13fe24a ci(workflows): run go tests on PR, adapt tests (#27)
Co-authored-by: abelanger5 <belanger@sas.upenn.edu>
2023-12-28 03:06:49 +07:00
Alexander Belanger
366c79441d first commit 2023-12-15 13:08:04 -05:00