Mohammed Nafees
ef498a6235
Introduce tenant Prometheus metrics ( #1875 )
...
* introduce tenant workflow completed metric
* expose tenant prom metrics via handler
* fix workflow and worker id in metrics
* correctly add workflow metrics from workflow controller
* use olap DB to gather information for workflow completion
* fix prom metrics endpoint for tenant
* workflow name from external id
* simplify tenant registry based metrics
* add docs for prometheus metrics
* fix docs lint
* run prettier fix
* WIP metrics work
* use federate prom server URL to proxy metrics
* implement workflow duration histogram metric
* separate prom stack docker compose
* fix duration metrics calls
* move scheduler metrics to prom tenant specific file
* update docs for prom metrics
* fix lint
* use proper indices to query for durations
* reorg tenant metrics
* fix lint for doc
* update docs with promql examples and casing around prom metrics enabled
* update prom server url
* fix lint
* enabled prom metrics for v1 only from controller
2025-06-27 11:46:31 -04:00
Gabe Ruttner
09f8bd6118
feat--frontend-overhaul ( #1537 )
...
* reintegration
* fix: relative routes
* dynamic routes
* up/downgrade ui
* memoize
* fix breadcrumbs
* cleanup
* titles
* title format
* additional meta
* cli upgrade
* button
* lint
* Update frontend/app/src/next/components/runs/run-id.tsx
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* build errors
* unbind on return
* lint
* Fe overhaul run form (#1547 )
* simple trigger
* populated
* feat:trigger modal
* clear
* Update frontend/app/src/next/hooks/use-runs.tsx
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Fe overhaul details (#1552 )
* simple trigger
* populated
* feat:trigger modal
* clear
* Update frontend/app/src/next/hooks/use-runs.tsx
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* wip activity log
* merged logs
* wip
* wip
* search box
* fudge sort
* wip improved worker sheet
* wip
* chore: improve error on dispatcher (#1538 )
* fix: empty billing context (#1553 )
* fix: empty
* precommit
* hotfix: priority nil pointer (#1555 )
* hotfix: priority on schedule workflow (#1556 )
* hotfix: priority on schedule workflow
* fix: build
* build
* lint
* build
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
Co-authored-by: abelanger5 <belanger@sas.upenn.edu >
* Fe overhaul burndown 1 (#1563 )
* chore: improve error on dispatcher (#1538 )
* fix: empty billing context (#1553 )
* fix: empty
* precommit
* hotfix: priority nil pointer (#1555 )
* hotfix: priority on schedule workflow (#1556 )
* hotfix: priority on schedule workflow
* fix: build
* Hotfix: Handle EOF Properly (#1557 )
* fix: handle EOF properly
* chore: version
* fix: debug logs
* fix: rm eof type
* hotfix: priority on cron workflow for v0 (#1558 )
* fix: one more possible null deref (#1560 )
* Hatchet Python Blog Post (#1526 )
* feat: initial pass at first parts of blog post
* feat: initial mkdocs setup
* feat: first pass at embedding mkdocs
* fix: config
* debug: paths
* fix: unwind docs hack
* feat: start working on mkdocs theme
* fix: paths
* feat: wrap up post
* fix: proof
* fix: doc links
* fix: rm docs
* fix: lint
* fix: lint
* fix: typos + tweak
* fix: tweaks
* fix: typo
* fix: cleanup
* fix: go signature and docs (#1561 )
* fix: go signature and docs
* Update examples/v1/workflows/concurrency-rr.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* feat: toggle doc sheet
* docs: concurrency cleanup (#1562 )
* feat: storage adapter
* docs--worker-config-options (#1535 )
* docs--worker-config-options
* Update frontend/docs/pages/home/workers.mdx
Co-authored-by: abelanger5 <belanger@sas.upenn.edu >
* Update worker-configuration-options.mdx
* lint
---------
Co-authored-by: abelanger5 <belanger@sas.upenn.edu >
---------
Co-authored-by: abelanger5 <belanger@sas.upenn.edu >
Co-authored-by: Matt Kaye <mrkaye97@gmail.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: bit of spacing
* single generated api
* feat: initial styling pass on runs views (#1586 )
* Feat: Workflows pages (#1577 )
* feat: simple workflows list page
* feat: refactor + add route for individual workflow
* feat: start wiring up workflow detail page
---------
Co-authored-by: Gabe Ruttner <gabriel.ruttner@gmail.com >
* fix: tsc
* fix: compiler
* fe overhaul runs bash (#1599 )
* wip sidebar
* wip: navigation
* fix nits
* tweaks
* clear filter button
* filter on click
* always scroll y
* wip time filters
* state changes
* wip
* wip
* functional, no style
* fix default
* wip
* queue metrics
* multi-select
* padding
* counts
* fix always num
* actions
* fix maxheight
* wip
* spacing
* fix: tenant state
* fix: tenant
* lint
* github page
* tenant alerting
* ingestors
* fix: tenant state
* build errrors
* empty state alerter
* dropdown only on mobile
* billing and limits
* fix form
* fix form
* wip
* wip
* env vars
* wip
* what a component
* wip
* fix inf render
* cleanup
* persistent tabs
* wip
* common actions
* deploy modal
* update
* delete
* you're a wizard harry
* Feat: Subrows in Runs View (#1595 )
* fix: type
* feat: start wiring up subrows
* fix: tsc
* fix: hook order
* fix: hack for tsc
* fix: add some margin
* fix: empty state
* fix: empty state centering
* fix: task detail links
* fix: handle error, tweak child run styling
* fix: lint
* fix: edge case
* all mw config
* wip
* fix breadcrumb hook
* fixes
* upgrade surfaces
---------
Co-authored-by: Matt Kaye <mrkaye97@gmail.com >
* bring back error toast
* toast improvements
* lint
* fix: cloud only surface
* cloud surface
* fix breadcrumbs
* feat: config view on workflow run page (#1607 )
* feat: config
* fix: rm schedule timeout
* Feat: Rework runs sidebar (#1612 )
* feat: rq devtools
* fix: badge hover
* feat: improve sidebar
* refactor: hook
* fix: input undefined
* feat: waterfall diagram and cleanup runs view (#1606 )
* merge
* merge
* fix merge issue
* rm debug lines
* case on count
* fix: display names
* colors and handle on click
---------
Co-authored-by: mrkaye97 <mrkaye97@gmail.com >
* fix: remove a ton of dead code (#1618 )
* fix: remove n+1 query (#1619 )
* Fe overhaul managed compute and onboarding (#1614 )
* fix tab state on state changes
* fix collapsed children
* setup
* layout
* layout
* move feature
* wip
* chore(deps): bump k8s.io/client-go from 0.32.3 to 0.33.0 (#1608 )
Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go ) from 0.32.3 to 0.33.0.
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md )
- [Commits](https://github.com/kubernetes/client-go/compare/v0.32.3...v0.33.0 )
---
updated-dependencies:
- dependency-name: k8s.io/client-go
dependency-version: 0.33.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* wip
* universal install
* initial migration guide
* fakefake tokens
* fake secret
* fixes
* lint
* lint
* lint
* tidy
* chore: expose clean docs on top level
* drop reo
* wip
* static
* functional
* bump go 1.23 -> 1.24
* fix: whitespace lint
* bump golangci-lint version
* wip
* try to set up go before pre commit runs
* lint
* names
* lint
* fix: session store
* wip
* normalized snips
* fix links
* blog: mergent migration (#1611 )
* universal install
* initial migration guide
* fakefake tokens
* fake secret
* fixes
* lint
* lint
* lint
* tidy
* bump go 1.23 -> 1.24
* fix: whitespace lint
* bump golangci-lint version
* try to set up go before pre commit runs
* lint
* names
* lint
* fix: session store
* fix links
---------
Co-authored-by: Alexander Belanger <alexander@hatchet.run >
* reusable state
* fix: rm unimplemented pages for now (#1615 )
* feature dir
* fixes (#1616 )
* with public auth
* completed typescript flow
* go blocks
* wip
* layout
* wip
* feat: read replica support and docs (#1617 )
* feat: read replica support and docs
* fix: load logic
* wip
* wip
* remove provider
* migrate to static
* fix block
* lint
* fix: loadtest
* add task for linting
* cleanup
* fix meta sync
* clean examples with highlights
* get snips into app
* sync
* remove fake highlights
* always gen before build
* ignore generated
* ignore
* ignore generated
* cleanup
* always start the dev server
* examples dependabot
* app examples built off sdk examples
* auto sync on main changes
* sync the quickstarts with updating dependencies
* examples and quickstart dependabots
* only examples
* functional onboarding
* fix worker detail page
* fix detail page
* lint
* init
* tested structure copy
* prepend
* ❓ to >, ‼️ to !!
* normalize
* separate example source
* multi source
* with index
* wip
* clean generation
* migrated
* cleanup
* should build
* sync before build
* fix steps
* inline
* mkdirs
* revert build
* gen docs
* static
* rm
* update examples
* lint
* fix: ts
* fix remove lines on match
* fix: client
* add snips
* dont lint examples
* fix source
* dont test examples
* exclude quickstart
* lint
* dont break examples
* cleanup
* cleanup
---------
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Alexander Belanger <alexander@hatchet.run >
Co-authored-by: Matt Kaye <mrkaye97@gmail.com >
Co-authored-by: abelanger5 <belanger@sas.upenn.edu >
* Merge main
* lint
* fix
* Update api-server-setup.mdx
* cleanup
* github state
* review and build
---------
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
Co-authored-by: abelanger5 <belanger@sas.upenn.edu >
Co-authored-by: Matt Kaye <mrkaye97@gmail.com >
Co-authored-by: Alexander Belanger <alexander@hatchet.run >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-05 08:17:29 -07:00
Gabe Ruttner
80e3ef6a13
Fix security patches ( #1462 )
...
* patch: next
* patch: axios
* nextra 2 -> 3
* final fixes for docs
* fix: v0-docs
* chore: update dependencies
* chore: update fe
* chore: final fixes
* fix: vite
* fix: generate
* chore: generate
* chore: generate
* lint
2025-04-01 06:40:42 -07:00
Matt Kaye
5062bf1e3e
V1 SDKs and Docs ( #1361 )
...
New SDKs and docs for the v1 release.
2025-03-25 15:45:07 -07:00
abelanger5
00c4bbff09
feat(v1): new gRPC API endpoints ( #1367 )
...
* wip: api contracts
* feat: implement put workflow version endpoint
* add support for match existing data, get scaffolding in place for additional triggers
* create additional matches
* feat: durable sleep, user event matching
* update protos
* fix: working poc of user events, durable sleep
* add migration
* fix: migration column
* feat: durable event listener
* fix: skip overrides
* fix: input -> output
2025-03-23 18:58:20 -07:00
abelanger5
677fe2d328
fix: spawn workflows should handle on failure properly, lite improvements ( #1336 )
2025-03-13 22:02:03 -04:00
abelanger5
ac968e94b8
fix: concurrency issues and a few small improvements ( #1324 )
2025-03-12 16:30:34 -04:00
abelanger5
1f2096313d
feat: v1 engine ( #1318 )
2025-03-11 14:57:13 -04:00
Gabe Ruttner
bc1d50b0ed
Fix db url env vars ( #1267 )
...
* fix: remove db url log
* docs: db config options
* chore: lint
* chore: lint
2025-02-13 07:31:59 -08:00
abelanger5
f821379997
fix: remove deprecated swagger-cli ( #1080 )
...
* fix: remove deprecated swagger-cli
* fix: use node:18-alpine
2024-12-03 16:01:46 +00:00
Sean Reilly
42afe083cf
Partition Step Run and Remove Prisma ( #982 )
...
* add in the migration for now
* Update step_runs.sql
remove TODO
* change the schema so we don't undo it
* add the migration for step run partition. remove prisma. add a helper task for recreating the db
* do a manual merge of the schema.sql
* add in the serial
* update docs
* PR feedback
* add Identity to all tables that don't have a Bigserial
* do the atlas hash with the new migration
* squash the migrations
---------
Co-authored-by: Sean Reilly <sean@hatchet.run >
2024-11-20 15:20:36 -08:00
Sean Reilly
15c50f46b5
Partial PR - need to generate SDK - Add endpoint to get the total free worker slots for a worker and the … ( #857 )
...
* Add endpoint to get the total free worker slots for a worker and the max runs
* update to use WorkerSempahoreCount instead of checking stepRunId
* modify the query for the new table and change the interface
* bump golangci-lint make changes to name of returned data
* revert the simple example
---------
Co-authored-by: Sean Reilly <sean@hatchet.run >
2024-09-19 10:11:16 -07:00
abelanger5
1ea4dfc5de
feat: deduplicated enqueue ( #735 )
...
* wip
* wip: functional query
* feat: expose affinity config
* feat: add weight to proto
* feat: upsert affinity state on worker start
* fix: linting
* feat: add upsert proto
* feat: upsert handler
* feat: revise model
* fix: labels
* feat: functional desired worker
* wip: ui
* feat: add state to step run events
* fix: filter empty keys
* fix: labels as badges
* feat: empty state and descriptive text
* chore: add todo
* chore: whitespace
* chore: cleanup
* chore: cleanup
* chore: fix hash
* chore: squash migrations
* fix: fair worker assignment
* fix: remaining slots on valid desired workers
* wip: sticky
* fix: count slots
* chore: rm log line
* feat: expose sticky config
* wip: sticky dag
* feat: expose desired worker id to trigger
* feat: trigger on desired worker
* feat: typescript docs
* feat: sticky python
* feat: py sticky children
* wip: py affinity
* serverless note
* feat: complete python examples
* linting
* feat: deduplicated enqueue
* fix: address changes from PR review
* chore: generate
---------
Co-authored-by: gabriel ruttner <gabriel.ruttner@gmail.com >
2024-07-26 16:47:46 +00:00
Luca Steeb
3e41279840
chore(packages): pin swagger api generation ( #675 )
2024-07-02 00:08:49 +00:00
Luca Steeb
1490d88954
feat: webhook workers ( #542 )
...
Adds serverless support via the concept of webhook workers. Allows any webhook to be registered as a serverless endpoint for executing a step.
2024-06-25 17:06:43 -04:00
abelanger5
12e0fe6671
docs: hatchet-lite setup guide ( #573 )
...
* docs: hatchet-lite setup guide
* chore: docs lint
* fix: add line
* fix: hatchet-lite init
* fix: auto-gen env file for guides
* fix: volume path
* fix: imports
2024-06-10 21:43:39 -04:00
abelanger5
3f70cf2a5c
fix: run rabbitmq-server with docker entrypoint ( #565 )
2024-06-08 17:28:07 -04:00
Gabe Ruttner
bbc4e58dd9
feat: limits ( #559 )
...
* feat: workflow run limits
* fix: resource exhausted 429
* feat: event limit
* feat: worker limit
* fix: sensible error
* fix: pb
* feat: expose limits api
* feat: default limits
* feat: add enable alert option
* feat: slack and email alerts
* fix: cron interval
* feat: make metered util
* wip: schedules and crons
* chore: squash migration
* fix: select or insert
* fix: remove unfinished meter
* chore: atlas migration
* fix: template format
* fix: shared ErrResourceExhausted
* feat: cache
* fix: limit can be nil
* fix: clarification
* fix: close meter ticker
* fix: friendly error for child workflows
2024-06-07 10:57:57 -07:00
abelanger5
3b1232cc28
fix: more casing in atlas-apply script ( #563 )
2024-06-06 14:41:54 -04:00
abelanger5
b0b2e26952
feat: hatchet-lite ( #560 )
...
* feat: hatchet-lite mvp
* fix: init shadow db
* fix: install atlas
* fix: correct env
* fix: wait for db ready
* fix: remove name flag
* fix: add hatchet-lite to build
2024-06-06 14:03:53 -04:00
Luca Steeb
d1a4d35830
chore(pre-commit): lint whitespace ( #494 )
...
Adds a whitespace linter to the pre-commit hook to ensure consistent formatting.
It also enables linting of other SQL files such as for SQLc queries.
2024-05-16 09:17:01 -04:00
Luca Steeb
7bb1015c99
chore(python-sdk): extract python sdk into its own repo ( #393 )
2024-04-22 16:35:53 +01:00
Abhiuday Gupta
c8a79aabf3
chore(format): added pre-commit hooks and formatted files ( #294 )
...
Co-authored-by: steebchen <contact@luca-steeb.com >
2024-04-16 22:19:21 +04:00
abelanger5
77e5d2b77c
feat(go-sdk): spawnWorkflow method and get up to speed with other sdks ( #297 )
...
* feat(go-sdk): spawnWorkflow method and get up to speed with other sdks
* fix: manual trigger example
* fix: linting errors
* fix: double serialization from go sdk
* fix: spawn workflow logic and procedural example
* test(e2e): add procedural test
* fix: panic in e2e test
* fix: e2e test preparation
* fix: api server url in test.yml
* fix: load test server url
* chore: make num children configurable
* address pr review
2024-03-29 14:07:39 -07:00
Luca Steeb
d91a8d7bc3
fix: handle inefficient assignments ( #263 )
2024-03-16 17:09:35 +07:00
Luca Steeb
c2b42508be
fix(typescript-sdk): extract typescript sdk into separate repo ( #239 )
2024-03-08 16:04:38 +07:00
abelanger5
d376b953aa
feat: python rest api ( #223 )
...
* feat: Adds a generated REST API client and exposes workflow methods for programmatic usage
2024-03-02 20:33:20 -05:00
Luca Steeb
f5a6e80fc7
fix(engine): add --no-graceful-shutdown flag for nodemon ( #221 )
2024-03-02 00:38:39 +07:00
abelanger5
1cbe710823
feat: bundle rest api with typescript sdk ( #151 )
...
* feat: (wip) bundle rest API in typescript client
* chore: add generated files
* chore: pnpm lock file
* extract tenant id from token subject
2024-02-07 16:45:32 -05:00
abelanger5
78685d0098
feat(security): multiple encryption options, API tokens, easier setup ( #125 )
...
* (wip) encryption
* feat: api tokens
* chore: add api token generation command
* fix: e2e tests
* chore: set timeout for e2e job
* fix: e2e tests, remove client-side certs
* chore: address PR review comments
* fix: token tests
* chore: address review comments and fix tests
2024-01-26 15:38:36 -05:00
Luca Steeb
6d8c7ab073
feat(test): introduce basic e2e tests ( #97 )
2024-01-11 13:36:15 -05:00
abelanger5
41f26b6bc8
chore: add docker builds ( #16 )
...
* chore: add docker builds and initial pre-release + release process
2023-12-19 14:42:50 -05:00
Luca Steeb
addf42988a
chore(contributing): adapt deps, hardcode generator versions ( #6 )
2023-12-19 22:04:13 +07:00
Luca Steeb
6fef81eb52
chore(contributing): improve onboarding ( #3 )
...
- shell scripts logging and fail on error
- use npx --yes instead of manual global install
- run go libs directly instead of installing and running binaries
- add missing brew dependencies
2023-12-18 13:36:48 -05:00
Alexander Belanger
366c79441d
first commit
2023-12-15 13:08:04 -05:00
