17 Commits

Author SHA1 Message Date
matt
c6e154fd03 Feat: OLAP Payloads (#2410)
* feat: olap payloads table

* feat: olap queue messages for payload puts

* feat: wire up writes on task write

* driveby: add + ignore psql-connect

* fix: down migration

* fix: use external id for pk

* fix: insert query

* fix: more external ids

* fix: bit more cleanup

* feat: dags

* fix: the rest of the refs

* fix: placeholder uuid

* fix: write external ids

* feat: wire up messages over the queue

* fix: panic

* Revert "fix: panic"

This reverts commit c0adccf2ea.

* Revert "feat: wire up messages over the queue"

This reverts commit 36f425f3c1.

* fix: rm unused method

* fix: rm more

* fix: rm cruft

* feat: wire up failures

* feat: start wiring up completed events

* fix: more wiring

* fix: finish wiring up completed event payloads

* fix: lint

* feat: start wiring up external ids in the core

* feat: olap pub

* fix: add returning

* fix: wiring

* debug: log lines for pubs

* fix: external id writes

* Revert "debug: log lines for pubs"

This reverts commit fe430840bd.

* fix: rm sample

* debug: rm pub buffer param

* Revert "debug: rm pub buffer param"

This reverts commit b42a5cacbb.

* debug: stuck queries

* debug: more logs

* debug: yet more logs

* fix: rename BulkRetrieve -> Retrieve

* chore: lint

* fix: naming

* fix: conn leak in putpayloads

* fix: revert debug

* Revert "debug: more logs"

This reverts commit 95da7de64f.

* Revert "debug: stuck queries"

This reverts commit 8fda64adc4.

* feat: improve getters, olap getter

* fix: key type

* feat: first pass at pulling olap payloads from the payload store

* fix: start fixing bugs

* fix: start reworking `includePayloads` param

* fix: include payloads wiring

* feat: analyze for payloads

* fix: simplify writes more + write event payloads

* feat: read out event payloads

* feat: env vars for dual writes

* refactor: clean up task prop drilling a bit

* feat: add include payloads params to python for tests

* fix: tx commit

* fix: dual writes

* fix: not null constraint

* fix: one more

* debug: logging

* fix: more debugging, tweak function sig

* fix: function sig

* fix: refs

* debug: more logging

* debug: more logging

* debug: fix condition

* debug: overwrite properly

* fix: revert debug

* fix: rm more drilling

* fix: comments

* fix: partitioning jobs

* chore: ver

* fix: bug, docs

* hack: dummy id and inserted at for payload offloads

* fix: bug

* fix: no need to handle offloads for task event data

* hack: jitter + current ts

* fix: short circuit

* fix: offload payloads in a tx

* fix: uncomment sampling

* fix: don't offload if external store is disabled

* chore: gen sqlc

* fix: migration

* fix: start reworking types

* fix: couple more

* fix: rm unused code

* fix: drill includePayloads down again

* fix: silence annoying error in some cases

* fix: always store payloads

* debug: use workflow run id for input

* fix: improve logging

* debug: logging on retrieve

* debug: task input

* fix: use correct field

* debug: write even null payloads to limit errors

* debug: hide error lines

* fix: quieting more errors

* fix: duplicate example names, remove print lines

* debug: add logging for olap event writes

* hack: immediate event offloads and cutovers

* fix: rm log line

* fix: import

* fix: short circuit events

* fix: duped names
2025-10-20 09:09:49 -04:00
matt
57bb24aef1 Fix: Auto-generate docs snippets and examples (#2139)
* fix: gitignore all the generated stuff

* debug: try fixing build

* debug: build error part ii

* debug: move more deps out of dev

* fix: lock

* debug: lockfile

* fix: make dir

* fix: ci

* fix: dir

* debug: sed

* fix: sed

* debug: allow skipping

* Revert "debug: allow skipping"

This reverts commit 88e0ff870d.

* debug: ci

* fix: corepack

* debug: dir

* debug: sed

* debug: path

* fix: rm sync docs for now

* fix: remove more stuff

* fix: rm unused stuff

* fix: rm copy:app

* chore: lint

* fix: rm prettier from boot

* fix: couple missing scripts

* feat: auto-gen examples on push to main

* debug: test on this branch

* fix: install pnpm

* fix: cd

* fix: cmd

* Auto-generate files after merge [skip ci]

* fix: only copy examples

* debug: dummy commit for examples check

* chore: regenerate examples

* fix: naming

* fix: unwind dummy

* fix: only run on main

* fix: pre commit

* fix: naming

* chore: gen, fix task pre

* feat: create pr

* feat: only push examples changes

* fix: don't run from this branch

* fix: regen lockfile

* fix: regen docs lockfile

---------

Co-authored-by: GitHub Action <action@github.com>
2025-08-14 18:17:29 -05:00
Gabe Ruttner
09f8bd6118 feat--frontend-overhaul (#1537)
* reintegration

* fix: relative routes

* dynamic routes

* up/downgrade ui

* memoize

* fix breadcrumbs

* cleanup

* titles

* title format

* additional meta

* cli upgrade

* button

* lint

* Update frontend/app/src/next/components/runs/run-id.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* build errors

* unbind on return

* lint

* Fe overhaul  run form (#1547)

* simple trigger

* populated

* feat:trigger modal

* clear

* Update frontend/app/src/next/hooks/use-runs.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fe overhaul  details (#1552)

* simple trigger

* populated

* feat:trigger modal

* clear

* Update frontend/app/src/next/hooks/use-runs.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* wip activity log

* merged logs

* wip

* wip

* search box

* fudge sort

* wip improved worker sheet

* wip

* chore: improve error on dispatcher (#1538)

* fix: empty billing context (#1553)

* fix: empty

* precommit

* hotfix: priority nil pointer (#1555)

* hotfix: priority on schedule workflow (#1556)

* hotfix: priority on schedule workflow

* fix: build

* build

* lint

* build

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: abelanger5 <belanger@sas.upenn.edu>

* Fe  overhaul  burndown 1 (#1563)

* chore: improve error on dispatcher (#1538)

* fix: empty billing context (#1553)

* fix: empty

* precommit

* hotfix: priority nil pointer (#1555)

* hotfix: priority on schedule workflow (#1556)

* hotfix: priority on schedule workflow

* fix: build

* Hotfix: Handle EOF Properly (#1557)

* fix: handle EOF properly

* chore: version

* fix: debug logs

* fix: rm eof type

* hotfix: priority on cron workflow for v0 (#1558)

* fix: one more possible null deref (#1560)

* Hatchet Python Blog Post (#1526)

* feat: initial pass at first parts of blog post

* feat: initial mkdocs setup

* feat: first pass at embedding mkdocs

* fix: config

* debug: paths

* fix: unwind docs hack

* feat: start working on mkdocs theme

* fix: paths

* feat: wrap up post

* fix: proof

* fix: doc links

* fix: rm docs

* fix: lint

* fix: lint

* fix: typos + tweak

* fix: tweaks

* fix: typo

* fix: cleanup

* fix: go signature and docs (#1561)

* fix: go signature and docs

* Update examples/v1/workflows/concurrency-rr.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* feat: toggle doc sheet

* docs: concurrency cleanup (#1562)

* feat: storage adapter

* docs--worker-config-options (#1535)

* docs--worker-config-options

* Update frontend/docs/pages/home/workers.mdx

Co-authored-by: abelanger5 <belanger@sas.upenn.edu>

* Update worker-configuration-options.mdx

* lint

---------

Co-authored-by: abelanger5 <belanger@sas.upenn.edu>

---------

Co-authored-by: abelanger5 <belanger@sas.upenn.edu>
Co-authored-by: Matt Kaye <mrkaye97@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: bit of spacing

* single generated api

* feat: initial styling pass on runs views (#1586)

* Feat: Workflows pages (#1577)

* feat: simple workflows list page

* feat: refactor + add route for individual workflow

* feat: start wiring up workflow detail page

---------

Co-authored-by: Gabe Ruttner <gabriel.ruttner@gmail.com>

* fix: tsc

* fix: compiler

* fe  overhaul runs bash (#1599)

* wip sidebar

* wip: navigation

* fix nits

* tweaks

* clear filter button

* filter on click

* always scroll y

* wip time filters

* state changes

* wip

* wip

* functional, no style

* fix default

* wip

* queue metrics

* multi-select

* padding

* counts

* fix always num

* actions

* fix maxheight

* wip

* spacing

* fix: tenant state

* fix: tenant

* lint

* github page

* tenant alerting

* ingestors

* fix: tenant state

* build errrors

* empty state alerter

* dropdown only on mobile

* billing and limits

* fix form

* fix form

* wip

* wip

* env vars

* wip

* what a component

* wip

* fix inf render

* cleanup

* persistent tabs

* wip

* common actions

* deploy modal

* update

* delete

* you're a wizard harry

* Feat: Subrows in Runs View (#1595)

* fix: type

* feat: start wiring up subrows

* fix: tsc

* fix: hook order

* fix: hack for tsc

* fix: add some margin

* fix: empty state

* fix: empty state centering

* fix: task detail links

* fix: handle error, tweak child run styling

* fix: lint

* fix: edge case

* all mw config

* wip

* fix breadcrumb hook

* fixes

* upgrade surfaces

---------

Co-authored-by: Matt Kaye <mrkaye97@gmail.com>

* bring back error toast

* toast improvements

* lint

* fix: cloud only surface

* cloud surface

* fix breadcrumbs

* feat: config view on workflow run page (#1607)

* feat: config

* fix: rm schedule timeout

* Feat: Rework runs sidebar (#1612)

* feat: rq devtools

* fix: badge hover

* feat: improve sidebar

* refactor: hook

* fix: input undefined

* feat: waterfall diagram and cleanup runs view (#1606)

* merge

* merge

* fix merge issue

* rm debug lines

* case on count

* fix: display names

* colors and handle on click

---------

Co-authored-by: mrkaye97 <mrkaye97@gmail.com>

* fix: remove a ton of dead code (#1618)

* fix: remove n+1 query (#1619)

* Fe  overhaul managed compute and onboarding (#1614)

* fix tab state on state changes

* fix collapsed children

* setup

* layout

* layout

* move feature

* wip

* chore(deps): bump k8s.io/client-go from 0.32.3 to 0.33.0 (#1608)

Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.32.3 to 0.33.0.
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kubernetes/client-go/compare/v0.32.3...v0.33.0)

---
updated-dependencies:
- dependency-name: k8s.io/client-go
  dependency-version: 0.33.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* wip

* universal install

* initial migration guide

* fakefake tokens

* fake secret

* fixes

* lint

* lint

* lint

* tidy

* chore: expose clean docs on top level

* drop reo

* wip

* static

* functional

* bump go 1.23 -> 1.24

* fix: whitespace lint

* bump golangci-lint version

* wip

* try to set up go before pre commit runs

* lint

* names

* lint

* fix: session store

* wip

* normalized snips

* fix links

* blog: mergent migration (#1611)

* universal install

* initial migration guide

* fakefake tokens

* fake secret

* fixes

* lint

* lint

* lint

* tidy

* bump go 1.23 -> 1.24

* fix: whitespace lint

* bump golangci-lint version

* try to set up go before pre commit runs

* lint

* names

* lint

* fix: session store

* fix links

---------

Co-authored-by: Alexander Belanger <alexander@hatchet.run>

* reusable state

* fix: rm unimplemented pages for now (#1615)

* feature dir

* fixes (#1616)

* with public auth

* completed typescript flow

* go blocks

* wip

* layout

* wip

* feat: read replica support and docs (#1617)

* feat: read replica support and docs

* fix: load logic

* wip

* wip

* remove provider

* migrate to static

* fix block

* lint

* fix: loadtest

* add task for linting

* cleanup

* fix meta sync

* clean examples with highlights

* get snips into app

* sync

* remove fake highlights

* always gen before build

* ignore generated

* ignore

* ignore generated

* cleanup

* always start the dev server

* examples dependabot

* app examples built off sdk examples

* auto sync on main changes

* sync the quickstarts with updating dependencies

* examples and quickstart dependabots

* only examples

* functional onboarding

* fix worker detail page

* fix detail page

* lint

* init

* tested structure copy

* prepend

* to >, ‼️ to !!

* normalize

* separate example source

* multi source

* with index

* wip

* clean generation

* migrated

* cleanup

* should build

* sync before build

* fix steps

* inline

* mkdirs

* revert build

* gen docs

* static

* rm

* update examples

* lint

* fix: ts

* fix remove lines on match

* fix: client

* add snips

* dont lint examples

* fix source

* dont test examples

* exclude quickstart

* lint

* dont break examples

* cleanup

* cleanup

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Alexander Belanger <alexander@hatchet.run>
Co-authored-by: Matt Kaye <mrkaye97@gmail.com>
Co-authored-by: abelanger5 <belanger@sas.upenn.edu>

* Merge main

* lint

* fix

* Update api-server-setup.mdx

* cleanup

* github state

* review and build

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: abelanger5 <belanger@sas.upenn.edu>
Co-authored-by: Matt Kaye <mrkaye97@gmail.com>
Co-authored-by: Alexander Belanger <alexander@hatchet.run>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-05 08:17:29 -07:00
Gabe Ruttner
697757879f feat: billing (#624)
* feat: init lago client

* feat: billable meter

* feat: db persistence

* wip: expose sub

* feat: rename page

* wip: billing section

* wip: lago integration

* feat: separate plan and period

* wip: webhook

* feat: improve empty state

* feat: update limits on plan changes

* feat: can change plans

* feat: change plan loading state

* feat: yearly filter

* feat: billing clarification

* fix: treatment

* feat: filter plans

* feat: prevent non-owner from changing plan

* fix: loading state

* fix: jit portal link

* fix: rm import

* fix: build errors

* fix: default to free

* fix: wrong files

* fix: select or insert customer

* fix: note

* feat: upgrade dependent on payment method state

* fix: dedupe

* chore: remove github-app from core

* chore: port to cloud

* chore: port to cloud

* chore: port to cloud

* chore: port to cloud

* chore: port to cloud

* add new components, repository callbacks

* chore: rm unused packages

* chore: fix generation

* chore: gen

* fix: cloud api references

* debug

* debug

* fix: actually set plans

* chore: rm debug

* fix: build

* feat: callbacks

* fix: add generated code

* chore: group cloud components

* chore: group by feature

* feat: alert change

* feat: confirm

* fix: confirm modal

* fix: ui

* fix: remove arrears

* fix: open in same tab

* fix: wan alert

* fix: call the callback

* fix: callback obj

* fix: disable if no cloud meta

---------

Co-authored-by: Alexander Belanger <alexander@hatchet.run>
2024-06-25 13:57:16 -04:00
Luca Steeb
2f7483a3be test(timeout): add test for timeout (#344) 2024-04-11 00:50:24 +07:00
Ikko Eltociear Ashimine
30fc42174b chore: fix typo in .gitignore (#252)
sentitive -> sensitive
2024-03-10 14:21:24 -04:00
abelanger5
d376b953aa feat: python rest api (#223)
* feat: Adds a generated REST API client and exposes workflow methods for programmatic usage
2024-03-02 20:33:20 -05:00
Luca Steeb
ae4841031b feat(engine): standalone tests and engine teardown (#172) 2024-02-28 00:15:25 +07:00
Gabe Ruttner
a7cff7f218 fix: worker listener retires (#181)
* hotfix: add repository for npm publish

* release(py-sdk): bump version

* chore: ignore venv

* fix: add timeout to retry

* chore: rm change

* fix: remove duplicate logic and favor retries in dispatcher

* release: bump version

* fix: retry on general failures

* chore: rm unused code

* fix: retries reset if greater than interval
2024-02-21 18:54:42 -08:00
Gabe Ruttner
45813fdb1b feat: dashboard playground ui (#162)
* hotfix: add repository for npm publish

* release(py-sdk): bump version

* chore: ignore venv

* feat(dashboard): collapsible sidebar

* fix: hangup workflow listener when workflow run finishes (#161)

* wip: class based listener pattern

* fix: workflow run listener hangups

* fix: hang up workflow listener on finished

* fix: case for current workflow run

* address review comments

* bump version

---------

Co-authored-by: g <gabriel.ruttner@gmail.com>

* wip: focus state and flat playground

* fix: focus state

* feat: unify state

* wip: playground state

* cleanup: rm logging

* fix: default output

* cleanup: rm deadcode

* feat: can replay individual steps

* feat: icon tabs

* feat: sticky output

* cleanup: linting

---------

Co-authored-by: abelanger5 <belanger@sas.upenn.edu>
2024-02-13 10:24:46 -07:00
abelanger5
1cbe710823 feat: bundle rest api with typescript sdk (#151)
* feat: (wip) bundle rest API in typescript client

* chore: add generated files

* chore: pnpm lock file

* extract tenant id from token subject
2024-02-07 16:45:32 -05:00
abelanger5
78685d0098 feat(security): multiple encryption options, API tokens, easier setup (#125)
* (wip) encryption

* feat: api tokens

* chore: add api token generation command

* fix: e2e tests

* chore: set timeout for e2e job

* fix: e2e tests, remove client-side certs

* chore: address PR review comments

* fix: token tests

* chore: address review comments and fix tests
2024-01-26 15:38:36 -05:00
Luca Steeb
8b379ee9d1 feat(events): add workflow filter (#114)
* feat(events): add workflow filter

* cast to uuid

---------

Co-authored-by: Alexander Belanger <belanger@sas.upenn.edu>
2024-01-21 22:33:58 -05:00
abelanger5
5937b9fd98 feat(go-sdk): add ability to create workflows from the Go SDK more easily, quickstart improvements (#30)
* feat: add initial docs site

* feat: allow workflows to be defined from go sdk

* fix release action

* chore: remove server dependencies from client

* fix: use correct certificate for server

* chore: add port and bind address to grpc config

* fix: add env for grpc config

* fix: nil pointer when output is null

* chore: support variation in output args

* fix unresolve merge conflict

* fix: quickstart improvements

* temp remove database url

* fix: action id not required for event

* fix: actionid validation for events

* Remove deleted files
2024-01-02 09:02:53 -05:00
abelanger5
14ecac8b81 feat: add hatchet-admin command and improve dockerfiles (#18)
* feat: add `hatchet-admin` command and improve dockerfiles

- Also bumps `prisma-client-go` version
2023-12-21 16:38:17 -05:00
Alexander Belanger
1f470da954 add openapi file 2023-12-18 12:13:31 -05:00
Alexander Belanger
366c79441d first commit 2023-12-15 13:08:04 -05:00