mirror of
https://github.com/hatchet-dev/hatchet.git
synced 2026-01-06 00:40:10 -06:00
feat: add initial docs site (#14)
* feat: add initial docs site * docs: create initial documentation and update readme
This commit is contained in:
290
README.md
290
README.md
@@ -1,293 +1,39 @@
|
||||
[](https://join.slack.com/t/hatchet-co/signup) [](https://opensource.org/licenses/MIT)
|
||||
|
||||
<!-- [](https://pkg.go.dev/github.com/hatchet-dev/hatchet) -->
|
||||
[](https://discord.gg/ZMeUafwH89) [](https://opensource.org/licenses/MIT) [](https://pkg.go.dev/github.com/hatchet-dev/hatchet)
|
||||
|
||||
## Introduction
|
||||
|
||||
_**Note:** Hatchet is in early development. Changes are not guaranteed to be backwards-compatible. If you'd like to run Hatchet in production, feel free to reach out on Slack for tips._
|
||||
_**Note:** Hatchet is in early development. Changes are not guaranteed to be backwards-compatible. If you'd like to run Hatchet in production, feel free to reach out on Discord for tips._
|
||||
|
||||
Hatchet is an event storage API and workflow engine for distributed applications. Using Hatchet, you can create workers which process a set of background tasks based on different triggers, like events created within your system or a cron schedule.
|
||||
Hatchet is a self-hostable workflow engine built for application developers.
|
||||
|
||||
As a simple example, let's say you want to perform 3 actions when a user has signed up for your app:
|
||||
**What is a workflow?**
|
||||
|
||||
1. Initialize a set of resources for the user (perhaps a sandbox environment for testing).
|
||||
2. Send the user an automated greeting over email
|
||||
3. Add the user to a newsletter campaign
|
||||
The term `workflow` tends to be overloaded, so let's make things more clear - in Hatchet, a workflow is a set of functions which are executed in response to an external trigger (an event, schedule, or API call). For example, if you'd like to send notifications to a user after they've signed up, you could create a workflow for that.
|
||||
|
||||
With Hatchet, this would look something like the following:
|
||||
**Why is that useful?**
|
||||
|
||||
```yaml
|
||||
name: "post-user-sign-up"
|
||||
version: v0.2.0
|
||||
triggers:
|
||||
events:
|
||||
- user:create
|
||||
jobs:
|
||||
create-resources:
|
||||
steps:
|
||||
- id: createSandbox
|
||||
action: sandbox:create
|
||||
timeout: 60s
|
||||
greet-user:
|
||||
steps:
|
||||
- id: greetUser
|
||||
action: postmark:email-from-template
|
||||
timeout: 15s
|
||||
with:
|
||||
firstName: "{{ .user.firstName }}"
|
||||
email: "{{ .user.email }}"
|
||||
add-to-newsletter:
|
||||
steps:
|
||||
- id: addUserToNewsletter
|
||||
action: newsletter:add-user
|
||||
timeout: 15s
|
||||
with:
|
||||
email: "{{ .user.email }}"
|
||||
```
|
||||
Instead of processing background tasks and functions in your application handlers, which can lead to complex code, hard-to-debug errors, and resource contention, you can distribute these workflows between a set of `workers`. Workers are long-running processes which listen for events, and execute the functions defined in your workflows.
|
||||
|
||||
In your codebase, you would then create a worker which could perform the following actions:
|
||||
**What is a workflow engine?**
|
||||
|
||||
- `sandbox:create` responsible for creating/tearing down a sandbox environment
|
||||
- `postmark:email-from-template` for sending an email from a template
|
||||
- `newsletter:add-user` for adding a user to a newsletter campaign
|
||||
A workflow engine orchestrates the execution of workflows. It schedules workflows on workers, retries failed workflows, and provides integrations for monitoring and debugging workflows.
|
||||
|
||||
Ultimately, the goal of Hatchet workflows are that you don't need to write these actions yourself -- creating a robust set of prebuilt integrations is one of the goals of the project.
|
||||
## Project Goals
|
||||
|
||||
### Why is this useful?
|
||||
Hatchet has the following high-level goals:
|
||||
|
||||
- When deploying a workflow engine or task queue, one of the first breaking points is requiring a robust event architecture for monitoring and replaying events. Hatchet provides this out of the box, allowing you to replay your events and retrigger your workflows.
|
||||
- No need to build all of your plumbing logic (action 1 -> event 1 -> action 2 -> event 2). Just define your jobs and steps and write your business logic. This is particularly useful the more complex your workflows become.
|
||||
- Using prebuilt integrations with a standard interface makes building auxiliary services like notification systems, billing, backups, and auditing much easier. **Please file an issue if you'd like to see an integration supported.** The following are on the roadmap:
|
||||
- Email providers: Sendgrid, Postmark, AWS SES
|
||||
- Stripe
|
||||
- AWS S3
|
||||
- Additionally, if you're already familiar with/using a workflow engine, making workflows declarative provides several benefits:
|
||||
- Makes spec'ing, debugging and visualizing workflows much simpler
|
||||
- Automatically updates triggers, schedules, and timeouts when they change, rather than doing this through a UI/CLI/SDK
|
||||
- Makes monitoring easier to build by logically separating units of work - jobs will automatically correspond to `BeginSpan`. OpenTelemetry support is on the roadmap.
|
||||
1. **Serve application developers:** we aim to support a broad set of languages and frameworks, to make it easier to support your existing applications. We currently support a Go SDK, with more languages coming soon.
|
||||
2. **Simple to setup:** we've seen too many overengineered stacks built on a fragile task queue with overly complex infrastructure. Hatchet is designed to be simple to setup, run locally, and deploy to your own infrastructure.
|
||||
3. **Flexibility when you need it:** as your application grows, you can use Hatchet to support complex, multi-step distributed workflows. Hatchet's backend is modular, allowing for customizing the implementation of the event storage API, queueing system, authentication, and more.
|
||||
|
||||
## Getting Started
|
||||
|
||||
For a set of end-to-end examples, see the [examples](./examples) directory.
|
||||
To get started, see the Hatchet documentation [here](https://docs.hatchet.run).
|
||||
|
||||
### Starting Hatchet
|
||||
## Github Issues
|
||||
|
||||
We are working on making it easier to start a Hatchet server. For now, see the [contributing guide](./CONTRIBUTING.md) for starting the Hatchet engine.
|
||||
|
||||
### Writing a Workflow
|
||||
|
||||
By default, Hatchet searches for workflows in the `.hatchet` folder relative to the directory you run your application in.
|
||||
|
||||
There are two main sections of a workflow file:
|
||||
|
||||
**`triggers`**
|
||||
|
||||
This section specifies what triggers a workflow. This can be events or a crontab-like schedule. For example, the following are valid triggers:
|
||||
|
||||
```yaml
|
||||
triggers:
|
||||
events:
|
||||
- event_key_1
|
||||
- event_key_2
|
||||
```
|
||||
|
||||
```yaml
|
||||
triggers:
|
||||
crons:
|
||||
- "* * * * *"
|
||||
```
|
||||
|
||||
**Jobs**
|
||||
|
||||
After defining your triggers, you define a list of jobs to run based on the triggers. **Jobs run in parallel.** Jobs contain the following fields:
|
||||
|
||||
```yaml
|
||||
# ...
|
||||
jobs:
|
||||
my-awesome-job:
|
||||
# (optional) A timeout value for the entire job
|
||||
timeout: 60s
|
||||
# (required) A set of steps for the job; see below
|
||||
steps: []
|
||||
```
|
||||
|
||||
Within each job, there are a set of **steps** which run sequentially. A step can contain the following fields:
|
||||
|
||||
```yaml
|
||||
# (required) the name of the step
|
||||
name: Step 1
|
||||
# (required) a unique id for the step (can be referenced by future steps)
|
||||
id: step-1
|
||||
# (required) the action id in the form of "integration_id:action".
|
||||
action: "slack:create-channel"
|
||||
# (required) the timeout of the individual step
|
||||
timeout: 15s
|
||||
# (optional or required, depending on integration) input data to the integration
|
||||
with:
|
||||
key: val
|
||||
```
|
||||
|
||||
### Creating a Worker
|
||||
|
||||
Workers can be created using:
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
"github.com/hatchet-dev/hatchet/cmd/cmdutils"
|
||||
"github.com/hatchet-dev/hatchet/pkg/client"
|
||||
"github.com/hatchet-dev/hatchet/pkg/worker"
|
||||
)
|
||||
|
||||
type userCreateEvent struct {
|
||||
Username string `json:"username"`
|
||||
UserId string `json:"user_id"`
|
||||
Data map[string]string `json:"data"`
|
||||
}
|
||||
|
||||
type actionInput struct {
|
||||
Message string `json:"message"`
|
||||
}
|
||||
|
||||
func main() {
|
||||
client, err := client.New(
|
||||
client.InitWorkflows(),
|
||||
)
|
||||
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
worker, err := worker.NewWorker(
|
||||
worker.WithDispatcherClient(
|
||||
client.Dispatcher(),
|
||||
),
|
||||
)
|
||||
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
err = worker.RegisterAction("echo:echo", func(ctx context.Context, input *actionInput) (result any, err error) {
|
||||
return map[string]interface{}{
|
||||
"message": input.Message,
|
||||
}, nil
|
||||
})
|
||||
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
interruptCtx, cancel := cmdutils.InterruptContext(cmdutils.InterruptChan())
|
||||
defer cancel()
|
||||
|
||||
worker.Start(interruptCtx)
|
||||
}
|
||||
```
|
||||
|
||||
You can configure the worker with your own set of workflow files using the `client.WithWorkflowFiles` option.
|
||||
|
||||
### Triggering Events
|
||||
|
||||
To trigger events from your main application, use the `client.Event().Push` method:
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/hatchet-dev/hatchet/pkg/client"
|
||||
)
|
||||
|
||||
type userCreateEvent struct {
|
||||
Username string `json:"username"`
|
||||
UserId string `json:"user_id"`
|
||||
Data map[string]string `json:"data"`
|
||||
}
|
||||
|
||||
func main() {
|
||||
client, err := client.New()
|
||||
|
||||
testEvent := userCreateEvent{
|
||||
Username: "echo-test",
|
||||
UserId: "1234",
|
||||
Data: map[string]string{
|
||||
"test": "test",
|
||||
},
|
||||
}
|
||||
|
||||
err = client.Event().Push(
|
||||
context.Background(),
|
||||
"user:create",
|
||||
testEvent,
|
||||
)
|
||||
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
You can configure the dispatcher with your own set of workflow files using the `dispatcher.WithWorkflowFiles` option.
|
||||
|
||||
## Why should I care?
|
||||
|
||||
**If you're unfamiliar with background task processing**
|
||||
|
||||
Many APIs start out without a task processing/worker service. You might not need it, but at a certain level of complexity, you probably will. There are a few use-cases where workers start to make sense:
|
||||
|
||||
1. You need to run scheduled tasks which that aren't triggered from your core API. For example, this may be a daily cleanup task, like traversing soft-deleted database entries or backing up data to S3.
|
||||
2. You need to run tasks which are triggered by API events, but aren't required for the core business logic of the handler. For example, you want to add a user to your CRM after they sign up.
|
||||
|
||||
For both of these cases, it's typical to re-use a lot of core functionality from your API, so the most natural place to start is by adding some automation within your API itself; for example, after returning `201 Created`, you might send a greeting to the user, initialize a sandbox environment, send an internal notification that a user signed up, etc, all within your API handlers. Let's say you've handled this case as following:
|
||||
|
||||
```go
|
||||
// Hypothetical handler called via a routing package, let's just pretend it returns an error
|
||||
func MyHandler(ctx context.Context, w http.ResponseWriter, r *http.Request) error {
|
||||
// Boilerplate code to parse the request
|
||||
var newUser User
|
||||
err := json.NewDecoder(r.Body).Decode(&newUser)
|
||||
if err != nil {
|
||||
http.Error(w, "Invalid user data", http.StatusBadRequest)
|
||||
return err
|
||||
}
|
||||
|
||||
// Validate email and password fields...
|
||||
// (Add your validation logic here)
|
||||
|
||||
// Create a user in the database
|
||||
user, err := createUser(ctx, newUser.Email, newUser.Password)
|
||||
if err != nil {
|
||||
// Handle database errors, such as unique constraint violation
|
||||
http.Error(w, "Error creating user", http.StatusInternalServerError)
|
||||
return err
|
||||
}
|
||||
|
||||
// Return 201 created with user type
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(http.StatusCreated)
|
||||
|
||||
// Send user a greeting
|
||||
err := email.SendGreetingEmail(context.Background(), user)
|
||||
|
||||
if err != nil {
|
||||
// can't return an error, since header is already set
|
||||
fmt.Println(err)
|
||||
}
|
||||
|
||||
// ... other post-signup operations
|
||||
}
|
||||
```
|
||||
|
||||
At some point, you realize all of these background operations don't really belong in the handler -- when they're part of the handler, they're more difficult to monitor and observe, difficult to retry (especially if a third-party service goes down), and bloat your handlers (which could cause goroutine leakage or memory issues).
|
||||
|
||||
This is where a service like Hatchet suited for background/task processing comes in.
|
||||
Please submit any bugs that you encounter via Github issues. However, please reach out on [Discord](https://discord.gg/ZMeUafwH89) before submitting a feature request - as the project is very early, we'd like to build a solid foundation before adding more complex features.
|
||||
|
||||
## I'd Like to Contribute
|
||||
|
||||
Hatchet is still in very early development -- as a result, there are very few development docs. However, please feel free to reach out on the #contributing channel on [Slack](https://join.slack.com/t/hatchet-co/signup) to shape the direction of the project.
|
||||
See the contributing docs [here](./CONTRIBUTING.md), and please let us know what you're interesting in working on in the #contributing channel on [Discord](https://discord.gg/ZMeUafwH89). This will help us shape the direction of the project and will make collaboration much easier!
|
||||
|
||||
2
frontend/docs/.gitignore
vendored
Normal file
2
frontend/docs/.gitignore
vendored
Normal file
@@ -0,0 +1,2 @@
|
||||
.next
|
||||
node_modules
|
||||
21
frontend/docs/LICENSE
Normal file
21
frontend/docs/LICENSE
Normal file
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2022 Shu Ding
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
23
frontend/docs/README.md
Normal file
23
frontend/docs/README.md
Normal file
@@ -0,0 +1,23 @@
|
||||
# Nextra Docs Template
|
||||
|
||||
This is a template for creating documentation with [Nextra](https://nextra.site).
|
||||
|
||||
[**Live Demo →**](https://nextra-docs-template.vercel.app)
|
||||
|
||||
[](https://nextra-docs-template.vercel.app)
|
||||
|
||||
## Quick Start
|
||||
|
||||
Click the button to clone this repository and deploy it on Vercel:
|
||||
|
||||
[](https://vercel.com/new/clone?s=https%3A%2F%2Fgithub.com%2Fshuding%2Fnextra-docs-template&showOptionalTeamCreation=false)
|
||||
|
||||
## Local Development
|
||||
|
||||
First, run `pnpm i` to install the dependencies.
|
||||
|
||||
Then, run `pnpm dev` to start the development server and visit localhost:3000.
|
||||
|
||||
## License
|
||||
|
||||
This project is licensed under the MIT License.
|
||||
6
frontend/docs/components/counters.module.css
Normal file
6
frontend/docs/components/counters.module.css
Normal file
@@ -0,0 +1,6 @@
|
||||
.counter {
|
||||
border: 1px solid #ccc;
|
||||
border-radius: 5px;
|
||||
padding: 2px 6px;
|
||||
margin: 12px 0 0;
|
||||
}
|
||||
24
frontend/docs/components/counters.tsx
Normal file
24
frontend/docs/components/counters.tsx
Normal file
@@ -0,0 +1,24 @@
|
||||
// Example from https://beta.reactjs.org/learn
|
||||
|
||||
import { useState } from 'react'
|
||||
import styles from './counters.module.css'
|
||||
|
||||
function MyButton() {
|
||||
const [count, setCount] = useState(0)
|
||||
|
||||
function handleClick() {
|
||||
setCount(count + 1)
|
||||
}
|
||||
|
||||
return (
|
||||
<div>
|
||||
<button onClick={handleClick} className={styles.counter}>
|
||||
Clicked {count} times
|
||||
</button>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
export default function MyApp() {
|
||||
return <MyButton />
|
||||
}
|
||||
5
frontend/docs/next-env.d.ts
vendored
Normal file
5
frontend/docs/next-env.d.ts
vendored
Normal file
@@ -0,0 +1,5 @@
|
||||
/// <reference types="next" />
|
||||
/// <reference types="next/image-types/global" />
|
||||
|
||||
// NOTE: This file should not be edited
|
||||
// see https://nextjs.org/docs/basic-features/typescript for more information.
|
||||
6
frontend/docs/next.config.js
Normal file
6
frontend/docs/next.config.js
Normal file
@@ -0,0 +1,6 @@
|
||||
const withNextra = require('nextra')({
|
||||
theme: 'nextra-theme-docs',
|
||||
themeConfig: './theme.config.tsx',
|
||||
})
|
||||
|
||||
module.exports = withNextra()
|
||||
8672
frontend/docs/package-lock.json
generated
Normal file
8672
frontend/docs/package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load Diff
31
frontend/docs/package.json
Normal file
31
frontend/docs/package.json
Normal file
@@ -0,0 +1,31 @@
|
||||
{
|
||||
"name": "nextra-docs-template",
|
||||
"version": "0.0.1",
|
||||
"description": "Nextra docs template",
|
||||
"scripts": {
|
||||
"dev": "next dev",
|
||||
"build": "next build",
|
||||
"start": "next start"
|
||||
},
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "git+https://github.com/shuding/nextra-docs-template.git"
|
||||
},
|
||||
"author": "Shu Ding <g@shud.in>",
|
||||
"license": "MIT",
|
||||
"bugs": {
|
||||
"url": "https://github.com/shuding/nextra-docs-template/issues"
|
||||
},
|
||||
"homepage": "https://github.com/shuding/nextra-docs-template#readme",
|
||||
"dependencies": {
|
||||
"next": "^13.0.6",
|
||||
"nextra": "latest",
|
||||
"nextra-theme-docs": "latest",
|
||||
"react": "^18.2.0",
|
||||
"react-dom": "^18.2.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "18.11.10",
|
||||
"typescript": "^4.9.3"
|
||||
}
|
||||
}
|
||||
5
frontend/docs/pages/_meta.json
Normal file
5
frontend/docs/pages/_meta.json
Normal file
@@ -0,0 +1,5 @@
|
||||
{
|
||||
"index": "Introduction",
|
||||
"quickstart": "Quickstart",
|
||||
"go-sdk": "Go SDK"
|
||||
}
|
||||
3
frontend/docs/pages/go-sdk.mdx
Normal file
3
frontend/docs/pages/go-sdk.mdx
Normal file
@@ -0,0 +1,3 @@
|
||||
# Go SDK
|
||||
|
||||
This contains the documentation for the Go SDK.
|
||||
58
frontend/docs/pages/go-sdk/creating-a-worker.mdx
Normal file
58
frontend/docs/pages/go-sdk/creating-a-worker.mdx
Normal file
@@ -0,0 +1,58 @@
|
||||
# Creating a Worker
|
||||
|
||||
Workers can be created via the `worker.NewWorker` method, which requires the `worker.WithClient` arg to work:
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"github.com/hatchet-dev/hatchet/pkg/client"
|
||||
"github.com/hatchet-dev/hatchet/pkg/worker"
|
||||
)
|
||||
|
||||
func main() {
|
||||
c, err := client.New(
|
||||
// this is the GRPC host and port of the Hatchet instance
|
||||
client.WithHostPort("127.0.0.1", 7077),
|
||||
)
|
||||
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
w, err := worker.NewWorker(
|
||||
worker.WithClient(c),
|
||||
)
|
||||
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
// ... workflow code
|
||||
|
||||
// start the worker
|
||||
w.Start(context.Background())
|
||||
}
|
||||
```
|
||||
|
||||
## Termination Signals
|
||||
|
||||
The worker will terminate when the context passed to `Start` is cancelled. Hatchet provides the `cmdutils.NewInterruptContext` method to create a context that will be cancelled when the process receives an interrupt signal (e.g. `SIGINT` or `SIGTERM`). This can be used via:
|
||||
|
||||
```go
|
||||
ctx, cancel := cmdutils.NewInterruptContext()
|
||||
|
||||
defer cancel()
|
||||
|
||||
w.Start(ctx)
|
||||
```
|
||||
|
||||
## All Worker Options
|
||||
|
||||
### `worker.WithClient`
|
||||
|
||||
The client to use to communicate with the Hatchet instance. This is required.
|
||||
|
||||
### `worker.WithName`
|
||||
|
||||
The name of the worker. This is used to identify the worker in the Hatchet UI.
|
||||
109
frontend/docs/pages/go-sdk/creating-a-workflow.mdx
Normal file
109
frontend/docs/pages/go-sdk/creating-a-workflow.mdx
Normal file
@@ -0,0 +1,109 @@
|
||||
# Creating a Workflow
|
||||
|
||||
The simplest way to define a workflow is by using the `worker.On` method. This method accepts two arguments: a workflow trigger and the workflow definition. For example, to trigger a workflow on the `user:created` event, you can do the following:
|
||||
|
||||
```go
|
||||
w.On(worker.Event("user:created"), &worker.WorkflowJob{
|
||||
Name: "post-user-sign-up",
|
||||
Description: "Workflow that executes after a user signs up.",
|
||||
Timeout: "60s",
|
||||
Steps: []worker.WorkflowStep{
|
||||
{
|
||||
Function: func(ctx context.Context) error {
|
||||
return nil
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
## Passing Arguments to Step Functions
|
||||
|
||||
Step functions must always accept a `context.Context` as the first argument, and must return an `error` as the last return value. They can optionally accept a second argument or return a value -- both of these must be a pointer to a struct. The following are valid step functions:
|
||||
|
||||
```go
|
||||
func (ctx context.Context) error
|
||||
func (ctx context.Context, in *myInput) error
|
||||
func (ctx context.Context, in *myInput) (*myOutput, error)
|
||||
func (ctx context.Context) (*myOutput, error)
|
||||
```
|
||||
|
||||
> Why pointers to structs? We use JSON marshalling/unmarshalling under the hood, and pointers to structs are the most predictable way to marshal and unmarshal values. You can use `json` tags and `MarshalJSON` + `UnmarshalJSON` methods to customize the marshalling/unmarshalling behavior.
|
||||
|
||||
Additionally, if the second argument of the first step function is a pointer to a struct, the event payload will be unmarshalled into that struct. For example, given the following event:
|
||||
|
||||
```go
|
||||
type MyEvent struct {
|
||||
Name string `json:"name"`
|
||||
}
|
||||
```
|
||||
|
||||
You can declare the following step function:
|
||||
|
||||
```go
|
||||
func FirstStep(ctx context.Context, event *MyEvent) error {
|
||||
fmt.Println("got event: ", event.Name)
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
If the second argument of a step function matches the type of the first return value of the previous step function, the return value of the previous step function will be passed to the next step function. For example, the `firstStepOutput` struct will be passed between step functions:
|
||||
|
||||
```go
|
||||
type firstStepOutput struct {
|
||||
Output string `json:"output"`
|
||||
}
|
||||
|
||||
func FirstStep(ctx context.Context) (*firstStepOutput, error) {
|
||||
return &firstStepOutput{Output: "foo"}, nil
|
||||
}
|
||||
|
||||
func SecondStep(ctx context.Context, in *firstStepOutput) error {
|
||||
fmt.Println("got first step output: ", in.Output)
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
## Services
|
||||
|
||||
Services are a way to logically group workflows into different categories. For example, you may have a `user` service that contains all workflows related to users. You can define a service by using the `worker.NewService` method. For example, to define a `user` service, you can do the following:
|
||||
|
||||
```go
|
||||
userService := w.NewService("user")
|
||||
|
||||
userService.On(worker.Event("user:created"), &worker.WorkflowJob{
|
||||
Name: "post-user-sign-up",
|
||||
Description: "Workflow that executes after a user signs up.",
|
||||
Timeout: "60s",
|
||||
Steps: []worker.WorkflowStep{
|
||||
{
|
||||
Function: func(ctx context.Context) error {
|
||||
fmt.Println("running post-user sign up")
|
||||
return nil
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
While this is mostly a convenience method at the moment, we plan to add more features to services in the future, like service-level metrics and service-level retries.
|
||||
|
||||
## Cron Schedules
|
||||
|
||||
You can declare a cron schedule by passing `worker.Cron` to the `worker.On` method. For example, to trigger a workflow every 5 minutes, you can do the following:
|
||||
|
||||
```go
|
||||
w.On(worker.Cron("*/5 * * * *"), &worker.WorkflowJob{
|
||||
Name: "my-cron-job",
|
||||
Description: "Cron workflow example.",
|
||||
Timeout: "60s",
|
||||
Steps: []worker.WorkflowStep{
|
||||
{
|
||||
Function: func(ctx context.Context) error {
|
||||
fmt.Println("triggered at:", time.Now())
|
||||
return nil
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
```
|
||||
23
frontend/docs/pages/go-sdk/pushing-events.mdx
Normal file
23
frontend/docs/pages/go-sdk/pushing-events.mdx
Normal file
@@ -0,0 +1,23 @@
|
||||
# Pushing Events
|
||||
|
||||
Events can be pushed via the client's `Event().Push` method:
|
||||
|
||||
```go
|
||||
c, err := client.New(
|
||||
client.WithHostPort("127.0.0.1", 7077),
|
||||
)
|
||||
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
c.Event().Push(
|
||||
context.Background(),
|
||||
"test-called",
|
||||
&events.TestEvent{
|
||||
Name: "testing",
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
Events are marshalled/unmarshalled using the `encoding/json` package, so any event type must be JSON serializable.
|
||||
58
frontend/docs/pages/index.mdx
Normal file
58
frontend/docs/pages/index.mdx
Normal file
@@ -0,0 +1,58 @@
|
||||
# Introduction
|
||||
|
||||
Welcome to the Hatchet documentation! Hatchet is a self-hostable workflow engine built for application developers.
|
||||
|
||||
**What is a workflow?**
|
||||
|
||||
The term `workflow` tends to be overloaded, so let's make things more clear - in Hatchet, a workflow is a set of functions which are executed in response to an external trigger (an event, schedule, or API call). For example, if you'd like to send notifications to a user after they've signed up, you could create a workflow for that.
|
||||
|
||||
**Why is that useful?**
|
||||
|
||||
Instead of processing background tasks and functions in your application handlers, which can lead to complex code, hard-to-debug errors, and resource contention, you can distribute these workflows between a set of `workers`. Workers are long-running processes which listen for events, and execute the functions defined in your workflows.
|
||||
|
||||
**What is a workflow engine?**
|
||||
|
||||
A workflow engine orchestrates the execution of workflows. It schedules workflows on workers, retries failed workflows, and provides integrations for monitoring and debugging workflows.
|
||||
|
||||
## Project Goals
|
||||
|
||||
Hatchet has the following high-level goals:
|
||||
|
||||
1. **Serve application developers:** we aim to support a broad set of languages and frameworks, to make it easier to support your existing applications. We currently support a Go SDK, with more languages coming soon.
|
||||
2. **Simple to setup:** we've seen too many overengineered stacks built on a fragile task queue with overly complex infrastructure. Hatchet is designed to be simple to setup, run locally, and deploy to your own infrastructure.
|
||||
3. **Flexibility when you need it:** as your application grows, you can use Hatchet to support complex, multi-step distributed workflows. Hatchet's backend is modular, allowing for customizing the implementation of the event storage API, queueing system, authentication, and more.
|
||||
|
||||
## Features
|
||||
|
||||
We currently support the following features, with many more on the roadmap:
|
||||
|
||||
- **Declarative workflows:** define workflows which run when events occur, with support for timeouts and parallel or sequential execution:
|
||||
|
||||
```go filename="worker.go" {3} copy
|
||||
w.On(
|
||||
worker.Event("event:test"),
|
||||
&worker.WorkflowJob{
|
||||
Name: "example-workflow",
|
||||
Description: "Example workflow.",
|
||||
Timeout: "60s",
|
||||
Steps: []worker.WorkflowStep{
|
||||
{
|
||||
Function: func(ctx context.Context, event *events.TestEvent) error {
|
||||
fmt.Println("got event: ", event.Name)
|
||||
return nil
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
- **Event storage API:** store events in a durable event log, with support for querying and filtering events:
|
||||
|
||||

|
||||
|
||||
- **Web platform for visibility and monitoring**: view the status of your workflows, and drill down into individual events:
|
||||
|
||||

|
||||
|
||||
Check out the [Quickstart](/quickstart) guide to work through a simple example of using Hatchet.
|
||||
88
frontend/docs/pages/quickstart.mdx
Normal file
88
frontend/docs/pages/quickstart.mdx
Normal file
@@ -0,0 +1,88 @@
|
||||
# Quickstart
|
||||
|
||||
> Note: this guide is written for Go developers. We have more SDKs on the roadmap, please [get in touch](mailto:contact@hatchet.run) if you would like to be notified when they're available (or you'd like to help us build one!).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
This quickstart example requires the following tools to work:
|
||||
|
||||
- `go 1.18+`
|
||||
- [`docker`](https://docs.docker.com/engine/install/)
|
||||
- [`caddy`](https://caddyserver.com/docs/install)
|
||||
|
||||
## Hatchet Instance Setup
|
||||
|
||||
1. Clone the quickstart repository:
|
||||
|
||||
```sh
|
||||
git clone https://github.com/hatchet-dev/hatchet-go-quickstart.git && cd hatchet-go-quickstart
|
||||
```
|
||||
|
||||
2. Run `go mod download` to fetch all packages.
|
||||
|
||||
3. Run `docker compose up` to start the Hatchet instance. This will take a few minutes, as the docker compose services set up the database and generate the required certificates to connect to the Hatchet instance. You can also run `docker compose up -d` to start this in the background. Once you start to see output from the `engine` and `api` services, you can move on to the next step.
|
||||
|
||||
4. Run `caddy start` to get an instance running. You should be able to navigate to [app.dev.hatchet-tools.com](https://app.dev.hatchet-tools.com) and use the following credentials to log in:
|
||||
|
||||
```
|
||||
Email: admin@example.com
|
||||
Password: Admin123!!
|
||||
```
|
||||
|
||||
5. Create the required environment variables via:
|
||||
|
||||
```sh
|
||||
cat > .env <<EOF
|
||||
HATCHET_CLIENT_TENANT_ID=707d0855-80ab-4e1f-a156-f1c4546cbf52
|
||||
HATCHET_CLIENT_TLS_ROOT_CA_FILE=./certs/ca.cert
|
||||
HATCHET_CLIENT_TLS_CERT_FILE=./certs/client-worker.pem
|
||||
HATCHET_CLIENT_TLS_KEY_FILE=./certs/client-worker.key
|
||||
HATCHET_CLIENT_TLS_SERVER_NAME=cluster
|
||||
EOF
|
||||
```
|
||||
|
||||
6. Run the server and worker in two separate shell sessions via: `go run ./cmd/server` and `go run ./cmd/worker`.
|
||||
|
||||
7. Run `curl http://localhost:1323/test` to test the endpoint. You will see events and workflows populated in the Hatchet dashboard:
|
||||
|
||||
<img width="1728" alt="image" src="https://github.com/hatchet-dev/hatchet-go-quickstart/assets/25448214/376e4ee8-7233-4a84-85b8-f71ad9e7402e" />
|
||||
|
||||
## Explanation
|
||||
|
||||
### Server
|
||||
|
||||
The server is a simple HTTP server that exposes a single endpoint at `/test`. When this endpoint is hit, it will create a new workflow and uses the Hatchet events API to send an event to Hatchet.
|
||||
|
||||
```go filename="server.go" copy
|
||||
hatchetClient.Event().Push(
|
||||
context.Background(),
|
||||
"test-called",
|
||||
&events.TestEvent{
|
||||
Name: "testing",
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
### Worker
|
||||
|
||||
The worker is a simple worker that listens for events with the key `test-called` from Hatchet. When an event is received, it will print the event to the console.
|
||||
|
||||
```go filename="worker.go" copy
|
||||
w.On(worker.Event("test-called"), &worker.WorkflowJob{
|
||||
Name: "event-test",
|
||||
Description: "Test workflow.",
|
||||
Timeout: "60s",
|
||||
Steps: []worker.WorkflowStep{
|
||||
{
|
||||
Function: func(ctx context.Context, event *events.TestEvent) error {
|
||||
fmt.Println("got event: ", event.Name)
|
||||
return nil
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
Hatchet has many powerful features that can be used to build complex workflows. Check out the [Go SDK](./go-sdk) for more information.
|
||||
2172
frontend/docs/pnpm-lock.yaml
generated
Normal file
2172
frontend/docs/pnpm-lock.yaml
generated
Normal file
File diff suppressed because it is too large
Load Diff
BIN
frontend/docs/public/favicon.ico
Normal file
BIN
frontend/docs/public/favicon.ico
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 15 KiB |
BIN
frontend/docs/public/features-1.png
Normal file
BIN
frontend/docs/public/features-1.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 824 KiB |
BIN
frontend/docs/public/features-2.png
Normal file
BIN
frontend/docs/public/features-2.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 755 KiB |
BIN
frontend/docs/public/features-3.png
Normal file
BIN
frontend/docs/public/features-3.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 983 KiB |
BIN
frontend/docs/public/hatchet_logo.png
Normal file
BIN
frontend/docs/public/hatchet_logo.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 8.0 KiB |
29
frontend/docs/theme.config.tsx
Normal file
29
frontend/docs/theme.config.tsx
Normal file
@@ -0,0 +1,29 @@
|
||||
import React from "react";
|
||||
import { DocsThemeConfig } from "nextra-theme-docs";
|
||||
import Image from "next/image";
|
||||
|
||||
const config: DocsThemeConfig = {
|
||||
logo: (
|
||||
<Image src="/hatchet_logo.png" alt="Hatchet logo" width={120} height={35} />
|
||||
),
|
||||
primaryHue: 210,
|
||||
primarySaturation: 10,
|
||||
project: {
|
||||
link: "https://github.com/hatchet-dev/hatchet",
|
||||
},
|
||||
chat: {
|
||||
link: "https://discord.gg/ZMeUafwH89",
|
||||
},
|
||||
docsRepositoryBase: "https://github.com/hatchet-dev/hatchet/frontend/docs",
|
||||
footer: {
|
||||
text: "Hatchet",
|
||||
},
|
||||
head: (
|
||||
<>
|
||||
<link rel="icon" type="image/png" href="/favicon.ico" />
|
||||
<title>Hatchet Documentation</title>{" "}
|
||||
</>
|
||||
),
|
||||
};
|
||||
|
||||
export default config;
|
||||
20
frontend/docs/tsconfig.json
Normal file
20
frontend/docs/tsconfig.json
Normal file
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "es5",
|
||||
"lib": ["dom", "dom.iterable", "esnext"],
|
||||
"allowJs": true,
|
||||
"skipLibCheck": true,
|
||||
"strict": false,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"noEmit": true,
|
||||
"incremental": true,
|
||||
"esModuleInterop": true,
|
||||
"module": "esnext",
|
||||
"moduleResolution": "node",
|
||||
"resolveJsonModule": true,
|
||||
"isolatedModules": true,
|
||||
"jsx": "preserve"
|
||||
},
|
||||
"include": ["next-env.d.ts", "**/*.ts", "**/*.tsx"],
|
||||
"exclude": ["node_modules"]
|
||||
}
|
||||
Reference in New Issue
Block a user