mirror/hatchet
1
0
Fork 0
mirror of https://github.com/hatchet-dev/hatchet.git synced 2026-04-22 18:19:17 -05:00
Code Issues Packages Projects Releases Wiki Activity

Feat: OLAP Table for CEL Eval Failures (#2012)

Browse Source 
* feat: add table, wire up partitioning

* feat: wire failures into the OLAP db from rabbit

* feat: bubble failures up to controller

* fix: naming

* fix: hack around enum type

* fix: typo

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: typos

* fix: migration name

* feat: log debug failure

* feat: pub message from debug endpoint to log failure

* fix: error handling

* fix: use ingestor

* fix: olap suffix

* fix: pass source through

* fix: dont log ingest failure

* fix: rm debug as enum opt

* chore: gen

* Feat: Webhooks (#1978)

* feat: migration + go gen

* feat: non unique source name

* feat: api types

* fix: rm cruft

* feat: initial api for webhooks

* feat: handle encryption of incoming keys

* fix: nil pointer errors

* fix: import

* feat: add endpoint for incoming webhooks

* fix: naming

* feat: start wiring up basic auth

* feat: wire up cel event parsing

* feat: implement authentication

* fix: hack for plain text content

* feat: add source to enum

* feat: add source name enum

* feat: db source name enum fix

* fix: use source name enums

* feat: nest sources

* feat: first pass at stripe

* fix: clean up source name passing

* fix: use unique name for webhook

* feat: populator test

* fix: null values

* fix: ordering

* fix: rm unnecessary index

* fix: validation

* feat: validation on create

* fix: lint

* fix: naming

* feat: wire triggering webhook name through to events table

* feat: cleanup + python gen + e2e test for basic auth

* feat: query to insert webhook validation errors

* refactor: auth handler

* fix: naming

* refactor: validation errors, part II

* feat: wire up writes through olap

* fix: linting, fallthrough case

* fix: validation

* feat: tests for failure cases for basic auth

* feat: expand tests

* fix: correctly return 404 out of task getter

* chore: generated stuff

* fix: rm cruft

* fix: longer sleep

* debug: print name + events to logs

* feat: limit to N

* feat: add limit env var

* debug: ci test

* fix: apply namespaces to keys

* fix: namespacing, part ii

* fix: sdk config

* fix: handle prefixing

* feat: handle partitioning logic

* chore: gen

* feat: add webhook limit

* feat: wire up limits

* fix: gen

* fix: reverse order of generic fallthrough

* fix: comment for potential unexpected behavior

* fix: add check constraints, improve error handling

* chore: gen

* chore: gen

* fix: improve naming

* feat: scaffold webhooks page

* feat: sidebar

* feat: first pass at page

* feat: improve feedback on UI

* feat: initial work on create modal

* feat: change default to basic

* fix: openapi spec discriminated union

* fix: go side

* feat: start wiring up placeholders for stripe and github

* feat: pre-populated fields for Stripe + Github

* feat: add name section

* feat: copy improvements, show URL

* feat: UI cleanup

* fix: check if tenant populator errors

* feat: add comments

* chore: gen again

* fix: default name

* fix: styling

* fix: improve stripe header processing

* feat: docs, part 1

* fix: lint

* fix: migration order

* feat: implement rate limit per-webhook

* feat: comment

* feat: clean up docs

* chore: gen

* fix: migration versions

* fix: olap naming

* fix: partitions

* chore: gen

* feat: store webhook cel eval failures properly

* fix: pk order

* fix: auth tweaks, move fetches out of populator

* fix: pgtype.Text instead of string pointer

* chore: gen

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
matt
2025-07-30 13:27:38 -04:00
committed by GitHub
parent 1b2a2bf566
commit d6f8be2c0f
111 changed files with 11294 additions and 374 deletions
Download Patch File Download Diff File Expand all files Collapse all files
api-contracts/openapi/components/schemas/_index.yaml
+10
View File
@@ -108,6 +108,14 @@ V1FilterList:
$ref: "./v1/filter.yaml#/V1FilterList"
V1Filter:
$ref: "./v1/filter.yaml#/V1Filter"
V1WebhookList:
$ref: "./v1/webhook.yaml#/V1WebhookList"
V1Webhook:
$ref: "./v1/webhook.yaml#/V1Webhook"
V1WebhookSourceName:
$ref: "./v1/webhook.yaml#/V1WebhookSourceName"
V1WebhookAuthType:
$ref: "./v1/webhook.yaml#/V1WebhookAuthType"
RateLimit:
$ref: "./rate_limits.yaml#/RateLimit"
RateLimitList:
@@ -346,6 +354,8 @@ V1TaskTimingList:
$ref: "./v1/task.yaml#/V1TaskTimingList"
V1CreateFilterRequest:
$ref: "./v1/filter.yaml#/V1CreateFilterRequest"
V1CreateWebhookRequest:
$ref: "./v1/webhook.yaml#/V1CreateWebhookRequest"
V1UpdateFilterRequest:
$ref: "./v1/filter.yaml#/V1UpdateFilterRequest"
V1CELDebugRequest:
api-contracts/openapi/components/schemas/v1/event.yaml
+3
View File
@@ -48,6 +48,9 @@ V1Event:
type: array
items:
$ref: "#/V1EventTriggeredRun"
triggeringWebhookName:
type: string
description: The name of the webhook that triggered this event, if applicable.
required:
- metadata
api-contracts/openapi/components/schemas/v1/webhook.yaml
+186
View File
@@ -0,0 +1,186 @@
V1Webhook:
properties:
metadata:
$ref: "../metadata.yaml#/APIResourceMeta"
tenantId:
type: string
description: The ID of the tenant associated with this webhook.
name:
type: string
description: The name of the webhook
sourceName:
$ref: "#/V1WebhookSourceName"
description: The name of the source for this webhook
eventKeyExpression:
type: string
description: The CEL expression to use for the event key. This is used to create the event key from the webhook payload.
authType:
$ref: "#/V1WebhookAuthType"
description: The type of authentication to use for the webhook
required:
- metadata
- tenantId
- name
- sourceName
- eventKeyExpression
- authType
V1WebhookList:
type: object
properties:
pagination:
$ref: "../metadata.yaml#/PaginationResponse"
rows:
type: array
items:
$ref: "#/V1Webhook"
V1WebhookAuthType:
type: string
enum:
- BASIC
- API_KEY
- HMAC
# TODO: Add more sources here as we support them
# IMPORTANT: Keep these in sync with the database enum v1_incoming_webhook_source_name
V1WebhookSourceName:
type: string
enum:
- GENERIC
- GITHUB
- STRIPE
V1WebhookHMACAlgorithm:
type: string
enum:
- SHA1
- SHA256
- SHA512
- MD5
V1WebhookHMACEncoding:
type: string
enum:
- HEX
- BASE64
- BASE64URL
V1WebhookBasicAuth:
type: object
properties:
username:
type: string
description: The username for basic auth
password:
type: string
description: The password for basic auth
required:
- username
- password
V1WebhookAPIKeyAuth:
type: object
properties:
headerName:
type: string
description: The name of the header to use for the API key
apiKey:
type: string
description: The API key to use for authentication
required:
- headerName
- apiKey
V1WebhookHMACAuth:
type: object
properties:
algorithm:
$ref: "#/V1WebhookHMACAlgorithm"
description: The HMAC algorithm to use for the webhook
encoding:
$ref: "#/V1WebhookHMACEncoding"
description: The encoding to use for the HMAC signature
signatureHeaderName:
type: string
description: The name of the header to use for the HMAC signature
signingSecret:
type: string
description: The secret key used to sign the HMAC signature
required:
- algorithm
- encoding
- signatureHeaderName
- signingSecret
V1CreateWebhookRequestBase:
type: object
properties:
sourceName:
$ref: "#/V1WebhookSourceName"
description: The name of the source for this webhook
name:
type: string
description: The name of the webhook
eventKeyExpression:
type: string
description: The CEL expression to use for the event key. This is used to create the event key from the webhook payload.
required:
- name
- sourceName
- eventKeyExpression
V1CreateWebhookRequestBasicAuth:
allOf:
- $ref: "#/V1CreateWebhookRequestBase"
- type: object
properties:
authType:
type: string
enum:
- BASIC
description: The type of authentication to use for the webhook
auth:
$ref: "#/V1WebhookBasicAuth"
required:
- authType
- auth
V1CreateWebhookRequestAPIKey:
allOf:
- $ref: "#/V1CreateWebhookRequestBase"
- type: object
properties:
authType:
type: string
enum:
- API_KEY
description: The type of authentication to use for the webhook
auth:
$ref: "#/V1WebhookAPIKeyAuth"
required:
- authType
- auth
V1CreateWebhookRequestHMAC:
allOf:
- $ref: "#/V1CreateWebhookRequestBase"
- type: object
properties:
authType:
type: string
enum:
- HMAC
description: The type of authentication to use for the webhook
auth:
$ref: "#/V1WebhookHMACAuth"
required:
- authType
- auth
V1CreateWebhookRequest:
oneOf:
- $ref: "#/V1CreateWebhookRequestBasicAuth"
- $ref: "#/V1CreateWebhookRequestAPIKey"
- $ref: "#/V1CreateWebhookRequestHMAC"
api-contracts/openapi/openapi.yaml
+7
View File
@@ -58,6 +58,13 @@ paths:
$ref: "./paths/v1/filters/filter.yaml#/V1FilterListCreate"
/api/v1/stable/tenants/{tenant}/filters/{v1-filter}:
$ref: "./paths/v1/filters/filter.yaml#/V1FilterGetDeleteUpdate"
/api/v1/stable/tenants/{tenant}/webhooks:
$ref: "./paths/v1/webhooks/webhook.yaml#/V1WebhookListCreate"
# !!! IMPORTANT: we directly reference this path in the webhook rate limiter in `webhook_rate_limit.go`
# changing this path will break the rate limiter. !!!
/api/v1/stable/tenants/{tenant}/webhooks/{v1-webhook}:
$ref: "./paths/v1/webhooks/webhook.yaml#/V1WebhookGetDeleteReceive"
/api/v1/stable/tenants/{tenant}/cel/debug:
$ref: "./paths/v1/cel/cel.yaml#/V1CELDebug"
/api/ready:
api-contracts/openapi/paths/v1/webhooks/webhook.yaml
+263
View File
@@ -0,0 +1,263 @@
V1WebhookGetDeleteReceive:
get:
description: Get a webhook by its name
operationId: v1-webhook:get
parameters:
- description: The tenant id
in: path
name: tenant
required: true
schema:
type: string
format: uuid
minLength: 36
maxLength: 36
- description: The webhook name
in: path
name: v1-webhook
required: true
schema:
type: string
responses:
"200":
content:
application/json:
schema:
$ref: "../../../components/schemas/_index.yaml#/V1Webhook"
description: Successfully got the webhook
"400":
content:
application/json:
schema:
$ref: "../../../components/schemas/_index.yaml#/APIErrors"
description: A malformed or bad request
"403":
content:
application/json:
schema:
$ref: "../../../components/schemas/_index.yaml#/APIErrors"
description: Forbidden
summary: Get a webhook
tags:
- Webhook
delete:
x-resources: ["tenant", "v1-webhook"]
description: Delete a webhook
operationId: v1-webhook:delete
parameters:
- description: The tenant id
in: path
name: tenant
required: true
schema:
type: string
format: uuid
minLength: 36
maxLength: 36
- description: The name of the webhook to delete
in: path
name: v1-webhook
required: true
schema:
type: string
responses:
"200":
content:
application/json:
schema:
$ref: "../../../components/schemas/_index.yaml#/V1Webhook"
description: Successfully deleted webhook
"400":
content:
application/json:
schema:
$ref: "../../../components/schemas/_index.yaml#/APIErrors"
description: A malformed or bad request
"403":
content:
application/json:
schema:
$ref: "../../../components/schemas/_index.yaml#/APIErrors"
description: Forbidden
"404":
content:
application/json:
schema:
$ref: "../../../components/schemas/_index.yaml#/APIErrors"
description: Not found
tags:
- Webhook
post:
x-resources: ["tenant", "v1-webhook"]
description: Post an incoming webhook message
operationId: v1-webhook:receive
security: []
parameters:
- description: The tenant id
in: path
name: tenant
required: true
schema:
type: string
format: uuid
minLength: 36
maxLength: 36
- description: The webhook name
in: path
name: v1-webhook
required: true
schema:
type: string
requestBody:
description: Any payload in any format
required: false
content:
text/plain:
schema:
type: string
responses:
'200':
description: 'Success'
required: false
content:
application/json:
schema:
type: object
properties:
message:
type: string
example: 'OK'
"400":
content:
application/json:
schema:
$ref: "../../../components/schemas/_index.yaml#/APIErrors"
description: A malformed or bad request
"403":
content:
application/json:
schema:
$ref: "../../../components/schemas/_index.yaml#/APIErrors"
description: Forbidden
summary: Post a webhook message
tags:
- Webhook
V1WebhookListCreate:
get:
x-resources: ["tenant"]
description: Lists all webhook for a tenant.
operationId: v1-webhook:list
parameters:
- description: The tenant id
in: path
name: tenant
required: true
schema:
type: string
format: uuid
minLength: 36
maxLength: 36
- description: The number to skip
in: query
name: offset
required: false
schema:
type: integer
format: int64
- description: The number to limit by
in: query
name: limit
required: false
schema:
type: integer
format: int64
- description: The source names to filter by
in: query
name: sourceNames
required: false
schema:
type: array
items:
$ref: "../../../components/schemas/_index.yaml#/V1WebhookSourceName"
description: The source name
- description: The webhook names to filter by
in: query
name: webhookNames
required: false
schema:
type: array
items:
type: string
description: The webhook name
responses:
"200":
content:
application/json:
schema:
$ref: "../../../components/schemas/_index.yaml#/V1WebhookList"
description: Successfully listed the webhooks
"400":
content:
application/json:
schema:
$ref: "../../../components/schemas/_index.yaml#/APIErrors"
description: A malformed or bad request
"403":
content:
application/json:
schema:
$ref: "../../../components/schemas/_index.yaml#/APIErrors"
description: Forbidden
summary: List webhooks
tags:
- Webhook
post:
x-resources: ["tenant"]
description: Create a new webhook
operationId: v1-webhook:create
parameters:
- description: The tenant id
in: path
name: tenant
required: true
schema:
type: string
format: uuid
minLength: 36
maxLength: 36
requestBody:
content:
application/json:
schema:
$ref: "../../../components/schemas/_index.yaml#/V1CreateWebhookRequest"
description: The input to the webhook creation
required: true
responses:
"200":
content:
application/json:
schema:
$ref: "../../../components/schemas/_index.yaml#/V1Webhook"
description: Successfully created the webhook
"400":
content:
application/json:
schema:
$ref: "../../../components/schemas/_index.yaml#/APIErrors"
description: A malformed or bad request
"403":
content:
application/json:
schema:
$ref: "../../../components/schemas/_index.yaml#/APIErrors"
description: Forbidden
"404":
content:
application/json:
schema:
$ref: "../../../components/schemas/_index.yaml#/APIErrors"
description: Not found
summary: Create a webhook
tags:
- Webhook
api/v1/server/authn/middleware.go
+3
View File
@@ -36,6 +36,7 @@ func NewAuthN(config *server.ServerConfig) *AuthN {
func (a *AuthN) Middleware(r *middleware.RouteInfo) echo.HandlerFunc {
return func(c echo.Context) error {
err := a.authenticate(c, r)
if err != nil {
return err
}
@@ -58,6 +59,7 @@ func (a *AuthN) authenticate(c echo.Context, r *middleware.RouteInfo) error {
if r.Security.CookieAuth() {
cookieErr = a.handleCookieAuth(c)
c.Set("auth_strategy", "cookie")
if cookieErr == nil {
@@ -73,6 +75,7 @@ func (a *AuthN) authenticate(c echo.Context, r *middleware.RouteInfo) error {
if r.Security.BearerAuth() {
bearerErr = a.handleBearerAuth(c)
c.Set("auth_strategy", "bearer")
if bearerErr == nil {
api/v1/server/handlers/events/create.go
+1 -1
View File
@@ -32,7 +32,7 @@ func (t *EventService) EventCreate(ctx echo.Context, request gen.EventCreateRequ
}
}
newEvent, err := t.config.Ingestor.IngestEvent(ctx.Request().Context(), tenant, request.Body.Key, dataBytes, additionalMetadata, request.Body.Priority, request.Body.Scope)
newEvent, err := t.config.Ingestor.IngestEvent(ctx.Request().Context(), tenant, request.Body.Key, dataBytes, additionalMetadata, request.Body.Priority, request.Body.Scope, nil)
if err != nil {
if err == metered.ErrResourceExhausted {
api/v1/server/handlers/ingestors/sns.go
+1 -1
View File
@@ -64,7 +64,7 @@ func (i *IngestorsService) SnsUpdate(ctx echo.Context, req gen.SnsUpdateRequestO
return nil, err
}
default:
_, err := i.config.Ingestor.IngestEvent(ctx.Request().Context(), tenant, req.Event, body, nil, nil, nil)
_, err := i.config.Ingestor.IngestEvent(ctx.Request().Context(), tenant, req.Event, body, nil, nil, nil, nil)
if err != nil {
return nil, err
api/v1/server/handlers/v1/tasks/get.go
+7 -1
View File
@@ -13,7 +13,13 @@ func (t *TasksService) V1TaskGet(ctx echo.Context, request gen.V1TaskGetRequestO
taskInterface := ctx.Get("task")
if taskInterface == nil {
return nil, echo.NewHTTPError(404, "Task not found")
return gen.V1TaskGet404JSONResponse{
Errors: []gen.APIError{
{
Description: "task not found",
},
},
}, nil
}
task, ok := taskInterface.(*sqlcv1.V1TasksOlap)
api/v1/server/handlers/v1/webhooks/create.go
+184
View File
@@ -0,0 +1,184 @@
package webhooksv1
import (
"encoding/json"
"fmt"
"github.com/hatchet-dev/hatchet/api/v1/server/oas/gen"
"github.com/hatchet-dev/hatchet/api/v1/server/oas/transformers/v1"
"github.com/hatchet-dev/hatchet/pkg/repository/postgres/dbsqlc"
"github.com/hatchet-dev/hatchet/pkg/repository/postgres/sqlchelpers"
v1 "github.com/hatchet-dev/hatchet/pkg/repository/v1"
"github.com/hatchet-dev/hatchet/pkg/repository/v1/sqlcv1"
"github.com/labstack/echo/v4"
)
func (w *V1WebhooksService) V1WebhookCreate(ctx echo.Context, request gen.V1WebhookCreateRequestObject) (gen.V1WebhookCreateResponseObject, error) {
tenant := ctx.Get("tenant").(*dbsqlc.Tenant)
canCreate, _, err := w.config.EntitlementRepository.TenantLimit().CanCreate(ctx.Request().Context(), dbsqlc.LimitResourceINCOMINGWEBHOOK, tenant.ID.String(), 1)
if err != nil {
return nil, fmt.Errorf("failed to check if webhook can be created: %w", err)
}
if !canCreate {
return gen.V1WebhookCreate400JSONResponse{
Errors: []gen.APIError{
{
Description: "incoming webhook limit reached",
},
},
}, nil
}
params, err := w.constructCreateOpts(tenant.ID.String(), *request.Body)
if err != nil {
return gen.V1WebhookCreate400JSONResponse{
Errors: []gen.APIError{
{
Description: fmt.Sprintf("failed to construct webhook create params: %v", err),
},
},
}, nil
}
webhook, err := w.config.V1.Webhooks().CreateWebhook(
ctx.Request().Context(),
tenant.ID.String(),
params,
)
if err != nil {
return nil, fmt.Errorf("failed to create webhook")
}
transformed := transformers.ToV1Webhook(webhook)
return gen.V1WebhookCreate200JSONResponse(transformed), nil
}
func extractAuthType(request gen.V1CreateWebhookRequest) (sqlcv1.V1IncomingWebhookAuthType, error) {
j, err := request.MarshalJSON()
if err != nil {
return "", fmt.Errorf("failed to get marshal request: %w", err)
}
parsedBody := make(map[string]interface{})
if err := json.Unmarshal(j, &parsedBody); err != nil {
return "", fmt.Errorf("failed to unmarshal request body: %w", err)
}
unparsedDiscriminator, ok := parsedBody["authType"]
if !ok {
return "", fmt.Errorf("authType field is missing in the request body")
}
discriminator := unparsedDiscriminator.(string)
authType := sqlcv1.V1IncomingWebhookAuthType(discriminator)
if authType == "" {
return "", fmt.Errorf("invalid auth type: %s", unparsedDiscriminator)
}
return authType, nil
}
func (w *V1WebhooksService) constructCreateOpts(tenantId string, request gen.V1CreateWebhookRequest) (v1.CreateWebhookOpts, error) {
params := v1.CreateWebhookOpts{
Tenantid: sqlchelpers.UUIDFromStr(tenantId),
}
discriminator, err := extractAuthType(request)
if err != nil {
return params, fmt.Errorf("failed to get discriminator: %w", err)
}
authConfig := v1.AuthConfig{}
switch discriminator {
case sqlcv1.V1IncomingWebhookAuthTypeBASIC:
basicAuth, err := request.AsV1CreateWebhookRequestBasicAuth()
if err != nil {
return params, fmt.Errorf("failed to parse basic auth: %w", err)
}
authConfig.Type = sqlcv1.V1IncomingWebhookAuthTypeBASIC
passwordEncrypted, err := w.config.Encryption.Encrypt([]byte(basicAuth.Auth.Password), "v1_webhook_basic_auth_password")
if err != nil {
return params, fmt.Errorf("failed to encrypt basic auth password: %s", err.Error())
}
authConfig.BasicAuth = &v1.BasicAuthCredentials{
Username: basicAuth.Auth.Username,
EncryptedPassword: passwordEncrypted,
}
params.Sourcename = sqlcv1.V1IncomingWebhookSourceName(basicAuth.SourceName)
params.Name = basicAuth.Name
params.Eventkeyexpression = basicAuth.EventKeyExpression
params.AuthConfig = authConfig
case sqlcv1.V1IncomingWebhookAuthTypeAPIKEY:
apiKeyAuth, err := request.AsV1CreateWebhookRequestAPIKey()
if err != nil {
return params, fmt.Errorf("failed to parse api key auth: %w", err)
}
authConfig.Type = sqlcv1.V1IncomingWebhookAuthTypeAPIKEY
authConfig := v1.AuthConfig{
Type: sqlcv1.V1IncomingWebhookAuthTypeAPIKEY,
}
apiKeyEncrypted, err := w.config.Encryption.Encrypt([]byte(apiKeyAuth.Auth.ApiKey), "v1_webhook_api_key")
if err != nil {
return params, fmt.Errorf("failed to encrypt api key: %s", err.Error())
}
authConfig.APIKeyAuth = &v1.APIKeyAuthCredentials{
HeaderName: apiKeyAuth.Auth.HeaderName,
EncryptedKey: apiKeyEncrypted,
}
params.Sourcename = sqlcv1.V1IncomingWebhookSourceName(apiKeyAuth.SourceName)
params.Name = apiKeyAuth.Name
params.Eventkeyexpression = apiKeyAuth.EventKeyExpression
params.AuthConfig = authConfig
case sqlcv1.V1IncomingWebhookAuthTypeHMAC:
hmacAuth, err := request.AsV1CreateWebhookRequestHMAC()
if err != nil {
return params, fmt.Errorf("failed to parse hmac auth: %w", err)
}
authConfig := v1.AuthConfig{
Type: sqlcv1.V1IncomingWebhookAuthTypeHMAC,
}
signingSecretEncrypted, err := w.config.Encryption.Encrypt([]byte(hmacAuth.Auth.SigningSecret), "v1_webhook_hmac_signing_secret")
if err != nil {
return params, fmt.Errorf("failed to encrypt api key: %s", err.Error())
}
authConfig.HMACAuth = &v1.HMACAuthCredentials{
Algorithm: sqlcv1.V1IncomingWebhookHmacAlgorithm(hmacAuth.Auth.Algorithm),
Encoding: sqlcv1.V1IncomingWebhookHmacEncoding(hmacAuth.Auth.Encoding),
SignatureHeaderName: hmacAuth.Auth.SignatureHeaderName,
EncryptedWebhookSigningSecret: signingSecretEncrypted,
}
params.Sourcename = sqlcv1.V1IncomingWebhookSourceName(hmacAuth.SourceName)
params.Name = hmacAuth.Name
params.Eventkeyexpression = hmacAuth.EventKeyExpression
params.AuthConfig = authConfig
default:
return params, fmt.Errorf("unsupported auth type: %s", discriminator)
}
return params, nil
}
api/v1/server/handlers/v1/webhooks/delete.go
+29
View File
@@ -0,0 +1,29 @@
package webhooksv1
import (
"github.com/hatchet-dev/hatchet/api/v1/server/oas/apierrors"
"github.com/hatchet-dev/hatchet/api/v1/server/oas/gen"
"github.com/hatchet-dev/hatchet/api/v1/server/oas/transformers/v1"
"github.com/hatchet-dev/hatchet/pkg/repository/v1/sqlcv1"
"github.com/labstack/echo/v4"
)
func (w *V1WebhooksService) V1WebhookDelete(ctx echo.Context, request gen.V1WebhookDeleteRequestObject) (gen.V1WebhookDeleteResponseObject, error) {
webhook := ctx.Get("v1-webhook").(*sqlcv1.V1IncomingWebhook)
webhook, err := w.config.V1.Webhooks().DeleteWebhook(
ctx.Request().Context(),
webhook.TenantID.String(),
webhook.Name,
)
if err != nil {
return gen.V1WebhookDelete400JSONResponse(apierrors.NewAPIErrors("failed to delete webhook")), nil
}
transformed := transformers.ToV1Webhook(webhook)
return gen.V1WebhookDelete200JSONResponse(
transformed,
), nil
}
api/v1/server/handlers/v1/webhooks/get.go
+18
View File
@@ -0,0 +1,18 @@
package webhooksv1
import (
"github.com/hatchet-dev/hatchet/api/v1/server/oas/gen"
"github.com/hatchet-dev/hatchet/api/v1/server/oas/transformers/v1"
"github.com/hatchet-dev/hatchet/pkg/repository/v1/sqlcv1"
"github.com/labstack/echo/v4"
)
func (w *V1WebhooksService) V1WebhookGet(ctx echo.Context, request gen.V1WebhookGetRequestObject) (gen.V1WebhookGetResponseObject, error) {
webhook := ctx.Get("v1-webhook").(*sqlcv1.V1IncomingWebhook)
transformed := transformers.ToV1Webhook(webhook)
return gen.V1WebhookGet200JSONResponse(
transformed,
), nil
}
api/v1/server/handlers/v1/webhooks/list.go
+49
View File
@@ -0,0 +1,49 @@
package webhooksv1
import (
"github.com/hatchet-dev/hatchet/api/v1/server/oas/apierrors"
"github.com/hatchet-dev/hatchet/api/v1/server/oas/gen"
"github.com/hatchet-dev/hatchet/api/v1/server/oas/transformers/v1"
"github.com/hatchet-dev/hatchet/pkg/repository/postgres/dbsqlc"
v1 "github.com/hatchet-dev/hatchet/pkg/repository/v1"
"github.com/hatchet-dev/hatchet/pkg/repository/v1/sqlcv1"
"github.com/labstack/echo/v4"
)
func (w *V1WebhooksService) V1WebhookList(ctx echo.Context, request gen.V1WebhookListRequestObject) (gen.V1WebhookListResponseObject, error) {
tenant := ctx.Get("tenant").(*dbsqlc.Tenant)
var sourceNames []sqlcv1.V1IncomingWebhookSourceName
var webhookNames []string
if request.Params.SourceNames != nil {
for _, sourceName := range *request.Params.SourceNames {
sourceNames = append(sourceNames, sqlcv1.V1IncomingWebhookSourceName(sourceName))
}
}
if request.Params.WebhookNames != nil {
webhookNames = *request.Params.WebhookNames
}
webhooks, err := w.config.V1.Webhooks().ListWebhooks(
ctx.Request().Context(),
tenant.ID.String(),
v1.ListWebhooksOpts{
WebhookNames: webhookNames,
WebhookSourceNames: sourceNames,
Limit: request.Params.Limit,
Offset: request.Params.Offset,
},
)
if err != nil {
return gen.V1WebhookList400JSONResponse(apierrors.NewAPIErrors("failed to list webhooks")), nil
}
transformed := transformers.ToV1WebhookList(webhooks)
return gen.V1WebhookList200JSONResponse(
transformed,
), nil
}
api/v1/server/handlers/v1/webhooks/receive.go
+459
View File
@@ -0,0 +1,459 @@
package webhooksv1
import (
"crypto/hmac"
"crypto/md5"
"crypto/sha1"
"crypto/sha256"
"crypto/sha512"
"encoding/base64"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"hash"
"io"
"net/http"
"strconv"
"strings"
"time"
"github.com/hatchet-dev/hatchet/api/v1/server/oas/gen"
"github.com/hatchet-dev/hatchet/internal/cel"
"github.com/hatchet-dev/hatchet/pkg/repository/v1/sqlcv1"
"github.com/labstack/echo/v4"
)
func (w *V1WebhooksService) V1WebhookReceive(ctx echo.Context, request gen.V1WebhookReceiveRequestObject) (gen.V1WebhookReceiveResponseObject, error) {
tenantId := request.Tenant.String()
webhookName := request.V1Webhook
tenant, err := w.config.APIRepository.Tenant().GetTenantByID(ctx.Request().Context(), tenantId)
if err != nil || tenant == nil {
return gen.V1WebhookReceive400JSONResponse{
Errors: []gen.APIError{
{
Description: "tenant not found",
},
},
}, nil
}
webhook, err := w.config.V1.Webhooks().GetWebhook(ctx.Request().Context(), tenantId, webhookName)
if err != nil || webhook == nil {
return gen.V1WebhookReceive400JSONResponse{
Errors: []gen.APIError{
{
Description: fmt.Sprintf("webhook %s not found for tenant %s", webhookName, tenantId),
},
},
}, nil
}
if webhook.TenantID.String() != tenantId {
return gen.V1WebhookReceive403JSONResponse{
Errors: []gen.APIError{
{
Description: fmt.Sprintf("webhook %s does not belong to tenant %s", webhookName, tenantId),
},
},
}, nil
}
rawBody, err := io.ReadAll(ctx.Request().Body)
if err != nil {
return nil, fmt.Errorf("failed to read request body: %w", err)
}
ok, validationError := w.validateWebhook(rawBody, *webhook, *ctx.Request())
if !ok {
err := w.config.Ingestor.IngestWebhookValidationFailure(
ctx.Request().Context(),
tenant,
webhook.Name,
validationError.ErrorText,
)
if err != nil {
return nil, fmt.Errorf("failed to ingest webhook validation failure: %w", err)
}
return validationError.ToResponse()
}
payloadMap := make(map[string]interface{})
if rawBody != nil {
err := json.Unmarshal(rawBody, &payloadMap)
if err != nil {
return gen.V1WebhookReceive400JSONResponse{
Errors: []gen.APIError{
{
Description: fmt.Sprintf("failed to unmarshal request body: %v", err),
},
},
}, nil
}
// This could cause unexpected behavior if the payload contains a key named "tenant" or "v1-webhook"
delete(payloadMap, "tenant")
delete(payloadMap, "v1-webhook")
}
eventKey, err := w.celParser.EvaluateIncomingWebhookExpression(webhook.EventKeyExpression, cel.NewInput(
cel.WithInput(payloadMap),
),
)
if err != nil {
if eventKey == "" {
err = fmt.Errorf("event key evaluted to an empty string")
}
ingestionErr := w.config.Ingestor.IngestCELEvaluationFailure(
ctx.Request().Context(),
tenant.ID.String(),
err.Error(),
sqlcv1.V1CelEvaluationFailureSourceWEBHOOK,
)
if ingestionErr != nil {
return nil, fmt.Errorf("failed to ingest webhook validation failure: %w", ingestionErr)
}
return gen.V1WebhookReceive400JSONResponse{
Errors: []gen.APIError{
{
Description: fmt.Sprintf("failed to evaluate event key expression: %v", err),
},
},
}, nil
}
payload, err := json.Marshal(payloadMap)
if err != nil {
return gen.V1WebhookReceive400JSONResponse{
Errors: []gen.APIError{
{
Description: fmt.Sprintf("failed to marshal request body: %v", err),
},
},
}, nil
}
_, err = w.config.Ingestor.IngestEvent(
ctx.Request().Context(),
tenant,
eventKey,
payload,
nil,
nil,
nil,
&webhook.Name,
)
if err != nil {
return nil, fmt.Errorf("failed to ingest event")
}
msg := "ok"
return gen.V1WebhookReceive200JSONResponse(
gen.V1WebhookReceive200JSONResponse{
Message: &msg,
},
), nil
}
func computeHMACSignature(payload []byte, secret []byte, algorithm sqlcv1.V1IncomingWebhookHmacAlgorithm, encoding sqlcv1.V1IncomingWebhookHmacEncoding) (string, error) {
var hashFunc func() hash.Hash
switch algorithm {
case sqlcv1.V1IncomingWebhookHmacAlgorithmSHA1:
hashFunc = sha1.New
case sqlcv1.V1IncomingWebhookHmacAlgorithmSHA256:
hashFunc = sha256.New
case sqlcv1.V1IncomingWebhookHmacAlgorithmSHA512:
hashFunc = sha512.New
case sqlcv1.V1IncomingWebhookHmacAlgorithmMD5:
hashFunc = md5.New
default:
return "", fmt.Errorf("unsupported HMAC algorithm: %s", algorithm)
}
h := hmac.New(hashFunc, secret)
h.Write(payload)
signature := h.Sum(nil)
switch encoding {
case sqlcv1.V1IncomingWebhookHmacEncodingHEX:
return hex.EncodeToString(signature), nil
case sqlcv1.V1IncomingWebhookHmacEncodingBASE64:
return base64.StdEncoding.EncodeToString(signature), nil
case sqlcv1.V1IncomingWebhookHmacEncodingBASE64URL:
return base64.URLEncoding.EncodeToString(signature), nil
default:
return "", fmt.Errorf("unsupported HMAC encoding: %s", encoding)
}
}
type HttpResponseCode int
const (
Http400 HttpResponseCode = iota
Http403
Http500
)
type ValidationError struct {
Code HttpResponseCode
ErrorText string
}
func (vr ValidationError) ToResponse() (gen.V1WebhookReceiveResponseObject, error) {
switch vr.Code {
case Http400:
return gen.V1WebhookReceive400JSONResponse{
Errors: []gen.APIError{
{
Description: vr.ErrorText,
},
},
}, nil
case Http403:
return gen.V1WebhookReceive403JSONResponse{
Errors: []gen.APIError{
{
Description: vr.ErrorText,
},
},
}, nil
case Http500:
return nil, errors.New(vr.ErrorText)
default:
return nil, fmt.Errorf("no validation error set")
}
}
func (w *V1WebhooksService) validateWebhook(webhookPayload []byte, webhook sqlcv1.V1IncomingWebhook, request http.Request) (
bool,
*ValidationError,
) {
switch webhook.SourceName {
case sqlcv1.V1IncomingWebhookSourceNameSTRIPE:
signatureHeader := request.Header.Get(webhook.AuthHmacSignatureHeaderName.String)
if signatureHeader == "" {
return false, &ValidationError{
Code: Http400,
ErrorText: fmt.Sprintf("missing or invalid signature header: %s", webhook.AuthHmacSignatureHeaderName.String),
}
}
splitHeader := strings.Split(signatureHeader, ",")
headersMap := make(map[string]string)
for _, header := range splitHeader {
parts := strings.Split(header, "=")
if len(parts) != 2 {
return false, &ValidationError{
Code: Http400,
ErrorText: fmt.Sprintf("invalid signature header format: %s", webhook.AuthHmacSignatureHeaderName.String),
}
}
headersMap[strings.TrimSpace(parts[0])] = strings.TrimSpace(parts[1])
}
timestampHeader, hasTimestampHeader := headersMap["t"]
v1SignatureHeader, hasV1SignatureHeader := headersMap["v1"]
if timestampHeader == "" || v1SignatureHeader == "" || !hasTimestampHeader || !hasV1SignatureHeader {
return false, &ValidationError{
Code: Http400,
ErrorText: fmt.Sprintf("missing or invalid signature header: %s", webhook.AuthHmacSignatureHeaderName.String),
}
}
timestamp := strings.TrimPrefix(timestampHeader, "t=")
signature := strings.TrimPrefix(v1SignatureHeader, "v1=")
if timestamp == "" || signature == "" {
return false, &ValidationError{
Code: Http400,
ErrorText: fmt.Sprintf("missing or invalid signature header: %s", webhook.AuthHmacSignatureHeaderName.String),
}
}
parsedTimestamp, err := strconv.ParseInt(timestamp, 10, 64)
if err != nil {
return false, &ValidationError{
Code: Http400,
ErrorText: fmt.Sprintf("invalid timestamp in signature header: %s", err),
}
}
if time.Unix(parsedTimestamp, 0).UTC().Before(time.Now().Add(-10 * time.Minute)) {
return false, &ValidationError{
Code: Http400,
ErrorText: "timestamp in signature header is out of range",
}
}
decryptedSigningSecret, err := w.config.Encryption.Decrypt(webhook.AuthHmacWebhookSigningSecret, "v1_webhook_hmac_signing_secret")
if err != nil {
return false, &ValidationError{
Code: Http500,
ErrorText: fmt.Sprintf("failed to decrypt HMAC signing secret: %s", err),
}
}
algorithm := webhook.AuthHmacAlgorithm.V1IncomingWebhookHmacAlgorithm
encoding := webhook.AuthHmacEncoding.V1IncomingWebhookHmacEncoding
signedPayload := fmt.Sprintf("%s.%s", timestamp, webhookPayload)
expectedSignature, err := computeHMACSignature([]byte(signedPayload), decryptedSigningSecret, algorithm, encoding)
if err != nil {
return false, &ValidationError{
Code: Http403,
ErrorText: fmt.Sprintf("failed to compute HMAC signature: %s", err),
}
}
if !signaturesMatch(signature, expectedSignature) {
return false, &ValidationError{
Code: Http403,
ErrorText: "invalid HMAC signature",
}
}
case sqlcv1.V1IncomingWebhookSourceNameGITHUB:
fallthrough
case sqlcv1.V1IncomingWebhookSourceNameGENERIC:
switch webhook.AuthMethod {
case sqlcv1.V1IncomingWebhookAuthTypeBASIC:
username, password, ok := request.BasicAuth()
if !ok {
return false, &ValidationError{
Code: Http403,
ErrorText: "missing or invalid authorization header",
}
}
decryptedPassword, err := w.config.Encryption.Decrypt(webhook.AuthBasicPassword, "v1_webhook_basic_auth_password")
if err != nil {
return false, &ValidationError{
Code: Http500,
ErrorText: fmt.Sprintf("failed to decrypt basic auth password: %s", err),
}
}
if username != webhook.AuthBasicUsername.String || password != string(decryptedPassword) {
return false, &ValidationError{
Code: Http403,
ErrorText: "invalid basic auth credentials",
}
}
case sqlcv1.V1IncomingWebhookAuthTypeAPIKEY:
apiKey := request.Header.Get(webhook.AuthApiKeyHeaderName.String)
if apiKey == "" {
return false, &ValidationError{
Code: Http403,
ErrorText: fmt.Sprintf("missing or invalid api key header: %s", webhook.AuthApiKeyHeaderName.String),
}
}
decryptedApiKey, err := w.config.Encryption.Decrypt(webhook.AuthApiKeyKey, "v1_webhook_api_key")
if err != nil {
return false, &ValidationError{
Code: Http500,
ErrorText: fmt.Sprintf("failed to decrypt api key: %s", err),
}
}
if apiKey != string(decryptedApiKey) {
return false, &ValidationError{
Code: Http403,
ErrorText: fmt.Sprintf("invalid api key: %s", webhook.AuthApiKeyHeaderName.String),
}
}
case sqlcv1.V1IncomingWebhookAuthTypeHMAC:
signature := request.Header.Get(webhook.AuthHmacSignatureHeaderName.String)
if signature == "" {
return false, &ValidationError{
Code: Http403,
ErrorText: fmt.Sprintf("missing or invalid signature header: %s", webhook.AuthHmacSignatureHeaderName.String),
}
}
decryptedSigningSecret, err := w.config.Encryption.Decrypt(webhook.AuthHmacWebhookSigningSecret, "v1_webhook_hmac_signing_secret")
if err != nil {
return false, &ValidationError{
Code: Http500,
ErrorText: fmt.Sprintf("failed to decrypt HMAC signing secret: %s", err),
}
}
algorithm := webhook.AuthHmacAlgorithm.V1IncomingWebhookHmacAlgorithm
encoding := webhook.AuthHmacEncoding.V1IncomingWebhookHmacEncoding
expectedSignature, err := computeHMACSignature(webhookPayload, decryptedSigningSecret, algorithm, encoding)
if err != nil {
return false, &ValidationError{
Code: Http500,
ErrorText: fmt.Sprintf("failed to compute HMAC signature: %s", err),
}
}
if !signaturesMatch(signature, expectedSignature) {
return false, &ValidationError{
Code: Http403,
ErrorText: "invalid HMAC signature",
}
}
default:
return false, &ValidationError{
Code: Http400,
ErrorText: fmt.Sprintf("unsupported auth type: %s", webhook.AuthMethod),
}
}
default:
return false, &ValidationError{
Code: Http400,
ErrorText: fmt.Sprintf("unsupported source name: %+v", webhook.SourceName),
}
}
return true, nil
}
func signaturesMatch(providedSignature, expectedSignature string) bool {
providedSignature = strings.TrimSpace(providedSignature)
expectedSignature = strings.TrimSpace(expectedSignature)
return hmac.Equal(
[]byte(removePrefixesFromSignature(providedSignature)),
[]byte(removePrefixesFromSignature(expectedSignature)),
)
}
func removePrefixesFromSignature(signature string) string {
signature = strings.TrimPrefix(signature, "sha1=")
signature = strings.TrimPrefix(signature, "sha256=")
signature = strings.TrimPrefix(signature, "sha512=")
signature = strings.TrimPrefix(signature, "md5=")
return signature
}
api/v1/server/handlers/v1/webhooks/service.go
+18
View File
@@ -0,0 +1,18 @@
package webhooksv1
import (
"github.com/hatchet-dev/hatchet/internal/cel"
"github.com/hatchet-dev/hatchet/pkg/config/server"
)
type V1WebhooksService struct {
config *server.ServerConfig
celParser *cel.CELParser
}
func NewV1WebhooksService(config *server.ServerConfig) *V1WebhooksService {
return &V1WebhooksService{
config: config,
celParser: cel.NewCELParser(),
}
}
api/v1/server/middleware/populator/populator_test.go
+41
View File
@@ -46,6 +46,15 @@ func topLevelResourceGetter(config *server.ServerConfig, parentId, id string) (i
}, "", nil
}
func invalidParentGetter(config *server.ServerConfig, parentId, id string) (interface{}, string, error) {
newUuid := uuid.NewString()
return &oneToManyResource{
ID: id,
ParentID: newUuid,
}, newUuid, nil
}
func TestPopulatorMiddleware(t *testing.T) {
e := echo.New()
req := httptest.NewRequest(http.MethodGet, "/", nil)
@@ -78,3 +87,35 @@ func TestPopulatorMiddleware(t *testing.T) {
assert.NotNil(t, c.Get("resource1"))
assert.NotNil(t, c.Get("resource2"))
}
func TestPopulatorMiddlewareParentDisagreement(t *testing.T) {
e := echo.New()
req := httptest.NewRequest(http.MethodGet, "/", nil)
rec := httptest.NewRecorder()
c := e.NewContext(req, rec)
// Mock RouteInfo
routeInfo := &middleware.RouteInfo{
Resources: []string{"resource1", "resource2"},
}
resource1Id := uuid.New().String()
resource2Id := uuid.New().String()
// Setting params for the context - NOTE: we need to set both params at once
c.SetParamNames("resource1", "resource2")
c.SetParamValues(resource1Id, resource2Id)
// Creating Populator with mock getter function
populator := NewPopulator(&server.ServerConfig{})
populator.RegisterGetter("resource1", topLevelResourceGetter)
populator.RegisterGetter("resource2", invalidParentGetter)
// Using the Populator middleware
middlewareFunc := populator.Middleware(routeInfo)
err := middlewareFunc(c)
// Assertions
assert.ErrorContains(t, err, "could not be populated")
}
api/v1/server/middleware/webhook_rate_limit.go
+43
View File
@@ -0,0 +1,43 @@
package middleware
import (
"fmt"
"net/http"
"time"
"github.com/labstack/echo/v4"
"github.com/labstack/echo/v4/middleware"
"github.com/rs/zerolog"
"golang.org/x/time/rate"
)
func WebhookRateLimitMiddleware(rateLimit rate.Limit, burst int, l *zerolog.Logger) echo.MiddlewareFunc {
config := middleware.RateLimiterConfig{
Skipper: func(c echo.Context) bool {
method := c.Request().Method
if method != http.MethodPost {
return true
}
tenantId := c.Param("tenant")
webhookName := c.Param("v1-webhook")
if tenantId == "" || webhookName == "" {
return true
}
return c.Request().URL.Path != fmt.Sprintf("/api/v1/stable/tenants/%s/webhooks/%s", tenantId, webhookName)
},
Store: middleware.NewRateLimiterMemoryStoreWithConfig(
middleware.RateLimiterMemoryStoreConfig{
Rate: rateLimit,
Burst: burst,
ExpiresIn: 10 * time.Minute,
},
),
}
return middleware.RateLimiterWithConfig(config)
}
api/v1/server/oas/gen/openapi.gen.go
+1076 -276
View File
File diff suppressed because it is too large Load Diff
api/v1/server/oas/transformers/v1/events.go
+5 -4
View File
@@ -92,10 +92,11 @@ func ToV1EventList(events []*v1.ListEventsRow, limit, offset, total int64) gen.V
Failed: row.FailedCount,
Running: row.RunningCount,
},
Payload: &payload,
SeenAt: &row.EventSeenAt.Time,
Scope: &row.EventScope,
TriggeredRuns: &triggeredRuns,
Payload: &payload,
SeenAt: &row.EventSeenAt.Time,
Scope: &row.EventScope,
TriggeredRuns: &triggeredRuns,
TriggeringWebhookName: row.TriggeringWebhookName,
}
}
api/v1/server/oas/transformers/v1/webhooks.go
+37
View File
@@ -0,0 +1,37 @@
package transformers
import (
"github.com/google/uuid"
"github.com/hatchet-dev/hatchet/api/v1/server/oas/gen"
"github.com/hatchet-dev/hatchet/pkg/repository/v1/sqlcv1"
)
func ToV1Webhook(webhook *sqlcv1.V1IncomingWebhook) gen.V1Webhook {
// Intentionally empty uuid
var id uuid.UUID
return gen.V1Webhook{
AuthType: gen.V1WebhookAuthType(webhook.AuthMethod),
Metadata: gen.APIResourceMeta{
CreatedAt: webhook.InsertedAt.Time,
UpdatedAt: webhook.UpdatedAt.Time,
Id: id.String(),
},
TenantId: webhook.TenantID.String(),
EventKeyExpression: webhook.EventKeyExpression,
Name: webhook.Name,
SourceName: gen.V1WebhookSourceName(webhook.SourceName),
}
}
func ToV1WebhookList(webhooks []*sqlcv1.V1IncomingWebhook) gen.V1WebhookList {
rows := make([]gen.V1Webhook, len(webhooks))
for i, webhook := range webhooks {
rows[i] = ToV1Webhook(webhook)
}
return gen.V1WebhookList{
Rows: &rows,
}
}
api/v1/server/run/run.go
+23
View File
@@ -30,6 +30,7 @@ import (
eventsv1 "github.com/hatchet-dev/hatchet/api/v1/server/handlers/v1/events"
filtersv1 "github.com/hatchet-dev/hatchet/api/v1/server/handlers/v1/filters"
"github.com/hatchet-dev/hatchet/api/v1/server/handlers/v1/tasks"
webhooksv1 "github.com/hatchet-dev/hatchet/api/v1/server/handlers/v1/webhooks"
workflowrunsv1 "github.com/hatchet-dev/hatchet/api/v1/server/handlers/v1/workflow-runs"
webhookworker "github.com/hatchet-dev/hatchet/api/v1/server/handlers/webhook-worker"
"github.com/hatchet-dev/hatchet/api/v1/server/handlers/workers"
@@ -41,6 +42,7 @@ import (
"github.com/hatchet-dev/hatchet/api/v1/server/oas/gen"
"github.com/hatchet-dev/hatchet/pkg/config/server"
"github.com/hatchet-dev/hatchet/pkg/repository/postgres/sqlchelpers"
"golang.org/x/time/rate"
)
type apiService struct {
@@ -64,6 +66,7 @@ type apiService struct {
*workflowrunsv1.V1WorkflowRunsService
*eventsv1.V1EventsService
*filtersv1.V1FiltersService
*webhooksv1.V1WebhooksService
*celv1.V1CELService
}
@@ -89,6 +92,7 @@ func newAPIService(config *server.ServerConfig) *apiService {
V1WorkflowRunsService: workflowrunsv1.NewV1WorkflowRunsService(config),
V1EventsService: eventsv1.NewV1EventsService(config),
V1FiltersService: filtersv1.NewV1FiltersService(config),
V1WebhooksService: webhooksv1.NewV1WebhooksService(config),
V1CELService: celv1.NewV1CELService(config),
}
}
@@ -417,6 +421,20 @@ func (t *APIServer) registerSpec(g *echo.Group, spec *openapi3.T, middlewares []
return filter, sqlchelpers.UUIDToStr(filter.TenantID), nil
})
populatorMW.RegisterGetter("v1-webhook", func(config *server.ServerConfig, parentId, id string) (result interface{}, uniqueParentId string, err error) {
webhook, err := t.config.V1.Webhooks().GetWebhook(
context.Background(),
parentId,
id,
)
if err != nil {
return nil, "", err
}
return webhook, sqlchelpers.UUIDToStr(webhook.TenantID), nil
})
authnMW := authn.NewAuthN(t.config)
authzMW := authz.NewAuthZ(t.config)
@@ -488,6 +506,11 @@ func (t *APIServer) registerSpec(g *echo.Group, spec *openapi3.T, middlewares []
loggerMiddleware,
middleware.Recover(),
allHatchetMiddleware,
hatchetmiddleware.WebhookRateLimitMiddleware(
rate.Limit(t.config.Runtime.WebhookRateLimit),
t.config.Runtime.WebhookRateLimitBurst,
t.config.Logger,
),
)
return populatorMW, nil
cmd/hatchet-migrate/migrate/migrations/20250725183217_v1_0_30.sql
+25
View File
@@ -0,0 +1,25 @@
-- +goose Up
-- +goose StatementBegin
CREATE TYPE v1_cel_evaluation_failure_source AS ENUM ('FILTER', 'WEBHOOK');
CREATE TABLE v1_cel_evaluation_failures_olap (
id BIGINT NOT NULL GENERATED ALWAYS AS IDENTITY,
tenant_id UUID NOT NULL,
source v1_cel_evaluation_failure_source NOT NULL,
error TEXT NOT NULL,
inserted_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (inserted_at, id)
) PARTITION BY RANGE(inserted_at);
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
DROP TABLE v1_cel_evaluation_failures_olap;
DROP TYPE v1_cel_evaluation_failure_source;
-- +goose StatementEnd
cmd/hatchet-migrate/migrate/migrations/20250726193737_v1_0_31.sql
+76
View File
@@ -0,0 +1,76 @@
-- +goose Up
-- +goose StatementBegin
CREATE TYPE v1_incoming_webhook_auth_type AS ENUM ('BASIC', 'API_KEY', 'HMAC');
CREATE TYPE v1_incoming_webhook_hmac_algorithm AS ENUM ('SHA1', 'SHA256', 'SHA512', 'MD5');
CREATE TYPE v1_incoming_webhook_hmac_encoding AS ENUM ('HEX', 'BASE64', 'BASE64URL');
-- Can add more sources in the future
CREATE TYPE v1_incoming_webhook_source_name AS ENUM ('GENERIC', 'GITHUB', 'STRIPE');
CREATE TABLE v1_incoming_webhook (
tenant_id UUID NOT NULL,
name TEXT NOT NULL,
source_name v1_incoming_webhook_source_name NOT NULL,
-- CEL expression that creates an event key
-- from the payload of the webhook
event_key_expression TEXT NOT NULL,
auth_method v1_incoming_webhook_auth_type NOT NULL,
auth__basic__username TEXT,
auth__basic__password BYTEA,
auth__api_key__header_name TEXT,
auth__api_key__key BYTEA,
auth__hmac__algorithm v1_incoming_webhook_hmac_algorithm,
auth__hmac__encoding v1_incoming_webhook_hmac_encoding,
auth__hmac__signature_header_name TEXT,
auth__hmac__webhook_signing_secret BYTEA,
inserted_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (tenant_id, name),
CHECK (
(
auth_method = 'BASIC'
AND (
auth__basic__username IS NOT NULL
AND auth__basic__password IS NOT NULL
)
)
OR
(
auth_method = 'API_KEY'
AND (
auth__api_key__header_name IS NOT NULL
AND auth__api_key__key IS NOT NULL
)
)
OR
(
auth_method = 'HMAC'
AND (
auth__hmac__algorithm IS NOT NULL
AND auth__hmac__encoding IS NOT NULL
AND auth__hmac__signature_header_name IS NOT NULL
AND auth__hmac__webhook_signing_secret IS NOT NULL
)
)
),
CHECK (LENGTH(event_key_expression) > 0),
CHECK (LENGTH(name) > 0)
);
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
DROP TABLE v1_incoming_webhook;
DROP TYPE v1_incoming_webhook_auth_type;
DROP TYPE v1_incoming_webhook_hmac_algorithm;
DROP TYPE v1_incoming_webhook_hmac_encoding;
DROP TYPE v1_incoming_webhook_source_name;
-- +goose StatementEnd
cmd/hatchet-migrate/migrate/migrations/20250726214427_v1_0_32.sql
+31
View File
@@ -0,0 +1,31 @@
-- +goose Up
-- +goose StatementBegin
ALTER TABLE v1_events_olap ADD COLUMN triggering_webhook_name TEXT;
CREATE TABLE v1_incoming_webhook_validation_failures_olap (
id BIGINT NOT NULL GENERATED ALWAYS AS IDENTITY,
tenant_id UUID NOT NULL,
-- webhook names are tenant-unique
incoming_webhook_name TEXT NOT NULL,
error TEXT NOT NULL,
inserted_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (inserted_at, id)
) PARTITION BY RANGE(inserted_at);
CREATE INDEX v1_incoming_webhook_validation_failures_olap_tenant_id_incoming_webhook_name_idx ON v1_incoming_webhook_validation_failures_olap (tenant_id, incoming_webhook_name);
SELECT create_v1_range_partition('v1_incoming_webhook_validation_failures_olap', NOW()::DATE);
SELECT create_v1_range_partition('v1_incoming_webhook_validation_failures_olap', (NOW() + INTERVAL '1 day')::DATE);
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
ALTER TABLE v1_events_olap DROP COLUMN triggering_webhook_name;
DROP TABLE v1_incoming_webhook_validation_failures_olap;
-- +goose StatementEnd
cmd/hatchet-migrate/migrate/migrations/20250726225025_v1_0_33.sql
+8
View File
@@ -0,0 +1,8 @@
-- +goose Up
-- +goose StatementBegin
ALTER TYPE "LimitResource" ADD VALUE IF NOT EXISTS 'INCOMING_WEBHOOK';
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
-- +goose StatementEnd
examples/go/bulk-operations/main.go
-4
View File
@@ -35,7 +35,6 @@ func main() {
selectedWorkflow := (*workflows.Rows)[0]
selectedWorkflowUUID := uuid.MustParse(selectedWorkflow.Metadata.Id)
// > List runs
workflowRuns, err := hatchet.Runs().List(ctx, rest.V1WorkflowRunListParams{
WorkflowIds: &[]types.UUID{selectedWorkflowUUID},
@@ -50,7 +49,6 @@ func main() {
runIds = append(runIds, uuid.MustParse(run.Metadata.Id))
}
// > Cancel by run ids
_, err = hatchet.Runs().Cancel(ctx, rest.V1CancelTaskRequest{
ExternalIds: &runIds,
@@ -59,7 +57,6 @@ func main() {
log.Fatalf("failed to cancel runs by ids: %v", err)
}
// > Cancel by filters
tNow := time.Now().UTC()
@@ -76,6 +73,5 @@ func main() {
log.Fatalf("failed to cancel runs by filters: %v", err)
}
fmt.Println("cancelled all runs for workflow", selectedWorkflow.Name)
}
examples/python/webhooks/test_webhooks.py
+643
View File
@@ -0,0 +1,643 @@
import asyncio
import base64
import hashlib
import hmac
import json
from collections.abc import AsyncGenerator
from contextlib import asynccontextmanager
from datetime import datetime, timezone
from typing import Any
from uuid import uuid4
import aiohttp
import pytest
from examples.webhooks.worker import WebhookInput
from hatchet_sdk import Hatchet
from hatchet_sdk.clients.rest.api.webhook_api import WebhookApi
from hatchet_sdk.clients.rest.models.v1_create_webhook_request import (
V1CreateWebhookRequest,
)
from hatchet_sdk.clients.rest.models.v1_create_webhook_request_api_key import (
V1CreateWebhookRequestAPIKey,
)
from hatchet_sdk.clients.rest.models.v1_create_webhook_request_basic_auth import (
V1CreateWebhookRequestBasicAuth,
)
from hatchet_sdk.clients.rest.models.v1_create_webhook_request_hmac import (
V1CreateWebhookRequestHMAC,
)
from hatchet_sdk.clients.rest.models.v1_event import V1Event
from hatchet_sdk.clients.rest.models.v1_task_status import V1TaskStatus
from hatchet_sdk.clients.rest.models.v1_task_summary import V1TaskSummary
from hatchet_sdk.clients.rest.models.v1_webhook import V1Webhook
from hatchet_sdk.clients.rest.models.v1_webhook_api_key_auth import V1WebhookAPIKeyAuth
from hatchet_sdk.clients.rest.models.v1_webhook_basic_auth import V1WebhookBasicAuth
from hatchet_sdk.clients.rest.models.v1_webhook_hmac_algorithm import (
V1WebhookHMACAlgorithm,
)
from hatchet_sdk.clients.rest.models.v1_webhook_hmac_auth import V1WebhookHMACAuth
from hatchet_sdk.clients.rest.models.v1_webhook_hmac_encoding import (
V1WebhookHMACEncoding,
)
from hatchet_sdk.clients.rest.models.v1_webhook_source_name import V1WebhookSourceName
TEST_BASIC_USERNAME = "test_user"
TEST_BASIC_PASSWORD = "test_password"
TEST_API_KEY_HEADER = "X-API-Key"
TEST_API_KEY_VALUE = "test_api_key_123"
TEST_HMAC_SIGNATURE_HEADER = "X-Signature"
TEST_HMAC_SECRET = "test_hmac_secret"
@pytest.fixture
def webhook_body() -> WebhookInput:
return WebhookInput(type="test", message="Hello, world!")
@pytest.fixture
def test_run_id() -> str:
return str(uuid4())
@pytest.fixture
def test_start() -> datetime:
return datetime.now(timezone.utc)
def create_hmac_signature(
payload: bytes,
secret: str,
algorithm: V1WebhookHMACAlgorithm = V1WebhookHMACAlgorithm.SHA256,
encoding: V1WebhookHMACEncoding = V1WebhookHMACEncoding.HEX,
) -> str:
algorithm_map = {
V1WebhookHMACAlgorithm.SHA1: hashlib.sha1,
V1WebhookHMACAlgorithm.SHA256: hashlib.sha256,
V1WebhookHMACAlgorithm.SHA512: hashlib.sha512,
V1WebhookHMACAlgorithm.MD5: hashlib.md5,
}
hash_func = algorithm_map[algorithm]
signature = hmac.new(secret.encode(), payload, hash_func).digest()
if encoding == V1WebhookHMACEncoding.HEX:
return signature.hex()
if encoding == V1WebhookHMACEncoding.BASE64:
return base64.b64encode(signature).decode()
if encoding == V1WebhookHMACEncoding.BASE64URL:
return base64.urlsafe_b64encode(signature).decode()
raise ValueError(f"Unsupported encoding: {encoding}")
async def send_webhook_request(
url: str,
body: WebhookInput,
auth_type: str,
auth_data: dict[str, Any] | None = None,
headers: dict[str, str] | None = None,
) -> aiohttp.ClientResponse:
request_headers = headers or {}
auth = None
if auth_type == "BASIC" and auth_data:
auth = aiohttp.BasicAuth(auth_data["username"], auth_data["password"])
elif auth_type == "API_KEY" and auth_data:
request_headers[auth_data["header_name"]] = auth_data["api_key"]
elif auth_type == "HMAC" and auth_data:
payload = json.dumps(body.model_dump()).encode()
signature = create_hmac_signature(
payload,
auth_data["secret"],
auth_data.get("algorithm", V1WebhookHMACAlgorithm.SHA256),
auth_data.get("encoding", V1WebhookHMACEncoding.HEX),
)
request_headers[auth_data["header_name"]] = signature
async with aiohttp.ClientSession() as session:
return await session.post(
url, json=body.model_dump(), auth=auth, headers=request_headers
)
async def wait_for_event(
hatchet: Hatchet,
webhook_name: str,
test_start: datetime,
) -> V1Event | None:
await asyncio.sleep(5)
events = await hatchet.event.aio_list(since=test_start)
if events.rows is None:
return None
return next(
(
event
for event in events.rows
if event.triggering_webhook_name == webhook_name
),
None,
)
async def wait_for_workflow_run(
hatchet: Hatchet, event_id: str, test_start: datetime
) -> V1TaskSummary | None:
await asyncio.sleep(5)
runs = await hatchet.runs.aio_list(
since=test_start,
additional_metadata={
"hatchet__event_id": event_id,
},
)
if len(runs.rows) == 0:
return None
return runs.rows[0]
@asynccontextmanager
async def basic_auth_webhook(
hatchet: Hatchet,
test_run_id: str,
username: str = TEST_BASIC_USERNAME,
password: str = TEST_BASIC_PASSWORD,
source_name: V1WebhookSourceName = V1WebhookSourceName.GENERIC,
) -> AsyncGenerator[V1Webhook, None]:
## Hack to get the API client
client = hatchet.metrics.client()
webhook_api = WebhookApi(client)
webhook_request = V1CreateWebhookRequestBasicAuth(
sourceName=source_name,
name=f"test-webhook-basic-{test_run_id}",
eventKeyExpression=f"'{hatchet.config.apply_namespace('webhook')}:' + input.type",
authType="BASIC",
auth=V1WebhookBasicAuth(
username=username,
password=password,
),
)
incoming_webhook = webhook_api.v1_webhook_create(
tenant=hatchet.tenant_id,
v1_create_webhook_request=V1CreateWebhookRequest(webhook_request),
)
try:
yield incoming_webhook
finally:
webhook_api.v1_webhook_delete(
tenant=hatchet.tenant_id,
v1_webhook=incoming_webhook.name,
)
@asynccontextmanager
async def api_key_webhook(
hatchet: Hatchet,
test_run_id: str,
header_name: str = TEST_API_KEY_HEADER,
api_key: str = TEST_API_KEY_VALUE,
source_name: V1WebhookSourceName = V1WebhookSourceName.GENERIC,
) -> AsyncGenerator[V1Webhook, None]:
client = hatchet.metrics.client()
webhook_api = WebhookApi(client)
webhook_request = V1CreateWebhookRequestAPIKey(
sourceName=source_name,
name=f"test-webhook-apikey-{test_run_id}",
eventKeyExpression=f"'{hatchet.config.apply_namespace('webhook')}:' + input.type",
authType="API_KEY",
auth=V1WebhookAPIKeyAuth(
headerName=header_name,
apiKey=api_key,
),
)
incoming_webhook = webhook_api.v1_webhook_create(
tenant=hatchet.tenant_id,
v1_create_webhook_request=V1CreateWebhookRequest(webhook_request),
)
try:
yield incoming_webhook
finally:
webhook_api.v1_webhook_delete(
tenant=hatchet.tenant_id,
v1_webhook=incoming_webhook.name,
)
@asynccontextmanager
async def hmac_webhook(
hatchet: Hatchet,
test_run_id: str,
signature_header_name: str = TEST_HMAC_SIGNATURE_HEADER,
signing_secret: str = TEST_HMAC_SECRET,
algorithm: V1WebhookHMACAlgorithm = V1WebhookHMACAlgorithm.SHA256,
encoding: V1WebhookHMACEncoding = V1WebhookHMACEncoding.HEX,
source_name: V1WebhookSourceName = V1WebhookSourceName.GENERIC,
) -> AsyncGenerator[V1Webhook, None]:
client = hatchet.metrics.client()
webhook_api = WebhookApi(client)
webhook_request = V1CreateWebhookRequestHMAC(
sourceName=source_name,
name=f"test-webhook-hmac-{test_run_id}",
eventKeyExpression=f"'{hatchet.config.apply_namespace('webhook')}:' + input.type",
authType="HMAC",
auth=V1WebhookHMACAuth(
algorithm=algorithm,
encoding=encoding,
signatureHeaderName=signature_header_name,
signingSecret=signing_secret,
),
)
incoming_webhook = webhook_api.v1_webhook_create(
tenant=hatchet.tenant_id,
v1_create_webhook_request=V1CreateWebhookRequest(webhook_request),
)
try:
yield incoming_webhook
finally:
webhook_api.v1_webhook_delete(
tenant=hatchet.tenant_id,
v1_webhook=incoming_webhook.name,
)
def url(tenant_id: str, webhook_name: str) -> str:
return f"http://localhost:8080/api/v1/stable/tenants/{tenant_id}/webhooks/{webhook_name}"
async def assert_has_runs(
hatchet: Hatchet,
test_start: datetime,
webhook_body: WebhookInput,
incoming_webhook: V1Webhook,
) -> None:
triggered_event = await wait_for_event(hatchet, incoming_webhook.name, test_start)
assert triggered_event is not None
assert (
triggered_event.key
== f"{hatchet.config.apply_namespace('webhook')}:{webhook_body.type}"
)
assert triggered_event.payload == webhook_body.model_dump()
workflow_run = await wait_for_workflow_run(
hatchet, triggered_event.metadata.id, test_start
)
assert workflow_run is not None
assert workflow_run.status == V1TaskStatus.COMPLETED
assert workflow_run.additional_metadata is not None
assert (
workflow_run.additional_metadata["hatchet__event_id"]
== triggered_event.metadata.id
)
assert workflow_run.additional_metadata["hatchet__event_key"] == triggered_event.key
assert workflow_run.status == V1TaskStatus.COMPLETED
async def assert_event_not_created(
hatchet: Hatchet,
test_start: datetime,
incoming_webhook: V1Webhook,
) -> None:
triggered_event = await wait_for_event(hatchet, incoming_webhook.name, test_start)
assert triggered_event is None
@pytest.mark.asyncio(loop_scope="session")
async def test_basic_auth_success(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
) -> None:
async with basic_auth_webhook(hatchet, test_run_id) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name),
webhook_body,
"BASIC",
{"username": TEST_BASIC_USERNAME, "password": TEST_BASIC_PASSWORD},
) as response:
assert response.status == 200
data = await response.json()
assert data == {"message": "ok"}
await assert_has_runs(
hatchet,
test_start,
webhook_body,
incoming_webhook,
)
@pytest.mark.parametrize(
"username,password",
[
("test_user", "incorrect_password"),
("incorrect_user", "test_password"),
("incorrect_user", "incorrect_password"),
("", ""),
],
)
@pytest.mark.asyncio(loop_scope="session")
async def test_basic_auth_failure(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
username: str,
password: str,
) -> None:
"""Test basic authentication failures."""
async with basic_auth_webhook(hatchet, test_run_id) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name),
webhook_body,
"BASIC",
{"username": username, "password": password},
) as response:
assert response.status == 403
await assert_event_not_created(
hatchet,
test_start,
incoming_webhook,
)
@pytest.mark.asyncio(loop_scope="session")
async def test_basic_auth_missing_credentials(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
) -> None:
async with basic_auth_webhook(hatchet, test_run_id) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name), webhook_body, "NONE"
) as response:
assert response.status == 403
await assert_event_not_created(
hatchet,
test_start,
incoming_webhook,
)
@pytest.mark.asyncio(loop_scope="session")
async def test_api_key_success(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
) -> None:
async with api_key_webhook(hatchet, test_run_id) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name),
webhook_body,
"API_KEY",
{"header_name": TEST_API_KEY_HEADER, "api_key": TEST_API_KEY_VALUE},
) as response:
assert response.status == 200
data = await response.json()
assert data == {"message": "ok"}
await assert_has_runs(
hatchet,
test_start,
webhook_body,
incoming_webhook,
)
@pytest.mark.parametrize(
"api_key",
[
"incorrect_api_key",
"",
"partial_key",
],
)
@pytest.mark.asyncio(loop_scope="session")
async def test_api_key_failure(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
api_key: str,
) -> None:
async with api_key_webhook(hatchet, test_run_id) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name),
webhook_body,
"API_KEY",
{"header_name": TEST_API_KEY_HEADER, "api_key": api_key},
) as response:
assert response.status == 403
await assert_event_not_created(
hatchet,
test_start,
incoming_webhook,
)
@pytest.mark.asyncio(loop_scope="session")
async def test_api_key_missing_header(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
) -> None:
async with api_key_webhook(hatchet, test_run_id) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name), webhook_body, "NONE"
) as response:
assert response.status == 403
await assert_event_not_created(
hatchet,
test_start,
incoming_webhook,
)
@pytest.mark.asyncio(loop_scope="session")
async def test_hmac_success(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
) -> None:
async with hmac_webhook(hatchet, test_run_id) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name),
webhook_body,
"HMAC",
{
"header_name": TEST_HMAC_SIGNATURE_HEADER,
"secret": TEST_HMAC_SECRET,
"algorithm": V1WebhookHMACAlgorithm.SHA256,
"encoding": V1WebhookHMACEncoding.HEX,
},
) as response:
assert response.status == 200
data = await response.json()
assert data == {"message": "ok"}
await assert_has_runs(
hatchet,
test_start,
webhook_body,
incoming_webhook,
)
@pytest.mark.parametrize(
"algorithm,encoding",
[
(V1WebhookHMACAlgorithm.SHA1, V1WebhookHMACEncoding.HEX),
(V1WebhookHMACAlgorithm.SHA256, V1WebhookHMACEncoding.BASE64),
(V1WebhookHMACAlgorithm.SHA512, V1WebhookHMACEncoding.BASE64URL),
(V1WebhookHMACAlgorithm.MD5, V1WebhookHMACEncoding.HEX),
],
)
@pytest.mark.asyncio(loop_scope="session")
async def test_hmac_different_algorithms_and_encodings(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
algorithm: V1WebhookHMACAlgorithm,
encoding: V1WebhookHMACEncoding,
) -> None:
async with hmac_webhook(
hatchet, test_run_id, algorithm=algorithm, encoding=encoding
) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name),
webhook_body,
"HMAC",
{
"header_name": TEST_HMAC_SIGNATURE_HEADER,
"secret": TEST_HMAC_SECRET,
"algorithm": algorithm,
"encoding": encoding,
},
) as response:
assert response.status == 200
data = await response.json()
assert data == {"message": "ok"}
await assert_has_runs(
hatchet,
test_start,
webhook_body,
incoming_webhook,
)
@pytest.mark.parametrize(
"secret",
[
"incorrect_secret",
"",
"partial_secret",
],
)
@pytest.mark.asyncio(loop_scope="session")
async def test_hmac_signature_failure(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
secret: str,
) -> None:
async with hmac_webhook(hatchet, test_run_id) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name),
webhook_body,
"HMAC",
{
"header_name": TEST_HMAC_SIGNATURE_HEADER,
"secret": secret,
"algorithm": V1WebhookHMACAlgorithm.SHA256,
"encoding": V1WebhookHMACEncoding.HEX,
},
) as response:
assert response.status == 403
await assert_event_not_created(
hatchet,
test_start,
incoming_webhook,
)
@pytest.mark.asyncio(loop_scope="session")
async def test_hmac_missing_signature_header(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
) -> None:
async with hmac_webhook(hatchet, test_run_id) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name), webhook_body, "NONE"
) as response:
assert response.status == 403
await assert_event_not_created(
hatchet,
test_start,
incoming_webhook,
)
@pytest.mark.parametrize(
"source_name",
[
V1WebhookSourceName.GENERIC,
V1WebhookSourceName.GITHUB,
],
)
@pytest.mark.asyncio(loop_scope="session")
async def test_different_source_types(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
source_name: V1WebhookSourceName,
) -> None:
async with basic_auth_webhook(
hatchet, test_run_id, source_name=source_name
) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name),
webhook_body,
"BASIC",
{"username": TEST_BASIC_USERNAME, "password": TEST_BASIC_PASSWORD},
) as response:
assert response.status == 200
data = await response.json()
assert data == {"message": "ok"}
await assert_has_runs(
hatchet,
test_start,
webhook_body,
incoming_webhook,
)
examples/python/webhooks/worker.py
+27
View File
@@ -0,0 +1,27 @@
# > Webhooks
from pydantic import BaseModel
from hatchet_sdk import Context, Hatchet
hatchet = Hatchet(debug=True)
class WebhookInput(BaseModel):
type: str
message: str
@hatchet.task(input_validator=WebhookInput, on_events=["webhook:test"])
def webhook(input: WebhookInput, ctx: Context) -> dict[str, str]:
return input.model_dump()
def main() -> None:
worker = hatchet.worker("webhook-worker", workflows=[webhook])
worker.start()
if __name__ == "__main__":
main()
examples/python/worker.py
+2
View File
@@ -26,6 +26,7 @@ from examples.on_failure.worker import on_failure_wf, on_failure_wf_with_details
from examples.return_exceptions.worker import return_exceptions_task
from examples.simple.worker import simple, simple_durable
from examples.timeout.worker import refresh_timeout_wf, timeout_wf
from examples.webhooks.worker import webhook
from hatchet_sdk import Hatchet
hatchet = Hatchet(debug=True)
@@ -66,6 +67,7 @@ def main() -> None:
bulk_replay_test_1,
bulk_replay_test_2,
bulk_replay_test_3,
webhook,
return_exceptions_task,
wait_for_sleep_twice,
],
frontend/app/src/lib/api/generated/Api.ts
+132
View File
@@ -89,6 +89,7 @@ import {
V1CancelTaskRequest,
V1CancelledTasks,
V1CreateFilterRequest,
V1CreateWebhookRequest,
V1DagChildren,
V1EventList,
V1Filter,
@@ -105,6 +106,9 @@ import {
V1TaskTimingList,
V1TriggerWorkflowRunRequest,
V1UpdateFilterRequest,
V1Webhook,
V1WebhookList,
V1WebhookSourceName,
V1WorkflowRunDetails,
V1WorkflowRunDisplayNameList,
WebhookWorkerCreateRequest,
@@ -788,6 +792,134 @@ export class Api<
format: "json",
...params,
});
/**
* @description Lists all webhook for a tenant.
*
* @tags Webhook
* @name V1WebhookList
* @summary List webhooks
* @request GET:/api/v1/stable/tenants/{tenant}/webhooks
* @secure
*/
v1WebhookList = (
tenant: string,
query?: {
/**
* The number to skip
* @format int64
*/
offset?: number;
/**
* The number to limit by
* @format int64
*/
limit?: number;
/** The source names to filter by */
sourceNames?: V1WebhookSourceName[];
/** The webhook names to filter by */
webhookNames?: string[];
},
params: RequestParams = {},
) =>
this.request<V1WebhookList, APIErrors>({
path: `/api/v1/stable/tenants/${tenant}/webhooks`,
method: "GET",
query: query,
secure: true,
format: "json",
...params,
});
/**
* @description Create a new webhook
*
* @tags Webhook
* @name V1WebhookCreate
* @summary Create a webhook
* @request POST:/api/v1/stable/tenants/{tenant}/webhooks
* @secure
*/
v1WebhookCreate = (
tenant: string,
data: V1CreateWebhookRequest,
params: RequestParams = {},
) =>
this.request<V1Webhook, APIErrors>({
path: `/api/v1/stable/tenants/${tenant}/webhooks`,
method: "POST",
body: data,
secure: true,
type: ContentType.Json,
format: "json",
...params,
});
/**
* @description Get a webhook by its name
*
* @tags Webhook
* @name V1WebhookGet
* @summary Get a webhook
* @request GET:/api/v1/stable/tenants/{tenant}/webhooks/{v1-webhook}
* @secure
*/
v1WebhookGet = (
tenant: string,
v1Webhook: string,
params: RequestParams = {},
) =>
this.request<V1Webhook, APIErrors>({
path: `/api/v1/stable/tenants/${tenant}/webhooks/${v1Webhook}`,
method: "GET",
secure: true,
format: "json",
...params,
});
/**
* @description Delete a webhook
*
* @tags Webhook
* @name V1WebhookDelete
* @request DELETE:/api/v1/stable/tenants/{tenant}/webhooks/{v1-webhook}
* @secure
*/
v1WebhookDelete = (
tenant: string,
v1Webhook: string,
params: RequestParams = {},
) =>
this.request<V1Webhook, APIErrors>({
path: `/api/v1/stable/tenants/${tenant}/webhooks/${v1Webhook}`,
method: "DELETE",
secure: true,
format: "json",
...params,
});
/**
* @description Post an incoming webhook message
*
* @tags Webhook
* @name V1WebhookReceive
* @summary Post a webhook message
* @request POST:/api/v1/stable/tenants/{tenant}/webhooks/{v1-webhook}
*/
v1WebhookReceive = (
tenant: string,
v1Webhook: string,
data?: any,
params: RequestParams = {},
) =>
this.request<
{
/** @example "OK" */
message?: string;
},
APIErrors
>({
path: `/api/v1/stable/tenants/${tenant}/webhooks/${v1Webhook}`,
method: "POST",
body: data,
format: "json",
...params,
});
/**
* @description Evaluate a CEL expression against provided input data.
*
frontend/app/src/lib/api/generated/data-contracts.ts
+103
View File
@@ -211,6 +211,31 @@ export enum V1CELDebugResponseStatus {
ERROR = "ERROR",
}
export enum V1WebhookHMACEncoding {
HEX = "HEX",
BASE64 = "BASE64",
BASE64URL = "BASE64URL",
}
export enum V1WebhookHMACAlgorithm {
SHA1 = "SHA1",
SHA256 = "SHA256",
SHA512 = "SHA512",
MD5 = "MD5",
}
export enum V1WebhookAuthType {
BASIC = "BASIC",
API_KEY = "API_KEY",
HMAC = "HMAC",
}
export enum V1WebhookSourceName {
GENERIC = "GENERIC",
GITHUB = "GITHUB",
STRIPE = "STRIPE",
}
export enum TenantUIVersion {
V0 = "V0",
V1 = "V1",
@@ -792,6 +817,8 @@ export interface V1Event {
seenAt?: string;
/** The external IDs of the runs that were triggered by this event. */
triggeredRuns?: V1EventTriggeredRun[];
/** The name of the webhook that triggered this event, if applicable. */
triggeringWebhookName?: string;
}
export interface V1EventList {
@@ -853,6 +880,82 @@ export interface V1UpdateFilterRequest {
payload?: object;
}
export interface V1Webhook {
metadata: APIResourceMeta;
/** The ID of the tenant associated with this webhook. */
tenantId: string;
/** The name of the webhook */
name: string;
/** The name of the source for this webhook */
sourceName: V1WebhookSourceName;
/** The CEL expression to use for the event key. This is used to create the event key from the webhook payload. */
eventKeyExpression: string;
/** The type of authentication to use for the webhook */
authType: V1WebhookAuthType;
}
export interface V1WebhookList {
pagination?: PaginationResponse;
rows?: V1Webhook[];
}
export interface V1CreateWebhookRequestBase {
/** The name of the source for this webhook */
sourceName: V1WebhookSourceName;
/** The name of the webhook */
name: string;
/** The CEL expression to use for the event key. This is used to create the event key from the webhook payload. */
eventKeyExpression: string;
}
export interface V1WebhookBasicAuth {
/** The username for basic auth */
username: string;
/** The password for basic auth */
password: string;
}
export type V1CreateWebhookRequestBasicAuth = V1CreateWebhookRequestBase & {
/** The type of authentication to use for the webhook */
authType: "BASIC";
auth: V1WebhookBasicAuth;
};
export interface V1WebhookAPIKeyAuth {
/** The name of the header to use for the API key */
headerName: string;
/** The API key to use for authentication */
apiKey: string;
}
export type V1CreateWebhookRequestAPIKey = V1CreateWebhookRequestBase & {
/** The type of authentication to use for the webhook */
authType: "API_KEY";
auth: V1WebhookAPIKeyAuth;
};
export interface V1WebhookHMACAuth {
/** The HMAC algorithm to use for the webhook */
algorithm: V1WebhookHMACAlgorithm;
/** The encoding to use for the HMAC signature */
encoding: V1WebhookHMACEncoding;
/** The name of the header to use for the HMAC signature */
signatureHeaderName: string;
/** The secret key used to sign the HMAC signature */
signingSecret: string;
}
export type V1CreateWebhookRequestHMAC = V1CreateWebhookRequestBase & {
/** The type of authentication to use for the webhook */
authType: "HMAC";
auth: V1WebhookHMACAuth;
};
export type V1CreateWebhookRequest =
| V1CreateWebhookRequestBasicAuth
| V1CreateWebhookRequestAPIKey
| V1CreateWebhookRequestHMAC;
export interface V1CELDebugRequest {
/** The CEL expression to evaluate */
expression: string;
frontend/app/src/lib/api/queries.ts
+7
View File
@@ -24,6 +24,7 @@ type V2TaskGetPointMetricsQuery = Parameters<
typeof api.v1TaskGetPointMetrics
>[1];
type GetTaskMetricsQuery = Parameters<typeof api.v1TaskListStatusMetrics>[1];
type ListWebhooksQuery = Parameters<typeof api.v1WebhookList>[1];
export const queries = createQueryKeyStore({
cloud: {
@@ -256,6 +257,12 @@ export const queries = createQueryKeyStore({
(await api.workflowRunListStepRunEvents(tenantId, workflowRun)).data,
}),
},
v1Webhooks: {
list: (tenant: string, params?: ListWebhooksQuery | undefined) => ({
queryKey: ['v1:webhook:list', tenant],
queryFn: async () => (await api.v1WebhookList(tenant, params)).data,
}),
},
v1Events: {
list: (tenant: string, query: V1EventListQuery) => ({
queryKey: ['v1:events:list', tenant, query],
frontend/app/src/lib/breadcrumbs.ts
+2
View File
@@ -10,6 +10,8 @@ const createRouteLabel = (path: TenantedPath): string => {
switch (path) {
case '/tenants/:tenant/events':
return 'Events';
case '/tenants/:tenant/webhooks':
return 'Webhooks';
case '/tenants/:tenant/rate-limits':
return 'Rate Limits';
case '/tenants/:tenant/scheduled':
frontend/app/src/next/lib/docs/generated/snips/python/index.ts
+2
View File
@@ -43,6 +43,7 @@ import * as sticky_workers from './sticky_workers';
import * as streaming from './streaming';
import * as timeout from './timeout';
import * as unit_testing from './unit_testing';
import * as webhooks from './webhooks';
import * as worker_existing_loop from './worker_existing_loop';
import * as workflow_registration from './workflow_registration';
@@ -91,5 +92,6 @@ export { sticky_workers };
export { streaming };
export { timeout };
export { unit_testing };
export { webhooks };
export { worker_existing_loop };
export { workflow_registration };
frontend/app/src/next/lib/docs/generated/snips/python/webhooks/index.ts
+5
View File
@@ -0,0 +1,5 @@
import test_webhooks from './test_webhooks';
import worker from './worker';
export { test_webhooks };
export { worker };
frontend/app/src/next/lib/docs/generated/snips/python/webhooks/test_webhooks.ts
+12
View File
File diff suppressed because one or more lines are too long
frontend/app/src/next/lib/docs/generated/snips/python/webhooks/worker.ts
+17
View File
@@ -0,0 +1,17 @@
import { Snippet } from '@/next/lib/docs/generated/snips/types';
const snippet: Snippet = {
language: 'python',
content:
'# > Webhooks\n\nfrom pydantic import BaseModel\n\nfrom hatchet_sdk import Context, Hatchet\n\nhatchet = Hatchet(debug=True)\n\n\nclass WebhookInput(BaseModel):\n type: str\n message: str\n\n\n@hatchet.task(input_validator=WebhookInput, on_events=["webhook:test"])\ndef webhook(input: WebhookInput, ctx: Context) -> dict[str, str]:\n return input.model_dump()\n\n\ndef main() -> None:\n worker = hatchet.worker("webhook-worker", workflows=[webhook])\n worker.start()\n\n\n\nif __name__ == "__main__":\n main()\n',
source: 'out/python/webhooks/worker.py',
blocks: {
webhooks: {
start: 2,
stop: 24,
},
},
highlights: {},
};
export default snippet;
frontend/app/src/next/lib/docs/generated/snips/python/worker.ts
+1 -1
View File
@@ -3,7 +3,7 @@ import { Snippet } from '@/next/lib/docs/generated/snips/types';
const snippet: Snippet = {
language: 'python',
content:
'from examples.affinity_workers.worker import affinity_worker_workflow\nfrom examples.bulk_fanout.worker import bulk_child_wf, bulk_parent_wf\nfrom examples.bulk_operations.worker import (\n bulk_replay_test_1,\n bulk_replay_test_2,\n bulk_replay_test_3,\n)\nfrom examples.cancellation.worker import cancellation_workflow\nfrom examples.concurrency_limit.worker import concurrency_limit_workflow\nfrom examples.concurrency_limit_rr.worker import concurrency_limit_rr_workflow\nfrom examples.concurrency_multiple_keys.worker import concurrency_multiple_keys_workflow\nfrom examples.concurrency_workflow_level.worker import (\n concurrency_workflow_level_workflow,\n)\nfrom examples.conditions.worker import task_condition_workflow\nfrom examples.dag.worker import dag_workflow\nfrom examples.dedupe.worker import dedupe_child_wf, dedupe_parent_wf\nfrom examples.durable.worker import durable_workflow, wait_for_sleep_twice\nfrom examples.events.worker import event_workflow\nfrom examples.fanout.worker import child_wf, parent_wf\nfrom examples.fanout_sync.worker import sync_fanout_child, sync_fanout_parent\nfrom examples.lifespans.simple import lifespan, lifespan_task\nfrom examples.logger.workflow import logging_workflow\nfrom examples.non_retryable.worker import non_retryable_workflow\nfrom examples.on_failure.worker import on_failure_wf, on_failure_wf_with_details\nfrom examples.return_exceptions.worker import return_exceptions_task\nfrom examples.simple.worker import simple, simple_durable\nfrom examples.timeout.worker import refresh_timeout_wf, timeout_wf\nfrom hatchet_sdk import Hatchet\n\nhatchet = Hatchet(debug=True)\n\n\ndef main() -> None:\n worker = hatchet.worker(\n "e2e-test-worker",\n slots=100,\n workflows=[\n affinity_worker_workflow,\n bulk_child_wf,\n bulk_parent_wf,\n concurrency_limit_workflow,\n concurrency_limit_rr_workflow,\n concurrency_multiple_keys_workflow,\n dag_workflow,\n dedupe_child_wf,\n dedupe_parent_wf,\n durable_workflow,\n child_wf,\n event_workflow,\n parent_wf,\n on_failure_wf,\n on_failure_wf_with_details,\n logging_workflow,\n timeout_wf,\n refresh_timeout_wf,\n task_condition_workflow,\n cancellation_workflow,\n sync_fanout_parent,\n sync_fanout_child,\n non_retryable_workflow,\n concurrency_workflow_level_workflow,\n lifespan_task,\n simple,\n simple_durable,\n bulk_replay_test_1,\n bulk_replay_test_2,\n bulk_replay_test_3,\n return_exceptions_task,\n wait_for_sleep_twice,\n ],\n lifespan=lifespan,\n )\n\n worker.start()\n\n\nif __name__ == "__main__":\n main()\n',
'from examples.affinity_workers.worker import affinity_worker_workflow\nfrom examples.bulk_fanout.worker import bulk_child_wf, bulk_parent_wf\nfrom examples.bulk_operations.worker import (\n bulk_replay_test_1,\n bulk_replay_test_2,\n bulk_replay_test_3,\n)\nfrom examples.cancellation.worker import cancellation_workflow\nfrom examples.concurrency_limit.worker import concurrency_limit_workflow\nfrom examples.concurrency_limit_rr.worker import concurrency_limit_rr_workflow\nfrom examples.concurrency_multiple_keys.worker import concurrency_multiple_keys_workflow\nfrom examples.concurrency_workflow_level.worker import (\n concurrency_workflow_level_workflow,\n)\nfrom examples.conditions.worker import task_condition_workflow\nfrom examples.dag.worker import dag_workflow\nfrom examples.dedupe.worker import dedupe_child_wf, dedupe_parent_wf\nfrom examples.durable.worker import durable_workflow, wait_for_sleep_twice\nfrom examples.events.worker import event_workflow\nfrom examples.fanout.worker import child_wf, parent_wf\nfrom examples.fanout_sync.worker import sync_fanout_child, sync_fanout_parent\nfrom examples.lifespans.simple import lifespan, lifespan_task\nfrom examples.logger.workflow import logging_workflow\nfrom examples.non_retryable.worker import non_retryable_workflow\nfrom examples.on_failure.worker import on_failure_wf, on_failure_wf_with_details\nfrom examples.return_exceptions.worker import return_exceptions_task\nfrom examples.simple.worker import simple, simple_durable\nfrom examples.timeout.worker import refresh_timeout_wf, timeout_wf\nfrom examples.webhooks.worker import webhook\nfrom hatchet_sdk import Hatchet\n\nhatchet = Hatchet(debug=True)\n\n\ndef main() -> None:\n worker = hatchet.worker(\n "e2e-test-worker",\n slots=100,\n workflows=[\n affinity_worker_workflow,\n bulk_child_wf,\n bulk_parent_wf,\n concurrency_limit_workflow,\n concurrency_limit_rr_workflow,\n concurrency_multiple_keys_workflow,\n dag_workflow,\n dedupe_child_wf,\n dedupe_parent_wf,\n durable_workflow,\n child_wf,\n event_workflow,\n parent_wf,\n on_failure_wf,\n on_failure_wf_with_details,\n logging_workflow,\n timeout_wf,\n refresh_timeout_wf,\n task_condition_workflow,\n cancellation_workflow,\n sync_fanout_parent,\n sync_fanout_child,\n non_retryable_workflow,\n concurrency_workflow_level_workflow,\n lifespan_task,\n simple,\n simple_durable,\n bulk_replay_test_1,\n bulk_replay_test_2,\n bulk_replay_test_3,\n webhook,\n return_exceptions_task,\n wait_for_sleep_twice,\n ],\n lifespan=lifespan,\n )\n\n worker.start()\n\n\nif __name__ == "__main__":\n main()\n',
source: 'out/python/worker.py',
blocks: {},
highlights: {},
frontend/app/src/pages/main/v1/index.tsx
+8 -1
View File
@@ -27,7 +27,7 @@ import {
import useCloudApiMeta from '@/pages/auth/hooks/use-cloud-api-meta';
import useCloudFeatureFlags from '@/pages/auth/hooks/use-cloud-feature-flags';
import { useSidebar } from '@/components/sidebar-provider';
import { SquareActivityIcon } from 'lucide-react';
import { SquareActivityIcon, WebhookIcon } from 'lucide-react';
import { useCurrentTenantId } from '@/hooks/use-tenant';
function Main() {
@@ -165,6 +165,13 @@ function Sidebar({ className, memberships }: SidebarProps) {
prefix={`/tenants/${tenantId}/workers`}
collapsibleChildren={workers}
/>
<SidebarButtonPrimary
key="webhooks"
onNavLinkClick={onNavLinkClick}
to={`/tenants/${tenantId}/webhooks`}
name="Webhooks"
icon={<WebhookIcon className="mr-2 h-4 w-4" />}
/>
{featureFlags?.data['managed-worker'] && (
<SidebarButtonPrimary
key="managed-compute"
frontend/app/src/pages/main/v1/webhooks/components/auth-method.tsx
+37
View File
@@ -0,0 +1,37 @@
import { V1WebhookAuthType } from '@/lib/api';
import { Key, ShieldCheck, UserCheck } from 'lucide-react';
export const AuthMethod = ({
authMethod,
}: {
authMethod: V1WebhookAuthType;
}) => {
switch (authMethod) {
case V1WebhookAuthType.BASIC:
return (
<span className="flex flex-row gap-x-2 items-center">
<UserCheck className="size-4" />
Basic
</span>
);
case V1WebhookAuthType.API_KEY:
return (
<span className="flex flex-row gap-x-2 items-center">
<Key className="size-4" />
API Key
</span>
);
case V1WebhookAuthType.HMAC:
return (
<span className="flex flex-row gap-x-2 items-center">
<ShieldCheck className="size-4" />
HMAC
</span>
);
default:
// eslint-disable-next-line no-case-declarations
const exhaustiveCheck: never = authMethod;
throw new Error(`Unhandled auth method: ${exhaustiveCheck}`);
}
};
frontend/app/src/pages/main/v1/webhooks/components/auth-setup.tsx
+251
View File
@@ -0,0 +1,251 @@
import { Input } from '@/components/v1/ui/input';
import { Label } from '@/components/v1/ui/label';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/v1/ui/select';
import { useForm } from 'react-hook-form';
import {
V1WebhookAuthType,
V1WebhookHMACAlgorithm,
V1WebhookHMACEncoding,
V1WebhookSourceName,
} from '@/lib/api';
import { WebhookFormData } from '../hooks/use-webhooks';
type BaseAuthMethodProps = {
register: ReturnType<typeof useForm<WebhookFormData>>['register'];
};
type HMACAuthProps = BaseAuthMethodProps & {
watch: ReturnType<typeof useForm<WebhookFormData>>['watch'];
setValue: ReturnType<typeof useForm<WebhookFormData>>['setValue'];
};
const BasicAuth = ({ register }: BaseAuthMethodProps) => (
<div className="space-y-4">
<div className="space-y-2">
<Label htmlFor="username" className="text-sm font-medium">
Username <span className="text-red-500">*</span>
</Label>
<Input
data-1p-ignore
id="username"
placeholder="username"
{...register('username')}
className="h-10"
/>
</div>
<div className="space-y-2">
<Label htmlFor="password" className="text-sm font-medium">
Password <span className="text-red-500">*</span>
</Label>
<div className="relative">
<Input
data-1p-ignore
id="password"
type={'text'}
placeholder="password"
{...register('password')}
className="h-10 pr-10"
/>
</div>
</div>
</div>
);
const APIKeyAuth = ({ register }: BaseAuthMethodProps) => (
<div className="space-y-4">
<div className="space-y-2">
<Label htmlFor="headerName" className="text-sm font-medium">
Header Name <span className="text-red-500">*</span>
</Label>
<Input
data-1p-ignore
id="headerName"
placeholder="X-API-Key"
{...register('headerName')}
className="h-10"
/>
</div>
<div className="space-y-2">
<Label htmlFor="apiKey" className="text-sm font-medium">
API Key <span className="text-red-500">*</span>
</Label>
<div className="relative">
<Input
data-1p-ignore
id="apiKey"
type={'text'}
placeholder="API key..."
{...register('apiKey')}
className="h-10 pr-10"
/>
</div>
</div>
</div>
);
const HMACAuth = ({ register, watch, setValue }: HMACAuthProps) => (
<div className="space-y-4">
<div className="space-y-2">
<Label htmlFor="signingSecret" className="text-sm font-medium">
Webhook Signing Secret <span className="text-red-500">*</span>
</Label>
<div className="relative">
<Input
data-1p-ignore
id="signingSecret"
type={'text'}
placeholder="Secret key..."
{...register('signingSecret')}
className="h-10 pr-10"
/>
</div>
</div>
<div className="grid grid-cols-2 gap-4">
<div className="space-y-2">
<Label htmlFor="algorithm" className="text-sm font-medium">
Algorithm
</Label>
<Select
value={watch('algorithm')}
onValueChange={(value: V1WebhookHMACAlgorithm) =>
setValue('algorithm', value)
}
>
<SelectTrigger className="h-10">
<SelectValue />
</SelectTrigger>
<SelectContent>
<SelectItem value="SHA256">SHA256</SelectItem>
<SelectItem value="SHA1">SHA1</SelectItem>
<SelectItem value="SHA512">SHA512</SelectItem>
<SelectItem value="MD5">MD5</SelectItem>
</SelectContent>
</Select>
</div>
<div className="space-y-2">
<Label htmlFor="encoding" className="text-sm font-medium">
Encoding
</Label>
<Select
value={watch('encoding')}
onValueChange={(value: V1WebhookHMACEncoding) =>
setValue('encoding', value)
}
>
<SelectTrigger className="h-10">
<SelectValue />
</SelectTrigger>
<SelectContent>
<SelectItem value="HEX">HEX</SelectItem>
<SelectItem value="BASE64">BASE64</SelectItem>
<SelectItem value="BASE64URL">BASE64URL</SelectItem>
</SelectContent>
</Select>
</div>
</div>
<div className="space-y-2">
<Label htmlFor="signatureHeaderName" className="text-sm font-medium">
Signature Header Name
</Label>
<Input
data-1p-ignore
id="signatureHeaderName"
placeholder="X-Signature"
{...register('signatureHeaderName')}
className="h-10"
/>
</div>
</div>
);
const StripeAuth = ({ register }: BaseAuthMethodProps) => (
// Stripe only requires a secret, as we know the header key and the encoding info (user doesn't need to provide them)
// See docs: https://docs.stripe.com/webhooks?verify=verify-manually#verify-manually
<div className="space-y-4">
<div className="space-y-2">
<Label htmlFor="signingSecret" className="text-sm font-medium">
Webhook Signing Secret <span className="text-red-500">*</span>
</Label>
<div className="relative">
<Input
data-1p-ignore
id="signingSecret"
type={'text'}
placeholder="whsec_..."
{...register('signingSecret')}
className="h-10 pr-10"
/>
</div>
</div>
</div>
);
const GithubAuth = ({ register }: BaseAuthMethodProps) => (
// Github only requires a secret, as we know the header key and the encoding info (user doesn't need to provide them)
// See docs: https://docs.github.com/en/webhooks/using-webhooks/validating-webhook-deliveries#validating-webhook-deliveries
<div className="space-y-4">
<div className="space-y-2">
<Label htmlFor="signingSecret" className="text-sm font-medium">
Secret <span className="text-red-500">*</span>
</Label>
<div className="relative">
<Input
data-1p-ignore
id="signingSecret"
type={'text'}
placeholder="super-secret"
{...register('signingSecret')}
className="h-10 pr-10"
/>
</div>
</div>
</div>
);
export const AuthSetup = ({
authMethod,
sourceName,
register,
watch,
setValue,
}: HMACAuthProps & {
authMethod: V1WebhookAuthType;
sourceName: V1WebhookSourceName;
}) => {
switch (sourceName) {
case V1WebhookSourceName.GENERIC:
switch (authMethod) {
case V1WebhookAuthType.BASIC:
return <BasicAuth register={register} />;
case V1WebhookAuthType.API_KEY:
return <APIKeyAuth register={register} />;
case V1WebhookAuthType.HMAC:
return (
<HMACAuth register={register} watch={watch} setValue={setValue} />
);
default:
// eslint-disable-next-line no-case-declarations
const exhaustiveCheck: never = authMethod;
throw new Error(`Unhandled auth method: ${exhaustiveCheck}`);
}
case V1WebhookSourceName.GITHUB:
return <GithubAuth register={register} />;
case V1WebhookSourceName.STRIPE:
return <StripeAuth register={register} />;
default:
// eslint-disable-next-line no-case-declarations
const exhaustiveCheck: never = sourceName;
throw new Error(`Unhandled source name: ${exhaustiveCheck}`);
}
};
frontend/app/src/pages/main/v1/webhooks/components/source-name.tsx
+39
View File
@@ -0,0 +1,39 @@
import { V1WebhookSourceName } from '@/lib/api';
import { GitHubLogoIcon } from '@radix-ui/react-icons';
import { Webhook } from 'lucide-react';
import { FaStripeS } from 'react-icons/fa';
export const SourceName = ({
sourceName,
}: {
sourceName: V1WebhookSourceName;
}) => {
switch (sourceName) {
case V1WebhookSourceName.GENERIC:
return (
<span className="flex flex-row gap-x-2 items-center">
<Webhook className="size-4" />
Generic
</span>
);
case V1WebhookSourceName.GITHUB:
return (
<span className="flex flex-row gap-x-2 items-center">
<GitHubLogoIcon className="size-4" />
GitHub
</span>
);
case V1WebhookSourceName.STRIPE:
return (
<span className="flex flex-row gap-x-2 items-center">
<FaStripeS className="size-4" />
Stripe
</span>
);
default:
// eslint-disable-next-line no-case-declarations
const exhaustiveCheck: never = sourceName;
throw new Error(`Unhandled source: ${exhaustiveCheck}`);
}
};
frontend/app/src/pages/main/v1/webhooks/components/webhook-columns.tsx
+139
View File
@@ -0,0 +1,139 @@
import { ColumnDef, Row } from '@tanstack/react-table';
import { V1Webhook } from '@/lib/api';
import { DataTableColumnHeader } from '@/components/v1/molecules/data-table/data-table-column-header';
import { DotsVerticalIcon } from '@radix-ui/react-icons';
import { Check, Copy, Loader, Trash2 } from 'lucide-react';
import { Button } from '@/components/v1/ui/button';
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuTrigger,
} from '@/components/v1/ui/dropdown-menu';
import { useState } from 'react';
import { useWebhooks } from '../hooks/use-webhooks';
import { SourceName } from './source-name';
import { AuthMethod } from './auth-method';
export const columns = (): ColumnDef<V1Webhook>[] => {
return [
{
accessorKey: 'name',
header: ({ column }) => (
<DataTableColumnHeader column={column} title="Name" />
),
cell: ({ row }) => <div className="w-full">{row.original.name}</div>,
enableSorting: false,
enableHiding: true,
},
{
accessorKey: 'sourceName',
header: ({ column }) => (
<DataTableColumnHeader column={column} title="Source" />
),
cell: ({ row }) => (
<div className="w-full">
<SourceName sourceName={row.original.sourceName} />
</div>
),
enableSorting: false,
enableHiding: true,
},
{
accessorKey: 'expression',
header: ({ column }) => (
<DataTableColumnHeader column={column} title="Expression" />
),
cell: ({ row }) => (
<code className="bg-muted relative rounded px-2 py-1 font-mono text-xs h-full">
{row.original.eventKeyExpression}
</code>
),
enableSorting: false,
enableHiding: true,
},
{
accessorKey: 'authType',
header: ({ column }) => (
<DataTableColumnHeader column={column} title="Auth Method" />
),
cell: ({ row }) => (
<div className="w-full">
<AuthMethod authMethod={row.original.authType} />
</div>
),
enableSorting: false,
enableHiding: true,
},
{
accessorKey: 'actions',
header: ({ column }) => (
<DataTableColumnHeader column={column} title="" />
),
cell: ({ row }) => <WebhookActionsCell row={row} />,
enableSorting: false,
enableHiding: true,
},
];
};
const WebhookActionsCell = ({ row }: { row: Row<V1Webhook> }) => {
const { mutations, createWebhookURL } = useWebhooks(() =>
setIsDropdownOpen(false),
);
const [isCopied, setIsCopied] = useState(false);
const [isDropdownOpen, setIsDropdownOpen] = useState(false);
const webhookUrl = createWebhookURL(row.original.name);
const handleCopy = (url: string) => {
navigator.clipboard.writeText(url);
setIsCopied(true);
setTimeout(() => setIsCopied(false), 1000);
setTimeout(() => setIsDropdownOpen(false), 400);
};
return (
<DropdownMenu open={isDropdownOpen} onOpenChange={setIsDropdownOpen}>
<DropdownMenuTrigger asChild>
<Button variant="ghost" className="h-8 w-8 p-0 hover:bg-muted/50">
<DotsVerticalIcon className="h-4 w-4 text-muted-foreground cursor-pointer" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem
className="flex flex-row gap-x-2"
onClick={(e) => {
e.stopPropagation();
e.preventDefault();
handleCopy(webhookUrl);
}}
>
{isCopied ? (
<Check className="h-4 w-4 text-green-600" />
) : (
<Copy className="size-4" />
)}
Copy Webhook URL
</DropdownMenuItem>
<DropdownMenuItem
className="flex flex-row gap-x-2"
onClick={(e) => {
e.stopPropagation();
e.preventDefault();
mutations.deleteWebhook({ webhookName: row.original.name });
}}
disabled={mutations.isDeletePending}
>
{mutations.isDeletePending ? (
<Loader className="size-4 animate-spin" />
) : (
<Trash2 className="size-4" />
)}
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
);
};
frontend/app/src/pages/main/v1/webhooks/hooks/use-webhooks.tsx
+80
View File
@@ -0,0 +1,80 @@
import { z } from 'zod';
import { useCurrentTenantId } from '@/hooks/use-tenant';
import api, {
queries,
V1CreateWebhookRequest,
V1WebhookAuthType,
V1WebhookHMACAlgorithm,
V1WebhookHMACEncoding,
V1WebhookSourceName,
} from '@/lib/api';
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
export const useWebhooks = (onDeleteSuccess?: () => void) => {
const queryClient = useQueryClient();
const { tenantId } = useCurrentTenantId();
const { data, isLoading, error } = useQuery({
...queries.v1Webhooks.list(tenantId),
});
const { mutate: deleteWebhook, isPending: isDeletePending } = useMutation({
mutationFn: async ({ webhookName }: { webhookName: string }) =>
api.v1WebhookDelete(tenantId, webhookName),
onSuccess: async () => {
if (onDeleteSuccess) {
onDeleteSuccess();
}
const queryKey = queries.v1Webhooks.list(tenantId).queryKey;
await queryClient.invalidateQueries({
queryKey,
});
},
});
const { mutate: createWebhook, isPending: isCreatePending } = useMutation({
mutationFn: async (webhookData: V1CreateWebhookRequest) =>
api.v1WebhookCreate(tenantId, webhookData),
onSuccess: async () => {
const queryKey = queries.v1Webhooks.list(tenantId).queryKey;
await queryClient.invalidateQueries({
queryKey,
});
},
});
const createWebhookURL = (name: string) => {
return `${window.location.protocol}//${window.location.hostname}/api/v1/stable/tenants/${tenantId}/webhooks/${name}`;
};
return {
data: data?.rows || [],
isLoading,
error,
createWebhookURL,
mutations: {
deleteWebhook,
isDeletePending,
createWebhook,
isCreatePending,
},
};
};
export const webhookFormSchema = z.object({
sourceName: z.nativeEnum(V1WebhookSourceName),
name: z.string().min(1, 'Name expression is required'),
eventKeyExpression: z.string().min(1, 'Event key expression is required'),
authType: z.nativeEnum(V1WebhookAuthType),
username: z.string().optional(),
password: z.string().optional(),
headerName: z.string().optional(),
apiKey: z.string().optional(),
signingSecret: z.string().optional(),
algorithm: z.nativeEnum(V1WebhookHMACAlgorithm).optional(),
encoding: z.nativeEnum(V1WebhookHMACEncoding).optional(),
signatureHeaderName: z.string().optional(),
});
export type WebhookFormData = z.infer<typeof webhookFormSchema>;
frontend/app/src/pages/main/v1/webhooks/index.tsx
+401
View File
@@ -0,0 +1,401 @@
import { columns } from './components/webhook-columns';
import { DataTable } from '@/components/v1/molecules/data-table/data-table';
import {
useWebhooks,
WebhookFormData,
webhookFormSchema,
} from './hooks/use-webhooks';
import { Button } from '@/components/v1/ui/button';
import {
Dialog,
DialogClose,
DialogContent,
DialogHeader,
DialogTitle,
DialogTrigger,
} from '@/components/v1/ui/dialog';
import { Input } from '@/components/v1/ui/input';
import { Label } from '@/components/v1/ui/label';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/v1/ui/select';
import { useCallback, useState } from 'react';
import { useForm } from 'react-hook-form';
import { zodResolver } from '@hookform/resolvers/zod';
import {
V1WebhookSourceName,
V1WebhookAuthType,
V1CreateWebhookRequest,
V1WebhookHMACAlgorithm,
V1WebhookHMACEncoding,
} from '@/lib/api';
import { Webhook, Copy, Check } from 'lucide-react';
import { Spinner } from '@/components/v1/ui/loading';
import { SourceName } from './components/source-name';
import { AuthMethod } from './components/auth-method';
import { AuthSetup } from './components/auth-setup';
export default function Webhooks() {
const { data, isLoading, error } = useWebhooks();
return (
<div>
<div className="flex flex-row justify-end w-full">
<CreateWebhookModal />
</div>
<DataTable
error={error}
isLoading={isLoading}
columns={columns()}
data={data}
filters={[]}
/>
</div>
);
}
const buildWebhookPayload = (data: WebhookFormData): V1CreateWebhookRequest => {
switch (data.sourceName) {
case V1WebhookSourceName.GENERIC:
switch (data.authType) {
case V1WebhookAuthType.BASIC:
if (!data.username || !data.password) {
throw new Error(
'Username and password are required for basic auth',
);
}
return {
sourceName: data.sourceName,
name: data.name,
eventKeyExpression: data.eventKeyExpression,
authType: data.authType,
auth: {
username: data.username,
password: data.password,
},
};
case V1WebhookAuthType.API_KEY:
if (!data.headerName || !data.apiKey) {
throw new Error(
'Header name and API key are required for API key auth',
);
}
return {
sourceName: data.sourceName,
name: data.name,
eventKeyExpression: data.eventKeyExpression,
authType: data.authType,
auth: {
headerName: data.headerName,
apiKey: data.apiKey,
},
};
case V1WebhookAuthType.HMAC:
if (
!data.algorithm ||
!data.encoding ||
!data.signatureHeaderName ||
!data.signingSecret
) {
throw new Error(
'Algorithm, encoding, signature header name, and signing secret are required for HMAC auth',
);
}
return {
sourceName: data.sourceName,
name: data.name,
eventKeyExpression: data.eventKeyExpression,
authType: data.authType,
auth: {
algorithm: data.algorithm,
encoding: data.encoding,
signatureHeaderName: data.signatureHeaderName,
signingSecret: data.signingSecret,
},
};
default:
// eslint-disable-next-line no-case-declarations
const exhaustiveCheck: never = data.authType;
throw new Error(`Unhandled auth type: ${exhaustiveCheck}`);
}
case V1WebhookSourceName.GITHUB:
if (!data.signingSecret) {
throw new Error('Signing secret is required for GitHub webhooks');
}
return {
sourceName: data.sourceName,
name: data.name,
eventKeyExpression: data.eventKeyExpression,
authType: V1WebhookAuthType.HMAC,
auth: {
// Header name is 'X-Hub-Signature-256'
// Encoding algorithm is SHA256
// Encoding type is HEX
// See GitHub docs: https://docs.github.com/en/webhooks/using-webhooks/validating-webhook-deliveries#validating-webhook-deliveries
algorithm: V1WebhookHMACAlgorithm.SHA256,
encoding: V1WebhookHMACEncoding.HEX,
signatureHeaderName: 'X-Hub-Signature-256',
signingSecret: data.signingSecret,
},
};
case V1WebhookSourceName.STRIPE:
if (!data.signingSecret) {
throw new Error('Signing secret is required for GitHub webhooks');
}
return {
sourceName: data.sourceName,
name: data.name,
eventKeyExpression: data.eventKeyExpression,
authType: V1WebhookAuthType.HMAC,
auth: {
// Header name is 'Stripe-Signature'
// Encoding algorithm is SHA256
// Encoding type is HEX
// See Stripe docs: https://docs.stripe.com/webhooks?verify=verify-manually#verify-manually
algorithm: V1WebhookHMACAlgorithm.SHA256,
encoding: V1WebhookHMACEncoding.HEX,
signatureHeaderName: 'Stripe-Signature',
signingSecret: data.signingSecret,
},
};
default:
// eslint-disable-next-line no-case-declarations
const exhaustiveCheck: never = data.sourceName;
throw new Error(`Unhandled source name: ${exhaustiveCheck}`);
}
};
const CreateWebhookModal = () => {
const { mutations, createWebhookURL } = useWebhooks();
const { createWebhook, isCreatePending } = mutations;
const [open, setOpen] = useState(false);
const [copied, setCopied] = useState(false);
const {
register,
handleSubmit,
watch,
setValue,
reset,
formState: { errors },
} = useForm<WebhookFormData>({
resolver: zodResolver(webhookFormSchema),
defaultValues: {
sourceName: V1WebhookSourceName.GENERIC,
authType: V1WebhookAuthType.BASIC,
name: '',
eventKeyExpression: 'input.id',
username: '',
password: '',
},
});
const sourceName = watch('sourceName');
const authType = watch('authType');
const webhookName = watch('name');
const copyToClipboard = useCallback(async () => {
if (webhookName) {
try {
await navigator.clipboard.writeText(createWebhookURL(webhookName));
setCopied(true);
setTimeout(() => setCopied(false), 2000);
} catch (err) {
console.error('Failed to copy URL:', err);
}
}
}, [webhookName, createWebhookURL]);
const onSubmit = useCallback(
(data: WebhookFormData) => {
const payload = buildWebhookPayload(data);
createWebhook(payload, {
onSuccess: () => {
setOpen(false);
reset();
},
});
},
[createWebhook, reset],
);
return (
<Dialog
open={open}
onOpenChange={(isOpen) => {
setOpen(isOpen);
if (!isOpen) {
reset();
setCopied(false);
}
}}
>
<DialogTrigger asChild>
<Button variant="default">Create Webhook</Button>
</DialogTrigger>
<DialogContent className="max-w-[90%] md:max-w-[80%] lg:max-w-[60%] xl:max-w-[50%] max-h-[90dvh] overflow-y-auto">
<DialogHeader>
<DialogTitle className="flex items-center gap-2">
<div className="flex items-center justify-center w-8 h-8 bg-blue-100 rounded-full">
<Webhook className="h-4 w-4 text-blue-600" />
</div>
Create a webhook
</DialogTitle>
</DialogHeader>
<form onSubmit={handleSubmit(onSubmit)} className="space-y-6">
<div className="space-y-2">
<Label htmlFor="name" className="text-sm font-medium">
Webhook ID <span className="text-red-500">*</span>
</Label>
<Input
data-1p-ignore
id="name"
placeholder="test-webhook"
{...register('name')}
className="h-10"
/>
{errors.name && (
<p className="text-xs text-red-500">{errors.name.message}</p>
)}
<div className="flex flex-col items-start gap-2 text-xs text-muted-foreground">
<span className="">Send incoming webhook requests to:</span>
<div className="flex flex-row items-center gap-2">
<code className="max-w-full font-mono bg-muted px-2 py-1 rounded text-xs">
{createWebhookURL(webhookName)}
</code>
<Button
type="button"
variant="ghost"
size="sm"
onClick={copyToClipboard}
className="h-6 w-6 p-0 flex-shrink-0"
disabled={!webhookName}
>
{copied ? (
<Check className="size-4 text-green-600" />
) : (
<Copy className="size-4" />
)}
</Button>
</div>
</div>
</div>
<div className="space-y-2">
<Label htmlFor="sourceName" className="text-sm font-medium">
Source <span className="text-red-500">*</span>
</Label>
<Select
value={sourceName}
onValueChange={(value: V1WebhookSourceName) =>
setValue('sourceName', value)
}
>
<SelectTrigger className="h-10">
<SelectValue>
<SourceName sourceName={sourceName} />
</SelectValue>
</SelectTrigger>
<SelectContent>
{Object.values(V1WebhookSourceName).map((source) => (
<SelectItem key={source} value={source}>
<SourceName sourceName={source} />
</SelectItem>
))}
</SelectContent>
</Select>
<p className="text-xs text-muted-foreground">
Represents the producer of your HTTP requests.
</p>
</div>
<div className="space-y-2">
<Label htmlFor="eventKeyExpression" className="text-sm font-medium">
Event Key Expression <span className="text-red-500">*</span>
</Label>
<Input
id="eventKeyExpression"
placeholder="input.id"
{...register('eventKeyExpression')}
className="h-10"
/>
{errors.eventKeyExpression && (
<p className="text-xs text-red-500">
{errors.eventKeyExpression.message}
</p>
)}
<p className="text-xs text-muted-foreground">
CEL expression to extract the event key from the webhook payload.
Use `input` to refer to the payload.
</p>
</div>
<div className="space-y-4">
<div className="space-y-4 pl-4 border-l-2 border-gray-200">
{sourceName === V1WebhookSourceName.GENERIC && (
<div className="space-y-2">
<Label htmlFor="authType" className="text-sm font-medium">
Authentication Type
</Label>
<Select
value={authType}
onValueChange={(value: V1WebhookAuthType) =>
setValue('authType', value)
}
>
<SelectTrigger className="h-10">
<SelectValue />
</SelectTrigger>
<SelectContent>
<SelectItem value={V1WebhookAuthType.BASIC}>
<AuthMethod authMethod={V1WebhookAuthType.BASIC} />
</SelectItem>
<SelectItem value={V1WebhookAuthType.API_KEY}>
<AuthMethod authMethod={V1WebhookAuthType.API_KEY} />
</SelectItem>
<SelectItem value={V1WebhookAuthType.HMAC}>
<AuthMethod authMethod={V1WebhookAuthType.HMAC} />
</SelectItem>
</SelectContent>
</Select>
</div>
)}
<AuthSetup
authMethod={authType}
sourceName={sourceName}
register={register}
watch={watch}
setValue={setValue}
/>
</div>
</div>
<div className="flex justify-end gap-3 pt-4">
<DialogClose asChild>
<Button type="button" variant="outline">
Cancel
</Button>
</DialogClose>
<Button type="submit" disabled={isCreatePending}>
{isCreatePending && <Spinner />}
Create Webhook
</Button>
</div>
</form>
</DialogContent>
</Dialog>
);
};
frontend/app/src/router.tsx
+11
View File
@@ -10,6 +10,7 @@ import Root from './pages/root.tsx';
export const tenantedPaths = [
'/tenants/:tenant/events',
'/tenants/:tenant/webhooks',
'/tenants/:tenant/rate-limits',
'/tenants/:tenant/scheduled',
'/tenants/:tenant/cron-jobs',
@@ -56,6 +57,16 @@ const createTenantedRoute = (path: TenantedPath): RouteObject => {
};
}),
};
case '/tenants/:tenant/webhooks':
return {
path,
lazy: async () =>
import('./pages/main/v1/webhooks').then((res) => {
return {
Component: res.default,
};
}),
};
case '/tenants/:tenant/rate-limits':
return {
path,
frontend/docs/lib/generated/snips/python/index.ts
+2
View File
@@ -43,6 +43,7 @@ import * as sticky_workers from './sticky_workers';
import * as streaming from './streaming';
import * as timeout from './timeout';
import * as unit_testing from './unit_testing';
import * as webhooks from './webhooks';
import * as worker_existing_loop from './worker_existing_loop';
import * as workflow_registration from './workflow_registration';
@@ -91,5 +92,6 @@ export { sticky_workers };
export { streaming };
export { timeout };
export { unit_testing };
export { webhooks };
export { worker_existing_loop };
export { workflow_registration };
frontend/docs/lib/generated/snips/python/webhooks/index.ts
+5
View File
@@ -0,0 +1,5 @@
import test_webhooks from './test_webhooks';
import worker from './worker';
export { test_webhooks }
export { worker }
frontend/docs/lib/generated/snips/python/webhooks/test_webhooks.ts
+11
View File
File diff suppressed because one or more lines are too long
frontend/docs/lib/generated/snips/python/webhooks/worker.ts
+16
View File
@@ -0,0 +1,16 @@
import { Snippet } from '@/lib/generated/snips/types';
const snippet: Snippet = {
"language": "python",
"content": "# > Webhooks\n\nfrom pydantic import BaseModel\n\nfrom hatchet_sdk import Context, Hatchet\n\nhatchet = Hatchet(debug=True)\n\n\nclass WebhookInput(BaseModel):\n type: str\n message: str\n\n\n@hatchet.task(input_validator=WebhookInput, on_events=[\"webhook:test\"])\ndef webhook(input: WebhookInput, ctx: Context) -> dict[str, str]:\n return input.model_dump()\n\n\ndef main() -> None:\n worker = hatchet.worker(\"webhook-worker\", workflows=[webhook])\n worker.start()\n\n\n\nif __name__ == \"__main__\":\n main()\n",
"source": "out/python/webhooks/worker.py",
"blocks": {
"webhooks": {
"start": 2,
"stop": 24
}
},
"highlights": {}
};
export default snippet;
frontend/docs/lib/generated/snips/python/worker.ts
+1 -1
View File
@@ -2,7 +2,7 @@ import { Snippet } from '@/lib/generated/snips/types';
const snippet: Snippet = {
"language": "python",
"content": "from examples.affinity_workers.worker import affinity_worker_workflow\nfrom examples.bulk_fanout.worker import bulk_child_wf, bulk_parent_wf\nfrom examples.bulk_operations.worker import (\n bulk_replay_test_1,\n bulk_replay_test_2,\n bulk_replay_test_3,\n)\nfrom examples.cancellation.worker import cancellation_workflow\nfrom examples.concurrency_limit.worker import concurrency_limit_workflow\nfrom examples.concurrency_limit_rr.worker import concurrency_limit_rr_workflow\nfrom examples.concurrency_multiple_keys.worker import concurrency_multiple_keys_workflow\nfrom examples.concurrency_workflow_level.worker import (\n concurrency_workflow_level_workflow,\n)\nfrom examples.conditions.worker import task_condition_workflow\nfrom examples.dag.worker import dag_workflow\nfrom examples.dedupe.worker import dedupe_child_wf, dedupe_parent_wf\nfrom examples.durable.worker import durable_workflow, wait_for_sleep_twice\nfrom examples.events.worker import event_workflow\nfrom examples.fanout.worker import child_wf, parent_wf\nfrom examples.fanout_sync.worker import sync_fanout_child, sync_fanout_parent\nfrom examples.lifespans.simple import lifespan, lifespan_task\nfrom examples.logger.workflow import logging_workflow\nfrom examples.non_retryable.worker import non_retryable_workflow\nfrom examples.on_failure.worker import on_failure_wf, on_failure_wf_with_details\nfrom examples.return_exceptions.worker import return_exceptions_task\nfrom examples.simple.worker import simple, simple_durable\nfrom examples.timeout.worker import refresh_timeout_wf, timeout_wf\nfrom hatchet_sdk import Hatchet\n\nhatchet = Hatchet(debug=True)\n\n\ndef main() -> None:\n worker = hatchet.worker(\n \"e2e-test-worker\",\n slots=100,\n workflows=[\n affinity_worker_workflow,\n bulk_child_wf,\n bulk_parent_wf,\n concurrency_limit_workflow,\n concurrency_limit_rr_workflow,\n concurrency_multiple_keys_workflow,\n dag_workflow,\n dedupe_child_wf,\n dedupe_parent_wf,\n durable_workflow,\n child_wf,\n event_workflow,\n parent_wf,\n on_failure_wf,\n on_failure_wf_with_details,\n logging_workflow,\n timeout_wf,\n refresh_timeout_wf,\n task_condition_workflow,\n cancellation_workflow,\n sync_fanout_parent,\n sync_fanout_child,\n non_retryable_workflow,\n concurrency_workflow_level_workflow,\n lifespan_task,\n simple,\n simple_durable,\n bulk_replay_test_1,\n bulk_replay_test_2,\n bulk_replay_test_3,\n return_exceptions_task,\n wait_for_sleep_twice,\n ],\n lifespan=lifespan,\n )\n\n worker.start()\n\n\nif __name__ == \"__main__\":\n main()\n",
"content": "from examples.affinity_workers.worker import affinity_worker_workflow\nfrom examples.bulk_fanout.worker import bulk_child_wf, bulk_parent_wf\nfrom examples.bulk_operations.worker import (\n bulk_replay_test_1,\n bulk_replay_test_2,\n bulk_replay_test_3,\n)\nfrom examples.cancellation.worker import cancellation_workflow\nfrom examples.concurrency_limit.worker import concurrency_limit_workflow\nfrom examples.concurrency_limit_rr.worker import concurrency_limit_rr_workflow\nfrom examples.concurrency_multiple_keys.worker import concurrency_multiple_keys_workflow\nfrom examples.concurrency_workflow_level.worker import (\n concurrency_workflow_level_workflow,\n)\nfrom examples.conditions.worker import task_condition_workflow\nfrom examples.dag.worker import dag_workflow\nfrom examples.dedupe.worker import dedupe_child_wf, dedupe_parent_wf\nfrom examples.durable.worker import durable_workflow\nfrom examples.events.worker import event_workflow\nfrom examples.fanout.worker import child_wf, parent_wf\nfrom examples.fanout_sync.worker import sync_fanout_child, sync_fanout_parent\nfrom examples.lifespans.simple import lifespan, lifespan_task\nfrom examples.logger.workflow import logging_workflow\nfrom examples.non_retryable.worker import non_retryable_workflow\nfrom examples.on_failure.worker import on_failure_wf, on_failure_wf_with_details\nfrom examples.return_exceptions.worker import return_exceptions_task\nfrom examples.simple.worker import simple, simple_durable\nfrom examples.timeout.worker import refresh_timeout_wf, timeout_wf\nfrom examples.webhooks.worker import webhook\nfrom hatchet_sdk import Hatchet\n\nhatchet = Hatchet(debug=True)\n\n\ndef main() -> None:\n worker = hatchet.worker(\n \"e2e-test-worker\",\n slots=100,\n workflows=[\n affinity_worker_workflow,\n bulk_child_wf,\n bulk_parent_wf,\n concurrency_limit_workflow,\n concurrency_limit_rr_workflow,\n concurrency_multiple_keys_workflow,\n dag_workflow,\n dedupe_child_wf,\n dedupe_parent_wf,\n durable_workflow,\n child_wf,\n event_workflow,\n parent_wf,\n on_failure_wf,\n on_failure_wf_with_details,\n logging_workflow,\n timeout_wf,\n refresh_timeout_wf,\n task_condition_workflow,\n cancellation_workflow,\n sync_fanout_parent,\n sync_fanout_child,\n non_retryable_workflow,\n concurrency_workflow_level_workflow,\n lifespan_task,\n simple,\n simple_durable,\n bulk_replay_test_1,\n bulk_replay_test_2,\n bulk_replay_test_3,\n webhook,\n return_exceptions_task,\n ],\n lifespan=lifespan,\n )\n\n worker.start()\n\n\nif __name__ == \"__main__\":\n main()\n",
"source": "out/python/worker.py",
"blocks": {},
"highlights": {}
frontend/docs/pages/home/webhooks.mdx
+58
View File
@@ -0,0 +1,58 @@
import snips from "@/lib/snips";
import { Snippet } from "@/components/code";
import { Callout, Card, Cards, Steps, Tabs } from "nextra/components";
import UniversalTabs from "@/components/UniversalTabs";
# Webhooks
Webhooks allow external systems to trigger Hatchet workflows by sending HTTP requests to dedicated endpoints. This enables real-time integration with third-party services like GitHub, Stripe, or any system that can send webhook events.
## Creating a webhook
To create a webhook, you'll need to fill out some fields that tell Hatchet how to determine which workflows to trigger from your webhook, and how to validate it when it arrives from the sender. In particular, you'll need to provide the following fields:
#### Name
The **Webhook Name** is tenant-unique (meaning a single tenant can only use each name once), and is used to create the URL for where the incoming webhook request should be sent. For instance, if your tenant id was `d60181b7-da6c-4d4c-92ec-8aa0fc74b3e5` and your webhook name was `my-webhook`, then the URL might look like `https://cloud.onhatchet.run/api/v1/stable/tenants/d60181b7-da6c-4d4c-92ec-8aa0fc74b3e5/webhooks/my-webhook`. Note that you can copy this URL in the dashboard.
#### Source
The **Source** indicates the source of the webhook, which can be a pre-provided one for easy setup like Stripe or Github, or a "generic" one, which lets you configure all of the necessary fields for your webhook integration based on what the webhook sender provides.
#### Event Key Expression
The **Event Key Expression** is a [CEL](https://cel.dev/) expression that you can use to create a dynamic event key from the payload of the incoming webhook. You can either set this to a constant value, like `webhook`, or you could set it to something like `'stripe:' + input.type`, where `'stripe:'` is a prefix for all keys indicating the webhook came from Stripe, and `input.type` selects the `type` field off of the webhook payload and uses it to create the final event key, which would look something like `stripe:payment_intent.created`.
<Callout type="info">
The result of the event key expression is what Hatchet will use as the event
key, so you'd need to set a matching event key as a trigger on your workflows
in order to trigger them from the webhooks you create. For instance, you might
add `on_events=["stripe:payment_intent.created"]` to listen for payment intent
created events in the previous example.
</Callout>
#### Authentication
Finally, you'll need to specify how Hatchet should authenticate incoming webhook requests. For non-generic sources like Stripe and Github, Hatchet has presets for most of the fields, so in most cases you'd only need to provide a secret.
If you're using a generic source, then you'll need to specify an authentication method (either basic auth, an API key, HMAC-based auth), and provide the required fields (such as a username and password in the basic auth case).
<Callout type="warning">
Hatchet encrypts any secrets you provide for validating incoming webhooks.
</Callout>
The different authentication methods require different fields to be provided:
- **Pre-configured sources** (Stripe, GitHub): Only require a webhook secret
- **Generic sources** require different fields depending on the selected authentication method:
- **Basic Auth**: Requires a username and password
- **API Key**: Requires header name containing the key on incoming requests, and secret key itself
- **HMAC**: Requires a header name containing the secret on incoming requests, the secret itself, an encoding method (e.g. hex, base64), and an algorithm (e.g. `SHA256`, `SHA1`, etc.).
## Usage
While you're creating your webhook (and also after you've created it), you can copy the webhook URL, which is what you'll provide to the webhook _sender_.
Once you've done that, the last thing to do is register the event keys you want your workers to listen for so that they can be triggered by incoming webhooks.
For examples on how to do this, see the [documentation on event triggers](./run-on-event.mdx).
internal/cel/cel.go
+41 -6
View File
@@ -17,9 +17,10 @@ import (
)
type CELParser struct {
workflowStrEnv *cel.Env
stepRunEnv *cel.Env
eventEnv *cel.Env
workflowStrEnv *cel.Env
stepRunEnv *cel.Env
eventEnv *cel.Env
incomingWebhookEnv *cel.Env
}
var checksumDecl = decls.NewFunction("checksum",
@@ -83,10 +84,18 @@ func NewCELParser() *CELParser {
ext.Strings(),
)
incomingWebhookEnv, _ := cel.NewEnv(
cel.Declarations(
decls.NewVar("input", decls.NewMapType(decls.String, decls.Dyn)),
checksumDecl,
),
)
return &CELParser{
workflowStrEnv: workflowStrEnv,
stepRunEnv: stepRunEnv,
eventEnv: eventEnv,
workflowStrEnv: workflowStrEnv,
stepRunEnv: stepRunEnv,
eventEnv: eventEnv,
incomingWebhookEnv: incomingWebhookEnv,
}
}
@@ -355,3 +364,29 @@ func (p *CELParser) EvaluateEventExpression(expr string, input Input) (bool, err
return out.Value().(bool), nil
}
func (p *CELParser) EvaluateIncomingWebhookExpression(expr string, input Input) (string, error) {
ast, issues := p.eventEnv.Compile(expr)
if issues != nil && issues.Err() != nil {
return "", fmt.Errorf("failed to compile expression: %w", issues.Err())
}
program, err := p.eventEnv.Program(ast)
if err != nil {
return "", fmt.Errorf("failed to create program: %w", err)
}
var inMap map[string]interface{} = input
out, _, err := program.Eval(inMap)
if err != nil {
return "", fmt.Errorf("failed to evaluate expression: %w", err)
}
if out.Type() != types.StringType {
return "", fmt.Errorf("expression did not evaluate to a string: got %s", out.Type().TypeName())
}
return out.Value().(string), nil
}
internal/services/controllers/v1/olap/controller.go
+67 -9
View File
@@ -337,11 +337,34 @@ func (tc *OLAPControllerImpl) handleBufferedMsgs(tenantId, msgId string, payload
return tc.handleCreateMonitoringEvent(context.Background(), tenantId, payloads)
case "created-event-trigger":
return tc.handleCreateEventTriggers(context.Background(), tenantId, payloads)
case "failed-webhook-validation":
return tc.handleFailedWebhookValidation(context.Background(), tenantId, payloads)
case "cel-evaluation-failure":
return tc.handleCelEvaluationFailure(context.Background(), tenantId, payloads)
}
return fmt.Errorf("unknown message id: %s", msgId)
}
func (tc *OLAPControllerImpl) handleCelEvaluationFailure(ctx context.Context, tenantId string, payloads [][]byte) error {
failures := make([]v1.CELEvaluationFailure, 0)
msgs := msgqueue.JSONConvert[tasktypes.CELEvaluationFailures](payloads)
for _, msg := range msgs {
for _, failure := range msg.Failures {
if !tc.sample(failure.ErrorMessage) {
tc.l.Debug().Msgf("skipping CEL evaluation failure %s for source %s", failure.ErrorMessage, failure.Source)
continue
}
failures = append(failures, failure)
}
}
return tc.repo.OLAP().StoreCELEvaluationFailures(ctx, tenantId, failures)
}
// handleCreatedTask is responsible for flushing a created task to the OLAP repository
func (tc *OLAPControllerImpl) handleCreatedTask(ctx context.Context, tenantId string, payloads [][]byte) error {
createTaskOpts := make([]*sqlcv1.V1Task, 0)
@@ -390,7 +413,8 @@ func (tc *OLAPControllerImpl) handleCreateEventTriggers(ctx context.Context, ten
keys := make([]string, 0)
payloadstoInsert := make([][]byte, 0)
additionalMetadatas := make([][]byte, 0)
scopes := make([]*string, 0)
scopes := make([]pgtype.Text, 0)
triggeringWebhookNames := make([]pgtype.Text, 0)
for _, msg := range msgs {
for _, payload := range msg.Payloads {
@@ -423,18 +447,32 @@ func (tc *OLAPControllerImpl) handleCreateEventTriggers(ctx context.Context, ten
keys = append(keys, payload.EventKey)
payloadstoInsert = append(payloadstoInsert, payload.EventPayload)
additionalMetadatas = append(additionalMetadatas, payload.EventAdditionalMetadata)
scopes = append(scopes, payload.EventScope)
var scope pgtype.Text
if payload.EventScope != nil {
scope = sqlchelpers.TextFromStr(*payload.EventScope)
}
scopes = append(scopes, scope)
var triggeringWebhookName pgtype.Text
if payload.TriggeringWebhookName != nil {
triggeringWebhookName = sqlchelpers.TextFromStr(*payload.TriggeringWebhookName)
}
triggeringWebhookNames = append(triggeringWebhookNames, triggeringWebhookName)
}
}
bulkCreateEventParams := sqlcv1.BulkCreateEventsParams{
Tenantids: tenantIds,
Externalids: externalIds,
Seenats: seenAts,
Keys: keys,
Payloads: payloadstoInsert,
Additionalmetadatas: additionalMetadatas,
Scopes: scopes,
Tenantids: tenantIds,
Externalids: externalIds,
Seenats: seenAts,
Keys: keys,
Payloads: payloadstoInsert,
Additionalmetadatas: additionalMetadatas,
Scopes: scopes,
TriggeringWebhookNames: triggeringWebhookNames,
}
return tc.repo.OLAP().BulkCreateEventsAndTriggers(
@@ -588,6 +626,26 @@ func (tc *OLAPControllerImpl) handleCreateMonitoringEvent(ctx context.Context, t
return tc.repo.OLAP().CreateTaskEvents(ctx, tenantId, opts)
}
func (tc *OLAPControllerImpl) handleFailedWebhookValidation(ctx context.Context, tenantId string, payloads [][]byte) error {
createFailedWebhookValidationOpts := make([]v1.CreateIncomingWebhookFailureLogOpts, 0)
msgs := msgqueue.JSONConvert[tasktypes.FailedWebhookValidationPayload](payloads)
for _, msg := range msgs {
if !tc.sample(msg.ErrorText) {
tc.l.Debug().Msgf("skipping failure logging for webhook %s", msg.WebhookName)
continue
}
createFailedWebhookValidationOpts = append(createFailedWebhookValidationOpts, v1.CreateIncomingWebhookFailureLogOpts{
WebhookName: msg.WebhookName,
ErrorText: msg.ErrorText,
})
}
return tc.repo.OLAP().CreateIncomingWebhookValidationFailureLogs(ctx, tenantId, createFailedWebhookValidationOpts)
}
func (tc *OLAPControllerImpl) sample(workflowRunID string) bool {
if tc.samplingHashThreshold == nil {
return true
internal/services/controllers/v1/task/controller.go
+24 -6
View File
@@ -911,12 +911,13 @@ func (tc *TasksControllerImpl) handleProcessUserEventTrigger(ctx context.Context
for _, msg := range msgs {
opt := v1.EventTriggerOpts{
ExternalId: msg.EventExternalId,
Key: msg.EventKey,
Data: msg.EventData,
AdditionalMetadata: msg.EventAdditionalMetadata,
Priority: msg.EventPriority,
Scope: msg.EventScope,
ExternalId: msg.EventExternalId,
Key: msg.EventKey,
Data: msg.EventData,
AdditionalMetadata: msg.EventAdditionalMetadata,
Priority: msg.EventPriority,
Scope: msg.EventScope,
TriggeringWebhookName: msg.TriggeringWebhookName,
}
opts = append(opts, opt)
@@ -945,6 +946,7 @@ func (tc *TasksControllerImpl) handleProcessUserEventTrigger(ctx context.Context
EventExternalId: opts.ExternalId,
EventPayload: opts.Data,
EventAdditionalMetadata: opts.AdditionalMetadata,
TriggeringWebhookName: opts.TriggeringWebhookName,
})
} else {
for _, run := range runs {
@@ -958,6 +960,7 @@ func (tc *TasksControllerImpl) handleProcessUserEventTrigger(ctx context.Context
EventAdditionalMetadata: opts.AdditionalMetadata,
EventScope: opts.Scope,
FilterId: run.FilterId,
TriggeringWebhookName: opts.TriggeringWebhookName,
})
}
}
@@ -980,6 +983,21 @@ func (tc *TasksControllerImpl) handleProcessUserEventTrigger(ctx context.Context
return fmt.Errorf("could not trigger tasks from events: %w", err)
}
evalFailuresMsg, err := tasktypes.CELEvaluationFailureMessage(
tenantId,
result.CELEvaluationFailures,
)
if err != nil {
return fmt.Errorf("could not create CEL evaluation failure message: %w", err)
}
err = tc.pubBuffer.Pub(ctx, msgqueue.OLAP_QUEUE, evalFailuresMsg, false)
if err != nil {
return fmt.Errorf("could not deliver CEL evaluation failure message: %w", err)
}
eg := &errgroup.Group{}
eg.Go(func() error {
internal/services/ingestor/ingestor.go
+19 -3
View File
@@ -17,14 +17,17 @@ import (
"github.com/hatchet-dev/hatchet/pkg/repository/postgres/dbsqlc"
"github.com/hatchet-dev/hatchet/pkg/repository/postgres/sqlchelpers"
v1 "github.com/hatchet-dev/hatchet/pkg/repository/v1"
"github.com/hatchet-dev/hatchet/pkg/repository/v1/sqlcv1"
"github.com/hatchet-dev/hatchet/pkg/validator"
)
type Ingestor interface {
contracts.EventsServiceServer
IngestEvent(ctx context.Context, tenant *dbsqlc.Tenant, eventName string, data []byte, metadata []byte, priority *int32, scope *string) (*dbsqlc.Event, error)
IngestEvent(ctx context.Context, tenant *dbsqlc.Tenant, eventName string, data []byte, metadata []byte, priority *int32, scope, triggeringWebhookName *string) (*dbsqlc.Event, error)
IngestWebhookValidationFailure(ctx context.Context, tenant *dbsqlc.Tenant, webhookName, errorText string) error
BulkIngestEvent(ctx context.Context, tenant *dbsqlc.Tenant, eventOpts []*repository.CreateEventOpts) ([]*dbsqlc.Event, error)
IngestReplayedEvent(ctx context.Context, tenant *dbsqlc.Tenant, replayedEvent *dbsqlc.Event) (*dbsqlc.Event, error)
IngestCELEvaluationFailure(ctx context.Context, tenantId, errorText string, source sqlcv1.V1CelEvaluationFailureSource) error
}
type IngestorOptFunc func(*IngestorOpts)
@@ -175,17 +178,21 @@ func NewIngestor(fs ...IngestorOptFunc) (Ingestor, error) {
}, nil
}
func (i *IngestorImpl) IngestEvent(ctx context.Context, tenant *dbsqlc.Tenant, key string, data []byte, metadata []byte, priority *int32, scope *string) (*dbsqlc.Event, error) {
func (i *IngestorImpl) IngestEvent(ctx context.Context, tenant *dbsqlc.Tenant, key string, data []byte, metadata []byte, priority *int32, scope, triggeringWebhookName *string) (*dbsqlc.Event, error) {
switch tenant.Version {
case dbsqlc.TenantMajorEngineVersionV0:
return i.ingestEventV0(ctx, tenant, key, data, metadata)
case dbsqlc.TenantMajorEngineVersionV1:
return i.ingestEventV1(ctx, tenant, key, data, metadata, priority, scope)
return i.ingestEventV1(ctx, tenant, key, data, metadata, priority, scope, triggeringWebhookName)
default:
return nil, fmt.Errorf("unsupported tenant version: %s", tenant.Version)
}
}
func (i *IngestorImpl) IngestWebhookValidationFailure(ctx context.Context, tenant *dbsqlc.Tenant, webhookName, errorText string) error {
return i.ingestWebhookValidationFailure(tenant.ID.String(), webhookName, errorText)
}
func (i *IngestorImpl) ingestEventV0(ctx context.Context, tenant *dbsqlc.Tenant, key string, data []byte, metadata []byte) (*dbsqlc.Event, error) {
ctx, span := telemetry.NewSpan(ctx, "ingest-event")
defer span.End()
@@ -339,3 +346,12 @@ func eventToTask(e *dbsqlc.Event) *msgqueue.Message {
Retries: 3,
}
}
func (i *IngestorImpl) IngestCELEvaluationFailure(ctx context.Context, tenantId, errorText string, source sqlcv1.V1CelEvaluationFailureSource) error {
return i.ingestCELEvaluationFailure(
ctx,
tenantId,
errorText,
source,
)
}
internal/services/ingestor/ingestor_v1.go
+69 -6
View File
@@ -15,6 +15,8 @@ import (
"github.com/hatchet-dev/hatchet/pkg/repository"
"github.com/hatchet-dev/hatchet/pkg/repository/postgres/dbsqlc"
"github.com/hatchet-dev/hatchet/pkg/repository/postgres/sqlchelpers"
v1 "github.com/hatchet-dev/hatchet/pkg/repository/v1"
"github.com/hatchet-dev/hatchet/pkg/repository/v1/sqlcv1"
)
type EventResult struct {
@@ -25,7 +27,7 @@ type EventResult struct {
AdditionalMetadata string
}
func (i *IngestorImpl) ingestEventV1(ctx context.Context, tenant *dbsqlc.Tenant, key string, data []byte, metadata []byte, priority *int32, scope *string) (*dbsqlc.Event, error) {
func (i *IngestorImpl) ingestEventV1(ctx context.Context, tenant *dbsqlc.Tenant, key string, data []byte, metadata []byte, priority *int32, scope, triggeringWebhookName *string) (*dbsqlc.Event, error) {
ctx, span := telemetry.NewSpan(ctx, "ingest-event")
defer span.End()
@@ -49,10 +51,10 @@ func (i *IngestorImpl) ingestEventV1(ctx context.Context, tenant *dbsqlc.Tenant,
)
}
return i.ingestSingleton(tenantId, key, data, metadata, priority, scope)
return i.ingestSingleton(tenantId, key, data, metadata, priority, scope, triggeringWebhookName)
}
func (i *IngestorImpl) ingestSingleton(tenantId, key string, data []byte, metadata []byte, priority *int32, scope *string) (*dbsqlc.Event, error) {
func (i *IngestorImpl) ingestSingleton(tenantId, key string, data []byte, metadata []byte, priority *int32, scope, triggeringWebhookName *string) (*dbsqlc.Event, error) {
eventId := uuid.New().String()
msg, err := eventToTaskV1(
@@ -63,6 +65,7 @@ func (i *IngestorImpl) ingestSingleton(tenantId, key string, data []byte, metada
metadata,
priority,
scope,
triggeringWebhookName,
)
if err != nil {
@@ -117,7 +120,7 @@ func (i *IngestorImpl) bulkIngestEventV1(ctx context.Context, tenant *dbsqlc.Ten
results := make([]*dbsqlc.Event, 0, len(eventOpts))
for _, event := range eventOpts {
res, err := i.ingestSingleton(tenantId, event.Key, event.Data, event.AdditionalMetadata, event.Priority, event.Scope)
res, err := i.ingestSingleton(tenantId, event.Key, event.Data, event.AdditionalMetadata, event.Priority, event.Scope, event.TriggeringWebhookName)
if err != nil {
return nil, fmt.Errorf("could not ingest event: %w", err)
@@ -135,10 +138,10 @@ func (i *IngestorImpl) ingestReplayedEventV1(ctx context.Context, tenant *dbsqlc
tenantId := sqlchelpers.UUIDToStr(tenant.ID)
return i.ingestSingleton(tenantId, replayedEvent.Key, replayedEvent.Data, replayedEvent.AdditionalMetadata, nil, nil)
return i.ingestSingleton(tenantId, replayedEvent.Key, replayedEvent.Data, replayedEvent.AdditionalMetadata, nil, nil, nil)
}
func eventToTaskV1(tenantId, eventExternalId, key string, data, additionalMeta []byte, priority *int32, scope *string) (*msgqueue.Message, error) {
func eventToTaskV1(tenantId, eventExternalId, key string, data, additionalMeta []byte, priority *int32, scope *string, triggeringWebhookName *string) (*msgqueue.Message, error) {
payloadTyped := tasktypes.UserEventTaskPayload{
EventExternalId: eventExternalId,
EventKey: key,
@@ -146,6 +149,7 @@ func eventToTaskV1(tenantId, eventExternalId, key string, data, additionalMeta [
EventAdditionalMetadata: additionalMeta,
EventPriority: priority,
EventScope: scope,
TriggeringWebhookName: triggeringWebhookName,
}
return msgqueue.NewTenantMessage(
@@ -156,3 +160,62 @@ func eventToTaskV1(tenantId, eventExternalId, key string, data, additionalMeta [
payloadTyped,
)
}
func createWebhookValidationFailureMsg(tenantId, webhookName, errorText string) (*msgqueue.Message, error) {
payloadTyped := tasktypes.FailedWebhookValidationPayload{
WebhookName: webhookName,
ErrorText: errorText,
}
return msgqueue.NewTenantMessage(
tenantId,
"failed-webhook-validation",
false,
true,
payloadTyped,
)
}
func (i *IngestorImpl) ingestWebhookValidationFailure(tenantId, webhookName, errorText string) error {
msg, err := createWebhookValidationFailureMsg(
tenantId,
webhookName,
errorText,
)
if err != nil {
return fmt.Errorf("could not create failed webhook validation payload: %w", err)
}
err = i.mqv1.SendMessage(context.Background(), msgqueue.OLAP_QUEUE, msg)
if err != nil {
return fmt.Errorf("could not add failed webhook validation to olap queue: %w", err)
}
return nil
}
func (i *IngestorImpl) ingestCELEvaluationFailure(ctx context.Context, tenantId, errorText string, source sqlcv1.V1CelEvaluationFailureSource) error {
msg, err := tasktypes.CELEvaluationFailureMessage(
tenantId,
[]v1.CELEvaluationFailure{
{
Source: source,
ErrorMessage: errorText,
},
},
)
if err != nil {
return fmt.Errorf("failed to create CEL evaluation failure message: %w", err)
}
err = i.mqv1.SendMessage(ctx, msgqueue.OLAP_QUEUE, msg)
if err != nil {
return fmt.Errorf("failed to send CEL evaluation failure message: %w", err)
}
return nil
}
internal/services/ingestor/server.go
+1 -1
View File
@@ -38,7 +38,7 @@ func (i *IngestorImpl) Push(ctx context.Context, req *contracts.PushEventRequest
return nil, status.Errorf(codes.InvalidArgument, "Invalid request: %s", err)
}
event, err := i.IngestEvent(ctx, tenant, req.Key, []byte(req.Payload), additionalMeta, req.Priority, req.Scope)
event, err := i.IngestEvent(ctx, tenant, req.Key, []byte(req.Payload), additionalMeta, req.Priority, req.Scope, nil)
if err == metered.ErrResourceExhausted {
return nil, status.Errorf(codes.ResourceExhausted, "resource exhausted: event limit exceeded for tenant")
internal/services/shared/tasktypes/v1/event.go
+1
View File
@@ -14,6 +14,7 @@ type UserEventTaskPayload struct {
EventAdditionalMetadata []byte `json:"event_additional_metadata"`
EventPriority *int32 `json:"event_priority,omitempty"`
EventScope *string `json:"event_scope,omitempty"`
TriggeringWebhookName *string `json:"triggering_webhook_name,omitempty"`
}
func NewInternalEventMessage(tenantId string, timestamp time.Time, events ...v1.InternalTaskEvent) (*msgqueue.Message, error) {
internal/services/shared/tasktypes/v1/olap.go
+17
View File
@@ -12,6 +12,22 @@ import (
"github.com/hatchet-dev/hatchet/pkg/repository/v1/sqlcv1"
)
type CELEvaluationFailures struct {
Failures []v1.CELEvaluationFailure
}
func CELEvaluationFailureMessage(tenantId string, failures []v1.CELEvaluationFailure) (*msgqueue.Message, error) {
return msgqueue.NewTenantMessage(
tenantId,
"cel-evaluation-failure",
false,
true,
CELEvaluationFailures{
Failures: failures,
},
)
}
type CreatedTaskPayload struct {
*sqlcv1.V1Task
}
@@ -54,6 +70,7 @@ type CreatedEventTriggerPayloadSingleton struct {
EventAdditionalMetadata []byte `json:"event_additional_metadata,omitempty"`
EventScope *string `json:"event_scope,omitempty"`
FilterId *string `json:"filter_id,omitempty"`
TriggeringWebhookName *string `json:"triggering_webhook_name,omitempty"`
}
type CreatedEventTriggerPayload struct {
internal/services/shared/tasktypes/v1/webhook.go
+6
View File
@@ -0,0 +1,6 @@
package v1
type FailedWebhookValidationPayload struct {
WebhookName string `json:"webhook_name" validate:"required"`
ErrorText string `json:"error_text" validate:"required"`
}
pkg/client/rest/gen.go
+1063 -2
View File
File diff suppressed because it is too large Load Diff
pkg/config/server/server.go
+13
View File
@@ -134,6 +134,12 @@ type ConfigFileRuntime struct {
// GRPCRateLimit is the rate limit for the grpc server. We count limits separately for the Workflow, Dispatcher and Events services. Workflow and Events service are set to this rate, Dispatcher is 10X this rate. The rate limit is per second, per engine, per api token.
GRPCRateLimit float64 `mapstructure:"grpcRateLimit" json:"grpcRateLimit,omitempty" default:"1000"`
// WebhookRateLimit is the rate limit for webhook endpoints per second, per webhook
WebhookRateLimit float64 `mapstructure:"webhookRateLimit" json:"webhookRateLimit,omitempty" default:"50"`
// WebhookRateLimitBurst is the burst size for webhook rate limiting
WebhookRateLimitBurst int `mapstructure:"webhookRateLimitBurst" json:"webhookRateLimitBurst,omitempty" default:"100"`
// ShutdownWait is the time between the readiness probe being offline when a shutdown is triggered and the actual start of cleaning up resources.
ShutdownWait time.Duration `mapstructure:"shutdownWait" json:"shutdownWait,omitempty" default:"20s"`
@@ -274,6 +280,9 @@ type LimitConfigFile struct {
DefaultScheduleLimit int `mapstructure:"defaultScheduleLimit" json:"defaultScheduleLimit,omitempty" default:"1000"`
DefaultScheduleAlarmLimit int `mapstructure:"defaultScheduleAlarmLimit" json:"defaultScheduleAlarmLimit,omitempty" default:"750"`
DefaultIncomingWebhookLimit int `mapstructure:"defaultIncomingWebhookLimit" json:"defaultIncomingWebhookLimit,omitempty" default:"5"`
DefaultIncomingWebhookAlarmLimit int `mapstructure:"defaultIncomingWebhookAlarmLimit" json:"defaultIncomingWebhookALarmLimit,omitempty" default:"4"`
}
// Alerting options
@@ -571,6 +580,8 @@ func BindAllEnv(v *viper.Viper) {
_ = v.BindEnv("runtime.grpcInsecure", "SERVER_GRPC_INSECURE")
_ = v.BindEnv("runtime.grpcMaxMsgSize", "SERVER_GRPC_MAX_MSG_SIZE")
_ = v.BindEnv("runtime.grpcRateLimit", "SERVER_GRPC_RATE_LIMIT")
_ = v.BindEnv("runtime.webhookRateLimit", "SERVER_INCOMING_WEBHOOK_RATE_LIMIT")
_ = v.BindEnv("runtime.webhookRateLimitBurst", "SERVER_INCOMING_WEBHOOK_RATE_LIMIT_BURST")
_ = v.BindEnv("runtime.schedulerConcurrencyRateLimit", "SCHEDULER_CONCURRENCY_RATE_LIMIT")
_ = v.BindEnv("runtime.shutdownWait", "SERVER_SHUTDOWN_WAIT")
_ = v.BindEnv("servicesString", "SERVER_SERVICES")
@@ -621,6 +632,8 @@ func BindAllEnv(v *viper.Viper) {
_ = v.BindEnv("runtime.limits.defaultScheduleLimit", "SERVER_LIMITS_DEFAULT_SCHEDULE_LIMIT")
_ = v.BindEnv("runtime.limits.defaultScheduleAlarmLimit", "SERVER_LIMITS_DEFAULT_SCHEDULE_ALARM_LIMIT")
_ = v.BindEnv("runtime.limits.defaultIncomingWebhookLimit", "SERVER_LIMITS_DEFAULT_INCOMING_WEBHOOK_LIMIT")
// buffer options
_ = v.BindEnv("runtime.workflowRunBuffer.waitForFlush", "SERVER_WORKFLOWRUNBUFFER_WAIT_FOR_FLUSH")
_ = v.BindEnv("runtime.workflowRunBuffer.maxConcurrent", "SERVER_WORKFLOWRUNBUFFER_MAX_CONCURRENT")
pkg/repository/event.go
+3
View File
@@ -33,6 +33,9 @@ type CreateEventOpts struct {
// (optional) the event scope
Scope *string `validate:"omitempty"`
// (optional) the triggering webhook name
TriggeringWebhookName *string `validate:"omitempty"`
}
type ListEventOpts struct {
pkg/repository/postgres/dbsqlc/models.go
+8 -7
View File
@@ -278,13 +278,14 @@ func (ns NullLeaseKind) Value() (driver.Value, error) {
type LimitResource string
const (
LimitResourceWORKFLOWRUN LimitResource = "WORKFLOW_RUN"
LimitResourceTASKRUN LimitResource = "TASK_RUN"
LimitResourceEVENT LimitResource = "EVENT"
LimitResourceWORKER LimitResource = "WORKER"
LimitResourceWORKERSLOT LimitResource = "WORKER_SLOT"
LimitResourceCRON LimitResource = "CRON"
LimitResourceSCHEDULE LimitResource = "SCHEDULE"
LimitResourceWORKFLOWRUN LimitResource = "WORKFLOW_RUN"
LimitResourceTASKRUN LimitResource = "TASK_RUN"
LimitResourceEVENT LimitResource = "EVENT"
LimitResourceWORKER LimitResource = "WORKER"
LimitResourceWORKERSLOT LimitResource = "WORKER_SLOT"
LimitResourceCRON LimitResource = "CRON"
LimitResourceSCHEDULE LimitResource = "SCHEDULE"
LimitResourceINCOMINGWEBHOOK LimitResource = "INCOMING_WEBHOOK"
)
func (e *LimitResource) Scan(src interface{}) error {
pkg/repository/postgres/dbsqlc/tenant_limits.sql
+1 -2
View File
@@ -39,8 +39,7 @@ WITH existing AS (
SELECT *
FROM "TenantResourceLimit"
WHERE "tenantId" = @tenantId::uuid AND "resource" = sqlc.narg('resource')::"LimitResource"
)
, insert_row AS (
), insert_row AS (
INSERT INTO "TenantResourceLimit" ("id", "tenantId", "resource", "value", "limitValue", "alarmValue", "window", "lastRefill", "customValueMeter")
SELECT gen_random_uuid(), @tenantId::uuid, sqlc.narg('resource')::"LimitResource", 0, sqlc.narg('limitValue')::int, sqlc.narg('alarmValue')::int, sqlc.narg('window')::text, CURRENT_TIMESTAMP, COALESCE(sqlc.narg('customValueMeter')::boolean, false)
WHERE NOT EXISTS (SELECT 1 FROM existing)
pkg/repository/postgres/dbsqlc/tenant_limits.sql.go
+1 -2
View File
@@ -246,8 +246,7 @@ WITH existing AS (
SELECT id, "createdAt", "updatedAt", resource, "tenantId", "limitValue", "alarmValue", value, "window", "lastRefill", "customValueMeter"
FROM "TenantResourceLimit"
WHERE "tenantId" = $1::uuid AND "resource" = $2::"LimitResource"
)
, insert_row AS (
), insert_row AS (
INSERT INTO "TenantResourceLimit" ("id", "tenantId", "resource", "value", "limitValue", "alarmValue", "window", "lastRefill", "customValueMeter")
SELECT gen_random_uuid(), $1::uuid, $2::"LimitResource", 0, $3::int, $4::int, $5::text, CURRENT_TIMESTAMP, COALESCE($6::boolean, false)
WHERE NOT EXISTS (SELECT 1 FROM existing)
pkg/repository/postgres/tenant_limit.go
+7
View File
@@ -85,6 +85,13 @@ func (t *tenantLimitRepository) DefaultLimits() []repository.Limit {
Window: nil,
CustomValueMeter: true,
},
{
Resource: dbsqlc.LimitResourceINCOMINGWEBHOOK,
Limit: int32(t.config.Limits.DefaultIncomingWebhookLimit), // nolint: gosec
Alarm: int32(t.config.Limits.DefaultIncomingWebhookAlarmLimit), // nolint: gosec
Window: nil,
CustomValueMeter: true,
},
}
}
pkg/repository/v1/olap.go
+56
View File
@@ -234,6 +234,9 @@ type OLAPRepository interface {
GetDagDurationsByDagIds(ctx context.Context, tenantId string, dagIds []int64, dagInsertedAts []pgtype.Timestamptz, readableStatuses []sqlcv1.V1ReadableStatusOlap) (map[string]*sqlcv1.GetDagDurationsByDagIdsRow, error)
GetTaskDurationsByTaskIds(ctx context.Context, tenantId string, taskIds []int64, taskInsertedAts []pgtype.Timestamptz, readableStatuses []sqlcv1.V1ReadableStatusOlap) (map[int64]*sqlcv1.GetTaskDurationsByTaskIdsRow, error)
CreateIncomingWebhookValidationFailureLogs(ctx context.Context, tenantId string, opts []CreateIncomingWebhookFailureLogOpts) error
StoreCELEvaluationFailures(ctx context.Context, tenantId string, failures []CELEvaluationFailure) error
}
type OLAPRepositoryImpl struct {
@@ -1588,6 +1591,7 @@ type ListEventsRow struct {
CancelledCount int64 `json:"cancelled_count"`
FailedCount int64 `json:"failed_count"`
TriggeredRuns []byte `json:"triggered_runs"`
TriggeringWebhookName *string `json:"triggering_webhook_name,omitempty"`
}
func (r *OLAPRepositoryImpl) ListEvents(ctx context.Context, opts sqlcv1.ListEventsParams) ([]*ListEventsRow, *int64, error) {
@@ -1638,8 +1642,14 @@ func (r *OLAPRepositoryImpl) ListEvents(ctx context.Context, opts sqlcv1.ListEve
for _, event := range events {
data, exists := externalIdToEventData[event.ExternalID]
var triggeringWebhookName *string
if event.TriggeringWebhookName.Valid {
triggeringWebhookName = &event.TriggeringWebhookName.String
}
if !exists || len(data) == 0 {
result = append(result, &ListEventsRow{
TenantID: event.TenantID,
EventID: event.ID,
@@ -1654,6 +1664,7 @@ func (r *OLAPRepositoryImpl) ListEvents(ctx context.Context, opts sqlcv1.ListEve
CompletedCount: 0,
CancelledCount: 0,
FailedCount: 0,
TriggeringWebhookName: triggeringWebhookName,
})
} else {
for _, d := range data {
@@ -1672,6 +1683,7 @@ func (r *OLAPRepositoryImpl) ListEvents(ctx context.Context, opts sqlcv1.ListEve
CancelledCount: d.CancelledCount,
FailedCount: d.FailedCount,
TriggeredRuns: d.TriggeredRuns,
TriggeringWebhookName: triggeringWebhookName,
})
}
}
@@ -1730,3 +1742,47 @@ func (r *OLAPRepositoryImpl) GetTaskDurationsByTaskIds(ctx context.Context, tena
return taskDurations, nil
}
type CreateIncomingWebhookFailureLogOpts struct {
WebhookName string
ErrorText string
}
func (r *OLAPRepositoryImpl) CreateIncomingWebhookValidationFailureLogs(ctx context.Context, tenantId string, opts []CreateIncomingWebhookFailureLogOpts) error {
incomingWebhookNames := make([]string, len(opts))
errors := make([]string, len(opts))
for i, opt := range opts {
incomingWebhookNames[i] = opt.WebhookName
errors[i] = opt.ErrorText
}
params := sqlcv1.CreateIncomingWebhookValidationFailureLogsParams{
Tenantid: sqlchelpers.UUIDFromStr(tenantId),
Incomingwebhooknames: incomingWebhookNames,
Errors: errors,
}
return r.queries.CreateIncomingWebhookValidationFailureLogs(ctx, r.pool, params)
}
type CELEvaluationFailure struct {
Source sqlcv1.V1CelEvaluationFailureSource `json:"source"`
ErrorMessage string `json:"error_message"`
}
func (r *OLAPRepositoryImpl) StoreCELEvaluationFailures(ctx context.Context, tenantId string, failures []CELEvaluationFailure) error {
errorMessages := make([]string, len(failures))
sources := make([]string, len(failures))
for i, failure := range failures {
errorMessages[i] = failure.ErrorMessage
sources[i] = string(failure.Source)
}
return r.queries.StoreCELEvaluationFailures(ctx, r.pool, sqlcv1.StoreCELEvaluationFailuresParams{
Tenantid: sqlchelpers.UUIDFromStr(tenantId),
Sources: sources,
Errors: errorMessages,
})
}
pkg/repository/v1/repository.go
+7
View File
@@ -23,6 +23,7 @@ type Repository interface {
Workflows() WorkflowRepository
Ticker() TickerRepository
Filters() FilterRepository
Webhooks() WebhookRepository
}
type repositoryImpl struct {
@@ -36,6 +37,7 @@ type repositoryImpl struct {
workflows WorkflowRepository
ticker TickerRepository
filters FilterRepository
webhooks WebhookRepository
}
func NewRepository(pool *pgxpool.Pool, l *zerolog.Logger, taskRetentionPeriod, olapRetentionPeriod time.Duration, maxInternalRetryCount int32, entitlements repository.EntitlementsRepository) (Repository, func() error) {
@@ -60,6 +62,7 @@ func NewRepository(pool *pgxpool.Pool, l *zerolog.Logger, taskRetentionPeriod, o
workflows: newWorkflowRepository(shared),
ticker: newTickerRepository(shared),
filters: newFilterRepository(shared),
webhooks: newWebhookRepository(shared),
}
return impl, func() error {
@@ -114,3 +117,7 @@ func (r *repositoryImpl) Ticker() TickerRepository {
func (r *repositoryImpl) Filters() FilterRepository {
return r.filters
}
func (r *repositoryImpl) Webhooks() WebhookRepository {
return r.webhooks
}
pkg/repository/v1/sqlcv1/models.go
+268 -15
View File
@@ -278,13 +278,14 @@ func (ns NullLeaseKind) Value() (driver.Value, error) {
type LimitResource string
const (
LimitResourceWORKFLOWRUN LimitResource = "WORKFLOW_RUN"
LimitResourceTASKRUN LimitResource = "TASK_RUN"
LimitResourceEVENT LimitResource = "EVENT"
LimitResourceWORKER LimitResource = "WORKER"
LimitResourceWORKERSLOT LimitResource = "WORKER_SLOT"
LimitResourceCRON LimitResource = "CRON"
LimitResourceSCHEDULE LimitResource = "SCHEDULE"
LimitResourceWORKFLOWRUN LimitResource = "WORKFLOW_RUN"
LimitResourceTASKRUN LimitResource = "TASK_RUN"
LimitResourceEVENT LimitResource = "EVENT"
LimitResourceWORKER LimitResource = "WORKER"
LimitResourceWORKERSLOT LimitResource = "WORKER_SLOT"
LimitResourceCRON LimitResource = "CRON"
LimitResourceSCHEDULE LimitResource = "SCHEDULE"
LimitResourceINCOMINGWEBHOOK LimitResource = "INCOMING_WEBHOOK"
)
func (e *LimitResource) Scan(src interface{}) error {
@@ -856,6 +857,48 @@ func (ns NullTenantResourceLimitAlertType) Value() (driver.Value, error) {
return string(ns.TenantResourceLimitAlertType), nil
}
type V1CelEvaluationFailureSource string
const (
V1CelEvaluationFailureSourceFILTER V1CelEvaluationFailureSource = "FILTER"
V1CelEvaluationFailureSourceWEBHOOK V1CelEvaluationFailureSource = "WEBHOOK"
)
func (e *V1CelEvaluationFailureSource) Scan(src interface{}) error {
switch s := src.(type) {
case []byte:
*e = V1CelEvaluationFailureSource(s)
case string:
*e = V1CelEvaluationFailureSource(s)
default:
return fmt.Errorf("unsupported scan type for V1CelEvaluationFailureSource: %T", src)
}
return nil
}
type NullV1CelEvaluationFailureSource struct {
V1CelEvaluationFailureSource V1CelEvaluationFailureSource `json:"v1_cel_evaluation_failure_source"`
Valid bool `json:"valid"` // Valid is true if V1CelEvaluationFailureSource is not NULL
}
// Scan implements the Scanner interface.
func (ns *NullV1CelEvaluationFailureSource) Scan(value interface{}) error {
if value == nil {
ns.V1CelEvaluationFailureSource, ns.Valid = "", false
return nil
}
ns.Valid = true
return ns.V1CelEvaluationFailureSource.Scan(value)
}
// Value implements the driver Valuer interface.
func (ns NullV1CelEvaluationFailureSource) Value() (driver.Value, error) {
if !ns.Valid {
return nil, nil
}
return string(ns.V1CelEvaluationFailureSource), nil
}
type V1ConcurrencyStrategy string
const (
@@ -1002,6 +1045,179 @@ func (ns NullV1EventTypeOlap) Value() (driver.Value, error) {
return string(ns.V1EventTypeOlap), nil
}
type V1IncomingWebhookAuthType string
const (
V1IncomingWebhookAuthTypeBASIC V1IncomingWebhookAuthType = "BASIC"
V1IncomingWebhookAuthTypeAPIKEY V1IncomingWebhookAuthType = "API_KEY"
V1IncomingWebhookAuthTypeHMAC V1IncomingWebhookAuthType = "HMAC"
)
func (e *V1IncomingWebhookAuthType) Scan(src interface{}) error {
switch s := src.(type) {
case []byte:
*e = V1IncomingWebhookAuthType(s)
case string:
*e = V1IncomingWebhookAuthType(s)
default:
return fmt.Errorf("unsupported scan type for V1IncomingWebhookAuthType: %T", src)
}
return nil
}
type NullV1IncomingWebhookAuthType struct {
V1IncomingWebhookAuthType V1IncomingWebhookAuthType `json:"v1_incoming_webhook_auth_type"`
Valid bool `json:"valid"` // Valid is true if V1IncomingWebhookAuthType is not NULL
}
// Scan implements the Scanner interface.
func (ns *NullV1IncomingWebhookAuthType) Scan(value interface{}) error {
if value == nil {
ns.V1IncomingWebhookAuthType, ns.Valid = "", false
return nil
}
ns.Valid = true
return ns.V1IncomingWebhookAuthType.Scan(value)
}
// Value implements the driver Valuer interface.
func (ns NullV1IncomingWebhookAuthType) Value() (driver.Value, error) {
if !ns.Valid {
return nil, nil
}
return string(ns.V1IncomingWebhookAuthType), nil
}
type V1IncomingWebhookHmacAlgorithm string
const (
V1IncomingWebhookHmacAlgorithmSHA1 V1IncomingWebhookHmacAlgorithm = "SHA1"
V1IncomingWebhookHmacAlgorithmSHA256 V1IncomingWebhookHmacAlgorithm = "SHA256"
V1IncomingWebhookHmacAlgorithmSHA512 V1IncomingWebhookHmacAlgorithm = "SHA512"
V1IncomingWebhookHmacAlgorithmMD5 V1IncomingWebhookHmacAlgorithm = "MD5"
)
func (e *V1IncomingWebhookHmacAlgorithm) Scan(src interface{}) error {
switch s := src.(type) {
case []byte:
*e = V1IncomingWebhookHmacAlgorithm(s)
case string:
*e = V1IncomingWebhookHmacAlgorithm(s)
default:
return fmt.Errorf("unsupported scan type for V1IncomingWebhookHmacAlgorithm: %T", src)
}
return nil
}
type NullV1IncomingWebhookHmacAlgorithm struct {
V1IncomingWebhookHmacAlgorithm V1IncomingWebhookHmacAlgorithm `json:"v1_incoming_webhook_hmac_algorithm"`
Valid bool `json:"valid"` // Valid is true if V1IncomingWebhookHmacAlgorithm is not NULL
}
// Scan implements the Scanner interface.
func (ns *NullV1IncomingWebhookHmacAlgorithm) Scan(value interface{}) error {
if value == nil {
ns.V1IncomingWebhookHmacAlgorithm, ns.Valid = "", false
return nil
}
ns.Valid = true
return ns.V1IncomingWebhookHmacAlgorithm.Scan(value)
}
// Value implements the driver Valuer interface.
func (ns NullV1IncomingWebhookHmacAlgorithm) Value() (driver.Value, error) {
if !ns.Valid {
return nil, nil
}
return string(ns.V1IncomingWebhookHmacAlgorithm), nil
}
type V1IncomingWebhookHmacEncoding string
const (
V1IncomingWebhookHmacEncodingHEX V1IncomingWebhookHmacEncoding = "HEX"
V1IncomingWebhookHmacEncodingBASE64 V1IncomingWebhookHmacEncoding = "BASE64"
V1IncomingWebhookHmacEncodingBASE64URL V1IncomingWebhookHmacEncoding = "BASE64URL"
)
func (e *V1IncomingWebhookHmacEncoding) Scan(src interface{}) error {
switch s := src.(type) {
case []byte:
*e = V1IncomingWebhookHmacEncoding(s)
case string:
*e = V1IncomingWebhookHmacEncoding(s)
default:
return fmt.Errorf("unsupported scan type for V1IncomingWebhookHmacEncoding: %T", src)
}
return nil
}
type NullV1IncomingWebhookHmacEncoding struct {
V1IncomingWebhookHmacEncoding V1IncomingWebhookHmacEncoding `json:"v1_incoming_webhook_hmac_encoding"`
Valid bool `json:"valid"` // Valid is true if V1IncomingWebhookHmacEncoding is not NULL
}
// Scan implements the Scanner interface.
func (ns *NullV1IncomingWebhookHmacEncoding) Scan(value interface{}) error {
if value == nil {
ns.V1IncomingWebhookHmacEncoding, ns.Valid = "", false
return nil
}
ns.Valid = true
return ns.V1IncomingWebhookHmacEncoding.Scan(value)
}
// Value implements the driver Valuer interface.
func (ns NullV1IncomingWebhookHmacEncoding) Value() (driver.Value, error) {
if !ns.Valid {
return nil, nil
}
return string(ns.V1IncomingWebhookHmacEncoding), nil
}
type V1IncomingWebhookSourceName string
const (
V1IncomingWebhookSourceNameGENERIC V1IncomingWebhookSourceName = "GENERIC"
V1IncomingWebhookSourceNameGITHUB V1IncomingWebhookSourceName = "GITHUB"
V1IncomingWebhookSourceNameSTRIPE V1IncomingWebhookSourceName = "STRIPE"
)
func (e *V1IncomingWebhookSourceName) Scan(src interface{}) error {
switch s := src.(type) {
case []byte:
*e = V1IncomingWebhookSourceName(s)
case string:
*e = V1IncomingWebhookSourceName(s)
default:
return fmt.Errorf("unsupported scan type for V1IncomingWebhookSourceName: %T", src)
}
return nil
}
type NullV1IncomingWebhookSourceName struct {
V1IncomingWebhookSourceName V1IncomingWebhookSourceName `json:"v1_incoming_webhook_source_name"`
Valid bool `json:"valid"` // Valid is true if V1IncomingWebhookSourceName is not NULL
}
// Scan implements the Scanner interface.
func (ns *NullV1IncomingWebhookSourceName) Scan(value interface{}) error {
if value == nil {
ns.V1IncomingWebhookSourceName, ns.Valid = "", false
return nil
}
ns.Valid = true
return ns.V1IncomingWebhookSourceName.Scan(value)
}
// Value implements the driver Valuer interface.
func (ns NullV1IncomingWebhookSourceName) Value() (driver.Value, error) {
if !ns.Valid {
return nil, nil
}
return string(ns.V1IncomingWebhookSourceName), nil
}
type V1LogLineLevel string
const (
@@ -2446,6 +2662,15 @@ type UserSession struct {
ExpiresAt pgtype.Timestamp `json:"expiresAt"`
}
type V1CelEvaluationFailuresOlap struct {
ID int64 `json:"id"`
TenantID pgtype.UUID `json:"tenant_id"`
Source V1CelEvaluationFailureSource `json:"source"`
Error string `json:"error"`
InsertedAt pgtype.Timestamptz `json:"inserted_at"`
UpdatedAt pgtype.Timestamptz `json:"updated_at"`
}
type V1ConcurrencySlot struct {
SortID pgtype.Int8 `json:"sort_id"`
TaskID int64 `json:"task_id"`
@@ -2538,14 +2763,15 @@ type V1EventToRunOlap struct {
}
type V1EventsOlap struct {
TenantID pgtype.UUID `json:"tenant_id"`
ID int64 `json:"id"`
ExternalID pgtype.UUID `json:"external_id"`
SeenAt pgtype.Timestamptz `json:"seen_at"`
Key string `json:"key"`
Payload []byte `json:"payload"`
AdditionalMetadata []byte `json:"additional_metadata"`
Scope pgtype.Text `json:"scope"`
TenantID pgtype.UUID `json:"tenant_id"`
ID int64 `json:"id"`
ExternalID pgtype.UUID `json:"external_id"`
SeenAt pgtype.Timestamptz `json:"seen_at"`
Key string `json:"key"`
Payload []byte `json:"payload"`
AdditionalMetadata []byte `json:"additional_metadata"`
Scope pgtype.Text `json:"scope"`
TriggeringWebhookName pgtype.Text `json:"triggering_webhook_name"`
}
type V1Filter struct {
@@ -2561,6 +2787,33 @@ type V1Filter struct {
UpdatedAt pgtype.Timestamptz `json:"updated_at"`
}
type V1IncomingWebhook struct {
TenantID pgtype.UUID `json:"tenant_id"`
Name string `json:"name"`
SourceName V1IncomingWebhookSourceName `json:"source_name"`
EventKeyExpression string `json:"event_key_expression"`
AuthMethod V1IncomingWebhookAuthType `json:"auth_method"`
AuthBasicUsername pgtype.Text `json:"auth__basic__username"`
AuthBasicPassword []byte `json:"auth__basic__password"`
AuthApiKeyHeaderName pgtype.Text `json:"auth__api_key__header_name"`
AuthApiKeyKey []byte `json:"auth__api_key__key"`
AuthHmacAlgorithm NullV1IncomingWebhookHmacAlgorithm `json:"auth__hmac__algorithm"`
AuthHmacEncoding NullV1IncomingWebhookHmacEncoding `json:"auth__hmac__encoding"`
AuthHmacSignatureHeaderName pgtype.Text `json:"auth__hmac__signature_header_name"`
AuthHmacWebhookSigningSecret []byte `json:"auth__hmac__webhook_signing_secret"`
InsertedAt pgtype.Timestamptz `json:"inserted_at"`
UpdatedAt pgtype.Timestamptz `json:"updated_at"`
}
type V1IncomingWebhookValidationFailuresOlap struct {
ID int64 `json:"id"`
TenantID pgtype.UUID `json:"tenant_id"`
IncomingWebhookName string `json:"incoming_webhook_name"`
Error string `json:"error"`
InsertedAt pgtype.Timestamptz `json:"inserted_at"`
UpdatedAt pgtype.Timestamptz `json:"updated_at"`
}
type V1LogLine struct {
ID int64 `json:"id"`
CreatedAt pgtype.Timestamptz `json:"created_at"`
pkg/repository/v1/sqlcv1/olap-overwrite.sql
+5 -2
View File
@@ -204,7 +204,9 @@ WITH to_insert AS (
UNNEST(@payloads::JSONB[]) AS payload,
UNNEST(@additionalMetadatas::JSONB[]) AS additional_metadata,
-- Scopes are nullable
UNNEST(@scopes::TEXT[]) AS scope
UNNEST(@scopes::TEXT[]) AS scope,
-- Webhook names are nullable
UNNEST(@triggeringWebhookName::TEXT[]) AS triggering_webhook_name
)
INSERT INTO v1_events_olap (
tenant_id,
@@ -213,7 +215,8 @@ INSERT INTO v1_events_olap (
key,
payload,
additional_metadata,
scope
scope,
triggering_webhook_name
)
SELECT *
FROM to_insert
pkg/repository/v1/sqlcv1/olap-overwrite.sql.go
+18 -11
View File
@@ -376,7 +376,10 @@ WITH to_insert AS (
UNNEST($5::JSONB[]) AS payload,
UNNEST($6::JSONB[]) AS additional_metadata,
-- Scopes are nullable
UNNEST($7::TEXT[]) AS scope
UNNEST($7::TEXT[]) AS scope,
-- Webhook names are nullable
UNNEST($8::TEXT[]) AS triggering_webhook_name
)
INSERT INTO v1_events_olap (
tenant_id,
@@ -385,21 +388,23 @@ INSERT INTO v1_events_olap (
key,
payload,
additional_metadata,
scope
scope,
triggering_webhook_name
)
SELECT tenant_id, external_id, seen_at, key, payload, additional_metadata, scope
SELECT tenant_id, external_id, seen_at, key, payload, additional_metadata, scope, triggering_webhook_name
FROM to_insert
RETURNING tenant_id, id, external_id, seen_at, key, payload, additional_metadata, scope
RETURNING tenant_id, id, external_id, seen_at, key, payload, additional_metadata, scope, triggering_webhook_name
`
type BulkCreateEventsParams struct {
Tenantids []pgtype.UUID `json:"tenantids"`
Externalids []pgtype.UUID `json:"externalids"`
Seenats []pgtype.Timestamptz `json:"seenats"`
Keys []string `json:"keys"`
Payloads [][]byte `json:"payloads"`
Additionalmetadatas [][]byte `json:"additionalmetadatas"`
Scopes []*string `json:"scopes"`
Tenantids []pgtype.UUID `json:"tenantids"`
Externalids []pgtype.UUID `json:"externalids"`
Seenats []pgtype.Timestamptz `json:"seenats"`
Keys []string `json:"keys"`
Payloads [][]byte `json:"payloads"`
Additionalmetadatas [][]byte `json:"additionalmetadatas"`
Scopes []pgtype.Text `json:"scopes"`
TriggeringWebhookNames []pgtype.Text `json:"triggeringWebhookName"`
}
func (q *Queries) BulkCreateEvents(ctx context.Context, db DBTX, arg BulkCreateEventsParams) ([]*V1EventsOlap, error) {
@@ -411,6 +416,7 @@ func (q *Queries) BulkCreateEvents(ctx context.Context, db DBTX, arg BulkCreateE
arg.Payloads,
arg.Additionalmetadatas,
arg.Scopes,
arg.TriggeringWebhookNames,
)
if err != nil {
return nil, err
@@ -428,6 +434,7 @@ func (q *Queries) BulkCreateEvents(ctx context.Context, db DBTX, arg BulkCreateE
&i.Payload,
&i.AdditionalMetadata,
&i.Scope,
&i.TriggeringWebhookName,
); err != nil {
return nil, err
}
pkg/repository/v1/sqlcv1/olap.sql
+54 -2
View File
@@ -11,7 +11,9 @@ SELECT
-- name: CreateOLAPEventPartitions :exec
SELECT
create_v1_range_partition('v1_events_olap'::text, @date::date),
create_v1_range_partition('v1_event_to_run_olap'::text, @date::date)
create_v1_range_partition('v1_event_to_run_olap'::text, @date::date),
create_v1_range_partition('v1_incoming_webhook_validation_failures_olap'::text, @date::date),
create_v1_range_partition('v1_cel_evaluation_failures_olap'::text, @date::date)
;
-- name: ListOLAPPartitionsBeforeDate :many
@@ -27,6 +29,10 @@ WITH task_partitions AS (
SELECT 'v1_event_to_run_olap' AS parent_table, p::TEXT AS partition_name FROM get_v1_partitions_before_date('v1_event_to_run_olap', @date::date) AS p
), events_lookup_table_partitions AS (
SELECT 'v1_event_lookup_table_olap' AS parent_table, p::TEXT AS partition_name FROM get_v1_partitions_before_date('v1_event_lookup_table_olap', @date::date) AS p
), incoming_webhook_validation_failure_partitions AS (
SELECT 'v1_incoming_webhook_validation_failures_olap' AS parent_table, p::TEXT AS partition_name FROM get_v1_partitions_before_date('v1_incoming_webhook_validation_failures_olap', @date::date) AS p
), cel_evaluation_failures_partitions AS (
SELECT 'v1_cel_evaluation_failures_olap' AS parent_table, p::TEXT AS partition_name FROM get_v1_partitions_before_date('v1_cel_evaluation_failures_olap', @date::date) AS p
), candidates AS (
SELECT
*
@@ -67,6 +73,20 @@ WITH task_partitions AS (
*
FROM
events_lookup_table_partitions
UNION ALL
SELECT
*
FROM
incoming_webhook_validation_failure_partitions
UNION ALL
SELECT
*
FROM
cel_evaluation_failures_partitions
)
SELECT *
@@ -74,7 +94,7 @@ FROM candidates
WHERE
CASE
WHEN @shouldPartitionEventsTables::BOOLEAN THEN TRUE
ELSE parent_table NOT IN ('v1_events_olap', 'v1_event_to_run_olap')
ELSE parent_table NOT IN ('v1_events_olap', 'v1_event_to_run_olap', 'v1_cel_evaluation_failures_olap', 'v1_incoming_webhook_validation_failures_olap')
END
;
@@ -1684,3 +1704,35 @@ FROM
LEFT JOIN
task_times tt ON (td.task_id, td.inserted_at) = (tt.task_id, tt.inserted_at)
ORDER BY td.task_id, td.inserted_at;
-- name: CreateIncomingWebhookValidationFailureLogs :exec
WITH inputs AS (
SELECT
UNNEST(@incomingWebhookNames::TEXT[]) AS incoming_webhook_name,
UNNEST(@errors::TEXT[]) AS error
)
INSERT INTO v1_incoming_webhook_validation_failures_olap(
tenant_id,
incoming_webhook_name,
error
)
SELECT
@tenantId::UUID,
i.incoming_webhook_name,
i.error
FROM inputs i;
-- name: StoreCELEvaluationFailures :exec
WITH inputs AS (
SELECT
UNNEST(CAST(@sources::TEXT[] AS v1_cel_evaluation_failure_source[])) AS source,
UNNEST(@errors::TEXT[]) AS error
)
INSERT INTO v1_cel_evaluation_failures_olap (
tenant_id,
source,
error
)
SELECT @tenantId::UUID, source, error
FROM inputs
;
pkg/repository/v1/sqlcv1/olap.sql.go
+80 -4
View File
@@ -21,7 +21,7 @@ type BulkCreateEventTriggersParams struct {
const countEvents = `-- name: CountEvents :one
WITH included_events AS (
SELECT e.tenant_id, e.id, e.external_id, e.seen_at, e.key, e.payload, e.additional_metadata, e.scope
SELECT e.tenant_id, e.id, e.external_id, e.seen_at, e.key, e.payload, e.additional_metadata, e.scope, e.triggering_webhook_name
FROM v1_event_lookup_table_olap elt
JOIN v1_events_olap e ON (elt.tenant_id, elt.event_id, elt.event_seen_at) = (e.tenant_id, e.id, e.seen_at)
WHERE
@@ -120,10 +120,41 @@ type CreateDAGsOLAPParams struct {
TotalTasks int32 `json:"total_tasks"`
}
const createIncomingWebhookValidationFailureLogs = `-- name: CreateIncomingWebhookValidationFailureLogs :exec
WITH inputs AS (
SELECT
UNNEST($2::TEXT[]) AS incoming_webhook_name,
UNNEST($3::TEXT[]) AS error
)
INSERT INTO v1_incoming_webhook_validation_failures_olap(
tenant_id,
incoming_webhook_name,
error
)
SELECT
$1::UUID,
i.incoming_webhook_name,
i.error
FROM inputs i
`
type CreateIncomingWebhookValidationFailureLogsParams struct {
Tenantid pgtype.UUID `json:"tenantid"`
Incomingwebhooknames []string `json:"incomingwebhooknames"`
Errors []string `json:"errors"`
}
func (q *Queries) CreateIncomingWebhookValidationFailureLogs(ctx context.Context, db DBTX, arg CreateIncomingWebhookValidationFailureLogsParams) error {
_, err := db.Exec(ctx, createIncomingWebhookValidationFailureLogs, arg.Tenantid, arg.Incomingwebhooknames, arg.Errors)
return err
}
const createOLAPEventPartitions = `-- name: CreateOLAPEventPartitions :exec
SELECT
create_v1_range_partition('v1_events_olap'::text, $1::date),
create_v1_range_partition('v1_event_to_run_olap'::text, $1::date)
create_v1_range_partition('v1_event_to_run_olap'::text, $1::date),
create_v1_range_partition('v1_incoming_webhook_validation_failures_olap'::text, $1::date),
create_v1_range_partition('v1_cel_evaluation_failures_olap'::text, $1::date)
`
func (q *Queries) CreateOLAPEventPartitions(ctx context.Context, db DBTX, date pgtype.Date) error {
@@ -879,7 +910,7 @@ func (q *Queries) ListEventKeys(ctx context.Context, db DBTX, tenantid pgtype.UU
}
const listEvents = `-- name: ListEvents :many
SELECT e.tenant_id, e.id, e.external_id, e.seen_at, e.key, e.payload, e.additional_metadata, e.scope
SELECT e.tenant_id, e.id, e.external_id, e.seen_at, e.key, e.payload, e.additional_metadata, e.scope, e.triggering_webhook_name
FROM v1_event_lookup_table_olap elt
JOIN v1_events_olap e ON (elt.tenant_id, elt.event_id, elt.event_seen_at) = (e.tenant_id, e.id, e.seen_at)
WHERE
@@ -978,6 +1009,7 @@ func (q *Queries) ListEvents(ctx context.Context, db DBTX, arg ListEventsParams)
&i.Payload,
&i.AdditionalMetadata,
&i.Scope,
&i.TriggeringWebhookName,
); err != nil {
return nil, err
}
@@ -1002,6 +1034,10 @@ WITH task_partitions AS (
SELECT 'v1_event_to_run_olap' AS parent_table, p::TEXT AS partition_name FROM get_v1_partitions_before_date('v1_event_to_run_olap', $2::date) AS p
), events_lookup_table_partitions AS (
SELECT 'v1_event_lookup_table_olap' AS parent_table, p::TEXT AS partition_name FROM get_v1_partitions_before_date('v1_event_lookup_table_olap', $2::date) AS p
), incoming_webhook_validation_failure_partitions AS (
SELECT 'v1_incoming_webhook_validation_failures_olap' AS parent_table, p::TEXT AS partition_name FROM get_v1_partitions_before_date('v1_incoming_webhook_validation_failures_olap', $2::date) AS p
), cel_evaluation_failures_partitions AS (
SELECT 'v1_cel_evaluation_failures_olap' AS parent_table, p::TEXT AS partition_name FROM get_v1_partitions_before_date('v1_cel_evaluation_failures_olap', $2::date) AS p
), candidates AS (
SELECT
parent_table, partition_name
@@ -1042,6 +1078,20 @@ WITH task_partitions AS (
parent_table, partition_name
FROM
events_lookup_table_partitions
UNION ALL
SELECT
parent_table, partition_name
FROM
incoming_webhook_validation_failure_partitions
UNION ALL
SELECT
parent_table, partition_name
FROM
cel_evaluation_failures_partitions
)
SELECT parent_table, partition_name
@@ -1049,7 +1099,7 @@ FROM candidates
WHERE
CASE
WHEN $1::BOOLEAN THEN TRUE
ELSE parent_table NOT IN ('v1_events_olap', 'v1_event_to_run_olap')
ELSE parent_table NOT IN ('v1_events_olap', 'v1_event_to_run_olap', 'v1_cel_evaluation_failures_olap', 'v1_incoming_webhook_validation_failures_olap')
END
`
@@ -2405,6 +2455,32 @@ func (q *Queries) ReadWorkflowRunByExternalId(ctx context.Context, db DBTX, work
return &i, err
}
const storeCELEvaluationFailures = `-- name: StoreCELEvaluationFailures :exec
WITH inputs AS (
SELECT
UNNEST(CAST($2::TEXT[] AS v1_cel_evaluation_failure_source[])) AS source,
UNNEST($3::TEXT[]) AS error
)
INSERT INTO v1_cel_evaluation_failures_olap (
tenant_id,
source,
error
)
SELECT $1::UUID, source, error
FROM inputs
`
type StoreCELEvaluationFailuresParams struct {
Tenantid pgtype.UUID `json:"tenantid"`
Sources []string `json:"sources"`
Errors []string `json:"errors"`
}
func (q *Queries) StoreCELEvaluationFailures(ctx context.Context, db DBTX, arg StoreCELEvaluationFailuresParams) error {
_, err := db.Exec(ctx, storeCELEvaluationFailures, arg.Tenantid, arg.Sources, arg.Errors)
return err
}
const updateDAGStatuses = `-- name: UpdateDAGStatuses :many
WITH tenants AS (
SELECT UNNEST(
pkg/repository/v1/sqlcv1/sqlc.yaml
+1
View File
@@ -19,6 +19,7 @@ sql:
- sleep.sql
- ticker.sql
- filters.sql
- webhooks.sql
schema:
- ../../../../sql/schema/v0.sql
- ../../../../sql/schema/v1-core.sql
pkg/repository/v1/sqlcv1/webhooks.sql
+71
View File
@@ -0,0 +1,71 @@
-- name: CreateWebhook :one
INSERT INTO v1_incoming_webhook (
tenant_id,
name,
source_name,
event_key_expression,
auth_method,
auth__basic__username,
auth__basic__password,
auth__api_key__header_name,
auth__api_key__key,
auth__hmac__algorithm,
auth__hmac__encoding,
auth__hmac__signature_header_name,
auth__hmac__webhook_signing_secret
) VALUES (
@tenantId::UUID,
@name::TEXT,
@sourceName::v1_incoming_webhook_source_name,
@eventKeyExpression::TEXT,
@authMethod::v1_incoming_webhook_auth_type,
sqlc.narg('authBasicUsername')::TEXT,
@authBasicPassword::BYTEA,
sqlc.narg('authApiKeyHeaderName')::TEXT,
@authApiKeyKey::BYTEA,
sqlc.narg('authHmacAlgorithm')::v1_incoming_webhook_hmac_algorithm,
sqlc.narg('authHmacEncoding')::v1_incoming_webhook_hmac_encoding,
sqlc.narg('authHmacSignatureHeaderName')::TEXT,
@authHmacWebhookSigningSecret::BYTEA
)
RETURNING *;
-- name: GetWebhook :one
SELECT *
FROM v1_incoming_webhook
WHERE
name = @name::TEXT
AND tenant_id = @tenantId::UUID;
-- name: DeleteWebhook :one
DELETE FROM v1_incoming_webhook
WHERE
tenant_id = @tenantId::UUID
AND name = @name::TEXT
RETURNING *;
-- name: ListWebhooks :many
SELECT *
FROM v1_incoming_webhook
WHERE
tenant_id = @tenantId::UUID
AND (
@sourceNames::v1_incoming_webhook_source_name[] IS NULL
OR source_name = ANY(@sourceNames::v1_incoming_webhook_source_name[])
)
AND (
@webhookNames::TEXT[] IS NULL
OR name = ANY(@webhookNames::TEXT[])
)
ORDER BY tenant_id, inserted_at DESC
LIMIT COALESCE(sqlc.narg('webhookLimit')::BIGINT, 20000)
OFFSET COALESCE(sqlc.narg('webhookOffset')::BIGINT, 0)
;
-- name: CanCreateWebhook :one
SELECT COUNT(*) < @webhookLimit::INT AS can_create_webhook
FROM v1_incoming_webhook
WHERE
tenant_id = @tenantId::UUID
;
pkg/repository/v1/sqlcv1/webhooks.sql.go
+258
View File
@@ -0,0 +1,258 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.29.0
// source: webhooks.sql
package sqlcv1
import (
"context"
"github.com/jackc/pgx/v5/pgtype"
)
const canCreateWebhook = `-- name: CanCreateWebhook :one
SELECT COUNT(*) < $1::INT AS can_create_webhook
FROM v1_incoming_webhook
WHERE
tenant_id = $2::UUID
`
type CanCreateWebhookParams struct {
Webhooklimit int32 `json:"webhooklimit"`
Tenantid pgtype.UUID `json:"tenantid"`
}
func (q *Queries) CanCreateWebhook(ctx context.Context, db DBTX, arg CanCreateWebhookParams) (bool, error) {
row := db.QueryRow(ctx, canCreateWebhook, arg.Webhooklimit, arg.Tenantid)
var can_create_webhook bool
err := row.Scan(&can_create_webhook)
return can_create_webhook, err
}
const createWebhook = `-- name: CreateWebhook :one
INSERT INTO v1_incoming_webhook (
tenant_id,
name,
source_name,
event_key_expression,
auth_method,
auth__basic__username,
auth__basic__password,
auth__api_key__header_name,
auth__api_key__key,
auth__hmac__algorithm,
auth__hmac__encoding,
auth__hmac__signature_header_name,
auth__hmac__webhook_signing_secret
) VALUES (
$1::UUID,
$2::TEXT,
$3::v1_incoming_webhook_source_name,
$4::TEXT,
$5::v1_incoming_webhook_auth_type,
$6::TEXT,
$7::BYTEA,
$8::TEXT,
$9::BYTEA,
$10::v1_incoming_webhook_hmac_algorithm,
$11::v1_incoming_webhook_hmac_encoding,
$12::TEXT,
$13::BYTEA
)
RETURNING tenant_id, name, source_name, event_key_expression, auth_method, auth__basic__username, auth__basic__password, auth__api_key__header_name, auth__api_key__key, auth__hmac__algorithm, auth__hmac__encoding, auth__hmac__signature_header_name, auth__hmac__webhook_signing_secret, inserted_at, updated_at
`
type CreateWebhookParams struct {
Tenantid pgtype.UUID `json:"tenantid"`
Name string `json:"name"`
Sourcename V1IncomingWebhookSourceName `json:"sourcename"`
Eventkeyexpression string `json:"eventkeyexpression"`
Authmethod V1IncomingWebhookAuthType `json:"authmethod"`
AuthBasicUsername pgtype.Text `json:"authBasicUsername"`
Authbasicpassword []byte `json:"authbasicpassword"`
AuthApiKeyHeaderName pgtype.Text `json:"authApiKeyHeaderName"`
Authapikeykey []byte `json:"authapikeykey"`
AuthHmacAlgorithm NullV1IncomingWebhookHmacAlgorithm `json:"authHmacAlgorithm"`
AuthHmacEncoding NullV1IncomingWebhookHmacEncoding `json:"authHmacEncoding"`
AuthHmacSignatureHeaderName pgtype.Text `json:"authHmacSignatureHeaderName"`
Authhmacwebhooksigningsecret []byte `json:"authhmacwebhooksigningsecret"`
}
func (q *Queries) CreateWebhook(ctx context.Context, db DBTX, arg CreateWebhookParams) (*V1IncomingWebhook, error) {
row := db.QueryRow(ctx, createWebhook,
arg.Tenantid,
arg.Name,
arg.Sourcename,
arg.Eventkeyexpression,
arg.Authmethod,
arg.AuthBasicUsername,
arg.Authbasicpassword,
arg.AuthApiKeyHeaderName,
arg.Authapikeykey,
arg.AuthHmacAlgorithm,
arg.AuthHmacEncoding,
arg.AuthHmacSignatureHeaderName,
arg.Authhmacwebhooksigningsecret,
)
var i V1IncomingWebhook
err := row.Scan(
&i.TenantID,
&i.Name,
&i.SourceName,
&i.EventKeyExpression,
&i.AuthMethod,
&i.AuthBasicUsername,
&i.AuthBasicPassword,
&i.AuthApiKeyHeaderName,
&i.AuthApiKeyKey,
&i.AuthHmacAlgorithm,
&i.AuthHmacEncoding,
&i.AuthHmacSignatureHeaderName,
&i.AuthHmacWebhookSigningSecret,
&i.InsertedAt,
&i.UpdatedAt,
)
return &i, err
}
const deleteWebhook = `-- name: DeleteWebhook :one
DELETE FROM v1_incoming_webhook
WHERE
tenant_id = $1::UUID
AND name = $2::TEXT
RETURNING tenant_id, name, source_name, event_key_expression, auth_method, auth__basic__username, auth__basic__password, auth__api_key__header_name, auth__api_key__key, auth__hmac__algorithm, auth__hmac__encoding, auth__hmac__signature_header_name, auth__hmac__webhook_signing_secret, inserted_at, updated_at
`
type DeleteWebhookParams struct {
Tenantid pgtype.UUID `json:"tenantid"`
Name string `json:"name"`
}
func (q *Queries) DeleteWebhook(ctx context.Context, db DBTX, arg DeleteWebhookParams) (*V1IncomingWebhook, error) {
row := db.QueryRow(ctx, deleteWebhook, arg.Tenantid, arg.Name)
var i V1IncomingWebhook
err := row.Scan(
&i.TenantID,
&i.Name,
&i.SourceName,
&i.EventKeyExpression,
&i.AuthMethod,
&i.AuthBasicUsername,
&i.AuthBasicPassword,
&i.AuthApiKeyHeaderName,
&i.AuthApiKeyKey,
&i.AuthHmacAlgorithm,
&i.AuthHmacEncoding,
&i.AuthHmacSignatureHeaderName,
&i.AuthHmacWebhookSigningSecret,
&i.InsertedAt,
&i.UpdatedAt,
)
return &i, err
}
const getWebhook = `-- name: GetWebhook :one
SELECT tenant_id, name, source_name, event_key_expression, auth_method, auth__basic__username, auth__basic__password, auth__api_key__header_name, auth__api_key__key, auth__hmac__algorithm, auth__hmac__encoding, auth__hmac__signature_header_name, auth__hmac__webhook_signing_secret, inserted_at, updated_at
FROM v1_incoming_webhook
WHERE
name = $1::TEXT
AND tenant_id = $2::UUID
`
type GetWebhookParams struct {
Name string `json:"name"`
Tenantid pgtype.UUID `json:"tenantid"`
}
func (q *Queries) GetWebhook(ctx context.Context, db DBTX, arg GetWebhookParams) (*V1IncomingWebhook, error) {
row := db.QueryRow(ctx, getWebhook, arg.Name, arg.Tenantid)
var i V1IncomingWebhook
err := row.Scan(
&i.TenantID,
&i.Name,
&i.SourceName,
&i.EventKeyExpression,
&i.AuthMethod,
&i.AuthBasicUsername,
&i.AuthBasicPassword,
&i.AuthApiKeyHeaderName,
&i.AuthApiKeyKey,
&i.AuthHmacAlgorithm,
&i.AuthHmacEncoding,
&i.AuthHmacSignatureHeaderName,
&i.AuthHmacWebhookSigningSecret,
&i.InsertedAt,
&i.UpdatedAt,
)
return &i, err
}
const listWebhooks = `-- name: ListWebhooks :many
SELECT tenant_id, name, source_name, event_key_expression, auth_method, auth__basic__username, auth__basic__password, auth__api_key__header_name, auth__api_key__key, auth__hmac__algorithm, auth__hmac__encoding, auth__hmac__signature_header_name, auth__hmac__webhook_signing_secret, inserted_at, updated_at
FROM v1_incoming_webhook
WHERE
tenant_id = $1::UUID
AND (
$2::v1_incoming_webhook_source_name[] IS NULL
OR source_name = ANY($2::v1_incoming_webhook_source_name[])
)
AND (
$3::TEXT[] IS NULL
OR name = ANY($3::TEXT[])
)
ORDER BY tenant_id, inserted_at DESC
LIMIT COALESCE($5::BIGINT, 20000)
OFFSET COALESCE($4::BIGINT, 0)
`
type ListWebhooksParams struct {
Tenantid pgtype.UUID `json:"tenantid"`
Sourcenames []V1IncomingWebhookSourceName `json:"sourcenames"`
Webhooknames []string `json:"webhooknames"`
WebhookOffset pgtype.Int8 `json:"webhookOffset"`
WebhookLimit pgtype.Int8 `json:"webhookLimit"`
}
func (q *Queries) ListWebhooks(ctx context.Context, db DBTX, arg ListWebhooksParams) ([]*V1IncomingWebhook, error) {
rows, err := db.Query(ctx, listWebhooks,
arg.Tenantid,
arg.Sourcenames,
arg.Webhooknames,
arg.WebhookOffset,
arg.WebhookLimit,
)
if err != nil {
return nil, err
}
defer rows.Close()
var items []*V1IncomingWebhook
for rows.Next() {
var i V1IncomingWebhook
if err := rows.Scan(
&i.TenantID,
&i.Name,
&i.SourceName,
&i.EventKeyExpression,
&i.AuthMethod,
&i.AuthBasicUsername,
&i.AuthBasicPassword,
&i.AuthApiKeyHeaderName,
&i.AuthApiKeyKey,
&i.AuthHmacAlgorithm,
&i.AuthHmacEncoding,
&i.AuthHmacSignatureHeaderName,
&i.AuthHmacWebhookSigningSecret,
&i.InsertedAt,
&i.UpdatedAt,
); err != nil {
return nil, err
}
items = append(items, &i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
pkg/repository/v1/trigger.go
+19 -5
View File
@@ -30,6 +30,8 @@ type EventTriggerOpts struct {
Priority *int32
Scope *string
TriggeringWebhookName *string
}
type TriggerTaskData struct {
@@ -118,6 +120,7 @@ type TriggerFromEventsResult struct {
Tasks []*sqlcv1.V1Task
Dags []*DAGWithData
EventExternalIdToRuns map[string][]*Run
CELEvaluationFailures []CELEvaluationFailure
}
type TriggerDecision struct {
@@ -126,7 +129,9 @@ type TriggerDecision struct {
FilterId *string
}
func (r *TriggerRepositoryImpl) makeTriggerDecisions(ctx context.Context, filters []*sqlcv1.V1Filter, hasAnyFilters bool, opt EventTriggerOpts) []TriggerDecision {
func (r *TriggerRepositoryImpl) makeTriggerDecisions(ctx context.Context, filters []*sqlcv1.V1Filter, hasAnyFilters bool, opt EventTriggerOpts) ([]TriggerDecision, []CELEvaluationFailure) {
celEvaluationFailures := make([]CELEvaluationFailure, 0)
// Cases to handle:
// 1. If there are no filters that exist for the workflow, we should trigger it.
// 2. If there _are_ filters that exist, but the list is empty, then there were no scope matches so we should _not_ trigger.
@@ -140,7 +145,7 @@ func (r *TriggerRepositoryImpl) makeTriggerDecisions(ctx context.Context, filter
FilterPayload: nil,
FilterId: nil,
},
}
}, celEvaluationFailures
}
// Case 2 - no filters were found matching the provided scope,
@@ -152,7 +157,7 @@ func (r *TriggerRepositoryImpl) makeTriggerDecisions(ctx context.Context, filter
FilterPayload: nil,
FilterId: nil,
},
}
}, celEvaluationFailures
}
// Case 3 - we have filters, so we should evaluate each expression and return a list of decisions
@@ -187,6 +192,11 @@ func (r *TriggerRepositoryImpl) makeTriggerDecisions(ctx context.Context, filter
FilterPayload: filter.Payload,
FilterId: &filterId,
})
celEvaluationFailures = append(celEvaluationFailures, CELEvaluationFailure{
Source: sqlcv1.V1CelEvaluationFailureSourceFILTER,
ErrorMessage: err.Error(),
})
}
decisions = append(decisions, TriggerDecision{
@@ -197,7 +207,7 @@ func (r *TriggerRepositoryImpl) makeTriggerDecisions(ctx context.Context, filter
}
}
return decisions
return decisions, celEvaluationFailures
}
type EventExternalIdFilterId struct {
@@ -315,6 +325,7 @@ func (r *TriggerRepositoryImpl) TriggerFromEvents(ctx context.Context, tenantId
// each (workflowVersionId, eventKey, opt) is a separate workflow that we need to create
triggerOpts := make([]triggerTuple, 0)
celEvaluationFailures := make([]CELEvaluationFailure, 0)
for _, workflow := range workflowVersionIdsAndEventKeys {
opts, ok := eventKeysToOpts[workflow.IncomingEventKey]
@@ -339,7 +350,9 @@ func (r *TriggerRepositoryImpl) TriggerFromEvents(ctx context.Context, tenantId
filters = workflowIdAndScopeToFilters[key]
}
triggerDecisions := r.makeTriggerDecisions(ctx, filters, hasAnyFilters, opt)
triggerDecisions, evalFailures := r.makeTriggerDecisions(ctx, filters, hasAnyFilters, opt)
celEvaluationFailures = append(celEvaluationFailures, evalFailures...)
for _, decision := range triggerDecisions {
if !decision.ShouldTrigger {
@@ -418,6 +431,7 @@ func (r *TriggerRepositoryImpl) TriggerFromEvents(ctx context.Context, tenantId
Tasks: tasks,
Dags: dags,
EventExternalIdToRuns: eventExternalIdToRuns,
CELEvaluationFailures: celEvaluationFailures,
}, nil
}
pkg/repository/v1/webhooks.go
+209
View File
@@ -0,0 +1,209 @@
package v1
import (
"context"
"fmt"
"github.com/hatchet-dev/hatchet/pkg/repository/postgres/sqlchelpers"
"github.com/hatchet-dev/hatchet/pkg/repository/v1/sqlcv1"
"github.com/jackc/pgx/v5/pgtype"
)
type WebhookRepository interface {
CreateWebhook(ctx context.Context, tenantId string, params CreateWebhookOpts) (*sqlcv1.V1IncomingWebhook, error)
ListWebhooks(ctx context.Context, tenantId string, params ListWebhooksOpts) ([]*sqlcv1.V1IncomingWebhook, error)
DeleteWebhook(ctx context.Context, tenantId, webhookId string) (*sqlcv1.V1IncomingWebhook, error)
GetWebhook(ctx context.Context, tenantId, webhookId string) (*sqlcv1.V1IncomingWebhook, error)
CanCreate(ctx context.Context, tenantId string, webhookLimit int32) (bool, error)
}
type webhookRepository struct {
*sharedRepository
}
func newWebhookRepository(shared *sharedRepository) WebhookRepository {
return &webhookRepository{
sharedRepository: shared,
}
}
type BasicAuthCredentials struct {
Username string `json:"username" validate:"required"`
EncryptedPassword []byte `json:"password" validate:"required"`
}
type APIKeyAuthCredentials struct {
HeaderName string `json:"header_name" validate:"required"`
EncryptedKey []byte `json:"key" validate:"required"`
}
type HMACAuthCredentials struct {
Algorithm sqlcv1.V1IncomingWebhookHmacAlgorithm `json:"algorithm" validate:"required"`
Encoding sqlcv1.V1IncomingWebhookHmacEncoding `json:"encoding" validate:"required"`
SignatureHeaderName string `json:"signature_header_name" validate:"required"`
EncryptedWebhookSigningSecret []byte `json:"webhook_signing_secret" validate:"required"`
}
type AuthConfig struct {
Type sqlcv1.V1IncomingWebhookAuthType `json:"type" validate:"required"`
BasicAuth *BasicAuthCredentials `json:"basic_auth,omitempty"`
APIKeyAuth *APIKeyAuthCredentials `json:"api_key_auth,omitempty"`
HMACAuth *HMACAuthCredentials `json:"hmac_auth,omitempty"`
}
func (ac *AuthConfig) Validate() error {
authMethodsSet := 0
if ac.BasicAuth != nil {
authMethodsSet++
}
if ac.APIKeyAuth != nil {
authMethodsSet++
}
if ac.HMACAuth != nil {
authMethodsSet++
}
if authMethodsSet != 1 {
return fmt.Errorf("exactly one auth method must be set, but %d were provided", authMethodsSet)
}
switch ac.Type {
case sqlcv1.V1IncomingWebhookAuthTypeBASIC:
if ac.BasicAuth == nil {
return fmt.Errorf("basic auth credentials must be provided when type is BASIC")
}
case sqlcv1.V1IncomingWebhookAuthTypeAPIKEY:
if ac.APIKeyAuth == nil {
return fmt.Errorf("api key auth credentials must be provided when type is API_KEY")
}
case sqlcv1.V1IncomingWebhookAuthTypeHMAC:
if ac.HMACAuth == nil {
return fmt.Errorf("hmac auth credentials must be provided when type is HMAC")
}
default:
return fmt.Errorf("unsupported auth type: %s", ac.Type)
}
return nil
}
type CreateWebhookOpts struct {
Tenantid pgtype.UUID `json:"tenantid"`
Sourcename sqlcv1.V1IncomingWebhookSourceName `json:"sourcename"`
Name string `json:"name" validate:"required"`
Eventkeyexpression string `json:"eventkeyexpression"`
AuthConfig AuthConfig `json:"auth_config,omitempty"`
}
func (r *webhookRepository) CreateWebhook(ctx context.Context, tenantId string, opts CreateWebhookOpts) (*sqlcv1.V1IncomingWebhook, error) {
if err := r.v.Validate(opts); err != nil {
return nil, err
}
if err := opts.AuthConfig.Validate(); err != nil {
return nil, err
}
params := sqlcv1.CreateWebhookParams{
Tenantid: sqlchelpers.UUIDFromStr(tenantId),
Sourcename: sqlcv1.V1IncomingWebhookSourceName(opts.Sourcename),
Name: opts.Name,
Eventkeyexpression: opts.Eventkeyexpression,
Authmethod: sqlcv1.V1IncomingWebhookAuthType(opts.AuthConfig.Type),
}
switch opts.AuthConfig.Type {
case sqlcv1.V1IncomingWebhookAuthTypeBASIC:
params.AuthBasicUsername = pgtype.Text{
String: opts.AuthConfig.BasicAuth.Username,
Valid: true,
}
params.Authbasicpassword = opts.AuthConfig.BasicAuth.EncryptedPassword
case sqlcv1.V1IncomingWebhookAuthTypeAPIKEY:
params.AuthApiKeyHeaderName = pgtype.Text{
String: opts.AuthConfig.APIKeyAuth.HeaderName,
Valid: true,
}
params.Authapikeykey = opts.AuthConfig.APIKeyAuth.EncryptedKey
case sqlcv1.V1IncomingWebhookAuthTypeHMAC:
params.AuthHmacAlgorithm = sqlcv1.NullV1IncomingWebhookHmacAlgorithm{
V1IncomingWebhookHmacAlgorithm: opts.AuthConfig.HMACAuth.Algorithm,
Valid: true,
}
params.AuthHmacEncoding = sqlcv1.NullV1IncomingWebhookHmacEncoding{
V1IncomingWebhookHmacEncoding: opts.AuthConfig.HMACAuth.Encoding,
Valid: true,
}
params.AuthHmacSignatureHeaderName = pgtype.Text{
String: opts.AuthConfig.HMACAuth.SignatureHeaderName,
Valid: true,
}
params.Authhmacwebhooksigningsecret = opts.AuthConfig.HMACAuth.EncryptedWebhookSigningSecret
default:
return nil, fmt.Errorf("unsupported auth type: %s", opts.AuthConfig.Type)
}
return r.queries.CreateWebhook(ctx, r.pool, params)
}
type ListWebhooksOpts struct {
WebhookNames []string `json:"webhook_names"`
WebhookSourceNames []sqlcv1.V1IncomingWebhookSourceName `json:"webhook_source_names"`
Limit *int64 `json:"limit" validate:"omitnil,min=1"`
Offset *int64 `json:"offset" validate:"omitnil,min=0"`
}
func (r *webhookRepository) ListWebhooks(ctx context.Context, tenantId string, opts ListWebhooksOpts) ([]*sqlcv1.V1IncomingWebhook, error) {
if err := r.v.Validate(opts); err != nil {
return nil, err
}
var limit pgtype.Int8
var offset pgtype.Int8
if opts.Limit != nil {
limit = pgtype.Int8{
Int64: *opts.Limit,
Valid: true,
}
}
if opts.Offset != nil {
offset = pgtype.Int8{
Int64: *opts.Offset,
Valid: true,
}
}
return r.queries.ListWebhooks(ctx, r.pool, sqlcv1.ListWebhooksParams{
Tenantid: sqlchelpers.UUIDFromStr(tenantId),
Webhooknames: opts.WebhookNames,
Sourcenames: opts.WebhookSourceNames,
WebhookLimit: limit,
WebhookOffset: offset,
})
}
func (r *webhookRepository) DeleteWebhook(ctx context.Context, tenantId, name string) (*sqlcv1.V1IncomingWebhook, error) {
return r.queries.DeleteWebhook(ctx, r.pool, sqlcv1.DeleteWebhookParams{
Tenantid: sqlchelpers.UUIDFromStr(tenantId),
Name: name,
})
}
func (r *webhookRepository) GetWebhook(ctx context.Context, tenantId, name string) (*sqlcv1.V1IncomingWebhook, error) {
return r.queries.GetWebhook(ctx, r.pool, sqlcv1.GetWebhookParams{
Tenantid: sqlchelpers.UUIDFromStr(tenantId),
Name: name,
})
}
func (r *webhookRepository) CanCreate(ctx context.Context, tenantId string, webhookLimit int32) (bool, error) {
return r.queries.CanCreateWebhook(ctx, r.pool, sqlcv1.CanCreateWebhookParams{
Tenantid: sqlchelpers.UUIDFromStr(tenantId),
Webhooklimit: webhookLimit,
})
}
sdks/python/examples/webhooks/test_webhooks.py
+643
View File
@@ -0,0 +1,643 @@
import asyncio
import base64
import hashlib
import hmac
import json
from collections.abc import AsyncGenerator
from contextlib import asynccontextmanager
from datetime import datetime, timezone
from typing import Any
from uuid import uuid4
import aiohttp
import pytest
from examples.webhooks.worker import WebhookInput
from hatchet_sdk import Hatchet
from hatchet_sdk.clients.rest.api.webhook_api import WebhookApi
from hatchet_sdk.clients.rest.models.v1_create_webhook_request import (
V1CreateWebhookRequest,
)
from hatchet_sdk.clients.rest.models.v1_create_webhook_request_api_key import (
V1CreateWebhookRequestAPIKey,
)
from hatchet_sdk.clients.rest.models.v1_create_webhook_request_basic_auth import (
V1CreateWebhookRequestBasicAuth,
)
from hatchet_sdk.clients.rest.models.v1_create_webhook_request_hmac import (
V1CreateWebhookRequestHMAC,
)
from hatchet_sdk.clients.rest.models.v1_event import V1Event
from hatchet_sdk.clients.rest.models.v1_task_status import V1TaskStatus
from hatchet_sdk.clients.rest.models.v1_task_summary import V1TaskSummary
from hatchet_sdk.clients.rest.models.v1_webhook import V1Webhook
from hatchet_sdk.clients.rest.models.v1_webhook_api_key_auth import V1WebhookAPIKeyAuth
from hatchet_sdk.clients.rest.models.v1_webhook_basic_auth import V1WebhookBasicAuth
from hatchet_sdk.clients.rest.models.v1_webhook_hmac_algorithm import (
V1WebhookHMACAlgorithm,
)
from hatchet_sdk.clients.rest.models.v1_webhook_hmac_auth import V1WebhookHMACAuth
from hatchet_sdk.clients.rest.models.v1_webhook_hmac_encoding import (
V1WebhookHMACEncoding,
)
from hatchet_sdk.clients.rest.models.v1_webhook_source_name import V1WebhookSourceName
TEST_BASIC_USERNAME = "test_user"
TEST_BASIC_PASSWORD = "test_password"
TEST_API_KEY_HEADER = "X-API-Key"
TEST_API_KEY_VALUE = "test_api_key_123"
TEST_HMAC_SIGNATURE_HEADER = "X-Signature"
TEST_HMAC_SECRET = "test_hmac_secret"
@pytest.fixture
def webhook_body() -> WebhookInput:
return WebhookInput(type="test", message="Hello, world!")
@pytest.fixture
def test_run_id() -> str:
return str(uuid4())
@pytest.fixture
def test_start() -> datetime:
return datetime.now(timezone.utc)
def create_hmac_signature(
payload: bytes,
secret: str,
algorithm: V1WebhookHMACAlgorithm = V1WebhookHMACAlgorithm.SHA256,
encoding: V1WebhookHMACEncoding = V1WebhookHMACEncoding.HEX,
) -> str:
algorithm_map = {
V1WebhookHMACAlgorithm.SHA1: hashlib.sha1,
V1WebhookHMACAlgorithm.SHA256: hashlib.sha256,
V1WebhookHMACAlgorithm.SHA512: hashlib.sha512,
V1WebhookHMACAlgorithm.MD5: hashlib.md5,
}
hash_func = algorithm_map[algorithm]
signature = hmac.new(secret.encode(), payload, hash_func).digest()
if encoding == V1WebhookHMACEncoding.HEX:
return signature.hex()
if encoding == V1WebhookHMACEncoding.BASE64:
return base64.b64encode(signature).decode()
if encoding == V1WebhookHMACEncoding.BASE64URL:
return base64.urlsafe_b64encode(signature).decode()
raise ValueError(f"Unsupported encoding: {encoding}")
async def send_webhook_request(
url: str,
body: WebhookInput,
auth_type: str,
auth_data: dict[str, Any] | None = None,
headers: dict[str, str] | None = None,
) -> aiohttp.ClientResponse:
request_headers = headers or {}
auth = None
if auth_type == "BASIC" and auth_data:
auth = aiohttp.BasicAuth(auth_data["username"], auth_data["password"])
elif auth_type == "API_KEY" and auth_data:
request_headers[auth_data["header_name"]] = auth_data["api_key"]
elif auth_type == "HMAC" and auth_data:
payload = json.dumps(body.model_dump()).encode()
signature = create_hmac_signature(
payload,
auth_data["secret"],
auth_data.get("algorithm", V1WebhookHMACAlgorithm.SHA256),
auth_data.get("encoding", V1WebhookHMACEncoding.HEX),
)
request_headers[auth_data["header_name"]] = signature
async with aiohttp.ClientSession() as session:
return await session.post(
url, json=body.model_dump(), auth=auth, headers=request_headers
)
async def wait_for_event(
hatchet: Hatchet,
webhook_name: str,
test_start: datetime,
) -> V1Event | None:
await asyncio.sleep(5)
events = await hatchet.event.aio_list(since=test_start)
if events.rows is None:
return None
return next(
(
event
for event in events.rows
if event.triggering_webhook_name == webhook_name
),
None,
)
async def wait_for_workflow_run(
hatchet: Hatchet, event_id: str, test_start: datetime
) -> V1TaskSummary | None:
await asyncio.sleep(5)
runs = await hatchet.runs.aio_list(
since=test_start,
additional_metadata={
"hatchet__event_id": event_id,
},
)
if len(runs.rows) == 0:
return None
return runs.rows[0]
@asynccontextmanager
async def basic_auth_webhook(
hatchet: Hatchet,
test_run_id: str,
username: str = TEST_BASIC_USERNAME,
password: str = TEST_BASIC_PASSWORD,
source_name: V1WebhookSourceName = V1WebhookSourceName.GENERIC,
) -> AsyncGenerator[V1Webhook, None]:
## Hack to get the API client
client = hatchet.metrics.client()
webhook_api = WebhookApi(client)
webhook_request = V1CreateWebhookRequestBasicAuth(
sourceName=source_name,
name=f"test-webhook-basic-{test_run_id}",
eventKeyExpression=f"'{hatchet.config.apply_namespace('webhook')}:' + input.type",
authType="BASIC",
auth=V1WebhookBasicAuth(
username=username,
password=password,
),
)
incoming_webhook = webhook_api.v1_webhook_create(
tenant=hatchet.tenant_id,
v1_create_webhook_request=V1CreateWebhookRequest(webhook_request),
)
try:
yield incoming_webhook
finally:
webhook_api.v1_webhook_delete(
tenant=hatchet.tenant_id,
v1_webhook=incoming_webhook.name,
)
@asynccontextmanager
async def api_key_webhook(
hatchet: Hatchet,
test_run_id: str,
header_name: str = TEST_API_KEY_HEADER,
api_key: str = TEST_API_KEY_VALUE,
source_name: V1WebhookSourceName = V1WebhookSourceName.GENERIC,
) -> AsyncGenerator[V1Webhook, None]:
client = hatchet.metrics.client()
webhook_api = WebhookApi(client)
webhook_request = V1CreateWebhookRequestAPIKey(
sourceName=source_name,
name=f"test-webhook-apikey-{test_run_id}",
eventKeyExpression=f"'{hatchet.config.apply_namespace('webhook')}:' + input.type",
authType="API_KEY",
auth=V1WebhookAPIKeyAuth(
headerName=header_name,
apiKey=api_key,
),
)
incoming_webhook = webhook_api.v1_webhook_create(
tenant=hatchet.tenant_id,
v1_create_webhook_request=V1CreateWebhookRequest(webhook_request),
)
try:
yield incoming_webhook
finally:
webhook_api.v1_webhook_delete(
tenant=hatchet.tenant_id,
v1_webhook=incoming_webhook.name,
)
@asynccontextmanager
async def hmac_webhook(
hatchet: Hatchet,
test_run_id: str,
signature_header_name: str = TEST_HMAC_SIGNATURE_HEADER,
signing_secret: str = TEST_HMAC_SECRET,
algorithm: V1WebhookHMACAlgorithm = V1WebhookHMACAlgorithm.SHA256,
encoding: V1WebhookHMACEncoding = V1WebhookHMACEncoding.HEX,
source_name: V1WebhookSourceName = V1WebhookSourceName.GENERIC,
) -> AsyncGenerator[V1Webhook, None]:
client = hatchet.metrics.client()
webhook_api = WebhookApi(client)
webhook_request = V1CreateWebhookRequestHMAC(
sourceName=source_name,
name=f"test-webhook-hmac-{test_run_id}",
eventKeyExpression=f"'{hatchet.config.apply_namespace('webhook')}:' + input.type",
authType="HMAC",
auth=V1WebhookHMACAuth(
algorithm=algorithm,
encoding=encoding,
signatureHeaderName=signature_header_name,
signingSecret=signing_secret,
),
)
incoming_webhook = webhook_api.v1_webhook_create(
tenant=hatchet.tenant_id,
v1_create_webhook_request=V1CreateWebhookRequest(webhook_request),
)
try:
yield incoming_webhook
finally:
webhook_api.v1_webhook_delete(
tenant=hatchet.tenant_id,
v1_webhook=incoming_webhook.name,
)
def url(tenant_id: str, webhook_name: str) -> str:
return f"http://localhost:8080/api/v1/stable/tenants/{tenant_id}/webhooks/{webhook_name}"
async def assert_has_runs(
hatchet: Hatchet,
test_start: datetime,
webhook_body: WebhookInput,
incoming_webhook: V1Webhook,
) -> None:
triggered_event = await wait_for_event(hatchet, incoming_webhook.name, test_start)
assert triggered_event is not None
assert (
triggered_event.key
== f"{hatchet.config.apply_namespace('webhook')}:{webhook_body.type}"
)
assert triggered_event.payload == webhook_body.model_dump()
workflow_run = await wait_for_workflow_run(
hatchet, triggered_event.metadata.id, test_start
)
assert workflow_run is not None
assert workflow_run.status == V1TaskStatus.COMPLETED
assert workflow_run.additional_metadata is not None
assert (
workflow_run.additional_metadata["hatchet__event_id"]
== triggered_event.metadata.id
)
assert workflow_run.additional_metadata["hatchet__event_key"] == triggered_event.key
assert workflow_run.status == V1TaskStatus.COMPLETED
async def assert_event_not_created(
hatchet: Hatchet,
test_start: datetime,
incoming_webhook: V1Webhook,
) -> None:
triggered_event = await wait_for_event(hatchet, incoming_webhook.name, test_start)
assert triggered_event is None
@pytest.mark.asyncio(loop_scope="session")
async def test_basic_auth_success(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
) -> None:
async with basic_auth_webhook(hatchet, test_run_id) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name),
webhook_body,
"BASIC",
{"username": TEST_BASIC_USERNAME, "password": TEST_BASIC_PASSWORD},
) as response:
assert response.status == 200
data = await response.json()
assert data == {"message": "ok"}
await assert_has_runs(
hatchet,
test_start,
webhook_body,
incoming_webhook,
)
@pytest.mark.parametrize(
"username,password",
[
("test_user", "incorrect_password"),
("incorrect_user", "test_password"),
("incorrect_user", "incorrect_password"),
("", ""),
],
)
@pytest.mark.asyncio(loop_scope="session")
async def test_basic_auth_failure(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
username: str,
password: str,
) -> None:
"""Test basic authentication failures."""
async with basic_auth_webhook(hatchet, test_run_id) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name),
webhook_body,
"BASIC",
{"username": username, "password": password},
) as response:
assert response.status == 403
await assert_event_not_created(
hatchet,
test_start,
incoming_webhook,
)
@pytest.mark.asyncio(loop_scope="session")
async def test_basic_auth_missing_credentials(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
) -> None:
async with basic_auth_webhook(hatchet, test_run_id) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name), webhook_body, "NONE"
) as response:
assert response.status == 403
await assert_event_not_created(
hatchet,
test_start,
incoming_webhook,
)
@pytest.mark.asyncio(loop_scope="session")
async def test_api_key_success(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
) -> None:
async with api_key_webhook(hatchet, test_run_id) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name),
webhook_body,
"API_KEY",
{"header_name": TEST_API_KEY_HEADER, "api_key": TEST_API_KEY_VALUE},
) as response:
assert response.status == 200
data = await response.json()
assert data == {"message": "ok"}
await assert_has_runs(
hatchet,
test_start,
webhook_body,
incoming_webhook,
)
@pytest.mark.parametrize(
"api_key",
[
"incorrect_api_key",
"",
"partial_key",
],
)
@pytest.mark.asyncio(loop_scope="session")
async def test_api_key_failure(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
api_key: str,
) -> None:
async with api_key_webhook(hatchet, test_run_id) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name),
webhook_body,
"API_KEY",
{"header_name": TEST_API_KEY_HEADER, "api_key": api_key},
) as response:
assert response.status == 403
await assert_event_not_created(
hatchet,
test_start,
incoming_webhook,
)
@pytest.mark.asyncio(loop_scope="session")
async def test_api_key_missing_header(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
) -> None:
async with api_key_webhook(hatchet, test_run_id) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name), webhook_body, "NONE"
) as response:
assert response.status == 403
await assert_event_not_created(
hatchet,
test_start,
incoming_webhook,
)
@pytest.mark.asyncio(loop_scope="session")
async def test_hmac_success(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
) -> None:
async with hmac_webhook(hatchet, test_run_id) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name),
webhook_body,
"HMAC",
{
"header_name": TEST_HMAC_SIGNATURE_HEADER,
"secret": TEST_HMAC_SECRET,
"algorithm": V1WebhookHMACAlgorithm.SHA256,
"encoding": V1WebhookHMACEncoding.HEX,
},
) as response:
assert response.status == 200
data = await response.json()
assert data == {"message": "ok"}
await assert_has_runs(
hatchet,
test_start,
webhook_body,
incoming_webhook,
)
@pytest.mark.parametrize(
"algorithm,encoding",
[
(V1WebhookHMACAlgorithm.SHA1, V1WebhookHMACEncoding.HEX),
(V1WebhookHMACAlgorithm.SHA256, V1WebhookHMACEncoding.BASE64),
(V1WebhookHMACAlgorithm.SHA512, V1WebhookHMACEncoding.BASE64URL),
(V1WebhookHMACAlgorithm.MD5, V1WebhookHMACEncoding.HEX),
],
)
@pytest.mark.asyncio(loop_scope="session")
async def test_hmac_different_algorithms_and_encodings(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
algorithm: V1WebhookHMACAlgorithm,
encoding: V1WebhookHMACEncoding,
) -> None:
async with hmac_webhook(
hatchet, test_run_id, algorithm=algorithm, encoding=encoding
) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name),
webhook_body,
"HMAC",
{
"header_name": TEST_HMAC_SIGNATURE_HEADER,
"secret": TEST_HMAC_SECRET,
"algorithm": algorithm,
"encoding": encoding,
},
) as response:
assert response.status == 200
data = await response.json()
assert data == {"message": "ok"}
await assert_has_runs(
hatchet,
test_start,
webhook_body,
incoming_webhook,
)
@pytest.mark.parametrize(
"secret",
[
"incorrect_secret",
"",
"partial_secret",
],
)
@pytest.mark.asyncio(loop_scope="session")
async def test_hmac_signature_failure(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
secret: str,
) -> None:
async with hmac_webhook(hatchet, test_run_id) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name),
webhook_body,
"HMAC",
{
"header_name": TEST_HMAC_SIGNATURE_HEADER,
"secret": secret,
"algorithm": V1WebhookHMACAlgorithm.SHA256,
"encoding": V1WebhookHMACEncoding.HEX,
},
) as response:
assert response.status == 403
await assert_event_not_created(
hatchet,
test_start,
incoming_webhook,
)
@pytest.mark.asyncio(loop_scope="session")
async def test_hmac_missing_signature_header(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
) -> None:
async with hmac_webhook(hatchet, test_run_id) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name), webhook_body, "NONE"
) as response:
assert response.status == 403
await assert_event_not_created(
hatchet,
test_start,
incoming_webhook,
)
@pytest.mark.parametrize(
"source_name",
[
V1WebhookSourceName.GENERIC,
V1WebhookSourceName.GITHUB,
],
)
@pytest.mark.asyncio(loop_scope="session")
async def test_different_source_types(
hatchet: Hatchet,
test_run_id: str,
test_start: datetime,
webhook_body: WebhookInput,
source_name: V1WebhookSourceName,
) -> None:
async with basic_auth_webhook(
hatchet, test_run_id, source_name=source_name
) as incoming_webhook:
async with await send_webhook_request(
url(hatchet.tenant_id, incoming_webhook.name),
webhook_body,
"BASIC",
{"username": TEST_BASIC_USERNAME, "password": TEST_BASIC_PASSWORD},
) as response:
assert response.status == 200
data = await response.json()
assert data == {"message": "ok"}
await assert_has_runs(
hatchet,
test_start,
webhook_body,
incoming_webhook,
)
sdks/python/examples/webhooks/worker.py
+28
View File
@@ -0,0 +1,28 @@
# > Webhooks
from pydantic import BaseModel
from hatchet_sdk import Context, Hatchet
hatchet = Hatchet(debug=True)
class WebhookInput(BaseModel):
type: str
message: str
@hatchet.task(input_validator=WebhookInput, on_events=["webhook:test"])
def webhook(input: WebhookInput, ctx: Context) -> dict[str, str]:
return input.model_dump()
def main() -> None:
worker = hatchet.worker("webhook-worker", workflows=[webhook])
worker.start()
# !!
if __name__ == "__main__":
main()
sdks/python/examples/worker.py
+2
View File
@@ -26,6 +26,7 @@ from examples.on_failure.worker import on_failure_wf, on_failure_wf_with_details
from examples.return_exceptions.worker import return_exceptions_task
from examples.simple.worker import simple, simple_durable
from examples.timeout.worker import refresh_timeout_wf, timeout_wf
from examples.webhooks.worker import webhook
from hatchet_sdk import Hatchet
hatchet = Hatchet(debug=True)
@@ -66,6 +67,7 @@ def main() -> None:
bulk_replay_test_1,
bulk_replay_test_2,
bulk_replay_test_3,
webhook,
return_exceptions_task,
wait_for_sleep_twice,
],
sdks/python/hatchet_sdk/clients/rest/__init__.py
+32
View File
@@ -33,6 +33,7 @@ from hatchet_sdk.clients.rest.api.step_run_api import StepRunApi
from hatchet_sdk.clients.rest.api.task_api import TaskApi
from hatchet_sdk.clients.rest.api.tenant_api import TenantApi
from hatchet_sdk.clients.rest.api.user_api import UserApi
from hatchet_sdk.clients.rest.api.webhook_api import WebhookApi
from hatchet_sdk.clients.rest.api.worker_api import WorkerApi
from hatchet_sdk.clients.rest.api.workflow_api import WorkflowApi
from hatchet_sdk.clients.rest.api.workflow_run_api import WorkflowRunApi
@@ -241,6 +242,21 @@ from hatchet_sdk.clients.rest.models.v1_cel_debug_response_status import (
from hatchet_sdk.clients.rest.models.v1_create_filter_request import (
V1CreateFilterRequest,
)
from hatchet_sdk.clients.rest.models.v1_create_webhook_request import (
V1CreateWebhookRequest,
)
from hatchet_sdk.clients.rest.models.v1_create_webhook_request_api_key import (
V1CreateWebhookRequestAPIKey,
)
from hatchet_sdk.clients.rest.models.v1_create_webhook_request_base import (
V1CreateWebhookRequestBase,
)
from hatchet_sdk.clients.rest.models.v1_create_webhook_request_basic_auth import (
V1CreateWebhookRequestBasicAuth,
)
from hatchet_sdk.clients.rest.models.v1_create_webhook_request_hmac import (
V1CreateWebhookRequestHMAC,
)
from hatchet_sdk.clients.rest.models.v1_dag_children import V1DagChildren
from hatchet_sdk.clients.rest.models.v1_event import V1Event
from hatchet_sdk.clients.rest.models.v1_event_list import V1EventList
@@ -274,6 +290,22 @@ from hatchet_sdk.clients.rest.models.v1_trigger_workflow_run_request import (
from hatchet_sdk.clients.rest.models.v1_update_filter_request import (
V1UpdateFilterRequest,
)
from hatchet_sdk.clients.rest.models.v1_webhook import V1Webhook
from hatchet_sdk.clients.rest.models.v1_webhook_api_key_auth import V1WebhookAPIKeyAuth
from hatchet_sdk.clients.rest.models.v1_webhook_auth_type import V1WebhookAuthType
from hatchet_sdk.clients.rest.models.v1_webhook_basic_auth import V1WebhookBasicAuth
from hatchet_sdk.clients.rest.models.v1_webhook_hmac_algorithm import (
V1WebhookHMACAlgorithm,
)
from hatchet_sdk.clients.rest.models.v1_webhook_hmac_auth import V1WebhookHMACAuth
from hatchet_sdk.clients.rest.models.v1_webhook_hmac_encoding import (
V1WebhookHMACEncoding,
)
from hatchet_sdk.clients.rest.models.v1_webhook_list import V1WebhookList
from hatchet_sdk.clients.rest.models.v1_webhook_receive200_response import (
V1WebhookReceive200Response,
)
from hatchet_sdk.clients.rest.models.v1_webhook_source_name import V1WebhookSourceName
from hatchet_sdk.clients.rest.models.v1_workflow_run import V1WorkflowRun
from hatchet_sdk.clients.rest.models.v1_workflow_run_details import V1WorkflowRunDetails
from hatchet_sdk.clients.rest.models.v1_workflow_run_display_name import (
sdks/python/hatchet_sdk/clients/rest/api/__init__.py
+1
View File
@@ -17,6 +17,7 @@ from hatchet_sdk.clients.rest.api.step_run_api import StepRunApi
from hatchet_sdk.clients.rest.api.task_api import TaskApi
from hatchet_sdk.clients.rest.api.tenant_api import TenantApi
from hatchet_sdk.clients.rest.api.user_api import UserApi
from hatchet_sdk.clients.rest.api.webhook_api import WebhookApi
from hatchet_sdk.clients.rest.api.worker_api import WorkerApi
from hatchet_sdk.clients.rest.api.workflow_api import WorkflowApi
from hatchet_sdk.clients.rest.api.workflow_run_api import WorkflowRunApi
sdks/python/hatchet_sdk/clients/rest/api/webhook_api.py
+1551
View File
File diff suppressed because it is too large Load Diff
sdks/python/hatchet_sdk/clients/rest/models/__init__.py
+31
View File
@@ -204,6 +204,21 @@ from hatchet_sdk.clients.rest.models.v1_cel_debug_response_status import (
from hatchet_sdk.clients.rest.models.v1_create_filter_request import (
V1CreateFilterRequest,
)
from hatchet_sdk.clients.rest.models.v1_create_webhook_request import (
V1CreateWebhookRequest,
)
from hatchet_sdk.clients.rest.models.v1_create_webhook_request_api_key import (
V1CreateWebhookRequestAPIKey,
)
from hatchet_sdk.clients.rest.models.v1_create_webhook_request_base import (
V1CreateWebhookRequestBase,
)
from hatchet_sdk.clients.rest.models.v1_create_webhook_request_basic_auth import (
V1CreateWebhookRequestBasicAuth,
)
from hatchet_sdk.clients.rest.models.v1_create_webhook_request_hmac import (
V1CreateWebhookRequestHMAC,
)
from hatchet_sdk.clients.rest.models.v1_dag_children import V1DagChildren
from hatchet_sdk.clients.rest.models.v1_event import V1Event
from hatchet_sdk.clients.rest.models.v1_event_list import V1EventList
@@ -237,6 +252,22 @@ from hatchet_sdk.clients.rest.models.v1_trigger_workflow_run_request import (
from hatchet_sdk.clients.rest.models.v1_update_filter_request import (
V1UpdateFilterRequest,
)
from hatchet_sdk.clients.rest.models.v1_webhook import V1Webhook
from hatchet_sdk.clients.rest.models.v1_webhook_api_key_auth import V1WebhookAPIKeyAuth
from hatchet_sdk.clients.rest.models.v1_webhook_auth_type import V1WebhookAuthType
from hatchet_sdk.clients.rest.models.v1_webhook_basic_auth import V1WebhookBasicAuth
from hatchet_sdk.clients.rest.models.v1_webhook_hmac_algorithm import (
V1WebhookHMACAlgorithm,
)
from hatchet_sdk.clients.rest.models.v1_webhook_hmac_auth import V1WebhookHMACAuth
from hatchet_sdk.clients.rest.models.v1_webhook_hmac_encoding import (
V1WebhookHMACEncoding,
)
from hatchet_sdk.clients.rest.models.v1_webhook_list import V1WebhookList
from hatchet_sdk.clients.rest.models.v1_webhook_receive200_response import (
V1WebhookReceive200Response,
)
from hatchet_sdk.clients.rest.models.v1_webhook_source_name import V1WebhookSourceName
from hatchet_sdk.clients.rest.models.v1_workflow_run import V1WorkflowRun
from hatchet_sdk.clients.rest.models.v1_workflow_run_details import V1WorkflowRunDetails
from hatchet_sdk.clients.rest.models.v1_workflow_run_display_name import (
sdks/python/hatchet_sdk/clients/rest/models/v1_create_webhook_request.py
+215
View File
@@ -0,0 +1,215 @@
# coding: utf-8
"""
Hatchet API
The Hatchet API
The version of the OpenAPI document: 1.0.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
""" # noqa: E501
from __future__ import annotations
import json
import pprint
from typing import Any, Dict, List, Optional, Set, Union
from pydantic import (
BaseModel,
ConfigDict,
Field,
StrictStr,
ValidationError,
field_validator,
)
from typing_extensions import Literal, Self
from hatchet_sdk.clients.rest.models.v1_create_webhook_request_api_key import (
V1CreateWebhookRequestAPIKey,
)
from hatchet_sdk.clients.rest.models.v1_create_webhook_request_basic_auth import (
V1CreateWebhookRequestBasicAuth,
)
from hatchet_sdk.clients.rest.models.v1_create_webhook_request_hmac import (
V1CreateWebhookRequestHMAC,
)
V1CREATEWEBHOOKREQUEST_ONE_OF_SCHEMAS = [
"V1CreateWebhookRequestAPIKey",
"V1CreateWebhookRequestBasicAuth",
"V1CreateWebhookRequestHMAC",
]
class V1CreateWebhookRequest(BaseModel):
"""
V1CreateWebhookRequest
"""
# data type: V1CreateWebhookRequestBasicAuth
oneof_schema_1_validator: Optional[V1CreateWebhookRequestBasicAuth] = None
# data type: V1CreateWebhookRequestAPIKey
oneof_schema_2_validator: Optional[V1CreateWebhookRequestAPIKey] = None
# data type: V1CreateWebhookRequestHMAC
oneof_schema_3_validator: Optional[V1CreateWebhookRequestHMAC] = None
actual_instance: Optional[
Union[
V1CreateWebhookRequestAPIKey,
V1CreateWebhookRequestBasicAuth,
V1CreateWebhookRequestHMAC,
]
] = None
one_of_schemas: Set[str] = {
"V1CreateWebhookRequestAPIKey",
"V1CreateWebhookRequestBasicAuth",
"V1CreateWebhookRequestHMAC",
}
model_config = ConfigDict(
validate_assignment=True,
protected_namespaces=(),
)
def __init__(self, *args, **kwargs) -> None:
if args:
if len(args) > 1:
raise ValueError(
"If a position argument is used, only 1 is allowed to set `actual_instance`"
)
if kwargs:
raise ValueError(
"If a position argument is used, keyword arguments cannot be used."
)
super().__init__(actual_instance=args[0])
else:
super().__init__(**kwargs)
@field_validator("actual_instance")
def actual_instance_must_validate_oneof(cls, v):
instance = V1CreateWebhookRequest.model_construct()
error_messages = []
match = 0
# validate data type: V1CreateWebhookRequestBasicAuth
if not isinstance(v, V1CreateWebhookRequestBasicAuth):
error_messages.append(
f"Error! Input type `{type(v)}` is not `V1CreateWebhookRequestBasicAuth`"
)
else:
match += 1
# validate data type: V1CreateWebhookRequestAPIKey
if not isinstance(v, V1CreateWebhookRequestAPIKey):
error_messages.append(
f"Error! Input type `{type(v)}` is not `V1CreateWebhookRequestAPIKey`"
)
else:
match += 1
# validate data type: V1CreateWebhookRequestHMAC
if not isinstance(v, V1CreateWebhookRequestHMAC):
error_messages.append(
f"Error! Input type `{type(v)}` is not `V1CreateWebhookRequestHMAC`"
)
else:
match += 1
if match > 1:
# more than 1 match
raise ValueError(
"Multiple matches found when setting `actual_instance` in V1CreateWebhookRequest with oneOf schemas: V1CreateWebhookRequestAPIKey, V1CreateWebhookRequestBasicAuth, V1CreateWebhookRequestHMAC. Details: "
+ ", ".join(error_messages)
)
elif match == 0:
# no match
raise ValueError(
"No match found when setting `actual_instance` in V1CreateWebhookRequest with oneOf schemas: V1CreateWebhookRequestAPIKey, V1CreateWebhookRequestBasicAuth, V1CreateWebhookRequestHMAC. Details: "
+ ", ".join(error_messages)
)
else:
return v
@classmethod
def from_dict(cls, obj: Union[str, Dict[str, Any]]) -> Self:
return cls.from_json(json.dumps(obj))
@classmethod
def from_json(cls, json_str: str) -> Self:
"""Returns the object represented by the json string"""
instance = cls.model_construct()
error_messages = []
match = 0
# deserialize data into V1CreateWebhookRequestBasicAuth
try:
instance.actual_instance = V1CreateWebhookRequestBasicAuth.from_json(
json_str
)
match += 1
except (ValidationError, ValueError) as e:
error_messages.append(str(e))
# deserialize data into V1CreateWebhookRequestAPIKey
try:
instance.actual_instance = V1CreateWebhookRequestAPIKey.from_json(json_str)
match += 1
except (ValidationError, ValueError) as e:
error_messages.append(str(e))
# deserialize data into V1CreateWebhookRequestHMAC
try:
instance.actual_instance = V1CreateWebhookRequestHMAC.from_json(json_str)
match += 1
except (ValidationError, ValueError) as e:
error_messages.append(str(e))
if match > 1:
# more than 1 match
raise ValueError(
"Multiple matches found when deserializing the JSON string into V1CreateWebhookRequest with oneOf schemas: V1CreateWebhookRequestAPIKey, V1CreateWebhookRequestBasicAuth, V1CreateWebhookRequestHMAC. Details: "
+ ", ".join(error_messages)
)
elif match == 0:
# no match
raise ValueError(
"No match found when deserializing the JSON string into V1CreateWebhookRequest with oneOf schemas: V1CreateWebhookRequestAPIKey, V1CreateWebhookRequestBasicAuth, V1CreateWebhookRequestHMAC. Details: "
+ ", ".join(error_messages)
)
else:
return instance
def to_json(self) -> str:
"""Returns the JSON representation of the actual instance"""
if self.actual_instance is None:
return "null"
if hasattr(self.actual_instance, "to_json") and callable(
self.actual_instance.to_json
):
return self.actual_instance.to_json()
else:
return json.dumps(self.actual_instance)
def to_dict(
self,
) -> Optional[
Union[
Dict[str, Any],
V1CreateWebhookRequestAPIKey,
V1CreateWebhookRequestBasicAuth,
V1CreateWebhookRequestHMAC,
]
]:
"""Returns the dict representation of the actual instance"""
if self.actual_instance is None:
return None
if hasattr(self.actual_instance, "to_dict") and callable(
self.actual_instance.to_dict
):
return self.actual_instance.to_dict()
else:
# primitive type
return self.actual_instance
def to_str(self) -> str:
"""Returns the string representation of the actual instance"""
return pprint.pformat(self.model_dump())
sdks/python/hatchet_sdk/clients/rest/models/v1_create_webhook_request_api_key.py
+126
View File
@@ -0,0 +1,126 @@
# coding: utf-8
"""
Hatchet API
The Hatchet API
The version of the OpenAPI document: 1.0.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
""" # noqa: E501
from __future__ import annotations
import json
import pprint
import re # noqa: F401
from typing import Any, ClassVar, Dict, List, Optional, Set
from pydantic import BaseModel, ConfigDict, Field, StrictStr, field_validator
from typing_extensions import Self
from hatchet_sdk.clients.rest.models.v1_webhook_api_key_auth import V1WebhookAPIKeyAuth
from hatchet_sdk.clients.rest.models.v1_webhook_source_name import V1WebhookSourceName
class V1CreateWebhookRequestAPIKey(BaseModel):
"""
V1CreateWebhookRequestAPIKey
""" # noqa: E501
source_name: V1WebhookSourceName = Field(
description="The name of the source for this webhook", alias="sourceName"
)
name: StrictStr = Field(description="The name of the webhook")
event_key_expression: StrictStr = Field(
description="The CEL expression to use for the event key. This is used to create the event key from the webhook payload.",
alias="eventKeyExpression",
)
auth_type: StrictStr = Field(
description="The type of authentication to use for the webhook",
alias="authType",
)
auth: V1WebhookAPIKeyAuth
__properties: ClassVar[List[str]] = [
"sourceName",
"name",
"eventKeyExpression",
"authType",
"auth",
]
@field_validator("auth_type")
def auth_type_validate_enum(cls, value):
"""Validates the enum"""
if value not in set(["API_KEY"]):
raise ValueError("must be one of enum values ('API_KEY')")
return value
model_config = ConfigDict(
populate_by_name=True,
validate_assignment=True,
protected_namespaces=(),
)
def to_str(self) -> str:
"""Returns the string representation of the model using alias"""
return pprint.pformat(self.model_dump(by_alias=True))
def to_json(self) -> str:
"""Returns the JSON representation of the model using alias"""
# TODO: pydantic v2: use .model_dump_json(by_alias=True, exclude_unset=True) instead
return json.dumps(self.to_dict())
@classmethod
def from_json(cls, json_str: str) -> Optional[Self]:
"""Create an instance of V1CreateWebhookRequestAPIKey from a JSON string"""
return cls.from_dict(json.loads(json_str))
def to_dict(self) -> Dict[str, Any]:
"""Return the dictionary representation of the model using alias.
This has the following differences from calling pydantic's
`self.model_dump(by_alias=True)`:
* `None` is only added to the output dict for nullable fields that
were set at model initialization. Other fields with value `None`
are ignored.
"""
excluded_fields: Set[str] = set([])
_dict = self.model_dump(
by_alias=True,
exclude=excluded_fields,
exclude_none=True,
)
# override the default output from pydantic by calling `to_dict()` of auth
if self.auth:
_dict["auth"] = self.auth.to_dict()
return _dict
@classmethod
def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]:
"""Create an instance of V1CreateWebhookRequestAPIKey from a dict"""
if obj is None:
return None
if not isinstance(obj, dict):
return cls.model_validate(obj)
_obj = cls.model_validate(
{
"sourceName": obj.get("sourceName"),
"name": obj.get("name"),
"eventKeyExpression": obj.get("eventKeyExpression"),
"authType": obj.get("authType"),
"auth": (
V1WebhookAPIKeyAuth.from_dict(obj["auth"])
if obj.get("auth") is not None
else None
),
}
)
return _obj
sdks/python/hatchet_sdk/clients/rest/models/v1_create_webhook_request_api_key_all_of_auth_type.py
+82
View File
@@ -0,0 +1,82 @@
# coding: utf-8
"""
Hatchet API
The Hatchet API
The version of the OpenAPI document: 1.0.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
""" # noqa: E501
from __future__ import annotations
import json
import pprint
import re # noqa: F401
from typing import Any, ClassVar, Dict, List, Optional, Set
from pydantic import BaseModel, ConfigDict
from typing_extensions import Self
class V1CreateWebhookRequestAPIKeyAllOfAuthType(BaseModel):
"""
V1CreateWebhookRequestAPIKeyAllOfAuthType
""" # noqa: E501
__properties: ClassVar[List[str]] = []
model_config = ConfigDict(
populate_by_name=True,
validate_assignment=True,
protected_namespaces=(),
)
def to_str(self) -> str:
"""Returns the string representation of the model using alias"""
return pprint.pformat(self.model_dump(by_alias=True))
def to_json(self) -> str:
"""Returns the JSON representation of the model using alias"""
# TODO: pydantic v2: use .model_dump_json(by_alias=True, exclude_unset=True) instead
return json.dumps(self.to_dict())
@classmethod
def from_json(cls, json_str: str) -> Optional[Self]:
"""Create an instance of V1CreateWebhookRequestAPIKeyAllOfAuthType from a JSON string"""
return cls.from_dict(json.loads(json_str))
def to_dict(self) -> Dict[str, Any]:
"""Return the dictionary representation of the model using alias.
This has the following differences from calling pydantic's
`self.model_dump(by_alias=True)`:
* `None` is only added to the output dict for nullable fields that
were set at model initialization. Other fields with value `None`
are ignored.
"""
excluded_fields: Set[str] = set([])
_dict = self.model_dump(
by_alias=True,
exclude=excluded_fields,
exclude_none=True,
)
return _dict
@classmethod
def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]:
"""Create an instance of V1CreateWebhookRequestAPIKeyAllOfAuthType from a dict"""
if obj is None:
return None
if not isinstance(obj, dict):
return cls.model_validate(obj)
_obj = cls.model_validate({})
return _obj
sdks/python/hatchet_sdk/clients/rest/models/v1_create_webhook_request_base.py
+98
View File
@@ -0,0 +1,98 @@
# coding: utf-8
"""
Hatchet API
The Hatchet API
The version of the OpenAPI document: 1.0.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
""" # noqa: E501
from __future__ import annotations
import json
import pprint
import re # noqa: F401
from typing import Any, ClassVar, Dict, List, Optional, Set
from pydantic import BaseModel, ConfigDict, Field, StrictStr
from typing_extensions import Self
from hatchet_sdk.clients.rest.models.v1_webhook_source_name import V1WebhookSourceName
class V1CreateWebhookRequestBase(BaseModel):
"""
V1CreateWebhookRequestBase
""" # noqa: E501
source_name: V1WebhookSourceName = Field(
description="The name of the source for this webhook", alias="sourceName"
)
name: StrictStr = Field(description="The name of the webhook")
event_key_expression: StrictStr = Field(
description="The CEL expression to use for the event key. This is used to create the event key from the webhook payload.",
alias="eventKeyExpression",
)
__properties: ClassVar[List[str]] = ["sourceName", "name", "eventKeyExpression"]
model_config = ConfigDict(
populate_by_name=True,
validate_assignment=True,
protected_namespaces=(),
)
def to_str(self) -> str:
"""Returns the string representation of the model using alias"""
return pprint.pformat(self.model_dump(by_alias=True))
def to_json(self) -> str:
"""Returns the JSON representation of the model using alias"""
# TODO: pydantic v2: use .model_dump_json(by_alias=True, exclude_unset=True) instead
return json.dumps(self.to_dict())
@classmethod
def from_json(cls, json_str: str) -> Optional[Self]:
"""Create an instance of V1CreateWebhookRequestBase from a JSON string"""
return cls.from_dict(json.loads(json_str))
def to_dict(self) -> Dict[str, Any]:
"""Return the dictionary representation of the model using alias.
This has the following differences from calling pydantic's
`self.model_dump(by_alias=True)`:
* `None` is only added to the output dict for nullable fields that
were set at model initialization. Other fields with value `None`
are ignored.
"""
excluded_fields: Set[str] = set([])
_dict = self.model_dump(
by_alias=True,
exclude=excluded_fields,
exclude_none=True,
)
return _dict
@classmethod
def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]:
"""Create an instance of V1CreateWebhookRequestBase from a dict"""
if obj is None:
return None
if not isinstance(obj, dict):
return cls.model_validate(obj)
_obj = cls.model_validate(
{
"sourceName": obj.get("sourceName"),
"name": obj.get("name"),
"eventKeyExpression": obj.get("eventKeyExpression"),
}
)
return _obj
sdks/python/hatchet_sdk/clients/rest/models/v1_create_webhook_request_basic_auth.py
+126
View File
@@ -0,0 +1,126 @@
# coding: utf-8
"""
Hatchet API
The Hatchet API
The version of the OpenAPI document: 1.0.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
""" # noqa: E501
from __future__ import annotations
import json
import pprint
import re # noqa: F401
from typing import Any, ClassVar, Dict, List, Optional, Set
from pydantic import BaseModel, ConfigDict, Field, StrictStr, field_validator
from typing_extensions import Self
from hatchet_sdk.clients.rest.models.v1_webhook_basic_auth import V1WebhookBasicAuth
from hatchet_sdk.clients.rest.models.v1_webhook_source_name import V1WebhookSourceName
class V1CreateWebhookRequestBasicAuth(BaseModel):
"""
V1CreateWebhookRequestBasicAuth
""" # noqa: E501
source_name: V1WebhookSourceName = Field(
description="The name of the source for this webhook", alias="sourceName"
)
name: StrictStr = Field(description="The name of the webhook")
event_key_expression: StrictStr = Field(
description="The CEL expression to use for the event key. This is used to create the event key from the webhook payload.",
alias="eventKeyExpression",
)
auth_type: StrictStr = Field(
description="The type of authentication to use for the webhook",
alias="authType",
)
auth: V1WebhookBasicAuth
__properties: ClassVar[List[str]] = [
"sourceName",
"name",
"eventKeyExpression",
"authType",
"auth",
]
@field_validator("auth_type")
def auth_type_validate_enum(cls, value):
"""Validates the enum"""
if value not in set(["BASIC"]):
raise ValueError("must be one of enum values ('BASIC')")
return value
model_config = ConfigDict(
populate_by_name=True,
validate_assignment=True,
protected_namespaces=(),
)
def to_str(self) -> str:
"""Returns the string representation of the model using alias"""
return pprint.pformat(self.model_dump(by_alias=True))
def to_json(self) -> str:
"""Returns the JSON representation of the model using alias"""
# TODO: pydantic v2: use .model_dump_json(by_alias=True, exclude_unset=True) instead
return json.dumps(self.to_dict())
@classmethod
def from_json(cls, json_str: str) -> Optional[Self]:
"""Create an instance of V1CreateWebhookRequestBasicAuth from a JSON string"""
return cls.from_dict(json.loads(json_str))
def to_dict(self) -> Dict[str, Any]:
"""Return the dictionary representation of the model using alias.
This has the following differences from calling pydantic's
`self.model_dump(by_alias=True)`:
* `None` is only added to the output dict for nullable fields that
were set at model initialization. Other fields with value `None`
are ignored.
"""
excluded_fields: Set[str] = set([])
_dict = self.model_dump(
by_alias=True,
exclude=excluded_fields,
exclude_none=True,
)
# override the default output from pydantic by calling `to_dict()` of auth
if self.auth:
_dict["auth"] = self.auth.to_dict()
return _dict
@classmethod
def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]:
"""Create an instance of V1CreateWebhookRequestBasicAuth from a dict"""
if obj is None:
return None
if not isinstance(obj, dict):
return cls.model_validate(obj)
_obj = cls.model_validate(
{
"sourceName": obj.get("sourceName"),
"name": obj.get("name"),
"eventKeyExpression": obj.get("eventKeyExpression"),
"authType": obj.get("authType"),
"auth": (
V1WebhookBasicAuth.from_dict(obj["auth"])
if obj.get("auth") is not None
else None
),
}
)
return _obj
sdks/python/hatchet_sdk/clients/rest/models/v1_create_webhook_request_basic_auth_all_of_auth_type.py
+82
View File
@@ -0,0 +1,82 @@
# coding: utf-8
"""
Hatchet API
The Hatchet API
The version of the OpenAPI document: 1.0.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
""" # noqa: E501
from __future__ import annotations
import json
import pprint
import re # noqa: F401
from typing import Any, ClassVar, Dict, List, Optional, Set
from pydantic import BaseModel, ConfigDict
from typing_extensions import Self
class V1CreateWebhookRequestBasicAuthAllOfAuthType(BaseModel):
"""
V1CreateWebhookRequestBasicAuthAllOfAuthType
""" # noqa: E501
__properties: ClassVar[List[str]] = []
model_config = ConfigDict(
populate_by_name=True,
validate_assignment=True,
protected_namespaces=(),
)
def to_str(self) -> str:
"""Returns the string representation of the model using alias"""
return pprint.pformat(self.model_dump(by_alias=True))
def to_json(self) -> str:
"""Returns the JSON representation of the model using alias"""
# TODO: pydantic v2: use .model_dump_json(by_alias=True, exclude_unset=True) instead
return json.dumps(self.to_dict())
@classmethod
def from_json(cls, json_str: str) -> Optional[Self]:
"""Create an instance of V1CreateWebhookRequestBasicAuthAllOfAuthType from a JSON string"""
return cls.from_dict(json.loads(json_str))
def to_dict(self) -> Dict[str, Any]:
"""Return the dictionary representation of the model using alias.
This has the following differences from calling pydantic's
`self.model_dump(by_alias=True)`:
* `None` is only added to the output dict for nullable fields that
were set at model initialization. Other fields with value `None`
are ignored.
"""
excluded_fields: Set[str] = set([])
_dict = self.model_dump(
by_alias=True,
exclude=excluded_fields,
exclude_none=True,
)
return _dict
@classmethod
def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]:
"""Create an instance of V1CreateWebhookRequestBasicAuthAllOfAuthType from a dict"""
if obj is None:
return None
if not isinstance(obj, dict):
return cls.model_validate(obj)
_obj = cls.model_validate({})
return _obj
sdks/python/hatchet_sdk/clients/rest/models/v1_create_webhook_request_hmac.py
+126
View File
@@ -0,0 +1,126 @@
# coding: utf-8
"""
Hatchet API
The Hatchet API
The version of the OpenAPI document: 1.0.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
""" # noqa: E501
from __future__ import annotations
import json
import pprint
import re # noqa: F401
from typing import Any, ClassVar, Dict, List, Optional, Set
from pydantic import BaseModel, ConfigDict, Field, StrictStr, field_validator
from typing_extensions import Self
from hatchet_sdk.clients.rest.models.v1_webhook_hmac_auth import V1WebhookHMACAuth
from hatchet_sdk.clients.rest.models.v1_webhook_source_name import V1WebhookSourceName
class V1CreateWebhookRequestHMAC(BaseModel):
"""
V1CreateWebhookRequestHMAC
""" # noqa: E501
source_name: V1WebhookSourceName = Field(
description="The name of the source for this webhook", alias="sourceName"
)
name: StrictStr = Field(description="The name of the webhook")
event_key_expression: StrictStr = Field(
description="The CEL expression to use for the event key. This is used to create the event key from the webhook payload.",
alias="eventKeyExpression",
)
auth_type: StrictStr = Field(
description="The type of authentication to use for the webhook",
alias="authType",
)
auth: V1WebhookHMACAuth
__properties: ClassVar[List[str]] = [
"sourceName",
"name",
"eventKeyExpression",
"authType",
"auth",
]
@field_validator("auth_type")
def auth_type_validate_enum(cls, value):
"""Validates the enum"""
if value not in set(["HMAC"]):
raise ValueError("must be one of enum values ('HMAC')")
return value
model_config = ConfigDict(
populate_by_name=True,
validate_assignment=True,
protected_namespaces=(),
)
def to_str(self) -> str:
"""Returns the string representation of the model using alias"""
return pprint.pformat(self.model_dump(by_alias=True))
def to_json(self) -> str:
"""Returns the JSON representation of the model using alias"""
# TODO: pydantic v2: use .model_dump_json(by_alias=True, exclude_unset=True) instead
return json.dumps(self.to_dict())
@classmethod
def from_json(cls, json_str: str) -> Optional[Self]:
"""Create an instance of V1CreateWebhookRequestHMAC from a JSON string"""
return cls.from_dict(json.loads(json_str))
def to_dict(self) -> Dict[str, Any]:
"""Return the dictionary representation of the model using alias.
This has the following differences from calling pydantic's
`self.model_dump(by_alias=True)`:
* `None` is only added to the output dict for nullable fields that
were set at model initialization. Other fields with value `None`
are ignored.
"""
excluded_fields: Set[str] = set([])
_dict = self.model_dump(
by_alias=True,
exclude=excluded_fields,
exclude_none=True,
)
# override the default output from pydantic by calling `to_dict()` of auth
if self.auth:
_dict["auth"] = self.auth.to_dict()
return _dict
@classmethod
def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]:
"""Create an instance of V1CreateWebhookRequestHMAC from a dict"""
if obj is None:
return None
if not isinstance(obj, dict):
return cls.model_validate(obj)
_obj = cls.model_validate(
{
"sourceName": obj.get("sourceName"),
"name": obj.get("name"),
"eventKeyExpression": obj.get("eventKeyExpression"),
"authType": obj.get("authType"),
"auth": (
V1WebhookHMACAuth.from_dict(obj["auth"])
if obj.get("auth") is not None
else None
),
}
)
return _obj
sdks/python/hatchet_sdk/clients/rest/models/v1_create_webhook_request_hmac_all_of_auth_type.py
+82
View File
@@ -0,0 +1,82 @@
# coding: utf-8
"""
Hatchet API
The Hatchet API
The version of the OpenAPI document: 1.0.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
""" # noqa: E501
from __future__ import annotations
import json
import pprint
import re # noqa: F401
from typing import Any, ClassVar, Dict, List, Optional, Set
from pydantic import BaseModel, ConfigDict
from typing_extensions import Self
class V1CreateWebhookRequestHMACAllOfAuthType(BaseModel):
"""
V1CreateWebhookRequestHMACAllOfAuthType
""" # noqa: E501
__properties: ClassVar[List[str]] = []
model_config = ConfigDict(
populate_by_name=True,
validate_assignment=True,
protected_namespaces=(),
)
def to_str(self) -> str:
"""Returns the string representation of the model using alias"""
return pprint.pformat(self.model_dump(by_alias=True))
def to_json(self) -> str:
"""Returns the JSON representation of the model using alias"""
# TODO: pydantic v2: use .model_dump_json(by_alias=True, exclude_unset=True) instead
return json.dumps(self.to_dict())
@classmethod
def from_json(cls, json_str: str) -> Optional[Self]:
"""Create an instance of V1CreateWebhookRequestHMACAllOfAuthType from a JSON string"""
return cls.from_dict(json.loads(json_str))
def to_dict(self) -> Dict[str, Any]:
"""Return the dictionary representation of the model using alias.
This has the following differences from calling pydantic's
`self.model_dump(by_alias=True)`:
* `None` is only added to the output dict for nullable fields that
were set at model initialization. Other fields with value `None`
are ignored.
"""
excluded_fields: Set[str] = set([])
_dict = self.model_dump(
by_alias=True,
exclude=excluded_fields,
exclude_none=True,
)
return _dict
@classmethod
def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]:
"""Create an instance of V1CreateWebhookRequestHMACAllOfAuthType from a dict"""
if obj is None:
return None
if not isinstance(obj, dict):
return cls.model_validate(obj)
_obj = cls.model_validate({})
return _obj
sdks/python/hatchet_sdk/clients/rest/models/v1_event.py
+7
View File
@@ -71,6 +71,11 @@ class V1Event(BaseModel):
description="The external IDs of the runs that were triggered by this event.",
alias="triggeredRuns",
)
triggering_webhook_name: Optional[StrictStr] = Field(
default=None,
description="The name of the webhook that triggered this event, if applicable.",
alias="triggeringWebhookName",
)
__properties: ClassVar[List[str]] = [
"metadata",
"key",
@@ -82,6 +87,7 @@ class V1Event(BaseModel):
"scope",
"seenAt",
"triggeredRuns",
"triggeringWebhookName",
]
model_config = ConfigDict(
@@ -179,6 +185,7 @@ class V1Event(BaseModel):
if obj.get("triggeredRuns") is not None
else None
),
"triggeringWebhookName": obj.get("triggeringWebhookName"),
}
)
return _obj
sdks/python/hatchet_sdk/clients/rest/models/v1_webhook.py
+126
View File
@@ -0,0 +1,126 @@
# coding: utf-8
"""
Hatchet API
The Hatchet API
The version of the OpenAPI document: 1.0.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
""" # noqa: E501
from __future__ import annotations
import json
import pprint
import re # noqa: F401
from typing import Any, ClassVar, Dict, List, Optional, Set
from pydantic import BaseModel, ConfigDict, Field, StrictStr
from typing_extensions import Self
from hatchet_sdk.clients.rest.models.api_resource_meta import APIResourceMeta
from hatchet_sdk.clients.rest.models.v1_webhook_auth_type import V1WebhookAuthType
from hatchet_sdk.clients.rest.models.v1_webhook_source_name import V1WebhookSourceName
class V1Webhook(BaseModel):
"""
V1Webhook
""" # noqa: E501
metadata: APIResourceMeta
tenant_id: StrictStr = Field(
description="The ID of the tenant associated with this webhook.",
alias="tenantId",
)
name: StrictStr = Field(description="The name of the webhook")
source_name: V1WebhookSourceName = Field(
description="The name of the source for this webhook", alias="sourceName"
)
event_key_expression: StrictStr = Field(
description="The CEL expression to use for the event key. This is used to create the event key from the webhook payload.",
alias="eventKeyExpression",
)
auth_type: V1WebhookAuthType = Field(
description="The type of authentication to use for the webhook",
alias="authType",
)
__properties: ClassVar[List[str]] = [
"metadata",
"tenantId",
"name",
"sourceName",
"eventKeyExpression",
"authType",
]
model_config = ConfigDict(
populate_by_name=True,
validate_assignment=True,
protected_namespaces=(),
)
def to_str(self) -> str:
"""Returns the string representation of the model using alias"""
return pprint.pformat(self.model_dump(by_alias=True))
def to_json(self) -> str:
"""Returns the JSON representation of the model using alias"""
# TODO: pydantic v2: use .model_dump_json(by_alias=True, exclude_unset=True) instead
return json.dumps(self.to_dict())
@classmethod
def from_json(cls, json_str: str) -> Optional[Self]:
"""Create an instance of V1Webhook from a JSON string"""
return cls.from_dict(json.loads(json_str))
def to_dict(self) -> Dict[str, Any]:
"""Return the dictionary representation of the model using alias.
This has the following differences from calling pydantic's
`self.model_dump(by_alias=True)`:
* `None` is only added to the output dict for nullable fields that
were set at model initialization. Other fields with value `None`
are ignored.
"""
excluded_fields: Set[str] = set([])
_dict = self.model_dump(
by_alias=True,
exclude=excluded_fields,
exclude_none=True,
)
# override the default output from pydantic by calling `to_dict()` of metadata
if self.metadata:
_dict["metadata"] = self.metadata.to_dict()
return _dict
@classmethod
def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]:
"""Create an instance of V1Webhook from a dict"""
if obj is None:
return None
if not isinstance(obj, dict):
return cls.model_validate(obj)
_obj = cls.model_validate(
{
"metadata": (
APIResourceMeta.from_dict(obj["metadata"])
if obj.get("metadata") is not None
else None
),
"tenantId": obj.get("tenantId"),
"name": obj.get("name"),
"sourceName": obj.get("sourceName"),
"eventKeyExpression": obj.get("eventKeyExpression"),
"authType": obj.get("authType"),
}
)
return _obj
sdks/python/hatchet_sdk/clients/rest/models/v1_webhook_api_key_auth.py
+90
View File
@@ -0,0 +1,90 @@
# coding: utf-8
"""
Hatchet API
The Hatchet API
The version of the OpenAPI document: 1.0.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
""" # noqa: E501
from __future__ import annotations
import json
import pprint
import re # noqa: F401
from typing import Any, ClassVar, Dict, List, Optional, Set
from pydantic import BaseModel, ConfigDict, Field, StrictStr
from typing_extensions import Self
class V1WebhookAPIKeyAuth(BaseModel):
"""
V1WebhookAPIKeyAuth
""" # noqa: E501
header_name: StrictStr = Field(
description="The name of the header to use for the API key", alias="headerName"
)
api_key: StrictStr = Field(
description="The API key to use for authentication", alias="apiKey"
)
__properties: ClassVar[List[str]] = ["headerName", "apiKey"]
model_config = ConfigDict(
populate_by_name=True,
validate_assignment=True,
protected_namespaces=(),
)
def to_str(self) -> str:
"""Returns the string representation of the model using alias"""
return pprint.pformat(self.model_dump(by_alias=True))
def to_json(self) -> str:
"""Returns the JSON representation of the model using alias"""
# TODO: pydantic v2: use .model_dump_json(by_alias=True, exclude_unset=True) instead
return json.dumps(self.to_dict())
@classmethod
def from_json(cls, json_str: str) -> Optional[Self]:
"""Create an instance of V1WebhookAPIKeyAuth from a JSON string"""
return cls.from_dict(json.loads(json_str))
def to_dict(self) -> Dict[str, Any]:
"""Return the dictionary representation of the model using alias.
This has the following differences from calling pydantic's
`self.model_dump(by_alias=True)`:
* `None` is only added to the output dict for nullable fields that
were set at model initialization. Other fields with value `None`
are ignored.
"""
excluded_fields: Set[str] = set([])
_dict = self.model_dump(
by_alias=True,
exclude=excluded_fields,
exclude_none=True,
)
return _dict
@classmethod
def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]:
"""Create an instance of V1WebhookAPIKeyAuth from a dict"""
if obj is None:
return None
if not isinstance(obj, dict):
return cls.model_validate(obj)
_obj = cls.model_validate(
{"headerName": obj.get("headerName"), "apiKey": obj.get("apiKey")}
)
return _obj
sdks/python/hatchet_sdk/clients/rest/models/v1_webhook_auth_type.py
+38
View File
@@ -0,0 +1,38 @@
# coding: utf-8
"""
Hatchet API
The Hatchet API
The version of the OpenAPI document: 1.0.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
""" # noqa: E501
from __future__ import annotations
import json
from enum import Enum
from typing_extensions import Self
class V1WebhookAuthType(str, Enum):
"""
V1WebhookAuthType
"""
"""
allowed enum values
"""
BASIC = "BASIC"
API_KEY = "API_KEY"
HMAC = "HMAC"
@classmethod
def from_json(cls, json_str: str) -> Self:
"""Create an instance of V1WebhookAuthType from a JSON string"""
return cls(json.loads(json_str))
sdks/python/hatchet_sdk/clients/rest/models/v1_webhook_basic_auth.py
+86
View File
@@ -0,0 +1,86 @@
# coding: utf-8
"""
Hatchet API
The Hatchet API
The version of the OpenAPI document: 1.0.0
Generated by OpenAPI Generator (https://openapi-generator.tech)
Do not edit the class manually.
""" # noqa: E501
from __future__ import annotations
import json
import pprint
import re # noqa: F401
from typing import Any, ClassVar, Dict, List, Optional, Set
from pydantic import BaseModel, ConfigDict, Field, StrictStr
from typing_extensions import Self
class V1WebhookBasicAuth(BaseModel):
"""
V1WebhookBasicAuth
""" # noqa: E501
username: StrictStr = Field(description="The username for basic auth")
password: StrictStr = Field(description="The password for basic auth")
__properties: ClassVar[List[str]] = ["username", "password"]
model_config = ConfigDict(
populate_by_name=True,
validate_assignment=True,
protected_namespaces=(),
)
def to_str(self) -> str:
"""Returns the string representation of the model using alias"""
return pprint.pformat(self.model_dump(by_alias=True))
def to_json(self) -> str:
"""Returns the JSON representation of the model using alias"""
# TODO: pydantic v2: use .model_dump_json(by_alias=True, exclude_unset=True) instead
return json.dumps(self.to_dict())
@classmethod
def from_json(cls, json_str: str) -> Optional[Self]:
"""Create an instance of V1WebhookBasicAuth from a JSON string"""
return cls.from_dict(json.loads(json_str))
def to_dict(self) -> Dict[str, Any]:
"""Return the dictionary representation of the model using alias.
This has the following differences from calling pydantic's
`self.model_dump(by_alias=True)`:
* `None` is only added to the output dict for nullable fields that
were set at model initialization. Other fields with value `None`
are ignored.
"""
excluded_fields: Set[str] = set([])
_dict = self.model_dump(
by_alias=True,
exclude=excluded_fields,
exclude_none=True,
)
return _dict
@classmethod
def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]:
"""Create an instance of V1WebhookBasicAuth from a dict"""
if obj is None:
return None
if not isinstance(obj, dict):
return cls.model_validate(obj)
_obj = cls.model_validate(
{"username": obj.get("username"), "password": obj.get("password")}
)
return _obj

Some files were not shown because too many files have changed in this diff Show More