Compare commits

...

14 Commits

Author SHA1 Message Date
Corentin Thomasset
5382019721 chore(release): update versions (#420)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-08-08 21:16:08 +02:00
Corentin Thomasset
b33fde35d3 feat(auth): improved feedback for invalid origin url (#455) 2025-08-08 18:10:54 +02:00
Corentin Thomasset
fd6f83f538 refactor(migrations): purged legacy migrations (#453) 2025-08-08 02:31:13 +02:00
Corentin Thomasset
7f7e5bffcb refactor(database): completely rewrote the db migration tooling (#452) 2025-08-08 02:18:22 +02:00
Corentin Thomasset
5868800bce fix(tags): fixed the impossibility to delete a tag that have been affected to a document (#448)
* fix(tags): fixed the impossibility to delete a tag that have been affected to a document

- Added user feedback for errors encountered during tag deletion in the client.
- Updated localization files to include new error messages for internal processing issues across multiple languages.
- Modified the document activity log migration to set foreign keys to null on delete for user and tag references.

* Update .changeset/green-teeth-fall.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update .changeset/cuddly-shoes-watch.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-08-02 00:41:43 +02:00
Corentin Thomasset
b5ccc135ba refactor(documents): document content extraction is now async (#447)
* refactor(documents): implement asynchronous document content extraction

- Updated dependencies for `@cadence-mq/core` and `@cadence-mq/driver-memory` to versions 0.2.1 and 0.2.0 respectively.
- Introduced a new task for extracting document file content asynchronously.
- Refactored document creation use case to schedule the extraction task.
- Added utility functions for stream conversion and text extraction from files.
- Updated relevant tests to accommodate the new asynchronous behavior and task services integration.

* Update .changeset/cyan-pots-begin.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-08-01 21:44:29 +00:00
Manuel Zavatta
5e46bb9e6a feat(i18n): added Italian translation
* Create it.yml

cloned from en.yml

* Update it.yml

italian translation

* Update i18n.constants.ts

* fix(i18n): lint and auto order

* chore(versioning): added changeset

---------

Co-authored-by: Corentin Thomasset <corentin.thomasset74@gmail.com>
2025-07-31 21:15:43 +00:00
Corentin Thomasset
41a113334a refactor(tasks): integrated cadence task services (#436) 2025-07-28 18:30:11 +00:00
Corentin Thomasset
6723baf98a feat(webhooks): add document update and tag events (#432) 2025-07-25 16:46:05 +02:00
Corentin Thomasset
bbe5fe74e2 test(lecture): added fixture test timeout (#431) 2025-07-25 12:56:46 +00:00
Corentin Thomasset
a8cff8cedc refactor(webhooks): updated webhooks signatures and payload to match standard-webhook spec (#430) 2025-07-25 11:29:26 +02:00
Corentin Thomasset
67b3b14cdf feat(lecture): added ocr support for scanned pdf (#429) 2025-07-24 22:21:10 +02:00
Osaf Ali Sayed
ffdae8db56 feat(intake-emails): redesigned intake email list (#412)
* feat(intake-emails): redesigned intake email list

* fix(intake-emails): fix linting

* fix(intake-emails): set drop down menu trigger size same as icon

* chore(version): added changeset

---------

Co-authored-by: Corentin Thomasset <corentin.thomasset74@gmail.com>
2025-07-14 13:28:48 +00:00
Edward205
7768840aa4 refactor(i18n): improved Romanian translation (#419)
* added diacritics and improved wording

* chore(version): added changeset

---------

Co-authored-by: Corentin Thomasset <corentin.thomasset74@gmail.com>
2025-07-14 11:36:10 +00:00
125 changed files with 5781 additions and 1819 deletions

View File

@@ -1,5 +1,11 @@
# @papra/docs
## 0.5.3
### Patch Changes
- [#455](https://github.com/papra-hq/papra/pull/455) [`b33fde3`](https://github.com/papra-hq/papra/commit/b33fde35d3e8622e31b51aadfe56875d8e48a2ef) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Improved feedback message in case of invalid origin configuration
## 0.5.2
### Patch Changes

View File

@@ -1,7 +1,7 @@
{
"name": "@papra/docs",
"type": "module",
"version": "0.5.2",
"version": "0.5.3",
"private": true,
"packageManager": "pnpm@10.12.3",
"description": "Papra documentation website",

View File

@@ -31,6 +31,7 @@ Launch Papra with default configuration using:
docker run -d \
--name papra \
--restart unless-stopped \
--env APP_BASE_URL=http://localhost:1221 \
-p 1221:1221 \
ghcr.io/papra-hq/papra:latest
```
@@ -69,6 +70,7 @@ For production deployments, mount host directories to preserve application data
docker run -d \
--name papra \
--restart unless-stopped \
--env APP_BASE_URL=http://localhost:1221 \
-p 1221:1221 \
-v $(pwd)/papra-data:/app/app-data \
--user $(id -u):$(id -g) \

View File

@@ -24,5 +24,17 @@ To fix this, you can either:
- Ensure that the directory is owned by the user running the container
- Run the server as root (not recommended)
## Invalid application origin
Papra ensures [CSRF](https://en.wikipedia.org/wiki/Cross-site_request_forgery) protection by validating the Origin header in requests. This check ensures that requests originate from the application or a trusted source. Any request that does not originate from a trusted origin will be rejected.
If you are self-hosting Papra, you may encounter an error stating that the application origin is invalid while trying to login or register.
To fix this, you can either:
- Update the `APP_BASE_URL` environment variable to match the url of your application (e.g. `https://papra.my-homelab.tld`)
- Add the current url to the `TRUSTED_ORIGINS` environment variable if you need to allow multiple origins, comma separated. By default the `TRUSTED_ORIGINS` is set to the `APP_BASE_URL`
- If you are using a reverse proxy, you may need to add the url to the `TRUSTED_ORIGINS` environment variable

View File

@@ -1,5 +1,25 @@
# @papra/app-client
## 0.8.0
### Minor Changes
- [#432](https://github.com/papra-hq/papra/pull/432) [`6723baf`](https://github.com/papra-hq/papra/commit/6723baf98ad46f989fe1e1e19ad0dd25622cca77) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Added new webhook events: document:updated, document:tag:added, document:tag:removed
- [#432](https://github.com/papra-hq/papra/pull/432) [`6723baf`](https://github.com/papra-hq/papra/commit/6723baf98ad46f989fe1e1e19ad0dd25622cca77) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Webhooks invocation is now defered
### Patch Changes
- [#419](https://github.com/papra-hq/papra/pull/419) [`7768840`](https://github.com/papra-hq/papra/commit/7768840aa4425a03cb96dc1c17605bfa8e6a0de4) Thanks [@Edward205](https://github.com/Edward205)! - Added diacritics and improved wording for Romanian translation
- [#448](https://github.com/papra-hq/papra/pull/448) [`5868800`](https://github.com/papra-hq/papra/commit/5868800bcec6ed69b5441b50e4445fae5cdb5bfb) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Added feedback when an error occurs while deleting a tag
- [#412](https://github.com/papra-hq/papra/pull/412) [`ffdae8d`](https://github.com/papra-hq/papra/commit/ffdae8db56c6ecfe63eb263ee606e9469eef8874) Thanks [@OsafAliSayed](https://github.com/OsafAliSayed)! - Simplified the organization intake email list
- [#441](https://github.com/papra-hq/papra/pull/441) [`5e46bb9`](https://github.com/papra-hq/papra/commit/5e46bb9e6a39cd16a83636018370607a27db042a) Thanks [@Zavy86](https://github.com/Zavy86)! - Added Italian (it) language support
- [#455](https://github.com/papra-hq/papra/pull/455) [`b33fde3`](https://github.com/papra-hq/papra/commit/b33fde35d3e8622e31b51aadfe56875d8e48a2ef) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Improved feedback message in case of invalid origin configuration
## 0.7.0
### Minor Changes

View File

@@ -1,7 +1,7 @@
{
"name": "@papra/app-client",
"type": "module",
"version": "0.7.0",
"version": "0.8.0",
"private": true,
"packageManager": "pnpm@10.12.3",
"description": "Papra frontend client",

View File

@@ -489,8 +489,12 @@ webhooks.delete.confirm.message: Sind Sie sicher, dass Sie diesen Webhook lösch
webhooks.delete.confirm.confirm-button: Löschen
webhooks.delete.confirm.cancel-button: Abbrechen
webhooks.events.documents.title: Dokumente Ereignisse
webhooks.events.documents.document:created.description: Dokument erstellt
webhooks.events.documents.document:deleted.description: Dokument gelöscht
webhooks.events.documents.document:updated.description: Dokument aktualisiert
webhooks.events.documents.document:tag:added.description: Ein Tag wurde zu einem Dokument hinzugefügt
webhooks.events.documents.document:tag:removed.description: Ein Tag wurde von einem Dokument entfernt
# Navigation
@@ -541,6 +545,8 @@ api-errors.user.already_in_organization: Dieser Benutzer ist bereits in dieser O
api-errors.user.organization_invitation_limit_reached: Die maximale Anzahl an Einladungen für heute wurde erreicht. Bitte versuchen Sie es morgen erneut.
api-errors.demo.not_available: Diese Funktion ist in der Demo nicht verfügbar
api-errors.tags.already_exists: Ein Tag mit diesem Namen existiert bereits für diese Organisation
api-errors.internal.error: Beim Verarbeiten Ihrer Anfrage ist ein Fehler aufgetreten. Bitte versuchen Sie es erneut.
api-errors.auth.invalid_origin: Ungültige Anwendungs-Ursprung. Wenn Sie Papra selbst hosten, stellen Sie sicher, dass Ihre APP_BASE_URL-Umgebungsvariable mit Ihrer aktuellen URL übereinstimmt. Weitere Details finden Sie unter https://docs.papra.app/resources/troubleshooting/#invalid-application-origin
# Not found

View File

@@ -489,8 +489,12 @@ webhooks.delete.confirm.message: Are you sure you want to delete this webhook?
webhooks.delete.confirm.confirm-button: Delete
webhooks.delete.confirm.cancel-button: Cancel
webhooks.events.documents.title: Documents events
webhooks.events.documents.document:created.description: Document created
webhooks.events.documents.document:deleted.description: Document deleted
webhooks.events.documents.document:updated.description: Document updated
webhooks.events.documents.document:tag:added.description: A tag is added to a document
webhooks.events.documents.document:tag:removed.description: A tag is removed from a document
# Navigation
@@ -541,6 +545,8 @@ api-errors.user.already_in_organization: This user is already in this organizati
api-errors.user.organization_invitation_limit_reached: The maximum number of invitations has been reached for today. Please try again tomorrow.
api-errors.demo.not_available: This feature is not available in demo
api-errors.tags.already_exists: A tag with this name already exists for this organization
api-errors.internal.error: An error occurred while processing your request. Please try again later.
api-errors.auth.invalid_origin: Invalid application origin. If you are self-hosting Papra, ensure your APP_BASE_URL environment variable matches your current url. For more details see https://docs.papra.app/resources/troubleshooting/#invalid-application-origin
# Not found

View File

@@ -489,8 +489,12 @@ webhooks.delete.confirm.message: ¿Estás seguro de que deseas eliminar este web
webhooks.delete.confirm.confirm-button: Eliminar
webhooks.delete.confirm.cancel-button: Cancelar
webhooks.events.documents.title: Eventos de documentos
webhooks.events.documents.document:created.description: Documento creado
webhooks.events.documents.document:deleted.description: Documento eliminado
webhooks.events.documents.document:updated.description: Documento actualizado
webhooks.events.documents.document:tag:added.description: Una etiqueta se ha añadido a un documento
webhooks.events.documents.document:tag:removed.description: Una etiqueta se ha eliminado de un documento
# Navigation
@@ -541,6 +545,8 @@ api-errors.user.already_in_organization: Este usuario ya está en esta organizac
api-errors.user.organization_invitation_limit_reached: Se ha alcanzado el número máximo de invitaciones para hoy. Por favor, inténtalo de nuevo mañana.
api-errors.demo.not_available: Esta función no está disponible en la demostración
api-errors.tags.already_exists: Ya existe una etiqueta con este nombre en esta organización
api-errors.internal.error: Ocurrió un error al procesar tu solicitud. Por favor, inténtalo de nuevo.
api-errors.auth.invalid_origin: Origen de la aplicación inválido. Si estás alojando Papra, asegúrate de que la variable de entorno APP_BASE_URL coincida con tu URL actual. Para más detalles, consulta https://docs.papra.app/resources/troubleshooting/#invalid-application-origin
# Not found

View File

@@ -489,8 +489,12 @@ webhooks.delete.confirm.message: Êtes-vous sûr de vouloir supprimer ce webhook
webhooks.delete.confirm.confirm-button: Supprimer
webhooks.delete.confirm.cancel-button: Annuler
webhooks.events.documents.title: Événements de documents
webhooks.events.documents.document:created.description: Document créé
webhooks.events.documents.document:deleted.description: Document supprimé
webhooks.events.documents.document:updated.description: Document mis à jour
webhooks.events.documents.document:tag:added.description: Un tag est ajouté à un document
webhooks.events.documents.document:tag:removed.description: Un tag est retiré d'un document
# Navigation
@@ -541,6 +545,8 @@ api-errors.user.already_in_organization: Cet utilisateur est déjà dans cette o
api-errors.user.organization_invitation_limit_reached: Le nombre maximum d'invitations a été atteint pour aujourd'hui. Veuillez réessayer demain.
api-errors.demo.not_available: Cette fonctionnalité n'est pas disponible dans la démo
api-errors.tags.already_exists: Un tag avec ce nom existe déjà pour cette organisation
api-errors.internal.error: Une erreur est survenue lors du traitement de votre requête. Veuillez réessayer.
api-errors.auth.invalid_origin: Origine de l'application invalide. Si vous hébergez Papra, assurez-vous que la variable d'environnement APP_BASE_URL correspond à votre URL actuelle. Pour plus de détails, consultez https://docs.papra.app/resources/troubleshooting/#invalid-application-origin
# Not found

View File

@@ -0,0 +1,571 @@
# Authentication
auth.request-password-reset.title: Reimposta la tua password
auth.request-password-reset.description: Inserisci la tua email per reimpostare la password.
auth.request-password-reset.requested: Se esiste un account per questa email, ti abbiamo inviato un'email per reimpostare la password.
auth.request-password-reset.back-to-login: Torna al login
auth.request-password-reset.form.email.label: Email
auth.request-password-reset.form.email.placeholder: 'Esempio: ada@papra.app'
auth.request-password-reset.form.email.required: Inserisci il tuo indirizzo email
auth.request-password-reset.form.email.invalid: Questo indirizzo email non è valido
auth.request-password-reset.form.submit: Richiedi reimpostazione password
auth.reset-password.title: Reimposta la tua password
auth.reset-password.description: Inserisci la nuova password per reimpostare la password.
auth.reset-password.reset: La tua password è stata reimpostata.
auth.reset-password.back-to-login: Torna al login
auth.reset-password.form.new-password.label: Nuova password
auth.reset-password.form.new-password.placeholder: 'Esempio: **********'
auth.reset-password.form.new-password.required: Inserisci la tua nuova password
auth.reset-password.form.new-password.min-length: La password deve essere di almeno {{ minLength }} caratteri
auth.reset-password.form.new-password.max-length: La password deve essere inferiore a {{ maxLength }} caratteri
auth.reset-password.form.submit: Reimposta password
auth.email-provider.open: Apri {{ provider }}
auth.login.title: Accedi a Papra
auth.login.description: Inserisci la tua email o usa un provider per accedere al tuo account Papra.
auth.login.login-with-provider: Accedi con {{ provider }}
auth.login.no-account: Non hai un account?
auth.login.register: Registrati
auth.login.form.email.label: Email
auth.login.form.email.placeholder: 'Esempio: ada@papra.app'
auth.login.form.email.required: Inserisci il tuo indirizzo email
auth.login.form.email.invalid: Questo indirizzo email non è valido
auth.login.form.password.label: Password
auth.login.form.password.placeholder: Imposta una password
auth.login.form.password.required: Inserisci la tua password
auth.login.form.remember-me.label: Ricordami
auth.login.form.forgot-password.label: Password dimenticata?
auth.login.form.submit: Accedi
auth.register.title: Registrati a Papra
auth.register.description: Crea un account per iniziare a usare Papra.
auth.register.register-with-email: Registrati tramite email
auth.register.register-with-provider: Registrati tramite {{ provider }}
auth.register.providers.google: Google
auth.register.providers.github: GitHub
auth.register.have-account: Hai già un account?
auth.register.login: Accedi
auth.register.registration-disabled.title: Registrazione disabilitata
auth.register.registration-disabled.description: La creazione di nuovi account è attualmente disabilitata su questa istanza di Papra. Solo gli utenti con account esistenti possono accedere. Se pensi che sia un errore, contatta l'amministratore di questa istanza.
auth.register.form.email.label: Email
auth.register.form.email.placeholder: 'Esempio: ada@papra.app'
auth.register.form.email.required: Inserisci il tuo indirizzo email
auth.register.form.email.invalid: Questo indirizzo email non è valido
auth.register.form.password.label: Password
auth.register.form.password.placeholder: Imposta una password
auth.register.form.password.required: Inserisci la tua password
auth.register.form.password.min-length: La password deve essere di almeno {{ minLength }} caratteri
auth.register.form.password.max-length: La password deve essere inferiore a {{ maxLength }} caratteri
auth.register.form.name.label: Nome
auth.register.form.name.placeholder: 'Esempio: Ada Lovelace'
auth.register.form.name.required: Inserisci il tuo nome
auth.register.form.name.max-length: Il nome deve essere inferiore a {{ maxLength }} caratteri
auth.register.form.submit: Registrati
auth.email-validation-required.title: Verifica la tua email
auth.email-validation-required.description: Una email di verifica è stata inviata al tuo indirizzo email. Verifica il tuo indirizzo cliccando il link nell'email.
auth.legal-links.description: Continuando, confermi di aver letto e accettato i {{ terms }} e l'{{ privacy }}.
auth.legal-links.terms: Termini di servizio
auth.legal-links.privacy: Informativa sulla privacy
auth.no-auth-provider.title: Nessun provider di autenticazione
auth.no-auth-provider.description: Nessun provider di autenticazione è abilitato su questa istanza di Papra. Contatta l'amministratore di questa istanza per abilitarli.
# User settings
user.settings.title: Impostazioni utente
user.settings.description: Gestisci qui le impostazioni del tuo account.
user.settings.email.title: Indirizzo email
user.settings.email.description: Il tuo indirizzo email non può essere modificato.
user.settings.email.label: Indirizzo email
user.settings.name.title: Nome completo
user.settings.name.description: Il tuo nome completo è visibile agli altri membri dell'organizzazione.
user.settings.name.label: Nome completo
user.settings.name.placeholder: Es. Mario Rossi
user.settings.name.update: Aggiorna nome
user.settings.name.updated: Il tuo nome completo è stato aggiornato
user.settings.logout.title: Logout
user.settings.logout.description: Esci dal tuo account. Potrai accedere nuovamente in seguito.
user.settings.logout.button: Esci
# Organizations
organizations.list.title: Le tue organizzazioni
organizations.list.description: Le organizzazioni sono un modo per raggruppare i tuoi documenti e gestire l'accesso. Puoi creare più organizzazioni e invitare i tuoi collaboratori.
organizations.list.create-new: Crea una nuova organizzazione
organizations.details.no-documents.title: Nessun documento
organizations.details.no-documents.description: Non ci sono ancora documenti in questa organizzazione. Inizia caricando dei documenti.
organizations.details.upload-documents: Carica documenti
organizations.details.documents-count: documenti in totale
organizations.details.total-size: dimensione totale
organizations.details.latest-documents: Ultimi documenti importati
organizations.create.title: Crea una nuova organizzazione
organizations.create.description: I tuoi documenti saranno raggruppati per organizzazione. Puoi creare più organizzazioni per separare i documenti, ad esempio per uso personale e lavorativo.
organizations.create.back: Indietro
organizations.create.error.max-count-reached: Hai raggiunto il numero massimo di organizzazioni che puoi creare, se hai bisogno di crearne altre contatta il supporto.
organizations.create.form.name.label: Nome organizzazione
organizations.create.form.name.placeholder: Es. Acme Inc.
organizations.create.form.name.required: Inserisci il nome dell'organizzazione
organizations.create.form.submit: Crea organizzazione
organizations.create.success: Organizzazione creata con successo
organizations.create-first.title: Crea la tua organizzazione
organizations.create-first.description: I tuoi documenti saranno raggruppati per organizzazione. Puoi creare più organizzazioni per separare i documenti, ad esempio per uso personale e lavorativo.
organizations.create-first.default-name: La mia organizzazione
organizations.create-first.user-name: 'Organizzazione di {{ name }}'
organization.settings.title: Impostazioni organizzazione
organization.settings.page.title: Impostazioni organizzazione
organization.settings.page.description: Gestisci qui le impostazioni della tua organizzazione.
organization.settings.name.title: Nome organizzazione
organization.settings.name.update: Aggiorna nome
organization.settings.name.placeholder: Es. Acme Inc.
organization.settings.name.updated: Nome organizzazione aggiornato
organization.settings.subscription.title: Sottoscrizione
organization.settings.subscription.description: Gestisci fatturazione, fatture e metodi di pagamento.
organization.settings.subscription.manage: Gestisci sottoscrizione
organization.settings.subscription.error: Impossibile ottenere l'URL del portale clienti
organization.settings.delete.title: Elimina organizzazione
organization.settings.delete.description: Eliminando questa organizzazione rimuoverai definitivamente tutti i dati associati.
organization.settings.delete.confirm.title: Elimina organizzazione
organization.settings.delete.confirm.message: Sei sicuro di voler eliminare questa organizzazione? Questa azione non può essere annullata e tutti i dati associati saranno rimossi in modo permanente.
organization.settings.delete.confirm.confirm-button: Elimina organizzazione
organization.settings.delete.confirm.cancel-button: Annulla
organization.settings.delete.success: Organizzazione eliminata
organizations.members.title: Membri
organizations.members.description: Gestisci i membri della tua organizzazione
organizations.members.invite-member: Invita membro
organizations.members.invite-member-disabled-tooltip: Solo gli amministratori o i proprietari possono invitare membri nell'organizzazione
organizations.members.remove-from-organization: Rimuovi dall'organizzazione
organizations.members.role: Ruolo
organizations.members.roles.owner: Proprietario
organizations.members.roles.admin: Amministratore
organizations.members.roles.member: Membro
organizations.members.delete.confirm.title: Rimuovi membro
organizations.members.delete.confirm.message: Sei sicuro di voler rimuovere questo membro dall'organizzazione?
organizations.members.delete.confirm.confirm-button: Rimuovi
organizations.members.delete.confirm.cancel-button: Annulla
organizations.members.delete.success: Membro rimosso dall'organizzazione
organizations.members.update-role.success: Ruolo del membro aggiornato
organizations.members.table.headers.name: Nome
organizations.members.table.headers.email: Email
organizations.members.table.headers.role: Ruolo
organizations.members.table.headers.created: Creato
organizations.members.table.headers.actions: Azioni
organizations.invite-member.title: Invita membro
organizations.invite-member.description: Invita un membro nella tua organizzazione
organizations.invite-member.form.email.label: Email
organizations.invite-member.form.email.placeholder: 'Esempio: ada@papra.app'
organizations.invite-member.form.email.required: Inserisci un indirizzo email valido
organizations.invite-member.form.role.label: Ruolo
organizations.invite-member.form.submit: Invita nell'organizzazione
organizations.invite-member.success.message: Membro invitato
organizations.invite-member.success.description: Il membro è stato invitato nell'organizzazione.
organizations.invite-member.error.message: Impossibile invitare il membro
organizations.invitations.title: Inviti
organizations.invitations.description: Gestisci gli inviti della tua organizzazione
organizations.invitations.list.cta: Invita membro
organizations.invitations.list.empty.title: Nessun invito in sospeso
organizations.invitations.list.empty.description: Non sei stato ancora invitato in nessuna organizzazione.
organizations.invitations.status.pending: In sospeso
organizations.invitations.status.accepted: Accettato
organizations.invitations.status.rejected: Rifiutato
organizations.invitations.status.expired: Scaduto
organizations.invitations.status.cancelled: Cancellato
organizations.invitations.resend: Invia di nuovo invito
organizations.invitations.cancel.title: Annulla invito
organizations.invitations.cancel.description: Sei sicuro di voler annullare questo invito?
organizations.invitations.cancel.confirm: Annulla invito
organizations.invitations.cancel.cancel: Annulla
organizations.invitations.resend.title: Invia di nuovo invito
organizations.invitations.resend.description: Sei sicuro di voler inviare nuovamente questo invito? Sarà inviata una nuova email al destinatario.
organizations.invitations.resend.confirm: Invia invito
organizations.invitations.resend.cancel: Annulla
invitations.list.title: Inviti
invitations.list.description: Gestisci gli inviti della tua organizzazione
invitations.list.empty.title: Nessun invito in sospeso
invitations.list.empty.description: Non sei stato ancora invitato in nessuna organizzazione.
invitations.list.headers.organization: Organizzazione
invitations.list.headers.status: Stato
invitations.list.headers.created: Creato
invitations.list.headers.actions: Azioni
invitations.list.actions.accept: Accetta
invitations.list.actions.reject: Rifiuta
invitations.list.actions.accept.success.message: Invito accettato
invitations.list.actions.accept.success.description: L'invito è stato accettato.
invitations.list.actions.reject.success.message: Invito rifiutato
invitations.list.actions.reject.success.description: L'invito è stato rifiutato.
# Documents
documents.list.title: Documenti
documents.list.no-documents.title: Nessun documento
documents.list.no-documents.description: Non ci sono ancora documenti in questa organizzazione. Inizia caricando dei documenti.
documents.list.no-results: Nessun documento trovato
documents.tabs.info: Info
documents.tabs.content: Contenuto
documents.tabs.activity: Attività
documents.deleted.message: Questo documento è stato eliminato e sarà rimosso definitivamente tra {{ days }} giorni.
documents.actions.download: Scarica
documents.actions.open-in-new-tab: Apri in una nuova scheda
documents.actions.restore: Ripristina
documents.actions.delete: Elimina
documents.actions.edit: Modifica
documents.actions.cancel: Annulla
documents.actions.save: Salva
documents.actions.saving: Salvataggio in corso...
documents.content.alert: Il contenuto del documento è estratto automaticamente al caricamento. È usato solo per la ricerca e l'indicizzazione.
documents.info.id: ID
documents.info.name: Nome
documents.info.type: Tipo
documents.info.size: Dimensione
documents.info.created-at: Creato il
documents.info.updated-at: Aggiornato il
documents.info.never: Mai
documents.rename.title: Rinomina documento
documents.rename.form.name.label: Nome
documents.rename.form.name.placeholder: 'Esempio: Fattura 2024'
documents.rename.form.name.required: Inserisci un nome per il documento
documents.rename.form.name.max-length: Il nome deve essere inferiore a 255 caratteri
documents.rename.form.submit: Rinomina documento
documents.rename.success: Documento rinominato con successo
documents.rename.cancel: Annulla
import-documents.title.error: '{{ count }} documenti non importati'
import-documents.title.success: '{{ count }} documenti importati'
import-documents.title.pending: '{{ count }} / {{ total }} documenti importati'
import-documents.title.none: Importa documenti
import-documents.no-import-in-progress: Nessuna importazione documenti in corso
documents.deleted.title: Documenti eliminati
documents.deleted.empty.title: Nessun documento eliminato
documents.deleted.empty.description: Non hai documenti eliminati. I documenti eliminati saranno spostati nel cestino per {{ days }} giorni.
documents.deleted.retention-notice: Tutti i documenti eliminati sono conservati nel cestino per {{ days }} giorni. Passato questo periodo, saranno eliminati definitivamente e non potrai recuperarli.
documents.deleted.deleted-at: Eliminato il
documents.deleted.restoring: Ripristino in corso...
documents.deleted.deleting: Eliminazione in corso...
documents.preview.unknown-file-type: Nessuna anteprima disponibile per questo tipo di file
documents.preview.binary-file: Sembra essere un file binario e non può essere visualizzato come testo
trash.delete-all.button: Elimina tutto
trash.delete-all.confirm.title: Eliminare definitivamente tutti i documenti?
trash.delete-all.confirm.description: Sei sicuro di voler eliminare definitivamente tutti i documenti dal cestino? Questa azione non può essere annullata.
trash.delete-all.confirm.label: Elimina
trash.delete-all.confirm.cancel: Annulla
trash.delete.button: Elimina
trash.delete.confirm.title: Eliminare definitivamente il documento?
trash.delete.confirm.description: Sei sicuro di voler eliminare definitivamente questo documento dal cestino? Questa azione non può essere annullata.
trash.delete.confirm.label: Elimina
trash.delete.confirm.cancel: Annulla
trash.deleted.success.title: Documento eliminato
trash.deleted.success.description: Il documento è stato eliminato definitivamente.
activity.document.created: Documento creato
activity.document.updated.single: Il campo {{ field }} è stato aggiornato
activity.document.updated.multiple: I campi {{ fields }} sono stati aggiornati
activity.document.updated: Documento aggiornato
activity.document.deleted: Documento eliminato
activity.document.restored: Documento ripristinato
activity.document.tagged: Tag {{ tag }} aggiunto
activity.document.untagged: Tag {{ tag }} rimosso
activity.document.user.name: da {{ name }}
activity.load-more: Carica altri
activity.no-more-activities: Nessuna altra attività per questo documento
# Tags
tags.no-tags.title: Nessun tag
tags.no-tags.description: Questa organizzazione non ha ancora tag. I tag vengono usati per categorizzare i documenti. Puoi aggiungere tag ai tuoi documenti per trovarli e organizzarli più facilmente.
tags.no-tags.create-tag: Crea tag
tags.title: Tag dei documenti
tags.description: I tag vengono usati per categorizzare i documenti. Puoi aggiungere tag ai tuoi documenti per trovarli e organizzarli più facilmente.
tags.create: Crea tag
tags.update: Aggiorna tag
tags.delete: Elimina tag
tags.delete.confirm.title: Elimina tag
tags.delete.confirm.message: Sei sicuro di voler eliminare questo tag? Il tag verrà rimosso da tutti i documenti.
tags.delete.confirm.confirm-button: Elimina
tags.delete.confirm.cancel-button: Annulla
tags.delete.success: Tag eliminato con successo
tags.create.success: Tag "{{ name }}" creato con successo.
tags.update.success: Tag "{{ name }}" aggiornato con successo.
tags.form.name.label: Nome
tags.form.name.placeholder: Es. Contratti
tags.form.name.required: Inserisci un nome per il tag
tags.form.name.max-length: Il nome del tag deve essere inferiore a 64 caratteri
tags.form.color.label: Colore
tags.form.color.required: Inserisci un colore
tags.form.color.invalid: Il colore hex non è formattato correttamente.
tags.form.description.label: Descrizione
tags.form.description.optional: (opzionale)
tags.form.description.placeholder: Es. Tutti i contratti firmati dall'azienda
tags.form.description.max-length: La descrizione deve essere inferiore a 256 caratteri
tags.form.no-description: Nessuna descrizione
tags.table.headers.tag: Tag
tags.table.headers.description: Descrizione
tags.table.headers.documents: Documenti
tags.table.headers.created: Creato
tags.table.headers.actions: Azioni
# Tagging rules
tagging-rules.field.name: nome documento
tagging-rules.field.content: contenuto documento
tagging-rules.operator.equals: uguale a
tagging-rules.operator.not-equals: diverso da
tagging-rules.operator.contains: contiene
tagging-rules.operator.not-contains: non contiene
tagging-rules.operator.starts-with: inizia con
tagging-rules.operator.ends-with: termina con
tagging-rules.list.title: Regole di tagging
tagging-rules.list.description: Gestisci le regole di tagging della tua organizzazione per taggare automaticamente i documenti in base a condizioni definite da te.
tagging-rules.list.demo-warning: 'Nota: Essendo un ambiente demo (senza server), le regole di tagging non verranno applicate ai nuovi documenti.'
tagging-rules.list.no-tagging-rules.title: Nessuna regola di tagging
tagging-rules.list.no-tagging-rules.description: Crea una regola per taggare automaticamente i documenti aggiunti in base a condizioni definite da te.
tagging-rules.list.no-tagging-rules.create-tagging-rule: Crea regola di tagging
tagging-rules.list.card.no-conditions: Nessuna condizione
tagging-rules.list.card.one-condition: 1 condizione
tagging-rules.list.card.conditions: '{{ count }} condizioni'
tagging-rules.list.card.delete: Elimina regola
tagging-rules.list.card.edit: Modifica regola
tagging-rules.create.title: Crea regola di tagging
tagging-rules.create.success: Regola di tagging creata con successo
tagging-rules.create.error: Errore nella creazione della regola di tagging
tagging-rules.create.submit: Crea regola
tagging-rules.form.name.label: Nome
tagging-rules.form.name.placeholder: 'Esempio: Tagga fatture'
tagging-rules.form.name.min-length: Inserisci un nome per la regola
tagging-rules.form.name.max-length: Il nome deve essere inferiore a 64 caratteri
tagging-rules.form.description.label: Descrizione
tagging-rules.form.description.placeholder: "Esempio: Tagga i documenti con 'fattura' nel nome"
tagging-rules.form.description.max-length: La descrizione deve essere inferiore a 256 caratteri
tagging-rules.form.conditions.label: Condizioni
tagging-rules.form.conditions.description: Definisci le condizioni che devono essere soddisfatte affinché la regola si applichi. Tutte le condizioni devono essere soddisfatte.
tagging-rules.form.conditions.add-condition: Aggiungi condizione
tagging-rules.form.conditions.no-conditions.title: Nessuna condizione
tagging-rules.form.conditions.no-conditions.description: Non hai aggiunto nessuna condizione a questa regola. Questa regola applicherà i suoi tag a tutti i documenti.
tagging-rules.form.conditions.no-conditions.confirm: Applica regola senza condizioni
tagging-rules.form.conditions.no-conditions.cancel: Annulla
tagging-rules.form.conditions.value.placeholder: 'Esempio: fattura'
tagging-rules.form.conditions.value.min-length: Inserisci un valore per la condizione
tagging-rules.form.tags.label: Tag
tagging-rules.form.tags.description: Seleziona i tag da applicare ai documenti che soddisfano le condizioni
tagging-rules.form.tags.min-length: È richiesto almeno un tag da applicare
tagging-rules.form.tags.add-tag: Crea tag
tagging-rules.form.submit: Crea regola
tagging-rules.update.title: Aggiorna regola di tagging
tagging-rules.update.error: Errore nell'aggiornamento della regola di tagging
tagging-rules.update.submit: Aggiorna regola
tagging-rules.update.cancel: Annulla
# Intake emails
intake-emails.title: Email di acquisizione
intake-emails.description: Gli indirizzi email di acquisizione vengono usati per importare automaticamente email in Papra. Basta inoltrare le email all'indirizzo di acquisizione e gli allegati saranno aggiunti ai documenti dell'organizzazione.
intake-emails.disabled.title: Email di acquisizione disabilitate
intake-emails.disabled.description: Le email di acquisizione sono disabilitate su questa istanza. Contatta il tuo amministratore per abilitarle. Consulta la {{ documentation }} per maggiori informazioni.
intake-emails.disabled.documentation: documentazione
intake-emails.info: Solo le email di acquisizione abilitate provenienti da origini consentite saranno processate. Puoi abilitare o disabilitare un'email di acquisizione in qualsiasi momento.
intake-emails.empty.title: Nessuna email di acquisizione
intake-emails.empty.description: Genera un indirizzo di acquisizione per importare facilmente allegati email.
intake-emails.empty.generate: Genera email di acquisizione
intake-emails.count: '{{ count }} email di acquisizione per questa organizzazione'
intake-emails.new: Nuova email di acquisizione
intake-emails.disabled-label: (Disabilitata)
intake-emails.no-origins: Nessuna origine email consentita
intake-emails.allowed-origins: Consentito da {{ count }} indirizzo/i
intake-emails.actions.enable: Abilita
intake-emails.actions.disable: Disabilita
intake-emails.actions.manage-origins: Gestisci indirizzi origine
intake-emails.actions.delete: Elimina
intake-emails.delete.confirm.title: Eliminare l'email di acquisizione?
intake-emails.delete.confirm.message: Sei sicuro di voler eliminare questa email di acquisizione? Questa azione non può essere annullata.
intake-emails.delete.confirm.confirm-button: Elimina email di acquisizione
intake-emails.delete.confirm.cancel-button: Annulla
intake-emails.delete.success: Email di acquisizione eliminata
intake-emails.create.success: Email di acquisizione creata
intake-emails.update.success.enabled: Email di acquisizione abilitata
intake-emails.update.success.disabled: Email di acquisizione disabilitata
intake-emails.allowed-origins.title: Origini consentite
intake-emails.allowed-origins.description: Solo le email inviate a {{ email }} da queste origini saranno processate. Se non sono specificate origini, tutte le email saranno scartate.
intake-emails.allowed-origins.add.label: Aggiungi email origine consentita
intake-emails.allowed-origins.add.placeholder: Es. ada@papra.app
intake-emails.allowed-origins.add.button: Aggiungi
intake-emails.allowed-origins.add.error.exists: Questa email è già tra le origini consentite per questa email di acquisizione
# API keys
api-keys.permissions.documents.title: Documenti
api-keys.permissions.documents.documents:create: Crea documenti
api-keys.permissions.documents.documents:read: Leggi documenti
api-keys.permissions.documents.documents:update: Aggiorna documenti
api-keys.permissions.documents.documents:delete: Elimina documenti
api-keys.permissions.tags.title: Tag
api-keys.permissions.tags.tags:create: Crea tag
api-keys.permissions.tags.tags:read: Leggi tag
api-keys.permissions.tags.tags:update: Aggiorna tag
api-keys.permissions.tags.tags:delete: Elimina tag
api-keys.create.title: Crea chiave API
api-keys.create.description: Crea una nuova chiave API per accedere all'API di Papra.
api-keys.create.success: La chiave API è stata creata con successo.
api-keys.create.back: Torna alle chiavi API
api-keys.create.form.name.label: Nome
api-keys.create.form.name.placeholder: 'Esempio: La mia chiave API'
api-keys.create.form.name.required: Inserisci un nome per la chiave API
api-keys.create.form.permissions.label: Permessi
api-keys.create.form.permissions.required: Seleziona almeno un permesso
api-keys.create.form.submit: Crea chiave API
api-keys.create.created.title: Chiave API creata
api-keys.create.created.description: La chiave API è stata creata con successo. Salvala in un luogo sicuro, non verrà più mostrata.
api-keys.list.title: Chiavi API
api-keys.list.description: Gestisci qui le tue chiavi API.
api-keys.list.create: Crea chiave API
api-keys.list.empty.title: Nessuna chiave API
api-keys.list.empty.description: Crea una chiave API per accedere all'API di Papra.
api-keys.list.card.last-used: Ultimo utilizzo
api-keys.list.card.never: Mai
api-keys.list.card.created: Creato
api-keys.delete.success: La chiave API è stata eliminata con successo
api-keys.delete.confirm.title: Eliminare la chiave API
api-keys.delete.confirm.message: Sei sicuro di voler eliminare questa chiave API? Questa azione non può essere annullata.
api-keys.delete.confirm.confirm-button: Elimina
api-keys.delete.confirm.cancel-button: Annulla
# Webhooks
webhooks.list.title: Webhook
webhooks.list.description: Gestisci i webhook della tua organizzazione
webhooks.list.empty.title: Nessun webhook
webhooks.list.empty.description: Crea il tuo primo webhook per iniziare a ricevere eventi
webhooks.list.create: Crea webhook
webhooks.list.card.last-triggered: Ultima attivazione
webhooks.list.card.never: Mai
webhooks.list.card.created: Creato
webhooks.create.title: Crea webhook
webhooks.create.description: Crea un nuovo webhook per ricevere eventi
webhooks.create.success: Webhook creato con successo
webhooks.create.back: Indietro
webhooks.create.form.submit: Crea webhook
webhooks.create.form.name.label: Nome webhook
webhooks.create.form.name.placeholder: Inserisci nome webhook
webhooks.create.form.name.required: Il nome è obbligatorio
webhooks.create.form.url.label: URL webhook
webhooks.create.form.url.placeholder: Inserisci URL webhook
webhooks.create.form.url.required: L'URL è obbligatorio
webhooks.create.form.url.invalid: L'URL non è valido
webhooks.create.form.secret.label: Segreto
webhooks.create.form.secret.placeholder: Inserisci il segreto del webhook
webhooks.create.form.events.label: Eventi
webhooks.create.form.events.required: È richiesto almeno un evento
webhooks.update.title: Modifica webhook
webhooks.update.description: Aggiorna i dettagli del webhook
webhooks.update.success: Webhook aggiornato con successo
webhooks.update.submit: Aggiorna webhook
webhooks.update.cancel: Annulla
webhooks.update.form.secret.placeholder: Inserisci nuovo segreto
webhooks.update.form.secret.placeholder-redacted: '[Segreto nascosto]'
webhooks.update.form.rotate-secret.button: Rigenera segreto
webhooks.delete.success: Webhook eliminato con successo
webhooks.delete.confirm.title: Eliminare webhook
webhooks.delete.confirm.message: Sei sicuro di voler eliminare questo webhook?
webhooks.delete.confirm.confirm-button: Elimina
webhooks.delete.confirm.cancel-button: Annulla
webhooks.events.documents.title: Eventi documenti
webhooks.events.documents.document:created.description: Documento creato
webhooks.events.documents.document:deleted.description: Documento eliminato
webhooks.events.documents.document:updated.description: Documento aggiornato
webhooks.events.documents.document:tag:added.description: Un tag è stato aggiunto a un documento
webhooks.events.documents.document:tag:removed.description: Un tag è stato rimosso da un documento
# Navigation
layout.menu.home: Home
layout.menu.documents: Documenti
layout.menu.tags: Tag
layout.menu.tagging-rules: Regole di tagging
layout.menu.deleted-documents: Documenti eliminati
layout.menu.organization-settings: Impostazioni
layout.menu.api-keys: Chiavi API
layout.menu.settings: Impostazioni
layout.menu.account: Account
layout.menu.general-settings: Impostazioni generali
layout.menu.intake-emails: Email di acquisizione
layout.menu.webhooks: Webhook
layout.menu.members: Membri
layout.menu.invitations: Inviti
layout.theme.light: Modalità chiara
layout.theme.dark: Modalità scura
layout.theme.system: Modalità sistema
layout.search.placeholder: Cerca...
layout.menu.import-document: Importa un documento
user-menu.account-settings: Impostazioni account
user-menu.api-keys: Chiavi API
user-menu.invitations: Inviti
user-menu.language: Lingua
user-menu.logout: Esci
# Command palette
command-palette.search.placeholder: Cerca comandi o documenti
command-palette.no-results: Nessun risultato trovato
command-palette.sections.documents: Documenti
command-palette.sections.theme: Tema
# API errors
api-errors.document.already_exists: Il documento esiste già
api-errors.document.file_too_big: Il file del documento è troppo grande
api-errors.intake_email.limit_reached: È stato raggiunto il numero massimo di email di acquisizione per questa organizzazione. Aggiorna il tuo piano per crearne altre.
api-errors.user.max_organization_count_reached: Hai raggiunto il numero massimo di organizzazioni che puoi creare, se hai bisogno di crearne altre contatta il supporto.
api-errors.default: Si è verificato un errore durante l'elaborazione della richiesta.
api-errors.organization.invitation_already_exists: Esiste già un invito per questa email in questa organizzazione.
api-errors.user.already_in_organization: Questo utente è già in questa organizzazione.
api-errors.user.organization_invitation_limit_reached: È stato raggiunto il numero massimo di inviti per oggi. Riprova domani.
api-errors.demo.not_available: Questa funzionalità non è disponibile nella demo
api-errors.tags.already_exists: Esiste già un tag con questo nome per questa organizzazione
api-errors.internal.error: Si è verificato un errore durante l'elaborazione della richiesta. Riprova.
api-errors.auth.invalid_origin: Origine dell'applicazione non valida. Se stai ospitando Papra, assicurati che la variabile di ambiente APP_BASE_URL corrisponda all'URL corrente. Per maggiori dettagli, consulta https://docs.papra.app/resources/troubleshooting/#invalid-application-origin
# Not found
not-found.title: 404 - Non trovato
not-found.description: Spiacenti, la pagina che stai cercando non sembra esistere. Controlla l'URL e riprova.
not-found.back-to-home: Torna alla home
# Demo
demo.popup.description: Questo è un ambiente demo, tutti i dati vengono salvati nello storage locale del browser.
demo.popup.discord: Unisciti a {{ discordLink }} per ricevere supporto, proporre funzionalità o semplicemente fare due chiacchiere.
demo.popup.discord-link-label: Server Discord
demo.popup.reset: Reimposta dati demo
demo.popup.hide: Nascondi
# Color picker
color-picker.hue: Tonalità
color-picker.saturation: Saturazione
color-picker.lightness: Luminosità
color-picker.select-color: Seleziona colore
color-picker.select-a-color: Seleziona un colore

View File

@@ -489,8 +489,12 @@ webhooks.delete.confirm.message: Czy na pewno chcesz usunąć ten webhook?
webhooks.delete.confirm.confirm-button: Usuń
webhooks.delete.confirm.cancel-button: Anuluj
webhooks.events.documents.title: Zdarzenia dokumentów
webhooks.events.documents.document:created.description: Utworzono dokument
webhooks.events.documents.document:deleted.description: Usunięto dokument
webhooks.events.documents.document:updated.description: Dokument został zaktualizowany
webhooks.events.documents.document:tag:added.description: Tag został dodany do dokumentu
webhooks.events.documents.document:tag:removed.description: Tag został usunięty z dokumentu
# Navigation
@@ -541,6 +545,8 @@ api-errors.user.already_in_organization: Ten użytkownik należy już do tej org
api-errors.user.organization_invitation_limit_reached: Osiągnięto maksymalną liczbę zaproszeń na dzisiaj. Spróbuj ponownie jutro.
api-errors.demo.not_available: Ta funkcja nie jest dostępna w wersji demo
api-errors.tags.already_exists: Tag o tej nazwie już istnieje w tej organizacji
api-errors.internal.error: Wystąpił błąd podczas przetwarzania żądania. Spróbuj ponownie później.
api-errors.auth.invalid_origin: Nieprawidłowa lokalizacja aplikacji. Jeśli hostujesz Papra, upewnij się, że zmienna środowiskowa APP_BASE_URL odpowiada bieżącemu adresowi URL. Aby uzyskać więcej informacji, zobacz https://docs.papra.app/resources/troubleshooting/#invalid-application-origin
# Not found

View File

@@ -489,8 +489,12 @@ webhooks.delete.confirm.message: Tem certeza de que deseja excluir este webhook?
webhooks.delete.confirm.confirm-button: Excluir
webhooks.delete.confirm.cancel-button: Cancelar
webhooks.events.documents.title: Eventos de documentos
webhooks.events.documents.document:created.description: Documento criado
webhooks.events.documents.document:deleted.description: Documento excluído
webhooks.events.documents.document:updated.description: Documento atualizado
webhooks.events.documents.document:tag:added.description: Uma tag foi adicionada a um documento
webhooks.events.documents.document:tag:removed.description: Uma tag foi removida de um documento
# Navigation
@@ -541,6 +545,8 @@ api-errors.user.already_in_organization: Este usuário já faz parte desta organ
api-errors.user.organization_invitation_limit_reached: O número máximo de convites por hoje foi atingido. Por favor, tente novamente amanhã.
api-errors.demo.not_available: Este recurso não está disponível em ambiente de demonstração
api-errors.tags.already_exists: Já existe uma tag com este nome nesta organização
api-errors.internal.error: Ocorreu um erro ao processar sua solicitação. Por favor, tente novamente.
api-errors.auth.invalid_origin: Origem da aplicação inválida. Se você está hospedando o Papra, certifique-se de que a variável de ambiente APP_BASE_URL corresponde à sua URL atual. Para mais detalhes, consulte https://docs.papra.app/resources/troubleshooting/#invalid-application-origin
# Not found

View File

@@ -71,6 +71,9 @@ auth.legal-links.description: Ao continuar, reconhece que compreende e concorda
auth.legal-links.terms: Termos de Serviço
auth.legal-links.privacy: Política de Privacidade
# auth.no-auth-provider.title: No authentication provider
# auth.no-auth-provider.description: There are no authentication providers enabled on this instance of Papra. Please contact the administrator of this instance to enable them.
# User settings
user.settings.title: Definições do utilizador
@@ -486,8 +489,12 @@ webhooks.delete.confirm.message: Tem a certeza de que deseja eliminar este webho
webhooks.delete.confirm.confirm-button: Eliminar
webhooks.delete.confirm.cancel-button: Cancelar
webhooks.events.documents.title: Eventos de documentos
webhooks.events.documents.document:created.description: Documento criado
webhooks.events.documents.document:deleted.description: Documento eliminado
webhooks.events.documents.document:updated.description: Documento atualizado
webhooks.events.documents.document:tag:added.description: Uma etiqueta foi adicionada a um documento
webhooks.events.documents.document:tag:removed.description: Uma etiqueta foi removida de um documento
# Navigation
@@ -538,6 +545,8 @@ api-errors.user.already_in_organization: Este utilizadpr já faz parte desta org
api-errors.user.organization_invitation_limit_reached: O número máximo de convites por hoje foi atingido. Por favor, tente novamente amanhã.
api-errors.demo.not_available: Este recurso não está disponível em ambiente de demonstração
api-errors.tags.already_exists: Já existe uma etiqueta com este nome nesta organização
api-errors.internal.error: Ocorreu um erro ao processar a solicitação. Por favor, tente novamente.
api-errors.auth.invalid_origin: Origem da aplicação inválida. Se você está hospedando o Papra, certifique-se de que a variável de ambiente APP_BASE_URL corresponde à sua URL atual. Para mais detalhes, consulte https://docs.papra.app/resources/troubleshooting/#invalid-application-origin
# Not found

File diff suppressed because it is too large Load Diff

View File

@@ -49,12 +49,21 @@ export async function authWithProvider({ provider, config }: { provider: SsoProv
const isCustomProvider = config.auth.providers.customs.some(({ providerId }) => providerId === provider.key);
if (isCustomProvider) {
signIn.oauth2({
const { error } = await signIn.oauth2({
providerId: provider.key,
callbackURL: config.baseUrl,
});
if (error) {
throw error;
}
return;
}
await signIn.social({ provider: provider.key as 'github' | 'google', callbackURL: config.baseUrl });
const { error } = await signIn.social({ provider: provider.key as 'github' | 'google', callbackURL: config.baseUrl });
if (error) {
throw error;
}
}

View File

@@ -1,34 +1,47 @@
import type { Component } from 'solid-js';
import { createSignal, Match, Switch } from 'solid-js';
import { useI18nApiErrors } from '@/modules/shared/http/composables/i18n-api-errors';
import { cn } from '@/modules/shared/style/cn';
import { Button } from '@/modules/ui/components/button';
export const SsoProviderButton: Component<{ name: string; icon?: string; onClick: () => Promise<void>; label: string }> = (props) => {
const [getIsLoading, setIsLoading] = createSignal(false);
const [getError, setError] = createSignal<string | undefined>(undefined);
const { getErrorMessage } = useI18nApiErrors();
const onClick = async () => {
setIsLoading(true);
await props.onClick();
try {
await props.onClick();
} catch (error) {
setError(getErrorMessage({ error }));
// reset loading only in catch as the auth via sso can take a while before the redirection happens
setIsLoading(false);
}
};
return (
<Button variant="secondary" class="block w-full flex items-center justify-center gap-2" onClick={onClick} disabled={getIsLoading()}>
<>
<Button variant="secondary" class="block w-full flex items-center justify-center gap-2" onClick={onClick} disabled={getIsLoading()}>
<Switch>
<Match when={getIsLoading()}>
<span class="i-tabler-loader-2 animate-spin" />
</Match>
<Switch>
<Match when={getIsLoading()}>
<span class="i-tabler-loader-2 animate-spin" />
</Match>
<Match when={props.icon?.startsWith('i-')}>
<span class={cn(`size-4.5`, props.icon)} />
</Match>
<Match when={props.icon?.startsWith('i-')}>
<span class={cn(`size-4.5`, props.icon)} />
</Match>
<Match when={props.icon}>
<img src={props.icon} alt={props.name} class="size-4.5" />
</Match>
</Switch>
<Match when={props.icon}>
<img src={props.icon} alt={props.name} class="size-4.5" />
</Match>
</Switch>
{props.label}
</Button>
{props.label}
</Button>
{getError() && <p class="text-red-500">{getError()}</p>}
</>
);
};

View File

@@ -6,6 +6,7 @@ import * as v from 'valibot';
import { useConfig } from '@/modules/config/config.provider';
import { useI18n } from '@/modules/i18n/i18n.provider';
import { createForm } from '@/modules/shared/form/form';
import { useI18nApiErrors } from '@/modules/shared/http/composables/i18n-api-errors';
import { Button } from '@/modules/ui/components/button';
import { Checkbox, CheckboxControl, CheckboxLabel } from '@/modules/ui/components/checkbox';
import { Separator } from '@/modules/ui/components/separator';
@@ -21,6 +22,7 @@ export const EmailLoginForm: Component = () => {
const navigate = useNavigate();
const { config } = useConfig();
const { t } = useI18n();
const { createI18nApiError } = useI18nApiErrors({ t });
const { form, Form, Field } = createForm({
onSubmit: async ({ email, password, rememberMe }) => {
@@ -31,7 +33,7 @@ export const EmailLoginForm: Component = () => {
}
if (error) {
throw error;
throw createI18nApiError({ error });
}
},
schema: v.object({

View File

@@ -6,6 +6,7 @@ import * as v from 'valibot';
import { useConfig } from '@/modules/config/config.provider';
import { useI18n } from '@/modules/i18n/i18n.provider';
import { createForm } from '@/modules/shared/form/form';
import { useI18nApiErrors } from '@/modules/shared/http/composables/i18n-api-errors';
import { Button } from '@/modules/ui/components/button';
import { Separator } from '@/modules/ui/components/separator';
import { TextField, TextFieldLabel, TextFieldRoot } from '@/modules/ui/components/textfield';
@@ -20,6 +21,8 @@ export const EmailRegisterForm: Component = () => {
const { config } = useConfig();
const navigate = useNavigate();
const { t } = useI18n();
const { createI18nApiError } = useI18nApiErrors({ t });
const { form, Form, Field } = createForm({
onSubmit: async ({ email, password, name }) => {
const { error } = await signUp.email({
@@ -30,7 +33,7 @@ export const EmailRegisterForm: Component = () => {
});
if (error) {
throw error;
throw createI18nApiError({ error });
}
if (config.auth.isEmailVerificationRequired) {

View File

@@ -7,4 +7,5 @@ export const locales = [
{ key: 'pl', name: 'Polski' },
{ key: 'ro', name: 'Română' },
{ key: 'es', name: 'Español' },
{ key: 'it', name: 'Italiano' },
] as const;

View File

@@ -70,13 +70,14 @@ describe('i18n models', () => {
expect(t('hello')).to.eql('Hello!');
});
test('the translator returns the key if the key is not in the dictionary', () => {
test('the translator returns undefined if the key is not in the dictionary', () => {
const dictionary = {
hello: 'Hello!',
};
const t = createTranslator({ getDictionary: () => dictionary });
expect(t('world' as any)).to.eql('world');
expect(t('world' as any)).to.eql(undefined);
expect(t('world' as any, { name: 'John' })).to.eql(undefined);
});
test('the translator replaces the placeholders in the translation', () => {

View File

@@ -36,15 +36,15 @@ export function createTranslator<Dict extends Record<string, string>>({ getDicti
console.warn(`Translation not found for key: ${String(key)}`);
}
let translation: string = translationFromDictionary ?? key;
if (args) {
for (const [key, value] of Object.entries(args)) {
translation = translation.replace(new RegExp(`{{\\s*${key}\\s*}}`, 'g'), String(value));
}
if (args && translationFromDictionary) {
return Object.entries(args)
.reduce(
(acc, [key, value]) => acc.replace(new RegExp(`{{\\s*${key}\\s*}}`, 'g'), String(value)),
String(translationFromDictionary),
);
}
return translation;
return translationFromDictionary;
};
}

View File

@@ -38,7 +38,8 @@ describe('locales', () => {
const dynamicKeysMatchers = [
/^api-errors\./, // api-errors.document.already_exists
/^auth\.register\.providers\.[a-z0-9:]+$/, // auth.register.providers.google
/^webhooks\.events\.documents\.[a-z0-9:]+.description$/, // webhooks.events.organization.organization:created
/^webhooks\.events\.[a-z0-9]+\.[a-z0-9:]+.description$/, // webhooks.events.documents.document:created.description
/^webhooks\.events\.[a-z0-9]+\.title$/, // webhooks.events.documents.title
/^api-keys\.permissions\.[a-z0-9:]+\.[a-z0-9:]+$/, // api-keys.permissions.documents.documents:delete
/^organizations\.members\.roles\.[a-z0-9]+$/, // organizations.members.roles.admin
/^activity\.document\.[a-z0-9:]+$/, // activity.document.created

View File

@@ -440,8 +440,12 @@ export type LocaleKeys =
| 'webhooks.delete.confirm.message'
| 'webhooks.delete.confirm.confirm-button'
| 'webhooks.delete.confirm.cancel-button'
| 'webhooks.events.documents.title'
| 'webhooks.events.documents.document:created.description'
| 'webhooks.events.documents.document:deleted.description'
| 'webhooks.events.documents.document:updated.description'
| 'webhooks.events.documents.document:tag:added.description'
| 'webhooks.events.documents.document:tag:removed.description'
| 'layout.menu.home'
| 'layout.menu.documents'
| 'layout.menu.tags'
@@ -480,6 +484,8 @@ export type LocaleKeys =
| 'api-errors.user.organization_invitation_limit_reached'
| 'api-errors.demo.not_available'
| 'api-errors.tags.already_exists'
| 'api-errors.internal.error'
| 'api-errors.auth.invalid_origin'
| 'not-found.title'
| 'not-found.description'
| 'not-found.back-to-home'

View File

@@ -17,16 +17,26 @@ import { Alert, AlertDescription } from '@/modules/ui/components/alert';
import { Button } from '@/modules/ui/components/button';
import { Card } from '@/modules/ui/components/card';
import { Dialog, DialogContent, DialogDescription, DialogHeader, DialogTitle, DialogTrigger } from '@/modules/ui/components/dialog';
import { DropdownMenu, DropdownMenuContent, DropdownMenuItem, DropdownMenuTrigger } from '@/modules/ui/components/dropdown-menu';
import { EmptyState } from '@/modules/ui/components/empty';
import { createToast } from '@/modules/ui/components/sonner';
import { TextField, TextFieldLabel, TextFieldRoot } from '@/modules/ui/components/textfield';
import { createIntakeEmail, deleteIntakeEmail, fetchIntakeEmails, updateIntakeEmail } from '../intake-emails.services';
const AllowedOriginsDialog: Component<{ children: (props: DialogTriggerProps) => JSX.Element; intakeEmails: IntakeEmail }> = (props) => {
const [getAllowedOrigins, setAllowedOrigins] = createSignal([...props.intakeEmails.allowedOrigins]);
const AllowedOriginsDialog: Component<{
children: (props: DialogTriggerProps) => JSX.Element;
intakeEmails: IntakeEmail;
open?: boolean;
onOpenChange?: (isOpen: boolean) => void;
}> = (props) => {
const [getAllowedOrigins, setAllowedOrigins] = createSignal(props.intakeEmails?.allowedOrigins || []);
const { t } = useI18n();
const update = async () => {
if (!props.intakeEmails) {
return;
}
await updateIntakeEmail({
organizationId: props.intakeEmails.organizationId,
intakeEmailId: props.intakeEmails.id,
@@ -58,13 +68,29 @@ const AllowedOriginsDialog: Component<{ children: (props: DialogTriggerProps) =>
});
async function invalidateQuery() {
if (!props.intakeEmails) {
return;
}
await queryClient.invalidateQueries({
queryKey: ['organizations', props.intakeEmails.organizationId, 'intake-emails'],
});
}
if (!props.intakeEmails) {
return null;
}
return (
<Dialog onOpenChange={isOpen => !isOpen && invalidateQuery()}>
<Dialog
open={props.open}
onOpenChange={(isOpen) => {
if (!isOpen) {
invalidateQuery();
}
props.onOpenChange?.(isOpen);
}}
>
<DialogTrigger as={props.children} />
<DialogContent>
@@ -129,6 +155,8 @@ const AllowedOriginsDialog: Component<{ children: (props: DialogTriggerProps) =>
export const IntakeEmailsPage: Component = () => {
const { config } = useConfig();
const { t, te } = useI18n();
const [selectedIntakeEmail, setSelectedIntakeEmail] = createSignal<IntakeEmail | null>(null);
const [openDropdownId, setOpenDropdownId] = createSignal<string | null>(null);
if (!config.intakeEmails.isEnabled) {
return (
@@ -225,6 +253,11 @@ export const IntakeEmailsPage: Component = () => {
});
};
const openAllowedOriginsDialog = (intakeEmail: IntakeEmail) => {
setOpenDropdownId(null);
setSelectedIntakeEmail(intakeEmail);
};
return (
<div class="p-6 max-w-screen-md mx-auto mt-10">
<h1 class="text-xl font-semibold">{t('intake-emails.title')}</h1>
@@ -313,39 +346,46 @@ export const IntakeEmailsPage: Component = () => {
</Show>
</div>
</div>
<div class="flex items-center gap-2">
<Button
variant="outline"
onClick={() => updateEmail({ intakeEmailId: intakeEmail.id, isEnabled: !intakeEmail.isEnabled })}
<DropdownMenu
open={openDropdownId() === intakeEmail.id}
onOpenChange={(isOpen) => {
setOpenDropdownId(isOpen ? intakeEmail.id : null);
}}
>
<div class="i-tabler-power size-4 mr-2" />
{intakeEmail.isEnabled ? t('intake-emails.actions.disable') : t('intake-emails.actions.enable')}
</Button>
<AllowedOriginsDialog intakeEmails={intakeEmail}>
{(props: DialogTriggerProps) => (
<Button
variant="outline"
aria-label="Edit intake email"
{...props}
class="flex items-center gap-2 leading-none"
<DropdownMenuTrigger as={Button} variant="outline" aria-label="More actions" size="icon">
<div class="i-tabler-dots-vertical size-4" />
</DropdownMenuTrigger>
<DropdownMenuContent>
<DropdownMenuItem
onClick={() => {
setOpenDropdownId(null);
updateEmail({ intakeEmailId: intakeEmail.id, isEnabled: !intakeEmail.isEnabled });
}}
>
<div class="i-tabler-edit size-4" />
{t('intake-emails.actions.manage-origins')}
</Button>
)}
</AllowedOriginsDialog>
<div class="i-tabler-power size-4 mr-2" />
{intakeEmail.isEnabled ? t('intake-emails.actions.disable') : t('intake-emails.actions.enable')}
</DropdownMenuItem>
<Button
variant="outline"
onClick={() => deleteEmail({ intakeEmailId: intakeEmail.id })}
aria-label="Delete intake email"
class="text-red"
>
<div class="i-tabler-trash size-4 mr-2" />
{t('intake-emails.actions.delete')}
</Button>
<DropdownMenuItem
onClick={() => openAllowedOriginsDialog(intakeEmail)}
>
<div class="i-tabler-edit size-4 mr-2" />
{t('intake-emails.actions.manage-origins')}
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => {
setOpenDropdownId(null);
deleteEmail({ intakeEmailId: intakeEmail.id });
}}
class="text-red"
>
<div class="i-tabler-trash size-4 mr-2" />
{t('intake-emails.actions.delete')}
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
</div>
)}
@@ -355,6 +395,22 @@ export const IntakeEmailsPage: Component = () => {
)}
</Show>
</Suspense>
<Show when={selectedIntakeEmail()}>
{intakeEmail => (
<AllowedOriginsDialog
intakeEmails={intakeEmail()}
open={true}
onOpenChange={(isOpen) => {
if (!isOpen) {
setSelectedIntakeEmail(null);
}
}}
>
{() => <div />}
</AllowedOriginsDialog>
)}
</Show>
</div>
);
};

View File

@@ -2,28 +2,46 @@ import type { LocaleKeys } from '@/modules/i18n/locales.types';
import { get } from 'lodash-es';
import { useI18n } from '@/modules/i18n/i18n.provider';
function codeToKey(code: string): LocaleKeys {
// Better auth may returns different error codes like INVALID_ORIGIN, INVALID_CALLBACKURL when the origin is invalid
// codes are here https://github.com/better-auth/better-auth/blob/canary/packages/better-auth/src/api/middlewares/origin-check.ts#L71 (in lower case)
if (code.match(/^INVALID_[A-Z]+URL$/) || code === 'INVALID_ORIGIN') {
return `api-errors.auth.invalid_origin`;
}
return `api-errors.${code}` as LocaleKeys;
}
export function useI18nApiErrors({ t = useI18n().t }: { t?: ReturnType<typeof useI18n>['t'] } = {}) {
const getTranslationFromApiErrorCode = ({ code }: { code: string }) => {
return t(`api-errors.${code}` as LocaleKeys);
};
const getDefaultErrorMessage = () => t('api-errors.default');
const getTranslationFromApiError = ({ error }: { error: unknown }) => {
const code = get(error, 'data.error.code') ?? get(error, 'code');
if (!code) {
return t('api-errors.default');
const getErrorMessage = (args: { error: unknown } | { code: string }) => {
if ('code' in args) {
const { code } = args;
return t(codeToKey(code)) ?? getDefaultErrorMessage();
}
return getTranslationFromApiErrorCode({ code });
if ('error' in args) {
const { error } = args;
const code = get(error, 'data.error.code') ?? get(error, 'code');
const translation = code ? t(codeToKey(code)) : undefined;
if (translation) {
return translation;
}
if (typeof error === 'object' && error && 'message' in error && typeof error.message === 'string') {
return error.message;
}
}
return getDefaultErrorMessage();
};
return {
getErrorMessage: (args: { error: unknown } | { code: string }) => {
if ('error' in args) {
return getTranslationFromApiError({ error: args.error });
}
return getTranslationFromApiErrorCode({ code: args.code });
getErrorMessage,
createI18nApiError: (args: { error: unknown } | { code: string }) => {
return new Error(getErrorMessage(args));
},
};
}

View File

@@ -228,6 +228,7 @@ export const TagsPage: Component = () => {
const params = useParams();
const { confirm } = useConfirmModal();
const { t } = useI18n();
const { getErrorMessage } = useI18nApiErrors({ t });
const query = useQuery(() => ({
queryKey: ['organizations', params.organizationId, 'tags'],
@@ -252,10 +253,19 @@ export const TagsPage: Component = () => {
return;
}
await deleteTag({
const [, error] = await safely(deleteTag({
organizationId: params.organizationId,
tagId: tag.id,
});
}));
if (error) {
createToast({
message: getErrorMessage({ error }),
type: 'error',
});
return;
}
await queryClient.invalidateQueries({
queryKey: ['organizations', params.organizationId],

View File

@@ -46,7 +46,8 @@ export const WebhookEventsPicker: Component<{ events: WebhookEvent[]; onChange:
};
return (
<div class="grid grid-cols-1 sm:grid-cols-2 gap-4">
<div>
{/* <div class="grid grid-cols-1 sm:grid-cols-2 gap-4"> */}
<For each={getEventsSections()}>
{section => (
<div>

View File

@@ -4,6 +4,9 @@ export const WEBHOOK_EVENTS = [
events: [
'document:created',
'document:deleted',
'document:updated',
'document:tag:added',
'document:tag:removed',
],
},

View File

@@ -1,5 +1,27 @@
# @papra/app-server
## 0.8.0
### Minor Changes
- [#452](https://github.com/papra-hq/papra/pull/452) [`7f7e5bf`](https://github.com/papra-hq/papra/commit/7f7e5bffcbcfb843f3b2458400dfb44409a44867) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Completely rewrote the migration mechanism
- [#447](https://github.com/papra-hq/papra/pull/447) [`b5ccc13`](https://github.com/papra-hq/papra/commit/b5ccc135ba7f4359eaf85221bcb40ee63ba7d6c7) Thanks [@CorentinTh](https://github.com/CorentinTh)! - The file content extraction (like OCR) is now done asynchronously by the task runner
- [#448](https://github.com/papra-hq/papra/pull/448) [`5868800`](https://github.com/papra-hq/papra/commit/5868800bcec6ed69b5441b50e4445fae5cdb5bfb) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Fixed the impossibility to delete a tag that has been assigned to a document
- [#432](https://github.com/papra-hq/papra/pull/432) [`6723baf`](https://github.com/papra-hq/papra/commit/6723baf98ad46f989fe1e1e19ad0dd25622cca77) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Added new webhook events: document:updated, document:tag:added, document:tag:removed
- [#432](https://github.com/papra-hq/papra/pull/432) [`6723baf`](https://github.com/papra-hq/papra/commit/6723baf98ad46f989fe1e1e19ad0dd25622cca77) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Webhooks invocation is now defered
### Patch Changes
- [#455](https://github.com/papra-hq/papra/pull/455) [`b33fde3`](https://github.com/papra-hq/papra/commit/b33fde35d3e8622e31b51aadfe56875d8e48a2ef) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Improved feedback message in case of invalid origin configuration
- Updated dependencies [[`a8cff8c`](https://github.com/papra-hq/papra/commit/a8cff8cedc062be3ed1d454e9de6e456553a4d8c), [`6723baf`](https://github.com/papra-hq/papra/commit/6723baf98ad46f989fe1e1e19ad0dd25622cca77), [`6723baf`](https://github.com/papra-hq/papra/commit/6723baf98ad46f989fe1e1e19ad0dd25622cca77), [`67b3b14`](https://github.com/papra-hq/papra/commit/67b3b14cdfa994874c695b9d854a93160ba6a911)]:
- @papra/webhooks@0.2.0
- @papra/lecture@0.1.0
## 0.7.0
### Minor Changes

View File

@@ -4,7 +4,7 @@ import { defineConfig } from 'drizzle-kit';
export default defineConfig({
schema: ['./src/modules/**/*.table.ts', './src/modules/**/*.tables.ts'],
dialect: 'turso',
out: './migrations',
out: './src/migrations',
dbCredentials: {
url: env.DATABASE_URL ?? 'file:./db.sqlite',
authToken: env.DATABASE_AUTH_TOKEN,

View File

@@ -1,172 +0,0 @@
CREATE TABLE `documents` (
`id` text PRIMARY KEY NOT NULL,
`created_at` integer NOT NULL,
`updated_at` integer NOT NULL,
`is_deleted` integer DEFAULT false NOT NULL,
`deleted_at` integer,
`organization_id` text NOT NULL,
`created_by` text,
`deleted_by` text,
`original_name` text NOT NULL,
`original_size` integer DEFAULT 0 NOT NULL,
`original_storage_key` text NOT NULL,
`original_sha256_hash` text NOT NULL,
`name` text NOT NULL,
`mime_type` text NOT NULL,
`content` text DEFAULT '' NOT NULL,
FOREIGN KEY (`organization_id`) REFERENCES `organizations`(`id`) ON UPDATE cascade ON DELETE cascade,
FOREIGN KEY (`created_by`) REFERENCES `users`(`id`) ON UPDATE cascade ON DELETE set null,
FOREIGN KEY (`deleted_by`) REFERENCES `users`(`id`) ON UPDATE cascade ON DELETE set null
);
--> statement-breakpoint
CREATE INDEX `documents_organization_id_is_deleted_created_at_index` ON `documents` (`organization_id`,`is_deleted`,`created_at`);--> statement-breakpoint
CREATE INDEX `documents_organization_id_is_deleted_index` ON `documents` (`organization_id`,`is_deleted`);--> statement-breakpoint
CREATE UNIQUE INDEX `documents_organization_id_original_sha256_hash_unique` ON `documents` (`organization_id`,`original_sha256_hash`);--> statement-breakpoint
CREATE INDEX `documents_original_sha256_hash_index` ON `documents` (`original_sha256_hash`);--> statement-breakpoint
CREATE INDEX `documents_organization_id_size_index` ON `documents` (`organization_id`,`original_size`);--> statement-breakpoint
CREATE TABLE `organization_invitations` (
`id` text PRIMARY KEY NOT NULL,
`created_at` integer NOT NULL,
`updated_at` integer NOT NULL,
`organization_id` text NOT NULL,
`email` text NOT NULL,
`role` text,
`status` text NOT NULL,
`expires_at` integer NOT NULL,
`inviter_id` text NOT NULL,
FOREIGN KEY (`organization_id`) REFERENCES `organizations`(`id`) ON UPDATE cascade ON DELETE cascade,
FOREIGN KEY (`inviter_id`) REFERENCES `users`(`id`) ON UPDATE cascade ON DELETE cascade
);
--> statement-breakpoint
CREATE TABLE `organization_members` (
`id` text PRIMARY KEY NOT NULL,
`created_at` integer NOT NULL,
`updated_at` integer NOT NULL,
`organization_id` text NOT NULL,
`user_id` text NOT NULL,
`role` text NOT NULL,
FOREIGN KEY (`organization_id`) REFERENCES `organizations`(`id`) ON UPDATE cascade ON DELETE cascade,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE cascade ON DELETE cascade
);
--> statement-breakpoint
CREATE UNIQUE INDEX `organization_members_user_organization_unique` ON `organization_members` (`organization_id`,`user_id`);--> statement-breakpoint
CREATE TABLE `organizations` (
`id` text PRIMARY KEY NOT NULL,
`created_at` integer NOT NULL,
`updated_at` integer NOT NULL,
`name` text NOT NULL,
`customer_id` text
);
--> statement-breakpoint
CREATE TABLE `user_roles` (
`id` text PRIMARY KEY NOT NULL,
`created_at` integer NOT NULL,
`updated_at` integer NOT NULL,
`user_id` text NOT NULL,
`role` text NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE cascade ON DELETE cascade
);
--> statement-breakpoint
CREATE INDEX `user_roles_role_index` ON `user_roles` (`role`);--> statement-breakpoint
CREATE UNIQUE INDEX `user_roles_user_id_role_unique_index` ON `user_roles` (`user_id`,`role`);--> statement-breakpoint
CREATE TABLE `documents_tags` (
`document_id` text NOT NULL,
`tag_id` text NOT NULL,
PRIMARY KEY(`document_id`, `tag_id`),
FOREIGN KEY (`document_id`) REFERENCES `documents`(`id`) ON UPDATE cascade ON DELETE cascade,
FOREIGN KEY (`tag_id`) REFERENCES `tags`(`id`) ON UPDATE cascade ON DELETE cascade
);
--> statement-breakpoint
CREATE TABLE `tags` (
`id` text PRIMARY KEY NOT NULL,
`created_at` integer NOT NULL,
`updated_at` integer NOT NULL,
`organization_id` text NOT NULL,
`name` text NOT NULL,
`color` text NOT NULL,
`description` text,
FOREIGN KEY (`organization_id`) REFERENCES `organizations`(`id`) ON UPDATE cascade ON DELETE cascade
);
--> statement-breakpoint
CREATE UNIQUE INDEX `tags_organization_id_name_unique` ON `tags` (`organization_id`,`name`);--> statement-breakpoint
CREATE TABLE `users` (
`id` text PRIMARY KEY NOT NULL,
`created_at` integer NOT NULL,
`updated_at` integer NOT NULL,
`email` text NOT NULL,
`email_verified` integer DEFAULT false NOT NULL,
`name` text,
`image` text,
`max_organization_count` integer
);
--> statement-breakpoint
CREATE UNIQUE INDEX `users_email_unique` ON `users` (`email`);--> statement-breakpoint
CREATE INDEX `users_email_index` ON `users` (`email`);--> statement-breakpoint
CREATE TABLE `auth_accounts` (
`id` text PRIMARY KEY NOT NULL,
`created_at` integer NOT NULL,
`updated_at` integer NOT NULL,
`user_id` text,
`account_id` text NOT NULL,
`provider_id` text NOT NULL,
`access_token` text,
`refresh_token` text,
`access_token_expires_at` integer,
`refresh_token_expires_at` integer,
`scope` text,
`id_token` text,
`password` text,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE cascade ON DELETE cascade
);
--> statement-breakpoint
CREATE TABLE `auth_sessions` (
`id` text PRIMARY KEY NOT NULL,
`created_at` integer NOT NULL,
`updated_at` integer NOT NULL,
`token` text NOT NULL,
`user_id` text,
`expires_at` integer NOT NULL,
`ip_address` text,
`user_agent` text,
`active_organization_id` text,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE cascade ON DELETE cascade,
FOREIGN KEY (`active_organization_id`) REFERENCES `organizations`(`id`) ON UPDATE cascade ON DELETE set null
);
--> statement-breakpoint
CREATE INDEX `auth_sessions_token_index` ON `auth_sessions` (`token`);--> statement-breakpoint
CREATE TABLE `auth_verifications` (
`id` text PRIMARY KEY NOT NULL,
`created_at` integer NOT NULL,
`updated_at` integer NOT NULL,
`identifier` text NOT NULL,
`value` text NOT NULL,
`expires_at` integer NOT NULL
);
--> statement-breakpoint
CREATE INDEX `auth_verifications_identifier_index` ON `auth_verifications` (`identifier`);--> statement-breakpoint
CREATE TABLE `intake_emails` (
`id` text PRIMARY KEY NOT NULL,
`created_at` integer NOT NULL,
`updated_at` integer NOT NULL,
`email_address` text NOT NULL,
`organization_id` text NOT NULL,
`allowed_origins` text DEFAULT '[]' NOT NULL,
`is_enabled` integer DEFAULT true NOT NULL,
FOREIGN KEY (`organization_id`) REFERENCES `organizations`(`id`) ON UPDATE cascade ON DELETE cascade
);
--> statement-breakpoint
CREATE UNIQUE INDEX `intake_emails_email_address_unique` ON `intake_emails` (`email_address`);--> statement-breakpoint
CREATE TABLE `organization_subscriptions` (
`id` text PRIMARY KEY NOT NULL,
`customer_id` text NOT NULL,
`organization_id` text NOT NULL,
`plan_id` text NOT NULL,
`status` text NOT NULL,
`seats_count` integer NOT NULL,
`current_period_end` integer NOT NULL,
`current_period_start` integer NOT NULL,
`cancel_at_period_end` integer DEFAULT false NOT NULL,
`created_at` integer NOT NULL,
`updated_at` integer NOT NULL,
FOREIGN KEY (`organization_id`) REFERENCES `organizations`(`id`) ON UPDATE cascade ON DELETE cascade
);

View File

@@ -1,23 +0,0 @@
-- Migration for adding full-text search virtual table for documents
CREATE VIRTUAL TABLE documents_fts USING fts5(id UNINDEXED, name, original_name, content, prefix='2 3 4');
--> statement-breakpoint
-- Copy data from documents to documents_fts for existing records
INSERT INTO documents_fts(id, name, original_name, content)
SELECT id, name, original_name, content FROM documents;
--> statement-breakpoint
CREATE TRIGGER trigger_documents_fts_insert AFTER INSERT ON documents BEGIN
INSERT INTO documents_fts(id, name, original_name, content) VALUES (new.id, new.name, new.original_name, new.content);
END;
--> statement-breakpoint
CREATE TRIGGER trigger_documents_fts_update AFTER UPDATE ON documents BEGIN
UPDATE documents_fts SET name = new.name, original_name = new.original_name, content = new.content WHERE id = new.id;
END;
--> statement-breakpoint
CREATE TRIGGER trigger_documents_fts_delete AFTER DELETE ON documents BEGIN
DELETE FROM documents_fts WHERE id = old.id;
END;

View File

@@ -1,32 +0,0 @@
CREATE TABLE `tagging_rule_actions` (
`id` text PRIMARY KEY NOT NULL,
`created_at` integer NOT NULL,
`updated_at` integer NOT NULL,
`tagging_rule_id` text NOT NULL,
`tag_id` text NOT NULL,
FOREIGN KEY (`tagging_rule_id`) REFERENCES `tagging_rules`(`id`) ON UPDATE cascade ON DELETE cascade,
FOREIGN KEY (`tag_id`) REFERENCES `tags`(`id`) ON UPDATE cascade ON DELETE cascade
);
--> statement-breakpoint
CREATE TABLE `tagging_rule_conditions` (
`id` text PRIMARY KEY NOT NULL,
`created_at` integer NOT NULL,
`updated_at` integer NOT NULL,
`tagging_rule_id` text NOT NULL,
`field` text NOT NULL,
`operator` text NOT NULL,
`value` text NOT NULL,
`is_case_sensitive` integer DEFAULT false NOT NULL,
FOREIGN KEY (`tagging_rule_id`) REFERENCES `tagging_rules`(`id`) ON UPDATE cascade ON DELETE cascade
);
--> statement-breakpoint
CREATE TABLE `tagging_rules` (
`id` text PRIMARY KEY NOT NULL,
`created_at` integer NOT NULL,
`updated_at` integer NOT NULL,
`organization_id` text NOT NULL,
`name` text NOT NULL,
`description` text,
`enabled` integer DEFAULT true NOT NULL,
FOREIGN KEY (`organization_id`) REFERENCES `organizations`(`id`) ON UPDATE cascade ON DELETE cascade
);

View File

@@ -1,24 +0,0 @@
CREATE TABLE `api_key_organizations` (
`api_key_id` text NOT NULL,
`organization_member_id` text NOT NULL,
FOREIGN KEY (`api_key_id`) REFERENCES `api_keys`(`id`) ON UPDATE cascade ON DELETE cascade,
FOREIGN KEY (`organization_member_id`) REFERENCES `organization_members`(`id`) ON UPDATE cascade ON DELETE cascade
);
--> statement-breakpoint
CREATE TABLE `api_keys` (
`id` text PRIMARY KEY NOT NULL,
`created_at` integer NOT NULL,
`updated_at` integer NOT NULL,
`name` text NOT NULL,
`key_hash` text NOT NULL,
`prefix` text NOT NULL,
`user_id` text NOT NULL,
`last_used_at` integer,
`expires_at` integer,
`permissions` text DEFAULT '[]' NOT NULL,
`all_organizations` integer DEFAULT false NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE cascade ON DELETE cascade
);
--> statement-breakpoint
CREATE UNIQUE INDEX `api_keys_key_hash_unique` ON `api_keys` (`key_hash`);--> statement-breakpoint
CREATE INDEX `key_hash_index` ON `api_keys` (`key_hash`);

View File

@@ -1,35 +0,0 @@
CREATE TABLE `webhook_deliveries` (
`id` text PRIMARY KEY NOT NULL,
`created_at` integer NOT NULL,
`updated_at` integer NOT NULL,
`webhook_id` text NOT NULL,
`event_name` text NOT NULL,
`request_payload` text NOT NULL,
`response_payload` text NOT NULL,
`response_status` integer NOT NULL,
FOREIGN KEY (`webhook_id`) REFERENCES `webhooks`(`id`) ON UPDATE cascade ON DELETE cascade
);
--> statement-breakpoint
CREATE TABLE `webhook_events` (
`id` text PRIMARY KEY NOT NULL,
`created_at` integer NOT NULL,
`updated_at` integer NOT NULL,
`webhook_id` text NOT NULL,
`event_name` text NOT NULL,
FOREIGN KEY (`webhook_id`) REFERENCES `webhooks`(`id`) ON UPDATE cascade ON DELETE cascade
);
--> statement-breakpoint
CREATE UNIQUE INDEX `webhook_events_webhook_id_event_name_unique` ON `webhook_events` (`webhook_id`,`event_name`);--> statement-breakpoint
CREATE TABLE `webhooks` (
`id` text PRIMARY KEY NOT NULL,
`created_at` integer NOT NULL,
`updated_at` integer NOT NULL,
`name` text NOT NULL,
`url` text NOT NULL,
`secret` text,
`enabled` integer DEFAULT true NOT NULL,
`created_by` text,
`organization_id` text,
FOREIGN KEY (`created_by`) REFERENCES `users`(`id`) ON UPDATE cascade ON DELETE set null,
FOREIGN KEY (`organization_id`) REFERENCES `organizations`(`id`) ON UPDATE cascade ON DELETE cascade
);

View File

@@ -1,4 +0,0 @@
ALTER TABLE `organization_invitations` ALTER COLUMN "role" TO "role" text NOT NULL;--> statement-breakpoint
CREATE UNIQUE INDEX `organization_invitations_organization_email_unique` ON `organization_invitations` (`organization_id`,`email`);--> statement-breakpoint
ALTER TABLE `organization_invitations` ALTER COLUMN "status" TO "status" text NOT NULL DEFAULT 'pending';

View File

@@ -1,12 +0,0 @@
CREATE TABLE `document_activity_log` (
`id` text PRIMARY KEY NOT NULL,
`created_at` integer NOT NULL,
`document_id` text NOT NULL,
`event` text NOT NULL,
`event_data` text,
`user_id` text,
`tag_id` text,
FOREIGN KEY (`document_id`) REFERENCES `documents`(`id`) ON UPDATE cascade ON DELETE cascade,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE cascade ON DELETE no action,
FOREIGN KEY (`tag_id`) REFERENCES `tags`(`id`) ON UPDATE cascade ON DELETE no action
);

View File

@@ -1,7 +1,7 @@
{
"name": "@papra/app-server",
"type": "module",
"version": "0.7.0",
"version": "0.8.0",
"private": true,
"packageManager": "pnpm@10.12.3",
"description": "Papra app server",
@@ -18,8 +18,9 @@
"test": "vitest run",
"test:watch": "vitest watch",
"typecheck": "tsc --noEmit",
"migrate:up": "tsx --env-file-if-exists=.env src/scripts/migrate-up.script.ts",
"migrate:up": "tsx --env-file-if-exists=.env src/scripts/migrate-up.script.ts | crowlog-pretty",
"migrate:push": "drizzle-kit push",
"migrate:create": "sh -c 'drizzle-kit generate --name \"$1\" && tsx --env-file-if-exists=.env src/scripts/create-migration.ts \"$1\" | crowlog-pretty' --",
"db:studio": "drizzle-kit studio",
"clean:dist": "rm -rf dist",
"clean:db": "rm db.sqlite",
@@ -33,6 +34,8 @@
"@aws-sdk/client-s3": "^3.835.0",
"@aws-sdk/lib-storage": "^3.835.0",
"@azure/storage-blob": "^12.27.0",
"@cadence-mq/core": "^0.2.1",
"@cadence-mq/driver-memory": "^0.2.0",
"@corentinth/chisels": "^1.3.1",
"@corentinth/friendly-ids": "^0.0.1",
"@crowlog/async-context-plugin": "^1.2.1",
@@ -83,6 +86,7 @@
"@vitest/coverage-v8": "catalog:",
"esbuild": "^0.24.2",
"eslint": "catalog:",
"magicast": "^0.3.5",
"memfs": "^4.17.2",
"typescript": "catalog:",
"vitest": "catalog:"

View File

@@ -7,8 +7,8 @@ import { createServer } from './modules/app/server';
import { parseConfig } from './modules/config/config';
import { createIngestionFolderWatcher } from './modules/ingestion-folders/ingestion-folders.usecases';
import { createLogger } from './modules/shared/logger/logger';
import { createTaskScheduler } from './modules/tasks/task-scheduler';
import { taskDefinitions } from './modules/tasks/tasks.defiitions';
import { registerTaskDefinitions } from './modules/tasks/tasks.definitions';
import { createTaskServices } from './modules/tasks/tasks.services';
const logger = createLogger({ namespace: 'app-server' });
@@ -17,8 +17,8 @@ const { config } = await parseConfig({ env });
await ensureLocalDatabaseDirectoryExists({ config });
const { db, client } = setupDatabase(config.database);
const { app } = await createServer({ config, db });
const { taskScheduler } = createTaskScheduler({ config, taskDefinitions, tasksArgs: { db } });
const taskServices = createTaskServices({ config });
const { app } = await createServer({ config, db, taskServices });
const server = serve(
{
@@ -30,6 +30,7 @@ const server = serve(
if (config.ingestionFolder.isEnabled) {
const { startWatchingIngestionFolders } = createIngestionFolderWatcher({
taskServices,
config,
db,
});
@@ -37,11 +38,12 @@ if (config.ingestionFolder.isEnabled) {
await startWatchingIngestionFolders();
}
taskScheduler.start();
await registerTaskDefinitions({ taskServices, db, config });
taskServices.start();
process.on('SIGINT', async () => {
server.close();
taskScheduler.stop();
client.close();
process.exit(0);

View File

@@ -0,0 +1,51 @@
import { sql } from 'drizzle-orm';
import { describe, expect, test } from 'vitest';
import { setupDatabase } from '../../modules/app/database/database';
import { initialSchemaSetupMigration } from './0001-initial-schema-setup.migration';
describe('0001-initial-schema-setup migration', () => {
describe('initialSchemaSetupMigration', () => {
test('the up setup some default tables', async () => {
const { db } = setupDatabase({ url: ':memory:' });
await initialSchemaSetupMigration.up({ db });
const { rows: existingTables } = await db.run(sql`SELECT name FROM sqlite_master WHERE name NOT LIKE 'sqlite_%'`);
expect(existingTables.map(({ name }) => name)).to.eql([
'documents',
'documents_organization_id_is_deleted_created_at_index',
'documents_organization_id_is_deleted_index',
'documents_organization_id_original_sha256_hash_unique',
'documents_original_sha256_hash_index',
'documents_organization_id_size_index',
'organization_invitations',
'organization_members',
'organization_members_user_organization_unique',
'organizations',
'user_roles',
'user_roles_role_index',
'user_roles_user_id_role_unique_index',
'documents_tags',
'tags',
'tags_organization_id_name_unique',
'users',
'users_email_unique',
'users_email_index',
'auth_accounts',
'auth_sessions',
'auth_sessions_token_index',
'auth_verifications',
'auth_verifications_identifier_index',
'intake_emails',
'intake_emails_email_address_unique',
'organization_subscriptions',
]);
await initialSchemaSetupMigration.down({ db });
const { rows: existingTablesAfterDown } = await db.run(sql`SELECT name FROM sqlite_master WHERE name NOT LIKE 'sqlite_%'`);
expect(existingTablesAfterDown.map(({ name }) => name)).to.eql([]);
});
});
});

View File

@@ -0,0 +1,220 @@
import type { Migration } from '../migrations.types';
import { sql } from 'drizzle-orm';
export const initialSchemaSetupMigration = {
name: 'initial-schema-setup',
description: 'Creation of the base tables for the application',
up: async ({ db }) => {
await db.batch([
db.run(sql`
CREATE TABLE IF NOT EXISTS "documents" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"updated_at" integer NOT NULL,
"is_deleted" integer DEFAULT false NOT NULL,
"deleted_at" integer,
"organization_id" text NOT NULL,
"created_by" text,
"deleted_by" text,
"original_name" text NOT NULL,
"original_size" integer DEFAULT 0 NOT NULL,
"original_storage_key" text NOT NULL,
"original_sha256_hash" text NOT NULL,
"name" text NOT NULL,
"mime_type" text NOT NULL,
"content" text DEFAULT '' NOT NULL,
FOREIGN KEY ("organization_id") REFERENCES "organizations"("id") ON UPDATE cascade ON DELETE cascade,
FOREIGN KEY ("created_by") REFERENCES "users"("id") ON UPDATE cascade ON DELETE set null,
FOREIGN KEY ("deleted_by") REFERENCES "users"("id") ON UPDATE cascade ON DELETE set null
);
`),
db.run(sql`CREATE INDEX IF NOT EXISTS "documents_organization_id_is_deleted_created_at_index" ON "documents" ("organization_id","is_deleted","created_at");`),
db.run(sql`CREATE INDEX IF NOT EXISTS "documents_organization_id_is_deleted_index" ON "documents" ("organization_id","is_deleted");`),
db.run(sql`CREATE UNIQUE INDEX IF NOT EXISTS "documents_organization_id_original_sha256_hash_unique" ON "documents" ("organization_id","original_sha256_hash");`),
db.run(sql`CREATE INDEX IF NOT EXISTS "documents_original_sha256_hash_index" ON "documents" ("original_sha256_hash");`),
db.run(sql`CREATE INDEX IF NOT EXISTS "documents_organization_id_size_index" ON "documents" ("organization_id","original_size");`),
db.run(sql`
CREATE TABLE IF NOT EXISTS "organization_invitations" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"updated_at" integer NOT NULL,
"organization_id" text NOT NULL,
"email" text NOT NULL,
"role" text,
"status" text NOT NULL,
"expires_at" integer NOT NULL,
"inviter_id" text NOT NULL,
FOREIGN KEY ("organization_id") REFERENCES "organizations"("id") ON UPDATE cascade ON DELETE cascade,
FOREIGN KEY ("inviter_id") REFERENCES "users"("id") ON UPDATE cascade ON DELETE cascade
);
`),
db.run(sql`CREATE TABLE IF NOT EXISTS "organization_members" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"updated_at" integer NOT NULL,
"organization_id" text NOT NULL,
"user_id" text NOT NULL,
"role" text NOT NULL,
FOREIGN KEY ("organization_id") REFERENCES "organizations"("id") ON UPDATE cascade ON DELETE cascade,
FOREIGN KEY ("user_id") REFERENCES "users"("id") ON UPDATE cascade ON DELETE cascade
);`),
db.run(sql`CREATE UNIQUE INDEX IF NOT EXISTS "organization_members_user_organization_unique" ON "organization_members" ("organization_id","user_id");`),
db.run(sql`CREATE TABLE IF NOT EXISTS "organizations" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"updated_at" integer NOT NULL,
"name" text NOT NULL,
"customer_id" text
);`),
db.run(sql`CREATE TABLE IF NOT EXISTS "user_roles" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"updated_at" integer NOT NULL,
"user_id" text NOT NULL,
"role" text NOT NULL,
FOREIGN KEY ("user_id") REFERENCES "users"("id") ON UPDATE cascade ON DELETE cascade
);`),
db.run(sql`CREATE INDEX IF NOT EXISTS "user_roles_role_index" ON "user_roles" ("role");`),
db.run(sql`CREATE UNIQUE INDEX IF NOT EXISTS "user_roles_user_id_role_unique_index" ON "user_roles" ("user_id","role");`),
db.run(sql`CREATE TABLE IF NOT EXISTS "documents_tags" (
"document_id" text NOT NULL,
"tag_id" text NOT NULL,
PRIMARY KEY("document_id", "tag_id"),
FOREIGN KEY ("document_id") REFERENCES "documents"("id") ON UPDATE cascade ON DELETE cascade,
FOREIGN KEY ("tag_id") REFERENCES "tags"("id") ON UPDATE cascade ON DELETE cascade
);`),
db.run(sql`CREATE TABLE IF NOT EXISTS "tags" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"updated_at" integer NOT NULL,
"organization_id" text NOT NULL,
"name" text NOT NULL,
"color" text NOT NULL,
"description" text,
FOREIGN KEY ("organization_id") REFERENCES "organizations"("id") ON UPDATE cascade ON DELETE cascade
);`),
db.run(sql`CREATE UNIQUE INDEX IF NOT EXISTS "tags_organization_id_name_unique" ON "tags" ("organization_id","name");`),
db.run(sql`
CREATE TABLE IF NOT EXISTS "users" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"updated_at" integer NOT NULL,
"email" text NOT NULL,
"email_verified" integer DEFAULT false NOT NULL,
"name" text,
"image" text,
"max_organization_count" integer
);
`),
db.run(sql`CREATE UNIQUE INDEX IF NOT EXISTS "users_email_unique" ON "users" ("email");`),
db.run(sql`CREATE INDEX IF NOT EXISTS "users_email_index" ON "users" ("email");`),
db.run(sql`CREATE TABLE IF NOT EXISTS "auth_accounts" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"updated_at" integer NOT NULL,
"user_id" text,
"account_id" text NOT NULL,
"provider_id" text NOT NULL,
"access_token" text,
"refresh_token" text,
"access_token_expires_at" integer,
"refresh_token_expires_at" integer,
"scope" text,
"id_token" text,
"password" text,
FOREIGN KEY ("user_id") REFERENCES "users"("id") ON UPDATE cascade ON DELETE cascade
);`),
db.run(sql`CREATE TABLE IF NOT EXISTS "auth_sessions" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"updated_at" integer NOT NULL,
"token" text NOT NULL,
"user_id" text,
"expires_at" integer NOT NULL,
"ip_address" text,
"user_agent" text,
"active_organization_id" text,
FOREIGN KEY ("user_id") REFERENCES "users"("id") ON UPDATE cascade ON DELETE cascade,
FOREIGN KEY ("active_organization_id") REFERENCES "organizations"("id") ON UPDATE cascade ON DELETE set null
);`),
db.run(sql`CREATE INDEX IF NOT EXISTS "auth_sessions_token_index" ON "auth_sessions" ("token");`),
db.run(sql`CREATE TABLE IF NOT EXISTS "auth_verifications" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"updated_at" integer NOT NULL,
"identifier" text NOT NULL,
"value" text NOT NULL,
"expires_at" integer NOT NULL
);`),
db.run(sql`CREATE INDEX IF NOT EXISTS "auth_verifications_identifier_index" ON "auth_verifications" ("identifier");`),
db.run(sql`CREATE TABLE IF NOT EXISTS "intake_emails" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"updated_at" integer NOT NULL,
"email_address" text NOT NULL,
"organization_id" text NOT NULL,
"allowed_origins" text DEFAULT '[]' NOT NULL,
"is_enabled" integer DEFAULT true NOT NULL,
FOREIGN KEY ("organization_id") REFERENCES "organizations"("id") ON UPDATE cascade ON DELETE cascade
);`),
db.run(sql`CREATE UNIQUE INDEX IF NOT EXISTS "intake_emails_email_address_unique" ON "intake_emails" ("email_address");`),
db.run(sql`CREATE TABLE IF NOT EXISTS "organization_subscriptions" (
"id" text PRIMARY KEY NOT NULL,
"customer_id" text NOT NULL,
"organization_id" text NOT NULL,
"plan_id" text NOT NULL,
"status" text NOT NULL,
"seats_count" integer NOT NULL,
"current_period_end" integer NOT NULL,
"current_period_start" integer NOT NULL,
"cancel_at_period_end" integer DEFAULT false NOT NULL,
"created_at" integer NOT NULL,
"updated_at" integer NOT NULL,
FOREIGN KEY ("organization_id") REFERENCES "organizations"("id") ON UPDATE cascade ON DELETE cascade
);`),
]);
},
down: async ({ db }) => {
await db.batch([
// Tables
db.run(sql`DROP TABLE IF EXISTS "organization_subscriptions";`),
db.run(sql`DROP TABLE IF EXISTS "intake_emails";`),
db.run(sql`DROP TABLE IF EXISTS "auth_verifications";`),
db.run(sql`DROP TABLE IF EXISTS "auth_sessions";`),
db.run(sql`DROP TABLE IF EXISTS "auth_accounts";`),
db.run(sql`DROP TABLE IF EXISTS "tags";`),
db.run(sql`DROP TABLE IF EXISTS "documents_tags";`),
db.run(sql`DROP TABLE IF EXISTS "user_roles";`),
db.run(sql`DROP TABLE IF EXISTS "organizations";`),
db.run(sql`DROP TABLE IF EXISTS "organization_members";`),
db.run(sql`DROP TABLE IF EXISTS "organization_invitations";`),
db.run(sql`DROP TABLE IF EXISTS "documents";`),
db.run(sql`DROP TABLE IF EXISTS "users";`),
// // Indexes
db.run(sql`DROP INDEX IF EXISTS "documents_organization_id_is_deleted_created_at_index";`),
db.run(sql`DROP INDEX IF EXISTS "documents_organization_id_is_deleted_index";`),
db.run(sql`DROP INDEX IF EXISTS "documents_organization_id_original_sha256_hash_unique";`),
db.run(sql`DROP INDEX IF EXISTS "documents_original_sha256_hash_index";`),
db.run(sql`DROP INDEX IF EXISTS "documents_organization_id_size_index";`),
db.run(sql`DROP INDEX IF EXISTS "user_roles_role_index";`),
db.run(sql`DROP INDEX IF EXISTS "user_roles_user_id_role_unique_index";`),
db.run(sql`DROP INDEX IF EXISTS "tags_organization_id_name_unique";`),
db.run(sql`DROP INDEX IF EXISTS "users_email_unique";`),
]);
},
} satisfies Migration;

View File

@@ -0,0 +1,37 @@
import type { Migration } from '../migrations.types';
import { sql } from 'drizzle-orm';
export const documentsFtsMigration = {
name: 'documents-fts',
up: async ({ db }) => {
await db.batch([
db.run(sql`CREATE VIRTUAL TABLE IF NOT EXISTS documents_fts USING fts5(id UNINDEXED, name, original_name, content, prefix='2 3 4')`),
db.run(sql`INSERT INTO documents_fts(id, name, original_name, content) SELECT id, name, original_name, content FROM documents`),
db.run(sql`
CREATE TRIGGER IF NOT EXISTS trigger_documents_fts_insert AFTER INSERT ON documents BEGIN
INSERT INTO documents_fts(id, name, original_name, content) VALUES (new.id, new.name, new.original_name, new.content);
END
`),
db.run(sql`
CREATE TRIGGER IF NOT EXISTS trigger_documents_fts_update AFTER UPDATE ON documents BEGIN
UPDATE documents_fts SET name = new.name, original_name = new.original_name, content = new.content WHERE id = new.id;
END
`),
db.run(sql`
CREATE TRIGGER IF NOT EXISTS trigger_documents_fts_delete AFTER DELETE ON documents BEGIN
DELETE FROM documents_fts WHERE id = old.id;
END
`),
]);
},
down: async ({ db }) => {
await db.batch([
db.run(sql`DROP TRIGGER IF EXISTS trigger_documents_fts_insert`),
db.run(sql`DROP TRIGGER IF EXISTS trigger_documents_fts_update`),
db.run(sql`DROP TRIGGER IF EXISTS trigger_documents_fts_delete`),
db.run(sql`DROP TABLE IF EXISTS documents_fts`),
]);
},
} satisfies Migration;

View File

@@ -0,0 +1,57 @@
import type { Migration } from '../migrations.types';
import { sql } from 'drizzle-orm';
export const taggingRulesMigration = {
name: 'tagging-rules',
up: async ({ db }) => {
await db.batch([
db.run(sql`
CREATE TABLE IF NOT EXISTS "tagging_rule_actions" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"updated_at" integer NOT NULL,
"tagging_rule_id" text NOT NULL,
"tag_id" text NOT NULL,
FOREIGN KEY ("tagging_rule_id") REFERENCES "tagging_rules"("id") ON UPDATE cascade ON DELETE cascade,
FOREIGN KEY ("tag_id") REFERENCES "tags"("id") ON UPDATE cascade ON DELETE cascade
);
`),
db.run(sql`
CREATE TABLE IF NOT EXISTS "tagging_rule_conditions" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"updated_at" integer NOT NULL,
"tagging_rule_id" text NOT NULL,
"field" text NOT NULL,
"operator" text NOT NULL,
"value" text NOT NULL,
"is_case_sensitive" integer DEFAULT false NOT NULL,
FOREIGN KEY ("tagging_rule_id") REFERENCES "tagging_rules"("id") ON UPDATE cascade ON DELETE cascade
);
`),
db.run(sql`
CREATE TABLE IF NOT EXISTS "tagging_rules" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"updated_at" integer NOT NULL,
"organization_id" text NOT NULL,
"name" text NOT NULL,
"description" text,
"enabled" integer DEFAULT true NOT NULL,
FOREIGN KEY ("organization_id") REFERENCES "organizations"("id") ON UPDATE cascade ON DELETE cascade
);
`),
]);
},
down: async ({ db }) => {
await db.batch([
db.run(sql`DROP TABLE IF EXISTS "tagging_rule_actions"`),
db.run(sql`DROP TABLE IF EXISTS "tagging_rule_conditions"`),
db.run(sql`DROP TABLE IF EXISTS "tagging_rules"`),
]);
},
} satisfies Migration;

View File

@@ -0,0 +1,46 @@
import type { Migration } from '../migrations.types';
import { sql } from 'drizzle-orm';
export const apiKeysMigration = {
name: 'api-keys',
up: async ({ db }) => {
await db.batch([
db.run(sql`
CREATE TABLE IF NOT EXISTS "api_key_organizations" (
"api_key_id" text NOT NULL,
"organization_member_id" text NOT NULL,
FOREIGN KEY ("api_key_id") REFERENCES "api_keys"("id") ON UPDATE cascade ON DELETE cascade,
FOREIGN KEY ("organization_member_id") REFERENCES "organization_members"("id") ON UPDATE cascade ON DELETE cascade
);
`),
db.run(sql`
CREATE TABLE IF NOT EXISTS "api_keys" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"updated_at" integer NOT NULL,
"name" text NOT NULL,
"key_hash" text NOT NULL,
"prefix" text NOT NULL,
"user_id" text NOT NULL,
"last_used_at" integer,
"expires_at" integer,
"permissions" text DEFAULT '[]' NOT NULL,
"all_organizations" integer DEFAULT false NOT NULL,
FOREIGN KEY ("user_id") REFERENCES "users"("id") ON UPDATE cascade ON DELETE cascade
);
`),
db.run(sql`CREATE UNIQUE INDEX IF NOT EXISTS "api_keys_key_hash_unique" ON "api_keys" ("key_hash")`),
db.run(sql`CREATE INDEX IF NOT EXISTS "key_hash_index" ON "api_keys" ("key_hash")`),
]);
},
down: async ({ db }) => {
await db.batch([
db.run(sql`DROP TABLE IF EXISTS "api_key_organizations"`),
db.run(sql`DROP TABLE IF EXISTS "api_keys"`),
db.run(sql`DROP INDEX IF EXISTS "api_keys_key_hash_unique"`),
db.run(sql`DROP INDEX IF EXISTS "key_hash_index"`),
]);
},
} satisfies Migration;

View File

@@ -0,0 +1,62 @@
import type { Migration } from '../migrations.types';
import { sql } from 'drizzle-orm';
export const organizationsWebhooksMigration = {
name: 'organizations-webhooks',
up: async ({ db }) => {
await db.batch([
db.run(sql`
CREATE TABLE IF NOT EXISTS "webhook_deliveries" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"updated_at" integer NOT NULL,
"webhook_id" text NOT NULL,
"event_name" text NOT NULL,
"request_payload" text NOT NULL,
"response_payload" text NOT NULL,
"response_status" integer NOT NULL,
FOREIGN KEY ("webhook_id") REFERENCES "webhooks"("id") ON UPDATE cascade ON DELETE cascade
);
`),
db.run(sql`
CREATE TABLE IF NOT EXISTS "webhook_events" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"updated_at" integer NOT NULL,
"webhook_id" text NOT NULL,
"event_name" text NOT NULL,
FOREIGN KEY ("webhook_id") REFERENCES "webhooks"("id") ON UPDATE cascade ON DELETE cascade
);
`),
db.run(sql`CREATE UNIQUE INDEX IF NOT EXISTS "webhook_events_webhook_id_event_name_unique" ON "webhook_events" ("webhook_id","event_name")`),
db.run(sql`
CREATE TABLE IF NOT EXISTS "webhooks" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"updated_at" integer NOT NULL,
"name" text NOT NULL,
"url" text NOT NULL,
"secret" text,
"enabled" integer DEFAULT true NOT NULL,
"created_by" text,
"organization_id" text,
FOREIGN KEY ("created_by") REFERENCES "users"("id") ON UPDATE cascade ON DELETE set null,
FOREIGN KEY ("organization_id") REFERENCES "organizations"("id") ON UPDATE cascade ON DELETE cascade
);
`),
]);
},
down: async ({ db }) => {
await db.batch([
db.run(sql`DROP TABLE IF EXISTS "webhook_deliveries"`),
db.run(sql`DROP TABLE IF EXISTS "webhook_events"`),
db.run(sql`DROP INDEX IF EXISTS "webhook_events_webhook_id_event_name_unique"`),
db.run(sql`DROP TABLE IF EXISTS "webhooks"`),
]);
},
} satisfies Migration;

View File

@@ -0,0 +1,22 @@
import type { Migration } from '../migrations.types';
import { sql } from 'drizzle-orm';
export const organizationsInvitationsImprovementMigration = {
name: 'organizations-invitations-improvement',
up: async ({ db }) => {
await db.batch([
db.run(sql`ALTER TABLE "organization_invitations" ALTER COLUMN "role" TO "role" text NOT NULL`),
db.run(sql`CREATE UNIQUE INDEX IF NOT EXISTS "organization_invitations_organization_email_unique" ON "organization_invitations" ("organization_id","email")`),
db.run(sql`ALTER TABLE "organization_invitations" ALTER COLUMN "status" TO "status" text NOT NULL DEFAULT 'pending'`),
]);
},
down: async ({ db }) => {
await db.batch([
db.run(sql`ALTER TABLE "organization_invitations" ALTER COLUMN "role" TO "role" text`),
db.run(sql`DROP INDEX IF EXISTS "organization_invitations_organization_email_unique"`),
db.run(sql`ALTER TABLE "organization_invitations" ALTER COLUMN "status" TO "status" text NOT NULL`),
]);
},
} satisfies Migration;

View File

@@ -0,0 +1,31 @@
import type { Migration } from '../migrations.types';
import { sql } from 'drizzle-orm';
export const documentActivityLogMigration = {
name: 'document-activity-log',
up: async ({ db }) => {
await db.batch([
db.run(sql`
CREATE TABLE IF NOT EXISTS "document_activity_log" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"document_id" text NOT NULL,
"event" text NOT NULL,
"event_data" text,
"user_id" text,
"tag_id" text,
FOREIGN KEY ("document_id") REFERENCES "documents"("id") ON UPDATE cascade ON DELETE cascade,
FOREIGN KEY ("user_id") REFERENCES "users"("id") ON UPDATE cascade ON DELETE no action,
FOREIGN KEY ("tag_id") REFERENCES "tags"("id") ON UPDATE cascade ON DELETE no action
);
`),
]);
},
down: async ({ db }) => {
await db.batch([
db.run(sql`DROP TABLE IF EXISTS "document_activity_log"`),
]);
},
} satisfies Migration;

View File

@@ -0,0 +1,56 @@
import type { Migration } from '../migrations.types';
import { sql } from 'drizzle-orm';
export const documentActivityLogOnDeleteSetNullMigration = {
name: 'document-activity-log-on-delete-set-null',
up: async ({ db }) => {
await db.batch([
db.run(sql`PRAGMA foreign_keys=OFF`),
db.run(sql`
CREATE TABLE "__new_document_activity_log" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"document_id" text NOT NULL,
"event" text NOT NULL,
"event_data" text,
"user_id" text,
"tag_id" text,
FOREIGN KEY ("document_id") REFERENCES "documents"("id") ON UPDATE cascade ON DELETE cascade,
FOREIGN KEY ("user_id") REFERENCES "users"("id") ON UPDATE cascade ON DELETE set null,
FOREIGN KEY ("tag_id") REFERENCES "tags"("id") ON UPDATE cascade ON DELETE set null
);
`),
db.run(sql`
INSERT INTO "__new_document_activity_log"("id", "created_at", "document_id", "event", "event_data", "user_id", "tag_id") SELECT "id", "created_at", "document_id", "event", "event_data", "user_id", "tag_id" FROM "document_activity_log";
`),
db.run(sql`DROP TABLE IF EXISTS "document_activity_log"`),
db.run(sql`ALTER TABLE "__new_document_activity_log" RENAME TO "document_activity_log"`),
db.run(sql`PRAGMA foreign_keys=ON`),
]);
},
down: async ({ db }) => {
await db.batch([
db.run(sql`PRAGMA foreign_keys=OFF`),
db.run(sql`
CREATE TABLE "__restore_document_activity_log" (
"id" text PRIMARY KEY NOT NULL,
"created_at" integer NOT NULL,
"document_id" text NOT NULL,
"event" text NOT NULL,
"event_data" text,
"user_id" text,
"tag_id" text,
FOREIGN KEY ("document_id") REFERENCES "documents"("id") ON UPDATE cascade ON DELETE cascade,
FOREIGN KEY ("user_id") REFERENCES "users"("id") ON UPDATE cascade ON DELETE no action,
FOREIGN KEY ("tag_id") REFERENCES "tags"("id") ON UPDATE cascade ON DELETE no action
);
`),
db.run(sql`INSERT INTO "__restore_document_activity_log"("id", "created_at", "document_id", "event", "event_data", "user_id", "tag_id") SELECT "id", "created_at", "document_id", "event", "event_data", "user_id", "tag_id" FROM "document_activity_log";`),
db.run(sql`DROP TABLE IF EXISTS "document_activity_log"`),
db.run(sql`ALTER TABLE "__restore_document_activity_log" RENAME TO "document_activity_log"`),
db.run(sql`PRAGMA foreign_keys=ON`),
]);
},
} satisfies Migration;

View File

@@ -0,0 +1,12 @@
import type { Migration } from '../migrations.types';
import { sql } from 'drizzle-orm';
export const dropLegacyMigrationsMigration = {
name: 'drop-legacy-migrations',
description: 'Drop the legacy migrations table as it is not used anymore',
up: async ({ db }) => {
await db.run(sql`DROP TABLE IF EXISTS "__drizzle_migrations"`);
},
} satisfies Migration;

File diff suppressed because it is too large Load Diff

View File

@@ -50,6 +50,13 @@
"when": 1748554484124,
"tag": "0006_document-activity-log",
"breakpoints": true
},
{
"idx": 7,
"version": "6",
"when": 1754086182584,
"tag": "0007_document-activity-log-on-delete-set-null",
"breakpoints": true
}
]
}

View File

@@ -0,0 +1,14 @@
import { index, integer, sqliteTable, text } from 'drizzle-orm/sqlite-core';
export const migrationsTable = sqliteTable(
'migrations',
{
id: integer('id').primaryKey({ autoIncrement: true }),
name: text('name').notNull(),
runAt: integer('run_at', { mode: 'timestamp_ms' }).notNull().$default(() => new Date()),
},
t => [
index('name_index').on(t.name),
index('run_at_index').on(t.runAt),
],
);

View File

@@ -0,0 +1,141 @@
import type { Migration } from './migrations.types';
import { sql } from 'drizzle-orm';
import { describe, expect, test } from 'vitest';
import { setupDatabase } from '../modules/app/database/database';
import { serializeSchema } from '../modules/app/database/database.test-utils';
import { migrations } from './migrations.registry';
import { rollbackLastAppliedMigration, runMigrations } from './migrations.usecases';
describe('migrations registry', () => {
describe('migrations', () => {
test('each migration should have a unique name', () => {
const migrationNames = migrations.map(m => m.name);
const duplicateMigrationNames = migrationNames.filter(name => migrationNames.filter(n => n === name).length > 1);
expect(duplicateMigrationNames).to.eql([], 'Each migration should have a unique name');
});
test('each migration should have a non empty name', () => {
const migrationNames = migrations.map(m => m.name);
const emptyMigrationNames = migrationNames.filter(name => name === '');
expect(emptyMigrationNames).to.eql([], 'Each migration should have a non empty name');
});
test('all migrations must be able to be applied without error and the database should be in a consistent state', async () => {
const { db } = setupDatabase({ url: ':memory:' });
// This will throw if any migration is not able to be applied
await runMigrations({ db, migrations });
// check foreign keys are enabled
const { rows } = await db.run(sql`pragma foreign_keys;`);
expect(rows).to.eql([{ foreign_keys: 1 }]);
});
test('we can stop to any migration and still have a consistent database state', async () => {
// Given like 3 migrations [A,B,C], creates [[A], [A,B], [A,B,C]]
const migrationCombinations = migrations.map((m, i) => migrations.slice(0, i + 1));
for (const migrationCombination of migrationCombinations) {
const { db } = setupDatabase({ url: ':memory:' });
await runMigrations({ db, migrations: migrationCombination });
}
});
test('when we rollback to a previous migration, the database should be in the state of the previous migration', async () => {
// Given like 3 migrations [A,B,C], creates [[A], [A,B], [A,B,C]]
const migrationCombinations = migrations.map((m, i) => migrations.slice(0, i + 1));
for (const [index, migrationCombination] of migrationCombinations.entries()) {
const { db } = setupDatabase({ url: ':memory:' });
const previousMigration = migrationCombinations[index - 1] ?? [] as Migration[];
await runMigrations({ db, migrations: previousMigration });
const previousDbState = await serializeSchema({ db });
await runMigrations({ db, migrations: migrationCombination });
await rollbackLastAppliedMigration({ db });
const currentDbState = await serializeSchema({ db });
expect(currentDbState).to.eql(previousDbState, `Downgrading from ${migrationCombination.at(-1)?.name ?? 'no migration'} should result in the same state as the previous migration`);
}
});
test('regression test of the database state after running migrations, update the snapshot when the database state changes', async () => {
const { db } = setupDatabase({ url: ':memory:' });
await runMigrations({ db, migrations });
expect(await serializeSchema({ db })).toMatchInlineSnapshot(`
"CREATE UNIQUE INDEX "api_keys_key_hash_unique" ON "api_keys" ("key_hash");
CREATE INDEX "auth_sessions_token_index" ON "auth_sessions" ("token");
CREATE INDEX "auth_verifications_identifier_index" ON "auth_verifications" ("identifier");
CREATE INDEX "documents_organization_id_is_deleted_created_at_index" ON "documents" ("organization_id","is_deleted","created_at");
CREATE INDEX "documents_organization_id_is_deleted_index" ON "documents" ("organization_id","is_deleted");
CREATE UNIQUE INDEX "documents_organization_id_original_sha256_hash_unique" ON "documents" ("organization_id","original_sha256_hash");
CREATE INDEX "documents_organization_id_size_index" ON "documents" ("organization_id","original_size");
CREATE INDEX "documents_original_sha256_hash_index" ON "documents" ("original_sha256_hash");
CREATE UNIQUE INDEX "intake_emails_email_address_unique" ON "intake_emails" ("email_address");
CREATE INDEX "key_hash_index" ON "api_keys" ("key_hash");
CREATE INDEX migrations_name_index ON migrations (name);
CREATE INDEX migrations_run_at_index ON migrations (run_at);
CREATE UNIQUE INDEX "organization_invitations_organization_email_unique" ON "organization_invitations" ("organization_id","email");
CREATE UNIQUE INDEX "organization_members_user_organization_unique" ON "organization_members" ("organization_id","user_id");
CREATE UNIQUE INDEX "tags_organization_id_name_unique" ON "tags" ("organization_id","name");
CREATE INDEX "user_roles_role_index" ON "user_roles" ("role");
CREATE UNIQUE INDEX "user_roles_user_id_role_unique_index" ON "user_roles" ("user_id","role");
CREATE INDEX "users_email_index" ON "users" ("email");
CREATE UNIQUE INDEX "users_email_unique" ON "users" ("email");
CREATE UNIQUE INDEX "webhook_events_webhook_id_event_name_unique" ON "webhook_events" ("webhook_id","event_name");
CREATE TABLE "api_key_organizations" ( "api_key_id" text NOT NULL, "organization_member_id" text NOT NULL, FOREIGN KEY ("api_key_id") REFERENCES "api_keys"("id") ON UPDATE cascade ON DELETE cascade, FOREIGN KEY ("organization_member_id") REFERENCES "organization_members"("id") ON UPDATE cascade ON DELETE cascade );
CREATE TABLE "api_keys" ( "id" text PRIMARY KEY NOT NULL, "created_at" integer NOT NULL, "updated_at" integer NOT NULL, "name" text NOT NULL, "key_hash" text NOT NULL, "prefix" text NOT NULL, "user_id" text NOT NULL, "last_used_at" integer, "expires_at" integer, "permissions" text DEFAULT '[]' NOT NULL, "all_organizations" integer DEFAULT false NOT NULL, FOREIGN KEY ("user_id") REFERENCES "users"("id") ON UPDATE cascade ON DELETE cascade );
CREATE TABLE "auth_accounts" ( "id" text PRIMARY KEY NOT NULL, "created_at" integer NOT NULL, "updated_at" integer NOT NULL, "user_id" text, "account_id" text NOT NULL, "provider_id" text NOT NULL, "access_token" text, "refresh_token" text, "access_token_expires_at" integer, "refresh_token_expires_at" integer, "scope" text, "id_token" text, "password" text, FOREIGN KEY ("user_id") REFERENCES "users"("id") ON UPDATE cascade ON DELETE cascade );
CREATE TABLE "auth_sessions" ( "id" text PRIMARY KEY NOT NULL, "created_at" integer NOT NULL, "updated_at" integer NOT NULL, "token" text NOT NULL, "user_id" text, "expires_at" integer NOT NULL, "ip_address" text, "user_agent" text, "active_organization_id" text, FOREIGN KEY ("user_id") REFERENCES "users"("id") ON UPDATE cascade ON DELETE cascade, FOREIGN KEY ("active_organization_id") REFERENCES "organizations"("id") ON UPDATE cascade ON DELETE set null );
CREATE TABLE "auth_verifications" ( "id" text PRIMARY KEY NOT NULL, "created_at" integer NOT NULL, "updated_at" integer NOT NULL, "identifier" text NOT NULL, "value" text NOT NULL, "expires_at" integer NOT NULL );
CREATE TABLE "document_activity_log" ( "id" text PRIMARY KEY NOT NULL, "created_at" integer NOT NULL, "document_id" text NOT NULL, "event" text NOT NULL, "event_data" text, "user_id" text, "tag_id" text, FOREIGN KEY ("document_id") REFERENCES "documents"("id") ON UPDATE cascade ON DELETE cascade, FOREIGN KEY ("user_id") REFERENCES "users"("id") ON UPDATE cascade ON DELETE set null, FOREIGN KEY ("tag_id") REFERENCES "tags"("id") ON UPDATE cascade ON DELETE set null );
CREATE TABLE "documents" ( "id" text PRIMARY KEY NOT NULL, "created_at" integer NOT NULL, "updated_at" integer NOT NULL, "is_deleted" integer DEFAULT false NOT NULL, "deleted_at" integer, "organization_id" text NOT NULL, "created_by" text, "deleted_by" text, "original_name" text NOT NULL, "original_size" integer DEFAULT 0 NOT NULL, "original_storage_key" text NOT NULL, "original_sha256_hash" text NOT NULL, "name" text NOT NULL, "mime_type" text NOT NULL, "content" text DEFAULT '' NOT NULL, FOREIGN KEY ("organization_id") REFERENCES "organizations"("id") ON UPDATE cascade ON DELETE cascade, FOREIGN KEY ("created_by") REFERENCES "users"("id") ON UPDATE cascade ON DELETE set null, FOREIGN KEY ("deleted_by") REFERENCES "users"("id") ON UPDATE cascade ON DELETE set null );
CREATE VIRTUAL TABLE documents_fts USING fts5(id UNINDEXED, name, original_name, content, prefix='2 3 4');
CREATE TABLE 'documents_fts_config'(k PRIMARY KEY, v) WITHOUT ROWID;
CREATE TABLE 'documents_fts_content'(id INTEGER PRIMARY KEY, c0, c1, c2, c3);
CREATE TABLE 'documents_fts_data'(id INTEGER PRIMARY KEY, block BLOB);
CREATE TABLE 'documents_fts_docsize'(id INTEGER PRIMARY KEY, sz BLOB);
CREATE TABLE 'documents_fts_idx'(segid, term, pgno, PRIMARY KEY(segid, term)) WITHOUT ROWID;
CREATE TABLE "documents_tags" ( "document_id" text NOT NULL, "tag_id" text NOT NULL, PRIMARY KEY("document_id", "tag_id"), FOREIGN KEY ("document_id") REFERENCES "documents"("id") ON UPDATE cascade ON DELETE cascade, FOREIGN KEY ("tag_id") REFERENCES "tags"("id") ON UPDATE cascade ON DELETE cascade );
CREATE TABLE "intake_emails" ( "id" text PRIMARY KEY NOT NULL, "created_at" integer NOT NULL, "updated_at" integer NOT NULL, "email_address" text NOT NULL, "organization_id" text NOT NULL, "allowed_origins" text DEFAULT '[]' NOT NULL, "is_enabled" integer DEFAULT true NOT NULL, FOREIGN KEY ("organization_id") REFERENCES "organizations"("id") ON UPDATE cascade ON DELETE cascade );
CREATE TABLE migrations (id INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT NOT NULL, run_at INTEGER NOT NULL);
CREATE TABLE "organization_invitations" ( "id" text PRIMARY KEY NOT NULL, "created_at" integer NOT NULL, "updated_at" integer NOT NULL, "organization_id" text NOT NULL, "email" text NOT NULL, "role" text NOT NULL, "status" text NOT NULL DEFAULT 'pending', "expires_at" integer NOT NULL, "inviter_id" text NOT NULL, FOREIGN KEY ("organization_id") REFERENCES "organizations"("id") ON UPDATE cascade ON DELETE cascade, FOREIGN KEY ("inviter_id") REFERENCES "users"("id") ON UPDATE cascade ON DELETE cascade );
CREATE TABLE "organization_members" ( "id" text PRIMARY KEY NOT NULL, "created_at" integer NOT NULL, "updated_at" integer NOT NULL, "organization_id" text NOT NULL, "user_id" text NOT NULL, "role" text NOT NULL, FOREIGN KEY ("organization_id") REFERENCES "organizations"("id") ON UPDATE cascade ON DELETE cascade, FOREIGN KEY ("user_id") REFERENCES "users"("id") ON UPDATE cascade ON DELETE cascade );
CREATE TABLE "organization_subscriptions" ( "id" text PRIMARY KEY NOT NULL, "customer_id" text NOT NULL, "organization_id" text NOT NULL, "plan_id" text NOT NULL, "status" text NOT NULL, "seats_count" integer NOT NULL, "current_period_end" integer NOT NULL, "current_period_start" integer NOT NULL, "cancel_at_period_end" integer DEFAULT false NOT NULL, "created_at" integer NOT NULL, "updated_at" integer NOT NULL, FOREIGN KEY ("organization_id") REFERENCES "organizations"("id") ON UPDATE cascade ON DELETE cascade );
CREATE TABLE "organizations" ( "id" text PRIMARY KEY NOT NULL, "created_at" integer NOT NULL, "updated_at" integer NOT NULL, "name" text NOT NULL, "customer_id" text );
CREATE TABLE sqlite_sequence(name,seq);
CREATE TABLE "tagging_rule_actions" ( "id" text PRIMARY KEY NOT NULL, "created_at" integer NOT NULL, "updated_at" integer NOT NULL, "tagging_rule_id" text NOT NULL, "tag_id" text NOT NULL, FOREIGN KEY ("tagging_rule_id") REFERENCES "tagging_rules"("id") ON UPDATE cascade ON DELETE cascade, FOREIGN KEY ("tag_id") REFERENCES "tags"("id") ON UPDATE cascade ON DELETE cascade );
CREATE TABLE "tagging_rule_conditions" ( "id" text PRIMARY KEY NOT NULL, "created_at" integer NOT NULL, "updated_at" integer NOT NULL, "tagging_rule_id" text NOT NULL, "field" text NOT NULL, "operator" text NOT NULL, "value" text NOT NULL, "is_case_sensitive" integer DEFAULT false NOT NULL, FOREIGN KEY ("tagging_rule_id") REFERENCES "tagging_rules"("id") ON UPDATE cascade ON DELETE cascade );
CREATE TABLE "tagging_rules" ( "id" text PRIMARY KEY NOT NULL, "created_at" integer NOT NULL, "updated_at" integer NOT NULL, "organization_id" text NOT NULL, "name" text NOT NULL, "description" text, "enabled" integer DEFAULT true NOT NULL, FOREIGN KEY ("organization_id") REFERENCES "organizations"("id") ON UPDATE cascade ON DELETE cascade );
CREATE TABLE "tags" ( "id" text PRIMARY KEY NOT NULL, "created_at" integer NOT NULL, "updated_at" integer NOT NULL, "organization_id" text NOT NULL, "name" text NOT NULL, "color" text NOT NULL, "description" text, FOREIGN KEY ("organization_id") REFERENCES "organizations"("id") ON UPDATE cascade ON DELETE cascade );
CREATE TABLE "user_roles" ( "id" text PRIMARY KEY NOT NULL, "created_at" integer NOT NULL, "updated_at" integer NOT NULL, "user_id" text NOT NULL, "role" text NOT NULL, FOREIGN KEY ("user_id") REFERENCES "users"("id") ON UPDATE cascade ON DELETE cascade );
CREATE TABLE "users" ( "id" text PRIMARY KEY NOT NULL, "created_at" integer NOT NULL, "updated_at" integer NOT NULL, "email" text NOT NULL, "email_verified" integer DEFAULT false NOT NULL, "name" text, "image" text, "max_organization_count" integer );
CREATE TABLE "webhook_deliveries" ( "id" text PRIMARY KEY NOT NULL, "created_at" integer NOT NULL, "updated_at" integer NOT NULL, "webhook_id" text NOT NULL, "event_name" text NOT NULL, "request_payload" text NOT NULL, "response_payload" text NOT NULL, "response_status" integer NOT NULL, FOREIGN KEY ("webhook_id") REFERENCES "webhooks"("id") ON UPDATE cascade ON DELETE cascade );
CREATE TABLE "webhook_events" ( "id" text PRIMARY KEY NOT NULL, "created_at" integer NOT NULL, "updated_at" integer NOT NULL, "webhook_id" text NOT NULL, "event_name" text NOT NULL, FOREIGN KEY ("webhook_id") REFERENCES "webhooks"("id") ON UPDATE cascade ON DELETE cascade );
CREATE TABLE "webhooks" ( "id" text PRIMARY KEY NOT NULL, "created_at" integer NOT NULL, "updated_at" integer NOT NULL, "name" text NOT NULL, "url" text NOT NULL, "secret" text, "enabled" integer DEFAULT true NOT NULL, "created_by" text, "organization_id" text, FOREIGN KEY ("created_by") REFERENCES "users"("id") ON UPDATE cascade ON DELETE set null, FOREIGN KEY ("organization_id") REFERENCES "organizations"("id") ON UPDATE cascade ON DELETE cascade );
CREATE TRIGGER trigger_documents_fts_delete AFTER DELETE ON documents BEGIN DELETE FROM documents_fts WHERE id = old.id; END;
CREATE TRIGGER trigger_documents_fts_insert AFTER INSERT ON documents BEGIN INSERT INTO documents_fts(id, name, original_name, content) VALUES (new.id, new.name, new.original_name, new.content); END;
CREATE TRIGGER trigger_documents_fts_update AFTER UPDATE ON documents BEGIN UPDATE documents_fts SET name = new.name, original_name = new.original_name, content = new.content WHERE id = new.id; END;"
`);
});
// Maybe a bit fragile, but it's to try to enforce to have migrations fail-safe
test('if for some reasons we drop the migrations table, we can reapply all migrations', async () => {
const { db } = setupDatabase({ url: ':memory:' });
await runMigrations({ db, migrations });
const dbState = await serializeSchema({ db });
await db.run(sql`DROP TABLE migrations`);
await runMigrations({ db, migrations });
expect(await serializeSchema({ db })).to.eq(dbState);
});
});
});

View File

@@ -0,0 +1,23 @@
import type { Migration } from './migrations.types';
import { initialSchemaSetupMigration } from './list/0001-initial-schema-setup.migration';
import { documentsFtsMigration } from './list/0002-documents-fts.migration';
import { taggingRulesMigration } from './list/0003-tagging-rules.migration';
import { apiKeysMigration } from './list/0004-api-keys.migration';
import { organizationsWebhooksMigration } from './list/0005-organizations-webhooks.migration';
import { organizationsInvitationsImprovementMigration } from './list/0006-organizations-invitations-improvement.migration';
import { documentActivityLogMigration } from './list/0007-document-activity-log.migration';
import { documentActivityLogOnDeleteSetNullMigration } from './list/0008-document-activity-log-on-delete-set-null.migration';
import { dropLegacyMigrationsMigration } from './list/0009-drop-legacy-migrations.migration';
export const migrations: Migration[] = [
initialSchemaSetupMigration,
documentsFtsMigration,
taggingRulesMigration,
apiKeysMigration,
organizationsWebhooksMigration,
organizationsInvitationsImprovementMigration,
documentActivityLogMigration,
documentActivityLogOnDeleteSetNullMigration,
dropLegacyMigrationsMigration,
];

View File

@@ -0,0 +1,29 @@
import type { Database } from '../modules/app/database/database.types';
import { asc, eq, sql } from 'drizzle-orm';
import { migrationsTable } from './migration.tables';
export async function setupMigrationTableIfNotExists({ db }: { db: Database }) {
await db.batch([
db.run(sql`CREATE TABLE IF NOT EXISTS migrations (id INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT NOT NULL, run_at INTEGER NOT NULL)`),
db.run(sql`CREATE INDEX IF NOT EXISTS migrations_name_index ON migrations (name)`),
db.run(sql`CREATE INDEX IF NOT EXISTS migrations_run_at_index ON migrations (run_at)`),
]);
}
export async function getMigrations({ db }: { db: Database }) {
const migrations = await db.select().from(migrationsTable).orderBy(asc(migrationsTable.runAt));
return { migrations };
}
export async function saveMigration({ db, migrationName, now = new Date() }: { db: Database; migrationName: string; now?: Date }) {
await db.insert(migrationsTable).values({ name: migrationName, runAt: now });
}
export async function deleteMigration({ db, migrationName }: { db: Database; migrationName: string }) {
await db.delete(migrationsTable).where(eq(migrationsTable.name, migrationName));
}
export async function deleteAllMigrations({ db }: { db: Database }) {
await db.delete(migrationsTable);
}

View File

@@ -0,0 +1,20 @@
import type { Database } from '../modules/app/database/database.types';
export type MigrationArguments = {
db: Database;
};
export type Migration = {
/**
* The name of the migration. Must be unique.
*/
name: string;
/**
* Optional description of the migration, serves to add more context to the migration for humans.
*/
description?: string;
up: (args: MigrationArguments) => Promise<unknown>;
down?: (args: MigrationArguments) => Promise<unknown>;
};

View File

@@ -0,0 +1,141 @@
import type { Migration } from './migrations.types';
import { sql } from 'drizzle-orm';
import { describe, expect, test } from 'vitest';
import { setupDatabase } from '../modules/app/database/database';
import { migrationsTable } from './migration.tables';
import { rollbackLastAppliedMigration, runMigrations } from './migrations.usecases';
const createTableUserMigration: Migration = {
name: 'create-table-user',
up: async ({ db }) => {
await db.run(sql`CREATE TABLE users (id INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT NOT NULL)`);
},
down: async ({ db }) => {
await db.run(sql`DROP TABLE users`);
},
};
const createTableOrganizationMigration: Migration = {
name: 'create-table-organization',
up: async ({ db }) => {
await db.batch([
db.run(sql`CREATE TABLE organizations (id INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT NOT NULL)`),
db.run(sql`CREATE TABLE organization_members (id INTEGER PRIMARY KEY AUTOINCREMENT, organization_id INTEGER NOT NULL, user_id INTEGER NOT NULL, role TEXT NOT NULL, created_at INTEGER NOT NULL)`),
]);
},
down: async ({ db }) => {
await db.batch([
db.run(sql`DROP TABLE organizations`),
db.run(sql`DROP TABLE organization_members`),
]);
},
};
const createTableDocumentMigration: Migration = {
name: 'create-table-document',
up: async ({ db }) => {
await db.batch([
db.run(sql`CREATE TABLE documents (id INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT NOT NULL, created_at INTEGER NOT NULL)`),
]);
},
down: async ({ db }) => {
await db.run(sql`DROP TABLE documents`);
},
};
describe('migrations usecases', () => {
describe('runMigrations', () => {
test('should run all migrations that are not already applied', async () => {
const { db } = setupDatabase({ url: ':memory:' });
const migrations = [createTableUserMigration, createTableOrganizationMigration];
await runMigrations({ db, migrations });
const migrationsInDb = await db.select().from(migrationsTable);
expect(migrationsInDb.map(({ id, name }) => ({ id, name }))).to.eql([
{ id: 1, name: 'create-table-user' },
{ id: 2, name: 'create-table-organization' },
]);
migrations.push(createTableDocumentMigration);
await runMigrations({ db, migrations });
const migrationsInDb2 = await db.select().from(migrationsTable);
expect(migrationsInDb2.map(({ id, name }) => ({ id, name }))).to.eql([
{ id: 1, name: 'create-table-user' },
{ id: 2, name: 'create-table-organization' },
{ id: 3, name: 'create-table-document' },
]);
const { rows: tables } = await db.run(sql`SELECT name FROM sqlite_master WHERE name NOT LIKE 'sqlite_%'`);
// Ensure all tables and indexes are created
expect(tables.map(t => t.name)).to.eql([
'migrations',
'migrations_name_index',
'migrations_run_at_index',
'users',
'organizations',
'organization_members',
'documents',
]);
});
});
describe('rollbackLastAppliedMigration', () => {
test('the last migration down is called', async () => {
const { db } = setupDatabase({ url: ':memory:' });
const migrations = [createTableUserMigration, createTableDocumentMigration];
await runMigrations({ db, migrations });
const initialMigrations = await db.select().from(migrationsTable);
expect(initialMigrations.map(({ id, name }) => ({ id, name }))).to.eql([
{ id: 1, name: 'create-table-user' },
{ id: 2, name: 'create-table-document' },
]);
// Ensure the tables exists, no error is thrown
await db.run(sql`SELECT * FROM users`);
await db.run(sql`SELECT * FROM documents`);
await rollbackLastAppliedMigration({ db, migrations });
const migrationsInDb = await db.select().from(migrationsTable);
expect(migrationsInDb.map(({ id, name }) => ({ id, name }))).to.eql([
{ id: 1, name: 'create-table-user' },
]);
// Ensure the table document is dropped
await db.run(sql`SELECT * FROM users`);
await expect(db.run(sql`SELECT * FROM documents`)).rejects.toThrow();
});
test('when their is no migration to rollback, nothing is done', async () => {
const { db } = setupDatabase({ url: ':memory:' });
await rollbackLastAppliedMigration({ db });
const migrationsInDb = await db.select().from(migrationsTable);
expect(migrationsInDb).to.eql([]);
});
test('when the last migration in the database does not exist in the migrations list, an error is thrown', async () => {
const { db } = setupDatabase({ url: ':memory:' });
await runMigrations({ db, migrations: [createTableUserMigration] });
await expect(
rollbackLastAppliedMigration({ db, migrations: [] }),
).rejects.toThrow('Migration create-table-user not found');
});
});
});

View File

@@ -0,0 +1,83 @@
import type { Database } from '../modules/app/database/database.types';
import type { Logger } from '../modules/shared/logger/logger';
import type { Migration } from './migrations.types';
import { safely } from '@corentinth/chisels';
import { createLogger } from '../modules/shared/logger/logger';
import { migrations as migrationsList } from './migrations.registry';
import { deleteMigration, getMigrations, saveMigration, setupMigrationTableIfNotExists } from './migrations.repository';
export async function runMigrations({ db, migrations = migrationsList, logger = createLogger({ namespace: 'migrations' }) }: { db: Database; migrations?: Migration[]; logger?: Logger }) {
await setupMigrationTableIfNotExists({ db });
if (migrations.length === 0) {
logger.info('No migrations to run, skipping');
return;
}
const { migrations: existingMigrations } = await getMigrations({ db });
const migrationsToRun = migrations.filter(migration => !existingMigrations.some(m => m.name === migration.name));
if (migrationsToRun.length === 0) {
logger.info('All migrations already applied');
return;
}
logger.debug({
migrations: migrations.map(m => m.name),
migrationsToRun: migrationsToRun.map(m => m.name),
existingMigrations: existingMigrations.map(m => m.name),
migrationsToRunCount: migrationsToRun.length,
existingMigrationsCount: existingMigrations.length,
}, 'Running migrations');
for (const migration of migrationsToRun) {
const [, error] = await safely(upMigration({ db, migration }));
if (error) {
logger.error({ error, migrationName: migration.name }, 'Failed to run migration');
throw error;
}
logger.info({ migrationName: migration.name }, 'Migration run successfully');
}
logger.info('All migrations run successfully');
}
async function upMigration({ db, migration }: { db: Database; migration: Migration }) {
const { name, up } = migration;
await up({ db });
await saveMigration({ db, migrationName: name });
}
export async function rollbackLastAppliedMigration({ db, migrations = migrationsList, logger = createLogger({ namespace: 'migrations' }) }: { db: Database; migrations?: Migration[]; logger?: Logger }) {
await setupMigrationTableIfNotExists({ db });
const { migrations: existingMigrations } = await getMigrations({ db });
const lastMigrationInDb = existingMigrations[existingMigrations.length - 1];
if (!lastMigrationInDb) {
logger.info('No migrations to rollback');
return;
}
const lastMigration = migrations.find(m => m.name === lastMigrationInDb.name);
if (!lastMigration) {
logger.error({ migrationName: lastMigrationInDb.name }, 'Migration in database not found in saved migrations');
throw new Error(`Migration ${lastMigrationInDb.name} not found`);
}
await downMigration({ db, migration: lastMigration });
logger.info({ migrationName: lastMigration.name }, 'Migration rolled back successfully');
}
async function downMigration({ db, migration }: { db: Database; migration: Migration }) {
const { name, down } = migration;
await down?.({ db });
await deleteMigration({ db, migrationName: name });
}

View File

@@ -66,7 +66,6 @@ describe('api-key e2e', () => {
originalSha256Hash: '9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08',
name: 'invoice.txt',
mimeType: 'text/plain',
content: 'test',
});
const fetchDocumentResponse = await app.request(`/api/organizations/org_222222222222222222222222/documents/${document.id}`, {

View File

@@ -107,6 +107,27 @@ export function getAuth({
deleteUser: { enabled: false },
},
plugins: [
// Would love to have this but it messes with the error handling in better-auth client
// {
// id: 'better-auth-error-adapter',
// onResponse: async (res) => {
// // Transform better auth error to our own error
// if (res.status < 400) {
// return { response: res };
// }
// const body = await res.clone().json();
// const code = get(body, 'code', 'unknown');
// throw createError({
// message: get(body, 'message', 'Unknown error'),
// code: `auth.${code.toLowerCase()}`,
// statusCode: res.status as ContentfulStatusCode,
// isInternal: res.status >= 500,
// });
// },
// },
...(config.auth.providers.customs.length > 0
? [genericOAuth({ config: config.auth.providers.customs })]
: []),

View File

@@ -1,19 +1,11 @@
import type { Config } from '../../config/config.types';
import type { Database } from './database.types';
import { dirname, join } from 'node:path';
import { migrate } from 'drizzle-orm/libsql/migrator';
import { dirname } from 'node:path';
import { ensureDirectoryExists } from '../../shared/fs/fs.services';
import { createLogger } from '../../shared/logger/logger';
import { fileUrlToPath, getRootDirPath } from '../../shared/path';
import { fileUrlToPath } from '../../shared/path';
const logger = createLogger({ namespace: 'database-services' });
export async function runMigrations({ db }: { db: Database }) {
const migrationsFolder = join(getRootDirPath(), 'migrations');
await migrate(db, { migrationsFolder });
}
export async function ensureLocalDatabaseDirectoryExists({ config }: { config: Config }) {
const { url } = config.database;

View File

@@ -0,0 +1,24 @@
import { sql } from 'drizzle-orm';
import { describe, expect, test } from 'vitest';
import { setupDatabase } from './database';
import { serializeSchema } from './database.test-utils';
describe('database-utils test', () => {
describe('serializeSchema', () => {
test('given a database with some tables, it should return the schema as a string, used for db state snapshot', async () => {
const { db } = setupDatabase({ url: ':memory:' });
await db.run(sql`CREATE TABLE test (id INTEGER PRIMARY KEY, name TEXT)`);
await db.run(sql`CREATE INDEX idx_test_name ON test (name)`);
await db.run(sql`CREATE VIEW test_view AS SELECT * FROM test`);
await db.run(sql`CREATE TRIGGER test_trigger AFTER INSERT ON test BEGIN SELECT 1; END`);
const schema = await serializeSchema({ db });
expect(schema).toMatchInlineSnapshot(`
"CREATE INDEX idx_test_name ON test (name);
CREATE TABLE test (id INTEGER PRIMARY KEY, name TEXT);
CREATE TRIGGER test_trigger AFTER INSERT ON test BEGIN SELECT 1; END;
CREATE VIEW test_view AS SELECT * FROM test;"
`);
});
});
});

View File

@@ -1,4 +1,6 @@
import type { Database } from './database.types';
import { sql } from 'drizzle-orm';
import { runMigrations } from '../../../migrations/migrations.usecases';
import { apiKeyOrganizationsTable, apiKeysTable } from '../../api-keys/api-keys.tables';
import { documentsTable } from '../../documents/documents.table';
import { intakeEmailsTable } from '../../intake-emails/intake-emails.tables';
@@ -9,7 +11,6 @@ import { documentsTagsTable, tagsTable } from '../../tags/tags.table';
import { usersTable } from '../../users/users.table';
import { webhookDeliveriesTable, webhookEventsTable, webhooksTable } from '../../webhooks/webhooks.tables';
import { setupDatabase } from './database';
import { runMigrations } from './database.services';
export { createInMemoryDatabase, seedDatabase };
@@ -60,3 +61,34 @@ async function seedDatabase({ db, ...seedRows }: { db: Database } & SeedTablesRo
),
);
}
/*
PRAGMA encoding;
PRAGMA page_size;
PRAGMA auto_vacuum;
PRAGMA journal_mode; -- WAL is persistent
PRAGMA user_version;
PRAGMA application_id;
*/
export async function serializeSchema({ db }: { db: Database }) {
const result = await db.batch([
// db.run(sql`PRAGMA encoding`),
// db.run(sql`PRAGMA page_size`),
// db.run(sql`PRAGMA auto_vacuum`),
// db.run(sql`PRAGMA journal_mode`),
// db.run(sql`PRAGMA user_version`),
// db.run(sql`PRAGMA application_id`),
db.run(sql`SELECT sql FROM sqlite_schema WHERE sql IS NOT NULL AND type IN ('table','index','view','trigger') ORDER BY type, name`),
]);
return Array
.from(result.values())
.flatMap(({ rows }) => rows.map(({ sql }) => minifyQuery(String(sql))))
.join('\n');
}
function minifyQuery(query: string) {
return `${query.replace(/\s+/g, ' ').trim().replace(/;$/, '')};`;
}

View File

@@ -6,6 +6,7 @@ import { parseConfig } from '../config/config';
import { createEmailsServices } from '../emails/emails.services';
import { createLoggerMiddleware } from '../shared/logger/logger.middleware';
import { createSubscriptionsServices } from '../subscriptions/subscriptions.services';
import { createTaskServices } from '../tasks/tasks.services';
import { createTrackingServices } from '../tracking/tracking.services';
import { createAuthEmailsServices } from './auth/auth.emails.services';
import { getAuth } from './auth/auth.services';
@@ -23,6 +24,7 @@ async function createGlobalDependencies(partialDeps: Partial<GlobalDependencies>
const trackingServices = createTrackingServices({ config });
const auth = partialDeps.auth ?? getAuth({ db, config, authEmailsServices: createAuthEmailsServices({ emailsServices }), trackingServices }).auth;
const subscriptionsServices = createSubscriptionsServices({ config });
const taskServices = partialDeps.taskServices ?? createTaskServices({ config });
return {
config,
@@ -31,6 +33,7 @@ async function createGlobalDependencies(partialDeps: Partial<GlobalDependencies>
emailsServices,
subscriptionsServices,
trackingServices,
taskServices,
};
}

View File

@@ -3,6 +3,7 @@ import type { ApiKey } from '../api-keys/api-keys.types';
import type { Config } from '../config/config.types';
import type { EmailsServices } from '../emails/emails.services';
import type { SubscriptionsServices } from '../subscriptions/subscriptions.services';
import type { TaskServices } from '../tasks/tasks.services';
import type { TrackingServices } from '../tracking/tracking.services';
import type { Auth } from './auth/auth.services';
import type { Session } from './auth/auth.types';
@@ -28,6 +29,7 @@ export type GlobalDependencies = {
emailsServices: EmailsServices;
subscriptionsServices: SubscriptionsServices;
trackingServices: TrackingServices;
taskServices: TaskServices;
};
export type RouteDefinitionContext = { app: ServerInstance } & GlobalDependencies;

View File

@@ -15,6 +15,6 @@ export const documentActivityLogTable = sqliteTable('document_activity_log', {
event: text('event', { enum: DOCUMENT_ACTIVITY_EVENT_LIST as NonEmptyArray<DocumentActivityEvent> }).notNull(),
eventData: text('event_data', { mode: 'json' }).$type<Record<string, unknown>>(),
userId: text('user_id').references(() => usersTable.id, { onDelete: 'no action', onUpdate: 'cascade' }),
tagId: text('tag_id').references(() => tagsTable.id, { onDelete: 'no action', onUpdate: 'cascade' }),
userId: text('user_id').references(() => usersTable.id, { onDelete: 'set null', onUpdate: 'cascade' }),
tagId: text('tag_id').references(() => tagsTable.id, { onDelete: 'set null', onUpdate: 'cascade' }),
});

View File

@@ -10,7 +10,7 @@ import { createError } from '../shared/errors/errors';
import { isNil } from '../shared/utils';
import { validateFormData, validateJsonBody, validateParams, validateQuery } from '../shared/validation/validation';
import { createWebhookRepository } from '../webhooks/webhook.repository';
import { triggerWebhooks } from '../webhooks/webhook.usecases';
import { deferTriggerWebhooks } from '../webhooks/webhook.usecases';
import { createDocumentActivityRepository } from './document-activity/document-activity.repository';
import { deferRegisterDocumentActivityLog } from './document-activity/document-activity.usecases';
import { createDocumentIsNotDeletedError } from './documents.errors';
@@ -35,7 +35,7 @@ export function registerDocumentsRoutes(context: RouteDefinitionContext) {
setupUpdateDocumentRoute(context);
}
function setupCreateDocumentRoute({ app, config, db, trackingServices }: RouteDefinitionContext) {
function setupCreateDocumentRoute({ app, config, db, trackingServices, taskServices }: RouteDefinitionContext) {
app.post(
'/api/organizations/:organizationId/documents',
requireAuthentication({ apiKeyPermissions: ['documents:create'] }),
@@ -93,6 +93,7 @@ function setupCreateDocumentRoute({ app, config, db, trackingServices }: RouteDe
const createDocument = await createDocumentCreationUsecase({
db,
config,
taskServices,
trackingServices,
ocrLanguages,
});
@@ -244,7 +245,7 @@ function setupDeleteDocumentRoute({ app, db }: RouteDefinitionContext) {
await documentsRepository.softDeleteDocument({ documentId, organizationId, userId });
await triggerWebhooks({
deferTriggerWebhooks({
webhookRepository,
organizationId,
event: 'document:deleted',
@@ -479,6 +480,7 @@ function setupUpdateDocumentRoute({ app, db }: RouteDefinitionContext) {
const documentsRepository = createDocumentsRepository({ db });
const organizationsRepository = createOrganizationsRepository({ db });
const documentActivityRepository = createDocumentActivityRepository({ db });
const webhookRepository = createWebhookRepository({ db });
await ensureUserIsInOrganization({ userId, organizationId, organizationsRepository });
await ensureDocumentExists({ documentId, organizationId, documentsRepository });
@@ -489,6 +491,13 @@ function setupUpdateDocumentRoute({ app, db }: RouteDefinitionContext) {
...updateData,
});
deferTriggerWebhooks({
webhookRepository,
organizationId,
event: 'document:updated',
payload: { documentId, organizationId, ...updateData },
});
deferRegisterDocumentActivityLog({
documentId,
event: 'updated',

View File

@@ -1,3 +1,7 @@
import type { Logger } from '@crowlog/logger';
import { extractTextFromFile } from '@papra/lecture';
import { createLogger } from '../shared/logger/logger';
export async function getFileSha256Hash({ file }: { file: File }) {
const arrayBuffer = await file.arrayBuffer();
const hash = await crypto.subtle.digest('SHA-256', arrayBuffer);
@@ -9,3 +13,23 @@ export async function getFileSha256Hash({ file }: { file: File }) {
hash: hashHex,
};
}
export async function extractDocumentText({
file,
ocrLanguages,
logger = createLogger({ namespace: 'documents:services' }),
}: {
file: File;
ocrLanguages?: string[];
logger?: Logger;
}) {
const { textContent, error, extractorName } = await extractTextFromFile({ file, config: { tesseract: { languages: ocrLanguages } } });
if (error) {
logger.error({ error, extractorName }, 'Error while extracting text from document');
}
return {
text: textContent ?? '',
};
}

View File

@@ -5,16 +5,18 @@ import { ORGANIZATION_ROLES } from '../organizations/organizations.constants';
import { nextTick } from '../shared/async/defer.test-utils';
import { collectReadableStreamToString } from '../shared/streams/readable-stream';
import { documentsTagsTable } from '../tags/tags.table';
import { createInMemoryTaskServices } from '../tasks/tasks.test-utils';
import { documentActivityLogTable } from './document-activity/document-activity.table';
import { createDocumentAlreadyExistsError } from './documents.errors';
import { createDocumentsRepository } from './documents.repository';
import { documentsTable } from './documents.table';
import { createDocumentCreationUsecase } from './documents.usecases';
import { createDocumentCreationUsecase, extractAndSaveDocumentFileContent } from './documents.usecases';
import { createDocumentStorageService } from './storage/documents.storage.services';
describe('documents usecases', () => {
describe('createDocument', () => {
test('creating a document save the file to the storage and registers a record in the db', async () => {
const taskServices = createInMemoryTaskServices();
const { db } = await createInMemoryDatabase({
users: [{ id: 'user-1', email: 'user-1@example.com' }],
organizations: [{ id: 'organization-1', name: 'Organization 1' }],
@@ -32,6 +34,7 @@ describe('documents usecases', () => {
config,
generateDocumentId: () => 'doc_1',
documentsStorageService,
taskServices,
});
const file = new File(['content'], 'file.txt', { type: 'text/plain' });
@@ -70,6 +73,7 @@ describe('documents usecases', () => {
});
test('in the same organization, we should not be able to have two documents with the same content, an error is raised if the document already exists', async () => {
const taskServices = createInMemoryTaskServices();
const { db } = await createInMemoryDatabase({
users: [{ id: 'user-1', email: 'user-1@example.com' }],
organizations: [{ id: 'organization-1', name: 'Organization 1' }],
@@ -89,6 +93,7 @@ describe('documents usecases', () => {
config,
generateDocumentId: () => `doc_${documentIdIndex++}`,
documentsStorageService,
taskServices,
});
const file = new File(['content'], 'file.txt', { type: 'text/plain' });
@@ -178,6 +183,8 @@ describe('documents usecases', () => {
],
});
const taskServices = createInMemoryTaskServices();
const config = overrideConfig({
organizationPlans: { isFreePlanUnlimited: true },
documentsStorage: { driver: 'in-memory' },
@@ -186,6 +193,7 @@ describe('documents usecases', () => {
const createDocument = await createDocumentCreationUsecase({
db,
config,
taskServices,
});
// 3. Re-create the document
@@ -219,6 +227,7 @@ describe('documents usecases', () => {
});
test('when there is an issue when inserting the document in the db, the file should not be saved in the storage', async () => {
const taskServices = createInMemoryTaskServices();
const { db } = await createInMemoryDatabase({
users: [{ id: 'user-1', email: 'user-1@example.com' }],
organizations: [{ id: 'organization-1', name: 'Organization 1' }],
@@ -243,6 +252,7 @@ describe('documents usecases', () => {
throw new Error('Macron, explosion!');
},
},
taskServices,
});
const file = new File(['content'], 'file.txt', { type: 'text/plain' });
@@ -267,6 +277,7 @@ describe('documents usecases', () => {
});
test('when a document is created by a user, a document activity log is registered with the user id', async () => {
const taskServices = createInMemoryTaskServices();
const { db } = await createInMemoryDatabase({
users: [{ id: 'user-1', email: 'user-1@example.com' }],
organizations: [{ id: 'organization-1', name: 'Organization 1' }],
@@ -283,6 +294,7 @@ describe('documents usecases', () => {
db,
config,
generateDocumentId: () => `doc_${documentIdIndex++}`,
taskServices,
});
await createDocument({
@@ -317,4 +329,53 @@ describe('documents usecases', () => {
});
});
});
describe('extractAndSaveDocumentFileContent', () => {
test('given a stored document, its content is extracted and saved in the db', async () => {
const { db } = await createInMemoryDatabase({
users: [{ id: 'user-1', email: 'user-1@example.com' }],
organizations: [{ id: 'organization-1', name: 'Organization 1' }],
organizationMembers: [{ organizationId: 'organization-1', userId: 'user-1', role: ORGANIZATION_ROLES.OWNER }],
});
const config = overrideConfig({
organizationPlans: { isFreePlanUnlimited: true },
documentsStorage: { driver: 'in-memory' },
});
const documentsRepository = createDocumentsRepository({ db });
const documentsStorageService = await createDocumentStorageService({ config });
await db.insert(documentsTable).values({
id: 'document-1',
organizationId: 'organization-1',
originalStorageKey: 'organization-1/originals/document-1.txt',
mimeType: 'text/plain',
name: 'file-1.txt',
originalName: 'file-1.txt',
originalSha256Hash: 'b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9',
});
await documentsStorageService.saveFile({
file: new File(['hello world'], 'file-1.txt', { type: 'text/plain' }),
storageKey: 'organization-1/originals/document-1.txt',
});
await extractAndSaveDocumentFileContent({
documentId: 'document-1',
organizationId: 'organization-1',
documentsRepository,
documentsStorageService,
});
const documentRecords = await db.select().from(documentsTable);
expect(documentRecords.length).to.eql(1);
expect(documentRecords[0]).to.deep.include({
id: 'document-1',
organizationId: 'organization-1',
content: 'hello world', // The content is extracted and saved in the db
});
});
});
});

View File

@@ -5,6 +5,7 @@ import type { Logger } from '../shared/logger/logger';
import type { SubscriptionsRepository } from '../subscriptions/subscriptions.repository';
import type { TaggingRulesRepository } from '../tagging-rules/tagging-rules.repository';
import type { TagsRepository } from '../tags/tags.repository';
import type { TaskServices } from '../tasks/tasks.services';
import type { TrackingServices } from '../tracking/tracking.services';
import type { WebhookRepository } from '../webhooks/webhook.repository';
import type { DocumentActivityRepository } from './document-activity/document-activity.repository';
@@ -12,11 +13,11 @@ import type { DocumentsRepository } from './documents.repository';
import type { Document } from './documents.types';
import type { DocumentStorageService } from './storage/documents.storage.services';
import { safely } from '@corentinth/chisels';
import { extractTextFromFile } from '@papra/lecture';
import pLimit from 'p-limit';
import { checkIfOrganizationCanCreateNewDocument } from '../organizations/organizations.usecases';
import { createPlansRepository } from '../plans/plans.repository';
import { createLogger } from '../shared/logger/logger';
import { collectStreamToFile } from '../shared/streams/stream.convertion';
import { isDefined } from '../shared/utils';
import { createSubscriptionsRepository } from '../subscriptions/subscriptions.repository';
import { createTaggingRulesRepository } from '../tagging-rules/tagging-rules.repository';
@@ -24,29 +25,15 @@ import { applyTaggingRules } from '../tagging-rules/tagging-rules.usecases';
import { createTagsRepository } from '../tags/tags.repository';
import { createTrackingServices } from '../tracking/tracking.services';
import { createWebhookRepository } from '../webhooks/webhook.repository';
import { triggerWebhooks } from '../webhooks/webhook.usecases';
import { deferTriggerWebhooks } from '../webhooks/webhook.usecases';
import { createDocumentActivityRepository } from './document-activity/document-activity.repository';
import { deferRegisterDocumentActivityLog } from './document-activity/document-activity.usecases';
import { createDocumentAlreadyExistsError, createDocumentNotDeletedError, createDocumentNotFoundError } from './documents.errors';
import { buildOriginalDocumentKey, generateDocumentId as generateDocumentIdImpl } from './documents.models';
import { createDocumentsRepository } from './documents.repository';
import { getFileSha256Hash } from './documents.services';
import { extractDocumentText, getFileSha256Hash } from './documents.services';
import { createDocumentStorageService } from './storage/documents.storage.services';
const logger = createLogger({ namespace: 'documents:usecases' });
export async function extractDocumentText({ file, ocrLanguages }: { file: File; ocrLanguages?: string[] }) {
const { textContent, error, extractorName } = await extractTextFromFile({ file, config: { tesseract: { languages: ocrLanguages } } });
if (error) {
logger.error({ error, extractorName }, 'Error while extracting text from document');
}
return {
text: textContent ?? '',
};
}
export async function createDocument({
file,
userId,
@@ -62,6 +49,7 @@ export async function createDocument({
tagsRepository,
webhookRepository,
documentActivityRepository,
taskServices,
logger = createLogger({ namespace: 'documents:usecases' }),
}: {
file: File;
@@ -78,6 +66,7 @@ export async function createDocument({
tagsRepository: TagsRepository;
webhookRepository: WebhookRepository;
documentActivityRepository: DocumentActivityRepository;
taskServices: TaskServices;
logger?: Logger;
}) {
const {
@@ -120,7 +109,6 @@ export async function createDocument({
documentsStorageService,
generateDocumentId,
trackingServices,
ocrLanguages,
logger,
});
@@ -133,7 +121,7 @@ export async function createDocument({
await applyTaggingRules({ document, taggingRulesRepository, tagsRepository });
await triggerWebhooks({
deferTriggerWebhooks({
webhookRepository,
organizationId,
event: 'document:created',
@@ -146,6 +134,11 @@ export async function createDocument({
},
});
await taskServices.scheduleJob({
taskName: 'extract-document-file-content',
data: { documentId: document.id, organizationId, ocrLanguages },
});
return { document };
}
@@ -155,9 +148,11 @@ export type DocumentUsecaseDependencies = Omit<Parameters<typeof createDocument>
export async function createDocumentCreationUsecase({
db,
config,
taskServices,
...initialDeps
}: {
db: Database;
taskServices: TaskServices;
config: Config;
} & Partial<DocumentUsecaseDependencies>) {
const deps = {
@@ -176,7 +171,7 @@ export async function createDocumentCreationUsecase({
logger: initialDeps.logger,
};
return async (args: { file: File; userId?: string; organizationId: string }) => createDocument({ ...args, ...deps });
return async (args: { file: File; userId?: string; organizationId: string }) => createDocument({ taskServices, ...args, ...deps });
}
async function handleExistingDocument({
@@ -222,7 +217,6 @@ async function createNewDocument({
documentsStorageService,
generateDocumentId,
trackingServices,
ocrLanguages,
logger,
}: {
file: File;
@@ -236,7 +230,6 @@ async function createNewDocument({
documentsStorageService: DocumentStorageService;
generateDocumentId: () => string;
trackingServices: TrackingServices;
ocrLanguages?: string[];
logger: Logger;
}) {
const documentId = generateDocumentId();
@@ -252,8 +245,6 @@ async function createNewDocument({
storageKey: originalDocumentStorageKey,
});
const { text } = await extractDocumentText({ file, ocrLanguages });
const [result, error] = await safely(documentsRepository.saveOrganizationDocument({
id: documentId,
name: fileName,
@@ -263,7 +254,6 @@ async function createNewDocument({
originalSize: size,
originalStorageKey: storageKey,
mimeType,
content: text,
originalSha256Hash: hash,
}));
@@ -412,3 +402,31 @@ export async function deleteAllTrashDocuments({
documents.map(async document => limit(async () => hardDeleteDocument({ document, documentsRepository, documentsStorageService }))),
);
}
export async function extractAndSaveDocumentFileContent({
documentId,
organizationId,
documentsRepository,
documentsStorageService,
ocrLanguages,
}: {
documentId: string;
ocrLanguages?: string[];
organizationId: string;
documentsRepository: DocumentsRepository;
documentsStorageService: DocumentStorageService;
}) {
const { document } = await documentsRepository.getDocumentById({ documentId, organizationId });
if (!document) {
throw createDocumentNotFoundError();
}
const { fileStream } = await documentsStorageService.getFileStream({ storageKey: document.originalStorageKey });
const { file } = await collectStreamToFile({ fileStream, fileName: document.name, mimeType: document.mimeType });
const { text } = await extractDocumentText({ file, ocrLanguages });
await documentsRepository.updateDocument({ documentId, organizationId, content: text });
}

View File

@@ -0,0 +1,29 @@
import type { Database } from '../../app/database/database.types';
import type { Config } from '../../config/config.types';
import type { TaskServices } from '../../tasks/tasks.services';
import { createDocumentsRepository } from '../documents.repository';
import { extractAndSaveDocumentFileContent } from '../documents.usecases';
import { createDocumentStorageService } from '../storage/documents.storage.services';
export async function registerExtractDocumentFileContentTask({ taskServices, db, config }: { taskServices: TaskServices; db: Database; config: Config }) {
const taskName = 'extract-document-file-content';
taskServices.registerTask({
taskName,
handler: async ({ data }) => {
const documentsRepository = createDocumentsRepository({ db });
const documentsStorageService = await createDocumentStorageService({ config });
// TODO: remove type cast
const { documentId, organizationId, ocrLanguages } = data as { documentId: string; organizationId: string; ocrLanguages: string[] };
await extractAndSaveDocumentFileContent({
documentId,
organizationId,
ocrLanguages,
documentsRepository,
documentsStorageService,
});
},
});
}

View File

@@ -1,24 +1,39 @@
import { defineTask } from '../../tasks/tasks.models';
import type { Database } from '../../app/database/database.types';
import type { Config } from '../../config/config.types';
import type { TaskServices } from '../../tasks/tasks.services';
import { createLogger } from '../../shared/logger/logger';
import { createDocumentsRepository } from '../documents.repository';
import { deleteExpiredDocuments } from '../documents.usecases';
import { createDocumentStorageService } from '../storage/documents.storage.services';
export const hardDeleteExpiredDocumentsTaskDefinition = defineTask({
name: 'hard-delete-expired-documents',
isEnabled: ({ config }) => config.tasks.hardDeleteExpiredDocuments.enabled,
cronSchedule: ({ config }) => config.tasks.hardDeleteExpiredDocuments.cron,
runOnStartup: ({ config }) => config.tasks.hardDeleteExpiredDocuments.runOnStartup,
handler: async ({ db, config, now, logger }) => {
const documentsRepository = createDocumentsRepository({ db });
const documentsStorageService = await createDocumentStorageService({ config });
const logger = createLogger({ namespace: 'documents:tasks:hardDeleteExpiredDocuments' });
const { deletedDocumentsCount } = await deleteExpiredDocuments({
config,
documentsRepository,
documentsStorageService,
now,
});
export async function registerHardDeleteExpiredDocumentsTask({ taskServices, db, config }: { taskServices: TaskServices; db: Database; config: Config }) {
const taskName = 'hard-delete-expired-documents';
const { cron, runOnStartup } = config.tasks.hardDeleteExpiredDocuments;
logger.info({ deletedDocumentsCount }, 'Expired documents deleted');
},
});
taskServices.registerTask({
taskName,
handler: async () => {
const documentsRepository = createDocumentsRepository({ db });
const documentsStorageService = await createDocumentStorageService({ config });
const { deletedDocumentsCount } = await deleteExpiredDocuments({
config,
documentsRepository,
documentsStorageService,
});
logger.info({ deletedDocumentsCount }, 'Expired documents deleted');
},
});
await taskServices.schedulePeriodicJob({
scheduleId: `periodic-${taskName}`,
taskName,
cron,
immediate: runOnStartup,
});
logger.info({ taskName, cron, runOnStartup }, 'Hard delete expired documents task registered');
}

View File

@@ -11,6 +11,7 @@ import { createOrganizationsRepository } from '../organizations/organizations.re
import { createInMemoryFsServices } from '../shared/fs/fs.in-memory';
import { createFsServices } from '../shared/fs/fs.services';
import { createTestLogger } from '../shared/logger/logger.test-utils';
import { createInMemoryTaskServices } from '../tasks/tasks.test-utils';
import { createInvalidPostProcessingStrategyError } from './ingestion-folders.errors';
import { moveIngestionFile, processFile } from './ingestion-folders.usecases';
@@ -18,6 +19,7 @@ describe('ingestion-folders usecases', () => {
describe('processFile', () => {
describe('when a file is added to an organization ingestion folder', () => {
test('if the post processing strategy is set to "move", the file is ingested and moved to the done folder', async () => {
const taskServices = createInMemoryTaskServices();
const { logger, getLogs } = createTestLogger();
const { db } = await createInMemoryDatabase({
@@ -53,7 +55,7 @@ describe('ingestion-folders usecases', () => {
organizationsRepository,
logger,
fs: createFsServices({ fs: vol.promises as unknown as FsNative }),
createDocument: await createDocumentCreationUsecase({ db, config, logger, documentsStorageService, generateDocumentId }),
createDocument: await createDocumentCreationUsecase({ db, config, logger, documentsStorageService, generateDocumentId, taskServices }),
});
// Check database
@@ -120,6 +122,7 @@ describe('ingestion-folders usecases', () => {
});
test('if the post processing strategy is set to "delete", the file is ingested and deleted', async () => {
const taskServices = createInMemoryTaskServices();
const { logger, getLogs } = createTestLogger();
const { db } = await createInMemoryDatabase({
@@ -154,7 +157,7 @@ describe('ingestion-folders usecases', () => {
organizationsRepository,
logger,
fs: createFsServices({ fs: vol.promises as unknown as FsNative }),
createDocument: await createDocumentCreationUsecase({ db, config, logger, documentsStorageService, generateDocumentId }),
createDocument: await createDocumentCreationUsecase({ db, config, logger, documentsStorageService, generateDocumentId, taskServices }),
});
// Check database
@@ -221,6 +224,7 @@ describe('ingestion-folders usecases', () => {
});
test('if the post processing strategy is not implemented, an error is thrown after the file has been ingested', async () => {
const taskServices = createInMemoryTaskServices();
const { logger } = createTestLogger();
const { db } = await createInMemoryDatabase({
@@ -256,7 +260,7 @@ describe('ingestion-folders usecases', () => {
organizationsRepository,
logger,
fs: createFsServices({ fs: vol.promises as unknown as FsNative }),
createDocument: await createDocumentCreationUsecase({ db, config, logger, documentsStorageService, generateDocumentId }),
createDocument: await createDocumentCreationUsecase({ db, config, logger, documentsStorageService, generateDocumentId, taskServices }),
}));
expect(error).to.deep.equal(createInvalidPostProcessingStrategyError({ strategy: 'unknown' }));
@@ -293,6 +297,7 @@ describe('ingestion-folders usecases', () => {
});
test('if for some reason the file cannot be read, a log is emitted and the processing stops', async () => {
const taskServices = createInMemoryTaskServices();
const { logger, getLogs } = createTestLogger();
const { db } = await createInMemoryDatabase({
@@ -333,7 +338,7 @@ describe('ingestion-folders usecases', () => {
throw new Error('File not found');
},
},
createDocument: await createDocumentCreationUsecase({ db, config, logger, documentsStorageService, generateDocumentId }),
createDocument: await createDocumentCreationUsecase({ db, config, logger, documentsStorageService, generateDocumentId, taskServices }),
});
// Check logs
@@ -433,6 +438,7 @@ describe('ingestion-folders usecases', () => {
});
test('when the document already exists in the database, it is not ingested, but the post-processing is still executed', async () => {
const taskServices = createInMemoryTaskServices();
const { logger, getLogs } = createTestLogger();
// This is the sha256 hash of the "lorem ipsum" text
@@ -472,7 +478,7 @@ describe('ingestion-folders usecases', () => {
organizationsRepository,
logger,
fs: createFsServices({ fs: vol.promises as unknown as FsNative }),
createDocument: await createDocumentCreationUsecase({ db, config, logger, documentsStorageService, generateDocumentId }),
createDocument: await createDocumentCreationUsecase({ db, config, logger, documentsStorageService, generateDocumentId, taskServices }),
});
// Check database

View File

@@ -5,6 +5,7 @@ import type { CreateDocumentUsecase } from '../documents/documents.usecases';
import type { OrganizationsRepository } from '../organizations/organizations.repository';
import type { FsServices } from '../shared/fs/fs.services';
import type { Logger } from '../shared/logger/logger';
import type { TaskServices } from '../tasks/tasks.services';
import { isAbsolute, join, parse } from 'node:path';
import { safely } from '@corentinth/chisels';
import chokidar from 'chokidar';
@@ -27,10 +28,12 @@ export function createIngestionFolderWatcher({
config,
logger = createLogger({ namespace: 'ingestion-folder-watcher' }),
db,
taskServices,
}: {
config: Config;
logger?: Logger;
db: Database;
taskServices: TaskServices;
}) {
const { folderRootPath, watcher: { usePolling, pollingInterval }, processingConcurrency } = config.ingestionFolder;
@@ -41,7 +44,7 @@ export function createIngestionFolderWatcher({
return {
startWatchingIngestionFolders: async () => {
const organizationsRepository = createOrganizationsRepository({ db });
const createDocument = await createDocumentCreationUsecase({ db, config, logger });
const createDocument = await createDocumentCreationUsecase({ db, config, logger, taskServices });
const ignored = await buildPathIgnoreFunction({ config, cwd, organizationsRepository });

View File

@@ -154,14 +154,13 @@ describe('intake-emails e2e', () => {
const [document] = documents;
expect(
pick(document, ['organizationId', 'createdBy', 'mimeType', 'originalName', 'originalSize', 'content']),
pick(document, ['organizationId', 'createdBy', 'mimeType', 'originalName', 'originalSize']),
).to.eql({
organizationId: 'org_1',
createdBy: null,
mimeType: 'text/plain',
originalName: 'test.txt',
originalSize: 11,
content: 'hello world',
});
});
});

View File

@@ -145,7 +145,7 @@ function setupUpdateIntakeEmailRoute({ app, db }: RouteDefinitionContext) {
);
}
function setupIngestIntakeEmailRoute({ app, db, config, trackingServices }: RouteDefinitionContext) {
function setupIngestIntakeEmailRoute({ app, db, config, trackingServices, taskServices }: RouteDefinitionContext) {
app.post(
INTAKE_EMAILS_INGEST_ROUTE,
validateFormData(z.object({
@@ -193,6 +193,7 @@ function setupIngestIntakeEmailRoute({ app, db, config, trackingServices }: Rout
db,
config,
trackingServices,
taskServices,
});
await processIntakeEmailIngestion({

View File

@@ -10,6 +10,7 @@ import { createDocumentCreationUsecase } from '../documents/documents.usecases';
import { PLUS_PLAN_ID } from '../plans/plans.constants';
import { createLogger } from '../shared/logger/logger';
import { createSubscriptionsRepository } from '../subscriptions/subscriptions.repository';
import { createInMemoryTaskServices } from '../tasks/tasks.test-utils';
import { createIntakeEmailLimitReachedError } from './intake-emails.errors';
import { createIntakeEmailsRepository } from './intake-emails.repository';
import { intakeEmailsTable } from './intake-emails.tables';
@@ -19,6 +20,7 @@ describe('intake-emails usecases', () => {
describe('ingestEmailForRecipient', () => {
describe('when a email is forwarded to papra api, we look for the recipient in the intake emails repository and create a papra document for each attachment', () => {
test(`when an intake email is is configured, enabled and match the recipient, and the sender is allowed, a document is created for each attachment`, async () => {
const taskServices = createInMemoryTaskServices();
const { db } = await createInMemoryDatabase({
organizations: [{ id: 'org-1', name: 'Organization 1' }],
intakeEmails: [{ id: 'ie-1', organizationId: 'org-1', allowedOrigins: ['foo@example.fr'], emailAddress: 'email-1@papra.email' }],
@@ -28,6 +30,7 @@ describe('intake-emails usecases', () => {
const createDocument = await createDocumentCreationUsecase({
db,
taskServices,
config: overrideConfig({
documentsStorage: { driver: 'in-memory' },
organizationPlans: { isFreePlanUnlimited: true },
@@ -48,10 +51,10 @@ describe('intake-emails usecases', () => {
const documents = await db.select().from(documentsTable).orderBy(asc(documentsTable.name));
expect(
documents.map(doc => pick(doc, ['organizationId', 'name', 'mimeType', 'originalName', 'content'])),
documents.map(doc => pick(doc, ['organizationId', 'name', 'mimeType', 'originalName'])),
).to.eql([
{ organizationId: 'org-1', name: 'file1.txt', mimeType: 'text/plain', originalName: 'file1.txt', content: 'content1' },
{ organizationId: 'org-1', name: 'file2.txt', mimeType: 'text/plain', originalName: 'file2.txt', content: 'content2' },
{ organizationId: 'org-1', name: 'file1.txt', mimeType: 'text/plain', originalName: 'file1.txt' },
{ organizationId: 'org-1', name: 'file2.txt', mimeType: 'text/plain', originalName: 'file2.txt' },
]);
});
@@ -59,6 +62,7 @@ describe('intake-emails usecases', () => {
const loggerTransport = createInMemoryLoggerTransport();
const logger = createLogger({ transports: [loggerTransport], namespace: 'test' });
const taskServices = createInMemoryTaskServices();
const { db } = await createInMemoryDatabase({
organizations: [{ id: 'org-1', name: 'Organization 1' }],
intakeEmails: [{ id: 'ie-1', organizationId: 'org-1', isEnabled: false, emailAddress: 'email-1@papra.email' }],
@@ -68,6 +72,7 @@ describe('intake-emails usecases', () => {
const createDocument = await createDocumentCreationUsecase({
db,
taskServices,
config: overrideConfig({
documentsStorage: { driver: 'in-memory' },
organizationPlans: { isFreePlanUnlimited: true },
@@ -90,6 +95,7 @@ describe('intake-emails usecases', () => {
});
test('when no intake email is found for the recipient, nothing happens, only a log is emitted', async () => {
const taskServices = createInMemoryTaskServices();
const loggerTransport = createInMemoryLoggerTransport();
const logger = createLogger({ transports: [loggerTransport], namespace: 'test' });
@@ -99,6 +105,7 @@ describe('intake-emails usecases', () => {
const createDocument = await createDocumentCreationUsecase({
db,
taskServices,
config: overrideConfig({
documentsStorage: { driver: 'in-memory' },
organizationPlans: { isFreePlanUnlimited: true },
@@ -123,6 +130,7 @@ describe('intake-emails usecases', () => {
test(`in order to be processed, the emitter of the email must be allowed for the intake email
it should be registered in the intake email allowed origins
if not, an error is logged and no document is created`, async () => {
const taskServices = createInMemoryTaskServices();
const loggerTransport = createInMemoryLoggerTransport();
const logger = createLogger({ transports: [loggerTransport], namespace: 'test' });
@@ -135,6 +143,7 @@ describe('intake-emails usecases', () => {
const createDocument = await createDocumentCreationUsecase({
db,
taskServices,
config: overrideConfig({
documentsStorage: { driver: 'in-memory' },
organizationPlans: { isFreePlanUnlimited: true },
@@ -167,6 +176,7 @@ describe('intake-emails usecases', () => {
describe('processIntakeEmailIngestion', () => {
test(`when an email is send to multiple intake emails from different organization, the attachments are processed for each of them`, async () => {
const taskServices = createInMemoryTaskServices();
const { db } = await createInMemoryDatabase({
organizations: [
{ id: 'org-1', name: 'Organization 1' },
@@ -182,6 +192,7 @@ describe('intake-emails usecases', () => {
const createDocument = await createDocumentCreationUsecase({
db,
taskServices,
config: overrideConfig({
documentsStorage: { driver: 'in-memory' },
organizationPlans: { isFreePlanUnlimited: true },
@@ -201,10 +212,10 @@ describe('intake-emails usecases', () => {
const documents = await db.select().from(documentsTable).orderBy(asc(documentsTable.organizationId));
expect(
documents.map(doc => pick(doc, ['organizationId', 'name', 'mimeType', 'originalName', 'content'])),
documents.map(doc => pick(doc, ['organizationId', 'name', 'mimeType', 'originalName'])),
).to.eql([
{ organizationId: 'org-1', name: 'file1.txt', mimeType: 'text/plain', originalName: 'file1.txt', content: 'content1' },
{ organizationId: 'org-2', name: 'file1.txt', mimeType: 'text/plain', originalName: 'file1.txt', content: 'content1' },
{ organizationId: 'org-1', name: 'file1.txt', mimeType: 'text/plain', originalName: 'file1.txt' },
{ organizationId: 'org-2', name: 'file1.txt', mimeType: 'text/plain', originalName: 'file1.txt' },
]);
});
});

View File

@@ -1,14 +1,32 @@
import { defineTask } from '../../tasks/tasks.models';
import type { Database } from '../../app/database/database.types';
import type { Config } from '../../config/config.types';
import type { TaskServices } from '../../tasks/tasks.services';
import { createLogger } from '../../shared/logger/logger';
import { createOrganizationsRepository } from '../organizations.repository';
export const expireInvitationsTaskDefinition = defineTask({
name: 'expire-invitations',
isEnabled: ({ config }) => config.tasks.expireInvitations.enabled,
cronSchedule: ({ config }) => config.tasks.expireInvitations.cron,
runOnStartup: ({ config }) => config.tasks.expireInvitations.runOnStartup,
handler: async ({ db, now }) => {
const organizationsRepository = createOrganizationsRepository({ db });
const logger = createLogger({ namespace: 'organizations:tasks:expireInvitations' });
await organizationsRepository.updateExpiredPendingInvitationsStatus({ now });
},
});
export async function registerExpireInvitationsTask({ taskServices, db, config }: { taskServices: TaskServices; db: Database; config: Config }) {
const taskName = 'expire-invitations';
const { cron, runOnStartup } = config.tasks.expireInvitations;
taskServices.registerTask({
taskName,
handler: async () => {
const organizationsRepository = createOrganizationsRepository({ db });
await organizationsRepository.updateExpiredPendingInvitationsStatus();
logger.info('Updated expired pending invitations status');
},
});
await taskServices.schedulePeriodicJob({
scheduleId: `periodic-${taskName}`,
taskName,
cron,
immediate: runOnStartup,
});
logger.info({ taskName, cron, runOnStartup }, 'Update expired pending invitations status task registered');
}

View File

@@ -0,0 +1,8 @@
export async function collectStreamToFile({ fileStream, fileName, mimeType }: { fileStream: ReadableStream; fileName: string; mimeType: string }): Promise<{ file: File }> {
const response = new Response(fileStream);
const blob = await response.blob();
const file = new File([blob], fileName, { type: mimeType });
return { file };
}

View File

@@ -11,3 +11,9 @@ export const createTagAlreadyExistsError = createErrorFactory({
code: 'tags.already_exists',
statusCode: 400,
});
export const createTagNotFoundError = createErrorFactory({
message: 'Tag not found',
code: 'tags.not_found',
statusCode: 404,
});

View File

@@ -15,6 +15,7 @@ export function createTagsRepository({ db }: { db: Database }) {
return injectArguments(
{
getOrganizationTags,
getTagById,
createTag,
deleteTag,
updateTag,
@@ -50,6 +51,20 @@ async function getOrganizationTags({ organizationId, db }: { organizationId: str
return { tags };
}
async function getTagById({ tagId, organizationId, db }: { tagId: string; organizationId: string; db: Database }) {
const [tag] = await db
.select()
.from(tagsTable)
.where(
and(
eq(tagsTable.id, tagId),
eq(tagsTable.organizationId, organizationId),
),
);
return { tag };
}
async function createTag({ tag, db }: { tag: DbInsertableTag; db: Database }) {
const [result, error] = await safely(db.insert(tagsTable).values(tag).returning());

View File

@@ -5,12 +5,17 @@ import { requireAuthentication } from '../app/auth/auth.middleware';
import { getUser } from '../app/auth/auth.models';
import { createDocumentActivityRepository } from '../documents/document-activity/document-activity.repository';
import { deferRegisterDocumentActivityLog } from '../documents/document-activity/document-activity.usecases';
import { createDocumentNotFoundError } from '../documents/documents.errors';
import { createDocumentsRepository } from '../documents/documents.repository';
import { documentIdSchema } from '../documents/documents.schemas';
import { organizationIdSchema } from '../organizations/organization.schemas';
import { createOrganizationsRepository } from '../organizations/organizations.repository';
import { ensureUserIsInOrganization } from '../organizations/organizations.usecases';
import { validateJsonBody, validateParams } from '../shared/validation/validation';
import { createWebhookRepository } from '../webhooks/webhook.repository';
import { deferTriggerWebhooks } from '../webhooks/webhook.usecases';
import { TagColorRegex } from './tags.constants';
import { createTagNotFoundError } from './tags.errors';
import { createTagsRepository } from './tags.repository';
import { tagIdSchema } from './tags.schemas';
@@ -161,12 +166,34 @@ function setupAddTagToDocumentRoute({ app, db }: RouteDefinitionContext) {
const tagsRepository = createTagsRepository({ db });
const organizationsRepository = createOrganizationsRepository({ db });
const webhookRepository = createWebhookRepository({ db });
const documentsRepository = createDocumentsRepository({ db });
const documentActivityRepository = createDocumentActivityRepository({ db });
await ensureUserIsInOrganization({ userId, organizationId, organizationsRepository });
const [{ document }, { tag }] = await Promise.all([
documentsRepository.getDocumentById({ organizationId, documentId }),
tagsRepository.getTagById({ tagId, organizationId }),
]);
if (!document) {
throw createDocumentNotFoundError();
}
if (!tag) {
throw createTagNotFoundError();
}
await tagsRepository.addTagToDocument({ tagId, documentId });
deferTriggerWebhooks({
webhookRepository,
organizationId,
event: 'document:tag:added',
payload: { documentId, organizationId, tagId, tagName: tag.name },
});
deferRegisterDocumentActivityLog({
documentId,
event: 'tagged',
@@ -197,12 +224,34 @@ function setupRemoveTagFromDocumentRoute({ app, db }: RouteDefinitionContext) {
const tagsRepository = createTagsRepository({ db });
const organizationsRepository = createOrganizationsRepository({ db });
const webhookRepository = createWebhookRepository({ db });
const documentActivityRepository = createDocumentActivityRepository({ db });
const documentsRepository = createDocumentsRepository({ db });
await ensureUserIsInOrganization({ userId, organizationId, organizationsRepository });
const [{ document }, { tag }] = await Promise.all([
documentsRepository.getDocumentById({ organizationId, documentId }),
tagsRepository.getTagById({ tagId, organizationId }),
]);
if (!document) {
throw createDocumentNotFoundError();
}
if (!tag) {
throw createTagNotFoundError();
}
await tagsRepository.removeTagFromDocument({ tagId, documentId });
deferTriggerWebhooks({
webhookRepository,
organizationId,
event: 'document:tag:removed',
payload: { documentId, organizationId, tagId, tagName: tag.name },
});
deferRegisterDocumentActivityLog({
documentId,
event: 'untagged',

View File

@@ -1,66 +0,0 @@
import type { Database } from '../app/database/database.types';
import type { Config } from '../config/config.types';
import type { TaskDefinition } from './tasks.models';
import cron from 'node-cron';
import { createLogger, wrapWithLoggerContext } from '../shared/logger/logger';
import { generateId } from '../shared/random/ids';
export { createTaskScheduler };
const logger = createLogger({ namespace: 'tasks:scheduler' });
function createTaskScheduler({
config,
taskDefinitions,
tasksArgs,
}: {
config: Config;
taskDefinitions: TaskDefinition[];
tasksArgs: { db: Database };
}) {
const scheduledTasks = taskDefinitions.map((taskDefinition) => {
const isEnabled = taskDefinition.getIsEnabled({ config });
const cronSchedule = taskDefinition.getCronSchedule({ config });
const runOnStartup = taskDefinition.getRunOnStartup({ config });
if (!isEnabled) {
return undefined;
}
const task = cron.schedule(
cronSchedule,
async () => wrapWithLoggerContext(
{
taskId: generateId({ prefix: 'task' }),
taskName: taskDefinition.taskName,
},
async () => taskDefinition.run({ ...tasksArgs, config }),
),
{
scheduled: false,
runOnInit: runOnStartup,
},
);
return { job: task, taskName: taskDefinition.taskName };
}).filter(Boolean);
return {
taskScheduler: {
scheduledTasks,
start() {
scheduledTasks.forEach(({ taskName, job }) => {
job.start();
logger.debug({ taskName }, 'Task scheduled');
});
},
stop() {
scheduledTasks.forEach(({ taskName, job }) => {
job.stop();
logger.debug({ taskName }, 'Task unscheduled');
});
},
},
};
}

View File

@@ -3,6 +3,21 @@ import { z } from 'zod';
import { booleanishSchema } from '../config/config.schemas';
export const tasksConfig = {
persistence: {
driver: {
doc: 'The driver to use for the tasks persistence',
schema: z.enum(['memory']),
default: 'memory',
env: 'TASKS_PERSISTENCE_DRIVER',
},
},
worker: {
id: {
doc: 'The id of the task worker, used to identify the worker in the Cadence cluster in case of multiple workers',
schema: z.string().optional(),
env: 'TASKS_WORKER_ID',
},
},
hardDeleteExpiredDocuments: {
enabled: {
doc: 'Whether the task to hard delete expired "soft deleted" documents is enabled',

View File

@@ -1,7 +0,0 @@
import { hardDeleteExpiredDocumentsTaskDefinition } from '../documents/tasks/hard-delete-expired-documents.task';
import { expireInvitationsTaskDefinition } from '../organizations/tasks/expire-invitations.task';
export const taskDefinitions = [
hardDeleteExpiredDocumentsTaskDefinition,
expireInvitationsTaskDefinition,
];

View File

@@ -0,0 +1,12 @@
import type { Database } from '../app/database/database.types';
import type { Config } from '../config/config.types';
import type { TaskServices } from './tasks.services';
import { registerExtractDocumentFileContentTask } from '../documents/tasks/extract-document-file-content.task';
import { registerHardDeleteExpiredDocumentsTask } from '../documents/tasks/hard-delete-expired-documents.task';
import { registerExpireInvitationsTask } from '../organizations/tasks/expire-invitations.task';
export async function registerTaskDefinitions({ taskServices, db, config }: { taskServices: TaskServices; db: Database; config: Config }) {
await registerHardDeleteExpiredDocumentsTask({ taskServices, db, config });
await registerExpireInvitationsTask({ taskServices, db, config });
await registerExtractDocumentFileContentTask({ taskServices, db, config });
}

View File

@@ -1,57 +0,0 @@
import type { Database } from '../app/database/database.types';
import type { Config } from '../config/config.types';
import type { Logger } from '../shared/logger/logger';
import { isFunction } from 'lodash-es';
import { createLogger } from '../shared/logger/logger';
export { defineTask };
export type TaskDefinition = ReturnType<typeof defineTask>;
function defineTask({
name: taskName,
cronSchedule,
isEnabled,
runOnStartup = false,
handler,
logger: taskLogger = createLogger({ namespace: `tasks:${taskName}` }),
}: {
name: string;
isEnabled: boolean | ((args: { config: Config }) => boolean);
cronSchedule: string | ((args: { config: Config }) => string);
runOnStartup?: boolean | ((args: { config: Config }) => boolean);
handler: (handlerArgs: { db: Database; config: Config; logger: Logger; now: Date }) => Promise<void>;
logger?: Logger;
}) {
const run = async ({
getNow = () => new Date(),
logger = taskLogger,
...handlerArgs
}: {
db: Database;
config: Config;
getNow?: () => Date;
logger?: Logger;
}) => {
const startedAt = getNow();
try {
logger.debug({ taskName, startedAt }, 'Task started');
await handler({ ...handlerArgs, logger, now: getNow() });
const durationMs = getNow().getTime() - startedAt.getTime();
logger.info({ taskName, durationMs, startedAt }, 'Task completed');
} catch (error) {
logger.error({ error, taskName, startedAt }, 'Task failed');
}
};
return {
taskName,
run,
getIsEnabled: (args: { config: Config }) => (isFunction(isEnabled) ? isEnabled(args) : isEnabled),
getCronSchedule: (args: { config: Config }) => (isFunction(cronSchedule) ? cronSchedule(args) : cronSchedule),
getRunOnStartup: (args: { config: Config }) => (isFunction(runOnStartup) ? runOnStartup(args) : runOnStartup),
};
}

View File

@@ -0,0 +1,28 @@
import type { Config } from '../config/config.types';
import { createCadence } from '@cadence-mq/core';
import { createMemoryDriver } from '@cadence-mq/driver-memory';
import { createLogger } from '../shared/logger/logger';
export type TaskServices = ReturnType<typeof createTaskServices>;
const logger = createLogger({ namespace: 'tasks:services' });
export function createTaskServices({ config }: { config: Config }) {
const workerId = config.tasks.worker.id ?? 'default';
const driver = createMemoryDriver();
const cadence = createCadence({ driver, logger });
return {
...cadence,
start: () => {
const worker = cadence.createWorker({ workerId });
worker.start();
logger.info({ workerId }, 'Task worker started');
return worker;
},
};
}

View File

@@ -0,0 +1,8 @@
import { overrideConfig } from '../config/config.test-utils';
import { createTaskServices } from './tasks.services';
export function createInMemoryTaskServices({ workerId = 'test' }: { workerId?: string } = {}) {
const config = overrideConfig({ tasks: { worker: { id: workerId } } });
return createTaskServices({ config });
}

View File

@@ -4,6 +4,7 @@ import type { WebhookRepository } from './webhook.repository';
import type { Webhook } from './webhooks.types';
import { triggerWebhook as triggerWebhookServiceImpl } from '@papra/webhooks';
import pLimit from 'p-limit';
import { createDeferable } from '../shared/async/defer';
import { createLogger } from '../shared/logger/logger';
import { createWebhookNotFoundError } from './webhook.errors';
@@ -107,6 +108,8 @@ export async function triggerWebhooks({
);
}
export const deferTriggerWebhooks = createDeferable(triggerWebhooks);
export async function triggerWebhook({
webhook,
webhookRepository,

View File

@@ -7,9 +7,7 @@ import { ensureLocalDatabaseDirectoryExists } from '../../modules/app/database/d
import { parseConfig } from '../../modules/config/config';
import { createLogger, wrapWithLoggerContext } from '../../modules/shared/logger/logger';
export { runScript };
async function runScript(
export async function runScriptWithDb(
{ scriptName }: { scriptName: string },
fn: (args: { isDryRun: boolean; logger: Logger; db: Database; config: Config }) => Promise<void> | void,
) {
@@ -25,18 +23,32 @@ async function runScript(
const { config } = await parseConfig({ env: process.env });
await ensureLocalDatabaseDirectoryExists({ config });
const { db, client } = setupDatabase({ ...config.database });
const { db } = setupDatabase({ ...config.database });
try {
logger.info('Script started');
await fn({ isDryRun, logger, db, config });
logger.info('Script finished');
} catch (error) {
logger.error({ error }, 'Script failed');
process.exit(1);
} finally {
client.close();
}
await executeScript({ logger, fn: async () => fn({ isDryRun, logger, db, config }) });
},
);
}
export async function runScript(
{ scriptName }: { scriptName: string },
fn: (args: { isDryRun: boolean; logger: Logger }) => Promise<void> | void,
) {
const isDryRun = process.argv.includes('--dry-run');
await wrapWithLoggerContext({ scriptName, isDryRun }, async () => {
const logger = createLogger({ namespace: 'scripts' });
await executeScript({ logger, fn: async () => fn({ isDryRun, logger }) });
});
}
async function executeScript({ logger, fn }: { logger: Logger; fn: () => Promise<unknown> }) {
try {
await fn();
logger.debug('Script finished');
} catch (error) {
logger.error({ error }, 'Script failed');
process.exit(1);
}
}

View File

@@ -0,0 +1,79 @@
import fs from 'node:fs/promises';
import path from 'node:path';
import process from 'node:process';
import { camelCase, kebabCase } from 'lodash-es';
import { builders, loadFile, writeFile } from 'magicast';
import { runScript } from './commons/run-script';
const currentDirectory = import.meta.dirname;
const migrationsDirectory = path.join(currentDirectory, '..', 'migrations', 'list');
async function getLastMigrationFilePrefixNumber() {
const migrations = await fs.readdir(migrationsDirectory);
const lastMigrationFileName = migrations.filter(file => file.endsWith('.migration.ts')).toSorted().pop();
if (lastMigrationFileName === undefined) {
return 0;
}
const [, lastMigrationNumber] = lastMigrationFileName.match(/^(\d+)/) ?? [];
return lastMigrationNumber === undefined ? 0 : Number.parseInt(lastMigrationNumber);
}
await runScript(
{ scriptName: 'create-migration' },
async ({ logger }) => {
const rawMigrationName = process.argv[2];
if (rawMigrationName === undefined || rawMigrationName === '') {
logger.error('Migration name is required, example: pnpm migrate:create <migration-name>');
process.exit(1);
}
const migrationName = kebabCase(rawMigrationName);
const lastMigrationPrefixNumber = await getLastMigrationFilePrefixNumber();
const prefixNumber = (lastMigrationPrefixNumber + 1).toString().padStart(4, '0');
const fileNameWithoutExtension = `${prefixNumber}-${migrationName}.migration`;
const fileName = `${fileNameWithoutExtension}.ts`;
const migrationPath = path.join(migrationsDirectory, fileName);
const migrationObjectIdentifier = `${camelCase(migrationName)}Migration`;
await fs.writeFile(migrationPath, `
import type { Migration } from '../migrations.types';
import { sql } from 'drizzle-orm';
export const ${migrationObjectIdentifier} = {
name: '${migrationName}',
up: async ({ db }) => {
await db.batch([
db.run(sql\`SELECT 1\`),
]);
},
down: async ({ db }) => {
await db.batch([
db.run(sql\`SELECT 1\`),
]);
},
} satisfies Migration;`.trim());
logger.info(`Migration ${fileName} created`);
const registry = await loadFile(path.join(migrationsDirectory, '..', 'migrations.registry.ts'));
registry.imports.$append({
imported: migrationObjectIdentifier,
from: `./list/${fileNameWithoutExtension}`,
});
// eslint-disable-next-line ts/no-unsafe-call, ts/no-unsafe-member-access
registry.exports.migrations.push(builders.raw(migrationObjectIdentifier));
await writeFile(registry, path.join(migrationsDirectory, '..', 'migrations.registry.ts'));
},
);

Some files were not shown because too many files have changed in this diff Show More