mirror of
https://github.com/papra-hq/papra.git
synced 2025-12-17 20:25:42 -06:00
Compare commits
4 Commits
storage-dr
...
@papra/app
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
40b0557553 | ||
|
|
b5a0317d24 | ||
|
|
9730a06468 | ||
|
|
ec0a437d86 |
@@ -1,5 +0,0 @@
|
||||
---
|
||||
"@papra/app-client": patch
|
||||
---
|
||||
|
||||
Lazy load the PDF viewer to reduce the main chunk size
|
||||
@@ -1,6 +0,0 @@
|
||||
---
|
||||
"@papra/app-client": patch
|
||||
"@papra/app-server": patch
|
||||
---
|
||||
|
||||
Allow for more complex intake-email origin adresses
|
||||
@@ -1,8 +0,0 @@
|
||||
---
|
||||
"@papra/webhooks": minor
|
||||
"@papra/api-sdk": minor
|
||||
"@papra/lecture": minor
|
||||
"@papra/cli": minor
|
||||
---
|
||||
|
||||
Ditched CommonJs build for packages
|
||||
@@ -1,5 +0,0 @@
|
||||
---
|
||||
"@papra/app-server": patch
|
||||
---
|
||||
|
||||
Use node file streams in ingestion folder for smaller RAM footprint
|
||||
@@ -1,5 +0,0 @@
|
||||
---
|
||||
"@papra/app-client": patch
|
||||
---
|
||||
|
||||
Simplified i18n tooling + improved performances
|
||||
@@ -1,5 +0,0 @@
|
||||
---
|
||||
"@papra/app-server": patch
|
||||
---
|
||||
|
||||
Fixed an issue where tags assigned to only deleted documents won't show up in the tag list
|
||||
@@ -1,5 +0,0 @@
|
||||
---
|
||||
"@papra/app-server": minor
|
||||
---
|
||||
|
||||
Dropped support for the dedicated backblaze b2 storage driver as b2 now fully support s3 client
|
||||
@@ -1,5 +0,0 @@
|
||||
---
|
||||
"@papra/app-client": patch
|
||||
---
|
||||
|
||||
Prevent infinit loading in search modal when an error occure
|
||||
@@ -1,6 +0,0 @@
|
||||
---
|
||||
"@papra/app-server": minor
|
||||
"@papra/docs": minor
|
||||
---
|
||||
|
||||
Added documents encryption layer
|
||||
@@ -1,5 +0,0 @@
|
||||
---
|
||||
"@papra/app-server": patch
|
||||
---
|
||||
|
||||
Properly handle missing files errors in storage drivers
|
||||
@@ -1,5 +0,0 @@
|
||||
---
|
||||
"@papra/app-client": patch
|
||||
---
|
||||
|
||||
Improved the UX of the document content edition panel
|
||||
@@ -1,5 +0,0 @@
|
||||
---
|
||||
"@papra/app-server": minor
|
||||
---
|
||||
|
||||
Stream file upload instead of full in-memory loading
|
||||
@@ -1,5 +0,0 @@
|
||||
---
|
||||
"@papra/app-client": patch
|
||||
---
|
||||
|
||||
Added content edition support in demo mode
|
||||
@@ -1,5 +1,11 @@
|
||||
# @papra/docs
|
||||
|
||||
## 0.6.0
|
||||
|
||||
### Minor Changes
|
||||
|
||||
- [#480](https://github.com/papra-hq/papra/pull/480) [`0a03f42`](https://github.com/papra-hq/papra/commit/0a03f42231f691d339c7ab5a5916c52385e31bd2) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Added documents encryption layer
|
||||
|
||||
## 0.5.3
|
||||
|
||||
### Patch Changes
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "@papra/docs",
|
||||
"type": "module",
|
||||
"version": "0.5.3",
|
||||
"version": "0.6.0",
|
||||
"private": true,
|
||||
"packageManager": "pnpm@10.12.3",
|
||||
"description": "Papra documentation website",
|
||||
|
||||
@@ -1,5 +1,21 @@
|
||||
# @papra/app-client
|
||||
|
||||
## 0.9.0
|
||||
|
||||
### Patch Changes
|
||||
|
||||
- [#471](https://github.com/papra-hq/papra/pull/471) [`e77a42f`](https://github.com/papra-hq/papra/commit/e77a42fbf14da011cd396426aa0bbea56c889740) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Lazy load the PDF viewer to reduce the main chunk size
|
||||
|
||||
- [#481](https://github.com/papra-hq/papra/pull/481) [`1606310`](https://github.com/papra-hq/papra/commit/1606310745e8edf405b527127078143481419e8c) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Allow for more complex intake-email origin adresses
|
||||
|
||||
- [#470](https://github.com/papra-hq/papra/pull/470) [`d488efe`](https://github.com/papra-hq/papra/commit/d488efe2cc4aa4f433cec4e9b8cc909b091eccc4) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Simplified i18n tooling + improved performances
|
||||
|
||||
- [#468](https://github.com/papra-hq/papra/pull/468) [`14c3587`](https://github.com/papra-hq/papra/commit/14c3587de07a605ec586bdc428d9e76956bf1c67) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Prevent infinit loading in search modal when an error occure
|
||||
|
||||
- [#468](https://github.com/papra-hq/papra/pull/468) [`14c3587`](https://github.com/papra-hq/papra/commit/14c3587de07a605ec586bdc428d9e76956bf1c67) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Improved the UX of the document content edition panel
|
||||
|
||||
- [#468](https://github.com/papra-hq/papra/pull/468) [`14c3587`](https://github.com/papra-hq/papra/commit/14c3587de07a605ec586bdc428d9e76956bf1c67) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Added content edition support in demo mode
|
||||
|
||||
## 0.8.2
|
||||
|
||||
## 0.8.1
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "@papra/app-client",
|
||||
"type": "module",
|
||||
"version": "0.8.2",
|
||||
"version": "0.9.0",
|
||||
"private": true,
|
||||
"packageManager": "pnpm@10.12.3",
|
||||
"description": "Papra frontend client",
|
||||
|
||||
@@ -1,5 +1,31 @@
|
||||
# @papra/app-server
|
||||
|
||||
## 0.9.0
|
||||
|
||||
### Minor Changes
|
||||
|
||||
- [#472](https://github.com/papra-hq/papra/pull/472) [`b08241f`](https://github.com/papra-hq/papra/commit/b08241f20fc326a65a8de0551a7bfa91d9e4c71d) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Dropped support for the dedicated backblaze b2 storage driver as b2 now fully support s3 client
|
||||
|
||||
- [#480](https://github.com/papra-hq/papra/pull/480) [`0a03f42`](https://github.com/papra-hq/papra/commit/0a03f42231f691d339c7ab5a5916c52385e31bd2) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Added documents encryption layer
|
||||
|
||||
- [#472](https://github.com/papra-hq/papra/pull/472) [`b08241f`](https://github.com/papra-hq/papra/commit/b08241f20fc326a65a8de0551a7bfa91d9e4c71d) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Stream file upload instead of full in-memory loading
|
||||
|
||||
### Patch Changes
|
||||
|
||||
- [#481](https://github.com/papra-hq/papra/pull/481) [`1606310`](https://github.com/papra-hq/papra/commit/1606310745e8edf405b527127078143481419e8c) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Allow for more complex intake-email origin adresses
|
||||
|
||||
- [#483](https://github.com/papra-hq/papra/pull/483) [`ec0a437`](https://github.com/papra-hq/papra/commit/ec0a437d86b4c8c0979ba9d0c2ff7b39f054cec0) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Fix a bug where the ingestion folder was not working when the done or error destination folder path (INGESTION_FOLDER_POST_PROCESSING_MOVE_FOLDER_PATH and INGESTION_FOLDER_ERROR_FOLDER_PATH) were absolute.
|
||||
|
||||
- [#475](https://github.com/papra-hq/papra/pull/475) [`ea9d90d`](https://github.com/papra-hq/papra/commit/ea9d90d6cff6954297152b3ad16f99170e8cd0dc) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Use node file streams in ingestion folder for smaller RAM footprint
|
||||
|
||||
- [#477](https://github.com/papra-hq/papra/pull/477) [`a62d376`](https://github.com/papra-hq/papra/commit/a62d3767729ab02ae203a1ac7b7fd6eb6e011d98) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Fixed an issue where tags assigned to only deleted documents won't show up in the tag list
|
||||
|
||||
- [#472](https://github.com/papra-hq/papra/pull/472) [`b08241f`](https://github.com/papra-hq/papra/commit/b08241f20fc326a65a8de0551a7bfa91d9e4c71d) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Properly handle missing files errors in storage drivers
|
||||
|
||||
- Updated dependencies [[`14bc2b8`](https://github.com/papra-hq/papra/commit/14bc2b8f8d0d6605062f37188e7c57bbc61b2c1a)]:
|
||||
- @papra/webhooks@0.3.0
|
||||
- @papra/lecture@0.2.0
|
||||
|
||||
## 0.8.2
|
||||
|
||||
### Patch Changes
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "@papra/app-server",
|
||||
"type": "module",
|
||||
"version": "0.8.2",
|
||||
"version": "0.9.0",
|
||||
"private": true,
|
||||
"packageManager": "pnpm@10.12.3",
|
||||
"description": "Papra app server",
|
||||
|
||||
@@ -18,7 +18,7 @@ const { config } = await parseConfig({ env });
|
||||
await ensureLocalDatabaseDirectoryExists({ config });
|
||||
const { db, client } = setupDatabase(config.database);
|
||||
|
||||
const documentsStorageService = createDocumentStorageService({ config });
|
||||
const documentsStorageService = createDocumentStorageService({ documentStorageConfig: config.documentsStorage });
|
||||
|
||||
const taskServices = createTaskServices({ config });
|
||||
const { app } = await createServer({ config, db, taskServices, documentsStorageService });
|
||||
|
||||
@@ -26,7 +26,7 @@ async function createGlobalDependencies(partialDeps: Partial<GlobalDependencies>
|
||||
const auth = partialDeps.auth ?? getAuth({ db, config, authEmailsServices: createAuthEmailsServices({ emailsServices }), trackingServices }).auth;
|
||||
const subscriptionsServices = createSubscriptionsServices({ config });
|
||||
const taskServices = partialDeps.taskServices ?? createTaskServices({ config });
|
||||
const documentsStorageService = partialDeps.documentsStorageService ?? createDocumentStorageService({ config });
|
||||
const documentsStorageService = partialDeps.documentsStorageService ?? createDocumentStorageService({ documentStorageConfig: config.documentsStorage });
|
||||
|
||||
return {
|
||||
documentsStorageService,
|
||||
|
||||
@@ -49,7 +49,7 @@ function setupCreateDocumentRoute({ app, ...deps }: RouteDefinitionContext) {
|
||||
headers: context.req.header(),
|
||||
});
|
||||
|
||||
const createDocument = await createDocumentCreationUsecase({ ...deps });
|
||||
const createDocument = createDocumentCreationUsecase({ ...deps });
|
||||
|
||||
const { document } = await createDocument({ fileStream, fileName, mimeType, userId, organizationId });
|
||||
|
||||
|
||||
@@ -33,9 +33,9 @@ describe('documents usecases', () => {
|
||||
organizationPlans: { isFreePlanUnlimited: true },
|
||||
documentsStorage: { driver: 'in-memory' },
|
||||
});
|
||||
const documentsStorageService = createDocumentStorageService({ config });
|
||||
const documentsStorageService = createDocumentStorageService({ documentStorageConfig: config.documentsStorage });
|
||||
|
||||
const createDocument = await createDocumentCreationUsecase({
|
||||
const createDocument = createDocumentCreationUsecase({
|
||||
db,
|
||||
config,
|
||||
generateDocumentId: () => 'doc_1',
|
||||
@@ -93,10 +93,10 @@ describe('documents usecases', () => {
|
||||
documentsStorage: { driver: 'in-memory' },
|
||||
});
|
||||
|
||||
const documentsStorageService = createDocumentStorageService({ config });
|
||||
const documentsStorageService = createDocumentStorageService({ documentStorageConfig: config.documentsStorage });
|
||||
|
||||
let documentIdIndex = 1;
|
||||
const createDocument = await createDocumentCreationUsecase({
|
||||
const createDocument = createDocumentCreationUsecase({
|
||||
db,
|
||||
config,
|
||||
generateDocumentId: () => `doc_${documentIdIndex++}`,
|
||||
@@ -201,7 +201,7 @@ describe('documents usecases', () => {
|
||||
organizationPlans: { isFreePlanUnlimited: true },
|
||||
});
|
||||
|
||||
const createDocument = await createDocumentCreationUsecase({
|
||||
const createDocument = createDocumentCreationUsecase({
|
||||
db,
|
||||
config,
|
||||
taskServices,
|
||||
@@ -254,9 +254,9 @@ describe('documents usecases', () => {
|
||||
});
|
||||
|
||||
const documentsRepository = createDocumentsRepository({ db });
|
||||
const documentsStorageService = createDocumentStorageService({ config });
|
||||
const documentsStorageService = createDocumentStorageService({ documentStorageConfig: config.documentsStorage });
|
||||
|
||||
const createDocument = await createDocumentCreationUsecase({
|
||||
const createDocument = createDocumentCreationUsecase({
|
||||
documentsStorageService,
|
||||
db,
|
||||
config,
|
||||
@@ -305,7 +305,7 @@ describe('documents usecases', () => {
|
||||
});
|
||||
|
||||
let documentIdIndex = 1;
|
||||
const createDocument = await createDocumentCreationUsecase({
|
||||
const createDocument = createDocumentCreationUsecase({
|
||||
db,
|
||||
config,
|
||||
generateDocumentId: () => `doc_${documentIdIndex++}`,
|
||||
@@ -369,7 +369,7 @@ describe('documents usecases', () => {
|
||||
}),
|
||||
} as PlansRepository;
|
||||
|
||||
const createDocument = await createDocumentCreationUsecase({
|
||||
const createDocument = createDocumentCreationUsecase({
|
||||
db,
|
||||
config: overrideConfig(),
|
||||
taskServices,
|
||||
@@ -434,7 +434,7 @@ describe('documents usecases', () => {
|
||||
}),
|
||||
} as PlansRepository;
|
||||
|
||||
const createDocument = await createDocumentCreationUsecase({
|
||||
const createDocument = createDocumentCreationUsecase({
|
||||
db,
|
||||
config: overrideConfig(),
|
||||
taskServices,
|
||||
@@ -492,7 +492,7 @@ describe('documents usecases', () => {
|
||||
}),
|
||||
} as PlansRepository;
|
||||
|
||||
const createDocument = await createDocumentCreationUsecase({
|
||||
const createDocument = createDocumentCreationUsecase({
|
||||
db,
|
||||
config: overrideConfig(),
|
||||
taskServices,
|
||||
@@ -533,7 +533,7 @@ describe('documents usecases', () => {
|
||||
});
|
||||
|
||||
const documentsRepository = createDocumentsRepository({ db });
|
||||
const documentsStorageService = createDocumentStorageService({ config });
|
||||
const documentsStorageService = createDocumentStorageService({ documentStorageConfig: config.documentsStorage });
|
||||
const taggingRulesRepository = createTaggingRulesRepository({ db });
|
||||
const tagsRepository = createTagsRepository({ db });
|
||||
|
||||
|
||||
@@ -176,7 +176,7 @@ export async function createDocument({
|
||||
export type CreateDocumentUsecase = Awaited<ReturnType<typeof createDocumentCreationUsecase>>;
|
||||
export type DocumentUsecaseDependencies = Omit<Parameters<typeof createDocument>[0], 'fileStream' | 'fileName' | 'mimeType' | 'userId' | 'organizationId'>;
|
||||
|
||||
export async function createDocumentCreationUsecase({
|
||||
export function createDocumentCreationUsecase({
|
||||
db,
|
||||
config,
|
||||
taskServices,
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import type { Config } from '../../config/config.types';
|
||||
import type { DocumentStorageConfig } from './documents.storage.types';
|
||||
import type { StorageDriver, StorageDriverFactory, StorageServices } from './drivers/drivers.models';
|
||||
import { createError } from '../../shared/errors/errors';
|
||||
import { isNil } from '../../shared/utils';
|
||||
@@ -17,8 +18,8 @@ const storageDriverFactories = {
|
||||
|
||||
export type DocumentStorageService = Awaited<ReturnType<typeof createDocumentStorageService>>;
|
||||
|
||||
export function createDocumentStorageService({ config }: { config: Config }): StorageServices {
|
||||
const storageDriverName = config.documentsStorage.driver;
|
||||
export function createDocumentStorageService({ documentStorageConfig }: { documentStorageConfig: DocumentStorageConfig }): StorageServices {
|
||||
const storageDriverName = documentStorageConfig.driver;
|
||||
|
||||
const storageDriverFactory: StorageDriverFactory | undefined = storageDriverFactories[storageDriverName];
|
||||
|
||||
@@ -31,11 +32,11 @@ export function createDocumentStorageService({ config }: { config: Config }): St
|
||||
});
|
||||
}
|
||||
|
||||
const storageDriver = storageDriverFactory({ config });
|
||||
const storageDriver = storageDriverFactory({ documentStorageConfig });
|
||||
|
||||
return createDocumentStorageServiceFromDriver({
|
||||
storageDriver,
|
||||
encryptionConfig: config.documentsStorage.encryption,
|
||||
encryptionConfig: documentStorageConfig.encryption,
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
import type { Config } from '../../config/config.types';
|
||||
|
||||
export type DocumentStorageConfig = Config['documentsStorage'];
|
||||
@@ -1,7 +1,7 @@
|
||||
import type { DocumentStorageConfig } from '../../documents.storage.types';
|
||||
import { AzuriteContainer } from '@testcontainers/azurite';
|
||||
import { describe } from 'vitest';
|
||||
import { TEST_CONTAINER_IMAGES } from '../../../../../../test/containers/images';
|
||||
import { overrideConfig } from '../../../../config/config.test-utils';
|
||||
import { runDriverTestSuites } from '../drivers.test-suite';
|
||||
import { azBlobStorageDriverFactory } from './az-blob.storage-driver';
|
||||
|
||||
@@ -13,11 +13,7 @@ describe('az-blob storage-driver', () => {
|
||||
const azuriteContainer = await new AzuriteContainer(TEST_CONTAINER_IMAGES.AZURITE).withInMemoryPersistence().start();
|
||||
const connectionString = azuriteContainer.getConnectionString();
|
||||
|
||||
const config = overrideConfig({
|
||||
documentsStorage: { drivers: { azureBlob: { connectionString, containerName: 'test-container' } } },
|
||||
});
|
||||
|
||||
const driver = azBlobStorageDriverFactory({ config });
|
||||
const driver = azBlobStorageDriverFactory({ documentStorageConfig: { drivers: { azureBlob: { connectionString, containerName: 'test-container' } } } as DocumentStorageConfig });
|
||||
const client = driver.getClient();
|
||||
await client.createContainer('test-container');
|
||||
|
||||
|
||||
@@ -11,8 +11,8 @@ function isAzureBlobNotFoundError(error: Error): boolean {
|
||||
return ('statusCode' in error && error.statusCode === 404) || ('code' in error && error.code === 'BlobNotFound');
|
||||
}
|
||||
|
||||
export const azBlobStorageDriverFactory = defineStorageDriver(({ config }) => {
|
||||
const { accountName, accountKey, containerName, connectionString } = config.documentsStorage.drivers.azureBlob;
|
||||
export const azBlobStorageDriverFactory = defineStorageDriver(({ documentStorageConfig }) => {
|
||||
const { accountName, accountKey, containerName, connectionString } = documentStorageConfig.drivers.azureBlob;
|
||||
|
||||
const blobServiceClient = connectionString !== undefined
|
||||
? BlobServiceClient.fromConnectionString(connectionString)
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import type { Readable } from 'node:stream';
|
||||
import type { Config } from '../../../config/config.types';
|
||||
import type { ExtendNamedArguments, ExtendReturnPromise } from '../../../shared/types';
|
||||
import type { DocumentStorageConfig } from '../documents.storage.types';
|
||||
|
||||
export type StorageDriver = {
|
||||
name: string;
|
||||
@@ -31,7 +31,7 @@ export type StorageServices = {
|
||||
deleteFile: StorageDriver['deleteFile'];
|
||||
};
|
||||
|
||||
export type StorageDriverFactory = (args: { config: Config }) => StorageDriver;
|
||||
export type StorageDriverFactory = (args: { documentStorageConfig: DocumentStorageConfig }) => StorageDriver;
|
||||
|
||||
export function defineStorageDriver<T extends StorageDriverFactory>(factory: T) {
|
||||
return factory;
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import type { Config } from '../../../../config/config.types';
|
||||
import type { DocumentStorageConfig } from '../../documents.storage.types';
|
||||
import fs from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import path, { join } from 'node:path';
|
||||
@@ -28,18 +28,10 @@ describe('storage driver', () => {
|
||||
createDriver: async () => {
|
||||
const tmpDirectory = await createTmpDirectory();
|
||||
|
||||
const config = {
|
||||
documentsStorage: {
|
||||
drivers: {
|
||||
filesystem: {
|
||||
root: tmpDirectory,
|
||||
},
|
||||
},
|
||||
},
|
||||
} as Config;
|
||||
|
||||
return {
|
||||
driver: fsStorageDriverFactory({ config }),
|
||||
driver: fsStorageDriverFactory({
|
||||
documentStorageConfig: { drivers: { filesystem: { root: tmpDirectory } } } as DocumentStorageConfig,
|
||||
}),
|
||||
[Symbol.asyncDispose]: async () => {
|
||||
await deleteTmpDirectory(tmpDirectory);
|
||||
},
|
||||
@@ -49,17 +41,7 @@ describe('storage driver', () => {
|
||||
|
||||
describe('saveFile', () => {
|
||||
test('persists the file to the filesystem', async () => {
|
||||
const config = {
|
||||
documentsStorage: {
|
||||
drivers: {
|
||||
filesystem: {
|
||||
root: tmpDirectory,
|
||||
},
|
||||
},
|
||||
},
|
||||
} as Config;
|
||||
|
||||
const fsStorageDriver = fsStorageDriverFactory({ config });
|
||||
const fsStorageDriver = fsStorageDriverFactory({ documentStorageConfig: { drivers: { filesystem: { root: tmpDirectory } } } as DocumentStorageConfig });
|
||||
|
||||
const { storageKey } = await fsStorageDriver.saveFile({
|
||||
fileStream: createReadableStream({ content: 'lorem ipsum' }),
|
||||
@@ -77,17 +59,7 @@ describe('storage driver', () => {
|
||||
});
|
||||
|
||||
test('an error is raised if the file already exists', async () => {
|
||||
const config = {
|
||||
documentsStorage: {
|
||||
drivers: {
|
||||
filesystem: {
|
||||
root: tmpDirectory,
|
||||
},
|
||||
},
|
||||
},
|
||||
} as Config;
|
||||
|
||||
const fsStorageDriver = fsStorageDriverFactory({ config });
|
||||
const fsStorageDriver = fsStorageDriverFactory({ documentStorageConfig: { drivers: { filesystem: { root: tmpDirectory } } } as DocumentStorageConfig });
|
||||
|
||||
await fsStorageDriver.saveFile({
|
||||
fileStream: createReadableStream({ content: 'lorem ipsum' }),
|
||||
@@ -109,17 +81,7 @@ describe('storage driver', () => {
|
||||
|
||||
describe('getFileStream', () => {
|
||||
test('get a readable stream of a stored file', async () => {
|
||||
const config = {
|
||||
documentsStorage: {
|
||||
drivers: {
|
||||
filesystem: {
|
||||
root: tmpDirectory,
|
||||
},
|
||||
},
|
||||
},
|
||||
} as Config;
|
||||
|
||||
const fsStorageDriver = fsStorageDriverFactory({ config });
|
||||
const fsStorageDriver = fsStorageDriverFactory({ documentStorageConfig: { drivers: { filesystem: { root: tmpDirectory } } } as DocumentStorageConfig });
|
||||
|
||||
await fsStorageDriver.saveFile({
|
||||
fileStream: createReadableStream({ content: 'lorem ipsum' }),
|
||||
@@ -139,17 +101,7 @@ describe('storage driver', () => {
|
||||
});
|
||||
|
||||
test('an error is raised if the file does not exist', async () => {
|
||||
const config = {
|
||||
documentsStorage: {
|
||||
drivers: {
|
||||
filesystem: {
|
||||
root: tmpDirectory,
|
||||
},
|
||||
},
|
||||
},
|
||||
} as Config;
|
||||
|
||||
const fsStorageDriver = fsStorageDriverFactory({ config });
|
||||
const fsStorageDriver = fsStorageDriverFactory({ documentStorageConfig: { drivers: { filesystem: { root: tmpDirectory } } } as DocumentStorageConfig });
|
||||
|
||||
await expect(fsStorageDriver.getFileStream({ storageKey: 'org_1/text-file.txt' })).rejects.toThrow(createFileNotFoundError());
|
||||
});
|
||||
@@ -157,17 +109,7 @@ describe('storage driver', () => {
|
||||
|
||||
describe('deleteFile', () => {
|
||||
test('deletes a stored file', async () => {
|
||||
const config = {
|
||||
documentsStorage: {
|
||||
drivers: {
|
||||
filesystem: {
|
||||
root: tmpDirectory,
|
||||
},
|
||||
},
|
||||
},
|
||||
} as Config;
|
||||
|
||||
const fsStorageDriver = fsStorageDriverFactory({ config });
|
||||
const fsStorageDriver = fsStorageDriverFactory({ documentStorageConfig: { drivers: { filesystem: { root: tmpDirectory } } } as DocumentStorageConfig });
|
||||
|
||||
await fsStorageDriver.saveFile({
|
||||
fileStream: createReadableStream({ content: 'lorem ipsum' }),
|
||||
@@ -189,17 +131,7 @@ describe('storage driver', () => {
|
||||
});
|
||||
|
||||
test('when the file does not exist, an error is raised', async () => {
|
||||
const config = {
|
||||
documentsStorage: {
|
||||
drivers: {
|
||||
filesystem: {
|
||||
root: tmpDirectory,
|
||||
},
|
||||
},
|
||||
},
|
||||
} as Config;
|
||||
|
||||
const fsStorageDriver = fsStorageDriverFactory({ config });
|
||||
const fsStorageDriver = fsStorageDriverFactory({ documentStorageConfig: { drivers: { filesystem: { root: tmpDirectory } } } as DocumentStorageConfig });
|
||||
|
||||
await expect(fsStorageDriver.deleteFile({ storageKey: 'org_1/text-file.txt' })).rejects.toThrow(createFileNotFoundError());
|
||||
});
|
||||
|
||||
@@ -8,8 +8,8 @@ import { createFileAlreadyExistsError } from './fs.storage-driver.errors';
|
||||
|
||||
export const FS_STORAGE_DRIVER_NAME = 'filesystem' as const;
|
||||
|
||||
export const fsStorageDriverFactory = defineStorageDriver(({ config }) => {
|
||||
const { root } = config.documentsStorage.drivers.filesystem;
|
||||
export const fsStorageDriverFactory = defineStorageDriver(({ documentStorageConfig }) => {
|
||||
const { root } = documentStorageConfig.drivers.filesystem;
|
||||
|
||||
const getStoragePath = ({ storageKey }: { storageKey: string }) => ({ storagePath: join(root, storageKey) });
|
||||
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
import type { DocumentStorageConfig } from '../../documents.storage.types';
|
||||
import { CreateBucketCommand } from '@aws-sdk/client-s3';
|
||||
import { LocalstackContainer } from '@testcontainers/localstack';
|
||||
import { describe } from 'vitest';
|
||||
import { TEST_CONTAINER_IMAGES } from '../../../../../../test/containers/images';
|
||||
import { overrideConfig } from '../../../../config/config.test-utils';
|
||||
import { runDriverTestSuites } from '../drivers.test-suite';
|
||||
import { s3StorageDriverFactory } from './s3.storage-driver';
|
||||
|
||||
@@ -16,8 +16,8 @@ describe('s3 storage-driver', () => {
|
||||
const localstackContainer = await new LocalstackContainer(TEST_CONTAINER_IMAGES.LOCALSTACK).start();
|
||||
const bucketName = 'test-bucket';
|
||||
|
||||
const config = overrideConfig({
|
||||
documentsStorage: {
|
||||
const driver = s3StorageDriverFactory({
|
||||
documentStorageConfig: {
|
||||
drivers: {
|
||||
s3: {
|
||||
accessKeyId: 'test',
|
||||
@@ -28,11 +28,9 @@ describe('s3 storage-driver', () => {
|
||||
forcePathStyle: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
} as DocumentStorageConfig,
|
||||
});
|
||||
|
||||
const driver = s3StorageDriverFactory({ config });
|
||||
|
||||
const s3Client = driver.getClient();
|
||||
|
||||
await s3Client.send(new CreateBucketCommand({ Bucket: bucketName }));
|
||||
|
||||
@@ -15,8 +15,8 @@ function isS3NotFoundError(error: Error) {
|
||||
|| ('Code' in error && typeof error.Code === 'string' && codes.includes(error.Code));
|
||||
}
|
||||
|
||||
export const s3StorageDriverFactory = defineStorageDriver(({ config }) => {
|
||||
const { accessKeyId, secretAccessKey, bucketName, region, endpoint, forcePathStyle } = config.documentsStorage.drivers.s3;
|
||||
export const s3StorageDriverFactory = defineStorageDriver(({ documentStorageConfig }) => {
|
||||
const { accessKeyId, secretAccessKey, bucketName, region, endpoint, forcePathStyle } = documentStorageConfig.drivers.s3;
|
||||
|
||||
const s3Client = new S3Client({
|
||||
region,
|
||||
|
||||
@@ -23,7 +23,7 @@ describe('document-encryption usecases', () => {
|
||||
|
||||
const storageDriver = inMemoryStorageDriverFactory();
|
||||
|
||||
const createDocumentWithoutEncryption = await createDocumentCreationUsecase({
|
||||
const createDocumentWithoutEncryption = createDocumentCreationUsecase({
|
||||
db,
|
||||
config: overrideConfig(),
|
||||
taskServices: noopTaskServices,
|
||||
@@ -61,7 +61,7 @@ describe('document-encryption usecases', () => {
|
||||
},
|
||||
});
|
||||
|
||||
const createDocumentWithEncryption = await createDocumentCreationUsecase({
|
||||
const createDocumentWithEncryption = createDocumentCreationUsecase({
|
||||
db,
|
||||
documentsStorageService: documentStorageServiceWithEncryption,
|
||||
config: overrideConfig(),
|
||||
|
||||
@@ -17,7 +17,12 @@ export async function encryptAllUnencryptedDocuments({
|
||||
deleteUnencryptedAfterEncryption?: boolean;
|
||||
}) {
|
||||
const documents = await db
|
||||
.select({ id: documentsTable.id, originalStorageKey: documentsTable.originalStorageKey, fileName: documentsTable.originalName, mimeType: documentsTable.mimeType })
|
||||
.select({
|
||||
id: documentsTable.id,
|
||||
originalStorageKey: documentsTable.originalStorageKey,
|
||||
fileName: documentsTable.originalName,
|
||||
mimeType: documentsTable.mimeType,
|
||||
})
|
||||
.from(documentsTable)
|
||||
.where(isNull(documentsTable.fileEncryptionKeyWrapped))
|
||||
.orderBy(documentsTable.id);
|
||||
@@ -34,15 +39,26 @@ export async function encryptAllUnencryptedDocuments({
|
||||
fileEncryptionKekVersion: null,
|
||||
});
|
||||
const newStorageKey = `${originalStorageKey}.enc`;
|
||||
const { storageKey, ...encryptionFields } = await documentStorageService.saveFile({ fileStream, fileName, mimeType, storageKey: newStorageKey });
|
||||
const { storageKey, ...encryptionFields }
|
||||
= await documentStorageService.saveFile({
|
||||
fileStream,
|
||||
fileName,
|
||||
mimeType,
|
||||
storageKey: newStorageKey,
|
||||
});
|
||||
|
||||
await db.update(documentsTable).set({
|
||||
...encryptionFields,
|
||||
originalStorageKey: storageKey,
|
||||
}).where(eq(documentsTable.id, id));
|
||||
await db
|
||||
.update(documentsTable)
|
||||
.set({
|
||||
...encryptionFields,
|
||||
originalStorageKey: storageKey,
|
||||
})
|
||||
.where(eq(documentsTable.id, id));
|
||||
|
||||
if (deleteUnencryptedAfterEncryption) {
|
||||
await documentStorageService.deleteFile({ storageKey: originalStorageKey });
|
||||
await documentStorageService.deleteFile({
|
||||
storageKey: originalStorageKey,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -52,7 +52,7 @@ describe('ingestion-folders usecases', () => {
|
||||
organizationsRepository,
|
||||
logger,
|
||||
fs,
|
||||
createDocument: await createDocumentCreationUsecase({ db, config, logger, documentsStorageService, generateDocumentId, taskServices }),
|
||||
createDocument: createDocumentCreationUsecase({ db, config, logger, documentsStorageService, generateDocumentId, taskServices }),
|
||||
});
|
||||
|
||||
// Check database
|
||||
@@ -154,7 +154,7 @@ describe('ingestion-folders usecases', () => {
|
||||
organizationsRepository,
|
||||
logger,
|
||||
fs,
|
||||
createDocument: await createDocumentCreationUsecase({ db, config, logger, documentsStorageService, generateDocumentId, taskServices }),
|
||||
createDocument: createDocumentCreationUsecase({ db, config, logger, documentsStorageService, generateDocumentId, taskServices }),
|
||||
});
|
||||
|
||||
// Check database
|
||||
@@ -257,7 +257,7 @@ describe('ingestion-folders usecases', () => {
|
||||
organizationsRepository,
|
||||
logger,
|
||||
fs,
|
||||
createDocument: await createDocumentCreationUsecase({ db, config, logger, documentsStorageService, generateDocumentId, taskServices }),
|
||||
createDocument: createDocumentCreationUsecase({ db, config, logger, documentsStorageService, generateDocumentId, taskServices }),
|
||||
}));
|
||||
|
||||
expect(error).to.deep.equal(createInvalidPostProcessingStrategyError({ strategy: 'unknown' }));
|
||||
@@ -335,7 +335,7 @@ describe('ingestion-folders usecases', () => {
|
||||
throw new Error('File not found');
|
||||
},
|
||||
},
|
||||
createDocument: await createDocumentCreationUsecase({ db, config, logger, documentsStorageService, generateDocumentId, taskServices }),
|
||||
createDocument: createDocumentCreationUsecase({ db, config, logger, documentsStorageService, generateDocumentId, taskServices }),
|
||||
});
|
||||
|
||||
// Check logs
|
||||
@@ -475,7 +475,7 @@ describe('ingestion-folders usecases', () => {
|
||||
organizationsRepository,
|
||||
logger,
|
||||
fs,
|
||||
createDocument: await createDocumentCreationUsecase({ db, config, logger, documentsStorageService, generateDocumentId, taskServices }),
|
||||
createDocument: createDocumentCreationUsecase({ db, config, logger, documentsStorageService, generateDocumentId, taskServices }),
|
||||
});
|
||||
|
||||
// Check database
|
||||
|
||||
@@ -47,7 +47,7 @@ export function createIngestionFolderWatcher({
|
||||
return {
|
||||
startWatchingIngestionFolders: async () => {
|
||||
const organizationsRepository = createOrganizationsRepository({ db });
|
||||
const createDocument = await createDocumentCreationUsecase({ db, config, logger, taskServices, documentsStorageService });
|
||||
const createDocument = createDocumentCreationUsecase({ db, config, logger, taskServices, documentsStorageService });
|
||||
|
||||
const ignored = await buildPathIgnoreFunction({ config, cwd, organizationsRepository });
|
||||
|
||||
@@ -235,10 +235,10 @@ async function buildPathIgnoreFunction({
|
||||
const { organizationIds } = await organizationsRepository.getAllOrganizationIds();
|
||||
|
||||
const doneFolders = strategy === 'move'
|
||||
? isAbsolute(moveToFolderPath) ? moveToFolderPath : uniq(organizationIds.map(id => join(cwd, folderRootPath, id, moveToFolderPath)))
|
||||
? isAbsolute(moveToFolderPath) ? [moveToFolderPath] : uniq(organizationIds.map(id => join(cwd, folderRootPath, id, moveToFolderPath)))
|
||||
: [];
|
||||
|
||||
const errorFolders = isAbsolute(errorFolder) ? errorFolder : uniq(organizationIds.map(id => join(cwd, folderRootPath, id, errorFolder)));
|
||||
const errorFolders = isAbsolute(errorFolder) ? [errorFolder] : uniq(organizationIds.map(id => join(cwd, folderRootPath, id, errorFolder)));
|
||||
|
||||
const ignoredFolders = [...doneFolders, ...errorFolders];
|
||||
const matchExcludedPatterns = picomatch(ignoredPatterns);
|
||||
|
||||
@@ -189,7 +189,7 @@ function setupIngestIntakeEmailRoute({ app, db, config, trackingServices, taskSe
|
||||
|
||||
const intakeEmailsRepository = createIntakeEmailsRepository({ db });
|
||||
|
||||
const createDocument = await createDocumentCreationUsecase({
|
||||
const createDocument = createDocumentCreationUsecase({
|
||||
documentsStorageService,
|
||||
db,
|
||||
config,
|
||||
|
||||
@@ -29,7 +29,7 @@ describe('intake-emails usecases', () => {
|
||||
|
||||
const intakeEmailsRepository = createIntakeEmailsRepository({ db });
|
||||
|
||||
const createDocument = await createDocumentCreationUsecase({
|
||||
const createDocument = createDocumentCreationUsecase({
|
||||
db,
|
||||
taskServices,
|
||||
documentsStorageService: inMemoryStorageDriverFactory(),
|
||||
@@ -71,7 +71,7 @@ describe('intake-emails usecases', () => {
|
||||
|
||||
const intakeEmailsRepository = createIntakeEmailsRepository({ db });
|
||||
|
||||
const createDocument = await createDocumentCreationUsecase({
|
||||
const createDocument = createDocumentCreationUsecase({
|
||||
db,
|
||||
taskServices,
|
||||
documentsStorageService: inMemoryStorageDriverFactory(),
|
||||
@@ -104,7 +104,7 @@ describe('intake-emails usecases', () => {
|
||||
|
||||
const intakeEmailsRepository = createIntakeEmailsRepository({ db });
|
||||
|
||||
const createDocument = await createDocumentCreationUsecase({
|
||||
const createDocument = createDocumentCreationUsecase({
|
||||
db,
|
||||
taskServices,
|
||||
documentsStorageService: inMemoryStorageDriverFactory(),
|
||||
@@ -142,7 +142,7 @@ describe('intake-emails usecases', () => {
|
||||
|
||||
const intakeEmailsRepository = createIntakeEmailsRepository({ db });
|
||||
|
||||
const createDocument = await createDocumentCreationUsecase({
|
||||
const createDocument = createDocumentCreationUsecase({
|
||||
db,
|
||||
taskServices,
|
||||
documentsStorageService: inMemoryStorageDriverFactory(),
|
||||
@@ -191,7 +191,7 @@ describe('intake-emails usecases', () => {
|
||||
|
||||
const intakeEmailsRepository = createIntakeEmailsRepository({ db });
|
||||
|
||||
const createDocument = await createDocumentCreationUsecase({
|
||||
const createDocument = createDocumentCreationUsecase({
|
||||
db,
|
||||
taskServices,
|
||||
documentsStorageService: inMemoryStorageDriverFactory(),
|
||||
|
||||
@@ -9,7 +9,7 @@ import { runScriptWithDb } from './commons/run-script';
|
||||
await runScriptWithDb(
|
||||
{ scriptName: 'encrypt-all-documents' },
|
||||
async ({ db, config, logger, isDryRun }) => {
|
||||
const documentStorageService = createDocumentStorageService({ config });
|
||||
const documentStorageService = createDocumentStorageService({ documentStorageConfig: config.documentsStorage });
|
||||
|
||||
if (!config.documentsStorage.encryption.isEncryptionEnabled) {
|
||||
logger.error('Document encryption is not enabled, skipping');
|
||||
|
||||
@@ -1,5 +1,11 @@
|
||||
# @papra/api-sdk
|
||||
|
||||
## 1.1.0
|
||||
|
||||
### Minor Changes
|
||||
|
||||
- [#464](https://github.com/papra-hq/papra/pull/464) [`14bc2b8`](https://github.com/papra-hq/papra/commit/14bc2b8f8d0d6605062f37188e7c57bbc61b2c1a) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Ditched CommonJs build for packages
|
||||
|
||||
## 1.0.2
|
||||
|
||||
### Patch Changes
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "@papra/api-sdk",
|
||||
"type": "module",
|
||||
"version": "1.0.2",
|
||||
"version": "1.1.0",
|
||||
"packageManager": "pnpm@10.12.3",
|
||||
"description": "Api SDK for Papra, the document archiving platform.",
|
||||
"author": "Corentin Thomasset <corentinth@proton.me> (https://corentin.tech)",
|
||||
|
||||
@@ -1,5 +1,16 @@
|
||||
# @papra/cli
|
||||
|
||||
## 0.1.0
|
||||
|
||||
### Minor Changes
|
||||
|
||||
- [#464](https://github.com/papra-hq/papra/pull/464) [`14bc2b8`](https://github.com/papra-hq/papra/commit/14bc2b8f8d0d6605062f37188e7c57bbc61b2c1a) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Ditched CommonJs build for packages
|
||||
|
||||
### Patch Changes
|
||||
|
||||
- Updated dependencies [[`14bc2b8`](https://github.com/papra-hq/papra/commit/14bc2b8f8d0d6605062f37188e7c57bbc61b2c1a)]:
|
||||
- @papra/api-sdk@1.1.0
|
||||
|
||||
## 0.0.2
|
||||
|
||||
### Patch Changes
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "@papra/cli",
|
||||
"type": "module",
|
||||
"version": "0.0.2",
|
||||
"version": "0.1.0",
|
||||
"packageManager": "pnpm@10.12.3",
|
||||
"description": "Command line interface for Papra, the document archiving platform.",
|
||||
"author": "Corentin Thomasset <corentinth@proton.me> (https://corentin.tech)",
|
||||
|
||||
@@ -1,5 +1,11 @@
|
||||
# @papra/lecture
|
||||
|
||||
## 0.2.0
|
||||
|
||||
### Minor Changes
|
||||
|
||||
- [#464](https://github.com/papra-hq/papra/pull/464) [`14bc2b8`](https://github.com/papra-hq/papra/commit/14bc2b8f8d0d6605062f37188e7c57bbc61b2c1a) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Ditched CommonJs build for packages
|
||||
|
||||
## 0.1.0
|
||||
|
||||
### Minor Changes
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "@papra/lecture",
|
||||
"type": "module",
|
||||
"version": "0.1.0",
|
||||
"version": "0.2.0",
|
||||
"packageManager": "pnpm@10.12.3",
|
||||
"description": "A simple library to extract text from files",
|
||||
"author": "Corentin Thomasset <corentinth@proton.me> (https://corentin.tech)",
|
||||
|
||||
@@ -1,5 +1,11 @@
|
||||
# @papra/webhooks
|
||||
|
||||
## 0.3.0
|
||||
|
||||
### Minor Changes
|
||||
|
||||
- [#464](https://github.com/papra-hq/papra/pull/464) [`14bc2b8`](https://github.com/papra-hq/papra/commit/14bc2b8f8d0d6605062f37188e7c57bbc61b2c1a) Thanks [@CorentinTh](https://github.com/CorentinTh)! - Ditched CommonJs build for packages
|
||||
|
||||
## 0.2.0
|
||||
|
||||
### Minor Changes
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "@papra/webhooks",
|
||||
"type": "module",
|
||||
"version": "0.2.0",
|
||||
"version": "0.3.0",
|
||||
"packageManager": "pnpm@10.12.3",
|
||||
"description": "Webhooks helper library for Papra, the document archiving platform.",
|
||||
"author": "Corentin Thomasset <corentinth@proton.me> (https://corentin.tech)",
|
||||
|
||||
Reference in New Issue
Block a user