readur/docs/configuration-reference.md

Configuration Reference

Complete Configuration Options for Readur

This document provides a comprehensive reference for all configuration options available in Readur, including the new S3 storage backend and per-user watch directories introduced in version 2.5.4.

Environment Variables

Core Configuration

Variable	Type	Default	Description
DATABASE_URL	String	postgresql://readur:readur@localhost/readur	PostgreSQL connection string
SERVER_ADDRESS	String	0.0.0.0:8000	Server bind address (host:port)
SERVER_HOST	String	0.0.0.0	Server host (used if SERVER_ADDRESS not set)
SERVER_PORT	String	8000	Server port (used if SERVER_ADDRESS not set)
JWT_SECRET	String	your-secret-key	Secret key for JWT token generation (CHANGE IN PRODUCTION)
UPLOAD_PATH	String	./uploads	Local directory for temporary file uploads
ALLOWED_FILE_TYPES	String	pdf,txt,doc,docx,png,jpg,jpeg	Comma-separated list of allowed file extensions

S3 Storage Configuration

Variable	Type	Default	Description
S3_ENABLED	Boolean	false	Enable S3 storage backend
S3_BUCKET_NAME	String	-	S3 bucket name (required when S3_ENABLED=true)
S3_ACCESS_KEY_ID	String	-	AWS Access Key ID (required when S3_ENABLED=true)
S3_SECRET_ACCESS_KEY	String	-	AWS Secret Access Key (required when S3_ENABLED=true)
S3_REGION	String	us-east-1	AWS region for S3 bucket
S3_ENDPOINT	String	-	Custom S3 endpoint URL (for S3-compatible services)

Watch Directory Configuration

Variable	Type	Default	Description
WATCH_FOLDER	String	./watch	Global watch directory for file ingestion
USER_WATCH_BASE_DIR	String	./user_watch	Base directory for per-user watch folders
ENABLE_PER_USER_WATCH	Boolean	false	Enable per-user watch directories feature
WATCH_INTERVAL_SECONDS	Integer	60	Interval between watch folder scans
FILE_STABILITY_CHECK_MS	Integer	2000	Time to wait for file size stability
MAX_FILE_AGE_HOURS	Integer	24	Maximum age of files to process

OCR Configuration

Variable	Type	Default	Description
OCR_LANGUAGE	String	eng	Tesseract language code for OCR
CONCURRENT_OCR_JOBS	Integer	4	Number of concurrent OCR jobs
OCR_TIMEOUT_SECONDS	Integer	300	Timeout for OCR processing per document
MAX_FILE_SIZE_MB	Integer	50	Maximum file size for processing

Performance Configuration

Variable	Type	Default	Description
MEMORY_LIMIT_MB	Integer	512	Memory limit for processing operations
CPU_PRIORITY	String	normal	CPU priority (low, normal, high)

OIDC Authentication Configuration

Variable	Type	Default	Description
OIDC_ENABLED	Boolean	false	Enable OpenID Connect authentication
OIDC_CLIENT_ID	String	-	OIDC client ID
OIDC_CLIENT_SECRET	String	-	OIDC client secret
OIDC_ISSUER_URL	String	-	OIDC issuer URL
OIDC_REDIRECT_URI	String	-	OIDC redirect URI

Configuration Examples

Basic Local Storage Setup

# .env file for local storage
DATABASE_URL=postgresql://readur:password@localhost/readur
SERVER_ADDRESS=0.0.0.0:8000
JWT_SECRET=your-secure-secret-key-change-this
UPLOAD_PATH=./uploads
WATCH_FOLDER=./watch
ALLOWED_FILE_TYPES=pdf,txt,doc,docx,png,jpg,jpeg,tiff,bmp
OCR_LANGUAGE=eng
CONCURRENT_OCR_JOBS=4

S3 Storage with AWS

# .env file for AWS S3
DATABASE_URL=postgresql://readur:password@localhost/readur
SERVER_ADDRESS=0.0.0.0:8000
JWT_SECRET=your-secure-secret-key-change-this

# S3 Configuration
S3_ENABLED=true
S3_BUCKET_NAME=readur-production
S3_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
S3_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
S3_REGION=us-west-2

# Still needed for temporary uploads
UPLOAD_PATH=./temp_uploads

S3 with MinIO

# .env file for MinIO
DATABASE_URL=postgresql://readur:password@localhost/readur
SERVER_ADDRESS=0.0.0.0:8000
JWT_SECRET=your-secure-secret-key-change-this

# MinIO S3 Configuration
S3_ENABLED=true
S3_BUCKET_NAME=readur-bucket
S3_ACCESS_KEY_ID=minioadmin
S3_SECRET_ACCESS_KEY=minioadmin
S3_REGION=us-east-1
S3_ENDPOINT=http://minio:9000

UPLOAD_PATH=./temp_uploads

Per-User Watch Directories

# .env file with per-user watch enabled
DATABASE_URL=postgresql://readur:password@localhost/readur
SERVER_ADDRESS=0.0.0.0:8000
JWT_SECRET=your-secure-secret-key-change-this

# Watch Directory Configuration
WATCH_FOLDER=./global_watch
USER_WATCH_BASE_DIR=/data/user_watches
ENABLE_PER_USER_WATCH=true
WATCH_INTERVAL_SECONDS=30
FILE_STABILITY_CHECK_MS=3000
MAX_FILE_AGE_HOURS=48

High-Performance Configuration

# .env file for high-performance setup
DATABASE_URL=postgresql://readur:password@db-server/readur
SERVER_ADDRESS=0.0.0.0:8000
JWT_SECRET=your-secure-secret-key-change-this

# S3 for scalable storage
S3_ENABLED=true
S3_BUCKET_NAME=readur-highperf
S3_ACCESS_KEY_ID=your-key
S3_SECRET_ACCESS_KEY=your-secret
S3_REGION=us-east-1

# Performance tuning
CONCURRENT_OCR_JOBS=8
OCR_TIMEOUT_SECONDS=600
MAX_FILE_SIZE_MB=200
MEMORY_LIMIT_MB=2048
CPU_PRIORITY=high

# Faster watch scanning
WATCH_INTERVAL_SECONDS=10
FILE_STABILITY_CHECK_MS=1000

OIDC with S3 Storage

# .env file for OIDC authentication with S3
DATABASE_URL=postgresql://readur:password@localhost/readur
SERVER_ADDRESS=0.0.0.0:8000
JWT_SECRET=your-secure-secret-key-change-this

# OIDC Configuration
OIDC_ENABLED=true
OIDC_CLIENT_ID=readur-client
OIDC_CLIENT_SECRET=your-oidc-secret
OIDC_ISSUER_URL=https://auth.example.com
OIDC_REDIRECT_URI=https://readur.example.com/api/auth/oidc/callback

# S3 Storage
S3_ENABLED=true
S3_BUCKET_NAME=readur-oidc
S3_ACCESS_KEY_ID=your-key
S3_SECRET_ACCESS_KEY=your-secret
S3_REGION=eu-west-1

Docker Configuration

Docker Compose with Environment File

version: '3.8'

services:
  readur:
    image: readur:latest
    env_file: .env
    ports:
      - "8000:8000"
    volumes:
      - ./uploads:/app/uploads
      - ./watch:/app/watch
      - ./user_watch:/app/user_watch
    depends_on:
      - postgres
      - minio

  postgres:
    image: postgres:15
    environment:
      POSTGRES_USER: readur
      POSTGRES_PASSWORD: password
      POSTGRES_DB: readur
    volumes:
      - postgres_data:/var/lib/postgresql/data

  minio:
    image: minio/minio:latest
    command: server /data --console-address ":9001"
    environment:
      MINIO_ROOT_USER: minioadmin
      MINIO_ROOT_PASSWORD: minioadmin
    ports:
      - "9000:9000"
      - "9001:9001"
    volumes:
      - minio_data:/data

volumes:
  postgres_data:
  minio_data:

Kubernetes ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  name: readur-config
data:
  DATABASE_URL: "postgresql://readur:password@postgres-service/readur"
  SERVER_ADDRESS: "0.0.0.0:8000"
  S3_ENABLED: "true"
  S3_BUCKET_NAME: "readur-k8s"
  S3_REGION: "us-east-1"
  ENABLE_PER_USER_WATCH: "true"
  USER_WATCH_BASE_DIR: "/data/user_watches"
  CONCURRENT_OCR_JOBS: "6"
  MAX_FILE_SIZE_MB: "100"

Configuration Validation

Required Variables

When S3 is enabled, the following variables are required:

• S3_BUCKET_NAME
• S3_ACCESS_KEY_ID
• S3_SECRET_ACCESS_KEY

When OIDC is enabled, the following variables are required:

• OIDC_CLIENT_ID
• OIDC_CLIENT_SECRET
• OIDC_ISSUER_URL
• OIDC_REDIRECT_URI

Validation Script

#!/bin/bash
# validate-config.sh

# Check required variables
check_var() {
    if [ -z "${!1}" ]; then
        echo "ERROR: $1 is not set"
        exit 1
    fi
}

# Load environment
source .env

# Always required
check_var DATABASE_URL
check_var JWT_SECRET

# Check S3 requirements
if [ "$S3_ENABLED" = "true" ]; then
    check_var S3_BUCKET_NAME
    check_var S3_ACCESS_KEY_ID
    check_var S3_SECRET_ACCESS_KEY
fi

# Check OIDC requirements
if [ "$OIDC_ENABLED" = "true" ]; then
    check_var OIDC_CLIENT_ID
    check_var OIDC_CLIENT_SECRET
    check_var OIDC_ISSUER_URL
    check_var OIDC_REDIRECT_URI
fi

echo "Configuration valid!"

Migration from Previous Versions

From 2.5.3 to 2.5.4

New configuration options in 2.5.4:

# New S3 storage options
S3_ENABLED=false
S3_BUCKET_NAME=
S3_ACCESS_KEY_ID=
S3_SECRET_ACCESS_KEY=
S3_REGION=us-east-1
S3_ENDPOINT=

# New per-user watch directories
USER_WATCH_BASE_DIR=./user_watch
ENABLE_PER_USER_WATCH=false

No changes required for existing installations unless you want to enable new features.

Troubleshooting Configuration

Common Issues

1. S3 Connection Failed

   • Verify S3_BUCKET_NAME exists
   • Check S3_ACCESS_KEY_ID and S3_SECRET_ACCESS_KEY are correct
   • Ensure S3_REGION matches bucket region
   • For S3-compatible services, verify S3_ENDPOINT is correct

2. Per-User Watch Not Working

   • Ensure ENABLE_PER_USER_WATCH=true
   • Verify USER_WATCH_BASE_DIR exists and is writable
   • Check directory permissions

3. JWT Authentication Failed

   • Ensure JWT_SECRET is consistent across restarts
   • Use a strong, unique secret in production

Debug Mode

Enable debug logging:

export RUST_LOG=debug
export RUST_BACKTRACE=1

Configuration Testing

Test S3 configuration:

aws s3 ls s3://$S3_BUCKET_NAME --profile readur-test

Test database connection:

psql $DATABASE_URL -c "SELECT version();"

Security Considerations

1. Never commit .env files to version control
2. Use strong, unique values for JWT_SECRET
3. Rotate S3 access keys regularly
4. Use IAM roles when running on AWS
5. Enable S3 bucket encryption
6. Restrict S3 bucket policies to minimum required permissions
7. Use HTTPS for S3_ENDPOINT when possible
8. Implement network security groups for database access