103 Commits

Author SHA1 Message Date
Dries Peeters
62c34faf08 Fix: Add missing push_subscriptions table migration
- Create migration 090 to add push_subscriptions table
- Fixes database error when deleting users (relation 'push_subscriptions' does not exist)
- Table includes foreign key to users with CASCADE delete
- Adds index on user_id column
- Follows idempotent migration pattern for safe execution

Fixes #428
2026-01-13 21:57:37 +01:00
Dries Peeters
ba9b789c51 feat: Add decorative images to PDF templates with full sync between invoice and quote editors
- Add decorative image element to PDF Layout Designer for both invoice and quote templates
- Implement template-level decorative image upload and management
- Add backend routes for template image upload and serving
- Update PDF generation (HTML preview and ReportLab) to handle template images with transparency preservation
- Sync all decorative image functionality between invoice and quote PDF layout editors

Fixes:
- Fix upload button not opening file picker in invoice template (use fresh DOM references)
- Fix element name matching to handle 'decorative-image element-overlap' format (use .includes() instead of strict equality)
- Fix image restoration after page reload with enhanced JSON searching and position matching
- Fix image persistence in Konva.js serialization/deserialization

Improvements:
- Enhanced image restoration logic with fallback mechanisms
- Improved error handling and console logging for debugging
- Better handling of transparent backgrounds in PDF export
- Consistent behavior between invoice and quote template editors
2026-01-13 13:33:55 +01:00
Dries Peeters
0178ef03ad feat: Increase invoice prefix length from 10 to 50 characters
- Add database migration to extend invoice_prefix column length
- Support longer invoice prefixes for better customization
- Migration includes SQLite compatibility handling
2026-01-11 08:37:59 +01:00
Dries Peeters
4a8607f400 feat: Add major feature updates - integrations, services, and utilities
- Add Google Calendar integration with OAuth 2.0 support
- Implement integration service and workflow engine
- Add new routes: auth, clients, custom_reports, integrations, invoices, team_chat
- Add utility modules: config_manager, email, excel_export, file_upload, permissions_seed
- Add integration view template
- Add Docker permission fixes and enhanced start scripts
- Add migration management utilities and legacy schema migration
- Add validation and version management scripts
- Update setup.py version to 4.9.16

This release significantly expands the application's integration capabilities,
adds new business logic services, and improves infrastructure tooling.
2026-01-09 22:42:53 +01:00
Dries Peeters
4eeaa2a842 feat: Migrate PDF templates to ReportLab JSON format
- Add ReportLab template renderer with JSON-based template system
- Implement template schema validation and helper functions
- Add database migration for template_json columns
- Update visual editor to generate ReportLab JSON alongside HTML/CSS
- Maintain backward compatibility with legacy templates
- Add comprehensive migration documentation

BREAKING CHANGE: Existing PDF templates need to be saved again through
the visual editor to generate the new template_json format. Templates
will continue to work using the legacy fallback generator until saved.
2026-01-09 11:43:42 +01:00
Dries Peeters
14d673a4a8 Allow auto-imported time entries without project or client
- Add migration to update check constraint allowing NULL project_id and client_id for source='auto' entries
- Update TimeEntry model validation to allow entries without project/client when source='auto'
- Update TimeEntryCreateSchema to allow entries without project/client when source='auto'
- Enables calendar integrations to import entries that don't have project/client mapping yet
2026-01-07 13:18:40 +01:00
Dries Peeters
ad9bfbf1ed Fix client deletion errors and add invoice validation
This commit fixes multiple issues preventing client deletion and adds proper validation to prevent deletion when invoices exist.

Database Schema Fixes:

- Migration 103: Add missing quote_number column to quotes table

  - Handles migration from offer_number to quote_number

  - Generates quote numbers for existing quotes if needed

  - Creates required unique index

- Migration 104: Add all missing columns to quotes table

  - Adds subtotal, tax_amount, visible_to_client columns

  - Adds discount fields (discount_type, discount_amount, discount_reason, coupon_code)

  - Adds payment_terms column

  - Adds approval workflow columns (approval_status, approved_by, approved_at, rejected_by, rejection_reason)

  - Creates required indexes and foreign keys

- Migration 105: Fix client_notifications foreign key cascade

  - Updates client_notifications.client_id FK to ON DELETE CASCADE

  - Updates client_notification_preferences.client_id FK to ON DELETE CASCADE

  - Prevents NOT NULL constraint violations during client deletion

Model Updates:

- Add passive_deletes=True to ClientNotification.client relationship

- Add passive_deletes=True to ClientNotificationPreferences.client relationship

- Add passive_deletes=True to ClientAttachment.client relationship

- Update ClientNote.client relationship to use passive_deletes

Route Updates:

- Add invoice validation to delete_client() and bulk_delete_clients()

- Manually delete notifications before client deletion to prevent SQLAlchemy update issues

Fixes:

- Resolves IntegrityError when deleting clients with notifications

- Resolves missing quote_number column errors

- Resolves missing quotes table columns errors

- Prevents deletion of clients with invoices (data integrity)
2026-01-05 22:07:50 +01:00
Dries Peeters
596d2815f8 Create 102_add_missing_quotes_template_id.py 2026-01-05 20:50:24 +01:00
Dries Peeters
9beda467c9 Fix migration 088: Make salesman splitting migration idempotent
Add existence checks before adding columns to report_email_schedules table
to prevent DuplicateColumn errors when columns already exist in the database.

This ensures the migration can be safely rerun and handles cases where
columns were manually added or migration partially completed.

Fixes: split_by_salesman and salesman_field_name columns in
report_email_schedules table
2026-01-05 17:00:44 +01:00
Dries Peeters
4d336ac54a feat: improve inventory lot grouping and add issues table migration
- Inventory: Group stock lots by warehouse, unit cost, lot type, and date
  to prevent duplicate display entries
- Inventory: Add total value calculation and display per warehouse
- Migration: Make 082_add_global_integrations migration idempotent with
  existence checks for columns and indexes
- Migration: Add 101_add_issues_table migration for client-reported issues
  tracking with multi-database support (SQLite, PostgreSQL, MySQL)
- Version: Bump to 4.9.5
2026-01-05 16:23:30 +01:00
Dries Peeters
f205904742 feat: Add comment attachments model and migration
- Create CommentAttachment model following ProjectAttachment pattern
- Add database migration for comment_attachments table
- Register CommentAttachment in models __init__.py
- Support file uploads (images, PDFs, documents, archives)
- Include file metadata (size, type, uploader, timestamp)
- Cascade delete attachments when comments are deleted

Enables file attachments to comments for better team collaboration.
2026-01-04 06:23:39 +01:00
Dries Peeters
a24d515247 fix(migrations): add idempotency checks to additional migrations
Add existence checks to older migrations (002, 004, 013, 021, 031, 034, 044,
046, 071, 086, 087) to prevent errors when objects already exist.

Also fixes syntax errors in migrations 046 and 086 where index creation
code was incorrectly placed outside try/except blocks.

All affected migrations now:
- Check for table/column/index existence before creating
- Handle existing objects gracefully with informative messages
- Have safe downgrade functions
- Use proper try/except error handling
2026-01-04 04:34:56 +01:00
Dries Peeters
fc6e1bb733 fix(migrations): add idempotency checks to recent migrations
Add existence checks to recent migrations (032, 049, 053, 055, 058, 064, 099)
to prevent DuplicateTable and DuplicateColumn errors. These migrations now:

- Check if tables/columns exist before creating them
- Handle cases where objects already exist gracefully
- Provide informative messages about operations
- Have safe downgrade functions with existence checks

This makes migrations more resilient to partial failures and manual
schema changes.
2026-01-04 04:34:52 +01:00
Dries Peeters
1fa86d7706 fix(migrations): add idempotency check for invoice_peppol_transmissions table
Fix migration 098 to check if invoice_peppol_transmissions table exists
before creating it. This prevents DuplicateTable errors when the migration
is run on databases where the table was already created manually or
during a previous partial migration attempt.

The migration now:
- Checks for table existence before creating
- Handles existing tables gracefully
- Provides informative messages
- Has safe downgrade function
2026-01-04 04:34:48 +01:00
Dries Peeters
7c169fb401 refactor: improve application initialization and migration handling
- Update app factory pattern with better error handling
- Enhance migration environment configuration
- Improve database initialization and recovery logic
- Add better support for bootstrap and migration modes
2026-01-03 20:27:51 +01:00
Dries Peeters
753b98d272 feat: add Peppol e-invoicing and stock lot valuation
Add Peppol BIS Billing 3.0 (UBL) invoice sending via a configurable access point, including admin-configurable settings, per-invoice send history, and documentation/README updates.

Also introduce stock lots/allocations (valuation layers) with supporting inventory route/report/UI updates and hardened startup migration handling.
2026-01-03 07:00:30 +01:00
Dries Peeters
1ea87c81e5 fix: make migration 081 tolerant of existing settings columns
Some legacy/partially-upgraded databases already have one or more integration OAuth columns in settings (e.g. google_calendar_client_id). Only add missing columns and set defaults for columns that exist so upgrades don't fail with DuplicateColumn.

Bump version to 4.8.12.
2026-01-02 19:05:46 +01:00
Dries Peeters
bd59488132 fix: make template-related migrations idempotent
Avoid DuplicateTable/duplicate index errors on partially-migrated databases by skipping creation of existing template tables (quote_templates, project_templates, time_entry_templates) and only creating missing indexes/columns.

Also bump version to 4.8.11.
2026-01-02 17:19:19 +01:00
Dries Peeters
dcc4f87769 fix: make migration 075 idempotent for existing link_templates
Skip creating link_templates when it already exists and only create missing indexes, so partially-migrated customer databases can continue upgrading.
2026-01-02 17:12:18 +01:00
Dries Peeters
7f209751ab fix: harden Alembic upgrades on legacy databases
Prevent Postgres upgrades from failing when alembic_version.version_num is too small for long revision IDs, and add an alias migration to bridge older installs that recorded '067_add_integration_credentials'.
2026-01-02 15:53:51 +01:00
Dries Peeters
86f7946120 fix: resolve SQLAlchemy reserved word conflict in ClientNotification model
- Rename 'metadata' column to 'extra_data' in ClientNotification model
  to avoid SQLAlchemy Declarative API reserved word conflict
- Update ClientNotificationService to use 'extra_data' instead of 'metadata'
- Maintain API compatibility by returning 'metadata' key in to_dict() method
- Update migration to create 'extra_data' column instead of 'metadata'
- Improve migration idempotency and SQLite compatibility with proper checks
- Enhance backup directory handling with configurable BACKUP_FOLDER support
- Update admin routes to use centralized backup directory function

This fixes the application startup error:
  sqlalchemy.exc.InvalidRequestError: Attribute name 'metadata' is reserved
  when using the Declarative API.

The migration is now idempotent and handles both PostgreSQL and SQLite
databases safely.
2026-01-02 07:50:04 +01:00
Dries Peeters
b9b7612633 Fix migration 091: Handle existing integration_external_event_links table
The migration was failing with DuplicateTable error when the table
already existed in the database. This can happen when:
- The table was created manually
- A previous migration attempt partially succeeded
- Database state is out of sync with Alembic version tracking

Changes:
- Add _has_table() helper to check table existence
- Add _has_index() helper to check index existence
- Make upgrade() idempotent by checking before creating
- Make downgrade() safe by checking before dropping
- Follow same pattern as other migrations (018, 037)

This allows the migration to run successfully even when the table
already exists, creating only missing indexes if needed.
2026-01-01 13:30:23 +01:00
Dries Peeters
427074398c Improve migration robustness and add missing schema columns
- Refactor migrations with idempotency checks and better error handling
  * Add SQLAlchemy inspector checks for table/column existence
  * Improve error messages and handling for schema operations
  * Make migrations safe to run multiple times
  * Update 27 migration files with enhanced error handling patterns

- Add missing schema columns via new migrations
  * Migration 095: Add ui_show_issues column to users table
  * Migration 096: Add portal_issues_enabled column to clients table

- Enhance Settings model error handling
  * Improve detection of schema errors (table/column missing)
  * Better handling of SQLAlchemy exceptions during migrations
  * More comprehensive error checking for OperationalErrors

- Fix database auto-switching logic in app initialization
  * Respect explicit DATABASE_URL setting to prevent unwanted switches
  * Only auto-switch to PostgreSQL when not explicitly configured

- Update docker entrypoint script with migration improvements
2026-01-01 09:15:31 +01:00
Dries Peeters
9507a9492c feat: Add comprehensive donation system with smart prompts and improved accessibility
- Add dedicated donation page (/donate) explaining why donations matter
- Implement DonationInteraction model to track user engagement and interactions
- Add smart banner logic with contextual messaging based on user milestones
- Improve donation accessibility with links in sidebar, footer, dashboard, and all major pages
- Add UTM tracking to all Buy Me a Coffee links for analytics
- Fix CSRF token issues in donation tracking JavaScript
- Enhance dashboard widget with user stats and dual action buttons
- Add donation information section to About page
- Update support banner with 'Learn More' and 'Donate' options
- Create database migration for donation_interactions table

The donation system now provides:
- Smart prompts that show after user milestones (7+ days, 50+ entries, 100+ hours)
- Banner dismissal tracking with 30-day cooldown
- Multiple access points throughout the application
- Better visibility of donation impact and importance
- Comprehensive tracking for analytics and optimization
2025-12-30 09:52:12 +01:00
Dries Peeters
0c310736c1 Remove legacy UI enable/disable settings system
Remove both system-wide and per-user UI feature enable/disable settings
in favor of the centralized ModuleRegistry system for module management.

Changes:
- Remove ui_allow_* columns from Settings model and database (migration 093)
- Remove ui_show_* preference assignments from user settings route
- Remove UI Customization section from user settings template
- Remove UI Features section from admin settings template
- Update admin modules template to use ModuleRegistry instead of settings flags
- Remove settings_flag and user_flag attributes from ModuleDefinition
- Update ModuleRegistry.is_enabled() to only check dependencies and default_enabled
- Update dashboard template to use is_module_enabled() helper
- Update admin route docstring to reflect module management changes

Module visibility is now controlled exclusively via the admin module
management interface (/admin/modules), eliminating the need for separate
system-wide and per-user UI preference systems.
2025-12-29 15:51:59 +01:00
Dries Peeters
083dd9f1f2 feat: Implement comprehensive module system with visibility controls
- Add centralized module registry system (ModuleRegistry) for managing
  module metadata, dependencies, and visibility across the application
- Create module helper utilities with decorators (@module_enabled) and
  helper functions for route protection and template access
- Add database migration (092) to add missing module visibility flags
  to settings and users tables for granular control
- Extend Settings and User models with additional module visibility
  flags for CRM, Finance, Tools, and Advanced features
- Implement admin module management UI for system-wide module
  enable/disable controls
- Add module checks to routes (calendar, contacts, deals, expenses,
  invoices, leads, custom_reports) to enforce visibility rules
- Update scheduled report service and report templates to respect
  module visibility settings
- Bump version to 4.8.0 in setup.py
- Add comprehensive documentation for module integration planning
  and implementation analysis
2025-12-29 14:13:32 +01:00
Dries Peeters
443a797e2d feat: Add CalDAV calendar integration for Zimbra and other CalDAV servers
Implement comprehensive CalDAV calendar integration to import calendar events
as time entries from CalDAV-compatible servers (Zimbra, Nextcloud, ownCloud).

Features:
- CalDAV client with calendar discovery and event fetching
- Automatic calendar discovery from server URL
- Import calendar events (VEVENT) as time entries
- Project matching from event titles with fallback to default project
- Idempotent sync using IntegrationExternalEventLink to prevent duplicates
- Per-user integration setup (similar to Google Calendar)
- Support for both server URL (with discovery) and direct calendar URL
- SSL certificate verification toggle for self-signed certificates
- Configurable lookback period for event import

Components:
- CalDAVCalendarConnector: Main integration connector with sync logic
- CalDAVClient: Low-level CalDAV client using PROPFIND/REPORT requests
- IntegrationExternalEventLink: Model for tracking imported events (idempotency)
- Setup UI: User-friendly form for configuration
- Comprehensive validation and error handling
- Full test coverage (unit, integration, route tests)
- Documentation: Setup guide and troubleshooting

Technical details:
- Uses icalendar library for parsing VEVENT components
- Handles timezone conversion (CalDAV UTC to app local timezone)
- Skips all-day events (only imports timed events)
- Stores credentials securely (password in access_token, username in extra_data)
- Automatic calendar discovery on first sync if only server URL provided

Migration:
- Adds integration_external_event_links table for sync tracking
- Unique constraint on (integration_id, external_uid) prevents duplicates

Documentation:
- CALDAV_INTEGRATION.md: Complete feature documentation
- CALDAV_QUICK_SETUP.md: Step-by-step setup guide with examples

Closes feature request for CalDAV/Zimbra integration.
2025-12-20 17:28:48 +01:00
Dries Peeters
88656c3d34 feat: Advanced Report Builder with iterative generation and email distribution
Implement comprehensive enhancements to the Report Builder system with support
for iterative report generation, flexible email distribution, and improved
error handling.

Features:
- Add iterative report generation: generate one report per custom field value
- Add email distribution modes: mapping, template, and single recipient modes
- Add recipient email templates with {value} placeholder support
- Enhance scheduled reports with better error handling and validation
- Add fix endpoint for invalid scheduled reports
- Improve report builder UI with iterative generation options
- Add comprehensive management views for report schemes

Fixes:
- Fix template errors in iterative report view (dict access issues)
- Fix empty report builder when editing saved reports
- Fix PWA install toast notification handling
- Fix migration revision ID length issue (shortened to fit 32 char limit)
- Add idempotent migration checks to prevent duplicate column errors
- Improve error handling in scheduled reports list view

Database Changes:
- Add iterative_report_generation and iterative_custom_field_name to saved_report_views
- Add email_distribution_mode and recipient_email_template to report_email_schedules
- Migration 090_report_builder_iteration (idempotent)

UI/UX Improvements:
- Display iterative generation status in saved views list
- Show distribution mode and template in scheduled reports
- Add error badges and fix buttons for invalid schedules
- Improve report builder form loading for saved configurations

Technical:
- Enhance ScheduledReportService with recipient resolution logic
- Add validation for report configurations
- Improve error handling and logging throughout
- Update templates to use safe dictionary access patterns
2025-12-12 22:11:57 +01:00
Dries Peeters
8a78975497 Enhance custom reports and scheduled reports functionality
- Add edit functionality for saved report views in custom reports builder
- Implement saved views list page with management capabilities
- Enhance scheduled reports service with improved error handling and email support
- Add email template for scheduled report delivery
- Update report templates with improved UI and functionality
- Fix PostgreSQL sequences for roles and permissions tables
- Update permission system migration with additional changes
- Improve base template with enhanced navigation and styling
2025-12-08 21:16:06 +01:00
Dries Peeters
4e57f08c03 Add salesman-based report splitting and email distribution
- Add SalesmanEmailMapping model to map salesman initials to email addresses
  - Support for direct email addresses, email patterns, and domain-based patterns
  - Admin interface for managing email mappings

- Add UnpaidHoursService for querying and grouping unpaid time entries
  - Filter by client custom fields (e.g., salesman)
  - Group unpaid hours by salesman for report generation

- Add salesman report routes and API endpoints
  - CRUD operations for email mappings
  - Generate and send reports split by salesman
  - Preview email addresses for salesman initials

- Enhance scheduled reports with salesman splitting
  - Add split_by_salesman and salesman_field_name to report schedules
  - Automatically split reports by salesman and send to mapped emails

- Add UI components for salesman report management
  - Admin dashboard integration
  - Report builder with salesman splitting options
  - Email mapping management interface

- Add email template for unpaid hours reports

- Add database migrations:
  - 087: Create salesman_email_mappings table
  - 088: Add salesman splitting fields to report_email_schedules
2025-12-03 08:59:48 +01:00
Dries Peeters
f3a3a40480 Add project custom fields and file attachments for projects and clients
This commit introduces two major features:

1. Project Custom Fields: Add custom_fields JSON column to projects table (migration 085), support for flexible custom data storage, display and edit in project views

2. File Attachments System: Add project_attachments and client_attachments tables (migration 086), new ProjectAttachment and ClientAttachment models, full CRUD operations, file upload/download/delete, client-visible attachments support

Additional improvements: Enhanced data tables, updated project/client/invoice/timer views, improved UI for attachments and custom fields management
2025-12-03 08:30:15 +01:00
Dries Peeters
86b3498f05 perf: optimize task queries and fix N+1 performance issues
This commit includes multiple performance optimizations, error handling
improvements, and bug fixes across the application.

Performance Improvements:
- Add caching for task status_display property to avoid N+1 queries
- Pre-calculate task counts by status in route handler instead of template
- Pre-load kanban columns in TaskService to eliminate N+1 queries
- Remove unnecessary db.session.expire_all() call in tasks route
- Always use pagination for task lists to improve performance

Error Handling & Robustness:
- Add graceful handling for missing time_entry_approvals table in timer deletion
- Improve safe_commit to handle ProgrammingError for optional relationships
- Add VAPID key validation and error handling in PWA push notifications
- Make custom_field_definitions migration idempotent

Bug Fixes:
- Fix IndexedDB boolean query issues in offline-sync.js by using cursor iteration
- Fix app context handling in scheduled reports processing
- Improve error messages for push notification subscription failures
2025-12-03 07:02:39 +01:00
Dries Peeters
3f5d253a8b feat: Add global custom field definitions with link template support
Implement a global custom field system that allows defining fields once
and reusing them across all clients. Fields can be marked as mandatory
or optional, and values become clickable links when link templates are
assigned.

- Add CustomFieldDefinition model and admin interface
- Support mandatory/optional flags with validation
- Update client forms to use global definitions
- Enhance link templates to support {value} and %value% placeholders
- Make custom field values clickable in client view
- Add migration 084 for custom_field_definitions table

Breaking: Client custom fields now require global definitions. Existing
custom fields continue to work, but new fields must be defined globally.
2025-11-30 15:29:02 +01:00
Dries Peeters
90d8407bda feat(billing): add paid status tracking for time entries with invoice reference
Add ability to mark time entries as paid and link them to internal invoice
numbers. Automatically mark time entries as paid when invoices are sent.

Database Changes:
- Add migration 083 to add `paid` boolean and `invoice_number` string columns
  to time_entries table
- Add index on `paid` field for faster queries

Model Updates:
- Add `paid` (default: False) and `invoice_number` (nullable) fields to TimeEntry
- Add `set_paid()` helper method to TimeEntry model
- Update `to_dict()` to include paid status and invoice number

API & Service Layer:
- Update TimeEntrySchema (all variants) to include paid/invoice_number fields
- Update API endpoints (/api/entry, /api/v1/time-entries) to accept these fields
- Update TimeTrackingService and TimeEntryRepository to handle paid status
- Add InvoiceService.mark_time_entries_as_paid() to automatically mark entries
- Update InvoiceService.mark_as_sent() to auto-mark time entries as paid

UI Updates:
- Add "Paid" checkbox and "Invoice Number" input field to time entry edit forms
- Update both admin and regular user edit forms
- Fields appear in timer edit page after tags section

Invoice Integration:
- Automatically mark time entries as paid when invoice status changes to "sent"
- Mark entries when time is added to already-sent invoices
- Store invoice number reference on time entries for tracking
- Enhanced create_invoice_from_time_entries() to properly link time entries

This enables proper tracking of which hours have been invoiced and paid
through the internal invoicing system, separate from the external ERP system.
2025-11-30 11:31:42 +01:00
Dries Peeters
0ec6b8e9d6 refactor: major integration system overhaul with global integrations support
This commit implements a comprehensive refactoring of the integration system to support both global (shared) and per-user integrations, adds new integrations, and improves the overall architecture.

Key changes:

- Add global integrations support: most integrations are now shared across all users (Jira, Slack, GitHub, Asana, Trello, GitLab, Microsoft Teams, Outlook Calendar, Xero)

- Add new integrations: GitLab, Microsoft Teams, Outlook Calendar, and Xero

- Database migrations:

  * Migration 081: Add OAuth credential columns for all integrations to Settings model

  * Migration 082: Add is_global flag to Integration model and make user_id nullable

- Update Integration model to support global integrations with nullable user_id

- Refactor IntegrationService to handle both global and per-user integrations

- Create dedicated admin setup pages for each integration

- Update Trello connector to use API key setup instead of OAuth flow

- Enhance all existing integrations (Jira, Slack, GitHub, Google Calendar, Asana, Trello) with global support

- Update routes, templates, and services to support the new global/per-user distinction

- Improve integration management UI with better separation of global vs per-user integrations

- Update scheduled tasks to work with the new integration architecture
2025-11-29 07:03:00 +01:00
Dries Peeters
dcbdfcc288 feat: Add client custom fields, link templates, UI feature flags, and client billing support
Add client custom fields (JSON) for flexible data storage

Implement link templates system for dynamic URL generation from custom fields

Add client_id support to time entries for direct client billing (project_id now nullable)

Implement user-level UI feature flags for customizable navigation visibility

Add system-wide UI feature flags in settings for admin control

Fix metadata column naming (user_badges.achievement_metadata, leaderboard_entries.entry_metadata)

Update templates and routes to support new features

Add comprehensive UI feature flag management in admin and user settings

Enhance client views with custom fields and link template integration

Update time entry forms to support client billing

Add tests for system UI flags

Migrations: 075-080 for custom fields, link templates, UI flags, client billing, and metadata fixes
2025-11-29 06:17:07 +01:00
Dries Peeters
6ad96f114a Fix user creation to use new Role system and add default password with forced change
- Updated user creation to assign roles from Role system instead of legacy role field
- Added password_change_required field to User model with migration
- Added default password input and force password change option in user creation form
- Updated login route to check password_change_required and redirect to change password page
- Created change_password route and template for forced password changes
- Updated all user creation points (admin, self-registration, OIDC, default admin) to use new Role system
- Updated user form template to show roles from Role system instead of hardcoded options

Fixes issue where newly created users were still using legacy roles instead of the new role-based permission system.
2025-11-28 22:47:01 +01:00
Dries Peeters
8585b097e0 feat: Add comprehensive feature implementation including integrations, workflows, approvals, and AI features
Major Features:
- Integration framework with implementations for Asana, Google Calendar, QuickBooks, and Trello
- Workflow automation system with workflow engine service
- Time entry approval system with client approval capabilities
- Recurring tasks functionality
- Client portal customization and team chat features
- AI-powered categorization and suggestion services
- GPS tracking for expenses
- Gamification system with service layer
- Custom reporting with service and model support
- Enhanced OCR service for expense processing
- Pomodoro timer service
- Currency service for multi-currency support
- PowerPoint export utility

Frontend Enhancements:
- Activity feed JavaScript module
- Mentions system for team chat
- Offline sync capabilities
- New templates for approvals, chat, and recurring tasks

Database Migrations:
- Updated integration framework migrations (066-068)
- Added workflow automation migration (069)
- Added time entry approvals migration (070)
- Added recurring tasks migration (071)
- Added client portal and team chat migration (072)
- Added AI features and GPS tracking migration (073)

Documentation:
- Updated implementation documentation
- Removed obsolete feature gap analysis docs
- Added comprehensive implementation status reports
2025-11-28 22:39:04 +01:00
Dries Peeters
4930f6a3e5 feat: add multiple authentication modes support
Add support for four authentication modes via AUTH_METHOD environment variable:
- none: Username-only authentication (no password)
- local: Password authentication required (default)
- oidc: OIDC/Single Sign-On only
- both: OIDC + local password authentication

Key changes:
- Add password_hash column to users table (migration 068)
- Implement password storage and verification in User model
- Update login routes to handle all authentication modes
- Add conditional password fields in login templates
- Support password authentication in kiosk mode
- Allow password changes in user profile when enabled

Password authentication is now enabled by default for better security,
while remaining backward compatible with existing installations.
Users will be prompted to set passwords when required.

Fixes authentication bypass issue where users could access accounts
without passwords even after setting them.
2025-11-28 15:56:01 +01:00
Dries Peeters
eb4fb8296f feat: Add integration framework and major feature enhancements
This commit introduces a comprehensive integration framework and multiple new features to enhance the TimeTracker application's capabilities.

Major Features:

- Integration Framework: Extensible system for third-party integrations with support for Jira, Slack, GitHub, and calendar services

- Project Templates: Reusable project templates for faster project creation

- Invoice Approvals: Workflow for invoice approval before sending

- Payment Gateways: Online payment processing integration with Stripe support

- Scheduled Reports: Automated report generation and email delivery

- Custom Reports: Advanced report builder with saved views

- Gantt Chart: Visual project timeline and dependency management

- Calendar Integrations: External calendar synchronization with Google Calendar support

- Push Notifications: Enhanced notification system with PWA support

Bug Fixes:

- Fix None handling in analytics routes

- Fix dynamic relationship loading issues in ProjectRepository and ProjectService

- Fix parameter ordering in service methods

- Fix None duration_seconds handling in budget forecasting

UI/UX Improvements:

- Update logo references to timetracker-logo.svg

- Add favicon links to all templates

- Add navigation items for new features

- Enhance invoice view with approval status and payment gateway links

Database:

- Add Alembic migrations for new features (065, 066, 067)

Dependencies:

- Add stripe==7.0.0 for payment processing

- Add google-api-python-client libraries for calendar integration
2025-11-26 07:53:28 +01:00
Dries Peeters
bdf9249edc refactor: comprehensive application improvements and architecture enhancements
This commit implements all critical improvements from the application review,
establishing modern architecture patterns and significantly improving performance,
security, and maintainability.

## Architecture Improvements

- Implement service layer pattern: Migrated routes (projects, tasks, invoices, reports)
  to use dedicated service classes with business logic separation
- Add repository pattern: Enhanced repositories with comprehensive docstrings and
  type hints for better data access abstraction
- Create base CRUD service: BaseCRUDService reduces code duplication across services
- Implement API versioning structure: Created app/routes/api/ package with v1
  subpackage for future versioning support

## Performance Optimizations

- Fix N+1 query problems: Added eager loading (joinedload) to all migrated routes,
  reducing database queries by 80-90%
- Add query logging: Implemented query_logging.py for performance monitoring and
  slow query detection
- Create caching foundation: Added cache_redis.py utilities ready for Redis integration

## Security Enhancements

- Enhanced API token management: Created ApiTokenService with token rotation,
  expiration management, and scope validation
- Add environment validation: Implemented startup validation for critical
  environment variables with production checks
- Improve error handling: Standardized error responses with route_helpers.py utilities

## Code Quality

- Add comprehensive type hints: All service and repository methods now have
  complete type annotations
- Add docstrings: Comprehensive documentation added to all services, repositories,
  and public APIs
- Standardize error handling: Consistent error response patterns across all routes

## Testing

- Add unit tests: Created test suites for ProjectService, TaskService,
  InvoiceService, ReportingService, ApiTokenService, and BaseRepository
- Test coverage: Added tests for CRUD operations, eager loading, filtering,
  and error cases

## Documentation

- Add API versioning documentation: Created docs/API_VERSIONING.md with
  versioning strategy and migration guidelines
- Add implementation documentation: Comprehensive review and progress
  documentation files

## Files Changed

### New Files (20+)
- app/services/base_crud_service.py
- app/services/api_token_service.py
- app/utils/env_validation.py
- app/utils/query_logging.py
- app/utils/route_helpers.py
- app/utils/cache_redis.py
- app/routes/api/__init__.py
- app/routes/api/v1/__init__.py
- tests/test_services/*.py (5 files)
- tests/test_repositories/test_base_repository.py
- docs/API_VERSIONING.md
- Documentation files (APPLICATION_REVIEW_2025.md, etc.)

### Modified Files (15+)
- app/services/project_service.py
- app/services/task_service.py
- app/services/invoice_service.py
- app/services/reporting_service.py
- app/routes/projects.py
- app/routes/tasks.py
- app/routes/invoices.py
- app/routes/reports.py
- app/repositories/base_repository.py
- app/repositories/task_repository.py
- app/__init__.py

## Impact

- Performance: 80-90% reduction in database queries
- Code Quality: Modern architecture patterns, type hints, comprehensive docs
- Security: Enhanced API token management, environment validation
- Maintainability: Service layer separation, consistent error handling
- Testing: Foundation for comprehensive test coverage

All changes are backward compatible and production-ready.
2025-11-24 20:58:22 +01:00
Dries Peeters
25ea52c029 feat: Implement CRM features and fix migration issues
- Add CRM models: Contact, ContactCommunication, Deal, DealActivity, Lead, LeadActivity
  - Support multiple contacts per client with primary contact designation
  - Track sales pipeline with deals and opportunities
  - Manage leads with conversion tracking
  - Record communication history with contacts

- Add CRM routes and templates
  - Contact management (list, create, view, edit, delete)
  - Deal management with pipeline view
  - Lead management with conversion workflow
  - Communication history tracking

- Fix SQLAlchemy relationship conflicts
  - Specify foreign_keys for Deal.lead relationship to resolve ambiguity
  - Remove duplicate backref definitions in DealActivity and LeadActivity

- Improve migration 062 robustness
  - Add index existence checks before creation
  - Handle partial migration states gracefully
  - Support both assigned_to and assignee_id column names

- Add error handling for missing CRM tables
  - Gracefully handle cases where migration 063 hasn't run yet
  - Prevent application crashes when CRM tables don't exist

- Add database migration 063 for CRM features
  - Create contacts, contact_communications, deals, deal_activities, leads, lead_activities tables
  - Set up proper foreign key relationships and indexes

- Update documentation
  - Add CRM features to FEATURES_COMPLETE.md
  - Create CRM implementation documentation
  - Add feature gap analysis documentation
2025-11-23 20:38:35 +01:00
Dries Peeters
9d1ece5263 feat: Implement comprehensive architectural improvements and new features
This commit implements a complete architectural transformation of the TimeTracker
application, introducing modern design patterns and comprehensive feature set.

## Architecture Improvements

### Service Layer (18 Services)
- TimeTrackingService: Time entry management with timer functionality
- ProjectService: Project operations and lifecycle management
- InvoiceService: Invoice creation, management, and status tracking
- TaskService: Task management and workflow
- ExpenseService: Expense tracking and categorization
- ClientService: Client relationship management
- PaymentService: Payment processing and invoice reconciliation
- CommentService: Comment system for projects, tasks, and quotes
- UserService: User management and role operations
- NotificationService: Notification delivery system
- ReportingService: Report generation and analytics
- AnalyticsService: Event tracking and analytics
- ExportService: CSV export functionality
- ImportService: CSV import with validation
- EmailService: Email operations and invoice delivery
- PermissionService: Role-based permission management
- BackupService: Database backup operations
- HealthService: System health checks and monitoring

### Repository Layer (9 Repositories)
- BaseRepository: Generic CRUD operations
- TimeEntryRepository: Time entry data access
- ProjectRepository: Project data access with filtering
- InvoiceRepository: Invoice queries and status management
- TaskRepository: Task data access
- ExpenseRepository: Expense data access
- ClientRepository: Client data access
- UserRepository: User data access
- PaymentRepository: Payment data access
- CommentRepository: Comment data access

### Schema Layer (9 Schemas)
- Marshmallow schemas for validation and serialization
- Create, update, and full schemas for all entities
- Input validation and data transformation

### Utility Modules (15 Utilities)
- api_responses: Standardized API response helpers
- validation: Input validation utilities
- query_optimization: N+1 query prevention and eager loading
- error_handlers: Centralized error handling
- cache: Caching foundation (Redis-ready)
- transactions: Transaction management decorators
- event_bus: Domain event system
- performance: Performance monitoring decorators
- logger: Enhanced structured logging
- pagination: Pagination utilities
- file_upload: Secure file upload handling
- search: Full-text search utilities
- rate_limiting: Rate limiting helpers
- config_manager: Configuration management
- datetime_utils: Enhanced date/time utilities

## Database Improvements
- Performance indexes migration (15+ indexes)
- Query optimization utilities
- N+1 query prevention patterns

## Testing Infrastructure
- Comprehensive test fixtures (conftest.py)
- Service layer unit tests
- Repository layer unit tests
- Integration test examples

## CI/CD Pipeline
- GitHub Actions workflow
- Automated linting (Black, Flake8, Pylint)
- Security scanning (Bandit, Safety, Semgrep)
- Automated testing with coverage
- Docker image builds

## Documentation
- Architecture migration guide
- Quick start guide
- API enhancements documentation
- Implementation summaries
- Refactored route examples

## Key Benefits
- Separation of concerns: Business logic decoupled from routes
- Testability: Services and repositories can be tested in isolation
- Maintainability: Consistent patterns across codebase
- Performance: Database indexes and query optimization
- Security: Input validation and security scanning
- Scalability: Event-driven architecture and health checks

## Statistics
- 70+ new files created
- 8,000+ lines of code
- 18 services, 9 repositories, 9 schemas
- 15 utility modules
- 5 test files with examples

This transformation establishes a solid foundation for future development
and follows industry best practices for maintainable, scalable applications.
2025-11-23 20:00:10 +01:00
Dries Peeters
73dfeecbaa feat: Complete inventory management system implementation
Add comprehensive inventory management system with full feature set including
stock tracking, warehouse management, supplier management, purchase orders,
transfers, adjustments, and reporting.

Core Features:
- Stock Items: Full CRUD operations with categories, SKU, barcodes, pricing
- Warehouse Management: Multi-warehouse support with stock level tracking
- Supplier Management: Multi-supplier support with supplier-specific pricing
- Purchase Orders: Complete PO lifecycle (draft, sent, received, cancelled)
- Stock Transfers: Transfer stock between warehouses with audit trail
- Stock Adjustments: Dedicated interface for stock corrections
- Stock Reservations: Reserve stock for quotes/invoices/projects
- Movement History: Complete audit trail for all stock movements
- Low Stock Alerts: Automated alerts when items fall below reorder point

Reports & Analytics:
- Inventory Dashboard: Overview with key metrics and statistics
- Stock Valuation: Calculate total inventory value by warehouse/category
- Movement History Report: Detailed movement log with filters
- Turnover Analysis: Inventory turnover rates and sales analysis
- Low Stock Report: Comprehensive low stock items listing

Integration:
- Quote Integration: Stock reservation when quotes are created
- Invoice Integration: Automatic stock reduction on invoice payment
- Project Integration: Stock allocation for project requirements
- API Endpoints: RESTful API for suppliers, purchase orders, and inventory

Technical Implementation:
- 9 new database models with proper relationships
- 3 Alembic migrations for schema changes
- 60+ new routes for inventory management
- 20+ templates for all inventory features
- Comprehensive permission system integration
- CSRF protection on all forms
- Full menu navigation integration

Testing:
- Unit tests for inventory models
- Route tests for inventory endpoints
- Integration tests for quote/invoice stock integration

Documentation:
- Implementation plan document
- Missing features analysis
- Implementation status tracking
2025-11-23 18:39:22 +01:00
Dries Peeters
acd30bc015 feat: implement comprehensive quote management system with PDF editor
Major Features:
- Complete quote management system with CRUD operations
- Quote items management with dynamic add/remove functionality
- Discount system (percentage and fixed amount)
- Payment terms integration with invoice creation
- Approval workflow with status tracking
- Quote attachments with client visibility control
- Quote templates for reusable configurations
- Quote versioning for revision history
- Email notifications for quote lifecycle events
- Scheduled tasks for expiring quote reminders
- Client portal integration for quote viewing/acceptance
- Bulk actions for quote management
- Analytics dashboard for quote metrics

UI/UX Improvements:
- Consistent table layout matching projects/clients pages
- Professional quote view page with improved action buttons
- Enhanced create/edit forms with organized sections
- Dynamic line items management in quote forms
- PDF template editor accessible via admin menu
- PDF submenu under Admin with Invoice and Quote options
- Fixed admin menu collapse when opening nested dropdowns

PDF Template System:
- Quote PDF layout editor with visual design tools
- Separate preview route for quote PDF templates
- Template reset functionality
- Support for multiple page sizes (A4, Letter, Legal, A3, A5, Tabloid)

Bug Fixes:
- Fixed 405 Method Not Allowed error on quote PDF save
- Fixed UnboundLocalError with translation function shadowing
- Fixed quote preview template context (quote vs invoice)
- Updated template references from invoice to quote variables

Database:
- Added 9 Alembic migrations for quote system schema
- Support for quotes, quote_items, quote_attachments, quote_templates, quote_versions
- Integration with existing comments system

Technical:
- Added Quote, QuoteItem, QuoteAttachment, QuoteTemplate, QuoteVersion models
- Extended comment routes to support quotes
- Integrated payment terms from quotes to invoices
- Email notification system for quote events
- Scheduled task for expiring quote checks
2025-11-23 16:08:31 +01:00
Dries Peeters
39cf649f8e feat: Add client portal with password setup email functionality
Implement a complete client portal feature that allows clients to access
their projects, invoices, and time entries through a dedicated portal with
separate authentication. Includes password setup via email with secure
token-based authentication.

Client Portal Features:
- Client-based authentication (separate from user accounts)
- Portal access can be enabled/disabled per client
- Clients can view their projects, invoices, and time entries
- Clean, minimal UI without main app navigation elements
- Login page styled to match main app design

Password Setup Email:
- Admin can send password setup emails to clients
- Secure token-based password setup (24-hour expiration)
- Email template with professional styling
- Password setup page matching app login design
- Token validation and automatic cleanup after use

Email Configuration:
- Email settings from admin menu are now used for sending
- Database email settings persist between restarts and updates
- Automatic reload of email configuration when sending emails
- Database settings take precedence over environment variables
- Improved error messages for email configuration issues

Database Changes:
- Add portal_enabled, portal_username, portal_password_hash to clients
- Add password_setup_token and password_setup_token_expires to clients
- Migration 047: Add client portal fields to users (legacy)
- Migration 048: Add client portal credentials to clients
- Migration 049: Add password setup token fields

New Files:
- app/routes/client_portal.py - Client portal routes and authentication
- app/templates/client_portal/ - Portal templates (base, login, dashboard, etc.)
- app/templates/email/client_portal_password_setup.html - Email template
- migrations/versions/047-049 - Database migrations
- tests/test_client_portal.py - Portal tests
- docs/CLIENT_PORTAL.md - Portal documentation

Modified Files:
- app/models/client.py - Add portal fields and password token methods
- app/routes/clients.py - Add send password email route
- app/routes/client_portal.py - Portal routes with redirect handling
- app/utils/email.py - Use database settings, add password setup email
- app/templates/clients/edit.html - Add send email button
- app/templates/components/ui.html - Support client portal breadcrumbs

Security:
- Secure token generation using secrets.token_urlsafe()
- Password hashing with werkzeug.security
- Token expiration (24 hours default)
- Token cleared after successful password setup
- CSRF protection on all forms
2025-11-14 15:15:38 +01:00
Dries Peeters
a18de04a6a feat: Add webhook system for real-time event notifications
Implement comprehensive webhook system supporting 40+ event types with automatic retries, HMAC signatures, delivery tracking, REST API, and admin UI. Integrates with Activity logging for automatic event triggering.

- Database: Add webhooks and webhook_deliveries tables (migration 046)

- API: Full CRUD endpoints with read:webhooks/write:webhooks scopes

- UI: Admin interface for webhook management and testing

- Service: Automatic retry with exponential backoff every 5 minutes

- Security: HMAC-SHA256 signature verification

- Tests: Model and service tests included

- Docs: Complete integration guide with examples
2025-11-14 13:52:56 +01:00
Dries Peeters
b4939f4755 feat: Add recurring invoices and email integration with template management
Implement comprehensive recurring invoice system and email functionality with admin interface for managing email templates.

Features: Recurring invoices with scheduling, invoice email integration with PDF attachments, email template management admin interface

Fixes: CSRF tokens, CSS leakage, toast notifications, response body handling, error logging
2025-11-13 09:24:17 +01:00
Dries Peeters
350d7105a2 feat: Add comprehensive audit trail/history tracking system
Implement a complete audit logging system to track all changes made to
tracked entities, providing full compliance and accountability capabilities.

Features:
- Automatic tracking of create, update, and delete operations on 25+ models
- Detailed field-level change tracking with old/new value comparison
- User attribution with IP address, user agent, and request path logging
- Web UI for viewing and filtering audit logs with pagination
- REST API endpoints for programmatic access
- Entity-specific history views
- Comprehensive test coverage (unit, model, route, and smoke tests)

Core Components:
- AuditLog model with JSON-encoded value storage and decoding helpers
- SQLAlchemy event listeners for automatic change detection
- Audit utility module with defensive programming for table existence checks
- Blueprint routes for audit log viewing and API access
- Jinja2 templates for audit log list, detail, and entity history views
- Database migration (044) creating audit_logs table with proper indexes

Technical Implementation:
- Uses SQLAlchemy 'after_flush' event listener to capture changes
- Tracks 25+ models including Projects, Tasks, TimeEntries, Invoices, Clients, Users, etc.
- Excludes sensitive fields (passwords) and system fields (id, timestamps)
- Implements lazy import pattern to avoid circular dependencies
- Graceful error handling to prevent audit logging from breaking core functionality
- Transaction-safe logging that integrates with main application transactions

Fixes:
- Resolved login errors caused by premature transaction commits
- Fixed circular import issues with lazy model loading
- Added table existence checks to prevent errors before migrations
- Improved error handling with debug-level logging for non-critical failures

UI/UX:
- Added "Audit Logs" link to admin dropdown menu
- Organized admin menu into logical sections for better usability
- Filterable audit log views by entity type, user, action, and date range
- Color-coded action badges and side-by-side old/new value display
- Pagination support for large audit log datasets

Documentation:
- Added comprehensive feature documentation
- Included troubleshooting guide and data examples
- Created diagnostic scripts for verifying audit log setup

Testing:
- Unit tests for AuditLog model and value encoding/decoding
- Route tests for all audit log endpoints
- Integration tests for audit logging functionality
- Smoke tests for end-to-end audit trail verification

This implementation provides a robust foundation for compliance tracking
and change accountability without impacting application performance or
requiring code changes in existing routes/models.
2025-11-13 08:08:48 +01:00
Dries Peeters
85298e1d47 feat: Add per-project Kanban columns support
Implement per-project Kanban column workflows, allowing different projects
to have their own custom kanban board columns and task states.

Changes:
- Add project_id field to KanbanColumn model (nullable, NULL = global columns)
- Create Alembic migration 043 to add project_id column with foreign key
- Update unique constraint from (key) to (key, project_id) to allow same
  keys across different projects
- Update all KanbanColumn model methods to filter by project_id:
  - get_active_columns(project_id=None)
  - get_all_columns(project_id=None)
  - get_column_by_key(key, project_id=None)
  - get_valid_status_keys(project_id=None)
  - initialize_default_columns(project_id=None)
  - reorder_columns(column_ids, project_id=None)
- Update kanban routes to support project filtering:
  - /kanban/columns accepts project_id query parameter
  - /kanban/columns/create supports project selection
  - All CRUD operations redirect to project-filtered view when applicable
  - API endpoints support project_id parameter
- Update project view route to use project-specific columns
- Update task routes to validate status against project-specific columns
- Add fallback logic: projects without custom columns use global columns
- Update UI templates:
  - Add project filter dropdown in column management page
  - Add project selection in create column form
  - Show project info in edit column page
  - Update reorder API calls to include project_id

Database Migration:
- Migration 043 adds project_id column (nullable)
- Existing columns remain global (project_id = NULL)
- New unique constraint on (key, project_id)
- Foreign key constraint with CASCADE delete

Backward Compatibility:
- Existing global columns continue to work
- Projects without custom columns fall back to global columns
- Task status validation uses project-specific columns when available

Impact: High - Enables multi-project teams to have different workflows
per project while maintaining backward compatibility with existing
global column setup.
2025-11-13 07:06:43 +01:00