Commit Graph

123 Commits

Author SHA1 Message Date
Dries Peeters 6881e554ce fix: restore audit logging by using before_flush for updates/deletes
Audit logs were not recording any changes because after_flush runs
after SQL is emitted; by then session.new, session.dirty, and
session.deleted can be cleared and attribute history for updates is
often consumed, so the handler saw nothing to log.

Changes:
- Add receive_before_flush: process session.dirty (updates) and
  session.deleted (deletes) while history is still valid; stash
  session.new (creates) in session.info for after_flush.
- Simplify receive_after_flush: only handle pending creates from
  session.info (instances now have ids), then session.flush() so
  audit rows are in the same transaction.
- Register receive_before_flush for before_flush on Session,
  sessionmaker class, and SignallingSession.
- Make receive_before_flush accept (session, flush_context, instances)
  to match SQLAlchemy's before_flush signature.
- Remove db.session.flush() from AuditLog.log_change to avoid
  nested flush; rely on main flush or explicit flush in after_flush.
- check_audit_logging.py: use entity_type='TimeEntry' to match
  get_entity_type (model __class__.__name__).
- test_audit_logging: assert at least one AuditLog for create/update/
  delete; use test_client for create; fix update to merge then mutate.
2026-01-21 13:59:13 +01:00
Dries Peeters 2615fefa91 feat(gantt): project bar colors and Pickr color picker
- Add Project.color (hex) and migration for projects.color
- Projects create/edit: Gantt color field with Pickr (swatch + hex input),
  Pickr theme CSS and gantt-color-picker.js for init and sync
- Gantt API: include color in JSON for projects and tasks (tasks use project color)
- Gantt view: set custom_class from color, inject CSS for .bar and .bar-progress,
  fix selectors for .gantt .bar-wrapper and :hover/.active overrides; add
  fallback styles for gantt-project and gantt-task
2026-01-20 21:12:51 +01:00
Dries Peeters dafefd5d67 fix(inventory): stock devaluation and lot logic (fixes #385)
InventoryReportService: use moved_at and reference_type/reference_id instead of non-existent movement_date and reference to avoid AttributeError in get_inventory_turnover, _get_stock_at_date, get_movement_history.

_ensure_legacy_lot: run only for outflows (movement_qty < 0) or record_devaluation (movement_qty None). For outflows use pre-movement total (updated_stock + abs(qty)) so FIFO consumption stays in sync with WarehouseStock. Skip for inbound to prevent double-count of new lots.

record_movement: stop swallowing exceptions from _apply_lot_changes; re-raise so callers can roll back and avoid inconsistent WarehouseStock vs StockLot state.

Movement form: dynamic quantity hint by type (return/waste/devaluation/default), required devaluation fields when shown, client-side sign checks for return (positive) and waste (negative).

Tests: test_first_inbound_with_no_lots_matches_warehouse_stock, test_first_outbound_with_no_lots_matches_warehouse_stock.
2026-01-20 19:59:28 +01:00
Dries Peeters 0b76df53e2 feat: Add configurable date format for PDF templates
- Add date_format column to invoice_pdf_templates and quote_pdf_templates tables
- Default date format set to DD.MM.YYYY (%d.%m.%Y)
- Update PDF generators to use template-specific date format
- Add date format configuration in admin PDF template editor
- Replace Babel date formatting with strftime for consistent formatting
- Update template filters to use DD.MM.YYYY format by default

This allows users to customize date formatting per PDF template while
maintaining backward compatibility with existing templates.
2026-01-14 21:15:18 +01:00
Dries Peeters ba9b789c51 feat: Add decorative images to PDF templates with full sync between invoice and quote editors
- Add decorative image element to PDF Layout Designer for both invoice and quote templates
- Implement template-level decorative image upload and management
- Add backend routes for template image upload and serving
- Update PDF generation (HTML preview and ReportLab) to handle template images with transparency preservation
- Sync all decorative image functionality between invoice and quote PDF layout editors

Fixes:
- Fix upload button not opening file picker in invoice template (use fresh DOM references)
- Fix element name matching to handle 'decorative-image element-overlap' format (use .includes() instead of strict equality)
- Fix image restoration after page reload with enhanced JSON searching and position matching
- Fix image persistence in Konva.js serialization/deserialization

Improvements:
- Enhanced image restoration logic with fallback mechanisms
- Improved error handling and console logging for debugging
- Better handling of transparent backgrounds in PDF export
- Consistent behavior between invoice and quote template editors
2026-01-13 13:33:55 +01:00
Dries Peeters 61bead4cb2 feat: add devaluation support to stock movements
- Add 'devaluation' to stock movement types
- Add validation to ensure trackable items for devaluation
- Improve error handling for devaluation operations
2026-01-12 20:09:00 +01:00
Dries Peeters bad397fbdb feat: Enhance invoice management, expenses, and PDF generation
- Improve invoice model with enhanced prefix handling and validation
- Enhance expense routes with better error handling and validation
- Refactor PDF generator with improved template support and formatting
- Update Google Calendar integration with improved error handling
- Enhance scheduled tasks with better logging and reliability
- Update admin routes with improved permission checks
- Improve email utility with better template handling
2026-01-11 08:37:48 +01:00
Dries Peeters 4a681f0f48 feat: Enhance invoice and quote management system
- Improve invoice model with additional fields and methods
- Enhance quote model with better validation and relationships
- Add invoice repository for data access layer abstraction
- Update invoice and quote routes with improved functionality
- Add quote service for business logic separation
- Improve quote view and edit templates with better UX
2026-01-09 11:43:51 +01:00
Dries Peeters 4eeaa2a842 feat: Migrate PDF templates to ReportLab JSON format
- Add ReportLab template renderer with JSON-based template system
- Implement template schema validation and helper functions
- Add database migration for template_json columns
- Update visual editor to generate ReportLab JSON alongside HTML/CSS
- Maintain backward compatibility with legacy templates
- Add comprehensive migration documentation

BREAKING CHANGE: Existing PDF templates need to be saved again through
the visual editor to generate the new template_json format. Templates
will continue to work using the legacy fallback generator until saved.
2026-01-09 11:43:42 +01:00
Dries Peeters 83a03f9a99 feat: enhance application features and routes
- Improve expenses route with additional functionality
- Enhance admin route with new features
- Update auth route with improved authentication handling
- Extend user model with new capabilities
- Update expenses view template
- Improve config manager utility
2026-01-07 20:04:50 +01:00
Dries Peeters 14d673a4a8 Allow auto-imported time entries without project or client
- Add migration to update check constraint allowing NULL project_id and client_id for source='auto' entries
- Update TimeEntry model validation to allow entries without project/client when source='auto'
- Update TimeEntryCreateSchema to allow entries without project/client when source='auto'
- Enables calendar integrations to import entries that don't have project/client mapping yet
2026-01-07 13:18:40 +01:00
Dries Peeters ad9bfbf1ed Fix client deletion errors and add invoice validation
This commit fixes multiple issues preventing client deletion and adds proper validation to prevent deletion when invoices exist.

Database Schema Fixes:

- Migration 103: Add missing quote_number column to quotes table

  - Handles migration from offer_number to quote_number

  - Generates quote numbers for existing quotes if needed

  - Creates required unique index

- Migration 104: Add all missing columns to quotes table

  - Adds subtotal, tax_amount, visible_to_client columns

  - Adds discount fields (discount_type, discount_amount, discount_reason, coupon_code)

  - Adds payment_terms column

  - Adds approval workflow columns (approval_status, approved_by, approved_at, rejected_by, rejection_reason)

  - Creates required indexes and foreign keys

- Migration 105: Fix client_notifications foreign key cascade

  - Updates client_notifications.client_id FK to ON DELETE CASCADE

  - Updates client_notification_preferences.client_id FK to ON DELETE CASCADE

  - Prevents NOT NULL constraint violations during client deletion

Model Updates:

- Add passive_deletes=True to ClientNotification.client relationship

- Add passive_deletes=True to ClientNotificationPreferences.client relationship

- Add passive_deletes=True to ClientAttachment.client relationship

- Update ClientNote.client relationship to use passive_deletes

Route Updates:

- Add invoice validation to delete_client() and bulk_delete_clients()

- Manually delete notifications before client deletion to prevent SQLAlchemy update issues

Fixes:

- Resolves IntegrityError when deleting clients with notifications

- Resolves missing quote_number column errors

- Resolves missing quotes table columns errors

- Prevents deletion of clients with invoices (data integrity)
2026-01-05 22:07:50 +01:00
Dries Peeters f160fac9c1 refactor: improve error handling with specific exceptions and logging
Replace bare except clauses with specific exception types and add
appropriate logging throughout the codebase. This improves:
- Debugging capabilities with proper error messages
- Code maintainability by catching specific exceptions
- Error tracking through structured logging

Changes include:
- Replace bare except: with specific exception types (JSONDecodeError,
  TypeError, ValueError, OSError, AttributeError, RuntimeError)
- Add logging for error conditions that were previously silently ignored
- Improve error messages with context information
2026-01-05 19:32:25 +01:00
Dries Peeters b5c65b0ce8 feat: Add comment attachment routes and API support
- Add upload_comment_attachment route with file validation
- Add download_attachment route for file downloads
- Add delete_attachment route with permission checks
- Enhance Comment.to_dict() to include attachments array
- Support file size limits (10 MB) and type validation
- Proper error handling and file cleanup on errors

Routes follow existing attachment patterns from projects/clients.
2026-01-04 06:23:42 +01:00
Dries Peeters f205904742 feat: Add comment attachments model and migration
- Create CommentAttachment model following ProjectAttachment pattern
- Add database migration for comment_attachments table
- Register CommentAttachment in models __init__.py
- Support file uploads (images, PDFs, documents, archives)
- Include file metadata (size, type, uploader, timestamp)
- Cascade delete attachments when comments are deleted

Enables file attachments to comments for better team collaboration.
2026-01-04 06:23:39 +01:00
Dries Peeters eb0cd0005e feat: enhance inventory management features
- Update stock movement model with improved functionality
- Enhance inventory routes and API endpoints
- Improve inventory templates for movements, reports, and stock items
- Add better history tracking and valuation reporting
2026-01-03 20:27:54 +01:00
Dries Peeters 753b98d272 feat: add Peppol e-invoicing and stock lot valuation
Add Peppol BIS Billing 3.0 (UBL) invoice sending via a configurable access point, including admin-configurable settings, per-invoice send history, and documentation/README updates.

Also introduce stock lots/allocations (valuation layers) with supporting inventory route/report/UI updates and hardened startup migration handling.
2026-01-03 07:00:30 +01:00
Dries Peeters 3218ab012a feat: expand client portal and approval workflows
Add new client portal pages (dashboard, approvals, notifications, documents, reports) and extend API/routes/services to support client approvals, invoices/quotes views, and related notifications.

Update email templates and docs; add/adjust tests for new models/routes.
2026-01-02 07:52:32 +01:00
Dries Peeters 86f7946120 fix: resolve SQLAlchemy reserved word conflict in ClientNotification model
- Rename 'metadata' column to 'extra_data' in ClientNotification model
  to avoid SQLAlchemy Declarative API reserved word conflict
- Update ClientNotificationService to use 'extra_data' instead of 'metadata'
- Maintain API compatibility by returning 'metadata' key in to_dict() method
- Update migration to create 'extra_data' column instead of 'metadata'
- Improve migration idempotency and SQLite compatibility with proper checks
- Enhance backup directory handling with configurable BACKUP_FOLDER support
- Update admin routes to use centralized backup directory function

This fixes the application startup error:
  sqlalchemy.exc.InvalidRequestError: Attribute name 'metadata' is reserved
  when using the Declarative API.

The migration is now idempotent and handles both PostgreSQL and SQLite
databases safely.
2026-01-02 07:50:04 +01:00
Dries Peeters 427074398c Improve migration robustness and add missing schema columns
- Refactor migrations with idempotency checks and better error handling
  * Add SQLAlchemy inspector checks for table/column existence
  * Improve error messages and handling for schema operations
  * Make migrations safe to run multiple times
  * Update 27 migration files with enhanced error handling patterns

- Add missing schema columns via new migrations
  * Migration 095: Add ui_show_issues column to users table
  * Migration 096: Add portal_issues_enabled column to clients table

- Enhance Settings model error handling
  * Improve detection of schema errors (table/column missing)
  * Better handling of SQLAlchemy exceptions during migrations
  * More comprehensive error checking for OperationalErrors

- Fix database auto-switching logic in app initialization
  * Respect explicit DATABASE_URL setting to prevent unwanted switches
  * Only auto-switch to PostgreSQL when not explicitly configured

- Update docker entrypoint script with migration improvements
2026-01-01 09:15:31 +01:00
Dries Peeters 9507a9492c feat: Add comprehensive donation system with smart prompts and improved accessibility
- Add dedicated donation page (/donate) explaining why donations matter
- Implement DonationInteraction model to track user engagement and interactions
- Add smart banner logic with contextual messaging based on user milestones
- Improve donation accessibility with links in sidebar, footer, dashboard, and all major pages
- Add UTM tracking to all Buy Me a Coffee links for analytics
- Fix CSRF token issues in donation tracking JavaScript
- Enhance dashboard widget with user stats and dual action buttons
- Add donation information section to About page
- Update support banner with 'Learn More' and 'Donate' options
- Create database migration for donation_interactions table

The donation system now provides:
- Smart prompts that show after user milestones (7+ days, 50+ entries, 100+ hours)
- Banner dismissal tracking with 30-day cooldown
- Multiple access points throughout the application
- Better visibility of donation impact and importance
- Comprehensive tracking for analytics and optimization
2025-12-30 09:52:12 +01:00
Dries Peeters 0c310736c1 Remove legacy UI enable/disable settings system
Remove both system-wide and per-user UI feature enable/disable settings
in favor of the centralized ModuleRegistry system for module management.

Changes:
- Remove ui_allow_* columns from Settings model and database (migration 093)
- Remove ui_show_* preference assignments from user settings route
- Remove UI Customization section from user settings template
- Remove UI Features section from admin settings template
- Update admin modules template to use ModuleRegistry instead of settings flags
- Remove settings_flag and user_flag attributes from ModuleDefinition
- Update ModuleRegistry.is_enabled() to only check dependencies and default_enabled
- Update dashboard template to use is_module_enabled() helper
- Update admin route docstring to reflect module management changes

Module visibility is now controlled exclusively via the admin module
management interface (/admin/modules), eliminating the need for separate
system-wide and per-user UI preference systems.
2025-12-29 15:51:59 +01:00
Dries Peeters 083dd9f1f2 feat: Implement comprehensive module system with visibility controls
- Add centralized module registry system (ModuleRegistry) for managing
  module metadata, dependencies, and visibility across the application
- Create module helper utilities with decorators (@module_enabled) and
  helper functions for route protection and template access
- Add database migration (092) to add missing module visibility flags
  to settings and users tables for granular control
- Extend Settings and User models with additional module visibility
  flags for CRM, Finance, Tools, and Advanced features
- Implement admin module management UI for system-wide module
  enable/disable controls
- Add module checks to routes (calendar, contacts, deals, expenses,
  invoices, leads, custom_reports) to enforce visibility rules
- Update scheduled report service and report templates to respect
  module visibility settings
- Bump version to 4.8.0 in setup.py
- Add comprehensive documentation for module integration planning
  and implementation analysis
2025-12-29 14:13:32 +01:00
Dries Peeters 92893b188d fix: complete backend implementations and integration improvements
This commit addresses multiple incomplete implementations identified in the
codebase analysis, focusing on security, functionality, and error handling.

Backend Fixes:
- Issues module: Implement proper permission filtering for non-admin users
  - Users can only see issues for projects they have access to
  - Added permission checks to view_issue and edit_issue routes
  - Statistics now respect user permissions

- Push notifications: Implement proper subscription storage
  - Created PushSubscription model for browser push notification subscriptions
  - Updated routes to use new model with proper CRUD operations
  - Added support for multiple subscriptions per user
  - Added endpoint to list user subscriptions

Integration Improvements:
- GitHub: Implement webhook signature verification
  - Added HMAC SHA-256 signature verification using webhook secret
  - Uses constant-time comparison to prevent timing attacks
  - Added webhook_secret field to config schema

- QuickBooks: Implement customer and account mapping
  - Added support for customer mappings (client → QuickBooks customer)
  - Added support for item mappings (invoice items → QuickBooks items)
  - Added support for account mappings (expense categories → accounts)
  - Added default expense account configuration
  - Improved error handling and logging

- Xero: Add customer and account mapping support
  - Added contact mappings (client → Xero Contact ID)
  - Added item mappings (invoice items → Xero item codes)
  - Added account mappings (expense categories → Xero account codes)
  - Added default expense account configuration

- CalDAV: Implement bidirectional sync
  - Added TimeTracker to Calendar sync direction
  - Implemented iCalendar event generation from time entries
  - Added create_or_update_event method to CalDAVClient
  - Supports bidirectional sync (both directions simultaneously)
  - Improved error handling for event creation/updates

- Trello: Implement bidirectional sync
  - Added TimeTracker to Trello sync direction
  - Implemented task to card creation and updates
  - Automatic board creation for projects if needed
  - Maps task status to Trello lists
  - Supports bidirectional sync

- Exception handling: Improve error logging in integrations
  - Replaced silent pass statements with proper error logging
  - Added debug logging for non-critical failures (user info fetch)
  - Improved error visibility for debugging
  - Affected: Google Calendar, Outlook Calendar, Microsoft Teams, Asana, GitLab

All changes include proper error handling, logging, and follow existing code
patterns. Database migration required for push_subscriptions table.
2025-12-29 12:31:52 +01:00
Dries Peeters 443a797e2d feat: Add CalDAV calendar integration for Zimbra and other CalDAV servers
Implement comprehensive CalDAV calendar integration to import calendar events
as time entries from CalDAV-compatible servers (Zimbra, Nextcloud, ownCloud).

Features:
- CalDAV client with calendar discovery and event fetching
- Automatic calendar discovery from server URL
- Import calendar events (VEVENT) as time entries
- Project matching from event titles with fallback to default project
- Idempotent sync using IntegrationExternalEventLink to prevent duplicates
- Per-user integration setup (similar to Google Calendar)
- Support for both server URL (with discovery) and direct calendar URL
- SSL certificate verification toggle for self-signed certificates
- Configurable lookback period for event import

Components:
- CalDAVCalendarConnector: Main integration connector with sync logic
- CalDAVClient: Low-level CalDAV client using PROPFIND/REPORT requests
- IntegrationExternalEventLink: Model for tracking imported events (idempotency)
- Setup UI: User-friendly form for configuration
- Comprehensive validation and error handling
- Full test coverage (unit, integration, route tests)
- Documentation: Setup guide and troubleshooting

Technical details:
- Uses icalendar library for parsing VEVENT components
- Handles timezone conversion (CalDAV UTC to app local timezone)
- Skips all-day events (only imports timed events)
- Stores credentials securely (password in access_token, username in extra_data)
- Automatic calendar discovery on first sync if only server URL provided

Migration:
- Adds integration_external_event_links table for sync tracking
- Unique constraint on (integration_id, external_uid) prevents duplicates

Documentation:
- CALDAV_INTEGRATION.md: Complete feature documentation
- CALDAV_QUICK_SETUP.md: Step-by-step setup guide with examples

Closes feature request for CalDAV/Zimbra integration.
2025-12-20 17:28:48 +01:00
Dries Peeters 7791e6ada0 feat: Add comprehensive issue/bug tracking system
Implement a complete issue management system with client portal integration
and internal admin interface for tracking and resolving client-reported issues.

Features:
- New Issue model with full lifecycle management (open, in_progress, resolved, closed, cancelled)
- Priority levels (low, medium, high, urgent) with visual indicators
- Issue linking to projects and tasks
- Create tasks directly from issues
- Client portal integration for issue reporting and viewing
- Internal admin routes for issue management, filtering, and assignment
- Comprehensive templates for both client and admin views
- Status filtering and search functionality
- Issue assignment to internal users
- Automatic timestamp tracking (created, updated, resolved, closed)

Client Portal:
- Clients can report new issues with project association
- View all issues with status filtering
- View individual issue details
- Submit issues with optional submitter name/email

Admin Interface:
- List all issues with advanced filtering (status, priority, client, project, assignee, search)
- View, edit, and delete issues
- Link issues to existing tasks
- Create tasks from issues
- Update issue status, priority, and assignment
- Issue statistics dashboard

Technical:
- Added Issue model with relationships to Client, Project, Task, and User
- New issues blueprint for internal management
- Extended client_portal routes with issue endpoints
- Updated model imports and relationships
- Added navigation links in base templates
- Version bump to 4.6.0
- Code cleanup in docker scripts and schema verification
2025-12-14 07:25:42 +01:00
Dries Peeters bde61c7f5d Fix permission and role management bugs
Fix multiple permission and role-related issues:

1. Gantt chart access: Replace is_admin check with view_projects permission
   - Users with custom roles having view_projects permission can now access
     Gantt charts, not just admins
   - Updated app/routes/gantt.py to check permissions properly

2. Task view filtering: Replace is_admin check with view_all_tasks permission
   - Users with custom roles having view_all_tasks permission can now see
     all tasks in the Tasks view, not just admins
   - Updated app/services/task_service.py to accept has_view_all_tasks parameter
   - Updated app/routes/tasks.py list_tasks and export_tasks to use permission check

3. Role assignment security: Prevent privilege escalation
   - Added is_super_admin property to User model
   - Only super_admins can assign super_admin role to users
   - Only super_admins can remove admin role from themselves or others
   - Prevents admins from escalating privileges or removing admin access
   - Updated app/routes/permissions.py manage_user_roles with validation

4. Version display consistency: Ensure consistent version display
   - Added APP_VERSION environment variable to docker-compose.example.yml
   - Ensures version is displayed consistently when using pre-built images

All changes maintain backward compatibility and follow the existing
permission system architecture.
2025-12-12 22:18:30 +01:00
Dries Peeters 88656c3d34 feat: Advanced Report Builder with iterative generation and email distribution
Implement comprehensive enhancements to the Report Builder system with support
for iterative report generation, flexible email distribution, and improved
error handling.

Features:
- Add iterative report generation: generate one report per custom field value
- Add email distribution modes: mapping, template, and single recipient modes
- Add recipient email templates with {value} placeholder support
- Enhance scheduled reports with better error handling and validation
- Add fix endpoint for invalid scheduled reports
- Improve report builder UI with iterative generation options
- Add comprehensive management views for report schemes

Fixes:
- Fix template errors in iterative report view (dict access issues)
- Fix empty report builder when editing saved reports
- Fix PWA install toast notification handling
- Fix migration revision ID length issue (shortened to fit 32 char limit)
- Add idempotent migration checks to prevent duplicate column errors
- Improve error handling in scheduled reports list view

Database Changes:
- Add iterative_report_generation and iterative_custom_field_name to saved_report_views
- Add email_distribution_mode and recipient_email_template to report_email_schedules
- Migration 090_report_builder_iteration (idempotent)

UI/UX Improvements:
- Display iterative generation status in saved views list
- Show distribution mode and template in scheduled reports
- Add error badges and fix buttons for invalid schedules
- Improve report builder form loading for saved configurations

Technical:
- Enhance ScheduledReportService with recipient resolution logic
- Add validation for report configurations
- Improve error handling and logging throughout
- Update templates to use safe dictionary access patterns
2025-12-12 22:11:57 +01:00
Dries Peeters a582e2af62 feat: improve error handling, performance logging, and PWA install UI
- Add session state clearing (expunge_all) after rollbacks in custom field
  definition error handlers to prevent stale session state
- Add graceful error handling for missing link_templates table with proper
  rollback and session cleanup, preventing app crashes when migrations
  haven't been run
- Add detailed performance logging to TaskService.list_tasks method to track
  timing of each query step for performance monitoring
- Improve PWA install prompt UI with better toast integration, dismiss button,
  and proper DOM manipulation using requestAnimationFrame
- Bump version to 4.5.0
2025-12-12 21:49:26 +01:00
Dries Peeters 98fefd5ec1 fix: handle missing custom_field_definitions table gracefully (fixes #344)
- Updated CustomFieldDefinition model methods to catch ProgrammingError
  when table doesn't exist (migration not run yet)
- Methods now return empty list/None instead of raising 500 errors
- Added transaction rollback to prevent subsequent query failures
- Prevents database errors when visiting Time Entries, Projects, or Clients
  pages before migration 084_add_custom_field_definitions is applied

The fix ensures the application continues to work even if the migration
hasn't been run, with appropriate warning logs suggesting to run
'flask db upgrade'.
2025-12-05 07:50:35 +01:00
Dries Peeters 4e57f08c03 Add salesman-based report splitting and email distribution
- Add SalesmanEmailMapping model to map salesman initials to email addresses
  - Support for direct email addresses, email patterns, and domain-based patterns
  - Admin interface for managing email mappings

- Add UnpaidHoursService for querying and grouping unpaid time entries
  - Filter by client custom fields (e.g., salesman)
  - Group unpaid hours by salesman for report generation

- Add salesman report routes and API endpoints
  - CRUD operations for email mappings
  - Generate and send reports split by salesman
  - Preview email addresses for salesman initials

- Enhance scheduled reports with salesman splitting
  - Add split_by_salesman and salesman_field_name to report schedules
  - Automatically split reports by salesman and send to mapped emails

- Add UI components for salesman report management
  - Admin dashboard integration
  - Report builder with salesman splitting options
  - Email mapping management interface

- Add email template for unpaid hours reports

- Add database migrations:
  - 087: Create salesman_email_mappings table
  - 088: Add salesman splitting fields to report_email_schedules
2025-12-03 08:59:48 +01:00
Dries Peeters f3a3a40480 Add project custom fields and file attachments for projects and clients
This commit introduces two major features:

1. Project Custom Fields: Add custom_fields JSON column to projects table (migration 085), support for flexible custom data storage, display and edit in project views

2. File Attachments System: Add project_attachments and client_attachments tables (migration 086), new ProjectAttachment and ClientAttachment models, full CRUD operations, file upload/download/delete, client-visible attachments support

Additional improvements: Enhanced data tables, updated project/client/invoice/timer views, improved UI for attachments and custom fields management
2025-12-03 08:30:15 +01:00
Dries Peeters 86b3498f05 perf: optimize task queries and fix N+1 performance issues
This commit includes multiple performance optimizations, error handling
improvements, and bug fixes across the application.

Performance Improvements:
- Add caching for task status_display property to avoid N+1 queries
- Pre-calculate task counts by status in route handler instead of template
- Pre-load kanban columns in TaskService to eliminate N+1 queries
- Remove unnecessary db.session.expire_all() call in tasks route
- Always use pagination for task lists to improve performance

Error Handling & Robustness:
- Add graceful handling for missing time_entry_approvals table in timer deletion
- Improve safe_commit to handle ProgrammingError for optional relationships
- Add VAPID key validation and error handling in PWA push notifications
- Make custom_field_definitions migration idempotent

Bug Fixes:
- Fix IndexedDB boolean query issues in offline-sync.js by using cursor iteration
- Fix app context handling in scheduled reports processing
- Improve error messages for push notification subscription failures
2025-12-03 07:02:39 +01:00
Dries Peeters ac19bebf2d feat: enhance offline sync and improve performance (v4.3.2)
- Add comprehensive offline sync improvements with enhanced IndexedDB support
- Optimize task model with cached total_hours calculation for better performance
- Improve task service query optimization and eager loading strategies
- Update CSP policy to allow CDN connections for improved resource loading
- Enhance service worker with better background sync capabilities
- Improve error handling and offline queue processing
- Update base template and comment templates for better UX
- Bump version to 4.3.2
2025-12-02 06:13:54 +01:00
Dries Peeters 2bd48d9e60 feat: Add client count tracking and cleanup for custom field definitions
- Add count_clients_with_value() method to CustomFieldDefinition model to track how many clients have values for each custom field

- Display client count in custom field definitions list view

- Automatically remove custom field values from all clients when a custom field definition is deleted

- Show user-friendly confirmation message indicating how many clients were affected when deleting a field definition

- Update client view to use custom field definitions for friendly field names instead of raw field keys

- Add comprehensive test suite for custom field definitions including model creation, client count functionality, deletion cleanup, and edge cases

- Update templates to display client counts and improve delete confirmation dialogs
2025-12-01 18:48:38 +01:00
Dries Peeters 9112a696dd feat: Enhance audit logging with improved error handling and diagnostic tools
- Improve audit logging error messages to distinguish table missing errors from other failures

- Add warning-level logging for audit_logs table missing scenarios with migration guidance

- Update audit event listener with better error detection and logging

- Add comprehensive diagnostic script for checking audit logging setup

- Update UI templates (base.html, admin forms, user settings, profile pages)

- Extend audit logging support across routes (admin, api, permissions, reports, timer, user)

- Add extensive test coverage for admin user management functionality

- Update time tracking service and user model with audit logging integration
2025-12-01 13:30:18 +01:00
Dries Peeters 3f5d253a8b feat: Add global custom field definitions with link template support
Implement a global custom field system that allows defining fields once
and reusing them across all clients. Fields can be marked as mandatory
or optional, and values become clickable links when link templates are
assigned.

- Add CustomFieldDefinition model and admin interface
- Support mandatory/optional flags with validation
- Update client forms to use global definitions
- Enhance link templates to support {value} and %value% placeholders
- Make custom field values clickable in client view
- Add migration 084 for custom_field_definitions table

Breaking: Client custom fields now require global definitions. Existing
custom fields continue to work, but new fields must be defined globally.
2025-11-30 15:29:02 +01:00
Dries Peeters 90d8407bda feat(billing): add paid status tracking for time entries with invoice reference
Add ability to mark time entries as paid and link them to internal invoice
numbers. Automatically mark time entries as paid when invoices are sent.

Database Changes:
- Add migration 083 to add `paid` boolean and `invoice_number` string columns
  to time_entries table
- Add index on `paid` field for faster queries

Model Updates:
- Add `paid` (default: False) and `invoice_number` (nullable) fields to TimeEntry
- Add `set_paid()` helper method to TimeEntry model
- Update `to_dict()` to include paid status and invoice number

API & Service Layer:
- Update TimeEntrySchema (all variants) to include paid/invoice_number fields
- Update API endpoints (/api/entry, /api/v1/time-entries) to accept these fields
- Update TimeTrackingService and TimeEntryRepository to handle paid status
- Add InvoiceService.mark_time_entries_as_paid() to automatically mark entries
- Update InvoiceService.mark_as_sent() to auto-mark time entries as paid

UI Updates:
- Add "Paid" checkbox and "Invoice Number" input field to time entry edit forms
- Update both admin and regular user edit forms
- Fields appear in timer edit page after tags section

Invoice Integration:
- Automatically mark time entries as paid when invoice status changes to "sent"
- Mark entries when time is added to already-sent invoices
- Store invoice number reference on time entries for tracking
- Enhanced create_invoice_from_time_entries() to properly link time entries

This enables proper tracking of which hours have been invoiced and paid
through the internal invoicing system, separate from the external ERP system.
2025-11-30 11:31:42 +01:00
Dries Peeters 583f9b6755 Format code with Black to fix code quality test 2025-11-29 07:13:23 +01:00
Dries Peeters 0ec6b8e9d6 refactor: major integration system overhaul with global integrations support
This commit implements a comprehensive refactoring of the integration system to support both global (shared) and per-user integrations, adds new integrations, and improves the overall architecture.

Key changes:

- Add global integrations support: most integrations are now shared across all users (Jira, Slack, GitHub, Asana, Trello, GitLab, Microsoft Teams, Outlook Calendar, Xero)

- Add new integrations: GitLab, Microsoft Teams, Outlook Calendar, and Xero

- Database migrations:

  * Migration 081: Add OAuth credential columns for all integrations to Settings model

  * Migration 082: Add is_global flag to Integration model and make user_id nullable

- Update Integration model to support global integrations with nullable user_id

- Refactor IntegrationService to handle both global and per-user integrations

- Create dedicated admin setup pages for each integration

- Update Trello connector to use API key setup instead of OAuth flow

- Enhance all existing integrations (Jira, Slack, GitHub, Google Calendar, Asana, Trello) with global support

- Update routes, templates, and services to support the new global/per-user distinction

- Improve integration management UI with better separation of global vs per-user integrations

- Update scheduled tasks to work with the new integration architecture
2025-11-29 07:03:00 +01:00
Dries Peeters dcbdfcc288 feat: Add client custom fields, link templates, UI feature flags, and client billing support
Add client custom fields (JSON) for flexible data storage

Implement link templates system for dynamic URL generation from custom fields

Add client_id support to time entries for direct client billing (project_id now nullable)

Implement user-level UI feature flags for customizable navigation visibility

Add system-wide UI feature flags in settings for admin control

Fix metadata column naming (user_badges.achievement_metadata, leaderboard_entries.entry_metadata)

Update templates and routes to support new features

Add comprehensive UI feature flag management in admin and user settings

Enhance client views with custom fields and link template integration

Update time entry forms to support client billing

Add tests for system UI flags

Migrations: 075-080 for custom fields, link templates, UI flags, client billing, and metadata fixes
2025-11-29 06:17:07 +01:00
Dries Peeters 6ad96f114a Fix user creation to use new Role system and add default password with forced change
- Updated user creation to assign roles from Role system instead of legacy role field
- Added password_change_required field to User model with migration
- Added default password input and force password change option in user creation form
- Updated login route to check password_change_required and redirect to change password page
- Created change_password route and template for forced password changes
- Updated all user creation points (admin, self-registration, OIDC, default admin) to use new Role system
- Updated user form template to show roles from Role system instead of hardcoded options

Fixes issue where newly created users were still using legacy roles instead of the new role-based permission system.
2025-11-28 22:47:01 +01:00
Dries Peeters 8585b097e0 feat: Add comprehensive feature implementation including integrations, workflows, approvals, and AI features
Major Features:
- Integration framework with implementations for Asana, Google Calendar, QuickBooks, and Trello
- Workflow automation system with workflow engine service
- Time entry approval system with client approval capabilities
- Recurring tasks functionality
- Client portal customization and team chat features
- AI-powered categorization and suggestion services
- GPS tracking for expenses
- Gamification system with service layer
- Custom reporting with service and model support
- Enhanced OCR service for expense processing
- Pomodoro timer service
- Currency service for multi-currency support
- PowerPoint export utility

Frontend Enhancements:
- Activity feed JavaScript module
- Mentions system for team chat
- Offline sync capabilities
- New templates for approvals, chat, and recurring tasks

Database Migrations:
- Updated integration framework migrations (066-068)
- Added workflow automation migration (069)
- Added time entry approvals migration (070)
- Added recurring tasks migration (071)
- Added client portal and team chat migration (072)
- Added AI features and GPS tracking migration (073)

Documentation:
- Updated implementation documentation
- Removed obsolete feature gap analysis docs
- Added comprehensive implementation status reports
2025-11-28 22:39:04 +01:00
Dries Peeters 579fc7af02 refactor: extract business logic to service layer and add comprehensive test coverage
Major refactoring to improve code organization and maintainability:

- Refactor API routes (api_v1.py) to delegate business logic to service layer
- Add new QuoteService for quote management operations
- Enhance existing services: ExpenseService, InvoiceService, PaymentService, ProjectService, TimeTrackingService
- Improve caching utilities with enhanced cache management
- Enhance API authentication utilities
- Add comprehensive test suite covering routes, services, and utilities
- Update routes to use service layer pattern (kiosk, main, projects, quotes, timer, time_entry_templates)
- Update time entry template model with additional functionality
- Update Docker configuration and startup scripts
- Update dependencies and setup configuration

This refactoring improves separation of concerns, testability, and code maintainability while preserving existing functionality.
2025-11-28 21:15:10 +01:00
Dries Peeters 90dde470da style: standardize code formatting and normalize line endings
- Normalize line endings from CRLF to LF across all files to match .editorconfig
- Standardize quote style from single quotes to double quotes
- Normalize whitespace and formatting throughout codebase
- Apply consistent code style across 372 files including:
  * Application code (models, routes, services, utils)
  * Test files
  * Configuration files
  * CI/CD workflows

This ensures consistency with the project's .editorconfig settings and
improves code maintainability.
2025-11-28 20:05:37 +01:00
Dries Peeters 50f9bbbbae feat: implement configuration priority system (WebUI > .env > defaults)
Implement a configuration management system where settings changed via
WebUI take priority over .env values, while .env values are used as initial
startup values.

Changes:
- Update ConfigManager.get_setting() to check Settings model first, then
  environment variables, ensuring WebUI changes have highest priority
- Add Settings._initialize_from_env() method to initialize new Settings
  instances from .env file values on first creation
- Update Settings.get_settings() to automatically initialize from .env
  when creating a new Settings instance
- Add Settings initialization in create_app() to ensure .env values are
  loaded on application startup
- Add comprehensive test suite (test_config_priority.py) covering:
  * Settings priority over environment variables
  * .env values used as initial startup values
  * WebUI changes persisting and taking priority
  * Proper type handling for different setting types

This ensures that:
1. .env file values are used as initial configuration on first startup
2. Settings changed via WebUI are saved to database and take priority
3. Configuration priority order: Settings (DB) > .env > app config > defaults

Fixes configuration management workflow where users can set initial values
in .env but override them permanently via WebUI without modifying .env.
2025-11-28 16:19:03 +01:00
Dries Peeters 4930f6a3e5 feat: add multiple authentication modes support
Add support for four authentication modes via AUTH_METHOD environment variable:
- none: Username-only authentication (no password)
- local: Password authentication required (default)
- oidc: OIDC/Single Sign-On only
- both: OIDC + local password authentication

Key changes:
- Add password_hash column to users table (migration 068)
- Implement password storage and verification in User model
- Update login routes to handle all authentication modes
- Add conditional password fields in login templates
- Support password authentication in kiosk mode
- Allow password changes in user profile when enabled

Password authentication is now enabled by default for better security,
while remaining backward compatible with existing installations.
Users will be prompted to set passwords when required.

Fixes authentication bypass issue where users could access accounts
without passwords even after setting them.
2025-11-28 15:56:01 +01:00
Dries Peeters eb4fb8296f feat: Add integration framework and major feature enhancements
This commit introduces a comprehensive integration framework and multiple new features to enhance the TimeTracker application's capabilities.

Major Features:

- Integration Framework: Extensible system for third-party integrations with support for Jira, Slack, GitHub, and calendar services

- Project Templates: Reusable project templates for faster project creation

- Invoice Approvals: Workflow for invoice approval before sending

- Payment Gateways: Online payment processing integration with Stripe support

- Scheduled Reports: Automated report generation and email delivery

- Custom Reports: Advanced report builder with saved views

- Gantt Chart: Visual project timeline and dependency management

- Calendar Integrations: External calendar synchronization with Google Calendar support

- Push Notifications: Enhanced notification system with PWA support

Bug Fixes:

- Fix None handling in analytics routes

- Fix dynamic relationship loading issues in ProjectRepository and ProjectService

- Fix parameter ordering in service methods

- Fix None duration_seconds handling in budget forecasting

UI/UX Improvements:

- Update logo references to timetracker-logo.svg

- Add favicon links to all templates

- Add navigation items for new features

- Enhance invoice view with approval status and payment gateway links

Database:

- Add Alembic migrations for new features (065, 066, 067)

Dependencies:

- Add stripe==7.0.0 for payment processing

- Add google-api-python-client libraries for calendar integration
2025-11-26 07:53:28 +01:00
Dries Peeters bdf9249edc refactor: comprehensive application improvements and architecture enhancements
This commit implements all critical improvements from the application review,
establishing modern architecture patterns and significantly improving performance,
security, and maintainability.

## Architecture Improvements

- Implement service layer pattern: Migrated routes (projects, tasks, invoices, reports)
  to use dedicated service classes with business logic separation
- Add repository pattern: Enhanced repositories with comprehensive docstrings and
  type hints for better data access abstraction
- Create base CRUD service: BaseCRUDService reduces code duplication across services
- Implement API versioning structure: Created app/routes/api/ package with v1
  subpackage for future versioning support

## Performance Optimizations

- Fix N+1 query problems: Added eager loading (joinedload) to all migrated routes,
  reducing database queries by 80-90%
- Add query logging: Implemented query_logging.py for performance monitoring and
  slow query detection
- Create caching foundation: Added cache_redis.py utilities ready for Redis integration

## Security Enhancements

- Enhanced API token management: Created ApiTokenService with token rotation,
  expiration management, and scope validation
- Add environment validation: Implemented startup validation for critical
  environment variables with production checks
- Improve error handling: Standardized error responses with route_helpers.py utilities

## Code Quality

- Add comprehensive type hints: All service and repository methods now have
  complete type annotations
- Add docstrings: Comprehensive documentation added to all services, repositories,
  and public APIs
- Standardize error handling: Consistent error response patterns across all routes

## Testing

- Add unit tests: Created test suites for ProjectService, TaskService,
  InvoiceService, ReportingService, ApiTokenService, and BaseRepository
- Test coverage: Added tests for CRUD operations, eager loading, filtering,
  and error cases

## Documentation

- Add API versioning documentation: Created docs/API_VERSIONING.md with
  versioning strategy and migration guidelines
- Add implementation documentation: Comprehensive review and progress
  documentation files

## Files Changed

### New Files (20+)
- app/services/base_crud_service.py
- app/services/api_token_service.py
- app/utils/env_validation.py
- app/utils/query_logging.py
- app/utils/route_helpers.py
- app/utils/cache_redis.py
- app/routes/api/__init__.py
- app/routes/api/v1/__init__.py
- tests/test_services/*.py (5 files)
- tests/test_repositories/test_base_repository.py
- docs/API_VERSIONING.md
- Documentation files (APPLICATION_REVIEW_2025.md, etc.)

### Modified Files (15+)
- app/services/project_service.py
- app/services/task_service.py
- app/services/invoice_service.py
- app/services/reporting_service.py
- app/routes/projects.py
- app/routes/tasks.py
- app/routes/invoices.py
- app/routes/reports.py
- app/repositories/base_repository.py
- app/repositories/task_repository.py
- app/__init__.py

## Impact

- Performance: 80-90% reduction in database queries
- Code Quality: Modern architecture patterns, type hints, comprehensive docs
- Security: Enhanced API token management, environment validation
- Maintainability: Service layer separation, consistent error handling
- Testing: Foundation for comprehensive test coverage

All changes are backward compatible and production-ready.
2025-11-24 20:58:22 +01:00
Dries Peeters 25ea52c029 feat: Implement CRM features and fix migration issues
- Add CRM models: Contact, ContactCommunication, Deal, DealActivity, Lead, LeadActivity
  - Support multiple contacts per client with primary contact designation
  - Track sales pipeline with deals and opportunities
  - Manage leads with conversion tracking
  - Record communication history with contacts

- Add CRM routes and templates
  - Contact management (list, create, view, edit, delete)
  - Deal management with pipeline view
  - Lead management with conversion workflow
  - Communication history tracking

- Fix SQLAlchemy relationship conflicts
  - Specify foreign_keys for Deal.lead relationship to resolve ambiguity
  - Remove duplicate backref definitions in DealActivity and LeadActivity

- Improve migration 062 robustness
  - Add index existence checks before creation
  - Handle partial migration states gracefully
  - Support both assigned_to and assignee_id column names

- Add error handling for missing CRM tables
  - Gracefully handle cases where migration 063 hasn't run yet
  - Prevent application crashes when CRM tables don't exist

- Add database migration 063 for CRM features
  - Create contacts, contact_communications, deals, deal_activities, leads, lead_activities tables
  - Set up proper foreign key relationships and indexes

- Update documentation
  - Add CRM features to FEATURES_COMPLETE.md
  - Create CRM implementation documentation
  - Add feature gap analysis documentation
2025-11-23 20:38:35 +01:00