mirror of
https://github.com/DRYTRIX/TimeTracker.git
synced 2026-02-12 23:28:48 -06:00
8a378b7078
- Clients: add model, routes, and templates
- app/models/client.py
- app/routes/clients.py
- templates/clients/{create,edit,list,view}.html
- docs/CLIENT_MANAGEMENT_README.md
- Database: add enhanced init/verify scripts, migrations, and docs
- docker/{init-database-enhanced.py,start-enhanced.py,verify-database.py}
- docs/ENHANCED_DATABASE_STARTUP.md
- migrations/{add_analytics_column.sql,add_analytics_setting.py,migrate_to_client_model.py}
- Scripts: add version manager and docker network test helpers
- scripts/version-manager.{bat,ps1,py,sh}
- scripts/test-docker-network.{bat,sh}
- docs/VERSION_MANAGEMENT.md
- UI: tweak base stylesheet
- app/static/base.css
- Tests: add client system test
- test_client_system.py
283 lines
9.9 KiB
Markdown
283 lines
9.9 KiB
Markdown
# Enhanced Database Startup Procedure
|
|
|
|
This document describes the improved database initialization and startup procedure for TimeTracker that ensures all tables are correctly created with proper schema and handles migrations automatically.
|
|
|
|
## Overview
|
|
|
|
The enhanced startup procedure addresses several issues found in the previous implementation:
|
|
|
|
1. **Missing Analytics Table**: The analytics functionality exists but there was no dedicated analytics table
|
|
2. **Schema Verification**: Previous scripts didn't verify that all required columns exist in existing tables
|
|
3. **Migration Handling**: Analytics setting migration wasn't integrated into the main startup process
|
|
4. **Table Recreation Logic**: Previous logic might drop and recreate tables unnecessarily
|
|
|
|
## New Components
|
|
|
|
### 1. Enhanced Database Initialization Script (`init-database-enhanced.py`)
|
|
|
|
This script provides comprehensive database setup:
|
|
|
|
- **Schema Definition**: Defines the complete expected database schema including all tables, columns, and relationships
|
|
- **Smart Table Creation**: Creates tables only if they don't exist, or adds missing columns if they do exist
|
|
- **Index Management**: Creates performance indexes for all tables
|
|
- **Trigger Setup**: Sets up automatic timestamp updates for `updated_at` columns
|
|
- **Data Initialization**: Inserts default admin user, project, and settings
|
|
- **Schema Verification**: Verifies that all required tables and columns exist after initialization
|
|
|
|
#### Tables Created
|
|
|
|
- `users` - User accounts and authentication
|
|
- `projects` - Project definitions and client information
|
|
- `time_entries` - Time tracking records
|
|
- `tasks` - Task management within projects
|
|
- `settings` - Application configuration and company branding
|
|
- `invoices` - Invoice management
|
|
- `invoice_items` - Individual invoice line items
|
|
|
|
#### Key Features
|
|
|
|
- **Non-destructive**: Never drops existing tables or data
|
|
- **Migration-friendly**: Automatically adds missing columns to existing tables
|
|
- **Comprehensive**: Handles all aspects of database setup in one script
|
|
- **Error-tolerant**: Continues operation even if some optional features fail
|
|
|
|
### 2. Enhanced Startup Script (`start-enhanced.py`)
|
|
|
|
Simplified startup process that:
|
|
|
|
- Uses the enhanced database initialization script
|
|
- Provides better error handling and logging
|
|
- Displays network information for debugging
|
|
- Ensures proper startup sequence
|
|
|
|
### 3. Database Verification Script (`verify-database.py`)
|
|
|
|
Independent verification tool that:
|
|
|
|
- Checks all tables exist with correct schema
|
|
- Verifies indexes and foreign keys
|
|
- Reports any missing or incorrect elements
|
|
- Can be run independently to diagnose issues
|
|
|
|
## Startup Sequence
|
|
|
|
1. **Container Start**: Docker container starts with the enhanced startup script
|
|
2. **Database Wait**: Waits for PostgreSQL database to be ready
|
|
3. **Enhanced Initialization**: Runs the comprehensive database setup script
|
|
4. **Schema Verification**: Verifies all tables and columns are correct
|
|
5. **Application Start**: Launches the Flask application with Gunicorn
|
|
|
|
## Database Schema
|
|
|
|
### Core Tables
|
|
|
|
#### users
|
|
```sql
|
|
id SERIAL PRIMARY KEY,
|
|
username VARCHAR(80) UNIQUE NOT NULL,
|
|
role VARCHAR(20) DEFAULT 'user' NOT NULL,
|
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL,
|
|
last_login TIMESTAMP,
|
|
is_active BOOLEAN DEFAULT true NOT NULL,
|
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL
|
|
```
|
|
|
|
#### projects
|
|
```sql
|
|
id SERIAL PRIMARY KEY,
|
|
name VARCHAR(200) NOT NULL,
|
|
client VARCHAR(200) NOT NULL,
|
|
description TEXT,
|
|
billable BOOLEAN DEFAULT true NOT NULL,
|
|
hourly_rate NUMERIC(9, 2),
|
|
billing_ref VARCHAR(100),
|
|
status VARCHAR(20) DEFAULT 'active' NOT NULL,
|
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
|
```
|
|
|
|
#### time_entries
|
|
```sql
|
|
id SERIAL PRIMARY KEY,
|
|
user_id INTEGER REFERENCES users(id) ON DELETE CASCADE,
|
|
project_id INTEGER REFERENCES projects(id) ON DELETE CASCADE,
|
|
task_id INTEGER REFERENCES tasks(id) ON DELETE SET NULL,
|
|
start_time TIMESTAMP NOT NULL,
|
|
end_time TIMESTAMP,
|
|
duration_seconds INTEGER,
|
|
notes TEXT,
|
|
tags VARCHAR(500),
|
|
source VARCHAR(20) DEFAULT 'manual' NOT NULL,
|
|
billable BOOLEAN DEFAULT true NOT NULL,
|
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
|
```
|
|
|
|
#### tasks
|
|
```sql
|
|
id SERIAL PRIMARY KEY,
|
|
project_id INTEGER REFERENCES projects(id) ON DELETE CASCADE NOT NULL,
|
|
name VARCHAR(200) NOT NULL,
|
|
description TEXT,
|
|
status VARCHAR(20) DEFAULT 'pending' NOT NULL,
|
|
priority VARCHAR(20) DEFAULT 'medium' NOT NULL,
|
|
assigned_to INTEGER REFERENCES users(id),
|
|
created_by INTEGER REFERENCES users(id) NOT NULL,
|
|
due_date DATE,
|
|
estimated_hours NUMERIC(5,2),
|
|
actual_hours NUMERIC(5,2),
|
|
started_at TIMESTAMP,
|
|
completed_at TIMESTAMP,
|
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL,
|
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL
|
|
```
|
|
|
|
#### settings
|
|
```sql
|
|
id SERIAL PRIMARY KEY,
|
|
timezone VARCHAR(50) DEFAULT 'Europe/Rome' NOT NULL,
|
|
currency VARCHAR(3) DEFAULT 'EUR' NOT NULL,
|
|
rounding_minutes INTEGER DEFAULT 1 NOT NULL,
|
|
single_active_timer BOOLEAN DEFAULT true NOT NULL,
|
|
allow_self_register BOOLEAN DEFAULT true NOT NULL,
|
|
idle_timeout_minutes INTEGER DEFAULT 30 NOT NULL,
|
|
backup_retention_days INTEGER DEFAULT 30 NOT NULL,
|
|
backup_time VARCHAR(5) DEFAULT '02:00' NOT NULL,
|
|
export_delimiter VARCHAR(1) DEFAULT ',' NOT NULL,
|
|
allow_analytics BOOLEAN DEFAULT true NOT NULL,
|
|
-- Company branding fields...
|
|
company_name VARCHAR(200) DEFAULT 'Your Company Name' NOT NULL,
|
|
company_address TEXT DEFAULT 'Your Company Address' NOT NULL,
|
|
company_email VARCHAR(200) DEFAULT 'info@yourcompany.com' NOT NULL,
|
|
company_phone VARCHAR(50) DEFAULT '+1 (555) 123-4567' NOT NULL,
|
|
company_website VARCHAR(200) DEFAULT 'www.yourcompany.com' NOT NULL,
|
|
company_logo_filename VARCHAR(255) DEFAULT '' NOT NULL,
|
|
company_tax_id VARCHAR(100) DEFAULT '' NOT NULL,
|
|
company_bank_info TEXT DEFAULT '' NOT NULL,
|
|
-- Invoice defaults...
|
|
invoice_prefix VARCHAR(10) DEFAULT 'INV' NOT NULL,
|
|
invoice_start_number INTEGER DEFAULT 1000 NOT NULL,
|
|
invoice_terms TEXT DEFAULT 'Payment is due within 30 days of invoice date.' NOT NULL,
|
|
invoice_notes TEXT DEFAULT 'Thank you for your business!' NOT NULL,
|
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL,
|
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL
|
|
```
|
|
|
|
#### invoices
|
|
```sql
|
|
id SERIAL PRIMARY KEY,
|
|
invoice_number VARCHAR(50) UNIQUE NOT NULL,
|
|
project_id INTEGER REFERENCES projects(id) ON DELETE CASCADE,
|
|
client_name VARCHAR(200) NOT NULL,
|
|
client_email VARCHAR(200),
|
|
client_address TEXT,
|
|
issue_date DATE NOT NULL,
|
|
due_date DATE NOT NULL,
|
|
status VARCHAR(20) DEFAULT 'draft' NOT NULL,
|
|
subtotal NUMERIC(10, 2) NOT NULL DEFAULT 0,
|
|
tax_rate NUMERIC(5, 2) NOT NULL DEFAULT 0,
|
|
tax_amount NUMERIC(10, 2) NOT NULL DEFAULT 0,
|
|
total_amount NUMERIC(10, 2) NOT NULL DEFAULT 0,
|
|
notes TEXT,
|
|
terms TEXT,
|
|
created_by INTEGER REFERENCES users(id) ON DELETE CASCADE,
|
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
|
```
|
|
|
|
#### invoice_items
|
|
```sql
|
|
id SERIAL PRIMARY KEY,
|
|
invoice_id INTEGER REFERENCES invoices(id) ON DELETE CASCADE,
|
|
description VARCHAR(500) NOT NULL,
|
|
quantity NUMERIC(10, 2) NOT NULL DEFAULT 1,
|
|
unit_price NUMERIC(10, 2) NOT NULL,
|
|
total_amount NUMERIC(10, 2) NOT NULL,
|
|
time_entry_ids VARCHAR(500),
|
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
|
```
|
|
|
|
## Performance Features
|
|
|
|
### Indexes
|
|
- User authentication: `username`, `role`
|
|
- Project queries: `client`, `status`
|
|
- Time tracking: `user_id`, `project_id`, `task_id`, `start_time`, `billable`
|
|
- Task management: `project_id`, `status`, `assigned_to`, `due_date`
|
|
- Invoice management: `project_id`, `status`, `issue_date`
|
|
|
|
### Triggers
|
|
- Automatic `updated_at` timestamp updates for all tables that have this column
|
|
- Ensures data consistency without application-level code
|
|
|
|
## Migration Support
|
|
|
|
The enhanced initialization script automatically handles:
|
|
|
|
- **New Tables**: Creates any missing tables
|
|
- **New Columns**: Adds missing columns to existing tables
|
|
- **Schema Updates**: Ensures existing databases are updated to current schema
|
|
- **Data Preservation**: Never drops existing data
|
|
|
|
## Verification
|
|
|
|
### Startup Verification
|
|
- Automatic verification after initialization
|
|
- Reports any issues found
|
|
- Ensures application only starts with correct database schema
|
|
|
|
### Manual Verification
|
|
```bash
|
|
# Run verification script independently
|
|
python docker/verify-database.py
|
|
|
|
# Check specific aspects
|
|
python docker/test-schema.py
|
|
```
|
|
|
|
## Error Handling
|
|
|
|
The enhanced startup procedure includes comprehensive error handling:
|
|
|
|
- **Database Connection**: Waits for database to be ready with retry logic
|
|
- **Schema Issues**: Reports specific problems and continues where possible
|
|
- **Non-critical Failures**: Continues operation even if some optional features fail
|
|
- **Detailed Logging**: Provides clear information about what succeeded and what failed
|
|
|
|
## Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
1. **Missing Tables**: Run `verify-database.py` to identify missing tables
|
|
2. **Schema Mismatches**: The enhanced script automatically adds missing columns
|
|
3. **Permission Issues**: Ensure database user has CREATE, ALTER, and INSERT privileges
|
|
4. **Connection Problems**: Check DATABASE_URL environment variable and network connectivity
|
|
|
|
### Debug Commands
|
|
|
|
```bash
|
|
# Check database schema
|
|
python docker/verify-database.py
|
|
|
|
# Test database connection
|
|
python docker/test-db.py
|
|
|
|
# View startup logs
|
|
docker logs <container_name>
|
|
```
|
|
|
|
## Benefits
|
|
|
|
1. **Reliability**: Ensures consistent database schema across all deployments
|
|
2. **Maintainability**: Single script handles all database setup
|
|
3. **Migration Safety**: Never loses existing data during updates
|
|
4. **Performance**: Proper indexes and triggers for optimal operation
|
|
5. **Debugging**: Comprehensive verification and error reporting
|
|
|
|
## Future Enhancements
|
|
|
|
- **Version Tracking**: Database schema version tracking for future migrations
|
|
- **Rollback Support**: Ability to rollback schema changes if needed
|
|
- **Performance Monitoring**: Database performance metrics collection
|
|
- **Automated Testing**: Integration with CI/CD pipeline for schema validation
|