Files
TimeTracker/FORCE_NO_CACHE_FIX.md
T
Dries Peeters 20824dbcb1 feat: Add customizable Kanban board columns and enhance CSRF configuration
This commit introduces a comprehensive Kanban board customization system and
improves CSRF token configuration for Docker deployments.

## Major Features

### 1. Customizable Kanban Board Columns
Add complete kanban column customization system allowing users to define
custom workflow states beyond the default columns.

**New Components:**
- Add KanbanColumn model with full CRUD operations (app/models/kanban_column.py)
- Add kanban routes blueprint with admin endpoints (app/routes/kanban.py)
- Add kanban column management templates (app/templates/kanban/)
- Add migration 019 for kanban_columns table (migrations/)

**Features:**
- Create unlimited custom columns with unique keys, labels, icons, and colors
- Drag-and-drop column reordering with position persistence
- Toggle column visibility without deletion
- Protected system columns (todo, in_progress, done) prevent accidental deletion
- Complete state marking for columns that should mark tasks as done
- Real-time updates via SocketIO broadcasts when columns change
- Font Awesome icon support (5000+ icons)
- Bootstrap color scheme integration
- Comprehensive validation and error handling

**Integration:**
- Update Task model to work with dynamic column statuses (app/models/task.py)
- Update task routes to use kanban column API (app/routes/tasks.py)
- Update project routes to fetch active columns (app/routes/projects.py)
- Add kanban column management links to base template (app/templates/base.html)
- Update kanban board templates to render dynamic columns (app/templates/tasks/)
- Add cache prevention headers to force fresh column data

**API Endpoints:**
- GET /api/kanban/columns - Fetch all active columns
- POST /api/kanban/columns/reorder - Reorder columns
- GET /kanban/columns - Column management interface (admin only)
- POST /kanban/columns/create - Create new column (admin only)
- POST /kanban/columns/<id>/edit - Edit column (admin only)
- POST /kanban/columns/<id>/delete - Delete column (admin only)
- POST /kanban/columns/<id>/toggle - Toggle column visibility (admin only)

### 2. Enhanced CSRF Configuration
Improve CSRF token configuration and documentation for Docker deployments.

**Configuration Updates:**
- Add WTF_CSRF_ENABLED environment variable to all docker-compose files
- Add WTF_CSRF_TIME_LIMIT environment variable with 1-hour default
- Update app/config.py to read CSRF settings from environment
- Add SECRET_KEY validation in app/__init__.py to prevent production deployment
  with default keys

**Docker Compose Updates:**
- docker-compose.yml: CSRF enabled by default for security testing
- docker-compose.remote.yml: CSRF always enabled in production
- docker-compose.remote-dev.yml: CSRF enabled with production-like settings
- docker-compose.local-test.yml: CSRF can be disabled for local testing
- Add helpful comments explaining each CSRF-related environment variable
- Update env.example with CSRF configuration examples

**Verification Scripts:**
- Add scripts/verify_csrf_config.sh for Unix systems
- Add scripts/verify_csrf_config.bat for Windows systems
- Scripts check SECRET_KEY, CSRF_ENABLED, and CSRF_TIME_LIMIT settings

### 3. Database Initialization Improvements
- Update app/__init__.py to run pending migrations on startup
- Add automatic kanban column initialization after migrations
- Improve error handling and logging during database setup

### 4. Configuration Management
- Update app/config.py with new CSRF and kanban-related settings
- Add environment variable parsing with sensible defaults
- Improve configuration validation and error messages

## Documentation

### New Documentation Files
- CUSTOM_KANBAN_README.md: Quick start guide for kanban customization
- KANBAN_CUSTOMIZATION.md: Detailed technical documentation
- IMPLEMENTATION_SUMMARY.md: Implementation details and architecture
- KANBAN_AUTO_REFRESH_COMPLETE.md: Real-time update system documentation
- KANBAN_REFRESH_FINAL_FIX.md: Cache and refresh troubleshooting
- KANBAN_REFRESH_SOLUTION.md: Technical solution for data freshness
- docs/CSRF_CONFIGURATION.md: Comprehensive CSRF setup guide
- CSRF_DOCKER_CONFIGURATION_SUMMARY.md: Docker-specific CSRF setup
- CSRF_TROUBLESHOOTING.md: Common CSRF issues and solutions
- APPLY_KANBAN_MIGRATION.md: Migration application guide
- APPLY_FIXES_NOW.md: Quick fix reference
- DEBUG_KANBAN_COLUMNS.md: Debugging guide
- DIAGNOSIS_STEPS.md: System diagnosis procedures
- BROWSER_CACHE_FIX.md: Browser cache troubleshooting
- FORCE_NO_CACHE_FIX.md: Cache prevention solutions
- SESSION_CLOSE_ERROR_FIX.md: Session handling fixes
- QUICK_FIX.md: Quick reference for common fixes

### Updated Documentation
- README.md: Add kanban customization feature description
- Update project documentation with new features

## Testing

### New Test Files
- test_kanban_refresh.py: Test kanban column refresh functionality

## Technical Details

**Database Changes:**
- New table: kanban_columns with 11 columns
- Indexes on: key, position
- Default data: 4 system columns (todo, in_progress, review, done)
- Support for both SQLite (development) and PostgreSQL (production)

**Real-Time Updates:**
- SocketIO events: 'kanban_columns_updated' with action type
- Automatic page refresh when columns are created/updated/deleted/reordered
- Prevents stale data by expiring SQLAlchemy caches after changes

**Security:**
- Admin-only access to column management
- CSRF protection on all column mutation endpoints
- API endpoints exempt from CSRF (use JSON and other auth mechanisms)
- System column protection prevents data integrity issues
- Validation prevents deletion of columns with active tasks

**Performance:**
- Efficient querying with position-based ordering
- Cached column data with cache invalidation on changes
- No-cache headers on API responses to prevent stale data
- Optimized database indexes for fast lookups

## Breaking Changes

None. This is a fully backward-compatible addition.

Existing workflows continue to work with the default columns.
Custom columns are opt-in via the admin interface.

## Migration Notes

1. Run migration 019 to create kanban_columns table
2. Default columns are initialized automatically on first run
3. No data migration needed for existing tasks
4. Existing task statuses map to new column keys

## Environment Variables

New environment variables (all optional with defaults):
- WTF_CSRF_ENABLED: Enable/disable CSRF protection (default: true)
- WTF_CSRF_TIME_LIMIT: CSRF token expiration in seconds (default: 3600)
- SECRET_KEY: Required in production, must be cryptographically secure

See env.example for complete configuration reference.

## Deployment Notes
2025-10-11 19:56:45 +02:00

4.2 KiB

Nuclear Option: Disable All Caching

If caching is still an issue, here's the nuclear option that FORCES fresh data every time:

Option 1: Add Query Expiration Decorator

Create a decorator that always expires before queries:

# In app/models/kanban_column.py

from functools import wraps
from app import db

def force_fresh(f):
    """Decorator to force fresh database queries"""
    @wraps(f)
    def wrapper(*args, **kwargs):
        db.session.expire_all()
        return f(*args, **kwargs)
    return wrapper

class KanbanColumn(db.Model):
    # ... existing code ...
    
    @classmethod
    @force_fresh
    def get_active_columns(cls):
        """Get all active columns ordered by position"""
        # Will always expire cache before running
        return db.session.query(cls).filter_by(is_active=True).order_by(cls.position.asc()).all()

Option 2: Disable SQLAlchemy Query Cache Entirely

In app/config.py or app/__init__.py:

# Disable query caching for KanbanColumn queries
app.config['SQLALCHEMY_ECHO'] = False  # Don't log queries
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False  # Already set

And in the model:

@classmethod
def get_active_columns(cls):
    """Get all active columns - always fresh from database"""
    # Use .from_statement() to bypass query cache
    from sqlalchemy import text
    result = db.session.execute(
        text("SELECT * FROM kanban_columns WHERE is_active = true ORDER BY position ASC")
    )
    return [cls(**dict(row)) for row in result]

Option 3: Add Cache Buster to Every Route

# In all routes that load columns
from time import time

@tasks_bp.route('/tasks')
@login_required
def list_tasks():
    # Force refresh
    db.session.commit()  # Commit any pending transactions
    db.session.expire_all()  # Clear cache
    db.session.close()  # Close session
    
    # Get fresh data
    kanban_columns = KanbanColumn.get_active_columns()
    
    # ... rest of route

Option 4: Restart Gunicorn Workers After Changes

Add this to column modification routes:

import os
import signal

# After successful column modification
if os.path.exists('/tmp/gunicorn.pid'):
    with open('/tmp/gunicorn.pid') as f:
        pid = int(f.read().strip())
    os.kill(pid, signal.SIGHUP)  # Reload workers

Option 5: Use Timestamp-Based Cache Busting

# Add to KanbanColumn model
import time

_last_modified = time.time()

@classmethod
def touch(cls):
    """Mark columns as modified"""
    global _last_modified
    _last_modified = time.time()
    db.session.expire_all()

@classmethod
def get_active_columns(cls):
    """Get columns with cache busting"""
    # Timestamp forces query to be different each time it changes
    return db.session.query(cls).filter_by(
        is_active=True
    ).order_by(
        cls.position.asc()
    ).all()

# Call after modifications
def create_column(...):
    # ... create column ...
    KanbanColumn.touch()

Option 6: Multiple Worker Issue

If you have multiple gunicorn workers, they each have their own cache. Fix:

# In docker-compose.yml or docker entrypoint
# Reduce to 1 worker temporarily
CMD ["gunicorn", "--workers=1", "--bind=0.0.0.0:8080", "app:app"]

Or use Redis for shared cache:

# Install redis
pip install redis flask-caching

# In app/__init__.py
from flask_caching import Cache
cache = Cache(config={'CACHE_TYPE': 'redis', 'CACHE_REDIS_URL': 'redis://redis:6379/0'})

# In models
@cache.memoize(timeout=5)  # 5 second cache
def get_active_columns():
    # ... query ...

# After modifications
cache.delete_memoized(get_active_columns)

Which One to Try?

Start with Option 3 (simplest):

  • Add cache clearing to every route
  • Most likely to work immediately

Then try Option 1 (cleanest):

  • Decorator approach is elegant
  • Easy to maintain

If still failing, try Option 6:

  • Multiple workers are probably the issue
  • Reduce to 1 worker temporarily to test

Test After Each Change

# 1. Make the change
# 2. Restart container
docker-compose restart app

# 3. Test
# Go to /kanban/columns, create column
# Go to /tasks, should appear immediately

# 4. Check logs
docker logs timetracker_app_1 | grep -i "kanban\|column"

Let me know which one you want to try first!