TimeTracker/docs/cicd/CI_CD_DOCUMENTATION.md

TimeTracker CI/CD Pipeline Documentation

📋 Table of Contents

1. Overview
2. Pipeline Architecture
3. Test Strategy
4. Workflow Details
5. Docker Registry
6. Deployment
7. Configuration
8. Troubleshooting

---

Overview

The TimeTracker project implements a comprehensive CI/CD pipeline using GitHub Actions. The pipeline automates:

• Continuous Integration (CI): Automated testing on every pull request
• Continuous Deployment (CD): Automated builds and deployments to container registry
• Quality Assurance: Code quality checks, security scanning, and comprehensive testing
• Multi-platform Support: Builds for AMD64 and ARM64 architectures

Key Features

✅ Multi-level Testing: Smoke, unit, integration, security, and database tests
✅ Parallel Execution: Fast feedback with parallel test jobs
✅ Multi-platform Builds: Support for x86_64 and ARM64
✅ Automated Releases: Automatic versioning and release creation
✅ Security Scanning: Bandit and Safety security checks
✅ Code Quality: Black, Flake8, and isort validation
✅ Coverage Reporting: Integrated Codecov support

---

Pipeline Architecture

┌─────────────────────────────────────────────────────────────┐
│                     Pull Request / Push                      │
└──────────────────────┬──────────────────────────────────────┘
                       │
                       ▼
           ┌───────────────────────┐
           │    Smoke Tests (5m)   │ ◄── Fastest critical tests
           └───────────┬───────────┘
                       │
       ┌───────────────┼───────────────┐
       ▼               ▼               ▼
  ┌─────────┐   ┌────────────┐  ┌──────────┐
  │  Unit   │   │Integration │  │ Security │
  │ Tests   │   │   Tests    │  │  Tests   │
  └────┬────┘   └─────┬──────┘  └────┬─────┘
       │              │              │
       └──────────────┴──────────────┘
                      │
                      ▼
              ┌──────────────┐
              │ Docker Build │
              └──────┬───────┘
                     │
        ┌────────────┴────────────┐
        ▼                         ▼
   ┌─────────┐            ┌──────────────┐
   │  develop│            │ main/master  │
   │  Branch │            │   Branch     │
   └────┬────┘            └──────┬───────┘
        │                        │
        ▼                        ▼
   ┌─────────┐            ┌─────────────┐
   │   Dev   │            │   Release   │
   │  Image  │            │   Image     │
   └─────────┘            └─────────────┘

---

Test Strategy

Test Levels

The CI pipeline uses pytest markers to organize tests into different levels:

1. Smoke Tests (@pytest.mark.smoke)

• Duration: < 1 minute
• Purpose: Quick sanity checks
• When: Every commit, fastest feedback
• Examples:
  • Health check endpoint
  • Basic model creation
  • Login page accessibility

2. Unit Tests (@pytest.mark.unit)

• Duration: 2-5 minutes
• Purpose: Test individual components in isolation
• When: Every PR, parallel execution
• Examples:
  • Model methods
  • Utility functions
  • Business logic

3. Integration Tests (@pytest.mark.integration)

• Duration: 5-10 minutes
• Purpose: Test component interactions
• When: Every PR
• Examples:
  • API endpoints
  • Database operations
  • Route handlers

4. Security Tests (@pytest.mark.security)

• Duration: 3-5 minutes
• Purpose: Security vulnerability testing
• When: Every PR
• Examples:
  • SQL injection
  • XSS protection
  • Authorization checks

5. Database Tests (@pytest.mark.database)

• Duration: 5-10 minutes
• Purpose: Database-specific testing
• When: Every PR (PostgreSQL & SQLite)
• Examples:
  • Migrations
  • Relationships
  • Cascade operations

Running Tests Locally

# Install test dependencies
pip install -r requirements.txt
pip install -r requirements-test.txt

# Run all tests
pytest

# Run specific test levels
pytest -m smoke          # Smoke tests only
pytest -m unit           # Unit tests only
pytest -m integration    # Integration tests only
pytest -m security       # Security tests only

# Run tests in parallel
pytest -n auto

# Run with coverage
pytest --cov=app --cov-report=html

# Run specific test file
pytest tests/test_routes.py

# Run specific test
pytest tests/test_routes.py::test_health_check

---

Workflow Details

1. CI - Comprehensive Pipeline (ci-comprehensive.yml)

Triggers:

• Pull requests to main or develop
• Pushes to develop

Jobs:

Smoke Tests (5 min)

- Fast critical tests
- No database required
- Fails fast on critical issues

Unit Tests (10 min, parallel)

- Runs in parallel for different components
- Models, routes, API, utils
- SQLite in-memory database

Integration Tests (15 min)

- PostgreSQL service
- Full database interactions
- API endpoint testing

Security Tests (10 min)

- Pytest security markers
- Bandit security linting
- Safety dependency scanning

Database Tests (15 min each)

- PostgreSQL 16
- SQLite
- Migration testing

Code Quality (10 min)

- Flake8 linting
- Black formatting check
- isort import sorting

Docker Build (20 min)

- Multi-platform build test
- Container startup verification
- Health check validation

2. CD - Development Builds (cd-development.yml)

Triggers:

• Pushes to develop branch
• Manual workflow dispatch

Process:

1. Quick test suite (smoke + unit + critical integration)
2. Build multi-platform Docker image (AMD64, ARM64)
3. Push to ghcr.io with tags:
   • develop (latest development)
   • dev-{date}-{time}
   • dev-{sha}
4. Create development release
5. Generate deployment manifest

Tags:

ghcr.io/{owner}/{repo}:develop
ghcr.io/{owner}/{repo}:dev-20240109-143022
ghcr.io/{owner}/{repo}:dev-abc1234

3. CD - Release Builds (cd-release.yml)

Triggers:

• Pushes to main/master branch
• Git tags matching v*.*.*
• GitHub releases
• Manual workflow dispatch

Process:

1. Full test suite (all tests, ~30 minutes)
2. Security audit (Bandit + Safety)
3. Version determination
4. Multi-platform build (AMD64, ARM64)
5. Push to registry with tags:
   • Semantic version tags
   • latest
   • stable
6. Create GitHub release with:
   • Changelog
   • Deployment manifests
   • Docker compose files
   • Kubernetes YAML

Tags:

ghcr.io/{owner}/{repo}:v1.2.3
ghcr.io/{owner}/{repo}:1.2
ghcr.io/{owner}/{repo}:1
ghcr.io/{owner}/{repo}:latest
ghcr.io/{owner}/{repo}:stable

4. Additional Workflows

Migration Check (migration-check.yml)

• Validates database migrations
• Checks for schema drift
• Tests rollback safety
• Runs on model/migration changes

Static Analysis (static.yml)

• CodeQL security scanning
• Dependency graph updates

---

Docker Registry

GitHub Container Registry (GHCR)

Images are published to GitHub Container Registry at:

ghcr.io/{owner}/{repo}

Authentication

# Login to GHCR
echo $GITHUB_TOKEN | docker login ghcr.io -u USERNAME --password-stdin

# Pull an image
docker pull ghcr.io/{owner}/{repo}:latest

# For private repositories
docker pull ghcr.io/{owner}/{repo}:develop

Image Tags

Tag	Purpose	Updated	Platforms
latest	Latest stable release	On release	AMD64, ARM64
stable	Last non-prerelease	On release	AMD64, ARM64
develop	Latest development	On develop push	AMD64, ARM64
v1.2.3	Specific version	On release	AMD64, ARM64
1.2	Minor version	On release	AMD64, ARM64
1	Major version	On release	AMD64, ARM64

Image Size Optimization

The Docker image includes:

• Python 3.11 slim base
• System dependencies (WeasyPrint, PostgreSQL client)
• Multi-stage builds (future enhancement)
• Layer caching for faster builds

---

Deployment

Development Environment

# Pull development image
docker pull ghcr.io/{owner}/{repo}:develop

# Run with docker-compose
docker-compose -f docker-compose.yml up -d

# Or use the generated deployment manifest
docker-compose -f deployment-dev.yml up -d

Production Environment

Docker Compose

# Download production compose file
wget https://github.com/{owner}/{repo}/releases/latest/download/docker-compose.production.yml

# Configure environment
cat > .env << EOF
SECRET_KEY=your-secret-key-here
POSTGRES_PASSWORD=your-db-password
POSTGRES_USER=timetracker
POSTGRES_DB=timetracker
TZ=Europe/Brussels
CURRENCY=EUR
EOF

# Deploy
docker-compose -f docker-compose.production.yml up -d

Kubernetes

# Download K8s manifest
wget https://github.com/{owner}/{repo}/releases/latest/download/k8s-deployment.yml

# Create secrets
kubectl create secret generic timetracker-secrets \
  --from-literal=database-url='postgresql://user:pass@host:5432/db' \
  --from-literal=secret-key='your-secret-key'

# Deploy
kubectl apply -f k8s-deployment.yml

# Check status
kubectl get pods -l app=timetracker
kubectl get svc timetracker

Manual Docker Run

docker run -d \
  --name timetracker \
  -p 8080:8080 \
  -e DATABASE_URL="postgresql://user:pass@host:5432/db" \
  -e SECRET_KEY="your-secret-key" \
  -e TZ="Europe/Brussels" \
  --restart unless-stopped \
  ghcr.io/{owner}/{repo}:latest

---

Configuration

GitHub Secrets

Required secrets (already configured via GITHUB_TOKEN):

• ✅ GITHUB_TOKEN - Automatic, used for GHCR authentication

Optional secrets:

• CODECOV_TOKEN - For Codecov integration
• SLACK_WEBHOOK - For Slack notifications
• DOCKER_HUB_USERNAME - If publishing to Docker Hub
• DOCKER_HUB_TOKEN - If publishing to Docker Hub

Environment Variables

Build-time (Docker)

ARG APP_VERSION=dev-0
ENV APP_VERSION=${APP_VERSION}

Runtime

# Required
DATABASE_URL=postgresql://user:pass@host:5432/db
SECRET_KEY=your-secret-key-here

# Optional
TZ=Europe/Brussels
CURRENCY=EUR
FLASK_ENV=production
LOG_LEVEL=INFO

Pytest Configuration

Configured in pytest.ini:

• Test discovery patterns
• Coverage settings
• Test markers
• Output options

Test Requirements

Specified in requirements-test.txt:

• pytest and plugins
• Code quality tools
• Security scanners
• Test utilities

---

Troubleshooting

Common Issues

1. Tests Failing Locally But Passing in CI

Cause: Database state, environment differences
Solution:

# Clean test database
rm -f test.db

# Use same Python version
pyenv install 3.11
pyenv local 3.11

# Reinstall dependencies
pip install -r requirements.txt -r requirements-test.txt

2. Docker Build Fails

Cause: Missing dependencies, network issues
Solution:

# Clear Docker cache
docker builder prune -a

# Build with no cache
docker build --no-cache -t timetracker:test .

# Check logs
docker logs <container-id>

3. Coverage Below Threshold

Cause: New code not tested
Solution:

# Run coverage locally
pytest --cov=app --cov-report=html

# Open coverage report
open htmlcov/index.html

# Add tests for uncovered code

4. Security Vulnerabilities Detected

Cause: Outdated dependencies
Solution:

# Check vulnerabilities
safety check --file requirements.txt

# Update dependencies
pip install --upgrade <package>

# Update requirements.txt
pip freeze > requirements.txt

5. Migration Tests Failing

Cause: Schema drift, missing migrations
Solution:

# Check current migration
flask db current

# Create new migration
flask db migrate -m "Description"

# Apply migration
flask db upgrade

# Test rollback
flask db downgrade -1
flask db upgrade

Debug Workflow Runs

View Logs

# Via GitHub CLI
gh run list
gh run view <run-id>
gh run view <run-id> --log

# Download artifacts
gh run download <run-id>

Re-run Failed Jobs

1. Go to Actions tab in GitHub
2. Select failed workflow run
3. Click "Re-run failed jobs"

Cancel Running Workflow

gh run cancel <run-id>

Performance Optimization

Speed Up Tests

# Run tests in parallel
pytest -n auto

# Skip slow tests
pytest -m "not slow"

# Run only failed tests
pytest --lf

Speed Up Builds

• Enable Docker layer caching
• Use build cache
• Parallelize test jobs
• Use smaller base images

---

Best Practices

For Developers

1. Run Tests Locally: Before pushing, run at least smoke and unit tests

   pytest -m "smoke or unit"
   

2. Write Tests: Add tests for new features using appropriate markers

   @pytest.mark.unit
   @pytest.mark.models
   def test_new_feature():
       pass
   

3. Keep PRs Small: Smaller PRs = faster CI, easier review

4. Fix Failures Quickly: Don't let broken tests sit

5. Check Coverage: Aim for >80% coverage on new code

For Maintainers

1. Review Test Reports: Check test results and coverage before merging

2. Monitor Build Times: Keep CI under 15 minutes for PRs

3. Update Dependencies: Regular security updates

4. Version Properly: Use semantic versioning

5. Document Changes: Update CHANGELOG.md

---

Metrics and Monitoring

CI/CD Metrics

Track these metrics for health:

• ✅ Test Success Rate: > 95%
• ✅ Build Time: < 15 minutes (PR), < 30 minutes (release)
• ✅ Coverage: > 80%
• ✅ Deployment Frequency: Multiple times per day (develop)
• ✅ Mean Time to Recovery: < 1 hour

Dashboard

View metrics at:

• GitHub Actions tab
• Codecov dashboard (if configured)
• Docker Hub / GHCR for image stats

---

Future Enhancements

Planned improvements:

• Automated performance testing
• E2E testing with Playwright/Selenium
• Automated security scanning (Dependabot)
• Blue-green deployments
• Canary releases
• Automated rollback on failures
• Multi-environment deployments (staging, production)
• Integration with monitoring (Datadog, Sentry)

---

Support

For issues or questions:

1. Check this documentation
2. Review GitHub Actions logs
3. Open an issue on GitHub
4. Contact the maintainers

---

Last Updated: 2025-01-09
Version: 1.0
Maintained by: TimeTracker Team