mirror/TimeTracker
1
0
Fork 0
mirror of https://github.com/DRYTRIX/TimeTracker.git synced 2026-05-17 10:29:49 -05:00
Code Issues Packages Projects Releases Wiki Activity
Files
96e0acbbea12e28dc877e54dc1373e32ade501fb
TimeTracker/app/integrations/gitlab.py
T
Dries Peeters 1836cb3c2d chore(typing): resolve mypy errors and harden type checking 
Drives ``mypy app/`` from 567 errors in 208 files to 0 errors across the
376 source files checked by ``./scripts/run-ci-local.sh code-quality``.

Configuration & dependencies
- pyproject.toml: enable implicit_optional (Flask-style ``x: str = None``
  defaults), silence truthy-function/truthy-bool (legitimate import-guard
  checks like ``KanbanColumn``), and disable warn_return_any (SQLAlchemy
  1.x ``Query`` API returns Any pervasively). Add module overrides for
  ``app.models.*``, repositories, base CRUD service, and known
  ``joinedload`` / ``Query.paginate`` callers where mypy cannot model the
  Flask-SQLAlchemy runtime API without a plugin.
- requirements-test.txt: pin ``types-requests``, ``types-bleach``,
  ``types-Markdown``, ``types-python-dateutil`` so mypy stops complaining
  about missing stubs.

Latent bugs fixed while driving mypy to zero
- app/utils/logger.py, app/utils/datetime_utils.py: drop imports of
  symbols that don't exist (``get_performance_metrics``,
  ``from_app_timezone``, ``to_app_timezone``) — these would have raised
  at import time on first use.
- app/services/currency_service.py: ``from typing import Decimal`` was a
  bug (typing has no Decimal); switch to ``decimal.Decimal`` and rename
  the ``D`` alias.
- app/utils/env_validation.py, app/utils/role_migration.py: ``Dict[str,
  any]`` → ``Dict[str, Any]`` (built-in ``any`` is not a type).
- app/utils/email.py: introduce ``send_template_email`` and update the
  three callers (``client_approval_service``,
  ``client_notification_service``, ``workflow_engine``) that were
  passing ``to=``/``template=``/etc. to ``send_email`` whose signature
  doesn't accept them — calls would have raised TypeError at runtime.
- app/services/permission_service.py: rewrite ``grant_permission`` /
  ``revoke_permission`` to use the actual ``Role`` ↔ ``Permission``
  many-to-many relationship; the old code referenced non-existent
  ``Permission.role_id`` / ``Permission.granted`` columns.
- app/services/gps_tracking_service.py: pass the required ``title`` and
  ``expense_date`` fields when creating mileage ``Expense`` rows.
- app/services/workflow_engine.py: ``_perform_action`` now forwards the
  ``rule`` argument to ``_action_log_time``, and ``_action_webhook``
  short-circuits when ``url`` is missing.
- app/services/time_tracking_service.py: validate ``start_time`` /
  ``end_time`` before comparing them.
- app/services/export_service.py: build CSV in a ``StringIO`` then wrap
  the bytes in ``BytesIO`` — ``csv.writer`` requires text I/O.
- app/integrations/peppol_smp.py: avoid attribute access on ``None`` in
  the SMP ``href`` fallback.
- app/integrations/{github,gitlab,slack}.py: coerce query-string params
  to strings so ``requests.get(params=...)`` matches the typed signature
  (and is what the HTTP layer expects anyway).
- app/integrations/{xero,quickbooks}.py: guard ``get_access_token()``
  returning ``None`` before calling private ``_api_request`` helpers.

Annotation-only changes
- Add ``Dict[str, Any]`` / ``list`` / ``Optional[...]`` annotations to
  service dict-literals that mypy could not infer from heterogeneous
  values (``ai_suggestion_service``, ``ai_categorization_service``,
  ``custom_report_service``, ``unpaid_hours_service``,
  ``integration_service``, ``invoice_service``, ``backup_service``,
  ``inventory_report_service``, ``analytics_service``, etc.).
- ``app/utils/event_bus.py``: ``emit_event`` accepts ``str |
  WebhookEvent`` and normalizes to ``str`` so all call-sites type-check.
- ``app/utils/api_responses.py``: introduce ``ApiResponse`` alias for
  ``Response | tuple[Response, int] | tuple[str, int]``.
- ``app/utils/budget_forecasting.py``: forecasting helpers return
  ``Optional[Dict]`` (they already returned ``None`` when the project
  was missing).
- ``app/utils/pdf_generator_reportlab.py``: ``_normalize_color`` is
  ``Optional[str]``.
- ``app/utils/pdfa3.py``: remove invalid ``force_version=None`` retry
  call.
- Narrow ``type: ignore`` markers on optional-dependency fallbacks
  (``redis``, ``bleach``, ``markdown``, ``babel``,
  ``powerpoint_export``) and on the documented ``requests.Session``
  / ``RotatingFileHandler`` typeshed limitations.
2026-05-13 10:32:06 +02:00

457 lines
18 KiB
Python
Raw Blame History

"""
GitLab integration connector.
Sync issues and track time from GitLab.
"""
import os
from datetime import datetime, timedelta
from typing import Any, Dict, List, Optional
import requests
from app.integrations.base import BaseConnector
class GitLabConnector(BaseConnector):
"""GitLab integration connector."""
display_name = "GitLab"
description = "Sync issues and track time from GitLab"
icon = "gitlab"
@property
def provider_name(self) -> str:
return "gitlab"
def _get_base_url(self) -> str:
"""Get GitLab instance URL from settings."""
from app.models import Settings
settings = Settings.get_settings()
creds = settings.get_integration_credentials("gitlab")
instance_url = creds.get("instance_url") or os.getenv("GITLAB_INSTANCE_URL", "https://gitlab.com")
return instance_url.rstrip("/")
def get_authorization_url(self, redirect_uri: str, state: str = None) -> str:
"""Get GitLab OAuth authorization URL."""
from app.models import Settings
settings = Settings.get_settings()
creds = settings.get_integration_credentials("gitlab")
client_id = creds.get("client_id") or os.getenv("GITLAB_CLIENT_ID")
base_url = self._get_base_url()
if not client_id:
raise ValueError("GITLAB_CLIENT_ID not configured")
scopes = ["api", "read_user", "read_repository", "write_repository"]
auth_url = f"{base_url}/oauth/authorize"
params = {
"client_id": client_id,
"redirect_uri": redirect_uri,
"response_type": "code",
"scope": " ".join(scopes),
"state": state or "",
}
query_string = "&".join([f"{k}={v}" for k, v in params.items()])
return f"{auth_url}?{query_string}"
def exchange_code_for_tokens(self, code: str, redirect_uri: str) -> Dict[str, Any]:
"""Exchange authorization code for tokens."""
from app.models import Settings
settings = Settings.get_settings()
creds = settings.get_integration_credentials("gitlab")
client_id = creds.get("client_id") or os.getenv("GITLAB_CLIENT_ID")
client_secret = creds.get("client_secret") or os.getenv("GITLAB_CLIENT_SECRET")
base_url = self._get_base_url()
if not client_id or not client_secret:
raise ValueError("GitLab OAuth credentials not configured")
token_url = f"{base_url}/oauth/token"
response = requests.post(
token_url,
data={
"client_id": client_id,
"client_secret": client_secret,
"code": code,
"grant_type": "authorization_code",
"redirect_uri": redirect_uri,
},
)
response.raise_for_status()
data = response.json()
expires_at = None
if "expires_in" in data:
expires_at = datetime.utcnow() + timedelta(seconds=data["expires_in"])
# Get user info
user_info = {}
if "access_token" in data:
try:
user_response = requests.get(
f"{base_url}/api/v4/user", headers={"Authorization": f"Bearer {data['access_token']}"}
)
if user_response.status_code == 200:
user_data = user_response.json()
user_info = {
"id": user_data.get("id"),
"username": user_data.get("username"),
"name": user_data.get("name"),
"email": user_data.get("email"),
}
except Exception as e:
# Log error but don't fail - user info is optional
import logging
logger = logging.getLogger(__name__)
logger.debug(f"Could not fetch GitLab user info: {e}")
return {
"access_token": data.get("access_token"),
"refresh_token": data.get("refresh_token"),
"expires_at": expires_at.isoformat() if expires_at else None,
"token_type": data.get("token_type", "Bearer"),
"scope": data.get("scope"),
"extra_data": user_info,
}
def refresh_access_token(self) -> Dict[str, Any]:
"""Refresh access token."""
if not self.credentials or not self.credentials.refresh_token:
raise ValueError("No refresh token available")
from app.models import Settings
settings = Settings.get_settings()
creds = settings.get_integration_credentials("gitlab")
client_id = creds.get("client_id") or os.getenv("GITLAB_CLIENT_ID")
client_secret = creds.get("client_secret") or os.getenv("GITLAB_CLIENT_SECRET")
base_url = self._get_base_url()
token_url = f"{base_url}/oauth/token"
response = requests.post(
token_url,
data={
"client_id": client_id,
"client_secret": client_secret,
"refresh_token": self.credentials.refresh_token,
"grant_type": "refresh_token",
},
)
response.raise_for_status()
data = response.json()
expires_at = None
if "expires_in" in data:
expires_at = datetime.utcnow() + timedelta(seconds=data["expires_in"])
# Update credentials
self.credentials.access_token = data.get("access_token")
if "refresh_token" in data:
self.credentials.refresh_token = data.get("refresh_token")
if expires_at:
self.credentials.expires_at = expires_at
from app.utils.db import safe_commit
safe_commit("refresh_gitlab_token", {"integration_id": self.integration.id})
return {
"access_token": data.get("access_token"),
"expires_at": expires_at.isoformat() if expires_at else None,
}
def test_connection(self) -> Dict[str, Any]:
"""Test connection to GitLab."""
token = self.get_access_token()
if not token:
return {"success": False, "message": "No access token available"}
base_url = self._get_base_url()
api_url = f"{base_url}/api/v4/user"
try:
response = requests.get(api_url, headers={"Authorization": f"Bearer {token}"})
if response.status_code == 200:
user_data = response.json()
return {"success": True, "message": f"Connected as {user_data.get('username', 'Unknown')}"}
else:
return {"success": False, "message": f"API returned status {response.status_code}"}
except Exception as e:
return {"success": False, "message": f"Connection error: {str(e)}"}
def sync_data(self, sync_type: str = "full") -> Dict[str, Any]:
"""Sync issues from GitLab repositories into TimeTracker projects and tasks."""
from app import db
from app.models import Project, Task
from app.utils.integration_sync_context import (
ensure_project_integration_fields,
find_project_by_integration_ref,
find_task_by_integration_ref,
require_sync_context,
set_task_integration_ref,
)
token = self.get_access_token()
if not token:
return {"success": False, "message": "No access token available"}
try:
actor_id, client_id = require_sync_context(self.integration)
except ValueError as e:
return {"success": False, "message": str(e)}
base_url = self._get_base_url()
headers = {"Authorization": f"Bearer {token}"}
synced_count = 0
errors = []
raw_ids = self.integration.config.get("repository_ids", []) if self.integration else []
repo_ids: List[int] = []
if isinstance(raw_ids, str):
for part in raw_ids.split(","):
part = part.strip()
if part.isdigit():
repo_ids.append(int(part))
elif isinstance(raw_ids, list):
for x in raw_ids:
try:
repo_ids.append(int(x))
except (TypeError, ValueError):
continue
try:
if not repo_ids:
projects_response = requests.get(
f"{base_url}/api/v4/projects",
headers=headers,
params={"membership": True, "per_page": 100},
timeout=30,
)
if projects_response.status_code != 200:
return {
"success": False,
"message": f"Could not list GitLab projects: HTTP {projects_response.status_code}",
}
repo_ids = [p["id"] for p in projects_response.json()[:20]]
for repo_id in repo_ids:
try:
pr = requests.get(f"{base_url}/api/v4/projects/{repo_id}", headers=headers, timeout=30)
if pr.status_code != 200:
errors.append(f"GitLab project {repo_id}: HTTP {pr.status_code}")
continue
gl_project = pr.json()
path = gl_project.get("path_with_namespace") or gl_project.get("name") or str(repo_id)
path = str(path)[:200]
project_ref = str(repo_id)
project = find_project_by_integration_ref(client_id, "gitlab", project_ref)
if not project:
project = Project.query.filter_by(client_id=client_id, name=path).first()
if not project:
project = Project(
name=path,
client_id=client_id,
description=(gl_project.get("description") or "") or f"GitLab: {path}",
status="active",
)
db.session.add(project)
db.session.flush()
ensure_project_integration_fields(
project,
source="gitlab",
ref=project_ref,
display_name=path,
description=(gl_project.get("description") or "") or f"GitLab: {path}",
)
issues_response = requests.get(
f"{base_url}/api/v4/projects/{repo_id}/issues",
headers=headers,
params={"state": "opened", "per_page": "100"},
timeout=30,
)
if issues_response.status_code != 200:
errors.append(f"GitLab issues for project {repo_id}: HTTP {issues_response.status_code}")
continue
for issue in issues_response.json():
iid = issue.get("iid")
if not iid:
continue
title = (issue.get("title") or "Issue").strip()[:180]
issue_ref = f"{repo_id}:{iid}"
desc = (issue.get("description") or "").strip()
web_url = issue.get("web_url") or ""
if web_url:
desc = f"{desc}\n\nGitLab: {web_url}" if desc else f"GitLab: {web_url}"
state = (issue.get("state") or "").lower()
task_status = "done" if state in ("closed", "merged") else "todo"
task_name = f"#{iid}: {title}"[:200]
task = find_task_by_integration_ref(project.id, issue_ref, source="gitlab")
if not task:
task = Task(
project_id=project.id,
name=task_name,
description=desc or None,
status=task_status,
created_by=actor_id,
)
db.session.add(task)
db.session.flush()
else:
task.name = task_name
task.description = desc or None
task.status = task_status
set_task_integration_ref(
task,
source="gitlab",
ref=issue_ref,
extra={
"gitlab_project_id": repo_id,
"iid": iid,
"id": issue.get("id"),
"url": web_url,
},
)
synced_count += 1
except Exception as e:
errors.append(f"Error syncing repository {repo_id}: {str(e)}")
db.session.commit()
msg = f"Sync completed. Upserted {synced_count} issue(s)."
if errors:
msg += f" {len(errors)} error(s)."
return {"success": True, "message": msg, "synced_items": synced_count, "errors": errors}
except Exception as e:
try:
db.session.rollback()
except Exception:
pass
return {"success": False, "message": f"Sync failed: {str(e)}", "errors": errors}
def get_config_schema(self) -> Dict[str, Any]:
"""Get configuration schema."""
return {
"fields": [
{
"name": "repository_ids",
"type": "text",
"label": "Repository IDs",
"required": False,
"placeholder": "123456, 789012",
"description": "Comma-separated list of GitLab project IDs to sync (leave empty to sync all accessible projects)",
"help": "Find project IDs in GitLab project settings or API. Leave empty to sync all projects you have access to.",
},
{
"name": "sync_direction",
"type": "select",
"label": "Sync Direction",
"options": [
{"value": "gitlab_to_timetracker", "label": "GitLab → TimeTracker (Import only)"},
{"value": "timetracker_to_gitlab", "label": "TimeTracker → GitLab (Export only)"},
{"value": "bidirectional", "label": "Bidirectional (Two-way sync)"},
],
"default": "gitlab_to_timetracker",
"description": "Choose how data flows between GitLab and TimeTracker",
},
{
"name": "sync_items",
"type": "array",
"label": "Items to Sync",
"options": [
{"value": "issues", "label": "Issues"},
{"value": "merge_requests", "label": "Merge Requests"},
{"value": "projects", "label": "Projects"},
],
"default": ["issues"],
"description": "Select which items to synchronize",
},
{
"name": "issue_states",
"type": "array",
"label": "Issue States to Sync",
"options": [
{"value": "opened", "label": "Open Issues"},
{"value": "closed", "label": "Closed Issues"},
{"value": "all", "label": "All Issues"},
],
"default": ["opened"],
"description": "Which issue states to include in sync",
},
{
"name": "auto_sync",
"type": "boolean",
"label": "Auto Sync",
"default": False,
"description": "Automatically sync when webhooks are received from GitLab",
},
{
"name": "sync_interval",
"type": "select",
"label": "Sync Schedule",
"options": [
{"value": "manual", "label": "Manual only"},
{"value": "hourly", "label": "Every hour"},
{"value": "daily", "label": "Daily"},
{"value": "weekly", "label": "Weekly"},
],
"default": "manual",
"description": "How often to automatically sync data",
},
{
"name": "create_projects",
"type": "boolean",
"label": "Create Projects",
"default": True,
"description": "Automatically create projects in TimeTracker from GitLab projects",
},
{
"name": "webhook_secret",
"label": "Webhook Secret",
"type": "password",
"required": False,
"placeholder": "Enter webhook secret from GitLab",
"help": "Secret token for verifying webhook signatures. Configure this in your GitLab project webhook settings.",
"description": "Security token for webhook verification",
},
],
"required": [],
"sections": [
{
"title": "Repository Settings",
"description": "Configure which GitLab projects to sync",
"fields": ["repository_ids", "create_projects"],
},
{
"title": "Sync Settings",
"description": "Configure what and how to sync",
"fields": ["sync_direction", "sync_items", "issue_states", "auto_sync", "sync_interval"],
},
{
"title": "Webhook Settings",
"description": "Configure webhook security",
"fields": ["webhook_secret"],
},
],
"sync_settings": {
"enabled": True,
"auto_sync": False,
"sync_interval": "manual",
"sync_direction": "gitlab_to_timetracker",
"sync_items": ["issues"],
},
}
Reference in New Issue View Git Blame Copy Permalink