Commit Graph

38 Commits

Author SHA1 Message Date
Self Hosters
474d78b801 Add telemetry features to Next.js frontend and version_checks to collector
- Add Community Analytics card to Next.js dashboard with toggle switch
- Expand telemetry settings page with collector management (add/remove/enable/disable)
- Add version_checks table to telemetry collector Database tab for viewing update checks
- Add telemetry types and API functions to Next.js frontend

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 10:54:37 -05:00
Self Hosters
fa367e2894 Add build_time to telemetry collector health endpoint
The collector's analytics dashboard already displays build time in the
version badge, but the health endpoint wasn't returning it. This adds
the build_time field to match the census server's health response.

This allows users to verify they're running the latest build by checking
the build timestamp in the UI.

Changes:
- Add build_time field to /health endpoint response
- Uses version.GetBuildTime() which is set via ldflags during build

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-14 14:39:24 -05:00
Self Hosters
e0a5fae237 Fix telemetry collector version check comparison logic
The telemetry collector was comparing its own version against GitHub
instead of comparing the requesting census server's version. This caused
update notifications to fail when the collector and census server had
different versions.

Changes:
- Export IsNewerVersion function from version package
- Update telemetry collector to compare req.CurrentVersion against GitHub latest
- Add backward compatible isNewerVersion alias

This fixes the issue where census server 2.0.3 wasn't being notified
about updates to 2.0.4/2.0.5 even though the collector knew about them.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-14 14:33:21 -05:00
Self Hosters
c91b84bbfc Implement session-based authentication for Next.js frontend
This commit adds complete authentication support to the Next.js frontend,
matching the existing vanilla JS implementation with session-based auth.

Frontend Changes (Next.js):
- Add AuthContext for global auth state management
- Create login page with gradient UI matching vanilla JS design
- Implement ProtectedRoute wrapper for route protection
- Add logout button to Header component
- Move all app pages to (protected)/ route group
- Update API client with improved 401 handling
- Add auth-related TypeScript types

Backend Changes:
- Update static file serving to support both Next.js and vanilla JS
- Add support for /login route (Next.js) in addition to /login.html
- Allow /_next/* routes for Next.js static assets
- Redirect unauthenticated users to /login instead of /login.html

Authentication Flow:
1. User visits app → AuthContext checks session by calling /api/containers
2. If 401 → Redirect to /login page
3. User submits credentials → POST /api/login
4. Backend creates session cookie → Frontend updates context
5. User can access protected routes with valid session
6. Logout destroys session and redirects to login

Key Features:
- Session-based auth with 7-day HTTP-only cookies
- Backward compatible with vanilla JS frontend
- AUTH_ENABLED=false support (skip login entirely)
- XSS protection via HTTP-only cookies
- CSRF protection via SameSite=Lax
- No passwords stored client-side

Files Added:
- web-next/src/contexts/AuthContext.tsx
- web-next/src/app/login/page.tsx
- web-next/src/components/auth/ProtectedRoute.tsx
- web-next/src/app/(protected)/layout.tsx

Files Modified:
- internal/api/handlers.go (static file routing)
- web-next/src/app/layout.tsx (AuthProvider wrapper)
- web-next/src/components/layout/Header.tsx (logout button)
- web-next/src/lib/api.ts (401 handling)
- web-next/src/types/index.ts (auth types)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-14 12:24:35 -05:00
Self Hosters
6fbf9dd597 Implement server-side version checking via telemetry collector
Changes the version check flow to properly track active installations:
- Server now checks for updates via telemetry collector (not GitHub directly)
- Version checks run on startup and daily at midnight
- Server caches results and exposes via /api/health endpoint
- UI now reads version info from server's /api/health (not collector directly)
- Telemetry collector can now accurately track active server installations

Backend:
- Add CheckViaCollector() function to internal/version/version.go
- Add checkForUpdates() and runDailyVersionCheck() in cmd/server/main.go
- Add getInstallationID() and getCollectorURL() helper functions
- Update /api/health endpoint to include latest_version, update_available, release_url
- Uses telemetry endpoint URL from database (falls back to community collector)

Frontend:
- Update UpdateBanner.tsx to use getHealth() instead of checkVersion()
- Update app/settings/page.tsx to use health endpoint for version checks
- Remove direct telemetry collector calls from UI
- Poll hourly instead of daily (server checks daily)

Flow:
1. Server → Telemetry Collector /api/version/check (startup + daily)
2. Collector → GitHub API (cached 24h) + records installation activity
3. Server → stores result in memory cache
4. UI → reads from server's /api/health endpoint
5. Collector → uses version_checks table for "active installations" chart

This ensures the collector accurately tracks active server installations
(not just browser sessions) for the analytics dashboard.

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-12 10:27:37 -05:00
Self Hosters
21c6360540 Add real-time progress indicator for container update checks
Implemented Server-Sent Events (SSE) to provide live progress updates
when checking multiple containers for updates, solving the "is it hung?"
problem for bulk operations.

Backend Changes:
- Add job tracking system (internal/api/update_jobs.go)
  - Thread-safe UpdateJobManager for in-memory job state
  - 1-hour TTL with automatic cleanup
  - Incremental result storage as checks complete

- Modify bulk check handler to run asynchronously
  - Returns job ID immediately instead of blocking
  - Launches goroutine to perform checks in background
  - Updates progress after each container check

- Add SSE progress endpoint (GET /api/containers/check-progress/{job_id})
  - Streams progress updates every 500ms
  - Sends 'progress' events with checked/total counts
  - Sends 'complete' event with final results

- Add hourly cleanup goroutine to prevent memory leaks

Frontend Changes (Next.js):
- Create useUpdateCheckProgress custom hook
  - Manages SSE connection lifecycle
  - Parses progress and completion events
  - Handles errors and cleanup gracefully

- Update BulkUpdateModal components (containers + NPM integration)
  - Show animated progress bar (0-100%)
  - Display live counter: "Checked X of Y containers"
  - Smooth CSS transitions with 300ms duration

- Update API client to handle new response format

User Experience:
- Before: Static "Checking N containers..." with no feedback
- After: Real-time progress bar + counter, updates every 500ms
- Especially helpful for large batches (50+ containers)

Technical Details:
- Uses SSE over WebSockets (simpler for one-way communication)
- Non-blocking: UI remains responsive during checks
- Memory efficient: Jobs auto-cleanup after 1 hour
- Browser compatible: Works in Chrome, Firefox, Safari

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-12 09:04:16 -05:00
Self Hosters
4aa3989829 Add Trivy database metadata reporting
- Add TrivyDBUpdatedAt field to agent Info struct
- Add GetTrivyDBMetadata() method to read Trivy's metadata.json
- Include trivy_db_updated_at in /api/vulnerabilities/summary response
- Agent /info endpoint now reports Trivy DB update timestamp
- Format: ISO 8601 timestamp (e.g., "2025-12-11T18:30:38Z")

Resolves "DB: Unknown" display issue in Security plugin modals.
Both local server and remote agents now report database freshness.

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-12 07:44:52 -05:00
Self Hosters
eeba4c116d Add missing database and infrastructure changes from previous session
Changes include:
- GetHostIDForImage method in storage with ORDER BY last_seen DESC
- Agent deployment script improvements
- Build time tracking in agent Dockerfile
- Additional vulnerability scanning infrastructure

These changes were made in previous session but not committed.

🤖 Generated with Claude Code
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-12 07:00:43 -05:00
Self Hosters
2cf3b7c0d6 Fix module path and add build time display to UI
Backend changes:
- Updated go.mod module path from github.com/container-census to
  github.com/selfhosters-cc to match correct GitHub organization
- Updated all import paths across codebase to use new module name
- This fixes ldflags injection of BuildTime during compilation
- BuildTime now correctly shows in /api/health response

Frontend changes:
- Added build time badge next to version in header
- Shows date and time in compact format (e.g., "🔨 12/11/2025 8:06 PM")
- Hover shows full timestamp
- Only displays if build_time is not "unknown"

The build script already sets BuildTime via ldflags, but it was being
ignored because the module path in go.mod didn't match the ldflags path.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-11 20:12:10 -05:00
Self Hosters
8f960fbf68 Fix sidebar navigation and plugin routing issues
- Always show "Manage Plugins" link in sidebar even when all plugins disabled
- Restore NPM plugin static page to avoid bundle.js 404 errors
- Remove npm from dynamic route generateStaticParams (uses static route)
- NPM plugin now properly uses its dedicated React component
- Graph and security plugins continue to use dynamic [pluginId] route

This fixes the issue where disabling all plugins made it impossible to
re-enable them, and resolves bundle.js loading errors for NPM plugin.
2025-12-07 20:22:52 -05:00
Self Hosters
1554740699 Improved graph plugin container node styling and simplified plugin architecture
Changes:
- Container nodes now use dynamic width based on label text (width: 'label')
- Fixed height of 40px for consistent rectangular appearance
- Added padding (8px) for better text readability
- Disabled text wrapping for cleaner display
- Removed all external plugin infrastructure (Phase 2 complete):
  - Deleted internal/plugins/external/ (1,629 lines)
  - Deleted internal/plugins/proto/ (658 lines + 143KB generated)
  - Simplified plugin manager (387 lines removed)
  - Simplified API handlers (125 lines removed)
  - Removed Census API gRPC server (39 lines removed)
- Updated CLAUDE.md with Plugin Architecture documentation
- Graph plugin fully functional as built-in plugin

Total code reduction: ~6,500 lines (-85% complexity)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-06 10:32:09 -05:00
Self Hosters
45036ca19e Convert graph plugin to built-in and fix UI issues
- Convert graph-visualizer from external to built-in plugin
- Add webpack build process for graph plugin frontend
- Fix history modal: reverse timeline sort (newest first)
- Fix history modal: correct lifetime calculation
- Fix container cards: deduplicate port displays
- Update build scripts to compile graph plugin bundle
- Fix plugin bundle URL routing in Next.js

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-05 16:53:13 -05:00
Self Hosters
bb2073ef56 Add external plugin system with gRPC support
- Implemented external plugin architecture with gRPC-based communication
- Added plugin manager for lifecycle management (start, stop, healthcheck)
- Created protobuf definitions for plugin API and Census API
- Added plugin discovery and loading from data/plugins directory
- Plugin features: custom tabs, HTTP routes, frontend assets, settings
- Added plugin management UI in Next.js frontend
- Added plugin SDK for frontend integration with fetch proxy and toast notifications
- Included cache busting for plugin asset loading
- Support for enabling/disabling plugins via UI
- Automatic plugin process management and health monitoring

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 14:26:35 -05:00
Self Hosters
8ac9ca8947 Add plugin architecture and NPM integration (WIP)
Plugin system infrastructure:
- Plugin interface with lifecycle management (Init, Start, Stop)
- Plugin manager for registration and route mounting
- Scoped database access for plugin data/settings
- Event bus for plugin communication
- Badge providers and container enrichers

NPM plugin (Nginx Proxy Manager):
- API client with JWT authentication
- Instance management (add/edit/delete/test/sync)
- Proxy host fetching and container matching
- Badge provider for exposed containers
- Tab UI with external JS loading

Container model updates:
- Added NetworkDetails (IP, aliases) for plugin matching
- Added StartedAt timestamp for uptime display
- Added PluginData map for plugin enrichment

Frontend plugin system:
- Plugin manager JS for loading tabs and badges
- Integrations dropdown in navigation
- External script loading with init function callbacks
- Container uptime display on cards

Note: Plugin tab JS execution has issues - Next.js migration planned.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-02 16:02:53 -05:00
Self Hosters
8dde58ba88 Add trending statistics to telemetry dashboard (hottest, movers, new entries)
- Add /api/stats/hottest endpoint for popular images by containers and adoption
- Add /api/stats/movers endpoint for week-over-week risers and fallers
- Add /api/stats/new-entries endpoint for newly discovered images
- Add image_stats_weekly table with automatic weekly snapshots
- Add Trends tab to vanilla JS dashboard with interactive charts
- Add collector-build.sh and collector-run.sh scripts for local development
- Add Next.js example components (HottestImagesChart, MoversChart, NewEntriesCard)
- Update telemetry-api.ts with new trending API methods and types

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-27 16:17:41 -05:00
Self Hosters
a3c7e6995f Release v1.7.1: Fix image update detection and add agent version display
Fixed:
- Image update detection now uses registry digest (RepoDigests) instead of
  local image ID, eliminating false positive updates for containers already
  running the latest image
- Multi-arch image timestamps now correctly fetched by resolving platform-
  specific manifest (linux/amd64) from manifest lists
- Logout button hidden when authentication is disabled
- JS files served with no-cache headers to ensure updates are seen without
  hard refresh

Added:
- Agent version display on Hosts page with version fetched on each scan
- Onboarding tour now re-shows on major/minor version upgrades to display
  "What's new" information to returning users

Changed:
- Update progress UI shows "Pulling image..." immediately when update starts

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-27 15:11:37 -05:00
Self Hosters
96b6cc1fb6 Replace HTTP Basic Auth with session-based authentication
This commit replaces the browser's native Basic Auth prompt with a
custom login page to improve user experience and avoid browser
credential caching issues.

Authentication Changes:
- Add gorilla/sessions for cookie-based session management
- Create login page with instructions for finding credentials
- Add logout button (🚪 icon) to navbar
- Root path (/) now redirects to /login.html when unauthenticated
- Maintain backward compatibility with Basic Auth for API clients
- Add SESSION_SECRET environment variable for session encryption

Implementation:
- internal/auth/session.go: Session middleware and management
- internal/api/auth_handlers.go: Login/logout HTTP endpoints
- internal/api/handlers.go: Updated routing with selective auth
- cmd/server/main.go: Session store initialization
- web/login.html: Login page with credential finding instructions
- web/login.js: Login form handling
- web/app.js: 401 redirect handling and logout function
- web/index.html: Logout button in navbar

Documentation:
- README.md: Added SESSION_SECRET to docker-compose example
- README.md: Added "Authentication Issues" troubleshooting section
- scripts/run-local.sh: Added auth prompt with qwerty credentials

Onboarding Tour:
- Restored "Join the Selfhosting Community" telemetry opt-in step
- Added updateTelemetrySettings() method
- Tour now has 5 steps including community contribution option

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-17 10:12:58 -05:00
Self Hosters
8bfb2e9b0d Add image version display and fix GHCR authentication
- Display actual image versions/tags on container cards
  - Shows version from org.opencontainers.image.version label
  - Displays multiple tags (e.g., "11.6.2 (latest)")
  - Helps verify image updates have taken effect
  - Database stores all image tags via image_tags field

- Fix GHCR authentication for image update checks
  - Added support for GitHub Container Registry (ghcr.io)
  - Anonymous token authentication for public GHCR images
  - Resolves 401 Unauthorized errors when checking ghcr.io images

- Add card design alternatives showcase (card-designs.html)
  - Three design options: Compact Metro, Spacious Material, Modern Dashboard
  - Comprehensive documentation in CARD_DESIGNS_README.md
  - Side-by-side comparison for design evaluation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-16 13:45:50 -05:00
Self Hosters
b4651f1e81 Added update ability 2025-11-15 17:49:41 -05:00
Self Hosters
55dc366545 Moved all configurations to the database and redid the onborading 2025-11-04 20:41:50 -05:00
Self Hosters
be8084a691 Almost have the scanning complete. Tour is implemented 2025-11-02 13:47:47 -05:00
Self Hosters
87b3c79094 Mostly working security tab 2025-11-01 20:20:26 -04:00
Self Hosters
878247e0dc Notification system almost complete 2025-10-30 22:45:48 -04:00
Self Hosters
9e3559897e Version 1.3.0
- Added CPU / Memory tracking
2025-10-30 14:37:01 -04:00
Self Hosters
d137b32d73 Support for putting the API token for the agent in the environment variables so you don't need to mount a volume 2025-10-28 20:04:35 -04:00
Self Hosters
ea98147342 Version 1.2.0
Added:
 - Server: History tab showing a timeline view of when  a container was first seen, image changes, state changes, etc
 - Server: Test connection buttons when adding new agents
 - Server: Database cleanup routine to remove scan details when it can be aggregated (no data lost for tracking trends)
 - Telemetry collector: Database view to see more granular details about submissions, making debugging easier
 - Added CHANGELOG.md

Fixed:
 - Agent: API token persistence - was generating a new token each time
2025-10-28 11:52:32 -04:00
Self Hosters
1c26c73843 Added database view to the telemetry collector 2025-10-27 23:45:55 -04:00
Self Hosters
1b776215e8 Updated telemetry charts and README 2025-10-23 21:19:42 -04:00
Self Hosters
fa3ce2dd78 Added a graph view of all nodes 2025-10-23 17:52:31 -04:00
Self Hosters
5ba569080a Added a table view of all containers 2025-10-20 21:27:59 -04:00
Self Hosters
f4c905b62c Added option to clear circuit breaker and cleaned up telemetry around docker hub as a default source 2025-10-18 19:28:26 -04:00
Self Hosters
f386429db5 Cleaning up collector handling 2025-10-18 12:17:16 -04:00
Self Hosters
e0927ebbf5 Added a fuil suite of API endpoints, sample pages, and documentation to expose collector stats to a third-party next.js (or other) application. 2025-10-17 08:40:38 -04:00
Self Hosters
9c1e075ab8 1. Updated the UI to allow all settings / configuration. Config file is only used for storage now, user doesn't have to edit it.
2. Added a visual indicator to the collector to show the user when new reports are arriving
3. Removed the reload config button as it's not needed
2025-10-17 00:29:35 -04:00
Self Hosters
28a6fbd082 Huge updates:
1. Added optional security to the UI for the server and telemtry collector
2. Expanded the telemetry being collected and the telemetry UI accordingly
3. Documentation update
4. Enhanced charting
2025-10-16 18:50:21 -04:00
Self Hosters
ffef935e75 Normalized image names for better tracking 2025-10-11 08:21:02 -04:00
Self Hosters
808e49e465 Telemetry introduced 2025-10-10 08:02:48 -04:00
Self Hosters
21117cf2b8 Initial commit 2025-10-07 16:26:32 -04:00