docs: add prometheus to monitoring docs (#5042)

docs/self-hosting/setup/monitoring.mdx

@@ -8,7 +8,52 @@ icon: "magnifying-glass-chart"
 
 Formbricks follows Next.js best practices with all logs being written to stdout/stderr, making it easy to collect and forward logs to your preferred logging solution.
 
-### Docker Container Logs
+### Log Levels
+
+- `debug`: Detailed information for debugging purposes.
+- `info`: General information about the system's operation.
+- `warn`: Potential issues that may require attention.
+- `error`: Errors that occur during the operation of the system.
+- `fatal`: Critical errors that cause the system to crash.
+
+### Log Format
+
+Formbricks uses JSON format for logs, which is structured and easy to parse. Each log entry includes:
+
+- `level`: The log level (e.g., info, error).
+- `time`: Timestamp of the log entry in milliseconds since epoch.
+- `pid`: Process ID of the application.
+- `hostname`: Hostname of the server where the log was generated.
+- `requestId`: Unique identifier for the request (if applicable).
+- `userId`: Unique identifier for the user (if applicable).
+- `msg`: The log message.
+- `stack`: Stack trace (if applicable).
+- `data`: Additional data related to the log entry.
+
+### Example Log Entry
+
+```json
+{
+  "hostname": "server-1",
+  "level": 30,
+  "msg": "User logged in successfully",
+  "pid": 12345,
+  "requestId": "abc-123",
+  "time": 1710000000000,
+  "userId": "user-789"
+}
+```
+
+### Configuring Log Levels
+
+You can configure the minimum log level using the `LOG_LEVEL` environment variable. Valid values: `debug`, `info`, `warn`, `error`, `fatal`.
+The default log level in production environments is `warn`, while in development environments it is `debug`.
+
+```env
+LOG_LEVEL=debug
+```
+
+### AccessDocker Container Logs
 
 ```bash
 # One-Click setup
@@ -20,7 +65,7 @@ docker logs <container-name>
 docker logs -f <container-name> # Follow logs
 ```
 
-### Kubernetes Pod Logs
+### Access Kubernetes Pod Logs
 
 ```bash
 kubectl logs <pod-name> -n <namespace>
@@ -37,22 +82,29 @@ Since all logs are written to stdout/stderr, you can integrate with various logg
 - Splunk
 - CloudWatch Logs (AWS)
 
-## OpenTelemetry Integration (Beta)
+## OpenTelemetry Integration
 
-Formbricks leverages Next.js's built-in OpenTelemetry instrumentation for comprehensive observability. When enabled, it automatically instruments various aspects of your application.
+Formbricks offers two complementary observability approaches:
 
-Set the following environment variables:
+1. **Next.js OpenTelemetry** - For tracing and APM integration
+2. **Prometheus Integration** - For metrics collection and monitoring
+
+### Next.js OpenTelemetry for Tracing
+
+Formbricks leverages Next.js's built-in OpenTelemetry instrumentation for tracing and APM integration. When enabled, it automatically instruments various aspects of your application, providing detailed insights into request flows and performance.
+
+To enable Next.js OpenTelemetry, set the following environment variables:
 
 ```env
 OTEL_ENABLED=true
-OTEL_ENDPOINT=<your-collector-endpoint>
+OTEL_ENDPOINT=<your-collector-endpoint> # e.g., http://localhost:4318/v1/traces for OTLP HTTP
 OTEL_SERVICE_NAME=formbricks
 NEXT_OTEL_VERBOSE=1 # Optional: enables detailed tracing
 ```
 
-### Default Instrumentation
+#### Default Next.js Instrumentation
 
-The OpenTelemetry integration automatically tracks:
+The Next.js OpenTelemetry integration automatically tracks:
 
 - HTTP requests and responses
 - Route rendering
@@ -61,24 +113,84 @@ The OpenTelemetry integration automatically tracks:
 - Database queries
 - External API calls
 
-### Supported Backends
+#### Supported Backends for Tracing
 
-OpenTelemetry can export data to:
+OpenTelemetry trace data can be exported to various observability platforms:
+
+##### Tracing Backends
 
 - Jaeger
 - Zipkin
-- Prometheus
+- Tempo
+
+##### APM & Full-Stack Observability Platforms
+
 - New Relic
 - Datadog
+- Dynatrace
 - Azure Monitor
+- AWS X-Ray
+- Google Cloud Trace
 
-### Key Metrics
+## Prometheus Integration
 
-- HTTP request duration
-- Database performance
-- Memory usage
-- Response times
-- Error rates
+Formbricks implements a dedicated Prometheus metrics exporter using OpenTelemetry. This integration runs a metrics server on a separate port and exposes metrics in Prometheus format for scraping. It focuses specifically on host and runtime metrics rather than application-specific traces.
+
+### Configuration
+
+To enable and configure the Prometheus exporter, set the following environment variables:
+
+```env
+PROMETHEUS_ENABLED=1
+PROMETHEUS_EXPORTER_PORT=9464  # Optional, defaults to 9464
+```
+
+The `PROMETHEUS_ENABLED` environment variable must be set to `1` to enable metrics collection. The `PROMETHEUS_EXPORTER_PORT` variable is optional and defaults to 9464 if not specified.
+
+The exporter listens on all network interfaces (0.0.0.0) and exposes metrics at the `/metrics` endpoint.
+
+### Available Metrics
+
+The metrics exported by the Prometheus integration include:
+
+- **Host Metrics**:
+
+  - CPU usage (user, system, idle)
+  - Memory usage (used, free, cached)
+  - Disk I/O (reads, writes)
+  - Network I/O (bytes in/out, packets in/out)
+
+- **HTTP Metrics**:
+
+  - Request counts
+  - Request durations
+  - Error rates
+
+- **Runtime Metrics**:
+  - Garbage collection frequency and duration
+  - Event loop lag
+  - Heap statistics (size, used, available)
+
+### Collecting Metrics
+
+You can scrape metrics from your Prometheus server by adding the following to your Prometheus configuration:
+
+```yaml
+scrape_configs:
+  - job_name: "formbricks"
+    static_configs:
+      - targets: ["your-formbricks-host:9464"]
+```
+
+### Resource Attributes
+
+The metrics include resource attributes automatically detected from:
+
+- Environment variables
+- Process information
+- Host information
+
+These attributes can help you filter and group metrics in your dashboards.
 
 ## Health Checks