updates

packaging-status-quo.mdx

@@ -0,0 +1,320 @@
+# Enterprise Edition Access Control Analysis
+
+## Current Implementation Overview
+
+The system currently has two parallel mechanisms for controlling enterprise features:
+
+### A. Cloud Implementation (Stripe-based)
+- Uses Stripe for subscription management
+- Plans are defined in the database with hardcoded limits
+- Features are controlled based on subscription plans (free, startup, scale, enterprise)
+- Key files:
+  - `apps/web/modules/ee/billing/components/pricing-table.tsx`
+  - `apps/web/modules/ee/billing/api/lib/subscription-created-or-updated.ts`
+  - `packages/database/zod/organizations.ts`
+
+#### Default Limits Definition and Usage
+The default limits for cloud plans are defined in multiple places and used in different contexts:
+
+1. **Primary Definition (`apps/web/lib/constants.ts`)**
+   ```typescript
+   export const BILLING_LIMITS = {
+     FREE: {
+       PROJECTS: 3,
+       RESPONSES: 1500,
+       MIU: 2000,
+     },
+     STARTUP: {
+       PROJECTS: 3,
+       RESPONSES: 5000,
+       MIU: 7500,
+     },
+     SCALE: {
+       PROJECTS: 5,
+       RESPONSES: 10000,
+       MIU: 30000,
+     },
+   } as const;
+   ```
+
+#### Stripe Metadata Handling
+The system uses Stripe product metadata to dynamically set limits for organizations. This is handled in several places:
+
+1. **Product Metadata Structure**
+   - Each Stripe product has metadata fields for:
+     - `responses`: Number of monthly responses allowed (or "unlimited")
+     - `miu`: Number of monthly identified users allowed (or "unlimited")
+     - `projects`: Number of projects allowed (or "unlimited")
+     - `plan`: The plan type (free, startup, scale, enterprise)
+     - `period`: Billing period (monthly, yearly)
+
+2. **Subscription Creation/Update Flow**
+   - When a subscription is created or updated (`subscription-created-or-updated.ts`):
+     ```typescript
+     // Extract limits from product metadata
+     if (product.metadata.responses === "unlimited") {
+       responses = null;
+     } else if (parseInt(product.metadata.responses) > 0) {
+       responses = parseInt(product.metadata.responses);
+     }
+     
+     // Similar handling for miu and projects
+     ```
+   - These limits are then stored in the organization's billing object
+
+3. **Checkout Session Handling**
+   - During checkout (`checkout-session-completed.ts`):
+     - Metadata is passed from the checkout session to the subscription
+     - Includes organization ID and limit information
+     - Updates customer metadata with organization details
+
+4. **Limit Enforcement**
+   - Limits are checked in various places:
+     - Response creation (`response.ts`) to send a notification to PostHog. So far we're not doing anything with that information.
+     - Project creation
+     - User identification
+   - When limits are reached:
+     - Events are sent to PostHog for tracking
+     - Users are notified of plan limits with a banner at the top of the screen
+
+5. **User Notifications**
+   - **Limits Reached Banner**
+     - Shows at the top of the screen when limits are reached
+     - Displays messages for MIU, response, or both limits
+     - Links to billing settings
+   - **Project Limit Modal**
+     - Appears when trying to create more projects than allowed
+     - Shows current limit and upgrade options
+   - **Billing Settings Page**
+     - Visual indicators for approaching limits
+     - Upgrade options when limits are reached
+   - **PostHog Events**
+     - Events sent when limits are reached
+     - Cached for 7 days to prevent spam
+   - **Error Messages**
+     - Clear error messages for limit violations
+     - Role permission errors
+
+6. **UI Display of Limits**
+   - Limits are displayed in the billing settings page (`pricing-table.tsx`):
+     ```typescript
+     // Unlimited checks for different metrics
+     const responsesUnlimitedCheck =
+       organization.billing.plan === "enterprise" && 
+       organization.billing.limits.monthly.responses === null;
+     const peopleUnlimitedCheck =
+       organization.billing.plan === "enterprise" && 
+       organization.billing.limits.monthly.miu === null;
+     const projectsUnlimitedCheck =
+       organization.billing.plan === "enterprise" && 
+       organization.billing.limits.projects === null;
+     ```
+   - Uses `BillingSlider` component to show:
+     - Current usage
+     - Limit thresholds
+     - Visual indicators for approaching limits
+   - Displays different UI states:
+     - Unlimited badges for enterprise plans
+     - Warning indicators when approaching limits
+     - Clear messaging about current plan limits
+   - Supports both monthly and yearly billing periods
+   - Shows upgrade options when limits are reached
+
+7. **Error Handling and Fallback Mechanisms**
+   - **API Error Handling**
+     - Retries on specific HTTP status codes (429, 502, 503, 504)
+     - Maximum retry attempts: 3
+     - Exponential backoff between retries
+     ```typescript
+     if (retryCount < CONFIG.CACHE.MAX_RETRIES && [429, 502, 503, 504].includes(res.status)) {
+       await sleep(CONFIG.CACHE.RETRY_DELAY_MS * Math.pow(2, retryCount));
+       return fetchLicenseFromServerInternal(retryCount + 1);
+     }
+     ```
+
+   - **Fallback Levels**
+     - "live": Direct API response
+     - "cached": Using cached license data
+     - "grace": Using previous valid result within grace period
+     - "default": Fallback to default limits
+     ```typescript
+     const fallbackLevel = getFallbackLevel(liveLicenseDetails, previousResult, currentTime);
+     ```
+
+   - **Grace Period System**
+     - Cache TTL: 24 hours
+     - Previous result TTL: 4 days
+     - Grace period: 3 days
+     ```typescript
+     const CONFIG = {
+       CACHE: {
+         FETCH_LICENSE_TTL_MS: 24 * 60 * 60 * 1000, // 24 hours
+         PREVIOUS_RESULT_TTL_MS: 4 * 24 * 60 * 60 * 1000, // 4 days
+         GRACE_PERIOD_MS: 3 * 24 * 60 * 60 * 1000, // 3 days
+       }
+     };
+     ```
+
+   - **Subscription Error Handling**
+     - Handles failed subscription updates
+     - Maintains previous valid state on errors
+     - Logs errors for debugging
+     ```typescript
+     try {
+       await updateOrganization(organizationId, {
+         billing: {
+           ...organization.billing,
+           plan: updatedBillingPlan,
+           limits: {
+             projects,
+             monthly: {
+               responses,
+               miu,
+             },
+           },
+         },
+       });
+     } catch (error) {
+       logger.error(error, "Failed to update organization billing");
+       // Maintain previous state
+     }
+     ```
+
+   - **Limit Validation**
+     - Validates metadata values before applying
+     - Falls back to default limits if invalid
+     - Logs validation errors
+     ```typescript
+     if (product.metadata.responses === "unlimited") {
+       responses = null;
+     } else if (parseInt(product.metadata.responses) > 0) {
+       responses = parseInt(product.metadata.responses);
+     } else {
+       logger.error({ responses: product.metadata.responses }, "Invalid responses metadata in product");
+       throw new Error("Invalid responses metadata in product");
+     }
+     ```
+
+### B. On-Premise Implementation (License-based)
+- Uses a license key system
+- Features are controlled through license validation
+- Makes API calls to `https://ee.formbricks.com/api/licenses/check`
+- Key files:
+  - `apps/web/modules/ee/license-check/lib/license.ts`
+  - `apps/web/modules/ee/license-check/lib/utils.ts`
+
+#### License Check Implementation Details
+1. **License Validation Flow**
+   - Validates license key against `ee.formbricks.com/api/licenses/check`
+   - Includes usage metrics (e.g., response count) in validation request
+   - Supports proxy configuration for enterprise networks
+   - Implements timeout and retry logic for API calls
+
+2. **Caching System**
+   - Uses a multi-level caching strategy:
+     - Live: Direct API response
+     - Cached: Using cached license data (24 hours TTL)
+     - Grace: Using previous valid result (3 days grace period)
+     - Default: Fallback to default limits
+   - Cache keys are hashed based on license key for security
+
+3. **Feature Access Control**
+   - Features are defined in `TEnterpriseLicenseFeatures`:
+   ```typescript
+   {
+     isMultiOrgEnabled: boolean,
+     contacts: boolean,
+     projects: number | null,
+     whitelabel: boolean,
+     removeBranding: boolean,
+     twoFactorAuth: boolean,
+     sso: boolean,
+     saml: boolean,
+     spamProtection: boolean,
+     ai: boolean
+   }
+   ```
+
+4. **Error Handling**
+   - Implements retry logic for specific HTTP status codes (429, 502, 503, 504)
+   - Maximum retry attempts: 3
+   - Exponential backoff between retries
+   - Grace period system for handling API failures
+
+#### Teams & Access Roles and Multi-language Surveys Implementation
+1. **Teams & Access Roles**
+   - Controlled by both license and billing plan
+   - Permission check implementation:
+   ```typescript
+   export const getRoleManagementPermission = async (
+     billingPlan: Organization["billing"]["plan"]
+   ): Promise<boolean> => {
+     const license = await getEnterpriseLicense();
+     if (IS_FORMBRICKS_CLOUD)
+       return (
+         license.active &&
+         (billingPlan === PROJECT_FEATURE_KEYS.SCALE || 
+          billingPlan === PROJECT_FEATURE_KEYS.ENTERPRISE)
+       );
+     else if (!IS_FORMBRICKS_CLOUD) return license.active;
+     return false;
+   };
+   ```
+   - Access control is implemented through:
+     - Organization roles (Owner, Manager, Billing, Member)
+     - Project-level permissions (Read, Read & Write, Manage)
+     - Team-level roles (Team Contributors, Team Admins)
+   - Permission checks are performed in:
+     - Team management actions
+     - Project access control
+     - Survey management
+     - Role updates
+
+2. **Multi-language Surveys**
+   - Controlled by both license and billing plan
+   - Permission check implementation:
+   ```typescript
+   export const getMultiLanguagePermission = async (
+     billingPlan: Organization["billing"]["plan"]
+   ): Promise<boolean> => {
+     const license = await getEnterpriseLicense();
+     if (IS_FORMBRICKS_CLOUD)
+       return (
+         license.active &&
+         (billingPlan === PROJECT_FEATURE_KEYS.SCALE || 
+          billingPlan === PROJECT_FEATURE_KEYS.ENTERPRISE)
+       );
+     else if (!IS_FORMBRICKS_CLOUD) return license.active;
+     return false;
+   };
+   ```
+   - Checks are performed at multiple levels:
+     - Survey creation
+     - Survey updates
+     - Language management
+     - Response handling
+
+
+## Current Issues
+
+1. **Dual System Complexity**
+   - Different code paths for cloud vs on-premise
+   - Duplicate feature checks in different places
+   - Inconsistent feature access patterns
+
+2. **Hardcoded Plans**
+   - Plans and limits are hardcoded in the database
+   - Stripe integration is tightly coupled with the application
+   - Difficult to modify plans without code changes
+   - Some limits are hardcoded while others come from Stripe metadata
+
+3. **Feature Access Control**
+   - Features are checked in multiple places with different logic
+   - No centralized feature management
+   - Inconsistent handling of feature flags
+
+4. **Error Handling**
+   - Current implementation has some error handling for license checks
+   - Uses a fallback system with grace periods
+   - But could be more robust for API failures
+

packaging-telemetry-plan.mdx

@@ -0,0 +1,271 @@
+# Unified Telemetry System Plan
+
+## 1. Core Architecture
+
+### Instance Identification
+- **Base Identifier System**
+  - Use `organizationId` as the primary identifier for all instances
+  - For Community Edition: Hash the `organizationId` before transmission
+  - For Enterprise Edition: Use raw `organizationId` for detailed insights
+  - Store mapping between hashed and raw IDs in a secure database for EE instances
+
+### Architecture Diagram
+```mermaid
+graph TD
+    subgraph "Formbricks Instance"
+        A[Instance Telemetry] -->|1. Collect Metrics| B[Telemetry Collector]
+        B -->|2. Format Data| C[Instance Telemetry]
+        C -->|3. Send to License Server| D[EE License Server]
+    end
+
+    subgraph "EE License Server"
+        D -->|4. Process & Validate| E[License Server Telemetry]
+        E -->|5. Store Data| F[(Telemetry DB)]
+        E -->|6. Forward to Analytics| G[PostHog]
+    end
+
+    subgraph "Analytics"
+        G -->|7. Group by Organization| H[PostHog Groups]
+        H -->|8. Track Metrics| I[PostHog Analytics]
+    end
+
+    style A fill:#f9f,stroke:#333,stroke-width:2px
+    style D fill:#bbf,stroke:#333,stroke-width:2px
+    style G fill:#bfb,stroke:#333,stroke-width:2px
+```
+
+### Data Collection Structure
+```typescript
+interface TelemetryData {
+  // Anonymous Metrics (Both Editions)
+  instanceId: string;                    // Hashed organizationId
+  alivePing: {
+    timestamp: string;
+    version: string;
+  };
+  activityMetrics: {
+    totalResponses: number;
+    totalUsers: number;
+    totalDisplays: number;
+    totalProjects: number;
+    totalContacts: number;
+    appSetupComplete: boolean;
+  };
+  
+  // Non-Anonymous Metrics (Enterprise Only)
+  enterpriseMetrics?: {
+    deploymentUrl: string;
+    adminEmail?: string;                 // Only if consented during setup
+    hashedLicenseKey: string;            // For EE license validation
+  };
+}
+```
+
+## 2. Implementation Details
+
+### Data Flow Architecture
+```typescript
+// apps/web/lib/telemetry/instance.ts
+export class InstanceTelemetry {
+  private static instance: InstanceTelemetry;
+  private isEnterprise: boolean;
+
+  private constructor() {
+    this.isEnterprise = await this.checkEnterpriseStatus();
+  }
+
+  public async sendTelemetry(organizationId: string) {
+    const metrics = await this.gatherMetrics(organizationId);
+    
+    // Send to our EE License Server
+    await fetch('https://license.formbricks.com/api/telemetry', {
+      method: 'POST',
+      headers: {
+        'Content-Type': 'application/json',
+        'Authorization': `Bearer ${process.env.LICENSE_SERVER_API_KEY}`
+      },
+      body: JSON.stringify({
+        organizationId,
+        metrics,
+        timestamp: new Date().toISOString(),
+        isEnterprise: this.isEnterprise
+      })
+    });
+  }
+}
+```
+
+
+
+### Data Collection Service
+```typescript
+// apps/web/lib/telemetry/collector.ts
+export class TelemetryCollector {
+  public async collectMetrics(organizationId: string) {
+    const [
+      responseCount,
+      userCount,
+      displayCount,
+      projectCount,
+      contactCount,
+      appSetupStatus
+    ] = await Promise.all([
+      this.getResponseCount(organizationId),
+      this.getUserCount(organizationId),
+      this.getDisplayCount(organizationId),
+      this.getProjectCount(organizationId),
+      this.getContactCount(organizationId),
+      this.getAppSetupStatus(organizationId)
+    ]);
+
+    return {
+      totalResponses: responseCount,
+      totalUsers: userCount,
+      totalDisplays: displayCount,
+      totalProjects: projectCount,
+      totalContacts: contactCount,
+      appSetupComplete: appSetupStatus
+    };
+  }
+}
+```
+
+## 3. Collection Schedule
+
+### Regular Collection Points
+1. **Alive Ping**
+   - Every 24 hours
+   - Aligned with EE license check
+   - Includes basic instance health
+
+2. **Activity Metrics**
+   - Every 6 hours
+   - Aggregated counts
+   - No personal data
+
+3. **Enterprise Metrics**
+   - On significant changes
+   - License updates
+   - Admin changes
+
+## 4. Privacy & Security
+
+### Data Handling
+- **Anonymous Data**
+  - All metrics except deployment URL, admin email, and license key
+  - Aggregated counts only
+  - No personal identifiers
+
+- **Enterprise Data**
+  - Stored separately
+  - Access controlled
+  - Encrypted at rest
+
+### Consent Management
+```typescript
+// apps/web/lib/telemetry/consent.ts
+export class ConsentManager {
+  public async checkConsent(organizationId: string) {
+    const organization = await prisma.organization.findUnique({
+      where: { id: organizationId },
+      select: { telemetryConsent: true }
+    });
+    
+    return organization?.telemetryConsent ?? false;
+  }
+}
+```
+
+## 5. Integration Points
+
+### Alive Ping Integration
+```typescript
+// apps/web/lib/telemetry/alive-ping.ts
+export class AlivePingService {
+  public async sendAlivePing(organizationId: string) {
+    const telemetry = new InstanceTelemetry();
+    
+    await telemetry.sendTelemetry({
+      organizationId,
+      alivePing: {
+        timestamp: new Date().toISOString(),
+        version: process.env.NEXT_PUBLIC_VERSION
+      }
+    });
+  }
+}
+```
+
+### License Check Integration
+```typescript
+// apps/web/modules/ee/license-check/lib/license.ts
+export const getEnterpriseLicense = reactCache(
+  async (): Promise<{
+    active: boolean;
+    features: TEnterpriseLicenseFeatures;
+    limits: YearlyLimit;
+  }> => {
+    const license = await fetchLicenseFromServerInternal();
+    
+    // Track license status through our server
+    if (license) {
+      const telemetry = new InstanceTelemetry();
+      await telemetry.sendTelemetry({
+        organizationId: env.ORGANIZATION_ID,
+        enterpriseMetrics: {
+          hashedLicenseKey: hashString(env.ENTERPRISE_LICENSE_KEY),
+          deploymentUrl: env.DEPLOYMENT_URL
+        }
+      });
+    }
+    
+    return license;
+  }
+);
+```
+
+## 6. Migration Strategy
+
+### Phase 1: Basic Metrics
+- Implement instance telemetry
+- Set up EE License Server endpoint
+- Add basic activity metrics
+
+### Phase 2: Enterprise Integration
+- Add enterprise-specific fields
+- Implement consent management
+- Set up license tracking
+
+### Phase 3: Validation & Cleanup
+- Verify data collection
+- Remove old telemetry system
+- Update documentation
+
+## 7. Monitoring & Validation
+
+### Health Checks
+```typescript
+// apps/web/lib/telemetry/health.ts
+export class TelemetryHealth {
+  public async validateCollection() {
+    const metrics = await this.collectMetrics();
+    const expectedFields = [
+      'totalResponses',
+      'totalUsers',
+      'totalDisplays',
+      'totalProjects',
+      'totalContacts',
+      'appSetupComplete'
+    ];
+    
+    return expectedFields.every(field => field in metrics);
+  }
+}
+```
+
+This plan provides a focused approach to telemetry that:
+1. Sends data through our EE License Server first
+2. Collects specific KPIs for both editions
+3. Maintains clear separation between anonymous and non-anonymous data
+4. Integrates with existing license check logic
+5. Provides flexibility to change analytics providers

packaging-telemetry-status-quo.mdx

@@ -0,0 +1,58 @@
+
+## Telemetry Implementation
+
+### Community Edition Telemetry
+The Community Edition currently implements basic telemetry through a simple system:
+
+1. **Basic Usage Metrics**
+   - Anonymous instance identification using hashed CRON_SECRET
+   - Basic usage statistics:
+     - Survey count
+     - Response count
+     - User count
+   - Version tracking
+   - Can be disabled via `TELEMETRY_DISABLED=1` environment variable
+
+2. **Implementation Details**
+   - Uses a dedicated telemetry endpoint (`telemetry.formbricks.com`)
+   - Data is collected anonymously
+   - No personal or customer data is transmitted
+   - Simple event-based system with minimal properties
+
+3. **Current Limitations**
+   - Very basic metrics only
+   - No feature usage tracking
+   - No error tracking
+   - No performance metrics
+   - No user behavior insights
+
+### Enterprise Edition Telemetry
+The Enterprise Edition currently has no dedicated telemetry system:
+
+1. **Current State**
+   - No specific telemetry for enterprise features
+   - No usage tracking for enterprise features
+   - No monitoring of license usage patterns
+   - No insights into feature adoption
+
+2. **Missing Capabilities**
+   - No tracking of enterprise feature usage
+   - No monitoring of license validation patterns
+   - No insights into limit usage and patterns
+   - No tracking of enterprise-specific errors
+   - No monitoring of enterprise feature performance
+
+3. **Impact**
+   - Limited ability to understand enterprise customer needs
+   - No data to drive enterprise feature development
+   - No insights into enterprise feature adoption
+   - Limited ability to proactively address issues
+   - No data to inform enterprise pricing decisions
+
+This lack of telemetry in the Enterprise Edition represents a significant gap in our ability to understand and improve the product for enterprise customers. It makes it difficult to:
+- Track feature adoption and usage patterns
+- Identify common issues and pain points
+- Make data-driven decisions about feature development
+- Provide proactive support
+- Understand enterprise customer needs and behaviors
+