Files
Evgenii Burmakin 8d2ade1bdc 0.37.0 (#2067)
* fix: move foreman to global gems to fix startup crash (#1971)

* Update exporting code to stream points data to file in batches to red… (#1980)

* Update exporting code to stream points data to file in batches to reduce memory usage

* Update changelog

* Update changelog

* Feature/maplibre frontend (#1953)

* Add a plan to use MapLibre GL JS for the frontend map rendering, replacing Leaflet

* Implement phase 1

* Phases 1-3 + part of 4

* Fix e2e tests

* Phase 6

* Implement fog of war

* Phase 7

* Next step: fix specs, phase 7 done

* Use our own map tiles

* Extract v2 map logic to separate manager classes

* Update settings panel on v2 map

* Update v2 e2e tests structure

* Reimplement location search in maps v2

* Update speed routes

* Implement visits and places creation in v2

* Fix last failing test

* Implement visits merging

* Fix a routes e2e test and simplify the routes layer styling.

* Extract js to modules from maps_v2_controller.js

* Implement area creation

* Fix spec problem

* Fix some e2e tests

* Implement live mode in v2 map

* Update icons and panel

* Extract some styles

* Remove unused file

* Start adding dark theme to popups on MapLibre maps

* Make popups respect dark theme

* Move v2 maps to maplibre namespace

* Update v2 references to maplibre

* Put place, area and visit info into side panel

* Update API to use safe settings config method

* Fix specs

* Fix method name to config in SafeSettings and update usages accordingly

* Add missing public files

* Add handling for real time points

* Fix remembering enabled/disabled layers of the v2 map

* Fix lots of e2e tests

* Add settings to select map version

* Use maps/v2 as main path for MapLibre maps

* Update routing

* Update live mode

* Update maplibre controller

* Update changelog

* Remove some console.log statements

* Pull only necessary data for map v2 points

* Feature/raw data archive (#2009)

* 0.36.2 (#2007)

* fix: move foreman to global gems to fix startup crash (#1971)

* Update exporting code to stream points data to file in batches to red… (#1980)

* Update exporting code to stream points data to file in batches to reduce memory usage

* Update changelog

* Update changelog

* Feature/maplibre frontend (#1953)

* Add a plan to use MapLibre GL JS for the frontend map rendering, replacing Leaflet

* Implement phase 1

* Phases 1-3 + part of 4

* Fix e2e tests

* Phase 6

* Implement fog of war

* Phase 7

* Next step: fix specs, phase 7 done

* Use our own map tiles

* Extract v2 map logic to separate manager classes

* Update settings panel on v2 map

* Update v2 e2e tests structure

* Reimplement location search in maps v2

* Update speed routes

* Implement visits and places creation in v2

* Fix last failing test

* Implement visits merging

* Fix a routes e2e test and simplify the routes layer styling.

* Extract js to modules from maps_v2_controller.js

* Implement area creation

* Fix spec problem

* Fix some e2e tests

* Implement live mode in v2 map

* Update icons and panel

* Extract some styles

* Remove unused file

* Start adding dark theme to popups on MapLibre maps

* Make popups respect dark theme

* Move v2 maps to maplibre namespace

* Update v2 references to maplibre

* Put place, area and visit info into side panel

* Update API to use safe settings config method

* Fix specs

* Fix method name to config in SafeSettings and update usages accordingly

* Add missing public files

* Add handling for real time points

* Fix remembering enabled/disabled layers of the v2 map

* Fix lots of e2e tests

* Add settings to select map version

* Use maps/v2 as main path for MapLibre maps

* Update routing

* Update live mode

* Update maplibre controller

* Update changelog

* Remove some console.log statements

---------

Co-authored-by: Robin Tuszik <mail@robin.gg>

* Remove esbuild scripts from package.json

* Remove sideEffects field from package.json

* Raw data archivation

* Add tests

* Fix tests

* Fix tests

* Update ExceptionReporter

* Add schedule to run raw data archival job monthly

* Change file structure for raw data archival feature

* Update changelog and version for raw data archival feature

---------

Co-authored-by: Robin Tuszik <mail@robin.gg>

* Set raw_data to an empty hash instead of nil when archiving

* Fix storage configuration and file extraction

* Consider MIN_MINUTES_SPENT_IN_CITY during stats calculation (#2018)

* Consider MIN_MINUTES_SPENT_IN_CITY during stats calculation

* Remove raw data from visited cities api endpoint

* Use user timezone to show dates on maps (#2020)

* Fix/pre epoch time (#2019)

* Use user timezone to show dates on maps

* Limit timestamps to valid range to prevent database errors when users enter pre-epoch dates.

* Limit timestamps to valid range to prevent database errors when users enter pre-epoch dates.

* Fix tests failing due to new index on stats table

* Fix failing specs

* Update redis client configuration to support unix socket connection

* Update changelog

* Fix kml kmz import issues (#2023)

* Fix kml kmz import issues

* Refactor KML importer to improve readability and maintainability

* Implement moving points in map v2 and fix route rendering logic to ma… (#2027)

* Implement moving points in map v2 and fix route rendering logic to match map v1.

* Fix route spec

* fix(maplibre): update date format to ISO 8601 (#2029)

* Add verification step to raw data archival process (#2028)

* Add verification step to raw data archival process

* Add actual verification of raw data archives after creation, and only clear raw_data for verified archives.

* Fix failing specs

* Eliminate zip-bomb risk

* Fix potential memory leak in js

* Return .keep files

* Use Toast instead of alert for notifications

* Add help section to navbar dropdown

* Update changelog

* Remove raw_data_archival_job

* Ensure file is being closed properly after reading in Archivable concern

* Add composite index to stats table if not exists

* Update changelog

* Update entrypoint to always sync static assets (not only new ones)

* Add family layer to MapLibre maps (#2055)

* Add family layer to MapLibre maps

* Update migration

* Don't show family toggle if feature is disabled

* Update changelog

* Return changelog

* Update changelog

* Update tailwind file

* Bump sentry-rails from 6.0.0 to 6.1.0 (#1945)

Bumps [sentry-rails](https://github.com/getsentry/sentry-ruby) from 6.0.0 to 6.1.0.
- [Release notes](https://github.com/getsentry/sentry-ruby/releases)
- [Changelog](https://github.com/getsentry/sentry-ruby/blob/master/CHANGELOG.md)
- [Commits](https://github.com/getsentry/sentry-ruby/compare/6.0.0...6.1.0)

---
updated-dependencies:
- dependency-name: sentry-rails
  dependency-version: 6.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump turbo-rails from 2.0.17 to 2.0.20 (#1944)

Bumps [turbo-rails](https://github.com/hotwired/turbo-rails) from 2.0.17 to 2.0.20.
- [Release notes](https://github.com/hotwired/turbo-rails/releases)
- [Commits](https://github.com/hotwired/turbo-rails/compare/v2.0.17...v2.0.20)

---
updated-dependencies:
- dependency-name: turbo-rails
  dependency-version: 2.0.20
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Evgenii Burmakin <Freika@users.noreply.github.com>

* Bump webmock from 3.25.1 to 3.26.1 (#1943)

Bumps [webmock](https://github.com/bblimke/webmock) from 3.25.1 to 3.26.1.
- [Release notes](https://github.com/bblimke/webmock/releases)
- [Changelog](https://github.com/bblimke/webmock/blob/master/CHANGELOG.md)
- [Commits](https://github.com/bblimke/webmock/compare/v3.25.1...v3.26.1)

---
updated-dependencies:
- dependency-name: webmock
  dependency-version: 3.26.1
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Evgenii Burmakin <Freika@users.noreply.github.com>

* Bump brakeman from 7.1.0 to 7.1.1 (#1942)

Bumps [brakeman](https://github.com/presidentbeef/brakeman) from 7.1.0 to 7.1.1.
- [Release notes](https://github.com/presidentbeef/brakeman/releases)
- [Changelog](https://github.com/presidentbeef/brakeman/blob/main/CHANGES.md)
- [Commits](https://github.com/presidentbeef/brakeman/compare/v7.1.0...v7.1.1)

---
updated-dependencies:
- dependency-name: brakeman
  dependency-version: 7.1.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump redis from 5.4.0 to 5.4.1 (#1941)

Bumps [redis](https://github.com/redis/redis-rb) from 5.4.0 to 5.4.1.
- [Changelog](https://github.com/redis/redis-rb/blob/master/CHANGELOG.md)
- [Commits](https://github.com/redis/redis-rb/compare/v5.4.0...v5.4.1)

---
updated-dependencies:
- dependency-name: redis
  dependency-version: 5.4.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Put import deletion into background job (#2045)

* Put import deletion into background job

* Update changelog

* fix null type error and update heatmap styling (#2037)

* fix: use constant weight for maplibre heatmap layer

* fix null type, update heatmap styling

* improve heatmap styling

* fix typo

* Fix stats calculation to recursively reduce H3 resolution when too ma… (#2065)

* Fix stats calculation to recursively reduce H3 resolution when too many hexagons are generated

* Update CHANGELOG.md

* Validate trip start and end dates (#2066)

* Validate trip start and end dates

* Update changelog

* Update migration to clean up duplicate stats before adding unique index

* Fix fog of war radius setting being ignored and applying settings causing errors (#2068)

* Update changelog

* Add Rack::Deflater middleware to config/application.rb to enable gzip compression for responses.

* Add composite index to points on user_id and timestamp

* Deduplicte points based on timestamp brought to unix time

* Fix/stats cache invalidation (#2072)

* Fix family layer toggle in Map v2 settings for non-selfhosted env

* Invalidate cache

* Remove comments

* Remove comment

* Add new indicies to improve performance and remove unused ones to opt… (#2078)

* Add new indicies to improve performance and remove unused ones to optimize database.

* Remove comments

* Update map search suggestions panel styling

* Add yearly digest (#2073)

* Add yearly digest

* Rename YearlyDigests to Users::Digests

* Minor changes

* Update yearly digest layout and styles

* Add flags and chart to email

* Update colors

* Fix layout of stats in yearly digest view

* Remove cron job for yearly digest scheduling

* Update CHANGELOG.md

* Update digest email setting handling

* Allow sharing digest for 1 week or 1 month

* Change Digests Distance to Bigint

* Fix settings page

* Update changelog

* Add RailsPulse (#2079)

* Add RailsPulse

* Add RailsPulse monitoring tool with basic HTTP authentication

* Bring points_count to integer

* Update migration and version

* Update rubocop issues

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Robin Tuszik <mail@robin.gg>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-30 17:33:56 +01:00
..
2025-12-30 17:33:56 +01:00

E2E Tests

End-to-end tests for Dawarich using Playwright.

Running Tests

# Run all tests
npx playwright test

# Run V1 map tests (Leaflet-based)
npx playwright test e2e/map/

# Run V2 map tests (MapLibre-based)
npx playwright test e2e/v2/map/

# Run specific test file
npx playwright test e2e/v2/map/settings.spec.js

# Run tests in headed mode (watch browser)
npx playwright test --headed

# Run tests in debug mode
npx playwright test --debug

# Run tests sequentially (avoid parallel issues)
npx playwright test --workers=1

# Run only non-destructive tests (safe for production data)
npx playwright test --grep-invert @destructive

# Run only destructive tests (use with caution!)
npx playwright test --grep @destructive

Test Structure

e2e/
├── setup/           # Test setup and authentication
├── helpers/         # Shared helper functions
├── map/             # V1 Map tests (Leaflet) - 81 tests
├── v2/              # V2 Map tests (MapLibre) - 52 tests
│   ├── helpers/     # V2-specific helpers
│   ├── map/         # V2 core map tests
│   │   └── layers/  # V2 layer-specific tests
│   └── realtime/    # V2 real-time features
└── temp/            # Playwright artifacts (screenshots, videos)

V1 Map Tests (Leaflet-based) - 81 tests

Map Tests

  • map-controls.spec.js - Basic map controls, zoom, tile layers (5 tests)
  • map-layers.spec.js - Layer toggles: Routes, Heatmap, Fog, etc. (8 tests)
  • map-points.spec.js - Point interactions and deletion (4 tests, 1 destructive)
  • map-visits.spec.js - Confirmed visit interactions and management (5 tests, 3 destructive)
  • map-suggested-visits.spec.js - Suggested visit interactions (6 tests, 3 destructive)
  • map-add-visit.spec.js - Add visit control and form (8 tests)
  • map-selection-tool.spec.js - Selection tool functionality (4 tests)
  • map-calendar-panel.spec.js - Calendar panel navigation (9 tests)
  • map-side-panel.spec.js - Side panel (visits drawer) functionality (13 tests)*
  • map-bulk-delete.spec.js - Bulk point deletion (12 tests, all destructive)
  • map-places-creation.spec.js - Creating new places on map (9 tests, 2 destructive)
  • map-places-layers.spec.js - Places layer visibility and filtering (10 tests)

* Some side panel tests may be skipped if demo data doesn't contain visits

V2 Map Tests (MapLibre-based) - 52 tests

Organized by feature domain:

Core Map Tests

  • v2/map/core.spec.js - Map initialization, lifecycle, loading states (8 tests)
  • v2/map/navigation.spec.js - Zoom controls, date picker navigation (4 tests)
  • v2/map/interactions.spec.js - Point clicks, hover effects, popups (2 tests)
  • v2/map/settings.spec.js - Settings panel, layer toggles, persistence (10 tests)
  • v2/map/performance.spec.js - Load time benchmarks, efficiency (2 tests)

Layer Tests

  • v2/map/layers/points.spec.js - Points display, GeoJSON data (3 tests)
  • v2/map/layers/routes.spec.js - Routes geometry, styling, ordering (8 tests)
  • v2/map/layers/heatmap.spec.js - Heatmap creation, toggle, persistence (3 tests)
  • v2/map/layers/visits.spec.js - Visits layer toggle and display (2 tests)
  • v2/map/layers/photos.spec.js - Photos layer toggle and display (2 tests)
  • v2/map/layers/areas.spec.js - Areas layer toggle and display (2 tests)
  • v2/map/layers/advanced.spec.js - Fog of war, scratch map (3 tests)

Real-time Features

  • v2/realtime/family.spec.js - Family tracking, ActionCable (2 tests, skipped)

V2 Test Organization Benefits

  • Feature-based hierarchy - Clear organization by domain
  • Zero duplication - All settings tests consolidated
  • Easy to navigate - Obvious file naming
  • Better maintainability - One feature = one file

Test Tags

Tests are tagged to enable selective execution:

  • @destructive (22 tests in V1) - Tests that delete or modify data:
    • Bulk delete operations (12 tests)
    • Point deletion (1 test)
    • Visit modification/deletion (3 tests)
    • Suggested visit actions (3 tests)
    • Place creation (3 tests)

Usage:

# Safe for staging/production - run only non-destructive tests
npx playwright test --grep-invert @destructive

# Use with caution - run only destructive tests
npx playwright test --grep @destructive

# Run specific destructive test file
npx playwright test e2e/map/map-bulk-delete.spec.js

Helper Functions

V1 Map Helpers (helpers/map.js)

  • waitForMap(page) - Wait for Leaflet map initialization
  • enableLayer(page, layerName) - Enable a map layer by name
  • clickConfirmedVisit(page) - Click first confirmed visit circle
  • clickSuggestedVisit(page) - Click first suggested visit circle
  • getMapZoom(page) - Get current map zoom level

V2 Map Helpers (v2/helpers/setup.js)

  • navigateToMapsV2(page) - Navigate to MapLibre map
  • navigateToMapsV2WithDate(page, startDate, endDate) - Navigate with date range
  • waitForMapLibre(page) - Wait for MapLibre initialization
  • waitForLoadingComplete(page) - Wait for data loading
  • hasMapInstance(page) - Check if map is initialized
  • getMapZoom(page) - Get current zoom level
  • getMapCenter(page) - Get map center coordinates
  • hasLayer(page, layerId) - Check if layer exists
  • getLayerVisibility(page, layerId) - Get layer visibility state
  • getPointsSourceData(page) - Get points source data
  • getRoutesSourceData(page) - Get routes source data
  • clickMapAt(page, x, y) - Click at specific coordinates
  • hasPopup(page) - Check if popup is visible

Navigation Helpers (helpers/navigation.js)

  • closeOnboardingModal(page) - Close getting started modal
  • navigateToDate(page, startDate, endDate) - Navigate to specific date range
  • navigateToMap(page) - Navigate to V1 map with setup

Selection Helpers (helpers/selection.js)

  • drawSelectionRectangle(page, options) - Draw selection on map
  • enableSelectionMode(page) - Enable area selection tool

Common Patterns

V1 Basic Test Template (Leaflet)

import { test, expect } from '@playwright/test';
import { navigateToMap } from '../helpers/navigation.js';
import { waitForMap } from '../helpers/map.js';

test('my test', async ({ page }) => {
  await navigateToMap(page);
  await waitForMap(page);
  // Your test logic
});

V2 Basic Test Template (MapLibre)

import { test, expect } from '@playwright/test';
import { closeOnboardingModal } from '../../helpers/navigation.js';
import {
  navigateToMapsV2,
  waitForMapLibre,
  waitForLoadingComplete
} from '../helpers/setup.js';

test.describe('My Feature', () => {
  test.beforeEach(async ({ page }) => {
    await navigateToMapsV2(page);
    await closeOnboardingModal(page);
    await waitForMapLibre(page);
    await waitForLoadingComplete(page);
  });

  test('my test', async ({ page }) => {
    // Your test logic
  });
});

V2 Testing Layer Visibility

import { getLayerVisibility } from '../helpers/setup.js';

// Check if layer is visible
const isVisible = await getLayerVisibility(page, 'points');
expect(isVisible).toBe(true);

// Wait for layer to exist
await page.waitForFunction(() => {
  const element = document.querySelector('[data-controller*="maps--maplibre"]');
  const app = window.Stimulus || window.Application;
  const controller = app?.getControllerForElementAndIdentifier(element, 'maps--maplibre');
  return controller?.map?.getLayer('routes') !== undefined;
}, { timeout: 5000 });

V2 Testing Settings Panel

// Open settings
await page.click('button[title="Open map settings"]');
await page.waitForTimeout(400);

// Switch to layers tab
await page.click('button[data-tab="layers"]');
await page.waitForTimeout(300);

// Check toggle state
const toggle = page.locator('label:has-text("Points")').first().locator('input.toggle');
const isChecked = await toggle.isChecked();

Debugging

View Test Artifacts

# Open HTML report
npx playwright show-report

# Screenshots and videos are in:
test-results/

Common Issues

V1 Tests

  • Flaky tests: Run with --workers=1 to avoid parallel interference
  • Timeout errors: Increase timeout in test or use page.waitForTimeout()
  • Map not loading: Ensure waitForMap() is called after navigation

V2 Tests

  • Layer not ready: Use page.waitForFunction() to wait for layer existence
  • Settings panel timing: Add waitForTimeout() after opening/closing
  • Parallel test failures: Some tests pass individually but fail in parallel - run with --workers=3 or --workers=1
  • Source data not available: Wait for source to be defined before accessing data

V2 Test Tips

  1. Always wait for MapLibre to initialize with waitForMapLibre(page)
  2. Wait for data loading with waitForLoadingComplete(page)
  3. Add layer existence checks before testing layer properties
  4. Use proper waits for settings panel animations
  5. Consider timing when testing layer toggles

CI/CD

Tests run with:

  • 1 worker (sequential)
  • 2 retries on failure
  • Screenshots/videos on failure
  • JUnit XML reports

See playwright.config.js for full configuration.

Important Considerations

  • We're using Rails 8 with Turbo, which might not cause full page reloads
  • V2 map uses MapLibre GL JS with Stimulus controllers
  • V2 settings are persisted to localStorage
  • V2 layer visibility is based on user settings (no hardcoded defaults)
  • Some V2 layers (routes, heatmap) are created dynamically based on data

Test Migration Notes

V2 tests were refactored from phase-based to feature-based organization:

  • Before: 9 phase files, 96 tests (many duplicates)
  • After: 13 feature files, 52 focused tests (zero duplication)
  • Code reduction: 56% (2,314 lines → 1,018 lines)
  • Pass rate: 94% (49/52 tests passing, 1 flaky, 2 skipped)

See E2E_REFACTORING_SUCCESS.md for complete migration details.