Skip to content

Improvements#325

Merged
jgauchia merged 127 commits intodevelfrom
devel_features
Mar 29, 2026
Merged

Improvements#325
jgauchia merged 127 commits intodevelfrom
devel_features

Conversation

@jgauchia
Copy link
Copy Markdown
Owner

@jgauchia jgauchia commented Jan 4, 2026

Changelog

This PR merges significant performance improvements and stability fixes across the IceNav-v3 system, focusing on rendering, navigation, memory, and UI responsiveness.

⚡ CORE OPTIMIZATIONS & RENDERING ENGINE

  • LVGL 9.5.0 Migration & Optimization:
    • Hardware SIMD Acceleration: Enabled conditional LV_DRAW_SW_ASM_ESP32_S3 in lv_conf.h for ESP32-S3 targets, leveraging native instructions for blending and image
      transformations.
    • Memory Alignment: Standardized draw buffer and PSRAM allocation alignment to 64 bytes (S3 Cache Line) for DMA efficiency.
    • Dynamic Memory Pool: Configured LVGL memory pool to 128KB in PSRAM for S3 targets (via ps_malloc) and 48KB in SRAM for other chips, ensuring optimal memory usage.
    • Layer Buffer: Increased LV_DRAW_LAYER_SIMPLE_BUF_SIZE to 128KB for smoother composition of complex UI elements.
    • Refresh Rate: Adjusted default refresh period to 33ms (aiming for 30 FPS) for a stable UI baseline.
    • Cache Enabled: Activated unified cache (LV_CACHE_DEF_SIZE = 32KB) to accelerate font glyph and icon rendering.
    • UI Fixes: Corrected checkbox/switch state logic using explicit LV_STATE_CHECKED flags for LVGL 9.5.0 compatibility.

🗺 MAP ENGINE & RENDERING

  • New Vector Engine (v0.5.0): High-performance vector map support using Pure Hilbert spatial indexing and binary deduplication (achieving ~30% faster SD access and
    ~15% smaller map files), optimized with VarInt, ZigZag, and Delta encoding with neutralized headers for fast culling.
  • CPU Optimization (Idle/Scroll):
    • Satellites Screen: Implemented "Dirty Flags" (isDOPChanged, hasLocationChange, satInView) to reduce CPU usage from 50% to ~10% in idle states.
    • Map Screen (NAV/PNG):
      • Standby CPU: Reduced from 30% to ~10-15% by applying a 1-degree deadband for heading and checking GPS movement before redrawing.
      • Continuous Scroll: Restored smooth, continuous map panning during manual drag, preventing high CPU when stationary but finger pressed.
  • Zero-Allocation Pipeline: Migrated to persistent projection buffers to eliminate heap fragmentation and rendering stutters.
  • Bbox Pre-Culling: Implemented geometry filtering to skip off-screen elements, significantly reducing CPU overhead.
  • Path Centralization: Centralized map storage directory paths into constants for better maintainability.

⚡ NAVIGATION & ROUTING

  • Hierarchical Search: Spatial indexing (O(log n)) using bounding box segments for instant track snapping and re-routing.
  • High-Performance Math: Optimized loops using squared Euclidean distances and equirectangular approximations to eliminate heavy trigonometric operations and redundant square
    roots.
  • Geographic Precision: Integrated segment-based projection with cosine latitude scaling for accurate tracking.
  • System Efficiency & Battery: Implemented adaptive main loop refresh rates (10/20/50ms) and decoupled navigation updates (10Hz) to maximize battery life and UI responsiveness.
  • UI Throttling: Added intelligent display update filtering to minimize redundant LVGL calls, prioritizing map rendering throughput.
  • Memory Stability: Implemented vector pre-reservation during GPX loading to prevent heap fragmentation and ensure stable performance during long sessions.

🛡 CORE OPTIMIZATION & STABILITY

  • Ultimate GPX Fix: Replaced TinyXML2 DOM parser with a custom Stream Parser for track loading, eliminating massive RAM usage and fixing persistent OOM errors.
  • Map Memory Stability: Converted dynamic map sprites to permanent buffers (up to 1024x1024), eradicating PSRAM fragmentation during navigation.
  • Zoom & Scroll Stability: Fixed map "snap-back" and geographic jumps by implementing resetScrollState and synchronizing coordinates immediately after zoom level changes.
  • Touch Responsiveness: Overhauled touch handling with drag filtering (30px) and optimized timing (200ms), plus prioritized UI tasks to prevent starvation during background
    loading.

🚀 SYSTEM & PERFORMANCE

  • Enhanced I/O: Added 4KB file buffering to drastically improve SD card read throughput.
  • Production Optimization: Refined build flags to disable internal logging and assertions, reducing binary size and latency.
  • Flash Cleanup: Removed unused Montserrat font definitions to optimize storage usage.
  • UI Refresh Rate: Optimized LVGL display refresh rate to 50 FPS (20ms) for balanced performance and resource consumption.

🎨 USER INTERFACE (UI)

  • Hardware Unification: Deduplicated screen code for T-Deck, centralizing layout logic into a single maintainable structure for navigation and satellite screens.
  • Satellite Screen Overhaul: Optimized redraw logic for satellite SNR bars and sky plots by only updating on actual GPS data changes (CPU usage drop in idle).
  • Sensor Responsiveness: Implemented independent state tracking for altitude and speed to ensure lag-free updates on the main screen.

🧹 CODE QUALITY & MAINTENANCE

  • Documentation: Completed comprehensive Doxygen documentation (@brief, @param, @return) across all C++ implementation files.
  • Orphaned Code Cleanup: Removed unused math utilities, legacy manual cache methods, and dead variables to reduce binary footprint and technical debt.
  • LVGL FreeRTOS Integration: Enabled proper LVGL integration with FreeRTOS (LV_USE_OS_FREERTOS) for enhanced multi-tasking stability and correctness.

🛠 STABILITY & BUG FIXES

  • Robust Startup: Resolved racing conditions between the Splash Screen and fast GPS fixes to ensure correct display duration.
  • Thread Safety: Implemented mutex protection for global GPS data access to prevent data corruption.
  • GPX Memory Optimization: Implemented PSRAM allocation for large GPX tracks, resolving std::bad_alloc crashes and ensuring safe track loading.
  • Linker & Code Fixes: Resolved "undefined reference" errors and various typos in core implementation files.

@jgauchia jgauchia marked this pull request as draft January 4, 2026 22:23
@jgauchia jgauchia self-assigned this Jan 4, 2026
@jgauchia jgauchia moved this to In Progress in IceNav roadmap Jan 5, 2026
@jgauchia jgauchia closed this Jan 5, 2026
@github-project-automation github-project-automation Bot moved this from In Progress to Done in IceNav roadmap Jan 5, 2026
@jgauchia jgauchia reopened this Jan 5, 2026
@jgauchia jgauchia moved this from Done to In Progress in IceNav roadmap Jan 5, 2026
- Replace MemoryGuard RAII (malloc/free) with persistent member vectors in Maps class.
- Add pre-allocation logic in initMap to reserve space for 2048 points, reducing heap fragmentation.
- Refactor renderNavLineString, renderNavPolygon and fillPolygonGeneral to use persistent buffers.
- Remove legacy Unified Memory Pool system and associated orphan code.
- Optimize performance by prioritizing fast SRAM for hot projection data while maintaining PSRAM compatibility.
- Add intersection check between feature bounding box and viewport in renderNavViewport.
- Calculate feature bounds on-the-fly using cheap integer comparisons before performing expensive projection math.
- Skip rendering of off-screen features to reduce CPU load and improve frame rates.
   - Set CORE_DEBUG_LEVEL to 0 in platformio.ini to disable internal ESP-IDF logging.
   - Add NDEBUG flag to strip assertions from the production binary.
   - Comment out runtime esp_log_level_set calls in setup() to allow the compiler to remove logging code.
   - Reduce binary size and execution latency by eliminating blocking serial log operations.
jgauchia added 16 commits March 11, 2026 22:14
   - Fix potential null pointer dereference in cli.cpp by validating partition pointer before access.
   - Initialize all member variables in global constructors (Battery, GPS, BME, Compass, KalmanFilter) to prevent RAM garbage issues.
   - Remove unused orphan files: InternalAllocator.hpp and the legacy psram_allocator.hpp.
   - Correct inverted Doxygen comments for tftOn (Wake up) and tftOff (Sleep) power modes in tft.cpp.
   - Replace hardcoded 3x3 (9 tiles) logic with dynamic `tilesGrid` constant.
   - Fix white gaps on Lilygo T4-S3 (452x600) by enabling full 4x4 (1024x1024) tile rendering.
   - Preserve spiral loading optimization for ICENAV_BOARD (3x3 grid) to maintain performance.
   - Synchronize `generateMap`, `mapRenderTask`, and `renderNavViewport` with hardware-specific grid settings.
   - Redesign rendering flow to use a screen-sized display buffer instead of a full-grid buffer.
   - Achieve PSRAM savings
   - Implement internal cropping and rotation logic in `displayMap` using LovyanGFX pivots and offsets.
   - Simplify GUI integration by fixing `mapCanvas` at (0,0), reducing memory bus traffic and improving LVGL composition.
   - Dynamize `mapSprite` creation based on actual screen resolution for universal compatibility.
   - Enable real-time viewport updates during manual drag by triggering `displayMap` in each GUI cycle.
   - Decouple map movement from LVGL object offsets; `mapCanvas` position is now static at (0,0).
   - Eliminate snapback behavior and improve overall scrolling fluidness.
New vector map render engine based on optimized vector map generator
   - Implement hardware SIMD acceleration (ASM) for ESP32-S3 targets.
   - Optimize memory alignment to 64 bytes (S3 Cache Line) for DMA-friendly buffers.
   - Implement dynamic memory pool: 128KB in PSRAM for S3, 48KB in SRAM for others.
   - Increase layer composition buffer to 128KB to reduce draw call slicing.
   - Stabilize UI refresh rate to 30 FPS (33ms) to reduce CPU and memory bus overhead.
   - Fix checkbox/switch state logic by using explicit LV_STATE_CHECKED flags.
   - Optimize SATTRACK screen using "Dirty Flags" for SNR, DOP, and position changes (CPU 50% -> 10%).
   - Implement smart map redrawing:
       - Added 1-degree deadband for heading updates to prevent noise-triggered redraws.
       - Optimized displayMap calls based on real offset changes during scroll and standby.
       - Reduced idle map CPU usage to ~10-15%.
   - Enabled LVGL 9.5.0 unified cache (32KB in PSRAM) to optimize font glyph and icon rendering speed.
   - Created `guiTask` on Core 1 with high priority (Prio 3) to handle `lv_timer_handler()` independently from the main loop.
   - Optimized `Storage::read()` to use direct `fread()` when the destination buffer is in internal SRAM (DMA capable), avoiding redundant `memcpy`.
   - Configured VFS mount with `allocation_unit_size = 0` for automatic sector size detection.
   - Ensured 64-byte alignment for the internal DMA buffer to maximize SPI transfer efficiency on ESP32-S3.
@jgauchia jgauchia linked an issue Mar 21, 2026 that may be closed by this pull request
   - Ported bitwise Hilbert curve math (xyToHilbert) to NavReader for efficient 2D-to-1D spatial mapping.
   - Transitioned from the legacy YTable structure to a flat Hilbert-indexed global binary search.
   - Achieved ~30% faster SD card access times and significantly smoother map scrolling fluidity.
   - Added support for the updated NPK2 header and neutral NAV1 tile headers (allowing binary deduplication).
   - Verified performance gains via granular telemetry and performed a complete cleanup of timing variables and logs.
   - Optimized map engine for the new v0.5.0 Pure Hilbert map ecosystem.
Implement Pure Hilbert lookup engine and support v0.5.0 maps
@jgauchia jgauchia marked this pull request as ready for review March 29, 2026 10:00
@jgauchia jgauchia merged commit 28c66d8 into devel Mar 29, 2026
1 check passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in IceNav roadmap Mar 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Crash in getActTile: lv_tileview_get_tile_act returns NULL during LV_EVENT_SCROLL Create waypoint crashing Incorrect SD size in webserver

1 participant