Skip to content

Packaging_Executable_Guide.md

Latest commit

 

History

History
184 lines (146 loc) · 7.57 KB

Packaging_Executable_Guide.md

File metadata and controls

184 lines (146 loc) · 7.57 KB

Packaging XPDRB (X-Plane Dataref Bridge) as an Executable

This document outlines the process, challenges, and solutions encountered when packaging the XPDRB application into a standalone executable using various tools.

Overview

The X-Plane Dataref Bridge (XPDRB) is a Python/PyQt6 application that interfaces with X-Plane flight simulator. It requires special handling during packaging due to its dependencies on serial communication libraries, PyQt6, qasync for asynchronous operations, and complex array dataref handling.

Challenges Encountered

1. Missing Dependencies Error

The most common issue during packaging was the "No module named 'qasync'" error, which occurs because packaging tools sometimes fail to detect all dependencies automatically, especially when dealing with complex modules like serial, hid, and PyQt6 components.

2. Array Dataref Handling

The application needed to expand array datarefs (e.g., LED_STATE[0], LED_STATE[1], ..., LED_STATE[7]) into individual elements in the Live Monitor UI, which required special handling in both the data model and UI layers.

3. Serial Backend Issues

The application uses pyserial for communication with Arduino devices, which requires proper inclusion of all serial backends (serialwin32, win32, serialutil, threaded) to function correctly on Windows.

Solutions Implemented

Solution 1: PyInstaller (Initial Attempts)

pip install pyinstaller
pyinstaller --onefile --windowed --icon=resources/icon.ico --hidden-import=serial --hidden-import=serial.tools --hidden-import=serial.tools.list_ports --hidden-import=PyQt6 --hidden-import=PyQt6.QtCore --hidden-import=PyQt6.QtGui --hidden-import=PyQt6.QtWidgets --hidden-import=qasync main.py

Issues with PyInstaller:

  • Had trouble detecting qasync module properly
  • Large executable size (~38MB)
  • Sometimes failed to include all necessary serial backends

Solution 2: Nuitka (Alternative Approach)

pip install nuitka
python -m nuitka --onefile --windows-disable-console --include-package=serial --include-package=PyQt6 --include-package=qasync --windows-icon-from-ico=resources/icon.ico main.py

Issues with Nuitka:

  • Compatibility problems with Python 3.14
  • Linking errors with PyQt6 modules
  • Experimental support for newer Python versions

Solution 3: cx_Freeze (Successful Implementation)

Setup Script (setup_cx.py)

import sys
from cx_Freeze import setup, Executable

build_exe_options = {
    "packages": [
        'serial',
        'serial.tools',
        'serial.tools.list_ports',
        'serial.serialwin32',
        'serial.win32',
        'serial.serialutil',
        'serial.threaded',
        'qasync',
        'PyQt6',
    ],
    "includes": [
        'serial.serialutil',
        'serial.threaded',
        'serial.urlhandler',
        'serial.urlhandler.protocol_socket',
        'serial.urlhandler.protocol_rfc2217',
        'serial.urlhandler.protocol_loop',
        'serial.urlhandler.protocol_alt',
        'serial.urlhandler.protocol_cp2110',
        'serial.urlhandler.protocol_hwgrep',
        'serial.urlhandler.protocol_spy',
    ],
    "excludes": [],
    "optimize": 0,
}

# For Windows GUI application without console
base = None
if sys.platform == "win32":
    base = "gui"  # Use 'gui' for Windows GUI applications

setup(
    name="X-Plane Dataref Bridge",
    version="1.0",
    description="X-Plane Dataref Bridge",
    options={"build_exe": build_exe_options},
    executables=[Executable("main.py", base=base, icon="resources/icon.ico",
                          target_name="X-Plane Dataref Bridge.exe")]
)

Building with cx_Freeze

python setup_cx.py build

Advantages of cx_Freeze:

  • Better automatic dependency detection
  • More reliable inclusion of PyQt6 components
  • Proper handling of qasync module
  • More predictable behavior with complex Python applications

Array Expansion Implementation

Core Changes Made

  1. Added array size parsing helper in core/dataref_manager.py:

    • get_array_size_from_type() method to extract array size from type strings (e.g., 'float[8]' -> 8)
    • get_expanded_datarefs() method to return datarefs with arrays expanded into per-element entries

  2. Enhanced Output Panel in gui/output_panel.py:

    • Added _parse_array_size() method to parse array sizes from type strings
    • Modified _add_output_row() to detect array datarefs and expand them into individual elements
    • Updated _subscribe_if_appropriate() to handle array subscriptions properly
    • Enhanced _update_datarefs_table_values() to route updates to correct array elements
    • Updated on_dataref_update() to handle array element updates
    • Modified restoration logic to handle array expansion when loading profiles

  3. Proper Subscription Handling:

    • For array elements, the system subscribes to the base array once but displays all elements separately in the UI
    • Updates are properly routed to individual array elements in the UI

Key Lessons Learned

1. Dependency Inclusion Strategy

Always explicitly include complex dependencies like:

  • Serial communication modules (serial, serial.tools, serial.tools.list_ports)
  • PyQt6 components (PyQt6.QtCore, PyQt6.QtGui, PyQt6.QtWidgets)
  • Async modules (qasync)
  • Hardware interface modules (hid)

2. Array Dataref Handling

  • Detect array sizes using regex pattern: r'\[(\d+)\]'
  • Expand arrays into individual elements in the UI (e.g., LED_STATE[0], LED_STATE[1])
  • Route updates to the correct array element based on the dataref name pattern
  • Maintain proper subscription to the base array while displaying individual elements

3. Platform-Specific Considerations

  • Use appropriate base for Windows GUI applications ("gui" in cx_Freeze)
  • Include Windows-specific serial backends (serial.serialwin32, serial.win32)
  • Handle platform-specific file paths and dependencies

4. Build Process Best Practices

  • Clean previous builds before creating new executables
  • Use --clean flag when available to ensure fresh builds
  • Test executables on clean systems without Python installed
  • Verify all functionality works in the packaged application

Final Working Solution

The application was successfully packaged using cx_Freeze with the following characteristics:

  • Standalone executable with all dependencies included
  • Proper handling of array dataref expansion
  • Reliable serial communication with Arduino devices
  • Full PyQt6 GUI functionality preserved
  • qasync module correctly included for async operations

Distribution Notes

  • The executable is located in build\exe.win-amd64-3.14\X-Plane Dataref Bridge.exe
  • Includes all necessary libraries and dependencies
  • Runs on Windows systems without requiring Python installation
  • Maintains all original functionality including array expansion features
  • Size is approximately 38MB with all dependencies included

Troubleshooting Tips

  1. If encountering "No module named 'qasync'":

    • Ensure qasync is included in the packages list in the setup script
    • Verify the module is installed in your Python environment before building

  2. If serial communication fails:

    • Check that all serial backends are properly included in the build
    • Verify that serial.serialwin32, serial.win32, and serial.serialutil are accessible

  3. If array expansion doesn't work:

    • Ensure the _parse_array_size method is correctly implemented
    • Verify that array elements are properly routed in the UI update methods

  4. For GUI-related issues:

    • Confirm that PyQt6 modules are properly included
    • Check that the correct base is used for GUI applications