Add configurable latency optimization for sub-second end-to-end latency

[jcork-intel]

Summary

This PR adds configurable latency optimization to achieve sub-second end-to-end latency on the full obj_detection_age_prediction pipeline.

Key Changes

1. Queue Optimization (src/pipelines/obj_detection_age_prediction.sh)

   • Added LOW_LATENCY mode: max-size-buffers=3 max-size-time=100000000 (0.1s)
   • Added MEDIUM_LATENCY mode: max-size-buffers=10 max-size-time=500000000 (0.5s)
   • Default queues can buffer up to 200 frames / 1 second per queue, adding significant latency

2. Configurable Inference Interval

   • Added INFERENCE_INTERVAL environment variable (default=3)
   • Set to 1 to process every frame for minimum latency

3. Docker-Compose Environment Passthrough (Critical fix)

   • Shell environment variables were being ignored because docker-compose loads .env directly
   • Added explicit passthrough for: LOW_LATENCY, MEDIUM_LATENCY, INFERENCE_INTERVAL, BATCH_SIZE_DETECT, BATCH_SIZE_CLASSIFY

4. NPU+GPU Hybrid Configuration (src/res/npu-gpu-flip.env)

   • YOLO11n detection on NPU
   • Classification on GPU with VA surface sharing
   • Separate inference options for each device type

Benchmark Results (Lunar Lake)

Metric	Default Config	Latency-Optimized
Single-stream latency	~1,509 ms	~957 ms
Max streams @ 14.95 FPS	22	25
Avg latency at max streams	1,264 ms	840 ms

Usage

# Run with latency optimization
LOW_LATENCY=1 INFERENCE_INTERVAL=1 \
  PIPELINE_SCRIPT=obj_detection_age_prediction.sh \
  DEVICE_ENV=res/npu-gpu-flip.env \
  make run-demo

Files Changed

• src/pipelines/obj_detection_age_prediction.sh - Queue optimization logic
• src/docker-compose.yml - Environment variable passthrough
• src/docker-compose-reg.yml - Environment variable passthrough (registry version)
• src/res/npu-gpu-flip.env - New NPU+GPU hybrid device configuration

Test Plan

• Tested on Lunar Lake with NPU+GPU configuration
• Verified LOW_LATENCY=1 achieves ~957ms single-stream latency
• Verified stream density of 25 streams @ 14.95 FPS target
• Test on other platforms (Meteor Lake, Raptor Lake)

[jcork-intel]

@sachinkaushik @avinash-palleti --> I created this PR just so you could see side by side the changes that I made in my fork when running my experiments.