Add second audio channel for voice#1625
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1625 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 25 25
Lines 4040 4042 +2
=========================================
+ Hits 4040 4042 +2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR extends the ESPHome voice assistant API audio stream to support a second audio channel by adding a new data2 field to VoiceAssistantAudio, plumbing it through the Python client callback, and adding test coverage for receiving the second channel.
Changes:
- Add
data2to theVoiceAssistantAudioprotobuf message to carry a second audio stream. - Update the voice assistant audio subscription path to forward
data2to thehandle_audiocallback. - Extend models/tests to represent and validate multi-channel audio reception.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
tests/test_client.py |
Updates voice assistant subscription test to validate data2 reception. |
aioesphomeapi/model.py |
Adds MULTI_CHANNEL_AUDIO feature flag and data2 field to the voice audio model. |
aioesphomeapi/client.py |
Changes handle_audio callback signature and forwards audio.data2 to it. |
aioesphomeapi/api.proto |
Adds bytes data2 = 3 to VoiceAssistantAudio. |
aioesphomeapi/api_pb2.py |
Regenerated protobuf output reflecting the updated schema (but currently inconsistent with api.proto). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Warning Rate limit exceeded
To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
WalkthroughThis PR adds multi-channel audio support to the voice assistant API by introducing a second audio data channel ( ChangesMulti-Channel Voice Assistant Audio
Estimated code review effort🎯 2 (Simple) | ⏱️ ~8 minutes Possibly related PRs
Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Tip 💬 Introducing Slack Agent: The best way for teams to turn conversations into code.Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.
Built for teams:
One agent for your entire SDLC. Right inside Slack. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@aioesphomeapi/client.py`:
- Around line 1809-1812: The handle_audio callback is now required to accept two
positional args which breaks existing one-arg consumers; update the type and
invocation to be backward-compatible by allowing a second optional bytes arg and
calling the callback with either one or two args depending on its signature.
Change the Callable annotation around handle_audio to indicate the second
parameter is optional (bytes | None), and when invoking handle_audio (the call
sites around the current definitions and at the later usage near lines
~1876-1882), detect the callback's parameter count via inspect.signature (or
fallback to a try/except TypeError) and call either handle_audio(audio.data) for
single-arg callbacks or handle_audio(audio.data, audio.data2) for two-arg
callbacks, passing None for audio.data2 when not available.
In `@aioesphomeapi/model.py`:
- Around line 1841-1845: The field VoiceAssistantAudioData.data2 is declared as
bytes | None but from_pb currently yields bytes (often b"") for absent
channel-2, so normalize the API by either changing the type to bytes or ensuring
from_pb sets data2 to b"" when the protobuf has no channel-2; update the
conversion in the VoiceAssistantAudioData.from_pb (or the factory that builds
this class from the protobuf) to always populate data2 as bytes (use b"" for
absent data) and adjust the model typing accordingly so callers can depend on a
single semantic.
In `@tests/test_client.py`:
- Around line 4257-4261: The callback handle_audio treats data2 as truthy which
drops empty bytes payloads; change the conditional in handle_audio to explicitly
check for None (use "if data2 is not None:") so empty b"" is counted while still
skipping an absent None, keeping behavior aligned with the protobuf bytes field
contract and ensuring data2_received is incremented for empty payloads.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 4d08ab95-b2c6-45fc-8073-3247b30e3bd5
📒 Files selected for processing (5)
aioesphomeapi/api.protoaioesphomeapi/api_pb2.pyaioesphomeapi/client.pyaioesphomeapi/model.pytests/test_client.py
| async def handle_audio(data: bytes, data2: bytes | None = None) -> None: | ||
| nonlocal data_received, data2_received | ||
| data_received += len(data) | ||
| if data2: | ||
| data2_received += len(data2) |
There was a problem hiding this comment.
Use if data2 is not None: instead of if data2:
bytes is falsy when empty (b""), which is the default value for protobuf bytes fields. Using if data2: means the test callback silently drops a legitimately-received empty data2 payload, misrepresenting the production callback contract. Although the current test data is non-empty (so the assertion still passes), this introduces a divergence between what the test exercises and what callers must actually handle.
🐛 Proposed fix
- if data2:
+ if data2 is not None:
data2_received += len(data2)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| async def handle_audio(data: bytes, data2: bytes | None = None) -> None: | |
| nonlocal data_received, data2_received | |
| data_received += len(data) | |
| if data2: | |
| data2_received += len(data2) | |
| async def handle_audio(data: bytes, data2: bytes | None = None) -> None: | |
| nonlocal data_received, data2_received | |
| data_received += len(data) | |
| if data2 is not None: | |
| data2_received += len(data2) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/test_client.py` around lines 4257 - 4261, The callback handle_audio
treats data2 as truthy which drops empty bytes payloads; change the conditional
in handle_audio to explicitly check for None (use "if data2 is not None:") so
empty b"" is counted while still skipping an absent None, keeping behavior
aligned with the protobuf bytes field contract and ensuring data2_received is
incremented for empty payloads.
What does this implement/fix?
Adds a second audio channel to voice API audio. This allows for providing the best audio for different parts of the voice pipeline: home-assistant/architecture#1364
When audio is streamed to Home Assistant, data from both microphone sources is sent in the
VoiceAssistantAudiomessage (asdataanddata2). Thehandle_audiomethod has been updated to pass the second channel to Home Assistant.Types of changes
Related issue or feature (if applicable):
Pull request in esphome (if applicable):
Checklist:
tests/folder).