Skip to content

feat: add Ascend NPU support for router multi-card deployment#4846

Open
scyiwei1986 wants to merge 1 commit intoopendatalab:masterfrom
scyiwei1986:feat/ascend-npu-router-support
Open

feat: add Ascend NPU support for router multi-card deployment#4846
scyiwei1986 wants to merge 1 commit intoopendatalab:masterfrom
scyiwei1986:feat/ascend-npu-router-support

Conversation

@scyiwei1986
Copy link
Copy Markdown

Set environment variable for NPU device mode.

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

This PR adds Ascend NPU (华为昇腾) support for the router module, enabling proper multi-card deployment on NPU devices. The router previously only set CUDA_VISIBLE_DEVICES for GPU worker isolation, which is not recognized by Ascend NPU runtime.

Modification

  • Detect MINERU_DEVICE_MODE environment variable to identify NPU deployment mode
  • Set ASCEND_RT_VISIBLE_DEVICES instead of CUDA_VISIBLE_DEVICES when MINERU_DEVICE_MODE=npu
  • Each router worker gets exclusive access to a single NPU device, preventing resource contention

BC-breaking (Optional)

This PR is intentionally minimal:

  • Only 3 lines of code changed
  • No new dependencies
  • Backward compatible with existing GPU deployments

Checklist

Before PR:

  • [√] Pre-commit or other linting tools are used to fix the potential lint issues.
  • [√] Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
  • [√] The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
  • [√] The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

  • [√] If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects.
  • [√] CLA has been signed and all committers have signed the CLA in this PR.

Set environment variable for NPU device mode.
@dosubot dosubot Bot added the size:XS This PR changes 0-9 lines, ignoring generated files. label Apr 23, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 23, 2026

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@dosubot dosubot Bot added the enhancement New feature or request label Apr 23, 2026
@scyiwei1986
Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

github-actions Bot added a commit that referenced this pull request Apr 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size:XS This PR changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant