4.3.0版本使用swift infer + vllm backend推理Kimi-k25模型报错”AttributeError: 'KimiK25VisionConfig' object has no attribute 'hidden_size'. Did you mean: 'mm_hidden_size'?“

[Jubileus-ryz]

Checklist / 检查清单

• I have searched existing issues, and this is a new bug report. / 我已经搜索过现有的 issues，确认这是一个新的 bug report。

Bug Description / Bug 描述

根因为Kimi官方的config里面，vit部分的配置被定义为

"vt_hidden_size": 1152,
"vt_intermediate_size": 4304,
"vt_num_attention_heads": 16,
"vt_num_hidden_layers": 27

但是vllm实际读取的时候 Vit部分统一用的还是不带有vt_的参数,例如这里

感觉是因为swift读config的时候没有加载vllm自定义的kimi_config.py

详细报错日志

[rank4]: Traceback (most recent call last):
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/swift/cli/infer.py", line 5, in <module>
[rank4]:     infer_main()
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/swift/pipelines/infer/infer.py", line 312, in infer_main
[rank4]:     return SwiftInfer(args).main()
[rank4]:            ^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/swift/pipelines/infer/infer.py", line 39, in __init__
[rank4]:     self.infer_engine = self.get_infer_engine(args, self.template)
[rank4]:                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/swift/pipelines/infer/infer.py", line 93, in get_infer_engine
[rank4]:     return infer_engine_cls(**kwargs)
[rank4]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/swift/infer_engine/vllm_engine.py", line 206, in __init__
[rank4]:     self._prepare_engine()
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/swift/infer_engine/vllm_engine.py", line 229, in _prepare_engine
[rank4]:     engine = llm_engine_cls.from_engine_args(self.engine_args)
[rank4]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/vllm/v1/engine/llm_engine.py", line 170, in from_engine_args
[rank4]:     return cls(
[rank4]:            ^^^^
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/vllm/v1/engine/llm_engine.py", line 104, in __init__
[rank4]:     self.engine_core = EngineCoreClient.make_client(
[rank4]:                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/vllm/v1/engine/core_client.py", line 103, in make_client
[rank4]:     return InprocClient(vllm_config, executor_class, log_stats)
[rank4]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/vllm/v1/engine/core_client.py", line 285, in __init__
[rank4]:     self.engine_core = EngineCore(*args, **kwargs)
[rank4]:                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/vllm/v1/engine/core.py", line 118, in __init__
[rank4]:     self.model_executor = executor_class(vllm_config)
[rank4]:                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
[rank4]:     return func(*args, **kwargs)
[rank4]:            ^^^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/vllm/v1/executor/abstract.py", line 109, in __init__
[rank4]:     self._init_executor()
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/vllm/v1/executor/uniproc_executor.py", line 172, in _init_executor
[rank4]:     super()._init_executor()
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/vllm/v1/executor/uniproc_executor.py", line 65, in _init_executor
[rank4]:     self.driver_worker.load_model()
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/vllm/v1/worker/gpu_worker.py", line 345, in load_model
[rank4]:     self.model_runner.load_model(load_dummy_weights=load_dummy_weights)
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
[rank4]:     return func(*args, **kwargs)
[rank4]:            ^^^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 4876, in load_model
[rank4]:     self.model = model_loader.load_model(
[rank4]:                  ^^^^^^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
[rank4]:     return func(*args, **kwargs)
[rank4]:            ^^^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/vllm/model_executor/model_loader/base_loader.py", line 55, in load_model
[rank4]:     model = initialize_model(
[rank4]:             ^^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
[rank4]:     return func(*args, **kwargs)
[rank4]:            ^^^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/vllm/model_executor/model_loader/utils.py", line 61, in initialize_model
[rank4]:     model = model_class(vllm_config=vllm_config, prefix=prefix)
[rank4]:             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/vllm/model_executor/models/kimi_k25.py", line 336, in __init__
[rank4]:     self.vision_tower = MoonViT3dPretrainedModel(
[rank4]:                         ^^^^^^^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/vllm/model_executor/models/kimi_k25_vit.py", line 568, in __init__
[rank4]:     out_dim=config.hidden_size,
[rank4]:             ^^^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/lib/python3.11/dist-packages/transformers/configuration_utils.py", line 434, in __getattribute__
[rank4]:     return super().__getattribute__(key)
[rank4]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank4]: AttributeError: 'KimiK25VisionConfig' object has no attribute 'hidden_size'. Did you mean: 'mm_hidden_size'?

How to Reproduce / 如何复现

vllm==0.21.0
ms-swift==4.3.0
transformers==5.8.0

swift infer
--model moonshotai/Kimi-K2.6
--infer_backend vllm
--val_dataset 'modelscope/coco_2014_caption:validation#5000'
--vllm_gpu_memory_utilization 0.9
--vllm_max_model_len 8192
--max_new_tokens 2048
--write_batch_size 1000
--result_path new_coco_dataset.jsonl

Additional Information / 补充信息

请问我应该在哪里增加这个比较合适?

[hjh0119]

https://github.com/huggingface/transformers/releases/tag/v5.13.0
plz retry with transformers>=5.13.0