ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Fix vllm gptq awq error (#12863)

Browse source 
* fix gptq awq error

* fix python style
This commit is contained in:
Wang, Jian4 2025-02-20 16:27:23 +08:00 committed by GitHub
parent a488981f3f
commit 348dc8056d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 6 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
python/llm/src/ipex_llm/vllm/xpu/model_convert.py
View file

@ -93,10 +93,12 @@ def get_load_function(low_bit):
vllm_config=new_vllm_config
)
if self.vllm_config.model_config.low_bit_model_path is None:
if "qwen" in self.vllm_config.model_config.model.lower() or \
"baichuan" in self.vllm_config.model_config.model.lower() or \
"codegeex4-all" in self.vllm_config.model_config.model.lower() or \
"chatglm" in self.vllm_config.model_config.model.lower():
if ("qwen" in self.vllm_config.model_config.model.lower() or
"baichuan" in self.vllm_config.model_config.model.lower() or
"codegeex4-all" in self.vllm_config.model_config.model.lower() or
"chatglm" in self.vllm_config.model_config.model.lower()) and \
"gptq" not in self.model_config.model.lower() and \
"awq" not in self.model_config.model.lower():
self.model.apply(padding_mlp)
from ipex_llm import optimize_model
not_convert_last_mlp = os.getenv("IPEX_LLM_NOT_CONVERT_LAST_MLP", None)