ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

vllm no padding glm4 to avoid nan error (#12062)

Browse source 
* no padding glm4

* add codegeex
This commit is contained in:
Wang, Jian4 2024-09-11 13:44:40 +08:00 committed by GitHub
parent 649390c464
commit c75f3dd874
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 2 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
python/llm/src/ipex_llm/vllm/xpu/model_convert.py
View file

@ -89,7 +89,8 @@ def get_load_function(low_bit):
) )
if "qwen" in self.model_config.model.lower() or \ if "qwen" in self.model_config.model.lower() or \
"baichuan" in self.model_config.model.lower() or \ "baichuan" in self.model_config.model.lower() or \
"glm" in self.model_config.model.lower(): "codegeex4-all" in self.model_config.model.lower() or \
"chatglm" in self.model_config.model.lower():
self.model.apply(padding_mlp) self.model.apply(padding_mlp)
from ipex_llm import optimize_model from ipex_llm import optimize_model
import os import os