ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Add half precision for fastchat models (#11130)

Browse source
This commit is contained in:
Guancheng Fu 2024-05-24 15:41:14 +08:00 committed by GitHub
parent 65f4212f89
commit 7f772c5a4f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 2 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
python/llm/src/ipex_llm/transformers/loader.py
View file

@ -91,6 +91,8 @@ def load_model(
if device == "xpu": if device == "xpu":
import intel_extension_for_pytorch as ipex import intel_extension_for_pytorch as ipex
print("Convert model to half precision...")
model = model.half()
model = model.to('xpu') model = model.to('xpu')
return model, tokenizer return model, tokenizer