ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Update PVC XMX condition (#9901)

Browse source 
* update pvc xmx condition

* update condition

* update conditon
This commit is contained in:
Guancheng Fu 2024-01-15 15:42:15 +08:00 committed by GitHub
parent 6637860ddf
commit 9f34da7cdb
1 changed files with 4 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
python/llm/src/bigdl/llm/transformers/models/utils.py
View file

@ -276,7 +276,10 @@ def use_xmx(x: torch.Tensor, qtype: int):
device in ["arc", "flex", "pvc"] device in ["arc", "flex", "pvc"]
and qtype in [SYM_INT4, SYM_INT8, FP8] and qtype in [SYM_INT4, SYM_INT8, FP8]
and ( and (
(device == "pvc" and 1 < x.size(0) <= 16)
or
(device != "pvc" and x.dtype == torch.float32 and 1 < x.size(0) <= 64) (device != "pvc" and x.dtype == torch.float32 and 1 < x.size(0) <= 64)
or 1 < x.size(0) <= 8 or
1 < x.size(0) <= 8
) )
) )