ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

fix multiple gpu usage of harness (#9444)

Browse source
This commit is contained in:
Chen, Zhentao 2023-11-13 16:53:23 +08:00 committed by GitHub
parent da6bbc8c11
commit 5747e2fe69
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
python/llm/dev/benchmark/harness/bigdl-llm.patch
View file

@ -83,7 +83,7 @@ index 00000000..74010da3
+ +
+ assert isinstance(pretrained, str) + assert isinstance(pretrained, str)
+ assert isinstance(batch_size, (int,str)) + assert isinstance(batch_size, (int,str))
+ if device == 'xpu': + if 'xpu' in device:
+ import intel_extension_for_pytorch as ipex + import intel_extension_for_pytorch as ipex
+ model = AutoModelForCausalLM.from_pretrained(pretrained, + model = AutoModelForCausalLM.from_pretrained(pretrained,
+ load_in_low_bit=load_in_low_bit, + load_in_low_bit=load_in_low_bit,