ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

use AutoTokenizer to enable more models (#9446)

Browse source
This commit is contained in:
Chen, Zhentao 2023-11-13 17:47:43 +08:00 committed by GitHub
parent ece5805572
commit 0ecb9efb05
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
python/llm/dev/benchmark/harness/bigdl-llm.patch
View file

@ -95,7 +95,7 @@ index 00000000..74010da3
+ self._device = device + self._device = device
+ self.model = model.to(device) + self.model = model.to(device)
+ +
+ self.tokenizer = LlamaTokenizer.from_pretrained(pretrained, trust_remote_code=True) + self.tokenizer = AutoTokenizer.from_pretrained(pretrained, trust_remote_code=True)
+ +
+ # setup for automatic batch size detection + # setup for automatic batch size detection
+ if batch_size == 'auto': + if batch_size == 'auto':