ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

fix unexpected keyword argument 'device' (#9982)

Browse source 
* add device for chatglm3 only

* add comment for this change

* fix style

* fix style

* fix style again..

* finally fixed style
This commit is contained in:
Chen, Zhentao 2024-01-24 13:20:46 +08:00 committed by GitHub
parent de27ddd81a
commit e0db44dcb6
1 changed files with 9 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
python/llm/src/bigdl/llm/transformers/model.py
View file

@ -505,6 +505,14 @@ class _BaseAutoModelClass:
if bigdl_lcmu_enabled: if bigdl_lcmu_enabled:
with ContextManagers(init_contexts): with ContextManagers(init_contexts):
if config.architectures is not None and config.architectures[0] in \
["ChatGLMModel", "ChatGLMForConditionalGeneration"]:
"""
ChatGLMModel uses skip_init by default, which will force modules placed on cpu
if the device is not specified. This will further cause replaced linear
allocating memory on cpu.
"""
kwargs["device"] = "meta" kwargs["device"] = "meta"
model = model_class(config, *model_args, **kwargs) model = model_class(config, *model_args, **kwargs)
else: else: