ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

LLM: improve transformers style API doc (#9113)

Browse source
This commit is contained in:
binbin Deng 2023-10-10 09:31:00 +08:00 committed by GitHub
parent 65212451cc
commit e4d1457a70
3 changed files with 66 additions and 43 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
docs/readthedocs/source/doc/PythonAPI/LLM/optimize.rst
View file

@ -1,7 +1,7 @@
BigDL-LLM PyTorch API BigDL-LLM PyTorch API
===================== =====================
llm.optimize optimize model
---------------------------------------- ----------------------------------------
.. automodule:: bigdl.llm.optimize .. automodule:: bigdl.llm.optimize

53
docs/readthedocs/source/doc/PythonAPI/LLM/transformers.rst
View file

@ -1,10 +1,16 @@
BigDL-LLM `transformers`-style API BigDL-LLM ``transformers``-style API
===================== ====================================
llm.transformers.model Hugging Face ``transformers`` AutoModel
--------------------------- ------------------------------------
.. autoclass:: bigdl.llm.transformers.model.AutoModelForCausalLM You can apply BigDL-LLM optimizations on any Hugging Face Transformers models by using the standard AutoModel APIs.
AutoModelForCausalLM
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. autoclass:: bigdl.llm.transformers.AutoModelForCausalLM
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
@ -13,8 +19,10 @@ llm.transformers.model
.. automethod:: load_convert .. automethod:: load_convert
.. automethod:: load_low_bit .. automethod:: load_low_bit
AutoModel
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. autoclass:: bigdl.llm.transformers.model.AutoModel .. autoclass:: bigdl.llm.transformers.AutoModel
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
@ -23,8 +31,10 @@ llm.transformers.model
.. automethod:: load_convert .. automethod:: load_convert
.. automethod:: load_low_bit .. automethod:: load_low_bit
AutoModelForSpeechSeq2Seq
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. autoclass:: bigdl.llm.transformers.model.AutoModelForSpeechSeq2Seq .. autoclass:: bigdl.llm.transformers.AutoModelForSpeechSeq2Seq
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
@ -33,8 +43,10 @@ llm.transformers.model
.. automethod:: load_convert .. automethod:: load_convert
.. automethod:: load_low_bit .. automethod:: load_low_bit
AutoModelForSeq2SeqLM
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. autoclass:: bigdl.llm.transformers.model.AutoModelForSeq2SeqLM .. autoclass:: bigdl.llm.transformers.AutoModelForSeq2SeqLM
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
@ -45,10 +57,17 @@ llm.transformers.model
llm.transformers.modelling_bigdl Native Model
---------------------------------------- ----------------------------------------
.. autoclass:: bigdl.llm.transformers.modelling_bigdl.LlamaForCausalLM For ``llama``/``chatglm``/``bloom``/``gptneox``/``starcoder`` model families, you may also convert and run LLM using the native (cpp) implementation for maximum performance.
.. tabs::
.. tab:: Llama
.. autoclass:: bigdl.llm.transformers.LlamaForCausalLM
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
@ -56,8 +75,9 @@ llm.transformers.modelling_bigdl
.. automethod:: from_pretrained .. automethod:: from_pretrained
.. tab:: ChatGLM
.. autoclass:: bigdl.llm.transformers.modelling_bigdl.ChatGLMForCausalLM .. autoclass:: bigdl.llm.transformers.ChatGLMForCausalLM
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
@ -65,8 +85,9 @@ llm.transformers.modelling_bigdl
.. automethod:: from_pretrained .. automethod:: from_pretrained
.. tab:: Gptneox
.. autoclass:: bigdl.llm.transformers.modelling_bigdl.GptneoxForCausalLM .. autoclass:: bigdl.llm.transformers.GptneoxForCausalLM
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
@ -74,8 +95,8 @@ llm.transformers.modelling_bigdl
.. automethod:: from_pretrained .. automethod:: from_pretrained
.. tab:: Bloom
.. autoclass:: bigdl.llm.transformers.modelling_bigdl.BloomForCausalLM .. autoclass:: bigdl.llm.transformers.BloomForCausalLM
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
@ -83,7 +104,9 @@ llm.transformers.modelling_bigdl
.. automethod:: from_pretrained .. automethod:: from_pretrained
.. autoclass:: bigdl.llm.transformers.modelling_bigdl.StarcoderForCausalLM .. tab:: Starcoder
.. autoclass:: bigdl.llm.transformers.StarcoderForCausalLM
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:

4
python/llm/src/bigdl/llm/utils/lazy_load_torch.py
View file

@ -46,7 +46,7 @@ from torch.serialization import StorageType
import pickle import pickle
import zipfile import zipfile
import io import io
from typing import Dict, IO, Any, Callable from typing import Dict, IO, Any, Callable, List
from dataclasses import dataclass from dataclasses import dataclass
from .common import invalidInputError from .common import invalidInputError
@ -69,7 +69,7 @@ class LazyStorage:
@dataclass @dataclass
class LazyTensor: class LazyTensor:
_load: Callable[[], torch.Tensor] _load: Callable[[], torch.Tensor]
shape: list[int] shape: List[int]
data_type: torch.dtype data_type: torch.dtype
description: str description: str