ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

LLM: update langchain api document page (#9222)

Browse source
This commit is contained in:
binbin Deng 2023-10-24 10:13:41 +08:00 committed by GitHub
parent 0c5055d38c
commit db37edae8a
7 changed files with 146 additions and 123 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
docs/readthedocs/source/doc/LLM/Overview/KeyFeatures/hugging_face_format.md
View file

@ -25,7 +25,7 @@ output = tokenizer.batch_decode(output_ids)
```eval_rst ```eval_rst
.. seealso:: .. seealso::
See the complete examples `here <https://github.com/intel-analytics/BigDL/blob/main/python/llm/example/transformers/transformers_int4>`_ See the complete CPU examples `here <https://github.com/intel-analytics/BigDL/tree/main/python/llm/example/CPU/HF-Transformers-AutoModels>`_ and GPU examples `here <https://github.com/intel-analytics/BigDL/tree/main/python/llm/example/GPU/HF-Transformers-AutoModels>`_.
.. note:: .. note::
@ -35,7 +35,7 @@ output = tokenizer.batch_decode(output_ids)
model = AutoModelForCausalLM.from_pretrained('/path/to/model/', load_in_low_bit="sym_int5") model = AutoModelForCausalLM.from_pretrained('/path/to/model/', load_in_low_bit="sym_int5")
See the complete example `here <https://github.com/intel-analytics/BigDL/blob/main/python/llm/example/transformers/transformers_low_bit>`_. See the CPU example `here <https://github.com/intel-analytics/BigDL/tree/main/python/llm/example/CPU/HF-Transformers-AutoModels/More-Data-Types>`_ and GPU example `here <https://github.com/intel-analytics/BigDL/tree/main/python/llm/example/GPU/HF-Transformers-AutoModels/More-Data-Types>`_.
``` ```
## Save & Load ## Save & Load
@ -50,5 +50,5 @@ new_model = AutoModelForCausalLM.load_low_bit(model_path)
```eval_rst ```eval_rst
.. seealso:: .. seealso::
See the examples `here <https://github.com/intel-analytics/BigDL/blob/main/python/llm/example/transformers/transformers_low_bit>`_ See the CPU example `here <https://github.com/intel-analytics/BigDL/tree/main/python/llm/example/CPU/HF-Transformers-AutoModels/Save-Load>`_ and GPU example `here <https://github.com/intel-analytics/BigDL/tree/main/python/llm/example/GPU/HF-Transformers-AutoModels/Save-Load>`_
``` ```

4
docs/readthedocs/source/doc/LLM/Overview/KeyFeatures/langchain_api.md
View file

@ -21,7 +21,7 @@ output = doc_chain.run(...)
```eval_rst ```eval_rst
.. seealso:: .. seealso::
See the examples `here <https://github.com/intel-analytics/BigDL/blob/main/python/llm/example/langchain/transformers_int4>`_. See the examples `here <https://github.com/intel-analytics/BigDL/tree/main/python/llm/example/CPU/LangChain/transformers_int4>`_.
``` ```
## Using Native INT4 Format ## Using Native INT4 Format
@ -53,5 +53,5 @@ doc_chain.run(...)
```eval_rst ```eval_rst
.. seealso:: .. seealso::
See the examples `here <https://github.com/intel-analytics/BigDL/blob/main/python/llm/example/langchain/native_int4>`_. See the examples `here <https://github.com/intel-analytics/BigDL/tree/main/python/llm/example/CPU/LangChain/native_int4>`_.
``` ```

2
docs/readthedocs/source/doc/LLM/Overview/KeyFeatures/native_format.md
View file

@ -28,5 +28,5 @@ output = llm.batch_decode(output_ids)
```eval_rst ```eval_rst
.. seealso:: .. seealso::
See the complete example `here <https://github.com/intel-analytics/BigDL/blob/main/python/llm/example/transformers/native_int4/native_int4_pipeline.py>`_ See the complete example `here <https://github.com/intel-analytics/BigDL/tree/main/python/llm/example/CPU/Native-Models>`_
``` ```

168
docs/readthedocs/source/doc/PythonAPI/LLM/langchain.rst
View file

@ -1,144 +1,164 @@
BigDL-LLM LangChain API BigDL-LLM LangChain API
===================== =====================
llm.langchain.embeddings.bigdlllm LLM Wrapper of LangChain
---------------------------------------- ----------------------------------------
.. autoclass:: bigdl.llm.langchain.embeddings.bigdlllm.LlamaEmbeddings Hugging Face ``transformers`` Format
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
BigDL-LLM provides ``TransformersLLM`` and ``TransformersPipelineLLM``, which implement the standard interface of LLM wrapper of LangChain.
.. tabs::
.. tab:: AutoModel
.. automodule:: bigdl.llm.langchain.llms.transformersllm
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
:exclude-members: ggml_model , ggml_module :exclude-members: model_id, model_kwargs, model, tokenizer, streaming, Config
.. automethod:: validate_environment .. tab:: pipeline
.. automethod:: embed_documents
.. automethod:: embed_query
.. automodule:: bigdl.llm.langchain.llms.transformerspipelinellm
.. autoclass:: bigdl.llm.langchain.embeddings.bigdlllm.BloomEmbeddings
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
:exclude-members: ggml_model , ggml_module :exclude-members: pipeline, model_id, model_kwargs, pipeline_kwargs, Config
.. automethod:: validate_environment
.. automethod:: embed_documents
.. automethod:: embed_query
.. autoclass:: bigdl.llm.langchain.embeddings.bigdlllm.GptneoxEmbeddings Native Model
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For ``llama``/``chatglm``/``bloom``/``gptneox``/``starcoder`` model families, you could also use the following LLM wrappers with the native (cpp) implementation for maximum performance.
.. tabs::
.. tab:: Llama
.. autoclass:: bigdl.llm.langchain.llms.bigdlllm.LlamaLLM
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
:exclude-members: ggml_model , ggml_module :exclude-members: ggml_model, ggml_module, client, model_path, kwargs
.. automethod:: validate_environment .. automethod:: validate_environment
.. automethod:: embed_documents .. automethod:: stream
.. automethod:: embed_query .. automethod:: get_num_tokens
.. tab:: ChatGLM
.. autoclass:: bigdl.llm.langchain.embeddings.bigdlllm.ChatGLMEmbeddings .. autoclass:: bigdl.llm.langchain.llms.bigdlllm.ChatGLMLLM
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
:exclude-members: ggml_model , ggml_module :exclude-members: ggml_model, ggml_module, client, model_path, kwargs
.. automethod:: validate_environment .. automethod:: validate_environment
.. automethod:: embed_documents .. automethod:: stream
.. automethod:: embed_query .. automethod:: get_num_tokens
.. tab:: Bloom
.. autoclass:: bigdl.llm.langchain.embeddings.bigdlllm.StarcoderEmbeddings .. autoclass:: bigdl.llm.langchain.llms.bigdlllm.BloomLLM
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
:exclude-members: ggml_model , ggml_module :exclude-members: ggml_model, ggml_module, client, model_path, kwargs
.. automethod:: validate_environment .. automethod:: validate_environment
.. automethod:: embed_documents .. automethod:: stream
.. automethod:: embed_query .. automethod:: get_num_tokens
.. tab:: Gptneox
.. autoclass:: bigdl.llm.langchain.llms.bigdlllm.GptneoxLLM
:members:
:undoc-members:
:show-inheritance:
:exclude-members: ggml_model, ggml_module, client, model_path, kwargs
.. automethod:: validate_environment
.. automethod:: stream
.. automethod:: get_num_tokens
.. tab:: Starcoder
.. autoclass:: bigdl.llm.langchain.llms.bigdlllm.StarcoderLLM
:members:
:undoc-members:
:show-inheritance:
:exclude-members: ggml_model, ggml_module, client, model_path, kwargs
.. automethod:: validate_environment
.. automethod:: stream
.. automethod:: get_num_tokens
llm.langchain.embeddings.transformersembeddings Embeddings Wrapper of LangChain
-------------------------------------------------- ----------------------------------------
Hugging Face ``transformers`` AutoModel
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. automodule:: bigdl.llm.langchain.embeddings.transformersembeddings .. automodule:: bigdl.llm.langchain.embeddings.transformersembeddings
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
:exclude-members: model, tokenizer, model_id, model_kwargs, encode_kwargs, Config
Native Model
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
llm.langchain.llms.bigdlllm For ``llama``/``bloom``/``gptneox``/``starcoder`` model families, you could also use the following wrappers.
----------------------------------------
.. autoclass:: bigdl.llm.langchain.llms.bigdlllm.LlamaLLM .. tabs::
.. tab:: Llama
.. autoclass:: bigdl.llm.langchain.embeddings.bigdlllm.LlamaEmbeddings
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
:exclude-members: ggml_model , ggml_module :exclude-members: ggml_model, ggml_module, client, model_path, kwargs
.. automethod:: validate_environment .. automethod:: validate_environment
.. automethod:: stream .. automethod:: embed_documents
.. automethod:: get_num_tokens .. automethod:: embed_query
.. tab:: Bloom
.. autoclass:: bigdl.llm.langchain.llms.bigdlllm.BloomLLM .. autoclass:: bigdl.llm.langchain.embeddings.bigdlllm.BloomEmbeddings
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
:exclude-members: ggml_model , ggml_module :exclude-members: ggml_model, ggml_module, client, model_path, kwargs
.. automethod:: validate_environment .. automethod:: validate_environment
.. automethod:: stream .. automethod:: embed_documents
.. automethod:: get_num_tokens .. automethod:: embed_query
.. tab:: Gptneox
.. autoclass:: bigdl.llm.langchain.llms.bigdlllm.GptneoxLLM .. autoclass:: bigdl.llm.langchain.embeddings.bigdlllm.GptneoxEmbeddings
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
:exclude-members: ggml_model , ggml_module :exclude-members: ggml_model, ggml_module, client, model_path, kwargs
.. automethod:: validate_environment .. automethod:: validate_environment
.. automethod:: stream .. automethod:: embed_documents
.. automethod:: get_num_tokens .. automethod:: embed_query
.. tab:: Starcoder
.. autoclass:: bigdl.llm.langchain.llms.bigdlllm.ChatGLMLLM .. autoclass:: bigdl.llm.langchain.embeddings.bigdlllm.StarcoderEmbeddings
:members: :members:
:undoc-members: :undoc-members:
:show-inheritance: :show-inheritance:
:exclude-members: ggml_model , ggml_module :exclude-members: ggml_model, ggml_module, client, model_path, kwargs
.. automethod:: validate_environment .. automethod:: validate_environment
.. automethod:: stream .. automethod:: embed_documents
.. automethod:: get_num_tokens .. automethod:: embed_query
.. autoclass:: bigdl.llm.langchain.llms.bigdlllm.StarcoderLLM
:members:
:undoc-members:
:show-inheritance:
:exclude-members: ggml_model , ggml_module
.. automethod:: validate_environment
.. automethod:: stream
.. automethod:: get_num_tokens
llm.langchain.llms.transformersllm
----------------------------------------
.. automodule:: bigdl.llm.langchain.llms.transformersllm
:members:
:undoc-members:
:show-inheritance:
llm.langchain.llms.transformerspipelinellm
---------------------------------------------
.. automodule:: bigdl.llm.langchain.llms.transformerspipelinellm
:members:
:undoc-members:
:show-inheritance:

4
docs/readthedocs/source/index.rst
View file

@ -24,8 +24,8 @@ BigDL-LLM: low-Bit LLM library
============================================ ============================================
Latest update Latest update
============================================ ============================================
- **[New]** ``bigdl-llm`` now supports QLoRA fintuning on Intel GPU; see the the example `here <https://github.com/intel-analytics/BigDL/tree/main/python/llm/example/gpu/qlora_finetuning>`_. - **[New]** ``bigdl-llm`` now supports QLoRA fintuning on Intel GPU; see the the example `here <https://github.com/intel-analytics/BigDL/tree/main/python/llm/example/GPU/QLoRA-FineTuning>`_.
- ``bigdl-llm`` now supports Intel GPU (including Arc, Flex and MAX); see the the latest GPU examples `here <https://github.com/intel-analytics/BigDL/tree/main/python/llm/example/gpu>`_. - ``bigdl-llm`` now supports Intel GPU (including Arc, Flex and MAX); see the the latest GPU examples `here <https://github.com/intel-analytics/BigDL/tree/main/python/llm/example/GPU>`_.
- ``bigdl-llm`` tutorial is released `here <https://github.com/intel-analytics/bigdl-llm-tutorial>`_. - ``bigdl-llm`` tutorial is released `here <https://github.com/intel-analytics/bigdl-llm-tutorial>`_.
- Over 20 models have been verified on ``bigdl-llm``, including *LLaMA/LLaMA2, ChatGLM/ChatGLM2, MPT, Falcon, Dolly, StarCoder, Whisper, InternLM, QWen, Baichuan, Aquila, MOSS* and more; see the complete list `here <https://github.com/intel-analytics/bigdl#verified-models>`_. - Over 20 models have been verified on ``bigdl-llm``, including *LLaMA/LLaMA2, ChatGLM/ChatGLM2, MPT, Falcon, Dolly, StarCoder, Whisper, InternLM, QWen, Baichuan, Aquila, MOSS* and more; see the complete list `here <https://github.com/intel-analytics/bigdl#verified-models>`_.

3
python/llm/src/bigdl/llm/langchain/embeddings/transformersembeddings.py
View file

@ -95,7 +95,8 @@ class TransformersEmbeddings(BaseModel, Embeddings):
model_kwargs: Keyword arguments that will be passed to the model and tokenizer. model_kwargs: Keyword arguments that will be passed to the model and tokenizer.
kwargs: Extra arguments that will be passed to the model and tokenizer. kwargs: Extra arguments that will be passed to the model and tokenizer.
Returns: An object of TransformersEmbeddings. Returns:
An object of TransformersEmbeddings.
""" """
try: try:
from bigdl.llm.transformers import AutoModel from bigdl.llm.transformers import AutoModel

6
python/llm/src/bigdl/llm/langchain/llms/transformersllm.py
View file

@ -101,7 +101,8 @@ class TransformersLLM(LLM):
model_kwargs: Keyword arguments that will be passed to the model and tokenizer. model_kwargs: Keyword arguments that will be passed to the model and tokenizer.
kwargs: Extra arguments that will be passed to the model and tokenizer. kwargs: Extra arguments that will be passed to the model and tokenizer.
Returns: An object of TransformersLLM. Returns:
An object of TransformersLLM.
""" """
try: try:
from bigdl.llm.transformers import ( from bigdl.llm.transformers import (
@ -159,7 +160,8 @@ class TransformersLLM(LLM):
model_kwargs: Keyword arguments that will be passed to the model and tokenizer. model_kwargs: Keyword arguments that will be passed to the model and tokenizer.
kwargs: Extra arguments that will be passed to the model and tokenizer. kwargs: Extra arguments that will be passed to the model and tokenizer.
Returns: An object of TransformersLLM. Returns:
An object of TransformersLLM.
""" """
try: try:
from bigdl.llm.transformers import ( from bigdl.llm.transformers import (