02c583144c
* langchain intergrations and examples * add licences and rename * add licences * fix license issues and change backbone to model_family * update examples to use model_family param * fix linting * fix code style * exclude langchain integration from stylecheck * update langchain examples and update integrations based on latets changes * update simple llama-cpp-python style API example * remove bloom in README * change default n_threads to 2 and remove redundant code --------- Co-authored-by: leonardozcm <changmin.zhao@intel.com> |
||
|---|---|---|
| .. | ||
| docqa.py | ||
| README.md | ||
| streamchat.py | ||
Langchain examples
The examples here shows how to use langchain with bigdl-llm.
Install bigdl-llm
Follow the instructions in bigdl-llm docs: Install.
Install Required Dependencies for langchain examples.
pip install langchain==0.0.184
pip install -U chromadb==0.3.25
pip install -U typing_extensions==4.5.0
Note that typing_extensions==4.5.0 is required, or you may encounter error TypeError: dataclass_transform() got an unexpected keyword argument 'field_specifiers' when running the examples.
Convert Models using bigdl-llm
Follow the instructions in bigdl-llm docs: Convert Models.
Run the examples
1. Streaming Chat
python ./streamchat.py -m MODEL_PATH -x MODEL_FAMILY -t THREAD_NUM -q "What is AI?"
arguments info:
-m MODEL_PATH: path to the converted model-x MODEL_FAMILY: the model family of the model specified in-m, available options arellama,gptneox-q QUESTION: question to ask. Default isWhat is AI?.-t THREAD_NUM: required argument defining the number of threads to use for inference. Default is2.
2. Question Answering over Docs
python ./docqa.py --t THREAD_NUM -m -x
arguments info:
-m CONVERTED_MODEL_PATH: path to the converted model in above step-x MODEL_FAMILY: the model family of the model specified in-m, available options arellama,gptneox-q QUESTION: question to ask, default question isWhat is AI?.-t THREAD_NUM: required argument defining the number of threads to use for inference. Default is2.