ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Update readme (#10214)

Browse source
This commit is contained in:
Jason Dai 2024-02-23 11:42:16 +08:00 committed by GitHub
parent f315c7f93a
commit 40584dec6d
2 changed files with 6 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
README.md
View file

@ -28,7 +28,7 @@
- [2023/10] `bigdl-llm` now supports [FastChat serving](python/llm/src/bigdl/llm/serving) on on both Intel CPU and GPU. - [2023/10] `bigdl-llm` now supports [FastChat serving](python/llm/src/bigdl/llm/serving) on on both Intel CPU and GPU.
- [2023/09] `bigdl-llm` now supports [Intel GPU](python/llm/example/GPU) (including iGPU, Arc, Flex and MAX). - [2023/09] `bigdl-llm` now supports [Intel GPU](python/llm/example/GPU) (including iGPU, Arc, Flex and MAX).
- [2023/09] `bigdl-llm` [tutorial](https://github.com/intel-analytics/bigdl-llm-tutorial) is released. - [2023/09] `bigdl-llm` [tutorial](https://github.com/intel-analytics/bigdl-llm-tutorial) is released.
- [2023/09] Over 30 models have been optimized/verified on `bigdl-llm`, including *LLaMA/LLaMA2, ChatGLM2/ChatGLM3, Mistral, Falcon, MPT, LLaVA, WizardCoder, Dolly, Whisper, Baichuan/Baichuan2, InternLM, Skywork, QWen/Qwen-VL, Aquila, MOSS,* and more; see the complete list [here](#verified-models). - [2023/09] Over 40 models have been optimized/verified on `bigdl-llm`, including *LLaMA/LLaMA2, ChatGLM2/ChatGLM3, Mistral, Falcon, MPT, LLaVA, WizardCoder, Dolly, Whisper, Baichuan/Baichuan2, InternLM, Skywork, QWen/Qwen-VL, Aquila, MOSS,* and more; see the complete list [here](#verified-models).
### `bigdl-llm` Demos ### `bigdl-llm` Demos
See the ***optimized performance*** of `chatglm2-6b` and `llama-2-13b-chat` models on 12th Gen Intel Core CPU and Intel Arc GPU below. See the ***optimized performance*** of `chatglm2-6b` and `llama-2-13b-chat` models on 12th Gen Intel Core CPU and Intel Arc GPU below.
@ -70,10 +70,11 @@ See the ***optimized performance*** of `chatglm2-6b` and `llama-2-13b-chat` mode
#### CPU INT4 #### CPU INT4
##### Install ##### Install
You may install **`bigdl-llm`** on Intel CPU as follows: You may install **`bigdl-llm`** on Intel CPU as follows:
> Note: See the [CPU installation guide](https://bigdl.readthedocs.io/en/latest/doc/LLM/Overview/install_cpu.html) for more details.
```bash ```bash
pip install --pre --upgrade bigdl-llm[all] pip install --pre --upgrade bigdl-llm[all]
``` ```
> Note: `bigdl-llm` has been tested on Python 3.9 > Note: `bigdl-llm` has been tested on Python 3.9, 3.10 and 3.11
##### Run Model ##### Run Model
You may apply INT4 optimizations to any Hugging Face *Transformers* models as follows. You may apply INT4 optimizations to any Hugging Face *Transformers* models as follows.
@ -100,7 +101,7 @@ You may install **`bigdl-llm`** on Intel GPU as follows:
# below command will install intel_extension_for_pytorch==2.1.10+xpu as default # below command will install intel_extension_for_pytorch==2.1.10+xpu as default
pip install --pre --upgrade bigdl-llm[xpu] -f https://developer.intel.com/ipex-whl-stable-xpu pip install --pre --upgrade bigdl-llm[xpu] -f https://developer.intel.com/ipex-whl-stable-xpu
``` ```
> Note: `bigdl-llm` has been tested on Python 3.9 > Note: `bigdl-llm` has been tested on Python 3.9, 3.10 and 3.11
##### Run Model ##### Run Model
You may apply INT4 optimizations to any Hugging Face *Transformers* models as follows. You may apply INT4 optimizations to any Hugging Face *Transformers* models as follows.

2
docs/readthedocs/source/doc/LLM/Overview/install_cpu.md
View file

@ -17,7 +17,7 @@ Please refer to [Environment Setup](#environment-setup) for more information.
.. important:: .. important::
``bigdl-llm`` is tested with Python 3.9, which is recommended for best practices. ``bigdl-llm`` is tested with Python 3.9, 3.10 and 3.11; Python 3.9 is recommended for best practices.
``` ```
## Recommended Requirements ## Recommended Requirements