LLM: Modify CPU Installation Command for documentation (#11042)
* init * refine * refine * refine * refine comments
This commit is contained in:
parent
fff067d240
commit
d963e95363
2 changed files with 46 additions and 8 deletions
|
|
@ -4,8 +4,20 @@
|
|||
|
||||
Install IPEX-LLM for CPU supports using pip through:
|
||||
|
||||
```bash
|
||||
pip install --pre --upgrade ipex-llm[all] # install the latest ipex-llm nightly build with 'all' option
|
||||
```eval_rst
|
||||
.. tabs::
|
||||
|
||||
.. tab:: Linux
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install --pre --upgrade ipex-llm[all] --extra-index-url https://download.pytorch.org/whl/cpu
|
||||
|
||||
.. tab:: Windows
|
||||
|
||||
.. code-block:: cmd
|
||||
|
||||
pip install --pre --upgrade ipex-llm[all]
|
||||
```
|
||||
|
||||
Please refer to [Environment Setup](#environment-setup) for more information.
|
||||
|
|
@ -41,11 +53,26 @@ For optimal performance with LLM models using IPEX-LLM optimizations on Intel CP
|
|||
|
||||
First we recommend using [Conda](https://docs.conda.io/en/latest/miniconda.html) to create a python 3.11 enviroment:
|
||||
|
||||
```bash
|
||||
```eval_rst
|
||||
.. tabs::
|
||||
|
||||
.. tab:: Linux
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
conda create -n llm python=3.11
|
||||
conda activate llm
|
||||
|
||||
pip install --pre --upgrade ipex-llm[all] # install the latest ipex-llm nightly build with 'all' option
|
||||
pip install --pre --upgrade ipex-llm[all] --extra-index-url https://download.pytorch.org/whl/cpu
|
||||
|
||||
.. tab:: Windows
|
||||
|
||||
.. code-block:: cmd
|
||||
|
||||
conda create -n llm python=3.11
|
||||
conda activate llm
|
||||
|
||||
pip install --pre --upgrade ipex-llm[all]
|
||||
```
|
||||
|
||||
Then for running a LLM model with IPEX-LLM optimizations (taking an `example.py` an example):
|
||||
|
|
|
|||
|
|
@ -8,14 +8,25 @@ To run these examples with IPEX-LLM, we have some recommended requirements for y
|
|||
In the example [generate.py](./generate.py), we show a basic use case for a Baichuan model to predict the next N tokens using `generate()` API, with IPEX-LLM INT4 optimizations.
|
||||
### 1. Install
|
||||
We suggest using conda to manage environment:
|
||||
|
||||
On Linux:
|
||||
```bash
|
||||
conda create -n llm python=3.11
|
||||
conda activate llm
|
||||
|
||||
pip install ipex-llm[all] # install ipex-llm with 'all' option
|
||||
pip install --pre --upgrade ipex-llm[all] --extra-index-url https://download.pytorch.org/whl/cpu # install ipex-llm with 'all' option
|
||||
pip install transformers_stream_generator # additional package required for Baichuan-13B-Chat to conduct generation
|
||||
```
|
||||
|
||||
On Windows:
|
||||
```cmd
|
||||
conda create -n llm python=3.11
|
||||
conda activate llm
|
||||
|
||||
pip install --pre --upgrade ipex-llm[all]
|
||||
pip install transformers_stream_generator
|
||||
```
|
||||
|
||||
### 2. Run
|
||||
```
|
||||
python ./generate.py --repo-id-or-model-path REPO_ID_OR_MODEL_PATH --prompt PROMPT --n-predict N_PREDICT
|
||||
|
|
@ -32,7 +43,7 @@ Arguments info:
|
|||
|
||||
#### 2.1 Client
|
||||
On client Windows machine, it is recommended to run directly with full utilization of all cores:
|
||||
```powershell
|
||||
```cmd
|
||||
python ./generate.py
|
||||
```
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue