LLM: Modify CPU Installation Command for documentation (#11042)
* init * refine * refine * refine * refine comments
This commit is contained in:
		
							parent
							
								
									fff067d240
								
							
						
					
					
						commit
						d963e95363
					
				
					 2 changed files with 46 additions and 8 deletions
				
			
		| 
						 | 
					@ -4,8 +4,20 @@
 | 
				
			||||||
 | 
					
 | 
				
			||||||
Install IPEX-LLM for CPU supports using pip through:
 | 
					Install IPEX-LLM for CPU supports using pip through:
 | 
				
			||||||
 | 
					
 | 
				
			||||||
```bash
 | 
					```eval_rst	
 | 
				
			||||||
pip install --pre --upgrade ipex-llm[all] # install the latest ipex-llm nightly build with 'all' option
 | 
					.. tabs::
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					   .. tab:: Linux
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					      .. code-block:: bash
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					         pip install --pre --upgrade ipex-llm[all] --extra-index-url https://download.pytorch.org/whl/cpu
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					   .. tab:: Windows
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					      .. code-block:: cmd
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					         pip install --pre --upgrade ipex-llm[all]
 | 
				
			||||||
```
 | 
					```
 | 
				
			||||||
 | 
					
 | 
				
			||||||
Please refer to [Environment Setup](#environment-setup) for more information.
 | 
					Please refer to [Environment Setup](#environment-setup) for more information.
 | 
				
			||||||
| 
						 | 
					@ -41,11 +53,26 @@ For optimal performance with LLM models using IPEX-LLM optimizations on Intel CP
 | 
				
			||||||
 | 
					
 | 
				
			||||||
First we recommend using [Conda](https://docs.conda.io/en/latest/miniconda.html) to create a python 3.11 enviroment:
 | 
					First we recommend using [Conda](https://docs.conda.io/en/latest/miniconda.html) to create a python 3.11 enviroment:
 | 
				
			||||||
 | 
					
 | 
				
			||||||
```bash
 | 
					```eval_rst	
 | 
				
			||||||
conda create -n llm python=3.11
 | 
					.. tabs::
 | 
				
			||||||
conda activate llm
 | 
					 | 
				
			||||||
 | 
					
 | 
				
			||||||
pip install --pre --upgrade ipex-llm[all] # install the latest ipex-llm nightly build with 'all' option
 | 
					   .. tab:: Linux
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					      .. code-block:: bash
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					         conda create -n llm python=3.11
 | 
				
			||||||
 | 
					         conda activate llm
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					         pip install --pre --upgrade ipex-llm[all] --extra-index-url https://download.pytorch.org/whl/cpu
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					   .. tab:: Windows
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					      .. code-block:: cmd
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					         conda create -n llm python=3.11
 | 
				
			||||||
 | 
					         conda activate llm
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					         pip install --pre --upgrade ipex-llm[all]
 | 
				
			||||||
```
 | 
					```
 | 
				
			||||||
 | 
					
 | 
				
			||||||
Then for running a LLM model with IPEX-LLM optimizations (taking an `example.py` an example):
 | 
					Then for running a LLM model with IPEX-LLM optimizations (taking an `example.py` an example):
 | 
				
			||||||
| 
						 | 
					
 | 
				
			||||||
| 
						 | 
					@ -8,14 +8,25 @@ To run these examples with IPEX-LLM, we have some recommended requirements for y
 | 
				
			||||||
In the example [generate.py](./generate.py), we show a basic use case for a Baichuan model to predict the next N tokens using `generate()` API, with IPEX-LLM INT4 optimizations.
 | 
					In the example [generate.py](./generate.py), we show a basic use case for a Baichuan model to predict the next N tokens using `generate()` API, with IPEX-LLM INT4 optimizations.
 | 
				
			||||||
### 1. Install
 | 
					### 1. Install
 | 
				
			||||||
We suggest using conda to manage environment:
 | 
					We suggest using conda to manage environment:
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					On Linux:
 | 
				
			||||||
```bash
 | 
					```bash
 | 
				
			||||||
conda create -n llm python=3.11
 | 
					conda create -n llm python=3.11
 | 
				
			||||||
conda activate llm
 | 
					conda activate llm
 | 
				
			||||||
 | 
					
 | 
				
			||||||
pip install ipex-llm[all] # install ipex-llm with 'all' option
 | 
					pip install --pre --upgrade ipex-llm[all] --extra-index-url https://download.pytorch.org/whl/cpu # install ipex-llm with 'all' option
 | 
				
			||||||
pip install transformers_stream_generator  # additional package required for Baichuan-13B-Chat to conduct generation
 | 
					pip install transformers_stream_generator  # additional package required for Baichuan-13B-Chat to conduct generation
 | 
				
			||||||
```
 | 
					```
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					On Windows:
 | 
				
			||||||
 | 
					```cmd
 | 
				
			||||||
 | 
					conda create -n llm python=3.11
 | 
				
			||||||
 | 
					conda activate llm
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					pip install --pre --upgrade ipex-llm[all]
 | 
				
			||||||
 | 
					pip install transformers_stream_generator
 | 
				
			||||||
 | 
					```
 | 
				
			||||||
 | 
					
 | 
				
			||||||
### 2. Run
 | 
					### 2. Run
 | 
				
			||||||
```
 | 
					```
 | 
				
			||||||
python ./generate.py --repo-id-or-model-path REPO_ID_OR_MODEL_PATH --prompt PROMPT --n-predict N_PREDICT
 | 
					python ./generate.py --repo-id-or-model-path REPO_ID_OR_MODEL_PATH --prompt PROMPT --n-predict N_PREDICT
 | 
				
			||||||
| 
						 | 
					@ -32,7 +43,7 @@ Arguments info:
 | 
				
			||||||
 | 
					
 | 
				
			||||||
#### 2.1 Client
 | 
					#### 2.1 Client
 | 
				
			||||||
On client Windows machine, it is recommended to run directly with full utilization of all cores:
 | 
					On client Windows machine, it is recommended to run directly with full utilization of all cores:
 | 
				
			||||||
```powershell
 | 
					```cmd
 | 
				
			||||||
python ./generate.py 
 | 
					python ./generate.py 
 | 
				
			||||||
```
 | 
					```
 | 
				
			||||||
 | 
					
 | 
				
			||||||
| 
						 | 
					
 | 
				
			||||||
		Loading…
	
		Reference in a new issue