Update ollama and llama.cpp readme (#12574)
This commit is contained in:
		
							parent
							
								
									e2ae42929a
								
							
						
					
					
						commit
						f7a2bd21cf
					
				
					 5 changed files with 5 additions and 9 deletions
				
			
		| 
						 | 
					@ -15,7 +15,7 @@
 | 
				
			||||||
> - ***70+ models** have been optimized/verified on `ipex-llm` (e.g., Llama, Phi, Mistral, Mixtral, Whisper, Qwen, MiniCPM, Qwen-VL, MiniCPM-V and more), with state-of-art **LLM optimizations**, **XPU acceleration** and **low-bit (FP8/FP6/FP4/INT4) support**; see the complete list [here](#verified-models).*
 | 
					> - ***70+ models** have been optimized/verified on `ipex-llm` (e.g., Llama, Phi, Mistral, Mixtral, Whisper, Qwen, MiniCPM, Qwen-VL, MiniCPM-V and more), with state-of-art **LLM optimizations**, **XPU acceleration** and **low-bit (FP8/FP6/FP4/INT4) support**; see the complete list [here](#verified-models).*
 | 
				
			||||||
 | 
					
 | 
				
			||||||
## Latest Update 🔥 
 | 
					## Latest Update 🔥 
 | 
				
			||||||
- [2024/12] We added support for running [Ollama 0.40.6](docs/mddocs/Quickstart/ollama_quickstart.md) on Intel GPU.
 | 
					- [2024/12] We added support for running [Ollama 0.4.6](docs/mddocs/Quickstart/ollama_quickstart.md) on Intel GPU.
 | 
				
			||||||
- [2024/12] We added both ***Python*** and ***C++*** support for Intel Core Ultra [NPU](docs/mddocs/Quickstart/npu_quickstart.md) (including 100H, 200V and 200K series).
 | 
					- [2024/12] We added both ***Python*** and ***C++*** support for Intel Core Ultra [NPU](docs/mddocs/Quickstart/npu_quickstart.md) (including 100H, 200V and 200K series).
 | 
				
			||||||
- [2024/11] We added support for running [vLLM 0.6.2](docs/mddocs/DockerGuides/vllm_docker_quickstart.md) on Intel Arc GPUs.
 | 
					- [2024/11] We added support for running [vLLM 0.6.2](docs/mddocs/DockerGuides/vllm_docker_quickstart.md) on Intel Arc GPUs.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
| 
						 | 
					
 | 
				
			||||||
| 
						 | 
					@ -17,9 +17,9 @@ See the demo of running LLaMA2-7B on Intel Arc GPU below.
 | 
				
			||||||
</table>
 | 
					</table>
 | 
				
			||||||
 | 
					
 | 
				
			||||||
> [!NOTE]
 | 
					> [!NOTE]
 | 
				
			||||||
> `ipex-llm[cpp]==2.2.0b20240826` is consistent with [62bfef5](https://github.com/ggerganov/llama.cpp/commit/62bfef5194d5582486d62da3db59bf44981b7912) of llama.cpp.
 | 
					> `ipex-llm[cpp]==2.2.0b20241204` is consistent with [a1631e5](https://github.com/ggerganov/llama.cpp/commit/a1631e53f6763e17da522ba219b030d8932900bd) of llama.cpp.
 | 
				
			||||||
>
 | 
					>
 | 
				
			||||||
> Our latest version is consistent with [a1631e5](https://github.com/ggerganov/llama.cpp/commit/a1631e53f6763e17da522ba219b030d8932900bd) of llama.cpp.
 | 
					> Our latest version is consistent with [3f1ae2e](https://github.com/ggerganov/llama.cpp/commit/3f1ae2e32cde00c39b96be6d01c2997c29bae555) of llama.cpp.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
> [!NOTE]
 | 
					> [!NOTE]
 | 
				
			||||||
> Starting from `ipex-llm[cpp]==2.2.0b20240912`, oneAPI dependency of `ipex-llm[cpp]` on Windows will switch from `2024.0.0` to `2024.2.1` .
 | 
					> Starting from `ipex-llm[cpp]==2.2.0b20240912`, oneAPI dependency of `ipex-llm[cpp]` on Windows will switch from `2024.0.0` to `2024.2.1` .
 | 
				
			||||||
| 
						 | 
					
 | 
				
			||||||
| 
						 | 
					@ -17,9 +17,9 @@
 | 
				
			||||||
</table>
 | 
					</table>
 | 
				
			||||||
 | 
					
 | 
				
			||||||
> [!NOTE]
 | 
					> [!NOTE]
 | 
				
			||||||
> `ipex-llm[cpp]==2.2.0b20240826` 版本与官方 llama.cpp 版本 [62bfef5](https://github.com/ggerganov/llama.cpp/commit/62bfef5194d5582486d62da3db59bf44981b7912) 一致。
 | 
					> `ipex-llm[cpp]==2.2.0b20241204` 版本与官方 llama.cpp 版本 [a1631e5](https://github.com/ggerganov/llama.cpp/commit/a1631e53f6763e17da522ba219b030d8932900bd) 一致。
 | 
				
			||||||
>
 | 
					>
 | 
				
			||||||
> `ipex-llm[cpp]` 的最新版本与官方 llama.cpp 版本 [a1631e5](https://github.com/ggerganov/llama.cpp/commit/a1631e53f6763e17da522ba219b030d8932900bd) 一致。
 | 
					> `ipex-llm[cpp]` 的最新版本与官方 llama.cpp 版本 [3f1ae2e](https://github.com/ggerganov/llama.cpp/commit/3f1ae2e32cde00c39b96be6d01c2997c29bae555) 一致。
 | 
				
			||||||
 | 
					
 | 
				
			||||||
> [!NOTE]
 | 
					> [!NOTE]
 | 
				
			||||||
> 从 `ipex-llm[cpp]==2.2.0b20240912` 版本开始,Windows 上 `ipex-llm[cpp]` 依赖的 oneAPI 版本已从 `2024.0.0` 更新到 `2024.2.1`。
 | 
					> 从 `ipex-llm[cpp]==2.2.0b20240912` 版本开始,Windows 上 `ipex-llm[cpp]` 依赖的 oneAPI 版本已从 `2024.0.0` 更新到 `2024.2.1`。
 | 
				
			||||||
| 
						 | 
					
 | 
				
			||||||
| 
						 | 
					@ -80,7 +80,6 @@ You may launch the Ollama service as below:
 | 
				
			||||||
  export ZES_ENABLE_SYSMAN=1
 | 
					  export ZES_ENABLE_SYSMAN=1
 | 
				
			||||||
  source /opt/intel/oneapi/setvars.sh
 | 
					  source /opt/intel/oneapi/setvars.sh
 | 
				
			||||||
  export SYCL_CACHE_PERSISTENT=1
 | 
					  export SYCL_CACHE_PERSISTENT=1
 | 
				
			||||||
  export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH
 | 
					 | 
				
			||||||
  # [optional] under most circumstances, the following environment variable may improve performance, but sometimes this may also cause performance degradation
 | 
					  # [optional] under most circumstances, the following environment variable may improve performance, but sometimes this may also cause performance degradation
 | 
				
			||||||
  export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
 | 
					  export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
 | 
				
			||||||
  # [optional] if you want to run on single GPU, use below command to limit GPU may improve performance
 | 
					  # [optional] if you want to run on single GPU, use below command to limit GPU may improve performance
 | 
				
			||||||
| 
						 | 
					@ -179,7 +178,6 @@ Then you can create the model in Ollama by `ollama create example -f Modelfile`
 | 
				
			||||||
  
 | 
					  
 | 
				
			||||||
  ```bash
 | 
					  ```bash
 | 
				
			||||||
  source /opt/intel/oneapi/setvars.sh
 | 
					  source /opt/intel/oneapi/setvars.sh
 | 
				
			||||||
  export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH
 | 
					 | 
				
			||||||
  export no_proxy=localhost,127.0.0.1
 | 
					  export no_proxy=localhost,127.0.0.1
 | 
				
			||||||
  ./ollama create example -f Modelfile
 | 
					  ./ollama create example -f Modelfile
 | 
				
			||||||
  ./ollama run example
 | 
					  ./ollama run example
 | 
				
			||||||
| 
						 | 
					
 | 
				
			||||||
| 
						 | 
					@ -80,7 +80,6 @@ IPEX-LLM 现在已支持在 Linux 和 Windows 系统上运行 `Ollama`。
 | 
				
			||||||
  export ZES_ENABLE_SYSMAN=1
 | 
					  export ZES_ENABLE_SYSMAN=1
 | 
				
			||||||
  source /opt/intel/oneapi/setvars.sh
 | 
					  source /opt/intel/oneapi/setvars.sh
 | 
				
			||||||
  export SYCL_CACHE_PERSISTENT=1
 | 
					  export SYCL_CACHE_PERSISTENT=1
 | 
				
			||||||
  export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH
 | 
					 | 
				
			||||||
  # [optional] under most circumstances, the following environment variable may improve performance, but sometimes this may also cause performance degradation
 | 
					  # [optional] under most circumstances, the following environment variable may improve performance, but sometimes this may also cause performance degradation
 | 
				
			||||||
  export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
 | 
					  export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
 | 
				
			||||||
  # [optional] if you want to run on single GPU, use below command to limit GPU may improve performance
 | 
					  # [optional] if you want to run on single GPU, use below command to limit GPU may improve performance
 | 
				
			||||||
| 
						 | 
					@ -176,7 +175,6 @@ PARAMETER num_predict 64
 | 
				
			||||||
  ```bash
 | 
					  ```bash
 | 
				
			||||||
  export no_proxy=localhost,127.0.0.1
 | 
					  export no_proxy=localhost,127.0.0.1
 | 
				
			||||||
  source /opt/intel/oneapi/setvars.sh
 | 
					  source /opt/intel/oneapi/setvars.sh
 | 
				
			||||||
  export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH
 | 
					 | 
				
			||||||
  ./ollama create example -f Modelfile
 | 
					  ./ollama create example -f Modelfile
 | 
				
			||||||
  ./ollama run example
 | 
					  ./ollama run example
 | 
				
			||||||
  ```
 | 
					  ```
 | 
				
			||||||
| 
						 | 
					
 | 
				
			||||||
		Loading…
	
		Reference in a new issue