diff --git a/docs/mddocs/Quickstart/ollama_portablze_zip_quickstart.md b/docs/mddocs/Quickstart/ollama_portablze_zip_quickstart.md index f34c974f..c31abbf9 100644 --- a/docs/mddocs/Quickstart/ollama_portablze_zip_quickstart.md +++ b/docs/mddocs/Quickstart/ollama_portablze_zip_quickstart.md @@ -79,4 +79,19 @@ To increase the context length, you could set environment variable `IPEX_LLM_NUM - Start Ollama serve through `start-ollama.bat` > [!TIP] -> `IPEX_LLM_NUM_CTX` has a higher priority than the `num_ctx` settings in a models' `Modelfile`. \ No newline at end of file +> `IPEX_LLM_NUM_CTX` has a higher priority than the `num_ctx` settings in a models' `Modelfile`. + +### Additional models supported after Ollama v0.5.4 + +The currently Ollama Portable Zip is based on Ollama v0.5.4; in addition, the following new models have also been supported in the Ollama Portable Zip: + + | Model | Download | Model Link | + | - | - | - | + | DeepSeek-R1 | `ollama run deepseek-r1` | [deepseek-r1](https://ollama.com/library/deepseek-r1) | + | Openthinker | `ollama run openthinker` | [openthinker](https://ollama.com/library/openthinker) | + | DeepScaleR | `ollama run deepscaler` | [deepscaler](https://ollama.com/library/deepscaler) | + | Phi-4 | `ollama run phi4` | [phi4](https://ollama.com/library/phi4) | + | Dolphin 3.0 | `ollama run dolphin3` | [dolphin3](https://ollama.com/library/dolphin3) | + | Smallthinker | `ollama run smallthinker` | [smallthinker](https://ollama.com/library/smallthinker) | + | Granite3.1-Dense | `ollama run granite3-dense` | [granite3.1-dense](https://ollama.com/library/granite3.1-dense) | + | Granite3.1-Moe-3B | `ollama run granite3-moe` | [granite3.1-moe](https://ollama.com/library/granite3.1-moe) | diff --git a/docs/mddocs/Quickstart/ollama_portablze_zip_quickstart.zh-CN.md b/docs/mddocs/Quickstart/ollama_portablze_zip_quickstart.zh-CN.md index d8d5e49b..dab796f0 100644 --- a/docs/mddocs/Quickstart/ollama_portablze_zip_quickstart.zh-CN.md +++ b/docs/mddocs/Quickstart/ollama_portablze_zip_quickstart.zh-CN.md @@ -13,6 +13,7 @@ - [步骤 1：下载和解压](#步骤-1下载和解压) - [步骤 2：启动 Ollama Serve](#步骤-2启动-ollama-serve) - [步骤 3：运行 Ollama](#步骤-3运行-ollama) +- [提示和故障排除](#提示和故障排除) ## 系统环境准备 @@ -46,3 +47,51 @@
+ +## 提示和故障排除 + +### 通过切换源提升模型下载速度 + +Ollama 默认从 [Ollama 库](https://ollama.com/library) 下载模型。在 [运行 Ollama](#步骤-3运行-ollama) 之前设置环境变量 `IPEX_LLM_MODEL_SOURCE` 为 `modelscope/ollama`，你可以切换模型的首选下载源。 + +例如，如果你想运行 `deepseek-r1:7b` 但从 Ollama 库的下载速度较慢，可以通过如下方式改用 [ModelScope](https://www.modelscope.cn/models) 的 [模型源](https://www.modelscope.cn/models/unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF)： + +- 打开 “命令提示符”（cmd），并通过 `cd /d PATH\TO\EXTRACTED\FOLDER` 命令进入解压后的文件夹 +- 在 “命令提示符” 中运行 `set IPEX_LLM_MODEL_SOURCE=modelscope` +- 运行 `ollama run deepseek-r1:7b` + +> [!Tip] +> 使用 `set IPEX_LLM_MODEL_SOURCE=modelscope` 下载的模型，在执行 `ollama list` 时仍会显示实际的模型 ID，例如： +> ``` +> NAME ID SIZE MODIFIED +> modelscope.cn/unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF:Q4_K_M f482d5af6aec 4.7 GB About a minute ago +> ``` +> 除了 `ollama run` 和 `ollama pull`，其他操作中模型应通过其实际 ID 进行识别，例如： `ollama rm modelscope.cn/unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF:Q4_K_M` + +### 在 Ollama 中增加上下文长度 + +默认情况下，Ollama 使用 2048 个 token 的上下文窗口运行模型。也就是说，模型最多能 “记住” 2048 个 token 的上下文。 + +要增加上下文长度，可以在 [启动 Ollama serve](#步骤-2启动-ollama-serve) 之前设置环境变量 `IPEX_LLM_NUM_CTX`，步骤如下： + +- 打开 “命令提示符”（cmd），并通过 `cd /d PATH\TO\EXTRACTED\FOLDER` 命令进入解压后的文件夹 +- 在 “命令提示符” 中将 `IPEX_LLM_NUM_CTX` 设置为所需长度，例如：`set IPEX_LLM_NUM_CTX=16384` +- 通过运行 `start-ollama.bat` 启动 Ollama serve + +> [!Tip] +> `IPEX_LLM_NUM_CTX` 的优先级高于模型 `Modelfile` 中设置的 `num_ctx`。 + +### Ollama v0.5.4 之后支持的其他模型 + +当前的 Ollama Portable Zip 基于 Ollama v0.5.4；此外，以下新模型也已在 Ollama Portable Zip 中得到支持： + + | 模型 | 下载 | 模型链接 | + | - | - | - | + | DeepSeek-R1 | `ollama run deepseek-r1` | [deepseek-r1](https://ollama.com/library/deepseek-r1) | + | Openthinker | `ollama run openthinker` | [openthinker](https://ollama.com/library/openthinker) | + | DeepScaleR | `ollama run deepscaler` | [deepscaler](https://ollama.com/library/deepscaler) | + | Phi-4 | `ollama run phi4` | [phi4](https://ollama.com/library/phi4) | + | Dolphin 3.0 | `ollama run dolphin3` | [dolphin3](https://ollama.com/library/dolphin3) | + | Smallthinker | `ollama run smallthinker` | [smallthinker](https://ollama.com/library/smallthinker) | + | Granite3.1-Dense | `ollama run granite3-dense` | [granite3.1-dense](https://ollama.com/library/granite3.1-dense) | + | Granite3.1-Moe-3B | `ollama run granite3-moe` | [granite3.1-moe](https://ollama.com/library/granite3.1-moe) | diff --git a/docs/mddocs/Quickstart/ollama_quickstart.md b/docs/mddocs/Quickstart/ollama_quickstart.md index 143d5a2f..f1d32ce7 100644 --- a/docs/mddocs/Quickstart/ollama_quickstart.md +++ b/docs/mddocs/Quickstart/ollama_quickstart.md @@ -28,9 +28,9 @@ See the demo of running LLaMA2-7B on Intel Arc GPU below. > [!NOTE] -> Starting from `ipex-llm[cpp]==2.2.0b20240912`, oneAPI dependency of `ipex-llm[cpp]` on Windows will switch from `2024.0.0` to `2024.2.1` . +> Starting from `ipex-llm[cpp]==2.2.0b20250207`, oneAPI dependency of `ipex-llm[cpp]` on Windows will switch from `2024.2.1` to `2025.0.1` . > -> For this update, it's necessary to create a new conda environment to install the latest version on Windows. If you directly upgrade to `ipex-llm[cpp]>=2.2.0b20240912` in the previous cpp conda environment, you may encounter the error `Can't find sycl7.dll`. +> For this update, it's necessary to create a new conda environment to install the latest version on Windows. If you directly upgrade to `ipex-llm[cpp]>=2.2.0b20250207` in the previous cpp conda environment, you may encounter the error `Can't find sycl8.dll`. ## Table of Contents - [Install IPEX-LLM for Ollama](./ollama_quickstart.md#1-install-ipex-llm-for-ollama) @@ -137,7 +137,6 @@ Keep the Ollama service on and open another terminal and run `./ollama pull - ### 5. Using Ollama #### Using Curl @@ -206,6 +205,7 @@ An example process of interacting with model with `ollama run example` looks lik + ### Troubleshooting #### 1. Unable to run the initialization script If you are unable to run `init-ollama.bat`, please make sure you have installed `ipex-llm[cpp]` in your conda environment. If you have installed it, please check if you have activated the correct conda environment. Also, if you are using Windows, please make sure you have run the script with administrator privilege in prompt terminal. diff --git a/docs/mddocs/Quickstart/ollama_quickstart.zh-CN.md b/docs/mddocs/Quickstart/ollama_quickstart.zh-CN.md index 34b158c5..dd579ec7 100644 --- a/docs/mddocs/Quickstart/ollama_quickstart.zh-CN.md +++ b/docs/mddocs/Quickstart/ollama_quickstart.zh-CN.md @@ -12,7 +12,7 @@ > 如果是在 Intel Arc B 系列 GPU 上安装(例如 **B580**)，请参阅本[指南](./bmg_quickstart.md)。 > [!NOTE] -> `ipex-llm[cpp]` 的最新版本与官方 ollama 的 [v0.5.1](https://github.com/ollama/ollama/releases/tag/v0.5.4) 版本保持一致。 +> `ipex-llm[cpp]` 的最新版本与官方 ollama 的 [v0.5.4](https://github.com/ollama/ollama/releases/tag/v0.5.4) 版本保持一致。 > > `ipex-llm[cpp]==2.2.0b20250123` 与官方 ollama 的 [v0.5.1](https://github.com/ollama/ollama/releases/tag/v0.5.1) 版本保持一致。 @@ -28,9 +28,9 @@ > [!NOTE] -> 从 `ipex-llm[cpp]==2.2.0b20240912` 版本开始，Windows 上 `ipex-llm[cpp]` 依赖的 oneAPI 版本已从 `2024.0.0` 更新到 `2024.2.1`。 +> 从 `ipex-llm[cpp]==2.2.0b20250207` 版本开始，Windows 上 `ipex-llm[cpp]` 依赖的 oneAPI 版本已从 `2024.2.1` 更新到 `2025.0.1`。 > -> 如果要将 `ipex-llm[cpp]` 升级到 `2.2.0b20240912` 或更高版本，在Windows环境下，你需要新建一个干净的 conda 环境来安装新版本。如果直接在旧的 conda 环境中卸载旧版本并升级，可能会遇到 `找不到 sycl7.dll` 的错误。 +> 如果要将 `ipex-llm[cpp]` 升级到 `2.2.0b20250207` 或更高版本，在Windows环境下，你需要新建一个干净的 conda 环境来安装新版本。如果直接在旧的 conda 环境中卸载旧版本并升级，可能会遇到 `找不到 sycl8.dll` 的错误。 ## 目录 - [安装 IPEX-LLM 来使用 Ollama](./ollama_quickstart.zh-CN.md#1-安装-ipex-llm-来使用-Ollama)