ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Replace runner doc in ollama quickstart (#12575)

Browse source
This commit is contained in:
SONG Ge 2024-12-18 19:05:28 +08:00 committed by GitHub
parent f7a2bd21cf
commit 28e81fda8e
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 2 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
docs/mddocs/Quickstart/ollama_quickstart.md
View file

@ -217,7 +217,7 @@ When launching `ollama serve` for the first time on Windows, it may get stuck du
#### 5. How to distinguish the community version of Ollama from the ipex-llm version of Ollama
In the server log of community version of Ollama, you may see `source=payload_common.go:139 msg="Dynamic LLM libraries [rocm_v60000 cpu_avx2 cuda_v11 cpu cpu_avx]"`.
But in the server log of ipex-llm version of Ollama, you should only see `source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2]"`.
But in the server log of ipex-llm version of Ollama, you should only see `source=common.go:49 msg="Dynamic LLM libraries" runners=[ipex_llm]`.
#### 6. Ollama hang when multiple different questions is asked or context is long
If you find ollama hang when multiple different questions is asked or context is long, and you see `update_slots : failed to free spaces in the KV cache` in the server log, this could be because that sometimes the LLM context is larger than the default `n_ctx` value, you may increase the `n_ctx` and try it again.

2
docs/mddocs/Quickstart/ollama_quickstart.zh-CN.md
View file

@ -212,7 +212,7 @@ Ollama 默认每 5 分钟从 GPU 内存卸载一次模型。针对 ollama 的最
在 Windows 中首次启动 `ollama serve` 时，可能会在模型加载阶段卡住。如果你在首次运行时发现程序长时间挂起，可以手动在服务器端输入空格或其他字符以确保程序正在运行。
#### 5. 如何区分社区版 Ollama 和 IPEX-LLM 版 Ollama
在社区版 Ollama 的服务器日志中，你可能会看到 `source=payload_common.go:139 msg="Dynamic LLM libraries [rocm_v60000 cpu_avx2 cuda_v11 cpu cpu_avx]"`。而在 IPEX-LLM 版 Ollama 的服务器日志中，你应该仅看到 `source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2]"`。
在社区版 Ollama 的服务器日志中，你可能会看到 `source=payload_common.go:139 msg="Dynamic LLM libraries [rocm_v60000 cpu_avx2 cuda_v11 cpu cpu_avx]"`。而在 IPEX-LLM 版 Ollama 的服务器日志中，你应该仅看到 `source=common.go:49 msg="Dynamic LLM libraries" runners=[ipex_llm]`。
#### 6. 当询问多个不同的问题或上下文很长时，Ollama 会挂起
如果你在询问多个不同问题或上下文很长时，发现 ollama 挂起，并且在服务器日志中看到 `update_slots : failed to free spaces in the KV cache`，这可能是因为 LLM 上下文大于默认 `n_ctx` 值导致的，你可以尝试增加 `n_ctx` 值后重试。