Update B580 doc (#12691)
This commit is contained in:
parent
66d4385cc9
commit
f9b29a4f56
3 changed files with 25 additions and 22 deletions
|
|
@ -11,11 +11,13 @@ This guide demonstrates how to install and use IPEX-LLM on the Intel Arc B-Serie
|
|||
|
||||
1. [Linux](#1-linux)
|
||||
1.1 [Install Prerequisites](#11-install-prerequisites)
|
||||
1.2 [Install IPEX-LLM](#12-install-ipex-llm)
|
||||
2. [Windows](#2-windows)
|
||||
1.2 [Install IPEX-LLM](#for-pytorch-and-huggingface) (for PyTorch and HuggingFace)
|
||||
1.3 [Install IPEX-LLM](#for-llamacpp-and-ollama) (for llama.cpp and Ollama)
|
||||
3. [Windows](#2-windows)
|
||||
2.1 [Install Prerequisites](#21-install-prerequisites)
|
||||
2.2 [Install IPEX-LLM](#22-install-ipex-llm)
|
||||
3. [Use Cases](#3-use-cases)
|
||||
2.2 [Install IPEX-LLM](#for-pytorch-and-huggingface-1) (for PyTorch and HuggingFace)
|
||||
2.3 [Install IPEX-LLM](#for-llamacpp-and-ollama-1) (for llama.cpp and Ollama)
|
||||
5. [Use Cases](#3-use-cases)
|
||||
3.1 [PyTorch](#31-pytorch)
|
||||
3.2 [Ollama](#32-ollama)
|
||||
3.3 [llama.cpp](#33-llamacpp)
|
||||
|
|
@ -59,7 +61,7 @@ conda activate llm
|
|||
|
||||
With the `llm` environment active, install the appropriate `ipex-llm` package based on your use case:
|
||||
|
||||
#### For PyTorch:
|
||||
#### For PyTorch and HuggingFace:
|
||||
Install the `ipex-llm[xpu-arc]` package. Choose either the US or CN website for `extra-index-url`:
|
||||
|
||||
- For **US**:
|
||||
|
|
@ -109,7 +111,7 @@ conda activate llm
|
|||
|
||||
With the `llm` environment active, install the appropriate `ipex-llm` package based on your use case:
|
||||
|
||||
#### For PyTorch:
|
||||
#### For PyTorch and HuggingFace:
|
||||
Install the `ipex-llm[xpu-arc]` package. Choose either the US or CN website for `extra-index-url`:
|
||||
|
||||
- For **US**:
|
||||
|
|
|
|||
|
|
@ -5,6 +5,14 @@
|
|||
|
||||
[ggerganov/llama.cpp](https://github.com/ggerganov/llama.cpp) prvoides fast LLM inference in pure C++ across a variety of hardware; you can now use the C++ interface of [`ipex-llm`](https://github.com/intel-analytics/ipex-llm) as an accelerated backend for `llama.cpp` running on Intel **GPU** *(e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Max)*.
|
||||
|
||||
> [!NOTE]
|
||||
> For installation on Intel Arc B-Series GPU (such as **B580**), please refer to this [guide](./bmg_quickstart.md).
|
||||
|
||||
> [!NOTE]
|
||||
> Our latest version is consistent with [3f1ae2e](https://github.com/ggerganov/llama.cpp/commit/3f1ae2e32cde00c39b96be6d01c2997c29bae555) of llama.cpp.
|
||||
>
|
||||
> `ipex-llm[cpp]==2.2.0b20241204` is consistent with [a1631e5](https://github.com/ggerganov/llama.cpp/commit/a1631e53f6763e17da522ba219b030d8932900bd) of llama.cpp.
|
||||
|
||||
See the demo of running LLaMA2-7B on Intel Arc GPU below.
|
||||
|
||||
<table width="100%">
|
||||
|
|
@ -16,16 +24,6 @@ See the demo of running LLaMA2-7B on Intel Arc GPU below.
|
|||
</tr>
|
||||
</table>
|
||||
|
||||
> [!NOTE]
|
||||
> `ipex-llm[cpp]==2.2.0b20241204` is consistent with [a1631e5](https://github.com/ggerganov/llama.cpp/commit/a1631e53f6763e17da522ba219b030d8932900bd) of llama.cpp.
|
||||
>
|
||||
> Our latest version is consistent with [3f1ae2e](https://github.com/ggerganov/llama.cpp/commit/3f1ae2e32cde00c39b96be6d01c2997c29bae555) of llama.cpp.
|
||||
|
||||
> [!NOTE]
|
||||
> Starting from `ipex-llm[cpp]==2.2.0b20240912`, oneAPI dependency of `ipex-llm[cpp]` on Windows will switch from `2024.0.0` to `2024.2.1` .
|
||||
>
|
||||
> For this update, it's necessary to create a new conda environment to install the latest version on Windows. If you directly upgrade to `ipex-llm[cpp]>=2.2.0b20240912` in the previous cpp conda environment, you may encounter the error `Can't find sycl7.dll`.
|
||||
|
||||
## Table of Contents
|
||||
- [Prerequisites](./llama_cpp_quickstart.md#0-prerequisites)
|
||||
- [Install IPEX-LLM for llama.cpp](./llama_cpp_quickstart.md#1-install-ipex-llm-for-llamacpp)
|
||||
|
|
@ -368,4 +366,4 @@ On latest version of `ipex-llm`, you might come across `native API failed` error
|
|||
If you meet this error, please check your Linux kernel version first. You may encounter this issue on higher kernel versions (like kernel 6.15). You can also refer to [this issue](https://github.com/intel-analytics/ipex-llm/issues/10955) to see if it helps.
|
||||
|
||||
#### 16. `backend buffer base cannot be NULL` error
|
||||
If you meet `ggml-backend.c:96: GGML_ASSERT(base != NULL && "backend buffer base cannot be NULL") failed`, simply adding `-c xx` parameter during inference, for example `-c 1024` would resolve this problem.
|
||||
If you meet `ggml-backend.c:96: GGML_ASSERT(base != NULL && "backend buffer base cannot be NULL") failed`, simply adding `-c xx` parameter during inference, for example `-c 1024` would resolve this problem.
|
||||
|
|
|
|||
|
|
@ -5,6 +5,14 @@
|
|||
|
||||
[ollama/ollama](https://github.com/ollama/ollama) is popular framework designed to build and run language models on a local machine; you can now use the C++ interface of [`ipex-llm`](https://github.com/intel-analytics/ipex-llm) as an accelerated backend for `ollama` running on Intel **GPU** *(e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Max)*.
|
||||
|
||||
> [!NOTE]
|
||||
> For installation on Intel Arc B-Series GPU (such as **B580**), please refer to this [guide](./bmg_quickstart.md).
|
||||
|
||||
> [!NOTE]
|
||||
> Our current version is consistent with [v0.4.6](https://github.com/ollama/ollama/releases/tag/v0.4.6) of ollama.
|
||||
>
|
||||
> `ipex-llm[cpp]==2.2.0b20241204` is consistent with [v0.3.6](https://github.com/ollama/ollama/releases/tag/v0.3.6) of ollama.
|
||||
|
||||
See the demo of running LLaMA2-7B on Intel Arc GPU below.
|
||||
|
||||
<table width="100%">
|
||||
|
|
@ -16,11 +24,6 @@ See the demo of running LLaMA2-7B on Intel Arc GPU below.
|
|||
</tr>
|
||||
</table>
|
||||
|
||||
> [!NOTE]
|
||||
> `ipex-llm[cpp]==2.2.0b20241204` is consistent with [v0.3.6](https://github.com/ollama/ollama/releases/tag/v0.3.6) of ollama.
|
||||
>
|
||||
> Our current version is consistent with [v0.4.6](https://github.com/ollama/ollama/releases/tag/v0.4.6) of ollama.
|
||||
|
||||
> [!NOTE]
|
||||
> Starting from `ipex-llm[cpp]==2.2.0b20240912`, oneAPI dependency of `ipex-llm[cpp]` on Windows will switch from `2024.0.0` to `2024.2.1` .
|
||||
>
|
||||
|
|
|
|||
Loading…
Reference in a new issue