ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Remove env variable BIGDL_LLM_XMX_DISABLED in documentation (#12445)

Browse source 
* fix: remove BIGDL_LLM_XMX_DISABLED in mddocs

* fix: remove set SYCL_CACHE_PERSISTENT=1 in example

* fix: remove BIGDL_LLM_XMX_DISABLED in workflows

* fix: merge igpu and A-series Graphics

* fix: remove set BIGDL_LLM_XMX_DISABLED=1 in example

* fix: remove BIGDL_LLM_XMX_DISABLED in workflows

* fix: merge igpu and A-series Graphics

* fix: textual adjustment

* fix: textual adjustment

* fix: textual adjustment
This commit is contained in:
Chu,Youcheng 2024-11-27 11:16:36 +08:00 committed by GitHub
parent f8c2bb2943
commit acd77d9e87
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
122 changed files with 232 additions and 1556 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

34
.github/workflows/llm_performance_tests.yml vendored
View file

@ -805,7 +805,6 @@ jobs:
call conda activate igpu-perf
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -841,7 +840,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -875,7 +873,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -910,7 +907,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -947,7 +943,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1003,7 +998,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1039,7 +1033,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1073,7 +1066,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1108,7 +1100,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1145,7 +1136,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1200,7 +1190,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1236,7 +1225,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1270,7 +1258,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1305,7 +1292,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1342,7 +1328,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1397,7 +1382,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1435,7 +1419,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1474,7 +1457,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1522,7 +1504,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1572,7 +1553,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1642,7 +1622,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1680,7 +1659,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1717,7 +1695,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1756,7 +1733,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1814,7 +1790,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1852,7 +1827,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1888,7 +1862,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1925,7 +1898,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -1964,7 +1936,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -2021,7 +1992,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -2059,7 +2029,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -2095,7 +2064,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -2132,7 +2100,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1
@ -2171,7 +2138,6 @@ jobs:
if "${{ matrix.platform }}"=="perf-mtl" (
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
)
if "${{ matrix.platform }}"=="perf-lnl" (
set SYCL_CACHE_PERSISTENT=1

4
docs/mddocs/DockerGuides/docker_pytorch_inference_gpu.md
View file

@ -97,10 +97,6 @@ root@arda-arc12:/# sycl-ls
> # Reduce memory accesses by fusing SDP ops.
> # Recommended for use on Intel Data Center GPU Max Series.
> export ENABLE_SDP_FUSION=1
>
> # Disable XMX computation.
> # Recommended for use on integrated GPUs.
> export BIGDL_LLM_XMX_DISABLED=1
> ```

9
docs/mddocs/Overview/install_gpu.md
View file

@ -150,13 +150,7 @@ call "C:\Program Files (x86)\Intel\oneAPI\setvars.bat"
Please also set the following environment variable if you would like to run LLMs on: -->
- For **Intel iGPU**:
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
- For **Intel Arc™ A-Series Graphics**:
- For **Intel iGPU** and **Intel Arc™ A-Series Graphics**:
```cmd
set SYCL_CACHE_PERSISTENT=1
```
@ -596,7 +590,6 @@ To use GPU acceleration on Linux, several environment variables are required or
source /opt/intel/oneapi/setvars.sh
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
> [!NOTE]

19
docs/mddocs/Quickstart/benchmark_quickstart.md
View file

@ -80,34 +80,19 @@ Some parameters in the yaml file that you can configure:
Please refer to [here](../Overview/install_gpu.md#runtime-configuration) to configure oneAPI environment variables. Choose corresponding commands base on your device.
- For **Intel iGPU**:
- For **Intel iGPU** and **Intel Arc™ A-Series Graphics**:
```bash
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
python run.py
```
- For **Intel Arc™ A300-Series or Pro A60**:
```bash
set SYCL_CACHE_PERSISTENT=1
python run.py
```
- For **Other Intel dGPU Series**:
```bash
# e.g. Arc™ A770
python run.py
```
## Run on Linux
Please choose corresponding commands base on your device.
- For **Intel Arc™ A-Series and Intel Data Center GPU Flex**:
- For **Intel Arc™ A-Series** and **Intel Data Center GPU Flex**:
For Intel Arc™ A-Series Graphics and Intel Data Center GPU Flex Series, we recommend:

1
docs/mddocs/Quickstart/install_linux_gpu.md
View file

@ -422,7 +422,6 @@ To use GPU acceleration on Linux, several environment variables are required or
source /opt/intel/oneapi/setvars.sh
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
> [!NOTE]

1
docs/mddocs/Quickstart/install_linux_gpu.zh-CN.md
View file

@ -390,7 +390,6 @@ conda activate llm
source /opt/intel/oneapi/setvars.sh
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
> [!NOTE]

11
docs/mddocs/Quickstart/install_windows_gpu.md
View file

@ -109,19 +109,12 @@ You can verify if `ipex-llm` is successfully installed following below steps.
- Set the following environment variables according to your device:
- For **Intel iGPU**:
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
- For **Intel Arc™ A770**:
- For **Intel iGPU** and **Intel Arc™ A770**:
```cmd
set SYCL_CACHE_PERSISTENT=1
```
> [!TIP]
> For other Intel dGPU Series, please refer to [this guide](../Overview/install_gpu.md#runtime-configuration) for more details regarding runtime configuration.

9
docs/mddocs/Quickstart/install_windows_gpu.zh-CN.md
View file

@ -108,14 +108,7 @@ conda activate llm
- 根据你的设备，设置以下环境参数：
- **Intel iGPU**:
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
- **Intel Arc™ A770**:
- **Intel iGPU** and **Intel Arc™ A770**:
```cmd
set SYCL_CACHE_PERSISTENT=1

5
docs/mddocs/Quickstart/webui_quickstart.md
View file

@ -70,11 +70,6 @@ Configure oneAPI variables by running the following command in **Miniforge Promp
set SYCL_CACHE_PERSISTENT=1
```
If you're running on iGPU, set additional environment variables by running the following commands:
```cmd
set BIGDL_LLM_XMX_DISABLED=1
```
#### Launch the Server
In **Miniforge Prompt** with the conda environment `llm` activated, navigate to the `text-generation-webui` folder and execute the following commands (You can optionally lanch the server with or without the API service):

15
python/llm/example/GPU/HuggingFace/LLM/aquila/README.md
View file

@ -77,7 +77,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -85,18 +84,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -104,6 +92,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/aquila2/README.md
View file

@ -77,7 +77,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -85,18 +84,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -104,6 +92,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/baichuan/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/baichuan2/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/bluelm/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/chatglm2/README.md
View file

@ -70,7 +70,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -78,18 +77,7 @@ export BIGDL_LLM_XMX_DISABLED=1
### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -97,6 +85,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.

15
python/llm/example/GPU/HuggingFace/LLM/chatglm3/README.md
View file

@ -70,7 +70,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -78,18 +77,7 @@ export BIGDL_LLM_XMX_DISABLED=1
### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -97,6 +85,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
## 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/chinese-llama2/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/codegeex2/README.md
View file

@ -84,7 +84,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -92,18 +91,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -111,6 +99,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.

15
python/llm/example/GPU/HuggingFace/LLM/codegemma/README.md
View file

@ -81,7 +81,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -89,18 +88,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -108,6 +96,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/codellama/readme.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/deciLM-7b/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.

15
python/llm/example/GPU/HuggingFace/LLM/deepseek/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.

15
python/llm/example/GPU/HuggingFace/LLM/dolly-v1/README.md
View file

@ -73,7 +73,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -81,18 +80,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -100,6 +88,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/dolly-v2/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/falcon/README.md
View file

@ -101,7 +101,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -109,18 +108,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 4.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -128,6 +116,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 5. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/flan-t5/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/gemma/README.md
View file

@ -79,7 +79,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -87,18 +86,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -106,6 +94,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/gemma2/README.md
View file

@ -81,7 +81,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -89,18 +88,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -108,6 +96,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/glm4/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
## 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/gpt-j/readme.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/internlm/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/internlm2/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.

15
python/llm/example/GPU/HuggingFace/LLM/llama2/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/llama3.1/README.md
View file

@ -79,7 +79,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -87,18 +86,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -106,6 +94,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/llama3.2/README.md
View file

@ -79,7 +79,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -87,18 +86,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -106,6 +94,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/llama3/README.md
View file

@ -77,7 +77,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -85,18 +84,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -104,6 +92,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/minicpm/README.md
View file

@ -73,7 +73,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -81,18 +80,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -100,6 +88,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/minicpm3/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/mistral/README.md
View file

@ -72,7 +72,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -80,18 +79,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -99,6 +87,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/mixtral/README.md
View file

@ -79,7 +79,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -87,18 +86,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -106,6 +94,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/mpt/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/phi-1_5/README.md
View file

@ -77,7 +77,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -85,18 +84,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -104,6 +92,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/phi-2/README.md
View file

@ -77,7 +77,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -85,18 +84,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -104,6 +92,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/phi-3/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/phixtral/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/qwen/README.md
View file

@ -77,7 +77,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -85,18 +84,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -104,6 +92,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/qwen1.5/README.md
View file

@ -79,7 +79,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -87,18 +86,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -106,6 +94,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/qwen2.5/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/qwen2/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/redpajama/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/replit/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/rwkv4/README.md
View file

@ -73,7 +73,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -81,18 +80,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -100,6 +88,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.

15
python/llm/example/GPU/HuggingFace/LLM/rwkv5/README.md
View file

@ -73,7 +73,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -81,18 +80,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -100,6 +88,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/solar/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/stablelm/README.md
View file

@ -77,7 +77,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -85,18 +84,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -104,6 +92,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/starcoder/readme.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/vicuna/README.md
View file

@ -73,7 +73,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -81,18 +80,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -100,6 +88,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/yi/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/LLM/yuan2/README.md
View file

@ -78,7 +78,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -86,18 +85,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -105,6 +93,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/Multimodal/MiniCPM-Llama3-V-2_5/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/Multimodal/MiniCPM-V-2/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/Multimodal/MiniCPM-V-2_6/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/Multimodal/MiniCPM-V/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/Multimodal/StableDiffusion/README.md
View file

@ -57,7 +57,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -65,18 +64,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -84,6 +72,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.

15
python/llm/example/GPU/HuggingFace/Multimodal/distil-whisper/README.md
View file

@ -76,7 +76,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -84,18 +83,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -103,6 +91,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/Multimodal/glm-4v/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/Multimodal/internvl2/readme.md
View file

@ -77,7 +77,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -85,18 +84,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -104,6 +92,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/Multimodal/phi-3-vision/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/Multimodal/qwen-vl/README.md
View file

@ -77,7 +77,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -85,18 +84,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -104,6 +92,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/Multimodal/qwen2-audio/README.md
View file

@ -82,7 +82,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -90,18 +89,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -109,6 +97,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/Multimodal/voiceassistant/README.md
View file

@ -86,7 +86,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -94,18 +93,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -113,6 +101,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/Multimodal/whisper/readme.md
View file

@ -78,7 +78,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -86,18 +85,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -105,6 +93,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/HuggingFace/Save-Load/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.

15
python/llm/example/GPU/LangChain/README.md
View file

@ -73,7 +73,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -81,18 +80,7 @@ export BIGDL_LLM_XMX_DISABLED=1
### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -100,6 +88,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.

15
python/llm/example/GPU/Lightweight-Serving/README.md
View file

@ -97,7 +97,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -105,18 +104,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -124,6 +112,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.

15
python/llm/example/GPU/LlamaIndex/README.md
View file

@ -130,7 +130,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -138,18 +137,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 4.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -157,6 +145,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.

15
python/llm/example/GPU/Long-Context/Chatglm3-32K/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.

15
python/llm/example/GPU/Long-Context/LLaMA2-32K/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.

15
python/llm/example/GPU/ModelScope-Models/README.md
View file

@ -74,7 +74,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -82,18 +81,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -101,6 +89,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.

15
python/llm/example/GPU/ModelScope-Models/Save-Load/README.md
View file

@ -73,7 +73,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -81,18 +80,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -100,6 +88,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.

15
python/llm/example/GPU/PyTorch-Models/Model/aquila2/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/PyTorch-Models/Model/baichuan/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/PyTorch-Models/Model/baichuan2/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/PyTorch-Models/Model/bark/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/PyTorch-Models/Model/bluelm/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/PyTorch-Models/Model/chatglm2/README.md
View file

@ -69,7 +69,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -77,18 +76,7 @@ export BIGDL_LLM_XMX_DISABLED=1
### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -96,6 +84,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.

15
python/llm/example/GPU/PyTorch-Models/Model/chatglm3/README.md
View file

@ -69,7 +69,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -77,18 +76,7 @@ export BIGDL_LLM_XMX_DISABLED=1
### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -96,6 +84,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
## 4. Running examples

15
python/llm/example/GPU/PyTorch-Models/Model/codegeex2/README.md
View file

@ -84,7 +84,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -92,18 +91,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -111,6 +99,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.

15
python/llm/example/GPU/PyTorch-Models/Model/codegemma/README.md
View file

@ -81,7 +81,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -89,18 +88,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -108,6 +96,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/PyTorch-Models/Model/codellama/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/PyTorch-Models/Model/deciLM-7b/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.

15
python/llm/example/GPU/PyTorch-Models/Model/deepseek/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/PyTorch-Models/Model/distil-whisper/README.md
View file

@ -76,7 +76,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -84,18 +83,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -103,6 +91,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/PyTorch-Models/Model/dolly-v1/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/PyTorch-Models/Model/dolly-v2/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/PyTorch-Models/Model/flan-t5/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/PyTorch-Models/Model/glm4/README.md
View file

@ -77,7 +77,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -85,18 +84,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -104,6 +92,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/PyTorch-Models/Model/internlm2/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.

15
python/llm/example/GPU/PyTorch-Models/Model/llama2/README.md
View file

@ -71,7 +71,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -79,18 +78,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -98,6 +86,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

15
python/llm/example/GPU/PyTorch-Models/Model/llama3.2-vision/README.md
View file

@ -75,7 +75,6 @@ export ENABLE_SDP_FUSION=1
```bash
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
</details>
@ -83,18 +82,7 @@ export BIGDL_LLM_XMX_DISABLED=1
#### 3.2 Configurations for Windows
<details>
<summary>For Intel iGPU</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1
```
</details>
<details>
<summary>For Intel Arc™ A-Series Graphics</summary>
<summary>For Intel iGPU and Intel Arc™ A-Series Graphics</summary>
```cmd
set SYCL_CACHE_PERSISTENT=1
@ -102,6 +90,7 @@ set SYCL_CACHE_PERSISTENT=1
</details>
> [!NOTE]
> For the first time that each model runs on Intel iGPU/Intel Arc™ A300-Series or Pro A60, it may take several minutes to compile.
### 4. Running examples

Some files were not shown because too many files have changed in this diff Show more