ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Add CPU and GPU Frequency Locking Instructions to Documentation (#12947)

Browse source
This commit is contained in:
Shaojun Liu 2025-03-07 09:20:40 +08:00 committed by GitHub
parent cb3c4b26ad
commit 015a4c8c43
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 60 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

20
docker/llm/serving/xpu/docker/README.md
View file

@ -179,6 +179,26 @@ The following example files are available in `/llm/` within the container:
## 4. Benchmarking
> [!TIP]
> Before running benchmarks, it's recommended to lock CPU and GPU frequencies to ensure more stable, reliable, and better performance data.
>
> **Lock CPU Frequency:**
> Use the following command to set the minimum CPU frequency (adjust based on your CPU model):
>
> ```bash
> sudo cpupower frequency-set -d 3.8GHz
> ```
>
> **Lock GPU Frequencies:**
> Use these commands to lock GPU frequencies to 2400MHz:
>
> ```bash
> sudo xpu-smi config -d 0 -t 0 --frequencyrange 2400,2400
> sudo xpu-smi config -d 1 -t 0 --frequencyrange 2400,2400
> sudo xpu-smi config -d 2 -t 0 --frequencyrange 2400,2400
> sudo xpu-smi config -d 3 -t 0 --frequencyrange 2400,2400
> ```
### 4.1 Online Benchmark through API Server
To benchmark the API server and estimate TPS (transactions per second), follow these steps:

20
docs/mddocs/DockerGuides/vllm_docker_quickstart.md
View file

@ -59,6 +59,26 @@ root@arda-arc12:/# sycl-ls
## Running vLLM serving with IPEX-LLM on Intel GPU in Docker
> [!TIP]
> Before running benchmarks, it's recommended to lock CPU and GPU frequencies to ensure more stable, reliable, and better performance data.
>
> **Lock CPU Frequency:**
> Use the following command to set the minimum CPU frequency (adjust based on your CPU model):
>
> ```bash
> sudo cpupower frequency-set -d 3.8GHz
> ```
>
> **Lock GPU Frequencies:**
> Use these commands to lock GPU frequencies to 2400MHz:
>
> ```bash
> sudo xpu-smi config -d 0 -t 0 --frequencyrange 2400,2400
> sudo xpu-smi config -d 1 -t 0 --frequencyrange 2400,2400
> sudo xpu-smi config -d 2 -t 0 --frequencyrange 2400,2400
> sudo xpu-smi config -d 3 -t 0 --frequencyrange 2400,2400
> ```
We have included multiple vLLM-related files in `/llm/`:
1. `vllm_offline_inference.py`: Used for vLLM offline inference example,

20
docs/mddocs/Quickstart/vLLM_quickstart.md
View file

@ -21,6 +21,26 @@ Currently, IPEX-LLM integrated vLLM only supports the following models:
## Quick Start
> [!TIP]
> Before running benchmarks, it's recommended to lock CPU and GPU frequencies to ensure more stable, reliable, and better performance data.
>
> **Lock CPU Frequency:**
> Use the following command to set the minimum CPU frequency (adjust based on your CPU model):
>
> ```bash
> sudo cpupower frequency-set -d 3.8GHz
> ```
>
> **Lock GPU Frequencies:**
> Use these commands to lock GPU frequencies to 2400MHz:
>
> ```bash
> sudo xpu-smi config -d 0 -t 0 --frequencyrange 2400,2400
> sudo xpu-smi config -d 1 -t 0 --frequencyrange 2400,2400
> sudo xpu-smi config -d 2 -t 0 --frequencyrange 2400,2400
> sudo xpu-smi config -d 3 -t 0 --frequencyrange 2400,2400
> ```
This quickstart guide walks you through installing and running `vLLM` with `ipex-llm`.
### 1. Install IPEX-LLM for vLLM