ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Update install_linux_gpu.md (#12353)

Browse source
This commit is contained in:
Xin Qiu 2024-11-07 16:08:01 +08:00 committed by GitHub
parent ad68c56573
commit 520af4e9b5
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 4 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
docs/mddocs/Quickstart/install_linux_gpu.md
View file

@ -166,6 +166,8 @@ whose output should contain `Intel(R) Arc(TM) Graphics` or `Intel(R) Graphics` b
linux-headers-$(uname -r) \
libc6-dev
sudo apt install intel-i915-dkms intel-fw-gpu
# Notice: if you are using Iris Graphics(integrated in 10-13 Gen Intel laptop CPU), please use below version of intel-i915-dkms instead.
# sudo apt install intel-i915-dkms=1.24.2.17.240301.20+i29-1 intel-fw-gpu=2024.17.5-329~22.04
# Install Compute Runtime
sudo apt-get install -y udev \
@ -229,6 +231,8 @@ whose output should contain `Intel(R) Arc(TM) Graphics` or `Intel(R) Graphics` b
linux-headers-$(uname -r) \
libc6-dev
sudo apt install -y intel-i915-dkms intel-fw-gpu
# Notice: if you are using Iris Graphics(integrated in 10-13 Gen Intel laptop CPU), please use below version of intel-i915-dkms instead.
# sudo apt install intel-i915-dkms=1.24.2.17.240301.20+i29-1 intel-fw-gpu=2024.17.5-329~22.04
# Install Compute Runtime
sudo apt-get install -y udev \
@ -487,4 +491,3 @@ Answer: AI stands for Artificial Intelligence, which is the simulation of human
### Warmup for optimial performance on first run
When running LLMs on GPU for the first time, you might notice the performance is lower than expected, with delays up to several minutes before the first token is generated. This delay occurs because the GPU kernels require compilation and initialization, which varies across different GPU types. To achieve optimal and consistent performance, we recommend a one-time warm-up by running `model.generate(...)` an additional time before starting your actual generation tasks. If you're developing an application, you can incorporate this warmup step into start-up or loading routine to enhance the user experience.