ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Update README.md (#10833)

Browse source
This commit is contained in:
Guancheng Fu 2024-04-22 11:07:10 +08:00 committed by GitHub
parent 5f95054f97
commit ae3b577537
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 3 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
python/llm/example/GPU/vLLM-Serving/README.md
View file

@ -136,6 +136,8 @@ Currently, for vLLM-v2, we support the following models:
Install the dependencies for vLLM-v2 as follows: Install the dependencies for vLLM-v2 as follows:
```bash ```bash
# This directory may change depends on where you install oneAPI-basekit
source /opt/intel/oneapi/setvars.sh
# First create an conda environment # First create an conda environment
conda create -n ipex-vllm python=3.11 conda create -n ipex-vllm python=3.11
conda activate ipex-vllm conda activate ipex-vllm
@ -200,4 +202,4 @@ Then you can access the api server as follows:
"max_tokens": 128, "max_tokens": 128,
"temperature": 0 "temperature": 0
}' & }' &
``` ```