ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Update fastchat demo script (#12367)

Browse source 
* Update README.md

* Update vllm_docker_quickstart.md
This commit is contained in:
Shaojun Liu 2024-11-08 15:42:17 +08:00 committed by GitHub
parent 51f7f87768
commit fad15c8ca0
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 6 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
docker/llm/serving/xpu/docker/README.md
View file

@ -102,7 +102,9 @@ To set up model serving using `IPEX-LLM` as backend using FastChat, you can refe
# start controller # start controller
python -m fastchat.serve.controller & python -m fastchat.serve.controller &
export USE_XETLA=OFF
export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=2
export TORCH_LLM_ALLREDUCE=0 export TORCH_LLM_ALLREDUCE=0
export CCL_DG2_ALLREDUCE=1 export CCL_DG2_ALLREDUCE=1

4
docs/mddocs/DockerGuides/vllm_docker_quickstart.md
View file

@ -831,7 +831,9 @@ We can set up model serving using `IPEX-LLM` as backend using FastChat, the foll
# start controller # start controller
python -m fastchat.serve.controller & python -m fastchat.serve.controller &
export USE_XETLA=OFF
export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=2
export TORCH_LLM_ALLREDUCE=0 export TORCH_LLM_ALLREDUCE=0
export CCL_DG2_ALLREDUCE=1 export CCL_DG2_ALLREDUCE=1