ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Update README.md (#10838)

Browse source
This commit is contained in:
Guancheng Fu 2024-04-22 14:18:51 +08:00 committed by GitHub
parent 3daad242b8
commit 3b82834aaf
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 3 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
docker/llm/README.md
View file

@ -152,14 +152,14 @@ Additionally, for examples related to Inference with Speculative Decoding, you c
## IPEX-LLM Inference on XPU ## IPEX-LLM Inference on XPU
### 1. Prepare ipex-llm-cpu Docker Image ### 1. Prepare ipex-llm-xpu Docker Image
Run the following command to pull image from dockerhub: Run the following command to pull image from dockerhub:
```bash ```bash
docker pull intelanalytics/ipex-llm-xpu:2.1.0-SNAPSHOT docker pull intelanalytics/ipex-llm-xpu:2.1.0-SNAPSHOT
``` ```
### 2. Start bigdl-llm-cpu Docker Container ### 2. Start ipex-llm-xpu Docker Container
To map the xpu into the container, you need to specify --device=/dev/dri when booting the container. To map the xpu into the container, you need to specify --device=/dev/dri when booting the container.
```bash ```bash
@ -650,4 +650,4 @@ Using the [export_merged_model.py](../../python/llm/example/GPU/LLM-Finetuning/Q
python ./export_merged_model.py --repo-id-or-model-path REPO_ID_OR_MODEL_PATH --adapter_path ./outputs/checkpoint-200 --output_path ./outputs/checkpoint-200-merged python ./export_merged_model.py --repo-id-or-model-path REPO_ID_OR_MODEL_PATH --adapter_path ./outputs/checkpoint-200 --output_path ./outputs/checkpoint-200-merged
``` ```
Then you can use `./outputs/checkpoint-200-merged` as a normal huggingface transformer model to do inference. Then you can use `./outputs/checkpoint-200-merged` as a normal huggingface transformer model to do inference.