add README.md
This commit is contained in:
parent
ca0e86062e
commit
8ca46d004f
1 changed files with 34 additions and 0 deletions
34
docker/llm/inference/cpu/docker/README.md
Normal file
34
docker/llm/inference/cpu/docker/README.md
Normal file
|
|
@ -0,0 +1,34 @@
|
|||
## Build/Use BigDL-LLM cpu image
|
||||
|
||||
### Build Image
|
||||
```bash
|
||||
docker build \
|
||||
--build-arg http_proxy=.. \
|
||||
--build-arg https_proxy=.. \
|
||||
--build-arg no_proxy=.. \
|
||||
--rm --no-cache -t intelanalytics/bigdl-llm-cpu:2.4.0-SNAPSHOT .
|
||||
```
|
||||
|
||||
|
||||
### Use the image for doing cpu inference
|
||||
|
||||
|
||||
An example could be:
|
||||
```bash
|
||||
#/bin/bash
|
||||
export DOCKER_IMAGE=intelanalytics/bigdl-llm-cpu:2.4.0-SNAPSHOT
|
||||
|
||||
sudo docker run -itd \
|
||||
--net=host \
|
||||
--cpuset-cpus="0-47" \
|
||||
--cpuset-mems="0" \
|
||||
--memory="32G" \
|
||||
--name=CONTAINER_NAME \
|
||||
--shm-size="16g" \
|
||||
$DOCKER_IMAGE
|
||||
```
|
||||
|
||||
|
||||
After the container is booted, you could get into the container through `docker exec`.
|
||||
|
||||
|
||||
Loading…
Reference in a new issue