add README.md for bigdl-llm-cpu image (#9026)
* modify Dockerfile * add README.md * add README.md
This commit is contained in:
parent
b943d73844
commit
9126abdf9b
2 changed files with 36 additions and 1 deletions
|
|
@ -13,6 +13,7 @@ RUN env DEBIAN_FRONTEND=noninteractive apt-get update && \
|
||||||
ln -s /usr/bin/python3 /usr/bin/python && \
|
ln -s /usr/bin/python3 /usr/bin/python && \
|
||||||
apt-get install -y python3-pip python3.9-dev python3-wheel python3.9-distutils && \
|
apt-get install -y python3-pip python3.9-dev python3-wheel python3.9-distutils && \
|
||||||
pip3 install --no-cache --upgrade requests argparse urllib3 && \
|
pip3 install --no-cache --upgrade requests argparse urllib3 && \
|
||||||
pip3 install --pre --upgrade bigdl-llm[all]
|
pip3 install --pre --upgrade bigdl-llm[all] && \
|
||||||
|
pip3 install --pre --upgrade bigdl-nano
|
||||||
|
|
||||||
ENTRYPOINT ["/bin/bash"]
|
ENTRYPOINT ["/bin/bash"]
|
||||||
34
docker/llm/inference/cpu/docker/README.md
Normal file
34
docker/llm/inference/cpu/docker/README.md
Normal file
|
|
@ -0,0 +1,34 @@
|
||||||
|
## Build/Use BigDL-LLM cpu image
|
||||||
|
|
||||||
|
### Build Image
|
||||||
|
```bash
|
||||||
|
docker build \
|
||||||
|
--build-arg http_proxy=.. \
|
||||||
|
--build-arg https_proxy=.. \
|
||||||
|
--build-arg no_proxy=.. \
|
||||||
|
--rm --no-cache -t intelanalytics/bigdl-llm-cpu:2.4.0-SNAPSHOT .
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
### Use the image for doing cpu inference
|
||||||
|
|
||||||
|
|
||||||
|
An example could be:
|
||||||
|
```bash
|
||||||
|
#/bin/bash
|
||||||
|
export DOCKER_IMAGE=intelanalytics/bigdl-llm-cpu:2.4.0-SNAPSHOT
|
||||||
|
|
||||||
|
sudo docker run -itd \
|
||||||
|
--net=host \
|
||||||
|
--cpuset-cpus="0-47" \
|
||||||
|
--cpuset-mems="0" \
|
||||||
|
--memory="32G" \
|
||||||
|
--name=CONTAINER_NAME \
|
||||||
|
--shm-size="16g" \
|
||||||
|
$DOCKER_IMAGE
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
After the container is booted, you could get into the container through `docker exec`.
|
||||||
|
|
||||||
|
To run inference using `BigDL-LLM` using cpu, you could refer to this [documentation](https://github.com/intel-analytics/BigDL/tree/main/python/llm#cpu-int4).
|
||||||
Loading…
Reference in a new issue