ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

update troubleshooting (#11960)

Browse source
This commit is contained in:
Yina Chen 2024-08-29 12:44:22 +03:00 committed by GitHub
parent 882f4a5ff7
commit 5f7ff76ea5
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 4 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
python/llm/example/NPU/HF-Transformers-AutoModels/LLM/README.md
View file

@ -126,6 +126,7 @@ Arguments info:
### Troubleshooting ### Troubleshooting
#### Output Problem
If you encounter output problem, please try to disable the optimization of transposing value cache with following command: If you encounter output problem, please try to disable the optimization of transposing value cache with following command:
```bash ```bash
# to run Llama-2-7b-chat-hf # to run Llama-2-7b-chat-hf
@ -144,6 +145,9 @@ python minicpm.py --disable-transpose-value-cache
python minicpm.py --repo-id-or-model-path openbmb/MiniCPM-2B-sft-bf16 --disable-transpose-value-cache python minicpm.py --repo-id-or-model-path openbmb/MiniCPM-2B-sft-bf16 --disable-transpose-value-cache
``` ```
#### High CPU Utilization
You can reduce CPU utilization by setting the environment variable with `set IPEX_LLM_CPU_LM_HEAD=0`.
### Sample Output ### Sample Output
#### [meta-llama/Llama-2-7b-chat-hf](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf) #### [meta-llama/Llama-2-7b-chat-hf](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf)