ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Update AWQ and GPTQ GPU example (#12300)

Browse source
This commit is contained in:
Zijie Li 2024-10-30 21:35:31 -04:00 committed by GitHub
parent 0763268e4c
commit 6f22133efc
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 0 additions and 6 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
python/llm/example/GPU/HuggingFace/Advanced-Quantizations/AWQ/README.md
View file

@ -37,15 +37,10 @@ conda create -n llm python=3.11
conda activate llm conda activate llm
# below command will install intel_extension_for_pytorch==2.1.10+xpu as default # below command will install intel_extension_for_pytorch==2.1.10+xpu as default
pip install --pre --upgrade ipex-llm[xpu] --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/ pip install --pre --upgrade ipex-llm[xpu] --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/
pip install transformers==4.35.0
pip install autoawq==0.1.8 --no-deps pip install autoawq==0.1.8 --no-deps
pip install accelerate==0.25.0 pip install accelerate==0.25.0
pip install einops pip install einops
``` ```
**Note: For Mixtral model, please use transformers 4.36.0:**
```bash
pip install transformers==4.36.0
```
### 2. Configures OneAPI environment variables ### 2. Configures OneAPI environment variables

1
python/llm/example/GPU/HuggingFace/Advanced-Quantizations/GPTQ/README.md
View file

@ -13,7 +13,6 @@ conda create -n llm python=3.11
conda activate llm conda activate llm
# below command will install intel_extension_for_pytorch==2.1.10+xpu as default # below command will install intel_extension_for_pytorch==2.1.10+xpu as default
pip install --pre --upgrade ipex-llm[xpu] --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/ pip install --pre --upgrade ipex-llm[xpu] --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/
pip install transformers==4.37.0
pip install auto_gptq==0.7.1 pip install auto_gptq==0.7.1
pip install optimum==1.14.0 pip install optimum==1.14.0
``` ```