ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
ipex-llm/python/llm/example/GPU/HF-Transformers-AutoModels
History
Jin Qiao 10ee786920
Replace with IPEX-LLM in example comments (#10671) 
* Replace with IPEX-LLM in example comments

* More replacement

* revert some changes
2024-04-07 13:29:51 +08:00
..
Advanced-Quantizations Replace with IPEX-LLM in example comments (#10671) 2024-04-07 13:29:51 +08:00
Model Replace with IPEX-LLM in example comments (#10671) 2024-04-07 13:29:51 +08:00
More-Data-Types Update pip install to use --extra-index-url for ipex package (#10557) 2024-03-28 09:56:23 +08:00
Save-Load Replace ipex with ipex-llm (#10554) 2024-03-28 13:54:40 +08:00
README.md Update readme (#10665) 2024-04-05 18:01:57 +08:00

README.md

Running HuggingFace transformers model using IPEX-LLM on Intel GPU

This folder contains examples of running any HuggingFace transformers model on IPEX-LLM (using the standard AutoModel APIs):

  • Model: examples of running HuggingFace transformers models (LLaMA, Mistral, ChatGLM, Qwen, Baichuan, Mixtral, Gemma, etc.) using INT4 optimizations
  • More-Data-Types: examples of applying other low bit optimizations (FP8/INT8/FP4, etc.)
  • Save-Load: examples of saving and loading low-bit models
  • Advanced-Quantizations: examples of loading GGUF/AWQ/GPTQ models