ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
ipex-llm/python/llm/example/CPU/PyTorch-Models/Model
History
Yining Wang 4a19f50d16 phi-1_5 CPU and GPU examples (#9173) 
* eee

* add examples on CPU and GPU

* fix

* fix

* optimize model examples

* have updated

* Warmup and configs added

* Update two tables
2023-10-24 15:08:04 +08:00
..
bark LLM: update example layout (#9046) 2023-10-09 15:36:39 +08:00
bert LLM: update example layout (#9046) 2023-10-09 15:36:39 +08:00
chatglm LLM: update example layout (#9046) 2023-10-09 15:36:39 +08:00
llama2 LLM: update example layout (#9046) 2023-10-09 15:36:39 +08:00
mistral LLM: add mistral examples (#9121) 2023-10-11 13:38:15 +08:00
openai-whisper LLM: update example layout (#9046) 2023-10-09 15:36:39 +08:00
phi-1_5 phi-1_5 CPU and GPU examples (#9173) 2023-10-24 15:08:04 +08:00
README.md LLM: add mistral examples (#9121) 2023-10-11 13:38:15 +08:00

README.md

BigDL-LLM INT4 Optimization for Large Language Model

You can use optimize_model API to accelerate general PyTorch models on Intel servers and PCs. This directory contains example scripts to help you quickly get started using BigDL-LLM to run some popular open-source models in the community. Each model has its own dedicated folder, where you can find detailed instructions on how to install and run it.

Verified models

Model Example
LLaMA 2 link
ChatGLM link
Openai Whisper link
BERT link
Bark link
Mistral link

Recommended Requirements

To run the examples, we recommend using Intel® Xeon® processors (server), or >= 12th Gen Intel® Core™ processor (client).

For OS, BigDL-LLM supports Ubuntu 20.04 or later, CentOS 7 or later, and Windows 10/11.

Best Known Configuration on Linux

For better performance, it is recommended to set environment variables on Linux with the help of BigDL-Nano:

pip install bigdl-nano
source bigdl-nano-init