ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
ipex-llm/python/llm/example/pytorch-models
History
binbin Deng 2d81521019 LLM: add optimize_model examples for llama2 and chatglm (#8894) 
* add llama2 and chatglm optimize_model examples

* update default usage

* update command and some descriptions

* move folder and remove general_int4 descriptions

* change folder name
2023-09-12 10:36:29 +08:00
..
chatglm LLM: add optimize_model examples for llama2 and chatglm (#8894) 2023-09-12 10:36:29 +08:00
llama2 LLM: add optimize_model examples for llama2 and chatglm (#8894) 2023-09-12 10:36:29 +08:00
openai-whisper LLM: add optimize_model examples for llama2 and chatglm (#8894) 2023-09-12 10:36:29 +08:00
README.md LLM: add optimize_model examples for llama2 and chatglm (#8894) 2023-09-12 10:36:29 +08:00

README.md

BigDL-LLM INT4 Optimization for Large Language Model

You can use optimize_model API to accelerate general PyTorch models on Intel servers and PCs. This directory contains example scripts to help you quickly get started using BigDL-LLM to run some popular open-source models in the community. Each model has its own dedicated folder, where you can find detailed instructions on how to install and run it.

Verified models

Model Example
LLaMA 2 link
ChatGLM link
Openai Whisper link

Recommended Requirements

To run the examples, we recommend using Intel® Xeon® processors (server), or >= 12th Gen Intel® Core™ processor (client).

For OS, BigDL-LLM supports Ubuntu 20.04 or later, CentOS 7 or later, and Windows 10/11.

Best Known Configuration on Linux

For better performance, it is recommended to set environment variables on Linux with the help of BigDL-Nano:

pip install bigdl-nano
source bigdl-nano-init