5a2ce421af
* add cpu and gpu examples of flan-t5 * address yuwen's comments * Add explanation why we add modules to not convert * Refine prompt and add a translation example * Add a empty line at the end of files * add examples of flan-t5 using optimize_mdoel api * address bin's comments * address binbin's comments * add flan-t5 in readme
1.5 KiB
1.5 KiB
BigDL-LLM INT4 Optimization for Large Language Model
You can use optimize_model API to accelerate general PyTorch models on Intel servers and PCs. This directory contains example scripts to help you quickly get started using BigDL-LLM to run some popular open-source models in the community. Each model has its own dedicated folder, where you can find detailed instructions on how to install and run it.
Verified models
| Model | Example |
|---|---|
| LLaMA 2 | link |
| ChatGLM | link |
| Openai Whisper | link |
| BERT | link |
| Bark | link |
| Mistral | link |
| Flan-t5 | link |
Recommended Requirements
To run the examples, we recommend using Intel® Xeon® processors (server), or >= 12th Gen Intel® Core™ processor (client).
For OS, BigDL-LLM supports Ubuntu 20.04 or later, CentOS 7 or later, and Windows 10/11.
Best Known Configuration on Linux
For better performance, it is recommended to set environment variables on Linux with the help of BigDL-Nano:
pip install bigdl-nano
source bigdl-nano-init