a7b66683f1
* Add initial support for llama3.2-1b/3b * move llama3.2 support into current llama_mp impl
2.5 KiB
2.5 KiB
IPEX-LLM Examples on Intel NPU
This folder contains examples of running IPEX-LLM on Intel NPU:
- LLM: examples of running large language models using IPEX-LLM optimizations
- Multimodal: examples of running large multimodal models using IPEX-LLM optimizations
Verified Models on Intel NPU
| Model | Model Link |
|---|---|
| Llama2 | meta-llama/Llama-2-7b-chat-hf |
| Llama3 | meta-llama/Meta-Llama-3-8B-Instruct |
| Llama3.2-1B | meta-llama/Llama-3.2-1B-Instruct |
| Llama3.2-3B | meta-llama/Llama-3.2-3B-Instruct |
| Chatglm3 | THUDM/chatglm3-6b |
| Chatglm2 | THUDM/chatglm2-6b |
| Qwen2 | Qwen/Qwen2-7B-Instruct, Qwen/Qwen2-1.5B-Instruct |
| Qwen2.5 | Qwen/Qwen2.5-7B-Instruct |
| MiniCPM | openbmb/MiniCPM-2B-sft-bf16 |
| Phi-3 | microsoft/Phi-3-mini-4k-instruct |
| Stablelm | stabilityai/stablelm-zephyr-3b |
| Baichuan2 | baichuan-inc/Baichuan2-7B-Chat |
| Deepseek | deepseek-ai/deepseek-coder-6.7b-instruct |
| Mistral | mistralai/Mistral-7B-Instruct-v0.1 |
| Phi-3-Vision | microsoft/Phi-3-vision-128k-instruct |
| MiniCPM-Llama3-V-2_5 | openbmb/MiniCPM-Llama3-V-2_5 |
| MiniCPM-V-2_6 | openbmb/MiniCPM-V-2_6 |
| Speech_Paraformer-Large | iic/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch |