ipex-llm/python/llm/example/NPU/HF-Transformers-AutoModels

IPEX-LLM Examples on Intel NPU

This folder contains examples of running IPEX-LLM on Intel NPU:

• LLM: examples of running large language models using IPEX-LLM optimizations
• Multimodal: examples of running large multimodal models using IPEX-LLM optimizations

Verified Models on Intel NPU

Model	Model Link
Llama2	meta-llama/Llama-2-7b-chat-hf
Llama3	meta-llama/Meta-Llama-3-8B-Instruct
Llama3.2-1B	meta-llama/Llama-3.2-1B-Instruct
Llama3.2-3B	meta-llama/Llama-3.2-3B-Instruct
Chatglm3	THUDM/chatglm3-6b
Chatglm2	THUDM/chatglm2-6b
GLM-Edge	THUDM/glm-edge-1.5b-chat, THUDM/glm-edge-4b-chat
Qwen2	Qwen/Qwen2-7B-Instruct, Qwen/Qwen2-1.5B-Instruct
Qwen2.5	Qwen/Qwen2.5-7B-Instruct
MiniCPM	openbmb/MiniCPM-2B-sft-bf16
Phi-3	microsoft/Phi-3-mini-4k-instruct
Stablelm	stabilityai/stablelm-zephyr-3b
Baichuan2	baichuan-inc/Baichuan2-7B-Chat
Deepseek	deepseek-ai/deepseek-coder-6.7b-instruct
Mistral	mistralai/Mistral-7B-Instruct-v0.1
Phi-3-Vision	microsoft/Phi-3-vision-128k-instruct
MiniCPM-Llama3-V-2_5	openbmb/MiniCPM-Llama3-V-2_5
MiniCPM-V-2_6	openbmb/MiniCPM-V-2_6
Bce-Embedding-Base-V1	maidalun1020/bce-embedding-base_v1
Speech_Paraformer-Large	iic/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch