ipex-llm/python/llm/example/GPU

BigDL-LLM Examples on Intel GPU

This folder contains examples of running BigDL-LLM on Intel GPU:

• HF-Transformers-AutoModels: running any Hugging Face Transformers model on BigDL-LLM (using the standard AutoModel APIs)
• QLoRA-FineTuning: running QLoRA finetuning using BigDL-LLM on Intel GPUs
• vLLM-Serving: running vLLM serving framework on intel GPUs (with BigDL-LLM low-bit optimized models)
• Deepspeed-AutoTP: running distributed inference using DeepSpeed AutoTP (with BigDL-LLM low-bit optimized models) on Intel GPUs
• PyTorch-Models: running any PyTorch model on BigDL-LLM (with "one-line code change")

System Support

Hardware:

• Intel Arc™ A-Series Graphics
• Intel Data Center GPU Flex Series
• Intel Data Center GPU Max Series

Operating System:

• Ubuntu 20.04 or later (Ubuntu 22.04 is preferred)

Requirements

To apply Intel GPU acceleration, there’re several steps for tools installation and environment preparation. See the GPU installation guide for mode details.