ce3f08b25a
* Fix ipex auto importer with Python builtins. * Raise errors if the user imports ipex manually before importing ipex_llm. Do nothing if they import ipex after importing ipex_llm. * Remove import ipex in examples. |
||
|---|---|---|
| .. | ||
| EAGLE | ||
| Self-Speculation | ||
| README.md | ||
Speculative-Decoding Examples on Intel GPU
This folder contains examples of running Speculative-Decoding Examples with IPEX-LLM on Intel GPU:
- Self-Speculation: running BF16 inference for Huggingface Transformer model with self-speculative decoding with IPEX-LLM on Intel GPUs
- EAGLE: running speculative sampling using EAGLE (Extrapolation Algorithm for Greater Language-model Efficiency) with IPEX-LLM on Intel GPUs