ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
ipex-llm/python/llm/example/CPU/Speculative-Decoding
History
Shaojun Liu 6d89c827a8
Fix PSIRT Vulnerability - Dependency Confusion in oneccl_bind_pt package (#13305) 
* Fix PSIRT Vulnerability - Dependency Confusion in oneccl_bind_pt package

* update

---------

Co-authored-by: YongZhuIntel <yong.zhu@intel.com>
2025-09-12 10:22:18 +08:00
..
EAGLE remove fschat in EAGLE example (#13005) 2025-03-25 15:48:48 +08:00
Self-Speculation Fix PSIRT Vulnerability - Dependency Confusion in oneccl_bind_pt package (#13305) 2025-09-12 10:22:18 +08:00
README.md Reconstruct Speculative Decoding example directory (#11136) 2024-05-29 13:15:27 -07:00

README.md

Speculative-Decoding Examples on Intel CPU

This folder contains examples of running Speculative-Decoding Examples with IPEX-LLM on Intel CPU:

  • Self-Speculation: running BF16 inference for Huggingface Transformer model with self-speculative decoding with IPEX-LLM on Intel CPUs
  • EAGLE: running speculative sampling using EAGLE (Extrapolation Algorithm for Greater Language-model Efficiency) with IPEX-LLM on Intel CPUs