8 commits

Author	SHA1	Message	Date
Xiangyu Tian	4359ab3172	LLM: Add /generate_stream endpoint for Pipeline-Parallel-FastAPI example (#11187) ... Add /generate_stream and OpenAI-formatted endpoint for Pipeline-Parallel-FastAPI example	2024-06-14 15:15:32 +08:00
Wang, Jian4	6f2684e5c9	Update pp llama.py to save memory (#11233)	2024-06-07 13:18:16 +08:00
Yuwen Hu	af96579c76	Update installation guide for pipeline parallel inference (#11224) ... * Update installation guide for pipeline parallel inference * Small fix * further fix * Small fix * Small fix * Update based on comments * Small fix * Small fix * Small fix	2024-06-05 17:54:29 +08:00
Qiyuan Gong	ce3f08b25a	Fix IPEX auto importer (#11192) ... * Fix ipex auto importer with Python builtins. * Raise errors if the user imports ipex manually before importing ipex_llm. Do nothing if they import ipex after importing ipex_llm. * Remove import ipex in examples.	2024-06-04 16:57:18 +08:00
Wang, Jian4	c0f1be6aea	Fix pp logic (#11175) ... * only send no none batch and rank1-n sending first * always send first	2024-05-30 16:40:59 +08:00
Xiangyu Tian	2299698b45	Refine Pipeline Parallel FastAPI example (#11168)	2024-05-29 17:16:50 +08:00
Xiangyu Tian	b44cf405e2	Refine Pipeline-Parallel-Fastapi example README (#11155)	2024-05-28 15:18:21 +08:00
Xiangyu Tian	5c8ccf0ba9	LLM: Add Pipeline-Parallel-FastAPI example (#10917) ... Add multi-stage Pipeline-Parallel-FastAPI example --------- Co-authored-by: hzjane <a1015616934@qq.com>	2024-05-27 14:46:29 +08:00