ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
2911 commits 1 branch 0 tags 37 MiB
Commit graph

433 commits

Author SHA1 Message Date
ZehuaCao
36eb8b2e96
Add llama3 speculative example (#10856) 
* Initial llama3 speculative example

* update README

* update README

* update README
 2024-04-23 17:03:54 +08:00
ZehuaCao
92ea54b512
Fix speculative decoding bug (#10855) 2024-04-23 14:28:31 +08:00
Wang, Jian4
18c032652d
LLM: Add mixtral speculative CPU example (#10830) 
* init mixtral sp example

* use different prompt_format

* update output

* update
 2024-04-23 10:05:51 +08:00
Qiyuan Gong
5494aa55f6
Downgrade datasets in axolotl example (#10849) 
* Downgrade datasets to 2.15.0 to address axolotl prepare issue https://github.com/OpenAccess-AI-Collective/axolotl/issues/1544

Tks to @kwaa for providing the solution in https://github.com/intel-analytics/ipex-llm/issues/10821#issuecomment-2068861571
 2024-04-23 09:41:58 +08:00
Guancheng Fu
47bd5f504c
[vLLM]Remove vllm-v1, refactor v2 (#10842) 
* remove vllm-v1

* fix format
 2024-04-22 17:51:32 +08:00
Wang, Jian4
23c6a52fb0
LLM: Fix ipex torchscript=True error (#10832) 
* remove

* update

* remove torchscript
 2024-04-22 15:53:09 +08:00
Heyang Sun
fc33aa3721
fix missing import (#10839) 2024-04-22 14:34:52 +08:00
Guancheng Fu
ae3b577537
Update README.md (#10833) 2024-04-22 11:07:10 +08:00
Wang, Jian4
5f95054f97
LLM：Add qwen moe example libs md (#10828) 2024-04-22 10:03:19 +08:00
Guancheng Fu
61c67af386
Fix vLLM-v2 install instructions(#10822) 2024-04-22 09:02:48 +08:00
Yang Wang
8153c3008e
Initial llama3 example (#10799) 
* Add initial hf huggingface GPU example

* Small fix

* Add llama3 gpu pytorch model example

* Add llama 3 hf transformers CPU example

* Add llama 3 pytorch model CPU example

* Fixes

* Small fix

* Small fixes

* Small fix

* Small fix

* Add links

* update repo id

* change prompt tuning url

* remove system header if there is no system prompt

---------

Co-authored-by: Yuwen Hu <yuwen.hu@intel.com>
Co-authored-by: Yuwen Hu <54161268+Oscilloscope98@users.noreply.github.com>
 2024-04-18 11:01:33 -07:00
Qiyuan Gong
e90e31719f
axolotl lora example (#10789) 
* Add axolotl lora example
* Modify readme
* Add comments in yml
 2024-04-18 16:38:32 +08:00
Guancheng Fu
cbe7b5753f
Add vLLM[xpu] related code (#10779) 
* Add ipex-llm side change

* add runable offline_inference

* refactor to call vllm2

* Verified async server

* add new v2 example

* add README

* fix

* change dir

* refactor readme.md

* add experimental

* fix
 2024-04-18 15:29:20 +08:00
Ziteng Zhang
ff040c8f01
LISA Finetuning Example (#10743) 
* enabling xetla only supports qtype=SYM_INT4 or FP8E5

* LISA Finetuning Example on gpu

* update readme

* add licence

* Explain parameters of lisa & Move backend codes to src dir

* fix style

* fix style

* update readme

* support chatglm

* fix style

* fix style

* update readme

* fix
 2024-04-18 13:48:10 +08:00
Heyang Sun
581ebf6104
GaLore Finetuning Example (#10722) 
* GaLore Finetuning Example

* Update README.md

* Update README.md

* change data to HuggingFaceH4/helpful_instructions

* Update README.md

* Update README.md

* shrink train size and delete cache before starting training to save memory

* Update README.md

* Update galore_finetuning.py

* change model to llama2 3b

* Update README.md
 2024-04-18 13:47:41 +08:00
Yina Chen
ea5b373a97
Add lookahead GPU example (#10785) 
* Add lookahead example

* fix style & attn mask

* fix typo

* address comments
 2024-04-17 17:41:55 +08:00
ZehuaCao
0646e2c062
Fix short prompt for IPEX_CPU speculative decoding cause no_attr error (#10783) 2024-04-17 16:19:57 +08:00
Cengguang Zhang
7ec82c6042
LLM: add README.md for Long-Context examples. (#10765) 
* LLM: add readme to long-context examples.

* add precision.

* update wording.

* add GPU type.

* add Long-Context example to GPU examples.

* fix comments.

* update max input length.

* update max length.

* add output length.

* fix wording.
 2024-04-17 15:34:59 +08:00
Qiyuan Gong
9e5069437f
Fix gradio version in axolotl example (#10776) 
* Change to gradio>=4.19.2
 2024-04-17 10:23:43 +08:00
Qiyuan Gong
f2e923b3ca
Axolotl v0.4.0 support (#10773) 
* Add Axolotl 0.4.0, remove legacy 0.3.0 support.
* replace is_torch_bf16_gpu_available
* Add HF_HUB_OFFLINE=1
* Move transformers out of requirement
* Refine readme and qlora.yml
 2024-04-17 09:49:11 +08:00
Heyang Sun
26cae0a39c
Update FLEX in Deepspeed README (#10774) 
* Update FLEX in Deepspeed README

* Update README.md
 2024-04-17 09:28:24 +08:00
Qiyuan Gong
d30b22a81b
Refine axolotl 0.3.0 documents and links (#10764) 
* Refine axolotl 0.3 based on comments
* Rename requirements to requirement-xpu
* Add comments for paged_adamw_32bit
* change lora_r from 8 to 16
 2024-04-16 14:47:45 +08:00
ZehuaCao
599a88db53
Add deepsped-autoTP-Fastapi serving (#10748) 
* add deepsped-autoTP-Fastapi serving

* add readme

* add license

* update

* update

* fix
 2024-04-16 14:03:23 +08:00
Jin Qiao
73a67804a4
GPU configuration update for examples (windows pip installer, etc.) (#10762) 
* renew chatglm3-6b gpu example readme

fix

fix

fix

* fix for comments

* fix

* fix

* fix

* fix

* fix

* apply on HF-Transformers-AutoModels

* apply on PyTorch-Models

* fix

* fix
 2024-04-15 17:42:52 +08:00
yb-peng
b5209d3ec1
Update example/GPU/PyTorch-Models/Model/llava/README.md (#10757) 
* Update example/GPU/PyTorch-Models/Model/llava/README.md

* Update README.md

fix path in windows installation
 2024-04-15 13:01:37 +08:00
Jiao Wang
9e668a5bf0
fix_internlm-chat-7b-8k repo name in examples (#10747) 2024-04-12 10:15:48 -07:00
Keyan (Kyrie) Zhang
1256a2cc4e
Add chatglm3 long input example (#10739) 
* Add long context input example for chatglm3

* Small fix

* Small fix

* Small fix
 2024-04-11 16:33:43 +08:00
Qiyuan Gong
2d64630757
Remove transformers version in axolotl example (#10736) 
* Remove transformers version in axolotl requirements.txt
 2024-04-11 14:02:31 +08:00
Xiangyu Tian
301504aa8d
Fix transformers version warning (#10732) 2024-04-11 13:12:49 +08:00
Shaojun Liu
29bf28bd6f
Upgrade python to 3.11 in Docker Image (#10718) 
* install python 3.11 for cpu-inference docker image

* update xpu-inference dockerfile

* update cpu-serving image

* update qlora image

* update lora image

* update document
 2024-04-10 14:41:27 +08:00
Qiyuan Gong
b727767f00
Add axolotl v0.3.0 with ipex-llm on Intel GPU (#10717) 
* Add axolotl v0.3.0 support on Intel GPU.
* Add finetune example on llama-2-7B with Alpaca dataset.
 2024-04-10 14:38:29 +08:00
Jiao Wang
d1eaea509f
update chatglm readme (#10659) 2024-04-09 14:24:46 -07:00
Jiao Wang
878a97077b
Fix llava example to support transformerds 4.36 (#10614) 
* fix llava example

* update
 2024-04-09 13:47:07 -07:00
Jiao Wang
1e817926ba
Fix low memory generation example issue in transformers 4.36 (#10702) 
* update cache in low memory generate

* update
 2024-04-09 09:56:52 -07:00
Shaojun Liu
f37a1f2a81
Upgrade to python 3.11 (#10711) 
* create conda env with python 3.11

* recommend to use Python 3.11

* update
 2024-04-09 17:41:17 +08:00
Cengguang Zhang
6a32216269
LLM: add llama2 8k input example. (#10696) 
* LLM: add llama2-32K example.

* refactor name.

* fix comments.

* add IPEX_LLM_LOW_MEM notes and update sample output.
 2024-04-09 16:02:37 +08:00
Keyan (Kyrie) Zhang
1e27e08322
Modify example from fp32 to fp16 (#10528) 
* Modify example from fp32 to fp16

* Remove Falcon from fp16 example for now

* Remove MPT from fp16 example
 2024-04-09 15:45:49 +08:00
binbin Deng
d9a1153b4e
LLM: upgrade deepspeed in AutoTP on GPU (#10647) 2024-04-07 14:05:19 +08:00
Zhicun
9d8ba64c0d
Llamaindex: add tokenizer_id and support chat (#10590) 
* add tokenizer_id

* fix

* modify

* add from_model_id and from_mode_id_low_bit

* fix typo and add comment

* fix python code style

---------

Co-authored-by: pengyb2001 <284261055@qq.com>
 2024-04-07 13:51:34 +08:00
Jin Qiao
10ee786920
Replace with IPEX-LLM in example comments (#10671) 
* Replace with IPEX-LLM in example comments

* More replacement

* revert some changes
 2024-04-07 13:29:51 +08:00
Jiao Wang
69bdbf5806
Fix vllm print error message issue (#10664) 
* update chatglm readme

* Add condition to invalidInputError

* update

* update

* style
 2024-04-05 15:08:13 -07:00
Jason Dai
29d97e4678
Update readme (#10665) 2024-04-05 18:01:57 +08:00
Jin Qiao
cc8b3be11c
Add GPU and CPU example for stablelm-zephyr-3b (#10643) 
* Add example for StableLM

* fix

* add to readme
 2024-04-03 16:28:31 +08:00
Heyang Sun
6000241b10
Add Deepspeed Example of FLEX Mistral (#10640) 2024-04-03 16:04:17 +08:00
Zhicun
b827f534d5
Add tokenizer_id in Langchain (#10588) 
* fix low-bit

* fix

* fix style

---------

Co-authored-by: arda <arda@arda-arc12.sh.intel.com>
 2024-04-03 14:25:35 +08:00
Zhicun
f6fef09933
fix prompt format for llama-2 in langchain (#10637) 2024-04-03 14:17:34 +08:00
Jiao Wang
330d4b4f4b
update readme (#10631) 2024-04-02 23:08:02 -07:00
Jiao Wang
4431134ec5
update readme (#10632) 2024-04-02 19:54:30 -07:00
Jiao Wang
654dc5ba57
Fix Qwen-VL example problem (#10582) 
* update

* update

* update

* update
 2024-04-02 12:17:30 -07:00
Ruonan Wang
d6af4877dd
LLM: remove ipex.optimize for gpt-j (#10606) 
* remove ipex.optimize

* fix

* fix
 2024-04-01 12:21:49 +08:00
Keyan (Kyrie) Zhang
848fa04dd6
Fix typo in Baichuan2 example (#10589) 2024-03-29 13:31:47 +08:00
ZehuaCao
52a2135d83
Replace ipex with ipex-llm (#10554) 
* fix ipex with ipex_llm

* fix ipex with ipex_llm

* update

* update

* update

* update

* update

* update

* update

* update
 2024-03-28 13:54:40 +08:00
Cheen Hau, 俊豪
1c5eb14128
Update pip install to use --extra-index-url for ipex package (#10557) 
* Change to 'pip install .. --extra-index-url' for readthedocs

* Change to 'pip install .. --extra-index-url' for examples

* Change to 'pip install .. --extra-index-url' for remaining files

* Fix URL for ipex

* Add links for ipex US and CN servers

* Update ipex cpu url

* remove readme

* Update for github actions

* Update for dockerfiles
 2024-03-28 09:56:23 +08:00
Cheen Hau, 俊豪
f239bc329b
Specify oneAPI minor version in documentation (#10561) 2024-03-27 17:58:57 +08:00
hxsz1997
d86477f14d
Remove native_int4 in LangChain examples (#10510) 
* rebase the modify to ipex-llm

* modify the typo
 2024-03-27 17:48:16 +08:00
Wang, Jian4
16b2ef49c6
Update_document by heyang (#30) 2024-03-25 10:06:02 +08:00
Wang, Jian4
9df70d95eb
Refactor bigdl.llm to ipex_llm (#24) 
* Rename bigdl/llm to ipex_llm

* rm python/llm/src/bigdl

* from bigdl.llm to from ipex_llm
 2024-03-22 15:41:21 +08:00
Jin Qiao
cc5806f4bc LLM: add save/load example for hf-transformers (#10432) 2024-03-22 13:57:47 +08:00
binbin Deng
2958ca49c0 LLM: add patching function for llm finetuning (#10247) 2024-03-21 16:01:01 +08:00
Zhicun
5b97fdb87b update deepseek example readme (#10420) 
* update readme

* update

* update readme
 2024-03-21 15:21:48 +08:00
hxsz1997
a5f35757a4 Migrate langchain rag cpu example to gpu (#10450) 
* add langchain rag on gpu

* add rag example in readme

* add trust_remote_code in TransformersEmbeddings.from_model_id

* add trust_remote_code in TransformersEmbeddings.from_model_id in cpu
 2024-03-21 15:20:46 +08:00
Ruonan Wang
28c315a5b9 LLM: fix deepspeed error of finetuning on xpu (#10484) 2024-03-21 09:46:25 +08:00
Cengguang Zhang
463a86cd5d LLM: fix qwen-vl interpolation gpu abnormal results. (#10457) 
* fix qwen-vl interpolation gpu abnormal results.

* fix style.

* update qwen-vl gpu example.

* fix comment and update example.

* fix style.
 2024-03-19 16:59:39 +08:00
Jiao Wang
f3fefdc9ce fix pad_token_id issue (#10425) 2024-03-18 23:30:28 -07:00
Yuxuan Xia
74e7490fda Fix Baichuan2 prompt format (#10334) 
* Fix Baichuan2 prompt format

* Fix Baichuan2 README

* Change baichuan2 prompt info

* Change baichuan2 prompt info
 2024-03-19 12:48:07 +08:00
Yang Wang
9e763b049c Support running pipeline parallel inference by vertically partitioning model to different devices (#10392) 
* support pipeline parallel inference

* fix logging

* remove benchmark file

* fic

* need to warmup twice

* support qwen and qwen2

* fix lint

* remove genxir

* refine
 2024-03-18 13:04:45 -07:00
Wang, Jian4
1de13ea578 LLM: remove CPU english_quotes dataset and update docker example (#10399) 
* update dataset

* update readme

* update docker cpu

* update xpu docker
 2024-03-18 10:45:14 +08:00
Jiao Wang
5ab52ef5b5 update (#10424) 2024-03-15 09:24:26 -07:00
Jin Qiao
ca372f6dab LLM: add save/load example for ModelScope (#10397) 
* LLM: add sl example for modelscope

* fix according to comments

* move file
 2024-03-15 15:17:50 +08:00
Wang, Jian4
fe8976a00f LLM: Support gguf models use low_bit and fix no json(#10408) 
* support others model use low_bit

* update readme

* update to add *.json
 2024-03-15 09:34:18 +08:00
Wang, Jian4
0193f29411 LLM : Enable gguf float16 and Yuan2 model (#10372) 
* enable float16

* add yun files

* enable yun

* enable set low_bit on yuan2

* update

* update license

* update generate

* update readme

* update python style

* update
 2024-03-13 10:19:18 +08:00
binbin Deng
5d7e044dbc LLM: add low bit option in deepspeed autotp example (#10382) 2024-03-12 17:07:09 +08:00
binbin Deng
df3bcc0e65 LLM: remove english_quotes dataset (#10370) 2024-03-12 16:57:40 +08:00
binbin Deng
fe27a6971c LLM: update modelscope version (#10367) 2024-03-11 16:18:27 +08:00
Zhicun
9026c08633 Fix llamaindex AutoTokenizer bug (#10345) 
* fix tokenizer

* fix AutoTokenizer bug

* modify code style
 2024-03-08 16:24:50 +08:00
Zhicun
2a10b53d73 rename docqa.py->rag.py (#10353) 2024-03-08 16:07:09 +08:00
Shengsheng Huang
370c52090c Langchain readme (#10348) 
* update langchain readme

* update readme

* create new README

* Update README_nativeint4.md
 2024-03-08 14:57:24 +08:00
hxsz1997
af11c53473 Add the installation step of postgresql and pgvector on windows in LlamaIndex GPU support (#10328) 
* add the installation of postgresql and pgvector of windows

* fix some format
 2024-03-05 18:31:19 +08:00
dingbaorong
1e6f0c6f1a Add llamaindex gpu example (#10314) 
* add llamaindex example

* fix core dump

* refine readme

* add trouble shooting

* refine readme

---------

Co-authored-by: Ariadne <wyn2000330@126.com>
 2024-03-05 13:36:00 +08:00
dingbaorong
fc7f10cd12 add langchain gpu example (#10277) 
* first draft

* fix

* add readme for transformer_int4_gpu

* fix doc

* check device_map

* add arc ut test

* fix ut test

* fix langchain ut

* Refine README

* fix gpu mem too high

* fix ut test

---------

Co-authored-by: Ariadne <wyn2000330@126.com>
 2024-03-05 13:33:57 +08:00
Xin Qiu
58208a5883 Update FAQ document. (#10300) 
* Update install_gpu.md

* Update resolve_error.md

* Update README.md

* Update resolve_error.md

* Update README.md

* Update resolve_error.md
 2024-03-04 08:35:11 +08:00
Xin Qiu
509e206de0 update doc about gemma random and unreadable output. (#10297) 
* Update install_gpu.md

* Update README.md

* Update README.md
 2024-03-01 15:41:16 +08:00
Shengsheng Huang
bcfad555df revise llamaindex readme (#10283) 2024-02-29 17:19:23 +08:00
Guancheng Fu
2d930bdca8 Add vLLM bf16 support (#10278) 
* add argument load_in_low_bit

* add docs

* modify gpu doc

* done

---------

Co-authored-by: ivy-lv11 <lvzc@lamda.nju.edu.cn>
 2024-02-29 16:33:42 +08:00
Zhicun
4e6cc424f1 Add LlamaIndex RAG (#10263) 
* run demo

* format code

* add llamaindex

* add custom LLM with bigdl

* update

* add readme

* begin ut

* add unit test

* add license

* add license

* revised

* update

* modify docs

* remove data folder

* update

* modify prompt

* fixed

* fixed

* fixed
 2024-02-29 15:21:19 +08:00
Ruonan Wang
a9fd20b6ba LLM: Update qkv fusion for GGUF-IQ2 (#10271) 
* first commit

* update mistral

* fix transformers==4.36.0

* fix

* disable qk for mixtral now

* fix style
 2024-02-29 12:49:53 +08:00
Shengsheng Huang
db0d129226 Revert "Add rwkv example (#9432)" (#10264) 
This reverts commit 6930422b42.
 2024-02-28 11:48:31 +08:00
Yining Wang
6930422b42 Add rwkv example (#9432) 
* codeshell fix wrong urls

* restart runner

* add RWKV CPU & GPU example (rwkv-4-world-7b)

* restart runner

* update submodule

* fix runner

* runner-test

---------

Co-authored-by: Shengsheng Huang <shengsheng.huang@intel.com>
 2024-02-28 11:41:00 +08:00
Keyan (Kyrie) Zhang
59861f73e5 Add Deepseek-6.7B (#9991) 
* Add new example Deepseek

* Add new example Deepseek

* Add new example Deepseek

* Add new example Deepseek

* Add new example Deepseek

* modify deepseek

* modify deepseek

* Add verified model in README

* Turn cpu_embedding=True in Deepseek example

---------

Co-authored-by: Shengsheng Huang <shengsheng.huang@intel.com>
 2024-02-28 11:36:39 +08:00
Yuxuan Xia
2524273198 Update AutoGen README (#10255) 
* Update AutoGen README

* Fix AutoGen README typos

* Update AutoGen README

* Update AutoGen README
 2024-02-28 11:34:45 +08:00
Zheng, Yi
2347f611cf Add cpu and gpu examples of Mamba (#9797) 
* Add mamba cpu example

* Add mamba gpu example

* Use a smaller model as the example

* minor fixes

---------

Co-authored-by: Shengsheng Huang <shengsheng.huang@intel.com>
 2024-02-28 11:33:29 +08:00
Zhao Changmin
937e1f7c74 rebase (#9104) 
Co-authored-by: leonardozcm <leonardozcm@gmail.com>
 2024-02-28 11:18:21 +08:00
Zhicun
308e637d0d Add DeepSeek-MoE-16B-Chat (#10155) 
* dsmoe-hf add

* add dsmoe pytorch

* update README

* modify comment

* remove GPU example

* update model name

* format code
 2024-02-28 10:12:09 +08:00
Guoqiong Song
f4a2e32106 Stream llm example for both GPU and CPU (#9390) 2024-02-27 15:54:47 -08:00
Keyan (Kyrie) Zhang
843fe546b0 Add CPU and GPU examples for DeciLM-7B (#9867) 
* Add cpu and gpu examples for DeciLM-7B

* Add cpu and gpu examples for DeciLM-7B

* Add DeciLM-7B to README table

* modify deciLM

* modify deciLM

* modify deciLM

* Add verified model in README

* Add cpu_embedding=True
 2024-02-27 13:15:49 +08:00
Heyang Sun
36a9e88104 Speculative Starcoder on CPU (#10138) 
* Speculative Starcoder on CPU

* enable kv-cache pre-allocation

* refine codes

* refine

* fix style

* fix style

* fix style

* refine

* refine

* Update speculative.py

* Update gptbigcode.py

* fix style

* Update speculative.py

* enable mixed-datatype layernorm on top of torch API

* adaptive dtype

* Update README.md
 2024-02-27 09:57:29 +08:00
Wang, Jian4
6c74b99a28 LLM: Update qwen readme (#10245) 2024-02-26 17:03:09 +08:00
Wang, Jian4
f9b75f900b LLM: Enable qwen target_model ipex (#10232) 
* change order

* enable qwen ipex

* update qwen example

* update

* fix style

* update
 2024-02-26 16:41:12 +08:00
Ziteng Zhang
ea23afc8ec [LLM]update ipex part in mistral example readme (#10239) 
* update ipex part in mistral example readme
 2024-02-26 14:35:20 +08:00
Xiangyu Tian
85a99e13e8 LLM: Fix ChatGLM3 Speculative Example (#10236) 
Fix ChatGLM3 Speculative Example.
 2024-02-26 10:57:28 +08:00