ee33b93464
* add nv longbench * LongBench: NV code to ipex-llm * ammend * add more models support * ammend * optimize LongBench's user experience * ammend * ammend * fix typo * ammend * remove cuda related information & add a readme * add license to python scripts & polish the readme * ammend * ammend --------- Co-authored-by: cyita <yitastudy@gmail.com> Co-authored-by: ATMxsp01 <shou.xu@intel.com> Co-authored-by: leonardozcm <leonardo1997zcm@gmail.com>
29 lines
474 B
YAML
29 lines
474 B
YAML
model_name:
|
|
# - "mistral-7B-instruct-v0.2"
|
|
- "llama2-7b-chat-4k"
|
|
# - "chatglm4-9b"
|
|
# - "qwen2-7b-instruct"
|
|
|
|
full_kv: True
|
|
optimize_model: True
|
|
dtype: 'fp16'
|
|
low_bit: 'sym_int4'
|
|
|
|
e: False
|
|
|
|
compress_kv:
|
|
- "ablation_c512_w32_k7_maxpool"
|
|
- "ablation_c1024_w32_k7_maxpool"
|
|
|
|
datasets:
|
|
- "multi_news"
|
|
- "qasper"
|
|
- "hotpotqa"
|
|
- "trec"
|
|
- "passage_count"
|
|
- "lcc"
|
|
# - "multifieldqa_zh"
|
|
# - "dureader"
|
|
# - "vcsum"
|
|
# - "lsht"
|
|
# - "passage_retrieval_zh"
|