ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

fix_internlm-chat-7b-8k repo name in examples (#10747)

Browse source
This commit is contained in:
Jiao Wang 2024-04-12 10:15:48 -07:00 committed by GitHub
parent c3fc8f4b90
commit 9e668a5bf0
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
9 changed files with 15 additions and 15 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
python/llm/example/CPU/HF-Transformers-AutoModels/Model/internlm/README.md
View file

@ -1,6 +1,6 @@
# InternLM
In this directory, you will find examples on how you could apply IPEX-LLM INT4 optimizations on InternLM models. For illustration purposes, we utilize the [internlm/internlm-chat-7b-8k](https://huggingface.co/internlm/internlm-chat-7b-8k) as a reference InternLM model.
In this directory, you will find examples on how you could apply IPEX-LLM INT4 optimizations on InternLM models. For illustration purposes, we utilize the [internlm/internlm-chat-7b](https://huggingface.co/internlm/internlm-chat-7b) as a reference InternLM model.
## 0. Requirements
To run these examples with IPEX-LLM, we have some recommended requirements for your machine, please refer to [here](../README.md#recommended-requirements) for more information.
@ -22,7 +22,7 @@ python ./generate.py --repo-id-or-model-path REPO_ID_OR_MODEL_PATH --prompt PROM
```
Arguments info:
- `--repo-id-or-model-path REPO_ID_OR_MODEL_PATH`: argument defining the huggingface repo id for the InternLM model to be downloaded, or the path to the huggingface checkpoint folder. It is default to be `'internlm/internlm-chat-7b-8k'`.
- `--repo-id-or-model-path REPO_ID_OR_MODEL_PATH`: argument defining the huggingface repo id for the InternLM model to be downloaded, or the path to the huggingface checkpoint folder. It is default to be `'internlm/internlm-chat-7b'`.
- `--prompt PROMPT`: argument defining the prompt to be infered (with integrated prompt format for chat). It is default to be `'AI是什么？'`.
- `--n-predict N_PREDICT`: argument defining the max number of tokens to predict. It is default to be `32`.
@ -50,7 +50,7 @@ numactl -C 0-47 -m 0 python ./generate.py
```
#### 2.3 Sample Output
#### [internlm/internlm-chat-7b-8k](https://huggingface.co/internlm/internlm-chat-7b-8k)
#### [internlm/internlm-chat-7b](https://huggingface.co/internlm/internlm-chat-7b)
```log
Inference time: xxxx s
-------------------- Prompt --------------------

4
python/llm/example/CPU/HF-Transformers-AutoModels/Model/internlm/generate.py
View file

@ -23,12 +23,12 @@ from ipex_llm.transformers import AutoModelForCausalLM
from transformers import AutoTokenizer
# you could tune the prompt based on your own model,
# here the prompt tuning refers to https://huggingface.co/internlm/internlm-chat-7b-8k/blob/main/modeling_internlm.py#L768
# here the prompt tuning refers to https://huggingface.co/internlm/internlm-chat-7b/blob/main/modeling_internlm.py#L1053
INTERNLM_PROMPT_FORMAT = "<|User|>:{prompt}\n<|Bot|>:"
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Predict Tokens using `generate()` API for InternLM model')
parser.add_argument('--repo-id-or-model-path', type=str, default="internlm/internlm-chat-7b-8k",
parser.add_argument('--repo-id-or-model-path', type=str, default="internlm/internlm-chat-7b",
help='The huggingface repo id for the InternLM model to be downloaded'
', or the path to the huggingface checkpoint folder')
parser.add_argument('--prompt', type=str, default="AI是什么？",

2
python/llm/example/CPU/HF-Transformers-AutoModels/Model/internlm2/generate.py
View file

@ -22,7 +22,7 @@ import numpy as np
from transformers import AutoTokenizer
# you could tune the prompt based on your own model,
# here the prompt tuning refers to https://huggingface.co/internlm/internlm-chat-7b-8k/blob/main/modeling_internlm.py#L768
# here the prompt tuning refers to https://huggingface.co/internlm/internlm-chat-7b/blob/main/modeling_internlm.py#L1053
INTERNLM_PROMPT_FORMAT = "<|User|>:{prompt}\n<|Bot|>:"
if __name__ == '__main__':

2
python/llm/example/CPU/PyTorch-Models/Model/internlm2/generate.py
View file

@ -23,7 +23,7 @@ from ipex_llm import optimize_model
from transformers import AutoTokenizer
# you could tune the prompt based on your own model,
# here the prompt tuning refers to https://huggingface.co/internlm/internlm-chat-7b-8k/blob/main/modeling_internlm.py#L768
# here the prompt tuning refers to https://huggingface.co/internlm/internlm-chat-7b/blob/main/modeling_internlm.py#L1053
INTERNLM_PROMPT_FORMAT = "<|User|>:{prompt}\n<|Bot|>:"
if __name__ == '__main__':

6
python/llm/example/GPU/HF-Transformers-AutoModels/Model/internlm/README.md
View file

@ -1,5 +1,5 @@
# InternLM
In this directory, you will find examples on how you could apply IPEX-LLM INT4 optimizations on InternLM models on [Intel GPUs](../../../README.md). For illustration purposes, we utilize the [internlm/internlm-chat-7b-8k](https://huggingface.co/internlm/internlm-chat-7b-8k) as a reference InternLM model.
In this directory, you will find examples on how you could apply IPEX-LLM INT4 optimizations on InternLM models on [Intel GPUs](../../../README.md). For illustration purposes, we utilize the [internlm/internlm-chat-7b](https://huggingface.co/internlm/internlm-chat-7b) as a reference InternLM model.
## 0. Requirements
To run these examples with IPEX-LLM on Intel GPUs, we have some recommended requirements for your machine, please refer to [here](../../../README.md#requirements) for more information.
@ -100,12 +100,12 @@ python ./generate.py --repo-id-or-model-path REPO_ID_OR_MODEL_PATH --prompt PROM
```
Arguments info:
- `--repo-id-or-model-path REPO_ID_OR_MODEL_PATH`: argument defining the huggingface repo id for the InternLM model (e.g. `internlm/internlm-chat-7b-8k`) to be downloaded, or the path to the huggingface checkpoint folder. It is default to be `'internlm/internlm-chat-7b-8k'`.
- `--repo-id-or-model-path REPO_ID_OR_MODEL_PATH`: argument defining the huggingface repo id for the InternLM model (e.g. `internlm/internlm-chat-7b`) to be downloaded, or the path to the huggingface checkpoint folder. It is default to be `'internlm/internlm-chat-7b'`.
- `--prompt PROMPT`: argument defining the prompt to be infered (with integrated prompt format for chat). It is default to be `'AI是什么？'`.
- `--n-predict N_PREDICT`: argument defining the max number of tokens to predict. It is default to be `32`.
#### Sample Output
#### [internlm/internlm-chat-7b-8k](https://huggingface.co/internlm/internlm-chat-7b-8k)
#### [internlm/internlm-chat-7b](https://huggingface.co/internlm/internlm-chat-7b)
```log
Inference time: xxxx s
-------------------- Prompt --------------------

4
python/llm/example/GPU/HF-Transformers-AutoModels/Model/internlm/generate.py
View file

@ -22,12 +22,12 @@ from ipex_llm.transformers import AutoModelForCausalLM
from transformers import AutoTokenizer
# you could tune the prompt based on your own model,
# here the prompt tuning refers to https://huggingface.co/internlm/internlm-chat-7b-8k/blob/main/modeling_internlm.py#L768
# here the prompt tuning refers to https://huggingface.co/internlm/internlm-chat-7b/blob/main/modeling_internlm.py#L1053
INTERNLM_PROMPT_FORMAT = "<|User|>:{prompt}\n<|Bot|>:"
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Predict Tokens using `generate()` API for InternLM model')
parser.add_argument('--repo-id-or-model-path', type=str, default="internlm/internlm-chat-7b-8k",
parser.add_argument('--repo-id-or-model-path', type=str, default="internlm/internlm-chat-7b",
help='The huggingface repo id for the InternLM model to be downloaded'
', or the path to the huggingface checkpoint folder')
parser.add_argument('--prompt', type=str, default="AI是什么？",

2
python/llm/example/GPU/HF-Transformers-AutoModels/Model/internlm2/generate.py
View file

@ -23,7 +23,7 @@ from ipex_llm import optimize_model
import intel_extension_for_pytorch as ipex
# you could tune the prompt based on your own model,
# here the prompt tuning refers to https://huggingface.co/internlm/internlm-chat-7b-8k/blob/main/modeling_internlm.py#L768
# here the prompt tuning refers to https://huggingface.co/internlm/internlm-chat-7b/blob/main/modeling_internlm.py#L1053
INTERNLM_PROMPT_FORMAT = "<|User|>:{prompt}\n<|Bot|>:"
if __name__ == '__main__':

2
python/llm/example/GPU/PyTorch-Models/Model/internlm2/generate.py
View file

@ -23,7 +23,7 @@ from ipex_llm import optimize_model
import intel_extension_for_pytorch as ipex
# you could tune the prompt based on your own model,
# here the prompt tuning refers to https://huggingface.co/internlm/internlm-chat-7b-8k/blob/main/modeling_internlm.py#L768
# here the prompt tuning refers to https://huggingface.co/internlm/internlm-chat-7b/blob/main/modeling_internlm.py#L1053
INTERNLM_PROMPT_FORMAT = "<|User|>:{prompt}\n<|Bot|>:"
if __name__ == '__main__':

2
python/llm/portable-zip/README.md
View file

@ -15,7 +15,7 @@ This portable zip includes everything you need to run an LLM with IPEX-LLM optim
- ChatGLM2-6b
- Baichuan-13B-Chat
- Baichuan2-7B-Chat
- internlm-chat-7b-8k
- internlm-chat-7b
- Llama-2-7b-chat-hf
## How to use