ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

[LLM] Transformers int4 example small typo fixes (#8550)

Browse source
This commit is contained in:
Yuwen Hu 2023-07-17 18:15:32 +08:00 committed by GitHub
parent 1344f50f75
commit ee70977c07
3 changed files with 5 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
python/llm/example/transformers/transformers_int4/mpt/README.md
View file

@ -22,7 +22,7 @@ python ./generate.py --repo-id-or-model-path REPO_ID_OR_MODEL_PATH --prompt PROM
``` ```
Arguments info: Arguments info:
- `--repo-id-or-model-path REPO_ID_OR_MODEL_PATH`: argument defining the huggingface repo id for the MPT model (e.g. `mosaicml/mpt-7b-chat` and `mosaicml/mpt-7b-chat`) to be downloaded, or the path to the huggingface checkpoint folder. It is default to be `'mosaicml/mpt-7b-chat'`. - `--repo-id-or-model-path REPO_ID_OR_MODEL_PATH`: argument defining the huggingface repo id for the MPT model (e.g. `mosaicml/mpt-7b-chat` and `mosaicml/mpt-30b-chat`) to be downloaded, or the path to the huggingface checkpoint folder. It is default to be `'mosaicml/mpt-7b-chat'`.
- `--prompt PROMPT`: argument defining the prompt to be infered (with integrated prompt format for chat). It is default to be `'What is AI?'`. - `--prompt PROMPT`: argument defining the prompt to be infered (with integrated prompt format for chat). It is default to be `'What is AI?'`.
- `--n-predict N_PREDICT`: argument defining the max number of tokens to predict. It is default to be `32`. - `--n-predict N_PREDICT`: argument defining the max number of tokens to predict. It is default to be `32`.
@ -67,6 +67,7 @@ AI, or artificial intelligence, is the simulation of human intelligence in machi
#### [mosaicml/mpt-30b-chat](https://huggingface.co/mosaicml/mpt-30b-chat) #### [mosaicml/mpt-30b-chat](https://huggingface.co/mosaicml/mpt-30b-chat)
```log ```log
Inference time: xxxx s
-------------------- Prompt -------------------- -------------------- Prompt --------------------
<|im_start|>user <|im_start|>user
What is AI?<|im_end|> What is AI?<|im_end|>

2
python/llm/example/transformers/transformers_int4/mpt/generate.py
View file

@ -29,7 +29,7 @@ if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Predict Tokens using `generate()` API for MPT model') parser = argparse.ArgumentParser(description='Predict Tokens using `generate()` API for MPT model')
parser.add_argument('--repo-id-or-model-path', type=str, default="mosaicml/mpt-7b-chat", parser.add_argument('--repo-id-or-model-path', type=str, default="mosaicml/mpt-7b-chat",
help='The huggingface repo id for the MPT models' help='The huggingface repo id for the MPT models'
'(e.g. `mosaicml/mpt-7b-chat` and `mosaicml/mpt-7b-chat`) to be downloaded' '(e.g. `mosaicml/mpt-7b-chat` and `mosaicml/mpt-30b-chat`) to be downloaded'
', or the path to the huggingface checkpoint folder') ', or the path to the huggingface checkpoint folder')
parser.add_argument('--prompt', type=str, default="What is AI?", parser.add_argument('--prompt', type=str, default="What is AI?",
help='Prompt to infer') help='Prompt to infer')