c619315131
* First push of bigdl-llm example for gptneox model family * Add some args and other small updates * Small updates * Add example for llama family models * Small fix * Small fix * Update for batch_decode api and change default model for llama example * Small fix * Small fix * Small fix * Small model family name fix and add example for bloom * Small fix * Small default prompt fix * Small fix * Change default prompt * Add sample output for inference * Hide example inference time |
||
|---|---|---|
| .. | ||
| dev | ||
| example | ||
| src/bigdl | ||
| test | ||
| README.md | ||
| setup.py | ||
BigDL LLM
llm-cli
llm-cli is a command-line interface tool that allows easy execution of llama/gptneox/bloom models and generates results based on the provided prompt.
Usage
llm-cli -x <llama/gptneox/bloom> [-h] [args]
args are the arguments provided to the specified model program. You can use -x MODEL_FAMILY -h
to retrieve the parameter list for a specific MODEL_FAMILY, for example:
llm-cli.sh -x llama -h
# Output:
# usage: main-llama [options]
#
# options:
# -h, --help show this help message and exit
# -i, --interactive run in interactive mode
# --interactive-first run in interactive mode and wait for input right away
# ...
Examples
Here are some examples of how to use the llm-cli tool:
Completion:
llm-cli.sh -t 16 -x llama -m ./llm-llama-model.bin -p 'Once upon a time,'
Chatting:
llm-cli.sh -t 16 -x llama -m ./llm-llama-model.bin -i --color
Feel free to explore different options and experiment with the llama/gptneox/bloom models using llm-cli!