diff --git a/README.md b/README.md index a34c880b..b211f604 100644 --- a/README.md +++ b/README.md @@ -319,7 +319,7 @@ Over 50 models have been optimized/verified on `ipex-llm`, including *LLaMA/LLaM | MiniCPM-V | | [link](python/llm/example/GPU/HuggingFace/Multimodal/MiniCPM-V) | | MiniCPM-V-2 | | [link](python/llm/example/GPU/HuggingFace/Multimodal/MiniCPM-V-2) | | MiniCPM-Llama3-V-2_5 | | [link](python/llm/example/GPU/HuggingFace/Multimodal/MiniCPM-Llama3-V-2_5) | -| MiniCPM-V-2_6 | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/minicpm-v) | [link](python/llm/example/GPU/HuggingFace/Multimodal/MiniCPM-V-2_6) | +| MiniCPM-V-2_6 | [link](python/llm/example/CPU/HF-Transformers-AutoModels/Model/minicpm-v-2_6) | [link](python/llm/example/GPU/HuggingFace/Multimodal/MiniCPM-V-2_6) | ## Get Support - Please report a bug or raise a feature request by opening a [Github Issue](https://github.com/intel-analytics/ipex-llm/issues) diff --git a/python/llm/example/CPU/HF-Transformers-AutoModels/Model/minicpm-v/README.md b/python/llm/example/CPU/HF-Transformers-AutoModels/Model/minicpm-v-2_6/README.md similarity index 88% rename from python/llm/example/CPU/HF-Transformers-AutoModels/Model/minicpm-v/README.md rename to python/llm/example/CPU/HF-Transformers-AutoModels/Model/minicpm-v-2_6/README.md index 640be289..4e0955b8 100644 --- a/python/llm/example/CPU/HF-Transformers-AutoModels/Model/minicpm-v/README.md +++ b/python/llm/example/CPU/HF-Transformers-AutoModels/Model/minicpm-v-2_6/README.md @@ -1,11 +1,11 @@ -# MiniCPM-V -In this directory, you will find examples on how you could apply IPEX-LLM INT4 optimizations on MiniCPM-V models. For illustration purposes, we utilize the [openbmb/MiniCPM-V-2_6](https://huggingface.co/openbmb/MiniCPM-V-2_6) as a reference MiniCPM-V model. +# MiniCPM-V-2_6 +In this directory, you will find examples on how you could apply IPEX-LLM INT4 optimizations on MiniCPM-V-2_6 models. For illustration purposes, we utilize the [openbmb/MiniCPM-V-2_6](https://huggingface.co/openbmb/MiniCPM-V-2_6) as a reference MiniCPM-V-2_6 model. ## 0. Requirements To run these examples with IPEX-LLM, we have some recommended requirements for your machine, please refer to [here](../README.md#recommended-requirements) for more information. ## Example: Predict Tokens using `chat()` API -In the example [chat.py](./chat.py), we show a basic use case for a MiniCPM-V model to predict the next N tokens using `chat()` API, with IPEX-LLM INT4 optimizations. +In the example [chat.py](./chat.py), we show a basic use case for a MiniCPM-V-2_6 model to predict the next N tokens using `chat()` API, with IPEX-LLM INT4 optimizations. ### 1. Install We suggest using conda to manage environment: @@ -47,7 +47,7 @@ pip install transformers==4.40.0 trl Arguments info: -- `--repo-id-or-model-path REPO_ID_OR_MODEL_PATH`: argument defining the huggingface repo id for the MiniCPM-V model (e.g. `openbmb/MiniCPM-V-2_6`) to be downloaded, or the path to the huggingface checkpoint folder. It is default to be `'openbmb/MiniCPM-V-2_6'`. +- `--repo-id-or-model-path REPO_ID_OR_MODEL_PATH`: argument defining the huggingface repo id for the MiniCPM-V-2_6 model (e.g. `openbmb/MiniCPM-V-2_6`) to be downloaded, or the path to the huggingface checkpoint folder. It is default to be `'openbmb/MiniCPM-V-2_6'`. - `--image-url-or-path IMAGE_URL_OR_PATH`: argument defining the image to be infered. It is default to be `'http://farm6.staticflickr.com/5268/5602445367_3504763978_z.jpg'`. - `--prompt PROMPT`: argument defining the prompt to be infered (with integrated prompt format for chat). It is default to be `'What is in the image?'`. - `--stream`: flag to chat in streaming mode diff --git a/python/llm/example/CPU/HF-Transformers-AutoModels/Model/minicpm-v/chat.py b/python/llm/example/CPU/HF-Transformers-AutoModels/Model/minicpm-v-2_6/chat.py similarity index 97% rename from python/llm/example/CPU/HF-Transformers-AutoModels/Model/minicpm-v/chat.py rename to python/llm/example/CPU/HF-Transformers-AutoModels/Model/minicpm-v-2_6/chat.py index e0a07c59..a6f44bd0 100644 --- a/python/llm/example/CPU/HF-Transformers-AutoModels/Model/minicpm-v/chat.py +++ b/python/llm/example/CPU/HF-Transformers-AutoModels/Model/minicpm-v-2_6/chat.py @@ -26,9 +26,9 @@ from transformers import AutoTokenizer if __name__ == '__main__': - parser = argparse.ArgumentParser(description='Predict Tokens using `chat()` API for MiniCPM-V model') + parser = argparse.ArgumentParser(description='Predict Tokens using `chat()` API for MiniCPM-V-2_6 model') parser.add_argument('--repo-id-or-model-path', type=str, default="openbmb/MiniCPM-V-2_6", - help='The huggingface repo id for the MiniCPM-V model to be downloaded' + help='The huggingface repo id for the MiniCPM-V-2_6 model to be downloaded' ', or the path to the huggingface checkpoint folder') parser.add_argument('--image-url-or-path', type=str, default='http://farm6.staticflickr.com/5268/5602445367_3504763978_z.jpg',