remove disco mixtral, update oneapi version (#9671)

python/llm/example/GPU/HF-Transformers-AutoModels/Model/mixtral/README.md

@@ -1,5 +1,5 @@
 # Mixtral
-In this directory, you will find examples on how you could apply BigDL-LLM INT4 optimizations on Mixtral models on [Intel GPUs](../README.md). For illustration purposes, we utilize the [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) and [DiscoResearch/mixtral-7b-8expert](https://huggingface.co/DiscoResearch/mixtral-7b-8expert) as reference Mixtral models.
+In this directory, you will find examples on how you could apply BigDL-LLM INT4 optimizations on Mixtral models on [Intel GPUs](../README.md). For illustration purposes, we utilize the [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) as a reference Mixtral model.
 
 ## Requirements
 To run these examples with BigDL-LLM on Intel GPUs, we have some recommended requirements for your machine, please refer to [here](../README.md#recommended-requirements) for more information.
@@ -24,29 +24,12 @@ pip install --pre --upgrade bigdl-llm[xpu] -f https://developer.intel.com/ipex-w
 pip install transformers==4.36.0
 ```
 
-### (Optional) 2. Download Model and Replace File
+### 2. Configures OneAPI environment variables
-To run [DiscoResearch/mixtral-7b-8expert](https://huggingface.co/DiscoResearch/mixtral-7b-8expert) model on Intel GPU, we have provided an updated version [DiscoResearch-mixtral-7b-8expert/modeling_moe_mistral.py](./DiscoResearch-mixtral-7b-8expert/modeling_moe_mistral.py) of `modeling_moe_mistral.py`.
-
-#### 2.1 Download Model
-You could use the following code to download [DiscoResearch/mixtral-7b-8expert](https://huggingface.co/DiscoResearch/mixtral-7b-8expert).
-
-```python
-from huggingface_hub import snapshot_download
-
-# for DiscoResearch/mixtral-7b-8expert
-model_path = snapshot_download(repo_id='DiscoResearch/mixtral-7b-8expert')
-print(f'DiscoResearch/mixtral-7b-8expert checkpoint is downloaded to {model_path}')
-```
-
-#### 2.2 Replace `modeling_moe_mistral.py`
-For `DiscoResearch/mixtral-7b-8expert`, you should replace the `modeling_moe_mistral.py` with [DiscoResearch-mixtral-7b-8expert/modeling_moe_mistral.py](./DiscoResearch-mixtral-7b-8expert/modeling_moe_mistral.py).
-
-### 3. Configures OneAPI environment variables
 ```bash
 source /opt/intel/oneapi/setvars.sh
 ```
 
-### 4. Run
+### 3. Run
 
 For optimal performance on Arc, it is recommended to set several environment variables.
 
@@ -61,7 +44,7 @@ python ./generate.py --prompt 'What is AI?'
 
 In the example, several arguments can be passed to satisfy your requirements:
 
-- `--repo-id-or-model-path REPO_ID_OR_MODEL_PATH`: argument defining the huggingface repo id for the Mixtral model (e.g. `mistralai/Mixtral-8x7B-Instruct-v0.1` and `DiscoResearch/mixtral-7b-8expert`) to be downloaded, or the path to the huggingface checkpoint folder. It is default to be `'mistralai/Mixtral-8x7B-Instruct-v0.1'`. For model `DiscoResearch/mixtral-7b-8expert`, you should input the path to the model folder in which `modeling_moe_mistral.py` has been replaced.
+- `--repo-id-or-model-path REPO_ID_OR_MODEL_PATH`: argument defining the huggingface repo id for the Mixtral model (e.g. `mistralai/Mixtral-8x7B-Instruct-v0.1`) to be downloaded, or the path to the huggingface checkpoint folder. It is default to be `'mistralai/Mixtral-8x7B-Instruct-v0.1'`.
 - `--prompt PROMPT`: argument defining the prompt to be infered (with integrated prompt format for chat). It is default to be `'What is AI?'`.
 - `--n-predict N_PREDICT`: argument defining the max number of tokens to predict. It is default to be `32`.
 
@@ -72,12 +55,3 @@ Inference time: xxxx s
 -------------------- Output --------------------
 [INST] What is AI? [/INST] AI, or Artificial Intelligence, refers to the development of computer systems that can perform tasks that would normally require human intelligence to accomplish. These tasks can include things
 ```
-
-#### [DiscoResearch/mixtral-7b-8expert](https://huggingface.co/DiscoResearch/mixtral-7b-8expert)
-```log
-Inference time: xxxx s
--------------------- Output --------------------
-[INST] What is AI? [/INST]
-
-[INST] Artificial Intelligence (AI) is the ability of a computer program or a machine to think and learn. It is also a field of
-```

python/llm/example/GPU/HF-Transformers-AutoModels/Model/mixtral/generate.py

@@ -29,9 +29,8 @@ MIXTRAL_PROMPT_FORMAT = """<s>[INST] {prompt} [/INST]"""
 if __name__ == '__main__':
     parser = argparse.ArgumentParser(description='Predict Tokens using `generate()` API for Mixtral model')
     parser.add_argument('--repo-id-or-model-path', type=str, default="'mistralai/Mixtral-8x7B-Instruct-v0.1'",
-                        help='The huggingface repo id for the Mixtral (e.g. `mistralai/Mixtral-8x7B-Instruct-v0.1` and `DiscoResearch/mixtral-7b-8expert`) to be downloaded,'
+                        help='The huggingface repo id for the Mixtral (e.g. `mistralai/Mixtral-8x7B-Instruct-v0.1`) to be downloaded,'
-                             ', or the path to the huggingface checkpoint folder. For model `DiscoResearch/mixtral-7b-8expert`, '
+                             ', or the path to the huggingface checkpoint folder.')
-                             'you should input the path to the model folder in which `modeling_moe_mistral.py` has been replaced.')
     parser.add_argument('--prompt', type=str, default="What is AI?",
                         help='Prompt to infer')
     parser.add_argument('--n-predict', type=int, default=32,

python/llm/example/GPU/PyTorch-Models/Model/mixtral/README.md

@@ -1,5 +1,5 @@
 # Mixtral
-In this directory, you will find examples on how you could use BigDL-LLM `optimize_model` API to accelerate Mixtral models. For illustration purposes, we utilize the [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) and [DiscoResearch/mixtral-7b-8expert](https://huggingface.co/DiscoResearch/mixtral-7b-8expert) as reference Mixtral models.
+In this directory, you will find examples on how you could use BigDL-LLM `optimize_model` API to accelerate Mixtral models. For illustration purposes, we utilize the [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) as a reference Mixtral model.
 
 ## Requirements
 To run these examples with BigDL-LLM on Intel GPUs, we have some recommended requirements for your machine, please refer to [here](../README.md#recommended-requirements) for more information.
@@ -24,30 +24,12 @@ pip install --pre --upgrade bigdl-llm[xpu] -f https://developer.intel.com/ipex-w
 pip install transformers==4.36.0
 ```
 
-### (Optional) 2. Download Model and Replace File
+### 2. Configures OneAPI environment variables
-To run [DiscoResearch/mixtral-7b-8expert](https://huggingface.co/DiscoResearch/mixtral-7b-8expert) model on Intel GPU, we have provided an updated version [DiscoResearch-mixtral-7b-8expert/modeling_moe_mistral.py](./DiscoResearch-mixtral-7b-8expert/modeling_moe_mistral.py) of `modeling_moe_mistral.py`.
-
-
-#### 2.1 Download Model
-You could use the following code to download [DiscoResearch/mixtral-7b-8expert](https://huggingface.co/DiscoResearch/mixtral-7b-8expert).
-
-```python
-from huggingface_hub import snapshot_download
-
-# for DiscoResearch/mixtral-7b-8expert
-model_path = snapshot_download(repo_id='DiscoResearch/mixtral-7b-8expert')
-print(f'DiscoResearch/mixtral-7b-8expert checkpoint is downloaded to {model_path}')
-```
-
-#### 2.2 Replace `modeling_moe_mistral.py`
-For `DiscoResearch/mixtral-7b-8expert`, you should replace the `modeling_moe_mistral.py` with [DiscoResearch-mixtral-7b-8expert/modeling_moe_mistral.py](./DiscoResearch-mixtral-7b-8expert/modeling_moe_mistral.py).
-
-### 3. Configures OneAPI environment variables
 ```bash
 source /opt/intel/oneapi/setvars.sh
 ```
 
-### 4. Run
+### 3. Run
 
 For optimal performance on Arc, it is recommended to set several environment variables.
 
@@ -62,7 +44,7 @@ python ./generate.py --prompt 'What is AI?'
 
 In the example, several arguments can be passed to satisfy your requirements:
 
-- `--repo-id-or-model-path REPO_ID_OR_MODEL_PATH`: argument defining the huggingface repo id for the Mixtral model (e.g. `mistralai/Mixtral-8x7B-Instruct-v0.1` and `DiscoResearch/mixtral-7b-8expert`) to be downloaded, or the path to the huggingface checkpoint folder. It is default to be `'mistralai/Mixtral-8x7B-Instruct-v0.1'`. For model `DiscoResearch/mixtral-7b-8expert`, you should input the path to the model folder in which `modeling_moe_mistral.py` has been replaced.
+- `--repo-id-or-model-path REPO_ID_OR_MODEL_PATH`: argument defining the huggingface repo id for the Mixtral model (e.g. `mistralai/Mixtral-8x7B-Instruct-v0.1`) to be downloaded, or the path to the huggingface checkpoint folder. It is default to be `'mistralai/Mixtral-8x7B-Instruct-v0.1'`.
 - `--prompt PROMPT`: argument defining the prompt to be infered (with integrated prompt format for chat). It is default to be `'What is AI?'`.
 - `--n-predict N_PREDICT`: argument defining the max number of tokens to predict. It is default to be `32`.
 
@@ -73,12 +55,3 @@ Inference time: xxxx s
 -------------------- Output --------------------
 [INST] What is AI? [/INST] AI, or Artificial Intelligence, refers to the development of computer systems that can perform tasks that would normally require human intelligence to accomplish. These tasks can include things
 ```
-
-#### [DiscoResearch/mixtral-7b-8expert](https://huggingface.co/DiscoResearch/mixtral-7b-8expert)
-```log
-Inference time: xxxx s
--------------------- Output --------------------
-[INST] What is AI? [/INST]
-
-[INST] Artificial Intelligence (AI) is the ability of a computer program or a machine to think and learn. It is also a field of
-```

python/llm/example/GPU/PyTorch-Models/Model/mixtral/generate.py

@@ -29,9 +29,8 @@ MIXTRAL_PROMPT_FORMAT = """<s>[INST] {prompt} [/INST]"""
 if __name__ == '__main__':
     parser = argparse.ArgumentParser(description='Predict Tokens using `generate()` API for Mixtral model')
     parser.add_argument('--repo-id-or-model-path', type=str, default="'mistralai/Mixtral-8x7B-Instruct-v0.1'",
-                        help='The huggingface repo id for the Mixtral (e.g. `mistralai/Mixtral-8x7B-Instruct-v0.1` and `DiscoResearch/mixtral-7b-8expert`) to be downloaded,'
+                        help='The huggingface repo id for the Mixtral (e.g. `mistralai/Mixtral-8x7B-Instruct-v0.1`) to be downloaded,'
-                             ', or the path to the huggingface checkpoint folder. For model `DiscoResearch/mixtral-7b-8expert`, '
+                             ', or the path to the huggingface checkpoint folder.')
-                             'you should input the path to the model folder in which `modeling_moe_mistral.py` has been replaced.')
     parser.add_argument('--prompt', type=str, default="What is AI?",
                         help='Prompt to infer')
     parser.add_argument('--n-predict', type=int, default=32,

python/llm/example/GPU/README.md

@@ -25,7 +25,7 @@ Step 1, please refer to our [driver installation](https://dgpu-docs.intel.com/dr
 > **Note**: IPEX 2.0.110+xpu requires Intel GPU Driver version is [Stable 647.21](https://dgpu-docs.intel.com/releases/stable_647_21_20230714.html).
 
 Step 2, you also need to download and install [Intel® oneAPI Base Toolkit](https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html). OneMKL and DPC++ compiler are needed, others are optional.
-> **Note**: IPEX 2.0.110+xpu requires Intel® oneAPI Base Toolkit's version >= 2023.2.0.
+> **Note**: IPEX 2.0.110+xpu requires Intel® oneAPI Base Toolkit's version == 2023.2.0.
 
 ## Best Known Configuration on Linux
 For better performance, it is recommended to set environment variables on Linux: