ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

remove disco mixtral, update oneapi version (#9671)

Browse source
This commit is contained in:
binbin Deng 2023-12-13 23:24:59 +08:00 committed by GitHub
parent 1456d30765
commit 68a4be762f
7 changed files with 13 additions and 2758 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1345
python/llm/example/GPU/HF-Transformers-AutoModels/Model/mixtral/DiscoResearch-mixtral-7b-8expert/modeling_moe_mistral.py
View file

File diff suppressed because it is too large Load diff

34
python/llm/example/GPU/HF-Transformers-AutoModels/Model/mixtral/README.md
View file

@ -1,5 +1,5 @@
# Mixtral
In this directory, you will find examples on how you could apply BigDL-LLM INT4 optimizations on Mixtral models on [Intel GPUs](../README.md). For illustration purposes, we utilize the [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) and [DiscoResearch/mixtral-7b-8expert](https://huggingface.co/DiscoResearch/mixtral-7b-8expert) as reference Mixtral models.
In this directory, you will find examples on how you could apply BigDL-LLM INT4 optimizations on Mixtral models on [Intel GPUs](../README.md). For illustration purposes, we utilize the [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) as a reference Mixtral model.
## Requirements
To run these examples with BigDL-LLM on Intel GPUs, we have some recommended requirements for your machine, please refer to [here](../README.md#recommended-requirements) for more information.
@ -24,29 +24,12 @@ pip install --pre --upgrade bigdl-llm[xpu] -f https://developer.intel.com/ipex-w
pip install transformers==4.36.0
```
### (Optional) 2. Download Model and Replace File
To run [DiscoResearch/mixtral-7b-8expert](https://huggingface.co/DiscoResearch/mixtral-7b-8expert) model on Intel GPU, we have provided an updated version [DiscoResearch-mixtral-7b-8expert/modeling_moe_mistral.py](./DiscoResearch-mixtral-7b-8expert/modeling_moe_mistral.py) of `modeling_moe_mistral.py`.
#### 2.1 Download Model
You could use the following code to download [DiscoResearch/mixtral-7b-8expert](https://huggingface.co/DiscoResearch/mixtral-7b-8expert).
```python
from huggingface_hub import snapshot_download
# for DiscoResearch/mixtral-7b-8expert
model_path = snapshot_download(repo_id='DiscoResearch/mixtral-7b-8expert')
print(f'DiscoResearch/mixtral-7b-8expert checkpoint is downloaded to {model_path}')
```
#### 2.2 Replace `modeling_moe_mistral.py`
For `DiscoResearch/mixtral-7b-8expert`, you should replace the `modeling_moe_mistral.py` with [DiscoResearch-mixtral-7b-8expert/modeling_moe_mistral.py](./DiscoResearch-mixtral-7b-8expert/modeling_moe_mistral.py).
### 3. Configures OneAPI environment variables
### 2. Configures OneAPI environment variables
```bash
source /opt/intel/oneapi/setvars.sh
```
### 4. Run
### 3. Run
For optimal performance on Arc, it is recommended to set several environment variables.
@ -61,7 +44,7 @@ python ./generate.py --prompt 'What is AI?'
In the example, several arguments can be passed to satisfy your requirements:
- `--repo-id-or-model-path REPO_ID_OR_MODEL_PATH`: argument defining the huggingface repo id for the Mixtral model (e.g. `mistralai/Mixtral-8x7B-Instruct-v0.1` and `DiscoResearch/mixtral-7b-8expert`) to be downloaded, or the path to the huggingface checkpoint folder. It is default to be `'mistralai/Mixtral-8x7B-Instruct-v0.1'`. For model `DiscoResearch/mixtral-7b-8expert`, you should input the path to the model folder in which `modeling_moe_mistral.py` has been replaced.
- `--repo-id-or-model-path REPO_ID_OR_MODEL_PATH`: argument defining the huggingface repo id for the Mixtral model (e.g. `mistralai/Mixtral-8x7B-Instruct-v0.1`) to be downloaded, or the path to the huggingface checkpoint folder. It is default to be `'mistralai/Mixtral-8x7B-Instruct-v0.1'`.
- `--prompt PROMPT`: argument defining the prompt to be infered (with integrated prompt format for chat). It is default to be `'What is AI?'`.
- `--n-predict N_PREDICT`: argument defining the max number of tokens to predict. It is default to be `32`.
@ -72,12 +55,3 @@ Inference time: xxxx s
-------------------- Output --------------------
[INST] What is AI? [/INST] AI, or Artificial Intelligence, refers to the development of computer systems that can perform tasks that would normally require human intelligence to accomplish. These tasks can include things
```
#### [DiscoResearch/mixtral-7b-8expert](https://huggingface.co/DiscoResearch/mixtral-7b-8expert)
```log
Inference time: xxxx s
-------------------- Output --------------------
[INST] What is AI? [/INST]
[INST] Artificial Intelligence (AI) is the ability of a computer program or a machine to think and learn. It is also a field of
```

5
python/llm/example/GPU/HF-Transformers-AutoModels/Model/mixtral/generate.py
View file

@ -29,9 +29,8 @@ MIXTRAL_PROMPT_FORMAT = """<s>[INST] {prompt} [/INST]"""
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Predict Tokens using `generate()` API for Mixtral model')
parser.add_argument('--repo-id-or-model-path', type=str, default="'mistralai/Mixtral-8x7B-Instruct-v0.1'",
help='The huggingface repo id for the Mixtral (e.g. `mistralai/Mixtral-8x7B-Instruct-v0.1` and `DiscoResearch/mixtral-7b-8expert`) to be downloaded,'
', or the path to the huggingface checkpoint folder. For model `DiscoResearch/mixtral-7b-8expert`, '
'you should input the path to the model folder in which `modeling_moe_mistral.py` has been replaced.')
help='The huggingface repo id for the Mixtral (e.g. `mistralai/Mixtral-8x7B-Instruct-v0.1`) to be downloaded,'
', or the path to the huggingface checkpoint folder.')
parser.add_argument('--prompt', type=str, default="What is AI?",
help='Prompt to infer')
parser.add_argument('--n-predict', type=int, default=32,

1345
python/llm/example/GPU/PyTorch-Models/Model/mixtral/DiscoResearch-mixtral-7b-8expert/modeling_moe_mistral.py
View file

File diff suppressed because it is too large Load diff

35
python/llm/example/GPU/PyTorch-Models/Model/mixtral/README.md
View file

@ -1,5 +1,5 @@
# Mixtral
In this directory, you will find examples on how you could use BigDL-LLM `optimize_model` API to accelerate Mixtral models. For illustration purposes, we utilize the [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) and [DiscoResearch/mixtral-7b-8expert](https://huggingface.co/DiscoResearch/mixtral-7b-8expert) as reference Mixtral models.
In this directory, you will find examples on how you could use BigDL-LLM `optimize_model` API to accelerate Mixtral models. For illustration purposes, we utilize the [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) as a reference Mixtral model.
## Requirements
To run these examples with BigDL-LLM on Intel GPUs, we have some recommended requirements for your machine, please refer to [here](../README.md#recommended-requirements) for more information.
@ -24,30 +24,12 @@ pip install --pre --upgrade bigdl-llm[xpu] -f https://developer.intel.com/ipex-w
pip install transformers==4.36.0
```
### (Optional) 2. Download Model and Replace File
To run [DiscoResearch/mixtral-7b-8expert](https://huggingface.co/DiscoResearch/mixtral-7b-8expert) model on Intel GPU, we have provided an updated version [DiscoResearch-mixtral-7b-8expert/modeling_moe_mistral.py](./DiscoResearch-mixtral-7b-8expert/modeling_moe_mistral.py) of `modeling_moe_mistral.py`.
#### 2.1 Download Model
You could use the following code to download [DiscoResearch/mixtral-7b-8expert](https://huggingface.co/DiscoResearch/mixtral-7b-8expert).
```python
from huggingface_hub import snapshot_download
# for DiscoResearch/mixtral-7b-8expert
model_path = snapshot_download(repo_id='DiscoResearch/mixtral-7b-8expert')
print(f'DiscoResearch/mixtral-7b-8expert checkpoint is downloaded to {model_path}')
```
#### 2.2 Replace `modeling_moe_mistral.py`
For `DiscoResearch/mixtral-7b-8expert`, you should replace the `modeling_moe_mistral.py` with [DiscoResearch-mixtral-7b-8expert/modeling_moe_mistral.py](./DiscoResearch-mixtral-7b-8expert/modeling_moe_mistral.py).
### 3. Configures OneAPI environment variables
### 2. Configures OneAPI environment variables
```bash
source /opt/intel/oneapi/setvars.sh
```
### 4. Run
### 3. Run
For optimal performance on Arc, it is recommended to set several environment variables.
@ -62,7 +44,7 @@ python ./generate.py --prompt 'What is AI?'
In the example, several arguments can be passed to satisfy your requirements:
- `--repo-id-or-model-path REPO_ID_OR_MODEL_PATH`: argument defining the huggingface repo id for the Mixtral model (e.g. `mistralai/Mixtral-8x7B-Instruct-v0.1` and `DiscoResearch/mixtral-7b-8expert`) to be downloaded, or the path to the huggingface checkpoint folder. It is default to be `'mistralai/Mixtral-8x7B-Instruct-v0.1'`. For model `DiscoResearch/mixtral-7b-8expert`, you should input the path to the model folder in which `modeling_moe_mistral.py` has been replaced.
- `--repo-id-or-model-path REPO_ID_OR_MODEL_PATH`: argument defining the huggingface repo id for the Mixtral model (e.g. `mistralai/Mixtral-8x7B-Instruct-v0.1`) to be downloaded, or the path to the huggingface checkpoint folder. It is default to be `'mistralai/Mixtral-8x7B-Instruct-v0.1'`.
- `--prompt PROMPT`: argument defining the prompt to be infered (with integrated prompt format for chat). It is default to be `'What is AI?'`.
- `--n-predict N_PREDICT`: argument defining the max number of tokens to predict. It is default to be `32`.
@ -73,12 +55,3 @@ Inference time: xxxx s
-------------------- Output --------------------
[INST] What is AI? [/INST] AI, or Artificial Intelligence, refers to the development of computer systems that can perform tasks that would normally require human intelligence to accomplish. These tasks can include things
```
#### [DiscoResearch/mixtral-7b-8expert](https://huggingface.co/DiscoResearch/mixtral-7b-8expert)
```log
Inference time: xxxx s
-------------------- Output --------------------
[INST] What is AI? [/INST]
[INST] Artificial Intelligence (AI) is the ability of a computer program or a machine to think and learn. It is also a field of
```

5
python/llm/example/GPU/PyTorch-Models/Model/mixtral/generate.py
View file

@ -29,9 +29,8 @@ MIXTRAL_PROMPT_FORMAT = """<s>[INST] {prompt} [/INST]"""
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Predict Tokens using `generate()` API for Mixtral model')
parser.add_argument('--repo-id-or-model-path', type=str, default="'mistralai/Mixtral-8x7B-Instruct-v0.1'",
help='The huggingface repo id for the Mixtral (e.g. `mistralai/Mixtral-8x7B-Instruct-v0.1` and `DiscoResearch/mixtral-7b-8expert`) to be downloaded,'
', or the path to the huggingface checkpoint folder. For model `DiscoResearch/mixtral-7b-8expert`, '
'you should input the path to the model folder in which `modeling_moe_mistral.py` has been replaced.')
help='The huggingface repo id for the Mixtral (e.g. `mistralai/Mixtral-8x7B-Instruct-v0.1`) to be downloaded,'
', or the path to the huggingface checkpoint folder.')
parser.add_argument('--prompt', type=str, default="What is AI?",
help='Prompt to infer')
parser.add_argument('--n-predict', type=int, default=32,

2
python/llm/example/GPU/README.md
View file

@ -25,7 +25,7 @@ Step 1, please refer to our [driver installation](https://dgpu-docs.intel.com/dr
> **Note**: IPEX 2.0.110+xpu requires Intel GPU Driver version is [Stable 647.21](https://dgpu-docs.intel.com/releases/stable_647_21_20230714.html).
Step 2, you also need to download and install [Intel® oneAPI Base Toolkit](https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html). OneMKL and DPC++ compiler are needed, others are optional.
> **Note**: IPEX 2.0.110+xpu requires Intel® oneAPI Base Toolkit's version >= 2023.2.0.
> **Note**: IPEX 2.0.110+xpu requires Intel® oneAPI Base Toolkit's version == 2023.2.0.
## Best Known Configuration on Linux
For better performance, it is recommended to set environment variables on Linux: