diff --git a/python/llm/example/pytorch-models/README.md b/python/llm/example/pytorch-models/README.md index ddbb5c21..004ac550 100644 --- a/python/llm/example/pytorch-models/README.md +++ b/python/llm/example/pytorch-models/README.md @@ -8,6 +8,7 @@ You can use `optimize_model` API to accelerate general PyTorch models on Intel s | ChatGLM | [link](chatglm) | | Openai Whisper | [link](openai-whisper) | | BERT | [link](bert) | +| Bark | [link](bark) | ## Recommended Requirements To run the examples, we recommend using Intel® Xeon® processors (server), or >= 12th Gen Intel® Core™ processor (client). diff --git a/python/llm/example/pytorch-models/bark/README.md b/python/llm/example/pytorch-models/bark/README.md new file mode 100644 index 00000000..4be37f75 --- /dev/null +++ b/python/llm/example/pytorch-models/bark/README.md @@ -0,0 +1,62 @@ +# Bark +In this directory, you will find examples on how you could use BigDL-LLM `optimize_model` API to accelerate Bark models. For illustration purposes, we utilize the [suno/bark](https://huggingface.co/suno/bark) as reference Bark models. + +## Requirements +To run these examples with BigDL-LLM, we have some recommended requirements for your machine, please refer to [here](../README.md#recommended-requirements) for more information. + +## Example: Synthesize speech with the given input text +In the example [synthesize_speech.py](./synthesize_speech.py), we show a basic use case for Bark model to synthesize speech based on the given text, with BigDL-LLM INT4 optimizations. +### 1. Install +We suggest using conda to manage the Python environment. For more information about conda installation, please refer to [here](https://docs.conda.io/en/latest/miniconda.html#). + +After installing conda, create a Python environment for BigDL-LLM: +```bash +conda create -n llm python=3.9 # recommend to use Python 3.9 +conda activate llm + +pip install --pre --upgrade bigdl-llm[all] # install the latest bigdl-llm nightly build with 'all' option +pip install TTS scipy +``` + +### 2. Download Bark model +Before running the example, you need to download Bark model to local folder: +```python +from huggingface_hub import snapshot_download + +model_path = snapshot_download(repo_id='suno/bark', + local_dir='bark/') # you can change `local_dir` parameter to specify any local folder +``` + +Please refer to [here](https://huggingface.co/docs/huggingface_hub/guides/download#download-files-to-local-folder) for more information about `snapshot_download`. + +### 3. Run +After setting up the Python environment and downloading Bark model, you could run the example by following steps. + +#### 3.1 Client +On client Windows machines, it is recommended to run directly with full utilization of all cores: +```powershell +# make sure `--model-path` corresponds to the local folder of downloaded model +python ./synthesize_speech.py --model-path 'bark/' --text "This is an example text for synthesize speech." +``` +More information about arguments can be found in [Arguments Info](#33-arguments-info) section. + +#### 3.2 Server +For optimal performance on server, it is recommended to set several environment variables (refer to [here](../README.md#best-known-configuration-on-linux) for more information), and run the example with all the physical cores of a single socket. + +E.g. on Linux, +```bash +# set BigDL-Nano env variables +source bigdl-nano-init + +# e.g. for a server with 48 cores per socket +export OMP_NUM_THREADS=48 +# make sure `--model-path` corresponds to the local folder of downloaded model +numactl -C 0-47 -m 0 python ./synthesize_speech.py --model-path 'bark/' --text "This is an example text for synthesize speech." +``` +More information about arguments can be found in [Arguments Info](#33-arguments-info) section. + +#### 3.3 Arguments Info +In the example, several arguments can be passed to satisfy your requirements: + +- `--model-path MODEL_PATH`: **required**, argument defining the local path to the Bark model checkpoint folder. +- `--text TEXT`: argument defining the text to synthesize speech. It is default to be `"This is an example text for synthesize speech."`. diff --git a/python/llm/example/pytorch-models/bark/synthesize_speech.py b/python/llm/example/pytorch-models/bark/synthesize_speech.py new file mode 100644 index 00000000..bb8a61b8 --- /dev/null +++ b/python/llm/example/pytorch-models/bark/synthesize_speech.py @@ -0,0 +1,53 @@ +# +# Copyright 2016 The BigDL Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import scipy +import time +import argparse + +from TTS.tts.configs.bark_config import BarkConfig +from TTS.tts.models.bark import Bark +from bigdl.llm import optimize_model + + +if __name__ == '__main__': + parser = argparse.ArgumentParser(description='Synthesize speech with the given input text using Bark model') + parser.add_argument('--model-path', type=str, required=True, + help='The local path to the Bark model checkpoint folder') + parser.add_argument('--text', type=str, default="This is an example text for synthesize speech.", + help='Text to synthesize speech') + + args = parser.parse_args() + model_path = args.model_path + + # Load model + config = BarkConfig() + model = Bark.init_from_config(config) + model.load_checkpoint(config, checkpoint_dir=model_path, eval=True) + + # With only one line to enable BigDL-LLM optimization on model + model = optimize_model(model) + + # Synthesize speech with the given input + text = args.text + st = time.time() + output_dict = model.synthesize(text, config, speaker_id="random", voice_dirs=None) # with random speaker + end = time.time() + print(f'Time cost: {end-st} s') + + # Save the speech as a .wav file using scipy + sampling_rate = model.config.sample_rate + scipy.io.wavfile.write("bark_out.wav", rate=sampling_rate, data=output_dict["wav"].squeeze())