perf(colab): simplify setup & speed up downloads (uv, hf-xet)
- use `uv pip -e` for faster, deterministic installs - enable hf-xet (HF_XET_HIGH_PERFORMANCE=1) for faster transfers - download model with `hf download` to /content/models/VibeVoice-1.5B (quiet, idempotent) - simplify flow: absolute paths, minimal cells, remove redundant steps - add binary T4 GPU check + clear runtime switch instructions - add “Open in Colab” badge; keep Risks & Limitations visible
This commit is contained in:
jonathanagustin
YaoyaoChang
parent
01a3005bd9
commit
12493567c4
1 changed files with 128 additions and 146 deletions
|
|
@ -2,166 +2,148 @@
|
|||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "AHLptWHtQmw-",
|
||||
"metadata": {
|
||||
"id": "AHLptWHtQmw-"
|
||||
"id": "view-in-github",
|
||||
"colab_type": "text"
|
||||
},
|
||||
"source": [
|
||||
"<a href=\"https://colab.research.google.com/github/microsoft/VibeVoice/blob/main/demo/VibeVoice_colab.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"# VibeVoice Colab — T4 Quickstart (1.5B)\n",
|
||||
"This page provides a quickstart guide to run VibeVoice on Colab with T4.\n",
|
||||
"\n",
|
||||
"The T4 GPU can only support the 1.5B model due to memory limitations. Please note that T4 can only use SDPA instead of flash_attention_2, which may result in unstable and lower audio quality. For the best TTS experience, we recommend trying the 7B model on a more powerful GPU."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "vzwhx5AtQ37g",
|
||||
"metadata": {
|
||||
"id": "vzwhx5AtQ37g"
|
||||
},
|
||||
"source": [
|
||||
"## Step 1: Use T4\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "ryxffqxlVbbP",
|
||||
"metadata": {
|
||||
"id": "ryxffqxlVbbP"
|
||||
},
|
||||
"source": [
|
||||
"Use T4 in Colab: go to Runtime → Change runtime type → Hardware accelerator: GPU → T4."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "Hek0yZKdVot_",
|
||||
"metadata": {
|
||||
"id": "Hek0yZKdVot_"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import torch\n",
|
||||
"print(torch.cuda.is_available())\n",
|
||||
"!nvidia-smi"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "S8D9WNSvWFwy",
|
||||
"metadata": {
|
||||
"id": "S8D9WNSvWFwy"
|
||||
},
|
||||
"source": [
|
||||
"## Step 2: Env Install"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "2xGbc7gKMD7A",
|
||||
"metadata": {
|
||||
"id": "2xGbc7gKMD7A"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!git clone https://github.com/microsoft/VibeVoice.git\n",
|
||||
"This notebook provides a quickstart guide to run VibeVoice on Colab with T4. The T4 GPU can only support the 1.5B model due to memory limitations. Please note that T4 can only use SDPA instead of flash_attention_2, which may result in unstable and lower audio quality. For the best TTS experience, we recommend trying the 7B model on a more powerful GPU.\n",
|
||||
"\n",
|
||||
"import os\n",
|
||||
"os.chdir(\"./VibeVoice\")\n",
|
||||
"## Risks and Limitations\n",
|
||||
"\n",
|
||||
"!apt update && apt install ffmpeg -y\n",
|
||||
"!pip install -e ."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "YmxjRFSFW4aE",
|
||||
"metadata": {
|
||||
"id": "YmxjRFSFW4aE"
|
||||
},
|
||||
"source": [
|
||||
"## Step 3: Run VibeVoice"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "MfQ0geOJQNS5",
|
||||
"metadata": {
|
||||
"id": "MfQ0geOJQNS5"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# First download checkpoint takes ~3 minutes\n",
|
||||
"!python demo/inference_from_file.py --model_path microsoft/VibeVoice-1.5B --txt_path demo/text_examples/2p_short.txt --speaker_names Alice Frank\n",
|
||||
"\n",
|
||||
"from IPython.display import Audio\n",
|
||||
"Audio(\"./outputs/2p_short_generated.wav\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "Pd6-KX2Hdswx",
|
||||
"metadata": {
|
||||
"id": "Pd6-KX2Hdswx"
|
||||
},
|
||||
"source": [
|
||||
"### TTS from your text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "ZB482MvXbg8M",
|
||||
"metadata": {
|
||||
"id": "ZB482MvXbg8M"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"text = \"\"\"Speaker 1: Can I try VibeVoice with my own example?\n",
|
||||
"Speaker 2: Of course! VibeVoice is open-source, built to benefit everyone — you’re welcome to try it out.\"\"\"\n",
|
||||
"with open(\"demo/text_examples/my_example.txt\", \"w\", encoding=\"utf-8\") as f:\n",
|
||||
" f.write(text)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "heoxL08yM-gf",
|
||||
"metadata": {
|
||||
"id": "heoxL08yM-gf"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!python demo/inference_from_file.py --model_path microsoft/VibeVoice-1.5B --txt_path demo/text_examples/my_example.txt --speaker_names Alice Frank\n",
|
||||
"Audio(\"./outputs/my_example_generated.wav\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "f09ff159",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Risk and Limitations"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "1e15d4e0",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"While efforts have been made to optimize it through various techniques, it may still produce outputs that are unexpected, biased, or inaccurate. VibeVoice inherits any biases, errors, or omissions produced by its base model (specifically, Qwen2.5 1.5b in this release). Potential for Deepfakes and Disinformation: High-quality synthetic speech can be misused to create convincing fake audio content for impersonation, fraud, or spreading disinformation. Users must ensure transcripts are reliable, check content accuracy, and avoid using generated content in misleading ways. Users are expected to use the generated content and to deploy the models in a lawful manner, in full compliance with all applicable laws and regulations in the relevant jurisdictions. It is best practice to disclose the use of AI when sharing AI-generated content."
|
||||
]
|
||||
],
|
||||
"metadata": {
|
||||
"id": "WvIaUJD2y0yU"
|
||||
},
|
||||
"id": "WvIaUJD2y0yU"
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Step 1: Setup Environment"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "e8fTKYGx7DZk"
|
||||
},
|
||||
"id": "e8fTKYGx7DZk"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"# Check for T4 GPU\n",
|
||||
"import torch\n",
|
||||
"if torch.cuda.is_available() and \"T4\" in torch.cuda.get_device_name(0):\n",
|
||||
" print(\"✅ T4 GPU detected\")\n",
|
||||
"else:\n",
|
||||
" print(\"\"\"\n",
|
||||
" ⚠️ WARNING: T4 GPU not detected\n",
|
||||
"\n",
|
||||
" The recommended runtime for this Colab notebook is \"T4 GPU\".\n",
|
||||
"\n",
|
||||
" To change the runtime type:\n",
|
||||
"\n",
|
||||
" 1. Click on \"Runtime\" in the top navigation menu\n",
|
||||
" 2. Click on \"Change runtime type\"\n",
|
||||
" 3. Select \"T4 GPU\"\n",
|
||||
" 4. Click \"OK\" if a \"Disconnect and delete runtime\" window appears\n",
|
||||
" 5. Click on \"Save\"\n",
|
||||
"\n",
|
||||
" \"\"\")\n",
|
||||
"\n",
|
||||
"# Clone the VibeVoice repository\n",
|
||||
"![ -d /content/VibeVoice ] || git clone --quiet --branch main --depth 1 https://github.com/microsoft/VibeVoice.git /content/VibeVoice\n",
|
||||
"print(\"✅ Cloned VibeVoice repository\")\n",
|
||||
"\n",
|
||||
"# Go to repo directory\n",
|
||||
"!cd /content/VibeVoice\n",
|
||||
"\n",
|
||||
"# Install project dependencies\n",
|
||||
"!uv pip --quiet install --system -e /content/VibeVoice\n",
|
||||
"print(\"✅ Installed dependencies\")\n",
|
||||
"\n",
|
||||
"# Download model (~3 minutes)\n",
|
||||
"!HF_XET_HIGH_PERFORMANCE=1 hf download microsoft/VibeVoice-1.5B --quiet --local-dir /content/models/VibeVoice-1.5B > /dev/null\n",
|
||||
"print(\"✅ Downloaded model: microsoft/VibeVoice-1.5B\")\n"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "4wxJ6QHM-ZOb"
|
||||
},
|
||||
"id": "4wxJ6QHM-ZOb",
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Step 2: Create Transcript"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "pgKlV7153Ifi"
|
||||
},
|
||||
"id": "pgKlV7153Ifi"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"%%writefile /content/my_transcript.txt\n",
|
||||
"Speaker 1: Can I try VibeVoice with my own example?\n",
|
||||
"Speaker 2: Of course! VibeVoice is open-source, built to benefit everyone - you're welcome to try it out.\n"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "Yc1N9EHswFxA"
|
||||
},
|
||||
"id": "Yc1N9EHswFxA",
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"## Step 3: Generate Audio"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "MBCC6s-F6_hP"
|
||||
},
|
||||
"id": "MBCC6s-F6_hP"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"# Run Python script to generate audio from transcript\n",
|
||||
"!python /content/VibeVoice/demo/inference_from_file.py \\\n",
|
||||
" --model_path /content/models/VibeVoice-1.5B \\\n",
|
||||
" --txt_path /content/my_transcript.txt \\\n",
|
||||
" --speaker_names Alice Frank\n",
|
||||
"\n",
|
||||
"# Display audio controls\n",
|
||||
"from IPython.display import Audio\n",
|
||||
"Audio(\"/content/outputs/my_transcript_generated.wav\")\n"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "dYWsLJ-n0Npm"
|
||||
},
|
||||
"id": "dYWsLJ-n0Npm",
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"accelerator": "GPU",
|
||||
"colab": {
|
||||
"gpuType": "T4",
|
||||
"provenance": []
|
||||
"provenance": [],
|
||||
"machine_shape": "hm",
|
||||
"name": "VibeVoice_Colab.ipynb",
|
||||
"include_colab_link": true
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
|
|
|
|||
Loading…
Reference in a new issue