ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
ipex-llm/python/llm/portable-zip/README-ui.md
Jin Qiao 6a128aee32 LLM: add ui for portable-zip (#9262)
2023-11-01 15:36:59 +08:00

1 KiB
Raw Blame History

BigDL-LLM Portable Zip with Web-UI For Windows: User Guide

Introduction

This portable zip includes everything you need to run an LLM with BigDL-LLM optimizations and chat with it in Web-UI. Please refer to How to use section to get started.

6B model running on an Intel 11-Gen Core PC (real-time screen capture)

Verified Models

  • ChatGLM2-6b

How to use

  1. Download the zip from link here.
  2. (Optional) You could also build the zip on your own. Run setup.bat --ui and it will generate the zip file.
  3. Unzip bigdl-llm.zip.
  4. Download the model to your computer.
  5. Go into the unzipped folder and double click chat-ui.bat. Input the path of the model (e.g. path\to\model, note that there's no slash at the end of the path). Press Enter and wait until it shows All service started. Visit 127.0.0.1:7860 in browser to chat.. Do NOT close the terminal window!
  6. Visit 127.0.0.1:7860 in your browser and enjoy chatting!
  7. If you want to stop the program, just close the terminal window.