ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

revise open webui quickstart and some indexes (#10715)

Browse source 
* update readme

* update openwebui readme and update index
This commit is contained in:
Shengsheng Huang 2024-04-09 22:44:03 +08:00 committed by GitHub
parent a0244527aa
commit 8924dbc3f9
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 71 additions and 40 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
docs/readthedocs/source/_templates/sidebar_quicklinks.html
View file

@ -29,13 +29,16 @@
<a href="doc/LLM/Quickstart/docker_windows_gpu.html">Install IPEX-LLM in Docker on Windows with Intel GPU</a>
</li>
<li>
<a href="doc/LLM/Quickstart/chatchat_quickstart.html">Run Langchain-Chatchat (RAG Application) on Intel GPU</a>
<a href="doc/LLM/Quickstart/chatchat_quickstart.html">Run Local RAG using Langchain-Chatchat on Intel GPU</a>
</li>
<li>
<a href="doc/LLM/Quickstart/webui_quickstart.html">Run Text Generation WebUI on Intel GPU</a>
</li>
<li>
<a href="doc/LLM/Quickstart/continue_quickstart.html">Run Code Copilot (Continue) in VSCode with Intel GPU</a>
<a href="doc/LLM/Quickstart/continue_quickstart.html">Run Coding Copilot (Continue) in VSCode with Intel GPU</a>
</li>
<li>
<a href="doc/LLM/Quickstart/open_webui_with_ollama_quickstart.html">Run Open WebUI with IPEX-LLM on Intel GPU</a>
</li>
<li>
<a href="doc/LLM/Quickstart/benchmark_quickstart.html">Run Performance Benchmarking with IPEX-LLM</a>

1
docs/readthedocs/source/_toc.yml
View file

@ -25,6 +25,7 @@ subtrees:
- file: doc/LLM/Quickstart/docker_windows_gpu
- file: doc/LLM/Quickstart/chatchat_quickstart
- file: doc/LLM/Quickstart/webui_quickstart
- file: doc/LLM/Quickstart/open_webui_with_ollama_quickstart
- file: doc/LLM/Quickstart/continue_quickstart
- file: doc/LLM/Quickstart/benchmark_quickstart
- file: doc/LLM/Quickstart/llama_cpp_quickstart

5
docs/readthedocs/source/doc/LLM/Quickstart/index.rst
View file

@ -13,9 +13,10 @@ This section includes efficient guide to show you how to:
* `Install IPEX-LLM on Windows with Intel GPU <./install_windows_gpu.html>`_
* `Install IPEX-LLM in Docker on Windows with Intel GPU <./docker_windows_gpu.html>`_
* `Run Performance Benchmarking with IPEX-LLM <./benchmark_quickstart.html>`_
* `Run Langchain-Chatchat (RAG Application) on Intel GPU <./chatchat_quickstart.html>`_
* `Run Local RAG using Langchain-Chatchat on Intel GPU <./chatchat_quickstart.html>`_
* `Run Text Generation WebUI on Intel GPU <./webui_quickstart.html>`_
* `Run Code Copilot (Continue) in VSCode with Intel GPU <./continue_quickstart.html>`_
* `Run Open WebUI on Intel GPU <./open_webui_with_ollama_quickstart.html>`_
* `Run Coding Copilot (Continue) in VSCode with Intel GPU <./continue_quickstart.html>`_
* `Run llama.cpp with IPEX-LLM on Intel GPU <./llama_cpp_quickstart.html>`_
* `Run Ollama with IPEX-LLM on Intel GPU <./ollama_quickstart.html>`_

98
docs/readthedocs/source/doc/LLM/Quickstart/open_webui_with_ollama_quickstart.md
View file

@ -1,56 +1,73 @@
# Run Open-Webui with Ollama on Linux with Intel GPU
# Run Open WebUI on Linux with Intel GPU
The [open-webui](https://github.com/open-webui/open-webui) provides a user friendly GUI for anyone to run LLM locally; by porting it to [ollama](https://github.com/ollama/ollama) integrated with [`ipex-llm`](https://github.com/intel-analytics/ipex-llm), users can now easily run LLM in [open-webui](https://github.com/open-webui/open-webui) on Intel **GPU** *(e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Max)*.
[Open WebUI](https://github.com/open-webui/open-webui) is a user friendly GUI for running LLM locally; by porting it to [`ipex-llm`](https://github.com/intel-analytics/ipex-llm), users can now easily run LLM in [Open WebUI](https://github.com/open-webui/open-webui) on Intel **GPU** *(e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Max)*.
See the demo of running Mistral:7B on Intel Arc A770 below.
<video src="https://llm-assets.readthedocs.io/en/latest/_images/open_webui_demo.mp4" width="100%" controls></video>
## Quickstart
This quickstart guide walks you through setting up and using the [open-webui](https://github.com/open-webui/open-webui) with Ollama.
This quickstart guide walks you through setting up and using [Open WebUI](https://github.com/open-webui/open-webui) with Ollama (using the C++ interface of [`ipex-llm`](https://github.com/intel-analytics/ipex-llm) as an accelerated backend).
### 1 Run Ollama on Linux with Intel GPU
To use the open-webui on Intel GPU, first ensure that you can run Ollama on Intel GPU. Follow the instructions on the [Run Ollama on Linux with Intel GPU](ollama_quickstart.md). Please keep ollama service running after completing the above steps.
Follow the instructions on the [Run Ollama on Linux with Intel GPU](ollama_quickstart.html) to install and run "Ollam Serve". Please ensure that the Ollama service continues to run while you're using the Open WebUI.
### 2 Install and Run Open-Webui
**Requirements**
#### Requirements
- Node.js (>= 20.10) or Bun (>= 1.0.21)
- Python (>= 3.11)
**Installation Steps**
#### Installation
```sh
# Optional: Use Hugging Face mirror for restricted areas
export HF_ENDPOINT=https://hf-mirror.com
1. Use `git` to clone the [open-webui repo](https://github.com/open-webui/open-webui.git) or download the open-webui source code zip from [this link](https://github.com/open-webui/open-webui/archive/refs/heads/main.zip) and unzip it to a directory, e.g. `~/open-webui`.
export no_proxy=localhost,127.0.0.1
2. Run below commands to install Open WebUI
```sh
cd ~/open-webui/
cp -RPp .env.example .env # Copy required .env file
git clone https://github.com/open-webui/open-webui.git
cd open-webui/
cp -RPp .env.example .env # Copy required .env file
# Build Frontend
npm i
npm run build
# Build Frontend
npm i
npm run build
# Serve Frontend with Backend
cd ./backend
pip install -r requirements.txt -U
```
# Serve Frontend with Backend
cd ./backend
pip install -r requirements.txt -U
bash start.sh
#### Start the service
```
export no_proxy=localhost,127.0.0.1
bash start.sh
```
```eval_rst
.. note::
If you have difficulty accessing the huggingface repositories, you may use a mirror, e.g. add `export HF_ENDPOINT=https://hf-mirror.com` before running `bash start.sh`.
```
open-webui will be accessible at http://localhost:8080/.
#### Access the WebUI
Upon successful launch, URLs to access the WebUI will be displayed in the terminal. Open the provided local URL in your browser to interact with the WebUI, e.g. http://localhost:8080/.
> For detailed information, visit the [open-webui official repository](https://github.com/open-webui/open-webui).
>
### 3. Using Open-Webui
#### Log-in and Pull Model
```eval_rst
.. note::
For detailed information about how to use Open WebUI, visit the README of `open-webui official repository <https://github.com/open-webui/open-webui>`_.
```
#### Log-in
If this is your first time using it, you need to register. After registering, log in with the registered account to access the interface.
@ -63,19 +80,30 @@ If this is your first time using it, you need to register. After registering, lo
<img src="https://llm-assets.readthedocs.io/en/latest/_images/open_webui_login.png" width="100%" />
</a>
Check your ollama service connection in `Settings`. The default Ollama Base URL is set to `https://localhost:11434`, you can also set your own url if you run Ollama service on another machine. Click this button to check if the Ollama service connection is functioning properly. If not, an alert will pop out as the below shows.
#### Configure `Ollama` service URL
Access the Ollama settings through **Settings -> Connections** in the menu. By default, the **Ollama Base URL** is preset to https://localhost:11434, as illustrated in the snapshot below. To verify the status of the Ollama service connection, click the **Refresh button** located next to the textbox. If the WebUI is unable to establish a connection with the Ollama server, you will see an error message stating, `WebUI could not connect to Ollama`.
<a href="https://llm-assets.readthedocs.io/en/latest/_images/open_webui_settings_0.png" target="_blank">
<img src="https://llm-assets.readthedocs.io/en/latest/_images/open_webui_settings_0.png" width="100%" />
</a>
If everything goes well, you will get a message as shown below.
If the connection is successful, you will see a message stating `Service Connection Verified`, as illustrated below.
<a href="https://llm-assets.readthedocs.io/en/latest/_images/open_webui_settings.png" target="_blank">
<img src="https://llm-assets.readthedocs.io/en/latest/_images/open_webui_settings.png" width="100%" />
</a>
Pull model in `Settings/Models`, click the download button and ollama will download the model you select automatically.
```eval_rst
.. note::
If you want to use an Ollama server hosted at a different URL, simply update the **Ollama Base URL** to the new URL and press the **Refresh** button to re-confirm the connection to Ollama.
```
#### Pull Model
Go to **Settings -> Models** in the menu, choose a model under **Pull a model from Ollama.com** using the drop-down menu, and then hit the **Download** button on the right. Ollama will automatically download the selected model for you.
<a href="https://llm-assets.readthedocs.io/en/latest/_images/open_webui_pull_models.png" target="_blank">
<img src="https://llm-assets.readthedocs.io/en/latest/_images/open_webui_pull_models.png" width="100%" />
@ -83,19 +111,17 @@ Pull model in `Settings/Models`, click the download button and ollama will downl
#### Chat with the Model
Start new conversations with **New chat**. Select a downloaded model here:
Start new conversations with **New chat** in the left-side menu.
<a href="https://llm-assets.readthedocs.io/en/latest/_images/open_webui_select_model.png" target="_blank">
<img src="https://llm-assets.readthedocs.io/en/latest/_images/open_webui_select_model.png" width="100%" />
</a>
On the right-side, choose a downloaded model from the **Select a model** drop-down menu at the top, input your questions into the **Send a Message** textbox at the bottom, and click the button on the right to get responses.
Enter prompts into the textbox at the bottom and press the send button to receive responses.
<a href="https://llm-assets.readthedocs.io/en/latest/_images/open_webui_select_model.png" target="_blank">
<img src="https://llm-assets.readthedocs.io/en/latest/_images/open_webui_select_model.png" width="100%" />
</a>
<a href="https://llm-assets.readthedocs.io/en/latest/_images/open_webui_chat_1.png" target="_blank">
<img src="https://llm-assets.readthedocs.io/en/latest/_images/open_webui_chat_1.png" width="100%" />
</a>
You can also drop files into the textbox for LLM to read.
<br/>
Additionally, you can drag and drop a document into the textbox, allowing the LLM to access its contents. The LLM will then generate answers based on the document provided.
<a href="https://llm-assets.readthedocs.io/en/latest/_images/open_webui_chat_2.png" target="_blank">
<img src="https://llm-assets.readthedocs.io/en/latest/_images/open_webui_chat_2.png" width="100%" />