Revert to use out-of-tree GPU driver (#11761)

* Revert to use out-of-tree GPU driver since the performance with out-of-tree driver is better than upsteam's

* add spaces

* add troubleshooting case

* update Troubleshooting

docs/mddocs/DockerGuides/docker_pytorch_inference_gpu.md

@@ -87,7 +87,7 @@ root@arda-arc12:/# sycl-ls
 > export USE_XETLA=OFF
 > 
 > # Enable immediate command lists mode for the Level Zero plugin. Improves performance on Intel Arc™ A-Series Graphics and Intel Data Center GPU Max Series; however, it depends on the Linux Kernel, and some Linux kernels may not necessarily provide acceleration.
-> # Recommended for use on Intel Arc™ A-Series Graphics and Intel Data Center GPU Max Series, but it depends on the Linux kernel, Non-i915 kernel drivers may cause performance regressions.
+> # Recommended for use on Intel Arc™ A-Series Graphics and Intel Data Center GPU Max Series, but it depends on the Linux kernel, Upstream i915 kernel drivers may cause performance regressions.
 > export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
 > 
 > # Controls persistent device compiled code cache. Set to '1' to turn on and '0' to turn off.

docs/mddocs/Overview/install_gpu.md

@@ -171,7 +171,7 @@ IPEX-LLM GPU support on Linux has been verified on:
 
       > **Tip**:
       >
-      > Please refer to our [driver installation](https://dgpu-docs.intel.com/driver/installation.html) for general purpose GPU capabilities.
+      > For client GPUs, such as the Intel® Arc™ A-series, please refer to [Client GPU Installation Guide](https://dgpu-docs.intel.com/driver/client/overview.html). For data center GPUs, including Intel® Data Center GPU Max Series and Intel® Data Center GPU Flex Series, please refer to our [Installation for Data Center GPU](https://dgpu-docs.intel.com/driver/installation.html) for general purpose GPU capabilities.
       >
       > See [release page](https://dgpu-docs.intel.com/releases/index.html) for latest version.
 
@@ -311,7 +311,7 @@ IPEX-LLM GPU support on Linux has been verified on:
 
       > **Tip**:
       >
-      >   Please refer to our [driver installation](https://dgpu-docs.intel.com/driver/installation.html) for general purpose GPU capabilities.
+      >   For client GPUs, such as the Intel® Arc™ A-series, please refer to [Client GPU Installation Guide](https://dgpu-docs.intel.com/driver/client/overview.html). For data center GPUs, including Intel® Data Center GPU Max Series and Intel® Data Center GPU Flex Series, please refer to our [Installation for Data Center GPU](https://dgpu-docs.intel.com/driver/installation.html) for general purpose GPU capabilities.
       >
       >   See [release page](https://dgpu-docs.intel.com/releases/index.html) for latest version.
 
@@ -623,3 +623,9 @@ The reason for such errors is that oneAPI has not been initialized properly befo
 * For oneAPI installed using APT or Offline Installer, make sure you execute `setvars.sh` of oneAPI Base Toolkit before running IPEX-LLM.
 * For PIP-installed oneAPI, activate your working environment and run ``echo $LD_LIBRARY_PATH`` to check if the installation path is properly configured for the environment. If the output does not contain oneAPI path (e.g. ``~/intel/oneapi/lib``), check [Prerequisites](#prerequisites-1) to re-install oneAPI with PIP installer.
 * Make sure you install matching versions of ipex-llm/pytorch/IPEX and oneAPI Base Toolkit. IPEX-LLM with PyTorch 2.1 should be used with oneAPI Base Toolkit version 2024.0. IPEX-LLM with PyTorch 2.0 should be used with oneAPI Base Toolkit version 2023.2.
+
+#### 2. `core dump` when running with GPU
+If encountered random  `core dump` when running with GPU, please remove out of tree driver.
+```
+sudo apt purge -y  intel-i915-dkms  intel-fw-gpu
+```

docs/mddocs/Quickstart/install_linux_gpu.md

@@ -34,13 +34,17 @@ IPEX-LLM currently supports the Ubuntu 20.04 operating system and later, and sup
 
     ```bash
     sudo apt-get update
+
+    # Install out-of-tree driver
     sudo apt-get -y install \
         gawk \
         dkms \
         linux-headers-$(uname -r) \
         libc6-dev
+    sudo apt install intel-i915-dkms intel-fw-gpu
 
-    sudo apt-get install -y gawk libc6-dev udev\
+    # Install Compute Runtime
+    sudo apt-get install -y udev \
         intel-opencl-icd intel-level-zero-gpu level-zero \
         intel-media-va-driver-non-free libmfx1 libmfxgen1 libvpl2 \
         libegl-mesa0 libegl1-mesa libegl1-mesa-dev libgbm1 libgl1-mesa-dev libgl1-mesa-dri \
@@ -82,13 +86,17 @@ IPEX-LLM currently supports the Ubuntu 20.04 operating system and later, and sup
 
     ```bash
     sudo apt-get update
+
+    # Install out-of-tree driver
     sudo apt-get -y install \
         gawk \
         dkms \
         linux-headers-$(uname -r) \
         libc6-dev
+    sudo apt install -y intel-i915-dkms intel-fw-gpu
 
-    sudo apt-get install -y gawk libc6-dev udev\
+    # Install Compute Runtime
+    sudo apt-get install -y udev \
         intel-opencl-icd intel-level-zero-gpu level-zero \
         intel-media-va-driver-non-free libmfx1 libmfxgen1 libvpl2 \
         libegl-mesa0 libegl1-mesa libegl1-mesa-dev libgbm1 libgl1-mesa-dev libgl1-mesa-dri \

docs/readthedocs/source/doc/LLM/Quickstart/install_linux_gpu.md

@@ -25,13 +25,17 @@ IPEX-LLM currently supports the Ubuntu 20.04 operating system and later, and sup
 
     ```bash
     sudo apt-get update
+
+    # Install out-of-tree driver
     sudo apt-get -y install \
         gawk \
         dkms \
         linux-headers-$(uname -r) \
         libc6-dev
+    sudo apt install intel-i915-dkms intel-fw-gpu
 
-    sudo apt-get install -y gawk libc6-dev udev\
+    # Install Compute Runtime
+    sudo apt-get install -y udev \
         intel-opencl-icd intel-level-zero-gpu level-zero \
         intel-media-va-driver-non-free libmfx1 libmfxgen1 libvpl2 \
         libegl-mesa0 libegl1-mesa libegl1-mesa-dev libgbm1 libgl1-mesa-dev libgl1-mesa-dri \
@@ -73,13 +77,17 @@ IPEX-LLM currently supports the Ubuntu 20.04 operating system and later, and sup
 
     ```bash
     sudo apt-get update
+
+    # Install out-of-tree driver
     sudo apt-get -y install \
         gawk \
         dkms \
         linux-headers-$(uname -r) \
         libc6-dev
+    sudo apt install intel-i915-dkms intel-fw-gpu
 
-    sudo apt-get install -y gawk libc6-dev udev\
+    # Install Compute Runtime
+    sudo apt-get install -y udev \
         intel-opencl-icd intel-level-zero-gpu level-zero \
         intel-media-va-driver-non-free libmfx1 libmfxgen1 libvpl2 \
         libegl-mesa0 libegl1-mesa libegl1-mesa-dev libgbm1 libgl1-mesa-dev libgl1-mesa-dri \