intel/ipex-llm - Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, DeepSpeed, Axolotl, etc.
7a5dfa350b
* feat: quantized tensor support QuantizedTensor inherits from Tensor with supporting part of methods. The memory is allocated from native code, so it only matains a pointer `getNativeStorage`. Note, We don't support automatically `free` method. Anyone using this method should think about the memory lacking, especially for memory optimization. test: quantized tensor tests * fix: test case failed on spark 1.6 |
||
|---|---|---|