intel/ipex-llm - Accelerate local LLM inference and finetuning on Intel XPUs
https://github.com/intel/ipex-llm/
7a5dfa350b
* feat: quantized tensor support QuantizedTensor inherits from Tensor with supporting part of methods. The memory is allocated from native code, so it only matains a pointer `getNativeStorage`. Note, We don't support automatically `free` method. Anyone using this method should think about the memory lacking, especially for memory optimization. test: quantized tensor tests * fix: test case failed on spark 1.6 |
||
|---|---|---|