Summary
Developed SYCL/DPCPP backend for llama.cpp referred from CUDA, achieving >10x performance gains on Intel GPU(Max, Flex, Arc) compared with OpenCL implementation. Co-work with the community owners and response to issues related to Intel. Published blog: Run LLMs on Intel GPUs Using llama.cpp https://medium.com/intel-analytics-software/run-llm-on-intel-gpus-using-llama-cpp-579d3595954e
Summary
Extending Hugging Face transformers APIs for Transformer-based models for collaborations with ecosystem Highly optimized hand written X86 assembly [kernels](https://github.com/intel/neural-speed/tree/main) for Intel hardware, targeting advanced compression algorithm especially for LLM. Develop [GPU kernels](https://github.com/intel/neural-speed/tree/xetla) for intel client GPU based on SYCL efficient low-level programming(ESIMD, "Explicit SIMD" SYCL extension)
Summary
Python package for SOTA low-bit LLM quantization. worked on ONNXRunTime backend and finally integrated by ONNX community.
Summary
Working on weight-only-quantization optimization for Intel Client GPU. enabled on Windows & Linux, achieving geomean >2x performance gains compared with normal F16 implementation.
Blogs:
Llama2 support on MTL
AI Framework Engineer