SLP-L10.3
I-LORA: AN ADAPTIVE RANK ALLOCATION APPROACH USING INTEGRATED GRADIENTS
Yunfei Li, Shanghai Jiao Tong University, China; Wenjin Yu, Shanghai United Imaging Medical Technology Cooperation, Limited, China; Ning Wen, Ruijin Hospital Shanghai Jiaotong University School of Medicine, China
Session:
SLP-L10: Efficient Learning and Inference for LLMs I Oral
Track:
Speech and Language Processing [SL]
Location:
Room 115
Presentation Time:
Wed, 6 May, 17:10 - 17:30
Presentation
Discussion
Resources
No resources available.
Session SLP-L10
SLP-L10.1: LOW RANK QUANTIZATION ADAPTATION FOR LARGE LANGUAGE MODEL
Xiwei Xu, Yuexiao Ma, Wenting Lin, Yuhang Wu, Yisheng Lin, Xiamen University, China; Zelan Yang, Wanchen Sui, Shen Li, Yong Li, Alibaba, China; Fei Chao, Xiawu Zheng, Rongrong Ji, Xiamen University, China
SLP-L10.2: TSQLORA: TOWARDS SENSITIVITY AND QUALITY LOW-RANK ADAPTATION FOR EFFICIENT FINE-TUNING
Yu Chen, South China University of Technology, China; Yifei Han, Long Zhang, Bin Li, Yue Du, Chinese Academy of Sciences, China
SLP-L10.3: I-LORA: AN ADAPTIVE RANK ALLOCATION APPROACH USING INTEGRATED GRADIENTS
Yunfei Li, Shanghai Jiao Tong University, China; Wenjin Yu, Shanghai United Imaging Medical Technology Cooperation, Limited, China; Ning Wen, Ruijin Hospital Shanghai Jiaotong University School of Medicine, China
SLP-L10.4: DIRA: Deep High-Rank Adaptation of Pre-trained Language Models
Shitong Cao, Dengtao Zhang, Xuejie Zhang, Jin Wang, Xiaobing Zhou, Yunnan University, China
SLP-L10.5: MIDAS: A Dynamic Cross-GPU KV Cache Offloading Framework For LLM On GPU Cluster Systems
Zixiao Zhang, Chaonong Xu, Zhibang Liu, Shaohui Zhi, Dan Ma, china university of petroleum(Beijing), China; Longxiang Yin, Chinese Academy of Sciences, China; Chao Li, Zhejiang Lab, China
SLP-L10.6: COMPRESSING KV CACHE FOR LONG-CONTEXT LLM INFERENCE WITH INTER-LAYER ATTENTION SIMILARITY
Da Ma, Lu Chen, Situo Zhang, Yuxun Miao, Shanghai Jiao Tong University, China; Su Zhu, Zhi Chen, ByteDance, China; Hongshen Xu, Hanqi Li, Shanghai Jiao Tong University, China; Shuai Fan, Lei Pan, AISpeech Co., Ltd., China; Kai Yu, Shanghai Jiao Tong University, China
Contacts