Chaoya Jiang | 蒋超亚
My name is Chaoya Jiang, and I am currently a fourth-year Ph.D. candidate in the School of Software and Microelectronics at Peking University, under the mentorship of Professor Shikun Zhang, Director of the National Engineering Research Center for Software Engineering. I previously earned my master’s degree from the Wangxuan Institute of Computer Technology, Peking University, and my bachelor’s degree in Computer Science from Nanjing University.
In addition to my academic achievements, I have gained significant industrial research experience as a long-term research intern at Alibaba’s Tongyi Laboratory.
WeChat: jcy690091936
Email: jiangchaoya@pku.edu.cn
Research Interest
My research interests lie in:
- Efficient Training and Inference For General Multimodal Large Language Models
- Embodied Intelligence
- Multimodal Intelligent Agents.
News
- [2024/11] Won the First Prize of Beijing Science and Technology Progress Award.(北京市科技进步一等奖)
- [2024/11] Won Chinese Electronic Society-Tencent Doctoral Incentive Program Grant (HunYuan Large Language Model) (中国电子学会-腾讯博士科研激励计划)
- [2024/09] One paper was accpeted by EMNLP 2024.
- [2024/09] One paper was accpeted by NeurlPS2024.
- [2024/07] One paper was accpeted by ACM MM2024 (Oral, Top 4%).
- [2024/05] One paper was accpeted by ACL2024.
- [2024/04] Won National Natural Science Foundation of China Youth Fundamental Research Project (Ph.D. Student Category) (首届国家自然科学基金青年学生基础研究项目（博士研究生）)
- [2024/02] One paper was accpeted by CVPR2024.
Research Grants
- National Natural Science Foundation of China Youth Fundamental Research Project (Ph.D. Student Category) (首届国家自然科学基金博士研究生青年学生基础研究项目) – 2023
- Principal Investigator (PI)
- Research on Efficient and Robust Training and Inference Techniques for Multimodal Large Language Models (多模态大型语言模型高效鲁棒训练与推理技术研究)
- Funding: 300,000 RMB
- Chinese Electronic Society-Tencent Doctoral Incentive Program Grant (HunYuan Large Language Model) （中国电子学会-腾讯博士研究生科研激励计划（混元大模型专项））- 2024
- Principal Investigator (PI)
- Research on Key Technologies for Retrieval-Augmented Generation in Multimodal Large Language Models for Complex Domain Knowledge Applications （面向复杂领域知识应用的多模态大模型检索增强生成关键技术研究）
- Funding: 100,000 RMB
Award
- First Prize in the Beijing Science and Technology Progress Award. 2023
- Awardee of the Peking University President’s Scholarship. 2024
- Outstanding Research Intern at Alibaba Tongyi Laboratory. 2023
Academic Services
- Program Commitee (PC) Member: ACL 2023, MM 2023, AAAI 2024, ACL2024, NIPS 2024, CVPR 2024, EMNLP 2024, ICLR 2025, CVPR 2025
Preprint
- Collaborative Object- and Patch-Text Alignment for Accelerating Vision-Language Foundation Models. (Under Review)
- Shikun Zhang*, Chaoya Jiang*(Equal Contribution), Haiyang Xu, Ming Yan, Ji Zhang, Wei Ye.
- SymDPO: Boosting In-Context Learning of Large Multimodal Models with Symbol Demonstration Direct Preference Optimization. (Under Review)
- Hongrui Jia*, Chaoya Jiang*(Equal Contribution), Haiyang Xu, Wei Ye, Mengfan Dong, Ming Yan, Ji Zhang, Fei Huang, Shikun Zhang.
- Efficient Vision-and-Language Pre-training with Text-Relevant Image Patch Selection. (Under Review)
- Wei Ye*, Chaoya Jiang*(Equal Contribution), Haiyang Xu, Chenhao Ye, Chenliang Li, Ming Yan, Shikun Zhang, Songhang Huang, Fei Huang.
Selected Publications
- MaVEn: An Effective Multi-granularity Hybrid Visual Encoding Framework for Multimodal Large Language Model. Accepted by NIPS2024 (CCF-A)
- Chaoya Jiang*, Hongruijia*(Equal Contribution), Wei Ye, Haiyang Xu, Ming Yan, Ji Zhang, Shikun Zhang.
- Hal-Eval: A Universal and Fine-grained Hallucination Evaluation Framework for Large Vision Language Models. MM 2024 (Oral,Top 4%, CCF-A)
- Chaoya Jiang, Wei Ye, Mengfan Dong, Jia Hongrui, Haiyang Xu, Ming Yan, Ji Zhang, Shikun Zhang.
- Hallucination Augmented Contrastive Learning for Multimodal Large Language Model. CVPR 2024 (CCF-A)
- Chaoya Jiang, Haiyang Xu, Mengfan Dong, Jiaxing Chen, Wei Ye, Ming Yan, Qinghao Ye, Ji Zhang, Fei Huang, Shikun Zhang.
- TiMix: Text-aware Image Mixing for Effective Vision-Language Pretraining. AAAI 2024 (Oral, CCF-A )
- Chaoya Jiang, Wei Ye, Haiyang Xu, Qinghao Ye, Ming Yan, Ji Zhang, Shikun Zhang.
- BUS: Efficient and Effective Vision-language Pre-training with Bottom-Up Patch Summarization. ICCV 2023 (CCF-A)
- Chaoya Jiang, Haiyang Xu, Wei Ye, Qinghao Ye, Chenliang Li, Mingshi Yan, Bin Bi, Shikun Zhang, Fei Huang, Songfang Huang.
- COPA : Efficient Vision-Language Pre-training through Collaborative Object- and Patch-Text Alignment. MM 2023 (CCF-A)
- Chaoya Jiang, Haiyang Xu, Wei Ye, Qinghao Ye, Chenliang Li, Mingshi Yan, Bin Bi, Shikun Zhang, Ji Zhang, Feiyan Huang.
- Vision Language Pre-training by Contrastive Learning with Cross-Modal Similarity Regulation. ACL 2023 (Oral, CCF-A)
- Chaoya Jiang, Wei Ye, Haiyang Xu, Miang yan, Shikun Zhang, Jie Zhang, Fei Huang.
- Exploiting Pseudo Image Captions for Multimodal Summarization. ACL 2023 Findings
- Chaoya Jiang, Rui Xie, Wei Ye, Jinan Sun, Shikun Zhang.
- TRIPS:Efficient Vision-and-Language Pre-training with Text-relevant Patch Selection. EMNLP2022 (CCF-B)
- Chaoya Jiang, Haiyang Xu, Chenliang Li, Ming Yan, Wei Ye, Shikun Zhang, Bin Bi, Songfang Huang.
- Similarity Learning For Cover Song Identification Using Cross-Similarity Matrices of Multi-Level Deep Sequences. ICASSP2020 (Oral, CCF-B)
- Chaoya Jiang, Deshun Yang, Xiaoou Chen.
- Learn A Robust Representation For Cover Song Identification Via Aggregating Local And Global Music Temporal Context. ICME2020 (Oral, CCF-B)
- Enhancing In-Context Learning via Implicit Demonstration Augmentation. ACL 2024 (Oral, CCF-A)
- Xiaoling Zhou, Wei Ye, Yidong Wang, Chaoya Jiang, Z Lee, Rui Xie, Shikun Zhang.
- PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization. ICLR 2024 (Poster)
- Yidong Wang, Zhuohao Yu, Zhengran Zeng, Linyi Yang, Cunxiang Wang, Hao Chen, Chaoya Jiang, Rui Xie, Jindong Wang, Xingxu Xie, Wei Ye, Shi-Bo Zhang, Yue Zhang.
- Mibench: Evaluating multimodal large language models over multiple images. EMNLP2024 (CCF-B)
- Haowei Liu, Xi Zhang, Haiyang Xu, Yaya Shi, Chaoya Jiang, Ming Yan, Ji Zhang, Fei Huang, Chunfeng Yuan, Bing Li, Weiming Hu.
