Ruiyang Xu   徐瑞阳

M.S. Student in CS

Shanghai Jiao Tong University (SJTU)

Minhang District, Shanghai, China

About

I am a graduate student in computer science at Shanghai Jiao Tong University, advised by Prof. Xie Chen.

My research focuses on cutting-edge Multimodal Large Language Models (MLLMs), specifically multimodal reasoning and understanding. As a Research Intern at Alibaba Qwen, I have contributed to Qwen3-Omni and Qwen3.5-Omni, focusing on detailed captioning and agents.

I have been involved in several research projects in multimodal reasoning, omni-modal perception, and audio understanding, some of which have appeared at ICML, ICLR, and NeurIPS.

In my spare time, I listen to Britpop and J-rock.

News

  • 2026.06 Released MMAE, the first comprehensive benchmark for instruction-based audio editing.
  • 2026.06 OmniAgent is now open-sourced — code, paper, and models all available.
  • 2026.05 OmniAgent introducing native active perception has been accepted to ICML 2026.
  • 2026.03 Released Qwen3.5-Omni, featuring superior performance in audio-visual detailed captioning.
  • 2026.03 Check out Omni-Cloze, a novel benchmark dedicated to multimodal detailed captioning.
  • 2026.01 Omni-Captioner accepted at ICLR 2026.
  • 2026.01 Join our Audio Reasoning Challenge at Interspeech 2026. Check the project page.
  • 2025.09 Qwen3-Omni is released. Check the blog.
  • 2025.09 MMAR benchmark is accepted at NeurIPS 2025.
  • 2025.05 SLAM-Omni is accepted to the ACL 2025 Findings.

Selected Publications

Native Active Perception as Reasoning for Omni-Modal Understanding
Zhenghao Xing*, Ruiyang Xu*, Yuxuan Wang*, Jinzheng He, Ziyang Ma, Qize Yang, Yunfei Chu, Jin Xu, Junyang Lin, Chi-Wing Fu, Pheng-Ann Heng
ICML 2026

* Equal contribution

Paper Code Models Figure
2026.06
The Interspeech 2026 Audio Reasoning Challenge: Evaluating Reasoning Process Quality for Audio Reasoning Models and Agents
Ziyang Ma, Ruiyang Xu, Yinghao Ma, Chao-Han Huck Yang, Bohan Li, Jaeyeon Kim, Jin Xu, Jinyu Li, Carlos Busso, Kai Yu, Eng Siong Chng, Xie Chen
Interspeech 2026
Paper Page
2026.02
Omni-Captioner: Data Pipeline, Models, and Benchmark for Omni Detailed Perception
Ziyang Ma*, Ruiyang Xu*, Zhenghao Xing*, Yunfei Chu, Yuxuan Wang, Jinzheng He, Jin Xu, Pheng-Ann Heng, Kai Yu, Junyang Lin, Eng Siong Chng, Xie Chen
ICLR 2026

* Equal contribution

Paper Code Benchmark
2025.10
MMAE: A Massive Multitask Audio Editing Benchmark
Ziyang Ma, Ruiqi Yan, Ruiyang Xu, Jie Fang, Zhikang Niu, Yi-Wen Chao, Wenming Tu, Tianrui Wang, Auden, Qi Chen, Wenxi Chen, Jiaying Chi, Yanru Huo, Zixuan Jiang, Xiquan Li, Yalin Li, Junxi Liu, Minghao Liu, Binghao Qiang, Yijia Shan, Zheshu Song, Tian Tan, Zixiang Wang, Zeyu Xie, Zhifei Xie, Xiaoyu Xing, Qixiang Xu, Chen Yang, Guanrou Yang, Shan Yang, Yifan Yang, Steve Yves, Haotian Zhang, Haina Zhu, Kai Yu, Liefeng Bo, Eng-Siong Chng, Xie Chen
arxiv
Paper Github Figure
2026.06
MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix
Ziyang Ma, Yinghao Ma, Yanqiao Zhu, Chen Yang, Yi-Wen Chao, Ruiyang Xu, Wenxi Chen, Yuanzhe Chen, Zhuo Chen, Jian Cong, Kai Li, Keliang Li, Siyou Li, Xinfeng Li, Xiquan Li, Zheng Lian, Yuzhe Liang, Minghao Liu, Zhikang Niu, Tianrui Wang, Yuping Wang, Yuxuan Wang, Yihao Wu, Guanrou Yang, Jianwei Yu, Ruibin Yuan, Zhisheng Zheng, Ziya Zhou, Haina Zhu, Wei Xue, Emmanouil Benetos, Kai Yu, Eng-Siong Chng, Xie Chen
NeurIPS 2025
Paper Github Figure
2025.05
SLAM-Omni: Timbre-Controllable Voice Interaction System with Single-Stage Training
Wenxi Chen, Ziyang Ma, Ruiqi Yan, Yuzhe Liang, Xiquan Li, Ruiyang Xu, Zhikang Niu, Yanqiao Zhu, Yifan Yang, Zhanxun Liu, Kai Yu, Yuxuan Hu, Jinyu Li, Yan Lu, Shujie Liu, Xie Chen
Findings of ACL 2025
Paper Anthology
2025.07
SLAM-LLM: A Modular, Open-Source Multimodal Large Language Model Framework and Best Practice for Speech, Language, Audio and Music Processing
Ziyang Ma, Guanrou Yang, Wenxi Chen, Zhifu Gao, Yexing Du, Xiquan Li, Zhisheng Zheng, Haina Zhu, Jianheng Zhuo, Zheshu Song, Ruiyang Xu, Tiranrui Wang, Yifan Yang, Yanqiao Zhu, Zhikang Niu, Liumeng Xue, Yinghao Ma, Ruibin Yuan, Shiliang Zhang, Kai Yu, Eng Siong Chng, Xie Chen
IEEE JSTSP
Paper Github
2026.01

Education

Shanghai Jiao Tong University
M.S. in Computer Science
2026 — Present
Shanghai Jiao Tong University
B.S. in Computer Science
2022 — 2026