Router Upcycling: Leveraging Mixture-of-Routers in Mixture-of-Experts Upcycling

Junfeng Ran, Guangxiang Zhao, Yuhan Wu, Dawei Zhu, Longyun Wu, Yikai Zhao, Tong Yang, Lin Sun, Xiangzheng Zhang, Sujian Li

January, 2025