Peiyu Liu/刘沛羽I am a PhD student at the Gaoling School of Artificial Intelligence at Renmin University of China. My PhD advisor is Prof. Wayne Xin Zhao. Currently, my primary research interests include natural language processing and model compression. I have a MS in Software Engineering in Information School, advised by Wei Xu. |
|
PublicationsI'm interested in machine leanring, model compression and natural language processing. |
|
Enabling Lightweight Fine-tuning for Pre-trained Language Model Compression based on Matrix Product OperatorsPeiyu Liu*, Ze-Feng Gao*, Wayne Xin Zhao, Z.Y. Xie, Zhong-Yi Lu and Ji-Rong Wen ACL 2021 main conference, 2021 paper / code / slides / This paper presents a novel pre-trained language models (PLM) compression approach based on the matrix product operator (short as MPO) from quantum many-body physics. |
|
Parameter-Efficient Mixture-of-Experts Architecture for Pre-trained Language ModelsZe-Feng Gao*, Peiyu Liu*, Wayne Xin Zhao, Zhong-Yi Lu, Ji-Rong Wen COLING 2022, Oral Presentation, 2022, 2021 paper / code / This study proposes a novel approach to reduce the parameter count of the MoE model, by using a shared central tensor among experts, resulting in a 27.2-fold reduction compared to the Switch Transformer. |
|
WenLan: Bridging Vision and Language by Large-Scale Multi-Modal Pre-TrainingYuqi Huo, Manli Zhang, Guangzhen Liu, Haoyu Lu, Yizhao Gao, Guoxing Yang, Jingyuan Wen, Heng Zhang, Baogui Xu, Weihao Zheng, Zongzheng Xi, Yueqian Yang, Anwen Hu, Jinming Zhao, Ruichen Li, Yida Zhao, Liang Zhang, Yuqing Song, Xin Hong, Wanqing Cui, Danyang Hou, Yingyan Li, Junyi Li, Peiyu Liu, Zheng Gong, Chuhao Jin, Yuchong Sun, Shizhe Chen, Zhiwu Lu*, Zhicheng Dou, Qin Jin, Yanyan Lan, Wayne Xin Zhao, Ruihua Song*, Ji-Rong Wen* Preprint, 2021 arxiv / |
ZhihuBesides my publications, I also post some work on the Zhihu site. |
|
NeurIPS 2020 之预训练语言模型压缩zhihu 2021-03-25 link / |
|
Design and source code from Jon Barron's website |