How to Improve AI Models While Training Only 0.1% of Parameters
:::info Authors: (1) Yaqing Wang, Purdue University (wang5075@purdue.edu); (2) Sahaj Agarwal, Microsoft (sahagar@microsoft.com); (3) Subhabrata Mukherjee, Microsoft Research (submukhe@microsoft.com); (4) Xiaodong Liu, Microsoft Research (xiaodl@microsoft.com); (5) Jing Gao, Purdue University (jinggao@purdue.edu); (6) Ahmed Hassan Awadallah, Microsoft Research (hassanam@microsoft.com); (7) Jianfeng Gao, Microsoft Research (jfgao@microsoft.com). ::: Table of Links Abstract and 1. Introduction Background 2.1 Mixture-of-Experts … Read more