讲座题目：Accommodating LLM training over decentralized computational resources

讲座时间：2023年10月13日 16:00-17:30

讲座人：袁彬航助理教授

地点：H301

讲座内容：

Training algorithms for large language models are often communication heavy. As a result, these models are trained dominantly in a centralized environment such as data centers with fast network connections. This strong dependency on fast interconnections is becoming the limiting factor of further scaling for the data center setting and alternative decentralized infrastructures such as spot instances and geo-distributed volunteer computes. In this talk, I will discuss our research in communication-efficient distributed learning and our current effort in training foundation models in a decentralized way

讲座人简介：

Binhang YUAN is an Assistant Professor at the Department of Computer Science and Engineering (CSE), the Hong Kong University of Science and Technology (HKUST). He received his Ph.D. and master's degrees from Rice University and his bachelor's degree from Fudan University. Before