Publications and Manuscripts

(* denotes equal contribution)

2023

• Provably Efficient Model Training over Centralized and Decentralized Datasets
  Yucheng Lu
  [PhD Thesis]

• Coordinating Distributed Example Orders for Provably Accelerated Training
  A. Feder Cooper*, Wentao Guo*, Khiem Pham*, Tiancheng Yuan, Charlie Ruan, Yucheng Lu, Chris De Sa
  In Proceedings of the 36th Neural Information Processing Systems Conference (NeurIPS) 2023.
  [Proceedings][Arxiv]

• CocktailSGD: Fine-tuning Foundation Models over 500Mbps Networks
  Jue Wang*, Yucheng Lu*, Binhang Yuan, Beidi Chen, Percy Liang, Chris De Sa, Chris Ré, Ce Zhang
  In the Fortieth International Conference on Machine Learning (ICML) 2023.
  [Proceedings]

• STEP: Learning N:M Structured Sparsity Masks from Scratch with Precondition
  Yucheng Lu, Shivani Agrawal, Suvinay Subramanian, Oleg Rybakov, Chris De Sa, Amir Yazdanbakhsh
  In the Fortieth International Conference on Machine Learning (ICML) 2023.
  [Proceedings][Arxiv]

• Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam
  Yucheng Lu, Conglong Li, Minjia Zhang, Chris De Sa, Yuxiong He
  In the Eleventh International Conference on Learning Representations (ICLR) 2023.
  [Arxiv][Tutorial][Code]

• Decentralized Learning: Theoretical Optimality and Practical Improvements
  Yucheng Lu, Chris De Sa
  In the Journal of Machine Learning Research (JMLR) 2023.
  [JMLR]
  

2022

• GraB: Finding Provably Better Data Permutations than Random Reshuffling
  Yucheng Lu, Wentao Guo, Chris De Sa
  In the Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS) 2022.
  [Proceedings][Arxiv][Code]

• A General Analysis of Example-Selection for Stochastic Gradient Descent
  Yucheng Lu*, Si Yi Meng*, Chris De Sa
  In the Tenth International Conference on Learning Representations (ICLR) 2022. Spotlight
  [Proceedings][Code]

2021

• Hyperparameter Optimization is Deceiving Us, and How to Stop It
  A. Feder Cooper, Yucheng Lu, Jessica Zosa Forde, Chris De Sa
  In the Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS) 2021.
  [Proceedings][Arxiv][Code]

• Variance Reduced Training with Stratified Sampling for Forecasting Models
  Yucheng Lu, Youngsuk Park, Lifan Chen, Yuyang Wang, Chris De Sa, Dean Foster
  In the Thirty-eighth International Conference on Machine Learning (ICML) 2021.
  [Proceedings][Arxiv][Code]

• Optimal Complexity in Decentralized Training
  Yucheng Lu, Chris De Sa
  In the Thirty-eighth International Conference on Machine Learning (ICML) 2021. Outstanding Paper Award Honorable Mention
  [Proceedings][Arxiv][Errata][Media Coverage (Chinese)]
  

2020

• MixML: A Unified Analysis of Weakly Consistent Parallel Learning
  Yucheng Lu, Jack Nash, Chris De Sa
  Unpublished Manuscript
  [Arxiv]

• Adaptive Diffusion of Sensitive Information In Online Social Networks
  Xudong Wu, Luoyi Fu, Huan Long, Dali Yang, Yucheng Lu, Xinbing Wang, Guihai Chen
  In IEEE Transactions on Knowledge and Data Engineering (TKDE) 2020.
  [Paper]

• Moniqua: Modulo Quantized Communication in Decentralized SGD
  Yucheng Lu, Chris De Sa
  In the Thirty-seventh International Conference on Machine Learning (ICML) 2020.
  [Proceedings][Arxiv]