Avrajit Ghosh

I am a postdoctoral fellow at the Simons Institute for the Theory of Computing and BAIR at UC Berkeley EECS. I am affiliated with the Machine Learning Research Pod, where I am fortunate to be advised jointly by Bin Yu and Peter Bartlett. I am also fortunate to collaborate with Manfred Warmuth.

I graduated from the Computational Mathematics, Science and Engineering department at Michigan State University, where I was advised by Rongrong Wang and Saiprasad Ravishankar.

My research studies optimization for deep learning, with a focus on implicit bias of optimizers and hyperparameters and the statistical benefits and limitations of implicit regularization.

Short bio

Avrajit Ghosh is a postdoctoral fellow at the Simons Institute for the Theory of Computing and BAIR at UC Berkeley EECS. His research focuses on optimization for deep learning, implicit bias of optimizers and hyperparameters, and the statistical benefits and limitations of implicit regularization. He received his Ph.D. from the Computational Mathematics, Science and Engineering department at Michigan State University, advised by Rongrong Wang and Saiprasad Ravishankar.

Research focus: Implicit regularization, optimization dynamics, inverse problems.

News

Older News
  • [2025] Fitch H. Beach Award, recognizing the most outstanding graduate researcher within the College of Engineering, MSU.
  • [2025] One paper accepted at ICLR 2025.
  • [2024] One paper accepted at ICML 2024.
  • [2024] One paper accepted at TMLR.
  • [2023] One paper accepted at ICLR 2023 as a Spotlight.
  • [2023] Top reviewer at NeurIPS 2023.

Publications

* indicates equal contribution.

Optimization Dynamics in Deep Learning

Variational Learning Finds Flatter Solutions at the Edge of Stability

Avrajit Ghosh, Bai Cong, Rio Yokota, Saiprasad Ravishankar, Rongrong Wang, Molei Tao, Mohammad Emtiyaz Khan, Thomas Möllenhoff

NeurIPS 2025, Spotlight, Top 3.1%

Learning Dynamics of Deep Matrix Factorization Beyond the Edge of Stability

Avrajit Ghosh*, Soo Min Kwon*, Rongrong Wang, Saiprasad Ravishankar, Qing Qu

ICLR 2025

Seminar talk at MPI-MIS

Implicit Regularization and Generalization

Representative Publication

Hard Labels Sampled from Sparse Targets Mislead Rotation-Invariant Algorithms

Avrajit Ghosh, Bin Yu, Manfred Warmuth, Peter Bartlett

ICML 2026

Manfred's Simons talk | Slides

Representative Publication

Implicit Regularization in Heavy-ball Momentum Accelerated Stochastic Gradient Descent

Avrajit Ghosh*, He Lyu*, Xitong Zhang, Rongrong Wang

ICLR 2023, Spotlight, Top 5%

ICLR oral talk

PAC-Bayes Generalization Bounds for Score Based Diffusion Models

Avrajit Ghosh, Rongrong Wang

NeurIPS 2025 DynaFront Workshop

Improving Generalization of Complex Models with Unbounded Loss Using PAC-Bayes Bounds

Xitong Zhang, Avrajit Ghosh, Guangliang Wang, Rongrong Wang

TMLR 2024

Inverse Problems and Learned Regularizers

Optimal Eye Surgeon: Finding Image Priors through Sparse Generators at Initialization

Avrajit Ghosh, Xitong Zhang, Kenneth Sun, Qing Qu, Saiprasad Ravishankar, Rongrong Wang

ICML 2024

Learning Sparsity Promoting Regularizers using Bilevel Optimization

Avrajit Ghosh, Michael McCann, Madeline Mitchell, Saiprasad Ravishankar

SIAM Journal on Imaging Sciences, 2024

Understanding Untrained Deep Models for Inverse Problems: Algorithms and Theory

Avrajit Ghosh*, Ismail Alkhouri*, Evan Bell*, Shijun Liang, Rongrong Wang, Saiprasad Ravishankar

IEEE SPM Special Issue on the Mathematics of Deep Learning, 2025

Bilevel Learning of L1 Regularizers with Closed-Form Gradients

Avrajit Ghosh, Michael McCann, Saiprasad Ravishankar

ICASSP 2022

Optimized Parallel Combination of Deep Networks and Sparsity Regularization for MR Image Reconstruction (OPCoNS)

Avrajit Ghosh, Shijun Liang, Anish Lahiri, Saiprasad Ravishankar

ISMRM 2022

Selected Awards

Invited Talks, Tutorials, and Conference Orals