Yushun Zhang

This article explores optimization challenges in Transformers and the effectiveness of Adam over SGD.

Adam-mini reduces memory usage for training large language models while maintaining performance.

MoFO helps large language models retain knowledge during fine-tuning without losing performance.

Discover efficient algorithm performance under strict time limits.