GPAS: Accelerating Convergence of LLM Pretraining via Gradient-Preserving Activation Scaling

Item #:

085713-0450

Details

DOI: https://doi.org/10.52202/085713-0450
Author(s): Tianhao Chen, Xin Xu, Zijing Liu, Pengxiang Li, Xinyuan Song, AJAY JAISWAL, Fan Zhang, Jishan Hu, Yang Wang, Hao CHEN, Shizhe Diao, Shiwei Liu, Yu Li, Lu Yin, Can Yang
Pages: 14950-14975 (26 pages)
Format: PDF Paper Download
Conference: Advances in Neural Information Processing Systems 38
Date/Location: Held 2-7 December 2025, San Diego, California, USA and 30 November - 5 December 2025, Mexico City, Mexico.
Series: Advances in Neural Information Processing Systems 38
Publisher: Neural Information Processing Systems Foundation, Inc. (NeurIPS)

DOI: https://doi.org/10.52202/085713-0450
Author(s): Tianhao Chen, Xin Xu, Zijing Liu, Pengxiang Li, Xinyuan Song, AJAY JAISWAL, Fan Zhang, Jishan Hu, Yang Wang, Hao CHEN, Shizhe Diao, Shiwei Liu, Yu Li, Lu Yin, Can Yang
Pages: 14950-14975 (26 pages)
Format: PDF Paper Download
Conference: Advances in Neural Information Processing Systems 38
Date/Location: Held 2-7 December 2025, San Diego, California, USA and 30 November - 5 December 2025, Mexico City, Mexico.
Series: Advances in Neural Information Processing Systems 38
Publisher: Neural Information Processing Systems Foundation, Inc. (NeurIPS)