D-CPT Law: Domain-specific Continual Pre-Training Scaling Law for Large Language Models

Item #:

079017-2867

Details

DOI: https://doi.org/10.52202/079017-2867
Author(s): Zhiqi Bai, Feiyu Duan, Jie Fu, Jiaheng Liu, Yinghao Ma, Lin Qu, Xingwei Qu, Haoran Que, Wenbo Su, Xu Tan, Jiakai Wang, Jiamang Wang, Chenchen Zhang, Ge Zhang, Yuanxing Zhang, Bo Zheng
Pages: 90318-90354 (37 pages)
Format: PDF Paper Download
Conference: Advances in Neural Information Processing Systems 37
Date/Location: Held 10-15 December 2024, Vancouver, Canada.
Series: Advances in Neural Information Processing Systems 37
Publisher: Neural Information Processing Systems Foundation, Inc. (NeurIPS)

DOI: https://doi.org/10.52202/079017-2867
Author(s): Zhiqi Bai, Feiyu Duan, Jie Fu, Jiaheng Liu, Yinghao Ma, Lin Qu, Xingwei Qu, Haoran Que, Wenbo Su, Xu Tan, Jiakai Wang, Jiamang Wang, Chenchen Zhang, Ge Zhang, Yuanxing Zhang, Bo Zheng
Pages: 90318-90354 (37 pages)
Format: PDF Paper Download
Conference: Advances in Neural Information Processing Systems 37
Date/Location: Held 10-15 December 2024, Vancouver, Canada.
Series: Advances in Neural Information Processing Systems 37
Publisher: Neural Information Processing Systems Foundation, Inc. (NeurIPS)