The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data Only

Item #:

075280-3464

Details

DOI: https://doi.org/10.52202/075280-3464
Author(s): Ebtesam Almazrouei, Hamza Alobeidli, Alessandro Cappelli, Ruxandra Cojocaru, Daniel Hesslow, Julien Launay, Quentin Malartic, Baptiste Pannier, Guilherme Penedo
Pages: 79155-79172 (18 pages)
Format: PDF Paper Download
Conference: Advances in Neural Information Processing Systems 36
Date/Location: Held 10-16 December 2023, New Orleans, Louisiana, USA.
Series: Advances in Neural Information Processing Systems 36
Publisher: Neural Information Processing Systems Foundation, Inc. (NeurIPS)

DOI: https://doi.org/10.52202/075280-3464
Author(s): Ebtesam Almazrouei, Hamza Alobeidli, Alessandro Cappelli, Ruxandra Cojocaru, Daniel Hesslow, Julien Launay, Quentin Malartic, Baptiste Pannier, Guilherme Penedo
Pages: 79155-79172 (18 pages)
Format: PDF Paper Download
Conference: Advances in Neural Information Processing Systems 36
Date/Location: Held 10-16 December 2023, New Orleans, Louisiana, USA.
Series: Advances in Neural Information Processing Systems 36
Publisher: Neural Information Processing Systems Foundation, Inc. (NeurIPS)