BLOOM (language model)
Lua error in package.lua at line 80: module 'strict' not found.
BigScience Large Open-science Open-access Multilingual Language Model (BLOOM[1]) is a transformer-based large language model. It was created by over 1000 AI researchers to provide a free large language model for everyone who wants to try. Trained on around 366 billion tokens over March through July 2022, it is considered an alternative to OpenAI's GPT-3 with its 176 billion parameters. BLOOM uses a decoder-only transformer model architecture modified from Megatron-LM GPT-2.
The BLOOM project[2] was started by a co-founder of Hugging Face. Six main groups of people were involved, including HuggingFace's BigScience team, the Microsoft DeepSpeed team, the NVIDIA Megatron-LM team, the IDRIS/GENCI team, the PyTorch team, and the volunteers in the BigScience Engineering workgroup.[2] BLOOM was trained using data of 46 natural languages and 13 programming languages. In total, 1.6 TeraByte pre-processed text was converted into 350 billion unique tokens as BLOOM's training datasets.[3]
References
<templatestyles src="Reflist/styles.css" />
Cite error: Invalid <references> tag; parameter "group" is allowed only.
<references />, or <references group="..." />
