Meta launched their latest Llama 3.1 405B language model (LLM) which is said to have higher performance than GPT-4o by OpenAi and Claude 3.5 Sonnet by Anthropic. It is also the largest open source LLM with a size of up to 405 billion trained using 16,000 NVIDIA H100 GPUs.
Simultaneously with the launch of Llama 3.1, Mark Zuckerberg predicted that Meta AI's virtual assistant will be the most widely used by the end of 20254. To achieve this dream, Meta is working with Microsoft, Amazon, Google, Nvidia, and Databricks to enable developers to use Llama 3.1 as the basis for artificial intelligence (AI) that they want to develop. Anyone can use Llama 3.1 but for those with over 700 million monthly users a license must be requested from Meta to obtain permission.
LLama 3.1 has a context window of up to 128,000 tokens which is equivalent to a 50 page book. More data is processed simultaneously for analysis tasks for example. It can also understand German, French, Italian, Portuguese, Hindi, Spanish and Thai in addition to English
Training is done with a data set of 15 trillion tokens or equivalent to 750 billion words. From where the data was taken to train Llama 3.1 was not revealed by Meta. Because it is also a multi-mode LLM, it can also answer math questions, identify images and videos and produce audio. However, this ability has not yet been given to the public.
In addition to Llama 3.1 405B, Meta also produces Llama 3.1 8B and Llama 3.170B which are smaller but also with a context window of 128,000 tokens.