Hey Reader,

Open-source software dominates the world of "traditional" coding, but so far, closed-source companies like (the very confusingly named) OpenAI and Anthropic are leading the pack in terms of large language model quality. Today, we dig into Meta's attempt to change that with the open-source Llama model.

•••

Together with WebAI

​WebAI’s biggest release yet​

One Platform. Tailored to you
​webAI’s Navigator gives you the power to build and deploy models utilizing your own infrastructure.​
​​
​Powerful​
Run huge models like Llama 70B on your own devices.
​
​Effortless​
Customize models quickly and easily. No coding needed.
​​
Private​
Your data never leaves your devices.
​
​Learn More about webAI →​

•••

Here are all the details from Nyasha on Meta's latest open-source large language model...

In a major move, Meta has unveiled Llama 3.1, the largest-ever open-source AI model, positioning itself at the forefront of the AI race. Meta claims that Llama 3.1 surpasses the performance of industry giants like GPT-4 and Anthropic’s Claude 3.5 Sonnet, making a significant impact in the AI landscape. This release marks a key moment, showcasing Meta's commitment to AI technology and accessibility.

Llama 3.1 boasts an impressive 405 billion parameters, far exceeding its predecessors and competing models. Meta utilized over 16,000 of Nvidia’s high-end H100 GPUs to train this behemoth, emphasizing the sizable investment required for such advancements. While the exact cost remains undisclosed, the expenditure on Nvidia chips alone suggests a development cost in the hundreds of millions of dollars.

Meta’s decision to release Llama 3.1 as an open-source model, despite the hefty development costs, aligns with CEO Mark Zuckerberg’s vision. He likens this to the success of the Open Compute Project, which saved Meta billions by involving external companies in improving and standardizing data center designs. Zuckerberg believes that open-source AI models will outpace proprietary ones, drawing parallels to the rise of Linux as the dominant open-source operating system.

To facilitate the widespread adoption of Llama 3.1, Meta is collaborating with over two dozen companies, including tech giants like Microsoft, Amazon, Google, Nvidia, and Databricks. These partnerships aim to help developers deploy customized versions of the model, which is touted to be more cost-effective than OpenAI’s GPT-4o. By releasing the model weights, Meta allows companies to fine-tune the model according to their specific needs, enhancing its versatility and applicability.

Despite the advancements, Meta has been secretive about the data used to train Llama 3.1. Critics argue that this lack of transparency is a tactic to avoid impending copyright lawsuits. However, Meta confirms that synthetic data played a crucial role in training the model, which could serve as a "teacher" for smaller models, enhancing their efficiency and cost-effectiveness. This approach reflects the ongoing challenges and ethical considerations in AI development.

Llama 3.1 is integrated into Meta’s AI assistant, accessible through popular platforms like WhatsApp, Instagram, and Facebook. Initially available in the U.S., the assistant will soon support multiple languages, including French, German, Hindi, Italian, and Spanish. However, due to the high operational costs, users will be switched to a scaled-back version of the model after a certain number of prompts. This tiered access model reflects the balance between providing advanced AI capabilities and managing operational expenses.

One notable feature of Meta AI is “Imagine Me,” which uses a phone’s camera to scan a user’s face and insert their likeness into generated images. This aims to meet the growing demand for AI-generated media, despite concerns about blurring the lines between real and synthetic content. Additionally, Meta AI will soon be integrated into the Quest headset, enhancing its functionality by replacing the voice command interface and providing real-time information about the user’s surroundings.

Meta’s release of Llama 3.1 marks a milestone in the AI industry. The model’s size and performance, coupled with its open-source availability, position Meta as a possible leader in AI development. While the high costs and potential legal challenges pose significant obstacles, the company’s strategic partnerships and commitment suggest a promising future.

Until next time,

– Rob, Nyasha and the IWAI Team

​
​PS. Have you checked out webAI yet?

• Unlock the real value of AI with local, ownable, and customizable models that you control.​
• No overpaying for cloud. No data-scraping. No gatekeepers.

​Learn more about our friends at webAI.