Gemma - New state-of-the-art open models by Google

Google has unveiled Gemma, a new suite of open models designed to empower developers and researchers to create AI responsibly. Gemma models, developed by Google DeepMind and other Google teams, are lightweight and state-of-the-art, drawing on the technology behind the Gemini models. These models come in two sizes, Gemma 2B and Gemma 7B, both available as pre-trained and instruction-tuned variants. Alongside these models, Google is releasing a Responsible Generative AI Toolkit and offering support across major frameworks like JAX, PyTorch, and TensorFlow. Gemma's integration extends to tools such as Hugging Face and NVIDIA NeMo, enabling easy adoption and deployment across various platforms, including laptops and Google Cloud services.

The Gemma models are designed to be both powerful and responsible, achieving top-tier performance while ensuring safety and reliability. Google has implemented measures to filter sensitive information from training data and has conducted extensive evaluations to align Gemma models with responsible AI practices. This includes manual red-teaming, automated adversarial testing, and reinforcement learning from human feedback (RLHF). To further aid developers in building safe AI applications, the Responsible Generative AI Toolkit offers resources for safety classification, debugging, and best practices based on Google's extensive experience in the field.

Gemma's flexible architecture allows for optimization across different frameworks, tools, and hardware, ensuring broad accessibility and industry-leading performance. With support for multiple AI hardware platforms, including NVIDIA GPUs and Google Cloud TPUs, Gemma models are optimized for a range of applications from summarization to retrieval-augmented generation (RAG). Google is also providing free access and credits for researchers and developers to encourage experimentation and innovation with Gemma, highlighting its commitment to the open community and the responsible development of AI technology.

Magic, a startup building AI software engineer coworkers just had a major breakthrough.

The recent investment of $100 million by former GitHub CEO Nat Friedman and Daniel Gross in Magic, an AI coding assistant developer, has sparked significant interest in the tech community. Unlike the semi-automated coding tools available, such as Microsoft’s GitHub Copilot, Magic aims to revolutionize the field by offering fully automated coding "coworkers." This ambitious goal is grounded in their development of a new large language model capable of handling an extensive amount of data.

Magic's innovation lies in its ability to process text inputs of up to 3.5 million words, vastly outperforming the capacity of existing models like Google’s Gemini LLM and even OpenAI’s GPT-4. This capability suggests a model with virtually an unlimited context window, which is a leap toward mimicking human information processing more closely. Such a technological advance could redefine the landscape of coding assistants by providing a more holistic and comprehensive tool for developers.

The substantial investment by Friedman and Gross in Magic reflects their belief in the startup’s potential to transform how coding is done. By pushing the boundaries of what AI can understand and process, Magic is positioned to offer solutions that go beyond the current limitations of coding assistants. This move could potentially set a new standard for AI's role in software development, making the work of programmers more efficient and intuitive.

OpenAI postmortem – Unexpected responses from ChatGPT

On February 20, 2024, a significant update on ChatGPT, aimed at enhancing the user experience inadvertently introduced a bug affecting the language processing capabilities of the model. Large Language Models (LLMs) like this one construct responses by selecting words through a stochastic process, relying heavily on the probabilities associated with various tokens. These tokens are essentially numerical values that correspond to specific pieces of language. However, due to the update, there was a flaw in the critical step where the model translates these numerical values into language tokens.

This bug manifested as a sort of "lost in translation" issue within the model's operation. Specifically, during the process of selecting numbers that map to tokens, the model began choosing slightly incorrect values. This minor deviation in number selection led to the generation of word sequences that did not logically connect, making the responses nonsensical. The root of the problem was traced back to inference kernels producing inaccurate results, an issue exacerbated when deployed across certain GPU configurations. This technical mishap disrupted the model's ability to accurately interpret and generate human-like language.

The team behind the model at OpenAI acted swiftly upon discovering the source of this discrepancy. A comprehensive fix was developed and deployed to address the bug, focusing on correcting the flawed step in the number-to-token translation process. Following the rollout of this fix, thorough testing confirmed that the issue had been effectively resolved. This swift action ensured that the integrity of the model's language processing capabilities was restored, maintaining the high standard of user experience expected from such advanced technology.

Why Google’s new AI Gemini accused of refusing to acknowledge the existence of white people

Gemini, Google's AI-powered chatbot, has sparked controversy among right-wing circles online, with accusations of bias against creating images of white people. Critics, including accounts like @EndWokeness and @WayOTWorld, argue that Gemini's diverse responses to prompts requesting images of historical figures and specific demographics are part of a broader agenda against white history and civilization. This backlash was amplified by comments from AI engineer Dbarghya Das and others on social media platforms, suggesting difficulties in getting Gemini to generate images of white individuals, and criticizing the AI for pushing back against requests perceived as reinforcing stereotypes.

Gemini's responses to various prompts, including those asking for images of "happy white people" or an "ideal nuclear family," have been designed to promote inclusivity and challenge harmful stereotypes, according to the bot. While some users claim that Gemini's refusal to generate images based on race alone is evidence of a "woke" agenda, tests conducted by the Daily Dot showed that the AI is capable of producing images featuring people of all races, including Caucasians. The bot's stance on not creating content that depicts stereotypes or promotes biased views has been met with mixed reactions, with some praising the effort to avoid discrimination and others calling for Google to be penalized for alleged misinformation.

The debate over Gemini's alleged bias highlights the ongoing challenges faced by AI developers in balancing the need for inclusivity with accusations of censorship and bias. While Google has made efforts to address past criticisms of AI perpetuating stereotypes and biases, the controversy surrounding Gemini suggests that achieving a consensus on the role of AI in reflecting and shaping societal values remains elusive. Critics argue that the bot's programming reflects a politically motivated agenda, while supporters view its responses as a step towards more responsible and equitable AI development.

Other stuff

• Reddit in AI content licensing deal with Google

• Are you Blacker than ChatGPT? Take this quiz to find out.

• Tinder Owner Signs ChatGPT Deal. Enjoy the AI Dating Tidal Wave

• Things I Don't Know About AI by Elad Gil

• How Bad Movie Dubbing Led to the Fake Biden Campaign Robocalls

• AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling

• Raising children on the eve of AI

• How much electricity does AI consume?

• AI hiring tools may be filtering out the best job applicants

• We Tested an AI Tutor for Kids. It Struggled With Basic Math.

Hope you enjoyed today's newsletter

⚡️ Join over 200,000 people using the Superpower ChatGPT extension on Chrome and Firefox.