Google’s new technique gives LLMs infinite context

Adobe’s ‘Ethical’ Firefly AI Was Trained on Midjourney Images

In today’s email:

  • 🤯 DeepMind’s New AI Saw 15,000,000,000 Chess Boards!

  • 👨🏽‍⚕️ Generative AI is coming for healthcare, and not everyone’s thrilled

  • 🔮 ChatGPT Can Predict the Future when it Tells Stories Set in the Future About the Past

  • 🧰 9 new AI-powered tools and resources. Make sure to check the online version for the full list of tools.

Top News

Google researchers have developed a new method called Infini-attention, designed to extend the context window of large language models (LLMs) to potentially infinite lengths without additional memory costs. This breakthrough addresses the traditional quadratic complexity of transformers, where memory and computation needs increase exponentially with input size. By integrating a compressive memory module into the classic attention mechanism, Infini-attention allows for efficient handling of extended inputs, maintaining high performance on very long sequences while drastically reducing memory requirements.

Infini-attention has been tested on various benchmarks, demonstrating its superiority in managing long-context tasks by maintaining lower perplexity scores and reducing memory usage by up to 114 times compared to other models. This technique enables the handling of sequences as long as one million tokens and shows promising results in tests like passkey retrieval and text summarization, significantly outperforming existing long-context models.

The introduction of LLMs capable of processing infinite context opens new possibilities for custom applications, reducing the need for complex engineering solutions like fine-tuning and retrieval-augmented generation. This could simplify the development of AI applications, allowing for more straightforward incorporation of extensive data directly into prompts, thereby facilitating the creation of tailored responses and enhancing model adaptability without extensive additional training.

DeepMind’s New AI Saw 15,000,000,000 Chess Boards!

Scientists at Google DeepMind have developed a groundbreaking AI that challenges the traditional methods used in creating chess-playing systems like those used by grandmasters. Unlike previous iterations that relied on self-play and intricate search algorithms, this new AI adopts a radically simplified approach by learning directly from Stockfish, a leading chess engine. This AI, without engaging in any games from start to finish, studied 15 billion board states and the corresponding moves made by Stockfish, focusing solely on single moves rather than entire games. Impressively, despite its simplicity and relatively smaller size compared to giants like GPT-4, it achieves grandmaster-level play and can process moves at a significant speed even on low-cost hardware.

This project diverges from the aim of building a superior chess engine. Instead, the focus is on demonstrating that transformer-type neural networks can effectively learn and replicate the expertise of a chess master through observation alone. This AI not only mimics but also understands and generalizes the strategies to new situations, showing a profound shift in how AI can learn complex tasks. This method of learning, which eschews deep strategic calculations for direct pattern recognition from expert moves, points to a new way of teaching AI complex decision-making.

The implications of this research extend far beyond the realm of chess. By learning algorithms through observation, this technique paves the way for advancements in various fields, such as autonomous driving and advanced computing algorithms. Such capabilities hint at a future where AI can decipher and replicate complex algorithms from observing tasks, offering a more intuitive and accessible way to harness AI for practical and innovative applications. This could revolutionize how we develop AI systems, making them more efficient and adaptable across diverse domains.

Adobe Inc. introduced its AI image-generating software, Firefly, which was promoted as being trained primarily on Adobe Stock images, boasting hundreds of millions of licensed images. This move was intended to make Firefly a commercially safe alternative in the AI image generator market, differentiating itself from competitors like Midjourney, who use internet-scraped images. However, it was later revealed that Adobe also used AI-generated images from these competitors as part of Firefly's training data, a fact not initially disclosed in their public representations.

In response to the growing use of AI-generated content, Adobe allowed creators to submit images generated by AI into the Adobe Stock marketplace, which contributed to the training dataset of Firefly. About 5% of the images used to train Firefly came from other AI platforms, despite Adobe's assurance that all content undergoes rigorous moderation to ensure no intellectual property or recognizable logos are included. The inclusion of AI-generated content in Firefly's training set has led to internal debates within Adobe about the ethics and future use of such data.

Adobe's strategy to differentiate Firefly by using controlled, legally obtained data has been undermined by their reliance on AI-generated content, which is often cheaper and readily available. This practice has brought up ethical concerns about transparency and the true uniqueness of Firefly compared to other AI tools that use openly scraped data. Although Adobe has been criticized for this approach, the company continues to promote Firefly as an ethical and commercially safer option for generating images.

The recent Google Cloud event in Las Vegas focused heavily on generative AI, emphasizing its potential to revolutionize various aspects of business and technology. Google showcased enhancements to their Gemini large language model (LLM), aiming to boost productivity and demonstrate the model's capabilities through various demos. However, the presentations sometimes seemed overly simplistic and primarily confined within the Google ecosystem, potentially limiting broader applicability.

Critics argue that while the demonstrations were impressive, they didn't always necessitate AI, such as an e-commerce transaction that could have been completed manually. Moreover, the event seemed to gloss over the substantial challenges and complexities involved in implementing such advanced technologies within large organizations. Despite the potential of generative AI, real-world application remains a significant hurdle, particularly for companies that are not as digitally advanced.

The discussions around data—the fundamental ingredient for training effective AI models—highlighted the principle of "garbage in, garbage out." For many companies, the prerequisite of having organized, clean data is still a major barrier. Thus, while Google's tools aim to simplify data integration and engineering, organizations lagging in digital transformation may find these tools less beneficial.

Additionally, concerns about governance, security, and ethical use of AI continue to be significant, underscoring the need for careful consideration beyond the initial excitement of new AI capabilities.

Other stuff

Superpower ChatGPT now supports voice 🎉

Text-to-Speech and Speech-to-Text. Easily have a conversation with ChatGPT on your computer

Superpower ChatGPT Extension on Chrome


Superpower ChatGPT Extension on Firefox


Tools & LinkS
Editor's Pick ✨

Suno Explore - Suno’s dedicated listening experience

Wozlo Chess - GPT AI-powered Chess Coach

Whisper Web - ML-powered speech recognition directly in your browser

Evelyn - Open-source AI tutor for quizzes, mindmaps & flashcards

Dovetail Magic - Transform customer data into insights like magic

Chapter One AI - Resume analyses by AI - Search for photorealistic AI gen images of celebrities

Book Summaries - Explore chapter-level summaries of bestselling books

Password GPT - Guess the secret password

Enhance Your Workflow with Superpower ChatGPT Pro: Introducing the Right-Click Menu Feature!

Unclassified 🌀 

How did you like today’s newsletter?

Login or Subscribe to participate in polls.

Help share Superpower

⚡️ Be the Highlight of Someone's Day - Think a friend would enjoy this? Go ahead and forward it. They'll thank you for it!

Hope you enjoyed today's newsletter

Follow me on Twitter and Linkedin for more AI news and resources.

Did you know you can add Superpower Daily to your RSS feed

⚡️ Join over 200,000 people using the Superpower ChatGPT extension on Chrome and Firefox.

Superpower ChatGPT Extension on Chrome


Superpower ChatGPT Extension on Firefox