New Claude AI can take over your computer

Anthropic has unveiled an upgraded version of its AI model, Claude 3.5 Sonnet, which can interact with desktop applications via a new "Computer Use" API, currently in open beta. This capability allows the model to emulate human actions like keystrokes, mouse clicks, and gestures, thus enabling it to use computer software directly. By leveraging this feature, developers can prompt Claude to perform tasks based on what it sees on a user's screen, positioning the model as an advanced tool for desktop-level automation. Notably, this new feature is accessible via Anthropic's API, Amazon Bedrock, and Google Cloud's Vertex AI platform.

Claude 3.5 Sonnet competes in an increasingly crowded AI agent market, where companies like OpenAI, Microsoft, and various startups are working on similar automation tools. However, Anthropic claims its model is particularly robust, outperforming other models on specific coding tasks and showing strong capabilities for managing multi-step operations. Despite these advancements, the model's performance is not flawless; during tests, it struggled with basic tasks like scrolling and handling short-lived notifications, and it completed only a portion of tasks successfully when applied to real-world scenarios like modifying flight reservations.

With the power to control desktop apps, Claude 3.5 Sonnet's release does raise concerns about safety and misuse. Anthropic acknowledges the risks and asserts that they have implemented measures to deter harmful use, such as not training the model on users' data and deploying classifiers to avoid high-risk actions. The company has also collaborated with safety institutes in the U.S. and U.K. to assess the model before its release and retains screenshots taken during its usage to monitor for abuse.

It’s time to change the way we build digital products

Keeping busy is easy. Driving business impact is what matters. Consistently align your people to the most strategic priorities, discover product opportunities from deep customer insights, and gain total visibility on execution with Airtable ProductCentral.

Get a first look at Airtable’s powerful new solution, built to answer the needs of modern product teams.

Google releases tech to watermark AI-generated text

Google has officially made its AI-generated text watermarking tool, SynthID Text, widely available. The technology, designed to help developers and businesses watermark and detect AI-generated content, is now accessible through the AI platform Hugging Face as well as Google’s Responsible GenAI Toolkit. SynthID Text was previously integrated with Google's Gemini models and has been positioned as a means to make generative AI outputs more identifiable, without sacrificing quality or accuracy.

SynthID Text works by embedding additional information into the token distribution of the output text. Generative AI models predict which token—a single character or word—comes next based on statistical likelihood. Google’s watermarking process involves modulating the probability scores associated with each token, which collectively form a unique watermark. This watermark can later be analyzed to determine whether a text was generated by AI or written independently.

Google claims the technology remains effective even if the generated text is modified, cropped, or paraphrased. However, there are limitations; SynthID struggles with shorter text passages and those that have been translated or rewritten in a different language. It also finds it challenging to work effectively with factual questions, where there is less room for adjusting probabilities without affecting accuracy.

Apple releases second wave of Intelligence features via new developer betas

Apple has begun releasing the second wave of its Apple Intelligence features with new developer betas for iOS 18.2, iPadOS 18.2, and macOS 15.2. The update includes generative AI tools such as Image Playground, Genmoji, and Image Wand, designed to create fun images in different styles like animation and illustration. With Genmoji, users can create custom emoji based on prompts or even faces from the Photos library. Image Playground allows users to generate themed images, and Image Wand enhances rough sketches into more detailed illustrations. Apple has implemented several safeguards to prevent the generation of inappropriate content, including prompt restrictions and user-reporting tools.

Apple Intelligence is also expanding its language processing capabilities. The new update adds more text-manipulation options to Writing Tools, letting users give custom prompts to adjust their writing. Additionally, Apple has introduced support for integrating ChatGPT, enabling Siri to dynamically pass complex queries to the AI, such as travel planning. Users can control their data privacy, and the feature does not store or use personal information by default. This integration will allow Siri to provide more conversational and versatile responses, bringing AI capabilities more in line with popular models like ChatGPT.

Visual Intelligence is another feature included in the latest developer betas, specifically for owners of the new iPhone 16 models. This tool allows the camera to identify objects, provide details, translate text, and even access more in-depth information via ChatGPT or Google search. The latest betas also expand Apple Intelligence's language support to include additional English dialects, such as those used in Canada, the UK, Australia, and South Africa, with more languages planned for 2025.

Related: Apple is ‘concerned’ about AI turning real photos into ‘fantasy’

Other stuff

• The mother of a 14-year-old Florida boy says he became obsessed with a chatbot on Character·AI before his death.

• Startup Hugging Face aims to cut AI costs with open source offering

• Create Professional-Grade Images with Stable Diffusion 3.5 AI

• Humane drops Ai Pin price by $200

• Marc Andreessen says AI model makers are in ‘a race to the bottom’ and it’s not good for business [Video]

Fact-based news without bias awaits. Make 1440 your choice today.

Overwhelmed by biased news? Cut through the clutter and get straight facts with your daily 1440 digest. From politics to sports, join millions who start their day informed.

Sign up now!

Hope you enjoyed today's newsletter

⚡️ Join over 200,000 people using the Superpower ChatGPT extension on Chrome and Firefox.