Anthropic has upgraded its Claude 3.5 Sonnet AI model, introducing a new feature called “computer use.” This capability allows the AI to perform tasks on a computer by simulating human actions like typing, clicking, and moving the cursor. Starting today, developers can access this feature in beta through the API.
In the official release post for the feature, Anthropic provides a detailed look at how it works. Sam Ringer, a researcher at Anthropic, demonstrates the tool by instructing Claude to complete a request form. The AI extracts relevant information from a spreadsheet and an online portal visible on the screen to accomplish the task.
The demonstration continues with the tool capturing screenshots of the information displayed on the screen. We also see it search an online portal, as instructed, to find details missing from the spreadsheet. In parallel, Claude opens the form and begins filling it out autonomously.
Anthropic highlights that Claude 3.5 Sonnet achieved a score of 14.9% in the screenshot-only category, significantly outperforming the next-best AI system’s 7.8%. Moreover, when given additional steps to complete tasks, Claude’s score improved to 22.0%.
Developers can leverage the computer use feature through the Anthropic API, Google’s Vertex AI, and Amazon Bedrock. Anthropic also points out that the updated Claude 3.5 Sonnet demonstrates notable improvements across several industry-leading benchmarks.
In addition, Anthropic has introduced a new smaller model, Claude 3.5 Haiku, which “improves across every skill set and even surpasses Claude 3 Opus.” The Claude 3.5 Haiku model will be available via the API later this month.
The idea of teaching general computer skills to an AI is certainly intriguing — it holds the potential to automate tasks and save significant time. What are your thoughts on Claude AI’s new computer use feature? Share your opinions in the comments below!
0 Comments