Anthropic has developed and released a new batch of enhanced artificial intelligence models designed to save keystrokes and headaches by autonomously performing tasks typically carried out by software application developers. Known for its family of AI models, Anthropic Claude is capable of natural language cognition, advanced reasoning including vision analysis, and, crucially here, its ability to “write” software code normally associated with command line functions.
Among Claude’s code generation pledges is the ability to start creating websites in HTML and CSS via its ability to turn images into structured JSON data, or debug complex code bases.
Anthropic’s chief science officer Jared Kaplan has explained how his firm’s new coder feature uses AI model functions to direct a software developer’s mouse within a given user interface. It then subsequently informs the user where and what to click, what to type for a defined development action, and in what order the tasks involved in a particular sequence should happen.
Essentially Agentic AI
As we move from deterministic AI functions that rely upon rules-based procedures (in human workflows, or indeed in developer coding actions) that rely upon a narrow set of values, inputs, information sources and data, we start to approach the world of non-deterministic AI agents that make use of large language models (LLMs) and generative AI to handle more unstructured complex tasks. That’s a lot of what is happening here i.e. this developer feature is an essentially agentic function capable of executing multi-step actions with little or no human intervention.
As AI itself now develops beyond the retrieval augmented generation (RAG) hype-cycle and into a new vortex, with almost every software vendor of any size at pains to develop an agentic or agent-based AI technology proposition, this is arguably a pretty significant move beyond intelligent document processing (IDP), robotic process automation (RPA) or “plain old” chatbots.
Again gravitating around code generation for web development, Anthropic has shown how its latest tools can code a website’s structure and functions at a comparatively basic level. It is also capable of dovetailing (presumably via open APIs) with services such as Google Search and Apple Maps to demonstrate how it might plan a travel outing of some kind for a user.
Claudes x3: Haiku, Sonnet & Opus
Anthropic offers developers three versions of Claude, each of which is offered at an incrementally higher price point depending on the power selected. The new developer feature known as “computer use” detailed here appears in the mid-tier Sonnet model and Haiku, the lightweight and lower end of the triumvirate.
Haiku is the fastest model that can execute lightweight actions. Sonnet is a combination of performance and speed for efficient, high-throughput tasks. Opus is the company’s highest-performing model, which can handle complex analysis, longer tasks with many steps and higher-order mathematics and coding tasks.
“We’re introducing ‘computer use’ as a groundbreaking new capability in public beta available on the API so that developers can direct Claude to use computers the way people do — by looking at a screen, moving a cursor, clicking buttons and typing text. Claude 3.5 Sonnet is the first frontier AI model to offer computer use in public beta. At this stage, it is still experimental — at times cumbersome and error-prone. We’re releasing computer use early for feedback from developers and expect the capability to improve rapidly over time,” notes Anthropic.
The company further notes that some actions that people perform effortlessly — such as scrolling, dragging and zooming — currently challenges for Claude, so the team encourages developers to begin exploration with low-risk tasks.
Because computer use may provide a new vector for more familiar threats such as spam, misinformation or fraud, Anthropic promises it is taking a proactive approach to promote its safe deployment. The team has developed “new classifiers” that can identify when computer use is being used and whether harm is occurring. As such, the Claude 3.5 Sonnet model was conducted by the U.S. AI Safety Institute (U.S. AISI) and the UK Safety Institute (UK AISI).
Consumer Capability Coming
Forthcoming capabilities from this technology are hoped to emerge for non-technical consumer use cases, and this is work that is being carried out inside the labs’ team division at Anthropic. This is an upgraded Claude 3.5 Sonnet, and the computer use beta on the Anthropic API is available to developers now.