As Matthew McConaughey runs around an airport wishing he had used an AI agent to book his flight, we could be forgiven for assuming that the technology is now a well-understood and integral part of everyday life for most people.
Of course, that’s not true. In reality, it’s quite far from the truth, even for businesses, but that is not to say that things aren’t going that way. In the future, many of our decisions will be handled by agents, or at least heavily informed by them. But in areas where we want more control, we will likely retain decision-making power, following a programmatic approach or workflow.
When booking a vacation, for example, we could consider different countries, cities and situations, the cuisine, the weather and the culture. Or we could just get agents to put together some options based on our preferences. The third option would be a mix of the two, such as choosing where you want to go and allowing agents to choose and book activities — again, based on our preferences and hobbies.
The choice will come down to how much autonomy we want to retain. And it’s the same with AI systems. In this article, I’ll explain the move from one decision-making method to another and show why both are still valid and complementary.
How Models Think
Traditional models have limited knowledge. They are trained on a finite amount of data, which puts some restrictions on the kinds of problems they can solve. For example, the first public iteration of ChatGPT could only deliver answers to prompts based on the information available up to 2022, as its underlying LLM, GPT-3.5, was trained on data up to that point. This is also why you will often hear or read about models that have been fine-tuned for specific use cases, such as HR or legal, as these use cases require additional steps and data to tailor them to the use case.
These models can be used to underpin generative AI applications within an organisation. They can therefore broadly answer queries relating to the models that have been fine-tuned, such as general HR policies or legal practices, but cannot answer specific questions employees might have that relate to their circumstances or organisational data. To unlock this ability, we can create a workflow that enables models to access and retrieve data from proprietary systems and internal corpora of data. This is what is known as a Retrieval-augmented generation (RAG) system, but one that is programmatic, i.e., a human has built in the capabilities that enable the model to access systems outside of its training data to extend its capabilities.
With tool-augmented LLMs, it is therefore possible to create advanced workflows that are limited only by the modular elements developers choose to include, meaning the control logic is determined by a human.
An agentic system uses an LLM’s ability to reason to determine the steps required to complete a task or fulfill a request. The difference between this approach and programmatic workflows boils down to the amount of autonomy we want our AI systems to have.
Structured workflows involve a linear progression to deal with problems. In many instances, this is the most appropriate approach, as it guarantees that the system won’t go beyond what it has been programmed to do. However, as the reasoning capabilities of LLMs have continued to develop, it is now possible to hand the control logic of compound AI systems over to the LLMs themselves. This is thanks to “chain-of-thought” reasoning that enables an LLM to break down problems, e.g., prompts, into logical steps, taking a little longer to consider each stage so that a more reliable and accurate output is delivered.
Orchestrating Agentic Autonomy
Orchestration frameworks, such as LangGraph, are used to build in the stages of reasoning that agents use to complete their tasks. This can include a stage in which the agent considers its initial answer and determines whether to proceed with or change its output. Orchestration frameworks also build in the tools that agents can use to complete their tasks. A simple example might be providing a specific tool that retrieves live meteorological information. By giving the agent access to this tool, it can bring in this live information for the use case that is being served, such as booking activities on vacation. No one wants to be struggling through outdoor activities when there’s a storm raging, which is why an agent who can reason on the information they get from calling a weather tool is particularly useful.
These agents are built in line with a framework known as ReAct (‘Re’ for ‘reasoning’ and ‘Act’ for ‘acting’), which ensures the agent follows the Thought-Action-Observation reasoning paradigm. These stages mean the agent will first consider the tools it has at its disposal to return a satisfactory response, such as a weather app, then take action by retrieving the information, observing the situation from the raw data and then presenting the information in a user-friendly format following a further reasoning stage.
Agents are the new kid on the block, but it doesn’t mean they’re here to usurp traditional workflows. The two work rather well in conjunction. Workflows can hand off to an agent, or team of agents, to complete tasks, just as with the use case I outlined at the beginning of this article — and it’s no different in a business context. The reverse can also be true, with agents handing off to pre-defined workflows once a course of action has been suggested. This can then lead to further feedback loops to iterate and ensure accuracy and better decision making.
Ultimately, the amount of autonomy we build into a system will always be determined by the use case, which will, in turn, be determined by the exact amount of control that is required.