Category: Techstrong Council

  • From Automation to Orchestration: The Key to Digital Success

    From Automation to Orchestration: The Key to Digital Success

    Digital transformation continues to be a key focus for many organizations, and this usually means the automation of processes and data.

    At its core, automation is meant to simplify and streamline business operations. However, if not implemented correctly, it can introduce complexity, risk and fragmentation. The proliferation of automation tools, combined with the rapid growth of interconnected systems has created a tangled web of interdependencies that are becoming increasingly difficult to manage.

    In fact, recent research has shown organizations now manage an average of 50 endpoints to execute tasks that are part of a process in their business (this is a 19% increase over the past five years). Without a fundamental shift in approach, organizations risk becoming overwhelmed by digital chaos – what’s more, 82% fear they could find themselves dealing with ‘Automation Armageddon’ if processes are left unchecked.

    Bringing Order to Automation Complexity

    As automation becomes more ingrained, organizations are facing new operational risks that threaten efficiency, compliance, and agility. One of the biggest challenges is the sheer volume of interconnected systems and workflows that businesses now rely on. When it comes to process complexity, more than three quarters of organizations say that a lack of control has resulted in increased risk that core business processes are not working anymore.

    At the same time, rising regulatory demands are making compliance more difficult. A lack of control has also increased business risk relating to compliance and many organizations lack the visibility to ensure compliance across all automated processes.

    As automation scales, it becomes harder to gain insights into what’s working, what’s redundant, and where bottlenecks exist. This lack of visibility makes it difficult for leaders to drive continuous improvement and ensure automation is delivering real value.

    AI: The Operationalizing and Scaling Challenge

    Despite AI being at a relatively early stage in terms of widespread adoption, AI is already revolutionizing industries, enhancing decision-making, improving customer experiences, and driving efficiencies across business processes. Many organizations are eager to expand their AI capabilities, integrating machine learning models, predictive analytics, and AI agents to uncover new opportunities.

    A significant challenge facing organizations is being able to operationalize and scale AI; 85% of organizations report such challenges, highlighting the difficulty of moving from isolated AI implementations to fully embedded AI-driven end-to-end processes. This fragmented approach makes it difficult for businesses to realize AI’s full value, as disconnected AI initiatives fail to work cohesively within larger business processes.

    Another hurdle is governance and compliance. As AI becomes more embedded in decision-making, organizations need clear oversight into how models are trained, deployed, and maintained. Transparency is crucial, particularly in regulated industries where AI-driven decisions must be explainable and auditable.

    To overcome these challenges, organizations must implement “guardrails” that combine deterministic workflows with the non-deterministic nature of AI capabilities. By clearly defining operational parameters, businesses can ensure AI operates autonomously while adhering to organizational policies and regulatory standards. This approach allows businesses to leverage AI’s adaptability and predictive power within structured, reliable processes without sacrificing compliance or control.

    How AI Can Be Integrated

    AI systems need to be seamlessly integrated into existing business processes to ensure they deliver real value. Without orchestration, AI often functions in isolation, requiring manual effort to synchronize with other process endpoints. This disconnect leads to inefficiencies and makes it impossible to “control” AI sufficiently. 

    Embedding AI within a structured orchestration framework provides clear governance and simplifies management – effectively balancing predictable, deterministic workflows with the adaptive, non-deterministic capabilities of AI, which are increasingly executed by one or more AI agents. Structured orchestration defines clear interactions between human and AI-driven tasks and delivers real-time visibility into operations. This reduces complexity, eliminates silos, and unlocks AI’s full potential as a strategic asset. 

    Process orchestration also enables organizations to build processes quicker with the help of AI, as modern orchestration tools can help teams in the process modelling phase. Furthermore, process orchestration allows organizations to gain insights into end-to-end processes in real-time and detect bottlenecks or other problems within an automated process. AI can then help to find solutions to solve the process issues,  enabling businesses to optimize operations, improve decision-making, and scale AI usage effectively. 

    To fully leverage AI, organizations must move beyond ad hoc implementations and toward a strategic, orchestrated approach. This way, they can maximize efficiency, maintain compliance, and ensure that AI contributes meaningfully to broader digital transformation efforts.

    Why Orchestration Helps Ensure AI and Automation Initiatives Play the Right Tune

    To regain control, organizations must embrace end-to-end process orchestration as a core part of their business and IT strategy. Process orchestration and automation allows organizations to tame complexity, operationalize AI, and accelerate transformation. Organizations can design, manage, and improve the processes that underpin their business, no matter what the processes entail or where they run. Additionally, it serves as a critical link between IT and business teams, aligning automation initiatives with real operational needs and long-term business goals.

    Automation Is Inevitable, But Chaos Isn’t

    Automation without orchestration leads to fragility, not resilience. Those organizations that prioritize process orchestration will be those who turn automation and AI into a competitive advantage and avoid the looming threat of digital chaos or ‘Automation Armageddon’.

  • Managing Day 2 Concerns for Agentic AI Architecture

    Managing Day 2 Concerns for Agentic AI Architecture

    We’re now in the fifth wave of computing. If that sounds dramatic, that’s because it is—and probably also because someone in marketing really wants to make “agentic AI” stick. For those keeping track at home, the first four waves were mainframes (your granddad’s computer), web (when we were all hopeful), cloud (when we were still kind of hopeful), and mobile (where that hope became a subscription to your attention span).

    And now? Agentic AI.

    Let’s get the buzzwords out of the way: Agentic AI is a fancy term for giving large language models (LLMs) persistent memory, looping logic, and callbacks to reach out to your APIs. It’s like giving a Roomba a graduate degree and asking it to run your IT department. These “agents” operate independently (ish), have goals (ish), and make decisions (ish) based on prompts, semantic search and however much context you can cram into a vector database.

    Sounds fantastic. Until you have to operate and maintain this stuff in production.

    Because Day 1 is always a party, demos run great on slide decks. Day 2 is when the ops team walks into the room, sees the cloud bill, and quietly walks back out.

    The Scale Problem (a.k.a. We Broke TPS)

    Let’s talk about scale. Traditional systems had thousands to millions of users. With mobile, we got billions. Agentic AI? Try trillions. Not just people, though. You’ll have agents helping users, agents helping agents and agents judging the work done by other agents. 

    Mainframe

    Web

    Cloud

    Mobile

    Agentic

    Users

    thousands

    millions

    100 million

    billions

    trillions

    TPS

    100

    500

    2,500

    10,000

    1,000,000

    5x

    5x

    4x

    100x

    Each one of those agents runs loops. Each loop makes calls. Each call hits an LLM. Every LLM call contributes to Transactions Per Second (TPS). Not all transactions or LLM interactions are the same. Humans have very different behavior than agents.

    The types of interaction behavior we might see:

    • Human users = finite and sleepy
    • Agents = infinite and caffeinated
    • LLM inference = expensive, slow, and shockingly unreliable
    • Therefore: system load = Who knows? How do we even predict this stuff?

    It isn’t unusual for pre-agentic systems to max out around 10,000 TPS. Agentic systems? You’re looking at a million TPS just to hit parity with your company’s Slack usage.

    But What About Cost?

    Ah yes, the CFO’s favorite section.

    Running agents is expensive. Running smart agents is very expensive. Running smart agents 24/7 with high-accuracy LLMs? Welcome to a world where your annual AI ops bill might be an order of magnitude higher than it was before.

    Let’s put that in relatable terms:

    • GPT-4 32K (86% accuracy): $98 per million tokens
    • Open source model (42% accuracy): $0.013 per million tokens
    • Database transaction: pocket change by comparison

    And don’t let the open-source optimism fool you. A 42% accurate agent might write emails like “Dear Sir/Madman, I have eaten the spreadsheet,” which is cute until it’s running your treasury. Remember that hallucinations and bad responses are so common, we already have patterns for using multiple other LLMs just to judge the work of the first. This means that your cost isn’t just the cost to ask an LLM an innocent question, it’s the cost to resolve tools, cost to make the actual request and cost to make potentially multiple judgment requests.

    So now you’re faced with the AI version of the eternal tradeoff: dumb and cheap vs. smart and broke.

    Latency, Reliability and Other Sad Stories

    LLMs are not built for low-latency, high-throughput work. They’re built for… storytelling. And hallucinating. And sometimes – if you ask nicely and are an expert in prompt engineering – even answering your questions.

    Now, imagine your core systems relying on that. Imagine your uptime metrics depending on the whims of an LLM inference API that thinks “return customer data” means “find a retail location where customers can bring and return their data”

    Agentic systems aren’t transactional. They’re conversational. They’re unpredictable. They can fail to give you an answer, they can stop answering in the middle of a stream. Don’t listen to what people tell you, you can’t blindly trust an LLM.

    And yet, we keep trying to run them like traditional software. We build orchestration platforms, routing layers, retry loops, and cost governors. Why? Because otherwise, these things will try to eat the entire internet in a single for-loop and send us the cloud bill.

    Capacity Planning Is Now Chaos Planning

    You used to think about load in terms of peak usage. Now? Your agents don’t sleep. They don’t go to lunch. They just keep going, looping away, slamming vector DBs and semantic search engines with relentless, existential enthusiasm. Your system lows are higher, your median times are higher, and your peaks are higher.

    This means:

    • Your database load goes up 100x
    • Your network saturates under constant agent chatter
    • Your infrastructure team turns into a stress-testing experiment
    • And you still get worse answers than just asking the intern

    You don’t plan for “peak load” anymore. You plan for “non-stop Reddit debates between bots.”

    The Blueprint (Such as It Is)

    In theory, building a sustainable agentic architecture looks like this:

    1. Semantic Search – All your domain knowledge gets shoved into a vector DB, ready to be retrieved by semantic similarity.
    2. Agentic Sessions – Every agent interaction is logged to provide context, continuity and accountability.
    3. LLM Invocation Loops – Prompt, retrieve, augment, call LLM, process, repeat until you get a satisfactory answer, or you run out of usage quota.
    4. Orchestration Layer – The puppet master that tells agents what to do, when, and for how long before they bankrupt you.

    This is all manageable, assuming you can hire a team of distributed systems engineers who also speak fluent “prompt engineering,” understand vector math, and have no plans for sleep.

    What Ops Teams Need to Do (Besides Cry)

    If you’re serious about building something in this space—and not just gluing ChatGPT to your CI pipeline, you need to:

    • Plan for real-time TPS: Millions of them, all at once, with unpredictable spikes.
    • Plan for augmentation (RAG): Simple one-off queries are rare. Instead, your agents will be making multiple round trips to the LLM to resolve tools and multiple passes at a vector database for context augmentation.
    • Instrument your agents: Know which model they’re using, how often, and at what cost.
    • Design for failure: Assume models will hallucinate, APIs will fail and your agents will suddenly think they’re writing sci-fi.
    • Stay modular: You’ll want to swap models faster than your CFO can say “cost optimization.”

    Final Thought: The AI Agency Gap

    The AI Agency Gap is the chasm between dumb bots and actual autonomous systems. It’s the space where most agents will live—semi-capable, semi-stable, semi-helpful. They’re the interns of the software world: energetic, inconsistent and always one bad prompt away from disaster.

    If you want to thrive in this new wave of computing, you’re going to need resilience, observability, guard rails, patience and some really good stress coping techniques. The future isn’t just about building agentic systems, it’s about knowing that you’ve made all of your microservices unpredictable and require 10x more resources.

    Good luck.

  • Hybrid Infrastructure Is a Mess—But It Doesn’t Have to Be

    Hybrid Infrastructure Is a Mess—But It Doesn’t Have to Be

    If you’re a CIO who is managing hybrid infrastructure, you’re like the peace-keeper in a dysfunctional family. On one side, you’ve got the old-school on-prem team—practical, cost-conscious, maybe a little stubborn. On the other hand, the cloud team is fast-moving, quickly adopting the latest tools, and sometimes a little dismissive of “legacy” infrastructure. And just like a family that doesn’t get along, they barely talk to each other.

    I see this all the time. A CIO will tell me, “I need to unify my infrastructure strategy,” but when I speak with their teams, it’s like talking to two different companies. The cloud engineers don’t know (or care) what the on-prem team does, and the on-prem team sees the cloud people as reckless spenders who don’t understand “real” infrastructure.

    Hybrid Isn’t Going Away—So We’d Better Get It Right

    The thing is, both sides have a point. On-prem infrastructure is more predictable in cost, often more secure, and sometimes just makes sense for certain workloads. And the flexibility and scalability of cloud infrastructure are real.

    This means hybrid is the future. But right now, it’s a future filled with inefficiency, wasted budgets and unnecessary complexity.

    The Reality: Two Worlds That Don’t Talk

    Here’s a real example. I was on a call with a large financial institution, and the goal was to show how to manage both on-prem resources with Ansible and cloud infrastructure with Terraform. But the on-prem person didn’t show up for some reason. It was a wasted meeting because the cloud team hadn’t a clue what the on-prem team did, so talk of cohesion between the two siloed departments was impossible.

    A week earlier, I had spoken to their CTO, who was practically begging for a solution to bring these teams together. His teams were using completely separate toolchains, paying over $1 million annually for tools that didn’t even integrate. He had no visibility, no way to make informed decisions, and worst of all, no real collaboration between his teams.

    This is very bad for business. It prevents the company from scaling, optimizing costs, and, most importantly, moving fast without breaking things.

    Hybrid Is the Future, Whether We Like It or Not

    In my opinion, hybrid is here to stay. Some workloads will always belong on-prem due to security, compliance or cost reasons. Others will thrive in the cloud, where elasticity and managed services make life easier. The problem is most organizations haven’t figured out how to make this coexistence work smoothly.

    Right now, hybrid infrastructure feels like owning two cars but having no idea which one to use on any given day. Imagine having to change the tires, handle maintenance, and refuel separately for each—without even knowing how often you’re driving them. That’s what most CIOs are dealing with when it comes to managing a mix of cloud and on-prem workloads. Without unified visibility, decisions get made in silos, costs spiral out of control and teams get stuck in an endless cycle of firefighting instead of innovating.

    Infrastructure as Code Orchestration: The Bridge We Need

    This is where infrastructure-as-code (IaC) orchestration comes in. If hybrid infrastructure is inherently complex, then the answer isn’t just automation—it’s orchestration.

    Automation alone won’t fix the fragmentation. You can automate tasks all day long, but if each team automates in its own way with different tools, you’re just adding complexity at scale. Orchestration, on the other hand, introduces a single layer of control—one set of workflows that unifies deployment and ongoing operational management across on-prem and cloud infrastructure.

    Here’s what this actually means in practice:

    ● Visibility – Instead of scattered tools and isolated teams, you get a single source of truth for your infrastructure.

    ● Efficiency – No more redundant work. Instead of two teams doing similar things in totally different ways, orchestration allows for shared workflows.

    ● Security & Compliance – Unified policy enforcement across all environments. No more guesswork or inconsistent controls.

    ● Developer Velocity Without Losing Control – The ability to move fast while ensuring infrastructure changes are auditable, reversible and compliant.

    I’ve seen companies try to duct-tape their way around this problem. They build custom pipelines with Jenkins or GitHub Actions, but eventually, they hit a wall. One company (who I can’t name, but let’s just say they handle a lot of digital documents) tried to do this with GitHub Actions, relying on one engineer to make it work. His salary alone cost them more than an orchestration platform, and if he left, they’d be completely stuck. That’s not a strategy—it’s a liability.

    The CIO’s Next Move

    If you’re a CIO dealing with hybrid headaches, here’s what you should be asking yourself:

    1. Do I actually have visibility into both my on-prem and cloud environments? (If the answer is no, that’s problem #1.)

    2. Are my teams working together, or are they operating as separate entities?

    3. How much are we spending on redundant tools that don’t integrate?

    4. What’s slowing down our infrastructure deployments, and is it avoidable?

    The companies that solve this problem first are the ones that will be best positioned for the future. They’ll move faster, spend smarter and avoid the kind of growing pains that turn into existential crises.

    If any of those questions made you uncomfortable, it’s time to rethink your approach. Hybrid doesn’t have to be a battleground. With the right orchestration, it can actually be an advantage