AI Agents: What We’ve Learned So Far

If 2024 was the year AI models went from powerful to pervasive, 2025 has emerged as the year they’ve become… agentic. From one-off prompt engineering to multi-step, autonomous goal-chasing AI systems, we’re now entering a fascinating and fast-moving phase: the era of AI Agents. 

At Saratoga, we’ve always had our eyes on the road ahead, especially when it comes to technology that could reshape how software is delivered, managed, and experienced. While much of the industry continues to talk about AI agents in abstract terms, we’ve already been elbows-deep in experimentation and architecture, learning from the inside out. And we’ve got a few reflections to share. 

Here’s what we’ve learned so far. 

What are AI Agents, Really? 

To appreciate where things are going, you have to get a grip on the terminology. There’s no shortage of jargon floating around in AI newsfeeds, but at the core, it’s quite simple. 

An AI Agent is an autonomous software entity that perceives its environment, processes information, and acts toward a defined goal, often without needing human input at every step. 

Agentic AI refers to the broader system of agents working in concert, often communicating with one another, augmented with tools, memory, and the ability to reason and adapt over time. 

So, while a single AI Agent might handle a task, Agentic AI systems tackle workflows. They coordinate, collaborate, and compound their intelligence, looping back through steps, refining results, and learning iteratively. 

You might find this read interesting: An Early, Adopters Guide to AI Agents for Financial Services 

 

The Hype is Real, but so is the Complexity 

Let’s be honest: AI agents sound idyllic on paper. Autonomous bots solving problems while you sleep? Sign us up. 

But getting from idea to implementation requires more than flipping a switch. It means working out the kinks in a fast, evolving landscape where best practices are still emerging, tools are maturing in real time, and the foundational questions remain unsettled. 

Questions like: 

• What frameworks should we build on? 

• How do we ensure explainability, traceability, and compliance? 

• How do we orchestrate multiple agents without things descending into chaos? 

• How best do we ensure model independence in order to take advantage of leaps in maturity as different vendors vie for dominance? 

• And the ever-present challenge: how do we strike the balance between automation and control?

We set out to explore these questions by building something real, a working prototype that could prove the concept, expose the pitfalls, and help us learn quickly. 

Related: AI and Machine Learning in Business Applications 

 

The Foundation: Choosing the Right Tools (For Now) 

When it came time to roll up our sleeves, we began with the goal of staying as open and vendor-agnostic as possible. The agent ecosystem is growing fast, and vendor lock-in is a real risk. We opted for tools that prioritised flexibility, interoperability, and transparency. 

While we won’t get into every specific choice here, a few guiding principles emerged from our early experiments: 

• Open standards matter. Whether it’s telemetry, data validation, or LLM integration, choosing tools that play well with others helps prevent dead ends. 

• Proxies are powerful. Having an intermediary that lets you switch between large language model providers without rewriting logic is invaluable for experimentation. 

• Debuggability is non-negotiable. Agentic AI can be difficult to trace, especially when things go off the rails. We learned quickly that observability and instrumentation need to be built in from day one. 

• Tooling trumps brute force. While LLMs are great at reasoning and language, they’re still bad at maths and painfully inconsistent with formats. Using dedicated tools for structured tasks, like parsing or calculating, makes the system far more reliable. 

 

Orchestrating Agents: The Magic (and Madness) of Delegation 

One of the most interesting aspects of Agentic AI is orchestration: the ability for one agent to call another or hand off a task to a different component in the system. This is essential for building systems that reflect real-world complexity.  

Most tasks aren’t linear.  

They involve branching decisions, tool use, retries, and results that need to be analysed and reprocessed. 

We explored several models of orchestration, from agent delegation (where one agent uses another like a subcontractor) to programmatic hand, off (where the application chooses which agent to trigger next, based on results). 

Here’s what we discovered 

• Modularity helps you scale. Designing agents that do one thing well makes them more composable. We’ve started thinking of our agents as microservices with brains. 

• Tool, augmented agents are more reliable. Rather than expecting an LLM to do everything, we supplement it with discrete tools, like parsers, calculators, and validators, so the heavy lifting is shared intelligently. 

• Monitoring orchestration flow is crucial. Multi-agent systems can become spaghetti very quickly. Instrumentation and visualisation help keep the architecture legible. 

 

A Real Use Case: AI Agents on Documents 

We didn’t just want to theorise. So, we built. 

One of our early test cases involved working with documents. Lots of documents. In various formats, with different structures and content types. The kind of thing that makes traditional parsing fragile and brittle. 

Here, we used agents to ingest PDFs, classify their types, extract relevant information, and even reason about what the documents meant. Different tools handled different document types, while a supporting tool did the calculations, because, let’s be real, LLMs are still hilariously bad at maths. 

The results? Surprisingly good. Possibly 90 percent of what we’d hoped for.  

We saw major improvements in processing time, consistency, and even in the quality of extracted insights. 

We’re still pushing for that final 10 percent that will make it truly remarkable. 

However, it was the clearest signal yet that agentic systems aren’t just hype, they’re useful. Today. 

 

Caveats, Constraints and Considerations 

Of course, it’s not all smooth sailing. While AI agents are exciting, they also come with very real limitations: 

• Performance vs. Explainability. Debugging a multi-agent system is no joke. As systems get more autonomous, it becomes harder to trace decisions. This raises obvious questions around compliance, especially in regulated industries. 

• Security and privacy. Agentic AI systems handle data in new ways. Depending on your implementation, that can mean new risks, and new responsibilities. 

• Compute costs. These systems don’t come cheap. Orchestrating multiple LLM calls, each potentially calling tools or other models, can rack up usage bills quickly. 

• Model variability. Different LLMs behave differently. What works with one provider might fall flat with another. That’s why interoperability matters. 

Related: Is A.I. a (real) threat to humanity? 

 

Where We Go From Here 

So, what’s next for us, and for the wider world of agentic AI? 

In our view, 2025 is the year of experimentation. It’s about building small, learning fast, and staying adaptable. As the tooling matures and the best practices solidify, we’ll be in a stronger position to scale our efforts, and help clients do the same. 

But perhaps more importantly, we’re seeing the rise of AI systems that feel less like tools and more like collaborators. Systems that reason. That adapt. That don’t just respond to prompts, but proactively pursue outcomes. 

We’re not saying AGI is around the corner, but the leap from AI, as a function to AI, as a collaborator, is well underway. 

 

Final Thoughts: A Word to the Cautiously Curious 

If you’re still on the fence about AI agents, we get it. The space moves fast. The risks are real. The jargon is heavy. But here’s what we’ve learned: 

You don’t need to build an AI-first product to benefit from Agentic AI. You just need a use case with enough complexity to make human-like decision-making valuable. Start small. Instrument everything. Stay vendor-agnostic where you can. 

At Saratoga, our mission is to help clients navigate complexity with confidence, and agentic AI fits right into that mission. It’s less about chasing buzzwords. It’s more about building systems that work smarter, learn faster, and adapt on the fly. 

Here’s to exploring the frontier, one agent at a time.