Research IT

lines of code

When an AI Application Isn't the Answer

Agent SDKs can introduce higher costs and limited observability compared with tools like Claude Code. This article explores why practicality, cost and efficiency should come before technical complexity when choosing an AI solution.


What Are Agent SDKs?

Agent software development kits (SDKs) - such as the Claude Agent SDK, the OpenAI Agents SDK, and similar offerings - are powerful tools for embedding AI capabilities into software applications. They make it possible to automate workflows, integrate large language models into existing systems, and build scalable AI-driven products. In many cases, they are exactly the right tool for the job.

Recently, however, I worked on a project where taking the obvious engineering approach - i.e., using an Agent SDK in an AI application - ultimately turned out not to be the most practical solution.

The Challenge

The project originated from a workflow used by the Office for Open Research (part of the University Library team) when they store papers published by University of Manchester researchers. They had discovered that Claude Code could reliably extract metadata from “Author-Accepted Manuscripts” (PDFs/DOCXs) and convert it into JATS XML, a format commonly used for scholarly publishing.

This process is very time consuming to do by hand and could be automated with AI very well, but it relied on an individual running Claude Code locally on their own machine. The goal of the project was to build a shared application that made this capability available to the wider team, while integrating it with the surrounding document management and publishing workflow.

The Initial Solution

The initial idea was straightforward: build an application around the Claude Agent SDK that would bring the existing Claude Code workflow into a shared web application.

Rather than relying on individuals running Claude Code locally, members of the library team would be able to log in, upload documents, manage their work, and integrate the converted JATS XML into the wider publishing workflow. On paper, embedding the Claude Code experience into an application via the Agent SDK seemed like the logical progression.

After building the application, however, two practical issues became apparent.

Issue 1 - limited built-in telemetry

In previous projects, I had worked directly with providers such as OpenAI and Groq using their APIs. Those platforms provide dashboards that expose useful operational metrics such as model usage, token consumption, and request statistics. Having access to that information makes it much easier to understand system behaviour, estimate costs, and optimise an application.

With the Claude Agent SDK, our experience was quite different. Since we weren't interacting with Anthropic's API in the same way, we didn't have access to comparable built-in telemetry. Beyond overall usage limits, there wasn't a straightforward way to determine exactly how different models or reasoning levels were contributing to consumption.

There are certainly third-party observability tools that can help fill this gap, but there wasn't an out-of-the-box solution that gave us the visibility we were expecting. For an application using multiple models and varying reasoning settings, this made performance and cost analysis significantly more difficult.

Issue 2 - the cost difference was enormous

The second - and ultimately much more significant - issue was cost.

When we compared running the workflow through the Claude Agent SDK with performing the same work using Claude Code, the difference was surprisingly large. In our testing, the Agent SDK proved to be roughly 18-20 times more expensive for the same document conversion task.

To put that into perspective, the cost of converting more than five documents using Claude Code was roughly equivalent to that of converting less than one document through the Agent SDK.

That fundamentally changed the economics of the project.

The Surprising Conclusion

As software engineers, there's often a natural instinct to build an application around a workflow that has already proven successful. Once you have something that works, building an application around it feels like the obvious next step.

In our case, though, the numbers told a different story.

Because the Agent SDK introduced such a significant increase in operating costs, the simpler solution turned out to be the better one. Having a small number of people continue processing documents directly with Claude Code was substantially cheaper than maintaining an application built around the Agent SDK. Despite requiring some manual effort, it was ultimately the more productive and cost-effective approach.

A screenshot of the app, including the xml rendering and the uploaded PDF, also showing utilisation of the Claude agent SDK
Fig. 1
A screenshot of the app, including the xml rendering and the uploaded PDF, also showing utilisation of the Claude agent SDK

Final Thoughts

This isn't a criticism of the Claude Agent SDK itself. Agent SDKs are valuable tools, and for many applications they make perfect sense. But this experience reinforced an important engineering lesson: the technically elegant solution isn't always the operationally sensible one.

Before investing time in building an AI-powered application, it's worth validating not only that the approach works technically, but also that it makes sense from the perspectives of cost, observability, and long-term maintenance.

Sometimes, the simplest workflow is the right one.

Get In Touch

If you're considering using Agent SDKs or developing AI-powered applications, feel free to contact Research IT for advice and guidance on choosing the right approach. You can get in touch via Connect to discuss your requirements and explore the most suitable solutions.

More Information

Aadityaa Mehrotra was a Year in Industry student in Research IT 2025/6; Erdem Atbas (Research Software Engineer) supervised Aadityaa and acted as tech lead on the project; Sarah Jaffa (Senior RSE) acted as Project Manager.