The New API Economy With LLMs

Large language models (LLMs) are becoming more advanced in understanding context in natural language. With this, a new paradigm is emerging — using LLMs as APIs.

Traditionally, an API call would be GET /users/123/orders and you would receive a JSON in return, which would return the orders for the user 123. APIs facilitate the interaction between different software systems.

But what if the query was more complex? What if the requirement was to interact with another system without having much knowledge about the APIs to do so? What if the person interacting with the other system is non-technical and does not know how to call the API?

This is where LLMs come into the picture. LLMs can function as a general-purpose API that can interpret the input and generate an output. In this case, the input is a natural language query, and the output is an API.

This changes how software is built and how users interact with the software. This article discusses this aspect from both angles — how the developers should build software that supports an LLM-type API and how users can use LLMs to interact with software.

You may wonder why now. Well, the answer is simple — there is an increased level of sophistication and accessibility of the models, which makes this shift possible. As the model outputs improve and the latencies decrease, this is becoming an easier shift.

From Code to Natural Language

Traditional APIs are deterministic: You make a specific request and get a known response. Every single time the traditional API is called, the response is the same. They are predictable, reliable, and rigid. But LLMs change that equation. Let’s see an example:

Traditional API:

GET /users/123/orders → JSON → Manual filtering & aggregation

LLM API:

"What did user 123 order last month?" → LLM → “User placed 3 orders totaling $249”

The two diagrams above show the difference in how the same task — retrieving and interpreting a user’s history is handled.

LLM-powered API access offers a transformative shift by enabling users to interact with systems using natural language instead of structured syntax. Unlike traditional APIs that require understanding endpoints, parameters, and authentication details, LLMs interpret user intent and generate the correct API calls behind the scenes. This removes the need for technical expertise, making API access available to non-developers like analysts or product managers. LLMs also handle vague queries (e.g., "recent orders") by inferring filters, reducing the need for back-and-forth query refinement. They can combine data from multiple sources, apply reasoning, and return user-friendly summaries instead of raw JSON. Overall, LLMs abstract complexity, shorten the path from question to insight, and unlock APIs for a broader set of users.

Below is another example :

• Traditional API: GET /weather?city=”SanFrancisco”
• LLM API: “What’s the weather in San Francisco this weekend? Should I pack a jacket?”

If you notice, the difference is very apparent. The LLMs are doing more than just getting the weather for San Francisco in the above example. They are also able to answer a question about whether a jacket is required or not. This shows the ability of the new API surface that the LLMs can provide. It not only queries the traditional API style to get an answer, but is also able to answer additional queries that may or may not be present in the data. The LLM could be programmed to query the weather API, see a forecast of 60°F with wind, and then access an internal knowledge base that correlates temperature and conditions with appropriate clothing, leading to the recommendation. LLMs are no longer assistants. They are becoming programmable, context-aware software interfaces.

How LLMs Are Changing the Way We Build and Use APIs

This fundamental shift from structured code to unstructured language demands adaptation from everyone involved in the software lifecycle, from the developers building the APIs to the end-users interacting with them.

API Developers

Developers now need to transition, as this demands a new skill set. APIs are contracts, and they should be maintained as such. From the calling user endpoint, they want to call into an API and expect to receive the exact same results each time, given that the parameters are the same. With LLMs hallucinating and relying on probabilities, maintaining the contract can become challenging. 

Consider the example used above: “What did user 123 order last month?", internally needs to call the same API and correctly extract the user id and the timeframe from the prompt. But what if the query changes to “For the last 30 days, what did user 123 order”. For a human, it is obvious that the question is still the same, just a different way of saying it. It becomes important for the API developers to be able to extract this information consistently and correctly with the LLM.

Developers need to focus on:

• Creating prompt templates to guide the model's behavior as per its use case.
• Slowly integrate LLM APIs into their code base. This is to ensure that it is a slow phased approach.
• Be understanding of the fact that it is not necessary to convert all the APIs to be used through an LLM API.
• Optimize for LLM latencies and token usage.
• Have good guardrails so that there is no misuse of the LLM APIs. For instance, a developer could implement a guardrail that prevents an LLM from processing a prompt containing 'delete all users’. The system would flag the suspicious intent before parsing it.
• Have a failure detection model to ensure that hallucination can be minimized. A failure detection model might work by having the LLM show its work. Before executing a database query, the LLM could output the query it plans to run. A separate, simpler validation model could then check that query against a set of allowed patterns, catching a potential hallucination before it returns incorrect data.
• Enable good access management even with LLM APIs. For example, it should not be made possible for an LLM API to bypass the permissions required to access certain data or perform certain actions.
• Ensuring the AI’s output is useful, safe, and aligned is as important as writing bug-free code.

This gives rise to LLMOps, a new discipline focused on the lifecycle management of LLM applications, including prompt monitoring, A/B testing prompt templates, and maintaining observability for failure detection and resolution. Building with LLMs is part coding, part prompt engineering, part optimization, and part product design.

API Users

Callers of these LLM APIs can rejoice. Since LLM APIs are becoming a thing, this gives the callers the ability to choose which kind of API to call. 

A developer might let an LLM handle a fuzzy user query by turning it into multiple API calls (via function calling or an agent), but use a direct API call for a straightforward, deterministic operation. Let us talk about the same example used above: GET /users/123/orders. For an API user, they need this information to populate a user interface. But to get insights from it, the prompt can become “Summarize user 123 orders and identify patterns within the data”. With this modification, there can be multiple API calls: one that gets the user 123's orders and one that identifies patterns within that data.

From the caller's perspective:

• Ensure that the API suits the purpose. While the developers will limit hallucinations or incorrect results, assume that mistakes can happen. By this logic, ensure critical work is called through well-defined APIs. Work that can benefit from LLM-level responses can be done using LLM APIs. Deciding when to use one versus the other becomes a key design choice.
• An important aspect is AI literacy, where callers need to understand the best way to structure their inputs to get the best outputs.
• Adapt their own software interactions with LLM APIs. For example, a travel booking application could use a traditional API to fetch flight prices. It could then use an LLM API to interpret a user's freeform request like, 'Find me a flight to Hawaii next month, I want to surf and stay somewhere family-friendly.' The LLM would parse the intent ('surfing,' 'family-friendly') to filter the structured data retrieved from the traditional API.

Typical roles: Support agents (partner engineers), business analysts, integration engineers

Non-Technical Users

LLMs don’t just empower developers; they open doors for everyone else. Previously, a designer may have had to use a complicated User Interface to perform a certain action. Internally, when a button was pressed, an API was being called to perform the action. But now, the designer can simply prompt the LLM to perform a certain action or even guide the designer on the best way to perform that action.

For this category of users, a query like GET /users/123/orders is hard to understand. Language provides the opportunity to ask so much more: “What did user 123 order within the last month, provide me with a summary and compare it with other users in the area”. This query functions as a multi-faceted question that directly serves the user's purpose.

For a product manager, instead of asking the developers how certain sequence flows take place, they can easily query the LLM and get answers to it.

For example, a marketing manager could ask, "Draft three social media posts for our new product launch, tailored for Twitter, LinkedIn, and Instagram, and schedule them for Monday at 9 AM." The LLM would not only generate the content but also interact with a scheduling API.

Or a sales representative could prompt, "Summarize my last five email interactions with Client X and suggest three potential next steps based on their stated needs." This replaces digging through a CRM with a direct, action-oriented query.

Typical roles: Product managers, designers, marketers, sales reps, operations staff

Language as the New API Surface

The biggest shift isn’t just technical — it’s conceptual.

With LLMs, the API surface becomes language. You’re not just coding functions, you’re authoring instructions. You’re not designing screens, you’re enabling interactions. You’re not calling specific APIs, you’re asking a question. LLMs are shifting the paradigm from rigid interfaces and predefined contracts to more adaptive, conversational, and intelligent systems.

It is not just the above categories that need to change. The biggest change has to be in how enterprises adopt them. LLMs expose interfaces and agents that can be customized. For enterprises, this means LLMs are not just another tool but a customizable and intelligent layer that can be wrapped around existing legacy systems. 

The transition to using language as an API is just the beginning. As these systems become more capable and integrated, the distinction between using software and having a conversation will reduce, fundamentally reshaping our relationship with technology. Treating LLMs like APIs is the first, crucial step into that future.