From Chat Completions to Responses: Why Is OpenAI Upgrading Its Core API?

This article was originally published on my blog. For the latest version and future updates, please visit the original post: https://jaketao.com/language/en/why-openai-upgrading-api.

If you’ve ever built a large-language-model application, you’ve most likely started with this endpoint:

POST /v1/chat/completions

In the era of GPT-3.5 and GPT-4, this endpoint was practically synonymous with the OpenAI API. Developers would pass in a set of messages, and the model would generate the next response based on the context.

But as applications have evolved from “chatbots” to “agents capable of invoking tools, executing tasks, and processing multimodal content,” the structure of the API has also begun to change. OpenAI has introduced a more unified approach:

POST /v1/responses

This doesn’t mean Chat Completions are obsolete; rather, it provides a more appropriate abstraction for the more complex workflows of agents.

Chat Completions: Conversation Messages at the Center

The core data structure of Chat Completions is messages. In each request round, the client must submit the context required for the model to understand the current task.

For example, a user requests the weather in Beijing:

{
  "model": "gpt-5.6",
  "messages": [
    {
      "role": "user",
      "content": "帮我查询北京天气"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "查询天气",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {
              "type": "string"
            }
          },
          "required": ["city"]
        }
      }
    }
  ]
}

After the model decides to call a tool, it will return a result similar to the following:

{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_weather_001",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"city\":\"北京\"}"
            }
          }
        ]
      }
    }
  ]
}

After the application executes get_weather, the next request must include the previous conversation, the tool calls initiated by the model, and the results of those tool executions:

{
  "model": "gpt-5.6",
  "messages": [
    {
      "role": "user",
      "content": "帮我查询北京天气"
    },
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        {
          "id": "call_weather_001",
          "type": "function",
          "function": {
            "name": "get_weather",
            "arguments": "{\"city\":\"北京\"}"
          }
        }
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "call_weather_001",
      "content": "北京晴，25°C"
    }
  ]
}

This approach is intuitive, well-established, and still suitable for most chat scenarios.

However, it has one obvious engineering shortcoming: context management is primarily handled by the client. As conversations grow longer and tool calls increase, the application must continuously maintain and replay historical messages.

Responses: Centered Around a Single “Task Response”

The Responses API takes a different approach: it treats model output not merely as a piece of text, but as a “response” that may include text, reasoning, tool calls, images, or structured results.

Let’s use the weather query as an example again:

{
  "model": "gpt-5.6",
  "input": "帮我查询北京天气",
  "tools": [
    {
      "type": "function",
      "name": "get_weather",
      "description": "查询天气",
      "parameters": {
        "type": "object",
        "properties": {
          "city": {
            "type": "string"
          }
        },
        "required": ["city"]
      }
    }
  ]
}

The model returns a responsecontaining a function call:

{
  "id": "resp_123",
  "output": [
    {
      "type": "function_call",
      "call_id": "call_weather_001",
      "name": "get_weather",
      "arguments": "{\"city\":\"北京\"}"
    }
  ]
}

After the tool completes execution, the next round only needs to submit the new results and reference the previous response:

{
  "model": "gpt-5.6",
  "previous_response_id": "resp_123",
  "input": [
    {
      "type": "function_call_output",
      "call_id": "call_weather_001",
      "output": "北京晴，25°C"
    }
  ]
}

OpenAI can use previous_response_id to associate the previous context with the tool call. The client does not need to manually replay the entire message history each time, making the Agent’s orchestration code more concise.

However, note that this does not mean “context no longer incurs costs.” Using previous_response_idreduces the complexity for the client in constructing and maintaining the message history; previous input tokens in the response chain will still be billed as input tokens.

Why Does the Agent Need the Responses API More?

In a question-and-answer scenario, messagesare natural; but Agents often need to constantly switch between conversations, tool calls, tool results, and structured data.

Chat Completions can also handle these tasks, but as the number of steps increases, the client must maintain a complex messages history on its own and ensure that tool calls are correctly mapped to their results.

The focus of the Responses API is not on adding a new capability, but on unifying these elements into response items and supporting the continuation of tasks based on the previous response, making it better suited for complex Agent workflows.

How Should an API Gateway Be Designed?

If the Gateway integrates models such as OpenAI, Claude, Gemini, and DeepSeek simultaneously, the key is not to rewrite all requests as Responses.

A more practical approach is to retain client-familiar interfaces  —  such as Chat Completions and Responses  —  for external use; once requests enter the system, they are parsed by the corresponding converters and routed into the same processing pipeline.

OwlVigil adopts precisely this approach: rather than replacing Chat with Responses, it allows different protocols to share the same set of gateway capabilities.

Client
  ├─ Chat Completions
  ├─ Responses
  ├─ Anthropic Messages
  └─ Gemini API
          ↓
      Inbound Converter
          ↓
   Unified LLM Request Model
          ↓
Model mapping, routing, rate limiting, retries
          ↓
       Outbound converter
          ↓
OpenAI
  ├─ Claude
  ├─ Gemini
  └─ DeepSeek

The term “unified” here does not mean forcing a binding to a single vendor’s protocol, but rather placing messages, tool calls, tool results, model parameters, and streaming responses into a single processing pipeline.