Text Summarization With OpenAI and Ruby on Rails

Modern applications deal with massive amounts of text — support tickets, CRM notes, blog posts, meeting transcripts, and internal documentation. The problem isn’t access to information anymore — it’s how quickly users can understand it.

In our CRM system, we allow publishing long-form articles to a blog. However, users rarely want to read everything up front.

To solve this, we introduced AI-powered summarization to generate short, readable previews.

This improves:

• Content scanability
• User engagement
• Time-to-information

In this article, we consider:

• What text summarization is
• Why AI summarization is powerful
• Setting up OpenAI in Rails
• Implementing a summarization service
• Building a controller endpoint
• Handling long documents
• Background processing with Sidekiq
• Real-world use cases

What Is Text Summarization?

Text summarization is the process of condensing a large body of text into a shorter version while preserving its key information. There are two main approaches:

1. Extractive Summarization

This selects the most important sentences directly from the original text. Example:

Original: Ruby on Rails is a powerful web framework designed to make programming easier by favoring convention over configuration.

Summary: Ruby on Rails is a web framework that simplifies development.

2. Abstractive Summarization

This generates new sentences that capture the meaning of the text. This is where large language models like OpenAI shine.

Why Use LLMs for Summarization?

Traditional NLP methods struggle with context and nuance.

OpenAI models can provide:

• Contextual understanding
• Multi-paragraph reasoning
• Domain adaptability
• Natural-sounding summaries

This makes them ideal for summarizing in a project with different types of information processes:

• Blog posts
• Documents
• Meeting transcripts
• Customer feedback
• Knowledge bases

Setting Up OpenAI in a Rails project

1. First of all, install the OpenAI Ruby Gem. Add the gem:

gem "ruby-openai"

2. Configure the API key. Add your API key to environment variables:

export OPENAI_API_KEY="your_api_key"

3. Example initializer:

OpenAI.configure do |config|
  config.access_token = ENV["OPENAI_API_KEY"]
end

Creating a Summarization Service

In Rails, the best practice is to encapsulate OpenAI logic in a separate service object.

Simple example:

module Openai
  class Summarizer
    def initialize(text)
      @text = text
      @client = OpenAI::Client.new
    end

    def call
      response = @client.chat(
        parameters: {
          model: "gpt-4.1-mini", #or you can select another one
          messages: [
            {
              role: "system",
              content: "You are a helpful assistant that summarizes text concisely." #you can define content with more detailed prompt
            },
            {
              role: "user",
              content: "Summarize the following text:\\n\\n#{@text}" #also here define more detailed expected response
            }
          ],
          temperature: 0.3
        }
      )

      response.dig("choices", 0, "message", "content")
    end
  end
end

Possible roles:

Role     Purpose
system - > instructions for the model
user - > input from the user
assistant - > previous AI responses

Usage:

summary = Openai::Summarizer.new(article.content).call

Example: Before and After

Input (CRM article excerpt):
Our platform allows teams to manage projects, track time, and generate reports across multiple departments...

Output (AI summary):

• Centralized platform for project and time tracking
• Supports multi-department workflows
• Provides reporting and analytics tools

This summary can be shown in:

• Blog preview cards
• Tooltips
• Search results

Example Controller Endpoint

Now we expose this functionality via an API endpoint.

Controller example:

class Api::SummariesController < ApplicationController
  def create
    text = params[:text]

    summary = Openai::Summarizer.new(text).call

    render json: {
      summary: summary
    }
  end
end

Temperature controls randomness in the output.

0.0 → deterministic
1.0 → very creative

Additional Useful Parameters

Also added parameters for improving summarization.

1. max_tokens: Limits the size of the generated response. Example:

max_tokens: 200 #This prevents extremely long outputs.

2. top_pAlternative randomness control.

Example:

top_p: 0.9 #Usually you adjust temperature or top_p, not both

3. frequency_penalty discourages repeated phrases.

Example:

frequency_penalty: 0.2 #Useful when summaries become repetitive

4. presence_penalty encourages introducing new ideas.

Example:

presence_penalty: 0.1 #Not usually necessary for summarization, but can be used in specific tasks

Prompt Engineering for Better Summaries and Why It Is Important

The prompt design significantly impacts the output quality. Instead of a generic prompt:

"Summarize this text"

Use structured instructions:

"Summarize the following text in 3 bullet points. Focus on the key ideas and avoid unnecessary details."

This simple change improves clarity, consistency, and usefulness of the generated summary.

In practice, prompt design becomes even more important when working with different types of content, such as technical documentation, CRM notes, etc. I wrote more about the features of Prompt Engineering on practical examples in my other article.

Example of a more structured prompt:

{
  role: "user",
  content: <<~PROMPT
    Summarize the following article in 5 bullet points.

    #{@text}
  PROMPT
}

Handling Very Long Documents

LLMs have token limits, so large texts must be processed in chunks.

Typical approach looks like this:

1. Split text into chunks
2. Summarize each chunk
3. Combine summaries
4. Generate a final summary

Avoid Naive Chunking

This is not ideal:

text.scan(/.{1,3000}/m)

It may cut sentences in half.

Prefer:

• Splitting by paragraphs
• Splitting by sentence boundaries

Text Chunking

class TextChunker
  def self.chunk(text)
    text.split("\\n\\n")
  end
end

Chunk Summarization

def summarize_long_text(text)
  chunks = TextChunker.chunk(text)

  partial_summaries = chunks.map do |chunk|
    Openai::Summarizer.new(chunk).call
  end

  Openai::Summarizer.new(partial_summaries.join("\\n")).call
end

Using Background Jobs for Summarization

Summarizing large text can take some time, so it’s better to process it asynchronously.

Example of our service usage with Sidekiq:

Generate Summary Job

class GenerateSummaryJob
  include Sidekiq::Job

  def perform(article_id)
    article = Article.find(article_id)

    summary = Openai::Summarizer.new(article.content).call

    article.update!(summary: summary)
  end
end

Error Handling

Always assume external APIs can fail.

rescueStandardError=>e
    Rails.logger.error(e.message)
    fallback_summary
end

Also consider:

• Retries
• Timeouts
• Monitoring

Cost Optimization

When you use AI features in production, cost management becomes critical. depends primarily on token usage, meaning the more text you send and receive, the more you pay.

Some tips for cost optimization that you need to know:

1. Limit Input Size

The most effective optimization is reducing the amount of text sent to the AI model.

Instead of summarizing an entire document, you can:

• Extract relevant sections
• Summarize those sections only

Example filtering before sending to OpenAI:

class TextPreprocessor
  MAX_LENGTH = 5000

  def self.clean(text)
    text.strip[0...MAX_LENGTH]
  end
end

Usage:

clean_text = TextPreprocessor.clean(article.content)
summary = Openai::Summarizer.new(clean_text).call

This ensures you never send extremely large inputs.

2. Choose the Right Model

Not every task requires the most powerful model. For summarization, smaller models often perform well.

Example:

model: "gpt-4.1-mini"

Advantages:

• Much cheaper
• Faster responses
• Good summarization quality

So, use larger models only for complex reasoning tasks.

3. Token Counting Before Requests

Sometimes the text is larger than expected. Using a token estimation step helps prevent sending oversized prompts.

Example:

def too_large?(text)
  text.length > 12000
end

If too large:

• chunk text into smaller chunks
• summarize in parts (chunks)

Conclusion

Throughout this article, we built a summarization pipeline in Rails using a clean service-oriented approach:

• Simple summarization service
• Prompt optimization
• Chunking for large documents
• Background processing with Sidekiq
• Cost and reliability improvements