Retrieval Augmented Generation With Spring AI 2.0, Claude, and PGvector

Language models become much more useful when they can answer questions about information they were never trained on, including your internal documentation, product manuals, policies, and other proprietary data. Prompting alone cannot solve this, because the model simply does not have access to that knowledge. Retrieval-Augmented Generation, or RAG, is the most common way to bridge that gap.

Spring AI comes with solid support for building RAG systems. It has been almost three years since Spring AI showed up, and in that time it has grown from an experimental member of the Spring portfolio into a mature layer over chat models, embedding models, vector stores, and the plumbing that sits between them, which happen to be exactly the pieces a RAG system needs.

In this article, we build a small but complete RAG service with Spring AI 2.0. The application reads a set of documents into a PostgreSQL vector store, retrieves the fragments that are relevant to a user question, and lets Anthropic's Claude put together the answer based on those fragments. Everything runs from a standard Spring Boot project, and every step can be reproduced on macOS, Windows, or Linux. The full project is available on GitHub. If you just want to see the finished result, or you would rather skip the step-by-step build below, you can clone the repository and run it as it is. Everyone else can follow along and generate this project from scratch.

The prompts themselves are kept deliberately simple. You can tune retrieval and prompts forever; here we care about the architecture and how the pieces fit together in Spring.

Approach

RAG is not really a single feature. It is more of a small pipeline, and the code below makes a lot more sense once its parts have names.

• Embedding: a vector of numbers that captures the meaning of a piece of text. Texts that mean similar things end up with vectors that are close to each other.
• Embedding model: the model that computes these embeddings. It is a different model from the chat model, and it has a different job.
• Vector store: a database that keeps text fragments together with their embeddings and can answer the question, "which stored fragments are closest in meaning to this query?"
• Chunking: documents are too large to embed and retrieve as a whole, so we split them into smaller fragments (chunks) before storing them.
• Similarity search: we embed the user question and fetch the top-k closest chunks from the store.
• Augmentation: we append the retrieved chunks to the user question before sending it to the chat model, so the model answers from the context we provided instead of from its training data.

One thing here is worth calling out, because it shapes the whole setup of the project: the LLM model used in chat and the embedding model are two separate choices. As of today, Anthropic offers LLM models but no embedding API, so a Claude-based RAG system always has to pair Claude with an embedding model from somewhere else. Rather than bringing in a second cloud provider and a second API key, this project computes embeddings locally (inside the JVM), using Spring AI's ONNX transformers module and the well-known all-MiniLM-L6-v2 sentence transformer. It is free and fast enough for this, and it keeps everything on one API key.

In our scenario, the service is an internal assistant for a fictional company called Nimbusfield Systems, and it answers employee questions based on the company handbook. The company and the handbook are fictional on purpose. Claude cannot possibly know about it, which makes it easy to verify that the answers really come from our documents and not from the model's own memory.

We build this in three steps:

1. Expose a /ask endpoint backed by Claude, with no retrieval, and show that the model cannot answer handbook questions.
2. Ingest the handbook into PGvector at application startup: read, chunk, embed, and store.
3. Attach Spring AI's QuestionAnswerAdvisor to the same ChatClient and ask again.

Prerequisites

• Java 21
• Maven 3.9.x (the Maven wrapper included in generated projects works too)
• Spring Boot 4.0.x
• Spring AI 2.0.0
• Docker Desktop (macOS/Windows) or Docker Engine (Linux), used only to run PostgreSQL.

A project skeleton can be generated at start.spring.io by selecting Web, Anthropic Claude, PGvector Vector Store, and Docker Compose Support. The remaining Spring AI modules are added manually below.

The Claude API Key

Sign in (or sign up) at the Anthropic Console, open Settings, then API Keys, and create a new key. New accounts may need a small prepaid credit before the API accepts requests, but the runs in this article cost only a few cents. The key is shown only once, so store it right away as an environment variable. If you would rather not spend anything at all, you can still follow along and read through the steps without running the calls yourself.

macOS/Linux:

export ANTHROPIC_API_KEY=sk-ant-...

Windows (PowerShell, persists across sessions after reopening the terminal):

setx ANTHROPIC_API_KEY "sk-ant-..."

Solution

Dependencies

With the Spring AI BOM in place, there is no need to repeat versions on the individual artifacts. Initializr expresses the BOM's own version as a property rather than a hardcoded literal, so there is a single place to bump it later:

<properties>
    <java.version>21</java.version>
    <spring-ai.version>2.0.0</spring-ai.version>
</properties>

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-bom</artifactId>
            <version>${spring-ai.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

A common source of confusion is that start.spring.io has no dependency literally named "Spring AI." Each provider- or store-specific starter (Anthropic Claude, PGvector Vector Database, and so on) is itself a Spring AI module, and picking one transitively pulls in the framework's core classes. (like ChatClient, VectorStore, etc.) Selecting any one of them is also what makes Initializr add the spring-ai-bom as shown above to the generated pom.xml for you. The BOM itself is never a separate item you tick on the Initializr dependency screen.

The application needs six Spring AI modules on top of the web starter, each one with a single responsibility.

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-webmvc</artifactId>
    </dependency>

    <!-- Chat model: Anthropic Claude -->
    <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-starter-model-anthropic</artifactId>
    </dependency>

    <!-- Embedding model: local ONNX sentence transformer -->
    <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-starter-model-transformers</artifactId>
    </dependency>

    <!-- Vector store: PostgreSQL + pgvector -->
    <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-starter-vector-store-pgvector</artifactId>
    </dependency>

    <!-- RAG advisor -->
    <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-vector-store-advisor</artifactId>
    </dependency>

    <!-- Document reading (PDF, Word, Markdown, HTML, and more) -->
    <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-tika-document-reader</artifactId>
    </dependency>

    <!-- Starts the database container on application startup -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-docker-compose</artifactId>
        <scope>runtime</scope>
        <optional>true</optional>
    </dependency>

    <!-- Docker Compose service connections for Spring AI vector stores -->
    <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-spring-boot-docker-compose</artifactId>
        <scope>runtime</scope>
        <optional>true</optional>
    </dependency>
</dependencies>

Two models are referenced from the code here. One is the chat model, Claude, which is served from the Anthropic API. The other is the embedding model, which runs locally, right inside the application. We will look at that local embedding model in the next section.

The Embedding Model

By default, the transformers starter fetches tokenizer.json and model.onnx from Spring AI's own GitHub repository the first time the application starts and then caches them locally. In practice, this default setup is a bit fragile. raw.githubusercontent.com may rate-limit unauthenticated requests, and model.onnx (which is roughly 90 MB) is stored via Git LFS, whose bandwidth quota can run out independently of the ordinary rate limit. When that happens, the endpoint serves the small LFS pointer stub instead of the binary, with a normal-looking HTTP 200, and the failure only shows up later as a cryptic ONNX Runtime protobuf-parsing error rather than a clear download error.

The fix is to bundle both files with the application instead of fetching them at startup. So we download them once:

mkdir -p src/main/resources/onnx/all-MiniLM-L6-v2

curl -fL -o src/main/resources/onnx/all-MiniLM-L6-v2/tokenizer.json \
  https://raw.githubusercontent.com/spring-projects/spring-ai/main/models/spring-ai-transformers/src/main/resources/onnx/all-MiniLM-L6-v2/tokenizer.json

curl -fL --http1.1 -o src/main/resources/onnx/all-MiniLM-L6-v2/model.onnx \
  https://media.githubusercontent.com/media/spring-projects/spring-ai/main/models/spring-ai-transformers/src/main/resources/onnx/all-MiniLM-L6-v2/model.onnx

Then we point the embedding model at these local files in our application.properties, overriding the GitHub-backed defaults:

spring.ai.embedding.transformer.onnx.model-uri=classpath:/onnx/all-MiniLM-L6-v2/model.onnx
spring.ai.embedding.transformer.tokenizer.uri=classpath:/onnx/all-MiniLM-L6-v2/tokenizer.json

With these two properties set, the application never touches the network for the embedding model, neither on the first run nor on any run after it.

The Database

The pgvector team publishes a PostgreSQL image with the extension already installed. A compose.yaml in the project root is all we need:

services:
  pgvector:
    image: "pgvector/pgvector:pg17"
    environment:
      - "POSTGRES_DB=nimbusfield"
      - "POSTGRES_USER=nimbusfield"
      - "POSTGRES_PASSWORD=nimbusfield"
    labels:
      - "org.springframework.boot.service-connection=postgres"
    ports:
      - "5432"

The labels entry is important. Spring Boot's Docker Compose support auto-detects connection details by matching the image name against a list of well-known images. Plain Postgres is on that list, but pgvector is not, since it is a third-party image. The label tells Spring Boot to treat this container as if it were the official Postgres image, and that is what actually makes the automatic connection wiring work. If we omit it, the container still starts, but Spring Boot never creates a ConnectionDetails bean for it, so the run fails with a connection error rather than falling back gracefully.

Because spring-boot-docker-compose is on the classpath, running the application starts the container automatically and injects the connection details. This works the same way on macOS and Windows, as long as Docker Desktop is running. Anyone who prefers to manage the container manually can run the same image with docker run -p 5432:5432 ..  and set the datasource properties explicitly.

Configuration

The complete application.properties, now including the embedding model overrides shown earlier:

spring.ai.anthropic.api-key=${ANTHROPIC_API_KEY}
spring.ai.anthropic.chat.model=claude-sonnet-5
spring.ai.anthropic.chat.max-tokens=1024

spring.ai.embedding.transformer.onnx.model-uri=classpath:/onnx/all-MiniLM-L6-v2/model.onnx
spring.ai.embedding.transformer.tokenizer.uri=classpath:/onnx/all-MiniLM-L6-v2/tokenizer.json

spring.ai.vectorstore.pgvector.initialize-schema=true
spring.ai.vectorstore.pgvector.dimensions=384
spring.ai.vectorstore.pgvector.index-type=HNSW
spring.ai.vectorstore.pgvector.distance-type=COSINE_DISTANCE

logging.level.org.springframework.ai.chat.client.advisor=DEBUG

Four details matter here. First, max-tokens is mandatory for the Anthropic API, which caps every response explicitly. Spring AI provides a default, but it is better stated than left implied. Second, the two spring.ai.embedding.transformer.* properties point the embedding model at the local files we bundled in the previous section, instead of Spring AI's own GitHub-backed defaults. See "The Embedding Model" above for why this matters. Third, initialize-schema=true enables the automatic creation of the vector-store table and the required extensions. (Since Spring AI 1.0, this no longer happens silently by default.) Fourth, dimensions=384 must match the embedding model. all-MiniLM-L6-v2 produces 384-dimensional vectors. If the embedding model changes later, the table has to be recreated, because the column type is vector(384).

The Documents

Two short Markdown files under src/main/resources/docs play the role of the company handbook.

remote-work-policy.md

# Nimbusfield Systems Remote Work Policy

Employees may work remotely up to three days per week. Remote days must be
registered in the portal by Thursday of the preceding week. Working
from abroad is permitted for a maximum of 30 calendar days per year and
requires prior approval from both the line manager and the People team.

travel-expenses.md:

# Nimbusfield Systems Travel and Expenses

The daily meal allowance for business trips is 65 EUR in Europe and 80 USD
elsewhere. Taxi rides are reimbursed only between airports, hotels, and
client sites. Flights longer than six hours may be booked in premium
economy. All expense reports are due within 15 working days after the trip
via the portal.

Thanks to the Tika reader used below, dropping PDFs or Word documents into the same folder works without any code changes.

Step 1: Chat Without Retrieval

We start with a service that wraps a ChatClient, built once from the auto-configured builder:

@Service
public class AssistantService {

    private final ChatClient chatClient;

    public AssistantService(ChatClient.Builder builder) {
        this.chatClient = builder
                .defaultSystem("""
                        You are the internal assistant of Nimbusfield Systems.
                        Answer employee questions precisely and briefly.
                        If you do not know the answer, say so.
                        """)
                .build();
    }

    public String ask(String question) {
        return chatClient.prompt()
                .user(question)
                .call()
                .content();
    }
}

And a  controller associated with it:

@RestController
public class AssistantController {

    private final AssistantService assistantService;

    public AssistantController(AssistantService assistantService) {
        this.assistantService = assistantService;
    }

    @GetMapping("/ask")
    public ResponseEntity<String> ask(@RequestParam("question") String question) {
        return ResponseEntity.ok(assistantService.ask(question));
    }
}

Start the application (./mvnw spring-boot:run on macOS/Linux, mvnw.cmd spring-boot:run on Windows) and ask it a handbook question:

http://localhost:8080/ask?question=What is the daily meal allowance for business trips in Europe?

The response, as we might expect, is:

I don't have that information in my available knowledge base. Nimbusfield Systems' specific travel and expense policy—including per diem rates for European business trips—isn't something I can confirm accurately. To get the correct figure, please check:

• The company's Travel & Expense Policy document (likely on the intranet/HR portal)
• Your Finance or HR department directly
• Your manager, if travel budgets are pre-approved per trip

Would you like help with anything else I can assist with more reliably?

This gives us a baseline. The model behaves correctly given what it knows, which is nothing at all about this company.

Step 2: The Ingestion Pipeline

Ingestion follows Spring AI's extract, transform, load structure: a DocumentReader extracts the text, a TextSplitter chunks it, and the VectorStore embeds and stores the chunks. The embedding call happens implicitly inside vectorStore.add() call. The auto-configured TransformersEmbeddingModel is wired into the PgVectorStore and each chunk is embedded into the table.

@Component
public class HandbookIngestion implements ApplicationRunner {

    private static final Logger log = LoggerFactory.getLogger(HandbookIngestion.class);

    private final VectorStore vectorStore;
    private final JdbcTemplate jdbcTemplate;
    private final Resource[] handbook;

    public HandbookIngestion(VectorStore vectorStore, JdbcTemplate jdbcTemplate,
                             @Value("classpath:docs/*.md") Resource[] handbook) {
        this.vectorStore = vectorStore;
        this.jdbcTemplate = jdbcTemplate;
        this.handbook = handbook;
    }

    @Override
    public void run(ApplicationArguments args) {
        Integer count = jdbcTemplate.queryForObject(
                "select count(*) from vector_store", Integer.class);
        if (count != null && count > 0) {
            log.info("Vector store already contains {} chunks, skipping ingestion", count);
            return;
        }

        TokenTextSplitter splitter = TokenTextSplitter.builder()
                .withChunkSize(300)
                .build();

        for (Resource resource : handbook) {
            List<Document> documents = new TikaDocumentReader(resource).get();
            documents.forEach(doc ->
                    doc.getMetadata().put("source", resource.getFilename()));

            List<Document> chunks = splitter.apply(documents);
            vectorStore.add(chunks);

            log.info("Ingested {} chunks from {}", chunks.size(), resource.getFilename());
        }
    }
}

The count check makes ingestion idempotent, so restarting the application does not duplicate every chunk. And the source metadata attached to each chunk enables filtered searches later, for instance restricting retrieval to a single document.

That same idempotency check has a practical downside worth pointing out. Once the vector store has data, restarting the application will not pick up edits to the handbook files, since the count check short-circuits before the splitter ever runs. To force a clean re-ingestion, for instance after changing a handbook document, tear down the container together with its data volume, not just the container:

docker compose down -v

The chunk size of 300 tokens is generous for documents this small. The splitter's default of 800 is aimed at larger, real-world content. Chunking is the least exciting and yet the most important knob in a RAG system: chunks that are too large dilute the similarity signals, while chunks that are too small lose their context. It is worth experimenting here: try a few different chunk sizes and see how the system behaves. Just remember to run docker compose down -v between runs, so the vector store is rebuilt from scratch each time.

Step 3: Attaching the Retrieval Advisor

Now we come back to the plain AssistantService from Step 1 and upgrade it, rather than writing something new. The ChatClient wiring we built earlier stays and what changes is what gets attached to it. Spring AI models the cross-cutting concerns around a chat call as "advisors", which are conceptually close to interceptors. The QuestionAnswerAdvisor embeds the incoming user question, runs a similarity search against the vector store, and appends the retrieved chunks to the prompt before it reaches Claude. Enabling RAG is therefore a change to how the ChatClient  is constructed, not to how the request is handled:

public AssistantService(ChatClient.Builder builder, VectorStore vectorStore) {
    this.chatClient = builder
            .defaultSystem("""
                    You are the internal assistant of Nimbusfield Systems.
                    Answer employee questions precisely and briefly.
                    If you do not know the answer, say so.
                    """)
            .defaultAdvisors(
                    QuestionAnswerAdvisor.builder(vectorStore)
                            .searchRequest(SearchRequest.builder()
                                    .topK(4)
                                    .similarityThreshold(0.5)
                                    .build())
                            .build(),
                    new SimpleLoggerAdvisor())
            .build();
}

topK(4) retrieves at most four chunks per question, and similarityThreshold(0.5) discards weak matches, so an entirely unrelated question augments the prompt with nothing rather than with noise. The SimpleLoggerAdvisor, combined with the DEBUG logging property we set earlier, prints the fully augmented prompt. This is the single most useful debugging tool while tuning retrieval, because it shows exactly what Claude was given.

We restart and repeat the same request:

http://localhost:8080/ask?question=What is the daily meal allowance for business trips in Europe?

The daily meal allowance for business trips in Europe is 65 EUR.

Same model, same question, and this time a precise answer grounded in the retrieved handbook chunk instead of a generic deflection. The debug log confirms what is going on behind the scenes: the user question arrives at Claude wrapped in a prompt that contains the retrieved handbook fragments as context.

Going Further

The default behavior of QuestionAnswerAdvisor is usable, but there are two refinements worth implementing if you want to take this pattern further.

The first one concerns grounding. Even with retrieved context, the model may fall back on its general knowledge when the context does not actually contain the answer. The advisor accepts a custom  PromptTemplate that controls how the question and the context are merged, and this is the place to enforce stricter behavior. The template must contain the query and question_answer_context  placeholders:

PromptTemplate strictTemplate = PromptTemplate.builder()
        .template("""
                {query}

                Answer strictly based on the context below.
                If the context does not contain the answer, reply exactly:
                "This is not covered by the handbook."

                ---------------------
                {question_answer_context}
                ---------------------
                """)
        .build();

QuestionAnswerAdvisor advisor = QuestionAnswerAdvisor.builder(vectorStore)
        .promptTemplate(strictTemplate)
        .build();

Asking about, say, the parental leave policy (which is absent from our two files) now produces the fixed refusal instead of an invention. If people are going to rely on it, you want this on.

The second refinement could be structured output, and it composes cleanly with retrieval. Declaring a record and calling .entity()  instead of .content() gives back a typed object, with Spring AI instructing the model to respond in the matching JSON schema:

public record HandbookAnswer(String answer, String sourceHint, boolean coveredByHandbook) {
}

public HandbookAnswer askStructured(String question) {
    return chatClient.prompt()
            .user(question)
            .call()
            .entity(HandbookAnswer.class);
}

A last note on the embedding choice. A local MiniLM model is not the strongest embedding model available, and for a large multilingual corpus a hosted embedding API or a bigger ONNX model would retrieve better. This choice is easy to reverse: EmbeddingModel is an interface, swapping the implementation is a matter of a dependency and a property, and the only hard constraint is the one mentioned earlier: the vector dimensions in PGvector have to match whatever the embedding model produces. 

Conclusion

In this article, we built the RAG flow step by step. We started with a plain chat endpoint that could not answer anything about the Nimbusfield handbook, because Claude had never seen it. We then ingested that handbook into PGvector, embedding each chunk locally, and attached Spring AI's QuestionAnswerAdvisor to the same client. That single change was enough to turn a generic model into a service that answers from your own documents. After that, we talked about how we can tighten the grounding, so the model says it does not know when the context has no answer, and pulled the response straight into a typed Java record.

If you want to take it further, clone the project, point it at your own documents, apply further the techniques we discussed in the Going Further section, play with different chunk sizes, retrieval settings, and prompts to see how the answers change. The Spring AI documentation goes deeper into advisors, vector stores, and retrieval configuration.

The complete, runnable project is available on GitHub.