Two Engines, One Query: DuckDB Inside Neo4j

An example that embeds DuckDB inside Neo4j as a custom Java procedure, letting DuckDB SQL queries run directly from Cypher, with results flowing back into the graph.

Akmal Chaudhri

CORE ·

Oct. 20, 26 · Tutorial

Likes (0)

Comment

Save

191 Views

If you've been following the database world lately, you'll have noticed DuckDB turning up in some unexpected places. One of the more interesting examples is pg_duckdb, a Postgres extension that embeds DuckDB directly into the Postgres process, letting analytical queries run through DuckDB's engine without leaving the database. It's a clever idea: keep your data where it is, but bring a faster analytical engine.

That got us wondering: could we do something similar with Neo4j?

The answer is not in the same way. Postgres has a mature extension architecture that allows foreign query engines to be embedded at a low level. Neo4j doesn't expose that kind of hook. But Neo4j does let you write custom procedures in Java that run inside the Neo4j JVM and are callable directly from Cypher. Also, DuckDB ships an official JDBC driver that bundles its native binary and loads it via JNI. Put those two things together and you've got something that feels surprisingly close to the pg_duckdb experience: DuckDB running in-process, queries fired from Cypher, results flowing back into a graph.

That's what we built. In this article, we'll walk through the whole thing from scratch.

The full source code is available on GitHub.

Prerequisites

Before starting, we'll need the following:

JDK 21 – check with java -version
Maven – check with mvn -version
Neo4j Desktop – with a local database instance created and ready to start

Note that this approach requires a self-managed Neo4j install. AuraDB doesn't support custom plugins, so Neo4j Desktop is the right choice here.

Version Summary

For reference, here are all the component versions used in this article:

Component	Version
OpenJDK	21.0.12.1
Maven	3.9.6
Neo4j Desktop	2.2.1
Neo4j	2026.07.0
Neo4j Procedure API	5.26.0
DuckDB JDBC	1.1.3
JUnit Jupiter	5.10.2
maven-compiler-plugin	3.11.0
maven-shade-plugin	3.5.2
maven-surefire-plugin	3.5.4

Getting the Code

The full source code is on GitHub. To get just this project:

    Shell
   
 

   cd ~
git clone --filter=blob:none --sparse https://github.com/VeryFatBoy/neo4j.git
cd neo4j
git sparse-checkout set duckdb-procedure
mv duckdb-procedure ../duckdb-procedure
cd ../duckdb-procedure/neo4j-duckdb-procedure
  

Or follow along from scratch using the steps below.

How It Works

The architecture is straightforward. We write two Java classes and package them as a fat jar:

DuckDBConnection manages a singleton DuckDB connection that lives for the lifetime of the Neo4j process. It uses double-checked locking to keep things thread-safe and reconnects automatically if the connection is ever closed.
DuckDBProcedure is the Neo4j procedure itself. It accepts a SQL string from Cypher, runs it against the shared DuckDB connection and streams the results back as rows, each one a Map<String, Object> that Cypher can destructure with dot notation.

DuckDB runs in-memory by default, which means it starts clean every time Neo4j restarts. That's fine for analytics against external files, and we'll discuss file-backed persistence in the article Summary section.

Project Setup

Start by creating the project directory:

    Shell
   
   mkdir -p neo4j-duckdb-procedure/src/main/java/com/example/duckdb
cd neo4j-duckdb-procedure

Verify the structure:

    Shell
   
   find . -type d

Now create pom.xml in the project root.

    XML
   
 

   <?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>
    <artifactId>neo4j-duckdb-procedure</artifactId>
    <version>1.0.0</version>
    <packaging>jar</packaging>

    <properties>
        <java.version>21</java.version>
        <maven.compiler.source>21</maven.compiler.source>
        <maven.compiler.target>21</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <neo4j.version>5.26.0</neo4j.version>
        <duckdb.version>1.1.3</duckdb.version>
    </properties>

    <dependencies>
        <!-- Neo4j procedure API, provided by Neo4j at runtime -->
        <dependency>
            <groupId>org.neo4j</groupId>
            <artifactId>neo4j</artifactId>
            <version>${neo4j.version}</version>
            <scope>provided</scope>
        </dependency>

        <!-- DuckDB JDBC driver, bundled with native binary -->
        <dependency>
            <groupId>org.duckdb</groupId>
            <artifactId>duckdb_jdbc</artifactId>
            <version>${duckdb.version}</version>
        </dependency>

        <!-- Test dependencies -->
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter</artifactId>
            <version>5.10.2</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.11.0</version>
                <configuration>
                    <source>21</source>
                    <target>21</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.5.4</version>
            </plugin>
            <!-- Fat jar: bundles DuckDB JDBC into the plugin jar -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>3.5.2</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>
                            <createDependencyReducedPom>false</createDependencyReducedPom>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

</project>
  

The Neo4j dependency is provided scope — it won't be bundled into the jar since Neo4j supplies it at runtime. The DuckDB JDBC driver is included at compile scope so it gets bundled along with its native binary. The JUnit dependency is test scope so it's excluded from the fat jar. The maven-shade-plugin builds the fat jar at package time.

Confirm Maven can resolve the dependencies:

    Shell
   
   mvn dependency:resolve

The Java Classes

Create DuckDBConnection.java at src/main/java/com/example/duckdb/:

    Java
   
 

   package com.example.duckdb;

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;

public class DuckDBConnection {

    private static volatile Connection instance;

    private DuckDBConnection() {}

    public static Connection getInstance() throws SQLException {
        if (instance == null || instance.isClosed()) {
            synchronized (DuckDBConnection.class) {
                if (instance == null || instance.isClosed()) {
                    instance = DriverManager.getConnection("jdbc:duckdb:");
                }
            }
        }
        return instance;
    }
}
  

The volatile keyword ensures the instance is visible across threads without full synchronization on every read. The jdbc:duckdb: connection string with no path gives us an in-memory DuckDB instance.

Now create DuckDBProcedure.java in the same package:

    Java
   
 

   package com.example.duckdb;

import org.neo4j.procedure.Description;
import org.neo4j.procedure.Name;
import org.neo4j.procedure.Procedure;

import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.ResultSetMetaData;
import java.sql.Statement;
import java.sql.Types;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Stream;

public class DuckDBProcedure {

    public static class QueryResult {
        public Map<String, Object> row;

        public QueryResult(Map<String, Object> row) {
            this.row = row;
        }
    }

    @Procedure(name = "duckdb.query")
    @Description("Run a DuckDB SQL query and return results as rows")
    public Stream<QueryResult> query(@Name("sql") String sql) throws Exception {
        List<QueryResult> results = new ArrayList<>();

        synchronized (DuckDBConnection.class) {
            Connection conn = DuckDBConnection.getInstance();
            try (Statement stmt = conn.createStatement()) {
                boolean hasResultSet = stmt.execute(sql);

                if (hasResultSet) {
                    try (ResultSet rs = stmt.getResultSet()) {
                        ResultSetMetaData meta = rs.getMetaData();
                        int columnCount = meta.getColumnCount();

                        while (rs.next()) {
                            Map<String, Object> row = new HashMap<>();
                            for (int i = 1; i <= columnCount; i++) {
                                String colName = meta.getColumnName(i);
                                int colType = meta.getColumnType(i);
                                String typeName = meta.getColumnTypeName(i);
                                Object value = mapValue(rs, i, colType, typeName);
                                row.put(colName, value);
                            }
                            results.add(new QueryResult(row));
                        }
                    }
                } else {
                    Map<String, Object> status = new HashMap<>();
                    status.put("status", "OK");
                    status.put("updateCount", (long) stmt.getUpdateCount());
                    results.add(new QueryResult(status));
                }
            }
        }

        return results.stream();
    }

    private Object mapValue(ResultSet rs, int index, int sqlType, String typeName) throws Exception {
        if (typeName != null && typeName.equalsIgnoreCase("HUGEINT")) {
            long val = rs.getLong(index);
            return rs.wasNull() ? null : val;
        }
        switch (sqlType) {
            case Types.TINYINT:
            case Types.SMALLINT:
            case Types.INTEGER: {
                long val = (long) rs.getInt(index);
                return rs.wasNull() ? null : val;
            }
            case Types.BIGINT: {
                long val = rs.getLong(index);
                return rs.wasNull() ? null : val;
            }
            case Types.FLOAT:
            case Types.REAL:
            case Types.DOUBLE:
            case Types.NUMERIC:
            case Types.DECIMAL: {
                double val = rs.getDouble(index);
                return rs.wasNull() ? null : val;
            }
            case Types.BOOLEAN:
            case Types.BIT: {
                boolean val = rs.getBoolean(index);
                return rs.wasNull() ? null : val;
            }
            case Types.NULL:
                return null;
            default:
                return rs.getString(index);
        }
    }
}
  

A few things worth noting about DuckDBProcedure:

We use stmt.execute(sql) rather than executeQuery() because DuckDB write operations like COPY don't return a ResultSet. The boolean return value tells us which path to take.
The mapValue method handles type mapping explicitly. DuckDB's HUGEINT type – used for aggregated sums — has no standard java.sql.Types mapping, so we check the type name directly. Every numeric read is followed by rs.wasNull() to avoid returning 0 for a null column.
We synchronize on DuckDBConnection.class to keep concurrent Cypher calls thread-safe against the shared connection.

Writing the Tests

We tried neo4j-harness first, which spins up a lightweight embedded Neo4j instance in JUnit — the same approach used in the Neo4j Procedure Template. In our environment, we found that the embedded server took longer than expected to become ready during the test run, which led us to a simpler alternative.

The fix is to skip neo4j-harness entirely and test the DuckDB logic directly via JDBC. Since DuckDBProcedure is a thin wrapper around a JDBC connection, we can exercise the same type mapping, null handling, and file reading logic without Neo4j in the loop at all. The tests run in under 4 seconds and are fully self-contained.

First, add the test directory structure and a src/test/resources folder to hold the test data files:

    Shell
   
   mkdir -p src/test/java/com/example/duckdb
mkdir -p src/test/resources

The test data files cover all the types and edge cases we want to verify:

    Plain Text
   
 

   src/test/resources/
├── test.csv
├── test_nulls.csv
├── test_floats.csv
├── test_booleans.csv
├── test_timezones.csv
└── test.json
  

Maven copies everything in src/test/resources to target/test-classes automatically, so the test class can resolve file paths via the classloader without hardcoding any absolute paths.

The test class at src/test/java/com/example/duckdb/DuckDBProcedureTest.java covers 12 cases:

Range queries
CSV reading
Parquet write and read
JSON reading
Aggregation returning numeric values rather than strings
Null handling
Floats
Booleans
Empty result sets
Bad SQL error surfacing
In-memory table persistence across calls
DuckDB-side JOIN between two files

Each test mirrors something we verified manually in Neo4j.

File paths are resolved portably so the tests work regardless of where the project is cloned:

    Java
   
   private String resourcePath(String filename) {
    return getClass().getClassLoader().getResource(filename).getPath();
}

The Parquet test is the one exception — since DuckDB generates the Parquet file rather than reading a pre-existing one, it writes to a JUnit @TempDir that's cleaned up automatically after the test run.

Run the full suite with:

    Shell
   
   mvn test

All 12 tests should pass in a few seconds. Running mvn package will also run the tests before building the fat jar, so any regression in the DuckDB logic is caught before the jar is deployed.

Building and Installing

Build the fat jar:

    Shell
   
   mvn package -DskipTests

Any warning about MANIFEST.MF from the shade plugin is harmless. The jar lands at target/neo4j-duckdb-procedure-1.0.0.jar.

Copy it to your Neo4j Desktop plugins folder. You can find the path in Neo4j Desktop under the three-dot menu for your database instance, then Open Folder > Plugins. Then copy the jar:

    Shell
   
   cp target/neo4j-duckdb-procedure-1.0.0.jar "/path/to/your/neo4j/plugins/"

Open neo4j.conf in the conf folder alongside plugins and add:

    Plain Text
   
   dbms.security.procedures.unrestricted=duckdb.*

Restart the database. Confirm the procedure is registered using Query from the left-hand navigation pane:

    Cypher
   
   SHOW PROCEDURES YIELD name
WHERE name STARTS WITH 'duckdb'
RETURN name

You should see duckdb.query.

If the procedure doesn't appear after restarting, also add the following in neo4j.conf:

    Plain Text
   
   dbms.security.procedures.allowlist=duckdb.*

Some Neo4j configurations require procedures to be explicitly allowlisted before they're loaded at startup.

Putting It to Work

The test data files used here are also available in src/test/resources in the project repository.

Let's start simple. DuckDB has a built-in range function we can use without any external data:

    Cypher
   
   CALL duckdb.query("SELECT range AS num FROM range(1, 6)")
YIELD row
RETURN row.num AS num

Five rows should be returned, confirming that DuckDB is running inside Neo4j. Note that DuckDB's range() function is exclusive of the upper bound, so range(1, 6) returns 1 through 5.

Now let's read a CSV file. Create a test file at /tmp/test.csv:

    Plain Text
   
 

   city,country,population
London,UK,9648110
Paris,France,2161000
Berlin,Germany,3645000
Tokyo,Japan,13960000
New York,USA,8336817
  

Query it from Cypher:

    Cypher
   
   CALL duckdb.query("SELECT city, country, population FROM read_csv_auto('/tmp/test.csv') ORDER BY population DESC")
YIELD row
RETURN row.city AS city, row.country AS country, row.population AS population

DuckDB reads and parses the CSV; Neo4j receives the result. We can also write the CSV out as a Parquet file:

    Cypher
   
   CALL duckdb.query("COPY (SELECT * FROM read_csv_auto('/tmp/test.csv')) TO '/tmp/test.parquet' (FORMAT PARQUET)")
YIELD row
RETURN row

And read it back:

    Cypher
   
   CALL duckdb.query("SELECT city, country, population FROM read_parquet('/tmp/test.parquet') ORDER BY population DESC")
YIELD row
RETURN row.city AS city, row.country AS country, row.population AS population

DuckDB handles JSON just as easily. Create /tmp/test.json:

    JSON
   
 

   [
  {"city": "London", "country": "UK", "population": 9648110},
  {"city": "Paris", "country": "France", "population": 2161000},
  {"city": "Berlin", "country": "Germany", "population": 3645000},
  {"city": "Tokyo", "country": "Japan", "population": 13960000},
  {"city": "New York", "country": "USA", "population": 8336817}
]
  

And read it:

    Cypher
   
   CALL duckdb.query("SELECT city, country, population FROM read_json_auto('/tmp/test.json') ORDER BY population DESC")
YIELD row
RETURN row.city AS city, row.country AS country, row.population AS population

We can also join two files entirely inside DuckDB before the result reaches Cypher. Create /tmp/test_timezones.csv:

    Plain Text
   
 

   city,timezone
London,GMT
Paris,CET
Berlin,CET
Tokyo,JST
New York,EST
  

Then join it against the population data:

    Cypher
   
   CALL duckdb.query("SELECT p.city, p.population, t.timezone FROM read_csv_auto('/tmp/test.csv') p JOIN read_csv_auto('/tmp/test_timezones.csv') t ON p.city = t.city ORDER BY p.population DESC")
YIELD row
RETURN row.city AS city, row.population AS population, row.timezone AS timezone

The join happens entirely inside DuckDB. Neo4j just receives the final result set.

Now for the most interesting demo: combining DuckDB analytics with Neo4j graph data in a single query. First, create some graph nodes:

    Cypher
   
 

   UNWIND [
  {city: "London", continent: "Europe"},
  {city: "Paris", continent: "Europe"},
  {city: "Berlin", continent: "Europe"},
  {city: "Tokyo", continent: "Asia"},
  {city: "New York", continent: "North America"}
] AS data
MERGE (c:City {name: data.city})
SET c.continent = data.continent
  

Now join DuckDB file data with Neo4j graph data by city name:

    Cypher
   
 

   CALL duckdb.query("SELECT city, population FROM read_parquet('/tmp/test.parquet')")
YIELD row
MATCH (c:City {name: row.city})
RETURN c.name AS city, c.continent AS continent, row.population AS population
ORDER BY row.population DESC
  

And aggregate by continent — population figures from DuckDB, continent groupings from the graph:

    Cypher
   
 

   CALL duckdb.query("SELECT city, population FROM read_parquet('/tmp/test.parquet')")
YIELD row
MATCH (c:City {name: row.city})
RETURN c.continent AS continent,
       count(c) AS city_count,
       sum(row.population) AS total_population
ORDER BY total_population DESC
  

That's two engines contributing to a single result: DuckDB handles the file analytics, Neo4j handles the graph traversal and aggregation.

What We Learned

Building this wasn't entirely smooth, and the bumps are worth documenting.

XML comments in a Maven POM can't contain double dashes anywhere except at the closing -->. That's a quirk of XML parsing that's easy to miss.
DuckDB's HUGEINT type – used for the result of aggregation functions like SUM – has no corresponding entry in the standard java.sql.Types constants. Our initial type mapping switch statement silently fell through to the string default, returning numeric values as strings. The fix is to check ResultSetMetaData.getColumnTypeName() by name before entering the switch.
Write operations like COPY can't go through executeQuery() since they don't return a ResultSet. Switching to stmt.execute() and checking the boolean return value handles both read and write paths cleanly.
Null numeric values in JDBC return 0 from getInt() and getLong(), not null. You must call rs.wasNull() immediately after each numeric read to distinguish a genuine zero from a missing value.

Comparing Approaches

It's also worth noting that APOC (Awesome Procedures on Cypher) Extended's apoc.load.jdbc procedure supports DuckDB via its JDBC driver and apoc.jdbc.analytics goes further by letting you export graph data into a temporary DuckDB table for analytical queries. These are complementary approaches worth exploring alongside the custom procedure built here.

The following table compares the three ways to use DuckDB with Neo4j, each with a different integration style.

Approach	How It Works	Direction	Persistent State	Requires
Custom procedure (this article)	DuckDB runs in-process inside the Neo4j JVM as a singleton connection	Files and SQL into Neo4j	Yes, across Cypher calls	Custom Java plugin
APOC apoc.load.jdbc	DuckDB treated as an external JDBC data source; driver jar dropped into plugins folder	Files and SQL into Neo4j	No	APOC Extended + DuckDB JDBC jar
APOC apoc.jdbc.analytics	Graph data exported into a temporary DuckDB table; analytical SQL runs against it	Graph data into DuckDB for analytics	No	APOC Extended + DuckDB JDBC jar

Summary

We started with a simple question: could DuckDB be embedded in Neo4j the way pg_duckdb embeds it in Postgres? The answer is not natively, but closer than you might expect.

With two Java classes, a fat jar, and a few lines of Neo4j configuration, we built a working duckdb.query procedure that runs DuckDB SQL directly from Cypher. Along the way we handled CSV, Parquet and JSON files, DuckDB-side joins, cross-engine aggregations with Neo4j graph data and a handful of type mapping edge cases that the 12-test suite now catches automatically.

There's plenty of room to grow this further. File-backed DuckDB, a proper connection pool, cleaner error handling, and possibly a maintained open-source plugin are all natural next steps. But the foundation is there.

What we've built here is a proof of concept — it works, and the tests confirm it, but anyone taking it further should treat robustness, connection pooling, and error handling as open items before putting it near production data.