DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Last call! Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • How to Convert XLSX to CSV in Java
  • Jakarta NoSQL 1.0: A Way To Bring Java and NoSQL Together
  • Handling Embedded Data in NoSQL With Java
  • Simplify NoSQL Database Integration in Java With Eclipse JNoSQL 1.1.3

Trending

  • Artificial Intelligence, Real Consequences: Balancing Good vs Evil AI [Infographic]
  • Ethical AI in Agile
  • Agentic AI for Automated Application Security and Vulnerability Management
  • From Zero to Production: Best Practices for Scaling LLMs in the Enterprise
  1. DZone
  2. Data Engineering
  3. Databases
  4. Efficiently Producing and Consuming CSV in Java Using HTTP-RPC

Efficiently Producing and Consuming CSV in Java Using HTTP-RPC

Want to learn how to produce and consume a CSV in Java? Check out this tutorial on producing CSVs, using HTTP-RPC.

By 
Greg Brown user avatar
Greg Brown
·
Aug. 14, 18 · Tutorial
Likes (2)
Comment
Save
Tweet
Share
14.1K Views

Join the DZone community and get the full member experience.

Join For Free

In a previous article, I discussed how the open-source HTTP-RPC framework can be used to efficiently transform JDBC query results into JSON. However, while JSON is an extremely common and well-supported data format, it may, in some cases, be preferable to return a CSV document instead. Because field keys are specified only at the beginning of a CSV document rather than being duplicated for every record, CSV generally requires less bandwidth than JSON. Additionally, consumers can begin processing CSV as soon as the first record arrives, rather than waiting for the entire document to download.

The previous article demonstrated how HTTP-RPC's ResultSetAdapter and JSONEncoder classes could be used to easily generate a JSON response from a SQL query on this table, taken from the MySQL sample database:

CREATE TABLE pet (
    name VARCHAR(20),
    owner VARCHAR(20),
    species VARCHAR(20), 
    sex CHAR(1), 
    birth DATE, 
    death DATE
);


The following service method returns the same results encoded as CSV. The only difference in this version is the use of HTTP-RPC's CSVEncoder class instead of JSONEncoder:

@RequestMethod("GET")
public void getPets(String owner) throws SQLException, IOException {
    try (Connection connection = DriverManager.getConnection(DB_URL)) {
        Parameters parameters = Parameters.parse("SELECT name, species, sex, birth FROM pet WHERE owner = :owner");

        parameters.put("owner", owner);

        try (PreparedStatement statement = connection.prepareStatement(parameters.getSQL())) {
            parameters.apply(statement);

            try (ResultSet resultSet = statement.executeQuery()) {
                CSVEncoder csvEncoder = new CSVEncoder();

                csvEncoder.writeValue(new ResultSetAdapter(resultSet), getResponse().getOutputStream());
            }
        }
    } finally {
        getResponse().flushBuffer();
    }
}


The response might look something like this, where each record in the document represents a row from the result set (date values are represented using epoch time):

"name","species","sex","birth"
"Claws","cat","m",763880400000
"Chirpy","bird","f",905486400000
"Whistler","bird",,881643600000


Consuming a CSV Response

On the other end, the HTTP-RPC's CSVDecoder class can be used to efficiently process the contents of a CSV document. Rather than reading the entire payload into memory and returning the data as a random-access list of map values, CSVDecoder returns a cursor over the records in the document. This allows a client to read records as they are being produced, reducing memory consumption and improving throughput.

For example, the following client code could be used to consume the response generated by the web service:

WebServiceProxy webServiceProxy = new WebServiceProxy("GET", new URL("http://localhost:8080/httprpc-test/pets"));

webServiceProxy.getArguments().put("owner", "Gwen");
webServiceProxy.getArguments().put("format", "csv");

webServiceProxy.invoke((inputStream, contentType) -> {
    CSVDecoder csvDecoder = new CSVDecoder();

    CSVDecoder.Cursor pets = csvDecoder.readValues(inputStream);

    for (Map<String, String> pet : pets) {
        System.out.println(String.format("%s is a %s", pet.get("name"), pet.get("species")));
    }

    return null;
});


This code produces the following output:

Claws is a cat
Chirpy is a bird
Whistler is a bird


Additionally, the adapt() method of the CSVDecoder.Cursor class can be used to facilitate typed iteration of CSV data. This method produces an Iterable sequence of values of a given type representing the rows in the document. The returned adapter uses dynamic proxy invocation to map properties declared by the interface to map values in the cursor. A single proxy instance is used for all rows to minimize heap allocation.

For example, the following interface might be used to model the pet records shown above:

public interface Pet {
    public String getName();
    public String getOwner();
    public String getSpecies();
    public String getSex();
    public Date getBirth();
}


This code uses adapt() to create an iterable sequence of Pet values from the CSV response:

CSVDecoder csvDecoder = new CSVDecoder();

CSVDecoder.Cursor pets = csvDecoder.readValues(inputStream);

for (Pet pet : pets.adapt(Pet.class)) {
    System.out.println(String.format("%s is a %s", pet.getName(), pet.getSpecies()));
}


The output is identical to the previous example; however, in this case, the fields of each record are retrieved in a type-safe manner (a feature whose value becomes much more obvious when handling CSV documents containing numeric or boolean values).

Final Thoughts

This article provided an example of how HTTP-RPC's CSVEncoder and CSVDecoder classes can be used to efficiently stream and process JDBC query results as CSV. For more information, see the project README.

CSV Database Java (programming language)

Opinions expressed by DZone contributors are their own.

Related

  • How to Convert XLSX to CSV in Java
  • Jakarta NoSQL 1.0: A Way To Bring Java and NoSQL Together
  • Handling Embedded Data in NoSQL With Java
  • Simplify NoSQL Database Integration in Java With Eclipse JNoSQL 1.1.3

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!