DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Coding
  3. Java
  4. Speed Up with Fast Java and File Serialization

Speed Up with Fast Java and File Serialization

Jakub Kubrynski user avatar by
Jakub Kubrynski
·
Aug. 06, 12 · Interview
Like (0)
Save
Tweet
Share
39.32K Views

Join the DZone community and get the full member experience.

Join For Free

Since the first version of Java, day-by-day many developers have been trying to achieve at least as good of performance as in C/C++. JVM vendors are doing their best by implementing some new JIT algorithms, but there is still a lot to do, especially in how we use Java.

For example, there is a lot to win in objects<->file serialization - notably in writing/reading objects that can readily fit in the memory. I’ll try to shed some light on that topic.

All the tests were executed on the simple object shown below: 

public class TestObject implements Serializable {

  private long longVariable;
  private long[] longArray;
  private String stringObject;
  private String secondStringObject; //just for testing nulls

  /* getters and setters */
}

To be more concise I’ll show only the write methods (though the other way is quite similar, too). Full source code is available on my GitHub (http://github.com/jkubrynski/serialization-tests).


The most standard java serialization (that we all start from) looks like this:

    public void testWriteBuffered(TestObject test, String fileName) throws IOException {
      ObjectOutputStream objectOutputStream = null;
      try {
        FileOutputStream fos = new FileOutputStream(fileName);
        BufferedOutputStream bos = new BufferedOutputStream(fos);
        objectOutputStream = new ObjectOutputStream(bos);
        objectOutputStream.writeObject(test);
      } finally {
        if (objectOutputStream != null) {
          objectOutputStream.close();
        }
      }
    } 

The easiest way to speed up the standard serialization is to use the RandomAccessFile object:

    public void testWriteBuffered(TestObject test, String fileName) throws IOException {
      ObjectOutputStream objectOutputStream = null;
      try {
        RandomAccessFile raf = new RandomAccessFile(fileName, "rw");
        FileOutputStream fos = new FileOutputStream(raf.getFD());
        objectOutputStream = new ObjectOutputStream(fos);
        objectOutputStream.writeObject(test);
      } finally {
        if (objectOutputStream != null) {
          objectOutputStream.close();
        }      
    } 

The more sophisticated technique is to use the Kryo framework. The difference between the old and the new version is vast. I’ve checked both. Because the performance comparison doesn’t show any spectacular dissimilarities, I’ll focus on the second version as it’s much more user-friendly and even somewhat faster.

    private static Kryo kryo = new Kryo(); // version 2.x

    public void testWriteBuffered(TestObject test, String fileName) throws IOException {
      Output output = null;
      try {
        RandomAccessFile raf = new RandomAccessFile(fileName, "rw");
        output = new Output(new FileOutputStream(raf.getFD()), MAX_BUFFER_SIZE);
        kryo.writeObject(output, test);
      } finally {
        if (output != null) {
          output.close();
        }
      }
    } 

The last option is a solution inspired by Martin Thompson’s article (http://mechanical-sympathy.blogspot.gr/2012/07/native-cc-like-performance-for-java.html). It shows how to play with the memory in the C++ way and in the Java :)

    public void testWriteBuffered(TestObject test, String fileName) throws IOException {
      RandomAccessFile raf = null;
      try {
        MemoryBuffer memoryBuffer = new MemoryBuffer(MAX_BUFFER_SIZE);
        raf = new RandomAccessFile(fileName, "rw");
        test.write(memoryBuffer);
        raf.write(memoryBuffer.getBuffer());
      } catch (IOException e) {
        if (raf != null) {
          raf.close();
        }
      }
    } 

TestObject write method is shown below:

  public void write(MemoryBuffer unsafeBuffer) {
    unsafeBuffer.putLong(longVariable);
    unsafeBuffer.putLongArray(longArray);
    // we support nulls
    boolean objectExists = stringObject != null;
    unsafeBuffer.putBoolean(objectExists);
    if (objectExists) {
      unsafeBuffer.putCharArray(stringObject.toCharArray());
    }
    objectExists = secondStringObject != null;
    unsafeBuffer.putBoolean(objectExists);
    if (objectExists) {
      unsafeBuffer.putCharArray(secondStringObject.toCharArray());
    }
  }   

Direct memory buffer class (shortened, just to show the idea):

public class MemoryBuffer {
  // getting Unsafe by reflection
  public static final Unsafe unsafe = UnsafeUtil.getUnsafe();

  private final byte[] buffer;

  private static final long byteArrayOffset = unsafe.arrayBaseOffset(byte[].class);
  private static final long longArrayOffset = unsafe.arrayBaseOffset(long[].class);
  /* other offsets */

  private static final int SIZE_OF_LONG = 8;
  /* other sizes */

  private long pos = 0;

  public MemoryBuffer(int bufferSize) {
    this.buffer = new byte[bufferSize];
  }

  public final byte[] getBuffer() {
    return buffer;
  }

  public final void putLong(long value) {
    unsafe.putLong(buffer, byteArrayOffset + pos, value);
    pos += SIZE_OF_LONG;
  }

  public final long getLong() {
    long result = unsafe.getLong(buffer, byteArrayOffset + pos);
    pos += SIZE_OF_LONG;
    return result;
  }

  public final void putLongArray(final long[] values) {
    putInt(values.length);
    long bytesToCopy = values.length << 3;
    unsafe.copyMemory(values, longArrayOffset, buffer, byteArrayOffset + pos, bytesToCopy);
    pos += bytesToCopy;
  }


  public final long[] getLongArray() {
    int arraySize = getInt();
    long[] values = new long[arraySize];
    long bytesToCopy = values.length << 3;
    unsafe.copyMemory(buffer, byteArrayOffset + pos, values, longArrayOffset, bytesToCopy);
    pos += bytesToCopy;
    return values;
  }

  /* other methods */
} 

Results of many hours of the Caliper’s runs are shown below:

 Full trip [ns] Standard deviation [ns] 
Standard  207307 2362
Standard on RAF 42661 733
KRYO 1.x  12027 112
KRYO 2.x 11479 259
Unsafe 8554 91

In the end we can draw a few conclusions:
  • Unsafe serialization is greater than 23 times faster than standard use of java.io.Serializable
  • Use of RandomAccessFile can speed up standard buffered serialization by almost 4 times
  • Kryo-dynamic serialization is about 35% slower than the hand-implemented direct buffer.

Finally, as we can see, there is still no golden hammer. For a lot of us, gaining 3000 ns (0.003ms) is not worth writing custom implementations for every object we want to serialize with files. And for standard solutions we’ll mostly choose Kryo. Nevertheless, in low-latency systems, where 100ns seems like an eternity, the choice will be completely different.

 

Serialization Java (programming language)

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • PHP vs React
  • Spring Cloud: How To Deal With Microservice Configuration (Part 1)
  • Why Open Source Is Much More Than Just a Free Tier
  • Why Does DevOps Recommend Shift-Left Testing Principles?

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: