DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
The Latest "Software Integration: The Intersection of APIs, Microservices, and Cloud-Based Systems" Trend Report
Get the report
  1. DZone
  2. Culture and Methodologies
  3. Career Development
  4. Spring Batch Job Restartability

Spring Batch Job Restartability

Developers should strive to be fundamentally and completely ordinary when writing code. Simplicity will lend understanding to the future.

Ryan McCullough user avatar by
Ryan McCullough
·
Apr. 07, 17 · Opinion
Like (2)
Save
Tweet
Share
16.86K Views

Join the DZone community and get the full member experience.

Join For Free

First of all, I’d like to give a big thank you to the wonderful folks from Spring who have given countless hours of their time to ensure the viability of Spring Batch jobs, as well as the seemingly magical ability to issue restart on a job! Thank you for this elegant toolset that permits us to tear through massive datasets while enabling us to dust ourselves off when we fall down!

While acknowledging that I still have much to learn, I’d like to share my well-learned lessons in the world of restartability. This post will include how to identify improper usage of Spring Batch’s Step and Job ExecutionContext as well as how to write good, wholesome components for Spring Batch.

Statefulness!

Statefulness is basically fancy talk for beans that have global variables that change.

As an example, take a one-dollar bill. It would be considered stateless, as its value is constant. On the other hand, take a stock like Google; its price fluctuates and its value would be considered variable (or stateful).

ExecutionContext

To maintain statefulness, Spring gives us access to the ExecutionContext for both Step and Job so that we may transact information that our job needs to stay on track and complete properly.

Anything stateful in your batch code base threatens the viability of its restartability. Processors, readers, writers, or anything that gets used by your batch operation should be considered at risk when stateful.

What Information Could Be Maintained in the ExecutionContext?

Technically, I guess any serializable object could be placed into an ExecutionContext anytime, but I’d say that it is a dangerous way to think. ExecutionContext updates should be handled in a very transactional way.

What Information Should Be Maintained in the ExecutionContext?

I would recommend only keeping primitive/pseudo-primitive simple values here. If you want to go to sleep easily at night, I would also recommend only writing these values through an ItemProcessor or a method annotated with @BeforeStep or @AfterStep.

What Should Not Happen?

The ExecutionContext should not be introduced and passed around in core business logic. Context values should not be updated in the middle of step execution. Additionally, you should avoid introducing a mutable value holder object into an ExecutionContext, as its reference can easily corrupt the values behind transactional boundaries.

When I see these types of examples in code, I consider them a threat to the application’s restartability and refuse to certify that application’s restartability.

In general, there is not a one-size-fits-all approach to ensuring that your job code is written in a way that guarantees that stateful information has been handled properly. However, I will tell you that you need to be thinking about:

  • How transactions are completed (distributed, partitioned, multithreaded, etc.).
  • How is chunk progress tracked?
  • How are your reads sorted/grouped?
  • What information will be needed at restart time?

Here is a general practice example for updating job relevant stateful information:

import org.springframework.batch.item.*;
import org.springframework.context.annotation.Scope;
import org.springframework.stereotype.Component;
/*This could be a reader, or a writer, or maybe a processor... 
  you need to identify when and where it is appropriate to 
  perform these tracking activities. Remember to think about restarts! */
@Component
@StepScope
public class FooComponent implements ItemStream{ 

// a perfectly acceptable way to read a value from an ExecutionContext from anywhere!
@Value(“#stepExecutionContext[‘fooStatefulCount’]”) 
long statefulCount = 0; // a read count perhaps?

public static final String KEY = "bar";
public void open(ExecutionContext cxt) throws ItemStreamException {
    cxt.put(KEY, statefulCount);
}
public void update(ExecutionContext cxt) throws ItemStreamException {
    cxt.put(KEY, statefulCount);
}   
public void close() throws ItemStreamException {}
}

If you want a more comprehensive example, go look through the open method in AbstractItemCountingItemStreamItemReader!

Final Thoughts

My end advice is for other developers to strive to be fundamentally and completely ordinary when writing code. Simplicity will lend understanding to the future, and subsequently, the business owners will cherish your gift of an application that is largely free of technical debt.

Spring Framework Spring Batch career

Published at DZone with permission of Ryan McCullough, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Demystifying the Infrastructure as Code Landscape
  • Create a CLI Chatbot With the ChatGPT API and Node.js
  • Steel Threads Are a Technique That Will Make You a Better Engineer
  • Integrate AWS Secrets Manager in Spring Boot Application

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: