Over a million developers have joined DZone.

Developing a Reusable File: Archival Schedulable Job

· DevOps Zone

The DevOps zone is brought to you in partnership with Sonatype Nexus. The Nexus suite helps scale your DevOps delivery with continuous component intelligence integrated into development tools, including Eclipse, IntelliJ, Jenkins, Bamboo, SonarQube and more. Schedule a demo today

DevOps and full-stack have been popular topics in our industry for a number of years now. Unfortunately, they don’t always mean the same thing to every organization or even individuals within an organization. Rather than debate what they are or what they should be, this post will work through a solution created by the development team to either facilitate a frequently executed task performed by the operations team or provide a solution worthy of use by developers functioning in a part-time operations role – file archival.

File archival needs are typically driven by a few factors – limits of physical space, business or compliance requirements for retention and information availability for support teams. Rather than develop a solution that is fixed to a given set of requirements, we want a reusable, flexible solution that can be used and adapted without a need for new development.

Configuration Parameters

This schedulable job will need to accept configuration parameters. They are:

  • Source Directory with optional filename pattern OR Source File(s)
  • Archival Directory
  • Rename pattern
  • Compress Y/N?
  • Delete original Y/N?

We will develop our job as an Obsidian schedulable job but you can adapt this easily to any scheduling platform you desire. In fact, here is the resulting Java source code available under the MIT open source licence for you do to as you wish. FileArchiveJob.java.

Job Code

The basic algorithm is as follows.

  • Iterate over files
  • Ensure archive directory exists
  • Archive file – applying compression if selected
  • Delete original if selected

Since our job is an Obsidian job, we use a source job’s results as the most flexible and powerful mechanism to get your archival list.

Here’s what our main archival method ends up looking like:

protected void processFile(Context context, File f) throws ParameterException, IOException, DBException {
  boolean gzip = Boolean.TRUE.equals(context.getConfig().getBoolean(GZIP));
  String newName = determineArchiveFilename(f);
  for (String dirPath : context.getConfig().getStringList(ARCHIVE_DIR)) {
    File dir = new File(dirPath);
    if (!dir.exists()) {
    if (!dir.isDirectory()) {
      throw new RuntimeException("Archive directory does not exist and could not be created: " + dir.getAbsolutePath());
    File archiveFile = new File(dir, newName);
    if (!Boolean.TRUE.equals(context.getConfig().getBoolean(OVERWRITE)) && archiveFile.exists()) {
      throw new RuntimeException("File already exists and overwrite is disabled: " + archiveFile.getAbsolutePath());
    InputStream src = null;
    OutputStream dest = null;
    FileOutputStream fos = null;
    try {
      src = new BufferedInputStream(new FileInputStream(f));
      fos = new FileOutputStream(archiveFile);
      if (gzip) {
        dest = new BufferedOutputStream(new GZIPOutputStream(fos));
      } else {
        dest = new BufferedOutputStream(fos);
      IOUtil.copyStream(src, dest, 4096, false);
      context.saveJobResult("archiveFile", archiveFile.getAbsolutePath());
    } finally {

And there’s a lot more this job offers. It is designed with customization in mind and has some additional optional features detailed on our wiki. Let us know what you think or take this job for a spin in your own free instance of Obsidian Scheduler.

Next up, we’ll take a look at a reusable file cleanup job.

The DevOps zone is brought to you in partnership with Sonatype Nexus. Use the Nexus Suite to automate your software supply chain and ensure you're using the highest quality open source components at every step of the development lifecycle. Get Nexus today


Published at DZone with permission of Craig Flichel, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}