DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workkloads.

Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • Auto-Instrumentation in Azure Application Insights With AKS
  • Deploying a Scala Play Application to Heroku: A Step-by-Step Guide
  • Why Mocking Sucks
  • Moving PeopleSoft ERP Data Between Databases With Data Mover Scripts

Trending

  • Start Coding With Google Cloud Workstations
  • Why I Started Using Dependency Injection in Python
  • The Role of Functional Programming in Modern Software Development
  • Testing SingleStore's MCP Server
  1. DZone
  2. Coding
  3. Java
  4. Fault Injection Testing - First Steps with JBoss Byteman

Fault Injection Testing - First Steps with JBoss Byteman

By 
Len DiMaggio user avatar
Len DiMaggio
·
Oct. 16, 09 · News
Likes (0)
Comment
Save
Tweet
Share
17.1K Views

Join the DZone community and get the full member experience.

Join For Free
Fault injection testing[1] is a very useful element of a comprehensive test strategy in that it enables you to concentrate on an area that can be difficult to test; the manner in which the application under test is able to handle exceptions.

It's always possible to perform exception testing in a black box mode, where you set up external conditions that will cause the application to fail, and then observe those application failures. Setting and automating (and reproducing) these such as these can, however, be time consuming. (And a pain in the neck, too!)

JBoss Byteman

I recently found a bytecode injection tool that makes it possible to automate fault injection tests. JBoss Byteman[2] is an open-source project that lets you write scripts in a Java-like syntax to insert events, exceptions, etc. into application code.

Byteman version 1.1.0 is available for download from: http://www.jboss.org/byteman - the download includes a programmer's guide. There's also a user forum for asking questions here: http://www.jboss.org/index.html?module=bb&op=viewforum&f=310, and a jboss.org JIRA project for submitted issues and feature requests here: https://jira.jboss.org/jira/browse/BYTEMAN

A Simple Example

The remainder of this post describes a simple example, on the scale of the classic "hello world" example, of using Byteman to insert an exception into a running application.

Let's start by defining the exception that we will inject into our application:
  package sample.byteman.test;

/**
* Simple exception class to demonstrate fault injection with byteman
*/

public class ApplicationException extends Exception {

private static final long serialVersionUID = 1L;
private int intError;
private String theMessage = "hello exception - default string";

public ApplicationException(int intErrNo, String exString) {
intError = intErrNo;
theMessage = exString;
}

public String toString() {
return "**********ApplicationException[" + intError + " " + theMessage + "]**********";
}

} /* class */


There's nothing complicated here, but note the string that is passed to the exception constructor at line 13.

Now, let's define our application class:
  package sample.byteman.test;

/**
* Simple class to demonstrate fault injection with byteman
*/

public class ExceptionTest {

public void doSomething(int counter) throws ApplicationException {
System.out.println("called doSomething(" + counter + ")");
if (counter > 10) {
throw new ApplicationException(counter, "bye!");
}
System.out.println("Exiting method normally...");
} /* doSomething() */

public static void main(String[] args) {
ExceptionTest theTest = new ExceptionTest();
try {
for (int i = 0; i < 12; i ++) {
theTest.doSomething (i);
}
} catch (ApplicationException e) {
System.out.println("caught ApplicationException: " + e);
}
}

} /* class*/


The application instantiates an instance of ExceptionTest at line 18, then runs the doSomething method in a loop until a counter is greater then 10. Then it raises the exception that we defined earlier. When we run the application, we see this output:
java -classpath bytemanTest.jar sample.byteman.test.ExceptionTest
called doSomething(0)
Exiting method normally...
called doSomething(1)
Exiting method normally...
called doSomething(2)
Exiting method normally...
called doSomething(3)
Exiting method normally...
called doSomething(4)
Exiting method normally...
called doSomething(5)
Exiting method normally...
called doSomething(6)
Exiting method normally...
called doSomething(7)
Exiting method normally...
called doSomething(8)
Exiting method normally...
called doSomething(9)
Exiting method normally...
called doSomething(10)
Exiting method normally...
called doSomething(11)
caught ApplicationException: **********ApplicationException[11 bye!]**********

OK. Nothing too exciting so far. Let's make things more interesting by scripting a Byteman rule to inject an exception before the doSomething method has a chance to print any output. Our Byteman script looks like this:
   #
# A simple script to demonstrate fault injection with byteman
#
RULE Simple byteman example - throw an exception
CLASS sample.byteman.test.ExceptionTest
METHOD doSomething(int)
AT INVOKE PrintStream.println
BIND buffer = 0
IF TRUE
DO throw sample.byteman.test.ApplicationException(1,"ha! byteman was here!")
ENDRULE


  • Line 4 - RULE defines the start of the RULE. The following text on this line is not executed
  • Line 5 - Reference to the class of the application to receive the injection
  • Line 6 - And the method in that class. Note that since if we had written this line as "METHOD doSomething", the rule would have matched any signature of the soSomething method
  • Line 7 - Our rule will fire when the PrintStream.println method is invoked
  • Line 8 - BIND determince values for variables which can be referenced in the rule body - in our example, the recipient of the doSomething method call that triggered the rule, is identified by the parameter reference $0
  • Line 9 - A rule has to include an IF clause - in our example, it's always true
  • Line 10 - When the rule is triggered, we throw an exception - note that we supply a string to the exception constructor
Now, before we try to run this run, we should check the its syntax. To do this, we build our application into a .jar (bytemanTest.jar in our case) and use bytemancheck.sh
sh bytemancheck.sh -cp bytemanTest.jar byteman.txt
checking rules in sample_byteman.txt
TestScript: parsed rule Simple byteman example - throw an exception
RULE Simple byteman example - throw an exception
CLASS sample.byteman.test.ExceptionTest
METHOD doSomething(int)
AT INVOKE PrintStream.println
BIND buffer : int = 0
IF TRUE
DO throw (1"ha! byteman was here!")

TestScript: checking rule Simple byteman example - throw an exception
TestScript: type checked rule Simple byteman example - throw an exception

TestScript: no errors

Once we get a clean result, we can run the application with Byteman. To do this, we run the application and specify an extra argument to the java command. Note that Byteman requires JDK 1.6 or newer.
java -javaagent:/opt/Byteman_1_1_0/build/lib/byteman.jar=script:sample_byteman.txt -classpath bytemanTest.jar sample.byteman.test.ExceptionTest

And the result is:
caught ApplicationException: **********ApplicationException[1 ha! byteman was here!]**********



Now that the Script Works, Let's Improve it!

Let's take a closer look and how we BIND to a method parameter. If we change the script to read as follows:
   #
# A simple script to demonstrate fault injection with byteman
#
RULE Simple byteman example - throw an exception
CLASS sample.byteman.test.ExceptionTest
METHOD doSomething(int)
AT INVOKE PrintStream.println
BIND counter = $1
IF TRUE
DO throw sample.byteman.test.ApplicationException(counter,"ha! byteman was here!")
ENDRULE


In line 8, the BIND clause now refers to the int method parameter by index using the syntax $1. This change makes the value available inside the rule body by enabling us to use the name "counter." The value of counter is then supplied as the argument to the constructor for the ApplicationException class. This new version of the rule demonstrates shows how we can use local state as derived from the trigger method
to construct our exception object.

But wait there's more! Let's use the "counter" value as a counter.

It's useful to be able to force an exception the first time a method is called. But, it's even more useful to be able to force an exception at a selected invocation of a method. Let's add a test for that counter value to the script:
   #
# A simple script to demonstrate fault injection with byteman
#
RULE Simple byteman example 2 - throw an exception at 3rd call
CLASS sample.byteman.test.ExceptionTest
METHOD doSomething(int)
AT INVOKE PrintStream.println
BIND counter = $1
IF counter == 3
DO throw sample.byteman.test.ApplicationException(counter,"ha! byteman was here!")
ENDRULE


In line 9, we've changed the IF clause to make use of the counter value. When we run the test with this script, the first 2 calls to doSomething succeed, but the third one fails.



One Last Thing - Changing the Script for a Running Process

So far, so good. We've been able to inject a fault/exception into our running application, and even specify which iteration of a loop in which it happens. Suppose, however, we want to change a value in a byteman script, while the application is running? No problem! Here's how.

First, we need to alter our application so that it can run for a long enough time for us to alter the byteman script. Here's a modified version of the doSomething method that waits for user input:
    public void doSomething(int counter) throws ApplicationException {

BufferedReader lineOfText = new BufferedReader(new InputStreamReader(System.in));
try {
System.out.println("Press <return>");
String textLine = lineOfText.readLine();
} catch (IOException e) {
e.printStackTrace();
}

System.out.println("called doSomething(" + counter + ")");
if (counter > 10) {
throw new ApplicationException(counter, "bye!");
}
System.out.println("Exiting method normally...");
}


If we run this version of the application, we'll see output like this:
Press <return>

called doSomething(0)
Exiting method normally...
Press <return>

called doSomething(1)
Exiting method normally...
Press <return>

called doSomething(2)
Exiting method normally...
caught ApplicationException: **********ApplicationException[3 ha! byteman was here!]**********


Let's run the application again, but this time, don't press <return>. While the application is waiting for input, create a copy of the byteman script. In this copy, change the IF clause to have a loop counter set to a different value, say '5.' Then, open up a second command shell window and enter this command:

Byteman_1_1_0/bin/submit.sh sample_byteman_changed.txt


Then, return to the first command shell window and start pressing return, and you'll see this output:
Press <return>
redefining rule Simple byteman example - throw an exception

called doSomething(0)
Exiting method normally...
Press <return>

called doSomething(1)
Exiting method normally...
Press <return>

called doSomething(2)
Exiting method normally...
Press <return>

called doSomething(3)
Exiting method normally...
Press <return>

called doSomething(4)
Exiting method normally...
caught ApplicationException: **********ApplicationException[5 ha! byteman was here!]**********


So, we were able to alter the value in the original byteman script, without stopping the application under test!


Pitfalls Along the Way

Some of the newbee mistakes that I made along the way were:
  • Each RULE needs an IF clause - even if you want the rule to always fire
  • The methods referenced in a RULE cannot be static - if they are static, then there is no $0 (aka this) to reference
  • Yes, I had several errors and some typos the first few times I tried this. A syntax checker is always my best friend. ;-)
Closing Thoughts

With this simple example, we're able to inject injections into a running application in an easily automated/scripted manner. But, We've only scratched the surface with Byteman. In subsequent posts, I'm hoping to explore using Byteman to cause more widespread havoc in software testing.

References

[1] http://en.wikipedia.org/wiki/Fault_injection

[2] http://www.jboss.org/byteman

(Special thanks to Andrew Dinn for his help! ;-)
Fault injection application JBoss

Published at DZone with permission of Len DiMaggio, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Auto-Instrumentation in Azure Application Insights With AKS
  • Deploying a Scala Play Application to Heroku: A Step-by-Step Guide
  • Why Mocking Sucks
  • Moving PeopleSoft ERP Data Between Databases With Data Mover Scripts

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!