DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. Deployment
  4. How I Learned to Stop Worrying and Love TestNG

How I Learned to Stop Worrying and Love TestNG

Sarah Goff-Dupont user avatar by
Sarah Goff-Dupont
·
Mar. 02, 13 · Interview
Like (1)
Save
Tweet
Share
6.16K Views

Join the DZone community and get the full member experience.

Join For Free

Think back to your early 20′s. Remember going out on Halloween or New Year’s Eve? No sooner do you walk into a bar than someone in your group starts lobbying to go to some other bar. ‘Cuz it’s gonna be way better. (In your exasperation – you just ordered a drink for pete’s sake! –  you vow that next year you’re gonna find a house party instead.)

Apologies in advance, but… I’m gonna be “that someone”. Replace New Year’s Eve with every Monday through Friday. The bar you just walked into is JUnit. That other bar you should, like, totally make a b-line for is TestNG - an open-source automated testing framework.

Now, where this analogy breaks down (assuming it ever had legs to begin with), is that unlike the human confines of singular spacio-temporal existence™, you can use both JUnit and TestNG at once. Here are 3 reasons you should.

Groups

This is hands-down my favorite thing about TestNG because it’s the most flexible way to split your tests into sub-suites. Let’s jump right to an example, inspired by Bamboo (a CI server from Atlassian). Here we see two tests around ordering sub-steps, or "Tasks", within a build job:


Each test has been assigned to groups representing functional area of the application, level of technology stack, maturity level, and whether it should be run as a post-deploy smoke test. This is incredibly powerful because you can use these assignments at run time to pull tests from disparate classes or packages and run them inside a single job. A job that runs all the API-level tests having to do with configuration functionality, for example. Or a job that runs all the tests for features that are still in development and not really expected to pass yet (TDD, anyone??).

And you guessed it: the logical next step is to create jobs that cover the fully body of tests you wish to execute, and run those jobs in parallel inside a stage. Check out this post for more implementation details.

@DataProvider

Let’s say you’re developing, oh I dunno… a continuous integration server. Let’s also say that your CI server needs to support switching between branches from build to build. There are loads of permutations you want to test, but you’re verifying the same functionality for each of them.

What’s a dev to do? Duplicate the hell outta their test code, making explicit sets of tests for each repo type/branch type combination? Oh, the humanity! But wait: up there in the sky! It’s a bird… no, it’s a plane… no! it’s @DataProvider! Here’s a simple example from TestNG’s site:


The output of that test will be: 


But wait. Junit supports parameterized tests as of v.4.4, which is essentially what this is. So why bother with TestNG? First, TestNG’s implementation is slicker and easier than JUnit’s. Second, and more importantly, the reporting in TestNG is much clearer. TestNG tells you exactly which inputs were used for each execution, making it much easier to diagnose failures.


JUnit just tells you which array the parameters came from, and their positions therein – 'bout as clear as mud.

Skipped Tests

Adherents to the D.R.Y principal (and that’s most of us) rely on @BeforeTest or @BeforeClass set-up methods. But if something goes awry in set-up, there’s no point in trying to run the actual test. We know it just gonna fail. With TestNG, the default behavior is to simply skip a test if its upstream dependency barfs. I love this for two reasons.

First, executing the actual test is a waste of time and just means you wait that much longer to find out this revision of your code no es bueno. Second, when you look at your test results, your count of skipped tests makes it immediately clear that something fundamental to your application is borked. Much faster than diving into 350 test failures and, somewhere around number 20, noticing that they’ve all failed on exactly the same call.

And it’s not just set-up methods that can trigger a skip. If you’ve declared TestA as being dependent upon TestB, and TestA fails, TestB will be skipped. Here’s what that looks like in your build logs:


Two Can Play at This Game…

Incorporating TestNG into your arsenal is easy, and again: you can use it in concert with JUnit. Add it as a dependency in your Maven POM, or as an Ant task. Then get the TestNG plugin for Eclipse or IDEA and start sprucing up your tests. Visit testng.org for details.

Among the goodies in Bamboo’s latest release is native support for parsing TestNG results - @DataProvider, skipped tests… the whole works. Just add the TestNG Parser as a final task in the jobs that execute tests. If you’re running the tests by way of Maven, Ant or Gradle, be sure to disable test reporting in those tasks so there are no conflicts. The TestNG plugin for Jenkins also supports all three features I've talked about. 

Try it out and see if you love TestNG, too!


ps: For more tips and best practices, be sure to check out our new automated testing page with all sorts of helpful info. Even automation gurus are sure to find at least one new idea Smile :)

TestNG Testing Continuous Integration/Deployment

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Quick Pattern-Matching Queries in PostgreSQL and YugabyteDB
  • What Is Policy-as-Code? An Introduction to Open Policy Agent
  • Agile Transformation With ChatGPT or McBoston?
  • Utilize OpenAI API to Extract Information From PDF Files

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: