Bar Exam Failures Highlight Importance of Performance Testing

originally written by craig lowell at the catchpoint blog.

it goes without saying that old-fashioned methods of doing business and keeping records are increasingly shifting into digital and/or online formats. usually that means making life easier for everyone involved. of course, when the systems behind such technological advancements fail, it can mean a nightmare instead.

this week, law students around the country have been going through the painstaking process of taking the bar exam, the culmination of weeks – sometimes months – of laborious studying that will determine whether they can actually practice law after pouring years of their life and tens of thousands of dollars into their education.

so you can imagine their frustration when, after completing the essay part of the exam, those students were unable to send their answers in .

examsoft, whose software is used by many states to conduct their bar exams (as well as by individual law schools for regular exams throughout the year), experienced server overload when it came time for those thousands of exam takers to submit their essays, as those trying to do so came face to face with examsoft’s version of the pinwheel of death:

the failures brought to many people’s minds the disaster that was the introduction of healthcare.gov so many months ago, while simultaneously putting to rest the idea that putting government services in the hands of a private company is a guarantee for success.

as the legions of test takers took to social media to vent their frustrations at the company, the team at examsoft went into panic mode as their it staff desperately tried to correct the problem while the rest supposedly began contacting the state bars in order to explain why so many people seemed to miss the deadline to submit their exams.

the process for taking the exam differs from state to state, and seeing as examsoft is used by several state bars, the terrible experiences on tuesday varied. however, one massachusetts law student, who wished to remain anonymous, reported people being told to try submitting from multiple locations – none of which worked – and then waiting for as much as three hours on the phone with customer service as they tried to solve the problem.

adding to the students’ fury is the fact that they have to pay a significant chunk of money simply for the “privilege” of using the software. to download it can cost as much as $150 (again, depending on the state), and several state bars have an additional triple-digit fee on top of that. after being forced to pay that much, one should be able to expect a certain degree of functionality, but obviously that didn’t happen on tuesday.

the problem was later blamed on a processing issue on examsoft’s end, which they apologized for on their facebook page , and those users who experienced the errors were given deadline extensions so that they won’t be penalized. but the fallout from this fiasco is far from over, as the company has also left itself open to potential legal ramifications (after all, these are future lawyers we’re talking about), seeing as those end users who had to spend hours dealing with these failures were forced to take valuable time away from the preparations for the second part of the exam the following day.

in the meantime, we once again see the importance of performance and capacity testing before undertaking a big online task. failure to do so can have wide-ranging ramifications for a long time.