The Internet of Things (IoT) is no longer "the wave of the future," but is instead "the wave of the present." We are already living in a world where our appliances talk to each other, our vehicles reach into the Internet to find traffic, and our cities are gearing up to regulate traffic, power, and other civic needs via smart devices. These devices will rely heavily on new software to drive existing or new hardware into the IoT.

Testing Cycles

• The starting point is simple functionality testing. This answers the question “Does the software do what it is designed to do with no unforeseen effects?”
• Once this gate has been cleared, the next step is to perform stress tests. These tests are designed to mark the limits of what the software can do, especially under extreme loads. Stress testing answers two questions, “What is the maximum amount of traffic this software can handle?” and “Does the software fail gracefully when it is overwhelmed?”
• When this gate is cleared, anomaly tests are then started. These tests strive to find out what the software will do when presented with information or instructions that are outside of expected values. Anomaly testing looks to find any ‘soft spots’ in the software, problems that could allow a wrong response or a clever attack unauthorized access to the device or perhaps crash the device itself. If the device is looking for a numeric response but receives an alpha-numeric response instead, will it simply re-prompt, or will it stay tied up in a logical loop rendering it essentially dead? Or perhaps it will receive the bad response and grant access to higher-level administrative functions in the device. Anomaly testing looks to find these issues out before the software is released.
• The last gate addresses the security tests that must be performed. Security testing exercises the software in ways designed to break it to ensure that, in the event of an attack, the software fails in a way that is safe. Security test suites have to be crafted with the knowledge of past types of attacks along with the same ‘outside the box’ thinking that hackers use to develop new, unforeseen types of attacks. Security testing answers the question “Can hackers or attackers gain control of this software and use that control to cause harm to the device or to devices connected to it?”

So the key to creating IoT devices and software is to ensure that it does what it is supposed to do and is hardened up enough to keep from being a security risk in the event of organic failures or cyber attacks.

The way to reach this endpoint is to have test suites developed that will address each of these four test scenarios. Having pre-crafted, repeatable tests that can be scaled up or down to match the testing scenario means having test runs that compare apples to apples every time.

Having those tests embedded in a device or test application means that your development and QA staff will have more time available to perform required tests, rather than having to create those tests for each new iteration of software.