Fuzzers: A Taxonomy

Fuzzing is an important capability in cybersecurity vulnerability analysis. You can use it to test applications, libraries, network services, you name it. There's a variety of tools available out there for this kind of thing, spanning both the open-source and commercial markets. That said, they still generally take a bit of work to apply to a specific problem.

Interestingly, though it's an approach equally applicable to more traditional software quality assurance, fuzzers have been more actively adopted by the cybersecurity research and development community than software testers. I'm not entirely sure why this is, but I suspect that it's because fuzzing is a much more efficient approach to vulnerability discovery when working with black-box software components than alternatives like source code analysis (especially as you'll only have access to disassembly or decompiled code rather than original code in black-box work).

There are roughly six types of fuzzers. You can organize fuzzers along three different dimensions — mutation or generation fuzzing, dumb or smart fuzzing, and black-box or white-box fuzzing (grey-box fuzzing is certainly a thing, but black-box fuzzing is usually really grey-box fuzzing as you almost always have some knowledge of the target).

Mutation and generation fuzzers handle test case generation differently. Mutation fuzzers mutate a supplied seed input object, while generation fuzzers generate new test cases from a supplied model. Some fuzzers can do both.

Intelligent and dumb buzzers are somewhat inaccurately named. Initially defined to indicate knowledge of and adherence to specific input formats, these terms today relate more to the way that specific fuzzers mutate or generate input rather than specific knowledge of input formats. Most interesting fuzzing requires some kind of a priori knowledge of input formats whether inputs are just randomly changed or tweaked based on some kind of internal algorithm.

Some dumb fuzzers have been very successful, believe it or not. American Fuzzy Lop (AFL) is a dumb fuzzer that has been very successful at finding bugs in a wide range of software products, though it requires access to source code to work best. AFL, though considered a dumb fuzzer as it just flips bits and replaces bytes, still uses a genetic algorithm to create new test cases. Smart fuzzers will change input based on a specific understanding of the input expected. Rather than randomly changing values, smart fuzzers could use specific values with conflicting semantics or generate specific optional metadata to use across a wide number of possible inputs. A dumb fuzzer, like AFL, can be very sophisticated, and smart fuzzers relatively simple, leading to a fair amount of confusion with respect to the application of these particular terms. Again, you can mix these approaches.

Finally, fuzzers can be grouped into either black- or white-box approaches. Black-box fuzzers are more commonly used by cybersecurity researchers looking for vulnerabilities in commercial products. White-box approaches by definition require access to all program artifacts and are usually used by red teams working for organizations responsible for systems or by software testing groups. This is also somewhat inaccurate. External cybersecurity researchers may not have access to source code, for example, but they frequently are able to access either disassembled or decompiled code, as well as other program artifacts like configuration files, registry settings, fonts, icons, or logos. These kinds of artifacts may not be able to be changed in all cases, but they are still usually accessible with some effort and can be used to guide fuzzing work.

Fuzzers can be used to test programs by tweaking input values, as well as networks services by submitting varying input over network connections (you could classify fuzzers along this dimension as well). Some fuzzers can be used in either area, while some are tied to a particular domain. Either way, if you're doing cybersecurity research or application auditing, fuzzers should be a key part of your overall technical approach.