Testing, Tools, and Frameworks

As the global community grapples with the urgent challenges of climate change, the role of technology and software becomes increasingly pivotal in the quest for sustainability. There exist optimization approaches at multiple levels that can help: Algorithmic efficiency: Algorithms that require fewer computations and resources can reduce energy consumption. A classic example here is optimized sorting algorithms in data processing. Cloud efficiency: Cloud services are energy-efficient alternatives to on-premises data centers. Migrating to cloud platforms that utilize renewable energy sources can significantly reduce the carbon footprint. Code optimization: Well-optimized code requires less processing power, reducing energy demand. Code reviews focusing on efficient logic, unit testing, and integration testing can lead to cleaner, greener software. Energy-aware architectural design: Energy-efficient design principles can be incorporated into software architecture. Ensuring, for example, that software hibernates when inactive or scales resources dynamically can save energy. Distributed, decentralized, and centralized options like choreography and orchestration can be evaluated. Renewable energy: Data centers and computing facilities can be powered with renewable energy sources to minimize reliance on fossil fuels and mitigate emissions. Green Software Standards: Industry standards and certifications for green software design can drive developers to create energy-efficient solutions. In this article, we will focus on code optimization via software testing. Software testing, a fundamental component of software development, can play a significant role in mitigating the environmental impact of technology. We explore the intersection of software testing and climate change, highlighting how testing can contribute to a more sustainable technological landscape. We begin by summarizing the role of software in the energy footprint of a number of industries. We then explore basic types of software testing that can be applied, giving specific examples. These types are by no means exhaustive. Other types of testing may well be used according to the energy optimization scenario. From Bytes to Carbon Telecommunications Telecommunication networks, including cellular and fixed-line networks, require software to manage signal routing, call routing, data transmission, and network optimization. The software that governs communication protocols, such as 4G, 5G, Wi-Fi, and other wireless technologies, also plays a crucial role in determining network efficiency and energy consumption. Traffic management, billing and customer management, service provisioning, remote monitoring, and management also involve software. As the telecommunications industry continues to evolve, a focus on sustainable software development and energy-efficient practices will be crucial to minimizing its environmental impact. E-Commerce Online shopping platforms require data centers to process transactions and manage inventories. Every click, search, and transaction contributes to the digital carbon footprint. Streamlining software operations and optimizing server usage can reduce energy consumption. Finance Financial institutions rely on software for trading, risk management, and customer service. High-frequency trading algorithms, for instance, demand significant computational power. By optimizing these algorithms and reducing unnecessary computations, the energy footprint can be curbed. Healthcare Electronic health records and medical imaging software are vital in healthcare. Reducing the processing power needed for rendering medical images or utilizing cloud services for storage can mitigate carbon emissions. Transportation Ride-sharing and navigation apps require real-time data processing, contributing to energy-intensive operations. Implementing efficient route optimization algorithms, for example, can reduce the carbon footprint of such apps. Entertainment Streaming platforms, gaming, and content delivery networks rely on data centers to provide seamless experiences. Employing content delivery strategies that minimize data transfer and optimize streaming quality can alleviate energy consumption. Manufacturing Industrial automation systems use software to control production processes. By optimizing these systems for energy efficiency, manufacturers can decrease the carbon footprint associated with production. Agriculture Precision farming relies on software for data analysis and decision-making. Ensuring that sensors and software are finely tuned reduces energy waste in the field. Education Online education platforms, virtual classrooms, and digital learning materials consume energy. Optimizing code, minimizing background processing, and encouraging offline access can lower energy consumption. Aerospace and Defense Aerospace and defense industries rely on sophisticated software for designing aircraft, simulations, and defense systems. Reducing resource-intensive calculations and optimizing software design can lower energy consumption and carbon emissions. Testing Types That Can Be Used As software is omnipresent, powering devices, applications, and infrastructure across industries, the development and deployment of software are not devoid of environmental implications. Unoptimized software that requires excessive computational resources can exacerbate this problem. Ensuring software is efficient and optimized through rigorous testing can significantly reduce its carbon footprint. Performance, Load, and Stress Testing for Efficiency and Scalability Performance testing evaluates how a system responds under varying workloads. By assessing software performance, developers can identify resource-intensive processes that may lead to energy waste. Optimizing resource utilization and minimizing processing time may lead to reduced energy consumption and, consequently, a smaller carbon footprint. Bear in mind that there is not always a linear correlation between resource optimization and energy consumption. In simple terms, the relationship between resource optimization and the energy consumption is not always straightforward. Optimizing certain processes might lead to more efficient energy use but can also involve complex trade-offs. For instance, reducing processing time might lead to more rapid completion of tasks. Still, it could also lead to higher peak resource usage, potentially negating energy savings due to shorter task duration. Such cases may be addressed by performance testing on the peak resource usage and optimization. Under the performance testing umbrella, load and stress testing can be useful, examining software's ability to handle increasing loads of traffic. When software is capable of efficiently accommodating user demands, it reduces the need for over-provisioning resources, which can lead to energy inefficiencies. A well-tested application that scales seamlessly promotes resource efficiency and sustainability. When the load increases beyond certain limits, stress testing can identify breaking points of a system or its capacity to handle excessive load, which could lead to performance degradation or failure. For example, in an e-commerce platform, a sudden surge in user traffic during a sale event overwhelms the website. By subjecting the platform to simulated high loads, developers can pinpoint areas that require optimization. Ensuring swift load times, efficient search queries, and seamless transaction processing not only enhances user experience but also reduces the energy required for server processing. In the financial sector, high-frequency trading platforms are reliant on efficient software to process complex calculations in microseconds. Performance testing identifies latency issues and helps developers optimize trading algorithms. By ensuring faster execution times and minimizing unnecessary computations, energy consumption is reduced, contributing to a more sustainable financial ecosystem. Streaming platforms witness varying levels of usage throughout the day. Load testing ensures that the platform can handle numerous concurrent viewers without buffering or quality degradation. Scalability ensures that the platform can allocate resources efficiently, reducing energy consumption during high-demand periods. In the transportation sector, load testing is essential for navigation apps that provide real-time route information to users. As users request navigation guidance during peak traffic hours, scalability testing ensures that the app can handle simultaneous queries without lag. A well-scaling app minimizes the need for over-provisioned servers during peak hours, promoting energy-efficient operations. Continuous Integration and Deployment (CI/CD) Implementing CI/CD pipelines with automated testing ensures that code changes are tested rigorously before deployment. By catching bugs early and preventing faulty code from entering production, CI/CD practices contribute to efficient software development, reducing the carbon footprint associated with bug fixes and maintenance. Industrial automation systems demand reliability to prevent production line interruptions. CI/CD guarantees that changes to these systems undergo comprehensive testing to avoid disruptions. Reduced downtimes translate to energy-efficient manufacturing processes. In healthcare software, where data accuracy and patient safety are paramount, CI/CD plays a vital role. Updates to electronic health records systems or medical imaging software undergo rigorous automated testing to prevent data corruption or processing errors. By avoiding situations that necessitate prolonged maintenance, CI/CD practices reduce energy consumption associated with emergency patches. Security Testing for Data Efficiency Security testing verifies the resilience of software against cyber threats. A secure system prevents data breaches and unauthorized access, reducing the risk of compromised data that could lead to unnecessary energy expenditure in data restoration and breach resolution. Security testing ensures that simulation software used in aerospace and defense remains impervious to hacking attempts. By protecting sensitive data, this practice prevents the energy-intensive task of identifying and repairing compromised simulations. In healthcare, finance, and e-commerce, for example, sensitive patient data, financial data, and e-commerce data can be protected. Restoring trust and credibility can be challenging and energy costly. Regression Testing for Code Stability Regression testing confirms that new code does not break existing functionality. Preventing regressions reduces the need for repeated testing and bug fixes, optimizing the software development lifecycle and minimizing unnecessary computational resources. Precision farming software relies on consistent data processing for optimal decision-making. Regression testing verifies that new code changes do not introduce inaccuracies in sensor data analysis. By preventing regressions, energy is conserved by avoiding the need to address erroneous data. Online education platforms introduce new features to enhance user experiences. Regression testing ensures these changes do not disrupt existing lessons or content delivery. By maintaining stability, energy is saved by minimizing the need for post-deployment fixes. Suppose a telecommunications company is rolling out a software update for its network infrastructure to improve data transmission efficiency and reduce latency. The update includes changes to the routing algorithms used to direct data traffic across the network. While the primary goal is to enhance network performance, there is a potential risk of introducing regressions that could disrupt existing services. Before deploying the software update to the entire network, the telecommunications company conducts thorough regression testing. Test cases cover various functionalities and scenarios, including those related to call routing, data transmission, and network optimization. The tests ensure that the new code does not break existing functionalities. If existing functionalities are compromised by the new code, the company may prevent the deployment of faulty updates that could lead to network disruptions. Avoiding such disruptions reduces the need for emergency fixes, saving computational resources that would otherwise be expended in resolving network outages. By ensuring that software updates are stable and do not introduce regressions, the telecommunications company maintains optimal network performance without frequent energy-intensive rollbacks or fixes. Emerging Trends Emerging trends that could shape the future of software development's impact on energy consumption include the rise of edge computing, where processing happens closer to the data source, reducing data transmission and energy consumption. Additionally, advancements in machine learning and artificial intelligence could lead to more sophisticated energy optimization algorithms, improving the efficiency of software operations. Quantum computing might also play a role in addressing complex optimization challenges. Wrapping Up There exist estimation scenarios that by 2030, the information and communications technology (ICT) sector could account for up to 23% of global energy consumption. This surge is fueled by various devices and software applications integral to modern life. The urgency of addressing climate change demands multifaceted approaches across various sectors, including technology. Software, being a core component of our modern lives, has a critical role to play in this endeavor. By integrating sustainable practices into software testing, developers can contribute to a more environmentally conscious technological landscape. As we continue to innovate and develop software solutions, it is important that we remain mindful of the environmental impact of our creations. Embracing a testing paradigm that focuses on performance optimization and resource efficiency can help reduce the carbon footprint of software. Ultimately, software development and testing could help towards harnessing the potential of technology to address climate change while delivering efficient, effective, and sustainable solutions to a rapidly changing world.

Beyond Unit Testing Test-driven development (TDD) is a well-regarded technique for an improved development process, whether developing new code or fixing bugs. First, write a test that fails, then get it to work minimally, then get it to work well; rinse and repeat. The process keeps the focus on value-added work and leverages the test process as a challenge to improving the design being tested rather than only verifying its behavior. This, in turn, also improves the quality of your tests, which become a more valued part of the overall process rather than a grudgingly necessary afterthought. The common discourse on TDD revolves around testing relatively small, in-process units, often just a single class. That works great, but what about the larger 'deliverable' units? When writing a microservice, it's the services that are of primary concern, while the various smaller implementation constructs are simply enablers for that goal. Testing of services is often thought of as outside the scope of a developer working within a single codebase. Such tests are often managed separately, perhaps by a separate team, using different tools and languages. This often makes such tests opaque and of lower quality and adds inefficiencies by requiring a commit/deploy as well as coordination with a separate team. This article explores how to minimize those drawbacks with test-driven development (TDD) principles applied at the service level. It addresses the corollary that such tests would naturally overlap with other API-level tests, such as integration tests, by progressively leveraging the same set of tests for multiple purposes. This can also be framed as a practical guide to shift-left testing from a design as well as implementation perspective. Service Contract Tests A Service Contract Test (SCT) is a functional test against a service API (black box) rather than the internal implementation mechanisms behind it (white box). In their purest form, SCTs do not include subversive mechanisms such as peeking into a database to verify results or rote comparisons against hard-coded JSON blobs. Even when run wholly within the same process, SCTs can loop back to localhost against an embedded HTTP server such as that available in Spring Boot. By limiting access through APIs in this manner, SCTs are agnostic as to whether the mechanisms behind the APIs are contained in the same or a different process(es), while all aspects of serialization/deserialization can be tested even in the simplest test configuration. The general structure of an SCT is: Establish a starting state (preferring to keep tests self-contained) One or more service calls (e.g., testing stateful transitions of updates followed by reads) Deep verification of the structural consistency and expected behavior of the results from each call and across multiple calls Because of the level they operate, SCTs may appear to be more like traditional integration tests (inter-process, involving coordination across external dependencies) than unit tests (intra-process operating wholly within a process space), but there are important differences. Traditional integration test codebases might be separated physically (separate repositories), by ownership (different teams), by implementation (different language and frameworks), by granularity (service vs. method focus), and by level of abstraction. These aspects can lead to costly communication overhead, and the lack of observability between such codebases can lead to redundancies, gaps, or problems tracking how those separately-versioned artifacts relate to each other. With the approach described herein, SCTs can operate at both levels, inter-process for integration-test level comprehensiveness as well as intra-process as part of the fast edit-compile-test cycle during development. By implication, SCTs operating at both levels Co-exist in the development codebase, which ensures that committed code and tests are always in lockstep Are defined using a uniform language and framework(s), which lowers the barriers to shared understanding and reduces communication overhead Reduce redundancy by enabling each test to serve multiple purposes Enable testers and developers to leverage each other’s work or even (depending on your process) remove the need for the dev/tester role distinction to exist in the first place Faking Real Challenges The distinguishing challenge to testing at the service level is the scope. A single service invocation can wind through many code paths across many classes and include interactions with external services and databases. While mocks are often used in unit tests to isolate the unit under test from its collaborators, they have downsides that become more pronounced when testing services. The collaborators at the service testing level are the external services and databases, which, while fewer in number than internal collaboration points, are often more complex. Mocks do not possess the attributes of good programming abstractions that drive modern language design; there is no abstraction, no encapsulation, and no cohesiveness. They simply exist in the context of a test as an assemblage of specific replies to specific method invocations. When testing services, those external collaboration points also tend to be called repeatedly across different tests. As mocks require a precise understanding and replication of collaborator requests/responses that are not even in your control, it is cumbersome to replicate and manage that malleable know-how across all your tests. A more suitable service-level alternative to mocks is fakes, which are an alternative form of test double. A fake object provides a working, stateful implementation of its interface with implementation shortcuts, making it not suitable for production. A fake, for example, may lack actual persistence while otherwise providing a fully (or mostly, as deemed necessary for testing purposes) functionally consistent representation of its 'real' counterpart. While mocks are told how to respond (when you see exactly this, do exactly that), fakes know themselves how to behave (according to their interface contract). Since we can make use of the full range of available programming constructs, such as classes, when building fakes, it is more natural to share them across tests as they encapsulate the complexities of external integration points that need not then be copied/pasted throughout your tests. While the unconstrained versatility of mocks does, at times, have its advantages, the inherent coherence, and shareability of fakes make them appealing as the primary implementation vehicle for the complexity behind SCTs. Alternately Configured Tests (ACTs) Being restricted to an appropriately high level of API abstraction, SCTs can be agnostic about whether fake or real integrations are running underneath. The same set of service contract tests can be run with either set. If the integrated entities, here referred to as task objects (because they often can be run in parallel as exemplified here), are written without assuming particular implementations of other task objects (in accordance with the "L" and "D" principles in SOLID), then different combinations of task implementations can be applied for any purpose. One configuration can run all fakes, another with fakes mixed with real, and another with all real. These Alternately Configured Tests (ACTs) suggest a process, starting with all fakes and moving to all real, possibly with intermediate points of mixing and matching. TDD begins in a walled-off garden with the 'all fakes' configuration, where there is no dependence on external data configurations and which runs fast because it is operating in process. Once all SCTs pass in that test configuration, subsequent configurations are run, each further verifying functionality while having only to focus on the changed elements with respect to the previous working test configuration. The last step is to configure as many “real” task implementations as required to match the intended level of integration testing. ACTs exist when there are at least two test configurations (color code red and green in the diagram above). This is often all that is needed, but at times, it can be useful in order to provide a more incremental sequence from the simplest to the most complex configuration. Intermediate test configurations might be a mixture of fake and real or semi-real task implementations that hit in-memory or containerized implementations of external integration points. Balancing SCTs and Unit Testing Relying on unit tests alone for test coverage of classes with multiple collaborators can be difficult because you're operating at several levels removed from the end result. Coverage tools tell you where there are untried code paths, but are those code paths important, do they have more or less no impact, and are they even executed at all? High test coverage does not necessarily equal confidence-engendering test coverage, which is the real goal. SCTs, in contrast, are by definition always relevant to and important for the purpose of writing services. Unit tests focus on the correctness of classes, while SCTs focus on the correctness of your API. This focus necessarily drives deep thinking about the semantics of your API, which in turn can drive deep thinking about the purpose of your class structure and how the individual parts contribute to the overall result. This has a big impact on the ability to evolve and change: tests against implementation artifacts must be changed when the implementation changes, while tests against services must change when there is a functional service-level change. While there are change scenarios that favor either case, refactoring freedom is often regarded as paramount from an agile perspective. Tests encourage refactoring when you have confidence that they will catch errors introduced by refactoring, but tests can also discourage refactoring to the extent that refactoring results in excessive test rework. Testing at the highest possible level of abstraction makes tests more stable while refactoring. Written at the appropriate level of abstraction, the accessibility of SCTs to a wider community (quality engineers, API consumers) also increases. The best way to understand a system is often through its tests; since those tests are expressed in the same API used by its consumers, they can not only read them but also possibly contribute to them in the spirit of Consumer Driven Contracts. Unit tests, on the other hand, are accessible only to those with deep familiarity with the implementation. Despite these differences, it is not a question of SCTs vs. unit tests, one excluding the other. They each have their purpose; there is a balance between them. SCTs, even in a test configuration with all fakes, can often achieve most of the required code coverage, while unit testing can fill in the gaps. SCTs also do not preclude the benefits of unit testing with TDD for classes with minimal collaborators and well-defined contracts. SCTs can significantly reduce the volume of unit tests against classes without those characteristics. The combination is synergistic. SCT Data Setup To fulfill its purpose, every test must work against a known state. This can be a more challenging problem for service tests than for unit tests since those external integration points are outside of the codebase. Traditional integration tests sometimes handle data setup through an out-of-band process, such as database seeding with automated or manual scripts. This makes tests difficult to understand without having to hunt down that external state or external processes and is subject to breaking at any time through circumstances outside your control. If updates are involved, care must be taken to reset or restore the state at the test start or end. If multiple users happen to run the tests at the same time, care must be taken to avoid update conflicts. A better approach tests that independently set up (and possibly tear down) their own non-conflicting (with other users) target state. For example, an SCT that tests the filtered retrieval of orders would first create an order with a unique ID and with field values set to the test's expectations before attempting to filter on it. Self-contained tests avoid the pitfalls of shared, separately controlled states and are much easier to read as well. Of course, direct data setup is not always directly possible since a given external service might not provide the mutator operations needed for your test setup. There are several ways to handle this: Add testing-only mutator operations. These might even go to a completely different service that isn't otherwise required for production execution. Provide a mixed fake/real test configuration using fakes for the update-constrained external service(s), then employ a mechanism to skip such tests for test configurations where those fake tasks are not active. This at least tests the real versions of other tasks. Externally pre-populated data can still be employed with SCTs and can still be run with fakes, provided those fakes expose equivalent results. For tests whose purpose is not actually validating updates (i.e., updates are only needed for test setup), this at least avoids any conflicts with multiple simultaneous test executions. Providing Early Working Services A test-filtering mechanism can be employed to only run tests against select test configurations. For example, a given SCT may initially work only against fakes but not against other test configurations. That restricted SCT can be checked into your code repository, even though it is not yet working across all test configurations. This orients toward smaller commits and can be useful for handing off work between team members who would then make that test work under more complex configurations. Done right, the follow-on work need only be focused on implementing the real task that doesn’t break the already-working SCTs. This benefit can be extended to API consumers. Fakes can serve to provide early, functionally rich implementations of services without those consumers having to wait for a complete solution. Real-task implementations can be incrementally introduced with little or no consumer code changes. Running Remote Because SCTs are embedded in the same executable space as your service code under test, all can run in the same process. This is beneficial for the initial design phases, including TDD, and running on the same machine provides a simple way for execution, even at the integration test level. Beyond that, it can sometimes be useful to run both on different machines. This might be done, for example, to bring up a test client against a fully integrated running system in staging or production, perhaps also for load/stress testing. An additional use case is for testing backward compatibility. A test client with a previous version of SCTs can be brought up separately from and run against the newer versioned server in order to verify that the older tests still run as expected. Within an automated build/test pipeline, several versions can be managed this way: Summary Service Contract Tests (SCTs) are tests against services. Alternatively, Configured Tests (ACTs) define multiple test configurations that each provide a different task implementation set. A single set of SCTs can be run against any test configuration. Even though SCT can be run with a test configuration that is entirely in process, the flexibility offered by ACTs distinguishes them from traditional unit/component tests. SCTs and unit tests complement one another. With this approach, Test Driven Development (SCT) can be applied to service development. This begins by creating SCTs against the simplest possible in-process test configuration, which is usually also the fastest to run. Once those tests have passed, they can be run against more complex configurations and ultimately against a test configuration of fully 'real' task implementations to achieve the traditional goals of integration or end-to-end testing. Leveraging the same set of SCTs across all configurations supports an incremental development process and yields great economies of scale.

In the evolving realm of IT, complexity emerges as a formidable challenge, ranking high on the list of concerns. Solutions should be designed to function across multiple platforms and cloud environments, ensuring portability. They should seamlessly integrate with existing legacy systems while also being tailored to accommodate a wide range of potential scenarios and requirements. Visualizing the average IT landscape often results in a tapestry of applications, hardware, and intricate interdependencies. The term "legacy systems" often conjures images of outdated software and obsolete hardware. Yet, beneath their seemingly archaic facade lies a critical piece of the IT puzzle that significantly influences the overall complexity of modern systems. These legacy systems, with their historical significance and enduring impact, continue to affect the way organizations manage and navigate their technological landscapes. They usually come in the form of large monolithic systems that encompass a wide range of functionalities and features. As projects evolved, new code was added in a way that was increasingly difficult to understand and manage. It was added in a way that strong coupling between components was introduced, leading to a convoluted codebase. An update or change to one part of the system will affect other parts of the system, creating a dependency labyrinth. Other types of dependencies, like dependencies on third-party libraries, frameworks, and tools, make updates and changes even more complicated. Other characteristics of legacy systems may include outdated technologies, technological and technical debt, and limited to no documentation. Scaling, deploying, and testing legacy systems can be tricky. As the load increases, it's often necessary to scale the entire system, even if only a specific part requires more resources. This can lead to inefficient resource utilization, slower deployments, and slower scaling processes. This article starts by examining key factors of IT complexity at an organizational decision level (vendor complexity, system integration complexity, consultants, and other internal dynamics) and mentality. We then move on to the implementation level of legacy systems. A blueprint is proposed for testing that is by no means exhaustive. It’s a minimal iterative testing process that can be tailored according to our needs. Vendor Complexity No matter how good a vendor is at software development, when the “build over buy” approach is adopted, building software introduces its own complexities. At best, as the software built in-house evolves, the complexity of maintaining it at a certain quality level can be manageable. At worst, the complexity may run out of control leading to unforeseen nightmares. As time passes, technologies and market needs evolve, and the systems developed may evolve into legacy systems. Many enterprises opt for a "buy over build" approach, procuring a substantial portion of their software and hardware from external vendors rather than creating in-house solutions. This inclination toward enterprise-grade software comes at a significant cost, with licenses consuming large parts of IT budgets. Software vendors glean advantages from complexity in several ways: Competing through extensive feature lists inadvertently fosters product complexity. The drive for features stems from the understanding that IT solutions are often evaluated based on the sheer presence of features, even if many of these features go unused. Savvy vendors adeptly tout their products: "Already using an application performance management framework? Ours is superior and seamlessly integrates with your existing setup!" Consequently, the enterprise landscape becomes fraught with multiple products addressing the same issue, contributing to complexity. Vendors introduce complexity into their product strategies, relentlessly coining new buzzwords and positioning improved solutions regularly to sustain their sales pipeline. Hardware vendors, too, derive benefits from complexity, as they can offer hardware-based solutions to problems rather than advocating for streamlined software alternatives. System Integration Complexity System integrators play a pivotal role in building enterprise software or integrating purchased solutions. They contribute specialized skill sets that internal IT might lack. System integrators are service providers offering consultancy services led by experts. Although cloaked in various arrangements, the consultancy economics ultimately translate into hourly billing for these consultants, aligning with the fundamental unit of consulting: the staff hour. More complexity invariably translates into more work and, consequently, increased revenue for system integrators. Consultants' Influence Consultants, often regarded as hired specialists, engage in solving intricate problems and shaping IT strategies. Yet, the imperative to remain employed sometimes may lead them to offer partial solutions, leaving room for further engagements. This behavior is termed "scoping" or "expectation management." Other Internal Dynamics In certain organizational contexts, the individual with a larger budget and greater operational complexity often garners more esteem. Paradoxically, complexity becomes a means of career advancement and ego elevation. Some IT managers may revel in showcasing the sophistication (i.e., complexity) of their operations. Legacy Systems To develop and maintain effectively any software system we need requirements with the following qualities: Documented: They must be written somewhere and should not just exist in our minds. Documentation may be as lightweight as possible as long as it’s easy to maintain and fulfills its purpose of being a single source of truth. Correct: We understand correctly what is required from the system and what is not required. Complete: There are no missing attributes or features. Understandable: It’s easy for all stakeholders to understand what they must do in order for the requirements to be fulfilled. Unambiguous: When we read the requirements, we can all understand the same thing. Consistent: The terminology is consistent with no contradictions. Testable: We must have an idea about how to test that the requirements are fulfilled. Traceable: We should be able to trace each requirement in code and tests. Viable: We should be able to implement them within the existing constraints (time, money, number of employees). Implementation independent: Requirements should only describe what should be done and not how. Consider a software system that, during the course of development, has been hit with most, if not all, of the qualities listed above. There may be little to no documentation that is difficult to understand, incomplete, and full of contradictions. One requirement contradicts another. Terminology is inconsistent, the code was written with no testability in mind, and there exist a few outdated UI tests. The people who have developed this system are no longer with the company, and no current employee understands exactly what it does or how it works. Somehow, however, the company has managed to keep this legacy system running for years despite the high maintenance costs and its outdated technologies. The problem with such a legacy system is that everyone is afraid to change anything about it. Small changes in the code may result in big failures. It could be code changes due to fixing bugs or glitches or due to developing and releasing new features. It could be changes like refactoring or improving performance and other non-functional issues. In the following sections, we will depict a set of steps to guide testing endeavors for legacy systems. Our blueprint is generic enough to be applied to a number of different contexts. Keep in mind, though, that some steps and the tasks per step may vary depending on the context. Step 1: Initial Assessment and Reconnaissance Try to understand the context. Determine why we are exploring the legacy system and what goals we aim to achieve overall. What is the definition of done for our testing efforts? Gather any available documentation. Collect existing documentation, even if outdated, to gain initial insights into the system's functionality and architecture. Identify stakeholders and ask questions. Identify individuals or teams who are currently or were previously involved with the system, including developers, product managers or analysts, business users, and technical support. The goal of step 1 is to understand the context and scope of testing. Make sure to gather as much information as possible to be able to get started. Keep in mind that there exist emergent characteristics in software systems that can only be found by exploration. As long as we’ve got a couple of ideas of where to start and what to test, we could be ready to go to step 2. Here are three questions to ask stakeholders at this stage that could help us get started. What are your plans for the legacy system under test? What is the current ecosystem that it is used for? Who uses this legacy system in order to solve what problem? Step 2: Explore by Pair Testing To learn about a legacy system quickly, it’s best to work with other explorers and discuss our findings. More people can cover the same ground more quickly. Each person brings a unique perspective and skills to play and thus will notice things that others won’t. As a group, we will discover more in the same amount of time by working together than if we all explored different aspects of the system individually. By pooling our insights, we make more effective use of our time. What to look for: Try to find out the core capabilities (if any). Systems exist to perform core capabilities, although, for some systems, that’s not always crystal clear. If not certain, try to answer question 3 from step 1. There may be multiple answers to that question that we need to explore. Try to find out the business rules (if any). For example, banking and accounting systems have rules that have to do with the balancing of accounts. Other industries like lottery and betting have a strict set of business rules that must be followed. Security, authorization, authentication, and single sign-on are also factors to look for. Try to identify the boundaries of the system under test. This may be challenging as the boundaries in legacy systems are usually blurry. Nevertheless, try to find out what the system does, how it interfaces with what, and when. How the pieces and parts all connect together. The goal of step 2 is to understand at least the basics of the system’s behavior. We are done if we have a set of questions that can lead to fruitful discussions in step 3 and further exploration. As a rule of thumb, the more we can answer any of the following questions, the more ready we are for step 3. What functions does the system perform beyond its core capabilities? How does the system take input? How (or where) does it produce output? What makes a basic input or sequence of actions, and what corresponds to the resulting output or outcome? How is the output influenced by the environment or configuration? Are there any alternative means to interact with the system that circumvents the intended interfaces? (For instance, are there concealed configuration files that can be manipulated? Is data stored in a location accessible directly? Are individual components of the system accessible directly, bypassing the more commonly used public interface?) How can error conditions be triggered? What potential outcomes might arise from using the system in unintended ways or intentionally provoking error conditions? Step 3: Seek More Information From Stakeholders This is when we can check if we did a good job in steps 1 and 2. We should be in a position to start answering and asking questions. If we can answer no questions and if the answers to our questions lead to no further exploration, then we may need to go back to steps 1 or 2. Once we are sufficiently fluent in our understanding of how our legacy system works, we’re ready to interview stakeholders. Questions to ask: Are there any alternatives? If yes, why would someone use this legacy system instead of the alternatives? Is there a sales pitch? If not, try to make a pitch in order to make me buy it. If nothing is working on the system, what would be the first thing that should work? If the legacy system is in telecommunications, aviation, finance, healthcare, defense, insurance, education, energy, public services, or other regulated industries, ask if there are industry standards that must be followed. Find out if one or more standards are not followed and why. How can we interact with the system under test at an API level? Any relevant documentation? Are we aware of any assumptions made when the system was built and the target ecosystems that the system was built for? Step 4: Iterate and Refine Once we gain information and insights about how the legacy system should work and how it should not work, we should end up with new use cases that lead to new test cases. Fixes to bugs that we may have found during our testing sessions may also introduce new problems that we need to be aware of. Extensive regression testing may be needed to address such problems, and if things work smoothly, such regression test suites are good candidates for automation testing. Organizing work: We should document our findings in a lightweight manner and capture key insights, models, and observations for reference. Create tables illustrating rules and conditions that govern the system's behavior. As we learn more and refine our understanding, we should update our documentation. Our findings should be used to create a set of repeatable smoke or regression tests that represent the core functionalities of the system. As more and more functionality is uncovered, keep thinking and asking: What is core functionality and what is not? Remember that this process is iterative. As we gain deeper insights, we may need to revisit previous steps and refine our approach. For example, new testing levels and types of testing may become necessary. A goal for each step is to gradually build a comprehensive understanding of the legacy system's behavior, uncover bugs, and develop strategies to mitigate them. Wrapping Up Legacy systems, while often deemed outdated, frequently coexist with cutting-edge technologies in modern enterprises. This interplay between the past and the present can create a labyrinth of integration challenges, requiring intricate compatibility and interoperability measures to ensure smooth operations. The legacy systems might not always seamlessly align with contemporary protocols and standards, necessitating workarounds, middleware, and custom interfaces. Consequently, this integration dance contributes significantly to the complexity of the overall IT landscape. We presented a high-to-low-level analysis of complexity in IT, and we also provided a blueprint for effective legacy system testing.

Kafka Non-Blocking Retries Non Blocking retries in Kafka are done via configuring retry topics for the main topic. An Additional Dead Letter Topic can also be configured if required. Events will be forwarded to DLT if all retries are exhausted. A lot of resources are available in the public domain to understand the technicalities. Kafka Consumer Non-Blocking Retry: Spring Retry Topics Spring Retry Kafka Consumer What To Test? It can be a challenging job when it comes to writing integration tests for the retry mechanism in your code. How do you test that the event has been retried for the required number of times? How do you test that retries are only performed when certain exceptions occur and not for others? How do you test if another retry is not done if the exception is resolved in the previous retry? How do you test that the nth attempt in the retry succeeds after (n-1) retry attempts have failed? How to test if the event has been sent to the Dead Letter Queue when all the retry attempts have been exhausted? Let’s see with some code. You can find a lot of good articles which show how to set up Non-Blocking retries using Spring Kafka. One such implementation is given below. This is accomplished using the @RetryableTopic and @DltHandler annotations from Spring-Kafka. Setting up the Retryable Consumer Java @Slf4j @Component @RequiredArgsConstructor public class CustomEventConsumer { private final CustomEventHandler handler; @RetryableTopic(attempts = "${retry.attempts}", backoff = @Backoff( delayExpression = "${retry.delay}", multiplierExpression = "${retry.delay.multiplier}" ), topicSuffixingStrategy = TopicSuffixingStrategy.SUFFIX_WITH_INDEX_VALUE, dltStrategy = FAIL_ON_ERROR, autoStartDltHandler = "true", autoCreateTopics = "false", include = {CustomRetryableException.class}) @KafkaListener(topics = "${topic}", id = "${default-consumer-group:default}") public void consume(CustomEvent event, @Header(KafkaHeaders.RECEIVED_TOPIC) String topic) { try { log.info("Received event on topic {}", topic); handler.handleEvent(event); } catch (Exception e) { log.error("Error occurred while processing event", e); throw e; } } @DltHandler public void listenOnDlt(@Payload CustomEvent event) { log.error("Received event on dlt."); handler.handleEventFromDlt(event); } } If you notice in the above code snippet, the include parameter contains CustomRetryableException.class. This tells the consumer to retry only in case CustomRetryableException is thrown by the CustomEventHandler#handleEvent method. You can add as many as you like. There is an exclude parameter as well, but any one of them can be used at a time. The event processing should be retried for a maximum of ${retry.attempts} times before publishing to the DLT. Setting up Test Infra To write integration tests, you need to make sure that you have a functioning Kafka broker (embedded preferred) and a fully functioning publisher. Let's set up our infrastructure: Java @EnableKafka @SpringBootTest @DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_CLASS) @EmbeddedKafka(partitions = 1, brokerProperties = {"listeners=" + "${kafka.broker.listeners}", "port=" + "${kafka.broker.port}"}, controlledShutdown = true, topics = {"test", "test-retry-0", "test-retry-1", "test-dlt"} ) @ActiveProfiles("test") class DocumentEventConsumerIntegrationTest { @Autowired private KafkaTemplate<String, CustomEvent> testKafkaTemplate; // tests } ** Configurations are imported from the application-test.yml file. When using an embedded kafka broker, it is important to mention the topics to be created. They will not be created automatically. In this case, we are creating four topics, namely "test", "test-retry-0", "test-retry-1", "test-dlt" We have set out the maximum retry attempts to three. Each topic corresponds to each of the retry attempts. So events should be forwarded to DLT if three retries are exhausted. Test Cases Retry should not be done if consumption is successful on the first attempt. This can be tested by the fact that the CustomEventHandler#handleEvent method is called only once. Further tests on Log statements can also be added. Java @Test void test_should_not_retry_if_consumption_is_successful() throws ExecutionException, InterruptedException { CustomEvent event = new CustomEvent("Hello"); // GIVEN doNothing().when(customEventHandler).handleEvent(any(CustomEvent.class)); // WHEN testKafkaTemplate.send("test", event).get(); // THEN verify(customEventHandler, timeout(2000).times(1)).handleEvent(any(CustomEvent.class)); verify(customEventHandler, timeout(2000).times(0)).handleEventFromDlt(any(CustomEvent.class)); } Retry should not be done if a non-retryable exception is raised. In this case, the CustomEventHandler#handleEvent method should be invoked only once: Java @Test void test_should_not_retry_if_non_retryable_exception_raised() throws ExecutionException, InterruptedException { CustomEvent event = new CustomEvent("Hello"); // GIVEN doThrow(CustomNonRetryableException.class).when(customEventHandler).handleEvent(any(CustomEvent.class)); // WHEN testKafkaTemplate.send("test", event).get(); // THEN verify(customEventHandler, timeout(2000).times(1)).handleEvent(any(CustomEvent.class)); verify(customEventHandler, timeout(2000).times(0)).handleEventFromDlt(any(CustomEvent.class)); } Retry for the maximum configured number of times if a RetryableException is thrown and subsequently should be published to Dead Letter Topic when retries are exhausted. In this case, the CustomEventHandler#handleEvent method should be invoked three (maxRetries) times and CustomEventHandler#handleEventFromDlt method should be invoked once. Java @Test void test_should_retry_maximum_times_and_publish_to_dlt_if_retryable_exception_raised() throws ExecutionException, InterruptedException { CustomEvent event = new CustomEvent("Hello"); // GIVEN doThrow(CustomRetryableException.class).when(customEventHandler).handleEvent(any(CustomEvent.class)); // WHEN testKafkaTemplate.send("test", event).get(); // THEN verify(customEventHandler, timeout(10000).times(maxRetries)).handleEvent(any(CustomEvent.class)); verify(customEventHandler, timeout(2000).times(1)).handleEventFromDlt(any(CustomEvent.class)); } **A considerable timeout has been added in the verification stage so that exponential back-off delay can be taken into consideration before the test is completed. This is important and may result in an assertion failure if not set properly. Should be retried until RetryableException is resolved And should not continue retrying if a non-retryable exception is raised or consumption eventually succeeds. The test has been set up such as to throw a RetryableException first and then throw a NonRetryable exception, such that retry is done once. Java @Test void test_should_retry_until_retryable_exception_is_resolved_by_non_retryable_exception() throws ExecutionException, InterruptedException { CustomEvent event = new CustomEvent("Hello"); // GIVEN doThrow(CustomRetryableException.class).doThrow(CustomNonRetryableException.class).when(customEventHandler).handleEvent(any(CustomEvent.class)); // WHEN testKafkaTemplate.send("test", event).get(); // THEN verify(customEventHandler, timeout(10000).times(2)).handleEvent(any(CustomEvent.class)); verify(customEventHandler, timeout(2000).times(0)).handleEventFromDlt(any(CustomEvent.class)); } Java @Test void test_should_retry_until_retryable_exception_is_resolved_by_successful_consumption() throws ExecutionException, InterruptedException { CustomEvent event = new CustomEvent("Hello"); // GIVEN doThrow(CustomRetryableException.class).doNothing().when(customEventHandler).handleEvent(any(CustomEvent.class)); // WHEN testKafkaTemplate.send("test", event).get(); // THEN verify(customEventHandler, timeout(10000).times(2)).handleEvent(any(CustomEvent.class)); verify(customEventHandler, timeout(2000).times(0)).handleEventFromDlt(any(CustomEvent.class)); } Conclusion So, you can see that the integration test is a mix and match of strategies, timeouts, delays, and verifications so as to foolproof the retry mechanism of your Kafka Event-Driven Architecture. Kudos. Feel Free to suggest improvements and reach out to me on LinkedIn. The full code can be found here.

In my previous article, I gave an overview of robotics software design, and what are the different approaches to testing we could do, and how we can approach testing at different layers. In this article, I will explain how to perform E2E automated testing for a home robot. Challenges in End-To-End Automation Challenges in robotic automation are plenty. The complexity of the software stack itself makes this type of automation super challenging. Testing could be done at any different layer, starting from the front end all the way to the low layer. End-to-end becomes difficult as it introduces a newer challenge with respect to the 3D space. Testing within a 3D space involving real home features is challenging too. Top of all, there are no specific tools and technology available to automate the testing process. This is a space that requires an immense understanding of the problem and great expertise in solving this challenge. Don't worry. I will share my experience here to help you understand and provide an overview of how things could be done. Real World Example Let's dive straight into a real-world example. You are working as a lead Software Engineer in Test in a robotic company making home robots, and you are responsible for a robot that is used in factories and warehouses for moving things around. The first thing you may face is getting overwhelmed with what are the scenarios to be tested and how you are going to implement your automation skills in this area to get started. In the example of a home robotic vacuum cleaner, a couple of simple real-world scenarios could be: How can I validate the robot moving from the living room to the kitchen? The robot could go to the bedroom and claim it is in the kitchen. How do we ensure 100/100 times it only goes to the kitchen? How do you ensure when there is a low battery, the robot goes back to the charging dock and charges itself? It might be overwhelming initially, but I am here to explain how these can be done. Tools and Technologies Available Unfortunately, there is no single tailor-made tool to solve this problem. There are some technologies that we can modify based on our use case. Below are some of the tools and technologies used in different streams which we can modify to solve our problem. April Tag and Raspberry Pi Combo HTC Vive motion tracking gaming software April Tag and Raspberry Pi Combo What Is a Raspberry Pi? Raspberry Pi is a small Linux-based computer that can do most of the tasks that a Linux machine can do. In the world of devices, Raspberry Pi is used because of its capabilities which it can offer and the cost it can save compared to large Linux/Ubuntu machines. What Is April Tag? Aril tag is something similar to QR codes, but it's different. Some Python modules can help understand the April tag and are very use full in our case. E2E Test Automation Architecture: Key items in this architecture are the Hostmachine, Raspberry Pi, Device Under Test, and April Tags. Host machine: Typically, a Linux/Ubuntu-based machine is used to drive the tests. This is the key element that talks to the Device under test, Raspberry Pi and runs the test framework. This is integral in pulling logs, uploading results, etc. Raspberry Pi: This hosts a restful service that will be mounted on the robot. Raspberry Pi has an inbuilt camera which is controlled by the restful service and can be invoked from the host machine. April Tags: These tags go on the ceiling, and each tag is associated with an ID, and the Id is mapped to a location or space in the real world. In our case, it will be mapped to Living Room, Kitchen, Bedroom, and other spaces which exist in the real world. Test Framework: The test framework handles sending commands to the device, pulling logs, decoding the April tag, and calculating the robot position with respect to the captured tag. This also talks to our Ci systems like Jenkins for continuous integration. Test Flow The test framework issues a command to the DUT to start the navigation. As the device is moving, the Raspberry Pi takes a complete video of the April tags. This video can be post-processed to understand what route the robot took. Later once the device comes to a complete stop, the Raspberry Pi is instructed to take a video where this is used to find the exact location of the robot in real space. With this architecture, we can automate the real scenario of a robot moving from one space to another by tagging each space with a name and post-processing the space in the real world. In the next topic, I will explain how HTC Vive, a motion detection/gaming software helps us solve the automation of robots.

Software-in-the-loop (SIL) testing is a crucial methodology in software development and verification processes. It plays a pivotal role in ensuring the software component's functionality, performance, and reliability by subjecting it to rigorous testing within a simulated environment. SIL testing involves integrating the software with a model of the system or component it will interact with and evaluating the combined behavior using simulation. To conduct SIL testing for medical devices, the first step is to create a comprehensive model that accurately represents the system or component with which the software will interact. This model typically includes the relevant physical and environmental factors the software needs to respond to, such as sensor inputs, external stimuli, and other critical parameters. By incorporating these factors, the model emulates real-world conditions and scenarios. Once the model is established, the software is integrated, forming a combined system that can be tested using simulation techniques. The simulated environment allows for the execution of a wide range of test cases, including typical and exceptional scenarios. For instance, in the case of a patient monitoring system, the software would be integrated with a model representing the patient's physiological systems, such as heart rate, blood pressure, and oxygen levels. The software would then be subjected to various simulated scenarios, such as changes in vital signs, alarms and alerts, and potential device malfunctions. During SIL testing, the behavior and performance of the software are closely monitored and analyzed. The testing team assesses how the software responds to different inputs and stimuli, ensuring it functions correctly and generates accurate outputs. By thoroughly testing the software under various conditions, SIL testing helps uncover any potential defects, anomalies, or issues that may arise during usage. One of the primary benefits of SIL testing in the context of medical devices is early Verification. By testing the software component before integration with the physical hardware, developers can identify and rectify potential issues early, reducing the likelihood of costly rework and ensuring higher software quality. SIL testing also saves cost and time by reducing dependency on physical prototypes and specialized testing equipment. Furthermore, simulating various scenarios and conditions during SIL testing enables comprehensive test coverage, including edge cases and rare events that may be challenging to reproduce in real-world situations. SIL testing is particularly valuable for safety validation in medical devices. It allows developers to evaluate the software's response to safety-critical functionalities and verify the effectiveness of fail-safe mechanisms. By simulating potential failure scenarios, SIL testing aids in identifying and mitigating risks associated with the software component, ultimately enhancing the safety of the medical device. There are several benefits to SIL testing, including: Early verification: SIL testing allows for early verification of the software's functionality and performance before integrating it with the actual hardware. This helps identify and rectify potential issues or defects in the software early, reducing the likelihood of costly rework later in the development process. Cost and time savings: By performing SIL testing, developers can significantly reduce the need for physical prototypes or specialized testing equipment. This leads to cost savings in terms of hardware procurement and setup. Additionally, SIL testing can be performed parallel to hardware development, resulting in time savings and faster time-to-market for medical devices. Test coverage: SIL testing enables comprehensive test coverage by controlling various test scenarios that may be difficult to reproduce in real-world conditions. This includes edge cases, rare events, and abnormal scenarios that may be challenging to recreate during traditional testing approaches. Safety validation: Medical devices require rigorous Validation to ensure patient well-being. SIL testing allows for the evaluation of safety-critical functionalities and the verification of fail-safe mechanisms. By simulating potential failure scenarios, SIL testing aids in identifying and mitigating risks associated with the software component of the device. Iterative development: SIL testing facilitates iterative development by providing a controlled environment for testing and refining the software. It allows quick modifications, updates, and enhancements to be tested without requiring extensive hardware integration, accelerating the development cycle and enabling agile development methodologies. Overall, SIL testing is a valuable technique for software development teams, as it allows them to test software in a simulated environment and detect potential issues early in the development process. SIL testing ensures software components' reliability, performance, and safety in medical devices, contributing to enhanced quality, reduced risks, and improved patient care.

Unit testing and integration testing have long since become established practices. Practically all Java programmers know how to write unit tests with JUnit. IDEs will help you with it and build tools like Maven and Gradle and run them as a matter of course. The same cannot be said of its (sort of) runtime counterpart: Defensive Programming — ensuring your program or method starts with a clean and workable set of inputs before continuing with the business logic. Null checks are the most common example of this. Yet it often seems like everything beyond that is treated as part of the business logic, even when it arguably isn't. If a method that calculates a price needs some value from a configuration file, is the presence of the configuration file part of the business logic? Probably not, but it should be checked nonetheless. This is where Klojang Check steps in. Klojang Check is a small Java library that enables you to separate precondition validation and business logic in a clean, elegant, and concise manner. Its take on precondition validation is rather different from, for example, Guava's Preconditions class or Apache's Validate class. It provides a set of syntactical constructs that make it easy to specify checks on program input, object state, method arguments, variables, etc. In addition, it comes with a set of common checks on values of various types. These checks are associated with short, informative error messages, so you don't have to invent them yourselves. Here is an example of Klojang Check in action: Java public class InteriorDesigner { private final int numChairs; public InteriorDesigner(int numChairs) { this.numChairs = Check.that(numChairs) .is(gt(), 0) .is(lte(), 4) .is(even()) .ok(); } public void applyColors(List<Color> colors) { Check.that(colors).is(notEmpty().and(contains(), noneOf(), RED, BLUE, PINK)); // apply the colors ... } public void addCouch(Couch couch) { Check.that(couch).isNot(Couch::isExpensive, ExpensiveCouchException::new); // add the couch ... } } Performance No one is going to use a library just to check things that aren't even related to their business logic if it is going to hog their CPU. Klojang Check incurs practically zero overhead. That's because it doesn't really do stuff. As mentioned, it only provides a set of syntactical constructs that make precondition validation more concise. Of course, if a value needs to be in a Map before it even makes sense to continue with the rest of a computation, you will have to do the lookup. There are no two ways around it. Klojang Check just lets you express this fact more clearly: Java Check.that(value).is(keyIn(), map); If you are interested, you can find the results of the JMH benchmarks here. Getting Started To start using Klojang Check, add the following dependency to your POM file: XML <dependency> <groupId>org.klojang</groupId> <artifactId>klojang-check</artifactId> <version>2.1.3</version> </dependency> Or Gradle script: Plain Text implementation group: 'org.klojang', name: 'klojang-check', version: '2.1.3' The Javadocs for Klojang Check can be found here. Common Checks The CommonChecks class is a grab bag of common checks on arguments, fields (a.k.a. state), and other types of program input. Here are some examples: Java import static org.klojang.check.CommonChecks.*; Check.that(length).is(gte(), 0); Check.that(divisor).isNot(zero()); Check.that(file).is(writable()); Check.that(firstName).is(substringOf(), fullName); Check.that(i).is(indexOf(), list); Think of all the if statements it would have taken to hand-code these checks! Testing Argument Properties With Klojang Check, you can test not just arguments but also argument properties.To do this, provide a Function that extracts the value to be tested from the argument. Java Check.that(fullName).has(String::length, lte(), 100); The CommonProperties class contains some useful functions that can make your life easier: Java import static org.klojang.check.CommonProperties.strlen; import static org.klojang.check.CommonProperties.type; import static org.klojang.check.CommonProperties.abs; Check.that(fullName).has(strlen(), lte(), 100); Check.that(foo).has(type(), instanceOf(), InputStream.class); Check.that(angle).has(abs(), lte(), 90); As the last example illustrates, the word "property" needs to be taken in the broadest sense here. These are really just functions that are passed the argument and return the value to be tested. Providing A Custom Error Message Klojang Check generates a short, informative error message if the input value failsa test: Java Check.that(length).is(gte(), 0); // error message: argument must be >= 0 (was -42) But you can provide your own error message if you prefer: Java Check.that(fullName).has(strlen(), lte(), 100, "full name must not exceed 100 characters"); The message may itself contain message arguments: Java Check.that(fullName).has(strlen(), lte(), maxLength, "full name must not exceed ${0} characters (was ${1})", maxLength fullName.length()); There are a few predefined message arguments that you can use in your error message: Java Check.that(fullName).has(strlen(), lte(), maxLength, "full name must not exceed ${obj} characters (was ${arg})"); This code snippet is exactly equivalent to the previous one, but this time you didn't have to provide any message arguments yourself! ${arg} is the value you are testing, while ${obj} is the value you are testing it against. The reason the latter message argument is called ${obj} is that it is the object of the less-than-or-equal-to relationship, while the argument is used as the subject of that relationship. (For more information, see here.) Throwing A Custom Exception By default, Klojang Check will throw an IllegalArgumentException if the inputvalue fails any of the checks following Check.that(...). This can be customized in two ways: By providing a function that takes a string (the error message) and returns the exception to be thrown; By providing a supplier that supplies the exception to be thrown. Here is an example for each of the two options: Java // Error message "stale connection" is passed to the constructor of IllegalStateException: Check.on(IllegalStateException::new, connection.isOpen()).is(yes(), "stale connection"); Check.that(connection.isOpen()).is(yes(), () -> new IllegalStateException("stale connection")); The CommonExceptions class contains exception factories for some common exceptions: Java import static org.klojang.check.CommonExceptions.STATE; import static org.klojang.check.CommonExceptions.illegalState; Check.on(STATE, connection.isOpen()).is(yes(), "stale connection"); Check.that(connection.isOpen()).is(yes(), illegalState("stale connection")); Combining Checks Sometimes you will want to do tests of form x must be either A or B or of the form either x must be A or y must be B: Java Check.that(collection).is(empty().or(contains(), "FOO")); Check.that(collection1).is(empty().or(collection2, contains(), "FOO")); The latter example nicely maintains the Klojang Check idiom, but if you prefer your code with less syntactical sugar, you can also just write: Java Check.that(collection1).is(empty().or(collection2.contains("FOO")); When combining checks, you can also employ quantifiers: Java import static org.klojang.check.relation.Quantifier.noneOf; import static org.klojang.check.CommonChecks.notEmpty; import static org.klojang.check.CommonChecks.contains; Check.that(collection).is(notEmpty().and(contains(), noneOf(), "FOO", "BAR"); Conclusion We hope this article has given you a flavor of how you can use Klojang Check to much more systematically separate precondition validation from business logic. Klojang Check's conciseness hopefully lowers the bar significantly to just list everything that should be the case for your program or method to continue normally — and write a check for it!

Building APIs and their related components is often a tedious task, especially when dealing with complex data models and architectures like microservices. Repetitive coding, setting up the configurations, and the overhead of building unit tests can quickly become time-consuming. Here's how AI tools, like ChatGPT, can be a game-changer. Harnessing AI Tools (Chat GPT) In API Development To understand the capabilities of ChatGPT, let's dive into a hands-on example. We're tasked with developing a REST API to manage users' Personally Identifiable Information (PII). The process entails: Database object creation (Stored procedure) Repository (Interface for Database Interaction and Data Access Object) Service (Business Logic Layer) REST Controller (Exposing Endpoints) Database Objects To build a stored procedure, first provide the table structure to ChatGPT. After that, supply a sample stored procedure. This will allow ChatGPT to anchor to the stored procedure pattern and construct a new procedure for the specified table. Below is the table we provided: MySQL CREATE TABLE `user_pii_data_repo` ( `id` bigint NOT NULL AUTO_INCREMENT, `user_id` int DEFAULT NULL, `loan_id` int DEFAULT NULL, `ext_partner_name` varchar(30) NOT NULL, `ext_partner_source` varchar(30) NOT NULL, `phase` varchar(20) NOT NULL, `session_uuid` varchar(36) DEFAULT NULL, `first_name` varchar(300) DEFAULT NULL, `last_name` varchar(300) DEFAULT NULL, `address1` varchar(300) DEFAULT NULL, `address2` varchar(300) DEFAULT NULL, `city` varchar(300) DEFAULT NULL, `state` char(2) DEFAULT NULL, `zip` varchar(300) DEFAULT NULL, `ssn` varchar(300) DEFAULT NULL, `dob` date DEFAULT NULL, `phone` varchar(300) DEFAULT NULL, `payload_path` varchar(500) DEFAULT NULL, PRIMARY KEY (`id`) ); Here is the stored procedure that has been generated: MySQL CREATE PROCEDURE `p_get_user_pii_data`(IN v_user_id INT, IN v_loan_id INT) BEGIN SELECT id, user_id, loan_id, ext_partner_name, ext_partner_source, phase, session_uuid, first_name, last_name, address1, address2, city, state, zip, ssn, dob, phone, payload_path FROM user_pii_data_repo WHERE user_id = v_user_id AND loan_id = v_loan_id; END; Here is the conversation link. Repository (Interface for Database Interaction and Data Access Object) In our endeavor to construct the repository layer, we employed a dedicated ChatGPT thread. We supplied it with the previously mentioned stored procedure and table definition. Leveraging ChatGPT's capabilities, we successfully generated: A JPA entity. A comprehensive Repository class encompassing both the interface and its implementation. Post-generation, we tasked ChatGPT with the creation of unit test cases. Impressively, the outcomes were near-perfect. A few minor adjustments were necessary, predominantly related to namespaces and some other minor details. Below are the generated classes and code: Repository Interface and Implementation Java package com.demo.app.api.core.repository.user_pii; import com.demo.app.api.core.model.user_pii.UserPiiData; import java.util.List; public interface UserPiiDataRepository { List<UserPiiData> getUserPiiData(Integer userId, Integer loanId); } Java package com.demo.app.api.core.repository.user_pii; import com.demo.app.api.core.model.user_pii.UserPiiData; import com.demo.app.api.core.repository.EntityManagerThreadLocalHelper; import com.demo.app.api.core.services.common.CryptoService; import com.demo.app.api.utils.Utility; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Qualifier; import org.springframework.stereotype.Repository; import javax.persistence.*; import java.util.List; @Repository @Qualifier("userPiiDataRepository") public class UserPiiDataRepositoryImpl implements UserPiiDataRepository { private final Logger logger = LoggerFactory.getLogger(this.getClass()); @Autowired EntityManagerThreadLocalHelper threadLocalHelper; @Autowired CryptoService cryptoService; @Override public List<UserPiiData> getUserPiiData(Integer userId, Integer loanId) { List<UserPiiData> results = null; try { EntityManager entityManager = threadLocalHelper.getEntityManager(); StoredProcedureQuery query = entityManager.createStoredProcedureQuery(UserPiiDataDbConstants.PROC_UW_AP_GET_USER_PII_DATA, UserPiiData.class); query.registerStoredProcedureParameter(UserPiiDataDbConstants.V_PARAM_USER_ID, Integer.class, ParameterMode.IN); query.registerStoredProcedureParameter(UserPiiDataDbConstants.V_PARAM_LOAN_ID, Integer.class, ParameterMode.IN); query.setParameter(UserPiiDataDbConstants.V_PARAM_USER_ID, userId); query.setParameter(UserPiiDataDbConstants.V_PARAM_LOAN_ID, loanId); query.execute(); results = query.getResultList(); for (UserPiiData userPiiData : results) { if (userPiiData != null) { if (userPiiData.getFirstName() != null) { String decryptedFirstName = this.cryptoService.decrypt(userPiiData.getFirstName()); userPiiData.setFirstName(decryptedFirstName); } ..... if (userPiiData.getDob() != null) { String formattedDate = Utility.getFormattedDobMMDDYYYY(userPiiData.getDob()); userPiiData.setFormattedDob(formattedDate); } } entityManager.detach(userPiiData); } threadLocalHelper.closeEntityManager(); } catch(Exception ex) { logger.error("UserPiiDataRepositoryImpl::getUserPiiData - an error occurred while fetching user pii data", ex); threadLocalHelper.closeEntityManager(); } return results; } } Java package com.demo.app.api.core.repository.user_pii; public interface UserPiiDataDbConstants { String PROC_UW_AP_GET_USER_PII_DATA = "p_uw_ap_get_user_pii_data"; String V_PARAM_USER_ID = "v_user_id"; String V_PARAM_LOAN_ID = "v_loan_id"; } Unit Tests Java import static org.junit.jupiter.api.Assertions.*; import static org.mockito.Mockito.*; import com.demo.app.api.core.model.user_pii.UserPiiData; import com.demo.app.api.core.repository.EntityManagerReadOnlyThreadLocalHelper; import com.demo.app.api.core.services.common.CryptoService; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.mockito.InjectMocks; import org.mockito.Mock; import org.mockito.MockitoAnnotations; import javax.persistence.EntityManager; import javax.persistence.StoredProcedureQuery; import java.util.Collections; import java.util.Date; import java.util.List; public class UserPiiDataRepositoryImplTest { @InjectMocks private UserPiiDataRepositoryImpl userPiiDataRepository; @Mock private EntityManagerReadOnlyThreadLocalHelper threadLocalHelper; @Mock private CryptoService cryptoService; @Mock private EntityManager entityManager; @Mock private StoredProcedureQuery storedProcedureQuery; @BeforeEach public void setUp() { MockitoAnnotations.openMocks(this); } @Test public void testGetUserPiiData() { // Arrange UserPiiData sampleData = new UserPiiData(); sampleData.setFirstName("encryptedFirstName"); sampleData.setLastName("encryptedLastName"); sampleData.setAddress1("encryptedAddress1"); sampleData.setCity("encryptedCity"); sampleData.setZip("encryptedZip"); sampleData.setSsn("encryptedSsn"); sampleData.setPhone("encryptedPhone"); sampleData.setDob(new Date()); when(threadLocalHelper.getEntityManager()).thenReturn(entityManager); when(entityManager.createStoredProcedureQuery(UserPiiDataDbConstants.PROC_UW_AP_GET_USER_PII_DATA, UserPiiData.class)).thenReturn(storedProcedureQuery); when(storedProcedureQuery.getResultList()).thenReturn(Collections.singletonList(sampleData)); when(cryptoService.decrypt("encryptedFirstName")).thenReturn("John"); when(cryptoService.decrypt("encryptedLastName")).thenReturn("Doe"); when(cryptoService.decrypt("encryptedAddress1")).thenReturn("123 Main St."); when(cryptoService.decrypt("encryptedCity")).thenReturn("Test City"); when(cryptoService.decrypt("encryptedZip")).thenReturn("12345"); when(cryptoService.decrypt("encryptedSsn")).thenReturn("123-45-6789"); when(cryptoService.decrypt("encryptedPhone")).thenReturn("123-456-7890"); // Act List<UserPiiData> result = userPiiDataRepository.getUserPiiData(1, 100); // Assert assertEquals(1, result.size()); UserPiiData fetchedData = result.get(0); assertEquals("John", fetchedData.getFirstName()); assertEquals("Doe", fetchedData.getLastName()); assertEquals("123 Main St.", fetchedData.getAddress1()); assertEquals("Test City", fetchedData.getCity()); assertEquals("12345", fetchedData.getZip()); assertEquals("123-45-6789", fetchedData.getSsn()); assertEquals("123-456-7890", fetchedData.getPhone()); } } During our initial execution, we encountered challenges with the code generated for the repository implementation. The data stored in our database was encrypted, necessitating decryption of the persisted values. We refined the repository layer's code to address this, integrating decryption mechanisms. After implementing the modifications, we supplied ChatGPT with the updated repository code, seeking a corresponding unit test case. Remarkably, ChatGPT furnished us with a perfectly functional unit test case. For those interested in the detailed dialogue, here's the link to our ChatGPT conversation thread. Building Business Service layer To deepen our exploration, we extended our requirements to an existing ChatGPT service sample, specifying dependencies like the JPA entity and the repository interface. The AI tool promptly generated classes. The Service Interface outlined a clear contract for fetching user PII data based on user and loan IDs. Its implementation, UserPIIDataServiceImpl, utilized the JPA repository to fetch the required data, and if null, would return an empty list — a thoughtful touch for avoiding potential null pointer exceptions. The unit test, UserPIIDataServiceImplTest, was comprehensive. Using Mockito for mock objects, the test verified the service's functionality by arranging mock data, calling the service, and asserting the expected results. Below are the generated classes and code: Service Interface and Implementation Java package com.demo.app.api.core.services.user_pii; import com.demo.app.api.core.model.user_pii.UserPiiData; import java.util.List; public interface UserPIIDataService { List<UserPiiData> getUserPiiData(Integer userId, Integer loanId); } Java package com.demo.app.api.core.services.user_pii; import com.demo.app.api.core.model.user_pii.UserPiiData; import com.demo.app.api.core.repository.user_pii.UserPiiDataRepository; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Service; import java.util.Collections; import java.util.List; @Service public class UserPIIDataServiceImpl implements UserPIIDataService { @Autowired UserPiiDataRepository userPiiDataRepository; @Override public List<UserPiiData> getUserPiiData(Integer userId, Integer loanId) { List<UserPiiData> piiData = userPiiDataRepository.getUserPiiData(userId, loanId); return piiData != null ? piiData : Collections.emptyList(); } } Unit Tests Java package com.demo.app.api.core.services.user_pii; import com.demo.app.api.core.model.user_pii.UserPiiData; import com.demo.app.api.core.repository.user_pii.UserPiiDataRepository; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.mockito.InjectMocks; import org.mockito.Mock; import org.mockito.MockitoAnnotations; import java.util.Arrays; import java.util.List; import static org.junit.jupiter.api.Assertions.assertEquals; import static org.mockito.Mockito.when; public class UserPIIDataServiceImplTest { @Mock private UserPiiDataRepository userPiiDataRepository; @InjectMocks private UserPIIDataServiceImpl userPIIDataService; @BeforeEach public void setup() { MockitoAnnotations.openMocks(this); } @Test public void testGetUserPiiData() { // Arrange Integer userId = 1; Integer loanId = 101; UserPiiData data1 = new UserPiiData(); data1.setUserId(userId); data1.setLoanId(loanId); data1.setFirstName("John"); UserPiiData data2 = new UserPiiData(); data2.setUserId(userId); data2.setLoanId(loanId); data2.setFirstName("Jane"); List<UserPiiData> mockResponse = Arrays.asList(data1, data2); when(userPiiDataRepository.getUserPiiData(userId, loanId)).thenReturn(mockResponse); // Act List<UserPiiData> result = userPIIDataService.getUserPiiData(userId, loanId); // Assert assertEquals(2, result.size()); assertEquals("John", result.get(0).getFirstName()); assertEquals("Jane", result.get(1).getFirstName()); } } Here is the chat thread for building the service layer. Final Step: REST Controller Generation Having set up our foundational layers, it was time to build our REST Controller. For this, we leveraged a previously existing REST Controller sample as a base template. We provided ChatGPT with this sample alongside the service interface and the necessary entity object. In response, ChatGPT generated a RestController that was tailored to our requirements. Below are the generated classes and code: Java package com.demo.app.api.controller.user_pii; import com.demo.app.api.core.model.user_pii.UserPiiData; import com.demo.app.api.core.services.user_pii.UserPIIDataService; import io.swagger.annotations.Api; import io.swagger.annotations.ApiOperation; import io.swagger.annotations.ApiResponse; import io.swagger.annotations.ApiResponses; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.bind.annotation.*; import java.util.List; @CrossOrigin(origins = "*") @RestController @RequestMapping("/api/mis/v1/ap") @Api(tags = "Controller for fetching User's PII data") public class UserPIIDataController { @Autowired UserPIIDataService userPIIDataService; @GetMapping(path = "/pii-data/user/{userId}/loan/{loanId}", produces = "application/json") @ApiOperation("Fetch user's PII data by user ID and loan ID") @ApiResponses(value = { @ApiResponse(code = 200, message = "Successfully fetched user's PII data"), @ApiResponse(code = 400, message = "Bad request"), @ApiResponse(code = 404, message = "Not found"), @ApiResponse(code = 500, message = "Internal server error") }) public List<UserPiiData> getUserPiiDataByUserIdAndLoanId( @PathVariable(name = "userId") Integer userId, @PathVariable(name = "loanId") Integer loanId) { return this.userPIIDataService.getUserPiiData(userId, loanId); } } A standout feature of the auto-generated code was the inclusion of annotations, specifically ApiResponse and ApiOperation. Without ChatGPT's intervention, these essential annotations might have been inadvertently overlooked, which can affect the clarity and documentation quality of the API. Subsequent to the REST Controller generation, we sought ChatGPT's assistance in producing unit test cases. Initially, the generated tests were designed as if they were calling the actual API. However, our intention was to strictly test the controller's code. After providing the necessary instructions to narrow down the focus, ChatGPT effectively delivered the tailored unit test code. Here's a snapshot of the generated unit test: Java package com.demo.app.api.controller.user_pii; import com.demo.app.api.core.model.user_pii.UserPiiData; import com.demo.app.api.core.services.user_pii.UserPIIDataService; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.mockito.InjectMocks; import org.mockito.Mock; import org.mockito.MockitoAnnotations; import java.util.Collections; import java.util.List; import static org.mockito.Mockito.when; import static org.junit.jupiter.api.Assertions.assertEquals; public class UserPIIDataControllerTest { @Mock private UserPIIDataService userPIIDataService; @InjectMocks private UserPIIDataController userPIIDataController; @BeforeEach public void setup() { MockitoAnnotations.openMocks(this); } @Test public void testGetUserPiiDataByUserIdAndLoanId() { Integer userId = 1; Integer loanId = 1; UserPiiData userPiiData = new UserPiiData(); userPiiData.setUserId(userId); userPiiData.setLoanId(loanId); userPiiData.setFirstName("John"); userPiiData.setLastName("Doe"); List<UserPiiData> expectedUserPiiDataList = Collections.singletonList(userPiiData); when(userPIIDataService.getUserPiiData(userId, loanId)).thenReturn(expectedUserPiiDataList); List<UserPiiData> actualUserPiiDataList = userPIIDataController.getUserPiiDataByUserIdAndLoanId(userId, loanId); assertEquals(expectedUserPiiDataList, actualUserPiiDataList); } } Finally, to ensure everything was functioning as anticipated, we conducted a verification of the API endpoint using Postman. Delightfully, the API behaved exactly as expected, showcasing the practical applicability and precision of the auto-generated code. Here is the chat thread. The above example might seem straightforward, but we've also applied this approach to build API endpoints for inserting data observing a consistent pattern. The real advantage shines through when dealing with extensive tables, say with 30 columns. Manually defining stored procedure parameters and constructing entities with a multitude of attributes — each requiring accurate column mapping — can be a tedious and error-prone process. However, leveraging tools like ChatGPT or other AI utilities eliminates these repetitive tasks. As a result, developers can produce more efficient, well-documented code with reduced effort. Conclusion The technological realm is evolving rapidly. With the advent of AI tools like ChatGPT, developers now possess powerful allies in their coding endeavors. By automating the more tedious and repetitive tasks in API development, these tools not only streamline the process but also drastically improve code quality and accuracy. The experiences shared in this article are a testament to the fact that AI's potential to revolutionize software development isn't mere speculation — it's a reality we're beginning to embrace. As we forge ahead, such collaborations between humans and machines will undoubtedly redefine the landscape of software engineering, opening doors to new possibilities and efficiency levels previously thought unattainable.

Quite possibly, nothing in web3 is more critical — and difficult to do well — than smart contract testing. For individual developers and small teams, the requisite testing tools are often too expensive and hard to use. Fortunately, a new breed of testing techniques is emerging, ones that are both affordable and accessible. In my previous article, I talked about one of the most popular techniques: fuzzing. Fuzzing is a dynamic testing technique capable of identifying errors and vulnerabilities that standard tests don’t typically identify. I also talked about how one of the most powerful fuzzing tools — Diligence Fuzzing — just released support for one of the most powerful development frameworks: Foundry. This combination complements your manual audits by providing new vulnerability detection techniques that you can use to avoid costly contract rewrites. Last time, we looked at Diligence and Foundry in detail and how they worked together. This time, we’ll review why the integration is a big deal and then run through a detailed tutorial on exactly how to implement fuzzing with Diligence Fuzzing and Foundry. Why Fuzzing Is Important As even novice dapp developers know, smart contracts running on blockchains tend to be immutable. In other words, if you deploy a contract and someone (including yourself) discovers a security loophole, there’s nothing you can do to prevent a malicious party from exploiting that weakness. So deploying defect-free contracts is incredibly important. Projects that handle the flow of assets of enormous value, therefore, typically spend thousands of dollars and many months auditing and stress testing their contracts. Fuzzing is a testing method that can supplement your testing practice and find defects that otherwise would have made it to production. Fuzzing works by inputting millions of invalid, unexpected, or (semi) random data points into your smart contract to cause unexpected behavior and stress test the code. The fuzzing tool identifies vulnerabilities and flags anything that doesn’t meet expectations through the Fuzzing dashboard. It’s an incredible tool for identifying edge cases that could result in those dreaded and expensive smart contract hacks. Diligence Fuzzing is a Fuzzing as a Service (FaaS) offering by ConsenSys, inspired by coverage-based fuzzing approaches like AFL and libFuzzer. Released in closed beta earlier this year, Diligence Fuzzing has routinely outperformed its counterparts in terms of overall code coverage, time to coverage, and number of bugs found. Foundry is an open-source framework for Ethereum development — and probably the most popular. With Foundry, you can create smart contracts, deploy, and more. The ability to use Diligence Fuzzing with Foundry was just recently released, pairing the leading fuzzing tool with the lead development framework. So, let’s jump into how easy it is to use the combination to test your Ethereum smart contracts. Testing an Ethereum Smart Contract With Foundry and Diligence Fuzzing Step 1: Install Python and Node The tools we are using in this tutorial are extremely diverse. In order to install Diligence Fuzzing, we will require Python and pip. You can do so by following the instructions available for your OS here. Check that they’ve been installed correctly by running the following commands: $ python --version$ pip --version Next, let’s install Node and npm using the instructions available here and check their version numbers by running: $ node -v$ npm -v Step 2: Create a Foundry Project Let’s now create a Foundry project! First, let’s install foundryup, a package that will make installing Foundry a breeze. (Without it, we would have to build Foundry from source using Rust and Cargo.) $ curl -L https://foundry.paradigm.xyz | bash Finally, let’s install Foundry by simply running: $ foundryup If all goes well, you should see a downloading screen in your terminal that looks something like this: Plain Text .xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx ╔═╗ ╔═╗ ╦ ╦ ╔╗╔ ╔╦╗ ╦═╗ ╦ ╦ Portable and modular toolkit ╠╣ ║ ║ ║ ║ ║║║ ║║ ╠╦╝ ╚╦╝ for Ethereum Application Development ╚ ╚═╝ ╚═╝ ╝╚╝ ═╩╝ ╩╚═ ╩ written in Rust. .xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx Repo : https://github.com/foundry-rs/ Book : https://book.getfoundry.sh/ Chat : https://t.me/foundry_rs/ Support : https://t.me/foundry_support/ Contribute : https://github.com/orgs/foundry-rs/projects/2/ .xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx foundryup: installing foundry (version nightly, tag nightly-6672134672c8e442684d7d9c51fa8f8717b0f600) foundryup: downloading latest forge, cast, anvil, and chisel Out of the tools that foundryup installs for us, the one we’re most interested in is Forge. Using forge, we can initialize a sample Foundry project as follows: $ forge init fuzz_project This will create a new folder in your repository called fuzz_project. Open the project in your favorite code editor (like VS Code). Step 3: Compile the Contract and Perform Normal Testing One of the main reasons Foundry has witnessed a monumental rise in popularity is because of its focus on testing. Foundry is one of the very few frameworks that allow developers to test Solidity smart contracts using Solidity itself (instead of a JavaScript testing framework like Chai or Mocha). In the sample project that Foundry has created for us, you will find a sample contract in the src/ folder and a test contract in the test/ folder. The main contract is extremely basic. It allows you to set a number and increment that number. Plain Text // SPDX-License-Identifier: UNLICENSED pragma solidity ^0.8.13; contract Counter { uint256 public number; function setNumber(uint256 newNumber) public { number = newNumber; } function increment() public { number++; } } Now, let’s take a look at the corresponding test contract. Plain Text // SPDX-License-Identifier: UNLICENSED pragma solidity ^0.8.13; import {Test, console2} from "forge-std/Test.sol"; import {Counter} from "../src/Counter.sol"; contract CounterTest is Test { Counter public counter; function setUp() public { counter = new Counter(); counter.setNumber(0); } function testIncrement() public { counter.increment(); assertEq(counter.number(), 1); } function testSetNumber(uint256 x) public { counter.setNumber(x); assertEq(counter.number(), x); } } Notice how simple this is in contrast to the huge amount of boilerplate you’re required to write while testing contracts using JS frameworks. Testing is also extremely simple. All you have to do is run… $ forge test …to get output that looks something like this: Plain Text [..] Compiling... [..] Compiling 22 files with 0.8.21 [..] Solc 0.8.21 finished in 3.60s Compiler run successful! Running 2 tests for test/Counter.t.sol:CounterTest [PASS] testIncrement() (gas: 28334) [PASS] testSetNumber(uint256) (runs: 256, μ: 27398, ~: 28409) Test result: ok. 2 passed; 0 failed; 0 skipped; finished in 8.32ms Ran 1 test suites: 2 tests passed, 0 failed, 0 skipped (2 total tests) Step 4: Install Diligence Fuzzing In order to perform Diligence Fuzzing in our Foundry project, we will need to install it first. This is extremely simple to do. Run the following command: $ pip3 install diligence-fuzzing Just installing the CLI tool is not enough, though. In order to perform fuzzing, we will require an API key from ConsenSys. To obtain this, create a free Diligence Fuzzing account. Next, choose a subscription tier that meets your needs. For this tutorial, the free tier should be more than sufficient. Once you’ve created an account, you will be redirected to a dashboard that looks something like this: Next, create an API key here. You can name the key anything you want. Once the key is created, keep it handy. We will require it in a later step. Step 5: Perform Diligence Fuzzing Believe it or not, performing fuzzing, one of the most advanced testing techniques for web3 ever created, can be done using a single command. By using Diligence, you seamlessly pick up the foundry fuzz tests and start fuzzing without any additional work. $ fuzz forge test -k <your_api_key> If all goes well, you should see an output that looks something like this: Parsing foundry configCompiling testsCollecting testsCollecting and validating campaigns for submissionPreparing the seed stateSubmitting campaignsYou can view campaign here: https://fuzzing.diligence.tools/campaigns/cmp_a39a98d29554496b91f4a977804d468dDone You can obtain detailed information about your test results by visiting the campaign above. It should look something like this: Conclusion Fuzzing with Diligence Fuzzing and Foundry makes it easy to add powerful fuzzing to your testing tools. With it, you should be able to catch many of the pressing vulnerabilities that may plague your smart contracts. With this integration, in fact, there’s no reason not to include fuzzing in your workflows.

When talking about writing code, we usually assume that we've got a nice, clean slate and everything is done from scratch. And this is understandable — when you're starting a new project, you've got the opportunity to do things just right, and you apply all your accumulated experience and wisdom. It's a chance to realize your vision in code. The problem is it's not often that you get to work on something brand new. Most of the time, we're maintaining something already built before us. When we're presented with a large and unfamiliar system, we call it legacy code, and working with it always takes more effort — invention is like going downhill, analysis is like climbing uphill. There are excellent books on dealing with legacy code once you're thrown in at the deep end. How does it come about, though? Can we prevent it from emerging altogether? Let's try to figure that out, but first, let's define legacy code. Definitions Let's start with the obvious and most used ones. Legacy code is code that uses technology that is not supported anymore or code whose author isn't around. However, those do not convey our feelings when dealing with a tangled mess of lines and using the word legacy as a curse. If we go with just this feeling, legacy code is code we don't want to touch or even code we're afraid to touch. It's tempting to say that this is all just tantrums that it's over-sensitive people badmouthing stuff that doesn't follow the latest fad in programming. But there is an actual problem here — legacy code is code we don't understand and code that is risky and/or difficult to change. That last one does sum up nicely the economic side of things. If the technology is obsolete, trying to change something might require major upgrades and refactoring parts of the system; if the author isn't around, we might break stuff so bad we'll be in a mess for weeks. And this is also the reason why we don't want to touch the code. But this points us to a larger problem than changing technology or rotating authors. Because: Legacy Code Is Useful Code If code is useless, it will end up in the garbage disposal. Nobody is going to be working on it — thus, it will never become legacy. On the other hand, if code is useful, people will likely be adding new functionality to it. That means more dependencies, more interconnectedness between parts of the system, more complexity, and more temptations to take shortcuts. Sometimes, this is called active rot - as people work on a system, they add entropy; it takes extreme discipline and diligence to keep reversing that entropy. This means that all useful code (=all code that people work on) tends to get sucked into the state of legacy. This is an important point. Dealing with legacy code isn't just a one-time thing that happens when you inherit someone else's code base. Unless you take specific measures to prevent it, the code you and your team have written will also become legacy. If we take that point of view, we can no longer explain the problems with legacy away by calling the author of the code an idiot or even saying that the problem is we can't talk to them directly. So, how do we prevent our code from becoming an unchangeable mess? This is where we come back to another definition of legacy code, the one by Michael Feathers: Legacy Code Is Code Without Tests Isn't that a bit too specific? We were just talking about general stuff like the cost of change. Well, automated tests will prevent your code from becoming legacy — provided they've been properly written. So, let's explore how your code can rot, how tests can prevent it, and how we've got to write such tests. Dealing With Creeping Legacy OOP, functional programming, and everything in between teach us how to avoid spaghetti code, each in its own way. The general principles here are well-known. Promote weak coupling and strong cohesion and avoid redundancy; otherwise, you'll end up with a bowl of pasta.Spaghetti code means any attempt to change something causes cascading changes all over the code, and the high cost of change means legacy code. The point we want to make is that tests provide you with an objective indicator of how well you're adhering to those well-known best practices. It makes sense, right? To test a piece of code, you need to run it in isolation. This is only possible if that piece is sufficiently modular. If your code has high coupling, running a part of it in a test will be difficult — you'll need to grab a lot of stuff from all over your code base. If code has low cohesion, then tests will be more expensive because each tested piece will have different dependencies. If object-oriented programming is your jam, then you know the value of polymorphism and encapsulation. Well, both of those enhance testability. Encapsulation means it's easier to isolate a piece of code; polymorphism means you're using interfaces and abstractions, which are not dependent on concrete implementations — making it easier to mock them. The modularity of your code impacts the cost of testing it. If it is sufficiently modular, you don't need to run your entire test suite for every change; you can limit yourself to tests that cover a part of the code. If, for instance, database access only happens from inside one package dedicated to that purpose, then you don't need to re-run all tests for that package whenever changes happen elsewhere. In other words, testability is an objective external indicator of your code quality; writing tests makes your code more reliable, modular, and easy on the eyes. Dealing With Rotating Developers and Dungeon Masters The turnover rate of software engineers is a significant problem — they rarely keep one job for more than two years. This means the code's author often isn't around to talk to; they usually don't get to experience the consequence of their design decisions, so they don't see how a model performs outside an ideal scenario. An engineer arrives at a greenfield project or convinces people that the current system just won't do; we simply have to redo it from scratch. They don't want to work with the legacy of the half-brained individual who worked on it before them. Two years pass, the project gets just as twisted and crooked, the engineer leaves on to a higher salaried position, and when a new programmer arrives, all the problems become the previous idiot's fault. This cycle is something we've witnessed many times. Things might take a different road, and the author of the new system - usually a genuinely talented developer — might turn into a dungeon master (Alberto Brandolini's term). In this case, the developer stays, and they are the only one who knows the system in and out. Consequently, any change in the system goes through them. This means change is costly, the processes are mostly manual, and there is a single person who is always a bottleneck. This is not the fault of that person; the problem is systemic, and paradoxically, it's the same problem as with rotation: knowledge about code only exists in the head of its creator. How do we get it out into the world? We use tools such as code reviews, comments, and documentation. And tests. Because tests are a great way of documenting your code. One of Python's PEPs says that a docstring for a method should basically say what we should feed it, what we should get from it, its side effects, exceptions, and restrictions. This is more or less what you can read from a well-written arrange-act-assert test, provided the assert is informative (see also this). Javas guidelines for Javadoc distinguish between two types of documentation: programming guides and API specs. The guides can contain quite a bit of prose. They describe concepts and terms; they're not what we're looking for here. The API specs, however, are similar in purpose to tests: they are even supposed to contain assertions. This does not mean you can do away with documenting altogether and just rely on tests (at the very least, you'll want your tests to be documented, too). If you're asking yourself why some gimmick was used in a function, you'll need documentation. But if you want to know how that function is used, a test will provide an excellent example, simulate all dependencies, etc. In other words, if the test base for your code is well-written (if the author isn't just testing what is easiest to test), then the code is much easier to understand because you've got a whole bunch of examples of how it should be used. Thus, it won't become legacy if the person who knows the code leaves. Dealing With Obsolete Technology and Changing Requirements No matter how well-written your code is or how good its documentation is, sooner or later, you'll need to migrate to a new technology, the requirements will change, or you'll want to refactor everything. XKCD has the truth of it: All code is rewritten at some point. And again, this is where tests can make your life easier. There's this thing called Ilizarov surgery, when you get a metal frame installed around a damaged limb, and this both preserves the limb and helps direct the growth of fractured bones. Essentially, that's what tests are. They fix the system in a particular state and ensure that whatever break you make inside doesn't affect functionality. This is how Michael Feathers defined refactoring: improving design without changing behavior. When you introduce a new technology, tests allow you to transition. When requirements change, tests preserve the rest of the functionality that hasn't changed and make sure that whatever you do to the code doesn't break what's already working. Big changes like this will always be painful, but with tests and other supporting practices, it is manageable, and your code doesn't become legacy. What This All Means for Writing Tests So, your tests can do many things: they push you to write cleaner code, present examples of how the code is supposed to be used, and serve as a safety net in transitions. How do you write such tests? First and foremost, writing tests should be a habit. If you've written a piece of code five months ago, and today a tester tells you it lacks testability — that's just annoyance. Now, you've got to measure the headache you get from digging into old code versus the headache from making it testable. But if you're covering your code with tests right away, it's all fresh in your mind, so zero headaches. And if you're doing this as a habit, you'll write testable code from the get-go, no changes necessary. If your tests are to serve as an Ilizarov frame for changes, you'll need to be able to run your tests often. First and foremost, this means having a good unit test base that you can run at the drop of a hat. Ideally — it means having a proper TestOps process where all tests can be run automatically. Running tests as a habit goes hand in hand with covering code with tests as a habit, and it means it's easier to localize errors. Of course, your tests need to run quickly for that to work. In addition to hardware requirements, this also means you should keep down the size of the test base. Here is where code modularity also helps, as it allows running just a part of your code base, provided that you're certain that changes won't propagate beyond that part. What also helps is keeping tests at their appropriate level of the pyramid: don't overcrowd the e2e level. It is only for testing business logic, so move everything lower if you can. If something can be run as a unit test without invoking costly dependencies - do that. Together, these things mean that a developer has to have QA as part of their work cycle. Keep tests in mind when writing code, follow the code with tests, and run the test base after finishing a chunk of work: It's also important that you've got the entire pyramid of tests: its different levels reinforce each other. Generally speaking, if you need the tests to be this frame that helps your code heal after a break, then the e2e tests are the ones you'll rely on most because unit tests tend to die along with the old code you're replacing. However, if you're refactoring the front end, the e2e tests will have to be replaced, but the unit- and other tests on the back end will still be of use. Also, writing new e2e tests will be easier if you're using the old ones as a guideline and not inventing everything from scratch. Another point is that if the test base is to provide information about the purpose of your code, then the assertion statements should be informative — as precise as possible. The traditional approach to testing is — change and then poke around to see if everything is all right. And it seems as if the more you poke, the more you care. But preventing creeping system rot is not so much about care as about using the right tools. You need a test base that is: Not haphazard, that covers every module and promotes modularity That informs you about the function of each module That can be run on demand to ensure the changes you're introducing don't change functionality Legacy Is a Team Problem Legacy code is a problem that arises when a team has to deal with code that was written "selfishly" - without following best practices regarding documentation, comments, tests, etc. How a team implements the best practices we've just discussed is also important. As well as being a personal habit for developers, writing tests should also be accepted practice at the organization level. In particular, the definition of done when setting deadlines should include documentation and tests. A deadline often means two completely different things for the programmer and the manager. The manager expects something polished and shiny; the programmer is happy if he can run it once and it doesn't crash. Or maybe the manager is afraid to bother the programmer too much, or maybe they think the programmer's time is more valuable than the tester's, so let's not waste it by making programmers write tests. And if the testers also live by the same deadline, then one month of developing + 1 month of testing will usually turn out to be 1.5 months of developing and then overtime for testers. Whatever the reason is, the only way you get a stable workflow without crunch time is when deadlines are set with the knowledge that code must be shipped along with tests and documentation, maybe even Architectural Decision Records (ADRs) and Requests for Comments (RFCs) — those are excellent at conveying the context and backstory to the future reader of the code. Without all this, even the author will forget their own code once a few months have passed, and the code will become legacy. Promoting testability is important for testers, but the ones who can actually improve it are developers. If the two departments are separate worlds, testing will always be a chore for developers, something that happens once the real work is done. And code is going to be a foreign country for testers. Everyone annoys each other, nobody is happy, and tests do very little to prevent system rot. The thing is, developers do care about writing modular code. There is no fundamental conflict here. It's just that testers need to know how to "speak code," developers need to write tests, and both of them need to talk to each other. It's important to share expertise and have software and QA engineers participate in decisions. This can be done through code reviews and meetings - you can learn each other's practices and things that annoy the other team. There are also the RFCs we've just talked about — as a dev, write down what you're planning to do before you do it, and request comments from other teams; if there is no issue, this won't take up much time, but if QA has other ideas, you might want to hear them before getting elbow-deep in code. In any case, you'll have a record of your thought process for the future, which means it's less likely your code will become legacy. Having a well-structured pyramid of tests requires the same things — proper communication and QA being everyone's concern. Let's talk about excessive test coverage. Of course, an individual programmer or tester can write too many tests all by themself — but very often, excessive coverage masks poor team structure and lack of expertise sharing. Say a developer has written a new piece of code. They'll also write unit tests for that code if they're diligent. Then, the testers will add integration and API tests on top of that. The unit test might already have checked all the corner cases and such in the new function, but the tester doesn't know about that, so they will redo all that work in their own tests. And then, as a final layer, you've got the end-to-end tests, which call everything from start to finish. In the end, the function gets called a dozen times every test run. And what if we need to change that function? If its correct behavior changes, each of the dozen tests that cover it must also be changed. And, of course, you'll forget one or two tests, and they will fail, and the tester who gets the failed result probably won't know why this happened etc., etc. And if the automated testers are a separate team, who also don't know anything about what the other team is doing, the problems multiply again. A personal bugbear of ours is tickets that say — hey, we've got some changes in the code, so please change the tests accordingly. And you'll be lucky if the ticket arrives before the test suite has run and failed because of the changes. This example shows how poor team structure and lack of visibility into each other's work can lead to cumbersome code (in this case, tests) that increases the cost of change. Conclusion Legacy code isn't a result of some accidental event, like a programmer leaving or technology becoming deprecated. A lot of software starts out in the head of one individual; if it's used, it inevitably outgrows that head. A large enterprise or public project might have millions of lines of code (e.g., in 2020, the size of Linux's kernel was 27.8 million lines of code). It doesn't matter how big your head is or how much you care; no single programmer can ever keep that much knowledge in. Legacy code is what you get when the hobby project approach is applied to large systems. The only way such a system doesn't collapse is if you use tools to control it and to share your knowledge with your peers. Automated tests are among the best tools for that purpose. A well-written test base will promote sound code structure, keep a record of how the code should be used, and help you improve design without affecting functionality. All of this goes towards one thing: reducing the cost and risk of change and thus preventing your code from becoming legacy. Providing such a test base isn't a job for one person (although individual responsibilities and habits are important building blocks). It is the responsibility of the team and should be part of deadline estimations. And finally, the team itself should have clear channels of communication. They should be able to talk about the code with each other. This is how code can make a healthy journey from a vision in the head of its creator to a large system supported by a team or a community.