Testing, Deployment, and Maintenance

This is an article from DZone's 2023 Automated Testing Trend Report.For more: Read the Report One of the core capabilities that has seen increased interest in the DevOps community is observability. Observability improves monitoring in several vital ways, making it easier and faster to understand business flows and allowing for enhanced issue resolution. Furthermore, observability goes beyond an operations capability and can be used for testing and quality assurance. Testing has traditionally faced the challenge of identifying the appropriate testing scope. "How much testing is enough?" and "What should we test?" are questions each testing executive asks, and the answers have been elusive. There are fewer arguments about testing new functionality; while not trivial, you know the functionality you built in new features and hence can derive the proper testing scope from your understanding of the functional scope. But what else should you test? What is a comprehensive general regression testing suite, and what previous functionality will be impacted by the new functionality you have developed and will release? Observability can help us with this as well as the unavoidable defect investigation. But before we get to this, let's take a closer look at observability. What Is Observability? Observability is not monitoring with a different name. Monitoring is usually limited to observing a specific aspect of a resource, like disk space or memory of a compute instance. Monitoring one specific characteristic can be helpful in an operations context, but it usually only detects a subset of what is concerning. All monitoring can show is that the system looks okay, but users can still be experiencing significant outages. Observability aims to make us see the state of the system by making data flows "observable." This means that we can identify when something starts to behave out of order and requires our attention. Observability combines logs, metrics, and traces from infrastructure and applications to gain insights. Ideally, it organizes these around workflows instead of system resources and, as such, creates a functional view of the system in use. Done correctly, it lets you see what functionality is being executed and how frequently, and it enables you to identify performance characteristics of the system and workflow. Figure 1: Observability combines metrics, logs, and traces for insights One benefit of observability is that it shows you the actual system. It is not biased by what the designers, architects, and engineers think should happen in production. It shows the unbiased flow of data. The users, over time (and sometimes from the very first day), find ways to use the system quite differently from what was designed. Observability makes such changes in behavior visible. Observability is incredibly powerful in debugging system issues as it allows us to navigate the system to see where problems occur. Observability requires a dedicated setup and some contextual knowledge similar to traceability. Traceability is the ability to follow a system transaction over time through all the different components of our application and infrastructure architecture, which means you have to have common information like an ID that enables this. OpenTelemetry is an open standard that can be used and provides useful guidance on how to set this up. Observability makes identifying production issues a lot easier. And we can use observability for our benefit in testing, too. Observability of Testing: How to Look Left Two aspects of observability make it useful in the testing context: Its ability to make the actual system usage observable and its usefulness in finding problem areas during debugging. Understanding the actual system behavior is most directly useful during performance testing. Performance testing is the pinnacle of testing since it tries to achieve as close to the realistic peak behavior of a system as possible. Unfortunately, performance testing scenarios are often based on human knowledge of the system instead of objective information. For example, performance testing might be based on the prediction of 10,000 customer interactions per hour during a sales campaign based on the information of the sales manager. Observability information can help define the testing scenarios by using the information to look for the times the system was under the most stress in production and then simulate similar situations in the performance test environment. We can use a system signature to compare behaviors. A system signature in the context of observability is the set of values for logs, metrics, and traces during a specific period. Take, for example, a marketing promotion for new customers. The signature of the system should change during that period to show more new account creations with its associated functionality and the related infrastructure showing up as being more "busy." If the signature does not change during the promotion, we would predict that we also don't see the business metrics move (e.g., user sign-ups). In this example, the business metrics and the signature can be easily matched. Figure 2: A system behaving differently in test, which shows up in the system signature In many other cases, this is not true. Imagine an example where we change the recommendation engine to use our warehouse data going forward. We expect the system signature to show increased data flows between the recommendation engine and our warehouse system. You can see how system signatures and the changes of the system signature can be useful for testing; any differences in signature between production and the testing systems should be explainable by the intended changes of the upcoming release. Otherwise, investigation is required. In the same way, information from the production observability system can be used to define a regression suite that reflects the functionality most frequently used in production. Observability can give you information about the workflows still actively in use and which workflows have stopped being relevant. This information can optimize your regression suite both from a maintenance perspective and, more importantly, from a risk perspective, making sure that core functionality, as experienced by the user, remains in a working state. Implementing observability in your test environments means you can use the power of observability for both production issues and your testing defects. It removes the need for debugging modes to some degree and relies upon the same system capability as production. This way, observability becomes how you work across both dev and ops, which helps break down silos. Observability for Test Insights: Looking Right In the previous section, we looked at using observability by looking left or backward, ensuring we have kept everything intact. Similarly, we can use observability to help us predict the success of the features we deliver. Think about a new feature you are developing. During the test cycles, we see how this new feature changes the workflows, which shows up in our observability solution. We can see the new features being used and other features changing in usage as a result. The signature of our application has changed when we consider the logs, traces, and metrics of our system in test. Once we go live, we predict that the signature of the production system will change in a very similar way. If that happens, we will be happy. But what if the signature of the production system does not change as predicted? Let's take an example: We created a new feature that leverages information from previous bookings to better serve our customers by allocating similar seats and menu options. During testing, we tested the new feature with our test data set, and we see an increase in accessing the bookings database while the customer booking is being collated. Once we go live, we realize that the workflows are not utilizing the customer booking database, and we leverage the information from our observability tooling to investigate. We have found a case where the users are not using our new features or are not using the features in the expected way. In either case, this information allows us to investigate further to see whether more change management is required for the users or whether our feature is just not solving the problem in the way we wanted it to. Another way to use observability is to evaluate the performance of your changes in test and the impact on the system signature — comparing this afterwards with the production system signature can give valuable insights and prevent overall performance degradation. Our testing efforts (and the associated predictions) have now become a valuable tool for the business to evaluate the success of a feature, which elevates testing to become a business tool and a real value investment. Figure 3: Using observability in test by looking left and looking right Conclusion While the popularity of observability is a somewhat recent development, it is exciting to see what benefits it can bring to testing. It will create objectiveness for defining testing efforts and results by evaluating them against the actual system behavior in production. It also provides value to developer, tester, and business communities, which makes it a valuable tool for breaking down barriers. Using the same practices and tools across communities drives a common culture — after all, culture is nothing but repeated behaviors. This is an article from DZone's 2023 Automated Testing Trend Report.For more: Read the Report

This is an article from DZone's 2023 Automated Testing Trend Report.For more: Read the Report Artificial intelligence (AI) has revolutionized the realm of software testing, introducing new possibilities and efficiencies. The demand for faster, more reliable, and efficient testing processes has grown exponentially with the increasing complexity of modern applications. To address these challenges, AI has emerged as a game-changing force, revolutionizing the field of automated software testing. By leveraging AI algorithms, machine learning (ML), and advanced analytics, software testing has undergone a remarkable transformation, enabling organizations to achieve unprecedented levels of speed, accuracy, and coverage in their testing endeavors. This article delves into the profound impact of AI on automated software testing, exploring its capabilities, benefits, and the potential it holds for the future of software quality assurance. An Overview of AI in Testing This introduction aims to shed light on the role of AI in software testing, focusing on key aspects that drive its transformative impact. Figure 1: AI in testing Elastically Scale Functional, Load, and Performance Tests AI-powered testing solutions enable the effortless allocation of testing resources, ensuring optimal utilization and adaptability to varying workloads. This scalability ensures comprehensive testing coverage while maintaining efficiency. AI-Powered Predictive Bots AI-powered predictive bots are a significant advancement in software testing. Bots leverage ML algorithms to analyze historical data, patterns, and trends, enabling them to make informed predictions about potential defects or high-risk areas. By proactively identifying potential issues, predictive bots contribute to more effective and efficient testing processes. Automatic Update of Test Cases With AI algorithms monitoring the application and its changes, test cases can be dynamically updated to reflect modifications in the software. This adaptability reduces the effort required for test maintenance and ensures that the test suite remains relevant and effective over time. AI-Powered Analytics of Test Automation Data By analyzing vast amounts of testing data, AI-powered analytical tools can identify patterns, trends, and anomalies, providing valuable information to enhance testing strategies and optimize testing efforts. This data-driven approach empowers testing teams to make informed decisions and uncover hidden patterns that traditional methods might overlook. Visual Locators Visual locators, a type of AI application in software testing, focus on visual elements such as user interfaces and graphical components. AI algorithms can analyze screenshots and images, enabling accurate identification of and interaction with visual elements during automated testing. This capability enhances the reliability and accuracy of visual testing, ensuring a seamless user experience. Self-Healing Tests AI algorithms continuously monitor test execution, analyzing results and detecting failures or inconsistencies. When issues arise, self-healing mechanisms automatically attempt to resolve the problem, adjusting the test environment or configuration. This intelligent resilience minimizes disruptions and optimizes the overall testing process. What Is AI-Augmented Software Testing? AI-augmented software testing refers to the utilization of AI techniques — such as ML, natural language processing, and data analytics — to enhance and optimize the entire software testing lifecycle. It involves automating test case generation, intelligent test prioritization, anomaly detection, predictive analysis, and adaptive testing, among other tasks. By harnessing the power of AI, organizations can improve test coverage, detect defects more efficiently, reduce manual effort, and ultimately deliver high-quality software with greater speed and accuracy. Benefits of AI-Powered Automated Testing AI-powered software testing offers a plethora of benefits that revolutionize the testing landscape. One significant advantage lies in its codeless nature, thus eliminating the need to memorize intricate syntax. Embracing simplicity, it empowers users to effortlessly create testing processes through intuitive drag-and-drop interfaces. Scalability becomes a reality as the workload can be efficiently distributed among multiple workstations, ensuring efficient utilization of resources. The cost-saving aspect is remarkable as minimal human intervention is required, resulting in substantial reductions in workforce expenses. With tasks executed by intelligent bots, accuracy reaches unprecedented heights, minimizing the risk of human errors. Furthermore, this automated approach amplifies productivity, enabling testers to achieve exceptional output levels. Irrespective of the software type — be it a web-based desktop application or mobile application — the flexibility of AI-powered testing seamlessly adapts to diverse environments, revolutionizing the testing realm altogether. Figure 2: Benefits of AI for test automation Mitigating the Challenges of AI-Powered Automated Testing AI-powered automated testing has revolutionized the software testing landscape, but it is not without its challenges. One of the primary hurdles is the need for high-quality training data. AI algorithms rely heavily on diverse and representative data to perform effectively. Therefore, organizations must invest time and effort in curating comprehensive and relevant datasets that encompass various scenarios, edge cases, and potential failures. Another challenge lies in the interpretability of AI models. Understanding why and how AI algorithms make specific decisions can be critical for gaining trust and ensuring accurate results. Addressing this challenge requires implementing techniques such as explainable AI, model auditing, and transparency. Furthermore, the dynamic nature of software environments poses a challenge in maintaining AI models' relevance and accuracy. Continuous monitoring, retraining, and adaptation of AI models become crucial to keeping pace with evolving software systems. Additionally, ethical considerations, data privacy, and bias mitigation should be diligently addressed to maintain fairness and accountability in AI-powered automated testing. AI models used in testing can sometimes produce false positives (incorrectly flagging a non-defect as a defect) or false negatives (failing to identify an actual defect). Balancing precision and recall of AI models is important to minimize false results. AI models can exhibit biases and may struggle to generalize new or uncommon scenarios. Adequate training and validation of AI models are necessary to mitigate biases and ensure their effectiveness across diverse testing scenarios. Human intervention plays a critical role in designing test suites by leveraging their domain knowledge and insights. They can identify critical test cases, edge cases, and scenarios that require human intuition or creativity, while leveraging AI to handle repetitive or computationally intensive tasks. Continuous improvement would be possible by encouraging a feedback loop between human testers and AI systems. Human experts can provide feedback on the accuracy and relevance of AI-generated test cases or predictions, helping improve the performance and adaptability of AI models. Human testers should play a role in the verification and validation of AI models, ensuring that they align with the intended objectives and requirements. They can evaluate the effectiveness, robustness, and limitations of AI models in specific testing contexts. AI-Driven Testing Approaches AI-driven testing approaches have ushered in a new era in software quality assurance, revolutionizing traditional testing methodologies. By harnessing the power of artificial intelligence, these innovative approaches optimize and enhance various aspects of testing, including test coverage, efficiency, accuracy, and adaptability. This section explores the key AI-driven testing approaches, including differential testing, visual testing, declarative testing, and self-healing automation. These techniques leverage AI algorithms and advanced analytics to elevate the effectiveness and efficiency of software testing, ensuring higher-quality applications that meet the demands of the rapidly evolving digital landscape: Differential testing assesses discrepancies between application versions and builds, categorizes the variances, and utilizes feedback to enhance the classification process through continuous learning. Visual testing utilizes image-based learning and screen comparisons to assess the visual aspects and user experience of an application, thereby ensuring the integrity of its look and feel. Declarative testing expresses the intention of a test using a natural or domain-specific language, allowing the system to autonomously determine the most appropriate approach to execute the test. Self-healing automation automatically rectifies element selection in tests when there are modifications to the user interface (UI), ensuring the continuity of reliable test execution. Key Considerations for Harnessing AI for Software Testing Many contemporary test automation tools infused with AI provide support for open-source test automation frameworks such as Selenium and Appium. AI-powered automated software testing encompasses essential features such as auto-code generation and the integration of exploratory testing techniques. Open-Source AI Tools To Test Software When selecting an open-source testing tool, it is essential to consider several factors. Firstly, it is crucial to verify that the tool is actively maintained and supported. Additionally, it is critical to assess whether the tool aligns with the skill set of the team. Furthermore, it is important to evaluate the features, benefits, and challenges presented by the tool to ensure they are in line with your specific testing requirements and organizational objectives. A few popular open-source options include, but are not limited to: Carina – AI-driven, free forever, scriptless approach to automate functional, performance, visual, and compatibility tests TestProject – Offered the industry's first free Appium AI tools in 2021, expanding upon the AI tools for Selenium that they had previously introduced in 2020 for self-healing technology Cerberus Testing – A low-code and scalable test automation solution that offers a self-healing feature called Erratum and has a forever-free plan Designing Automated Tests With AI and Self-Testing AI has made significant strides in transforming the landscape of automated testing, offering a range of techniques and applications that revolutionize software quality assurance. Some of the prominent techniques and algorithms are provided in the tables below, along with the purposes they serve: KEY TECHNIQUES AND APPLICATIONS OF AI IN AUTOMATED TESTING Key Technique Applications Machine learning Analyze large volumes of testing data, identify patterns, and make predictions for test optimization, anomaly detection, and test case generation Natural language processing Facilitate the creation of intelligent chatbots, voice-based testing interfaces, and natural language test case generation Computer vision Analyze image and visual data in areas such as visual testing, UI testing, and defect detection Reinforcement learning Optimize test execution strategies, generate adaptive test scripts, and dynamically adjust test scenarios based on feedback from the system under test Table 1 KEY ALGORITHMS USED FOR AI-POWERED AUTOMATED TESTING Algorithm Purpose Applications Clustering algorithms Segmentation k-means and hierarchical clustering are used to group similar test cases, identify patterns, and detect anomalies Sequence generation models: recurrent neural networks or transformers Text classification and sequence prediction Trained to generate sequences such as test scripts or sequences of user interactions for log analysis Bayesian networks Dependencies and relationships between variables Test coverage analysis, defect prediction, and risk assessment Convolutional neural networks Image analysis Visual testing Evolutionary algorithms: genetic algorithms Natural selection Optimize test case generation, test suite prioritization, and test execution strategies by applying genetic operators like mutation and crossover on existing test cases to create new variants, which are then evaluated based on fitness criteria Decision trees, fandom forests, support vector machines, and neural networks Classification Classification of software components Variational autoencoders and generative adversarial networks Generative AI Used to generate new test cases that cover different scenarios or edge cases by test data generation, creating synthetic data that resembles real-world scenarios Table 2 Real-World Examples of AI-Powered Automated Testing AI-powered visual testing platforms perform automated visual validation of web and mobile applications. They use computer vision algorithms to compare screenshots and identify visual discrepancies, enabling efficient visual testing across multiple platforms and devices. NLP and ML are combined to generate test cases from plain English descriptions. They automatically execute these test cases, detect bugs, and provide actionable insights to improve software quality. Self-healing capabilities are also provided by automatically adapting test cases to changes in the application's UI, improving test maintenance efficiency. Quantum AI-Powered Automated Testing: The Road Ahead The future of quantum AI-powered automated software testing holds great potential for transforming the way testing is conducted. Figure 3: Transition of automated testing from AI to Quantum AI Quantum computing's ability to handle complex optimization problems can significantly improve test case generation, test suite optimization, and resource allocation in automated testing. Quantum ML algorithms can enable more sophisticated and accurate models for anomaly detection, regression testing, and predictive analytics. Quantum computing's ability to perform parallel computations can greatly accelerate the execution of complex test scenarios and large-scale test suites. Quantum algorithms can help enhance security testing by efficiently simulating and analyzing cryptographic algorithms and protocols. Quantum simulation capabilities can be leveraged to model and simulate complex systems, enabling more realistic and comprehensive testing of software applications in various domains, such as finance, healthcare, and transportation. Parting Thoughts AI has significantly revolutionized the traditional landscape of testing, enhancing the effectiveness, efficiency, and reliability of software quality assurance processes. AI-driven techniques such as ML, anomaly detection, NLP, and intelligent test prioritization have enabled organizations to achieve higher test coverage, early defect detection, streamlined test script creation, and adaptive test maintenance. The integration of AI in automated testing not only accelerates the testing process but also improves overall software quality, leading to enhanced customer satisfaction and reduced time to market. As AI continues to evolve and mature, it holds immense potential for further advancements in automated testing, paving the way for a future where AI-driven approaches become the norm in ensuring the delivery of robust, high-quality software applications. Embracing the power of AI in automated testing is not only a strategic imperative but also a competitive advantage for organizations looking to thrive in today's rapidly evolving technological landscape. This is an article from DZone's 2023 Automated Testing Trend Report.For more: Read the Report

Working more than 15 years in IT, I rarely met programmers who enjoy writing tests and only a few people who use something like TDD. Is this really such an uninteresting part of the software development process? In this article, I’d like to share my experience of using TDD. In most of the teams I worked with, programmers wrote code. Often, they didn't write tests at all or added them symbolically. The only mention of the TDD abbreviation made programmers panic. The main reason is that many people misunderstand the meaning of TDD and try to avoid writing tests. It is generally assumed that TDD are usual tests but written before implementation. But this is not quite true. TDD is a culture of writing code. This approach implies a certain order of solving a task and a specific way of thinking. TDD implies solving a task using loops or iterations. Formally, the cycle consists of three phases: Writing a test that gives something to the input and checks the output. In this case, the test doesn’t pass. Writing the simplest implementation with which the test passes. Refactoring. Changing the code without changing the test. The cycle repeats itself until the problem is solved. TDD Cycle I use a slightly different algorithm. In my approach, refactoring is most often the cycle. That is, I write the test, and then I write the code. Next, I write the test again and write the code because refactoring still often requires editing to the test (various mocks, generation of instances, links to existing modules, etc.), but not always. The general algorithm of what we will do I think I won’t describe it as it is done in textbooks. I'll just show you an example of how it works for me. Example Imagine we got a task. No matter how it is described, we can clarify it ourselves, coordinate it with the customer, and solve it. Let’s suppose that the task is described something like this: "Add an endpoint that returns the current time and user information (id, first name, last name, phone number). Also, it is necessary to sign this information based on a secret key." I will not complicate the task to demonstrate it. But in real life, you may need to make a full-fledged digital signature and supplement it with encryption, and this endpoint needs to be added to an existing project. For academic purposes, we will have to create it from scratch. Let's do it using FastAPI. Most programmers just start working on this task without detailed study. They keep everything they can in their head. After all, such a task does not need to be divided into subtasks since it is quite simple and quickly implemented. While working on it, they clarify the requirements of stakeholders and ask questions. And at the end, they write tests anxiously. But we will do it differently. It may seem unexpected, but let's take something from the Agile methodology. Firstly, this task can be divided into logically completed subtasks. You can immediately clarify all the requirements. Secondly, it can be done iteratively, having something working at each step (even incorrectly) that can be demonstrated. Planning Let’s start with the following partition. The First Subtask Make an empty FastAPI app work with one method. Acceptance Criteria There is a FastAPI app, and it can be launched. The GET request "/my-info" returns a response with code 200 and body {} - empty json. The Second Subtask Add a model/User object. At this stage, it will just be a pedantic scheme for the response. You will have to agree with the business on the name of the fields and whether it is necessary to somehow convert the values (filter, clean, or something else). Acceptance Criteria The GET request "/my-info" returns a response with code 200 and body {"user":{"id":1,"firstname":"John","lastname":"Smith","phone":"+995000000000"}. The Third Subtask Add the current time to the response. Again, we need to agree on the time format and the name of the field. Acceptance Criteria The GET request "/my-info" returns a response with code 200 and body {"user":{added earlier},"timestamp":1691377675}. The Fourth Subtask Add a signature. Immediately, some questions to the business appear: Where to add? How to form it? Where to get the key? Where to store? Who has access? And so on… As a result, we use a simple algorithm: We get base64 from the JSON response body. We concatenate with the private key. First, we use an empty string as a key. Then, we take md5 from the received string. We add the result to the X-Signature header. Acceptance Criteria The GET request "/my-info" returns a response described earlier without changes, but with an additional header: "X-Signature":"638e4c9e30b157cc56fadc9296af813a" For this step, the X-Signature is calculated manually. Base64 = eyJ1c2VyIjp7ImlkIjoxLCJmaXJzdG5hbWUiOiJKb2huIiwibGFzdG5hbWUiOiJTbWl0aCIsInBob25lIjoiKzk5NTAwMDAwMDAwMCJ9LCJ0aW1lc3RhbXAiOjE2OTEzNzc2NzV9. Note that the endpoint returns hard-coded values. To what level tasks should be split is up to you. This is just an example. The most important thing will be described further. These four subtasks result in the endpoint that always returns the same response. But there is a question: why have we described the stub in such detail? Here is the reason: these subtasks don’t have to be physically present. They are just steps. They are needed to use the TDD practice. However, their presence on any storage medium other than our memory will make our work much easier. So, let’s begin. Implementation The First Subtask We add the main.py file to the app directory. Python from fastapi import FastAPI app = FastAPI() @app.get("/my-info") async def my_info(): return {} Right after that, we add one test. For example, to the same directory: test_main.py. Python from fastapi.testclient import TestClient from .main import app client = TestClient(app) def test_my_info_success(): response = client.get("/my-info") assert response.status_code == 200 assert response.json() == {} As a result of the first subtask, we added just a few lines of code and a test. At the very beginning, a simple test appeared. It does not cover business requirements at all. It checks only one case — one step. Obviously, writing such a test does not cause much negativity. And at the same time, we have a working code that can be demonstrated. The Second Subtask We add JSON to the verification. To do this, replace the last line in the test. Python result = { "user": { "id": 1, "firstname": "John", "lastname": "Smith", "phone": "+995000000000", }, } assert response.json() == result ❌ Now, the test fails. We change the code so that the test passes. We add the schema file. Python from pydantic import BaseModel class User(BaseModel): id: int firstname: str lastname: str phone: str class MyInfoResponse(BaseModel): user: User We change the main file. We add import. Python from .scheme import MyInfoResponse, User We change the router function. Python @app.get("/my-info", response_model=MyInfoResponse) async def my_info(): my_info_response = MyInfoResponse( user=User( id=1, firstname="John", lastname="Smith", phone="+995000000000", ), ) return my_info_response ✅ Now, the test passes. And we got a working code again. The Third Subtask We add "timestamp": 1691377675 to the test. Python result = { "user": { "id": 1, "firstname": "John", "lastname": "Smith", "phone": "+995000000000", }, "timestamp": 1691377675, } ❌ The test fails again. We change the code so that the test passes. To do this, we add timestamp to the scheme. Python class MyInfoResponse(BaseModel): user: User timestamp: int We add its initialization to the main file. Python my_info_response = MyInfoResponse( user=User( id=1, firstname="John", lastname="Smith", phone="+995000000000", ), timestamp=1691377675, ) ✅ The test passes again. The Fourth Subtask We add the "X-Signature" header verification to the test: "54977504fbe6c7aec318722d9fbcaec8". Python assert response.headers.get("X-Signature") == "638e4c9e30b157cc56fadc9296af813a" ❌ The test fails again. We add this header to the application's response. To do this, we add middleware. After all, we will most likely need a signature for other endpoints of the application. But this is just our choice, which in reality can be different so as not to complicate the code. Let's do it to understand this. We add import Request. Python from fastapi import FastAPI, Request And the middleware function. Python @app.middleware("http") async def add_signature_header(request: Request, call_next): response = await call_next(request) response.headers["X-Signature"] = "638e4c9e30b157cc56fadc9296af813a" return response ✅ The test passes again. At this stage, we have received a ready-made working test for the endpoint. Next, we will change the application, converting it from a stub into a fully working code while checking it with just one ready-made test. This step can already be considered as refactoring. But we will do it in exactly the same small steps. The Fifth Subtask Implement signature calculation. The algorithm is described above, as well as the acceptance criteria, but the signature should change depending on the user's data and timestamp. Let's implement it. ✅ The test passes, and we don't do anything to it at this step. That is, we do a full-fledged refactoring. We add the signature.py file. It contains the following code: Python import base64 import hashlib def generate_signature(data: bytes) -> str: m = hashlib.md5() b64data = base64.b64encode(data) m.update(b64data + b"") return m.hexdigest() We change main.py. We add import. Python from fastapi import FastAPI, Request, Response from .signature import generate_signature We change middleware. Python @app.middleware("http") async def add_signature_header(request: Request, call_next): response = await call_next(request) body = b"" async for chunk in response.body_iterator: body += chunk response.headers["X-Signature"] = generate_signature(body) return Response( content=body, status_code=response.status_code, headers=dict(response.headers), media_type=response.media_type, ) Here is the result of our complication, which wasn’t necessary for us to do. We did not get the best solution since we have to calculate the entire body of the response and form our own Response. But it is quite suitable for our purposes. ✅ The test still passes. The Sixth Subtask Replace timestamp with the actual value of the current time. Acceptance Criteria timestamp in the response returns the actual current time value. The signature is generated correctly. To generate the time, we will use int(time.time()) First, we edit the test. Now, we have to freeze the current time. Import: Python from datetime import datetime from freezegun import freeze_time We make the test look like the one below. Since freezegun accepts either an object or a string with a date, but not unix timestamp, it will have to be converted. Python def test_my_info_success(): initial_datetime = 1691377675 with freeze_time(datetime.utcfromtimestamp(initial_datetime)): response = client.get("/my-info") assert response.status_code == 200 result = { "user": { "id": 1, "firstname": "John", "lastname": "Smith", "phone": "+995000000000", }, "timestamp": initial_datetime, } assert response.json() == result assert response.headers.get("X-Signature") == "638e4c9e30b157cc56fadc9296af813a" Nothing has changed. ✅ That’s why the test still passes. So, we continue refactoring. Changes to the main.py code. Import: Python import time In the response, we replace the time-hard code with a method call. Python timestamp=int(time.time()), ✅ We launch the test — it works. In tests, one often tries to dynamically generate input data and write duplicate functions to calculate the results. I don’t share the idea of this approach as it can potentially contain errors and requires testing as well. The simplest and most reliable way is to input and output data prepared in advance. The only thing that can be used at the same time is configuration data, settings, and some proven fixtures. Now, we will add the settings. The Seventh Subtask Add a private key. We will take it from the settings environment variables. Acceptance Criteria There is a private key (not an empty string). It is part of the signature generation process according to the algorithm described above. The application gets it from the environment variables. For the test, we use the private key: 6hsjkJnsd)s-_=2$%723 As a result, our signature will change to: 479bb02f0f5f1249760573846de2dbc1 We replace the signature verification in the test: Python assert response.headers.get("X-Signature") == "479bb02f0f5f1249760573846de2dbc1" ❌ Now, the test fails. We add file settings.py to get the settings from environment variables. Python from pydantic_settings import BaseSettings class Settings(BaseSettings): security_key: bytes settings = Settings() We add the code for using this key to signature.py. Import: Python from .settings import settings And we replace the string with concatenation with: Python m.update(b64data + settings.security_key) ❌ Now, the test fails. Now, before running the tests, we need to set the environment variable with the correct key. This can be done right before the call, for example, like this: export security_key='6hsjkJnsd)s-_=2$%723' ✅ Now, the test passes. I would not recommend setting the default value in the settings.py file. The variable must be defined. Setting a default incorrect value can lead to hiding an error in production if the value of this variable is not set during deployment. The application will start without errors, but it will give incorrect results. However, in some cases, a working application with incorrect functionality is better than error 503. It's up to you as a developer to decide. The next steps may be replacing the stub of the User object with real values from the database writing additional validation tests and negative scenarios. In any case, you will have to add more acceptance tests at the end. The most important thing here is dividing the task into micro-tasks, writing a simple test for each subtask, and then writing the application code, and after that, if necessary, refactoring. This order in development really helps: Focus on the problem See the result of the subtask clearly Be able to quickly verify the written code Reduce negativity when writing tests Always have at least one test per task As a result, the number of situations when a programmer "overdoes it" and spends much more time solving a problem than he could with a structured approach decreases. Thus, the development time of the feature is reduced, and the quality of the code is improved. In the long term, changes, refactoring, and updates of package versions in the code are easily controlled and implemented with minimal losses. And here is what’s important: TDD should improve development, make it faster, and strengthen it. This is what the word Driven in the abbreviation means. Therefore, it is not necessary to try to write a complete test or acceptance test of the entire task before the start of development. An iterative approach is needed. Tests are only needed to verify the next small step in development. TDD helps answer the question: how do I know that I have achieved my goal (I mean, that the code fragment I wrote works)? The examples can be found here.

Are you looking to get away from proprietary instrumentation? Are you interested in open-source observability but lack the knowledge to just dive right in? This workshop is for you, designed to expand your knowledge and understanding of open-source observability tooling that is available to you today. Dive right into a free, online, self-paced, hands-on workshop introducing you to Prometheus. Prometheus is an open-source systems monitoring and alerting tool kit that enables you to hit the ground running with discovering, collecting, and querying your observability today. Over the course of this workshop, you will learn what Prometheus is, what it is not, install it, start collecting metrics, and learn all the things you need to know to become effective at running Prometheus in your observability stack. Previously, I shared an introduction to Prometheus, installing Prometheus, an introduction to the query language, exploring basic queries, using advanced queries, relabeling metrics in Prometheus, and discovering service targets as free online labs. In this article, you'll learn all about instrumenting your applications using Prometheus client libraries. Your learning path takes you into the wonderful world of instrumenting applications in Prometheus, where you learn all about client libraries for the languages you code in. Note this article is only a short summary, so please see the complete lab found online to work through it in its entirety yourself. The following is a short overview of what is in this specific lab of the workshop. Each lab starts with a goal. In this case, it is as follows: This lab introduces client libraries and shows you how to use them to add Prometheus metrics to applications and services. You'll get hands-on and instrument a sample application to start collecting metrics. You start in this lab reviewing how Prometheus metrics collection works, exploring client library architectures, and reviewing the four basic metrics types (counters, gauges, histograms, and summaries). If you've never collected any type of metrics data before, you're given two systems to help you get started. One is known as the USE method and is known for systems or infrastructure metrics. The other is the RED method, which targets more applications and services. The introduction finishes with a few best practices around naming your metrics and warnings on how to avoid cardinality bombs. Instrumentation in Java For the rest of this lab, you'll be working on exercises that walk you through instrumenting a simple Java application using the Prometheus Java client library. No previous Java experience is required, but there are assumptions made that you have minimum versions of Java and Maven installed. You are provided with a Java project that you can easily download and work from using your favorite IDE. If you don't work in an IDE, use any editor you like as the coding you'll be doing is possible with just cutting and pasting from the lab slides. To install the project locally: Download and unzip the Prometheus Java Metrics Demo from GitLab. Unzip the prometheus-java-metrics-demo-main.zip file in your workshop directory. Open the project in your favorite IDE (examples shown in the lab use VSCode). You'll be building and running the Java application, which is a basic empty service where comments are used to show where your application code would go. Before that block, you see that the instrumentation has been provided for all four of the basic metric types. Once you have built and started the Java JAR file, the output will show you that the setup has been successful: $ cd prometheus-java-metrics-demo-main/ $ mvn clean install (watch for BUILD SUCCESS) $ java -jar target/java_metrics-1.0-SNAPSHOT-jar-with-dependencies.jar Java example metrics setup successful... Java example service started... Now it's just waiting for you to validate the endpoint at localhost:7777/metrics, which displays the metrics: # HELP java_app_s is a summary metric (request size in bytes) # TYPE java_app_s summary java_app_s{quantile="0.5",} 2.679717814859738 java_app_s{quantile="0.9",} 4.566657867333372 java_app_s{quantile="0.99",} 4.927313848318692 java_app_s_count 512.0 java_app_s_sum 1343.9017287309503 # HELP java_app_h is a histogram metric # TYPE java_app_h histogram java_app_h_bucket{le="0.005",} 1.0 java_app_h_bucket{le="0.01",} 1.0 ... java_app_h_bucket{le="10.0",} 512.0 java_app_h_bucket{le="+Inf",} 512.0 java_app_h_count 512.0 java_app_h_sum 1291.5300871683055 # HELP java_app_c is a counter metric # TYPE java_app_c counter java_app_c 512.0 # HELP java_app_g is a gauge metric # TYPE java_app_g gauge java_app_g 5.5811320747117765 While the metrics are exposed in this example on localhost:7777, they will not be scraped by Prometheus until you have updated its configuration to add this new endpoint. Let's update our workshop-prometheus.yml file to add the Java application job as shown along with comments for clarity (this is the minimum needed, with a few custom labels for fun): # workshop config global: scrape_interval: 5s scrape_configs: # Scraping Prometheus. - job_name: "prometheus" static_configs: - targets: ["localhost:9090"] # Scraping java metrics. - job_name: "java_app" static_configs: - targets: ["localhost:7777"] labels: job: "java_app" env: "workshop-lab8" Start the Prometheus instance (for container Prometheus, see the workshop for details) and then watch the console where you started the Java application as it will report each time a new scrape is done by logging Handled :/metrics: $./prometheus --config.file=support/workshop-prometheus.yml ===========Java application log=============== Java example metrics setup successful... Java example service started... Handled :/metrics Handled :/metrics Handled :/metrics Handled :/metrics ... You can validate that the Java metrics you just instrumented in your application are available in the Prometheus console localhost:9090 as shown. Feel free to query and explore: Next up, you'll be creating your own Java metrics application starting with the minimal setup needed to get your Java application running and exposing the path /metrics. Instead of coding it all by hand, you're given a starting point class file found in the project. Instrumenting Basic Metrics Java was chosen as the language due to many developers using this in enterprises, and exposing you to the Prometheus client library usage for a common developer language is a good baseline. The rest of the lab walks through multiple exercises where you start from a blank application template that's provided and code step-by-step the four basic metrics types. You're also walked through a custom build and run of the application each step of the way, with the following process used for each metric type as you work from implementation, to build, to validating that it works: Add the necessary Java client library import statements for the metric type you are adding. Add the code to construct the metric type you are defining. Initialize the new metric in a thread with basic numerical values (often random numbers). Rebuilding the basic Java application to create an updated JAR file you can run. Starting the application and validating the new metric is available on localhost:9999/metrics. Once all four of the basic metric types have been implemented and tested, you learn to update your Prometheus configuration to pick up your application: # workshop config global: scrape_interval: 5s scrape_configs: # Scraping Prometheus. - job_name: "prometheus" static_configs: - targets: ["localhost:9090"] # Scraping java metrics. - job_name: "java_app" static_configs: - targets: ["localhost:9999"] labels: job: "java_app" env: "workshop-lab8" Finally, you verify that you are collecting your Java-instrumented application data by checking through the Prometheus query console: Miss Previous Labs? This is one lab in the more extensive free online workshop. Feel free to start from the very beginning of this workshop here if you missed anything previously. You can always proceed at your own pace and return any time you like as you work your way through this workshop. Just stop and later restart Perses to pick up where you left off. Coming up Next I'll be taking you through the final lab in this workshop where you'll learn all about metrics monitoring at scale and understanding some of the pain points with Prometheus that arise as you start to scale out your observability architecture and start caring more about reliability.. Stay tuned for more hands-on material to help you with your cloud-native observability journey.

(Note: A list of links for all articles in this series can be found at the conclusion of this article.) In Part 4 of this multi-part series on continuous compliance, we presented designs for Compliance Policy Administration Centers (CPAC) that facilitate the management of various compliance artifacts connecting the Regulatory Policies expressed as Compliance-as-Code with technical policies implemented as Policy-as-Code. The separation of Compliance-As-Code and Policy-As-Code is purposeful, as different personas (see Part 1) need to independently manage their respective responsibilities according to their expertise, be they controls and parameters selection, crosswalks mapping across regulations, or policy check implementations. The CPAC enables users to deploy and run technical policy checks according to different Regulatory Policies on different Policy Validation Points (PVPs) and, depending upon the level of generality or specialty of the inspected systems, the CPAC performs specific normalization and aggregation transformations. We presented three different designs for CPAC: two for handling specialized PVPs with their declarative vs. imperative policies, and one for orchestrating diverse PVP formats across heterogeneous IT stack levels and cloud services. In this blog, we present an example implementation of CPAC that supports the declarative policy in Kubernetes, whose design was introduced in section 2 of COMPASS Part 4. There are various policy engines in Kubernetes, such as GateKeeper/OPA, Kyverno, Kube-bench, etc. Here, we explore a CPAC using Open Cluster Management (OCM) to administer the different policy engines. This design is just one example of how a CPAC can be integrated with a PVP, and a CPAC is not limited to this design only. We flexibly allow the extension of our CPAC through plugins to any specific PVP, as we will see in upcoming blog posts in this series. We also describe how our CPAC can connect the compliance artifacts from Compliance-as-Code produced using our OSCAL-based Agile Authoring methodology to artifacts in Policy-as-Code. This bridging is the key enabler of end-to-end continuous compliance: from authoring controls and profiles to mapping to technical policies and rules, to collecting assessment results from PVPs, to aggregating them against regulatory compliance into an encompassing posture for the whole environment. We assume the compliance artifacts have been authored and approved for production runtime using our open-source Trestle-based Agile Authoring tool. Now the challenge is how to deal with the runtime policy execution and integrate the policy with compliance artifacts represented in NIST OSCAL. In this blog, we focus on the Kubernetes policies and related PVPs and show end-to-end compliance management with NIST OSCAL and the technical policies for Kubernetes. Using Open Cluster Management for Managing Policies in Kubernetes In Kubernetes, the cluster configuration comprises policies that are written in a YAML manifest, and its format depends upon which particular policy engine is used. In order to accommodate the differences among policy engines, we have used Open Cluster Management (OCM) in our CPAC. OCM provides various functionalities for managing Kubernetes clusters: Governance Policy Framework to distribute manifests to managed clusters (by a unified object called OCM Policy) and collect the status from managed clusters, PolicyGenerator to compose OCM Policy from raw Kubernetes manifests, Template function to embed parameters in OCM Policy, PolicySets for grouping of policies, and Placement (or PlacementRule)/PlacementBinding for cluster selection. Once an OCM Policy is composed from a Kubernetes manifest specific to a policy engine, it can be deployed and compliance posture status can be collected using the OCM unified approach. The OCM community maintains OCM Policies in the Policy Collection repository. However, these policies are published with compliance metadata and PlacementRule/PlacementBinding embedded, making it difficult to maintain and reuse policies across regulation programs without constant editing of the policies themselves, while considering them regulation agnostic. Figure 1 is a schematic diagram of policy-kyverno-image-pull-policy.yaml. It illustrates the OCM Policy containing not only the Kubernetes manifests, but also additional compliance metadata, PlacementRule, and PlacementBinding. Figure 1: Example of Today's OCM Policy. Compliance metadata, PlacementRule, and PlacementBinding are embedded in OCM Policy In order to make the policies reusable and composable by the OCM PolicyGenerator, we decompose from each policy its set of Kubernetes manifests. We call this manifest set “Policy Resource." Figure 2 is an example of a decomposed policy that contains three raw Kubernetes manifests (in the middle), along with a PolicyGenerator manifest and its associated kustomization.yaml (on the right). The original OCM Policy can be re-composed by running PolicyGenerator in the directory displayed on the left. Figure 2: Decomposed OCM Policy C2P for End-To-End Compliance Automation Enablement Now that we have completely decoupled compliance and policy as OSCAL artifacts and Policy Resource, we bridge compliance into policy that takes compliance artifacts in OSCAL format and applies policies (including installing policy engines) on managed Kubernetes clusters. We call this bridging process "Compliance-to-Policy" (C2P). The Component Definition is an OSCAL entity that provides a mapping of controls to specific rules for a service and its implementation (check) by a PVP. For example, we can have a Component Definition defined for Kubernetes that specifies that cm-6 in NIST SP 800-53 maps to a rule checked by policy-kyverno-image-pull-policy in Kubernetes. Then, C2P interprets this Component Definition by fetching policy-kyverno-image-pull-policy directory and running PolicyGenerator with given compliance metadata to generate OCM Policy. The generated OCM Policy is pushed to GitHub along with Placement and PlacementBinding. OCM automatically distributes the OCM policy to managed clusters specified in Placement and PlacementBinding. Each managed cluster periodically updates the status field of OCM policy in the OCM Hub. C2P collects and summarizes the OCM policy statuses from OCM Hub and pushes it as the compliance posture. Figure 3 illustrates the end-to-end flow diagram of the compliance management and policy administration thus achieved. Figure 3: Diagram of end-to-end of C2P with Trestle, OSCAL, and OCM for multiple Kubernetes clusters Figure 3 depicts the end-to-end flow steps as follows: Regulators provide OSCAL Catalog and Profile by using Trestle-based agile authoring methodology (see also COMPASS Part 3). Vendors or service providers create Component Definition referring to Catalog, Profile, and Policy Resource by Trestle (Component Definition representation in the spreadsheet below). Compliance officers or auditors create Compliance Deployment CR that defines: Compliance information OSCAL artifact URLs Policy Resources URL Inventory information clusterGroups for grouping clusters by label selectors Binding of cluster group and compliance OCM connection information The example Compliance Deployment CR is as follows: YAML apiVersion: compliance-to-policy.io/v1alpha1 kind: ComplianceDeployment metadata: name: nist-high spec: compliance: name: NIST_SP-800-53-HIGH # name of compliance catalog: url: https://raw.githubusercontent.com/usnistgov/oscal-content/main/nist.gov/SP800-53/rev5/json/NIST_SP-800-53_rev5_catalog.json profile: url: https://raw.githubusercontent.com/usnistgov/oscal-content/main/nist.gov/SP800-53/rev5/json/NIST_SP-800-53_rev5_HIGH-baseline_profile.json componentDefinition: url: https://raw.githubusercontent.com/IBM/compliance-to-policy/template/oscal/component-definition.json policyResources: url: https://github.com/IBM/compliance-to-policy/tree/template/policy-resources clusterGroups: - name: cluster-nist-high # name of clusterGroup matchLabels: level: nist-high # label's key value pair of clusterlabel binding: compliance: NIST_SP-800-53-HIGH # compliance name clusterGroups: - cluster-nist-high # clusterGroup name ocm: url: http://localhost:8080 # OCM Hub URL token: secretName: secret-ocm-hub-token # name of secret volume that stores access to hub namespace: c2p # namespace to which C2P deploys generated resources 4. C2P takes OSCAL artifacts and CR, retrieves required policies from Policy Resources, generates OCM Policy using PolicyGenerator, and pushes the generated policies with Placement/PlacementBindingto GitHub. GitOps engine (for example, ArgoCD) pulls the OCM Policies and Placement/PlacementBinding into OCM Hub. OCM Hub distributes them to managed clusters. OCM Hub updates the statuses of OCM Policies of each managed cluster. 5. C2P periodically fetches the statuses of OCM Policy from OCM Hub and pushes compliance posture summary to GitHub. An example compliance posture summary: 6. Compliance officers or auditors check the compliance posture and take appropriate actions. As a result of the decoupling of Compliance and Policy and bridging them by C2P, each persona can effectively play their role without needing to be aware of the specifics of different Kubernetes Policy Engines. Conclusion In this blog, we detailed the making of a Compliance and Policy Administration Center implementation for integrating Regulatory Programs with supportive Kubernetes declarative policies and showed how this design can be applied for the compliance management of the Kubernetes multi-cluster environment. Coming Next Besides configuration policies, regulatory programs also require complex processes and procedures that entail batch processing for their validation such as provided by Policy Validation Points which support imperative language for policies. In our next blog, we will introduce another design of CPAC for integrating PVPs supporting imperative policies such as Auditree. Learn More If you would like to use our C2P tool, see the compliance-to-policy GitHub project. For our open-source Trestle SDK see compliance-trestle to learn about various Trestle CLIs and their usage. For more details on the markdown formats and commands for authoring various compliance artifacts see this tutorial from Trestle. Below are the links to other articles in this series: Compliance Automated Standard Solution (COMPASS), Part 1: Personas and Roles Compliance Automated Standard Solution (COMPASS), Part 2: Trestle SDK Compliance Automated Standard Solution (COMPASS), Part 3: Artifacts and Personas Compliance Automated Standard Solution (COMPASS), Part 4: Topologies of Compliance Policy Administration Centers Compliance Automated Standard Solution (COMPASS), Part 5: A Lack of Network Boundaries Invites a Lack of Compliance

In today's rapidly evolving world of software development and deployment, containerization has emerged as a transformative technology. It has revolutionized the way applications are built, packaged, and deployed, providing agility, scalability, and consistency to development and operations teams alike. Two of the most popular containerization tools, Docker and Kubernetes, play pivotal roles in this paradigm shift. In this blog, we'll dive deep into containerization technologies, explore how Docker and Kubernetes work together, and understand their significance in modern application deployment. Understanding Containerization A containerization is a lightweight form of virtualization that allows you to package an application and its dependencies into a single, portable unit called a container. Containers are isolated, ensuring that an application runs consistently across different environments, from development to production. Unlike traditional virtual machines (VMs), containers share the host OS kernel, which makes them extremely efficient in terms of resource utilization and startup times. Example: Containerizing a Python Web Application Let's consider a Python web application using Flask, a microweb framework. We'll containerize this application using Docker, a popular containerization tool. Step 1: Create the Python Web Application Python # app.py from flask import Flask app = Flask(__name__) @app.route('/') def hello(): return "Hello, Containerization!" if __name__ == '__main__': app.run(debug=True, host='0.0.0.0') Step 2: Create a Dockerfile Dockerfile # Use an official Python runtime as a parent image FROM python:3.9-slim # Set the working directory to /app WORKDIR /app # Copy the current directory contents into the container at /app COPY . /app # Install any needed packages specified in requirements.txt RUN pip install -r requirements.txt # Make port 80 available to the world outside this container EXPOSE 80 # Define environment variable ENV NAME World # Run app.py when the container launches CMD ["python", "app.py"] Step 3: Build and Run the Docker Container Shell # Build the Docker image docker build -t flask-app . # Run the Docker container, mapping host port 4000 to container port 80 docker run -p 4000:80 flask-app This demonstrates containerization by encapsulating the Python web application and its dependencies within a Docker container. The containerized app can be run consistently on various environments, promoting portability and ease of deployment. Containerization simplifies application deployment, ensures consistency, and optimizes resource utilization, making it a crucial technology in modern software development and deployment pipelines. Docker: The Containerization Pioneer Docker, developed in 2013, is widely regarded as the pioneer of containerization technology. It introduced a simple yet powerful way to create, manage, and deploy containers. Here are some key Docker components: Docker Engine The Docker Engine is the core component responsible for running containers. It includes the Docker daemon, which manages containers, and the Docker CLI (Command Line Interface), which allows users to interact with Docker. Docker Images Docker images are lightweight, stand-alone, and executable packages that contain all the necessary code and dependencies to run an application. They serve as the blueprints for containers. Docker Containers Containers are instances of Docker images. They are isolated environments where applications run. Containers are highly portable and can be executed consistently across various environments. Docker's simplicity and ease of use made it a go-to choice for developers and operators. However, managing a large number of containers at scale and ensuring high availability required a more sophisticated solution, which led to the rise of Kubernetes. Kubernetes: Orchestrating Containers at Scale Kubernetes, often abbreviated as K8s, is an open-source container orchestration platform originally developed by Google. It provides a framework for automating the deployment, scaling, and management of containerized applications. Here's a glimpse of Kubernetes' core components: Master Node The Kubernetes master node is responsible for controlling the cluster. It manages container orchestration, scaling, and load balancing. Worker Nodes Worker nodes, also known as Minions, host containers and run the tasks assigned by the master node. They provide the computing resources needed to run containers. Pods Pods are the smallest deployable units in Kubernetes. They can contain one or more containers that share the same network namespace, storage, and IP address. Services Kubernetes services enable network communication between different sets of pods. They abstract the network and ensure that applications can discover and communicate with each other reliably. Deployments Deployments in Kubernetes allow you to declaratively define the desired state of your application and ensure that the current state matches it. This enables rolling updates and automatic rollbacks in case of failures. The Docker-Kubernetes Synergy Docker and Kubernetes are often used together to create a comprehensive containerization and orchestration solution. Docker simplifies the packaging and distribution of containerized applications, while Kubernetes takes care of their deployment and management at scale. Here's how Docker and Kubernetes work together: Building Docker Images: Developers use Docker to build and package their applications into Docker images. These images are then pushed to a container registry, such as Docker Hub or Google Container Registry. Kubernetes Deployments: Kubernetes takes the Docker images and orchestrates the deployment of containers across a cluster of nodes. Developers define the desired state of their application using Kubernetes YAML manifests, including the number of replicas, resource requirements, and networking settings. Scaling and Load Balancing: Kubernetes can automatically scale the number of container replicas based on resource utilization or traffic load. It also manages load balancing to ensure high availability and efficient resource utilization. Service Discovery: Kubernetes services enable easy discovery and communication between different parts of an application. Services can be exposed internally or externally, depending on the use case. Rolling Updates: Kubernetes supports rolling updates and rollbacks, allowing applications to be updated with minimal downtime and the ability to revert to a previous version in case of issues. The Significance in Modern Application Deployment The adoption of Docker and Kubernetes has had a profound impact on modern application deployment practices. Here's why they are crucial: Portability: Containers encapsulate everything an application needs, making it highly portable. Developers can build once and run anywhere, from their local development environment to a public cloud or on-premises data center. Efficiency: Containers are lightweight and start quickly, making them efficient in terms of resource utilization and time to deployment. Scalability: Kubernetes allows applications to scale up or down automatically based on demand, ensuring optimal resource allocation and high availability. Consistency: Containers provide consistency across different environments, reducing the "it works on my machine" problem and streamlining the development and operations pipeline. DevOps Enablement: Docker and Kubernetes promote DevOps practices by enabling developers and operators to collaborate seamlessly, automate repetitive tasks, and accelerate the software delivery lifecycle. Conclusion In conclusion, Docker and Kubernetes are at the forefront of containerization and container orchestration technologies. They have reshaped the way applications are developed, deployed, and managed in the modern era. By combining the simplicity of Docker with the power of Kubernetes, organizations can achieve agility, scalability, and reliability in their application deployment processes. Embracing these technologies is not just a trend but a strategic move for staying competitive in the ever-evolving world of software development. As you embark on your containerization journey with Docker and Kubernetes, remember that continuous learning and best practices are key to success. Stay curious, explore new features, and leverage the vibrant communities surrounding these technologies to unlock their full potential in your organization's quest for innovation and efficiency. Containerization is not just a technology; it's a mindset that empowers you to build, ship, and run your applications with confidence in a rapidly changing digital landscape.

Programming, regardless of the era, has been riddled with bugs that vary in nature but often remain consistent in their basic problems. Whether we're talking about mobile, desktop, server, or different operating systems and languages, bugs have always been a constant challenge. Here's a dive into the nature of these bugs and how we can tackle them effectively. As a side note, if you like the content of this and the other posts in this series, check out my Debugging book that covers this subject. If you have friends who are learning to code, I'd appreciate a reference to my Java Basics book. If you want to get back to Java after a while, check out my Java 8 to 21 book. Memory Management: The Past and the Present Memory management, with its intricacies and nuances, has always posed unique challenges for developers. Debugging memory issues, in particular, has transformed considerably over the decades. Here's a dive into the world of memory-related bugs and how debugging strategies have evolved. The Classic Challenges: Memory Leaks and Corruption In the days of manual memory management, one of the primary culprits behind application crashes or slowdowns was the dreaded memory leak. This would occur when a program consumes memory but fails to release it back to the system, leading to eventual resource exhaustion. Debugging such leaks was tedious. Developers would pour over code, looking for allocations without corresponding deallocations. Tools like Valgrind or Purify were often employed, which would track memory allocations and highlight potential leaks. They provided valuable insights but came with their own performance overheads. Memory corruption was another notorious issue. When a program writes data outside the boundaries of allocated memory, it corrupts other data structures, leading to unpredictable program behavior. Debugging this required understanding the entire flow of the application and checking each memory access. Enter Garbage Collection: A Mixed Blessing The introduction of garbage collectors (GC) in languages brought in its own set of challenges and advantages. On the bright side, many manual errors were now handled automatically. The system would clean up objects not in use, drastically reducing memory leaks. However, new debugging challenges arose. For instance, in some cases, objects remained in memory because unintentional references prevented the GC from recognizing them as garbage. Detecting these unintentional references became a new form of memory leak debugging. Tools like Java's VisualVM or .NET's Memory Profiler emerged to help developers visualize object references and track down these lurking references. Memory Profiling: The Contemporary Solution Today, one of the most effective methods for debugging memory issues is memory profiling. These profilers provide a holistic view of an application's memory consumption. Developers can see which parts of their program consume the most memory, track allocation, and deallocation rates, and even detect memory leaks. Some profilers can also detect potential concurrency issues, making them invaluable in multi-threaded applications. They help bridge the gap between the manual memory management of the past and the automated, concurrent future. Concurrency: A Double-Edged Sword Concurrency, the art of making software execute multiple tasks in overlapping periods, has transformed how programs are designed and executed. However, with the myriad of benefits it introduces, like improved performance and resource utilization, concurrency also presents unique and often challenging debugging hurdles. Let's delve deeper into the dual nature of concurrency in the context of debugging. The Bright Side: Predictable Threading Managed languages, those with built-in memory management systems, have been a boon to concurrent programming. Languages like Java or C# made threading more approachable and predictable, especially for applications that require simultaneous tasks but not necessarily high-frequency context switches. These languages provide in-built safeguards and structures, helping developers avoid many pitfalls that previously plagued multi-threaded applications. Moreover, tools and paradigms, such as promises in JavaScript, have abstracted away much of the manual overhead of managing concurrency. These tools ensure smoother data flow, handle callbacks, and aid in better structuring asynchronous code, making potential bugs less frequent. The Murky Waters: Multi-Container Concurrency However, as technology progressed, the landscape became more intricate. Now, we're not just looking at threads within a single application. Modern architectures often involve multiple concurrent containers, microservices, or functions, especially in cloud environments, all potentially accessing shared resources. When multiple concurrent entities, perhaps running on separate machines or even data centers, try to manipulate shared data, the debugging complexity escalates. Issues arising from these scenarios are far more challenging than traditional localized threading issues. Tracing a bug may involve traversing logs from multiple systems, understanding inter-service communication, and discerning the sequence of operations across distributed components. Reproducing The Elusive: Threading Bugs Thread-related problems have earned a reputation for being some of the hardest to solve. One of the primary reasons is their often non-deterministic nature. A multi-threaded application may run smoothly most of the time but occasionally produce an error under specific conditions, which can be exceptionally challenging to reproduce. One approach to identifying such elusive issues is logging the current thread and/or stack within potentially problematic code blocks. By observing logs, developers can spot patterns or anomalies that hint at concurrency violations. Furthermore, tools that create "markers" or labels for threads can help in visualizing the sequence of operations across threads, making anomalies more evident. Deadlocks, where two or more threads indefinitely wait for each other to release resources, although tricky, can be more straightforward to debug once identified. Modern debuggers can highlight which threads are stuck, waiting for which resources, and which other threads are holding them. In contrast, livelocks present a more deceptive problem. Threads involved in a livelock are technically operational, but they're caught in a loop of actions that render them effectively unproductive. Debugging this requires meticulous observation, often stepping through each thread's operations to spot a potential loop or repeated resource contention without progress. Race Conditions: The Ever-Present Ghost One of the most notorious concurrency-related bugs is the race condition. It occurs when software's behavior becomes erratic due to the relative timing of events, like two threads trying to modify the same piece of data. Debugging race conditions involves a paradigm shift: one shouldn't view it just as a threading issue but as a state issue. Some effective strategies involve field watchpoints, which trigger alerts when particular fields are accessed or modified, allowing developers to monitor unexpected or premature data changes. The Pervasiveness of State Bugs Software, at its core, represents and manipulates data. This data can represent everything from user preferences and current context to more ephemeral states, like the progress of a download. The correctness of software heavily relies on managing these states accurately and predictably. State bugs, which arise from incorrect management or understanding of this data, are among the most common and treacherous issues developers face. Let's delve deeper into the realm of state bugs and understand why they're so pervasive. What Are State Bugs? State bugs manifest when the software enters an unexpected state, leading to malfunction. This might mean a video player that believes it's playing while paused, an online shopping cart that thinks it's empty when items have been added, or a security system that assumes it's armed when it's not. From Simple Variables to Complex Data Structures One reason state bugs are so widespread is the breadth and depth of data structures involved. It's not just about simple variables. Software systems manage vast, intricate data structures like lists, trees, or graphs. These structures can interact, affecting one another's states. An error in one structure or a misinterpreted interaction between two structures can introduce state inconsistencies. Interactions and Events: Where Timing Matters Software rarely acts in isolation. It responds to user input, system events, network messages, and more. Each of these interactions can change the state of the system. When multiple events occur closely together or in an unexpected order, they can lead to unforeseen state transitions. Consider a web application handling user requests. If two requests to modify a user's profile come almost simultaneously, the end state might depend heavily on the precise ordering and processing time of these requests, leading to potential state bugs. Persistence: When Bugs Linger The state doesn't always reside temporarily in memory. Much of it gets stored persistently, be it in databases, files, or cloud storage. When errors creep into this persistent state, they can be particularly challenging to rectify. They linger, causing repeated issues until detected and addressed. For example, if a software bug erroneously marks an e-commerce product as "out of stock" in the database, it will consistently present that incorrect status to all users until the incorrect state is fixed, even if the bug causing the error has been resolved. Concurrency Compounds State Issues As software becomes more concurrent, managing the state becomes even more of a juggling act. Concurrent processes or threads may try to read or modify shared state simultaneously. Without proper safeguards like locks or semaphores, this can lead to race conditions, where the final state depends on the precise timing of these operations. Tools and Strategies to Combat State Bugs To tackle state bugs, developers have an arsenal of tools and strategies: Unit tests: These ensure individual components handle state transitions as expected. State machine diagrams: Visualizing potential states and transitions can help in identifying problematic or missing transitions. Logging and monitoring: Keeping a close eye on state changes in real-time can offer insights into unexpected transitions or states. Database constraints: Using database-level checks and constraints can act as a final line of defense against incorrect persistent states. Exceptions: The Noisy Neighbor When navigating the labyrinth of software debugging, few things stand out quite as prominently as exceptions. They are, in many ways, like a noisy neighbor in an otherwise quiet neighborhood: impossible to ignore and often disruptive. But just as understanding the reasons behind a neighbor's raucous behavior can lead to a peaceful resolution, diving deep into exceptions can pave the way for a smoother software experience. What Are Exceptions? At their core, exceptions are disruptions in the normal flow of a program. They occur when the software encounters a situation it wasn't expecting or doesn't know how to handle. Examples include attempting to divide by zero, accessing a null reference, or failing to open a file that doesn't exist. The Informative Nature of Exceptions Unlike a silent bug that might cause the software to produce incorrect results without any overt indications, exceptions are typically loud and informative. They often come with a stack trace, pinpointing the exact location in the code where the issue arose. This stack trace acts as a map, guiding developers directly to the problem's epicenter. Causes of Exceptions There's a myriad of reasons why exceptions might occur, but some common culprits include: Input errors: Software often makes assumptions about the kind of input it will receive. When these assumptions are violated, exceptions can arise. For instance, a program expecting a date in the format "MM/DD/YYYY" might throw an exception if given "DD/MM/YYYY" instead. Resource limitations: If the software tries to allocate memory when none is available or opens more files than the system allows, exceptions can be triggered. External system failures: When software depends on external systems, like databases or web services, failures in these systems can lead to exceptions. This could be due to network issues, service downtimes, or unexpected changes in the external systems. Programming errors: These are straightforward mistakes in the code. For instance, trying to access an element beyond the end of a list or forgetting to initialize a variable. Handling Exceptions: A Delicate Balance While it's tempting to wrap every operation in try-catch blocks and suppress exceptions, such a strategy can lead to more significant problems down the road. Silenced exceptions can hide underlying issues that might manifest in more severe ways later. Best practices recommend: Graceful degradation: If a non-essential feature encounters an exception, allow the main functionality to continue working while perhaps disabling or providing alternative functionality for the affected feature. Informative reporting: Rather than displaying technical stack traces to end-users, provide friendly error messages that inform them of the problem and potential solutions or workarounds. Logging: Even if an exception is handled gracefully, it's essential to log it for developers to review later. These logs can be invaluable in identifying patterns, understanding root causes, and improving the software. Retry mechanisms: For transient issues, like a brief network glitch, implementing a retry mechanism can be effective. However, it's crucial to distinguish between transient and persistent errors to avoid endless retries. Proactive Prevention Like most issues in software, prevention is often better than cure. Static code analysis tools, rigorous testing practices, and code reviews can help identify and rectify potential causes of exceptions before the software even reaches the end user. Faults: Beyond the Surface When a software system falters or produces unexpected results, the term "fault" often comes into the conversation. Faults, in a software context, refer to the underlying causes or conditions that lead to an observable malfunction, known as an error. While errors are the outward manifestations we observe and experience, faults are the underlying glitches in the system, hidden beneath layers of code and logic. To understand faults and how to manage them, we need to dive deeper than the superficial symptoms and explore the realm below the surface. What Constitutes a Fault? A fault can be seen as a discrepancy or flaw within the software system, be it in the code, data, or even the software's specification. It's like a broken gear within a clock. You may not immediately see the gear, but you'll notice the clock's hands aren't moving correctly. Similarly, a software fault may remain hidden until specific conditions bring it to the surface as an error. Origins of Faults Design shortcomings: Sometimes, the very blueprint of the software can introduce faults. This might stem from misunderstandings of requirements, inadequate system design, or failure to foresee certain user behaviors or system states. Coding mistakes: These are the more "classic" faults where a developer might introduce bugs due to oversights, misunderstandings, or simply human error. This can range from off-by-one errors of incorrectly initialized variables to complex logic errors. External influences: Software doesn't operate in a vacuum. It interacts with other software, hardware, and the environment. Changes or failures in any of these external components can introduce faults into a system. Concurrency issues: In modern multi-threaded and distributed systems, race conditions, deadlocks, or synchronization issues can introduce faults that are particularly hard to reproduce and diagnose. Detecting and Isolating Faults Unearthing faults requires a combination of techniques: Testing: Rigorous and comprehensive testing, including unit, integration, and system testing, can help identify faults by triggering the conditions under which they manifest as errors. Static analysis: Tools that examine the code without executing it can identify potential faults based on patterns, coding standards, or known problematic constructs. Dynamic analysis: By monitoring the software as it runs, dynamic analysis tools can identify issues like memory leaks or race conditions, pointing to potential faults in the system. Logs and monitoring: Continuous monitoring of the software in production, combined with detailed logging, can offer insights into when and where faults manifest, even if they don't always cause immediate or overt errors. Addressing Faults Correction: This involves fixing the actual code or logic where the fault resides. It's the most direct approach but requires accurate diagnosis. Compensation: In some cases, especially with legacy systems, directly fixing a fault might be too risky or costly. Instead, additional layers or mechanisms might be introduced to counteract or compensate for the fault. Redundancy: In critical systems, redundancy can be used to mask faults. For example, if one component fails due to a fault, a backup can take over, ensuring continuous operation. The Value of Learning From Faults Every fault presents a learning opportunity. By analyzing faults, their origins, and their manifestations, development teams can improve their processes, making future versions of the software more robust and reliable. Feedback loops, where lessons from faults in production inform earlier stages of the development cycle, can be instrumental in creating better software over time. Thread Bugs: Unraveling the Knot In the vast tapestry of software development, threads represent a potent yet intricate tool. While they empower developers to create highly efficient and responsive applications by executing multiple operations simultaneously, they also introduce a class of bugs that can be maddeningly elusive and notoriously hard to reproduce: thread bugs. This is such a difficult problem that some platforms eliminated the concept of threads entirely. This created a performance problem in some cases or shifted the complexity of concurrency to a different area. These are inherent complexities, and while the platform can alleviate some of the difficulties, the core complexity is inherent and unavoidable. A Glimpse into Thread Bugs Thread bugs emerge when multiple threads in an application interfere with each other, leading to unpredictable behavior. Because threads operate concurrently, their relative timing can vary from one run to another, causing issues that might appear sporadically. The Common Culprits Behind Thread Bugs Race conditions: This is perhaps the most notorious type of thread bug. A race condition occurs when the behavior of a piece of software depends on the relative timing of events, such as the order in which threads reach and execute certain sections of code. The outcome of a race can be unpredictable, and tiny changes in the environment can lead to vastly different results. Deadlocks: These occur when two or more threads are unable to proceed with their tasks because they're each waiting for the other to release some resources. It's the software equivalent of a stand-off, where neither side is willing to budge. Starvation: In this scenario, a thread is perpetually denied access to resources and thus can't make progress. While other threads might be operating just fine, the starved thread is left in the lurch, causing parts of the application to become unresponsive or slow. Thread thrashing: This happens when too many threads are competing for the system's resources, causing the system to spend more time switching between threads than actually executing them. It's like having too many chefs in a kitchen, leading to chaos rather than productivity. Diagnosing the Tangle Spotting thread bugs can be quite challenging due to their sporadic nature. However, some tools and strategies can help: Thread sanitizers: These are tools specifically designed to detect thread-related issues in programs. They can identify problems like race conditions and provide insights into where the issues are occurring. Logging: Detailed logging of thread behavior can help identify patterns that lead to problematic conditions. Timestamped logs can be especially useful in reconstructing the sequence of events. Stress testing: By artificially increasing the load on an application, developers can exacerbate thread contention, making thread bugs more apparent. Visualization tools: Some tools can visualize thread interactions, helping developers see where threads might be clashing or waiting on each other. Untangling the Knot Addressing thread bugs often requires a blend of preventive and corrective measures: Mutexes and locks: Using mutexes or locks can ensure that only one thread accesses a critical section of code or resource at a time. However, overusing them can lead to performance bottlenecks, so they should be used judiciously. Thread-safe data structures: Instead of retrofitting thread safety onto existing structures, using inherently thread-safe structures can prevent many thread-related issues. Concurrency libraries: Modern languages often come with libraries designed to handle common concurrency patterns, reducing the likelihood of introducing thread bugs. Code reviews: Given the complexity of multithreaded programming, having multiple eyes review thread-related code can be invaluable in spotting potential issues. Race Conditions: Always a Step Ahead The digital realm, while primarily rooted in binary logic and deterministic processes, is not exempt from its share of unpredictable chaos. One of the primary culprits behind this unpredictability is the race condition, a subtle foe that always seems to be one step ahead, defying the predictable nature we expect from our software. What Exactly Is a Race Condition? A race condition emerges when two or more operations must execute in a sequence or combination to operate correctly, but the system's actual execution order is not guaranteed. The term "race" perfectly encapsulates the problem: these operations are in a race, and the outcome depends on who finishes first. If one operation 'wins' the race in one scenario, the system might work as intended. If another 'wins' in a different run, chaos might ensue. Why Are Race Conditions So Tricky? Sporadic occurrence: One of the defining characteristics of race conditions is that they don't always manifest. Depending on a myriad of factors, such as system load, available resources, or even sheer randomness, the outcome of the race can differ, leading to a bug that's incredibly hard to reproduce consistently. Silent errors: Sometimes, race conditions don't crash the system or produce visible errors. Instead, they might introduce minor inconsistencies—data might be slightly off, a log entry might get missed, or a transaction might not get recorded. Complex interdependencies: Often, race conditions involve multiple parts of a system or even multiple systems. Tracing the interaction that causes the problem can be like finding a needle in a haystack. Guarding Against the Unpredictable While race conditions might seem like unpredictable beasts, various strategies can be employed to tame them: Synchronization mechanisms: Using tools like mutexes, semaphores, or locks can enforce a predictable order of operations. For example, if two threads are racing to access a shared resource, a mutex can ensure that only one gets access at a time. Atomic operations: These are operations that run completely independently of any other operations and are uninterruptible. Once they start, they run straight through to completion without being stopped, altered, or interfered with. Timeouts: For operations that might hang or get stuck due to race conditions, setting a timeout can be a useful fail-safe. If the operation isn't complete within the expected time frame, it's terminated to prevent it from causing further issues. Avoid shared state: By designing systems that minimize shared state or shared resources, the potential for races can be significantly reduced. Testing for Races Given the unpredictable nature of race conditions, traditional debugging techniques often fall short. However: Stress testing: Pushing the system to its limits can increase the likelihood of race conditions manifesting, making them easier to spot. Race detectors: Some tools are designed to detect potential race conditions in code. They can't catch everything, but they can be invaluable in spotting obvious issues. Code reviews: Human eyes are excellent at spotting patterns and potential pitfalls. Regular reviews, especially by those familiar with concurrency issues, can be a strong defense against race conditions. Performance Pitfalls: Monitor Contention and Resource Starvation Performance optimization is at the heart of ensuring that software runs efficiently and meets the expected requirements of end users. However, two of the most overlooked yet impactful performance pitfalls developers face are monitor contention and resource starvation. By understanding and navigating these challenges, developers can significantly enhance software performance. Monitor Contention: A Bottleneck in Disguise Monitor contention occurs when multiple threads attempt to acquire a lock on a shared resource, but only one succeeds, causing the others to wait. This creates a bottleneck as multiple threads are contending for the same lock, slowing down the overall performance. Why It's Problematic Delays and deadlocks: Contention can cause significant delays in multi-threaded applications. Worse, if not managed correctly, it can even lead to deadlocks where threads wait indefinitely. Inefficient resource utilization: When threads are stuck waiting, they aren't doing productive work, leading to wasted computational power. Mitigation Strategies Fine-grained locking: Instead of having a single lock for a large resource, divide the resource and use multiple locks. This reduces the chances of multiple threads waiting for a single lock. Lock-free data structures: These structures are designed to manage concurrent access without locks, thus avoiding contention altogether. Timeouts: Set a limit on how long a thread will wait for a lock. This prevents indefinite waiting and can help in identifying contention issues. Resource Starvation: The Silent Performance Killer Resource starvation arises when a process or thread is perpetually denied the resources it needs to perform its task. While it's waiting, other processes might continue to grab available resources, pushing the starving process further down the queue. The Impact Degraded performance: Starved processes or threads slow down, causing the system's overall performance to dip. Unpredictability: Starvation can make system behavior unpredictable. A process that should typically be completed quickly might take much longer, leading to inconsistencies. Potential system failure: In extreme cases, if essential processes are starved for critical resources, it might lead to system crashes or failures. Solutions to Counteract Starvation Fair allocation algorithms: Implement scheduling algorithms that ensure each process gets a fair share of resources. Resource reservation: Reserve specific resources for critical tasks, ensuring they always have what they need to function. Prioritization: Assign priorities to tasks or processes. While this might seem counterintuitive, ensuring critical tasks get resources first can prevent system-wide failures. However, be cautious, as this can sometimes lead to starvation for lower-priority tasks. The Bigger Picture Both monitor contention and resource starvation can degrade system performance in ways that are often hard to diagnose. A holistic understanding of these issues, paired with proactive monitoring and thoughtful design, can help developers anticipate and mitigate these performance pitfalls. This not only results in faster and more efficient systems but also in a smoother and more predictable user experience. Final Word Bugs, in their many forms, will always be a part of programming. But with a deeper understanding of their nature and the tools at our disposal, we can tackle them more effectively. Remember, every bug unraveled adds to our experience, making us better equipped for future challenges. In previous posts in the blog, I delved into some of the tools and techniques mentioned in this post.

I recently began a new role as a software engineer, and in my current position, I spend a lot of time in the terminal. Even though I have been a long-time Linux user, I embarked on my Linux journey after becoming frustrated with setting up a Node.js environment on Windows during my college days. It was during that time that I discovered Ubuntu, and it was then that I fell in love with the simplicity and power of the Linux terminal. Despite starting my Linux journey with Ubuntu, my curiosity led me to try other distributions, such as Manjaro Linux, and ultimately Arch Linux. Without a doubt, I have a deep affection for Arch Linux. However, at my day job, I used macOS, and gradually, I also developed a love for macOS. Now, I have transitioned to macOS as my daily driver. Nevertheless, my love for Linux, especially Arch Linux and the extensive customization it offers, remains unchanged. Anyway, in this post, I will be discussing grep and how I utilize it to analyze logs and uncover insights. Without a doubt, grep has proven to be an exceptionally powerful tool. However, before we delve into grep, let’s first grasp what grep is and how it works. What Is grep and How Does It Work? grep is a powerful command-line utility in Unix-like operating systems used for searching text or regular expressions (patterns) within files. The name “grep” stands for “Global Regular Expression Print.” It’s an essential tool for system administrators, programmers, and anyone working with text files and logs. How It Works When you use grep, you provide it with a search pattern and a list of files to search through. The basic syntax is: grep [options] pattern [file...] Here’s a simple understanding of how it works: Search pattern: You provide a search pattern, which can be a simple string or a complex regular expression. This pattern defines what you’re searching for within the files. Files to search: You can specify one or more files (or even directories) in which grep should search for the pattern. If you don’t specify any files, grep reads from the standard input (which allows you to pipe in data from other commands). Matching lines:grep scans through each line of the specified files (or standard input) and checks if the search pattern matches the content of the line. Output: When a line containing a match is found, grep prints that line to the standard output. If you’re searching within multiple files, grep also prefixes the matching lines with the file name. Options:grep offers various options that allow you to control its behavior. For example, you can make the search case-insensitive, display line numbers alongside matches, invert the match to show lines that don’t match and more. Backstory of Development grep was created by Ken Thompson, one of the early developers of Unix, and its development dates back to the late 1960s. The context of its creation lies in the evolution of the Unix operating system at Bell Labs. Ken Thompson, along with Dennis Ritchie and others, was involved in developing Unix in the late 1960s. As part of this effort, they were building tools and utilities to make the system more practical and user-friendly. One of the tasks was to develop a way to search for patterns within text files efficiently. The concept of regular expressions was already established in the field of formal language theory, and Thompson drew inspiration from this. He created a program that utilized a simple form of regular expressions for searching and printing lines that matched the provided pattern. This program eventually became grep. The initial version of grep used a simple and efficient algorithm to perform the search, which is based on the use of finite automata. This approach allowed for fast pattern matching, making grep a highly useful tool, especially in the early days of Unix when computing resources were limited. Over the years, grep has become an integral part of Unix-like systems, and its functionality and capabilities have been extended. The basic concept of searching for patterns in text using regular expressions, however, remains at the core of grep’s functionality. grep and Log Analysis So you might be wondering how grep can be used for log analysis. Well, grep is a powerful tool that can be used to analyze logs and uncover insights. In this section, I will be discussing how I use grep to analyze logs and find insights. Isolating Errors Debugging often starts with identifying errors in logs. To isolate errors using grep, I use the following techniques: Search for error keywords: Start by searching for common error keywords such as "error", "exception", "fail" or "invalid" . Use case-insensitive searches with the -i flag to ensure you capture variations in case. Multiple pattern search: Use the -e flag to search for multiple patterns simultaneously. For instance, you could search for both "error" and "warning" messages to cover a wider range of potential issues. Contextual search: Use the -C flag to display a certain number of lines of context around each match. This helps you understand the context in which an error occurred. Tracking Down Issues Once you’ve isolated errors, it’s time to dig deeper and trace the source of the issue: Timestamp-based search: If your logs include timestamps, use them to track down the sequence of events leading to an issue. You can use grep along with regular expressions to match specific time ranges. Unique identifiers: If your application generates unique identifiers for events, use these to track the flow of events across log entries. Search for these identifiers using grep. Combining with other tools: Combine grep with other command-line tools like sort, uniq, and awk to aggregate and analyze log entries based on various criteria. Identifying Patterns Log analysis is not just about finding errors; it’s also about identifying patterns that might provide insights into performance or user behavior: Frequency analysis: Use grep to count the occurrence of specific patterns. This can help you identify frequently occurring events or errors. Custom pattern matching: Leverage regular expressions to define custom patterns based on your application’s unique log formats. Anomaly detection: Regular expressions can also help you detect anomalies by defining what “normal” log entries look like and searching for deviations from that pattern. Conclusion In the world of debugging and log analysis, grep is a tool that can make a significant difference. Its powerful pattern-matching capabilities, combined with its versatility in handling regular expressions, allow you to efficiently isolate errors, track down issues, and identify meaningful patterns in your log files. With these techniques in your toolkit, you’ll be better equipped to unravel the mysteries hidden within your logs and ensure the smooth operation of your systems and applications. Happy log hunting! Remember, practice is key. The more you experiment with grep and apply these techniques to your real-world scenarios, the more proficient you’ll become at navigating through log files and gaining insights from them. Examples Isolating Errors Search for lines containing the word “error” in a log file: grep -i "error" application.log Search for lines containing either “error” or “warning” in a log file: grep -i -e "error" -e "warning" application.log Display lines containing the word “error” along with 2 lines of context before and after: grep -C 2 "error" application.log Tracking Down Issues Search for log entries within a specific time range (using regular expressions for timestamp matching): grep "^\[2023-08-31 10:..:..]" application.log Search for entries associated with a specific transaction ID: grep "TransactionID: 12345" application.log Count the occurrences of a specific error: grep -c "Connection refused" application.log Identifying Patterns Count the occurrences of each type of error in a log file: grep -i -o "error" application.log | sort | uniq -c Search for log entries containing IP addresses: grep -E "[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+" application.log Detect unusual patterns using negative lookaheads in regular expressions: grep -E "^(?!.*normal).*error" application.log Lastly, I hope you enjoyed reading this and got a chance to learn something new from this post. If you have any grep tips or how you started your Linux journey feel free to comment below as I would love to hear them.

Beyond Unit Testing Test-driven development (TDD) is a well-regarded technique for an improved development process, whether developing new code or fixing bugs. First, write a test that fails, then get it to work minimally, then get it to work well; rinse and repeat. The process keeps the focus on value-added work and leverages the test process as a challenge to improving the design being tested rather than only verifying its behavior. This, in turn, also improves the quality of your tests, which become a more valued part of the overall process rather than a grudgingly necessary afterthought. The common discourse on TDD revolves around testing relatively small, in-process units, often just a single class. That works great, but what about the larger 'deliverable' units? When writing a microservice, it's the services that are of primary concern, while the various smaller implementation constructs are simply enablers for that goal. Testing of services is often thought of as outside the scope of a developer working within a single codebase. Such tests are often managed separately, perhaps by a separate team, using different tools and languages. This often makes such tests opaque and of lower quality and adds inefficiencies by requiring a commit/deploy as well as coordination with a separate team. This article explores how to minimize those drawbacks with test-driven development (TDD) principles applied at the service level. It addresses the corollary that such tests would naturally overlap with other API-level tests, such as integration tests, by progressively leveraging the same set of tests for multiple purposes. This can also be framed as a practical guide to shift-left testing from a design as well as implementation perspective. Service Contract Tests A Service Contract Test (SCT) is a functional test against a service API (black box) rather than the internal implementation mechanisms behind it (white box). In their purest form, SCTs do not include subversive mechanisms such as peeking into a database to verify results or rote comparisons against hard-coded JSON blobs. Even when run wholly within the same process, SCTs can loop back to localhost against an embedded HTTP server such as that available in Spring Boot. By limiting access through APIs in this manner, SCTs are agnostic as to whether the mechanisms behind the APIs are contained in the same or a different process(es), while all aspects of serialization/deserialization can be tested even in the simplest test configuration. The general structure of an SCT is: Establish a starting state (preferring to keep tests self-contained) One or more service calls (e.g., testing stateful transitions of updates followed by reads) Deep verification of the structural consistency and expected behavior of the results from each call and across multiple calls Because of the level they operate, SCTs may appear to be more like traditional integration tests (inter-process, involving coordination across external dependencies) than unit tests (intra-process operating wholly within a process space), but there are important differences. Traditional integration test codebases might be separated physically (separate repositories), by ownership (different teams), by implementation (different language and frameworks), by granularity (service vs. method focus), and by level of abstraction. These aspects can lead to costly communication overhead, and the lack of observability between such codebases can lead to redundancies, gaps, or problems tracking how those separately-versioned artifacts relate to each other. With the approach described herein, SCTs can operate at both levels, inter-process for integration-test level comprehensiveness as well as intra-process as part of the fast edit-compile-test cycle during development. By implication, SCTs operating at both levels Co-exist in the development codebase, which ensures that committed code and tests are always in lockstep Are defined using a uniform language and framework(s), which lowers the barriers to shared understanding and reduces communication overhead Reduce redundancy by enabling each test to serve multiple purposes Enable testers and developers to leverage each other’s work or even (depending on your process) remove the need for the dev/tester role distinction to exist in the first place Faking Real Challenges The distinguishing challenge to testing at the service level is the scope. A single service invocation can wind through many code paths across many classes and include interactions with external services and databases. While mocks are often used in unit tests to isolate the unit under test from its collaborators, they have downsides that become more pronounced when testing services. The collaborators at the service testing level are the external services and databases, which, while fewer in number than internal collaboration points, are often more complex. Mocks do not possess the attributes of good programming abstractions that drive modern language design; there is no abstraction, no encapsulation, and no cohesiveness. They simply exist in the context of a test as an assemblage of specific replies to specific method invocations. When testing services, those external collaboration points also tend to be called repeatedly across different tests. As mocks require a precise understanding and replication of collaborator requests/responses that are not even in your control, it is cumbersome to replicate and manage that malleable know-how across all your tests. A more suitable service-level alternative to mocks is fakes, which are an alternative form of test double. A fake object provides a working, stateful implementation of its interface with implementation shortcuts, making it not suitable for production. A fake, for example, may lack actual persistence while otherwise providing a fully (or mostly, as deemed necessary for testing purposes) functionally consistent representation of its 'real' counterpart. While mocks are told how to respond (when you see exactly this, do exactly that), fakes know themselves how to behave (according to their interface contract). Since we can make use of the full range of available programming constructs, such as classes, when building fakes, it is more natural to share them across tests as they encapsulate the complexities of external integration points that need not then be copied/pasted throughout your tests. While the unconstrained versatility of mocks does, at times, have its advantages, the inherent coherence, and shareability of fakes make them appealing as the primary implementation vehicle for the complexity behind SCTs. Alternately Configured Tests (ACTs) Being restricted to an appropriately high level of API abstraction, SCTs can be agnostic about whether fake or real integrations are running underneath. The same set of service contract tests can be run with either set. If the integrated entities, here referred to as task objects (because they often can be run in parallel as exemplified here), are written without assuming particular implementations of other task objects (in accordance with the "L" and "D" principles in SOLID), then different combinations of task implementations can be applied for any purpose. One configuration can run all fakes, another with fakes mixed with real, and another with all real. These Alternately Configured Tests (ACTs) suggest a process, starting with all fakes and moving to all real, possibly with intermediate points of mixing and matching. TDD begins in a walled-off garden with the 'all fakes' configuration, where there is no dependence on external data configurations and which runs fast because it is operating in process. Once all SCTs pass in that test configuration, subsequent configurations are run, each further verifying functionality while having only to focus on the changed elements with respect to the previous working test configuration. The last step is to configure as many “real” task implementations as required to match the intended level of integration testing. ACTs exist when there are at least two test configurations (color code red and green in the diagram above). This is often all that is needed, but at times, it can be useful in order to provide a more incremental sequence from the simplest to the most complex configuration. Intermediate test configurations might be a mixture of fake and real or semi-real task implementations that hit in-memory or containerized implementations of external integration points. Balancing SCTs and Unit Testing Relying on unit tests alone for test coverage of classes with multiple collaborators can be difficult because you're operating at several levels removed from the end result. Coverage tools tell you where there are untried code paths, but are those code paths important, do they have more or less no impact, and are they even executed at all? High test coverage does not necessarily equal confidence-engendering test coverage, which is the real goal. SCTs, in contrast, are by definition always relevant to and important for the purpose of writing services. Unit tests focus on the correctness of classes, while SCTs focus on the correctness of your API. This focus necessarily drives deep thinking about the semantics of your API, which in turn can drive deep thinking about the purpose of your class structure and how the individual parts contribute to the overall result. This has a big impact on the ability to evolve and change: tests against implementation artifacts must be changed when the implementation changes, while tests against services must change when there is a functional service-level change. While there are change scenarios that favor either case, refactoring freedom is often regarded as paramount from an agile perspective. Tests encourage refactoring when you have confidence that they will catch errors introduced by refactoring, but tests can also discourage refactoring to the extent that refactoring results in excessive test rework. Testing at the highest possible level of abstraction makes tests more stable while refactoring. Written at the appropriate level of abstraction, the accessibility of SCTs to a wider community (quality engineers, API consumers) also increases. The best way to understand a system is often through its tests; since those tests are expressed in the same API used by its consumers, they can not only read them but also possibly contribute to them in the spirit of Consumer Driven Contracts. Unit tests, on the other hand, are accessible only to those with deep familiarity with the implementation. Despite these differences, it is not a question of SCTs vs. unit tests, one excluding the other. They each have their purpose; there is a balance between them. SCTs, even in a test configuration with all fakes, can often achieve most of the required code coverage, while unit testing can fill in the gaps. SCTs also do not preclude the benefits of unit testing with TDD for classes with minimal collaborators and well-defined contracts. SCTs can significantly reduce the volume of unit tests against classes without those characteristics. The combination is synergistic. SCT Data Setup To fulfill its purpose, every test must work against a known state. This can be a more challenging problem for service tests than for unit tests since those external integration points are outside of the codebase. Traditional integration tests sometimes handle data setup through an out-of-band process, such as database seeding with automated or manual scripts. This makes tests difficult to understand without having to hunt down that external state or external processes and is subject to breaking at any time through circumstances outside your control. If updates are involved, care must be taken to reset or restore the state at the test start or end. If multiple users happen to run the tests at the same time, care must be taken to avoid update conflicts. A better approach tests that independently set up (and possibly tear down) their own non-conflicting (with other users) target state. For example, an SCT that tests the filtered retrieval of orders would first create an order with a unique ID and with field values set to the test's expectations before attempting to filter on it. Self-contained tests avoid the pitfalls of shared, separately controlled states and are much easier to read as well. Of course, direct data setup is not always directly possible since a given external service might not provide the mutator operations needed for your test setup. There are several ways to handle this: Add testing-only mutator operations. These might even go to a completely different service that isn't otherwise required for production execution. Provide a mixed fake/real test configuration using fakes for the update-constrained external service(s), then employ a mechanism to skip such tests for test configurations where those fake tasks are not active. This at least tests the real versions of other tasks. Externally pre-populated data can still be employed with SCTs and can still be run with fakes, provided those fakes expose equivalent results. For tests whose purpose is not actually validating updates (i.e., updates are only needed for test setup), this at least avoids any conflicts with multiple simultaneous test executions. Providing Early Working Services A test-filtering mechanism can be employed to only run tests against select test configurations. For example, a given SCT may initially work only against fakes but not against other test configurations. That restricted SCT can be checked into your code repository, even though it is not yet working across all test configurations. This orients toward smaller commits and can be useful for handing off work between team members who would then make that test work under more complex configurations. Done right, the follow-on work need only be focused on implementing the real task that doesn’t break the already-working SCTs. This benefit can be extended to API consumers. Fakes can serve to provide early, functionally rich implementations of services without those consumers having to wait for a complete solution. Real-task implementations can be incrementally introduced with little or no consumer code changes. Running Remote Because SCTs are embedded in the same executable space as your service code under test, all can run in the same process. This is beneficial for the initial design phases, including TDD, and running on the same machine provides a simple way for execution, even at the integration test level. Beyond that, it can sometimes be useful to run both on different machines. This might be done, for example, to bring up a test client against a fully integrated running system in staging or production, perhaps also for load/stress testing. An additional use case is for testing backward compatibility. A test client with a previous version of SCTs can be brought up separately from and run against the newer versioned server in order to verify that the older tests still run as expected. Within an automated build/test pipeline, several versions can be managed this way: Summary Service Contract Tests (SCTs) are tests against services. Alternatively, Configured Tests (ACTs) define multiple test configurations that each provide a different task implementation set. A single set of SCTs can be run against any test configuration. Even though SCT can be run with a test configuration that is entirely in process, the flexibility offered by ACTs distinguishes them from traditional unit/component tests. SCTs and unit tests complement one another. With this approach, Test Driven Development (SCT) can be applied to service development. This begins by creating SCTs against the simplest possible in-process test configuration, which is usually also the fastest to run. Once those tests have passed, they can be run against more complex configurations and ultimately against a test configuration of fully 'real' task implementations to achieve the traditional goals of integration or end-to-end testing. Leveraging the same set of SCTs across all configurations supports an incremental development process and yields great economies of scale.

This GitHub Actions workflow builds a Docker image, tags it, and pushes it to one of three container registries. Here’s a Gist with the boilerplate code. Building Docker Images and Pushing to a Container Registry If you haven’t yet integrated GitHub Actions with your private container registry, this tutorial is a good place to start. The resulting workflow will log in to your private registry using the provided credentials, build existing Docker images by path, and push the resulting images to a container registry. We’ll discuss how to do this for GHCR, Docker Hub, and Harbor. Benefits and Use Cases Building and pushing Docker images using your CI/CD platform is a best practice. Here’s how it can improve your developer QoL: Shared builds: Streamline the process, configuration, and dependencies across all builds for easy reproducibility. Saves build minutes: Team members can access existing images instead of rebuilding from the source. Version control: Easily duplicate previous builds with image tags, allowing teams to trace and pinpoint bugs. Building a Docker Image Using GitHub Actions to automate Docker builds will ensure you keep your build config consistent. This only requires substituting your existing build command(s) into the workflow YAML. In this workflow, the image is named after your GitHub repo using the GITHUB_REPOSITORY environment variable as {{ github.repository }. YAML name: Build Docker image on: push: branches: - main jobs: build: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@v3 - name: Build and tag image COMMIT_SHA=$(echo $GITHUB_SHA | cut -c1-7) run: docker build -t ${{ github.repository }:$COMMIT_SHA -f path/to/Dockerfile . Versioning Your Docker Image Tags Never rely on latest tags to version your images. We recommend choosing one of these two versioning conventions when tagging your images: using the GitHub commit hash or following the SemVer spec. Using the GitHub Hash GitHub Actions sets default environment variables that you can access within your workflow. Among these is GITHUB_SHA, which is the commit hash that triggered the workflow. This is a valuable versioning approach because you can trace each image back to its corresponding commit. In general, this convention uses the hash's first seven digits. Here's how we can access the variable and extract these digits: YAML - name: Build and tag image run: | COMMIT_SHA=$(echo $GITHUB_SHA | cut -c1-7) docker build -t ${{ github.repository }:$COMMIT_SHA -f path/to/Dockerfile . Semantic Versioning When using version numbers, it is best practice to follow the SemVer spec. This way, you can increment your version numbers following a consistent structure when releasing new updates and patches. Assuming you store your app’s version in a root file version.txt, you can extract the version number from this file and tag the image in two separate actions: YAML - name: Get version run: | export VERSION=$(cat version.txt) echo "Version: $VERSION" - name: Build and tag image run: docker build -t ${{ github.repository }:$VERSION -f path/to/Dockerfile . Pushing a Docker Image to a Container Registry You can easily build, tag, and push your Docker image to your private container registry of choice within only two or three actions. Here’s a high-level overview of what you’ll be doing: Manually set your authentication token or access credential(s) as repository secrets. Use the echo command to pipe credentials to standard input for registry authentication. This way, no action is required on the user’s part. Populate the workflow with your custom build command. Remember to follow your registry’s tagging convention. Add the push command. You can find the proper syntax in your registry's docs. You may prefer to split each item into its own action for better traceability on a workflow failure. Pushing to GHCR Step 1: Setting up GHCR Credentials In order to access the GitHub API, you’ll want to generate a personal access token. You can do this by going to Settings → Developer → New personal access token (classic) from where you’ll generate a custom token to allow package access. Make sure to select write:packages in the Select scopes section. Store this token as a repository secret called GHCR_TOKEN. Step 2: Action Recipe To Push to GHCR You can add the following actions to your GitHub Actions workflow. This code will log into GHCR, build, and push your Docker image. YAML - name: Log in to ghcr.io run: echo "${{ secrets.GHCR_TOKEN }" | docker login ghcr.io -u ${{ github.actor } --password-stdin - name: Build and tag image run: | COMMIT_SHA=$(echo $GITHUB_SHA | cut -c1-7) docker build -t ghcr.io/${{ github.repository_owner }/${{ github.repository }:$COMMIT_SHA -f path/to/Dockerfile . - name: Push image to GHCR run: docker push ghcr.io/${{ github.repository_owner }/${{ github.repository }:$COMMIT_SHA Pushing to Docker Hub Step 1: Store Your Docker Hub Credentials Using your Docker Hub login credentials, set the following repository secrets: DOCKERHUB_USERNAME DOCKERHUB_PASSWORD Note: You'll need to set up a repo on Docker Hub before you can push your image. Step 2: Action Recipe To Push to Docker Hub Adding these actions to your workflow will automate logging in to Docker Hub, building and tagging an image, and pushing it. YAML - name: Log in to Docker Hub run: | echo ${{ secrets.DOCKERHUB_PASSWORD } | docker login -u ${{ secrets.DOCKERHUB_USERNAME } --password-stdin - name: Build and tag image run: | COMMIT_SHA=$(echo $GITHUB_SHA | cut -c1-7) docker build -t ${{ secrets.DOCKERHUB_USERNAME }/${{ github.repository }:$COMMIT_SHA -f path/to/Dockerfile . - name: Push image to Docker Hub run: docker push ${{ secrets.DOCKERHUB_USERNAME }/${{ github.repository }:$COMMIT_SHA Pushing to Harbor Step 1: Store Your Harbor Access Credentials Create two new repository secrets to store the following info: HARBOR_CREDENTIALS: Your Harbor username and password formatted as username:password HARBOR_REGISTRY_URL: The URL corresponding to your personal Harbor registry Note: You'll need to create a Harbor project before you can push an image to Harbor. Step 2: Action Recipe To Push to Harbor The actions below will authenticate into Harbor, build and tag an image using Harbor-specific conventions, and push the image. YAML - name: Log in to Harbor run: | echo ${{ secrets.HARBOR_CREDENTIALS } | base64 --decode | docker login -u $(cut -d ':' -f1 <<< "${{ secrets.HARBOR_CREDENTIALS }") --password-stdin ${{ secrets.HARBOR_REGISTRY_URL } - name: Build and tag image run: | COMMIT_SHA=$(echo $GITHUB_SHA | cut -c1-7) docker build -t ${{ secrets.HARBOR_REGISTRY_URL }/project-name/${{ github.repository }:$COMMIT_SHA -f path/to/Dockerfile . - name: Push image to Harbor run: docker push ${{ secrets.HARBOR_REGISTRY_URL }/project-name/${{ github.repository }:$COMMIT_SHA Thanks for Reading! I hope you enjoyed today's featured recipes. I'm looking forward to sharing more easy ways you can automate repetitive tasks and chores with GitHub Actions.