Understanding API Technologies: A Comparative Analysis of REST, GraphQL, and Asynchronous APIs
Navigating the Landscape of Smaller Language Models
Modern API Management
When assessing prominent topics across DZone — and the software engineering space more broadly — it simply felt incomplete to conduct research on the larger impacts of data and the cloud without talking about such a crucial component of modern software architectures: APIs. Communication is key in an era when applications and data capabilities are growing increasingly complex. Therefore, we set our sights on investigating the emerging ways in which data that would otherwise be isolated can better integrate with and work alongside other app components and across systems.For DZone's 2024 Modern API Management Trend Report, we focused our research specifically on APIs' growing influence across domains, prevalent paradigms and implementation techniques, security strategies, AI, and automation. Alongside observations from our original research, practicing tech professionals from the DZone Community contributed articles addressing key topics in the API space, including automated API generation via no and low code; communication architecture design among systems, APIs, and microservices; GraphQL vs. REST; and the role of APIs in the modern cloud-native landscape.
Open Source Migration Practices and Patterns
MongoDB Essentials
Integrating assets from diverse platforms and ecosystems presents a significant challenge in enterprise application development, where projects often span multiple technologies and languages. Seamlessly incorporating web-based assets such as JavaScript, CSS, and other resources is a common yet complex requirement in Java web applications. The diversity of development ecosystems — each with its tools, package managers, and distribution methods — complicates including these assets in a unified development workflow. This fragmentation can lead to inefficiencies, increased development time, and potential for errors as developers navigate the intricacies of integrating disparate systems. Recognizing this challenge, the open-source project Npm2Mvn offers a solution to streamline the inclusion of NPM packages into Java workspaces, thereby bridging the gap between the JavaScript and Java ecosystems. Understanding NPM and Maven Before diving into the intricacies of Npm2Mvn, it's essential to understand the platforms it connects: NPM and Maven. NPM (Node Package Manager) is the default package manager for Node.js, primarily used for managing dependencies of various JavaScript projects. It hosts thousands of packages developers provide worldwide, facilitating the sharing and distribution of code. NPM simplifies adding, updating, and managing libraries and tools in your projects, making it an indispensable tool for JavaScript developers. Maven, on the other hand, is a powerful build automation tool used primarily for Java projects. It goes beyond simple build tasks by managing project dependencies, documentation, SCM (Source Code Management), and releases. Maven utilizes a Project Object Model (POM) file to manage a project's build configuration, dependencies, and other elements, ensuring developers can easily manage and build their Java applications. The Genesis of Npm2Mvn Npm2Mvn emerges as a solution to a familiar challenge developers face: incorporating the vast array of JavaScript libraries and frameworks available on NPM into Java projects. While Java and JavaScript operate in markedly different environments, the demand for utilizing web assets (like CSS, JavaScript files, and fonts) within Java applications has grown exponentially. It is particularly relevant for projects that require rich client interfaces or the server-side rendering of front-end components. Many Javascript projects are distributed exclusively through NPM, so like me, if you have found yourself copying and pasting assets from an NPM archive across to your Java web application workspace, then Npm2Mvn is just the solution you need. Key Features of Npm2Mvn Designed to automate the transformation of NPM packages into Maven-compatible jar files, Npm2Mvn makes NPM packages readily consumable by Java developers. This process involves several key steps: Standard Maven repository presentation: Utilizing another open-source project, uHTTPD, NPM2MVN presents itself as a standard Maven repository. Automatic package conversion: When a request for a Maven artifact in the group npm is received, NPM2MVN fetches the package metadata and tarball from NPM. It then enriches the package with additional metadata required for Maven, such as POM files and MANIFEST.MF. Inclusion of additional metadata: Besides standard Maven metadata, NPM2MVN adds specific metadata for Graal native images, enhancing compatibility and performance for projects leveraging GraalVM. Seamless integration into local Maven cache: The final jar file, enriched with the necessary metadata, is placed in the local Maven cache, just like any other artifact, ensuring that using NPM packages in Java projects is as straightforward as adding a Maven dependency. Benefits for Java Developers Npm2Mvn offers several compelling benefits for Java developers: Access to a vast repository of JavaScript libraries: By bridging NPM and Maven, Java developers can easily incorporate thousands of JavaScript libraries and frameworks into their projects. This access significantly expands the resources for enhancing Java applications, especially for UI/UX design, without leaving the familiar Maven ecosystem. Simplified dependency management: Managing dependencies across different ecosystems can be cumbersome. Npm2Mvn streamlines this process, allowing developers to handle NPM packages with the Maven commands they are accustomed to. Enhanced productivity: By automating the conversion of NPM packages to Maven artifacts, NPM2MVN saves developers considerable time and effort. This efficiency boost enables developers to focus more on building their applications than wrestling with package management intricacies. Real-world applications: Projects like Fontawesome, Xterm, and Bootstrap, staples for frontend development, can seamlessly integrate into Java applications. How To Use Using Npm2Mvn is straightforward. Jadaptive, the project's developers, host a repository here. This repository is open and free to use. You can also download a copy of the server to host in a private build environment. To use this service, add the repository entry to your POM file. XML <repositories> <repository> <id>npm2mvn</id> <url>https://npm2mvn.jadaptive.com</url> </repository> </repositories> Now, declare your NPM packages. For example, I am including the JQuery NPM package here. XML <dependency> <groupId>npm</groupId> <artifactId>jquery</artifactId> <version>3.7.1</version> </dependency> That's all we need to include and version manage NPM packages into the classpath. Consuming the NPM Resources in Your Java Application The resources of the NPM package are placed in the jar under a fixed prefix, allowing multiple versions of multiple NPM packages to be available to the JVM via the classpath or module path. For example, if the NPM package bootstrap@v5.3.1 contains a resource with the path css/bootstrap.css, then the Npm2Mvn package will make that resource available at the resource path /npm2mvn/npm/bootstrap/5.3.1/css/boostrap.css. Now that you know the path of the resources in your classpath, you can prepare to consume them in your Java web application by implementing a Servlet or other mechanism to serve the resources from the classpath. How you do this depends on your web application platform and any framework you use. In Spring Boot, we would add a resource handler as demonstrated below. Java @Configuration @EnableWebMvc public class MvcConfig implements WebMvcConfigurer { @Override public void addResourceHandlers(ResourceHandlerRegistry registry) { registry .addResourceHandler("/npm2mvn/**") .addResourceLocations("classpath:/npm2mvn/"); } } With this configuration in a Spring Boot application, we can now reference NPM assets directly in HTML files we use in the application. HTML <script type="text/javascript" src="/npm2mvn/npm/jquery/3.7.1/dist/jquery.min.js"> But What About NPM Scopes? NPM version 2 supports scopes which, according to their website: ... allows you to create a package with the same name as a package created by another user or organization without conflict. In the examples above, we are not using scopes. If the package you require uses a scope, you must modify your pom.xml dependency and the resource path. Taking the FontAwesome project as an example, to include the @fortawesome/fontawesome-free module in our Maven build, we modify the groupId to include the scope as demonstrated below. XML <dependency> <groupId>npm.fortawesome</groupId> <artifactId>fontawesome-free</artifactId> <version>6.5.1</version> </dependency> Similarly, in the resource path, we change the second path value from 'npm' to the same groupId we used above. HTML <link rel="stylesheet" href="/npm2nvm/npm.fortawesome/fontawesome-free/6.5.1/css/all.css"/> You can download a full working Spring Boot example that integrates the Xterm NPM module and add-ons from GitHub. Dependency Generator The website at the hosted version of Npm2Mvn provides a useful utility that developers can use to get the correct syntax for the dependencies needed to build the artifacts. Here we have entered the scope, package, and version to get the correct dependency entry for the Maven build. If the project does not have a scope simply leave the first field blank. Conclusion Npm2Mvn bridges the JavaScript and Java worlds, enhancing developers' capabilities and project possibilities. By simplifying the integration of NPM packages into Java workspaces, Npm2Mvn promotes a more interconnected and efficient development environment. It empowers developers to leverage the best of both ecosystems in their applications.
Debugging is not just about identifying errors — it's about instituting a reliable process for ensuring software health and longevity. In this post, we discuss the role of software testing in debugging, including foundational concepts and how they converge to improve software quality. As a side note, if you like the content of this and the other posts in this series check out my Debugging book that covers this subject. If you have friends that are learning to code I'd appreciate a reference to my Java Basics book. If you want to get back to Java after a while check out my Java 8 to 21 book. The Intersection of Debugging and Testing Debugging and testing play distinct roles in software development. Debugging is the targeted process of identifying and fixing known bugs. Testing, on the other hand, encompasses an adjacent scope, identifying unknown issues by validating expected software behavior across a variety of scenarios. Both are a part of the debug fix cycle which is a core concept in debugging. Before we cover the cycle we should first make sure we're aligned on the basic terminology. Unit Tests Unit tests are tightly linked to debugging efforts, focusing on isolated parts of the application—typically individual functions or methods. Their purpose is to validate that each unit operates correctly in isolation, making them a swift and efficient tool in the debugging arsenal. These tests are characterized by their speed and consistency, enabling developers to run them frequently, sometimes even automatically as code is written within the IDE. Since software is so tightly bound it is nearly impossible to compose unit tests without extensive mocking. Mocking involves substituting a genuine component with a stand-in that returns predefined results, thus a test method can simulate scenarios without relying on the actual object. This is a powerful yet controversial tool. By using mocking we're in effect creating a synthetic environment that might misrepresent the real world. We're reducing the scope of the test which might perpetuate some bugs. Integration Tests Opposite to unit tests, integration tests examine the interactions between multiple units, providing a more comprehensive picture of the system's health. While they cover broader scenarios, their setup can be more complex due to the interactions involved. However, they are crucial in catching bugs that arise from the interplay between different software components. In general, mocking can be used in integration tests but it is discouraged. They take longer to run and are sometimes harder to set up. However, many developers (myself included) would argue that they are the only benchmark for quality. Most bugs express themselves in the seams between the modules and integration tests are better at detecting that. Since they are far more important some developers would argue that unit tests are unnecessary. This isn't true, unit test failures are much easier to read and understand. Since they are faster we can run them during development, even while typing. In that sense, the balance between the two approaches is the important part. Coverage Coverage is a metric that helps quantify the effectiveness of testing by indicating the proportion of code exercised by tests. It helps identify potential areas of the code that have not been tested, which could harbor undetected bugs. However, striving for 100% coverage can be a case of diminishing returns; the focus should remain on the quality and relevance of the tests rather than the metric itself. In my experience, chasing high coverage numbers often results in bad test practices that persist in problems. It is my opinion that unit tests should be excluded from coverage metrics due to the importance of integration tests to overall quality. To get a sense of quality coverage should focus on integration and end-to-end tests. The Debug-Fix Cycle The debug-fix cycle is a structured approach that integrates testing into the debugging process. The stages include identifying the bug, creating a test that reproduces the bug, fixing the bug, verifying the fix with the test, and finally, running the application to ensure the fix works in the live environment. This cycle emphasizes the importance of testing in not only identifying but also in preventing the recurrence of bugs. Notice that this is a simplified version of the cycle with a focus on the testing aspect only. The full cycle includes a discussion of the issue tracking and versioning as part of the whole process. I discuss this more in-depth in other posts in the series and my book. Composing Tests With Debuggers A powerful feature of using debuggers in test composition is their ability to "jump to line" or "set value." Developers can effectively reset the execution to a point before the test and rerun it with different conditions, without recompiling or rerunning the entire suite. This iterative process is invaluable for achieving desired test constraints and improves the quality of unit tests by refining the input parameters and expected outcomes. Increasing test coverage is about more than hitting a percentage; it's about ensuring that tests are meaningful and that they contribute to software quality. A debugger can significantly assist in this by identifying untested paths. When a test coverage tool highlights lines or conditions not reached by current tests, the debugger can be used to force execution down those paths. This helps in crafting additional tests that cover missed scenarios, ensuring that the coverage metric is not just a number but a true reflection of the software's tested state. In this case, you will notice that the next line in the body is a rejectValue call which will throw an exception. I don’t want an exception thrown as I still want to test all the permutations of the method. I can drag the execution pointer (arrow on the left) and place it back at the start of the method. Test-Driven Development How does all of this fit with disciplines like Test-Driven Development (TDD)? It doesn't fit well. Before we get into that let's revisit the basics of TDD. Weak TDD typically means just writing tests before writing the code. Strong TDD involves a red-green-refactor cycle: Red: Write a test that fails because the feature it tests isn't implemented yet. Green: Write the minimum amount of code necessary to make the test pass. Refactor: Clean up the code while ensuring that tests continue to pass. This rigorous cycle guarantees that new code is continually tested and refactored, reducing the likelihood of complex bugs. It also means that when bugs do appear, they are often easier to isolate and fix due to the modular and well-tested nature of the codebase. At least, that's the theory. TDD can be especially advantageous for scripting and loosely typed languages. In environments lacking the rigid structure of compilers and linters, TDD steps in to provide the necessary checks that would otherwise be performed during compilation in statically typed languages. It becomes a crucial substitute for compiler/linter checks, ensuring that type and logic errors are caught early. In real-world application development, TDD's utility is nuanced. While it encourages thorough testing and upfront design, it can sometimes hinder the natural flow of development, especially in complex systems that evolve through numerous iterations. The requirement for 100% test coverage can lead to an unnecessary focus on fulfilling metrics rather than writing meaningful tests. The biggest problem in TDD is its focus on unit testing. TDD is impractical with integration tests as the process would take too long. But as we determined at the start of this post, integration tests are the true benchmark for quality. In that test TDD is a methodology that provides great quality for arbitrary tests, but not necessarily great quality for the final product. You might have the best cog in the world, but if doesn't fit well into the machine then it isn't great. Final Word Debugging is a tool that not only fixes bugs but also actively aids in crafting tests that bolster software quality. By utilizing debuggers in test composition and increasing coverage, developers can create a suite of tests that not only identifies existing issues but also guards against future ones, thus ensuring the delivery of reliable, high-quality software. Debugging lets us increase coverage and verify edge cases effectively. It's part of a standardized process for issue resolution that's critical for reliability and prevents regressions.
In the dynamic landscape of artificial intelligence (AI), two groundbreaking technologies — Large Language Models (LLM) and Retrieval-Augmented Generation (RAG) — stand out for their transformative potential in understanding and generating human-like text. This article embarks on a comparative journey between LLM and RAG, shedding light on their mechanisms, applications, and the unique advantages they offer to the AI field. Large Language Models (LLM): Foundations and Applications LLMs, such as GPT (Generative Pre-trained Transformer), have revolutionized the AI scene with their ability to generate coherent and contextually relevant text across a wide array of topics. At their core, LLMs rely on vast amounts of text data and sophisticated neural network architectures to learn language patterns, grammar, and knowledge from the textual content they have been trained on. The strength of LLMs lies in their generalization capabilities: they can perform a variety of language-related tasks without task-specific training. This includes translating languages, answering questions, and even writing articles. However, LLMs are not without their challenges. They sometimes generate plausible-sounding but incorrect or nonsensical answers, a phenomenon known as a "hallucination." Additionally, the quality of their output heavily depends on the quality and breadth of their training data. Core Aspects Scale: The hallmark of LLMs is their vast parameter count, reaching into the billions, which captures a wide linguistic range. Training regime: They undergo pre-training on diverse text data, subsequently fine-tuned for tailored tasks, embedding a deep understanding of language nuances. Utility spectrum: LLMs find their use across various fronts, from aiding in content creation to facilitating language translation. Example: Generating Text With an LLM To illustrate, consider the following Python code snippet that uses an LLM to generate a text sample: Python from transformers import GPT2Tokenizer, GPT2LMHeadModel # Input prompt = "How long have Australia held on to the Ashes?" # Encode the inputs with GPT2 Tokenizer tokenizer = GPT2Tokenizer.from_pretrained('gpt2') inputs = tokenizer.encode(prompt, return_tensors='pt') ## using pyTorch ('tf' to use TensorFlow) # Generate outputs with gpt2 Model model = GPT2LMHeadModel.from_pretrained('gpt2') outputs = model.generate(inputs, max_length=25) # Decode and print the result result = tokenizer.decode(outputs[0], skip_special_tokens=True) print("Generated text:", result) This code initializes a text generation pipeline using GPT-2, a popular LLM, and generates text based on a given prompt. Retrieval-Augmented Generation (RAG): An Overview and Use Cases RAG introduces a novel approach by combining the generative capabilities of models like GPT with a retrieval mechanism. This mechanism searches a database of text (such as Wikipedia) in real time to find relevant information that can be used to inform the model's responses. This blending of retrieval and generation allows RAG to produce answers that are not only contextually relevant but also grounded in factual information. One of the main advantages of RAG over traditional LLMs is its ability to provide more accurate and specific information by referencing up-to-date sources. This makes RAG particularly useful for applications where accuracy and timeliness of information are critical, such as in news reporting or academic research assistance. However, the reliance on external databases means that RAG's performance can suffer if the database is not comprehensive or if the retrieval process is inefficient. Furthermore, integrating retrieval mechanisms into the generative process adds complexity to the model, potentially increasing the computational resources required. Core Aspects Hybrid nature: RAG models first retrieve pertinent documents, and then utilize this context for informed generation. Dynamic knowledge access: Unlike LLMs, RAG models can tap into the latest or domain-specific data, offering enhanced versatility. Application areas: RAG shines in scenarios demanding external knowledge, such as in-depth question answering and factual content generation. Example: Implementing RAG for Information Retrieval Below is a simplified example of how one might implement a basic RAG system for retrieving and generating text: Python from transformers import RagTokenizer, RagRetriever, RagSequenceForGeneration # A sample query to ask the model query = "How long have Australia held on to the Ashes?" tokenizer = RagTokenizer.from_pretrained("facebook/rag-sequence-nq") ## Get the tokenizer from the pretrained model tokenized_text = tokenizer(query, return_tensors='pt', max_length=100, truncation=True) ## Encode/Tokenize the query # Find results with RAG-Sequence model (uncased model) using wiki_dpr dataset retriever = RagRetriever.from_pretrained("facebook/rag-sequence-nq", index_name="exact", use_dummy_dataset=True) ## Uses a pretrained DPR dataset (wiki_dpr) https://huggingface.co/datasets/wiki_dpr model = RagSequenceForGeneration.from_pretrained("facebook/rag-sequence-nq", retriever=retriever) model_generated_tokens = model.generate(input_ids=tokenized_text["input_ids"], max_new_tokens=1000) ## Find the relavant information from the dataset (tokens) print(tokenizer.batch_decode(model_generated_tokens, skip_special_tokens=True)[0]) ## Decode the data to find the answer This code utilizes Facebook's RAG model to answer a query by first tokenizing the input and then generating a response based on information retrieved in real time. Comparative Insights: LLM vs RAG The choice between LLM and RAG hinges on specific task requirements. Here’s how they stack up: Knowledge Accessibility LLMs rely on their pre-training corpus, possibly leading to outdated information. RAG, with its retrieval capability, ensures access to the most current data. Implementation Complexity RAG models, owing to their dual-step nature, present a higher complexity and necessitate more resources than LLMs. Flexibility and Application Both model types offer broad application potential. LLMs serve as a robust foundation for varied NLP tasks, while RAG models excel where instant access to external, detailed data is paramount. Conclusion: Navigating the LLM and RAG Landscape Both LLM and RAG represent significant strides in AI's capability to understand and generate human-like text. Selecting between LLM and RAG models involves weighing the unique demands of your NLP project. LLMs offer versatility and generalization, making them suitable for a wide range of applications and a go-to for diverse language tasks. In contrast, RAG's strength lies in its ability to provide accurate, information-rich responses, particularly valuable in knowledge-intensive tasks and ideal for situations where the incorporation of the latest or specific detailed information is crucial. As AI continues to evolve, the comparative analysis of LLM and RAG underscores the importance of selecting the right tool for the right task. Developers and researchers are encouraged to weigh these technologies' benefits and limitations in the context of their specific needs, aiming to leverage AI's full potential in creating intelligent, responsive, and context-aware applications.
APIs and SDKs are how you provide a bridge to developers for your underlying platform, allowing them to compile and use applications and integrate your platform with their personal projects. Building APIs and SDKs that developers love to use can make or break your platform and is no mere feat. In the following article, I will provide some of the most effective practices I have seen in the industry. I place these four necessary strategies that should be at the heart of any API/SDK program: simplicity, resilience, community building, and continuous improvement. Prioritize Simplicity Simplicity is the most essential factor to consider while designing APIs and SDKs. Firms are more likely to adopt and stay with you if APIs and SDKs usage is intuitive, well-documented, and easy to plug into other projects. Do not over-engineer or overcomplicate APIs/SDKs. Preferring clarity, consistency, and compliance with industry standards, draft intuitive and user-friendly APIs and SDKs. Create endpoints with concise, descriptive naming best practices that accurately convey their purpose. Enforce a comprehensive guideline to your whole API or SDK, with naming opportunities and design scenarios followed consistently. Align with widely adopted standards and paradigms, such as RESTful principles, appropriate HTTP methods, language-specific conventions, and secure authentication mechanisms, to provide a seamless and familiar experience for developers. Here are a few good examples to consider: API Design Guidelines APIs Design API Standards Style Guide Blindly following a style guide without considering the unique requirements and goals of your platform can lead to suboptimal outcomes. It is important to strike a balance between catering to developers' needs and doing what's right for the long-term success and viability of your platform. While it might be tempting to fulfill every feature request from your users, you must make hard choices to prioritize the health and maintainability of your platform (the adage applies: "Put on your own oxygen mask first before assisting others"). Nothing erodes trust like a platform that lacks stability, and security or cannot scale, so work hard to find the right balance between a good developer-friendly experience while ensuring the aforementioned criteria are not in peril. Designing for Resilience While designing APIs and SDKs, it is essential to place error handling at its core. To provide a dependable developer experience, a platform needs to have a comprehensive and well-documented error code system that covers a significant range of possible failure scenarios with dozens of unique error codes designed to cover various categories of errors like authentication failures, validation errors, resource not found, rate limiting, and other server-side errors. Furthermore, error messages should not only inform the developer about the nature of an error but offer guidance on how to resolve it. Offer retry mechanisms to developers when dealing with partial failures. Provide them with the means to configure the retry behavior, such as the maximum number of retries and initial retry delay. Additionally, set timeout values to prevent requests to services from hanging or being blocked indefinitely. Allows developers to customize the timeout setting and provides them with a way to gracefully cancel a long-running request. Follow an all-or-nothing approach when it comes to transactional operations. Keep data integrity and consistency in the forefront whenever a batch operation is invoked, either all operations in the batch should succeed or none of them should. The developer should be notified about which items in the batch were successful, and which items were erroneous. Ensure that your APIs and SDKs include robust logging capabilities that can help developers troubleshoot and debug issues. Log relevant information such as request/response details, error messages, and stack traces. Allow developers to configure logging verbosity and opt in/out of logging entirely in production. Define a consistent and clear versioning policy for your APIs and SDKs. Follow semantic versioning. Fostering a Developer Community Building a strong developer community around your APIs and SDKs is critical to drive adoption, educate developers, and promote innovation. Provide comprehensive documentation for your APIs and SDKs that thoroughly covers all they have to offer. Include getting started guides, tutorials, code samples, reference documentation, and more. Build an interactive developer portal that serves as the central hub for all developer-related content. Include features such as API consoles, sandbox environments, and interactive documentation that allow developers to experiment and try out their integrations in a controlled setting. Engage with developers through popular developer platforms, social media, webinars, and in-person workshops. Participate in discussions, answer questions, and provide support for developers who are using your APIs and SDKs. Create an environment where developers can easily provide feedback and help test and improve your offerings. Set up bug trackers, feature requests, and general feedback submission processes. Foster community-driven support by encouraging developers to help each other in forums, establish a community-driven knowledge base, and provide moderation to ensure a positive and inclusive community. Make sure your support team is responsive and knowledgeable, reply to developer questions promptly, and provide value-added responses. Keep a detailed internal knowledge base or a dedicated FAQ section containing solutions to common questions and challenges. This ensures your support and field teams can quickly understand and resolve customer issues, delivering a seamless experience to the developers using your APIs and SDKs. Organize developer events and conferences to gather developers and encourage one-on-one communication. Invite veterans and industry experts to educate and enlighten, and enable developers to present their own projects to learn from one another. Gather feedback, announce features or changes, and bond with your developer community. Growing a thriving developer community ensures you have a supportive environment that cultivates collaboration, education, and innovation, driving your APIs and SDKs to become more popular and successful. Iterate and Improve Develop a structured approach for assessing and ordering the feedback by its effect, urgency, and relationship with your company’s objectives. Regularly consult your development team and stakeholders to review the feedback and determine what changes and features should be implemented into your roadmap. Devote resources and set deadlines to implement the modifications. Ensure your development cycle includes complete testing and quality assurance procedures to uphold the integrity and dependability of your APIs and SDKs. Update your documentation and announce the changes to your developer community. Establish key performance indicators – API adoption rates, developer satisfaction, support ticket response time, for example – to evaluate your changes’ performance. Regularly monitor and assess this data to evaluate the effect of your changes and identify potential improvements. Lastly, build a culture of continuous learning and improvement within your organization. Ensure that your team keeps up with the latest trends in the industry, attends conferences and workshops, and participates in developer communities. Knowledge of the current trends equips you with relevant insights to stay ahead by addressing the current developers’ needs. More importantly, have processes that enable you to iterate and enhance the APIs and SDKs provided. Having a process that can effectively iterate shows developers that you are serious about delivering quality products and can quickly switch to another provider should their expectations be compromised. This way, you build trust and relationships that last, and your platform becomes a reliable and innovative tool that keeps attracting developers in the market. In conclusion, designing developer-friendly APIs and SDKs is a vital element of platform strategy. Prioritize simplicity, resilience, community, and continuous improvement. Remember, developers will only love your platform if they first enjoy using it. Hence, invest in making their experience better, meet their nowadays-changing needs, and enhance their satisfaction. Such actions enable you to get the best out of your platform, introduce more innovations, and thrive in the dynamic technology landscape.
I would like to to introduce you a Java class with less than 170 lines of code to facilitate work with SQL queries called via the JDBC API. What makes this solution interesting? The class can be embedded in a Java version 17 script. Using a Java Script The advantage of a Java script is easy portability in text format and the possibility of running without prior compilation, while we have considerable resources available from the language's standard library at runtime. The use of scripts is offered for various prototypes, in which even more complicated data exports or data conversions can be solved (after connecting to the database). Scripts are useful wherever we don't want to (or can't) put the implementation into a standard Java project. However, the use of the script has some limitations. For example, the code must be written in a single file. We can include all the necessary libraries when we run the script, but these will likely have additional dependencies, and simply listing them on the command line can be frustrating. The complications associated with the distribution of such a script probably do not need to be emphasized. For the above reasons, I believe that external libraries in scripts are best avoided. If we still want to go the script route, the choice falls on pure JDBC. Multi-line text literals can be advantageously used for writing SQL queries, and the automatic closing of objects like PreparedStatement (implementing the interface AutoCloseable). So what's the problem? Mapping SQL Parameter Values For security reasons, it is advisable to map SQL parameter values to question marks. I consider the main handicap of JDBC to be the mapping of parameters using the sequence number of the question mark (starting with one). The first version of the parameter mapping to the SQL script often turns out well, but the risk of error increases as the number of parameters and additional SQL modifications increase. I remind you that by inserting a new parameter in the first position, the following row must be renumbered. Another complication is the use of the operator IN because for each value of the enumeration, a question mark must be written in the SQL template which must be mapped to a separate parameter. If the parameter list is dynamic, the list of question marks in the SQL template must also be dynamic. Debugging a larger number of more complex SQLs can start to take a significant amount of time. For inserting SQL parameters using String Templates we will have to wait a little longer. However, inserting SQL parameters could be facilitated by a simple wrapper over the interfacePreparedStatement, which would (before calling the SQL statement) append the parameters using JPA-style named tags (alphanumeric text starting with a colon). A wrapper could also simplify reading data from the database (with a SELECT statement) if it allowed the necessary methods to be chained into a single statement, preferably with a return type Stream<ResultSet>. SqlParamBuilder Class Visualization of the SQL command with attached parameters would sometimes be useful for debugging or logging the SQL query. I present to you the class SqlParamBuilder. The priority of the implementation was to cover the stated requirements with a single Java class with minimalistic code. The programming interface was inspired by the library JDBI. The samples use the H2 database in in-memory mode. However, connecting the database driver will be necessary. Java void mainStart(Connection dbConnection) throws Exception { try (var builder = new SqlParamBuilder(dbConnection)) { System.out.println("# CREATE TABLE"); builder.sql(""" CREATE TABLE employee ( id INTEGER PRIMARY KEY , name VARCHAR(256) DEFAULT 'test' , code VARCHAR(1) , created DATE NOT NULL ) """) .execute(); System.out.println("# SINGLE INSERT"); builder.sql(""" INSERT INTO employee ( id, code, created ) VALUES ( :id, :code, :created ) """) .bind("id", 1) .bind("code", "T") .bind("created", someDate) .execute(); System.out.println("# MULTI INSERT"); builder.sql(""" INSERT INTO employee (id,code,created) VALUES (:id1,:code,:created), (:id2,:code,:created) """) .bind("id1", 2) .bind("id2", 3) .bind("code", "T") .bind("created", someDate.plusDays(7)) .execute(); builder.bind("id1", 11) .bind("id2", 12) .bind("code", "V") .execute(); System.out.println("# SELECT"); List<Employee> employees = builder.sql(""" SELECT t.id, t.name, t.created FROM employee t WHERE t.id < :id AND t.code IN (:code) ORDER BY t.id """) .bind("id", 10) .bind("code", "T", "V") .streamMap(rs -> new Employee( rs.getInt("id"), rs.getString("name"), rs.getObject("created", LocalDate.class))) .toList(); System.out.printf("# PRINT RESULT OF: %s%n", builder.toStringLine()); employees.stream() .forEach((Employee employee) -> System.out.println(employee)); assertEquals(3, employees.size()); assertEquals(1, employees.get(0).id); assertEquals("test", employees.get(0).name); assertEquals(someDate, employees.get(0).created); } } record Employee (int id, String name, LocalDate created) {} static class SqlParamBuilder {…} Usage Notes and Final Thoughts An instance of the type SqlParamBuilder can be recycled for multiple SQL statements. After calling the command, the parameters can be changed and the command can be run again. The parameters are assigned to the last used object PreparedStatement. Method sql() automatically closes the internal object PrepradedStatement (if there was one open before). If we change the group of parameters (typically for the IN operator), we need to send the same number for the same PreparedStatement. Otherwise, the method againsql() will need to be used. An object is required after the last command execution to explicitly close the SqlParamBuilder. However, since we are implementing an interface AutoCloseable, just enclose the entire block in a try block. Closing does not affect the contained database connection. In the Bash shell, the sample can be run with a script SqlExecutor.sh, which can download the necessary JDBC driver (here, for the H2 database). If we prefer Kotlin, we can try a Bash script SqlExecutorKt.sh, which migrates the prepared Kotlin code to a script and runs it. Let's not get confused by the fact that the class is stored in a Maven-type project. One reason is the ease of running JUnit tests. The class is licensed under the Apache License, Version 2.0. Probably the fastest way to create your own implementation is to download the example script, redesign the method mainRun(), and modify the connection parameters to your own database. Use your own JDBC driver to run.
1. Use "&&" to Link Two or More Commands Use “&&” to link two or more commands when you want the previous command to be succeeded before the next command. If you use “;” then it would still run the next command after “;” even if the command before “;” failed. So you would have to wait and run each command one by one. However, using "&&" ensures that the next command will only run if the preceding command finishes successfully. This allows you to add commands without waiting, move on to the next task, and check later. If the last command ran, it indicates that all previous commands ran successfully. Example: Shell ls /path/to/file.txt && cp /path/to/file.txt /backup/ The above example ensures that the previous command runs successfully and that the file "file.txt" exists. If the file doesn't exist, the second command after "&&" won't run and won't attempt to copy it. 2. Use “grep” With -A and -B Options One common use of the "grep" command is to identify specific errors from log files. However, using it with the -A and -B options provides additional context within a single command, and it displays lines after and before the searched text, which enhances visibility into related content. Example: Shell % grep -A 2 "java.io.IOException" logfile.txt java.io.IOException: Permission denied (open /path/to/file.txt) at java.io.FileOutputStream.<init>(FileOutputStream.java:53) at com.pkg.TestClass.writeFile(TestClass.java:258) Using grep with -A here will also show 2 lines after the “java.io.IOException” was found from the logfile.txt. Similarly, Shell grep "Ramesh" -B 3 rank-file.txt Name: John Wright, Rank: 23 Name: David Ross, Rank: 45 Name: Peter Taylor, Rank: 68 Name Ramesh Kumar, Rank: 36 Here, grep with -B option will also show 3 lines before the “Ramesh” was found from the rank-file.txt 3. Use “>” to Create an Empty File Just write > and then the filename to create an empty file with the name provided after > Example: Shell >my-file.txt It will create an empty file with "my-file.txt" name in the current directory. 4. Use “rsync” for Backups "rsync" is a useful command for regular backups as it saves time by transferring only the differences between the source and destination. This feature is especially beneficial when creating backups over a network. Example: Shell rsync -avz /path/to/source_directory/ user@remotehost:/path/to/destination_directory/ 5. Use Tab Completion Using tab completion as a habit is faster than manually selecting filenames and pressing Enter. Typing the initial letters of filenames and utilizing Tab completion streamlines the process and is more efficient. 6. Use “man” Pages Instead of reaching the web to find the usage of a command, a quick way would be to use the “man” command to find out the manual of that command. This approach not only saves time but also ensures accuracy, as command options can vary based on the installed version. By accessing the manual directly, you get precise details tailored to your existing version. Example: Shell man ps It will get the manual page for the “ps” command 7. Create Scripts For repetitive tasks, create small shell scripts that chain commands and perform actions based on conditions. This saves time and reduces risks in complex operations. Conclusion In conclusion, becoming familiar with these Linux commands and tips can significantly boost productivity and streamline workflow on the command line. By using techniques like command chaining, context-aware searching, efficient file management, and automation through scripts, users can save time, reduce errors, and optimize their Linux experience.
Debugging Terraform providers is crucial for ensuring the reliability and functionality of infrastructure deployments. Terraform providers, written in languages like Go, can have complex logic that requires careful debugging when issues arise. One powerful tool for debugging Terraform providers is Delve, a debugger for the Go programming language. Delve allows developers to set breakpoints, inspect variables, and step through code, making it easier to identify and resolve bugs. In this blog, we will explore how to use Delve effectively for debugging Terraform providers. Setup Delve for Debugging Terraform Provider Shell # For Linux sudo apt-get install -y delve # For macOS brew instal delve Refer here for more details on the installation. Debug Terraform Provider Using VS Code Follow the below steps to debug the provider Download the provider code. We will use IBM Cloud Terraform Provider for this debugging example. Update the provider’s main.go code to the below to support debugging Go package main import ( "flag" "log" "github.com/IBM-Cloud/terraform-provider-ibm/ibm/provider" "github.com/IBM-Cloud/terraform-provider-ibm/version" "github.com/hashicorp/terraform-plugin-sdk/v2/plugin" ) func main() { var debug bool flag.BoolVar(&debug, "debug", true, "Set to true to enable debugging mode using delve") flag.Parse() opts := &plugin.ServeOpts{ Debug: debug, ProviderAddr: "registry.terraform.io/IBM-Cloud/ibm", ProviderFunc: provider.Provider, } log.Println("IBM Cloud Provider version", version.Version) plugin.Serve(opts) } Launch VS Code in debug mode. Refer here if you are new to debugging in VS Code. Create the launch.json using the below configuration. JSON { "version": "0.2.0", "configurations": [ { "name": "Debug Terraform Provider IBM with Delve", "type": "go", "request": "launch", "mode": "debug", "program": "${workspaceFolder}", "internalConsoleOptions": "openOnSessionStart", "args": [ "-debug" ] } ] } In VS Code click “Start Debugging”. Starting the debugging starts the provider for debugging. To attach the Terraform CLI to the debugger, console prints the environment variable TF_REATTACH_PROVIDERS. Copy this from the console. Set this as an environment variable in the terminal running the Terraform code. Now in the VS Code where the provider code is in debug mode, open the go code to set up break points. To know more on breakpoints in VS Code refer here. Execute 'terraform plan' followed by 'terraform apply', to notice the Terraform provider breakpoint to be triggered as part of the terraform apply execution. This helps to debug the Terraform execution and comprehend the behavior of the provider code for the particular inputs supplied in Terraform. Debug Terraform Provider Using DLV Command Line Follow the below steps to debug the provider using the command line. To know more about the dlv command line commands refer here. Follow the 1& 2 steps mentioned in Debug Terraform provider using VS Code In the terminal navigate to the provider go code and issue go build -gcflags="all=-N -l" to compile the code To execute the precompiled Terraform provider binary and begin a debug session, run dlv exec --accept-multiclient --continue --headless <path to the binary> -- -debug where the build file is present. For IBM Cloud Terraform provider use dlv exec --accept-multiclient --continue --headless ./terraform-provider-ibm -- -debug In another terminal where the Terraform code would be run, set the TF_REATTACH_PROVIDERS as an environment variable. Notice the “API server” details in the above command output. In another (third) terminal connect to the DLV server and start issuing the DLV client commands Set the breakpoint using the break command Now we are set to debug the Terraform provider when Terraform scripts are executed. Issue continue in the DLV client terminal to continue until the breakpoints are set. Now execute the terraform plan and terraform apply to notice the client waiting on the breakpoint. Use DLV CLI commands to stepin / stepout / continue the execution. This provides a way to debug the terraform provider from the command line. Remote Debugging and CI/CD Pipeline Debugging Following are the extensions to the debugging using the dlv command line tool. Remote Debugging Remote debugging allows you to debug a Terraform provider running on a remote machine or environment. Debugging in CI/CD Pipelines Debugging in CI/CD pipelines involves setting up your pipeline to run Delve and attach to your Terraform provider for debugging. This can be challenging due to the ephemeral nature of CI/CD environments. One approach is to use conditional logic in your pipeline configuration to only enable debugging when a specific environment variable is set. For example, you can use the following script in your pipeline configuration to start Delve and attach to your Terraform provider – YAML - name: Debug Terraform Provider if: env(DEBUG) == 'true' run: | dlv debug --headless --listen=:2345 --api-version=2 & sleep 5 # Wait for Delve to start export TF_LOG=TRACE terraform init terraform apply Best Practices for Effective Debugging With Delve Here are some best practices for effective debugging with Delve, along with tips for improving efficiency and minimizing downtime: Use version control: Always work with version-controlled code. This allows you to easily revert changes if debugging introduces new issues. Start small: Begin debugging with a minimal, reproducible test case. This helps isolate the problem and reduces the complexity of debugging. Understand the code: Familiarize yourself with the codebase before debugging. Knowing the code structure and expected behavior can speed up the debugging process. Use logging: Add logging statements to your code to track the flow of execution and the values of important variables. This can provide valuable insights during debugging. Use breakpoints wisely: Set breakpoints strategically at critical points in your code. Too many breakpoints can slow down the debugging process. Inspect variables: Use the print (p) command in Delve to inspect the values of variables. This can help you understand the state of your program at different points in time. Use conditional breakpoints: Use conditional breakpoints to break execution only when certain conditions are met. This can help you focus on specific scenarios or issues. Use stack traces: Use the stack command in Delve to view the call stack. This can help you understand the sequence of function calls leading to an issue. Use goroutine debugging: If your code uses goroutines, use Delve's goroutine debugging features to track down issues related to concurrency. Automate debugging: If you're debugging in a CI/CD pipeline, automate the process as much as possible to minimize downtime and speed up resolution. By following these best practices, you can improve the efficiency of your debugging process and minimize downtime caused by issues in your code. Conclusion In conclusion, mastering the art of debugging Terraform providers with Delve is a valuable skill that can significantly improve the reliability and performance of your infrastructure deployments. By setting up Delve for debugging, exploring advanced techniques like remote debugging and CI/CD pipeline debugging, and following best practices for effective debugging, you can effectively troubleshoot issues in your Terraform provider code. Debugging is not just about fixing bugs; it's also about understanding your code better and improving its overall quality. Dive deep into Terraform provider debugging with Delve, and empower yourself to build a more robust and efficient infrastructure with Terraform.
The amount of data generated by modern systems has become a double-edged sword for security teams. While it offers valuable insights, sifting through mountains of logs and alerts manually to identify malicious activity is no longer feasible. Here's where rule-based incident detection steps in, offering a way to automate the process by leveraging predefined rules to flag suspicious activity. However, the choice of tool for processing high-volume data for real-time insights is crucial. This article delves into the strengths and weaknesses of two popular options: Splunk, a leading batch search tool, and Flink, a powerful stream processing framework, specifically in the context of rule-based security incident detection. Splunk: Powerhouse Search and Reporting Splunk has become a go-to platform for making application and infrastructure logs readily available for ad-hoc search. Its core strength lies in its ability to ingest log data from various sources, centralize it, and enable users to explore it through powerful search queries. This empowers security teams to build comprehensive dashboards and reports, providing a holistic view of their security posture. Additionally, Splunk supports scheduled searches, allowing users to automate repetitive queries and receive regular updates on specific security metrics. This can be particularly valuable for configuring rule-based detections, monitoring key security indicators, and identifying trends over time. Flink: The Stream Processing Champion Apache Flink, on the other hand, takes a fundamentally different approach. It is a distributed processing engine designed to handle stateful computations over unbounded and bounded data streams. Unlike Splunk's batch processing, Flink excels at real-time processing, enabling it to analyze data as it arrives, offering near-instantaneous insights. This makes it ideal for scenarios where immediate detection and response are paramount, such as identifying ongoing security threats or preventing fraudulent transactions in real time. Flink's ability to scale horizontally across clusters makes it suitable for handling massive data volumes, a critical factor for organizations wrestling with ever-growing security data. Case Study: Detecting User Login Attacks Let's consider a practical example: a rule designed to detect potential brute-force login attempts. This rule aims to identify users who experience a high number of failed login attempts within a specific timeframe (e.g., an hour). Here's how the rule implementation would differ in Splunk and Flink: Splunk Implementation sourcetype=login_logs (result="failure" OR "failed") | stats count by user within 1h | search count > 5 | alert "Potential Brute Force Login Attempt for user: $user$" This Splunk search query filters login logs for failed attempts, calculates the count of failed attempts per user within an hour window, and then triggers an alert if the count exceeds a predefined threshold (5). While efficient for basic detection, it relies on batch processing, potentially introducing latency in identifying ongoing attacks. Flink Implementation SQL SELECT user, COUNT(*) AS failed_attempts FROM login_logs WHERE result = 'failure' OR result = 'failed' GROUP BY user, TUMBLE(event_time, INTERVAL '1 HOUR') HAVING failed_attempts > 5; Flink takes a more real-time approach. As each login event arrives, Flink checks the user and result. If it's a failed attempt, a counter for that user's window (1 hour) is incremented. If the count surpasses the threshold (5) within the window, Flink triggers an alert. This provides near-instantaneous detection of suspicious login activity. A Deep Dive: Splunk vs. Flink for Detecting User Login Attacks The underlying processing models of Splunk and Flink lead to fundamental differences in how they handle security incident detection. Here's a closer look at the key areas: Batch vs. Stream Processing Splunk Splunk operates on historical data. Security analysts write search queries that retrieve and analyze relevant logs. These queries can be configured to run periodically automatically. This is a batch processing approach, meaning Splunk needs to search through potentially a large volume of data to identify anomalies or trends. For the login attempt example, Splunk would need to query all login logs within the past hour every time the search is run to calculate the failed login count per user. This can introduce significant latency in detecting, and increase the cost of compute, especially when dealing with large datasets. Flink Flink analyzes data streams in real-time. As each login event arrives, Flink processes it immediately. This stream-processing approach allows Flink to maintain a continuous state and update it with each incoming event. In the login attempt scenario, Flink keeps track of failed login attempts per user within a rolling one-hour window. With each new login event, Flink checks the user and result. If it's a failed attempt, the counter for that user's window is incremented. This eliminates the need to query a large amount of historical data every time a check is needed. Windowing Splunk Splunk performs windowing calculations after retrieving all relevant logs. In our example, the search stats count by user within 1h retrieves all login attempts within the past hour and then calculates the count for each user. This approach can be inefficient for real-time analysis, especially as data volume increases. Flink Flink maintains a rolling window and continuously updates the state based on incoming events. Flink uses a concept called "time windows" to partition the data stream into specific time intervals (e.g., one hour). For each window, Flink keeps track of relevant information, such as the number of failed login attempts per user. As new data arrives, Flink updates the state for the current window. This eliminates the need for a separate post-processing step to calculate windowed aggregations. Alerting Infrastructure Splunk Splunk relies on pre-configured alerting actions within the platform. Splunk allows users to define search queries that trigger alerts when specific conditions are met. These alerts can be delivered through various channels such as email, SMS, or integrations with other security tools. Flink Flink might require integration with external tools for alerts. While Flink can identify anomalies in real time, it may not have built-in alerting functionalities like Splunk. Security teams often integrate Flink with external Security Information and Event Management (SIEM) solutions for alert generation and management. In essence, Splunk operates like a detective sifting through historical evidence, while Flink functions as a security guard constantly monitoring activity. Splunk is a valuable tool for forensic analysis and identifying historical trends. However, for real-time threat detection and faster response times, Flink's stream processing capabilities offer a significant advantage. Choosing the Right Tool: A Balancing Act While Splunk provides a user-friendly interface and simplifies rule creation, its batch processing introduces latency, which can be detrimental to real-time security needs. Flink excels in real-time processing and scalability, but it requires more technical expertise to set up and manage. Beyond Latency and Ease of Use: Additional Considerations The decision between Splunk and Flink goes beyond just real-time processing and ease of use. Here are some additional factors to consider: Data Volume and Variety Security teams are often overwhelmed by the sheer volume and variety of data they need to analyze. Splunk excels at handling structured data like logs but struggles with real-time ingestion and analysis of unstructured data like network traffic or social media feeds. Flink, with its distributed architecture, can handle diverse data types at scale. Alerting and Response Both Splunk and Flink can trigger alerts based on rule violations. However, Splunk integrates seamlessly with existing Security Information and Event Management (SIEM) systems, streamlining the incident response workflow. Flink might require additional development effort to integrate with external alerting and response tools. Cost Splunk's licensing costs are based on data ingestion volume, which can become expensive for organizations with massive security data sets. Flink, being open-source, eliminates licensing fees. However, the cost of technical expertise for setup, maintenance, and rule development for Flink needs to be factored in. The Evolving Security Landscape: A Hybrid Approach The security landscape is constantly evolving, demanding a multifaceted approach. Many organizations find value in a hybrid approach, leveraging the strengths of both Splunk and Flink. Splunk as the security hub: Splunk can serve as a central repository for security data, integrating logs from various sources, including real-time data feeds from Flink. Security analysts can utilize Splunk's powerful search capabilities for historical analysis, threat hunting, and investigation. Flink for real-time detection and response: Flink can be deployed for real-time processing of critical security data streams, focusing on identifying and responding to ongoing threats. This combination allows security teams to enjoy the benefits of both worlds: Comprehensive security visibility: Splunk provides a holistic view of historical and current security data. Real-time threat detection and response: Flink enables near-instantaneous identification and mitigation of ongoing security incidents. Conclusion: Choosing the Right Tool for the Job Neither Splunk nor Flink is a one-size-fits-all solution for rule-based incident detection. The optimal choice depends on your specific security needs, data volume, technical expertise, and budget. Security teams should carefully assess these factors and potentially consider a hybrid approach to leverage the strengths of both Splunk and Flink for a robust and comprehensive security posture. By understanding the strengths and weaknesses of each tool, security teams can make informed decisions about how to best utilize them to detect and respond to security threats in a timely and effective manner.
Java adoption has shifted from version 1.8 to at least Java 17. Concurrently, Spring Boot has advanced from version 2.x to 3.2.2. The springdoc project has transitioned from the older library 'springdoc-openapi-ui' to 'springdoc-openapi-starter-webmvc-ui' for its functionality. These updates mean that readers relying on older articles may find themselves years behind in these technologies. The author has updated this article so that readers are using the latest versions and don't struggle with outdated information during migration. This is part one of a three-part series. You can check out the other articles below. OpenAPI 3 Documentation With Spring Boot Doing More With Springdoc OpenAPI Extending Swagger and Springdoc Open API In this tutorial, we are going to try out a Spring Boot Open API 3-enabled REST project and explore some of its capabilities. The springdoc-openapi Java library has quickly become very compelling. We are going to refer to Building a RESTful Web Service and springdoc-openapi v2.5.0. Prerequisites Java 17.x Maven 3.x Steps Start by creating a Maven JAR project. Below, you will see the pom.xml to use: XML <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>3.2.2</version> <relativePath ></relativePath> <!-- lookup parent from repository --> </parent> <groupId>com.example</groupId> <artifactId>sample</artifactId> <version>0.0.1</version> <name>sample</name> <description>Demo project for Spring Boot with openapi 3 documentation</description> <properties> <java.version>17</java.version> </properties> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-validation</artifactId> </dependency> <dependency> <groupId>org.springdoc</groupId> <artifactId>springdoc-openapi-starter-webmvc-ui</artifactId> <version>2.5.0</version> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> </plugin> </plugins> </build> </project> Note the "springdoc-openapi-starter-webmvc-ui" dependency. Now, let's create a small Java bean class. Java package sample; import org.hibernate.validator.constraints.CreditCardNumber; import jakarta.validation.constraints.Email; import jakarta.validation.constraints.Max; import jakarta.validation.constraints.Min; import jakarta.validation.constraints.NotBlank; import jakarta.validation.constraints.NotNull; import jakarta.validation.constraints.Pattern; import jakarta.validation.constraints.Size; public class Person { private long id; private String firstName; @NotNull @NotBlank private String lastName; @Pattern(regexp = ".+@.+\\..+", message = "Please provide a valid email address" ) private String email; @Email() private String email1; @Min(18) @Max(30) private int age; @CreditCardNumber private String creditCardNumber; public String getCreditCardNumber() { return creditCardNumber; } public void setCreditCardNumber(String creditCardNumber) { this.creditCardNumber = creditCardNumber; } public long getId() { return id; } public void setId(long id) { this.id = id; } public String getEmail1() { return email1; } public void setEmail1(String email1) { this.email1 = email1; } @Size(min = 2) public String getFirstName() { return firstName; } public void setFirstName(String firstName) { this.firstName = firstName; } public String getLastName() { return lastName; } public void setLastName(String lastName) { this.lastName = lastName; } public String getEmail() { return email; } public void setEmail(String email) { this.email = email; } public int getAge() { return age; } public void setAge(int age) { this.age = age; } } - This is an example of a Java bean. Now, let's create a controller. Java package sample; import org.springframework.web.bind.annotation.RequestBody; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMethod; import org.springframework.web.bind.annotation.RestController; import io.swagger.v3.oas.annotations.media.Content; import io.swagger.v3.oas.annotations.media.ExampleObject; import jakarta.validation.Valid; @RestController public class PersonController { @RequestMapping(path = "/person", method = RequestMethod.POST) @io.swagger.v3.oas.annotations.parameters.RequestBody(required = true, content = @Content(examples = { @ExampleObject(value = INVALID_REQUEST, name = "invalidRequest", description = "Invalid Request"), @ExampleObject(value = VALID_REQUEST, name = "validRequest", description = "Valid Request") })) public Person person(@Valid @RequestBody Person person) { return person; } private static final String VALID_REQUEST = """ { "id": 0, "firstName": "string", "lastName": "string", "email": "abc@abc.com", "email1": "abc@abc.com", "age": 20, "creditCardNumber": "4111111111111111" }"""; private static final String INVALID_REQUEST = """ { "id": 0, "firstName": "string", "lastName": "string", "email": "abcabc.com", "email1": "abcabc.com", "age": 17, "creditCardNumber": "411111111111111" }"""; } - Above is a sample REST Controller. Side Note: Normally I don't like to clutter already annotation-cluttered code with additional annotations, but I do think having ready-made examples like these can be useful. Another reason that forced me to do this was the default examples now generated from Swagger UI appear to be generating some confusing text when using @Pattern. It appears to be a Spring UI issue and not a Springdoc issue. Let's make some entries in src\main\resources\application.properties. Properties files application-description=@project.description@ application-version=@project.version@ logging.level.org.springframework.boot.autoconfigure=ERROR # server.error.include-binding-errors is now needed if we # want to display the errors as shown in this article # this can also be avoided in other ways as we will see # in later articles server.error.include-binding-errors=always The above entries will pass on Maven build-related information to the OpenAPI documentation and also include the new server.error.include-binding-errors property. Finally, let's write the Spring Boot application class: Java package sample; import org.springframework.beans.factory.annotation.Value; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.context.annotation.Bean; import io.swagger.v3.oas.models.OpenAPI; import io.swagger.v3.oas.models.info.Info; import io.swagger.v3.oas.models.info.License; @SpringBootApplication public class SampleApplication { public static void main(String[] args) { SpringApplication.run(SampleApplication.class, args); } @Bean public OpenAPI customOpenAPI(@Value("${application-description}") String appDesciption, @Value("${application-version}") String appVersion) { return new OpenAPI() .info(new Info() .title("sample application API") .version(appVersion) .description(appDesciption) .termsOfService("http://swagger.io/terms/") .license(new License().name("Apache 2.0").url("http://springdoc.org"))); } } - Also, note how the API version and description are being leveraged from application.properties. At this stage, this is what the project looks like in Eclipse: The project contents are above. Next, execute the mvn clean package from the command prompt or terminal. Then, execute java -jar target\sample-0.0.1.jar. You can also launch the application by running the SampleApplication.java class from your IDE. Now, let's visit the Swagger UI — http://localhost:8080/swagger-ui.html. Click the green Post button and expand the > symbol on the right of Person under Schemas. Let's expand the last schemas section a bit more: The nice thing is how the contract is automatically detailed leveraging JSR-303 annotations on the model. It out-of-the-box covers many of the important annotations and documents them. However, I did not see it support out of the box @javax.validation.constraints.Email and @org.hibernate.validator.constraints.CreditCardNumber at this point. The issue is that they are not documented in the generated Swagger specs, but those constraints are functional. We will discuss more on this in the subsequent article. For completeness, let's post a request. Press the Try it out button. Press the blue Execute button. Let's feed in a valid input by copying the below or by selecting the valid Input drop-down. JSON { "id": 0, "firstName": "string", "lastName": "string", "email": "abc@abc.com", "email1": "abc@abc.com", "age": 20, "creditCardNumber": "4111111111111111" } Let's feed that valid input into the Request body section. (We can also select "validRequest" from the Examples dropdown as shown below.) Upon pressing the blue Execute button, we see the below: This was only a brief introduction to the capabilities of the dependency: XML <dependency> <groupId>org.springdoc</groupId> <artifactId>springdoc-openapi-starter-webmvc-ui</artifactId> <version>2.5.0</version> </dependency> Troubleshooting Tips Ensure prerequisites. If using the Eclipse IDE, we might need to do a Maven update on the project after creating all the files. In the Swagger UI, if you are unable to access the “Schema” definitions link, it might be because you need to come out of the “try it out “ mode. Click on one or two Cancel buttons that might be visible. Source code Git Clone URL, Branch: springdoc-openapi-intro-update1.
As we delve into the dynamic world of Kubernetes, understanding its core components and functionalities becomes pivotal for anyone looking to make a mark in the cloud computing and containerization arena. Among these components, static pods hold a unique place, often overshadowed by more commonly discussed resources like deployments and services. In this comprehensive guide, we will unveil the power of static pods, elucidating their utility, operational principles, and how they can be an asset in your Kubernetes arsenal. Understanding Static Pods Static pods are Kubernetes pods that are managed directly by the kubelet daemon on a specific node, without the API server observing them. Unlike other pods that are controlled by the Kubernetes API server, static pods are defined by placing their configuration files directly on a node's filesystem, which the kubelet periodically scans and ensures that the pods defined in these configurations are running. Why Use Static Pods? Static pods serve several critical functions in a Kubernetes environment: Cluster Bootstrapping They are essential for bootstrapping a Kubernetes cluster before the API server is up and running. Since they do not depend on the API server, they can be used to deploy the control plane components as static pods. Node-Level System Pods Static pods are ideal for running node-level system components, ensuring that these essential services remain running, even if the Kubernetes API server is unreachable. Simplicity and Reliability For simpler deployments or edge environments where high availability is not a primary concern, static pods offer a straightforward and reliable deployment option. Creating Your First Static Pod Let’s walk through the process of creating a static pod. You'll need access to a Kubernetes node to follow along. 1. Access Your Kubernetes Node First, SSH into your Kubernetes node: ssh your_username@your_kubernetes_node 2. Create a Pod Definition File Create a simple pod definition file. Let’s deploy an Nginx static pod as an example. Save the following configuration in /etc/kubernetes/manifests/nginx-static-pod.yaml: apiVersion: v1 kind: Pod metadata: name: nginx-static-pod labels: role: myrole spec: containers: - name: nginx image: nginx ports: - containerPort: 80 3. Configure the kubelet to Use This Directory Ensure the kubelet is configured to monitor the /etc/kubernetes/manifests directory for pod manifests. This is typically set by the --pod-manifest-path kubelet command-line option. 4. Verify the Pod Is Running After a few moments, use the docker ps command (or crictl ps if you're using CRI-O or containerd) to check that the Nginx container is running: docker ps | grep nginx Or, if your cluster allows it, you can check from the Kubernetes API server with: kubectl get pods --all-namespaces | grep nginx-static-pod Note that while you can see the static pod through the API server, you cannot manage it (delete, scale, etc.) through the API server. Advantages of Static Pods Simplicity: Static pods are straightforward to set up and manage on a node-by-node basis. Self-sufficiency: They can operate independently of the Kubernetes API server, making them resilient in scenarios where the API server is unavailable. Control plane bootstrapping: Static pods are instrumental in the initial setup of a Kubernetes cluster, particularly for deploying control plane components. Considerations and Best Practices While static pods offer simplicity and independence from the Kubernetes API server, they also come with considerations that should not be overlooked: Cluster management: Static pods are not managed by the API server, which means they do not benefit from some of the orchestration features like scaling, lifecycle management, and health checks. Deployment strategy: They are best used for node-specific tasks or cluster bootstrapping, rather than general application deployment. Monitoring and logging: Ensure that your node-level monitoring and logging tools are configured to include static pods. Conclusion Static pods, despite their simplicity, play a critical role in the Kubernetes ecosystem. They offer a reliable method for running system-level services directly on nodes, independent of the cluster's control plane. By understanding how to deploy and manage static pods, you can ensure your Kubernetes clusters are more robust and resilient. Whether you're bootstrapping a new cluster or managing node-specific services, static pods are a tool worth mastering. This beginner's guide aims to demystify static pods and highlight their importance within Kubernetes architectures. As you advance in your Kubernetes journey, remember that the power of Kubernetes lies in its flexibility and the diversity of options it offers for running containerized applications. Static pods are just one piece of the puzzle, offering a unique blend of simplicity and reliability for specific use cases. I encourage you to explore static pods further, experiment with deploying different applications as static pods, and integrate them into your Kubernetes strategy where appropriate. Happy Kubernetes-ing!
Jira For Product Managers: Useful Features Explained
April 24, 2024 by
Handling Vectors in AI Context via PostgreSQL pgVector
April 25, 2024 by
VMware vSphere Backup Methods: Ensuring Data Safety and Recovery
April 25, 2024 by
Explainable AI: Making the Black Box Transparent
May 16, 2023 by CORE
How To Analyze Node.js Garbage Collection Traces
April 25, 2024 by CORE
Abusing Kubernetes Resources to the Moon
April 25, 2024 by
Low Code vs. Traditional Development: A Comprehensive Comparison
May 16, 2023 by
Enhancing Web Scraping With Large Language Models: A Modern Approach
April 25, 2024 by
How To Analyze Node.js Garbage Collection Traces
April 25, 2024 by CORE
How To Analyze Node.js Garbage Collection Traces
April 25, 2024 by CORE
Handling Vectors in AI Context via PostgreSQL pgVector
April 25, 2024 by
Five IntelliJ Idea Plugins That Will Change the Way You Code
May 15, 2023 by