DZone Spotlight

Jakarta EE 10 is probably the most important event of this year in the Java world. Since this fall, software editors providing Jakarta EE-compliant platforms are working hard to validate their respective implementations against the TCK (Technology Compatibility Kit) supplied by the Eclipse Foundation. At Payara, as much as in bigger companies like Oracle, Red Hat, or IBM, they aren't left behind, and, as of last September, they announced the availability of the Payara 6 Platform, declined in three versions: Server, Micro, and Cloud. As an implementation of Jakarta EE 10 Web, Core, and Micro Profile, Payara Server 6 is itself proposed in two editions: Community and Enterprise. But how does this impact Java developers? What does it mean in terms of application development and portability? Is it easier or more difficult to write and deploy code compliant to the new specifications than it was with Release 9 or 8 of the Jakarta EE drafts? Well, it depends. While any new Jakarta EE release aims at simplifying the whole bunch of the API (Application Programming Interface) set and at facilitating the developers' work, the fact that, on a total of 20 specifications, 16 have been updated, and a new one has been added, shows how dynamic the communities and the working groups involved in this process are. Which isn't without some difficulties when trying to transition to the newest releases with a minimal impact. This is especially true when it comes to combining, for example, JAX-RS 4.0 and its implementation by Jersey 3.1.0 with JSON-B 3.0 and its Yasson provider by Eclipse or when experiencing NoSuchMethodException, due to an inconvenient combination of versions. Or when noticing that a transitive Maven dependency, pulled out by a nonupdated library like RESTassured, still uses the old javax namespace. In order to avoid all these troubles, as marginal as they may be, one of the most practical solutions is to use Maven archetypes. A Maven archetype is a set of templates used to generate a Java project skeleton. They use Velocity placeholders which, at the generation time, are replaced by actual values that make sense in the current context. Hence, software editors, different OSS communities and work groups, or even individuals provide such Maven archetypes. The Apache community, for example, provides several hundreds of such Maven archetypes, and one may find one for almost any type of Java project. The advantage of using them is that the developers can generate a basic and clean skeleton of their Java project, on which they can build while avoiding some minor but painful annoyances. Jakarta EE 10 is so recent that most of its implementations are still in beta testing and consequently, the Maven archetypes dedicated to the Java projects using this release aren't yet available. In this blog post, I'm demonstrating such an archetype that generates a Jakarta EE 10 web applications skeleton and its associated artifacts to be deployed on a Payara 6 server. The code might be found here. The figure below shows the structure of our Maven archetype: A Maven archetype is a Maven project like any other, and, as such, it is driven by a pom.xml file, which the most essential part is reproduced below: XML <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>fr.simplex-software.archetypes</groupId> <artifactId>jakartaee10-basic-archetype</artifactId> <version>1.0-SNAPSHOT</version> <name>Basic Java EE 10 project archetype</name> ... <packaging>maven-archetype</packaging> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> </properties> <build> <extensions> <extension> <groupId>org.apache.maven.archetype</groupId> <artifactId>archetype-packaging</artifactId> <version>3.1.1</version> </extension> </extensions> </build> </project> The only notable thing that our pom.xml file should contain is the declaration of the archetype-packaging Maven plugin that will be used to generate our Java project skeleton. In order to do that, proceed as follows: Shell $ git clone https://github.com/nicolasduminil/jakartaee10-basic-archetype.git $ cd jakartaee10-basic-archetype $ mvn clean install [INFO] Scanning for projects... [INFO] [INFO] -----< fr.simplex-software.archetypes:jakartaee10-basic-archetype >----- [INFO] Building Basic Java EE 10 project archetype 1.0-SNAPSHOT [INFO] --------------------------[ maven-archetype ]--------------------------- [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ jakartaee10-basic-archetype --- [INFO] [INFO] --- maven-resources-plugin:3.3.0:resources (default-resources) @ jakartaee10-basic-archetype --- [INFO] Copying 11 resources [INFO] [INFO] --- maven-resources-plugin:3.3.0:testResources (default-testResources) @ jakartaee10-basic-archetype --- [INFO] skip non existing resourceDirectory /home/nicolas/jakartaee10-basic-archetype/src/test/resources [INFO] [INFO] --- maven-archetype-plugin:3.2.1:jar (default-jar) @ jakartaee10-basic-archetype --- [INFO] Building archetype jar: /home/nicolas/jakartaee10-basic-archetype/target/jakartaee10-basic-archetype-1.0-SNAPSHOT.jar [INFO] Building jar: /home/nicolas/jakartaee10-basic-archetype/target/jakartaee10-basic-archetype-1.0-SNAPSHOT.jar [INFO] [INFO] --- maven-archetype-plugin:3.2.1:integration-test (default-integration-test) @ jakartaee10-basic-archetype --- [WARNING] No Archetype IT projects: root 'projects' directory not found. [INFO] [INFO] --- maven-install-plugin:3.1.0:install (default-install) @ jakartaee10-basic-archetype --- [INFO] Installing /home/nicolas/jakartaee10-basic-archetype/pom.xml to /home/nicolas/.m2/repository/fr/simplex-software/archetypes/jakartaee10-basic-archetype/1.0-SNAPSHOT/jakartaee10-basic-archetype-1.0-SNAPSHOT.pom [INFO] Installing /home/nicolas/jakartaee10-basic-archetype/target/jakartaee10-basic-archetype-1.0-SNAPSHOT.jar to /home/nicolas/.m2/repository/fr/simplex-software/archetypes/jakartaee10-basic-archetype/1.0-SNAPSHOT/jakartaee10-basic-archetype-1.0-SNAPSHOT.jar [INFO] [INFO] --- maven-archetype-plugin:3.2.1:update-local-catalog (default-update-local-catalog) @ jakartaee10-basic-archetype --- [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 1.151 s [INFO] Finished at: 2022-12-02T13:32:35+01:00 [INFO] ------------------------------------------------------------------------ Here we're cloning first the GIT repository containing our archetype, and then, we're installing it in our local Maven repository, such that we can use it in order to generate projects. As explained earlier, our archetype is a set of template files located in the directory src/main/resources/atchetype-resources. These files use the Velocity notation to express placeholders that will be processed and replaced during the generation process. For example, looking at the file MyResource.java which exposes a simple REST API: Java package $package; import jakarta.ws.rs.*; import jakarta.ws.rs.core.*; import jakarta.inject.*; import org.eclipse.microprofile.config.inject.*; @Path("myresource") public class MyResource { @Inject @ConfigProperty(name = "message") private String message; @GET @Produces(MediaType.TEXT_PLAIN) public String getIt() { return message; } } Here, the placeholder $package will be replaced by the actual Java package name of the generated class. The whole bunch of resources that will be included in the generated project are described by the file archetype-metadata.xml located in src/main/resources/META-INF/maven. XML <archetype-descriptor xsi:schemaLocation="http://maven.apache.org/plugins/maven-archetype-plugin/archetype-descriptor/1.0.0 http://maven.apache.org/xsd/archetype-descriptor-1.0.0.xsd" xmlns="http://maven.apache.org/plugins/maven-archetype-plugin/archetype-descriptor/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" name="jakarta-ee-10-webapp"> <fileSets> <fileSet filtered="true" packaged="false" encoding="UTF-8"> <directory>src/main/java</directory> <includes> <include>**/*.java</include> </includes> </fileSet> <fileSet filtered="true" packaged="false" encoding="UTF-8"> <directory>src/test/java</directory> <includes> <include>**/*.java</include> </includes> </fileSet> <fileSet filtered="true" packaged="false" encoding="UTF-8"> <directory>src/main/resources</directory> <includes> <include>**/*.xml</include> <include>**/*.properties</include> </includes> </fileSet> <fileSet filtered="false" packaged="false" encoding="UTF-8"> <directory>src/main/webapp</directory> <includes> <include>index.jsp</include> </includes> </fileSet> <fileSet filtered="false" packaged="false" encoding="UTF-8"> <directory></directory> <includes> <include>.gitignore</include> </includes> </fileSet> <fileSet filtered="true" packaged="false" encoding="UTF-8"> <directory></directory> <includes> <include>README.md</include> <include>Dockerfile</include> <include>build.sh</include> </includes> </fileSet> </fileSets> </archetype-descriptor> The syntax above is self-descriptive and probably already known by all the Maven users. Once we installed our Maven archetype in our local Maven repository, we can proceed with the generation process: Shell $ cd example $ ../jakartaee10-basic-archetype/generate.sh [INFO] Scanning for projects... [INFO] [INFO] ------------------< org.apache.maven:standalone-pom >------------------- [INFO] Building Maven Stub Project (No POM) 1 [INFO] --------------------------------[ pom ]--------------------------------- [INFO] [INFO] >>> maven-archetype-plugin:3.2.1:generate (default-cli) > generate-sources @ standalone-pom >>> [INFO] [INFO] <<< maven-archetype-plugin:3.2.1:generate (default-cli) < generate-sources @ standalone-pom <<< [INFO] [INFO] [INFO] --- maven-archetype-plugin:3.2.1:generate (default-cli) @ standalone-pom --- [INFO] Generating project in Batch mode [INFO] Archetype repository not defined. Using the one from [fr.simplex-software.archetypes:jakartaee10-basic-archetype:1.0-SNAPSHOT] found in catalog local [INFO] ---------------------------------------------------------------------------- [INFO] Using following parameters for creating project from Archetype: jakartaee10-basic-archetype:1.0-SNAPSHOT [INFO] ---------------------------------------------------------------------------- [INFO] Parameter: groupId, Value: com.exemple [INFO] Parameter: artifactId, Value: test [INFO] Parameter: version, Value: 1.0-SNAPSHOT [INFO] Parameter: package, Value: com.exemple [INFO] Parameter: packageInPathFormat, Value: com/exemple [INFO] Parameter: package, Value: com.exemple [INFO] Parameter: groupId, Value: com.exemple [INFO] Parameter: artifactId, Value: test [INFO] Parameter: version, Value: 1.0-SNAPSHOT [INFO] Project created from Archetype in dir: /home/nicolas/toto/test [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 1.898 s [INFO] Finished at: 2022-12-02T13:53:32+01:00 [INFO] ------------------------------------------------------------------------ $ cd test $ ./build.sh [INFO] Scanning for projects... [INFO] [INFO] --------------------------< com.exemple:test >-------------------------- [INFO] Building test 1.0-SNAPSHOT [INFO] --------------------------------[ war ]--------------------------------- [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ test --- [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ test --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 1 resource [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ test --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 2 source files to /home/nicolas/toto/test/target/classes [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ test --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /home/nicolas/toto/test/src/test/resources [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ test --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 1 source file to /home/nicolas/toto/test/target/test-classes [INFO] [INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ test --- [INFO] [INFO] --- maven-war-plugin:3.3.1:war (default-war) @ test --- [INFO] Packaging webapp [INFO] Assembling webapp [test] in [/home/nicolas/toto/test/target/test] [INFO] Processing war project [INFO] Copying webapp resources [/home/nicolas/toto/test/src/main/webapp] [INFO] Building war: /home/nicolas/toto/test/target/test.war [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2.620 s [INFO] Finished at: 2022-12-02T13:54:33+01:00 [INFO] ------------------------------------------------------------------------ Sending build context to Docker daemon 13.66MB Step 1/2 : FROM payara/server-full:6.2022.1 ---> ada23f507bd2 Step 2/2 : COPY ./target/test.war $DEPLOY_DIR ---> 96650dc307b0 Successfully built 96650dc307b0 Successfully tagged com.exemple/test:latest Error: No such container: test 39934e82c8b164c4e6cd91036df7e2b0731254cdb869d7f2321ad1f2aaf37350 The generate.sh script that we're running above only contains the maven archetype:generate goal, as shown: Shell #!/bin/sh mvn -B archetype:generate \ -DarchetypeGroupId=fr.simplex-software.archetypes \ -DarchetypeArtifactId=jakartaee10-basic-archetype \ -DarchetypeVersion=1.0-SNAPSHOT \ -DgroupId=com.exemple \ -DartifactId=test Here we're using our Maven archetype in order to generate a new artifact which GAV (GroupID, ArtifactID, Version) are: com.example:test:1.0-SNAPSHOT. Once generated, the new project may be imported in your preferred IDE. As you may see, it consists of a simple REST API exposing and endpoint returning some text. For this purpose, we leverage Jakarta JAX-RS 4.0 and its Jersey 3.1 implementation with the Eclipse Microprofile Configuration 5.0. Please take some time to look at the generation project, including the pom.xml file and the dependencies used with their associated versions. All these dependencies are mandatory in order to get a valid artifact. We just generated our new project; let's build it now. In the listing above, we did that by running the script build.sh. Shell #!/bin/sh mvn clean package && docker build -t ${groupId}/${artifactId} . docker rm -f ${artifactId} || true && docker run -d -p 8080:8080 -p 4848:4848 --name ${artifactId} ${groupId}/${artifactId} This script is first packaging the newly generated Java project in a WAR, and after that, it builds a new Docker image based on the Dockerfile below: Dockerfile FROM payara/server-full:6.2022.1 COPY ./target/${artifactId}.war $DEPLOY_DIR As you may see, this Dockerfile is just extending the standard Payara server Docker image provided by the company to copy the WAR that was previously packaged into the auto-deployment server directory. Copying the WAR in the mentioned directory, which by the way is /opt/payara/deployments, automatically deploys the packaged application. Once this new Docker image built, we run it under the same name as our Maven artifactId, by mapping the ports 8080 and 4848. Please notice the way that the Velocity placeholders are again used. Once the Maven build process is successfully finished, a Docker container of the name of the test should be running. Of course, you need to have a running Docker daemon. You can test that everything is okay using the following curl request: Shell curl http://localhost:8080/test/api/myresource or by executing the script myresource.sh. An integration test is generated as well. It leverages test containers to execute an instance of Payara server 6 in a Docker container in which the application has been deployed. Then it uses the JAX-RS client, as implemented by Jersey Client 3.1, to perform HTTP requests to the exposed endpoint. You can experience it by running the following maven command: Shell $ mvn verify Please notice that this command can only be run after having previously executed the build.sh script or having manually run: Shell $ mvn -DskipTests clean package This is because the integration test uses testcontainers to deploy the WAR and, consequently, the WAR has to exist then. Hence, the package goal which creates the WAR should have been executed already. And we need to skip tests in order to avoid trying to execute them before packaging. Enjoy!

During my work on different projects and using different languages, frameworks, styles, and idioms, I found out that there are no silver bullets on how to design software. Starting from a set of requirements we need to implement, we have preferences that we should first write code and then test; on the other side, we have the TDD approach emerging for years now, as well as some other approaches (design-first, etc.). Here I want to explain one approach that I find to work very well when designing and implementing software, especially if it is a component or library. The main question here is how you start designing your code. Do you start with some kind of drawing, write tests first (TDD), or start imidate with an implementation? This is the method I developed over the years, which I found working very well in my development workflow. I call it Interface-Driven Development (IDD), and it is like the TDD process to some extent. This concept already existed in some areas, such as Protocol-oriented programming in Swift or Interface-based programming in Java, and it is based on Design by Contract by Bertrand Meyer, described in his book “Object-Oriented Software Construction” [3]. In the book, he discusses standards for contracts between a method and a caller. Also, Hunt and Thomas rely upon a similar concept in their “The Pragmatic Programmer” book [2], in the section on Prototyping Architecture: “Most prototypes are constructed to model the entire system under consideration. As opposed to tracer bullets, none of the modules in the prototype system need to be particularly functional. What you are looking for is how the system hangs together as a whole, again deferring details.“ The problem that this process needs to solve is components that are vaguely defined during design, and we tend to give more responsibility to some components than is necessary. A usual implication of such design is bad and untestable code. A Method The method consists of five consecutive steps, as follows: 1. Create a High-Level Idea When I start to think about the problem I need to solve, the focus is always on a user, developers, or other components which need to interact with a system I’m creating. This originates in User-Driven Design (UDD), where we always start by thinking about how anyone else will use our code. What actions are going to happen, and who is going to do them? What kind of events do we need to handle? Here we need to know what kind of messages our code will communicate with other components in the system, what we receive and what we can expect. At this point, you can use any tool which is in help for you, whiteboard, post-it cards, UML, or some other kind of diagrams you like. The point is to understand which elements you have in your design and how they will communicate with each other. Here we don’t care about actual implementation. 2. Design the Public Interface First Now, we create interfaces from the components we have and all public methods needed for them to communicate with each other. It is important to note here that we don’t have all information we need, but we create a minimal set of interfaces and methods based on the knowledge we have currently. Later, we will add more to this, and it is an iterative process. Here we can describe public contracts as a list of operations involved, including preconditions and postconditions, their parameters, return types, and eventual errors. To know how operations would be called in a sequence, we need to set some conditions, and that can be done in a form of a use case. A use case describes an interaction between a client and our interface that fulfills the goal. Use cases are usually expressed in technology-independent terms; work cases might include the names of the methods in the interface. E.g. for one case, we can write a series of steps we need to do to achieve it. We should tend to have simple interfaces but deep classes when designing interfaces. So we need to define such an interface, which is clear and easy to use, with a few parameters only, but an implementation of such methods should be deep as needed. 3. Write Test Cases Now, we want to test our interface to ensure that an implementation of an interface meets its contract. We can specify a contract in documentation, yet a test clarifies the contractual obligation, and we can verify it. One general rule here is that interface definition is not done until we have tested it for at least one implementation. So, here do black-box testing, where we test an interface without looking inside to see how it’s implemented. For our defined interface, our aim here is to write minimal passing test cases that will use our interfaces (unit tests). Yet, without an implementation yet and all test cases should fail (like TDD). And for the implementation, we will use mocks or stubs. With mocks and stubs, we go fast, and we don’t lose time for implementation. This process gives us two main things. First, we will check how our interface interacts with other interfaces or classes in the system, and it will allow us to revise our public interface if needed before we go to the implementation. Along with these advantages, this allows us to better understand how our piece of code will work in the system, but also enable us to adapt it if needed, as it is very cheap to do at this point. Here we can see if we need to introduce many dependencies; it is a sign that our code is not written properly and needs to be refactored. 4. Refactor Interfaces If we find any issues during running our tests and also in the way how our interface interacts with other components in the system, here we do adjustments and refactor our public interface. It is a fast way to do it, as we don’t have any underlying implementation to change. And here, we iterate between points 3. and 4. until we are satisfied without changes. When our interface is designed properly and all tests are green, we go with the next step. 5. Implement Real Code When we have our public interfaces written and tested, real implementation of those methods can start by using communication patterns we created with interfaces. Here we should say that during the implementation process, we may need to alter again our public interfaces, but our initial design should stay stable with minimal adjustments in the end. An Example So, when we want to implement something, select a couple of use cases first for each requirement we need to implement. From here, we continue with the process (IDD) as described above. Here we will see the process in the example of creating a simple microblogging platform in C#. Here I want to add blogging functionality to my existing website, where I want to post an article with some tags, have the possibility to list all blog posts on my site, have a preview of each blog post, and search blog posts via tags or direct text search. We need to be able to write down our blog posts to some data storage and retrieve it, and on top of that, we want to cache them for better efficiency. For such a platform, we need a few cases (C stands for a case): C1. I can post blogs with tags. C2. Blogs can be listed. C3. A blog post can be viewed. C4. Blogs can be searched by using tags. From here, it is obvious that we have some existing Website service that needs to communicate with some kind of Blogging service. This service will be our main component for this requirement. Also, based on requirements, it is obvious that we need also some way to store or cache data. Now, we need to understand our components and define their boundaries because we don’t one that one component to take much responsibility for itself (e.g., Blogging service to store and read data, do cache, and everything else), as this would break Single-Responsibility Principle (SRP) [5]. Here we want to move responsibilities to their respective components. So, our design could look to this: From here, we need the following services: Website service -> editing capabilities, converting our text to HTML, preview it. Blogging service -> main service to handle all main use cases. Data service -> our storage engine, which could be Firebase or local files (in JSON, Raw data, or HTML format). Caching service -> cache data in memory, read, and write. So, let’s start with implementing use cases in IDD style: C1. Add a Blog Post To add a blog post, we first need some kind of a BloggingService to handle all these requirements. The use case here would be: creating a blog post with no precondition and one postcondition that a blog post is created. So, let’s start with the interface first and a signature: public interface IBloggingService { public bool AddBlogPost(string text, string[] tags); } Here we still don’t know the implementation of this method. How and where that blog post will be stored actually. From here, we call some other service or repository to store it. Now, our existing Website service can call this service to add a blog post. As an example: public class IWebsiteService { // Start service // Stop service // Render elements // Layouting // ... public bool AddBlogPost(string htmlText) { // Sanitize inputs // Validation bloggingService.AddBlogPost(text, tags); // Success } } From this, we can already draw some conclusions. How these two components will work, what data is passed around, what kind of communication we will have, etc. For other use cases, it is similar, as follows. C2. List All Blog Posts Now we want to list all blog posts so that we list them on our Website. For this, we need to add a new method to the Blogging service. public interface IBloggingService { public bool AddBlogPost(string text, string[] tags); // New method public List<Post> GetAllBlogPosts(); } C3. A Post Can Be Viewed When a user clicks on one blog post, we need to retrieve it from our system and show it. This we can do through a passed ID from the upper component (WebsiteService): public class WebsiteService { public Post GetBlogPostById(string ID) { // Sanitize inputs // Validation bloggingService.GetBlogPostById(ID); // Success } } and our interface will look like this: public interface IBloggingService { public bool AddBlogPost(string text, string[] tags); public List<Post> GetAllBlogPosts(); // New method public Post GetBlogPostById(string ID); } C4. Search Blog Posts By a Tag As our blog posts have one or more tags, we want to have the opportunity to search them by those tags. public interface IBloggingService { public bool AddBlogPost(string text, string[] tags); public List<Post> GetAllBlogPosts(); public Post GetBlogPostById(string ID); // New method public List<Post> SearchPostsByTag(string tag); } So now we know what our interface will look like, and now we write test cases for each one. Test cases We will show here an example of one test case (c1). For others, we would follow a similar approach. C1. Add a Blog Post Now, we want to implement our main method for adding a new blog post and using the underlying service which is going to store it. We implement a method without an implementation: public class BloggingService: IBloggingService { public bool AddBlogPost(string text, string[] tags) { throw new NotImplementedException(); } } and here we write a test case for it: [TestClass] public class WebsiteServiceTest { // ... Setup [TestMethod] public void AddBlogPost_OnePost_BlogPostIsAdded() { // Arrange var bloggingServiceStub = new Mock<IBloggingService>(); var sut = new WebsiteService(bloggingServiceStub.Object); var blogPost = @"This is a formatted <b>blog post</b> with @tag1 @tag2"; // Act var result = sut.AddBlogPost(blogPost); // Assert Assert.IsFalse(result); // Fail bloggingServiceStub.Verify(x => x.AddBlogPost(blogPost), Times.Once); } } The test will fail. At this point, we don’t use the real implementation of our method, but we will mock it. [TestClass] public class WebsiteServiceTest { // ... Setup [TestMethod] public void AddBlogPost_OnePost_BlogPostIsAdded() { // Arrange var bloggingServiceStub = new Mock<IBloggingService>(); var blogPostSanitized; bloggingServiceStub.Setup(x => x.AddBlogPost(It.IsAny<string>(), It.IsAny<string[]>())) .Callback<string>(callbackResult => blogPostSanitized = callbackResult) .Verifiable(); var sut = new WebsiteService(bloggingServiceStub.Object); var blogPost = @"This is a formatted <b>blog post</b> with @tag1 @tag2"; // Act var result = sut.AddBlogPost(blogPost); // Assert Assert.IsTrue(result); // Success Assert.AreEqual(blogPostSanitized, "This is a formatted <b>blog post</b>"); bloggingServiceStub.Verify(x => x.AddBlogPost(blogPost), Times.Once); } } We can continue the development of our IBloggingService, adjusting it and understanding its dependencies. At this point, the interface can change, but also a test. If we want to test the relation with other services, such as Data or Caching services, we can also create a stub and test it. Now, when we are sure that our design works, we can continue with implementing our method. public class BloggingService : IBloggingService { // Some props and dependencies IDataService dataService; ICacheService cacheService; public(IDataService dataService, ICacheService cacheService) { this.dataService = dataService; this.cacheService = cacheService; } public bool AddBlogPost(string text, string[] tags) { // First we check do we have the same blog post already // if yes, we update // Store text with data service var result = dataService.StoreBlogPost(text, tags); return result; // Success } } Here we can see what kind of an interface we need on the data service side, so we can start with its definition too: public interface IDataService { public bool StoreBlogPost(string text, string[] tags); // More methods } At this point, we don’t know any of the internals of data service, how it will store or fetch data, in which format, etc. And now, the cycle continues; we create a test for this method, check how they work together, and later implement it. In the same manner, we would do it for a caching service. Conclusion In this text, I presented one way to design and implement software systems called Interface-Driven Development (IDD). It starts from the high-level idea, creating interfaces and method signatures but not implementing them. We write tests by using mocks and stubs and do the necessary corrections here. When we are sure how those components (and their interfaces will work), we implement them. This process looks to the TDD to some extent, yet, the focus is here on proper design, while the implementation is left for the last step.

The nature of FinTech makes customized DevOps a necessity. Most modern, sophisticated companies recognize the value of developing a DevOps culture that removes organizational silos, changes how they do business to deliver value rapidly, and creates iterative feedback loops to enable continuous improvement. However, there are several unique aspects of the FinTech industry that make a tailored approach to DevOps vital. Delivering innovative value to the customer while navigating an ever-changing regulatory and security scene is challenging, but a FinTech-centric approach to DevOps will help you succeed. In this article, we’ll explore how DevOps fits into a FinTech world. We’ll look at the approach you should take in developing your unique DevOps culture and engineering practices, and we’ll touch upon successful DevOps practices used by top FinTech companies. This article assumes you’re familiar with the basic concepts of DevOps, including the idea that DevOps stresses cultural change, not simply the adoption and implementation of technology tools. What Makes FinTech Different? While most tech companies share many common concerns, several aspects of the FinTech space make it substantially different from many other industries. If you want to develop an effective, personalized DevOps culture in your FinTech, then consider these unique factors. Although the traditional software industry already changes rapidly, the FinTech landscape is evolving more rapidly. FinTech is a relatively new domain even though it operates in one of the oldest domains of human enterprise: financial services. However, recent advances in technology combined with changing consumer expectations and comfort result in a segment of the global economy that is constantly responding to market changes. Consumer-Driven Market Changes Some of these changes are driven by consumers, who only recently have come to trust technological solutions for their basic financial needs. People are beginning to demand increasingly sophisticated FinTech solutions for financial tasks that have been performed manually for decades or even centuries. For example, FinTech services are taking the real estate industry by storm. Of the companies listed in Forbes FinTech 50 for 2022, five FinTech companies are in the real estate sector. From offering new ways to invest in rental properties to disrupting the mortgage experience for today’s homebuyers, FinTechs in real estate are driven by a younger, technology-savvy demographic. This next generation of consumers demands the same ease of use in the property investment and mortgage experience that they have come to expect in routine point-of-sale and personal banking services. Legislation-Driven Regulatory Demands Other changes in the FinTech space are driven by the regulatory environment. Many traditional financial institutions have been slow to adopt digital technology. Much of the legislation in place around the planet was crafted for a financial industry that bears little resemblance to the modern FinTech industry. As governments attempt to reconcile finance law with the innovative financial products and services provided by FinTech companies, they’re constantly studying new products. For example, the Consumer Financial Protection Bureau (CFPB) analyzed the growth of buy-now-pay-later (BNPL) in its new report. Therefore, FinTech companies often find themselves needing to balance their pace of innovation against regulatory considerations. A company in the FinTech space is not like any old software company. Its business landscape is different, so it’s sure to have unique processes. This includes the area of DevOps. With its use of automation and feedback to achieve rapid delivery of value to the customer, DevOps is essential to the strategy of any successful software company. FinTechs, however, have additional DevOps considerations that are unique to their space. Crafting Your DevOps Culture Adopting a DevOps methodology impacts a company’s developer culture as well as its practices. In FinTech companies, certain cultural elements of DevOps should receive special attention. Decentralized Ownership One of the most important cultural elements to consider is the idea of “decentralized ownership.” In order to have the agility to respond effectively to changing consumer demands and regulations, ownership of specific product features or infrastructure requirements cannot be walled off in siloes. Any competent employee — or team — who identifies a critical need or task should be empowered and encouraged to work towards its resolution, independent of organizational structure or rigid job descriptions. Valuing Self-Criticism A related element to integrate into your DevOps approach is to stress the value of being self-critical. As operators in the vanguard of consumer financial technology, FinTechs will inevitably encounter things going wrong. If your culture doesn't make it acceptable (and encouraged) for teams to examine what role they played in how something went wrong, you won't be able to innovate or respond quickly enough. Readiness for Breach As FinTech services become more vital for the modern consumer, FinTech companies will increasingly become the target of security threats. Mature teams take an approach to security that prepares for breach, assuming that a significant security incident will occur at some point. This posture will impact their approach to the software development life cycle (SDLC) and DevOps strategy. Empowering Your Engineers As DevOps has so much to do with the culture we create in a company, it can be helpful for FinTech leaders to consider their engineering culture specifically. High Communication Regarding Requirements In FinTech, it’s especially important to communicate business goals and constraints to your engineers. Of course, every engineering team would like this; but in FinTech, we often have to modify our product and its features not just to create value for our customers but also to satisfy regulatory requirements. For an engineer who takes their craft seriously, understanding technical requirements without the accompanying rationale can be a challenge. To be realistic, however, even the best efforts at high communication may fall short. Sufficient communication across the organization or the engineering team is not always achievable. Where there may be gaps or the possibility of gaps, put “policy as code” in place to act as preventative controls for what humans might get wrong. Strategic Innovation Amidst a Constantly Changing Landscape Another critical area of engineering culture that you should give thought to is how your engineers should spend their “innovation energy.” In an industry with so much change influenced by external factors, your engineering resources are extremely valuable. Any time your engineers can free up for innovation work should be used with judgment. As an organization, discuss when and how to use your resources toward innovation. Also, keep in mind that innovation is not always about doing more. Sometimes, resources can be put to answer the question: Can we do less? When innovating in this space, the answer may often be, “Yes, we can save some energy here.” Balancing Security and Agility Security is, of course, a vital concern for any tech company. In FinTech companies that deal with sensitive personal and financial data, the consequences of a security breach can be devastating. A key influence on your DevOps culture is the tension between the desire to move fast and the necessity to build secure services. FinTech companies work to create a culture where this conflict isn't resolved with an either-or solution. Which element should we sacrifice — security or innovation? Instead, these top companies push their teams to always look for both-and solutions. How do we create an innovative feature AND meet our challenging SLA and security targets? As part of your DevOps approach, you’ll certainly use automation to improve the throughput of your software delivery machine. Because of this, it’s important to give a lot of thought to the targets you choose for automation. You’ll need an evaluation process in place that helps you quickly determine what to automate. This way, your automation work will produce the biggest improvements in delivering value to the customer while addressing legal and regulatory requirements. Best Practices: Insights from an Industry Leader On the subject of DevOps, I had a conversation with Jeffrey Hamblin. Hamblin is an engineering lead at Marqeta, a FinTech company focused on innovative payment solutions through modern card issuing. In 2021 alone, Marqeta processed over $110 billion in payment transaction volume. As a leader in the FinTech space, Marqeta has had many opportunities to explore the DevOps space and develop a successful DevOps strategy. Here are some of the specific best practices that Marqeta found useful in their DevOps culture: Automation of Source Code Chain of Custody In their code repositories, Marqeta automates the enforcement of branch protection rules. This isn’t because they don’t trust people (although this practice does help reduce error-prone and time-consuming manual work). Instead, this enables Marqeta to assert to auditors that reasonable dual-human control exists for production changes. This type of enforcement needs to be accompanied by education to build a culture of compliance. Putting up guardrails can cause friction, especially if not everybody in the organization is on the same page about compliance requirements. In a highly-regulated industry like finance, every member of the team needs to understand that compliance regulations are non-negotiables that can get your business heavily fined or shut down if you don’t give them heed. On this topic, Hamblin commented, “Discovering ways to make the necessary assertions of compliance that simultaneously benefit your culture or your quality is a rewarding way to creatively engineer the processes for your team.” Continuous Measurement of DevOps Performance No company can know ahead of time how a change to the SDLC process might deliver more value to the customer (for example, through faster releases of features or better code quality). However, by continuously measuring DevOps performance, a company can test the effectiveness of any changes and respond early to this feedback. For Marqeta, their most important metric is commit-to-deploy latency, or the amount of time between when a change is “done” and when that change is adding value in production. They found that an increase in this time indicated sub-par functioning of their engineering systems, while a decrease usually accompanied higher confidence in the process and the development process. Whether it’s a DevOps team adopting chaos testing or going through incident management training, measurements help Marqeta track how those initiatives translate to business outcomes. By tracking metrics related to sprint health or DORA metrics like mean time to recovery (MTTR), an organization has data to decide how best to use its resources. Provide Motivated Teams With Great Tools Hamblin shared the following: One of the most fulfilling events occurred when we integrated a code scanning and analysis tool into our CI pipelines. Some teams saw that and really took the initiative to raise the bar. Discussions of “what should be a blocking gate” on merges began to take place. Shining the light on that mindset and celebrating those teams is the name of the game. What’s the takeaway? Choose tools that can bring to light what DevOps teams are doing well and concentrate on that, balancing out the near-constant focus on what isn't going well. When a team is motivated, provide them with tools to help them lean into their strengths, and you might see even higher capacity and performance. Conclusion The unique character of the FinTech world requires a thoughtful, forward-thinking approach to your DevOps strategy. Striking the right balance between security and agility is key. You’re responsible for creating a culture. Why not create a culture of possibility where members of your DevOps team ask themselves, “Instead of security or agility, why can't we have security and agility?” Taking the time upfront to discuss the DevOps culture that makes sense for your organization is a worthwhile endeavor. And be sure to connect with and learn from fellow FinTech devs when you can. After all, it's your culture and your people that will guarantee your survival and prosperity in the wild world of FinTech.

Throughout this article series, we’ve introduced you to the Flow blockchain, its smart contract language Cadence, and some of the most essential tools developers should know, all while comparing and contrasting to Ethereum. In this article, we will talk about best practices and patterns that should be followed when using the Cadence language and developing solutions on the Flow network, as well as patterns to avoid. Best Practices Use Flow Test Network and Flow Emulator One of the most important steps in developing on any blockchain is testing your projects in a simulated environment. Flow blockchain is no different. For this purpose, the Flow testnet and Flow Emulator are both crucial components to include in your development process. Not only do they allow you to understand how your dApp will perform, but they will also help you catch any critical bugs before deploying to mainnet. Use Standard Contracts Two of the most common use cases for any blockchain are digital currencies and providing proof of digital ownership through NFTs. Instead of trying to rewrite the necessary contracts from scratch, Flow developers should always import the official core contracts to ensure user safety and consistency with the rest of the network. There are currently three standard contracts you should use: Fungible Token - Used to create new tokens Non-Fungible Token - Used to create NFTs Metadata Views - Used as a metadata pattern for NFTs These contracts are essentially interfaces that force whoever is implementing them to add their variable and method declarations, enabling them to be interoperable with other smart contracts in the Flow ecosystem. They already exist on both testnet and mainnet networks, so be sure to import from the official addresses linked above when implementing them in your contract. You can find other core contracts in the Flow documentation. Create Tests for Your Smart Contracts, Transactions, and Scripts The Flow JavaScript Testing Framework is crucial for testing deployment scenarios for your contracts. However, it’s important to note that you’ll need the Flow CLI running in the background for full functionality. With these, you can test creating new accounts, sending transactions, running queries, executing scripts, and more. Additionally, it integrates with the Flow Emulator so you can create, run, and stop emulator instances. Add Your Contract to the Flow NFT Catalog The Flow NFT Catalog exists as a database to store NFT contracts and metadata. By uploading your contract, your NFTs become interoperable with the rest of the Flow ecosystem, and other developers can easily support your collection in their marketplaces or other dApps. You can add your contract to the catalog by completing a simple four-step process. Be Specific With Types Cadence is a strong and statically typed language that allows the developer to specify which types contain or return variables, interfaces, and functions. Therefore, you should be as specific as possible when making declarations; use generic types only in necessary situations. Not doing so can result in clumsy errors. Access Control Whenever possible, you should be deliberate with access control. Luckily, on the Flow blockchain, a caller cannot access objects in another user’s account storage without a reference to it. This is called reference-based security, and it means that nothing is truly public by default. However, when writing Cadence code, care must be taken when declaring variables, structs, functions, resources, and so on. There are four levels of access control a developer can include in their declarations: pub/access(all) - accessible/visible in all scopes access(account) - only accessible in the entire account where it’s defined (other contracts in the same account can access) access(contract) - only accessible in the scope of the contract in which it is defined (cannot be accessed outside of the contract) priv/access(self) - only accessible in current and inner scopes Avoid using pub/access(all) whenever possible. Check out the Flow documentation for more information on access control. Create StoragePath and PublicPath Constants Inside Your Contract The paths to your contract’s resources are extremely important and must be consistent across all transactions and scripts. To ensure uniformity, you should create constants for both PublicPath and StoragePath. pub contract BestPractices { pub let CollectionStoragePath: StoragePath pub let CollectionPublicPath: PublicPath init(){ self.CollectionStoragePath = /storage/bestPracticesCollection self.CollectionPublicPath = /public/bestPracticesCollection } } When you create a transaction that uses one of these paths, you only need to call its respective variable with the imported contract to reference the required path. //This script checks if a user has the public Collection from BestPractices contract import BestPractices from 0x... pub fun main(addr: Address): Bool { return getAccount(addr).getLinkTarget(BestPractices.CollectionPublicPath) != nil } Admin Resource It’s good practice to create an administrative resource that contains specific functions to perform actions under the contract, such as a mint function. This convention ensures that only accounts with that resource or capability can perform administrative functions. pub contract BestPractices { pub let CollectionStoragePath: StoragePath pub let CollectionPublicPath: PublicPath pub let AdminStoragePath: StoragePath pub resource AdminResource { pub fun mintNFT(){ //your mint NFT code here! } } init(){ self.CollectionStoragePath = /storage/bestPracticesCollection self.CollectionPublicPath = /public/bestPracticesCollection self.AdminStoragePath = /storage/bestPracticesAdmin //Create the adminResource and store it inside Contract Deployer account let adminResource <- create AdminResource() self.account.save(<- adminResource, to: self.AdminStoragePath) } } Create Custom Events Events are values that can be emitted during the execution of your Cadence code. For example, when defining important actions in your contracts, you can emit events to signal their completion or deliver a specific value. As a result, transactions that interact with your smart contracts can receive additional information through these events. pub contract BestPractices { pub let CollectionStoragePath: StoragePath pub let CollectionPublicPath: PublicPath pub let AdminStoragePath: StoragePath //Create your own events pub event AdminCreated() pub resource AdminResource { pub fun mintNFT(){ //your mint NFT code here! } } init(){ self.CollectionStoragePath = /storage/bestPracticesCollection self.CollectionPublicPath = /public/bestPracticesCollection self.AdminStoragePath = /storage/bestPracticesAdmin //Create the adminResource and store it inside Contract Deployer account let adminResource <- create AdminResource() self.account.save(<- adminResource, to: self.AdminStoragePath) } } Patterns on Cadence/Flow Blockchain Be Specific With Types in Type Constraints One of the most powerful features of the Cadence language is undoubtedly capabilities. Through capabilities, the scope of resource access expands. An important point when creating a capability is to specify which features of your resource should be available to others. This can be done at the time of link creation using type constraints. In this example, we use the ExampleNFT contract to create a basic functionality where any account that wants to receive an ExampleNFT must have a collection. import NonFungibleToken from 0x... import ExampleNFT from 0x... transaction{ prepare(acct: AuthAccount){ let collection <- ExampleNFT.createEmptyCollection() // Put the new collection in storage acct.save(<-collection, to: ExampleNFT.CollectionStoragePath) // Create a public Capability for the collection acct.link<&ExampleNFT.Collection{ExampleNFT.CollectionPublic}>(ExampleNFT.CollectionPublicPath, target: ExampleNFT.CollectionStoragePath) } } The "&" symbol in &ExampleNFT specifies that we are using a reference. After the reference symbol, we add the type to which the capability we are creating will have access. At this point, we need to be as specific as possible. This pattern strengthens security and limits the functionality that the user calling the borrow function of this capability can use. Omitting the {ExampleNFT.CollectionPublic} type will give you access to all the functions that exist in the ExampleNFT.Collection reference, including the withdraw function, so that anyone can access the user's collection and steal their NFTs. Use the Borrow Function To use the resource's features, you could call the Load function to remove the resource from the account, use its features, and call the Save function to save it again. However, this approach is costly and inefficient. To avoid this, use the borrow function instead. It allows you to use a reference to the resource you are calling. This method makes your transaction much more efficient and cost-effective. Use the check and getLinkTarget Functions When building applications on the Flow blockchain, you will discover the user’s account plays a vital role. Unlike other blockchains, such as Ethereum, Flow stores resources, assets, and more directly in the user's account rather than as a reference to an address on a public digital ledger. This approach requires the account to have a specific storage location, such as a collection or vault for fungible tokens. However, this also adds complexity. One must be sure that the user either does or does not have the collection in their account. Both the check() function (which checks whether a capability exists in a given path) and the getLinkTarget() function (which returns the path of a given capability) must be used when adding collections and capabilities to the user account. These functions ensure that the transaction executes without problems. Use Panic Panic is a built-in function in Cadence that allows you to terminate a program unconditionally. This can occur during the execution of your smart contract code and returns an error message, which makes it easier to understand when something does not go as expected. When declaring variables, it is possible to define them as optional; meaning, if they are not of the specified type, they have a value of nil. Thus, in Cadence, it is possible to use two question marks followed by the panic("treatment message") function when querying the value of a particular variable or function that returns an optional. let optionalAccount: AuthAccount? = //... let account = optionalAccount ?? panic("missing account") This command ??panic("treatment message") attempts to return the value with the specified type. If the returned value is the wrong type, or nil, the execution aborts, and the selected treatment message displays on the console. Anti-Patterns Although Cadence is designed to avoid many of the potential bugs and exploits found in other blockchain ecosystems, there are some anti-patterns developers should be aware of while building. Listed below are a few important ones to consider. For a complete list of anti-patterns, check out the Flow documentation. Failing to Specify the Type When Using the Borrow Function Developers should use the borrow function mentioned above to take advantage of the features available in a capability. However, it should be clear that users can store anything in their memory. Therefore, it is vital to make sure that what is borrowed is of the correct type. Not specifying the type is an anti-pattern that can end up causing errors or even breakage in your transaction and application. //Bad Practice. Should be avoided let collection = getAccount(address).getCapability(ExampleNFT.CollectionPublicPath) .borrow<&ExampleNFT.Collection>()?? panic("Could not borrow a reference to the nft collection") //Correct! let collection = getAccount(address).getCapability(ExampleNFT.CollectionPublicPath) .borrow<&ExampleNFT.Collection{NonFungibleToken.CollectionPublic}>() ?? panic("Could not borrow a reference to the nft collection") Using AuthAccount as a Function Parameter For a transaction in the preparation phase, it is possible to access the AuthAccount field of the user. This object allows access to the memory storage and all other private areas of the account for the user who provides it. Passing this field as an argument is not recommended and should be avoided, as cases where this method is necessary are extremely rare. //Bad Practice. Should be avoided transaction() { prepare(acct: AuthAccount){ //You take sensitive and important user data out of the scope of the transaction prepare phase ExampleNFT.function(acct: acct) } } Using Public Access Control on Dictionaries and Arrays By storing dictionaries and array variables in your contract with pub/access(all) scope, your smart contract becomes vulnerable, as anyone can manipulate and change these values. Creating Capabilities and References Using the Auth Keyword Creating capabilities and references with the auth keyword exposes the value to downcasting, which could provide access to functionality that was not originally intended. //AVOID THIS! signer.link<auth &ExampleNFT.CollectionPublic{NonFungibleToken.Receiver}>( /public/exampleNFT, target: /storage/exampleNFT ) Conclusion When developing on the Flow blockchain using Cadence, it helps to be aware of design patterns, anti-patterns, and best practices. By following these best practices, we can ensure a consistent level of security throughout the Flow ecosystem, thus providing the best experience for users. For a more thorough understanding, read through the Flow documentation. Have a really great day!

As an engineering leader, one of your top priorities is improving the effectiveness and productivity of the developers on your team. The first step to managing and improving your engineering team is adopting a metric-driven approach to identifying the problem areas that threaten your team’s performance. Successful teams keep track of their performance through a set of chosen indicators called software engineering metrics. With these metrics, engineering leaders can visualize progress, identify bottlenecks, watch for anomalous trends, and predict when something’s off before a deadline is missed. One such important but often overlooked metric in software development is code churn. In this guide, we’ll unpack what code churn is, why high levels of churn can be detrimental to a project, and what to do when you notice an unexpected spike in churn. What Is Code Churn? Code churn, also known as code rework, is when a developer deletes or rewrites their own code shortly after it has been composed. Code churn is a normal part of software development and watching trends in code churn can help managers notice when a deadline is at risk, when an engineer is stuck or struggling, problematic code areas, or when issues concerning external stakeholders come up. It is common for newly composed code to go through multiple changes. The volume and frequency of code changes in a given period of time can vary due to several factors and code churn can be good or bad depending upon when and why it is taking place. For example, engineers frequently test, rewrite, and examine several solutions to an issue particularly at the beginning of a new project or task when they are experimenting with solutions to the task at hand. In this case, code churn is good, because it is a result of creative problem-solving. Code churn can be good or bad depending upon when and why it is taking place. Code Churn Metric Breakdown RefactorCode that is modified after 21 days of committing is called refactored code. Refactored code is usually an acceptable change that is needed for maintenance hence, is distinct from code churn so as not to raise any red flags. New WorkCode that is newly added and that is not replacing or rewriting existing code. Help othersCode that is replaced by engineers other than the author of the original code within 21 days of authoring. This helps you measure to what extent developers are helping their teammates to improve code quality and delivery. Watching trends across this spectrum of metrics during a development lifecycle creates a better ground for effectively debugging the root cause and gaining potential insights such as: Which team members are spending more time helping others, than perhaps working on their own work? The percentage of time engineers spend on new features (new work) vs. application maintenance (refactoring) Anomaly alerts when any or all of these indicators trend out of the anticipated range can equip managers to combat challenges, preempt risks to delivery, and gain visibility into critical processes that might require an improvement. How to Detect Unproductive Code Churn Code churn varies depending on many factors. For instance, when engineers work on a fairly new problem, churn would most likely be higher than the benchmark, whereas when developers work on a familiar problem or a relatively easier problem, churn could most likely be lower. Churn could also vary depending on the stage of a project in the development lifecycle. Hence, it is important for engineering managers and leaders to develop a sense of the patterns or benchmarks of churn level for different teams and individuals across the organization. While code churn, by itself, is neither good nor bad, there is a cause for concern only when churn levels digress from team or individual benchmarks for the particular project that is being worked on. When such digression occurs, it is important to identify the factors contributing to unproductive code churn. What Can High Code Churn Indicate? Complicated Tasks A higher level of churn is to be expected when an engineer is exploring and backtracking with a particularly challenging problem at hand. It is when the exploration has gone on for too long that it is a call for concern. An unusual high churn level might indicate that an engineer did not completely understand the assignment, or neglected to fully comprehend the issue, or didn’t have the expertise to address the assignment. In many cases, engineers feel that they have successfully handled the issue, perhaps even sending it off for review, and then finding that significant areas of it needed to be changed. Unclear Requirements or Changing Requests from External Stakeholders Factors outside the normal development process such as a poor PRD (product requirements document) or unclear or indecisive stakeholders can also lead to high code churn. A sudden increase in churn or a sudden spike in new work, especially in the final phases of a project, is usually an indication that a miscommunication between the stakeholders or new requirements led to the final code undergoing changes. When this pattern is seen sprint over sprint with the same team it can damage both morale and progress and can lead to frustration in the team over time. An Indicator of Future Quality Problems Measuring code churn equips managers with foresight to predict and preempt potential future problems. The most problematic code is the one that is complicated to grasp and altered frequently. A high level of churn exposes potential code hotspots, and if these frequent changes are performed on complicated and critical code, the code tends to be more error prone. Hence, code churn can be a predictor of high-risk code.These code hotspots, if not recognized early during refactoring efforts, can result in developers accumulating huge amounts of technical debt. This debt grows as more opportunities for code refactoring are missed and, as a result, new development becomes difficult, especially when features are built upon legacy code. Deadline Is At Risk A higher percentage of reworking and code deletion resulting from experimentation is commonly seen at the beginning of a project — especially when a project is new or unfamiliar. A similar trend, sometimes called “exploratory churn,” is expected in the case of particularly challenging problems. Although code churn resulting from creative problem solving is a positive outcome, it becomes a risk to meeting project deadlines when such experimental coding continues for a long period of time, risking the timeline of the development cycle. Similarly, churn should stabilize as a project nears the release timeline. An early indication that the delivery ought to be pushed back is when you start seeing a high volume of churn leading up to a release. How to Prevent High Code Churn When faced with high unproductive code churn, here are some potential actions managers can implement. Better Planning Managers should assign developers to projects and tasks based on programming language and code complexity. Using data-driven and factual insights for planning team and task allocation can improve instances of unproductive code churn. High rates of churn in particular code hotspots can likely be an instance where an engineer, for a prolonged period of time, remains unwaveringly focused on a particular region of the codebase, making just little tweaks here and there. This could be an early sign of a burnout. Data-driven planning provides managers the opportunity to assign a new set of tasks or projects to such engineers which would help them navigate to new areas of the codebase. Training Leaders should ensure that their developers receive the right training and learning so that they have the right skillset to create the features that the application requires. A widely used and successful training method pairs programming sessions with senior engineers who naturally tend to help others. Such pair-learning exercises also help in boosting the morale and effectiveness of the team. Clear Requirements If the specs are poorly defined or inadequate, the developer is forced to work with hazy requirements, forcing them to rely on their best reasonable guess to decode and fill in any gaps. To avoid this, managers have to ensure that their developers get the most up-to-date requirements so that they can create appropriate solutions and avoid rework. Conclusion For far too long, engineering leaders have relied on limited signals and their own intuition to assess the performance of and adequately help their teams. The aim of tracking code churn and other metrics is to enable fact and data-driven decision-making. Data-driven feedback loops assist in identifying process improvement possibilities and tuning engineering routines in real-time. Code churn is frequently ignored and underutilized by several software organizations. However, tracking and managing churn can lead to teams discovering severe issues not just inside their codebases but also in their developer education and in engineering routines. Measuring code churn will certainly help engineering leaders to manage and optimize their team’s performance and productivity.

A survey has found that 59% of project managers manage between 2 and 5 projects, 11% manage 6 to 10 projects, and 15% manage more than 10 projects at a time. Only 15% of project managers manage one project at a time. Managing even a single project can turn out to be an overwhelming task for project managers, leave alone running multiple projects at the same time. The task is made even more difficult when working with remote teams to get the job done. Project managers have to juggle multiple responsibilities at the same time—planning projects, creating tasks and assigning them, monitoring the team performance, checking the project progress, managing deliverables, and much more. It’s easy for project managers to lose track of vital project information amid all the chaos. The standard fields offered in some project management tools usually do not match the unique requirements of your business or workflow. How can you add additional data to tasks? How do you track and prioritize work effectively? How can you configure your projects to track what matters to you? If these questions spring up in your mind often, then it’s evident that you need a team collaboration and project management tool that offers you clarity and detail into task requirements - Custom Fields. What Are the Limitations of Standard Default Fields in a Project Management Software? There are some project management tools that do not offer users the option to add custom fields. Standard default fields do not give users the leverage to add additional information that is necessary to your project’s workflow. The fact is no two projects are alike. They differ in terms of scope, deadlines, and budget. Some projects are relatively straightforward, while others are tough and complex. Hence, the workflow of every project is different from the others, even though exceptions are always there. When it comes to project management, standard default fields do not allow users to add details that are unique to the project. Neither can you configure tasks based on project requirements. The lack of custom fields also hampers your ability to track various activities within a project. What Are Custom Fields? As the term suggests, custom fields allow you to add and store additional relevant data to your projects. Custom fields in project management software allow users to add more detail to their tasks beyond the standard default filters. With custom fields, you can add more fields to suit the unique needs of your team’s workflow. For example, in addition to default fields, you can create a field for Text, Date, Currency, Percentage, Text area, Numbers, Dropdown, or any other field that’s important to your organization, team, and workflow. This allows project managers and team members to have a granular view of different tasks and other activities across the project. What Are Different Types of Custom Fields in a Project Management Software? There are many project management tools available in the market. Only a handful of them offers users the option of custom fields. Though no two PM software offers the same options in custom fields, you can expect some common data types, which are as follows. 1. Text You can use the text field to store information of any type and size. This custom field is generally used by team members to add task descriptions or specific instructions for those who are assigned to the task. 2. Numbers This custom field is meant specifically for adding and storing numeric data. It is useful for storing figures like IP addresses, logged working hours, tracking numbers, contact numbers, etc. 3. Percentage The percentage custom field is usually added to denote the progress of tasks in the form of percentage data. For example, if the task is in progress, it would be denoted as 50%. 4. Tags Tags are like labels. You can create and add them and use these tags again and again to organize and track your work. You can also customize them using different color codes. 5. Date This custom field can be created and added to store start and due dates, estimated dates, and set dates for tasks. 6. Dropdown Dropdown custom fields can be used to create and add a list of options for users to select from. You can add any number of options and color code them for a better view. How To Use Custom Fields in a Project Management Software Whether you’re a project manager or a team member, adding custom fields in project management software is as easy as it can get. You must add custom fields to selected tasks per your project workflow requirements. This will allow easier access, filtering, and sorting of tasks. Using custom fields, you can be more descriptive about your tasks. When you add fields to tasks, these automatically get added as sections when tasks are viewed in a table view. Be it resource allocation, daily task management, or work prioritization, you can easily track anything within your project with the help of this innovative feature. Some examples of custom field applications are as follows: Budget Management — You can add custom fields to manage and track project costs besides restricting the visibility of your sensitive financial data to only authorized users. Task Progress — Custom fields allow you to be more descriptive about task statuses. Apart from standard “In-progress,” “Pending,” or “Finished Statuses,” you can choose to display task progress in the form of a percentage or by adding insightful options like Ideas, Approval, Review, Submitted, Published, Declined, etc. Categorization Of Tasks — You can classify your tasks into different categories, which makes it easy to search and filter them. You can also prioritize tasks according to project requirements. Add Tags — You can add tags to certain tasks using Custom Fields. Tags work like labels, which is an effective way to categorize, organize, and prioritize your work. Create Dropdown — You can create dropdown menus and different options for team members to choose from. For example, you can create a dropdown and add options for users to choose from, like Stage 1, Stage 2, Stage 3, etc. Multiple Uses — The main advantage of custom fields is that these can be edited according to your project requirements and store pivotal information. The application of custom fields is not limited to specific project details. For example, the number field can be used to add, store, and track numerical data like IP addresses, payroll, ticket numbers, budget, costs, and more. Hence, every custom field can be used as per your workflow and project requirements. The Final Word Custom fields bring flexibility and customizability to project management. There are many project management tools that offer project teams default project management processes, which leaves them with little to no room for editing and customizing things. Custom fields enable users to work on different projects without any confusion. When you choose a project management software for planning, organizing, and executing projects efficiently, make sure it does offer you the much-required feature — custom fields.

TL; DR: Business Agility, Scrum, and Generative AI’s Take on Getting There There has been a lot of talking about generative AI recently, mainly fueled by excellent work results in the text and graphics area. A few days ago, OpenAI made a new model available that “interacts in a conversational way. The dialogue format makes it possible for ChatGPT to answer follow-up questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests.” (Source.) I thought it might be fun to ask ChatGPT a few questions about business agility in general and Scrum in particular. Will Google Be Toast Anytime Soon? Paul Buchheit, Google’s 23rd employee, who created Gmail, has a pessimistic outlook on Google concerning the growing capabilities of generative AI. The question is obvious: What if a Generative AI can not just answer questions but also learn while constantly improving its model? Wouldn’t that lead to the long-run obsolescence of many knowledge workers, for example, Scrum Masters, agile coaches, product managers, and Product Owners? Let’s put the model to the test! (Moreover, please try ChatGPT yourself; access is free for the time being.) My Questions to the Generative AI on Business Agility and Scrum Chatting with OpenAI’s ChatGPT is straightforward: you add a question to the entry field and trigger the reply, which typically takes no more than 1-2 seconds. Then you can rate the response, thumb-up or thumb-down, and ChatGPT provides an opportunity to add some remarks, clarifying your feedback. Moreover, you can ask ChatGPT to make another attempt to answer the question. (There are a few examples of that below.) I chatted with the Generative AI on December 3, 2022. OpenAI’s ChatGPT on Business Agility Let's get started with some general questions: Question 1: “Why is agility beneficial?” Answer: “Agility is beneficial because it allows people and organizations to respond quickly and effectively to changes in their environment. This can help them stay competitive, adapt to new situations, and make the most of opportunities as they arise. In sports, agility is important for making quick changes in direction, avoiding obstacles, and staying balanced. In business, it can help organizations respond to market trends, changing customer needs, and new technologies. In general, agility is an important quality to have because it enables people to be flexible, adaptable, and responsive to the world around them.” We are off to a decent start; responding to change seems to be the hook, and ChatGPT distinguishes between business and sports. Question 2: “Why is agility beneficial in business?” Answer: “Agility is beneficial in business because it allows organizations to respond quickly and effectively to changes in their environment. This can help them stay competitive, adapt to new situations, and make the most of opportunities as they arise. For example, if a company is agile, it can quickly adapt to changes in market trends, changing customer needs, and new technologies. This can help the company maintain its competitive edge and continue to grow and succeed in a rapidly changing business environment. Additionally, being agile can help a company be more flexible and adaptable, which can make it better able to respond to unexpected challenges and opportunities.” Now responding to change is related to preserving competitiveness in the face of changing customer needs and technology. Also, ChatGPT adds a bit of uncertainty to the mix. Question 3: “What defines business agility?” Answer: “Business agility refers to the ability of a company to quickly and effectively respond to changes in its environment. This can include changes in market trends, customer needs, technologies, and other factors that can affect the company's operations. A company that is agile is able to adapt to these changes in a flexible and effective manner, which can help it stay competitive and successful in a rapidly changing business environment. Business agility is often achieved through the use of agile practices and methodologies, such as agile project management and agile software development. These practices help companies to be more flexible, adaptable, and responsive to changes in their environment.” It gets a bit repetitive; however, the Generative AI adds agile practices and refers directly to agile software development. Let's drill down a bit. Question 4: “What factors contribute to business agility?” Answer: “There are several factors that can contribute to business agility, including the following: A culture of continuous learning and improvement: In order to be agile, companies need to create a culture that encourages learning and growth. This means creating an environment where employees are encouraged to learn new skills, share knowledge, and try new things. Flexible and adaptable processes: Companies need to have processes in place that are flexible and adaptable. This means being open to trying new approaches and methods, and being willing to change course if something isn't working. The ability to make decisions quickly: In order to be agile, companies need to be able to make decisions quickly and effectively. This means having the necessary information and data available, as well as the ability to analyze it and make informed decisions. Collaboration and communication: Agile companies rely on collaboration and communication to get things done. This means having strong communication channels and mechanisms in place to allow teams to work together effectively. The use of agile practices and methodologies: Many companies use agile practices and methodologies, such as agile project management and agile software development, to help them be more agile. These practices help to create a more flexible and adaptable environment, and can help companies respond to changes in their environment more quickly and effectively.” So far, that is OpenAI’s ChatGPT most prolific answer, ticking many boxes at the heart of successful transformation, from Kaizen spirit to flexible processes to fast decision-making, and, lastly, the importance of collaboration and communication. Question 5: “What are agile practices and methodologies?” Answer: “Agile practices and methodologies are a set of principles and approaches that are used to help organizations be more flexible and adaptable. These practices and methodologies are often used in the context of software development, but they can also be applied to other areas of business, such as project management and product development. Agile practices and methodologies are based on the idea that organizations need to be able to respond quickly and effectively to changes in their environment. They focus on creating flexible and adaptable processes that can be easily changed as needed, and on fostering collaboration and communication among teams. Some of the most common agile practices and methodologies include agile project management, agile software development, and Scrum.” So, our Generative AI knows Scrum and that agile practices can be used outside of software development. However, there is no mention of other practices, for example, Kanban or Lean, or the Manifesto for Agile Software Development. Question 6: “What is the role of the management in agile?” Answer: “The role of management in agile is to provide support and guidance to teams as they work using agile methodologies. This can include providing resources, training, and support to teams, as well as helping to remove any obstacles or challenges that they may face. Additionally, management plays a key role in fostering a culture of continuous learning and improvement, and in promoting collaboration and communication among team members. In general, the role of management in agile is to provide the support and guidance that teams need to be successful, and to help create an environment that is conducive to using agile methodologies.” OpenAI’s ChatGPT has a remarkably enlightened stance on management in an agile environment. Question 7: “Do we need managers in an agile organization?” Answer: “It is not necessary for an agile organization to have traditional managers. In fact, many agile organizations use a more decentralized and flat management structure, in which teams are self-organizing and empowered to make their own decisions. This approach is based on the idea that teams are best equipped to make decisions about the work that they are doing, and that they are more likely to be successful if they are given the freedom and autonomy to do so. However, this does not mean that agile organizations do not need any form of management. Agile organizations still need leaders who can provide guidance and support to teams, and who can help to create an environment that is conducive to using agile methodologies. These leaders may not have the same title or role as traditional managers, but they still play an important role in helping the organization to be successful.” No traditional managers are needed, but decentralized organizations and flat hierarchies — hear, hear! I like ChatGPT pointing out the necessity that managers in agile organizations create an environment, a culture that enables agility to flourish. Question 8: “Do managers hinder or advance agility?” Answer: “It depends on the approach that managers take to agility. In some cases, managers can hinder agility by being resistant to change, failing to provide sufficient training and support, or not setting clear goals and objectives. These actions can make it difficult for teams to implement agile practices effectively, and can hinder the organization's efforts to become more agile. On the other hand, managers who are supportive of agility, who provide the necessary training and support, and who foster a culture of continuous learning and improvement can help to advance agility within the organization. In general, the role of managers in promoting agility will depend on their approach and their willingness to support and enable agile practices.” The Generative AI mentions goals and objectives for the first time. Question 9: “Can we fund an agile transformation by letting go of the managers?” Answer: “It is possible to fund an agile transformation by letting go of some managers, but this may not necessarily be the best approach. While cutting costs can be one way to fund a transformation, it is important to consider the potential consequences of letting go of managers. In many cases, managers play a critical role in providing guidance and support to teams, and in helping to create an environment that is conducive to using agile methodologies. If managers are let go without adequate planning and support, it can create disruptions and challenges for teams, and can hinder the organization's ability to successfully implement agile practices. Additionally, simply cutting costs may not be enough to fully fund a transformation, and the organization may need to explore other options, such as securing additional funding or finding new sources of revenue. In general, while cutting costs can be one way to fund a transformation, it is important to carefully consider the potential consequences and to have a comprehensive plan in place.” The answer seems more driven by linking change with cost-cutting than considering the particular case of becoming a learning, self-managing organization with a flatter hierarchy. Generative AI ChatGPT on Scrum Now that we have a first idea of the model’s capability at the meta-level, let’s have a look at Generative AI’s idea of Scrum: Question 10: “What is Scrum?” Answer 10.1: “Scrum is an agile framework that is used to manage complex projects and products. It is a flexible, iterative approach that allows teams to quickly and effectively respond to changes in their environment. Scrum is based on the idea that projects and products are best managed through collaboration, continuous learning, and iterative development. It consists of a set of roles, meetings, and artifacts that help teams to work together and deliver value to their customers. The key roles in Scrum include the Scrum Master, the Product Owner, and the development team. Scrum meetings include the daily stand-up, the sprint planning meeting, the sprint review, and the sprint retrospective. And the key artifacts in Scrum include the product backlog, the sprint backlog, and the increment.” Answer 10.2: “Scrum is an agile framework that is used to manage and control complex projects. It is a flexible, iterative approach that focuses on delivering high-quality products and increasing efficiency and productivity. Scrum is based on the idea that projects are complex and unpredictable, and that the best way to manage them is through collaboration, transparency, and adaptability. In Scrum, teams work in short, iterative cycles called "sprints," in which they identify and prioritize the most important tasks, and then work together to complete them. Scrum emphasizes regular communication and collaboration among team members, as well as regular feedback and reviews to ensure that the project stays on track.” Here I tried the opportunity to have another answer from ChatGPT. Generally, it is aware of the basic Scrum principles; however, the answer looks superficial to me. Somehow, OpenAI’s ChatGPT seems stuck with the 2017 Scrum Guide, still referring to the Development team and not mentioning commitments. Moreover, there are only three artifacts. Also, it repeats a typical sloppiness with the terminology, for example, talking about the “daily stand-up.” But, at least it got two out of three elements of empiricism, and we know that “two out of three ain’t bad.” Question 11: “What is the purpose of the Scrum Master?” Answer: “The Scrum Master is a key role in the Scrum framework. The purpose of the Scrum Master is to facilitate the Scrum process and help the team follow the Scrum values and principles. The Scrum Master is responsible for ensuring that the team has the tools, resources, and support they need to be successful. This can include helping the team to identify and prioritize their work, facilitating communication and collaboration among team members, and providing guidance and support as needed. The Scrum Master is also responsible for protecting the team from distractions and disruptions, and for facilitating regular meetings and reviews to ensure that the project stays on track. In general, the Scrum Master is an essential part of the Scrum process, and plays a crucial role in helping teams to be successful.” Spot on. Question 12: “A Scrum Team member refuses to participate in retrospectives. As a Scrum Master, what can I do?” Answer 12.1: “As a Scrum Master, it is your responsibility to facilitate the Scrum process and ensure that all team members are participating and contributing to the best of their abilities. If a team member is refusing to participate in retrospectives, it is important to have a conversation with them to understand their reasons and address any concerns they may have. It may be helpful to remind them of the importance of retrospectives in the Scrum process and how they can help the team improve and become more effective. If the team member continues to refuse to participate, it may be necessary to escalate the issue to the team's Product Owner or other relevant stakeholders. Ultimately, it is important for all team members to be fully committed to the Scrum process in order for it to be successful.” Answer 12.2: “As a Scrum Master, you can try the following approaches to address the situation: Address the issue directly: Talk to the team member and find out the reason behind their refusal to participate in retrospectives. It could be due to lack of understanding of the importance of retrospectives or any other personal issues. Try to address their concerns and explain the benefits of retrospectives for the team and the individual. Facilitate the retrospectives: As a Scrum Master, you can facilitate the retrospectives to make sure that everyone is participating actively. Use different techniques and activities to engage the team members and make the retrospectives more interactive and enjoyable. Provide feedback: After each retrospective, provide feedback to the team member on their participation and highlight the areas where they can improve. This can motivate them to participate more actively in the next retrospective. Involve the team: Encourage the team members to share their feedback and ideas with the team member who is not participating in the retrospectives. This can help the team member to understand the importance of retrospectives and also encourage them to participate in the future retrospectives. Follow up: After the retrospective, follow up with the team member and check if they are willing to participate in the next retrospective. If not, repeat the above steps and continue to provide support and guidance to help the team member understand the importance of retrospectives.” Coming up with a definition of an element of a framework is a thing. Applying the spirit of a framework like Scrum to a practical issue is a different beast. I consider answer #1 good up to the moment when ChatGPT suggests escalating “the issue to the team’s Product Owner or other relevant stakeholders.” Answer #2 starts convincing but quickly falls into some meta-level talking about facilitation. The suggested “guidance” from the Scrum Master looks more like line managing to me. Question 13: “What are the first steps to introduce Scrum?” Answer 13: “The first steps to introduce Scrum in an organization are as follows: Educate yourself and your team about Scrum: Before introducing Scrum, it's important to understand the basics of the framework and how it works. This means learning about the Scrum values, principles, and practices, and how they can be applied to your specific project or organization. Identify the project or team that will use Scrum: The next step is to identify the project or team that will use Scrum. This will typically be a project or team that is complex and unpredictable, and that would benefit from the flexibility and adaptability of Scrum. Create a cross-functional Scrum team: In Scrum, teams are cross-functional, which means that they have all the skills and expertise they need to complete their work. This means that you will need to bring together team members from different departments and disciplines to form a Scrum team. Appoint a Scrum Master: The Scrum Master is a key role in the Scrum framework, and is responsible for facilitating the Scrum process and helping the team follow the Scrum values and principles. You will need to appoint a Scrum Master for your team who has a good understanding of Scrum and can provide the necessary guidance and support. Identify and prioritize the work: The next step is to identify and prioritize the work that the team will complete during the first sprint. This will involve creating a list of tasks and objectives, and then prioritizing them based on their importance and value to the project. Begin the first sprint: Once the work has been identified and prioritized, the team can begin the first sprint. This will involve working together to complete the tasks and objectives that have been identified, and regularly reviewing and reassessing the work to ensure that the project stays on track.” Again, for my taste, too much classic management is included in the Generative AI’s answer: identify a team, appoint a Scrum Master, and identify and prioritize the work. OpenAI’s ChatGPT has a limited understanding of the importance of self-management and how that will manifest itself in practice; for example, by the self-selection of Scrum teams. Question 14: “What factors contribute to the failure of Scrum?” Answer 14: “There are several factors that can contribute to the failure of Scrum. Some of the most common reasons include the following: Lack of commitment from team members: In order for Scrum to be successful, all team members must be fully committed to the process. If some team members are not fully invested in the project, or if they are not willing to follow the Scrum values and principles, the process can break down and the project may fail. Insufficient training or support: In order for Scrum to be effective, team members need to have a clear understanding of the process and how it works. If team members are not properly trained or supported, they may struggle to understand their roles and responsibilities, and the project may suffer as a result. Poorly defined goals or objectives: In order for Scrum to be successful, the goals and objectives of the project need to be clearly defined. If the goals are not well-defined, or if they are not aligned with the broader business objectives, the team may struggle to make progress and the project may fail. Lack of collaboration and communication: Scrum relies on collaboration and communication among team members to be successful. If team members are not communicating effectively, or if they are not working together as a team, the project may suffer. Resistance to change: Scrum is a flexible and adaptable process, but it requires teams to be open to change. If team members are resistant to change, or if they are not willing to adapt to new approaches and methods, the project may fail.” While the answer covers five critical reasons why transformations fail, the answer is not Scrum-specific but applicable to any project management framework. Question 15: “Can you force Scrum upon an organization?” Answer 15: “While it is possible to introduce Scrum to an organization, it is not recommended to try to force it upon the organization. Scrum is a flexible and adaptable framework, but it requires commitment and buy-in from all team members in order to be successful. If team members are not fully committed to the Scrum process, or if they are not willing to follow the Scrum values and principles, the process can break down and the project may fail. Instead of trying to force Scrum upon an organization, it is better to educate people about the benefits of Scrum, and to work with them to create a plan for introducing it in a way that will be accepted and embraced by the organization.” The suggestion is a correct, universal answer to the question of how to bring about change. It is not Scrum-specific. Question 16: “What should be avoided if you want Scrum to succeed?” Answer 16: “There are several things that should be avoided if you want Scrum to succeed. Some of the most important things to avoid include the following: Trying to force Scrum on the team: As mentioned earlier, Scrum is a flexible and adaptable framework, but it requires commitment and buy-in from all team members in order to be successful. If team members are not fully committed to the Scrum process, or if they are not willing to follow the Scrum values and principles, the process can break down and the project may fail. Not providing sufficient training or support: In order for Scrum to be effective, team members need to have a clear understanding of the process and how it works. If team members are not properly trained or supported, they may struggle to understand their roles and responsibilities, and the project may suffer as a result. Not having clearly defined goals and objectives: In order for Scrum to be successful, the goals and objectives of the project need to be clearly defined. If the goals are not well-defined, or if they are not aligned with the broader business objectives, the team may struggle to make progress and the project may fail. Not fostering collaboration and communication: Scrum relies on collaboration and communication among team members to be successful. If team members are not communicating effectively, or if they are not working together as a team, the project may suffer. Not being open to change: Scrum is a flexible and adaptable process, but it requires teams to be open to change. If team members are resistant to change, or if they are not willing to adapt to new approaches and methods, the project may fail.” Cool, our Generative AI seems to learn while we chat, referring to the previous question and answer. Otherwise, it is primarily a repetition of answer 14, a set of generic answers from a project management handbook. Generative AI — Conclusion We do not know what data was used to train OpenAI’s ChatGPT model. There are indications that, as far as business agility, Scrum, and other agile practices are concerned, the data seems to be limited or not up-to-date. Nevertheless, the answers are often acceptable, some even good, despite some repetitiveness and superficiality. At this stage, though, I am not concerned that a Generative AI will soon render agile practitioners obsolete in practicing Scrum, Kanban, or other agile frameworks.

Let's continue building our Java-based, command-line text editor that we started in Part 1 and Part 2. Here in Part 3, we will cover the following: How to implement page-up and page-down functionality How to make the end key work properly, including cursor snapping How to make our text editor work on all operating systems, including macOS and Windows - not just Linux You're in for a ride! What’s in the Video In the previous episode, we got vertical scrolling with the arrow keys working. Unfortunately, when you press page up or page down, nothing happens - and we need to change that! We'll use a tiny trick to simulate the page up/down functionality, mapping it to pressing the arrow up/down for the number of rows our screen has. It serves as a good initial implementation, though there are a couple of edge cases we need to iron out. Once we have the page up and down working, it's time to care about horizontal scrolling. At the moment, our text viewer renders lines overflowing the screen, leading to heavy flickering whenever we vertically move our cursor. Ideally, we only want to render as much text as we have columns on the screen - and then we want to move the screen's contents horizontally, whenever we press the left or right keys at the beginning or end of the screen. To implement horizontal scrolling we can take most of the code for vertical scrolling, copy and paste it, and just replace a couple of key variables - done! After horizontal scrolling, let's take care of a couple of minor editing issues: first of all, the end key. It currently makes the user jump to the end of the screen. Ideally, we'd like the end key to only jump to the end of the current line. With a couple of small changes to our moveCursor() function, we can implement that behavior. This opens up another problem: when we are at the end of a line and then move vertically upwards or downwards, we also want to automatically snap to the end of the new line, not just end up somewhere in the middle. So, we'll need to fix our cursor-snapping implementation. In between, I'll leave a couple of notes for you regarding cursor line wrapping. We don't have enough time to implement it in this episode, but it would serve as a great exercise, for you, the watcher, to implement. Last but not least, we'll need to fix a couple of issues for our macOS and Windows platform support. The issue with macOS is that while it uses the same OS APIs as Linux, it uses different values for the OS calls. Hence, we'll need to invent an abstraction/delegation layer that detects if the current OS is macOS or Linux, and then use the corresponding, OS-specific classes. Windows uses a completely different API to put terminals into raw mode or get the current terminal size, and we'll have to dig deep into Microsoft's API documentation to find out which Windows methods we'll need to implement on our JNA side. That's it for today! See you in the next episode, where we'll implement searching across your text file.

The subject of Web resource caching is as old as the World Wide Web itself. However, I'd like to offer an as-exhaustive-as-possible catalogue of how one can improve performance by caching. Web resource caching can happen in two different places: client-side - on the browser and server-side. This post is dedicated to the former; the next post will focus on the latter. Caching 101 The idea behind caching is simple: if a resource is time- or resource-consuming to compute, do it once and store the result. When somebody requests the resource afterwards, return the stored result instead of computing it a second time. It looks simple - and it is, but the devil is in the detail, as they say. The problem is that a "computation" is not a mathematical one. In mathematics, the result of a computation is constant over time. On the Web, the resource you requested yesterday may be different if you request it today. Think about the weather forecast, for example. It all boils down to two related concepts: freshness and staleness. A fresh response is one whose age has not yet exceeded its freshness lifetime. Conversely, a stale response is one where it has. A response's freshness lifetime is the length of time between its generation by the origin server and its expiration time. An explicit expiration time is the time at which the origin server intends that a stored response can no longer be used by a Cache without further validation, whereas a heuristic expiration time is assigned by a Cache when no explicit expiration time is available. A response's age is the time that has passed since it was generated by, or successfully validated with, the origin server. When a response is "fresh" in the cache, it can be used to satisfy subsequent requests without contacting the origin server, thereby improving efficiency. -- RFC 7234 - 4.2. Freshness Early Web Resource Caching Remember that the WWW was relatively simple at its beginning compared to nowadays. The client would send a request, and the server would return the requested resource. When the resource was a page, whether it was a static page or a server-rendered page was unimportant. Hence, early client-side caching was pretty "rustic". The first specification of Web caching is defined in RFC 7234, aka HTTP/1.1 Caching, in 2014. Note that it has been superseded by RFC 9111 since 2022. I won't talk here about the Pragma HTTP header since it's deprecated. The most straightforward cache management is through the Expire response header. When the server returns the resource, it specifies after which timestamp the cache is stale. The browser has two options when a cached resource is requested: Either the current time is before the expiry timestamp: the resource is considered fresh, and the browser serves it from the local cache Or it's after: the resource is considered stale, and the browser requires the resource from the server as it was not cached The benefit of Expire is that it's a purely local decision. It doesn't need to send a request to the server. However, it has two main issues: The decision to use the locally cached resource (or not) is based on heuristics. The resource may have changed server-side despite the Expiry value being in the future, so the browser serves as an out-of-date resource. Conversely, the browser may send a request because the time has expired, but the resource hasn't changed. Moreover, Expire is pretty basic. A resource is either fresh or stale; either return it Cache or send the request again. We may want to have more control. Cache-Control to the Rescue The Cache-Control header aims to address the following requirements: Never cache a resource at all Validate if a resource should be served from the cache before serving it Can intermediate caches (proxies) cache the resource? Cache-Control is an HTTP header used on the request and the response? The header can contain different directives separated by commas. Exact directives vary depending on whether they're part of the request or the response. All in all, Cache-Control is quite complex. It might be well the subject of a dedicated post; I won't paraphrase the specification. However, here's a visual help on how to configure Cache-Control response headers. The Cache-Control page of Mozilla Developer Network has some significant use cases of Cache-Control, complete with configuration. As Expire, Cache-Control is also local: the browser serves the resource from its cache, if needed, without any request to the server. Last-Modified and ETag To avoid the risk of serving an out-of-date resource, the browser must send a request to the server. Enters the Last-Modified response header. Last-Modified works in conjunction with the If-Modified-Since request header: The If-Modified-Since request HTTP header makes the request conditional: the server sends back the requested resource, with a 200 status, only if it has been last modified after the given date. If the resource has not been modified since, the response is a 304 without any body; the Last-Modified response header of a previous request contains the date of last modification. Unlike If-Unmodified-Since, If-Modified-Since can only be used with a GET or HEAD. -- If-Modified-Since Let's use a diagram to make clear how they interact: Note: the If-Unmodified-Since has the opposite function for POST and other non-idempotent methods. It returns a 412 Precondition Failed HTTP error to avoid overwriting resources that have changed. The problem with timestamps in distributed systems is that it's impossible to guarantee that all clocks in the system have the same time. Clocks drift at different paces and need to synchronize to the same time at regular intervals. Hence, if the server that generated the Last-Modified header and the one that receives the If-Modified-Since header is different, the results could be unexpected depending on their drift. Note that it also applies to the Expire header. Etags are an alternative to timestamps to avoid the above issue. The server computes the hash of the served resource and sends the ETag header containing the value along with the resource. When a new request comes in If-None-Match containing the hash value, the server compares it with the current hash. If they match, it returns an 304 as above. It has the slight overhead of computing the hash vs. just handing the timestamp, but it's nowadays considered a good practice. The Cache API The most recent way to cache on the client side is via the Cache API. It offers a general cache interface: you can think of it as a local key value provided by the browser. Here are the provided methods: Cache.match(request, options) Returns a Promise that resolves the response associated with the first matching request in the Cache object. Cache.matchAll(request, options) Returns a Promise that resolves to an array of all matching responses in the Cache object. Cache.add(request) Takes a URL, retrieves it and adds the resulting response object to the given cache. This is functionally equivalent to calling fetch(), then using put() to add the results to the cache. Cache.addAll(requests) Takes an array of URLs, retrieves them, and adds the resulting response objects to the given cache. Cache.put(request, response) Takes both a request and its response and adds it to the given cache. Cache.delete(request, options) Finds the Cache entry whose key is the request, returning a Promise that resolves to true if a matching Cache entry is found and deleted. If no Cache entry is found, the Promise resolves to false. Cache.keys(request, options) Returns a Promise that resolves to an array of Cache keys. The Cache API works in conjunction with Service Workers. The flow is simple: You register a service worker on a URL The browser calls the worker before the URL fetch call From the worker, you can return resources from the cache and avoid any request to the server It allows us to put resources in the cache after the initial load so that the client can work offline - depending on the use case. Summary Here's a summary of the above alternatives to cache resources client-side. Order Alternative Managed by Local Pros Cons 1 Service worker + Cache API You Yes Flexible Requires JavaScript coding skills Coding and maintenance time 2 Expire Browser Yes Easy configuration Guess-based Simplistic Cache-Control Browser Yes Fine-grained control Guess-based Complex configuration 3 Last-Modified Browser No Just works Sensible to clock drift ETag Browser No Just works Slightly more resource-sensitive to compute the hash Note that those alternatives aren't exclusive. You may have a short Expire header and rely on ETag. You should probably use both a level 2 alternative and a level 3 one. A Bit of Practice Let's put the theory that we have seen above into practice. I'll set up a two-tiered HTTP cache: The first tier caches resources locally for 10 seconds using Cache-Control The second tier uses ETag to avoid optimizing the data load over the network I'll use Apache APISIX. APISIX sits on the shoulder of giants, namely NGINX. NGINX adds ETag response headers by default. We only need to add the Cache-Control response header. We achieve it with the response-rewrite plugin: YAML upstreams: - id: 1 type: roundrobin nodes: "content:8080": 1 routes: - uri: /* upstream_id: 1 plugins: response-rewrite: headers: set: Cache-Control: "max-age=10" Let's do it without a browser first. Shell curl -v localhost:9080 Plain Text HTTP/1.1 200 OK Content-Type: text/html; charset=utf-8 Content-Length: 147 Connection: keep-alive Date: Thu, 24 Nov 2022 08:21:36 GMT Accept-Ranges: bytes Last-Modified: Wed, 23 Nov 2022 13:58:55 GMT ETag: "637e271f-93" Server: APISIX/3.0.0 Cache-Control: max-age=10 To prevent the server from sending the same resource, we can use the ETag value in an If-None-Match request header: Shell curl -H 'If-None-Match: "637e271f-93"' -v localhost:9080 The result is a 304 Not Modified as expected: Plain Text HTTP/1.1 304 Not Modified Content-Type: text/html; charset=utf-8 Content-Length: 147 Connection: keep-alive Date: Thu, 24 Nov 2022 08:26:17 GMT Accept-Ranges: bytes Last-Modified: Wed, 23 Nov 2022 13:58:55 GMT ETag: "637e271f-93" Server: APISIX/3.0.0 Cache-Control: max-age=10 Now, we can do the same inside a browser. If we use the resend feature a second time before 10 seconds have passed, the browser returns the resource from the cache without sending the request to the server. Conclusion In this post, I described several alternatives to cache web resources: Expiry and Cache-Control, Last-Modified and ETag, and the Cache API and web workers. You can easily set the HTTP response headers via a reverse proxy or an API Gateway. With Apache APISIX, ETags are enabled by default, and other headers are easily set up. In the next post, I will describe caching server-side. You can find the source code for this post on GitHub. To go further: RFC 7234: HTTP/1.1: Caching (obsolete) RFC 9111: HTTP Caching HTTP caching Cache-Control Cache API

Using YOLOv5 in PyTorch YOLO, an acronym for 'You only look once,' is an open-source software tool utilized for its efficient capability of detecting objects in a given image in real time. The YOLO algorithm uses convolutional neural network (CNN) models to detect objects in an image. The algorithm requires only one forward propagation through a given neural network to detect all objects in the image. This gives the YOLO algorithm an edge in speed over others, making it one of the most well-known detection algorithms to date. What Is YOLO Object Detection? An object detection algorithm is an algorithm that is capable of detecting certain objects or shapes in a given frame. For example, simple detection algorithms may be capable of detecting and identifying shapes in an image, such as circles or squares, while more advanced detection algorithms can detect more complex objects, such as humans, bicycles, cars, etc. Not only does the YOLO algorithm offer high detection speed and performance through its one-forward propagation capability, but it also detects them with great accuracy and precision. In this tutorial, we will focus on YOLOv5, which is the fifth and latest version of the YOLO software. It was originally released on the 18th of May, 2020. The YOLO open-source code can be found on GitHub. We will be using YOLO with the well-known PyTorch library. PyTorch is a deep learning open-source package that is based on the well-known Torch library. It's also a Python-based library that is more commonly used for natural language processing and computer vision. How Does the YOLO Algorithm Work? Step 1: Residual Blocks (Dividing the Image Into Smaller, Grid-Like Boxes) In this step, the complete (whole) frame is divided into smaller boxes or grids. All the grids are drawn over the original image sharing the exact shape and size. The idea behind these divisions is that each grid box will detect the different objects inside it. Step 2: Bounding Box Regression (Identifying the Object Inside a Bounding Box) After detecting a given object in an image, a bounding box is drawn surrounding it. The bounding box has parameters such as the center point, height, width, and class (object type detected). Step 3: Intersection Over Union (IOU) The IOU, short for intersection over union, is used to calculate our model's accuracy. This is achieved by quantifying the degree of intersection of two boxes: the real value box (red box in image) and the box returned from our result (blue box in image). In the tutorial portion of this article, we identified our IOU value as 40 percent, meaning that if the intersection of the two boxes is below 40 percent, then this prediction should not be taken into consideration. This is done to help us calculate the accuracy of our predictions. Below is an image showing the complete process of the YOLO detection algorithm For additional information on how the YOLO algorithm works, view the Introduction to YOLO algorithm. What Are We Trying to Achieve With Our Model? The main goal of the example in this tutorial is to use the YOLO algorithm to detect a list of chest diseases in a given image. As with any machine learning model, we will run ours using thousands of chest-scanned images. The goal is for the YOLO algorithm to successfully detect all lesions in the given image. Data Set The VinBigData 512 image Dataset used in this tutorial can be found on Kaggle. The data set is divided into two parts, the training, and the testing data sets. The training data set contains 15,000 images, while the testing data set contains 3,000. This division of data between the training and the testing is somehow optimal as the training data set is usually 4 to 5 times the size of the testing data set. The other part of the data set contains the label for all the images. Inside this data set, each image is labeled with a class name (chest disease found), along with the class ID, width and height of the image, etc. Check the below image to view all the columns available. YOLOv5 Tutorial Note: You can view the original code used in this example on Kaggle. Step 1: Importing the Necessary Libraries To start with, we will import the required libraries and packages at the very beginning of our code. First, let's explain some of the more common libraries that we just imported. NumPy is an open-source numerical Python library that allows users to create matrices and perform a number of mathematical operations on them. import pandas as pd import os import numpy as np import shutil import ast from sklearn import model_selection from tqdm import tqdm import wandb from sklearn.model_selection import GroupKFold\ from IPython.display import Image, clear_output # to display images from os import listdir from os.path import isfile from glob import glob import yaml # clear_output() Step 2: Defining Our Paths To make our life easier, we will start by defining the direct paths to the labels and the images of the training and testing data sets. TRAIN_LABELS_PATH = './vinbigdata/labels/train' VAL_LABELS_PATH = './vinbigdata/labels/val' TRAIN_IMAGES_PATH = './vinbigdata/images/train' #12000 VAL_IMAGES_PATH = './vinbigdata/images/val' #3000 External_DIR = '../input/vinbigdata-512-image-dataset/vinbigdata/train' # 15000 os.makedirs(TRAIN_LABELS_PATH, exist_ok = True) os.makedirs(VAL_LABELS_PATH, exist_ok = True) os.makedirs(TRAIN_IMAGES_PATH, exist_ok = True) os.makedirs(VAL_IMAGES_PATH, exist_ok = True) size = 51 Step 3: Importing and Reading the Textual Dataset Here we will import and read the textual data set. This data is stored as rows and columns in a CSV file format. df = pd.read_csv('../input/vinbigdata-512-image-dataset/vinbigdata/train.csv') df.head() Note: the df.head() function prints the first 5 rows of the given data set. Step 4: Filtering and Cleaning the Data Set As no data set is perfect, most of the time, a filtering process is necessary to optimize a data set, thus optimizing our model’s performance. In this step, we would drop any row with a class id that is equal to 14. This class id stands for a no finding in the disease class. The reason we dropped this class is that it may confuse our model. Moreover, it will slow it down because our data set will be slightly bigger. df = df[df.class_id!=14].reset_index(drop = True) Step 5: Calculating the Coordinates of the Bounding Box for YOLO As mentioned previously in the 'How does the YOLO algorithm work section' (particularly steps 1 and 2), the YOLO algorithm expects the dataset to be in a certain format. Here we will be going through the dataframe and applying a few transformations. The end goal of the below code is to calculate the new x-mid, y-mid, width, and height dimensions for each data point. df['x_min'] = df.apply(lambda row: (row.x_min)/row.width, axis = 1)*float(size) df['y_min'] = df.apply(lambda row: (row.y_min)/row.height, axis = 1)*float(size) df['x_max'] = df.apply(lambda row: (row.x_max)/row.width, axis =1)*float(size) df['y_max'] = df.apply(lambda row: (row.y_max)/row.height, axis =1)*float(size) df['x_mid'] = df.apply(lambda row: (row.x_max+row.x_min)/2, axis =1) df['y_mid'] = df.apply(lambda row: (row.y_max+row.y_min)/2, axis =1) df['w'] = df.apply(lambda row: (row.x_max-row.x_min), axis =1) df['h'] = df.apply(lambda row: (row.y_max-row.y_min), axis =1) df['x_mid'] /= float(size) df['y_mid'] /= float(size) df['w'] /= float(size) df['h'] /= float(size) Step 6: Changing the Provided Data Format In this part of the code, we will change the given data format of all rows in the dataset into the following columns; <class> <x_center> <y_center> <width> <height>. This is necessary since the YOLOv5 algorithm can only read the data in this specific format. # <class> <x_center> <y_center> <width> <height> def preproccess_data(df, labels_path, images_path): for column, row in tqdm(df.iterrows(), total=len(df)): attributes = row[['class_id','x_mid','y_mid','w','h']].values attributes = np.array(attributes) np.savetxt(os.path.join(labels_path, f"{row['image_id']}.txt"), [attributes], fmt = ['%d', '%f', '%f', '%f', '%f']) shutil.copy(os.path.join('/kaggle/input/vinbigdata-512-image-dataset/vinbigdata/train', f"{row['image_id']}.png"),images_path) We will then run the preproccess_data function two times, once with the training data set and its images and the second with the testing data set and its images. preproccess_data(df, TRAIN_LABELS_PATH, TRAIN_IMAGES_PATH) preproccess_data(val_df, VAL_LABELS_PATH, VAL_IMAGES_PATH) Using the line below, we will clone the YOLOv5 algorithm into our model. !git clone https://github.com/ultralytics/yolov5.git Step 7: Defining our Model’s Classes Here we will define the available 14 chest diseases in our models as classes. These are the actual diseases that can be identified in the data set’s images. classes = [ 'Aortic enlargement', 'Atelectasis', 'Calcification', 'Cardiomegaly', 'Consolidation', 'ILD', 'Infiltration', 'Lung Opacity', 'Nodule/Mass', 'Other lesion', 'Pleural effusion', 'Pleural thickening', 'Pneumothorax', 'Pulmonary fibrosis'] data = dict( train = '../vinbigdata/images/train', val = '../vinbigdata/images/val', nc = 14, names = classes ) with open('./yolov5/vinbigdata.yaml', 'w') as outfile: yaml.dump(data, outfile, default_flow_style=False) f = open('./yolov5/vinbigdata.yaml', 'r') print('\nyaml:') print(f.read()) Step 8: Training the Model To start, we will open the YOLOv5 directory. Then we will use pip in order to install all the libraries written inside the requirements file. The requirements file contains all the required libraries that the code base needs to work. We will also install other libraries such as pycocotools, seaborn, and pandas. %cd ./yolov5 !pip install -U -r requirements.txt !pip install pycocotools>=2.0 seaborn>=0.11.0 pandas thop clear_output() Wandb, short for weights and biases, allows us to monitor a given neural network model. # b39dd18eed49a73a53fccd7b684ea7ecaed75b08 wandb.login() Now we will train the YOLOv5 on the vinbigdata set provided for 100 epochs. We’ll also pass some other flags, such as --img 512, which tells the model that our image size is 512 pixels, --batch 16 will allow our model to take 16 images per batch. Using the --data ./vinbigdata.yaml flag, we will pass our dataset, which is the vinbigdata.yaml data set. !python train.py --img 512 --batch 16 --epochs 100 --data ./vinbigdata.yaml --cfg models/yolov5x.yaml --weights yolov5x.pt --cache --name vin Step 9: Evaluating the Model First, we will identify the testing data set directory along with the weights directory. test_dir = f'/kaggle/input/vinbigdata-{size}-image-dataset/vinbigdata/test' weights_dir = './runs/train/vin3/weights/best.pt' os.listdir('./runs/train/vin3/weights') In this part, we will use the detect.py as our inference to check the accuracy of our predictions. We will also pass some flags, such as --conf 0.15\, which is the model's confidence threshold. If the confidence rate of a detected object is under 15 percent, remove it from our output. The --iou 0.4\ flag informs our model that if the intersection over the union of two boxes is below 40 percent, it should be removed. !python detect.py --weights $weights_dir\ --img 512\ --conf 0.15\ --iou 0.4\ --source $test_dir\ --save-txt --save-conf --exist-ok Final Thoughts on Using YOLOv5 in PyTorch In this article, we explained what YOLOv5 is and how the basic YOLO algorithm works. Next, we went on to briefly explain PyTorch. Then we covered a couple of reasons why you should use YOLO over other similar detection algorithms. Finally, we walked you through a machine-learning model that is capable of detecting chest diseases in x-ray images. In this example, we used YOLO as our main detection algorithm to find and locate chest lesions. We then classified each lesion into a given class or disease. If you are interested in machine learning and building your own models, especially models that require the detection of multiple objects in a given image or video representation, then YOLOv5 is definitely worth a try.