In a traditional software project’s development, there are some involved mandatory activities that should be done during the software development life cycle (SDLC). These activities are time-consuming and cost wasters. They are either done manually, or in basic approaches by different roles. Some of these activities are: version control, project structuring, development environment preparation, software testing, building, packaging, and deployment, along with 3rd party library and API dependency management. In this article, we will discuss the rationale about these activities, open source, mature, and free tools that are already applied in the Java industry. Moreover, we will discuss the current limitations of these tools towards reducing the cost and time of building high-quality software solutions.
Who Is This Article For?
This article is for beginning to intermediate level technical people seeking to understnd the full picture of continuous delivery.
In any software project, there are a lot of assets that should be managed, maintained and tracked, such as documentation, diagrams, and most importantly, source code. This process of managing and maintaining these assets is known as Version Control (SVC), which is part of the Software Configuration Management (SCM).
SVC is not a luxury in the software development industry anymore. The unavailability of SVC may cause a big direct impact on the business, affecting a software project’s success. This includes the ability to provide an effective way of asset sharing, the ability to recover deleted resources, resolving conflicts between resources, logical copying for tagging and branching, and others.
On the other hand, and based on the “Don’t re-invent the wheel” concept, the usage of 3rd party libraries and API’s has become an important factor to reduce the cost. However, managing these libraries and versions has become a big challenge, which resulted in the concept of dependency management.
In property programming languages, like Microsoft's .NET platform, creating new projects has never been an issue, since Visual Studio — the official Integrated Development Environment (IDE) for Microsoft — has been the only available option for developing projects in .NET and any other Microsoft technology projects.
This, however, is not the case in the open source community with a big list of IDE options for developing software applications, especially for Java (e.g. Eclipse, NetBeans, IntelliJ and JDeveloper). This has become a real challenge in terms of migrating projects between the long list of available IDEs, which has become a show-stopper since it either enforces a project’s instability caused by the usage of different IDEs, or lowers the productivity of developers by enforcing a unified IDE based on the company’s policy.
In traditional software projects and in the SDLC, testing is the phase that ensures the quality of the software. Testing can be done manually by either applying written test-scenarios, or based on the knowledge of the Business Analyst (B.A.) or domain expert. The manual approach, however, will not be practical in large and long-term projects, which makes the automated test cases a better approach. Automating the test scenarios as software components will validate the functionality against the requirements in a more granular approach, which may be run in agile methodologies as task based, be part of a full integrated test for the whole module or system, or both. Although the automated test cases will make it easier to test the system, it will take lots of effort and time to write and maintain. This, in turn, will increase the software cost in the short-term.
After the finalization of software development, software package should be prepared, deployed and installed either for testing or final usage by the end-user. The packing, deployment, and installation may be done in many ways. It may be done manually by collecting all the required resources using script tools (e.g. Apache Ant) or using proprietary features in IDEs. These methods have a limitation of the availability of an expert in the specific field, along with non-portability between tools, IDEs, and environments.
II. Software Industry Best Practices and Proposed Solutions
In the software industry and open source community, a lot of solutions, tools, and frameworks have been created to automate a lot of the activities and tasks required for software projects. In this section, we will explore open source tools that are widely adopted in the industry, as well as discuss their features and limitations.
A) Version Control
Version control (VC) is mainly concerned with managing project assets, especially source code. There are mainly two classifications for available VC tools and software: Centralized (CVC) and De-centralized (DCVC). Choosing one of these approaches depends on many factors, including project scale, team geographical location, team expertise, and agility.
Currently, the most popular and available open source tool for CVC is Subversion, and for DCVC is Git.
In small-scale projects with relatively close geographical areas and low levels of expertise, CVC is the preferred way, since it is easier to manage and will reduce the chance of conflicts. However, CVC requires centralized senior expertise to insure stability through continuous code review processes. In addition, the software development policy in this case should ensure that only low-level and short-term tasks are assigned to developers by an experienced architect or software designer.
DCVC, On the other hand, depends on the situation where developers have higher maturity and expertise to handle more higher-level and long term tasks. It has the ability to take architectural and design decisions. Moreover, DCVC should insure the standardization of coding and naming conversions, as well as design and architecture decisions. This leads to an enhanced consistency of the project. Failing to do so may increase the cost of development.
B) Repository and Library managamanet
The usage of 3rd party libraries, APIs, and frameworks in software projects (artifacts) has become a key part of increasing the productivity of developers to deliver software projects successfully and on time. However, manual maintenance of these artifacts is not an easy task between different versions of the project or between the different environments of development, testing, and production. Currently, there are many open source tools that can be used to cover this need. The most popular is Sonatype's Nexus, which is used in the largest open source central repository: Maven, with more than 1,334,000 artifacts available. In many cases, however, there is a need to upload local or 3rd party artifacts on the organizational level either for privacy, non-availability on central repository, or to reduce network traffic and resources.
C) Software Project Structure
Building the software project structure has become a very important task as well. It should be done very carefully to ensure smooth compatibility and migration between different IDEs. One of the most popular and mature available options for project structure standardization and organization is Apache Maven. Maven has been designed and built around the concept of the project object model, where all the technical aspects of the project are set in specific configuration files to enable smooth IDE integration along with easy packaging, dependency management, and lookup.
D) Test Cases
To increase the quality of software, test driven development approaches have been used for a while to automate the unit and integration tests process. Apache Junit is one of the more mature available open source tools, which is wildly used in different levels of projects.
The cost of writing the test case, however, is relatively high, especially with large projects. We believe that a dynamic approach of creating test cases has become mandatory in order to increase productivity and quality, while also reducing the cost.
E) Packaging, Build, and Deployment
The process of making the final version of the application includes many steps. It starts with getting the required version of your source code from version control, followed by compiling and packaging the application, and applying the automated test cases to ensure software integrity. All these actions and processes are defined as Continuous Integration (CI). After the package is prepared, it is either published to the dependency repository manager, directly deployed on the designated environment, or both. The whole process, including CI, is called Continuous Delivery (CD). The above process will help insuring that high-quality and consistent software package is deployed and applied. However, it will not be an easy task without a tool that automates all of the above processes.
Jenkins is one of the popular open-source tools available nowadays for CI. It is relatively easy to configure and have a big set of plugins to integrate with most of the necessary tools (e.g. SVN, Git, Maven, etc.). Furthermore, Jenkins can be used to deploy changes to production or test environments using different types of plugins (e.g. deployments to JBoss using JBoss CLI commands triggered after a successful build process).
In this article, we talked about the steps, tools, and rationale about various Continuous Delivery tools and practices. We do believe that there is a good space for improvement, including building dynamic test cases especially for medium-to-large and long-term software systems, as well as standardizing the continuous integration of different processes to make tools interchangeable without affecting the processes.
I would first like to thank my dear friend Dr. Muhanna Muhanna, Assistant President for Accreditation and QA, and Assistant Professor at Princess Sumaya University for technology for his continuous support, auditing and review.
Oracle certified Expert, Java Server Faces
Sun Certified Java Developer
Sun Certified Java Programmer
Sun Certified Java Web Component Developer
Sun Certified Java Business Components Developer
Sun Certified Java Associate
Masters in Science, Enterprise Systems Engineering