Information about Kafka Connect sourced from Spark Summit East 2016.

The open source NGINX v. 1.9.11 is out, and features an awesome addition: dynamic modules. These let you load separate shared object files at runtime. Learn more about the latest open source NGINX 1.9.11 release!

A presentation and slide deck on using several different tools including Graylog Splunk, and TIBCO to analyze log data.

How the engineering team at SignalFX came to use CircleCI when listing their DevOps requirements.

One thing almost universal to programmers? We're lazy. Rather we're resourceful, and don't want to do the same thing twice. Here's how to implement the Lazy class.

A recap from a recent Java survey to determine what kinds of big data tools are popular amongst Java developers.

The basics of messaging services for Java apps, along with a client example based on ActiveMQ.

This just in, Wordpress is moving to Node.js. Check out what Node.js is, and why you'd want to move from PHP to Node.js.

Docker is an open platform for orchestrating distributed applications.; open source NGINX (and the enhanced and commercially supported version, NGINX Plus) are great use cases for Docker, and this article describes how you can deploy the open source product.

Game developer Nikolaus Gebhardt talks open source game engines, challenges of creating an engine, and working with open source projects.

Hear the case from CUBA why YARG is a solid entry into the already large field of report generators.

Go is awesome and you should learn it. Here's how.

Smart farming is a concept quickly catching on in the agricultural business. Offering high-precision crop control, useful data collection, and automated farming techniques, there are clearly many advantages a networked farm has to offer.

How to create a lwIP project, which is an open source TCP/IP for small systems, using the Kinetis SDK and FreeRTOS on the FRDM-K64F board.

On the heels of the Kinesis Firehose announcement, more people are going to be looking to integrate Kinesis with logging systems. Here is one take on solving that problem that integrates syslog-ng with Kinesis.

What’s good about online schedulers is that you can get an access to your data from any of your devices. Your schedule will be safe if something happens to your laptop or smartphone. But the overall quantity of available schedulers can confuse you. There are JavaScript and .NET scheduling tools. There are third-party plug-ins for big frameworks and built-in parts of these frameworks. And finally, there are stand-alone tools which can be integrated with other components. This article’s aim is to take a look at the four most popular .NET scheduling components to make clear which of them is worthy of attention and good for beginners. Here are the scheduling tools: DayPilot ASP.NET Event Calendar DHTMLX Scheduler .NET Devexpress ASP.NET Scheduler Telerik .NET Scheduler We’ll use Microsoft Visual Studio 2013 to create demo applications. We will check how the basic calendar looks like and behaves. The other important thing is documentation. Good structured documentation pages with understandable step-by-step guides will be a plus. DayPilot ASP.NET Event Calendar DayPilot provides you with different kinds of time management tools: ASP.NET Event Calendar is an event calendar that allows you to plan your day hour by hour: ASP.NET Monthly Calendar will help you create a month schedule: ASP.NET Scheduler is useful if you want to group you events into different categories: ASP.NET Gantt Chart for those who want to build a Gantt Chart: Since Scheduler and Gantt Chart are more specialized tools we’ll take a look at something more commonly used. Let’s say, the Event Calendar. Basic features: Integrated message bar Drag-and-drop Export. Available file formats are BMP, PNG, GIF and JPG Customizable appearance The full list of features is available on this page. If you want to check what you can get from it, check this demo page. The tutorial page is a little bit messy at first sight. There are pages for different kinds of features for all of the DayPilot components. However, it’s not too hard to find the page you want. Which is, in our case, this basic calendar tutorial page. It’s a step-by-step guide which shows how to create a project in Visual Studio, create a database and attach it to your project, etc. It’s a little bit hard to get started with it if you haven’t used Visual Studio before. If you’re lazy enough, you can simply download the source code for this example here. After the job is done, you can check the results: You can’t do much with this simple event calendar. This testing event can be dragged through the calendar grid to change its beginning date. It was created during the calendar creation and by default, there’s no possibility to create a new one. To add this functionality you should spend some more time with tutorials. And you can’t change the scale of the calendar. Therefore, if you want to get a possibility to create long term events as well, you should use both Event Calendar and Monthly Calendar in your project. But here’s some good news. There is an open-source version of Event Calendar available. The cost of DayPilot calendar may vary from $499 to $4999 depending on the license and product version. Well, let’s move along. DHTMLX Scheduler .NET DHTMLX Scheduler .NET is another stand-alone tool that was made to help you create your own online scheduler. Check this demo page to see how it looks like. Basic features: Different views: Decade view, Grid view, Year view Recurring events Drag-and-drop Touchscreen support Easy customizable Integration with Google Maps and Google Calendar You can check all its features on this page. There is more than one possible way to get this scheduler. You can download the package with required files or install them via NuGet or Package Manager Console. This install guide can help you choose the proper way. Let’s talk about documentation. Since DHTMLX Scheduler .NET is basically a .NET wrapper for this JavaScript scheduler, the documentation consists of two parts: server side and client side articles. There are basic calendar creation guides and a bunch of configuration guides. Moreover, there is a video tutorial available. But what we’re interested in right now is how we can create a basic calendar. The developer’s blog contains this tutorial explaining how to build a basic calendar in ASP.NET MVC5. It’s quite detailed and contains a bunch of screenshots. And here’s what we got: This time we can change the scale. There are “Day”, “Week” and “Month” buttons available. The current day is highlighted and there is a “Today” button that will help you in case you lost yourself within the calendar grid. There are several ways of how you can create a new task. You can use drag-and-drop with this calendar. So, select the time period you want and the event property window will appear on your screen: You can also use right click to get access to this window, but in this case you’ll have to set the time period by yourself. Scale changing feature is useful if you want to create a long term event: You can drag the event to change its start date. This is all that you can get by default. The important thing is that, unlike the DayPilot tool, this scheduler can be used to show days, weeks and months. This basic app, as you can see, gives you a possibility to add and change the tasks. Documentation structure is easy to understand, so it’s not too hard to use it to improve your application. The cost of DHTMLX Scheduler .NET is $299 for Commercial License and $979 for Enterprise license. Devexpress ASP.NET Scheduler DevExpress provides you with huge amount of different tools. There are WinForms, iOS, ASP.NET components for desktop and online applications. Very detailed subscription of Devexpress scheduler is available on this page. You can get the installation file on this download page. There’s no possibility to install the scheduler separately from other .NET and MVC components. There’s a lot of documentation. Tons of it. For every single component. The scheduler documentation page is separated into several parts: “Start Here”, Learn More”, “How To” and “Reference”. Each of them consists of chapters that contain a bunch of articles on a specific topic. Everything that you need to know about the every single tool. But if you want to create a basic scheduler fast, you can check two video tutorials available on YouTube: The Project Wizard which will tell you how to create a Devexpress project and Getting Started with the ASPxScheduler which is about the scheduler creation process. This 14 minutes is all that you need to create your first online application. The result is shown below: As you can see, there are a couple of scales available: “Day”, “Work Week”, “Full Week” and “Month”. Moreover, there is a Timeline mode, which can help you trace the current week events. A navigation panel on the right side will help you switch between month and years. Drag-and-drop behavior is available, so you can select the proper area of the calendar grid to create a new event. After the right-click you can choose what you want to do: The properties window will appear. It allows you to set the basic properties (Start time, End time, Subject, etc.) and a couple of additional ones (Label, for example): According to these properties, different events will look differently, which is pretty handy. Here’s the Timeline mode as an example: The cost of Devexpress scheduer is $899.99 for the .NET bundle. Telerik .NET Scheduler Just like the previous developer, Telerik provides you with a wide variety of tools: a mobile application platform for iOS, Android and Windows Mobile; HTML5 and JavaScript widgets for web applications; .NET components for desktop, mobile and online applications and much more. But we’re here for one reason. RadScheduler page contains a detailed list of features with a demo page for each of them. And, like in case of Devexpress, this time we can’t download the scheduler separately. We should get the whole .NET bundle. The documentation page contains comprehensive information about available components. There are tutorials, API documentation, videos, infographics and other useful stuff. It may be a little bit complicated to define where to start from. But there is a YouTube video tutorial that allows you to dive straight into the action instead of reading tons of docs. First 13 minutes are enough to get required basic information about how the things work. As a result you will get something like this: There is something familiar with this one. We can choose between “Day”, “Week”, “Month” and “Timeline” modes. The last one works pretty much the same as the Devexpress Timeline option. This time we have a mini calendar which is useful for navigation. There’s also a “Show full/business hours” toggle button that allows you to separate your work events from the private ones: New event creation is intuitive and similar to other calendars. This time there is a wide variety of properties that allows you to configure the recurrent events. And, as usual, drag-and-drop is available. The cost of Telerik scheduler is $999 for the .NET bundle. Conclusion Described schedulers can be divided into two groups. The first two of them are stand-alone tools. In case of DayPilot there’s no possibility to get the calendar you like. The only possible way to get it is to buy the whole bundle. DHTMLX Scheduler .NET is the only component that can be bought separately from any other tool, which is great if you’re not interested in additional components. The remaining two components are the part of big .NET bundles. And maybe it’s not a good idea to buy them if you want to create a simple online calendar. DayPilot and DHTMLX Scheduler .NET DayPilot components look great. The documentation contains detailed tutorials. The bad thing is that the basic scheduler which was built according to the beginner’s guide is kinda low functional. To make something really useful you should spend some more time with it. There’s also no possibility to change the scale of a calendar, so you have to chose either Event Calendar or Monthly Calendar to plan your events properly. The good news is that, besides the paid components, there is an open-source version available. You can get it here. DHTMLX Scheduler .NET is more attractive, especially for beginners. There are step-by-step guides of different levels of complexity and a video tutorial. The basic app looks pretty useful. It allows you to change the calendar scale, add and configure the tasks. There are Commercial and Enterprise versions available on this page. You can also get the 30-days trial version. As an alternative, you can try the JavaScript version of this scheduler. You have two options with this one: commercial Pro Edition and open-source Standard edition. Devexpress ASP.NET Scheduler and Telerik .NET Scheduler Those two are the components of big and advanced sets of tools. They’re integrated with Visual Studio. As a result, there’s a project wizard for both of them that allows you to configure your application properties. The documentation contains dozens of pages and it may confuse you at the first glance. But you can find an article about the every single property if you spend some time with it. There are video tutorials available as well, so everyone can find the most convenient way of learning. The Devexpress documentation structure looks more comfortable. Every big theme is separated into several parts which are grouped according to the difficulty level. The basic scheduler is pretty functional. For example, there is a possibility to set a proper label for a task. There are a lot of possible combinations of tools that you can buy. But it’s always good to try some before to make a decision and trial versions will help you with that. Check the details on this page. Telerik documentation is not so well grouped which can confuse the beginners. Scheduler creation is intuitive and the basic app is pretty usable. Just like in the previous case, Telerik provides you with a wide variety of possible bundles of tools. And, once again, you can try it before buy it. Trial versions are waiting for you.

[This article was written by Juergen Hoeller.] Spring is well-known to actively support the latest versions of common open source projects out there, e.g. Hibernate and Jackson but also common server engines such as Tomcat and Jetty. We usually do this in a backwards-compatible fashion, supporting older versions at the same time - either through reflective adaptation or through separate support packages. This allows for applications to selectively decide about upgrades, e.g. upgrading to the latest Spring and Jackson versions while preserving an existing Hibernate 3 investment. With the upcoming Spring Framework 4.2, we are taking the opportunity to support quite a list of new open source project versions, including some rather major ones: Hibernate ORM 5.0 Hibernate Validator 5.2 Undertow 1.2 / WildFly 9 Jackson 2.6 Jetty 9.3 Reactor 2.0 SockJS 1.0 final Moneta 1.0 (the JSR-354 Money & Currency reference implementation) While early support for the above is shipping in the Spring Framework 4.2 RCs already, the ultimate point that we’re working towards is of course 4.2 GA - scheduled for July 15th. At this point, we’re eagerly waiting for Hibernate ORM 5.0 and Hibernate Validator 5.2 to GA (both of them are currently at RC1), as well as WildFly 9.0 (currently at RC2) and Jackson 2.6 (currently at RC3). Tight timing… By our own 4.2 GA on July 15th, we’ll keep supporting the latest release candidates, rolling any remaining GA support into our 4.2.1 if necessary. If you’d like to give some of those current release candidates a try with Spring, let us know how it goes. Now is a perfect time for such feedback towards Spring Framework 4.2 GA! P.S.: Note that you may of course keep using e.g. Hibernate ORM 3.6+ and Hibernate Validator 4.3+ even with Spring Framework 4.2. A migration to Hibernate ORM 5.0 in particular is likely to affect quite a bit of your setup, so we only recommend it in a major revision of your application, whereas Spring Framework 4.2 itself is designed as a straightforward upgrade path with no impact on existing code and therefore immediately recommended to all users.

Rework is Hell “Software may be eating the world, but rework is choking software”, tweeted John Jeremiah (@j_jeremiah). To shed more light on what is choking software, new data was released last week in the 2015 State of the Software Supply Chain Report. In its discussion of application quality and integrity, the report revealed that the average application includes 106 open source components. It is clear that the use of these components has benefited development tremendously in helping to speed time to market and improve innovation. While the benefits are undeniable, development teams are also delivering applications that are “insecure by design”. Of the 106 components per application, the report’s analysis revealed an average of 24 (i.e., 23%) have known critical or severe security vulnerabilities. Those same apps also showed an average of 9 restrictive license types (e.g., GPL, AGPL, LGPL). By electively sourcing components with known vulnerabilities and potential license risks, software development teams are building up higher levels of technical and security debt. And in order to reduce some of that debt, unplanned and unscheduled rework is often required. No Developer Intentionally Uses Bad Parts I have never met a developer that intentionally wanted to use “bad” components in their applications. That said, I have also never met a developer that wanted to spend four hours researching the latest versions, open source licenses and known security vulnerabilities for components (including dependencies) they wanted to use in their applications. While 57% of development organizations have internal policies in place that direct “thou shall not use components with poor quality and integrity”, many have stated that the policies are not followed or are simply not enforced. The problem is one of volume and velocity, while people clutch to old processes that simply can’t keep pace. The analysis in the 2015 State of the Software Supply Chain report revealed that billions of open source downloads occurred in 2014, of which 6.2% had known vulnerabilities (1 in 16 downloads). For some large companies, the volume of downloads exceeded 240,000 — where 15,000 (7.5%) had known security vulnerabilities. At these volumes, manual processes and paper-based policies to prevent poor quality or risky components from getting into your software have no possibility of keeping pace. While software development practices aim for higher velocity, through component-based development as well as Agile, continuous and DevOps practices, the processes to keep up with it have not changed enough. The only way to keep pace with this velocity is through automation. And if we want to avoid unplanned rework associated with remediating known security vulnerabilities or selecting alternative components with approved license types, automation has to become the norm rather than the exception within development. At Speed “If I look at the mass, I will never act. If I look at the one, I will.” — Mother Teresa Looking at the massive volume and velocity of open source and other artifact consumption can be discouraging and overwhelming. However, the volume and velocity within our software supply chains will not diminish—and without a new approach, the volume of unchecked quality and integrity of parts being consumed will continue to build up as technical debt. Automation in areas of testing, build, and deployment has provided significant performance benefits. Likewise, investments in software supply chain automation have shown markedly improved efficiency and controlled risk, as the best practices in the 2015 Report illustrate. Automation can unleash the potential of an organization’s development capacity. Rarely is there such an opportunity to simultaneously increase speed, efficiency, and quality. In order to improve the quality and integrity of components used in our applications, we need to release our clutch on old practices. We can further embrace automation across our software supply chains — and improve the quality of our applications — by considering these three steps: 1. Start with creating a software bill of materials for one application. Visibility into one application can help you better understand your current component usage. A number of free and paid services are available to help you create a software bill of materials within a few minutes. The bill of materials will help you to identify the unique component parts used within your application and the suppliers who contributed them. These reports list all components used, and several services also identify component age, popularity, version numbers, licenses, and known vulnerabilities. 2. Design approval processes to be frictionless, scalable, and automated. Manual reviews of components and suppliers cannot keep pace with the current volume and velocity of consumption. Your organization must not only define your policies for supplier and part selection but also find practical ways to enforce them without slowing down development or inadvertently encouraging workarounds. Policies must be agile enough to keep pace with modern development. Strive to automate policy enforcement and minimize drag on developers. 3. Enable developer decision support. Developers are primary consumers of components in software supply chains. They initiate every component request. Help developers by automating the availability of information on component versions, age, popularity, licenses, and known vulnerabilities within their existing development tools so it is easy to pick the best components from the start. By selecting the highest-quality components from the highest-quality suppliers early, you will improve developer productivity and reduce costs. Here is the complete 2015 State of the Software Supply Chain Report. I will also be discussing more of the Report findings and best practices on a webinar scheduled for Wednesday, June 24, 12pm ET (register or view it on-demand). The previous blog in this series was, Fewer and Better Suppliers. Next up, I will be reviewing Visibility and Traceability practices across the software supply chain with more commentary from the 2015 State of the Software Supply Chain Report.

That Supplier is Better For You Since releasing the 2015 State of the Software Supply Chain Report, there has been a lot of great discussion across the industry on best practices for managing the complexity introduced by the volume and velocity of the components used across your software supply chain. Today I want to focus on the huge ecosystem of open source projects (“suppliers”) that feed a steady stream of innovative components into our software supply chains. In the Java ecosystem alone, there are now over 108,000 suppliers of open source components. Across all component types available to developers (e.g., RubyGems, NuGet, npm, Bower, PyPI, etc.), estimates now reach over 650,000 suppliers of open source projects. Source: ModuleCounts.com However, like in traditional manufacturing, not all suppliers deliver parts of comparable quality and integrity. My latest research, the 2015 State of the Software Supply Chain Report, shows that some open source projects use restrictive licenses and vulnerable sub-components, while other projects are far more diligent at updating the overall quality of their components. Choosing the best and fewest suppliers can improve the quality and integrity of the applications we deliver to our customers. While I am hosting a webinar next week to share many of the detailed report findings, I wanted to share a few of the more meaningful stats here. Your 7,600 Suppliers My research for the report revealed many new perspectives on “suppliers” across the software supply chains. First of all, I saw that the average large development organization consumed over 240,000 open source components last year — sourced from over 7,600 open source projects. Average number of open source component downloads per organization in calendar 2014 On the surface, the huge reliance on open source projects is a great thing. Development teams have chosen to not write those pieces themselves, but have sourced the needed components from outside suppliers. This practice speeds development, enables more innovation, and ensures time-to-release goals are achieved. The use of open source is so prolific today, few of us could ever imagine reducing the use of those components and their suppliers in the future. At the same time that we benefit from open source, our high paced, high volume consumption practices don’t allow us the time needed to do the due diligence on the suppliers or open source projects where we source our component parts from. Average number of known vulnerabilities downloaded by organization in calendar 2014. For example, of the 240,000 average component downloads in 2014, the same businesses sourced an average of 15,000 components that included known security vulnerabilities. In many cases, developers were downloading vulnerable component versions, when safer versions of those same components were available from the open source projects. While no one intends to download components with known vulnerabilities, the problem is exacerbated due to the lack the visibility into a better recommended version. Fewer Suppliers, Less Context Switching Choosing an open source project supplier should be considered an important strategic decision in organizations because changing a supplier (“open source project”) used is far more effort than swapping out a specific component. Like traditional suppliers, open source projects have good and bad practices impacting the overall quality of their component parts. Traditional manufacturing supply chains intentionally select specific parts from approved suppliers. They also rely on formalized sourcing and procurement practices. This practice also focuses the organization on using the best and fewest suppliers — an effort that improves quality, reduces context switching, and also accelerates mean time to repair when defects are discovered. One industry example from the report describes how Toyota manages 125 suppliers for their Prius to help sustain competitive advantages over GM who manages over 800 suppliers for the Chevy Volt. By contrast, development teams working with software supply chains often rely on an unchecked variety of supply, where each developer or development team can make their own sourcing and procurement decisions. The effort of managing over 7,600 suppliers introduces a drag on development and is contrary to their need to develop faster as part of agile, continuous delivery and devops practices. Coming to Terms When you come to terms with the volume of consumption and the massive ecosystem of suppliers you can source your components from, you quickly realize it is impossible to address this issue with a manual review process. Any organizations clutching to these outdated manual practices are will continue to be outgunned by the velocity by their software supply chains. Just as traditional manufacturing supply chains have turned to automation, software development teams need to take the same approach by further automating their software supply chains. Information about suppliers and the quality of their projects needs to be made available to developers at the time they are selecting components. Information about the latest versions, features, licenses, known vulnerabilities, popularity of versions being used, and the cadence of new releases should be made available to developers in an automated way. Automating the availability of this information about suppliers can lead to better and fewer suppliers being used. Be sure to read the full 2015 State of the Software Supply Chain Report for more information about open source suppliers and organizations sourcing practices. The report also highlights current and best practices being used in organizations that are managing their use of suppliers that feed their software supply chains.

The use of OpenStack in the private cloud is invaluable for DevOps. It provides engineers the ability to innovate quickly and deal with uncertainty. It also maximizes existing infrastructure and provides a programmable, software-defined IaC. Openstack in the private cloud = agile development OpenStack has emerged as the de facto standard for IaaS in the private cloud. It gives engineers a vital self-service capability to provision (and de-provision) environments, allowing them to act autonomously, in the moment. This helps to eliminate the downstream bottleneck caused by waiting for operations staff to find time to do the provisioning. As OpenStack is open source it is vendor agnostic, allowing you to take advantage of competitive pricing rather than suffering from vendor lock-in. A private cloud means lower cost for the same capacity in a public cloud, which is especially useful for enterprises with high data needs. For security reasons, OpenStack is still mainly used in the private cloud by developers and QA, i.e. in a non-production context. However, OpenStack gives an ability to optimize application performance and/or security by having more control compared to public cloud. The software is increasingly backed by the critical mass of leading IT infrastructure vendors such as IBM, CICSO and HP. Gartner assumes that “by 2019, OpenStack enterprise deployments will grow tenfold, up from just hundreds of production deployments today, due to increased maturity and growing ecosystem support.”1 Challenges to consider OpenStack implementation skills are still rare in the market, so experimentation and self-learning is necessary. Although this takes time, it is offset by the fact the software is free and represents a good opportunity to gain internal expertise. This is particularly valid if you class infrastructure as a core competence. The maturity and functionality of OpenStack projects vary widely - while it covers storage, network and compute, the main adoption currently happens around compute (Nova) and block storage (Cinder), with object storage and network (Neutron) lacking significantly behind. However, without leveraging virtualized network services as part of a private cloud, full-stack environment provisioning is not possible, so don’t forget to add necessary network services to your private cloud. Where to begin Integrating OpenStack clouds with existing infrastructure can be a challenge. It is hardly plug and play. At first, it is best to focus on relatively isolated DevOps environments, such as Gartner’s “mode two”2 applications rather than introducing open stack across the board straight away, (Bimodal IT “refers to having two modes of IT, each designed to develop and deliver information – and technology – intensive services in its own way. Mode 1 is traditional, emphasizing scalability, efficiency, safety and accuracy. Mode 2 is nonsequential, emphasizing agility and speed.”3) As with any open source software, new functions and upgrades are frequently released. This means keeping up with changes in functionality and filling gaps with customizations or third-party products. Upgrades are complex and typically require planned downtime. For these reasons, we recommend choosing a hardened distribution and sticking with it. Openstack is the most complete vendor agnostic solution for storage, network and compute services. The ability for developers to instantly spin up environments at any time is invaluable for a fully agile DevOps environment, and is well worth the effort it takes to acclimatize to Openstack. 1 http://www.prnewswire.com/news-releases/suse-openstack-cloud-5-to-simplify-private-cloud-management-300048721.html 2 http://www.gartner.com/it-glossary/bimodal 3 http://www.gartner.com/it-glossary/bimodal