Don’t miss the opportunity to engage in an interactive discussion with your peers around digital transformation at MuleSoft Forum, and learn why an API-led connectivity has become IT's secret weapon. Register a seat for yourself using the following links: Gurgaon - 9th July Mumbai - 14th July Bangalore - 16th July What's in store at MuleSoft Forum: Whether you're the CIO, an Enterprise Architect, or Developer, MuleSoft Forum will provide you with a number of innovative ways to improve and transform your business. At MuleSoft Forum, you'll have the opportunity to: Learn how Mule ESB Enterprise Edition offers reliability, performance, scalability, security – all out-of-the box. See MuleSoft's connectivity platform in action in a short live demo Discover how API-Led Connectivity enables major business initiatives Attend the exclusive MuleSoft ecosystem networking reception

Survey of open source PostgreSQL users found adoption continues to rise with 55% of users deploying it for mission-critical applications Bedford, MA – June 23, 2015 – EnterpriseDB (EDB), the leading provider of enterprise-class Postgres products and database compatibility solutions, today announced the results of its “PostgreSQL Adoption Survey 2015,” a biennial survey of open source PostgreSQL users. Conducted by EnterpriseDB, the survey found PostgreSQL adoption continuing to rise, with 55% of users – up from 40% two years ago – deploying it for mission-critical applications and 77% of users are dedicating all new application deployments to PostgreSQL. These findings give voice to end users and confirm such industry indicators as increasing job listings and monthly rankings on DB-Engines that have pointed to rising interest in and demand for PostgreSQL, also called Postgres. The growing popularity of Postgres also comes as traditional software vendors suffer setbacks in the marketplace. The enterprise-class performance, security and stability of Postgres, on par with traditional database vendors for most corporate workloads, meanwhile have helped position Postgres among the solutions from the world’s largest vendors. The opportunity to transform their data center economics has helped fuel downloads of Postgres as well. End users reported cutting costs with Postgres, with 41% reporting they had first-year cost savings of 50% or more. They’re using Postgres to build web 2.0 applications using unstructured data as evidenced by the 64% of respondents who said they were working with JSON/JSONB and the 47% who said they were using Postgres for collaboration applications. “Postgres is empowering organizations to transform the economics of IT. IT can invest in the customer engagement applications that differentiate their operations from their competition instead of continuing to pay the steep and rising licensing and support fees charged by traditional database vendors,” said Marc Linster, senior vice president of products and services of EnterpriseDB. “With the expanding adoption, EnterpriseDB has experienced dramatic growth year over year, providing the software, services and support that organizations need to be successful with Postgres.” Database Migrations, Replacements The findings also support statements in a recent Gartner report that reflect the widespread acceptance of open source databases. “By 2018, more than 70% of new in-house applications will be developed on an OSDBMS, and 50% of existing commercial RDBMS instances will have been converted or will be in process,” according to the April 2015 Gartner report, The State of Open-Source RDBMs, 2015.* Among Postgres users, the survey findings show migrations are already under way with 37% reporting they had migrated applications from Oracle or Microsoft SQL Server to Postgres. Many users were still planning further migrations, with 37% of PostgreSQL users saying they will gradually replace their legacy systems with Postgres, compared to 29% who said that in the 2013 survey. Further, end users predict their deployments of Postgres will expand significantly, with 32% saying they anticipate production deployments of Postgres to increase by at least 50% over the next year. The survey, conducted by EnterpriseDB using an online tool in May 2015, queried registered users of PostgreSQL and drew 274 respondents worldwide from government organizations and companies ranging in size and industry. *The State of Open-Source RDBMs, 2015, by Donald Feinberg and Merv Adrian, published on April 21, 2015. Connect with EnterpriseDB Read the blog: http://blogs.enterprisedb.com/ Follow us on Twitter: http://www.twitter.com/enterprisedb Become a fan on Facebook: http://www.facebook.com/EnterpriseDB?ref=ts Join us on Google+: https://plus.google.com/108046988421677398468 Connect on LinkedIn: http://www.linkedin.com/company/enterprisedb

This week we're in Boston at DevNation, an awesome, young (second ever), and relatively intimate (~500 attendees) conference on anything and everything hard-core, cool-and-hot (DevOps, big data, Angular, IoT, you name it), and of course -- since the conference is organized by Red Hat -- totally open-source. So far I've had in-depth conversations with five super-amazing engineers, attended several inspiring keynotes, and chatted with one skilled developer after another. We'll transcribe the deeper interviews shortly, including some on topics totally unrelated to this post. But meanwhile I'd like to offer some thoughts inspired by the first day of the event. The general theme is: we're just beginning to get serious about separation of concerns. The metaphor that keeps popping into my head comes from the first keynote: machines have finally grown up. Imperatives: telling really unintelligent agents what to do (and then they sort of do whatever they please) It is trivial to observe that computers are incredibly stupid. Turing's fundamental paper is about how to figure out whether a theoretical computer will keep calculating the values of a function until the heat death of the universe (okay that's a slight oversimplification). The fact that Edsger Dijkstra felt the need to rail gently against all goto statements in any higher-level language than machine code suggests that, in 1968, far too many computers needed instructions about how to read the instructions that tell them what to do in the first place. Richard Feynman's famous lecture on computer heuristics is the condescension of the man who conceived quantum computing to the level of functional composition (hmmm) and file systems (double sigh). Stupid agents need to be told exactly what to do. Then they need to be told to pay attention to the exact part of the command that tells them exactly what they have been told to do (dude, just goto line 1343 already and shut up). Then they don't do what you told them (optimistically we call this an 'exception'), and then you send them into time out / set a break point and try to figure out where the idiot state muted off the rails. They stare blankly at the wall / variable / register and either do nothing or repeat another unintelligibly wrong result until you notice that your increment is (apparently meaninglessly to you) one bracket too deep. You sigh and tell them what to do again, and after a while they hit age thirty (life-years/debug-hours) and maybe do something useful with their (process-)lives. Well, maybe I'm straining the metaphor a little here, but you get the point because it cuts too close to home. We spend far too much time fixing stupid mistakes that we didn't even know we were making because -- like all actual human beings -- we assumed that the agent we commanded will use their common sense to iron out those few whiffs of, admit it, frank nonsense that our step-by-step instructions will probably always contain. So, at least, goes the imperative programming paradigm. The machine does what you tell it to; and the universe collapses onto itself before the last real number is computed. Functions: reliable, predictable adults Time to give credit where it's due: I'm really just riffing on the metaphor Venkat Subramanian offered in his highly enjoyable keynote on The Joy of Functional Programming yesterday morning His not-so-smart agents -- the 'programmed' of imperative programming -- were toddlers. Since I don't have any kids, I can't presume to understand this experience fully (although I did grow up with three younger brothers..). But the general idea is: imperative programming is tricky because, when you spell everything out super literally, it's very hard to tell exactly why what you thought should happen didn't. Venkat's talk was a whirlwind of functional concepts, from the thrill of immutability to the self-evident utility of memoization. For random (Myers-Briggs?) reasons, the object-oriented paradigm never seemed very intuitive to me -- I've gravitated towards functional style even when the problem domain wasn't actually modeled very well by functions -- but Venkat's side-by-side implementations of simple calculations in OO and functional Java showed the readability delta very clearly. Functional code is beautiful because it looks like its purpose. It tells you flat-out: here is what I do; and then it does it. But immutable functions are also beautiful because they do exactly the same thing every time. I couldn't count on my two year old brother very much at all because given a certain input I had pretty much no idea what would come out. But we all count on our grown-up collaborators to output exactly what they should, given a definite input, predictably and reliably every time. Of course, people also do more than expected -- every intervention of intelligence is an injection of creativity, not generated by the definition of the function -- but at least they do what you need them to do and no less. Containers: grown-ups with good boundaries I'm picking out just one aspect of the resurgent 'joy' of functional programming because the renaissance of containerization (another 'old' technology that is just now really taking off) is, I think, a part of the same shift toward, let's say, treating computers as adults. If functions are reliable agents, then applications in well-defined containers are self-sufficient agents who know exactly what they need from others and neither require nor demand anything more. If apps on dedicated VMs are teenagers negotiating personal boundaries by waking/booting up independently (and taking far too long -- and far too many resources -- to do so, given their meager output) -- or bubble boys, isolated in ways that are unfortunate in order to isolate in ways that are absolutely necessary -- then containerized applications are subway-riders who jam into the train without offending anyone or campers who can live anywhere with just a backpack of just the stuff they need. Of course, subway-riders and campers do more than just not-mess-up. But what's kind of neat about containers is that -- like an adult with good boundaries -- clearly defined bounds and interfaces free up the application / mind to do whatever world-changing thing the developer / human has cooked up. I'll come back to this metaphor in a later article. (Mesh networks, SDN, and ad-hoc computing are all part of the same picture, I think. Kubernetes probably is too, along with event-driven and reactive programming, the actor model, dreams of Smalltalk, and of course REST, at least of the HATEOAS flavor.) But maybe this isn't a good way to think about some of these recent sparks in devworld within a single paradigm -- and maybe my perpetual discomfort with OO is influencing me too much. What do you think?

An effective governance framework is essential for a well-managed intranet. It can be the deciding factor between a good user experience, greatly valued, and a poor user experience with little benefit. Every intranet is different depending on the size, type, and culture of the organisation it supports. However, there are some key governance principles that are common to their success. Recently I spoke at Intranatverk about this based on my book ‘Digital success or digital disaster?‘ which is a practical, experience-based approach to growing and managing a successful intranet. My slides ‘7 principles of good intranet governance’ are avilable for you to share. The alternative to governance can be chaotic anarchy. Posing risks to security and intellectual property provides an awful experience for those who still use your intranet. Where governance can start to get confusing and difficult is in how it is applied. Applying these governance principles leads to a good outcome: Know your organisation Define the scope Put people first Use all resources Compare and benchmark Do what you say you will do Keep it legal Think about how you build a house with the foundations, walls, floors, windows, doors and finally the roof. It would not make sense for you to have windows, doors, and a roof only. The same applies to your governance framework. These principles for good governance are not like a menu that you choose which items to have and leave others alone. You need to follow all of these to build a strong foundation to improve your intranet and implement your strategy. Read the introductory chapter of my new governance book to find out more. A license to share the ebook within your whole organisation is also available.

[This article was written by Kevin Casey] Welcome to This Week in Modern Software, orTWiMS, New Relic’s weekly roundup of the need-to-know news, stories, and events of interest surrounding software analytics, cloud computing, application monitoring, development methodologies, programming languages, and the myriad of other issues that influence modern software. This week, our top story goes inside President Obama’s secret team of tech geeks, 140 of them and counting: TWiMS Top Story: Inside Obama’s Stealth Startup—Fast Company What it’s about:If the President of the United States walked into the room and personally recruited you to rebuild the country’s technology infrastructure, could you turn him down? He’s serious, and that room is theRoosevelt Room in the West Wing of the White House, by the way. AsLisa Gelobtersays: “What are you going to say that?” Gelobter’s answer was “Yes”—she’s now chief digital officer for the US Department of Education, part of a 140-person-and-counting tech team that’s functioning something like an elite startup embedded inside the federal government. Its business? Only modernizing the technical infrastructure, applications, and processes of just about every federal agency. Why you should care:What was once something of a tech desert—the federal government—is beginning to draw top private-sector talent inside the Beltway. The team, led by Mikey Dickerson (who helped lead the team that rescuedHealthcare.gov) andformer US CTO Todd Park, also includes the likes of former Googler Matthew Weaver, and it hopes to hit 500 people by the end 2016, shortly before President Obama will leave office. Its challenges are immense, from tackling government bureaucracy (to test just how entrenched the suits were, Weaver requested the official title “Rogue Leader”—and he got it) to the fact that its recruiting pitch includes the phrase: “You’ll have to take a pay cut.” But its mission is both noble and necessary, and the appeal of working on major problems with enormous public impacts appears to be working. Recommended reading. Further reading: Mikey Dickerson’s 10 Tips for Dealing with Bureaucracy—New Relic Blog [Video] Airbnb Open Sources Software to Lure Talent Amid ‘Insane’ Competition—CIO Journal What it’s about:Airbnb added three new apps to its open source portfolio earlier this month, but the motivation wasn’t just trying to give employees the best business tools or contribute to the software community at large. Sure, that might have been part of the equation, but the rental booking site hopes open-sourcing some of its toolkit will help recruit the best software talent in the face of what director of engineeringMike Curtiscalls “insane” competition in the Silicon Valley labor market. Why you should care:In the software arms race, any little edge counts. Curtis tellsCIO Journalthat Airbnb will keep the proprietary stuff closely guarded, of course. But it will open source “generic” tools with wider industry use cases, such as its recently releasedAerosolvemachine-learning package and itsAirpalcloud-based data querying tool. The latter, which works with Facebook’s open sourcePrestoDB, aims to simplify SQL queries to the point where you don’t need to be a big data wonk or business intelligence guru to run it. Indeed, one in three Airbnb employees have run a query on it in the year since it launched. Airbnb has contributed a dozen open source tools on its aptly namedNerds site(gotta love that!) to date, something the company hopes both contributes to greater good but also advertises its software innovation to potential hires. Google Is Wielding Its Own Secret Weapon in the Cloud—The New York Times What it’s about:In thecutthroat competitionfor public cloud business, Google may be its own best customer testimonial. In advance of this week’sOpen Network Summit, theTimes’Bits bloglooked at Google’s plan to not only unveil cloud customers such as HTC but reveal much more than ever before about its own infrastructure. Google did just that on Wednesday, offering a look inside itsdata center networking, including its massive-capacity, lightning-fast Jupiter network. Why you should care:As major cloud players continue to zap prices with their shrink-rays, it’s increasingly clear that features and underlying platforms will distinguish one from the other when enterprise users make their pick. Google is taking a big step toward writing its own story in this regard, and the synopsis might read something like: “We’re pretty good at this stuff.” Its Jupiter fabrics deliver 1 petabit per second of bisection bandwidth, according to Google, or “enough for 100,000 servers to exchange information at 10Gb/s each, enough to read the entire scanned contents of the Library of Congress in less than 1/10th of a second.” If it sounds like a bit of bragging, well, yeah—it is. But it’s bragging with a purpose: Attracting devs who want access to the same technology without having to build it themselves.Google’s Amin Vahdat connected the dots in a blog post: “The same networks that power all of Google’s internal infrastructure and services also power Google Cloud Platform.” Move Over, Meeker: Byron Deeter’s State of the Cloud Report—Bessemer Venture Partners What it’s about:With a nod to Mary Meeker’s classicState of the Internet report,Bessemer Venture Partners’Byron Deeterchecks in with his 2015 State of the Cloud Report. Given cloud computing’s relative youth and rampant ascension, it’s no surprise the stats are staggering. Here’s one to start: Cloud revenues have increased tenfold in the last six years, from a scant $5.6 billion in 2008 to more than $56 billion in 2014. And it’s going to double again in the next four years, according to BVP’s projections, to $127.5 billion in 2018. Why you should care:Deeter’s full presentation is worth a weekend watch or read, but it’s the forward-looking slides that may be most compelling for software pros. Deeter notes both the immense risks and opportunities in cloud security, unveiling a 10-point security plan for cloud startups on slide 37. To underscore the security landscape, Deeter quotes an unnamed cloud CEO who says aDDoSattack that took down the firm’s API caused more customer churn in one day than in the rest of its history. Wow. He also addresses the exploding market for cloud services built specifically for developers including, yes, New Relic. And for mobile developers, slide 44 underscores something we’ve talked about before in this space:the real money’s in enterprise apps, and it’s still a largely untapped market. Click through thefull slide deck hereorwatch video of Deeter’s presentation here. Bandwidth: The Next Frontier of Cloud Computing—ZDnet What it’s about:Is networking the next big thing in the everything-as-a-service age? It just might be, as firms likePacnetvie to deliver networking capacity on a pay-for-what-you-use model that some industry folks say better suits cloud environments facing significant but uneven networking needs. Why you should care:As author Drew Turney notes, there’s a common blind spot when it comes to cloud computing’s many shapes and sizes: Moving all that data from points A to Z, and everywhere in between, which can cause both performance problems and undue financial pressures. The promise of Networking-as-a-Service (NaaS), industry execs tell Turney, is that it can provide more efficient, scalable networking for short-term usage bursts such as customer traffic spikes or large cloud backup-and-storage jobs, enabling companies to later dial down their capacity as needed. Combined withSoftware-Defined Networking (SDN),NaaS makes it possible to build intelligent applications that manage their own networking needs, which might be the most significant enterprise potential of NaaS, saysNuage NetworksarchitectMarten Hauville. Page Bloat: Average Web Page Now More Than 2MB—The Performance Beacon (SOASTA) What it’s about:Do you need to put your website on a diet? Apparently so: The average Web page topped 2 MB as of May 2015, according to ongoing tracking atThe Performance Beacon. That’s double the average page weight from just three years ago. The site projects average page weight will exceed 3 MB in late 2017. Why you should care:Performance, performance, performance:Slow speedsare a killerin the modern software era. While author andSOASTAUX evangelistTammy Evertsrightly notes that page weight is not the only factor in Web optimization, we’re simply not paying it enough attention when designing and building Web pages. Images are the big culprit in the Web’s expanding waistline: they comprise nearly two-thirds of the average page’s weight, and video is a growing part of our Web diet, too. But other factors such as custom fonts play a role, adding weight even as the Web sheds previous performance hogs like Flash. The ideal weight? 1 MB, she says, which will save crucial seconds in load times. Sounds like it’s time to hit the virtual treadmill.

Opsmatic Inc., a company with a focus on creating tools to improve the effectiveness of development and operations teams, announced today the expansion of its live-state monitoring service to include Enterprise and On-Premises Editions. Opsmatic officially came out of stealth last month and continues to deliver on the promise of supporting the needs of DevOps teams, whether in the cloud or inside a corporate firewall. The Opsmatic live-state monitoring service is the only solution that provides a precise, real-time picture of the detailed configuration and changes that affect an enterprise’s computing infrastructure. The new offerings include all the features of the Professional Edition (announced last month), with the addition of a single sign-on and dedicated support in the Enterprise and On-Premises Editions. The On-Premises Edition is designed for customers with isolated infrastructure who require an internally deployed solution, while leveraging the same real-time visibility to quickly troubleshoot problems and reduce costly downtime. Opsmatic services are delivered through a purpose-built, intelligent data platform which enables customers to easily integrate events from other services for greater context (PagerDuty, Nagios, Zabbix), incorporate their own custom event data (deployment events, backups, etc.); and extend their live state host data to include custom configuration and state data from their own services to provide deeper, more complete insight. Alerts can also be posted to Slack, HipChat, DataDog and PagerDuty, to better support team collaboration and communication. “Our new single sign-on capability gives the entire technical team access to a central source of truth, reducing the number of configuration-related issues while increasing team velocity,” said Jim Stoneham, CEO, Opsmatic. “Our customers have said that incident triage that used to take them hours now takes minutes.” Opsmatic live-state monitoring features include: Real-time insight The Opsmatic service monitors the state of every host in real-time, providing a current and accurate picture of the configuration, as well as an instantaneous feed of any changes happening to that host. Any variation, or “drift” in configuration across host groups is also immediately visible to enable teams to fix minor issues before they escalate into downtime events, saving hours of detective work and remediation. Infrastructure search and Assertions Live-state inventory data can be instantly searched to find vulnerable packages, to identify every version of an open-source package that is deployed, or find hosts that are running a specific service. Specific policies (“Assertions”) can be easily defined and run against live-state data to enforce dependencies, verify configuration, or instantly identify potential issues. Customers can also add internal service configuration and other types of data with a simple inject utility to provide deeper service-level checks. Configuration management monitoring Deep integrations with popular Configuration Management tools from Chef, Puppet, Ansible, SaltStack, as well as Docker, enable tracking and reporting of automation runs, file integrity monitoring, and detailed visibility into the key host attributes set by each run. In addition, Opsmatic reaches beyond the policies deployed through the CM tool to track the entire host and report on changes made outside automation runs. Intelligent alerts Using Assertions and saved searches, Opsmatic gives teams control over alert noise and the fatigue it can cause, enabling them to focus on the changes or conditions that really matter. Any alert can be classified by host or host group, and can be fed into specific chat channels (Slack, HipChat), or emailed to the right person on team to investigate the issue. Robust Software-as-a-Service The Professional and Enterprise Editions are delivered as cloud-based services, supporting any cloud infrastructure, datacenter, or hybrid environment across a range of OS platforms. The services are hosted in a hardened data center with SAS 70 Type II and SSAE 16 certifications and 24x7 monitoring. Availability Opsmatic Enterprise Edition is sold as a monthly subscription, with billing based on usage: $7 per host, per month, based on the peak number of hosts being monitored each month. The Opsmatic On-Premises Edition is sold as a yearly contract by quote at [email protected]. The company offers an unlimited-use, 30-day free trial of Opsmatic Professional that is available at www.opsmatic.com/signup. No credit card is required for the free trial. General inquiries should be directed to [email protected]. About Opsmatic Inc. Opsmatic provides real-time visibility of any change in the live state of computing infrastructure and intelligently alerts users before trouble begins. The SaaS service is built on an underlying data platform with a robust API, and is integrated with popular monitoring and code automation tools to give customers complete context and provide the shared visibility required by modern DevOps teams. Founded in 2013, the Opsmatic team comprises experienced development and operations professionals who were involved at the beginning of the DevOps movement at major web-scale companies. The company is backed by leading investors in the cloud technology space, including AME Cloud Ventures (Jerry Yang), Freestyle Ventures, Illuminate Ventures and Index Ventures. For more info, please visit opsmatic.com or follow @Opsmatic on Twitter.

Last month we shared the news about the debut Opsmatic and their live state monitoring service, a solution that delivers a precise, real-time picture of the detailed configuration – as well as changes that affect a computing infrastructure. We wanted to let you know that Opsmatic continues to expand its service capabilities with the announcement of two new versions. The Enterprise version of the Opsmatic service includes all the features we discussed last month in their Professional edition with the addition of single sign-on and dedicated support. The On-Premises edition embodies features contained in the Enterprise version in a solution designed for customers with an isolated infrastructure who require an internally-deployed solution. Details of Opsmatic’s new service offerings are outlined in the press release below. Should you have any questions, Opsmatic would be happy to respond to them.

The complexity of big data makes it a difficult concept for many to grasp, and utilizing it effectively is one of the biggest challenges businesses face today. There is little doubt that big data offers organizations a number of clear advantages, but applying them across the entire enterprise is one obstacle that can truly be described as formidable, even daunting, to even the most technologically savvy companies. One department might be able to create its own business solutions through big data analytics, while another department might come up with answers of their own, but lack of true coordination and collaboration remains a significant problem. Businesses aren’t without help in this area, however, because they’ve encountered similar problems before. Many companies have encountered issues such as virtualization technology sprawl, and the lessons learned from addressing that problem could prove to be exceptionally valuable when dealing with big data true cost of ownership (TCO). To understand the problem and the solution, we must first look back at the rapid growth of virtualization technology, more specifically server virtualization. As businesses adopted virtualization, the mainframe systems soon diverged into multiple systems. The more popular virtualization became, the more projects were taken on and the more technologies diverged. Larger companies eventually sought technology specialists to work within their areas of expertise. The result of the use of these individual teams was virtualization technology sprawl, an inefficient development that eventually lead to even higher operational costs. For all the benefits virtualization technology offered, many of them were outweighed by the increased demands and greater management complexity that came from technology sprawl. Businesses were quick to come up with new solutions for the problem. The most common was to adopt a converged infrastructure . This strategy directly addressed the higher operational costs that resulted from technology sprawl, basically breaking through the silos by taking multiple technologies and combining them into single stacks for computing, storage, and networking. This made the management of virtualization technology much easier since operational complexity was significantly reduced. In other words, management of this technology was kept at a reasonable size. The same principle can apply to big data management across an entire organization. When it comes to management of big data and hadoop security, it’s easy to get caught up in the immensity of it all. The fact that big data is so versatile and can be applied to so many different use cases also means it can apply to any number of different divisions within a company. This creates silos and a general desire to hold onto data sets. In other words, big data ends up in a sprawl of its own, becoming that much more unwieldy and complicated, which is a major problem for a technology that’s already so complex to begin with. The lesson that every company should take away from the solution to virtualization technology sprawl is the breaking down of barriers to big data management. It all comes down to ready access to all the necessary data no matter what roles an employee may have within a company. Businesses shouldn’t have to worry over the cost it takes to store and process data since the insights gained from big data analytics are particularly valuable. Most importantly, it’s about avoiding big data from getting too big, to the point where it becomes unmanageable and merely adds to the overall operating costs of a company. It’s true that big data introduces more complexity, but businesses that have learned how to store and process it efficiently, sometimes through big data platforms or cloud-based services, are in a more advantageous position than companies still dealing with technology sprawl. The lessons learned from previous problems can indeed play a helpful role in solving the problems many experience today.

London, UK – FusionExperience, the business and data solutions provider, today announces the success of its first salesforce.com partnership with Cloud Consulting Ltd. (CCL). CCL was working with an international airline client to migrate a legacy charter and group booking application from one Salesforce.com instance to a new one. Very early on in the project CCL discovered that there were considerable elements of unsupported custom code and that these had to be redesigned and redeveloped. The airline took the opportunity at this stage to request changes and improve the application in line with their new business processes. CCL worked with FusionExperience to migrate the application to the latest salesforce.com environment and re-architected the booking engine functionality and complex pricing algorithms using Apex and VisualForce. For business reasons the airline had a strict project deadline and despite all the unknowns involved the project timescales were maintained and FusionExperience delivered on time and to budget. The airline went live with the application on schedule without any post-production problems or warranty fixes required. They now have an up to date system that has achieved a game changing transformation in the way it does business. Robin James, Platform Evangelist for FusionExperience said; “The ability to seamlessly work with our partners on salesforce.com projects enables rapid scaling of resources and capabilities. This ensures that the client is delighted by the results, yet unaware of the complex extended ecosystem that has been involved. This is facilitated by that fact that we all speak the same salesforce.com language. Cloud Consulting is an ideal partner to work with in this way, as our delivery and technical strengths are well matched with their intimate client facing approach.” Tim Pullen, Managing Director of CCL added: “We already had a close relationship with FusionExperience and it was natural for us to turn to them for help with this suddenly extremely challenging project. The combination of cleaning, segmenting and splitting the data in Salesforce.com, extracting the system configuration and custom code and then creating a new system was tough enough to start but then having to redevelop the application from scratch took it to a new level. Right from the start Robin James and his team took everything in their stride and provided a level of comfort, reassurance, skill and professionalism that we’d never experienced before from other partners. Bear in mind that the old system had no user or technical documentation plus undocumented code and you begin to understand just how good the end result has been for the airline. Thank you Fusion!”

From the DevNation Agenda site: You've probably heard a lot about Linux containers and the exciting potential they hold. In this presentation, Matt Hicks will cover how Docker and Kubernetes have evolved to fundamentally change how you will approach development and operations. If you are looking for an understanding of the technology and how it relates to the common roles in IT today, this is the talk to watch. Speaker: Matt Hicks -- Vice President of engineering, Red Hat Matt Hicks is a founding member of the OpenShift by Red Hat team. He has spent more than a decade in software engineering, with a variety of roles in development, operations, architecture, and management. His real expertise is in bridging the gap between developing code and actually running it in production. An expert in IT and cloud-based architectures, he spends his time these days evolving OpenShift to use the power of cloud and make developers more productive.

Written by Leena Joshi Microsoft recently announced that Microsoft AppFabric 1.1 for Windows Server will be at the end of support on April 2, 2016. Less than a year away! Don’t panic yet, there is another, better solution. Redis is the product of choice for thousands of developers worldwide who want to accelerate their applications. Redis is the fastest growing NoSQL datastore, that runs in-memory and delivers millions of transactions at sub millisecond latencies. Redis Labs provides enterprise class Redis: as a downloadable product Redis Labs Enterprise Cluster (RLEC) or as a seamlessly scalable, highly available service, Redis Cloud. Here are the top 5 reasons to move applications using Microsoft AppFabric to Redis Cloud or Redis Labs Enterprise Cluster : Performance: One of the biggest reasons to move is the blazing fast performance and versatility of Redis. While AppFabric performs simple GET and SET commands, Redis comes with a variety of data structures (strings, lists, hashes, sets, sorted sets) and a sophisticated set of commands, embedded Lua scripting and bit operations that helps you address more than high speed caching – it lets you implement high speed transactions, real time analytics, in-app social functionality, messaging, job and queue management and much more. Scalability: The current AppFabric replacement offered by MS runs on Azure, however it is not a service that scales seamlessly like Redis Cloud. Redis Cloud with 4900+ customers to date is proven to be highly scalable and available and provides all the functionality of Redis with none of the deployment overhead. Also, if you don’t really want a cloud solution, RLEC (Redis Labs Enterprise Cluster) runs on-premises or wherever you are deployed and relieves you of any provisioning, configuring, scaling, clustering, monitoring – it fully automates all those tasks and makes it super-easy to deploy Redis clusters. Portability: AppFabric is Windows and .Net specific while Redis supports many environments and languages.Thanks to our community, Redis supports many languages including Python, Ruby, Java, PHP, Node, C, C# . So, if you have a mixed environment, now you can even extend your use of in-memory, high speed technologies. Built-in Monitoring: Both Redis Cloud and RLEC come with a dashboard to view different operational metrics as well as built-in alerts to receive notification on important events. This is very much more cumbersome with AppFabric and you would have had to use external tools to get the same level of manageability. Ease of use: Redis is simple yet very sophisticated and used by thousands of developers worldwide. It is #3 among NoSQL database adoption (source:DBengines) and #12 among all tools used by developers (source:Stackshare). This means that a talent pool of Redis users and a worldwide community is always there to help! RLEC is free to download and Redis Cloud has a free tier as well – so it is really easy to try out both those products. If you have any questions about migrating from AppFabric to Redis Labs, our solutions consultants are always here to help. Email us at [email protected] and we can set up some time to walk you through the migration! - See more at: https://redislabs.com/blog/appfabric-coming-apart-top-5-reasons-to-move-to-redis#.VYSGzucYGL4

Welcome to Useful PC Guide, we are covering latest technology news with many topics on computing, mobile, programming, technology, computer games, games, mobile games, Apple iOS, and Android apps as well as online tutorials, guides and how-to articles. UsefulPCGuide.com website also regularly updates new Windows OS tips and tricks to resolve your problems, as well as iOS and Android issues. You can read an example tutorial from us about how to fix your connection is not private error in Google Chrome in Windows OS. This guide will help you to learn more about causes of this error, and appropriate ways to troubleshoot the issues on your Chrome browser. Most of our tips and tricks are include images and very easy to read and follow up the instructions. Visit usefulguide.com for more good news and tutorials.

You will aggregate a lot of logs over the lifetime of your product and codebase, so it’s important to be able to search through them. In the rare case of a security issue, not having that capability is incredibly painful. You might be able to use services that allow you to search through the logs of the last two weeks quickly. But what if you want to search through the last six months, a year, or even further? That availability can be rather expensive or not even an option at all with existing services. Many hosted log services provide S3 archival support which we can use to build a long-term log analysis infrastructure with AWS Redshift. Recently I’ve set up scripts to be able to create that infrastructure whenever we need it at Codeship. AWS Redshift AWS Redshift is a data warehousing solution by AWS. It has an easy clustering and ingestion mechanism ideal for loading large log files and then searching through them with SQL. As it automatically balances your log files across several machines, you can easily scale up if you need more speed. As I said earlier, looking through large amounts of log files is a relatively rare occasion; you don’t need this infrastructure to be around all the time, which makes it a perfect use case for AWS. Setting Up Your Log Analysis Let’s walk through the scripts that drive our long-term log analysis infrastructure. You can check them out in the flomotlik/redshift-logging GitHub repository. I’ll take you step by step through configuring the whole setup of the environment variables needed, as well as starting the creation of the cluster and searching the logs. But first, let’s get a high-level overview of what the setup script is doing before going into all the different options that you can set: Creates an AWS Redshift cluster. You can configure the number of servers and which server type should be used. Waits for the cluster to become ready. Creates a SQL table inside the Redshift cluster to load the log files into. Ingests all log files into the Redshift cluster from AWS S3. Cleans up the database and prints the psql access command to connect into the cluster. Be sure to check out the script on GitHub before we go into all the different options that you can set through the .env file. Options to set The following is a list of all the options available to you. You can simply copy the .env.template file to .env and then fill in all the options to get picked up. AWS_ACCESS_KEY_ID AWS key of the account that should run the Redshift cluster. AWS_SECRET_ACCESS_KEY AWS secret key of the account that should run the Redshift cluster. AWS_REGION=us-east-1 AWS region the cluster should run in, default us-east-1. Make sure to use the same region that is used for archiving your logs to S3 to have them close. REDSHIFT_USERNAME Username to connect with psql into the cluster. REDSHIFT_PASSWORD Password to connect with psql into the cluster. S3_AWS_ACCESS_KEY_ID AWS key that has access to the S3 bucket you want to pull your logs from. We run the log analysis cluster in our AWS Sandbox account but pull the logs from our production AWS account so the Redshift cluster doesn’t impact production in any way. S3_AWS_SECRET_ACCESS_KEY AWS secret key that has access to the S3 bucket you want to pull your logs from. PORT=5439 Port to connect to with psql. CLUSTER_TYPE=single-node The cluster type can be single-node or multi-node. Multi-node clusters get auto-balanced which gives you more speed at a higher cost. NODE_TYPE Instance type that’s used for the nodes of the cluster. Check out the Redshift Documentation for details on the instance types and their differences. NUMBER_OF_NODES=10 Number of nodes when running in multi-mode. CLUSTER_IDENTIFIER=log-analysis DB_NAME=log-analysis S3_PATH=s3://your_s3_bucket/papertrail/logs/862693/dt=2015 Database format and failed loads When ingesting log statements into the cluster, make sure to check the amount of failed loads that are happening. You might have to edit the database format to fit to your specific log output style. You can debug this easily by creating a single-node cluster first that only loads a small subset of your logs and is very fast as a result. Make sure to have none or nearly no failed loads before you extend to the whole cluster. In case there are issues, check out the documentation of the copy command which loads your logs into the database and the parameters in the setup script for that. Example and benchmarks It’s a quick thing to set up the whole cluster and run example queries against it. For example, I’ll load all of our logs of the last nine months into a Redshift cluster and run several queries against it. I haven’t spent any time on optimizing the table, but you could definitely gain some more speed out of the whole system if necessary. It’s just fast enough already for us out of the box. As you can see here, loading all logs of May — more than 600 million log lines — took only 12 minutes on a cluster of 10 machines. We could easily load more than one month into that 10-machine cluster since there’s more than enough storage available, but for this post, one month is enough. After that, we’re able to search through the history of all of our applications and past servers through SQL. We connect with our psql client and send of SQL queries against the “events’ database. For example, what if we want to know how many build servers reported logs in May: loganalysis=# select count(distinct(source_name)) from events where source_name LIKE 'i-%'; count ------- 801 (1 row) So in May, we had 801 EC2 build servers running for our customers. That query took ~3 seconds to finish. Or let’s say we want to know how many people accessed the configuration page of our main repository (the project ID is hidden with XXXX): loganalysis=# select count(*) from events where source_name = 'mothership' and program LIKE 'app/web%' and message LIKE 'method=GET path=/projects/XXXX/configure_tests%'; count ------- 15 (1 row) So now we know that there were 15 accesses on that configuration page throughout May. We can also get all the details, including who accessed it when through our logs. This could help in case of any security issues we’d need to look into. The query took about 40 seconds to go though all of our logs, but it could be optimized on Redshift even more. Those are just some of the queries you could use to look through your logs, gaining more insight into your customers’ use of your system. And you et all of that with a setup that costs $2.50 an hour, can be shut down immediately, and recreated any time you need access to that data again. Conclusions Being able to search through and learn from your history is incredibly important for building a large infrastructure. You need to be able to look into your history easily, especially when it comes to security issues. With AWS Redshift, you have a great tool in hand that allows you to start an ad hoc analytics infrastructure that’s fast and cheap for short-term reviews. Of course, Redshift can do a lot more as well. Let us know what your processes and tools around logging, storage, and search are in the comments.

Written by Mark Pollack. Today, we are pleased to announce the general availability of Spring XD 1.2, Spring XD 1.1.3 and the release of Flo for Spring XD Beta. 1.2.0.GA: zip 1.1.3.RELEASE: zip Flo for Spring XD Beta You can also install XD 1.2 using brew and rpm The 1.2 release includes a wide range of new features and improvements. The release journey was an eventful one, mainly due to Spring XD’s popularity with so many different groups, each with their respective request priorities. However the Spring XD team rose to the challenge and it is rewarding to look back and review the amount of innovation delivered to meet our commitments toward simplifying big data complexity. Here is a summary of what we have been busy with for the last 3 months and the value created for the community and our customers. Flo for Spring XD and UI improvements Flo for Spring XD is an HTML5 canvas application that runs on top of the Spring XD runtime, offering a graphical interface for creation, management and monitoring streaming data pipelines. Here is a short screencast showing you how to build an advanced stream definition. You can browse the documentation for additional information and links to additional screen casts of Flo in action. The XD admin screen also includes a new Analytics section that allows you to easily view gauges, counters, field-value counters and aggregate counters. Performance Improvements Anticipating increased high-throughput and low-latency IoT requirements, we’ve made several performance optimizations within the underlying message-bus implementation to deliver several million messages per second transported between Spring XD containers using Kafka as a transport. With these optimizations, we are now on par with the performance from Kafka’s own testing tools. However, we are using the more feature rich Spring Integration Kafka client instead of Kafka’s high level consumer library. For anyone who is interested in reproducing these numbers, please refer to the XD benchmarking blog, which describes the tests performed and infrastructure used in detail. Apache Ambari and Pivotal HD To help automate the deployment of Spring XD on an Apache HadoopⓇ cluster, we added an Apache AmbariⓇ plugin for Spring XD. The plugin is supported on both Pivotal HD 3.0 and Hortonworks HDP 2.2 distributions. We also added support in Spring XD for Pivotal HD 3.0, bringing the total number of Hadoop versions supported to five. New Sources, Processors, Sinks, and Batch Jobs One of Spring XD’s biggest value propositions is its complete set of out-of-the-box data connectivity adapters that can be used to create real-time and batch-based data pipelines, and these require little to no user-code for common use-cases. With the help of community contributions, we now have MongoDB, VideCap, and FTP as source modules, an XSLT-transformer processor, and FTP sink module. The XD team also developed a Cassandra sink and a language-detection processor. Recognizing the important role in the Pivotal Big Data portfolio, we have also added native integration with Pivotal Greenplum Database and Pivotal HAWQ through gpfdist sink for real-time streaming and also support for gpload based batch jobs. Adding to our developer productivity theme and the use of Spring XD in production for high-volume data ingest use-cases, we are delighted to recognize Simon Tao and Yu Cao (EMC² Office of The CTO & Labs China), who have been operationalizing Spring XD data pipelines in production since 2014 and also for the VideCap source module contribution. Their use-case and implementation specifics (in their own words) are below. “There are significant demands to extract insights from large magnitude of unstructured video streams for the video surveillance industry. Prior to being analyzed by data scientists, the video surveillance data needs to be ingested in the first place. To tackle this challenge, we built a highly scalable and extensible video-data ingestion platform using Spring XD. This platform is operationally ready to ingest different kinds of video sources into a centralized Big Data Lake. Given the out-of-the-box features within Spring XD, the platform is designed to allow rich video content processing capabilities such as video transcoding and object detection, etc. The platform also supports various types of video sources—data processors and data exporting destinations (e.g. HDFS, Gemfire XD and Spark)—which are built as custom modules in Spring XD and are highly reusable and composable. With a declarative DSL, a video ingestion stream will be handled by a video ingestion pipeline defined as Directed Acyclic Graph of modules. The pipeline is designed to be deployed in a clustered environment with upstream modules transferring data to downstream ones efficiently via the message bus. The Spring-XD distributed runtime allows each module in the pipeline to have multiple instances that run in parallel on different nodes. By scaling out horizontally, our system is capable of supporting large scale video surveillance deployment with high volume of video data and complex data processing workloads.” Custom Module Registry and HA Support Though we have had the flexibility to configure shared network location for distributed availability of custom modules (via: xd.customModule.home), we also recognized the importance of having the module-registry resilient under failure scenarios—hence, we have an HDFS backed module registry. Having this setup for production deployment provides consistent availability of custom module bits and the flexibility of choices, as needed by the business requirements. Pivotal Cloud Foundry Integration Furthering the Pivotal Cloud Foundry integration efforts, we have made several foundation-level changes to the Spring XD runtime, so we are able to run Spring XD modules as cloud-native Apps in Lattice and Diego. We have aggressive roadmap plans to launch Spring XD on Diego proper. While studying Diego’s Receptor API (written in Go!), we created a Java Receptor API, which is now proposed to Cloud Foundry for incubation. Next Steps We have some very interesting developments on the horizon. Perhaps the most important, we will be launching new projects that focus on message-driven and batch-oriented “data microservices”. These will be built directly on Spring Boot as well as Spring Integration and Spring Batch, respectively. Our main goal is to provide the simplest possible developer experience for creating cloud-native, data-centric microservice apps. In turn, Spring XD 2.0 will be refactored as a layer above those projects, to support the composition of those data microservices into streams and jobs as well as all of the “as a service” aspects that it provides today, but it will have a major focus on deployment to Cloud Foundry and Lattice. We will be posting more on these new projects soon, so stay tuned! Feedback is very important, so please get in touch with questions and comments via * StackOverflowspring-xd tag * Spring JIRA or GitHub Issues Editor’s Note: ©2015 Pivotal Software, Inc. All rights reserved. Pivotal, Pivotal HD, Pivotal Greenplum Database, Pivotal Gemfire and Pivotal Cloud Foundry are trademarks and/or registered trademarks of Pivotal Software, Inc. in the United States and/or other countries. Apache, Apache Hadoop, Hadoop and Apache Ambari are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All Posts Engineering Releases News and Events

Every now and then I get work done in the home office. I’ve written previously about my setup, but after churning out some solution design today, I sat back and really took some time to appreciate the workspace. I’m really pleased with the configuration, it’s probably the best setup I’ve had in years. The desk is a former QLD police desk from the 1940s, so it wasn’t built for modern computers – not a problem, the cables run down the back which is just a minor annoyance. The keyboard and mouse are gaming varieties so that they perform well – the old Sennheiser (RF) wireless headset has been with me since 2006 and still works very well. The wooden clock (recently reviewed) acts as external speakers, a Bluetooth receiver and has a built in microphone so it can be used as a hands-free option for conference calls. It also features Qi wireless charging capability and also features a thermostat. Under the second monitor is a HDD caddy which supports USB3, and features 4 bays which can be used in parallel. I try to keep the desk reasonably neat, and there’s plenty of space so it doesn’t get too cluttered. I have a nice view out the window to a small courtyard which gets early morning sun.

Mobile app development has become vital for enterprises as they look to support new devices (phones, tablets, wearables, etc.) for internal use while also reaching out to their increasingly mobile customers. This approach makes sense: According to a comScore report, the number of mobile Internet users outnumbered desktop ones for the first time at some point in late 2013, and has since achieved significant separation. Many companies have responded to this change by implementing bring-your-own-device policies and building mobile apps that complement their full websites, mobile Web presence and/or desktop applications. Watch out for pitfalls in mobile apps: General risks and the recent Starbucks example However, both BYOD policies and mobile app development require due diligence around cybersecurity if they are to be worthwhile. Safety starts with well-designed applications that are strongly authenticated, do not leak sensitive data and are safe from popular attack vectors like brute-force password guessing. Unfortunately, many apps still have a long way to go on these fronts. An early 2014 study from MetaIntell discovered that 92 percent of the top 500 most popular Android apps at the time created privacy risks due to data leakage. Wary of leaky apps as well as what kinds of information users put into them, enterprises have understandably been concerned about the impact of mobile apps on their operations and BYOD initiatives. Security is often the biggest barrier to effective BYOD, and justifiably so considering that barely more than 40 percent of employees are required to have a security tool installed, according to Webroot. To get a sense of what could go wrong with today's mobile apps, consider what recently happened to Starbucks. The company's app is a mainstay on many phones, and at one time it accounted for the bulk of all mobile payments made in North America. The issue that arose over the last few months involved unauthorized card reloads and apparent account hijackings. The causes may have been mixed, with poor password management on the part of users possibly exacerbated by exploitation of the app's auto-reload feature and an April 2015 outage of the coffee chain's point-of-sale systems. At the end of the day, Starbucks implemented additional security questions and has been urged to add two-factor authentication into the app to prevent erroneous transactions. Catching mobile app security issues with a test management solution As we can see, mobile app security is multifactorial, requiring best efforts on the parts of end users, developers and infrastructure/network providers. For enterprises, the best approach to ensuring long-term security is to catch potential vulnerabilities early and often with a test management system. A test management solution supports both automated and manual testing, and receiving updates in real-time offers you the ability to make important decisions once issues arise. Regardless of how many tests, sprints and projects your company is running, all of them should be conveniently viewed from a lone interface, enabling a single source of truth that keeps your mobile app development initiatives on track.

The software development life cycle is an extremely intensive process for developers and quality assurance professionals alike. If even one element is neglected, it can delay project schedules and affect user performance. Security is one aspect that must be built in from the inception of any app, and here are a few reasons why: Breaches can cost your business Let's say that an organization uses its application to order and manage inventory, payroll and other operational needs. If a malicious entity were to access this information, it could easily make fraudulent transactions, costing the company more than what was intended. Not to mention it will create a massive headache to set the record straight. TechTarget contributor Peter Gregory noted that this can happen when programs lack audit trails and processes required for secure purchasing. By building in this functionality early on, this type of situation can be avoided, allowing organizations to retain customer trust and money. "Organizations that fail to involve information security in the life cycle will pay the price in the form of costly and disruptive events," Gregory wrote. "Many bad things can happen to information systems that lack the required security interfaces and characteristics." Access to confidential data can be damaging If a business aims to use an app for information sharing and availability, protection must be at the forefront of this project throughout its life cycle. While some data may not be as costly to leak, the loss of confidential reports and documents can severely affect the organization's ability tofunction. QA teams must ensure that security practices are implemented and built upon constantly. TechTarget contributor Nick Lewis noted that firewalls and traditional methods will not be enough to keep targeted attacks at bay. Instead, testing the app for insufficient process validation, abuse of functionality, weak password recovery validation and information leakage will be critical toguarding the program. Analyze initial risk before jumping in One SDLC security practice to observe is a primary risk assessment before the start of a new project. Not all applications are equal, which means each program will be labeled with a different risk level. Some software will be publicly accessible, whereas others will be more business-critical and involve processing sensitive data. These uses will largely determine how much risk would be involved with a breach on such activities. This information will give QA teams a clear picture of the security roadmap needed, and can be implemented. "Doing the preliminary risk assessment to establish the need for the system helps identify any security show stoppers before too much time and effort goes into the next SDLC phases," a SANS white paper stated. "It also gets the design team thinking about security issues early in the design process." Cyberattacks and malware in the headlines have made security more prominent than ever before. By building in protections early in the SDLC, QA teams can ensure that they will be better able tohandle these threats without interruptions to regular business activities.

Introduction Continuous integration is a practice that helps developers deliver better software in a more reliable and predictable manner. This article deals with the problems developers face while writing, testing and delivering software to end users. Through exploring continuous integration, we will cover how we can overcome these issues. The Problem First, we will take a look at the source of the problem, which lies in the software development cycle. Next, we will cover some of the change conflicts that can take place during that process, and finally we will explore the main factors that can make these problems escalate, followed by an explanation of how continuous integration solves these issues. The Source of the Problem Let's take a look at what a traditional software development cycle looks like. Each developer gets a copy of the code from the central repository. The starting point is usually the latest stable version of the application. All developers begin at the same starting point, and work on adding a new feature or fixing a bug. Each developer makes progress by working on their own or in a team. They add or change classes, methods and functions, shaping the code to meet their needs, and eventually they complete the task they were assigned to do. Meanwhile, the other developers and teams continue working on their own tasks, changing the code or adding new code, solving the problems they have been assigned. If we take a step back and look at the big picture, i.e. the entire project, we can see that all developers working on a project are changing the context for the other developers as they are working on the source code. As teams finish their tasks, they copy their code to the central repository. There are two scenarios that can take place at this point. The code in the central repository is unchanged The code is the same as the initial copy. If this is the case, things are simple, because the system is unchanged. All the ideas we had about the system still stand. This is always the case if you are the only developer working on the application and if you have finished your work before the other members of your team. Either way, things are looking good for you. The system you have created and tested can be delivered to users without additional changes. The code in the central repository has changed The second scenario is that the application you have been working on has changed, and you discover this at the point when you try to copy your code over to the central repository. Changes in the code may or may not be in conflict with the ones you've made. If there are conflicts, you need to resolve them in order to be able to successfully deliver your code to the users. In this case, things could get complicated. Next, we'll explore the types of conflicts that can happen and what you may need to do to resolve them. Change Conflicts There are several types of change conflicts that can occur when integrating code. Here are some of the most common ones. We'll start with the simplest scenarios, and gradually explore the more complex ones. The implementation details have changed - You refactored a method, but so did the developer that has already integrated their code into the central repository. The behavior of the method is the same in all three implementations. You will need to pick the version that will stay, and remove the other implementations. You can even come up with a fourth implementation. This is a simple type of conflict, which you can usually resolve within a few minutes. The APIs you have been relying on have changed - For instance, the behavior of a certain method has changed. This could affect your code in a number of ways — from minor changes that you might need to make, to major structural changes. There is no silver bullet in such cases. You will need to carefully study the changes and make all the fixes. An entire subsystem of the application behaves in a different way - in such cases you will almost certainly be facing a partial, if not a full rewrite of your solution. If this is the case, you will probably need to speak with all the developers working on the application, because such a significant change should not happen without letting the rest of the team know about it. These and a number of other issues could come up, caused by various factors. Different versions of frameworks, libraries, databases are another potential source of conflicts. Once you have updated your code so it can be compiled or interpreted, you also need to remember to repeat all the tests that you have previously ran. These examples show that the amount of work needed to solve a problem that was initially assigned to a developer can easily double. Escalating Factors Here are some of the main factors that can make these problems escalate. The size of the team working on the project. The number of changes that are being pushed back into the main repository is proportional to the number of people on the project. This makes the process of integrating code into the main repository significantly harder. The amount of time passed since the developer got the latest version of the code from the central repository. As time passes, other people working on the same project are integrating more and more of their work, and changing the context in which your code needs to run. Sometimes the changes in the main repository are so big that it's easier to do a complete rewrite of your solution. A large number of changes in the system make integration events more complex and can have a huge effect on the productivity of the team. Such situations are even referred to as "integration hell". This process has a number of other negative consequences for your business. Testing and fixing bugs can take forever. Your releases are running late. Teams are stressed out because of long and unpredictable release cycles, and morale deteriorates. Solution: Integrate Continuously The solution to the problem of managing a large number of changes in big integration events is conceptually simple. We need to split these big integration events into much smaller integration events. This way, developers need to deal with a much smaller number of changes, which are easier to understand and manage. To keep integration events small and easily manageable, we need them to happen often. A couple of times a day is ideal. The practice of doing small integrations often is called Continuous Integration. The idea is simple, but at the same time it often appears to be impossible to implement in practice. This is because changing the process requires us to change some of our own habits, and changing habits is difficult. The Practice of Continuous Integration In order to avoid the previously described issues, developers need to integrate their partially complete work back into the main repository on a daily basis, or even a couple of times a day. To accomplish this, they first need to pull in all the changes added to the main repository while they were working on the code. They also must make sure that their code will work once it is integrated into the main repository. The only way to ensure this is to test every feature of the application. What first comes into mind when we start considering continuous integration is that the developers would need to spend half of their time every day testing the code in order not to break the code in the main repository for everyone else. This is why the prerequisite for continuous integration is having an automated test suite. Automated tests take away the burden of the manual, repetitive, and error-prone testing process from the developers. They also make the entire testing process much quicker. A computer can replace hours of manual testing with just minutes of automated testing. Behavior-driven and test-driven development are techniques that help developers write clean, maintainable code while writing tests at the same time. Testing techniques are out of the scope of this article, and you can read more about them in other articles on Semaphore Community. Tests make sense only if they are executed every time the source code changes, without exception. A continuous integration service such as Semaphore CI is a tool which can automate this process by monitoring the central code repository and running tests on every change in the source code. Apart from running tests, they also collect test results and communicate those results to the entire team working on the project. The result of continuous integration is so important that many teams have a rule to stop working on their current task if the version in the central repository is broken. They join the team which is working on fixing the code until tests are passing again. The role of a continuous integration service is to improve the communication between developers by communicating the status of a project's source code. How to Adopt Continuous Integration Continuous integration as a practice makes a big contribution to improving the development process, but also calls for essential changes in the everyday development routine. Adopting it comes with challenges that are easy to overcome if the process is introduced gradually. One of the biggest challenges teams face is the lack of an automated testing suite. A good recipe for overcoming this situation is to start adding automated tests for all new features as they are being developed. At the same time, the developer working on a bug fix should also work to cover the related code with tests. Whenever a bug is reported, the team should first write a failing test to demonstrate the existence of bug. Once the fix is created, the tests should pass. Over time, the automated tests suite gradually becomes more comprehensive, and the developers begin relying on it more and more. Adopting a continuous integration service to communicate the status of the tests to the entire team in the early stages of a project is also important, because it raises awareness of the project status among team members. Conclusion Introducing continuous integration and automated testing into the development process changes the way software is developed from the ground up. It requires effort from all team members, and a cultural shift in the organization. Big changes in the workflow are not easy to pull off quickly. Changes have to be introduced gradually, and all team members and stakeholders need to be on board with the idea. Educating team members about the practice of continuous integration practice and building the automated tests suite needs to be done systematically. Once the first steps have been taken, the process usually continues on its own, as both developers and stakeholders begin seeing the benefits of automated testing suites and the peace of mind that this practice brings to the entire team. Article originally posted on the Semaphore Community.

robert annett wrote a post titled diagrams for system evolution where he describes a simple approach to showing how to visually describe changes to a software architecture. in essence, in order to show how a system is to change, he'll draw different versions of the same diagram and use colour-coding to highlight the elements/relationships that will be added, removed or modified. i've typically used a similar approach for describing as-is and to-be architectures in the past too. it's a technique that works well. although you can version control diagrams, it's still tricky to diff them using a tool. one solution that addresses this problem is to not create diagrams, but instead create a textual description of your software architecture model that is then subsequently rendered with some tooling. you could do this with an architecture description language (such as darwin ) although i would much rather use my regular programming language instead. creating a software architecture model as code this is exactly what structurizr is designed to do. i've recreated robert's diagrams with structurizr as follows. and since the diagrams were created by a model described as java code , that description can be diff'ed using your regular toolchain. code provides opportunities this perhaps isn't as obvious as robert's visual approach, and i would likely still highlight the actual differences on diagrams using notation as robert did too. creating a textual description of a software architecture model does provide some interesting opportunities though.

Learn about using the microservice architecture pattern to build microservices and API gateways--compared to the usage of monolithic application architecture.