Docker deployments can be very dynamic with containers being started and stopped, moved around the YARN or Mesos-managed clusters, having very short life spans (the so-called pets) or long uptimes (aka cattle). Getting insight into the current and historical state of such clusters goes beyond collecting container performance metrics and sending alert notifications. If a container dies or gets paused, for example, you may want to know about it, right? Or maybe you’d want to be able to see that a container went belly up in retrospect when troubleshooting, wouldn’t you? Just two weeks ago we added Docker Monitoring (docker image is right here for your pulling pleasure) to SPM. We didn’t stop there — we’ve now expanded SPM’s Docker support by adding Docker Event collection, charting, and correlation. Every time a container is created or destroyed, started, stopped, or when it dies, spm-agent-docker captures the appropriate event so you can later see what happened where and when, correlate it with metrics, alerts, anomalies — all of which are captured in SPM — or with any other information you have at your disposal. The functionality and the value this brings should be pretty obvious from the annotated screenshot below. Like this post? Please tweet about Docker Events and Docker Metrics Monitoring Know somebody who’d find this post useful? Please let them know… Here’s the list of Docker events SPM Docker monitoring agent currently captures: Version Information on Startup: server-info – created by spm-agent framework with node.js and OS version info on startup docker-info – Docker Version, API Version, Kernel Version on startup Docker Status Events: Container Lifecycle Events like create, exec_create, destroy, export Container Runtime Events like die, exec_start, kill, oom, pause, restart, start, stop, unpause Every time a Docker container emits one of these events spm-agent-docker will capture it in real-time, ship it over to SPM, and you’ll be able to see it as shown in the above screenshot. Oh, and if you’re running CoreOS, you may also want to see how to index CoreOS logs into ELK/Logsene. Why? Because then you can have not only metrics and container events in one place, but also all container and application logs, too! If you’re using Docker, we hope you find this useful! Anything else you’d like us to add to SPM (for Docker or anyother integration)? Leave a comment, ping @sematext, or send us email – tell us what you’d like to get for early Christmas!

DevOps can help reduce technical debt in some fundamental ways. Continuous Delivery/Deployment First, building a Continuous Delivery/Deployment pipeline, automating the work of migration and deployment, will force you to clean up inconsistencies and holes in configuration and code deployment, and inconsistencies between development, test and production environments. And automated Continuous Delivery and Infrastructure as Code gets rid of dangerous one-of-a-kind snowflakes and configuration drift caused by making configuration changes and applying patches manually over time. Which makes systems easier to setup and manage, and reduces the risk of an un-patched system becoming the target of a security attack or the cause of an operational problem. A CD pipeline also makes it easier, cheaper and faster to pay down other kinds of technical debt. With Continuous Delivery/Deployment, you can test and push out patches and refactoring changes and platform upgrades faster and with more confidence. Positive Feedback The Lean feedback cycle and Just-in-Time prioritization in DevOps ensures that you’re working on whatever is most important to the business. This means that bugs and usability issues and security vulnerabilities don’t have to wait until after the next feature release to get fixed. Instead, problems that impact operations or the users will get fixed immediately. Teams that do Blameless Post-Mortems and Root Cause(s) Analysis when problems come up will go even further, and fix problems at the source and improve in fundamental and important ways. But there’s a negative side to DevOps that can add to technical debt costs. Erosive Change Michael Feathers’ research has shown that constant, iterative change is erosive: the same code gets changed over and over, the same classes and methods become bloated (because it is naturally easier to add code to an existing method or a method to an existing class), structure breaks down and the design is eventually lost. DevOps can make this even worse. DevOps and Continuous Delivery/Deployment involves pushing out lots of small changes, running experiments and iteratively tuning features and the user experience based on continuous feedback from production use. Many DevOps teams work directly on the code mainline, “branching in code” to “dark launch” code changes, while code is still being developed, using conditional logic and flags to skip over sections of code at run-time. This can make the code hard to understand, and potentially dangerous: if a feature toggle is turned on before the code is ready, bad things can happen. Feature flags are also used to run A/B experiments and control risk on release, by rolling out a change incrementally to a few users to start. But the longer that feature flags are left in the code, the harder it is to understand and change. There is a lot of housekeeping that needs to be done in DevOps: upgrading the CD pipeline and making sure that all of the tests are working; maintaining Puppet or Chef (or whatever configuration management tool you are using) recipes; disciplined, day-to-day refactoring; keeping track of features and options and cleaning them up when they are no longer needed, getting rid of dead code and trying to keep the code as simple as possible. Microservices and Technology Choices Microservices are a popular architectural approach for DevOps teams. This is because loosely-coupled Microservices are easier for individual teams to independently deploy, change, refactor or even replace. And a Microservices-based approach provides developers with more freedom when deciding on language or technology stack: teams don’t necessarily have to work the same way, they can choose the right tool for the job, as long as they support an API contract for the rest of the system. In the short term there are obvious advantages to giving teams more freedom in making technology choices. They can deliver code faster, quickly try out prototypes, and teams get a chance to experiment and learn about different technologies and languages. But Microservices “are not a free lunch”. As you add more services, system testing costs and complexity increase. Debugging and problem solving gets harder. And as more teams choose different languages and frameworks, it’s harder to track vulnerabilities, harder to operate, and harder for people to switch between teams. Code gets duplicated because teams want to minimize coupling and it is difficult or impossible to share libraries in a polyglot environment. Data is often duplicated between services for the same reason, and data inconsistencies creep in over time. Negative Feedback There is a potentially negative side to the Lean delivery feedback cycle too. Constantly responding to production feedback, always working on what’s most immediately important to the organization, doesn’t leave much space or time to consider bigger, longer-term technical issues, and to work on paying off deeper architectural and technical design debt that result from poor early decisions or incorrect assumptions. Smaller, more immediate problems get fixed fast in DevOps. Bugs that matter to operations and the users can get fixed right away instead of waiting until all the features are done, and patches and upgrades to the run-time can be pushed out more often. Which means that you can pay off a lot of debt before costs start to compound. But behind-the-scenes, strategic debt will continue to add up. Nothing’s broke, so you don’t have to fix anything right away. And you can’t refactor your way out of it either, at least not easily. So you end up living with a poor design or an aging technology platform, slowly slowing down your ability to respond to changes, to come up with new solutions. Or forcing you to continue filling in security holes as they come up, or scrambling to scale as load increases. DevOps can reduce technical debt. But only if you work in a highly disciplined way. And only if you raise your head up from tactical optimization to deal with bigger, more strategic issues before they become real problems.

Leading today, VentureBeat reports on the quiet launch of Google’s Cloud Source Repositories. This seems like something we should have heard more about, but I don’t remember seeing anything about it. Amazon AWS announces the availability of all things Alexa, the Skills Kit, the Voice Service and a Fund. Last but not least, we have AJ Kohn, from Blind Five Year Old, talking about click-through rate being a ranking signal on Google’s search results. I don’t talk about SEO much, but reading AJ’s work is always fascinating. As always, enjoy today’s items, and please participate in the discussions on these sites. Top Stories Google has quietly launched a GitHub competitor, Cloud Source Repositories | VentureBeat Alexa Skills Kit, Alexa Voice Service, Alexa Fund | AWS Official Blog Startups, Career and Process Why offices are where work goes to die | Swizec Teller Unleashing the power of small teams | Andreas Papathanasis What Is A Tester? | Developsense Blog What happens when you stop relying on resumes | Aline Lerner Design and Development Swift 2: SIMD | Russ Bishop Why is Git better than Mercurial? | Javalobby Create a Maven archetype | Javalobby pip -t: A simple and transparent alternative to virtualenv | Zoomer Analytics Killing Off Wasabi – Part 1 | Fog Creek Blog WebAssembly- Explained | Modus Create Generating JSON Schema from XSD with JAXB and Jackson | Inspired by Actual Events AI, Machine Learning, Research and Advanced Algorithms Applying Machine Learning to Text Mining with Amazon S3 and RapidMiner | Amazon AWS Big Data, Visualization, SQL and NoSQL Is Click Through Rate A Ranking Signal? | Blind Five Year Old Cache-friendly binary search | Bannalia Discovering the Computer Science Behind Postgres Indexes | Java Code Geeks How an open-source competitive benchmark helped to improve databases | ArangoDB Security, Encryption and Cryptography Cracking JXcore… Again | Mark Haase Link Collections The Daily Six Pack: June 25, 2015 | Dirk Strauss Double Shot #1517 | A Fresh Cup Dew Drop – June 25, 2015 (#2042) | Morning Dew The Daily Six Pack: June 26, 2015 | Dirk Strauss

When a team works with a lot of maven projects it becomes quickly painful to do some basic tasks like: manage versionning and connections between the different projects. releasing and opening versions, especially when the maven-release plugin needs to be run on many projects and when versionning is not standard. managing external dependencies also can become complex, and ensuring that a single version of a dependency is used accross different projects is sometimes not a trivial question to ask. in a one word, applying transformations on a dependency graph is difficult. mind-mapping the dependency graph is difficult when the number of projects grows. That can increase the amount of time needed by new people to understand a project graph, and that also makes maintaining and changing things difficult. checking consistency and optimizing the dependency graph is not an easy task neither. having an always up-to-date build of snapshots and release is not easy when projects are distributed everywhere. Pom Explorer So there is the Pom Explorer tool which tries to address those problems by providing those functionalities : release a graph : release a pom or all poms and its/theirs dependencies and updates all dependent poms accross multiple repositories and projects, change a gav : updates a project’s gav and make all the project which depends on it follow this change. manages properties, dependency management, and so on. Pom-explorer knows what pom.xml to update and where to update. If a dependency specifies ${foobar.version}, pom-explorer will go to update the foobar.version property. query the dependency graph to retrieve pertinent information about your projects, statistics and check functions are also available, display 3d interactive graph, export graphml files, find not used dependencies and other similar problems, list java classes provided by artifacts, list java classes referenced by artifacts, runs a light and efficient web server so local and shared usage is possible. The tool will also support automatically building projects in order to always have such or such project always up to date. Use cases In this article, I will show some common use cases possible with this tool. Installation First one needs to install and run the software. Put yourself in a temporary directory and type those commands : git clone https://github.com/ltearno/pom-explorer.git cd pom-explorer java -jar target/pom-explorer.jar The program should welcome you and ask you to go to this address : http://localhost:90 This is the console to the application. You can type commands in the prompt, they will be sent to the server and it will answer. You can use up and down arrows to recall past commands. Let’s start by typing ? to get the available commands : Analyze of repositories OK. First we will analyse a directory where there are many maven projects, then we will work a bit to optimize those projects. You will have to adapt the exercise to your computer. Let’s analyse my git repositories directory : analyze directory c:\documents\repos This will analyze my projects and construct an in-memory graph of the dependency graph : Now, the program knows about everything on my projects, let’s start asking questions ! List of GAVs… Let’s get the list of all existing GAVs (groupId, artifactId, version) in the graph. There will be my projects and all the GAVs on which they depend. Type this command : gav li Note that you can type only the first letters of a command, as long as there is no ambiguity. Here li stands for list. Find dependencies on an obsolete artifact As I look through the list of GAVs, I remark that there are still an old snapshot version of the hexa.binding artifact hanging around. The latest released version is 1.3 and the working version is 1.4-SNAPSHOT so the version 1.3-SNAPSHOTshould not be used anymore. Which is the project still depending on this very deprecated this version ? Let’s ask the question : depends on fr.lteconsulting:hexa.binding:1.3-SNAPSHOT Here it is ! the project rigpa.org:regsys-clients:1.0.0-SNAPSHOT is still using an old snapshot. Let’s arrange that. Pom Explorer is able to change the pom properties and dependencies by itself. Updating this wrong dependency What we want is to change fr.lteconsulting:hexa.binding:1.3-SNAPSHOT tofr.lteconsulting:hexa.binding:1.3 so that the project uses the latest release available. We could desire to change for fr.lteconsulting:hexa.binding:1.4-SNAPSHOT which would be possible with the same command as we’ll see. For that we will use the change gav command : cha ga fr.lteconsulting:hexa.binding:1.3-SNAPSHOT fr.lteconsulting:hexa.binding:1.3 Here is what Pom Explorer answers : So first Pom Explorer finds what needs to be changed in the graph. This might be the project itself and all projects which depend on it. After that the program begins a loop in which all changes are checked and appropriately transformed when needed. For instance changing a dependency version can become changing a property value. Changes are first resolved as described before and they are then transformed in a change list to apply to be applied to pom.xml files. In the ouput, there is first a little warning saying thefr.lteconsulting:hexa.binding:1.3-SNAPSHOT project was not found. That’s normal because the project in now in version 1.4-SNAPSHOT. So there is no need to modify it. Then in the change list section, the changes that are to be applied to pom.xml files are listed. The first one says ‘project not found’ and that’s ok as seen before. The second one says to modify the C:\documents\repos\regsys-clients\pom.xmland change the dependency ([DEPENDENCY])fr.lteconsulting:hexa.binding:1.3-SNAPSHOT tofr.lteconsulting:hexa.binding:1.3. The “causes” message is useful when a change is caused by other changes (as said before a dependency change can become one or several property changes). If we had properties involved, Pom Explorer would have found them and included them in the change set. Now that we reviewed the proposed changes and agreed with them, let’s apply them by using the same command with the -apply flag : cha ga fr.lteconsulting:hexa.binding:1.3-SNAPSHOT fr.lteconsulting:hexa.binding:1.3 -apply We see that at the end of the same process, the program updated the dependency in the right pom.xml file. Let’s have a look at the file it self : fr.lteconsulting hexa.binding 1.3 compile OK, the file is correct now… Oh well no ! I just find other dependencies in SNAPSHOT versions ! Finding more duplicate and obsolete dependencies Let’s accept it, our projects are not up to date. Well let’s see how many of those artifacts there are with multiple versions used. For that i type the checkcommand : Ok there is some work to do ! Opening a version Now let’s look at another use case. Say that the hexa.binding project is in version 1.3 and i want to open the version 1.4-SNAPSHOT. I also want all the projects which depend on version 1.3to move to 1.4-SNAPSHOT. On the way, I want all modified projects still in a release version to be SNAPSHOT-ized too. And i want this to happen recursively as new projects are opened. With Pom Explorer, that’s only one command : change gav fr.lteconsulting:hexa.binding:1.3 fr.lteconsulting:hexa.binding:1.4-SNAPSHOT As you can see, warnings are generated when projects are reopened : Those are normal warnings, they are just here so that you know what happens. Then, there is a big list of changes to be made, because the hexa.bindingartifact is used in many central projects that were in a release state. Glad that we didn’t do that by hand ! Even with the maven-update-version plugin, there would have been a lot of repositories to go to open and update. Let’s apply the changes with the -apply flag : cha ga fr.lteconsulting:hexa.binding:1.3 fr.lteconsulting:hexa.binding:1.4-SNAPSHOT -apply All the changes have been made, about 30 of them. In one go ! Refresh the page so that a new session is created from the changed files. We can see that many of the projects have been reopened : You now have to commit all the repositories with this update. Pom explorer does not do that yet, but maybe in the future ! Releasing many poms Imagine the sprint is almost finished now and it’s now time to release the projects. Type the gav li fr.lteconsulting again to get the GAVs list (fr.lteconsulting is my projects package name, so I filter GAVs with that), choose one and let’s release it : fr.lteconsulting:hexa.binding.samples:1.4-SNAPSHOT The thing in the release is to have all direct and transitive dependencies released too. That’s what Pom Explorer checks. It then generates a change list to materialize your requirements. Other use cases Listing provided and referenced classes You can ask which Java classes are provided and referenced by GAVs. That’s sometimes a useful information to have. Try those commands : classes providedBy fr.lteconsulting:hexa.css:1.3 classes referencedBy fr.lteconsulting:hexa.css:1.3 Optimizing your project’s dependencies Sometimes, you ask yourself “do I still need this and that dependency ?” but you are not very sure, and since you lack time to investigate, eventually the dependency stays in your project for a long time, causing of course maintenance issues sometime. Let’s have Pom Explorer help us in the quest for the obsolete dependency. garbage dep fr.lteconsulting:carousel:1.0-SNAPSHOT This will give you something like that : You can refer to the project documentation to find how to use those informations. But sure that it can help you give away those useless dependencies ! Other goodies : graphs ! Pom Explorer can do two other things to help you visualize your dependency graph : export GraphML files so you can use them in another graph software (like yEd for instance). display an interactive 3d graph Exporting GraphML files GraphML is an open format to describe graphs. With the graph exportcommand, you can get graphml files of your working session. The program will create two files and display the links to them. Those two files are corresponding to two graphs : the dependency graph as usual, and the dependency graph between the git repositories containing your projects. Sometime one git repository can contain multiple projects and a view of the dependencies at the repository level is useful in those cases. This is the kind of picture you can get easily from editors like yEd : Interactive 3D graph Thanks to the WebGL standard which allows direct access to the 3D hardware on the running machine and thanks to libraries like three.js and ngraph.pixel, it is possible to display an interacive 3d graph. More over it is possible to customize the appearance of the graph to give account of different perspectives. Type the graph command and click on the link. This will open another tab containing the living 3d graph of your projects. When focus is given to the 3d viewport, the W, A, S, F and arrow keys allow to move in the 3d space. On the right, there is a text area where you can edit some javascript callback to customize the graph appearance. You can also stop the moving of the particle with the checkbox at the bottom right of the screen. It is not necessarilly useful, but sometimes it is relaxing to admire your work in the form of a living and moving graph ! Conclusion There are many other functions in Pom Explorer, but they are for you to discover now. This tool finds easily its place in the daily workflow because of the functions it provides. The fact that one can run it locally or on a shared server allows to use it as you wish. It is still in early development phase so many more functionalities could come up. On this subject, don’t hesitate submitting a little pull request on the GitHub repository… Pom Explorer is made with love by LTE Consulting

[This article was written by Wyatt Lindsay] New Relic’s Community Forum is designed to be a place for our users to share their experiences, questions, problems, and fixes. The collective expertise and creativity of the New Relic community has generated some outstanding solutions to everyday issues, and we want to call out some of them in the area of .NET agent deployment excellence. Basic installation Installing New Relic’s .NET agent is designed to be simple: run the installer on the target host and choose the features you want to include. For Microsoft Azuredeployments, install one of our NuGet packages. Installation requires a reset of IIS for the software to load into your application. To upgrade, we recommend first stopping IIS, installing the newer agent version, and then starting up IIS again. It’s also possible to perform a “silent” (manual) install using msiexec.exe, for example: msiexec.exe /i C:\NewRelicAgent.msi /qb NR_LICENSE_KEY= INSTALLLEVEL=1 See the documentation for complete manual installation options. Leveraging PowerShell Scripting the silent installation provides convenience and flexibility. Here’s an example script provided by community member Jon Carl in this post: $msiName = $licenseKey = $arguments = "/i $msiName /L*v install.log /qn NR_LICENSE_KEY=$licenseKey" if ($msiName -ne $null) { $exitCode = (Start-Process -FilePath "msiexec" -ArgumentList $arguments -Wait -PassThru).ExitCode; if($exitCode -eq 0) { Write-Host "Installation successful!" -ForegroundColor Green } else { Write-Host "Installation unsuccessful. Exitcode: $exitCode" -ForegroundColor Red } } This script works great when run directly on the target machine. Another forum user (Kym McGain) noticed that the installation didn’t complete before the session ended when executing the script remotely. This caused the installer to quit partway through. Kym posted this script that uses a ‘while’ loop to ensure the installer completes. As a bonus, it stops IIS before and restarts it after the software installs. As mentioned above, these steps are usually needed when upgrading. $installNewRelic = { $runProcess = { param($process,$arguments) $res = Start-Process -FilePath $process -ArgumentList $arguments -Wait -PassThru while ($res.HasExited -eq $false) { Write-Host "Waiting for $process..." Start-Sleep -s 1 } $exitCode = $res.ExitCode if($exitCode -eq 0) { Write-Host "$process successful!" -ForegroundColor Green } else { Write-Host "$process unsuccessful. Exitcode: $exitCode" -ForegroundColor Red } } $msiName = $licenseKey = $arguments = "/i $msiName /L*v install.log /qn NR_LICENSE_KEY=$licenseKey" Invoke-Command $runProcess -ArgumentList "IISRESET","/STOP" Invoke-Command $runProcess -ArgumentList "msiexec.exe",$arguments Invoke-Command $runProcess -ArgumentList "IISRESET","/START" } Chef, Puppet, and Chocolatey Deployment options abound for modern Web developers. These solutions often require a known download path and installer name. New Relic offers a consistent filepath and MSI name for the agent in an effort to make automated deployment easier for .NET customers: http://download.newrelic.com/dot_net_agent/release/NewRelicDotNetAgent_x64.msi http://download.newrelic.com/dot_net_agent/release/NewRelicDotNetAgent_x86.msi Several Community members have created packages for these utilities. Chocolatey users are invited to use the following NuGet package created by kireevco: https://chocolatey.org/packages/newrelic-dotnet New Relic community member ePitty built a Puppet module to handle .NET agent deployment: https://github.com/epitty1023/puppet-newrelicappmon Chef users, meanwhile, can check out the following cookbook for .NET and many other platforms New Relic supports: http://community.opscode.com/cookbooks/newrelic New Relic Community member E_Bow wrote a Chef recipe that goes a step further by stopping IIS before the installation and starting it again after completion: #Stop IIS iis_site 'Website' do action [:stop] end # install latest Newrelic agent from web include_recipe 'newrelic::repository' include_recipe node['newrelic']['dotnet-agent']['dotnet_recipe'] license = node['newrelic']['application_monitoring']['license'] windows_package 'Install New Relic .NET Agent' do source node['newrelic']['dotnet-agent']['https_download'] options "/qb NR_LICENSE_KEY=#{license} INSTALLLEVEL=#{node['newrelic']['dotnet-agent']['install_level']}" installer_type :msi action :install end #Start IIS iis_site 'Website' do action [:start] end The author states that they were unable to pull the New Relic license key from the configuration JSON in Chef Overrides, requiring them to modify the config file on each machine and manually enter the key. We invite any Chef experts out there to extend and improve this recipe so that it correctly pulls the license key. We are continually impressed by the smarts and spirit of our New Relic Community Forum members, and jump at the chance to highlight their contributions. Look for more Forum projects in the New Relic blog in the future. Do you have your own approach, tips, or recipes? Please share them in the New Relic Community Forum.

Spring Integration Kafka 1.2 is out with a major performance overhaul.

Last week’s Gartner IT Operations Strategies & Solutions Summit in Orlando, Fla., was exactly what you’d expect—a place to talk about the IT operations issues impacting some of the largest companies in the world. Even so, there were a few interesting surprises. Among them: 1. Bi-modal is big. Not everyone will succeed. Gartner continued to tell its customers to employ two modes of IT—a traditional, slower moving capability for older, typically internal systems of record; and a high-speed, experimental one for new, typically customer-facing Web and mobile apps. “This is a time of experimentation and innovation,” said Gartner VP and distinguished analyst Chris Howard in his opening keynote. Organizations can’t ignore that there are multiple speeds and they should participate in all. Gartner managing VPRonni Colville added that by 2017, 75% of IT orgs will have this “bi-modal” IT capability. See also: Bi-Modal IT: Gartner Endorses Both Disruptive and Conservative Approaches to Technology However, “50% will make a mess of it,” Colville said. Why? Not necessarily because of technology failings, but more often because of a lack of people skills. 2. IT success is all about people. Donna Scott, also a Gartner VP and distinguished analyst, told her keynote audience that “you will be judged on agility, speed, and innovation.” However, the biggest problems Gartner sees for infrastructure and operations team engagement and innovation are lack of time, company culture that’s not conducive to these approaches, and a lack of business skills in IT. More than half of the people responding to an in-room poll said “people” are the part of IT ops that must change first. Not technology. Gartner research director George Spafford underscored similar issues in large organizations trying to use DevOps at scale: people and “human factors” are the biggest concerns from his in-room poll. All these probably contributed to hiring best-selling author Daniel Pink as a keynote speaker on the opening day of the conference. His focus? Not IT or architecture. Instead, he pounded home the importance of influencing people and selling internally. 3. Big orgs are trying DevOps. But the issues are different at scale. In numerous sessions I saw many hands go up when analysts asked, “Who here is trying DevOps?” Clearly, the approach is getting traction in large companies. But there’s lots of learning still to do. In fact, that was Spafford’s biggest bit of advice. “Always be learning,” he said, “trying to see what works and what breaks, especially at scale.” And, even once you’ve had some initial success, keep learning. “If you’ve done ‪DevOps, stay humble,” he advised. 4. Looking to innovative organizations for ideas … analytics on the rise. Many sessions addressed how large organizations are taking on ideas fostered by smaller, more risk-tolerant companies, and offered advice for doing so successfully. In addition to multiple discussions of DevOps, an entire session was devoted to establishing your own “Genius Bar®—a “walk-up IT support center” as explained in this CIO article. As at previous conferences, Gartner research VP Cameron Haight ran several sessions on lessons learned from firms running massive, Web-scale IT systems. “You need lots of data … and access to it inexpensively,” he said. Some commercial monitoring companies (New Relic included!) got a shout out for taking the lessons of Web scale IT to heart in their offerings. In addition, Haight said, “Analytics are increasingly important for application performance monitoring given the huge amount of data now available.” 5. Cloud: Enterprises want it, but aren’t very good at it yet. Gartner research director Dennis Smith talked through the enterprise’s interest in cloud computing. A huge majority of his in-room poll wanted some mix of both public and private cloud, while only 9% wanted to use only a private cloud environment and a measly 4% were looking to move entirely to the public cloud. The most popular choice (41%) was an 80/20 split between private and public cloud infrastructure. “Enterprises don’t make the dean’s list,” for cloud usage, Smith said, earning no more than a C average in his opinion. Large organizations are doing well at visibility, governance, and delivering standardized stacks, he said, but are less skilled at optimizing for these new environments. Still, Smith said the trends point toward enterprises improving on all fronts. 6. Cloud security can be better than yours. Importantly, Gartner VP and distinguished analyst Neil MacDonald gave the cloud a vote of confidence: noting that, for a variety of reasons, “Well-managed public cloud can be more secure than your own data center.” For example, on-premise software can pose serious security risks, he said, because of “deployment lag” where customers are stuck using software releases with unpatched security vulnerabilities. With a cloud-based Software-as-a-Service (SaaS), security updates can be more quickly rolled out to all customers. But cloud security can be different, requiring a shift to information-level security from OS-level security. Best practices include doing away with a huge pool of all-powerful sysadmins in favor of JEA, or “just enough administration,” where sysadmins have just enough privileges to do their job, and no more. An analogous security practice for compute resources is “least privilege,” where apps and microservices can’t talk to each other unless they specifically need to do so. Audience polling supported MacDonald’s optimistic view of cloud security, which suggests that large enterprises may struggle less with their cloud policies moving forward. 7. Containers: Try ’em! Ahead of this week’s DockerCon in San Francisco, Gartner devoted significant airtime to educating the audience on containers and microservices. My summary of ‪Gartner VP and distinguished analyst Tom Bittman’s advice on containers was simple: Try ’em. Now. Complement them with VMs. ‪And Docker (the company) is important, but not the be-all and end-all in this space. Bittman (copping to some deja vu from Gartner presentations he made on server virtualization 13 years ago) noted that while virtualization has been focused on admin and ops functions, containers are focused on value for developers. But because containers are well suited for driving up VM utilization for workloads that share the same OS, we can expect to see more combinations of containers and server virtualization. Finally, Bittman underscored that Gartner doesn’t see containers having much impact on premise, but making a huge difference in the cloud. That doesn’t necessarily fit with what’s been shown in other research, such as this 2015 State of Containers Survey sponsored by VMblog.com and StackEngine, so we’ll want to watch how this plays out. This is all a lot to digest. The Gartner IT Operations Strategies & Solutions Summitacknowledges the importance of dealing with existing IT systems and practices as well as promising new technologies and thinking, and tries to point a way forward. In fact, Haight had a very good quote about microservices that I thought also served to wrap up the entire event: “If you want to run with the big dogs, you need to rethink application architecture,” he said. That can be very difficult for an enterprise to fully implement … but also very appealing. Note: Al Sargent contributed to this post. All product and company names herein may be trademarks of their registered owners. Server, tortoise and hare, business team, and cloud security images courtesy ofShutterstock.com.

I remain captivated by the ideals of Literate Programming. My fork of PyLit (https://github.com/slott56/PyLit-3) coupled with Sphinx seems to handle LP programming in a very elegant way. It works like this. Write RST files describing the problem and the solution. This includes the actual implementation code. And everything else that's relevant. Run PyLit3 to build final Python code from the RST documentation. This should include the setup.py so that it can be installed properly. Run Sphinx to build pretty HTML pages (and LaTeX) from the RST documentation. I often include the unit tests along with the sphinx build so that I'm sure that things are working. The challenge is final presentation of the whole package. The HTML can be easy to publish, but it can't (trivially) be used to recover the code. We have to upload two separate and distinct things. (We could use BeautifulSoup to recover RST from HTML and then PyLit to rebuild the code. But that sounds crazy.) The RST is easy to publish, but hard to read and it requires a pass with PyLit to emit the code and then another pass with Sphinx to produce the HTML. A single upload doesn't work well. If we publish only the Python code we've defeated the point of literate programming. Even if we focus on the Python, we need to do a separate upload of HTML to providing the supporting documentation. After working with this for a while, I've found that it's simplest to have one source and several targets. I use RST ⇒ (.py, .html, .tex). This encourages me to write documentation first. I often fail, and have blocks of code with tiny summaries and non-existent explanations. PyLit allows one to use .py ⇒ .rst ⇒ .html, .tex. I've messed with this a bit and don't like it as much. Code first leaves the documentation as a kind of afterthought. How can we publish simply and cleanly: without separate uploads? Enter GitHub and gh-pages. See the "sphinxdoc-test" project for an example. Also thishttps://github.com/daler/sphinxdoc-test. The bulk of this is useful advice on a way to create the gh-pages branch from your RST source via Sphinx and some GitHub commands. Following this line of thinking, we almost have the case for three branches in a LP project. The "master" branch with the RST source. And nothing more. The "code" branch with the generated Python code created by PyLit. The "gh-pages" branch with the generated HTML created by Sphinx. I think I like this. We need three top-level directories. One has RST source. A build script would run PyLit to populate the (separate) directory for the code branch. The build script would also run Sphinx to populate a third top-level directory for the gh-pages branch. The downside of this shows up when you need to create a branch for a separate effort. You have a "some-major-change" branch to master. Where's the code? Where's the doco? You don't want to commit either of those derived work products until you merge the "some-major-change" back into master. GitHub Literate Programming There are many LP projects on GitHub. There are perhaps a dozen which focus on publishing with the Github-flavored Markdown as the source language. Because Markdown is about as easy to parse as RST, the tooling is simple. Because Markdown lacks semantic richness, I'm not switching. I've found that semantically rich markup is essential. This is a key feature of RST. It's carried forward by Sphinx to create very sophisticated markup. Think:code:`sample` vs. :py:func:`sample` vs. :py:mod:`sample` vs.:py:exc:`sample`. The final typesetting may be similar, but they are clearly semantically distinct and create separate index entries. A focus on Markdown seems to be a limitation. It's encouraging to see folks experiment with literate programming using Markdown and GitHub. Perhaps other folks will look at more sophisticated markup languages like RST. Previous Exercises See https://sourceforge.net/projects/stingrayreader/ for a seriously large literate programming effort. The HTML is also hosted at SourceForge: http://stingrayreader.sourceforge.net/index.html. This project is awkward because -- well -- I have to do a separate FTP upload of the finished pages after a change. It's done with a script, not a simple "git push." SourceForge has a GitHub repository. https://sourceforge.net/p/stingrayreader/code/ci/master/tree/. But. SourceForge doesn't use GitHub.com's UI, so it's not clear if it supports the gh-pages feature. I assume it doesn't, but, maybe it does. (I can't even login to SourceForge with Safari... I should really stop using SourceForge and switch to GitHub.) See https://github.com/slott56/HamCalc-2.1 for another complex, LP effort. This predates my dim understanding of the gh-pages branch, so it's got HTML (in doc/build/html), but it doesn't show it elegantly. I'm still not sure this three-branch Literate Programming approach is sensible. My first step should probably be to rearrange the PyLit3 project into this three-branch structure.

New Integration Provides Seamless Connections Between Perforce Helix and Leading Application Lifecycle Management Systems WOKINGHAM, UK. (June 24, 2015) – Perforce Software, the leader in software configuration management (SCM) and collaboration, and Go2Group, an Atlassian Platinum and Enterprise Expert, today announced the Perforce ConnectALL Adapter. The new adapter for Go2Group’s ConnectALL ALM Router connects Perforce Helix to Application Lifecycle Management (ALM) systems supported by ConnectALL. The companies also announced that they have expanded their partnership, which first began in 2002. “Very few SCMs can handle binary data, and no other SCM solution supports large file formats that scale across globally distributed enterprises like Helix,” said Brett Taylor, president of Go2Group. “Our customers demand future-proof solutions, and with Perforce we know they don’t have to worry about outgrowing their systems—it will serve them well whether they’re a team of 50 or 50,000.” With the Perforce adapter, ConnectALL automatically synchronises data and workflow with other ALM systems and integrates ALM systems components within minutes. “We’re excited to be a part of the ConnectALL ecosystem of adapters and to enable companies to more easily design, configure, synchronise, manage, and monitor their integrations with Perforce,” said Dave Robertson, vice president of Channels at Perforce. “We’re glad to extend our partnership with Go2Group to new technologies and markets.” Go2Group is part of Perforce’s network of sales partners across Europe, the Middle East, Africa, Asia Pacific and India. Perforce partners serve customers in more than 100 countries worldwide. The Perforce ConnectALL Adapter is available for purchase from the Go2Group website.

Combining BioConnect with Digitus server cabinet access control makes biometric identity management more effective and easier ENTERTECH SYSTEMSandDigitus Biometricshave just announced a new technology partnership between ENTERTECH’S BioConnect identity management platform and the Digitus db Bus and db Cabinet Sentry for server cabinet access control. The result is a new, fully-integrated solution called db BioConnect. The newdb BioConnectlets company data centres, co-location data centres and IT room customers simplify biometric implementation, enrollment and management for access control of perimeter doors, interior rooms, cages and now server racks all integrated into one identity platform. This game-changing solution now extends enterprise access security platforms into the data centre, making investments in Access Control Management Systems far more economical and effective. “Digitus Biometrics is the leader in providing biometric access control to the critical infrastructure market," said Rob Douglas, ENTERTECH SYSTEMS CEO. "With Digitus technology integrated to BioConnect, all 15 of our certified access control partners will now be able to offer it to their customers. For the first time, end users will have access to an integrated biometric solution that secures access control from the data centre entrance to the server cabinet instead of having to deal with stand-alone systems.” The list of BioConnect certified access control partners can be found atwww.bioconnect.com/partners. ENTERTECH SYSTEMS’BioConnectis the most advanced identity management platform on the global market today. Simple, secure and scalable, it provides Suprema biometric authentication across leading access control systems. As an application for security professionals, BioConnect helps enterprises successfully implement identity solutions by making deployment and use of biometrics easier than ever. BioConnect addresses deployment challenges by reducing costs, overcoming complexity and making it easier to on-board users. The platform provides seamless synchronization of data such as new cardholders, changes and deletions, and is tailor-made for enterprises where true identity is critical for secure access to physical facilities and software applications. “Our biometric access control solutions are designed to meet the needs of a diverse range of clients," said David Orischak, Digitus Biometrics CEO. "This technology integration to create db BioConnect will let us offer a single, centralised, highly advanced access control solution that's easy to deploy and use. An industry first, our customers will be able to use one centralised system facility-wide to secure a company’s critical infrastructure and data." db Bus ServerRack Access Controlis designed for data centres needing to secure large volumes of server cabinets with only one small component per cabinet. A single db Bus controller allows the user to secure up to 32 racks with a single 48V power supply and may be infinitely scaled.db Cabinet Sentryis designed for data centres with a structured cabling scheme. It is Digitus’ most compact, cost-effective and energy-efficient means of securing server cabinets. This extremely versatile unit can be deployed in networked or stand-alone environments, using power over Ethernet (PoE) or an external power supply.db BioLockis a server cabinet lock that uses biometric identification with prints for up to 10 fingers per user. Through this new technology partnership, these db products will be integrated with BioConnect to create the new integrated db BioConnect solution. “Digitus’ use of leading Suprema biometric technology in their server cabinet access control solutions is a natural fit for ENTERTECH SYSTEMS as the operating partner for Suprema in the US, Canada, UK, Ireland and Puerto Rico,” added Douglas. “The implications to the market are significant," added Orischak. "The integrated db BioConnect solution can be used to manage any cabinet system where biometric access control is warranted – even SCIF’s and locked areas housing sensitive assets such as pharmaceuticals, hazardous materials, intelligence archives, customer and patient records, as well as critical IP.” For more information on the db BioConnect integrated solution, visitwww.bioconnect.com/db.

[This article was written by Ophir Prusak] Guest author Ophir Prusak is chief evangelist atBlazeMeter. To learn more about load and performance testing automation, he invites readers toattend a meetupthis Wednesday, June 24, at New Relic’s San Francisco offices. Performance and load testing are kind of like flossing your teeth. You know you need to do it, but you might not be doing it as much as you should. When your site goes down because it couldn’t handle the load, you look back and realize you might have easily prevented it with a little more testing in advance. That’s why companies are automating their application testing in an effort to lower costs, increase efficiency, and reduce the time needed to release new features. The importance of automated testing in a continuous delivery era Continuous Delivery (CD) is rapidly emerging as the “new normal” in software development, as Perforce discovered in an independent survey, with an estimated 80% of SaaS companies and 51% of non-SaaS companies adopting this practice. Companies that provide Software-as-a-Service know they need to be continuously creating new features, updating their websites, and optimizing their backend. But while software development has adapted nicely in terms of automation, the testing side has moved more slowly. For a fully Continuous Delivery and Integration process to be realized, performance testing must be automated. As the need for testing increases, doing it manually can dramatically increase your time to release. Automating testing throughout the CD process can help detect errors instantly and deliver software faster. Making it work JMeter is the de facto standard in open source load testing. It’s the most widely used open source tool for performance testing for a good reason. There’s virtually nothing it can’t test (websites, native mobile applications, APIs, and Web applications) and it’s extremely powerful and fully featured. Yet there are challenges. JMeter poses a steep learning curve in terms of integration and ease of use. Additionally, it doesn’t integrate easily with APM and Continuous Integration (CI) tools. Many developers have been looking for a way to conduct performance testing with less time and effort—and fewer hiccups along the way. Taurus: An effort to simplify test automation A new open source project called Taurus (Test AUtomation Running Smoothly) is designed to provide exactly that—a way to remove most of the pain of using JMeter on its own. Taurus can give you the ability to Create and define a load test even without using JMeter. Override existing JMeter files or tests configurations. Create human-readable configuration files and testing scripts that are easily added to source control systems like GitHub. Integrate into CI tools like Jenkins. Run multiple tests in parallel. Provide pass/fail criteria back into the CI tool for easier automation of test-results analysis. Make analysis of test results easier and more intuitive. Taurus still uses JMeter under the hood, but is designed to have a much easier learning curve, especially for simple tests. Taurus also offers a built-in result analysis engine that provides both console-based reporting features and result analysis. Performance testing and optimizing your applications is not simple, yet there are solutions available that make the process easier and more successful. I’m looking forward to seeing how the technology evolves even further in the near future. If you want to learn more about Taurus, check out the project on GitHub. Better yet, you are invited to come to a meetup this Wednesday, June 24, at New Relic’s San Francisco offices. You can learn a lot more about Taurus and how you can use it to help scale load and performance testing automation.

Microsoft’s recent announcement that Windows 10 will be the last big-bang release for the operating system was a surprise to many. But what on Earth is going on? Why have they suddenly turned round and decided to upgrade their software with a series of frequent updates instead? The answer is a term not many chief executives will be familiar with, but an important one nevertheless: continuous delivery. Continuous delivery for software applications means that new features can be released faster and companies can be more competitive. Although it requires a change in processes and an investment in the tools that make it possible, the return on investment is high – and proven. The 2014 State of DevOps Report from Puppet Labs found that high-performing IT organisations that use practices such as continuous delivery are twice as likely to exceed their profitability, market share and productivity goals. WHAT ABOUT THE DATABASE? A stumbling block many organisations find in their pursuit of continuous delivery is the database. Databases hold critical information and their stability is vital to the bottom line. Often they’re tied into those same applications that can gain from continuous delivery, but they’re seen as too risky to include in the process. In fact, the DZone Guide to Continuous Delivery 2015 showed that while 61 per cent of companies have already implemented continuous delivery for their applications, it falls to half that number when it comes to the database.The result? Application development is faster, smoother and more cost effective, while database development lags behind. “Clients such as Yorkshire Water are seeing a return on investment of 700 per cent after investing in continuous delivery for their databases” At Redgate, we’ve been working on a way to resolve the issue with a Database Lifecycle Management (DLM) solution: one that brings all the advantages of continuous delivery to the database, while protecting the data at the same time. We have some pedigree here. Our tools are already used in more than 90 per cent of Fortune 100 companies, and are trusted in areas such as finance, healthcare and manufacturing, where performance and reliability are not optional. It’s working too. Clients such as Yorkshire Water are seeing a return on investment of 700 per cent after investing in continuous delivery for their databases. Similarly, StateServ, a medical equipment supplier with customers across the United States, has adopted Redgate’s SQL Source Control and SQL Compare to reduce the time it takes to release new versions of its website by 50 per cent. Property services provider Move With Us has also adopted these tools to reduce the cost of releasing database updates by 97 per cent. As Anthony Hall, IT operations manager, says: “We spend less time deploying and more time developing better software, which keeps us ahead of our competitors. It results in a product that more closely matches what the customer wants, at a cheaper price. That equals happy customers, a happier development team and bigger profits for the company.” I love comments like that. It demonstrates that all the effort we’ve put into developing our tools and the DLM process to go with it is paying off. Not for us – for our clients. By aligning the continuous delivery of the application with the continuous delivery of the database, a company will inevitably see increased productivity, reduced risk and higher quality end-products. This suddenly means that your technology becomes a strategic advantage, as opposed to an unpredictable risk, and can be used to deliver real financial benefits to the business. CASE STUDY KEEPING DATA FLOWING FOR YORKSHIRE WATER Yorkshire Water manage the collection, treatment and distribution of water in Yorkshire, supplying around 1.24 billion litres of drinking water daily. At the same time they collect, treat and dispose of about one billion litres of waste water safely back into the environment. As might be imagined, their applications and databases are diverse and technically challenging, and deploying changes to the databases used to be manual and time consuming. The company turned to Redgate’s database development tools to automate the changes, saving time as well as avoiding errors. Using SQL Source Control and SQL Compare, they achieved every aim. In 25 days, they moved their first project from a manual deployment process to full automation and it now takes half a day to start auto-deploying a new project. As software development team leader Carl Davison says: “It’s now a minor overhead to deploy. We predict the return on investment to be in the order of 700 per cent over the next five years. We can deploy database changes as soon as the business needs them, without delays or problems.”

Originally written by David Brimley. The Death Spiral. A new feature in the 3.5 release of Hazelcast is the Cluster Quorum. In this instance we’re not talking about a Quorum in its traditional distributed systems sense, think of a Cluster Quorum as a kind of gatekeeper, protecting your cluster during times of unexpected member loss. You can use Cluster Quorums to restrict operations on Maps or indeed the entire cluster based upon environmental criteria. This sounds great you say, but I’m still not sure how this can help me? OK. Let's take a look at a scenario… Imagine a cluster that has a very high number of writes to a certain map. We also have other maps that are not updated quite so frequently and all the while we have hundreds of clients all reading from the cluster but not at the same frequency as the data that is entering the system. In normal circumstances if a machine or a number of machines were to die in the cluster we may still have enough memory available to store our data, but the amount of threads available to process requests would be reduced. We now have less cores available and the partition threads in the cluster could quickly become overwhelmed by the one map that is updated rapidly. This could mean other clients becoming starved of threads, unable to service requests. It’s also possible that the remaining members would become so consumed that they’re unable to respond to membership pings, the knock on effect could result in the member being forced out of the cluster on the assumption that it is dead. To protect the rest of the cluster in the event of member loss we need a way to stop the writes to the high frequency map whilst allowing operations to the other data structures. We can then continue to provide a good service to our other users whilst the crashed machines are restored to the cluster. Bring on the Quorum! As of Hazelcast 3.5 we now have the ability to restrict operations on distinct data structures. We do this via a Quorum configuration. We observed that other IMDG products provide Quorums that have protection at a cluster level,we decided to go one step further and provide Quorum protection around data structures as well. In the example below we create a very simple Quorum on the default map. The ‘default’ map in Hazelcast is the configuration used if no other match is found. In this instance no operations will be allowed unless the cluster has a minimum of 3 members. You’ll also note that the Quorum configuration is separate from the Map. This means that you can have multiple Quorums in a cluster attached to many different structures. If the Quorum thresholds are not satisfied then a QuorumException is thrown when we try to interact with the default map in any way. Be it from a client or another member. 3 quorumRuleWithThreeNodes Quorum Functions It’s simple to set up a Quorum check based on cluster size as we’ve seen above, but if you want to make a slightly more complex check you can do this by applying a Quorum Function. QuorumConfig quorumConfig = new QuorumConfig(); quorumConfig.setName("MyQuorum"); quorumConfig.setEnabled(true); quorumConfig.setType(QuorumType.WRITE); quorumConfig.setQuorumFunctionImplementation(new QuorumFunction() { @Override public boolean apply(Collection members) { return (members.size() >= 3) && (someOtherExternalClusterState); } }); In the example above we use Configuration API to set-up the Quorum to disallowwrites if the boolean returned from the QuorumFunction is false. In the function we test if the size of the cluster is greater than 3 and also if a variable namedsomeOtherExternalClusterState is equal true. You now get the idea that by using a function you can test for other state and not just cluster member. Listen In. Another nice feature of Quorums is the ability to listen in to Quorum Events. You can register a new callback interface called not surprisingly a QuorumListener. Quorum listeners are local to the node that they are registered, so they receive only events occurred on that local node. 3 com.company.quorum.ThreeNodeQuorumListener quorumRuleWithThreeNodes The QuorumListener has just one method that is called passing you aQuorumEvent. package com.hazelcast.quorum; import java.util.EventListener; /** * Listener to get notified when a quorum state is changed */ public interface QuorumListener extends EventListener { /** * Called when quorum presence state is changed. * * @param quorumEvent provides information about quorum presence and current member list. */ void onChange(QuorumEvent quorumEvent); } The QuorumEvent itself allows you to determine if a Quorum has been established or if it has been lost via its isPresent() method call. Additionally it provides the required cluster members to form a quorum and also the current membership list. Query the Quorums. Above we saw how we could receive callbacks, but in some cases we may just wish to make an immediate check to see if the Quorum is established or not. We can do this via the QuorumService. HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(config); QuorumService quorumService = hazelcastInstance.getQuorumService(); Quorum quorum = quorumService.getQuorum(quorumName); boolean quorumPresence = quorum.isPresent(); In Conclusion The Cluster Quorum feature is another important tool for you to manage your cluster. In future versions of Hazelcast there are plans to add other data structures, for example you’ll be able to protect operations against Topics or Queues.

[This article was written by Andrew Marshall] We’ve been talking a lot about Docker over the past few weeks—with good reason. Docker’s explosive growth in popularity within the enterprise has enabled new distributed application architectures and with it a need for app-centric monitoring of your Docker containers within the context of the rest of your infrastructure. We’re thrilled to announce today that New Relic’s Docker monitoring is now generally available to New Relic customers, just in time for DockerCon 2015! (And as we noted last week, New Relic’s Docker monitoring solution has been selected by Docker for its Ecosystem Technology Partner program as a proven container monitoring solution.) Why app-centric monitoring? If you’re a software business using Docker containers, chances are you’ve done so to gain efficiencies from your system resources or portability across environments to shorten the cycle between writing and running code. Either way, adding Docker containers to your app development meant a new tier of infrastructure to monitor, which equated to a “black box” in your data—one that you had no visibility into from a monitoring perspective, Docker monitoring with New Relic is designed to “fix” this lack of monitoring visibility by adding an app-centric view of Docker containers to the existing New Relic Servers interface you already use. Now, instead of having a gap between the application and server monitoring views, we’ve added the ability to see containers with the same “first-class“ experience as you would with virtual machines and servers. You can now drill down from the application (which is really what you care about) to the individual Docker container, and then to the physical server. No more blind spots! As we strive to do with all of our products, we took the approach of “important” over “impressive” when it comes to the container information we provide to users. Based on direct feedback from customers, we’ve tried to take the mystery out of finding the right container to help you get back to developing your applications. As the way people use containers changes over time, we plan to continue to listen to our customers to help shape how we approach Docker container monitoring. Restoring 360-degree view of your application environment One example of how app-centric monitoring can impact a team moving to microservices or distributed application environments is Motus, a mobile workforce management company. Motus has been a New Relic customer for more than four years and recently has been shifting to a microservices architecture with approximately 95% of its production workload now running in Docker containers. While Docker helpd Motus gain speed and agility while reducing infrastructure complexity, the link between the application and what was happening with the container it was running on was broken. During the trial of New Relic’s Docker monitoring, Motus was able to more easily identify which container an app was running on, all the way down to the node. That was a big help when they needed to investigate an issue and determine if a new container was required.. During the beta alone, Motus estimates that using New Relic helped them to reduce the time to investigate and fix problems with its Docker containers by 30%! Motus isn’t just using New Relic to diagnose when a problem occurs. Docker monitoring with New Relic has helped Motus analyze and “right size” its containers for the application to better allocate resources for performance and budget. Get started with New Relic’s Docker monitoring today, for more information, please stop by our booth at DockerCon, June 22-23 in San Francisco! Resources: Motus Docker Monitoring Case Study Docker Monitoring with New Relic Enabling Docker Monitoring with New Relic Docker in the New Relic Community Forum

I wanted to display my Android screen on Fedora during a Summit presentation which includes Feedhenry. I found an easy way to mirror my android screen on Fedora so I can show it through the presentation device (TV, Projector, etc.). So I compiled the steps below from some different references. Download the latest Android SDK from Google: Android SDK Extract the TGZ file to your home/YOUR-USERNAME directory To get ADB, you need to install the SDK: Installing the SDK Run chmode on android in tools Run android under tools and then install the Android SDK Tools On your phone turn on Debugging in Developer Settings, click Settings > Developer Options turn on debugging and make sure USB Debugging is on. If you are running 64-bit then to run adb you will have to enable 32-bit # yum install glibc.i686 #yum install zlib.i686 libstdc++.i686 ncurses-libs.i686 libgcc.i686 You need to add a udev rules file that contains a USB configuration for each type of device you want to use for development. In the rules file, each device manufacturer is identified by a unique vendor ID, as specified by the ATTR{idVendor} property. For a list of vendor IDs, see USB Vendor IDs, To set up device detection on Linux: Log in as root and create this file: /etc/udev/rules.d/51-android.rules. Use this format to add each vendor to the file: SUBSYSTEM=="usb", ATTR{idVendor}=="xxxx", MODE="0666" [summit2015@localhost tools]$ cat /etc/udev/rules.d/51-android.rules SUBSYSTEM=="usb", ATTR{idVendor}=="22b8", MODE="0666" [summit2015@localhost tools]$ Note: The rule syntax may vary slightly depending on your environment. Consult the udevdocumentation for your system as needed. For an overview of rule syntax, see this guide towriting udev rules. Now execute: chmod a+r /etc/udev/rules.d/51-android.rules When plugged in over USB, you can verify that your device is connected by executing adb devices from your SDK platform-tools/ directory. If connected, you'll see the device name listed as a "device." [summit2015@localhost platform-tools]$ ./adb devices List of devices attached 0A3D267016016004 device [summit2015@localhost platform-tools]$ NOTE: I ran android update adb and adb server-start to test prior to the above command but these shouldn't be required Next I download Droid@Screen and then ran java -jar droidAtScreen-1.1.jar That's all that is required!

Written by Stephane Combaudon. A question I often hear when customers want to set up a production PXC cluster is: “How many nodes should we use?” Three nodes is the most common deployment, but when are more nodes needed? They also ask: “Do we always need to use an even number of nodes?” This is what we’ll clarify in this post. This is all about quorum I explained in a previous post that a quorum vote is held each time one node becomes unreachable. With this vote, the remaining nodes will estimate whether it is safe to keep on serving queries. If quorum is not reached, all remaining nodes will set themselves in a state where they cannot process any query (even reads). To get the right size for you cluster, the only question you should answer is: how many nodes can simultaneously fail while leaving the cluster operational? If the answer is 1 node, then you need 3 nodes: when 1 node fails, the two remaining nodes have quorum. If the answer is 2 nodes, then you need 5 nodes. If the answer is 3 nodes, then you need 7 nodes. And so on and so forth. Remember that group communication is not free, so the more nodes in the cluster, the more expensive group communication will be. That’s why it would be a bad idea to have a cluster with 15 nodes for instance. In general we recommend that you talk to us if you think you need more than 10 nodes. What about an even number of nodes? The recommendation above always specifies odd number of nodes, so is there anything bad with an even number of nodes? Let’s take a 4-node cluster and see what happens if nodes fail: If 1 node fails, 3 nodes are remaining: they have quorum. If 2 nodes fail, 2 nodes are remaining: they no longer have quorum (remember 50% is NOT quorum). Conclusion: availability of a 4-node cluster is no better than the availability of a 3-node cluster, so why bother with a 4th node? The next question is: is a 4-node cluster less available than a 3-node cluster? Many people think so, specifically after reading this sentence from the manual: Clusters that have an even number of nodes risk split-brain conditions. Many people read this as “as soon as one node fails, this is a split-brain condition and the whole cluster stop working”. This is not correct! In a 4-node cluster, you can lose 1 node without any problem, exactly like in a 3-node cluster. This is not better but not worse. By the way the manual is not wrong! The sentence makes sense with its context. There could actually reasons why you might want to have an even number of nodes, but we will discuss that topic in the next section. Quorum with multiple data centers To provide more availability, spreading nodes in several datacenters is a common practice: if power fails in one DC, nodes are available elsewhere. The typical implementation is 3 nodes in 2 DCs: Notice that while this setup can handle any single node failure, it can’t handle all single DC failures: if we lose DC1, 2 nodes leave the cluster and the remaining node has not quorum. You can try with 4, 5 or any number of nodes and it will be easy to convince yourself that in all cases, losing one DC can make the whole cluster stop operating. If you want to be resilient to a single DC failure, you must have 3 DCs, for instance like this: Other considerations Sometimes other factors will make you choose a higher number of nodes. For instance, look at these requirements: All traffic is directed to a single node. The application should be able to fail over to another node in the same datacenter if possible. The cluster must keep operating even if one datacenter fails. The following architecture is an option (and yes, it has an even number of nodes!): Conclusion Regarding availability, it is easy to estimate the number of nodes you need for your PXC cluster. But node failures are not the only aspect to consider: Resilience to a datacenter failure can, for instance, influence the number of nodes you will be using.

Recently, I puzzled while looking for friendly email client for my Android device. And, the situation was not unique as I found all have their drawbacks. Let's look at the most popular e-mail clients for Android and try to find the best. As a result of wanderings in Google Play, I selected the six most popular Mail clients which are successful in my opinion. 1. Blue Mail Blue Mail is a popular e-mail client with support for a large number of postal services. It has very user-friendly interface and minimum setting - you must enter the mailbox password. After setting up the client immediately begins syncing - and after a few moments you will already see rows of folders with your letters. That's great - subfolders marked as investment, but, unfortunately, turn into the main folder cannot be all the folders displaying a single list. Appendix is perfectly adapted for touch screens - controlled via a mail is very convenient. It is possible to set the reminder time for a specific letter, the ability to customize the appearance and gestures, you can add a large number of accounts (different postal services, support for Exchange). Pros: free; There is no limit on the number of accounts (may, of course, is - but a lot; Not tied to a specific mail service; Convenient management (ie control gestures); Additional features (reminder filter to display messages in the application, configure the appearance of the application). Cons: You cannot collapse the subfolders to the main (if subfolders much, it turns out a long list, which is not convenient to work); The interface is not smooth - when scrolling and withdraw sidebars sometimes retarding. 2. myMail myMail is an excellent mail client aimed at an international audience. It’s offering quality services to the customers, minimum setting (no mail server, you need not). Completely free, not tied to a specific mail service, there is no limit to the number of connected accounts. This is management friendly and comfortable, but as such no control gestures. In the list of folders nested shifted to the right with respect to the others, in spite of that they cannot be minimized - quite convenient to use (unless, of course, subfolders are not too many). Even if the letters and added a lot of accounts - runs very smoothly. Unfortunately, there is no appearance settings, and the interface at the application seems specific enough (many may seem too "flashy"), the application has filters (for sorting mail). Pros: free; There is no limit on the number of accounts; Not tied to a specific mail service; Friendly visual interface, subfolders are shifted relative to the other; Smooth operation. Cons: Cannot be minimized subfolders in the main; Very bright interface without customization; when you reply to or forward the original formatting is lost letters. 3. Mailbox Mailbox is the popular mail client for Google and iCloud. After logging interface meets us in soothing and pleasing colors. The letters are in chronological order, depending on the strength and direction of pulling letters perform various actions - for example, just to delay the letter to the right - it is marked as executed, pull on - letter moves to the basket, if you pull the left - there also its actions. Managing convenient, but you get used - I, for example, first letter ever mistakenly moved to the Trash. It is possible to create reminders for time / date, there is an interesting sort mode analyzing user actions and, based on starts sorting the mail (of course you can create categories to sort). Application works smoothly, small braking when scrolling noticed only at the moment when there is synchronization. The app is counted in the list of best email apps for Android . Pros: Free; There is no limit on the number of accounts; Simple operation; Interesting feature of intellectual sorting mail and other options. Smoothly and fast. Cons: The ability to use only Google Accounts and iCloud; It takes time to get used to; it is impossible to beat in the category of writing more on some lines. 4. MailDroid MailDroid is a popular app with advertising and a large number of settings. On the first pages of the settings immediately struck by the quality of translation. Navigation is intuitive, but in my opinion not very convenient. Smooth operation is no different. MailDroid supports most popular email services, so configuration problems will not arise. Interface is made in soft blue and white, there are additional appearance settings. Default opens the folder "Inbox" in order to get to the other - need to go to a separate menu and there select the folder you want. The application has a large number of settings, it is possible to buy additional extension (for example, Spam plug-in), and it is possible to archive emails, have export / import settings and much more. Pros: Free; There is no limit on the number of accounts; Friendly controls; Many additional options / settings. Cons: Hype; Enlargement will have to re-buy for a fee; Despite the easy management interface, it seemed to me not very logical; It does not work smoothly. 5. Aqua Mail Aqua Mail is Conditionally free (free limited), very convenient mail client for Android. Free version of the application is limited - you can link only two e-mail accounts, plus all send letters added a signature with a link to the official website of the program. The full version can be unlocked for 4.95$ (the price at the beginning of 2015). The application meets us with a nice interface, something significant of the material design from Google, when you first turn on each step tooltips that help to understand the basic functions and features of the application. The interface is very smooth, good management is optimized for touch screens, is very convenient to work with a large number of letters. Working with folders organized as follows - the default application displays only one folder "Inbox", the others can be added manually (choose only those which are needed on your phone). There is an interesting feature, "Smart Folder" - to get the letters on any criteria require attention, naturally configured criteria (for example, letters received in the last X days). Pros: Convenient to work with a large number of letters; A variety of settings; A handy feature "Smart Folder"; Easy navigation; Smooth operation. Cons: The free version has some limitations. 6. CloudMagic CloudMagic is Simple e-mail client with the ability to password protection, a good alternative to the standard Android mail client. The application supports most of the international postal services. The controls are very simple, intuitive interface, it works smoothly. Folders are displayed all at once invested labeled as investment, but in the main folder, they do not collapse. Settings least of the features - the ability to enable password protection, longer any special settings there. pros: Free; There is no limit on the number of accounts; Simple and clear controls; It operates smoothly and quickly; The ability to password protect. cons: Few options / additional options.

if you use https to communicate with your git repository, es, github or visualstudioonline, you usually setup credential manager to avoid entering credential for each command that contact the server. with latest versions of git you can configure wincred with this simple command. git config --global credential.helper wincred this morning i start getting error while i’m trying to push some commits to github. $ git push remote: invalid username or password. fatal: authentication failed for 'https://github.com/proximosrl/jarvis.documents tore.git/' if i remove credential helper (git config –global credential.helper unset) everything works, git ask me for user name and password and i’m able to do everything, but as soon as i re-enable credential helper, the error returned. this problem is probably originated by some corruption of stored credentials, and usually you can simply clear stored credentials and at the next operation you will be prompted for credentials and everything starts worked again. the question is, where are stored credential for wincred? if you use wincred for credential.helper, git is storing your credentials in standard windows credential manager you can simply open credential manager on your computer, figure 1: credential manager in your control panel settings opening credential manager you can manage windows and web credentials. now simply have a look to both web credentials and windows credentials, and delete everything related to github or the server you are using. the next time you issue a git command that requires authentication, you will be prompted for credentials again and the credentials will be stored again in the store. gian maria.

This week we're in Boston at DevNation, an awesome, young (second ever), and relatively intimate (~500 attendees) conference on anything and everything hard-core, cool-and-hot (DevOps, big data, Angular, IoT, you name it), and of course -- since the conference is organized by Red Hat -- totally open-source. So far I've had in-depth conversations with five super-amazing engineers, attended several inspiring keynotes, and chatted with one skilled developer after another. We'll transcribe the deeper interviews shortly, including some on topics totally unrelated to this post. But meanwhile I'd like to offer some thoughts inspired by the first day of the event. The general theme is: we're just beginning to get serious about separation of concerns. The metaphor that keeps popping into my head comes from the first keynote: machines have finally grown up. Imperatives: telling really unintelligent agents what to do (and then they sort of do whatever they please) It is trivial to observe that computers are incredibly stupid. Turing's fundamental paper is about how to figure out whether a theoretical computer will keep calculating the values of a function until the heat death of the universe (okay that's a slight oversimplification). The fact that Edsger Dijkstra felt the need to rail gently against all goto statements in any higher-level language than machine code suggests that, in 1968, far too many computers needed instructions about how to read the instructions that tell them what to do in the first place. Richard Feynman's famous lecture on computer heuristics is the condescension of the man who conceived quantum computing to the level of functional composition (hmmm) and file systems (double sigh). Stupid agents need to be told exactly what to do. Then they need to be told to pay attention to the exact part of the command that tells them exactly what they have been told to do (dude, just goto line 1343 already and shut up). Then they don't do what you told them (optimistically we call this an 'exception'), and then you send them into time out / set a break point and try to figure out where the idiot state muted off the rails. They stare blankly at the wall / variable / register and either do nothing or repeat another unintelligibly wrong result until you notice that your increment is (apparently meaninglessly to you) one bracket too deep. You sigh and tell them what to do again, and after a while they hit age thirty (life-years/debug-hours) and maybe do something useful with their (process-)lives. Well, maybe I'm straining the metaphor a little here, but you get the point because it cuts too close to home. We spend far too much time fixing stupid mistakes that we didn't even know we were making because -- like all actual human beings -- we assumed that the agent we commanded will use their common sense to iron out those few whiffs of, admit it, frank nonsense that our step-by-step instructions will probably always contain. So, at least, goes the imperative programming paradigm. The machine does what you tell it to; and the universe collapses onto itself before the last real number is computed. Functions: reliable, predictable adults Time to give credit where it's due: I'm really just riffing on the metaphor Venkat Subramanian offered in his highly enjoyable keynote on The Joy of Functional Programming yesterday morning His not-so-smart agents -- the 'programmed' of imperative programming -- were toddlers. Since I don't have any kids, I can't presume to understand this experience fully (although I did grow up with three younger brothers..). But the general idea is: imperative programming is tricky because, when you spell everything out super literally, it's very hard to tell exactly why what you thought should happen didn't. Venkat's talk was a whirlwind of functional concepts, from the thrill of immutability to the self-evident utility of memoization. For random (Myers-Briggs?) reasons, the object-oriented paradigm never seemed very intuitive to me -- I've gravitated towards functional style even when the problem domain wasn't actually modeled very well by functions -- but Venkat's side-by-side implementations of simple calculations in OO and functional Java showed the readability delta very clearly. Functional code is beautiful because it looks like its purpose. It tells you flat-out: here is what I do; and then it does it. But immutable functions are also beautiful because they do exactly the same thing every time. I couldn't count on my two year old brother very much at all because given a certain input I had pretty much no idea what would come out. But we all count on our grown-up collaborators to output exactly what they should, given a definite input, predictably and reliably every time. Of course, people also do more than expected -- every intervention of intelligence is an injection of creativity, not generated by the definition of the function -- but at least they do what you need them to do and no less. Containers: grown-ups with good boundaries I'm picking out just one aspect of the resurgent 'joy' of functional programming because the renaissance of containerization (another 'old' technology that is just now really taking off) is, I think, a part of the same shift toward, let's say, treating computers as adults. If functions are reliable agents, then applications in well-defined containers are self-sufficient agents who know exactly what they need from others and neither require nor demand anything more. If apps on dedicated VMs are teenagers negotiating personal boundaries by waking/booting up independently (and taking far too long -- and far too many resources -- to do so, given their meager output) -- or bubble boys, isolated in ways that are unfortunate in order to isolate in ways that are absolutely necessary -- then containerized applications are subway-riders who jam into the train without offending anyone or campers who can live anywhere with just a backpack of just the stuff they need. Of course, subway-riders and campers do more than just not-mess-up. But what's kind of neat about containers is that -- like an adult with good boundaries -- clearly defined bounds and interfaces free up the application / mind to do whatever world-changing thing the developer / human has cooked up. I'll come back to this metaphor in a later article. (Mesh networks, SDN, and ad-hoc computing are all part of the same picture, I think. Kubernetes probably is too, along with event-driven and reactive programming, the actor model, dreams of Smalltalk, and of course REST, at least of the HATEOAS flavor.) But maybe this isn't a good way to think about some of these recent sparks in devworld within a single paradigm -- and maybe my perpetual discomfort with OO is influencing me too much. What do you think?

[This article was written by Kevin Casey] Welcome to This Week in Modern Software, orTWiMS, New Relic’s weekly roundup of the need-to-know news, stories, and events of interest surrounding software analytics, cloud computing, application monitoring, development methodologies, programming languages, and the myriad of other issues that influence modern software. This week, our top story goes inside President Obama’s secret team of tech geeks, 140 of them and counting: TWiMS Top Story: Inside Obama’s Stealth Startup—Fast Company What it’s about:If the President of the United States walked into the room and personally recruited you to rebuild the country’s technology infrastructure, could you turn him down? He’s serious, and that room is theRoosevelt Room in the West Wing of the White House, by the way. AsLisa Gelobtersays: “What are you going to say that?” Gelobter’s answer was “Yes”—she’s now chief digital officer for the US Department of Education, part of a 140-person-and-counting tech team that’s functioning something like an elite startup embedded inside the federal government. Its business? Only modernizing the technical infrastructure, applications, and processes of just about every federal agency. Why you should care:What was once something of a tech desert—the federal government—is beginning to draw top private-sector talent inside the Beltway. The team, led by Mikey Dickerson (who helped lead the team that rescuedHealthcare.gov) andformer US CTO Todd Park, also includes the likes of former Googler Matthew Weaver, and it hopes to hit 500 people by the end 2016, shortly before President Obama will leave office. Its challenges are immense, from tackling government bureaucracy (to test just how entrenched the suits were, Weaver requested the official title “Rogue Leader”—and he got it) to the fact that its recruiting pitch includes the phrase: “You’ll have to take a pay cut.” But its mission is both noble and necessary, and the appeal of working on major problems with enormous public impacts appears to be working. Recommended reading. Further reading: Mikey Dickerson’s 10 Tips for Dealing with Bureaucracy—New Relic Blog [Video] Airbnb Open Sources Software to Lure Talent Amid ‘Insane’ Competition—CIO Journal What it’s about:Airbnb added three new apps to its open source portfolio earlier this month, but the motivation wasn’t just trying to give employees the best business tools or contribute to the software community at large. Sure, that might have been part of the equation, but the rental booking site hopes open-sourcing some of its toolkit will help recruit the best software talent in the face of what director of engineeringMike Curtiscalls “insane” competition in the Silicon Valley labor market. Why you should care:In the software arms race, any little edge counts. Curtis tellsCIO Journalthat Airbnb will keep the proprietary stuff closely guarded, of course. But it will open source “generic” tools with wider industry use cases, such as its recently releasedAerosolvemachine-learning package and itsAirpalcloud-based data querying tool. The latter, which works with Facebook’s open sourcePrestoDB, aims to simplify SQL queries to the point where you don’t need to be a big data wonk or business intelligence guru to run it. Indeed, one in three Airbnb employees have run a query on it in the year since it launched. Airbnb has contributed a dozen open source tools on its aptly namedNerds site(gotta love that!) to date, something the company hopes both contributes to greater good but also advertises its software innovation to potential hires. Google Is Wielding Its Own Secret Weapon in the Cloud—The New York Times What it’s about:In thecutthroat competitionfor public cloud business, Google may be its own best customer testimonial. In advance of this week’sOpen Network Summit, theTimes’Bits bloglooked at Google’s plan to not only unveil cloud customers such as HTC but reveal much more than ever before about its own infrastructure. Google did just that on Wednesday, offering a look inside itsdata center networking, including its massive-capacity, lightning-fast Jupiter network. Why you should care:As major cloud players continue to zap prices with their shrink-rays, it’s increasingly clear that features and underlying platforms will distinguish one from the other when enterprise users make their pick. Google is taking a big step toward writing its own story in this regard, and the synopsis might read something like: “We’re pretty good at this stuff.” Its Jupiter fabrics deliver 1 petabit per second of bisection bandwidth, according to Google, or “enough for 100,000 servers to exchange information at 10Gb/s each, enough to read the entire scanned contents of the Library of Congress in less than 1/10th of a second.” If it sounds like a bit of bragging, well, yeah—it is. But it’s bragging with a purpose: Attracting devs who want access to the same technology without having to build it themselves.Google’s Amin Vahdat connected the dots in a blog post: “The same networks that power all of Google’s internal infrastructure and services also power Google Cloud Platform.” Move Over, Meeker: Byron Deeter’s State of the Cloud Report—Bessemer Venture Partners What it’s about:With a nod to Mary Meeker’s classicState of the Internet report,Bessemer Venture Partners’Byron Deeterchecks in with his 2015 State of the Cloud Report. Given cloud computing’s relative youth and rampant ascension, it’s no surprise the stats are staggering. Here’s one to start: Cloud revenues have increased tenfold in the last six years, from a scant $5.6 billion in 2008 to more than $56 billion in 2014. And it’s going to double again in the next four years, according to BVP’s projections, to $127.5 billion in 2018. Why you should care:Deeter’s full presentation is worth a weekend watch or read, but it’s the forward-looking slides that may be most compelling for software pros. Deeter notes both the immense risks and opportunities in cloud security, unveiling a 10-point security plan for cloud startups on slide 37. To underscore the security landscape, Deeter quotes an unnamed cloud CEO who says aDDoSattack that took down the firm’s API caused more customer churn in one day than in the rest of its history. Wow. He also addresses the exploding market for cloud services built specifically for developers including, yes, New Relic. And for mobile developers, slide 44 underscores something we’ve talked about before in this space:the real money’s in enterprise apps, and it’s still a largely untapped market. Click through thefull slide deck hereorwatch video of Deeter’s presentation here. Bandwidth: The Next Frontier of Cloud Computing—ZDnet What it’s about:Is networking the next big thing in the everything-as-a-service age? It just might be, as firms likePacnetvie to deliver networking capacity on a pay-for-what-you-use model that some industry folks say better suits cloud environments facing significant but uneven networking needs. Why you should care:As author Drew Turney notes, there’s a common blind spot when it comes to cloud computing’s many shapes and sizes: Moving all that data from points A to Z, and everywhere in between, which can cause both performance problems and undue financial pressures. The promise of Networking-as-a-Service (NaaS), industry execs tell Turney, is that it can provide more efficient, scalable networking for short-term usage bursts such as customer traffic spikes or large cloud backup-and-storage jobs, enabling companies to later dial down their capacity as needed. Combined withSoftware-Defined Networking (SDN),NaaS makes it possible to build intelligent applications that manage their own networking needs, which might be the most significant enterprise potential of NaaS, saysNuage NetworksarchitectMarten Hauville. Page Bloat: Average Web Page Now More Than 2MB—The Performance Beacon (SOASTA) What it’s about:Do you need to put your website on a diet? Apparently so: The average Web page topped 2 MB as of May 2015, according to ongoing tracking atThe Performance Beacon. That’s double the average page weight from just three years ago. The site projects average page weight will exceed 3 MB in late 2017. Why you should care:Performance, performance, performance:Slow speedsare a killerin the modern software era. While author andSOASTAUX evangelistTammy Evertsrightly notes that page weight is not the only factor in Web optimization, we’re simply not paying it enough attention when designing and building Web pages. Images are the big culprit in the Web’s expanding waistline: they comprise nearly two-thirds of the average page’s weight, and video is a growing part of our Web diet, too. But other factors such as custom fonts play a role, adding weight even as the Web sheds previous performance hogs like Flash. The ideal weight? 1 MB, she says, which will save crucial seconds in load times. Sounds like it’s time to hit the virtual treadmill.