New Integration Provides Seamless Connections Between Perforce Helix and Leading Application Lifecycle Management Systems WOKINGHAM, UK. (June 24, 2015) – Perforce Software, the leader in software configuration management (SCM) and collaboration, and Go2Group, an Atlassian Platinum and Enterprise Expert, today announced the Perforce ConnectALL Adapter. The new adapter for Go2Group’s ConnectALL ALM Router connects Perforce Helix to Application Lifecycle Management (ALM) systems supported by ConnectALL. The companies also announced that they have expanded their partnership, which first began in 2002. “Very few SCMs can handle binary data, and no other SCM solution supports large file formats that scale across globally distributed enterprises like Helix,” said Brett Taylor, president of Go2Group. “Our customers demand future-proof solutions, and with Perforce we know they don’t have to worry about outgrowing their systems—it will serve them well whether they’re a team of 50 or 50,000.” With the Perforce adapter, ConnectALL automatically synchronises data and workflow with other ALM systems and integrates ALM systems components within minutes. “We’re excited to be a part of the ConnectALL ecosystem of adapters and to enable companies to more easily design, configure, synchronise, manage, and monitor their integrations with Perforce,” said Dave Robertson, vice president of Channels at Perforce. “We’re glad to extend our partnership with Go2Group to new technologies and markets.” Go2Group is part of Perforce’s network of sales partners across Europe, the Middle East, Africa, Asia Pacific and India. Perforce partners serve customers in more than 100 countries worldwide. The Perforce ConnectALL Adapter is available for purchase from the Go2Group website.

Combining BioConnect with Digitus server cabinet access control makes biometric identity management more effective and easier ENTERTECH SYSTEMSandDigitus Biometricshave just announced a new technology partnership between ENTERTECH’S BioConnect identity management platform and the Digitus db Bus and db Cabinet Sentry for server cabinet access control. The result is a new, fully-integrated solution called db BioConnect. The newdb BioConnectlets company data centres, co-location data centres and IT room customers simplify biometric implementation, enrollment and management for access control of perimeter doors, interior rooms, cages and now server racks all integrated into one identity platform. This game-changing solution now extends enterprise access security platforms into the data centre, making investments in Access Control Management Systems far more economical and effective. “Digitus Biometrics is the leader in providing biometric access control to the critical infrastructure market," said Rob Douglas, ENTERTECH SYSTEMS CEO. "With Digitus technology integrated to BioConnect, all 15 of our certified access control partners will now be able to offer it to their customers. For the first time, end users will have access to an integrated biometric solution that secures access control from the data centre entrance to the server cabinet instead of having to deal with stand-alone systems.” The list of BioConnect certified access control partners can be found atwww.bioconnect.com/partners. ENTERTECH SYSTEMS’BioConnectis the most advanced identity management platform on the global market today. Simple, secure and scalable, it provides Suprema biometric authentication across leading access control systems. As an application for security professionals, BioConnect helps enterprises successfully implement identity solutions by making deployment and use of biometrics easier than ever. BioConnect addresses deployment challenges by reducing costs, overcoming complexity and making it easier to on-board users. The platform provides seamless synchronization of data such as new cardholders, changes and deletions, and is tailor-made for enterprises where true identity is critical for secure access to physical facilities and software applications. “Our biometric access control solutions are designed to meet the needs of a diverse range of clients," said David Orischak, Digitus Biometrics CEO. "This technology integration to create db BioConnect will let us offer a single, centralised, highly advanced access control solution that's easy to deploy and use. An industry first, our customers will be able to use one centralised system facility-wide to secure a company’s critical infrastructure and data." db Bus ServerRack Access Controlis designed for data centres needing to secure large volumes of server cabinets with only one small component per cabinet. A single db Bus controller allows the user to secure up to 32 racks with a single 48V power supply and may be infinitely scaled.db Cabinet Sentryis designed for data centres with a structured cabling scheme. It is Digitus’ most compact, cost-effective and energy-efficient means of securing server cabinets. This extremely versatile unit can be deployed in networked or stand-alone environments, using power over Ethernet (PoE) or an external power supply.db BioLockis a server cabinet lock that uses biometric identification with prints for up to 10 fingers per user. Through this new technology partnership, these db products will be integrated with BioConnect to create the new integrated db BioConnect solution. “Digitus’ use of leading Suprema biometric technology in their server cabinet access control solutions is a natural fit for ENTERTECH SYSTEMS as the operating partner for Suprema in the US, Canada, UK, Ireland and Puerto Rico,” added Douglas. “The implications to the market are significant," added Orischak. "The integrated db BioConnect solution can be used to manage any cabinet system where biometric access control is warranted – even SCIF’s and locked areas housing sensitive assets such as pharmaceuticals, hazardous materials, intelligence archives, customer and patient records, as well as critical IP.” For more information on the db BioConnect integrated solution, visitwww.bioconnect.com/db.

This technical tip explains how to .NET developers can split and Merge PST files using Aspose.Email. Aspose.Email API provides the capability to split a single PST file into multiple PST files of desired file size. It can also merge multiple PST files into a single PST file. Both the splitting and merging of PSTs operations can be tracked by adding events to these operations. Aspose.Email for .NET is a set of components allowing developers to easily implement email functionality within their ASP.NET web applications, web services & Windows applications. It Supports Outlook PST, EML, MSG & MHT formats. //Code sample for Splitting a Single PST into multiple PSTs [C#] using (PersonalStorage pst = PersonalStorage.FromFile(@"D:\test\source.pst")) { // The events subscription is an optional step for the tracking process only. pst.StorageProcessed += PstSplit_OnStorageProcessed; pst.ItemMoved += PstSplit_OnItemMoved; // Splits into pst chunks with the size of 5mb pst.SplitInto(5000000, @"D:\test\chunks\"); } [VB.NET] Using pst As PersonalStorage = PersonalStorage.FromFile("D:\test\source.pst") ' The events subscription is an optional step for the tracking process only. pst.StorageProcessed += PstSplit_OnStorageProcessed pst.ItemMoved += PstSplit_OnItemMoved ' Splits into pst chunks with the size of 5mb pst.SplitInto(5000000, "D:\test\chunks\") //Code sample for Merging of Multiple PSTs into a single PST [C#] totalAdded = 0; using (PersonalStorage pst = PersonalStorage.FromFile(@"D:\test\destination.pst")) { // The events subscription is an optional step for the tracking process only. pst.StorageProcessed += PstMerge_OnStorageProcessed; pst.ItemMoved += PstMerge_OnItemMoved; // Merges with the pst files that are located in separate folder. pst.MergeWith(Directory.GetFiles(@"D:\test\sources\")); Console.WriteLine("Total messages added: {0}", totalAdded); } [VB.NET] totalAdded = 0 Using pst As PersonalStorage = PersonalStorage.FromFile("D:\test\destination.pst") ' The events subscription is an optional step for the tracking process only. pst.StorageProcessed += PstMerge_OnStorageProcessed pst.ItemMoved += PstMerge_OnItemMoved ' Merges with the pst files that are located in separate folder. pst.MergeWith(Directory.GetFiles("D:\test\sources\")) Console.WriteLine("Total messages added: {0}", totalAdded) End Using //Code sample Merging Folders from another PST [C#] totalAdded = 0; using (PersonalStorage destinationPst = PersonalStorage.FromFile(@"D:\test\destination.pst")) using (PersonalStorage sourcePst = PersonalStorage.FromFile(@"D:\test\source.pst")) { FolderInfo destinationFolder = destinationPst.RootFolder.AddSubFolder("FolderFromAnotherPst"); FolderInfo sourceFolder = sourcePst.GetPredefinedFolder(StandardIpmFolder.DeletedItems); // The events subscription is an optional step for the tracking process only. destinationFolder.ItemMoved += destinationFolder_ItemMoved; // Merges with the folder from another pst. destinationFolder.MergeWith(sourceFolder); Console.WriteLine("Total messages added: {0}", totalAdded); } [VB.NET] totalAdded = 0 Using destinationPst As PersonalStorage = PersonalStorage.FromFile("D:\test\destination.pst") Using sourcePst As PersonalStorage = PersonalStorage.FromFile("D:\test\source.pst") Dim destinationFolder As FolderInfo = destinationPst.RootFolder.AddSubFolder("FolderFromAnotherPst")t Dim sourceFolder As FolderInfo = sourcePst.GetPredefinedFolder(StandardIpmFolder.DeletedItems) ' The events subscription is an optional step for the tracking process only. destinationFolder.ItemMoved += destinationFolder_ItemMoved ' Merges with the folder from another pst. destinationFolder.MergeWith(sourceFolder) Console.WriteLine("Total messages added: {0}", totalAdded) End Using End Using //Helping Methods void destinationFolder_ItemMoved(object sender, ItemMovedEventArgs e) { totalAdded++; } void PstMerge_OnStorageProcessed(object sender, StorageProcessedEventArgs e) { Console.WriteLine("*** The storage is merging: {0}", e.FileName); } void PstMerge_OnItemMoved(object sender, ItemMovedEventArgs e) { if (currentFolder == null) { currentFolder = e.DestinationFolder.RetrieveFullPath(); } string folderPath = e.DestinationFolder.RetrieveFullPath(); if (currentFolder != folderPath) { Console.WriteLine(" Added {0} messages to \"{1}\"", messageCount, currentFolder); messageCount = 0; currentFolder = folderPath; } messageCount++; totalAdded++; } void PstSplit_OnStorageProcessed(object sender, StorageProcessedEventArgs e) { if (currentFolder != null) { Console.WriteLine(" Added {0} messages to \"{1}\"", messageCount, currentFolder); } messageCount = 0; currentFolder = null; Console.WriteLine("*** The chunk is processed: {0}", e.FileName); } void PstSplit_OnItemMoved(object sender, ItemMovedEventArgs e) { if (currentFolder == null) { currentFolder = e.DestinationFolder.RetrieveFullPath(); } string folderPath = e.DestinationFolder.RetrieveFullPath(); if (currentFolder != folderPath) { Console.WriteLine(" Added {0} messages to \"{1}\"", messageCount, currentFolder); messageCount = 0; currentFolder = folderPath; } messageCount++; }

Downloading various files (either text or binary) is a bread and butter of every enterprise application. PDF documents, attachments, media, executables, CSV, very large files, etc. Almost every application, sooner or later, will have to provide some form of download. Downloading is implemented in terms of HTTP, so it's important to fully embrace this protocol and take full advantage of it. Especially in Internet facing applications features like caching or user experience are worth considering. This series of articles provides a list of aspects that you might want to consider when implementing all sorts of download servers. Note that I avoid "best practices" term, these are just guidelines that I find useful but are not necessarily always applicable. One of the biggest scalability issues is loading whole file into memory before streaming it. Loading full file into byte[] to later return it e.g. from Spring MVC controller is unpredictable and doesn't scale. The amount of memory your server will consume depends linearly on number of concurrent connections times average file size - factors you don't really want to depend on so much. It's extremely easy to stream contents of a file directly from your server to the client byte-by-byte (with buffering), there are actually many techniques to achieve that. The easiest one is to copy bytes manually: @RequestMapping(method = GET) public void download(OutputStream output) throws IOException { try(final InputStream myFile = openFile()) { IOUtils.copy(myFile, output); } } Your InputStream doesn't even have to be buffered, IOUtils.copy() will take care of that. However this implementation is rather low-level and hard to unit test. Instead I suggest returning Resource: @RestController @RequestMapping("/download") public class DownloadController { private final FileStorage storage; @Autowired public DownloadController(FileStorage storage) { this.storage = storage; } @RequestMapping(method = GET, value = "/{uuid}") public Resource download(@PathVariable UUID uuid) { return storage .findFile(uuid) .map(this::prepareResponse) .orElseGet(this::notFound); } private Resource prepareResponse(FilePointer filePointer) { final InputStream inputStream = filePointer.open(); return new InputStreamResource(inputStream); } private Resource notFound() { throw new NotFoundException(); } } @ResponseStatus(value= HttpStatus.NOT_FOUND) public class NotFoundException extends RuntimeException { } Two abstractions were created to decouple Spring controller from file storage mechanism.FilePointer is a file descriptor, irrespective to where that file was taken. Currently we use one method from it: public interface FilePointer { InputStream open(); //more to come } open() allows reading the actual file, no matter where it comes from (file system, database BLOB, Amazon S3, etc.) We will gradually extend FilePointer to support more advanced features, like file size and MIME type. The process of finding and creatingFilePointers is governed by FileStorage abstraction: public interface FileStorage { Optional findFile(UUID uuid); } Streaming allows us to handle hundreds of concurrent requests without significant impact on memory and GC (only a small buffer is allocated in IOUtils). BTW I am using UUID to identify files rather than names or other form of sequence number. This makes it harder to guess individual resource names, thus more secure (obscure). More on that in next articles. Having this basic setup we can reliably serve lots of concurrent connections with minimal impact on memory. Remember that many components in Spring framework and other libraries (e.g. servlet filters) may buffer full response before returning it. Therefore it's really important to have an integration test trying to download huge file (in tens of GiB) and making sure the application doesn't crash. Writing a download server Part I: Always stream, never keep fully in memory Part II: headers: Last-Modified, ETag and If-None-Match Part III: headers: Content-length and Range Part IV: Implement HEAD operation (efficiently) Part V: Throttle download speed Part VI: Describe what you send (Content-type, et.al.) The sample application developed throughout these articles is available on GitHub.

[This article was written by Ophir Prusak] Guest author Ophir Prusak is chief evangelist atBlazeMeter. To learn more about load and performance testing automation, he invites readers toattend a meetupthis Wednesday, June 24, at New Relic’s San Francisco offices. Performance and load testing are kind of like flossing your teeth. You know you need to do it, but you might not be doing it as much as you should. When your site goes down because it couldn’t handle the load, you look back and realize you might have easily prevented it with a little more testing in advance. That’s why companies are automating their application testing in an effort to lower costs, increase efficiency, and reduce the time needed to release new features. The importance of automated testing in a continuous delivery era Continuous Delivery (CD) is rapidly emerging as the “new normal” in software development, as Perforce discovered in an independent survey, with an estimated 80% of SaaS companies and 51% of non-SaaS companies adopting this practice. Companies that provide Software-as-a-Service know they need to be continuously creating new features, updating their websites, and optimizing their backend. But while software development has adapted nicely in terms of automation, the testing side has moved more slowly. For a fully Continuous Delivery and Integration process to be realized, performance testing must be automated. As the need for testing increases, doing it manually can dramatically increase your time to release. Automating testing throughout the CD process can help detect errors instantly and deliver software faster. Making it work JMeter is the de facto standard in open source load testing. It’s the most widely used open source tool for performance testing for a good reason. There’s virtually nothing it can’t test (websites, native mobile applications, APIs, and Web applications) and it’s extremely powerful and fully featured. Yet there are challenges. JMeter poses a steep learning curve in terms of integration and ease of use. Additionally, it doesn’t integrate easily with APM and Continuous Integration (CI) tools. Many developers have been looking for a way to conduct performance testing with less time and effort—and fewer hiccups along the way. Taurus: An effort to simplify test automation A new open source project called Taurus (Test AUtomation Running Smoothly) is designed to provide exactly that—a way to remove most of the pain of using JMeter on its own. Taurus can give you the ability to Create and define a load test even without using JMeter. Override existing JMeter files or tests configurations. Create human-readable configuration files and testing scripts that are easily added to source control systems like GitHub. Integrate into CI tools like Jenkins. Run multiple tests in parallel. Provide pass/fail criteria back into the CI tool for easier automation of test-results analysis. Make analysis of test results easier and more intuitive. Taurus still uses JMeter under the hood, but is designed to have a much easier learning curve, especially for simple tests. Taurus also offers a built-in result analysis engine that provides both console-based reporting features and result analysis. Performance testing and optimizing your applications is not simple, yet there are solutions available that make the process easier and more successful. I’m looking forward to seeing how the technology evolves even further in the near future. If you want to learn more about Taurus, check out the project on GitHub. Better yet, you are invited to come to a meetup this Wednesday, June 24, at New Relic’s San Francisco offices. You can learn a lot more about Taurus and how you can use it to help scale load and performance testing automation.

Microsoft’s recent announcement that Windows 10 will be the last big-bang release for the operating system was a surprise to many. But what on Earth is going on? Why have they suddenly turned round and decided to upgrade their software with a series of frequent updates instead? The answer is a term not many chief executives will be familiar with, but an important one nevertheless: continuous delivery. Continuous delivery for software applications means that new features can be released faster and companies can be more competitive. Although it requires a change in processes and an investment in the tools that make it possible, the return on investment is high – and proven. The 2014 State of DevOps Report from Puppet Labs found that high-performing IT organisations that use practices such as continuous delivery are twice as likely to exceed their profitability, market share and productivity goals. WHAT ABOUT THE DATABASE? A stumbling block many organisations find in their pursuit of continuous delivery is the database. Databases hold critical information and their stability is vital to the bottom line. Often they’re tied into those same applications that can gain from continuous delivery, but they’re seen as too risky to include in the process. In fact, the DZone Guide to Continuous Delivery 2015 showed that while 61 per cent of companies have already implemented continuous delivery for their applications, it falls to half that number when it comes to the database.The result? Application development is faster, smoother and more cost effective, while database development lags behind. “Clients such as Yorkshire Water are seeing a return on investment of 700 per cent after investing in continuous delivery for their databases” At Redgate, we’ve been working on a way to resolve the issue with a Database Lifecycle Management (DLM) solution: one that brings all the advantages of continuous delivery to the database, while protecting the data at the same time. We have some pedigree here. Our tools are already used in more than 90 per cent of Fortune 100 companies, and are trusted in areas such as finance, healthcare and manufacturing, where performance and reliability are not optional. It’s working too. Clients such as Yorkshire Water are seeing a return on investment of 700 per cent after investing in continuous delivery for their databases. Similarly, StateServ, a medical equipment supplier with customers across the United States, has adopted Redgate’s SQL Source Control and SQL Compare to reduce the time it takes to release new versions of its website by 50 per cent. Property services provider Move With Us has also adopted these tools to reduce the cost of releasing database updates by 97 per cent. As Anthony Hall, IT operations manager, says: “We spend less time deploying and more time developing better software, which keeps us ahead of our competitors. It results in a product that more closely matches what the customer wants, at a cheaper price. That equals happy customers, a happier development team and bigger profits for the company.” I love comments like that. It demonstrates that all the effort we’ve put into developing our tools and the DLM process to go with it is paying off. Not for us – for our clients. By aligning the continuous delivery of the application with the continuous delivery of the database, a company will inevitably see increased productivity, reduced risk and higher quality end-products. This suddenly means that your technology becomes a strategic advantage, as opposed to an unpredictable risk, and can be used to deliver real financial benefits to the business. CASE STUDY KEEPING DATA FLOWING FOR YORKSHIRE WATER Yorkshire Water manage the collection, treatment and distribution of water in Yorkshire, supplying around 1.24 billion litres of drinking water daily. At the same time they collect, treat and dispose of about one billion litres of waste water safely back into the environment. As might be imagined, their applications and databases are diverse and technically challenging, and deploying changes to the databases used to be manual and time consuming. The company turned to Redgate’s database development tools to automate the changes, saving time as well as avoiding errors. Using SQL Source Control and SQL Compare, they achieved every aim. In 25 days, they moved their first project from a manual deployment process to full automation and it now takes half a day to start auto-deploying a new project. As software development team leader Carl Davison says: “It’s now a minor overhead to deploy. We predict the return on investment to be in the order of 700 per cent over the next five years. We can deploy database changes as soon as the business needs them, without delays or problems.”

Originally written by David Brimley. The Death Spiral. A new feature in the 3.5 release of Hazelcast is the Cluster Quorum. In this instance we’re not talking about a Quorum in its traditional distributed systems sense, think of a Cluster Quorum as a kind of gatekeeper, protecting your cluster during times of unexpected member loss. You can use Cluster Quorums to restrict operations on Maps or indeed the entire cluster based upon environmental criteria. This sounds great you say, but I’m still not sure how this can help me? OK. Let's take a look at a scenario… Imagine a cluster that has a very high number of writes to a certain map. We also have other maps that are not updated quite so frequently and all the while we have hundreds of clients all reading from the cluster but not at the same frequency as the data that is entering the system. In normal circumstances if a machine or a number of machines were to die in the cluster we may still have enough memory available to store our data, but the amount of threads available to process requests would be reduced. We now have less cores available and the partition threads in the cluster could quickly become overwhelmed by the one map that is updated rapidly. This could mean other clients becoming starved of threads, unable to service requests. It’s also possible that the remaining members would become so consumed that they’re unable to respond to membership pings, the knock on effect could result in the member being forced out of the cluster on the assumption that it is dead. To protect the rest of the cluster in the event of member loss we need a way to stop the writes to the high frequency map whilst allowing operations to the other data structures. We can then continue to provide a good service to our other users whilst the crashed machines are restored to the cluster. Bring on the Quorum! As of Hazelcast 3.5 we now have the ability to restrict operations on distinct data structures. We do this via a Quorum configuration. We observed that other IMDG products provide Quorums that have protection at a cluster level,we decided to go one step further and provide Quorum protection around data structures as well. In the example below we create a very simple Quorum on the default map. The ‘default’ map in Hazelcast is the configuration used if no other match is found. In this instance no operations will be allowed unless the cluster has a minimum of 3 members. You’ll also note that the Quorum configuration is separate from the Map. This means that you can have multiple Quorums in a cluster attached to many different structures. If the Quorum thresholds are not satisfied then a QuorumException is thrown when we try to interact with the default map in any way. Be it from a client or another member. 3 quorumRuleWithThreeNodes Quorum Functions It’s simple to set up a Quorum check based on cluster size as we’ve seen above, but if you want to make a slightly more complex check you can do this by applying a Quorum Function. QuorumConfig quorumConfig = new QuorumConfig(); quorumConfig.setName("MyQuorum"); quorumConfig.setEnabled(true); quorumConfig.setType(QuorumType.WRITE); quorumConfig.setQuorumFunctionImplementation(new QuorumFunction() { @Override public boolean apply(Collection members) { return (members.size() >= 3) && (someOtherExternalClusterState); } }); In the example above we use Configuration API to set-up the Quorum to disallowwrites if the boolean returned from the QuorumFunction is false. In the function we test if the size of the cluster is greater than 3 and also if a variable namedsomeOtherExternalClusterState is equal true. You now get the idea that by using a function you can test for other state and not just cluster member. Listen In. Another nice feature of Quorums is the ability to listen in to Quorum Events. You can register a new callback interface called not surprisingly a QuorumListener. Quorum listeners are local to the node that they are registered, so they receive only events occurred on that local node. 3 com.company.quorum.ThreeNodeQuorumListener quorumRuleWithThreeNodes The QuorumListener has just one method that is called passing you aQuorumEvent. package com.hazelcast.quorum; import java.util.EventListener; /** * Listener to get notified when a quorum state is changed */ public interface QuorumListener extends EventListener { /** * Called when quorum presence state is changed. * * @param quorumEvent provides information about quorum presence and current member list. */ void onChange(QuorumEvent quorumEvent); } The QuorumEvent itself allows you to determine if a Quorum has been established or if it has been lost via its isPresent() method call. Additionally it provides the required cluster members to form a quorum and also the current membership list. Query the Quorums. Above we saw how we could receive callbacks, but in some cases we may just wish to make an immediate check to see if the Quorum is established or not. We can do this via the QuorumService. HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(config); QuorumService quorumService = hazelcastInstance.getQuorumService(); Quorum quorum = quorumService.getQuorum(quorumName); boolean quorumPresence = quorum.isPresent(); In Conclusion The Cluster Quorum feature is another important tool for you to manage your cluster. In future versions of Hazelcast there are plans to add other data structures, for example you’ll be able to protect operations against Topics or Queues.

[This article was written by Andrew Marshall] We’ve been talking a lot about Docker over the past few weeks—with good reason. Docker’s explosive growth in popularity within the enterprise has enabled new distributed application architectures and with it a need for app-centric monitoring of your Docker containers within the context of the rest of your infrastructure. We’re thrilled to announce today that New Relic’s Docker monitoring is now generally available to New Relic customers, just in time for DockerCon 2015! (And as we noted last week, New Relic’s Docker monitoring solution has been selected by Docker for its Ecosystem Technology Partner program as a proven container monitoring solution.) Why app-centric monitoring? If you’re a software business using Docker containers, chances are you’ve done so to gain efficiencies from your system resources or portability across environments to shorten the cycle between writing and running code. Either way, adding Docker containers to your app development meant a new tier of infrastructure to monitor, which equated to a “black box” in your data—one that you had no visibility into from a monitoring perspective, Docker monitoring with New Relic is designed to “fix” this lack of monitoring visibility by adding an app-centric view of Docker containers to the existing New Relic Servers interface you already use. Now, instead of having a gap between the application and server monitoring views, we’ve added the ability to see containers with the same “first-class“ experience as you would with virtual machines and servers. You can now drill down from the application (which is really what you care about) to the individual Docker container, and then to the physical server. No more blind spots! As we strive to do with all of our products, we took the approach of “important” over “impressive” when it comes to the container information we provide to users. Based on direct feedback from customers, we’ve tried to take the mystery out of finding the right container to help you get back to developing your applications. As the way people use containers changes over time, we plan to continue to listen to our customers to help shape how we approach Docker container monitoring. Restoring 360-degree view of your application environment One example of how app-centric monitoring can impact a team moving to microservices or distributed application environments is Motus, a mobile workforce management company. Motus has been a New Relic customer for more than four years and recently has been shifting to a microservices architecture with approximately 95% of its production workload now running in Docker containers. While Docker helpd Motus gain speed and agility while reducing infrastructure complexity, the link between the application and what was happening with the container it was running on was broken. During the trial of New Relic’s Docker monitoring, Motus was able to more easily identify which container an app was running on, all the way down to the node. That was a big help when they needed to investigate an issue and determine if a new container was required.. During the beta alone, Motus estimates that using New Relic helped them to reduce the time to investigate and fix problems with its Docker containers by 30%! Motus isn’t just using New Relic to diagnose when a problem occurs. Docker monitoring with New Relic has helped Motus analyze and “right size” its containers for the application to better allocate resources for performance and budget. Get started with New Relic’s Docker monitoring today, for more information, please stop by our booth at DockerCon, June 22-23 in San Francisco! Resources: Motus Docker Monitoring Case Study Docker Monitoring with New Relic Enabling Docker Monitoring with New Relic Docker in the New Relic Community Forum

I wanted to display my Android screen on Fedora during a Summit presentation which includes Feedhenry. I found an easy way to mirror my android screen on Fedora so I can show it through the presentation device (TV, Projector, etc.). So I compiled the steps below from some different references. Download the latest Android SDK from Google: Android SDK Extract the TGZ file to your home/YOUR-USERNAME directory To get ADB, you need to install the SDK: Installing the SDK Run chmode on android in tools Run android under tools and then install the Android SDK Tools On your phone turn on Debugging in Developer Settings, click Settings > Developer Options turn on debugging and make sure USB Debugging is on. If you are running 64-bit then to run adb you will have to enable 32-bit # yum install glibc.i686 #yum install zlib.i686 libstdc++.i686 ncurses-libs.i686 libgcc.i686 You need to add a udev rules file that contains a USB configuration for each type of device you want to use for development. In the rules file, each device manufacturer is identified by a unique vendor ID, as specified by the ATTR{idVendor} property. For a list of vendor IDs, see USB Vendor IDs, To set up device detection on Linux: Log in as root and create this file: /etc/udev/rules.d/51-android.rules. Use this format to add each vendor to the file: SUBSYSTEM=="usb", ATTR{idVendor}=="xxxx", MODE="0666" [summit2015@localhost tools]$ cat /etc/udev/rules.d/51-android.rules SUBSYSTEM=="usb", ATTR{idVendor}=="22b8", MODE="0666" [summit2015@localhost tools]$ Note: The rule syntax may vary slightly depending on your environment. Consult the udevdocumentation for your system as needed. For an overview of rule syntax, see this guide towriting udev rules. Now execute: chmod a+r /etc/udev/rules.d/51-android.rules When plugged in over USB, you can verify that your device is connected by executing adb devices from your SDK platform-tools/ directory. If connected, you'll see the device name listed as a "device." [summit2015@localhost platform-tools]$ ./adb devices List of devices attached 0A3D267016016004 device [summit2015@localhost platform-tools]$ NOTE: I ran android update adb and adb server-start to test prior to the above command but these shouldn't be required Next I download Droid@Screen and then ran java -jar droidAtScreen-1.1.jar That's all that is required!

Written by Stephane Combaudon. A question I often hear when customers want to set up a production PXC cluster is: “How many nodes should we use?” Three nodes is the most common deployment, but when are more nodes needed? They also ask: “Do we always need to use an even number of nodes?” This is what we’ll clarify in this post. This is all about quorum I explained in a previous post that a quorum vote is held each time one node becomes unreachable. With this vote, the remaining nodes will estimate whether it is safe to keep on serving queries. If quorum is not reached, all remaining nodes will set themselves in a state where they cannot process any query (even reads). To get the right size for you cluster, the only question you should answer is: how many nodes can simultaneously fail while leaving the cluster operational? If the answer is 1 node, then you need 3 nodes: when 1 node fails, the two remaining nodes have quorum. If the answer is 2 nodes, then you need 5 nodes. If the answer is 3 nodes, then you need 7 nodes. And so on and so forth. Remember that group communication is not free, so the more nodes in the cluster, the more expensive group communication will be. That’s why it would be a bad idea to have a cluster with 15 nodes for instance. In general we recommend that you talk to us if you think you need more than 10 nodes. What about an even number of nodes? The recommendation above always specifies odd number of nodes, so is there anything bad with an even number of nodes? Let’s take a 4-node cluster and see what happens if nodes fail: If 1 node fails, 3 nodes are remaining: they have quorum. If 2 nodes fail, 2 nodes are remaining: they no longer have quorum (remember 50% is NOT quorum). Conclusion: availability of a 4-node cluster is no better than the availability of a 3-node cluster, so why bother with a 4th node? The next question is: is a 4-node cluster less available than a 3-node cluster? Many people think so, specifically after reading this sentence from the manual: Clusters that have an even number of nodes risk split-brain conditions. Many people read this as “as soon as one node fails, this is a split-brain condition and the whole cluster stop working”. This is not correct! In a 4-node cluster, you can lose 1 node without any problem, exactly like in a 3-node cluster. This is not better but not worse. By the way the manual is not wrong! The sentence makes sense with its context. There could actually reasons why you might want to have an even number of nodes, but we will discuss that topic in the next section. Quorum with multiple data centers To provide more availability, spreading nodes in several datacenters is a common practice: if power fails in one DC, nodes are available elsewhere. The typical implementation is 3 nodes in 2 DCs: Notice that while this setup can handle any single node failure, it can’t handle all single DC failures: if we lose DC1, 2 nodes leave the cluster and the remaining node has not quorum. You can try with 4, 5 or any number of nodes and it will be easy to convince yourself that in all cases, losing one DC can make the whole cluster stop operating. If you want to be resilient to a single DC failure, you must have 3 DCs, for instance like this: Other considerations Sometimes other factors will make you choose a higher number of nodes. For instance, look at these requirements: All traffic is directed to a single node. The application should be able to fail over to another node in the same datacenter if possible. The cluster must keep operating even if one datacenter fails. The following architecture is an option (and yes, it has an even number of nodes!): Conclusion Regarding availability, it is easy to estimate the number of nodes you need for your PXC cluster. But node failures are not the only aspect to consider: Resilience to a datacenter failure can, for instance, influence the number of nodes you will be using.

Recently, I puzzled while looking for friendly email client for my Android device. And, the situation was not unique as I found all have their drawbacks. Let's look at the most popular e-mail clients for Android and try to find the best. As a result of wanderings in Google Play, I selected the six most popular Mail clients which are successful in my opinion. 1. Blue Mail Blue Mail is a popular e-mail client with support for a large number of postal services. It has very user-friendly interface and minimum setting - you must enter the mailbox password. After setting up the client immediately begins syncing - and after a few moments you will already see rows of folders with your letters. That's great - subfolders marked as investment, but, unfortunately, turn into the main folder cannot be all the folders displaying a single list. Appendix is perfectly adapted for touch screens - controlled via a mail is very convenient. It is possible to set the reminder time for a specific letter, the ability to customize the appearance and gestures, you can add a large number of accounts (different postal services, support for Exchange). Pros: free; There is no limit on the number of accounts (may, of course, is - but a lot; Not tied to a specific mail service; Convenient management (ie control gestures); Additional features (reminder filter to display messages in the application, configure the appearance of the application). Cons: You cannot collapse the subfolders to the main (if subfolders much, it turns out a long list, which is not convenient to work); The interface is not smooth - when scrolling and withdraw sidebars sometimes retarding. 2. myMail myMail is an excellent mail client aimed at an international audience. It’s offering quality services to the customers, minimum setting (no mail server, you need not). Completely free, not tied to a specific mail service, there is no limit to the number of connected accounts. This is management friendly and comfortable, but as such no control gestures. In the list of folders nested shifted to the right with respect to the others, in spite of that they cannot be minimized - quite convenient to use (unless, of course, subfolders are not too many). Even if the letters and added a lot of accounts - runs very smoothly. Unfortunately, there is no appearance settings, and the interface at the application seems specific enough (many may seem too "flashy"), the application has filters (for sorting mail). Pros: free; There is no limit on the number of accounts; Not tied to a specific mail service; Friendly visual interface, subfolders are shifted relative to the other; Smooth operation. Cons: Cannot be minimized subfolders in the main; Very bright interface without customization; when you reply to or forward the original formatting is lost letters. 3. Mailbox Mailbox is the popular mail client for Google and iCloud. After logging interface meets us in soothing and pleasing colors. The letters are in chronological order, depending on the strength and direction of pulling letters perform various actions - for example, just to delay the letter to the right - it is marked as executed, pull on - letter moves to the basket, if you pull the left - there also its actions. Managing convenient, but you get used - I, for example, first letter ever mistakenly moved to the Trash. It is possible to create reminders for time / date, there is an interesting sort mode analyzing user actions and, based on starts sorting the mail (of course you can create categories to sort). Application works smoothly, small braking when scrolling noticed only at the moment when there is synchronization. The app is counted in the list of best email apps for Android . Pros: Free; There is no limit on the number of accounts; Simple operation; Interesting feature of intellectual sorting mail and other options. Smoothly and fast. Cons: The ability to use only Google Accounts and iCloud; It takes time to get used to; it is impossible to beat in the category of writing more on some lines. 4. MailDroid MailDroid is a popular app with advertising and a large number of settings. On the first pages of the settings immediately struck by the quality of translation. Navigation is intuitive, but in my opinion not very convenient. Smooth operation is no different. MailDroid supports most popular email services, so configuration problems will not arise. Interface is made in soft blue and white, there are additional appearance settings. Default opens the folder "Inbox" in order to get to the other - need to go to a separate menu and there select the folder you want. The application has a large number of settings, it is possible to buy additional extension (for example, Spam plug-in), and it is possible to archive emails, have export / import settings and much more. Pros: Free; There is no limit on the number of accounts; Friendly controls; Many additional options / settings. Cons: Hype; Enlargement will have to re-buy for a fee; Despite the easy management interface, it seemed to me not very logical; It does not work smoothly. 5. Aqua Mail Aqua Mail is Conditionally free (free limited), very convenient mail client for Android. Free version of the application is limited - you can link only two e-mail accounts, plus all send letters added a signature with a link to the official website of the program. The full version can be unlocked for 4.95$ (the price at the beginning of 2015). The application meets us with a nice interface, something significant of the material design from Google, when you first turn on each step tooltips that help to understand the basic functions and features of the application. The interface is very smooth, good management is optimized for touch screens, is very convenient to work with a large number of letters. Working with folders organized as follows - the default application displays only one folder "Inbox", the others can be added manually (choose only those which are needed on your phone). There is an interesting feature, "Smart Folder" - to get the letters on any criteria require attention, naturally configured criteria (for example, letters received in the last X days). Pros: Convenient to work with a large number of letters; A variety of settings; A handy feature "Smart Folder"; Easy navigation; Smooth operation. Cons: The free version has some limitations. 6. CloudMagic CloudMagic is Simple e-mail client with the ability to password protection, a good alternative to the standard Android mail client. The application supports most of the international postal services. The controls are very simple, intuitive interface, it works smoothly. Folders are displayed all at once invested labeled as investment, but in the main folder, they do not collapse. Settings least of the features - the ability to enable password protection, longer any special settings there. pros: Free; There is no limit on the number of accounts; Simple and clear controls; It operates smoothly and quickly; The ability to password protect. cons: Few options / additional options.

if you use https to communicate with your git repository, es, github or visualstudioonline, you usually setup credential manager to avoid entering credential for each command that contact the server. with latest versions of git you can configure wincred with this simple command. git config --global credential.helper wincred this morning i start getting error while i’m trying to push some commits to github. $ git push remote: invalid username or password. fatal: authentication failed for 'https://github.com/proximosrl/jarvis.documents tore.git/' if i remove credential helper (git config –global credential.helper unset) everything works, git ask me for user name and password and i’m able to do everything, but as soon as i re-enable credential helper, the error returned. this problem is probably originated by some corruption of stored credentials, and usually you can simply clear stored credentials and at the next operation you will be prompted for credentials and everything starts worked again. the question is, where are stored credential for wincred? if you use wincred for credential.helper, git is storing your credentials in standard windows credential manager you can simply open credential manager on your computer, figure 1: credential manager in your control panel settings opening credential manager you can manage windows and web credentials. now simply have a look to both web credentials and windows credentials, and delete everything related to github or the server you are using. the next time you issue a git command that requires authentication, you will be prompted for credentials again and the credentials will be stored again in the store. gian maria.

This week we're in Boston at DevNation, an awesome, young (second ever), and relatively intimate (~500 attendees) conference on anything and everything hard-core, cool-and-hot (DevOps, big data, Angular, IoT, you name it), and of course -- since the conference is organized by Red Hat -- totally open-source. So far I've had in-depth conversations with five super-amazing engineers, attended several inspiring keynotes, and chatted with one skilled developer after another. We'll transcribe the deeper interviews shortly, including some on topics totally unrelated to this post. But meanwhile I'd like to offer some thoughts inspired by the first day of the event. The general theme is: we're just beginning to get serious about separation of concerns. The metaphor that keeps popping into my head comes from the first keynote: machines have finally grown up. Imperatives: telling really unintelligent agents what to do (and then they sort of do whatever they please) It is trivial to observe that computers are incredibly stupid. Turing's fundamental paper is about how to figure out whether a theoretical computer will keep calculating the values of a function until the heat death of the universe (okay that's a slight oversimplification). The fact that Edsger Dijkstra felt the need to rail gently against all goto statements in any higher-level language than machine code suggests that, in 1968, far too many computers needed instructions about how to read the instructions that tell them what to do in the first place. Richard Feynman's famous lecture on computer heuristics is the condescension of the man who conceived quantum computing to the level of functional composition (hmmm) and file systems (double sigh). Stupid agents need to be told exactly what to do. Then they need to be told to pay attention to the exact part of the command that tells them exactly what they have been told to do (dude, just goto line 1343 already and shut up). Then they don't do what you told them (optimistically we call this an 'exception'), and then you send them into time out / set a break point and try to figure out where the idiot state muted off the rails. They stare blankly at the wall / variable / register and either do nothing or repeat another unintelligibly wrong result until you notice that your increment is (apparently meaninglessly to you) one bracket too deep. You sigh and tell them what to do again, and after a while they hit age thirty (life-years/debug-hours) and maybe do something useful with their (process-)lives. Well, maybe I'm straining the metaphor a little here, but you get the point because it cuts too close to home. We spend far too much time fixing stupid mistakes that we didn't even know we were making because -- like all actual human beings -- we assumed that the agent we commanded will use their common sense to iron out those few whiffs of, admit it, frank nonsense that our step-by-step instructions will probably always contain. So, at least, goes the imperative programming paradigm. The machine does what you tell it to; and the universe collapses onto itself before the last real number is computed. Functions: reliable, predictable adults Time to give credit where it's due: I'm really just riffing on the metaphor Venkat Subramanian offered in his highly enjoyable keynote on The Joy of Functional Programming yesterday morning His not-so-smart agents -- the 'programmed' of imperative programming -- were toddlers. Since I don't have any kids, I can't presume to understand this experience fully (although I did grow up with three younger brothers..). But the general idea is: imperative programming is tricky because, when you spell everything out super literally, it's very hard to tell exactly why what you thought should happen didn't. Venkat's talk was a whirlwind of functional concepts, from the thrill of immutability to the self-evident utility of memoization. For random (Myers-Briggs?) reasons, the object-oriented paradigm never seemed very intuitive to me -- I've gravitated towards functional style even when the problem domain wasn't actually modeled very well by functions -- but Venkat's side-by-side implementations of simple calculations in OO and functional Java showed the readability delta very clearly. Functional code is beautiful because it looks like its purpose. It tells you flat-out: here is what I do; and then it does it. But immutable functions are also beautiful because they do exactly the same thing every time. I couldn't count on my two year old brother very much at all because given a certain input I had pretty much no idea what would come out. But we all count on our grown-up collaborators to output exactly what they should, given a definite input, predictably and reliably every time. Of course, people also do more than expected -- every intervention of intelligence is an injection of creativity, not generated by the definition of the function -- but at least they do what you need them to do and no less. Containers: grown-ups with good boundaries I'm picking out just one aspect of the resurgent 'joy' of functional programming because the renaissance of containerization (another 'old' technology that is just now really taking off) is, I think, a part of the same shift toward, let's say, treating computers as adults. If functions are reliable agents, then applications in well-defined containers are self-sufficient agents who know exactly what they need from others and neither require nor demand anything more. If apps on dedicated VMs are teenagers negotiating personal boundaries by waking/booting up independently (and taking far too long -- and far too many resources -- to do so, given their meager output) -- or bubble boys, isolated in ways that are unfortunate in order to isolate in ways that are absolutely necessary -- then containerized applications are subway-riders who jam into the train without offending anyone or campers who can live anywhere with just a backpack of just the stuff they need. Of course, subway-riders and campers do more than just not-mess-up. But what's kind of neat about containers is that -- like an adult with good boundaries -- clearly defined bounds and interfaces free up the application / mind to do whatever world-changing thing the developer / human has cooked up. I'll come back to this metaphor in a later article. (Mesh networks, SDN, and ad-hoc computing are all part of the same picture, I think. Kubernetes probably is too, along with event-driven and reactive programming, the actor model, dreams of Smalltalk, and of course REST, at least of the HATEOAS flavor.) But maybe this isn't a good way to think about some of these recent sparks in devworld within a single paradigm -- and maybe my perpetual discomfort with OO is influencing me too much. What do you think?

[This article was written by Kevin Casey] Welcome to This Week in Modern Software, orTWiMS, New Relic’s weekly roundup of the need-to-know news, stories, and events of interest surrounding software analytics, cloud computing, application monitoring, development methodologies, programming languages, and the myriad of other issues that influence modern software. This week, our top story goes inside President Obama’s secret team of tech geeks, 140 of them and counting: TWiMS Top Story: Inside Obama’s Stealth Startup—Fast Company What it’s about:If the President of the United States walked into the room and personally recruited you to rebuild the country’s technology infrastructure, could you turn him down? He’s serious, and that room is theRoosevelt Room in the West Wing of the White House, by the way. AsLisa Gelobtersays: “What are you going to say that?” Gelobter’s answer was “Yes”—she’s now chief digital officer for the US Department of Education, part of a 140-person-and-counting tech team that’s functioning something like an elite startup embedded inside the federal government. Its business? Only modernizing the technical infrastructure, applications, and processes of just about every federal agency. Why you should care:What was once something of a tech desert—the federal government—is beginning to draw top private-sector talent inside the Beltway. The team, led by Mikey Dickerson (who helped lead the team that rescuedHealthcare.gov) andformer US CTO Todd Park, also includes the likes of former Googler Matthew Weaver, and it hopes to hit 500 people by the end 2016, shortly before President Obama will leave office. Its challenges are immense, from tackling government bureaucracy (to test just how entrenched the suits were, Weaver requested the official title “Rogue Leader”—and he got it) to the fact that its recruiting pitch includes the phrase: “You’ll have to take a pay cut.” But its mission is both noble and necessary, and the appeal of working on major problems with enormous public impacts appears to be working. Recommended reading. Further reading: Mikey Dickerson’s 10 Tips for Dealing with Bureaucracy—New Relic Blog [Video] Airbnb Open Sources Software to Lure Talent Amid ‘Insane’ Competition—CIO Journal What it’s about:Airbnb added three new apps to its open source portfolio earlier this month, but the motivation wasn’t just trying to give employees the best business tools or contribute to the software community at large. Sure, that might have been part of the equation, but the rental booking site hopes open-sourcing some of its toolkit will help recruit the best software talent in the face of what director of engineeringMike Curtiscalls “insane” competition in the Silicon Valley labor market. Why you should care:In the software arms race, any little edge counts. Curtis tellsCIO Journalthat Airbnb will keep the proprietary stuff closely guarded, of course. But it will open source “generic” tools with wider industry use cases, such as its recently releasedAerosolvemachine-learning package and itsAirpalcloud-based data querying tool. The latter, which works with Facebook’s open sourcePrestoDB, aims to simplify SQL queries to the point where you don’t need to be a big data wonk or business intelligence guru to run it. Indeed, one in three Airbnb employees have run a query on it in the year since it launched. Airbnb has contributed a dozen open source tools on its aptly namedNerds site(gotta love that!) to date, something the company hopes both contributes to greater good but also advertises its software innovation to potential hires. Google Is Wielding Its Own Secret Weapon in the Cloud—The New York Times What it’s about:In thecutthroat competitionfor public cloud business, Google may be its own best customer testimonial. In advance of this week’sOpen Network Summit, theTimes’Bits bloglooked at Google’s plan to not only unveil cloud customers such as HTC but reveal much more than ever before about its own infrastructure. Google did just that on Wednesday, offering a look inside itsdata center networking, including its massive-capacity, lightning-fast Jupiter network. Why you should care:As major cloud players continue to zap prices with their shrink-rays, it’s increasingly clear that features and underlying platforms will distinguish one from the other when enterprise users make their pick. Google is taking a big step toward writing its own story in this regard, and the synopsis might read something like: “We’re pretty good at this stuff.” Its Jupiter fabrics deliver 1 petabit per second of bisection bandwidth, according to Google, or “enough for 100,000 servers to exchange information at 10Gb/s each, enough to read the entire scanned contents of the Library of Congress in less than 1/10th of a second.” If it sounds like a bit of bragging, well, yeah—it is. But it’s bragging with a purpose: Attracting devs who want access to the same technology without having to build it themselves.Google’s Amin Vahdat connected the dots in a blog post: “The same networks that power all of Google’s internal infrastructure and services also power Google Cloud Platform.” Move Over, Meeker: Byron Deeter’s State of the Cloud Report—Bessemer Venture Partners What it’s about:With a nod to Mary Meeker’s classicState of the Internet report,Bessemer Venture Partners’Byron Deeterchecks in with his 2015 State of the Cloud Report. Given cloud computing’s relative youth and rampant ascension, it’s no surprise the stats are staggering. Here’s one to start: Cloud revenues have increased tenfold in the last six years, from a scant $5.6 billion in 2008 to more than $56 billion in 2014. And it’s going to double again in the next four years, according to BVP’s projections, to $127.5 billion in 2018. Why you should care:Deeter’s full presentation is worth a weekend watch or read, but it’s the forward-looking slides that may be most compelling for software pros. Deeter notes both the immense risks and opportunities in cloud security, unveiling a 10-point security plan for cloud startups on slide 37. To underscore the security landscape, Deeter quotes an unnamed cloud CEO who says aDDoSattack that took down the firm’s API caused more customer churn in one day than in the rest of its history. Wow. He also addresses the exploding market for cloud services built specifically for developers including, yes, New Relic. And for mobile developers, slide 44 underscores something we’ve talked about before in this space:the real money’s in enterprise apps, and it’s still a largely untapped market. Click through thefull slide deck hereorwatch video of Deeter’s presentation here. Bandwidth: The Next Frontier of Cloud Computing—ZDnet What it’s about:Is networking the next big thing in the everything-as-a-service age? It just might be, as firms likePacnetvie to deliver networking capacity on a pay-for-what-you-use model that some industry folks say better suits cloud environments facing significant but uneven networking needs. Why you should care:As author Drew Turney notes, there’s a common blind spot when it comes to cloud computing’s many shapes and sizes: Moving all that data from points A to Z, and everywhere in between, which can cause both performance problems and undue financial pressures. The promise of Networking-as-a-Service (NaaS), industry execs tell Turney, is that it can provide more efficient, scalable networking for short-term usage bursts such as customer traffic spikes or large cloud backup-and-storage jobs, enabling companies to later dial down their capacity as needed. Combined withSoftware-Defined Networking (SDN),NaaS makes it possible to build intelligent applications that manage their own networking needs, which might be the most significant enterprise potential of NaaS, saysNuage NetworksarchitectMarten Hauville. Page Bloat: Average Web Page Now More Than 2MB—The Performance Beacon (SOASTA) What it’s about:Do you need to put your website on a diet? Apparently so: The average Web page topped 2 MB as of May 2015, according to ongoing tracking atThe Performance Beacon. That’s double the average page weight from just three years ago. The site projects average page weight will exceed 3 MB in late 2017. Why you should care:Performance, performance, performance:Slow speedsare a killerin the modern software era. While author andSOASTAUX evangelistTammy Evertsrightly notes that page weight is not the only factor in Web optimization, we’re simply not paying it enough attention when designing and building Web pages. Images are the big culprit in the Web’s expanding waistline: they comprise nearly two-thirds of the average page’s weight, and video is a growing part of our Web diet, too. But other factors such as custom fonts play a role, adding weight even as the Web sheds previous performance hogs like Flash. The ideal weight? 1 MB, she says, which will save crucial seconds in load times. Sounds like it’s time to hit the virtual treadmill.

Last month we shared the news about the debut Opsmatic and their live state monitoring service, a solution that delivers a precise, real-time picture of the detailed configuration – as well as changes that affect a computing infrastructure. We wanted to let you know that Opsmatic continues to expand its service capabilities with the announcement of two new versions. The Enterprise version of the Opsmatic service includes all the features we discussed last month in their Professional edition with the addition of single sign-on and dedicated support. The On-Premises edition embodies features contained in the Enterprise version in a solution designed for customers with an isolated infrastructure who require an internally-deployed solution. Details of Opsmatic’s new service offerings are outlined in the press release below. Should you have any questions, Opsmatic would be happy to respond to them.

London, UK – FusionExperience, the business and data solutions provider, today announces the success of its first salesforce.com partnership with Cloud Consulting Ltd. (CCL). CCL was working with an international airline client to migrate a legacy charter and group booking application from one Salesforce.com instance to a new one. Very early on in the project CCL discovered that there were considerable elements of unsupported custom code and that these had to be redesigned and redeveloped. The airline took the opportunity at this stage to request changes and improve the application in line with their new business processes. CCL worked with FusionExperience to migrate the application to the latest salesforce.com environment and re-architected the booking engine functionality and complex pricing algorithms using Apex and VisualForce. For business reasons the airline had a strict project deadline and despite all the unknowns involved the project timescales were maintained and FusionExperience delivered on time and to budget. The airline went live with the application on schedule without any post-production problems or warranty fixes required. They now have an up to date system that has achieved a game changing transformation in the way it does business. Robin James, Platform Evangelist for FusionExperience said; “The ability to seamlessly work with our partners on salesforce.com projects enables rapid scaling of resources and capabilities. This ensures that the client is delighted by the results, yet unaware of the complex extended ecosystem that has been involved. This is facilitated by that fact that we all speak the same salesforce.com language. Cloud Consulting is an ideal partner to work with in this way, as our delivery and technical strengths are well matched with their intimate client facing approach.” Tim Pullen, Managing Director of CCL added: “We already had a close relationship with FusionExperience and it was natural for us to turn to them for help with this suddenly extremely challenging project. The combination of cleaning, segmenting and splitting the data in Salesforce.com, extracting the system configuration and custom code and then creating a new system was tough enough to start but then having to redevelop the application from scratch took it to a new level. Right from the start Robin James and his team took everything in their stride and provided a level of comfort, reassurance, skill and professionalism that we’d never experienced before from other partners. Bear in mind that the old system had no user or technical documentation plus undocumented code and you begin to understand just how good the end result has been for the airline. Thank you Fusion!”

From the DevNation Agenda site: You've probably heard a lot about Linux containers and the exciting potential they hold. In this presentation, Matt Hicks will cover how Docker and Kubernetes have evolved to fundamentally change how you will approach development and operations. If you are looking for an understanding of the technology and how it relates to the common roles in IT today, this is the talk to watch. Speaker: Matt Hicks -- Vice President of engineering, Red Hat Matt Hicks is a founding member of the OpenShift by Red Hat team. He has spent more than a decade in software engineering, with a variety of roles in development, operations, architecture, and management. His real expertise is in bridging the gap between developing code and actually running it in production. An expert in IT and cloud-based architectures, he spends his time these days evolving OpenShift to use the power of cloud and make developers more productive.

Welcome to Useful PC Guide, we are covering latest technology news with many topics on computing, mobile, programming, technology, computer games, games, mobile games, Apple iOS, and Android apps as well as online tutorials, guides and how-to articles. UsefulPCGuide.com website also regularly updates new Windows OS tips and tricks to resolve your problems, as well as iOS and Android issues. You can read an example tutorial from us about how to fix your connection is not private error in Google Chrome in Windows OS. This guide will help you to learn more about causes of this error, and appropriate ways to troubleshoot the issues on your Chrome browser. Most of our tips and tricks are include images and very easy to read and follow up the instructions. Visit usefulguide.com for more good news and tutorials.

You will aggregate a lot of logs over the lifetime of your product and codebase, so it’s important to be able to search through them. In the rare case of a security issue, not having that capability is incredibly painful. You might be able to use services that allow you to search through the logs of the last two weeks quickly. But what if you want to search through the last six months, a year, or even further? That availability can be rather expensive or not even an option at all with existing services. Many hosted log services provide S3 archival support which we can use to build a long-term log analysis infrastructure with AWS Redshift. Recently I’ve set up scripts to be able to create that infrastructure whenever we need it at Codeship. AWS Redshift AWS Redshift is a data warehousing solution by AWS. It has an easy clustering and ingestion mechanism ideal for loading large log files and then searching through them with SQL. As it automatically balances your log files across several machines, you can easily scale up if you need more speed. As I said earlier, looking through large amounts of log files is a relatively rare occasion; you don’t need this infrastructure to be around all the time, which makes it a perfect use case for AWS. Setting Up Your Log Analysis Let’s walk through the scripts that drive our long-term log analysis infrastructure. You can check them out in the flomotlik/redshift-logging GitHub repository. I’ll take you step by step through configuring the whole setup of the environment variables needed, as well as starting the creation of the cluster and searching the logs. But first, let’s get a high-level overview of what the setup script is doing before going into all the different options that you can set: Creates an AWS Redshift cluster. You can configure the number of servers and which server type should be used. Waits for the cluster to become ready. Creates a SQL table inside the Redshift cluster to load the log files into. Ingests all log files into the Redshift cluster from AWS S3. Cleans up the database and prints the psql access command to connect into the cluster. Be sure to check out the script on GitHub before we go into all the different options that you can set through the .env file. Options to set The following is a list of all the options available to you. You can simply copy the .env.template file to .env and then fill in all the options to get picked up. AWS_ACCESS_KEY_ID AWS key of the account that should run the Redshift cluster. AWS_SECRET_ACCESS_KEY AWS secret key of the account that should run the Redshift cluster. AWS_REGION=us-east-1 AWS region the cluster should run in, default us-east-1. Make sure to use the same region that is used for archiving your logs to S3 to have them close. REDSHIFT_USERNAME Username to connect with psql into the cluster. REDSHIFT_PASSWORD Password to connect with psql into the cluster. S3_AWS_ACCESS_KEY_ID AWS key that has access to the S3 bucket you want to pull your logs from. We run the log analysis cluster in our AWS Sandbox account but pull the logs from our production AWS account so the Redshift cluster doesn’t impact production in any way. S3_AWS_SECRET_ACCESS_KEY AWS secret key that has access to the S3 bucket you want to pull your logs from. PORT=5439 Port to connect to with psql. CLUSTER_TYPE=single-node The cluster type can be single-node or multi-node. Multi-node clusters get auto-balanced which gives you more speed at a higher cost. NODE_TYPE Instance type that’s used for the nodes of the cluster. Check out the Redshift Documentation for details on the instance types and their differences. NUMBER_OF_NODES=10 Number of nodes when running in multi-mode. CLUSTER_IDENTIFIER=log-analysis DB_NAME=log-analysis S3_PATH=s3://your_s3_bucket/papertrail/logs/862693/dt=2015 Database format and failed loads When ingesting log statements into the cluster, make sure to check the amount of failed loads that are happening. You might have to edit the database format to fit to your specific log output style. You can debug this easily by creating a single-node cluster first that only loads a small subset of your logs and is very fast as a result. Make sure to have none or nearly no failed loads before you extend to the whole cluster. In case there are issues, check out the documentation of the copy command which loads your logs into the database and the parameters in the setup script for that. Example and benchmarks It’s a quick thing to set up the whole cluster and run example queries against it. For example, I’ll load all of our logs of the last nine months into a Redshift cluster and run several queries against it. I haven’t spent any time on optimizing the table, but you could definitely gain some more speed out of the whole system if necessary. It’s just fast enough already for us out of the box. As you can see here, loading all logs of May — more than 600 million log lines — took only 12 minutes on a cluster of 10 machines. We could easily load more than one month into that 10-machine cluster since there’s more than enough storage available, but for this post, one month is enough. After that, we’re able to search through the history of all of our applications and past servers through SQL. We connect with our psql client and send of SQL queries against the “events’ database. For example, what if we want to know how many build servers reported logs in May: loganalysis=# select count(distinct(source_name)) from events where source_name LIKE 'i-%'; count ------- 801 (1 row) So in May, we had 801 EC2 build servers running for our customers. That query took ~3 seconds to finish. Or let’s say we want to know how many people accessed the configuration page of our main repository (the project ID is hidden with XXXX): loganalysis=# select count(*) from events where source_name = 'mothership' and program LIKE 'app/web%' and message LIKE 'method=GET path=/projects/XXXX/configure_tests%'; count ------- 15 (1 row) So now we know that there were 15 accesses on that configuration page throughout May. We can also get all the details, including who accessed it when through our logs. This could help in case of any security issues we’d need to look into. The query took about 40 seconds to go though all of our logs, but it could be optimized on Redshift even more. Those are just some of the queries you could use to look through your logs, gaining more insight into your customers’ use of your system. And you et all of that with a setup that costs $2.50 an hour, can be shut down immediately, and recreated any time you need access to that data again. Conclusions Being able to search through and learn from your history is incredibly important for building a large infrastructure. You need to be able to look into your history easily, especially when it comes to security issues. With AWS Redshift, you have a great tool in hand that allows you to start an ad hoc analytics infrastructure that’s fast and cheap for short-term reviews. Of course, Redshift can do a lot more as well. Let us know what your processes and tools around logging, storage, and search are in the comments.

Written by Mark Pollack. Today, we are pleased to announce the general availability of Spring XD 1.2, Spring XD 1.1.3 and the release of Flo for Spring XD Beta. 1.2.0.GA: zip 1.1.3.RELEASE: zip Flo for Spring XD Beta You can also install XD 1.2 using brew and rpm The 1.2 release includes a wide range of new features and improvements. The release journey was an eventful one, mainly due to Spring XD’s popularity with so many different groups, each with their respective request priorities. However the Spring XD team rose to the challenge and it is rewarding to look back and review the amount of innovation delivered to meet our commitments toward simplifying big data complexity. Here is a summary of what we have been busy with for the last 3 months and the value created for the community and our customers. Flo for Spring XD and UI improvements Flo for Spring XD is an HTML5 canvas application that runs on top of the Spring XD runtime, offering a graphical interface for creation, management and monitoring streaming data pipelines. Here is a short screencast showing you how to build an advanced stream definition. You can browse the documentation for additional information and links to additional screen casts of Flo in action. The XD admin screen also includes a new Analytics section that allows you to easily view gauges, counters, field-value counters and aggregate counters. Performance Improvements Anticipating increased high-throughput and low-latency IoT requirements, we’ve made several performance optimizations within the underlying message-bus implementation to deliver several million messages per second transported between Spring XD containers using Kafka as a transport. With these optimizations, we are now on par with the performance from Kafka’s own testing tools. However, we are using the more feature rich Spring Integration Kafka client instead of Kafka’s high level consumer library. For anyone who is interested in reproducing these numbers, please refer to the XD benchmarking blog, which describes the tests performed and infrastructure used in detail. Apache Ambari and Pivotal HD To help automate the deployment of Spring XD on an Apache HadoopⓇ cluster, we added an Apache AmbariⓇ plugin for Spring XD. The plugin is supported on both Pivotal HD 3.0 and Hortonworks HDP 2.2 distributions. We also added support in Spring XD for Pivotal HD 3.0, bringing the total number of Hadoop versions supported to five. New Sources, Processors, Sinks, and Batch Jobs One of Spring XD’s biggest value propositions is its complete set of out-of-the-box data connectivity adapters that can be used to create real-time and batch-based data pipelines, and these require little to no user-code for common use-cases. With the help of community contributions, we now have MongoDB, VideCap, and FTP as source modules, an XSLT-transformer processor, and FTP sink module. The XD team also developed a Cassandra sink and a language-detection processor. Recognizing the important role in the Pivotal Big Data portfolio, we have also added native integration with Pivotal Greenplum Database and Pivotal HAWQ through gpfdist sink for real-time streaming and also support for gpload based batch jobs. Adding to our developer productivity theme and the use of Spring XD in production for high-volume data ingest use-cases, we are delighted to recognize Simon Tao and Yu Cao (EMC² Office of The CTO & Labs China), who have been operationalizing Spring XD data pipelines in production since 2014 and also for the VideCap source module contribution. Their use-case and implementation specifics (in their own words) are below. “There are significant demands to extract insights from large magnitude of unstructured video streams for the video surveillance industry. Prior to being analyzed by data scientists, the video surveillance data needs to be ingested in the first place. To tackle this challenge, we built a highly scalable and extensible video-data ingestion platform using Spring XD. This platform is operationally ready to ingest different kinds of video sources into a centralized Big Data Lake. Given the out-of-the-box features within Spring XD, the platform is designed to allow rich video content processing capabilities such as video transcoding and object detection, etc. The platform also supports various types of video sources—data processors and data exporting destinations (e.g. HDFS, Gemfire XD and Spark)—which are built as custom modules in Spring XD and are highly reusable and composable. With a declarative DSL, a video ingestion stream will be handled by a video ingestion pipeline defined as Directed Acyclic Graph of modules. The pipeline is designed to be deployed in a clustered environment with upstream modules transferring data to downstream ones efficiently via the message bus. The Spring-XD distributed runtime allows each module in the pipeline to have multiple instances that run in parallel on different nodes. By scaling out horizontally, our system is capable of supporting large scale video surveillance deployment with high volume of video data and complex data processing workloads.” Custom Module Registry and HA Support Though we have had the flexibility to configure shared network location for distributed availability of custom modules (via: xd.customModule.home), we also recognized the importance of having the module-registry resilient under failure scenarios—hence, we have an HDFS backed module registry. Having this setup for production deployment provides consistent availability of custom module bits and the flexibility of choices, as needed by the business requirements. Pivotal Cloud Foundry Integration Furthering the Pivotal Cloud Foundry integration efforts, we have made several foundation-level changes to the Spring XD runtime, so we are able to run Spring XD modules as cloud-native Apps in Lattice and Diego. We have aggressive roadmap plans to launch Spring XD on Diego proper. While studying Diego’s Receptor API (written in Go!), we created a Java Receptor API, which is now proposed to Cloud Foundry for incubation. Next Steps We have some very interesting developments on the horizon. Perhaps the most important, we will be launching new projects that focus on message-driven and batch-oriented “data microservices”. These will be built directly on Spring Boot as well as Spring Integration and Spring Batch, respectively. Our main goal is to provide the simplest possible developer experience for creating cloud-native, data-centric microservice apps. In turn, Spring XD 2.0 will be refactored as a layer above those projects, to support the composition of those data microservices into streams and jobs as well as all of the “as a service” aspects that it provides today, but it will have a major focus on deployment to Cloud Foundry and Lattice. We will be posting more on these new projects soon, so stay tuned! Feedback is very important, so please get in touch with questions and comments via * StackOverflowspring-xd tag * Spring JIRA or GitHub Issues Editor’s Note: ©2015 Pivotal Software, Inc. All rights reserved. Pivotal, Pivotal HD, Pivotal Greenplum Database, Pivotal Gemfire and Pivotal Cloud Foundry are trademarks and/or registered trademarks of Pivotal Software, Inc. in the United States and/or other countries. Apache, Apache Hadoop, Hadoop and Apache Ambari are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All Posts Engineering Releases News and Events