Written by Norberto Leite Introduction In this 2-part blog series, you will learn how to use MongoDB, Mongoose Object Data Mapping (ODM) with Express.js and Node.js. These technologies use a uniform language - JavaScript - providing performance gains in the software and productivity gains for developers. In this first part, we will describe the basic mechanics of our application and undertake data modeling. In the second part, we will create tests that validate the behavior of our application and then describe how to set-up and run the application. No prior experience with these technologies is assumed and developers of all skill levels should benefit from this blog series. So, if you have no previous experience using MongoDB, JavaScript or building a REST API, don’t worry - we will cover these topics with enough detail to get you past the simplistic examples one tends to find online, including authentication, structuring code in multiple files, and writing test cases. Let’s begin by defining the MEAN stack. What is the MEAN stack? The MEAN stack can be summarized as follows: M = MongoDB/Mongoose.js: the popular database, and an elegant ODM for node.js. E = Express.js: a lightweight web application framework. A = Angular.js: a robust framework for creating HTML5 and JavaScript-rich web applications. N = Node.js: a server-side JavaScript interpreter. The MEAN stack is a modern replacement for the LAMP (Linux, Apache, MySQL, PHP/Python) stack that became the popular way for building web applications in the late 1990s. In our application, we won’t be using Angular.js, as we are not building an HTML user interface. Instead, we are building a REST API which has no user interface, but could instead serve as the basis for any kind of interface, such as a website, an Android application, or an iOS application. You might say we are building our REST API on the ME(a)N stack, but we have no idea how to pronounce that! What is a REST API? REST stands for Representational State Transfer. It is a lighter weight alternative to SOAP and WSDL XML-based API protocols. REST uses a client-server model, where the server is an HTTP server and the client sends HTTP verbs (GET, POST, PUT, DELETE), along with a URL and variable parameters that are URL-encoded. The URL describes the object to act upon and the server replies with a result code and valid JavaScript Object Notation (JSON). Because the server replies with JSON, it makes the MEAN stack particularly well suited for our application, as all the components are in JavaScript and MongoDB interacts well with JSON. We will see some JSON examples later, when we start defining our Data Models. The CRUD acronym is often used to describe database operations. CRUD stands for CREATE, READ, UPDATE, and DELETE. These database operations map very nicely to the HTTP verbs, as follows: POST: A client wants to insert or create an object. GET: A client wants to read an object. PUT: A client wants to update an object. DELETE: A client wants to delete an object. These operations will become clear later when define our API. Some of the common HTTP result codes that are often used inside REST APIs are as follows: 200 - “OK”. 201 - “Created” (Used with POST). 400 - “Bad Request” (Perhaps missing required parameters). 401 - “Unauthorized” (Missing authentication parameters). 403 - “Forbidden” (You were authenticated but lacking required privileges). 404 - “Not Found”. A complete description can be found in the RFC document, listed in the resources section at the end of this blog. We will use these result codes in our application and you will see some examples shortly. Why Are We Starting with a REST API? Developing a REST API enables us to create a foundation upon which we can build all other applications. As previously mentioned, these applications may be web-based or designed for specific platforms, such as Android or iOS. Today, there are also many companies that are building applications that do not use an HTTP or web interface, such as Uber, WhatsApp, Postmates, and Wash.io. A REST API also makes it easy to implement other interfaces or applications over time, turning the initial project from a single application into a powerful platform. Creating our REST API The application that we will be building will be an RSS Aggregator, similar to Google Reader. Our application will have two main components: The REST API Feed Grabber (similar to Google Reader) In this blog series we will focus on building the REST API, and we will not cover the intricacies of RSS feeds. However, code for Feed Grabber is available in a github repository, listed in the resources section of this blog. Let’s now describe the process we will follow in building our API. We will begin by defining the data model for the following requirements: Store user information in user accounts Track RSS feeds that need to be monitored Pull feed entries into the database Track user feed subscriptions Track which feed entry a user has already read Users will need to be able to do the following: Create an account Subscribe/unsubscribe to feeds Read feed entries Mark feeds/entries as read or unread Modeling Our Data An in-depth discussion on data modeling in MongoDB is beyond the scope of this article, so see the references section for good resources on this topic. We will need 4 collections to manage this information: Feed collection Feed entry collection User collection User-feed-entry mapping collection Let’s take a closer look at each of these collections. Feed Collection Lets now look at some code. To model a feed collection, we can use the following JSON document: If you are familiar with relational database technology, then you will know about databases, tables, rows and columns. In MongoDB, there is a mapping to most of these Relational concepts. At the highest level, a MongoDB deployment supports one or more databases. A database contains one or more collections, which are the similar to tables in a relational database. Collections hold documents. Each document in a collection is, at a highest level, similar to a row in a relational table. However, documents do not follow a fixed schema with pre-defined columns of simple values. Instead, each document consists of one or more key-value pairs where the value can be simple (e.g., a date), or more sophisticated (e.g., an array of address objects). Our JSON document above is an example of one RSS feed for the Eater Blog, which tracks information about restaurants in New York City. We can see that there are a number of different fields but the key ones that our client application may be interested in include the URL of the feed and the feed description. The description is important so that if we create a mobile application, it would show a nice summary of the feed. The remaining fields in our JSON document are for internal use. A very important field is _id. In MongoDB, every document must have a field called _id. If you create a document without this field, at the point where you save the document, MongoDB will create it for you. In MongoDB, this field is a primary key and MongoDB will guarantee that within a collection, this value is unique. Feed Entry Collection After feeds, we want to track feed entries. Here is an example of a document in the feed entry collection: Again, we can see that there is a _id field. There are also some other fields, such as description, title and summary. For the content field, note that we are using an array, and the array is also storing a document. MongoDB allows us to store sub-documents in this way and this can be very useful in some situations, where we want to hold all information together. The entryID field uses the tag format to avoid duplicate feed entries. Notice also the feedID field that is of type ObjectId - the value is the _id of the Eater Blog document, described earlier. This provides a referential model, similar to a foreign key in a relational database. So, if we were interested to see the feed document associated with this ObjectId, we could take the value 523b1153a2aa6a3233a913f8 and query the feed collection on _id, and it would return the Eater Blog document. User Collection Here is the document we could use to keep track of users: A user has an email address, first name and last name. There is also an sp_api_key_id and sp_api_key_secret - we will use these later with Stormpath, a user management API. The last field, called subs, is a subscription array. The subs field tells us which feeds this user is subscribed-to. User-Feed-Entry Mapping Collection The last collection allows us to map users to feeds and to track which feeds have been read. We use a Boolean (true/false) to mark the feed as read or unread. Functional Requirements for the REST API As previously mentioned, users need to be able to do the following: Create an account. Subscribe/unsubscribe to feeds. Read feed entries. Mark feeds/entries as read or unread. Additionally, a user should be able to reset their password. The following table shows how these operations can be mapped to HTTP routes and verbs. Route Verb Description Variables /user/enroll POST Register a new user firstName lastName email password /user/resetPassword PUT Password Reset email /feeds GET Get feed subscriptions for each user with description and unread count /feeds/subscribe PUT Subscribe to a new feed feedURL /feeds/entries GET Get all entries for feeds the user is subscribed to /feeds/&ltfeedid>/entries GET Get all entries for a specific feed /feeds/&ltfeedid> PUT Mark all entries for a specific feed as read or unread read = &lttrue | false> /feeds/&ltfeedid>/entries/&ltentryid> PUT Mark a specific entry as either read or unread read = &lttrue | false> /feeds/&ltfeedid> DELETE Unsubscribe from this particular feed In a production environment, the use of secure HTTP (HTTPS) would be the standard approach when sending sensitive details, such as passwords. Real World Authentication with Stormpath In robust real-world applications it is important to provide user authentication. We need a secure approach to manage users, passwords, and password resets. There are a number of ways we could authenticate users for our application. One possibility is to use Node.js with the Passport Plugin, which could be useful if we wanted to authenticate with social media accounts, such as Facebook or Twitter. However, another possibility is to use Stormpath. Stormpath provides User Management as a Service and supports authentication and authorization through API keys. Basically, Stormpath maintains a database of user details and passwords and a client application REST API would call the Stormpath REST API to perform user authentication. The following diagram shows the flow of requests and responses using Stormpath. In detail, Stormpath will provide a secret key for each “Application” that is defined with their service. For example, we could define an application as “Reader Production” or “Reader Test”. This could be very useful when we are still developing and testing our application, as we may be frequently adding and deleting test users. Stormpath will also provide an API Key Properties file. Stormpath also allows us to define password strength requirements for each application, such as: Must have >= 8 characters. Must include lowercase and uppercase. Must include a number. Must include a non-alphabetic character Stormpath keeps track of all of our users and assigns them API keys, which we can use for our REST API authentication. This greatly simplifies the task of building our application, as we don’t have to focus on writing code for authenticating users. Node.js Node.js is a runtime environment for server-side and network applications. Node.js uses JavaScript and it is available for many different platforms, such as Linux, Microsoft Windows and Apple OS X. Node.js applications are built using many library modules and there is a very rich ecosystem of libraries available, some of which we will use to build our application. To start using Node.js, we need to define a package.json file describing our application and all of its library dependencies. The Node.js Package Manager installs copies of the libraries in a subdirectory, called node_modules/, in the application directory. This has benefits, as it isolates the library versions for each application and so avoids code compatibility problems if the libraries were to be installed in a standard system location, such as /usr/lib, for example. The command npm install will create the node_modules/ directory, with all of the required libraries. Here is the JavaScript from our package.json file: Our application is called reader-api. The main file is called server.js. Then we have a list of the dependent libraries and their versions. Some of these libraries are designed for parsing the HTTP queries. The test harness we will use is called frisby. The jasmine-node is used to run frisby scripts. One library that is particularly important is async. If you have never used node.js, it is important to understand that node.js is designed to be asynchronous. So, any function which does blocking input/output (I/O), such as reading from a socket or querying a database, will take a callback function as the last parameter, and then continue with the control flow, only returning to that callback function once the blocking operation has completed. Let’s look at the following simple example to demonstrate this. In the above example, we may think that the output would be: one two but in fact it might be: two one because the line that prints “one” might happen later, asynchronously, in the callback. We say “might” because if conditions are just right, “one” might print before “two”. This element of uncertainty in asynchronous programming is called non-deterministic execution. For many programming tasks, this is actually desirable and permits high performance, but clearly there are times when we want to execute functions in a particular order. The following example shows how we could use the async library to achieve the desired result of printing the numbers in the correct order: In the above code, we are guaranteed that function two will only be called after function one has completed. Wrapping Up Part 1 Now that we have seen the basic mechanics of node.js and async function setup, we are ready to move on. Rather than move into creating the application, we will instead start by creating tests that validate the behavior of the application. This approach is called test-driven development and has two very good features: It helps the developer really understand how data and functions are consumed and often exposes subtle needs like the ability to return 2 or more things in an array instead of just one thing. By writing tests before building the application, the paradigm becomes “broken / unimplemented until proven tested OK” instead of “assumed to be working until a test fails.” The former is a “safer” way to keep the code healthy.

We’re going back to the classics this month, with the latest enterprise collaboration news round-up focussing on cloud strategy and considerations and benefits when it comes to implementing collaboration software. Mashable shared an infographic it created in conjunction with Hewlett Packard, which compiles data and research suggesting that the use of hybrid and private cloud computing is on the rise. The article quotes statistics from Rightscale, which states that 82% of enterprises have a multi-cloud strategy already, and of these 14% use multiple private clouds, 13% use multiple public clouds, and 55% use hybrid clouds. Mashable quotes Technology Business Research, which states that “there is continued migration of enterprise vendors in mature markets such as the U.S. to hybrid and private cloud platforms to provide software vendors an opportunity to generate adoption for management technologies, as customers require next-generation tools to manage heterogeneous IT infrastructures efficiently.” In his article for eWEEK, Chris Preimesberger outlines 10 ways IT and business leaders must collaborate on cloud strategies. Chris explains that a decision to use cloud services is no longer simply down to the IT department. During the last nine years, he says, entire businesses have become necessarily immersed in IT strategies in order to harness the cloud for economics, innovation, operations and growth. He shares a slide show which provides advice for how technical and business leaders can collaborate to build a secure cloud strategy. The slide show states that usage indicate that private clouds are expected to grow at double the rate of public cloud, a result of ongoing concerns about data security and privacy. Gary Audin asks the question cloud economics or flexibility? in his article for No Jitter. Gary explains that although the cost of cloud can be attractive, that might not be the real draw for enterprises. He states that knowing what costs to consider as part of a cloud service implementation is vital to making the right decision about cloud. Gary points out the benefits of the cloud as being far more than simply a matter of cost. He explains that the cloud allows rapid response for an enterprise as it contends with change due to situations such as staff growth or reduction, market fluctuations, financial limitations, or new opportunities. Above all, Gary explains, the cloud delivers flexibility and it is this which makes it the most attractive option for enterprises. In his article for MSP Mentor, Michael Brown reveals the result of a recent report on cloud adoption in the enterprise. The report, by Skyhigh Networks, revealed that enterprise cloud adoption grew by 43% in 2014. Michael highlights findings on the file sharing front, revealing that 37 percent of employees were found to be uploading sensitive business data to consumer file sharing services. Consumer file sharing services are one element of a growing trend towards BYOC (bring your own cloud, content and collaboration). Robert Bamforth explains that BYOC is an evolution of BYOD (bring your own device) which posed a challenge to IT departments since the rise of the smartphone. Robert explains that BYOC is a new challenge for IT departments in controlling their organisation’s digital assets while liberating employee productivity and information sharing. Robert states that the BYOC conundrum should change as enterprise-strength security features and tools continue to evolve to have more consumer-like interfaces, which will make asking employees to use enterprise tools much easier. He gives some suggestions to help enterprises in the mean time: understand the appeal of consumer tools, make sure everyone understands security risks, forget trying to apply strong rules to trivial information, get a mobile-ready solution, look for and pre-plug data leaks, and above all don’t stop collaboration if it’s happening. In his article for ZDNet, Dion Hinchcliffe reflects on the state of the digital collaboration industry. Far from maturing, Dion says, the collaboration tool space is busier than ever evolving, branching out, and multiplying. But, he asks, are organizations able to adopt so many different ways of working together? Dion observes that instead of settling down, the collaboration software space is actually get more interesting and varied, and he is seeing new technologies, such as applications that focus on optimizing collaboration for mobile devices or for team analytics. It’s now time for organizations to design a strong foundation for digital collaboration, says Dion, as the near future promises many key new innovations that must be considered and incorporated to stay competitive, both to customers and the workforce. When businesses do decide to adopt one or more digital collaboration platforms, Andre Bourque offers some helpful ways in which to measure ROI. Andre quotes a Mashable report which states that cloud collaboration drives creativity and engagement, leading to happier employees and a better company culture, but this is not a metric that is easily measurable. Andre explains that it’s hard to find definitive examples of ROI, as most are anecdotal or “in process”, and merely counting user adoption rate of a collaborative platform is inadequate. Instead, Andre quotes Angela Ashenden, of MWD Advisors, who offers the following metrics to consider: reduced travel time and costs; creating new business opportunities and services; increased employee retention rates, cost savings across the organisation, and faster on-boarding for new users. Do you have any metrics that you find useful to measure ROI on your collaboration platform in your organisation?

Docker deployments can be very dynamic with containers being started and stopped, moved around the YARN or Mesos-managed clusters, having very short life spans (the so-called pets) or long uptimes (aka cattle). Getting insight into the current and historical state of such clusters goes beyond collecting container performance metrics and sending alert notifications. If a container dies or gets paused, for example, you may want to know about it, right? Or maybe you’d want to be able to see that a container went belly up in retrospect when troubleshooting, wouldn’t you? Just two weeks ago we added Docker Monitoring (docker image is right here for your pulling pleasure) to SPM. We didn’t stop there — we’ve now expanded SPM’s Docker support by adding Docker Event collection, charting, and correlation. Every time a container is created or destroyed, started, stopped, or when it dies, spm-agent-docker captures the appropriate event so you can later see what happened where and when, correlate it with metrics, alerts, anomalies — all of which are captured in SPM — or with any other information you have at your disposal. The functionality and the value this brings should be pretty obvious from the annotated screenshot below. Like this post? Please tweet about Docker Events and Docker Metrics Monitoring Know somebody who’d find this post useful? Please let them know… Here’s the list of Docker events SPM Docker monitoring agent currently captures: Version Information on Startup: server-info – created by spm-agent framework with node.js and OS version info on startup docker-info – Docker Version, API Version, Kernel Version on startup Docker Status Events: Container Lifecycle Events like create, exec_create, destroy, export Container Runtime Events like die, exec_start, kill, oom, pause, restart, start, stop, unpause Every time a Docker container emits one of these events spm-agent-docker will capture it in real-time, ship it over to SPM, and you’ll be able to see it as shown in the above screenshot. Oh, and if you’re running CoreOS, you may also want to see how to index CoreOS logs into ELK/Logsene. Why? Because then you can have not only metrics and container events in one place, but also all container and application logs, too! If you’re using Docker, we hope you find this useful! Anything else you’d like us to add to SPM (for Docker or anyother integration)? Leave a comment, ping @sematext, or send us email – tell us what you’d like to get for early Christmas!

If you are building mobile apps then you are connecting to some REST API. For example, if you want to resolve an address to a latitude/longitude information to display on a map, you might use the Google Geocoding API: Google Geocoding API: https://maps.googleapis.com/maps/api/geocode/json?address=San Francisco,CA&key=AIzaSyDvFMYGjeR02RH If you are invoking the API from the client, then the API key also has to be present on the client. But, this is also the problem. It’s very easy to look at the app source in the browser and get access to the API key. If someone has access to your API key, they can send requests on your behalf (without you knowing), and use up your request quota. Even if you are building a hybrid app, it’s still the same problem. A hybrid app is HTML/JavaScript inside a native wrapper, it’s possible to download the app, un-package it and gain access to API keys or any sensitive information stored in the app. Even native apps are not immune to this. For example, an Android app is just a Java application and a Java application can be de-compiled to view the original source. The next image shows how to get access to an API key in the browser: Viewing app source in browser A good solution is to never expose the API key (or any other sensitive data) on the client. How do you do that? You keep the API key and any other sensitive information on the server. Appery.io Secure Proxy Appery.io Secure Proxy (part of Backend Services) enables app developers to keep sensitive app data on the server. Your API keys or any other data is never exposed on the client. Watch this 5-minute video on how to use Secure Proxy: Before using the Secure Proxy, you need to store the data on the server. To store the data you are going to use the Appery.io Database. It’s as simple as creating a collection with two columns. The first column is the value name, the second column is the actual value. This is how the database looks when storing the API key for Google Geocoding API: Saving API key in database As this key is stored on the server, no one (but you) has access to it. You can store other data as well such as URLs, tokens or anything else that shouldn’t be exposed on the client. The next step is to setup the proxy that will use the information stored in the database. This step is also very simple, this is how it looks: Secure proxy linked to a database You give the proxy a name and then link it to a database which stores your data. The above proxy is linked to Secrets_db database, Credentials collection, and secretName, secretValue columns. The last step is to link a REST API service to the proxy. In the service editor you select the secure proxy created: REST API service using secure proxy then in the Request tab you reference the API key stored in the database (the name stored in secretName column): Request parameter substitution will happen on the server and that’s it. When the API service is invoked, the call will go through the secure proxy (server) where the API key will substituted: API key is not exposed on the client For web apps, you can add an extra layer of security by specifying from which page URLs the proxy should accept requests: URL-based security The proxy will only accept requests from page URLs listed in the table. Another option to keep API keys private is to invoke the API from the server using Server Code, I will cover this in another post. Setting up an using the Appery.io Secure Proxy is simple. It provides a very important feature by allowing to keep sensitive and private data on the server, never exposing it on the client, and adding an extra security layer to your app.

LastPass, the password manager that lets you manage your passwords between different devices, was recently hacked. From this there has been a fair amount of FUD circulating and not enough rational thought. With that in mind, this seems like a good time to talk about password security and LastPass with some rational ideas. Since we can't get rid of passwords just yet we need to manage them well. 1. LastPass Detected The Breach No useful system is impenetrable. Computers not connected to the Internet, that don't even have a network card, have been hacked across an air gap using their speakers and mic. The most up to date systems still suffer from zero-day exploits. Two of the elements of an organization that takes security seriously are keeping certain pieces of data separate and detecting when a breach occurs. From the LastPass announcement of the breach we can see these two things in action. While some information was obtained the actual vaults of passwords were not downloaded. And, they detected there was a problem and enough monitoring in place to distinguish what was effected. I can't overstate how nice that is to hear. Many organizations won't detect if they have been breached. Even many of those that could detect a breach wouldn't be able to tell you what was affected. That's right, many of the places you put personal information couldn't do what LastPass did. 2. LastPass Responded To The Breach Even though the password vaults were not taken LastPass is having everyone change their vault password. They detected the problem and are going the extra mile to protect their users. Now, let's consider an alternative option. Consider a 1Password or KeePass user who stores their information in Dropbox or a similar service. A malicious program on one of their systems could have taken their vault and sent it to an attacker. Those users would not have known. Or, the service could have been hacked but since it's not password specific who would have suggested changing the master password? I'm not trying to defend LastPass. It's a matter of considering the alternatives and the security measures around them. Is a 1Password or KeePass alternative setup actually more secure in practice? 3. Different Passwords For Different Sites There's a good reason to have a different password or passphrase for different sites. You can't trust that a site you submit it to will store it securely so that it won't be misused to access other sites. With all the sites we connect to we it's difficult to remember a different password for each site. Congratulations if you can do that. For the rest of us we need a system to help. This is where a password manager is useful. That is, until we can stop using passwords for something better. So, use a password manager if you can't otherwise have a different password for each site. It's more secure than using the same password everywhere. Note, I'm not recommending a particular password manager on purpose. Use a good one. 4. Security Is Not About Perfection There is no such thing as perfect security. Security needs to be practical. For example, for most people it's more security to use a password manager than to use the same password everywhere. Neither is perfect but when you weigh the differences the password manager comes out as more secure. In a distributed device world where we need passwords on more than one system it's good to go with a system that does this for you. A system that focuses on security and handling issues that come up. You could roll your own solution. But, will it be more secure? For most people the professional solution is the more secure one. When considering password security choose the one that's more secure for you rather than seeking the perfect option. 5. When Not To Use A Password Manager There are some places I would recommend not using a password manager. For example, I would recommend not using one for your financial sites. Those few places that are very important use a passphrase. 6. Encrypt Your Password Store In our multi-device world you'll likely need to share your password datastore between devices. And, any device can be hacked even if you don't need to use multiple devices. Imagine a virus on a computer looking for your password excel file and uploading that to someone bad. It happens. Use an encrypted datastore. This is why password managers are important. They are designed to store your data in an encrypted manner. This way, if someone gets your data store they will have a very hard time reading it. Before they can get to anything they'll need to break the encryption which isn't so easy. That means, even if an attacker had gotten the password vaults from LastPass, which they didn't, they would not have been able to read the data in them. 7. Limit Your Attack Vector One of the problems with LastPass is that they are a known password manager. That makes them a target if someone wants to try and get passwords. Alternatives that store their distributed information in general purpose systems pose a different attack vector. For example, if you use 1Password or KeePass and store your information in Dropbox you can still be hacked. Dropbox has been externally hacked in the past and other applications can access your Dropbox folder. Using alternatives to LastPass doesn't mean you won't be attacked. Take a few minutes and consider the attack vectors of the different solutions you're considering and how each of those will detect a breach and respond to that. For example, I could self host my encrypted file on the Internet somewhere. This would be managed by me and wouldn't be a known system for someone to target. But, the IPv4 addresses are regularly checked for known vulnerabilities so attackers and get onto a system a poke around. That is the entire IPv4 space, which is still the only space routable for all things, is regular checked. Will I keep everything on that system up to date? Will I detect if someone broke into the system? Will I respond appropriately? All of this needs to be taken into account. Final Thoughts On LastPass I'm not trying to defend LastPass. I'm trying to give a little more of a holistic picture of security. It's complicated and any alternatives to a LastPass or LastPass-like solutions need to have their security considered. Viewing the options with security and attacks in mind keeps everything in perspective.

Leading today, VentureBeat reports on the quiet launch of Google’s Cloud Source Repositories. This seems like something we should have heard more about, but I don’t remember seeing anything about it. Amazon AWS announces the availability of all things Alexa, the Skills Kit, the Voice Service and a Fund. Last but not least, we have AJ Kohn, from Blind Five Year Old, talking about click-through rate being a ranking signal on Google’s search results. I don’t talk about SEO much, but reading AJ’s work is always fascinating. As always, enjoy today’s items, and please participate in the discussions on these sites. Top Stories Google has quietly launched a GitHub competitor, Cloud Source Repositories | VentureBeat Alexa Skills Kit, Alexa Voice Service, Alexa Fund | AWS Official Blog Startups, Career and Process Why offices are where work goes to die | Swizec Teller Unleashing the power of small teams | Andreas Papathanasis What Is A Tester? | Developsense Blog What happens when you stop relying on resumes | Aline Lerner Design and Development Swift 2: SIMD | Russ Bishop Why is Git better than Mercurial? | Javalobby Create a Maven archetype | Javalobby pip -t: A simple and transparent alternative to virtualenv | Zoomer Analytics Killing Off Wasabi – Part 1 | Fog Creek Blog WebAssembly- Explained | Modus Create Generating JSON Schema from XSD with JAXB and Jackson | Inspired by Actual Events AI, Machine Learning, Research and Advanced Algorithms Applying Machine Learning to Text Mining with Amazon S3 and RapidMiner | Amazon AWS Big Data, Visualization, SQL and NoSQL Is Click Through Rate A Ranking Signal? | Blind Five Year Old Cache-friendly binary search | Bannalia Discovering the Computer Science Behind Postgres Indexes | Java Code Geeks How an open-source competitive benchmark helped to improve databases | ArangoDB Security, Encryption and Cryptography Cracking JXcore… Again | Mark Haase Link Collections The Daily Six Pack: June 25, 2015 | Dirk Strauss Double Shot #1517 | A Fresh Cup Dew Drop – June 25, 2015 (#2042) | Morning Dew The Daily Six Pack: June 26, 2015 | Dirk Strauss

Every business has unique characteristics that set it apart from other organizations, even within the same industry. For this reason, it shouldn't be surprising that there's not a one-size-fits-all approach to app security testing. Each company has certain protection expectations and regulations to adhere to, making it essential to find the best way to achieve these goals. Here are a few examples of security testing methodologies available for quality assurance teams to leverage: Black box With black box testing, QA professionals put themselves in the shoes of the hacker and attempt to break the app through various attack vectors. This methodology can yield a lot of information and help better secure the program from actual threats. A white paper by Security Innovation noted that software testers first analyze the system's architecture and business model to identify any security vulnerabilities. Looking over the software logic in this way can uncover subtle security and privacy issues that may not have been noticed otherwise, such as defects in design, input, system dependency, authentication, cryptography and information disclosure. "Although white box code inspection is good for analyzing static behavior, only black box exploratory testing can determine the dynamic behavior of how a system is implemented and used, the coupling between systems and the interactions of the distributed systems," Security Innovation wrote. Dynamic For QA teams that like to execute code, dynamic testing is the approach for them. This methodology checks the running application for how it behaves and responds to a variety of inputs. This is done to ensure that the product meets up with established regulations and is giving the expected outcomes. IBM noted that dynamic analysis is especially useful to identify code coverage, as it can discover bugs in paths that have gone untested. While dynamic testing can be manual work for testers, it can also yield significant information that will help mitigate defects and produce quality products. Static In contrast to dynamic testing, static approaches directly review the source code, often through an automated test management solution. TechTarget contributor Michael Cobb noted that this methodology occurs at the implementation phase, rather than when the app is running, and often helps mitigate vulnerabilities involved with industry compliance standards. Automation in this area can reduce the amount of time it takes to complete these tasks. However, it may not be able to detect sophisticated threats, which can be supplemented by dynamic security testing. "A thorough source code review has an advantage over dynamic testing," Cobb wrote. "Nothing is hidden from analysts during a source code review, so they can examine exactly how data flows through a program. By solving the problem at the code level, static testing reduces the number of security-related design and coding defects, and the severity of any defects that make it through to the release version, thus dramatically improving the overall security of the application." There are a number of security testing methodologies that organizations can pursue, and designing a combination of approaches may result with a solution that's best for their needs. Using these strategies, companies can better ensure the protection of sensitive information while providing users with the software testing tools needed to succeed.

CryTek just released its CryEngine 3.8.1, an update packed with features. The CryEngine website dubs this the heftiest upgrade since debuting their Engine-as-a-Service in May. Among the many new features is virtual reality (VR) support, OpenGL compatibility, and Linux support. One of the biggest trends in gaming is VR, and CryEngine 3.8.1 adds VR support. Initially, it’s limited to the Oculus Rift, but chances are as more headsets emerge and see adoption among both developers and gamers, compatibility will expand to support these as well. Epic Games’ Unreal Engine also added VR support, along with Unity3D’s most recent release. There’s a neat VR demo on the CryTek website, which developers will surely want to check out. Another significant change is the addition of Linux support. While Wine and Playonlinux have both helped many games to run on a variety of Linux-based operating systems (OSes). However, native Linux support means easier use for developers. As more games add Linux compatibility, spearheaded by Steam’s SteamOS, the CryEngine itself can now be run on Linux. This latest update means that the CryEngine will join Unity3D, the Unreal Engine, and Source as a powerful game development engine with Linux and VR support. Virtual reality is seeing widespread adoption among the developer community, and Linux compatibility in gaming is a huge trend. While CryEngine 3.8.1’s ability to run on Linux won’t necessarily mean games developed will be compatible on the popular open source OS, it certainly makes it easier to ensure Linux support. There’s also OpenGL support, which will also aid cross-platform development. The CryEngine has been used to create many games known for their gorgeous eye-candy. Notably, the CryTek’s aptly names Crysis series is built with the CryEngine, as is Ryse: Son of Rome, State of Decay, and Kingdom Come: Deliverance.

[This article was written by Matt Kiernan] We are excited to announce that Logentries’ new SQL-like query language, LEQL, is now available for more advanced analytics and easy extraction of valuable insights from your log data. A SQL-Like Query Language If you’ve ever used SQL, LEQL should feel familiar. In fact, Logentries already supports a number of SQL-like search functions, including: SUM: Sums a set of values COUNT: Counts the number of times a value occurs GROUPBY: Groups values by a unique key UNIQUE: Enables the count of only unique values With the rollout of LEQL, we’ll be introducing four new query functions: MIN: Calculate the minimum value of a specified key MAX: Calculate the maximum value of a specified key SORT: Display results sorted either ascending or descending TIMESLICE: Specifies how to group by time (e.g. by specific number of minutes, hours or days) A Consistent Yet Expressive Syntax We believe a reliable query language depends on a consistently enforced syntax. For this reason, we’ll be enforcing how queries are structured. Here’s an example of how an old query would change with LEQL: Old pages>0 | GroupBY(dbName) | SUM(pages) New where(pages>0) groupby (dbName) calculate(SUM:pages) *In this example, pages & dbName are Key names in log events Notice how the search logic gets wrapped in a where() clause, used for refining your search to return only results that match your search criteria (i.e. where events include the text or Key “pages”.) groupBy() is an optional clause that enables you to organize your search results into groups by specifying a Key from a Key-Value Pair (i.e. key: value). Calculations made within your query get utilized in the calculate()clause. When building your query, you no longer need to separate sections with pipes “|”. Though we believe in the value of a consistent query syntax, we also believe in the importance of giving users an expressive language that is easy to use and delivers expected results. We’re taking the following steps to make LEQL easy to use: Outdated saved queries will automatically be converted into LEQL – no effort required where clauses will automatically be added to any new query you write LEQL terms will not be case sensitive An updated search bar will provide a query builder and validator An updated search bar & query builder An Updated Search Bar & Query Builder As we rollout LEQL, we’ll be introducing a new search bar, allowing users to switch between a simple & advanced modes based on their preference. Simple mode “Simple mode” provides an easy way to build queries by providing a list of the available functions. Type-Assist will show a list of keys to associate with each functions, or new keys can be typed manually. Advanced mode “Advanced mode” will allow users to type their queries manually. Type-Assist will autocomplete key names while the new search bar will automatically validate query syntax. July 1st Rollout The LEQL rollout will take place in phases, beginning July 1st and will continue over the next few weeks to update all plans. If you’d like early beta access to LEQL, or have any questions, feel free to reach us at [email protected].

Occasionally we are asked about JBoss BPM Suite integration with other products and layers in an enterprises architecture. We have published articles talking about how to achieve this with various aspects such as: Microservices integration Data integration Articles are one thing, but seeing is believing, so we have done a few webinars to show you live how to tackle integration: Data integration webinar PEX webinar Along with these articles we have always published demo projects that give you a closer look and chance to get hands on with these integration strategies: JBoss BPM Suite & JBoss Fuse Travel microservices story JBoss BPM Suite & JBoss Data Virtualization integration Imported Signavio Process Editor mortgage workflow. There is another integration story yet to be told about how one can leverage other tooling together with JBoss BPM Suite. This article will introduce one such company,Signavio, that provides a Signavio Process Editor so"...you can start modeling and engaging your organization in improving operational efficiency through the development of optimal models..." The following demo project provides a working example of how you can model an example mortgage process in Signavio Process Editor and then bring it into JBoss BPM Suite where you can add implementation details, integration details and other implementation details to finally execute the mortgage process end-to-end. Demo project As always we bring you not only a story, but a reusable demo project you can easily spin up yourself to explore the details around how a JBoss BPM project would integrate with the model designed in Signavio Process Editor. The project is called the JBoss BPM Suite & Signavio Process Editor Integration Demo. The project installs JBoss BPM Suite 6.1 with an example mortgage project with rules, process, forms and other artifacts. It also includes a copy of an exported Signavio Process Editor mortgage process that we then show how to import. Final mortgage workflow project with implementation details and integration details completed. Ready to run! This gives you the initial starting point after importing the Signavio process and the completely integrated final mortgage project that you can run side-by-side. To setup this project there are just a few simple steps to get going and will be up and running minutes: Installation Download and unzip. Add products to installs directory. Run 'init.sh' or 'init.bat' file. 'init.bat' must be run with Administrative privileges. Start JBoss BPMS Server by running 'standalone.sh' or 'standalone.bat' in the /target/jboss-eap-6.1/bin directory. Login to http://localhost:8080/business-central - login for admin, appraisor, broker, and manager roles (u:erics / p:bpmsuite1!) Mortgage Loan demo pre-installed as project. Using process designer, import the Signavio process that was exported to the file found in: support/MortgageDemoSignavio.bpmn Looking to Automate your business? See screenshots provided in project for how this should look and note that the JBoss BPM Suite process designer included validation that puts messages about tasks not specified, this is correct as at this point you need to start implementing the process tasks. You can examine the imported process and note the various details captured during initial workshops have been put into the process details for each step in the workflow. After implementing these steps you will find the final process ready to run. You can now explore the final project by deploying it and starting a new instance. We hope you enjoy this example project and feel free to browse for more at JBoss Demo Central.

The use of OpenStack in the private cloud is invaluable for DevOps. It provides engineers the ability to innovate quickly and deal with uncertainty. It also maximizes existing infrastructure and provides a programmable, software-defined IaC. Openstack in the private cloud = agile development OpenStack has emerged as the de facto standard for IaaS in the private cloud. It gives engineers a vital self-service capability to provision (and de-provision) environments, allowing them to act autonomously, in the moment. This helps to eliminate the downstream bottleneck caused by waiting for operations staff to find time to do the provisioning. As OpenStack is open source it is vendor agnostic, allowing you to take advantage of competitive pricing rather than suffering from vendor lock-in. A private cloud means lower cost for the same capacity in a public cloud, which is especially useful for enterprises with high data needs. For security reasons, OpenStack is still mainly used in the private cloud by developers and QA, i.e. in a non-production context. However, OpenStack gives an ability to optimize application performance and/or security by having more control compared to public cloud. The software is increasingly backed by the critical mass of leading IT infrastructure vendors such as IBM, CICSO and HP. Gartner assumes that “by 2019, OpenStack enterprise deployments will grow tenfold, up from just hundreds of production deployments today, due to increased maturity and growing ecosystem support.”1 Challenges to consider OpenStack implementation skills are still rare in the market, so experimentation and self-learning is necessary. Although this takes time, it is offset by the fact the software is free and represents a good opportunity to gain internal expertise. This is particularly valid if you class infrastructure as a core competence. The maturity and functionality of OpenStack projects vary widely - while it covers storage, network and compute, the main adoption currently happens around compute (Nova) and block storage (Cinder), with object storage and network (Neutron) lacking significantly behind. However, without leveraging virtualized network services as part of a private cloud, full-stack environment provisioning is not possible, so don’t forget to add necessary network services to your private cloud. Where to begin Integrating OpenStack clouds with existing infrastructure can be a challenge. It is hardly plug and play. At first, it is best to focus on relatively isolated DevOps environments, such as Gartner’s “mode two”2 applications rather than introducing open stack across the board straight away, (Bimodal IT “refers to having two modes of IT, each designed to develop and deliver information – and technology – intensive services in its own way. Mode 1 is traditional, emphasizing scalability, efficiency, safety and accuracy. Mode 2 is nonsequential, emphasizing agility and speed.”3) As with any open source software, new functions and upgrades are frequently released. This means keeping up with changes in functionality and filling gaps with customizations or third-party products. Upgrades are complex and typically require planned downtime. For these reasons, we recommend choosing a hardened distribution and sticking with it. Openstack is the most complete vendor agnostic solution for storage, network and compute services. The ability for developers to instantly spin up environments at any time is invaluable for a fully agile DevOps environment, and is well worth the effort it takes to acclimatize to Openstack. 1 http://www.prnewswire.com/news-releases/suse-openstack-cloud-5-to-simplify-private-cloud-management-300048721.html 2 http://www.gartner.com/it-glossary/bimodal 3 http://www.gartner.com/it-glossary/bimodal

Written by Yves Trudeau Ever since MySQL replication has existed, people have dreamed of a good solution to automatically split read from write operations, sending the writes to the MySQL master and load balancing the reads over a set of MySQL slaves. While if at first it seems easy to solve, the reality is far more complex. First, the tool needs to make sure it parses and analyses correctly all the forms of SQL MySQL supports in order to sort writes from reads, something that is not as easy as it seems. Second, it needs to take into account if a session is in a transaction or not. While in a transaction, the default transaction isolation level in InnoDB, Repeatable-read, and the MVCC framework insure that you’ll get a consistent view for the duration of the transaction. That means all statements executed inside a transaction must run on the master but, when the transaction commits or rollbacks, the following select statements on the session can be again load balanced to the slaves, if the session is in autocommit mode of course. Then, what do you do with sessions that set variables? Do you restrict those sessions to the master or you replay them to the slave? If you replay the set variable commands, you need to associate the client connection to a set of MySQL backend connections, made of at least a master and a slave. What about temporary objects like with “create temporary table…”? How do you deal when a slave lags behind or what if worse, replication is broken? Those are just a few of the challenges you face when you want to build a tool to perform read/write splitting. Over the last few years, a few products have tried to tackle the read/write split challenge. The MySQL_proxy was the first attempt I am aware of at solving this problem but it ended up with many limitations. ScaleARC does a much better job and is very usable but it stills has some limitations. The latest contender is MaxScale from MariaDB and this post is a road story of my first implementation of MaxScale for a customer. Let me first introduce what is MaxScale exactly. MaxScale is an open source project, developed by MariaDB, that aims to be a modular proxy for MySQL. Most of the functionality in MaxScale is implemented as modules, which includes for example, modules for the MySQL protocol, client side and server side. Other families of available modules are routers, monitors and filters. Routers are used to determine where to send a query, Read/Write splitting is accomplished by the readwritesplit router. The readwritesplit router uses an embedded MySQL server to parse the queries… quite clever and hard to beat in term of query parsing. There are other routers available, the readconnrouter is basically a round-robin load balancer with optional weights, the schemarouter is a way to shard your data by schema and the binlog router is useful to manage a large number of slaves (have a look at Booking.com’s Jean-François Gagné’s talk at PLMCE15 to see how it can be used). Monitors are modules that maintain information about the backend MySQL servers. There are monitors for a replicating setup, for Galera and for NDB cluster. Finally, the filters are modules that can be inserted in the software stack to manipulate the queries and the resultsets. All those modules have well defined APIs and thus, writing a custom module is rather easy, even for a non-developer like me, basic C skills are needed though. All event handling in MaxScale uses epoll and it supports multiple threads. Over the last few months I worked with a customer having a challenging problem. On a PXC cluster, they have more than 30k queries/s and because of their write pattern and to avoid certification issues, they want to have the possibility to write to a single node and to load balance the reads. The application is not able to do the Read/Write splitting so, without a tool to do the splitting, only one node can be used for all the traffic. Of course, to make things easy, they use a lot of Java code that set tons of sessions variables. Furthermore, for ISO 27001 compliance, they want to be able to log all the queries for security analysis (and also for performance analysis, why not?). So, high query rate, Read/Write splitting and full query logging, like I said a challenging problem. We experimented with a few solutions. One was a hardware load balancer that failed miserably – the implementation was just too simple, using only regular expressions. Another solution we tried was ScaleArc but it needed many rules to whitelist the set session variables and to repeat them to multiple servers. ScaleArc could have done the job but all the rules increases the CPU load and the cost is per CPU. The queries could have been sent to rsyslog and aggregated for analysis. Finally, the HA implementation is rather minimalist and we had some issues with it. Then, we tried MaxScale. At the time, it was not GA and was (is still) young. Nevertheless, I wrote a query logging filter module to send all the queries to a Kafka cluster and we gave it a try. Kafka is extremely well suited to record a large flow of queries like that. In fact, at 30k qps, the 3 Kafka nodes are barely moving with cpu under 5% of one core. Although we encountered some issues, remember MaxScale is very young, it appeared to be the solution with the best potential and so we moved forward. The folks at MariaDB behind MaxScale have been very responsive to the problems we encountered and we finally got to a very usable point and the test in the pilot environment was successful. The solution is now been deployed in the staging environment and if all goes well, it will be in production soon. The following figure is simplified view of the internals of MaxScale as configured for the customer: The blocks in the figure are nearly all defined in the configuration file. We define a TCP listener using the MySQL protocol (client side) which is linked with a router, either the readwritesplit router or the readconn router. The first step when routing a query is to assign the backends. This is where the read/write splitting decision is made. Also, as part of the steps required to route a query, 2 filters are called, regexp (optional) and Genlog. The regexp filter may be used to hot patch a query and the Genlog filter is the logging filter I wrote for them. The Genlog filter will send a json string containing about what can be found in the MySQL general query log plus the execution time. Authentication attempts are also logged but the process is not illustrated in the figure. A key point to note, the authentication information is cached by MaxScale and is refreshed upon authentication failure, the refresh process is throttled to avoid overloading the backend servers. The servers are continuously monitored, the interval is adjustable, and the server status are used when the decision to assign a backend for a query is done. In term of HA, I wrote a simple Pacemaker resource agent for MaxScale that does a few fancy things like load balancing with IPTables (I’ll talk about that in future post). With Pacemaker, we have a full fledge HA solution with quorum and fencing on which we can rely. Performance wise, it is very good – a single core in a virtual environment was able to read/write split and log to Kafka about 10k queries per second. Although MaxScale supports multiple threads, we are still using a single thread per process, simply because it yields a slightly higher throughput and the custom Pacemaker agent deals with the use of a clone set of MaxScale instances. Remember we started early using MaxScale and the beta versions were not dealing gracefully with threads so we built around multiple single threaded instances. So, since a conclusion is needed, MaxScale has proven to be a very useful and flexible tool that allows to elaborate solutions to problems that were very hard to tackle before. In particular, if you need to perform read/write splitting, then, try MaxScale, it is best solution for that purpose I have found so far. Keep in touch, I’ll surely write other posts about MaxScale in the near future.

I have project where I need to output some reports as CSV-files. I found a good library called CsvHelper from NuGet and it works perfect for me. After some playing with it I was able to generate CSV-files that were shown correctly in Excel. Here is some sample code and also extensions that make it easier to work with DataTables. Simple report Here’s the simple fragment of code that illustrates how to use CsvHelper. using (var writer = new StreamWriter(Response.OutputStream)) using (var csvWriter = new CsvWriter(writer)) { csvWriter.Configuration.Delimiter = ";"; csvWriter.WriteField("Task No"); csvWriter.WriteField("Customer"); csvWriter.WriteField("Title"); csvWriter.WriteField("Manager"); csvWriter.NextRecord(); foreach (var project in data) { csvWriter.WriteField(project.Code); csvWriter.WriteField(project.CustomerName); csvWriter.WriteField(project.Name); csvWriter.WriteField(project.ProjectManagerName); csvWriter.NextRecord(); } } Of course, you can use other methods to output whole object or object list with one shot. I just needed here custom headers that doesn’t match property names 1:1. Generic helper for DataTable Some of my projects come from service layer as DataTable. I don’t want to add new models or Data Transfer Objects (DTO) with no good reason and DataTable is actually flexible enough if you need to add new fields to report and you want to do it fast. As DataTables are not supported by default (yet?), I wrote simple extension methods that work on DataTable views. When called on DataTable it selects default view automatically. The idea is – you can set filter on default data view and leave out the rows you don’t need. If you just want to show DataTable to screen as table then check out my posting Simple view to display contents of DataTable. public static class CsvHelperExtensions { public static void WriteDataTable(this CsvWriter csvWriter, DataTable table) { WriteDataView(csvWriter, table.DefaultView); } public static void WriteDataView(this CsvWriter csvWriter, DataView view) { foreach (DataColumn col in view.Table.Columns) { csvWriter.WriteField(col.ColumnName); } csvWriter.NextRecord(); foreach (DataRowView row in view) { foreach (DataColumn col in view.Table.Columns) { csvWriter.WriteField(row[col.ColumnName]); } csvWriter.NextRecord(); } } } And here is simple MVC controller action that gets data as DataTable and returns it as CSV-file. The result is CSV-file that opens correctly in Excel. [HttpPost] public void ExportIncomesReport() { var data = // Get DataTable here Response.ContentType = "text/csv"; Response.AddHeader("Content-disposition", "attachment;filename=IncomesReport.csv"); var preamble = Encoding.UTF8.GetPreamble(); Response.OutputStream.Write(preamble, 0, preamble.Length); using (var writer = new StreamWriter(Response.OutputStream)) using (var csvWriter = new CsvWriter(writer)) { csvWriter.Configuration.Delimiter = ";"; csvWriter.WriteDataTable(data); } } One thing to notice – with CsvHelper we have full control over a stream where we write data and this way we can write more performant code. Related Posts .Net Framework 4.0: string.IsNullOrWhiteSpace() method Exporting GridView Data to Excel Code Contracts: Hiding ContractException How to dump object properties My object to object mapper source released The post Generating CSV-files on .NET appeared first on Gunnar Peipman - Programming Blog.

BQ deploys its call centre and telephony solution on Interoute Virtual Data Centre to improve international customer and employee communications Madrid, June 25th, 2015 - Interoute, owner operator of Europe's largest cloud services platform has announced that BQ, a leading European technology company, has chosen Interoute Virtual Data Centre (VDC), to host its new customer and employee unified communications solution. BQ has deployed a new telephony and call centre solution on Interoute VDC, leveraging the throughput, flexibility and scalability provided by this cloud platform. The BQ solution supports its 1,000 employees across different international offices, using Interoute VDC to provide the global reach they need. The solution is complemented with telephony services and worldwide DDIs from Interoute with great cost savings thanks to the economies of scale provided by Interoute's global infrastructure. Since it was founded in Spain, BQ has grown its business inside and outside the country thanks to its latest generation technology devices catalogue and highly competitive prices, as well as its full commitment to its users through a comprehensive support service. Mario Fernández, IT Manager at BQ, has said: "One of the main BQ objectives is to give the best user support. So, we chose Interoute to provide and guarantee the performance of our telephony service. The VoIP solution provided by Interoute meets all our needs: hosted private cloud, high availability and the ability to quickly scale and expand when needed." Interoute Virtual Data Centre is Interoute's scalable, fully automated Infrastructure as a Service (IaaS) solution. Interoute VDC provides on-demand computing, storage and applications integrated into the heart of its customers' IT infrastructure. This networked cloud replaces the need to buy, manage and maintain physical IT infrastructure and is built into Interoute's fibre connected physical Data Centres world-wide. It's simple to provision, scalable, compliant and cost effective. Diego Matas, General Manager at Interoute Iberia, has added: "We are proud that a Spanish company such as BQ, committed to education and pioneering innovation in exciting fields like robotics and 3D printing, has chosen our cloud platform for its networked communications. Interoute's networked cloud will enable BQ to continue to build upon its excellent reputation for high quality service. We look forward to working with this innovative company to support its future ICT needs."

- Now available with limited-time pricing, package brings together InfinityQS’ cloud-based enterprise quality hub, ProFicient on Demand, with training and ongoing services to ensure a successful deployment - InfinityQS International, Inc., the global authority on real-time quality and Manufacturing Intelligence, announces the launch of ProFicient Now!, a program that blends InfinityQS’ cloud-based enterprise quality hub, ProFicient on Demand, with training and ongoing services to ensure a successful deployment. Available with limited-time pricing, ProFicient Now! aims to give manufacturers the knowledge, tools and continued guidance needed to realise the benefits of a cloud-based quality management program and quickly gain a competitive advantage. “In today’s fast-paced market, manufacturers are looking for ways to better align their quality systems with overall manufacturing excellence goals,” said Doug Fair, Chief Operating Officer, InfinityQS. “By combining the power of the cloud with ongoing expert guidance from our engineering team, ProFicient Now! helps both new and existing clients track towards their goals and achieve a competitive edge through their quality initiatives.” With ProFicient Now!, manufacturers receive expert engineering guidance that leads them through their deployment. Included in the ProFicient Now! package is: Training: Administrators obtain comprehensive skills for building and maintaining the system. Solution Design: The client works closely with InfinityQS to examine the current environment and establish goals for the deployment. Onsite Services: An InfinityQS engineer creates the initial system configurations. Quarterly Consultations: InfinityQS experts guide clients in data analysis and help uncover opportunities for improvement and cost reduction. Executive Review: The InfinityQS engineer leads a review with the client senior management team to review successes, quality enhancements and opportunities for improvement that were uncovered during the use of ProFicient Now! InfinityQS ProFicient is a proven enterprise quality hub powered by a robust, centralised Statistical Process Control (SPC) software engine. ProFicient enables global manufacturers to proactively monitor, analyse and report on Manufacturing Intelligence to improve quality, decrease costs and make smarter business decisions. With a cloud-based deployment option, ProFicient streamlines global data collection and analysis with a unified data archive. For more information about ProFicient Now, including its limited-time pricing, visit here: http://www.infinityqs.com/ProFicient-Now-EMEA

[This article was written by Laura Strassman] AlertSite is now integrated with Ready! API. This means that developers, testers and operations teams can collaborate together on API quality using the same tests and metrics, simplifying configuration of monitoring assets and ultimately turning around performance problems in real time. There are several advantages to this approach: You should be monitoring your APIs in production. When your API moves into production from test, the environment changes – there is no way to know if theAPI performanceis compromised unless you look. Furthermore, you can find problems that may be a result of the location or variance in traffic. There is no easier method. You can simply click a button from right in the Ready! API interface and see the status of your APIs in production it can’t get any easier. You take your already created test cases and turn them into monitors whenever you have a new API or test you want to keep an eye on. Troubleshooting is like shooting fish in a barrel. You wrote the functional test, you know it works, and when something comes back as not working you can isolate it quickly. All of this makes it easy to be ahead of problems, solve them quickly when they happen and keep customers happy. You can be monitoring your APIs in less than 3 minutes: <br>

When running a Maven build with many plugins (e.g. the jOOQ or Flyway plugins), you may want to have a closer look under the hood to see what’s going on internally in those plugins, or in your extensions of those plugins. This may not appear obvious when you’re running Maven from the command line, e.g. via: C:\Users\jOOQ\workspace>mvn clean install Luckily, it is rather easy to debug Maven. In order to do so, just create the following batch file on Windows: @ECHO OFF IF "%1" == "off" ( SET MAVEN_OPTS= ) ELSE ( SET MAVEN_OPTS=-Xdebug -Xnoagent -Djava.compile=NONE -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=5005 ) Of course, you can do the same also on a MacOS X or Linux box, by usingexport intead of SET. Now, run the above batch file and proceed again with building: C:\Users\jOOQ\workspace>mvn_debug C:\Users\jOOQ\workspace>mvn clean install Listening for transport dt_socket at address: 5005 Your Maven build will now wait for a debugger client to connect to your JVM on port 5005 (change to any other suitable port). We’ll do that now with Eclipse. Just add a new Remote Java Application that connects on a socket, and hit “Debug”: That’s it. We can now set breakpoints and debug through our Maven process like through any other similar kind of server process. Of course, things work exactly the same way with IntelliJ or NetBeans. Once you’re done debugging your Maven process, simply call the batch again with parameter off: C:\Users\jOOQ\workspace>mvn_debug off C:\Users\jOOQ\workspace>mvn clean install And your Maven builds will no longer be debugged. Happy debugging!

Spring Integration Kafka 1.2 is out with a major performance overhaul.

Last week’s Gartner IT Operations Strategies & Solutions Summit in Orlando, Fla., was exactly what you’d expect—a place to talk about the IT operations issues impacting some of the largest companies in the world. Even so, there were a few interesting surprises. Among them: 1. Bi-modal is big. Not everyone will succeed. Gartner continued to tell its customers to employ two modes of IT—a traditional, slower moving capability for older, typically internal systems of record; and a high-speed, experimental one for new, typically customer-facing Web and mobile apps. “This is a time of experimentation and innovation,” said Gartner VP and distinguished analyst Chris Howard in his opening keynote. Organizations can’t ignore that there are multiple speeds and they should participate in all. Gartner managing VPRonni Colville added that by 2017, 75% of IT orgs will have this “bi-modal” IT capability. See also: Bi-Modal IT: Gartner Endorses Both Disruptive and Conservative Approaches to Technology However, “50% will make a mess of it,” Colville said. Why? Not necessarily because of technology failings, but more often because of a lack of people skills. 2. IT success is all about people. Donna Scott, also a Gartner VP and distinguished analyst, told her keynote audience that “you will be judged on agility, speed, and innovation.” However, the biggest problems Gartner sees for infrastructure and operations team engagement and innovation are lack of time, company culture that’s not conducive to these approaches, and a lack of business skills in IT. More than half of the people responding to an in-room poll said “people” are the part of IT ops that must change first. Not technology. Gartner research director George Spafford underscored similar issues in large organizations trying to use DevOps at scale: people and “human factors” are the biggest concerns from his in-room poll. All these probably contributed to hiring best-selling author Daniel Pink as a keynote speaker on the opening day of the conference. His focus? Not IT or architecture. Instead, he pounded home the importance of influencing people and selling internally. 3. Big orgs are trying DevOps. But the issues are different at scale. In numerous sessions I saw many hands go up when analysts asked, “Who here is trying DevOps?” Clearly, the approach is getting traction in large companies. But there’s lots of learning still to do. In fact, that was Spafford’s biggest bit of advice. “Always be learning,” he said, “trying to see what works and what breaks, especially at scale.” And, even once you’ve had some initial success, keep learning. “If you’ve done ‪DevOps, stay humble,” he advised. 4. Looking to innovative organizations for ideas … analytics on the rise. Many sessions addressed how large organizations are taking on ideas fostered by smaller, more risk-tolerant companies, and offered advice for doing so successfully. In addition to multiple discussions of DevOps, an entire session was devoted to establishing your own “Genius Bar®—a “walk-up IT support center” as explained in this CIO article. As at previous conferences, Gartner research VP Cameron Haight ran several sessions on lessons learned from firms running massive, Web-scale IT systems. “You need lots of data … and access to it inexpensively,” he said. Some commercial monitoring companies (New Relic included!) got a shout out for taking the lessons of Web scale IT to heart in their offerings. In addition, Haight said, “Analytics are increasingly important for application performance monitoring given the huge amount of data now available.” 5. Cloud: Enterprises want it, but aren’t very good at it yet. Gartner research director Dennis Smith talked through the enterprise’s interest in cloud computing. A huge majority of his in-room poll wanted some mix of both public and private cloud, while only 9% wanted to use only a private cloud environment and a measly 4% were looking to move entirely to the public cloud. The most popular choice (41%) was an 80/20 split between private and public cloud infrastructure. “Enterprises don’t make the dean’s list,” for cloud usage, Smith said, earning no more than a C average in his opinion. Large organizations are doing well at visibility, governance, and delivering standardized stacks, he said, but are less skilled at optimizing for these new environments. Still, Smith said the trends point toward enterprises improving on all fronts. 6. Cloud security can be better than yours. Importantly, Gartner VP and distinguished analyst Neil MacDonald gave the cloud a vote of confidence: noting that, for a variety of reasons, “Well-managed public cloud can be more secure than your own data center.” For example, on-premise software can pose serious security risks, he said, because of “deployment lag” where customers are stuck using software releases with unpatched security vulnerabilities. With a cloud-based Software-as-a-Service (SaaS), security updates can be more quickly rolled out to all customers. But cloud security can be different, requiring a shift to information-level security from OS-level security. Best practices include doing away with a huge pool of all-powerful sysadmins in favor of JEA, or “just enough administration,” where sysadmins have just enough privileges to do their job, and no more. An analogous security practice for compute resources is “least privilege,” where apps and microservices can’t talk to each other unless they specifically need to do so. Audience polling supported MacDonald’s optimistic view of cloud security, which suggests that large enterprises may struggle less with their cloud policies moving forward. 7. Containers: Try ’em! Ahead of this week’s DockerCon in San Francisco, Gartner devoted significant airtime to educating the audience on containers and microservices. My summary of ‪Gartner VP and distinguished analyst Tom Bittman’s advice on containers was simple: Try ’em. Now. Complement them with VMs. ‪And Docker (the company) is important, but not the be-all and end-all in this space. Bittman (copping to some deja vu from Gartner presentations he made on server virtualization 13 years ago) noted that while virtualization has been focused on admin and ops functions, containers are focused on value for developers. But because containers are well suited for driving up VM utilization for workloads that share the same OS, we can expect to see more combinations of containers and server virtualization. Finally, Bittman underscored that Gartner doesn’t see containers having much impact on premise, but making a huge difference in the cloud. That doesn’t necessarily fit with what’s been shown in other research, such as this 2015 State of Containers Survey sponsored by VMblog.com and StackEngine, so we’ll want to watch how this plays out. This is all a lot to digest. The Gartner IT Operations Strategies & Solutions Summitacknowledges the importance of dealing with existing IT systems and practices as well as promising new technologies and thinking, and tries to point a way forward. In fact, Haight had a very good quote about microservices that I thought also served to wrap up the entire event: “If you want to run with the big dogs, you need to rethink application architecture,” he said. That can be very difficult for an enterprise to fully implement … but also very appealing. Note: Al Sargent contributed to this post. All product and company names herein may be trademarks of their registered owners. Server, tortoise and hare, business team, and cloud security images courtesy ofShutterstock.com.

New Integration Provides Seamless Connections Between Perforce Helix and Leading Application Lifecycle Management Systems WOKINGHAM, UK. (June 24, 2015) – Perforce Software, the leader in software configuration management (SCM) and collaboration, and Go2Group, an Atlassian Platinum and Enterprise Expert, today announced the Perforce ConnectALL Adapter. The new adapter for Go2Group’s ConnectALL ALM Router connects Perforce Helix to Application Lifecycle Management (ALM) systems supported by ConnectALL. The companies also announced that they have expanded their partnership, which first began in 2002. “Very few SCMs can handle binary data, and no other SCM solution supports large file formats that scale across globally distributed enterprises like Helix,” said Brett Taylor, president of Go2Group. “Our customers demand future-proof solutions, and with Perforce we know they don’t have to worry about outgrowing their systems—it will serve them well whether they’re a team of 50 or 50,000.” With the Perforce adapter, ConnectALL automatically synchronises data and workflow with other ALM systems and integrates ALM systems components within minutes. “We’re excited to be a part of the ConnectALL ecosystem of adapters and to enable companies to more easily design, configure, synchronise, manage, and monitor their integrations with Perforce,” said Dave Robertson, vice president of Channels at Perforce. “We’re glad to extend our partnership with Go2Group to new technologies and markets.” Go2Group is part of Perforce’s network of sales partners across Europe, the Middle East, Africa, Asia Pacific and India. Perforce partners serve customers in more than 100 countries worldwide. The Perforce ConnectALL Adapter is available for purchase from the Go2Group website.