Written by Jim Magers for LeadingAgile. When I’m teaching an agile bootcamp class and talking about work in process, I always make a point (usually multiple times) to tell the attendees that agile is about finishing work…not about starting work. I reinforce this by pointing out that you can have a glorious looking burndown chart for the duration of the sprint but completely fail in your mission to meet your commitments and finish stories. The team can be burning down hours beautifully on a daily basis, with the remaining task hours looking like they are tracking right along the ideal line, and then boom… It’s closing time for the sprint and no stories actually got completed. Remember that notion of building working, tested software? Didn’t happen. The team started too many stories at once and ended up not being able to bring any of them across the finish line when the bell rang. This notion of finishing work applies to sprint planning as well. If you short-change the time it takes to do good sprint planning, and the team meanders off to begin writing code and test plans too soon, there is a risk that the team is going to struggle to be successful. Remember what we do in sprint planning. Consider velocity, and load the sprint backlog with high priority stories from the product backlog. Check. Determine capacity for the team to work on sprint tasks over the coming sprint. Check. Break stories into tasks and determine who is doing what. Check. Make sure the work is going to fit. Check. Commit to the work. Check. But what can happen when you don’t take the time to thoughtfully break down tasks and estimate hours of effort? Consider the following burn down chart. In this example, the team left sprint planning thinking that they were committing to 830 hours. But just two days into the sprint they discovered additional task hours and instead found themselves in the awkward position of actually needing almost 1200 hours of capacity to complete the committed stories. Guess what…they did not have 1200 hours of capacity to give, especially since they were now a full two days into the work. In looking more closely at their burndown chart over the course of the two-week sprint, it took them almost 6 days of work to get back to the point where they had 830 hours of work left to do. Six days just to get back to where they thought was their starting point when they concluded their sprint planning meeting. And surprise, they didn’t finish the sprint successfully. So, don’t short-change the value that good sprint planning affords. Yes, it takes time. Yes, it can seem tedious. Yes, the team is anxious to get started. But good sprint planning pays dividends. Remember that it is about finishing work, and not about starting work.

Introduction As we all know about the Identity columns and Merge statement. We are not going to discuss any boring theoretical tropics related to it. Better we are discussing here with a practical scenario of merging records. Hope all of you must enjoy it and it will be informative. The Scenario We have a Table with Identity Columns named #tbl_TempStudentRecords. The table details are mentioned bellow. Column Name SrlNo StudentName StudentClass StudentSection We have another table named #tbl_TempStudrntMsrks. The table details are mentioned bellow. Column Name SrlNo StubjectName MarksObtain What we want to do is, we have another set of table called tbl_StudentDetails mentioned bellow. Column Name StdRoll (PK) StudentName StudentClass StudentSection Another table named tbl_StudentMarks Column Name IdNo (PK) StdRoll (FK) References [tbl_StudentDetails].[StdRoll] SubjectName MarksObtain Her we can insert records very easily in tbl_StudentDetails from #tbl_TempStudentRecords very easily. But the main problem is the IDENTITY columns in the Table named [tbl_StudentDetails].[ StdRoll]. When we insert records the Identity columns values generate automatically. When we are trying to insert records into the table named tbl_StudentMarks from Table named #tbl_TempStudrntMsrks we have to provide the StdRoll values, which is the Foreign Key References to the Table named [tbl_StudentDetails].[ StdRoll]. Think one minute with the case scenario. Hope you can understand the problem. Now we have to solve it and we are not using any LOOP for that and NOT even any DDL operation to change the structure of base table. We are just using the SET BASED operation to make performance high. How to Solve it Step – 1 [ Create the Base Table First ] IF OBJECT_ID('tempdb..#tbl_TempStudentRecords')IS NOT NULL BEGIN DROP TABLE #tbl_TempStudentRecords; END GO CREATE TABLE #tbl_TempStudentRecords ( SrlNo BIGINT NOT NULL, StudentName VARCHAR(50) NOT NULL, StudentClass INT NOT NULL, StudentSection CHAR(1) NOT NULL ); GO IF OBJECT_ID('tempdb..#tbl_TempStudrntMsrks')IS NOT NULL BEGIN DROP TABLE #tbl_TempStudrntMsrks; END GO CREATE TABLE #tbl_TempStudrntMsrks ( SrlNo BIGINT NOT NULL, StubjectName VARCHAR(50) NOT NULL, MarksObtain INT NOT NULL ); GO IF OBJECT_ID(N'[dbo].[tbl_StudentDetails]', N'U')IS NOT NULL BEGIN DROP TABLE [dbo].[tbl_StudentDetails]; END GO CREATE TABLE [dbo].[tbl_StudentDetails] ( StdRoll BIGINT NOT NULL IDENTITY(100,1) PRIMARY KEY, StudentName VARCHAR(50) NOT NULL, StudentClass INT NOT NULL, StudentSection CHAR(1) NOT NULL ); GO IF OBJECT_ID(N'[dbo].[tbl_StudentMarks]', N'U')IS NOT NULL BEGIN DROP TABLE [dbo].[tbl_StudentMarks]; END GO CREATE TABLE [dbo].[tbl_StudentMarks] ( IdNo BIGINT NOT NULL IDENTITY(1,1) PRIMARY KEY, StdRoll BIGINT NOT NULL, SubjectName VARCHAR(50) NOT NULL, MarksObtain INT NOT NULL ); GO ALTER TABLE [dbo].[tbl_StudentMarks] ADD CONSTRAINT FK_StdRoll_tbl_StudentMarks FOREIGN KEY(StdRoll) REFERENCES [dbo].[tbl_StudentDetails](StdRoll); Step – 2 [ Inserting Records in Temp Table ] INSERT INTO #tbl_TempStudentRecords (SrlNo, StudentName, StudentClass, StudentSection) VALUES(1, 'Joydeep Das', 1, 'A'), (2, 'Preeti Sharma', 1, 'A'), (3, 'Deepasree Das', 1, 'A'); INSERT INTO #tbl_TempStudrntMsrks (SrlNo, StubjectName, MarksObtain) VALUES (1, 'Bengali', 50), (1, 'English', 70), (1, 'Math', 80), (2, 'Bengali', 0), (2, 'English', 70), (2, 'Math', 80), (3, 'Bengali', 20), (3, 'English', 90), (3, 'Math', 95); Step – 3 [ Now Solve it By MERGE Statement ] BEGIN DECLARE @MappingTable TABLE ([NewRecordID] BIGINT, [OldRecordID] BIGINT) MERGE [dbo].[tbl_StudentDetails] AS target USING (SELECT [SrlNo] AS RecordID_Original ,[StudentName] ,[StudentClass] ,[StudentSection] FROM #tbl_TempStudentRecords ) AS source ON (target.StdRoll = NULL) WHEN NOT MATCHED THEN INSERT ([StudentName], [StudentClass], [StudentSection]) VALUES (source.[StudentName],source.[StudentClass], source.[StudentSection]) OUTPUT inserted.[StdRoll], source.[RecordID_Original] INTO @MappingTable; --- Now Map table is ready and we can use it --- INSERT INTO [dbo].[tbl_StudentMarks] (StdRoll, SubjectName, MarksObtain) SELECT b.NewRecordID, a.StubjectName, a.MarksObtain FROM #tbl_TempStudrntMsrks AS a INNER JOIN @MappingTable AS b ON a.SrlNo = b.OldRecordID; END GO Step – 4 [ Observation ] SELECT * FROM [dbo].[tbl_StudentDetails]; GO SELECT * FROM [dbo].[tbl_StudentMarks]; GO StdRoll StudentName StudentClass StudentSection 100 Joydeep Das 1 A 101 Preeti Sharma 1 A 102 Deepasree Das 1 A IdNo StdRoll SubjectName MarksObtain 1 100 Bengali 50 2 100 English 70 3 100 Math 80 4 101 Bengali 0 5 101 English 70 6 101 Math 80 7 102 Bengali 20 8 102 English 90 9 102 Math 95

Here is the third post in a six part series focusing on DevOps Days Austin. Following Cote’s presentation, next in the line up of speakers on day one was Paul Read of Release Engineering Approaches. Paul’s talk tackles the “importance of continually breaking your organization with experiments, some surprising examples, and how to do it in such a way that you aren’t left with a broken organization.” Take a listen, and check out his awesome tie! (also see his slides below) <br> Some of the ground Paul covers: Learning from Toyota Kata The culture of continuous learning and experimentation/improvement Making “the pain visible” Where Paul sees DevOps going in the future Still to come Cameron Haight – Gartner John Willis – Docker Barton George (Sputnik ignite talk) – Dell Extra credit reading The coming donkey apocalypse — Cote Opening Keynote – Damon Edwards Pau for now…

We’re going back to the classics this month, with the latest enterprise collaboration news round-up focussing on cloud strategy and considerations and benefits when it comes to implementing collaboration software. Mashable shared an infographic it created in conjunction with Hewlett Packard, which compiles data and research suggesting that the use of hybrid and private cloud computing is on the rise. The article quotes statistics from Rightscale, which states that 82% of enterprises have a multi-cloud strategy already, and of these 14% use multiple private clouds, 13% use multiple public clouds, and 55% use hybrid clouds. Mashable quotes Technology Business Research, which states that “there is continued migration of enterprise vendors in mature markets such as the U.S. to hybrid and private cloud platforms to provide software vendors an opportunity to generate adoption for management technologies, as customers require next-generation tools to manage heterogeneous IT infrastructures efficiently.” In his article for eWEEK, Chris Preimesberger outlines 10 ways IT and business leaders must collaborate on cloud strategies. Chris explains that a decision to use cloud services is no longer simply down to the IT department. During the last nine years, he says, entire businesses have become necessarily immersed in IT strategies in order to harness the cloud for economics, innovation, operations and growth. He shares a slide show which provides advice for how technical and business leaders can collaborate to build a secure cloud strategy. The slide show states that usage indicate that private clouds are expected to grow at double the rate of public cloud, a result of ongoing concerns about data security and privacy. Gary Audin asks the question cloud economics or flexibility? in his article for No Jitter. Gary explains that although the cost of cloud can be attractive, that might not be the real draw for enterprises. He states that knowing what costs to consider as part of a cloud service implementation is vital to making the right decision about cloud. Gary points out the benefits of the cloud as being far more than simply a matter of cost. He explains that the cloud allows rapid response for an enterprise as it contends with change due to situations such as staff growth or reduction, market fluctuations, financial limitations, or new opportunities. Above all, Gary explains, the cloud delivers flexibility and it is this which makes it the most attractive option for enterprises. In his article for MSP Mentor, Michael Brown reveals the result of a recent report on cloud adoption in the enterprise. The report, by Skyhigh Networks, revealed that enterprise cloud adoption grew by 43% in 2014. Michael highlights findings on the file sharing front, revealing that 37 percent of employees were found to be uploading sensitive business data to consumer file sharing services. Consumer file sharing services are one element of a growing trend towards BYOC (bring your own cloud, content and collaboration). Robert Bamforth explains that BYOC is an evolution of BYOD (bring your own device) which posed a challenge to IT departments since the rise of the smartphone. Robert explains that BYOC is a new challenge for IT departments in controlling their organisation’s digital assets while liberating employee productivity and information sharing. Robert states that the BYOC conundrum should change as enterprise-strength security features and tools continue to evolve to have more consumer-like interfaces, which will make asking employees to use enterprise tools much easier. He gives some suggestions to help enterprises in the mean time: understand the appeal of consumer tools, make sure everyone understands security risks, forget trying to apply strong rules to trivial information, get a mobile-ready solution, look for and pre-plug data leaks, and above all don’t stop collaboration if it’s happening. In his article for ZDNet, Dion Hinchcliffe reflects on the state of the digital collaboration industry. Far from maturing, Dion says, the collaboration tool space is busier than ever evolving, branching out, and multiplying. But, he asks, are organizations able to adopt so many different ways of working together? Dion observes that instead of settling down, the collaboration software space is actually get more interesting and varied, and he is seeing new technologies, such as applications that focus on optimizing collaboration for mobile devices or for team analytics. It’s now time for organizations to design a strong foundation for digital collaboration, says Dion, as the near future promises many key new innovations that must be considered and incorporated to stay competitive, both to customers and the workforce. When businesses do decide to adopt one or more digital collaboration platforms, Andre Bourque offers some helpful ways in which to measure ROI. Andre quotes a Mashable report which states that cloud collaboration drives creativity and engagement, leading to happier employees and a better company culture, but this is not a metric that is easily measurable. Andre explains that it’s hard to find definitive examples of ROI, as most are anecdotal or “in process”, and merely counting user adoption rate of a collaborative platform is inadequate. Instead, Andre quotes Angela Ashenden, of MWD Advisors, who offers the following metrics to consider: reduced travel time and costs; creating new business opportunities and services; increased employee retention rates, cost savings across the organisation, and faster on-boarding for new users. Do you have any metrics that you find useful to measure ROI on your collaboration platform in your organisation?

Docker deployments can be very dynamic with containers being started and stopped, moved around the YARN or Mesos-managed clusters, having very short life spans (the so-called pets) or long uptimes (aka cattle). Getting insight into the current and historical state of such clusters goes beyond collecting container performance metrics and sending alert notifications. If a container dies or gets paused, for example, you may want to know about it, right? Or maybe you’d want to be able to see that a container went belly up in retrospect when troubleshooting, wouldn’t you? Just two weeks ago we added Docker Monitoring (docker image is right here for your pulling pleasure) to SPM. We didn’t stop there — we’ve now expanded SPM’s Docker support by adding Docker Event collection, charting, and correlation. Every time a container is created or destroyed, started, stopped, or when it dies, spm-agent-docker captures the appropriate event so you can later see what happened where and when, correlate it with metrics, alerts, anomalies — all of which are captured in SPM — or with any other information you have at your disposal. The functionality and the value this brings should be pretty obvious from the annotated screenshot below. Like this post? Please tweet about Docker Events and Docker Metrics Monitoring Know somebody who’d find this post useful? Please let them know… Here’s the list of Docker events SPM Docker monitoring agent currently captures: Version Information on Startup: server-info – created by spm-agent framework with node.js and OS version info on startup docker-info – Docker Version, API Version, Kernel Version on startup Docker Status Events: Container Lifecycle Events like create, exec_create, destroy, export Container Runtime Events like die, exec_start, kill, oom, pause, restart, start, stop, unpause Every time a Docker container emits one of these events spm-agent-docker will capture it in real-time, ship it over to SPM, and you’ll be able to see it as shown in the above screenshot. Oh, and if you’re running CoreOS, you may also want to see how to index CoreOS logs into ELK/Logsene. Why? Because then you can have not only metrics and container events in one place, but also all container and application logs, too! If you’re using Docker, we hope you find this useful! Anything else you’d like us to add to SPM (for Docker or anyother integration)? Leave a comment, ping @sematext, or send us email – tell us what you’d like to get for early Christmas!

DevOps can help reduce technical debt in some fundamental ways. Continuous Delivery/Deployment First, building a Continuous Delivery/Deployment pipeline, automating the work of migration and deployment, will force you to clean up inconsistencies and holes in configuration and code deployment, and inconsistencies between development, test and production environments. And automated Continuous Delivery and Infrastructure as Code gets rid of dangerous one-of-a-kind snowflakes and configuration drift caused by making configuration changes and applying patches manually over time. Which makes systems easier to setup and manage, and reduces the risk of an un-patched system becoming the target of a security attack or the cause of an operational problem. A CD pipeline also makes it easier, cheaper and faster to pay down other kinds of technical debt. With Continuous Delivery/Deployment, you can test and push out patches and refactoring changes and platform upgrades faster and with more confidence. Positive Feedback The Lean feedback cycle and Just-in-Time prioritization in DevOps ensures that you’re working on whatever is most important to the business. This means that bugs and usability issues and security vulnerabilities don’t have to wait until after the next feature release to get fixed. Instead, problems that impact operations or the users will get fixed immediately. Teams that do Blameless Post-Mortems and Root Cause(s) Analysis when problems come up will go even further, and fix problems at the source and improve in fundamental and important ways. But there’s a negative side to DevOps that can add to technical debt costs. Erosive Change Michael Feathers’ research has shown that constant, iterative change is erosive: the same code gets changed over and over, the same classes and methods become bloated (because it is naturally easier to add code to an existing method or a method to an existing class), structure breaks down and the design is eventually lost. DevOps can make this even worse. DevOps and Continuous Delivery/Deployment involves pushing out lots of small changes, running experiments and iteratively tuning features and the user experience based on continuous feedback from production use. Many DevOps teams work directly on the code mainline, “branching in code” to “dark launch” code changes, while code is still being developed, using conditional logic and flags to skip over sections of code at run-time. This can make the code hard to understand, and potentially dangerous: if a feature toggle is turned on before the code is ready, bad things can happen. Feature flags are also used to run A/B experiments and control risk on release, by rolling out a change incrementally to a few users to start. But the longer that feature flags are left in the code, the harder it is to understand and change. There is a lot of housekeeping that needs to be done in DevOps: upgrading the CD pipeline and making sure that all of the tests are working; maintaining Puppet or Chef (or whatever configuration management tool you are using) recipes; disciplined, day-to-day refactoring; keeping track of features and options and cleaning them up when they are no longer needed, getting rid of dead code and trying to keep the code as simple as possible. Microservices and Technology Choices Microservices are a popular architectural approach for DevOps teams. This is because loosely-coupled Microservices are easier for individual teams to independently deploy, change, refactor or even replace. And a Microservices-based approach provides developers with more freedom when deciding on language or technology stack: teams don’t necessarily have to work the same way, they can choose the right tool for the job, as long as they support an API contract for the rest of the system. In the short term there are obvious advantages to giving teams more freedom in making technology choices. They can deliver code faster, quickly try out prototypes, and teams get a chance to experiment and learn about different technologies and languages. But Microservices “are not a free lunch”. As you add more services, system testing costs and complexity increase. Debugging and problem solving gets harder. And as more teams choose different languages and frameworks, it’s harder to track vulnerabilities, harder to operate, and harder for people to switch between teams. Code gets duplicated because teams want to minimize coupling and it is difficult or impossible to share libraries in a polyglot environment. Data is often duplicated between services for the same reason, and data inconsistencies creep in over time. Negative Feedback There is a potentially negative side to the Lean delivery feedback cycle too. Constantly responding to production feedback, always working on what’s most immediately important to the organization, doesn’t leave much space or time to consider bigger, longer-term technical issues, and to work on paying off deeper architectural and technical design debt that result from poor early decisions or incorrect assumptions. Smaller, more immediate problems get fixed fast in DevOps. Bugs that matter to operations and the users can get fixed right away instead of waiting until all the features are done, and patches and upgrades to the run-time can be pushed out more often. Which means that you can pay off a lot of debt before costs start to compound. But behind-the-scenes, strategic debt will continue to add up. Nothing’s broke, so you don’t have to fix anything right away. And you can’t refactor your way out of it either, at least not easily. So you end up living with a poor design or an aging technology platform, slowly slowing down your ability to respond to changes, to come up with new solutions. Or forcing you to continue filling in security holes as they come up, or scrambling to scale as load increases. DevOps can reduce technical debt. But only if you work in a highly disciplined way. And only if you raise your head up from tactical optimization to deal with bigger, more strategic issues before they become real problems.

I am sure by now most people understand that there is strong correlation between employee satisfaction and business results. If you need more convincing have a read of these two articles: Forbes & Research Paper So how do you best go about measuring it? On my current project I have decided to go with the following 4 questions: I would recommend this account and my project as a good place to work I have the tools and resources to do my role well I rarely think about rolling off this account or project My role makes good use of my skills and abilities For those of you who have read Jez Humble’s “Lean Enterprise”, these questions will look familiar. I have adopted them to the project setting that I work within. We have just set out on a cultural transformation to become truly Agile and adopt DevOps in a large complex legacy environment. To me measuring the above will give me the best indicator that we are doing the right thing. Of course there will be other measures who determine the quality of the outcomes and the levels of automation among others, but changing the culture of an organisation is critical if your Agile and DevOps adoption is to be successful. I will report back throughout that journey to tell you what my experiences is with the above questions. IT delivery is complex and it is not always clear what the right solution is. I found in the past that it is near impossible to create processes and tools that work by itself, you need to have the right mindset that people use the processes and tools with the right intent. It’s very frustrating when you implement great automation only to see a few months later that the solution has degraded. It is with hindsight that I understand that the solution is to not just implement process and tools but to instill the right culture and mindset for progression, a culture where we blamelessly identify a way to avoid the same mistake again rather than looking for the person in fault, a culture where we strive for automation and lean processes and are not concerned about the size of our teams or budgets, a culture where you don’t have to protect your fiefdom and where you are happy to collaborate with others to solve problems no matter where the root cause lies. I think we all in IT need to understand this dynamic between employee satisfaction and outcomes better, I for sure believe that I have come across a magic potion that I aim to bring to all my future projects. About these ads

Leading today, VentureBeat reports on the quiet launch of Google’s Cloud Source Repositories. This seems like something we should have heard more about, but I don’t remember seeing anything about it. Amazon AWS announces the availability of all things Alexa, the Skills Kit, the Voice Service and a Fund. Last but not least, we have AJ Kohn, from Blind Five Year Old, talking about click-through rate being a ranking signal on Google’s search results. I don’t talk about SEO much, but reading AJ’s work is always fascinating. As always, enjoy today’s items, and please participate in the discussions on these sites. Top Stories Google has quietly launched a GitHub competitor, Cloud Source Repositories | VentureBeat Alexa Skills Kit, Alexa Voice Service, Alexa Fund | AWS Official Blog Startups, Career and Process Why offices are where work goes to die | Swizec Teller Unleashing the power of small teams | Andreas Papathanasis What Is A Tester? | Developsense Blog What happens when you stop relying on resumes | Aline Lerner Design and Development Swift 2: SIMD | Russ Bishop Why is Git better than Mercurial? | Javalobby Create a Maven archetype | Javalobby pip -t: A simple and transparent alternative to virtualenv | Zoomer Analytics Killing Off Wasabi – Part 1 | Fog Creek Blog WebAssembly- Explained | Modus Create Generating JSON Schema from XSD with JAXB and Jackson | Inspired by Actual Events AI, Machine Learning, Research and Advanced Algorithms Applying Machine Learning to Text Mining with Amazon S3 and RapidMiner | Amazon AWS Big Data, Visualization, SQL and NoSQL Is Click Through Rate A Ranking Signal? | Blind Five Year Old Cache-friendly binary search | Bannalia Discovering the Computer Science Behind Postgres Indexes | Java Code Geeks How an open-source competitive benchmark helped to improve databases | ArangoDB Security, Encryption and Cryptography Cracking JXcore… Again | Mark Haase Link Collections The Daily Six Pack: June 25, 2015 | Dirk Strauss Double Shot #1517 | A Fresh Cup Dew Drop – June 25, 2015 (#2042) | Morning Dew The Daily Six Pack: June 26, 2015 | Dirk Strauss

Every business has unique characteristics that set it apart from other organizations, even within the same industry. For this reason, it shouldn't be surprising that there's not a one-size-fits-all approach to app security testing. Each company has certain protection expectations and regulations to adhere to, making it essential to find the best way to achieve these goals. Here are a few examples of security testing methodologies available for quality assurance teams to leverage: Black box With black box testing, QA professionals put themselves in the shoes of the hacker and attempt to break the app through various attack vectors. This methodology can yield a lot of information and help better secure the program from actual threats. A white paper by Security Innovation noted that software testers first analyze the system's architecture and business model to identify any security vulnerabilities. Looking over the software logic in this way can uncover subtle security and privacy issues that may not have been noticed otherwise, such as defects in design, input, system dependency, authentication, cryptography and information disclosure. "Although white box code inspection is good for analyzing static behavior, only black box exploratory testing can determine the dynamic behavior of how a system is implemented and used, the coupling between systems and the interactions of the distributed systems," Security Innovation wrote. Dynamic For QA teams that like to execute code, dynamic testing is the approach for them. This methodology checks the running application for how it behaves and responds to a variety of inputs. This is done to ensure that the product meets up with established regulations and is giving the expected outcomes. IBM noted that dynamic analysis is especially useful to identify code coverage, as it can discover bugs in paths that have gone untested. While dynamic testing can be manual work for testers, it can also yield significant information that will help mitigate defects and produce quality products. Static In contrast to dynamic testing, static approaches directly review the source code, often through an automated test management solution. TechTarget contributor Michael Cobb noted that this methodology occurs at the implementation phase, rather than when the app is running, and often helps mitigate vulnerabilities involved with industry compliance standards. Automation in this area can reduce the amount of time it takes to complete these tasks. However, it may not be able to detect sophisticated threats, which can be supplemented by dynamic security testing. "A thorough source code review has an advantage over dynamic testing," Cobb wrote. "Nothing is hidden from analysts during a source code review, so they can examine exactly how data flows through a program. By solving the problem at the code level, static testing reduces the number of security-related design and coding defects, and the severity of any defects that make it through to the release version, thus dramatically improving the overall security of the application." There are a number of security testing methodologies that organizations can pursue, and designing a combination of approaches may result with a solution that's best for their needs. Using these strategies, companies can better ensure the protection of sensitive information while providing users with the software testing tools needed to succeed.

When a team works with a lot of maven projects it becomes quickly painful to do some basic tasks like: manage versionning and connections between the different projects. releasing and opening versions, especially when the maven-release plugin needs to be run on many projects and when versionning is not standard. managing external dependencies also can become complex, and ensuring that a single version of a dependency is used accross different projects is sometimes not a trivial question to ask. in a one word, applying transformations on a dependency graph is difficult. mind-mapping the dependency graph is difficult when the number of projects grows. That can increase the amount of time needed by new people to understand a project graph, and that also makes maintaining and changing things difficult. checking consistency and optimizing the dependency graph is not an easy task neither. having an always up-to-date build of snapshots and release is not easy when projects are distributed everywhere. Pom Explorer So there is the Pom Explorer tool which tries to address those problems by providing those functionalities : release a graph : release a pom or all poms and its/theirs dependencies and updates all dependent poms accross multiple repositories and projects, change a gav : updates a project’s gav and make all the project which depends on it follow this change. manages properties, dependency management, and so on. Pom-explorer knows what pom.xml to update and where to update. If a dependency specifies ${foobar.version}, pom-explorer will go to update the foobar.version property. query the dependency graph to retrieve pertinent information about your projects, statistics and check functions are also available, display 3d interactive graph, export graphml files, find not used dependencies and other similar problems, list java classes provided by artifacts, list java classes referenced by artifacts, runs a light and efficient web server so local and shared usage is possible. The tool will also support automatically building projects in order to always have such or such project always up to date. Use cases In this article, I will show some common use cases possible with this tool. Installation First one needs to install and run the software. Put yourself in a temporary directory and type those commands : git clone https://github.com/ltearno/pom-explorer.git cd pom-explorer java -jar target/pom-explorer.jar The program should welcome you and ask you to go to this address : http://localhost:90 This is the console to the application. You can type commands in the prompt, they will be sent to the server and it will answer. You can use up and down arrows to recall past commands. Let’s start by typing ? to get the available commands : Analyze of repositories OK. First we will analyse a directory where there are many maven projects, then we will work a bit to optimize those projects. You will have to adapt the exercise to your computer. Let’s analyse my git repositories directory : analyze directory c:\documents\repos This will analyze my projects and construct an in-memory graph of the dependency graph : Now, the program knows about everything on my projects, let’s start asking questions ! List of GAVs… Let’s get the list of all existing GAVs (groupId, artifactId, version) in the graph. There will be my projects and all the GAVs on which they depend. Type this command : gav li Note that you can type only the first letters of a command, as long as there is no ambiguity. Here li stands for list. Find dependencies on an obsolete artifact As I look through the list of GAVs, I remark that there are still an old snapshot version of the hexa.binding artifact hanging around. The latest released version is 1.3 and the working version is 1.4-SNAPSHOT so the version 1.3-SNAPSHOTshould not be used anymore. Which is the project still depending on this very deprecated this version ? Let’s ask the question : depends on fr.lteconsulting:hexa.binding:1.3-SNAPSHOT Here it is ! the project rigpa.org:regsys-clients:1.0.0-SNAPSHOT is still using an old snapshot. Let’s arrange that. Pom Explorer is able to change the pom properties and dependencies by itself. Updating this wrong dependency What we want is to change fr.lteconsulting:hexa.binding:1.3-SNAPSHOT tofr.lteconsulting:hexa.binding:1.3 so that the project uses the latest release available. We could desire to change for fr.lteconsulting:hexa.binding:1.4-SNAPSHOT which would be possible with the same command as we’ll see. For that we will use the change gav command : cha ga fr.lteconsulting:hexa.binding:1.3-SNAPSHOT fr.lteconsulting:hexa.binding:1.3 Here is what Pom Explorer answers : So first Pom Explorer finds what needs to be changed in the graph. This might be the project itself and all projects which depend on it. After that the program begins a loop in which all changes are checked and appropriately transformed when needed. For instance changing a dependency version can become changing a property value. Changes are first resolved as described before and they are then transformed in a change list to apply to be applied to pom.xml files. In the ouput, there is first a little warning saying thefr.lteconsulting:hexa.binding:1.3-SNAPSHOT project was not found. That’s normal because the project in now in version 1.4-SNAPSHOT. So there is no need to modify it. Then in the change list section, the changes that are to be applied to pom.xml files are listed. The first one says ‘project not found’ and that’s ok as seen before. The second one says to modify the C:\documents\repos\regsys-clients\pom.xmland change the dependency ([DEPENDENCY])fr.lteconsulting:hexa.binding:1.3-SNAPSHOT tofr.lteconsulting:hexa.binding:1.3. The “causes” message is useful when a change is caused by other changes (as said before a dependency change can become one or several property changes). If we had properties involved, Pom Explorer would have found them and included them in the change set. Now that we reviewed the proposed changes and agreed with them, let’s apply them by using the same command with the -apply flag : cha ga fr.lteconsulting:hexa.binding:1.3-SNAPSHOT fr.lteconsulting:hexa.binding:1.3 -apply We see that at the end of the same process, the program updated the dependency in the right pom.xml file. Let’s have a look at the file it self : fr.lteconsulting hexa.binding 1.3 compile OK, the file is correct now… Oh well no ! I just find other dependencies in SNAPSHOT versions ! Finding more duplicate and obsolete dependencies Let’s accept it, our projects are not up to date. Well let’s see how many of those artifacts there are with multiple versions used. For that i type the checkcommand : Ok there is some work to do ! Opening a version Now let’s look at another use case. Say that the hexa.binding project is in version 1.3 and i want to open the version 1.4-SNAPSHOT. I also want all the projects which depend on version 1.3to move to 1.4-SNAPSHOT. On the way, I want all modified projects still in a release version to be SNAPSHOT-ized too. And i want this to happen recursively as new projects are opened. With Pom Explorer, that’s only one command : change gav fr.lteconsulting:hexa.binding:1.3 fr.lteconsulting:hexa.binding:1.4-SNAPSHOT As you can see, warnings are generated when projects are reopened : Those are normal warnings, they are just here so that you know what happens. Then, there is a big list of changes to be made, because the hexa.bindingartifact is used in many central projects that were in a release state. Glad that we didn’t do that by hand ! Even with the maven-update-version plugin, there would have been a lot of repositories to go to open and update. Let’s apply the changes with the -apply flag : cha ga fr.lteconsulting:hexa.binding:1.3 fr.lteconsulting:hexa.binding:1.4-SNAPSHOT -apply All the changes have been made, about 30 of them. In one go ! Refresh the page so that a new session is created from the changed files. We can see that many of the projects have been reopened : You now have to commit all the repositories with this update. Pom explorer does not do that yet, but maybe in the future ! Releasing many poms Imagine the sprint is almost finished now and it’s now time to release the projects. Type the gav li fr.lteconsulting again to get the GAVs list (fr.lteconsulting is my projects package name, so I filter GAVs with that), choose one and let’s release it : fr.lteconsulting:hexa.binding.samples:1.4-SNAPSHOT The thing in the release is to have all direct and transitive dependencies released too. That’s what Pom Explorer checks. It then generates a change list to materialize your requirements. Other use cases Listing provided and referenced classes You can ask which Java classes are provided and referenced by GAVs. That’s sometimes a useful information to have. Try those commands : classes providedBy fr.lteconsulting:hexa.css:1.3 classes referencedBy fr.lteconsulting:hexa.css:1.3 Optimizing your project’s dependencies Sometimes, you ask yourself “do I still need this and that dependency ?” but you are not very sure, and since you lack time to investigate, eventually the dependency stays in your project for a long time, causing of course maintenance issues sometime. Let’s have Pom Explorer help us in the quest for the obsolete dependency. garbage dep fr.lteconsulting:carousel:1.0-SNAPSHOT This will give you something like that : You can refer to the project documentation to find how to use those informations. But sure that it can help you give away those useless dependencies ! Other goodies : graphs ! Pom Explorer can do two other things to help you visualize your dependency graph : export GraphML files so you can use them in another graph software (like yEd for instance). display an interactive 3d graph Exporting GraphML files GraphML is an open format to describe graphs. With the graph exportcommand, you can get graphml files of your working session. The program will create two files and display the links to them. Those two files are corresponding to two graphs : the dependency graph as usual, and the dependency graph between the git repositories containing your projects. Sometime one git repository can contain multiple projects and a view of the dependencies at the repository level is useful in those cases. This is the kind of picture you can get easily from editors like yEd : Interactive 3D graph Thanks to the WebGL standard which allows direct access to the 3D hardware on the running machine and thanks to libraries like three.js and ngraph.pixel, it is possible to display an interacive 3d graph. More over it is possible to customize the appearance of the graph to give account of different perspectives. Type the graph command and click on the link. This will open another tab containing the living 3d graph of your projects. When focus is given to the 3d viewport, the W, A, S, F and arrow keys allow to move in the 3d space. On the right, there is a text area where you can edit some javascript callback to customize the graph appearance. You can also stop the moving of the particle with the checkbox at the bottom right of the screen. It is not necessarilly useful, but sometimes it is relaxing to admire your work in the form of a living and moving graph ! Conclusion There are many other functions in Pom Explorer, but they are for you to discover now. This tool finds easily its place in the daily workflow because of the functions it provides. The fact that one can run it locally or on a shared server allows to use it as you wish. It is still in early development phase so many more functionalities could come up. On this subject, don’t hesitate submitting a little pull request on the GitHub repository… Pom Explorer is made with love by LTE Consulting

[This article was written by Wyatt Lindsay] New Relic’s Community Forum is designed to be a place for our users to share their experiences, questions, problems, and fixes. The collective expertise and creativity of the New Relic community has generated some outstanding solutions to everyday issues, and we want to call out some of them in the area of .NET agent deployment excellence. Basic installation Installing New Relic’s .NET agent is designed to be simple: run the installer on the target host and choose the features you want to include. For Microsoft Azuredeployments, install one of our NuGet packages. Installation requires a reset of IIS for the software to load into your application. To upgrade, we recommend first stopping IIS, installing the newer agent version, and then starting up IIS again. It’s also possible to perform a “silent” (manual) install using msiexec.exe, for example: msiexec.exe /i C:\NewRelicAgent.msi /qb NR_LICENSE_KEY= INSTALLLEVEL=1 See the documentation for complete manual installation options. Leveraging PowerShell Scripting the silent installation provides convenience and flexibility. Here’s an example script provided by community member Jon Carl in this post: $msiName = $licenseKey = $arguments = "/i $msiName /L*v install.log /qn NR_LICENSE_KEY=$licenseKey" if ($msiName -ne $null) { $exitCode = (Start-Process -FilePath "msiexec" -ArgumentList $arguments -Wait -PassThru).ExitCode; if($exitCode -eq 0) { Write-Host "Installation successful!" -ForegroundColor Green } else { Write-Host "Installation unsuccessful. Exitcode: $exitCode" -ForegroundColor Red } } This script works great when run directly on the target machine. Another forum user (Kym McGain) noticed that the installation didn’t complete before the session ended when executing the script remotely. This caused the installer to quit partway through. Kym posted this script that uses a ‘while’ loop to ensure the installer completes. As a bonus, it stops IIS before and restarts it after the software installs. As mentioned above, these steps are usually needed when upgrading. $installNewRelic = { $runProcess = { param($process,$arguments) $res = Start-Process -FilePath $process -ArgumentList $arguments -Wait -PassThru while ($res.HasExited -eq $false) { Write-Host "Waiting for $process..." Start-Sleep -s 1 } $exitCode = $res.ExitCode if($exitCode -eq 0) { Write-Host "$process successful!" -ForegroundColor Green } else { Write-Host "$process unsuccessful. Exitcode: $exitCode" -ForegroundColor Red } } $msiName = $licenseKey = $arguments = "/i $msiName /L*v install.log /qn NR_LICENSE_KEY=$licenseKey" Invoke-Command $runProcess -ArgumentList "IISRESET","/STOP" Invoke-Command $runProcess -ArgumentList "msiexec.exe",$arguments Invoke-Command $runProcess -ArgumentList "IISRESET","/START" } Chef, Puppet, and Chocolatey Deployment options abound for modern Web developers. These solutions often require a known download path and installer name. New Relic offers a consistent filepath and MSI name for the agent in an effort to make automated deployment easier for .NET customers: http://download.newrelic.com/dot_net_agent/release/NewRelicDotNetAgent_x64.msi http://download.newrelic.com/dot_net_agent/release/NewRelicDotNetAgent_x86.msi Several Community members have created packages for these utilities. Chocolatey users are invited to use the following NuGet package created by kireevco: https://chocolatey.org/packages/newrelic-dotnet New Relic community member ePitty built a Puppet module to handle .NET agent deployment: https://github.com/epitty1023/puppet-newrelicappmon Chef users, meanwhile, can check out the following cookbook for .NET and many other platforms New Relic supports: http://community.opscode.com/cookbooks/newrelic New Relic Community member E_Bow wrote a Chef recipe that goes a step further by stopping IIS before the installation and starting it again after completion: #Stop IIS iis_site 'Website' do action [:stop] end # install latest Newrelic agent from web include_recipe 'newrelic::repository' include_recipe node['newrelic']['dotnet-agent']['dotnet_recipe'] license = node['newrelic']['application_monitoring']['license'] windows_package 'Install New Relic .NET Agent' do source node['newrelic']['dotnet-agent']['https_download'] options "/qb NR_LICENSE_KEY=#{license} INSTALLLEVEL=#{node['newrelic']['dotnet-agent']['install_level']}" installer_type :msi action :install end #Start IIS iis_site 'Website' do action [:start] end The author states that they were unable to pull the New Relic license key from the configuration JSON in Chef Overrides, requiring them to modify the config file on each machine and manually enter the key. We invite any Chef experts out there to extend and improve this recipe so that it correctly pulls the license key. We are continually impressed by the smarts and spirit of our New Relic Community Forum members, and jump at the chance to highlight their contributions. Look for more Forum projects in the New Relic blog in the future. Do you have your own approach, tips, or recipes? Please share them in the New Relic Community Forum.

This past week, I headed to London for the Rugged DevOps event, where I had the chance to catch up with a few more industry thought leaders. First, I caught up with Gareth Rushgrove from Puppet Labs, who also runs the DevOps Weekly newsletter. In this episode of the DevOps Leadership Series, Gareth explains why the importance of security tests within DevOps practices is going to be a big topic for years to come. Gareth anticipates that “In five years time we will look back on this and ask ‘why were we not always doing this’?” I then caught up with Helen Beal, Head of DevOps at Ranger4, where she discussed the importance of DevOps and security. While the two ultimately need to work hand and hand, she voiced concerns about DevOps practices sometimes circumventing controls that are essential to a business’s safety. She also said that DevOps supports security in a number of ways: from making things consistent to relying on more automation. Finally, I caught up with Justin Arbuckle, Vice President, EMEA & Chief Enterprise Architect at Chef. He discusses key trends he expects to see in over the next 9-12 months regarding compliance and security. Justin explains that compliance as a core driver of DevOps is something that will start to shape our conversations over the next year, whereas the two have previously been seen as contradictory. He believes that we will see that high velocity organizations are able to improve compliance continuously, and tells us to look out for the changing role of the security officer in 2015. Next up in the series, I head to the United States Capitol, Washington DC, for DevOps Days DC. NOTE: If you have missed any of the other videos from this series, you can find them here. (We’re up to 15 so far).

Occasionally we are asked about JBoss BPM Suite integration with other products and layers in an enterprises architecture. We have published articles talking about how to achieve this with various aspects such as: Microservices integration Data integration Articles are one thing, but seeing is believing, so we have done a few webinars to show you live how to tackle integration: Data integration webinar PEX webinar Along with these articles we have always published demo projects that give you a closer look and chance to get hands on with these integration strategies: JBoss BPM Suite & JBoss Fuse Travel microservices story JBoss BPM Suite & JBoss Data Virtualization integration Imported Signavio Process Editor mortgage workflow. There is another integration story yet to be told about how one can leverage other tooling together with JBoss BPM Suite. This article will introduce one such company,Signavio, that provides a Signavio Process Editor so"...you can start modeling and engaging your organization in improving operational efficiency through the development of optimal models..." The following demo project provides a working example of how you can model an example mortgage process in Signavio Process Editor and then bring it into JBoss BPM Suite where you can add implementation details, integration details and other implementation details to finally execute the mortgage process end-to-end. Demo project As always we bring you not only a story, but a reusable demo project you can easily spin up yourself to explore the details around how a JBoss BPM project would integrate with the model designed in Signavio Process Editor. The project is called the JBoss BPM Suite & Signavio Process Editor Integration Demo. The project installs JBoss BPM Suite 6.1 with an example mortgage project with rules, process, forms and other artifacts. It also includes a copy of an exported Signavio Process Editor mortgage process that we then show how to import. Final mortgage workflow project with implementation details and integration details completed. Ready to run! This gives you the initial starting point after importing the Signavio process and the completely integrated final mortgage project that you can run side-by-side. To setup this project there are just a few simple steps to get going and will be up and running minutes: Installation Download and unzip. Add products to installs directory. Run 'init.sh' or 'init.bat' file. 'init.bat' must be run with Administrative privileges. Start JBoss BPMS Server by running 'standalone.sh' or 'standalone.bat' in the /target/jboss-eap-6.1/bin directory. Login to http://localhost:8080/business-central - login for admin, appraisor, broker, and manager roles (u:erics / p:bpmsuite1!) Mortgage Loan demo pre-installed as project. Using process designer, import the Signavio process that was exported to the file found in: support/MortgageDemoSignavio.bpmn Looking to Automate your business? See screenshots provided in project for how this should look and note that the JBoss BPM Suite process designer included validation that puts messages about tasks not specified, this is correct as at this point you need to start implementing the process tasks. You can examine the imported process and note the various details captured during initial workshops have been put into the process details for each step in the workflow. After implementing these steps you will find the final process ready to run. You can now explore the final project by deploying it and starting a new instance. We hope you enjoy this example project and feel free to browse for more at JBoss Demo Central.

The use of OpenStack in the private cloud is invaluable for DevOps. It provides engineers the ability to innovate quickly and deal with uncertainty. It also maximizes existing infrastructure and provides a programmable, software-defined IaC. Openstack in the private cloud = agile development OpenStack has emerged as the de facto standard for IaaS in the private cloud. It gives engineers a vital self-service capability to provision (and de-provision) environments, allowing them to act autonomously, in the moment. This helps to eliminate the downstream bottleneck caused by waiting for operations staff to find time to do the provisioning. As OpenStack is open source it is vendor agnostic, allowing you to take advantage of competitive pricing rather than suffering from vendor lock-in. A private cloud means lower cost for the same capacity in a public cloud, which is especially useful for enterprises with high data needs. For security reasons, OpenStack is still mainly used in the private cloud by developers and QA, i.e. in a non-production context. However, OpenStack gives an ability to optimize application performance and/or security by having more control compared to public cloud. The software is increasingly backed by the critical mass of leading IT infrastructure vendors such as IBM, CICSO and HP. Gartner assumes that “by 2019, OpenStack enterprise deployments will grow tenfold, up from just hundreds of production deployments today, due to increased maturity and growing ecosystem support.”1 Challenges to consider OpenStack implementation skills are still rare in the market, so experimentation and self-learning is necessary. Although this takes time, it is offset by the fact the software is free and represents a good opportunity to gain internal expertise. This is particularly valid if you class infrastructure as a core competence. The maturity and functionality of OpenStack projects vary widely - while it covers storage, network and compute, the main adoption currently happens around compute (Nova) and block storage (Cinder), with object storage and network (Neutron) lacking significantly behind. However, without leveraging virtualized network services as part of a private cloud, full-stack environment provisioning is not possible, so don’t forget to add necessary network services to your private cloud. Where to begin Integrating OpenStack clouds with existing infrastructure can be a challenge. It is hardly plug and play. At first, it is best to focus on relatively isolated DevOps environments, such as Gartner’s “mode two”2 applications rather than introducing open stack across the board straight away, (Bimodal IT “refers to having two modes of IT, each designed to develop and deliver information – and technology – intensive services in its own way. Mode 1 is traditional, emphasizing scalability, efficiency, safety and accuracy. Mode 2 is nonsequential, emphasizing agility and speed.”3) As with any open source software, new functions and upgrades are frequently released. This means keeping up with changes in functionality and filling gaps with customizations or third-party products. Upgrades are complex and typically require planned downtime. For these reasons, we recommend choosing a hardened distribution and sticking with it. Openstack is the most complete vendor agnostic solution for storage, network and compute services. The ability for developers to instantly spin up environments at any time is invaluable for a fully agile DevOps environment, and is well worth the effort it takes to acclimatize to Openstack. 1 http://www.prnewswire.com/news-releases/suse-openstack-cloud-5-to-simplify-private-cloud-management-300048721.html 2 http://www.gartner.com/it-glossary/bimodal 3 http://www.gartner.com/it-glossary/bimodal

[This article was written by Laura Strassman] AlertSite is now integrated with Ready! API. This means that developers, testers and operations teams can collaborate together on API quality using the same tests and metrics, simplifying configuration of monitoring assets and ultimately turning around performance problems in real time. There are several advantages to this approach: You should be monitoring your APIs in production. When your API moves into production from test, the environment changes – there is no way to know if theAPI performanceis compromised unless you look. Furthermore, you can find problems that may be a result of the location or variance in traffic. There is no easier method. You can simply click a button from right in the Ready! API interface and see the status of your APIs in production it can’t get any easier. You take your already created test cases and turn them into monitors whenever you have a new API or test you want to keep an eye on. Troubleshooting is like shooting fish in a barrel. You wrote the functional test, you know it works, and when something comes back as not working you can isolate it quickly. All of this makes it easy to be ahead of problems, solve them quickly when they happen and keep customers happy. You can be monitoring your APIs in less than 3 minutes: &amp;amp;lt;br&amp;amp;gt;

When running a Maven build with many plugins (e.g. the jOOQ or Flyway plugins), you may want to have a closer look under the hood to see what’s going on internally in those plugins, or in your extensions of those plugins. This may not appear obvious when you’re running Maven from the command line, e.g. via: C:\Users\jOOQ\workspace>mvn clean install Luckily, it is rather easy to debug Maven. In order to do so, just create the following batch file on Windows: @ECHO OFF IF "%1" == "off" ( SET MAVEN_OPTS= ) ELSE ( SET MAVEN_OPTS=-Xdebug -Xnoagent -Djava.compile=NONE -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=5005 ) Of course, you can do the same also on a MacOS X or Linux box, by usingexport intead of SET. Now, run the above batch file and proceed again with building: C:\Users\jOOQ\workspace>mvn_debug C:\Users\jOOQ\workspace>mvn clean install Listening for transport dt_socket at address: 5005 Your Maven build will now wait for a debugger client to connect to your JVM on port 5005 (change to any other suitable port). We’ll do that now with Eclipse. Just add a new Remote Java Application that connects on a socket, and hit “Debug”: That’s it. We can now set breakpoints and debug through our Maven process like through any other similar kind of server process. Of course, things work exactly the same way with IntelliJ or NetBeans. Once you’re done debugging your Maven process, simply call the batch again with parameter off: C:\Users\jOOQ\workspace>mvn_debug off C:\Users\jOOQ\workspace>mvn clean install And your Maven builds will no longer be debugged. Happy debugging!

Spring Integration Kafka 1.2 is out with a major performance overhaul.

Having mentored and invested in startups I have come to learn what works and doesn’t websites. The reality of getting out of startup mode and scaling takes vision and the ability to anticipate how you may need to pivot. There are two main reasons why startups do not scale. The first is a lack of experience and mentorship. This is closely followed by a lack of a working capital. An effective entrepreneur understands not only how to pivot but how to utilize capital investment. This article aims to demystify the need for investing in site infrastructure. When scaling a business there are several huge issues that executives tend to forget. whether you are a startup or a fortune 500 company your website is the organizations public face. Maintaining a clean and secure site will help to avoid deep routed problems that could potentially destroy not only you site by your reputation. Hackers can be devastating and end up costing you millions. Another aspect of your site to consider is funnel optimization. By making this a priority you will be able to effective guide customers step by step into a conversion. American With Disabilities Act (ADA) The ADA can be a freighting legal area for many entrepreneurs. Many think that ADA only applies to physical boundaries such as implementing ramps for wheelchairs and accessible bathroom stalls. However this is far from the case. Few people know that the digital world also counts. Is your website written in HTML5? If not you are in violation of ADA. This newest version of the HTML coding language allows for an audible version of a web site for those facing impaired vision. However there are some quick an easy ways to gain some ADA points to allow for more leeway in other more long-term solutions. Implementing closed captioning into your promotional videos will allow for increased accessibility for those facing an auditory disability. Investing in infrastructure is not a single step but a constant to building a good company culture where employees can feel proud of where they work and remain safe. Please comment on this article if you have any additional suggestions.

Last week’s Gartner IT Operations Strategies & Solutions Summit in Orlando, Fla., was exactly what you’d expect—a place to talk about the IT operations issues impacting some of the largest companies in the world. Even so, there were a few interesting surprises. Among them: 1. Bi-modal is big. Not everyone will succeed. Gartner continued to tell its customers to employ two modes of IT—a traditional, slower moving capability for older, typically internal systems of record; and a high-speed, experimental one for new, typically customer-facing Web and mobile apps. “This is a time of experimentation and innovation,” said Gartner VP and distinguished analyst Chris Howard in his opening keynote. Organizations can’t ignore that there are multiple speeds and they should participate in all. Gartner managing VPRonni Colville added that by 2017, 75% of IT orgs will have this “bi-modal” IT capability. See also: Bi-Modal IT: Gartner Endorses Both Disruptive and Conservative Approaches to Technology However, “50% will make a mess of it,” Colville said. Why? Not necessarily because of technology failings, but more often because of a lack of people skills. 2. IT success is all about people. Donna Scott, also a Gartner VP and distinguished analyst, told her keynote audience that “you will be judged on agility, speed, and innovation.” However, the biggest problems Gartner sees for infrastructure and operations team engagement and innovation are lack of time, company culture that’s not conducive to these approaches, and a lack of business skills in IT. More than half of the people responding to an in-room poll said “people” are the part of IT ops that must change first. Not technology. Gartner research director George Spafford underscored similar issues in large organizations trying to use DevOps at scale: people and “human factors” are the biggest concerns from his in-room poll. All these probably contributed to hiring best-selling author Daniel Pink as a keynote speaker on the opening day of the conference. His focus? Not IT or architecture. Instead, he pounded home the importance of influencing people and selling internally. 3. Big orgs are trying DevOps. But the issues are different at scale. In numerous sessions I saw many hands go up when analysts asked, “Who here is trying DevOps?” Clearly, the approach is getting traction in large companies. But there’s lots of learning still to do. In fact, that was Spafford’s biggest bit of advice. “Always be learning,” he said, “trying to see what works and what breaks, especially at scale.” And, even once you’ve had some initial success, keep learning. “If you’ve done ‪DevOps, stay humble,” he advised. 4. Looking to innovative organizations for ideas … analytics on the rise. Many sessions addressed how large organizations are taking on ideas fostered by smaller, more risk-tolerant companies, and offered advice for doing so successfully. In addition to multiple discussions of DevOps, an entire session was devoted to establishing your own “Genius Bar®—a “walk-up IT support center” as explained in this CIO article. As at previous conferences, Gartner research VP Cameron Haight ran several sessions on lessons learned from firms running massive, Web-scale IT systems. “You need lots of data … and access to it inexpensively,” he said. Some commercial monitoring companies (New Relic included!) got a shout out for taking the lessons of Web scale IT to heart in their offerings. In addition, Haight said, “Analytics are increasingly important for application performance monitoring given the huge amount of data now available.” 5. Cloud: Enterprises want it, but aren’t very good at it yet. Gartner research director Dennis Smith talked through the enterprise’s interest in cloud computing. A huge majority of his in-room poll wanted some mix of both public and private cloud, while only 9% wanted to use only a private cloud environment and a measly 4% were looking to move entirely to the public cloud. The most popular choice (41%) was an 80/20 split between private and public cloud infrastructure. “Enterprises don’t make the dean’s list,” for cloud usage, Smith said, earning no more than a C average in his opinion. Large organizations are doing well at visibility, governance, and delivering standardized stacks, he said, but are less skilled at optimizing for these new environments. Still, Smith said the trends point toward enterprises improving on all fronts. 6. Cloud security can be better than yours. Importantly, Gartner VP and distinguished analyst Neil MacDonald gave the cloud a vote of confidence: noting that, for a variety of reasons, “Well-managed public cloud can be more secure than your own data center.” For example, on-premise software can pose serious security risks, he said, because of “deployment lag” where customers are stuck using software releases with unpatched security vulnerabilities. With a cloud-based Software-as-a-Service (SaaS), security updates can be more quickly rolled out to all customers. But cloud security can be different, requiring a shift to information-level security from OS-level security. Best practices include doing away with a huge pool of all-powerful sysadmins in favor of JEA, or “just enough administration,” where sysadmins have just enough privileges to do their job, and no more. An analogous security practice for compute resources is “least privilege,” where apps and microservices can’t talk to each other unless they specifically need to do so. Audience polling supported MacDonald’s optimistic view of cloud security, which suggests that large enterprises may struggle less with their cloud policies moving forward. 7. Containers: Try ’em! Ahead of this week’s DockerCon in San Francisco, Gartner devoted significant airtime to educating the audience on containers and microservices. My summary of ‪Gartner VP and distinguished analyst Tom Bittman’s advice on containers was simple: Try ’em. Now. Complement them with VMs. ‪And Docker (the company) is important, but not the be-all and end-all in this space. Bittman (copping to some deja vu from Gartner presentations he made on server virtualization 13 years ago) noted that while virtualization has been focused on admin and ops functions, containers are focused on value for developers. But because containers are well suited for driving up VM utilization for workloads that share the same OS, we can expect to see more combinations of containers and server virtualization. Finally, Bittman underscored that Gartner doesn’t see containers having much impact on premise, but making a huge difference in the cloud. That doesn’t necessarily fit with what’s been shown in other research, such as this 2015 State of Containers Survey sponsored by VMblog.com and StackEngine, so we’ll want to watch how this plays out. This is all a lot to digest. The Gartner IT Operations Strategies & Solutions Summitacknowledges the importance of dealing with existing IT systems and practices as well as promising new technologies and thinking, and tries to point a way forward. In fact, Haight had a very good quote about microservices that I thought also served to wrap up the entire event: “If you want to run with the big dogs, you need to rethink application architecture,” he said. That can be very difficult for an enterprise to fully implement … but also very appealing. Note: Al Sargent contributed to this post. All product and company names herein may be trademarks of their registered owners. Server, tortoise and hare, business team, and cloud security images courtesy ofShutterstock.com.

I remain captivated by the ideals of Literate Programming. My fork of PyLit (https://github.com/slott56/PyLit-3) coupled with Sphinx seems to handle LP programming in a very elegant way. It works like this. Write RST files describing the problem and the solution. This includes the actual implementation code. And everything else that's relevant. Run PyLit3 to build final Python code from the RST documentation. This should include the setup.py so that it can be installed properly. Run Sphinx to build pretty HTML pages (and LaTeX) from the RST documentation. I often include the unit tests along with the sphinx build so that I'm sure that things are working. The challenge is final presentation of the whole package. The HTML can be easy to publish, but it can't (trivially) be used to recover the code. We have to upload two separate and distinct things. (We could use BeautifulSoup to recover RST from HTML and then PyLit to rebuild the code. But that sounds crazy.) The RST is easy to publish, but hard to read and it requires a pass with PyLit to emit the code and then another pass with Sphinx to produce the HTML. A single upload doesn't work well. If we publish only the Python code we've defeated the point of literate programming. Even if we focus on the Python, we need to do a separate upload of HTML to providing the supporting documentation. After working with this for a while, I've found that it's simplest to have one source and several targets. I use RST ⇒ (.py, .html, .tex). This encourages me to write documentation first. I often fail, and have blocks of code with tiny summaries and non-existent explanations. PyLit allows one to use .py ⇒ .rst ⇒ .html, .tex. I've messed with this a bit and don't like it as much. Code first leaves the documentation as a kind of afterthought. How can we publish simply and cleanly: without separate uploads? Enter GitHub and gh-pages. See the "sphinxdoc-test" project for an example. Also thishttps://github.com/daler/sphinxdoc-test. The bulk of this is useful advice on a way to create the gh-pages branch from your RST source via Sphinx and some GitHub commands. Following this line of thinking, we almost have the case for three branches in a LP project. The "master" branch with the RST source. And nothing more. The "code" branch with the generated Python code created by PyLit. The "gh-pages" branch with the generated HTML created by Sphinx. I think I like this. We need three top-level directories. One has RST source. A build script would run PyLit to populate the (separate) directory for the code branch. The build script would also run Sphinx to populate a third top-level directory for the gh-pages branch. The downside of this shows up when you need to create a branch for a separate effort. You have a "some-major-change" branch to master. Where's the code? Where's the doco? You don't want to commit either of those derived work products until you merge the "some-major-change" back into master. GitHub Literate Programming There are many LP projects on GitHub. There are perhaps a dozen which focus on publishing with the Github-flavored Markdown as the source language. Because Markdown is about as easy to parse as RST, the tooling is simple. Because Markdown lacks semantic richness, I'm not switching. I've found that semantically rich markup is essential. This is a key feature of RST. It's carried forward by Sphinx to create very sophisticated markup. Think:code:`sample` vs. :py:func:`sample` vs. :py:mod:`sample` vs.:py:exc:`sample`. The final typesetting may be similar, but they are clearly semantically distinct and create separate index entries. A focus on Markdown seems to be a limitation. It's encouraging to see folks experiment with literate programming using Markdown and GitHub. Perhaps other folks will look at more sophisticated markup languages like RST. Previous Exercises See https://sourceforge.net/projects/stingrayreader/ for a seriously large literate programming effort. The HTML is also hosted at SourceForge: http://stingrayreader.sourceforge.net/index.html. This project is awkward because -- well -- I have to do a separate FTP upload of the finished pages after a change. It's done with a script, not a simple "git push." SourceForge has a GitHub repository. https://sourceforge.net/p/stingrayreader/code/ci/master/tree/. But. SourceForge doesn't use GitHub.com's UI, so it's not clear if it supports the gh-pages feature. I assume it doesn't, but, maybe it does. (I can't even login to SourceForge with Safari... I should really stop using SourceForge and switch to GitHub.) See https://github.com/slott56/HamCalc-2.1 for another complex, LP effort. This predates my dim understanding of the gh-pages branch, so it's got HTML (in doc/build/html), but it doesn't show it elegantly. I'm still not sure this three-branch Literate Programming approach is sensible. My first step should probably be to rearrange the PyLit3 project into this three-branch structure.