Many organizations in which I've coached are concerned when the people on their teams aren't being "utilized" at 100% of their capacity. If someone is at work for 8 hours per day, minus an hour for lunch and breaks, that's 7 hours of potential capacity. Some organizations are progressive enough to see that the organization's overhead of administrative activities lowers that value to 6 hours per day. By extension, a team's capacity is simply a multiple of the number of team members and the number of hours available to be utilized, i.e. a team of 5 has 30 person-hours per day. So, anything less than 30 person-hours per day spent on tasks means that a team isn't being utilized 100%. Which is bad, apparently. A Thought Exercise If you're reading this, you have a computer and it's connected to a network. Your computer might be a small one that makes phone calls or a large one that can also handle the complex scenery of games at 60 frames per second without breaking a sweat. Regardless, it's a computer. Thanks to John von Neumann, almost all computers today have the same basic architecture, the heart of which is the Central Processing Unit or CPU. When the CPU on your computer, regardless of its size, reaches 100% utilization, what happens? Your computer slows down. Significantly. The CPU is saturated with instructions and does its best to process them, but it has a finite limit on how quickly it can do that. There's a finite limit to the speed at which the instructions and the data they use can be passed around among the internal components of the CPU. That processor just can't go any faster. To a person using the computer, it appears to be locked up. To see this in action, try running a modern game like Faster Than Light on a 1st generation iPad. The older processor can't keep up with the computing demand from a game targeted towards more a powerful CPU. The communications network to which your computer is connected might be wireless or have a "hard" connection, but behind the scenes the public internet is dominated by TCP/IP and Ethernet. What happens when the network to which your computer is connected is running at 100% capacity? The network slows down. Significantly. The equipment moving the packets of data about will experience numerous collisions and will have to send requests back to your computer to resend data. The equipment will also begin to simply "drop" packets because it can't process them quickly enough. As far as you're concerned the network has become either very slow or has simply stopped. To see this in real life, look at what happens to a web site when it's the subject of a Denial of Service (DoS) attack, which effectively saturates that site's network with data requests. The site either crashes completely or appears to be unresponsive because it can't handle the traffic. Knowledge work such as software development requires a huge amount of thinking (processing) and a huge amount of collaboration (communication). The human brain is a processor, and like the von Neumann architecture has subcomponents (lobes), working memory, etc. What happens when our processor hits 100% utilization? We simply can't process all the inputs fast enough and our ability to process information slows down dramatically. We even reach a state where the decision-making centre in our brain shuts itself down. Coupled with a decreasing ability to process information and make decisions, we experience increasing levels of anxiety. Now, let's look at teams. They represent a communications network, with the people being the nodes in the network. When the team reaches 100% utilization, the individual processors (the brains of the people!) begin to slow as the ability to process information diminishes and anxiety increases. This has the effect of slowing the team's ability to communicate and operate effectively. When one or multiple people reach the point of shutting down, the network collapses. Why does anyone think that 100% utilization is a good thing? In manufacturing, 100% utilization of the capacity of your workers is actually desirable. The thinking aspects have already been done in the design and engineering of the item being created, and the assembly process is infinitely repeatable until a new item comes along. This approach to the process was taken from the manufacturing world and applied to the development of software, with the thinking that assembly was done by the programmers after all the design and engineering had taken place. Well, that simply isn't true. The "assembly" of software isn't programming, it's the compilation and packing into something deployable. Writing software is a confluence of design and engineering that has creative and technical aspects. It's not an assembly line, and probably won't ever be within my lifetime. As long ago as the late 1950's, management guru Peter Drucker realized that there were people who "think for a living". He coined the term knowledge worker to describe that type of work. Developing software is a classic knowledge worker role and therefore has different rules for productivity. As Drucker said in his 1999 book Management Challenges of the 21st Century, Productivity of the knowledge worker is not — at least not primarily — a matter of the quantity of output. Quality is at least as important. If we're pushing individuals and teams to 100% capacity, the quality of work and therefore the productivity of the team will be reduced. In 2001, Tom deMarco released Slack: Getting Past Burnout, Busywork and the Myth of Total Efficiency, an entire book describing the problems created by trying to achieve 100% utilization. I highly recommend the book, and also recommend you give your manager a copy. :) Here are some selected quotes that are pertinent to reducing the load on our processor and communication network: Very successful companies have never struck me as particularly busy; in fact, they are, as a group, rather laid back. Energy is evident in the workplace, but it's not the energy tinged with fear that comes from being slightly behind on everything. The principal resource needed for invention is slack. When companies can't invent, it's usually because their people are too damn busy. People under time pressure don't think faster. - Tim Lister How Can We Deal With This? First and foremost, stop trying to do so damned much! This is truly a slow down to go faster type of solution. For a Team If you're an agile team using iterations or sprints, pull less work into a sprint even if your velocity says you should be able to pull more. The extra time can be used to increase the quality of the work that you do complete, which is in itself a motivating factor for knowledge workers. Increased quality means decreased rework, which means the team as a whole delivers faster over the long term. Similarly, the reduced anxiety and stress on the team members increases their ability to think about what they're doing, meaning better and more innovative solutions to problems. Also, ensure that teams are focusing on activities that contribute directly to their work. Meetings tend to be the worst offenders of this, including the meetings defined within agile processes. Some suggestions: No meeting can be scheduled unless there is a specific question to be answered or specific information to be passed along. Reduce the default meeting length from 1 hour to 30 minutes in whatever calendar tool you use. Even better, reduce it to 20 minutes. Choose blocks of time that are deemed meeting free zones. No one, regardless of their position in the org chart can schedule a meeting with the team or the team members during that time. Minimize meetings where people are on a phone. My observation is that people who call into meetings tend to speak longer than when they are in the room with everyone else. I know that I do this, and I see it often in others. For an Individual Learn to say, "No." At the very least, learn to say, "Not yet." It's really difficult to do this, and I speak from personal experience. We want to help others, we want to be team players. However, if we don't say no, we take on too much, our processing and communication ability suffers and we end up disappointing those we were trying to help! Also, learn to take breaks. Yes, take breaks. The same concepts from Slack apply here, with brief diversions clearing our mind and allowing us to work better afterwards. At the extreme, if you feel like you can't keep your eyes open, take a nap! I highly recommend the Pomodoro Technique as at least a starting point for giving yourself some slack. Step away from your work for just a few minutes and return recharged. Conclusion We don't want the CPU in our computer to be working at 100% and we don't want the communications network to which it's connected to be at 100% capacity either. Our brains are processors and teams are a communications network, so why would we want those to be 100% busy all the time? In fact, ensuring that teams and the people on them are always busy is a provably wrong approach for software development. It has it's roots in a manufacturing mindset and developing software is knowledge work. Software development requires substantially different ways to make teams and the individual people on them as productive as possible. Finally, there is no magic number for what percentage of utilization is best. People are variable and 90% utilization for a person might be fine on a given day while %30 is all that the same person can handle on the next. Don't aim for a number, aim for an environment where people know that they don't have to appear busy. That will leave plenty of capacity for each person's processor and their team's communications network to run smoothly and deliver real value more effectively.

These days a lot of organizations talk about Multi-Speed IT, so I thought I’d share my thoughts on this. I think the concept has been around for a while but now there is a nice label to associate this idea with. Let’s start by looking at why Multi-Speed IT is important. The idea is best illustrated by a picture of two interlocking gears of different sizes and by using a simple example to explain the concept. Different Speeds for Different Needs One easy way to recall what multi-speed IT looks like is to remember that there are multiple speeds for multiple needs. This is to say that there are different IT programs that may be most useful at various speeds. Some departments and applications need to move very rapidly, but others can move at a slower pace that works best for them. Regardless of which department you are focused on at the moment, it is important to know that it will have specialized needs that you need to look after, and that is why so many people are now looking at multi-speed IT as the best way to accomplish what they set out to accomplish. The smaller gear moves much faster than the larger one, but where the two gears interlock they remain aligned to not stop the motion. But what does this mean in reality? Think about a banking app on your mobile. Your bank might update the app on a weekly basis with new functionality like reporting and/or an improved user interface. That is a reasonable fast release cycle. The mainframe system that sits in the background and provides the mobile app with your account balance and transaction details does not have to change at the same speed. In fact, it might only have to provide a new service for the mobile app once every quarter. Nonetheless, the changes between those two systems need to align when new functionality is rolled out. However, it doesn’t mean both systems need to release at the same speed. In general, the customer-facing systems are the fast applications (Systems of Engagement, Digital) and the slower ones are the Systems of Record or backend systems. The release cycles should take this into consideration. So how do you get ready for the Multi-Speed IT Delivery Model? Release Strategy (Agile) – Identify functionality that requires changes in multiple systems and ones that can be done in isolation. If you follow an Agile approach, you can align every n-th release for releasing functionality that is aligned while the releases in between can deliver isolated changes for the fast-moving applications. Application Architecture – Use versioned interface agreements so that you can decouple the gears (read applications) temporarily. This means you can release a new version of a backend system or a front-end system without impacting the current functionality of the other. Once the other system catches up, new functionality becomes available across the system. This allows you to keep to your individual release schedule, which in turn means delivery is a lot less complex and interdependent. In the picture I used above, think of this as the clutch that temporarily disengages the gears. Technical Practices and Tools (DevOps) – If the application architecture decoupling is the clutch, then the technical practices and tools are the grease. This is where DevOps comes into the picture. The whole idea of Multi-Speed IT is to make the delivery of functionality less interdependent. On the flip side, you need to spend more effort on getting the right practices and tools in place to support this. For example, you want to make sure that you can quickly test the different interface versions with automated testing, you need to have good version control to make sure you have in place the right components for each application, and you also want to make sure you can manage your code line very well through abstractions and branching where required. And the basics of configuration management, packaging, and deployment will become even more important as you want to reduce the number of variables you have to deal with in your environments. You better remove those variables introduced through manual steps by having these processes completely automated. Testing strategies – Given that you are now dealing with multiple versions of components being in the environment at the same time, you have to rethink your testing strategies. The rules of combinatorics make it very clear that it only takes a few different variables before it becomes unmanageable to test all permutations. So we need to think about different testing strategies that focus on valid permutations and risk profiles. After all, functionality that is not yet live requires less testing than the ones that will go live next. The above points cover the technical aspects but to get there you will also have to solve some of the organizational challenges. Let me just highlight 3 of them here: Partnership with delivery partners – It will be important to choose your partners wisely. Perhaps it helps to think of your partner ecosystem in three categories: Innovators (the ones who work with you in innovative spaces and with new technologies), Workhorses(the ones who support your core business applications that continue to change) and Commodities (the ones who run legacy applications that don’t require much new functionality and attention). It should be clear that you need to treat them differently in regards to contracts and incentives. I will blog later about the best way to incentivize your workhorses, the area that I see most challenges in. Application Portfolio Management - Of course, to find the right partner you first need to understand what your needs are. Look across your application portfolio and determine where your applications sit across the following dimensions: Importance to business, exposure to customers, frequency of change, and volume of change. Based on this you can find the right partner to optimize the outcome for each application. Governance – Last but not least, governance is very important. In a multi-speed IT world you will need flexible governance. One size fits all will not be good enough. You will need lightweight system-driven governance for your high-speed applications and you can probably afford a more PowerPoint/Excel-driven manual governance for your slower-changing applications. If you can run status reports of live systems (like Jira, RTC, or TFS) for your fast applications you are another step closer to mastering the multi-speed IT world.

Cracking the coding interview is the holy grail of many programmers and software developers, but is cracking the coding interview really possible?

If you go to Google and search for "measuring software developer productivity" you will find a whole lot of nothing. Seriously -- nothing. Nick Hodges, Measuring Developer Productivity By now we should all know that we don’t know how to measure programmer productivity. There is no clear cut way to measure which programmers are doing a better or faster job, or to compare productivity across teams. We “know” who the stars on a team are, who we can depend on to deliver, and who is struggling. And we know if a team is kicking ass – or dragging their asses. But how do we prove it? How can we quantify it? All sorts of stupid and evil things can happen when you try to measure programmer productivity. But let’s do it anyways. We’re Writing More Code, So We Must Be More Productive Developers are paid to write code. So why not measure how much code they write – how many lines of code get delivered? Because we've known since the 1980s that this is a lousy way to measure productivity. Lines of code can’t be compared across languages (of course), or even between programmers using the same language working in different frameworks or following different styles. Which is why Function Points were invented – an attempt to standardize and compare the size of work in different environments. Sounds good, but Function Points haven’t made it into the mainstream, and probably never will – very few people know how Function Points work, how to calculate them and how they should be used. The more fundamental problem is that measuring productivity by lines (or Function Points or other derivatives) typed doesn’t make any sense. A lot of important work in software development, the most important work, involves thinking and learning – not typing. The best programmers spend a lot of time understanding and solving hard problems, or helping other people understand and solve hard problems, instead of typing. They find ways to simplify code and eliminate duplication. And a lot of the code that they do write won’t count anyways, as they iterate through experiments and build prototypes and throw all of it away in order to get to an optimal solution. The flaws in these measures are obvious if we consider the ideal outcomes: the fewest lines of code possible in order to solve a problem, and the creation of simplified, common processes and customer interactions that reduce complexity in IT systems. Our most productive people are those that find ingenious ways to avoid writing any code at all. Jez Humble, The Lean Enterprise This is clearly one of those cases where size doesn’t matter. We’re Making (or Saving) More Money, so We Must Be Working Better We could try to measure productivity at a high level using profitability or financial return on what each team is delivering, or some other business measure such as how many customers are using the system – if developers are making more money for the business (or saving more money), they must be doing something right. Using financial measures seems like a good idea at the executive level, especially now that “every company is a software company”. These are organizational measures that developers should share in. But they are not effective – or fair – measures of developer productivity. There are too many business factors are outside of the development team’s control. Some products or services succeed even if the people delivering them are doing a lousy job, or fail even if the team did a great job. Focusing on cost savings in particular leads many managers to cut people and try “to do more with less” instead of investing in real productivity improvements. And as Martin Fowler points out there is a time lag, especially in large organizations – it can sometimes take months or years to see real financial results from an IT project, or from productivity improvements. We need to look somewhere else to find meaningful productivity metrics. We’re Going Faster, so We Must Be Getting More Productive Measuring speed of development – velocity in Agile – looks like another way to measure productivity at the team level. After all, the point of software development is to deliver working software. The faster that a team delivers, the better. But velocity (how much work, measured in story points or feature points or ideal days, that the team delivers in a period of time) is really a measure of predictability, not productivity. Velocity is intended to be used by a team to measure how much work they can take on, to calibrate their estimates and plan their work forward. Once a team’s velocity has stabilized, you can measure changes in velocity within the team as a relative measure of productivity. If the team’s velocity is decelerating, it could be an indicator of problems in the team or the project or the system. Or you can use velocity to measure the impact of process improvements, to see if training or new tools or new practices actually make the team’s work measurably faster. But you will have to account for changes in the team, as people join or leave. And you will have to remember that velocity is a measure that only makes sense within a team – that you can’t compare velocity between teams. Although this doesn't stop people from trying. Some shops use the idea of a well-known reference story that all teams in a program understand and use to base their story points estimates on. As long as teams aren't given much freedom on how they come up with estimates, and as long as the teams are working in the same project or program with the same constraints and assumptions, you might be able to do rough comparison of velocity between teams. But Mike Cohn warns that If teams feel the slightest indication that velocities will be compared between teams there will be gradual but consistent “point inflation.” ThoughtWorks explains that velocity <> productivity in their latest Technology Radar: We continue to see teams and organizations equating velocity with productivity. When properly used, velocity allows the incorporation of “yesterday's weather” into a team’s internal iteration planning process. The key here is that velocity is an internal measure for a team, it is just a capacity estimate for that given team at that given time. Organizations and managers who equate internal velocity with external productivity start to set targets for velocity, forgetting that what actually matters is working software in production. Treating velocity as productivity leads to unproductive team behaviors that optimize this metric at the expense of actual working software. Next: Just Stay Busy, Measure Outcomes, not Output; and more... Just Stay Busy One manager I know says that instead of trying to measure productivity “We just stay busy. If we’re busy working away like maniacs, we can look out for problems and bottlenecks and fix them and keep going”. In this case you would measure – and optimize for – cycle time, like in Lean manufacturing. Cycle time – turnaround time or change lead time, from when the business asks for something to when they get it in their hands and see it working – is something that the business cares about, and something that everyone can see and measure. And once you start looking closely, waste and delays will show up as you measure waiting/idle time, value-add vs. non-value-add work, and process cycle efficiency (total value-add time / total cycle time). “It’s not important to define productivity, or to measure it. It’s much more important to identify non-productive activities and drive them down to zero.” Erik Simmons, Intel Teams can use Kanban to monitor – and limit – work in progress and identify delays and bottlenecks. And Value Stream Mapping to understand the steps, queues, delays and information flows which need to be optimized. To be effective, you have to look at the end-to-end process from when requests are first made to when they are delivered and running, and optimize all along the path, not just the work in development. This may mean changing how the business prioritizes, how decisions are made and who makes the decisions. In almost every case we have seen, making one process block more efficient will have a minimal effect on the overall value stream. Since rework and wait times are some of the biggest contributors to overall delivery time, adopting “agile” processes within a single function (such as development) generally has little impact on the overall value stream, and hence on customer outcomes. Jezz Humble, The Lean Enterprise The down side of equating delivery speed with productivity? Optimizing for cycle time/speed of delivery by itself could lead to problems over the long term, because this incents people to think short term, and to cut corners and take on technical debt. We’re Writing Better Software, so We Must Be More Productive “The paradox is that when managers focus on productivity, long-term improvements are rarely made. On the other hand, when managers focus on quality, productivity improves continuously.” John Seddon, quoted in The Lean Enterprise We know that fixing bugs later costs more. Whether it’s 10x or 100+x, it doesn't really matter. And that projects with fewer bugs are delivered faster – at least up to a point of diminishing returns for safety-critical and life-critical systems. And we know that the costs of bugs and mistakes in software to the business can be significant. Not just development rework costs and maintenance and support costs. But direct costs to the business. Downtime. Security breaches. Lost IP. Lost customers. Fines. Lawsuits. Business failure. It’s easy to measure that you are writing good – or bad – software. Defect density. Defect escape rates (especially defects – including security vulnerabilities – that escape to production). Static analysis metrics on the code base, using tools like SonarQube. And we know how to write good software - or we should know by now. But is software quality enough to define productivity? Devops – Measuring and Improving IT Performance Devops teams who build/maintain and operate/support systems extend productivity from dev into ops. They measure productivity across two dimensions that we have already looked at: speed of delivery, and quality. But devops isn't limited to just building and delivering code – instead it looks at performance metrics for end-to-end IT service delivery: Delivery Throughput: deployment frequency and lead time, maximizing the flow of work into production Service Quality: change failure rate and MTTR It’s not a matter of just delivering software faster or better. It’s dev and ops working together to deliver services better and faster, striking a balance between moving too fast or trying to do too much at a time, and excessive bureaucracy and over-caution resulting in waste and delays. Dev and ops need to share responsibility and accountability for the outcome, and for measuring and improving productivity and quality. As I pointed out in an earlier post this makes operational metrics more important than developer metrics. According to recent studies, success in achieving these goals lead to improvements in business success: not just productivity, but market share and profitability. Measure Outcomes, not Output In The Lean Enterprise (which you can tell I just finished reading), Jez Jumble talks about the importance of measuring productivity by outcome – measuring things that matter to the organization – not output. “It doesn't matter how many stories we complete if we don’t achieve the business outcomes we set out to achieve in the form of program-level target conditions”. Stop trying to measure individual developer productivity. It’s a waste of time. Everyone knows who the top performers are. Point them in the right direction, and keep them happy. Everyone knows the people who are struggling. Get them the help that they need to succeed. Everyone knows who doesn't fit in. Move them out. Measuring and improving productivity at the team or (better) organization level will give you much more meaningful returns. When it comes to productivity: Measure things that matter – things that will make a difference to the team or to the organization. Measures that are clear, important, and that aren't easy to game. Use metrics for good, not for evil – to drive learning and improvement, not to compare output between teams or to rank people. I can see why measuring productivity is so seductive. If we could do it we could assess software much more easily and objectively than we can now. But false measures only make things worse. Martin Fowler, CannotMeasureProductivity

Over the past year, Datical has had amazing success with our flagship product, Datical DB. We’ve seen multiple visionary, sector-leading companies select Datical DB to drive their Application Schema changes. Now that the number has grown rapidly over the past year, we can begin to see patterns in why customers choose Datical DB. One of them turns out to be pretty emblematic of our other customers. So, let’s examine the reasons why they chose to adopt Datical DB. Customer Facing Applications are the Front Door When your competitor is only a mobile screen swipe away, this Datical customer focuses on brand reputation and customer satisfaction. They know that application uptime and fast delivery are key to customer retention and account expansion. All applications have a database backend. Though, when something goes wrong with the database, it is not apparent to the consumer. The consumer just knows that the app isn’t working. An unresponsive mobile app or website is often enough to make the customer take their business elsewhere. Customer stickiness due to the hassle of changing providers continues to lessen as the cost to change continues to drop. Therefore, immediate and fast access is a must have for today’s companies. Remember: Consumer facing applications don’t have business hours. They must ALWAYS be open. Cross Team Collaboration This Datical customer had difficulty in determining who made what change to the database. Moreover, answering which database and why was near impossible. All of the changes were stored in a single document. That solution is not multi-tenant, meaning that the team had to distribute the document to share information. Moreover, there was a single point of failure in the tracking process. Too often, people were required to visit the datacenter for days at a time during application changes. There was simply no method to notify team members of changes, when they occurred, and the change impact. This lack of communication led to almost 80% of all database change requests being rejected. There was clearly a need to increase communication of changes and increase the requested changes’ quality. Increase Staff Productivity With over 70% of the Database team’s time spent on managing change due to application requirements, the Database team was taxed in meeting other demands. Needs to improve scalability and reliability of the database servers became a lower priority as the DBAs struggled to keep up with change requests. Furthermore, development teams spent almost 10% of their time managing these change requests, including reworks of failed requests. By eliminating a manual, time-consuming process, this customer can now focus resources on addressing other needs such as managing continuity during server failure, better allocation of server resources, and certainly scalability concerns. Go Faster With the adoption of IBM UrbanCode Deploy, this customer quickly streamlined their deployments with all but the database automated. With all other components automated, the database changes required by application changes was clearly the weak link in the deployment chain. Truly, database automation is the last mile necessary to realize the promise of Agile, Continuous Delivery, and DevOps. Until the customer was able to apply Agile and Continuous Delivery to database changes, the entire application stack was, in effect, still using out-of-date development and deployment methods. To see the complete benefits of Agile and Continuous Delivery, automation throughout the entire stack, including the database, was absolutely necessary. Leverage Existing Infrastructure and Processes With out of the box integration with UrbanCode Deploy, Datical DB did not require an independent separate server. Furthermore, with Datical DB’s ability to utilize the customer’s existing source code control repository, the implementation cycle was shortened significantly. Too often, customers are asked by vendors to spend money on more server resources or make strange, unnatural changes to their network security to support potential solutions. By utilizing existing infrastructure and processes, Datical DB was able to deliver to this customer lightening quick ROI. We measure ROI in days and weeks, not months and years. Please join us for a webcast next Wednesday, February 4th, from 12:00 – 1:00 pm EST, as we discuss these customer benefits in detail and show how Datical DB integrates seamlessly with IBM UrbanCode Deploy.

One of the goals of JBoss AS7 was to make it much more secure by default, when compared to previous versions. One of the areas which was directly impacted by this goal was that you could no longer expect the server to expose some service on a port and get access to it without any authentication/authorization. Remember that in previous versions of JBoss AS you could access the JNDI port, the JMX port without any authentication/authorization, as long as those ports were opened for communication remotely. Finer grained authorizations on such ports for communications, in JBoss AS7, allows the server to control who gets to invoke operations over that port. Of course, this is not just limited to JBoss AS7 but continues to be the goal in WildFly (which is the rename of JBoss Application Server). In fact, WildFly has gone one step further and now has the feature of "one single port" for all communication. JMX communication in JBoss AS7 and WildFly With that background, we'll now focus on JMX communication in JBoss AS7 and WildFly. I'll use WildFly (8.2.0 Final) as a reference for the rest of this article, but the same details apply (with minor changes) to other major versions of JBoss AS7 and WildFly, that have been released till date. WildFly server is composed of "subsystems", each of which expose a particular set of functionality. For example, there's the EE subsystem which supports the Java EE feature set. Then there's the Undertow subsystem which supports web/HTTP server functionality. Similarly, there's a JMX subsystem which exposes the JMX feature set on the server. As you all are aware, I'm sure, JMX service is standardly used for monitoring and even managing Java servers and this includes managing the servers remotely. The JMX subsystem in WildFly allows remote access to the JMX service and port 9990 is what is used for that remote JMX communication. JConsole for remote JMX access against JBoss AS7 and WildFly Java (JDK) comes bundled with the JConsole tool which allows connecting to local or remote Java runtimes which expose the JMX service. The tool is easy to use, all you have to do is run the jconsole command it will show up a graphical menu listing any local Java processes and also an option to specify a remote URL to connect to a remote process: # Start the JConsole $JAVA_HOME/bin/jconsole Let's assume that you have started WildFly standalone server, locally. Now when you start the jconsole, you'll notice that the WildFly Java process is listed in the local running processes to which you can connect to. When you select the WildFly Java instance, you'll be auto connected to it and you'll notice MBeans that are exposed by the server. However, in the context of this article, this "local process" mode in JConsole isn't what we are interested in. Let's use the "Remote process" option in that JConsole menu which allows you to specify the remote URL to connect to the Java runtime and username and password to use to connect to that instance. Even though our WildFly server is running locally, we can use this "Remote process" option to try and connect to it. So let's try it out. Before that though, let's consider a the following few points: Remember that the JMX subsystem in WildFly allows remote access on port 9990 For remote access to JMX, the URL is of the format - service:jmx:[vendor-specific-protocol]://[host]:[port]. The vendor specific protocol is the interesting bit here. In the case of WildFly that vendor-specific-protocol is http-remoting-jmx. Remember that WildFly is secure by default which means that just because the JMX subsystem exposes 9990 port for remote communication, it doesn't mean it's open for communication to anyone. In order to be allowed to communicate over this port, the caller client is expected to be authenticated and authorized. This is backed by the "ManagementRealm" in WildFly. Users authenticated and authorized against this realm are allowed access to that port. Keeping those points in mind, let's first create a user in the Management Realm. This can be done using the add-user command line script (which is present in JBOSS_HOME/bin folder). I won't go into the details of that since there's enough documentation for that. Let's just assume that I created a user named "wflyadmin" with an appropriate password in the Management Realm. To verify that the user has been properly created, in the right realm, let's access the WildFly admin console at the URL http://localhost:9990/console. You'll be asked for username and password for access. Use the same username and password of the newly created user. If the login works, then you are good. If not, then make sure you have done things right while adding the new user (as I said I won't go into the details of adding a new user since it's going to just stretch this article unnecessarily long). So at this point we have created a user named "wflyadmin" belonging to ManagementRealm. We'll be using this same user account for accessing the JMX service on WildFly, through JConsole. So let's now bring up the jconsole as usual: $JAVA_HOME/bin/jconsole On the JConsole menu let's again select the "Remote process" option and use the following URL in the URL text box: service:jmx:http-remoting-jmx://localhost:9990 Note: For JBoss AS 7.x and JBoss EAP 6.x, the vendor specific protocol is remoting-jmx and the port for communication is 9999. So the URL will be service:jmx:remoting-jmx://localhost:9999 In the username and password textboxes, use the same user/pass that you newly created. Finally, click on Connect. What do you see? It doesn't work! The connection fails. So what went wrong? Why isn't the JConsole remote access to WildFly not working? You did all the obvious things necessary to access the WildFly JMX service remotely but you keep seeing that JConsole can't connect to it. What could be the reason? Remember, in one of those points earlier, I noted that the "vendor specific protocol" is an interesting bit? We use http-remoting-jmx and that protocol internally relies on certain WildFly/JBoss specific libraries, primarily for remote communication and authentication and authorization. These libraries are WildFly server specific and hence aren't part of the standard Java runtime environment. When you start jconsole, it uses a standard classpath which just has the relevant libraries that are part of the JDK/JRE. To solve this problem, what you need to do is bring in the WildFly server specific libraries into the classpath of JConsole. Before looking into how to do that, let's see which are the WildFly specific libraries that are needed. All the necessary classes for this to work are part of the jboss-cli-client.jar which is present in JBOSS_HOME/bin/client/ folder. So all we need to do in include this jar in the classpath of the jconsole tool. To do that we use the -J option of jconsole tool which allows passing parameters to the Java runtime of jconsole. The command to do that is: $JAVA_HOME/bin/jconsole -J-Djava.class.path=$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/jconsole.jar:/opt/wildfly-8.2.0.Final/bin/client/jboss-cli-client.jar (Note that for Windows the classpath separator is the semi-colon character instead of the colon) Note, the server specific jar for JBoss AS 7.x and JBoss EAP 6.x is named jboss-client.jar and is present at the same JBOSS_HOME/bin/client directory location. So we are passing -Djava.class.path as the parameter to the jconsole Java runtime, using the -J option. Notice that we have specified more than just our server specific jar in that classpath. That's because, using the -Djava.class.path is expected to contain the complete classpath. We are including the jars from the Java JDK lib folder that are necessary for JConsole and also our server specific jar in that classpath. Running that command should bring up JConsole as usual and let's go ahead and select the "Remote process" option and specify the same URL as before: service:jmx:http-remoting-jmx://localhost:9990 and the same username and password as before and click Connect. This time you should be able to connect and should start seeing the MBeans and others services exposed over JMX. How about providing a script which does this necessary classpath setup? Since it's a common thing to try and use JConsole for remote access against WildFly, it's reasonable to expect to have a script which sets up the classpath (as above) and you could then just use that script. That's why WildFly ships such a script. It's in the JBOSS_HOME/bin folder and is called jconsole.sh (and jconsole.bat for Windows). This is just a wrapper script which internally invokes the jconsole tool present in Java JDK, after setting up the classpath appropriately. All you have to do is run: $JBOSS_HOME/bin/jconsole.sh What about using JConsole from a really remote machine, against WildFly? So far we were using the jconsole tool that was present on the same machine as the WildFly instance, which meant that we have filesystem access to the WildFly server specific jars present in the WildFly installation directory on the filesystem. This allowed us to setup the classpath for jconsole to point to the jar on the local filesystem? What if you wanted to run jconsole from a remote machine against a WildFly server which is installed and running on a different machine. In that case, your remote client machine won't be having filesystem access to the WildFly installation directory. So to get jconsole running in such a scenario, you will have to copy over the JBOSS_HOME/bin/jboss-cli-client.jar to your remote client machine, to a directory of your choice and then setup the classpath for jconsole tool as explained earlier and point it to that jar location. That should get you access to JMX services of WildFly from jconsole on a remote machine. More questions? If you still have problems getting this to work or have other questions, please start a discussion in the JBoss community forums here https://developer.jboss.org/en/wildfly/content.

Java Mission Control (JMC) enables you to monitor and manage Java applications without introducing the performance overhead normally associated with these types of tools. It uses data which is already getting collected for normal dynamic optimization of the JVM resulting in a very lightweight approach to observe and analyze problems in the application code. The JMC consists of three different types of tools. A JMX browser which let's you browse all available JVM instances on a machine and a JMX console which let's you browse through the JMX tree on a connected JVM. Last but not least the most interesting aspect is the Java Flight Recorder (JFR). This is exactly the part of the tooling which does the low overhead profiling of JVM instances. Disclaimer: A Word On Licensing The tooling is part of the Oracle JDK downloads. In particular the JMC 5.4 is part of JDK 8u20 and JDK 7u71 and is distributed under the Oracle Binary Code License Agreement for Java SE Platform products and commercially available features for Java SE Advanced and Java SE Suite. IANAL, but as far as I know this allows for using it for your personal education and potentially also as part of your developer tests. Make sure to check back with whomever you know that could answer this question. This blog post looks at it as a small little how-to and assumes, that you know what you are doing from a license perspective. Adding Java Optional Parameters Unlocking the JFR features requires you to put in some optional parameters to your WildFly 8.x/EAP 6.x configuration. Find the $JBOSS_HOME/bin/standalone.conf|conf.bat and add the following parameters: -XX:+UnlockCommercialFeatures -XX:+FlightRecorder You can now use jcmd command like described in this knowledge-base entry to start a recording. Another way is actually to start a recording directly from JMC. Starting A Recording From JMC First step is to start JMC. Find it in the %JAVA_HOME%/bin folder. After it started you can use the JVM Browser to find the WildFly/EAP instance you want to connect to. Right click on it to see all the available options. You can either start the JMX Console or start a Flight Recording. The JMX console is a bit fancier than the JConsole and allows for a bunch of metrics and statistics. It also allows you to set a bunch of triggers and browser MBeans and whatnot. Please look at the documentation for all the details. What is really interesting is the function to start a Flight Recording. If you select this option, a new wizard pops up and lets you tweak the settings a bit. Beside having to select a folder where the recording gets stored you also have the choice between different recording templates. A one minute recording with the "Server Profiling" template with barely any load on the server results in a 1.5 MB file. So, better keep an eye on the volume you're storing all that stuff at. You can also decide the profiling granularity for a bunch of parameters further down the dialogues. But at the end, you click "Finish" and the recording session starts. You can decide to push it to the background and keep working while the data gets captured. Analyzing Flight Recorder Files This is pretty easy. You can open the recording with JMC and click through the results. If you enabled the default recording with the additional parameter: -XX:FlightRecorderOptions=defaultrecording=true you can also directly dump the recording via the JVM browser. It is easy to pick a time-frame that you want to download the data for or alternatively you can also decide to download the complete recording.

serenity bdd (previously known as thucydides ) is an open source reporting library that helps you write better structured, more maintainable automated acceptance criteria, and also produces rich meaningful test reports (or "living documentation") that not only report on the test results, but also what features have been tested. and for when your automated acceptance tests exercise a web interface, serenity comes with a host of features that make writing your automated web tests easier and faster. 1. bdd fundamentals but before we get into the nitty-gritty details, let’s talk about behaviour driven development, which is a core concept underlying many of serenity’s features. behaviour driven development, or bdd, is an approach where teams use conversations around concrete examples to build up a shared understanding of the features they are supposed to build. for example, suppose you are building a site where artists and craftspeople can sell their good online. one important feature for such a site would be the search feature. you might express this feature using a story-card format commonly used in agile projects like this: in order for buyers to find what they are looking for more efficiently as a seller i want buyers to be able to search for articles by keywords to build up a shared understanding of this requirement, you could talk through a few concrete examples. the converstaion might go something like this: "so give me an example of how a search might work." "well, if i search for wool , then i should see only woolen products." "sound’s simple enough. are there any other variations on the search feature that would produce different outcomes?" "well, i could also filter the search results; for example, i could look for only handmade woolen products." and so on. in practice, many of the examples that get discussed become "acceptance criteria" for the features. and many of these acceptance criteria become automated acceptance tests. automating acceptence tests provides valuable feedback to the whole team, as these tests, unlike unit and integrationt tests, are typically expressed in business terms, and can be easily understood by non-developers. and, as we will se later on in this article, the reports that are produced when these teste are executed give a clear picture of the state of the application. 2. serenity bdd and junit in this article, we will learn how to use serenity bdd using nothing more than junit, serenity bdd, and a little selenium webdriver. automated acceptance tests can use more specialized bdd tools such as cucumber or jbehave, but many teams like to keep it simple, and use more conventional unit testing tools like junit. this is fine: the essence of the bdd approach lies in the conversations that the teams have to discuss the requirements and discover the acceptance criteria. 2.1. writing the acceptance test let’s start off with a simple example. the first example that was discussed was searching for wool . the corresponding automated acceptance test for this example in junit looks like this: @runwith(serenityrunner.class) public class whensearchingbykeyword { @managed(driver="chrome", uniquesession = true) webdriver driver; @steps buyersteps buyer; @test public void should_see_a_list_of_items_related_to_the_specified_keyword() { // given buyer.opens_etsy_home_page(); // when buyer.searches_for_items_containing("wool"); // then. buyer.should_see_items_related_to("wool"); } } the serenity test runner sets up the test and records the test results this is a web test, and serenity will manage the webdriver driver for us we hide implementation details about how the test will be executed in a "step library" our test itself is reduced to the bare essential business logic that we want to demonstrate there are several things to point out here. when you use serenity with junit, you need to use the serenityrunner test runner. this instruments the junit class and instantiates the webdriver driver (if it is a web test), as well as any step libraries and page objects that you use in your test (more on these later). the @managed annotation tells serenity that this is a web test. serenity takes care of instantiating the webdriver instance, opening the browser, and shutting it down at the end of the test. you can also use this annotation to specify what browser you want to use, or if you want to keep the browser open during all of the tests in this test case. the @steps annotation tells serenity that this variable is a step library. in serenity, we use step libraries to add a layer of abstraction between the "what" and the "how" of our acceptance tests. at the top level, the step methods document "what" the acceptance test is doing, in fairly implementation-neutral, business-friendly terms. so we say "searches for items containing wool ", not "enters wool into the search field and clicks on the search button". this layered approach makes the tests both easier to understand and to maintain, and helps build up a great library of reusable business-level steps that we can use in other tests. 2.2. the step library the step library class is just an ordinary java class, with methods annotated with the @step annotation: public class buyersteps { homepage homepage; searchresultspage searchresultspage; @step public void opens_etsy_home_page() { homepage.open(); } @step public void searches_for_items_containing(string keywords) { homepage.searchfor(keywords); } @step public void should_see_items_related_to(string keywords) { list resulttitles = searchresultspage.getresulttitles(); resulttitles.stream().foreach(title -> assertthat(title.contains(keywords))); } } //end:tail step libraries often use page objects, which are automatically instantiated the @step annotation indicates a method that will appear as a step in the test reports for automated web tests, the step library methods do not call webdriver directly, but rather they typically interact with page objects . 2.3. the page objects page objects encapsulate how a test interacts with a particular web page. they hide the webdriver implementation details about how elements on a page are accessed and manipulated behind more business-friendly methods. like steps, page objects are reusable components that make the tests easier to understand and to maintain. serenity automatically instantiates page objects for you, and injects the current webdriver instance. all you need to worry about is the webdriver code that interacts with the page. and serenity provides a few shortcuts to make this easier as well. for example, here is the page object for the home page: @defaulturl("http://www.etsy.com") public class homepage extends pageobject { @findby(css = "button[value='search']") webelement searchbutton; public void searchfor(string keywords) { $("#search-query").sendkeys(keywords); searchbutton.click(); } } what url should be used by default when we call the open() method a serenity page object must extend the pageobject class you can use the $ method to access elements directly using css or xpath expressions or you may use a member variable annotated with the @findby annotation and here is the second page object we use: public class searchresultspage extends pageobject { @findby(css=".listing-card") list listingcards; public list getresulttitles() { return listingcards.stream() .map(element -> element.gettext()) .collect(collectors.tolist()); } } in both cases, we are hiding the webdriver implementation of how we access the page elements inside the page object methods. this makes the code both easier to read and reduces the places you need to change if a page is modified. this approach encourages a very high degree of reuse. for example, the second example mentioned at the start of this article involved filtering results by type. the corresponding automated acceptance criteria might look like this: @test public void should_be_able_to_filter_by_item_type() { // given buyer.opens_etsy_home_page(); // when buyer.searches_for_items_containing("wool"); int unfiltereditemcount = buyer.get_matching_item_count(); // and buyer.filters_results_by_type("handmade"); // then buyer.should_see_items_related_to("wool"); // and buyer.should_see_item_count(lessthan(unfiltereditemcount)); } @test public void should_be_able_to_view_details_about_a_searched_item() { // given buyer.opens_etsy_home_page(); // when buyer.searches_for_items_containing("wool"); buyer.selects_item_number(5); // then buyer.should_see_matching_details(); } notice how most of the methods here are reused from the previous steps: in fact, only two new methods are required. 3. reporting and living documentation reporting is one of serenity’s fortes. serenity not only reports on whether a test passes or fails, but documents what it did, in a step-by-step narrative format that inculdes test data and screenshots for web tests. for example, the following page illustrates the test results for our first acceptance criteria: figure 1. test results reported in serenity but test outcomes are only part of the picture. it is also important to know what work has been done, and what is work in progress. serenity provides the @pending annotation, that lets you indicate that a scenario is not yet completed, but has been scheduled for work, as illustrated here: @runwith(serenityrunner.class) public class whenputtingitemsintheshoppingcart { @pending @test public void shouldupdateshippingpricefordifferentdestinationcountries() { } } this test will appear in the reports as pending (blue in the graphs): figure 2. test result overview we can also organize our acceptance tests in terms of the features or requirements they are testing. one simple approach is to organize your requirements in suitably-named packages: |----net | |----serenity_bdd | | |----samples | | | |----etsy | | | | |----features | | | | | |----search | | | | | | |----whensearchingbykeyword.java | | | | | | |----whenviewingitemdetails.java | | | | | |----shopping_cart | | | | | | |----whenputtingitemsintheshoppingcart.java | | | | |----pages | | | | | |----homepage.java | | | | | |----itemdetailspage.java | | | | | |----registerpage.java | | | | | |----searchresultspage.java | | | | | |----shoppingcartpage.java | | | | |----steps | | | | | |----buyersteps.java all the test cases are organized under the features directory. test cass related to the search feature test cases related to the ‘shopping cart’ feature serenity can use this package structure to group and aggregate the test results for each feature. you need to tell serenity the root package that you are using, and what terms you use for your requirements. you do this in a special file called (for historical reasons) thucydides.properties , which lives in the root directory of your project: thucydides.test.root=net.serenity_bdd.samples.etsy.features thucydides.requirement.types=feature,story with this configured, serenity will report about how well each requirement has been tested, and will also tell you about the requirements that have not been tested: figure 3. serenity reports on requirements as well as tests 4. conclusion hopefully this will be enough to get you started with serenity. that said, we have barely scratched the surface of what serenity can do for your automated acceptance tests. you can read more about serenity, and the principles behind it, by reading the users manual , or by reading bdd in action , which devotes several chapters to these practices. and be sure to check out the online courses at parleys . you can get the source code for the project discussed in this article on github .

recently some pwc tech supremos wrote an article: agile coding in enterprise it: code small and local . subsections: moving away from the monolith why microservices? msa: a think-small approach for rapid development thinking the msa way: minimalism is a must where msa makes sense in msa, integration is the problem, not the solution conclusion msa is short for microservices architecture(s), in the above article. the article posits that microservices is the antidote to monoliths. it doesn’t mention cookie cutter scaling at all, which is another antidote to monoliths, with the right build infrastructure and devops. here’s a view of hypothetical architecture a company could deploy if they were doing microservices: w is web server. p and q don’t stand for anything in particular. here’s the same solution as cookie-cutter scaling, and the alternate (historical) choice of monolith to the right of it: the cookie cutter approach will often leverage components that are dependency injected into each other, and though monoliths might be the same today, pre 2004 they were probably hairballs of singletons (the design patten, not the springframework idiom). continuous delivery, agile? here’s one excerpt that confuses me: " … makes no sense to design and develop software over an 18-month process to accommodate all possible use cases when those use cases can change unexpectedly and the life span of code modules might be less than 18 months…. as i recall, the 18 month-delay problem was solved previously. agile methodologies principally, and continuous delivery/deployment in more recent times. it does not matter whether you’re compiling a monolith, a cookie-cutter solution, old soa services, or microservices, the 18-month fear isn’t real if you’re doing agile and/or cd. agile and cd were increasing the release cadence, and allowing the organization to pivot faster before microservices. it doesn’t matter whether you’ve got a monolith, something cookie-cutter scaled, or soa (micro or not), you’re going to be able to benefit from agile practices and devops setup that facilitates cd. in something like 30 thoughtworks client engagements since 2002, i have not seen the 18-month process at all. in fact i last encountered it in 1997 on an as/400 project, which was the last time i saw a waterfall process being championed. build(s) and trunk elsewhere there is a suggestion: “each microservice [has] its own build, to avoid trunk conflict”. that isn’t unique to microservices, of course. component based systems today also have a multiple build file (module) structure in a source tree. hopefully “trunk” mentioned is alluding to trunk based development, as i would recommend. build technologies this is a expansion on the above, and you can skip this paragraph if you want. hierarchical build systems like maven has allow you to have one build file per module (whether that’s a service or a simple jar destined for the classpath of a bigger thing). buck has a build grammar that allows for a build to grow/shrink/change based on what is being built (from implicitly shared source). maven is for the java ecosystem, while buck promises to be multi-language. both are doing multi-module builds for the sake of a composed or servicified deployment. both maven and buck are presently competing to draw the most reduced set of compile/test/deploy operations for the changes since last build for a hierarchy of modules. anyway, what is it we are striving for? what we want is to develop cheaply, and to deploy smoothly and often, without defect. we want the ability to deploy without large permanent or temporary headcount overseeing or participating in deployment. aside from development costs, and support/operation, deployment costs are a potentially big factor in total cost of ownership. what i like about cookie-cutter is the uniformity of the deployable things. the team size for deployment of such a thing doesn’t grow with the numbers of nodes that binary is being deployed to. at least, if you’re able to automate the deployment to those nodes, and have a strategy for handling the users connected to the stack at redeployment time somehow (sessions or stateless). the uniformity of the deployment is a cheapener, i think. when you have a number of dissimilar services, you might be able to minimize release personnel if you’re only doing one service. if more than one service is being updated in a particular deployment, you’re going to have to concentrate to make sure you don’t experience a multiplier effect for the participants. it is possible of course, to keep the headcount small, but the practice needed beforehand is bigger, which in turn allows for some calmness around the actual deployment. if we’ve stepped away from the project management office thinking that suggests three buggy releases a year (which is more usual than 18 month schedules of old), then we can employ continuous deployment to further eliminate personnel costs around going live. this is something that microservices does well at, but because the most adept proponents design forwards & backwards compatibility into the permutations most likely to co-exist in production. it is at least much quicker to redeploy and bounce one small service, n times than the the cookie-cutter uniform deployment.

In this blog post I will look into the basics of configuring Role Based Access Control (RBAC) in EAP and Wildfly. RBAC was introduced in EAP 6.2 and WildFly 8 so you will need either of those if you wish to use RBAC. For the purposes of this blog I will be using the following: OS - Ubuntu 14 Java - 1.7.0_67 JBoss - EAP 6.3 Although I'm using EAP these instructions should work just the same on Wildfly. What is RBAC? Role Based Access Control is designed to restrict system access by specifying permissions for management users. Each user with management access is given a role and that role defines what they can and cannot access. In EAP 6.2+ and Wildfly 8+ there are seven predefined roles each of which has different permissions. Details on each of the roles can be found here: https://access.redhat.com/documentation/en-US/JBoss_Enterprise_Application_Platform/6.2/html/Security_Guide/Supported_Roles.html In order to authenticate users one of the three standard authentication providers must be used. These are: Local User - The local user is automatically added as a SuperUser so a user on the server machine has full access. This user should be removed in a production system and access locked down to named users. Username/Password - using either the mgmt-users.properties file, or an LDAP server. Client Certificate - using a trust store For the purposes of this blog and to keep things simple we will use username/passwords and the mgmt-users.properties file Why do we need RBAC? The easiest way to show this is through a practical demo. Configuration can be done either via the Management Console or via the Command Line Interface (CLI). However, only a limited set of tasks can be done via the management console whereas all tasks are available via the CLI. Therefore, for the purposes of this blog I will be doing all configuration via the CLI. In our test scenario we have 4 users: Andy - This user is the main sys-admin and therefore we want him to be able to access everything. Bob - This user is a lead developer and therefore will need to be able to deploy apps and make changes to certain application resources. Clare & Dave - These users are standard developers and will need to be able to view application resources but should not be able to make changes. First of all we will set up a number of users. In order to do so we will use the add-user.sh script which can be found in: /bin Create the following users in the stated groups. (Enter No for the final question for all users) Andy - no group Bob - lead-developers Clare - standard-developers Dave - standard-developers In /domain/configuration you will find a file called mgmt-users.properties. At the bottom of this file you will see a list of the users we've created similar to this: Andy=82153e0297590cceb14e7620ccd3b6ed Bob=06a61e836d9d2d5be98517b468ab72cc Clare=63a8ff615a122c56b1d47fc098ff5124 Dave=2df8d1e02e7f3d13dcea7f4b022d0165 In the same directory you will find a a file called mgmt-groups.properties, at the bottom of this file you will see a list of users and the groups they are in, like so: Andy= Bob=lead-developers Clare=developers Dave=developers Now point a browser at http://localhost:9990 and log in as the user Dave. Navigate around and you will see you have full access to everything. This is precisely why RBAC is needed! Allowing all users to not only access the management console but to be able to access and alter anything is a recipe for disaster and guaranteed to cause issues further down the line. Often users don't understand the implications of the changes they have made, it may just be a quick fix to resolve an immediate issue but it may have long term consequences that are not noticed until much further down the line when the changes that were made have been forgotten about or are not documented. As someone who works in support we see these kind of issues on a regular basis and they can be difficult to track down with no audit trail and users not realising that the minor change they made to one part of the system is now causing a major issue in some other part of the system. OK, so we now have our users set up but at the moment they have full access to everything. Next up we will configure these users and assign them to roles. First of all start up the CLI. Run the following command: /bin/jboss-cli.sh -c Change directory to the authorisation node cd /core-service=management/access=authorization Running the following command lists the current role names and the standard role names along with two other attributes ls -l The two we are interested in here are permission-combination-policy and provider. The permission-combination-policy defines how permissions are determined if a user is assigned more than one role. The default setting is permissive. This means that if a user is assigned to any role that allows a particular action then the user can perform that action. The opposite of this is rejecting. This means that if a user is assigned to multiple roles then all those roles must permit an action for a user to be able to perform that action. The other attribute of interest here is provider. This can be set to either simple (which is the default) or rbac. In simple mode all management users can access everything and make changes, as we have seen. In rbac mode users are assigned roles and each of those roles has difference privileges. Switching on RBAC OK, lets turn on RBAC... Run the following commands to turn on RBAC cd /core-service=management/access=authorization :write-attribute(name=provider, value=rbac) Restart JBoss Now point a browser at http://localhost:9990 and try to log in as the user Andy (who should be able to access everything). You should see the following message : Insufficient privileges to access this interface. This is because at the moment the user Andy isn't mapped to any role. Let's fix that now: If you look in domain.xml in the management element you will see the following: This shows that at the moment only the local user is mapped to the SuperUser role. Mapping users and groups to roles We need to map our users to the relevant roles to allow them access. In order to do this we need the following command: role-mapping=ROLENAME/include=ALIAS:add(name=USERNAME, type=USER) Where rolename is one of the pre-configured roles, alias is a unique name for the mapping and user is the name of the user to map. So, lets map the user Andy to the SuperUser role. ./role-mapping=SuperUser/include=user-Andy:add(name=Andy, type=USER) In domain.xml you will see that our user has been added to the SuperUser role: Now point a browser at http://localhost:9990 you should now be able to log in as the user Andy and have full access to everything. Next we need to add mappings for the other roles we want to use. ./role-mapping=Deployer:add ./role-mapping=Monitor:add Now we need to give role mappings to all our other users. As we have them in groups we can assign the groups to roles, rather than mapping by user. The command is basically the same as for a user but the type is GROUP rather than user. Here we are mapping lead developers to the Deployer role and standard developers to the Monitor role. ./role-mapping=Deployer/include=group-lead-devs:add(name=lead-developers, type=GROUP) ./role-mapping=Monitor/include=group-standard-devs:add(name=developers, type=GROUP) If you look in domain.xml you should now see the following showing that the user Andy is mapped to the SuperUser role and the two groups are mapped to the Deployer and Monitor roles. You can also view the role mappings in the admin console. Click on the Administration tab. Expand the Access Control item on the left and select Role Assignment. Select the Users tab - this shows users that are mapped to roles. Select the Groups tab and you will see the mapping between groups and roles. Log in as the different users and see the differences between what you can and can't access. Conclusion So, that's it for Part One. We have switched on RBAC, set up a number of users and groups and mapped those users and groups to particular roles to give them different levels of access. In Part Two of this blog I will look at constraints which allow more fine grained permission setting, scoped roles which allow you to set permissions on individual servers and audit logging which allows you to see who is accessing the management console and see what changes they are making.

Have an interview coming up? Let us help you prep with these JVA and garbage collection basics.

I often get asked what it takes to be an effective product manager or product owner, which product skills the individuals should have, and how a company can strengthen its product management function. Answering these questions requires an understanding of what effective product management looks like in the digital age. The following picture shows how I view product management: It depicts a product management framework that consists of six core knowledge areas and by six supporting ones. The core areas are orange and placed centrally. The supporting ones are purple and located at the edge of the circle. You can download the picture for free by simply clicking on it or from romanpichler.com/tools/product-management-framework. The core areas are particularly important for doing a great job as a product manager or product owner. You should hence strive to become knowledgeable in all of them. The supporting areas are also important for your work, especially when you manage commercial products, but generally not as crucial. If the product management framework with its knowledge areas feels overwhelming then don’t worry: Product management is a complex and demanding discipline that is not easy to master. It takes time and effort to become a competent product manager or product owner. The good news is that you can use the framework to spot gaps in your skill set so you can address them. The Core Knowledge Areas Vision and Leadership: Working as an effective product managers or product owner requires vision and leadership skills. You should be able to establish a shared vision, set realistic goals, and describe the benefits your product should deliver. You should be able to actively listen to others and negotiate to reach agreement and get buy-in. At the same time, you should not shy away from making the right product decisions even if they are tough and do not please everyone. You should be able to manage the stakeholders including customers and users, senior management, development, marketing, sales, support, and other business groups that have to contribute to the product success. You should be able to effectively communicate with and influence them. You be comfortable working with a broad range of people from diverse backgrounds including a cross-functional development team. Product Lifecycle Management: Managing a product successfully involves more than getting it built and released. You should understand the product lifecycle with its stages and the key events in the life of your product including launch, product-market fit, and end of sales; you should know how the lifecycle helps you maximise the benefits your product creates across its entire life; this includes the lifecycle’s impact on the product performance (revenue and profits), the product goals, the pricing and the marketing strategy; the options to revive growth as your product matures and growth starts to stagnate; and the process best suited for each lifecycle stage. (An iterative, Lean Startup and Scrum-based process tends to beneficial while your product is young; Kanban is usually preferable when your product starts to mature.) Product Strategy and Market Research: Your product exists to serve a market or market segment, a group of people whose need the product addresses. You therefore should be able to identify your target users and customers and segment the market; you should be able to clearly state the value proposition of your product, why people would want to use and buy it and why your product does a great job at creating value for them. You should be able to carry out a competitor analysis to understand their respective strengths and weaknesses; you should be able to position your product, and determine the values the brand needs to communicate. You should be able to perform the necessary market research work to test your ideas and assumptions about the market segment and the value proposition. This includes qualitative and quantitative methods including problem interviews, direct observations, and employing minimum viable products (MVPs); you should be able to leverage data to make the right decisions. This includes using an analytics tool, analysing the data effectively, and deciding if you should pivot and change your strategy or if you should persevere and refine it. Business Model and Financials: To provide an investment incentive for your company and to make developing and providing the product sustainable, you have to be able to determine the value the product creates for your firm. You should be able to formulate and prioritise business goals, for instance, enter a new market, meet a revenue or profit goal, save cost, or develop the brand. You should be able to describe how your product’s value proposition is monetised and capture how the business model works including the revenue sources and the main cost factors. You should also be able to create a financial forecast or business case that describes when a break-even is likely to occur and when your product may become profitable. In practice, you may want to partner with a colleague from the finance department to carry out this work. Product Roadmap: Many people have to contribute to the success of a digital product. To help them do their work and to provide visibility of how your product is likely to evolve, you should be able to create and use a product roadmap. This includes formulating realistic product goals (benefits), metrics and key performance indicators (KPIs), release dates or timeframes, and key features (deliverables or results). You should be clear on the relationship between the product strategy and the product roadmap. You should be able to formulate a go-to-market strategy and capture it in your roadmap. You should understand when the roadmap should be reviewed and changed. User Experience and Product Backlog: A great product has to offer a great user experience (UX). You should be able to describe the desired user experience. This includes describing users and customer as personas, capturing the user interaction, the visual design, the functional and the non-functional aspects of your product together with the help of the cross-functional team (a UX/UI expert should be part of the team). You should be able to create scenarios, epics, user stories, storyboards, workflow diagrams and storymaps, and be able to work with user interface sketches and mock-ups. You should be able to stock and manage the product backlog, prioritise it effectively, and select sprint goals. You should know how to understand if you develop a product with the right features and the right UX, how to test the appropriate aspects of your product and how to collect the relevant feedback and data. This includes the ability to perform product demoes, solution interviews, usability tests, A/B tests, and direct observation. You should be able to use an analytics tool to retrieve the relevant data and be able to analyse it effectively. You be able to change (or “groom”) the product backlog using the newly gained insights. Continue reading... The Supporting Knowledge Areas General Market Knowledge: Understand who your current customers and users are, what product you offer them today including their value proposition and business model, what competitors you have, how big your market share currently is, and which market segments you serve well. Development/Technologies: Be a competent partner for development/IT/engineering, have an interest in software technologies, be comfortable collaborating with a cross-functional technical team. Marketing: Be a respected partner for (product) marketing; be able to help select the right select the right marketing channels and to determine the right marketing mix; help marketing with creating the marketing collateral. Sales and Support: Be a respected partner for sales and support; be able to help select the right sales channels and create the sales collateral and training. Project/Release management: Be able to determine the primary success factor for a major release/product version and to steer the development project; be able to determine the project progress to forecast the progress, for instance, using a release burndown chart; be able to work with the Definition of Done; be able to trade-off scope, time, and budget. Process: Have a good understanding of ideation and innovation processes to generate and select ideas and to bring new products and new features to life. These should include Customer Development/Lean Startup, Business Model Generation, Scrum, and Kanban. Defining Product Roles with the Framework My product management framework helps you define product roles and the skills and responsibilities they should have. Using the framework, I can, for instance, define the role of a product owner in the following way: As the picture above shows, a product owner should have strategic product management skills such as product strategy and roadmapping as well as tactical ones (UX and product backlog). I have circled the areas, which are required by Scrum – the framework in which the role originated – in dark orange. The other areas are necessary to allow the product owner to do a great job and achieve product success even though they are not mandated by Scrum. You may, of course, disagree with my take on the product owner role and may want to use the framework to capture your definition of the role. Another example of how you can apply the framework is the description of the role of a tech product manager, a product manager who looks after a technical product and requires more in-depth technology/development skills, as the following picture illustrates. For instance, one of my clients is a major games development company, which has its own in-house developed physics engine, a complex piece of software that does all the clever animation. The product owner of the physics engine is a former developer. This makes sense, as the individual requires a detailed technical knowledge about the product and has to be able to communicate effectively with the users, the game developers. If you work as a product manager who looks after digital products that are developed and used in-house, for instance, a finance or HR application, then you probably have to tailor the supporting areas as the following picture shows: In the picture above, I have removed “Marketing” and replaced “Sales and Support” with “Operations”. I have kept “General Market Knowledge” as it is desirable for the product manager of a finance application to understand the market, that is, how the finance group works, what problems people struggle with, which products they use, and so forth. Determining Learning Measures with the Framework You can also use the product management framework to identify gaps in your skill set. Use the knowledge areas and reflect on your own knowledge. Then identify the areas where you lack some knowledge and skills, as I have done in the picture below by high-lightening the areas, which product owners often need to strengthen in my experience. Then rank them by determining how much the lack of knowledge is preventing you from doing a great job. For instance, a lack of product lifecycle management and product roadmap skills may be hurting you most if you manage a product that is in the growth stage. Finally identify how you best close the gap, for instance, reading one or more books, or blogs, attending a training course, finding someone to mentor or coach you, forming a community of practice with your fellow product managers or product owners to share knowledge and support each other. You can do the same exercise for a group of product managers or product owners to identify learning measure for the entire function. Learn More You can learn more about specific areas such as vision and leadership, product strategy and market research, product roadmap, or user experience and product backlog by attending one of my training courses. I also teach my courses onsite and in from of interactive virtual training sessions. If you would like me to help you apply the framework, define roles, or identify the right learning and development measures for product managers and product owners, then please contact me.

The article represents the 3rd set of 10 interview questions. The following are previous two sets that have been published earlier on our website. Following are other sets that we recommend you to go through. Interview questions Set 1 Interview questions Set 2 Q1: Directives can be applied to which all element type? Ans: Following represents the element type and directive declaration style: `E` – Element name: `` `A` – Attribute (default): `` `C` – Class: `` `M` – Comment: `` Q2. What is notion of “isolate” scope object when creating a custom directive? How is it different from the normal scope object? Ans: When creating a custom directive, there is a property called as “scope” which can be assigned different values such as true/false or {}. When assigned with the value “{}”, then a new “isolate” scope is created. The ‘isolate’ scope differs from normal scope in that it does not prototypically inherit from the parent scope. This is useful when creating reusable components, which should not accidentally read or modify data in the parent scope. Q3. What are different return types from compile function? Ans: A compile function can have a return value which can be either a function or an object. A (post-link) function: It is equivalent to registering the linking function via the `link` property of the config object when the compile function is empty. An object with function(s) registered via `pre` and `post` properties. It allows you to control when a linking function should be called during the linking phase. Q4. WHich API need to be invoked on the rootScope service to get the child scopes? Ans: $new Q5. Explain the relationship between scope.$apply & scope.$digest? Ans: As an event such as text change in a textfield happens, the event is caught with an eventhandler which then invokes $apply method on the scope object. The $apply method in turn evaluates the expression and finally invokes $digest method on the scope object. Following code does it all: $apply: function(expr) { try { beginPhase('$apply'); return this.$eval(expr); } catch (e) { $exceptionHandler(e); } finally { clearPhase(); try { $rootScope.$digest(); } catch (e) { $exceptionHandler(e); throw e; } } } Q6. Which angular module is loaded by default? Ans: ng Q7. What angular function is used to manually start an application? Ans: angular.bootstrap Q8. Name some of the methods that could be called on a module instance? For example, say, you instantiated a module such as ‘var helloApp = angular.module( “helloApp”, [] );’. What are different methods that could be called on helloApp instance? Ans: Following are some of the methods: controller factory directive filter constant service provider config Q9. Which angular function is used to wrap a raw DOM element or HTML string as a jQuery element? Ans: angular.element; If jQuery is available, `angular.element` is an alias for the jQuery function. If jQuery is not available, `angular.element` delegates to Angular’s built-in subset of jQuery, called “jQuery lite” or “jqLite.” Q10. Write sample code representing an injector that could be used to kick off your application? var $injector = angular.injector(['ng', 'appName']); $injector.invoke(function($rootScope, $compile, $document){ $compile($document)($rootScope); $rootScope.$digest(); }); Feel free to suggest any changes in above answers if you feel so.

Gephi comes with tools to analyse properties of a graph.

I’ve been working on migrating some batch jobs for Podcastpedia.org to Spring Batch. Before, these jobs were developed in my own kind of way, and I thought it was high time to use a more “standardized” approach. Because I had never used Spring with java configuration before, I thought this were a good opportunity to learn about it, by configuring the Spring Batch jobs in java. And since I am all into trying new things with Spring, why not also throw Spring Boot into the boat… Before you begin with this tutorial I recommend you read first Spring’s Getting started – Creating a Batch Service, because the structure and the code presented here builds on that original. 1. What I’ll build So, as mentioned, in this post I will present Spring Batch in the context of configuring it and developing with it some batch jobs for Podcastpedia.org. Here’s a short description of the two jobs that are currently part of the Podcastpedia-batch project: addNewPodcastJob reads podcast metadata (feed url, identifier, categories etc.) from a flat file transforms (parses and prepares episodes to be inserted with Http Apache Client) the data and in the last step, insert it to the Podcastpedia database and inform the submitter via emailabout it notifyEmailSubscribersJob – people can subscribe to their favorite podcasts on Podcastpedia.orgvia email. For those who did it is checked on a regular basis (DAILY, WEEKLY, MONTHLY) if new episodes are available, and if they are the subscribers are informed via email about those; read from database, expand read data via JPA, re-group it and notify subscriber via email Source code: The source code for this tutorial is available on GitHub – Podcastpedia-batch. Note: Before you start I also highly recommend you read the Domain Language of Batch, so that terms like “Jobs”, “Steps” or “ItemReaders” don’t sound strange to you. 2. What you’ll need A favorite text editor or IDE JDK 1.7 or later Maven 3.0+ 3. Set up the project The project is built with Maven. It uses Spring Boot, which makes it easy to create stand-alone Spring based Applications that you can “just run”. You can learn more about the Spring Boot by visiting theproject’s website. 3.1. Maven build file Because it uses Spring Boot it will have the spring-boot-starter-parent as its parent, and a couple of other spring-boot-starters that will get for us some libraries required in the project: pom.xml of the podcastpedia-batch project 4.0.0 org.podcastpedia.batch podcastpedia-batch 0.1.0 1.1.6.RELEASE 1.7 org.springframework.boot spring-boot-starter-parent 1.1.6.RELEASE org.springframework.boot spring-boot-starter-batch org.springframework.boot spring-boot-starter-data-jpa org.apache.httpcomponents httpclient 4.3.5 org.apache.httpcomponents httpcore 4.3.2 org.apache.velocity velocity 1.7 org.apache.velocity velocity-tools 2.0 org.apache.struts struts-core rome rome 1.0 rome rome-fetcher 1.0 org.jdom jdom 1.1 xerces xercesImpl 2.9.1 mysql mysql-connector-java 5.1.31 org.springframework.boot spring-boot-starter-freemarker org.springframework.boot spring-boot-starter-remote-shell javax.mail mail javax.mail mail 1.4.7 javax.inject javax.inject 1 org.twitter4j twitter4j-core [4.0,) org.springframework.boot spring-boot-starter-test maven-compiler-plugin org.springframework.boot spring-boot-maven-plugin Note: One big advantage of using the spring-boot-starter-parent as the project’s parent is that you only have to upgrade the version of the parent and it will get the “latest” libraries for you. When I started the project spring boot was in version 1.1.3.RELEASE and by the time of finishing to write this post is already at 1.1.6.RELEASE. 3.2. Project directory structure I structured the project in the following way: └── src └── main └── java └── org └── podcastpedia └── batch └── common └── jobs └── addpodcast └── notifysubscribers Note: the org.podcastpedia.batch.jobs package contains sub-packages having specific classes to particular jobs. the org.podcastpedia.batch.jobs.common package contains classes used by all the jobs, like for example the JPA entities that both the current jobs require. 4. Create a batch Job configuration I will start by presenting the Java configuration class for the first batch job: package org.podcastpedia.batch.jobs.addpodcast; import org.podcastpedia.batch.common.configuration.DatabaseAccessConfiguration; import org.podcastpedia.batch.common.listeners.LogProcessListener; import org.podcastpedia.batch.common.listeners.ProtocolListener; import org.podcastpedia.batch.jobs.addpodcast.model.SuggestedPodcast; import org.springframework.batch.core.Job; import org.springframework.batch.core.Step; import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; import org.springframework.batch.core.configuration.annotation.JobBuilderFactory; import org.springframework.batch.core.configuration.annotation.StepBuilderFactory; import org.springframework.batch.item.ItemProcessor; import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.ItemWriter; import org.springframework.batch.item.file.FlatFileItemReader; import org.springframework.batch.item.file.LineMapper; import org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper; import org.springframework.batch.item.file.mapping.DefaultLineMapper; import org.springframework.batch.item.file.transform.DelimitedLineTokenizer; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Import; import org.springframework.core.io.ClassPathResource; import com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException; @Configuration @EnableBatchProcessing @Import({DatabaseAccessConfiguration.class, ServicesConfiguration.class}) public class AddPodcastJobConfiguration { @Autowired private JobBuilderFactory jobs; @Autowired private StepBuilderFactory stepBuilderFactory; // tag::jobstep[] @Bean public Job addNewPodcastJob(){ return jobs.get("addNewPodcastJob") .listener(protocolListener()) .start(step()) .build(); } @Bean public Step step(){ return stepBuilderFactory.get("step") .chunk(1) //important to be one in this case to commit after every line read .reader(reader()) .processor(processor()) .writer(writer()) .listener(logProcessListener()) .faultTolerant() .skipLimit(10) //default is set to 0 .skip(MySQLIntegrityConstraintViolationException.class) .build(); } // end::jobstep[] // tag::readerwriterprocessor[] @Bean public ItemReader reader(){ FlatFileItemReader reader = new FlatFileItemReader(); reader.setLinesToSkip(1);//first line is title definition reader.setResource(new ClassPathResource("suggested-podcasts.txt")); reader.setLineMapper(lineMapper()); return reader; } @Bean public LineMapper lineMapper() { DefaultLineMapper lineMapper = new DefaultLineMapper(); DelimitedLineTokenizer lineTokenizer = new DelimitedLineTokenizer(); lineTokenizer.setDelimiter(";"); lineTokenizer.setStrict(false); lineTokenizer.setNames(new String[]{"FEED_URL", "IDENTIFIER_ON_PODCASTPEDIA", "CATEGORIES", "LANGUAGE", "MEDIA_TYPE", "UPDATE_FREQUENCY", "KEYWORDS", "FB_PAGE", "TWITTER_PAGE", "GPLUS_PAGE", "NAME_SUBMITTER", "EMAIL_SUBMITTER"}); BeanWrapperFieldSetMapper fieldSetMapper = new BeanWrapperFieldSetMapper(); fieldSetMapper.setTargetType(SuggestedPodcast.class); lineMapper.setLineTokenizer(lineTokenizer); lineMapper.setFieldSetMapper(suggestedPodcastFieldSetMapper()); return lineMapper; } @Bean public SuggestedPodcastFieldSetMapper suggestedPodcastFieldSetMapper() { return new SuggestedPodcastFieldSetMapper(); } /** configure the processor related stuff */ @Bean public ItemProcessor processor() { return new SuggestedPodcastItemProcessor(); } @Bean public ItemWriter writer() { return new Writer(); } // end::readerwriterprocessor[] @Bean public ProtocolListener protocolListener(){ return new ProtocolListener(); } @Bean public LogProcessListener logProcessListener(){ return new LogProcessListener(); } } The @EnableBatchProcessing annotation adds many critical beans that support jobs and saves us configuration work. For example you will also be able to @Autowired some useful stuff into your context: a JobRepository (bean name “jobRepository”) a JobLauncher (bean name “jobLauncher”) a JobRegistry (bean name “jobRegistry”) a PlatformTransactionManager (bean name “transactionManager”) a JobBuilderFactory (bean name “jobBuilders”) as a convenience to prevent you from having to inject the job repository into every job, as in the examples above a StepBuilderFactory (bean name “stepBuilders”) as a convenience to prevent you from having to inject the job repository and transaction manager into every step The first part focuses on the actual job configuration: @Bean public Job addNewPodcastJob(){ return jobs.get("addNewPodcastJob") .listener(protocolListener()) .start(step()) .build(); } @Bean public Step step(){ return stepBuilderFactory.get("step") .chunk(1) //important to be one in this case to commit after every line read .reader(reader()) .processor(processor()) .writer(writer()) .listener(logProcessListener()) .faultTolerant() .skipLimit(10) //default is set to 0 .skip(MySQLIntegrityConstraintViolationException.class) .build(); } The first method defines a job and the second one defines a single step. As you’ve read in The Domain Language of Batch, jobs are built from steps, where each step can involve a reader, a processor, and a writer. In the step definition, you define how much data to write at a time (in our case 1 record at a time). Next you specify the reader, processor and writer. 5. Spring Batch processing units Most of the batch processing can be described as reading data, doing some transformation on it and then writing the result out. This mirrors somehow the Extract, Transform, Load (ETL) process, in case you know more about that. Spring Batch provides three key interfaces to help perform bulk reading and writing: ItemReader, ItemProcessor and ItemWriter. 5.1. Readers ItemReader is an abstraction providing the mean to retrieve data from many different types of input: flat files, xml files, database, jms etc., one item at a time. See the Appendix A. List of ItemReaders and ItemWriters for a complete list of available item readers. In the Podcastpedia batch jobs I use the following specialized ItemReaders: 5.1.1. FlatFileItemReader which, as the name implies, reads lines of data from a flat file that typically describe records with fields of data defined by fixed positions in the file or delimited by some special character (e.g. Comma). This type of ItemReader is being used in the first batch job, addNewPodcastJob. The input file used is named suggested-podcasts.in, resides in the classpath (src/main/resources) and looks something like the following: FEED_URL; IDENTIFIER_ON_PODCASTPEDIA; CATEGORIES; LANGUAGE; MEDIA_TYPE; UPDATE_FREQUENCY; KEYWORDS; FB_PAGE; TWITTER_PAGE; GPLUS_PAGE; NAME_SUBMITTER; EMAIL_SUBMITTER http://www.5minutebiographies.com/feed/; 5minutebiographies; people_society, history; en; Audio; WEEKLY; biography, biographies, short biography, short biographies, 5 minute biographies, five minute biographies, 5 minute biography, five minute biography; https://www.facebook.com/5minutebiographies;https://twitter.com/5MinuteBios; ; Adrian Matei; [email protected] http://notanotherpodcast.libsyn.com/rss; NotAnotherPodcast; entertainment; en; Audio; WEEKLY; Comedy, Sports, Cinema, Movies, Pop Culture, Food, Games; https://www.facebook.com/notanotherpodcastusa;https://twitter.com/NAPodcastUSA;https://plus.google.com/u/0/103089891373760354121/posts; Adrian Matei; [email protected] As you can see the first line defines the names of the “columns”, and the following lines contain the actual data (delimited by “;”), that needs translating to domain objects relevant in the context. Let’s see now how to configure the FlatFileItemReader: @Bean public ItemReader reader(){ FlatFileItemReader reader = new FlatFileItemReader(); reader.setLinesToSkip(1);//first line is title definition reader.setResource(new ClassPathResource("suggested-podcasts.in")); reader.setLineMapper(lineMapper()); return reader; } You can specify, among other things, the input resource, the number of lines to skip, and a line mapper. 5.1.1.1. LineMapper The LineMapper is an interface for mapping lines (strings) to domain objects, typically used to map lines read from a file to domain objects on a per line basis. For the Podcastpedia job I used the DefaultLineMapper, which is two-phase implementation consisting of tokenization of the line into a FieldSet followed by mapping to item: @Bean public LineMapper lineMapper() { DefaultLineMapper lineMapper = new DefaultLineMapper(); DelimitedLineTokenizer lineTokenizer = new DelimitedLineTokenizer(); lineTokenizer.setDelimiter(";"); lineTokenizer.setStrict(false); lineTokenizer.setNames(new String[]{"FEED_URL", "IDENTIFIER_ON_PODCASTPEDIA", "CATEGORIES", "LANGUAGE", "MEDIA_TYPE", "UPDATE_FREQUENCY", "KEYWORDS", "FB_PAGE", "TWITTER_PAGE", "GPLUS_PAGE", "NAME_SUBMITTER", "EMAIL_SUBMITTER"}); BeanWrapperFieldSetMapper fieldSetMapper = new BeanWrapperFieldSetMapper(); fieldSetMapper.setTargetType(SuggestedPodcast.class); lineMapper.setLineTokenizer(lineTokenizer); lineMapper.setFieldSetMapper(suggestedPodcastFieldSetMapper()); return lineMapper; } the DelimitedLineTokenizer splits the input String via the “;” delimiter. if you set the strict flag to false then lines with less tokens will be tolerated and padded with empty columns, and lines with more tokens will simply be truncated. the columns names from the first line are set lineTokenizer.setNames(...); and the fieldMapper is set (line 14) Note: The FieldSet is an “interface used by flat file input sources to encapsulate concerns of converting an array of Strings to Java native types. A bit like the role played by ResultSet in JDBC, clients will know the name or position of strongly typed fields that they want to extract.“ 5.1.1.2. FieldSetMapper The FieldSetMapper is an interface that is used to map data obtained from a FieldSet into an object. Here’s my implementation which maps the fieldSet to the SuggestedPodcast domain object that will be further passed to the processor: public class SuggestedPodcastFieldSetMapper implements FieldSetMapper { @Override public SuggestedPodcast mapFieldSet(FieldSet fieldSet) throws BindException { SuggestedPodcast suggestedPodcast = new SuggestedPodcast(); suggestedPodcast.setCategories(fieldSet.readString("CATEGORIES")); suggestedPodcast.setEmail(fieldSet.readString("EMAIL_SUBMITTER")); suggestedPodcast.setName(fieldSet.readString("NAME_SUBMITTER")); suggestedPodcast.setTags(fieldSet.readString("KEYWORDS")); //some of the attributes we can map directly into the Podcast entity that we'll insert later into the database Podcast podcast = new Podcast(); podcast.setUrl(fieldSet.readString("FEED_URL")); podcast.setIdentifier(fieldSet.readString("IDENTIFIER_ON_PODCASTPEDIA")); podcast.setLanguageCode(LanguageCode.valueOf(fieldSet.readString("LANGUAGE"))); podcast.setMediaType(MediaType.valueOf(fieldSet.readString("MEDIA_TYPE"))); podcast.setUpdateFrequency(UpdateFrequency.valueOf(fieldSet.readString("UPDATE_FREQUENCY"))); podcast.setFbPage(fieldSet.readString("FB_PAGE")); podcast.setTwitterPage(fieldSet.readString("TWITTER_PAGE")); podcast.setGplusPage(fieldSet.readString("GPLUS_PAGE")); suggestedPodcast.setPodcast(podcast); return suggestedPodcast; } } 5.2. JdbcCursorItemReader In the second job, notifyEmailSubscribersJob, in the reader, I only read email subscribers from a single database table, but further in the processor a more detailed read(via JPA) is executed to retrieve all the new episodes of the podcasts the user subscribed to. This is a common pattern employed in the batch world. Follow this link for more Common Batch Patterns. For the initial read, I chose the JdbcCursorItemReader, which is a simple reader implementation that opens a JDBC cursor and continually retrieves the next row in the ResultSet: @Bean public ItemReader notifySubscribersReader(){ JdbcCursorItemReader reader = new JdbcCursorItemReader(); String sql = "select * from users where is_email_subscriber is not null"; reader.setSql(sql); reader.setDataSource(dataSource); reader.setRowMapper(rowMapper()); return reader; } Note I had to set the sql, the datasource to read from and a RowMapper. 5.2.1. RowMapper The RowMapper is an interface used by JdbcTemplate for mapping rows of a Result’set on a per-row basis. My implementation of this interface, , performs the actual work of mapping each row to a result object, but I don’t need to worry about exception handling: public class UserRowMapper implements RowMapper { @Override public User mapRow(ResultSet rs, int rowNum) throws SQLException { User user = new User(); user.setEmail(rs.getString("email")); return user; } } 5.2. Writers ItemWriter is an abstraction that represents the output of a Step, one batch or chunk of items at a time. Generally, an item writer has no knowledge of the input it will receive next, only the item that was passed in its current invocation. The writers for the two jobs presented are quite simple. They just use external services to send email notifications and post tweets on Podcastpedia’s account. Here is the implementation of the ItemWriterfor the first job – addNewPodcast: package org.podcastpedia.batch.jobs.addpodcast; import java.util.Date; import java.util.List; import javax.inject.Inject; import javax.persistence.EntityManager; import org.podcastpedia.batch.common.entities.Podcast; import org.podcastpedia.batch.jobs.addpodcast.model.SuggestedPodcast; import org.podcastpedia.batch.jobs.addpodcast.service.EmailNotificationService; import org.podcastpedia.batch.jobs.addpodcast.service.SocialMediaService; import org.springframework.batch.item.ItemWriter; import org.springframework.beans.factory.annotation.Autowired; public class Writer implements ItemWriter{ @Autowired private EntityManager entityManager; @Inject private EmailNotificationService emailNotificationService; @Inject private SocialMediaService socialMediaService; @Override public void write(List items) throws Exception { if(items.get(0) != null){ SuggestedPodcast suggestedPodcast = items.get(0); //first insert the data in the database Podcast podcast = suggestedPodcast.getPodcast(); podcast.setInsertionDate(new Date()); entityManager.persist(podcast); entityManager.flush(); //notify submitter about the insertion and post a twitt about it String url = buildUrlOnPodcastpedia(podcast); emailNotificationService.sendPodcastAdditionConfirmation( suggestedPodcast.getName(), suggestedPodcast.getEmail(), url); if(podcast.getTwitterPage() != null){ socialMediaService.postOnTwitterAboutNewPodcast(podcast, url); } } } private String buildUrlOnPodcastpedia(Podcast podcast) { StringBuffer urlOnPodcastpedia = new StringBuffer( "http://www.podcastpedia.org"); if (podcast.getIdentifier() != null) { urlOnPodcastpedia.append("/" + podcast.getIdentifier()); } else { urlOnPodcastpedia.append("/podcasts/"); urlOnPodcastpedia.append(String.valueOf(podcast.getPodcastId())); urlOnPodcastpedia.append("/" + podcast.getTitleInUrl()); } String url = urlOnPodcastpedia.toString(); return url; } } As you can see there’s nothing special here, except that the write method has to be overriden and this is where the injected external services EmailNotificationService and SocialMediaService are used to inform via email the podcast submitter about the addition to the podcast directory, and if a Twitter page was submitted a tweet will be posted on the Podcastpedia’s wall. You can find detailed explanation on how to send email via Velocity and how to post on Twitter from Java in the following posts: How to compose html emails in Java with Spring and Velocity How to post to Twittter from Java with Twitter4J in 10 minutes 5.3. Processors ItemProcessor is an abstraction that represents the business processing of an item. While theItemReader reads one item, and the ItemWriter writes them, the ItemProcessor provides access to transform or apply other business processing. When using your own Processors you have to implement the ItemProcessor interface, with its only method O process(I item) throws Exception, returning a potentially modified or a new item for continued processing. If the returned result is null, it is assumed that processing of the item should not continue. While the processor of the first job requires a little bit of more logic, because I have to set the etag andlast-modified header attributes, the feed attributes, episodes, categories and keywords of the podcast: public class SuggestedPodcastItemProcessor implements ItemProcessor { private static final int TIMEOUT = 10; @Autowired ReadDao readDao; @Autowired PodcastAndEpisodeAttributesService podcastAndEpisodeAttributesService; @Autowired private PoolingHttpClientConnectionManager poolingHttpClientConnectionManager; @Autowired private SyndFeedService syndFeedService; /** * Method used to build the categories, tags and episodes of the podcast */ @Override public SuggestedPodcast process(SuggestedPodcast item) throws Exception { if(isPodcastAlreadyInTheDirectory(item.getPodcast().getUrl())) { return null; } String[] categories = item.getCategories().trim().split("\\s*,\\s*"); item.getPodcast().setAvailability(org.apache.http.HttpStatus.SC_OK); //set etag and last modified attributes for the podcast setHeaderFieldAttributes(item.getPodcast()); //set the other attributes of the podcast from the feed podcastAndEpisodeAttributesService.setPodcastFeedAttributes(item.getPodcast()); //set the categories List categoriesByNames = readDao.findCategoriesByNames(categories); item.getPodcast().setCategories(categoriesByNames); //set the tags setTagsForPodcast(item); //build the episodes setEpisodesForPodcast(item.getPodcast()); return item; } ...... } the processor from the second job uses the ‘Driving Query’ approach, where I expand the data retrieved from the Reader with another “JPA-read” and I group the items on podcasts with episodes so that it looks nice in the emails that I am sending out to subscribers: @Scope("step") public class NotifySubscribersItemProcessor implements ItemProcessor { @Autowired EntityManager em; @Value("#{jobParameters[updateFrequency]}") String updateFrequency; @Override public User process(User item) throws Exception { String sqlInnerJoinEpisodes = "select e from User u JOIN u.podcasts p JOIN p.episodes e WHERE u.email=?1 AND p.updateFrequency=?2 AND" + " e.isNew IS NOT NULL AND e.availability=200 ORDER BY e.podcast.podcastId ASC, e.publicationDate ASC"; TypedQuery queryInnerJoinepisodes = em.createQuery(sqlInnerJoinEpisodes, Episode.class); queryInnerJoinepisodes.setParameter(1, item.getEmail()); queryInnerJoinepisodes.setParameter(2, UpdateFrequency.valueOf(updateFrequency)); List newEpisodes = queryInnerJoinepisodes.getResultList(); return regroupPodcastsWithEpisodes(item, newEpisodes); } ....... } Note: If you’d like to find out more how to use the Apache Http Client, to get the etag and last-modifiedheaders, you can have a look at my post – How to use the new Apache Http Client to make a HEAD request 6. Execute the batch application Batch processing can be embedded in web applications and WAR files, but I chose in the beginning the simpler approach that creates a standalone application, that can be started by the Java main() method: package org.podcastpedia.batch; //imports ...; @ComponentScan @EnableAutoConfiguration public class Application { private static final String NEW_EPISODES_NOTIFICATION_JOB = "newEpisodesNotificationJob"; private static final String ADD_NEW_PODCAST_JOB = "addNewPodcastJob"; public static void main(String[] args) throws BeansException, JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException, JobParametersInvalidException, InterruptedException { Log log = LogFactory.getLog(Application.class); SpringApplication app = new SpringApplication(Application.class); app.setWebEnvironment(false); ConfigurableApplicationContext ctx= app.run(args); JobLauncher jobLauncher = ctx.getBean(JobLauncher.class); if(ADD_NEW_PODCAST_JOB.equals(args[0])){ //addNewPodcastJob Job addNewPodcastJob = ctx.getBean(ADD_NEW_PODCAST_JOB, Job.class); JobParameters jobParameters = new JobParametersBuilder() .addDate("date", new Date()) .toJobParameters(); JobExecution jobExecution = jobLauncher.run(addNewPodcastJob, jobParameters); BatchStatus batchStatus = jobExecution.getStatus(); while(batchStatus.isRunning()){ log.info("*********** Still running.... **************"); Thread.sleep(1000); } log.info(String.format("*********** Exit status: %s", jobExecution.getExitStatus().getExitCode())); JobInstance jobInstance = jobExecution.getJobInstance(); log.info(String.format("********* Name of the job %s", jobInstance.getJobName())); log.info(String.format("*********** job instance Id: %d", jobInstance.getId())); System.exit(0); } else if(NEW_EPISODES_NOTIFICATION_JOB.equals(args[0])){ JobParameters jobParameters = new JobParametersBuilder() .addDate("date", new Date()) .addString("updateFrequency", args[1]) .toJobParameters(); jobLauncher.run(ctx.getBean(NEW_EPISODES_NOTIFICATION_JOB, Job.class), jobParameters); } else { throw new IllegalArgumentException("Please provide a valid Job name as first application parameter"); } System.exit(0); } } The best explanation for SpringApplication-, @ComponentScan- and @EnableAutoConfiguration-magic you get from the source – Getting Started – Creating a Batch Service: “The main() method defers to the SpringApplication helper class, providing Application.class as an argument to its run() method. This tells Spring to read the annotation metadata from Application and to manage it as a component in the Spring application context. The @ComponentScan annotation tells Spring to search recursively through theorg.podcastpedia.batchpackage and its children for classes marked directly or indirectly with Spring’s @Component annotation. This directive ensures that Spring finds and registers BatchConfiguration, because it is marked with @Configuration, which in turn is a kind of @Component annotation. The @EnableAutoConfiguration annotation switches on reasonable default behaviors based on the content of your classpath. For example, it looks for any class that implements the CommandLineRunner interface and invokes its run() method.” Execution construction steps: the JobLauncher, which is a simple interface for controlling jobs, is retrieved from the ApplicationContext. Remember this is automatically made available via the@EnableBatchProcessing annotation. now based on the first parameter of the application (args[0]), I will retrieve the correspondingJob from the ApplicationContext then the JobParameters are prepared, where I use the current date - .addDate("date", new Date()), so that the job executions are always unique. once everything is in place, the job can be executed: JobExecution jobExecution = jobLauncher.run(addNewPodcastJob, jobParameters); you can use the returned jobExecution to gain access to BatchStatus, exit code, or job name and id. Note: I highly recommend you read and understand the Meta-Data Schema for Spring Batch. It will also help you better understand the Spring Batch Domain objects. 6.1. Running the application on dev and prod environments To be able to run the Spring Batch / Spring Boot application on different environments I make use of the Spring Profiles capability. By default the application runs with development data (database). But if I want the job to use the production database I have to do the following: provide the following environment argument -Dspring.profiles.active=prod have the production database properties configured in the application-prod.properties file in the classpath, right besides the default application.properties file Summary In this tutorial we’ve learned how to configure a Spring Batch project with Spring Boot and Java configuration, how to use some of the most common readers in batch processing, how to configure some simple jobs, and how to start Spring Batch jobs from a main method. Note: As I mentioned, I am fairly new to Spring Batch, and especially to Spring Boot and Spring Configuration with Java, so if you see any potential for improvement (code, job design etc.) please make a pull request or leave a comment below. Thanks a lot.

Security is one of the most important features while running a JBoss server in a production environment. Implementing SSL and securing communications is a must do, to avoid malicious use. This blogs details the steps you could take to secure JBoss EAP 6 running in Domain mode. These are probably documented by RedHat but the documentation seems a bit scattered. The idea behind this blog is to put together everything in one place. In Order to enhance security in JBoss EAP 6, SSL/encryption can be implemented for the following Admin console access – enable https access for admin console Domain Controller – Host controller communication – Communication between the main domain controller and all the other host controllers should be secured. Jboss CLI – enable ssl for the command line interface The below example uses a single keystore being both the key and truststore and also uses CA signed certificates. You could use self-signed certificates and/or separated keystores and truststores if required. Create the keystores (certificates for each of the servers) keytool -genkeypair -alias testServer.prd -keyalg RSA -keysize 2048 -validity 730 -keystore testServer.prd.jks Generate a certificate signing request (CSR) for the Java keystore keytool -certreq -alias testServer.prd -keystore testServer.prd.jks -file testServer.prd.csr Get the CSR signed by the Certificate Authorities Import a root or intermediate CA certificate to the existing Java keystore keytool -import -trustcacerts -alias root -file rootCA.crt -keystore testServer.prd.jks Import the signed primary certificate to the existing Java keystore. Keytool -importcert -keystore testServer.prd.jks -trustcacerts -alias testServer.prd -file testServer.prd.crt Repeat steps 1-6 for each of the servers. In order to establish trust between the master and slave hosts, Import the signed certificates of all the (slave) servers that the Domain Controller must trust onto the Domain Controllers Keystore keytool -importcert -keystore testServer.prd.jks -trustcacerts -alias slaveServer.prd -file slaveServers.prd.crt repeat step for all slave hosts. Import the signed certificate of the Domain controller onto the slave hosts keytool -importcert -keystore slaveServer.prd.jks -trustcacerts -alias testServer.prd -file testServer.prd.crt repeat steps for all slave hosts This has be to done because (as per RedHat’s Documentation) There is a problem with this methodology when trying to configure one way SSL between the servers, because there the HC's and the DC (depending on what action is being performed) switch roles (client, server). Because of this one way SSL configuration will not work and it is recommended that if you need SSL between these two endpoints that you configure two way SSL Once this is done, we now have signed certificates loaded onto the java keystore. In Jboss EAP 6 , the http-interface which provides access to the admin console, by default uses the ManagementRealm to provide file based authentication. (mgmt.-users.properties).The next step is to modify the configurations in the host.xml, to make the ManagementRealm use the certificates we created above. The host.xml should be modified to look like: view source print? 01. 02. 03. 04. 05. 06. 07. 08. 09. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. On the Slave hosts, In addition to the above configuration, the following needs to be changed view source print? 1. 2. 3. " 4. 5. Once you make the above changes and restart the servers, you should be able to access the admin console via https. https://testServer.prd:9443/console Finally, in order to secure cli authentication Modify /opt/jboss/jboss-eap-6.1/bin/jboss-cli.xml for each server and add view source print? 01. 02. 03. testServer.prd 04. 05. /opt/jboss/jboss-eap-6.1/domain/configuration/testServer.prd.jks 06. 07. xxxx 08. 09. /opt/jboss/jboss-eap-6.1/domain/configuration/testServer.prd.jks 10. 11. xxxx 12. 13. true 14. 15.

Graphene is pure carbon in the form of a very thin, nearly transparent sheet, one atom thick. It is remarkably strong for its very low weight and it conducts heat and electricity with great efficiency. Wikipedia If you have spent any time at all working in an Agile software development environment, you've heard the mantra to split your Stories as thin as you possibly can while still delivering value. This is indeed great advice, but the term "thin" is relative - our notion of what thin means is anchored by our previous experience! To help communicate what I mean when I say that Stories should be thinly sliced, I'm going to provide examples from a recent client who was building a relatively standard system for entering orders from their wholesale customers. To start, here's a very simplified model of the entities we'll be working with (using GML - the Galactic Modeling Language). In plain language, the system creates Orders, which have a Customer and from 1 to some number of Products. For initial planning, the team (consisting of developers, QA person, business analyst and product owner) used Jeff Patton's Story Mapping technique to determine what work the salespeople did at a relatively high level. Not surprisingly, the group determined that creating an Order was the most fundamental part of the system! So we started with a simple sticky note with the title Create Order. Note that there wasn't any other descriptive text, no acceptance criteria, no indication of performance requirements, or even mention of how someone using the system would actually create the order. At this point, we simply didn't need all that - we just needed a placeholder for the conversations that were required for the action of creating orders. Everyone in the group knew that an order would need a customer and some number of products. When I asked what the first Story should be, reminding them that it should be very, very thin, the first suggestion was to have the user select a customer from a list, select products from a list, and save the order. I asked if it was possible to make that story even thinner, and received puzzled looks. I followed with another question - why did we need to select a customer at this point, or even products for that matter? More puzzled looks. Couldn't we just hard-code a customer ID and a single product SKU? More puzzled looks. Another question - if we presented a simple page with a button that when pressed created the order in our database with a hard-coded customer ID and a single, hard-coded product SKU, what percentage of the overall system infrastructure would be fleshed out. That was the point at which people started to understand. The response I received was, "Almost all of it... we need to render a page, capture the submit action, write the order to an Orders table, the product to the Order Lines table and the customer & product to the Customers and Products tables respectively". My response was, "Great!! But it's still a little too thick." More puzzled looks... how could that story be made any thinner? I asked, "Why do we need tables for Customers and Products right now?" More puzzled looks, followed by a sheepish, "Um, because...". The need to create those tables was an assumption at that point. Everyone in the room, myself included, knew that the database would have Customers and Products tables at some point, but we just didn't need them right now. All that was required from a persistence perspective was the Orders and Order Lines tables. Someone asked, "How do we know that the Customer ID and Product SKU are valid?" My response was, "We don't! And at this point, we don't care!" This is another way to split stories - assume the happy path where everything is valid, and build validation and error-handling later. So, this very thin story became, The acceptance criteria for the story were very simple as well: When you navigate to the page index.html, it should be displayed with a Submit button When the Submit button is clicked, the application creates one row in the Order table with a Customer ID of XXX, and one row in the Order Line table with a Product ID of YYY. Note that I could have gone even further and argued that we didn't even need the database yet. We could have simply used a flat file and broken all the rules of relational data modeling by putting all the Order and Order Line data in a single line in the file. Why? BECAUSE WE DIDN'T NEED IT YET! However, in the interest of establishing the overall architecture early, we decided to proceed with the database. It did add some complexity to the story, but it was on the order of hours rather than days or weeks. We also "gold-plated" the story to an extent by providing a rudimentary confirmation page that was displayed after submitting the Order. After the group created this one, simple, thin story with hard-coded values, the next steps were to gradually replace the hard-coding with the selections and searches that you would expect in this type of application. Subsequent stories were, You can now see the progression where the creation of an Order is gradually built out using very thin slices of functionality. Also note that the last two stories, which "wire up" the selection of Customers and Products to an external web service, could be implemented in either order - there is nothing in the story that indicates a dependency on anything else. Splitting Techniques In these stories we used several common splitting techniques: Hard coding values; Simple interface; Defer validation; Zero, then One, then Many (with the Products); Defer complexity. This is only a small handful of the available techniques, but it gave the group a powerful way to work in very, very small increments while still delivering some value with each one. We Can't Ship That!! The moment I suggested the first story with all the hard-coding, I was challenged by the group. Why would you want to build something that isn't production-worthy? The answer is that some aspects of the feature aren't production worthy when you look at the story in isolation. However, the whole concept of a Minimal Viable Product or Minimal Viable Feature is each consists of 1 to many smaller slices (stories in our case) that have been built together to provide enough value to warrant the cost of being put into production. The Product Owner or Product Manager is the person who decides when enough value has accrued to meet that threshold. The Benefits of Thin Slicing I've provide a number of examples from a real-world project but haven't really dug into the reasons why this is a good approach. There are a number of them: You are deferring design decisions as long as possible which gives you more options when it does come time to implement a Story; Having options also allows the Product Owner/Manager to defer business decisions as long as possible, providing the ability to make changes to the product that better suit the actual needs of the business; Thin slices (in conjunction with acceptance criteria) help prevent assumptions which can lead to bloated Stories that are seemingly never done, Agile's version of scope creep; Thin slices are much easier to verify since they are only providing a small increment of functionality beyond what has already been implemented; Thin slices allow a product to be reviewed sooner by both the Product Owner/Manager and users/stakeholders, such that their feedback can be incorporated sooner in the process; Thin slices allow the Product Owner/Manager decides to change or drop a Story without the burden of the Sunk Cost Fallacy. While these are all great reasons to split Stories into thin slices, here's the one that my experience suggests is by far the most important: With Stories sliced as thinly as possible, they all become nearly the same size in terms of effort. Therefore, no estimation of individual Stories is required and a team's capacity can simply be derived by counting the number of Stories completed for some unit of time. That means no haggling over whether a Story is a 3 or 5 on the modified Fibonacci series many teams use for sizing Stories. It means that the need for spending any time estimating individual Stories is no longer required, an activity that is downright painful for some groups. The group from which these examples came was in the situation where there was a hard release date to be met. This stories gave the Product Owner/Manager the ability to easily tweak the what was and wasn't going to be in that initial release and, in several cases which features could be released partially completed. If a group is in the situation where they are being asked for an estimate of when some work could be completed, my experience has also been that Story Mapping coupled with a technique called Blink Estimation provides a high-level estimate of the overall work that's much more grounded in reality. Once a group begins working on thinly-sliced Stories, you begin to have data points with respect to how many they can complete per some unit of time. Those real numbers can then be used to make better business decisions, and deliver a product without the pressure of Crunch Time. Conclusion The group with whom I worked for this example system never spent any time estimating individual Stories. Ever. There were some instances where a Story was larger than it first appeared, but those were either balanced out by Stories that were very small, or by further splitting. Like the graphene example at the beginning of the post, thin stories have remarkable properties far beyond the fact that they are just "thin". The value gained by learning how to split Stories effectively is enormous owing to the flexibility it provides by deferring decisions as long as possible and the removal of the need to estimate at a granular level. Splitting Stories this thin isn't really a specialized skill - it's something that can be learned by anyone in any domain. If you need help learning this technique, let's have a chat.

in this post i will describe how to enable monitoring of a remote mule instance using jmx. in addition i will also enable the mx4j web interface that will expose the jmx properties of the mule instance in a web application and i will install the jolokia mule agent, which makes it possible to use hawtio to monitor the mule instance. enabling jmx for a mule instance using a mule application as of writing this, the finest granularity for which jmx monitoring can be enabled is an entire mule instance. thus it is not possible to enable jmx monitoring per application basis in a mule instance. this of course has advantages and disadvantages and we are going to use this fact to our advantage by deploying an application to a mule server that does nothing but enable jmx monitoring for the server. create a directory named “mulejmxenabler”. in this new directory, create a file named “mule-config.xml” with the following contents: in the above file, replace the ip address “192.168.1.73” with either the external ip address or the dns name of the computer running the mule server that you want to monitor. optionally, you may change the port number 1096 to any port that you rather use. deploy the application by copying the directory mulejmxenabler with its contents to the apps directory of the mule instance that is to be monitored. verify that the application was successfully deployed in the log of the mule instance (mule.log if you are using the community edition, mule_ee.log if you are using the enterprise edition). you should see something like this: info 2014-08-24 16:24:24,009 [mule.app.deployer.monitor.1.thread.1] org.mule.module.launcher.muledeploymentservice: ================== new exploded application: mulejmxenabler info 2014-08-24 16:24:24,010 [mule.app.deployer.monitor.1.thread.1] org.mule.module.launcher.application.defaultmuleapplication: ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + new app 'mulejmxenabler' + ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ info 2014-08-24 16:24:24,668 [mule.app.deployer.monitor.1.thread.1] org.mule.module.launcher.muledeploymentservice: ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + started app 'mulejmxenabler' + ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ the above is the minimum configuration that i need to enable remote jmx access to a mule instance. try connect to the mule instance using some jmx monitoring application, like jvisualvm or java mission control . the following steps describe how to do when using java mission control: launch java mission control. right-click in the jvm browser pane and select new connection. in the host field, enter the ip address or dns name that you used earlier in the mule-config.xml file. in my case that will be “192.168.1.73”. in the port field, enter the port number that you also entered in the mule-config.xml file earlier. in my case it was “1096”. click the test connection button. the status should be reported as ok as in this picture. click the finish button. you are now ready to connect to the mbean server of the mule instance and start monitoring it. this approach has the advantage of allowing modifications to the jmx configuration of a mule instance without having to restart it. if i want to make any modifications to the jmx configuration, enable or disable jmx in the mule instance i just edit the mule-config.xml file in the apps/mulejmxenabler directory in the mule instance home and save it. if hot deployment has not been disabled, the mule server will automatically re-deploy the application and pick up the changes. enabling jmx for a mule instance using wrapper parameters one alternative for enabling jmx monitoring of a mule instance that i have had success with is to add a set of java vm parameters in the wrapper configuration file of the mule instance. the wrapper configuration file is named “wrapper.conf” and can be found in the conf directory of the mule instance. to enable remote jmx access, add the following additional jvm parameters to your wrapper.conf file: # enables remote jmx management without authentication or ssl over port 1096. wrapper.java.additional.4=-dcom.sun.management.jmxremote wrapper.java.additional.5=-dcom.sun.management.jmxremote.port=1096 wrapper.java.additional.6=-dcom.sun.management.jmxremote.authenticate=false wrapper.java.additional.7=-dcom.sun.management.jmxremote.ssl=false wrapper.java.additional.8=-djava.rmi.server.hostname=192.168.1.73 important notes! you must adjust the numbering of the additional jmv parameters, as described in the comments in the wrapper.conf file! thus 4, 5, 6, 7 and 8 may not be the correct numbers for your wrapper.conf file. you must change the ip address to the remote ip address of the computer on which the mule server is running. you may want to change the port number. the mule instance must be restarted after the modifications, in order for them to come into effect. the drawback with this approach is that the mule instance needs to be restarted each time there is a modification to the jmx-related configuration parameters. with the target computer running mac os x, i need to chose one of the two described ways to enable remote jmx monitoring – if i use wrapper parameters, the mulejmxenabler application will not start properly. on ubuntu i can chose either of the two approaches but was also able to use both approaches simultaneously without errors. enabling the mx4j web interface if you are using the first approach, enabling jmx using a mule application, then adding the following xml element to the mule-config.xml file as a child element of the element enables the mx4j web interface exposing the jmx beans of the mule instance in question: as before, the ip address needs to be changed to the external ip address of the computer on which the mule instance is run. the port number may be changed from 1100 to any free port that you rather use. the mx4j web interface may, in my case, then be accessed using the url http://192.168.1.73:1100. install the jolokia mule agent with the first approach to enable jmx in a mule instance described above, you can also enable jolokia – the jmx-to-http bridge which enables use of the hawtio – an extensible web-based management console for java applications. note! version 1.2.2 of the jolokia mule agent, which at the time of writing is the latest, does not work with mule 3.5 due to the jetty libraries having been updated from version 6 to version 8. in this example i used mule 3.4, which is the latest version of mule that still use the jetty 6 libraries, as far as i know. download the jolokia mule agent from http://www.jolokia.org/download.html move the agent jar file to the lib/opt directory in the mule instance that you want to monitor. add the following configuration as a child element of the element in the mule-config.xml of the mulejmxenabler application created earlier: the port number may be modified as desired. open the url http://192.168.1.73:1095/jolokia/ in a web browser. note that you have to modify the ip address to the remote ip address of the computer running the mule instance and the port to the port number entered in the configuration earlier. if the jolokia agent was successfully enabled, you should see something like this in your web browser: {"timestamp":1408898064,"status":200,"request":{"type":"version"},"value":{"protocol":"7.2","config":{"maxdepth":"5","maxobjects":"10000","historymaxentries":"10","agentid":"192.168.1.73-1277-41c62ae4-mule","agenttype":"servlet","debug":"false","debugmaxentries":"100"},"agent":"1.2.2","info":{"product":"jetty","vendor":"mortbay","version":"6.1.26"}} with the jolokia agent in place, we can now monitor the mule instance using hawtio: download the hawtio jar from http://hawt.io/getstarted/index.html launch hawtio using the command “java -jar hawtio-app-1.4.17.jar” (the name of the jar file needs to match that of the file you downloaded earlier). hawtio should open a web browser and display a welcome page. if not, try the url http://localhost:8080/hawtio/welcome click the connect button in the upper left corner of the hawtio webpage. click the remote button. on the right, enter the remote ip of the computer on which your mule instance is running in the host field. 192.168.1.73 in my case. enter the port, 1095 in my case, from the configuration file in the port field. make sure the use proxy checkbox is checked. click the connect to remote server button. a new window or tab should open in your web browser. click the dashboard button in the upper left corner. you should now see information about the target computer, such as cpu and memory usage. click the jmx button next to the dashboard button. here you find the regular jmx management features. click the threads button next to the jmx button. this tab shows information about the different threads running in the target jvm. final words there are more things to tweak, but i hope that this will get you started with remote jmx monitoring of a mule instance. from there you can start modifying the parameters, adding security etc according to your requirements. more information on mule jmx management can be found here: http://www.mulesoft.org/documentation/display/current/jmx+management the oracle webpage on jmx technology can be found here: http://www.oracle.com/technetwork/java/javase/tech/javamanagement-140525.html finally, the oracle jmx tutorial can be found here: http://docs.oracle.com/javase/tutorial/jmx/

1 Start with Personas The first step towards writing the right user stories is to understand your target users and customers. After all, user stories want to tell a story about the users using the product. If you don’t know who the users are and what problem we want to solve then it’s impossible to write the right stories and you end up with a long wish list rather than a description of the relevant product functionality. Personas offer a great way to capture the users and the customers with their needs. They are fictional characters that have a name and picture; relevant characteristics such as a role, activities, behaviours, and attitudes; and a goal, which is the problem that has to be addressed or the benefit that should be provided. Let’s look at an example. Say we want to create a game for children, which is fun to play and which educates the kids about music and dancing. We would then create at least two personas, one to represent the children, and one for the parents, as the following picture illustrates. The two sample personas above use my simple yet effective persona template. It encourages you to keep your personas concise, to focus on what really matters and to leave out the rest. You can download the template from romanpichler.com/tools/persona-template where more information on writing personas and using the template is available. Once you have created a cast of characters, select a primary persona, the persona you are mainly designing and building the product for. This helps you make the right product decision and get the user experience (UX) right. In the example above, I have chosen Yasmin as the primary persona. 2 Derive Epics from the Persona Goals Once you have created your personas, use their goals personas to identify the product functionality. Ask yourself what the product should do to address the personas’ problems or to create the desired benefits for them, as the following picture shows. Start with your primary persona and capture the functionality as epics, as coarse-grained, high-level stories. Write all the epics necessary to meet the persona goals but keep them rough and sketchy at this stage. For the dance game, we could write the epics below assuming that the game will be initially launched as an iPad app: As the epics above show, the game should allow the players to select different characters, to make them dance, to choose different dance floors and music tracks, to play the game with their friends, and to post a snapshot of their game on Facebook. While epics are great to sketch the product’s functionality, there is more to your product than epics and stories: You should also capture the user interaction and the sequences in which the epics are used, the visual design of your product, and the important nonfunctional qualities such as interoperability and performance. Use, for instance, workflow diagrams, story maps, storyboards, sketches, mock-ups, and constraint cards to describe them. You can find out more about describing the different product aspects in my post “User Stories are Not Enough to Create a Great User Experience”. 3 Progressively Decompose the Epics into User Stories With a holistic but coarse-grained description of your product in place start progressively decomposing your epics. Rather than detailing all epics and writing all user stories in one go, you derive your stories step by step as the following picture shows. As long as there are some significant risks present and you are figuring out what the product should look like and do, it’s best to derive just enough user stories just in time for the next sprint. Use your sprint goal or hypothesis to determine which epics to decompose and which stories to write as the following diagram illustrates. The approach depicted above minimises the amount of detailed items in your product backlog. This makes it easier to integrate new insights derived from exposing product increments or minimum viable products (MVPs) to users and customers. Say that we want to address the risk of creating the wrong game characters by developing an executable prototype that allows us to run a usability test with selected children. We could then write the following user stories: The stories above are derived from the epics “Choose character” and “Play with character”. The resulting prototype only partially implements the two epics – just to the extent of being able to test if the characters resonate with the users. Once you understand better how to meet the customer and user needs, you can start pre-writing user stories and have a larger inventory of detailed items on your product backlog as you are unlikely to experience bigger changes to your epics and your overall backlog. 4 Get the Stories Ready Before the development team starts working on the stories, check that each user story is ready: clear, feasible, and testable. A story is clear if there is a shared understanding between the product owner and the team about its meaning. It is feasible if it can be delivered in the next sprint according to the Definition of Done. This implies that the story is small enough to fit into the sprint but also that the necessary user interface design, test, and documentation work can be carried out. In the case of the sample stories above, we would have to add acceptance criteria, ensure that the stories are small enough to fit into the next sprint, and consider creating some very rough design sketches to indicate what the characters look like. For instance, to get the story “Yas chooses the little girl” ready, we could create the following rough sketch: The sketch above complement the user story and allows the team to implement the entire story including the visual design in the next sprint. With ready user stories in place the development team is in a good position to progress your product in an effective manner. For more details on getting user stories ready please take a look at my post “The Definition of Ready in Scrum”. Learn More You can learn more about writing the right epics and user stories by attending my Writing Great User Stories training course. If you want to learn more about the creating the UX artefacts mentioned in this post, then attend my Agile UX and Scrum training course. Please contact me if you are interested in having the courses delivered at your office. The persona pictures and the Manga girl sketch were created by Ole Størksen. Thanks Ole!

The objectives of Test Driven Development and unit testing are generally misunderstood. The problem is the word ‘test’, it is much less about testing and much more about specification of requirements, showing your working – as in maths, and the impact it has on design. TDD is much more important than only testing. Robert C Martin has a good analogy, he likens TDD to double entry bookkeeping: Software is a remarkably sensitive discipline. If you reach into a base of code and you change one bit you can crash the software. Go into the memory and twiddle one bit at random and very likely you will elicit some form of crash. Very, very few systems are that sensitive. You could go out to one of these bridges over here, start taking bolts out and they probably wouldn’t fall. I could pull out a gun and start shooting randomly and I probably wouldn’t kill too many people. I might wound a few but — you know — you get a bullet in the leg or a lung and you’d probably survive. People are resilient — they can survive the loss of a leg and so forth. Bridges are resilient — they survive the loss of components. But software isn’t resilient at all: one bit changes and — BANG! — it crashes. Very few disciplines are that sensitive. But there is one other [discipline] that is, and that’s Accounting. The right mistake at exactly the right time on the right spreadsheet — that one-digit error can crash the company and send the offenders off to jail. How do accountants deal with that sensitivity? Well, they have disciplines. And one of the primary disciplines is dual-entry bookkeeping. Everything is said twice. Every transaction is entered two times — once on the credit side and once on the debit side. Those two transactions follow separate mathematical pathways until they end up at this wonderful subtraction on the balance sheet that has to yield to zero. This is what test-driven development is: dual-entry bookkeeping. Everything is said twice — once on the test side and once on the production code side and everything runs in an execution that yields either a green bar or a red bar just like the zero on the balance sheet. It seems like that’s a good practice for us: to manage these sensitivities of our discipline… -Robert C. Martin The sensitivity of software is a good point to reflect upon, there is little in human experience that is so complex and yet so fragile. Without a strong focus on showing your working, no matter how good you are as a developer, if you omit the tests., your software will be worse than it could have been. The double-entry bookkeeping analogy only holds up though if you do test first development. If you write your test after the code it is generally not sufficiently independent to provide a valid “separate path” check. Test first is the idea that your write the test before you write the code that is being tested. This seems like a bizarre idea to many people at first, but actually makes perfect sense. If you write the test first and run it, you get to see it fail, so you are testing the test. If you write the test first then you are expressing what you want of your software from the outside in. It leads you to design for behaviour and so you have less of a tendency to get lost in irrelevant technicalities. This is a much more effective design approach than testing after you have written the code, and as a by product it leads inevitably to software that is easy to test – you have to be pretty dumb to write a test before you have written the code for an idea that can’t be tested! Finally there is a virtuous circle here. Software is easy to test when it is modular. It is easy to test when dependencies are externalised and it is easy to test when there is a clear separation of concerns. Now the software industry is famous for change, but if there is any idea that has remained constant for, literally, decades it is that quality software is modular, has well defined dependencies and clear separation of concerns – sound familiar? This has been how computer science has defined quality since before I started, and that was a very long time ago! Using TDD as a practice makes you produce higher quality software, not because it is well tested (though that is a nice by-product) but because it improves the quality of your designs. Want more detail: http://c2.com/cgi/wiki?TestDrivenDevelopment http://www.agiledata.org/essays/tdd.html http://butunclebob.com/ArticleS.UncleBob.TheThreeRulesOfTdd http://unitmm.sourceforge.net/fibonacci_example.shtml http://clean-cpp.org/test-driven-development/ http://agile2007.agilealliance.org/downloads/presentations/TDD-Cpp-Agile2007-HandsOnTddInCpp.ppt_801.pdf http://www.growing-object-oriented-software.com/