A recent Java production performance problem forced me to revisit and truly appreciate the Java VM Just-In-Time (JIT) compiler. Most Java developers and support individuals have heard of this JVM run time performance optimization but how many truly understand and appreciate its benefits? This article will share with you a troubleshooting exercise I was involved with following the addition of a new virtual server (capacity improvement and horizontal scaling project). For a more in-depth coverage of JIT, I recommend the following articles: ## Just-in-time compilation http://en.wikipedia.org/wiki/Just-in-time_compilation ## The Java HotSpot Performance Engine Architecture http://www.oracle.com/technetwork/java/whitepaper-135217.html ## Understanding Just-In-Time Compilation and Optimization http://docs.oracle.com/cd/E15289_01/doc.40/e15058/underst_jit.htm ## How the JIT compiler optimizes code http://pic.dhe.ibm.com/infocenter/java7sdk/v7r0/index.jsp?topic=%2Fcom.ibm.java.zos.70.doc%2Fdiag%2Funderstanding%2Fjit_overview.html JIT compilation overview The JIT compilation is essentially a process that improves the performance of your Java applications at run time. The diagram below illustrates the different JVM layers and interaction. It describes the following high level process: Java source files are compiled by the Java compiler into platform independent bytecode or Java class files. After your fire your Java application, the JVM loads the compiled classes at run time and execute the proper computation semantic via the Java interpreter. When JIT is enabled, the JVM will analyze the Java application method calls and compile the bytecode (after some internal thresholds are reached) into native, more efficient, machine code. The JIT process is normally prioritized by the busiest method calls first. Once such method call is compiled into machine code, the JVM executes it directlyinstead of “interpreting” it. The above process leads to improved run time performance over time. Case study Now here is the background on the project I was referring to earlier. The primary goal was to add a new IBM P7 AIX virtual server (LPAR) to the production environment in order to improve the platform’s capacity. Find below the specifications of the platform itself: Java EE server: IBM WAS 6.1.0.37 & IBM WCC 7.0.1 OS: AIX 6.1 JDK: IBM J2RE 1.5.0 (SR12 FP3 +IZ94331) @64-bit RDBMS: Oracle 10g Platform type: Middle tier and batch processing In order to achieve the existing application performance levels, the exact same hardware specifications were purchased. The AIX OS version and other IBM software’s were also installed using the same version as per existing production. The following items (check list) were all verified in order to guarantee the same performance level of the application: Hardware specifications (# CPU cores, physical RAM, SAN…). OS version and patch level; including AIX kernel parameters. IBM WAS & IBM WCC version, patch level; including tuning parameters. IBM JRE version, patch level and tuning parameters (start-up arguments, Java heap size…). The network connectivity and performance were also assessed properly. After the new production server build was completed, functional testing was performed which did also confirm a proper behaviour of the online and batch applications. However, a major performance problem was detected on the very first day of its production operation. You will find below a summary matrix of the performance problems observed. Production server Operation elapsed time Volume processed (# orders) CPU % (average) Middleware health Existing server 10 hours 250 000 (baseline) 20% healthy *New* server 10 hours 50 000 -500% 80% +400% High thread utilization As you can see from the above view, the performance results were quite disastrous on the first production day. Not only much less orders were processed by the new production server but the physical resource utilization such as CPU % was much higher compared with the existing production servers. The situation was quite puzzling given the amount of time spent ensuring that the new server was built exactly like the existing ones. At that point, another core team was engaged in order to perform extra troubleshooting and identify the source of the performance problem. Troubleshooting: searching for the culprit... The troubleshooting team was split in 2 in order to focus on the items below: Identify the source of CPU % from the IBM WAS container and compare the CPU footprint with the existing production server. Perform more data and file compares between the existing and new production server. In order to understand the source of the CPU %, we did perform an AIX CPU per Thread analysis from the IBM JVM running IBM WAS and IBM WCC. As you can see from the screenshot below, many threads were found using between 5-20% each. The same analysis performed on the existing production server did reveal fewer # of threads with CPU footprint always around 5%. Conclusion: the same type of business process was using 3-4 times more CPU vs. the existing production server. In order to understand the type of processing performed, JVM thread dumps were captured at the same time of the CPU per Thread data. Now the first thing that we realized after reviewing the JVM thread dump (Java core) is that JIT was indeed disabled! The problem was also confirmed by running the java –version command from the running JVM processes. This finding was quite major, especially given that JIT was enabled on the existing production servers. Around the same time, the other team responsible of comparing the servers did finally find differences between the environment variables of the AIX user used to start the application. Such compare exercise was missed from the earlier gap analysis. What they found is that the new AIX production server had the following extra entry: JAVA_COMPILER=NONE As per the IBM documentation, adding such environment variable is one of the ways to disableJIT. Complex root cause analysis, simple solution In order to understand the impact of disabling JIT in our environment, you have to understand its implication. Disabling JIT essentially means that the entire JVM is now running ininterpretation mode. For our application, running in full interpretation mode not only reduces the application throughput significantly but also increases the pressure point on the server CPU utilization since each request/thread takes 3-4 more times CPU than a request executed with JIT (remember, when JIT is enabled, the JVM will perform many calls to the machine/native code directly). As expected, the removal of this environment variable along with the restart of the affected JVM processes did resolve the problem and restore the performnance level. Assessing the JIT benefits for your application I hope you appreciated this case study and short revisit of the JVM JIT compilation process. In order to understand the impact of not using JIT for your Java application, I recommend that you preform the following experiment: Generate load to your application with JIT enabled and capture some baseline data such as CPU %, response time, # requests etc. Disable JIT. Redo the same testing and compare the results. I’m looking forward for your comments and please share any experience you may have with JIT.

In a previous Damn Cool Algorithms post, we learned about Fountain Codes, a clever probabilistic algorithm that allows you break a large file up into a virtually infinite number of small chunks, such that you can collect any subset of those chunks - as long as you collect a few more than the volume of the original file - and be able to reconstruct the original file. This is a very cool construction, but as we observed last time, it has one major flaw when it comes to use in situations with untrusted users, such as peer to peer networks: there doesn't seem to be a practical way to verify if a peer is sending you valid blocks until you decode the file, which happens very near the end - far too late to detect and punish abuse. It's here that Homomorphic Hashes come to our rescue. A homomorphic hash is a construction that's simple in principle: a hash function such that you can compute the hash of a composite block from the hashes of the individual blocks. With a construction like this, we could distribute a list of individual hashes to users, and they could use those to verify incoming blocks as they arrive, solving our problem. Homomorphic Hashing is described in the paper On-the-fly verification of rateless erasure codes for efficient content distribution by Krohn et al. It's a clever construction, but rather difficult to understand at first, so in this article, we'll start with a strawman construction of a possible homomorphic hash, then improve upon it until it resembles the one in the paper - at which point you will hopefully have a better idea as to how it works. We'll also discuss the shortcomings and issues of the final hash, as well as how the authors propose to resolve them. Before we continue, a small disclaimer is needed: I'm a computer scientist, not a mathematician, and my discrete math knowledge is far rustier than I'd like. This paper stretches the boundaries of my understanding, and describing the full theoretical underpinnings of it is something I'm likely to make a hash of. So my goal here is to provide a basic explanation of the principles, sufficient for an intuition of how the construction works, and leave the rest for further exploration by the interested reader. A homomorphic hash that isn't We can construct a very simple candidate for a homomorphic hash by using one very simple mathematical identity: the observation that gx0 * gx1 = gx0 + x1. So, for instance, 23 * 22 = 25. We can make use of this by the following procedure: Pick a random number g For each element x in our message, take gx. This is the hash of the given element. Using the identity above, we can see that if we sum several message blocks together, we can compute their hash by multiplying the hashes of the individual blocks, and get the same result as if we 'hash' the sum. Unfortunately, this construction has a couple of obvious issues: Our 'hash' really isn't - the hashes are way longer than the message elements themselves! Any attacker can compute the original message block by taking the logarithm of the hash for that block. If we had a real hash with collisions, a similar procedure would let them generate a collision easily. A better hash with modular arithmetic Fortunately, there's a way we can fix both problems in one shot: by using modular arithmetic. Modular arithmetic keeps our numbers bounded, which solves our first problem, while also making our attacker's life more difficult: finding a preimage for one of our hashes now requires solving the discrete log problem, a major unsolved problem in mathematics, and the foundation for several cryptosystems. Here, unfortunately, is where the theory starts to get a little more complicated - and I start to get a little more vague. Bear with me. First, we need to pick a modulus for adding blocks together - we'll call it q. For the purposes of this example, let's say we want to add numbers between 0 and 255, so let's pick the smallest prime greater than 255 - which is 257. We'll also need another modulus under which to perform exponentiation and multiplication. We'll call this p. For reasons relating to Fermat's Little Theorem, this also needs to be a prime, and further, needs to be chosen such that p - 1 is a multiple of q (written q | (p - 1), or equivalently, p % q == 1). For the purposes of this example, we'll choose 1543, which is 257 * 6 + 1. Using a finite field also puts some constraints on the number, g, that we use for the base of the exponent. Briefly, it has to be 'of order q', meaning that gq mod p must equal 1. For our example, we'll use 47, since47257 % 1543 == 1. So now we can reformulate our hash to work like this: To hash a message block, we compute gb mod p - in our example, 47b mod 1543 - where b is the message block. To combine hashes, we simply multiply them mod p, and to combine message blocks, we add them mod q. Let's try it out. Suppose our message is the sequence [72, 101, 108, 108, 111] - that's "Hello" in ASCII. We can compute the hash of the first number as 4772 mod 1543, which is 883. Following the same procedure for the other elements gives us our list of hashes: [883, 958, 81, 81, 313]. We can now see how the properties of the hash play out. The sum of all the elements of the message is 500, which is 243 mod 257. The hash of 243 is 47243 mod 1543, or 376. And the product of our hashes is883 * 958 * 81 * 81 * 313 mod 1543 - also 376! Feel free to try this for yourself with other messages and other subsets - they'll always match, as you would expect. A practical hash Of course, our improved hash still has a couple of issues: The domain of our input values is small enough that an attacker could simply try them all out to find collisions. And the domain of our output values is small enough the attacker could attempt to find discrete logarithms by brute force, too. Although our hashes are shorter than they were without modular arithmetic, they're still longer than the input. The first of these is fairly straightforward to resolve: we can simply pick larger primes for p and q. If we choose ones that are sufficiently large, both enumerating all inputs and brute force logarithm finding will become impractical. The second problem is a little trickier, but not hugely so; we just have to reorganize our message a bit. Instead of breaking the message down into elements between 0 and q, and treating each of those as a block, we can break the message into arrays of elements between 0 and q. For instance, suppose we have a message that is 1024 bytes long. Instead of breaking it down into 1024 blocks of 1 byte each, let's break it down into, say, 64 blocks of 16 bytes. We then modify our hashing scheme a little bit to accommodate this: Instead of picking a single random number as the base of our exponent, g, we pick 16 of them, g0 - g16. To hash a block, we take each number gi and raise it to the power of the corresponding sub-block. The resulting output is the same length as when we were hashing only a single block per hash, but we're taking 16 elements as input instead of a single one. When adding blocks together, we add all the corresponding sub-blocks individually. All the properties we had earlier still hold. Better, we've given ourselves another tuneable parameter: the number of sub blocks per block. This will be invaluable in getting the right tradeoff between security, granularity of blocks, and protocol overhead. Practical applications What we've arrived at now is pretty much the construction described in the paper, and hopefully you can see how it would be applied to a system utilizing fountain codes. Simply pick two primes of about the right size - the paper recommends 257 bits for q and 1024 bits for p - figure out how big you want each block to be - and hence how many sub-blocks per block - and figure out a way for everyone to agree on the random numbers for g - such as by using a random number generator with a well defined seed value. The construction we have now, although useful, is still not perfect, and has a couple more issues we should address. First of these is one you may have noticed yourself already: our input values pack neatly into bytes - integers between 0 and 255 in our example - but after summing them in a finite field, the domain has grown, and we can no longer pack them back into the same number of bits. There are two solutions to this: the tidy one and the ugly one. The tidy one is what you'd expect: Since each value has grown by one bit, chop off the leading bit and transmit it along with the rest of the block. This allows you to transmit your block reasonably sanely and with minimal expansion in size, but is a bit messy to implement and seems - at least to me - inelegant. The ugly solution is this: Pick the smallest prime number larger than your chosen power of 2 for q, and simply ignore or discard overflows. At first glance this seems like a terrible solution, but consider: the smallest prime larger than 2256 is 2256 + 297. The chance that a random number in that range is larger than 2256 is approximately 1 in 3.9 * 1074, or approximately one in 2247. This is way smaller than the probability of, say, two randomly generated texts having the same SHA-1 hash. Thus, I think there's a reasonable argument for picking a prime using that method, then simply ignoring the possibility of overflows. Or, if you want to be paranoid, you can check for them, and throw out any encoded blocks that cause overflows - there won't be many of them, to say the least. Performance and how to improve it Another thing you may be wondering about this scheme is just how well it performs. Unfortunately, the short answer is "not well". Using the example parameters in the paper, for each sub-block we're raising a 1024 bit number to the power of a 257 bit number; even on modern hardware this is not fast. We're doing this for every 256 bits of the file, so to hash an entire 1 gigabyte file, for instance, we have to compute over 33 million exponentiations. This is an algorithm that promises to really put the assumption that it's always worth spending CPU to save bandwidth to the test. The paper offers two solutions to this problem; one for the content creator and one for the distributors. For the content creator, the authors demonstrate that there is a way to generate the random constants g, used as the bases of the exponents using a secret value. With this secret value, the content creator can generate the hashes for their files much more quickly than without it. However, anyone with the secret value can also trivially generate hash collisions, so in such a scheme, the publisher must be careful not to disclose the value to anyone, and only distribute the computed constants gi. Further, the set of constants themselves aren't small - with the example parameters, a full set of constants weighs in at about the size of 4 data blocks. Thus, you need a good way to distribute the per-publisher constants in addition to the data itself. Anyone interested in this scheme should consult section C of the paper, titled "Per-Publisher Homomorphic Hashing". For distributors, the authors offer a probabilistic check that works on batches of blocks, described in section D, "Computational Efficiency Improvements". Another easier to understand variant is this: Instead of verifying blocks individually as they arrive, accumulate blocks in a batch. When you have enough blocks, sum them all together, and calculate an expected hash by taking the product of the expected hashes of the individual blocks. Compute the composite block's hash. If it verifies, all the individual blocks are valid! If it doesn't, divide and conquer: split your batch in half and check each, winnowing out valid blocks until you're left with any invalid ones. The nice thing about either of these procedures is that they allow you to trade off verification work with your vulnerability window. You can even dedicate a certain amount of CPU time to verification, and simply batch up incoming blocks until the current computation finishes, ensuring you're always verifying the last batch as you receive the next. Conclusion Homomorphic Hashing provides a neat solution to the problem of verifying data from untrusted peers when using a fountain coding system, but it's not without its own drawbacks. It's complicated to implement and computationally expensive to compute, and requires careful tuning of the parameters to minimise the volume of the hash data without compromising security. Used correctly in conjunction with fountain codes, however, Homomorphic Hashing could be used to create an impressively fast and efficient content distribution network. As a side-note, I'm intending to resume more regular blogging with more Damn Cool Algorithms posts. Have an algorithm you think is Damn Cool and would like to hear more about? Post it in the comments!

Introduction The purpose of this article is to see how to integrate Spring Security on top of an Openxava standalone application. Openxava builds portlets as well as standalone applications. When working with portlets deployed on a portal such as Liferay, they handle secured access by configuration. A standalone application lets you have to handle this functionality yourself. This page will illustrate how to add spring security (authentication/authorisation) functionalities. The focus will be on the authorizations aspects since authorization is often enterprise-environment specific. To demonstrate the integration, this article will use the minuteproject Lazuly showcase application generated for Openxava. The first part identifies and explains the actions to undertake. The second part explains what minuteproject can do to fasten your development by generated a customed spring-security integration for you Openxava application. Eventually a set of tests will ensure that the resulting application is correctly protected for URL direct access as well as content display. Furthermore, the integration is technologically non-intruisive. You do not have to change Openxava code for it to work. Spring-Security Openxava integration Technical Access URL access The url pattern is the following http://servername:port/applicationcontext/xava/module.jsp?application=appName&module=moduleName given like that it is hard to protect. The module and application are passed as parameters. The URL has to be revisited with http://servername:port/applicationcontext/applicationPath/module And the 'parameter' access are banned. Enabling new URL access Add a servlet package net.sf.minuteproject.openxava.web.servlet; import java.io.*; import javax.servlet.*; import javax.servlet.http.*; public class ModuleHomeServlet extends HttpServlet { protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { RequestDispatcher dispatcher; String [] uri = request.getRequestURI().split("/"); if (uri.length < 4) { dispatcher = request.getRequestDispatcher("/xava/homeMenu.jsp"); } else { dispatcher = request.getRequestDispatcher( "/xava/home.jsp?application=" + uri[1] + "&module=" + uri[3]); } dispatcher.forward(request, response); } protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { doGet(request, response); } } homeMenu.jsp is a page including a header with menu (to protect and whose menu link URL are correspond to the secured format) and a footer. Add a servlet configuration Servlet configuration snippet done in Openxava servlets.xml. moduleHome net.sf.minuteproject.openxava.web.servlet.ModuleHomeServlet moduleHome /MenuModules/* This snippet will be package in war web.xml at build time by OpenXava ant script. Jsp access Prohibit any Openxava jsp access except the one of the menu To do that add an spring applicationContext-security.xml in you classpath (ex: Openxava src folder). ... This means that all path after xava will be accessible (ex: css...) safe jsp expect one homeMenu.jsp is available to all registered user (ie having role ROLE_APPLICATION_USER cf attribution at authorisation part further). Of course ensure that the role ROLE_NOT_PRESENT is really not present in your app. Business Access The idea is to give CRUD access on a entity base on role. Define roles and UC To be more explicit, I define 3 roles with their scope. Administrator can administrate ROLE and COUNTRY entities Application_user can manage all the other conference related tables safe the master data table mentionned above. Reviewer can access to the statistic views but not the administration. Both reviewer and Administrator can do what Application_user can do. In applicationContext-security.xml the role can be mapped to specific URLs Impact of the roles access on your model modal navigation Be coherent As I said before 'the CRUD access on a entity is role based' but the affectation mechanism has to reflect that. OpenXava has annotation to create an entity from another one. It is then logical that we cannot create entity B from entity A, if we do not have CRUD rights on entity B. The mechanism will consist in this case of affectation only with search functionalities. In our scenario it means that a user with 'application_user' only can select a country but can not create any (no create or update icons available). It is also true at the menu level, a user is entitled to see only its menu items corresponding to its profile. Here the menu is done in JSP. To secure the access you can wrap to code to secure with taglib code coming with spring security or add a little taglib such as the following isUserInRole.tag located in web/WEB-INF/tags/common. Wrap the code to protect here the administrator menu and each menu item Administration CountryRole Authentication/Authorization For the user to operate, he must be authenticated and authorised (moment where his role profile is loaded granting him with business access rights). I use an simple authentication and authorisation based a DB information. Of course you are not supposed to use that in production ;) In applicationContext-security.xml add the following snippet. java:comp/env/jdbc/conferenceDS Both authorisation and authentication queries have to be valid. Here, they are done on top of views, which means that you have to implement 2 views: user_authentication and user_authorisation. The datasource is the same as the one of the Openxava application View gives you flexibility because if you have indirection level of granularity such as (user-role-permission), your view can associate user to role Authentication flow Eventually you need to handle an authentication flow composed of welcome page login page access denied page logout link The flow is handled by applicationContext-security.xml Add the following snippet. Login.jsp is strongly inspired by spring petclinic sample Login test Locale is: Your login attempt was not successful, try again. Reason: . User:Password:Don't ask for my password for two weeks index.jsp Welcome to Conference login accessDenied.jsp Access denied! Not to forget a logout functionality here added on the menu Logoff Spring security dependencies Add spring security jars into web/WEB-INF/lib spring-aop-3.0.4.RELEASE.jar spring-asm-3.0.4.RELEASE.jar spring-beans-3.0.4.RELEASE.jar spring-context-3.0.4.RELEASE.jar spring-core-3.0.4.RELEASE.jar spring-expression-3.0.4.RELEASE.jar spring-jdbc-3.0.4.RELEASE.jar spring-security-acl-2.0.3.jar spring-security-config-3.1.0.M1.jar spring-security-core-2.0.3.jar spring-security-core-3.1.0.M1.jar spring-security-core-tiger-2.0.3.jar spring-security-taglibs-2.0.3.jar spring-security-web-3.1.0.M1.jar spring-tx-3.0.4.RELEASE.jar spring-web-3.0.4.RELEASE.jar Spring security context Spring security context had been mentioned at different level, here is the complete version java:comp/env/jdbc/conferenceDS Reference the context Openxava listeners.xml is the place where you can set web.xml-snippets to be package in web.xml at Openxava build time Add the following snippet org.springframework.web.context.ContextLoaderListener contextConfigLocation classpath:applicationContext-security.xml springSecurityFilterChain org.springframework.web.filter.DelegatingFilterProxy springSecurityFilterChain /* The minuteproject way Doing the integration can be time consuming. As you can notice there is some effort to have the code compliant for a webapp here Openxava to be bodyguard by Spring-Security. Meanwhile when dealing with data centric application, this knowledge can be crystalized to be instantly available. Because...there is an underlying concept that guides our choice and lead to best pratices. It is one thing to execute them, it is another to state it. The question is how do we specify which entity to access and to which role. The idea is to express with simplicity the relationship between role or permission and action. In our case the actions are: a full CRUD an affectation mechanism The full CRUD is associated to a specific role. The affection (linkage of an entity from another by search) is when to entities are linked but not all the role of the main entities are the same as the roles of the target. Otherwise affection goes with creation and update. And the roles are: Administrator Application_user Reviewer Now it is time for a primary school exercice If you represent an entity-relationship diagram, you should see boxes and links. Boxes for entities and links for relationships. Give each role/permission a color. Paint all the boxes that are full CRUD with the corresponding role color... Yes, you may paint the same box twice (resulting is color combination). The result gives you the Color access spectrum of your DB. Of course, we can further decline the gradient with other function (read-only, controller specific...) But the underlying idea is evident. What Minuteproject allows you to do it by enriching your model with this color spectrum at the entity level or at the package level. This enables you to work with concept only closed to UC agnostic of technology implementations. Minuteproject configuration snippet Generation Minuteproject configuration full The configuration is similar to lazuly show case enhanced with security aspects org.gjt.mm.mysql.Driver jdbc:mysql://127.0.0.1:3306/conference root mysql The main points are exclude entities starting with user_ (i.e. the security entity used by spring configuration) add security access on package level package admin is accessible by role administrator only package statistics is accessible by role reviewer only default package (conference) is accessible by any application_user add spring-security track in the target add reference in openxava to spring-security The track springsecurity holding the configuration is not yet bundled in minuteproject release 0.8 but will be present for 0.8.1+. Set up Database Implement the views Here a very dummy implementation. create view user_authentication as select email as username, first_name as password, '1' as active from conference_member ; create view user_authorisation as select cm.email as username, r.name as role from conference_member cm, role r, member_role mr where mr.role_id = r.id and mr.conference_member_id = cm.id union select cm.email as username, concat('ROLE_',r.name) as role from conference_member cm, role r, member_role mr where mr.role_id = r.id and mr.conference_member_id = cm.id ; As you can not there is a little redundancy in the user_authentication view, since sometimes the role administrator is refered sometimes role_administrator. This will be homogenized in next release. Add some default value Here a very dummy implementation. INSERT INTO country (id, name, iso_name) VALUES (-1, 'France', 'FR'); INSERT INTO address (id, street1, street2, country_id) VALUES(-1, 'rue 1', 'rue 2', -1); INSERT INTO conference_member (id, conference_id, first_name, last_name, email, address_id, status ) VALUES (-1, -1, 'f', 'a', '[email protected]', -1, 'ACTIVE' ); INSERT INTO role (id, name) VALUES (-1, 'ADMINSTRATOR' ); INSERT INTO role (id, name) VALUES (-2, 'ROLE_APPLICATION_USER' ); INSERT INTO member_role (conference_member_id, role_id) VALUES (-1, -1); INSERT INTO member_role (conference_member_id, role_id) VALUES (-1, -2); So when user [email protected] connects he will get the role Administrator which allows him to access the administrator menu and create a new role called 'REVIEWER'. He can also create a new conference member and associate with the role 'REVIEWER'. Set up Application Download the lazuly-openxava-springsecurity minuteproject configuration from google code minuteproject. Copy file into /mywork/config Execute In /mywork/config: model-generation.cmd mp-config-LAZULY-Openxava-with-spring-security.xml The generated code goes to /DEV/output/openxava-springsecurity/conference Packaging Here the packaging/deployment is a 2 steps exercices (unfortunately): there is no more the start-tomcat/stop-tomcat command in OX distribution spring dependencies are not included Steps Check that Openxava 4.3 is available, and OX_HOME is set to Openxava 4.3 from /DEV/output/openxava-springsecurity/conference run build-conference(.cmd/sh). This will trigger the build that is successful but not the deployment due to information before. Open the project generated by the build in Openxava workspace Add Spring security dependencies Start tomcat server (remark: The Datasource for the application is present in tomcat/config/context.xml) Deploy Enjoy Testing Welcome page Default URL at context root of the application. Login page Any other direct called where the user is not authenticated will be intercepted and routed to this page Contextual Menu The user have access to the admin and conference part not the statistics. The URLs have been modified. When the user tries to access the standard OX style URL he recieves an access denied (ex: module.jsp) Add role reviewer Add user Affect user with role reviewer and default (application_user) Logoff (click logoff) Login as Reviewer On login page enter [email protected] and password=b In the contextual menu you do see the 'admin' package' And you get an access deny when manipulating directly the URL Now the application is secured. Conclusion This article showed the configuration and manipulation to integrate spring security with openxava in a non-intrusive manner. It stressed a new concept 'DB color access spectrum' and how to densify the security information in minuteproject configuration. DB color access spectrum is a concept which ask only to be extended: Ad-hoc functions, controllers Store procedures It is simple to express and analyst friendly. It is not bound to a technology. It is a step in easily defining fine grain access, its combination with profile based access and state based access (to do manually... for the moment ;)) could pave the way to intuitive and implicit workflows instead of heavy BPM solutions.

Strategy Pattern is one of the patterns from the Design Patterns : Elements of Reusable Object book. The intent of the strategy pattern as stated in the book is: Define a family of algorithms, encapsulate each one, and make them interchangeable. Strategy lets the algorithm vary independently from clients that use it. In this post I would like to give an example or two on strategy pattern and then rewrite the same example using lambda expressions to be introduced in Java 8. Strategy Pattern: An example Consider an interface declaring the strategy: interface Strategy{ public void performTask(); } Consider two implementations of this strategy: class LazyStratgey implements Strategy{ @Override public void performTask() { System.out.println("Perform task a day before deadline!"); } } class ActiveStratgey implements Strategy{ @Override public void performTask() { System.out.println("Perform task now!"); } } The above strategies are naive and I have kept it simple to help readers grasp it quickly. And lets see these strategies in action: public class StartegyPatternOldWay { public static void main(String[] args) { List strategies = Arrays.asList( new LazyStratgey(), new ActiveStratgey() ); for(Strategy stg : strategies){ stg.performTask(); } } } The output for the above is: Perform task a day before deadline! Perform task now! Strategy Pattern: An example with Lambda expressions Lets look at the same example using Lambda expressions. For this we will retain our Strategy interface, but we need not create different implementation of the interface, instead we make use of lambda expressions to create different implementations of the strategy. The below code shows it in action: import java.util.Arrays; import java.util.List; public class StrategyPatternOnSteroids { public static void main(String[] args) { System.out.println("Strategy pattern on Steroids"); List strategies = Arrays.asList( () -> {System.out.println("Perform task a day before deadline!");}, () -> {System.out.println("Perform task now!");} ); strategies.forEach((elem) -> elem.performTask()); } } The output for the above is: Strategy pattern on Steroids Perform task a day before deadline! Perform task now! In the example using lambda expression, we avoided the use of class declaration for different strategies implementation and instead made use of the lambda expressions. Strategy Pattern: Another Example This example is inspired from Neal Ford’s article on IBM Developer works: Functional Design Pattern-1. The idea of the example is exactly similar, but Neal Ford uses Scala and I am using Java for the same with a few changes in the naming conventions. Lets look at an interface Computation which also declares a generic type T apart from a method compute which takes in two parameters. interface Computation { public T compute(T n, T m); } We can have different implementations of the computation like: IntSum – which returns the sum of two integers, IntDifference – which returns the difference of two integers and IntProduct – which returns the product of two integers. class IntSum implements Computation { @Override public Integer compute(Integer n, Integer m) { return n + m; } } class IntProduct implements Computation { @Override public Integer compute(Integer n, Integer m) { return n * m; } } class IntDifference implements Computation { @Override public Integer compute(Integer n, Integer m) { return n - m; } } Now lets look at these strategies in action in the below code: public class AnotherStrategyPattern { public static void main(String[] args) { List computations = Arrays.asList( new IntSum(), new IntDifference(), new IntProduct() ); for (Computation comp : computations) { System.out.println(comp.compute(10, 4)); } } }public class AnotherStrategyPattern { public static void main(String[] args) { List computations = Arrays.asList( new IntSum(), new IntDifference(), new IntProduct() ); for (Computation comp : computations) { System.out.println(comp.compute(10, 4)); } } } The output for the above is: 14 6 40 Strategy Pattern: Another Example with lambda expressions Now lets look at the same example using Lambda expressions. As in the previous example as well we need not declare classes for different implementation of the strategy i.e the Computation interface, instead we make use of lambda expressions to achieve the same. Lets look at an example: public class AnotherStrategyPatternWithLambdas { public static void main(String[] args) { List> computations = Arrays.asList( (n, m)-> { return n+m; }, (n, m)-> { return n*m; }, (n, m)-> { return n-m; } ); computations.forEach((comp) -> System.out.println(comp.compute(10, 4))); } } The output for above is 14 6 40 From the above examples we can see that using Lambda expressions will help in reducing lot of boilerplate code to achieve more concise code. And with practice one can get used to reading lambda expressions.

in-line predicates can create a maintenance nightmare. writing in-line lambda expressions and using the stream interfaces to perform common operations on collections can be awesome. assume the following example: list getadultmales (list persons) { return persons.stream().filter(p -> p.getage() > adult && p.getsex() == sexenum.male ).collect(collectors.tolist()); } that’s fun! but things like this also lead to software that is costly to maintain. at least in an enterprise application, where most of your code handles business logic, your development team will grow the tenancy to write the same similar set of predicate rules again and again. that is not what you want on your project. it breaks three important principles for growing maintainable and stable enterprise applications: dry (don’t repeat yourself): writing code more than once is not a good fit for a lazy developer it also makes your software more difficult to maintain because it becomes harder to make your business logic consistent readability : following clean-code best practices, 80% of writing code is reading the code that already exists. having complicated lambda expressions is still a bit hard to read compared to a simple one-line statement. testability : your business logic needs to be well-tested. it is adviced to unit-test your complex predicates. and that is just much easier to do when you separate your business predicate from your operational code. and from a personal point of view… that method still contains too much boilerplate code… imports to the rescue! fortunately, we have a very good suggestion in the world of unit testing on how we could improve on this. imagine the following example: import static somepackage.personpredicate; ... list getadultmales (list persons) { return persons.stream().filter( isadultmale() ).collect(collectors.tolist()); } what we did here was: create a personpredicate class define a “factory” method that creates the lambda predicate for us statically import the factory method into our old class this is how such a predicate class could look like, located next to your person domain entity: public personpredicate { public static predicate isadultmale() { return p -> p.getage() > adult && p.getsex() == sexenum.male; } } wait… why don’t we just create a “ismaleadult” boolean function on the person class itself like we would do in domain driven development? i agreed, that is also an option… but as time goes on and your software project becomes bigger and loaded with functionality and data… you will again break your clean code principles: the class becomes bloated with all kind of function and conditions your class and tests become huge, more difficult to handle and change (*) (*) and yes… even if you do your best to separate your concerns and use composition patterns adding some defaults… working with domain objects, we can imagine that some operations (such as filter) are often executed on domain entities. taking that into account, it would make sense to let our entities implement some interface that offers us some default methods. for example: public interface domainoperations { default list filter(predicate predicate) { return persons.stream().filter( predicate ) .collect(collectors.tolist()); } } when our person entity implements this interface, we can clean-up our code even more: list getadultmales (list persons) { return persons.filter( isadultmale() ); } and there we go… conclusion moving your predicates to a predicate helper class offers some good advantages in the long run: predicate classes are easy to test and change your domain objects remain clean and focussed on representing your domain, not your business logic you optimize the re-usability of your code and, in the end, reduce your maintenance you seperate your business from operational concerns references clean code: a handbook of agile software craftsmanship [robert c. martin] practical unit testing with junit and mockito [tomek kaczanowski] state of the collections [http://cr.openjdk.java.net/~briangoetz/lambda/collections-overview.html] notes the code above is served as an example to illustrate the principles i wanted to discuss. however, i did not proof-run this code yet (it’s still on my todo list). some modifications may be needed for your project.

In a previous post, I described Babylon.js, a brand new 3D engine for WebGL and JavaScript. Among others features, Babylon.js is capable of loading a JSON file through the .babylon file format. During this post, I will show you how to use Babylon.js API to load a scene created with Blender. Creating a scene and exporting a .babylon file with Blender In my previous post, I already described how to install the .babylon exporter in Blender, but for the sake of comprehension, I copy/paste the process here: First of all, please download the exporter script right here: http://www.babylonjs.com/Blender2Babylon.zip. To install it in Blender, please follow this small guide: Unzip the file to your Blender’s plugins folder (Should be C:\Program Files\Blender Foundation\Blender\2.67\scripts\addons for Blender 2.67 x64). Launch Blender and go to File/User Préférences/Addon and select Import-Export category. You will be able to activate Babylon.js exporter. Create your scene Go to File/Export and select Babylon.js format. Choose a filename and you are done ! Once the exporter is installed, you can unleash your artist side and create the most beautiful scene your imagination can produce. In my case, it will be fairly simple: A camera A point light A plane for the ground A sphere Just to be something a bit less austere, I will add some colors for the ground and the sphere: I will also add a texture for the sphere. This texture will be used for the diffuse channel of the material: Please pay attention to: Use Alpha checkbox to indicate to Babylon.js to use alpha values from the texture Color checkbox to indicate that this texture must be use for diffuse color Once you are satisfied (You can obviously create a more complex scene), just go to File/Export/Babylon.js to create your .babylon file. Loading your .babylon Inside your page/app First of all, you should create a simple html web page: This page is pretty simple because all you need is just a canvas and a reference to babylon.js. Then you will have to use BABYLON.SceneLoader object to load your scene. To do so, just add this script block right after the canvas: the Load function takes the following parameters: scene folder (can be empty to use the same folder as your page) scene file name a reference to the engine a callback to give you the loaded scene (in my case, I use this callback to attach the camera to the canvas and to launch my render loop) a callback for progress report Once the scene is loaded, just wait for the textures and shaders to be ready, connect the camera to the canvas and let’s go! Fairly simple, isn’t it? Please note that the textures and the .babylon file must be side by side Another function is also available to interact with .babylon files: BABYLON.SceneLoader.importMesh: BABYLON.SceneLoader.ImportMesh("spaceship", "Scenes/SpaceDek/", "SpaceDek.babylon", scene, function (newMeshes, particleSystems) { }); This function is intended to import meshes (with their materials and particle systems) from a scene to another. It takes the following parameters: object name (if you omit this parameter, all the objects are imported) scene folder (can be empty to use the same folder as your page) scene file name a reference to the target scene a callback to give you the list of imported meshes and particle systems Playing with your scene The result is as expected: a orange plane lighted by a point light with a floating sphere using an RGBA texture for its diffuse color. You can use the mouse and the cursors keys to move: &lt;br&gt; For IE11 preview, you can also directly try the result just here: http://www.babylonjs.com/tutorials/blogs/loadScene/loadscene.html The full source code is also available there: http://www.babylonjs.com/tutorials/blogs/loadScene/loadScene.zip Enjoy! Others chapters If you want to go more deeply into babylon.js, here are some useful links: Introducing Babylon.js: http://blogs.msdn.com/b/eternalcoding/archive/2013/06/27/babylon-js-a-complete-javascript-framework-for-building-3d-games-with-html-5-and-webgl.aspx How to load a scene exported from Blender: http://blogs.msdn.com/b/eternalcoding/archive/2013/06/28/babylon-js-how-to-load-a-babylon-file-produced-with-blender.aspx

A lot of times you need to to assign some functionality to some keyboard shortcut like F5 or Ctrl+R in your application. JavaFx also provides KeyCodeCombination API for handling multiple key events. scene.setOnKeyPressed(new EventHandler() { public void handle(final KeyEvent keyEvent) { if (keyEvent.getCode() == KeyCode.F5) { System.out.println("F5 pressed"); //Stop letting it do anything else keyEvent.consume(); } } }); final KeyCombination keyComb1 = new KeyCodeCombination(KeyCode.R, KeyCombination.CONTROL_DOWN); scene.addEventHandler(KeyEvent.KEY_RELEASED, new EventHandler() { @Override public void handle(KeyEvent event) { if (keyComb1.match(event)) { System.out.println("Ctrl+R pressed"); } } });

This technical tip shows how to Add, Delete & Get Attachment in a PDF Document using Aspose.Pdf for Java. In order to add attachment in a PDF document, you need to create a FileSpecification object with the file, which needs to be added, and the file description. After that the FileSpecification object can be added to EmbeddedFiles collection of Document object using add(..) method of EmbeddedFiles collection. The attachments of the PDF document can found in the EmbeddedFiles collection of the Document object. In order to delete all the attachments, you only need to call the delete(..) method of the EmbeddedFiles collection and then save the updated file using save method of the Document object. //Add attachment in a PDF document. //open document com.aspose.pdf.Document pdfDocument = new com.aspose.pdf.Document("input.pdf"); //setup new file to be added as attachment com.aspose.pdf.FileSpecification fileSpecification = new com.aspose.pdf.FileSpecification("sample.txt", "Sample text file"); //add attachment to document's attachment collection pdfDocument.getEmbeddedFiles().add(fileSpecification); // Save updated document containing table object pdfDocument.save("output.pdf"); //Delete all the attachments from the PDF document. //open document com.aspose.pdf.Document pdfDocument = new com.aspose.pdf.Document("input.pdf"); //delete all attachments pdfDocument.getEmbeddedFiles().delete(); //save updated file pdfDocument.save("output.pdf"); //Get an individual attachment from the PDF document. //open document com.aspose.pdf.Document pdfDocument = new com.aspose.pdf.Document("input.pdf"); //get particular embedded file com.aspose.pdf.FileSpecification fileSpecification = pdfDocument.getEmbeddedFiles().get_Item(1); //get the file properties System.out.printf("Name: - " + fileSpecification.getName()); System.out.printf("\nDescription: - " + fileSpecification.getDescription()); System.out.printf("\nMime Type: - " + fileSpecification.getMIMEType()); // get attachment form PDF file try { InputStream input = fileSpecification.getContents(); File file = new File(fileSpecification.getName()); // create path for file from pdf file.getParentFile().mkdirs(); // create and extract file from pdf java.io.FileOutputStream output = new java.io.FileOutputStream(fileSpecification.getName(), true); byte[] buffer = new byte[4096]; int n = 0; while (-1 != (n = input.read(buffer))) output.write(buffer, 0, n); // close InputStream object input.close(); output.close(); } catch (IOException e) { e.printStackTrace(); } // close Document object pdfDocument.dispose();

In this blog post, I will show you how to "ETL" all kinds of data to Amazon’s cloud data warehouse Redshift wit Talend’s big data components. Let’s begin with a short introduction to Amazon Redshift (copied from website): "Amazon Redshift is [part of Amazon Web Services (AWS) and] a fast and powerful, fully managed, petabyte-scale data warehouse service in the cloud. With a few clicks in the AWS Management Console, customers can launch a Redshift cluster, starting with a few hundred gigabytes and scaling to a petabyte or more, for under $1,000 per terabyte per year. Traditional data warehouses require significant time and resource to administer, especially for large datasets. In addition, the financial cost associated with building, maintaining, and growing self-managed, on-premise data warehouses is very high. Amazon Redshift not only significantly lowers the cost of a data warehouse, but also makes it easy to analyze large amounts of data very quickly.“ Sounds interesting! And indeed, we already see companies using Talend’s Redshift connectors. From Talend perspective it is not much more than just another database. If you have ever used a Talend connector, you can integrate to Redshift within some minutes. In the next sections, I will describe all necessary steps and give some hints regarding configuration issues and performance improvements. Be aware: You need Talend Open Studio for Data Integration (Apache License, open source) or any Talend Enterprise Edition / Platform which contains the Cloud components to see and use Amazon Redshift connectors. The open source edition offers all connectors and functionality to integrate with Amazon Redshift. However, Enterprise versions offer some more features (e.g. versioning), comfort (e.g. wizards) and commercial support. Setup Amazon Redshift Setup of Amazon Redshift is very easy. Just follow Amazon‘s getting started guide: http://docs.aws.amazon.com/redshift/latest/gsg/welcome.html. Like every other AWS guide, it is very easy to understand and use. Be aware, that you just have to do step 1, 2 and 3 of the getting started guide for using it with Talend. Some hints: - Step 1 („before you begin“): Just sign up. Client tools and drivers are not necessary because they are already installed within Talend Studio. - Step 2 („launch a cluster“): Yes, please start your cluster! - Step 3(„authorize access“): If you are not sure what to do here, select Connection Type = CIDR/IP. Find out your IP address (http://whatismyipaddress.com) and enter it with „/32“ at the end. Example: „192.168.1.1/32“ Now you can connect to Amazon Redshift from your Talend Studio on your local computer. Step 4 (connect) and step 5 (create table, data, queries) are not necessary, this will be done from Talend Studio. Of course, you should not forget to delete your cluster (step 7) when you are done. Otherwise, you will pay for every hour, even if you do not access your DWH. Connect to Amazon Redshift from Talend Studio Create a new connection to Amazon Redshift database as you do with every other relational database. The easiest way is to use „DB Connection Wizard“ in metadata. Just enter your connection information and check if it works. You get all information about configuration from Amazon Web Console. The connection string looks something like this: „jdbc:paraccel://talend-demo-cluster.cp8t6c5.eu-west-1.redshift.amazonaws.com:5439/dev“ Next, right click on the created connection and select „retrieve schema“. „public“ is the default schema which you (have to) use. Now, you are ready to use this connection within Talend Jobs to write to Amazon Redshift and read from it. Create Talend Jobs (Write, Read, Delete) Amazon Redshift components work like any other Talend (relational) database components. Look at www.help.talend.com for more information if you have not used them before (or just try them out, they are very self-explanatory). You just have to drag&drop your connection from metadata . Afterwards, you can easily write data (tRedShiftOutput), read data (tRedshiftInput), or do any other queries such as delete or copy (tRedShiftRow). In the following job, I start with deleting all content in the Amazon Redshift table. Then, I read data from a MySQL table and insert it into an Amazon Redshift table. The table is created automatically (as I have configured it this way). After this subjob is finished, I read the data again, and store it to a CSV file (which is also created automatically). Of course, this is no business use case, but it shows how to use different Amazon Redshift components. Query Data from Amazon Redshift You can connect to Amazon Redshift directly from Talend Studio to explore and query data of the DWH. Thus, no other database tool is required. Just right click on your Amazon Redshift connection in metadata and select „edit queries“. Here you can define, execute and save SQL queries. Improve Performance Write performance of Amazon Redshift is relatively low compared to „classical“ relational databases (in your data center) as you have to upload all data into the cloud. Different alternatives exist to improve performance: - Bulk inserts: „Extended insert“ (in advanced settings) improves performance a lot, but still not to hyperspeed… Also, as it is bulk, you can just do inserts! It is not compatible to „rejects“ or „updates“ - AWS S3 and COPY command: S3 is Amazon’s „simple storage service“, a key-value store – also called NoSQL today – for storing very large objects. You can use Amazon Redshift’s COPY command (http://docs.aws.amazon.com/redshift/latest/dg/r_COPY.html) to transfer data from S3 to Amazon Redshift with good performance. Though, you still have to copy data to S3 before, same „cloud problem“ here. The COPY command can be used with tRedshiftRow, so no problem at all from Talend perspective. To transfer data to S3, you can either use the Talend S3 components from Talendforge, Talend’s open source community (http://www.talendforge.org/exchange), or use camel-s3, an Apache Camel component which is included in Talend ESB. The latter is an option, if you use Talend Data Services which combines Talend DI and Talend ESB in its unified platform. Summary You need not be a cloud or DWH expert, or an expert developer to integrate with Amazon’s cloud data warehouse Redshift. It is very easy with Talend’s integration solutions. Just drag&drop, configure, do some graphical mappings / transformations (if necessary), that’s it. Code is generated. Job runs. You can integrate Amazon Redshift almost as simple as any other relational database. Just be aware of some cloud specific security and performance issues. With Talend, you can easily „ETL“ all data from different sources to Redshift and store it there for under $1,000 per terabyte per year – even with the open source version! Best regards, Kai Wähner (Contact and feedback via @KaiWaehner, www.kai-waehner.de, LinkedIn / Xing) This is content from my blog: http://www.kai-waehner.de/blog/2013/06/26/integration-of-amazon-redshift-cloud-data-warehouse-aws-saas-dwh-with-talend-data-integration-di-big-data-bd-enterprise-service-bus-esb/

My team has recently started a new Java web application project and we picked gradle as our build tool. Most of us were extremely familiar with maven, but decided to give gradle a try. Today I had to figure out how to do resource filtering in gradle. And to be honest it wasn't as easy as I thought it should be; as least coming from a maven background. I eventually figured it out, but wanted to post my solution to make it easier for others. What is Resource Filtering? First, for those that may not know, what is resource filtering? It's basically a way to avoid hard coding values in files and make them more dynamic. For example, I may want to display the version of my application in my application. The version is usually defined in your build file and this value can be injected or replaced in your configuration file during assembly. So I could have a file called config.properties under src/main/resources with the following content: application.version=${application.version}. With resource filtering the ${application.version} value gets replaced with 1.0.0 during assembly, then my application can load config.properties and display the application version. It's an extremely valuable and powerful feature in build tools like maven and one that I took advantage of often. Resource Filtering in Gradle With this being my first gradle project, I needed to find the recommended way to enable resource filtering in gradle. My first problem I had to figure out was where to define the property. In maven this would typically be defined in the project's pom.xml file as a maven property: 1.0.0 For gradle the appropriate place seemed to be the project's gradle.properties file. So you would add the following to your project's gradle.properties file (Note, I'm not suggesting you would hardcode the modules version in the gradle.properties file. Obviously the value would be derived from the version property in your project. I'm just using this for a simple example): application.version=1.0.0 The next, and most difficult, problem I had to track down was how to actually enable resource filtering. I was hoping to just set some enableFiltering option and define the includes/excludes list, but that doesn't seem to be the case (extra tip: don't do filtering on binary files like images). I did find some resources online, but this one seemed to be the best approach. So you will need to add the following to your build.gradle file: import org.apache.tools.ant.filters.* processResources { filter ReplaceTokens, tokens: [ "application.version": project.property("application.version") ] } Next you need to update your resource file. So put a config.properties file under src/main/resources and add this: [email protected]@ Note, the use of @ instead of ${}. This is because gradle is based on ant, and ant by default uses the @ character as the token identifier whereas maven uses ${}. Finally, if you build your project you can look under build/resources/main and you should see a config.properties file with a value of 1.0.0. You can also open up your artifact and see the same result. Dot notation One thing to note is I typically use a period or dot to separate words for properties: application.version instead of applicationVersion. So you will notice the surrounding quotes around "application.version" in the build.gradle file. This is required as failing to surround the key by quotes will fail the build. Probably because groovy's dynamic nature thinks you are traversing an object. Overriding I also investigated the best approach to overriding properties in gradle, as this appeared to be slightly different then how it's done in maven. In maven, properties can be overridden by properties defined in the user's setting.xml file or on the command line with the -D option. To override application.version in gradle on the command line I had to run the following: gradle assemble -Papplication.version=2.0.0 If you want to override it for all projects you can add the property in your gradle.properties file under /user_home/.gradle. Also, if you are overriding the value via the command line and your property value contains special characters like a single quote, you can wrap the value with double quotes like the following to get it to work: gradle assemble -Papplication.version="2.0.0'6589" Summary Well I hope this helps and if anyone from the gradle community sees a better way to perform resource filtering I'd love to hear about it. I'd also like to see something as important as resource filtering becoming easier to perform in gradle. I think it's crazy having to add an import statement to perform something so simple.

Hi, I'm excited to announce the release of our QuartzDesk product. QuartzDesk is an advanced Java Quartz scheduler management and monitoring GUI / tool with many powerful and unique features. To name just a few: Support for Quartz 1.x and 2.x schedulers. Persistent job execution history. Job execution log message capturing. Notifications (email, all popular IM protocols, web-service). Interactive execution statistics and charts. REST API for job / trigger / scheduler monitoring. QuartzAnywhere web-service to manage / monitor Quartz schedulers from applications. and more To keep this announcement short, I kindly refer you to the QuartzDesk Features page for details and screenshots. The product is aimed at Java developers and system administrators. Jan Moravec (Founder) & The QuartzDesk Team

I was recently working a bit with Twitter’s Storm, and it got me wondering, how does it compare to another high-performance, concurrent-data-processing framework, Akka. WHAT’S AKKA AND STORM? Let’s start with a short description of both systems. Storm is a distributed, real-time computation system. On a Storm cluster, you execute topologies, which process streams of tuples (data). Each topology is a graph consisting of spouts (which produce tuples) and bolts (which transform tuples). Storm takes care of cluster communication, fail-over and distributing topologies across cluster nodes. Akka is a toolkit for building distributed, concurrent, fault-tolerant applications. In an Akka application, the basic construct is an actor; actors process messages asynchronously, and each actor instance is guaranteed to be run using at most one thread at a time, making concurrency much easier. Actors can also be deployed remotely. There’s a clustering module coming, which will handle automatic fail-over and distribution of actors across cluster nodes. Both systems scale very well and can handle large amounts of data. But when to use one, and when to use the other? There’s another good blog post on the subject, but I wanted to take the comparison a bit further: let’s see how elementary constructs in Storm compare to elementary constructs in Akka. COMPARING THE BASICS Firstly, the basic unit of data in Storm is a tuple. A tuple can have any number of elements, and each tuple element can be any object, as long as there’s a serializer for it. In Akka, the basic unit is amessage, which can be any object, but it should be serializable as well (for sending it to remote actors). So here the concepts are almost equivalent. Let’s take a look at the basic unit of computation. In Storm, we have components: bolts andsprouts. A bolt can be any piece of code, which does arbitrary processing on the incoming tuples. It can also store some mutable data, e.g. to accumulate results. Moreover, bolts run in a single thread, so unless you start additional threads in your bolts, you don’t have to worry about concurrent access to the bolt’s data. This is very similar to an actor, isn’t it? Hence a Storm bolt/sprout corresponds to an Akka actor. How do these two compare in detail? Actors can receive arbitrary messages; bolts can receive arbitrary tuples. Both are expected to do some processing basing on the data received. Both have internal state, which is private and protected from concurrent thread access. ACTORS & BOLTS: DIFFERENCES One crucial difference is how actors and bolts communicate. An actor can send a message to any other actor, as long as it has the ActorRef (and if not, an actor can be looked up by-name). It can also send back a reply to the sender of the message that is being handled. Storm, on the other hand is one-way. You cannot send back messages; you also can’t send messages to arbitrary bolts. You can also send a tuple to a named channel (stream), which will cause the tuple (message) to be broadcast to all listeners, defined in the topology. (Bolts also ack messages, which is also a form of communication, to the ackers.) In Storm, multiple copies of a bolt’s/sprout’s code can be run in parallel (depending on theparallelism setting). So this corresponds to a set of (potentially remote) actors, with a load-balancer actor in front of them; a concept well-known from Akka’s routing. There are a couple of choices on how tuples are routed to bolt instances in Storm (random, consistent hashing on a field), and this roughly corresponds to the various router options in Akka (round robin, consistent hashing on the message). There’s also a difference in the “weight” of a bolt and an actor. In Akka, it is normal to have lots of actors (up to millions). In Storm, the expected number of bolts is significantly smaller; this isn’t in any case a downside of Storm, but rather a design decision. Also, Akka actors typically share threads, while each bolt instance tends to have a dedicated thread. OTHER FEATURES Storm also has one crucial feature which isn’t implemented in Akka out-of-the-box: guaranteed message delivery. Storm tracks the whole tree of tuples that originate from any tuple produced by a sprout. If all tuples aren’t acknowledged, the tuple will be replayed. Also the cluster management of Storm is more advanced (automatic fail-over, automatic balancing of workers across the cluster; based on Zookeeper); however the upcoming Akka clustering module should address that. Finally, the layout of the communication in Storm – the topology – is static and defined upfront. In Akka, the communication patterns can change over time and can be totally dynamic; actors can send messages to any other actors, or can even send addresses (ActorRefs). So overall, Storm implements a specific range of usages very well, while Akka is more of a general-purpose toolkit. It would be possible to build a Storm-like system on top of Akka, but not the other way round (at least it would be very hard).

During any sort of programming you will always get situations where you need to compare values with each other, if the values are boolean or integers then the comparison is simple. But if you want to compare strings or parts of strings then there can be more to the comparison such as case of the string you are comparing. In this tutorial we are going to look at all the different ways you can compare strings in PHP using a number of built in PHP functions. == operator The most common way you will see of comparing two strings is simply by using the == operator if the two strings are equal to each other then it returns true. if('string1' == 'string1') { echo ' Strings match. '; } else { echo ' Strings do not match. '; } This code will return that the strings match, but what if the strings were not in the same case it will not match. If all the letters in one string were in uppercase then this will return false and that the strings do not match. if('string1' == 'STRING1') { echo ' Strings match. '; } else { echo ' Strings do not match. '; } This means that we can't use the == operator when comparing strings from user inputs, even if the first letter is in uppercase it will still return false. So we need to use some other function to help compare the strings. strcmp Function Another way to compare strings is to use the PHP function strcmp, this is a binary safe string comparison function that will return a 0 if the strings match. if(strcmp('string1', 'string1') == 0) { echo ' Strings match. '; } else { echo ' Strings do not match. '; } This if statement will return true and echo that the strings match. But this function is case sensitive so if one of the strings has an uppercase letter then the function will not return 0. strcasecmp Function The previous examples will not allow you to compare different case strings, the following function will allow you to compare case insensitive strings. if(strcasecmp('string1', 'string1') == 0) { echo ' Strings match. '; } else { echo ' Strings do not match. '; } if(strcasecmp('string1', 'String1') == 0) { echo ' Strings match. '; } else { echo ' Strings do not match. '; } if(strcasecmp('string1', 'STRING1') == 0) { echo ' Strings match. '; } else { echo ' Strings do not match. '; } All of these if statements will return that the strings match, which means that we can use this function when comparing strings that are input by the user.

"Chart JS Library" provides API for drawing different charts. Drawing is based on HTML CANVAS Element. Download Link:- http://www.chartjs.org/ In this Demo, "We will draw a Radar Chart .The Student input data is JSON in nature.The Servlet returns the JSON data when called by Jquery Ajax method.The Student Java class object is converted to JSON representation using GSON Library". The Java web project structure, The Student Servlet StudentJsonDataServlet.java , package com.sandeep.chartjs.servlet; import java.io.IOException; import java.util.ArrayList; import java.util.List; import javax.servlet.ServletException; import javax.servlet.annotation.WebServlet; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import com.google.gson.Gson; import com.sandeep.chartjs.data.Student; @WebServlet("/StudentJsonDataServlet") public class StudentJsonDataServlet extends HttpServlet { private static final long serialVersionUID = 1L; public StudentJsonDataServlet() { super(); } protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { List listOfStudent = getStudentData(); Gson gson = new Gson(); String jsonString = gson.toJson(listOfStudent); response.setContentType("application/json"); response.getWriter().write(jsonString); } private List getStudentData() { List listOfStudent = new ArrayList(); Student s1 = new Student(); s1.setName("Sandeep"); s1.setComputerMark(75); s1.setMathematicsMark(26); s1.setGeographyMark(91); s1.setHistoryMark(55); s1.setLitratureMark(36); listOfStudent.add(s1); return listOfStudent; } } The HTML markup chartjs-demo.html, The java script file for radar chart ts-chart-script.js, var TUTORIAL_SAVVY ={ /*Makes the AJAX calll (synchronous) to load a Student Data*/ loadStudentData : function(){ var formattedstudentListArray =[]; $.ajax({ async: false, url: "StudentJsonDataServlet", dataType:"json", success: function(studentJsonData) { console.log(studentJsonData); $.each(studentJsonData,function(index,aStudent){ formattedstudentListArray.push([aStudent.mathematicsMark,aStudent.computerMark,aStudent.historyMark,aStudent.litratureMark,aStudent.geographyMark]); }); } }); return formattedstudentListArray; }, /*Crate the custom Object with the data*/ createChartData : function(jsonData){ console.log(jsonData); return { labels : ["Mathematics", "Computers", "History","Literature", "Geography"], datasets : [ { fillColor : "rgba(255,0,0,0.3)", strokeColor : "rgba(0,255,0,1)", pointColor : "rgba(0,0,255,1)", pointStrokeColor : "rgba(0,0,255,1)", /*As Ajax response data is a multidimensional array, we have 'student' data in 0th position*/ data : jsonData[0] } ] }; }, /*Renders the Chart on a canvas and returns the reference to chart*/ renderStudenrRadarChart:function(radarChartData){ var context2D = document.getElementById("canvas").getContext("2d"), myRadar = new Chart(context2D). Radar(radarChartData,{ scaleShowLabels : false, pointLabelFontSize : 10 }); return myRadar; }, /*Initalization Student render chart*/ initRadarChart : function(){ var studentData = TUTORIAL_SAVVY.loadStudentData(); chartData = TUTORIAL_SAVVY.createChartData(studentData); radarChartObj = TUTORIAL_SAVVY.renderStudenrRadarChart(chartData); } }; $(document).ready(function(){ TUTORIAL_SAVVY.initRadarChart(); }); The response Json data format for student, The Firebug console shows DOM Element, The output in browser will look like,This radar chart shows the a student('sandeep') marks in different subject,

cdi (context and dependency injection) is a complete and lightweight injection technology designed for java ee environment. special container objects (ejb,entitymanager), primitive data type elements and java class/objects written by you can be easily managed and injected as well through cdi. every defined java class in each application that configured in cdi standard is a candidate to become an injectable cdi object. this default behavior is provided by @default annotation that was installed per each java class secretly. there is an car class which has a vehicle implementation in the above uml diagram. a random int value is produced in sayvelocity() method and there is an output to the console such as “the car is running at the speed of x” in work() method, where x is represents a number produced randomly. @default // optional public class car implements vehicle { public string work() { return "car is working in "+ sayvelocity()+" kmh."; } public int sayvelocity(){ return threadlocalrandom.current().nextint(20, 240) ; } } if the above car class is in an application activated in cdi environment, it becomes a candidate to be an object managed by cdi. so, what is implied by the activation of cdi? first of all, necessary dependencies need to be included in the classpath for the activation of cdi environment. if you are using an application server like glassfish, cdi can be used without any extra definition because of the existence of cdi libraries on the application server. but if your application is a java se application or is running in lightweight containers such as tomcat, jetty; a cdi library must be added to the project. the reference library of cdi technology is the jboss weld archetype. for this reason, if jboss weld dependencies are added to the project such as the following, first phase of the cdi activation is realized. org.jboss.weld.se weld-se 1.1.10.final to activate the cdi environment, a blank cdi configuration file named beans.xml must be in the application as a requirement of the standard. this file must be located on the /meta-inf/beans.xml path for java se applications and /web-inf/beans.xml path for java ee web applications. the existence necessity of this file may sound silly initially, but the existence of beans.xml in the directories specified above can be considered as a permission given to the container in order to activate cdi environment. activation of cdi environment is automatically started in java ee web applications when beans.xml file is encountered in the /web-inf directory. but it is not the same for a java se application, starting a cdi container is the developer’s job. this case can be seen clearly if you pay attention to the following gallery class: public class gallery { @inject // injection point private vehicle vehicle; public static void main(string[] args) { weld weld = new weld(); weldcontainer weldcontainer = weld.initialize(); gallery gallery = weldcontainer.instance().select(gallery.class).get(); string message= gallery.vehicle.work(); system.out.println("> "+message); } } weldcontainer type object reference in gallery class access to a cdi object which represents the initiated container environment and after this point, cdi objects are accessible and can be made injection procedures through weldcontainer object. after this point, it is used by adopting a cdi object that is a gallery class type through the container instance. in here, a programmatic access to the gallery object is provided. programmatic access is required for the startup of java se applications (as in spring), but you can easily access to all cdi objects in the environment also by annotation-based injection method through the obtained gallery cdi object, e.g. [ private @inject vehicle vehicle; ] an output as the following occurs when the gallery class is being run; the real content and sample code can be accessed in http://en.kodcu.com/2013/06/cdi-default-and-inject-annotations/ hope to see you again..

If you’re using any of these tools for Web Services – Axis2, CXF etc. – that internally make use of Woodstox XML processor (wstx), and you're getting an exception like this during webservice calls, javax.xml.ws.soap.SOAPFaultException: Error reading XMLStreamReader. at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:...) ... Caused by: com.ctc.wstx.exc.WstxUnexpectedCharException: Unexpected character ... at com.ctc.wstx.sr.StreamScanner.throwUnexpectedChar(StreamScanner.java:...) at com.ctc.wstx.sr.BasicStreamReader.nextFromProlog(BasicStreamReader.java:...) at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:...) at com.ctc.wstx.sr.BasicStreamReader.nextTag(BasicStreamReader.java:...) the problem is that the wstx tokenizer/parser encountered unexpected (but not necessarily invalid per se) character; character that is not legal in current context. Could happen, for example, if white space was missing between attribute value and name of next attribute, according to API docs (http://woodstox.codehaus.org/3.2.9/javadoc/com/ctc/wstx/exc/WstxUnexpectedCharException.html). This simply means that you’re receiving an ill-formed SOAP XML as response. You need to check the SOAP response construction logic/code at the other end you’re communicating to.

implementation of mixins using aop (aspectj) or source-code modification (jamopp) in object-oriented programming languages, a mixin refers to a defined amount of functionality which can be added to a class. an important aspect of this is that it makes it possible to concentrate more on the properties of a particular behaviour than on the inheritance structures during development. in scala for example, a variant of mixins can be found under the name of “traits”. although java does not provide direct support for mixins, these can easily be added on with a few annotations, interfaces and some tool support. occasionally you read in a few online articles that mixins are incorporated into java version 8. unfortunately, this is not the case. a feature of the lambda project ( jsr-335 ) are the so-called “virtual extension methods” (vem). whilst these are similar to mixins, they do have a different background and are significantly more limited in functionality. the motivation for the introduction of vems is the problem of backward compatibility in the introduction of new methods in interfaces . as “real” mixins are not expected in the java language in the near future, this article intends to demonstrate how it is already possible to create mixin support in java projects now, using simple methods. to do this, we will discuss two approaches: using aop with aspectj and using source-code modification with jamopp . why not just inheritance? when asked at an event “ what would you change about java if you could reinvent it? ” james gosling , the inventor of java is said to have answered “ i would get rid of the classes “. after the laughter had died down, he explained what he meant by that: inheritance in java, which is expressed with the “extends” relationship, should – wherever possible – be replaced by interfaces [ why extends is evil ]. any experienced developer knows what he meant here: inheritance should be used sparingly. it is very easy to misuse it as a technical construct to reuse code, and not to model a technically motivated parent-child relationship with it. but even if one considers such a technically motivated code reuse as legitimate, one quickly reaches its limits, as java does not allow multiple inheritance. mixins are always useful if several classes have similar properties or define a similar behaviour, but these cannot be reasonably modelled simply via slim relationship hierarchies. in english, terms which end in “able” (e.g. “sortable”, “comparable” or “commentable”) are often an indicator for applications of mixins. also, when starting to write “utility” methods in order to avoid a code duplication in the implementation of interfaces, this can be an indication of a meaningful case of application. mixins with aop so-called inter-type declarations are an extremely simple possibility for implementing mixins, offered by the aspectj eclipse project. with these, it is possible – among other things – to add new instance variables and methods to any target class. this will be shown in the following, based on a small example in listing 1. for this, we will use the following terms: basis-interface describes the desired behaviour. classes which the mixin should not use can use this interface. mixin-interface intermediate interface used in the aspect and implemented by classes which the mixin is to use. mixin-provider aspect which provides the implementation for the mixin. mixin-user class which uses (implements) one or more mixin interfaces. // === listing 1 === /** base-interface */ public interface named { public string getname(); } /** mixin-interface */ public interface namedmixin extends named { } /** mixin-provider */ public aspect namedaspect { private string namedmixin.name; public final void namedmixin.setname(string name) { this.name = name; } public final string namedmixin.getname() { return name; } } /** mixin-user */ public class myclass implements namedmixin { // could have more methods or use different mixins } listing 1 shows a complete aop-based mixin example. if aspectj is set up correctly, the following source text should compile and run without errors: myclass myobj = new myclass(); myobj.setname("abc"); system.out.println(myobj.getname()); it is possible to work quite comfortably with aop variants, but there are also a few disadvantages which will be explored here. first of all, inter-type declarations cannot deal with generic types in the target class. this is not absolutely necessary in many cases, but can be very practical. for example, it is possible to define the “named” interface just as well with a generic type instead of “string”. it would then define the behaviour for any name types. the class used could then determine how the type of name should look. a further disadvantage is that the methods generated by aspectj follow their own naming conventions. this makes it difficult to search the classes using reflection, as you would have to reckon with method names such as “ajc$intermethoddispatch …” last but not least, without the support of the development environment, you cannot see the source code in the target class and are dependent on the interface declaration alone. this could, however, be seen as an advantage, since the using classes contain less code. appearance: java model parser and printer (jamopp) an alternative to the implementation of mixins with aspektj is offered by java model parser and printer (jamopp). simply put, jamopp can read java source code, present it as an object graph in the memory and transform (i.e. write) it back into text. with jamopp, it is therefore possible to programmatically process java code and thus automate refactoring or implement your own code analyses, for example. technologically, jamopp is based on the eclipse modeling framework (emf) and emftext . jamopp is jointly developed by the technical university of dresden and devboost gmbh and is freely available on github as an open-source project. mixins with jamopp in the following, we would like to take up the example from the aop mixins and expand this slightly. for this, we will first define a few annotations: @mixinintf indicates a mixin interface. @mixinprovider indicates a class which provides the implementation for a mixin. the implemented mixin interface is specified as the only parameter. @mixingenerated marks methods and instance variables which have been generated by the mixin. the only parameter is the class of the mixin provider. in the following, we will also be expanding the interfaces and classes from listing 1 with a generic type for the name. only the class using the mixin defines which concrete type the name should actually have. // === listing 2 === /** base-interface (extended with generic parameter) */ public interface named { public t getname(); } /** mixin-interface */ @mixinintf public interface namedmixin extends named { } /** mixin-provider */ @mixinprovider(namedmixin.class) public final class namedmixinprovider implements named { @mixingenerated(namedmixinprovider.class) private t name; @mixingenerated(namedmixinprovider.class) public void setname(t name) { this.name = name; } @override @mixingenerated(namedmixinprovider.class) public t getname() { return name; } } /** special name type (alternative to string) */ public final class myname { private final string name; public myname(string name) { super(); if (name == null) { throw new illegalargumentexception("name == null"); } if (name.trim().length() == 0) { throw new illegalargumentexception("name is empty"); } this.name = name; } @override public string tostring() { return name; } } in the class which the mixin is to use, the mixin interface is now implemented again as shown in listing 3. in order to “blend” the fields and methods defined by the mixin provider into the myclass class, a code generator is used. with the help of jamopp, this modifies the myclass class and adds the instance variables and methods provided by the mixin provider. // === listing 3 === /** mixin-user */ public class myclass implements namedmixin { // could have more methods or use different mixins } in doing this, the code generator does the following. it reads the source code of every class, similarly to the normal java compiler, and, in doing so, examines the amount of implemented interfaces. if a mixin interface is present, i.e. an interface with the annotation @mixinintf, the corresponding provider is found and the instance variables and methods are copied into the class which is implementing the mixin. in order to initiate the generation of mixin codes, there are currently two options: using an eclipse plug-in directly when saving or as a maven plug-in as part of the build. installation instructions and the source code of both plug-ins can be found on github in the small srcmixins4j project. there is also an on-screen video available there, which demonstrates the use of the eclipse plug-in. listing 4 shows the how the modified target class then looks. // === listing 4 === /** mixin-user */ public class myclass implements namedmixin { @mixingenerated(namedmixinprovider.class) private myname name; @mixingenerated(namedmixinprovider.class) public void setname(myname name) { this.name = name; } @override @mixingenerated(namedmixinprovider.class) public myname getname() { return name; } } if the mixin interface is removed from the “implements” section, all of the provider’s fields and methods annotated with “@mixingenerated” will be deleted automatically. generated code can be overridden at any time by removing the “@mixingenerated” annotation. click on the following image to open a flash video that demonstrates the eclipse plugin: conclusion as native support of mixins in the java language standard is not expected in the foreseeable future, it is currently possible to make do with just some aop or source-code generation. which of the two options you choose depends essentially on whether you prefer to keep the mixin code separate from your own application code or whether you want them directly in the respective classes. in any case, the speed of development is significantly increased and you will concentrate less on inheritance hierarchies and more on the definition of functional behaviour. neither approach is perfect. in particular, conflicts are not automatically resolved. methods with the same signature from different interfaces which are provided by different mixin providers will, for example, lead to an error in a class which uses both mixins. those seeking anything more would have to transfer to another language with native mixin support, such as scala. about these ads

RabbitMQ is a popular message broker typically used for building integration between applications or different components of the same application using messages. This post is a very basic introduction on how to get started using RabbitMQ and assumes you already have setup the rabbitmq server. RabbitMQ is written in Erlang and has drivers/clients available for most major languages. We are using Java for this post therefore we will first get hold of the java client. The maven dependency for the java client is given below. com.rabbitmq amqp-client 3.0.4 While message brokers such as RabbitMQ can be used to model a variety of schemes such as one to one message delivery or publisher/subscriber, our application will be simple enough and have two basic components, a single producer, that will produce a message and a single consumer that will consume that message. In our example, the producer will produce a large number of messages, each message carrying a sequence number while the consumer will consume the messages in a separate thread. The EndPoint Abstract class: Let’s first write a class that generalizes both producers and consumers as ‘endpoints’ of a queue. Whether you are a producer or a consumer, the code to connect to a queue remains the same therefore we can generalize it in this class. package co.syntx.examples.rabbitmq; import java.io.IOException; import com.rabbitmq.client.Channel; import com.rabbitmq.client.Connection; import com.rabbitmq.client.ConnectionFactory; /** * Represents a connection with a queue * @author syntx * */ public abstract class EndPoint{ protected Channel channel; protected Connection connection; protected String endPointName; public EndPoint(String endpointName) throws IOException{ this.endPointName = endpointName; //Create a connection factory ConnectionFactory factory = new ConnectionFactory(); //hostname of your rabbitmq server factory.setHost("localhost"); //getting a connection connection = factory.newConnection(); //creating a channel channel = connection.createChannel(); //declaring a queue for this channel. If queue does not exist, //it will be created on the server. channel.queueDeclare(endpointName, false, false, false, null); } /** * Close channel and connection. Not necessary as it happens implicitly any way. * @throws IOException */ public void close() throws IOException{ this.channel.close(); this.connection.close(); } } The Producer: The producer class is what is responsible for writing a message onto a queue. We are using Apache Commons Lang to convert a Serializable java object to a byte array. The maven dependency for commons lang is commons-lang commons-lang 2.6 package co.syntx.examples.rabbitmq; import java.io.IOException; import java.io.Serializable; import org.apache.commons.lang.SerializationUtils; /** * The producer endpoint that writes to the queue. * @author syntx * */ public class Producer extends EndPoint{ public Producer(String endPointName) throws IOException{ super(endPointName); } public void sendMessage(Serializable object) throws IOException { channel.basicPublish("",endPointName, null, SerializationUtils.serialize(object)); } } The Consumer: The consumer, which can be run as a thread, has callback functions for various events, most important of which is the availability of a new message. package co.syntx.examples.rabbitmq; import java.io.IOException; import java.util.HashMap; import java.util.Map; import org.apache.commons.lang.SerializationUtils; import com.rabbitmq.client.AMQP.BasicProperties; import com.rabbitmq.client.Consumer; import com.rabbitmq.client.Envelope; import com.rabbitmq.client.ShutdownSignalException; /** * The endpoint that consumes messages off of the queue. Happens to be runnable. * @author syntx * */ public class QueueConsumer extends EndPoint implements Runnable, Consumer{ public QueueConsumer(String endPointName) throws IOException{ super(endPointName); } public void run() { try { //start consuming messages. Auto acknowledge messages. channel.basicConsume(endPointName, true,this); } catch (IOException e) { e.printStackTrace(); } } /** * Called when consumer is registered. */ public void handleConsumeOk(String consumerTag) { System.out.println("Consumer "+consumerTag +" registered"); } /** * Called when new message is available. */ public void handleDelivery(String consumerTag, Envelope env, BasicProperties props, byte[] body) throws IOException { Map map = (HashMap)SerializationUtils.deserialize(body); System.out.println("Message Number "+ map.get("message number") + " received."); } public void handleCancel(String consumerTag) {} public void handleCancelOk(String consumerTag) {} public void handleRecoverOk(String consumerTag) {} public void handleShutdownSignal(String consumerTag, ShutdownSignalException arg1) {} } Putting it together: In our driver class, we start a consumer thread and then proceed to generate a large number of messages that will be consumed by the consumer. package co.syntx.examples.rabbitmq; import java.io.IOException; import java.sql.SQLException; import java.util.HashMap; public class Main { public Main() throws Exception{ QueueConsumer consumer = new QueueConsumer("queue"); Thread consumerThread = new Thread(consumer); consumerThread.start(); Producer producer = new Producer("queue"); for (int i = 0; i < 100000; i++) { HashMap message = new HashMap(); message.put("message number", i); producer.sendMessage(message); System.out.println("Message Number "+ i +" sent."); } } /** * @param args * @throws SQLException * @throws IOException */ public static void main(String[] args) throws Exception{ new Main(); } }

This post comes from Vladimir Šor at the Plumbr blog. Some of you have been there. You have added -Xmx option to your startup scripts and sat back relaxed knowing that there is no way your Java process is going to eat up more memory than your fine-tuned option had permitted. And then you were up for a nasty surprise. Either by yourself by checking a process table in your development / test box or if things got really bad then by operations who calls you in the middle of the night telling that the 4G memory you had asked for the production is exhausted. And that the application just died. So what the heck is happening under the hood? Why is the process consuming more memory than you allocated? Is it a bug or something completely normal? Bear with me and I will guide you through what is happening. First of all, part of it can definitely be a malicious native code leaking memory. But on 99% of the cases it is completely normal behaviour of the JVM. What you have specified via the -Xmx switches is limiting the memory consumed by your application heap. Besides heap there are other regions in memory which your application is using under the hood – namely permgen and stack sizes. So in order to limit those you should also specify the -XX:MaxPermSize and -Xss options respectively. In a short, you can predict your application memory usage with the following formula Max memory = [-Xmx] + [-XX:MaxPermSize] + number_of_threads * [-Xss] But besides the memory consumed by your application, the JVM itself also needs some elbow room. The need for it derives from several different reasons: Garbage collection. As you might recall, Java is a garbage collected language. In order for the garbage collector to know which objects are eligible for collection, it needs to keep track of the object graphs. So this is one part of the memory lost for this internal bookkeeping. Especially G1 is known for its excessive appetite for additional memory, so be aware of this. JIT optimization. Java Virtual Machine optimizes the code during the runtime. Again, to know which parts to optimize it needs to keep track of the execution of certain code parts. So again, you are going to lose memory. Off-heap allocations. If you happen to use off-heap memory, for example while using direct or mapped ByteBuffers yourself or via some clever 3rd party API then voila – you are extending your heap to something you actually cannot control via JVM configuration. JNI code. When you are using native code for example in the format of Type 2 database drivers then again, you are loading code in the native memory. Metaspace. If you are an early adopter of Java 8, you are using metaspace instead of the good old permgen to store class declarations. This is unlimited and in a native part of the JVM. You can end up using memory for other reasons than listed above as well, but I hope I managed to convince you that there is a significant amount of memory eaten up by the JVM internals. But is there a way to predict how much memory is actually going to be needed? Or at least understand where it disappears in order to optimize? As we have found out via painful experience – it is not possible to predict it with a reasonable precision. The JVM overhead can range from anything between just a few percentages to several hundred %. Your best friend is again the good old trial and error. So you need to run your application with loads similar to production environment and measure. Measuring the additional overhead is trivial – just monitor the process with the OS built-in tools (top on Linux, Activity Monitor on OS X, Task Manager on Windows) to find out the real memory consumption. Subtract the heap and permgen sizes from the real consumption and you see the overhead posed. Now if you need to reduce to overhead you would like to understand where it actually disappears. We have found vmmap on Mac OS X and pmap on Linux to be a truly helpful tools in this case. We have not used the vmmap port to Windows by ourselves, but it seems there is a tool for Windows fanboys as well. The following example illustrates this situation. I have launched my Jetty with the following startup parameters: -Xmx168m -Xms168m -XX:PermSize=32m -XX:MaxPermSize=32m -Xss1m Knowing that I have 30 threads launched in my application I might expect that my memory usage does not exceed 230M no matter what. But now when I look at the Activity Monitor on my Mac OS X, I see something different The real memory usage has exceeded 320M. Now digging under the hood how the process with the help of the vmmap output we start to understand where the memory is disappearing. Lets go through some samples: The following says we have lost close to 2MB is lost to memory mapped rt.jar library. mapped file 00000001178b9000-0000000117a88000 [ 1852K] r--/r-x SM=ALI /Library/Java/JavaVirtualMachines/jdk1.7.0_21.jdk/Contents/Home/jre/lib/rt.jar - Next section explains that we are using ~6MB for a particular Dynamic Library loaded __TEXT 0000000104573000-0000000104c00000 [ 6708K] r-x/rwx SM=COW /Library/Java/JavaVirtualMachines/jdk1.7.0_21.jdk/Contents/Home/jre/lib/server/libjvm.dylib - See more at: http://plumbr.eu/blog/why-does-my-java-process-consume-more-memory-than-xmx?utm_source=rss&utm_medium=rss&utm_campaign=rss20130618#sthash.G8fx60eX.dpuf And here we have threads no 25-30 each allocating 1MB for their stacks and stack guards Stack 000000011a5f1000-000000011a6f0000 [ 1020K] rw-/rwx SM=ZER thread 25 Stack 000000011aa8c000-000000011ab8b000 [ 1020K] rw-/rwx SM=ZER thread 27 Stack 000000011ab8f000-000000011ac8e000 [ 1020K] rw-/rwx SM=ZER thread 28 Stack 000000011ac92000-000000011ad91000 [ 1020K] rw-/rwx SM=ZER thread 29 Stack 000000011af0f000-000000011b00e000 [ 1020K] rw-/rwx SM=ZER thread 30 - See more at: http://plumbr.eu/blog/why-does-my-java-process-consume-more-memory-than-xmx?utm_source=rss&utm_medium=rss&utm_campaign=rss20130618#sthash.G8fx60eX.dpuf STACK GUARD 000000011a5ed000-000000011a5ee000 [ 4K] ---/rwx SM=NUL stack guard for thread 25 STACK GUARD 000000011aa88000-000000011aa89000 [ 4K] ---/rwx SM=NUL stack guard for thread 27 STACK GUARD 000000011ab8b000-000000011ab8c000 [ 4K] ---/rwx SM=NUL stack guard for thread 28 STACK GUARD 000000011ac8e000-000000011ac8f000 [ 4K] ---/rwx SM=NUL stack guard for thread 29 STACK GUARD 000000011af0b000-000000011af0c000 [ 4K] ---/rwx SM=NUL stack guard for thread 30 - See more at: http://plumbr.eu/blog/why-does-my-java-process-consume-more-memory-than-xmx?utm_source=rss&utm_medium=rss&utm_campaign=rss20130618#sthash.G8fx60eX.dpuf I hope I managed to shed some light upon the tricky task of predicting and measuring the actual memory consumption. If you enjoyed the content – subscribe to our RSS feed or start following us in Twitter to be notified on future interesting posts.

What is an Object? An object is a collection of data and actions. An object is an instance of a class. Objects have states and behaviors. In the real-world, we can find so many objects around us, for example Cars, Birds, Humans etc. All these objects have a state and behavior. If we consider a Car then it have some data speed, lights on, direction, etc. and have some actions turn right, accelerate, turn lights on, etc. If you compare the java object with a real world object, both of them have similar characteristics. Java objects also have a state and behavior. A Java object's state is stored in fields and behavior is shown via methods. Technically speaking Car, Bird and Human are considered as Class in Java. Brian Christopher is an object of human and Vehicle XKMV-669 is the object of car. Creating Object Using new keyword is the most common way to create an object in java. Syntax:- ClassName Obj.Name = new ClassName(); // Human brianChristopher= new Human(); // Car vehicleXKMV_669 = new Car(); The first statement creates a new Human object and second statement creates Car object. This single statement performs three actions, Declaration, Instantiation, and Initialization. Here, Human brianChristopher is a variable declaration which simply declares to the compiler that the name brianChristopher will be used to refer to an object whose type is Human, the new operator instantiates the Human class (thereby creating a new Human object), and Human initializes the object. Object Lifecycle In Java, it has seven states in Object lifecycle. They are, Created In use Invisible Unreachable Collected Finalized De-allocated Created The following are the some actions performed when an object is created,New memory is allocated for an object. Once the object has been created, assuming that it is assigned to some variable and then it directly moves to the In Use state. In use Objects that are held by at least one strong reference are considered to be “In Use”. Invisible An object is in the “Invisible” state when there are no longer any strong references that are accessible to the program, even though there might still be references. Unreachable An object enters an “unreachable” state when no more strong references to it exist. When an object is unreachable then it is a state for collection. It is important to note that not just any strong reference will hold an object in memory. These must be references that chain from a garbage collection root. Garbage collection roots are a special class of variable that includes,Temporary variables on the stack Collected An object is in the “collected” state when the garbage collector has recognized an object as unreachable and readies it for final processing as a precursor to de-allocation. If the object has a finalize method, then it is marked for finalization. Finalized An object is in the “finalized” state if it is still unreachable after it’s finalize method, if any, has been run. A finalized object is awaiting de-allocation. If you are considering using a finalizer to ensure that important resources are freed in a timely manner, you might want to reconsider. To lengthening object lifetimes, finalize methods can increase object size. De-allocated The de-allocated state is the final step in garbage collection. If an object is still unreachable after all the above work has done, then this is the state for de-allocation. For more detailed discussion about Object Lifecycle with real-world examples download OCAJP 7 Training Lab from EPractize Labs.