Lots and lots of web applications need to page through information. From customer records, to the albums in your itunes collection. So as web developers and architects, it’s important that we do all this efficiently. Start by looking at how you’re fetching information from your MySQL database. We’ve outlined three ways to do just that. 1. Paging without discarding records Ultimately we’re trying to avoid discarding records. After all if the server doesn’t fetch them, we save big. How else can we avoid this extra work. How about remember the last name. For example: select id, name, address, phone FROM customers WHERE id > 990 ORDER BY id LIMIT 10; Of course such a solution would only work if you were paging by ID. If you page by name, it might get messier as there may be more than one person with the same name. If ID doesn’t work for your application, perhaps returning paged users by USERNAME might work. Those would be unique: SELECT id, username FROM customers WHERE username > '[email protected]' ORDER BY username LIMIT 10; Paging queries can be slow with SQL as they often involve the OFFSET keyword which instructs the server you only want a subset. However it typically scans collects and then discards those rows first. With deferred join or by maintaining a place or position column you can avoid this, and speedup your database dramatically. 2. Try using a Deferred Join This is an interesting trick. Suppose you have pages of customers. Each page displays ten customers. The query will use LIMIT to get ten records, and OFFSET to skip all the previous page results. When you get to the 100th page, it’s doing LIMIT 10 OFFSET 990. So the server has to go and read all those records, then discard them. SELECT id, name, address, phone FROM customers ORDER BY name LIMIT 10 OFFSET 990; MySQL is first scanning an index then retrieving rows in the table by primary key id. So it’s doing double lookups and so forth. Turns out you can make this faster with a tricky thing called a deferred join. The inside piece just uses the primary key. An explain plan shows us “using index” which we love! SELECT id FROM customers ORDER BY name LIMIT 10 OFFSET 990; Now combine this using an INNER JOIN to get the ten rows and data you want: SELECT id, name, address, phone FROM customers INNER JOIN ( SELECT id FROM customers ORDER BY name LIMIT 10 OFFSET 990) AS my_results USING(id); That’s pretty cool! 3. Maintain a Page or Place column Another way to trick the optimizer from retrieving rows it doesn’t need is to maintain a column for the page, place or position. Yes you need to update that column whenever you (a) INSERT a row (b) DELETE a row ( c) move a row with UPDATE. This could get messy with page, but a straight place or position might work easier. SELECT id, name, address, phone FROM customers WHERE page = 100 ORDER BY name; Or with place column something like this: SELECT id, name, address, phone FROM customers WHERE place BETWEEN 990 AND 999 ORDER BY name;

"Chart JS Library" provides API for drawing different charts. Drawing is based on HTML CANVAS Element. Download Link:- http://www.chartjs.org/ In this Demo, "We will draw a Radar Chart .The Student input data is JSON in nature.The Servlet returns the JSON data when called by Jquery Ajax method.The Student Java class object is converted to JSON representation using GSON Library". The Java web project structure, The Student Servlet StudentJsonDataServlet.java , package com.sandeep.chartjs.servlet; import java.io.IOException; import java.util.ArrayList; import java.util.List; import javax.servlet.ServletException; import javax.servlet.annotation.WebServlet; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import com.google.gson.Gson; import com.sandeep.chartjs.data.Student; @WebServlet("/StudentJsonDataServlet") public class StudentJsonDataServlet extends HttpServlet { private static final long serialVersionUID = 1L; public StudentJsonDataServlet() { super(); } protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { List listOfStudent = getStudentData(); Gson gson = new Gson(); String jsonString = gson.toJson(listOfStudent); response.setContentType("application/json"); response.getWriter().write(jsonString); } private List getStudentData() { List listOfStudent = new ArrayList(); Student s1 = new Student(); s1.setName("Sandeep"); s1.setComputerMark(75); s1.setMathematicsMark(26); s1.setGeographyMark(91); s1.setHistoryMark(55); s1.setLitratureMark(36); listOfStudent.add(s1); return listOfStudent; } } The HTML markup chartjs-demo.html, The java script file for radar chart ts-chart-script.js, var TUTORIAL_SAVVY ={ /*Makes the AJAX calll (synchronous) to load a Student Data*/ loadStudentData : function(){ var formattedstudentListArray =[]; $.ajax({ async: false, url: "StudentJsonDataServlet", dataType:"json", success: function(studentJsonData) { console.log(studentJsonData); $.each(studentJsonData,function(index,aStudent){ formattedstudentListArray.push([aStudent.mathematicsMark,aStudent.computerMark,aStudent.historyMark,aStudent.litratureMark,aStudent.geographyMark]); }); } }); return formattedstudentListArray; }, /*Crate the custom Object with the data*/ createChartData : function(jsonData){ console.log(jsonData); return { labels : ["Mathematics", "Computers", "History","Literature", "Geography"], datasets : [ { fillColor : "rgba(255,0,0,0.3)", strokeColor : "rgba(0,255,0,1)", pointColor : "rgba(0,0,255,1)", pointStrokeColor : "rgba(0,0,255,1)", /*As Ajax response data is a multidimensional array, we have 'student' data in 0th position*/ data : jsonData[0] } ] }; }, /*Renders the Chart on a canvas and returns the reference to chart*/ renderStudenrRadarChart:function(radarChartData){ var context2D = document.getElementById("canvas").getContext("2d"), myRadar = new Chart(context2D). Radar(radarChartData,{ scaleShowLabels : false, pointLabelFontSize : 10 }); return myRadar; }, /*Initalization Student render chart*/ initRadarChart : function(){ var studentData = TUTORIAL_SAVVY.loadStudentData(); chartData = TUTORIAL_SAVVY.createChartData(studentData); radarChartObj = TUTORIAL_SAVVY.renderStudenrRadarChart(chartData); } }; $(document).ready(function(){ TUTORIAL_SAVVY.initRadarChart(); }); The response Json data format for student, The Firebug console shows DOM Element, The output in browser will look like,This radar chart shows the a student('sandeep') marks in different subject,

cdi (context and dependency injection) is a complete and lightweight injection technology designed for java ee environment. special container objects (ejb,entitymanager), primitive data type elements and java class/objects written by you can be easily managed and injected as well through cdi. every defined java class in each application that configured in cdi standard is a candidate to become an injectable cdi object. this default behavior is provided by @default annotation that was installed per each java class secretly. there is an car class which has a vehicle implementation in the above uml diagram. a random int value is produced in sayvelocity() method and there is an output to the console such as “the car is running at the speed of x” in work() method, where x is represents a number produced randomly. @default // optional public class car implements vehicle { public string work() { return "car is working in "+ sayvelocity()+" kmh."; } public int sayvelocity(){ return threadlocalrandom.current().nextint(20, 240) ; } } if the above car class is in an application activated in cdi environment, it becomes a candidate to be an object managed by cdi. so, what is implied by the activation of cdi? first of all, necessary dependencies need to be included in the classpath for the activation of cdi environment. if you are using an application server like glassfish, cdi can be used without any extra definition because of the existence of cdi libraries on the application server. but if your application is a java se application or is running in lightweight containers such as tomcat, jetty; a cdi library must be added to the project. the reference library of cdi technology is the jboss weld archetype. for this reason, if jboss weld dependencies are added to the project such as the following, first phase of the cdi activation is realized. org.jboss.weld.se weld-se 1.1.10.final to activate the cdi environment, a blank cdi configuration file named beans.xml must be in the application as a requirement of the standard. this file must be located on the /meta-inf/beans.xml path for java se applications and /web-inf/beans.xml path for java ee web applications. the existence necessity of this file may sound silly initially, but the existence of beans.xml in the directories specified above can be considered as a permission given to the container in order to activate cdi environment. activation of cdi environment is automatically started in java ee web applications when beans.xml file is encountered in the /web-inf directory. but it is not the same for a java se application, starting a cdi container is the developer’s job. this case can be seen clearly if you pay attention to the following gallery class: public class gallery { @inject // injection point private vehicle vehicle; public static void main(string[] args) { weld weld = new weld(); weldcontainer weldcontainer = weld.initialize(); gallery gallery = weldcontainer.instance().select(gallery.class).get(); string message= gallery.vehicle.work(); system.out.println("> "+message); } } weldcontainer type object reference in gallery class access to a cdi object which represents the initiated container environment and after this point, cdi objects are accessible and can be made injection procedures through weldcontainer object. after this point, it is used by adopting a cdi object that is a gallery class type through the container instance. in here, a programmatic access to the gallery object is provided. programmatic access is required for the startup of java se applications (as in spring), but you can easily access to all cdi objects in the environment also by annotation-based injection method through the obtained gallery cdi object, e.g. [ private @inject vehicle vehicle; ] an output as the following occurs when the gallery class is being run; the real content and sample code can be accessed in http://en.kodcu.com/2013/06/cdi-default-and-inject-annotations/ hope to see you again..

I have a number of Spring MVC based RESTful services that return JSON. In 90% of the cases, the state of objects these services return will not change within a 24 hour period. This makes them (the JSON objects) perfect candidates for simple caching enabled by memcached. The idea was to have every request to Spring controllers intercepted, cache key generated and checked against the cache. If the key and corresponding value (JSON string) is available (a cache hit), it is returned to the caller as-is without making a full round trip to the database. However, if the cache has no entry for the key and hence no corresponding value (a cache miss), the call is forwarded to the controller, which in turn calls the logic to fetch desired object from the database and not only return it to the caller but also update the cache with the returned content. Keys are generated using the URL of the service in case of GET requests and the URL concatenated with POSTed input (as JSON) in case of POST requests. The resultant strings are encoded with MD5 to come up with a 32 character cache key which is well within the 250 character key length limit of memcached. Performance impact of using MD5 is yet to be evaluated during our load testing cycle. I started off trying to get hold of JSON response in the postHandle method of a Spring HandlerInterceptor. However since we are using @ResponseBody annotation in our controller, the JSON would be written directly to the stream. The ModelAndView was of course null because of this reason. If we removed the annotation and returned ModelAndView from the controller, the intended JSON object got enclosed in a map wrapper. A quick question on stack overflow didn’t help as the only suggestion I got was to extract my original object from the map wrapper. I wanted to keep this option (as discussed here as well ) as my last resort. The solution I eventually came up with involved Replacing the HandlerInterceptor with Servlet Filters Using DelegatingFilterProxy to make my filters spring application context aware Using HttpServletRequestWrapper to get control of the POST request body in the filter on the way in Using HttpServletResponseWrapper to get control of the response content in the filter on the way out True, its probably a more complex solution than just overriding MappingJacksonJsonView and extracting my JSON object, but it is more generic as it does not assume that all my content will always be JSON. Lets first start with the filter definition in the web.xml cacheFilter org.springframework.web.filter.DelegatingFilterProxy ... cacheFilter /* A standard filter configuration except for the fact that the filter class is always going to be org.springframework.web.filter.DelegatingFilterProxy. Where do you specify your own class ? As a bean in your spring context xml. The name of the filter and the name of the bean must be the same for the delegation to happen. Using the DelegatingFilterProxy allowed me to use my Filters with Spring. I can inject my dependencies as I would normally. Next, lets look at my MemcacheFilter filter Memcache Filter Class public class MemcacheFilter implements Filter { private static Logger logger = Logger.getLogger(MemcacheFilter.class); private CacheConfig cacheConfig; /** * Memcached lookup is being performed in this method. Firstly, keys are * generated depending on the request method (GET/POST). Then a cache lookup * is performed. If a value is obtained, the value is written to the * response otherwise, the actual target (in this case, Spring's Dispatcher * Servlet) is called by calling doFilter on the filteChain. The dispatcher * servlet calls the controller to produce the desire response which is * intercepted when the doFilter method returns. The Response is added to * the cache if the reponse code was 200(OK). * * @param request * @param response * @param filterChain * @throws IOException * @throws ServletException */ public void doFilter(ServletRequest request, ServletResponse response, FilterChain filterChain) throws IOException, ServletException { try { if ((request instanceof HttpServletRequest) && (response instanceof HttpServletResponse)) { // Wrapping the response in HTTPServletResponseWrapper MemcacheResponseWrapper responseWrap = new MemcacheResponseWrapper((HttpServletResponse) response); // Wrapping the request in HTTPServletResponseWrapper MemcacheRequestWrapper requestWrap = new MemcacheRequestWrapper((HttpServletRequest) request); // Get Memcached Client Instance MemcachedClient client = cacheConfig.getMemcachedClient(); Key keyGenerator = getKeyGenerator(requestWrap); if (keyGenerator != null) { String key = keyGenerator.getKey(requestWrap, cacheConfig); String value = (String) client.get(key); if (value == null) { // cache miss logger.info("Cache miss for key " + key); // call next filter/actual target for value filterChain.doFilter(requestWrap, responseWrap); if (responseWrap.getStatus() == HttpServletResponse.SC_OK) { // obtaining response content from // HttpServletResponseWrapper value = responseWrap.getOutputStream().toString(); // adding response to cache client.add(key, 0, value); logger.info("Adding response to cache: "+ (value.length() > 50 ? value.substring(0,50) + "..." : value)); } else { logger.warn("Did not add content to cache as response status is not 200"); } } else { // This case is a cache hit logger.info("Cache hit for key " + key); response.getWriter().println(value); } } else { logger.warn("Request skipped because no key generator could be found for the request's method"); // attempting call to actual target filterChain.doFilter(request, response); } } } catch (Exception ex) { logger.info("Cache functionality skipped due to exception", ex); // attempting call to actual target filterChain.doFilter(request, response); } } /** * Factory method that returns KeyGenerator based on the request method. * * @param httpRequest * @return */ private Key getKeyGenerator(HttpServletRequest httpRequest) { Key keyGenerator = null; if (httpRequest.getMethod().equalsIgnoreCase("GET")) { keyGenerator = new GetRequestKey(); } else if (httpRequest.getMethod().equalsIgnoreCase("POST")) { keyGenerator = new PostRequestKey(); } return keyGenerator; } public void init(FilterConfig arg0) throws ServletException { logger.debug("init"); } public CacheConfig getCacheConfig() { return cacheConfig; } public void setCacheConfig(CacheConfig cacheConfig) { this.cacheConfig = cacheConfig; } public void destroy() { logger.debug("destroy"); } } 1. I first wrap my request and response objects in the following statements. I have had to create the wrappers as well. Will get to those later. // Wrapping the response in HTTPServletResponseWrapper MemcacheResponseWrapper responseWrap = new MemcacheResponseWrapper((HttpServletResponse) response); // Wrapping the request in HTTPServletResponseWrapper MemcacheRequestWrapper requestWrap = new MemcacheRequestWrapper((HttpServletRequest) request); 2. Next, I have one of my injected classes, CacheConfig, provide me with a memcache client which I will use later to look up the cache. // Get Memcached Client Instance MemcachedClient client = cacheConfig.getMemcachedClient(); 3. I make a call to a function that tells me which key generator I should use, a GET one or a POST one depending on the request method. Key keyGenerator = getKeyGenerator(requestWrap); /** * Factory method that returns KeyGenerator based on the request method. * * @param httpRequest * @return */ private Key getKeyGenerator(HttpServletRequest httpRequest) { Key keyGenerator = null; if (httpRequest.getMethod().equalsIgnoreCase("GET")) { keyGenerator = new GetRequestKey(); } else if (httpRequest.getMethod().equalsIgnoreCase("POST")) { keyGenerator = new PostRequestKey(); } return keyGenerator; } 4. Check for a cache hit using the Key returned by the Key Generator. If its a miss, call next filter or target to compute actual value, get value from the response wrapper, and add it to the cache. if (keyGenerator != null) { String key = keyGenerator.getKey(requestWrap, cacheConfig); String value = (String) client.get(key); if (value == null) { // cache miss logger.info("Cache miss for key " + key); // call next filter/actual target for value filterChain.doFilter(requestWrap, responseWrap); if (responseWrap.getStatus() == HttpServletResponse.SC_OK) { // obtaining response content from // HttpServletResponseWrapper value = responseWrap.getOutputStream().toString(); // adding response to cache client.add(key, 0, value); logger.info("Adding response to cache: "+ (value.length() > 50 ? value.substring(0,50) + "..." : value)); } 5. If its a cache hit, just get return cached value else { // This case is a cache hit logger.info("Cache hit for key " + key); response.getWriter().println(value); } Lets take a look at each of the Wrappers. I am not going into a a lot of detail into how each of these work. Request Wrapper Class On the way in, the original POST content is extracted from the request and put in a String Buffer. To the filter, this content is returned via the toString() method of the WrappedInputStream class whereas the subsequently called controller calls the read method. public class MemcacheRequestWrapper extends HttpServletRequestWrapper { protected ServletInputStream stream; protected HttpServletRequest origRequest = null; protected BufferedReader reader = null; public MemcacheRequestWrapper(HttpServletRequest request) throws IOException { super(request); origRequest = request; } public ServletInputStream createInputStream() throws IOException { return (new WrappedInputStream(origRequest)); } @Override public ServletInputStream getInputStream() throws IOException { if (reader != null) { throw new IllegalStateException("getReader() has already been called for this request"); } if (stream == null) { stream = createInputStream(); } return stream; } @Override public BufferedReader getReader() throws IOException { if (reader != null) { return reader; } if (stream != null) { throw new IllegalStateException("getReader() has already been called for this request"); } stream = createInputStream(); reader = new BufferedReader(new InputStreamReader(stream)); return reader; } private class WrappedInputStream extends ServletInputStream { private StringBuffer originalInput = new StringBuffer(); private HttpServletRequest originalRequest; private ByteArrayInputStream byteArrayInputStream; public WrappedInputStream(HttpServletRequest request) throws IOException { this.originalRequest = request; BufferedReader bufferedReader = null; try { InputStream inputStream = request.getInputStream(); if (inputStream != null) { bufferedReader = new BufferedReader(new InputStreamReader(inputStream)); char[] charBuffer = new char[128]; int bytesRead = -1; while ((bytesRead = bufferedReader.read(charBuffer)) > 0) { originalInput.append(charBuffer, 0, bytesRead); } } byteArrayInputStream = new ByteArrayInputStream(originalInput.toString().getBytes()); } catch (IOException ex) { throw ex; } finally { if (bufferedReader != null) { try { bufferedReader.close(); } catch (IOException ex) { throw ex; } } } } @Override public String toString() { return this.originalInput.toString(); } @Override public int read() throws IOException { return byteArrayInputStream.read(); } } } Response Wrapper Class The response wrapper is similar to the request wrapper. Instead of the read method, there is a write method, called by the controller when its writing JSON content. This is stored in the wrapper and called in the filter. public class MemcacheResponseWrapper extends HttpServletResponseWrapper { protected ServletOutputStream stream; protected PrintWriter writer = null; protected HttpServletResponse origResponse = null; private int httpStatus = 200; public MemcacheResponseWrapper(HttpServletResponse response) { super(response); response.setContentType("application/json"); origResponse = response; } public ServletOutputStream createOutputStream() throws IOException { return (new WrappedOutputStream(origResponse)); } public ServletOutputStream getOutputStream() throws IOException { if (writer != null) { throw new IllegalStateException("getWriter() has already been called for this response"); } if (stream == null) { stream = createOutputStream(); } return stream; } public PrintWriter getWriter() throws IOException { if (writer != null) { return writer; } if (stream != null) { throw new IllegalStateException("getOutputStream() has already been called for this response"); } stream = createOutputStream(); writer = new PrintWriter(stream); return writer; } @Override public void sendError(int sc) throws IOException { httpStatus = sc; super.sendError(sc); } @Override public void sendError(int sc, String msg) throws IOException { httpStatus = sc; super.sendError(sc, msg); } @Override public void setStatus(int sc) { httpStatus = sc; super.setStatus(sc); } public int getStatus() { return httpStatus; } private class WrappedOutputStream extends ServletOutputStream { private StringBuffer originalOutput = new StringBuffer(); private HttpServletResponse originalResponse; public WrappedOutputStream(HttpServletResponse response) { this.originalResponse = response; } @Override public String toString() { return this.originalOutput.toString(); } @Override public void write(int arg0) throws IOException { originalOutput.append((char) arg0); originalResponse.getOutputStream().write(arg0); } } }

If you’re using any of these tools for Web Services – Axis2, CXF etc. – that internally make use of Woodstox XML processor (wstx), and you're getting an exception like this during webservice calls, javax.xml.ws.soap.SOAPFaultException: Error reading XMLStreamReader. at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:...) ... Caused by: com.ctc.wstx.exc.WstxUnexpectedCharException: Unexpected character ... at com.ctc.wstx.sr.StreamScanner.throwUnexpectedChar(StreamScanner.java:...) at com.ctc.wstx.sr.BasicStreamReader.nextFromProlog(BasicStreamReader.java:...) at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:...) at com.ctc.wstx.sr.BasicStreamReader.nextTag(BasicStreamReader.java:...) the problem is that the wstx tokenizer/parser encountered unexpected (but not necessarily invalid per se) character; character that is not legal in current context. Could happen, for example, if white space was missing between attribute value and name of next attribute, according to API docs (http://woodstox.codehaus.org/3.2.9/javadoc/com/ctc/wstx/exc/WstxUnexpectedCharException.html). This simply means that you’re receiving an ill-formed SOAP XML as response. You need to check the SOAP response construction logic/code at the other end you’re communicating to.

When you are working on a multinational website you may need the functionality to translate your website into different languages. There are many ways you can translate a website, in this article I will not go through the different ways you can translate a website. But one thing is that you will always need is to know what language you want display when a user hits your page. There are different options you can do with this, either have a default language and allow the user to switch to the language they want to use, or detect the language set in the browser and switch the language automatically. You can detect the language the browser is set to by looking at the server variable HTTP_ACCEPT_LANGUAGE. // Detect browser language $_SERVER['HTTP_ACCEPT_LANGUAGE']; This variable will display all the languages that you can set in your browser en-GB,en,en-US. But because you can select multiple languages for your browser they are returned as a comma separated string. From these languages you can explode the string and now you have a list of all languages that the user can read, looping through this list you can find supported languages and display these to the user. $supportedLangs = array('en-GB', 'fr', 'de'); $languages = explode(',',$_SERVER['HTTP_ACCEPT_LANGUAGE']); foreach($languages as $lang) { if(in_array($lang, $supportedLangs)) { // Set the page locale to the first supported language found $page->setLocale($lang); break; } }

implementation of mixins using aop (aspectj) or source-code modification (jamopp) in object-oriented programming languages, a mixin refers to a defined amount of functionality which can be added to a class. an important aspect of this is that it makes it possible to concentrate more on the properties of a particular behaviour than on the inheritance structures during development. in scala for example, a variant of mixins can be found under the name of “traits”. although java does not provide direct support for mixins, these can easily be added on with a few annotations, interfaces and some tool support. occasionally you read in a few online articles that mixins are incorporated into java version 8. unfortunately, this is not the case. a feature of the lambda project ( jsr-335 ) are the so-called “virtual extension methods” (vem). whilst these are similar to mixins, they do have a different background and are significantly more limited in functionality. the motivation for the introduction of vems is the problem of backward compatibility in the introduction of new methods in interfaces . as “real” mixins are not expected in the java language in the near future, this article intends to demonstrate how it is already possible to create mixin support in java projects now, using simple methods. to do this, we will discuss two approaches: using aop with aspectj and using source-code modification with jamopp . why not just inheritance? when asked at an event “ what would you change about java if you could reinvent it? ” james gosling , the inventor of java is said to have answered “ i would get rid of the classes “. after the laughter had died down, he explained what he meant by that: inheritance in java, which is expressed with the “extends” relationship, should – wherever possible – be replaced by interfaces [ why extends is evil ]. any experienced developer knows what he meant here: inheritance should be used sparingly. it is very easy to misuse it as a technical construct to reuse code, and not to model a technically motivated parent-child relationship with it. but even if one considers such a technically motivated code reuse as legitimate, one quickly reaches its limits, as java does not allow multiple inheritance. mixins are always useful if several classes have similar properties or define a similar behaviour, but these cannot be reasonably modelled simply via slim relationship hierarchies. in english, terms which end in “able” (e.g. “sortable”, “comparable” or “commentable”) are often an indicator for applications of mixins. also, when starting to write “utility” methods in order to avoid a code duplication in the implementation of interfaces, this can be an indication of a meaningful case of application. mixins with aop so-called inter-type declarations are an extremely simple possibility for implementing mixins, offered by the aspectj eclipse project. with these, it is possible – among other things – to add new instance variables and methods to any target class. this will be shown in the following, based on a small example in listing 1. for this, we will use the following terms: basis-interface describes the desired behaviour. classes which the mixin should not use can use this interface. mixin-interface intermediate interface used in the aspect and implemented by classes which the mixin is to use. mixin-provider aspect which provides the implementation for the mixin. mixin-user class which uses (implements) one or more mixin interfaces. // === listing 1 === /** base-interface */ public interface named { public string getname(); } /** mixin-interface */ public interface namedmixin extends named { } /** mixin-provider */ public aspect namedaspect { private string namedmixin.name; public final void namedmixin.setname(string name) { this.name = name; } public final string namedmixin.getname() { return name; } } /** mixin-user */ public class myclass implements namedmixin { // could have more methods or use different mixins } listing 1 shows a complete aop-based mixin example. if aspectj is set up correctly, the following source text should compile and run without errors: myclass myobj = new myclass(); myobj.setname("abc"); system.out.println(myobj.getname()); it is possible to work quite comfortably with aop variants, but there are also a few disadvantages which will be explored here. first of all, inter-type declarations cannot deal with generic types in the target class. this is not absolutely necessary in many cases, but can be very practical. for example, it is possible to define the “named” interface just as well with a generic type instead of “string”. it would then define the behaviour for any name types. the class used could then determine how the type of name should look. a further disadvantage is that the methods generated by aspectj follow their own naming conventions. this makes it difficult to search the classes using reflection, as you would have to reckon with method names such as “ajc$intermethoddispatch …” last but not least, without the support of the development environment, you cannot see the source code in the target class and are dependent on the interface declaration alone. this could, however, be seen as an advantage, since the using classes contain less code. appearance: java model parser and printer (jamopp) an alternative to the implementation of mixins with aspektj is offered by java model parser and printer (jamopp). simply put, jamopp can read java source code, present it as an object graph in the memory and transform (i.e. write) it back into text. with jamopp, it is therefore possible to programmatically process java code and thus automate refactoring or implement your own code analyses, for example. technologically, jamopp is based on the eclipse modeling framework (emf) and emftext . jamopp is jointly developed by the technical university of dresden and devboost gmbh and is freely available on github as an open-source project. mixins with jamopp in the following, we would like to take up the example from the aop mixins and expand this slightly. for this, we will first define a few annotations: @mixinintf indicates a mixin interface. @mixinprovider indicates a class which provides the implementation for a mixin. the implemented mixin interface is specified as the only parameter. @mixingenerated marks methods and instance variables which have been generated by the mixin. the only parameter is the class of the mixin provider. in the following, we will also be expanding the interfaces and classes from listing 1 with a generic type for the name. only the class using the mixin defines which concrete type the name should actually have. // === listing 2 === /** base-interface (extended with generic parameter) */ public interface named { public t getname(); } /** mixin-interface */ @mixinintf public interface namedmixin extends named { } /** mixin-provider */ @mixinprovider(namedmixin.class) public final class namedmixinprovider implements named { @mixingenerated(namedmixinprovider.class) private t name; @mixingenerated(namedmixinprovider.class) public void setname(t name) { this.name = name; } @override @mixingenerated(namedmixinprovider.class) public t getname() { return name; } } /** special name type (alternative to string) */ public final class myname { private final string name; public myname(string name) { super(); if (name == null) { throw new illegalargumentexception("name == null"); } if (name.trim().length() == 0) { throw new illegalargumentexception("name is empty"); } this.name = name; } @override public string tostring() { return name; } } in the class which the mixin is to use, the mixin interface is now implemented again as shown in listing 3. in order to “blend” the fields and methods defined by the mixin provider into the myclass class, a code generator is used. with the help of jamopp, this modifies the myclass class and adds the instance variables and methods provided by the mixin provider. // === listing 3 === /** mixin-user */ public class myclass implements namedmixin { // could have more methods or use different mixins } in doing this, the code generator does the following. it reads the source code of every class, similarly to the normal java compiler, and, in doing so, examines the amount of implemented interfaces. if a mixin interface is present, i.e. an interface with the annotation @mixinintf, the corresponding provider is found and the instance variables and methods are copied into the class which is implementing the mixin. in order to initiate the generation of mixin codes, there are currently two options: using an eclipse plug-in directly when saving or as a maven plug-in as part of the build. installation instructions and the source code of both plug-ins can be found on github in the small srcmixins4j project. there is also an on-screen video available there, which demonstrates the use of the eclipse plug-in. listing 4 shows the how the modified target class then looks. // === listing 4 === /** mixin-user */ public class myclass implements namedmixin { @mixingenerated(namedmixinprovider.class) private myname name; @mixingenerated(namedmixinprovider.class) public void setname(myname name) { this.name = name; } @override @mixingenerated(namedmixinprovider.class) public myname getname() { return name; } } if the mixin interface is removed from the “implements” section, all of the provider’s fields and methods annotated with “@mixingenerated” will be deleted automatically. generated code can be overridden at any time by removing the “@mixingenerated” annotation. click on the following image to open a flash video that demonstrates the eclipse plugin: conclusion as native support of mixins in the java language standard is not expected in the foreseeable future, it is currently possible to make do with just some aop or source-code generation. which of the two options you choose depends essentially on whether you prefer to keep the mixin code separate from your own application code or whether you want them directly in the respective classes. in any case, the speed of development is significantly increased and you will concentrate less on inheritance hierarchies and more on the definition of functional behaviour. neither approach is perfect. in particular, conflicts are not automatically resolved. methods with the same signature from different interfaces which are provided by different mixin providers will, for example, lead to an error in a class which uses both mixins. those seeking anything more would have to transfer to another language with native mixin support, such as scala. about these ads

RabbitMQ is a popular message broker typically used for building integration between applications or different components of the same application using messages. This post is a very basic introduction on how to get started using RabbitMQ and assumes you already have setup the rabbitmq server. RabbitMQ is written in Erlang and has drivers/clients available for most major languages. We are using Java for this post therefore we will first get hold of the java client. The maven dependency for the java client is given below. com.rabbitmq amqp-client 3.0.4 While message brokers such as RabbitMQ can be used to model a variety of schemes such as one to one message delivery or publisher/subscriber, our application will be simple enough and have two basic components, a single producer, that will produce a message and a single consumer that will consume that message. In our example, the producer will produce a large number of messages, each message carrying a sequence number while the consumer will consume the messages in a separate thread. The EndPoint Abstract class: Let’s first write a class that generalizes both producers and consumers as ‘endpoints’ of a queue. Whether you are a producer or a consumer, the code to connect to a queue remains the same therefore we can generalize it in this class. package co.syntx.examples.rabbitmq; import java.io.IOException; import com.rabbitmq.client.Channel; import com.rabbitmq.client.Connection; import com.rabbitmq.client.ConnectionFactory; /** * Represents a connection with a queue * @author syntx * */ public abstract class EndPoint{ protected Channel channel; protected Connection connection; protected String endPointName; public EndPoint(String endpointName) throws IOException{ this.endPointName = endpointName; //Create a connection factory ConnectionFactory factory = new ConnectionFactory(); //hostname of your rabbitmq server factory.setHost("localhost"); //getting a connection connection = factory.newConnection(); //creating a channel channel = connection.createChannel(); //declaring a queue for this channel. If queue does not exist, //it will be created on the server. channel.queueDeclare(endpointName, false, false, false, null); } /** * Close channel and connection. Not necessary as it happens implicitly any way. * @throws IOException */ public void close() throws IOException{ this.channel.close(); this.connection.close(); } } The Producer: The producer class is what is responsible for writing a message onto a queue. We are using Apache Commons Lang to convert a Serializable java object to a byte array. The maven dependency for commons lang is commons-lang commons-lang 2.6 package co.syntx.examples.rabbitmq; import java.io.IOException; import java.io.Serializable; import org.apache.commons.lang.SerializationUtils; /** * The producer endpoint that writes to the queue. * @author syntx * */ public class Producer extends EndPoint{ public Producer(String endPointName) throws IOException{ super(endPointName); } public void sendMessage(Serializable object) throws IOException { channel.basicPublish("",endPointName, null, SerializationUtils.serialize(object)); } } The Consumer: The consumer, which can be run as a thread, has callback functions for various events, most important of which is the availability of a new message. package co.syntx.examples.rabbitmq; import java.io.IOException; import java.util.HashMap; import java.util.Map; import org.apache.commons.lang.SerializationUtils; import com.rabbitmq.client.AMQP.BasicProperties; import com.rabbitmq.client.Consumer; import com.rabbitmq.client.Envelope; import com.rabbitmq.client.ShutdownSignalException; /** * The endpoint that consumes messages off of the queue. Happens to be runnable. * @author syntx * */ public class QueueConsumer extends EndPoint implements Runnable, Consumer{ public QueueConsumer(String endPointName) throws IOException{ super(endPointName); } public void run() { try { //start consuming messages. Auto acknowledge messages. channel.basicConsume(endPointName, true,this); } catch (IOException e) { e.printStackTrace(); } } /** * Called when consumer is registered. */ public void handleConsumeOk(String consumerTag) { System.out.println("Consumer "+consumerTag +" registered"); } /** * Called when new message is available. */ public void handleDelivery(String consumerTag, Envelope env, BasicProperties props, byte[] body) throws IOException { Map map = (HashMap)SerializationUtils.deserialize(body); System.out.println("Message Number "+ map.get("message number") + " received."); } public void handleCancel(String consumerTag) {} public void handleCancelOk(String consumerTag) {} public void handleRecoverOk(String consumerTag) {} public void handleShutdownSignal(String consumerTag, ShutdownSignalException arg1) {} } Putting it together: In our driver class, we start a consumer thread and then proceed to generate a large number of messages that will be consumed by the consumer. package co.syntx.examples.rabbitmq; import java.io.IOException; import java.sql.SQLException; import java.util.HashMap; public class Main { public Main() throws Exception{ QueueConsumer consumer = new QueueConsumer("queue"); Thread consumerThread = new Thread(consumer); consumerThread.start(); Producer producer = new Producer("queue"); for (int i = 0; i < 100000; i++) { HashMap message = new HashMap(); message.put("message number", i); producer.sendMessage(message); System.out.println("Message Number "+ i +" sent."); } } /** * @param args * @throws SQLException * @throws IOException */ public static void main(String[] args) throws Exception{ new Main(); } }

People often ask me how they can map a java.util.Map such that the keys become the node names. In this post I will demonstrate how this can be done using the new Variable Node mapping that we have added in EclipseLink MOXy. You can try this out today using a nightly build of EclipseLink 2.6.0: http://www.eclipse.org/eclipselink/downloads/nightly.php Input/Output Below are the XML and JSON representations we will use in this example. Each has node names that correspond to keys and contents that correspond to values of a Map. XML (input.xml) 1 2 JSON (Output) { "A" : 1, "B" : 2 } Java Model (Root) We want a non-default XML (and JSON) representation for Map so we will use an XmlAdapter (see: JAXB and java.util.Map). MOXy's @XmlPath extension will be used to prevent the contents of the adapted Map from being wrapped in a parent element (see: XPath Based Mapping). package blog.variablenode.map; import java.util.*; import javax.xml.bind.annotation.*; import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter; import org.eclipse.persistence.oxm.annotations.XmlPath; @XmlRootElement @XmlAccessorType(XmlAccessType.FIELD) public class Root { @XmlPath(".") @XmlJavaTypeAdapter(MapAdapter.class) private Map map = new HashMap(); } XmlAdapter (MapAdapter) An XmlAdapter is used to convert an object for the purposes of marshalling/unmarshalling (see: XmlAdapter - JAXB's Secret Weapon). In this example we convert the Map to an AdaptedMap that has a List of AdaptedEntry values. These AdaptedEntry values have a field (key) to represent the key. It is this field that we will use with @XmlVariableNode (line 13). We will mark the key field with @XmlTransient to prevent if from being marshalled/unmarshalled (line 20). package blog.variablenode.map; import java.util.*; import java.util.Map.Entry; import javax.xml.bind.annotation.*; import javax.xml.bind.annotation.adapters.XmlAdapter; import org.eclipse.persistence.oxm.annotations.XmlVariableNode; public class MapAdapter extends XmlAdapter> { public static class AdaptedMap { @XmlVariableNode("key") List entries = new ArrayList(); } public static class AdaptedEntry { @XmlTransient public String key; @XmlValue public Integer value; } @Override public AdaptedMap marshal(Map map) throws Exception { AdaptedMap adaptedMap = new AdaptedMap(); for(Entry entry : map.entrySet()) { AdaptedEntry adaptedEntry = new AdaptedEntry(); adaptedEntry.key = entry.getKey(); adaptedEntry.value = entry.getValue(); adaptedMap.entries.add(adaptedEntry); } return adaptedMap; } @Override public Map unmarshal(AdaptedMap adaptedMap) throws Exception { List adaptedEntries = adaptedMap.entries; Map map = new HashMap(adaptedEntries.size()); for(AdaptedEntry adaptedEntry : adaptedEntries) { map.put(adaptedEntry.key, adaptedEntry.value); } return map; } } Demo Below is some demo code that can be run to prove that everything works. The Root object will be instantiated from XML input and marshalled to create the JSON output. package blog.variablenode.map; import java.io.File; import javax.xml.bind.*; import org.eclipse.persistence.jaxb.MarshallerProperties; public class Demo { public static void main(String[] args) throws Exception { JAXBContext jc = JAXBContext.newInstance(Root.class); Unmarshaller unmarshaller = jc.createUnmarshaller(); File xml = new File("src/blog/variablenode/map/input.xml"); Root root = (Root) unmarshaller.unmarshal(xml); Marshaller marshaller = jc.createMarshaller(); marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true); marshaller.setProperty(MarshallerProperties.MEDIA_TYPE, "application/json"); marshaller.setProperty(MarshallerProperties.JSON_INCLUDE_ROOT, false); marshaller.marshal(root, System.out); } } Further Reading If you enjoyed this post then you may also be interested in: MOXy's @XmlVariableNode - JSON Schema Example Specifying EclipseLink MOXy as your JAXB Provider JAXB and java.util.Map JSON Binding with EclipseLink MOXy - Twitter Example

This post comes from Vladimir Šor at the Plumbr blog. Some of you have been there. You have added -Xmx option to your startup scripts and sat back relaxed knowing that there is no way your Java process is going to eat up more memory than your fine-tuned option had permitted. And then you were up for a nasty surprise. Either by yourself by checking a process table in your development / test box or if things got really bad then by operations who calls you in the middle of the night telling that the 4G memory you had asked for the production is exhausted. And that the application just died. So what the heck is happening under the hood? Why is the process consuming more memory than you allocated? Is it a bug or something completely normal? Bear with me and I will guide you through what is happening. First of all, part of it can definitely be a malicious native code leaking memory. But on 99% of the cases it is completely normal behaviour of the JVM. What you have specified via the -Xmx switches is limiting the memory consumed by your application heap. Besides heap there are other regions in memory which your application is using under the hood – namely permgen and stack sizes. So in order to limit those you should also specify the -XX:MaxPermSize and -Xss options respectively. In a short, you can predict your application memory usage with the following formula Max memory = [-Xmx] + [-XX:MaxPermSize] + number_of_threads * [-Xss] But besides the memory consumed by your application, the JVM itself also needs some elbow room. The need for it derives from several different reasons: Garbage collection. As you might recall, Java is a garbage collected language. In order for the garbage collector to know which objects are eligible for collection, it needs to keep track of the object graphs. So this is one part of the memory lost for this internal bookkeeping. Especially G1 is known for its excessive appetite for additional memory, so be aware of this. JIT optimization. Java Virtual Machine optimizes the code during the runtime. Again, to know which parts to optimize it needs to keep track of the execution of certain code parts. So again, you are going to lose memory. Off-heap allocations. If you happen to use off-heap memory, for example while using direct or mapped ByteBuffers yourself or via some clever 3rd party API then voila – you are extending your heap to something you actually cannot control via JVM configuration. JNI code. When you are using native code for example in the format of Type 2 database drivers then again, you are loading code in the native memory. Metaspace. If you are an early adopter of Java 8, you are using metaspace instead of the good old permgen to store class declarations. This is unlimited and in a native part of the JVM. You can end up using memory for other reasons than listed above as well, but I hope I managed to convince you that there is a significant amount of memory eaten up by the JVM internals. But is there a way to predict how much memory is actually going to be needed? Or at least understand where it disappears in order to optimize? As we have found out via painful experience – it is not possible to predict it with a reasonable precision. The JVM overhead can range from anything between just a few percentages to several hundred %. Your best friend is again the good old trial and error. So you need to run your application with loads similar to production environment and measure. Measuring the additional overhead is trivial – just monitor the process with the OS built-in tools (top on Linux, Activity Monitor on OS X, Task Manager on Windows) to find out the real memory consumption. Subtract the heap and permgen sizes from the real consumption and you see the overhead posed. Now if you need to reduce to overhead you would like to understand where it actually disappears. We have found vmmap on Mac OS X and pmap on Linux to be a truly helpful tools in this case. We have not used the vmmap port to Windows by ourselves, but it seems there is a tool for Windows fanboys as well. The following example illustrates this situation. I have launched my Jetty with the following startup parameters: -Xmx168m -Xms168m -XX:PermSize=32m -XX:MaxPermSize=32m -Xss1m Knowing that I have 30 threads launched in my application I might expect that my memory usage does not exceed 230M no matter what. But now when I look at the Activity Monitor on my Mac OS X, I see something different The real memory usage has exceeded 320M. Now digging under the hood how the process with the help of the vmmap output we start to understand where the memory is disappearing. Lets go through some samples: The following says we have lost close to 2MB is lost to memory mapped rt.jar library. mapped file 00000001178b9000-0000000117a88000 [ 1852K] r--/r-x SM=ALI /Library/Java/JavaVirtualMachines/jdk1.7.0_21.jdk/Contents/Home/jre/lib/rt.jar - Next section explains that we are using ~6MB for a particular Dynamic Library loaded __TEXT 0000000104573000-0000000104c00000 [ 6708K] r-x/rwx SM=COW /Library/Java/JavaVirtualMachines/jdk1.7.0_21.jdk/Contents/Home/jre/lib/server/libjvm.dylib - See more at: http://plumbr.eu/blog/why-does-my-java-process-consume-more-memory-than-xmx?utm_source=rss&utm_medium=rss&utm_campaign=rss20130618#sthash.G8fx60eX.dpuf And here we have threads no 25-30 each allocating 1MB for their stacks and stack guards Stack 000000011a5f1000-000000011a6f0000 [ 1020K] rw-/rwx SM=ZER thread 25 Stack 000000011aa8c000-000000011ab8b000 [ 1020K] rw-/rwx SM=ZER thread 27 Stack 000000011ab8f000-000000011ac8e000 [ 1020K] rw-/rwx SM=ZER thread 28 Stack 000000011ac92000-000000011ad91000 [ 1020K] rw-/rwx SM=ZER thread 29 Stack 000000011af0f000-000000011b00e000 [ 1020K] rw-/rwx SM=ZER thread 30 - See more at: http://plumbr.eu/blog/why-does-my-java-process-consume-more-memory-than-xmx?utm_source=rss&utm_medium=rss&utm_campaign=rss20130618#sthash.G8fx60eX.dpuf STACK GUARD 000000011a5ed000-000000011a5ee000 [ 4K] ---/rwx SM=NUL stack guard for thread 25 STACK GUARD 000000011aa88000-000000011aa89000 [ 4K] ---/rwx SM=NUL stack guard for thread 27 STACK GUARD 000000011ab8b000-000000011ab8c000 [ 4K] ---/rwx SM=NUL stack guard for thread 28 STACK GUARD 000000011ac8e000-000000011ac8f000 [ 4K] ---/rwx SM=NUL stack guard for thread 29 STACK GUARD 000000011af0b000-000000011af0c000 [ 4K] ---/rwx SM=NUL stack guard for thread 30 - See more at: http://plumbr.eu/blog/why-does-my-java-process-consume-more-memory-than-xmx?utm_source=rss&utm_medium=rss&utm_campaign=rss20130618#sthash.G8fx60eX.dpuf I hope I managed to shed some light upon the tricky task of predicting and measuring the actual memory consumption. If you enjoyed the content – subscribe to our RSS feed or start following us in Twitter to be notified on future interesting posts.

There are two ways to speedup UNIONs in a MySQL database. First use UNION ALL if at all possible, and second try to push down your conditions. 1. UNION ALL is much faster than UNION How does a UNION work? Imagine you have two tables for shirts. The short_sleeve table looks like this: blue green gray black And long_sleeve another that looks like this: red green yellow blue Related: Why Generalists are Better at Scaling the Web If you UNION those two tables, first MySQL will sort the combined set into a temp table like this: black blue blue gray green green red yellow Once it’s done this sort, it can easily remove the duplicate blue & duplicate green for this resulting set: black blue gray green red yellow See also: Mythical MySQL DBA – the talent drought. Why does it do this? UNION is defined that way in SQL. Duplicates must be removed and this is an efficient way for the MySQL engine to remove them. Combine results, sort, remove duplicates and return the set. Queries with UNION can be accelerated in two ways. Switch to UNION ALL or try to push ORDER BY, LIMIT and WHERE conditions inside each subquery. You’ll be glad you did! What if we did UNION ALL? The result would look like this: blue green gray black red green yellow blue Read this: MySQL DBA Interview & Hiring Guide. It doesn’t have to sort, and doesn’t have to remove duplicates. If you imagine combining two 10 million row tables, and don’t have to sort, this speedup can be HUGE. 2. Use Push-down Conditions to speedup UNION in MySQL Imagine with our example above the shirts have a design date, the year they were released. Yes we’re keeping this example very simple to illustrate the concept. Here is the short_sleeve table: blue 2013 green 2013 green 2012 gray 2011 black 2009 black 2011 And long_sleeve table looks like this: red 2012 red 2013 green 2011 yellow 2010 blue 2011 For 2013 designs could combine them like this: (SELECT type, release FROM short_sleeve) UNION (SELECT type, release FROM long_sleeve); WHERE release >=2013; See also: 5 More Things Deadly to Scalability and the original 5 Things Toxic to Scalability.. Here the WHERE clause works on this 11 record temp table: black 2009 black 2011 blue 2011 blue 2013 gray 2011 green 2013 green 2012 green 2011 red 2012 red 2013 yellow 2010 But it would be much faster to move the WHERE inside each subquery like this: (SELECT type, release FROM short_sleeve WHERE release >=2013) UNION (SELECT type, release FROM long_sleeve WHERE release >=2013); That would be operating on a combined 3 record table. Faster to sort & remove duplicates. Smaller result sets cache better too, providing a pay forward dividend. That’s what performance optimization is all about! Read this: RDS or MySQL – 10 Use Cases. Remember multi-million row sets in each part of this query will quickly illustrate the optimization. We’re using very small results to make visualizing easier. You can also use this optimization for ORDER BY and for LIMIT conditions. By reducing the number of records returned by EACH PART of the UNION, you reduce the work that happens at the stage where they are all combined. If you’re seeing some UNION queries in your slow query log, I suggest you try this optimization out and see if you can tweak it.

Andres and I recently found ourselves wanting to delete a remote branch which had the same name as a tag and therefore the normal way of doing that wasn’t worked out as well as we’d hoped. I created a dummy repository to recreate the state we’d got ourselves into: $ echo "mark" > README $ git commit -am "readme" $ echo "for the branch" >> README $ git commit -am "for the branch" $ git checkout -b same Switched to a new branch 'same' $ git push origin same Counting objects: 5, done. Writing objects: 100% (3/3), 263 bytes, done. Total 3 (delta 0), reused 0 (delta 0) To ssh://[email protected]/markhneedham/branch-tag-test.git * [new branch] same -> same $ git checkout master $ echo "for the tag" >> README $ git commit -am "for the tag" $ git tag same $ git push origin refs/tags/same Counting objects: 5, done. Writing objects: 100% (3/3), 266 bytes, done. Total 3 (delta 0), reused 0 (delta 0) To ssh://[email protected]/markhneedham/branch-tag-test.git * [new tag] same -> same We wanted to delete the remote ‘same’ branch and the following command would work if we hadn’t created a tag with the same name. Instead it throws an error: $ git push origin :same error: dst refspec same matches more than one. error: failed to push some refs to 'ssh://[email protected]/markhneedham/branch-tag-test.git' We learnt that what we needed to do was refer to the full path for the branch when trying to delete it remotely: $ git push origin :refs/heads/same To ssh://[email protected]/markhneedham/branch-tag-test.git - [deleted] same To delete the tag we could do the same thing: $ git push origin :refs/tags/same remote: warning: Deleting a non-existent ref. To ssh://[email protected]/markhneedham/branch-tag-test.git - [deleted] same Of course the tag and branch still exist locally: $ ls -alh .git/refs/heads/ total 16 drwxr-xr-x 4 markhneedham wheel 136B 13 Jun 23:09 . drwxr-xr-x 5 markhneedham wheel 170B 13 Jun 22:39 .. -rw-r--r-- 1 markhneedham wheel 41B 13 Jun 23:08 master -rw-r--r-- 1 markhneedham wheel 41B 13 Jun 23:08 same $ ls -alh .git/refs/tags/ total 8 drwxr-xr-x 3 markhneedham wheel 102B 13 Jun 23:08 . drwxr-xr-x 5 markhneedham wheel 170B 13 Jun 22:39 .. -rw-r--r-- 1 markhneedham wheel 41B 13 Jun 23:08 same So we got rid of them as well: $ git checkout master Switched to branch 'master' $ git branch -d same Deleted branch same (was 08ad88c). $ git tag -d same Deleted tag 'same' (was 1187891) And now they are gone: $ ls -alh .git/refs/heads/ total 8 drwxr-xr-x 3 markhneedham wheel 102B 13 Jun 23:16 . drwxr-xr-x 5 markhneedham wheel 170B 13 Jun 22:39 .. -rw-r--r-- 1 markhneedham wheel 41B 13 Jun 23:08 master $ ls -alh .git/refs/tags/ total 0 drwxr-xr-x 2 markhneedham wheel 68B 13 Jun 23:16 . drwxr-xr-x 5 markhneedham wheel 170B 13 Jun 22:39 .. Out of interest we’d ended up with this situation by mistake rather than by design but it was still fun to do a little bit of git digging to figure out how to solve the problem we’d created for ourselves.

What is an Object? An object is a collection of data and actions. An object is an instance of a class. Objects have states and behaviors. In the real-world, we can find so many objects around us, for example Cars, Birds, Humans etc. All these objects have a state and behavior. If we consider a Car then it have some data speed, lights on, direction, etc. and have some actions turn right, accelerate, turn lights on, etc. If you compare the java object with a real world object, both of them have similar characteristics. Java objects also have a state and behavior. A Java object's state is stored in fields and behavior is shown via methods. Technically speaking Car, Bird and Human are considered as Class in Java. Brian Christopher is an object of human and Vehicle XKMV-669 is the object of car. Creating Object Using new keyword is the most common way to create an object in java. Syntax:- ClassName Obj.Name = new ClassName(); // Human brianChristopher= new Human(); // Car vehicleXKMV_669 = new Car(); The first statement creates a new Human object and second statement creates Car object. This single statement performs three actions, Declaration, Instantiation, and Initialization. Here, Human brianChristopher is a variable declaration which simply declares to the compiler that the name brianChristopher will be used to refer to an object whose type is Human, the new operator instantiates the Human class (thereby creating a new Human object), and Human initializes the object. Object Lifecycle In Java, it has seven states in Object lifecycle. They are, Created In use Invisible Unreachable Collected Finalized De-allocated Created The following are the some actions performed when an object is created,New memory is allocated for an object. Once the object has been created, assuming that it is assigned to some variable and then it directly moves to the In Use state. In use Objects that are held by at least one strong reference are considered to be “In Use”. Invisible An object is in the “Invisible” state when there are no longer any strong references that are accessible to the program, even though there might still be references. Unreachable An object enters an “unreachable” state when no more strong references to it exist. When an object is unreachable then it is a state for collection. It is important to note that not just any strong reference will hold an object in memory. These must be references that chain from a garbage collection root. Garbage collection roots are a special class of variable that includes,Temporary variables on the stack Collected An object is in the “collected” state when the garbage collector has recognized an object as unreachable and readies it for final processing as a precursor to de-allocation. If the object has a finalize method, then it is marked for finalization. Finalized An object is in the “finalized” state if it is still unreachable after it’s finalize method, if any, has been run. A finalized object is awaiting de-allocation. If you are considering using a finalizer to ensure that important resources are freed in a timely manner, you might want to reconsider. To lengthening object lifetimes, finalize methods can increase object size. De-allocated The de-allocated state is the final step in garbage collection. If an object is still unreachable after all the above work has done, then this is the state for de-allocation. For more detailed discussion about Object Lifecycle with real-world examples download OCAJP 7 Training Lab from EPractize Labs.

In the code I have quite recently came across a really bad piece of code that based on class casting in terms of performing some actions on objects. Of course the code needed to be refactored but sometimes you can't do it / or don't want to do it (and it should be understandable) if first you don't have unit tests of that functionality. In the following post I will show how to test such code, how to refactor it and in fact what I think about such code ;) Let's take a look at the project structure: As presented in the post regarding Mocktio RETURNS_DEEP_STUBS Answer for JAXB yet again we have the JAXB generated classes by the JAXB compiler in thecom.blogspot.toomuchcoding.model package. Let's ommit the discussion over the pom.xml file since it's exactly the same as in the previous post. In the com.blogspot.toomuchcoding.adapter package we have adapters over the JAXB PlayerDetails class that provides access to the Player interface. There is the CommonPlayerAdapter.java package com.blogspot.toomuchcoding.adapter; import com.blogspot.toomuchcoding.model.Player; import com.blogspot.toomuchcoding.model.PlayerDetails; /** * User: mgrzejszczak * Date: 09.06.13 * Time: 15:42 */ public class CommonPlayerAdapter implements Player { private final PlayerDetails playerDetails; public CommonPlayerAdapter(PlayerDetails playerDetails){ this.playerDetails = playerDetails; } @Override public void run() { System.out.printf("Run %s. Run!%n", playerDetails.getName()); } public PlayerDetails getPlayerDetails() { return playerDetails; } } DefencePlayerAdapter.java package com.blogspot.toomuchcoding.adapter; import com.blogspot.toomuchcoding.model.DJ; import com.blogspot.toomuchcoding.model.DefensivePlayer; import com.blogspot.toomuchcoding.model.JavaDeveloper; import com.blogspot.toomuchcoding.model.PlayerDetails; /** * User: mgrzejszczak * Date: 09.06.13 * Time: 15:42 */ public class DefencePlayerAdapter extends CommonPlayerAdapter implements DefensivePlayer, DJ, JavaDeveloper { public DefencePlayerAdapter(PlayerDetails playerDetails){ super(playerDetails); } @Override public void defend(){ System.out.printf("Defence! %s. Defence!%n", getPlayerDetails().getName()); } @Override public void playSomeMusic() { System.out.println("Oops I did it again...!"); } @Override public void doSomeSeriousCoding() { System.out.println("System.out.println(\"Hello world\");"); } } OffensivePlayerAdapter.java package com.blogspot.toomuchcoding.adapter; import com.blogspot.toomuchcoding.model.OffensivePlayer; import com.blogspot.toomuchcoding.model.PlayerDetails; /** * User: mgrzejszczak * Date: 09.06.13 * Time: 15:42 */ public class OffensivePlayerAdapter extends CommonPlayerAdapter implements OffensivePlayer { public OffensivePlayerAdapter(PlayerDetails playerDetails){ super(playerDetails); } @Override public void shoot(){ System.out.printf("%s Shooooot!.%n", getPlayerDetails().getName()); } } Ok, now let's go to the more interesting part. Let us assume that we have a very simple factory of players: PlayerFactoryImpl.java package com.blogspot.toomuchcoding.factory; import com.blogspot.toomuchcoding.adapter.CommonPlayerAdapter; import com.blogspot.toomuchcoding.adapter.DefencePlayerAdapter; import com.blogspot.toomuchcoding.adapter.OffensivePlayerAdapter; import com.blogspot.toomuchcoding.model.Player; import com.blogspot.toomuchcoding.model.PlayerDetails; import com.blogspot.toomuchcoding.model.PositionType; /** * User: mgrzejszczak * Date: 09.06.13 * Time: 15:53 */ public class PlayerFactoryImpl implements PlayerFactory { @Override public Player createPlayer(PositionType positionType) { PlayerDetails player = createCommonPlayer(positionType); switch (positionType){ case ATT: return new OffensivePlayerAdapter(player); case MID: return new OffensivePlayerAdapter(player); case DEF: return new DefencePlayerAdapter(player); case GK: return new DefencePlayerAdapter(player); default: return new CommonPlayerAdapter(player); } } private PlayerDetails createCommonPlayer(PositionType positionType){ PlayerDetails playerDetails = new PlayerDetails(); playerDetails.setPosition(positionType); return playerDetails; } } Ok so we have the factory that builds Players. Let's take a look at the Service that uses the factory: PlayerServiceImpl.java package com.blogspot.toomuchcoding.service; import com.blogspot.toomuchcoding.factory.PlayerFactory; import com.blogspot.toomuchcoding.model.*; /** * User: mgrzejszczak * Date: 08.06.13 * Time: 19:02 */ public class PlayerServiceImpl implements PlayerService { private PlayerFactory playerFactory; @Override public Player playAGameWithAPlayerOfPosition(PositionType positionType) { Player player = playerFactory.createPlayer(positionType); player.run(); performAdditionalActions(player); return player; } private void performAdditionalActions(Player player) { if(player instanceof OffensivePlayer){ OffensivePlayer offensivePlayer = (OffensivePlayer) player; performAdditionalActionsForTheOffensivePlayer(offensivePlayer); }else if(player instanceof DefensivePlayer){ DefensivePlayer defensivePlayer = (DefensivePlayer) player; performAdditionalActionsForTheDefensivePlayer(defensivePlayer); } } private void performAdditionalActionsForTheOffensivePlayer(OffensivePlayer offensivePlayer){ offensivePlayer.shoot(); } private void performAdditionalActionsForTheDefensivePlayer(DefensivePlayer defensivePlayer){ defensivePlayer.defend(); try{ DJ dj = (DJ)defensivePlayer; dj.playSomeMusic(); JavaDeveloper javaDeveloper = (JavaDeveloper)defensivePlayer; javaDeveloper.doSomeSeriousCoding(); }catch(ClassCastException exception){ System.err.println("Sorry, I can't do more than just play football..."); } } public PlayerFactory getPlayerFactory() { return playerFactory; } public void setPlayerFactory(PlayerFactory playerFactory) { this.playerFactory = playerFactory; } } Let's admit it... this code is bad. Internally when you look at it (regardless of the fact whether it used instance of operator or not) you feel that it is evil :) As you can see in the code we have some class casts going on... How on earth can we test it? In the majority of testing frameworks you can't do such class casts on mocks since they are built with the CGLIB library and there can be some ClassCastExceptions thrown. You could still not return mocks and real implementations (assuming that those will not perform any ugly stuff in the construction process) and it could actually work but still - this is bad code :P Mockito comes to the rescue (although you shouldn't overuse this feature - in fact if you need to use it please consider refactoring it) with its extraInterfaces feature: extraInterfaces MockSettings extraInterfaces(java.lang.Class... interfaces) Specifies extra interfaces the mock should implement. Might be useful for legacy code or some corner cases. For background, see issue 51 hereThis mysterious feature should be used very occasionally. The object under test should know exactly its collaborators & dependencies. If you happen to use it often than please make sure you are really producing simple, clean & readable code. Examples: Foo foo = mock(Foo.class, withSettings().extraInterfaces(Bar.class, Baz.class)); //now, the mock implements extra interfaces, so following casting is possible: Bar bar = (Bar) foo; Baz baz = (Baz) foo; Parameters:interfaces - extra interfaces the should implement. Returns:settings instance so that you can fluently specify other settings Now let's take a look at the test: PlayerServiceImplTest.java package com.blogspot.toomuchcoding.service; import com.blogspot.toomuchcoding.factory.PlayerFactory; import com.blogspot.toomuchcoding.model.*; import org.junit.Test; import org.junit.runner.RunWith; import org.mockito.InjectMocks; import org.mockito.Mock; import org.mockito.invocation.InvocationOnMock; import org.mockito.runners.MockitoJUnitRunner; import org.mockito.stubbing.Answer; import static org.hamcrest.CoreMatchers.is; import static org.junit.Assert.assertThat; import static org.mockito.BDDMockito.*; /** * User: mgrzejszczak * Date: 08.06.13 * Time: 19:26 */ @RunWith(MockitoJUnitRunner.class) public class PlayerServiceImplTest { @Mock PlayerFactory playerFactory; @InjectMocks PlayerServiceImpl objectUnderTest; @Mock(extraInterfaces = {DJ.class, JavaDeveloper.class}) DefensivePlayer defensivePlayerWithDjAndJavaDevSkills; @Mock DefensivePlayer defensivePlayer; @Mock OffensivePlayer offensivePlayer; @Mock Player commonPlayer; @Test public void shouldReturnOffensivePlayerThatRan() throws Exception { //given given(playerFactory.createPlayer(PositionType.ATT)).willReturn(offensivePlayer); //when Player createdPlayer = objectUnderTest.playAGameWithAPlayerOfPosition(PositionType.ATT); //then assertThat(createdPlayer == offensivePlayer, is(true)); verify(offensivePlayer).run(); } @Test public void shouldReturnDefensivePlayerButHeWontBeADjNorAJavaDev() throws Exception { //given given(playerFactory.createPlayer(PositionType.GK)).willReturn(defensivePlayer); //when Player createdPlayer = objectUnderTest.playAGameWithAPlayerOfPosition(PositionType.GK); //then assertThat(createdPlayer == defensivePlayer, is(true)); verify(defensivePlayer).run(); verify(defensivePlayer).defend(); verifyNoMoreInteractions(defensivePlayer); } @Test public void shouldReturnDefensivePlayerBeingADjAndAJavaDev() throws Exception { //given given(playerFactory.createPlayer(PositionType.GK)).willReturn(defensivePlayerWithDjAndJavaDevSkills); doAnswer(new Answer

NetBeans IDE 7.3.1 is an update to NetBeans IDE 7.3 and includes the following highlights: Support for Java EE 7 development Deployment to GlassFish 4 Support for major Java EE 7 specifications: JSF 2.2, JPA 2.1, JAX-RS 2.0, WebSocket 1.0 and more Support for WebLogic 12.1.2 and JBoss 7.x Integration of recent patches There are two ways to get the recent changes: To use the new Java EE 7 support, it is recommended to download and install NetBeans IDE 7.3.1. To get only the integration of recent patches: Launch your current installation of NetBeans IDE 7.3. An update notification will appear in the IDE. Click the notification box to install the updates. OR to perform the update manually, in the IDE select Help-->Check for Updates. NetBeans IDE 7.3.1 is available in English, Brazilian Portuguese, Japanese, Russian, and Simplified Chinese.

RPC/encoded is a vestige from before SOAP objects were defined with XML Schema. It’s not widely supported anymore. You will need to generate the stubs using Apache Axis 1.0, which is from the same era. java org.apache.axis.wsdl.WSDL2Java http://someurl?WSDL You will need the following jars or equivalents in the -cp classpath param: axis-1.4.jar commons-logging-1.1.ja commons-discovery-0.2.jar jaxrpc-1.1.jar saaj-1.1.jar wsdl4j-1.4.jar activation-1.1.jar mail-1.4.jar This will generate similar stubs to wsimport. Alternatively, if you are not using the parts of the schema that require rpc/encoded, you can download a copy of the WSDL and comment out those bits. Then run wsimport against the local file. If you look at the WSDL, the following bits are using rpc/encoded: Sources 1. http://bitkickers.blogspot.com/2008/12/rpcencoded-web-services-on-java-16.html 2. http://stackoverflow.com/questions/412772/java-rpc-encoded-wsdls-are-not-supported-in-jaxws-2-0

Sorry for not having written for some time but I was busy with writing the JBoss Drools Refcard for DZone and I am in the middle of writing a book about Mockito so I don't have too much time left for blogging... Anyway quite recently on my current project I had an interesting situation regarding unit testing with Mockito and JAXB structures. We have very deeply nested JAXB structures generated from schemas that are provided for us which means that we can't change it in anyway. Let's take a look at the project structure: The project structure is pretty simple - there is a Player.xsd schema file that thanks to using the jaxb2-maven-plugin produces the generated JAXB Java classes corresponding to the schema in the target/jaxb/ folder in the appropriate package that is defined in the pom.xml. Speaking of which let's take a look at the pom.xml file. The pom.xml : 4.0.0 com.blogspot.toomuchcoding mockito-deep_stubs 0.0.1-SNAPSHOT UTF-8 1.6 1.6 spring-release http://maven.springframework.org/release maven-us-nuxeo https://maven-us.nuxeo.org/nexus/content/groups/public junit junit 4.10 org.mockito mockito-all 1.9.5 test org.apache.maven.plugins maven-compiler-plugin 2.5.1 org.codehaus.mojo jaxb2-maven-plugin 1.5 xjc xjc com.blogspot.toomuchcoding.model ${project.basedir}/src/main/resources/xsd Apart from the previously defined project dependencies, as mentioned previously in the jaxb2-maven-plugin in the configuration node you can define the packageName value that defines to which package should the JAXB classes be generated basing on the schemaDirectory value where the plugin can find the proper schema files. Speaking of which let's check the Player.xsd schema file (simillar to the one that was present in the Spring JMS automatic message conversion article of mine): As you can see I'm defining some complex types that even though might have no business sense but you can find such examples in the real life :) Let's find out how the method that we would like to test looks like. Here we have the PlayerServiceImpl that implements the PlayerService interface: package com.blogspot.toomuchcoding.service; import com.blogspot.toomuchcoding.model.PlayerDetails; /** * User: mgrzejszczak * Date: 08.06.13 * Time: 19:02 */ public class PlayerServiceImpl implements PlayerService { @Override public boolean isPlayerOfGivenCountry(PlayerDetails playerDetails, String country) { String countryValue = playerDetails.getClubDetails().getCountry().getCountryCode().getCountryCode().value(); return countryValue.equalsIgnoreCase(country); } } We are getting the nested elements from the JAXB generated classes. Although it violates the Law of Demeter it is quite common to call methods of structures because JAXB generated classes are in fact structures so in fact I fully agree with Martin Fowler that it should be called the Suggestion of Demeter. Anyway let's see how you could test the method: @Test public void shouldReturnTrueIfCountryCodeIsTheSame() throws Exception { //given PlayerDetails playerDetails = new PlayerDetails(); ClubDetails clubDetails = new ClubDetails(); CountryDetails countryDetails = new CountryDetails(); CountryCodeDetails countryCodeDetails = new CountryCodeDetails(); playerDetails.setClubDetails(clubDetails); clubDetails.setCountry(countryDetails); countryDetails.setCountryCode(countryCodeDetails); countryCodeDetails.setCountryCode(CountryCodeType.ENG); //when boolean playerOfGivenCountry = objectUnderTest.isPlayerOfGivenCountry(playerDetails, COUNTRY_CODE_ENG); //then assertThat(playerOfGivenCountry, is(true)); } The function checks if, once you have the same Country Code, you get a true boolean from the method. The only problem is the amount of sets and instantiations that take place when you want to create the input message. In our projects we have twice as many nested elements so you can only imagine the number of code that we would have to produce to create the input object... So what can be done to improve this code? Mockito comes to the rescue to together with the RETURN_DEEP_STUBS default answer to the Mockito.mock(...) method: @Test public void shouldReturnTrueIfCountryCodeIsTheSameUsingMockitoReturnDeepStubs() throws Exception { //given PlayerDetails playerDetailsMock = mock(PlayerDetails.class, RETURNS_DEEP_STUBS); CountryCodeType countryCodeType = CountryCodeType.ENG; when(playerDetailsMock.getClubDetails().getCountry().getCountryCode().getCountryCode()).thenReturn(countryCodeType); //when boolean playerOfGivenCountry = objectUnderTest.isPlayerOfGivenCountry(playerDetailsMock, COUNTRY_CODE_ENG); //then assertThat(playerOfGivenCountry, is(true)); } So what happened here is that you use the Mockito.mock(...) method and provide the RETURNS_DEEP_STUBS answer that will create mocks automatically for you. Mind you that Enums can't be mocked that's why you can't write in the Mockito.when(...) functionplayerDetailsMock.getClubDetails().getCountry().getCountryCode().getCountryCode().getValue(). Summing it up you can compare the readability of both tests and see how clearer it is to work with JAXB structures by using Mockito RETURNS_DEEP_STUBS default answer. Naturally sources for this example are available at BitBucket and GitHub.

Elasticsearch is a flexible and powerful open source, distributed real-time search and analytics engine for the cloud based on Apache Lucene which provides full text search capabilities. It is document oriented and schema free. Asciidoctor is a pure Ruby processor for converting AsciiDoc source files and strings into HTML 5, DocBook 4.5 and other formats. Apart of Asciidoctor Ruby part, there is an Asciidoctor-java-integration project which let us call Asciidoctor functions from Java without noticing that Ruby code is being executed. In this post we are going to see how we can use Elasticsearch over AsciiDocdocuments to make them searchable by their header information or by their content. Let's add required dependencies: junit junit 4.11 test com.googlecode.lambdaj lambdaj 2.3.3 org.elasticsearch elasticsearch 0.90.1 org.asciidoctor asciidoctor-java-integration 0.1.3 Lambdaj library is used to convert AsciiDoc files to a json documents. Now we can start an Elasticsearch instance which in our case it is going to be an embedded instance. node = nodeBuilder().local(true).node(); Next step is parse AsciiDoc document header, read its content and convert them into a json document. An example of json document stored in Elasticsearch can be: { "title":"Asciidoctor Maven plugin 0.1.2 released!", "authors":[ { "author":"Jason Porter", "email":"[email protected]" } ], "version":null, "content":"= Asciidoctor Maven plugin 0.1.2 released!.....", "tags":[ "release", "plugin" ] } And for converting an AsciiDoc File to a json document we are going to useXContentBuilder class which is provided by ElasticsearchJava API to create jsondocuments programmatically. package com.lordofthejars.asciidoctor; import static org.elasticsearch.common.xcontent.XContentFactory.*; import java.io.File; import java.io.FileInputStream; import java.io.FileNotFoundException; import java.io.IOException; import java.util.List; import org.asciidoctor.Asciidoctor; import org.asciidoctor.Author; import org.asciidoctor.DocumentHeader; import org.asciidoctor.internal.IOUtils; import org.elasticsearch.common.xcontent.XContentBuilder; import ch.lambdaj.function.convert.Converter; public class AsciidoctorFileJsonConverter implements Converter { private Asciidoctor asciidoctor; public AsciidoctorFileJsonConverter() { this.asciidoctor = Asciidoctor.Factory.create(); } public XContentBuilder convert(File asciidoctor) { DocumentHeader documentHeader = this.asciidoctor.readDocumentHeader(asciidoctor); XContentBuilder jsonContent = null; try { jsonContent = jsonBuilder() .startObject() .field("title", documentHeader.getDocumentTitle()) .startArray("authors"); Author mainAuthor = documentHeader.getAuthor(); jsonContent.startObject() .field("author", mainAuthor.getFullName()) .field("email", mainAuthor.getEmail()) .endObject(); List authors = documentHeader.getAuthors(); for (Author author : authors) { jsonContent.startObject() .field("author", author.getFullName()) .field("email", author.getEmail()) .endObject(); } jsonContent.endArray() .field("version", documentHeader.getRevisionInfo().getNumber()) .field("content", readContent(asciidoctor)) .array("tags", parseTags((String)documentHeader.getAttributes().get("tags"))) .endObject(); } catch (IOException e) { throw new IllegalArgumentException(e); } return jsonContent; } private String[] parseTags(String tags) { tags = tags.substring(1, tags.length()-1); return tags.split(", "); } private String readContent(File content) throws FileNotFoundException { return IOUtils.readFull(new FileInputStream(content)); } } Basically we are building the json document by calling startObject methods to start a new object, field method to add new fields, and startArray to start an array. Then this builder will be used to render the equivalent object in json format. Notice that we are using readDocumentHeader method from Asciidoctor class which returns header attributes from AsciiDoc file without reading and rendering the whole document. And finally content field is set with all document content. And now we are ready to start indexing documents. Note that populateData method receives as parameter a Client object. This object is from Elasticsearch Java APIand represents a connection to Elasticsearch database. import static ch.lambdaj.Lambda.convert; //.... private void populateData(Client client) throws IOException { List asciidoctorFiles = new ArrayList() {{ add(new File("target/test-classes/java_release.adoc")); add(new File("target/test-classes/maven_release.adoc")); }; List jsonDocuments = convertAsciidoctorFilesToJson(asciidoctorFiles); for (int i=0; i < jsonDocuments.size(); i++) { client.prepareIndex("docs", "asciidoctor", Integer.toString(i)).setSource(jsonDocuments.get(i)).execute().actionGet(); } client.admin().indices().refresh(new RefreshRequest("docs")).actionGet(); } private List convertAsciidoctorFilesToJson(List asciidoctorFiles) { return convert(asciidoctorFiles, new AsciidoctorFileJsonConverter()); } It is important to note that the first part of the algorithm is converting all our AsciiDocfiles (in our case two) to XContentBuilder instances by using previous converter class and the method convert of Lambdaj project. If you want you can take a look to both documents used in this example in https://github.com/asciidoctor/asciidoctor.github.com/blob/develop/news/asciidoctor-java-integration-0-1-3-released.adoc and https://github.com/asciidoctor/asciidoctor.github.com/blob/develop/news/asciidoctor-maven-plugin-0-1-2-released.adoc. Next part is inserting documents inside one index. This is done by using prepareIndexmethod, which requires an index name (docs), an index type (asciidoctor), and the idof the document being inserted. Then we call setSource method which transforms theXContentBuilder object to json, and finally by calling execute().actionGet(), data is sent to database. The final step is only required because we are using an embedded instance ofElasticsearch (in production this part should not be required), which refresh the indexes by calling refresh method. After that point we can start querying Elasticsearch for retrieving information from our AsciiDoc documents. Let's start with very simple example, which returns all documents inserted: SearchResponse response = client.prepareSearch().execute().actionGet(); Next we are going to search for all documents that has been written by Alex Sotowhich in our case is one. import static org.elasticsearch.index.query.QueryBuilders.matchQuery; //.... QueryBuilder matchQuery = matchQuery("author", "Alex Soto"); QueryBuilder matchQuery = matchQuery("author", "Alexander Soto"); Note that I am searching for field author the string Alex Soto, which returns only one. The other document is written by Jason. But it is interesting to say that if you search for Alexander Soto, the same document will be returned; Elasticsearch is smart enough to know that Alex and Alexander are very similar names so it returns the document too. More queries, how about finding documents written by someone who is called Alex, but not Soto. import static org.elasticsearch.index.query.QueryBuilders.fieldQuery; //.... QueryBuilder matchQuery = fieldQuery("author", "+Alex -Soto"); And of course no results are returned in this case. See that in this case we are using afield query instead of a term query, and we use +, and - symbols to exclude and include words. Also you can find all documents which contains the word released on title. import static org.elasticsearch.index.query.QueryBuilders.matchQuery; //.... QueryBuilder matchQuery = matchQuery("title", "released"); And finally let's find all documents that talks about 0.1.2 release, in this case only one document talks about it, the other one talks about 0.1.3. QueryBuilder matchQuery = matchQuery("content", "0.1.2"); Now we only have to send the query to Elasticsearch database, which is done by using prepareSearch method. SearchResponse response = client.prepareSearch("docs") .setTypes("asciidoctor") .setQuery(matchQuery) .execute() .actionGet(); SearchHits hits = response.getHits(); for (SearchHit searchHit : hits) { System.out.println(searchHit.getSource().get("content")); } Note that in this case we are printing the AsciiDoc content through console, but you could use asciidoctor.render(String content, Options options) method to render the content into required format. So in this post we have seen how to index documents using Elasticsearch, how to get some important information from AsciiDoc files using Asciidoctor-java-integration project, and finally how to execute some queries to inserted documents. Of course there are more kind of queries in Elasticsearch, but the intend of this post wasn't to explore all possibilities of Elasticsearch. Also as corollary, note how important it is using AsciiDoc format for writing your documents. Without much effort you can build a search engine for your documentation. On the other side, imagine all code that would be required to implement the same using any proprietary binary format like Microsoft Word. So we have shown another reason to use AsciiDoc instead of other formats.

I’ve recently had the need to automate configuration of Nginx on an Ubuntu server. Of course, in UNIX land we like to use SSH (Secure Shell) to log into our servers and manage them remotely. Wouldn’t it be nice, I thought, if there was a managed SSH library somewhere so that I could automate logging onto my Ubuntu server, run various commands and transfer files. A short Google turned up SSH.NET by the somewhat mysterious Olegkap (at least I couldn’t find out anything else about them) which turned out to be just what I wanted. Here’s the blurb on the CodePlex site: “This project was inspired by Sharp.SSH library which was ported from java and it seems like was not supported for quite some time. This library is complete rewrite using .NET 4.0, without any third party dependencies and to utilize the parallelism as much as possible to allow best performance I can get.” It does exactly what it says on the tin. It’s on NuGet, so you can grab it with: PM> Install-Package SSH.NET Here’s how you run a remote command. First you need to build a ConnectionInfo object: public ConnectionInfo CreateConnectionInfo() { const string privateKeyFilePath = @"C:\some\private\key.pem"; ConnectionInfo connectionInfo; using (var stream = new FileStream(privateKeyFilePath, FileMode.Open, FileAccess.Read)) { var privateKeyFile = new PrivateKeyFile(stream); AuthenticationMethod authenticationMethod = new PrivateKeyAuthenticationMethod("ubuntu", privateKeyFile); connectionInfo = new ConnectionInfo( "my.server.com", "ubuntu", authenticationMethod); } return connectionInfo; } Then you simply create an SshClient instance and run commands: public void Connect() { using (var ssh = new SshClient(CreateConnectionInfo())) { ssh.Connect(); var command = ssh.CreateCommand("uptime"); var result = command.Execute(); Console.Out.WriteLine(result); ssh.Disconnect(); } } Here I’m running the ‘uptime’ command which output this when I ran it just now: 14:37:46 up 22 days, 3:59, 0 users, load average: 0.08, 0.03, 0.05 To transfer a file, just use the ScpClient: public void GetConfigurationFiles() { using (var scp = new ScpClient(CreateNginxServerConnectionInfo())) { scp.Connect(); scp.Download("/etc/nginx/", new DirectoryInfo(@"D:\Temp\ScpDownloadTest")); scp.Disconnect(); } } Which grabs all my Nginx configuration and transfers it to a directory tree on my windows machine. All in all a very nice little library that’s been working well for me so far. Give it a try if you need to interact with a UNIX-like machine from .NET code.

about an hour ago i gave a presentation on indexeddb. one of the attendees asked about dates and being able to filter based on a date range. i told him that my assumption was that you would need to convert the dates into numbers and use a number-based range. turns out i was wrong. here is an example. i began by creating an objectstore that used an index on the created field. since our intent is to search via a date field, i decided "created" would be a good name. i also named my objectstore as "data". boring, but it works. var openrequest = indexeddb.open("idbpreso_date1",1); openrequest.onupgradeneeded = function(e) { var thisdb = e.target.result; if(!thisdb.objectstorenames.contains("data")) { var os = thisdb.createobjectstore("data", {autoincrement:true}); os.createindex("created", "created", {unique:false}); } } next - i built a simple way to seed data. i based on a button click event to add 10 objects. each object will have one property, created, and the date object will be based on a random date from now till 7 days in the future. function doseed() { var now = new date(); for(var i=0; i<10; i++) { var daydiff = getrandomint(1, 7); var thisdate = new date(); thisdate.setdate(now.getdate() + daydiff); db.transaction(["data"],"readwrite").objectstore("data").add({created:thisdate}); } } //credit: mozilla developer center function getrandomint (min, max) { return math.floor(math.random() * (max - min + 1)) + min; } note that since indexeddb calls are asynchronous, my code should handle updating the user to let them know when the operation is done. since this is just a quick demo though, and since that add operation will complete incredibly fast, i decided to not worry about it. so at this point we'd have an application that lets us add data containing a created property with a valid javascript date. note i didn't change it to milliseconds. i just passed it in as is. for the final portion i added two date fields on my page. in chrome this is rendered nicely: based on these, i can then create an indexeddb range of either bounds, lowerbounds, or upperbounds. i.e., give me crap either after a date, before a date, or inside a date range. function dosearch() { var fromdate = document.queryselector("#fromdate").value; var todate = document.queryselector("#todate").value; var range; if(fromdate == "" && todate == "") return; var transaction = db.transaction(["data"],"readonly"); var store = transaction.objectstore("data"); var index = store.index("created"); if(fromdate != "") fromdate = new date(fromdate); if(todate != "") todate = new date(todate); if(fromdate != "" && todate != "") { range = idbkeyrange.bound(fromdate, todate); } else if(fromdate == "") { range = idbkeyrange.upperbound(todate); } else { range = idbkeyrange.lowerbound(fromdate); } var s = ""; index.opencursor(range).onsuccess = function(e) { var cursor = e.target.result; if(cursor) { s += "key "+cursor.key+""; for(var field in cursor.value) { s+= field+"="+cursor.value[field]+""; } s+=""; cursor.continue(); } document.queryselector("#status").innerhtml = s; } } the only conversion required here was to take the user input and turn it into "real" date objects. once done, everything works great: you can run the full demo below.