Visa Card ^4[0-9]{12}(?:[0-9]{3})?$^5[1-5][0-9]{14}$ Amex Card ^3[47][0-9]{13}$ Carte Blanche Card ^389[0-9]{11}$ Diners Club Card ^3(?:0[0-5]|[68][0-9])[0-9]{11}$ Discover Card ^65[4-9][0-9]{13}|64[4-9][0-9]{13}|6011[0-9]{12}|(622(?:12[6-9]|1[3-9][0-9]|[2-8][0-9][0-9]|9[01][0-9]|92[0-5])[0-9]{10})$ JCB Card ^(?:2131|1800|35\d{3})\d{11}$ Visa Master Card ^(?:4[0-9]{12}(?:[0-9]{3})?|5[1-5][0-9]{14})$ Insta Payment Card ^63[7-9][0-9]{13}$ Laser Card ^(6304|6706|6709|6771)[0-9]{12,15}$ Maestro Card ^(5018|5020|5038|6304|6759|6761|6763)[0-9]{8,15}$ Solo Card ^(6334|6767)[0-9]{12}|(6334|6767)[0-9]{14}|(6334|6767)[0-9]{15}$ Switch Card ^(4903|4905|4911|4936|6333|6759)[0-9]{12}|(4903|4905|4911|4936|6333|6759)[0-9]{14}|(4903|4905|4911|4936|6333|6759)[0-9]{15}|564182[0-9]{10}|564182[0-9]{12}|564182[0-9]{13}|633110[0-9]{10}|633110[0-9]{12}|633110[0-9]{13}$ Union Pay Card ^(62[0-9]{14,17})$ KoreanLocalCard ^9[0-9]{15}$ BCGlobal ^(6541|6556)[0-9]{12}$ As you can see, regular expressions are incredibly powerful and the above examples are very basic. Regular expressions are essential for all sorts of application from web scraping, form validation and pattern matching. Hopefully you will find this resource useful and would come back to refer again and again.

Facts and Terminology As you probably know, Java uses UTF-16 to represent Strings. In order to understand the confusion about String.length(), you need to be familiar with some Encoding/Unicode terms. Code Point: A unique integer value which represents a character in the code space. Code Unit: A bit sequence used to encode characters (Code Points). One or more Code Units may be required to represent a Code Point. UTF-16 Unicode Code Points are logically divided into 17 planes. The first plane, the Basic Multilingual Plane (BMP) contains the “classic” characters (from U+0000 to U+FFFF). The other planes contain the supplementary characters (from U+10000 to U+10FFFF). Characters (Code Points) from the first plane are encoded in one 16-bit Code Unit with the same value. Supplementary characters (Code Points) are encoded in two Code Units (encoding-specific, see Wiki for the explanation). Example Character: A Unicode Code Point: U+0041 UTF-16 Code Unit(s): 0041 Character: Mathematical double-struck capital A Unicode Code Point: U+1D538 UTF-16 Code Unit(s): D835 DD38 As you can see here, there are characters which are encoded in two Code Units. String.length() Let’s take a look at the Javadoc of the length() method: public int length() Returns the length of this string. The length is equal to the number of Unicode code units in the string. So if you have one supplementary character which consists of two code units, the length of that single character is two. // Mathematical double-struck capital A String str = "\uD835\uDD38"; System.out.println(str); System.out.println(str.length()); //prints 2 Which is correct according to the documentation, but maybe it’s not expected. ~Solution You need to count the code points not the code units: String str = "\uD835\uDD38"; System.out.println(str); System.out.println(str.codePointCount(0, str.length())); See: codePointCount(int beginIndex, int endIndex) References/Sources The Java Language Specification Unicode Glossary: Code Point Wiki: Code Point Unicode Glossary: Code Unit Wiki: Code Unit Wiki: Unicode Wiki: UTF-16 Supplementary Characters in the Java Platform Wiki: Unicode Planes

The Map interface provides some handy new methods in JDK 8. Because the Map methods I cover in this post are implemented as default methods, all existing implementations of the Map interface enjoy the default behaviors defined in the default methods without any new code. The JDK 8 introduced Map methods covered in this post are getOrDefault(Object, V), putIfAbsent(K, V), remove(Object, Object), remove(Object, Object), replace(K, V), and replace(K, V, V). Example Map for Demonstrations I will be using the Map declared and initialized as shown in the following code throughout the examples in this blog post. The statesAndCapitals field is a class-level static field. I intentionally have only included a small subset of the fifty states in the United States for reading clarity and to allow easier demonstration of some of the new JDK 8 Map default methods. private final static Map statesAndCapitals; static { statesAndCapitals = new HashMap<>(); statesAndCapitals.put("Alaska", "Anchorage"); statesAndCapitals.put("California", "Sacramento"); statesAndCapitals.put("Colorado", "Denver"); statesAndCapitals.put("Florida", "Tallahassee"); statesAndCapitals.put("Nevada", "Las Vegas"); statesAndCapitals.put("New Mexico", "Sante Fe"); statesAndCapitals.put("Utah", "Salt Lake City"); statesAndCapitals.put("Wyoming", "Cheyenne"); } Map.getOrDefault(Object, V) Map's new method getOrDefault(Object, V) allows the caller to specify in a single statement to get the value of the map that corresponds to the provided key or else return a provided "default value" if no match is found for the provided key. The next code listing compares how checking for a value matching a provided key in a map or else using a default if no match is found was implemented before JDK 8 and how it can now be implemented with JDK 8. /* * Demonstrate Map.getOrDefault and compare to pre-JDK 8 approach. The JDK 8 * addition of Map.getOrDefault requires fewer lines of code than the * traditional approach and allows the returned value to be assigned to a * "final" variable. */ // pre-JDK 8 approach String capitalGeorgia = statesAndCapitals.get("Georgia"); if (capitalGeorgia == null) { capitalGeorgia = "Unknown"; } // JDK 8 approach final String capitalWisconsin = statesAndCapitals.getOrDefault("Wisconsin", "Unknown"); The Apache Commons class DefaultedMap provides functionality similar to the newMap.getOrDefault(Object, V) method. The Groovy GDK includes a similar method for Groovy,Map.get(Object, Object), but that one's behavior is a bit different because it not only returns the provided default if the "key" is not found, but also adds the key with the default value to the underlying map. Map.putIfAbsent(K, V) Map's new method putIfAbsent(K, V) has Javadoc advertising its default implementation equivalent: The default implementation is equivalent to, for this map: V v = map.get(key); if (v == null) v = map.put(key, value); return v; This is illustrated with another code sample that compares the pre-JDK 8 approach to the JDK 8 approach. /* * Demonstrate Map.putIfAbsent and compare to pre-JDK 8 approach. The JDK 8 * addition of Map.putIfAbsent requires fewer lines of code than the * traditional approach and allows the returned value to be assigned to a * "final" variable. */ // pre-JDK 8 approach String capitalMississippi = statesAndCapitals.get("Mississippi"); if (capitalMississippi == null) { capitalMississippi = statesAndCapitals.put("Mississippi", "Jackson"); } // JDK 8 approach final String capitalNewYork = statesAndCapitals.putIfAbsent("New York", "Albany"); Alternate solutions in the Java space before the addition of this putIfAbsent method are discussed in theStackOverflow thread Java map.get(key) - automatically do put(key) and return if key doesn't exist?. It's worth noting that before JDK 8, the ConcurrentMap interface (extends Map) already provided a putIfAbsent(K, V)method. Map.remove(Object, Object) Map's new remove(Object, Object) method goes beyond the long-available Map.remove(Object) method to remove a map entry only if both the provided key and provided value match an entry in the map (the previously available version only looked for a "key" match to remove). The Javadoc comment for this method explains the how the default method's implementation works in terms of equivalent pre-JDK 8 Java code: The default implementation is equivalent to, for this map: if (map.containsKey(key) && Objects.equals(map.get(key), value)) { map.remove(key); return true; } else return false; A concrete comparison of the new approach to the pre-JDK 8 approach is shown in the next code listing. /* * Demonstrate Map.remove(Object, Object) and compare to pre-JDK 8 approach. * The JDK 8 addition of Map.remove(Object, Object) requires fewer lines of * code than the traditional approach and allows the returned value to be * assigned to a "final" variable. */ // pre-JDK 8 approach boolean removed = false; if ( statesAndCapitals.containsKey("New Mexico") && Objects.equals(statesAndCapitals.get("New Mexico"), "Sante Fe")) { statesAndCapitals.remove("New Mexico", "Sante Fe"); removed = true; } // JDK 8 approach final boolean removedJdk8 = statesAndCapitals.remove("California", "Sacramento"); Map.replace(K, V) The first of the two new Map "replace" methods sets the specified value to be mapped to the specified key only if the specified key already exists with some mapped value. The Javadoc comment explains the Java equivalent of this default method implementation: The default implementation is equivalent to, for this map: if (map.containsKey(key)) { return map.put(key, value); } else return null; The comparison of this new approach to the pre-JDK 8 approach is shown next. /* * Demonstrate Map.replace(K, V) and compare to pre-JDK 8 approach. The JDK 8 * addition of replace(K, V) requires fewer lines of code than the traditional * approach and allows the returned value to be assigned to a "final" * variable. */ // pre-JDK 8 approach String replacedCapitalCity; if (statesAndCapitals.containsKey("Alaska")) { replacedCapitalCity = statesAndCapitals.put("Alaska", "Juneau"); } // JDK 8 approach final String replacedJdk8City = statesAndCapitals.replace("Alaska", "Juneau"); Map.replace(K, V, V) The second newly added Map "replace" method is more narrow in its interpretation of which existing values are replaced. While the method just covered replaces any value in a value available for the specified key in the mapping, this "replace" method that accepts an additional (third) argument will only replace the value of a mapped entry that has both a matching key and a matching value. The Javadoc comment shows the default method's implementation: The default implementation is equivalent to, for this map: if (map.containsKey(key) && Objects.equals(map.get(key), value)) { map.put(key, newValue); return true; } else return false; My comparison of this approach to the pre-JDK 8 approach is shown in the next code listing. /* * Demonstrate Map.replace(K, V, V) and compare to pre-JDK 8 approach. The * JDK 8 addition of replace(K, V, V) requires fewer lines of code than the * traditional approach and allows the returned value to be assigned to a * "final" variable. */ // pre-JDK 8 approach boolean replaced = false; if ( statesAndCapitals.containsKey("Nevada") && Objects.equals(statesAndCapitals.get("Nevada"), "Las Vegas")) { statesAndCapitals.put("Nevada", "Carson City"); replaced = true; } // JDK 8 approach final boolean replacedJdk8 = statesAndCapitals.replace("Nevada", "Las Vegas", "Carson City"); Observations and Conclusion There are several observations to make from this post. The Javadoc methods for these new JDK 8 Map methods are very useful, especially in terms of describing how the new methods behave in terms of pre-JDK 8 code. I discussed these methods' Javadoc in a more general discussion on JDK 8 Javadoc-based API documentation. As the equivalent Java code in these methods' Javadoc comments indicates, these new methods do not generally check for null before accessing map keys and values. Therefore, one can expect the same issues with nulls using these methods as one would find when using "equivalent" code as shown in the Javadoc comments. In fact, the Javadoc comments generally warn about the potential forNullPointerException and issues related to some Map implementations allowing null and some not for keys and values. The new Map methods discussed in this post are "default methods," meaning that implementations of Map "inherit" these implementations automatically. The new Map methods discussed in this post allow for cleaner and more concise code. In most of my examples, they allowed the client code to be converted from multiple state-impacting statements to a single statement that can set a local variable once and for all. The new Map methods covered in this post are not ground-breaking or earth-shattering, but they are conveniences that many Java developers previously implemented more verbose code for, wrote their own similar methods for, or used a third-party library for. JDK 8 brings these standardized methods to the Java masses without need for custom implementation or third-party frameworks. Because default methods are the implementation mechanism, even Map implementations that have been around for quite a while suddenly and automatically have access to these new methods without any code changes to the implementations.

There are many instances where we will have to write custom queries with hibernate. I always hated writing custom queries due to following reasons Hibernate returns List of Object arrays (List getSalaryByDepartment() { Session session = HibernateUtil.getSessionFactory().openSession(); try { List salaryByDepartments = new ArrayList(); for (Object object : contactList) { Object[] result = (Object[]) object; SalaryByDepartment salaryByDepartment = new SalaryByDepartment(); salaryByDepartment.setDeptId((Integer) result[0]); salaryByDepartment.setDepartmentName((String) result[1]); salaryByDepartment.setSalary((Double) result[2]); salaryByDepartments.add(salaryByDepartment); } return salaryByDepartments; } catch (HibernateException e) { e.printStackTrace(); return null; } finally { session.close(); } } There is a better mechanism for writing the custom queries called 'select new' but it is not widely used for some reason. It might be an overkill to write all custom queries using this mechanism, but is good for queries which return more than 2 columns. public List getNewSalaryByDepartment() { Session session = HibernateUtil.getSessionFactory().openSession(); try { List salaryByDept = session.createQuery("select " + "new hsqldb.results.SalaryByDepartment(department.id, department.departmentName, sum(employee.salary)) " + "from Employee employee, Department department " + "where employee.department.id = department.id group by department.id") .list(); return salaryByDept; } catch (HibernateException e) { e.printStackTrace(); return null; } finally { session.close(); } }

After reading the interesting blog post of my colleague Rik van Bruggen on “Media, Politics and Graphs” I thought it would be really cool to render it as a GrapGist. Especially, as he already shared all the queries as a GitHub Gist. Unfortunately the dataset was a bit large for a sensible GraphGist representation, so I thought about means of extracting a smaller sample of his raw data that he made available (see his blog post for the link). Considering my last blog post on creating data from sampling a cross product, this should be much easier. We know we want to have all nodes with the labels PARTY, SHOW and GENDER in our graph as well as a sample of GUEST nodes with their relationships. The first part is easy: MATCH (n) WHERE n:PARTY OR n:SHOW OR n:GENDER RETURN n; The second part uses something that was not helpful in my last exploration, namely that random sampling when applied directly to a match, is used to filter the first node-pattern in the match and then still traverse all relationships/paths emanating from that node. MATCH(n:GUEST)-[r]->() WHERE rand() < 0.1 RETURN n,r; The number you compare rand() to is the percentage you want to get back, in this example 10%. Now I have two nice queries, that can get me the data, how can I bring them together? With UNION ALL MATCH (n) WHERE n:PARTY OR n:SHOW OR n:GENDER RETURN n, null as r UNION ALL MATCH(n:GUEST)-[r]->() WHERE rand() < 0.1 RETURN n,r; And where do I get the Cypher statements from, that I can use to populate my GraphGist database setup? Fortunately my dump command made it into the Neo4j-Shell, so that we can just run it on the command-line and redirect the output into a file: bin/neo4j-shell -path talkshow/graph.db \ -c 'dump MATCH (n) WHERE n:PARTY OR n:SHOW OR n:GENDER RETURN n, null as r UNION ALL MATCH(n:GUEST)-[r]->() WHERE rand() < 0.1 RETURN n,r;' \ > talkshow/sample.cql Don’t forget the semicolon at the end! Looking at sample.cql we see something like: begin create (_0:`SHOW` {`Modularity Name`:"B&vD", `id`:"B&vD", `label`:"B&vD", `modularity_class`:3, `weighted outdegree`:0.000000}) create (_1:`SHOW` {`Modularity Name`:"P&W", `id`:"P&W", `label`:"P&W", `modularity_class`:4, `weighted outdegree`:0.000000}) create (_2:`SHOW` {`Modularity Name`:"DWDD", `id`:"DWDD", `label`:"DWDD", `modularity_class`:5, `weighted outdegree`:0.000000}) ... ... create _509-[:`VISITED` {`quantity`:1}]->_5 create _509-[:`VISITED` {`quantity`:1}]->_2 create _509-[:`VISITED` {`quantity`:1}]->_1 create _509-[:`VISITED` {`quantity`:1}]->_0 ; commit Which we can now use to populate our database for our GraphGist, and here it is in all its beauty – GraphGist: “Media, Politics and Graphs”. But actually I chose not to use Rik’s GitHub Gist with the queries, but to copy the nice text and pictures from his blog post into the GraphGist. You might notice that some of the parties go without connections. That would need some tweaking of the sampling which I leave as exercise for you. Have fun Michael

A few days ago the second beta of Groovy 2.3 got released. One of the major new Groovy 2.3 features are traits. A trait is a reusable set of methods and fields that can be added to one or more classes. A class can be composed out of multiple traits without using multiple inheritance (and therefore avoiding the diamond problem). Basic usage The following piece of code shows a basic definition of a trait in Groovy 2.3. trait SwimmingAbility { def swim() { println "swimming.." } } A trait definition looks very similar to a class definition. This trait defines a single method swim(). We can add this trait to a class using the implements keyword: class Goldfish implements SwimmingAbility { .. } Now we are able to call the swim() methods on Goldfish objects: def goldfish = new Goldfish() goldfish.swim() So far we could have accomplished the same using inheritance. The difference is that we can add multiple traits to a single class. So let's define another trait: trait FlyingAbility { def fly() { println "flying.." } } We can now create a new class that makes use of both traits: class Duck implements SwimmingAbility, FlyingAbility { .. } Ducks can swim and fly now: def duck = new Duck() duck.swim() duck.fly() this inside traits Inside traits the this keyword represents the implementing instance. This means you can write the following: trait FlyingAbility { def fly() { println "${this.class.name} is flying.." } } In case of the Duck class from above this will print Duck is flying.. if we call duck.fly(). A more complex example Now let's look at an example that shows some more features of Groovy traits trait Trader { int availableMoney = 0 private int tradesDone = 0 def buy(Item item) { if (item.price <= availableMoney) { availableMoney -= item.price tradesDone += 1 println "${getName()} bought ${item.name}" } } def sell(Item item) { .. } abstract String getName() } Like Groovy classes traits support properties. Here the property availableMoney will become private and public getter / setter methods will be generated. These methods can be accessed on implementing classes.tradesDone is a private variable that cannot be accessed outside the Trader trait. Within this trait we defined an abstract method getName(). This method has to be implemented by classes that make use of this trait. Let's create a class that implements our Trader trait: class Merchant implements Trader { String name String getName() { return this.name } } A Merchant is now be able to buy items: def bike = new Item(name: 'big red bike', price: 750) def paul = new Merchant(name: 'paul') paul.availableMoney = 2000 paul.buy(bike) // prints "paul bought big red bike" println paul.availableMoney // 1250 Extending from traits, Overriding and conflict resolution A trait can inherit functionality from another trait using the extends keyword: trait Dealer { def getProfit() { ... } def deal() { ... } } trait CarDealer extends Dealer { def deal() { ... } } Here the CarDealer trait extends Dealer and overrides the deal() method of Dealer. Trait methods can also be overwritten in implementing classes: class OnlineCarDealer implements CarDealer { def deal() { ... } } If a class implements more than one trait it is possible to create a conflict. That's the case if two or more traits define a method with an identical signature: trait Car { def drive() { ... } } trait Bike { def drive() { ... } } class DrivingThing implements Car, Bike { ... } In such a situation the last declared trait wins (Bike in this example). Conclusion I think traits are a very useful concept and I am happy to see them in Groovy. Other than Groovy mixins traits work at compile time and can therefore be accessed from Java code. For further reading I can recommend the Groovy 2.3 Trait documentation.

If you need to compare the java.io.file.Path object, be aware that Path.endsWith(String) will ONLY match another sub-element of Path object in your original path, not the path name string portion! If you want to match the string name portion, you would need to call the Path.toString() first. For example // Match all jar files. Files.walk(dir).forEach(path -> { if (path.toString().endsWith(".jar")) System.out.println(path); }); With out the "toString()" you will spend many fruitless hours wonder why your program didn't work.

There are many false ideas around about Java 8. Among these is the idea that Java 8 brings closures to Java.

What's New in this Release? Aspose development team is happy to announce the monthly release of Aspose.Words for Java &.NET 14.3.0. Aspose.Words now supports insertion of OLE objects such as another Microsoft Word document or a Microsoft Excel chart. A new public method, InsertOleObject, has been introduced in the DocumentBuilder class. This method can be used to insert an embedded or linked OLE object from a file into a Word document. Aspose.Words’ rendering engine now partially supports EUDC (End-User-Defined-Characters) fonts. Please find below the description of how EUDC fonts works on Windows. In this first implementation, Aspose.Words uses a single EUDC font. When rendering a document to fixed-page formats, this font is searched among the specified font sources by “EUDC” family name. Starting from Aspose.Words 14.3.0, Best Fit position of data labels in pie charts is partially supported. In previous versions labels with best fit position were rendered as if they had the inside end position. Currently we use a modified Open Office algorithm to set the best fit position of data labels. The list of new and improved features in this release are listed below Public API for insertion of OLE objects both linked and embedded Outline, Shadow, Reflection, Glow and Fill text effects for rendering text inside DrawingML shapes EUDC fonts rendering partially supported “PDF Logical Structure” export reworked, significantly improving memory usage Support OLE embedding of documents and files Feature Write an article about how to work with Table of contents in Aspose.Words Add tag support Support BestFit position of Pie chart's data labels. Support style:text-decoration attribute of Paragraph tag. Preserve private characters in EUDC.TTE during rendering to Pdf Support HTML table row borders in HTML import Support text outline effect. Support text fill effect. Support text shadow effect. Support text reflection effect Support text glow effect. Support text effects applied to text in Dml Shape or in SmartArt. Implement reading text effects from rPr in DML. Support inheriting styles from parent objects. Add image compression options for different image types Add a link to the online documentation in the DLLs only release Default run properties lose lang attribute value on DOCX to DOC conversions Padding for image in table is lost when rendering to Pdf Consider using "Don't vertically align cells containing floating objects" compatibility option when exporting to HTML Shape in table's cell is improperly horizontally aligned to center after export to HTML is now fixed Floating shape is now properly vertically positioned after export to HTML Warning : Unkno/wn ProgId value 'Visio.Drawing.11'. This might cause inaccessible OLE embedding Charts (DrawingML) issue fixed and now render correctly in Pdf file Aspose.Words now properly work in Jdeveloper IDE OLE object cannot be edited after re saving the document Diagram connectors are inverted/flipped after conversion from Docx to pdf WordArt letters are condensing issue is resolved Condensed character spacing is lost is fixed Relative hyperlink with Unicode is now properly saved to Pdf Add pre-built Document Explorer JAR to Java release. Aspose.Words does not take in account style set in . Table now looks correctly while converting html to doc. Hyperlinks split into multiple fragments/links in output PDF Horizontal table position is corrected. Document.UpdateFields Does not Update TOC in DOCX Content in the output html is overlapped at many places in Html Different table justify alignment for different compatibilityMode values. CSS selectors now work for , , and elements Chart Legend/Series now render correctly in output Pdf file Data labels with best fit position are rendering at correct places in Chart Offset of the hyperlink text line is off by a few pixels in output Pdf Hyperlinks split into multiple fragments/links in output PDF is fixed Document.UpdateFields is enhanced and update SUM formula field Paragraph's first line indent increases is fixed when exported to HTM Space before a paragraph following a floating element is too large after HTML to DOCX conversion is now fixd Block-level SVG image become inline after HTML to DOCX conversion Problem with vertical paragraph spacing is resolved when importing HTML using InsertHtml method Shape rotation is fixed after conversion from Doc to Pdf Text color is changed after conversion from Docx to WordML/Doc Word Table indentation is now corrected when is placed inside Arrows on the Lines gets distorted when converting to Pdf is now fixed A space character is exported to PDF output between Japanese and Numeric characters is fixed Line Shape from Header is merged with the top border of Table in Body Docx to WordML conversion issue resolved with content control Junk text is rendered in fixed page formats Positions of some DrawingML circles are now preserved during rendering A row and some content is rendering at the bottom of previous page is fixed Header table rows and images are now preserved in PDF List items issue fixed, now line up correctly after conversion from RTF to HTML Conversion from Docm to Doc creates corrupted document now fixed Hyperlink for an icon is now preserved during HTML to PDF conversion Text overlapping is fixed after conversion from Docx to Pdf Text position change is fixed after conversion from Docx to Pdf Other most recent bug fixes are also included in this release Newly added documentation pages and articles Some new tips and articles have now been added into Aspose.Words for .NET documentation that may guide you briefly how to use Aspose.Words for performing different tasks like the followings. How Aspose.Words Uses True Type Fonts How to Extract Images from a Document Overview: Aspose.Words Aspose.Words is a word processing component that enables .NET, Java & Android applications to read, write and modify Word documents without using Microsoft Word. Other useful features include document creation, content and formatting manipulation, mail merge abilities, reporting features, TOC updated/rebuilt, Embedded OOXML, Footnotes rendering and support of DOCX, DOC, WordprocessingML, HTML, XHTML, TXT and PDF formats (requires Aspose.Pdf). It supports both 32-bit and 64-bit operating systems. You can even use Aspose.Words for .NET to build applications with Mono. More about Aspose.Words Homepage Aspose.Words for .NET Homepage Java Word Library Download Aspose.Words for .NET Download Aspose.Words for Java Demos for Aspose.Words Contact Information Aspose Pty Ltd Suite 163, 79 Longueville Road Lane Cove, NSW, 2066 Australia Aspose - Your File Format Experts [email protected] Phone: 888.277.6734 Fax: 866.810.9465

Camel is very often used in distributed environments for accessing remote resources. Remote services may fail for various reasons and periods. For services that are temporarily unavailable and recoverable after short period of time, a retry strategy may help. But some services can fail or hang for longer period of time making the calling application unresponsive and slow. A good strategy to prevent from cascading failures and exhaustion of critical resources is the Circuit Breaker pattern described by Michael Nygard in the Release It! book. Circuit Breaker is a stateful pattern that wraps the failure-prone resource and monitors for errors. Initially the Circuit Breaker is in closed state and passes all calls to the wrapped resource. When the failures reaches a certain threshold, the circuit moves to open state where it returns error to the caller without actually calling the wrapped resource. This prevents from overloading the already failing resource. While at this state, we need a mechanism to detect whether the failures are over and start calling the protected resource. This is where the third state called half-open comes into play. This state is reached after a certain time following the last failure. At this state, the calls are passed through to the protected resource, but the result of the call is important. If the call is successful, it is assumed that the protected resource has recovered and the circuit is moved into closed state, and if the call fails, the timeout is reset, and the circuit is moved back to open state where all calls are rejected. Here is the state diagram of Circuit Breaker from Martin Fowler's post: How Circuit Breaker is implemented in Camel? Circuit Breaker is available in the latest snapshot version of Camel as a Load balancer policy. Camel Load Balancer already has policies for Round Robin, Random, Failover, etc. and now also CircuiBreaker policy. Here is an example load balancer that uses Circuit Breaker policy with threshold of 2 errors and halfOpenAfter timeout of 1 second. Notice also that this policy applies only to errors caused by MyCustomException. new RouteBuilder() { public void configure() { from("direct:start").loadBalance() .circuitBreaker(2, 1000L, MyCustomException.class) .to("mock:result"); } }; And here is the same example using Spring XML DSL: MyCustomException

This post was originally written by Vlad Lesin for the MySQL Performance Blog. Percona Server 5.6.11-60.3 introduces a new “log archiving” feature. Percona XtraBackup 2.1.5 supports “apply archived logs.” What does it mean and how it can be used? Percona products propose three kinds of incremental backups. The first is full scan of data files and comparison the data with backup data to find some delta. This approach provides a history of changes and saves disk space by storing only data deltas. But the disadvantage is a full-data file scan that adds load to the disk subsystem. The second kind of incremental backup avoids extra disk load during data file scans. The idea is in reading only changed data pages. The information about what specific pages were changed is provided by the server itself which writes files with the information during work. It’s a good alternative but changed-pages tracking adds some small load. And Percona XtraBackup’s delta reading leads to non-sequential disk io. This is good alternative but there is one more option. The Innodb engine has a data log. It writes all operations which modify database pages to log files. This log is used in the case of unexpected server terminating to recover data. The Innodb log consists of the several log files which are filled sequentially in circular. The idea is to save those files somewhere and apply all modifications from archived logs to backup data files. The disadvantage of this approach is in using extra disk space. The advantage is there is no need to do an “explicit” backup on the host server. A simple script could sit and wait for logs to appear then scp/netcat them over to another machine. But why not use good-old replication? Maybe replication does not have such performance as logs recovering but it is more controlled and well-known. Archived logs allows you to do any number of things with them from just storing them to doing periodic log applying. You can not recover from a ‘DROP TABLE’, etc with replication. But with this framework one could maintain the idea of “point in time” backups. So the “archived logs” feature is one more option to organize incremental backups. It is not widely used as it was issued not so far and there is not A good understanding of how it works and how it can be used. We are open to any suggestions about its suggest improvements and use cases. The subject of this post is to describe how it works in depth. As log archiving is closely tied with innodb redo logs the internals of redo logs will be covered too. This post would be useful not only for DBA but also for Software Engineers because not only common principles are considered but the specific code too, and knowledge from this post can be used for further MySQL code exploring and patching. What is the innodb log and how it is written? Let’s remember what are innodb logs, why they are written, what they are used for. The Innodb engine has buffer pool. This is a cache of database pages. Any changes are done on page in buffer pool, then page is considered as “dirty,” which means it must be flushed, and pushed to the flush list which is processed periodically by special thread. If pages are not flushed to disk and server is terminated unexpectedly the changes will be lost. To avoid this innodb writes changes to redo log and recover data from redo log during start. This technique allows to delay buffer pool pages flushing. It can increase performance because several changes of one page can be accumulated in memory and then flushed by one io. Except that flushed pages can be grouped to decrease the number of non-sequential io’s. But the down-side of this approach is time for data recovering. Let’s consider how this log is stored, generated and used for data recovering. Log files Redo log consists of a several log files which are treated as a circular buffer. The number and the size of log files can be configured. Each log file has a header. The description of this header can be found in “storage/innobase/include/log0log.h” by “LOG_GROUP_ID” keyword. Each log file contains log records. Redo log records are written sequentially by log blocks of OS_FILE_LOG_BLOCK_SIZE size which is equal to 512 bytes by default and can be changed with innodb option. Each record has its LSN. LSN is a “Log Sequence Number” – the number of bytes written to log from the log creation to the certain log record. Each block consists of header, trailer and log records. Log blocks Let’s consider log block header. The first 4 bytes of the header is log block number. The block number is very similar as LSN but LSN is measured in bytes and block number is measured by OS_FILE_LOG_BLOCK_SIZE. Here is the simple formula how LSN is converted to block number: return(((ulint) (lsn / OS_FILE_LOG_BLOCK_SIZE) & 0x3FFFFFFFUL) + 1); This formula can be found in log_block_convert_lsn_to_no() function. The next two bytes is the number of bytes used in the block. The next two bytes is the offset of the first MTR log record in this block. What is MTR will be described below. Currently it can be considered as a synonym of bunch of log records which are gathered together as a description of some logical operation. For example it can be a group of log records for inserting new row to some table. This field is used when there are records of several MTR’s in one block. The next four bytes is a checkpoint number. The trailer is four bytes of log block checksum. The above description can be found in “storage/innobase/include/log0log.h” by “LOG_BLOCK_HDR_NO” keyword. Before writing to disk log blocks must be somehow formed and stored. And the question is: How log blocks are stored in memory and on disk? Where log blocks are stored before flushing to disk and how they are written and flushed? Global log object and log buffer The answer to the first part of the question is log buffer. Server holds very important global object log_sys in memory. It contains a lot of useful information about logging state. Log buffer is pointed by log_sys->buf pointer which is initialized in log_init(). I would highlight the following log_sys fields that are used for work with log buffer and flushing: log_sys->buf_size – the size of log buffer, can be set with innodb-log-buffer-size variable, the default value is 8M; log_sys->buf_free – the offset from which the next log record will be written; log_sys->max_buf_free – if log_sys->buf_free is greater then this value log buffer must be flushed, see log_free_check(); log_sys->buf_next_to_write – the offset of the next log record to write to disk; log_sys->write_lsn – the LSN up to which log is written; log_sys->flushed_to_disk_lsn – the LSN up to which log is written and flushed; log_sys->lsn – the last LSN in log buffer; So log_sys->buf_next_to_write is between 0 and log_sys->buf_free, log_sys->write_lsn is equal or less log_sys->lsn, log_sys->flushed_to_disk_lsn is less or equal to log_sys->write_lsn. The relationships for those fields can be easily traced with debugged by setting up watchpoints. Ok, we have log buffer, but how do log records come to this buffer? Where log records come from? Innodb has special objects that allow you to gather redo log records for some operations in one bunch before writing them to log buffer. These objects are called “mini-transactions” and corresponding functions and data types have “mtr” prefix in the code. The objects itself are described in mtr_t “c” structure. The most interesting fields of this structure are the following: mtr_t::log – contains log records for the mini-transaction, mtr_t::memo – contains pointers to pages which are changed or locked by the mini-transaction, it is used to push pages to flush list and release locks after logs records are copied to log buffer in mtr_commit() (see mtr_memo_pop_all() called in mtr_commit()). mtr_start() function initializes an object of mtr_t type and mtr_commit() writes log records from mtr_t::log to log_sys->buf + log_sys->buf_free. So the typical sequence of any operation which changes data is the following: mtr_start(); // initialize mtr object some_ops... // operations on data which are logged in mtr_t::log mtr_commit(); // write logged operations from mtr_t::log to log buffer log_sys->buf page_cur_insert_rec_write_log() is a good example of how mtr records can be written and mtr::memo can be filled. The low-level function which writes data to log buffer is log_write_low(). This function is invoked inside of mtr_commit() and not only copy the log records from mtr_t object to log buffer log_sys->buf but also creates a new log blocks inside of log_sys->buf, fills their header, trailer, calculates checksum. So log buffer contains log blocks which are sequentially filled with log records which are grouped in “mini-transactions” which logically can be treated as some logical operation over data which consists of a sequence of mini-operations(log records). As log records are written sequentially in log buffer one mini-transaction and even one log record can be written in two neighbour blocks. That is why the header field which would contain the offset of the first MTR in the block is necessary to calculate the point from which log records parsing can be started. This field was described in 2.2. So we have a buffer of log blocks in a memory. How is data from this buffer written to disk? The mysql documentation says that this depends on innodb_flush_log_at_trx_commit option. There can be three cases depending on the value of this option. Let’s consider each of them. Writing log buffer to disk: innodb_flush_log_at_trx_commit is 1 or 2. The first two cases is when innodb_flush_log_at_trx_commit is 1 or 2. In these cases flush log records are written for 2 and flushed for 1 on each transaction commit. If innodb_flush_log_at_trx_commit is 2 log records are flushed periodically by special thread which will be considered later. The low-level function which writes log records from buffer to file is log_group_write_buf(). But in the most cases it is not called directly but it is called from more high level log_write_up_to(). For the current case the calling stack is the following: (trx_commit_in_memory() or trx_commit_complete_for_mysql() or trx_prepare() e.t.c)-> trx_flush_log_if_needed()-> trx_flush_log_if_needed_low()-> log_write_up_to()-> log_group_write_buf(). It is quite easy to find the higher levels of calling stack, just set up breakpoint on log_group_write_buf() and execute any sql query that modifies innodb data. For example for the simple “insert” sql query the higher levels of calling stack are the following: mysql_execute_command()-> trans_commit_stmt()-> ha_commit_trans()-> TC_LOG_DUMMY::commit()-> ha_commit_low()-> innobase_commit()-> trx_commit_complete_for_mysql()-> trx_flush_log_if_needed()-> ... . log_io_complete() callback is invoked when i/o is finished for log files (see fil_aio_wait()). log_io_complete() flushes log files if this is not forbidden by innodb_flush_method or innodb_flush_log_at_trx_commit options. Writing log buffer to disk: innodb_flush_log_at_trx_commit is equal to 0 The third case is when innodb_flush_log_at_trx_commit is equal to 0. For this case log buffer is NOT written to disk on transaction commit, it is written and flushed periodically by separate thread “srv_master_thread”. If innodb_flush_log_at_trx_commit = 0 log files are flushed in the same thread by the same calls. The calling stack is the following: srv_master_thread()-> (srv_master_do_active_tasks() or srv_master_do_idle_tasks() or srv_master_do_shutdown_tasks())-> srv_sync_log_buffer_in_background()-> log_buffer_sync_in_background()->log_write_up_to()->... . Special cases for logs flushing While log_io_complete() do flushing depending on innodb_flush_log_at_trx_commit value among others log_write_up_to() has it’s own flushing criteria. This is flush_to_disk function argument. So it is possible to force log files flushing even if innodb_flush_log_at_trx_commit = 0. Here are examples of such cases: 1) buf_flush_write_block_low() Each page contains information about the last applied LSN(buf_flush_write_block_low::newest_modification), each log record is a description of change on certain page. Imagine we flushed some changed pages but log records for these pages were not flushed and server goes down. After starting the server some pages will have the newest modifications, but some of them were not flushed and the correspondent log records are lost too. We will have inconsistent database in this case. That is why log records must be flushed before the pages they refer. 2) srv_sync_log_buffer_in_background() As it was described above this function is called periodically by special thread and forces flushing. 3) log_checkpoint() When checkpoint is made log files must be reliably flushed. 4) The special handlerton innobase_flush_logs() which can be called through ha_flush_logs() from mysql server. For example ha_flush_logs() is called from MYSQL_BIN_LOG::reset_logs() when “RESET MASTER” or “RESET SLAVE” are executed. 5) srv_master_do_shutdown_tasks() – on shutdown, ha_innobase::create() – on table creating, ha_innobase::delete_table() – on table removing, innobase_drop_database() – on all database tables removing, innobase_rename_table() – on table rename e.t.c If log files are treated as circular buffer what happens when the buffer is overflown? Briefly. Innodb has a mechanism which allows you to avoid overflowing. It is called “checkpoints.” The checkpoint is a state when log files are synchronized with data files. In this case there is no need to keep the history of changes before checkpoint because all pages with the last modifications LSN less or equal to checkpoint LSN are flushed and the log files space from the last written LSN to the last checkpoint LSN can be reused. We will not describe a checkpoint process here because it is a separate interesting subject. The only thing we need to know is when checkpoint happens all pages with modification LSN less or equal to checkpoint LSN are reliably flushed. How archived logs are written by server. So the log contains information about page changes. But as we said, log files are the circular buffer. This means that they occupy fixed disk size and the oldest records can be rewritten by the newest ones as there are points when data files are synchronized with log files called checkpoints and there is no need to store the previous history of log records to guarantee database consistency. The idea is to save somewhere all log records to have the possibility of applying them to backuped data to have some kind if incremental backup. For example if we want to have an archive of log records. As log consists of log files it is reasonable to store log records in such files too, and these files are called “archived logs.” Archived log files are written to the directory which can be set with special innodb option. Each file has the same size as innodb log size and the suffix of each archived file is the LSN from which it is started. As well as log writing system log archiving system stores its data in global log_sys object. Here are the most valuable fields in log_sys from my point of view: log_sys->archive_buf, log_sys->archive_buf_size – logs archive buffer and its size, log records are copied from log buffer log_sys->buf to this buffer before writing to disk; log_sys->archiving_phase – the current phase of log archiving: LOG_ARCHIVE_READ when log records are being copied from log_sys->buf to log_sys->archive_buf, LOG_ARCHIVE_WRITE when log_sys->archive_buf is being written to disk; log_sys->archived_lsn – the LSN to which log files are written; log_sys->next_archived_lsn – the LSN to which write operations was invoked but not yet finished; log_sys->max_archived_lsn_age – the maximum difference between log_sys->lsn and log_sys->archived_lsn, if this difference exceeds the log are being archived synchronously, i.e. the difference is decreased; log_sys->archive_lock – this is rw-lock which is used for synchronizing LOG_ARCHIVE_WRITE and LOG_ARCHIVE_READ phases, it is x-locked on LOG_ARCHIVE_WRITE phase. So how is data copied from log_sys->buf to log_sys->archived_buf? log_archive_do() is used for this. It is not only set the proper state for archived log fields in log_sys but also invokes log_group_read_log_seg() with corresponding arguments which not only copy data from log buffer to archived log buffer but also invokes asynchronous write operation for archived log buffer. log_archive_do() can wait until io operations are finished using log_sys->archive_lock if corresponding function parameter is set. The main question is on what circumstances log_archive_do() is invoked, i.e. when log records are being written to archived log files. The first call stack is the following: log_free_check()-> log_check_margins()-> log_archive_margin()-> log_archive_do(). Here is text of log_free_check() with comments: /*********************************************************************// Checks if there is need for a log buffer flush or a new checkpoint, and does this if yes. Any database operation should call this when it has modified more than about 4 pages. NOTE that this function may only be called when the OS thread owns no synchronization objects except the dictionary mutex. */ UNIV_INLINE void log_free_check(void) /*================*/ { #ifdef UNIV_SYNC_DEBUG ut_ad(sync_thread_levels_empty_except_dict()); #endif /* UNIV_SYNC_DEBUG */ if (log_sys->check_flush_or_checkpoint) { log_check_margins(); } } log_sys->check_flush_or_checkpoint is set when there is no enough free space in log buffer or it is time to do checkpoint or any other bound case. log_archive_margin() is invoked only if the limit if the difference between log_sys->lsn and log_sys->archived_lsn is exceeded. Let’s refer to this difference as archived lsn age. One more call log_archive_do() is from log_open() when archived lsn age exceeds some limit. log_open() is called on each mtr_commit(). And for this case archived logs are written synchronously. The next synchronous call is from log_archive_all() during shutdown. Summarizing all above archived logs begins to be written when the log buffer is full enough to be written or when checkpoint happens or when the server is in the process of shut down. And there is no any delay between writing to archive log buffer and writing to disk. I mean there is no way to say that archived logs must be written once a second as it is possible for redo logs with innodb_flush_log_at_trx_commit = 0. As soon as data is copied to the buffer the write operation is invoked immediately for this buffer. Archived log buffer is not filled on each mtr_commit() so it does not slow down the usual logging process. The exception is when there are a lot of io operations what can be the reason of archive log age is too big. The result of big archive log age is the synchronous archived logs writing during mtr_commit(). Memory to memory copying is quite fast operation that is why the data is copied to archived log buffer and is written to disk asynchronously minimizing delays which can be caused by logs archiving. PS: Here is another call stack for writing archived log buffer to archived log files: log_io_complete()->log_io_complete_archive()->log_archive_check_completion_low()->log_archive_groups(). I propose to explore this stack yourself. Logs recovery process, how it is started and works inside. Archived logs applying. So we discovered how innodb redo logging works, and how redo logs are archived. And the last uncovered thing is how recovery works and how archived logs are applied. These two processes are very similar – that is why they are discussed in one section of this post. The story begins with innobase_start_or_create_for_mysql() which is invoked from innobase_init(). The following trident in innobase_start_or_create_for_mysql() can be used to search the relevant code: if (create_new_db) { ... } else if (srv_archive_recovery) { ... } else { ... } The second condition and the last one is the place from which archived logs applying and innodb logs recovery processes correspondingly start. These two blocks wrap two pairs of functions: recv_recovery_from_archive_start() recv_recovery_from_archive_finish() and recv_recovery_from_checkpoint_start() recv_recovery_from_checkpoint_finish() And all the magic happens in these pairs. As well as global log_sys object for redo logging there is global recv_sys object for innodb recovery and archived logs applying. It is created and initialized in recv_sys_create() and recv_sys_init() functions correspondingly. The following fields if recv_sys object are the most important from my point: recv_sys->limit_lsn – the LSN up to which recovery should be made, this value is initialized with the maximum value of uint64_t(see #define LSN_MAX) for the recovery process and with certain value which is passed as an argument of recv_recovery_from_archive_start() function and can be set via xtrabackup option for log applying; recv_sys->parse_start_lsn – the LSN from which logs parsing is started, for the the logs recovery this value equals to the last checkpoint LSN, for logs applying this is last applied LSN; recv_sys->scanned_lsn – the LSN up to which log files are scanned; recv_sys->recovered_lsn – the LSN up to which log records are applied, this value <= recv_sys->scanned_lsn; The first thing that must be done for starting recovery process is to find out the point in log files where the recovery must be started from. This is the last checkpoint LSN. recv_find_max_checkpoint() proceed this. As we can see in log_group_checkpoint() the following code writes checkpoint info into two places in the first log file depending on the checkpoint number: /* We alternate the physical place of the checkpoint info in the first log file */ if ((log_sys->next_checkpoint_no & 1) == 0) { write_offset = LOG_CHECKPOINT_1; } else { write_offset = LOG_CHECKPOINT_2; } So recv_find_max_checkpoint() reads checkpoint info from both places and selects the latest checkpoint. The same idea is applied for logs, too, but the last applied LSN instead of last checkpoint LSN must be found. Here is the call stack for reading last applied LSN: innobase_start_or_create_for_mysql()-> open_or_create_data_files()-> fil_read_first_page(). The last applied LSN is stored in the first page of data files in (min|max)_flushed_lsn fields(see FIL_PAGE_FILE_FLUSH_LSN offset). These values are written in fil_write_flushed_lsn_to_data_files() function on server shutdown. So the main difference between logs applying and recovery process at this stage is the manner of calculating LSN from which log records will be read. For logs applying the last flushed LSN is used but for recovery process it is the last checkpoint LSN. Why does this difference take place? Logs can be applied periodically. Assume we gather archived logs and apply them once an hour to have fresh backup. After applying the previous bunch of log files there can be unfinished transactions. For the recovery process any unfinished transactions are rolled back to have consistent db state at server starting. But for the logs applying process there is no need to roll back them because any unfinished transactions can be finished during the next logs applying. After calculating the start LSN the sequence of actions is the same for both recovering and applying. The next step is reading and parsing log records. See recv_group_scan_log_recs() which is invoked from recv_recovery_from_checkpoint_start_func() for logs recovering and recv_recovery_from_archive_start()->log_group_recover_from_archive_file() for logs applying. The first we read log records to some buffer and then invoke recv_scan_log_recs() to parse them. recv_scan_log_recs() checks each log block on consistency(checksum + comparing the log block number written in log block with log block number calculated from log block LSN) and other edge cases and copy it to parsing buffer recv_sys->buf with recv_sys_add_to_parsing_buf() function. The parsing buffer is then parsed by recv_parse_log_recs(). Log records are stored in hash table recv_sys->addr_hash. The key for this hash table is calculated basing on space id and page number pair. This pair refers to the page to which log records must be applied. The value of the hash table is object of recv_addr_t type. recv_addr_t type contains rec_list field which is the list of log records for applying to the (space id, page num) page (see recv_add_to_hash_table(). After parsing and storing log record in hash table recv_sys->addr_hash log records are applied. The function which is responsible for log records applying is recv_apply_hashed_log_recs(). It is invoked from recv_scan_log_recs() if there is no enough memory to store log records and at the end of recovering/applying process. For each element of recv_sys->addr_hash, i.e. for each DB page which must be changed with log records recv_recover_page() is invoked. It can be invoked as from recv_apply_hashed_log_recs() in the case if page is already in buffer pool of from buf_page_io_complete() on io completion, i.e. just after page was read from storage. Applying log records on page read completion is necessary and very convenient. Assume log records have not yet applied as we had enough memory to store the whole recovery log records. But we want for example to boot DB dictionary. I this case any records that concern to the pages of the dictionary will be applied to those pages just after reading them from storage to buffer pool. The function which applies log records to the certain page is recv_recover_page_func(). It gets the list of log records for the certain page from recv_sys->addr_hash hash table, for each element of this list it compares the lsn of last page changes with the LSN of the record, and if the former is greater the later it applies log record to the page. After applying all log records from archived logs xtrabackup writes last applied LSN to (min|max)_flushed LSN fields of each data file and finishes execution. The logs recovery process rollbacks all unfinished transactions unless this is forbidden with innodb-force-recovery parameter. Conclusion We covered the processes of redo logs writing and recovery in depth. These are very important processes as they provide data consistency on crashes. These two processes became a base for logs archiving and applying features. As log records can describe any data changes the idea is to store these records somewhere and then apply them to backups for organizing some kind of incremental backup. The features were implemented a short time ago and currently they are not widely used. So if you have something to say about them you are welcome to comment for discussion.

I am a recent convert to thymeleaf for view templating in Spring based web applications, preferring it over jsp's. All the arguments that thymeleaf documentation makes on why thymeleaf over jsp holds water and I am definitely sold. One of the big reasons for me, apart from being able to preview the template, is the way the view is rendered at runtime. Whereas the application stack has to defer the rendering of jsp to the servlet container, it has full control over the rendering of thymeleaf templates. To clarify this a little more, with jsp as the view technology an application only returns the location of the jsp and it is upto the servlet container to render the jsp. So why again is this a big reason - because using the mvc test support in spring-test module, now the actual rendered content can be asserted on rather than just the name of the view. Consider a sample Spring MVC controller : @Controller @RequestMapping("/shop") public class ShopController { ... @RequestMapping("/products") public String listProducts(Model model) { model.addAttribute("products", this.productRepository.findAll()); return "products/list"; } } Had the view been jsp based, I would have had a test which looks like this: @RunWith(SpringJUnit4ClassRunner.class) @WebAppConfiguration @ContextConfiguration(classes = SampleWebApplication.class) public class ShopControllerWebTests { @Autowired private WebApplicationContext wac; private MockMvc mockMvc; @Before public void setup() { this.mockMvc = MockMvcBuilders.webAppContextSetup(this.wac).build(); } @Test public void testListProducts() throws Exception { this.mockMvc.perform(get("/shop/products")) .andExpect(status().isOk()) .andExpect(view().name("products/list")); } } the assertion is only on the name of the view. Now, consider a test with thymeleaf used as the view technology: @Test public void testListProducts() throws Exception { this.mockMvc.perform(get("/shop/products")) .andExpect(status().isOk()) .andExpect(content().string(containsString("Dummy Book1"))); } Here, I am asserting on the actual rendered content. This is really good, whereas with jsp I would had to validate that the jsp is rendered correctly at runtime with a real container, with thymeleaf I can validate that rendering is clean purely using tests.

Before some time I have written a blog post – Converting a C# object into JSON string in that post one of reader, Thomas Levesque commented that mostly people are using JSON.NET a popular high performance JSON for creating for .NET Created by James Newton- King. I agree with him if we are using .NET Framework 4.0 or higher version for earlier version still JavaScriptSerializer is good. So in this post we are going to learn How we can convert C# object into JSON string with JSON.NET framework. What is JSON.NET: JSON.NET is a very high performance framework compared to other serializer for converting C# object into JSON string. It is created by James Newton-Kind. You can find more information about this framework from following link. http://james.newtonking.com/json How to convert C# object into JSON string with JSON.NET framework: For this I am going to use old application that I have used in previous post. Following is a employee class with two properties first name and last name. public class Employee { public string FirstName { get; set; } public string LastName { get; set; } } I have created same object of “Employee” class as I have created in previous post like below. Employee employee=new Employee {FirstName = "Jalpesh", LastName = "Vadgama"}; Now it’s time to add JSON.NET Nuget package. You install Nuget package via following command. I have installed like below. Now we are done with adding NuGet package. Following is code I have written to convert C# object into JSON string. string jsonString = Newtonsoft.Json.JsonConvert.SerializeObject(employee); Console.WriteLine(jsonString); Let's run application and following is a output as expected. That’s it. It’s very easy. Hope you like it. Stay tuned for more.

In this post, we will take a look at how we can create an object pool in Java. In recent times, JVM performance has been multiplied manifold and so object creation is no longer considered as expensive as it was done earlier. But there are few objects, for which creation of new object still seems to be slight costly as they are not considered as lightweight objects. e.g.: database connection objects, parser objects, thread creation etc. In any application we need to create multiple such objects. Since creation of such objects is costly, it’s a sure hit for the performance of any application. It would be great if we can reuse the same object again and again. Object Pools are used for this purpose. Basically, object pools can be visualized as a storage where we can store such objects so that stored objects can be used and reused dynamically. Object pools also controls the life-cycle of pooled objects. As we understood the requirement, let’s come to real stuff. Fortunately, there are various open source object pooling frameworks available, so we do not need to reinvent the wheel. In this post we will be using apache commons pool to create our own object pool. At the time of writing this post Version 2.2 is the latest, so let us use this. The basic thing we need to create is- 1. A pool to store heavyweight objects (pooled objects). 2. A simple interface, so that client can - a.) Borrow pooled object for its use. b.) Return the borrowed object after its use. Let’s start with Parser Objects. Parsers are normally designed to parse some document like xml files, html files or something else. Creating new xml parser for each xml file (having same structure) is really costly. One would really like to reuse the same (or few in concurrent environment) parser object(s) for xml parsing. In such scenario, we can put some parser objects into pool so that they can be reused as and when needed. Below is a simple parser declaration: package blog.techcypher.parser; /** * Abstract definition of Parser. * * @author abhishek * */ public interface Parser { /** * Parse the element E and set the result back into target object T. * * @param elementToBeParsed * @param result * @throws Exception */ public void parse(E elementToBeParsed, T result) throws Exception; /** * Tells whether this parser is valid or not. This will ensure the we * will never be using an invalid/corrupt parser. * * @return */ public boolean isValid(); /** * Reset parser state back to the original, so that it will be as * good as new parser. * */ public void reset(); } Let’s implement a simple XML Parser over this as below: package blog.techcypher.parser.impl; import blog.techcypher.parser.Parser; /** * Parser for parsing xml documents. * * @author abhishek * * @param * @param */ public class XmlParser implements Parser { private Exception exception; @Override public void parse(E elementToBeParsed, T result) throws Exception { try { System.out.println("[" + Thread.currentThread().getName()+ "]: Parser Instance:" + this); // Do some real parsing stuff. } catch(Exception e) { this.exception = e; e.printStackTrace(System.err); throw e; } } @Override public boolean isValid() { return this.exception == null; } @Override public void reset() { this.exception = null; } } At this point, as we have parser object we should create a pool to store these objects. Here, we will be using GenericObjectPool to store the parse objects. Apache commons pool has already build-in classes for pool implementation. GenericObjectPool can be used to store any object. Each pool can contain same kind of object and they have factory associated with them. GenericObjectPool provides a wide variety of configuration options, including the ability to cap the number of idle or active instances, to evict instances as they sit idle in the pool, etc. If you want to create multiple pools for different kind of objects (e.g. parsers, converters, device connections etc.) then you should use GenericKeyedObjectPool . package blog.techcypher.parser.pool; import org.apache.commons.pool2.PooledObjectFactory; import org.apache.commons.pool2.impl.GenericObjectPool; import org.apache.commons.pool2.impl.GenericObjectPoolConfig; import blog.techcypher.parser.Parser; /** * Pool Implementation for Parser Objects. * It is an implementation of ObjectPool. * * It can be visualized as- * +-------------------------------------------------------------+ * | ParserPool | * +-------------------------------------------------------------+ * | [Parser@1, Parser@2,...., Parser@N] | * +-------------------------------------------------------------+ * * @author abhishek * * @param * @param */ public class ParserPool extends GenericObjectPool>{ /** * Constructor. * * It uses the default configuration for pool provided by * apache-commons-pool2. * * @param factory */ public ParserPool(PooledObjectFactory> factory) { super(factory); } /** * Constructor. * * This can be used to have full control over the pool using configuration * object. * * @param factory * @param config */ public ParserPool(PooledObjectFactory> factory, GenericObjectPoolConfig config) { super(factory, config); } } As we can see, the constructor of pool requires a factory to manage lifecycle of pooled objects. So we need to create a parser factory which can create parser objects. Commons pool provide generic interface for defining a factory(PooledObjectFactory). PooledObjectFactory create and manage PooledObjects . These object wrappers maintain object pooling state, enabling PooledObjectFactory methods to have access to data such as instance creation time or time of last use. A DefaultPooledObject is provided, with natural implementations for pooling state methods. The simplest way to implement a PoolableObjectFactory is to have it extend BasePooledObjectFactory . This factory provides a makeObject() that returns wrap(create()) where create and wrap are abstract. We provide an implementation of create to create the underlying objects that we want to manage in the pool and wrap to wrap created instances in PooledObjects. So, here is our factory implementation for parser objects- package blog.techcypher.parser.pool; import org.apache.commons.pool2.BasePooledObjectFactory; import org.apache.commons.pool2.PooledObject; import org.apache.commons.pool2.impl.DefaultPooledObject; import blog.techcypher.parser.Parser; import blog.techcypher.parser.impl.XmlParser; /** * Factory to create parser object(s). * * @author abhishek * * @param * @param */ public class ParserFactory extends BasePooledObjectFactory> { @Override public Parser create() throws Exception { return new XmlParser(); } @Override public PooledObject> wrap(Parser parser) { return new DefaultPooledObject>(parser); } @Override public void passivateObject(PooledObject> parser) throws Exception { parser.getObject().reset(); } @Override public boolean validateObject(PooledObject> parser) { return parser.getObject().isValid(); } } Now, at this point we have successfully created our pool to store parser objects and we have a factory as well to manage the life-cycle of parser objects. You should notice that, we have implemented couple of extra methods- 1. boolean validateObject(PooledObject obj): This is used to validate an object borrowed from the pool or returned to the pool based on configuration. By default, validation remains off. Implementing this ensures that client will always get a valid object from the pool. 2. void passivateObject(PooledObject obj): This is used while returning an object back to pool. In the implementation we can reset the object state, so that the object behaves as good as a new object on another borrow. Since, we have everything in place, let’s create a test to test this pool. Pool clients can – 1. Get object by calling pool.borrowObject() 2. Return the object back to pool by calling pool.returnObject(object) Below is our code to test Parser Pool- package blog.techcypher.parser; import static org.junit.Assert.fail; import java.util.concurrent.ArrayBlockingQueue; import java.util.concurrent.ExecutorService; import java.util.concurrent.ThreadPoolExecutor; import java.util.concurrent.TimeUnit; import java.util.concurrent.atomic.AtomicInteger; import junit.framework.Assert; import org.apache.commons.pool2.impl.GenericObjectPoolConfig; import org.junit.Before; import org.junit.Test; import blog.techcypher.parser.pool.ParserFactory; import blog.techcypher.parser.pool.ParserPool; /** * Test case to test- * 1. object creation by factory * 2. object borrow from pool. * 3. returning object back to pool. * * @author abhishek * */ public class ParserFactoryTest { private ParserPool pool; private AtomicInteger count = new AtomicInteger(0); @Before public void setUp() throws Exception { GenericObjectPoolConfig config = new GenericObjectPoolConfig(); config.setMaxIdle(1); config.setMaxTotal(1); /*---------------------------------------------------------------------+ |TestOnBorrow=true --> To ensure that we get a valid object from pool | |TestOnReturn=true --> To ensure that valid object is returned to pool | +---------------------------------------------------------------------*/ config.setTestOnBorrow(true); config.setTestOnReturn(true); pool = new ParserPool(new ParserFactory(), config); } @Test public void test() { try { int limit = 10; ExecutorService es = new ThreadPoolExecutor(10, 10, 0L, TimeUnit.MILLISECONDS, new ArrayBlockingQueue(limit)); for (int i=0; i parser = null; try { parser = pool.borrowObject(); count.getAndIncrement(); parser.parse(null, null); } catch (Exception e) { e.printStackTrace(System.err); } finally { if (parser != null) { pool.returnObject(parser); } } } }; es.submit(r); } es.shutdown(); try { es.awaitTermination(1, TimeUnit.MINUTES); } catch (InterruptedException ignored) {} System.out.println("Pool Stats:\n Created:[" + pool.getCreatedCount() + "], Borrowed:[" + pool.getBorrowedCount() + "]"); Assert.assertEquals(limit, count.get()); Assert.assertEquals(count.get(), pool.getBorrowedCount()); Assert.assertEquals(1, pool.getCreatedCount()); } catch (Exception ex) { fail("Exception:" + ex); } } } Result: [pool-1-thread-1]: Parser Instance:blog.techcypher.parser.impl.XmlParser@fcfa52 [pool-1-thread-2]: Parser Instance:blog.techcypher.parser.impl.XmlParser@fcfa52 [pool-1-thread-3]: Parser Instance:blog.techcypher.parser.impl.XmlParser@fcfa52 [pool-1-thread-4]: Parser Instance:blog.techcypher.parser.impl.XmlParser@fcfa52 [pool-1-thread-5]: Parser Instance:blog.techcypher.parser.impl.XmlParser@fcfa52 [pool-1-thread-8]: Parser Instance:blog.techcypher.parser.impl.XmlParser@fcfa52 [pool-1-thread-7]: Parser Instance:blog.techcypher.parser.impl.XmlParser@fcfa52 [pool-1-thread-9]: Parser Instance:blog.techcypher.parser.impl.XmlParser@fcfa52 [pool-1-thread-6]: Parser Instance:blog.techcypher.parser.impl.XmlParser@fcfa52 [pool-1-thread-10]: Parser Instance:blog.techcypher.parser.impl.XmlParser@fcfa52 Pool Stats: Created:[1], Borrowed:[10] You can easily see that single parser object was created and reused dynamically. Commons Pool 2 stands far better in term of performance and scalability over Commons Pool 1. Also, version 2 includes robust instance tracking and pool monitoring. Commons Pool 2 requires JDK 1.6 or above. There are lots of configuration options to control and manage the life-cycle of pooled objects. And so ends our long post… :-) Hope this article helped. Keep learning!

In the last week I was working on a key project to migrate a BI platform from MySQL to MongoDB. We chose that database due to its support and scalability.

Introduction All developers are very much concerned about performance. If someone say that this increase performance, all the developer are running behind it. It is not a bad practice at all. Rather as per my point of view we must span all our effort related improve the performance of query. “One common question that we find that, if we change the ordering of table join in case of inner join will effect or increase performance” To understand it lets take a simple example of Inner join. There is two tables named Table-A and Table-B. We can us the Inner Join on both the table. Like this FROM [Table-A] AS a INNER JOIN [Table-B] AS b ON a.IDNO = b.IDNO OR FROM [Table-B] AS a INNER JOIN [Table-A] AS b ON a.IDNO = b.IDNO Which one is best for performance? To answer this question we all know that whenever a SQL Query is executed the MS SQL server create several query plans with different join Order and choose the best one. That means the Join order that we are writing in the query may not be executed by execution plan. May be different join order is used by the execution plan. In the above case the execution plan decide which Join order he will chose depends on best possible costing of execution. Here [Table-A] JOIN [Table-B] or [Table-B] JOIN [Table-A], MS SQL Server knows it well that both are same. To understand it Details Lets take an Example Step-1 [ Create Base Table and Insert Some Records ] -- Item Master IF OBJECT_ID(N'dbo.tbl_ITEMDTLS', N'U')IS NOT NULL BEGIN DROP TABLE [dbo].[tbl_ITEMDTLS]; END GO CREATE TABLE [dbo].[tbl_ITEMDTLS] ( ITEMCD INT NOT NULL IDENTITY PRIMARY KEY, ITEMNAME VARCHAR(50) NOT NULL ) GO -- Inserting Records INSERT INTO [dbo].[tbl_ITEMDTLS] (ITEMNAME) VALUES ('ITEM-1'),('ITEM-2'),('ITEM-3'); -- Item UOM Master IF OBJECT_ID(N'dbo.tbl_UOMDTLS', N'U')IS NOT NULL BEGIN DROP TABLE [dbo].[tbl_UOMDTLS]; END GO CREATE TABLE [dbo].[tbl_UOMDTLS] ( UOMCD INT NOT NULL IDENTITY PRIMARY KEY, UOMNAME VARCHAR(50) NOT NULL ) GO -- Inserting Records INSERT INTO [dbo].[tbl_UOMDTLS] (UOMNAME) VALUES ('KG'),('LTR'),('GRM'); GO -- Transaction Table IF OBJECT_ID(N'dbo.tbl_SBILL', N'U')IS NOT NULL BEGIN DROP TABLE [dbo].[tbl_SBILL]; END GO CREATE TABLE [dbo].[tbl_SBILL] ( TRID INT NOT NULL IDENTITY PRIMARY KEY, ITEMCD INT NOT NULL, UOMCD INT NOT NULL, QTY DECIMAL(18,3) NOT NULL, RATE DECIMAL(18,2) NOT NULL, AMOUNT AS QTY * RATE ); GO -- Foreign Key Constraint ALTER TABLE [dbo].[tbl_SBILL] ADD CONSTRAINT FK_ITEM_tbl_SBILL FOREIGN KEY(ITEMCD) REFERENCES [dbo].[tbl_ITEMDTLS](ITEMCD); GO ALTER TABLE [dbo].[tbl_SBILL] ADD CONSTRAINT FK_UOMCD_tbl_SBILL FOREIGN KEY(UOMCD) REFERENCES [dbo].[tbl_UOMDTLS](UOMCD); -- Insert Records INSERT INTO [dbo].[tbl_SBILL] (ITEMCD, UOMCD, QTY, RATE) VALUES (1, 1, 20, 2000),(2, 3, 23, 1400); Step-2 [ Now Make Some JOIN ] SELECT b.TRID, b.ITEMCD, a.ITEMNAME, b.UOMCD, c.UOMNAME, b.QTY, b.RATE, b.AMOUNT FROM [dbo].[tbl_ITEMDTLS] AS a INNER JOIN [dbo].[tbl_SBILL] AS b ON a.ITEMCD = b.ITEMCD INNER JOIN [dbo].[tbl_UOMDTLS] AS c ON b.UOMCD = c.UOMCD; Here [tbl_ITEMDETAILS] JOIN [tbl_SALES] JOIN [tbl_UOMDETAILS] If we look at the Execution Plan We find that [tbl_SALES] JOIN [tbl_ITEMDETAILS] JOIN [tbl_UOMDETAILS] Step-2 [ Now we need to Force Order Hint to maintain Join Order ] SELECT b.TRID, b.ITEMCD, a.ITEMNAME, b.UOMCD, c.UOMNAME, b.QTY, b.RATE, b.AMOUNT FROM [dbo].[tbl_ITEMDTLS] AS a INNER JOIN [dbo].[tbl_SBILL] AS b ON a.ITEMCD = b.ITEMCD INNER JOIN [dbo].[tbl_UOMDTLS] AS c ON b.UOMCD = c.UOMCD OPTION ( QUERYRULEOFF JoinCommute); For this we need the FORCE ORDER Hint. The query optimizer uses different rules to evaluate different plan and one of the rules is called JoinCommute. We can turn it off using the undocumented query hint QUERYRULEOFF. Hope you like it.

Here is how I enable remote debugging with WebLogic Server (11g) and Eclipse IDE. (Actually the java option is for any JVM, just the instruction here is WLS specific.) 1. Edit /bin/setDomainEnv.sh file and add this on top: JAVA_OPTIONS="$JAVA_OPTIONS -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y" The suspend=y will start your server and wait for you to connect with IDE before continue. If you don't want this, then set to suspend=n instead. 2. Start/restart your WLS with /bin/startWebLogic.sh 3. Once WLS is running, you may connect to it using Eclipse IDE. Go to Menu: Run > Debug Configuration ... > Remote Java Application and create a new entry. Ensure your port number is matching to what you used above. Read more java debugging options here: http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html#DebuggingOptions

If you've read any of the previous blogs in this series, you may remember that I'm developing a small but almost industrial strength application that searches log files for exceptions. You may also remember that I now have a class that can contain a whole bunch of results that will need sending to any one whose interested. This will be done by implementing my simple Publisher interface shown below. public interface Publisher { public boolean publish(T report);} If you remember, the requirement was: 7 . Publish the report using email or some other technique. In this blog I’m dealing with the concrete part of the requirement: sending a report by email. As this is a Spring app, then the simplest way of sending an email is to use Spring’s email classes. Unlike those stalwarts of the Spring API, template classes such as JdbcTemplate and JmsTemplate, the Spring email classes are based around a couple of interfaces and their implementations. The interfaces are: MailSender JavaMailSender extends MailSender MailMessage …and the implementations are: JavaMailSenderImpl implements JavaMailSender SimpleMailMessage implements MailMessage Note that these are the ‘basic’ classes; you can send nicer looking, more sophisticated email content using classes such as: MimeMailMessage, MimeMailMessageHelper, ConfigurableMimeFileTypeMap and MimeMessagePreparator. Before getting down to some code, there’s the little matter of project configuration. To use the Spring email classes, you need the following entry in your Maven POM file: javax.mail mail 1.4 This ensures that the underlying Java Mail classes are available to your application. Once the Java Mail classes are configured in the build, the next thing to do is to set up the Spring XML config. For the purposes of this app, which is sending out automated reports, I’ve included two Spring beans: mailSender and mailMessage.mailSender, is a JavaMailSenderImpl instance configured to use a specific SMTP mail server, with all other properties, such as TCP port, left as defaults. The second Spring bean is mailMessage, an instance of SimpleMailMessage. This time I’ve pre-configured three properties: ‘to’, ‘from’ and ‘subject’. This is because, being automated messages, these values are always identical. You can of course configure these programatically, something you’d probably need to do if you were creating a mail GUI. All this XML makes the implementation of the Publisher very simple. @Service public class EmailPublisher implements Publisher { private static final Logger logger = LoggerFactory.getLogger(EmailPublisher.class); @Autowired private MailSender mailSender; @Autowired private SimpleMailMessage mailMessage; @Override public boolean publish(T report) { logger.debug("Sending report by email..."); boolean retVal = false; try { String message = (String) report; mailMessage.setText(message); mailSender.send(mailMessage); retVal = true; } catch (Exception e) { logger.error("Can't send email... " + e.getMessage(), e); } return retVal;} } The Publisher class contains one method: publish, which takes a generic argument T report. This, as I’ve said before, has to be the same type as the argument returned by the Formatter implementation from my previous blog. There are only really three steps in this code to consider: firstly, the generic T is cast to a String (this is where it’ll all fall over if the argument T report isn’t a String. The second step is to attach the email’s text to the mailMessage and then to send the message using mailSender.send(…). The final step is fulfil the Publisher contract by returning true, unless the email fails to send in which case the exception is logged and the return value is false. In terms of developing the code that’s about it. The next step is to sort out the scheduling, so that the report is generated on time, but more on that later… The code for this blog is available on Github at: https://github.com/roghughe/captaindebug/tree/master/error-track. If you want to look at other blogs in this series take a look here… Tracking Application Exceptions With Spring Tracking Exceptions With Spring - Part 2 - Delegate Pattern Error Tracking Reports - Part 3 - Strategy and Package Private

Welcome to part 5 of “Be a lazy but a productive android developer” series. If you are a lazy Android developer and looking for image loading library, which could help you to load image(s) asynchronously without writing a logic for downloading and caching images then this article is for you. This series so far: Part 1: We looked at RoboGuice, a dependency injection library by which we can reduce the boiler plate code, save time and there by achieve productivity during Android app development. Part 2: We saw and explored about Genymotion, which is a rocket speed emulator and super-fast emulator as compared to native emulator. And we can use Genymotion while developing apps and can quickly test apps and there by can achieve productivity. Part 3: We understood and explored about JSON Parsing libraries (GSON and Jackson), using which we can increase app performance, we can decrease boilerplate code and there by can optimize productivity. Part 4: We talked about Card UI and explored card library, also created a basic card and simple card list demo. In this part In this part, we are going to talk about some image libraries using which we can load image(s) asynchronously, can cache images and also can download images into the local storage. Required features for loading images Almost every android app has a need to load remote images. While loading remote images, we have to take care of below things: Image loading process must be done in background (i.e. asynchronously) to avoid blocking UI main thread. Image recycling image should be done. Image should be displayed once its loaded successfully. Images should be cached in local memory for the later use. If remote image gets failed (due to network connection or bad url or any other reasons) to load then it should be managed perfectly for avoiding duplicate requests to load the same again, instead it should load if and only if net connection is available. Memory management should be done efficiently. In short, we have to write a code to manage each and every aspects of image loading but there are some awesome libraries available, using which we can load/download image asynchronously. We just have to call the load image method and success/failure callbacks. Asynchronous image loading Consider a case where we are having 50 images and 50 titles and we try to load all the images/text into the listview, it won’t display anything until all the images get downloaded. Here Asynchronous image loading process comes in picture. Asynchronous image loading is nothing but a loading process which happens in background so that it doesn’t block main UI thread and let user to play with other loaded data on the screen. Images will be getting displayed as and when it gets downloaded from background threads. Asynchronous image loading libraries Nostra’s Universal Image loader – https://github.com/nostra13/Android-Universal-Image-Loader Picasso – http://square.github.io/picasso/ UrlImageViewHelper by Koush Volley - By Android team members @ Google Novoda’s Image loader – https://github.com/novoda/ImageLoader Let’s have a look at examples using Picasso and Universal Image loader libraries. Example 1: Nostra’s Universal Image loader Step 1: Initialize ImageLoader configuration ? public class MyApplication extends Application{ @Override public void onCreate() { // TODO Auto-generated method stub super.onCreate(); // Create global configuration and initialize ImageLoader with this configuration ImageLoaderConfiguration config = new ImageLoaderConfiguration.Builder(getApplicationContext()).build(); ImageLoader.getInstance().init(config); } } Step 2: Declare application class inside Application tag in AndroidManifest.xml file ? Step 3: Load image and display into ImageView ? ImageLoader.getInstance().displayImage(objVideo.getThumb(), holder.imgVideo); Now, Universal Image loader also provides a functionality to implement success/failure callback to check whether image loading is failed or successful. ? ImageLoader.getInstance().displayImage(photoUrl, imgView, new ImageLoadingListener() { @Override public void onLoadingStarted(String arg0, View arg1) { // TODO Auto-generated method stub findViewById(R.id.EL3002).setVisibility(View.VISIBLE); } @Override public void onLoadingFailed(String arg0, View arg1, FailReason arg2) { // TODO Auto-generated method stub findViewById(R.id.EL3002).setVisibility(View.GONE); } @Override public void onLoadingComplete(String arg0, View arg1, Bitmap arg2) { // TODO Auto-generated method stub findViewById(R.id.EL3002).setVisibility(View.GONE); } @Override public void onLoadingCancelled(String arg0, View arg1) { // TODO Auto-generated method stub findViewById(R.id.EL3002).setVisibility(View.GONE); } }); Example 2: Picasso Image loading straight way: ? Picasso.with(context).load("http://postimg.org/image/wjidfl5pd/").into(imageView); Image re-sizing: ? Picasso.with(context) .load(imageUrl) .resize(100, 100) .centerCrop() .into(imageView) Example 3: UrlImageViewHelper library It’s an android library that sets an ImageView’s contents from a url, manages image downloading, caching, and makes your coffee too. UrlImageViewHelper will automatically download and manage all the web images and ImageViews. Duplicate urls will not be loaded into memory twice. Bitmap memory is managed by using a weak reference hash table, so as soon as the image is no longer used by you, it will be garbage collected automatically. Image loading straight way: ? UrlImageViewHelper.setUrlDrawable(imgView, "http://yourwebsite.com/image.png"); Placeholder image when image is being downloaded: ? UrlImageViewHelper.setUrlDrawable(imgView, "http://yourwebsite.com/image.png", R.drawable.loadingPlaceHolder); Cache images for a minute only: ? UrlImageViewHelper.setUrlDrawable(imgView, "http://yourwebsite.com/image.png", null, 60000); Example 4: Volley library Yes Volley is a library developed and being managed by some android team members at Google, it was announced by Ficus Kirkpatrick during the last I/O. I wrote an article about Volley library 10 months back , read it and give it a try if you haven’t used it yet. Let’s look at an example of image loading using Volley. Step 1: Take a NetworkImageView inside your xml layout. ? Step 2: Define a ImageCache class Yes you are reading title perfectly, we have to define an ImageCache class for initializing ImageLoader object. ? public class BitmapLruCache extends LruCache implements ImageLoader.ImageCache { public BitmapLruCache() { this(getDefaultLruCacheSize()); } public BitmapLruCache(int sizeInKiloBytes) { super(sizeInKiloBytes); } @Override protected int sizeOf(String key, Bitmap value) { return value.getRowBytes() * value.getHeight() / 1024; } @Override public Bitmap getBitmap(String url) { return get(url); } @Override public void putBitmap(String url, Bitmap bitmap) { put(url, bitmap); } public static int getDefaultLruCacheSize() { final int maxMemory = (int) (Runtime.getRuntime().maxMemory() / 1024); final int cacheSize = maxMemory / 8; return cacheSize; } } Step 3: Create an ImageLoader object and load image Create an ImageLoader object and initialize it with ImageCache object and RequestQueue object. ? ImageLoader.ImageCache imageCache = new BitmapLruCache(); ImageLoader imageLoader = new ImageLoader(Volley.newRequestQueue(context), imageCache); Step 4: Load an image into ImageView ? NetworkImageView imgAvatar = (NetworkImageView) findViewById(R.id.imgDemo); imageView.setImageUrl(url, imageLoader); Which library to use? Can you decide which library you would use? Let us know which and what are the reasons? Selection of the library is always depends on the requirement. Let’s look at the few fact points about each library so that you would able to compare exactly and can take decision. Picasso: It’s just a one liner code to load image using Picasso. No need to initialize ImageLoader and to prepare a singleton instance of image loader. Picasso allows you to specify exact target image size. It’s useful when you have memory pressure or performance issues, you can trade off some image quality for speed. Picasso doesn’t provide a way to prepare and store thumbnails of local images. Sometimes you need to check image loading process is in which state, loading, finished execution, failed or cancelled image loading. Surprisingly It doesn’t provide a callback functionality to check any state. “fetch()” dose not pass back anything. “get()” is for synchronously read, and “load()” is for asynchronously draw a view. Universal Image loader (UIL): It’s the most popular image loading library out there. Actually, it’s based on the Fedor Vlasov’s project which was again probably a very first complete solution and also a most voted answer (for the image loading solution) on Stackoverflow. UIL library is better in documentation and even there’s a demo example which highlights almost all the features. UIL provides an easy way to download image. UIL uses builders for customization. Almost everything can be configured. UIL doesn’t not provide a way to specify image size directly you want to load into a view. It uses some rules based on the size of the view. Indirectly you can do it by mentioning ImageSize argument in the source code and bypass the view size checking. It’s not as flexible as Picasso. Volley: It’s officially by Android dev team, Google but still it’s not documented. It’s just not an image loading library only but an asynchronous networking library Developer has to define ImageCache class their self and has to initialize ImageLoader object with RequestQueue and ImageCache objects. So now I am sure now you can be able to compare libraries. Choosing library is a bit difficult talk because it always depends on the requirement and type of projects. If the project is large then you should go for Picasso or Universal Image loader. If the project is small then you can consider to use Volley librar, because Volley isn’t an image loading library only but it tries to solve a more generic solution.). I suggest you to start with Picasso. If you want more control and customization, go for UIL. Read more: http://blog.bignerdranch.com/3177-solving-the-android-image-loading-problem-volley-vs-picasso/ http://stackoverflow.com/questions/19995007/local-image-caching-solution-for-android-square-picasso-vs-universal-image-load https://plus.google.com/103583939320326217147/posts/bfAFC5YZ3mq Hope you liked this part of “Lazy android developer: Be productive” series. Till the next part, keep exploring image loading libraries mentioned above and enjoy!

Recently I had to integrate a Node.js based server application with a C# DLL. Our software (a web-app) offers the possibility to execute payments over a POS terminal. This latter one is controllable through a dedicated DLL which exposes interfaces like ExecutePayment(operation, amount) and so on. As I mentioned, there is the Node.js server that somehow exposes the functionality of the POS (and some more) as a REST api. (The choice for using Node.js had specific reasons which I wouldn't want to outline right now). When you start with such an undertaking, then there are different possibilities. One is to use Edge.js which allows you to embed, reference and invoke .Net CLR objects from within your Node.js based applications. Something like this: var hello = require('edge').func({ assemblyFile: 'My.Edge.Samples.dll', typeName: 'Samples.FooBar.MyType', methodName: 'MyMethod' // Func> }); hello('Node.js', function (error, result) { ... }); Edge is a very interesting project and has a lot of potential. In fact, I just tried it quickly with a simple DLL and it worked right away. However, when using it from my Node app within node-webkit it didn't work. I'm not yet sure whether it was related to node-webkit or the POS DLL itself (because it might be COM exposed etc..). However, if you need simple integrations this might work well for you. Process invocation A second option that came to my mind is to design the DLL as a self-contained process and to invoke it using Node.js's process api. Turns out this is quite simple. Just prepare your C# application to read it's invocation arguments s.t. you can do something like.. IntegrationConsole.exe ExecutePayment 1 100 ..to "ExecutePayment" with operation number 1 and an amount of 1€. The C# console application needs to communicate it's return values to the STDOUT (you may use JSON for creating a more structured information exchange protocol format). Once you have this, you can simply execute the process from Node.js and read the according STDOUT: var process = require('child_process'); ... process.exec(execCmd, function (error, stdout, stderr) { var result = stdout; ... writeToResponse(stdout); }); execCmd is holds the instructions required to launch the EXE with the required invocation arguments. In this approach you execute the process, it does its job, returns the response and terminates. If for some reason however, you need to keep the process running for having a longer, kind of more interactive communication between the two components, you can communicate through the STDIN/STDOUT of the process. Your C# console application starts and listens on the STDIN.. static void Main(string[] args) { ... string line; do { line = Console.ReadLine(); try { // do something meaningful with the input // write to STDOUT to respond to the caller } catch (Exception e) { Console.WriteLine(e.Message); } } while (line != null); } On the Node.js side you do not exec your process, but instead you spawn a child process. var spawn = require('child_process').spawn; ... var posProc = spawn('IntegrationConsole.exe', ['ExecutePayment', 1, 100]); For getting the responses, you simply register on the STDOUT of the process... posProc.stdout.once('data', function (data) { // write it back on the response object writeToResponse(data); }); ..and you may also want to listen for when the process dies to eventually perform some cleanup. posProc.on('exit', function (code) { ... }); Writing to the STDIN of the process is simple as well: posProc.stdin.setEncoding ='utf-8'; posProc.stdin.write('...'); In this way you have a more interactive, "stateful communication", where you send a command to the EXE which responds (STDOUT) and based on the response you again react and send some other command (STDIN). Embedding this in the Request/Response pattern To expose everything as a REST api (on Node), you need to pay some attention on the registration of the event handlers on STDOUT. Suppose you do something like app.post('/someEndpoint',function(req, res){ posProc = spawn('IntegrationConsole.exe',['ExecutePayment',1,100]);... posProc.stdout.on('data',function(data){// return the result of this execution// on the response});}), app.post('/someOtherEndpoint',function(req, res){... posProc.stdout.on('data',function(data){// return the result of this execution// on the response});// write to the stdin of the before created child process posProc.stdin.setEncoding ='utf-8'; posProc.stdin.write('...');}); I excluded proper edge case handling like what happens if your process died before etc.. but the key point here is that you cannot register your events by using on(..), as otherwise you'll end up having multiple data event handlers on the stdout. So you can either register and de-register the event by using the removeListener('event name', callback) syntax or use the more handy once registration mechanism (as I did already in my samples at the beginning of the article): posProc.stdout.once('data',function(data){// write it back on the response object writeToResponse(data);});