My objective here is to recreate a similar set-up in a smaller unit test mode.

Recently I wanted to extract certain data from an output log. Here’s part of the log file: 2015-01-06 11:33:03 b.s.d.task [INFO] Emitting: eVentToRequestsBolt __ack_ack [-6722594615019711369 -1335723027906100557] 2015-01-06 11:33:03 c.s.p.d.PackagesProvider [INFO] ===---> Loaded package com.foo.bar 2015-01-06 11:33:04 b.s.d.executor [INFO] Processing received message source: eventToManageBolt:2, stream: __ack_ack, id: {}, [-6722594615019711369 -1335723027906100557] 2015-01-06 11:33:04 c.s.p.d.PackagesProvider [INFO] ===---> Loaded package co.il.boo 2015-01-06 11:33:04 c.s.p.d.PackagesProvider [INFO] ===---> Loaded package dot.org.biz I decided to do it using the Java8 Stream and Lambda Expression features. Read the file First, I needed to read the log file and put the lines in a Stream: Stream lines = Files.lines(Paths.get(args[1])); Filter relevant lines I needed to get the packages names and write them into another file. Not all lines contained the data I need, hence filter only relevant ones. lines.filter(line -> line.contains("===---> Loaded package")) Parsing the relevant lines Then, I needed to parse the relevant lines. I did it by first splitting each line to an array of Strings and then taking the last element in that array. In other words, I did a double mapping. First a line to an array and then an array to a String. .map(line -> line.split(" ")) .map(arr -> arr[arr.length - 1]) Writing to output file The last part was taking each string and write it to a file. That was the terminal operation. .forEach(package -> writeToFile(fw, package)); writeToFile is a method I created. The reason is that Java File System throws IOException. You can’t use checked exceptions in lambda expressions. Here’s a full example (note, I don’t check input) import java.io.FileWriter; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Paths; import java.util.Arrays; import java.util.List; import java.util.stream.Stream; public class App { public static void main(String[] args) throws IOException { Stream lines = null; if (args.length == 2) { lines = Files.lines(Paths.get(args[1])); } else { String s1 = "2015-01-06 11:33:03 b.s.d.task [INFO] Emitting: adEventToRequestsBolt __ack_ack [-6722594615019711369 -1335723027906100557]"; String s2 = "2015-01-06 11:33:03 b.s.d.executor [INFO] Processing received message source: eventToManageBolt:2, stream: __ack_ack, id: {}, [-6722594615019711369 -1335723027906100557]"; String s3 = "2015-01-06 11:33:04 c.s.p.d.PackagesProvider [INFO] ===---> Loaded package com.foo.bar"; String s4 = "2015-01-06 11:33:04 c.s.p.d.PackagesProvider [INFO] ===---> Loaded package co.il.boo"; String s5 = "2015-01-06 11:33:04 c.s.p.d.PackagesProvider [INFO] ===---> Loaded package dot.org.biz"; List rows = Arrays.asList(s1, s2, s3, s4, s5); lines = rows.stream(); } new App().parse(lines, args[0]); } private void parse(Stream lines, String output) throws IOException { final FileWriter fw = new FileWriter(output); //@formatter:off lines.filter(line -> line.contains("===---> Loaded package")) .map(line -> line.split(" ")) .map(arr -> arr[arr.length - 1]) .forEach(package -> writeToFile(fw, package)); //@formatter:on fw.close(); lines.close(); } private void writeToFile(FileWriter fw, String package) { try { fw.write(String.format("%s%n", package)); } catch (IOException e) { throw new RuntimeException(e); } } } If you enjoyed this article and want to learn more about Java Streams, check out this collection of tutorials and articles on all things Java Streams.

Byte Buddy is a code generation library for creating Java classes during the runtime of a Java application and without the help of a compiler. Other than the code generation utilities that ship with the Java Class Library, Byte Buddy allows the creation of arbitrary classes and is not limited to implementing interfaces for the creation of runtime proxies. In order to use Byte Buddy, one does not require an understanding of Java byte code or the class file format. In contrast, Byte Buddy’s API aims for code that is concise and easy to understand for everybody. Nevertheless, Byte Buddy remains fully customizable down to the possibility of defining custom byte code. Furthermore, the API was designed to be as non-intrusive as possible and as a result, Byte Buddy does not leave any trace in the classes that were created by it. For this reason, the generated classes can exist without requiring Byte Buddy on the class path. Because of this feature, Byte Buddy’s mascot was chosen to be a ghost. Byte Buddy is written in Java 6 but supports the generation of classes for any Java version. Byte Buddy is a light-weight library and only depends on the visitor API of the Java byte code parser library ASM which does itself not require any further dependencies. At first sight, runtime code generation can appear to be some sort of black magic that should be avoided and only few developers write applications that explicitly generate code during their runtime. However, this picture changes when creating libraries that need to interact with arbitrary code and types that are unknown at compile time. In this context, a library implementer must often choose between either requiring a user to implement library-proprietary interfaces or to generate code at runtime when the user’s types becomes first known to the library. Many known libraries such as for example Spring or Hibernate choose the latter approach which is popular among their users under the term of using Plain Old Java Objects. As a result, code generation has become an ubiquitous concept in the Java space. Byte Buddy is an attempt to innovate the runtime creation of Java types in order to provide a better tool set to those relying on code generation. Hello World Saying Hello World with Byte Buddy is as easy as it can get. Any creation of a Java class starts with an instance of the ByteBuddy class which represents a configuration for creating new types: Class dynamicType = new ByteBuddy() .subclass(Object.class) .method(named("toString")).intercept(FixedValue.value("Hello World!")) .make() .load(getClass().getClassLoader(), ClassLoadingStrategy.Default.WRAPPER) .getLoaded(); assertThat(dynamicType.newInstance().toString(), is("Hello World!")); The default ByteBuddy configuration which is used in the above example creatse a Java class in the newest version of the class file format that is understood by the processing Java virtual machine. As hopefully obvious from the example code, the created type will extend the Object class and intercept its toString method which should return a fixed value of Hello World!. The method to be intercepted is identified by a so-called method matcher. In the above example, a predefined method matcher named(String) is used which identifies a method by its exact name. Byte Buddy comes with numerous predefined and well-tested method matchers which are collected in the MethodMatchers class. The creation of custom matchers is however as simple as implementing the (functional) MethodMatcher interface. For implementing the toString method, the FixedValue class defines a constant return value for the intercepted method. Defining a constant value is only one example of many method interceptors that ship with Byte Buddy. By implementing the Instrumentation interface, a method could however even be defined by custom byte code. Finally, the described Java class is created and then loaded into the Java virtual machine. For this purpose, a target class loader is required as well as a class loading strategy where we choose a wrapper strategy. The latter creates a new child class loader which wraps the given class loader and only knows about the newly created dynamic type. Eventually, we can convince ourselves of the result by calling the toString method on an instance of the created class and finding the return value to represent the constant value we expected. Where to go from here? Byte Buddy is a comprehensive library and we only scratched the surface of Byte Buddy's capabilities. However, Byte Buddy aims for being easy to use by providing a domain-specific language for creating classes. Most runtime code generation can be done by writing readable code and without any knowledge of Java's class file format. If you want to learn more about Byte Buddy, you can find such a tutorial on Byte Buddy's web page. Furthermore, Byte Buddy comes with a detailed in-code documentation and extensive test case coverage which can also serve as example code. you can also find the source code of Byte Buddy on GitHub .

Introduction So your thinking on designing your own DSL or to use an existing grammar from ANTLR, but how do you execute it? That is what this tutorial is about. Groovy is great for building DSL's, but the expressiveness is limited to language features already included in the Groovy syntax. ANTLR offers a much larger set of language features that you can use. Just look at the language's already available at: GitHub - ANTLR 4 Grammars. You could pick one grammar file, or two, modify them to your needs and rapidly create your DSL. One thing is missing though from all books and articles on DSL's with groovy and ANTLR, at least from the ones I've read, and that is, how to run a script written in your newly created DSL? With ANTLR, you can parse the script, but you'll need to write a Java or Groovy program with a main class to read in the file. This may be sufficient for certain use cases, for example when translating between two syntaxes, or when reading in a configuraiton file, or when you have a command language for your server, but in certain cases you just want the customers of your DSL to be able to use it as a 'real' language. In this article I will demonstrate how the powerful Groovy AST transformations can be used to execute a script written in a ANTLR grammar with a custom extension as a Groovy script. If you're new to Groovy DSL's and AST transformations I recommend to read the following articles as well: Joe's Groovy Blog on AST Transformations Global AST Transformations Groovy Global AST Transformations Groovy makes a distinction between a global and a local AST Transformation. A global AST Transformation is fired once and has the entire script, or SourceUnit, as scope. A local AST Transformation is fired when found and has the statement where it is found as scope. Global transformations are always executed when present on the classpath, every time the groovy compiler is invoked. Local transformations are only invoked when they are present in a script. Local transformations are useful when a single statement needs to be transformed. Global transformations are useful when the entire script needs to be transformed. In this example we will use global transformations. Global transformations have an extra cool feature: you can specify your own file extension. This feature will be used to specify a syntax with ANTLR, and then run it with Groovy. Our DSL: the Cymbol language Cymbol is a little language written by Terence Parr and published in his book: The Definitive ANTLR 4 Reference. Cymbol only contains the basic C-style syntax needed to write declarations and functions. It is a good point to start if you want to implement your own DSL with some extra features. The grammar file is available at GitHub: Cymbol.g4. This file is included in the grammar directory of the sample workspace. You can install the ANTLRWorks workbench, if you want to try the examples out for yourself. The workbench can be found at: Tunnel Vision Labs. The ANTLR Workbench was used to generate the necessary classes to parse the DSL. This is however not necessary to run this example and all needed classes are already included in the attached workspace zip file. A sample Cymbol program could look as follows (taken from the ANTLR 4 Reference, p. 99): // Cymbol test int g = 9; // a global variable int fact(int x) { // a factorial function if x==0 then return 1; return x * fact(x-1); } Set up the workspace Eclipse is used as IDE. To start, you can download the Cymbol_workspace.zip file and import it in Eclipse. After that, you'll need to download a few libraries, the antlr_4.4_complete.jar, hamcrest_1.3.jar and junit_4.11.jar. Due to upload size limitations I couldn't include the libraries in the zip file. Links to the download sites are provided in the README.txt file in the libs directory of the Eclipse project. Add the libraries to the build path and the project set up is complete. Now you can execute DslScriptTransformTest.java as JUnit test and you should see some output in the console. What is going on behind the scenes here? Generate the Parser Open the Cymbol.g4 grammar file in the ANTLR Workbench. Go to: Run > Generate Recognizer... Select the ouput directory, tick the boxes in front of Generate Listener and Generate Visitor. Don't forget to set the package to com.cymbol.parser. This will generate all classes that you'll need to parse a script written with your DSL's syntax. Create the ASTTransformation The DslScriptTransform.java file contains the ASTTransformation. The transformation class needs to implement the ASTTransformation interface from Groovy. An AST Transformation is annotated with: @GroovyASTTransformation(phase = CompilePhase.SEMANTIC_ANALYSIS) This annotation tells the Groovy compiler that this class is an AST Transformation that should be applied in the specified phase. Global transformations can occur in any compiler phase so here are a few words on how to select the right phase. The Groovy compiler hase nine phases. Of these, only the second, third and fourth are relevant for us: Initialization The resources and environment are initialized. Parsing The script text is parsed and a Concrete Syntax Tree (CST) is constructed according to the Groovy grammar. Conversion The Astract Syntaxt Tree (AST) is constructed from the CST. Semantic Analysis In this phase, the AST is checked, classes, static imports and variables are resolved. After this phase, the AST is complete and the input is seen as a valid source. Our DSL script has been transformed into a Groovy script with one method: getScriptText(). In the semantic analysis phase, all necessary Groovy structures have been initialised and we can safely modify the AST. The only thing that needs to be done now is to implement the run() method to call the ANTLR Parser and parse the script text. First, we check whether the file has the required extension: @Override public void visit(ASTNode[] nodes, SourceUnit source) { if(!source.getName().endsWith(".cymbol")) { return; } ... Second, the main class is retrieved from the AST: private ClassNode getMainClass(ModuleNode moduleNode) { for(ClassNode classNode : moduleNode.getClasses()) { if(classNode.getNameWithoutPackage().equals(moduleNode.getMainClassName())) { return classNode; } } return null; } And the run method's body is imlemented. The run method's body will look something like follows: @Override public Object run() { DslScriptTransformHelper helper = new DslScriptTransformHelper(); String scriptText = this.getScriptText(); helper.parse(scriptText); } The @Override annotation is not actually there but is included here for clarity. The annotation could be added as well but it is left out to keep the example simple. The implementation is kept as minimal as possible because writing AST Transformations is rather cumbersome. As much as posisble is factored out into a delegate called DslScriptTransformHelper. The helper class contains the actual code to parse the DSL script and to do something with it. The benefit is that complex logic can be implemented that can be checked by the compiler and by your IDE. Third, the AST Transformation to implement the run method's body is as follows: private void addRunMethod(ClassNode scriptClass) { List statements = new ArrayList<>(); /* * initialise the parser helper: * * DslScriptTransformHelper helper = new DslScriptTransformHelper() */ ClassNode helperClassNode = new ClassNode(DslScriptTransformHelper.class); ConstructorCallExpression initParserHelper = new ConstructorCallExpression(helperClassNode, new ArgumentListExpression()); VariableExpression helperVar = new VariableExpression(HELPER_VAR, helperClassNode); DeclarationExpression assignHelperExpr = new DeclarationExpression(helperVar, new Token(Types.EQUALS, "=", -1, -1), initParserHelper); ExpressionStatement initHelperExpr = new ExpressionStatement(assignHelperExpr); statements.add(initHelperExpr); /* * get the script text and assign it to a variable: * * String scriptText = this.getScriptText() */ MethodCallExpression getScriptTextExpr = new MethodCallExpression(new VariableExpression("this"), new ConstantExpression(GET_SCRIPT_TEXT_METHOD), MethodCallExpression.NO_ARGUMENTS); VariableExpression scriptTextVar = new VariableExpression(SCRIPT_TEXT_VAR, new ClassNode(String.class)); DeclarationExpression declareScriptTextExpr = new DeclarationExpression(scriptTextVar, new Token(Types.EQUALS, "=", -1, -1), getScriptTextExpr); ExpressionStatement getScriptTextStmt = new ExpressionStatement(declareScriptTextExpr); statements.add(getScriptTextStmt); /* * call the parse method on the helper: * * helper.parse(scriptText) */ MethodCallExpression callParserExpr = new MethodCallExpression(helperVar, PARSE_METHOD, new ArgumentListExpression(new VariableExpression(SCRIPT_TEXT_VAR))); ExpressionStatement callParserStmt = new ExpressionStatement(callParserExpr); statements.add(callParserStmt); // implement the run method's body BlockStatement methodBody = new BlockStatement(statements, new VariableScope()); MethodNode runMethod = scriptClass.getMethod(RUN_METHOD, new Parameter[0]); runMethod.setCode(methodBody); } All the above is needed to implement three lines of code! Fifth, the DslScriptTransformHelper class contains a parse method that calls the ANTLR parser and then does something with the ParseTree returned by ANTLR. You will probably want a semantic model, and I included a visitor that can iterate over the model to do something with it. This bit is left unimplemented because it requires, a lot, of design and coding to get this right. Here is a simple example: public class DslScriptTransformHelper { public SemanticModel parse(String scriptText) throws IOException { ANTLRInputStream is = new ANTLRInputStream(scriptText); CymbolLexer lexer = new CymbolLexer(is); CommonTokenStream tokens = new CommonTokenStream(lexer); CymbolParser parser = new CymbolParser(tokens); ParseTree tree = parser.file(); SemanticModel model = initSemanticModel(tree); SemanticModelVisitor visitor = initVisitor(); visitor.visit(model); return model; } Here, the generated Lexer and Parser are called. If you would want to create your own DSL with ANTLR, you will have to modify the DslScriptTransformHelper. The AST Transformation, DslScriptTransform, need not be modified. As much logic is factored out of the AST Transformation as posisble into a helper class. As you will see in the next section, creating an AST Transformation is difficult, time consuming, error prone, and generally not something that you will enjoy doing. by factoring out behaviour into a helper you can write logic and still have IDE support, and Unit testability for your logic. All you need to do is to initialize the helper and call a method on the helper with the DSL script's text as argument. I included two interfaces that are left unimplemented: SemanticModel and SemanticModelVisitor. private SemanticModel initSemanticModel(ParseTree tree) { SemanticModel model = new SemanticModel() { @Override public void init(ParseTree tree) { System.out.println("init: " + tree.toStringTree()); } }; model.init(tree); return model; } private SemanticModelVisitor initVisitor() { SemanticModelVisitor visitor = new SemanticModelVisitor() { @Override public void visit(SemanticModel model) { System.out.println("visit"); } }; return visitor; } } The purpose of these is that the semantic model represents, well, your semantic model. This is so to say your domain model. In the design of your DSL, deciding on a semantic model is probably the most important activity. The semantic model models what the DSL means. The grammar specifies how to write a valid script, but the model specifies what it means. Think of it as follows: not all valid English sentences are meaningful. Martin Fowler discusses this in his book: Domain Specific Languages. The visitor is included so that after you've constructed a valid semantic model, you can do something with it, if you want. As you can see from the above example, performing an AST Transformation is rather cumbersome, at least in Java. Surely Groovy offers something better? Groovy does, and there are several excellent examples on this available at: GitHub. To me, the major downside to using the Groovy Builder DSL to specify AST Transformations is that the types can't be checked. It is rather difficult to get everything right, especially without type checking. The Cymbol DSL implementation uses the Groovy compiler, but doesn't actually contain any Groovy code. I think that is a benefit because fiddling with compilers and AST's is difficult enough, even with strict typing and IDE support. The examples can be implemented in Groovy as well if you wish so. Create the Groovy configuration for the AST Transformations Groovy needs the following configuration files to know that an AST Transformation should be called, and that it should accept files with a custom extension. Two files must be added: META-INF/services/org.codehaus.groovy.source.Extensions The extension is just declared in the org.codehaus.groovy.source.Extensions file: .cymbol META-INF/services/org.codehaus.groovy.transform.ASTTransformation This file only contains the name with package of the class that contains the AST Transformation: com.dsl.transformation.DslScriptTransform Create a Source Pre-Processor Groovy supports custom compiler configurations. They are documented at: Advanced Compiler Configuration. We will use a custom compiler configuration to create a source pre-processor that reads the source of a script, a Cymbol script in our case, and that modifies the source. This happens before the Groovy compiler tries to parse the source and thus can be used as a source pre-processor. The text of the source is wrapped in a String and a method that returns that string. The Cymbol example is transformed into the following Groovy script: String getScriptText() { return '''// Cymbol test int g = 9; // a global variable int fact(int x) { // a factorial function if x==0 then return 1; return x * fact(x-1); }''' } The method getScriptText() now returns the DSL script as string. The Groovy source pre-processor is implemented as follows. First, a custom CustomParserPlugin is created. The CustomParserPlugin extends the Groovy AntlrParserPlugin and overrides the parseCST() method. This method is similar to the following example from Guillaume Laforge: Custom antlr parser plugin. Here the example is taken a step further however by calling ANTLR's StringTemplate to operate on the entire source, instead of modifiying the Groovy source by remapping the bindings, as is done in Guillaume's example. public class CustomParserPlugin extends AntlrParserPlugin { @Override public Reduction parseCST(final SourceUnit sourceUnit, Reader reader) throws CompilationFailedException { StringBuffer bfr = new StringBuffer(); int intValueOfChar; try { while ((intValueOfChar = reader.read()) != -1) { bfr.append((char) intValueOfChar); } } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } String text = modifyTextSource(bfr.toString()); StringReader stringReader = new StringReader(text); SourceUnit modifiedSource = SourceUnit.create(sourceUnit.getName(), text); return super.parseCST(modifiedSource, stringReader); } String modifyTextSource(String text) { ST textAsStringConstant = new ST("String getScriptText() { return '''''' }"); textAsStringConstant.add("text", text); return textAsStringConstant.render(); } } A CompilerConfiguration is equipped with a ParserPluginFactory and we also create a custom implementation that returns our CustomParserPlugin: public class SourcePreProcessor extends ParserPluginFactory { @Override public ParserPlugin createParserPlugin() { return new CustomParserPlugin(); } } A CompilerConfiguration needs to be created with our implementation of the ParserPluginFactory. Several ways of doing this are available to us: Use Groovy embedded (GroovyShell, GroovyScriptEngine, ...) and add a CompilerConfiguration Use the groovyc compiler with a configscript Both techniques will be presented. Embedded Groovy is used in the Unit test and the Groovy compiler with the configuration script is invoked from the command line. Create a Unit test with embedded groovy The Unit test simply evaluates the DSL script with a GroovyShell that has a custom CompilerConfiguration with our SourcePreProcessor: public class DslScriptTransformTest { String scriptPath = "tests"; @Test public void testPreProcessor() throws CompilationFailedException, IOException { String testScript = scriptPath + "/test1.cymbol"; File scriptFile = new File(testScript); ParserPluginFactory ppf = new SourcePreProcessor(); CompilerConfiguration cc = new CompilerConfiguration(); cc.setPluginFactory(ppf); Binding binding = new Binding(); GroovyShell shell = new GroovyShell(binding, cc); shell.evaluate(scriptFile); } } If you now call the testPreProcessor() method you should see some output on the console with the output from ANTLR's tree.toStringTree() method call and a message that the semantic models has been visited. Create a compiler configuration script First, a CompilerConfiguration is created with a custom ParserPluginFactory, and passed to the GroovyShell. Second, the groovyc compiler is invoked with the --configscript flag. The config script has a CompilerConfiguration injected before any files are compiled. The CompilerConfiguration is exposed as a variable named configuration. The compiler configuration script config.groovy looks as follows: configuration.setPluginFactory(new com.dsl.transformation.SourcePreProcessor()) We can simply set the SourcePreProcessor on the configuration as a PluginFactory. Export the Jar file With Eclipse, you can export the project as a Jar file by selecting: Export > Java > JAR File You don't need the libs folder because Jar files that are included in another Jar file aren't available on the class path. What you can do is to create a Manifest.MF file with the following class path entries: Class-Path: libs/antlr-4.4-complete.jar libs/hamcrest-core-1.3.jar This makes sure that if you run the jar file from the right location, the required libraries are on the class path. This is not bullet proof production code, but for testing it is sufficient. Now that you exported a Jar file all you need to do is to include the Jar file with your AST Transformations on the class path and Groovy will automatically understand what to do. The DSL script is invoked from the command line with the following command: groovy -cp Cymbol.jar --configscript config.groovy tests\test1.cymbol You can also invoke the groovyc compile and compile your DSL script into a Java class. The command is the same, really: groovyc -cp Cymbol.jar --configscript config.groovy tests\test1.cymbol In case you have a DSL in which you have customers that write scripts with it, you can now also compile it with a build system such as Ant or Gradle and package the results in a JAR file and deliver it to a production system. Conclusion In the previous sections you have seen a technique to develop a DSL and execute it as a Groovy script, or to compile the DSL script into a Java class file. It requires quite a lot of groovy's DSL features, but it is a good show case on what you can do with Groovy's superb DSL features and how to succesfully apply them. And I didn't even write a single line of Groovy code! The techniques that have been covered are: Generate an ANTLR Parser from a grammar file Create a Groovy source pre-processor Create Global AST Transformations that delegate to a helper class for easy implementations Execute a DSL script with a custom CompilerConfiguration with embedded Groovy or from the command line Compile a DSL script with an ANTLR grammar into a Java .class file with a build system That was a lot of ground to cover. One open point remains tool support. Now that I have my DSL, can I at least have syntax high-lighting? I hope to return to this, and other topics involving DSL design and development, in a next article.

History I think it’s important to take a look at the evolution of Interceptors in Java EE because of the simple fact that it started as an EJB-specific item and later evolved into a separate spec which is now open for extension by other Java EE specifications. Version 1.0 Interceptors were first introduced in EJB 3.0 (part of Java EE 5). Interceptors did not have a dedicated spec but they were versioned 1.0 and bought basic AOP related features to managed beans (POJOs) via simple annotations @AroundInvoke – to annotate methods containing the interception logic for target class methods @Intercerptors – to bind the the interceptor classes with their target classes/methods Capability to configure interceptors for an entire module (EJB JAR) via the deployment descriptor @ExcludeDefaultInterceptors – to mute default interceptors defined in the deployment descriptor @ExcludeClassInterceptors – to mute a globally defined (class level) interceptor for a particular method/constructor of the class Interceptors 1.1 Along came Java EE 6 with EJB 3.1 – Interceptors 1.1 was still included in the EJB spec document @InterceptorBinding – a type safe way of specifying interceptors of a class or a method. Please note that this annotation was leveraged by CDI 1.0 (another specification introduced in Java EE 6) and its details are present in the CDI 1.0 spec doc rather than EJB 3.1 (light bulb moment … at least for me) @Interceptor – Used to explicitly declare a class containing an interception logic in a specific method (annotated with @AroundInvoke etc) as an interceptor along with an appropriate Interceptor Binding. This too was mentioned in the CDI 1.0 documentation only. @AroundTimeout – used to intercept time outs of EJB timers along with a way to obtain an instance of the Timer being intercepted (viajavax.interceptor.InvocationContext.getTimer()) Interceptors 1.2 Interceptors were split off into an individual spec in Java EE 7 and thus Interceptors 1.2came into being Interceptors 1.2 was a maintenance release on top of 1.1 and hence the JSR number still remained the same as EJB 3.1 (JSR 318) Interceptor.Priority (static class) – to provide capability to define the order (priority) in which the interceptors need to invoked. @AroundConstruct – used to intercept the construction of the target class i.e. invoke logic prior to the constructor of the target class is invoked It’s important to bear in mind that Interceptors are applicable to managed beans in general. Managed Beans themselves are simple POJOs which are privileged to basic services by the container – Interceptors are one of them along with life cycle callbacks, resource injection. Memory Aid It’s helpful to think of Interceptors as components which can interpose on beans throughout their life cycle before they are even constructed – @AroundConstruct after they are constructed – @PostConstruct during their life time (method invocation) – @AroundInvoke prior to destruction – @PreDestroy time outs of EJBs – @AroundTimeout Let’s look at some of the traits of Interceptors in more detail and try to answer questions like where are they applied and what do they intercept ? how to bind interceptors to the target (class) they are supposed to intercept ? Interceptors Types (based on the intercepted component) Method Interceptors Achieved by @AroundInvoke public class MethodInterceptor{ @AroundInvoke public Object interceptorMethod(InvocationContext ictx) throws Exception{ //logic goes here } } @Stateless public class AnEJB{ @Interceptors(MethodInterceptor.class) public void bizMethod(){ //any calls to this method will be intercepted by MethodInterceptor.interceptorMethod() } } The method containing the logic can be part of separate class as well as the target class (class to be intercepted) itself. Lifecycle Callback interceptors Decorate the method with @AroundConstruct in order to intercept the constructor invocation for a class public class ConstructorInterceptor{ @AroundConstruct public Object interceptorMethod(InvocationContext ictx) throws Exception{ //logic goes here } } public class APOJO{ @Interceptors(ConstructorInterceptor.class) public APOJO(){ //any calls to this constructor will be intercepted by ConstructorInterceptor.interceptorMethod() } } The method annotated with @AroundConstruct cannot be a part of the intercepted class. It has to be defined using a separate Interceptor class Use the @PostConstruct annotation on a method in order to intercept a call back method on a managed bean. Just to clarify again – the Interceptor spec does not define a new annotation as such. One needs to reuse the @PostConstruct (part of theCommon Annotations spec) on the interceptor method. public class PostConstructInterceptor{ @PostConstruct public void interceptorMethod(InvocationContext ictx) throws Exception{ //logic goes here } } @Interceptors(PostConstructInterceptor.class) public class APOJO{ @PostConstruct public void bizMethod(){ //any calls to this method will be intercepted by PostConstructInterceptor.interceptorMethod() } } The @PreDestroy (another call back annotation defined in Common Annotations spec) annotation is used in a similar fashion Time-out Interceptors As mentioned above – @AroundTimeout used to intercept time outs of EJB timers along with a way to obtain an instance of the Timer being intercepted (viajavax.interceptor.InvocationContext.getTimer()) Applying/Binding Interceptors Using @Interceptors As shown in above examples – just use the @Interceptors annotation to specify the interceptor classes @Interceptors can be applied on a class level (automatically applicable to all the methods of a class), to a particular method or multiple methods and constructor in case of a constructor specific interceptor using @AroundConstruct Using @IntercerptorBinding Interceptor Bindings (explained above) – Use @IntercerptorBinding annotation to define a binding annotation which is further used on the interceptor class as well as the target class (whose method, constructor etc needs to be intercepted) @InterceptorBinding @Target({TYPE, METHOD, CONSTRUCTOR}) @Retention(RUNTIME) public @interface @Auditable { } @Auditable @Interceptor public class AuditInterceptor { @AroundInvoke public Object audit(InvocationContext ictx) throws Exception{ //logic goes here } } @Stateless @Auditable public class AnEJB{ public void bizMethod(){ //any calls to this method will be intercepted by AuditInterceptor.audit() } } Deployment Descriptor One can also use deployment descriptors to bind interceptors and target classes either in an explicit fashion as well as in override mode to annotations. This was a rather quick overview of Java EE interceptors. Hopefully the right trigger for you to dig deeper :-) Cheers !

This is a step by step guide on How to Configure MySQL Metastore for Hive in place of Derby Metastore (Default).

Recently at DevSKiller.com we've decided to move majority of our stuff to simple containers. It was pretty easy due to use of Spring Boot uber-jars, but the problem was in NewRelic agents which should have to be included separately. That caused uncomfortable situation so we decided to solve it by including NewRelic agent into our uber-jar applications. If you also want to simplify your life please follow provided instructions :) At first we have to add proper dependency into our pom.xml descriptor: com.newrelic.agent.java< newrelic-agent 3.12.1 provided Now since we have proper jar included into our project it's time to unpack the dependency to have all necessary classes in our application jar file: org.apache.maven.plugins maven-dependency-plugin 2.9 prepare-package unpack-dependencies newrelic-agent ${project.build.outputDirectory} After this step we've all agent related classes accessible directly from our jar. But still the file cannot be used as an agent jar. There are some important manifest entries that have to be present in every agent jar. The most important is the Premain-Class attribute specifying main agent class including premain() method. In case of NewRelic it's also important to include Can-Redefine-Classes and Can-Retransform-Classes attributes. The easiest way to do that is to extend maven-jar-plugin configuration: org.apache.maven.plugins maven-jar-plugin 2.5 com.newrelic.bootstrap.BootstrapAgent true true Now is coming the tricky part :) NewRelic agent also contains class with main() method which causes that Spring Boot repackager plugin is unable to find single main() method so build fails. It's not a problem but we have to remember to specify proper main class in spring-boot-maven-plugin (or in gradle plugin): my.custom.Application That's all! You can execute your application with following command: java -javaagent:myapp.jar -jar myapp.jar Last but not least: don't forget to include NewRelic configuration file (newrelic.yml) in the same directory as your application jar. The other solution is to set newrelic.config.file system property to point the fully qualified file name.

I was recently asked about the benefits and wisdom of using off heap memory in Java. The answers may be of interest to others facing the same choices. Off heap memory is nothing special. The thread stacks, application code, NIO buffers are all off heap. In fact in C and C++, you only have unmanaged memory as it does not have a managed heap by default. The use of managed memory or "heap" in Java is a special feature of the language. Note: Java is not the only language to do this. new Object() vs Object pool vs Off Heap memory. new Object() Before Java 5.0, using object pools was very popular. Creating objects was still very expensive. However, from Java 5.0, object allocation and garbage cleanup was made much cheaper, and developers found they got a performance speed up and a simplification of their code by removing object pools and just creating new objects whenever needed. Before Java 5.0, almost any object pool, even an object pool which used objects provided an improvement, from Java 5.0 pooling only expensive objects obviously made sense e.g. threads, socket and database connections. Object pools In the low latency space it was still apparent that recycling mutable objects improved performance by reduced pressure on your CPU caches. These objects have to have simple life cycles and have a simple structure, but you could see significant improvements in performance and jitter by using them. Another area where it made sense to use object pools is when loading large amounts of data with many duplicate objects. With a significant reduction in memory usage and a reduction in the number of objects the GC had to manage, you saw a reduction in GC times and an increase in throughput. These object pools were designed to be more light weight than say using a synchronized HashMap, and so they still helped. Take this StringInterner class as an example. You pass it a recycled mutable StringBuilder of the text you want as a String and it will provide a String which matches. Passing a String would be inefficient as you would have already created the object. The StringBuilder can be recycled. Note: this structure has an interesting property that requires no additional thread safety features, like volatile or synchronized, other than is provided by the minimum Java guarantees. i.e. you can see the final fields in a String correctly and only read consistent references. public class StringInterner { private final String[] interner; private final int mask; public StringInterner(int capacity) { int n = Maths.nextPower2(capacity, 128); interner = new String[n]; mask = n - 1; } private static boolean isEqual(@Nullable CharSequence s, @NotNull CharSequence cs) { if (s == null) return false; if (s.length() != cs.length()) return false; for (int i = 0; i < cs.length(); i++) if (s.charAt(i) != cs.charAt(i)) return false; return true; } @NotNull public String intern(@NotNull CharSequence cs) { long hash = 0; for (int i = 0; i < cs.length(); i++) hash = 57 * hash + cs.charAt(i); int h = (int) Maths.hash(hash) & mask; String s = interner[h]; if (isEqual(s, cs)) return s; String s2 = cs.toString(); return interner[h] = s2; } } Off heap memory usage Using off heap memory and using object pools both help reduce GC pauses, this is their only similarity. Object pools are good for short lived mutable objects, expensive to create objects and long live immutable objects where there is a lot of duplication. Medium lived mutable objects, or complex objects are more likely to be better left to the GC to handle. However, medium to long lived mutable objects suffer in a number of ways which off heap memory solves. Off heap memory provides; Scalability to large memory sizes e.g. over 1 TB and larger than main memory. Notional impact on GC pause times. Sharing between processes, reducing duplication between JVMs, and making it easier to split JVMs. Persistence for faster restarts or replying of production data in test. The use of off heap memory gives you more options in terms of how you design your system. The most important improvement is not performance, but determinism. Off heap and testing One of the biggest challenges in high performance computing is reproducing obscure bugs and being able to prove you have fixed them. By storing all your input events and data off heap in a persisted way you can turn your critical systems into a series of complex state machines. (Or in simple cases, just one state machine) In this way you get reproducible behaviour and performance between test and production.A number of investment banks use this technique to replay a system reliably to any event in the day and work out exactly why that event was processed the way it was. More importantly, once you have a fix you can show that you have fixed the issue which occurred in production, instead of finding an issue and hoping this was the issue.Along with deterministic behaviour comes deterministic performance. In test environments, you can replay the events with realistic timings and show the latency distribution you expect to get in production. Some system jitter can't be reproduce esp if the hardware is not the same, but you can get pretty close when you take a statistical view. To avoid taking a day to replay a day of data you can add a threshold. e.g. if the time between events is more than 10 ms you might only wait 10 ms. This can allow you to replay a day of events with realistic timing in under an hour and see whether your changes have improved your latency distribution or not. By going more low level don't you lose some of "compile once, run anywhere"? To some degree this is true, but it is far less than you might think. When you are working closer the processor and so you are more dependant on how the processor, or OS behaves. Fortunately, most systems use AMD/Intel processors and even ARM processors are becoming more compatible in terms of the low level guarantees they provide. There is also differences in the OSes, and these techniques tend to work better on Linux than Windows. However, if you develop on MacOSX or Windows and use Linux for production, you shouldn't have any issues. This is what we do at Higher Frequency Trading. What new problems are we creating by using off heap? Nothing comes for free, and this is the case with off heap. The biggest issue with off heap is your data structures become less natural. You either need a simple data structure which can be mapped directly to off heap, or you have a complex data structure which serializes and deserializes to put it off heap. Obvious using serialization has its own headaches and performance hit. Using serialization thus much slower than on heap objects.In the financial world, most high ticking data structure are flat and simple, full of primitives which maps nicely off heap with little overhead. However, this doesn't apply in all applications and you can get complex nested data structures e.g. graphs, which you can end up having to cache some objects on-heap as well.Another problem is that the JVM limits how much of the system you can use. You don't have to worry about the JVM overloading the system so much. With off heap, some limitations are lifted and you can use data structures much larger than main memory, and you start having to worry about what kind of disk sub-system you have if you do this. For example, you don't want to be paging to a HDD which has 80 IOPS, instead you are likely to want an SSD with 80,000 IOPS (Input/Ouput Operations per Second) or better i.e. 1000x faster. How does OpenHFT help? OpenHFT has a number of libraries to hide the fact you are really using native memory to store your data. These data structures are persisted and can be used with little or no garbage. These are used in applications which run all day without a minor collection Chronicle Queue - Persisted queue of events. Supports concurrent writers across JVMs on the same machine and concurrent readers across machines. Micro-second latencies and sustained throughputs in the millions of messages per second. Chronicle Map - Native or Persisted storage of a key-value Map. Can be shared between JVMs on the same machine, replicated via UDP or TCP and/or accessed remotely via TCP. Micro-second latencies and sustained read/write rates in the millions of operations per second per machine. Thread Affinity - Binding of critical threads to isolated cores or logical cpus to minimise jitter. Can reduce jitter by a factor of 1000. Which API to use? If you need to record every event -> Chronicle Queue If you only need the latest result for a unique key -> Chronicle Map If you care about 20 micro-second jitter -> Thread Affinity Conclusion Off heap memory can have challenges but also come with a lot of benefits. Where you see the biggest gain and compares with other solutions introduced to achieve scalability. Off heap is likely to be simpler and much faster than using partitioned/sharded on heap caches, messaging solutions, or out of process databases. By being faster, you may find that some of the tricks you need to do to give you the performance you need are no longer required. e.g. off heap solutions can support synchronous writes to the OS, instead of having to perform them asynchronously with the risk of data loss.The biggest gain however, can be your startup time, giving you a production system which restarts much faster. e.g. mapping in a 1 TB data set can take 10 milli-seconds, and ease of reproducibility in test by replaying every event in order you get the same behaviour every time. This allows you to produce quality systems you can rely on.

This article represents code samples representing lambda expression and the related ease with which one could print key and value of a Map object using one liner. Please feel free to comment/suggest if I missed to mention one or more important points. Also, sorry for the typos. Code Sample – Printing Map using BiConsumer Functional Interface Following is detail for Map.forEach API in Java 8. Read further on this page. default void forEach(BiConsumer action) Performs the given action for each entry in this map until all entries have been processed or the action throws an exception. Unless otherwise specified by the implementing class, actions are performed in the order of entry set iteration (if an iteration order is specified.) Exceptions thrown by the action are relayed to the caller. Pay attention to following: Traditional way of printing key & value would require one to get an iterator of Map.Entry objects and print key and values Lambda expression way represents defining a BiConsumer implementation by passing two input arguments as key and value of Map and printing their values. public static void main(String[] args) { Map map = new HashMap(); String[][] tempStrArr = {{"Chris","USA"}, {"Raju","India"}, {"Lynda","Canada"} }; // Create a Map using String Array for( int i = 0; i < tempStrArr.length; i++ ) { map.put( tempStrArr[i][0], tempStrArr[i][1] ); } // Traditional way of printing key, value Iterator> iter = map.entrySet().iterator(); if( iter != null ) { while( iter.hasNext() ) { Map.Entry entry = iter.next(); System.out.println( "Key: " + entry.getKey() + "\t" + " Value: " + entry.getValue() ); } } // Using Lambda Expression: All in One line map.forEach( (key, value) -> { System.out.println( "Key: " + key + "\t" + " Value: " + value ); }); }

JSR 354 defines a new Java API for working with Money and Currencies, which is planned to be included in Java 9. In this post we will look at the current state of the reference implementation: JavaMoney. Like my post about the Java 8 Date/Time API this post will be mainly driven by code that shows the new API. But before we start, I want to quote a short section from the specification that pretty much sums up the motivation for this new API: Monetary values are a key feature of many applications, yet the JDK provides little or no support. The existing java.util.Currency class is strictly a structure used for representing current ISO 4217 currencies, but not associated values or custom currencies. The JDK also provides no support for monetary arithmetic or currency conversion, nor for a standard value type to represent a monetary amount. If you use Maven, you can easily try the current state of the reference implementation by adding the following dependency to your project: org.javamoney moneta 0.9 All specification classes and interfaces are located in the javax.money.* package. We will start with the two core interfaces CurrencyUnit and MonetaryAmount. After that, we will look into exchange rates, currency conversion and formatting. CurrencyUnit and MonetaryAmount CurrencyUnit models a currency. CurrencyUnit is very similar to the existing java.util.Currency class, except it allows custom implementations. According to the specification it should be possible that java.util.Currency implements CurrencyUnit. CurrencyUnit instances can be obtained using the MonetaryCurrencies factory: // getting CurrencyUnits by currency code CurrencyUnit euro = MonetaryCurrencies.getCurrency("EUR"); CurrencyUnit usDollar = MonetaryCurrencies.getCurrency("USD"); // getting CurrencyUnits by locale CurrencyUnit yen = MonetaryCurrencies.getCurrency(Locale.JAPAN); CurrencyUnit canadianDollar = MonetaryCurrencies.getCurrency(Locale.CANADA); MontetaryAmount represents a concrete numeric representation of a monetary amount. A MonetaryAmount is always bound to a CurrencyUnit. Like CurrencyUnit, MonetaryAmount is an interface that supports different implementations. CurrencyUnit and MonetaryAmount implementations must be immutable, thread safe, serializable and comparable. // get MonetaryAmount from CurrencyUnit CurrencyUnit euro = MonetaryCurrencies.getCurrency("EUR"); MonetaryAmount fiveEuro = Money.of(5, euro); // get MonetaryAmount from currency code MonetaryAmount tenUsDollar = Money.of(10, "USD"); // FastMoney is an alternative MonetaryAmount factory that focuses on performance MonetaryAmount sevenEuro = FastMoney.of(7, euro); Money and FastMoney are two MonetaryAmount implementations of JavaMoney. Money is the default implementation that stores number values using BigDecimal. FastMoney is an alternative implementation which stores amounts in long fields. According to the documentation operations on FastMoney are 10-15 times faster compared to Money. However, FastMoney is limited by the size and precision of the long type. Please note that Money and FastMoney are implementation specific classes (located in org.javamoney.moneta.* instead of javax.money.*). If you want to avoid implementation specific classes, you have to obtain a MonetaryAmountFactory to create a MonetaryAmount instance: MonetaryAmount specAmount = MonetaryAmounts.getDefaultAmountFactory() .setNumber(123.45) .setCurrency("USD") .create(); Two MontetaryAmount instances are considered equal if the implementation classes, the currency units and the numeric values are equal: MonetaryAmount oneEuro = Money.of(1, MonetaryCurrencies.getCurrency("EUR")); boolean isEqual = oneEuro.equals(Money.of(1, "EUR")); // true boolean isEqualFast = oneEuro.equals(FastMoney.of(1, "EUR")); // false MonetaryAmount has various methods that allow accessing the assigned currency, the numeric amount, its precision and more: MonetaryAmount monetaryAmount = Money.of(123.45, euro); CurrencyUnit currency = monetaryAmount.getCurrency(); NumberValue numberValue = monetaryAmount.getNumber(); int intValue = numberValue.intValue(); // 123 double doubleValue = numberValue.doubleValue(); // 123.45 long fractionDenominator = numberValue.getAmountFractionDenominator(); // 100 long fractionNumerator = numberValue.getAmountFractionNumerator(); // 45 int precision = numberValue.getPrecision(); // 5 // NumberValue extends java.lang.Number. // So we assign numberValue to a variable of type Number Number number = numberValue; Working with MonetaryAmounts Mathematical operations can be performed with MonetaryAmount: MonetaryAmount twelveEuro = fiveEuro.add(sevenEuro); // "EUR 12" MonetaryAmount twoEuro = sevenEuro.subtract(fiveEuro); // "EUR 2" MonetaryAmount sevenPointFiveEuro = fiveEuro.multiply(1.5); // "EUR 7.5" // MonetaryAmount can have a negative NumberValue MonetaryAmount minusTwoEuro = fiveEuro.subtract(sevenEuro); // "EUR -2" // some useful utility methods boolean greaterThan = sevenEuro.isGreaterThan(fiveEuro); // true boolean positive = sevenEuro.isPositive(); // true boolean zero = sevenEuro.isZero(); // false // Note that MonetaryAmounts need to have the same CurrencyUnit to do mathematical operations // this fails with: javax.money.MonetaryException: Currency mismatch: EUR/USD fiveEuro.add(tenUsDollar); Rounding is another important part when working with money. MonetaryAmounts can be rounded using a rounding operator: CurrencyUnit usd = MonetaryCurrencies.getCurrency("USD"); MonetaryAmount dollars = Money.of(12.34567, usd); MonetaryOperator roundingOperator = MonetaryRoundings.getRounding(usd); MonetaryAmount roundedDollars = dollars.with(roundingOperator); // USD 12.35 Here 12.3456 US Dollars are rounded with the default rounding for this currency. When working with collections of MonetaryAmounts, some nice utility methods for filtering, sorting and grouping are available. These methods can be used together with the Java 8 Stream API. Consider the following collection: List amounts = new ArrayList<>(); amounts.add(Money.of(2, "EUR")); amounts.add(Money.of(42, "USD")); amounts.add(Money.of(7, "USD")); amounts.add(Money.of(13.37, "JPY")); amounts.add(Money.of(18, "USD")); We can now filter amounts by CurrencyUnit: CurrencyUnit yen = MonetaryCurrencies.getCurrency("JPY"); CurrencyUnit dollar = MonetaryCurrencies.getCurrency("USD"); // filter by currency, get only dollars // result is [USD 18, USD 7, USD 42] List onlyDollar = amounts.stream() .filter(MonetaryFunctions.isCurrency(dollar)) .collect(Collectors.toList()); // filter by currency, get only dollars and yen // [USD 18, USD 7, JPY 13.37, USD 42] List onlyDollarAndYen = amounts.stream() .filter(MonetaryFunctions.isCurrency(dollar, yen)) .collect(Collectors.toList()); We can also filter out MonetaryAmounts smaller or greater than a specific threshold: MonetaryAmount tenDollar = Money.of(10, dollar); // [USD 42, USD 18] List greaterThanTenDollar = amounts.stream() .filter(MonetaryFunctions.isCurrency(dollar)) .filter(MonetaryFunctions.isGreaterThan(tenDollar)) .collect(Collectors.toList()); Sorting works in a similar way: // Sorting dollar values by number value // [USD 7, USD 18, USD 42] List sortedByAmount = onlyDollar.stream() .sorted(MonetaryFunctions.sortNumber()) .collect(Collectors.toList()); // Sorting by CurrencyUnit // [EUR 2, JPY 13.37, USD 42, USD 7, USD 18] List sortedByCurrencyUnit = amounts.stream() .sorted(MonetaryFunctions.sortCurrencyUnit()) .collect(Collectors.toList()); Grouping functions: // Grouping by CurrencyUnit // {USD=[USD 42, USD 7, USD 18], EUR=[EUR 2], JPY=[JPY 13.37]} Map> groupedByCurrency = amounts.stream() .collect(MonetaryFunctions.groupByCurrencyUnit()); // Grouping by summarizing MonetaryAmounts Map summary = amounts.stream() .collect(MonetaryFunctions.groupBySummarizingMonetary()).get(); // get summary for CurrencyUnit USD MonetarySummaryStatistics dollarSummary = summary.get(dollar); MonetaryAmount average = dollarSummary.getAverage(); // "USD 22.333333333333333333.." MonetaryAmount min = dollarSummary.getMin(); // "USD 7" MonetaryAmount max = dollarSummary.getMax(); // "USD 42" MonetaryAmount sum = dollarSummary.getSum(); // "USD 67" long count = dollarSummary.getCount(); // 3 MonetaryFunctions also provides reduction function that can be used to obtain the max, min and sum of a MonetaryAmount collection: List amounts = new ArrayList<>(); amounts.add(Money.of(10, "EUR")); amounts.add(Money.of(7.5, "EUR")); amounts.add(Money.of(12, "EUR")); Optional max = amounts.stream().reduce(MonetaryFunctions.max()); // "EUR 7.5" Optional min = amounts.stream().reduce(MonetaryFunctions.min()); // "EUR 12" Optional sum = amounts.stream().reduce(MonetaryFunctions.sum()); // "EUR 29.5" Custom MonetaryAmount operations MonetaryAmount provides a nice extension point called MonetaryOperator. MonetaryOperator is a functional interface that takes a MonetaryAmount as input and creates a new MonetaryAmount based on the input. // A monetary operator that returns 10% of the input MonetaryAmount // Implemented using Java 8 Lambdas MonetaryOperator tenPercentOperator = (MonetaryAmount amount) -> { BigDecimal baseAmount = amount.getNumber().numberValue(BigDecimal.class); BigDecimal tenPercent = baseAmount.multiply(new BigDecimal("0.1")); return Money.of(tenPercent, amount.getCurrency()); }; MonetaryAmount dollars = Money.of(12.34567, "USD"); // apply tenPercentOperator to MonetaryAmount MonetaryAmount tenPercentDollars = dollars.with(tenPercentOperator); // USD 1.234567 Some standard API features are implemented as MonetaryOperator. For example, the rounding features we saw above are implemented as MonetaryOperator. Exchange rates Currency exchange rates can be obtained using an ExchangeRateProvider. JavaMoney comes with multiple different ExchangeRateProvider implementations. The two most important implementations are ECBCurrentRateProvider and IMFRateProvider. ECBCurrentRateProvider queries the European Central Bank (ECB) data feed for getting current exchange rates while IMFRateProvider uses International Monetary Fund (IMF) conversion rates. // get the default ExchangeRateProvider (CompoundRateProvider) ExchangeRateProvider exchangeRateProvider = MonetaryConversions.getExchangeRateProvider(); // get the names of the default provider chain // [IDENT, ECB, IMF, ECB-HIST] List defaultProviderChain = MonetaryConversions.getDefaultProviderChain(); // get a specific ExchangeRateProvider (here ECB) ExchangeRateProvider ecbExchangeRateProvider = MonetaryConversions.getExchangeRateProvider("ECB"); If no specific ExchangeRateProvider is requested a CompoundRateProvider will be returned. CompoundRateProvider delegates exchange rate requests to a chain of ExchangeRateProviders and returns the result from the first provider that returns an adequate result. // get the exchange rate from euro to us dollar ExchangeRate rate = exchangeRateProvider.getExchangeRate("EUR", "USD"); NumberValue factor = rate.getFactor(); // 1.2537 (at time writing) CurrencyUnit baseCurrency = rate.getBaseCurrency(); // EUR CurrencyUnit targetCurrency = rate.getCurrency(); // USD Currency conversion Conversion between currencies is be done with CurrencyConversions that can be obtained from ExchangeRateProviders: // get the CurrencyConversion from the default provider chain CurrencyConversion dollarConversion = MonetaryConversions.getConversion("USD"); // get the CurrencyConversion from a specific provider CurrencyConversion ecbDollarConversion = ecbExchangeRateProvider.getCurrencyConversion("USD"); MonetaryAmount tenEuro = Money.of(10, "EUR"); // convert 10 euro to us dollar MonetaryAmount inDollar = tenEuro.with(dollarConversion); "USD 12.537" (at the time writing) Note that CurrencyConversion implements MonetaryOperator. Like other operators it can be applied using MonetaryAmount.with(). Formatting and parsing MonetaryAmounts can be parsed/formatted from/to string using a MonetaryAmountFormat: // formatting by locale specific formats MonetaryAmountFormat germanFormat = MonetaryFormats.getAmountFormat(Locale.GERMANY); MonetaryAmountFormat usFormat = MonetaryFormats.getAmountFormat(Locale.CANADA); MonetaryAmount amount = Money.of(12345.67, "USD"); String usFormatted = usFormat.format(amount); // "USD12,345.67" String germanFormatted = germanFormat.format(amount); // 12.345,67 USD // A MonetaryAmountFormat can also be used to parse MonetaryAmounts from strings MonetaryAmount parsed = germanFormat.parse("12,4 USD"); With AmountFormatQueryBuilder custom formats can be created: // Creating a custom MonetaryAmountFormat MonetaryAmountFormat customFormat = MonetaryFormats.getAmountFormat( AmountFormatQueryBuilder.of(Locale.US) .set(CurrencyStyle.NAME) .set("pattern", "00,00,00,00.00 ¤") .build()); // results in "00,01,23,45.67 US Dollar" String formatted = customFormat.format(amount); Note that the ¤ symbol (\u00A) is used as currency placeholder inside the pattern string. Summary We looked at many parts of the new Money and Currency API. The implementation already looks quite solid (but definitely needs some more documentation). I am looking forward to see this API in Java 9 :-) You can find all the examples shown here on GitHub.

MySQL offers the ability to use regular expressions to perform complex searches against your data. A regular expression is a tool that provides for a concise and flexible way to identify strings of text based on user-defined patterns. This article will discuss the MySQL regular expression operators, review their use and syntax, and identify the constructs and special characters that can be used in a MySQL regular expression, as well as provide a few examples of their use. MySQL Regular Expression Operators The following operators are used in MySQL to perform regular expression operations. These are used in a WHERE clause similar to the well-known and often used LIKE operator. REGEXP: The pattern matching operator for using regular expressions. NOT REGEXP: The negation of the REGEXP operator. RLIKE: A synonym for the REGEXP operator. MySQL Regular Expression Syntax The basic syntax used for MySQL regular expression operations is: -- For the REGEXP Operator SELECT {COLUMN_NAME} FROM {TABLE_NAME} WHERE {COLUMN_NAME} REGEXP '{REGEXP_PATTERN}'; -- For the NOT REGEXP Operator SELECT {COLUMN_NAME} FROM {TABLE_NAME} WHERE {COLUMN_NAME} NOT REGEXP '{REGEXP_PATTERN}'; -- For the RLIKE Alias Operator SELECT {COLUMN_NAME} FROM {TABLE_NAME} WHERE {COLUMN_NAME} RLIKE '{REGEXP_PATTERN}'; To provide more detailed, yet simple, example of a MySQL regular expression operation, take the following statement. It will retrieve all the columns of each record in the table PRICE where the PRICELIST_ID matches the pattern specified (starts with the numeric range 0-9 occurring one or more times, followed by an ‘_’ (underscore), and then the character sequence ‘USD’. SELECT * FROM PRICE WHERE PRICELIST_ID REGEXP '^[0-9]+_USD'; Another example of a MySQL regular expression operation, can be shown in the following statement. It will retrieve all columns of each record from the PRICE table where the PRICE_ID matches the pattern specified (starts with an O, followed by and ‘_’ (underscore), then the numeric range 0-9 occurring one or more times, followed by and ‘_’ (underscore), then ending with either the character sequence USD, or BRA. SELECT * FROM PRICE WHERE PRICE_ID REGEXP '^O_[0-9]+_[USD|BRA]'; MySQL REGEXP Constructs and Special Characters A MySQL regular expression may use any of the following constructs and special characters to construct a pattern for use with the REGEXP operators. The construct or special character is shown, followed by a description of each and what operations in performs within the pattern for the regular expression. ^ : Match the beginning of a string. $ : Match the end of a string. . : Match any character (including carriage return and newline characters). a* : Match any sequence of zero or more a characters. a+ : Match any sequence of one or more a characters. a? : Match either zero or one a characters. de|abc : Match either of the character sequences, de or abc. (abc)* : Match zero or more instances of the character sequence abc. {1},{2,3} : Provides a more general way of writing regular expressions that match many occurences of the previous atom (or “piece”) of the pattern. i.e. a? can be written as a{0,1}. [a-dX],[^a-dX] : Matches any character that is (or is not, if ^ is used) either a, b, c, d, or X. A “-” character between two other characters forms a range that maches all characters from the first character to the second. [.characters.] : Within a bracket expression (using “[” and “]”), matches the sequence of characters of that collating element. i.e. the pattern [[.period.]] would match the ‘.’ (period) character. [=character_class=] : Within a bracket expression, represents an equivalence class. It matches all characters with the same collation value, including itself. [:character_class:] : Within a bracket expression, represents a character class that matches all characters belonging to that class. i.e. the pattern [[:alpha:]] would match against a string that is all aphabetic characters. [[:<:]],[[:>:]] : These markers stand for word boundaries, and as such they match the beginning and ending of words, respectively. * NOTE: MySQL interprets the “\” (backslash) character as an escape character. If you choose to use the “\” character as part of your pattern in a regular expression it will need to escaped with another backslash “\\”. For further documentation on the MySQL regular expression operator, please visit Regular Expressions in the MySQL Reference Manual (v5.1 currently linked).

Learn all about creating a microservices architecture on Java in this great tutorial.

this article represents the high level concept and code samples which could be used to create a binary search tree in java. please feel free to comment/suggest if i missed to mention one or more important points. also, sorry for the typos. following are the key points described later in this article: what is a binary search tree? what are different kind of traversals? code samples what is a binary search tree? a binary search tree is a binary tree in which every node contains a key that satisfies following criteria: the key in left child is less than the key in the parent node the key in the right child is more than the parent node the left and right child are again binary search trees. following diagram represents a binary search tree: what are different kind of traversals? following are three different kind of traversals: preorder traversal : in preorder traversal, the node is visted first and then, left and right sub-trees. inorder traversal : in inorder traversal, the node is visited between left and right sub-tree. postorder traversal : in postorder traversal, the node is visited after left and right subtrees. code sample – how to create a binary search tree if the numbers such as {20, 15, 200, 25, -5, 0, 100, 20, 12, 126, 1000, -150} are to be stored in a binarytree (represented by code below), following would get printed using different kind of traversal mechanism: //preorder traversal 20, 15, -5, -150, 0, 12, 200, 25, 20, 100, 126, 1000 // inorder traversal -150, -5, 0, 12, 15, 20, 20, 25, 100, 126, 200, 1000 //postorder traversal -150, 12, 0, -5, 15, 20, 126, 100, 25, 1000, 200, 20 following is the code for creating binary tree that uses following binarytree class and traversals: binarytree tree = new binarytree( 20 ); int[] nums = {15, 200, 25, -5, 0, 100, 20, 12, 126, 1000, -150}; for(int i : nums ) { tree.addnode( i ); } tree.traversepreorder(); tree.traverseinorder(); tree.traversepostorder(); following is the code for binarytree class: public class binarytree { private int data; private binarytree left; private binarytree right; public binarytree(int num) { this.data = num; this.left = null; this.right = null; } // as a convention, if the key to be inserted is less than the key of root node, then key is inserted in // left sub-tree; if key is greater, it is inserted in right sub-tree. if it is equal, as a convention, it // is inserted in right sub-tree public void addnode(int num) { if (num < this.data) { if (this.left != null) { this.left.addnode(num); } else { this.left = new binarytree(num); } } else { if (this.right != null) { this.right.addnode(num); } else { this.right = new binarytree(num); } } } // visit the node first, then left and right sub-trees public void traversepreorder() { system.out.println( this.data ); if( this.left != null ) { this.left.traversepreorder(); } if( this.right != null ) { this.right.traversepreorder(); } } // visit left sub-tree, then node and then, right sub-tree public void traverseinorder() { if( this.left != null ) { this.left.traverseinorder(); } system.out.println( this.data ); if( this.right != null ) { this.right.traverseinorder(); } } // visit left sub-tree, then right sub-tree and then the node public void traversepostorder() { if( this.left != null ) { this.left.traversepostorder(); } if( this.right != null ) { this.right.traversepostorder(); } system.out.println( this.data ); } }

If you ever have a need to process messages serially with RabbitMQ with a cluster of listeners processing the messages, the best way that I have seen is to use a "exclusive consumer" flag on a listener with 1 thread on each listener processing the messages. Exclusive consumer flag ensures that only 1 consumer can read messages from the specific queue, and 1 thread on that consumer ensures that the messages are processed serially. There is a catch however, I will go over it later. Let me demonstrate this behavior with a Spring Boot and Spring Integration based RabbitMQ message consumer. First, this is the configuration for setting up a queue using Spring java configuration, note that since this is a Spring Boot application, it automatically creates a RabbitMQ connection factory when the Spring-amqp library is added to the list of dependencies: @Configuration @Configuration public class RabbitConfig { @Autowired private ConnectionFactory rabbitConnectionFactory; @Bean public Queue sampleQueue() { return new Queue("sample.queue", true, false, false); } } Given this sample queue, a listener which gets the messages from this queue and processes them looks like this, the flow is written using the excellent Spring integration Java DSL library: @Configuration public class RabbitInboundFlow { private static final Logger logger = LoggerFactory.getLogger(RabbitInboundFlow.class); @Autowired private RabbitConfig rabbitConfig; @Autowired private ConnectionFactory connectionFactory; @Bean public SimpleMessageListenerContainer simpleMessageListenerContainer() { SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer(); listenerContainer.setConnectionFactory(this.connectionFactory); listenerContainer.setQueues(this.rabbitConfig.sampleQueue()); listenerContainer.setConcurrentConsumers(1); listenerContainer.setExclusive(true); return listenerContainer; } @Bean public IntegrationFlow inboundFlow() { return IntegrationFlows.from(Amqp.inboundAdapter(simpleMessageListenerContainer())) .transform(Transformers.objectToString()) .handle((m) -> { logger.info("Processed {}", m.getPayload()); }) .get(); } } The flow is very concisely expressed in the inboundFlow method, a message payload from RabbitMQ is transformed from byte array to String and finally processed by simply logging the message to the logs The important part of the flow is the listener configuration, note the flag which sets the consumer to be an exclusive consumer and within this consumer the number of threads processing is set to 1. Given this even if multiple instances of the application is started up only 1 of the listeners will be able to connect and process messages. Now for the catch, consider a case where the processing of messages takes a while to complete and rolls back during processing of the message. If the instance of the application handling the message were to be stopped in the middle of processing such a message, then the behavior is a different instance will start handling the messages in the queue, when the stopped instance rolls back the message, the rolled back message is then delivered to the new exclusive consumer, thus getting a message out of order. If you are interested in exploring this further, here is a github project to play with this feature: https://github.com/bijukunjummen/test-rabbit-exclusive

Java Mission Control (JMC) enables you to monitor and manage Java applications without introducing the performance overhead normally associated with these types of tools. It uses data which is already getting collected for normal dynamic optimization of the JVM resulting in a very lightweight approach to observe and analyze problems in the application code. The JMC consists of three different types of tools. A JMX browser which let's you browse all available JVM instances on a machine and a JMX console which let's you browse through the JMX tree on a connected JVM. Last but not least the most interesting aspect is the Java Flight Recorder (JFR). This is exactly the part of the tooling which does the low overhead profiling of JVM instances. Disclaimer: A Word On Licensing The tooling is part of the Oracle JDK downloads. In particular the JMC 5.4 is part of JDK 8u20 and JDK 7u71 and is distributed under the Oracle Binary Code License Agreement for Java SE Platform products and commercially available features for Java SE Advanced and Java SE Suite. IANAL, but as far as I know this allows for using it for your personal education and potentially also as part of your developer tests. Make sure to check back with whomever you know that could answer this question. This blog post looks at it as a small little how-to and assumes, that you know what you are doing from a license perspective. Adding Java Optional Parameters Unlocking the JFR features requires you to put in some optional parameters to your WildFly 8.x/EAP 6.x configuration. Find the $JBOSS_HOME/bin/standalone.conf|conf.bat and add the following parameters: -XX:+UnlockCommercialFeatures -XX:+FlightRecorder You can now use jcmd command like described in this knowledge-base entry to start a recording. Another way is actually to start a recording directly from JMC. Starting A Recording From JMC First step is to start JMC. Find it in the %JAVA_HOME%/bin folder. After it started you can use the JVM Browser to find the WildFly/EAP instance you want to connect to. Right click on it to see all the available options. You can either start the JMX Console or start a Flight Recording. The JMX console is a bit fancier than the JConsole and allows for a bunch of metrics and statistics. It also allows you to set a bunch of triggers and browser MBeans and whatnot. Please look at the documentation for all the details. What is really interesting is the function to start a Flight Recording. If you select this option, a new wizard pops up and lets you tweak the settings a bit. Beside having to select a folder where the recording gets stored you also have the choice between different recording templates. A one minute recording with the "Server Profiling" template with barely any load on the server results in a 1.5 MB file. So, better keep an eye on the volume you're storing all that stuff at. You can also decide the profiling granularity for a bunch of parameters further down the dialogues. But at the end, you click "Finish" and the recording session starts. You can decide to push it to the background and keep working while the data gets captured. Analyzing Flight Recorder Files This is pretty easy. You can open the recording with JMC and click through the results. If you enabled the default recording with the additional parameter: -XX:FlightRecorderOptions=defaultrecording=true you can also directly dump the recording via the JVM browser. It is easy to pick a time-frame that you want to download the data for or alternatively you can also decide to download the complete recording.

{editor's note: thanks to yegor bugayenko, a new mvb at dzone. among other things, yegor blogs about java and devops. we're pleased to have him on board as a most valuable blogger. check out his blog, yegor256.com .} tl;dr orm is a terrible anti-pattern that violates all principles of object-oriented programming, tearing objects apart and turning them into dumb and passive data bags. there is no excuse for orm existence in any application, be it a small web app or an enterprise-size system with thousands of tables and crud manipulations on them. what is the alternative? sql-speaking objects . vinni-pukh (1969) by fyodor khitruk how orm works object-relational mapping (orm) is a technique (a.k.a. design pattern) of accessing a relational database from an object-oriented language (java, for example). there are multiple implementations of orm in almost every language; for example: hibernate for java, activerecord for ruby on rails, doctrine for php, and sqlalchemy for python. in java, the orm design is even standardized as jpa . first, let's see how orm works, by example. let's use java, postgresql, and hibernate. let's say we have a single table in the database, called post : +-----+------------+--------------------------+ | id | date | title | +-----+------------+--------------------------+ | 9 | 10/24/2014 | how to cook a sandwich | | 13 | 11/03/2014 | my favorite movies | | 27 | 11/17/2014 | how much i love my job | +-----+------------+--------------------------+ now we want to crud-manipulate this table from our java app (crud stands for create, read, update, and delete). first, we should create a post class (i'm sorry it's so long, but that's the best i can do): @entity @table(name = "post") public class post { private int id; private date date; private string title; @id @generatedvalue public int getid() { return this.id; } @temporal(temporaltype.timestamp) public date getdate() { return this.date; } public title gettitle() { return this.title; } public void setdate(date when) { this.date = when; } public void settitle(string txt) { this.title = txt; } } before any operation with hibernate, we have to create a session factory: sessionfactory factory = new annotationconfiguration() .configure() .addannotatedclass(post.class) .buildsessionfactory(); this factory will give us "sessions" every time we want to manipulate with post objects. every manipulation with the session should be wrapped in this code block: session session = factory.opensession(); try { transaction txn = session.begintransaction(); // your manipulations with the orm, see below txn.commit(); } catch (hibernateexception ex) { txn.rollback(); } finally { session.close(); } when the session is ready, here is how we get a list of all posts from that database table: list posts = session.createquery("from post").list(); for (post post : (list) posts){ system.out.println("title: " + post.gettitle()); } i think it's clear what's going on here. hibernate is a big, powerful engine that makes a connection to the database, executes necessary sql select requests, and retrieves the data. then it makes instances of class post and stuffs them with the data. when the object comes to us, it is filled with data, and we should use getters to take them out, like we're using gettitle() above. when we want to do a reverse operation and send an object to the database, we do all of the same but in reverse order. we make an instance of class post , stuff it with the data, and ask hibernate to save it: post post = new post(); post.setdate(new date()); post.settitle("how to cook an omelette"); session.save(post); this is how almost every orm works. the basic principle is always the same — orm objects are anemic envelopes with data. we are talking with the orm framework, and the framework is talking to the database. objects only help us send our requests to the orm framework and understand its response. besides getters and setters, objects have no other methods. they don't even know which database they came from. this is how object-relational mapping works. what's wrong with it, you may ask? everything! what's wrong with orm? seriously, what is wrong? hibernate has been one of the most popular java libraries for more than 10 years already. almost every sql-intensive application in the world is using it. each java tutorial would mention hibernate (or maybe some other orm like toplink or openjpa) for a database-connected application. it's a standard de-facto and still i'm saying that it's wrong? yes. i'm claiming that the entire idea behind orm is wrong. its invention was maybe the second big mistake in oop after null reference . actually, i'm not the only one saying something like this, and definitely not the first. a lot about this subject has already been published by very respected authors, including ormhate by martin fowler, object-relational mapping is the vietnam of computer science by jeff atwood, the vietnam of computer science by ted neward, orm is an anti-pattern by laurie voss, and many others. however, my argument is different than what they're saying. even though their reasons are practical and valid, like "orm is slow" or "database upgrades are hard", they miss the main point. you can see a very good, practical answer to these practical arguments given by bozhidar bozhanov in his orm haters don’t get it blog post. the main point is that orm, instead of encapsulating database interaction inside an object, extracts it away, literally tearing a solid and cohesive living organism apart. one part of the object keeps the data while another one, implemented inside the orm engine (session factory), knows how to deal with this data and transfers it to the relational database. look at this picture; it illustrates what orm is doing. i, being a reader of posts, have to deal with two components: 1) the orm and 2) the "obtruncated" object returned to me. the behavior i'm interacting with is supposed to be provided through a single entry point, which is an object in oop. in the case of orm, i'm getting this behavior via two entry points — the orm and the "thing", which we can't even call an object. because of this terrible and offensive violation of the object-oriented paradigm, we have a lot of practical issues already mentioned in respected publications. i can only add a few more. sql is not hidden . users of orm should speak sql (or its dialect, like hql ). see the example above; we're calling session.createquery("from post") in order to get all posts. even though it's not sql, it is very similar to it. thus, the relational model is not encapsulated inside objects. instead, it is exposed to the entire application. everybody, with each object, inevitably has to deal with a relational model in order to get or save something. thus, orm doesn't hide and wrap the sql but pollutes the entire application with it. difficult to test . when some object is working a list of posts, it needs to deal with an instance of sessionfactory . how can we mock this dependency? we have to create a mock of it? how complex is this task? look at the code above, and you will realize how verbose and cumbersome that unit test will be. instead, we can write integration tests and connect the entire application to a test version of postgresql. in that case, there is no need to mock sessionfactory , but such tests will be rather slow, and even more important, our having-nothing-to-do-with-the-database objects will be tested against the database instance. a terrible design. again, let me reiterate. practical problems of orm are just consequences. the fundamental drawback is that orm tears objects apart, terribly and offensively violating the very idea of what an object is . sql-speaking objects what is the alternative? let me show it to you by example. let's try to design that class, post , my way. we'll have to break it down into two classes: post and posts , singular and plural. i already mentioned in one of my previous articles that a good object is always an abstraction of a real-life entity. here is how this principle works in practice. we have two entities: database table and table row. that's why we'll make two classes; posts will represent the table, and post will represent the row. as i also mentioned in that article , every object should work by contract and implement an interface. let's start our design with two interfaces. of course, our objects will be immutable. here is how posts would look: @immutable interface posts { iterable iterate(); post add(date date, string title); } this is how a single post would look: @immutable interface post { int id(); date date(); string title(); } here is how we will list all posts in the database table: posts posts = // we'll discuss this right now for (post post : posts.iterate()){ system.out.println("title: " + post.title()); } here is how we will create a new post: posts posts = // we'll discuss this right now posts.add(new date(), "how to cook an omelette"); as you see, we have true objects now. they are in charge of all operations, and they perfectly hide their implementation details. there are no transactions, sessions, or factories. we don't even know whether these objects are actually talking to the postgresql or if they keep all the data in text files. all we need from posts is an ability to list all posts for us and to create a new one. implementation details are perfectly hidden inside. now let's see how we can implement these two classes. i'm going to use jcabi-jdbc as a jdbc wrapper, but you can use something else or just plain jdbc if you like. it doesn't really matter. what matters is that your database interactions are hidden inside objects. let's start with posts and implement it in class pgposts ("pg" stands for postgresql): @immutable final class pgposts implements posts { private final source dbase; public pgposts(datasource data) { this.dbase = data; } public iterable iterate() { return new jdbcsession(this.dbase) .sql("select id from post") .select( new listoutcome( new listoutcome.mapping() { @override public post map(final resultset rset) { return new pgpost(rset.getinteger(1)); } } ) ); } public post add(date date, string title) { return new pgpost( this.dbase, new jdbcsession(this.dbase) .sql("insert into post (date, title) values (?, ?)") .set(new utc(date)) .set(title) .insert(new singleoutcome(integer.class)) ); } } next, let's implement the post interface in class pgpost : @immutable final class pgpost implements post { private final source dbase; private final int number; public pgpost(datasource data, int id) { this.dbase = data; this.number = id; } public int id() { return this.number; } public date date() { return new jdbcsession(this.dbase) .sql("select date from post where id = ?") .set(this.number) .select(new singleoutcome(utc.class)); } public string title() { return new jdbcsession(this.dbase) .sql("select title from post where id = ?") .set(this.number) .select(new singleoutcome(string.class)); } } this is how a full database interaction scenario would look like using the classes we just created: posts posts = new pgposts(dbase); for (post post : posts.iterate()){ system.out.println("title: " + post.title()); } post post = posts.add(new date(), "how to cook an omelette"); system.out.println("just added post #" + post.id()); you can see a full practical example here . it's an open source web app that works with postgresql using the exact approach explained above — sql-speaking objects. what about performance? i can hear you screaming, "what about performance?" in that script a few lines above, we're making many redundant round trips to the database. first, we retrieve post ids with select id and then, in order to get their titles, we make an extra select title call for each post. this is inefficient, or simply put, too slow. no worries; this is object-oriented programming, which means it is flexible! let's create a decorator of pgpost that will accept all data in its constructor and cache it internally, forever: @immutable final class constpost implements post { private final post origin; private final date dte; private final string ttl; public constpost(post post, date date, string title) { this.origin = post; this.dte = date; this.ttl = title; } public int id() { return this.origin.id(); } public date date() { return this.dte; } public string title() { return this.ttl; } } pay attention: this decorator doesn't know anything about postgresql or jdbc. it just decorates an object of type post and pre-caches the date and title. as usual, this decorator is also immutable. now let's create another implementation of posts that will return the "constant" objects: @immutable final class constpgposts implements posts { // ... public iterable iterate() { return new jdbcsession(this.dbase) .sql("select * from post") .select( new listoutcome( new listoutcome.mapping() { @override public post map(final resultset rset) { return new constpost( new pgpost(rset.getinteger(1)), utc.gettimestamp(rset, 2), rset.getstring(3) ); } } ) ); } } now all posts returned by iterate() of this new class are pre-equipped with dates and titles fetched in one round trip to the database. using decorators and multiple implementations of the same interface, you can compose any functionality you wish. what is the most important is that while functionality is being extended, the complexity of the design is not escalating, because classes don't grow in size. instead, we're introducing new classes that stay cohesive and solid, because they are small. what about transactions? every object should deal with its own transactions and encapsulate them the same way as select or insert queries. this will lead to nested transactions, which is perfectly fine provided the database server supports them. if there is no such support, create a session-wide transaction object that will accept a "callable" class. for example: final class txn { private final datasource dbase; public t call(callable callable) { jdbcsession session = new jdbcsession(this.dbase); try { session.sql("start transaction").exec(); t result = callable.call(); session.sql("commit").exec(); return result; } catch (exception ex) { session.sql("rollback").exec(); throw ex; } } } then, when you want to wrap a few object manipulations in one transaction, do it like this: new txn(dbase).call( new callable() { @override public integer call() { posts posts = new pgposts(dbase); post post = posts.add(new date(), "how to cook an omelette"); posts.comments().post("this is my first comment!"); return post.id(); } } ); this code will create a new post and post a comment to it. if one of the calls fail, the entire transaction will be rolled back. this approach looks object-oriented to me. i'm calling it "sql-speaking objects", because they know how to speak sql with the database server. it's their skill, perfectly encapsulated inside their borders. related posts you may also find these posts interesting: how much your objects encapsulate? how an immutable object can have state and behavior? seven virtues of a good object how immutability helps paired brackets

What's New in this Release? The long awaited version of Aspose.Slides for Java (14.9.0) has been released. Aspose team has introduced support for replacing an image in a presentation image collection with a different one. As a result of this, the image is replaced in all instances in the presentation that refers to it. This release also introduced support for setting the fill format for SmartArt nodes and devlopers can now set the fill color or pattern for SmartArt nodes. More details about this feature are available in the article Setting Fill Format for SmartArt Node in the documentation section. Aspose team has introduced support for generating a PDF for a specific number of slides. Users can find more details by visiting and reading the article Exporting Presentation to PDF in the documentation section. Support for generating HTML files for individual presentation slides has also been included in this new release. Now, Aspose.Slides for Java makes it possible to get warning callbacks for font substitution in case the used font is not available on the machine during the rendering process. Warning callbacks are helpful when debugging issues of missing or inaccessible fonts during rendering. Developers can find more details about this feature by visiting the article Getting Warning Callbacks for Fonts Substitution in Aspose.Slides. Aspose team has rectified exception issues that appeared when accessing, saving and rendering presentation to PDF, HTML or slide thumbnails, which resulted in different exceptions like KeyNotFoundException, UnKnownFileFormatException, NullReference, ArgumentException, and IndexOutOfRange in previous releases. It has taken a leap towards improving the presentation rendering support for exported PDF, SVG, HTML and slide thumbnails in this release. Several issues pertaining to improper text, wrong shape, improper charts, unfitting SmartArt and wrong font rendering have been addressed in this regard. Some important enhancement & bug fixes included in this release are given below Implementation of IWarningCallback in font substitution scenario Convert a specific PPTX slide to PDF file Get warnings for Fonts substitution in Aspose.Slides Attaching an XLS file in a PPT file Detecting symbols with position in text strings Missing feature to replace image in presentation file in the new unified version Slide per file when coverting PPTX to HTML Setting Fill format for SmartArt node Support for setting the background color of individual nodes in SmartArt shape Implement animation timeline serialization to PPT Thread blocking in Aspose.Slides for Java PPTX to PDF takes immense memory resources An element with the same key already exists in the dictionary Exception is thrown while opening the PPTX file Double Underline in a table cell text is not working properly Converting PPTX to HTML and saving images as other formats rather than SVG Page number position is not proper in the generated PDF file Aspose.Slides is unable to complete the PPTX to PDF conversion process PPTX to HTML Conversion issue: Logo on the bottom is coming with black background PPTX to HTML Conversion issue: Text background color is not proper HTML to PDF Conversion issue: graph background color is not proper Bullets are coming as junk characters in generated PDF file Aspose.Slides escaping issue in PPTX file Unknown file format exception is thrown on opening the file. Font changes after conversion from PPT to PNG Pie Chart Series Labels are outside the chart in generated PNG file Chart Title position in generated PPTX are not same as in Original PPTX KeyNotFoundException on exporting to PDF Details required for Presentations to HTML with Externally Linked Images InvertIfNegative values are rendered with colors in generated PDF Setting fill picture for Media Player control does not work Concurrent processing of slides takes more time then sequential threading Aspose.Slides failed to work in Scala Framework The line links are missing for rendered smart art in generated PDF Hyperlinks Addresses Changing on opening and saving presentations using Aspose.Slides Bullet position and shadow effects lost for text in exported PDF Vertical text is rendered horizontally in generated thumbnail Unable to remove shape border PPT to PDF conversion issue Missing shapes in generated thumbnails when used in JDK 1.4 font Typeface ignored when Bold property is used Text position is lost in generated thumbnail Table height sets successfully only if it is first read. Newly added documentation pages and articles Some new tips and articles have now been added into Aspose.Slides for Java documentation that may guide youl briefly how to use Aspose.Slides for performing different tasks like the followings. Setting Fill Format for SmartArt Node Converting Presentation to HTML Overview: Aspose.Slides for Java Aspose.Slides is a Java component to create, read, write and modify a PowerPoint document without using Microsoft PowerPoint. It supports PHP applications and provides all advanced features for managing presentations, slides, shapes, tables and supports PPT, POT, POS PowerPoint formats. Now you can add, access, copy, clone, edit and delete slides in your presentations. It also supports audio & video frames, adding pictures, text frames and saving presentations as streams or SVG format. Homepage of Aspose.Slides for Java Downlaod Aspose.Slides for Java

if you don't have enough connections open to your mysql server, your users will begin to receive a "too many connections" error while trying to use your service. to fix this, you can increase the maximum number of connections to the database that are allowed, but there are some things to take into consideration before simply ramping up this number. items to consider before you increase the connections limit, you will want to ensure that the machine on which the database is housed can handle the additional workload. the maximum number of connections that can be supported depends on the following variables: the available ram – the system will need to have enough ram to handle the additional workload. the thread library quality of the platform - this will vary based on the platform. for example, windows can be limited by the posix compatibility layer it uses (though the limit no longer applies to mysql v5.5 and up). however, there remains memoray usage concerns depending on the architecture (x86 vs. x64) and how much memory can be consumed per application process. the required response time - increasing the number could increase the amount of time to respond to request. this should be tested to ensure it meets your needs before going into production. the amount of ram used per connection - again, ram is important, so you will need to know if the ram used per connection will overload the system or not. the workload required for each connection - the workload will also factor in to what system resources are needed to handle the additional connections. another issue to consider is that you may also need to increase the open files limit–this may be necessary so that enough handles are available. checking the connection limit to see what the current connection limit is, you can run the following from the mysql command line or from many of the available mysql tools such as phpmyadmin : the show variables command. this will display a nicely formatted result for you: example result of the show variables command. increasing the connection limit to increase the global number of connections temporarily, you can run the following from the command line: an example of setting the max_connections global. if you want to make the increase permanent, you will need to edit the my.cnf configuration file. you will need to determine the location of this file for your operating system (linux systems often store the file in the /etc folder, for example). open this file add a line that includes max_connections, followed by an equal sign, followed by the number you want to use, as in the following example: example of setting the max_connections the next time you restart mysql, the new setting will take effect and will remain in place unless or until this is changed again. easily scale a mysql database instead of worrying about these settings on your own system, you could opt to use a service like morpheus , which offers databases as a service on the cloud. with morpheus, you can easily and quickly set up your choice of several databases (including mysql, mongodb, redis, and elasticsearch). in addition, mysql and redis have automatic back ups, and each database instance is replicated, archived, and deployed on a high performance infrastructure with solid state drives.

this mobile application tutorial shows you how to create a user registration, login and logout backend using mongodb and mongoose. this article is part of a series of mobile application development tutorials that i have been publishing on my blog jorgeramon.me, which shows you how to create a meeting room booking mobile application. this app will be used to browse an inventory of meeting rooms and reserve rooms for conference calls and other types of events. the backend that we will create in this article will connect with the user account management screens that we built in a previous chapter of this series . this backend will consist of the following modules: router (using node.js and express ) controller data model to represent a user (using mongoose ) database (using mongodb ) the router receives http requests from the mobile application and forwards them to the controller, which in turn creates, reads, updates and deletes data models defined with the mongoose library . the user profiles and sessions information will reside in a mongodb database . the router also receives data from the controller and bundles it in http responses that it sends to the mobile app. in this article we will create the controller, model and database modules using mongoose and mongodb. we will test the controller and build the router in the next article of this series. let’s proceed to install the software that we will use to create the node js, mongodb and mongoose endpoint. installing node.js node.js is a platform for building network apps that you can use to build backend endpoints for your mobile applications. you can get node.js at node.js . this tutorial doesn’t require you to have extensive knowledge of node, but you should try to learn about it as much as you can in order to take full advantage of its capabilities. to start, i would recommend the tutorials over at node school . installing express express is a framework for building web applications with node.js. express’ installation page shows you how to install the framework. in this particular article we will only use the request routing capabilities of express. in the feature we will take advantage of other features. installing mongodb mongodb is a leading nosql database at the time of this writing. in the databases ecosystem, mongodb falls under the document databases category. these are databases where each record and its associated data is thought of as a “document”. document databases have characteristics that make them a good choice for storing unstructured data across multiple servers. there is abundant online documentation on this subject. if you want to learn more, you can start with the document databases page on mongodb’s website. to install mongodb you need to head to the downloads page on mongodb.org and grab the mongodb installer for your platform. if you haven’t worked with mongo, i recommend that at a minimum you go over mongodb’s interactive tutorial so you become familiar with it. installing mongoose mongoose is a javascript library that makes it easy to move data between your application and mongodb databases. it is a layer of abstraction that allows you to create schemas for the data that your application uses, and provides facilities for connecting to mongodb and validating, saving, updating, deleting and retrieving instances of these schemas. the picture below will give you an idea of where mongoose fits in our application’s architecture: you can find installation instructions and a very good introduction to the library on mongoose’s getting started page. designing the public interface of the controller the role of the controller module in our express backend will be to fulfill requests received from the mobile application: as at this point in this series of tutorials we are only concerned with the user registration, login and logout features of the application, we will create controller methods to handle these functions. we need the controller module to respond to the following requests: register user log on a user log off a user initiate a password reset for a user finalize a password reset for a user based on these requests, we will design a controller with the following public methods. controller.register(newuser, callback): this method will register a new user with the backend by saving the user’s profile in the mongodb database. parameters: newuser – the user to register in the database callback – a function that will receive the results of the registration attempt. returns: callback – the callback function passed in the arguments. controller.logon(email, password, callback): this method will logon a user if the supplied email and password are valid. if the logon attempt succeeds, the method will add the user’s profile to a private “session” variable in the controller. parameters: email – the user’s email. password – the user’s password. callback – a function that will receive the results of the logon attempt. returns: callback – the callback function passed in the arguments. controller.logoff(): this method will log off a user by delete the user’s profile data stored in the controller’s private “session” variable. controller.resetpassword(email, callback): this method will send the user an email containing a password reset link. the link will contain a unique identifier string that will be used in the controller.resetpasswordfinal method. parameters: email – the user’s email address. callback – a function that will receive the results of the reset password attempt. returns: callback – the callback function passed in the arguments. controller.resetpasswordfinal(email, newpassword, passwordresethash, callback): this method will reset a user’s password. parameters: email – the user’s email address. newpassword – the user’s new password. passwordresethash – a unique identifier sent to the user via email from the controller.resetpassword method. callback – a function that will receive the results of the reset password attempt. returns: callback – the callback function passed in the arguments. controller.setsession(session): this method will set the controller’s private “session” variable. parameters: session – the value for the controller’s “session” variable. controller.getsession(): this method will return a reference to the controller’s private session variable. returns: session – the internal “session” variable. creating a model using mongoose as explained in the mongoose documentation , the mongoose model automatically inherits a number of methods (such as create, save, remove and find) that allow us to store and retrieve model instances from a mongodb database. we will use mongoose’s help to create a model of a user. let’s create the user.js file in the model directory. in the file, we will define the following mongoose schema: var mongoose = require('mongoose'); var schema = mongoose.schema; var userschema = new schema({ email: string, firstname: string, lastname: string, passwordhash: string, passwordsalt: string }); module.exports = mongoose.model('user', userschema); the model’s properties are the user’s attributes we want to capture (email, first name and last name), as well as a hash of the user’s password and the salt value that we used to create the password’s hash. as you will see later, storing a password’s hash and salt will allow us to authenticate users without needing to store their passwords in our database. the apiresponse class i mentioned a class called apiresponse in the majority of the methods that make the controller’s public interface. this is a data transfer class that will help us move data out of the controller. let’s create the api-response.js file in the models directory. in the file, let’s type the following definition: var apiresponse = function (cnf) { this.success = cnf.success; this.extras = cnf.extras; }; module.exports = apiresponse; any request sent to the controller will eventually produce an apiresponse instance. as its name indicates, the success property of apiresponse will signal whether the request succeeded or not. the extras property will be a javascript object containing any additional data that the controller wants to send out as part of the response. the apimessages class when the success property of the apiresponse instance is false, the data sent in the extras property can include information about what caused the failure. we will define these causes in a class that we will call apimessages. let’s create the api-messages.js file in the models directory. we will define the apimessages class as follows: var apimessages = function () { }; apimessages.prototype.email_not_found = 0; apimessages.prototype.invalid_pwd = 1; apimessages.prototype.db_error = 2; apimessages.prototype.not_found = 3; apimessages.prototype.email_already_exists = 4; apimessages.prototype.could_not_create_user = 5; apimessages.prototype.password_reset_expired = 6; apimessages.prototype.password_reset_hash_mismatch = 7; apimessages.prototype.password_reset_email_mismatch = 8; apimessages.prototype.could_not_reset_password = 9; module.exports = apimessages; as the code indicates, we are defining the reasons that can cause a controller request to fail. they are basically the different error conditions that we anticipate can occur inside the controller. creating the userprofilemodel class the data sent in the extras property of an apiresponse instance can also include a read-only version of the user’s profile. we will create the userprofilemodel class to model this entity. instances of this class will help us pass user data from the database to the outer layers of the backend, and ultimately the mobile application, without exposing sensitive information such as the password hash and salt values. in the models folder, let’s create the user-profile.js file. then, type the userprofilemodel definition: var userprofilemodel = function(cnf) { this.email = cnf.email, this.firstname = cnf.firstname, this.lastname = cnf.lastname }; module.exports = userprofilemodel; in the model we defined three properties to hold the user’s first name, last name and email. this gives us a nice data transfer object that we can send from the controller out to the mobile app when the mobile app needs to display these data. we are not storing password information in instances of this model so there is no opportunity for this information to be pulled from the database and sent out as part of an http response. creating the controller it’s finally time to turn our attention to the controller itself. let’s create the account.js file in the controllers directory. we will declare the controller as follows: var accountcontroller = function (usermodel, session, mailer) { this.crypto = require('crypto'); this.uuid = require('node-uuid'); this.apiresponse = require('../models/api-response.js'); this.apimessages = require('../models/api-messages.js'); this.userprofilemodel = require('../models/user-profile.js'); this.usermodel = usermodel; this.session = session; this.mailer = mailer; }; module.exports = accountcontroller; notice that we are injecting three dependencies into the controller. the usermodel argument is an instance of the user mongoose class that we created a few minutes ago. as you already know, this is an object that knows how to save and retrieve user data from the mondodb database. the session argument is an object that the controller will use to store session data. the mailer argument is a helper object that the controller will use to send the password reset email to the user. what we are doing here is using a dependency injection approach by passing to the controller some of the entities it needs to do its job. this will make it really easy for us to test the controller using mock objects, without having to instance the database, session and mailer objects that we will use in production. in the next chapter of this tutorial you will see how this is done when we create the tests for the controller. we are also declaring a number of variables inside the controller. the crypto and uuid variables refer to the node.crypto and node-uuid modules, which we will use to generate password hashes and unique identifiers needed when we register and log on users. the apiresponse, apimessages and userprofile internal variables refer to the model classes with the same names that we created a few minutes ago. the session getter and setter methods let’s move on to implementing the controller’s public interface that we designed earlier. first, we will create the setter and getter methods for the session, immediately below the controller’s declaration: accountcontroller.prototype.getsession = function () { return this.session; }; accountcontroller.prototype.setsession = function (session) { this.session = session; }; we will use these methods to set or grab a reference to the controller’s session variable. the hashpassword method we will use this method to create a cryptographically-strong pseudo random hash of a password: accountcontroller.prototype.hashpassword = function (password, salt, callback) { // we use pbkdf2 to hash and iterate 10k times by default var iterations = 10000, keylen = 64; // 64 bit. this.crypto.pbkdf2(password, salt, iterations, keylen, callback); }; within hashpassword, we call crypto.pbkdf2, which uses a pseudorandom function to derive a key of the given length from the given password, salt and number of iterations. remember that we will save this hash in the database, instead of saving the password in clear text or encrypted. this is a good security measure because it’s very difficult to use the hash to obtain the original password without knowing the function used, salt, iteration and keylen values. the logon method next, we will create the logon method: accountcontroller.prototype.logon = function(email, password, callback) { var me = this; me.usermodel.findone({ email: email }, function (err, user) { if (err) { return callback(err, new me.apiresponse({ success: false, extras: { msg: me.apimessages.db_error } })); } if (user) { me.hashpassword(password, user.passwordsalt, function (err, passwordhash) { if (passwordhash == user.passwordhash) { var userprofilemodel = new me.userprofilemodel({ email: user.email, firstname: user.firstname, lastname: user.lastname }); me.session.userprofilemodel = userprofilemodel; return callback(err, new me.apiresponse({ success: true, extras: { userprofilemodel:userprofilemodel } })); } else { return callback(err, new me.apiresponse({ success: false, extras: { msg: me.apimessages.invalid_pwd } })); } }); } else { return callback(err, new me.apiresponse({ success: false, extras: { msg: me.apimessages.email_not_found } })); } }); }; inside logon we first create the me variable to hold a reference to the accountcontroller instance that we can use inside callback functions that we will create inline. next, we call the findone method of the usermodel instance to try to find a user with the same email in the mongodb database. the findmethod is provided by mongoose. remember that usermodel is an instance of the user model that we create with mongoose’s help. if the call to findone produces an error, we immediately invoke the callback argument, passing an apiresponse instance where the success property is set to false and the extra property contains a message that explains that there was a database error. if the call to findone produces a user, we proceed to hash the password provided by the user who is attempting to log on, and compare the hash to the password hash of the user that we found in the database. if the hashes are equal, it means that the user attempting to log on provided a valid password and we can move on to create a userprofile instance and save it to the controller’s session variable. we then invoke the callback function, setting the response’s success property to true and passing the userprofile instance in the extras property of the response. when the hashes don’t match, we invoke the callback function, setting the response’s success property to false and passing an “invalid password” reason in the extras property. finally, if the call to findone does not produce a user, we invoke the callback function with a response where the extras property contains a message indicating that the provided email was not found. the logoff method we will use the logoff method to terminate a user’s session: accountcontroller.prototype.logoff = function () { if (this.session.userprofilemodel) delete this.session.userprofilemodel; return; }; to terminate the session we simply destroy the userprofile instance that we previously saved in the controller’s session variable. the register method the controller’s register method allows a user to register with the application: accountcontroller.prototype.register = function (newuser, callback) { var me = this; me.usermodel.findone({ email: newuser.email }, function (err, user) { if (err) { return callback(err, new me.apiresponse({ success: false, extras: { msg: me.apimessages.db_error } })); } if (user) { return callback(err, new me.apiresponse({ success: false, extras: { msg: me.apimessages.email_already_exists } })); } else { newuser.save(function (err, user, numberaffected) { if (err) { return callback(err, new me.apiresponse({ success: false, extras: { msg: me.apimessages.db_error } })); } if (numberaffected === 1) { var userprofilemodel = new me.userprofilemodel({ email: user.email, firstname: user.firstname, lastname: user.lastname }); return callback(err, new me.apiresponse({ success: true, extras: { userprofilemodel: userprofilemodel } })); } else { return callback(err, new me.apiresponse({ success: false, extras: { msg: me.apimessages.could_not_create_user } })); } }); } }); }; the first step that we take in register is to check if a user with the same email address of the user that is attempting to register exists in the database. as we did in the logon method, if there is a database error we will immediately invoke the callback function and send out an apiresponse instance explaining that there was a database error. if we find an user that has the same email address of the user that is attempting to register, we also stop the registration process, as we cannot have two users with the same email address. in this case the extras property of the apiresponse instance that we send out contains a message explaining that the email address already exists. if we don’t find the email address in the database, we proceed to save the new user by invoking save method (inherited from mongooose) of the user class. the save method produces a numberaffected argument in its callback function. we check numberaffected to make sure that the new user was saved. if numberaffected is 1, we create a userprofile instance and send it out embedded in an apiresponse object. if numberaffected is not 1, we produce an apiresponse indicating that the registration failed. the resetpassword method the resetpassword method is the first step of the password reset workflow that we defined in the mobile application user registration, login and logout screens tutorial of this series. the method consists of the following code: accountcontroller.prototype.resetpassword = function (email, callback) { var me = this; me.usermodel.findone({ email: email }, function (err, user) { if (err) { return callback(err, new me.apiresponse({ success: false, extras: { msg: me.apimessages.db_error } })); } // save the user's email and a password reset hash in session. we will use var passwordresethash = me.uuid.v4(); me.session.passwordresethash = passwordresethash; me.session.emailwhorequestedpasswordreset = email; me.mailer.sendpasswordresethash(email, passwordresethash); return callback(err, new me.apiresponse({ success: true, extras: { passwordresethash: passwordresethash } })); }) }; in order to initiate a password reset sequence, users need to provide their email address. inside resetpassword we use the provided email address to retrieve the user’s record from the database. if the record exists, we create a unique identifier called passwordresethash, and pass this identifier and the user’s email address to the mailer object’s sendpasswordresethash method. this method sends a message to the user, containing the unique identifier and a password reset link that they can use to change their password. we will implement the mailer module in the next chapter of this tutorial. inside resetpassword we also save the password reset hash and the user’s email in the controller’s session variable so we can later compare them to the values provided by the user in the final step of the password reset process. if the database doesn’t have a record for the provided email address, we return an apiresponse whose extras property explains that the email was not found. the resetpasswordfinal method users will invoke this method when they access a special web page using the “password reset” link inside the email that they will receive after they perform the first step of the password reset process. here’s the code for the method: accountcontroller.prototype.resetpasswordfinal = function (email, newpassword, passwordresethash, callback) { var me = this; if (!me.session || !me.session.passwordresethash) { return callback(null, new me.apiresponse({ success: false, extras: { msg: me.apimessages.password_reset_expired } })); } if (me.session.passwordresethash !== passwordresethash) { return callback(null, new me.apiresponse({ success: false, extras: { msg: me.apimessages.password_reset_hash_mismatch } })); } if (me.session.emailwhorequestedpasswordreset !== email) { return callback(null, new me.apiresponse({ success: false, extras: { msg: me.apimessages.password_reset_email_mismatch } })); } var passwordsalt = this.uuid.v4(); me.hashpassword(newpassword, passwordsalt, function (err, passwordhash) { me.usermodel.update({ email: email }, { passwordhash: passwordhash, passwordsalt: passwordsalt }, function (err, numberaffected, raw) { if (err) { return callback(err, new me.apiresponse({ success: false, extras: { msg: me.apimessages.db_error } })); } if (numberaffected < 1) { return callback(err, new me.apiresponse({ success: false, extras: { msg: me.apimessages.could_not_reset_password } })); } else { return callback(err, new me.apiresponse({ success: true, extras: null })); } }); }); }; to reset their password a user will need to provide their email address and a new password, along with the password reset hash that we sent them in the password reset email generated from the resetpassword method. we will save the user from having to type the password reset hash by embedding the hash in the link inside the password reset email. in the next chapter of this series we will create the mailer class and implement the email features. inside resetpasswordfinal, we first check that the password reset hash is also saved in the controller’s session variable. if the hash does not exist, we return an apiresponse whose extras property explains that the password reset period expired. as a security measure, we want to limit the period of time during which a user can reset their password to the length of a session timeout period. if the password reset hash value stored in the session and the value supplied by the user do not match, we will assume that the user who requested the password reset and the user who is providing the new password are not the same. in such a case we return an apiresponse explaining that there is a mismatch of the hashes. the same logic applies when the email value stored in the session and the value supplied by the user do not match, in which case we return an apiresponse explaining that there is a mismatch of the email addresses. if the password reset hash and email address validations are successful, we proceed to hash the new password and save it by calling the user model’s update method, which is inherited from mongoose. the update method returns the number of records affected by the update operation. we check this value and return an apiresponse that signals to the outside world if the update operation succeeded or not. summary and next steps we just began building the backend for a meeting room booking application that we defined in the first chapter of this series . this is a mongodb and mongoose backend paired to a node.js and express web server. our focus in this article was building a controller module that will handle the user registration, login and logout features of the application. we implemented the controller’s public interface, along with a number of helper classes that will allow the controller to do its work. in the next chapter of this tutorial we will turn our attention to testing the controller, which will take us through choosing a testing library and implementing the tests for the controller’s features. make sure to sign up for miamicoder’s newsletter so you can be among the first to know when next part of this tutorial is available. download the source code download the mongodb and mongoose backend tutorial here: mongodb and mongoose backend for mobile application previous chapters of this series these are the previous parts of this series: mobile app tutorial: the meeting room booking app, part 1 mobile app tutorial: the meeting room booking app, part 2 mobile app tutorial: the meeting room booking app, part 3 mobile ui patterns – a flowchart for user registration, login and logout

In previous post we gave brief introduction on Recursive Descent Parsers and we implemented parser that was able to parse and calculate simple arithmetic expressions with addition and subtraction. To be (True) or !(To be True)? This time we will try to tackle little bit more complex example that will parse and evaluate Boolean logic expressions that will include negation and parenthesis. Examples of expressions we want to be able to parse and evaluate are: True And True And False True !False (!(False)) and (!(True) etc Let’s assemble a EBNF grammar for this type of expressions: Expression := [ "!" ] { BooleanOperator Boolean } Boolean := BooleanConstant | Expression | "(" ")" BooleanOperator := "And" | "Or" BooleanConstant := "True" | "False" You can see that our Terminal Symbols are “And”, “Or” (BooleanOperator) and “True”, “False” (BooleanConstant) and off course “!” and parenthesis. Expression can have optional negation symbol “!” and then Boolean (which can be BooleanConstant or Expression or Expression in parenthesis). Every next Boolean expression is optional but if its there, it must be preceded by BooleanOperator so that we can parse the final value by combining it with previous Boolean value. Obviously we will have some recursion there, but more on that later when we start implementing the parser. Always Tokenize everything! Before looking into the parser, we have to implement the Tokenizer class that will parse the raw text of the expression, tokenize it and return IEnumerable so that our parser can have less to worry about. Here is the Tokenizer class: public class Tokenizer { private readonly StringReader _reader; private string _text; public Tokenizer(string text) { _text = text; _reader = new StringReader(text); } public IEnumerable Tokenize() { var tokens = new List(); while (_reader.Peek() != -1) { while (Char.IsWhiteSpace((char) _reader.Peek())) { _reader.Read(); } if (_reader.Peek() == -1) break; var c = (char) _reader.Peek(); switch (c) { case '!': tokens.Add(new NegationToken()); _reader.Read(); break; case '(': tokens.Add(new OpenParenthesisToken()); _reader.Read(); break; case ')': tokens.Add(new ClosedParenthesisToken()); _reader.Read(); break; default: if (Char.IsLetter(c)) { var token = ParseKeyword(); tokens.Add(token); } else { var remainingText = _reader.ReadToEnd() ?? string.Empty; throw new Exception(string.Format("Unknown grammar found at position {0} : '{1}'", _text.Length - remainingText.Length, remainingText)); } break; } } return tokens; } private Token ParseKeyword() { var text = new StringBuilder(); while (Char.IsLetter((char) _reader.Peek())) { text.Append((char) _reader.Read()); } var potentialKeyword = text.ToString().ToLower(); switch (potentialKeyword) { case "true": return new TrueToken(); case "false": return new FalseToken(); case "and": return new AndToken(); case "or": return new OrToken(); default: throw new Exception("Expected keyword (True, False, And, Or) but found "+ potentialKeyword); } } } Not much happening there really, we just go through the characters of the expression, and if its negation or parenthesis we return proper sub classes of Token and if we detect letters we try to parse one of our keywords (“True”, “False”, “And”, “Or”). If we encounter unknown keyword we throw exception to be on the safe side. I deliberately did not do much validation of the expression in this class since this is done later in the Parser – but nothing would stop us from doing it here also – i will leave that exercise to the reader. The Parser Inside of our parser we have main Parse method that will start the process of parsing the tokens, handle the negation, and continue parsing sub-expressions while it encounters one of the OperandTokens (AndToken or OrToken). public bool Parse() { while (_tokens.Current != null) { var isNegated = _tokens.Current is NegationToken; if (isNegated) _tokens.MoveNext(); var boolean = ParseBoolean(); if (isNegated) boolean = !boolean; while (_tokens.Current is OperandToken) { var operand = _tokens.Current; if (!_tokens.MoveNext()) { throw new Exception("Missing expression after operand"); } var nextBoolean = ParseBoolean(); if (operand is AndToken) boolean = boolean && nextBoolean; else boolean = boolean || nextBoolean; } return boolean; } throw new Exception("Empty expression"); } Parsing of the sub-expressions is handled in the ParseBoolean method: private bool ParseBoolean() { if (_tokens.Current is BooleanValueToken) { var current = _tokens.Current; _tokens.MoveNext(); if (current is TrueToken) return true; return false; } if (_tokens.Current is OpenParenthesisToken) { _tokens.MoveNext(); var expInPars = Parse(); if (!(_tokens.Current is ClosedParenthesisToken)) throw new Exception("Expecting Closing Parenthesis"); _tokens.MoveNext(); return expInPars; } if (_tokens.Current is ClosedParenthesisToken) throw new Exception("Unexpected Closed Parenthesis"); // since its not a BooleanConstant or Expression in parenthesis, it must be a expression again var val = Parse(); return val; } This method tries to parse the simplest BooleanValueToken, then if it encounter OpenParenthesisToken it handles the Expressions in parenthesis by skipping the OpenParenthesisToken and then calling back the Parse to get the value of expressions and then again skipping the ClosedParenthesisToken once parsing of inner expression is done. If it does not find BooleanValueToken or OpenParenthesisToken – method simply assumes that what follows is again an expression so it calls back Parse method to start the process of parsing again. To be logical is to be simple As you see, we implemented the parser in less then 90 lines of C# code. Maybe this code is not particularity useful but its good exercise on how to build parsing logic recursively. It could be further improved by adding more logic to throw exceptions when unexpected Tokens are encountered but again – i leave that to the reader (for example expression like “true)” should throw exception, but in this version of code it will not do that, it will still parse the expression correctly by ignoring the closing parenthesis). Tests Here are some of the Unit Tests i built to test the parser: [TestCase("true", ExpectedResult = true)] [TestCase(")", ExpectedException = (typeof(Exception)))] [TestCase("az", ExpectedException = (typeof(Exception)))] [TestCase("", ExpectedException = (typeof(Exception)))] [TestCase("()", ExpectedException = typeof(Exception))] [TestCase("true and", ExpectedException = typeof(Exception))] [TestCase("false", ExpectedResult = false)] [TestCase("true ", ExpectedResult = true)] [TestCase("false ", ExpectedResult = false)] [TestCase(" true", ExpectedResult = true)] [TestCase(" false", ExpectedResult = false)] [TestCase(" true ", ExpectedResult = true)] [TestCase(" false ", ExpectedResult = false)] [TestCase("(false)", ExpectedResult = false)] [TestCase("(true)", ExpectedResult = true)] [TestCase("true and false", ExpectedResult = false)] [TestCase("false and true", ExpectedResult = false)] [TestCase("false and false", ExpectedResult = false)] [TestCase("true and true", ExpectedResult = true)] [TestCase("!true", ExpectedResult = false)] [TestCase("!(true)", ExpectedResult = false)] [TestCase("!(true", ExpectedException = typeof(Exception))] [TestCase("!(!(true))", ExpectedResult = true)] [TestCase("!false", ExpectedResult = true)] [TestCase("!(false)", ExpectedResult = true)] [TestCase("(!(false)) and (!(true))", ExpectedResult = false)] [TestCase("!((!(false)) and (!(true)))", ExpectedResult = true)] [TestCase("!false and !true", ExpectedResult = false)] [TestCase("false and true and true", ExpectedResult = false)] [TestCase("false or true or false", ExpectedResult = true)] public bool CanParseSingleToken(string expression) { var tokens = new Tokenizer(expression).Tokenize(); var parser = new Parser(tokens); return parser.Parse(); } Full source code of the whole solution is available at my BooleanLogicExpressionParser GitHub repo. Stay tuned because next time we will implement parser for more complex arithmetical expressions.