If you use Java for programming, read on to learn about tools like Eclipse IDE, the Java Development Kit, and other must-know tools.

eclipse projects have the nice features that they can link to files and folders: so instead of having the physical file, it is just a pointer to a file. this is very cool as that way i can point to shared files, or keep files in a common place referenced from projects, and so on. linked folder and file in eclipse as with most things in eclipse, there is not a single way how to do things. so i’m showing in this post several ways how to link to files and folders. creating a new link select the folder/project where to create a link to a file. use the context menu with n ew > file (or the menu file > new > other > general > file ): new file that opens a dialog to create a new file. i can select which project/folder i want to use for this: new file dialog next, click on ‘advanced’, enable ‘link to file in the file system’ and browse to the file you want to link to: link to file this will set the link to the file: specified link to file as an absolute path (c:\….) is probably not really what you want, there are eclipse path variables you can use: path variables :idea: note that these are eclipse path variables (not eclipse build variables), see “ eclipse build variables “. to use the path variable for the link, use the ‘extend…’ button and then select the file, then press ok: extending path variable this now uses a path variable for that link. note that it shows as well to which file it resolves: resolved path variable for linked file press finish, and the link will be created. note the small ‘arrow’ in the icon (see “ icon and label decorators in eclipse “) to show a linked file: linked file modifying a link if you right-click on that linked file and select ‘properties’ of it, you can see that it is really a linked file, with the link information, and you can change/edit that link any time: linked file properties linked folder as for link to files, its possible to create ‘link to folders’. it works the same way: select the folder, then use file > new > folder: creating new folder use default location : this creates a normal folder. virtual folder : this does not create a physical folder, but a virtual ‘container’ where i can place links or other virtual folders. this is useful to organize links and virtual folders, without the need for a physical folder. linked folder : like linked files, this links to a folder. :idea: the cool thing about linked (source) folders is: when i add new files to that folder where it links to, the projects with that linked folder to it will ‘see’ the extra files too, and that way new files are automatically added to the project. i do this many times, and it is like a ‘library’ folder for me: i can add a new source file to that ‘library’ folder, and every project linking to that folder will automatically have it added. :-) deleting links linked files and linked folders can be deleted from the project too. in that case, the destination of the link is *not* deleted, only the link: deleting a link to a folder using drag & drop as mentioned at the beginning: there are multiple ways to do the same thing in eclipse. instead using the top menu, or using the context menu, i can use drag & drop. to create links, i need to hold the ctrl key: drag and drop with ctrl pressed to create a link :idea: notice that during the drag&drop with ctrl key pressed the icon gets a ‘+’ to show copy/linking mode. when i drop the file: i get the usual choices how i want to create the link: link to files and folders dialog drag&drop of files and folders do not work inside eclipse. what works as well under windows is to drag&drop a file or folder from the windows explorer :-). summary links to files and folders are a cool thing in eclipse, and they can be created with menus or even simpler with drag&drop. this is not limited to inside eclipse: i can drag&drop from outside with the windows explorer and that way can link to files and folders everywhere :-) happy linking :-)

in “ semihosting with kinetis design studio ” i used printf() to exchange text and data between the target board and the host using the debug connection. kinetis design studio (kds) has that semihosting baked into its libraries. what about if using the gnu arm embedded (launchpad) tools and libraries (see “ switching arm gnu tool chain and libraries in kinetis design studio “)? actually it requires two more steps, but is very easy too. semihosting output there are three things to be in place to use semihosting with the gnu arm embedded (launchpad) libraries: option in the gnu linker settings enabling semihosting in the debugger settings initializing the gnu libraries linker option to enable semihosting for the gnu arm embedded ( launchpad ) libraries, i need to add --specs=rdimon.specs to the linker options: linker option to enable semihosting in case i’m using newlib-nano and want to use printf() and/or scanf() with floating point support, i need to pull in some symbols explicitly with the linker options ‘u': -u _scanf_float -u _printf_float debugger settings in the gnu arm eclipse plugins, i need to enable semihosting. segger j-link for segger j-link, i enable the console in the launch configuration: allocated semihosting console for segger additionally i enable semihosting options in the startup options of the debugger: enabled semihosting in the startup options for segger p&e multilink for p&e the following settings are used: semihosting settings for pne settings for openocd the following settings are used for openocd: openocd semihosting settings initializing the gnu libraries if you would now try to use semihosting with running the debugger, you probably will get error messages like this (e.g. from segger j-link): warning: semihosting command sys_flen failed. handle is 0. warning: semihosting command sys_write failed. handle is 0. warning: semihosting command sys_write failed. handle is 0. warning: semihosting command sys_write failed. handle is 0. the reason is that the semihosting needs to be enabled by the application. i need to call initialise_monitor_handles() before i’m using printf() : 1 2 3 4 5 6 7 8 extern void initialise_monitor_handles( void ); /* prototype */ int main( void ) { initialise_monitor_handles(); /* initialize handles */ for (;;) { printf ( "hello world!\r\n" ); } } with this, i can use printf() and scanf() through a debugger connection. semihosting printf output summary while i don’t like printf() for many reasons, sometimes it is useful to exchange data with the host. using semihosting no physical connection is required, as the communication goes through the debugger. it is somewhat intrusive, and adds code and data overhead, but the gnu arm embedded (launchpad) libraries (both newlib and newlib-nano) have semihosting built-in. it is a matter to enable it in the linker and debugger settings, and to initialize the handles in the application. happy semihosting :-)

Apache Camel 2.14 is being released later this month. There is a slight holdup due some Apache infrastructure issue which is being worked on. This blog post is to talk about one of the new functions we have added to this release. Thanks to Lauri Kimmel who donated a camel-metrics component, we integrated with the excellent codehale metrics library. So I took this component one step further and integrated it with the Camel routes so we have additional metrics about the route performances using codehale metrics. This allows end users to seamless feed Camel routing information together with existing data they are gathering using codehale metrics. Also take note we have a lot of existing metrics from camel-core which of course is still around. What codehale brings to the table is that they have additional statistical data which we do not have in camel-core. To use the codehale metics all you need to do is: add camel-metrics component enable route metrics in XML or Java code To enable in XML you declare a as shown below: &;t;bean id="metricsRoutePolicyFactory" class="org.apache.camel.component.metrics. routepolicy.MetricsRoutePolicyFactory"/> And doing so in Java code is easy as well by calling this method on your CamelContext. context.addRoutePolicyFactory(new MetricsRoutePolicyFactory()); Now performance metrics is only useable if you have a way of displaying them, and for that you can use hawtio. Notice you can use any kind of monitoring tooling which can integrate with JMX, as the metrics is available over JMX. The actual data is 100% codehale json format, where a piece of the data is shown in the figure below. Sample of the route metrics JSON data The next release of hawtio supports Camel 2.14 and automatic detects if you have enabled route metrics and if so, then shows a sub, where the information can be seen in real time in a graphical charts. hawtio have detected that we have route metrics enabled, and shows a sub tab where we can see the data in real time The screenshot above is from the new camel-example-servlet-rest-tomcat which we ship out of the box. This example demonstrates another new functionality in Camel 2.14 which is the Rest DSL (I will do a blog about that later). This example enables the route metrics out of the box, so what I did was to deploy this example together with hawtio (the hawtio-default WAR) in Apache Tomcat 8. With hawtio you can also build custom dashboards, so here at the end I have put together a dashboard with various screens from hawtio to have a custom view of a Camel application. hawtio dashboard with Camel route and metrics as well control panel to control the route(s), and the logs in the bottom.

The NetBeans Team has released NetBeans IDE 8.0.1, with significant enhancements to features relating to HTML5, JavaScript, and CSS3. An update to NetBeans IDE 8.0, this release includes the following notable changes: Modularity and enterprise JavaScript development support via integration of RequireJS Tools for working seamlessly with Grunt, Karma, and Bower Enhanced Java Editor, PHP Editor, and Git Tools Support for new versions of WebLogic, GlassFish, Tomcat, WildFly, and PrimeFaces Performance improvements and bug fixes Complete list of features: http://wiki.netbeans.org/NewAndNoteworthyNB801 To get the recent changes: Download and install NetBeans 8.0.1, which includes the recently released GlassFish 4.1 OR If you already have NetBeans IDE 8.0 installed, launch the IDE and an update notification will appear. Alternatively, choose Help | Check for Updates. Click the alert-box to install the updates. For details on upgrading, see the YouTube screencast "How to Upgrade to NetBeans IDE 8.0.1 from NetBeans IDE 8.0" on the NetBeans YouTube channel. NetBeans IDE 8.0.1 is available in English, Brazilian Portuguese, Japanese, Russian, and Simplified Chinese. We welcome feedback about your use of NetBeans software. Share your thoughts on the NetBeans mailing lists and forums; and keep track of NetBeans news by subscribing to the NetBeans Weekly Newsletter and following NetBeans on Twitter, Facebook, and YouTube.

Overview : Recently, I was working on a project which uses a REST services layer to communicate with the client application (GWT application). So I have spent a lot of to time to figure out how to secure the REST services with Spring Security. This article describes the solution I found, and I have implemented. I hope that this solution will be helpful to someone and will save a much valuable time. The solution : In a normal web application, whenever a secured resource is accessed Spring Security check the security context for the current user and will decide either to forward him to login page (if the user is not authenticated), or to forward him to the resource not authorised page (if he doesn’t have the required permissions). In our scenario this is different, because we don’t have pages to forward to, we need to adapt and override Spring Security to communicate using HTTP protocols status only, below I liste the things to do to make Spring Security works best : The authentication is going to be managed by the normal form login, the only difference is that the response will be on JSON along with an HTTP status which can either code 200 (if the autentication passed) or code 401 (if the authentication failed) ; Override the AuthenticationFailureHandler to return the code 401 UNAUTHORIZED ; Override the AuthenticationSuccessHandler to return the code 20 OK, the body of the HTTP response contain the JSON data of the current authenticated user ; Override the AuthenticationEntryPoint to always return the code 401 UNAUTHORIZED. This will override the default behavior of Spring Security which is forwarding the user to the login page if he don’t meet the security requirements, because on REST we don’t have any login page ; Override the LogoutSuccessHandler to return the code 20 OK ; Like a normal web application secured by Spring Security, before accessing a protected service, it is mandatory to first authenticate by submitting the password and username to the Login URL. Note: The following solution requires Spring Security in version minimum 3.2. Overriding the AuthenticationEntryPoint : Class extends org.springframework.security.web.AuthenticationEntryPoint, and implements only one method, which sends response error (with 401 status code) in cause of unauthorized attempt. @Component public class HttpAuthenticationEntryPoint implements AuthenticationEntryPoint { @Override public void commence(HttpServletRequest request, HttpServletResponse response, AuthenticationException authException) throws IOException { response.sendError(HttpServletResponse.SC_UNAUTHORIZED, authException.getMessage()); } } Overriding the AuthenticationSuccessHandler : The AuthenticationSuccessHandler is responsible of what to do after a successful authentication, by default it will redirect to an URL, but in our case we want it to send an HTTP response with data. @Component public class AuthSuccessHandler extends SavedRequestAwareAuthenticationSuccessHandler { private static final Logger LOGGER = LoggerFactory.getLogger(AuthSuccessHandler.class); private final ObjectMapper mapper; @Autowired AuthSuccessHandler(MappingJackson2HttpMessageConverter messageConverter) { this.mapper = messageConverter.getObjectMapper(); } @Override public void onAuthenticationSuccess(HttpServletRequest request, HttpServletResponse response, Authentication authentication) throws IOException, ServletException { response.setStatus(HttpServletResponse.SC_OK); NuvolaUserDetails userDetails = (NuvolaUserDetails) authentication.getPrincipal(); User user = userDetails.getUser(); userDetails.setUser(user); LOGGER.info(userDetails.getUsername() + " got is connected "); PrintWriter writer = response.getWriter(); mapper.writeValue(writer, user); writer.flush(); } } Overriding the AuthenticationFailureHandler : The AuthenticationFaillureHandler is responsible of what to after a failed authentication, by default it will redirect to the login page URL, but in our case we just want it to send an HTTP response with the 401 UNAUTHORIZED code. @Component public class AuthFailureHandler extends SimpleUrlAuthenticationFailureHandler { @Override public void onAuthenticationFailure(HttpServletRequest request, HttpServletResponse response, AuthenticationException exception) throws IOException, ServletException { response.setStatus(HttpServletResponse.SC_UNAUTHORIZED); PrintWriter writer = response.getWriter(); writer.write(exception.getMessage()); writer.flush(); } } Overriding the LogoutSuccessHandler : The LogoutSuccessHandler decide what to do if the user logged out successfully, by default it will redirect to the login page URL, because we don’t have that I did override it to return an HTTP response with the 20 OK code. @Component public class HttpLogoutSuccessHandler implements LogoutSuccessHandler { @Override public void onLogoutSuccess(HttpServletRequest request, HttpServletResponse response, Authentication authentication) throws IOException { response.setStatus(HttpServletResponse.SC_OK); response.getWriter().flush(); } } Spring security configuration : This is the final step, to put all what we did together, I prefer using the new way to configure Spring Security which is with Java no XML, but you can easily adapt this configuration to XML. @Configuration @EnableWebSecurity public class WebSecurityConfig extends WebSecurityConfigurerAdapter { private static final String LOGIN_PATH = ApiPaths.ROOT + ApiPaths.User.ROOT + ApiPaths.User.LOGIN; @Autowired private NuvolaUserDetailsService userDetailsService; @Autowired private HttpAuthenticationEntryPoint authenticationEntryPoint; @Autowired private AuthSuccessHandler authSuccessHandler; @Autowired private AuthFailureHandler authFailureHandler; @Autowired private HttpLogoutSuccessHandler logoutSuccessHandler; @Bean @Override public AuthenticationManager authenticationManagerBean() throws Exception { return super.authenticationManagerBean(); } @Bean @Override public UserDetailsService userDetailsServiceBean() throws Exception { return super.userDetailsServiceBean(); } @Bean public AuthenticationProvider authenticationProvider() { DaoAuthenticationProvider authenticationProvider = new DaoAuthenticationProvider(); authenticationProvider.setUserDetailsService(userDetailsService); authenticationProvider.setPasswordEncoder(new ShaPasswordEncoder()); return authenticationProvider; } @Override protected void configure(AuthenticationManagerBuilder auth) throws Exception { auth.authenticationProvider(authenticationProvider()); } @Override protected AuthenticationManager authenticationManager() throws Exception { return super.authenticationManager(); } @Override protected void configure(HttpSecurity http) throws Exception { http.csrf().disable() .authenticationProvider(authenticationProvider()) .exceptionHandling() .authenticationEntryPoint(authenticationEntryPoint) .and() .formLogin() .permitAll() .loginProcessingUrl(LOGIN_PATH) .usernameParameter(USERNAME) .passwordParameter(PASSWORD) .successHandler(authSuccessHandler) .failureHandler(authFailureHandler) .and() .logout() .permitAll() .logoutRequestMatcher(new AntPathRequestMatcher(LOGIN_PATH, "DELETE")) .logoutSuccessHandler(logoutSuccessHandler) .and() .sessionManagement() .maximumSessions(1); http.authorizeRequests().anyRequest().authenticated(); } } This was a sneak peak at the overall configuration, I attached in this article a Github repository containing a sample project https://github.com/imrabti/gwtp-spring-security. I hope this will help some of you developers struggling to figure out a solution, please feel free to ask any questions, or post any enhancements that can make this solution better.

i am eclipse user. so when i wanted to analyze my code with pmd, i needed to use “pmd for eclipse” plugin. this plugin used to be very buggy, which was enhanced in later versions (currently 4.0.3). but the performance is really bad sometimes. especially when you are dealing with relatively big codebase and have option “check code after saving” on. ecplise-pmd plugin so when i realized that there is new alternative pmd plugin called eclipse-pmd out there i evaluated it immediately with great happiness. installation uses modern eclipse marketplace method. you just need to go “help” -> “eclipse marketplace…” and search for “eclipse-pmd” . than hit “install” and follow instructions. after installation i was a little bit confused because i didn’t find any configuration options it general settings ( “window” -> “preferences” ). i discovered that you need to turn on pmd for each project separately. which make sense, because you can have different rule set per project. so to turn it on, right click on project -> “preferences” -> “pmd” (there would be two pmd sections if you didn’t uninstall old pmd plugin) -> “enable pmd for this project” -> “add…” . now you should pick a location of pmd ruleset file. unlike old pmd plugin, eclipse-pmd don’t import ruleset. it is using ruleset file directly. this is very handy, because typically you want to have it in source control. when you pull changes to ruleset file from source control system, they are applied without re-import (re-import was needed for old pmd plugin). problem can be when you (or your team) don’t have existing ruleset. i would suggest to start with full ruleset and exclude rules you don’t want to use. your ruleset would evolve anyway, so starting with most restrictive (default) deck make perfect sense for me. unfortunately eclipse-pmd plugin doesn’t provide option to generate ruleset file. so i created full ruleset for pmd 5.1.1 (5.1.1 is pmd version not plugin version) . i have to admit that it was created with help of old pmd plugin. you can see that i literally included all the rule categories. i would suggest to specify your set this way and exclude/configure rules explicitly as needed. here is link to pmd site that explains how to customize your ruleset . this approach can be handy when pmd version will be updated. new rules can appear in category and they will be automatically included into your ruleset when you are listing categories, not rules individually. but you have to keep eye on new rules/categories when updating pmd version anyway, because categories often change with new pmd version. so now we should have rulset configured and working. here are some screen shots of rules in action: when you hover over left side panel warning: when you hover over problematic snippet: when you do quick fix on problematic snippet: generating suppress warning annotation for pmd rules is very nice feature. it also provide quick fixes for some rules. take a look at its change log site for full list. these pmd warning sometimes clash with eclipse native warnings, so there is possibility to make them more visible. go to “window” -> “preferences” -> “general” -> “editors” -> “text editors” -> “annotations” and find “pmd violations” . here you can configure your own style of highlighting pmd issues. this is mine: to explore full feature list of this plugin take a look at its change log site. there are some features in old plugin i miss though. for example i would appreciate some quick link or full description of the rule. short description provided is sometimes not enough. i encourage you to take a look at full pmd rule description if you are not sure what’s source of the problem. you will learn a lot about java language itself or about libraries you are using. quick links would help a lot in such case. also some rules doesn’t use code highlighting (only side panel markers). it is sometimes hard to distinguish between compiler and pmd issues. this is problem for me because our team doesn’t use javadoc warnings but i do. so i get a lot of javadoc warnings from code written by teammates. and sometimes i can miss pmd issue because it is lost in javadoc warnings. (fortunately svn commit is rejected if i forget to fix some rule). conclusion this plugin enhanced my eclipse workflow. no more disruptions because of endless “checking code…” processing by old plugin.

Learn How to Enable Swagger in your Spring Boot Web Application

In this article we will discuss how we can perform CRUD operations on a backbone model using a REST based HTTP service. Background Earlier we have discussed about the benefits of using backbone.js and we also looked at the backbone models. Link to complete series: BackBone Tutorial – Part 1: Introduction to Backbone.Js BackBone Tutorial – Part 2: Understanding the basics of Backbone Models BackBone Tutorial – Part 3: More about Backbone Models BackBone Tutorial – Part 4: CRUD Operations on BackboneJs Models using HTTP REST Service[^] BackBone Tutorial – Part 5: Understanding Backbone.js Collections[^] BackBone Tutorial – Part 6: Understanding Backbone.js Views[^] BackBone Tutorial – Part 7: Understanding Backbone.js Routes and History[^] In this article we will look at performing the CRUD operations on backbone models using a REST based web service. Using the code The first thing we will do is that we will create a simple REST based web api that can be used to save the data on the server using our simple backbone application. For this I have created a simple database with a single table as: The ID field is configured to auto increment and this is the primary key of the table. so while creating a new model we don’t have to provide this to the server. Now on top of this model, I have written a simple ASP.NET web api that will provide us the RESTful api. This API is configured to run on my local machine at: http://localhost:51377/. The API details are as follows: Create: POST http://localhost:51377/api/values Read: GET http://localhost:51377/api/values/{id} Update: PUT http://localhost:51377/api/values/{id} Delete: DELETE http://localhost:51377/api/values/{id} Once we have the API running, we can start working on our backbone model. We had create the backbone model in our previous article as: var Book = Backbone.Model.extend({ defaults: { ID: "", BookName: "" }, idAttribute: "ID", initialize: function () { console.log('Book has been initialized'); this.on("invalid", function (model, error) { console.log("Houston, we have a problem: " + error) }); }, constructor: function (attributes, options) { console.log('Book\'s constructor had been called'); Backbone.Model.apply(this, arguments); }, validate: function (attr) { if (!attr.BookName) { return "Invalid BookName supplied." } } }); The backbone models inherently supports saving on the server using a restful web api. To save the model using a HTTP REST service, we need to specify the urlRoot in the backbone model. To actually save the model, we can call the save on the backbone model.The save method will trigger the validations and if the validations are successful, it will try to identify the action to be performed i.e. create or update and based on that action, it will use urlRoot and call the appropriate REST API to perform the operation. Let us specify the URL root to enable this model to use our web api service. var Book = Backbone.Model.extend({ defaults: { ID: "", BookName: "" }, idAttribute: "ID", initialize: function () { console.log('Book has been initialized'); this.on("invalid", function (model, error) { console.log("Houston, we have a problem: " + error) }); }, constructor: function (attributes, options) { console.log('Book\'s constructor had been called'); Backbone.Model.apply(this, arguments); }, validate: function (attr) { if (!attr.BookName) { return "Invalid BookName supplied." } }, urlRoot: 'http://localhost:51377/api/Books' }); Now let us try to perform CRUD operations on this model. Create To create a new entity on the server, we need to populate the non identity fields in the model (other than ID in this case) and then call the Save method on the model. // Lets perform a create operation [CREATE] var book = new Book({ BookName: "Backbone Book 43" }); book.save({}, { success: function (model, respose, options) { console.log("The model has been saved to the server"); }, error: function (model, xhr, options) { console.log("Something went wrong while saving the model"); } }); Read To read a single book entity, we need to create the book entity with the identity attribute populated, i.e., the ID of the book we want to read. Then we need to call the fetch method on the model object. // Now let us try to retrieve a book [READ] var book1 = new Book({ ID: 40 }); book1.fetch({ success: function (bookResponse) { console.log("Found the book: " + bookResponse.get("BookName")); } }); Update Now let’s say we want to update the name of the book retrieved in the earlier fetch call. All we need to do is set the attributes we need to update and call the save method again. // Lets try to update a book [UPDATE] var book1 = new Book({ ID: 40 }); book1.fetch({ success: function (bookResponse) { console.log("Found the book: " + bookResponse.get("BookName")); // Let us update this retreived book now (doing it in the callback) [UPDATE] bookResponse.set("BookName", bookResponse.get("BookName") + "_updated"); bookResponse.save({}, { success: function (model, respose, options) { console.log("The model has been updated to the server"); }, error: function (model, xhr, options) { console.log("Something went wrong while updating the model"); } }); } }); Delete Now to delete a Model, we just need to call the destroy method of the model object. // Let us delete the model with id 13 [DELETE] var book2 = new Book({ ID: 40 }); book2.destroy({ success: function (model, respose, options) { console.log("The model has deleted the server"); }, error: function (model, xhr, options) { console.log("Something went wrong while deleting the model"); } }); Custom URLs to perform CRUD operation on models There are few scenarios where we might want to have provide custom URLs for the individual operations. This can be achieved by overriding the sync function and providing custom URL for each action. Let us create one more model BookEx to see how this can be done. var BookEx = Backbone.Model.extend({ defaults: { ID: "", BookName: "" }, idAttribute: "ID", // Lets create function which will return the custom URL based on the method type getCustomUrl: function (method) { switch (method) { case 'read': return 'http://localhost:51377/api/Books/' + this.id; break; case 'create': return 'http://localhost:51377/api/Books'; break; case 'update': return 'http://localhost:51377/api/Books/' + this.id; break; case 'delete': return 'http://localhost:51377/api/Books/' + this.id; break; } }, // Now lets override the sync function to use our custom URLs sync: function (method, model, options) { options || (options = {}); options.url = this.getCustomUrl(method.toLowerCase()); // Lets notify backbone to use our URLs and do follow default course return Backbone.sync.apply(this, arguments); } }); Now we can perform the CRUD operations on this model in the same way as we did for the previous model. Point of interest In this article we have looked at how to perform CRUD operations on backbone models using HTTP based REST service. This has been written from a beginner’s perspective. I hope this has been informative. Download sample Web API code: WebAPISample Download sample backbone app code: backboneSample

the article represents steps required to configure tomcat 8 with eclipse kepler. download tomcat 8 and place it within any local folder. download eclipse java ee kepler as of date, tomcat 8 is not supported in eclipse javeee kepler. however, you could add the tomcat 8 by doing following: go to the wtp downloads page, select the latest version (currently 3.6), and download the zip. here’s the current link . copy the all of the files in features and plugins directories of the downloaded wtp into the corresponding eclipse directories in your eclipse folder (overwriting the existing files). start eclipse and click on “servers” tab in the workbench. go ahead and try adding a new server. you would find option for tomcat 8 available for selection as shown below. after clicking finish, you would see a new server added with the name as “tomcat v8.0 server at localhost”. start the server. check http://localhost:8080 (provided you installed tomcat 8 and set http port as 8080) interestingly, you would not see the welcome page, but the 404 error page. to get rid of that, double click on ”tomcat v8.0 server at localhost”. in the window that opens up, select “use tomcat installation” and, change deploy path from wtpwebapps to webapps. look at the figure below. restart the server and access http://localhost:8080 . you are all set.

I have a small Spring boot based application that uses a Postgres database as a datastore. I wanted to document the steps involved in deploying this sample application to Cloud Foundry. Some of the steps are described in the Spring Boot reference guide, however the guides do not sufficiently explain how to integrate with the datastore provided in a cloud based environment. Spring-cloud provides the glue to connect Spring based applications deployed on a Cloud to discover and connect to bound services, so the first step is to pull in the Spring-cloud libraries into the project with the following pom entries: org.springframework.cloud spring-cloud-spring-service-connector 1.0.0.RELEASE org.springframework.cloud spring-cloud-cloudfoundry-connector 1.0.0.RELEASE Once this dependency is pulled in, connecting to a bound service is easy, just define a configuration along these lines: @Configuration public class PostgresCloudConfig extends AbstractCloudConfig { @Bean public DataSource dataSource() { return connectionFactory().dataSource(); } } Spring-Cloud understands that the application is deployed on a specific Cloud(currently Cloud Foundry and Heroku by looking for certain characteristics of the deployed Cloud platform), discovers the bound services, recognizes that there is a bound service using which a Postgres based datasource can be created and returns the datasource as a Spring bean. This application can now deploy cleanly to a Cloud Foundry based Cloud. The sample application can be tried out in a version of Cloud Foundry deployed with bosh-lite, these are how the steps in my machine looks like once Cloud Foundry is up and running with bosh-lite: The following command creates a user provided service in Cloud Foundry: cf create-user-provided-service psgservice -p '{"uri":"postgres://postgres:[email protected]:5432/hotelsdb"}' Now, push the app, however don't start it up. We can do that once the service above is bound to the app: cf push spring-boot-mvc-test -p target/spring-boot-mvc-test-1.0.0-SNAPSHOT.war --no-start Bind the service to the app and restart the app: cf bind-service spring-boot-mvc-test psgservice cf restart spring-boot-mvc-test That is essentially it, Spring Cloud should ideally take over at the point and cleanly parse the credentials from the bound service which within Cloud Foundry translates to an environment variable called VCAP_SERVICES, and create the datasource from it. There is however an issue with this approach - once the datasource bean is created using spring-cloud approach, it does not work in a local environment anymore. The potential fix for this is to use Spring profiles, assume that there is a different "cloud" Spring profile available in Cloud environment where the Spring-cloud based datasource gets returned: @Profile("cloud") @Configuration public class PostgresCloudConfig extends AbstractCloudConfig { @Bean public DataSource dataSource() { return connectionFactory().dataSource(); } } and let Spring-boot auto-configuration create a datasource in the default local environment, this way the configuration works both local as well as in Cloud. Where does this "cloud" profile come from, it can be created using a ApplicationContextInitializer, and looks this way: public class SampleWebApplicationInitializer implementsApplicationContextInitializer { private static final Log logger = LogFactory.getLog(SampleWebApplicationInitializer.class); @Override public void initialize(AnnotationConfigEmbeddedWebApplicationContext applicationContext) { Cloud cloud = getCloud(); ConfigurableEnvironment appEnvironment = applicationContext.getEnvironment(); if (cloud!=null) { appEnvironment.addActiveProfile("cloud"); } logger.info("Cloud profile active"); } private Cloud getCloud() { try { CloudFactory cloudFactory = new CloudFactory(); return cloudFactory.getCloud(); } catch (CloudException ce) { return null; } } } This initializer makes use of the Spring-cloud's scanning capabilities to activate the "cloud" profile. One last thing which I wanted to try was to make my local behave like Cloud atleast in the eyes of Spring-Cloud and this can be done by adding in some environment variables using which Spring-Cloud makes the determination of the type of cloud where the application is deployed, the following is my startup script in local for the app to pretend as if it is deployed in Cloud Foundry: read -r -d '' VCAP_APPLICATION <<'ENDOFVAR' {"application_version":"1","application_name":"spring-boot-mvc-test","application_uris":[""],"version":"1.0","name":"spring-boot-mvc-test","instance_id":"abcd","instance_index":0,"host":"0.0.0.0","port":61008} ENDOFVAR export VCAP_APPLICATION=$VCAP_APPLICATION read -r -d '' VCAP_SERVICES <<'ENDOFVAR' {"postgres":[{"name":"psgservice","label":"postgresql","tags":["postgresql"],"plan":"Standard","credentials":{"uri":"postgres://postgres:[email protected]:5432/hotelsdb"}]} ENDOFVAR export VCAP_SERVICES=$VCAP_SERVICES mvn spring-boot:run This entire sample is available at this github location:https://github.com/bijukunjummen/spring-boot-mvc-test Conclusion Spring Boot along with Spring-Cloud project now provide an excellent toolset to create Spring-powered cloud ready applications, and hopefully these notes are useful in integrating Spring Boot with Spring-Cloud and using these for seamless local and Cloud deployments.

Spring Integration (SI) is a framework enabling a collection of individual applications to integrate together to deliver a business enterprise system. The framework is essentially a lightweight messaging system that enables spring based applications to communicate with one another and supports integration with external systems via declarative adaptors. It is based on the 'filters and pipes' design architecture. A key feature of it is that it achieves this integration in a minimally intrusive way. The framework is built on 3 main components: Messages Encapsulate the data to be transferred from one place to another. They comprise of a header (holds meta data such as message-id, timestamp, etc) and a payload (your data typically in the form of a POJO). Channels Provide a mechanism to transport messages from one endpoint to another. Represents the pipes in the pipes & filters architecture. SI offers two types of channels, namely Pollable and Subscribable Channels. The former rely on consumers to periodically check for messages whereas the latter is directly responsible for notifying registered consumers when messages become available. Endpoints Consumer/Producer of messages. Performs some action based on the payload. Endpoints come in various flavours, each performing a different function. These include Transformers (transform data), Routers (route data), Filters (filter data), Splitter (splits messages), Aggregator (aggregates group of messages into single message), Service Activator (connecting messages to Services) and Channel Adapters (connect channels to external applications). The basic idea behind the SI framework is that applications communicate with each other by sending/receiving messages. These messages would typically contain the information (payload) required by the next application in the process pipeline. The transport of messages from one application to another is performed by Channel components. The Endpoints perform some action based on the payload. This could be routing the messages to another endpoint or processing the payload itself. The objective of this post is to provide an introduction to Spring Integration. To help achieve this, I developed a sample application which will be discussed below. The source for this sample application is available at here. The project was built and run using spring-integration-4.0.0, maven 3.2.1 and jdk1.6. The main dependency is for the relevant spring-integration jar as declared in the pom.xml: org.springframework.integration spring-integration-stream 4.0.0.RC1 I ran the application using the maven exec plugin. This allows me to clean, package and run the application by invoking mvn clean package exec:java -P OnlineShop from the command line. Developing a sample application: Tabernus My goal as usual was to build something very simple which would help me to become familiar with key concepts of this framework and to this end I've knocked up a simple app which does not connect up individual systems but rather invokes methods on a POJO. Extending this to actual working applications shouldn't be too difficult. The scenario I'm going to model revolves around purchasing items from an online store (Tabernus). This store only sells 3 types of items: Books, Music CDs, and software. During a Sale, the owners have decided to apply different discounts based on the item type. In this instance books, music and software benefit from discounts of 5%, 10%, and 15% respectively. The following diagram shows our domain entities. The class diagram shows that a Customer can place an Order comprising of a number of OrderItems which are of type Book, MusicCD or Software. The problem I need to solve is to design a system which can interrogate each Order and apply the correct discount based on the item type. Subsequently it should be able to compute the total cost of the order once the discounts have been applied. To model this using Spring Integration we need the following pipeline The above diagram shows various components most of which can be divided into 2 categories, channels (blue cylinder shapes) and endpoints (rectangular boxes). The exception to this is the Poller component whose purpose is to enable the various endpoints to function correctly and discussion of it will be given later. We'll start off by briefly covering the various stages in this pipeline as indicated by the numbers in red. Following this we will delve deeper into how we build this pipeline using the SI framework. The pipeline is comprised of 6 major stages as reflected by the numbers in the diagram, The Gateway component represents the entry point to the messaging system. All new Orders will be submitted to this component which will in turn wrap them as messages and place them into the channel appropriately named ordersChannel. Using the Splitter component - each Order is decomposed into a collection of it's constituent OrderItem instances. Each of these is wrapped in a Message and placed in the orderItemsChannel. The Router component considers each OrderItem in turn and places it in the relevant channel, e.g. Book items will be placed in the bookItemsChannel etc. This allows us to consider the different item types separately. The ServiceActivator needs to consider messages within each of the 3 channels and calculate the correct discount based on the channel. After completing the calculation for each OrderItem, it will place the OrderItem in the processedItemsChannel. The Aggregator component will collect all OrderItem instances placed in the processedItemsChannel and reconstruct the original Order. This will subsequently be placed in the deliveriesChannel, which represents the end of the pipeline. The Poller Component is required to configure how often the various endpoints will interrogate their respective input channels for messages. To implement the pipeline shown above using the SI framework, we need to implement the various end points. configure the pipeline in an xml file (Shop.xml) - identifying the various channels and endpoints and how they wire up together. At this point I should mention that SI offers 2 approach to configuring your process pipeline, annotations based and xml. In this article I'll be using the latter. Let's start to look at some code. We'll consider each stage described above and show the java implementation of the endpoint and xml configuration required to wire up the components. Step 1 - Gateway To begin with, we need to implement the Client that will invoke the Gateway component to place the Order. The client (OnlineShop.java) is shown below, public class OnlineShop { public static void main(String[] args) { AbstractApplicationContext context = new ClassPathXmlApplicationContext("/META-INF/com/prodcod/shop.xml", OnlineShop.class); Shop shop = (Shop) context.getBean("shop"); final Order order = createOrder(); shop.placeOrder(order); context.close(); } The logic here is quite simple. The client creates a dummy Order and passes this as an argument when it invokes the placeOrder() method on the gateway component. The gateway component referred here as Shop is injected by Spring. The Gateway component looks like: // Gateway component public interface Shop { @Gateway(requestChannel="ordersChannel") void placeOrder(Order order); } As you can see, this is simply an interface, whose implementation will be provided by Spring when it is injected into the client application. This is achieved by the use of the @Gateway annotation which informs Spring that this is a Gateway component and it needs to provide the implementation. Additionally the annotation accepts an attribute, requestChannel which defines the channel on which the Order instance will be placed. The framework does this by simply wrapping our instance of Order within a Message instance and placing it in the channel, 'ordersChannel'. The Gateway component and the 'ordersChannel' are declared as follows in the file shop.xml Step 2 - Splitter The next end point is the Splitter component. Appropriately named, it's role is to take a single message containing a payload of a collection of items and splitting it into a number of messages, each of which contains a single element from the collection. In our case, we want to decompose the Order into it's constituent OrderItem instances. It does this by taking a Message containing the payload of Order from 'ordersChannel' and then processing it before sending messages (each containing an OrderItem instance) to the 'orderItemsChannel'. Our implementation of the splitter is called OrderSplitter and is defined as below, public class OrderSplitter extends AbstractMessageSplitter{ @Override protected Object splitMessage(Message message) { return ((Order)message.getPayload()).getOrderItems(); } } Implementing a splitter is quite easy and involves extending the AbstractMessageSplitter class and overriding the splitMessage() method. This simply takes a message containing the payload of Order and returns it's collection of OrderItems. Step 3 - Router Having decomposed the Order into it's constituent OrderItems, we now need to separate them into groups of Books, MusicCD, and Software. This is achieved using a router. Our implementation of the Router looks like, public class OrderItemRouter { public String routeOrder(OrderItem orderItem) { String channel = ""; if(isBook(orderItem)) { channel = "bookItemsChannel"; } else if(isMusic(orderItem)) { channel = "musicItemsChannel"; } else if(isSoftware(orderItem)) { channel = "softwareItemsChannel"; } return channel; } ..................... ..................... } Nothing too complicated here. For each OrderItem, the method routeOrder() will determine it's item type and return the name of the channel that this message should be sent to. The channel name is returned by the method. Spring will then ensure that the message containing the OrderItem is relayed to the named channel. The configuration for OrderItemRouter looks like, The config identifies that the class OrderItemRouter is a Router component which will consume messages from the orderItemsChannel. Further Spring needs to invoke the method routeOrder() which contains the logic to perform the routing. The channels for each item type are declared as follows Step 4 - ServiceActivator The next step is to calculate the discounted price for each item type and this is performed by a ServiceActivator component. This is implemented as follows public class Shopkeeper { private static final BigDecimal BOOK_DISCOUNT = new BigDecimal(0.05); private static final BigDecimal MUSIC_DISCOUNT = new BigDecimal(0.10); private static final BigDecimal SOFTWARE_DISCOUNT = new BigDecimal(0.15); /** * Performs discount on books * @param bookOrderItem OrderItem comprising of a book item * @return OrderItem with discount price newly calculated */ public OrderItem processBooks(OrderItem bookOrderItem){ final BigDecimal finalPrice = calculateDiscountedPrice(bookOrderItem, BOOK_DISCOUNT); bookOrderItem.setDiscountedPrice(finalPrice); return bookOrderItem; } /** * Performs discount on music * @param musicOrderItem OrderItem comprising of a music item * @return OrderItem with discount price newly calculated */ public OrderItem processMusic(OrderItem musicOrderItem){ final BigDecimal finalPrice = calculateDiscountedPrice(musicOrderItem, MUSIC_DISCOUNT); musicOrderItem.setDiscountedPrice(finalPrice); return musicOrderItem; } /** * Performs discount on software * @param softwareOrderItem OrderItem comprising of a book item * @return OrderItem with discount price newly calculated */ public OrderItem processSoftware(OrderItem softwareOrderItem){ final BigDecimal finalPrice = calculateDiscountedPrice(softwareOrderItem, SOFTWARE_DISCOUNT); softwareOrderItem.setDiscountedPrice(finalPrice); return softwareOrderItem; } } This class exposes 3 methods to compute the new discounted price for each item type. Each method returns the OrderItem instance with the new price. The ServiceActivator is configured as follows: This tells Spring that the Shopkeeper class is a ServiceActivator and will consume messages from any of the 3 channels defined in the input-channel attribute. When a message appears in one of these channels, Spring will invoke the appropriate method on the ServiceActivator class as specfied by the attribute method. Anything returned from all three methods will be placed in the processedItems channel, ready for the next step of the processing pipeline. Step 5 - Aggregator The final stage is to take the individual OrderItems with their newly computed discounted prices and reconstruct the Order. This is achieved using an aggregator. Our implementation of an aggregator is listed below public class OrderCompleter { public Order prepareDelivery(List orderItems) { final Order order = new Order(); order.setOrderItems(orderItems); return order; } } The aggregator exposes a method that takes a collection of OrderItem objects. These will come from the processedItems channel declared as Recall this is the output channel for the service activator class as discussed above. The aggregator is configured in the xml file as The configuration tells Spring that the aggregator component will consume messages from the processedItems channel. These will be processed by the method prepareDelivery on the class OrderCompleter. Any output from this class will be relayed to the channel-adaptor deliveries, which is declared as The stdout-channel-adapter component writes to the systems STDOUT output stream. Step 6 - Poller To complete the setup we have to configure a poller component. This is required to enable the channels to work correctly. All our channels are of a queue type and so their respective consumers need to know when to query them. This is achieved using a poller mechanism. It is configured in the following way In this case, we have declared a global poller (as indicated by the default attribute). This will be used by the various end points to determine when they should interrogate their respective input-channels for messages. The second attribute fixed-delay is used to configure the polling interval. Running the Application Building and running the app shows the following output: The logging shows that the Customer submitted an Order for 3 items, one of each type. All items cost £100 each. The Order was then split into 3 OrderItems each of which was routed to the correct processing channel based on the item type. The ServiceActivator (Shopkeeper) then calculated the discount for each item and this was set on the OrderItem instance. The OrderItems were then aggregated using the OrderCompleter class which displays the final discounted price of £270 to be paid by the Customer. Note that the messages are logged to be in different stages of the processing pipeline despite starting off in the same order. This completes the tutorial on the Spring Integration Framework. Any comments relating to corrections, omissions, etc are welcome.

This is the third blog in C2B2 series looking at Glassfish 4. The previous two are available here: Part 1 - Getting started with Glassfish 4 Part 2 - Glassfish 4 - Features For High Availability In this blog I will look at 3 areas: Performance Tuning, where I will look at some of the areas to look at when setting up a system for production. Monitoring, where I will look at some of the tools we use for monitoring a system both during performance testing and tuning and once a system is up and running. Troubleshooting, where I will look at some of the tools you can use to help diagnose and detect performance issues. Performance Tuning Glassfish out of the box (as with most app servers) is optimised for development purposes. Developers want the ability to deploy and undeploy continuously, create and remove resources, debug, etc. However, this configuration is not suitable for a production system. When configuring any application server you have to take into account what you are trying to achieve and what is best suited for the applications you intend to run. One size does not fit all! It can be a long and complex process and I'm afraid I can't give you a one-stop solution. However, I can give you some pointers to some of the things you can do to prepare your system for production. So, what kind of things do we look at when we are looking to performance tune a Glassfish system. Some of the most common things are: JVM Settings Garbage Collection Glassfish Settings Logging JVM Settings The standard JVM defaults are not suitable for a production system. One of the simplest changes that can be made is to use the -server flag, rather than the default -client. Although the Server and Client VMs are similar, the Server VM has been specially tuned to maximise peak operating speed. It is intended for executing long-running server applications, which need the fastest possible operating speed more than a fast start-up time or smaller runtime memory footprint. Allocate more memory to the JVM by modifying the value of the -Xmx flag. How much depends on the size and complexity of your enterprise application and how much memory you have available. In addition we also want to make sure we allocate all of the memory on startup. This is done with the -Xms flag. We set the minimum and maximum perm gen to the same value in order to avoid allocation failures & subsequent full garbage collections. Garbage Collection There are a number of settings that can be tweaked regarding Garbage Collection. I'm not going to cover GC tuning as that is a whole topic all of it's own but here are some of the settings we would always recommend regarding GC in a production environment: Firstly we want to ensure we log all Garbage Collection information as this can prove extremely useful in diagnosing issues. -verbose:gc Next we want to make sure we log GC information to a file. This will make it easier to separate the GC from other details in the log files. -Xloggc:/path_to_log_file/gc.log We also want to ensure we have as much detail as possible. -XX:+PrintGCDetails and that the information is timestamped for easier diagnosis of long running errors and to be able to ascertain what normal levels are over time. -XX:+PrintGCDateStamps Finally, we want to ensure that developers aren't making explicit calls to System.gc(). Hopefully they don’t anyway and if they are you need to look into why (doing so is a bad idea since this forces major collections) but this will disable it just in case. -XX:+DisableExplicitGC Heap Dumps Heap dumps can be extremely useful for diagnosing memory issues. There are two settings we would definitely recommend. These tell the JVM to generate a heap dump when an allocation from the Java heap or the permanent generation cannot be satisfied. There is no overhead in running with these options but they can be useful for production systems where OutOfMemoryErrors can take a long time to surface. -XX:-HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/dumps/glassfish.hprof Configuring Glassfish There are three ways to configure Glassfish: Through the admin console By directly editing the config files Using the asadmin tool Although making changes through the admin console can often be the easiest way to make changes we’d recommend where possible to script all changes so you have a repeatable production server build. Also you should ensure copies of all config files are kept in Config Control so you know you have a working copy and can roll back to a previous version when needed. Turn off development features Turn off auto-deploy and dynamic application reloading. Both of these features are great for development, but can affect performance. Configure the JSP servlet not to check JSP files for changes on every request. Also, set the parameter genStrAsCharArray to true. This will ensure all String values are declared as static char arrays. One reason for this is that the array has less memory overhead than String. These changes will mean you cannot change JSP pages on your production server without redeploying the application, but on a production system this is generally what you want. Acceptor Threads and Request Threads There are two main thread values we would recommend setting, acceptor threads and request threads. Acceptor threads are used to accept new connections to the server and to schedule existing connections when a new request comes in. Set this value equal to the number of CPU cores in your server. So, if you have two quad core CPUs, this value should be set to eight. Request threads run HTTP requests. You want enough of these to keep the machine busy, but not so many that they compete for CPU resources which would cause your throughput to suffer greatly. Static resources By default, GlassFish does not tell the client to cache static resources. It is recommended to cache static resources, like CSS files and images particularly if you have a lot of them. Thread pools Max thread pool and min pool size should be set to the same value. Specifying the same value will allow GlassFish to use a slightly more optimised thread pool. This configuration should be considered unless the load on the server varies significantly. Increasing this value will reduce HTTP response latency times. What to set these values to depends heavily on what your application is doing. In order to get this value right you should look to incrementally increase the thread count and to monitor performance after each incremental increase. When performance stops improving stop increasing the thread count. Logging You should look to turn off as much logging as possible. In a production environment we would generally recommend logging at WARN and above. This includes the logging done by Glassfish as well as your own applications. Monitoring The fewer monitoring options that are enabled, the better the server's performance. All Glassfish monitoring is turned off by default. Switching monitoring on can be very useful when diagnosing issues and when doing initial system testing and performance tuning for monitoring what changes. What to monitor Used Heap Size - Compare this number with the maximum allowed heap size to see what portion of the heap is in use. If the used heap size nears the max heap size, the garbage collector urgently attempts to free memory and this is something that should be avoided where possible. Number of loaded classes - Useful for detecting performance and application development trends. JVM Threads - Important for performance tuning and for troubleshooting JVM crashes. Some of the most essential indicators are the current active JVM thread count and the peak values. Thread pools - You should compare a pools current usage with the maximum number allowed. Problems can start to occur when the current count nears the max threads number. JVM Tools for Monitoring The following is a list of a a few of the tools that come with the JDK that are useful for monitoring information from the JVM. jstat - This tool displays performance statistics regarding usage of the perm gen, new gen and old gen. It also provides class loading and compilation statistics jmap - Gives you visibility of memory usage, can produce a class histogram and can dump the memory to a file jconsole/jvisualvm - These tools can display all the previously mentioned monitoring indicators and graph them over time. This allows you to spot trends and to get a better overall picture of your normal performance levels and changes over time. Note - These should NOT be left running permanently on a production system! Troubleshooting Unfortunately, no matter how much tuning and testing you do all systems WILL go wrong from time to time. So, what should you do when your production server bursts into flames? Well, in that situation you should call the fire service but for more general problems: Gather data - get as much data as you can, there is no such thing as too much! Analyse that data - Data is worthless when you don’t know what it means. Visualise where possible – graphs and charts reveal trends and patterns over time Make educated decisions - Only make decisions based on data. If you go with your “gut instinct” and what “feels right” you will probably make things worse Gathering data First up, for most of the JVM tools you will need the process ID of the server. You can get this information in various ways. Two of the simplest are: jps -v This will list all current running Java processes. The -v flag is for verbose output. ps aux | grep glassfish The ps command with the options aux will show all processes from all users. This will display a LOT of information so pipe it through grep to filter for the glassfish process As mentioned earlier the jstat tool can be used for gathering info on JVM performance. Other useful tools include: jstack This will produce thread stack dumps for all threads running in the JVM. This can be very useful for discovering stuck threads or long running threads. jmap This tool can be used to create a heap dump. It outputs to a file in .hprof format which can be read by a number of analysis tools jrcmd and jrmc These tools are only available with the jRockit JDK. I won't go into any detail here as I have previously blogged about jrcmd here: http://blog.c2b2.co.uk/2012/11/troubleshooting-jrockit-using-jrcmd.html and my colleague has blogged about jrmc here: http://blog.c2b2.co.uk/2012/10/weblogic-troubleshooting-with-jrockit.html Glassfish asadmin The Glassfish asadmin tool has a built in command which will provide similar functionality to the above tools but without the need for the PID. asadmin generate-jvm-report --type=[type] Analysing the data There are various tools available for analysing performance data. The following are some of the most useful: IBM Support Assistant is a free troubleshooting application that helps you research, analyze, and resolve problems using various support features and tools. It contains a Garbage Collection and Memory Visualiser as well as a Heap Analyser. It will also provide a report telling you where issues might exist, and listing red flags with advice on what to change in your applications jRockit Mission Control is a very powerful tool which can be used to monitor live systems or analyse historical data in the form of flight recordings. JVisualVM GCViewer is an optional plugin for jVisualVM which can transform a tool which is already great for live monitoring into a powerful analysis tool jhat is a Java Heap Analysis Tool. It processes heap dump files and produces HTML reports. There are better analysis tools, but it’s always freely available if you’re running a JDK. Others There are many open source and freely available tools and projects to help you, here we’ve covered some very common and widely used ones, but our list is by no means exhaustive! Conclusion Remember, Glassfish out of the box (or out of the zip file!) is not designed to be run 'as is'. You should also note that there is no ideal configuration that will work for all systems. It will take time and effort to get the best configuration for what you require. Hopefully in this blog I have given you some useful guidelines and pointers. You should take time to work out what you want in terms of services, then strip back your config to match that. You should test, test and test again to ensure that your configuration matches the requirements with regards to the applications you will be running on your server. You should tune your JVM to ensure you have the best settings for your particular configuration. You should ensure you have monitoring in place to keep a check on everything and ensure that if your server does crash you have as much information as possible at hand to diagnose what caused it. The next blog in this series looks at Migrating to Glassfish 4: http://blog.c2b2.co.uk/2013/07/glassfish-4-migrating-to-glassfish.html

a powerful multi-format xml schema (xsd) documentation generator and a tool for rapid development of custom xsd documentation generators according to user needs. about docflex/xml "xsddoc" template set template processor template designer integrations generation of xsd diagrams apache ant & maven links about docflex/xml docflex/xml is a java-based software system for development and execution of high performance template-driven documentation generators from any data stored in xml files. the actual doc/report generators are programmed in the form of special templates using a graphic template designer , which represents the templates visually in a form resembling the output they generate. further, the templates are interpreted by a template processor , which takes on input the xml files and produces by them the result documentation. this article describes an application of docflex/xml for the task of generation of high-quality xml schema documentation. that includes the following features of docflex/xml system: " xsddoc " template set that implements the ready-to-use xml schema documentation generator itself. template processor makes the templates works. currently, it provides three interchangeable output generators for html, rtf, txt (plain text) formats. template designer provides a high quality gui to design/modify templates. if you need a special xml schema doc generator, the simplest way to create it is to modify the standard xsddoc templates. the template designer enables you to do that. integrations with altova xmlspy and oxygen xml editor . if you are a user of one of those popular xml editors, you can turn it also into a dynamically linked diagramming engine for docflex, so that to include automatically the xsd diagrams generated by xmlspy/oxygenxml into the xml schema documentation generated by docflex (with the full support of hyperlinks). "xsddoc" template set it is the implementation of xml schema documentation itself, which provides the following functionality: generation of single documentation by any number of xml schema (xsd) files together, in particular: highly navigable framed (javadoc-like) html documentation single-file html documentation rtf documentation (further convertible to pdf) processing of any referenced xml schemas, in particular: correct processing of all , , elements found across all involved xsd files. automatic loading and processing (i.e. inclusion in the documentation scope) all directly/indirectly referenced xsd files. sophisticated documenting of xsd components , including: component diagrams (with hyperlinks to everything depicted on them; see also integrations ) xml representation summary (a textual alternative to diagrams) lists of related components. for elements this includes also the list of possible containing elements . (such a list is never present in the output generated by xslt-based doc generators). list of usage locations support of any xml schema design patterns . this comes down mainly to the following: special treatment of local elements (see below) support and documenting of substitution groups support of importing, inclusion and redefinition of schema files special documenting of local elements . local elements are those components that are declared locally within other xsd components. w3c xml schema spec allows you to declare any number of local elements that may share the same name but have different content. that's because their meaning is local and there will be no collisions with other declarations. that, however, creates a problem for documenting, because in a documentation both global and local elements may appear simultaneously in various lists according to their common properties. if each element component is identified only by its name, you will get the lists with multiple repeating names but little clue what they mean. moreover, some xml schemas may contain lots of identical local element declarations (that is, they have the same both name and content). so, you'll get in those lists a mess of repeating names, some of which referencing to effectively the same entities, whereas others to complete different ones. in xsddoc , those problems are solved in two ways: adding extensions to local element names. the extension provides more information about the element (e.g. where it can be inserted or its global type or where it is defined). that makes the whole string identifying the element unique. here is how it looks. the grey text is the name extension: unifying local elements by type. on the left you can see a documentation generated with such unification. on the right, all local elements are documented straight as they are. click on each screenshot to view the docs: we believe the first documentation (on the left) is easier to understand and use. processing of xhtml markup . you can format your xml schema annotations with xhtml tags, which will be recognized and rendered with the appropriate formatting in both html and rtf output, as shown on the following screenshots (click to see more details): here, on the left you can see the xml source of an xml schema, whose annotations are heavily laden with xhtml markup (including insertion of images). the next is the html documentation generated by that schema. on the right is a page of rtf documentation also generated by that schema. possibility of unlimited customization : xsddoc is controlled by more than 400 parameters, which allow you to adjust the generated documentation within huge range of included details. template parameters serve the same role as options in traditional doc generators. the difference is that docflex template architecture makes the support/implementation of template parameters very cheap (typically, the most of efforts takes writing their descriptions). so, there may be hundreds of parameters controlling a large template application. if parameters are not enough, you can modify the templates themselves using the template designer . in case of html output, you can also apply your own css styles to change how the generated documentation looks. template processor the template processor (also called simply "generator") makes everything work. it consists of two logical parts: 1. template interpreter 2. output generator the output generator actually has three different implementations for each currently supported output format: html, rtf, txt (plain text). the plain-text output can be used to generate documentation in formats not supported directly by docflex. the template processor is started directly from java command line with the following arguments: ● main template ● template parameters ● initial xsd files to be processed (documented) ● xml catalogs (to redirect physical location of input files) ● destination directory/file ● output format (this selects which output generator will be used) ● output format options (specify settings to control the selected output generator) actually, the number of settings may be so large that the template processor provides a special gui to specify everything interactively (click to enlarge): template designer although docflex templates are stored as plain-text files (with an xml-like format), they are not supposed for editing manually. rather, a special graphic template designer must be used, which visualizes the templates in the form of template components they are made of. those components are the actual constructs of the template language (not some textual statements, operators, blocks etc.) the following screenshots show templates open in the template designer (click to see a lot more): that approach has a number of advantages, among them: the processing structures represented by template components may be displayed in a way that visually expresses what a component does (for instance, it may resemble the output it generates). that representation may be both expressive and compact (after all, it is not just a text), which allows you easily to navigate a template, understand what it does and modify anything you need. as template components are visual and interactive, they may have very complex internal structure, for instance, contain lots of properties and nested components. at that, you don't need to scroll and navigate some kind of enormous text, which encodes all of this (as it would be in case of a script). rather, you just need to invoke some property dialogs and expand/collapse some component sections. a template component may be easily copied, pasted and deleted as a whole. at that, you don't need to bother that the template syntax is restored after that. the template designer will also ensure that each component is created, copied or moved only in the allowed place. the highly structured nature of templates eliminates the need for most of various named identifiers. many connections between different template components are also maintained by the template designer (i.e. modified automatically when necessary). as template files are stored and read only programmatically, there is no need to know and understand their syntax. there will be no syntax errors either. the actual syntax of template files may be optimized not for human programmers, but for faster loading and processing of templates by the template processor . there is no need in a compilation phase. the separation of template semantics from the particular structure of template files helps for faster and easier evolution of the template language. the obsolete constructs of older template versions can be automatically converted into new structures. both old and new templates will look and work up-to-date. integrations generation of xsd diagrams docflex/xml is able to work with any kind of diagrams (i.e. inserting them automatically in the generated output). that is supported on the level of templates, along with the generation of hypertext imagemaps, as shown on the following screenshot (click to see a lot more): docflex/xml provides no diagramming engine of its own. instead, it includes integrations with two most popular xml editors that do generate xsd diagrams: ● altova xmlspy ● oxygen xml editor effectively, the third-party software is used as dynamically linked diagramming engine. the advantage of such integrations is that when you are the user of one of those xml editors, you will get in the documentation generated by docflex the same diagrams as you see in your xml editor. here is how such a documentation with diagrams looks (click on a screenshot to view the real html): apache ant & maven as a pure java application, docflex/xml can be run in any environment that runs java itself. the template processor can be easily integrated with ant (that can be specified just in the ant build file). in case of maven, docflex/xml includes a simple maven plugin. it is possible also to use all diagraming integrations with both ant and maven. links docflex/xml (home page): http://www.filigris.com/docflex-xml/ docflex/xml xsddoc: http://www.filigris.com/docflex-xml/xsddoc/ xsddoc examples: http://www.filigris.com/docflex-xml/xsddoc/examples/ xmlspy integration: http://www.filigris.com/docflex-xml/xmlspy/ oxygenxml integration: http://www.filigris.com/docflex-xml/oxygenxml/ free downloads: http://www.filigris.com/downloads/ this original article: http://www.filigris.com/ann/docflex-xsd/

This is a sample that I have wanted to try for sometime - A Websocket application to tail the contents of a file. The following is the final view of the web-application: There are a few parts to this application: Generating a File to tail: I chose to use a set of 100 random quotes as a source of the file content, every few seconds the application generates a quote and writes this quote to the temporary file. Spring Integration is used for wiring this flow for writing the contents to the file: Just a quick note, Spring Integration flows can now also be written using a Java Based DSL, and this flow using Java is available here Tailing the file and sending the content to a broker The actual tailing of the file itself can be accomplished by OS specific tail command or by using a library like Apache Commons IO. Again in my case I decided to use Spring Integration which provides Inbound channel adapters to tail a file purely using configuration, this flow looks like this: and its working Java equivalent There is a reference to a "fileContentRecordingService" above, this is the component which will direct the lines of the file to a place where the Websocket client will subscribe to. Websocket server configuration Spring Websocket support makes it super simple to write a Websocket based application, in this instance the entire working configuration is the following: @Configuration @EnableWebSocketMessageBroker public class WebSocketDefaultConfig extends AbstractWebSocketMessageBrokerConfigurer { @Override public void configureMessageBroker(MessageBrokerRegistry config) { //config.enableStompBrokerRelay("/topic/", "/queue/"); config.enableSimpleBroker("/topic/", "/queue/"); config.setApplicationDestinationPrefixes("/app"); } @Override public void registerStompEndpoints(StompEndpointRegistry registry) { registry.addEndpoint("/tailfilesep").withSockJS(); } } This may seem a little over the top, but what these few lines of configuration does is very powerful and the configuration can be better understood by going through the reference here. In brief, it sets up a websocket endpoint at '/tailfileep' uri, this endpoint is enhanced with SockJS support, Stomp is used as a sub-protocol, endpoints `/topic` and `/queue` is configured to a real broker like RabbitMQ or ActiveMQ but in this specific to an in-memory one. Going back to the "fileContentRecordingService" once more, this component essentially takes the line of the file and sends it this in-memory broker, SimpMessagingTemplate facilitates this wiring: public class FileContentRecordingService { @Autowired private SimpMessagingTemplate simpMessagingTemplate; public void sendLinesToTopic(String line) { this.simpMessagingTemplate.convertAndSend("/topic/tailfiles", line); } } Websocket UI configuration The UI is angularjs based, the client controller is set up this way and internally uses the javascript libraries for sockjs and stomp support: var tailFilesApp = angular.module("tailFilesApp",[]); tailFilesApp.controller("TailFilesCtrl", function ($scope) { function init() { $scope.buffer = new CircularBuffer(20); } $scope.initSockets = function() { $scope.socket={}; $scope.socket.client = new SockJS("/tailfilesep); $scope.socket.stomp = Stomp.over($scope.socket.client); $scope.socket.stomp.connect({}, function() { $scope.socket.stomp.subscribe("/topic/tailfiles", $scope.notify); }); $scope.socket.client.onclose = $scope.reconnect; }; $scope.notify = function(message) { $scope.$apply(function() { $scope.buffer.add(angular.fromJson(message.body)); }); }; $scope.reconnect = function() { setTimeout($scope.initSockets, 10000); }; init(); $scope.initSockets(); }); The meat of this code is the "notify" function which the callback acting on the messages from the server, in this instance the new lines coming into the file and showing it in a textarea. This wraps up the entire application to tail a file. A complete working sample without any external dependencies is available at this github location, instructions to start it up is also available at that location. Conclusion Spring Websockets provides a concise way to create Websocket based applications, this sample provides a good demonstration of this support. I had presented on this topic recently at my local JUG (IndyJUG) and a deck with the presentation is available here

In this post, I will show how to integrate Apache Tiles 3 with Spring MVC.

In an earlier blog entry I had touched on a fictitious rube goldberg flow for capitalizing a string through a complicated series of steps, the premise of the article was to introduce Spring Integration Java DSL as an alternative to defining integration flows through xml configuration files. I learned a few new things after writing that blog entry, thanks to Artem Bilan and wanted to document those learnings here: So, first my original sample, here I have the following flow(the one's in bold): Take in a message of this type - "hello from spring integ" Split it up into individual words(hello, from, spring, integ) Send each word to a ActiveMQ queue Pick up the word fragments from the queue and capitalize each word Place the response back into a response queue Pick up the message, re-sequence based on the original sequence of the words Aggregate back into a sentence("HELLO FROM SPRING INTEG") and Return the sentence back to the calling application. EchoFlowOutbound.java: @Bean public DirectChannel sequenceChannel() { return new DirectChannel(); } @Bean public DirectChannel requestChannel() { return new DirectChannel(); } @Bean public IntegrationFlow toOutboundQueueFlow() { return IntegrationFlows.from(requestChannel()) .split(s -> s.applySequence(true).get().getT2().setDelimiters("\\s")) .handle(jmsOutboundGateway()) .get(); } @Bean public IntegrationFlow flowOnReturnOfMessage() { return IntegrationFlows.from(sequenceChannel()) .resequence() .aggregate(aggregate -> aggregate.outputProcessor(g -> Joiner.on(" ").join(g.getMessages() .stream() .map(m -> (String) m.getPayload()).collect(toList()))) , null) .get(); } @Bean public JmsOutboundGateway jmsOutboundGateway() { JmsOutboundGateway jmsOutboundGateway = new JmsOutboundGateway(); jmsOutboundGateway.setConnectionFactory(this.connectionFactory); jmsOutboundGateway.setRequestDestinationName("amq.outbound"); jmsOutboundGateway.setReplyChannel(sequenceChannel()); return jmsOutboundGateway; } It turns out, based on Artem Bilan's feedback, that a few things can be optimized here. First notice how I have explicitly defined two direct channels, "requestChannel" for starting the flow that takes in the string message and the "sequenceChannel" to handle the message once it returns back from the jms message queue, these can actually be totally removed and the flow made a little more concise this way: @Bean public IntegrationFlow toOutboundQueueFlow() { return IntegrationFlows.from("requestChannel") .split(s -> s.applySequence(true).get().getT2().setDelimiters("\\s")) .handle(jmsOutboundGateway()) .resequence() .aggregate(aggregate -> aggregate.outputProcessor(g -> Joiner.on(" ").join(g.getMessages() .stream() .map(m -> (String) m.getPayload()).collect(toList()))) , null) .get(); } @Bean public JmsOutboundGateway jmsOutboundGateway() { JmsOutboundGateway jmsOutboundGateway = new JmsOutboundGateway(); jmsOutboundGateway.setConnectionFactory(this.connectionFactory); jmsOutboundGateway.setRequestDestinationName("amq.outbound"); return jmsOutboundGateway; } "requestChannel" is now being implicitly created just by declaring a name for it. The sequence channel is more interesting, quoting Artem Bilan - do not specify outputChannel for AbstractReplyProducingMessageHandler and rely on DSL , what it means is that here jmsOutboundGateway is a AbstractReplyProducingMessageHandler and its reply channel is implicitly derived by the DSL. Further, two methods which were earlier handling the flows for sending out the message to the queue and then continuing once the message is back, is collapsed into one. And IMHO it does read a little better because of this change. The second good change and the topic of this article is the introduction of the Jms namespace factories, when I had written the previous blog article, DSL had support for defining the AMQ inbound/outbound adapter/gateway, now there is support for Jms based inbound/adapter adapter/gateways also, this simplifies the flow even further, the flow now looks like this: @Bean public IntegrationFlow toOutboundQueueFlow() { return IntegrationFlows.from("requestChannel") .split(s -> s.applySequence(true).get().getT2().setDelimiters("\\s")) .handle(Jms.outboundGateway(connectionFactory) .requestDestination("amq.outbound")) .resequence() .aggregate(aggregate -> aggregate.outputProcessor(g -> Joiner.on(" ").join(g.getMessages() .stream() .map(m -> (String) m.getPayload()).collect(toList()))) , null) .get(); } The inbound Jms part of the flow also simplifies to the following: @Bean public IntegrationFlow inboundFlow() { return IntegrationFlows.from(Jms.inboundGateway(connectionFactory) .destination("amq.outbound")) .transform((String s) -> s.toUpperCase()) .get(); } Thus, to conclude, Spring Integration Java DSL is an exciting new way to concisely configure Spring Integration flows. It is already very impressive in how it simplifies the readability of flows, the introduction of the Jms namespace factories takes it even further for JMS based flows.

We had a lengthy discussion recently during code review whether scala.Option.fold() is idiomatic and clever or maybe unreadable and tricky? Let's first describe what the problem is. Option.fold does two things: maps a function f overOption's value (if any) or returns an alternative alt if it's absent. Using simple pattern matching we can implement it as follows: val option: Option[T] = //... def alt: R = //... def f(in: T): R = //... val x: R = option match { case Some(v) => f(v) case None => alt } If you prefer one-liner, fold is actually a combination of map and getOrElse val x: R = option map f getOrElse alt Or, if you are a C programmer that still wants to write in C, but using Scala compiler: val x: R = if (option.isDefined) f(option.get) else alt Interestingly this is similar to how fold() is actually implemented, but that's an implementation detail. OK, all of the above can be replaced with single Option.fold(): val x: R = option.fold(alt)(f) Technically you can even use /: and \: operators (alt /: option) - but that would be simply masochistic. I have three problems with option.fold() idiom. First of all - it's anything but readable. We are folding (reducing) over Option - which doesn't really make much sense. Secondly it reverses the ordinary positive-then-negative-case flow by starting with failure (absence, alt) condition followed by presence block (f function; see also: Refactoring map-getOrElse to fold). Interestingly this method would work great for me if it was named mapOrElse: ** * Hypothetical in Option */ def mapOrElse[B](f: A => B, alt: => B): B = this map f getOrElse alt Actually there is already such method in Scalaz, called OptionW.cata. cata. Here is whatMartin Odersky has to say about it: "I personally find methods like cata that take two closures as arguments are often overdoing it. Do you really gain in readability over map + getOrElse? Think of a newcomer to your code[...]"While cata has some theoretical background, Option.fold just sounds like a random name collision that doesn't bring anything to the table, apart from confusion. I know what you'll say, that TraversableOnce has fold and we are sort-of doing the same thing. Why it's a random collision rather than extending the contract described inTraversableOnce? fold() method in Scala collections typically just delegates to one offoldLeft()/foldRight() (the one that works better for given data structure), thus it doesn't guarantee order and folding function has to be associative. But inOption.fold() the contract is different: folding function takes just one parameter rather than two. If you read my previous article about folds you know that reducing function always takes two parameters: current element and accumulated value (initial value during first iteration). But Option.fold() takes just one parameter: current Option value! This breaks the consistency, especially when realizing Option.foldLeft() andOption.foldRight() have correct contract (but it doesn't mean they are more readable). The only way to understand folding over option is to imagine Option as a sequence with0 or 1 elements. Then it sort of makes sense, right? No. def double(x: Int) = x * 2 Some(21).fold(-1)(double) //OK: 42 None.fold(-1)(double) //OK: -1 but: Some(21).toList.fold(-1)(double) : error: type mismatch; found : Int => Int required: (Int, Int) => Int Some(21).toList.fold(-1)(double) ^ If we treat Option[T] as a List[T], awkward Option.fold() breaks because it has different type than TraversableOnce.fold(). This is my biggest concern. I can't understand why folding wasn't defined in terms of the type system (trait?) and implemented strictly. As an example take a look at: Data.Foldable in Haskell (advanced) Data.Foldable typeclass describes various flavours of folding in Haskell. There are familiar foldl/foldr/foldl1/foldr1, in Scala namedfoldLeft/foldRight/reduceLeft/reduceRight accordingly. They have the same type as Scala and behave unsurprisingly with all types that you can fold over, including Maybe, lists, arrays, etc. There is also a function named fold, but it has a completely different meaning: class Foldable t where fold :: Monoid m => t m -> m While other folds are quite complex, this one barely takes a foldable container of ms (which have to be Monoids) and returns the same Monoid type. A quick recap: a type can be aMonoid if there exists a neutral value of that type and an operation that takes two values and produces just one. Applying that function with one of the arguments being neutral value yields the other argument. String ([Char]) is a good example with empty string being neutral value (mempty) and string concatenation being such operation (mappend). Notice that there are two different ways you can construct monoids for numbers: under addition with neutral value being 0 (x + 0 == 0 + x == x for any x) and under multiplication with neutral 1 (x * 1 == 1 * x == x for any x). Let's stick to strings. If I fold empty list of strings, I'll get an empty string. But when a list contains many elements, they are being concatenated: > fold ([] :: [String]) "" > fold [] :: String "" > fold ["foo", "bar"] "foobar" In the first example we have to explicitly say what is the type of empty list []. Otherwise Haskell compiler can't figure out what is the type of elements in a list, thus which monoid instance to choose. In second example we declare that whatever is returned from fold [], it should be a String. From that the compiler infers that [] actually must have a type of [String]. Last fold is the simplest: the program folds over elements in list and concatenates them because concatenation is the operation defined in Monoid Stringtypeclass instance. Back to options (or more precisely Maybe). Folding over Maybe monad having type parameter being Monoid (I can't believe I just said it) has an interesting interpretation: it either returns value inside Maybe or a default Monoid value: > fold (Just "abc") "abc" > fold Nothing :: String "" Just "abc" is same as Some("abc") in Scala. You can see here that if Maybe Stringis Nothing, neutral String monoid value is returned, that is an empty string. Summary Haskell shows that folding (also over Maybe) can be at least consistent. In ScalaOption.fold is unrelated to List.fold, confusing and unreadable. I advise avoiding it and staying with slightly more verbose map/getOrElse transformations or pattern matching. PS: Did I mention there is also Either.fold() (with even different contract) but noTry.fold()?

We have published the results of the Eclipse Community Survey 2014. Thank you to everyone who participated in the survey this year. The complete results and data are available for anyone to download [xls] [ods]. As in other years, I think the results provide an interesting perspective on what tools software developers are using and the type of applications they are building. Here are some key highlights from the results this year: 1) Git #1 Code Management Tool. Git has finally surpassed Subversion to be the top code management tool used by software developers. A third of developers (33.3%) report they use Git as their primary code management tool compared to 30.7% using Subversion. Subversion continues to show a downward trend from previous years when it was used by more than half the developers. Of note, 9.6% claim GitHub is their primary code management tool so the prevalence of overall Git usage is becoming dominate. 2) Maven and Jenkins Key Tools. For Build and Release tools, Maven and Jenkins continue to be key tools used by developers. Of interest is the growth of Gradle from 2013 (4.5%) to 2014 (11%). 3) Top 3 Application Servers. Tomcat (32.6%), JBoss (11.8%) and Jetty (7.2%) continue to be the top 3 application servers. 4) Java 8 Adoption. Java 8 was released in March 2014 and already 9.2% of Java developers have migrated to Java 8 as their primary version of Java. 59.2% are using Java 7 but close to a quarter are using Java 6 or before. 5) Majority of Developers Use JavaScript. More and more software developers use multiple languages to develop software. Due to the Eclipse biased of the survey, Java is not surprisingly the top primary language. However, when asked what other languages developers might use, JavaScript stands out to be a popular language with a the majority of developers (56.2%) claiming it as a secondary language. 6) Developers Experimenting With Open Hardware. The Internet of Things (IoT) and Open Hardware have become important industry trends in the last couple of years. Over a third (35.7%) of software developers are spending their own personal time learning about devices like the BeagleBone, Arduino and Raspberry Pi. Thanks again to everyone who participated in the survey. I hope everyone finds the results of interest.

This post exists to help with an MSDN Magazine article that I am authoring It provides some of the low-level details for the article How to install Mono and root certificates on a raspberry pi How to create an Azure mobile service How to create a Custom API inside Azure mobile services that the raspberry pi can call into How to create an Azure storage account MONO - HOW TO INSTALL ON A RASPBERRY PI Why Mono? How to install Mono on a raspberry pi Installing trusted root certificates on to the raspberry pi http://www.mono-project.com/Main_Page An open source, cross-platform, implementation of C# and the CLR that is binary compatible with Microsoft.NET Mono is a free and open source project led by Xamarin (formerly by Novell) that provides a .NET Framework-compatible set of tools including, among others, a C# compiler and a Common Language Runtime WHY MONO? Because it lets us write .net code compiled on Windows We can simply copy the binary files from Windows to Linux and run it as is From a raspberry pi device, it is possible to use a .net application to take a photo and upload it to Windows Azure storage HOW TO INSTALL ON A RASPBERRY PI RUNNING LINUX You will issue the following commands: pi@raspberrypi ~ $ sudo apt-get update pi@raspberrypi ~ $ sudo apt-get install mono-complete The first command makes sure all the local package index are up to date with the changes made in repositories. Second command installs the complete Mono tooling and runtime. MAKING SURE THAT YOUR MONO APPLICATIONS CAN MAKE A HTTPS REST-BASED CALLS This command downloads the trusted root certificates from the Mozilla LXR web site into the Mono certificate store. Once complete, the Raspberry PI will be capable of making web requests using HTTPS requests within Mono. pi@raspberrypi ~ $ mozroots --import --ask-remove --machine CREATING A NEW AZURE MOBILE SERVICES ACCOUNT The mobile services account is needed to host a Node.js application that provides shared access signatures to raspberry pi devices The shared access signature is needed by the raspberry pi, so that it can directly and securely upload photos to Azure storage STEPS TO CREATE AN AZURE MOBILE SERVICE The steps below will create an Azure mobile service The service will be used to host a Node.js application interacting with a raspberry pi devices We will provision a SQL database, although it will not be used initially FOLLOW THESE STEPS TO CREATE THE MOBILE SERVICE Login into the Azure Portal Select MOBILE SERVICES from the left menu pane at the Azure Portal. In the lower left corner select "+NEW" to create a new Azure Mobile Service. Make sure you've selected, "COMPUTE / MOBILE SERVICE / CREATE." You will now enter a url. We will call this service raspberrymobileservice. For the DATABASE, we will choose "Create a new SQL database instance." The REGION we chose is "West US." The BACKEND is "JavaScript." Click the "->" arrow to proceed to the next screen. In this screen you will "Specify database settings." The NAME of your database will based on the URL you entered previously. In this case, the database is called "raspberrymobileservice_db." You will need to choose a SERVER. We will choose "New SQL database server" from the drop-down list. You will need to provide a SERVER LOGIN NAME and a SERVER LOGIN PASSWORD. Take note of the login you provided as it will be needed later CREATING A CUSTOM API Azure mobile services allows you to create a custom API written in JavaScript that can be called from a raspberry pi device using REST This custom API is really just a Node.js application running in the server CREATING THE API TO RESPOND TO THE DEVICE TRYING TO UPLOAD PHOTOS Now that the service is established, we will turn our attention to creating an API that the device can call into to upload a photo. Login into the Azure Portal Your mobile service will take a few minutes to complete, and you should see the "Ready" flag as the "Status" for your service. Once it is ready you can drill into your service to customize its behavior. Just to the right of the service name, click the right arrow key "->" to drill into the service details. The top menu bar will offer many options, but we are interested in the one titled "API." The API allows you to create a series of node.JS API calls that a device can call into using rest-based approaches. Click on "API." from there, select "CREATE A CUSTOM API." You will be asked to provide an API name. Type in "photos" for the API name. Below you will see a series of drop-down combo boxes that relate to permission. We will keep the default value of "Anybody with the application key." This might not be the best option for all scenarios. You can read more about this here. http://msdn.microsoft.com/en-us/library/azure/jj193161.aspx. Click the checkmark to complete the process. The name of the AP you just created, "Photos," should be visible on the portal interface. To drill into the photos API click on the right arrow key "->". The right arrow key will be just to the right of the name of the API "Photos". At this point you should see a basic script that has been provided by default. We will overwrite this default script with our own script as described in the MSDN Magazine article. CREATING A STORAGE ACCOUNT TO STORE THE PHOTOS Navigate to the portal and create a storage account Create a container for the photos Obtain the: Storage Account Name (you will provide a name) Storage Account Access key (generated for you) Container Name (you will create) CREATING A STORAGE ACCOUNT We will need a storage account so that we can upload photos to it. The steps are well documented here: http://azure.microsoft.com/en-us/documentation/articles/storage-create-storage-account/ In our case we call the storage account raspberrystorage. This means that the URL that the device will use to upload photos is https://raspberrystorage.blob.core.windows.net/. As you complete these steps make sure that you choose the storage account location to be the same location as was used for your mobile services account. This avoids any unnecessary latency or bandwidth costs between data centers. Once the storage account is created, we will need to create a container within it. Photos or any blob for that matter, are always stored within a container. To create a container drill into your newly created storage account and select CONTAINERS from the top menu. From there, select CREATE A CONTAINER. The new container dialog box will ask for a name for your container. Take note of the name you provide. We are calling our container ?photocontainer.? When the raspberry pi device uploads photos to the storage account, it will target a specific container, such as the one we just created. You will next be asked to indicate ACCESS rights. To keep things simple we will select access rights of Public Blob. ENTERING APP SETTINGS Rather than hard-code storage account information inside your JavaScript/Node.js applications, you should consider using apps settings inside of the Azure mobile services portal This post also discusses it well: http://blogs.msdn.com/b/carlosfigueira/archive/2013/12/09/application-settings-in-azure-mobile-services.aspx ?The idea of application settings is a set of key-value pairs which can be set for the mobile service (either via the portal or via the command-line interface), and those values could be then read in the service runtime.? NAVIGATING TO APP SETTINGS Navigate to the Azure Mobile Services section of the portal. Drill into the specific service by hitting the arrow below Select from the Configure Menu at the top Scroll down to the very bottom to see app settings Note that we need to enter: - We need to get this from Azure Storage - PhotoContainerName - AccountName - AccountKey We get this information from the Azure Storage Section of the Portal. Note that you need to have provisioned a Storage Account to have this information. How to get the AccountKey with Azure Storage Services Now you can get the access keys HOW NODE.JS WILL ACCESS THE APP SETTINGS You will create a Node.js application inside of Azure Mobile Services See previous steps THE NODE.JS APPLICATION READING APP SETTINGS You will starting by going back to Azure Mobile Services and drill down into your newly minted service We called ours raspberrymobileservice Once you click API, you should see: Notice the app settings are being read on lines 12 to 14.