DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Newsletter
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Latest Articles - DZone

article thumbnail
Installing JDK 7 on Mac OS X
To get JDK 7 up, I downloaded the JDK from Oracle. They have a nice dmg file, which makes it easy to install. After reading their installation instructions and running /usr/libexec/java_home (which I didn't even know about), it still wasn't defaulting to JDK 7. Surgery required. So, I headed over to: /System/Library/Frameworks/JavaVM.framework/Versions This is where the system jvm's are stored. You'll notice a symbolic link for CurrentJDK. It probably points to: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents You're going to want to point that to the new JDK, which java_home tells us is located in: bone@zen:/usr/libexec$ /usr/libexec/java_home /Library/Java/JavaVirtualMachines/jdk1.7.0_09.jdk/Contents/Home So, the magic commands you need are: bone@zen:/System/Library/Frameworks/JavaVM.framework/Versions$ sudo rm CurrentJDK bone@zen:/System/Library/Frameworks/JavaVM.framework/Versions$ sudo ln -s /Library/Java/JavaVirtualMachines/jdk1.7.0_09.jdk/Contents/ CurrentJDK Then, you should be good: bone@zen:/System/Library/Frameworks/JavaVM.framework/Versions$ java -version java version "1.7.0_09" Java(TM) SE Runtime Environment (build 1.7.0_09-b05) Java HotSpot(TM) 64-Bit Server VM (build 23.5-b02, mixed mode)
November 29, 2012
by Brian O' Neill
· 72,147 Views
article thumbnail
Easy Integration Testing with Spring+Hibernate
I am guilty of not writing integration testing (At least for database related transactions) up until now. So in order to eradicate the guilt i read up on how one can achieve this with minimal effort during the weekend. Came up with a small example depicting how to achieve this with ease using Spring and Hibernate. With integration testing, you can test your DAO(Data access object) layer without ever having to deploy the application. For me this is a huge plus since now i can even test my criteria's, named queries and the sort without having to run the application. There is a property in hibernate that allows you to specify an sql script to run when the Session factory is initialized. With this, i can now populate tables with data that required by my DAO layer. The property is as follows; import.sql According to the hibernate documentation, you can have many comma separated sql scripts.One gotcha here is that you cannot create tables using the script. Because the schema needs to be created first in order for the script to run. Even if you issue a create table statement within the script, this is ignored when executing the script as i saw it. Let me first show you the DAO class i am going to test; package com.unittest.session.example1.dao; import org.springframework.transaction.annotation.Propagation; import org.springframework.transaction.annotation.Transactional; import com.unittest.session.example1.domain.Employee; @Transactional(propagation = Propagation.REQUIRED) public interface EmployeeDAO { public Long createEmployee(Employee emp); public Employee getEmployeeById(Long id); } package com.unittest.session.example1.dao.hibernate; import org.springframework.orm.hibernate3.support.HibernateDaoSupport; import com.unittest.session.example1.dao.EmployeeDAO; import com.unittest.session.example1.domain.Employee; public class EmployeeHibernateDAOImpl extends HibernateDaoSupport implements EmployeeDAO { @Override public Long createEmployee(Employee emp) { getHibernateTemplate().persist(emp); return emp.getEmpId(); } public Employee getEmployeeById(Long id) { return getHibernateTemplate().get(Employee.class, id); } } Nothing major, just a simple DAO with two methods where one is to persist and one is to retrieve. For me to test the retrieval method i need to populate the Employee table with some data. This is where the import sql script which was explained before comes into play. The import.sql file is as follows; insert into Employee (empId,emp_name) values (1,'Emp test'); This is just a basic script in which i am inserting one record to the employee table. Note again here that the employee table should be created through the hibernate auto create DDL option in order for the sql script to run. More info can be found here. Also the import.sql script in my instance is within the classpath. This is required in order for it to be picked up to be executed when the Session factory is created. Next up let us see how easy it is to run integration tests with Spring. package com.unittest.session.example1.dao.hibernate; import static org.junit.Assert.*; import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import org.springframework.test.context.transaction.TransactionConfiguration; import com.unittest.session.example1.dao.EmployeeDAO; import com.unittest.session.example1.domain.Employee; @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations="classpath:spring-context.xml") @TransactionConfiguration(defaultRollback=true,transactionManager="transactionManager") public class EmployeeHibernateDAOImplTest { @Autowired private EmployeeDAO employeeDAO; @Test public void testGetEmployeeById() { Employee emp = employeeDAO.getEmployeeById(1L); assertNotNull(emp); } @Test public void testCreateEmployee() { Employee emp = new Employee(); emp.setName("Emp123"); Long key = employeeDAO.createEmployee(emp); assertEquals(2L, key.longValue()); } } A few things to note here is that you need to instruct to run the test within a Spring context. We use the SpringJUnit4ClassRunner for this. Also the transction attribute is set to defaultRollback=true. Note that with MySQL, for this to work, your tables must have the InnoDB engine set as the MyISAM engine does not support transactions. And finally i present the spring configuration which wires everything up; com.unittest.session.example1.**.* org.hibernate.dialect.MySQLDialect com.mysql.jdbc.Driver jdbc:mysql://localhost:3306/hbmex1 root password true org.hibernate.dialect.MySQLDialect create import.sql That is about it. Personally i would much rather use a more light weight in-memory database such as hsqldb in order to run my integration tests. Here is the eclipse project for anyone who would like to run the program and try it out.
November 27, 2012
by Dinuka Arseculeratne
· 56,477 Views · 2 Likes
article thumbnail
Enterprise-ready Tool Support for Apache Camel
apache camel is my favorite integration framework on the java platform due to great dsls, a huge community, and so many different components. camel is used by many developers from different companies all over the world. however, most guys are not aware that some really cool and – more important – enterprise-ready tooling is available for camel, too. many people ask me about camel tooling when i do talks at conferences. this is the reason for this short blog post about camel tooling. [fyi: i work for talend (one of the vendors).] ide support camel consists of a set of normal java libraries and is therefore usable with any java ide (such as eclipse, netbeans or intellij idea) or even a classic text editor. programming dsls are available for java, groovy, and scala. even a kotlin dsl is in the works, thanks to camel’s founder james strachan. all familiar ide features such as code completion or javadoc view are available for these dsls. in the spring xml dsl, the eclipse-based springsource tool suite (sts) should be emphasized, which provides the best support for the spring framework and xml configurations. camel-specific tooling besides classical ide support, further products are available to provide additional functionality. integration problems can be modeled with the help of enterprise integration patterns (eip, http://www.eaipatterns.com/). eips are implemented by camel. visual designers are available to help modeling integration problems with these eips. these tools even generate the corresponding source code automatically. ideally, developers do not have to write any source code by hand. camel tooling is offered by talend with talend esb (http://de.talend.com/products/esb) and jboss, formerly fusesource, with fuse ide (http://fusesource.com/products/fuse-ide). both companies also provide full-time committers for the apache camel project. let’s take a short look at these two products in the following. open studio for talend esb talend esb is an eclipse-based integration platform within the talend unified platform. the familiar “look and feel” and the intuitive use of eclipse remain. the esb is open source and freely available. the paid enterprise version offers additional features and support. the esb can be used independently or in combination with other parts of the talend unified platform, such as BPM, big data, or master data management. the great benefit is that everything can be done within one suite using the same gui and concepts, based on eclipse. the entire talend unified platform is based on the “zero-coding” approach. this way, a very efficient implementation of integration problems is possible using the eips and components. routes are modeled and configured with intuitive tool support, all source code is generated. of course, custom integration logic can still be written and included, for example, pojos, spring beans, scripts in different languages, or own camel components. plenty of other components besides camel’s ones are available for talend esb – for example connectors to alfresco, jasper, sap, salesforce, or host systems. figure 1: visual designer of talend’s esb fuse ide the fuse ide is an eclipse plugin, which is installed from the eclipse update site. the visual designer (see figure 2) generates camel routes as xml code using the spring xml dsl. the generated code is editable vice-versa, i.e. the developer can change the source code. the graphical model applies changes automatically. fuse ide is intuitive to use for creating camel routes. fusesource offers some other products, which can be used in combination with fuse ide – such as management console or fuse mq for messaging. under fusesource, fuse ide was a proprietary product. however, fusesource was recently taken over by redhat (http://www.redhat.com/about/news/press-archive/2012/6/red-hat-to-acquire-fusesource) and now belongs to the jboss division. in the new roadmap, the fuse ide is still included. it will probably be integrated into the jboss enterprise soa platform and become “open sourced”. the integration of fusesource will take at least a few more months time to complete (http://www.redhat.com/promo/jboss_integration_week/). jboss now “owns” three esb products (jboss esb, switchyard and fuse esb). probably, these will be merged into one product in the end (switchyard is also based on camel). nevertheless, the fusesource products will also be supported for some time – primarily in order to satisfy existing customers (my guess). figure 2: visual designer of fuse ide (jboss, former fusesource) enterprise-ready tooling is already available for apache camel! the bottom line is that enterprise-ready tooling is already available for apache camel. it is great to see different companies working on tooling for apache camel. the winner definitely is apache camel… and there is no loser! talend esb and fuse ide are two different approaches for different kinds of projects. if you like the „zero-coding“ approach, then take a closer look at talend’s esb. it is really easy and efficient to realize integration projects without writing source code – nevertheless, there is enough flexibility for customization and adding own source code. the combination with bpm, mdm or big data (based on hadoop) is also supported within the unified platform using the same open source and „zero-coding“ concepts. if you „insist“ on writing and refactoring all source code by yourself within the text editor of an ide, then take a look at fuse ide. your best would be to try out both and see which one fits best into your next enterprise integration project. if you know any other cool camel tooling (no matter if it is enterprise-ready or not), or if you have any other feedback, please write a comment. thank you. best regards, kai wähner (twitter: @kaiwaehner) content from my blog: http://www.kai-waehner.de/blog/2012/11/23/enterprise-ready-tool-support-for-apache-camel/
November 26, 2012
by Kai Wähner DZone Core CORE
· 15,717 Views
article thumbnail
CentOS Minimal Installation Network Configuration
By default CentOS minimal install does not come with pre-configured network, here’s how to make it work: $ ping google.com ping: unknown host google.com To fix this we’ll need to edit the set up for the ethernet. Let’s start with editing this file: $ vim /etc/sysconfig/network-scripts/ifcfg-eth0 IPADDR=x.x.x.x BOOTPROTO=none NETMASK=255.255.255.0 GATEWAY=y.y.y.y DNS1=y.y.y.y DNS2=y.y.y.y USERCTL=yes HWADDR='your mac address' where x.x.x.x is your static ip, and y.y.y.y is your router ip If you’re not sure what your mac address is, run this command $ ifconfig eth0 | grep -o -E '([[:xdigit:]]{1,2}:){5}[[:xdigit:]]{1,2}' now edit the networks config and make sure you added the line below: $ vi /etc/sysconfig/network Add the line: GATEWAY = y.y.y.y Now restart the network interface: $ /etc/init.d/networking restart Now ping the router: $ ping y.y.y.y Request timeout for icmp_seq 0 64 bytes from y.y.y.y: icmp_seq=1 ttl=56 time=1.792 ms Request timeout for icmp_seq 1 64 bytes from y.y.y.y: icmp_seq=3 ttl=56 time=1.790 ms 64 bytes from y.y.y.y: icmp_seq=4 ttl=56 time=1.762 ms Looks good, now let’s see if we can see anything outside. $ ping google.com PING google.com (173.194.67.138) 56(84) bytes of data. 64 bytes from wi-in-f138.1e100.net (173.194.67.138): icmp_seq=1 ttl=49 time=7.88 ms 64 bytes from wi-in-f138.1e100.net (173.194.67.138): icmp_seq=2 ttl=49 time=7.35 ms 64 bytes from wi-in-f138.1e100.net (173.194.67.138): icmp_seq=3 ttl=49 time=7.13 ms Now you can connect to the internet, and get all the packages you need.
November 26, 2012
by Kasia Gogolek
· 90,209 Views · 1 Like
article thumbnail
IBM AIX: Java process Size Monitoring
This article will provide you with a quick reference guide on how to calculate the Java process size memory footprint for Java VM processes running on IBM AIX 5.3+ OS. This is a complementary post to my original article on this subject: how to monitor the Java native memory on AIX. I highly recommend this article to any individual involved in production support or development of Java applications deployed on AIX. Why is this knowledge important? From my perspective, basic knowledge on how the OS is managing the memory allocation of your JVM processes is very important. We often overlook this monitoring aspect and only focus on the Java heap itself. From my experience, most Java memory related problems are observed from the Java heap itself such as garbage collection problems, leaks etc. However, I’m confident that you will face situations in the future involving native memory problems or OS memory challenges. Proper knowledge of your OS and virtual memory management is crucial for proper root causes analysis, recommendations and solutions. AIX memory vs. pages As you may have seen from my earlier post, the AIX Virtual Memory Manager (VMM) is responsible to manage memory requests from the system and its applications. The actual physical memory is converted and partitioned in units called pages; allocated either in physical RAM or stored on disk until it is needed. Each page can have a size of 4 KB (small page), 64 KB (medium page) or 16 MB (large page). Typically for a 64-bit Java process you will see a mix of all of the above. What about the topas command? The typical reflex when supporting applications on AIX is to run the topas command, similar to Solaris top. Find below an example of output from AIX 5.3: As you can see, the topas command is not very helpful to get a clear view on the memory utilization since it is not providing the breakdown view that we need for our analysis. It is still useful to get a rough idea of the paging space utilization which can give you a quick idea of your top "paging space" consumer processes. Same can be achieved via the ps aux command. AIX OS command to the rescue: svmon The AIX svmon command is by far my preferred command to deep dive into the Java process memory utilization. This is a very powerful command, similar to Solaris pmap. It allows you to monitor the current memory “pages” allocation along with each segment e.g. Java Heap vs. native heap segments. Analyzing the svmon output will allow you to calculate the memory footprint for each page type (4 KB, 64 KB, and 16 MB). Now find below a real example which will allow you to understand how the calculation is done: # 64-bit JVM with -Xms2048m & -Xmx2048m (2 GB Java Heap) # Command: svmon –P As you can see, the total footprint of our Java process size was found at 2.2 GB which is aligned with current Java heap settings. You should be able to easily perform the same memory footprint analysis from your AIX environment. I hope this article has helped you to understand how to calculate the Java process size on AIX OS. Please feel free to post any comment or question.
November 26, 2012
by Pierre - Hugues Charbonneau
· 14,604 Views
article thumbnail
Web Application Deployment Using Webistrano and Capistrano
One of the 12 steps to better code, according to Joel Spolsky’s blog is one-click deployment. I’ve worked in places where the deployment process contained about 20 steps and involved a lot of copy-pasting of the paths and environment variables. It was obviously prone to errors, and very stressful for the developers involved. God forbid your wiki database crashed, and you lost all the steps! One click deployment means faster, less buggy deployment. No more wondering, which environment you’re in at the moment, and if you just run that migration script on staging or production. You can even ask your Jimmy to deploy the lovely application for you. There are quite a few tools that can help you deploy your applications: Ant/Phing Bash scripts Jenkins Aegir Today, however I will focus on the mix of Capistrano / Webistrano. Both of the packages are ruby on rails based, and can be easily installed using gems. Capistrano is an open source tool that allows to run scripts on multiple servers. Capistrano automates the process of making new versions of an application as well as supporting tasks such as updating databases, rsyncing media assets etc. Webistrano is a RoR open source UI for managing Capistrano deployments. It provides a descriptive interface for your projects and stages, and easy way to add environment specific settings. Installation Installing Capistrano is quick and easy as long as you have gems and ruby installed: $ gem install capistrano With capistrano installed, let’s move on with Webistrano installation. I want my install to be visible externally on http://webistrano.server.com. For this, I’m going to use Phusion Passanger. For now I’m going to install Webistrano straight in my httpdocs $ cd /var/www/vhosts/webistrano.server.com/httpdocs git clone git://github.com/peritor/webistrano.git . Webistrano uses mysql to store the information about your project, stages and deployments, so let’s log in and set it up: $ mysql mysql> CREATE DATABASE `webistrano`; mysql> CREATE USER 'webistrano'@'localhost' IDENTIFIED BY 'password'; mysql> GRANT ALL PRIVILEGES ON `webistrano`.* TO 'webistrano'@'localhost' WITH GRANT OPTION; Now we need to configure it before we build it with rake. Two files that we need to copy: database (database.yml), and general, webistrano config file (webistrano_config.rb). $ cp config/webistrano_config.rb.sample config/webistrano_config.rb cp config/database.yml.sample config/database.yml Now edit the file with your mysql details and other options you want to set. And the last, but not least, before we start the build. Webistrano creates all the stylesheets on the first run, so the public folder has to have writeable permissions: $ chmod -R 755 public That’s it for the basic set up. What’s left is building webistrano. We’re going to need bundler to install it. If you don’t have bundler yet, use gem to install it: $ gem install bundler Now run rake to build webistrano: $ RAILS_ENV=production rake db:migrate If like me, you get the following error bundler (~> 1.0.10) ruby, otherwise move to the next step (bundle install) rake aborted! Bundler could not find compatible versions for gem "bundler": In Gemfile: bundler (~> 1.0.10) ruby Current Bundler version: bundler (1.2.1)[/code] You will have to install an older version of the bundler. Gems allows you to keep different versions of the same package so this won't clash with your other ruby applications: $ gem install bundler --version=1.0.10 If this was successful, or you haven't had any issues with your bundler, simply run bundle install $ bundle install Fetching source index for http://rubygems.org/ Installing rake (0.9.2.2) Installing activesupport (2.3.11) Installing rack (1.1.3) Installing actionpack (2.3.11) Installing actionmailer (2.3.11) Installing activerecord (2.3.11) Installing activeresource (2.3.11) Using bundler (1.0.10) Installing highline (1.6.15) Installing jruby-pageant (1.1.1) Installing net-ssh (2.6.0) Installing net-scp (1.0.4) Installing net-sftp (2.0.5) Installing net-ssh-gateway (1.1.0) Installing capistrano (2.6.0) Installing erubis (2.7.0) Installing exception_notification (2.3.3.0) Installing mocha (0.9.8) Installing mysql (2.8.1) with native extensions Installing open4 (0.9.3) Installing rails (2.3.11) Installing syntax (1.0.0) Your bundle is complete! It was installed into ./vendor/bundler You need to build with rake again $ RAILS_ENV=production rake db:migrate WARNING: 'require 'rake/rdoctask'' is deprecated. Please use 'require 'rdoc/task' (in RDoc 2.4.2+)' instead. at /var/www/vhosts/webistrano.server.com/httpdocs/vendor/bundler/ruby/1.8/gems/rake-0.9.2.2/lib/rake/rdoctask.rb rake aborted! no such file to load -- /var/www/vhosts/webistrano.server.com/httpdocs/config/webistrano_config Tasks: TOP => db:migrate => environment (See full trace by running task with --trace) If you get this error find out what rake gems you've got installed by running gem list $ gem list => rake You'll have to specify the version of rake manually in your gem file: $ vim Gemfile gem rake, '0.8.7' You will have to run bundle install again. We'll start from unlocking the version of rake that bundle currently has set to use, by updating bundle: $ bundle update rake Now install the correct bundle $ bundle install and run rake using bundle exec $ bundle exec rake db:migrate --trace RAILS_ENV=production Hurrah! You have successfully installed Webistrano! Running Webistrano ** EDIT ** IN my original post I've been advising to use Phusion Passenger for Apache to run websitrano, but since it occurred it only causes trouble with permissions during deployment. Phusion Passenger should run as the user that has the ownership over conf/environment.rb which in my case was webistrano. All seemed ok, and logging the id of my user running Webistrano was pointing to the correct one, but it seems that there was some sort of conflict and the process was actually running as nobody. You can find more about the issue on stack overflow With this in mind, I would recommend using mongrel instead. $ sudo gem install mongrel $ su - webistrano $ cd /var/www/vhosts/webistrano.server.com/httpdocs $ mongrel_rails start -e production -d -p 3000 Now go to http://webistrano.server.com:3000 and you should be able to see the login screen login: admin password: admin Success! You can now see the main Webistrano screen. If this has wet your apetite, please read my next post about deploying PHP applications with Webistrano. Also, feel free to leave your comments below. I hope you found this post useful.
November 26, 2012
by Kasia Gogolek
· 20,034 Views
article thumbnail
IndexedDBViewer: Take a Look Inside your IndexedDB
Some days ago I released a new version of the IndexedDBViewer 1.1.0. The IndexedDBViewer is intended for web developers who want to sneak into their indexedDB database. It allows you to inspect the structure of your database as well as the data stored inside this structure. The difference with the previous version is that it no longer needs the jQueryUI library. This way I eliminated at least one reference. The following references are still necessary: jQuery (version 1.8.2 or higher) linq2indexedDB (version 1.0.19 or higher) indexedDBViewer (1.1.0 – JavaScript + css file) If you are using nuget, you can get all the resource by searching for the indexedDBViewer. The second major change is that the viewer can easily be added to an existing page. The only thing you need to do is add a div with “indexedDBViewer” as id and data-dbName attribute to pass the database you want to inspect. The rest will be handled by the script in the viewer. Once this is done and you navigate to the page with the viewer, you will get the following result In the bottom you will see the view appear. On the left pane you get an overview of the database structure. This a list with on top the name of the database. Under that you will find child nodes that represent the object stores present in the database. If we descend an other level we can see the indexes present on the object store. If you click on the “+” or “-“ next to the names, you can expand or hide the structure beneath. If you click on the database name in the navigation pane, you will get information about the database and it’s structure. In the general block you will see the name of the database and the version it is currently in The object stores block gives you an overview of all the object stores present and how they are configured. The indexes block shows all the present indexes and how they are configured. When you click on one of the object store names in the navigation pane, you will get all the data present in the object store. Because the data is saved as a key/value pair, you will see the key with his corresponding value. If the value is an object or contains objects, then you can inspect them by clicking on the “+” to expand and “-” to hide the details. If you click on one of the index names in the navigation pane, you will get – similar as for object stores – all the data present in the object store. But in this case you will see a little more. Besides the key of the index and the value you will see the key the value has in the object store. This can be found under the “primary” key column. As last there are some little extra features: If you click on the top border of the viewer and drag it up or down, then you can change the height of the viewer. if you click on the “-” in the right top of the viewer, you can hide the viewer. If you want it to appear again, then you have to click on the “+” on the right bottom of the page. Conclusion With this Chrome like indexedDBViewer you can inspect the structure of your database inclusive all data stored within it. This with the advantage that it runs inside the browser, so you can use it cross-browser.
November 26, 2012
by Kristof Degrave
· 5,571 Views
article thumbnail
Lightweight RPC with ZeroMQ (ØMQ) and Protocol Buffers
A frequent issue I come across writing integration applications with Mule is deciding how to communicate back and forth between my front end application, typically a web or mobile application, and a flow hosted on Mule. I could use web services and do something like annotate a component with JAX-RS and expose this out over HTTP. This is potentially overkill, particularly if I only want to host a few methods, the methods are asynchronous or I don’t want to deal with the overhead of HTTP. It also could be a lot of extra effort if the only consumers of the API, at least initially, are internal facing applications. Another choice is to use “synchronous” JMS with temporary reply queues. While Mule makes this easy to do, particularly with MuleClient, I now have to deal with the overhead of spinning up a JMS infrastructure. I could also be limited to Java only clients, depending on which JMS broker I choose. The latter is particularly signifcant, as Java probably isn’t the technology of choice on the web or mobile layer. ØMQ for RPC ØMQ, or ZeroMQ, is a networking library designed from the ground up to ease integration between distributed applications. In addition to supporting a variety of messaging patterns, which are enumerated in the extremely well written guide, the library is written in platform agnostic C with wrappers for different languages like Java, Python and Ruby. These features make it a good candidate to solve the challenges I introduced above, particularly since a community contributed module for ØMQ was released recently. Let’s consider a simple service that accepts a request for a range of stock quotes and returns the results and see how we can host this service with Mule and expose it out with the ØMQ Module. Data Serialization with Protocol Buffers Data is transported back and forth over ØMQ as byte arrays. We, as such, need to decide on a way to serialize our stock quote request and responses “on the wire.” Before we do that, however, let’s take a look at the Java canonical data model we’re using on the client and server side. The following Gists show the important bits of the StockQuote and StockQuoteResponse classes. public class StockQuote implements Serializable { String symbol; Date date; Double open; Double high; Double low; Double close; Long volume; Double adjustedClose; public class StockQuoteRequest implements Serializable { String symbol; Date startDate; Date endDate; public interface StockDataService { public List getQuote(StockQuoteRequest request); } We could use Java serialization to get the objects into byte arrays. Ignoring the other deficiencies of default Java serialization, the main drawback is that it limits our clients to one’s running on a JVM. XML or JSON provide better alternatives, but for the purposes of this example we’ll assume we want a more compact representation of the data (this isn’t totally unrealistic, stock quote data can be extremely time sensitive and we probably want to minimize serialization and deserialization overhead.) Protocol Buffers provide a good middle ground and also boast a Mule Module to provide the necessary transformers we need to move back and forth from the byte array representations. Let’s define two .proto files to define the wire format and generate the intermediary stubs for serialization. package com.acmesoft.zeromq; option java_package = "com.acmesoft.stock.model.serialization.protobuf"; option optimize_for = SPEED;package com.acmesoft.zeromq; option java_package = "com.acmesoft.stock.model.serialization.protobuf"; option optimize_for = SPEED; option java_multiple_files = true; message StockQuoteResponseBuffer { repeated StockQuoteBuffer result = 1; } message StockQuoteBuffer { required string symbol = 1; required int64 date = 2; required double open = 3; required double high = 4; required double low = 5; required double close = 6; required int64 volume = 7; required double adjustedClose = 8; } option java_multiple_files = true; message StockQuoteRequestBuffer { required string symbol = 1; required int64 start = 2; required int64 end = 3; } You typically would use the “protoc” compiler to generate the Java stubs. This is tedious, however, so we’ll instead modify the pom.xml of our project to compile the protoc files during the compile goals: com.google.protobuf.tools maven-protoc-plugin /usr/local/bin/protoc compile testCompile Since we already have a domain model we’ll add some helper classes to simplify the serialization tasks on the client side. public byte[] toProtocolBufferAsBytes() { return StockQuoteRequestBuffer.newBuilder() .setSymbol(symbol) .setStart(startDate.getTime()) .setEnd(endDate.getTime()).build().toByteArray(); } public static StockQuoteRequest fromProtocolBuffer(StockQuoteRequestBuffer buffer) { StockQuoteRequest request = new StockQuoteRequest(); request.setSymbol(buffer.getSymbol()); request.setStartDate(new Date(buffer.getStart())); request.setEndDate(new Date(buffer.getEnd())); return request; } public static StockQuoteResponseBuffer toProtocolBuffer(List quotes) { StockQuoteResponseBuffer.Builder responseBuilder = StockQuoteResponseBuffer.newBuilder(); for (StockQuote quote : quotes) { responseBuilder.addResult(StockQuoteBuffer.newBuilder() .setAdjustedClose(quote.getAdjustedClose()) .setClose(quote.getClose()) .setDate(quote.getDate().getTime()) .setHigh(quote.getHigh()) .setLow(quote.getLow()) .setOpen(quote.getOpen()) .setSymbol(quote.getSymbol()) .setVolume(quote.getVolume()).build()); } return responseBuilder.build(); } public static List listOfStockQuotesFromBytes(byte[] bytes) { List buffer; try { buffer = StockQuoteResponseBuffer.parseFrom(bytes).getResultList(); } catch (InvalidProtocolBufferException e) { throw new SerializationException(e); } List quotes = new ArrayList(); for (StockQuoteBuffer stockQuoteBuffer : buffer) { StockQuote stockQuote = new StockQuote(); stockQuote.setClose(stockQuoteBuffer.getClose()); stockQuote.setDate(new Date(stockQuoteBuffer.getDate())); stockQuote.setHigh(stockQuoteBuffer.getHigh()); stockQuote.setOpen(stockQuoteBuffer.getOpen()); stockQuote.setSymbol(stockQuoteBuffer.getSymbol()); stockQuote.setVolume(stockQuoteBuffer.getVolume()); stockQuote.setAdjustedClose(stockQuoteBuffer.getAdjustedClose()); stockQuote.setLow(stockQuoteBuffer.getLow()); quotes.add(stockQuote); } return quotes; } Configuring StockDataService Now that we have a canonical data model and a wire format defined we’re ready to wire up a Mule flow to expose the service out. Note that for this to work you need to have jzmq installed locally on your system. The following dependency needs to be added to your pom.xml once its installed: org.zeromq zmq 2.2.0 /usr/local/lib/zmq.jar system Where systemPath is the location of the zmq.jar on your filesystem. Once that’s out of the way we can configure the flow, as illustrated below: The ZeroMQ inbound-endpoint will be bound to TCP port 9090 with a request-response exchange pattern. The deserialize MP in the protobuf module will deserialize the byte array to the generated StockQuoteRequestBuffer class. From there we’ll use MEL to invoke the helper method on StockQuoteRequest to transform the intermediary class to the domain model. The List of StockQuotes returned from StockDataService will be transformed by the MEL expression using the “toProtocolBuffer” helper method on the domain model. The Protocol Buffer Module is then smart enough to implicitly transform the intermediary object to a byte array for the response. Consuming the Service from the Client Side Now that the server is ready we can turn our attention to the client side code to invoke the remote service. Let’s take a look at how this works: StockQuoteRequest stockQuoteRequest = new StockQuoteRequest(); stockQuoteRequest.setSymbol("FB"); stockQuoteRequest.setStartDate(new Date( new Date().getTime() - (86400000 * 7))); stockQuoteRequest.setEndDate(new Date()); ZMQ.Socket zmqSocket = zmqContext.socket(ZMQ.REQ); zmqSocket.setReceiveTimeOut(RECEIVE_TIMEOUT); zmqSocket.connect("tcp://localhost:9090"); zmqSocket.send(stockQuoteRequest.toProtocolBufferAsBytes(), 0); List quotes = StockQuote.listOfStockQuotesFromBytes(zmqSocket.recv(0)); We start off by defining the StockQuoteRequest object to give us all the quotes for Facebook stock from the last week. We can then open up a ZMQ socket, set the timeout, connect to the ZMQ socket on the remote Mule instance and send the byte representation of the StockQuoteRequest to it. zmqSocket.recv is then used to receive the bytes back from Mule. From here we can use the listOfStockQuotesFromBytes helper method we wrote above to convert the Protocol Buffer representation to a List of StockQuotes. Despite the fair bit of plumbing we did above, this is a pretty concise bit of client side code to invoke the remote service. Conclusion This blog post only touched on the features of ØMQ and the ØMQ Mule Module. In addition to request-reply, other exchange-patterns are supported, like one-way, push and pull. This effectively gives you the benefits of a reliable, asynchronous messaging layer without a centralized infrastructure. I hope to cover this in a later post. Protocol buffers also seem like a natural fit as a wire format for ØMQ. protobuffers echo ØMQ’s principals of being lightweight, fast and platform agnostic. These are also, not coincidently, principals Mule shares as an integration framework. The project for this example is available on GitHub.
November 26, 2012
by John D'Emic
· 28,583 Views
article thumbnail
Moving the Decimal Place in a Double
This is taken from a popular answer to the question Moving decimal places over in a double Question So I have a double set to equal 1234, I want to move a decimal place over to make it 12.34 So to do this I multiply .1 to 1234 two times, kinda like this double x = 1234; for(int i = 1; i <= 2; i++) { x = x*.1; } System.out.println(x); This will print the result, "12.340000000000002" Is there a way, without simply formatting it to two decimal places, to have the double store 12.34 correctly? Answer If you use double or float, you should use rounding or expect to see some rounding errors. If you can't do this, use BigDecimal. The problem you have is that 0.1 is not an exact representation, and by performing the calculation twice, you are compounding that error. However, 100 can be represented accurately, so try: double x = 1234; x /= 100; System.out.println(x); which prints: 12.34 This works because Double.toString(d) performs a small amount of rounding on your behalf, but it is not much. If you are wondering what it might look like without rounding: System.out.println(new BigDecimal(0.1)); System.out.println(new BigDecimal(x)); prints 0.100000000000000005551115123125782702118158340454101562 12.339999999999999857891452847979962825775146484375 In short, rounding is unavoidable for sensible answers in floating point whether you are doing this explicitly or not. Note: x / 100 and x * 0.01 are not exactly the same when it comes to rounding error. This is because the round error for the first expression depends on the values of x, whereas the 0.01 in the second has a fixed round error. for(int i = 0; i < 200; i++) { double d1 = (double) i / 100; double d2 = i * 0.01; if (d1 != d2) System.out.println(d1 + " != "+d2); } prints 0.35 != 0.35000000000000003 0.41 != 0.41000000000000003 0.47 != 0.47000000000000003 0.57 != 0.5700000000000001 0.69 != 0.6900000000000001 0.7 != 0.7000000000000001 0.82 != 0.8200000000000001 0.83 != 0.8300000000000001 0.94 != 0.9400000000000001 0.95 != 0.9500000000000001 1.13 != 1.1300000000000001 1.14 != 1.1400000000000001 1.15 != 1.1500000000000001 1.38 != 1.3800000000000001 1.39 != 1.3900000000000001 1.4 != 1.4000000000000001 1.63 != 1.6300000000000001 1.64 != 1.6400000000000001 1.65 != 1.6500000000000001 1.66 != 1.6600000000000001 1.88 != 1.8800000000000001 1.89 != 1.8900000000000001 1.9 != 1.9000000000000001 1.91 != 1.9100000000000001
November 25, 2012
by Peter Lawrey
· 15,600 Views
article thumbnail
Migrating from BIND DNS servers to Amazon Route 53 by using cli53
Amazon Route 53 offers DNS as a hosted service. They maintain robust DNS service for you with easy to user web interface and API. Name servers are part of any serious hosting infrastructure. They have been traditionally run on BIND or similar UNIX daemons. The usual hosting tasks including mapping domain names and subdomains (A, CNAME, et. al records) to servers and delivering email (MX and SPF records). Usually a basic hosting plan is only good for setting A records for few subdomains like www, but on the instant you need something more advanced like routing MX records to Google Apps email you need to start considering a real DNS service. GoDaddy and other big name registrants provide tools for this, but they are usually specific only for top level domains (TLDs) the registrant supports. If you come from a cold country like .fi you cannot centralize your DNS management to big popular domain registrants. The bad thing with running your own name servers is that when the service goes down the whole your infrastructure becomes virtually inaccessible – web sites, email, anything. Thus you usually run at least two BIND daemons on two physically separated servers just in case one blows up the other can still keep DNS records running. Also DNS, being core part of internet infrastructure, is subject for many kind of attacks and keeping your skills and knowledge up-to-date with BIND may be time consuming. Like with email servers, hosting name servers is a thing you really don’t want to do. Enter Amazon Route 53. (tha name Route 53 probably comes from the famous U.S. road Route 67 and DNS port 53?). Route 53 is ridiculous easy to use. Just go to the site and register one for yourself. However if you have legacy BIND servers running around there exist ways to import data automatically instead of manually reading through zone files and typing them in to Route 53 by hand. 1. Setting up cli53 cli53 is a Python based command line interface for Route 53. It uses buildout for installation (of Python package dependencies). These instructions have been tested with Ubuntu 8.04 server. Prerequisitements for using this stuff is knowing basic Ubuntu server management from the command line. Run everything as root. Install wget and unzip apt-get install wget unzip Download ZIP copy of cli53 from Github (we don’t do git clone here – no need to install git on the server): wget --no-check-certificate https://github.com/barnybug/cli53/zipball/master Extract unzip master Install as user local under the extracted folder (note that the folder varies across GitHub exports) cd barnybug-cli53-3b468b7 python bootstrap.py bin/buildout This will generate the local bin/cli53 command. Now you need to set up AWS credentials. Route 53, and other AWS service, are operated using access keys which you can get from Security Credentials page in AWS web console. Set the access keys for your shell/SSH session using enviroment variables: export AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXX export AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXX Side note: as this point you need to check that your server clock is in correct time your AWS remote operations will fail (300 seconds thresold).. Use date command to see the server clock time and correct it with the same date command using this obscure date command syntax. Now you can run a test by listing your Amazon Route 53 Hosted Zones (Hosted Zone = zonefile = one per top level domain usually) using list command: bin/cli53 list The proper output for one domain which has been created throught-the-web in Route 53 control panel is something like: HostedZones: - CallerReference: 9265CCC3-9C41-98CE-8820-E26C0356C478 Config: Comment: test Id: /hostedzone/Z2L3NT0WCS8OWA Name: xxx.fi. 2. Importing zone files Now when cli53 is running and working it is time to rock’n'roll. On our server configuration BIND zonefiles lived in /srv/dns/zones though this is not the default location for Ubuntu BIND. There is one file per each zone and the filename is the domain name. [root@foobar][23:02][/srv/dns/zones]# ls -l total 176 -rw-r--r-- 1 root root 425 Dec 30 2008 abc.com -rw-r--r-- 1 root root 1713 Dec 30 2008 abcabc.com -rw-r--r-- 1 root root 1719 Dec 30 2008 abcabcabc.com We can simple create an one-liner shell script which will loop through all the files and import them to Amazon Route 53. However – there is a trick. You’ll get unknown origin exception if your zone files lack a certain line: Traceback (most recent call last): File "/root/barnybug-cli53-3b468b7/bin/cli53", line 22, in cli53.cli53.main() File "/root/barnybug-cli53-3b468b7/src/cli53/cli53.py", line 495, in main args.func(args) File "/root/barnybug-cli53-3b468b7/src/cli53/cli53.py", line 268, in cmd_import raise Exception, 'Could not find origin' Exception: Could not find origin This is because the zone file doesn’t have $ORIGIN directive. it’s optional with BIND as BIND uses the filename as $ORIGIN if it’s not defined, but the directive is mandatory for cli53 (cli53 doesn’t yet synthetize $ORIGIN). You can simple add $ORIGIN to the beginning of each file and copy files to a working directory with the following shell one-liner (it’s always safe to make copies of processed files than try to fix them in place):./transfer/$i ; done mkdir ../transfer for i in * ; do echo "\$ORIGIN $i." > ../transfer/$i ; cat $i >> ../transfer/$i ; done Then do a test run for one import: ~/barnybug-cli53-3b468b7/bin/cli53 create xxx.fi ~/barnybug-cli53-3b468b7/bin/cli53 import xxx.fi --file xxx.fi --replace --wait Note: deleting zones in Route 53 control panel is painful difficult, so make sure you import zones you only really need. If everything looks good we are ok to upload everything to Route 53. Again, a shell one-liner does the trick for us: for i in * ; do ~/barnybug-cli53-3b468b7/bin/cli53 create $i ; ~/barnybug-cli53-3b468b7/bin/cli53 import $i --file $i --wait ; done This will take abour 15-20 seconds per domain. Note: if you need to re-import add –replace flag. 3. The ugly part Each zone has Source of Authority record with authoritative nameserves (NS records). These will be change from your BIND server IP addresses to Amazon ones. Route 53 will re-assign its own name servers for each imported Hosted Zone. However, you cannot know beforehand what name servers the dice has chosen for your Hosted Zone, so automatizing this process is little bit difficult. NS records servers are overriden when the zone file is uploaded to Route 53. You can see new SOA and NS records when you choose the domain in Route 53 control panel and press Go to Record Sets. To make things easier later on here is a script which will dump all the name servers of all domains you have in Route 53 and collect are info to domain-info.txt file: for i in * ; do ~/barnybug-cli53-3b468b7/bin/cli53 info $i >> /tmp/domain-info.txt ; done 4. Testing Route 53 DNS Browser through imported domain records in Route 53 control panel and see if they look ok. You can also test the actual DNS functionality. When you use ping command, your web browser or any other “normal” mean which queries DNS records they use the Source of Authority name servers as set by in your domain registrant settings. When in the middle of migration to Route 53, the authorative name servers are still pointing to the old name servers in this point. dig command can be used to query a specific name server for DNS records. In Route 53 control panel pick any domain and any of its name servers Use dig command to query the records of this domain Example: dig -t ANY @ns-1340.awsdns-39.org yourdomainname.com Output should be something like this: ; <<>> DiG 9.4.2-P1 <<>> -t ANY @ns-1340.awsdns-39.org yourdomain.com ; (1 server found) ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 9352 ;; flags: qr aa rd; QUERY: 1, ANSWER: 10, AUTHORITY: 0, ADDITIONAL: 0 ;; WARNING: recursion requested but not available ;; QUESTION SECTION: ;yourdomain.com. IN ANY ;; ANSWER SECTION: yourdomain.com. 172800 IN NS ns-470.awsdns-58.com. yourdomain.com. 172800 IN NS ns-553.awsdns-05.net. yourdomain.com. 172800 IN NS ns-1340.awsdns-39.org. yourdomain.com. 172800 IN NS ns-1706.awsdns-21.co.uk. yourdomain.com. 900 IN SOA ns-1340.awsdns-39.org. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400 yourdomain.com. 0 IN MX 10 aspmx.l.google.com. yourdomain.com. 0 IN MX 20 alt1.aspmx.l.google.com. yourdomain.com. 0 IN MX 20 alt2.aspmx.l.google.com. yourdomain.com. 0 IN TXT "v=spf1 a mx a:smtp.something.fi a:auth-smtp.something.fi include:aspmx.googlemail.com ~all" yourdomain.com. 0 IN A 84.20.128.49 ;; Query time: 32 msec ;; SERVER: 205.251.197.60#53(205.251.197.60) ;; WHEN: Wed Nov 23 00:04:53 2011 ;; MSG SIZE rcvd: 413 (subdomains are not listed) 5. Finalizing it Go to your domain name registrant and update the domain name servers point to your Route 53 servers. For each domain in domain-info.txt Go to the corresponding registrant from whom you obtained the domain (e.g. domain.ficora.fi) Update the authoritative name servers to be as stated in the file Wait 24-48 hours or whatever time-to-live time you have for your name servers – usually this is longish period See that your services still run Take down your old name servers 6. More info http://hathology.com/2011/02/how-to-import-bind-zone-files-into-amazon-route-53/ http://docs.amazonwebservices.com/Route53/latest/DeveloperGuide/index.html?MigratingDNS.html#Step_UpdateRegistrar
November 23, 2012
by Mikko Ohtamaa
· 13,775 Views
article thumbnail
How to Resolve java.lang.ClassNotFoundException
This article is intended for Java beginners currently facing java.lang.ClassNotFoundException challenges. It will provide you with an overview of this common Java exception, a sample Java program to support your learning process and resolution strategies. If you are interested on more advanced class loader related problems, I recommended that you review my article series on java.lang.NoClassDefFoundError since these Java exceptions are closely related. java.lang.ClassNotFoundException: Overview As per the Oracle documentation, ClassNotFoundException is thrown following the failure of a class loading call, using its string name, as per below: The Class.forName method The ClassLoader.findSystemClass method The ClassLoader.loadClass method In other words, it means that one particular Java class was not found or could not be loaded at “runtime” from your application current context class loader. This problem can be particularly confusing for Java beginners. This is why I always recommend to Java developers to learn and refine their knowledge on Java class loaders. Unless you are involved in dynamic class loading and using the Java Reflection API, chances are that the ClassNotFoundException error you are getting is not from your application code but from a referencing API. Another common problem pattern is a wrong packaging of your application code. We will get back to the resolution strategies at the end of the article. java.lang.ClassNotFoundException: Sample Java program Now find below a very simple Java program which simulates the 2 most common ClassNotFoundException scenarios via Class.forName() & ClassLoader.loadClass(). Please simply copy/paste and run the program with the IDE of your choice (Eclipse IDE was used for this example). The Java program allows you to choose between problem scenario #1 or problem scenario #2 as per below. Simply change to 1 or 2 depending of the scenario you want to study. # Class.forName() private static final int PROBLEM_SCENARIO = 1; # ClassLoader.loadClass() private static final int PROBLEM_SCENARIO = 2; # ClassNotFoundExceptionSimulator package org.ph.javaee.training5; /** * ClassNotFoundExceptionSimulator * @author Pierre-Hugues Charbonneau * */ public class ClassNotFoundExceptionSimulator { private static final String CLASS_TO_LOAD = "org.ph.javaee.training5.ClassA"; private static final int PROBLEM_SCENARIO = 1; /** * @param args */ public static void main(String[] args) { System.out.println("java.lang.ClassNotFoundException Simulator - Training 5"); System.out.println("Author: Pierre-Hugues Charbonneau"); System.out.println("http://javaeesupportpatterns.blogspot.com"); switch(PROBLEM_SCENARIO) { // Scenario #1 - Class.forName() case 1: System.out.println("\n** Problem scenario #1: Class.forName() **\n"); try { Class newClass = Class.forName(CLASS_TO_LOAD); System.out.println("Class "+newClass+" found successfully!"); } catch (ClassNotFoundException ex) { ex.printStackTrace(); System.out.println("Class "+CLASS_TO_LOAD+" not found!"); } catch (Throwable any) { System.out.println("Unexpected error! "+any); } break; // Scenario #2 - ClassLoader.loadClass() case 2: System.out.println("\n** Problem scenario #2: ClassLoader.loadClass() **\n"); try { ClassLoader classLoader = Thread.currentThread().getContextClassLoader(); Class callerClass = classLoader.loadClass(CLASS_TO_LOAD); Object newClassAInstance = callerClass.newInstance(); System.out.println("SUCCESS!: "+newClassAInstance); } catch (ClassNotFoundException ex) { ex.printStackTrace(); System.out.println("Class "+CLASS_TO_LOAD+" not found!"); } catch (Throwable any) { System.out.println("Unexpected error! "+any); } break; } System.out.println("\nSimulator done!"); } } # ClassA package org.ph.javaee.training5; /** * ClassA * @author Pierre-Hugues Charbonneau * */ public class ClassA { private final static Class CLAZZ = ClassA.class; static { System.out.println("Class loading of "+CLAZZ+" from ClassLoader '"+CLAZZ.getClassLoader()+"' in progress..."); } public ClassA() { System.out.println("Creating a new instance of "+ClassA.class.getName()+"..."); doSomething(); } private void doSomething() { // Nothing to do... } } f you run the program as is, you will see the output as per below for each scenario: #Scenario 1 output (baseline) java.lang.ClassNotFoundException Simulator - Training 5 Author: Pierre-Hugues Charbonneau http://javaeesupportpatterns.blogspot.com ** Problem scenario #1: Class.forName() ** Class loading of class org.ph.javaee.training5.ClassA from ClassLoader 'sun.misc.Launcher$AppClassLoader@bfbdb0' in progress... Class class org.ph.javaee.training5.ClassA found successfully! Simulator done! #Scenario 2 output (baseline) java.lang.ClassNotFoundException Simulator - Training 5 Author: Pierre-Hugues Charbonneau http://javaeesupportpatterns.blogspot.com ** Problem scenario #2: ClassLoader.loadClass() ** Class loading of class org.ph.javaee.training5.ClassA from ClassLoader 'sun.misc.Launcher$AppClassLoader@2a340e' in progress... Creating a new instance of org.ph.javaee.training5.ClassA... SUCCESS!: org.ph.javaee.training5.ClassA@6eb38a Simulator done! For the “baseline” run, the Java program is able to load ClassA successfully. Now let’s voluntary change the full name of ClassA and re-run the program for each scenario. The following output can be observed: #ClassA changed to ClassB private static final String CLASS_TO_LOAD = "org.ph.javaee.training5.ClassB"; #Scenario 1 output (problem replication) java.lang.ClassNotFoundException Simulator - Training 5 Author: Pierre-Hugues Charbonneau http://javaeesupportpatterns.blogspot.com ** Problem scenario #1: Class.forName() ** java.lang.ClassNotFoundException: org.ph.javaee.training5.ClassB at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:186) at org.ph.javaee.training5.ClassNotFoundExceptionSimulator.main(ClassNotFoundExceptionSimulator.java:29) Class org.ph.javaee.training5.ClassB not found! Simulator done! #Scenario 2 output (problem replication) java.lang.ClassNotFoundException Simulator - Training 5 Author: Pierre-Hugues Charbonneau http://javaeesupportpatterns.blogspot.com ** Problem scenario #2: ClassLoader.loadClass() ** java.lang.ClassNotFoundException: org.ph.javaee.training5.ClassB at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) at org.ph.javaee.training5.ClassNotFoundExceptionSimulator.main(ClassNotFoundExceptionSimulator.java:51) Class org.ph.javaee.training5.ClassB not found! Simulator done! What happened? Well since we changed the full class name to org.ph.javaee.training5.ClassB, such class was not found at runtime (does not exist), causing both Class.forName() and ClassLoader.loadClass() calls to fail. You can also replicate this problem by packaging each class of this program to its own JAR file and then omit the jar file containing ClassA.class from the main class path Please try this and see the results for yourself…(hint: NoClassDefFoundError) Now let’s jump to the resolution strategies. java.lang.ClassNotFoundException: Resolution strategies Now that you understand this problem, it is now time to resolve it. Resolution can be fairly simple or very complex depending of the root cause. Don’t jump on complex root causes too quickly, rule out the simplest causes first. First review the java.lang.ClassNotFoundException stack trace as per the above and determine which Java class was not loaded properly at runtime e.g. application code, third party API, Java EE container itself etc. Identify the caller e.g. Java class you see from the stack trace just before the Class.forName() or ClassLoader.loadClass() calls. This will help you understand if your application code is at fault vs. a third party API. Determine if your application code is not packaged properly e.g. missing JAR file(s) from your classpath If the missing Java class is not from your application code, then identify if it belongs to a third party API you are using as per of your Java application. Once you identify it, you will need to add the missing JAR file(s) to your runtime classpath or web application WAR/EAR file. If still struggling after multiple resolution attempts, this could means a more complex class loader hierarchy problem. In this case, please review my NoClassDefFoundError article series for more examples and resolution strategies I hope this article has helped you to understand and revisit this common Java exception. Please feel free to post any comment or question if you are still struggling with your java.lang.ClassNotFoundException problem.
November 23, 2012
by Pierre - Hugues Charbonneau
· 221,657 Views · 6 Likes
article thumbnail
A Node.js speed dilemma: AJAX or Socket.IO?
Originally posted by Daniel Chirca One the first things I stumbled upon when I started my first Node.js project was how to handle the communication between the browser (the client) and my middleware (the middleware being a Node.js application using the CUBRID Node.js driver (node-cubrid) to exchange information with a CUBRID 8.4.1 database). I am already familiar with AJAX (btw, thanks God for jQuery!! ) but, while studying Node.js, I found out about the Socket.IO module and even found some pretty nice code examples on the internet... Examples which were very-very easy to (re)use... So this quickly become a dilemma: what to choose, AJAX or sockets.io? Obviously, as my experience was quite limited, I needed first more information from out there... In other words, it was time to do some quality Google search :) There’s a lot of information available and, obviously, one would need to filter out all the “noise” and keep what is really useful. Let me share with you some of the goods links I found on the topic: http://stackoverflow.com/questions/7193033/nodejs-ajax-vs-socket-io-pros-and-cons http://podefr.tumblr.com/post/22553968711/an-innovative-way-to-replace-ajax-and-jsonp-using http://stackoverflow.com/questions/4848642/what-is-the-disadvantage-of-using-websocket-socket-io-where-ajax-will-do?rq=1 http://howtonode.org/websockets-socketio To summarize, here’s what I quickly found: Socket.IO (usually) uses persistent connection between the client and the server (the middleware), so you can reach a maximum limit of concurrent connections depending on the resources you have on server side (while more AJAX async requests can be served with the same resources). With AJAX you can do RESTful requests. This means that you can take advantage of existing HTTP-infrastructure like e.g. proxies to cache requests and use conditional get requests. There is more (communication) data overhead in AJAX when compared to Socket.IO (HTTP headers, cookies etc.) AJAX is usually faster than Socket.IO to “code”... When using Socket.IO, it is possible to have a two-way communication where each side – client or server - can initiate a request. In AJAX, it is only the client who can initiate a request! Socket.IO has more transport options, including Adobe Flash. Now, for my own application, what I was most interested in was the speed of making requests and getting data from the (Node.js) server! Regarding the middleware data communication with the CUBRID database, as ~90% of my data access was read-only, a good data caching mechanism is obviously a great way to go! But about this, I’ll talk next time. So I decided to put up their (AJAX and socket.io) speed to test, to see which one is faster (at least on my hardware & software environment)....! My middleware was setup to run on an i5 processor, 8GB of RAM and an Intel X25 SSD drive. But seriously, every speed test and, generally speaking, any performance test depends so much(!) on your hardware and software configuration, that it is always a great idea to try the things on your own environment, rely less on various information you find on internet and more on your own findings! The tests I decided to do have to meet the following requirements: Test: AJAX Socket.IO persistent connection Socket.IO non-persistent connections Test 10, 100, 250 and 500 data exchanges between the client and the server Each data exchange between the middleware SERVER (a Node.js web server) and the client (a browser) is a 4KBytes random data string Run the server in release (not debug) mode Use Firefox as the client Minimize the console messages output, for both server and client Do each test after a client full page reload Repeat each test at least 3 times, to make sure the results are consistent Testing Socket.IO, using a persistent connection I've created a small Node.js server, which was handling the client requests: io.sockets.on('connection', function (client) { client.on('send_me_data', function (idx) { client.emit('you_have_data', idx, random_string(4096)); }); }); And this is the JS client script I used for test: var socket = io.connect(document.location.href); socket.on('you_have_data', function (idx, data) { var end_time = new Date(); total_time += end_time - start_time; logMsg(total_time + '(ms.) [' + idx + '] - Received ' + data.length + ' bytes.'); if (idx++ < countMax) { setTimeout(function () { start_time = new Date(); socket.emit('send_me_data', idx); }, 500); } }); Testing Socket.IO, using NON-persistent connection This time, for each data exchange, I opened a new socket-io connection. The Node.js server code was similar with the previous one, but I decided to send back the client data immediately after connect, as a new connection was initiated every time, for each data exchange: io.sockets.on('connection', function (client) { client.emit('you_have_data', random_string(4096)); }); The client test code was: function exchange(idx) { var start_time = new Date(); var socket = io.connect(document.location.href, {'force new connection' : true}); socket.on('you_have_data', function (data) { var end_time = new Date(); total_time += end_time - start_time; socket.removeAllListeners(); socket.disconnect(); logMsg(total_time + '(ms.) [' + idx + '] - Received ' + data.length + ' bytes.'); if (idx++ < countMax) { setTimeout(function () { exchange(idx); }, 500); } }); } Testing AJAX Finally, I put AJAX to test... The Node.js server code was, again, not that different from the previous ones: res.writeHead(200, {'Content-Type' : 'text/plain'}); res.end('_testcb(\'{"message": "' + random_string(4096) + '"}\')'); As for the client code, this is what I used to test: function exchange(idx) { var start_time = new Date(); $.ajax({ url : 'http://localhost:8080/', dataType : "jsonp", jsonpCallback : "_testcb", timeout : 300, success : function (data) { var end_time = new Date(); total_time += end_time - start_time; logMsg(total_time + '(ms.) [' + idx + '] - Received ' + data.length + ' bytes.'); if (idx++ < countMax) { setTimeout(function () { exchange(idx); }, 500); } }, error : function (jqXHR, textStatus, errorThrown) { alert('Error: ' + textStatus + " " + errorThrown); } }); } Remember, when coding together AJAX and Node.js, you need to take into account the you might be doing cross-domain requests and violating same origin policy, therefore you should use the JSONP based format! Btw, as you can see, I quoted only the most significant parts of the test code, to save space. If anyone needs the full code, server and client, please let me know – I’ll be happy to share them. OK – it’s time now to see what we got after all this work! I have run each test for 10, 100, 250 and 500 data exchanges and this is what I got in the end: Data exchanges Socket.IO NON-persistent (ms.) AJAX (ms.) Socket.IO persistent (ms.) 10 90 40 32 100 900 320 340 250 2,400 800 830 500 4,900 1,500 1,600 Looking into the results, we can notice a few things right away: For each type of test, the results behave quite linear; this is good – it shows that the results are consistent. The results clearly show that when using Socket.IO non-persistent connections, the performance numbers are significantly worse than others. It doesn’t seem to be a big difference between AJAX and the Socket.IO persistent connections – we are talking only about some milliseconds differences. This means that if you can live with less than 10,000 data exchanges per day, for example, there are high chances that the user won’t notice a speed difference... The graph below illustrates the numbers I obtained in test: ...So what’s next...? ...Well, I have to figure out what kind of traffic I need to support and then I will re-run the tests for those numbers, but this time excluding Socket.IO non-persistent connections. That’s because it is obvious that I need to choose between AJAX and persistent Socket.IO connections. And I also learned that, most probably, the difference in speed would not be as much as one would expect... at least not for a “small-traffic” web site, so I need to start looking into other advantages and disadvantages for each approach/technology when choosing my solution! That’s pretty much for this post - see you next time with a post about Node.js and caching! P.S. Here are a few more nice resources to find interesting stuff about Node.js, Socket.IO and AJAX: http://socket.io/#how-to-use http://www.hacksparrow.com/jquery-with-node-js.html http://www.slideshare.net/toddeichel/nodejs-talk-at-jquery-pittsburgh http://tech.burningbird.net/article/node-references-and-resources http://davidwalsh.name/websocket
November 22, 2012
by Esen Sagynov
· 17,600 Views
article thumbnail
IBM AIX: Java Process Size Monitoring
This article will provide you with a quick reference guide on how to calculate the Java process size memory footprint for Java VM processes running on IBM AIX 5.3+ OS. This is a complementary post to my original article on this subject: how to monitor the Java native memory on AIX. I highly recommend this read to any individual involved in production support or development of Java applications deployed on AIX. Why is this knowledge important? From my perspective, basic knowledge on how the OS is managing the memory allocation of your JVM processes is very important. We often overlook this monitoring aspect and only focus on the Java heap itself. From my experience, most Java memory related problems are observed from the Java heap itself such as garbage collection problems, leaks etc. However, I’m confident that you will face situations in the future involving native memory problems or OS memory challenges. Proper knowledge of your OS and virtual memory management is crucial for proper root causes analysis, recommendations and solutions. AIX memory vs. pages As you may have seen from my earlier post, the AIX Virtual Memory Manager (VMM) is responsible to manage memory requests from the system and its applications. The actual physical memory is converted and partitioned in units called pages; allocated either in physical RAM or stored on disk until it is needed. Each page can have a size of 4 KB (small page), 64 KB (medium page) or 16 MB (large page). Typically for a 64-bit Java process you will see a mix of all of the above. What about the topas command? The typical reflex when supporting applications on AIX is to run the topas command, similar to Solaris top. Find below an example of output from AIX 5.3: As you can see, the topas command is not very helpful to get a clear view on the memory utilization since it is not providing the breakdown view that we need for our analysis. It is still useful to get a rough idea of the paging space utilization which can give you a quick idea of your top "paging space" consumer processes. Same can be achieved via the ps aux command. AIX OS command to the rescue: svmon The AIX svmon command is by far my preferred command to deep dive into the Java process memory utilization. This is a very powerful command, similar to Solaris pmap. It allows you to monitor the current memory “pages” allocation along with each segment e.g. Java Heap vs. native heap segments. Analyzing the svmon output will allow you to calculate the memory footprint for each page type (4 KB, 64 KB, and 16 MB). Now find below a real example which will allow you to understand how the calculation is done: # 64-bit JVM with -Xms2048m & -Xmx2048m (2 GB Java Heap) # Command: svmon –P As you can see, the total footprint of our Java process size was found at 2.2 GB which is aligned with current Java heap settings. You should be able to easily perform the same memory footprint analysis from your AIX environment I hope this article has helped you to understand how to calculate the Java process size on AIX OS. Please feel free to post any comment or question.
November 22, 2012
by Pierre - Hugues Charbonneau
· 14,208 Views · 1 Like
article thumbnail
Setting Up Custom Instrumentation Using the New Relic Java Agent
New Relic (Remember, there's a free Lite version) lets you identify the slow transactions of an application out of the box. This means you can simply download an agent, start your application running with an agent and soon see slow transactions being reported. While identifying slow transactions is key to performance tuning, there are times when I need more information about a specific transaction that’s not necessarily the slowest. Sometimes I just want to know how many times a specific method is being called. At others, I want timing information on a specific method not automatically instrumented. Or maybe I just want to exclude all calls to a recursive method from a transaction trace. New Relic provides these features through custom instrumentation. Custom Instrumentation Using the New Relic Java Agent Since I work on the New Relic Java agent, this post will focus on the three mechanisms it provides to get metric information about certain transactions. If you’re interested in setting up custom instrumentation with any of our other agents, see the links at the end of this post. One way to set up custom instrumentation with the Java Agent is to use the New Relic API. Our API allows you to create metrics, increment counters, notice errors, ignore transactions, and more within your code. To get started: 1. Simply add the newrelic-api.jar to your class path. 2. Call the appropriate static method from the NewRelic API in your code. 3. Recompile and restart your application with the Java agent. If you prefer annotations, this second option might be for you: 1. Set enable_custom_tracing to true in your newrelic.yml file. Be sure to add the flag if it does not already exist. 2. Add the newrelic-api.jar to your class path. 3. Add the Trace annotation to the method you want to monitor. 4. Recompile and restart your application with the Java agent. If modifying the source code is not possible or desired, you should use New Relic’s third mechanism for custom instrumentation. New Relic allows you to create an Extension XML file specifying the class and method combinations that you want to monitor. New Relic will then read the file on startup and instrument the appropriate classes. Example Lets take a closer look at these three options for custom instrumentation through an example. Suppose you have the following class: package com.example; import java.util.concurrent.Executors; import java.util.concurrent.ScheduledExecutorService; import java.util.concurrent.TimeUnit; public class CustomSample { public void sampleMethod() throws Exception { Runnable myRunnable = new Runnable() { @Override public void run() { firstPart(); secondPart(); } }; ScheduledExecutorService scheduledExecutor = Executors .newScheduledThreadPool(1); scheduledExecutor.scheduleWithFixedDelay(myRunnable, 0, 10000, TimeUnit.MILLISECONDS); } private void firstPart() { System.out.println("In the first part."); } private void secondPart() { System.out.println("In the second part."); } } Option 1 – The New Relic API To use the New Relic API, first put the newrelic-api.jar on your class path. Then the class itself must be modified. Suppose you want to count the number of times the method run is called. This can be accomplished by adding a call to the incrementCounter method in the New Relic API. The one input parameter to this method is the name of the metric. Below I have chosen to call the metric ‘CustomSample.run’. @Override public void run() { firstPart(); secondPart(); NewRelic.incrementCounter(“CustomSample.run”); } Meanwhile, you can use the method recordMetric from the New Relic API to record the amount of time the method firstPart is taking. This method takes in the name of the metric and the value of the metric as parameters. While I have chosen to provide the time below, the value can be anything that fits into a float. For example, I could have set the metric value to a float value resulting from some business logic. If you do choose to measure method times yourself, be sure to use the System.nanoTime() method instead of the System.currentTimeMillis() method. The millisecond time is based on the system clock which can get reset, resulting in start times that are later than stop times. private void firstPart() { long start = System.nanoTime(); System.out.println("In the first part."); long timeDifference = System.nanoTime() -start; NewRelic.recordMetric(“CustomSample.firstPart”, timeDifference); } Once you have made the code changes, recompile and restart your application with the Java agent. Option 2 – Annotations In order to use New Relics’s annotations, remember to set the property enable_custom_tracing to true in your newrelic.yml configuration file and put the newrelic-api.jar on your class path. If you want metrics on the run method, then the Trace annotation needs to be added. Additionally, since this method will likely be the start of a transaction, you need to include ‘dispatcher=true’ as shown below. When this property is set to true, a new transaction is started when the method is reached if a transaction is not already in progress. If the method is encountered after a transaction has been started, then that transaction will continue and a new one will not be created. The dispatcher property is defaulted to false. @Override @Trace(dispatcher=true) public void run() { firstPart(); secondPart(); } To monitor the method firstPart, you also need to add the Trace annotation to this method. Since the method firstPart will be called within the run method, meaning after the transaction has already been started, the dispatcher property can be defaulted to false. @Trace private void firstPart() { System.out.println("In the first part."); } Once you have made the code changes, recompile and restart your application with the Java agent. Option 3 – XML Extension Files In order to use XML extension files, you first need to create the XML file. XML extension files must follow the schema definition newrelic-extension.xsd which can be found within the Java agent zip file starting with version 2.10.0. You will also find an example file called newrelic-extension-example.xml which instruments some of the methods found in the JDK. Here are a few pointers when creating the file: 1. Be sure to always include a name and version. If you have two XML extension files with the same name, only the one with the higher version will be implemented. 2. The metricPrefix is an optional parameter on the instrumentation node which is used as part of the metric name. If not set, it is defaulted to ‘CUSTOM”. 3. Methods to be monitor are put into point cuts. A separate point cut must be used for each class. However, multiple methods from that class can be listed within the same point cut. The XML extension file for instrumenting the run method is shown below. Since run is actually in an inner class, the class name is com.example.CustomSample$1. Additionally, since run should be the start of a transaction, the attribute transactionStartPoint is set to true on the point cut node. com.example.CustomSample$1 run To also monitor the firstPart and secondPart methods from the CustomSample class, a point cut needs to be added to the XML above. This point cut is shown below. Note that the attribute transactionStartPoint is left to its default of false since both these methods should be called from within the run method and thus will be called within a transaction. Also note that if you wanted to exclude these methods from the stack trace, you could set the attribute excludeFromTransactionTrace to true on the point cut node. com.example.CustomSample firstPart secondPart Once the XML extension file is complete, be sure to validate the file. This can be accomplished with the following command. java -jar newrelic.jar instrument -file /path/to/extension.xml This validation application will check the XML syntax and validate that all classes and methods found in the XML file are present on the classpath. It will then either print out ‘PASS’ or ‘FAIL’ to the console. If the XML file fails validation, the reason it failed will also be printed out. Once the XML file is valid, set the property ‘extensions.dir’ in your newrelic.yml file to the directory where your XML file is located. Then restart your application with the agent. Conclusion I find these options for custom instrumentation to be very useful. However, a word of caution: custom instrumentation is designed to be used for only a few methods. You should not try to instrument every method of every class. Doing so will slow down the performance of your application. That said, I encourage you to try out custom instrumentation for yourself. More examples and documentation on the Java agent can be found here.
November 21, 2012
by Leigh Shevchik
· 14,752 Views
article thumbnail
Using JAXB With XSLT to Produce HTML
JAXB (JSR-222)enables Java to treat a domain model as XML. In this post I will demonstrate how to leverage this by applying an XSLT stylesheet to a JAXB model to produce HTML. This approach is often leveraged when creating JAX-RS applications. Java Model Below is the Java model that will be used for this example. It represents information about a collection of books. Library package blog.jaxbsource.xslt; import java.util.*; import javax.xml.bind.annotation.*; @XmlRootElement public class Library { private List books = new ArrayList(); @XmlElement(name="book") public List getBooks() { return books; } } Book package blog.jaxbsource.xslt; public class Book { private String title; private String author; public String getTitle() { return title; } public void setTitle(String title) { this.title = title; } public String getAuthor() { return author; } public void setAuthor(String author) { this.author = author; } } XML Structure Our JAXB model can be used to produce XML documents that conform to the following XML schema. The above XML schema was produced using the following code. package blog.jaxbsource.xslt; import java.io.IOException; import javax.xml.bind.*; import javax.xml.transform.Result; import javax.xml.transform.stream.StreamResult; public class GenSchema { public static void main(String[] args) throws Exception { JAXBContext jc = JAXBContext.newInstance(Library.class); jc.generateSchema(new SchemaOutputResolver() { @Override public Result createOutput(String namespaceURI, String suggestedFileName) throws IOException { StreamResult result = new StreamResult(System.out); result.setSystemId(suggestedFileName); return result; } }); } } Stylesheet Below is the XSLT stylesheet that we will use to convert the XML (JAXB model) to HTML. My Library TitleAuthor Demo Code In our demo code we will use the standard javax.xml.transform APIs to do the conversion. Our JAXB input is wrapped in an instance of JAXBSource, and the output is wrapped in an instance of StreamResult. package blog.jaxbsource.xslt; import javax.xml.bind.*; import javax.xml.bind.util.JAXBSource; import javax.xml.transform.*; import javax.xml.transform.stream.*; public class Demo { public static void main(String[] args) throws Exception { // XML Data Book book1 = new Book(); book1.setAuthor("Jane Doe"); book1.setTitle("Some Book"); Book book2 = new Book(); book2.setAuthor("John Smith"); book2.setTitle("Another Novel"); Library catalog = new Library(); catalog.getBooks().add(book1); catalog.getBooks().add(book2); // Create Transformer TransformerFactory tf = TransformerFactory.newInstance(); StreamSource xslt = new StreamSource( "src/blog/jaxbsource/xslt/stylesheet.xsl"); Transformer transformer = tf.newTransformer(xslt); // Source JAXBContext jc = JAXBContext.newInstance(Library.class); JAXBSource source = new JAXBSource(jc, catalog); // Result StreamResult result = new StreamResult(System.out); // Transform transformer.transform(source, result); } } Output The following HTML output was produced as a result of running the demo code. My Library TitleAuthorSome BookJane DoeAnother NovelJohn Smith
November 21, 2012
by Blaise Doughan
· 23,584 Views
article thumbnail
API Server Design - Making De-Normalization the Norm
In database design classes in Computer Science, we learn that normalization is a good thing. And it certainly is a good thing, for databases. In the case of APIs, it is a different story. If a client must do multiple GETs to obtain the data it needs, or multiple PUTs or POSTs to send up data, just because your database happens to be normalized, then something is wrong. One of the functions of an API Server is to de-normalize your data so that clients are spared from making extra REST API calls, with all of the overhead which goes with that. Mugunth Kumar explains this very well in this excellent presentation, using Twitter as an example. When you do a GET on a tweet, it not only returns you the Tweet itself, but also other information (e.g. description of the Twitter user who sent the tweet). This saves the API client (often a mobile app) from making another request for that data. Effectively, the API Server has gathered up that data, which may come from different database tables, and de-normalized it for the response. You can try it out yourself here, by looking at the JSON which comes back from this Twitter API GET the most recent Tweet from my timeline. Many Vordel customers are using the API Server to gather together the data which is returned to the API clients, often taking this data from multiple sources (not only databases, but also message queues and even from other APIs). This data is then amalgamated into single JSON or XML structures. It often then cached at the API Server, in this structure. In this way, clients are spared from doing multiple calls, and instead (like the Twitter API example above) get the data they need in one request, or can PUT or POST up data in one action, rather than piecemeal. De-normalization is key to this process, and is one of the great benefits of an API Server.
November 21, 2012
by Oren Eini
· 9,965 Views
article thumbnail
5 Good and useful .NET Profilers
I have been making some R&D on the tools or the profilers to test the performance of one of my .NET application. Below are list of some of the compiled collection of .NET Profilers that I could gather. 1. JetBrains dotTrace JetBrains dotTrace is a performance and memory profilers for .NET apps from JetBrains. It lets you to profile the performance of the applications targeting .NET 1.0 to 4.5 and detect bottlenecks quickly. Know more about JetBrains dotTrace at JetBrains dotTrace’s product website 2. ANTS Performance Profiler ANTS Performance Profiler is from red gate and lets you profile .NET, ASP.NET and MVC apps easily and lets you optimize .NET application performance quickly. Know more about ANTS Performance Profiler at ANTS Performance Profiler’s product page 3. EQATEC Profiler Another good .NET Profilers and comes in different versions like free, standard, professional, corporate. Know more about EQATEC Profiler and download it from EQATEC Profiler’s product download page 4. Telerik Just Trace Telerik Just Trace is from Telerik and helps you identify memory leaks and also resolve the performance issues easily. Know more about Telerik Just Trace at Telerik Just Trace’s product page 5. .NET Memory Profiler An in depth .NET Memory Profiling tool and lets you find memory leaks and automate memory testing. Know more about .NET Memory Profiler from .NET Memory Profiler’s product page The above list is just a partial one that I was able to explore and list them. If you feel that there are any other .NET Profilers to be included to the above list, please add them in the comments section.
November 21, 2012
by Senthil Kumar
· 57,258 Views
article thumbnail
Testing Legacy Code With Golden Master
As a warm up for SCNA, the Chicago Software Craftsmanship Community ran a hands-on coding session where developers, working in pairs, should test and refactor some legacy code. For that they used the Gilded Rose kata. You can find links to versions in java, C# and ruby here and for clojure here. We ran the same session for the London Software Craftsmanship Community (LSCC) early this year and back then I decided to write my tests BDD-style (I used JBehave for that). You can check my solution here. This time, instead of writing unit tests or BDD / Spec By Example to test every branch of that horrible code, I decided to solve it using a test style called Golden Master. The Golden Master approach Before making any change to the production code, do the following: Create X number of random inputs, always using the same random seed, so you can generate always the same set over and over again. You will probably want a few thousand random inputs. Bombard the class or system under test with these random inputs. Capture the outputs for each individual random input When you run it for the first time, record the outputs in a file (or database, etc). From then on, you can start changing your code, run the test and compare the execution output with the original output data you recorded. If they match, keep refactoring, otherwise, revert back your change and you should be back to green. Approval Tests An easy way to do Golden Master testing in Java (also available to C# and Ruby) is to use Approval Tests. It does all the file handling for you, storing and comparing it. Here is an example: package org.craftedsw.gildedrose; import java.util.ArrayList; import java.util.List; import java.util.Random; import org.approvaltests.Approvals; import org.junit.Before; import org.junit.Test; public class GildedRoseTest { private static final int FIXED_SEED = 100; private static final int NUMBER_OF_RANDOM_ITEMS = 2000; private static final int MINIMUM = -50; private static final int MAXIMUN = 101; private String[] itemNames = {"+5 Dexterity Vest", "Aged Brie", "Elixir of the Mongoose", "Sulfuras, Hand of Ragnaros", "Backstage passes to a TAFKAL80ETC concert", "Conjured Mana Cake"}; private Random random = new Random(FIXED_SEED); private GildedRose gildedRose; @Before public void initialise() { gildedRose = new GildedRose(); } @Test public void should_generate_update_quality_output() throws Exception { List items = generateRandomItems(NUMBER_OF_RANDOM_ITEMS); gildedRose.updateQuality(items); Approvals.verify(getStringRepresentationFor(items)); } private List generateRandomItems(int totalNumberOfRandomItems) { List items = new ArrayList(); for (int cnt = 0; cnt < totalNumberOfRandomItems; cnt++) { items.add(new Item(itemName(), sellIn(), quality())); } return items; } private String itemName() { return itemNames[0 + random.nextInt(itemNames.length)]; } private int sellIn() { return randomNumberBetween(MINIMUM, MAXIMUN); } private int quality() { return randomNumberBetween(MINIMUM, MAXIMUN); } private int randomNumberBetween(int minimum, int maximum) { return minimum + random.nextInt(maximum); } private String getStringRepresentationFor(List items) { StringBuilder builder = new StringBuilder(); for (Item item : items) { builder.append(item).append("\r"); } return builder.toString(); } } For those not familiar with the kata, after passing a list of items to the GildedRose class, it will iterate through them and according to many different rules, it will change their "sellIn" and "quality" attributes. I've made a small change in the Item class, adding a automatically generated toString() method to it: public class Item { private String name; private int sellIn; private int quality; public Item(String name, int sellIn, int quality) { this.setName(name); this.setSellIn(sellIn); this.setQuality(quality); } // all getters and setters here @Override public String toString() { return "Item [name=" + name + ", sellIn=" + sellIn + ", quality=" + quality + "]"; } } The first time the test method is executed, the line: Approvals.verify(getStringRepresentationFor(items)); will generate a text file, in the same folder where the test class is, called: GildedRoseTest.should_generate_update_quality_output.received.txt. That mean, ..received.txt ApprovalTests then will display the following message in the console: To approve run : mv /Users/sandromancuso/development/projects/ java/gildedrose_goldemaster/./src/test/java/org/craftedsw/ gildedrose/GildedRoseTest.should_generate_update_quality_output.received.txt /Users/sandromancuso/development/projects/java/gildedrose_goldemaster/./src/ test/java/org/craftedsw/gildedrose/GildedRoseTest.should_generate_update_quality_output.approved.txt Basically, after inspecting the file, if we are happy, we just need to change the .received with .approved to approve the output. Once this is done, every time we run the test, ApprovalTests will compare the output with the approved file. Here is an example of how the file looks like: Item [name=Aged Brie, sellIn=-23, quality=-44] Item [name=Elixir of the Mongoose, sellIn=-9, quality=45] Item [name=Conjured Mana Cake, sellIn=-28, quality=1] Item [name=Aged Brie, sellIn=10, quality=-2] Item [name=+5 Dexterity Vest, sellIn=31, quality=5] Now you are ready to rip the GildedRose horrible code apart. Just make sure you run the tests every time you make a change. :) Infinitest If you are using Eclipse or IntelliJ, you can also use Infinitest. It automatically runs your tests every time you save a production or test class. It is smart enough to run just the relevant tests and not the entire test suite. In Eclipse, it displays a bar at the bottom-left corner that can be red, green or yellow (in case there are compilation errors and the tests can't be run). With this, approach, refactoring legacy code becomes a piece of cake. You make a change, save it, look at the bar at the bottom of the screen. If it is green, keep refactoring, if it is red, just hit CTRL-Z and you are back in the green. Wonderful. :) Thanks Thanks to Robert Taylor and Balint Pato for showing me this approach for the first time in one of the LSCC meetings early this year. It was fun to finally do it myself.
November 19, 2012
by Sandro Mancuso
· 8,292 Views
article thumbnail
Using CSS/HTML to Make a Responsive Website in 3 Easy Steps
Today, a website must not look good only on a desktop screen, but also on tablets and smartphones. A website is responsive if it is able to adapt to the screen of the client. In this article, I’ll show you how to easily make a website responsive in three easy steps. 1 – The layout When building a responsive website, or making responsive an existing site, the first element to look at is the layout. When I build responsive websites, I always start by creating a non-responsive layout, fixed at the default size. For example, CatsWhoCode.com default width is 1100px. When I’m pleased with the non-responsive version, I add media queries and slight changes to my code to make the code responsive. It’s way easier to focus on one task at a time. When you’re done with your non-responsive website, the first thing to do is to paste the following lines within the and tags on your html page. This will set the view on all screens at a 1×1 aspect ratio and remove the default functionality from iPhones and other smartphone browsers which render websites at full-view and allow users to zoom into the layout by pinching. It’s now time to add some media queries. According to the W3C site, a media query consists of a media type and zero or more expressions that check for the conditions of particular media features. By using media queries, presentations can be tailored to a specific range of output devices without changing the content itself. In other words, media queries allows your website to look good on all kinds of displays, from smartphones to big screens. Media queries depends of your website layout, so it’s kinda difficult for me to provide you a ready to use code snippet. However, the code below is a good starting point for most websites. In this example, #primary is the main content area, and #secondary the sidebar. By having a look at the code, you can see that I defined two sizes: The first have a maximum width of 1060px and is optimized for tablet landscape display. #primary occupies 67% of its parent container, and #secondary 30%, plus a 3% left margin. The second size is designed for tablet portrait and smaller sizes. Due to the small sizes of smartphones screens, I decided to give #primary a 100% width. #secondary also have a 100% width, and will be displayed below #primary. As I already said, you’ll probably have to adapt this code a bit to fit the specific needs of your website. Paste it on your site .css file. /* Tablet Landscape */ @media screen and (max-width: 1060px) { #primary { width:67%; } #secondary { width:30%; margin-left:3%;} } /* Tabled Portrait */ @media screen and (max-width: 768px) { #primary { width:100%; } #secondary { width:100%; margin:0; border:none; } } Once done, let’s see how responsive your layout is. To do so, I use this awesome tool created by Matt Kersley. 2 – Medias A responsive layout is the first step to a fully responsive website. Now, let’s focus on a very important aspect of a modern website: medias, such as videos or images. The CSS code below will ensure that your images will never be bigger than their parent container. It’s super simple and it works for most websites. Please note that the max-width directive is not recognized by older browsers such as IE6. In order to work, this code snippet have to be inserted into your CSS stylesheet. img { max-width: 100%; } Although the technique above is efficient, sometimes you may need to have more control over images and display a different image according to the client display size. Here is a technique developed by Nicolas Gallagher. Let’s start with the html: As you can see, we used the data-* attribute to store replacement images urls. Now, let’s use the full power of CSS3 to replace the default image by one of the specified replacement images if the min-device-width condition is matched: @media (min-device-width:600px) { img[data-src-600px] { content: attr(data-src-600px, url); } } @media (min-device-width:800px) { img[data-src-800px] { content: attr(data-src-800px, url); } } Impressive, isn’t it? Now let’s have a look to another very important media in today’s websites, videos. As most websites are using videos from third parties sites such as YouTube or Vimeo, I decided to focus on the elastic video technique by Nick La. This technique allows you to make embedded videos responsive. The html: And now, the CSS: .video-container { position: relative; padding-bottom: 56.25%; padding-top: 30px; height: 0; overflow: hidden; } .video-container iframe, .video-container object, .video-container embed { position: absolute; top: 0; left: 0; width: 100%; height: 100%; } Once you applied this code to your website, embedded videos are now responsive. 3 – Typography The last step of this tutorial is definitely important, but it is often neglected by developers when it comes to responsive websites: Typography. Until now, most developers (including myself!) used pixels to define font sizes. While pixels are ok when your website has a fixed width, a responsive website should have a responsive font. Indeed, a responsive font size should be related to its parent container width, so it can adapt to the screen of the client. The CSS3 specification included a new unit named rems. They work almost identically to the em unit, but are relative to the html element, which make them a lot easier to use than ems. As rems are relative to the html element, don’t forget to reset html font size: html { font-size:100%; } Once done, you can define responsive font sizes as shown below: @media (min-width: 640px) { body {font-size:1rem;} } @media (min-width:960px) { body {font-size:1.2rem;} } @media (min-width:1100px) { body {font-size:1.5rem;} } Please note that the rem unit is not recognized by older browers, so don’t forget to implement a fallback. That’s all for today – I hope you enjoyed this tutorial!
November 19, 2012
by Jean-Baptiste Jung
· 532,332 Views · 3 Likes
article thumbnail
A Web Server in C
I implemented a web server in C language using only the standard libraries and thought it would be useful for you guys if I share the code. The server runs on Linux and includes features like handling HTTP GET request, handling content types(txt, html, jpg, zip. rar, pdf, php etc.), sending proper HTTP error codes, serving the files from a web root, change in web root in a config file, zero copy optimization using sendfile method and php file handling. A port number should be provided as a command line argument. After the server is up and running you can request for files using a web browser like Firefox. For an example assume port number is "9000" and if you want to request a file called "test.php" which is in the webroot, use http://localhost:9000/test.php WebServer.c /* * WebServer.c * * Created on: Nov 3, 2012 * Author: pavithra * * A web server in C language using only the standard libraries. * The port number is passed as an argument. * */ #include #include #include #include #include #include #include #include #include #define EOL "\r\n" #define EOL_SIZE 2 typedef struct { char *ext; char *mediatype; } extn; //Possible media types extn extensions[] ={ {"gif", "image/gif" }, {"txt", "text/plain" }, {"jpg", "image/jpg" }, {"jpeg","image/jpeg"}, {"png", "image/png" }, {"ico", "image/ico" }, {"zip", "image/zip" }, {"gz", "image/gz" }, {"tar", "image/tar" }, {"htm", "text/html" }, {"html","text/html" }, {"php", "text/html" }, {"pdf","application/pdf"}, {"zip","application/octet-stream"}, {"rar","application/octet-stream"}, {0,0} }; /* A helper function */ void error(const char *msg) { perror(msg); exit(1); } /* A helper function */ int get_file_size(int fd) { struct stat stat_struct; if (fstat(fd, &stat_struct) == -1) return (1); return (int) stat_struct.st_size; } /* A helper function */ void send_new(int fd, char *msg) { int len = strlen(msg); if (send(fd, msg, len, 0) == -1) { printf("Error in send\n"); } } /* This function recieves the buffer until an "End of line(EOL)" byte is recieved */ int recv_new(int fd, char *buffer) { char *p = buffer; // Use of a pointer to the buffer rather than dealing with the buffer directly int eol_matched = 0; // Use to check whether the recieved byte is matched with the buffer byte or not while (recv(fd, p, 1, 0) != 0) // Start receiving 1 byte at a time { if (*p == EOL[eol_matched]) // if the byte matches with the first eol byte that is '\r' { ++eol_matched; if (eol_matched == EOL_SIZE) // if both the bytes matches with the EOL { *(p + 1 - EOL_SIZE) = '\0'; // End the string return (strlen(buffer)); // Return the bytes recieved } } else { eol_matched = 0; } p++; // Increment the pointer to receive next byte } return (0); } /* A helper function: Returns the web root location. */ char* webroot() { // open the file "conf" for reading FILE *in = fopen("conf", "rt"); // read the first line from the file char buff[1000]; fgets(buff, 1000, in); // close the stream fclose(in); char* nl_ptr = strrchr(buff, '\n'); if (nl_ptr != NULL) *nl_ptr = '\0'; return strdup(buff); } /* Handles php requests */ void php_cgi(char* script_path, int fd) { send_new(fd, "HTTP/1.1 200 OK\n Server: Web Server in C\n Connection: close\n"); dup2(fd, STDOUT_FILENO); char script[500]; strcpy(script, "SCRIPT_FILENAME="); strcat(script, script_path); putenv("GATEWAY_INTERFACE=CGI/1.1"); putenv(script); putenv("QUERY_STRING="); putenv("REQUEST_METHOD=GET"); putenv("REDIRECT_STATUS=true"); putenv("SERVER_PROTOCOL=HTTP/1.1"); putenv("REMOTE_HOST=127.0.0.1"); execl("/usr/bin/php-cgi", "php-cgi", NULL); } /* This function parses the HTTP requests, arrange resource locations, check for supported media types, serves files in a web root, sends the HTTP error codes. */ int connection(int fd) { char request[500], resource[500], *ptr; int fd1, length; if (recv_new(fd, request) == 0) { printf("Recieve Failed\n"); } printf("%s\n", request); // Check for a valid browser request ptr = strstr(request, " HTTP/"); if (ptr == NULL) { printf("NOT HTTP !\n"); } else { *ptr = 0; ptr = NULL; if (strncmp(request, "GET ", 4) == 0) { ptr = request + 4; } if (ptr == NULL) { printf("Unknown Request ! \n"); } else { if (ptr[strlen(ptr) - 1] == '/') { strcat(ptr, "index.html"); } strcpy(resource, webroot()); strcat(resource, ptr); char* s = strchr(ptr, '.'); int i; for (i = 0; extensions[i].ext != NULL; i++) { if (strcmp(s + 1, extensions[i].ext) == 0) { fd1 = open(resource, O_RDONLY, 0); printf("Opening \"%s\"\n", resource); if (fd1 == -1) { printf("404 File not found Error\n"); send_new(fd, "HTTP/1.1 404 Not Found\r\n"); send_new(fd, "Server : Web Server in C\r\n\r\n"); send_new(fd, ""); send_new(fd, "404 Not Found: The requested resource could not be found!"); //Handling php requests } else if (strcmp(extensions[i].ext, "php") == 0) { php_cgi(resource, fd); sleep(1); close(fd); exit(1); } else { printf("200 OK, Content-Type: %s\n\n", extensions[i].mediatype); send_new(fd, "HTTP/1.1 200 OK\r\n"); send_new(fd, "Server : Web Server in C\r\n\r\n"); if (ptr == request + 4) // if it is a GET request { if ((length = get_file_size(fd1)) == -1) printf("Error in getting size !\n"); size_t total_bytes_sent = 0; ssize_t bytes_sent; while (total_bytes_sent < length) { //Zero copy optimization if ((bytes_sent = sendfile(fd, fd1, 0, length - total_bytes_sent)) <= 0) { if (errno == EINTR || errno == EAGAIN) { continue; } perror("sendfile"); return -1; } total_bytes_sent += bytes_sent; } } } break; } int size = sizeof(extensions) / sizeof(extensions[0]); if (i == size - 2) { printf("415 Unsupported Media Type\n"); send_new(fd, "HTTP/1.1 415 Unsupported Media Type\r\n"); send_new(fd, "Server : Web Server in C\r\n\r\n"); send_new(fd, ""); send_new(fd, "415 Unsupported Media Type!"); } } close(fd); } } shutdown(fd, SHUT_RDWR); } int main(int argc, char *argv[]) { int sockfd, newsockfd, portno, pid; socklen_t clilen; struct sockaddr_in serv_addr, cli_addr; if (argc < 2) { fprintf(stderr, "ERROR, no port provided\n"); exit(1); } sockfd = socket(AF_INET, SOCK_STREAM, 0); if (sockfd < 0) error("ERROR opening socket"); bzero((char *) &serv_addr, sizeof(serv_addr)); portno = atoi(argv[1]); serv_addr.sin_family = AF_INET; serv_addr.sin_addr.s_addr = INADDR_ANY; serv_addr.sin_port = htons(portno); if (bind(sockfd, (struct sockaddr *) &serv_addr, sizeof(serv_addr)) < 0) error("ERROR on binding"); listen(sockfd, 5); clilen = sizeof(cli_addr); /* Server runs forever, forking off a separate process for each connection. */ while (1) { newsockfd = accept(sockfd, (struct sockaddr *) &cli_addr, &clilen); if (newsockfd < 0) error("ERROR on accept"); pid = fork(); if (pid < 0) error("ERROR on fork"); if (pid == 0) { close(sockfd); connection(newsockfd); exit(0); } else close(newsockfd); } /* end of while */ close(sockfd); return 0; /* we never get here */ } Make sure you have installed php and there's exist a "conf" file consisted of the webroot before running this. Enjoy!
November 17, 2012
by Pavithra Gunasekara
· 43,873 Views · 1 Like
  • Previous
  • ...
  • 1567
  • 1568
  • 1569
  • 1570
  • 1571
  • 1572
  • 1573
  • 1574
  • 1575
  • 1576
  • ...
  • Next
  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook
×