DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

The Latest Integration Topics

article thumbnail
OpenHFT Java Lang Project
Overview OpenHFT/Java Lang started as an Apache 2.0 library to provide the low level functionality used by Java Chronicle without the need to persist to a file. This allows serializable and deserialization of data and random access to memory in native space (off heap) It supports writing and reading enumerable types with object pooling. e.g. writing and reading String without creating an object (if it has been pooled). It also supports writing and read primitive types in binary and text without creating any garbage. Small messages can be serialized and deserialized in under a micro-second. Recent additions Java Lang supports a DirectStore which is like a ByteBuffer but can be any size (up to 40 to 48-bit on most systems) It support 64-bit sizes and offset. It support compacted types, and object serialization. It also supports thread safety features such as volatile reads, ordered (lazy) writes, CAS operations and using an int (4 bytes) as a lock in native memory. Testing a native memory lock in Java This test has one lock and a value which is toggled. One thread changes the value from 0 to 1 and the other switches it from 1 to 0. This goes around 20 million times, but has been run for longer final DirectStore store1 = DirectStore.allocate(1L << 12); final int lockCount = 20 * 1000 * 1000; new Thread(new Runnable() { @Override public void run() { manyToggles(store1, lockCount, 1, 0); } }).start(); manyToggles(store1, lockCount, 0, 1); store1.free(); The manyToggles method is more interesting. Note is using the 4 bytes at offset 0 as a lock. You can arrange any number of locks in native space this way. E.g. you might have fixed length records and want to be able to lock them before updating or access them. You can place a lock at the "head" of the record. private void manyToggles(DirectStore store1, int lockCount, int from, int to) { long id = Thread.currentThread().getId(); assertEquals(0, id >>> 24); System.out.println("Thread " + id); DirectBytes slice1 = store1.createSlice(); for (int i = 0; i < lockCount; i++) { assertTrue( slice1.tryLockNanosInt(0L, 10 * 1000 * 1000)); int toggle1 = slice1.readInt(4); if (toggle1 == from) { slice1.writeInt(4L, to); } else { i--; } slice1.unlockInt(0L); } } The size of the DataStore and the offsets within it are long, allowing you to allocate a continuous block of native memory into the many GB, and access it as you required. On my 2.6 GHz i5 laptop I get the following output for this test Contended lock rate was 9,096,824 per second This looks great but under heavy contention, one thread can be staved out. This is more useful for lots of locks and lower contention. Note: if I drop the timeout from 10 ms to 1 ms, it eventually fails meaning sometimes it takes more then 1 ms to get a lock! Conclusion The Java Lang library is taking the step of making it easier to use native memory with the same functionality available on the heap. The language support is not as good, but if you need to store say 128 GB of data you will get a much better GC behavior using off heap memory.
July 11, 2013
by Peter Lawrey
· 16,856 Views
article thumbnail
Easy Messaging with STOMP over WebSockets using ActiveMQ and HornetQ
Expose two very popular JMS implementations, Apache ActiveMQand JBoss HornetQ, to be available to web front-end (JavaScript) using STOMP over Websockets.
July 2, 2013
by Andriy Redko
· 52,865 Views · 3 Likes
article thumbnail
Spire.Barcode for .NET
This is a package of C#, VB.NET Example Project for Spire.BarCode for .NET. Spire.BarCode for .NET is a professional and reliable barcode generation and recognition component. It enables developers to quickly and easily add barcode generation and recognition functionality to their Microsoft .NET applications (ASP.NET, WinForms and .NET) and it supports in C#, VB.NET. Spire.Barcode for.NET is 100% FREE barcode component. First Glance of Spire.BarCode for .NET http://www.e-iceblue.com/images/url/BarCode.png Spire.BarCode for .NET Main Features: Supports rich Barcode types, more than 37 different barcodes. • Code bar Barcode • Code 1 of 1 Barcode • Standard 2 of 5 barcode • Code 3 of 9 barcode • Extended Code 3 of 9 barcode • Code 9 of 3 Barcode • Extended Code 9 of 3 Barcode • Code 128 barcode • EAN-8 barcode • EAN-13 barcode • EAN-128 barcode • EAN-14 barcode • SCC14 barcode • SSCC18 barcode • ITF14 Barcode • ITF-6 Barcode • UPCA barcode • UPCE barcode • Postnet barcode • Planet barcode • MSI barcode • 2D Barcode DataMatrix • QR Code barcode • Pdf417 barcode • Pdf417 Macro barcode • RSS14 barcode • RSS-14 Truncated barcode • RSS Limited Barcode • RSS Expanded Barcode • USPS OneCode barcode • Swiss Post Parcel Barcode • PZN Barcode • OPC(Optical Product Code) Barcode • Deutschen Post Barcode • Deutsche Post Leitcode Barcode • Royal Mail 4-state Customer Code Barcode • Singapore Post Barcode 1.Robust Barcode Recognize and Generation 1D & 2D Barcode. Developers can read most often used Linear, 2D and Postal barcodes, detecting them anywhere, with any orientation. 2.High performance for generating and reading barcode image Developers can create barcode images in any desired output image format like Bitmap, JPG, PNG, EMF, TIFF, GIF and WMF. 3.Superior performance support for reading and writing barcode Developers can easily set barcode image borders, border colors, style, margins and width. You can also rotate barcode images to any angle and produce high quality barcode images. 4.Easy Integration Spire.Barcode for .NET can be easily integrated into any .net applications. There are two main ways to integrate Spire.Barcode in .NET applications, API Mode and Component Mode. • API Mode is just one line of code to create, recognizes barcode. • Component Mode use Visual way to create barcode, then drag Spire.Barcode component to your .NET, Windows or ASP.NET Form. No more code needs. Download Spire.BarCode: Spire.BarCode for .NET is a free barcode library used in .NET applications (in ASP.NET, WinForms and .NET). And you can download Spire.BarCode for .NET and install it on your system. With Spire.BarCode, you can add Enterprise-level barcode formats to your NET applications easily and quickly. Feedback and Support E-iceblue welcomes any kind of questions, bug reports and feedback about this product from our customers. As long as you leave them on Spire.BarCode for .NET Forum or contact us by e-mail, we will offer full support within one business day. Related Links Website:www.e-iceblue.com Product Introduction: http://www.e-iceblue.com/Introduce/barcode-for-net-introduce.html#.UdEuk9hp4Zu Download: http://www.e-iceblue.com/Download/download-barcode-for-net-now.html . Spire.BarCode for .NET is a FREE and professional barcode component specially designed for .NET developers (C#, VB.NET, ASP.NET) to generate, read 1D & 2D barcodes. Developers and programmers can use Spire.BarCode to add Enterprise-Level barcode formats to their .net applications (ASP.NET, WinForms and Web Service) quickly and easily. Spire.BarCode for .NET provides a very easy way to integrate barcode processing. With just one line of code to create, read 1D & 2D barcode, Spire.BarCode supports variable common image formats, such as Bitmap, JPG, PNG, EMF, TIFF, GIF and WMF. Spire.BarCode for .NET is 100% FREE BarCode component, no risk to integrate in your .NET application.
July 1, 2013
by Chen Steve
· 5,235 Views
article thumbnail
Integration of Amazon Redshift Data Warehouse with Talend Data Integration
In this blog post, I will show you how to "ETL" all kinds of data to Amazon’s cloud data warehouse Redshift wit Talend’s big data components. Let’s begin with a short introduction to Amazon Redshift (copied from website): "Amazon Redshift is [part of Amazon Web Services (AWS) and] a fast and powerful, fully managed, petabyte-scale data warehouse service in the cloud. With a few clicks in the AWS Management Console, customers can launch a Redshift cluster, starting with a few hundred gigabytes and scaling to a petabyte or more, for under $1,000 per terabyte per year. Traditional data warehouses require significant time and resource to administer, especially for large datasets. In addition, the financial cost associated with building, maintaining, and growing self-managed, on-premise data warehouses is very high. Amazon Redshift not only significantly lowers the cost of a data warehouse, but also makes it easy to analyze large amounts of data very quickly.“ Sounds interesting! And indeed, we already see companies using Talend’s Redshift connectors. From Talend perspective it is not much more than just another database. If you have ever used a Talend connector, you can integrate to Redshift within some minutes. In the next sections, I will describe all necessary steps and give some hints regarding configuration issues and performance improvements. Be aware: You need Talend Open Studio for Data Integration (Apache License, open source) or any Talend Enterprise Edition / Platform which contains the Cloud components to see and use Amazon Redshift connectors. The open source edition offers all connectors and functionality to integrate with Amazon Redshift. However, Enterprise versions offer some more features (e.g. versioning), comfort (e.g. wizards) and commercial support. Setup Amazon Redshift Setup of Amazon Redshift is very easy. Just follow Amazon‘s getting started guide: http://docs.aws.amazon.com/redshift/latest/gsg/welcome.html. Like every other AWS guide, it is very easy to understand and use. Be aware, that you just have to do step 1, 2 and 3 of the getting started guide for using it with Talend. Some hints: - Step 1 („before you begin“): Just sign up. Client tools and drivers are not necessary because they are already installed within Talend Studio. - Step 2 („launch a cluster“): Yes, please start your cluster! - Step 3(„authorize access“): If you are not sure what to do here, select Connection Type = CIDR/IP. Find out your IP address (http://whatismyipaddress.com) and enter it with „/32“ at the end. Example: „192.168.1.1/32“ Now you can connect to Amazon Redshift from your Talend Studio on your local computer. Step 4 (connect) and step 5 (create table, data, queries) are not necessary, this will be done from Talend Studio. Of course, you should not forget to delete your cluster (step 7) when you are done. Otherwise, you will pay for every hour, even if you do not access your DWH. Connect to Amazon Redshift from Talend Studio Create a new connection to Amazon Redshift database as you do with every other relational database. The easiest way is to use „DB Connection Wizard“ in metadata. Just enter your connection information and check if it works. You get all information about configuration from Amazon Web Console. The connection string looks something like this: „jdbc:paraccel://talend-demo-cluster.cp8t6c5.eu-west-1.redshift.amazonaws.com:5439/dev“ Next, right click on the created connection and select „retrieve schema“. „public“ is the default schema which you (have to) use. Now, you are ready to use this connection within Talend Jobs to write to Amazon Redshift and read from it. Create Talend Jobs (Write, Read, Delete) Amazon Redshift components work like any other Talend (relational) database components. Look at www.help.talend.com for more information if you have not used them before (or just try them out, they are very self-explanatory). You just have to drag&drop your connection from metadata . Afterwards, you can easily write data (tRedShiftOutput), read data (tRedshiftInput), or do any other queries such as delete or copy (tRedShiftRow). In the following job, I start with deleting all content in the Amazon Redshift table. Then, I read data from a MySQL table and insert it into an Amazon Redshift table. The table is created automatically (as I have configured it this way). After this subjob is finished, I read the data again, and store it to a CSV file (which is also created automatically). Of course, this is no business use case, but it shows how to use different Amazon Redshift components. Query Data from Amazon Redshift You can connect to Amazon Redshift directly from Talend Studio to explore and query data of the DWH. Thus, no other database tool is required. Just right click on your Amazon Redshift connection in metadata and select „edit queries“. Here you can define, execute and save SQL queries. Improve Performance Write performance of Amazon Redshift is relatively low compared to „classical“ relational databases (in your data center) as you have to upload all data into the cloud. Different alternatives exist to improve performance: - Bulk inserts: „Extended insert“ (in advanced settings) improves performance a lot, but still not to hyperspeed… Also, as it is bulk, you can just do inserts! It is not compatible to „rejects“ or „updates“ - AWS S3 and COPY command: S3 is Amazon’s „simple storage service“, a key-value store – also called NoSQL today – for storing very large objects. You can use Amazon Redshift’s COPY command (http://docs.aws.amazon.com/redshift/latest/dg/r_COPY.html) to transfer data from S3 to Amazon Redshift with good performance. Though, you still have to copy data to S3 before, same „cloud problem“ here. The COPY command can be used with tRedshiftRow, so no problem at all from Talend perspective. To transfer data to S3, you can either use the Talend S3 components from Talendforge, Talend’s open source community (http://www.talendforge.org/exchange), or use camel-s3, an Apache Camel component which is included in Talend ESB. The latter is an option, if you use Talend Data Services which combines Talend DI and Talend ESB in its unified platform. Summary You need not be a cloud or DWH expert, or an expert developer to integrate with Amazon’s cloud data warehouse Redshift. It is very easy with Talend’s integration solutions. Just drag&drop, configure, do some graphical mappings / transformations (if necessary), that’s it. Code is generated. Job runs. You can integrate Amazon Redshift almost as simple as any other relational database. Just be aware of some cloud specific security and performance issues. With Talend, you can easily „ETL“ all data from different sources to Redshift and store it there for under $1,000 per terabyte per year – even with the open source version! Best regards, Kai Wähner (Contact and feedback via @KaiWaehner, www.kai-waehner.de, LinkedIn / Xing) This is content from my blog: http://www.kai-waehner.de/blog/2013/06/26/integration-of-amazon-redshift-cloud-data-warehouse-aws-saas-dwh-with-talend-data-integration-di-big-data-bd-enterprise-service-bus-esb/
June 27, 2013
by Kai Wähner DZone Core CORE
· 20,612 Views · 1 Like
article thumbnail
Implementing Memcached a Servlet Filter for Spring MVC-Based RESTful Services
I have a number of Spring MVC based RESTful services that return JSON. In 90% of the cases, the state of objects these services return will not change within a 24 hour period. This makes them (the JSON objects) perfect candidates for simple caching enabled by memcached. The idea was to have every request to Spring controllers intercepted, cache key generated and checked against the cache. If the key and corresponding value (JSON string) is available (a cache hit), it is returned to the caller as-is without making a full round trip to the database. However, if the cache has no entry for the key and hence no corresponding value (a cache miss), the call is forwarded to the controller, which in turn calls the logic to fetch desired object from the database and not only return it to the caller but also update the cache with the returned content. Keys are generated using the URL of the service in case of GET requests and the URL concatenated with POSTed input (as JSON) in case of POST requests. The resultant strings are encoded with MD5 to come up with a 32 character cache key which is well within the 250 character key length limit of memcached. Performance impact of using MD5 is yet to be evaluated during our load testing cycle. I started off trying to get hold of JSON response in the postHandle method of a Spring HandlerInterceptor. However since we are using @ResponseBody annotation in our controller, the JSON would be written directly to the stream. The ModelAndView was of course null because of this reason. If we removed the annotation and returned ModelAndView from the controller, the intended JSON object got enclosed in a map wrapper. A quick question on stack overflow didn’t help as the only suggestion I got was to extract my original object from the map wrapper. I wanted to keep this option (as discussed here as well ) as my last resort. The solution I eventually came up with involved Replacing the HandlerInterceptor with Servlet Filters Using DelegatingFilterProxy to make my filters spring application context aware Using HttpServletRequestWrapper to get control of the POST request body in the filter on the way in Using HttpServletResponseWrapper to get control of the response content in the filter on the way out True, its probably a more complex solution than just overriding MappingJacksonJsonView and extracting my JSON object, but it is more generic as it does not assume that all my content will always be JSON. Lets first start with the filter definition in the web.xml cacheFilter org.springframework.web.filter.DelegatingFilterProxy ... cacheFilter /* A standard filter configuration except for the fact that the filter class is always going to be org.springframework.web.filter.DelegatingFilterProxy. Where do you specify your own class ? As a bean in your spring context xml. The name of the filter and the name of the bean must be the same for the delegation to happen. Using the DelegatingFilterProxy allowed me to use my Filters with Spring. I can inject my dependencies as I would normally. Next, lets look at my MemcacheFilter filter Memcache Filter Class public class MemcacheFilter implements Filter { private static Logger logger = Logger.getLogger(MemcacheFilter.class); private CacheConfig cacheConfig; /** * Memcached lookup is being performed in this method. Firstly, keys are * generated depending on the request method (GET/POST). Then a cache lookup * is performed. If a value is obtained, the value is written to the * response otherwise, the actual target (in this case, Spring's Dispatcher * Servlet) is called by calling doFilter on the filteChain. The dispatcher * servlet calls the controller to produce the desire response which is * intercepted when the doFilter method returns. The Response is added to * the cache if the reponse code was 200(OK). * * @param request * @param response * @param filterChain * @throws IOException * @throws ServletException */ public void doFilter(ServletRequest request, ServletResponse response, FilterChain filterChain) throws IOException, ServletException { try { if ((request instanceof HttpServletRequest) && (response instanceof HttpServletResponse)) { // Wrapping the response in HTTPServletResponseWrapper MemcacheResponseWrapper responseWrap = new MemcacheResponseWrapper((HttpServletResponse) response); // Wrapping the request in HTTPServletResponseWrapper MemcacheRequestWrapper requestWrap = new MemcacheRequestWrapper((HttpServletRequest) request); // Get Memcached Client Instance MemcachedClient client = cacheConfig.getMemcachedClient(); Key keyGenerator = getKeyGenerator(requestWrap); if (keyGenerator != null) { String key = keyGenerator.getKey(requestWrap, cacheConfig); String value = (String) client.get(key); if (value == null) { // cache miss logger.info("Cache miss for key " + key); // call next filter/actual target for value filterChain.doFilter(requestWrap, responseWrap); if (responseWrap.getStatus() == HttpServletResponse.SC_OK) { // obtaining response content from // HttpServletResponseWrapper value = responseWrap.getOutputStream().toString(); // adding response to cache client.add(key, 0, value); logger.info("Adding response to cache: "+ (value.length() > 50 ? value.substring(0,50) + "..." : value)); } else { logger.warn("Did not add content to cache as response status is not 200"); } } else { // This case is a cache hit logger.info("Cache hit for key " + key); response.getWriter().println(value); } } else { logger.warn("Request skipped because no key generator could be found for the request's method"); // attempting call to actual target filterChain.doFilter(request, response); } } } catch (Exception ex) { logger.info("Cache functionality skipped due to exception", ex); // attempting call to actual target filterChain.doFilter(request, response); } } /** * Factory method that returns KeyGenerator based on the request method. * * @param httpRequest * @return */ private Key getKeyGenerator(HttpServletRequest httpRequest) { Key keyGenerator = null; if (httpRequest.getMethod().equalsIgnoreCase("GET")) { keyGenerator = new GetRequestKey(); } else if (httpRequest.getMethod().equalsIgnoreCase("POST")) { keyGenerator = new PostRequestKey(); } return keyGenerator; } public void init(FilterConfig arg0) throws ServletException { logger.debug("init"); } public CacheConfig getCacheConfig() { return cacheConfig; } public void setCacheConfig(CacheConfig cacheConfig) { this.cacheConfig = cacheConfig; } public void destroy() { logger.debug("destroy"); } } 1. I first wrap my request and response objects in the following statements. I have had to create the wrappers as well. Will get to those later. // Wrapping the response in HTTPServletResponseWrapper MemcacheResponseWrapper responseWrap = new MemcacheResponseWrapper((HttpServletResponse) response); // Wrapping the request in HTTPServletResponseWrapper MemcacheRequestWrapper requestWrap = new MemcacheRequestWrapper((HttpServletRequest) request); 2. Next, I have one of my injected classes, CacheConfig, provide me with a memcache client which I will use later to look up the cache. // Get Memcached Client Instance MemcachedClient client = cacheConfig.getMemcachedClient(); 3. I make a call to a function that tells me which key generator I should use, a GET one or a POST one depending on the request method. Key keyGenerator = getKeyGenerator(requestWrap); /** * Factory method that returns KeyGenerator based on the request method. * * @param httpRequest * @return */ private Key getKeyGenerator(HttpServletRequest httpRequest) { Key keyGenerator = null; if (httpRequest.getMethod().equalsIgnoreCase("GET")) { keyGenerator = new GetRequestKey(); } else if (httpRequest.getMethod().equalsIgnoreCase("POST")) { keyGenerator = new PostRequestKey(); } return keyGenerator; } 4. Check for a cache hit using the Key returned by the Key Generator. If its a miss, call next filter or target to compute actual value, get value from the response wrapper, and add it to the cache. if (keyGenerator != null) { String key = keyGenerator.getKey(requestWrap, cacheConfig); String value = (String) client.get(key); if (value == null) { // cache miss logger.info("Cache miss for key " + key); // call next filter/actual target for value filterChain.doFilter(requestWrap, responseWrap); if (responseWrap.getStatus() == HttpServletResponse.SC_OK) { // obtaining response content from // HttpServletResponseWrapper value = responseWrap.getOutputStream().toString(); // adding response to cache client.add(key, 0, value); logger.info("Adding response to cache: "+ (value.length() > 50 ? value.substring(0,50) + "..." : value)); } 5. If its a cache hit, just get return cached value else { // This case is a cache hit logger.info("Cache hit for key " + key); response.getWriter().println(value); } Lets take a look at each of the Wrappers. I am not going into a a lot of detail into how each of these work. Request Wrapper Class On the way in, the original POST content is extracted from the request and put in a String Buffer. To the filter, this content is returned via the toString() method of the WrappedInputStream class whereas the subsequently called controller calls the read method. public class MemcacheRequestWrapper extends HttpServletRequestWrapper { protected ServletInputStream stream; protected HttpServletRequest origRequest = null; protected BufferedReader reader = null; public MemcacheRequestWrapper(HttpServletRequest request) throws IOException { super(request); origRequest = request; } public ServletInputStream createInputStream() throws IOException { return (new WrappedInputStream(origRequest)); } @Override public ServletInputStream getInputStream() throws IOException { if (reader != null) { throw new IllegalStateException("getReader() has already been called for this request"); } if (stream == null) { stream = createInputStream(); } return stream; } @Override public BufferedReader getReader() throws IOException { if (reader != null) { return reader; } if (stream != null) { throw new IllegalStateException("getReader() has already been called for this request"); } stream = createInputStream(); reader = new BufferedReader(new InputStreamReader(stream)); return reader; } private class WrappedInputStream extends ServletInputStream { private StringBuffer originalInput = new StringBuffer(); private HttpServletRequest originalRequest; private ByteArrayInputStream byteArrayInputStream; public WrappedInputStream(HttpServletRequest request) throws IOException { this.originalRequest = request; BufferedReader bufferedReader = null; try { InputStream inputStream = request.getInputStream(); if (inputStream != null) { bufferedReader = new BufferedReader(new InputStreamReader(inputStream)); char[] charBuffer = new char[128]; int bytesRead = -1; while ((bytesRead = bufferedReader.read(charBuffer)) > 0) { originalInput.append(charBuffer, 0, bytesRead); } } byteArrayInputStream = new ByteArrayInputStream(originalInput.toString().getBytes()); } catch (IOException ex) { throw ex; } finally { if (bufferedReader != null) { try { bufferedReader.close(); } catch (IOException ex) { throw ex; } } } } @Override public String toString() { return this.originalInput.toString(); } @Override public int read() throws IOException { return byteArrayInputStream.read(); } } } Response Wrapper Class The response wrapper is similar to the request wrapper. Instead of the read method, there is a write method, called by the controller when its writing JSON content. This is stored in the wrapper and called in the filter. public class MemcacheResponseWrapper extends HttpServletResponseWrapper { protected ServletOutputStream stream; protected PrintWriter writer = null; protected HttpServletResponse origResponse = null; private int httpStatus = 200; public MemcacheResponseWrapper(HttpServletResponse response) { super(response); response.setContentType("application/json"); origResponse = response; } public ServletOutputStream createOutputStream() throws IOException { return (new WrappedOutputStream(origResponse)); } public ServletOutputStream getOutputStream() throws IOException { if (writer != null) { throw new IllegalStateException("getWriter() has already been called for this response"); } if (stream == null) { stream = createOutputStream(); } return stream; } public PrintWriter getWriter() throws IOException { if (writer != null) { return writer; } if (stream != null) { throw new IllegalStateException("getOutputStream() has already been called for this response"); } stream = createOutputStream(); writer = new PrintWriter(stream); return writer; } @Override public void sendError(int sc) throws IOException { httpStatus = sc; super.sendError(sc); } @Override public void sendError(int sc, String msg) throws IOException { httpStatus = sc; super.sendError(sc, msg); } @Override public void setStatus(int sc) { httpStatus = sc; super.setStatus(sc); } public int getStatus() { return httpStatus; } private class WrappedOutputStream extends ServletOutputStream { private StringBuffer originalOutput = new StringBuffer(); private HttpServletResponse originalResponse; public WrappedOutputStream(HttpServletResponse response) { this.originalResponse = response; } @Override public String toString() { return this.originalOutput.toString(); } @Override public void write(int arg0) throws IOException { originalOutput.append((char) arg0); originalResponse.getOutputStream().write(arg0); } } }
June 25, 2013
by Faheem Sohail
· 22,628 Views · 1 Like
article thumbnail
Resolving SOAPFaultException caused by com.ctc.wstx.exc. WstxUnexpectedCharException
If you’re using any of these tools for Web Services – Axis2, CXF etc. – that internally make use of Woodstox XML processor (wstx), and you're getting an exception like this during webservice calls, javax.xml.ws.soap.SOAPFaultException: Error reading XMLStreamReader. at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:...) ... Caused by: com.ctc.wstx.exc.WstxUnexpectedCharException: Unexpected character ... at com.ctc.wstx.sr.StreamScanner.throwUnexpectedChar(StreamScanner.java:...) at com.ctc.wstx.sr.BasicStreamReader.nextFromProlog(BasicStreamReader.java:...) at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:...) at com.ctc.wstx.sr.BasicStreamReader.nextTag(BasicStreamReader.java:...) the problem is that the wstx tokenizer/parser encountered unexpected (but not necessarily invalid per se) character; character that is not legal in current context. Could happen, for example, if white space was missing between attribute value and name of next attribute, according to API docs (http://woodstox.codehaus.org/3.2.9/javadoc/com/ctc/wstx/exc/WstxUnexpectedCharException.html). This simply means that you’re receiving an ill-formed SOAP XML as response. You need to check the SOAP response construction logic/code at the other end you’re communicating to.
June 24, 2013
by Singaram Subramanian
· 21,102 Views
article thumbnail
Resolving CertPathValidatorException: Path does not chain with any of the trust anchors Error in Axis2
I was getting this error (see below) in one of our axis2 based web service, and this is what I did to resolve it. org.apache.axis2.AxisFault: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors at org.apache.axis2.AxisFault.makeFault(AxisFault.java:430) at org.apache.axis2.transport.http.SOAPMessageFormatter.writeTo(SOAPMessageFormatter.java:83) … Caused by: com.ctc.wstx.exc.WstxIOException: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors at com.ctc.wstx.sw.BaseStreamWriter.flush(BaseStreamWriter.java:313) at org.apache.axiom.om.impl.MTOMXMLStreamWriter.flush(MTOMXMLStreamWriter.java:146) … Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:150) … Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:187) … Caused by: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors at sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:195) at java.security.cert.CertPathValidator.validate(CertPathValidator.java:206) at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:182) … 49 more Solution Axis2 uses commons-httpclient library (now, a part of Apache HttpComponents™ project) for making http/https connections. I’m doing a little tweak for it accept any server certificate like this: (By the way, I didn’t bother at all about whether the certificate was valid, self-signed, or has a valid trust chain) Stub stub = ; . . . //Line #1 org.apache.commons.httpclient.protocol.Protocol.unregisterProtocol("https"); //Line #2 org.apache.commons.httpclient.protocol.Protocol.registerProtocol ("https", new Protocol("https", (ProtocolSocketFactory) new org.apache.commons.httpclient.contrib.ssl.EasySSLProtocolSocketFactory(), 13087)); Line #1: Unregistered the default socket factory for the https URI protocol scheme Line #2: Used a custom socket factory – EasySSLProtocolSocketFactory – used to create SSL connections that allow the target server to authenticate with a self-signed certificate (to put it simple, it accepts any self-signed certificate). Remember, this socket factory SHOULD NOT be used for productive systems due to security reasons, unless it is a concious decision and you are perfectly aware of security implications of accepting self-signed certificates To use this custom socket factory, you need to include not-yet-commons-ssl-0.3.9.jar in classpath and it’s available here (as of writing this post): http://repository.jboss.org/maven2/org/apache/commons/not-yet-commons-ssl/0.3.9/. if you don’t find here, you can google and get it. About the commons-httpclient, it provides full support for HTTP over Secure Sockets Layer (SSL) or IETF Transport Layer Security (TLS) protocols by leveraging the Java Secure Socket Extension (JSSE). JSSE has been integrated into the Java 2 platform as of version 1.4 and works with HttpClient out of the box.
June 20, 2013
by Singaram Subramanian
· 37,732 Views
article thumbnail
Automating Nginx Reverse Proxy Configuration
It’s really nice if you can decouple your external API from the details of application segregation and deployment. In a previous post I explained some of the benefits of using a reverse proxy. On my current project we’ve building a distributed service oriented architecture that also exposes an HTTP API, and we’re using a reverse proxy to route requests addressed to our API to individual components. We have chosen the excellent Nginx web server to serve as our reverse proxy; it’s fast, reliable and easy to configure. We use it to aggregate multiple services exposing HTTP APIs into a single URL space. So, for example, when you type: http://api.example.com/product/pinstripe_suit It gets routed to: http://10.0.1.101:8001/product/pinstripe_suit But when you go to: http://api.example.com/customer/103474783 It gets routed to http://10.0.1.104:8003/customer/103474783 To the consumer of the API it appears that they are exploring a single URL space (http://api.example.com/blah/blah), but behind the scenes the different top level segments of the URL route to different back end servers. /product/… routes to 10.0.1.101:8001, but /customer/… routes to 10.0.1.104:8003. We also want this to be self-configuring. So, say I want to create a new component of the system that records stock levels. Rather than extending an existing component, I want to be able to write a stand-alone executable or service that exposes an HTTP endpoint, have it be automatically deployed to one of the hosts in my cloud infrastructure, and have Nginx automatically route requests addressed http://api.example.com/stock/whatever to my new component. We also want to load balance these back end services. We might want to deploy several instances of our new stock API and have Nginx automatically round robin between them. We call each top level segment ( /stock, /product, /customer ) a claim. A component publishes an ‘AddApiClaim’ message over RabbitMQ when it comes on line. This message has 3 fields: ‘Claim', ‘ipAddress’, and ‘PortNumber’. We have a special component, ProxyAutomation, that subscribes to these messages and rewrites the Nginx configuration as required. It uses SSH and SCP to log into the Nginx server, transfer the various configuration files, and instruct Nginx to reload its configuration. We use the excellent SSH.NET library to automate this. A really nice thing about Nginx configuration is wildcard includes. Take a look at our top level configuration file: ... http { include /etc/nginx/mime.types; default_type application/octet-stream; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log /var/log/nginx/access.log main; sendfile on; keepalive_timeout 65; include /etc/nginx/conf.d/*.conf; } Line 16 says, take any *.conf file in the conf.d directory and add it here. Inside conf.d is a single file for all api.example.com requests: include /etc/nginx/conf.d/api.example.com.conf.d/upstream.*.conf; server { listen 80; server_name api.example.com; include /etc/nginx/conf.d/api.example.com.conf.d/location.*.conf; location / { root /usr/share/nginx/api.example.com; index index.html index.htm; } } This is basically saying listen on port 80 for any requests with a host header ‘api.example.com’. This has two includes. The first one at line 1, I’ll talk about later. At line 7 it says ‘take any file named location.*.conf in the subdirectory ‘api.example.com.conf.d’ and add it to the configuration. Our proxy automation component adds new components (AKA API claims) by dropping new location.*.conf files in this directory. For example, for our stock component it might create a file, ‘location.stock.conf’, like this: location /stock/ { proxy_pass http://stock; } This simply tells Nginx to proxy all requests addressed to api.example.com/stock/… to the upstream servers defined at ‘stock’. This is where the other include mentioned above comes in, ‘upstream.*.conf’. The proxy automation component also drops in a file named upstream.stock.conf that looks something like this: upstream stock { server 10.0.0.23:8001; server 10.0.0.23:8002; } This tells Nginx to round-robin all requests to api.example.com/stock/ to the given sockets. In this example it’s two components on the same machine (10.0.0.23), one on port 8001 and the other on port 8002. As instances of the stock component get deployed, new entries are added to upstream.stock.conf. Similarly, when components get uninstalled, the entry is removed. When the last entry is removed, the whole file is also deleted. This infrastructure allows us to decouple infrastructure configuration from component deployment. We can scale the application up and down by simply adding new component instances as required. As a component developer, I don’t need to do any proxy configuration, just make sure my component publishes add and remove API claim messages and I’m good to go.
June 19, 2013
by Mike Hadlow
· 59,177 Views
article thumbnail
How to Configure Timeout Duration on the Client Side for Axis2 Web Services
Axis2 uses CommonsHTTPTransportSender by default, which is based on commons-httpclient-3.1. At transport level, there’re two types of timeouts that can be set: 1. Socket Timeout 2. Connection Timeout Here’s how you can configure the above ones: Way #1: Configuring timeouts in axis2.xml Socket Timeout some_integer_value Connection timeout some_integer_value Way #2: Configuring timeouts in code … Options options = new Options(); options.setProperty(HTTPConstants.SO_TIMEOUT, new Integer(timeOutInMilliSeconds)); options.setProperty(HTTPConstants.CONNECTION_TIMEOUT, new Integer(timeOutInMilliSeconds)); // or options.setTimeOutInMilliSeconds(timeOutInMilliSeconds); … Real-life code: How to set timeout for a Axis2 Stub? long timeout = 2 * 60 * 1000; // Two minutes Stub stub = new TestStub(); stub._getServiceClient().getOptions().setTimeOutInMilliSeconds(soTimeout); //or long timeout = 2 * 60 * 1000; // Two minutes Stub stub = new TestStub(); stub._getServiceClient().getOptions().setProperty( HTTPConstants.SO_TIMEOUT, new Integer(timeOutInMilliSeconds)); stub._getServiceClient().getOptions().setProperty( HTTPConstants.CONNECTION_TIMEOUT, new Integer(timeOutInMilliSeconds));
June 18, 2013
by Singaram Subramanian
· 21,094 Views
article thumbnail
NetBeans IDE 7.3.1 Now Available with Java EE 7 Support
NetBeans IDE 7.3.1 is an update to NetBeans IDE 7.3 and includes the following highlights: Support for Java EE 7 development Deployment to GlassFish 4 Support for major Java EE 7 specifications: JSF 2.2, JPA 2.1, JAX-RS 2.0, WebSocket 1.0 and more Support for WebLogic 12.1.2 and JBoss 7.x Integration of recent patches There are two ways to get the recent changes: To use the new Java EE 7 support, it is recommended to download and install NetBeans IDE 7.3.1. To get only the integration of recent patches: Launch your current installation of NetBeans IDE 7.3. An update notification will appear in the IDE. Click the notification box to install the updates. OR to perform the update manually, in the IDE select Help-->Check for Updates. NetBeans IDE 7.3.1 is available in English, Brazilian Portuguese, Japanese, Russian, and Simplified Chinese.
June 12, 2013
by Tinu Awopetu
· 9,465 Views
article thumbnail
WSDLToJava Error: Rpc/Encoded WSDLs Are Not Supported with CXF
RPC/encoded is a vestige from before SOAP objects were defined with XML Schema. It’s not widely supported anymore. You will need to generate the stubs using Apache Axis 1.0, which is from the same era. java org.apache.axis.wsdl.WSDL2Java http://someurl?WSDL You will need the following jars or equivalents in the -cp classpath param: axis-1.4.jar commons-logging-1.1.ja commons-discovery-0.2.jar jaxrpc-1.1.jar saaj-1.1.jar wsdl4j-1.4.jar activation-1.1.jar mail-1.4.jar This will generate similar stubs to wsimport. Alternatively, if you are not using the parts of the schema that require rpc/encoded, you can download a copy of the WSDL and comment out those bits. Then run wsimport against the local file. If you look at the WSDL, the following bits are using rpc/encoded: Sources 1. http://bitkickers.blogspot.com/2008/12/rpcencoded-web-services-on-java-16.html 2. http://stackoverflow.com/questions/412772/java-rpc-encoded-wsdls-are-not-supported-in-jaxws-2-0
June 12, 2013
by Singaram Subramanian
· 40,564 Views · 9 Likes
article thumbnail
Hadoop REST API - WebHDFS
Hadoop provides a Java native API to support file system operations..
June 3, 2013
by Istvan Szegedi
· 57,522 Views · 5 Likes
article thumbnail
Web API in ASP.NET Web Forms Application
With the release of ASP.NET MVC 4 one of the exciting features packed in the release was ASP.NET Web API.
May 24, 2013
by Lohith Nagaraj
· 51,650 Views
article thumbnail
WSO2 ESB Filter Mediator Tutorial
This post will be for WSO2 ESB Filter Mediator and it will be cover simple usecase with basic Filter Mediator functions. It can be used for XPath filtering of messages. There are two modes of operation Specifies the XPath (boolean expression), return true or false XPath will be matched against the regular expression return true or false Syntax mediator+ Usecase I have services call 'BusService' where I can give rootId (road name) and get list bus number that going on that root. In the same services it have some train deatils also. When client call busService it must give busService and also If client ask for train details rather bus system must give it also. Here is busServices calls request: root1 respond: root1Colombo Negombo Galle getTraingNo request: root1 respond: 12-Colombo 13-Muthu 01-Bange Now I have write simple WSO2 ESB proxy with filter mediator. 1. Download wso2 esb 4.6.0 2. Start wso2 esb /bin/wso2server.bat (offset 1) Other services expose in wso2 AS in offset 0 3. Go to https://localhost:9444/carbon/ 4. Then Create "Pass Through Proxy" 5. Here I am adding WSO2 Filter Mediator Specify As: XPath or a Regular expression. XPath: XPath expression if you selected the "Specify As" option to "XPath". Source: which is going match with the reguilar expression Regex: Regular expression to match with the source value. 6. In Here I am filtering for the action of the WS request and it log the client request is it bus or train request? Here is proxy Source View 6. Go to https://localhost:9444/services/transportProxy?tryit# Make bus request and train request and see console log You can improve this usecase with some WSO2 ESB mediator if you wish!!
May 15, 2013
by Madhuka Udantha
· 13,244 Views
article thumbnail
Why Choose Apache Camel with Apache Tomcat
apache camel with apache tomcat provides a low-cost and lightweight integration framework. is apache camel with apache tomcat a good fit for your project requirements? apache tomcat is known for it’s ease-of-use and minimal footprint when building servlet and javaserver page applications, while apache camel is known for supporting enterprise integration patterns, routing and mediation rules in a variety of domain-specific languages, including a java-based fluent api, spring or blueprint xml configuration files, and a scala dsl. developers and architects find a straightforward learning curve when using apache camel’s java based dsl, yet they find better tools exist when building simple connections or implementing large integration projects. see kai wahner’s writeup on lightweight frameworks . for larger integration projects requiring reliable messaging, scalability, eventing, business process execution, or web agent hosting, selecting an enterprise service bus provides a better fit . kai has another good article placing esb and integration suites in context. apache camel is often integrated with activemq, servicemix, or fuse to obtain additional capabilities required to deliver medium to complex integration projects. the wso2 esb team is looking to embrace the simplicity of apache camel (by incorporating the project similar to embedding apache cxf ), and extend with multi-tenancy, failover, performance, and scalability enhancements. similar to redhat jboss fuse, wso2 esb delivers service container clustering and reliable failover functions. in addition to extensive mediation primitives, the products provide service monitoring and management support not available in the basic apache camel with apache tomcat combination. to combat server proliferation, wso2 esb inherently supports multi-tenancy. the multi-tenancy goes beyond simple tomcat virtual domains by using osgi class loaders and security managers to provide adequate tenant isolation and separate administration console interfaces. a single wso2 esb instance can support multiple business units with appropriate data, logic, and execution isolation. springsource, mulesoft, and wso2 have extended apache tomcat to provide better server management and ability to install features within the integration platform. wso2 esb can install over 100+ features (e.g. business process execution, complex event execution, business activity monitoring) into the integration platform. from a performance perspective, apache camel with apache tomcat depends on the tomcat transport to provide high performant message transfer. the wso2 esb pass through transport and binary relay transports are optimized to provide the best streaming, non-blocking performance by tightly integrating the transport and mediation layers. camel + tomcat depends on what ever the tomcat transport support but i believe esb pt and nhttp transports are preforming efficiently here but i also don’t have any reference. if you install apache camel on top of apache tomcat then you are not going to get the same performance and scalability. the latest esb performance benchmarks are posted for reference and replication.
May 9, 2013
by Chris Haddad
· 11,470 Views
article thumbnail
Apache CXF vs. Apache AXIS vs. Spring WS
This blog does not try to compare all available Web Services Development Frameworks but focuses only on three popular approaches
May 8, 2013
by Ankur Kumar
· 146,124 Views · 12 Likes
article thumbnail
How to Create a Web Service Using Java, Eclipse, and Tomcat
This tutorial runs through a method for building a Java web service in Eclipse using Apache Tomcat and Apache Axis. The process takes under ten minutes.
May 8, 2013
by Mitch Pronschinske
· 175,784 Views · 1 Like
article thumbnail
Synchronising Multithreaded Integration Tests revisited
I recently stumbled upon an article Synchronising Multithreaded Integration Tests on Captain Debug's Blog. That post emphasizes the problem of designing integration tests involving class under test running business logic asynchronously. This contrived example was given (I stripped some comments): public class ThreadWrapper { public void doWork() { Thread thread = new Thread() { @Override public void run() { System.out.println("Start of the thread"); addDataToDB(); System.out.println("End of the thread method"); } private void addDataToDB() { // Dummy Code... try { Thread.sleep(4000); } catch (InterruptedException e) { e.printStackTrace(); } } }; thread.start(); System.out.println("Off and running..."); } } This is only an example of common pattern where business logic is delegated to some asynchronous job pool we have no control over. Roger Hughes (the author) enumerates few techniques of testing such code, including: arbitrary ("long enough") sleep() in test method to make sure background logic finishes refactoring doWork() so that it accepts CountDownLatch and agrees to notify it when job is done making the method above package private and @VisibleForTesting only "The" solution - refactoring doWork() so that it accepts arbitrary Runnable. In test we can wrap this Runnable (decorator pattern) and wait for inner Runnable to complete Last solution is not bad but it changes the responsibilities of ThreadWrapper significantly. Now it's up to the caller to decide what kind of job should be executed asynchronously while previously ThreadWrapper was encapsulating business logic completely. I am not saying it's a bad design, but it's drastically different from original method. Awaitility Can we write a test without such a massive refactoring? First solution involves handy library called Awaitility. This library is not a silver bullet, it simply evaluates given condition periodically and makes sure it's fulfilled within given time. It's the kind of code you probably wrote once or twice - wrapped in a library with well designed API. So here is our initial approach: import static com.jayway.awaitility.Awaitility.await; import static java.util.concurrent.TimeUnit.SECONDS; //... await().atMost(10, SECONDS).until(recordInserted()); //... private Callable recordInserted() { return new Callable() { @Override public Boolean call() throws Exception { return dataExists(); } }; } I think there is nothing to explain here. dataExists() is simply a boolean method that initially returns false but will eventually return true once the background task (addDataToDB()) is done. In other words we assume that background task introduces some side effect and dataExists() can detect that side effect. BTW I happened to have JDK 8 with Lambda support installed and IntelliJ IDEA gives me this nice tooltip: Suddenly I get this Java 8-compatible alternative suggested: private Callable recordInserted() { return () -> dataExists(); } But there's more: Which transforms my code to: private Callable recordInserted() { return this::dataExists; } this:: prefix means that recordInsterted is a method of current object. Just as well we can say someDao::dataExists. Simply put this syntax turns method into a function object we can pass around (this process is called eta expansion in Scala). By now recordInsterted() method is no longer that needed so I can inline it and remove it completely: await().atMost(10, SECONDS).until(this::dataExists); I am not sure what I love more - the new lambda syntax or how IntelliJ IDEA takes pre-Java 8 code and retrofits it for me automatically (well, it's still a bit experimental, just reported IDEA-106670). I can run this intention in IntelliJ project-wide, Lambda-enabling my whole code base in seconds. Sweet! But back to original problem. Awaitility helps a lot by providing decent API and some handy features. I use it extensively in combination with FluentLenium. But periodically polling for state changes feels a bit like a workaround and still introduces minimal latency. But notice that running and synchronizing on asynchronous tasks is quite common and JDK already provides necessary facilities: Future abstraction! java.util.concurrent.Future To limit the scope of refactoring I will leave the original new Thread() approach for now and use SettableFuture from Guava. It is a Future implementation that allows triggering completion or failure at any time, from any thread (see DeferredResult - asynchronous processing in Spring MVC for more advanced usage). As you can see the changes are quite small: public class ThreadWrapper { public ListenableFuture doWork() { final SettableFuture future = SettableFuture.create(); Thread thread = new Thread() { @Override public void run() { addDataToDB() //... //last instruction future.set(null); } private void addDataToDB() { // Dummy Code... // ... } }; thread.start(); return future; } } doWork() now returns ListenableFuture with lifecycle controlled inside asynchronous task. We use Void but in reality you might want to return some asynchronous result instead. future.set(null) invocation in the end is crucial. It signals that future is fulfilled and all threads waiting for that future will be notified. Once again, in practice you would use e.g. Future and then instead of null we would say future.set(someInteger). Here null is just a placeholder for Void type. How does this help us? Test code can now rely on future completion: final ListenableFuture future = wrapper.doWork(); future.get(10, SECONDS); future.get() blocks until future is done (with timeout), i.e. until we call future.set(...). BTW I use ListenableFuture from Guava but Java 8 introduces equivalent and standard CompletableFuture - I will write about it soon. So, we are getting somewhere. Future is a useful abstraction for waiting and signalling completion of background jobs. But there is also one immense advantage of Future which are not taking, ekhm, advantage from - exception handling and propagation. Future.get() will block until future is complete and return asynchronous result or throw an exception initially thrown from our job. This is really useful for asynchronous tests. Currently if Thread.run() throws an exception it may or may not be logged or visible to us and future will never be completed. With Awaitility it's slightly better - it will timeout without any meaningful reason, which have to be tracked down manually in console/logs. But with minor modification our test is much more verbose: public void run() { try { addDataToDB() //... future.set(null); } catch (Exception e) { future.setException(e); } } If some exception occurs in asynchronous job, it will pop-up and be shown as JUnit/TestNG failure reason. (Listening)ExecutorService That's it. If addDataToDB() throws an exception it will not be lost. Instead our future.get() in test will re-throw that exception for us. Our test won't simply timeout leaving us with no clue what went wrong. Great, but do we really have to create this special SettableFuture instance, can't we just use existing libraries that already give us Future with correct underlying implementation? Of course! By this requires further refactoring: import com.google.common.util.concurrent.ListeningExecutorService; import com.google.common.util.concurrent.MoreExecutors; import java.util.concurrent.Executors; import java.util.concurrent.Future; public class ThreadWrapper { private final ListeningExecutorService executorService = MoreExecutors.listeningDecorator( Executors.newSingleThreadExecutor() ); public ListenableFuture doWork() { Runnable job = new Runnable() { @Override public void run() { //... } }; return executorService.submit(job); } } This is what you've all been waiting for. Don't start new Thread all the time, use thread pool! I actually went one step further by using ListeningExecutorService - an extension to ExecutorService that returns ListenableFuture instances (see why you want that). But the solution doesn't require this, I just spread good practices. As you can see Future instance is now created and managed for us. The test is exactly the same but production code is cleaner and more robust. MoreExecutors.sameThreadExecutor() The final trick I want to show you involves dependency injection. First let's externalize the creation of a thread pool from ThreadWrapper class: private final ListeningExecutorService executorService; public ThreadWrapper() { this(Executors.newSingleThreadExecutor()); } public ThreadWrapper(ExecutorService executorService) { this.executorService = MoreExecutors.listeningDecorator(executorService); } We can now optionally supply custom ExecutorService. This is good for various other reasons, but for us it opens brand new testing opportunity: MoreExecutors.sameThreadExecutor(). This time we modify our test slightly: final ThreadWrapper wrapper = new ThreadWrapper(MoreExecutors.sameThreadExecutor()); wrapper.doWork().get(); See how we pass custom ExecutorService? It's a very special implementation that doesn't really maintain thread pool of any kind. Every time you submit() some task to that "pool" it will be executed in the same thread in a blocking manner. This means that we no longer have asynchronous test, even though the production code wasn't changed that much! wrapper.doWork() will block until "background" job finishes. The extra call to get() is still needed to make sure exceptions are propagated, but is guaranteed to never block (because the job is already done). Using the same thread to execute asynchronous task instead of a thread pool might have an unexpected results if you somehow depend on thread-based properties, e.g. transactions, security, ThreadLocal. However if you use standard ThreadPoolExecutor with CallerRunsPolicy, JDK already behaves this way if thread pool is overflowed. So it's not that unusual. Summary Testing asynchronous code is hard, but you have options. Several options. But one conclusion that strikes me is the side effect of our efforts. We refactored original code in order to make it testable. But the final production code is not only testable, but also much better structured and robust. Surprisingly it's even source-code compatible with previous version as we barely changed return type from void to Future. It seems to be a rule - testable code is often better designed and implemented. Unit test is the first client code using our library. It naturally forces us to to think more about consumers, not the implementation.
May 7, 2013
by Tomasz Nurkiewicz
· 9,015 Views · 1 Like
article thumbnail
Publish/Subscribe Pattern with Apache Camel
Publish/Subscribe is a simple messaging pattern where a publisher sends messages to a channel without the knowledge of who is going to receive them. Then it is the responsibility of the channel to deliver a copy of the messages to each subscriber. This messaging model enables creation of loosely coupled and scalable systems. It is a very common messaging pattern and there are so many ways to create a kind of pub-sub in Apache Camel. But bear in mind that they are all different and have different characteristics. From the simplest to more complex, here is a list: Multicast - works only with a static list of subscribers, can deliver the message to subscriber in parallel, stops or continues on exception if one of the subscribers fails. Recipient List - it is similar to multicast, but allows the subscribers to be defined at run time, for example in the message header. SEDA - this component provides asynchronous SEDA behaviour using BlockingQueue. When multipleConsumers option is set, it can be used for asynchronous pub-sub messaging. It also has possibilities to block when full, set queue size or time out publishing if the message is not consumed on time. VM - same as SEDA, but works cross multiple CamelContexts, as long as they are in the same JMV. It is a nice mechanism for sending messages between webapps in a web-container or bundles in OSGI container. Spring-redis - Redis has pubsub feature which allows publishing messages to multiple receivers. It is possible to subscribe to a channel by name or using pattern-matching. When pattern-matching is used, the subscriber will receive messages from all the channels matching the pattern. Keep in mind that in this case it is possible to receive a message more than once, if the multiple patterns matches the same channel where the message was sent. JMS (ActiveMQ) - that's probably the best know way for doing pub-sub including durable subscriptions. For a complete list of features check ActiveMQ website. Amazon SNS/SQS - if you need a really scalable and reliable solution, SNS is the way to go. Subscribing a SQS queue to the topic, turns it into a durable subscriber and allows polling the messages later. The important point to remember in this case is that it is not very fast and most importantly, Amazon doesn't guarantee FIFO order for your messages. There are also less popular Camel components which offer publish-subscribe messaging model: websocket - it uses Eclipse Jetty Server and can sends message to all clients which are currently connected. hazelcast - SEDA implements a work-queue in order to support asynchronous SEDA architectures. guava-eventbus - integration bridge between Camel and Google Guava EventBus infrastructure. spring-event - provides access to the Spring ApplicationEvent objects. eventadmin - on OSGi environment to receive OSGI events. xmpp - implements XMPP (Jabber) transport. Posting a message in chat room is also pub-sub;) mqtt - for communicating with MQTT compliant message brokers. amqp - supports the AMQP protocol using the Client API of the Qpid project. javaspace - a transport for working with any JavaSpace compliant implementation. Can you name any other way for doing publish-subscribe?
May 2, 2013
by Bilgin Ibryam
· 11,865 Views
article thumbnail
How to Format Java Code Using Eclipse JDT?
Yo probably format your code often by pressing Ctrl+Shift+F or right clicking Source -> Format. This function is also provide in JDT, so you can also format your Java code in code. However finding correct class to do this function is not straight-forward, because one of them is a internal class. The following is the code to format Java code by using DefaultCodeFormatter. import org.eclipse.jdt.core.ToolFactory; import org.eclipse.jdt.core.formatter.CodeFormatter; import org.eclipse.jface.text.BadLocationException; import org.eclipse.jface.text.Document; import org.eclipse.jface.text.IDocument; import org.eclipse.text.edits.MalformedTreeException; import org.eclipse.text.edits.TextEdit; public class FormatterTest { public static void main(String[] args) { String code = "public class TestFormatter{public static void main(String[] args){System.out.println(\"Hello World\");}"; CodeFormatter codeFormatter = ToolFactory.createCodeFormatter(null); TextEdit textEdit = codeFormatter.format(CodeFormatter.K_COMPILATION_UNIT, code, 0, code.length(), 0, null); IDocument doc = new Document(code); try { textEdit.apply(doc); System.out.println(doc.get()); } catch (MalformedTreeException e) { e.printStackTrace(); } catch (BadLocationException e) { e.printStackTrace(); } } } he apply() method in TextEdit class is the key to this problem. It applies the edit tree rooted by this edit to the GIVEN document. Output in console: Depending on your Eclipse version, you will need the following jar files: org.eclipse.core.contenttype_3.4.1.R35x_v20090826-0451.jar org.eclipse.core.jobs_3.4.100.v20090429-1800.jar org.eclipse.core.resources_3.5.2.R35x_v20091203-1235.jar org.eclipse.equinox.common_3.5.1.R35x_v20090807-1100.jar org.eclipse.equinox.preferences_3.2.301.R35x_v20091117.jar org.eclipse.jdt.core_3.5.2.v_981_R35x.jar org.eclipse.osgi_3.5.2.R35x_v20100126.jar org.eclipse.text_3.5.101.v20110928-1504.jar org.eclipse.core.runtime_3.5.0.v20090525.jar
April 22, 2013
by Ryan Wang
· 6,760 Views
  • Previous
  • ...
  • 255
  • 256
  • 257
  • 258
  • 259
  • 260
  • 261
  • 262
  • 263
  • 264
  • ...
  • Next
  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook
×