DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

The Latest Tools Topics

article thumbnail
Step-by-Step: Live Migrate Multiple (Clustered) VMs in One Line of PowerShell - Revisited
A while back, I wrote an article showing how to Live Migrate Your VMs in One Line of Powershell between non-clustered Windows Server 2012 Hyper-V hosts using Shared Nothing Live Migration. Since then, I’ve been asked a few times for how this type of parallel Live Migration would be performed for highly available virtual machines between Hyper-V hosts within a cluster. In this article, we’ll walk through the steps of doing exactly that … via Windows PowerShell on Windows Server 2012 or 2012 R2 or our FREE Hyper-V Server 2012 R2 bare-metal, enterprise-grade hypervisor in a clustered configuration. Wait! Do I need PowerShell to Live Migrate multiple VMs within a Cluster? Well, actually … No. You could certainly use the Failover Cluster Manager GUI tool to select multiple highly available virtual machines, right-click and select Move | Live Migration … Failover Cluster Manager – Performing Multi-VM Live Migration But, you may wish to script this process for other reasons … perhaps to efficiently drain all VM’s from a host as part of a maintenance script that will be performing other tasks. Can I use the same PowerShell cmdlets for Live Migrating within a Cluster? Well, actually … No again. When VMs are made highly available resources within a cluster, they’re managed as cluster group resources instead of being standalone VM resources. As a result, we have a different set of Cluster-aware PowerShell cmdlets that we use when managing these cluster groups. To perform a scripted multi-VM Live Migration, we’ll be leveraging three of these cmdlets: Get-ClusterNode, Get-ClusterGroup and Move-ClusterVirtualMachineRole Now, let’s see that one line of PowerShell! Before getting to the point of actually performing the multi-VM Live Migration in a single PowerShell command line, we first need to setup a few variables to handle the "what" and "where" of moving these VMs. First, let’s specify the name of the cluster with which we’ll be working. We’ll store it in a $clusterName variable. $clusterName = read-host -Prompt "Cluster name" Next, we’ll need to select the cluster node to which we’ll be Live Migrating the VMs. Lets use the Get-ClusterNode and Out-GridView cmdlets together to prompt for the cluster node and store the value in a $targetClusterNode variable. $targetClusterNode = Get-ClusterNode -Cluster $clusterName | Out-GridView -Title "Select Target Cluster Node" ` -OutputMode Single And then, we’ll need to create a list of all the VMs currently running in the cluster. We can use the Get-ClusterGroup cmdlet to retrieve this list. Below, we have an example where we are combining this cmdlet with a Where-Object cmdlet to return only the virtual machine cluster groups that are running on any node except the selected target cluster node. After all, it really doesn’t make any sense to Live Migrate a VM to the same node on which it’s currently running! $haVMs = Get-ClusterGroup -Cluster $clusterName | Where-Object {($_.GroupType -eq "VirtualMachine") ` -and ($_.OwnerNode -ne $targetClusterNode.Name)} We’ve stored the resulting list of VMs in a $haVMs variable. Ready to Live Migrate! OK … Now we have all of our variables defined for the cluster, the target cluster node and the list of VMs from which to choose. Here’s our single line of PowerShell to do the magic … $haVMs | Out-GridView -Title "Select VMs to Move" –PassThru | Move-ClusterVirtualMachineRole -MigrationType Live ` -Node $targetClusterNode.Name -Wait 0 Proceed with care: Keep in mind that your target cluster node will need to have sufficient available resources to run the VM's that you select for Live Migration. Of course, it's best to initially test tasks like this in your lab environment first. Here’s what is happening in this single PowerShell command line: We’re passing the list of VMs stored in the $haVMs variable to the Out-GridView cmdlet. Out-GridView prompts for which VMs to Live Migrate and then passes the selected VMs down the PowerShell object pipeline to the Move-ClusterVirtualMachineRole cmdlet. This cmdlet initiates the Live Migration for each selected VM, and because it’s using a –Wait 0 parameter, it initiates each Live Migration one-after-another without waiting for the prior task to finish. As a result, all of the selected VMs will Live Migrate in parallel, up to the maximum number of concurrent Live Migrations that you’ve configured on these cluster nodes. The VMs selected beyond this maximum will simply queue up and wait their turn. Unlike some competing hypervisors, Hyper-V doesn't impose an artificial hard-coded limit on how many VMs for you can Live Migrate concurrently. Instead, it's up to you to set the maximum to a sensible value based on your hardware and network capacity. Do you have your own PowerShell automation ideas for Hyper-V? Feel free to share your ideas in the Comments section below. See you in the Clouds! - Keith
March 3, 2014
by Keith Mayer
· 10,723 Views
article thumbnail
Generating a War File From a Plain IntelliJ Web Project
Sometimes you just want to create a quick web project in IntelliJ IDEA, and you would use their wizard and with web or Java EE module as starter project. But these projects will not have Ant nor Maven script generated for you automatically, and the IDEA Build would only compile your classes. So if you want an war file generated, try the following: 1) Menu: File > Project Structure > Artifacts 2) Click the green + icon and create a "Web Application: Archive", then OK 3) Menu: Build > Build Artifacts ... > Web: war By default it should generate it under your /out/artifacts/web_war.war Note that IntelliJ also allows you to setup "Web Application: Exploded" artifact, which great for development that run and deploy to an application server within your IDE.
March 3, 2014
by Zemian Deng
· 76,302 Views · 2 Likes
article thumbnail
Jersey: Ignoring SSL certificate – javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException
Last week Alistair and I were working on an internal application and we needed to make a HTTPS request directly to an AWS machine using a certificate signed to a different host. We use jersey-client so our code looked something like this: Client client = Client.create(); client.resource("https://some-aws-host.compute-1.amazonaws.com").post(); // and so on When we ran this we predictably ran into trouble: com.sun.jersey.api.client.ClientHandlerException: javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative DNS name matching some-aws-host.compute-1.amazonaws.com found. at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149) at com.sun.jersey.api.client.Client.handle(Client.java:648) at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670) at com.sun.jersey.api.client.WebResource.post(WebResource.java:241) at com.neotechnology.testlab.manager.bootstrap.ManagerAdmin.takeBackup(ManagerAdmin.java:33) at com.neotechnology.testlab.manager.bootstrap.ManagerAdminTest.foo(ManagerAdminTest.java:11) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.junit.runner.JUnitCore.run(JUnitCore.java:157) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:202) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120) Caused by: javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative DNS name matching some-aws-host.compute-1.amazonaws.com found. at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1884) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:276) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) at sun.security.ssl.Handshaker.process_record(Handshaker.java:804) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1016) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323) at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:563) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1300) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:240) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147) ... 31 more Caused by: java.security.cert.CertificateException: No subject alternative DNS name matching some-aws-host.compute-1.amazonaws.com found. at sun.security.util.HostnameChecker.matchDNS(HostnameChecker.java:191) at sun.security.util.HostnameChecker.match(HostnameChecker.java:93) at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:347) at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:203) at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:126) at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1323) ... 45 more We figured that we needed to get our client to ignore the certificate and came across this Stack Overflow thread which had some suggestions on how to do this. None of the suggestions worked on their own but we ended up with a combination of a couple of the suggestions which did the trick: public Client hostIgnoringClient() { try { SSLContext sslcontext = SSLContext.getInstance( "TLS" ); sslcontext.init( null, null, null ); DefaultClientConfig config = new DefaultClientConfig(); Map properties = config.getProperties(); HTTPSProperties httpsProperties = new HTTPSProperties( new HostnameVerifier() { @Override public boolean verify( String s, SSLSession sslSession ) { return true; } }, sslcontext ); properties.put( HTTPSProperties.PROPERTY_HTTPS_PROPERTIES, httpsProperties ); config.getClasses().add( JacksonJsonProvider.class ); return Client.create( config ); } catch ( KeyManagementException | NoSuchAlgorithmException e ) { throw new RuntimeException( e ); } } You’re welcome Future Mark.
March 2, 2014
by Mark Needham
· 43,089 Views · 8 Likes
article thumbnail
Running Hadoop MapReduce Application from Eclipse Kepler
it's very important to learn hadoop by practice. one of the learning curves is how to write the first map reduce app and debug in favorite ide, eclipse. do we need any eclipse plugins? no, we do not. we can do hadoop development without map reduce plugins this tutorial will show you how to set up eclipse and run your map reduce project and mapreduce job right from your ide. before you read further, you should have setup hadoop single node cluster and your machine. you can download the eclipse project from github . use case: we will explore the weather data to find maximum temperature from tom white’s book hadoop: definitive guide (3rd edition) chapter 2 and run it using toolrunner i am using linux mint 15 on virtualbox vm instance. in addition, you should have hadoop (mrv1 am using 1.2.1) single node cluster installed and running, if you have not done so, would strongly recommend you do it from here download eclipse ide, as of writing this, latest version of eclipse is kepler 1. create new java project 2. add dependencies jars right click on project properties and select java build path add all jars from $hadoop_home/lib and $hadoop_home (where hadoop core and tools jar lives) 3. create mapper package com.letsdobigdata; import java.io.ioexception; import org.apache.hadoop.io.intwritable; import org.apache.hadoop.io.longwritable; import org.apache.hadoop.io.text; import org.apache.hadoop.mapreduce.mapper; public class maxtemperaturemapper extends mapper { private static final int missing = 9999; @override public void map(longwritable key, text value, context context) throws ioexception, interruptedexception { string line = value.tostring(); string year = line.substring(15, 19); int airtemperature; if (line.charat(87) == '+') { // parseint doesn't like leading plus // signs airtemperature = integer.parseint(line.substring(88, 92)); } else { airtemperature = integer.parseint(line.substring(87, 92)); } string quality = line.substring(92, 93); if (airtemperature != missing && quality.matches("[01459]")) { context.write(new text(year), new intwritable(airtemperature)); } } } 4. create reducer package com.letsdobigdata; import java.io.ioexception; import org.apache.hadoop.io.intwritable; import org.apache.hadoop.io.text; import org.apache.hadoop.mapreduce.reducer; public class maxtemperaturereducer extends reducer { @override public void reduce(text key, iterable values, context context) throws ioexception, interruptedexception { int maxvalue = integer.min_value; for (intwritable value : values) { maxvalue = math.max(maxvalue, value.get()); } context.write(key, new intwritable(maxvalue)); } } 5. create driver for mapreduce job map reduce job is executed by useful hadoop utility class toolrunner package com.letsdobigdata; import org.apache.hadoop.conf.configured; import org.apache.hadoop.fs.path; import org.apache.hadoop.io.intwritable; import org.apache.hadoop.io.text; import org.apache.hadoop.mapreduce.job; import org.apache.hadoop.mapreduce.lib.input.fileinputformat; import org.apache.hadoop.mapreduce.lib.output.fileoutputformat; import org.apache.hadoop.util.tool; import org.apache.hadoop.util.toolrunner; /*this class is responsible for running map reduce job*/ public class maxtemperaturedriver extends configured implements tool{ public int run(string[] args) throws exception { if(args.length !=2) { system.err.println("usage: maxtemperaturedriver "); system.exit(-1); } job job = new job(); job.setjarbyclass(maxtemperaturedriver.class); job.setjobname("max temperature"); fileinputformat.addinputpath(job, new path(args[0])); fileoutputformat.setoutputpath(job,new path(args[1])); job.setmapperclass(maxtemperaturemapper.class); job.setreducerclass(maxtemperaturereducer.class); job.setoutputkeyclass(text.class); job.setoutputvalueclass(intwritable.class); system.exit(job.waitforcompletion(true) ? 0:1); boolean success = job.waitforcompletion(true); return success ? 0 : 1; } public static void main(string[] args) throws exception { maxtemperaturedriver driver = new maxtemperaturedriver(); int exitcode = toolrunner.run(driver, args); system.exit(exitcode); } } 6. supply input and output we need to supply input file that will be used during map phase and the final output will be generated in output directory by reduct task. edit run configuration and supply command line arguments. sample.txt reside in the project root. your project explorer should contain following ] 7. map reduce job execution 8. final output if you managed to come this far, once the job is complete, it will create output directory with _success and part_nnnnn , double click to view it in eclipse editor and you will see we have supplied 5 rows of weather data (downloaded from ncdc weather) and we wanted to find out the maximum temperature in a given year from input file and the output will contain 2 rows with max temperature in (centigrade) for each supplied year 1949 111 (11.1 c) 1950 22 (2.2 c) make sure you delete the output directory next time running your application else you will get an error from hadoop saying directory already exists. happy hadooping!
February 21, 2014
by Hardik Pandya
· 144,762 Views · 2 Likes
article thumbnail
Getting Started with HTML5 WebSocket on Oracle WebLogic 12c
The current release of Oracle WebLogic (12.1.2) added support to HTML5 WebSocket (RFC-6455) providing the bi-directional communication over a single TCP connection between clients and servers. Unlike the HTTP communication model, client and server can send and receive data independently from each other. The use of WebSocket is becoming very popular for highly interactive web applications that depend on time critical data delivery, requirements for true bi-directional data flow and higher throughput. To initiate the WebSocket connection, the client sends a handshake request to the server. The connection is established if the handshake request passes validation, and the server accepts the request. When a WebSocket connection is established, a browser client can send data to a WebLogic Server instance while simultaneously receiving data from that server instance. The WebLogic server implementation has the following components: The WebSocket protocol implementation that handles connection upgrades and establishes and manages connections as well as exchanges with the client. The WebLogic server fully implements the WebSocket protocol using its existing threading and networking infrastructure. The WebLogic WebSocket Java API, through the weblogic.websocket package, allows you to create WebSocket-based server side applications handling client connections, WebSocket messages, providing context information for a particular WebSocket connection and managing the request/response handshake. For additional information about the WebLogic WebSocket Java API interfaces and classes, visit the following resource: http://docs.oracle.com/middleware/1212/wls/WLPRG/websockets.htm#BABJEFFD To declare a WebSocket endpoint you use the @WebSocket annotation that allow you to mark a class as a WebSocket listener that's ready to be exposed and handle events. The annotated class must implement the WebSocketListener interface or extend from the WebSocketAdapter class. When you deploy the WebSocket-based application on WebLogic, you basically follow the same approach using the standard Java EE Web Application archives (WARs), either as standalone or WAR module. Either way, the WebLogic Application Server detects the @WebSocket annotation on the class and automatically establishes it as a WebSocket endpoint. Here is a simple application that creates a WebSocket endpoint at the /echo/ location path, receives messages from the client and sends the same message back to the client. import weblogic.websocket.WebSocketAdapter; import weblogic.websocket.WebSocketConnection; import weblogic.websocket.annotation.WebSocket; @WebSocket(pathPatterns={"/echo/*"}) public class Echo extends WebSocketAdapter{ @Override public void onMessage(WebSocketConnection connection, String msg){ try{ connection.send(msg); }catch(IOException ioe){ //Handle error condition } } } n the client side, you typically use the WebSocket JavaScript API that most of the web browsers already support. There are many samples out there that you can use to test the implementation above. One quick way of doing that is to navigate to http://www.websocket.org/echo.html and then point the location field to your WebLogic server. If you follow the sample available on https://github.com/mjabali/HelloWorld_WebSocket then you'll end up with something like ws://localhost:7001/HelloWorld_WebSocket/echo/ for the WebSocket server application. The sample client provided will be available at http://localhost:7001/HelloWorld_WebSocket/index.html after the WebLogic deployment. See the README.md file on GitHub for additional instructions. Have fun!
February 10, 2014
by Marcelo Jabali
· 12,068 Views
article thumbnail
Java: Handling a RuntimeException in a Runnable
At the end of last year I was playing around with running scheduled tasks to monitor a Neo4j cluster and one of the problems I ran into was that the monitoring would sometimes exit. I eventually realised that this was because a RuntimeException was being thrown inside the Runnable method and I wasn’t handling it. The following code demonstrates the problem: import java.util.ArrayList; import java.util.List; import java.util.concurrent.*; public class RunnableBlog { public static void main(String[] args) throws ExecutionException, InterruptedException { ScheduledExecutorService executor = Executors.newSingleThreadScheduledExecutor(); executor.scheduleAtFixedRate(new Runnable() { @Override public void run() { System.out.println(Thread.currentThread().getName() + " -> " + System.currentTimeMillis()); throw new RuntimeException("game over"); } }, 0, 1000, TimeUnit.MILLISECONDS).get(); System.out.println("exit"); executor.shutdown(); } } If we run that code we’ll see the RuntimeException but the executor won’t exit because the thread died without informing it: Exception in thread "main" pool-1-thread-1 -> 1391212558074 java.util.concurrent.ExecutionException: java.lang.RuntimeException: game over at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) at java.util.concurrent.FutureTask.get(FutureTask.java:111) at RunnableBlog.main(RunnableBlog.java:11) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120) Caused by: java.lang.RuntimeException: game over at RunnableBlog$1.run(RunnableBlog.java:16) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) At the time I ended up adding a try catch block and printing the exception like so: public class RunnableBlog { public static void main(String[] args) throws ExecutionException, InterruptedException { ScheduledExecutorService executor = Executors.newSingleThreadScheduledExecutor(); executor.scheduleAtFixedRate(new Runnable() { @Override public void run() { try { System.out.println(Thread.currentThread().getName() + " -> " + System.currentTimeMillis()); throw new RuntimeException("game over"); } catch (RuntimeException e) { e.printStackTrace(); } } }, 0, 1000, TimeUnit.MILLISECONDS).get(); System.out.println("exit"); executor.shutdown(); } } This allows the exception to be recognised and as far as I can tell means that the thread executing the Runnable doesn’t die. java.lang.RuntimeException: game over pool-1-thread-1 -> 1391212651955 at RunnableBlog$1.run(RunnableBlog.java:16) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) pool-1-thread-1 -> 1391212652956 java.lang.RuntimeException: game over at RunnableBlog$1.run(RunnableBlog.java:16) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) pool-1-thread-1 -> 1391212653955 java.lang.RuntimeException: game over at RunnableBlog$1.run(RunnableBlog.java:16) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) This worked well and allowed me to keep monitoring the cluster. However, I recently started reading ‘Java Concurrency in Practice‘ (only 6 years after I bought it!) and realised that this might not be the proper way of handling the RuntimeException. public class RunnableBlog { public static void main(String[] args) throws ExecutionException, InterruptedException { ScheduledExecutorService executor = Executors.newSingleThreadScheduledExecutor(); executor.scheduleAtFixedRate(new Runnable() { @Override public void run() { try { System.out.println(Thread.currentThread().getName() + " -> " + System.currentTimeMillis()); throw new RuntimeException("game over"); } catch (RuntimeException e) { Thread t = Thread.currentThread(); t.getUncaughtExceptionHandler().uncaughtException(t, e); } } }, 0, 1000, TimeUnit.MILLISECONDS).get(); System.out.println("exit"); executor.shutdown(); } } I don’t see much difference between the two approaches so it’d be great if someone could explain to me why this approach is better than my previous one of catching the exception and printing the stack trace.
February 6, 2014
by Mark Needham
· 19,657 Views
article thumbnail
Getting Started with Intellij IDEA and WebLogic Server
Before starting, you would need the Ultimate version of IDEA to run WebLogic Server (yes, the paid version or the 30 days trial). The Community edition of IDEA will not support Application Server deployment. I also assume you have already setup WebLogic Server and a user domain as per my previous blog instructions. So now let's setup the IDE to boost your development. Create a simple HelloWorld web application in IDEA. For your HelloWorld, you can go into the Project Settings > Artifacts, and add "web:war exploded" entry for your application. You will add this into your app server later. Ensure you have added the Application Server Views plugin with WebLogic Server. (It's under Settings > IDE Settings > Application Server) Click + and enter Name: WebLogic 12.1.2 WebLogic Home: C:\apps\wls12120 Back to your editor, select Menu: Run > Edit Configuration Click + and add "WebLogic Server" > Local Name: WLS On Server tab, ensure DomainPath is set: C:\apps\wls12120\mydomain On Deployment tab, select "web:war exploded" for your HelloWorld project. Click OK Now Menu: Run > "Run WLS" Your WebLogic Server should now start and running your web application inside. You may visit the browser on http://localhost:7001/web_war_exploded Some goodies with Intellij IDEA and WLS are: Redeploy WAR only without restarting server Deploy application in exploded mode and have IDE auto make and sync Debug application with server running within IDE Full control on server settings NOTE: As noted in previous blog, if you do not set MW_HOME as system variable, then you must add this in IDEA's Run Configuration. Or you edit your "mydomain/bin/startWebLogic.cmd" and"stopWebLogic.cmd" scripts directly.
February 5, 2014
by Zemian Deng
· 47,214 Views
article thumbnail
Java: Exception Translation with AspectJ
Within this blog post I describe how you can use AspectJ to automatically translate one type of exception to another. The problem Sometimes we are in situations where we have to convert an exception (often thrown by a third-party library) to another type of exception. Assume you are using a persistence framework like hibernate and you do not want to leak hibernate specific exceptions out of a certain application layer. Maybe you are using more than one persistence technology and you want to wrap technology specific exceptions into a common base exception. In such situations, one can end with code like this: public class MyRepository { public Object getSomeData() { try { // assume hibernate is used to access some data } catch(HibernateException e) { // wrap hibernate specific exception into a general DataAccessException throw new DataAccessException(e); } } } Of course this becomes ugly if you have to do this every time you access a certain framework. The AspectJ way AspectJ is an aspect oriented programming (AOP) extension for Java. With AspectJ we can define Cross-cutting concerns that take care of the exception translation process for us. To get started we first have to add the AspectJ dependency to our project: org.aspectj aspectjrt 1.7.4 Next we have to set up ajc, the compiler and bytecode weaver for AspectJ. This step depends on the developing environment you are using, so I will not go into details here. Eclipse users should have a look at theAspectJ Development Tools (AJDT) for Eclipse. IntelliJ IDEA users should make sure the AspectJ plugin is enabled. There is also an AspectJ Maven plugin available (check this pom.xml for an example configuration). Now let's define our aspect using AspectJ annotations: @Aspect public class ExceptionTranslationAspect { @Around("execution(* com.mscharhag.exceptiontranslation.repository..*(..))") public Object translateToDataAccessException(ProceedingJoinPoint pjp) throws Throwable { try { return pjp.proceed(); } catch (HibernateException e) { throw new DataAccessException(e); } } } Using the @Aspect annotation we can declare a new aspect. Within this aspect we use the @Aroundannotation to define an advice that is always executed if the passed pointcut is matched. Here, the pointcut execution(* com.mscharhag.exceptiontranslation.repository..*(..)) tells AspectJ to call translateToDataAccessException() every time a method of a class inside thecom.mscharhag.exceptiontranslation.repository package is executed. Within translateToDataAccessException() we can use the passed ProceedingJoinPoint object to proceed the method execution we intercepted. In this example we just add a try/catch block around the method execution. Using the ProceedingJoinPoint instance we could also do more interesting things like analyzing the method signature using pjp.getSignature() or accessing method parameters withpjp.getArgs(). We can now remove the try/catch block from the example repository implementation shown above and use a simple test to verify our aspect is working: public class MyRepositoryTest { private MyRepository repository = new MyRepository(); @Test(expected = DataAccessException.class) public void testExceptionTranslation() { this.repository.getSomeData(); } } Conclusion Using AspectJ we can easily automate the conversion of Java runtime exceptions. This simplifies our code by removing try/catch blocks that would otherwise be required for exception translation. You can find the full source of the example project on GitHub.
February 4, 2014
by Michael Scharhag
· 17,271 Views · 2 Likes
article thumbnail
How to Build a Fat JAR using NetBeans IDE
In this post I would like to describe how to build a fat JAR using NetBeans IDE.
February 4, 2014
by Aruna Karunarathna
· 41,256 Views · 1 Like
article thumbnail
Business Process Modeling in NetBeans IDE
gaurav gupta is a senior software engineer working on the bpmn workflow engine. he's created several plugins for netbeans ide, notably the js css minify compress plugin ( here ) and the jbpmn plugin, which is the topic of this interview. the plugin provides netbeans ide tools for working with business process modeling notation (bpmn). the plugin can be accessed here, including for netbeans ide 7.4 and 8.0 beta, together with screencasts and tutorials: http://plugins.netbeans.org/plugin/50735/jbpmn hi, gaurav, why did you create the bpmn plugin? business process modeling notation is an increasingly important standard for process modelling and has enjoyed high levels of adoption, so the specific intent of the jbpmn plugin is to create a bpmn modeler that supports the complete bpmn specification and can be integrated by multiple bpmn engine vendors into netbeans ide. as a result, different bpmn engine vendors will not need to create separate bpmn modelers for netbeans ide. another reason i created this plugin was to bring the netbeans community closer to the bpm community because bpmn has become the de-facto standard for business process modeling. also, when we talk about "netbeans vs eclipse", the basic conclusion is that netbeans is much more intuitive and easy to use, while eclipse has a wider range of third-party plugins support from more companies. in that context, i have taken this initiative to create the bpm plugin to help the netbeans community. what would you consider to be the best features of the bpmn plugin? the jbpmn nb modeler is a graphical modelling tool which allows the creation and editing of bpmn process diagrams. it provides debugging supports for jbpm 5.0. it is bpmn engine vendor neutral, can be used by any vendor or generated xml, and can run on any bpmn engine which adopts the bpmn 2.0 standard. here's another screenshot: what are the future development plans for this plugin? to cover the complete bpmn ssecification with user friendly gui and properties support. jbpmn currently supports only the bpmn process model, it will also support the bpmn conversation model. it will also provide support to extends the modeller with your own palette elements, properties, and generated xml tags. it will provides debugging supports for jbpm 6.0, activities, and also extensions so that any bpmn engine vendor can integrate debugging functionality within jbpmn. it will also provide a netbeans modeler platform api, which any business modeller can use to easily build solutions such as a bpmn conversation model.
January 26, 2014
by Geertjan Wielenga
· 15,803 Views
article thumbnail
How to Set Up a Multi-Node Hadoop Cluster on Amazon EC2, Part 1
Learn how to set up a four node Hadoop cluster using AWS EC2, PuTTy(gen), and WinSCP.
January 23, 2014
by Hardik Pandya
· 136,013 Views · 3 Likes
article thumbnail
Google's vs Facebook's Trunk-Based Development
i’ve been pushing this branching model for something like 14 years now. it’s nice to see facebook say a little more about their trunk based development . of course they’re not doing it because they read anything i wrote, as the practice isn’t mine, it’s been hanging around in the industry for many years, but always as bridesmaid so to speak. if not trunk, what? mainline? mainline as popularized by clearcase is what we’re trying to kill. at least historically. it’s very different to trunk based development, and even having vastly improved merge tools doesn’t make it better – you still risk regressions, and huge nerves around ordering of releases. clearcase’s best-practices also foisted a ‘many repos’ (vobs) on teams using it, and that courted the whole conway’s law prophesy. i mentioned conway’s law before in scaling trunk based development and it concerns undue self-importance of teams around arbitrary separations. multiple small repos for a dvcs ? there is a great statement by a reddit user in the programming section of reddit, in conjunction with the facebook announcement: comment ref or all comments this redditor is right, there’s a lack of atomicity around a many-repos design, that stymies bisect. it could be that git subtrees (not submodules) are a way of getting that back (thanks @chris_stevenson on a back channel). there’s also a real problem moving code easily between repos (with history) though @offbytwo (back channel again) points out that subtrees carefully used can help do that. trunk at google vs facebook tuesday’s announcement was from facebook, and to give some balance, there’s deeper info on google’s trunk design in: google’s scaled trunk based development . subsetting the trunk for checkouts tl;dr: different google have many thousands of buildable and deployable things, which have very different release schedules. facebook don’t as they substantially have the php web-app, and apps for ios and android in different repos. well at least the main php web-app is in the mercurial trunk they talked about on tuesday. i’m not sure how the ios and android apps are managed, but at least the android one is outside the main trunk. google subset their trunk. i posted about that on monday . in that article i pointed out that the checkout can grow (or shrink) depending on the nature of the change being undertaken. it’s very different to a multiple-small-repos design. facebook don’t subset their trunk on checkout, as they do not need to; the head revisions of everything in that trunk are not big enough for a c: drive or ide to buckle. there’s also no compile stage for php , for regular development work. maximized sharing of code tl;dr: the same code is shared using globbed directories within the source tree. it’s shared as source files, in situ, rather than classes in a jar (or equivalent). refactoring tl;dr: the same developers take on refactorings where appropriate. sure it means a bigger atomic commit, but knowing all the affected source is in front of you as you do the refactoring is comforting. at least, knowing that if intellij (or eclipse, etc) completes the refactoring there’s a very strong possibility that the build will stay green, and that you’re only going to have a slight impact on other people’s working copy, and only if they are concurrently editing the same files. bigger refactoring probably still require a warning email. super tooling of the build phase tl;dr: the same google have what amounts to a super-computer doing the compilation for them (all languages that are compiled). all developers and all ci daemons leverage it. and by effective super-computer, i mean previous-compiled bits and pieces are pulled out of an internal cloud-map-thing for source permutations that have been compiled before. the distributed hashmap is possibly lru centric rather that everything forever. facebook don’t have that big hashmap of recently compiled bits and pieces, but they do have hiphop in the toolchain (originally a php to c++ compiler) which is interesting because at face value php is an interpreted language and ‘compile’ makes no sense. hiphop was created to reduce the server footprint and requirements for production deployments, while still being 100% functionally identical to the interpreted php app. it’s also faster in production. more recently hiphop became a virtual machine. it continues to be incrementally improved. like google, facebook can measure cost-benefit of continued work on it (prod rack space & prod electricity vs developer salaries). source-control weapons of choice tl;dr: different google use perforce for their trunk (with additional tooling), and many (but not all) developers use git on their local workstation to gain local-branching with an inhouse developed bridge for interop with perforce. facebook uses mercurial with additional tooling for the central server/repo. it’s unclear whether developers, by habit, exist with the mercurial client software or use git which can interop with mercurial backends. both google and facebook do trunk based development of course. branches & merge pain tl;dr: the same they don’t have merge pain, because as a rule developers are not merging to/from branches. at least up to the central repo’s server they are not. on workstations, developers may be merging to/from local branches, and rebasing when the push something that’s “done” back to the central repo. release engineers might cherry-pick defect fixes from time to time, but regular developers are not merging (you should not count to-working-copy merges) eating own dog-food tl;dr: mostly different all staff at facebook use a not-live-yet version of the web-app for all of their communication, documentation, management etc. if there’s a bug everyone feels it – though selenium2 functional tests and zillions of unit-tests guard against that happening too often. google has too many different apps for the team making each to be said to be a daily user of it. for example the adsense developer may use a dog-food version of gmail, but they are making adsense, so are hardly hurting themselves as they are not minute by minute using the interface as part of their regular existence at google. code review tl;dr: same both google and facebook insist on code reviews before the commit is accepted into the remote repo’s trunk for all others to use. there’s no mechanism of code review that’s more efficient or effective. google back in 2009 were pivoting incoming changes to the trunk around the code-review process managed by mondrian. i wrote about that in “continuous review #1” in december . i think they are unchanged in that respect: developers actively push their commit after a code review has been completed. facebook have just flipped to mercurial (from subversion). in the article linked to at the top of the page, facebook have not mentioned “pull request” or “patch queue”, or indeed “code review”. the article was mostly about speed, robustness and scale. i suspect they are sitting within the semantics of mercurials patch-queue processing though, although assigning a bot to it rather than a human. update: simon stewart pinged me and reminded me that they use (and made) phabricator. he spoke about it in a mobile@scale presentation, and that video is here . in the video he says the review is queue based now, but that they experimenting with landing the change sets into the master now. the video is from november, and was for the android + ios platforms, but it is likely to be used today for the main trunk for the php web-app. automated testing tl;dr: same heavy reliance on unit tests (not necessarily made in a tdd style). later in an build pipeline, selenium2 tests (for web-apps at least) kick in to guard the functional quality of deployed app. manual qa tl;dr: mostly the same both companies have progressively moved way from manual qa and dedicated testing professionals, towards developers testing their own stuff at discrete moments (note the dog-food item above too). prod release frequency tl;dr: it varies. facebook for the main web app, are twice a day presently (at least on weekdays). i published info on that at the start of last year. google have many apps with different release schedules, and some are “many times a day”, while others are “planned releases every few weeks”. many are in between. prod db deployment tl;dr: mostly the same database (or equivalent) table shapes (or equivalent) are designed to be forwards/backwards compatible as far as possible. pull requests as part of workflow tl;dr: same etsy, github, and other high throughput organizations are trunking by some definition, but using pull-requests to merge in things being done. it has different obligations if done, but google and facebook are not doing this in their trunks – they both essentially push (after review). refer the ‘code review’ section above. common code ownership tl;dr: the same you can commit to any part of the source tree, provided it passed a fair code review. notional owners of directories within the source tree take a boy-scout pledge to do their best with unsolicited incoming change-lists. there are strong permissions in the google perforce implementation, but the pledge means that contributions are not often rejected if the merit is there. build is ever broken tl;dr: the same almost never. directionality of merge for prod bug fixes tl;dr: the same trunk receives the defect fix, it gets cherry picked to the release branch. the release branch might have been made from a tag, if it didn’t exist before. binary dependencies tl;dr: the same checked into source-control without version suffixing (harmonized versions across all apps). e.g. – log4j.jar rather than log4j-1.2.8.jar.
January 21, 2014
by Paul Hammant
· 18,858 Views
article thumbnail
Python Script to Delete Merged Git Branches
One of the great things about git is how fast it is. You can create a new branch, or switch to another branch, almost as fast as you can type the command. This tends to lower the impedance of branching. As a result, many individuals and teams will naturally converge on a process where they create many, many branches. If you’re like me, you may have 30 branches at any given time. This can make viewing all the branches unwieldy. Once I week or so, I would go on a branch deletion spree by manually copying and pasting multiple branch names into a git branch -D statement. The basic use case is that you want to delete any branches that are already merged into master. Here is a python script that automated just that. from subprocess import check_output import sys def get_merged_branches(): ''' a list of merged branches, not couting the current branch or master ''' raw_results = check_output('git branch --merged upstream/master', shell=True) return [b.strip() for b in raw_results.split('\n') if b.strip() and not b.startswith('*') and b.strip() != 'master'] def delete_branch(branch): return check_output('git branch -D %s' % branch, shell=True).strip() if __name__ == '__main__': dry_run = '--confirm' not in sys.argv for branch in get_merged_branches(): if dry_run: print branch else: print delete_branch(branch) if dry_run: print '*****************************************************************' print 'Did not actually delete anything yet, pass in --confirm to delete' print '*****************************************************************' To print the branches that would be deleted, just execute python delete_merged_branches.py. To actually delete the branches, execute python delete_merged_branches.py --confirm.
January 21, 2014
by Chase Seibert
· 8,137 Views
article thumbnail
Building a Samlple Java WebSocket Client
Learn more about creating Java-based WebSocket clients, including code for the server side WebSocket application and the corresponding JavaScript/HTML client.
January 8, 2014
by Ulas Ergin
· 224,574 Views · 10 Likes
article thumbnail
Introduction to Codenvy
what is codenvy exactly? well, their website states: codenvy is a cloud environment for coding, building, and debugging apps. basically, it’s an ide in the cloud (“ide as a service?”) accessible by all the major browsers . it started out as an additional feature to the exo platform in early 2009 and gained a lot of traction after the first paas (openshift) and git integration was added mid-2011. codenvy targets me as a (java) software developer to run and debug applications in their hosted cloud ide, while being able to share and collaborate during development and finally publish to a repository – e.g. git – or a number of deployment platforms – e.g. amazon, openshift or google app engine. i first encountered their booth at javaone last september, but they couldn’t demo their product right there on the spot over the wifi, because their on-line demo workspace never finished loading well i got the t-shirt instead then, but now’s the time to see what codenvy has in store as a cloud ide. signing up signing up took 3 seconds. all you have to do is go to codenvy.com , use the “sign up” button, choose an email address and a name for your workspace , confirm the email they’ll send you and you’re done. the “workspace” holds all your projects and is part of the url codenvy will create for you, like “ https://codenvy.com/ide/ . although not very clear during the registration process – which of course nowadays is usually minimalistic as can be – it seems that i’ve signed up for codenvy’s free community plan , which gives me an unlimited number of public projects. you can even start coding without registration. after confirming the registration mail, i’m in. finally i’ll end up in the browser where your (empty) workspace has been opened. empty workspace a few options a possible for here on, as seen in the figure above: create a new project from scratch – generate an empty project from predefined project types import from github – import projects from your github account clone a git repository – create a new project from any public git reposiroty browse documentation invite people – get team members on board support – questions, feedback and troubleshooting let’s… create a new project from scratch this option allows you to name the new project – e.g. “myproject”, choose a technology and a paas . the technology is a defined set of languages of frameworks to develop with. available technologies at the moment the technologies are: java jar java war java spring javascript ruby on rails python php node.js android maven multi-module at the time of writing java 1.6 is supported. available paas at the moment the available platforms are: amazon webservices (aws) elastic beanstalk savvis cloud appfrog cloudbees google app engine (gae) heroku manymo android emulator red hat’s openshift none depending on the choice of technology, or or more paas options become available. a single jar can not be deployed onto any of the platforms, leaving only the option “none” available. a java web application (war) can be deployed onto any number of platforms, except heroku and manymo. node.js can only be deployed to openshift. creating a simple jar project after having selected a jar (and no platform) one can select a project template . e.g. if webapplication (war) would have been selected, codenvy would present project templates, such as google app engine java project illustrating simple examples that use the search api , java web project with datasource usage or a demonstration of accessing amazon s3 buckets using the java sdk . the jar technology has only one project: simple jar project . after having finished the wizard, our jar project has been created in our workspace. we’ll see two views of our project: a project explorer and a package explorer. project- and package explorer what we can see is that our jar project has been given a maven pom.xml with the following content: view source print ? 01. < project xmlns = " http://maven.apache.org/pom/4.0.0 " xmlns:xsi = " http://www.w3.org/2001/xmlschema-instance " 02. xsi:schemalocation = " http://maven.apache.org/pom/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd " > 03. < modelversion >4.0.0 04. < groupid >com.codenvy.workspaceyug8g52wjwb5im13 05. < artifactid >testjarproject 06. < version >1.0-snapshot 07. < packaging >jar 08. 09. < name >sample-lib 10. 11. < properties > 12. < project.build.sourceencoding >utf-8 13. 14. 15. < dependencies > 16. < dependency > 17. < groupid >junit 18. < artifactid >junit 19. < version >3.8.1 20. < scope >test 21. 22. 23. we have a generated group id com.codenvy.workspaceyug8g52wjwb5im13 , our own artifact id and the junit dependency, which is a decent choice for many java developers use it as a testing framework. the source encoding has already been set to utf-8, which is also a sensible choice. as a convenience we’ve also been given a hello.sayhello class, so we know we’re actually in a java project say hello file & project management so what about the browser-based editor we’re working in? on top we’re seeing a few menu’s, like file, project, edit, view, run, git, paas, window, share and help . i’ll be highlighting a few. file- and project menu the file menu allows to creating folders , packages and various kind of filetypes , such as text, xml (1.0 at time of writing) , html (4.1) , css (2.0), java classes and jsp’s (2.1). although i’m in a jar project, i am still also able to create here e.g. ruby, php or python files. a very convenient feature is to upload existing files to the workspace, either separately or in zip archives. i’ve tried dropping a file onto the package explorer from the file system, but the browser (in this case, chrome) tries to open it instead the project menu allows to create new projects, either by launching the create project wizard again, but also allows for importing from github . in order to clone a repository, you’ll have to authorize codenvy to access github.com to be able to import a project. after having authorized github, codeenvy presents me with a list of projects to choose from. after having imported all necessary stuff, it somehow needs to know what kind of project i’m importing. selecting a file type after importing a project from github the project i imported didn’t give codenvy any clues as to what kind of project it is (which is right since i only had a readme.md in it), so it lists a few options to choose from. i chose the maven multi-module type after which the output window shows: [email protected]:tvinke/examples.git was successfully cloned. [info] project type updated. if you’d have a pom.xml in the root of your project, it would immediately recognize it a s a maven project. apart from going through the project > import from github option, you can also go directly to the git menu, and choose clone repository . this allows you to manually enter the remote repository uri, wanted project name and the remote name (e.g. “origin”). cloning a repository one you have pulled in a git project, the git menu allows all kinds of common operations, such as adding and removing files, committing, pushing, pulling and much more. git menu the ssh keys can be found under menu window > preferences where you can view the github.com entry, where one can view the details or delete it. also a new key can be either generated or uploaded here. sharing the project one of the unique selling points of codenvy are their collaboration possibilities which come along with any project. you can: invite other developers with read-only rights or full read-write rights to your workspace and every project in it.when you’re pair-programming like this, or co-editing a file with a colleague, you can also send each other code pointers – small shortcuts to code lines. use factories to create temporary workspaces , through cloning, off one source project (“factory”) and represent the cloning mechanism as a url which can be given to other developers. a use case might be to get a colleague quickly started on a project by providing a fully working development environment.there’s a lot more about creating factories in the docs (such as through rest), but the nice thing is that once you have a factory url, you can embed it as a button, send it through email of publish it somewhere for others! a factory url to load up e.g. their twitter bootstrap sample – as they use on their website themselves – looks like: https://codenvy.com/factory?v=1.0&pname=sample-twitterbootstrap&wname=codenvy-factories&vcs=git&vcsurl=http%3a%2f%2fcodenvy.com%2fgit%2f04%2f0f%2f7f%2fworkspacegcpv6cdxy1q34n1i%2fsample-twitterbootstrap&idcommit=c1443ecea63471f5797f172c081cd802bac6e6b0&action=openproject&ptype=javascript conclusion applications are run in the cloud nowadays, so why not create them there too? codenvy brings some interesting features, such as being able to instantly provision workspaces (through factory urls) and share projects in real-time. it supports common operations with projects, files and version control. with a slew of languages and platforms and as an ide being always accessible through the internet, it could lower the barrier to actually code anytime and anywhere. in a future post i will try and see whether or not it can actually replace my conventional desktop ide for java development.
January 4, 2014
by Ted Vinke
· 7,964 Views
article thumbnail
Top Posts of 2013: Google's Big Data Papers
I’ll review Google’s most important Big Data publications and discuss where they are (as far as they’ve disclosed).
December 30, 2013
by Mikio Braun
· 117,185 Views
article thumbnail
Building and Testing a WebSocket Server with Undertow
the upcoming version of jboss application server will no longer use tomcat as the integrated webserver, but instead, will replace it with undertow . the architecture of undertow is based on handlers that can be added dynamically via a builder api to the server. this approach is similar to the way of constructing a webserver in node.js . it allows developers to embed the undertow webserver easily into their applications. as the addition of features is done via the builder api, one can only add the features that are really required in one’s application. beyond that undertow supports websockets and the servlet api in version 3.1. it can be run as a blocking or non-blocking server and it is reported that first benchmarks have proven that undertow is the fastest webserver written in java. as all of this sounds very promising, so let’s try to set up a simple websocket server. as usual we start by creating a simple java project and add the undertow maven dependency: io.undertow undertow-core 1.0.0.beta20 with undertow’s builder api our buildandstartserver() method looks like this: public void buildandstartserver(int port, string host) { server = undertow.builder() .addlistener(port, host) .sethandler(getwebsockethandler()) .build(); server.start(); } we just add a listener that specifies the port and host to listen for incoming connections and afterwards add a websocket handler. as the websocket handler code is a little bit more comprehensive, i have put it into its own method: private pathhandler getwebsockethandler() { return path().addpath("/websocket", websocket(new websocketconnectioncallback() { @override public void onconnect(websockethttpexchange exchange, websocketchannel channel) { channel.getreceivesetter().set(new abstractreceivelistener() { @override protected void onfulltextmessage(websocketchannel channel, bufferedtextmessage message) { string data = message.getdata(); lastreceivedmessage = data; logger.info("received data: "+data); websockets.sendtext(data, channel, null); } }); channel.resumereceives(); } })) .addpath("/", resource(new classpathresourcemanager(websocketserver.class.getclassloader(), websocketserver.class.getpackage())) .addwelcomefiles("index.html")); } let’s go line by line through this code snippet. first of all, we add a new path: /websocket. the second argument of the addpath() methods lets us specify what kind of protocol we want to use for this path. in our case we create a new websocket. the anonymous implementation has a onconnect() method in which we set an implementation of abstractreceivelistener. here we have a convenient method onfulltextmessage() that is called when a client has sent us a text message. a call of getdata() fetches the actual message we have received. in this simple example we just echo this string back to client to validate that the roundtrip from the client to server and back works. to perform some simple manual tests we also add a second resource under the path / which serves some static html and javascript files. the directory that contains these files is given as an instance of classpathresourcemanager. the call of addwelcomefiles() tells undertow which file to server when the client asks for the path /. the index.html looks like this: our javascript code is swapped out to the websocket.js file. we use jquery and the jquery-plugin gracefulwebsocket to ease the client side development: var ws = $.gracefulwebsocket("ws://127.0.0.1:8080/websocket"); ws.onmessage = function(event) { var messagefromserver = event.data; $('#output').append('received: '+messagefromserver+''); } function send(message) { ws.send(message); } after having created a websocket object by calling $.gracefulwebsocket() we can register a callback function for incoming messages. in this method we only append the message string to the dom of the page. the send() method is just a call to gracefulwebsocket’s send() method. when we now start our application and open the url http://127.0.0.1:8080/ in our webbrowser we see the following page: entering some string and hitting the “send web socket data” button sends the message to the server, which in response echos it back to the client. now that we know that everything works as expected, we want to protect our code against regression with a junit test case. as a websocket client i have chosen the library jetty-websocket: org.eclipse.jetty jetty-websocket 8.1.0.rc5 test in the test case we build and start the websocket server to open a new connection to the websocket port. the websocket implementation of jetty-websocket allows us to implement two callback methods for the open and close events. within the open callback we send the test message to the client. the rest of the code waits for the connection to be established, closes it and asserts that the server has received the message: @test public void teststartandbuild() throws exception { subject = new websocketserver(); subject.buildandstartserver(8080, "127.0.0.1"); websocketclient client = new websocketclient(); future connectionfuture = client.open(new uri("ws://localhost:8080/websocket"), new websocket() { @override public void onopen(connection connection) { logger.info("onopen"); try { connection.sendmessage("testmessage"); } catch (ioexception e) { logger.error("failed to send message: "+e.getmessage(), e); } } @override public void onclose(int i, string s) { logger.info("onclose"); } }); websocket.connection connection = connectionfuture.get(2, timeunit.seconds); assertthat(connection, is(notnullvalue())); connection.close(); subject.stopserver(); thread.sleep(1000); assertthat(subject.lastreceivedmessage, is("testmessage")); } as usual you can find the source code on github . conclusion: undertow’s builder api makes it easy to construct a websocket server and in general an embedded webserver that fits your needs. this also eases automatic testing as you do not need any specific maven plugin that starts and stops your server before and after your integration tests. beyond that the jquery plugin jquery-graceful-websocket lets you send and receive messages over websockets with only a few lines of code.
December 23, 2013
by Martin Mois
· 27,563 Views
article thumbnail
Logging, Processing and Monitoring Data using Talend, ElasticSearch, Logstash and Kibana
Your mission-critical projects need complex event processing, realtime management and monitoring. Talend 5.4 (released in December 2013, https://www.talend.com) offers a great new feature: Talend Event Logging. It allows logging, processing and monitoring of all technical events and business data. In this article, I will focus on how to process, filter and monitor business data. You can find more details about monitoring technical events and logs (e.g. OSGi events of the ESB container) in Talend’s documentation (www.help.talend.com). This new feature is very powerful, but also extendable to fit custom requirements. You can solve and monitor much more complex scenarios than the one I describe here. Talend Event Logging with Logstash, ElasticSearch and Kibana First, let’s take a look at the components / projects which are integrated and extended into Talend’s products for implementing this new feature. logstash (http://logstash.net) is a tool for managing events and logs. You can use it to collect different (distributed) logs, parse them, and store them for later use. Speaking of searching, logstash comes with a simple, but fine web interface for searching and drilling into all of your logs. It uses ElasticSearch under the hood. So, you can easily query through all your logs for specific errors or business analytics (e.g. searching for all lines matching an unique order id). Additional to the pure collection of events, the Event Logging feature supports custom processing (e.g. custom filtering, customer data enrichment/reduction), aggregation, signing and also server side custom pre- and post-processing of events - e.g. to send them to an intrusion detection system or to any other kind of potential higher level log processing /management system. Kibana (http://www.elasticsearch.org/overview/kibana) is a browser based analytics and search interface to logstash and other timestamped data sets stored in ElasticSearch. Kibana strives to be easy to get started with, while also being flexible and powerful, just like logstash and ElasticSearch. Main difference to logstash is a much more powerful HTML5 based web interface. You can • use multiple concurrent search inputs • highlight to drill down bar charts • create line charts, stacked, unstacked, filled or unfilled, with or without points • create Pie and donut charts that compare top terms or the results of multiple queries • create custom dashboards with multiple charts • and much more… Therefore, logstash, Elasticsearch and Kibana are a perfect combination. You can use Kibana to analyze and monitor your data as you do with logstash, however, Kibana’s web interface is much more powerful and comfortable than logstash. There is a great book about logstash: “The logstash book” – for just 9.99 USD. I can really recommend this book for getting started: http://www.logstashbook.com/. For Elasticsearch, you can find several books on Amazon. Unfortunately, Kibana has no good and extensive documentation yet. I heard from its developers that this topic is addressed for Q1 2014. Integration of Event Logging and Monitoring into Talend’s Unified Platform Talend 5.4 has integrated logstash and Kibana into its Unified Platform. Talend Administration Center (TAC) is Talend’s central web application for management and monitoring. It got a new logging view: Here, you can use very flexible and powerful realtime search capabilities of Elasticsearch. Some dashboards are available by default. You can also create your own custom dashboards easily within this site thanks to Kibana. Many panel types are available, such as pie, histogram, table, hits or trends. You can analyze every technical event or business data down to the message level: By default, you see fields such as message, source, timestamp, type, and others. Of course, you can also add custom fields suitable for your business case. This way, you have a central monitoring capability which allows analyzing data on distributed clusters easily. Under the hood, this data comes from logstash. Many alternative inputs are available for logstash, such as log4j input or tcp input. For business data, I often use file input (http://logstash.net/docs/1.2.2/inputs/file) to analyze files such as CSV. Adding new inputs is very simple. You just have to add an input to your logstash configuration file of your logstash server. As I mentioned already, all this is integrated into the Talend’s Unified Platform: In this example, there are three log4j inputs and one file input. I use the file input to analyze text files in a specific directory. In this case, output is an embedded Elasticsearch instance. In production, you should use an external Elasticsearch cluster, of course. Processing such as filtering can also be configured in this file. Thus, you can process, analyze and monitor all your different inputs within one central monitoring application thanks to Kibana. Building Talend Integration Jobs, Routes and Web Services As mentioned before, you can analyze almost all data with logstash, Elasticsearch and Kibana. It does not matter if your input is technical events from a container (e.g. OSGi events) or any business data such as CSV files, log4j logs, or something else. Talend implicitly supports technical events which are created by the ESB container, by MDM, etc. However, you can also add your custom business data from your Talend jobs (integration perspective), SOAP / REST Web Services (integration perspective) or Talend routes (mediation perspective), easily: This is just one example (part of Talend’s DI demos which are included in every DI installation). The job generates some random data and stores it to a CSV file. You just have to add the configured file or directory of tFileOutputDelimited to your logstash configuration using file input (with wild cards for more complex scenarios). That’s it. You can now monitor and analyze the business data in realtime. This example showed a Talend DI Job (i.e. ETL job). However, you can also monitor your business data from SOAP / REST Web Services or Talend Routes the same way. Conclusion Your mission-critical projects need management and monitoring. Today, this is not just possible with complex and expensive tools of large vendors, but also with Talend’s Unified Platform products such as Talend ESB or Talend MDM. Under the hood, Talend integrates and extends widely used open source products: logstash, Elasticsearch and Kibana. Have fun with Talend 5.4’s new event logging and monitoring features… Best regards, Kai Wähner (@KaiWaehner) CONTENT FROM MY BLOG: http://www.kai-waehner.de/blog/2013/12/17/realtime-event-logging-complex-event-processing-cep-and-monitoring-with-talends-unified-platform-5-4-di-esb-dq-mdm-bpm-using-elasticsearch-logstash-and-kibana/
December 19, 2013
by Kai Wähner DZone Core CORE
· 31,566 Views
article thumbnail
Java WebSockets (JSR-356) on Jetty 9.1
Jetty 9.1 is finally released, bringing Java WebSockets (JSR-356) to non-EE environments. It's awesome news and today's post will be about using this great new API along with Spring Framework. JSR-356 defines concise, annotation-based model to allow modern Java web applications easily create bidirectional communication channels using WebSockets API. It covers not only server-side, but client-side as well, making this API really simple to use everywhere. Let's get started! Our goal would be to build a WebSockets server which accepts messages from the clients and broadcasts them to all other clients currently connected. To begin with, let's define the message format, which server and client will be exchanging, as this simple Message class. We can limit ourselves to something like a String, but I would like to introduce to you the power of another new API - Java API for JSON Processing (JSR-353). package com.example.services; public class Message { private String username; private String message; public Message() { } public Message( final String username, final String message ) { this.username = username; this.message = message; } public String getMessage() { return message; } public String getUsername() { return username; } public void setMessage( final String message ) { this.message = message; } public void setUsername( final String username ) { this.username = username; } } To separate the declarations related to the server and the client, JSR-356 defines two basic annotations:@ServerEndpoint and @ClientEndpoit respectively. Our client endpoint, let's call itBroadcastClientEndpoint, will simply listen for messages server sends: package com.example.services; import java.io.IOException; import java.util.logging.Logger; import javax.websocket.ClientEndpoint; import javax.websocket.EncodeException; import javax.websocket.OnMessage; import javax.websocket.OnOpen; import javax.websocket.Session; @ClientEndpoint public class BroadcastClientEndpoint { private static final Logger log = Logger.getLogger( BroadcastClientEndpoint.class.getName() ); @OnOpen public void onOpen( final Session session ) throws IOException, EncodeException { session.getBasicRemote().sendObject( new Message( "Client", "Hello!" ) ); } @OnMessage public void onMessage( final Message message ) { log.info( String.format( "Received message '%s' from '%s'", message.getMessage(), message.getUsername() ) ); } } That's literally it! Very clean, self-explanatory piece of code: @OnOpen is being called when client got connected to the server and @OnMessage is being called every time server sends a message to the client. Yes, it's very simple but there is a caveat: JSR-356 implementation can handle any simple objects but not the complex ones like Message is. To manage that, JSR-356 introduces concept of encoders and decoders. We all love JSON, so why don't we define our own JSON encoder and decoder? It's an easy task which Java API for JSON Processing (JSR-353) can handle for us. To create an encoder, you only need to implementEncoder.Text< Message > and basically serialize your object to some string, in our case to JSON string, using JsonObjectBuilder. package com.example.services; import javax.json.Json; import javax.json.JsonReaderFactory; import javax.websocket.EncodeException; import javax.websocket.Encoder; import javax.websocket.EndpointConfig; public class Message { public static class MessageEncoder implements Encoder.Text< Message > { @Override public void init( final EndpointConfig config ) { } @Override public String encode( final Message message ) throws EncodeException { return Json.createObjectBuilder() .add( "username", message.getUsername() ) .add( "message", message.getMessage() ) .build() .toString(); } @Override public void destroy() { } } } For decoder part, everything looks very similar, we have to implement Decoder.Text< Message > and deserialize our object from string, this time using JsonReader. package com.example.services; import javax.json.JsonObject; import javax.json.JsonReader; import javax.json.JsonReaderFactory; import javax.websocket.DecodeException; import javax.websocket.Decoder; public class Message { public static class MessageDecoder implements Decoder.Text< Message > { private JsonReaderFactory factory = Json.createReaderFactory( Collections.< String, Object >emptyMap() ); @Override public void init( final EndpointConfig config ) { } @Override public Message decode( final String str ) throws DecodeException { final Message message = new Message(); try( final JsonReader reader = factory.createReader( new StringReader( str ) ) ) { final JsonObject json = reader.readObject(); message.setUsername( json.getString( "username" ) ); message.setMessage( json.getString( "message" ) ); } return message; } @Override public boolean willDecode( final String str ) { return true; } @Override public void destroy() { } } } And as a final step, we need to tell the client (and the server, they share same decoders and encoders) that we have encoder and decoder for our messages. The easiest thing to do that is just by declaring them as part of @ServerEndpoint and @ClientEndpoit annotations. import com.example.services.Message.MessageDecoder; import com.example.services.Message.MessageEncoder; @ClientEndpoint( encoders = { MessageEncoder.class }, decoders = { MessageDecoder.class } ) public class BroadcastClientEndpoint { } To make client's example complete, we need some way to connect to the server usingBroadcastClientEndpoint and basically exchange messages. The ClientStarter class finalizes the picture: package com.example.ws; import java.net.URI; import java.util.UUID; import javax.websocket.ContainerProvider; import javax.websocket.Session; import javax.websocket.WebSocketContainer; import org.eclipse.jetty.websocket.jsr356.ClientContainer; import com.example.services.BroadcastClientEndpoint; import com.example.services.Message; public class ClientStarter { public static void main( final String[] args ) throws Exception { final String client = UUID.randomUUID().toString().substring( 0, 8 ); final WebSocketContainer container = ContainerProvider.getWebSocketContainer(); final String uri = "ws://localhost:8080/broadcast"; try( Session session = container.connectToServer( BroadcastClientEndpoint.class, URI.create( uri ) ) ) { for( int i = 1; i <= 10; ++i ) { session.getBasicRemote().sendObject( new Message( client, "Message #" + i ) ); Thread.sleep( 1000 ); } } // Application doesn't exit if container's threads are still running ( ( ClientContainer )container ).stop(); } } Just couple of comments what this code does: we are connecting to WebSockets endpoint atws://localhost:8080/broadcast, randomly picking some client name (from UUID) and generating 10 messages, every with 1 second delay (just to be sure we have time to receive them all back). Server part doesn't look very different and at this point could be understood without any additional comments (except may be the fact that server just broadcasts every message it receives to all connected clients). Important to mention here: new instance of the server endpoint is created every time new client connects to it (that's why peers collection is static), it's a default behavior and could be easily changed. package com.example.services; import java.io.IOException; import java.util.Collections; import java.util.HashSet; import java.util.Set; import javax.websocket.EncodeException; import javax.websocket.OnClose; import javax.websocket.OnMessage; import javax.websocket.OnOpen; import javax.websocket.Session; import javax.websocket.server.ServerEndpoint; import com.example.services.Message.MessageDecoder; import com.example.services.Message.MessageEncoder; @ServerEndpoint( value = "/broadcast", encoders = { MessageEncoder.class }, decoders = { MessageDecoder.class } ) public class BroadcastServerEndpoint { private static final Set< Session > sessions = Collections.synchronizedSet( new HashSet< Session >() ); @OnOpen public void onOpen( final Session session ) { sessions.add( session ); } @OnClose public void onClose( final Session session ) { sessions.remove( session ); } @OnMessage public void onMessage( final Message message, final Session client ) throws IOException, EncodeException { for( final Session session: sessions ) { session.getBasicRemote().sendObject( message ); } } } In order this endpoint to be available for connection, we should start the WebSockets container and register this endpoint inside it. As always, Jetty 9.1 is runnable in embedded mode effortlessly: package com.example.ws; import org.eclipse.jetty.server.Server; import org.eclipse.jetty.servlet.DefaultServlet; import org.eclipse.jetty.servlet.ServletContextHandler; import org.eclipse.jetty.servlet.ServletHolder; import org.eclipse.jetty.websocket.jsr356.server.deploy.WebSocketServerContainerInitializer; import org.springframework.web.context.ContextLoaderListener; import org.springframework.web.context.support.AnnotationConfigWebApplicationContext; import com.example.config.AppConfig; public class ServerStarter { public static void main( String[] args ) throws Exception { Server server = new Server( 8080 ); // Create the 'root' Spring application context final ServletHolder servletHolder = new ServletHolder( new DefaultServlet() ); final ServletContextHandler context = new ServletContextHandler(); context.setContextPath( "/" ); context.addServlet( servletHolder, "/*" ); context.addEventListener( new ContextLoaderListener() ); context.setInitParameter( "contextClass", AnnotationConfigWebApplicationContext.class.getName() ); context.setInitParameter( "contextConfigLocation", AppConfig.class.getName() ); server.setHandler( context ); WebSocketServerContainerInitializer.configureContext( context ); server.start(); server.join(); } } The most important part of the snippet above is WebSocketServerContainerInitializer.configureContext: it's actually creates the instance of WebSockets container. Because we haven't added any endpoints yet, the container basically sits here and does nothing. Spring Framework and AppConfig configuration class will do this last wiring for us. package com.example.config; import javax.annotation.PostConstruct; import javax.inject.Inject; import javax.websocket.DeploymentException; import javax.websocket.server.ServerContainer; import javax.websocket.server.ServerEndpoint; import javax.websocket.server.ServerEndpointConfig; import org.eclipse.jetty.websocket.jsr356.server.AnnotatedServerEndpointConfig; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.web.context.WebApplicationContext; import com.example.services.BroadcastServerEndpoint; @Configuration public class AppConfig { @Inject private WebApplicationContext context; private ServerContainer container; public class SpringServerEndpointConfigurator extends ServerEndpointConfig.Configurator { @Override public < T > T getEndpointInstance( Class< T > endpointClass ) throws InstantiationException { return context.getAutowireCapableBeanFactory().createBean( endpointClass ); } } @Bean public ServerEndpointConfig.Configurator configurator() { return new SpringServerEndpointConfigurator(); } @PostConstruct public void init() throws DeploymentException { container = ( ServerContainer )context.getServletContext(). getAttribute( javax.websocket.server.ServerContainer.class.getName() ); container.addEndpoint( new AnnotatedServerEndpointConfig( BroadcastServerEndpoint.class, BroadcastServerEndpoint.class.getAnnotation( ServerEndpoint.class ) ) { @Override public Configurator getConfigurator() { return configurator(); } } ); } } As we mentioned earlier, by default container will create new instance of server endpoint every time new client connects, and it does so by calling constructor, in our caseBroadcastServerEndpoint.class.newInstance(). It might be a desired behavior but because we are usingSpring Framework and dependency injection, such new objects are basically unmanaged beans. Thanks to very well-thought (in my opinion) design of JSR-356, it's actually quite easy to provide your own way of creating endpoint instances by implementing ServerEndpointConfig.Configurator. TheSpringServerEndpointConfigurator is an example of such implementation: it's creates new managed bean every time new endpoint instance is being asked (if you want single instance, you can create singleton of the endpoint as a bean in AppConfig and return it all the time). The way we retrieve the WebSockets container is Jetty-specific: from the attribute of the context with name"javax.websocket.server.ServerContainer" (it probably might change in the future). Once container is there, we are just adding new (managed!) endpoint by providing our own ServerEndpointConfig (based onAnnotatedServerEndpointConfig which Jetty kindly provides already). To build and run our server and clients, we need just do that: mvn clean package java -jar target\jetty-web-sockets-jsr356-0.0.1-SNAPSHOT-server.jar // run server java -jar target/jetty-web-sockets-jsr356-0.0.1-SNAPSHOT-client.jar // run yet another client As an example, by running server and couple of clients (I run 4 of them, '392f68ef', '8e3a869d', 'ca3a06d0', '6cb82119') you might see by the output in the console that each client receives all the messages from all other clients (including its own messages): Nov 29, 2013 9:21:29 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Hello!' from 'Client' Nov 29, 2013 9:21:29 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #1' from '392f68ef' Nov 29, 2013 9:21:29 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #2' from '8e3a869d' Nov 29, 2013 9:21:29 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #7' from 'ca3a06d0' Nov 29, 2013 9:21:30 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #4' from '6cb82119' Nov 29, 2013 9:21:30 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #2' from '392f68ef' Nov 29, 2013 9:21:30 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #3' from '8e3a869d' Nov 29, 2013 9:21:30 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #8' from 'ca3a06d0' Nov 29, 2013 9:21:31 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #5' from '6cb82119' Nov 29, 2013 9:21:31 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #3' from '392f68ef' Nov 29, 2013 9:21:31 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #4' from '8e3a869d' Nov 29, 2013 9:21:31 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #9' from 'ca3a06d0' Nov 29, 2013 9:21:32 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #6' from '6cb82119' Nov 29, 2013 9:21:32 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #4' from '392f68ef' Nov 29, 2013 9:21:32 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #5' from '8e3a869d' Nov 29, 2013 9:21:32 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #10' from 'ca3a06d0' Nov 29, 2013 9:21:33 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #7' from '6cb82119' Nov 29, 2013 9:21:33 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #5' from '392f68ef' Nov 29, 2013 9:21:33 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #6' from '8e3a869d' Nov 29, 2013 9:21:34 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #8' from '6cb82119' Nov 29, 2013 9:21:34 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #6' from '392f68ef' Nov 29, 2013 9:21:34 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #7' from '8e3a869d' Nov 29, 2013 9:21:35 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #9' from '6cb82119' Nov 29, 2013 9:21:35 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #7' from '392f68ef' Nov 29, 2013 9:21:35 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #8' from '8e3a869d' Nov 29, 2013 9:21:36 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #10' from '6cb82119' Nov 29, 2013 9:21:36 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #8' from '392f68ef' Nov 29, 2013 9:21:36 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #9' from '8e3a869d' Nov 29, 2013 9:21:37 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #9' from '392f68ef' Nov 29, 2013 9:21:37 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #10' from '8e3a869d' Nov 29, 2013 9:21:38 PM com.example.services.BroadcastClientEndpoint onMessage INFO: Received message 'Message #10' from '392f68ef' 2013-11-29 21:21:39.260:INFO:oejwc.WebSocketClient:main: Stopped org.eclipse.jetty.websocket.client.WebSocketClient@3af5f6dc Awesome! I hope this introductory blog post shows how easy it became to use modern web communication protocols in Java, thanks to Java WebSockets (JSR-356), Java API for JSON Processing (JSR-353) and great projects such as Jetty 9.1! As always, complete project is available on GitHub.
December 6, 2013
by Andriy Redko
· 40,766 Views · 2 Likes
article thumbnail
Deconstructing the Azure Point-to-Site VPN for Command Line usage
when configuring an azure virtual network one of the most common things you'll want to do is setup a point-to-site vpn so that you can actually get to your servers to manage and maintain them. azure point-to-site vpns use client certificates to secure connections which can be quite complicated to configure so microsoft has gone the extra mile to make it easy for you to configure and get setup – sadly at the cost of losing the ability to connect through the command line or through powershell – let's change that. current state of play == no command line vpn connections normally when you want to launch a vpn from the cli or powershell in windows you can simply use the following command: rasdial "my home vpn" the azure pre-packaged vpn doesn't allow this because it's really just not a normal vpn. it's something else , something mysterious - not a normal native windows vpn connection. when you run the azure vpn through the command line you get this (you'll see a hint as to why i'd be using azure point-to-site in this screenshot): azure vpns don't appear to support this. if you want to keep your servers behind a private network in azure and use continuous deployment to get your code into production this makes it hard to deploy without a human being around. not really the best case scenario – especially when you remind yourself that automated builds aim to do away with human error altogether. what the azure point-to-site looks like out of the box when you first go to setup a point-to-site vpn into your azure virtual network microsoft points you at a page that walks you through creating a client certificate on your local machine to use as authentication. they then get you to download a package for setting up the azure vpn ras dialler on your local machine. this is accessed from within the azure "networks" page for your virtual network. you install this package and then whenever connecting you're greeted with a connection screen that you might of seen in a previous life. and by seen i don't mean that windows azure virtual networks have been around for ages. but more that the login screen may look familiar. this is because this login screen is a microsoft " connection manager " login screen and has been around for a while. example from technet (note extremely dated bitmap awesomeness): connection manager is used to pre-package vpn and dial up connections for easy-install distribution in a large organisation. this also means we can reconstruct the underlying vpn connection and use it as a normal vpn – claiming back our cli super powers. digging through the details so what we really want to know is: what is this mystical vpn technology the people at microsoft have bestowed upon us? here's how i started getting more information about the implementation: connecting once successfully then disconnect. open it up again to connect and click on properties then clicking on view log you'll then be greeted by something that looks like this: ****************************************************************** operating system : windows nt 6.2 dialler version : 7.2.9200.16384 connection name : my azure virtual network all users/single user : single user start date/time : 24/11/2013, 7:50:31 ****************************************************************** module name, time, log id, log item name, other info for connection type, 0=dial-up, 1=vpn, 2=vpn over dial-up ****************************************************************** [cmdial32] 7:50:31 03 pre-init event callingprocess = c:\windows\system32\cmmon32.exe [cmdial32] 7:50:39 04 pre-connect event connectiontype = 1 [cmdial32] 7:50:39 06 pre-tunnel event username = myclientsslcertificate domain = dunsetting = [obfuscated azure gateway id] tunnel devicename = tunneladdress = [obfuscated azure gateway id].cloudapp.net [cmdial32] 7:50:44 07 connect event [cmdial32] 7:50:44 08 custom action dll actiontype = connect actions description = to update your routing table actionpath = c:\users\doug\appdata\roaming\microsoft\network\connections\cm\[obfuscated azure gateway id]\cmroute.dll returnvalue = 0x0 [cmmon32] 7:56:21 23 external disconnect [cmdial32] 7:56:21 13 disconnect event callingprocess = c:\windows\explorer.exe more importantly you'll see this path included in the connection: within this folder is all the magic connection manager odds and ends. apologies for the [obfuscated], simply the path contains information to my azure endpoint. within this folder you'll see a bunch of files: most importantly there is a pbk file – a personal phonebook. this is what stores the connect settings for the vpn as is a commonly distributed way of sending out connection settings in the enterprise. if you run this on its own you'll actually be able to connect to the vpn directly (without your network routes being updated). this phonebook is where we can steal our settings from to recreate a command line driven connection. setting it up open up the properties of your azure point-to-site vpn phonebook above, and copy the connection address. it will look like this: azuregateway-[guid].cloudapp.net open network sharing centre , and create a new connection. then select connect to a workplace . select that you'll "use my internet connection". then enter your azure point-to-site vpn address and then give your new connection a name. remember this name for later then click create to save your vpn. now open the connection properties for your newly created vpn. this is where we'll use the settings in your azure diallers config to setup your connection. i'll save you the hassle of showing you me copying the settings from one connection to another and instead i'll just focus on what you need to set them to. flick over to the options tab and then click ppp settings . click the 2 missing options enable software compression and negotiate multi-link for single-link connections . set the type of vpn to secure socket tunnelling protocol (sstp), turn on eap and select microsoft: smart card of other certificate as the authentication type. then click on properties . select "use a certificate on this computer", un-tick "connect to these servers", and then select the certificate that uses your azure endpoint uri as its certificate name and then save out. then flick over to the network tab. open tcp/ipv4 then advanced then untick use default gateway on remote network . this setting stops internet traffic going over the vpn while you're connected so you can still surf reddit while managing your azure environment. close the vpn configuration panel. you now have a working vpn connection to azure. when you connect using windows you'll be asked to select the name of the client certificate you'll be authenticating with. you select the certificate you created and uploaded into azure before you setup your connection. when you connect using the command line you don't need to specify your certificate: rasdial "azure vpn" but there's one catch: your local machine's route table doesn't know when to send any traffic to your azure virtual network. the network link is there, but windows doesn't know what to send over your internet link and what to send over the vpn link. you see microsoft did a few things when they packaged your connection manager, and one of these things was to also copy a file called "cmroute.dll" and call this after connection to route your traffic onto your virtual network. this file altered your routing table to route traffic to your virtual network subnets through the vpn connection . we can do the same thing – so lets go about it. what's this about routing... rooting (for the english speakers in the room) my azure virtual network consists of the following network range: 10.0.0.0/8 i also have the following subnets for different machines groups. 10.0.1.0/24 (web servers) 10.0.2.0/24 (application servers) 10.0.3.0/24 (management services) my pptp connections, or point-to-site connections sit on the range: 172.16.0/24 this means that when i connect to the azure vpn i will get an ip address in this range. example: 172.16.0.17 when this happens we need to tell windows to route all traffic going to my 10.0.x.x range ip addresses through the ip address that has been given to us by azure's vpn rras service. you can see your current routing table by entering route print into a command prompt or powershell console. automating the routing additions luckily the windows task scheduler supports event listeners that allow us to watch for vpn connections and run commands off the back of them. take the below powershell script below and save it for arguments sake in c:\scripts\updateroutetableforazurevpn.ps1 ############################################################# # adds ip routes to azure vpn through the point-to-site vpn ############################################################# # define your azure subnets $ips = @("10.0.1.0", "10.0.2.0","10.0.3.0") # point-to-site ip address range # should be the first 4 octets of the ip address '172.16.0.14' == '172.16.0. $azurepptprange = "172.16.0." # find the current new dhcp assigned ip address from azure $azureipaddress = ipconfig | findstr $azurepptprange # if azure hasn't given us one yet, exit and let u know if (!$azureipaddress){ "you do not currently have an ip address in your azure subnet." exit 1 } $azureipaddress = $azureipaddress.split(": ") $azureipaddress = $azureipaddress[$azureipaddress.length-1] $azureipaddress = $azureipaddress.trim() # delete any previous configured routes for these ip ranges foreach($ip in $ips) { $routeexists = route print | findstr $ip if($routeexists) { "deleting route to azure: " + $ip route delete $ip } } # add our new routes to azure virtual network foreach($subnet in $ips) { "adding route to azure: " + $subnet echo "route add $ip mask 255.255.255.0 $azureipaddress" route add $subnet mask 255.255.255.0 $azureipaddress } now execute the following from an elevated command prompt window. this tells windows to add an event listener based task that looks for events to our "azure vpn" connection and if it sees them, it runs our powershell script. schtasks /create /f /tn "vpn connection update" /tr "powershell.exe -noninteractive -command c:\scripts\updateroutetableforazurevpn.ps1" /sc onevent /ec application /mo "*[system[(level=4 or level=0) and (eventid=20225)]] and *[eventdata[data='azure vpn']] " if i then connect to my vpn the above script should execute. after connecting if i check my routing table by entering route print into a console application we have our routes to azure added correctly. we're done! with that we're now able to fully use an azure point-to-site vpn simply from the command line. this means we can use it as part of a build server deployment, or if you're working on it all the time you can simply set it up to connect every time you login to windows . command line usage rasdial "[connection name]" rasdial "[connection name]" /disconnect for my connection named "azure vpn" this command line usage becomes: rasdial "azure vpn" rasdial "azure vpn" /disconnect
November 29, 2013
by Douglas Rathbone
· 10,612 Views
  • Previous
  • ...
  • 302
  • 303
  • 304
  • 305
  • 306
  • 307
  • 308
  • 309
  • 310
  • 311
  • ...
  • Next
  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook
×