Java Collections allows you to add one or more elements with the same key by using the MultiValueMap class, found in the Apache org.apache.commons.collections package (http://commons.apache.org/collections/): package multihashmap.example; import java.util.Iterator; import java.util.List; import java.util.Map; import java.util.Set; import org.apache.commons.collections.map.MultiValueMap; public class MultiHashMapExample { public static void main(String[] args) { List list; MultiValueMap map = new MultiValueMap(); map.put("A", 4); map.put("A", 6); map.put("B", 7); map.put("C", 1); map.put("B", 9); map.put("A", 5); Set entrySet = map.entrySet(); Iterator it = entrySet.iterator(); System.out.println(" Object key Object value"); while (it.hasNext()) { Map.Entry mapEntry = (Map.Entry) it.next(); list = (List) map.get(mapEntry.getKey()); for (int j = 0; j < list.size(); j++) { System.out.println("\t" + mapEntry.getKey() + "\t " + list.get(j)); } } } } Since Java Core does’t come with some solutions for supporting multiple keys, using the org.apache.commons.collections seems to be a proper way to deal with multiple keys. And the output is: From http://e-blog-java.blogspot.com/2012/02/how-to-allow-duplicate-key-in-java.html

Today we continue our HTML5 canvas examples. Today I want to share with you a method of applying a sepia effect to images. This is not a very difficult method, anyone can repeat it. In our demo we can play with different images by adding a sepia effect to them, as well as ‘export’ our result on the image element (). Here are our demo and downloadable package: Live Demo download in package Ok, download the example files and lets start coding ! Step 1. HTML Markup This is the markup of our demo page. Here it is: index.html HTML5 Image Effects - Sepia Back to original tutorial on Script Tutorials Canvas Object Image Object Next imageApply Sepia EffectTo Image Basically – it contains just one canvas object, one image, and three ‘buttons’ (div elements). Step 2. CSS Here are our stylesheets: css/main.css *{ margin:0; padding:0; } body { background-image:url(../images/bg.png); color:#fff; font:14px/1.3 Arial,sans-serif; } header { background-color:#212121; box-shadow: 0 -1px 2px #111111; display:block; height:70px; position:relative; width:100%; z-index:100; } header h2{ font-size:22px; font-weight:normal; left:50%; margin-left:-400px; padding:22px 0; position:absolute; width:540px; } header a.stuts,a.stuts:visited { border:none; text-decoration:none; color:#fcfcfc; font-size:14px; left:50%; line-height:31px; margin:23px 0 0 110px; position:absolute; top:0; } header .stuts span { font-size:22px; font-weight:bold; margin-left:5px; } .container { color: #000000; margin: 20px auto; overflow: hidden; position: relative; width: 1005px; } table { background-color: rgba(255, 255, 255, 0.7); } table td { border: 1px inset #888888; position: relative; text-align: center; } table td p { display: block; padding: 10px 0; } .button { cursor: pointer; height: 20px; padding: 15px 0; position: relative; text-align: center; width: 500px; -moz-user-select: none; -khtml-user-select: none; user-select: none; } Step 3. JS Finally – our Javascript code generating the sepia effect: js/script.js // variables var canvas, ctx; var imgObj; // set of sepia colors var r = [0, 0, 0, 1, 1, 2, 3, 3, 3, 4, 4, 4, 5, 5, 5, 6, 6, 7, 7, 7, 7, 8, 8, 8, 9, 9, 9, 9, 10, 10, 10, 10, 11, 11, 12, 12, 12, 12, 13, 13, 13, 14, 14, 15, 15, 16, 16, 17, 17, 17, 18, 19, 19, 20, 21, 22, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 39, 40, 41, 42, 44, 45, 47, 48, 49, 52, 54, 55, 57, 59, 60, 62, 65, 67, 69, 70, 72, 74, 77, 79, 81, 83, 86, 88, 90, 92, 94, 97, 99, 101, 103, 107, 109, 111, 112, 116, 118, 120, 124, 126, 127, 129, 133, 135, 136, 140, 142, 143, 145, 149, 150, 152, 155, 157, 159, 162, 163, 165, 167, 170, 171, 173, 176, 177, 178, 180, 183, 184, 185, 188, 189, 190, 192, 194, 195, 196, 198, 200, 201, 202, 203, 204, 206, 207, 208, 209, 211, 212, 213, 214, 215, 216, 218, 219, 219, 220, 221, 222, 223, 224, 225, 226, 227, 227, 228, 229, 229, 230, 231, 232, 232, 233, 234, 234, 235, 236, 236, 237, 238, 238, 239, 239, 240, 241, 241, 242, 242, 243, 244, 244, 245, 245, 245, 246, 247, 247, 248, 248, 249, 249, 249, 250, 251, 251, 252, 252, 252, 253, 254, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255], g = [0, 0, 1, 2, 2, 3, 5, 5, 6, 7, 8, 8, 10, 11, 11, 12, 13, 15, 15, 16, 17, 18, 18, 19, 21, 22, 22, 23, 24, 26, 26, 27, 28, 29, 31, 31, 32, 33, 34, 35, 35, 37, 38, 39, 40, 41, 43, 44, 44, 45, 46, 47, 48, 50, 51, 52, 53, 54, 56, 57, 58, 59, 60, 61, 63, 64, 65, 66, 67, 68, 69, 71, 72, 73, 74, 75, 76, 77, 79, 80, 81, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 97, 100, 101, 102, 103, 105, 106, 107, 108, 109, 111, 113, 114, 115, 117, 118, 119, 120, 122, 123, 124, 126, 127, 128, 129, 131, 132, 133, 135, 136, 137, 138, 140, 141, 142, 144, 145, 146, 148, 149, 150, 151, 153, 154, 155, 157, 158, 159, 160, 162, 163, 164, 166, 167, 168, 169, 171, 172, 173, 174, 175, 176, 177, 178, 179, 181, 182, 183, 184, 186, 186, 187, 188, 189, 190, 192, 193, 194, 195, 195, 196, 197, 199, 200, 201, 202, 202, 203, 204, 205, 206, 207, 208, 208, 209, 210, 211, 212, 213, 214, 214, 215, 216, 217, 218, 219, 219, 220, 221, 222, 223, 223, 224, 225, 226, 226, 227, 228, 228, 229, 230, 231, 232, 232, 232, 233, 234, 235, 235, 236, 236, 237, 238, 238, 239, 239, 240, 240, 241, 242, 242, 242, 243, 244, 245, 245, 246, 246, 247, 247, 248, 249, 249, 249, 250, 251, 251, 252, 252, 252, 253, 254, 255], b = [53, 53, 53, 54, 54, 54, 55, 55, 55, 56, 57, 57, 57, 58, 58, 58, 59, 59, 59, 60, 61, 61, 61, 62, 62, 63, 63, 63, 64, 65, 65, 65, 66, 66, 67, 67, 67, 68, 69, 69, 69, 70, 70, 71, 71, 72, 73, 73, 73, 74, 74, 75, 75, 76, 77, 77, 78, 78, 79, 79, 80, 81, 81, 82, 82, 83, 83, 84, 85, 85, 86, 86, 87, 87, 88, 89, 89, 90, 90, 91, 91, 93, 93, 94, 94, 95, 95, 96, 97, 98, 98, 99, 99, 100, 101, 102, 102, 103, 104, 105, 105, 106, 106, 107, 108, 109, 109, 110, 111, 111, 112, 113, 114, 114, 115, 116, 117, 117, 118, 119, 119, 121, 121, 122, 122, 123, 124, 125, 126, 126, 127, 128, 129, 129, 130, 131, 132, 132, 133, 134, 134, 135, 136, 137, 137, 138, 139, 140, 140, 141, 142, 142, 143, 144, 145, 145, 146, 146, 148, 148, 149, 149, 150, 151, 152, 152, 153, 153, 154, 155, 156, 156, 157, 157, 158, 159, 160, 160, 161, 161, 162, 162, 163, 164, 164, 165, 165, 166, 166, 167, 168, 168, 169, 169, 170, 170, 171, 172, 172, 173, 173, 174, 174, 175, 176, 176, 177, 177, 177, 178, 178, 179, 180, 180, 181, 181, 181, 182, 182, 183, 184, 184, 184, 185, 185, 186, 186, 186, 187, 188, 188, 188, 189, 189, 189, 190, 190, 191, 191, 192, 192, 193, 193, 193, 194, 194, 194, 195, 196, 196, 196, 197, 197, 197, 198, 199]; // noise value var noise = 20; function processSepia() { // get current image data var imageData = ctx.getImageData(0,0,canvas.width,canvas.height); for (var i=0; i < imageData.data.length; i+=4) { // change image colors imageData.data[i] = r[imageData.data[i]]; imageData.data[i+1] = g[imageData.data[i+1]]; imageData.data[i+2] = b[imageData.data[i+2]]; // apply noise if (noise > 0) { var noise = Math.round(noise - Math.random() * noise); for(var j=0; j<3; j++){ var iPN = noise + imageData.data[i+j]; imageData.data[i+j] = (iPN > 255) ? 255 : iPN; } } } // put image data back to context ctx.putImageData(imageData, 0, 0); }; $(function () { // create canvas and context objects canvas = document.getElementById('source'); ctx = canvas.getContext('2d'); // load source image imgObj = new Image(); imgObj.onload = function () { // draw image ctx.drawImage(this, 0, 0, this.width, this.height, 0, 0, canvas.width, canvas.height); } imgObj.src="images/pic1.jpg"; // different onclick handlers var iCur = 1; $('#next').click(function () { iCur++; if (iCur > 6) iCur = 1; imgObj.src="images/pic" + iCur + '.jpg'; }); $('#sepia').click(function () { processSepia(); }); $('#toImage').click(function () { $('#img').attr('src', canvas.toDataURL('image/jpeg')); }); }); Main idea: as we know – sepia images have their own, quite predefined colors. So our script will walk through all pixels of the original image, and change the pixel color to the corresponding sepia color. Plus, we will add a little of noise to our image. Live Demo download in package Conclusion I hope that our demo looks fine. Today we have added a new interesting effect to our html5 application – Sepia. I will be glad to see your thanks and comments. Good luck!

this won't be a terribly long post, nor one that is probably informative to a lot of people, but i finally got around to looking at the html5 specification for multiple file uploads (by that i mean allowing a user to pick multiple files from one file form field). i knew it existed i just never got around to actually looking at it. turns out it is incredibly simple. you take a file form field: and you add multiple to it (or multiple="multiple"). and... yeah, that's it. even better, it degrades perfectly. so in chrome it works, and your label switches from... to... i kept waiting for the "but, wait" moment and it never came. in ie where it fails to work, it simply remains a file upload control that works with one file, not many. because it's so simple and fallback is so good, i can't see really bothering to use another solution, like a flash uploader, but you could do a bit of massaging for it. so for example, consider if my form had the following label: multiple file: in ie, the user won't be able to select multiple files. while not the end of the world, it kind of bugs me that the label may mislead the user into trying something they can't do. (and as we know, the user will blame us, not ie.) so what to do? i did some basic research into how to use javascript to detect features. i found a few good examples that basically suggested creating a temporary dom item like so: var el = document.createelement("input"); but here's where i had issues. most of the links i found discussed how to detect new html form types , like range for example. these techniques worked by setting the type on the new element to the type you want to test and then immediately checking to see if the type still has the same value. but what about attributes? turns out dive into html5 has a great article on this. when you have your dom item, you can simply ask if the item exists. here is an example: function supportmultiple() { //do i support input type=file/multiple var el = document.createelement("input"); return ("multiple" in el); } so given that, i modified my mark up a bit. i changed the label in front of my form and hid the word "multiple": multiple file: i then added an init method to my body tag and used the following code: function supportmultiple() { //do i support input type=file/multiple var el = document.createelement("input"); return ("multiple" in el); } function init() { if(supportmultiple()) { document.queryselector("#multiplefilelabel").setattribute("style",""); } } and that worked like a charm. ie say just "file" whereas chrome and firefox had the new hotness. so yeah - that's a lot of writing about a simple little thing, but... dare i say it... i'm really getting jazzed up about html again! here's my entire test template if you want to cut and paste: normal text: multiple file: p.s. so how you actually process the upload depends on your server. my coldfusion readers are probably wondering how it handles this. good news, bad news. the bad news is that coldfusion can't (as far as i and other smarter peoiple know) handle this at all. if you want to do this in coldfusion 9, use the flash based multifile uploader instead. the good news is that coldfusion 10 handles it just file. be sure to use action value of "uploadall" of your cffile tag.

Here’s a brief step by step guide to running more than one instance of Tomcat on a single machine. Step 1: Install the Tomcat files Download Tomcat 5.5, 6.x, or 7.x and unzip it into an appropriate directory. I usually put it in /usr/local, so it ends up in a directory called /usr/local/apache-tomcat-5.5.17, and make a symlink named /usr/local/tomcat to that directory. When later versions come out, I can unzip them and relink, leaving the older version in case things don’t work out (which rarely if ever happens, but I’m paranoid). Step 2: Make directories for each instance For each instance of Tomcat you’re going to run, you’ll need a directory that will be CATALINA_BASE. For example, you might make them /var/tomcat/serverA and /var/tomcat/serverB. In each of these directories you need the following subdirectories: conf, logs, temp, webapps, and work. Put a server.xml and web.xml file in the conf directory. You can get these from the conf directory of the directory where you put the tomcat installation files, although of course you should tighten up your server.xml a bit. The webapps directory is where you’ll put the web applications you want to run on the particular instance of Tomcat. I like to have the Tomcat manager webapp installed on each instance, so I can play with the webapps, and see how many active sessions there are. See my instructions for configuring the Tomcat manager webapp. Step 3: Configure the ports and/or addresses for each instance Tomcat listens to at least two network ports, one for the shutdown command, and one or more for accepting requests. Two instances of Tomcat can’t listen to the same port number on the same IP address, so you will need to edit your server.xml files to change the ports they listen to. The first port to look at is the shutdown port. This is used by the command line shutdown script (actually, but the Java code it runs) to tell the Tomcat instance to shut itself down. This port is defined at the top of the server.xml file for the instance. Make sure each instance uses a different port value. The port value will normally need to be higher than 1024, and shouldn’t conflict with any other network service running on the same system. The shutdown string is the value that is sent to shut the server down. Note that Tomcat won’t accept shutdown commands that come from other machines. Unlike the other ports Tomcat listens to, the shutdown port can’t be configured to listen to its port on a different IP address. It always listens on 127.0.0.1. The other ports Tomcat listens to are configured with the elements, for instance the HTTP or JK listeners. The port attribute configures which port to listen to. Setting this to a different value on the different Tomcat instances on a machine will avoid conflict. Of course, you’ll need to configure whatever connects to that Connector to use the different port. If a web server is used as the front end using mod_jk, mod_proxy, or the like, then this is simple enough - change your web server’s configuration. In some cases you may not want to do this, for instance you may not want to use a port other than 8080 for HTTP connectors. If you want all of your Tomcat intances to use the same port number, you’ll need to use different IP addresses. The server system must be configured with multiple IP addresses, and the address attribute of the element for each Tomcat instance will be set to the appropriate IP address. Step 4: Startup Startup scripts are a whole other topic, but here’s the brief rundown. The main different from running a single Tomcat instance is you need to set CATALINA_BASE to the directory you set up for the particular instance you want to start (or stop). Here’s a typical startup routine: JAVA_HOME=/usr/java JAVA_OPTS="-Xmx800m -Xms800m" CATALINA_HOME=/usr/local/tomcat CATALINA_BASE=/var/tomcat/serverA export JAVA_HOME JAVA_OPTS CATALINA_HOME CATALINA_BASE $CATALINA_HOME/bin/catalina.sh start Source: http://kief.com/running-multiple-tomcat-instances-on-one-server.html

v8js is a new PHP extension able to run JavaScript code inside V8, Google's JavaScript interpreter that powers for example Chrome and NodeJS. This extension is highly alpha - and its API would probably change in the months ahead. Since documentation is lacking, I invite you to repeat the discovering process I follow in this post in case you find some differences in a new version of v8js. Installation V8 must be present on the machine in order to install the extension. On a Debian/Ubuntu system, run the following: sudo apt-get install libv8-dev libv8-dbg libv8-dev will also install libv8 in its latest version. Afterwards, download and compile the extension with: sudo pecl install v8js There are no more requirements for compilation apart from the usual dependencies for PECL packages, like build-essential. After that, add: extension=v8js.so to php.ini or to a section in conf.d. $ php -m | grep v8 v8js will confirm you the extension is loaded. Some introspection PHP's reflection let us take a look at the classes and methods provided by this extension, even without documentation available. It probably hasn't been written yet, due to the unstable API. $ php -r 'var_dump(get_declared_classes());' | grep V8 string(8) "V8Object" string(10) "V8Function" string(4) "V8Js" string(13) "V8JsException" $ php -r '$class = new ReflectionClass("V8Js"); var_dump($class->getMethods());' array(5) { [0]=> &object(ReflectionMethod)#2 (2) { ["name"]=> string(11) "__construct" ["class"]=> string(4) "V8Js" } [1]=> &object(ReflectionMethod)#3 (2) { ["name"]=> string(13) "executeString" ["class"]=> string(4) "V8Js" } [2]=> &object(ReflectionMethod)#4 (2) { ["name"]=> string(19) "getPendingException" ["class"]=> string(4) "V8Js" } [3]=> &object(ReflectionMethod)#5 (2) { ["name"]=> string(17) "registerExtension" ["class"]=> string(4) "V8Js" } [4]=> &object(ReflectionMethod)#6 (2) { ["name"]=> string(13) "getExtensions" ["class"]=> string(4) "V8Js" } } $ php -r '$v8 = new V8Js(); var_dump($v8->executeString("1+2+3"));' int(6) Interesting! We have just executed our first JavaScript expression inside a PHP process. Apparently the last statement's value is returned by the executeString() method, with a rough conversion preserving the type: $ php -r '$v8 = new V8Js(); var_dump($v8->executeString("var obj = {}; obj.field = 1; obj.field++; obj.field;"));' int(2) Syntax or runtime errors are signaled with a V8JsException: $ php -r '$v8 = new V8Js(); var_dump($v8->executeString("var obj = {"));' PHP Fatal error: Uncaught exception 'V8JsException' with message 'V8Js::executeString():1: SyntaxError: Unexpected end of input' in Command line code:1 Stack trace: #0 Command line code(1): V8Js->executeString('var obj = {') #1 {main} thrown in Command line code on line 1 Let's add more difficulty The FizzBuzz kata OO solution is an example of JavaScript code creating an object and executing anonymous functions: it's a good test bench for our integration.Since evaluating a variable as the last line returns it, that is our channel of communication, supporting integers, strings, floats, booleans, arrays (not objects at this time). Meanwhile, input for JavaScript code can be embedded into the executed string. This code will output string(8) "FizzBuzz": executeString($javaScriptCode)); By changing it a bit, we can build a JSON string by backslashing the double quotes ("), and returns it in lieu of an object to communicate to the PHP process a complex result: ... var myFizzBuzz = new FizzBuzz({3 : "Fizz", 5 : "Buzz"}); "{\"15\" : \"" + myFizzBuzz.accept(15) + "\", \"5\" : \"" + myFizzBuzz.accept(5) + "\"}"; '; $v8 = new V8Js(); $result = $v8->executeString($javaScriptCode); var_dump($result); var_dump(json_decode($result)); The output is: string(33) "{"15" : "FizzBuzz", "5" : "Buzz"}" object(stdClass)#2 (2) { ["15"]=> string(8) "FizzBuzz" ["5"]=> string(4) "Buzz" } Wiring Executing an external script would be nice: it would provide better stack traces, with traceable line numbers at the JavaScript level. It would also mean we won't need backslashing for single quotes, simplifying the syntax. We cannot load scripts on the JavaScript side out of the box, due to V8 missing this functionality (Node JS adds this feature) but we can load the code on the PHP Side: // test.js $ php loadingfiles.php string(24) "test.js file was loaded." // loadingfiles.php executeString($javascriptCode); var_dump($result); Execution results in: $ php loadingfiles.php string(24) "test.js file was loaded." However, we still miss the capability of including JavaScript libraries. Conclusion There are many possible use cases for v8js, like sandboxed scripting or the integration of some code which was written for the client side. That would not be the most clean solution, but it's a Turing complete approach, so why not? After all, it's already possible to run PHP on Java or Python and Ruby on .NET. JavaScript is becoming ubiquitous, so why not providing support for it, even in a caged box?

HTML 5 capable browsers support multimedia files as first class citizens - HTML is not a language for describing documents anymore, but more a medium for building applications, which can leverage every bit of built-in functionality to run fast and to not weigh heavily on the shoulders of developers for maintenance. In this article, we'll see all the ways to play audio inside a modern browser via HTML 5. The element is a new tag not only able to load a sound file and play it, but also to feature a series of control buttons for playing, pausing and pausing the reproduction. Of course there are many use cases where you won't need controls, like for the background music of a game. is not more complex than : you specify a mandatory src attribute, and other options to configure the sound player where needed. The attributes shown in this example (and a few others) are: controls: when present, displays a browser-specific player to control the playback. The HTML 5 specification says that with these controls the user should be able to "begin playback, pause playback, seek to an arbitrary position in the content (if the content supports arbitrary seeking), change the volume" and a bunch of other features that I have not seen supported yet. autoplay: when present, tells the browser to begin playing the sound file as soon as it can. loop: when present, specifies the sound file should start again as soon as it finishes. muted: when present, sets the volume at 0 as a default. preload: specifies a policy for loading the sound file. "none" corresponds to no prefetching, which minimizes traffic; "metadata" prefetches only the start of the file, in order to read metadata like a song's duration. "auto" authorizes the browser to prefetch the whole file. Audio object Apparently the combination of HTML 5 and CSS 3 is a Turing complete language: however, it is more practical to code behavior of a web application in JavaScript. That's why eveyrthing you can do with an element can also be accomplished programmatically with JavaScript Audio objects: Thus via JavaScript you can control an audio element by calling a series of methods such as play() and pause(), and adjust properties like *volume*. Audio objects implement an interface named HtmlMediaElement, where you can find the list of all methods, plus modifiable and readonly properties available. For example, doorBell.seekable.length is a range, while doorBell.currentTime is an assignable property to make reproduction jump to a certain point of the file. Formats and support The real problem with HTML 5 Audio right now is its support in different browsers; not the support for and Audio - which is pretty consistent - but for the codecs to play all the different formats an audio file can come in. There are mainly two sides in the issue: Firefox, Chrome and Opera being as usual friendly to open source and open formats, and being the first to support Ogg Vorbis. Ogg is an open container for audio and video files, and Vorbis is a free lossy codec like MP3, but without any patent on it. Internet Explorer and Safari supporting MP3 instead. Let's not even talk about support on mobile browsers, which is close to non-existent with respect to codecs. By the way, for experimenting with stick to Chrome, Firefox and Opera. Fortunately, a series of elements can be embedded in an (or , equivalently) element to provide alternative formats, listed by MIME type: Encoding lots of files in different formats just to please browsers can be a pain, but it's viable in the case of small clips and sound effects.

Originally Authored By Shrikant Sharat If you haven't checked out @kennethreitz's excellent python-requests library yet, I suggest you go do that immediately. Go on, I'll wait for you. Had your candy? That is one of the most beatiful pieces of python code I've read. And its an excellent library with a very humane API. Recently, I have been using this library for a few of my company's internal projects and at a point I needed to serialize and save Session objects for later. That wasn't as straightforward as I first thought it'd be, so I am sharing my experience here. First off, let's make a simple http server which we are going to contact with python-requests. The server should be able to handle cookie based sessions and also have basic auth, as these things are handled by python-requests' Session objects on the client side. I won't discuss the code for the server here, you can get it from bitbucket. Once you have the server running, now for the client, lets do requests! import requests as req URL_ROOT = 'http://localhost:5050' def get_logged_in_session(name): session = req.session(auth=('user', 'pass')) login_response = session.post(URL_ROOT + '/login', data={'name': name}) login_response.raise_for_status() return session def get_whoami(session): response = session.get(URL_ROOT + '/whoami') response.raise_for_status() return response.text I defined two functions here. The get_logged_in_session will create a new session and login to the http server and return that session. Any subsequent requests using this sesssion will be made as if you have logged in. That's what will be tested with the get_whoami function, which will just return the response from /whoami. Lets test this out. Make sure the server.py is running and in another terminal, $ python -i client.py >>> s = get_logged_in_session('sharat') >>> get_whoami(s) u'You are sharat' >>> get_whoami(req.session(auth=('user', 'pass'))) u'You are a guest' Works perfectly. If we pass it the logged in session, it gives us the username and if we pass it a new session, it gives us a guest. Now, lets assume we have two functions, serialize_session and deserialize_session which do exactly what their names say. We can test them out by running a small test.py, as from client import get_logged_in_session, get_whoami from serializer import deserialize_session, serialize_session session = get_logged_in_session('sharat') dsession = deserialize_session(serialize_session(session)) assert get_whoami(session) == get_whoami(dsession) print 'Success' and a dummy serializer.py def serialize_session(session): return session def deserialize_session(session): return session And with that, of course, the test will not fail $ python test.py Success Serializing Now, to implement the functions in serializer.py. A simple one, would be to use pickle. Lets try import pickle as pk def serialize_session(session): return pk.dumps(session) def deserialize_session(data): return pk.loads(data) If you run test.py now, python is going to yell at you. $ python test.py Traceback (most recent call last): File "test.py", line 10, in dsession = deserialize_session(serialize_session(session)) [ ... ] raise TypeError, "can't pickle %s objects" % base.__name__ TypeError: can't pickle lock objects Oh well, it was worth a try I suppose. Update: The Session class can be made to implement the pickle protocol if you want to use pickle. Next plan I had was to pick up attributes and data from a Session object, just enough to recreate this object using the Session constructor, and serialize those attributes as a json. After all, the Session's API is very easy to use, how hard can picking attributes from it be? :) So, I dug in the sessions.py module of python-requests library. And here's what the signature of the constructor for Session objects looks like def __init__(self, headers=None, cookies=None, auth=None, timeout=None, proxies=None, hooks=None, params=None, config=None, verify=True): # ... So, if I pick up just these values, I should be able to recreate the session object. Sweet. import json import requests as req def serialize_session(session): attrs = ['headers', 'cookies', 'auth', 'timeout', 'proxies', 'hooks', 'params', 'config', 'verify'] session_data = {} for attr in attrs: session_data[attr] = getattr(session, attr) return json.dumps(session_data) def deserialize_session(data): return req.session(**json.loads(data)) And let's try this out $ python test.py Traceback (most recent call last): File "test.py", line 12, in assert get_whoami(session) == get_whoami(dsession) [ ... ] [...]requests/models.py", line 447, in send r = self.auth(self) TypeError: 'list' object is not callable Okay, that error message is very wierd. Why would anyone call a list object? Go dig in the models.py module. See this [ ... ] if isinstance(self.auth, tuple) and len(self.auth) == 2: # special-case basic HTTP auth self.auth = HTTPBasicAuth(*self.auth) # Allow auth to make its changes. r = self.auth(self) [ ... ] There. Its not a list that's being called. Not directly at least. The problem here is that the auth we are passing to session() is not a tuple. Duh! While I like it that auth is restricted to be a tuple, I wish there was a better error message for when auth is a list instead of a tuple. I personally wouldn't want it to accept a list for auth though. So, what went wrong? json does not differentiate between a tuple and a list. It only does lists. So, when serializing and deserializing, the auth tuple is turned to a list. Lets turn it back def deserialize_session(data): session_data = json.loads(data) if 'auth' in session_data: session_data['auth'] = tuple(session_data['auth']) return req.session(**session_data) And $ python test.py Traceback (most recent call last): File "test.py", line 12, in assert get_whoami(session) == get_whoami(dsession) [ ... ] File "/usr/lib/python2.7/string.py", line 493, in translate return s.translate(table, deletions) TypeError: translate() takes exactly one argument (2 given) Wait. What? Now we have an error from stdlib? This just keeps getting better and better. If this looks like something that can frustrate you, go get some coffee :) If you look at the complete stack trace, the second file from bottom, File "[...]site-packages/requests/packages/oreos/monkeys.py", line 470, in set if "" != translate(key, idmap, LegalChars): This thing seems to be calling the translate method incorrectly. With a bit of debugging and yelling at my monitor, I found out the problem and for a moment, lost my grip on reality. str.translate takes 2 arguments, but unicode.translate takes only 1. I have no idea why this is done this way but I sure as hell didn't enjoy it. The code in oreos/monkeys.py assumes that the key is a str. However, what json.loads gives you, is unicode stuff. So, we need to convert just the parts in the deserialized dict we get from json.loads which are being used by the oreos/monkeys.py, from unicode to str. Reading a bit more code around the oreos library, it didn't take long to figure out that those were the keys in the cookies dict. Lo def deserialize_session(data): session_data = json.loads(data) if 'auth' in session_data: session_data['auth'] = tuple(session_data['auth']) if 'cookies' in session_data: session_data['cookies'] = dict((key.encode(), val) for key, val in session_data['cookies'].items()) return req.session(**session_data) And so $ python test.py Success ! All the code is on a bitbucket repository. Update: Pickling can also work As Daslch pointed out in his comment on reddit, by implementing the pickle protocol on the Session class, we can get pickling to work. From the documentation, we need two methods, __getstate__ and __setstate__. Adding those methods as follows to sessions.Session class def __getstate__(self): attrs = ['headers', 'cookies', 'auth', 'timeout', 'proxies', 'hooks', 'params', 'config', 'verify'] return dict((attr, getattr(self, attr)) for attr in attrs) def __setstate__(self, state): for name, value in state.items(): setattr(self, name, value) self.poolmanager = PoolManager( num_pools=self.config.get('pool_connections'), maxsize=self.config.get('pool_maxsize') ) with this as the version of serializer.py that uses pickle, we do get a Success. The creation of new poolmanager in __setstate__ is a piece of code copied from __init__ of the same class. This should probably be turned to a method to avoid code repetition. Update 2: Created an issue about this. Update 3: This has been merged and Session objects are pickleable as of version 0.10.3. See requests history. Source: http://sharats.me/serializing-python-requests-session-objects-for-fun-and-profit.html

In this post, I will try to give an overview of the free tools available for developers to move databases from SQL Server to SQL Server Compact and vice versa. I will also show how you can do this with the SQL Server Compact Toolbox (add-in and standalone editions). Moving databases from SQL Server Compact to SQL Server This can be useful for situations where you already have developed an application that depends on SQL Server Compact, and would like the increased power of SQL Server or would like to use some feature, that is not available on SQL Server Compact. I have an informal comparison of the two products here. Microsoft offers a GUI based tool and a command line tool to do this: WebMatrix and MsDeploy. You can also use the ExportSqlCe command line tool or the SQL Server Compact Toolbox to do this. To use the ExportSqlCE (or ExportSqlCE40) command line, use a command similar to: ExportSQLCE.exe "Data Source=D:\Northwind.sdf;" Northwind.sql The resulting script file (Northwind.sql) can the be run against a SQL Server database, using for example the SQL Server sqlcmd command line utility: sqlcmd -S mySQLServer –d NorthWindSQL -i C:\Northwind.sql To use the SQL Server Compact Toolbox: Connect the Toolbox to the database file that you want to move to SQL Server: Right click the database connection, and select to script Schema and Data: Optionally, select which tables to script and click OK: Enter the filename for the script, default extension is .sqlce: Click OK to the confirmation message: You can now open the generated script in Management Studio and execute it against a SQL Server database, or run it with sqlcmd as described above. Moving databases from SQL Server to SQL Server Compact Microsoft offers no tools for doing this “downsizing” of a SQL Server database to SQL Server Compact, and of course not all objects in a SQL Server database CAN be downsized, as only tables exists in a SQL Server Compact database, so no stored procedures, views, triggers, users, schema and so on. I have blogged about how this can be done from the command line, and you can also do this with the SQL Server Compact Toolbox (of course): From the root node, select Script Server Data and Schema: Follow a procedure like the one above, but connecting to a SQL Server database instead. The export process will convert the SQL Server data types to a matching SQL Server Compact data type, for example varchar(50) becomes nvarchar(50) and so on. Any unsupported data types will be ignored, this includes for example computed columns and sql_variant. The new date types in SQL Server 2008+, like date, time, datetime2 will be converted to nvarchar based data types, as only datetime is supported in SQL Server Compact. A full list of the SQL Server Compact data types is available here.

One day at work, I was told that we had a feature request for one of my programs. They wanted a “dark mode” for when they used my application at night as the normal colors were kind of glaring. My program is used in laptops in police cars, so I could understand their frustration. I spent some time looking into the matter and got a mostly working script put together which I’m going to share with my readers. Of course, if you’re a long time reader, you probably know I’m talking about a wxPython program. I write almost all my GUIs using wxPython. Anyway, let’s get on with the story! Into the Darkness Getting the widgets to change color in wxPython is quite easy. The only two methods you need are SetBackgroundColour and SetForegroundColour. The only major problem I ran into when I was doing this was getting my ListCtrl / ObjectListView widget to change colors appropriately. You need to loop over each ListItem and change their colors individually. I alternate row colors, so that made things more interesting. The other problem I had was restoring the ListCtrl’s background color. Normally you can set a widget’s background color to wx.NullColour (or wx.NullColor) and it will go back to its default color. However, some widgets don’t work that way and you have to actually specify a color. It should also be noted that some widgets don’t seem to pay any attention to SetBackgroundColour at all. One such widget that I’ve found is the wx.ToggleButton. Now you know what I know, so let’s look at the code I came up with to solve my issue: import wx try: from ObjectListView import ObjectListView except: ObjectListView = False #---------------------------------------------------------------------- def getWidgets(parent): """ Return a list of all the child widgets """ items = [parent] for item in parent.GetChildren(): items.append(item) if hasattr(item, "GetChildren"): for child in item.GetChildren(): items.append(child) return items #---------------------------------------------------------------------- def darkRowFormatter(listctrl, dark=False): """ Toggles the rows in a ListCtrl or ObjectListView widget. Based loosely on the following documentation: http://objectlistview.sourceforge.net/python/recipes.html#recipe-formatter and http://objectlistview.sourceforge.net/python/cellEditing.html """ listItems = [listctrl.GetItem(i) for i in range(listctrl.GetItemCount())] for index, item in enumerate(listItems): if dark: if index % 2: item.SetBackgroundColour("Dark Grey") else: item.SetBackgroundColour("Light Grey") else: if index % 2: item.SetBackgroundColour("Light Blue") else: item.SetBackgroundColour("Yellow") listctrl.SetItem(item) #---------------------------------------------------------------------- def darkMode(self, normalPanelColor): """ Toggles dark mode """ widgets = getWidgets(self) panel = widgets[0] if normalPanelColor == panel.GetBackgroundColour(): dark_mode = True else: dark_mode = False for widget in widgets: if dark_mode: if isinstance(widget, ObjectListView) or isinstance(widget, wx.ListCtrl): darkRowFormatter(widget, dark=True) widget.SetBackgroundColour("Dark Grey") widget.SetForegroundColour("White") else: if isinstance(widget, ObjectListView) or isinstance(widget, wx.ListCtrl): darkRowFormatter(widget) widget.SetBackgroundColour("White") widget.SetForegroundColour("Black") continue widget.SetBackgroundColour(wx.NullColor) widget.SetForegroundColour("Black") self.Refresh() return dark_mode This code is a little convoluted, but it gets the job done. Let’s break it down a bit and see how it works. First off, we try to import ObjectListView, a cool 3rd party widget that wraps wx.ListCtrl and makes it a LOT easier to use. However, it’s not part of wxPython right now, so you need to test for it’s existence. I just set it to False if it doesn’t exist. The GetWidgets function takes a parent parameter, which would usually be a wx.Frame or wx.Panel and goes through all of its children to create a list of widgets, which it then returns to the calling function. The main function is darkMode. It takes two parameters too, the poorly named “self”, which refers to a parent widget, and a default panel color. It calls GetWidgets and then uses a conditional statement to decide if dark mode should be enabled or not. Next it loops over the widgets and changes the colors accordingly. When it’s done, it will refresh the passed in parent and return a bool to let you know if dark mode is on or off. There is one more function called darkRowFormatter that is only for setting the colors of the ListItems in a wx.ListCtrl or an ObjectListView widget. Here we use a list comprehension to create a list of wx.ListItems that we then iterate over, changing their colors. To actually apply the color change, we need to call SetItem and pass it a wx.ListItem object instance. Trying Out Dark Mode So now you’re probably wondering how to actually use the script above. Well, this section will show you how it’s done. Here’s a simple program with a list control in it and a toggle button too! import wx import darkMode ######################################################################## class MyPanel(wx.Panel): """""" #---------------------------------------------------------------------- def __init__(self, parent): """Constructor""" wx.Panel.__init__(self, parent) self.defaultColor = self.GetBackgroundColour() rows = [("Ford", "Taurus", "1996", "Blue"), ("Nissan", "370Z", "2010", "Green"), ("Porche", "911", "2009", "Red") ] self.list_ctrl = wx.ListCtrl(self, style=wx.LC_REPORT) self.list_ctrl.InsertColumn(0, "Make") self.list_ctrl.InsertColumn(1, "Model") self.list_ctrl.InsertColumn(2, "Year") self.list_ctrl.InsertColumn(3, "Color") index = 0 for row in rows: self.list_ctrl.InsertStringItem(index, row[0]) self.list_ctrl.SetStringItem(index, 1, row[1]) self.list_ctrl.SetStringItem(index, 2, row[2]) self.list_ctrl.SetStringItem(index, 3, row[3]) if index % 2: self.list_ctrl.SetItemBackgroundColour(index, "white") else: self.list_ctrl.SetItemBackgroundColour(index, "yellow") index += 1 btn = wx.ToggleButton(self, label="Toggle Dark") btn.Bind(wx.EVT_TOGGLEBUTTON, self.onToggleDark) normalBtn = wx.Button(self, label="Test") sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.list_ctrl, 0, wx.ALL|wx.EXPAND, 5) sizer.Add(btn, 0, wx.ALL, 5) sizer.Add(normalBtn, 0, wx.ALL, 5) self.SetSizer(sizer) #---------------------------------------------------------------------- def onToggleDark(self, event): """""" darkMode.darkMode(self, self.defaultColor) ######################################################################## class MyFrame(wx.Frame): """""" #---------------------------------------------------------------------- def __init__(self): """Constructor""" wx.Frame.__init__(self, None, wx.ID_ANY, "MvP ListCtrl Dark Mode Demo") panel = MyPanel(self) self.Show() #---------------------------------------------------------------------- if __name__ == "__main__": app = wx.App(False) frame = MyFrame() app.MainLoop() If you run the program above, you should see something like this: If you click the ToggleButton, you should see something like this: Notice how the toggle button was unaffected by the SetBackgroundColour method. Also notice that the list control’s column headers don’t change colors either. Unfortunately, wxPython doesn’t expose access to the column headers, so there’s no way to manipulate their color. Anyway, let’s take a moment to see how the dark mode code is used. First we need to import it. In this case, the module is called darkMode. To actually call it, we need to look at the ToggleButton’s event handler: darkMode.darkMode(self, self.defaultColor) As you can see, all we did was call darkMode.darkMode with the panel object (the “self) and a defaultColor that we set at the beginning of the wx.Panel’s init method. That’s all we had to do too. We should probably set it up with a variable to catch the returned value, but for this example we don’t really care. Wrapping Up Now we’re done and you too can create a “dark mode” for your applications. At some point, I’d like to generalize this some more to make into a color changer script where I can pass whatever colors I want to it. What would be really cool is to make it into a mixin. But that’s something for the future. For now, enjoy! Further Reading ObjectListView documentation An ObjectListView tutorial wx.ListCtrl documentation Source Code 2011-11-5-wxPython-dark-mode You can also pull the source from Bitbucket Source: http://www.blog.pythonlibrary.org/2011/11/05/wxpython-creating-a-dark-mode/

domain-driven design background there are a series of domain model patterns that describe objects and objects group built with domain-driven design. aggregates describe cohesive object graph with a single point of entry, called root: the internal objects of the aggregate cannot be persistently references from the outside. the domain classes whose instances are inside aggregates are subdivided into entities and value objects: the former have a lifecycle (like a post or a user), while the latter are just values with methods, equivalent to strings and other domain concepts. a prerequisite of these patterns is the immutability of value objects , which can then be shared between aggregates, just like string instances can be in many languages. value objects such as numbers and colors are modified by calling a method on them that return a new instance: every change to their state should produce a new value object. repositories are collection of aggregates: they model operations such as finding an aggregate or persisting a new one. a great departure of modern ddd from the entity/relationship modelling everyone knows is the duplication of data between aggregates to support new scenarios: it's possible some field or object is repeated in different aggregates. when there is an update to an aggregate, it's not necessarily atomically reflected to the other copies of its data. i'll refer to writing calls for generality, to indicate the command side of the command query sepration, which corresponds to everything that causes a change in state in the domain objects (in opposition to the reading side). events as mail messages thus it has become common to copy data between objects in different aggregates : for example, think of a document and invoice object that share the same start/finish date interval. traditionally this duplication is dealt with by extracting a common object, mapped to a common row in the database, with a name invented on the spot. domain events are an alternative that allows for duplicating these data: they reflect changes happened in a single aggregate, and are sent to other aggregates so that they can update themselves. technically speaking, domain events are plain old $yourlanguage objects, containing the modified data but not related to the orm like the main domain objects. domain events are handy for modelling "when" rules that should always be respected no matter who is writing to an aggregate; moreover, their handling can take place in the same transaction or even in a new one. my skeptic view of events was that it can be unclear which events are communicated between objects. after a while, i accepted that unit tests tell us that; moreover, communicating with events is a further level of abstraction which is unnecessary in simple domains but just a giant observer pattern in others. the underlying idea is that no matter who applies a command or modifies a domain object, we already configured the event handling mechanism so that consistency across aggregates is reached according to our policy defined in the event handlers (which may be immediate consistency, or eventual one. or it may result in sending a mail to a human asking him to review the changes: whatever you want.) the only alternative to propagate changes between aggregates would be to have many collaborators passed to the various repositories, but this solution couples the aggregates with each other in many way, while with events you're forced to define one-way messages. the event generator does not make any assumption about who will listen to the event and if it will be listened to at all: events are a point of decoupling like interface are for object collaboration. and it's not that we call static methods by passing a string. we have a clear contract, a domainevents static class, and we publish interesting events (like createdcar or updatedvoyageplan) as plain old domain object which contain all the information about the update, often even composing the relevant domain object. udi dahan discourages the reference to domain objects, and consider events just special value objects ; indeed as our solution matures we are moving towards simpler objects. this choice may force us to consider just what needs to be inserted in the message instead of a full reference (where and if serialization is used to transmit the event, it's simpler to use a value object in fact). moreover, it avoids possible further accidental writing calls to the domain object originating the event. in the application layer events are published by calling a static domain class: as a result event launchers cannot be decoupled from the event (as in udi dahan's approach). we launch events from the repository after an update has been performed, either by choosing an event class directly (in case of an update or creation) or by collecting the events from a queue on the relevant domain object, usually the root of the aggregate. this was a nice idea from a colleague of mine that let us decouple at least entities from the domainevents static class. for now we do not have the requirement to decouple the handling of events from the transaction , so the application layer (which is over the domain layer) open and commits/aborts a transaction, while reconstituting an aggregate, doing some "writing" work (updating it or executing a command) and saving it. the save triggers the event launch, which may trigger work on other aggregates through the configured handler: in case of an error the whole transaction is aborted, ensuring immediate consistency. so we aren't getting the scaling advantages of deferred handling (we're not interested in that for now), but the simplicity of communicating with events while writing code. dynamic language this a php-specific section: however, domain events are an approach typical of java or .net enterprise applications. we use php classes (or interfaces) for routing the events with instanceof; php is a shared nothing environment, so event configuration is done now on a per-action basis to avoid having to create all the objects handling events on each request. however, we want to move the configuration to the application level , with some lazy-loading: for example, configuring lazy event handlers as methods on factory objects that create the real handler and return it along with the name of the method to call. all communication between aggregates happen in a single process and a single address space (for now), so we don't use a bit of the decoupling properties of events. we map value objects into the relational database either as on the parent entity's table (decomposing their fields onto the entity) or as row of their own table. in any case, we have to ensure immutability via encapsulation and only assignign to $this->anyfield into the constructor. our standard pattern is to define setters as new self($this->field1, ..., $nwfieldvalue, ..., $this->fieldn); where n is a small number of fields. we map all domain object with the hibernate-equivalent doctrine 2. we are investigating how to deal with orphaned value objects, which are not reached anymore by any other entity.

Originally Authored by Dejan Noveski I'm not going to dive into how good or extendible Vim is. If you are reading this article, you probably know that. The thing that makes Vim so good, is the scripting environment behind it called VimL. Using this scripting language, you can write any functionality/plugin you need for Vim. Each plugin you use is written in this language. Here's the best part. You only need very little knowledge of VimL to be able to write plugins, if you know Python (or Ruby). What's a vim plugin anyway A Vim plugin is a .vim script that defines functions, mappings, syntax rules, commands that may, or may not, manipulate the windows, buffers, lines. It is a complete piece of code with some specific functionality. Usually, a plugin consists of several functions mappings command definitions and event hooks. When writing vim plugins with Python, often, everything outside the functions is written in VimL. But those are vim commands and they can be learned fast. In fact, VimL can be learned fast, but using python gives so much flexibility. Think about using urllib/httplib/simplejson for accessing some web service that helps editing in Vim. This is why most of the plugins that work with web services are usually done in VimL+Python. Any prerequisites? You must have vim compiled with +python support. You can check that using the command: vim --version | grep +python Vim package in Ubuntu and it's derivatives comes with +python support. To Work - Vimmit.vim What's better than starting with a simple example? This is a plugin that, when called, will retrieve the homepage of Reddit and will display it in the current buffer. Start by opening "vimmit.vim" file (in vim). Since we are writing python code, its good to check if Vim supports Python: if !has('python') echo "Error: Required vim compiled with +python" finish endif This piece is writen in VimL. It's best if we stick to VimL for things like this, mappings and event hooks. This function will check if Vim has python support or it will end the script with an error message. We continue with the main function Reddit(). This is where we use Python and do the main functionality: " Vim comments start with a double quote. " Function definition is VimL. We can mix VimL and Python in " function definition. function! Reddit() " We start the python code like the next line. python << EOF # the vim module contains everything we need to interface with vim from # python. We need urllib2 for the web service consumer. import vim, urllib2 # we need json for parsing the response import json # we define a timeout that we'll use in the API call. We don't want # users to wait much. TIMEOUT = 20 URL = "http://reddit.com/.json" try: # Get the posts and parse the json response response = urllib2.urlopen(URL, None, TIMEOUT).read() json_response = json.loads(response) posts = json_response.get("data", "").get("children", "") # vim.current.buffer is the current buffer. It's list-like object. # each line is an item in the list. We can loop through them delete # them, alter them etc. # Here we delete all lines in the current buffer del vim.current.buffer[:] # Here we append some lines above. Aesthetics. vim.current.buffer[0] = 80*"-" for post in posts: # In the next few lines, we get the post details post_data = post.get("data", {}) up = post_data.get("ups", 0) down = post_data.get("downs", 0) title = post_data.get("title", "NO TITLE").encode("utf-8") score = post_data.get("score", 0) permalink = post_data.get("permalink").encode("utf-8") url = post_data.get("url").encode("utf-8") comments = post_data.get("num_comments") # And here we append line by line to the buffer. # First the upvotes vim.current.buffer.append("↑ %s"%up) # Then the title and the url vim.current.buffer.append(" %s [%s]"%(title, url,)) # Then the downvotes and number of comments vim.current.buffer.append("↓ %s | comments: %s [%s]"%(down, comments, permalink,)) # And last we append some "-" for visual appeal. vim.current.buffer.append(80*"-") except Exception, e: print e EOF " Here the python code is closed. We can continue writing VimL or python again. endfunction Save the file, source it in vim (:source vimmit.vim) and: :call Reddit() Now, the way we call the function is not so elegant. So we define a command: command! -nargs=0 Reddit call Reddit() We define the command :Reddit to call the function. After adding this, open a new bufer and do :Reddit . Home page will be loaded in the buffer. The -nargs argument states how many arguments the command will take. Function Arguments, Eval and Command Q: How does one access functional arguments? function! SomeName(arg1, arg2, arg3) " Get the first argument by name in VimL let firstarg=a:arg1 " Get the second argument by position in Viml let secondarg=a:1 " Get the arguments in python python << EOF import vim first_argument = vim.eval("a:arg1") #or vim.eval("a:0") second_argument = vim.eval("a:arg2") #or vim.eval("a:1") You can define a function with arbitrary number of arguments by putting "..." instead of argument names. You can access these arguments only by position, and you can mix them with named arguments (arg1, arg2, ...) Q: How can I call Vim commands from Python? vim.command("[vim-command-here]") Q: How to define global variables and access them in VimL and Python? Global vars are prefixed with g:. If you want to define one in your script, best thing to do is check if it exists and if doesn't define it and assign some default value to it: if !exists("g:reddit_apicall_timeout") let g:reddit_apicall_timeout=40 endif You can access it from python using the vim module: TIMEOUT = vim.eval("g:reddit_apicall_timeout") If you want to override this setting, you can write: let g:reddit_apicall_timeout=60 in .vimrc . Additional Notes VimL is pretty easy once you try it. Remember that print works and everything you can do with python, you can do in here. Here you can find the documentation for the vim python module. Vimdoc is the possibly the only resource you will need when writing vim plugins. You can also check this IBM developerWorks article . Now, try to extend "vimmit.vim" so the user is able to choose a subreddit (as a first functional argument). Source: http://brainacle.com/how-to-write-vim-plugins-with-python.html

The @XmlAnyElement annotation enables a property to handle arbitrary XML elements.

Erlang is a language and platform designed by Ericsson, whose scope is supporting distributed, real-time and fault-tolerant computation. It has become famous in the programming world probably as the platform on which CouchDB was developed. Erlang has some peculiarities that make it different from the usual imperative, mainstream languages: it is a dynamic, functional language; even more than Clojure in some aspects. As a result, it forces single assignments and immutability: you cannot change existing variables, only create new ones. All state is kept on the stack in the form of function calls and their parameters. Concurrency and message passing is supported at the language level, as actors (called processes in Erlang). Actor model Every process of the Erlang virtual machine is an actor: actors execute independently and communicate only with one-way messages. Actors can create other actors. When an actor finished its computation, it disappears. Like all paradigms, the actor model is defined by what it takes away from the programmer: no shared memory between processes. No remote procedure call: only messages sent via asynchronous invocations. As a result, no return values since messages go in one direction (although of course this can be simulated with another, separate message.) Note that Erlang processes run inside one or more virtual machines on one or more nodes of a network, so they do not map to OS processes. An echo server Without taking too much consideration for Erlang's syntax, let's see it in action. An echo server is a process sending you back the messages you send to it. This first example will serve to use both to learn a bit of Erlang's look&feel, and to introduce the primitives for creating actors. -module(echo). -export([start/0]). loop() -> receive {Sender, Num} -> Sender ! Num, loop() end. start() -> spawn(fun loop/0). This is contained in a echo.erl file, where we define a module named echo. All identifiers starting with lowercase are atoms, while variables start with an uppercase letter. This means Num is a variable, while num would be a constant symbol that cannot be assigned but only passed around (like Ruby symbols or Clojure keywords). We define two functions loop() and start(), but export only start/0 (the start version with 0 arguments); the complete signature of a function always comprehends its arity (the number of arguments), so start() and start(Variable) would be two different functions. In loop/0, we use the receive primitive to receive one new message from the queue for this process. We define a pattern for the message - we only accept tuples containing two variables. After receival, we send back a message to the process identified by the first half of the tuple, containing the number in its second half. We then restart listening by tail recurring on loop() until a new message is delivered. start/0 is instead a primitive that has to be called from the originating process. It would call spawn/1 to create another process which will execute the loop function; it will then return the control to the calling process. Here's how to use this echo server. The shell will be our originating process, while the process for echo will be created: [giorgio@Desmond:~]$ erl Erlang R14B02 (erts-5.8.3) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] Eshell V5.8.3 (abort with ^G) We compile the echo.erl source file: 1> c(echo). {ok,echo} We call start() from this process. Another process is spawned, but spawn() returns its process id. Since it is the last statement of start(), also start() returns the same value, which we can save in a variable. The value of the id is shown, as all other return values of local function calls: 2> Pid = echo:start(). <0.39.0> Now we send a message to Pid, a tuple containing two values. A tuple is a simple data structure containing a fixed number of values (it is the generalization of a pair or of a triplet): 3> Pid ! {self(), 42}. {<0.32.0>,42} Now the other process will handle the message and send one back to us. When we want to receive it, we can invoke receive with an identity handler, just displaying what we have got back: 4> receive Value -> Value end. 42 Let's complicate this a bit Let's try to encapsulate some state into our server. We code a new module called var, which will represent a number we can add to or display. -module(var). -export([init/0, add/1, get/0]). init() -> Pid = spawn(fun() -> loop(0) end), register(variable, Pid). % we keep track of the process id loop(N) -> receive {add, X} -> loop(N+X); % we accept more than one type of message: the first has the atom add as the first element for identification {Parent, get} -> % in case of get, before restarting the loop we send a message to Parent with the value Parent ! N, loop(N) end. add(X) -> % these primitives can be called from the parent process, like init() variable ! {add, X}. get() -> variable ! {self(), get}, receive Result -> Result end. The server has the same structure, but it accepts more than one type of message, distinguishing between them via pattern matching. Moreover, it performs some operations like Parent ! N before tail recurring. The add/1 and get/0 primitives are meant to be called from the client process, and encapsulate the messages to send to the server process, along with the blocking for receiving results. To manage multiple created processes, we would have to save pids in a list. Conclusion As you know, different approaches to concurrency such as functional programming and the Actor model are likely to play a role in the near future due to the rise of parallel and distributed computing. To delve into this new world, I'm currently test-driving a distributed tuple space (like JavaSpaces) in Erlang: the mix of a functional language and the actor model is a big change for an OO developer to deal with.

This is an experiment for a real-time tracking of a football match using only web technologies. In this case I’ve chosen to track a football match from Futbol Club Barcelona (fcb). All the work is made by the browser and it’s interactive (there are four checkboxes to activate/deactivate each tracker). It is based on tracking the location of colored objects in the frames using the HSV colour space. As a practical use for the tracker, I’ve added a smart volume feature that activates the sound only when there are goal chances (every time the ball is near the area). So you can browse other pages and change to the video one only when chances are happening. It works based on the quantity of green color in the sides of the video. The path tracker shows ‘possible’ passes between fcb players If you want to watch it in action here is the demo So far it only works in chrome and firefox and because of the low resolution of the video (and my limited skills), the tracking is not perfect, but it could be improved for tracking videos from external sources like Youtube, Veetle, etc. Here is the code: function rgbToHsv(r, g, b){ r = r/255, g = g/255, b = b/255; var max = Math.max(r, g, b), min = Math.min(r, g, b); var h, s, v = max; var d = max - min; s = max == 0 ? 0 : d / max; if(max == min){ h = 0; // achromatic }else{ switch(max){ case r: h = (g - b) / d + (g < b ? 6 : 0); break; case g: h = (b - r) / d + 2; break; case b: h = (r - g) / d + 4; break; } h /= 6; } return [h, s, v]; } //Main object var processor = { isFirefox35: function() { // Let a chance to the navigator to deal with it: return true; var ua = navigator.userAgent; // Gecko ? if (ua.indexOf("Gecko") == -1) return false; // Geck >= 1.9.1 ? return !(ua.indexOf("rv:1.9.1") == -1 && ua.indexOf("rv:1.9.2") == -1); }, // Init doLoad: function() { if (!this.isFirefox35()) { document.getElementById("nofirefoxbeta").style.display = "block"; } // Some init this.displayBackground = true; this.video = document.getElementById("video"); this.mirrorVideo = document.getElementById("mirrorVideo"); this.mirrorVideoCtx = this.mirrorVideo.getContext("2d"); var self = this; // If the videos end, play again this.video.addEventListener("ended", function(){ try { clearTimeout(self.timeout); } catch(e){} self.video.play(); // Work around: https://bugzilla.mozilla.org/show_bug.cgi?id=488287 self.videoIsPlaying(); }, true); // Set the events listeners for the main video (update button) this.video.addEventListener("pause", function() { self.updateButtons(false); }, false); this.pageLoaded = true; this.startPlayer(); }, videoIsPlaying: function() { this.updateButtons(true); this.timerCallback(); }, videoIsReady: function() { this.videoLoaded = true; this.startPlayer(); }, startPlayer: function() { if (!this.videoLoaded || !this.pageLoaded) return; document.getElementById("wait").style.display = "none"; document.getElementById("player").style.display = "block"; this.width = this.video.videoWidth; this.height = this.video.videoHeight; this.mirrorVideo.width = this.width; this.mirrorVideo.height = this.height; }, playVideo:function(){ this.video.play(); this.videoIsPlaying(); }, stopVideo: function(){ this.video.pause(); clearTimeout(this.timeout); }, volumeDown:function(){ if(this.video.volume>0) this.video.volume=Math.round((this.video.volume - 0.1)*10)/10; }, volumeUp:function(){ if(this.video.volume<1) this.video.volume=Math.round((this.video.volume + 0.1)*10)/10; }, // Main loop timerCallback: function() { if (this.video.paused || this.video.ended) { return; } this.computeFrame(); var self = this; this.timeout = setTimeout(function () { self.timerCallback(); }, 50); }, // Update the SVG button updateButtons: function(play) { document.getElementById("playButton").setAttribute("play", play); document.getElementById("stopButton").setAttribute("play", play); }, // Handling some patterns (text, drawing) has_green_around: function(frameData, pos) { pos_left = pos+ 24; pos_right = pos - 24; r_left = frameData[pos_left+0]; g_left = frameData[pos_left+1]; b_left = frameData[pos_left+2]; r_right = frameData[pos_right+0]; g_right = frameData[pos_right+1]; b_right = frameData[pos_right+2]; return ((r_left < 125 && g_left > 125 && b_left < 80) && (r_right < 125 && g_right > 125 && b_right < 80)); }, is_team_color: function(frameData, pos){ for (i=pos-4; i<=pos+4; i+=4){ r = frameData[i+0]; g = frameData[i+1]; b = frameData[i+2]; hsv = rgbToHsv(r,g,b); //if(hsv[0] < 0.60|| hsv[1] < 0.35) if((hsv[0] < 0.40 || hsv[1] < 0.25 || hsv[2] < 0) || (hsv[0] > 0.70 || hsv[1] > 1 || hsv[2] > 1)) return false; } return true; }, is_ball_color: function(frameData, pos){ for (i=pos-4; i<=pos+4; i+=4){ r = frameData[i+0]; g = frameData[i+1]; b = frameData[i+2]; hsv = rgbToHsv(r,g,b); if((hsv[0] < 0.23 || hsv[1] < 0.40 || hsv[2] < 0.90) || (hsv[0] > 0.27 || hsv[1] > 0.50 || hsv[2] > 1)) return false; } return true; }, dist: function(x1, y1, x2, y2) { return Math.sqrt((x1 - x2) * (x1 - x2) + (y1 - y2) * (y1 - y2)); }, computeFrame: function() { try { this.mirrorVideoCtx.clearRect(0, 0, this.width, this.height); this.mirrorVideoCtx.drawImage(this.video, 0, 0, this.width, this.height); } catch(e) { return; } var frame = this.mirrorVideoCtx.getImageData(0, 0, 640, 360); this.mirrorVideoCtx.fillStyle = 'rgba(200,200,200,0.5)'; this.mirrorVideoCtx.strokeStyle = 'rgba(200,200,200,0.5)'; this.mirrorVideoCtx.lineWidth = 1; var x, y; var weight = 0; var team_shape = null; var ball_shape = null; var ball_found = false; var ball_tracker = document.getElementById("balltracker").checked; var team_tracker = document.getElementById("teamtracker").checked; var path_tracker = document.getElementById("pathtracker").checked; var smart_volume = document.getElementById("smartvolume").checked; var shapes = []; var x, y; var green_bar_left = 0; var green_bar_rigth =0; var green_bar_down = 0; var frame_length = frame.data.length/4; // We dont' need to compute each pixels var step = 4; for (var i = 0; i < frame_length; i += step) { pos = i*4; x = i % this.width; y = Math.round(i / this.width); if (x == 4) green_bar_left += g; else if (x == this.width-4) { green_bar_rigth += g; if (y == 4) green_bar_down += g; } ball_found = ball_tracker && this.is_ball_color(frame.data, pos); team_found = team_tracker && this.is_team_color(frame.data, pos); if (this.has_green_around(frame.data, pos) && (ball_found || team_found)){ if (ball_found) { ball_shape = {}; ball_shape.x = x; ball_shape.y = y; } if(team_found) { if (!team_shape) { // no shape yet, create the first one team_shape = {}; team_shape.x = x; team_shape.y = y; team_shape.weight = 1; shapes.push(team_shape); } else { var d = this.dist(x, y, team_shape.x, team_shape.y); if (d>25){ team_shape = {}; team_shape.x = x; team_shape.y = y; team_shape.weight = 1; shapes.push(team_shape); } } } } } if (shapes.length>0) { if (path_tracker){ this.mirrorVideoCtx.beginPath(); this.mirrorVideoCtx.moveTo(shapes[0].x, shapes[0].y); } for( var s=0; s0 && Math.abs(green_left_average-green_right_average)>15){ //this.volumeUp(); this.video.volume=1; document.getElementById("soundIcon").setAttribute("mute", false); }else{ this.volumeDown(); //this.video.volume=0; document.getElementById("soundIcon").setAttribute("mute", true); } } return; } } Source: http://lusob.com/2012/02/tracking-a-football-match-with-html5-and-javascript/

In January, Google released LevelDB, "a fast and lightweight key/value database library." In a recent post on the "High Availability MySQL" blog has generated a discussion around the possibility of LevelDB being a storage engine for MySQL due to its performance benefits. The discussion generated some insight LevelDB's comparative performance to MySQL. The LevelDB site provides some insight into these performance benefits. When creating a brand new database, various methods shows a range of speeds from .4 MB/s to 62.7 MB/s in Write performance. In Read performance, LevelDB ranged from 152 MB/s to 232 MB/s. You can see a more detailed explanation of these benchmarks by checking out the LewisDB site here. The "High Availability MySQL" blog also suggests that LevelDB may be a "great fit" for MongoDB because it does not require multi-statement transactions. Commenters pointed out a few more details about LevelDB that may limit its performance: Unfortunately, there is a trade off between number of SST files and query latency variation: the larger single storage file is - the more time will require to compact it -- Vladmir Rodionov A recent GitHub post also compared MySQL and LevelDB. For sequential insert performance, LevelDB was found to get higher throughput/lower latency overall, although MySQL was more stable. For both average latency and update performance, MySQL and LevelDB performed essentially the same. Have you had a chance to use LevelDB? How does it compare to other libraries? Please post your comments below.

In this post I will demonstrate how to use the propOrder property on the @XmlType annotation to control the ordering of XML elements.

PDF files aren't pleasant. The good news is that they're documented (http://www.adobe.com/devnet/pdf/pdf_reference.html). The bad news is that they're rather complex. I found four Python packages for reading PDF files. http://pybrary.net/pyPdf/ - weak http://www.swftools.org/gfx_tutorial.html - depends on binary XPDF http://blog.didierstevens.com/programs/pdf-tools/ - limited http://www.unixuser.org/~euske/python/pdfminer/ - acceptable I elected to work with PDFMiner for two reasons. (1) Pure Python, (2) Reasonably Complete. This is not, however, much of an endorsement. The implementation (while seemingly correct for my purposes) needs a fair amount of cleanup. Here's one example of remarkably poor programming. # Connect the parser and document objects. parser.set_document(doc) doc.set_parser(parser) Only one of these two is needed; the other is trivially handled as part of the setter method. Also, the package seems to rely on a huge volume of isinstance type checking. It's not clear if proper polymorphism is even possible. But some kind of filter that picked elements by type might be nicer than a lot of isinstance checks. Annotation Extraction While shabby, the good news is that PDFMiner seems to reliably extract the annotations on a PDF form. In a couple of hours, I had this example of how to read a PDF document and collect the data filled into the form. from pdfminer.pdfparser import PDFParser, PDFDocument from pdfminer.psparser import PSLiteral from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter, PDFTextExtractionNotAllowed from pdfminer.pdfdevice import PDFDevice from pdfminer.pdftypes import PDFObjRef from pdfminer.layout import LAParams, LTTextBoxHorizontal from pdfminer.converter import PDFPageAggregator from collections import defaultdict, namedtuple TextBlock= namedtuple("TextBlock", ["x", "y", "text"]) class Parser( object ): """Parse the PDF. 1. Get the annotations into the self.fields dictionary. 2. Get the text into a dictionary of text blocks. The key to the dictionary is page number (1-based). The value in the dictionary is a sequence of items in (-y, x) order. That is approximately top-to-bottom, left-to-right. """ def __init__( self ): self.fields = {} self.text= {} def load( self, open_file ): self.fields = {} self.text= {} # Create a PDF parser object associated with the file object. parser = PDFParser(open_file) # Create a PDF document object that stores the document structure. doc = PDFDocument() # Connect the parser and document objects. parser.set_document(doc) doc.set_parser(parser) # Supply the password for initialization. # (If no password is set, give an empty string.) doc.initialize('') # Check if the document allows text extraction. If not, abort. if not doc.is_extractable: raise PDFTextExtractionNotAllowed # Create a PDF resource manager object that stores shared resources. rsrcmgr = PDFResourceManager() # Set parameters for analysis. laparams = LAParams() # Create a PDF page aggregator object. device = PDFPageAggregator(rsrcmgr, laparams=laparams) # Create a PDF interpreter object. interpreter = PDFPageInterpreter(rsrcmgr, device) # Process each page contained in the document. for pgnum, page in enumerate( doc.get_pages() ): interpreter.process_page(page) if page.annots: self._build_annotations( page ) txt= self._get_text( device ) self.text[pgnum+1]= txt def _build_annotations( self, page ): for annot in page.annots.resolve(): if isinstance( annot, PDFObjRef ): annot= annot.resolve() assert annot['Type'].name == "Annot", repr(annot) if annot['Subtype'].name == "Widget": if annot['FT'].name == "Btn": assert annot['T'] not in self.fields self.fields[ annot['T'] ] = annot['V'].name elif annot['FT'].name == "Tx": assert annot['T'] not in self.fields self.fields[ annot['T'] ] = annot['V'] elif annot['FT'].name == "Ch": assert annot['T'] not in self.fields self.fields[ annot['T'] ] = annot['V'] # Alternative choices in annot['Opt'] ) else: raise Exception( "Unknown Widget" ) else: raise Exception( "Unknown Annotation" ) def _get_text( self, device ): text= [] layout = device.get_result() for obj in layout: if isinstance( obj, LTTextBoxHorizontal ): if obj.get_text().strip(): text.append( TextBlock(obj.x0, obj.y1, obj.get_text().strip()) ) text.sort( key=lambda row: (-row.y, row.x) ) return text def is_recognized( self ): """Check for Copyright as well as Revision information on each page.""" bottom_page_1 = self.text[1][-3:] bottom_page_2 = self.text[2][-3:] pg1_rev= "Rev 2011.01.17" == bottom_page_1[2].text pg2_rev= "Rev 2011.01.17" == bottom_page_2[0].text return pg1_rev and pg2_rev This gives us a dictionary of field names and values. Essentially transforming the PDF form into the same kind of data that comes from an HTML POST request. An important part is that we don't want much of the background text. Just enough to confirm the version of the form file itself. The cryptic text.sort( key=lambda row: (-row.y, row.x) ) will sort the text blocks into order from top-to-bottom and left-to-right. For the most part, a page footer will show up last. This is not guaranteed, however. In a multi-column layout, the footer can be so close to the bottom of a column that PDFMiner may put the two text blocks together. The other unfortunate part is the extremely long (and opaque) setup required to get the data from the page. Source: http://slott-softwarearchitect.blogspot.com/2012/02/pdf-reading.html

Since EF was released I have been a fan. However, every once in a while I’ll run into a table design situation that I am not sure how to handle with EF. This week, I needed to setup a self-referencing table in order to store some hierarchical data. A self referencing table is a table where the primary key on the table is also defined as a foreign key. Sounds a little confusing right? Let’s clarify the solution with an example. Let’s say I am building an application where I have a list of categories and subcategories. One of my top level categories is “Programming Languages” and under programming languages I have to subcategories which are “C#” and “Java”. In order to store this data I can use a single table with the following structure: The actual data would look like this: Just to clarify, a top level category will have a null value for the ParentId field. For all child categories the ParentId field is used as to represent its parent’s primary key value. As a programmer you may want to think about the ParentId field as a pointer. To complete the example lets take a look at the SQL used to create the table. CREATE TABLE [dbo].[Categories] ( [CategoryId] [int] IDENTITY(1,1) NOT NULL, [Name] [nvarchar](255) NOT NULL, [ParentId] [int] NULL, PRIMARY KEY CLUSTERED ( [CategoryId] ASC )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY] ) ON [PRIMARY] GO ALTER TABLE [dbo].[Categories] WITH CHECK ADD CONSTRAINT [Category_Parent] FOREIGN KEY([ParentId]) REFERENCES [dbo].[Categories] ([CategoryId]) GO ALTER TABLE [dbo].[Categories] CHECK CONSTRAINT [Category_Parent] GO Upon examining the SQL, you should have noticed that the CategoryId is the primary key on the table and the ParentId field is a foreign key which points back to the CategoryId field. Since we have a key referencing a another key on the same table we can classify this this as a self-referencing table. Now that we fully understand what a self-referencing table is, we can move forward to the Entity Framework code. To get started we first need to create a simple C# object to represent the Category table. Of course, keep in mind that if you are using EF Code first you do not need to create the table or database ahead of time. I only showed the table first because I wanted to better illustrate what a self referencing table is. public class Category { public int CategoryId { get; set; } public string Name { get; set; } public int? ParentId { get; set; } } So far the Category class is very simple. However, we really want to add a few more properties in order to make this class useful. For example, if you are a child category you really want to be able to use dot notation to get the name of the parent category (e.g. subCategory.Parent.Name). Using EF, we will create a virtual property named Parent. By making the property virtual we are letting EF know that when this property is accessed we want to load some data. Based on your configuration settings and the code you use to retrieve your data (whether or not you used DbSet.Include), EF will lazy load or eager load this data. public class Category { public int CategoryId { get; set; } public string Name { get; set; } public int? ParentId { get; set; } public virtual Category Parent { get; set; } } Finally, we also want a property called Children so we can use dot notation to enumerate over the child categories. Once again, here is the modified class: public class Category { public int CategoryId { get; set; } public string Name { get; set; } public int? ParentId { get; set; } public virtual Category Parent { get; set; } public virtual ICollection Children { get; set; } } The final step is to let EF know how these properties are related to one another. This can be done using EF's fluent API. If you are new to EF and are unaware of the fluent API then you may want to read this article first. public class CommodityCategoryMap : EntityTypeConfiguration { public CommodityCategoryMap() { HasKey(x => x.CategoryId); Property(x => x.CategoryId) .HasDatabaseGeneratedOption(DatabaseGeneratedOption.Identity); Property(x => x.Name) .IsRequired() .HasMaxLength(255); HasOptional(x => x.Parent) .WithMany(x => x.Children) .HasForeignKey(x => x.ParentId) .WillCascadeOnDelete(false); } } Hopefully you paid careful attention to the last section of code where we state the a Category has an optional Parent property. In database speak, this simply means that the ParentID field is nullable. The code also states that if a Category object can have zero or many children. In order to specify that a record is a child, we leverage the ParentId field to hold the primary key value of the parent record. As I mentioned earlier, if you are a programmer its easier to think of the ParentId field as a pointer. Finally, I disabled the cascade on delete option. This step is optional and probably based on your own personal preferences. If you enable cascade on delete and you delete a category that has 100 children then you will effectively remove 101 records. For whatever reason this scares me a little bit. Perhaps, my short career as a DBA caused me to not trust people with large volume delete statements. However, you may decide differently depending on your circumstances. Hopefully, this short EF tutorial will help you if you are working through a scenario where you need to capture and manipulate hierarchical data. If you have any questions please leave a comment.

There is an increasing number of libraries which are described as high performance and have benchmarks to back that claim up. Here is a selection that I am aware of.

XML is for dinosaurs, right? Everybody uses JSON these days. So you do, don’t you? But what about things like XSD, XSLT, JAXB, XPath, etc – is it all evil? In this article, I’d like to introduce the StAXON project (APL2) which tries to give you the best from both worlds: JSON outside, but XML inside. One benefit from this is that you can integrate JSON with powerful XML-related technologies for free. StAXON lets you read and write JSON using the Java Streaming API for XML (javax.xml.stream), also known as StAX. More specifically, StAXON provides implementations of the StAX Cursor API (XMLStreamReader and XMLStreamWriter) StAX Event API (XMLEventReader and XMLEventWriter) StAX Factory API (XMLInputFactory and XMLOutputFactory) for JSON. You may know the Jettison project, which also has XMLStreamReader and XMLStreamWriter implementations. However, StAXON aims to provide a more comprehensive and consistent solution and tries to avoid some of the issues users are having with Jettison. Anyway, let’s get started and see what this “anti-aging substance” for XML can do. Setup Add the following dependency to your Maven POM file: de.odysseus.staxon staxon 1.0 or get the latest StAXON JAR from the Downloads page and add it to your classpath. Mapping Convention The purpose of StAXON’s mapping convention is to generate a more compact JSON. It borrows the "$" syntax for text elements from the Badgerfish convention but attempts to avoid needless text-only JSON objects: Element names become object properties: <–> {"alice":null} Attributes go in properties whose name begin with "@": <–> {"alice":{"@charlie":"david"} Text-only elements go to a simple key/value property: bob <–> {"alice":"bob"} Otherwise, text content is mapped to the "$" property: bob <–> {"alice":{"@charlie":"david","$":"bob"} Nested elements go to nested properties: charlie <–> {"alice":{"bob":"charlie"} A default namespace declaration goes in the element’s "@xmlns" property: <–> {"alice":{"@xmlns":"http://foo.com"} A prefixed namespace declaration goes in the element’s "@xmlns:" property: John Doe555-1111 However, with our JSON-based writer, the output is {"customer":{"name":"John Doe","phone":"555-1111"} Reading JSON Create a JSON-based reader: String json = "{\"customer\":{\"name\":\"John Doe\",\"phone\":\"555-1111\"}"; XMLInputFactory factory = new JsonXMLInputFactory(); XMLStreamReader reader = factory.createXMLStreamReader(new StringReader(json)); Read your document: assert reader.getEventType() == XMLStreamConstants.START_DOCUMENT; reader.nextTag(); assert reader.isStartElement() && "customer".equals(reader.getLocalName()); reader.next(); assert reader.isStartElement() && "name".equals(reader.getLocalName()); reader.next(); assert reader.hasText() && "John Doe".equals(reader.getText()); reader.nextTag(); assert reader.isEndElement(); reader.next(); assert reader.isStartElement() && "phone".equals(reader.getLocalName()); reader.next(); assert reader.hasText() && "555-111".equals(reader.getText()); reader.nextTag(); assert reader.isEndElement(); reader.next(); assert reader.isEndElement(); reader.next(); assert reader.getEventType() == XMLStreamConstants.END_DOCUMENT; reader.close(); Factory Configuration The JsonXMLInputFactory and JsonXMLOutputFactory classes can be configured via the standard setProperty(String, Object) API. The factory classes define several constants for properties they support. However, the JsonXMLConfig interface provides a convenient way to hold the configuration of both - input and output - factories: JsonXMLConfig config = new JsonXMLConfigBuilder(). virtualRoot("customer"). prettyPrint(true). build(); XMLInputFactory inputFactory = new JsonXMLInputFactory(config); ... XMLOutputFactory outputFactory = new JsonXMLOutputFactory(config); ... Virtual Roots Set the virtualRoot configuration property to strip the root element from the JSON representation, e.g. { "name" : "John Doe", "phone" : "555-1111" } As XML requires a single root element, but JSON documents often don’t have one, this is an important feature required to read and write existing JSON formats. Mastering Arrays What about JSON arrays? Unfortunately, there’s nothing like this in XML. And to be honest, this causes most of the trouble when writing JSON via an XML API like StAX. Simply omitting the array boundaries would lead to non-unique JSON properties, which is usually not desired. StAXON provides several ways to deal with JSON arrays. At the core is the idea to leverage XML processing instructions to tell the writer about to start an array: the processing instruction maps a sequence of XML elements with the same name to a JSON array. The processing instruction optionally takes the array element tag name (with prefix) as data. There’s no end array hint as StAXON detects the end of an array sequence and closes it automatically. Consider the following JSON document: { "alice" : { "bob" : [ "edgar", "charlie" ], "peter" : null } } In order to get a "bob" array instead of two separate "bob" properties, we need to provide XML events corresponding to edgar charlie I.e., with the cursor API, you would just insert writer.writeProcessingInstruction(JsonXMLStreamConstants.MULTIPLE_PI_TARGET); // to start an array. Initiating Arrays with Element Paths Sometimes it is not desired or even impossible to generate processing instruction to control arrays. This may be the case if the actual writing isn’t done by your code, but some other framework like JAXB or similar, and you only provide a stream writer. Addressing such a scenario, wouldn’t it be nice being able to tell the writer beforehand, which elements should trigger a JSON array? This is where the XMLMultipleStreamWriter and XMLMultipleEventWriter wrappers step in. E.g., to specify a sequence of bob elements below root element alice as a multiple path: writer = new XMLMultipleStreamWriter(writer, true, "/alice/bob"); The boolean parameter specifies whether our paths include the root node (alice) from the paths. That is, we could also use writer = new XMLMultipleStreamWriter(writer, false, "/bob"); To wrap all bob fields into arrays (not just alice children), we can use a relative path, without a leading slash: writer = new XMLMultipleStreamWriter(writer, false, "bob"); Now we (or some legacy code, framework, …) may write our document, and the writer will take care to trigger the bob array for us. Triggering Arrays automatically Finally, if nothing else works for you, you may also let StAXON fully automatically determine array boundaries. Use this only if you cannot provide processing instructions and cannot provide the paths of the elements that should be wrapped into JSON arrays. However, using this method has several drawbacks: The writer basically needs to cache the entire document in memory, eating both space and time. The writer will not be able to produce empty arrays or arrays with a single element. To enable this feature, set the JsonXMLOutputFactory.PROP_AUTO_ARRAY property to true. Triggering Document Arrays StAXON’s writer implementation allows you to wrap a sequence of documents into a JSON array. To do this, write the PI before writing anything else: writer.writeProcessingInstruction(JsonXMLStreamConstants.MULTIPLE_PI_TARGET); writer.writeStartDocument(); // first array component ... writer.writeEndDocument(); writer.writeStartDocument(); // second array component ... writer.writeEndDocument(); ... writer.close(); The writer.close() call is crucial here, as it will close the JSON array. Using JAXB Consider a JAXB-annotated Customer class: @JsonXML(virtualRoot = true, prettyPrint = true, multiplePaths = "phone") @XmlRootElement public class Customer { public String name; public List phone; } The @JsonXML annotation is used to configure the mapping details. In the above example, the customer root element is stripped from the JSON representation, phone elements are wrapped into an array and JSON output is nicely formatted, e.g. { "name" : "John Doe", "phone" : [ "555-1111" ] } Now, the JsonXMLMapper class enables for dead-simple mapping to and from JSON: /* * Create mapper instance. */ JsonXMLMapper mapper = new JsonXMLMapper(Customer.class); /* * Read customer. */ InputStream input = getClass().getResourceAsStream("input.json"); Customer customer = mapper.readObject(input); input.close(); /* * Write back to console */ mapper.writeObject(System.out, customer); Using JAX-RS StAXON provides the staxon-jaxrs module, which enables your RESTful services to serialize/deserialize JAXB-annotated classes to/from JSON. It includes the following JAX-RS @Provider classes: de.odysseus.staxon.json.jaxrs.jaxb.JsonXMLObjectProvider is used to read and write JSON objects de.odysseus.staxon.json.jaxrs.jaxb.JsonXMLArrayProvider is used to read and write JSON arrays In order to select the StAXON message body readers/writers for your resource, a @JsonXML annotation is required. When used with JAX-RS, the @JsonXML annotation can be placed on a model type (@XmlRootElement or @XmlType) to configure its serialization and deserialization a JAX-RS resource method to configure serialization of the result type a parameter of a JAX-RS resource method to configure deserialization of the parameter type If a @JsonXML annotation is present at a model type and a resource method or parameter, the latter will override the model type annotation. If neither is present, StAXON will not handle the resource. You can find a sample project using Jersey with StAXON here. Using XPath XPath is another standard that can be easily adopted for use with JSON. The Java XPath API (javax.xml.xpath) doesn’t let us provide an XMLStreamReader or similar as a source, but requires a Document Object Model (DOM). Therefore, we need to read our JSON into a DOM first to apply expressions against that DOM. This could be done by performing an XSLT identity transformation to a DOMResult. However, StAXON provides the DOMEventConsumer class to translate XML events to DOM nodes, which should be faster and simpler than leveraging XSLT. Once we have a DOM, there’s nothing special with applying XPath expressions. StringReader json = new StringReader("{\"edgar\":\"david\",\"bob\":\"charlie\"}"); /* * Our sample JSON has no root element, so specify "alice" as virtual root */ JsonXMLConfig config = new JsonXMLConfigBuilder().virtualRoot("alice").build(); /* * create event reader */ XMLEventReader reader = new JsonXMLInputFactory(config).createXMLEventReader(json); /* * parse JSON into Document Object Model (DOM) */ Document document = DOMEventConsumer.consume(reader); /* * evaluate an XPath expression */ XPath xpath = XPathFactory.newInstance().newXPath(); System.out.println(xpath.evaluate("//alice/bob", document)); Running the above sample will print charlie to the console. What else? In the end, using an XML API to read and write JSON may still look like a compromise, but it may turn out to be a good choice. The availability of a StAX implementation for JSON acts as a door opener to powerful XML related technologies and easily enables for dual-format (XML and JSON) services. There’s more we can do with StAXON: XSD, XSLT, XQuery, XML-JSON/JSON-XML conversions, to name a few. Please check the Wiki for some of those.