In this blog post I am going to Explain How we create Default values for model Entities.

Many frameworks for storing objects in an off-line or cached manner, use standard Java Serialization to encode the object as bytes which can be turned back into the original object. Java Serialization is generic and can serialise just about any type of object. Why avoid it The main problem with Java Serialization is performance and efficiency. Java serialization is much slower than using in memory stores and tends to significantly expand the size of the object. Java Serialization also creates a lot of garbage. Access performance Say you have a collection and you want to update a field of many elements. Something like for (MutableTypes mt : mts) { mt.setInt(mt.getInt()); } If you update one million elements for about five seconds how long does each one take. Huge Collection update one field, took an average 5.1 ns. List update one field took an average 6.5 ns. List with Externalizable update one field took an average 5,841 ns. List update one field took an average 23,217 ns. If you update ten million elements for five seconds or more Huge Collection update one field, took an average 5.4 ns. List, update one field took an average 6.6 ns. List with readObject/writeObject update one field took an average 6,073 ns. List update one field took an average 22,943 ns. Huge Collection stores information in a column based based, so accessing just one field is much more CPU cache efficient than using JavaBeans. If you were to update every field, it would be about 2x or more times slower. Using an optimised Externalizable is much faster than the default Serializable, however is it 400x slower than using a a JavaBean Memory efficiency The per object memory used is also important as it impacts how many object you can store and the performance of accessing those objects. Collection type Heap used per million Direct memory per million Garbage produced per million Huge Collection 0.09 MB 34 MB 80 bytes List 68 MB none 30 bytes List using Externalizable 140 MB none 5,941 MB List 506 MB none 16,746 MB This test was performed on a collection of one million elements. To test the amount of garbage produced I set the Eden size target than 15 GB so no GC would be performed. -mx22g -XX:NewSize=20g -XX:-UseTLAB -verbosegc Conclusion Having an optimised readExternal/writeExternal can improve performance and the size of a serialised object by 2-4 times, however if you need to maximise performance and efficiency you can gain much more by not using it. From http://vanillajava.blogspot.com/2011/08/avoiding-java-serialization-to-increase.html

Most of you are certainly familiar with this piece of code: List This is the syntax for generic (parameterized) types, which has been available in Java since 2004. And of course, you know what you’re supposed to put between the brackets: a type (class or interface). But have you ever wondered what else we could use there, besides types? For example, how about: List<2> // not legal Java Instead of a type, I’m now using a value of a certain type (an integer) as a type parameter. What could this possibly mean? Well, since I’m making up the syntax (and I’ll use that liberty a bit more in the rest of this post), I might as well make up what it means as well. How about we say that the integer here represents the size of the list? In other words: List<2> l = List(1, 2); // compiles List<2> l = List(1); // does not compile Or we could decide that if the type that we are indexing (Integer) supports the + operator, then that value is the sum of all its elements: List<2> l = List(2); // compiles List<2> l = List(0, 1, 1); // compiles List<2> l = List(0, 1); // does not compile You probably guessed it by now: List<2> is a dependent type. A type parameterized not just by other types but by values of other types. The correct terminology here is that the type is “indexed” by the value (not to be confused with the meaning of “index” in the context of a list). Different values specify different types, and of course, we don’t have to stop there: the value could be of a different type than Integer, e.g String (List<"abc">>, etc… Obviously, you’re not limited to List either, you can imagine having maps, trees or any other parameterized type. Unfortunately, the fun stops there. While the appeal of dependent types looks strong, it’s actually very hard to cross over in the realm of concrete code. Let’s get back to our list: how do we manipulate a dependent list with a fixed size, so that the compiler will flag any attempt to violate this constraint by refusing to type check our program? I used the easy situation in the code snippets above by initializing my lists with constants, but obviously, this is not very useful: you pretty much never use lists that are initialized with constants at declaration time, except maybe in tests. Very often, we create a list, we populate it at runtime with some algorithm and if it’s mutable, we alter it in various ways. First, we need to create our list without putting any elements in it, but doesn’t this violate the constraint? List<2> l = new ArrayList<2>(); // should not compile If we can’t even compile this, then we’ll never be able to use anything but lists initialized with constants, so let’s imagine that the compiler will let us get away with it. How do we populate our list now? As it turns out, we haven’t made much progress: List<2> l = new ArrayList<2>(); // assume this compiles l.add(1); // will not compile l.add(2); // would compile if the previous linecompiled Having a list with one element would be a type error, so the second line in the snippet above should not compile. You could work around this limitation by having add() return a different list of a different type. It will still be a List, but indexed by the value +1 of the original one. And now you have to determine how a List<1> and a List<2> can interact and how compatible they are.. We are at an impasse, and if we want to make progress, we have to configure the compiler to increasingly release the dependent typing condition. For example, maybe it would only apply the type checker on operations that access the content of the list (read) but it wouldn’t perform any checking for operations that alter it (write). Kind of a builder pattern waiver, where the object is allowed to be in an illegal state within well delineated boundaries. Even so, the burden of type checking will become quite big and will require the compiler to not just generate code but actually perform branch analysis to make sure that the number of elements of the list always satisfies the dependency at certain given points. Note also that I glossed over another non-trivial aspect: how can we teach the compiler what the syntax above means? Should it be configurable or part of the language definition? What types are allowed to be indexed? What types are allowed as indices? How do types of the same family but with different indices interoperate? By now, it should be increasingly clear that mutable types can’t realistically be dependent, which confines dependent types to immutable ones. How useful is this, really? Still working on that. If the subject interests you, the best paper I have found so far is Dependent types at work (PDF), which is a short introduction of dependent types through Agda. Agda is probably the first language you should look at since it’s much more recent than Coq (which I was already studying as part of my PhD fifteen years ago, since I was working at the same research institute that created Coq). This paper (and most of the literature on the subject) doesn’t touch any of the topics I described in this short introduction, they are more focused on describing how you can use Agda to formalize the proofs of your type system, which in turn will give you guidance on how your type checker should be implemented. And if you have some interesting practical examples of dependent types, comment away! From http://beust.com/weblog/2011/08/18/a-gentle-introduction-to-dependent-types/

In Eclipse, when you press Ctrl button and click on any Class names, the IDE will take you to the source file for that class. This is the normal behavior for the classes you have in your project. But, in case you want the same behavior for Java’s core classes too, you can have it by attaching the Java source with the Eclipse IDE. Once you attach the source, thereafter when you Ctrl+Click any Java class names (String for example), Eclipse will open the source code of that class. To attach the Java source code with Eclipse, When you install the JDK, you must have selected the option to install the Java source files too. This will copy the src.zip file in the installation directory. In Eclipse, go to Window -> Preferences -> Java -> Installed JREs -> Add and choose the JDK you have in your system. Eclipse will now list the JARs found in the dialog box. There, select the rt.jar and choose Source Attachment. By default, this will be pointing to the correct src.zip. If not, choose the src.zip file which you have in your java installation directory. Similarly, if you have the javadoc downloaded in your machine, you can configure that too in this dialog box. Done! Here after, for all the projects for which you are using the above JDK, you’ll be able to browse the Java’s source code just like how you browse your own code. From http://veerasundar.com/blog/2011/08/attaching-java-source-with-eclipse-ide

JSDoc is the de facto standard for documenting JavaScript code. You need to know at least its syntax (which is also used by many other tools) if you publish code. Alas, documentation is still scarce, but this post can help – it shows you how to run JSDoc and how its syntax works. (The JSDoc wiki [2] is the main source of this post, some examples are borrowed from it.) As a tool, JSDoc takes JavaScript code with special /** */ comments and produces HTML documentation for it. For example: Given the following code. /** @namespace */ var util = { /** * Repeat str several times. * @param {string} str The string to repeat. * @param {number} [times=1] How many times to repeat the string. * @returns {string} */ repeat: function(str, times) { if (times === undefined || times < 1) { times = 1; } return new Array(times+1).join(str); } }; The generated HTML looks as follows in a web browser: This post begins with a quick start, so can try out JSDoc immediately if you are impatient. Afterwards, more background information is given. 1. Quick start For the steps described below, you need to have Java installed. JSDoc includes the shell script jsrun.sh that requires Unix (including OS X and Linux) to run. But it should be easy to translate that script to a Windows batch file. Download the latest jsdoc_toolkit. Unpack the archive into, say, $HOME/jsdoc-toolkit. Make the script $HOME/jsdoc-toolkit/jsrun.sh executable and tell it where to look for the JSDoc binary and the template (which controls what the result looks like). JSDOCDIR="$HOME/local/jsdoc-toolkit" JSDOCTEMPLATEDIR="$JSDOCDIR/templates/jsdoc" Now you can move the script anywhere you want to, e.g. a bin/ directory. For the purpose of this demonstration, we don’t move the script. Use jsrun.sh on a directory of JavaScript files: $HOME/jsdoc-toolkit/jsrun.sh -d=$HOME/doc $HOME/js Input: $HOME/js – a directory of JavaScript files (see below for an example). Output: $HOME/doc – where to write the generated files. If you put the JavaScript code at the beginning of this post into a file $HOME/js/util.js then JSDoc produces the following files: $HOME/doc +-- files.html +-- index.html +-- symbols +-- _global_.html +-- src ¦ +-- util.js.html +-- util.html 2. Introduction: What is JSDoc? It’s a common programming problem: You have written JavaScript code that is to be used by others and need a nice-looking HTML documentation of its API. Java has pioneered this domain via its JavaDoc tool. The quasi-standard in the JavaScript world is JSDoc. As seen above, you document an entity by putting before it a special comment that starts with two asterisks. Templates. In order to output anything, JSDoc always needs a template, a mix of JavaScript and specially marked-up HTML that tells it how to translate the parsed documentation to HTML. JSDoc comes with a built-in template, but there are others that you can download [3]. 2.1. Terminology and conventions of JSDoc Doclet: JSDoc calls its comments doclets which clashes with JavaDoc terminology where such comments are called doc comments and a doclet is similar to a JSDoc template, but written in Java. Variable: The term variable in JSDoc often refers to all documentable entities which include global variables, object properties, and inner members. Instance properties: In JavaScript one typically puts methods into a prototype to share them with all instances of a class, while fields (non-function-valued properties) are put into each instance. JSDoc conflates shared properties and per-instance properties and calls them instance properties. Class properties, static properties: are properties of classes, usually of constructor functions. For example, Object.create is a class property of Object. Inner members: An inner member is data nested inside a function. Most relevant for documentation is instance-private data nested inside a constructor function. function MyClass() { var privateCounter = 0; // an inner member this.inc = function() { // an instance property privateCounter++; }; } 2.2. Syntax Let’s review the comment shown at the beginning: /** * Repeat str several times. * @param {string} str The string to repeat. * @param {number} [times=1] How many times to repeat the string. * @returns {string} */ This demonstrates some of the JSDoc syntax which consists of the following pieces. JSDoc comment: is a JavaScript block comment whose first character is an asterisk. This creates the illusion that the token /** starts such a comment. Tags: Comments are structured by starting lines with tags, keywords that are prefixed with an @ symbol. @param is an example above. HTML: You can freely use HTML in JSDoc comments; for example, to display a word in a monospaced font. Type annotations: You can document the type of a value by putting the type name in braces after the appropriate tags. Variations: Single type: @param {string} name Multiple types: @param {string|number} idCode Arrays of a type: @param {string[]} names Name paths: are used to refer to variables inside JSDoc comments. The syntax of such paths is as follows. myFunction MyConstructor MyConstructor.classProperty MyConstructor#instanceProperty MyConstructor-innerMember 2.3. A word on types There are two kinds of values in JavaScript: primitives and objects [7]. Primitive types: boolean, number, string. The values undefined and null are also considered primitive. Object types: All other types are object types, including arrays and functions. Watch out: the names of primitive types start with a lowercase letter. Each primitive type has a corresponding wrapper type, an object type with a capital name whose instances are objects: The wrapper type of boolean is Boolean. The wrapper type of number is Number. The wrapper type of string is String. Getting the name of the type of a value: Primitive value p: via typeof p. > typeof "" 'string' Compare: an instance of the wrapper type is an object. > typeof new String() 'object' Object value o: via o.constructor.name. Example: > new String().constructor.name 'String' 3. Basic tags Meta-data: @fileOverview: marks a JSDoc comment that describes the whole file. @author: Who has written the variable being documented? @deprecated: indicates that the variable is not supported, any more. It is a good practice to document what to use instead. @example: contains a code example, illustrating how the given entity should be used. /** * @example * var str = "abc"; * console.log(repeat(str, 3)); // abcabcabc */ Linking: @see: points to a related resource. /** * @see MyClass#myInstanceMethod * @see The Example Project. */ {@link ...}: works like @see, but can be used inside other tags. @requires resourceDescription: a resource that the documented entity needs. The resource description is either a name path or a natural language description. Versioning: @version versionNumber: indicates the version of the documented entity. Example: @version 10.3.1 @since versionNumber: indicates since which version the documented entity has been available. Example: @since 10.2.0 4. Documenting functions and methods For functions and methods, one can document parameters, return values, and exceptions they might throw. @param {paramType} paramName description: describes the parameter whose name is paramName. Type and description are optional. Examples: @param str @param str The string to repeat. @param {string} str @param {string} str The string to repeat. Advanced features: Optional parameter: @param {number} [times] The number of times is optional. Optional parameter with default value: @param {number} [times=1] The number of times is optional. @returns {returnType} description: describes the return value of the function or method. Either type or description can be omitted. @throws {exceptionType} description: describes an exception that might be thrown during the execution of the function or method. Either type or description can be omitted. 4.1. Inline type information (“inline doc comments”) There are two ways of providing type information for parameters and return values. First, you can add a type annotation to @param and @returns. /** * @param {String} name * @returns {Object} */ function getPerson(name) { } Second, you can inline the type information: function getPerson(/**String*/ name) /**Object*/ { } 5. Documenting variables and fields Fields are properties with non-function values. Because instance fields are often created inside a constructor, you have to document them there. @type {typeName}: What type does the documented variable have? Example: /** @constructor */ function Car(make, owner) { /** @type {string} */ this.make = make; /** @type {Person} */ this.owner = owner; } @type {Person} This tag can also be used to document the return type of functions, but @returns is preferable in this case. @constant: A flag that indicates that the documented variable has a constant value. @default defaultValue: What is the default value of a variable? Example: /** @constructor */ function Page(title) { /** * @default "Untitled" */ this.title = title || "Untitled"; } @property {propType} propName description: Document an instance property in the class comment. Example: /** * @class * @property {string} name The name of the person. */ function Person(name) { this.name = name; } Without this tag, instance properties are documented as follows. /** * @class */ function Person(name) { /** * The name of the person. * @type {string} */ this.name = name; } Which one of those styles to use is a matter of taste. @property does introduce redundancies, though. 6. Documenting classes JavaScript’s built-in means for defining classes are weak, which is why there are many APIs that help with this task [5]. These APIs differ, often radically, so you have to help JSDoc with figuring out what is going on. There are three basic ways of defining a class: Constructor function: You must mark a constructor function, otherwise it will not be documented as a class. That is, capitalization alone does not mark a function as a constructor. /** * @constructor */ function Person(name) { } @class is a synonym for @constructor, but it also allows you to describe the class – as opposed to the function setting up an instance (see tag documentation below for an example). API call and object literal: You need two markers. First, you need to tell JSDoc that a given variable holds a class. Second, you need to mark an object literal as defining a class. The latter is done via the @lends tag. /** @class */ var Person = makeClass( /** @lends Person# */ { say: function(message) { return "This person says: " + message; } } ); API call and object literal with a constructor method: If one of the methods in an object literal performs the task of a constructor (setting up instance data, [8]), you need to mark it as such so that fields are found by JSDoc. Then the documentation of the class moves to that method. var Person = makeClass( /** @lends Person# */ { /** * A class for managing persons. * @constructs */ initialize: function(name) { this.name = name; }, say: function(message) { return this.name + " says: " + message; } } ); Tags: @constructor: marks a function as a constructor. @class: marks a variable as a class or a function as a constructor. Can be used in a constructor comment to separate the description of the constructor (first line below) from the description of the class (second line below). /** * Creates a new instance of class Person. * @class Represents a person. */ Person = function() { } @constructs: marks a method in an object literal as taking up the duties of a constructor. That is, setting up instance data. In such a case, the class must be documented there. Works in tandem with @lends. @lends namePath: specifies to which class the following object literal contributes. There are two ways of contributing. @lends Person# – the object literal contributes instance properties to Person. @lends Person – the object literal contributes class properties to Person. 6.1. Inheritance, namespacing JavaScript has no simple support for subclassing and no real namespaces [6]. You thus have to help JSDoc see what is going on when you are using work-arounds. @extends namePath: indicates that the documented class is the subclass of another one. Example: /** * @constructor * @extends Person */ function Programmer(name) { Person.call(this, name); ... } // Remaining code for subclassing omitted @augments: a synonym for @extends. @namespace: One can use objects to simulate namespaces in JavaScript. This tag marks such objects. Example: /** @namespace */ var util = { ... }; 7. Meta-tags Meta-tags are tags that are added to several variables. You put them in a comment that starts with “/**#@+”. They are then added to all variables until JSDoc encounters the closing comment “/**#@-*/”. Example: /**#@+ * @private * @memberOf Foo */ function baz() {} function zop() {} function pez() {} /**#@-*/ 8. Rarely used tags @ignore: ignore a variable. Note that variables without /** comments are ignored, anyway. @borrows otherNamePath as this.propName: A variable is just a reference to somewhere else; it is documented there. Example: /** * @constructor * @borrows Remote#transfer as this.send */ function SpecialWriter() { this.send = Remote.prototype.transfer; } @description text: Provide a description, the same as all of the text before the first tag. Manual categorization. Sometimes JSDoc misinterprets what a variable is. Then you can help it via one of the following tags: Tag Mark variable as @function function @field non-function value @public public (especially inner variables) @private private @inner inner and thus also private @static accessible without instantiation Name and membership: @name namePath: override the parsed name and use the given name, instead. @memberOf parentNamePath: the documented variable is a member of the specified object. Not explained here: @event, see [2]. 9. Related reading jsdoc-toolkit - A documentation generator for JavaScript: JSDoc homepage on Google Code. Includes a link to the downloads. JSDoc wiki: the official documentation of JSDoc and source of this post. JSDoc wiki – TemplateGallery: lists available JSDoc templates. JSDoc wiki – TagReference: a handy cheat-sheet for JSDoc tags. Lightweight JavaScript inheritance APIs Modules and namespaces in JavaScript JavaScript values: not everything is an object Prototypes as classes – an introduction to JavaScript inheritance From http://www.2ality.com/2011/08/jsdoc-intro.html

One of the rules of simple design is the necessity to minimize the number of moving parts, like classes and methods, as long as the tests are satisfied and we are not accepting duplication or feeling the lack of an explicit concept. Thus, a rule that aids simple design is to use primitive types unless a field has already some behavior attached: we don't create a class for the user's name or the user's password; we just use some strings. As we make progress, however, we must be able to revise our decisions via refactoring: if a field gains some logic, this behavior shouldn't be modelled by methods in the containing class, but by a new object. The code in this new class can be reused, while the containing object will change from case to case and you will end up duplicating the same methods. Transforming a scalar value into an object is the essence of the Replace Data Value with Object refactoring. In most of the cases, a Value Object or a Parameter Object come out as a result: while DDD pursue Value Objects as concepts in the domain layer, this refactoring is more general and can be applied anywhere. For instance, in a project we started introducing Data Transfer Objects to model the data sent by the controller to a Service Layer. Data values in PHP In PHP, all scalar values are by nature data values as they cannot host methods: string, integers, and booleans are proper scalar. arrays are not scalar in the Perl or mathematical sense, but they are still a primitive type. On the borderline, we find some simple objects used as data containers in PHP: ArrayObjects. SplHeap and other SPL data structures. The classes on the borderline may host methods, but the original class is out of reach for modification, and an indirection has to be introduced."Local Extension" Steps Create the new class: it should contain as a private field just the value you want to substitute. The methods you immediately need have to be chosen between a constructor, getters, and setters (where needed). Change the field in the containing class. Update the constructor to also create the new object and populate the field, or accept injection (a rarer case). Update the original getter to delegate to the new one. Update the original setter to delegate to the new one (where present) or to create a new object. Run tests at the functional level; the changes should be propagated to the construction phases, while the external usage should not change very much. Example In the initial state, magic arrays are passed around. It's very easy to build an array where a key is missing or is called incorrectly. newPassword(array( 'userId' => 42, 'oldPassword' => 'gismo', 'newPassword' => 'supersecret', 'repeatNewPassword' => 'supersecret' )); $this->markTestIncomplete('This refactoring is about the introduction of an object; it suffices that the test does not explode.'); } } class UserService { public function newPassword($changePasswordData) { /* it's not interesting to do something here */ } } After the introduction of an ArrayObject extension, a little type safety is ensure and we gained a place to put methods at a little cost. newPassword(new ChangePasswordCommand(array( 'userId' => 42, 'oldPassword' => 'gismo', 'newPassword' => 'supersecret', 'repeatNewPassword' => 'supersecret' ))); $this->markTestIncomplete('This refactoring is about the introduction of an object; it suffices that the test does not explode.'); } } class UserService { public function newPassword(ChangePasswordCommand $changePasswordData) { /* it's not interesting to do something here */ } } class ChangePasswordCommand extends ArrayObject { } We add methods to implement logic on this object; in this case, validation logic; in general cases, any kind of code that should not be duplicated by the different clients. For a stricter implementation, wrap an array or another data structure (scalars, SPL objects) instead of extending ArrayObject as you gain immutability and encapsulation (but this kind of objects need little encapsulation.) class ChangePasswordCommand extends ArrayObject { public function __construct($data) { if (!isset($data['userId'])) { throw new Exception('User id is missing.'); } parent::__construct($data); } public function getPassword() { if ($this['newPassword'] != $this['repeatNewPassword']) { throw new Exception('Password do not match.'); } return $this['newPassword']; } } Being this a refactoring however, this is the less invasive kind of introduction of objects you can make as the client code can still use the ArrayAccess interface and treat the object as a scalar array.

About one year ago I wrote a blog post about JavaScriptSerializer and the Serialize and Deserialize methods it supports. Note: This blog post has been in draft for sometime now, so I decided to complete it and publish it. There might be situation when you want to serialize to JSON string only specific properties of a given class. You can do that using JavaScriptSerializer in combination with LINQ. Let’s say we have the following class definition public class Customer { public string Name { get; set; } public string Surname { get; set; } public string Email { get; set; } public int Age { get; set; } public bool Drinker { get; set; } public bool Smoker { get; set; } public bool Single { get; set; } } Next, lets create method that will create sample data for our demo private List GetListOfCustomers() { List customers = new List(); customers.Add(new Customer() { Name = "Hajan", Surname = "Selmani", Age = 25, Drinker = false, Smoker = false, Single = false, Email = "[email protected]" }); customers.Add(new Customer() { Name = "John", Surname = "Doe", Age = 29, Drinker = false, Smoker = true, Single = false, Email = "[email protected]" }); customers.Add(new Customer() { Name = "Mark", Surname = "Moris", Age = 34, Drinker = true, Smoker = true, Single = true, Email = "[email protected]" }); return customers; } So, we have three customers with some property values for each of them. Now, lets serialize some of their properties using JavaScriptSerializer. First, you must put the following directive: using System.Web.Script.Serialization; Next, we create list of customers that will get the returned value from GetListOfCustomers method and we create instance of JavaScriptSerializer class List customers = GetListOfCustomers(); JavaScriptSerializer serializer = new JavaScriptSerializer(); Now, lets say we want to serialize as JSON string and retrieve only the Age property data… We do that with only one simple line of code: //this will serialize only the 'Age' property string jsonString = serializer.Serialize(customers.Select(x => x.Age)); The result will be: Nice! Now, what if we want to serialize multiple properties at once, but not all class properties? string jsonStringMultiple = serializer.Serialize(customers.Select(x => new { x.Name, x.Surname, x.Age })); The result will be: You see, the result is an array of objects with the four properties and their corresponding values we have selected using the LINQ query above. You can see that integer and boolean values are without quotes, which is correct way of serialization. Now, you probably saw a difference somewhere? Namely, in the first example where we have selected only one property, there are only the values of the property (no property name), while in the second example we have the property name and it’s corresponding value… Why is it like that? It’s because in the second query, we use new { … } to specify multiple properties in the select statement. Therefore, the anonymous new { … } creates an object of each found item. So, if you are interested to make some more tests, run the following two lines of code: var customers1 = customers.Select(x => x.Name).ToList(); var customers2 = customers.Select(x=> new { x.Name } ).ToList(); and you will obviously see the difference. If we use the new { } way for single property selection, like in the following example string jsonString2 = serializer.Serialize(customers.Select(x => new { x.Age })); the result will be: The complete demo code used for this blog post: List customers = GetListOfCustomers(); JavaScriptSerializer serializer = new JavaScriptSerializer(); //this will serialize only the 'Age' property string jsonString = serializer.Serialize(customers.Select(x => x.Age )); string jsonStringMultiple = serializer.Serialize(customers.Select(x => new { x.Name, x.Surname, x.Age, x.Drinker })); var customers1 = customers.Select(x => x.Name).ToList(); var customers2 = customers.Select(x=> new { x.Name } ).ToList(); string jsonString2 = serializer.Serialize(customers.Select(x => new { x.Age })); You can download the demo project here.

There are a number of problems with having a large number of records in memory. One way around this is to use direct memory, but this is too low level for most developers. Is there a way to make this more friendly? Limitations of large numbers of objects The overhead per object is between 12 and 16 bytes for 64-bit JVMs. If the object is relatively small, this is significant and could be more than the data itself. The GC pause time increases with the number of objects. Pause times can be around one second per GB of objects. Collections and arrays only support two billion elements Huge collections One way to store more data and still follow object orientated principles is have wrappers for direct ByteBuffers. This can be tedious to write, but very efficient. What would be ideal is to have these wrappers generated automatically. Small JavaBean Example This is an example of JavaBean which would have far more overhead than actual data contained. interface MutableByte { public void setByte(byte b); public byte getByte(); } It is also small enough that I can create billions of these on my machine. This example creates a List with 16 billion elements. final long length = 16_000_000_000L; HugeArrayList hugeList = new HugeArrayBuilder() {{ allocationSize = 4 * 1024 * 1024; capacity = length; }.create(); List list = hugeList; assertEquals(0, list.size()); hugeList.setSize(length); // add a GC to see what the GC times are like. System.gc(); assertEquals(Integer.MAX_VALUE, list.size()); assertEquals(length, hugeList.longSize()); byte b = 0; for (MutableByte mb : list) mb.setByte(b++); b = 0; for (MutableByte mb : list) { byte b2 = mb.getByte(); byte expected = b++; if (b2 != expected) assertEquals(expected, b2); } From start to finish, the heap memory used is as follows. with -verbosegc 0 sec - 3100 KB used [GC 9671K->1520K(370496K), 0.0020330 secs] [Full GC 1520K->1407K(370496K), 0.0063500 secs] 10 sec - 3885 KB used 20 sec - 4428 KB used 30 sec - 4428 KB used ... deleted ... 1380 sec - 4475 KB used 1390 sec - 4476 KB used 1400 sec - 4476 KB used 1410 sec - 4476 KB used The only GC is one triggered explicitly. Without the System.gc(); no GC logs appear. After 20 sec, the increase in memory used is from logging how much memory was used. Conclusion The library is relatively slow. Each get or set takes about 40 ns which really adds up when there are so many calls to make. I plan to work on it so it is much faster. ;) On the upside, it wouldn't be possible to create 16 billion objects with the memory I have, nor could it be put in an ArrayList, so having it a little slow is still better than not working at all. From http://vanillajava.blogspot.com/2011/08/collection-with-billions-of-entries.html

I'm forever forgetting what the rules are for dependency declarations in MANIFEST.MF and feature.xml for osgi plugins and features. And Googling often results in frustration rather than an answer. So, because today I actually found a concise list of the rules, I thought I'd repost them here, with some minor edits to help clarify. OSGi Plugin Version Ranges Dependencies on bundles and packages have an associated version range which is specified using an interval notation: a square bracket “[” or “]” denotes an inclusive end of the range and a round bracket “(” or “)” denotes an exclusive end of the range. Where one end of the range is to be included and the other excluded, it is permitted to pair a round bracket with a square bracket. The examples below make this clear. If a single version number is used where a version range is required this does not indicate a single version, but the range starting from that version and including all higher versions. There are four common cases: A “strict” version range, such as [1.2.3,1.2.3], which denotes that version and only that version. A “half-open” range, such as [1.2.3,2.0.0), which has an inclusive lower limit and an exclusive upper limit, denoting version 1.2.3 and any version after this, up to, but not including, version 2.0.0. An “unbounded” version range, such as 1.2.3, which denotes version 1.2.3 and all later versions. No version range, which denotes any version will be acceptable. NOT RECOMMENDED. The complete text of the above snippet can be seen here (or here as PDF). Example: Require-Bundle: org.eclipse.core.runtime;bundle-version="[3.4.0,4.0.0)", org.eclipse.core.resources;bundle-version="[3.4.0,4.0.0)", org.eclipse.ui.ide;bundle-version="[3.4.0,4.0.0)", org.eclipse.ui.navigator;bundle-version="3.5.100", com.ibm.icu See also: plugin manifest (plugin.xml) osgi bundle manifest (MANIFEST.MF) In terms of feature manifest (feature.xml) rules, help.eclipse.org has pretty good documentation, but the most important thing to remember - and what I often have to look up - is how to state the matching rules for required upstream features & plugins. Experience says it's always better to state things explicitly so there's no downstream guesswork needed and anyone reading your manifest knows EXACTLY what version(s) are required for or compatible with your feature. Plus, while YOU might be using PDE UI to build, someone else might be using Tycho and Maven, and every tool can interpret missing metadata their own way. When in doubt, spell it out. Valid values and processing are as follows: if version attribute is not specified, the match attribute (if specified) is ignored. perfect - dependent plug-in version must match exactly the specified version. If "patch" is "true", "perfect" is assumed and other values cannot be set. [1.2.3,1.2.3] equivalent - dependent plug-in version must be at least at the version specified, or at a higher service level (major and minor version levels must equal the specified version). [1.2.3,1.3) compatible - dependent plug-in version must be at least at the version specified, or at a higher service level or minor level (major version level must equal the specified version). [1.2.3,2.0) greaterOrEqual - dependent plug-in version must be at least at the version specified, or at a higher service, minor or major level. 1.2.3 The complete text of the above snippet can be seen here. Example: From http://divby0.blogspot.com/2011/07/manifestmf-and-featurexml-versioning.html

My recent slow-down in posting is because I've been trying to write a post explaining memory barriers and their applicability in the Disruptor. The problem is, no matter how much I read and no matter how many times I ask the ever-patient Martin and Mike questions trying to clarify some point, I just don't intuitively grasp the subject. I guess I don't have the deep background knowledge required to fully understand. So, rather than make an idiot of myself trying to explain something I don't really get, I'm going to try and cover, at an abstract / massive-simplification level, what I do understand in the area. Martin has written a post going into memory barriers in some detail, so hopefully I can get away with skimming the subject. Disclaimer: any errors in the explanation are completely my own, and no reflection on the implementation of the Disruptor or on the LMAX guys who actually do know about this stuff. What's the point? My main aim in this series of blog posts is to explain how the Disruptor works and, to a slightly lesser extent, why. In theory I should be able to provide a bridge between the code and the technical paper by talking about it from the point of view of a developer who might want to use it. The paper mentioned memory barriers, and I wanted to understand what they were, and how they apply. What's a Memory Barrier? It's a CPU instruction. Yes, once again, we're thinking about CPU-level stuff in order to get the performance we need (Martin's famous Mechanical Sympathy). Basically it's an instruction to a) ensure the order in which certain operations are executed and b) influence visibility of some data (which might be the result of executing some instruction). Compilers and CPUs can re-order instructions, provided the end result is the same, to try and optimise performance. Inserting a memory barrier tells the CPU and the compiler that what happened before that command needs to stay before that command, and what happens after needs to stay after. All similarities to a trip to Vegas are entirely in your own mind. The other thing a memory barrier does is force an update of the various CPU caches - for example, a write barrier will flush all the data that was written before the barrier out to cache, therefore any other thread that tries to read that data will get the most up-to-date version regardless of which core or which socket it might be executing by. What's this got to do with Java? Now I know what you're thinking - this isn't assembler. It's Java. The magic incantation here is the word volatile (something I felt was never clearly explained in the Java certification). If your field is volatile, the Java Memory Model inserts a write barrier instruction after you write to it, and a read barrier instruction before you read from it. This means if you write to a volatile field, you know that: Any thread accessing that field after the point at which you wrote to it will get the updated value Anything you did before you wrote that field is guaranteed to have happened and any updated data values will also be visible, because the memory barrier flushed all earlier writes to the cache. Example please! So glad you asked. It's about time I started drawing doughnuts again. The RingBuffer cursor is one of these magic volatile thingies, and it's one of the reasons we can get away with implementing the Disruptor without locking. The Producer will obtain the next Entry (or batch of them) and do whatever it needs to do to the entries, updating them with whatever values it wants to place in there. As you know, at the end of all the changes the producer calls the commit method on the ring buffer, which updates the sequence number. This write of the volatile field (cursor) creates a memory barrier which ultimately brings all the caches up to date (or at least invalidates them accordingly). At this point, the consumers can get the updated sequence number (8), and because the memory barrier also guarantees the ordering of the instructions that happened before then, the consumers can be confident that all changes the producer did to to the Entry at position 7 are also available. ...and on the Consumer side? The sequence number on the Consumer is volatile, and read by a number of external objects - other downstream consumers might be tracking this consumer, and the ProducerBarrier/RingBuffer (depending on whether you're looking at older or newer code) tracks it to make sure the the ring doesn't wrap. So, if your downstream consumer (C2) sees that an earlier consumer (C1) reaches number 12, when C2 reads entries up to 12 from the ring buffer it will get all updates C1 made to the entries before it updated its sequence number. Basically everything that happens after C2 gets the updated sequence number (shown in blue above) must occur after everything C1 did to the ring buffer before updating its sequence number (shown in black). Impact on performance Memory barriers, being another CPU-level instruction, don't have the same cost as locks - the kernel isn't interfering and arbitrating between multiple threads. But nothing comes for free. Memory barriers do have a cost - the compiler/CPU cannot re-order instructions, which could potentially lead to not using the CPU as efficiently as possible, and refreshing the caches obviously has a performance impact. So don't think that using volatile instead of locking will get you away scot free. You'll notice that the Disruptor implementation tries to read from and write to the sequence number as infrequently as possible. Every read or write of a volatile field is a relatively costly operation. However, recognising this also plays in quite nicely with batching behaviour - if you know you shouldn't read from or write to the sequences too frequently, it makes sense to grab a whole batch of Entries and process them before updating the sequence number, both on the Producer and Consumer side. Here's an example from BatchConsumer: long nextSequence = sequence + 1; while (running) { try { final long availableSequence = consumerBarrier.waitFor(nextSequence); while (nextSequence <= availableSequence) { entry = consumerBarrier.getEntry(nextSequence); handler.onAvailable(entry); nextSequence++; } handler.onEndOfBatch(); sequence = entry.getSequence(); } ... catch (final Exception ex) { exceptionHandler.handle(ex, entry); sequence = entry.getSequence(); nextSequence = entry.getSequence() + 1; } } (You'll note this is the "old" code and naming conventions, because this is inline with my previous blog posts, I thought it was slightly less confusing than switching straight to the new conventions). In the code above, we use a local variable to increment during our loop over the entries the consumer is processing. This means we read from and write to the volatile sequence field (shown in bold) as infrequently as we can get away with. In Summary Memory barriers are CPU instructions that allow you to make certain assumptions about when data will be visible to other processes. In Java, you implement them with the volatile keyword. Using volatile means you don't necessarily have to add locks willy nilly, and will give you performance improvements over using them. However you need to think a little more carefully about your design, in particular how frequently you use volatile fields, and how frequently you read and write them. PS Given that the New World Order in the Disruptor uses totally different naming conventions now to everything I've blogged about so far, I guess the next post is mapping the old world to the new one. From http://mechanitis.blogspot.com/2011/08/dissecting-disruptor-why-its-so-fast.html

This is what Wikipedia writes about the watermelon: The Watermelon (Citrullus lanatus (Thunb.), family Cucurbitaceae) can be both the fruit and the plant of a vine-like (scrambler and trailer) plant originally from southern Africa, and is one of the most common types of melon. [...] The watermelon fruit, loosely considered a type of melon (although not in the genus Cucumis), has a smooth exterior rind (green, yellow and sometimes white) and a juicy, sweet interior flesh (usually pink, but sometimes orange, yellow, red and sometimes green if not ripe). Watermelon (Citrullus lanatus (Thunb.), family Cucurbitaceae) can be both the fruit and the plant of a vine-like (scrambler and trailer) plant originally from southern Africa, and is one of the most common types of melon. This flowering plant produces a special type of fruit known by botanists as a pepo, a berry which has a thick rind (exocarp) and fleshy center (mesocarp and endocarp); pepos are derived from an inferior ovary, and are characteristic of the Cucurbitaceae. The watermelon fruit, loosely considered a type of melon (although not in the genus Cucumis), has a smooth exterior rind (green, yellow and sometimes white) and a juicy, sweet interior flesh (usually pink, but sometimes orange, yellow, red and sometimes green if not ripe). For my metaphor, I’ll use the one with red flesh but orange and yellow would work too. I think most of us experienced the phenomenon when the project status is red but is getting greener and greener when climbing the management ladder. The project’s core is red but for the management it has a nice green paring, so it looks like a watermelon. This is why I call this phenomenon Watermelon Reporting. But why are we creating such reports and how can we avoid it? Why? The bearer of bad news already had a bad time in the ancient world. If he was lucky, they gave him the chop but in other cases they simply chopped his head of. This hasn’t changed until now but fortunately only in a figurative sense. Some bosses aren’t interested that there are problems with a project in their responsibility because if they know about it, they are in charge. So what do they do to avoid incurring the wrath of their boss ? They tweak the project status just a bit and the melon starts growing. Another reason could be that nobody wants to be in the focus of management, thus they embellish the project status in the hope that everything turns for the better. And as we all know hope is the last to die. In the end the result is the same.. Eventually the overripe melon bursts and there is no rescue for the project anymore. How to avoid it? The answer is easy: Transparency, transparency and transparency. If there is no way to hide the current status the watermelon can’t grow. Fortunately Scrum and other agile frameworks provide tools like burndown charts and backlogs to help the team with their transparency. But there are also tools like dashboards or kanban boards to do this job, but this will be the subject of one of my next blog posts. Conclusion The nuts and bolts of any project are transparency. If the project status is transparent, the watermelons can’t arise. If anybody is able to get the information, it will be difficult to hide something.

This article shows *one* way to mock the JMS infrastructure in a Spring JMS application. This allows us to test our JMS infrastructure without actually having to depend on a physical connection being available. If you are reading this article, chances are that you are also frustrated with failing tests in your continuous integration environment due to a JMS server being (temporarily) unavailable. By mocking the JMS provider, developers are left free to test not only the functionality of their API (unit tests) but also the plumbing of the different components, e.g. in a Spring container. In this article I show how a Spring JMS Hello World application can be fully tested without the need of a physical JMS connection. I would like to stress the fact that the code in this article is by no means meant for production and that the approach shown is just one of many. The infrastructure For this article I use the following infrastructure: Apache ActiveMQ, an open source JMS provider, running on an Ubuntu installation Spring 3 Java 6 MockRunner Eclipse as development environment, running on Windows 7 The Spring configuration It's my belief that using what I define as Spring Configuration Strategy Pattern (SCSP) is the right solution in almost all cases when there is the need for a sound testing infrastructure. I will dedicate an entire article to SCSP, for now this is how it looks: The Spring application context Here follows the content of jemosJms-appContext.xml The only important thing to note here is that there are some services which rely on an existing bean named jmsConnectionFactory but that such bean is not defined in this file. This is key to the SCSP and I will illustrate this in one of my future articles. The Spring application context implementation Here follows the content of jemosJms-appContextImpl.xml which could be seen as an implementation of the Spring application context defined above This Spring context file imports the Spring application context defined above and it is this application context which declared the connection factory. This decoupling of the bean requirement (in the super context) from its actual declaration (Spring application context implementation) represents the cornerstore of SCSP. Mocking the JMS provider - The Spring Test application context and MockRunner Following the same approach I used above, I can now declare a fake connection factory which does not require a physical connection to a JMS provider. Here follows the content of jemosJmsTest-appContext.xml. Please note that this file should reside in the test resources of your project, i.e. it should never make it to production. Here the Spring test application context file imports the Spring application context (not its implementation) and it declares a fake connection factory, thanks to the MockRunner MockQueueConnectionFactory class. A POJO listener The job of handling the message is delegated to a simple POJO, which happens to be declared also as a bean: package uk.co.jemos.experiments; public class HelloWorldHandler { /** The application logger */ private static final org.apache.log4j.Logger LOG = org.apache.log4j.Logger .getLogger(HelloWorldHandler.class); public void handleHelloWorld(String msg) { LOG.info("Received message: " + msg); } } There is nothing glamorous about this class. In real life this should have probably be the implementation of an interface, but here I wanted to keep things simple. A simple JMS message producer Here follows an example of a JMS message producer, which would use the real JMS infrastructure to send messages: package uk.co.jemos.experiments; import org.springframework.context.ApplicationContext; import org.springframework.context.support.ClassPathXmlApplicationContext; import org.springframework.jms.core.JmsTemplate; public class JmsTest { /** The application logger */ private static final org.apache.log4j.Logger LOG = org.apache.log4j.Logger .getLogger(JmsTest.class); /** * @param args */ public static void main(String[] args) { ApplicationContext ctx = new ClassPathXmlApplicationContext( "classpath:jemosJms-appContextImpl.xml"); JmsTemplate jmsTemplate = ctx.getBean(JmsTemplate.class); jmsTemplate.send("jemos.tests", new HelloWorldMessageCreator()); LOG.info("Message sent successfully"); } } The only thing of interest here is that this class retrieves the real JmsTemplate to send a message to the queue. Now if I was to run this class as is, I would obtain the following: 2011-07-31 17:09:46 ClassPathXmlApplicationContext [INFO] Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@19e0ff2f: startup date [Sun Jul 31 17:09:46 BST 2011]; root of context hierarchy 2011-07-31 17:09:46 XmlBeanDefinitionReader [INFO] Loading XML bean definitions from class path resource [jemosJms-appContextImpl.xml] 2011-07-31 17:09:46 XmlBeanDefinitionReader [INFO] Loading XML bean definitions from class path resource [jemosJms-appContext.xml] 2011-07-31 17:09:46 DefaultListableBeanFactory [INFO] Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@3479e304: defining beans [helloWorldConsumer,jmsTemplate,org.springframework.jms.listener.DefaultMessageListenerContainer#0,jmsConnectionFactory]; root of factory hierarchy 2011-07-31 17:09:46 DefaultLifecycleProcessor [INFO] Starting beans in phase 2147483647 2011-07-31 17:09:47 HelloWorldHandler [INFO] Received message: Hello World 2011-07-31 17:09:47 JmsTest [INFO] Message sent successfully Writing the integration test There are various interpretations as to what different types of tests mean and I don't pretend to have the only answer; my interpreation is that an integration test is a functional test which also wires up different components together but which does not interact with real external infrastructure (e.g. a Dao integration test fakes data, a JMS integration test fakes the JMS physical connection, an HTTP integration test fakes the remote Web host, etc). Whereas in my opinion, the main purpose of a unit (aka functional) test is to let the API emerge from the tests, the main goal of an integration test is to test that the plumbing amongst components works as expected so as to avoid surprises in a production environment. Both unit (functional) and integration tests should run very fast (e.g. under 10 minutes) as they constitute what can be considered the "development token". If unit and integration tests are green one should feel pretty confident that 90% of the functionality works as expected; in my projects when both unit and integration tests are green I let developers free to release the token. This does not mean that the other 10% (e.g. the interaction with the real infrastructure) should not be tested, but this can be delegated to system tests which run nightly and don't require the development token. Because unit and integration tests need to run fast, interaction with external infrastructure should be mocked whenever possible. Here follows an integration test for the Hello World handler: package uk.co.jemos.experiments.test.integration; import javax.annotation.Resource; import javax.jms.TextMessage; import junit.framework.Assert; import org.junit.Before; import org.junit.Test; import org.springframework.jms.core.JmsTemplate; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.AbstractJUnit4SpringContextTests; import uk.co.jemos.experiments.HelloWorldHandler; import uk.co.jemos.experiments.HelloWorldMessageCreator; import com.mockrunner.jms.DestinationManager; import com.mockrunner.mock.jms.MockQueue; /** * @author mtedone * */ @ContextConfiguration(locations = { "classpath:jemosJmsTest-appContextImpl.xml" }) public class HelloWorldHandlerIntegrationTest extends AbstractJUnit4SpringContextTests { @Resource private JmsTemplate jmsTemplate; @Resource private DestinationManager mockDestinationManager; @Resource private HelloWorldHandler helloWorldHandler; @Before public void init() { Assert.assertNotNull(jmsTemplate); Assert.assertNotNull(mockDestinationManager); Assert.assertNotNull(helloWorldHandler); } @Test public void helloWorld() throws Exception { MockQueue mockQueue = mockDestinationManager.createQueue("jemos.tests"); jmsTemplate.send(mockQueue, new HelloWorldMessageCreator()); TextMessage message = (TextMessage) jmsTemplate.receive(mockQueue); Assert.assertNotNull("The text message cannot be null!", message.getText()); helloWorldHandler.handleHelloWorld(message.getText()); } } And here follows the output: 2011-07-31 17:17:26 XmlBeanDefinitionReader [INFO] Loading XML bean definitions from class path resource [jemosJmsTest-appContextImpl.xml] 2011-07-31 17:17:26 XmlBeanDefinitionReader [INFO] Loading XML bean definitions from class path resource [jemosJms-appContext.xml] 2011-07-31 17:17:26 GenericApplicationContext [INFO] Refreshing org.springframework.context.support.GenericApplicationContext@f01a1e: startup date [Sun Jul 31 17:17:26 BST 2011]; root of context hierarchy 2011-07-31 17:17:27 DefaultListableBeanFactory [INFO] Pre-instantiating singletons in org.springframework.beans.factory.support. DefaultListableBeanFactory@39478a43: defining beans [helloWorldConsumer,jmsTemplate,org.springframework.jms.listener.DefaultMessageListener Container#0,destinationManager,configurationManager,jmsConnectionFactory,org.springframework.context.annotation.internalConfigurationAnnotation Processor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequired AnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor]; root of factory hierarchy 2011-07-31 17:17:27 DefaultLifecycleProcessor [INFO] Starting beans in phase 2147483647 2011-07-31 17:17:27 HelloWorldHandler [INFO] Received message: Hello World 2011-07-31 17:17:27 GenericApplicationContext [INFO] Closing org.springframework.context.support.GenericApplicationContext@f01a1e: startup date [Sun Jul 31 17:17:26 BST 2011]; root of context hierarchy 2011-07-31 17:17:27 DefaultLifecycleProcessor [INFO] Stopping beans in phase 2147483647 2011-07-31 17:17:32 DefaultMessageListenerContainer [WARN] Setup of JMS message listener invoker failed for destination 'jemos.tests' - trying to recover. Cause: Queue with name jemos.tests not found 2011-07-31 17:17:32 DefaultListableBeanFactory [INFO] Destroying singletons in org.springframework.beans.factory.support. DefaultListableBeanFactory@39478a43: defining beans [helloWorldConsumer,jmsTemplate,org.springframework.jms.listener.DefaultMessageListener Container#0,destinationManager,configurationManager,jmsConnectionFactory,org.springframework.context.annotation.internalConfigurationAnnotationProcessor ,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context. annotation.internalCommonAnnotationProcessor]; root of factory hierarchy In this test, although we simulated a message roundtrip to a JMS queue, the message never left the current JVM and it the whole execution did not depend on a JMS infrastructure being up. This gives us the power to simulate the JMS infrastructure, to test the integration of our business components without having to fear a red from time to time due to JMS infrastructure being down or inaccessible. Please note that in the output there are some warnings because the JMS listener container declared in the jemosJms-appContext.xml does not find a queue named "jemos.test" in the fake connection factory, but this is fine; it's a warning and does not impede the test from running successfully. The Maven configuration Here follows the Maven pom.xml to compile the example: 4.0.0 uk.co.jemos.experiments jmx-experiments 0.0.1-SNAPSHOT Jemos JMS experiments junit junit 4.8.2 test com.mockrunner mockrunner 0.3.1 test log4j log4j 1.2.16 compile org.slf4j slf4j-api 1.6.1 compile org.slf4j slf4j-simple 1.6.1 compile org.apache.activemq activemq-all 5.5.0 compile org.springframework spring-beans 3.0.5.RELEASE org.springframework spring-context 3.0.5.RELEASE org.springframework spring-core 3.0.5.RELEASE org.springframework spring-jms 3.0.5.RELEASE org.springframework spring-test 3.0.5.RELEASE test From http://tedone.typepad.com/blog/2011/07/mocking-spring-jms-with-mockrunner.html

There are a number of articles on whether synchronized or Locks are faster.

i often get asked about ‘rest to soap’ transformation use cases these days. using an soa gateway like securespan to perform this type of transformation at runtime is trivial to setup. with securespan in front of any existing web service (in the dmz for example), you can virtualize a rest version of this same service. using an example, here is a description of the steps to perform this conversion. imagine the geoloc web service for recording geographical locations. it has two methods, one for setting a location and one for getting a location. see below what this would look like in soap. request: 34802398402 response: 52.37706 4.889721 request: 34802398402 52.37706 4.889721 response: ok here is the equivalent rest target that i want to support at the edge. payloads could be xml, but let’s use json to make it more interesting. get /position/34802398402 http 200 ok content-type: text/json { 'latitude' : 52.37706 'longitude' : 4.889721 } post /position/34802398402 content-type: text/json { 'latitude' : 52.37706 'longitude' : 4.889721 } http 200 ok ok now let’s implement this rest version of the service using securespan. i’m assuming that you already have a securespan gateway deployed between the potential rest requesters and the existing soap web service. first, i will create a new service endpoint on the gateway for this service and assign anything that comes at the uri pattern /position/* to this service. i will also allow the http verbs get and post for this service. rest geoloc service properties next, let’s isolate the resource id from the uri and save this as a context variable named ‘trackerid’. we can use a simple regex assertion to accomplish this. also, i will branch on the incoming http verb using an or statement. i am just focusing on get and post for this example but you could add additional logic for other http verbs that you want to support for this rest service. regex for rest service resource identification policy branching for get vs post for get requests, the transformation is very simple, we just declare a message variable using a soap skeleton into which we refer to the trackerid variable. soap request template this soap message is routed to the existing web service and the essential elements are isolated using xpath assertions. processing soap response the rest response is then constructed back using a template response. template json response a similar logic is performed for the post message. see below for the full policy logic. complete policy you’re done for virtualizing the rest service. setting this up with securespan took less than an hour, did not require any change on the existing soap web service and did not require the deployment of an additional component. from there, you would probably enrich the policy to perform some json schema validation , some url and query parameter validation, perhaps some authentication, authorization , etc.

This blog is about my new Spring-CDI modules effort. It's pupose is to make useful CDI patterns like decorators or interceptors available to a Spring application. I do believe that the explicit pattern implementation in CDI is very useful. It makes it obvious and simple to use patterns for not so experienced developers. Therefore I decided to investigate how to make those patterns and the corresponding CDI annotations available for Spring managed beans. Here is the current status of my work. If you're interested and you have some time left, take a look or try out my early version of the Spring-CDI decorator module. The set-up is straight forward. You'll find all you need below. Please notice: The intention of my blog is to share and discuss ideas. If you use any of this in your applications you're acting at your own risk. JSR-299 decorator pattern implementation Features The decorator module provides the following features: - Use JSR-299 @Decorator and @Delegate in Spring managed beans - Support chains of multiple decorators for the same target delegate bean - Allow to qualify decorators to decorate multiple implementations of the same interface with different decorators - Support scoped beans, allow scoped decorators - Integrate with Spring AOP, both dynamic JDK proxies and CGLIB proxies - Allow definition of custom decorator and delegate annotations Download Link The Spring-CDI decorator module is an usual Spring IoC-container extension delivered as JAR archive. You can download the module JAR and put that on the classpath of your Spring application. Compiled Spring-CDI decorator module JAR: download here Sources: download here API-Doc: view here Everything is hosted on a git repository on Github.com. Dependencies org.springframework spring-context ${spring.version} org.springframework org.springframework.aop ${spring.version} javax.enterprise cdi-api 1.0 cglib cglib 2.2.2 Configuration If the Spring-CDI decorator module JAR and its dependencies are on your classpath, all you need to do is: (1) register DecoratorAwareBeanFactoryPostProcessor in your application context (2) define an include-filter to include javax.decorator.Decorator as component annotation in your context:component-scan tag Use Case The following code snippets show how you can use the decorator pattern ones you have configured your Spring application as described above. For more complex scenarios see my unit test cases. Let's assume you have a business interface called: MyService package com.mycompany.springapp.springcdi; public interface MyService { String sayHello(); } This is your implementation of the service. package com.mycompany.springapp.springcdi; import org.springframework.stereotype.Component; @Component public class MyServiceImpl implements MyService { public String sayHello() { return "Hello"; } } You want to do some transaction and security stuff, but you do not want to mess up the business code with it. For security you'd write a decorator that points to the MyService business service. package com.mycompany.springapp.springcdi; import javax.decorator.Decorator; import javax.decorator.Delegate; @Decorator public class MyServiceSecurityDecorator implements MyService { @Delegate private MyService delegate; public String sayHello() { // do some security stuff return delegate.sayHello(); } } To seperate the cross-cutting-concerns you write another decorator for transaction handling that points to the MyService business service. package com.mycompany.springapp.springcdi; import javax.decorator.Decorator; import javax.decorator.Delegate; @Decorator public class MyServiceTransactionDecorator implements MyService{ @Delegate private MyService delegate; public String sayHello() { // do some transaction stuff return delegate.sayHello(); } } Then you can just use standard Spring @Autowired annotation to make that work. The injected bean will be decorated with your new security and transaction decorator. package com.mycompany.springapp.springcdi; import org.junit.Assert; import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; @ContextConfiguration("/test-decorator-context.xml") @RunWith(SpringJUnit4ClassRunner.class) public class DecoratorTestCase { // This injected bean will be a decorated MyServiceImpl @Autowired private MyService service; @Test public void testHelloWorld() { Assert.assertTrue(service.sayHello().equals("Hello")); } } How it works The core is the DecoratorAwareBeanFactoryPostProcessor that scans the registered bean definitions for existing decorators. It gathers meta data and stores that data in the DecoratorMetaDataBean. The DecoratorAwareBeanPostProcessor uses the meta data to wire the decorators into a chain and creates a CGLIB proxy that intercepts method calls to the target delegate bean. It redirects those calls to the decorator chain. The DecoratorAutowireCandidateResolver applies autowiring rules specific to the CDI decorator pattern. It also uses meta data to do that. The two modes The DecoratorAwareBeanFactoryPostProcessor accepts two runtime modes. The 'processor' (default) mode uses DecoratorAwareBeanPostProcessor and the DecoratorChainingStrategy to wire the decorator chain. The 'resolver' mode uses DecoratorAwareAutowireCandidateResolver to implement custom wiring logic based on complex wiring rules implemented in ResolverCDIAutowiringRules. The 'resolver' mode was just another option how one can implement such complex logic. I tried two different options and both work. The 'processor' alternative however implements simpler logic. Therefore it's my prefered mode at the moment. Decorator Meta Data Model The DecoratorAwareBeanFactoryPostProcessor scans bean definitions and stores meta data about the decorators and delegates in the application context. These are the model beans in their hierarchical access order: DecoratorMetaDataBean.java: Top level entry point to the meta-data. Registered and available in the application context. QualifiedDecoratorChain.java: A chain of decorators for the same target delegate bean. DecoratorInfo.java: A decorator bean definition wrapper class. DelegateField.java: Contains the delegate field of the decorator implementation. Strategies The Spring-CDI decorator module is easy to adopt by users through the use of strategy pattern in many places. These are the strategies that allow users to change processing logic if required: DecoratorChainingStrategy.java: Wires the decorators for a specific target delegate bean. DecoratorOrderingStrategy.java: Orders the decorators for a specific target delegate bean. DecoratorResolutionStrategy.java: Scans the bean factory for available decorator beans. DelegateResolutionStrategy.java: Searches the delegate bean for a specific decorator bean. Decorator Autowiring Rules The 'processor' mode and the 'resolver' mode both use a custom AutowireCandidateResolver applied to the current bean factory. The class is called DecoratorAwareAutowireCandidateResolver and it is applied to the bean factory in the DecoratorAwareBeanFactoryPostProcessor. The custom resolver works with different rule sets. In the 'processor' mode it works with a very simple rule set called BeanPostProcessorCDIAutowiringRules. In the 'resolver' mode it uses ResolverCDIAutowiringRules which is far more complex. If these rule sets are not sufficient for your autowiring logic, it's easy to apply additional rule sets by implementing a custom SpringCDIPlugin and adding it to the DecoratorAwareAutowireCandidateResolver. Spring-CDI Plugin System The Spring-CDI decorator module contains two infrastructure interfaces that allow the modularized approach of Spring-CDI project: SpringCDIPlugin and SpringCDIInfrastructure. When I implement additional modules - like the interceptor module - users can decide which modules to use and import into their projects. It's not required to add all Spring-CDI functionality if one only needs decorators. From http://niklasschlimm.blogspot.com/2011/08/jsr-299-cdi-decorators-for-spring-beans.html

A new method is needed, as we are factoring out some lines of code. It would be nice to have it available on a class which already has all the fields to execute it, but unfortunately we cannot modify original code (because it's part of a library, or it's bundled with PHP.) A classic example of Introduce Foreign Method is cause by a missing method on ArrayObject, or SplQueue, or other external classes. The first place to place a new method is the object the code works with, since it's cohesive with the rest of the data. The second best place is the client object, although the method could get duplicated in different clients. A Foreign Method is a method created in the client code to complete the functionality of a source object. How it works The source object becomes a first additional argument of the Foreign Method: Python and some other languages reflects this with the self first argument, which makes refactoring Foreign Methods into normal ones easier. For a PHP- example, consider PHPUnit: I use Foreign Methods in my test cases to wrap $this->getMock() multiple usages. I factor away the adaptation to the Api (use of false arguments or MockBuilder) and leave as arguments the variability of the mocks, like the number of calls or the expected parameters. A Foreign Method is a work-around: if you have the possibility to change the source class, definitely go for that. Another alternative is to introduce a wrapper, when you can modify the lifecycle in order to inject a wrapper in the client code instead of the original object. It's not the case for PHPUnit (you extend the class), but it is for SplQueue and other collection objects; wrapping has also other benefits which will be shown in the next articles. Steps Create the method in the client class. Make the server object the first parameter of the method, if it cannot be passed already through $this (in case it's referenced by a field). Substitute the duplicated code with method calls. Fowler suggests also to comment the method calling it Foreign Method and saying it should be moved onto the server object when it will become possible. Example Initially, we're using an ArrayObject and continuing to ordering it after each addition of an element: addUrl('twitter.com'); $links->add('plus.google.com', 'Google+'); $links->add('facebook.com', 'Facebook'); $expected = "Facebook\n" . "Google+\n" . "twitter.com"; $this->assertEquals($expected, $links->__toString()); } } class LinkGroup { private $links; public function __construct() { $this->links = new ArrayObject(); } public function add($url, $text) { $this->links[$url] = $text; $this->links->asort(); } public function addUrl($url) { $this->links[$url] = $url; $this->links->asort(); } public function __toString() { $links = array(); foreach ($this->links as $url => $text) { $links[] = "$text"; } return implode("\n", $links); } } We introduce a Foreign Method, newLink(): class LinkGroup { private $links; public function __construct() { $this->links = new ArrayObject(); } public function add($url, $text) { $this->newLink($url, $text); } public function addUrl($url) { $this->newLink($url, $url); } public function __toString() { $links = array(); foreach ($this->links as $url => $text) { $links[] = "$text"; } return implode("\n", $links); } private function newLink($url, $text) { $this->links[$url] = $text; $this->links->asort(); } } We also add some comments noting that if more and more Foreign Methods pop out, a radical solution would be needed: /** * Foreign Method of the ArrayObject. Should be moved onto a newly extracted * collaborator which wraps the ArrayObject, or an heap-like data structure * should be used. */ private function newLink($url, $text) { $this->links[$url] = $text; $this->links->asort(); }

Below is a list of some tools that can help you examine your Java source code for potential problems: 1. PMD from http://pmd.sourceforge.net/ License: PMD is licensed under a “BSD-style” license PMD scans Java source code and looks for potential problems like: * Possible bugs – empty try/catch/finally/switch statements * Dead code – unused local variables, parameters and private methods * Suboptimal code – wasteful String/StringBuffer usage * Overcomplicated expressions – unnecessary if statements, for loops that could be while loops * Duplicate code – copied/pasted code means copied/pasted bugs You can download everything from here, and you can get an overview of all the rules at the rulesets index page. PMD is integrated with JDeveloper, Eclipse, JEdit, JBuilder, BlueJ, CodeGuide, NetBeans/Sun Java Studio Enterprise/Creator, IntelliJ IDEA, TextPad, Maven, Ant, Gel, JCreator, and Emacs. 2. FindBug from http://findbugs.sourceforge.net License: L-GPL FindBugs, a program which uses static analysis to look for bugs in Java code. And since this is a project from my alumni university (IEEE – University of Maryland, College Park – Bill Pugh) , I have to definitely add this contribution to this list. 3. Clover from http://www.cenqua.com/clover/ License: Free for Open Source (more like a GPL) Measures statement, method, and branch coverage and has XML, HTML, and GUI reporting. and comprehensive plug-ins for major IDEs. * Improve Test Quality * Increase Testing Productivity * Keep Team on Track Fully integrated plugins for NetBeans, Eclipse , IntelliJ IDEA, JBuilder and JDeveloper. These plugins allow you to measure and inspect coverage results without leaving the IDE. Seamless Integration with projects using Apache Ant and Maven. * Easy integration into legacy build systems with command line interface and API. Fast, accurate, configurable, detailed coverage reporting of Method, Statement, and Branch coverage. Rich reporting in HTML, PDF, XML or a Swing GUI Precise control over the coverage gathering with source-level filtering. Historical charting of code coverage and other metrics. Fully compatible with JUnit 3.x & 4.x, TestNG, JTiger and other testing frameworks. Can also be used with manual, functional or integration testing. 4. Macker from http://innig.net/macker/ License: GPL Macker is a build-time architectural rule checking utility for Java developers. It’s meant to model the architectural ideals programmers always dream up for their projects, and then break — it helps keep code clean and consistent. You can tailor a rules file to suit a specific project’s structure, or write some general “good practice” rules for your code. Macker doesn’t try to shove anybody else’s rules down your throat; it’s flexible, and writing a rules file is part of the development process for each unique project. 5 EMMA from http://emma.sourceforge.net/ License: EMMA is distributed under the terms of Common Public License v1.0 and is thus free for both open-source and commercial development. Reports on class, method, basic block, and line coverage (text, HTML, and XML). EMMA can instrument classes for coverage either offline (before they are loaded) or on the fly (using an instrumenting application classloader). Supported coverage types: class, method, line, basic block. EMMA can detect when a single source code line is covered only partially. Coverage stats are aggregated at method, class, package, and “all classes” levels. Output report types: plain text, HTML, XML. All report types support drill-down, to a user-controlled detail depth. The HTML report supports source code linking. Output reports can highlight items with coverage levels below user-provided thresholds. Coverage data obtained in different instrumentation or test runs can be merged together. EMMA does not require access to the source code and degrades gracefully with decreasing amount of debug information available in the input classes. EMMA can instrument individial .class files or entire .jars (in place, if desired). Efficient coverage subset filtering is possible, too. Makefile and ANT build integration are supported on equal footing. EMMA is quite fast: the runtime overhead of added instrumentation is small (5-20%) and the bytecode instrumentor itself is very fast (mostly limited by file I/O speed). Memory overhead is a few hundred bytes per Java class. EMMA is 100% pure Java, has no external library dependencies, and works in any Java 2 JVM (even 1.2.x). 6. XRadar from http://xradar.sourceforge.net/ License: BSD (me thinks) The XRadar is an open extensible code report tool currently supporting all Java based systems. The batch-processing framework produces HTML/SVG reports of the systems current state and the development over time – all presented in sexy tables and graphs. The XRadar gives measurements on standard software metrics such as package metrics and dependencies, code size and complexity, code duplications, coding violations and code-style violations. 7. Hammurapi from Hammurapi Group License: (if anyone knows the license for this email me Venkatt.Guhesan at Y! dot com) Hammurapi is a tool for execution of automated inspection of Java program code. Following the example of 282 rules of Hammurabi’s code, we are offered over 120 Java classes, the so-called inspectors, which can, at three levels (source code, packages, repository of Java files), state whether the analysed source code contains violations of commonly accepted standards of coding. Relevant Links: http://en.sdjournal.org/products/articleInfo/93 http://wiki.hammurapi.biz/index.php?title=Hammurapi_4_Quick_Start 8. Relief from http://www.workingfrog.org/ License: GPL Relief is a design tool providing a new look on Java projects. Relying on our ability to deal with real objects by examining their shape, size or relative place in space it gives a “physical” view on java packages, types and fields and their relationships, making them easier to handle. 9. Hudson from http://hudson-ci.org/ License: MIT Hudson is a continuous integration (CI) tool written in Java, which runs in a servlet container, such as Apache Tomcat or the GlassFish application server. It supports SCM tools including CVS, Subversion, Git and Clearcase and can execute Apache Ant and Apache Maven based projects, as well as arbitrary shell scripts and Windows batch commands. 10. Cobertura from http://cobertura.sourceforge.net/ License: GNU GPL Cobertura is a free Java tool that calculates the percentage of code accessed by tests. It can be used to identify which parts of your Java program are lacking test coverage. It is based on jcoverage. 11. SonarSource from http://www.sonarsource.org/ (recommended by Vishwanath Krishnamurthi – thanks) License: LGPL Sonar is an open platform to manage code quality. As such, it covers the 7 axes of code quality: Architecture & Design, Duplications, Unit Tests, Complexity, Potential bugs, Coding rules, Comments. From http://mythinkpond.wordpress.com/2011/07/14/java-tools-for-source-code-optimization-and-analysis/

I have seen a number of tests where a JVM has 10K threads. However, what happens if you go beyond this? My recommendation is to consider having more servers once your total reaches 10K. You can get a decent server for $2K and a powerful one for $10K. Creating threads gets slower The time it takes to create a thread increases as you create more thread. For the 32-bit JVM, the stack size appears to limit the number of threads you can create. This may be due to the limited address space. In any case, the memory used by each thread's stack add up. If you have a stack of 128KB and you have 20K threads it will use 2.5 GB of virtual memory. Bitness Stack Size Max threads 32-bit 64K 32,073 32-bit 128K 20,549 32-bit 256K 11,216 64-bit 64K stack too small 64-bit 128K 32,072 64-bit 512K 32,072 Note: in the last case, the thread stacks total 16 GB of virtual memory. Java 6 update 26 32-bit,-XX:ThreadStackSize=64 4,000 threads: Time to create 4,000 threads was 0.522 seconds 8,000 threads: Time to create 4,000 threads was 1.281 seconds 12,000 threads: Time to create 4,000 threads was 1.874 seconds 16,000 threads: Time to create 4,000 threads was 2.725 seconds 20,000 threads: Time to create 4,000 threads was 3.333 seconds 24,000 threads: Time to create 4,000 threads was 4.151 seconds 28,000 threads: Time to create 4,000 threads was 5.293 seconds 32,000 threads: Time to create 4,000 threads was 6.636 seconds After creating 32,073 threads, java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:640) at com.google.code.java.core.threads.MaxThreadsMain.addThread(MaxThreadsMain.java:46) at com.google.code.java.core.threads.MaxThreadsMain.main(MaxThreadsMain.java:16) Java 6 update 26 32-bit,-XX:ThreadStackSize=128 4,000 threads: Time to create 4,000 threads was 0.525 seconds 8,000 threads: Time to create 4,000 threads was 1.239 seconds 12,000 threads: Time to create 4,000 threads was 1.902 seconds 16,000 threads: Time to create 4,000 threads was 2.529 seconds 20,000 threads: Time to create 4,000 threads was 3.165 seconds After creating 20,549 threads, java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:640) at com.google.code.java.core.threads.MaxThreadsMain.addThread(MaxThreadsMain.java:46) at com.google.code.java.core.threads.MaxThreadsMain.main(MaxThreadsMain.java:16) Java 6 update 26 32-bit,-XX:ThreadStackSize=128 4,000 threads: Time to create 4,000 threads was 0.526 seconds 8,000 threads: Time to create 4,000 threads was 1.212 seconds After creating 11,216 threads, java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:640) at com.google.code.java.core.threads.MaxThreadsMain.addThread(MaxThreadsMain.java:46) at com.google.code.java.core.threads.MaxThreadsMain.main(MaxThreadsMain.java:16) Java 6 update 26 64-bit,-XX:ThreadStackSize=128 4,000 threads: Time to create 4,000 threads was 0.577 seconds 8,000 threads: Time to create 4,000 threads was 1.292 seconds 12,000 threads: Time to create 4,000 threads was 1.995 seconds 16,000 threads: Time to create 4,000 threads was 2.653 seconds 20,000 threads: Time to create 4,000 threads was 3.456 seconds 24,000 threads: Time to create 4,000 threads was 4.663 seconds 28,000 threads: Time to create 4,000 threads was 5.818 seconds 32,000 threads: Time to create 4,000 threads was 6.792 seconds After creating 32,072 threads, java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:640) at com.google.code.java.core.threads.MaxThreadsMain.addThread(MaxThreadsMain.java:46) at com.google.code.java.core.threads.MaxThreadsMain.main(MaxThreadsMain.java:16) Java 6 update 26 64-bit,-XX:ThreadStackSize=512 4,000 threads: Time to create 4,000 threads was 0.577 seconds 8,000 threads: Time to create 4,000 threads was 1.292 seconds 12,000 threads: Time to create 4,000 threads was 1.995 seconds 16,000 threads: Time to create 4,000 threads was 2.653 seconds 20,000 threads: Time to create 4,000 threads was 3.456 seconds 24,000 threads: Time to create 4,000 threads was 4.663 seconds 28,000 threads: Time to create 4,000 threads was 5.818 seconds 32,000 threads: Time to create 4,000 threads was 6.792 seconds After creating 32,072 threads, java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:640) at com.google.code.java.core.threads.MaxThreadsMain.addThread(MaxThreadsMain.java:46) at com.google.code.java.core.threads.MaxThreadsMain.main(MaxThreadsMain.java:16) The Code MaxThreadsMain.java From http://vanillajava.blogspot.com/2011/07/java-what-is-limit-to-number-of-threads.html

I believe that every JSF fan has heard about PrimeFaces, the open source component suite for Java Server Faces (PrimeFaces). From my point of view, it is a real pleasure to explore PrimeFaces components and to get play with their capabilities. Recently, I had to use a customized growl component, meaning that I needed to make it appear in different locations and changing it’s aspect. Let me share this experience with you! The final result (after customization) is shown in the below image: - Replacing the default info icon with a custom image can be accomplished by using the infoIcon attribute of component: - In addition, you can customize the image’s CSS properties, like width and height, by modifying the CSS class ui-growl-image: .ui-growl-image { width: 100px; height: 95px; float: left; } - By default, the growl’s message will be displayed for 6000 ms and after that will be hidden. You can change this period by setting the sticky attribute to true (the message will never get hide) or changing the duration in the life attribute. - The growl background it is also customizable through CSS. This time operate over ui-growl-item-container class: .ui-growl-item-container { background-image: url('./imgs/background.jpg'); } - The growl closing icon is set by default in the top left corner. If you want it to appear in a different place, like top right corner, you need to modify the CSS ui-growl-icon-close class: .ui-growl-icon-close { cursor: pointer; position: absolute; left: 335px; top: 15px; } - You can also change the font and style of the summary and the detail message, like below: .ui-growl-title { font: 28px "Lucida Console", Monaco, monospace; text-align: center; } .ui-growl-message p { font: 20px "Lucida Console", Monaco, monospace; letter-spacing: -1px; } - Finally, you can place the growl anywhere you want, top left, bottom left, bottom right, center and so on by ui-growl class: .ui-growl { position:fixed; top: 40%; left: 37%; width: 360px; height: 110px; z-index:9999; } That’s it ! Hope you like it! From http://e-blog-java.blogspot.com/2011/07/how-to-customize-growl-primefaces.html

as i'm sure you know python doesn't have a concept of private members. one trick that is sometimes used is to hide an object inside a python closure, and provide a proxy object that only permits limited access to the original object. here's a simple example of a hide function that takes an object and returns a proxy. the proxy allows you to access any attribute of the original, but not to set or change any attributes. def hide(obj): class proxy(object): __slots__ = () def __getattr__(self, name): return getattr(obj, name) return proxy() here it is in action: >>> class foo(object): ... def __init__(self, a, b): ... self.a = a ... self.b = b ... >>> f = foo(1, 2) >>> p = hide(f) >>> p.a, p.b (1, 2) >>> p.a = 3 traceback (most recent call last): ... attributeerror: 'proxy' object has no attribute 'a' after the hide function has returned the proxy object the __getattr__ method is able to access the original object through the closure . this is stored on the __getattr__ method as the func_closure attribute (python 2) or the __closure__ attribute (python 3). this is a "cell object" and you can access the contents of the cell using the cell_contents attribute: >>> cell_obj = p.__getattr__.func_closure[0] >>> cell_obj.cell_contents <__main__.foo object at 0x...> this makes hide useless for actually preventing access to the original object. anyone who wants access to it can just fish it out of the cell_contents . what we can't do from pure-python is*set* the contents of the cell, but nothing is really private in python - or at least not in cpython. there are two python c api functions, pycell_get and pycell_set , that provide access to the contents of closures. from ctypes we can call these functions and both introspect and modify values inside the cell object: >>> import ctypes >>> ctypes.pythonapi.pycell_get.restype = ctypes.py_object >>> py_obj = ctypes.py_object(cell_obj) >>> f2 = ctypes.pythonapi.pycell_get(py_obj) >>> f2 is f true >>> new_py_obj = ctypes.py_object(foo(5, 6)) >>> ctypes.pythonapi.pycell_set(py_obj, new_py_obj) 0 >>> p.a, p.b (5, 6) as you can see, after the call to pycell_set the proxy object is using the new object we put in the closure instead of the original. using ctypes may seem like cheating, but it would only take a trivial amount of c code to do the same. two notes about this code. it isn't (of course) portable across different python implementations don't ever do this, it's for illustration purposes only! still, an interesting poke around the cpython internals with ctypes. interestingly i have heard of one potential use case for code like this. it is alleged that at some point armin ronacher was using a similar technique in jinja2 for improving tracebacks. (tracebacks from templating languages can be very tricky because the compiled python code usually bears a quite distant relationship to the original text based template.) just because armin does it doesn't mean you can though...