Hi there back again. Hope everyone is already exploiting the power of LINQ on a fairly regular basis. Okay, everyone knows by now how simple LINQ queries with a where and select (and orderby, and Take and Skip and Sum, etc) are translated from a query comprehension into an equivalent expression for further translation: from p in products where p.Price > 100 select p.Name becomes products.Where(p => p.Price > 100).Select(p => p.Name) All blue syntax highlighting has gone; the compiler is happy with what remains and takes it from there in a left-to-right fashion (so, it depends on the signature of the found Where method whether or not we take the route of anonymous methods or, in case of an Expression<…> signature, the route of expression trees). But let’s make things slightly more complicated and abstract: from i in afrom j in bwhere i > jselect i + j It’s more complicated because we have two from clauses; it’s more abstract because we’re using names with no intrinsic meaning. Let’s assume a and b are IEnumerable sequences in what follows. Actually what the above query means in abstract terms is: (a X b).Where((i, j) => i > j).Select((i, j) => i + j) where X is a hypothetical Cartesian product operator, i.e. given a = { 1, 4, 7 } and b = { 2, 5, 8 }, it produces { (1,2), (1,5), (1,8), (4,2), (4,5), (4,8), (7,2), (7,5), (7,8) }, or all the possible pairs with elements from the first sequence combined with an element from the second sequence. For the record, the generalized from of such a pair – having any number of elements – would be a tuple. If we would have this capability, Where would get a sequence of such tuples, and it could identify a tuple in its lambda expression as a set of parameters (i, j). Similarly, Select would do the same and everyone would be happy. You can verify the result would be { 6, 9, 12 }. Back to reality now: we don’t have the direct equivalent of Cartesian product in a form that produces tuples. In addition to this, the Where operator in LINQ has a signature like this: IEnumerable Where(this IEnumerable source, Func predicate) where the predicate parameter is a function of one – and only one – argument. The lambda (i, j) => i > j isn’t compatible with this since it has two arguments. A similar remark holds for Select. So, how can we get around this restriction? SelectMany is the answer. Demystifying SelectMany What’s the magic SelectMany all about? Where could we better start our investigation than by looking at one of its signatures? IEnumerable SelectMany( this IEnumerable source, Func> collectionSelector, Func resultSelector) Wow, might be a little overwhelming at first. What does it do? Given a sequence of elements (called source) of type TSource, it asks every such element (using collectionSelector) for a sequence of – in some way related – elements of type TCollection. Next, it combines the currently selected TSource element with all of the TCollection elements in the returned sequence and feed it in to resultSelector to produce a TResult that’s returned. Still not clear? The implementation says it all and is barely three lines: foreach (TSource item in source) foreach (TCollection subItem in collectionSelector(item)) yield return resultSelector(item, subItem); This already gives us a tremendous amount of power. Here’s a sample: products.SelectMany(p => p.Categories, (p, c) => p.Name + “ has category “ + c.Name) How can we use this construct to translate multiple from clauses you might wonder? Well, there’s no reason the function passed in as the first argument (really the second after rewriting the extension method, i.e. the collectionSelector) uses the TSource argument to determine the IEnumerable result. For example: products.SelectMany(p => new int[] { 1, 2, 3 }, (p, i) => p.Name + “ with irrelevant number “ + i) will produce a sequence of strings like “Chai with irrelevant number 1”, “Chai with irrelevant number 2”, “Chai with irrelevant number 3”, and similar for all subsequent products. This sample doesn’t make sense but it illustrates that SelectMany can be used to form a Cartesian product-like sequence. Let’s focus on our initial sample: var a = new [] { 1, 4, 7 };var b = new [] { 2, 5, 8 };from i in afrom j in bselect i + j; I’ve dropped the where clause for now to simplify things a bit. With our knowledge of SelectMany above we can now translate the LINQ query into: a.SelectMany(i => b, …) This means: for every i in a, “extract” the sequence b and feed it into …. What’s the …’s signature? Something from a (i.e. an int) and something from the result of the collectionSelector (i.e. an int from b), is mapped onto some result. Well, in this case we can combine those two values by summing them, therefore translating the select clause in one go: a.SelectMany(i => b, (i, j) => i + j) What happens when we introduce a seemingly innocent where clause in between? from i in afrom j in bwhere i > jselect i + j; The first two lines again look like: a.SelectMany(i => b, …) However, going forward from there we’ll need to be able to reference i (from a) and j (from b) in both the where and select clause that follow but both the corresponding Where and Select methods only take in “single values”: IEnumerable Where(this IEnumerable source, Func predicate);IEnumerable Select(this IEnumerable source, Func projection); So what can we do to combine the value i and j into one single object? Right, use an anonymous type: a.SelectMany(i => b, (i, j) => new { i = i, j = j }) This produces a sequence of objects that have two public properties “i” and “j” (since it’s anonymous we don’t care much about casing, and indeed the type never bubbles up to the surface in the query above, because of what follows: a.SelectMany(i => b, (i, j) => new { i = i, j = j }).Where(anon => anon.i > anon.j).Select(anon => anon.i + anon.j) In other words, all references to i and j in the where and select clauses in the original query expression have been replaced by references to the corresponding properties in the anonymous type spawned by SelectMany. Lost in translation This whole translation of this little query above puts quite some work on the shoulder of the compiler (assuming a and b are IEnumerable and nothing more, i.e. no IQueryable): The lambda expression i => b captures variable b, hence a closure is needed. That same lambda expression acts as a parameter to SelectMany, so an anonymous method will be created inside the closure class. For new { i = i, j = j } an anonymous type needs to be generated. SelectMany’s second argument, Where’s first argument and Select’s first argument are all lambda expressions that generate anonymous methods as well. As a little hot summer evening exercise, I wrote all of this plumbing manually to show how much code would be needed in C# 2.0 minus closures and anonymous methods (more or less C# 1.0 plus generics). Here’s where we start from: class Q{ IEnumerable GetData(IEnumerable a, IEnumerable b) { return from i in a from j in b where i > j select i + j; } This translates into: class Q{ IEnumerable GetData(IEnumerable a, IEnumerable b) { Closure0 __closure = new Closure0(); __closure.b = b; return Enumerable.Select( Enumerable.Where( Enumerable.SelectMany( a, new Func>(__closure.__selectMany1), new Func>(__selectMany2) ), new Func, bool>(__where1) ), new Func, int>(__select1) ); } private class Closure0 { public IEnumerable b; public IEnumerable __selectMany1(int i) { return b; } } private static Anon0 __selectMany2(int i, int j) { return new Anon0(i, j); } private static bool __where1(Anon0 anon) { return anon.i > anon.j; } private static int __select1(Anon0 anon) { return anon.i + anon.j; }private class Anon0 // generics allow reuse of type for all anonymous types with 2 properties, hence the use of EqualityComparers in the implementation{ private readonly TI _i; private readonly TJ _j; public Anon0(TI i, TJ t2) { _i = i; _j = j; } public TI i { get { return _i; } } public TJ j { get { return _j; } } public override bool Equals(object o) { Anon0 anonO = o as Anon0; return anonO != null && EqualityComparer.Default.Equals(_i, anonO._i) && EqualityComparer.Default.Equals(_j, anonO._j); } public override int GetHashCode() { return EqualityComparer.Default.GetHashCode(_i) ^ EqualityComparer.Default.GetHashCode(_j); // lame quick-and-dirty hash code } public override string ToString() { return “( i = “ + i + “, j = ” + j + “ }”; // lame without StringBuilder } Just a little thought… Would you like to go through this burden to write a query? “Syntactical sugar” might have some bad connotation to some, but it can be oh so sweet baby! Bind in disguise Fans of “monads”, a term from category theory that has yielded great results in the domain of functional programming as a way to make side-effects explicit through the type system (e.g. the IO monad in Haskell), will recognize SelectMany’s (limited) signature to match the one of bind: IEnumerable SelectMany( this IEnumerable source, Func> collectionSelector) corresponds to: (>>=) :: M x –> (x –> M y) –> M y Which is Haskell’s bind operator. For those familiar with Haskell, the “do” notation – that allows the visual illusion of embedding semi-colon curly brace style of “imperative programming” in Haskell code – is syntactical sugar on top of this operator, defined (recursively) as follows: do { e } = edo { e; s } = e >>= \_ –> do { s }do { x <- e; s } = e >>= (\x –> do { s })do { let x = e; s } = let x = e in do { s } Rename to SelectMany, replace M x by IEnumerable and assume a non-curried form and you end up with: SelectMany :: (IEnumerable, x –> IEnumerable) –> IEnumerable Identifying x with TSource, y with TResult and turning a –> b into Func yields: SelectMany :: Func, Func>, IEnumerable> and you got identically the same signature as the SelectMany we started from. For the curious, M in the original form acts as a type constructor, something the CLR doesn’t support since it lacks higher-order kinded polymorphism; it’s yet another abstraction one level higher than generics that math freaks love to use in category theory. The idea is that if you can prove laws to be true in some “structure” and you can map that structure onto an another “target structure” by means of some mapping function, corresponding laws will hold true in the “target structure” as well. For instance: ({ even, odd }, +) and ({ pos, neg }, *) can be mapped onto each other pairwise and recursively, making it possible to map laws from the first one to the second one, e.g. even + odd –> oddpos * neg –> neg This is a largely simplified sample of course, I’d recommend everyone who’s interested to get a decent book on category theory to get into the gory details. A word of caution Now that you know how SelectMany works, can you think of a possible implication when selecting from multiple sources? Let me give you a tip: nested foreachs. This is an uninteresting sentence that acts as a placeholder in the time space while you’re thinking about the question. Got it? Indeed, order matters. Writing the following two lines of code produces a different query with a radically different execution pattern: from i in a from j in b …from j in b from i in a … Those roughly correspond to: foreach (var i in a) foreach (var j in b) … versus foreach (var j in b) foreach (var i in a) … But isn’t this much ado about nothing? No, not really. What if iterating over b is much more costly than iterating over a? For example, from p in localCollectionOfProductsfrom c in sqlTableOfCategories… This means that for every product iterated locally, we’ll reach out to the database to iterate over the (retrieved) categories. If both were local, there wouldn’t be a problem of course; if both were remote, the (e.g.) SQL translation would take care of it to keep the heavy work on the remote machine. If you want to see the difference yourself, you can use the following simulation: using System; using System.Collections.Generic; using System.Diagnostics; using System.Linq; using System.Threading; class Q { static void Main() { Stopwatch sw = new Stopwatch(); Console.WriteLine("Slow first"); sw.Start(); foreach (var s in Perf(Slow(), Fast())) Console.WriteLine(s); sw.Stop(); Console.WriteLine(sw.Elapsed); sw.Reset(); Console.WriteLine("Fast first"); sw.Start(); foreach (var s in Perf(Fast(), Slow())) Console.WriteLine(s); sw.Stop(); Console.WriteLine(sw.Elapsed); } static IEnumerable Perf(IEnumerable a, IEnumerable b) { return from i in a from j in b select i + "," + j; } static IEnumerable Slow() { Console.Write("Connecting... "); Thread.Sleep(2000); // mimic query overhead (e.g. remote server) Console.WriteLine("Done!"); yield return 1; yield return 2; yield return 3; } static IEnumerable Fast() { return new [] { 'a', 'b', 'c' }; } } This produces: [img_assist|nid=4625|title=|desc=|link=none|align=none|width=259|height=374] Obviously, it might be the case you’re constructing a query that can only execute by reaching out to the server multiple times, e.g. because order of the result matters (see screenshot above for an illustration of the ordering influence – but some local sorting operation might help too in order to satisfy such a requirement) or because the second query source depends on the first one (from i in a from j in b(i) …). There’s no silver bullet for a solution but knowing what happens underneath the covers certainly provides the necessary insights to come up with scenario-specific solutions. Happy binding!

Last week I started working on very short proof of concept with a team that I am currently coaching at a short term insurance company. We hit a very common design decision: when do we use a factory pattern and when do we use a builder pattern. In the problem at hand, we needed to describe an item that will appear in an insurance policy that must be covered for fire damage. It turns out that these items are not trivial in their structure, and many things influence the premium that will paid by the policy holder for fire insurance. So, the first bit of code (in Java) in the test looked something like this. FireItem fireItem = new FireItem(); fireItem.setIndustry("Building Construction"); fireItem.setOccupationCode("Workshop"); // the postal/zip code for the location of the insured item fireItem.setAreaCode(7800); // ignore the number, we actually created a Money class fireItem.setSumInsured(200000.00); fireItem.setRoofing("Non-Standard"); After the test passed, we refactored and I sneaked in a factory method which will be used to honor default values and at the same time threw in a fluent interface (the term coined by Eric Evans and Martin Fowler. After all, I was also quietly introducing Domain Driven Design without actually saying that). The code looked like this. FireItem fireItem = FireItem.create() .inIndustry("Building Construction") .withOccupation("Workshop") .InAreaWithPostalCode(7800) .forSumInsured(200000.00) .havingRoofing("Non-Standard"); The FireItem class looked like this: public class FireItem { String industry; String occupation; // other properties ... private FireItem() { } public static FireItem create() { return new FireItem(); } public FireItem inIndustry(String industry) { this.industry = industry; return this; } // other chained methods follow a similar style returning "this" ... } Nice! Much more readable. But, we then realised that it’s easy for someone to miss one of the methods in the chain. That will result in the item having an incomplete structure. Not good! One of the things I tend to do as a coach, is to let the team I am working with, experience the problem, solution and any rewards and smells as well. Sometimes I even throw in red herring for sake of experience ;-). So, the third pass at refactoring was to introduce a validate() method on the item which throws an exception if everything was not in place. try { FireItem fireItem = FireItem.create() .inIndustry("Building Construction") .withOccupation("Workshop") .InAreaWithPostalCode(7800) .forSumInsured(200000.00) .havingRoofing("Non-Standard") .validate(); } catch (FireItemException e) { // handle the exception } Now the user of this class needs to know that the validate() method must be called before they really want to use an item object. Yuck, that’s smelly! So, for the fourth design refactoring, I introduced a builder and moved the fluent interface to the builder, still using method chaining but introduced a build() method that did the work of the previous validate() method before returning the well structured item. The FireItem class now needs the traditional bunch of getters and setters (rant - the framework goodies need them anyway!!) import static insurance.FireItemBuilder.fireItem; // ... try { FireItem fireItem = fireItem().inIndustry("Building Construction") .withOccupation("Workshop") .InAreaWithPostalCode(7800) .forSumInsured(200000) .havingRoofing("Non-Standard") .build(); } catch (FireItemException e) { // handle the exception } Much better! Note the use of the static import which gives us the liberty to use the static method without specifying the class in code body. The FireItemBuilder class looked like this. public class FireItemBuilder { private final FireItem fireItem; private FireItemBuilder() { fireItem = new FireItem(); } public static FireItemBuilder fireItem() { return new FireItemBuilder(); } public FireItemBuilder inIndustry(String industry) { fireItem.setIndustry(industry); return this; } // other chained methods follow a similar style returning "this" ... public FireItem build() throws FireItemBuilderException { validate(); return fireItem; } private void validate() throws FireItemBuilderException { // do all validations on the fire item itself and throw an exception if something fails } } Sure, we can improve the bubbling of the exception from validate() to build() and we could do with a better name for validate(). And perhaps, validate() should be on the FireItem class. But let’s stick to factories and builders and fluent interfaces. I think these three things work nicely “together”, when used for the right purpose. In a nutshell, factories are great for creating objects where defaults and invariants are easily honored during the simple message to the factory. However, if the structure of the object is more complex which makes long argument liss in messages ugly, and some form of validation is necessary before we can use an object; then a builder works beautifully. An alternative is to allow the object to have an invalid structure but you track it with an invalid state, perhaps using a state pattern. This is not exactly what the state pattern was meant for, but it will work nonetheless. Also, note that the fluent interface was used to improve readability and kick off a tiny little DSL for describing insurance items. Yes, I know it's Java and that's it's not the best language for creating internal DSL's but the point is that you can maximize the use of your language to create a DSL. After all, DSL's are not about syntactic sugar alone.

Discussion of functional programming concepts, focusing on fold operations and their implementation in C# using LINQ and expression trees.

The bulk of my career has been spent working with and implementing distributed middleware. In the mid-90's I worked for the parent company of Open Environment Corporation working on DCE tools. Later on, I worked for Iona developing their next generation CORBA ORB. Currently, I work for the JBoss division of Red Hat, which is entrenched in Java middleware. So, you could say that I have a pretty rich perspective when it comes to middleware. Over a year ago, I became exposed to a new way of writing Web Services called REST. REST is about using the principles of the World Wide Web to build applications. REST stands for REpresentational State Transfer and was first defined within a PhD thesis by Roy Fielding. REST is a set of architectural principles which ask the following questions: Why is the World Wide Web so prevalent and ubiquitous? What makes the Web scale? How can I apply the architecture of the Web to my own applications? While REST has many similarities to the more traditional ways of writing SOA applications, in many important ways it is very different. You would think that my background would be an asset to understanding this new way of creating web services, but unfortunately this was not the case. The reason is that some of the concepts of REST are hard to swallow, especially if you have written successful SOAP or CORBA applications. If your career has a foundation in one of these older technologies, there's a bit of emotional baggage you will have to overcome. For me, it took a few months and a lot of reading. For you, it may be easier. For others, they will never pick REST over something like SOAP and WS-*. I just ask that you keep an open mind and do some research if I fail to convince you that REST is an intriguing alternative to WS-*. So... RESTful Architectural Principles REST isn't protocol specific, but when people talk about REST they usually mean REST over HTTP. Technologies like SOAP use HTTP strictly as a transport protocol and thus use a very small subset of its capabilities. Many would say that WS-* uses HTTP solely to tunnel through firewalls. HTTP is actually a very rich application protocol which gives us things like content negotiation and distributed caching. RESTful web services try to leverage HTTP in its entirety using specific architectural principles. What are those RESTful principles? Addressable Resources. Every “thing” on your network should have an ID. With REST over HTTP, every object will have its own specific URI. A Uniform, Constrained Interface. When applying REST over HTTP, stick to the methods provided by the protocol. This means following the meaning of GET, POST, PUT, and DELETE religiously. Representation oriented. You interact with services using representations of that service. An object referenced by one URI can have different formats available. Different platforms need different formats. AJAX may need JSON. A Java application may need XML. Communicate statelessly. Stateless applications are easier to scale. Let's go into more detail on each of these individual principles. Addressability Addressability is the idea that every object and resource in your system is reachable through a unique identifier. This seems like a no-brainer, but, if you think about it, standardized object identity isn't available in many environments. If you have tried to implement a portable J2EE application you probably know what I mean. In J2EE, distributed and even local references to services are not standardized so it make portability really difficult. This isn't such a big deal for one application, but with the new popularity of SOA, we're heading to a world where disparate applications must integrate and interact. Not having something as simple as service addressability standardized adds a whole complex dimension to integration efforts. In the REST world, addressability is addressed through the use of URIs. URIs are standardized and well-known. Anybody who has ever used a browser is familiar with URIs. From a URI we know the object's protocol. In other words, we know how to communicate with the object. We know its host and port or rather, where it is on the network. Finally, we know the resource's path on its host, which is its identity on the server it resides. Using a unique URI to identify each of your services make each of your resources linkable. Service references can be embedded in documents or even emails. For instance, consider the situation where somebody calls your company's help desk with a problem with your SOA application. They can email a link to the developers on what exact service there was problems with. Furthermore, the data which services publish can also be composed into larger data streams fairly easily. Figure 1-1 http://customers.myintranet.com/customers/32133 5 http://products.myintranet.com/products/111 ... In this example, we have an XML document that describes a e-commerce order entry. We can reference data provided by different divisions in a company. From this reference we can not only obtain information about the linked customer and products that were bought, but we also have the identifier of the service this data comes from. We know exactly where we can further interact and manipulate this data if we so desired. The Uniform, Constrained Interface The REST principle of a constrained interface is perhaps the hardest pill for an experienced CORBA or SOAP developer to swallow. The idea behind it is that you stick to the finite set of operations of the application protocol you're distributing your services upon. For HTTP, this means that services are restricted to using the methods GET, PUT, DELETE, and POST. Let's explain each of these methods: GET is a read only operation. It is both an idempotent and safe operation. Idempotent means that no matter how many times you apply the operation, the result is always the same. The act of reading an HTML document shouldn't change the document. Safe means that invoking a GET does not change the state of the server at all. That, other than request load, the operation will not affect the server. PUT is usually modeled as an insert or update. It is also idempotent. When using PUT, the client knows the identity of the resource it is creating or updating. It is idempotent because sending the same PUT message more than once has no affect on the underlying service. An analogy is an MS Word document that you are editing. No matter how many times you click the “save” button, the file that stores your document will logically be the same document. DELETE is used to remove services. It is idempotent as well. POST is the only non-idempotent and unsafe operation of HTTP. It is a method where the constraints are relaxed to give some flexibility to the user. In a RESTFul system, POST usually models a factory service. Where with PUT you know exactly which object you are creating, with POST you are relying on a factory service to create the object for you. You may be scratching your head and thinking, “How is it possible to write a distributed service with only 4 methods?” Well... SQL only has 4 operations: SELECT, INSERT, UPDATE, and DELETE. JMS and other MOMs really only have two: send and receive. How powerful are both of these tools? For both SQL and JMS, the complexity of the interaction is confined purely to the data model. The addressability and operations are well defined and finite and the hard stuff is delegated to the data model (in SQL's case) or the message body(JMS's case). Why is the Uniform Interface Important? Constraining the interface for your web services has many more advantages than disadvantages. Let's look at a few: Familiarity If you have a URI that points to a service you know exactly what methods are available on that resource. You don't need a IDL-like file describing what methods are available. You don't need stubs. All you need is an HTTP client library. If you have a document that is composed of links to data provided by many different services, you already know what method to call to pull in data from those links. Interoperability HTTP is a very ubiquitous protocol. Most programming languages have an HTTP client library available to them. So, if your web service is exposed via REST, there is a very high probably that people that want to use your service will be able to without any additional requirements beyond being able to exchange the data formats the service is expecting. With CORBA or WS-* you have to install vendor specific client libraries. How many of you have had the problem of getting CORBA or WS-* vendors to interoperate? It has traditionally been very problematic. The WS-* set of specifications have also been a moving target over the years. So with WS-* and CORBA, you not only have to worry about vendor interoperability, you have to make sure that your client and server are using the same specification version. With REST over HTTP, you don't have to worry about either of these things and can just focus on understanding the data format of the service. I like to think that you are focusing on what is really important: application interoperability, rather than vendor interoperability. Scalability Because REST constrains you to a well-defined set of methods, you have predictable behavior which can have incredible performance benefits. GET is the strongest example. Because GET is a read method that is both idempotent and safe, browsers and HTTP proxies can cache responses to servers which can save a huge amount of network traffic and hits to your website. Add the capabilities of HTTP 1.1's Cache-Control header, and you have a incredibly rich way of defining caching policies for your services. It doesn't end with caching though. Consider both PUT and DELETE. Because they are idempotent, the client, nor the server have to worry about handling duplicate message delivery. This saves a lot of book keeping and complex code. Representation Oriented The third architectural principle of REST is that your services should be representation oriented. Each service is addressable through a specific URI and representations are exchanged between the client and service. With a GET operation you are receiving a representation of the current state of that resource. A PUT or POST passes a representation of the resource to the server so that the underlying resource's state can change. In a RESTful system, the complexity of the client-server interaction is within the representations being passed back and forth. These representations could be XML, JSON, YAML, or really any format you can come up with. One really cool thing about HTTP is that it provides a simple content negotiation protocol between the client and server. Through the Content-Type header, the client specifies the representation's type. With the Accept header, the client can list its preferred response formats. AJAX clients can ask for JSON, Java for XML, Ruby for YAML. Another thing this is very useful for is versioning of services. The same service can be available through the same URI with the same methods (GET, POST, etc.), and all that changes is the mime type. For example, the mime type could be “application/xml” for an old service while newer services could exchange “application/xml;schemaVersion=1.1” mime types. All and all, because REST and HTTP have a layered approach to addressability, method choice, and data format, you have a much more decoupled protocol that allows your service to interact with a wide variety of different clients in a consistent way. Communicate Statelessly The last RESTful principle I will discuss is the idea of statelessness. When I talk about statelessness though, I don't mean that your applications can't have state. In REST, stateless means that there is no client session data stored on the server. The server only records and manages the state of the resources it exposes. If there needs to be session specific data, it should be held and maintained by the client and transfered to the server with each request as needed. A service layer that does not have to maintain client sessions is a lot easier to scale as it has to do a lot less expensive replications in a clustered environment. Its a lot easier to scale up as all you have to do is add machines. A world without server maintained session data isn't so hard to imagine if you look back 12-15 years ago. Back then many distributed applications had a fat GUI client written in Visual Basic, Power Builder, or Visual C++ talking RPCs to a middle-tier that sat in front of a database. The server was stateless and just processed data. The fat client held all session state. The problem with this architecture was an operations one. It was very hard for operations to upgrade, patch, and maintain client GUIs in large environments. Web applications solved this problem because the applications could be delivered from a central server and rendered by the browser. We started maintaining client sessions on the server because of the limitations of the browser. Now, circa 2008, with the growing popularity of AJAX, Flex, and Java FX, the browsers are sophisticated enough to maintain their own session state like their fat-client counterparts in the mid-90s used to do. We can now go back to that stateless scalable middle tier that we enjoyed in the past. Its funny how things go full circle sometimes. Conclusion REST identifies the key architectural principles of why the World Wide Web is so prevalent and scalable. The next step in the evolution of the web is to apply these principles to the semantic web and the world of web services. REST offers a simple, interoperable, and flexible way of writing web services that can be very different than the RPC mechanisms like CORBA and WS-* that so many of us have had training in. This article is the first of a two part series. In this article I wanted to introduce you to the basic concepts of REST. In my next article “Putting Java to REST”, we will build a very simple RESTful service in Java using the new JCP standard JAX-RS. In other words, you'll get to see the theory being put into action. Until then, I urge you to read more about REST. Below are some interesting links. Read Part II of this series: Putting Java to REST Links O'Reilly's "RESTful Web Services" book Addressing Doubts about REST The REST WIKI About the Author Bill Burke is an engineer and Fellow at the JBoss division of Red Hat. He blogs regularly at bill.burkecentral.com.

Explore the importance of mocking, including mock objects.

String cmd = "ls -al"; Runtime run = Runtime.getRuntime(); Process pr = run.exec(cmd); pr.waitFor(); BufferedReader buf = new BufferedReader(new InputStreamReader(pr.getInputStream())); String line = ""; while ((line=buf.readLine())!=null) { System.out.println(line); }

This tutorial takes a look at SOAP Message handlers and how easy it is to write handlers using JAX-WS 2.0. JAX-WS 2.0 allows both regular Java classes and stateless EJBs(Session beans) to be exposed as web services. The JAX-WS 2.0 is the core specification that defines the web services standard for Java EE 5 specification. JAX-WS 2.0 is an extension of the Java API for XML-RPC (JAX-RPC) 1.0. SOAP message handlers are used to intercept the SOAP message as they make their way from the client to the end-point service and vice-versa. These handlers intercept the SOAP message for both the request and response of the Web Service. If you are familiar with EJB interceptors, handlers are similar to EJB interceptors and are defined in an XML file. A few typical scenarios where you would be using SOAP Message handlers are: to encrypt and decrypt messages, to support logging, caching and in some cases auditing, and in rare cases to provide transaction management as well. So much for the theory. Lets see the three basic steps to use a simple log handler to intercept and print our SOAP messages (request and response). In this tutorial, we are going to expose an EJB 3 stateless session bean as a web service which is simple and can be done by adding the @WebService annotation. So, here comes the source code for the interface as well as the implementation class which is self explanatory: Listing 1: Remote Interface package com.ws;import javax.ejb.Remote;/** * * @author meerasubbarao */@Remotepublic interface HelloWebServiceRemote { String sayHello(String name); } Listing 2: Bean Implementation package com.ws;import javax.ejb.Stateless;import javax.jws.WebMethod;import javax.jws.WebParam;import javax.jws.WebService;/** * * @author meerasubbarao */@WebService@Statelesspublic class HelloWebServiceBean implements HelloWebServiceRemote { @WebMethod(operationName = "sayHello") public String sayHello(@WebParam(name = "name") String name) { return "Hello " + name; } } You can package the two source files in a jar, deploy to your application server, and test it. Quite simple, right? For this tutorial, I am using GlassFish V2 application server and SoapUI to test my web services. Shown below are the request and response from SoapUI: Now that we know our web services work, lets start writing the message handler, which as I said earlier is just 3 steps. So, what are these SOAP message handlers? They are simple Java classes that can used to modify SOAP messages; both request as well as response. These handlers have access to both the SOAP header as well as the body of the message. Lets move on to create SOAP message handlers: Step 1. Implement the SOAPHandler interface. package com.ws;import java.io.IOException;import java.util.Collections;import java.util.Set;import java.util.logging.Level;import java.util.logging.Logger;import javax.xml.namespace.QName;import javax.xml.soap.SOAPException;import javax.xml.soap.SOAPMessage;import javax.xml.ws.handler.MessageContext;import javax.xml.ws.handler.soap.SOAPHandler;import javax.xml.ws.handler.soap.SOAPMessageContext;/** * * @author meerasubbarao */public class LogMessageHandler implements SOAPHandler { public boolean handleMessage(SOAPMessageContext messageContext) { return true; } public Set getHeaders() { return Collections.EMPTY_SET; } public boolean handleFault(SOAPMessageContext messageContext) { return true; } public void close(MessageContext context) { } The handleMessage() method is invoked for both incoming as well as outgoing messages. Lets add a new method called log() and invoke this method from the handleMessage method. Shown below are both the methods: private void log(SOAPMessageContext messageContext) { SOAPMessage msg = messageContext.getMessage(); //Line 1 try { msg.writeTo(System.out); //Line 3 } catch (SOAPException ex) { Logger.getLogger(LogMessageHandler.class.getName()).log(Level.SEVERE, null, ex); } catch (IOException ex) { Logger.getLogger(LogMessageHandler.class.getName()).log(Level.SEVERE, null, ex); } } In line 1, we are retrieving the SOAPMessage from the message context. Line 3 will print the incoming and outgoing messages in our GlassFish console. Invoke this method from within the handleMessage() as shown: public boolean handleMessage(SOAPMessageContext messageContext) { log(messageContext); return true; } Step 2: Create the XML file for the Handler Chain. Create this XML file in the same package as the web service with the name LogMessage_handler.xml. com.ws.LogMessageHandler com.ws.LogMessageHandler Lets take a close look at the various elements used above: 1. is the root element that will contain a list of all handler chains that are defined for the Web Service. 2. The child element of the element lists all the handlers in the handler chain. 3. Within the element is defined the , each handler element must in turn specify the name and also the fully qualified name of the Java class that implements the handler. If you have more than one handler, specify each one of them within the handler-chain element. Step 3: Invoking the Handler The @HandlerChain annotation is used to define a set of handlers that are invoked in response to a SOAP message. So, within our HelloWebServiceBean implementaion, you need to make a simple change to invoke the Log Handler as shown below in Line 1: package com.ws;import javax.ejb.Stateless;import javax.jws.HandlerChain;import javax.jws.WebMethod;import javax.jws.WebParam;import javax.jws.WebService;/** * * @author meerasubbarao */@WebService@Stateless@HandlerChain(file = "LogMessage_handler.xml") // Line 1public class HelloWebServiceBean implements HelloWebServiceRemote { @WebMethod(operationName = "sayHello") public String sayHello(@WebParam(name = "name") String name) { return "Hello " + name; } } We added the annotation for the handlers; lets recompile, repackage and redeploy the application. Now invoke the web service from SoapUI and see if it works. If it does, we should be able to see the request and response logged in our GlassFish console window. Here is the final output: **RemoteBusinessJndiName: com.ws.CustomerManagerRemote; remoteBusIntf: com.ws.CustomerManagerRemote LDR5010: All ejb(s) of [EJBWebServices] loaded successfully! Javalobby Hello Javalobby In this article, we saw how simple and easy it was to create and use SOAP Handlers to intercept request and response of SOAP messages. We implemented the SOAPHandler interface, wrote minimal XML to define the handler chain, and finally added one simple annotation to the web service implementation class. We were also able to test these web service using SoapUI. In Part 2 of this series, we will see how to actually parse the SOAP Headers, and also use multiple handlers. Stay tuned for more..

Object-oriented programming languages and techniques provide a powerful means for designing and building applications. These techniques do not always translate well into a service oriented paradigm. Service orientation demands a different set of design guidelines and requirements than an object-oriented application. Understanding how an object-oriented design can negatively impact a service-oriented design is key to building services that support an agile enterprise. This article examines where the two designs impact each other as well as methods for addressing the incompatibilities between the two while still leveraging the power of both. by Larry Guger Introduction Object orientation is a good thing. I would like to believe that I write code in a well-defined object oriented manner, taking advantage of all the goodness that is provided, such as encapsulation, polymorphism and inheritance. These are important concepts that make modern software applications easier to develop, enhance and maintain. I’m sold. Service-orientation is a good thing too. As the industry moves toward service-orientation we naturally take along a lot of what we have learned over the years and apply this to the new way of doing things. Visual Studio, the .NET framework, and especially Windows Communication Foundation (WCF) support the development of service-oriented applications. This was one of the core design goals behind WCF, but moving from object-oriented design techniques to service-oriented design techniques is not without its challenges. Part of the reason can be apportioned towards the tooling. We have very mature tools to support object-oriented design and development but fewer tools that emphasize service-oriented design. This is partly because we, as an industry, are still really figuring out what service-orientation really means. This article explores what I am referring to as the “impedance mismatch” between the two design paradigms. Note: Most of the concepts and ideas presented are not specific to .NET or Visual Studio until I refer to the namespace generation problem later in the discussion (as it manifests itself in Visual Studio). However, it is worth noting that this problem can present itself on any platform. Designing an OO System Here is an example of one model that would make sense in the object-oriented world: Figure 1 Figure 1 is a simplified model designed to support some form of purchasing functionality provided by the application. A customer contains a mailing address and a shipping address and the customer can also hold many contracts with the company that is supplying products. The customer is able to place orders against these contracts with any given order containing one or more line items each of which is an order for a product. In addition, the model permits the developer to navigate from a PurchaseOrder object to the Contract under which the order was placed by using an object reference. Likewise, a Contract will contain a collection of all of the PurchaseOrders placed under it. This model is also repeated between the Customer and its Contracts as well as PurchaseOrders and OrderLineItems. We now have a nice object model that permits navigation between related objects in any direction. Service Enablement In a distributed computing environment such as is found in almost every enterprise today there is a business logic/service tier that resides on some central server farm that exposes services for working with the contained business functionality. This service tier is accessed by a client tier, whether the client is a Web application, a rich client application or a B2B service implementation. To support the object model above, one can imagine a collection of services that are targeted at a handful of business needs: customer services, contract services and order services. Each of these services has its own endpoint with a few methods to support working with each of the primary business types identified. To be specific, a service could be developed to support working with customer data that would contain methods such as CreateCustomer, SearchCustomers, and GetCustomer. Each of these methods would either accept or return customer objects and if you retrieved a customer object using the GetCustomer service you could inspect the contracts that the customer has as well as the orders placed under each contract. Chances are that if you retrieved a customer using the GetCustomer method you would not be getting the populated contract objects along with it at that time. You would need to make additional calls to retrieve individual contracts and associated orders if that was the information you were after. The same concept should hold for working with contracts or orders. As you can see, the object hierarchy is maintained. Assuming that this is the approach taken and the GetCustomer method returns a serialized object of type Customer once that customer object is deserialized in the client application it is easy enough to create contract objects, attach them to the contracts collection on the customer and create order objects and attach them to the contract objects and we have the same object oriented goodness on the client as we have on the server. This is a standard approach when first building service enabled applications. Unfortunately this does not work well for service-oriented applications that are intended to be reused throughout the enterprise for other purposes beyond the initial application. Here’s why. Service Referencing Let’s think about developing the client side portion of our application, whether it is a web, WinForms or WPF application does not matter. To begin, we create a user interface for dealing with all things related to customers. We can create new customers, modify existing ones and search for customers based on various criteria. Once we have the user interface defined we add a service reference, using Visual Studio, to our previously created service which in turn generates our classes and proxies. We instantiate objects, add data supplied by the user and submit these objects to our services. Pretty standard stuff. Next we move on to developing the portion of the user interface that deals with contracts. We follow the same pattern and things are working well until we get to a namespace collision. When we added a reference to the customer related service, Visual Studio generated code for the classes that make up the return types and the request types our service supports. This will include all of the serializable types in the customer object graph. When we add a reference to the contract service, Visual Studio will generate the code for the classes that make up the return and request types that this service supports. This will also include all of the serializable types in the contract object graph. In other words we end up with generated code for all of the classes described above twice! This is because each reference generates the classes in a distinct namespace. Even if you use the same reference name Visual Studio will alter them slightly to make them distinct. For example if both the first and second service references are given the name “localhost”, Visual Studio will append a “1” to the end of the first reference making the namespace begin with “localhost1”. You now have two Customer classes, localhost.Customer and localhost1.Customer, as well as two of every class in the respective object graphs. Now your code is duplicated and stored in different types. You cannot create an instance of a localhost.Customer object and assign it to a variable of type localhost1.Customer. Not only do you not get object compatibility but you also end up with a whole bunch of equivalent classes under different namespaces. There are a few commonly used ways to deal with this problem: 1. Add a reference to the assembly that contains your objects to your UI projects, and alter the service references to use that/those assemblies rather than generating the code. 2. Alter the generated code to remove the duplicates. 3. Alter the code generation process to reference the assemblies containing your data objects. 4. Develop mapping code to translate from one type to another. There are challenges with all of these. The first and third options may not be possible if you don’t have access to the assemblies, perhaps you are referencing services that you did not develop, and perhaps the objects contain code that shouldn’t reside on the UI side of things such as database access logic. The second option is simply fraught with perils as the code will be updated and need re-altering after every reference update, and if this is in mid development there may be many updates. The fourth option adds extra work and who needs extra work? The Service-oriented Approach The correct approach is to avoid these problems completely when developing your services. Here’s how. A Customer service should know about customers and data directly related to a customer only. Your Customer service methods should return a slightly different object graph than the object graph defined above. You will still have the customer object and you will still have the addresses for that customer but that’s it. Break the graph at the collection of contracts. When the client requires the contracts for a customer they need to submit a request to the Contract service asking for the contracts for a particular customer. This should actually be the same approach regardless of the object graph in use. Whether the customer explicitly asks for the contracts or the system hides the implementation details and makes the call to retrieve contracts are simply implementation details. The fact that the object graph does not contain direct links to the contracts would not impact the user experience. Let me explain. Regardless of the size of the object graph in use it should be a rare case in which the entire graph is populated and returned by a service call. Generally you will find that only a portion of the objects are in use for any given user action. To minimize the amount of data that is sent over a network only the relevant objects should be populated and returned. The code that is developed on the client side takes the responsibility for calling the appropriate services to populate further objects in the graph as the user requests them - this is often termed “lazy loading”. The goal of “lazy loading” is to retrieve only the data that is required so as to improve performance of the application. In fact, the simplified object graph better supports this design approach than the more complex graph does. With the complex graph, if you have a Contract object and need to perform some work with the related Customer for that contract you still need a sparsely populated Customer object that contains, at a minimum, the customer Id that can be used to retrieve the full customer object from the service. With the simplified graph the Contract class contains a customer Id directly. The relations between the objects are still maintained, however with the simplified version the relationships are more explicit than implicit, as they are with the complex version. To bring this back to the user experience, the user should not know whether the underlying code has been developed using the simplified or complex graph. They should be able to navigate from a contract to the related customer just as easily. It is up to you, the developer, to make this experience seemless. Figure 2 Again the object graph is kept simple as shown in Figure 2. As with the customer, contracts would not maintain a full customer object reference as part of the contract definition on the client side of the equation. A Contract object passed from the service would consist of only itself, no customer, and no PurchaseOrders. In place of the customer reference each contract object would maintain the customer identifier so as to be able to uniquely identify which customer the contract belongs to and provide the means to “navigate” to the customer object when required. As a side note, when requesting the collection of contracts for any given customer the collection of data returned should only consist of enough data in each object to uniquely identify the contract being sought, whether these objects are sparsely populated contract objects or a “light” version of the contract class does not really matter. You can then query for the full contract object based on the unique identifier that would be part of the collection of query results from the first request that returned the collection. Again, these are implementation details that are hidden from the user experience and need to be determined based on application needs. The End Result The end results of using this approach are: • Returned result data is kept to the relevant bits. Even though this goal can still be achieved with a more complex object graph this approach enforces it. • When adding a service reference using tools like Visual Studio the generated classes have a smaller object graph which avoids having multiple identical classes in different namespaces. • Cleaner separation of duties. Once you have adjusted to using this approach you will find that you need to create a mapping layer just behind the services - to map to and from the interfaces exposed by the services and your more complex object graphs. Either that or you can use simpler object graphs on the server side. However, these simple object graphs may not provide the functionality needed by the consumer of the service, especially if the consumer is a rich client application. In that case, a more complex object model can be designed and mapped to the simpler objects returned by the service. This keeps the service architecture simple while allowing the ability to use all of the OO principles that we’ve learned over the years. When it’s not needed, the simpler objects can simply be used as is with no additional work required. By keeping the server side object graphs at the same simplicity as the service interfaces you will find that your code becomes cleaner, more modular, your classes become more cohesive and there is less coupling between your objects. In addition, the flexibility to reuse the customer service with other services which rely on customer information, but are sourced from a different repository, is easier to achieve. Merging multiple customer data systems into a single customer service also becomes much easier if the customer data is de-coupled from any other data. This now leads to a potential increase in service-orientation reuse, and a happier enterprise. Conclusion Both object oriented and service-oriented design and develop techniques have their place in modern systems development. Object oriented systems fit well in a stateful environment while a service-oriented approach requires a stateless environment. There is nothing wrong with the strong object oriented approach as described at the start of this paper however it will not serve you well if you try and expose those object graphs through a service. With years of OO experience it’s easy to fall into OO design by default, but when designing systems we need to shift our mindset and think about what we are designing for. If it’s an SOA system, a traditional OO approach may not be the best. The tight coupling will get you in trouble as you expand the reach and reuse of your services throughout your enterprise. Keep the interfaces into your services simple and focused and you will find that your services become much easier to manage and become much more scalable. To summarize, OO is, by its nature, stateful while SOA is, by its nature, stateless. This is where the impedance mismatch shows itself.

in a nutshell, grid computing is a way to distribute your computations across multiple computers (nodes). however, even jms does that, but jms is not a grid computing product - it's a messaging protocol. to correctly classify grid computing products we have to split them into 2 categories: compute grids and data grids. compute grid compute grids allow you to take a computation, optionally split it into multiple parts, and execute them on different grid nodes in parallel. the obvious benefit here is that your computation will perform faster as it now can use resources from all grid nodes in parallel. one of the most common design patterns for parallel execution is mapreduce . however, compute grids are useful even if you don't need to split your computation - they help you improve overall scalability and fault-tolerance of your system by offloading your computations onto most available nodes. some of the "must have" compute grid features are: automatic deployment - allows for automatic deployment of classes and resources onto grid without any extra steps from user. this feature alone provides one of the largest productivity boosts in distributed systems. users usually are able to simply execute a task from one grid node and as task execution penetrates the grid, all classes and resources are also automatically deployed. topology resolution - allows to provision nodes based on any node characteristic or user-specific configuration. for example, you can decide to only include linux nodes for execution, or to only include a certain group of nodes within certain time window. you should also be able to choose all nodes with cpu loaded, say, under 50% that have more than 2gb of available heap memory. collision resolution - allows users to control which jobs get executed, which jobs get rejected, how many jobs can be executed in parallel, order of overall execution, etc. load balancing - allows to balance properly balance your system load within grid. usually range of load balancing policies varies within products. some of the most common ones are round robin, random, or adaptive. more advanced vendors also provide affinity load balancing where grid jobs always end up on the same node based on job's affinity key. this policy works well with data grids described below. fail-over - grid jobs should automatically fail-over onto other nodes in case of node crash or some other job failure. checkpoints - long running jobs should be able to periodically store their intermediate state. this is useful for fail-overs, when a failed job should be able to pick up its execution from the latest checkpoint, rather than start from scratch. grid events - a querying mechanism for all grid events is essential. any grid node should be able to query all events that happened on remote grid nodes during grid task execution. node metrics - a good compute grid solution should be able to provide dynamic grid metrics for all grid nodes. metrics should include vital node statistics, from cpu load to average job execution time. this is especially useful for load balancing, when the system or user need to pick the least loaded node for execution. pluggability - in order to blend into any environment a good compute grid should have well thought out pluggability points. for example, if running on top of jboss, a compute grid should totally reuse jboss communication and discovery protocols. data grid integration - it is important that compute grid are able to natively integrate with data grids as quite often businesses will need both, computational and data features working within same application. some compute grid vendors: - gridgain - professional open source - jppf - open source data grid data grids allow you to distribute your data across the grid. most of us are used to the term distributed cache rather than data grid (data grid does sound more savvy though). the main goal of data grid is to provide as much data as possible from memory on every grid node and to ensure data coherency. some of the important data grid features include: data replication - all data is fully replicated to all nodes in the grid. this strategy consumes the most resources, however it is the most effective solution for read-mostly scenarios, as data is available everywhere for immediate access. data invalidation - in this scenario, nodes load data on demand. whenever data changes on one of the nodes, then the same data on all other nodes is purged (invalidated). then this data will be loaded on-demand the next time it is accessed. distributed transactions - transactions are required to ensure data coherency. cache updates must work just like database updates - whenever an update failed, then the whole transaction must be rolled back. most data grid support various transaction policies, such as read committed, write committed, serializable, etc... data backups - useful for fail-over. some data grid products provide ability to assign backup nodes for the data. this way whenever a node crashes, the data is immediately available from another node. data affinity/partitioning - data affinity allows you to split/partition your whole data set into multiple subsets and assign every subset to a grid node. in the purest form, data is not replicated between nodes at all, every node is only responsible for it's own subset of data. however, various data grid products may provide different flavors of data affinity, such as replication only to back up nodes for example. data affinity is one of the more advanced features, and is not provided by every vendor. to my knowledge, according to product websites, out of commercial vendors oracle coherence and gemstone have it (there may be others). in professional open source space you can take a look at combination of gridgain with affinity load balancing and jbosscache . some data grid/cache vendors: - oracle coherence - commercial - gemstone - commercial - gigaspaces - commercial - jbosscache - professional open source - ehcache - open source

Petr Pisl is the lead engineer of the PHP work scheduled for NetBeans IDE 6.5. Here he shares an update on some of the latest work that's been done in this area. -- Geertjan Wielenga, NetBeans Zone leader This morning I have committed a feature to the NetBeans PHP support that will be part of NetBeans IDE 6.5, which offers generating of constructor, getters and setters in a PHP class. To invoke the functionality, the caret position has to be inside a PHP class and you have to press shortcut ALT+Insert (CTLRL+I on Mac). After invoking the shortcut, all possible generators are offered. Let's explain on a simple example. I have very simple PHP class, where I have only two properties - $name and $age. At first I want to add constructor. After pressing ALT+Insert (CTLRL+I on Mac), the constructor generator is offered - Constructor... item in the Generate menu. After invoking this item, the Generate Constructor dialog is displayed. There I can choose parameters of the constructor. It is possible to chose any property, then the constructor is generated without any parameter and empty. If I check / uncheck the User class, then all properties are selected / unselected. For my demonstration I have chose $name property. After pressing OK button, the constructor with one parameter is generated. Now I want to generate getter for the $name. After invoking Generate menu, you can notice that the Constructor... item is not offer anymore, because the class already has a constructor. I choose Getter... item and Generate Getters dialog is displayed. In the dialog you can choose, for which properties you want to generate getters. Again you can check / uncheck the class node, which selects / unselects all properties. As the next step I want to generate getter and setter for $age property. After choosing Getter and Setter... item in Generate menu, the dialog offers only $age property, because only this property has neither getter nor setter. When I invoke Generate menu after creating getter and setter for $age property, only setter generator is offered, because there is not defined only setter for $name property. The generate functionality is designed that you can work just with the keyboard and you don't have to use mouse. From: http://blogs.sun.com/netbeansphp

OFX4J has been released! OFX4J is a Java implementation of Open Financial Exchange, which defines web service APIs for interfacing with financial institutions. The OFX4J library includes support for both client-side and server-side implementations of both version 1 and version 2 of the OFX specification. As an example, let's consider the OFX4J client-side interface, which can be used, for example, to download your financial transactions from your bank or credit card institution. OFX4J comes with an initial set of financial institution data, but you can load your own if your financial institution isn't included. Consider the following code for downloading your bank statement: //the financial institution data is loaded or created //the FinancialInstitutionData interface defines //what is required to interface with a financial institution. FinancialInstitutionData data = ...; FinancialInstitutionService service = new FinancialInstitutionServiceImpl(); FinancialInstitution fi = service.getFinancialInstitution(data); //get a reference to a specific bank account at the FI BankAccountDetails bankAccountDetails = new BankAccountDetails(); //routing number to the bank. bankAccountDetails.setRoutingNumber("11111111"); //bank account number. bankAccountDetails.setAccountNumber("1234-5678"); //it's a checking account bankAccountDetails.setAccountType(AccountType.CHECKING); BankAccount bankAccount = fi.loadBankAccount(bankAccountDetails, "username", "password"); //read the statement (transaction details, etc.) // for a given time period. Date startDate = ...; Date endDate = ...; AccountStatement statement = bankAccount.readStatement(startDate, endDate); // get a reference to a specific credit card // account at your FI CreditCardAccountDetails ccDetails = new CreditCardAccountDetails(); ccDetails.setAccountNumber("1234-567890-1111"); CreditCardAccount ccAccount = fi.loadCreditCardAccount(ccDetails, "username", "password"); // read the statement (transaction details, etc.) // for a given time period. Date startDate = ...; Date endDate = ...; AccountStatement statement = ccAccount.readStatement(startDate, endDate); Server-side development is just straightforward, consisting of implementing a simple interface and publishing it with a simple Servlet implementation: /** * Basic interface for an OFX server. * * @author Ryan Heaton */ public interface OFXServer { /** * Get a response for the given request. * * @param request The request. * @return The response. */ ResponseEnvelope getResponse(RequestEnvelope request); }

Any input provided by a user in a GUI application must typically be validated in one way or another. There is a number of ways this gets done, while some applications have just ignored the matter altogether. When crafting an Eclipse RCP application, there are some help provided by SWT and JFace. We can add ModifyListeners and VerifyListeners to certain SWT widgets. JFace also provides ControlDecorations to help us indicate to the user where a problem with a specific input value exists. The problem is that these are at a low level, and we need to do a lot of "monkey"-coding just to add basic validation and error indication to a widget, and then we're not even touching the world of input masks. If you're like me, you want to concentrate on solving your business problem, and don't want to write lots of basic UI code over and over. This is where the RCP Toolbox is very useful. It provides a light-weight validation framework (among other features) that makes it much easier to add validation and input masks to SWT Text, Combo and CCombo widgets. The goal Let us have a look at how to define a basic wizard for creating a new Booking. This wizard must capture the following fields from the user: Name: the name of the booking, typically the name of the person making the booking. May not be empty, and preferably not less than three characters. Date: the date and time of the booking. Must be any time from the current time to the end of the year. Number of Persons: the number of persons to book for. Must be a number from 1 to 10. Telephone Number: the telephone number of the person making the booking. Must be in the form +(country code) (area code) number, e.g. +44 (33) 555-1111. And of course we want indicators next to each field when an error or warning condition exists in the field, as well as a message being written to the WizardDialog's message area. For fun we want the user to be able to get a quick-fix option on the date field for setting it to the current time. The Validation framework The RCP Toolbox provides a number of custom widgets and a easy to use validation framework. Adding validation starts with the ValidationToolkit class. This class gets instantiated to work with a specific type of contents, and is then used to create ValidatingField instances that can handle that type of contents. The rest of the framework deals with interfaces and default implementations to facilitate the validation of contents, definition of input masks, provision of quick-fixes, error-handling and conversion of the input text to specific class types. The WizardPage and the ValidationToolkits We start by first defining our BookingWizardPage class and instantiating the necessary ValidationToolkit instances. //Not all imports are shown import com.richclientgui.toolbox.validation.IFieldErrorMessageHandler; import com.richclientgui.toolbox.validation.ValidationToolkit; import com.richclientgui.toolbox.validation.converter.DateStringConverter; import com.richclientgui.toolbox.validation.converter.IntegerStringConverter; import com.richclientgui.toolbox.validation.string.StringValidationToolkit; public class BookingWizardPage extends WizardPage { private static final int DECORATOR_POSITION = SWT.TOP | SWT.LEFT; private static final int DECORATOR_MARGIN_WIDTH = 1; private static final int DEFAULT_WIDTH_HINT = 150; private StringValidationToolkit strValToolkit = null; private ValidationToolkit dateValToolkit = null; private ValidationToolkit intValToolkit = null; private final IFieldErrorMessageHandler errorMessageHandler; public BookingWizardPage() { super("booking.pageone","New Booking Entry", null); errorMessageHandler = new WizardPageErrorHandler(); } public void createControl(Composite parent) { final Composite composite = new Composite(parent, SWT.NONE); composite.setLayout(new GridLayout(2, false)); strValToolkit = new StringValidationToolkit(DECORATOR_POSITION, DECORATOR_MARGIN_WIDTH, true); strValToolkit.setDefaultErrorMessageHandler(errorMessageHandler); intValToolkit = new ValidationToolkit(new IntegerStringConverter(), DECORATOR_POSITION, DECORATOR_MARGIN_WIDTH, true); intValToolkit.setDefaultErrorMessageHandler(errorMessageHandler); dateValToolkit = new ValidationToolkit(new DateStringConverter(), DECORATOR_POSITION, DECORATOR_MARGIN_WIDTH, true); dateValToolkit.setDefaultErrorMessageHandler(errorMessageHandler); //TODO: create ValidatingFields setControl(composite); } } The StringValidationToolkit class we instantiate in line 28 is a ValidationToolkit that deals specifically with ValidatingFields that have normal String contents. In line 32 we instantiate a typed instance of ValidationToolkit that will create ValidatingFields that only takes Integers as input. We must provide a way that the contents of the fields are converted from a String to the correct content type. This is done with a set of coverter classes provided by the framework. In lines 32 and 36 we specify a IntegerStringConverter and a DateStringConverter to convert Integer and java.util.Date values respectively. The framework makes use of the JFace org.eclipse.jface.fieldassist.ControlDecoration and related classes to indicate whether a field has an error or warning condition, whether it is a required field, and if there is a quick-fix available (by right-clicking on the decorator icon) for the current error or warning condition. The position of these decorator icons relative to the input widgets as well as the margin width between the decorator icon and the widget can be specified when constructing a ValidationToolkit. All the fields created by this ValidationToolkit instance will use the same settings for there decorator icons. In lines 9 - 10 we have defined some constants for the decorator position and margins, and we use this for constructing all the ValidationToolkit instances. Handling the error messages We also make use of an IFieldErrorMessageHandler to get feedback from the validation process. The validation framework will call these error handlers when error or warning conditions occur, and allow us to do something with those messages. By default these messages are only displayed in the tooltips of the decorator icons. A default error handler can be specified for each toolkit instance, or a separate handler can be set for each ValidatingField if so required. The inner class WizardPageErrorHandler implements the IFieldErrorMessageHandler interface and basically just set the messages on the WizardPage's message area. //inner class of BookingWizardPage class WizardPageErrorHandler implements IFieldErrorMessageHandler { public void handleErrorMessage(String message, String input) { setMessage(null, DialogPage.WARNING); setErrorMessage(message); } public void handleWarningMessage(String message, String input) { setErrorMessage(null); setMessage(message, DialogPage.WARNING); } public void clearMessage() { setErrorMessage(null); setMessage(null, DialogPage.WARNING); } } The actual error or warning messages are generated by the various IFieldValidator implementations (we'll get to those), and can easily be customized by implementing custom validators. Creating a simple ValidatingField Of course just having some toolkit instances does not help us much. We need actual input widgets that are being validated. The first step is to update the createControl method. public void createControl(Composite parent) { final Composite composite = new Composite(parent, SWT.NONE); composite.setLayout(new GridLayout(2, false)); strValToolkit = new StringValidationToolkit(DECORATOR_POSITION, DECORATOR_MARGIN_WIDTH, true); strValToolkit.setDefaultErrorMessageHandler(errorMessageHandler); intValToolkit = new ValidationToolkit(new IntegerStringConverter(), DECORATOR_POSITION, DECORATOR_MARGIN_WIDTH, true); intValToolkit.setDefaultErrorMessageHandler(errorMessageHandler); dateValToolkit = new ValidationToolkit(new DateStringConverter(), DECORATOR_POSITION, DECORATOR_MARGIN_WIDTH, true); dateValToolkit.setDefaultErrorMessageHandler(errorMessageHandler); createNameField(composite); createDateField(composite); createNumberPersonsField(composite); createTelephoneNumberField(composite); setControl(composite); } Then we can look at creating our first validated input field that makes use of a SWT Text widget to capture the name of the person doing the booking. private void createNameField(Composite composite) { new Label(composite, SWT.NONE).setText("Booking Name:"); final ValidatingField nameField = strValToolkit.createTextField( composite, new IFieldValidator(){ public String getErrorMessage() { return "Name may not be empty."; } public String getWarningMessage() { return "That's a very short name..."; } public boolean isValid(String contents) { return !(contents.length()==0); } public boolean warningExist(String contents) { return contents.length() < 3; } }, true, ""); GridData gd = new GridData(SWT.LEFT, SWT.CENTER, false, false); gd.widthHint = DEFAULT_WIDTH_HINT; nameField.getControl().setLayoutData(gd); } Since this field works with String contents, we make use of the strValToolkit instance to create the field in line 4 above. We specify the parent composite that the input widget must be added to, the IFieldValidator that will be used to validate the field contents, whether this is a required field or not (thus whether the required decorator icon must be shown or not) and an initial empty string value for the field. Note that this call will also create a Text widget to be used for the field, but the API allows that you can create your own Text, Combo or CCombo instance and pass that to the toolkit to use when creating a new ValidatingField. An anonymous inner class implementation of IFieldValidator is specified in lines 5 - 23. We're doing some very basic validation checks in this example, but it is easy to implement validators that makes use of other heavy-weight business validation frameworks. Our validator will indicate an error condition if the contents of the field is empty (line 16), in which case the error message "Name may not be empty." will be displayed (line 8). This validator will also indicate a warning condition if the name field contains less than 3 characters (line 20) with the message "That's a very short name..." (line 12). In lines 24 - 26 we set the layout of the input widget on the composite. Dating an input mask Dates have always been a difficult input type to deal with. The easiest way for us developers to deal with them are to use widgets that pop up a calendar from where the user can choose a day, and possibly a time as well. However, this way of dealing with dates are not always a favourite with touch-typing end-users. They prefer some masked field where they only need to fill in the bits of the date that matter. Using the mouse should be restricted as far as possible. Luckily the RCP Toolbox provides a way of specifying input masks, as well as specific implementations of validators and converters for dealing with Dates. We want to create a field that only takes dates as input, where the date entered must be of the form yyyy-MM-dd HH:mm (e.g 2008-07-19 21:00), and fall in the date range starting at the current time and ending at the end of the year 2008. private void createDateField(Composite composite) { new Label(composite, SWT.NONE).setText("Booking Time:"); final Date endYear = getEndYearDate(); //we create a Date field that takes input of form yyyy-MM-dd HH:mm //and only allows values from now till the end of the year final ValidatingField rangedDateField = dateValToolkit.createTextField(composite, new RangedDateFieldValidator( DateFieldValidator.DATE_TIME_HHMM_DASH, dateValToolkit.getStringConverter(), new Date(), endYear), true, new Date()); GridData gd = new GridData(SWT.LEFT, SWT.CENTER, false, false); gd.widthHint = DEFAULT_WIDTH_HINT; rangedDateField.getControl().setLayoutData(gd); } Here we used the dateValToolkit instance to create the field as the contents of the field must be a java.util.Date. We specify an instance of RangedDateFieldValidator (provided by the framework) that makes use of the specified Date input mask pattern DateFieldValidator.DATE_TIME_HHMM_DASH (line 9) to validate the contents of the field. Other patterns are available, or custom ones can also be defined. The DateStringConverter specified when dateValToolkit was constructed will be used to convert from Dates to Strings and vice versa. In line 11 we specify the valid date range for the field, from the current date and time to the end of the year, and in line 13 we set the initial value of the field to the current time. Providing quick-fixes I found that developers using Eclipse RCP to develop their applications like to make the experience for the end-user as good as possible. So we should not stop at just validating input; we must also try and help them quickly fix mistakes, where possible. In this example we can do that by specifying a IQuickFixProvider. //add at end of createDateField(..) method //we add a quickfix that will set it to the current date rangedDateField.setQuickFixProvider(new IQuickFixProvider(){ public boolean doQuickFix(ValidatingField field) { field.setContents(new Date()); return true; } public String getQuickFixMenuText() { return "Set to current time"; } public boolean hasQuickFix(Date contents) { //would typically first check contents to determine if quickfix //is possible return true; } }); The above is a very simple quick-fixer. It always says it has a quick-fix available (line 17), where a more complex provider will first check the contents of the field to determine if there is a quick-fix. When the user performs the quick-fix, it just sets the contents of the field to the current date and time (line 6). When the validation framework detects there is an error condition on a field, it will see if there is a IQuickFixProvider available with a quick-fix. If this is the case, it will add the quick-fix option to a context-menu on the decorator icon with the text specified in line 11. All the user then needs to do is right-click on the decorator icon and select the quick-fix to perform. Validating Combos A Combo widget would be just the thing to use for capturing the number of persons for the booking. Our requirements say we must limit the number to a maximum of 10 people (and a minimum of 1 goes without saying). Once again we do not want to force the user to use numerous mouse-clicks or keystrokes to select the number, so we do not make the Combo read-only, and rather decide to add validation to it. private void createNumberPersonsField(Composite composite) { new Label(composite, SWT.NONE).setText("Number of persons:"); final ValidatingField numberPersonsField = intValToolkit.createComboField( composite, new StrictRangedNumberFieldValidator(1, 10){ public String getErrorMessage() { return "Bookings for groups bigger than 10 not allowed"; } public String getWarningMessage() { return null; } public boolean warningExist(Integer contents) { return false; } }, true, 2, new Integer[]{1,2,3,4,5,6,7,8,9,10}); GridData gd = new GridData(SWT.LEFT, SWT.CENTER, false, false); gd.widthHint = DEFAULT_WIDTH_HINT; numberPersonsField.getControl().setLayoutData(gd); } Here we make use of the intValToolkit.createComboField method to create a field containing a Combo widget with contents of type Integer. We specify a StrictRangedNumberFieldValidator (line 6) to ensure that the entered value only consists of digits and falls in the range 1 to 10. No warning conditions are checked. In line 22 we populate the Combo with a list of Integers from 1 to 10, and in line 21 we select a default value of 2. As easy as counting to 5. And don't forget the telephone number Freeform telephone number fields are used a lot, but unfortunately end-users can easily make mistakes in such fields. We want to force our user to input the telephone number in a specific form, thus at least preventing the cases where digits are missed. To do this, we make use of the framework's TelephoneNumberValidator. This validator allows telephone number to be entered in either international format (e.g. +44 (55) 555-5555) or in domestic format (e.g. (055) 555-5555). private void createTelephoneNumberField(Composite composite) { new Label(composite, SWT.NONE).setText("Contact Telephone Nr:"); final ValidatingField telephoneField = strValToolkit.createTextField( composite, new TelephoneNumberValidator(true), true, "+44 (55) 555-4321"); GridData gd = new GridData(SWT.LEFT, SWT.CENTER, false, false); gd.widthHint = DEFAULT_WIDTH_HINT; telephoneField.getControl().setLayoutData(gd); } We are using strValToolkit to create the field, since the contents will be managed as a String. Then we specify a TelephoneNumberValidator as the validator in line 7, with the true parameter indicating that we want to use the international format. In line 9 we provide an initial value. Conclusion This article describes a very simple example of how to add validation to SWT widgets using the RCP Toolbox. In a real-world application the actual business validations to be done might be more complex, but if this validation framework is used, the UI code related to validation would remain as simple as the code in these examples. This validation framework really made our development much easier, allowing us to concentrate on the business code. It is very easy to extend the framework with custom validators, converters, quick-fix providers and error handlers that ties into existing business code or other validation code, rules engines etc. Note that the examples in this article need Eclipse RCP 3.3 or 3.4 as well as RCP Toolbox v1.0.1, created and distributed by www.richclientgui.com.

Continuing the tour in the ASP.NET client side state management our current stop is the query string technique. You can read my previous posts in the state management subject in the following links: Client side state management introduction ViewState technique Hidden fields technique What are Query Strings? Query strings are data that is appended to the end of a page URL. They are commonly used to hold data like page numbers or search terms or other data that isn't confidential. Unlike ViewState and hidden fields, the user can see the values which the query string holds without using special operations like View Source. An example of a query string can look like http://www.srl.co.il?a=1;b=2. Query strings are included in bookmarks and in URLs that you pass in an e-mail. They are the only way to save a page state when copying and pasting a URL. The Query String Structure As written earlier, query strings are appended to the end of a URL. First a question mark is appended to the URL's end and then every parameter that we want to hold in the query string. The parameters declare the parameter name followed by = symbol which followed by the data to hold. Every parameter is separated with the ampersand symbol. You should always use the HttpUtility.UrlEncode method on the data itself before appending it. Query String Limitations You can use query string technique when passing from one page to another but that is all. If the first page need to pass non secure data to the other page it can build a URL with a query string and then redirect. You should always keep in mind that a query string isn't secure and therefore always validate the data you received. There are a few browser limitation when using query strings. For example, there are browsers that impose a length limitation on the query string. Another limitation is that query strings are passed only in HTTP GET command. How To Use Query Strings When you need to use a query string data you do it in the following way: string queryStringData = Request.QueryString["data"]; In the example I extract a data query string. The structure of the URL can look like url?data=somthing. After getting to data parameter value you should validate it in order not to enable security breaches. The next example is a code to help inject a query string into a URL: public string BuildQueryString(string url, NameValueCollection parameters){ StringBuilder sb = new StringBuilder(url); sb.Append("?"); IEnumerator enumerator = parameters.GetEnumerator(); while (enumerator.MoveNext()) { // get the current query parameter string key = enumerator.Current.ToString(); // insert the parameter into the url sb.Append(string.Format("{0}={1}&", key, HttpUtility.UrlEncode(parameters[key]))); } // remove the last ampersand sb.Remove(sb.Length - 1, 1); return sb.ToString(); } Summary To sum up the post, query string is another ASP.NET client side state management technique. It is most helpful for page number state or search terms. The technique isn't secured so avoid using it with confidential data. In the next post in this series I'll explain the how to use cookies.

A few weeks ago I posted an article describing how my current team built a publish/subscribe message bus using WCF and MSMQ. At that time we had only deployed the application in a single-server test environment. While there were a few tricks to getting that working, once we tried deploying to a multiple server environment we found a whole lot of new challenges. In the end they were all quite solvable, but it seems that not a lot of people have attempted to use the MSMQ bindings for WCF, hosted in IIS 7 WAS, so there isn't a lot of help out on the web. The best source of information I'd found is an MSDN article, Troubleshooting Queued Messaging (which unfortunately I didn't find until after we'd already solved most of our problems). But even that article is a bit lacking, so I thought I'd share some of the things we learned about getting this all working. The Scenario The goal here is to set up reliable, asynchronous communication between a client application and a service, which may be on different machines. We will be using MSMQ as a transport mechanism, as it supports reliable queued communication. MSMQ will be deployed on a third server (typically clustered to eliminate a single point of failure). The client application will use WCF's NetMsmqBinding to send messages to a private queue on the MSMQ server. The service will be hosted in IIS 7, and will use Windows Activation Services (WAS) to listen for new messages on the message queue. This listening is done by a Windows Service called SMSvcHost.exe. When a message arrives, it activates the service within an IIS worker process, and the service will process the message. The overall architecture is shown in the following diagram. [img_assist|nid=4105|title=|desc=|link=none|align=none|width=615|height=310] The Basics Let's start simple by setting everything up on a single server, with no security or transactions to complicate things. This first instalment is a bit of a recap of my earlier post, but I'm including it again here as it will be an important foundation for the more complex steps shown in the next instalments. Install the necessary Windows components Before writing any code, make sure you're running Windows Vista or Windows Server 2008, and that you've installed the following components (I've taken the names from Vista's "Windows Features" dialog; Windows Server 2008 has slightly different options but all should be there somewhere). Microsoft Message Queue (MSMQ) Server > MSMQ Server Core and MSMQ Active Directory Domain Services Integration (needed for Transport Security in Part 2) Microsoft .NET Framework 3.0 > Windows Communication Foundation Non-HTTP Activation Internet Information Services > World Wide Web Services Windows Process Activation Service Distributed Transaction Controller (DTC) - Always installed with Windows Vista, may need to be added for Windows Server 2008 Of course, you'll also want Visual Studio 2005 or 2008 installed so you can write the necessary code. Define the contract As with all WCF applications, a great starting point is to define the service contract. The only real trick when building MSMQ services is to ensure that every operation contract is defined with IsOneWay=true. In my example we'll have just one very simple operation, but you could easily add more or use more complicated data contracts. [ServiceContract] public interface IMsmqContract { [OperationContract(IsOneWay = true)] void SendMessage(string message); } I won't bother with showing any sample client code to call the service, as this is no different from any other WCF client. Create the Message Queue Message Queues don't just create themselves, so if you want to build a MSMQ-based application, you'll need to create yourself some queues. The easiest way to do this is from the Computer Management console in Windows Vista, or Server Manager in Windows Server 2008. In general, message queues can be called whatever you want. However when you are hosting your MSMQ-enabled service in IIS 7 WAS, the queue name must match the URI of your service's .svc file. In this example we'll be hosting the service in an application called MsmqService with an .svc file called MsmqService.svc, so the queue must be called MsmqService/MsmqService.svc. Queues used for WCF services should always be private. While the term "private queue" could imply that the queue cannot be accessed from external machines, this isn't actually true - the only thing that makes a public queue public is that it is published in Active Directory. Since all of our queue paths will be coded into WCF config files, there really isn't any value in publishing the queues to AD. In this first stage, we won't be using a transactional queue, so make sure you don't click the Transactional checkbox. Transactional queues can add some complexity, but they also provide significantly more reliability so we'll be moving to transactional queues later in the article. At this time, it's a good idea to configure the security for the queue. You want to make sure that the account running the client is allowed to send messages to the queue, and the account running the service is able to receive messages from the queue. Since the service will be hosted in IIS, by default it will be using the NETWORK SERVICE account. Configure the Client Now we know the name of the message queue, we can configure the client to send messages to the correct place. First you need to configure a suitable binding. We'll be using the NetMsmqBinding, which is normally the best option when both the client and service are using WCF. For now we will not be using and security or transactions, so we'll need to specify that in the binding (the exactlyOnce="false" attribute means it's non-transactional). The endpoint definition is defined in the same way as any WCF client endpoint. One thing to look out for is the address syntax for MSMQ services. Rather than using the format name syntax that you may have used in other MSMQ applications, WCF has a new (and simpler) syntax. The key differences are that all slashes go forwards, and you use "private" instead of "private$". So the address for our locally hosted queue will be net.msmq://localhost/private/MsmqService/MsmqService.svc. Here's the complete config file for the client: Configure the Service To create the service, start by setting up a new ASP.NET application, hosted in IIS - just as you would for a normal HTTP-based WCF service. This includes creating a .svc file for the service endpoint, and of course a class that implements the service contract. Again, I won't bother showing this code as it's not specific to MSMQ. You'll also need to modify the service's web.config file to include the configuration details for your WCF service. Not surprisingly, this will look very similar to what we configured on the client. Enable the MSMQ WAS Listener The last step is to configure IIS 7 to use WAS to listen to the message queue and activate your service when new messages arrive. There are two parts to this: first you need to activate the net.msmq listener for the entire web site, and second you need to enable the protocols needed for your specific application. You can perform both of these steps using either the appcmd.exe tool (located under C:\Windows\System32\Inetsrv) or by editing the C:\Windows\System32\Inetsrv\config\ApplicationHost.config file in a text editor. Let's go with the former, since it's a bit less dangerous. To enable the net.msmq listener for the entire web site, use the following command. Note that the bindingInformation='localhost' bit is what tells the listener which machine is hosting the queues that it should listen to. This will be important when we want to start listening to remote queues. appcmd set site "Default Web Site" -+bindings.[protocol='net.msmq',bindingInformation='localhost'] To enable the net.msmq protocol for our specific application, use the following command. Note that you can configure multiple protocols for a single application, should you want it to be activated in more than one way (for example, to allow either MSMQ or HTTP you could say /enabledProtocols:net.msmq,http). appcmd set app "Default Web Site/MsmqService" /enabledProtocols:net.msmq Troubleshooting Steps If all has gone to plan, you should be able to successfully send messages from the client to the service, and have the service process them correctly. However if you're anything like me, this probably won't work first time. Troubleshooting MSMQ issues can be somewhat of an art form, but I've listed a few techniques that I've found to be helpful to resolve issues. Check queue permissions. Make sure that you've correctly set the ACLs on your message queue so that the user accounts running the client and service are able to send and receive respectively. Check the dead letter queues. In many circumstances, MSMQ will send messages to the Dead Letter Queue (or Transactional Dead Letter Queue) if it couldn't successfully be delivered for any reason. Often the details on the dead letter message will explain why it ended up there (for example, you tried sending a non-transactional message to a transactional queue). If you're using MSMQ across multiple machines, make sure you check the Dead Letter Queues on all servers, as it could end up in different places depending on what caused the delivery failure. Enable Journaling. Sometimes it can be hard to tell whether a message never arrived at all, or if it arrived and subsequently got "lost". If you enable the "journal" feature on a message queue, you'll see a record of every message that passed through. However use this feature sparingly, as you can very easily end up with a huge number of journal messages after a few hours of testing. Shut down the service listener. When troubleshooting, it can be useful to focus on just the client or just the service. For example, if you aren't sure if the client is sending messages properly, you may want to completely disable the service so you can see if the client's messages are arriving on the queue. To do this, you can shut down the IIS service or application pool, or shut down the Net.Msmq Listener Adapter Service. Make sure the MSMQ storage isn't maxed out. MSMQ is designed to be resistant against all sorts of failures, such as temporary network outages. However it seems that if you reach the limit of MSMQ's allocated storage, messages will not be delivered at all. This has happened to us a few times after large amounts of messages ended up in the Dead Letter Queue, or when journaling has been left on for too long. It's easy enough to increase the storage limit, but normally when you reach the limit during testing the best thing to do is purge all of your queues. Try pinging the service using the browser. When you are working on WCF services exposed through HTTP, you're probably used to hitting the .svc file in a web browser to check that you can receive the metadata correctly and that there are no configuration problems. Unfortunately there isn't any equivalent way to "browse" to an MSMQ service, so simple configuration errors can be very hard to track down. However if you enable the HTTP protocol for your site, you will be able to hit the .svc file in the browser, even if you haven't configured an HTTP endpoint for your service. If you get the standard WCF service page, that means the service is probably configured correctly. That's it for the basics. In Part 2 of this article, we'll look at what's required to get this application deployed on multiple servers, and specifically focus on what you'll need to do for the security configuration.

Caching may address new challenges when developing performing applications.

Garbage collection in Java is great, but in some cases you want to listen when an object is garbage collected. In my case I had the following scenario: I have a throw away key (a composition of objects) and a value (statistics) in some map. When one of the components of the key is garbage collected, the key and value should be removed from the map. The java.util.WeakHashMap was the first thing that came to mind, but there are some problems: It is not thread safe. And wrapping it in a synchronized block is not acceptable for scalability reasons. The key (a WeakReference containing the real key) is removed when the real key has been garbage collected. But in my case this isn’t going to work. I don’t want my throw-away key to be removed when it is garbage collected because it would be garbage collected immediately since nobody but the WeakReference is holding a reference to it. So I needed to go a step deeper. How can I listen to the garbage collection on an object? This can be done using a java.lang.ref.Reference (java.lang.ref.WeakReference for example) and a java.lang.ref.ReferenceQueue. When the object inside the WeakReference is garbage collected, the WeakReference is put on a ReferenceQueue. This makes it possible to do some cleanup by taking items from this ReferenceQueue. This is the first big step to listen to garbage collection. The other two constraints are now easy to solve: thread safety: use a ConcurrentHashMap implementation listen to an arbitrary object instead of the key: wrap the object inside a WeakReference and store the key & map inside. If you create a thread that consumes these references from the ReferenceQueue, this thread is now able to remove the key (and value) from the map This weekend I created a proof of concept implementation and it can be found bellow. public class GarbageCollectingConcurrentMap { private final static ReferenceQueue referenceQueue = new ReferenceQueue(); static { new CleanupThread().start(); } private final ConcurrentMap> map = new ConcurrentHashMap>(); public void clear() { map.clear(); } public V get(K key) { GarbageReference ref = map.get(key); return ref == null ? null : ref.value; } public Object getGarbageObject(K key){ GarbageReference ref=map.get(key); return ref == null ? null : ref.get(); } public Collection keySet() { return map.keySet(); } public void put(K key, V value, Object garbageObject) { if (key == null || value == null || garbageObject == null) throw new NullPointerException(); if (key == garbageObject) throw new IllegalArgumentException("key can't be equal to garbageObject for gc to work"); if (value == garbageObject) throw new IllegalArgumentException("value can't be equal to garbageObject for gc to work"); GarbageReference reference = new GarbageReference(garbageObject, key, value, map); map.put(key, reference); } static class GarbageReference extends WeakReference { final K key; final V value; final ConcurrentMap map; GarbageReference(Object referent, K key, V value, ConcurrentMap map) { super(referent, referenceQueue); this.key = key; this.value = value; this.map = map; } } static class CleanupThread extends Thread { CleanupThread() { setPriority(Thread.MAX_PRIORITY); setName("GarbageCollectingConcurrentMap-cleanupthread"); setDaemon(true); } public void run() { while (true) { try { GarbageReference ref = (GarbageReference) referenceQueue.remove(); while (true) { ref.map.remove(ref.key); ref = (GarbageReference) referenceQueue.remove(); } } catch (InterruptedException e) { //ignore } } } } } I am now able to run my concurrency detector on large projects like JBoss instead of running out of memory.

The code-base that I am currently working with consists of a large set of binaries that are all signed. The savvy .NET devs out there will know that any assembly that’s used/referenced by a signed assembly must also be signed. This is an issue when dealing with third-party libraries that are not signed. Sometimes you’ll be lucky enough to be dealing with vendor that is happy to provide a set of signed assemblies, other times you won’t. If your scenario fits the latter (as a recent one did for my colleagues and I), you need to sign the assemblies yourself. Here’s how. Note: delay signing is not covered in this article. Scenario 1 - Foo and Bar Foo is the component that you’re building which has to be signed. Bar is the third-party component that you’re forced to use that isn’t. [img_assist|nid=4058|title=|desc=|link=none|align=none|width=412|height=170] Grab Bar.dll and project along with Foo.dll and project to see a source sample. You’ll notice Foo has a .snk which is used to sign Foo.dll. When you attempt to compile Foo you get the following error message: Assembly generation failed — Referenced assembly ‘Bar’ does not have a strong name We need to sign Bar in order for Foo to compile. [img_assist|nid=4059|title=|desc=|link=none|align=right|width=161|height=145]Step 1 - Disassemble Bar We need to open a command prompt which has the .NET framework binaries in the PATH environment variable. The easiest way to do this is to open a Visual Studio command prompt (which is usually under the “Visual Studio Tools” subfolder of “Visual Studio 200X” in your programs menu). Change directory so that you’re in the folder which contains Bar.dll. Use ildasm.exe to disassemble the file using the /all and /out, like so: C:\Foo\bin> ildasm /all /out=Bar.il Bar.dll The result of the command is a new file, Bar.il, which contains a dissassembled listing of Bar.dll. [img_assist|nid=4060|title=|desc=|link=none|align=right|width=129|height=145]Step 2 - Rebuild and Sign Bar We can now use ilasm to reassemble Bar.il back into Bar.dll, but at the same time specify a strong-name key to use to sign the resulting assembly. We pass in the value Foo.snk to the /key switch on the command line, like so: C:\Foo\bin> ilasm /dll /key=Foo.snk Bar.il Microsoft (R) .NET Framework IL Assembler. Version 2.0.50727.1434 Copyright (c) Microsoft Corporation. All rights reserved. Assembling 'Bar.il' to DLL --> 'Bar.dll' Source file is ANSI Assembled method Bar.Bar::get_SecretMessage Assembled method Bar.Bar::.ctor Creating PE file Emitting classes: Class 1: Bar.Bar Emitting fields and methods: Global Class 1 Methods: 2; Resolving local member refs: 1 -> 1 defs, 0 refs, 0 unresolved Emitting events and properties: Global Class 1 Props: 1; Resolving local member refs: 0 -> 0 defs, 0 refs, 0 unresolved Writing PE file Signing file with strong name Operation completed successfully Bar.dll is now signed! All we have to do is reopen Foo’s project, remove the reference to Bar.dll, re-add the reference to the new signed assembly and rebuild. Sorted! Scenario 2 - Foo, Bar and Baz Foo is the component that you’re building which has to be signed. Bar is the third-party component that you’re forced to use that isn’t. Baz is another third-party component that is required in order for you to use Bar. [img_assist|nid=4061|title=|desc=|link=none|align=none|width=640|height=177] Grab Baz.dll and project, Bar.dll and project along with Foo.dll and project for a sample source. When you attempt to build Foo you get the same error as you do in the previous scenario. Bear in mind that this time, both Bar.dll and Baz.dll need to be signed. So first of all, follow the steps in Scenario 1 for both Bar.dll and Baz.dll. Done? OK. When you attempt to build Foo.dll after pointing the project at the new Bar.dll no compiler errors will be shown. Don’t get too excited. When you attempt to use Foo.dll your world will come crashing down. The reason is because Bar.dll was originally built with a reference to an unsigned version of Baz.dll. Now that Baz.dll is signed we need to force Bar.dll to reference the signed version of Baz.dll. [img_assist|nid=4062|title=|desc=|link=none|align=right|width=147|height=154]Step 1 - Hack the Disassembled IL Just like we did in the previous steps we need to disassemble the binary that we need to fix. This time, make sure you disassemble the new binary that you created in the previous step (this binary has been signed, and will contain the signature block for the strong name). Once Bar.il has been created using ildasm, open it up in a text editor. Search for the reference to Baz — this should be located a fair way down the file, somewhere near the top of the actual code listing, just after the comments. Here’s what it looks like on my machine: .assembly extern /*23000002*/ Baz { .ver 1:0:0:0 } This external assembly reference is missing the all-important public key token reference. Before we can add it, we need to know what the public key token is for Bar.dll. To determine this, we can use the sn.exe utility, like so: C:\Foo\bin> sn -Tp Baz.dll Microsoft (R) .NET Framework Strong Name Utility Version 3.5.21022.8 Copyright (c) Microsoft Corporation. All rights reserved. Public key is 0024000004800000940000000602000000240000525341310004000001000100a59cd85e10658d 9229d54de16c69d0b53b31f60bb4404b86eb3b8804203aca9d65412a249dfb8e7b9869d09ce80b 0d9bdccd4943c0004c4e76b95fdcdbc6043765f51a1ee331fdd55ad25400d496808b792723fc76 dee74d3db67403572cddd530cadfa7fbdd974cef7700be93c00c81121d978a3398b07a9dc1077f b331ca9c Public key token is 2ed7bbec811020ec Now we return to Bar.il and modify the assembly reference so that the public key token is specified. This is what it should look like after modification: .assembly extern /*23000002*/ Baz { .publickeytoken = (2E D7 BB EC 81 10 20 EC ) .ver 1:0:0:0 } Save your changes. [img_assist|nid=4064|title=|desc=|link=none|align=right|width=131|height=155]Step 2 - Reassemble Bar This step is just a repeat of previous steps. We are again using ilasm to reassemble Bar.dll, but this time from the new “hacked” Bar.il file. We must use the exact same command line as we did previously, and we still need to specify the Foo.snk for signing the assembly. To save you having to scroll up, here it is again: C:\Foo\bin> ilasm /dll /key=Foo.snk Bar.il Microsoft (R) .NET Framework IL Assembler. Version 2.0.50727.1434 Copyright (c) Microsoft Corporation. All rights reserved. Assembling 'Bar.il' to DLL --> 'Bar.dll' Source file is ANSI Assembled method Bar.Bar::get_SecretMessage Assembled method Bar.Bar::.ctor Creating PE file Emitting classes: Class 1: Bar.Bar Emitting fields and methods: Global Class 1 Fields: 1; Methods: 2; Resolving local member refs: 3 -> 3 defs, 0 refs, 0 unresolved Emitting events and properties: Global Class 1 Props: 1; Resolving local member refs: 0 -> 0 defs, 0 refs, 0 unresolved Writing PE file Signing file with strong name Operation completed successfully Open up Foo’s project, remove and re-add the reference to Bar.dll, making sure you point to the new version that you just created. Foo.dll will not only build, but this time it will run! Disclaimer “Hacking” third-party binaries in this manner may breach the license agreement of those binaries. Please make sure that you are not breaking the license agreement before adopting this technique. Original Author Original article written by Oliver Reeves

Returns the number of line breaks in a string. function lineBreakCount(str){ /* counts \n */ try { return((str.match(/[^\n]*\n[^\n]*/gi).length)); } catch(e) { return 0; } }

[img_assist|nid=3421|title=|desc=|link=url|url=http://www.manning.com/affiliate/idevaffiliate.php?id|align=left|width=208|height=388]The core of every GWT project is the project layout and the basic components required—host pages, entry points, and modules. To begin a GWT project, you need to create the default layout and generate the initial files. The easiest way to do this is to use the provided ApplicationCreator tool. Generating a project ApplicationCreator is provided by GWT to create the default starting points and layout for a GWT project. ApplicationCreator, like the GWT shell, supports several command-line parameters, which are listed in table 1. ApplicationCreator [-eclipse projectName] [-out dir] [-overwrite] [-ignore] className Table 1 ApplicationCreator command-line parameters Parameter Description -eclipse Creates a debug launch configuration for the named eclipse project -out The directory to which output files will be written (defaults to the current directory) -overwrite Overwrites any existing files -ignore Ignores any existing files; does not overwrite className The fully qualified name of the application class to be created To stub out an example calculator project, we’ll use ApplicationCreator based on a relative GWT_HOME path, and a className of com.manning.gwtip.calculator.client.Calculator, as follows: mkdir [PROJECT_HOME] cd [PROJECT_HOME] [GWT_HOME]/applicationCreator com.manning.gwtip.calculator.client.Calculator GWT_HOME It is recommended that you establish GWT_HOME as an environment variable referring to the filesystem location where you have unpacked GWT. Additionally, you may want to add GWT_HOME to your PATH for further convenience. We use GWT_HOME when referencing the location where GWT is installed and PROJECT_HOME to refer to the location of the current project. PATH SEPARATORS For convenience, when referring to filesystem paths, we'll use forward slashes, which work for two-thirds of supported GWT platforms. If you are using Windows, please adjust the path separators to use backward slashes. Running ApplicationCreator as described creates the default src directory structure and the starting-point GWT file resources. The standard directory structure Even though it's quite simple, the GWT layout is very important because the toolkit can operate in keeping with a Convention over Configuration design approach. As we’ll see, several parts of the GWT compilation process make assumptions about the default layout. Because of this, not everything has to be explicitly defined in every instance (which cuts down on the amount of configuration required). Taking a look at the output of the ApplicationCreator script execution, you will see a specific structure and related contents, as shown in listing 1. This represents the default configuration for a GWT project. Listing 1 ApplicationCreator output, showing the default GWT project structure: src src/com src/com/manning src/com/manning/gwtip src/com/manning/gwtip/calculator src/com/manning/gwtip/calculator/Calculator.gwt.xml src/com/manning/gwtip/calculator/client src/com/manning/gwtip/calculator/client/Calculator.java src/com/manning/gwtip/calculator/public src/com/manning/gwtip/calculator/public/Calculator.html Calculator-shell.sh Calculator-compile.sh The package name, com.manning.gwtip.calculator, is represented in the structure as a series of subdirectories in the src tree. This is the standard Java convention, and there are notably separate client and public subdirectories within. The client directory is intended for resources that will be compiled into JavaScript . Client items are translatable, or serializable, and will ultimately be downloaded to a client browser—these are Java resources in the source. The client package is known in GWT terminology as the source path. The public directory denotes files that will also be distributed to the client, but that do not require compilation and translation to JavaScript . This typically includes CSS, images, static HTML, and any other such assets that should not be translated, including existing JavaScript. The public package is known as the public path. Note that our client-side example does not use any server resources, but GWT does include the concept of a server path/package for server-side resources. Figure 1 illustrates this default GWT project layout. [img_assist|nid=4037|title=|desc=|link=none|align=none|width=293|height=284] ApplicationCreator generates the structure and a required set of minimal files for a GWT project. The generated files include the XML configuration module definition, the entry point Java class, and the HTML host page. These are some of the basic GWT project concepts. Along with the module definition, entry point, and host page, some shortcut scripts have also been created for use with the GWTShell and GWTCompiler tools. These scripts run the shell and compiler for the project. Table 2 lists all of the files created by ApplicationCreator: the basic resources and shortcut scripts needed for a GWT project. Table 2 ApplicationCreator-generated initial project files that serve as a starting point for GWT applications File Name Purpose GWT module file ProjectName.gwt.xml Defines the project configuration Entry point class ProjectName.java Starting class invoked by the module Host page ProjectName.html Initial HTML page that loads the module GWTShell shortcut invoker script ProjectName-shell.sh Invokes GWTShell for the project GWTCompiler shortcut invoker script ProjectName-compile.sh Invokes GWTCompiler for the project The starting points ApplicationCreator provides essentially wire up all the moving parts for you and stub out your project. You take it from there and modify these generated files to begin building a GWT application. If the toolkit did not provide these files via ApplicationCreator, getting a project started, at least initially, would be much more time consuming and confusing. Once you are experienced in the GWT ways, you may wind up using other tools to kick off a project: an IDE plugin, a Maven “archetype,” or your own scripts. ApplicationCreator, though, is the helpful default. The contents and structure that ApplicationCreator provides are themselves a working GWT “hello world” example. You get “hello world” for free, out of the box. "Hello world", however, is not that interesting. The connection of all the moving parts is what is really important; how a host page includes a module, how a module describes project resources, and how an entry point invokes project code. These concepts are applicable to all levels of GWT projects—the basic ones and beyond. Understanding these parts is key to gaining an overall understanding of GWT. Next, we’ll take a closer look at each of these concepts, beginning with the host page. Host pages A host page is the initial HTML page that invokes a GWT application. A host page contains a script tag that references a special GWT JavaScript file, Module.nocache.js. This JavaScript file, which the toolkit provides when you compile your project, kicks off the GWT application loading process. Along with the script reference that loads the project resources, you can also specify several GWT-related tags in the host page. These tag options are not present in the default host page created by ApplicationCreator, but it’s still important to be aware of them. The GWT tags that are supported in a host page are listed in table 3, as a reference. Table 3 GWT tags supported in host pages Meta tag Syntax Purpose gwt:module (Legacy, pre GWT 1.4.) Specifies the module to be loaded gwt:property Statically defines a deferred binding client property gwt:onPropertyErrorFn Specifies the name of a function to call if a client property is set to an invalid value (meaning that no matching compilation will be found) gwt:onLoadErrorFn Specifies the name of a function to call if an exception happens during bootstrapping or if a module throws an exception out of onModuleLoad(); the function should take a message parameter Thus, a host page includes a script reference that gets the GWT process started and refers to all the required project resources. The required resources for a project are assembled by the GWT compilation process, and are based on the module configuration. Modules GWT applications inhabit a challenging environment. This is partly because of the scope of responsibility GWT has elected to take on and partly because of the Internet landscape. Being a rich Internet-based platform and using only the basic built-in browser support for HTML, CSS, and JavaScript makes GWT quite elegant and impressive, but this combination is tough to achieve. Browsers that are “guided” by standards, but that don’t always stick to them, add to the pressure. Couple that environment with an approach that aims to bring static types, code standards, profiling and debugging, inheritance, and reuse to the web tier, and you have a tall order. To help with this large task, GWT uses modules as configuration and execution units that handle discreet areas of responsibility. Modules enable the GWT compiler to optimize the Java code it gets fed, create variants for all possible situations from a single code base, and make inheritance and property support possible. One of the most important resources generated by the ApplicationCreator is the Module.gwt.xml module descriptor for your project. This file exists in the top-level directory of your project’s package and provides a means to define resource locations and structure. In a default generated module file, there are only two elements: and . An element simply includes the configuration for another named GWT module in the current definition, and defines a class that kicks things off and moves from configuration to code. Table 4 provides an overview of the most common GWT module descriptor elements. Table 4 A summary of the most common elements supported by the GWT module descriptor Module element Description Identifies additional GWT modules that should be inherited into the current module Specifies which EntryPoint class should be invoked when starting a GWT project Identifies where the source code that should be translated into JavaScript by the GWT compiler is located Identifies where assets that are not translatable source code, such as images and CSS files, are located

Have been using IntelliJ for about 5 years now. It is still the Roll Royce of all IDEs, and of that there is no doubt. But having said that, you should once in a while jump the ship, and try out something different and it is with this intention that I am currently trying out NetBeans IDE 6.1. I was a NetBeans IDE user starting from its “Forte for Java” days to about 2003, and did also develop some plugins on it. So it feels great to be back home. Well not quite. Though NetBeans IDE has now added almost every trick in the bag, it is still challenging even for a veteran, to pick up from where they left off. To ease this pain of transition, here is the link to an IntelliJ Key Map, which when applied to NetBeans IDE, will make it feel more like IntelliJ to you. If you find yourself pressing CTRL+ALT+L, CTRL+ALT+O and CTRL+F12, this keymap will be a real time saver. From: IntelliJ Keymap for NetBeans