DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Latest Articles - DZone

article thumbnail
Caching With WCF Services
This is the first part of a two part article about caching in WCF services. In this part I will explain the in-process memory cache available in .NET 4.0. In the second part I will describe the Windows AppFabric distributed memory cache. The .NET framework has provided a cache for ASP.NET applications since version 1.0. For other types of applications like WPF applications or console application, caching was never possible out of the box. Only WCF services were able to use the ASP.NET cache if they were configured to run in ASP.NET compatibility mode. But this mode has some performance drawbacks and only works when the WCF service is hosted inside IIS and uses an HTTP-based binding. With the release of the .NET 4.0 framework this has luckily changed. Microsoft has now developed an in-process memory cache that does not rely on the ASP.NET framework. This cache can be found in the “System.Runtime.Caching.dll” assembly. In order to explain the working of the cache, I have a created a simple sample application. It consists of a very slow repository called “SlowRepository”. public class SlowRepository { public IEnumerable GetPizzas() { Thread.Sleep(10000); return new List() { "Hawaii", "Pepperoni", "Bolognaise" }; } } This repository is used by my sample WCF service to gets its data. public class PizzaService : IPizzaService { private const string CacheKey = "availablePizzas"; private SlowRepository repository; public PizzaService() { this.repository = new SlowRepository(); } public IEnumerable GetAvailablePizzas() { ObjectCache cache = MemoryCache.Default; if(cache.Contains(CacheKey)) return (IEnumerable)cache.Get(CacheKey); else { IEnumerable availablePizzas = repository.GetPizzas(); // Store data in the cache CacheItemPolicy cacheItemPolicy = new CacheItemPolicy(); cacheItemPolicy.AbsoluteExpiration = DateTime.Now.AddHours(1.0); cache.Add(CacheKey, availablePizzas, cacheItemPolicy); return availablePizzas; } } } When the WCF service method GetAvailablePizzas is called, the service first retrieves the default memory cache instance ObjectCache cache = MemoryCache.Default; Next, it checks if the data is already available in the cache. If so, the cached data is used. If not, the repository is called to get the data and afterwards the data is stored in the cache. For my sample service, I also choose to restrict the maximum memory to 20% of the total physical memory. This can be done in the web.config.
April 13, 2012
by Pieter De Rycke
· 22,055 Views · 1 Like
article thumbnail
Playing Sounds in Android
Let's take a closer look at how to play sounds on an Android device with SoundPool and MediaPlayer.
April 13, 2012
by Tony Siciliani
· 95,609 Views · 2 Likes
article thumbnail
Quartz Scheduler Misfire Instructions Explained
Sometimes Quartz is not capable of running your job at the time when you desired. There are three reasons for that: all worker threads were busy running other jobs (probably with higher priority) the scheduler itself was down the job was scheduled with start time in the past (probably a coding error) You can increase the number of worker threads by simply customizing the org.quartz.threadPool.threadCount in quartz.properties (default is 10). But you cannot really do anything when the whole application/server/scheduler was down. The situation when Quartz was incapable of firing given trigger is called misfire. Do you know what Quartz is doing when it happens? Turns out there are various strategies (called misfire instructions) Quartz can take and also there are some defaults if you haven't thought about it. But in order to make your application robust and predictable (especially under heavy load or maintenance) you should really make sure your triggers and jobs are configured conciously. There are different configuration options (available misfire instructions) depending on the trigger chosen. Also Quartz behaves differently depending on trigger setup (so called smart policy). Although the misfire instructions are described in the documentation, I found it hard to understand what do they really mean. So I created this small summary article. Before I dive into the details, there is yet another configuration option that should be described. It is org.quartz.jobStore.misfireThreshold (in milliseconds), defaulting to 60000 (a minute). It defines how late the trigger should be to be considered misfired. With default setup if trigger was suppose to be fired 30 seconds ago, Quartz will happily just run it. Such delay is not considered misfiring. However if the trigger is discovered 61 seconds after the scheduled time - the special misfire handler thread takes care of it, obeying the misfire instruction. For test purposes we will set this parameter to 1000 (1 second) so that we can test misfiring quickly. Simple trigger without repeating In our first example we will see how misfiring is handled by simple triggers scheduled to run only once: val trigger = newTrigger(). startAt(DateUtils.addSeconds(new Date(), -10)). build() The same trigger but with explicitly set misfire instruction handler: val trigger = newTrigger(). startAt(DateUtils.addSeconds(new Date(), -10)). withSchedule( simpleSchedule(). withMisfireHandlingInstructionFireNow() //MISFIRE_INSTRUCTION_FIRE_NOW ). build() For the purpose of testing I am simply scheduling the trigger to run 10 seconds ago (so it is 10 seconds late by the time it is created!) In real world you would normally never schedule triggers like that. Instead imagine the trigger was set correctly but by the time it was scheduled the scheduler was down or didn't have any free worker threads. Nevertheless, how will Quartz handle this extraordinary situation? In the first code snippet above no misfire handling instruction is set (so called smart policy is used in that case). The second code snippet explicitly defines what kind of behaviour do we expect when misfiring occurs. See the table: Instruction Meaning smart policy - default See: withMisfireHandlingInstructionFireNow withMisfireHandlingInstructionFireNow MISFIRE_INSTRUCTION_FIRE_NOW The job is executed immediately after the scheduler discovers misfire situation. This is the smart policy. Example scenario: you have scheduled some system clean up at 2 AM. Unfortunately the application was down due to maintenance by that time and brought back on 3 AM. So the trigger misfired and the scheduler tries to save the situation by running it as soon as it can - at 3 AM. withMisfireHandlingInstructionIgnoreMisfires MISFIRE_INSTRUCTION_IGNORE_MISFIRE_POLICY QTZ-283 See: withMisfireHandlingInstructionFireNow withMisfireHandlingInstructionNextWithExistingCount MISFIRE_INSTRUCTION_RESCHEDULE_NEXT_WITH_EXISTING_COUNT See: withMisfireHandlingInstructionNextWithRemainingCount withMisfireHandlingInstructionNextWithRemainingCount MISFIRE_INSTRUCTION_RESCHEDULE_NEXT_WITH_REMAINING_COUNT Does nothing, misfired execution is ignored and there is no next execution. Use this instruction when you want to completely discard the misfired execution. Example scenario: the trigger was suppose to start recording of a program in TV. There is no point of starting recording when the trigger misfired and is already 2 hours late. withMisfireHandlingInstructionNowWithExistingCount MISFIRE_INSTRUCTION_RESCHEDULE_NOW_WITH_EXISTING_REPEAT_COUNT See: withMisfireHandlingInstructionFireNow withMisfireHandlingInstructionNowWithRemainingCount MISFIRE_INSTRUCTION_RESCHEDULE_NOW_WITH_REMAINING_REPEAT_COUNT See: withMisfireHandlingInstructionFireNow Simple trigger repeating fixed number of times This scenario is much more complicated. Imagine we have scheduled some job to repeat fixed number of times: val trigger = newTrigger(). startAt(dateOf(9, 0, 0)). withSchedule( simpleSchedule(). withRepeatCount(7). withIntervalInHours(1). WithMisfireHandlingInstructionFireNow() //or other ). build() In this example the trigger is suppose to fire 8 times (first execution + 7 repetitions) every hour, beginning at 9 AM today (startAt(dateOf(9, 0, 0)). Thus the last execution should occur at 4 PM. However assume that due to some reason the scheduler was not capable of running jobs at 9 and 10 AM and it discovered that fact at 10:15 AM, i.e. 2 firings misfired. How will the scheduler behave in this situation? Instruction Meaning smart policy - default See: withMisfireHandlingInstructionNowWithExistingCount withMisfireHandlingInstructionFireNow MISFIRE_INSTRUCTION_FIRE_NOW See: withMisfireHandlingInstructionNowWithRemainingCount withMisfireHandlingInstructionIgnoreMisfires MISFIRE_INSTRUCTION_IGNORE_MISFIRE_POLICYQTZ-283 Fires all triggers that were missed as soon as possible and then goes back to ordinary schedule. Example scenario: With this strategy in our example the scheduler will fire jobs scheduled at 9 and 10 AM immediately. Then it will wait to 11 AM and go back to ordinary schedule. Note: When handling misfires it is equally important to realize that the actual job execution time might be way after the scheduled time. This means you cannot simply rely on current system date, but you need to use JobExecutionContext .getScheduledFireTime(): def execute(context: JobExecutionContext) { val date = context.getScheduledFireTime //... } withMisfireHandlingInstructionNextWithExistingCount MISFIRE_INSTRUCTION_RESCHEDULE_NEXT_WITH_EXISTING_COUNT The scheduler won't do anything immediately. Instead it will wait for next scheduled time and run all triggers with scheduled intervals. See also: withMisfireHandlingInstructionNextWithRemainingCount Example scenario: at 10:15 the scheduler discovers 2 misfired executions. It waits until next scheduled time (11 AM) and fires all 8 scheduled executions every hour, stopping at 6 PM (the trigger should have stopped at 4 PM). withMisfireHandlingInstructionNextWithRemainingCount MISFIRE_INSTRUCTION_RESCHEDULE_NEXT_WITH_REMAINING_COUNT The scheduler discards misfired executions and waits for the next scheduled time. The total number of trigger executions will be less then configured. Example scenario: at 10:15 two misfired executions are discarded. The scheduler waits for next scheduled time (11 AM) and fires remaining triggers up to 4 PM. Effectively it behaves as if misfire never occurred. withMisfireHandlingInstructionNowWithExistingCount MISFIRE_INSTRUCTION_RESCHEDULE_NOW_WITH_EXISTING_REPEAT_COUNT First misfired trigger is executed immediately. Then the scheduler waits desired interval and executes all remaining triggers. Effectively the first fire time of the misfired trigger is moved to current time with no other changes. Example scenario: at 10:15 the scheduler runs the first misfired execution. Then it waits 1 hour and fires the second one at 11:15 AM. All 8 executions are performed, the last one at 5:15 PM withMisfireHandlingInstructionNowWithRemainingCount MISFIRE_INSTRUCTION_RESCHEDULE_NOW_WITH_REMAINING_REPEAT_COUNT First misfired execution runs immediately. Remaining misfired executions are discarded. Triggers that were not misfired are executed with desired interval. Example scenario: at 10:15 the scheduler runs the first misfired execution (from 9 AM). It discards remaining misfired executions (the one from 10 AM) and waits 1 hour to execute six more triggers: 11:15, 12:15, … 4:15 PM Simple trigger repeating infinitely In this scenario trigger repeats infinite number of times at a given interval: val trigger = newTrigger(). startAt(dateOf(9, 0, 0)). withSchedule( simpleSchedule(). withRepeatCount(SimpleTrigger.REPEAT_INDEFINITELY). withIntervalInHours(1). WithMisfireHandlingInstructionFireNow() //or other ). build() Once again trigger should fire on every hour, beginning at 9 AM today (startAt(dateOf(9, 0, 0)). However the scheduler was not capable of running jobs at 9 and 10 AM and it discovered that fact at 10:15 AM, i.e. 2 firings misfired. This is a more general situation compared to simple trigger running fixed number of times. Instruction Meaning smart policy - default See: withMisfireHandlingInstructionNextWithRemainingCount withMisfireHandlingInstructionFireNow MISFIRE_INSTRUCTION_FIRE_NOW See: withMisfireHandlingInstructionNowWithRemainingCount withMisfireHandlingInstructionIgnoreMisfires MISFIRE_INSTRUCTION_IGNORE_MISFIRE_POLICYQTZ-283 The scheduler will immediately run all misfired triggers, then continue on schedule. Example scenario: the triggers scheduled at 9 and 10 AM are executed immediately. Future invocations (next scheduled at 11 AM) are executed according to the plan. withMisfireHandlingInstructionNextWithExistingCount MISFIRE_INSTRUCTION_RESCHEDULE_NEXT_WITH_EXISTING_COUNT See: withMisfireHandlingInstructionNextWithRemainingCount withMisfireHandlingInstructionNextWithRemainingCount MISFIRE_INSTRUCTION_RESCHEDULE_NEXT_WITH_REMAINING_COUNT Does nothing, misfired executions are discarded. Then the scheduler waits for next scheduled interval and goes back to schedule. Example scenario: Misfired execution at 9 and 10 AM are discarded. The first execution occurs at 11 AM. withMisfireHandlingInstructionNowWithExistingCount MISFIRE_INSTRUCTION_RESCHEDULE_NOW_WITH_EXISTING_REPEAT_COUNT See: withMisfireHandlingInstructionNowWithRemainingCount withMisfireHandlingInstructionNowWithRemainingCount MISFIRE_INSTRUCTION_RESCHEDULE_NOW_WITH_REMAINING_REPEAT_COUNT The first misfired execution is run immediately, remaining are discarded. Next execution happens after desired interval. Effectively the first execution time is moved to current time. Example scenario: the scheduler fires misfired trigger immediately at 10:15 AM. Then waits an hour and runs the second one at 11:15 AM and continues with 1 hour interval. CRON triggers CRON triggers are the most popular ones amongst Quartz users. However there are also two other available triggers: DailyTimeIntervalTrigger (e.g. fire every 25 minutes) and CalendarIntervalTrigger (e.g. fire every 5 months). They support triggering policies not possible in both CRON and simple triggers. However they understand the same misfire handling instructions as CRON trigger. val trigger = newTrigger(). withSchedule( cronSchedule("0 0 9-17 ? * MON-FRI"). withMisfireHandlingInstructionFireAndProceed() //or other ). build() In this example the trigger should fire every hour between 9 AM and 5 PM, from Monday to Friday. But once again first two invocations were missed (so the trigger misfired) and this situation was discovered at 10:15 AM. Note that available misfire instructions are different compared to simple triggers: Instruction Meaning smart policy - default See: withMisfireHandlingInstructionFireAndProceed withMisfireHandlingInstructionIgnoreMisfires MISFIRE_INSTRUCTION_IGNORE_MISFIRE_POLICYQTZ-283 All misfired executions are immediately executed, then the trigger runs back on schedule. Example scenario: the executions scheduled at 9 and 10 AM are executed immediately. The next scheduled execution (at 11 AM) runs on time. withMisfireHandlingInstructionFireAndProceed MISFIRE_INSTRUCTION_FIRE_ONCE_NOW Immediately executes first misfired execution and discards other (i.e. all misfired executions are merged together). Then back to schedule. No matter how many trigger executions were missed, only single immediate execution is performed. Example scenario: the executions scheduled at 9 and 10 AM are merged and executed only once (in other words: the execution scheduled at 10 AM is discarded). The next scheduled execution (at 11 AM) runs on time. withMisfireHandlingInstructionDoNothing MISFIRE_INSTRUCTION_DO_NOTHING All misfired executions are discarded, the scheduler simply waits for next scheduled time. Example scenario: the executions scheduled at 9 and 10 AM are discarded, so basically nothing happens. The next scheduled execution (at 11 AM) runs on time. QTZ-283Note: QTZ-283: MISFIRE_INSTRUCTION_IGNORE_MISFIRE_POLICY not working with JDBCJobStore - apparently there is a bug when JDBCJobStore is used, keep an eye on that issue. As you can see various triggers behave differently based on the actual setup. Moreover, even though the so called smart policy is provided, often the decision is based on business requirements. Essentially there are three major strategies: ignore, run immediately and continue and discard and wait for next. They all have different use-cases: Use ignore policies when you want to make sure all scheduled executions were triggered, even if it means multiple misfired triggers will fire. Think about a job that generates report every hour based on orders placed during that last hour. If the server was down for 8 hours, you still want to have that reports generated, as soon as you can. In this case the ignore policies will simply run all triggers scheduled during that 8 hour as fast as scheduler can. They will be several hours late, but will eventually be executed. Use now* policies when there are jobs executing periodically and upon misfire situation they should run as soon as possible, but only once. Think of a job that cleans /tmp directory every minute. If the scheduler was busy for 20 minutes and finally can run this job, you don't want to run in 20 times! One is enough, but make sure it runs as fast it can. Then back to your normal one-minute intervals. Finally next* policies are good when you want to make sure your job runs at particular points in time. For example you need to fetch stock prices quarter past every hour. They change rapidly so if your job misfired and it is already 20 minutes past full hour, don't bother. You missed the correct time by 5 minutes and now you don't really care. It is better to have a gap rather than an inaccurate value. In this case Quartz will skip all misfired executions and simply wait for the next one.
April 13, 2012
by Tomasz Nurkiewicz
· 109,537 Views · 13 Likes
article thumbnail
How to Use Sigma.js with Neo4j
i’ve done a few posts recently using d3.js and now i want to show you how to use two other great javascript libraries to visualize your graphs. we’ll start with sigma.js and soon i’ll do another post with three.js . we’re going to create our graph and group our nodes into five clusters. you’ll notice later on that we’re going to give our clustered nodes colors using rgb values so we’ll be able to see them move around until they find their right place in our layout. we’ll be using two sigma.js plugins, the gefx (graph exchange xml format) parser and the forceatlas2 layout. you can see what a gefx file looks like below. notice it comes from gephi which is an interactive visualization and exploration platform, which runs on all major operating systems, is open source, and is free. ... ... in order to build this file, we will need to get the nodes and edges from the graph and create an xml file. get '/graph.xml' do @nodes = nodes @edges = edges builder :graph end we’ll use cypher to get our nodes and edges: def nodes neo = neography::rest.new cypher_query = " start node = node:nodes_index(type='user')" cypher_query << " return id(node), node" neo.execute_query(cypher_query)["data"].collect{|n| {"id" => n[0]}.merge(n[1]["data"])} end we need the node and relationship ids, so notice i’m using the id() function in both cases. def edges neo = neography::rest.new cypher_query = " start source = node:nodes_index(type='user')" cypher_query << " match source -[rel]-> target" cypher_query << " return id(rel), id(source), id(target)" neo.execute_query(cypher_query)["data"].collect{|n| {"id" => n[0], "source" => n[1], "target" => n[2]} } end so far we have seen graphs represented as json, and we’ve built these manually. today we’ll take advantage of the builder ruby gem to build our graph in xml. xml.instruct! :xml xml.gexf 'xmlns' => "http://www.gephi.org/gexf", 'xmlns:viz' => "http://www.gephi.org/gexf/viz" do xml.graph 'defaultedgetype' => "directed", 'idtype' => "string", 'type' => "static" do xml.nodes :count => @nodes.size do @nodes.each do |n| xml.node :id => n["id"], :label => n["name"] do xml.tag!("viz:size", :value => n["size"]) xml.tag!("viz:color", :b => n["b"], :g => n["g"], :r => n["r"]) xml.tag!("viz:position", :x => n["x"], :y => n["y"]) end end end xml.edges :count => @edges.size do @edges.each do |e| xml.edge:id => e["id"], :source => e["source"], :target => e["target"] end end end end you can get the code on github as usual and see it running live on heroku. you will want to see it live on heroku so you can see the nodes in random positions and then move to form clusters. use your mouse wheel to zoom in, and click and drag to move around. credit goes out to alexis jacomy and mathieu jacomy . you’ve seen me create numerous random graphs, but for completeness here is the code for this graph. notice how i create 5 clusters and for each node i assign half its relationships to other nodes in their cluster and half to random nodes? this is so the forceatlas2 layout plugin clusters our nodes neatly. def create_graph neo = neography::rest.new graph_exists = neo.get_node_properties(1) return if graph_exists && graph_exists['name'] names = 500.times.collect{|x| generate_text} clusters = 5.times.collect{|x| {:r => rand(256), :g => rand(256), :b => rand(256)} } commands = [] names.each_index do |n| cluster = clusters[n % clusters.size] commands << [:create_node, {:name => names[n], :size => 5.0 + rand(20.0), :r => cluster[:r], :g => cluster[:g], :b => cluster[:b], :x => rand(600) - 300, :y => rand(150) - 150 }] end names.each_index do |from| commands << [:add_node_to_index, "nodes_index", "type", "user", "{#{from}"] connected = [] # create clustered relationships members = 20.times.collect{|x| x * 10 + (from % clusters.size)} members.delete(from) rels = 3 rels.times do |x| to = members[x] connected << to commands << [:create_relationship, "follows", "{#{from}", "{#{to}"] unless to == from end # create random relationships rels = 3 rels.times do |x| to = rand(names.size) commands << [:create_relationship, "follows", "{#{from}", "{#{to}"] unless (to == from) || connected.include?(to) end end batch_result = neo.batch *commands end
April 12, 2012
by Max De Marzi
· 15,495 Views
article thumbnail
F1 Live Timing Map
this is a live timing map application for f1 championship races made using javascript and google maps markers. the live timing data is supplied by formula1.com. it’s interactive, you can press over a driver to track him or press into an empty map zone to untrack and have a general view. it has also been made with a responsive design to adapt it to mobile browsers using jquerymobile framework. how it works: the client side: until the race start date a countdown and a demo race is showed. when the countdown finishes it will connect to server (using ajax) to get the live timing data from server (every five seconds) and the interface will be updated using this data. the server side: it uses a django app for the web page and the static race data (circuit, laps, drivers) is put into the html using the django template system. for the dynamic data (live timing) i have modified the source of a c program for the linux terminal called live-f1 to generate a json with the data that the client requires instead of printing it on terminal screen. enjoy the race!
April 12, 2012
by Luis Sobrecueva
· 16,134 Views
article thumbnail
Secret Powers of foldLeft() in Scala
The foldLeft() method, available for all collections in Scala, allows a given 2-argument function to run against consecutive elements of that collection, where the result of that function is passed as the first argument in the next invocation. Second argument is always the current item in the collection. Doesn't sound very encouraging but as we will see soon there are great some use-cases waiting to be discovered. Before we dive into foldLeft, let us have a look at reduce - simplified version of foldLeft. I always believed that a working code is worth a thousand words: val input = List(3, 5, 7, 11) input.reduce((total, cur) => total + cur) or more readable: def op(total: Int, cur: Int) = total + cur input reduce op The result is 26 (sum). The code is more-or-less readable: to reduce method we are passing 2-argument function op (operation). Both parameters of that function (and its return value) need to have the same type as the collection. reduce() will invoke that operation on two first items of the collection: op(3, 5) //8 The result (8) is passed as a first argument to a subsequent invocation of op where the second argument is the next collection element: op(8, 7) //15 and finally: op(15, 11) //26 From the logical standpoint the following composed operation has been invoked: op(op(op(3, 5), 7), 11) When we realize that op() is basically an addition: (((3 + 5) + 7) + 11) So far so good - reduce() reduces a collection of a given type to a single value of the same type. Example use-cases include adding up numbers, concatenating a sequence of strings, etc.: List("Foo", "Bar", "Buzz").reduce(_ + _) Note the shorthand notation for code block without naming the parameters: _ + _. Obviously we are not limited to addition operator: def factorial(x: Int) = (2 to x).reduce(_ * _) It is worth to mention two special cases: when the collection has only one element, reduce() returns this very element. When it is empty, reduce() will throw an exception. Let's face it, typically we implement factorial for the first (and last) time somewhere at the beginning of the university and to add up numbers we have a convenience method: input.sum Besides the problem with empty collections is a bit painful - after all the sum of empty set of numbers is intuitively equal to 0 and the concatenation of an empty set of strings is... an empty string. Here is where foldLeft() enters with the ability to specify initial value: input.foldLeft(0)(op) In this case the op() function is first called with initial value 0 as the first argument and with the first collection element: op(0, 3) The subsequent iterations remain the same. If the collection is empty, foldLeft() returns the initial value. It is sad how many tutorial stop right here. After all we can simply prepend initial value to the input list and happily use reduce(): (0 :: input).reduce(op) (0 :: Nil).reduce(op) //empty list is prepended by 0 Even worse, many suggest “simplified" foldLeft() syntax, I doubt it simplifies anything: (0 /: input)(op) This is equivalent to input.foldLeft(0)(op) but intended for people who love Perl. So, closing this way too long introduction, let us see the true power behind foldLeft(). Let us assume that we have an object of type [T] on which we would like to perform a set of transformations. Transformation is nothing more than a function that accepts and returns an object of type [T]. We can return the same instance (no-op transformation), wrap the original object (the Decorator pattern) or mutate it. It is not hard to imagine that the order of transformations is important. For example let us use an ordinary string and set of transformations represented by functions of String => String: val reverse = (s: String) => s.reverse val toUpper = (s: String) => s.toUpperCase val appendBar = (s: String) => s + "bar" Remembering that a result of a first transformation is an argument of the second one we can say: appendBar(toUpper(reverse("foo"))) //OOFbar toUpper(reverse(appendBar("foo"))) //RABOOF I think that's obvious. Unfortunately we need a method taking an arbitrary (possibly empty or created dynamically) list of transformations to apply: def applyTransformations(initial: String, transformations: Seq[String => String]) = //??? applyTransformations("foo", List(reverse, toUpper, appendBar)) applyTransformations("foo", List(appendBar, reverse, toUpper)) applyTransformations("foo", List.fill(7)(appendBar)) The last line performs appendBar transformation 7 times on an initial value "foo". How to implement applyTransformations method? The programmer with highly imperative background would probably come up with something like this: def applyTransformations(initial: String, transformations: Seq[String => String]) = { var cur = initial for(transformation <- transformations) { cur = transformation(cur) } cur } Boring loop over all transformations, the intermediate result is stored in a variable. This implementation has several drawbacks. First - it's imperative (!) Scala tries to embrace the functional programming paradigm and this code seems very low-level. Our second take is much more idiomatic as far as Scala is concerned - we use recursion and pattern matching: @tailrec def applyTransformations(initial: String, transformations: Seq[String => String]): String = transformations match { case head :: tail => applyTransformations(head(initial), tail) case Nil => initial } A little bit harder to comprehend compared to imperative solution. If the list of transformations is empty - return current value. If it's not, apply the first transformation (head(initial)) and recursively call myself with the rest of the transformations (tail). Turns out this problem can be implemented in much, much more concise way, without explicit loops and recursion. Have you noticed how the problem with nested transformations (appendBar(toUpper(reverse("foo")))) is similar to how the foldLeft() works (op(op(op(3, 5), 7), 11))? def applyTransformations(initial: String, transformations: Seq[String => String]) = transformations.foldLeft(initial) { (cur, transformation) => transformation(cur) } Understanding how the code above works requires a little bit of time - but it is really rewarding afterwards. Also it allows you to fully grasp the power of foldLeft(). Before you go further try to figure this out yourself. Few tips: The type of foldLeft() result [B] doesn't necessarily have to be the same as the collection type [A]. It is the type of the initial value. In our example the input collection contains functions but the initial value is String. Function passed as an argument to foldLeft() does not need to accept both arguments of [A] type and return that type as well - as it was with reduce(). In fact, the signature of foldLeft() is as follows: def foldLeft[B](initial: B)(op: (B, A) => B): B The value returned by op function should be of the same type as its first argument. Also the whole foldLeft() invocation will have the same type. Let's think about it: the type of the first argument of op() is compatible with the initial value (initial: B) because in the first iteration it is the initial value that is passed as the first argument of op. A second argument is the first element of the input collection of type [A]. In the second iteration the result of op() invocation (of type [B]) is passed as the first argument of subsequent invocation of op. This time the second element of the input collection is used as the second argument. And it goes on until it reaches the end of the collection. I think the pseudo-code would be much easier to comprehend. First some example invocation: List(reverse, toUpper, appendBar).foldLeft("foo") { (cur, transformation) => transformation(cur) } Subsequent iterations (pseudo-code): val initial = "foo" val temp1 = (initial, reverse) => reverse(initial) val temp2 = (temp1, toUpper) => toUpper(temp1) val temp3 = (temp2, appendBar) => appendBar(temp2) And after inlining temporary variables: val initial = "foo" appendBar(toUpper(reverse(initial))) Isn't this the result we've been waiting for? As it turns out, foldLeft() is not only useful when we need to reduce (aggregate) collection to a single value, like adding up numbers - in fact, reduce() or sum() are better suited in this case. foldLeft() seems to be a great fit when we need to iterate over an arbitrary collection but every iteration requires some sort of result from previous one. By the way this is the reason why fold and reduce operations can't be executed in parallel. In comments to the original article Cezary Bartoszuk suggested an alternative way of using foldLeft() in this problem: def composeAll[A](ts: Seq[A => A]): A => A = ts.foldLeft(identity[A] _)(_ compose _) def applyTransformations(init: String, ts: Seq[String => String]): String = composeAll(ts.reverse)(init) If this solution isn't clear to your, once again few tips. First of all identity[A] _ is an identity function - always returning an argument untouched. Secondly val composed = appendBar compose toUpper is equivalent to: val composed = (s: String) => appendBar(toUpper(s)) So another mathematical term: function composition. This was a translation of my article "Ukryta potęga foldLeft()" originally published on scala.net.pl.
April 12, 2012
by Tomasz Nurkiewicz
· 68,517 Views · 3 Likes
article thumbnail
JMS Message Groups in Apache Camel
Message groups in JMS provide a way to identify a set of related messages. The messages could be related by anything - a customer order number, for example. Basically a JMS broker provides a guarantee that any messages that belong to a specific group will always be consumed by a common consumer. For instance, imagine that we’ve used the splitter pattern to split out line items from an order but want to aggregate those line items together later in a route. In order to perform that aggregation you need to guarantee that all of the messages being aggregated together are consumed by the same consumer. Below is an example of using message groups with ActiveMQ within Apache Camel. package com.brinksys.camel; import org.apache.activemq.ActiveMQConnectionFactory; import org.apache.activemq.broker.BrokerService; import org.apache.activemq.camel.component.ActiveMQComponent; import org.apache.activemq.pool.PooledConnectionFactory; import org.apache.camel.CamelContext; import org.apache.camel.Exchange; import org.apache.camel.Processor; import org.apache.camel.ProducerTemplate; import org.apache.camel.builder.RouteBuilder; import org.apache.camel.impl.DefaultCamelContext; import java.util.concurrent.TimeUnit; public class App { private static BrokerService broker; public static void main(String[] args) throws Exception { try { startBroker(); CamelContext ctx = createCamelContext(); ctx.start(); ctx.addRoutes(new RouteBuilder() { @Override public void configure() throws Exception { /* Our direct route will take a message, and set the message to group 1 if the body is an integer, * otherwise set the group to 2. * * This demonstrates the following concepts: * 1) Header Manipulation * 2) Checking the payload type of the body and using it in a choice. * 3) JMS Message groups */ from("direct:begin") .choice() .when(body().isInstanceOf(Integer.class)).setHeader("JMSXGroupID",constant("1")) .otherwise().setHeader("JMSXGroupID",constant("2")) .end() .to("amq:queue:Message.Group.Test"); /* These two are competing consumers */ from("amq:queue:Message.Group.Test").routeId("Route A").log("Received: ${body}"); from("amq:queue:Message.Group.Test").routeId("Route B").log("Received: ${body}"); } }); sendMessages(ctx.createProducerTemplate()); Thread.sleep(TimeUnit.SECONDS.toMillis(10)); stopBroker(); } catch (Exception e) { e.printStackTrace(); } } private static CamelContext createCamelContext() throws Exception { CamelContext camelContext = new DefaultCamelContext(); ActiveMQConnectionFactory activeMQConnectionFactory = new ActiveMQConnectionFactory("vm://localhost/"); PooledConnectionFactory pooledConnectionFactory = new PooledConnectionFactory(activeMQConnectionFactory); pooledConnectionFactory.setMaxConnections(8); pooledConnectionFactory.setMaximumActive(500); ActiveMQComponent activeMQComponent = ActiveMQComponent.activeMQComponent(); activeMQComponent.setUsePooledConnection(true); activeMQComponent.setConnectionFactory(pooledConnectionFactory); camelContext.addComponent("amq", activeMQComponent); return camelContext; } private static void sendMessages(ProducerTemplate pt) throws Exception { for (int i = 0; i < 10; i++) { pt.sendBody("direct:begin", Integer.valueOf(i)); } for (int i = 0; i < 10; i++) { pt.sendBody("direct:begin", "next group"); } pt.sendBody("direct:begin", Integer.valueOf(1)); pt.sendBody("direct:begin", "foo"); pt.sendBody("direct:begin", Integer.valueOf(2)); } private static void startBroker() throws Exception { broker = new BrokerService(); broker.addConnector("vm://localhost"); broker.start(); } private static void stopBroker() throws Exception { broker.stop(); } } The result of running this main method is as follows: 2445 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 0 2447 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 1 2460 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 2 2466 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 3 2472 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 4 2479 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 5 2482 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 6 2485 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 7 2488 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 8 2490 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 9 2493 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: next group 2496 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: next group 2499 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: next group 2501 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: next group 2504 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: next group 2505 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: next group 2508 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: next group 2510 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: next group 2513 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: next group 2515 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: next group 2517 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 1 2535 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: foo 2538 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 2 You’ll notice that all messages with a groupId of 1 are consumed by one route and the messages with a groupId of 2 are consumed by the other consumer. You’ll also see how relatively simple it is to inspect the body of our original message to check it’s type and set the header in the route that begins our orchestration. If you wish to run this source code, I’ve set up a little Git repository on github for hosting some camel examples. As of the time I write this, only the message group example is available, but others should appear soon.
April 11, 2012
by Jason Whaley
· 15,456 Views
article thumbnail
A Regular Expression HashMap Implementation in Java
Below is an implementation of a Regular Expression HashMap. It works with key-value pairs which the key is a regular expression. It compiles the key (regular expression) while adding (i.e. putting), so there is no compile time while getting. Once getting an element, you don't give regular expression; you give any possible value of a regular expression. As a result, this behaviour provides to map numerous values of a regular expression into the same value. The class does not depend to any external libraries, uses only default java.util. So, it will be used simply when a behaviour like that is required. import java.util.ArrayList; import java.util.HashMap; import java.util.regex.Pattern; /** * This class is an extended version of Java HashMap * and includes pattern-value lists which are used to * evaluate regular expression values. If given item * is a regular expression, it is saved in regexp lists. * If requested item matches with a regular expression, * its value is get from regexp lists. * * @author cb * * @param : Key of the map item. * @param : Value of the map item. */ public class RegExHashMap extends HashMap { // list of regular expression patterns private ArrayList regExPatterns = new ArrayList(); // list of regular expression values which match patterns private ArrayList regExValues = new ArrayList(); /** * Compile regular expression and add it to the regexp list as key. */ @Override public V put(K key, V value) { regExPatterns.add(Pattern.compile(key.toString())); regExValues.add(value); return value; } /** * If requested value matches with a regular expression, * returns it from regexp lists. */ @Override public V get(Object key) { CharSequence cs = new String(key.toString()); for (int i = 0; i < regExPatterns.size(); i++) { if (regExPatterns.get(i).matcher(cs).matches()) { return regExValues.get(i); } } return super.get(key); } }
April 11, 2012
by Cagdas Basaraner
· 24,747 Views
article thumbnail
How to Pad a Number With a Leading Zero With C#
Recently I was working with a project where I was in need to format a number in such a way which can apply leading zero for particular format. So after doing such R and D I have found a great way to apply this leading zero format. I was having need that I need to pad number in 5 digit format. So following is a table in which format I need my leading zero format. 1-> 00001 20->00020 300->00300 4000->04000 50000->5000 So in the above example you can see that 1 will become 00001 and 20 will become 00200 format so on. So to display an integer value in decimal format I have applied interger.Tostring(String) method where I have passed “Dn” as the value of the format parameter, where n represents the minimum length of the string. So if we pass 5 it will have padding up to 5 digits. So let’s create a simple console application and see how its works. Following is a code for that. using System; namespace LeadingZero { class Program { static void Main(string[] args) { int a = 1; int b = 20; int c = 300; int d = 4000; int e = 50000; Console.WriteLine(string.Format("{0}------>{1}",a,a.ToString("D5"))); Console.WriteLine(string.Format("{0}------>{1}", b, b.ToString("D5"))); Console.WriteLine(string.Format("{0}------>{1}", c, c.ToString("D5"))); Console.WriteLine(string.Format("{0}------>{1}", d, d.ToString("D5"))); Console.WriteLine(string.Format("{0}------>{1}", e, e.ToString("D5"))); Console.ReadKey(); } } } As you can see in the above code I have use string.Format function to display value of integer and after using integer value’s ToString method. Now Let’s run the console application and following is the output as expected. Here you can see the integer number are converted into the exact output that we requires. That’s it you can see it’s very easy. We have written code in nice clean way and without writing any extra code or loop. Hope you liked it. Stay tuned for more.. Till than happy programming.
April 10, 2012
by Jalpesh Vadgama
· 30,387 Views
article thumbnail
Externalizing Application Logic : Business Rules Approach with Drools
i originally written this post in our company's blog and i decided to share it here as my first contributed article. this topic is about externalizing application logic through business rules approach using the drools business rules engine. enterprise applications usually consists of multiple layers. primarily presentation layer, business logiclayer and persistence layer. the business logic layer is considered the heart among these layers, and this is where all processes and decisions take place. requirements for this layer also changes more often than the rest of the application. as requirements continues to change, it is very easy for the code in this layer to end up tangled in a situation known as “spaghetti code”, lots of nested if-else conditions, and as new conditions are added, readability suffers. moreover, to change the behavior of the system, a recompile and rebuild must be done. this post aims to introduce the business rules approach , a development methodology where rules, decisions and processes are used by, but does not have to be embedded in business process management systems. you’ll be introduced to this methodology through the use of a business rule engine – jboss drools expert , part of jboss’ business logic integration platform. what are business rules? a business rule is simply a statement that defines or constrains some aspects of the business. business rules always resolves to either true or false. business rule can also be defined as an abstraction of the policies and practices of a business organization. example: a customer can have several reservations but only one car can be rented at a time. (car rental) when a registration is coming from a country listed in the banned country list, reject the registration. (ptc site registration) what is a rule engine? a business rule engine is the heart of the technology behind the business rules approach. it is a system that evaluates and executes one or more business rules in a runtime production system against known facts (domain models). why use a rule engine? logic and data separation – your data is in your domain objects, the logic is in the rules. speed and scalability – business rules engine uses algorithms design to efficiently match rule patterns to your domain objects, for example the rete algorithm as used in drools expert. also in terms of scalability, rules can be shared in different systems allowing us to create or integrate sub-systems that uses some of the rules used by other sub-systems. centralization of knowledge – since logic are externalized in the form of rules, there is only a single source of truth, and that is your repository of rules / knowledge (a knowledge base). explanation facility – rule systems effectively provide an “explanation facility” by being able to log the decisions made by the rule engine along with why the decisions were made. understandable rules – in drools expert, rules can be written using dsls (domain specific languages) allowing non-technical domain experts to write rules which are close to natural language. sample rules figure 1: an example drools rule file hello message rule and goodbye message rule. we can read these rules this way: hello message rule – “when the message object’s status is hello, display the message attribute of the message object and change its message and status to “goodbye cruel world” and goodbye respectively”. goodbye message rule – “when the message object’s status is goodbye, display the message attribute of the message object.” make your applications rules-driven drools expert is a sub-project of jboss business logic integration platform and this is the business rule engine we’ll be using in this post. to make your applications rules-driven using drools expert, you need first to understand the following important concepts: facts – facts are simply your domain objects / models. rules are run against these facts to match conditions declared in the rules with the state of the fact currently being used and this determines the rule / rules to run. rule – a rule is a statement that test a condition against a fact and is composed of 3 important sections: name – name of the rule lhs (condition) – this is the “when” section (see above). this section declares the condition the rule would test. rhs (consequence / action – this is the “then” section (see above). this section declares the consequence or the action to be performed when the condition declared in the “when” section is met. knowledge base – a repository of rule / knowledge knowledge session – a session is an established interaction between your application and the rules engine. a session is created from a knowledgebase and can be either stateless or stateful. the application uses the established session to run the engine against the inserted facts. example code: package com.ideyatech.drools; import org.drools.knowledgebase; import org.drools.knowledgebasefactory; import org.drools.builder.knowledgebuilder; import org.drools.builder.knowledgebuilderfactory; import org.drools.builder.resourcetype; import org.drools.io.resourcefactory; import org.drools.runtime.statelessknowledgesession; import com.ideyatech.drools.bean.message; public class main { public static void main(string[] args) { final knowledgebase knowledgebase = createknowledgebase(); final statelessknowledgesession session = knowledgebase.newstatelessknowledgesession(); try { final message message = new message(); message.setstatus(message.hello); session.execute(message); }catch (throwable e) { e.printstacktrace(); } } private static knowledgebase createknowledgebase() { final knowledgebuilder builder = createknowledgebuilder(); final knowledgebase knowledgebase = knowledgebasefactory .newknowledgebase(); knowledgebase.addknowledgepackages( builder.getknowledgepackages()); return knowledgebase; } private static knowledgebuilder createknowledgebuilder() { final knowledgebuilder builder = knowledgebuilderfactory.newknowledgebuilder(); final string rulepath = "com/ideyatech/drools/rule/hello-world-rule.drl"; builder.add(resourcefactory.newclasspathresource( rulepath), resourcetype.drl); if (builder.haserrors()) { throw new runtimeexception(builder.geterrors().tostring()); } return builder; } } code explanation: the code above contains two helper methods: createknowledgebuilder() and createknowledgebase(). the createknowledgebuilder() method creates an instance of knowledgebuilder which is responsible for taking source files such as .drl (drools rule) files and turning them into a knowledgepackage of rule and process definitions which a knowledgebase can consume. in this method we added to the knowledgebuilder the rule file “hello-world-rule.drl” which is a classpath resource. take note that rules can also obtained as a file resource or a url resource. the createknowledgebase() method creates an instance of knowledgebase which is a repository of all the application’s knowledge definitions. this method uses the createknowledgebuilder() and gets all knowledge packages from the created knowledgebuilder to be used by the created knowledgebase. in the main method after creating our knowledgebase, we then created a statelesssession and then asked the created session to execute all rules against the message object with a status hello. when we run this code, this produces the following result: human-readable rules using dsl in drools, rules can also be written in a syntax which is close to natural language (english for example). this allows non-technical domain experts to author rules using plain english statements and a few drools rule syntax which doesn’t require advanced technical knowledge to understand. the example below shows a simple rule written in dsl format. figure 2: dsl file figure 3: dslr file in the above figures, we have 2 files; a dsl and a dslr file. the dsl file contains the mapping of the conditions and consequences used in the dslr file. for example, the dsl condition “there is a customer with firstname {name}” is mapped to “$customer : customer(firstname == {name})”. this mapping / translation simply tells the engine how to interpret the rule written in the dslr file. conclusion truly, by using a business rule engine and allowing business rules to drive our applications, we can increase our development agility, improve software maintainability, and allow us to deal much easier with evolving requirement complexity. however, also take note that by introducing business rule engine to your applications, you also add another layer of complexity to your application architecture. using a business rule engine should be considered when we need to create pluggable systems, expert systems or when the need to handle ever-changing requirements is very high. for application requirements of moderate complexity, the standard approach of embedding logic through imperative programming language may suffice.
April 10, 2012
by Arjay Nacion
· 14,393 Views · 1 Like
article thumbnail
Ternary Operator in VB.NET
We all know about the ternary operator in C#.NET. I am a big fan of the ternary operator and I like to use it instead of using IF..Else. Those who don’t know about ternary operator please go through below link. http://msdn.microsoft.com/en-us/library/ty67wk28(v=vs.80).aspx Here you can see ternary operator returns one of the two values based on the condition. See following example. bool value = false; string output=string.Empty; //using If condition if (value==true) output ="True"; else output="False"; //using tenary operator output = value == true ? "True" : "False"; In the above example you can see how we produce same output with the ternary operator without using If..Else statement. Recently in one of the project I was working with VB.NET language and I was eager to know if there is a ternary operator equivalent there or not. After searching on internet I have found two ways to do it. IF operator which works for VB.NET 2008 and higher version and IIF operator which is there since VB 6.0. So let’s check same above example with both of this operators. So let’s create a console application which has following code. Module Module1 Sub Main() Dim value As Boolean = False Dim output As String = String.Empty ''Output using if else statement If value = True Then output = "True" Else output = "False" Console.WriteLine("Output Using If Loop") Console.WriteLine(output) output = If(value = True, "True", "False") Console.WriteLine("Output using If operator") Console.WriteLine(output) output = IIf(value = True, "True", "False") Console.WriteLine("Output using IIF Operator") Console.WriteLine(output) Console.ReadKey() End If End Sub End Module As you can see in the above code I have written all three-way to condition check using If.Else statement and If operator and IIf operator. You can see that both IIF and If operator has three parameter first parameter is the condition which you need to check and then another parameter is true part of you need to put thing which you need as output when condition is ‘true’. Same way third parameter is for the false part where you need to put things which you need as output when condition as ‘false’. Now let’s run that application and following is the output as expected. That’s it. You can see all three ways are producing same output. Hope you like it. Stay tuned for more..Till then Happy Programming.
April 10, 2012
by Jalpesh Vadgama
· 34,549 Views · 1 Like
article thumbnail
The Hidden Treasure of Quartz Scheduler Plugins
Although briefly described in the official documentation, I believe Quartz plugins aren't known enough, looking at how useful they are. Essentially plugins in Quartz are convenient classes wrapping registration of underlying listeners. You are free to write your own plugins but we will focus on existing ones shipped with Quartz. LoggingTriggerHistoryPlugin First some background. Two main abstractions in Quartz are jobs and triggers. Job is a piece of code that we would like to schedule. Trigger instructs the scheduler when this code should run. CRON (e.g. run every Friday between 9 AM and 5 PM until November) and simple (run 100 times every 2 hours) triggers are most commonly used. You associate any number of triggers to a single job. Believe it or not, Quartz by default provides no logging or monitoring whatsoever of executed jobs and triggers. There is an API, but no built-in logging is implemented. It won't show you that it now executes this particular job due to this trigger firing. So the first thing you should do is adding the following lines to your quartz.properties: org.quartz.plugin.triggerHistory.class=org.quartz.plugins.history.LoggingTriggerHistoryPlugin org.quartz.plugin.triggerHistory.triggerFiredMessage=Trigger [{1}.{0}] fired job [{6}.{5}] scheduled at: {2, date, dd-MM-yyyy HH:mm:ss.SSS}, next scheduled at: {3, date, dd-MM-yyyy HH:mm:ss.SSS} org.quartz.plugin.triggerHistory.triggerCompleteMessage=Trigger [{1}.{0}] completed firing job [{6}.{5}] with resulting trigger instruction code: {9}. Next scheduled at: {3, date, dd-MM-yyyy HH:mm:ss.SSS} org.quartz.plugin.triggerHistory.triggerMisfiredMessage=Trigger [{1}.{0}] misfired job [{6}.{5}]. Should have fired at: {3, date, dd-MM-yyyy HH:mm:ss.SSS} The first line (and the only required) loads the plugin class LoggingTriggerHistoryPlugin. The remaining lines are configuring the plugin, customizing the logging messages. I found the built-in defaults not very well thought, e.g. they display current time which is already part of the logging framework message. You are free to construct any logging message, see the API for details. Adding these extra few lines makes debugging and monitoring much easier: LoggingTriggerHistoryPlugin | Trigger [Demo.Every-few-seconds] fired job [Demo.Print-message] scheduled at: 04-04-2012 23:23:47.036, next scheduled at: 04-04-2012 23:23:51.036 //...job output LoggingTriggerHistoryPlugin | Trigger [Demo.Every-few-seconds] completed firing job [Demo.Print-message] with resulting trigger instruction code: DO NOTHING. Next scheduled at: 04-04-2012 23:23:51.036 You see now why naming your triggers (Demo.Every-few-seconds) and jobs (Demo.Print-message) is so important. LoggingJobHistoryPlugin There is another handy plugin related to logging: org.quartz.plugin.jobHistory.class=org.quartz.plugins.history.LoggingJobHistoryPlugin org.quartz.plugin.jobHistory.jobToBeFiredMessage=Job [{1}.{0}] to be fired by trigger [{4}.{3}], re-fire: {7} org.quartz.plugin.jobHistory.jobSuccessMessage=Job [{1}.{0}] execution complete and reports: {8} org.quartz.plugin.jobHistory.jobFailedMessage=Job [{1}.{0}] execution failed with exception: {8} org.quartz.plugin.jobHistory.jobWasVetoedMessage=Job [{1}.{0}] was vetoed. It was to be fired by trigger [{4}.{3}] at: {2, date, dd-MM-yyyy HH:mm:ss.SSS} The rule is the same - plugin + extra configuration. See JavaDoc of LoggingJobHistoryPlugin for details and possible placeholders. Quick look at logs reveals very descriptive output: Trigger [Demo.Every-few-seconds] fired job [Demo.Print-message] scheduled at: 04-04-2012 23:34:53.739, next scheduled at: 04-04-2012 23:34:57.739 Job [Demo.Print-message] to be fired by trigger [Demo.Every-few-seconds], re-fire: 0 //...job output Job [Demo.Print-message] execution complete and reports: null Trigger [Demo.Every-few-seconds] completed firing job [Demo.Print-message] with resulting trigger instruction code: DO NOTHING. Next scheduled at: 04-04-2012 23:34:57.739 I have no idea why these plugins aren't enabled by default. After all, if you don't want such a verbose output, you can turn it off in your logging framework. Never mind, I think it is a good idea to have them in place when troubleshooting Quartz execution. XMLSchedulingDataProcessorPlugin This is a pretty comprehensive plugin. It reads XML file (by default named quartz_data.xml) containing jobs and triggers definitions and adds them to the scheduler. This is especially useful when you have a global job that you need to add once. Plugin can either update the existing jobs/triggers or ignore the XML file if they already exist - very useful when JDBCJobStore is used. org.quartz.plugin.xmlScheduling.class=org.quartz.plugins.xml.XMLSchedulingDataProcessorPlugin In the aforementioned article we have been manually adding job to the scheduler: val trigger = newTrigger(). withIdentity("Every-few-seconds", "Demo"). withSchedule( simpleSchedule(). withIntervalInSeconds(4). repeatForever() ). build() val job = newJob(classOf[PrintMessageJob]). withIdentity("Print-message", "Demo"). usingJobData("msg", "Hello, world!"). build() scheduler.scheduleJob(job, trigger) The same can be achieved with XML configuration, just place the following quartz_data.xml in your CLASSPATH: false true Every-few-seconds Demo Print-message Demo -1 4000 Print-message Demo com.blogspot.nurkiewicz.quartz.demo.PrintMessageJob msg Hello, World! The file supports both simple and CRON triggers and is well described using XML Schema. It is even possible to point out to an XML files somewhere in the file system and periodically scan them for changes (!) (see: XMLSchedulingDataProcessorPlugin.setScanInterval(). Guess what is Quartz using to schedule periodic scanning? org.quartz.plugin.xmlScheduling.fileNames=/etc/quartz/system-jobs.xml,/home/johnny/my-jobs.xml org.quartz.plugin.xmlScheduling.scanInterval=60 ShutdownHookPlugin Last but not least, ShutdownHookPlugin. Small but probably useful plugin that register shutdown hook in the JVM in order to gently stop the scheduler. However I recommend turning cleanShutdown off - if the system already tries to abruptly stop the application (typically scheduler shutdown is called by Spring via SchedulerFactoryBean) or the user hit Ctrl+C - waiting for currently running jobs seems like a bad idea. After all, maybe we are killing the application because some jobs are running for too long/hanging? org.quartz.plugin.shutdownHook.class=org.quartz.plugins.management.ShutdownHookPlugin org.quartz.plugin.shutdownHook.cleanShutdown=false As you can see Qurtz ships with few quite interesting plugins. For some reason they aren't described in detail in the official documentation, but they work pretty well and are a valuable addition to scheduler. The source code with applied plugins is available on GitHub.
April 9, 2012
by Tomasz Nurkiewicz
· 18,992 Views · 1 Like
article thumbnail
Creating Dynamic Breadcrumbs in ASP.NET MVC With MvcSiteMap
I created a new MVC 3 web application called breadcrumb and I added a reference to the site map provider via the NuGet Package Manager.
April 8, 2012
by Jalpesh Vadgama
· 42,158 Views
article thumbnail
Using New Relic with Supervisord and Gunicorn
New Relic recently added support for python to their awesome web application performance tool, and I have been playing with it on a number of projects. Installing and configuring new relic is pretty well covered in their own documentation, so there is no reason for me to repeat that here. One thing that isn't covered in the documentation is how to use new relic if you are using supervisord to control your gunicorn processes, and I'll take this time right now to show you what I did. Setting up new relic with supervisord and gunicorn is pretty easy. All that you need to do, is change your supervisor.conf file and then update your supevisor config, and you are good to go. Here is the supervisor.conf file for my awesome app, before I installed new relic. Note: These are not my real conf files, they have been changed to protect the guilty, so please excuse any typos. [program:awesome_app] directory=/opt/apps/awesome_home/awesome_app/ command=/opt/apps/awesome_home/bin/python2.6 /opt/apps/awesome_home/bin/gunicorn_django -c /opt/apps/awesome_home/awesome_app/conf/gunicorn.conf user=aweman autostart=true autorestart=true environment=HOME='/opt/apps/awesome_home/awesome_app/',DJANGO_SETTINGS_MODULE='settings' After I installed new relic. All you need to do is add the 'newrelic-admin run-program' command before the 'gunicorn_django' command and add an ENV variable called NEW_RELIC_CONFIG_FILE that is pointing to your newrelic.ini file. [program:awesome_app] directory=/opt/apps/awesome_home/awesome_app/ command=/opt/apps/awesome_home/bin/newrelic-admin run-program /opt/apps/awesome_home/bin/gunicorn_django -c /opt/apps/awesome_home/awesome_app/conf/gunicorn.conf user=aweman autostart=true autorestart=true environment=HOME='/opt/apps/awesome_home/awesome_app/',DJANGO_SETTINGS_MODULE='settings',NEW_RELIC_CONFIG_FILE=/opt/apps/awesome_home/awesome_app/conf/newrelic.ini Now that you have the new configuration setup, you will need to let supervisord know that you have changed t he configuration for that app. If you run the update command it will prompt supervisord to reread the configuration file for that app, and reload the config, and then restart the application with the new configuration. $ supervisorctl update Another thing that is important to note here, is the fact that New Relic currently doesn't work well with Gunicorn in gevent mode. If you try to use gevent with gunicorn and new relic, it may not start up at all, or just not work as it should. Here is what they say in the Known Issues section of their docs. Gunicorn gevent mode - When using gevent mode of gunicorn and the 'newrelic-admin run-program' command is used to wrap the invocation of gunicorn, the hosted web application can fail in strange ways. One way this is manifesting is with requests blocking for a period of 1 minute. The cause of the problem is believed in this case to specifically relate to the order in which module imports are occuring. The monkey patching performed by gevent is not working properly for the case where the Python threading module is imported before the gevent monkey patching routine is run. Because of this, I have changed my gunicorn's to use eventlet when using new relic, and that seems to work fine. I normally prefer to use gevent, so hopefully they will be able to fix the issue with gevent so I can revert back to that setup. All and all I have been pretty happy with new relic, it has helped us find issues with our code that would have been a pain otherwise. There support has been awesome, and they have been adding new fixes/ improvements all the time. Can't wait to see what else they have in store for the future. I would try it out if you can, they have a lite version that is free which even includes server monitoring.
April 8, 2012
by Ken Cochrane
· 7,152 Views
article thumbnail
What AnyCPU Really Means As Of .NET 4.5 and Visual Studio 11
the 32-bit and 64-bit development story on windows seemingly never stops causing problems for developers. it’s been a decade since 64-bit processors have started popping up in the windows consumer environment, but we just can’t get it right . if you forget some of the gory details, here are a couple of reminders: on a 64-bit windows system, both the 32-bit and 64-bit versions of system dlls are stored. the 64-bit dlls are in c:\windows\system32, and the 32-bit dlls are in c:\windows\syswow64. when a 32-bit process opens a file in c:\program files, it actually reads/writes to c:\program files (x86). there are separate views of (most of) the registry for 32-bit and 64-bit applications. you can change the 64-bit registry location and it wouldn’t be visible to 32-bit applications. these differences are hardly elegant as they are, but they allow 32-bit applications to run successfully on a 64-bit windows system. while unmanaged applications always had to choose the native target—x86, x64, or ia64 in the visual studio case—managed code has the additional choice of anycpu . what anycpu used to mean up to .net 4.0 (and visual studio 2010) is the following: if the process runs on a 32-bit windows system, it runs as a 32-bit process. il is compiled to x86 machine code. if the process runs on a 64-bit windows system, it runs as a 64-bit process. il is compiled to x64 machine code. if the process runs on an itanium windows system (has anyone got one? ;-)), it runs as a 64-bit process. il is compiled to itanium machine code. prior to visual studio 2010, anycpu was the default for most .net projects, which was confusing to some developers: when they ran the application on a 64-bit windows system, the process was a 64-bit process, which may cause unexpected results. for example, if the application relies on an unmanaged dll of which only a 32-bit version is available, its 64-bit version won’t be able to load that component. in visual studio 2010, x86 (and not anycpu) became the default for most .net projects—but otherwise the semantics haven’t changed. in .net 4.5 and visual studio 11 the cheese has been moved. the default for most .net projects is again anycpu, but there is more than one meaning to anycpu now. there is an additional sub-type of anycpu, “any cpu 32-bit preferred”, which is the new default (overall, there are now five options for the /platform c# compiler switch : x86, itanium, x64, anycpu, and anycpu32bitpreferred). when using that flavor of anycpu, the semantics are the following: if the process runs on a 32-bit windows system, it runs as a 32-bit process. il is compiled to x86 machine code. if the process runs on a 64-bit windows system, it runs as a 32-bit process. il is compiled to x86 machine code. if the process runs on an arm windows system, it runs as a 32-bit process. il is compiled to arm machine code. the difference, then, between “any cpu 32-bit preferred” and “x86” is only this: a .net application compiled to x86 will fail to run on an arm windows system, but an “any cpu 32-bit preferred” application will run successfully. to inspect these changes, i created a new c# console application in visual studio 11 that prints the values of environment.is64bitoperatingsystem and environment.is64bitprocess . when i ran it on my 64-bit windows system, the result was as follows: is64bitoperatingsystem = true is64bitprocess = false inspecting the project’s properties shows the following (in the current visual studio ui “prefer 32-bit” is grayed out and unchecked, where in actuality it is enabled…): inspecting the executable with corflags.exe shows the following: version : v4.0.30319 clr header: 2.5 pe : pe32 corflags : 131075 ilonly : 1 32bitreq : 0 32bitpref : 1 signed : 0 after changing the 32bitpref setting with corflags.exe (using the /32bitpref- option), the output was as follows: is64bitoperatingsystem = true is64bitprocess = true
April 7, 2012
by Sasha Goldshtein
· 40,228 Views
article thumbnail
Configuring Quartz With JDBCJobStore in Spring
I am starting a little series about Quartz scheduler internals, tips and tricks, this is chapter 0 - how to configure persistent job store.
April 7, 2012
by Tomasz Nurkiewicz
· 37,827 Views
article thumbnail
How Facebook Does Deployment
Some interesting insider info was released this week about the Facebook deployment process. I've compiled a TL;DR version here in case you don't want to read the whole piece, but I recommend you do. The information comes straight from Chuck Rossi, the head of Facebook's release engineering team, who was interviewed by Ars Technica's Ryan Paul: An important aspect of Facebook's development culture is the idea that developers are fully responsible for how their code behaves in production. This philosophy mirrors the "DevOps" movement, which encourages lowering the wall between software development and IT operations. If any of the code in a Facebook update causes problems in production, the developer who wrote it is on the hook for making sure that the issue gets resolved as quickly as possible. --Ryan Paul, Ars Technica "Exclusive: a behind-the-scenes look at Facebook release engineering" I think we've got more evidence here that "DevOps" is just a name for what many super-successful companies have already been doing. On to the TL;DR! For deployment Facebook uses Uses custom BitTorrent P2P system Site updates take 15-30 minutes JS, CSS, and Graphics are hosted on distributed CDNs Deployment steps Update goes to an "a2" tier which rolls update out to a small, random collection of users. Check-in procedure on company IRC where all developers who submitted code for the update have confirm that they are ready to respond if there is a problem when the update goes out. Head of release engineering issues command in terminal to begin deployment Watch web-based monitoring dashboards as the update rolls out. A small amount of servers will fail during most deployments, but it usually doesn't cause any issues. That's because any of Facebook's servers can handle any sort of page request, so they don't have to worry about serialization and migration of user session states. The servers can keep handling incoming page requests during their software updates. No-downtime, as we all know. Frequency of deployments: One minor update on most business days One major update on a weekly basis, usually Tuesdays "Release early and often." The mantra rings true at Google, Facebook, and many other companies that we look up to as the best in the business. Make sure you check out the whole article for the full story on Facebook's release engineering.
April 6, 2012
by Mitch Pronschinske
· 35,688 Views · 1 Like
article thumbnail
A-Z of JavaScript
Here is an A - Z list of some Javascript idioms and patterns. The idea is to convey in simple terms some features of the actual Javascript language (rather than how it can interact with DOM). Enjoy... Array Literals An array literal can be defined using a comma separated list in square brackets. var months = ['jan', 'feb', 'mar', 'apr', 'may', 'jun', 'jul', 'aug', 'sep', 'oct', 'nov', 'dec']; console.log(months[0]); // outputs jan console.log(months.length) // outputs 12 Arrays in javascript have a wide selection methods including push() and pop(). Suppose the world got taken over by a dictator who wanted to get rid of the last month of the year? The dictator would just do... months.pop(); And of course, the dictator will eventually want to add a month after himself when everyone will have to worship him: months.push("me"); Callbacks Since functions are objects, they can be passed as arguments to other functions. function peakOil(callback) { //... code callback(); // the parentheses mean the function is executed! } function changeCivilisationCallback(){ //... } // Now pass the changeCivilisationCallback to peakOil. // Note: no changeCivilisationCallback parentheses because it is not // executed at this point. // It will be excuted later inside peak oil. peakOil(changeCivilisationCallback); In the example above, the chanceCivilisationCallback callback function is invoked by peakOil. Logic could be added to check if the energy returns from solar panels and wind farms were sufficient in which case another callback, other than changeCivilisationCallback could be added. Configuration Object Instead of passing around a bunch of related properties... function addCar(colour, wheelsize, regplate) {...} Use a configuration object function addCar(carConf) {...} var myCarConf = { colour: "blue", wheelsize: "32", regplate: "00D98788" }; addCar(myCarConf); The use of a configuration object makes it makes it easier to write clean APIs that don't need to take a huge long list of parameters. They also means you are less likely to get silly errors if parameters are in the wrong order. Closures There are three ways to creats objects in Javascript: using literals, using the constuctor function and by using a closure. What closures offer that the other two approaches do not is encapsulation. Closures make it possible to hide away functions and variables. var counter = function(count) { console.log(">> setting count to " + this.count); return { getCount: function(){ return ++count; } } } mycounter = counter(0); console.log(mycounter.getCount()); // outputs 1 console.log(mycounter.getCount()); // outputs 2 console.log(mycounter.getCount()); // outputs 3 console.log(mycounter.getCount()); // outputs 4 // Same again with offset this time. mycounterWithOffset = counter(10); console.log(mycounterWithOffset .getCount()); // outputs 11 console.log(mycounterWithOffset .getCount()); // outputs 12 console.log(mycounterWithOffset .getCount()); // outputs 13 console.log(mycounterWithOffset .getCount()); // outputs 14 Note: The closure is the object literal returned from annoymous function. It "closes" over the count variable. No-one can access it except for the closure. It is encapsulated. The closure also has a sense of state. Note also how the it maintains the value of the counter. Constructor Functions (Built in) There are no classes in Javascript but there are construtor functions which use the new keyword syntax similar to the class based object creation in Java or other languages. Javascript has some built-in constructor functions. These include Object(), Date(), String() etc. var person = new Object(); // person variable is an Object person.name = "alex"; // properties can then be dynamically added Constructor Functions (Custom) When a function is invoked with the keyword new, it is referred to as a Constructor function. The new means that the new object will have a hidden link to value of the function's prototype member and the this keyword will be bound to the new object. function MyConstrutorFunction() { this.goodblog = "dublintech.blogspot.com"; } var newObject = new MyConstrutorFunction(); console.log(typeof newObject); // "object" console.log(newObject.goodblog); // "dublintech.blogspot.com" var noNewObject = MyConstrutorFunction(); console.log(typeof noNewObject); // "undefined" console.log(window.tastes); // "yummy" The convention is that constructor functions should begin with a capital letter. Note: if the new keyword is not used, then the 'this' variable inside the function will refer to the global object. Can you smell a potential mess? Hence why the capital letter convention for constructor functions is used. The capital letter means: "I am a constructor function, please use the new keyword". Currying Currying is the process of reducing the number of arguments passed to a function by setting some argument(s) to predefined values. Consider this function. function outputNumbers(begin, end) { var i; for (i = begin; i <= end; i++) { print(i); } } outputNumbers(0, 5); // outputs 0, 1, 2, 3, 4, 5 outputNumbers(1, 5); // outputs 1, 2, 3, 4, 5 Suppose, we want a similar function with a fixed "begin" value. Let's say the "begin" value was always 1. We could do: function outputNumbersFixedStart(start) { return function(end) { return outputNumbers(start, end); } } And then define a variable to be this new function... var outputFromOne = outputNumbersFixedStart(1); outputFromOne(3); 1, 2, 3 outputFromOne(5); 1, 2, 3, 4, 5 Delete Operator The delete operator can be used to remove properties from objects and arrays. var person = {name: 'Alex', age: 56}; // damn I don't want them to know my age remove it delete person.age; console.log("name" in person); // outputs true because it is still there console.log("age" in person); // outputs false var colours = ['red', 'green', 'blue'] // is red really in the array? console.log(colours.indexOf('red') > -1); // outputs true. // remove red, it's going out of fashion! delete colours[colours.indexOf('red')]; console.log(colours.indexOf('red') > -1); // outputs false console.log(colours.length) // length is still three, remember it's javascript! You cannot delete global variables or prototype attributes. console.log(delete Object.prototype) // can't be deleted, outputs false function MyFunction() { // ... } console.log(delete MyFunction.prototype) // can't be deleted, outputs false var myglobalVar = 1; console.log(delete this.myglobalVar) // can't be delete, outputs false Dynamic Arguments Arguments for a function do not have to be specifed in the function definition function myFunction(){ // ... Note myfunction has no arguments in signature for(var i=0; i < arguments.length; i++){ alert(arguments[i].value); } } myFunction("tony", "Magoo"); // any argument can be specified The arguments parameter is an array available to functions and gives access to all arguments that were specified in the invocation. for-in iterations for-in loops (also called enumeration) should be used to iterate over nonarray objects. var counties = { dublin: "good", kildare: "not bad", cork: "avoid" } for (var i in counties) { if (counties.hasOwnProperty(i)) { // filter out prototype properties console.log(i, ":", counties[i]); } } Function declaration In a function declaration, the function stands on its own and does not need to be assigned to anything. function multiple(a, b) { return a * b; } // Note, no semi colon is needed Function expressions When function is defined as part of something else's definition, it is considered a function expression. multiply = function multiplyFunction(a, b) { return a * b; }; // Note the semi colan must be placed after the function definition console.log(multiply(5, 10)); // outputs 50 In the above example, the function is named. It can also be anonymous, in which case the name property will be a blank string. multiply = function (a, b) { return a * b; } // Note the semi colan must be placed after the function definition console.log(multiply(5, 10)); // outputs 50 Functional Inheritance Functional inheritance is mechanism of inheritance that provides encapsulation by using closures. Before trying to understand the syntax, take an example first. Suppose we want to represent planets in the solar system. We decided to have a planet base object and then several planet child objects which inherit from the base object. Here is the base planet object: var planet = function(spec) { var that = {}; that.getName = function() { return spec.radius; }; that.getNumberOfMoons()= function() { return spec.numberOfMoons; }; return that; } Now for some planets. Let's start with Earth and Jupiter and to amuse ourselves let's add a function for Earth for people to leave and a function to Jupiter for people arriving. Sarah Palin has taken over and things have got pretty bad!!! var earth = function(spec) { var that = planet{spec}; // No need for new keyword! that.peopleLeave = function() { // ... people leave } return that; } var jupiter = function(spec) { var that = planet(spec); that.peopleArrive = function() { // .. people arrive } return that; } Now put the earth and jupiter in motion... var myEarth = earth({name:"earth",numberofmoons:1}); var myjupiter=jupiter({name:"jupiter",numberofmoons:66}); The three key points here: There is code reuse. There is encapsulation. The name and numberOfMoons properties are encapsulated. The child objects can add in their own specific functionality. Now an explanation of the syntax: The base object planet accepts some data in the spec object. The base object planet creates a closures called that which is returned. The that object has access to everything in the spec object. But, nothing else does. This provides a layer of encapsulation. The child objects, earth and jupiter, set up their own data and pass it to base planet object. The planet object returns a closure which contains base functionality. The child classes receive this closure and add further methods and variables to it. Hoisting No matter where var's are declared in a function, javascript will "hoist" them meaning that they behave as if they were declared at the top of the function. mylocation = "dublin"; // global variable function outputPosition() { console.log(mylocation); // outputs "undefined" not "dublin" var mylocation = "fingal" ; console.log(mylocation); // outputs "fingal" } outputPosition(); In the function above, the var declaration in the function means that the first log will "see" the mylocation in the function scope and not the one declared in the global scope. After declaration, the local mylocation var will have the value "undefined", hence why this is outputted first. Functions that are assigned to variables can also be hoisted. The only difference being that when functions are hoisted, their definitions also are - not just their declarations. Immediate Function Expressions Immediate function expression are executed as soon as they are defined. (function() { console.log("I ain't waiting around"); }()); There are two aspects of the syntax to note here. Firsty, there is a () immediately after the function definiton, this makes it execute. Secondly, the function can only execute if it is a function expression as opposed to a function declaration. The outer () make the function an expression. Another way to define a an immediate function expression is: var anotherWay = function() { console.log("I ain't waiting around"); }() JSON JavaScript Object Notation (JSON) is a notation used to represent objects. It is very similar to the format used for Javascript Object literals except the property names must be wrapped in quotes. The JSON format is not exclusive to javascript; it can be used by any language (Python, Ruby etc). JSON makes it very easy to see what's an array and what's an object. In XML this would be much harder. An external document - such as XSD - would have to be consulted. In this example, Mitt Romney has an array describing who might vore for him and an object which is his son. {"name": "Mitt Romney", "party": "republicans", "scary": "of course", "romneysMostLikelyVoters": ["oilguzzlers", "conservatives"], son : {name:'George Romney} Loose typing Javascript is loosely typed. This means that variables do not need to be typed. It also means there is no complex class hierarchies and there is no casting. var number1 = 50; var number2 = "51"; function output(varToOutput) { // function does not care about what type the parameter passed is. console.log(varToOutput); } output(number1); // outputs 50 output(number2); // outputs 51 Memoization Memoization is a mechanism whereby functions can cache data from previous executions. function myFunc(param){ if (!myFunc.cache) { myFunc.cache = {}; // If the cache doesn't exist, create it. } if (!myFunc.cache[param]) { //... Imagine the code to work out result below // is computationally intensive. var result = { //... }; myFunc.cache[param] = result; // now result is cached. } return myFunc.cache[param]; } Method When a function is stored as a property of an object, it is referred to as a method. var myObject { myProperty: function () { //... // the this keyword in here will refer to the myObject instance. // This means the "method" can read and change variables in the // object. } } Modules The goal of modules is to enable javascript code bases to more modular. Functions and variables are collated into a module and then the module can decide what functions and what variables the outside world can see - in the same way as encapsulations works in the object orientated paradigms. In javascript we create modules by combining characteristics of closures and immediate function expressions. var bankAccountModule = (function moduleScope() { var balance = 0; //private function doSomethingPrivate(){ // private method //... } return { //exposed to public addMoney: function(money) { //... }, withDrawMoney: function(money) { //... }, getBalance: function() { return balance; } }()); In the example above, we have a bank account module: The function expression moduleScope has its own scope. The private variable balance and the private function doSomethingPrivate, exist only within this scope and are only visible to functions within this scope. The moduleScope function returns an object literal. This is a closure which has access to the private variables and functions of moduleScope. The returned object's properties are "public" and accesible to the outside world. The returned object is automatically assigned to bankAccountModule The immediate function ()) syntax is used. This means that the module is initialised immediately. Because the returned object (the closure) is assigned to bankAccountModule, it means we can access the bankAccountModule as: bankAccountModule.addMoney(20); bankAccoumtModule.withdrawMoney(15); By convention, the filename of a module should match its namespace. So in this example, the filename should be bankAccountModule.js. Namespace Pattern Javascript doesn't have namespaces built into the language, meaning it is easy for variables to clash. Unless variables are defined in a function, they are considered global. However, it is possible to use "." in variables names. Meaning you can pretend you have name spaces. DUBLINTECH.myName = "Alex" DUBLINTECH.myAddress = "Dublin Object Literal Notation In javascript you can define an object as collection of name value pairs. The values can be property values or functions. var ireland = { capital: "Dublin", getCapital: function () { return this.capital; } }; Prototype properties (inheritance) Every object has a prototype object. It is useful when you want to add a property to all instances of a particular object. Suppose you have a constructor function, which representent Irish people who bought in the boom. function IrishPersonBoughtInTheBoom(){ } var mary = new IrishPersonBoughtInTheBoom (); var tony = new IrishPersonBoughtInTheBoom (); var peter = new IrishPersonBoughtInTheBoom (); ... Now, the Irish economy goes belly up, the property bubble explodes and you want to add a debt property to all instances of this function. To do this you would do: IrishPersonBoughtInTheBoom.prototype.debt = "ouch"; Then... console.log(mary.debt); // outputs "ouch" console.log(tony.debt); // outputs "ouch" console.log(peter.debt); // outputs "ouch" Now, when this approach is used, all instances of IrishPersonBoughtInTheBoom share the save copy of the debt property. This means, that they all have the same value as illustrated in this example. Returning functions A function always returns a value. If return is not specified for a function, the undefined value type will be returned. Javascript functions can also return some data or another function. var counter = function() { //... var count = 0; return function () { return count = count + 1; } } var nextValue = counter(); nextValue(); // outputs 1 nextValue(); // outputs 2 Note, in this case the inner function which is returned "closes" over the count variable - making it a closure - since it encapsulates its own count variable. This means it gets its own copy which is different to the variable return by nextValue.count. this keyword The this keyword in Java has different meanings, depending on the context it is used. In summary: In a method context, this refers to the object that contains the method. In a function context, this refers to the global object. Unless the function is a property of another object. In which case the this refers to that object. If this is used in a constructor, the this in the constructor function refers to the object which uses the constructor function. When the apply or call methods are used the value of this refers to what was explictly specified in the apply or call invocation. typeof typeof is a unary operator with one operand. It is used to determine the types of things (a bit like getClass() in Java). The values outputted by typeof are "number", "string", "boolean", "undefined", "function", "object". console.log(typeof "tony"); // outputs string console.log(typeof 6); // outputs number console.log(false); // outputs boolean console.log(this.doesNotExist); // outputs undefined if the global scope has no such var console.log(typeof function(){}); // outputs function console.log(typeof {name:"I am an object"}); //outputs object console.log(typeof ["I am an array"]) // typedef outputs object for arrays console.log(typeof null) // typedef outputs object for nulls Some implementations return "object" for typeof for regular expressions; others return "function". But the biggest problem with typeof is that it returns object for null. To test for null, use strict equality... if (myobject === null) { ... } Self-redefining functions This is a good performance technique. Suppose you have a function and the first time it is called you want it to perform some set up code that you never want to perfom again. You can execute the set up code and then make the function redefine itself after that so that the setup code is never re-excuted. var myFunction = function () { //set up code only to this once alert("set up, only called once"); // set up code now complete. // redefine function so that set up code is not re-executed myFunction = function() { alert("no set up code"); } } myFunction(); // outputs - Set up, only called once myFunction(); // outputs - no set up code this time myFunction(); // outputs - no set up code this time Note, any properties added to the set up part of this function will be lost when the function redefines itself. In addition, if this function is used with a different name (i.e. it is assigned to a variable), the re-definition will not happen and the set up code will re-execute. Scope In javascript there is a global scope and a function scope available for variables. The var keyword does not need to be used to define variable in the global scope but it must be used to define variable in the local function scope. When a variable is scoped to a local function shares the name with a global variable, the local scope takes precedence - unless var was not used to declare the local variable in which case any local references are pointing to the global reference. There is no block scope in javascript. By block we mean the code between {}, aka curly braces. var myFunction = function () { var noBlockScope = function ( ) { if (true) { // you'd think that d would only be visible to this if statement var d = 24; } if (true) { // this if statement can see the variable defined in the other if statement console.log(d); } } noBlockScope(); Single var pattern You can define all variables used by a function in one place. It is ensures tidy code and is considered best practise. function scrum() { var numberOfProps = 2, numberOfHookers = 1, numberOfSecondRows = 2, numberOfBackRow = 3 // function body... } If a variable is declared but not initialized with a value it will have the value undefined. Strict Equality In javascript it is possible to compare two objects using ==. However, in some cases this will perform type conversion which can yield unexpected equality matches. To ensure there is strict comparison (i.e. no type conversions) use the === syntax. console.log(1 == true) // outputs true console.log(1 === true) // outputs false console.log(45 == "45") // outputs true console.log(45 === "45") // outputs false Truthy and Falsey When javascript expects a boolean, you may specify a value of any type. Values that convert to true are said to be truthy and values that convert to false are said to be falsey. Example of truthy values are objects, arrays, functions, strings and numbers: // This will output 'Wow, they were all true' if ({} && {sillyproperty:"sillyvalue"} && [] && ['element'] && function() {} && "string" && 89) { console.log("wow, they were all true"); } Examples of falsey values are empty strings, undefined, null and the value 0. // This will out put: 'none of them were true' if (!("" || undefined || null || 0)) { console.log("none of them were true"); } Undefined and null In javascript, the undefined value means not initialised or unknown where null means an absence of a value. References JavaScript patterns Stoyan Stefanov JavaScript, The Definitive Guide David Flanagan JavaScript, The Good Parts Doug Crockford.
April 4, 2012
by Alex Staveley
· 31,413 Views
article thumbnail
Algorithm of the Week: Rabin-Karp String Searching
Brute force string matching is a very basic sub-string matching algorithm, but it’s good for some reasons. For example it doesn’t require preprocessing of the text or the pattern. The problem is that it’s very slow. That is why in many cases brute force matching can’t be very useful. For pattern matching we need something faster, but to understand other sub-string matching algorithms let’s take a look once again at brute force matching. In brute force sub-string matching we checked every single character from the text with the first character of the pattern. Once we have a match between them we shift the comparison between the second character of the pattern with the next character of the text, as shown on the picture below. This algorithm is slow for mainly two reasons. First, we have to check every single character from the text. On the other hand even if we find a match between a text character and the first character of the pattern we continue to check step by step (character by character) every single symbol of the pattern in order to find whether it is in the text. So is there any other approach to find whether the text contains the pattern? In fact there is a “faster” approach. In this case, in order to avoid the comparison between the pattern and the text character by character, we’ll try to compare them all at once, so we need a good hash function. With its help we can hash the pattern and check against hashed sub-strings of the text. We must be sure that the hash function is returning “small” hash codes for larger sub-strings. Another problem is that for larger patterns we can’t expect to have short hashes. But besides this the approach should be quite effective compared to the brute force string matching. This approach is known as Rabin-Karp algorithm. Overview Michael O. Rabin and Richard M. Karp came up with the idea of hashing the pattern and to check it against a hashed sub-string from the text in 1987. In general the idea seems quite simple, the only thing is that we need a hash function that gives different hashes for different sub-strings. Said hash function, for instance, may use the ASCII codes for every character, but we must be careful for multi-lingual support. The hash function may vary depending on many things, so it may consist of ASCII char to number converting, but it can also be anything else. The only thing we need is to convert a string (pattern) into some hash that is faster to compare. Let’s say we have the string “hello world”, and let’s assume that its hash is hash(‘hello world’) = 12345. So if hash(‘he’) = 1 we can say that the pattern “he” is contained in the text “hello world”. So in every step, we take from the text a sub-string with the length of m, where m is the pattern length. Thus we hash this sub-string and we can directly compare it to the hashed pattern, as in the picture above. Implementation So far we saw some diagrams explaining the Rabin-Karp algorithm, but let’s take a look at its implementation here, in this very basic example where a simple hash table is used in order to convert the characters into integers. The code is PHP and it’s used only to illustrate the principles of this algorithm. function hash_string($str, $len) { $hash = ''; $hash_table = array( 'h' => 1, 'e' => 2, 'l' => 3, 'o' => 4, 'w' => 5, 'r' => 6, 'd' => 7, ); for ($i = 0; $i < $len; $i++) { $hash .= $hash_table[$str{$i}]; } return (int)$hash; } function rabin_karp($text, $pattern) { $n = strlen($text); $m = strlen($pattern); $text_hash = hash_string(substr($text, 0, $m), $m); $pattern_hash = hash_string($pattern, $m); for ($i = 0; $i < $n-$m+1; $i++) { if ($text_hash == $pattern_hash) { return $i; } $text_hash = hash_string(substr($text, $i, $m), $m); } return -1; } // 2 echo rabin_karp('hello world', 'ello'); Multiple Pattern Match It’s great to say that the Rabin-Karp algorithm is great for multiple pattern match. Indeed its nature is supposed to support such functionality, which is its advantage in comparison to other string searching algorithms. Complexity The Rabin-Karp algorithm has the complexity of O(nm) where n, of course, is the length of the text, while m is the length of the pattern. So where is it compared to brute-force matching? Well, brute force matching complexity is O(nm), so as it seems there’s not much of a gain in performance. However, it’s considered that Rabin-Karp’s complexity is O(n+m) in practice, and that makes it a bit faster, as shown on the chart below. Note that the Rabin-Karp algorithm also needs O(m) preprocessing time. Application As we saw Rabin-Karp is not much faster than brute force matching. So where we should use it? 3 Reasons Why Rabin-Karp is Cool 1. Good for plagiarism, because it can deal with multiple pattern matching! 2. Not faster than brute force matching in theory, but in practice its complexity is O(n+m)! 3. With a good hashing function it can be quite effective and it’s easy to implement! 2 Reasons Why Rabin-Karp is Not Cool 1. There are lots of string matching algorithms that are faster than O(n+m) 2. It’s practically as slow as brute force matching and it requires additional space Final Words Rabin-Karp is a great algorithm for one simple reason – it can be used to match against multiple patterns. This makes it perfect to detect plagiarism even for larger phrases.
April 3, 2012
by Stoimen Popov
· 36,776 Views
article thumbnail
Wrapping Begin/End Async API Into C#5 Tasks
Microsoft offered programmers several different ways of dealing with the asynchronous programming since .NET 1.0. The first model was Asynchronous programming model or APM for short. The pattern is implemented with two methods named BeginOperation and EndOperation. .NET 4 introduced new pattern – Task Asynchronous Pattern and with the introduction of .NET 4.5, Microsoft added language support for language integrated asynchronous coding style. You can check the MSDN for more samples and information. I will assume that you are familiar with it and have written code using it. You can wrap existing APM pattern into TPL pattern using the Task.Factory.FromAsync methods. For example: public static Task> ExecuteAsync(this DataServiceQuery query, object state) { return Task.Factory.FromAsync>(query.BeginExecute, query.EndExecute, state); } It is easy to wrap most of the asynchronous functions this way, but some cannot be since the wrapper functions assume that the last two parameters to the BeginOperation are AsyncCallback and object, and there are some versions of asynchronous operations that have different specifications. Examples: Extra parameters after the object state parameter: IAsyncResult DataServiceContext.BeginExecuteBatch( AsyncCallback callback, object state, params DataServiceRequest[] queries); Missing the expected object state parameter and different return type: ICancelableAsyncResult BeginQuery(AsyncCallback callBack); WorkItemCollection EndQuery(ICancelableAsyncResult car); Short solution for the first example The short and elegant way for wrapping the first example is to provide the following wrapper: public static Task ExecuteBatchAsync(this DataServiceContext context, object state, params DataServiceRequest[] queries) { if (context == null) throw new ArgumentNullException("context"); return Task.Factory.FromAsync( context.BeginExecuteBatch(null, state, queries), context.EndExecuteBatch); } We simply call the Begin method ourselves and then wrap it using an another overload for FromAsync function. The longer way However, we can fully wrap it ourselves by simulating what the FromAsync wrapper does. The complete code is listed below. public static Task ExecuteBatchAsync(this DataServiceContext context, object state, params DataServiceRequest[] queries) { // this will be our sentry that will know when our async operation is completed var tcs = new TaskCompletionSource(); try { context.BeginExecuteBatch((iar) => { try { var result = context.EndExecuteBatch(iar as ICancelableAsyncResult); tcs.TrySetResult(result); } catch (OperationCanceledException ex) { // if the inner operation was canceled, this task is cancelled too tcs.TrySetCanceled(); } catch (Exception ex) { // general exception has been set bool flag = tcs.TrySetException(ex); if (flag && ex as ThreadAbortException != null) { tcs.Task.m_contingentProperties.m_exceptionsHolder.MarkAsHandled(false); } } }, state, queries); } catch { tcs.TrySetResult(default(DataServiceResponse)); // propagate exceptions to the outside throw; } return tcs.Task; } Besides educational benefits, writing the full wrapper code allows us to add cancellation, logging and diagnostic information. Once we understand how to wrap APM pattern, We can now tackle the second problem easily. Handling the BeginQuery/EndQuery We will first create our own wrapper function in the spirit of the above code with the notable difference that we use the ICancelableAsyncResult interface instead of the IAsyncResult. public static class TaskEx { public static Task FromAsync(Func beginMethod, Func endMethod) { if (beginMethod == null) throw new ArgumentNullException("beginMethod"); if (endMethod == null) throw new ArgumentNullException("endMethod"); var tcs = new TaskCompletionSource(); try { beginMethod((iar) => { try { var result = endMethod(iar as ICancelableAsyncResult); tcs.TrySetResult(result); } catch (OperationCanceledException ex) { tcs.TrySetCanceled(); } catch (Exception ex) { bool flag = tcs.TrySetException(ex); if (flag && ex as ThreadAbortException != null) { tcs.Task.m_contingentProperties.m_exceptionsHolder.MarkAsHandled(false); } } }); } catch { tcs.TrySetResult(default(TResult)); throw; } return tcs.Task; } } The code is pretty self-explanatory and we can go ahead with the wrapping. There are four different operations that are exposed both in synchronous and asynchronous version: Query, LinkQuery, CountOnlyQuery and RegularQuery. The extension methods are short since we have already created our generic wrapper above: public static Task RunQueryAsync(this Query query) { return TaskEx.FromAsync(query.BeginQuery, query.EndQuery); } public static Task RunLinkQueryAsync(this Query query) { return TaskEx.FromAsync(query.BeginLinkQuery, query.EndLinkQuery); } public static Task RunCountOnlyQueryAsync(this Query query) { return TaskEx.FromAsync(query.BeginCountOnlyQuery, query.EndCountOnlyQuery); } public static Task RunRegularQueryAsync(this Query query) { return TaskEx.FromAsync(query.BeginRegularQuery, query.EndRegularQuery); } That is it for today, you can write your own handy extensions easily for APM functions out there.
April 2, 2012
by Toni Petrina
· 11,332 Views
  • Previous
  • ...
  • 1567
  • 1568
  • 1569
  • 1570
  • 1571
  • 1572
  • 1573
  • 1574
  • 1575
  • 1576
  • ...
  • Next
  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook
×