Last week I tried to create a PHP stream context which set multiple headers; an Authorization header and a Content-Type header. All the examples I could find showed headers built up as a string with newlines added manually, which seemed pretty clunky and not-streams-like to me. In fact, you've been able to pass this as an array since PHP 5.2.10, so to set multiple headers in the stream context, I just used this: [ "method" => "POST", "header" => ["Authorization: token " . $access_token, "Content-Type: application/json"], "content" => $data ]]; $context = stream_context_create($options); The $access_token had been set elsewhere (in fact I usually put credentials in a separate file and exclude it from source control in an effort not to spread my access credentials further than I mean to!), and $data is already encoded as JSON. For completeness, you can make the POST request like this:

not a lot of developers today know assembly, which - regardless of your professional line of work - is a good skill to have. assembly teaches you think on a much lower level, going beyond the abstracted out layer provided by many of the high-level languages. today we're going to look at a way to implement a string concatenation function. specifically, i want to follow the following procedure for building the final result: ask the user for input append a crlf (carriage return + line feed) to the entered string append the entered string to the existing composite string follow back from step 1 until the user enters a terminator character display the composite string let's assume that you have zero knowledge of assembly. if that is the case, i would recommend starting here . in this example, i am using visual studio 2012 to test the code, but you might as well use an older version of the ide if you want. for convenience purposes, i would recommend downloading the basic framework code that comes for free from the writer of the introduction to 80x86 assembly language and computer architecture book: visual studio 2012 visual studio 2010 visual studio 2008 first, you have the standard declarations: .586 .model flat include io.h ; header file for input/output cr equ 0dh ; carriage return character lf equ 0ah ; line feed .stack 4096 .data prompt byte cr, lf, "original string? ",0 restitle byte "final result",0 stringin byte 1024 dup (?) stringout byte 1024 dup (?) linefeed byte cr, lf notice the reference to io.h - at this point you want a way to receive user input and display output data through standard winapi channels, and io.h does just that. some asm experts might argue that it is not a good idea to use winapi hooks in the context of a "pure" assembly program, for educational purposes, but in this situation the focus is on the inner workings of a different function. note: the program is adapted to the scenario where the execution of the string concatenation function is the sole purpose. as you will get a hang of the execution flow, you can easily adapt it to a scenario where some of the registers can be re-used. let's start by clearing the ecx and edx registers: .code _mainproc proc ; clear the ecx and edx registers because these will ; be used for length counters and sequential increments. xor ecx, ecx xor edx, edx once the strings will be entered by the user, i will need to find out the length of the string to append, in order to have a correct sequential memory address. now i need to get user input: input_data: ; prompt the user to enter the string he ultimately ; wants appended to the main string buffer. input prompt, stringin, 40 ; read ascii characters ; make sure that the string doesn't start with the $ character ; which would automatically mean that we need to terminate the ; reading process cmp stringin, '$' je done lea eax, [stringout + edx] ; destination address push eax ; push the destination on the stack lea eax, [stringin] ; source address push eax ; push the source on the stack call strcopy ; call the string copy procedure once the string is entered, i can check whether the terminator character - "$", was used. one of the great things about the cmp instruction is the fact that it checks the starting address of the entered string, therefore i can simply compare the entered data with a single character. in case the character is encountered, the program flow terminates at done, where the output is displayed: done: ; output the new data. output restitle, stringout mov eax, 0 ret strcopy is an internal procedure that will simply copy a string from one memory address to another: strcopy proc near32 push ebp mov ebp, esp push edi push esi pushf mov esi, [ebp+8] mov edi, [ebp+12] cld whilenonull: cmp byte ptr [esi], 0 je endwhilenonull movsb jmp whilenonull endwhilenonull: mov byte ptr [edi], 0 popf pop esi pop edi pop ebp ret 8 strcopy endp to make sure that the next string is properly appended, i need to find out the length of the previous one, for a correct memory address offset: ; let's get the length of the current string - move it ; to the proper register so that we can perform the measurement mov edi, eax ; find the length of the string that was just entered sub ecx, ecx sub al, al not ecx cld repne scasb not ecx dec ecx add edx, ecx repne scasb is used for an in-string iterative null terminator search (you can read more about it here ). it will decrement ecx for each character. ; we need to append the linefeed (crlf) to the string so we apply ; the same string concatenation procedure for that sequence. lea eax, [stringout + edx] ; destination address push eax ; first parameter lea eax, [linefeed] ; source push eax ; second parameter call strcopy ; call string copy procedure mov edi, eax ; we know that the crlf characters are 2 entities, therefore ; increment the overall counter by 2. add edx, 2 ; ask for more input because no terminator character was used. jmp input_data once the basic input data is processed, i can append the crlf sequence and increment edx for the proper offset, after which the program flow is being reset from the point where the user has to enter the next character sequence.

Introduction StringBuffer is a synchronized class for mutable strings. The main problem with making it synchronized is that It was usually used as a local variable so making it synchronized just made it slower. It was never a good idea to use it in a multi-threaded way. This problem is that developers assumed that methods which used StringBuffer were themselves thread safe when they were not. The problem with StringBuffer This is an example from a real class which is used in production in many trading systems. It's not a commonly used but you might assume that StringBuffer gives you thread safety, when it doesn't. private StringBuffer sb = new StringBuffer(); public void addProperty(String name, String value) { if (value != null && value.length() > 0) { if (sb.length() > 0) { sb.append(','); } sb.append(name).append('=').append(value); } } While individual calls are thread safe, multiple calls are not. It is almost impossible to find a good use for StringBuffer that doesn't involve multiple calls (including toString) A puzzle Imagine three threads call (in no particular order) T1: addProperty("a", "b"); T2: addProperty("c", "d"); T3: sb.toString(); write a program which will generate every possible output of T3's sb.toString() I found 89. With thread safety, you might reduce this to 4. Note: if you used StringBuilder it would be worse, but at least you might not assume your method is thread safe when it is not. e.g. SimpleDateFormat uses StringBuffer ;)

Originally authored by Jason Harwig The Square Engineering Blog has a great article on Smoother Signatures for Android, but I didn't find anything specifically about iOS. So, what is the best way to capture a users signature on an iOS device? Although I didn't find any articles on signature capture, there are good implementations on the App Store. My target user experience was the iPad application Paper by 53, a drawing application with beautiful and responsive brushes. All code is available in the Github repository: SignatureDemo. Connecting the Dots The simplest approach is to capture the touches and connect them with straight lines. In the initializer of a UIView subclass, create the path and gesture recognizer to capture touch events. // Create a path to connect lines path = [UIBezierPath bezierPath]; // Capture touches UIPanGestureRecognizer *pan = [[UIPanGestureRecognizer alloc] initWithTarget:self action:@selector(pan:)]; pan.maximumNumberOfTouches = pan.minimumNumberOfTouches = 1; [self addGestureRecognizer:pan]; Capture the pan events into a bézier path by connecting the points with lines. - (void)pan:(UIPanGestureRecognizer *)pan { CGPoint currentPoint = [pan locationInView:self]; if (pan.state == UIGestureRecognizerStateBegan) { [path moveToPoint:currentPoint]; } else if (pan.state == UIGestureRecognizerStateChanged) [path addLineToPoint:currentPoint]; [self setNeedsDisplay]; } Stroke the path - (void)drawRect:(CGRect)rect { [[UIColor blackColor] setStroke]; [path stroke]; } An example "J" character rendered using this technique reveals some issues. At slow velocities iOS captures enough touch resolution that the lines aren't noticeable, but faster movement shows large gaps between touches that accentuates the lines. The 2012 Apple Developer Conference included a session Building Advanced Gesture Recognizers that addresses this issue using math. Quadratic Bézier Curves Instead of connected lines between the touch points, quadratic bézier curves connect the points using the technique discussed in the aforementioned WWDC session (Seek to 42:15.) Connect the touch points with a quadratic curve using the touch points as the control points and the mid points as start and end. Adding quadratic curves to the previous code requires the storing the previous touch point, so add an instance variable for that. CGPoint previousPoint; Create a function to calculate the midpoint of two points. static CGPoint midpoint(CGPoint p0, CGPoint p1) { return (CGPoint) { (p0.x + p1.x) / 2.0, (p0.y + p1.y) / 2.0 }; } Update the pan gesture handler to add quadratic curves instead of straight lines - (void)pan:(UIPanGestureRecognizer *)pan { CGPoint currentPoint = [pan locationInView:self]; CGPoint midPoint = midpoint(previousPoint, currentPoint); if (pan.state == UIGestureRecognizerStateBegan) { [path moveToPoint:currentPoint]; } else if (pan.state == UIGestureRecognizerStateChanged) { [path addQuadCurveToPoint:midPoint controlPoint:previousPoint]; } previousPoint = currentPoint; [self setNeedsDisplay]; } Not much code and already we see a big difference. The touch points are no longer visible, but it looks a little bland when drawing a signature. Every curve is the same width, which doesn't match the physics of a real pen. Variable Stroke Width The width can be varied based on the touch velocity to create a more natural stroke. The UIPanGestureRecognizer already includes a method called velocityInView: that returns the current touch velocity as a CGPoint. To render a stroke of varying width, I switched to OpenGL ES and a technique called tesselation to convert the stroke into triangles – specifically, triangle strips (OpenGL has support for drawing lines, but iOS doesn't support variable line widths with smoothing.) The quadratic points along a curve also need to be calculated, but is beyond the scope of this article. Check the source on github for details. Given two points, a perpendicular vector is calculated and its magnitude set to half the current thickness. Given the nature of GL_TRIANGLE_STRIP only two points are needed to create the next rectangle segment with two triangles. Here is an example of the final output using quadratic bézier curves, and velocity based stroke thickness creating a visually appealing and natural signature.

let’s say you have a large piece of text and a dictionary of keywords. how do you quickly locate all the keywords? aho-corasick algorithm diagram well, there are many ways really, you could even iterate through the whole thing and compare words to keywords. but it turns out that’s going to be very slow. at least o(n_keywords * n_words) complexity. essentially you’re making as many passes over the text as your dictionary is big. in 1975 a couple of ibm researchers – alfred aho and margaret corasick – discovered an algorithm that can do this in a single pass. the aho-corasick string matching algorithm . i implemented it in haskell and it takes 0.005s to find 8 different keywords in oscar wilde’s the nightingale and the rose – a 12kb text. a quick naive keyword search implemented in python takes 0.023s . not a big difference practically speaking, but imagine a situation with megabytes of text and thousands of words in the dictionary. the authors mention printing out the result as a major bottleneck in their assessment of the algorithm. yep, printing . the aho-corasick algorithm at the core of this algorithm are three functions: the three functions of aho-corasick algorithm a parser based on a state machine, which maps (state, char) pairs to states and occasionally emits an output. this is called the goto function a failure function, which tells the goto function which state to jump into when the character it just read doesn’t match anything an output function, which maps states to outputs – potentially more than one per state the algorithm works in two stages. it will first construct the goto, failure and output functions. the complexity of this operation hinges solely on the size of our dictionary. then it iterates over the input text to produce all the matches. using state machines for parsing text is a well known trick – the real genius of this algorithm rests in that failure function if you ask me. it makes lateral transitions between states when the algorithm climbs itself into a wall. say you have she and hers in the dictionary. the goto machine eats your input string one character at the time. let’s say it’s already read s h . the next input is an e so it outputs she and reaches a final state. next it reads an r , but the state didn’t expect any more inputs, so the failure function puts us on the path towards hers . this is a bit tricky to explain in text, i suggest you look at the picture from the original article and look at what’s happening. my haskell implementation the first implementation i tried, relied on manully mapping inputs to outputs for the goto, failure and output functions by using pattern recognition. not very pretty, extremely hardcoded, but it worked and was easy to make. building the functions dynamically proved a bit trickier. type goto = map (int, char) int type failure = map int int type output = map int [string] first off, we build the goto function. -- builds the goto function build_goto::goto -> string -> (goto, string) build_goto m s = (add_one 0 m s, s) -- adds one string to goto function add_one::int -> goto -> [char] -> goto add_one _ m [] = m add_one state m (c:rest) | member key m = add_one (frommaybe 0 $ map.lookup key m) m rest | otherwise = add_one max (map.insert key max m) rest where key = (state, c) max = (size m)+1 essentially this builds a flattened prefix tree in a hashmap of (state, char) pairs mapping to the next state. it makes sure to avoid adding new edges to the three as much as possible. the reason it’s not simply a prefix tree are those lateral transitions; doing them in a tree would require backtracking and repeating of steps, so we haven’t achieved anything. once we have the goto function, building the output is trivial. -- builds the output function build_output::(?m::goto) => [string] -> output build_output [] = empty build_output (s:rest) = map.insert (fin 0 s) (list.filter (\x -> elem x dictionary) $ list.tails s) $ build_output rest -- returns the state in which an input string ends without using failures fin::(?m::goto) => int -> [char] -> int fin state [] = state fin state (c:rest) = fin next rest where next = frommaybe 0 $ map.lookup (state, c) ?m we are essentially going over the dictionary, finding the final state for each word and building a hash table mapping final states to their outputs. building the failure function was trickiest, because we need a way to iterate over the depths at which nodes are position in the goto state machine. but we threw that info away by using a hashmap. -- tells us which nodes in the goto state machine are at which traversal depth nodes_at_depths::(?m::goto) => [[int]] nodes_at_depths = list.map (\i -> list.filter (>0) $ list.map (\l -> if i < length l then l!!i else -1) paths) [0..(maximum $ list.map length paths)-1] where paths = list.map (path 0) dictionary we now have a list of lists, that tells us at which depth certain nodes are. -- builds the failure function build_fail::(?m::goto) => [[int]] -> int -> failure build_fail nodes 0 = fst $ mapaccuml (\f state -> (map.insert state 0 f, state)) empty (nodes!!0) build_fail nodes d = fst $ mapaccuml (\f state -> (map.insert state (decide_fail state lower) f, state)) lower (nodes!!d) where lower = build_fail nodes (d-1) -- inner step of building the failure function decide_fail::(?m::goto) => int -> failure -> int decide_fail state lower = findwithdefault 0 (s, c) ?m where (s', c) = key' state $ assocs ?m s = findwithdefault 0 s' lower -- gives us the key associated with a certain state (how to get there) key'::int -> [((int, char), int)] -> (int, char) key' _ [] = (-1, '_') -- this is ugly, being of maybe type would be better key' state ((k, v):rest) | state == v = k | otherwise = key' state rest here we are going over the list of nodes at depths and deciding what the failure should be for each depth based on the failures of depth-1. at depth zero, all failures go to the zeroth state. an important part of this process was inverting the goto hashmap so values point to keys, which is essentially what the key’ function does. finally, we can use the whole algorithm like this: main = do let ?m = fst $ mapaccuml build_goto empty dictionary let ?f = build_fail nodes_at_depths $ (length $ nodes_at_depths)-1 ?out = build_output dictionary print $ ahocorasick text a bit more involved than the usual example of haskell found online, it’s still pretty cool you can see the whole code on github here .

Let’s say you are writing an application in Ruby. You are probably talking to every API under the sun and are happily writing tests to make sure your code isn’t failing. Because you don’t want to rely on 3rd parties or an internet connection to make your tests pass or fail you mock everything with let’s say, Webmock. This also makes your tests much much faster. After all even the fastest internet is much slower than the processor talking to its memory. If you’re too lazy to mock out every API under the sun, you might use VCR to record requests and play them back later. The main advantage being, you don’t have to worry about meticulously reimplementing everything, and you can nuke the recordings at any time to make sure your code still works against the real API. Life is good. Enter Javascript, stage left Then Javascript becomes more and more prominent. Suddenly your application’s logic is shifting from backend to browser and before you know it, most of your tests are pretty irrelevant. You’re fine for a while with Capybara or Cucumber. Launch a headless browser, click around the site from the comfort of RSpec, make sure users see what they’re supposed to. Balance restored. Then you add a payment form. Or something. Suddenly your frontend is talking to an API. In case of Stripe or Balanced it’s even a feature. A great benefit for the user. jQuery(function($) { $('#payment-form').submit(function(event) { var $form = $(this); // Disable the submit button to prevent repeated clicks $form.find('button').prop('disabled', true); Stripe.createToken($form, stripeResponseHandler); // Prevent the form from submitting with the default action return false; }); }); Well that sucks, you’re suddenly back to square one. Your tests take minutes to execute. Your tests fail without an internet connection. Your tests rely on some 3rd party service being up. Your tests suck. Who wants to code when running ~5 tests takes 3 minutes? Nobody. Enter puffing-billy, stage right The problem is that neither Webmock nor VCR can handle requests originating in a browser because they happen in a different thread and they can’t mess around with those. Luckily, a year ago Olly Smith, created puffing-billy. The idea was great – spin up a web proxy, tell your headless browser to use it, when your code makes a request it will go through the proxy, which will try to use a Webmock to handle it, otherwise pass it on to the vast internet. But who wants to mock everything out manually? Over the past few weeks I set upon the task of fixing this problem and restoring sanity to my life. Good tests are transparent to the application and I’ll be damned if I use any of the suggested solutions on the internet like “Well you just put a switch in your code that knows if you’re in a test and then doesn’t talk to Stripe” Screw that. This morning I submitted a pull request to puffing-billy. I added the ability for puffing-billy to behave like it was VCR, but for your browser. When a request is made, it gets cached. The cache is then persisted between sessions, and requests are played back to the browser as needed. It’s not as sophisticated as VCR just yet, but it gets the job done and my test runtime has gone from 3 minutes to just under a minute. That’s a big deal in my book! The caching even understands that some URL’s are needlessly different on every request (social buttons, analytics etc.) so you can configure it to normalize those requests to a single recording that is played back every time. Your tests don’t really rely on gAnalytics working right? And the best thing is, you don’t even have to change your tests. You add something like this in your spec_helper.rb: Billy.configure do |c| c.cache = true c.ignore_params = ["http://www.google-analytics.com/__utm.gif", "http://b.siftscience.com/i.gif", "https://r.twimg.com/jot", "http://p.twitter.com/t.gif", "http://p.twitter.com/f.gif", "http://www.facebook.com/plugins/like.php", "https://www.facebook.com/dialog/oauth", "http://cdn.api.twitter.com/1/urls/count.json"] c.persist_cache = true c.cache_path = 'spec/req_cache/' end # need to call this because of a race condition between persist_cache # being set and the proxy being loaded for the first time Billy.proxy.restore_cache Capybara.javascript_driver = :poltergeist_billy A test for the payment form looks the same as usual: scenario "physical product" do product = start_buying build(:product, :physical, user: @seller, active: true) VCR.use_cassette('Balanced/purchase_with_cc') do within '#new_order' do fill_in 'order_email', with: Faker::Internet.safe_email fill_in_address fill_in_card click_on 'Buy Now' end page.should have_css('#receipt', :visible => true) end validate_receipt product, @seller end Puffing-billy will transparently cache every requests the browser makes and VCR records any requests made by your backend logic. It’s pretty sweet. What do you guys think? I only have 20 days of Ruby experience and the internet has told me it really wants something like this, but I couldn’t find anyone who’s already made it.

node.js is well-know for its good connectivity with nosql databases. a less know fact is that it's also very efficient with relational databases. among the dozens orms out there in javascript, one stands out for relational databases: sequelize . it's quite easy to learn but there are not many pointers about how to organize model code with this module. here are a few tips we learned by using sequelize in a medium size project. sequelize 101 sequelize claims to supports mysql, postgresql and sqlite. the sequelize docs explain the first steps with the javascript orm. first, initialize a database connection, then a few models, without worrying about primary or foreign keys: var sequelize = new sequelize('database', 'username'[, 'password']) var project = sequelize.define('project', { title: sequelize.string, description: sequelize.text }); var task = sequelize.define('task', { title: sequelize.string, description: sequelize.text, deadline: sequelize.date }); project.hasmany(task); next, create new instances and persist them, query the database, etc. // create an instance var task = task.build({title: 'very important task'}) task.title // ==> 'very important task' // persist an instance task.save() .error(function(err) { // error callback }) .success(function() { // success callback }); // query persistence for instances var tasks = task.all({ where: ['dealdine < ?', new date()] }) .error(function(err) { // error callback }) .success(function() { // success callback }); sequelize uses promises so you can chain error and success callbacks, and it all plays well with unit tests. all that is pretty well documented, but the sequelize documentation only covers the basic usage. once you start using sequelize in real world projects, finding the right way to implement a feature gets trickier. model file structure all the examples in the sequelize documentation show all model declarations grouped in a single file. once a project reaches production size, this is not a viable approach. the alternative is to import models from a module using sequelize.import() . the problem is that relationships rely on several models. when you declare a relationship, models from both sides of the relationship must already be imported. you should not import model files from other model files because of node.js module caching policy (more on that later on); instead, you can define relationships in a standalone file. here is the file structure we've been working with: models/ index.js # import all models and creates relationships phonenumber.js task.js user.js ... and here is how the main models/index.js initializes the entire model: var sequelize = require('sequelize'); var config = require('config').database; // we use node-config to handle environments // initialize database connection var sequelize = new sequelize( config.name, config.username, config.password, config.options ); // load models var models = [ 'phonenumber', 'task', 'user' ]; models.foreach(function(model) { module.exports[model] = sequelize.import(__dirname + '/' + model); }); // describe relationships (function(m) { m.phonenumber.belongsto(m.user); m.task.belongsto(m.user); m.user.hasmany(m.task); m.user.hasmany(m.phonenumber); })(module.exports); // export connection module.exports.sequelize = sequelize; using models in code from other parts of the application, if you need a model class, require the models/index.js instead of the standalone model file. that way, you don't have to repeat the sequelize initialization. var models = require('./models'); var user = models.user; var user = user.build({ first_name: "john", last_name: "doe "}); the problem is, when you require the models/index.js file, node may use a cached version of the module... or not: from nodejs.org : multiple calls to require('foo') may not cause the module code to be executed multiple times. (...) modules are cached based on their resolved filename. since modules may resolve to a different filename based on the location of the calling module (loading from node_modules folders), it is not a guarantee that require('foo') will always return the exact same object, if it would resolve to different files. that means that using require('./models') to get the models may create more than one connection to the database. to avoid that, the models variable must be somehow singleton-esque. this can be easily achieved, if you're using a framework like expressjs , by attaching the models module to the application: app.set('models', require('./models')); and when you need to require a class of the model in a controller, use this application setting rather than a direct import: var user = app.get('models').user; accessing other models sequelize models can be extended with class and instance methods. you can add abilities to model classes, much like in a true activerecord implementation. but a problem arises when adding a method that depends on another model: how can a model access another one? // in models/user.js module.exports = function(sequelize, datatypes) { return sequelize.define('user', { first_name: datatypes.string, last_name: datatypes.string, }, { instancemethods: { counttasks: function() { // how to implement this method ? } } }); }; if the two models share a relationship, there is a way. here, one user has many tasks , that makes the task model accessible through user.associations['tasks'].target . and here is yet another problem: since sequelize doesn't use prototype-based inheritance, how can a user instance gain access to the user class? digging into the sequelize source brings the protected __factory to the light. with all this undocumented knowledge, it is now possible to write the counttasks() instance method: counttasks: function() { return this.__factory.associations['tasks'].target.count({ where: { user_id: this.id } }); } note that task.count() returns a promise, so counttasks() also returns a promise: user.counttasks().success(function(nbtasks) { // do somethig with the user task count }); extending models (a.k.a behaviors) what if you need to reuse several methods across several models? sequelize doesn't have a behavior system per se (or "concerns" in the ruby on rails terminology), although it's quite easy to implement . for now, you're condemned to import common methods before the call to sequelize.define() and use sequelize.utils._.extend() to add it to the instancemethods or classmethods object: // in models/friendlyurl.js module.exports = function(keys) { return { geturl: function() { var ret = ''; keys.foreach(function(key) { ret += this[key]; }) return ret .tolowercase() .replace(/^\s+|\s+$/g, "") // trim whitespace .replace(/[_|\s]+/g, "-") .replace(/[^a-z0-9-]+/g, "") .replace(/[-]+/g, "-") .replace(/^-+|-+$/g, ""); } }; } // in models/user.js var friendlyurlmethods = require('./friendlyurl')(['first_name', 'last_name']); module.exports = function(sequelize, datatypes) { return sequelize.define('user', { first_name: datatypes.string, last_name: datatypes.string, }, { instancemethods: sequelize.utils._.extend({}, friendlyurlmethods, { counttasks: function() { return this.__factory.associations['tasks'].target.count({ where: { user_id: this.id } }); } }); }) }; now the user model instances gain access to a geturl() method: var user = user.build({ first_name: 'john', last_name: 'doe' }); user.geturl(); // 'john_doe' a limitation of this trick is that you must define behaviors before the actual model. this forbids behaviors accessing other models. query series sequelize provides a tool called the querychainer to ease the resynchronization of queries. new sequelize.utils.querychainer() .add(user, 'find', [id]) .add(task, 'findall') .error(function(err) { /* hmm not good :> */ }) .success(function(results) { var user = results[0]; var tasks = results[1]; // do things with the results }); after using it a little, this utility turns out to be very limited. most notably, querychainer executes all queries in parallel by default. and you only get access to the results of the queries in the final callback - no way to pass values from one query to the other. we've found it much more convenient to use a generic resynchronizing module like async , which provides the wonderful async.auto() utility. this method lets you list tasks together with dependencies, and determines which task can be run in parallel, and which must be run in series. async.auto([ user: function(next) { user.find(id).complete(next); }, tasks: ['user', function(next) { tasks.findall({ where: { user_id: user.id } }).complete(next); }] ], function(err, results) { var user = results.user; var tasks = results.tasks; // do things with the results }); notice the complete() method, which is an alternative to the two success() and error() callbacks. complete() accepts a callback with the signature (err, res) , which is more natural in the node.js world, and compatible with async . prefetching one thing orms are usually good at is minimizing queries. sequelize offers a prefetching feature, allowing to group two queries in a single one using a join. for instance, if you want to retrieve a task together with the related user, write the query as follows: task.find({ where: { id: id } }, include: ['user']) .error(function(err) { // error callback }) .success(function(task) { task.getuser(); // does not trigger a new query }); this is another undocumented feature, although the documentation should be updated soon . migrations sequelize provides a migration command line utility. but because it only allows modifying the model using sequelize commands (and not calling any asynchronous method of your own ), this migration command falls short. as of now, we've been handling migrations manually using numbered sql files and a custom utility to run them in order. custom sql queries sequelize is built over database adapters, and as such provides a way to execute arbitrary sql queries against the database. here is an example: var util = require('util'); var query = 'select * from `task` ' + 'left join `user` on `task`.`userid` = `user`.`id` ' + 'where `user`.`last_name` = %s'; var escapedname = sequelize.constructor.utils.escape(last_name); querywithparams = util.format(query, escapedname); sequelize.query(querywithparams, task) .error(function(err) { // error callback }) .success(function(tasks) { task.getuser(); // does not trigger a new query }); sequelize.query() returns a promise just like other query functions. if you provide the model to use for hydration ( task in this case), the query() method returns model instances rather than a simple json. note that you must escape values by hand before concatenating them into the sql query. for strings, sequelize.constructor.utils.escape() is your friend. for integers, util.format('%d') should do the trick. conclusion is sequelize ready for prime time ? almost. the learning curve is made longer by an incomplete documentation, but most of the features required by a production-level orm are there. however, i wouldn't recommend it for production just yet if you're not ready to run on your own fork, since the rate at which prs are merged in the sequelize github repository is low.

This article wants to bring some hints about how to use HTML5 canvas with Apache Wicket web framework. Inside a Wicket application we want to have a panel with something drawn inside a HTML5 canvas. To make this happen we have to think about following: Do we really need HTML5? If we need HTML5, how to do it? What to do if browser version is an issue and it does not support HTML5? 1. First we should ask if we really need HTML5 If we need just an image then we should consider to draw inside a Java2D Graphics object. If we need some animation we should consider to draw inside a HTML5 canvas, but even in this case we need a simple Java2D image implementation if browser version is a concern and canvas is not supported. Wicket has a RenderedDynamicImageResource class which is very handy for this because we can do Java2D stuff inside render(Graphics2D g2) method. A simple example may look like the following: public class MyDynamicImageResource extends RenderedDynamicImageResource { private int width; private int height; private MyData data; public MyDynamicImageResource (int width, int height, MYData data) { super(width, height); this.width = width; this.height = height; this.data = data; } protected boolean render(Graphics2D g2) { g2.setRenderingHint(RenderingHints.KEY_ANTIALIASING, RenderingHints.VALUE_ANTIALIAS_ON); g2.setRenderingHint(RenderingHints.KEY_RENDERING, RenderingHints.VALUE_RENDER_QUALITY); // your code } } Because Java2d is used, we can set anti-aliasing to make the image look good. Then we can use this dynamic resource to create our panel. We will use Wicket's NonCachingImage class, a subclass of Image that adds random noise to the url at every request to prevent the browser from caching the image. If you do not care that browser caches your image then you should use a simple Image instead. public class MyJava2DImagePanel extends Panel { private MyDynamicImageResource imageResource; public MyJava2DImagePanel(String id, final int width, final int height, final IModel model) { super(id, model); NonCachingImage image = new NonCachingImage("myImage", new PropertyModel(this, "imageResource")) { private static final long serialVersionUID = 1L; @Override protected void onBeforeRender() { imageResource = new MyDynamicImageResource(width, height, model.getObject()); super.onBeforeRender(); } }; add(image); } } Markup html file for MyJava2DImagePanel will contain the image: 2. If we need some animation for our image, then we should think to draw it on a HTML5 canvas. We should pay attention to draw things just once, meaning for example if we draw a text twice in same position , then our result will look ugly (pix-elated) because an anti-aliasing for canvas cannot be set as for Java2D Graphics object. First we need to create our java script code. We can obtain a Java 2d context and use it to draw our image. I won't talk about canvas context and its methods here. For animation we use jquery in following snippet, but you can use anything you like. Knowing two values (from, to) we can have for example a drawColor method which can paint different segments, creating this way a filling effect which takes in this example 1000ms : var myWidget = function(id, color) { var can = document.getElementById(id); var ctx = can.getContext('2d'); // clear canvas ctx.clearRect(0, 0, can.width, can.height); // draw your image on ctx ..... // animate color fill $({ n: from }).animate({ n: to}, { duration: 1000, step: function(now, fx) { drawColor(id, now); } }); } } Second we have to create our Wicket panel. Canvas is just a WebMarkupContainer and we set width and height through some AttributeAppenders: public class MyHTML5Panel extends Panel { private final ResourceReference MY_JS = new JavaScriptResourceReference(MyHTML5Panel.class, "my.js"); public MyHTML5Panel(String id, String width, String height, IModel model) { super(id, model); WebMarkupContainer container = new WebMarkupContainer("canvas"); container.setOutputMarkupId(true); container.add(new AttributeAppender("width", width)); container.add(new AttributeAppender("height", height)); add(container); } @Override public void renderHead(IHeaderResponse response) { response.renderOnLoadJavaScript(getJavascriptCall()); //include js file response.renderJavaScriptReference(MY_JS); } private String getJavascriptCall() { MyData data = getModel().getObject(); StringBuilder sb = new StringBuilder(); sb.append("myWidget(\""). append(get("canvas").getMarkupId()). append("\",\"").append(data.getColor()). append("\");"); return sb.toString(); } } renderHead(IHeaderResponse response) method from Panel can use the IHeaderResponse object to render our java script call. Also, on the response object we should render our java script reference file. We can use one of the following methods: /** * Renders javascript that is executed right after the DOM is built, before external resources * (e.g. images) are loaded. * * @param javascript */ public void renderOnDomReadyJavaScript(String javascript); /** * Renders javascript that is executed after the entire page is loaded. * * @param javascript */ public void renderOnLoadJavaScript(String javascript); There are situations when we should call one or another depending on our business. As an example, if we need to expose our wicket component to an external iframe, we must call onLoad instead of onDomReady to make it appear inside iframe because $(document).ready in the iframe seems to be fired too soon and the iframe content isn't even loaded yet. HTML markup file MyHTML5Panel.html will contain the canvas tag: 3. If we choose to use HTML5 panel but we also have to think about older browser that cannot support canvas tag, we will have to create both a Java2D and a HTML5 panel and see what to render by ourselves. A solution is to have a wrapper panel with a container which initially contains an EmptyPanel and we add a Wicket Behavior to the container. That behavior will choose what to render (html5 or simple image): ..... container = new WebMarkupContainer("container"); container.setOutputMarkupId(true); container.add(new EmptyPanel("image")); add(container); add(new MyHTML5Behavior()); ....... The following java-script code is a way to test if canvas tag is supported by browser: function isCanvasEnabled() { return !!document.createElement('canvas').getContext; } This function starts by creating a dummy element which is never attached to the page, so no one will ever see it. As soon as we create the dummy element, we test for the presence of a getContext() method. This method will only exist if browser supports the canvas API. Finally, we use the double-negative trick to force the result to a Boolean value (true or false). To call this java script and make the result available to Wicket we use wicketAjaxGet javascript method as seen in following code. We append a result parameter to callback url and inside respond method we can read the value of this parameter. class MyHTML5Behavior extends AbstractDefaultAjaxBehavior { private String width; private String height; private String PARAM = "Param"; public MyHTML5Behavior() { super(); } @Override public void renderHead(Component component, IHeaderResponse response) { super.renderHead(component, response); //include js file response.renderJavaScriptReference(MY_UTIL_JS); response.renderOnLoadJavaScript(getJavascript()); } @Override protected void respond(AjaxRequestTarget target) { String param = this.getComponent().getRequest().getRequestParameters().getParameterValue(PARAM).toString(); // test if html5 canvas tag is supported if (Boolean.parseBoolean(param)) { container.replace(new MyHTML5Panel("image", width, height, model).setOutputMarkupId(true)); } else { container.replace(new MyImagePanel("image", width, height, model).setOutputMarkupId(true)); } target.add(container); } // this javascript call will make the PARAM available to wicket and can be read in respond method private String getJavascript() { StringBuilder sb = new StringBuilder(); sb.append("var data = isCanvasEnabled();"); sb.append("wicketAjaxGet('" + getCallbackUrl() + "&" + PARAM + "='+ data" + ", null, null, function() { return true; })"); return sb.toString(); } } These are just some hints on how to use HTML5 canvas inside Apache Wicket framework. I hope it will help others.

Almost all the implementation I see today are based on OAuth 2.0 Bearer Token Profile. Of course its an RFC proposed standard today. OAuth 2.0 Bearer Token profile brings a simplified scheme for authentication. This specification describes how to use bearer tokens in HTTP requests to access OAuth 2.0 protected resources. Any party in possession of a bearer token (a "bearer") can use it to get access to the associated resources (without demonstrating possession of a cryptographic key). To prevent misuse, bearer tokens need to be protected from disclosure in storage and in transport. Before dig in to the OAuth 2.0 MAC profile lets have quick high-level overview of OAuth 2.0 message flow. OAuth 2.0 has mainly three phases. 1. Requesting an Authorization Grant. 2. Exchanging the Authorization Grant for an Access Token. 3. Access the resources with the Access Token. Where does the token type come in to action ? OAuth 2.0 core specification does not mandate any token type. At the same time at any point token requester - client - cannot decide which token type it needs. It's purely up to the Authorization Server to decide which token type to be returned in the Access Token response. So, the token type comes in to action in phase-2 when Authorization Server returning back the OAuth 2.0 Access Token. The access token type provides the client with the information required to successfully utilize the access token to make a protected resource request (along with type-specific attributes). The client must not use an access token if it does not understand the token type. Each access token type definition specifies the additional attributes (if any) sent to the client together with the "access_token" response parameter. It also defines the HTTP authentication method used to include the access token when making a protected resource request. For example following is what you get for Access Token response irrespective of which grant type you use. HTTP/1.1 200 OK Content-Type: application/json;charset=UTF-8 Cache-Control: no-store Pragma: no-cache { "access_token":"mF_9.B5f-4.1JqM", "token_type":"Bearer", "expires_in":3600, "refresh_token":"tGzv3JOkF0XG5Qx2TlKWIA" } The above is for Bearer - following is for MAC. HTTP/1.1 200 OK Content-Type: application/json Cache-Control: no-store { "access_token":"SlAV32hkKG", "token_type":"mac", "expires_in":3600, "refresh_token":"8xLOxBtZp8", "mac_key":"adijq39jdlaska9asud", "mac_algorithm":"hmac-sha-256" } Here you can see MAC Access Token response has two additional attributes. mac_key and the mac_algorithm. Let me rephrase this - "Each access token type definition specifies the additional attributes (if any) sent to the client together with the "access_token" response parameter". This MAC Token Profile defines the HTTP MAC access authentication scheme, providing a method for making authenticated HTTP requests with partial cryptographic verification of the request, covering the HTTP method, request URI, and host. In the above response access_token is the MAC key identifier. Unlike in Bearer, MAC token profile never passes it's top secret over the wire. The access_token or the MAC key identifier is a string identifying the MAC key used to calculate the request MAC. The string is usually opaque to the client. The server typically assigns a specific scope and lifetime to each set of MAC credentials. The identifier may denote a unique value used to retrieve the authorization information (e.g. from a database), or self-contain the authorization information in a verifiable manner (i.e. a string consisting of some data and a signature). The mac_key is a shared symmetric secret used as the MAC algorithm key. The server will not reissue a previously issued MAC key and MAC key identifier combination. Now let's see what happens in phase-3. Following shows how the Authorization HTTP header looks like when Bearer Token been used. Authorization: Bearer mF_9.B5f-4.1JqM This adds very low overhead on client side. It simply needs to pass the exact access_token it got from the Authorization Server in phase-2. Under MAC token profile, this is how it looks like. Authorization: MAC id="h480djs93hd8", ts="1336363200", nonce="dj83hs9s", mac="bhCQXTVyfj5cmA9uKkPFx1zeOXM=" This needs bit more attention. id is the MAC key identifier or the access_token from the phase-2. ts the request timestamp. The value is a positive integer set by the client when making each request to the number of seconds elapsed from a fixed point in time (e.g. January 1, 1970 00:00:00 GMT). This value is unique across all requests with the same timestamp and MAC key identifier combination. nonce is a unique string generated by the client. The value is unique across all requests with the same timestamp and MAC key identifier combination. The client uses the MAC algorithm and the MAC key to calculate the request mac. This is how you derive the normalized string to generate the HMAC. The normalized request string is a consistent, reproducible concatenation of several of the HTTP request elements into a single string. By normalizing the request into a reproducible string, the client and server can both calculate the request MAC over the exact same value. The string is constructed by concatenating together, in order, the following HTTP request elements, each followed by a new line character (%x0A): 1. The timestamp value calculated for the request. 2. The nonce value generated for the request. 3. The HTTP request method in upper case. For example: "HEAD", "GET", "POST", etc. 4. The HTTP request-URI as defined by [RFC2616] section 5.1.2. 5. The hostname included in the HTTP request using the "Host" request header field in lower case. 6. The port as included in the HTTP request using the "Host" request header field. If the header field does not include a port, the default value for the scheme MUST be used (e.g. 80 for HTTP and 443 for HTTPS). 7. The value of the "ext" "Authorization" request header field attribute if one was included in the request (this is optional), otherwise, an empty string. Each element is followed by a new line character (%x0A) including the last element and even when an element value is an empty string. Either you use Bearer of MAC - the end user or the resource owner is identified using the access_token. Authorization, throttling, monitoring or any other quality of service operations can be carried out against the access_token irrespective of which token profile you use.

This is a continuation from the previous post. Specification Many games have a start screen or main menu of some sort. (Though I love games like Braid that bypass the whole notion.) Let’s begin by designing our start screen. We’ll have a solid color background. Perhaps the ever lovely cornflower blue. Then we’ll draw the name of our game and provide an instruction to the player. In order to make sure we have the player’s attention, we’ll animate the color of the instruction. It will morph from black to red and back again. Finally, when the player clicks the screen we’ll transition to the main game. Or at least we’ll stub out the transition. Here’s a demo based on the code we’ll cover later in this post (as well as that from the previous post.) Implementation Here’s the code to implement our start screen. // `input` will be defined elsewhere, it's a means // for us to capture the state of input from the player var startScreen = (function(input) { // the red component of rgb var hue = 0; // are we moving toward red or black? var direction = 1; var transitioning = false; // record the input state from last frame // because we need to compare it in the // current frame var wasButtonDown = false; // a helper function // used internally to draw the text in // in the center of the canvas (with respect // to the x coordinate) function centerText(ctx, text, y) { var measurement = ctx.measureText(text); var x = (ctx.canvas.width - measurement.width) / 2; ctx.fillText(text, x, y); } // draw the main menu to the canvas function draw(ctx, elapsed) { // let's draw the text in the middle of the canvas // note that it's ineffecient to calculate this // in every frame since it never changes // however, I leave it here for simplicity var y = ctx.canvas.height / 2; // create a css color from the `hue` var color = 'rgb(' + hue + ',0,0)'; // clear the entire canvas // (this is not strictly necessary since we are always // updating the same pixels for this screen, however it // is generally what you need to do.) ctx.clearRect(0, 0, ctx.canvas.width, ctx.canvas.height); // draw the title of the game // this is static and doesn't change ctx.fillStyle = 'white'; ctx.font = '48px monospace'; centerText(ctx, 'My Awesome Game', y); // draw instructions to the player // this animates the color based on the value of `hue` ctx.fillStyle = color; ctx.font = '24px monospace'; centerText(ctx, 'click to begin', y + 30); } // update the color we're drawing and // check for input from the user function update() { // we want `hue` to oscillate between 0 and 255 hue += 1 * direction; if (hue > 255) direction = -1; if (hue < 0) direction = 1; // note that this logic is dependent on the frame rate, // that means if the frame rate is slow then the animation // is slow. // we could make it indepedent on the frame rate, but we'll // come to that later. // here we magically capture the state of the mouse // notice that we are not dealing with events inside the game // loop. // we'll come back to this too. var isButtonDown = input.isButtonDown(); // we want to know if the input (mouse click) _just_ happened // that means we only want to transition away from the menu to the // game if there _was_ input on the last frame _but none_ on the // current one. var mouseJustClicked = !isButtonDown && wasButtonDown; // we also check the value of `transitioning` so that we don't // initiate the transition logic more the once (like if the player // clicked the mouse repeatedly before we finished transitioning) if (mouseJustClicked && !transitioning) { transitioning = true; // do something here to transition to the actual game } // record the state of input for use in the next frame wasButtonDown = isButtonDown; } // this is the object that will be `startScreen` return { draw: draw, update: update }; }()); Explanation Recall that our start screen is meant to be invoked by our game loop. The game loop doesn’t know about the specifics of the start screen, but it does expect it to have a certain shape. This enables us to swap out screen objects without having to modify the game loop itself. The shape that the game loop expects is this: { update: function(timeElapsedSinceLastFrame) { }, draw: function(drawingContext) { } } Update Let’s begin with the start screen’s update function. The first bit of logic is this: hue += 1 * direction; if (hue > 255) direction = -1; if (hue < 0) direction = 1; Perhaps hue is not the best choice of variable names. It represents the red component for an RGB color value. The range of values for this component is 0 (no red) to 255 (all the reds!). On each iteration of our loop we “move” the hue towards either the red or black. The variable direction can be either 1 or -1. A value of 1 means we are moving towards 255 and a value of -1 means we are moving towards 0. When we cross a boundary, we flip the direction. Keen observers will ask why we bother with 1 * direction. In our current logic, it’s an unnecessary step and unnecessary steps in game development are generally bad. In this case, I wanted to separate the rate of change from the direction. In order words, you could modify that expression to 2 * direction and the color would change twice as fast. This leads us to another important point. Our rate of change is tied to how quickly our loop iterates; most likely 60fps. However, it’s not guaranteed to be 60fps and that makes this approach a dangerous practice. Once way to detach ourselves from the loop’s speed would be to use the elapsed time that is being passed into our update function. Let’s say that we want to it to take 2 full seconds to go from red to black regardless of how often the update function is called. There’s a span of 256 discrete values between red and black. To make our calculations clear, let’s say there are 256 units and we’ll label these units R. Also, the elapsed time will be in milliseconds (ms). For a given frame, if were are given a slice of elapsed time in ms, we’ll want to calculate how many R units to increase (or decrease) hue by for that slice. Our rate of change can be defined as 256 **R** / 2000 **ms** or 0.128 R/ms. (You can read that as “0.128 units of red per millisecond”.) This rate of change is a constant for our start screen and as such we can define it once (as opposed to calculating it inside the update function). Now that we have the rate of change , we only need to multiply it by the elapsed time received in update to determine how many Rs we want. A revised version of the function would look like this: var rate = 0.128; // R/ms function update(elapsed) { var amount = rate * elapsed; hue += amount * direction; if (hue > 255) direction = -1; if (hue < 0) direction = 1; } One consequence of this change is that hue will no longer be integral values (as much as that can be said in JavaScript.) This means that we’d really want to have two values for the hue: an actual value and a rounded value. This is because the RBG model requires an integral value for each color component. function update(elapsed) { var amount = rate * elapsed; hue += amount * direction; if (hue > 255) direction = -1; if (hue < 0) direction = 1; rounded_hue = Math.round(hue); } Draw Let’s turn our attention to draw for a moment. One of the first things you generally do is to clear the entire screen. This is simple to do with the canvas API’s clearRect method. ctx.clearRect(0, 0, ctx.canvas.width, ctx.canvas.height); Notice that ctx is an instance of CanvasRenderingContext2D and not a HTMLCanvasElement. However, there is a handy back reference to the canvas element that we use to grab the actual width and height. There are other options other than clearing the entire canvas, but I’m not going to address this in this post. Also, there are some performance considerations. See the article listed under references. After clearing the screen, we want to draw something new. In this case, the game title and the instructions. In both cases I want to center the text horizontally. I created a helper function that I can provide with the text to render as well as the vertical position (y). function centerText(ctx, text, y) { var measurement = ctx.measureText(text); var x = (ctx.canvas.width - measurement.width) / 2; ctx.fillText(text, x, y); } measureText returns the width in pixels that the rendered text will take up. We use this in combination with the canvas element’s width to determine the x position for the text. fillText is responsible for actually drawing the text. The rendering context ctx is stateful. Meaning that, what happens when you call methods like measureText or fillText depends on the state of the rendering context. The state can be modified by setting its properties. var y = ctx.canvas.height / 2; ctx.fillStyle = 'white'; ctx.font = '48px monospace'; centerText(ctx, 'My Awesome Game', y); The properties fillStyle and font change the state of the rendering context and hence affect the methods calls inside of centerText. This state applies to all future methods calls. This means that all calls to fillText will use the color white until you can the fillStyle. Notice too that we are calculating the x and y values for the text on every frame. This is potentially wasteful since these values are unlikely to change. However, if we want to respond to changes in canvas size (or even changes to the text itself) then we’d want to continue calculating these on every frame. Otherwise, if we were confident that we didn’t need to do this, we could calculate these values once and cache them. Now let’s use the red component calculated in update to render the instructional text. var color = 'rgb(' + hue + ',0,0)'; ctx.fillStyle = color; ctx.font = '24px monospace'; centerText(ctx, 'click to begin', y + 30); fillStyle can be set in a number of ways. Earlier, we used the simple value white. Here were are using rgb() to set the individual components explicitly. Any CSS color should work with fillStyle. (I won’t be too surprised if some don’t though.) Now you might be wondering why we bothered calculating hue inside update since hue is all about what to draw on the screen. The reason is that draw is concerned with the mechanics of rendering. Anything that is modeling the game state should live in update. The tell in this example is that hue is dependent on elapsed time and the draw doesn’t know anything about that. Update (again) Moving back to update, the next bit deals with input from the player. In the sample code I’ve extracted the input logic away. The key thing here is that we are not relying on events to tell us about input from the player. Instead we have some helper, input in this case, that gives us the current state of the input. If event-driven logic says “tell me when this happens” then our game logic says “tell me if this is happening now”. The primary reason for this is to be deterministic. We can establish at the beginning of our update what the current input state is and that it won’t change before the next invocation of the function. In simple games this might be inconsequential, but in others it can be a subtle source of bugs. var isButtonDown = input.isButtonDown(); var mouseJustClicked = !isButtonDown && wasButtonDown; if (mouseJustClicked && !transitioning) { transitioning = true; // do something here to transition to the actual game } wasButtonDown = isButtonDown; We only want transition when the mouse button has been released. In this case, “released” is defined as “down on the last frame but up on this one”. Hence, we need to track what the mouse button’s state was on the last frame. That’s wasButtonDown and it lives outside of update. Secondly, we don’t want to trigger multiple transitions. That is, if our transition takes some time (perhaps due to animation) then we want to ignore subsequent clicks. We have our transitioning variable outside of update to track that for us. More to come… Update: I just realized that I didn't include a shim for requestAnimationFrame for the demo on jsfiddle. That means the demo will fail on many browsers. (Of course, it will also fail if there's no canvas support either.) If it doesn't work, check your console for errors.

Use jQuery's UI To drag and drop within a single or multiple lists in AngularJS.

Originally posted by Daniel Chirca One the first things I stumbled upon when I started my first Node.js project was how to handle the communication between the browser (the client) and my middleware (the middleware being a Node.js application using the CUBRID Node.js driver (node-cubrid) to exchange information with a CUBRID 8.4.1 database). I am already familiar with AJAX (btw, thanks God for jQuery!! ) but, while studying Node.js, I found out about the Socket.IO module and even found some pretty nice code examples on the internet... Examples which were very-very easy to (re)use... So this quickly become a dilemma: what to choose, AJAX or sockets.io? Obviously, as my experience was quite limited, I needed first more information from out there... In other words, it was time to do some quality Google search :) There’s a lot of information available and, obviously, one would need to filter out all the “noise” and keep what is really useful. Let me share with you some of the goods links I found on the topic: http://stackoverflow.com/questions/7193033/nodejs-ajax-vs-socket-io-pros-and-cons http://podefr.tumblr.com/post/22553968711/an-innovative-way-to-replace-ajax-and-jsonp-using http://stackoverflow.com/questions/4848642/what-is-the-disadvantage-of-using-websocket-socket-io-where-ajax-will-do?rq=1 http://howtonode.org/websockets-socketio To summarize, here’s what I quickly found: Socket.IO (usually) uses persistent connection between the client and the server (the middleware), so you can reach a maximum limit of concurrent connections depending on the resources you have on server side (while more AJAX async requests can be served with the same resources). With AJAX you can do RESTful requests. This means that you can take advantage of existing HTTP-infrastructure like e.g. proxies to cache requests and use conditional get requests. There is more (communication) data overhead in AJAX when compared to Socket.IO (HTTP headers, cookies etc.) AJAX is usually faster than Socket.IO to “code”... When using Socket.IO, it is possible to have a two-way communication where each side – client or server - can initiate a request. In AJAX, it is only the client who can initiate a request! Socket.IO has more transport options, including Adobe Flash. Now, for my own application, what I was most interested in was the speed of making requests and getting data from the (Node.js) server! Regarding the middleware data communication with the CUBRID database, as ~90% of my data access was read-only, a good data caching mechanism is obviously a great way to go! But about this, I’ll talk next time. So I decided to put up their (AJAX and socket.io) speed to test, to see which one is faster (at least on my hardware & software environment)....! My middleware was setup to run on an i5 processor, 8GB of RAM and an Intel X25 SSD drive. But seriously, every speed test and, generally speaking, any performance test depends so much(!) on your hardware and software configuration, that it is always a great idea to try the things on your own environment, rely less on various information you find on internet and more on your own findings! The tests I decided to do have to meet the following requirements: Test: AJAX Socket.IO persistent connection Socket.IO non-persistent connections Test 10, 100, 250 and 500 data exchanges between the client and the server Each data exchange between the middleware SERVER (a Node.js web server) and the client (a browser) is a 4KBytes random data string Run the server in release (not debug) mode Use Firefox as the client Minimize the console messages output, for both server and client Do each test after a client full page reload Repeat each test at least 3 times, to make sure the results are consistent Testing Socket.IO, using a persistent connection I've created a small Node.js server, which was handling the client requests: io.sockets.on('connection', function (client) { client.on('send_me_data', function (idx) { client.emit('you_have_data', idx, random_string(4096)); }); }); And this is the JS client script I used for test: var socket = io.connect(document.location.href); socket.on('you_have_data', function (idx, data) { var end_time = new Date(); total_time += end_time - start_time; logMsg(total_time + '(ms.) [' + idx + '] - Received ' + data.length + ' bytes.'); if (idx++ < countMax) { setTimeout(function () { start_time = new Date(); socket.emit('send_me_data', idx); }, 500); } }); Testing Socket.IO, using NON-persistent connection This time, for each data exchange, I opened a new socket-io connection. The Node.js server code was similar with the previous one, but I decided to send back the client data immediately after connect, as a new connection was initiated every time, for each data exchange: io.sockets.on('connection', function (client) { client.emit('you_have_data', random_string(4096)); }); The client test code was: function exchange(idx) { var start_time = new Date(); var socket = io.connect(document.location.href, {'force new connection' : true}); socket.on('you_have_data', function (data) { var end_time = new Date(); total_time += end_time - start_time; socket.removeAllListeners(); socket.disconnect(); logMsg(total_time + '(ms.) [' + idx + '] - Received ' + data.length + ' bytes.'); if (idx++ < countMax) { setTimeout(function () { exchange(idx); }, 500); } }); } Testing AJAX Finally, I put AJAX to test... The Node.js server code was, again, not that different from the previous ones: res.writeHead(200, {'Content-Type' : 'text/plain'}); res.end('_testcb(\'{"message": "' + random_string(4096) + '"}\')'); As for the client code, this is what I used to test: function exchange(idx) { var start_time = new Date(); $.ajax({ url : 'http://localhost:8080/', dataType : "jsonp", jsonpCallback : "_testcb", timeout : 300, success : function (data) { var end_time = new Date(); total_time += end_time - start_time; logMsg(total_time + '(ms.) [' + idx + '] - Received ' + data.length + ' bytes.'); if (idx++ < countMax) { setTimeout(function () { exchange(idx); }, 500); } }, error : function (jqXHR, textStatus, errorThrown) { alert('Error: ' + textStatus + " " + errorThrown); } }); } Remember, when coding together AJAX and Node.js, you need to take into account the you might be doing cross-domain requests and violating same origin policy, therefore you should use the JSONP based format! Btw, as you can see, I quoted only the most significant parts of the test code, to save space. If anyone needs the full code, server and client, please let me know – I’ll be happy to share them. OK – it’s time now to see what we got after all this work! I have run each test for 10, 100, 250 and 500 data exchanges and this is what I got in the end: Data exchanges Socket.IO NON-persistent (ms.) AJAX (ms.) Socket.IO persistent (ms.) 10 90 40 32 100 900 320 340 250 2,400 800 830 500 4,900 1,500 1,600 Looking into the results, we can notice a few things right away: For each type of test, the results behave quite linear; this is good – it shows that the results are consistent. The results clearly show that when using Socket.IO non-persistent connections, the performance numbers are significantly worse than others. It doesn’t seem to be a big difference between AJAX and the Socket.IO persistent connections – we are talking only about some milliseconds differences. This means that if you can live with less than 10,000 data exchanges per day, for example, there are high chances that the user won’t notice a speed difference... The graph below illustrates the numbers I obtained in test: ...So what’s next...? ...Well, I have to figure out what kind of traffic I need to support and then I will re-run the tests for those numbers, but this time excluding Socket.IO non-persistent connections. That’s because it is obvious that I need to choose between AJAX and persistent Socket.IO connections. And I also learned that, most probably, the difference in speed would not be as much as one would expect... at least not for a “small-traffic” web site, so I need to start looking into other advantages and disadvantages for each approach/technology when choosing my solution! That’s pretty much for this post - see you next time with a post about Node.js and caching! P.S. Here are a few more nice resources to find interesting stuff about Node.js, Socket.IO and AJAX: http://socket.io/#how-to-use http://www.hacksparrow.com/jquery-with-node-js.html http://www.slideshare.net/toddeichel/nodejs-talk-at-jquery-pittsburgh http://tech.burningbird.net/article/node-references-and-resources http://davidwalsh.name/websocket

Today I would like to talk about picture manipulation. Not the Direct2D way I used in my previous article but the pure JavaScript way. The test case The test application is simple. On the left a picture to manipulate and on the right the updated result (a sepia tone effect is applied): The page itself is simple and is described as follow: The overall process to apply a sepia tone effect requires you to compute a new value for every pixel of the picture: finalRed= (red * 0.393) + (green * 0.769) + (blue * 0.189); finalGreen = (red * 0.349) + (green * 0.686) + (blue * 0.168); finalBlue= (red * 0.272) + (green * 0.534) + (blue * 0.131); To make it more realistic I added a bit of random in the formula so the final JavaScript code to apply to every pixel is: function noise() { return Math.random() * 0.5 + 0.5; }; function colorDistance(scale, dest, src) { return (scale * dest + (1 - scale) * src); }; var processSepia = function (pixel) { pixel.r = colorDistance(noise(), (pixel.r * 0.393) + (pixel.g * 0.769) + (pixel.b * 0.189), pixel.r); pixel.g = colorDistance(noise(), (pixel.r * 0.349) + (pixel.g * 0.686) + (pixel.b * 0.168), pixel.g); pixel.b = colorDistance(noise(), (pixel.r * 0.272) + (pixel.g * 0.534) + (pixel.b * 0.131), pixel.b); }; Brutal force Obviously the very first solution can consist to the use of a brutal approach with a function that apply the previous code on every pixel. To get access to the pixels, you can use the canvas context with the following code: var source = document.getElementById("source"); source.onload = function () { var canvas = document.getElementById("target"); canvas.width = source.clientWidth; canvas.height = source.clientHeight; tempContext.drawImage(source, 0, 0, canvas.width, canvas.height); var canvasData = tempContext.getImageData(0, 0, canvas.width, canvas.height); var binaryData = canvasData.data; } The binaryData object contains an array of every pixel and can be used to quickly read or write data directly to the canvas. So with this in mind, we can apply the whole effect with the following code: var source = document.getElementById("source"); source.onload = function () { var start = new Date(); var canvas = document.getElementById("target"); canvas.width = source.clientWidth; canvas.height = source.clientHeight; if (!canvas.getContext) { log.innerText = "Canvas not supported. Please install a HTML5 compatible browser."; return; } var tempContext = canvas.getContext("2d"); var len = canvas.width * canvas.height * 4; tempContext.drawImage(source, 0, 0, canvas.width, canvas.height); var canvasData = tempContext.getImageData(0, 0, canvas.width, canvas.height); var binaryData = canvasData.data; processSepia(binaryData, len); tempContext.putImageData(canvasData, 0, 0); var diff = new Date() - start; log.innerText = "Process done in " + diff + " ms (no web workers)"; } The processSepia function is just an variation of the previous one: var processSepia = function (binaryData, l) { for (var i = 0; i < l; i += 4) { var r = binaryData[i]; var g = binaryData[i + 1]; var b = binaryData[i + 2]; binaryData[i] = colorDistance(noise(), (r * 0.393) + (g * 0.769) + (b * 0.189), r); binaryData[i + 1] = colorDistance(noise(), (r * 0.349) + (g * 0.686) + (b * 0.168), g); binaryData[i + 2] = colorDistance(noise(), (r * 0.272) + (g * 0.534) + (b * 0.131), b); } }; With this solution, on my Intel Extreme processor (12 cores), the main process takes 150ms and obviously only use one processor: Adding web workers The best thing you can do when dealing with SIMD (single instruction multiple data) is to use a parallelization approach. Especially when you want to work with low-end hardware (such as phone devices) with limited resources. With JavaScript, to enjoy the power of parallelization, you have to use the Web Workers (my friend David Rousset wrote an excellent paper on this subject: http://blogs.msdn.com/b/davrous/archive/2011/07/15/introduction-to-the-html5-web-workers-the-javascript-multithreading-approach.aspx). Picture processing is a really good candidate for parallelization because (in the case of sepia tone) every processing is independent and so the following approach is possible: To do so, first of all you have to create a tools.js file to be used as a reference by other scripts: function noise() { return Math.random() * 0.5 + 0.5; }; function colorDistance(scale, dest, src) { return (scale * dest + (1 - scale) * src); }; var processSepia = function (binaryData, l) { for (var i = 0; i < l; i += 4) { var r = binaryData[i]; var g = binaryData[i + 1]; var b = binaryData[i + 2]; binaryData[i] = colorDistance(noise(), (r * 0.393) + (g * 0.769) + (b * 0.189), r); binaryData[i + 1] = colorDistance(noise(), (r * 0.349) + (g * 0.686) + (b * 0.168), g); binaryData[i + 2] = colorDistance(noise(), (r * 0.272) + (g * 0.534) + (b * 0.131), b); } }; The processSepia function will be applied to every bunch of the picture by a dedicated worker. The code of each worker is included in a pictureprocessor.js file: importScripts("tools.js"); self.onmessage = function (e) { var canvasData = e.data.data; var binaryData = canvasData.data; var l = e.data.length; var index = e.data.index; processSepia(binaryData, l); self.postMessage({ result: canvasData, index: index }); }; The main point here is that the canvas data (actually a part of it according to the current block to process) is cloned by JavaScript and passed to the worker. The worker is not working on the initial source but on a copy of it (using a specified algorithm: the structured clone algorithm). The copy itself is really quick and limited to a specific part of the picture. The main client page (default.js) has to create 4 workers and give them the right part of the picture. Then every worker will callback a function in the main thread using the messaging API (postMessage / onmessage) to give back the result: var source = document.getElementById("source"); source.onload = function () { var start = new Date(); var canvas = document.getElementById("target"); canvas.width = source.clientWidth; canvas.height = source.clientHeight; // Testing canvas support if (!canvas.getContext) { log.innerText = "Canvas not supported. Please install a HTML5 compatible browser."; return; } var tempContext = canvas.getContext("2d"); var len = canvas.width * canvas.height * 4; // Drawing the source image into the target canvas tempContext.drawImage(source, 0, 0, canvas.width, canvas.height); // If workers are not supported if (!window.Worker) { // Getting all the canvas data var canvasData = tempContext.getImageData(0, 0, canvas.width, canvas.height); var binaryData = canvasData.data; // Processing all the pixel with the main thread processSepia(binaryData, len); // Copying back canvas data to canvas tempContext.putImageData(canvasData, 0, 0); var diff = new Date() - start; log.innerText = "Process done in " + diff + " ms (no web workers)"; return; } // Let say we want to use 4 workers var workersCount = 4; var finished = 0; var segmentLength = len / workersCount; // This is the length of array sent to the worker var blockSize = canvas.height / workersCount; // Height of the picture chunck for every worker // Function called when a job is finished var onWorkEnded = function (e) { // Data is retrieved using a memory clone operation var canvasData = e.data.result; var index = e.data.index; // Copying back canvas data to canvas tempContext.putImageData(canvasData, 0, blockSize * index); finished++; if (finished == workersCount) { var diff = new Date() - start; log.innerText = "Process done in " + diff + " ms"; } }; // Launching every worker for (var index = 0; index < workersCount; index++) { var worker = new Worker("pictureProcessor.js"); worker.onmessage = onWorkEnded; // Getting the picture var canvasData = tempContext.getImageData(0, blockSize * index, canvas.width, blockSize); // Sending canvas data to the worker using a copy memory operation worker.postMessage({ data: canvasData, index: index, length: segmentLength }); } }; Using this technique, the complete process lasts only 80ms (from 150ms) on my computer and obviously uses 4 processors: On my low-end hardware (based on dual core system), the process falls to 500ms (from 900ms). The final code is available here: http://www.catuhe.com/msdn/pictureworkers.zip And the live version is right there: http://www.catuhe.com/msdn/workers/default.html (For comparison, the no web workers version: http://www.catuhe.com/msdn/workers/defaultnoworker.html) A important point to note is that on recent computers the difference can be thin or even in favor of the code without workers. The overhead of the memory copy must be balanced by a complex code used by the workers. The sepia tone could not be enough in some cases. However, the web workers will really be useful on low-end hardware. Porting to Windows 8 Finally I was not able to resist to the pleasure of porting my JavaScript code to create a Windows 8 application. It took me about 10 minutes to create a blank JavaScript project and copy/paste the JavaScript code inside (feel the power of native JavaScript code for Windows 8!) So feel free to grab the Windows 8 app code here: http://www.catuhe.com/msdn/Win8PictureWorkers.zip

Today we're going to build a simple TCP proxy server. The scenario: we've got one host (the client) that establishes a TCP connection to another one (the remote). client —> remote We want to set up a proxy server in the middle, so the client will establish the connection with the proxy and the proxy will forward it to the remote, keeping in mind the remote response also. With node.js is really simple to perform those kind of network operations. client —> proxy -> remote var net = require('net'); var LOCAL_PORT = 6512; var REMOTE_PORT = 6512; var REMOTE_ADDR = "192.168.1.25"; var server = net.createServer(function (socket) { socket.on('data', function (msg) { console.log(' ** START **'); console.log('<< From client to proxy ', msg.toString()); var serviceSocket = new net.Socket(); serviceSocket.connect(parseInt(REMOTE_PORT), REMOTE_ADDR, function () { console.log('>> From proxy to remote', msg.toString()); serviceSocket.write(msg); }); serviceSocket.on("data", function (data) { console.log('<< From remote to proxy', data.toString()); socket.write(data); console.log('>> From proxy to client', data.toString()); }); }); }); server.listen(LOCAL_PORT); console.log("TCP server accepting connection on port: " + LOCAL_PORT); Simple, isn’t it? Source code in github

At work, recently I did a code cleanup of an existing Java project. After that exercise, I could see a common set of code violations that occur again and again in the code. So, I came up with a list of such common violations and shared it with my peers so that an awareness would help to improve the code quality and maintainability. I’m sharing the list here to a bigger audience. The list is not in any particular order and all derived from the rules enforced by code quality tools such as CheckStyle, FindBugs and PMD. Here we go! Format source code and Organize imports in Eclipse: Eclipse provides the option to auto-format the source code and organize the imports (thereby removing unused ones). You can use the following shortcut keys to invoke these functions. Ctrl + Shift + F – Formats the source code. Ctrl + Shift + O – Organizes the imports and removes the unused ones. Instead of you manually invoking these two functions, you can tell Eclipse to auto-format and auto-organize whenever you save a file. To do this, in Eclipse, go to Window -> Preferences -> Java -> Editor -> Save Actions and then enable Perform the selected actions on save and check Format source code + Organize imports. Avoid multiple returns (exit points) in methods: In your methods, make sure that you have only one exit point. Do not use returns in more than one places in a method body. For example, the below code is NOT RECOMMENDED because it has more then one exit points (return statements). private boolean isEligible(int age){ if(age > 18){ return true; }else{ return false; } } The above code can be rewritten like this (of course, the below code can be still improved, but that’ll be later). private boolean isEligible(int age){ boolean result; if(age > 18){ result = true; }else{ result = false; } return result; } Simplify if-else methods: We write several utility methods that takes a parameter, checks for some conditions and returns a value based on the condition. For example, consider the isEligible method that you just saw in the previous point. private boolean isEligible(int age){ boolean result; if(age > 18){ result = true; }else{ result = false; } return result; } The entire method can be re-written as a single return statement as below. private boolean isEligible(int age){ return age > 18; } Do not create new instances of Boolean, Integer or String: Avoid creating new instances of Boolean, Integer, String etc. For example, instead of using new Boolean(true), use Boolean.valueOf(true). The later statement has the same effect of the former one but it has improved performance. Use curly braces around block statements. Never forget to use curly braces around block level statements such as if, for, while. This reduces the ambiguity of your code and avoids the chances of introducing a new bug when you modify the block level statement. NOT RECOMMENDED if(age > 18) return true; else return false; RECOMMENDED if(age > 18){ return true; }else{ return false; } Mark method parameters as final, wherever applicable: Always mark the method parameters as final wherever applicable. If you do so, when you accidentally modify the value of the parameter, you’ll get a compiler warning. Also, it makes the compiler to optimize the byte code in a better way. RECOMMENDED private boolean isEligible(final int age){ ... } Name public static final fields in UPPERCASE: Always name the public static final fields (also known as Constants) in UPPERCASE. This lets you to easily differentiate constant fields from the local variables. NOT RECOMMENDED public static final String testAccountNo = "12345678"; RECOMMENDED public static final String TEST_ACCOUNT_NO = "12345678";, Combine multiple if statements into one: Wherever possible, try to combine multiple if statements into single one. For example, the below code; if(age > 18){ if( voted == false){ // eligible to vote. } } can be combined into single if statements, as: if(age > 18 && !voted){ // eligible to vote } switch should have default: Always add a default case for the switch statements. Avoid duplicate string literals, instead create a constant: If you have to use a string in several places, avoid using it as a literal. Instead create a String constant and use it. For example, from the below code, private void someMethod(){ logger.log("My Application" + e); .... .... logger.log("My Application" + f); } The string literal “My Application” can be made as an Constant and used in the code. public static final String MY_APP = "My Application"; private void someMethod(){ logger.log(MY_APP + e); .... .... logger.log(MY_APP + f); } Additional Resources: A collection of Java best practices. List of available Checkstyle checks. List of PMD Rule sets

Yes, Perl5 can be embedded in node.js! First of all, do a npm install perl. (P.S. node-perl requires a perl5 binary built with -fPIC and -Duseshrplib.) This is synchronous but useful embedded Perl5 for node.js. If you want to try any version of perl, you must check out perl-node. #>git clone git://github.com/hideo55/node-perl.git #>cd node-perl #>node-waf configure #>node-waf build #>node-waf install And then: var Perl = require('perl').Perl(); var perl = new Perl(); perl.Run({ opts : ["-Mfeature=say","-e","say 'Hello world'"] }, function(out,err){ console.log(out); }); perl.Run({ script : 'example.pl', args : ['foo', 'bar'] }); If you opted for Perl5: var Perl = require('perl-simple').Perl; var perl = new Perl(); var ret = perl.evaluate("reverse 'yoeman'"); console.log(ret); // => nameoy var Perl = require('../index.js').Perl; var perl = new Perl(); perl.use('LWP::UserAgent'); var ua = perl.getClass('LWP::UserAgent').new(); var res = ua.get('http://utf-8.jp/'); console.log(res.as_string()); Happy hacking!

Entity Framework Migrations are handled from the package manager console in Visual Studio. The usage is shown in various tutorials, but I haven’t found a complete list of the commands available and their usage, so I created my own. There are four available commands. Enable-Migrations: Enables Code First Migrations in a project. Add-Migration: Scaffolds a migration script for any pending model changes. Update-Database: Applies any pending migrations to the database. Get-Migrations: Displays the migrations that have been applied to the target database. The information here is the output of running get-help command-name -detailed for each of the commands in the package manager console (running EF 4.3.1). I’ve also added some own comments where I think some information is missing. My own comments are placed under the Additional Information heading. Please note that all commands should be entered on the same line. I’ve added line breaks to avoid vertical scrollbars. Enable-Migrations Enables Code First Migrations in a project. Syntax Enable-Migrations [-EnableAutomaticMigrations] [[-ProjectName] ] [-Force] [] Description Enables Migrations by scaffolding a migrations configuration class in the project. If the target database was created by an initializer, an initial migration will be created (unless automatic migrations are enabled via the EnableAutomaticMigrations parameter). Parameters -EnableAutomaticMigrations Specifies whether automatic migrations will be enabled in the scaffolded migrations configuration. If ommitted, automatic migrations will be disabled. -ProjectName Specifies the project that the scaffolded migrations configuration class will be added to. If omitted, the default project selected in package manager console is used. -Force Specifies that the migrations configuration be overwritten when running more than once for given project. This cmdlet supports the common parameters: Verbose, Debug, ErrorAction, ErrorVariable, WarningAction, WarningVariable, OutBuffer and OutVariable. For more information, type: get-help about_commonparameters. Remarks To see the examples, type: get-help Enable-Migrations -examples. For more information, type: get-help Enable-Migrations -detailed. For technical information, type: get-help Enable-Migrations -full. Additional Information The flag for enabling automatic migrations is saved in the Migrations\Configuration.cs file, in the constructor. To later change the option, just change the assignment in the file. public Configuration() { AutomaticMigrationsEnabled = false; } Add-Migration Scaffolds a migration script for any pending model changes. Syntax Add-Migration [-Name] [-Force] [-ProjectName ] [-StartUpProjectName ] [-ConfigurationTypeName ] [-ConnectionStringName ] [-IgnoreChanges] [] Add-Migration [-Name] [-Force] [-ProjectName ] [-StartUpProjectName ] [-ConfigurationTypeName ] -ConnectionString -ConnectionProviderName [-IgnoreChanges] [] Description Scaffolds a new migration script and adds it to the project. Parameters -Name Specifies the name of the custom script. -Force Specifies that the migration user code be overwritten when re-scaffolding an existing migration. -ProjectName Specifies the project that contains the migration configuration type to be used. If ommitted, the default project selected in package manager console is used. -StartUpProjectName Specifies the configuration file to use for named connection strings. If omitted, the specified project’s configuration file is used. -ConfigurationTypeName Specifies the migrations configuration to use. If omitted, migrations will attempt to locate a single migrations configuration type in the target project. -ConnectionStringName Specifies the name of a connection string to use from the application’s configuration file. -ConnectionString Specifies the the connection string to use. If omitted, the context’s default connection will be used. -ConnectionProviderName Specifies the provider invariant name of the connection string. -IgnoreChanges Scaffolds an empty migration ignoring any pending changes detected in the current model. This can be used to create an initial, empty migration to enable Migrations for an existing database. N.B. Doing this assumes that the target database schema is compatible with the current model. This cmdlet supports the common parameters: Verbose, Debug, ErrorAction, ErrorVariable, WarningAction, WarningVariable, OutBuffer and OutVariable. For more information, type: get-help about_commonparameters. Remarks To see the examples, type: get-help Add-Migration -examples. For more information, type: get-help Add-Migration -detailed. For technical information, type: get-help Add-Migration -full. Update-Database Applies any pending migrations to the database. Syntax Update-Database [-SourceMigration ] [-TargetMigration ] [-Script] [-Force] [-ProjectName ] [-StartUpProjectName ] [-ConfigurationTypeName ] [-ConnectionStringName ] [] Update-Database [-SourceMigration ] [-TargetMigration ] [-Script] [-Force] [-ProjectName ] [-StartUpProjectName ] [-ConfigurationTypeName ] -ConnectionString -ConnectionProviderName [] Description Updates the database to the current model by applying pending migrations. Parameters -SourceMigration Only valid with -Script. Specifies the name of a particular migration to use as the update’s starting point. If ommitted, the last applied migration in the database will be used. -TargetMigration Specifies the name of a particular migration to update the database to. If ommitted, the current model will be used. -Script Generate a SQL script rather than executing the pending changes directly. -Force Specifies that data loss is acceptable during automatic migration of the database. -ProjectName Specifies the project that contains the migration configuration type to be used. If ommitted, the default project selected in package manager console is used. -StartUpProjectName Specifies the configuration file to use for named connection strings. If omitted, the specified project’s configuration file is used. -ConfigurationTypeName Specifies the migrations configuration to use. If omitted, migrations will attempt to locate a single migrations configuration type in the target project. -ConnectionStringName Specifies the name of a connection string to use from the application’s configuration file. -ConnectionString Specifies the the connection string to use. If omitted, the context’s default connection will be used. -ConnectionProviderName Specifies the provider invariant name of the connection string. This cmdlet supports the common parameters: Verbose, Debug, ErrorAction, ErrorVariable, WarningAction, WarningVariable, OutBuffer and OutVariable. For more information, type: get-help about_commonparameters. Remarks To see the examples, type: get-help Update-Database -examples. For more information, type: get-help Update-Database -detailed. For technical information, type: get-help Update-Database -full. Additional Information The command always runs any pending code-based migrations first. If the database is still incompatible with the model the additional changes required are applied as an separate automatic migration step if automatic migrations are enabled. If automatic migrations are disabled an error message is shown. Get-Migrations Displays the migrations that have been applied to the target database. Syntax Get-Migrations [-ProjectName ] [-StartUpProjectName ] [-ConfigurationTypeName ] [-ConnectionStringName ] [] Get-Migrations [-ProjectName ] [-StartUpProjectName ] [-ConfigurationTypeName ] -ConnectionString -ConnectionProviderName [] Description Displays the migrations that have been applied to the target database. Parameters -ProjectName Specifies the project that contains the migration configuration type to be used. If ommitted, the default project selected in package manager console is used. -StartUpProjectName Specifies the configuration file to use for named connection strings. If omitted, the specified project’s configuration file is used. -ConfigurationTypeName Specifies the migrations configuration to use. If omitted, migrations will attempt to locate a single migrations configuration type in the target project. -ConnectionStringName Specifies the name of a connection string to use from the application’s configuration file. -ConnectionString Specifies the the connection string to use. If omitted, the context’s default connection will be used. -ConnectionProviderName Specifies the provider invariant name of the connection string. This cmdlet supports the common parameters: Verbose, Debug, ErrorAction, ErrorVariable, WarningAction, WarningVariable, OutBuffer and OutVariable. For more information, type: get-help about_commonparameters. Remarks To see the examples, type: get-help Get-Migrations -examples. For more information, type: get-help Get-Migrations -detailed. For technical information, type: get-help Get-Migrations -full. Additional Information The powershell commands are complex powershell functions, located in the tools\EntityFramework.psm1 file of the Entity Framework installation. The powershell code is mostly a wrapper around the System.Data.Entity.Migrations.MigrationsCommands found in the tools\EntityFramework\EntityFramework.PowerShell.dll file. First a MigrationsCommands object is instantiated with all configuration parameters. Then there is a public method on the MigrationsCommands object for each of the available commands.

Generate a random alpha numeric string whose length is the number of characters specified. Characters will be chosen from the set of alpha-numeric characters. Count is the length of random string to create. private static final String ALPHA_NUMERIC_STRING = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"; public static String randomAlphaNumeric(int count) { StringBuilder builder = new StringBuilder(); while (count-- != 0) { int character = (int)(Math.random()*ALPHA_NUMERIC_STRING.length()); builder.append(ALPHA_NUMERIC_STRING.charAt(character)); } return builder.toString(); }

This article is based on my answer on StackOverflow. I am trying to explain how String class stores the texts, how interning and constant pool works. The main point to understand here is the distinction between String Java object and its contents - char[] under private value field. String is basically a wrapper around char[] array, encapsulating it and making it impossible to modify so the String can remain immutable. Also the String class remembers which parts of this array is actually used (see below). This all means that you can have two different String objects (quite lightweight) pointing to the same char[]. I will show you few examples, together with hashCode() of each String and hashCode() of internal char[] value field (I will call it text to distinguish it from string). Finally I'll show javap -c -verbose output, together with constant pool for my test class. Please do not confuse class constant pool with string literal pool. They are not quite the same. See also Understanding javap's output for the Constant Pool Prerequisites For the purpose of testing I created such a utility method that breaks String encapsulation: private int showInternalCharArrayHashCode(String s) { final Field value = String.class.getDeclaredField("value"); value.setAccessible(true); return value.get(s).hashCode(); } It will print hashCode() of char[] value, effectively helping us understand whether this particular String points to the same char[] text or not. Two string literals in a class Let's start from the simplest example. Java code String one = "abc"; String two = "abc"; BTW if you simply write "ab" + "c", Java compiler will perform concatenation at compile time and the generated code will be exactly the same. This only works if all strings are known at compile time. Class constant pool Each class has its own constant pool - a list of constant values that can be reused if they occur several times in the source code. It includes common strings, numbers, method names, etc. Here are the contents of the constant pool in our example above: const #2 = String #38; // abc //... const #38 = Asciz abc; The important thing to note is the distinction between String constant object (#2) and Unicode encoded text "abc" (#38) that the string points to. Byte code Here is generated byte code. Note that both one and two references are assigned with the same #2 constant pointing to "abc" string: ldc #2; //String abc astore_1 //one ldc #2; //String abc astore_2 //two Output For each example I am printing the following values: System.out.println("one.value: " + showInternalCharArrayHashCode(one)); System.out.println("two.value: " + showInternalCharArrayHashCode(two)); System.out.println("one" + System.identityHashCode(one)); System.out.println("two" + System.identityHashCode(two)); No surprise that both pairs are equal: one.value: 23583040 two.value: 23583040 one: 8918249 two: 8918249 Which means that not only both objects point to the same char[] (the same text underneath) so equals() test will pass. But even more, one and two are the exact same references! So one == two is true as well. Obviously if one and two point to the same object then one.value and two.value must be equal. Literal and new String() Java code Now the example we all waited for - one string literal and one new String using the same literal. How will this work? String one = "abc"; String two = new String("abc"); The fact that "abc" constant is used two times in the source code should give you some hint... Class constant pool Same as above. Byte code ldc #2; //String abc astore_1 //one new #3; //class java/lang/String dup ldc #2; //String abc invokespecial #4; //Method java/lang/String."":(Ljava/lang/String;)V astore_2 //two Look carefully! The first object is created the same way as above, no surprise. It just takes a constant reference to already created String (#2) from the constant pool. However the second object is created via normal constructor call. But! The first String is passed as an argument. This can be decompiled to: String two = new String(one); Output The output is a bit surprising. The second pair, representing references to String object is understandable - we created two String objects - one was created for us in the constant pool and the second one was created manually for two. But why, on earth the first pair suggests that both String objects point to the same char[] value array?! one.value: 41771 two.value: 41771 one: 8388097 two: 16585653 It becomes clear when you look at how String(String) constructor works (greatly simplified here): public String(String original) { this.offset = original.offset; this.count = original.count; this.value = original.value; } See? When you are creating new String object based on existing one, it reuses char[] value. Strings are immutable, there is no need to copy data structure that is known to be never modified. Moreover, since new String(someString) creates an exact copy of existing string and strings are immutable, there is clearly no reason for the two to exist at the same time. I think this is the clue of some misunderstandings: even if you have two String objects, they might still point to the same contents. And as you can see the String object itself is quite small. Runtime modification and intern() Java code Let's say you initially used two different strings but after some modifications they are all the same: String one = "abc"; String two = "?abc".substring(1); //also two = "abc" The Java compiler (at least mine) is not clever enough to perform such operation at compile time, have a look: Class constant pool Suddenly we ended up with two constant strings pointing to two different constant texts: const #2 = String #44; // abc const #3 = String #45; // ?abc const #44 = Asciz abc; const #45 = Asciz ?abc; Byte Code ldc #2; //String abc astore_1 //one ldc #3; //String ?abc iconst_1 invokevirtual #4; //Method String.substring:(I)Ljava/lang/String; astore_2 //two The fist string is constructed as usual. The second is created by first loading the constant "?abc" string and then calling substring(1) on it. Output No surprise here - we have two different strings, pointing to two different char[] texts in memory: one.value: 27379847 two.value: 7615385 one: 8388097 two: 16585653 Well, the texts aren't really different, equals() method will still yield true. We have two unnecessary copies of the same text. Now we should run two exercises. First, try running: two = two.intern(); before printing hash codes. Not only both one and two point to the same text, but they are the same reference! one.value: 11108810 two.value: 11108810 one: 15184449 two: 15184449 This means both one.equals(two) and one == two tests will pass. Also we saved some memory because "abc" text appears only once in memory (the second copy will be garbage collected). The second exercise is slightly different, check out this: String one = "abc"; String two = "abc".substring(1); Obviously one and two are two different objects, pointing to two different texts. But how come the output suggests that they both point to the same char[] array?!? one.value: 23583040two.value: 23583040one: 11108810two: 8918249 I'll leave the answer to you. It'll teach you how substring() works, what are the advantages of such approach and when it can lead to big troubles. Lessons learnt String object itself is rather cheap. It's the text it points to that consumes most of the memory String is just a thin wrapper around char[] to preserve immutability new String("abc") isn't really that expensive as the internal text representation is reused. But still avoid such construct. When String is concatenated from constant values known at compile time, concatenation is done by the compiler, not by the JVM substring() is tricky, but most importantly, it is very cheap, both in terms of used memory and run time (constant in both cases)

This blog post explores JavaScript’s eval function by implementing the foundation for an interactive command line. As a bonus, you’ll get to work with ECMAScript.next’s generators (which can already be tried out on current Firefox versions). Writing an evaluator Let’s say you want to implement an interactive command line for JavaScript (such as [1]). On one hand, you would need to get the graphical user interface right: The user inputs JavaScript code, the command line evaluates the code and displays the result. On the other hand, you would have to implement the evaluation. That’s what we will take on here. It is more complex that it initially seems and teaches us a lot about eval. For starters, let’s write a constructor Evaluator: function Evaluator() { } Evaluator.prototype.evaluate = function (str) { return JSON.stringify(eval(str)); }; To use the evaluator, we create an instance and send JavaScript code to it: > var e = new Evaluator(); > e.evaluate("Math.pow(2, 53)") '9007199254740992' > e.evaluate("3 * 7") '21' > e.evaluate("'foo'+'bar'") '"foobar"' JSON.stringify is used so that the evaluation results can be shown to the user and look like the input. Without stringify, things look as follows: > console.log(123) // OK 123 > console.log("abc") // not OK abc With stringify, everything looks OK: > console.log(JSON.stringify(123)) 123 > console.log(JSON.stringify("abc")) "abc" Note that undefined is not valid JSON, but stringify converts it to undefined (the value, not the string), which is fine for our purposes. What we have implemented so far works for basic things, but still has several problems. Let’s tackle them one at a time. Problem: declarations You can evaluate variable and function declarations, but they are forgotten immediately afterwards: > e.evaluate("var x = 12;") undefined > e.evaluate("x") ReferenceError: x is not defined How do we fix this? The following code is a solution: function Evaluator() { this.env = {}; } Evaluator.prototype.evaluate = function (str) { str = rewriteDeclarations(str); var __environment__ = this.env; // (1) with (__environment__) { // (2) return JSON.stringify(eval(str)); } }; function rewriteDeclarations(str) { // Prefix a newline so that search and replace is simpler str = "\n" + str; str = str.replace(/\nvar\s+(\w+)\s*=/g, "\n__environment__.$1 ="); // (3) str = str.replace(/\nfunction\s+(\w+)/g, "\n__environment__.$1 = function"); return str.slice(1); // remove prefixed newline } this.env holds all variable declarations and function declarations in its properties. We make it accessible to the input in two steps. Step 1 – declare: We assign this.env to __environment__ (1) and rewrite the input so that, among other things, each var declaration assigns to __environment__ (3). That demonstrates one important aspect of eval: it sees all variables in surrounding scopes. That is, if you invoke eval inside your function, you expose all of its internals. The only way to keep those internals secret is to put the eval call in a separate function and call that function. Step 2 – access: Use a with statement so that the properties of __environment__ appear as variables to the eval-ed code. This is not an ideal solution, more of a compromise: with should be avoided [2] and can’t be used in the advantageous strict mode [3]. But it is a quick solution for us now. A work-around is quite complex [4]. > var e = new Evaluator(); > e.evaluate("var x = 123;") '123' > e.evaluate("x") '123' Minor drawback: Normal var declarations have the result undefined; due to our rewriting we now get the value that is assigned to the variable. Problem: exceptions Right now, throwing an exception in evaluate’s input means that the method will throw: > e.evaluate("* 3") SyntaxError: Unexpected token * That is obviously unacceptable: In a graphical user interface, we want to report errors back to the user, not (invisibly) throw an exception. Here is one simple way of doing so: Evaluator.prototype.evaluate = function (str) { try { str = rewriteDeclarations(str); var __environment__ = this.env; with (__environment__) { return JSON.stringify(eval(str)); } } catch (e) { return e.toString(); } }; There is nothing surprising in this code, we simply use try-catch and report back what happened. More sophisticated solutions will want to do more, e.g. display the exception’s stack trace. The new evaluator in action: > var e = new Evaluator(); > e.evaluate("* 3") 'SyntaxError: Unexpected token *' Problem: console.log How do we handle calls to console.log in the input? Logged messages should be shown to the user, not be sent to the browser’s console. The solution is surprisingly easy: function Evaluator(cons) { this.env = {}; this.cons = cons; } Evaluator.prototype.evaluate = function (str) { try { str = rewriteDeclarations(str); var __environment__ = this.env; var console = this.cons; with (__environment__) { return JSON.stringify(eval(str)); } } catch (e) { return e.toString(); } }; The constructor now receives a custom implementation of console and assigns it to this.cons. By assigning that object to a local variable named console (1), we temporarily shadow the global console for eval, there is no need to replace it. Beware that that shadowing affects all of the function, you won’t be able to use the browser’s console anywhere in evaluate. The new evaluator in action: > var cons = { log: function (m) { console.log("### "+m) } }; > var e = new Evaluator(cons); > e.evaluate("console.log('hello')") ### hello undefined Problem: eval creates bindings inside the function One scary feature of eval is that it creates variable bindings inside the function that invokes it: > (function () { eval("var x=3"); return x }()) 3 Fortunately, the fix is easy: use strict mode. > (function () { "use strict"; eval("var x=3"); return x }()) ReferenceError: x is not defined You can’t use with in strict mode, so you’ll have to replace it with a work-around [4]. Keeping declarations in an environment An environment is where JavaScript keeps the parameters and variables of a function. It maps variable names to values and is thus similar to an object. We might be able to avoid rewriting the input and manage declarations via environments. The idea is as follows. eval puts declarations in some environment: Non-strict mode: the environment of the surrounding function. Strict mode: a newly created environment. What if we could reuse that environment for the next invocation of eval, instead of throwing it away? Then eval would properly remember prior declarations. Strict mode gives us no way to access the temporary environment it creates for each invocation. However, in non-strict mode, we might be able to keep the environment of the surrounding function around. The following subsections explore two ways of doing so. Declarations via nested scopes If you create a function g inside another function f, then g permanently retains a reference to f’s current environment envf. Whenever g is called, a new g-specific environment envg is created. But envg points to its parent environment envf. Variables that can’t be found in g’s scope (as managed via envg), are looked up in f’s scope (via envf). Thus, envf is not lost, as long as g exists. That gives us a strategy for keeping the environment of the function that calls eval around. In the following code that function is called evalHelper and creates a new function that has to be used for the next call of eval. Hence, declarations made in the former function are accessible in the later function. function Evaluator() { var that = this; that.evalHelper = function (str) { that.evalHelper = function (str) { return eval(str); }; return eval(str); }; } Evaluator.prototype.evaluate = function (str) { return this.evalHelper(str); }; The fatal problem of this implementation is that you cannot nest to arbitrary depth. But, for the above depth of 2, it works perfectly: > var e = new Evaluator(); > e.evaluate("var x = 7;"); undefined > e.evaluate("x * 3") 21 Declarations via a generator It would be great if we could “restart” the function that calls eval, re-enter it with its previous environment still in place. ECMAScript.next’s generators [5] let you do that. Current versions of Firefox already support generators. Here is a demonstration of how they work in these versions (in ECMAScript.next, you will have to write function*, but apart from that, the code is the same): function mygen() { console.log((yield 0) + " @ 0"); console.log((yield 1) + " @ 1"); console.log((yield 2) + " @ 2"); } The above is a generator function. Invoke it and it will create a generator object. On that object, you first need to invoke the next() method to start execution. A yield x inside the code pauses execution and returns x to the the previously called generator object method. After the first next(), you can either call next() or send(y). The latter means that the currently paused yield will continue and produce the value y. The former is equivalent to send(undefined). The following interaction shows mygen in use: > var g = mygen(); > g.next() // can’t use send() the first time 0 > g.send("a") // continue after yield 0, pause again a @ 0 1 > g.send("b") b @ 1 2 The following is an implementation of Evaluator that calls eval via the generator evalGenerator. Because of that, eval always sees the same environment and remembers declarations. function evalGenerator(console) { var str = yield; while(true) { try { var result = JSON.stringify(eval(str)); str = yield result; } catch (e) { str = yield e.toString(); } } } function Evaluator(cons) { this.evalGen = evalGenerator(cons); this.evalGen.next(); // start } Evaluator.prototype.evaluate = function (str) { return this.evalGen.send(str); }; The new evaluator works as expected. > var e = new Evaluator(); > e.evaluate("var x = 7;") undefined > e.evaluate("x * 2") "14" > e.evaluate("* syntax_error") "SyntaxError: missing ; before statement" The biggest problem with this solution is that it uses the deprecated features non-strict eval together with the new feature generators. There will probably be a way in ECMAScript.next to make this combination work, but it will be a hack and should thus be avoided. Conclusion We have used eval to implement a helper type for a command line. While doing so, we learned a few interesting things about eval: Letting it remember declarations between invocations is complicated; it can access all variables in the scopes surrounding its invocation; and in non-strict mode, it can even create new variables inside the invoking function. The best solution for remembering declarations would be for eval to have an optional parameter for an environment (to be reused), but that is not in the cards. Therefore, the only truly safe solution in pure JavaScript is to use a full-featured JavaScript parser such as esprima to rewrite critical parts of the input code. That is left as an exercise to the reader. References Combining code editing with a command line JavaScript’s with statement and why it’s deprecated JavaScript’s strict mode: a summary Handing variables to eval Asynchronous programming and continuation-passing style in JavaScript