Or “How to never re-use cached filter query results even though you meant to”: Filter queries (“fq” clauses) are a means to restrict the number of documents that are considered for scoring. A common use of “fq” clauses is to restrict the dates of documents returned, things like “in the last day”, “in the last week” etc. You find this pattern often used in conjunction with faceting. Filter queries make use of a filterCache (see solrconfig.xml) to calculate the set of documents satisfying the query once and then re-use that result set. Often, using NOW in filter queries causes this caching to be useless. Here’s why. Solr maintains a filterCache, where it stores the results of “fq” clauses. You can think of it as a map, where the key is the “fq” clause and the value is the set of documents satisfying that clause. I’m going to skip the details of how the document set (the “value” in this map) is stored, since this post is really concentrating on the key. So, let’s say you have two filter queries (whether they’re in the same query or not is irrelevant), something like: “fq=category:books&fq=source:library”. There will be two entries in the filterCache, something like: category:books => 1, 2, 5, 89… source:library => 7, 45, 101… All well and good so far. I’ll add one short diversion here. This bears on why it is often better to have several “fq” clauses than a single one. The same results could be obtained by “fq=category:books AND source:library”, but then the filter cache would look like: category:books AND source:library => 1, 2, 5, 7, 45, 89, 101….. and an fq like “fq=category:books” would NOT re-use the cache since the key is much different. But enough of a diversion… OK, you mentioned dates. Get to the point. It’s common to have date ranges as filter queries, things like “in the last day”, “in the last week”, etc. And there’s the convenient date math to make this easy. So it’s tempting, very tempting to have filter clauses on date ranges like “fq=date:[NOW-1DAY TO NOW]“. Be careful when using NOW! Here’s the problem. In the above example, date:[NOW-1DAY TO NOW] is not what’s used as the key for the fq in the filterCache, the expansion is used as the key. This translates into a form like: “date:[2012-01-20T08:56:23Z TO 2012-01-27T8:56:23Z]” for the key into the filter cache. Now the user adds a term to the “q” and re-submits the query 30 seconds after the first one. The fq clause now looks something like: “fq=date:[2012-01-20T08:56:53Z TO 2012-01-27T8:56:53Z]” note that the seconds went from 23 to 53! The key for this fq does not match the key for the first, even though it’s often the case that the intent is that submitting this kind of fq 30 seconds later would result in the same set of documents matching the filter. Bare NOW entries in filter clauses will pretty much guarantee that the cached result sets will never be reused. Fine. What do you do to make it better? Here’s where rounding makes sense. Using midnight can make sense from two perspectives. The sense you often want is “anything with a timestamp in a particular day” (or month or year or hour or….). So just using NOW for the lower bound would miss anything published between midnight and whenever the user happens to submit the query on the day (in this example) of the lower bound. Re-using the filter cache can substantially speed up your queries, especially if you’re providing links like “in the last day”, “1-7 days ago” etc. So your fq clauses start to look like “fq=date:[NOW/DAY-7DAYS TO NOW/DAY+1DAY]“. The thing to note about the date math “/” operator is that is is a “round down” operator. So let’s break this up a bit: NOW/DAY rounds down to midnight last night. -7DAYS subtracts 7 whole days. So the lower limit is really “midnight 7 days ago”. Similarly, NOW/DAY rounds to midnight last night and +1DAY moves that to midnight tonight for the upper limit. These clauses are invariant until after midnight tonight so these clauses will return the same results all day today, and only the first submission of this fq will incur the cost of figuring out which documents satisfy it, all the queries after the first will just read the cached result set from the filterCache. Of course the caches are invalidated if you update your index and/or a replication happens, but that’s always the case. You will note that there is a bit of “slop” here. If your index has dates in the future, you may get them too. Suppose you have a situation where your index contains documents you don’t want the users to see until it’s later than their timestamp. I actually have a hard time contriving an example here, but let’s just assume it’s the case. Also say it’s noon and your index contains timestamps on documents through midnight tonight. The above technique will show documents that will be officially published at, say, 15:00 even though it’s only 12:00 and you may not want that. In that case, you’ll have to use a bare NOW clause and live with the fact that your cache isn’t being used for these clauses. Like I said, this is contrived, but I mention it for completeness’ sake. A couple of notes about dates: Before I finish, a couple of notes about dates. Use the coarsest dates you can and still satisfy your use case. This is especially true if you’re sorting by dates. The sorting resource requirements go up by the number of unique terms. So storing millisecond resolution when all you care about is day can be wasteful. This is also true when faceting. It’s often useful to index multiple fields with some date data, especially if you intend to facet. The above examples in the 3.x code line have a slight problem when more than one adjoining range is required. The range operator “[]” is inclusive, so if you have a document indexed at exactly midnight in these examples, it might be included in two ranges. Trunk Solr (4.0) allows mixing inclusive “[]” and exclusive “{}” endpoints, so expressions like “date:[NOW/DAY-1DAY TO NOW/DAY+1DAY}” are possible. An exercise for the reader: What are the consequences of using different kinds of rounding? E.g. NOW/5MIN, NOW/72MIN (does this even work?).

Originally Authored By Shrikant Sharat If you haven't checked out @kennethreitz's excellent python-requests library yet, I suggest you go do that immediately. Go on, I'll wait for you. Had your candy? That is one of the most beatiful pieces of python code I've read. And its an excellent library with a very humane API. Recently, I have been using this library for a few of my company's internal projects and at a point I needed to serialize and save Session objects for later. That wasn't as straightforward as I first thought it'd be, so I am sharing my experience here. First off, let's make a simple http server which we are going to contact with python-requests. The server should be able to handle cookie based sessions and also have basic auth, as these things are handled by python-requests' Session objects on the client side. I won't discuss the code for the server here, you can get it from bitbucket. Once you have the server running, now for the client, lets do requests! import requests as req URL_ROOT = 'http://localhost:5050' def get_logged_in_session(name): session = req.session(auth=('user', 'pass')) login_response = session.post(URL_ROOT + '/login', data={'name': name}) login_response.raise_for_status() return session def get_whoami(session): response = session.get(URL_ROOT + '/whoami') response.raise_for_status() return response.text I defined two functions here. The get_logged_in_session will create a new session and login to the http server and return that session. Any subsequent requests using this sesssion will be made as if you have logged in. That's what will be tested with the get_whoami function, which will just return the response from /whoami. Lets test this out. Make sure the server.py is running and in another terminal, $ python -i client.py >>> s = get_logged_in_session('sharat') >>> get_whoami(s) u'You are sharat' >>> get_whoami(req.session(auth=('user', 'pass'))) u'You are a guest' Works perfectly. If we pass it the logged in session, it gives us the username and if we pass it a new session, it gives us a guest. Now, lets assume we have two functions, serialize_session and deserialize_session which do exactly what their names say. We can test them out by running a small test.py, as from client import get_logged_in_session, get_whoami from serializer import deserialize_session, serialize_session session = get_logged_in_session('sharat') dsession = deserialize_session(serialize_session(session)) assert get_whoami(session) == get_whoami(dsession) print 'Success' and a dummy serializer.py def serialize_session(session): return session def deserialize_session(session): return session And with that, of course, the test will not fail $ python test.py Success Serializing Now, to implement the functions in serializer.py. A simple one, would be to use pickle. Lets try import pickle as pk def serialize_session(session): return pk.dumps(session) def deserialize_session(data): return pk.loads(data) If you run test.py now, python is going to yell at you. $ python test.py Traceback (most recent call last): File "test.py", line 10, in dsession = deserialize_session(serialize_session(session)) [ ... ] raise TypeError, "can't pickle %s objects" % base.__name__ TypeError: can't pickle lock objects Oh well, it was worth a try I suppose. Update: The Session class can be made to implement the pickle protocol if you want to use pickle. Next plan I had was to pick up attributes and data from a Session object, just enough to recreate this object using the Session constructor, and serialize those attributes as a json. After all, the Session's API is very easy to use, how hard can picking attributes from it be? :) So, I dug in the sessions.py module of python-requests library. And here's what the signature of the constructor for Session objects looks like def __init__(self, headers=None, cookies=None, auth=None, timeout=None, proxies=None, hooks=None, params=None, config=None, verify=True): # ... So, if I pick up just these values, I should be able to recreate the session object. Sweet. import json import requests as req def serialize_session(session): attrs = ['headers', 'cookies', 'auth', 'timeout', 'proxies', 'hooks', 'params', 'config', 'verify'] session_data = {} for attr in attrs: session_data[attr] = getattr(session, attr) return json.dumps(session_data) def deserialize_session(data): return req.session(**json.loads(data)) And let's try this out $ python test.py Traceback (most recent call last): File "test.py", line 12, in assert get_whoami(session) == get_whoami(dsession) [ ... ] [...]requests/models.py", line 447, in send r = self.auth(self) TypeError: 'list' object is not callable Okay, that error message is very wierd. Why would anyone call a list object? Go dig in the models.py module. See this [ ... ] if isinstance(self.auth, tuple) and len(self.auth) == 2: # special-case basic HTTP auth self.auth = HTTPBasicAuth(*self.auth) # Allow auth to make its changes. r = self.auth(self) [ ... ] There. Its not a list that's being called. Not directly at least. The problem here is that the auth we are passing to session() is not a tuple. Duh! While I like it that auth is restricted to be a tuple, I wish there was a better error message for when auth is a list instead of a tuple. I personally wouldn't want it to accept a list for auth though. So, what went wrong? json does not differentiate between a tuple and a list. It only does lists. So, when serializing and deserializing, the auth tuple is turned to a list. Lets turn it back def deserialize_session(data): session_data = json.loads(data) if 'auth' in session_data: session_data['auth'] = tuple(session_data['auth']) return req.session(**session_data) And $ python test.py Traceback (most recent call last): File "test.py", line 12, in assert get_whoami(session) == get_whoami(dsession) [ ... ] File "/usr/lib/python2.7/string.py", line 493, in translate return s.translate(table, deletions) TypeError: translate() takes exactly one argument (2 given) Wait. What? Now we have an error from stdlib? This just keeps getting better and better. If this looks like something that can frustrate you, go get some coffee :) If you look at the complete stack trace, the second file from bottom, File "[...]site-packages/requests/packages/oreos/monkeys.py", line 470, in set if "" != translate(key, idmap, LegalChars): This thing seems to be calling the translate method incorrectly. With a bit of debugging and yelling at my monitor, I found out the problem and for a moment, lost my grip on reality. str.translate takes 2 arguments, but unicode.translate takes only 1. I have no idea why this is done this way but I sure as hell didn't enjoy it. The code in oreos/monkeys.py assumes that the key is a str. However, what json.loads gives you, is unicode stuff. So, we need to convert just the parts in the deserialized dict we get from json.loads which are being used by the oreos/monkeys.py, from unicode to str. Reading a bit more code around the oreos library, it didn't take long to figure out that those were the keys in the cookies dict. Lo def deserialize_session(data): session_data = json.loads(data) if 'auth' in session_data: session_data['auth'] = tuple(session_data['auth']) if 'cookies' in session_data: session_data['cookies'] = dict((key.encode(), val) for key, val in session_data['cookies'].items()) return req.session(**session_data) And so $ python test.py Success ! All the code is on a bitbucket repository. Update: Pickling can also work As Daslch pointed out in his comment on reddit, by implementing the pickle protocol on the Session class, we can get pickling to work. From the documentation, we need two methods, __getstate__ and __setstate__. Adding those methods as follows to sessions.Session class def __getstate__(self): attrs = ['headers', 'cookies', 'auth', 'timeout', 'proxies', 'hooks', 'params', 'config', 'verify'] return dict((attr, getattr(self, attr)) for attr in attrs) def __setstate__(self, state): for name, value in state.items(): setattr(self, name, value) self.poolmanager = PoolManager( num_pools=self.config.get('pool_connections'), maxsize=self.config.get('pool_maxsize') ) with this as the version of serializer.py that uses pickle, we do get a Success. The creation of new poolmanager in __setstate__ is a piece of code copied from __init__ of the same class. This should probably be turned to a method to avoid code repetition. Update 2: Created an issue about this. Update 3: This has been merged and Session objects are pickleable as of version 0.10.3. See requests history. Source: http://sharats.me/serializing-python-requests-session-objects-for-fun-and-profit.html

In this post, I will try to give an overview of the free tools available for developers to move databases from SQL Server to SQL Server Compact and vice versa. I will also show how you can do this with the SQL Server Compact Toolbox (add-in and standalone editions). Moving databases from SQL Server Compact to SQL Server This can be useful for situations where you already have developed an application that depends on SQL Server Compact, and would like the increased power of SQL Server or would like to use some feature, that is not available on SQL Server Compact. I have an informal comparison of the two products here. Microsoft offers a GUI based tool and a command line tool to do this: WebMatrix and MsDeploy. You can also use the ExportSqlCe command line tool or the SQL Server Compact Toolbox to do this. To use the ExportSqlCE (or ExportSqlCE40) command line, use a command similar to: ExportSQLCE.exe "Data Source=D:\Northwind.sdf;" Northwind.sql The resulting script file (Northwind.sql) can the be run against a SQL Server database, using for example the SQL Server sqlcmd command line utility: sqlcmd -S mySQLServer –d NorthWindSQL -i C:\Northwind.sql To use the SQL Server Compact Toolbox: Connect the Toolbox to the database file that you want to move to SQL Server: Right click the database connection, and select to script Schema and Data: Optionally, select which tables to script and click OK: Enter the filename for the script, default extension is .sqlce: Click OK to the confirmation message: You can now open the generated script in Management Studio and execute it against a SQL Server database, or run it with sqlcmd as described above. Moving databases from SQL Server to SQL Server Compact Microsoft offers no tools for doing this “downsizing” of a SQL Server database to SQL Server Compact, and of course not all objects in a SQL Server database CAN be downsized, as only tables exists in a SQL Server Compact database, so no stored procedures, views, triggers, users, schema and so on. I have blogged about how this can be done from the command line, and you can also do this with the SQL Server Compact Toolbox (of course): From the root node, select Script Server Data and Schema: Follow a procedure like the one above, but connecting to a SQL Server database instead. The export process will convert the SQL Server data types to a matching SQL Server Compact data type, for example varchar(50) becomes nvarchar(50) and so on. Any unsupported data types will be ignored, this includes for example computed columns and sql_variant. The new date types in SQL Server 2008+, like date, time, datetime2 will be converted to nvarchar based data types, as only datetime is supported in SQL Server Compact. A full list of the SQL Server Compact data types is available here.

One day at work, I was told that we had a feature request for one of my programs. They wanted a “dark mode” for when they used my application at night as the normal colors were kind of glaring. My program is used in laptops in police cars, so I could understand their frustration. I spent some time looking into the matter and got a mostly working script put together which I’m going to share with my readers. Of course, if you’re a long time reader, you probably know I’m talking about a wxPython program. I write almost all my GUIs using wxPython. Anyway, let’s get on with the story! Into the Darkness Getting the widgets to change color in wxPython is quite easy. The only two methods you need are SetBackgroundColour and SetForegroundColour. The only major problem I ran into when I was doing this was getting my ListCtrl / ObjectListView widget to change colors appropriately. You need to loop over each ListItem and change their colors individually. I alternate row colors, so that made things more interesting. The other problem I had was restoring the ListCtrl’s background color. Normally you can set a widget’s background color to wx.NullColour (or wx.NullColor) and it will go back to its default color. However, some widgets don’t work that way and you have to actually specify a color. It should also be noted that some widgets don’t seem to pay any attention to SetBackgroundColour at all. One such widget that I’ve found is the wx.ToggleButton. Now you know what I know, so let’s look at the code I came up with to solve my issue: import wx try: from ObjectListView import ObjectListView except: ObjectListView = False #---------------------------------------------------------------------- def getWidgets(parent): """ Return a list of all the child widgets """ items = [parent] for item in parent.GetChildren(): items.append(item) if hasattr(item, "GetChildren"): for child in item.GetChildren(): items.append(child) return items #---------------------------------------------------------------------- def darkRowFormatter(listctrl, dark=False): """ Toggles the rows in a ListCtrl or ObjectListView widget. Based loosely on the following documentation: http://objectlistview.sourceforge.net/python/recipes.html#recipe-formatter and http://objectlistview.sourceforge.net/python/cellEditing.html """ listItems = [listctrl.GetItem(i) for i in range(listctrl.GetItemCount())] for index, item in enumerate(listItems): if dark: if index % 2: item.SetBackgroundColour("Dark Grey") else: item.SetBackgroundColour("Light Grey") else: if index % 2: item.SetBackgroundColour("Light Blue") else: item.SetBackgroundColour("Yellow") listctrl.SetItem(item) #---------------------------------------------------------------------- def darkMode(self, normalPanelColor): """ Toggles dark mode """ widgets = getWidgets(self) panel = widgets[0] if normalPanelColor == panel.GetBackgroundColour(): dark_mode = True else: dark_mode = False for widget in widgets: if dark_mode: if isinstance(widget, ObjectListView) or isinstance(widget, wx.ListCtrl): darkRowFormatter(widget, dark=True) widget.SetBackgroundColour("Dark Grey") widget.SetForegroundColour("White") else: if isinstance(widget, ObjectListView) or isinstance(widget, wx.ListCtrl): darkRowFormatter(widget) widget.SetBackgroundColour("White") widget.SetForegroundColour("Black") continue widget.SetBackgroundColour(wx.NullColor) widget.SetForegroundColour("Black") self.Refresh() return dark_mode This code is a little convoluted, but it gets the job done. Let’s break it down a bit and see how it works. First off, we try to import ObjectListView, a cool 3rd party widget that wraps wx.ListCtrl and makes it a LOT easier to use. However, it’s not part of wxPython right now, so you need to test for it’s existence. I just set it to False if it doesn’t exist. The GetWidgets function takes a parent parameter, which would usually be a wx.Frame or wx.Panel and goes through all of its children to create a list of widgets, which it then returns to the calling function. The main function is darkMode. It takes two parameters too, the poorly named “self”, which refers to a parent widget, and a default panel color. It calls GetWidgets and then uses a conditional statement to decide if dark mode should be enabled or not. Next it loops over the widgets and changes the colors accordingly. When it’s done, it will refresh the passed in parent and return a bool to let you know if dark mode is on or off. There is one more function called darkRowFormatter that is only for setting the colors of the ListItems in a wx.ListCtrl or an ObjectListView widget. Here we use a list comprehension to create a list of wx.ListItems that we then iterate over, changing their colors. To actually apply the color change, we need to call SetItem and pass it a wx.ListItem object instance. Trying Out Dark Mode So now you’re probably wondering how to actually use the script above. Well, this section will show you how it’s done. Here’s a simple program with a list control in it and a toggle button too! import wx import darkMode ######################################################################## class MyPanel(wx.Panel): """""" #---------------------------------------------------------------------- def __init__(self, parent): """Constructor""" wx.Panel.__init__(self, parent) self.defaultColor = self.GetBackgroundColour() rows = [("Ford", "Taurus", "1996", "Blue"), ("Nissan", "370Z", "2010", "Green"), ("Porche", "911", "2009", "Red") ] self.list_ctrl = wx.ListCtrl(self, style=wx.LC_REPORT) self.list_ctrl.InsertColumn(0, "Make") self.list_ctrl.InsertColumn(1, "Model") self.list_ctrl.InsertColumn(2, "Year") self.list_ctrl.InsertColumn(3, "Color") index = 0 for row in rows: self.list_ctrl.InsertStringItem(index, row[0]) self.list_ctrl.SetStringItem(index, 1, row[1]) self.list_ctrl.SetStringItem(index, 2, row[2]) self.list_ctrl.SetStringItem(index, 3, row[3]) if index % 2: self.list_ctrl.SetItemBackgroundColour(index, "white") else: self.list_ctrl.SetItemBackgroundColour(index, "yellow") index += 1 btn = wx.ToggleButton(self, label="Toggle Dark") btn.Bind(wx.EVT_TOGGLEBUTTON, self.onToggleDark) normalBtn = wx.Button(self, label="Test") sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.list_ctrl, 0, wx.ALL|wx.EXPAND, 5) sizer.Add(btn, 0, wx.ALL, 5) sizer.Add(normalBtn, 0, wx.ALL, 5) self.SetSizer(sizer) #---------------------------------------------------------------------- def onToggleDark(self, event): """""" darkMode.darkMode(self, self.defaultColor) ######################################################################## class MyFrame(wx.Frame): """""" #---------------------------------------------------------------------- def __init__(self): """Constructor""" wx.Frame.__init__(self, None, wx.ID_ANY, "MvP ListCtrl Dark Mode Demo") panel = MyPanel(self) self.Show() #---------------------------------------------------------------------- if __name__ == "__main__": app = wx.App(False) frame = MyFrame() app.MainLoop() If you run the program above, you should see something like this: If you click the ToggleButton, you should see something like this: Notice how the toggle button was unaffected by the SetBackgroundColour method. Also notice that the list control’s column headers don’t change colors either. Unfortunately, wxPython doesn’t expose access to the column headers, so there’s no way to manipulate their color. Anyway, let’s take a moment to see how the dark mode code is used. First we need to import it. In this case, the module is called darkMode. To actually call it, we need to look at the ToggleButton’s event handler: darkMode.darkMode(self, self.defaultColor) As you can see, all we did was call darkMode.darkMode with the panel object (the “self) and a defaultColor that we set at the beginning of the wx.Panel’s init method. That’s all we had to do too. We should probably set it up with a variable to catch the returned value, but for this example we don’t really care. Wrapping Up Now we’re done and you too can create a “dark mode” for your applications. At some point, I’d like to generalize this some more to make into a color changer script where I can pass whatever colors I want to it. What would be really cool is to make it into a mixin. But that’s something for the future. For now, enjoy! Further Reading ObjectListView documentation An ObjectListView tutorial wx.ListCtrl documentation Source Code 2011-11-5-wxPython-dark-mode You can also pull the source from Bitbucket Source: http://www.blog.pythonlibrary.org/2011/11/05/wxpython-creating-a-dark-mode/

(get wcf.js on github!) Take a look at the following code: var binding = new WSHttpBinding( { MessageEncoding: "Mtom" , SecurityMode:"TransportWithMessageCredential" }) , proxy = new Proxy(binding) proxy.ClientCredentials.Username.Username = "yaron"; proxy.ClientCredentials.Username.Password = "1234"; proxy.send(message, function(response) { console.log(response) }); Do you see anything...um, special? Well c# already has the "var" keyword since version 3.0 so maybe it is some kind of a c#-ish dialect? Or maybe it is a CTP for javascript as a CLR language? Or something related to the azure sdk for node.js? Not at all. This is a snippet from wcf.js - a pure javascript node.js module that makes wcf and node.js work together! As node assumes its central place in modern web development, many developers build node apps that must consume downstream wcf services. Now if these services use WCF Web API ASP.NET Web API it is very easy. It is also a breeze if you are in a position to add a basic http binding to the Wcf service, and just a little bit of more work if you plan to employ a wcf router to do the protocol bridging. Wcf.js is a library that aims to provide a pure-javascript development experiece for such scenarios. Note that building new node.js ws-* based services is a non-goal for this project. Putting aside all the religious wars, Soap is not the "node way", so you should stick to Rest where you'll get good language support (json) and built-in libraries. "Hello, Wcf... from node" You are closer than you think to consume your first Wcf service node.js: 1. Create a new wcf web site in VS and call it "Wcf2Node". If you use .Net 4 than BasicHttpBinding is the default, otherwise in web.config replace WsHttp with BasicHttp. No need to deploy, just run the service in VS using F5. 2. Create anywhere a folder for the node side and from the command line enter its root and execute: $> npm install wcf.js 3. In the same folder create test.js: var BasicHttpBinding = require('wcf.js').BasicHttpBinding , Proxy = require('wcf.js').Proxy , binding = new BasicHttpBinding() , proxy = new Proxy(binding, " http://localhost:12/Wcf2Node/Service.svc") , message = '' + '' + '' + '' + '123' + '' + '' + '' proxy.send(message, "http://tempuri.org/IService/GetData", function(response, ctx) { console.log(response) }); 4. In test.js, change the port 12 (don't ask...) to the port your service runs on. 5. Now we can execute node: $> node test.js 6. You should now see the output soap on the console. Of course this sample is not very interesting and you may be better off sending the raw soap using request. Let's see something more interesting. If your service uses ssl + username token (transport with message credential), the config may look like this: The following modifications to the previous example will allow to consume it from node: ... binding = new WSHttpBinding( { SecurityMode: "TransportWithMessageCredential" , MessageClientCredentialType: "UserName" }) ... proxy.ClientCredentials.Username.Username = "yaron"; proxy.ClientCredentials.Username.Password = "1234"; proxy.send(...) And here is the wire soap: 2012-02-26T11:03:40Z 2012-02-26T11:08:40Z yaron 1234 123 If you use Mtom check out this code: (The formatting here is a bit strage due to my blog layout - it looks much better in github!) var CustomBinding = require('wcf.js').CustomBinding , MtomMessageEncodingBindingElement = require('wcf.js').MtomMessageEncodingBindingElement , HttpTransportBindingElement = require('wcf.js').HttpTransportBindingElement , Proxy = require('wcf.js').Proxy , fs = require('fs') , binding = new CustomBinding( [ new MtomMessageEncodingBindingElement({MessageVersion: "Soap12WSAddressing10"}), , new HttpTransportBindingElement() ]) , proxy = new Proxy(binding, "http://localhost:7171/Service/mtom") , message = '' + '' + '' + '' + '' + '' + '' + '' + '' + '' proxy.addAttachment("//*[local-name(.)='File1']", "me.jpg"); proxy.send(message, "http://tempuri.org/IService/EchoFiles", function(response, ctx) { var file = proxy.getAttachment("//*[local-name(.)='File1']") fs.writeFileSync("result.jpg", file) }); Mtom is a little bit trickier since wcf.js needs to know which nodes are binary. Using simple xpath can help you achieve that. Getting your hands dirty with Soap Wcf.js uses soap in its raw format. Code generation of proxies does not resonate well with a dynamic language like javascript. I also assume you are consuming an existing service which already has working clients so you should be able to get a working soap sample. And if you do like some level of abstraction between you and your soap I recommend node-soap, though it still does not integrate with wcf.js. If you will use raw soap requests and responses you would need a good xml library. And while node has plenty of dom / xpath libraries, they are not windows friendly. My next post will be on a good match here. Supported standards Wcf implements many of the ws-* standards and even more via proprietary extensions. The first version of wcf.js supports the following: MTOM WS-Security (Username token only) WS-Addressing (all versions) HTTP(S) The supported binding are: BasicHttpBinding WSHttpBinding CustomBinding What do you want to see next? Let me know. Get the code Wcf.js is hosted in GitHub, and everyone is welcome to contribute features and fixes if needed. Wcf.js is powered by ws.js, the actual standards implementation, which I will introduce in an upcoming post.

domain-driven design background there are a series of domain model patterns that describe objects and objects group built with domain-driven design. aggregates describe cohesive object graph with a single point of entry, called root: the internal objects of the aggregate cannot be persistently references from the outside. the domain classes whose instances are inside aggregates are subdivided into entities and value objects: the former have a lifecycle (like a post or a user), while the latter are just values with methods, equivalent to strings and other domain concepts. a prerequisite of these patterns is the immutability of value objects , which can then be shared between aggregates, just like string instances can be in many languages. value objects such as numbers and colors are modified by calling a method on them that return a new instance: every change to their state should produce a new value object. repositories are collection of aggregates: they model operations such as finding an aggregate or persisting a new one. a great departure of modern ddd from the entity/relationship modelling everyone knows is the duplication of data between aggregates to support new scenarios: it's possible some field or object is repeated in different aggregates. when there is an update to an aggregate, it's not necessarily atomically reflected to the other copies of its data. i'll refer to writing calls for generality, to indicate the command side of the command query sepration, which corresponds to everything that causes a change in state in the domain objects (in opposition to the reading side). events as mail messages thus it has become common to copy data between objects in different aggregates : for example, think of a document and invoice object that share the same start/finish date interval. traditionally this duplication is dealt with by extracting a common object, mapped to a common row in the database, with a name invented on the spot. domain events are an alternative that allows for duplicating these data: they reflect changes happened in a single aggregate, and are sent to other aggregates so that they can update themselves. technically speaking, domain events are plain old $yourlanguage objects, containing the modified data but not related to the orm like the main domain objects. domain events are handy for modelling "when" rules that should always be respected no matter who is writing to an aggregate; moreover, their handling can take place in the same transaction or even in a new one. my skeptic view of events was that it can be unclear which events are communicated between objects. after a while, i accepted that unit tests tell us that; moreover, communicating with events is a further level of abstraction which is unnecessary in simple domains but just a giant observer pattern in others. the underlying idea is that no matter who applies a command or modifies a domain object, we already configured the event handling mechanism so that consistency across aggregates is reached according to our policy defined in the event handlers (which may be immediate consistency, or eventual one. or it may result in sending a mail to a human asking him to review the changes: whatever you want.) the only alternative to propagate changes between aggregates would be to have many collaborators passed to the various repositories, but this solution couples the aggregates with each other in many way, while with events you're forced to define one-way messages. the event generator does not make any assumption about who will listen to the event and if it will be listened to at all: events are a point of decoupling like interface are for object collaboration. and it's not that we call static methods by passing a string. we have a clear contract, a domainevents static class, and we publish interesting events (like createdcar or updatedvoyageplan) as plain old domain object which contain all the information about the update, often even composing the relevant domain object. udi dahan discourages the reference to domain objects, and consider events just special value objects ; indeed as our solution matures we are moving towards simpler objects. this choice may force us to consider just what needs to be inserted in the message instead of a full reference (where and if serialization is used to transmit the event, it's simpler to use a value object in fact). moreover, it avoids possible further accidental writing calls to the domain object originating the event. in the application layer events are published by calling a static domain class: as a result event launchers cannot be decoupled from the event (as in udi dahan's approach). we launch events from the repository after an update has been performed, either by choosing an event class directly (in case of an update or creation) or by collecting the events from a queue on the relevant domain object, usually the root of the aggregate. this was a nice idea from a colleague of mine that let us decouple at least entities from the domainevents static class. for now we do not have the requirement to decouple the handling of events from the transaction , so the application layer (which is over the domain layer) open and commits/aborts a transaction, while reconstituting an aggregate, doing some "writing" work (updating it or executing a command) and saving it. the save triggers the event launch, which may trigger work on other aggregates through the configured handler: in case of an error the whole transaction is aborted, ensuring immediate consistency. so we aren't getting the scaling advantages of deferred handling (we're not interested in that for now), but the simplicity of communicating with events while writing code. dynamic language this a php-specific section: however, domain events are an approach typical of java or .net enterprise applications. we use php classes (or interfaces) for routing the events with instanceof; php is a shared nothing environment, so event configuration is done now on a per-action basis to avoid having to create all the objects handling events on each request. however, we want to move the configuration to the application level , with some lazy-loading: for example, configuring lazy event handlers as methods on factory objects that create the real handler and return it along with the name of the method to call. all communication between aggregates happen in a single process and a single address space (for now), so we don't use a bit of the decoupling properties of events. we map value objects into the relational database either as on the parent entity's table (decomposing their fields onto the entity) or as row of their own table. in any case, we have to ensure immutability via encapsulation and only assignign to $this->anyfield into the constructor. our standard pattern is to define setters as new self($this->field1, ..., $nwfieldvalue, ..., $this->fieldn); where n is a small number of fields. we map all domain object with the hibernate-equivalent doctrine 2. we are investigating how to deal with orphaned value objects, which are not reached anymore by any other entity.

Originally Authored by Dejan Noveski I'm not going to dive into how good or extendible Vim is. If you are reading this article, you probably know that. The thing that makes Vim so good, is the scripting environment behind it called VimL. Using this scripting language, you can write any functionality/plugin you need for Vim. Each plugin you use is written in this language. Here's the best part. You only need very little knowledge of VimL to be able to write plugins, if you know Python (or Ruby). What's a vim plugin anyway A Vim plugin is a .vim script that defines functions, mappings, syntax rules, commands that may, or may not, manipulate the windows, buffers, lines. It is a complete piece of code with some specific functionality. Usually, a plugin consists of several functions mappings command definitions and event hooks. When writing vim plugins with Python, often, everything outside the functions is written in VimL. But those are vim commands and they can be learned fast. In fact, VimL can be learned fast, but using python gives so much flexibility. Think about using urllib/httplib/simplejson for accessing some web service that helps editing in Vim. This is why most of the plugins that work with web services are usually done in VimL+Python. Any prerequisites? You must have vim compiled with +python support. You can check that using the command: vim --version | grep +python Vim package in Ubuntu and it's derivatives comes with +python support. To Work - Vimmit.vim What's better than starting with a simple example? This is a plugin that, when called, will retrieve the homepage of Reddit and will display it in the current buffer. Start by opening "vimmit.vim" file (in vim). Since we are writing python code, its good to check if Vim supports Python: if !has('python') echo "Error: Required vim compiled with +python" finish endif This piece is writen in VimL. It's best if we stick to VimL for things like this, mappings and event hooks. This function will check if Vim has python support or it will end the script with an error message. We continue with the main function Reddit(). This is where we use Python and do the main functionality: " Vim comments start with a double quote. " Function definition is VimL. We can mix VimL and Python in " function definition. function! Reddit() " We start the python code like the next line. python << EOF # the vim module contains everything we need to interface with vim from # python. We need urllib2 for the web service consumer. import vim, urllib2 # we need json for parsing the response import json # we define a timeout that we'll use in the API call. We don't want # users to wait much. TIMEOUT = 20 URL = "http://reddit.com/.json" try: # Get the posts and parse the json response response = urllib2.urlopen(URL, None, TIMEOUT).read() json_response = json.loads(response) posts = json_response.get("data", "").get("children", "") # vim.current.buffer is the current buffer. It's list-like object. # each line is an item in the list. We can loop through them delete # them, alter them etc. # Here we delete all lines in the current buffer del vim.current.buffer[:] # Here we append some lines above. Aesthetics. vim.current.buffer[0] = 80*"-" for post in posts: # In the next few lines, we get the post details post_data = post.get("data", {}) up = post_data.get("ups", 0) down = post_data.get("downs", 0) title = post_data.get("title", "NO TITLE").encode("utf-8") score = post_data.get("score", 0) permalink = post_data.get("permalink").encode("utf-8") url = post_data.get("url").encode("utf-8") comments = post_data.get("num_comments") # And here we append line by line to the buffer. # First the upvotes vim.current.buffer.append("↑ %s"%up) # Then the title and the url vim.current.buffer.append(" %s [%s]"%(title, url,)) # Then the downvotes and number of comments vim.current.buffer.append("↓ %s | comments: %s [%s]"%(down, comments, permalink,)) # And last we append some "-" for visual appeal. vim.current.buffer.append(80*"-") except Exception, e: print e EOF " Here the python code is closed. We can continue writing VimL or python again. endfunction Save the file, source it in vim (:source vimmit.vim) and: :call Reddit() Now, the way we call the function is not so elegant. So we define a command: command! -nargs=0 Reddit call Reddit() We define the command :Reddit to call the function. After adding this, open a new bufer and do :Reddit . Home page will be loaded in the buffer. The -nargs argument states how many arguments the command will take. Function Arguments, Eval and Command Q: How does one access functional arguments? function! SomeName(arg1, arg2, arg3) " Get the first argument by name in VimL let firstarg=a:arg1 " Get the second argument by position in Viml let secondarg=a:1 " Get the arguments in python python << EOF import vim first_argument = vim.eval("a:arg1") #or vim.eval("a:0") second_argument = vim.eval("a:arg2") #or vim.eval("a:1") You can define a function with arbitrary number of arguments by putting "..." instead of argument names. You can access these arguments only by position, and you can mix them with named arguments (arg1, arg2, ...) Q: How can I call Vim commands from Python? vim.command("[vim-command-here]") Q: How to define global variables and access them in VimL and Python? Global vars are prefixed with g:. If you want to define one in your script, best thing to do is check if it exists and if doesn't define it and assign some default value to it: if !exists("g:reddit_apicall_timeout") let g:reddit_apicall_timeout=40 endif You can access it from python using the vim module: TIMEOUT = vim.eval("g:reddit_apicall_timeout") If you want to override this setting, you can write: let g:reddit_apicall_timeout=60 in .vimrc . Additional Notes VimL is pretty easy once you try it. Remember that print works and everything you can do with python, you can do in here. Here you can find the documentation for the vim python module. Vimdoc is the possibly the only resource you will need when writing vim plugins. You can also check this IBM developerWorks article . Now, try to extend "vimmit.vim" so the user is able to choose a subreddit (as a first functional argument). Source: http://brainacle.com/how-to-write-vim-plugins-with-python.html

The @XmlAnyElement annotation enables a property to handle arbitrary XML elements.

A disparity map contains information related to the distance of the objects of a scene from a viewpoint. In this example we will see how to compute a disparity map from a stereo pair and how to use the map to cut the objects far from the cameras. The stereo pair is represented by two input images, these images are taken with two cameras separated by a distance and the disparity map is derived from the offset of the objects between them. There are various algorithm to compute a disparity map, the one implemented in OpenCV is the graph cut algorithm. To use it we have to call the function CreateStereoGCState() to initialize the data structure needed by the algorithm and use the function FindStereoCorrespondenceGC() to get the disparity map. Let's see the code: def cut(disparity, image, threshold): for i in range(0, image.height): for j in range(0, image.width): # keep closer object if cv.GetReal2D(disparity,i,j) > threshold: cv.Set2D(disparity,i,j,cv.Get2D(image,i,j)) # loading the stereo pair left = cv.LoadImage('scene_l.bmp',cv.CV_LOAD_IMAGE_GRAYSCALE) right = cv.LoadImage('scene_r.bmp',cv.CV_LOAD_IMAGE_GRAYSCALE) disparity_left = cv.CreateMat(left.height, left.width, cv.CV_16S) disparity_right = cv.CreateMat(left.height, left.width, cv.CV_16S) # data structure initialization state = cv.CreateStereoGCState(16,2) # running the graph-cut algorithm cv.FindStereoCorrespondenceGC(left,right, disparity_left,disparity_right,state) disp_left_visual = cv.CreateMat(left.height, left.width, cv.CV_8U) cv.ConvertScale( disparity_left, disp_left_visual, -20 ); cv.Save( "disparity.pgm", disp_left_visual ); # save the map # cutting the object farthest of a threshold (120) cut(disp_left_visual,left,120) cv.NamedWindow('Disparity map', cv.CV_WINDOW_AUTOSIZE) cv.ShowImage('Disparity map', disp_left_visual) cv.WaitKey() These are the two input image I used to test the program (respectively left and right): Result using threshold = 100 Result using threshold = 120 Result using threshold = 180 Source: http://glowingpython.blogspot.com/2011/11/computing-disparity-map-in-opencv.html

In my Django applications, I tend to use custom middleware extensively for common tasks. I have middleware that logs page runtime, middleware that sets context that most views will end up needing anyway, and middleware that copies the HTTP_REFERRER header from an entry page into the session scope for use later in the session. At some point, I inadvertently created a middleware class invalidated the browser cache for certain views. Typically, just wrapping a view in @cache_control(max_age=3600) is enough to have the browser cache that view for an hour. But if you do something innocuous like evaluate request.user.is_authenticated() in a middleware class, then Django will set the Vary: Cookie header, invalidating the cache. In my case, what I really wanted was a decorator that I could attach to a view that would skip my custom middleware, like an exclude list. Of course, you could just attach your middleware explicitly to each view that needs it, but that's needless code repetition if a middleware should wrap almost all views. You could also change each of your middleware classes to exclude particular views by URL, but you might end up having to alter many different middleware classes with that logic. As another option, you can use the following decorator/middleware pair to short-circuit the middleware execution of any view, for any middleware defined in your settings file AFTER this one. """ Allows short-curcuiting of ALL remaining middleware by attaching the @shortcircuitmiddleware decorator as the TOP LEVEL decorator of a view. Example settings.py: MIDDLEWARE_CLASSES = ( 'django.middleware.common.CommonMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', # THIS MIDDLEWARE 'myapp.middleware.shortcircuit.ShortCircuitMiddleware', # SOME OTHER MIDDLE WARE YOU WANT TO SKIP SOMETIMES 'myapp.middleware.package.MostOfTheTimeMiddleware', # MORE MIDDLEWARE YOU WANT TO SKIP SOMETIMES HERE ) Example view to exclude from MostOfTheTimeMiddleware (and any subsequent): @shortcircuitmiddleware def myview(request): ... """ def shortcircuitmiddleware(f): """ view decorator, the sole purpose to is 'rename' the function '_shortcircuitmiddleware' """ def _shortcircuitmiddleware(*args, **kwargs): return f(*args, **kwargs) return _shortcircuitmiddleware class ShortCircuitMiddleware(object): """ Middleware; looks for a view function named '_shortcircuitmiddleware' and short-circuits. Relies on the fact that if you return an HttpResponse from a view, it will short-circuit other middleware, see: https://docs.djangoproject.com/en/dev/topics/http/middleware/#process-request """ def process_view(self, request, view_func, view_args, view_kwargs): if view_func.func_name == "_shortcircuitmiddleware": return view_func(request, *view_args, **view_kwargs) return None Source: http://bitkickers.blogspot.com/2011/08/django-exclude-some-views-from.html

kendo ui is html 5 and jquery based framework and it helps you to create modern web applications. kendo ui helps you in data binding in animations with ui widgets like grid and chart with drag and drop api in touch support. download kendo ui from here once you download you get these folders: navigate to the 'example' folder for examples of various widgets. if you want to start developing web applications using kendoui then you need to add the required file in your project. you need to add the below files in the script folder: and you need to add the below files in the style folder: even though i have added script files and css files in the script folder and style folders respectively, you are free to keep them anywhere you want. after adding these files you need to link them in the header of the html page. you can add the reference as below: in a later post i will go into the details of kendo ui and play around with all other aspects. however, working with any widgets is very intuitive. for example, if you want to work with kendo autocomplete , you can do that as below: and using jquery you can assign the value as below: putting all html and script code together: test.htm kendo ui demo when you run test.htm in your browser, you should get this output: in later posts i will get into detail about all widgets. i hope this post is useful. thanks for reading. source: http://debugmode.net/2012/02/18/introduction-to-telerik-kedno-ui/

Erlang is a language and platform designed by Ericsson, whose scope is supporting distributed, real-time and fault-tolerant computation. It has become famous in the programming world probably as the platform on which CouchDB was developed. Erlang has some peculiarities that make it different from the usual imperative, mainstream languages: it is a dynamic, functional language; even more than Clojure in some aspects. As a result, it forces single assignments and immutability: you cannot change existing variables, only create new ones. All state is kept on the stack in the form of function calls and their parameters. Concurrency and message passing is supported at the language level, as actors (called processes in Erlang). Actor model Every process of the Erlang virtual machine is an actor: actors execute independently and communicate only with one-way messages. Actors can create other actors. When an actor finished its computation, it disappears. Like all paradigms, the actor model is defined by what it takes away from the programmer: no shared memory between processes. No remote procedure call: only messages sent via asynchronous invocations. As a result, no return values since messages go in one direction (although of course this can be simulated with another, separate message.) Note that Erlang processes run inside one or more virtual machines on one or more nodes of a network, so they do not map to OS processes. An echo server Without taking too much consideration for Erlang's syntax, let's see it in action. An echo server is a process sending you back the messages you send to it. This first example will serve to use both to learn a bit of Erlang's look&feel, and to introduce the primitives for creating actors. -module(echo). -export([start/0]). loop() -> receive {Sender, Num} -> Sender ! Num, loop() end. start() -> spawn(fun loop/0). This is contained in a echo.erl file, where we define a module named echo. All identifiers starting with lowercase are atoms, while variables start with an uppercase letter. This means Num is a variable, while num would be a constant symbol that cannot be assigned but only passed around (like Ruby symbols or Clojure keywords). We define two functions loop() and start(), but export only start/0 (the start version with 0 arguments); the complete signature of a function always comprehends its arity (the number of arguments), so start() and start(Variable) would be two different functions. In loop/0, we use the receive primitive to receive one new message from the queue for this process. We define a pattern for the message - we only accept tuples containing two variables. After receival, we send back a message to the process identified by the first half of the tuple, containing the number in its second half. We then restart listening by tail recurring on loop() until a new message is delivered. start/0 is instead a primitive that has to be called from the originating process. It would call spawn/1 to create another process which will execute the loop function; it will then return the control to the calling process. Here's how to use this echo server. The shell will be our originating process, while the process for echo will be created: [giorgio@Desmond:~]$ erl Erlang R14B02 (erts-5.8.3) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] Eshell V5.8.3 (abort with ^G) We compile the echo.erl source file: 1> c(echo). {ok,echo} We call start() from this process. Another process is spawned, but spawn() returns its process id. Since it is the last statement of start(), also start() returns the same value, which we can save in a variable. The value of the id is shown, as all other return values of local function calls: 2> Pid = echo:start(). <0.39.0> Now we send a message to Pid, a tuple containing two values. A tuple is a simple data structure containing a fixed number of values (it is the generalization of a pair or of a triplet): 3> Pid ! {self(), 42}. {<0.32.0>,42} Now the other process will handle the message and send one back to us. When we want to receive it, we can invoke receive with an identity handler, just displaying what we have got back: 4> receive Value -> Value end. 42 Let's complicate this a bit Let's try to encapsulate some state into our server. We code a new module called var, which will represent a number we can add to or display. -module(var). -export([init/0, add/1, get/0]). init() -> Pid = spawn(fun() -> loop(0) end), register(variable, Pid). % we keep track of the process id loop(N) -> receive {add, X} -> loop(N+X); % we accept more than one type of message: the first has the atom add as the first element for identification {Parent, get} -> % in case of get, before restarting the loop we send a message to Parent with the value Parent ! N, loop(N) end. add(X) -> % these primitives can be called from the parent process, like init() variable ! {add, X}. get() -> variable ! {self(), get}, receive Result -> Result end. The server has the same structure, but it accepts more than one type of message, distinguishing between them via pattern matching. Moreover, it performs some operations like Parent ! N before tail recurring. The add/1 and get/0 primitives are meant to be called from the client process, and encapsulate the messages to send to the server process, along with the blocking for receiving results. To manage multiple created processes, we would have to save pids in a list. Conclusion As you know, different approaches to concurrency such as functional programming and the Actor model are likely to play a role in the near future due to the rise of parallel and distributed computing. To delve into this new world, I'm currently test-driving a distributed tuple space (like JavaSpaces) in Erlang: the mix of a functional language and the actor model is a big change for an OO developer to deal with.

This is an experiment for a real-time tracking of a football match using only web technologies. In this case I’ve chosen to track a football match from Futbol Club Barcelona (fcb). All the work is made by the browser and it’s interactive (there are four checkboxes to activate/deactivate each tracker). It is based on tracking the location of colored objects in the frames using the HSV colour space. As a practical use for the tracker, I’ve added a smart volume feature that activates the sound only when there are goal chances (every time the ball is near the area). So you can browse other pages and change to the video one only when chances are happening. It works based on the quantity of green color in the sides of the video. The path tracker shows ‘possible’ passes between fcb players If you want to watch it in action here is the demo So far it only works in chrome and firefox and because of the low resolution of the video (and my limited skills), the tracking is not perfect, but it could be improved for tracking videos from external sources like Youtube, Veetle, etc. Here is the code: function rgbToHsv(r, g, b){ r = r/255, g = g/255, b = b/255; var max = Math.max(r, g, b), min = Math.min(r, g, b); var h, s, v = max; var d = max - min; s = max == 0 ? 0 : d / max; if(max == min){ h = 0; // achromatic }else{ switch(max){ case r: h = (g - b) / d + (g < b ? 6 : 0); break; case g: h = (b - r) / d + 2; break; case b: h = (r - g) / d + 4; break; } h /= 6; } return [h, s, v]; } //Main object var processor = { isFirefox35: function() { // Let a chance to the navigator to deal with it: return true; var ua = navigator.userAgent; // Gecko ? if (ua.indexOf("Gecko") == -1) return false; // Geck >= 1.9.1 ? return !(ua.indexOf("rv:1.9.1") == -1 && ua.indexOf("rv:1.9.2") == -1); }, // Init doLoad: function() { if (!this.isFirefox35()) { document.getElementById("nofirefoxbeta").style.display = "block"; } // Some init this.displayBackground = true; this.video = document.getElementById("video"); this.mirrorVideo = document.getElementById("mirrorVideo"); this.mirrorVideoCtx = this.mirrorVideo.getContext("2d"); var self = this; // If the videos end, play again this.video.addEventListener("ended", function(){ try { clearTimeout(self.timeout); } catch(e){} self.video.play(); // Work around: https://bugzilla.mozilla.org/show_bug.cgi?id=488287 self.videoIsPlaying(); }, true); // Set the events listeners for the main video (update button) this.video.addEventListener("pause", function() { self.updateButtons(false); }, false); this.pageLoaded = true; this.startPlayer(); }, videoIsPlaying: function() { this.updateButtons(true); this.timerCallback(); }, videoIsReady: function() { this.videoLoaded = true; this.startPlayer(); }, startPlayer: function() { if (!this.videoLoaded || !this.pageLoaded) return; document.getElementById("wait").style.display = "none"; document.getElementById("player").style.display = "block"; this.width = this.video.videoWidth; this.height = this.video.videoHeight; this.mirrorVideo.width = this.width; this.mirrorVideo.height = this.height; }, playVideo:function(){ this.video.play(); this.videoIsPlaying(); }, stopVideo: function(){ this.video.pause(); clearTimeout(this.timeout); }, volumeDown:function(){ if(this.video.volume>0) this.video.volume=Math.round((this.video.volume - 0.1)*10)/10; }, volumeUp:function(){ if(this.video.volume<1) this.video.volume=Math.round((this.video.volume + 0.1)*10)/10; }, // Main loop timerCallback: function() { if (this.video.paused || this.video.ended) { return; } this.computeFrame(); var self = this; this.timeout = setTimeout(function () { self.timerCallback(); }, 50); }, // Update the SVG button updateButtons: function(play) { document.getElementById("playButton").setAttribute("play", play); document.getElementById("stopButton").setAttribute("play", play); }, // Handling some patterns (text, drawing) has_green_around: function(frameData, pos) { pos_left = pos+ 24; pos_right = pos - 24; r_left = frameData[pos_left+0]; g_left = frameData[pos_left+1]; b_left = frameData[pos_left+2]; r_right = frameData[pos_right+0]; g_right = frameData[pos_right+1]; b_right = frameData[pos_right+2]; return ((r_left < 125 && g_left > 125 && b_left < 80) && (r_right < 125 && g_right > 125 && b_right < 80)); }, is_team_color: function(frameData, pos){ for (i=pos-4; i<=pos+4; i+=4){ r = frameData[i+0]; g = frameData[i+1]; b = frameData[i+2]; hsv = rgbToHsv(r,g,b); //if(hsv[0] < 0.60|| hsv[1] < 0.35) if((hsv[0] < 0.40 || hsv[1] < 0.25 || hsv[2] < 0) || (hsv[0] > 0.70 || hsv[1] > 1 || hsv[2] > 1)) return false; } return true; }, is_ball_color: function(frameData, pos){ for (i=pos-4; i<=pos+4; i+=4){ r = frameData[i+0]; g = frameData[i+1]; b = frameData[i+2]; hsv = rgbToHsv(r,g,b); if((hsv[0] < 0.23 || hsv[1] < 0.40 || hsv[2] < 0.90) || (hsv[0] > 0.27 || hsv[1] > 0.50 || hsv[2] > 1)) return false; } return true; }, dist: function(x1, y1, x2, y2) { return Math.sqrt((x1 - x2) * (x1 - x2) + (y1 - y2) * (y1 - y2)); }, computeFrame: function() { try { this.mirrorVideoCtx.clearRect(0, 0, this.width, this.height); this.mirrorVideoCtx.drawImage(this.video, 0, 0, this.width, this.height); } catch(e) { return; } var frame = this.mirrorVideoCtx.getImageData(0, 0, 640, 360); this.mirrorVideoCtx.fillStyle = 'rgba(200,200,200,0.5)'; this.mirrorVideoCtx.strokeStyle = 'rgba(200,200,200,0.5)'; this.mirrorVideoCtx.lineWidth = 1; var x, y; var weight = 0; var team_shape = null; var ball_shape = null; var ball_found = false; var ball_tracker = document.getElementById("balltracker").checked; var team_tracker = document.getElementById("teamtracker").checked; var path_tracker = document.getElementById("pathtracker").checked; var smart_volume = document.getElementById("smartvolume").checked; var shapes = []; var x, y; var green_bar_left = 0; var green_bar_rigth =0; var green_bar_down = 0; var frame_length = frame.data.length/4; // We dont' need to compute each pixels var step = 4; for (var i = 0; i < frame_length; i += step) { pos = i*4; x = i % this.width; y = Math.round(i / this.width); if (x == 4) green_bar_left += g; else if (x == this.width-4) { green_bar_rigth += g; if (y == 4) green_bar_down += g; } ball_found = ball_tracker && this.is_ball_color(frame.data, pos); team_found = team_tracker && this.is_team_color(frame.data, pos); if (this.has_green_around(frame.data, pos) && (ball_found || team_found)){ if (ball_found) { ball_shape = {}; ball_shape.x = x; ball_shape.y = y; } if(team_found) { if (!team_shape) { // no shape yet, create the first one team_shape = {}; team_shape.x = x; team_shape.y = y; team_shape.weight = 1; shapes.push(team_shape); } else { var d = this.dist(x, y, team_shape.x, team_shape.y); if (d>25){ team_shape = {}; team_shape.x = x; team_shape.y = y; team_shape.weight = 1; shapes.push(team_shape); } } } } } if (shapes.length>0) { if (path_tracker){ this.mirrorVideoCtx.beginPath(); this.mirrorVideoCtx.moveTo(shapes[0].x, shapes[0].y); } for( var s=0; s0 && Math.abs(green_left_average-green_right_average)>15){ //this.volumeUp(); this.video.volume=1; document.getElementById("soundIcon").setAttribute("mute", false); }else{ this.volumeDown(); //this.video.volume=0; document.getElementById("soundIcon").setAttribute("mute", true); } } return; } } Source: http://lusob.com/2012/02/tracking-a-football-match-with-html5-and-javascript/

Few years back I blogged about adding OpenID login support in ASP.NET application. This time I am blogging about adding Google login support in ASP.NET application. A friend of mine is trying to integrate multiple third party authentication support for one of the application he is developing for his client. He is using DotNetOpenAuth for Google authentication. the code I am using here is from my friend and I am sharing it with his explicit permission. First, download the latest version of DotNetOpenAuth and add its reference in your web application and these two namespaces. using DotNetOpenAuth.OpenId; using DotNetOpenAuth.OpenId.RelyingParty; After adding the reference, add a normal button with CommandArgument to point https://www.google.com/accounts/o8/id. On the button click event on the server side: protected void btnGoogleLogin_Click(object sender, CommandEventArgs e) { string discoveryUri = e.CommandArgument.ToString(); OpenIdRelyingParty openid = new OpenIdRelyingParty(); var URIbuilder = new UriBuilder(Request.Url) { Query = "" }; var req = openid.CreateRequest(discoveryUri, URIbuilder.Uri, URIbuilder.Uri); req.RedirectToProvider(); } Now when you click the button it will take you to Google login page which look something like this. You can see on the right side of the screen with the information of the site requesting the authentication. Once you get successfully authenticated with your entered user name and password, you will then redirect to the confirmation page: As I am using my local development server, you will see Locahost. Once you deploy the application in the production environment it will automatically get the name of the domain. Clicking on the Sign in button you will then be redirected to the main page, but before you get to the main page you need to check whether the authentication was successful or was it cancelled by the user or was failed. To make sure use the below code on the load event of the login page: protected void Page_Load(object sender, EventArgs e) { OpenIdRelyingParty rp = new OpenIdRelyingParty(); var response = rp.GetResponse(); if (response != null) { switch (response.Status) { case AuthenticationStatus.Authenticated: Session["GoogleIdentifier"] = response.ClaimedIdentifier.ToString(); Response.Redirect("Default.aspx"); break; case AuthenticationStatus.Canceled: Session["GoogleIdentifier"] = "Cancelled."; break; case AuthenticationStatus.Failed: Session["GoogleIdentifier"] = "Login Failed."; break; } } } On Default.aspx page I have set the ClaimedIdentifier: The response/status returned from Google will be checked here and we will redirect the application to work the way accordingly. My friend sends me the above code to find out whether there is any way we can logout from the service. Well, unfortunately there isn't any specific way to log out using DotNetOpenAuth? But there is a workaround. I don't know if it is a good practice or bad but it worked out for me. To logout, I am just calling this logout URL used by Google. http://accounts.google.com/logout If you have some suggestions or you know a better way or approach of doing this then please drop a line in the comments sections.

Using regular expressions in eclipse to search and replace text in a file, is straight forward and very handy! All you need to know is that the matched text is going to be available in the replace textbox using the '$' dollar sign. So to access the first matched element use $1, for the second $2 and so on. I had an old method with hundreds of lines doing calling a getAttribute("X") and casting the result to a string. (String)object1.getAttribute("X") (String)object2.getAttribute("Y") (String)objectN.getAttribute("Z") I had to change them all to use a new method that checks if the attribute is null. So the new line would be getSafeStringAttribute(object1,"X") getSafeStringAttribute(object2,"Y") getSafeStringAttribute(objectN,"Z") With this simple regEx you can do a replace all! find : \(String\)(.+)\.getAttribute\("(.+)"\) replace: getSafeStringAttribute($1,"$2") The first (.+) will match the objectX part while the second will match the attribute name. The best thing is that when you select some text and type CTRL + F (if the Regular Expressions checkbox is ticked) you string in the find will be already escaped from characters like '(', ')' etc! From http://www.devinprogress.info/2012/02/eclipse-regular-expressions-find.html

In January, Google released LevelDB, "a fast and lightweight key/value database library." In a recent post on the "High Availability MySQL" blog has generated a discussion around the possibility of LevelDB being a storage engine for MySQL due to its performance benefits. The discussion generated some insight LevelDB's comparative performance to MySQL. The LevelDB site provides some insight into these performance benefits. When creating a brand new database, various methods shows a range of speeds from .4 MB/s to 62.7 MB/s in Write performance. In Read performance, LevelDB ranged from 152 MB/s to 232 MB/s. You can see a more detailed explanation of these benchmarks by checking out the LewisDB site here. The "High Availability MySQL" blog also suggests that LevelDB may be a "great fit" for MongoDB because it does not require multi-statement transactions. Commenters pointed out a few more details about LevelDB that may limit its performance: Unfortunately, there is a trade off between number of SST files and query latency variation: the larger single storage file is - the more time will require to compact it -- Vladmir Rodionov A recent GitHub post also compared MySQL and LevelDB. For sequential insert performance, LevelDB was found to get higher throughput/lower latency overall, although MySQL was more stable. For both average latency and update performance, MySQL and LevelDB performed essentially the same. Have you had a chance to use LevelDB? How does it compare to other libraries? Please post your comments below.

My last blog was the first in a short series of blogs examining Spring 3 MVC’s exception handling annotations. It covered the basic usage of the @ExceptionHandler annotation with a few pieces of demo code and no contrived scenarios. Today’s blog continues where I left off and examines @ExceptionHandler in more detail. You may recall that my last blog demonstrated1 that the @ExceptionHandler obeys the usual rules of exception handling in that any annotated method would catch the specified exception and its subclasses. My demo code caught IOException, which means that ChangedCharSetException, CharacterCodingException, CharConversionException, ClosedChannelException, EOFException,FileLockInterruptionException, FileNotFoundException, FilerException, etc. etc... would also be caught, if thrown by the sample application. So, what happens if I want to catch two or more unrelated exceptions? This is achieved by specifying @ExceptionHandler with an array of exception classes. For example, given the two flaky controller methods below, one throwing NullPointerException and the other throwing NoSuchRequestHandlingMethodException... @RequestMapping(value = "/my404", method = RequestMethod.GET) public String throwNoSuchRequestHandlingMethodException(Locale locale, Model model) throws NoSuchRequestHandlingMethodException { logger.info("This will throw a NoSuchRequestHandlingMethodException, which is Spring's 404 not found"); boolean throwException = true; if (throwException) { throw new NoSuchRequestHandlingMethodException( "This is my NoSuchRequestHandlingMethodException", this.getClass()); } return "home"; } @RequestMapping(value = "/nullpointer", method = RequestMethod.GET) public String throwNullPointerException(Locale locale, Model model) throws NoSuchRequestHandlingMethodException { logger.info("This will throw a NullPointerException"); String str = null; // Ensure that this is null. str.length(); return "home"; } … then I can catch them both by writing an exception handler that looks like this: @ExceptionHandler({ NullPointerException.class, NoSuchRequestHandlingMethodException.class }) public ModelAndView handleExceptionArray(Exception ex) { logger.info("handleExceptionArray - Catching: " + ex.getClass().getSimpleName()); return errorModelAndView(ex); } /** * Get the users details for the 'personal' page */ private ModelAndView errorModelAndView(Exception ex) { ModelAndView modelAndView = new ModelAndView(); modelAndView.setViewName("error"); modelAndView.addObject("name", ex.getClass().getSimpleName()); modelAndView.addObject("user", userDao.readUserName()); return modelAndView; } There is a slight GOTCHA to avoid here. If you choose to pass the exception in as an argument as I have, then you need to ensure that it’s a super class of all the exceptions in the array. In the above case I’ve used a straight forward exception class: public ModelAndView handleExceptionArray(Exception ex) If you get this wrong, then you won’t know about it until run-time when you’ll see the following exception in your server’s log file: ERROR: org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerExceptionResolver - Invoking request method resulted in exception : public org.springframework.web.servlet.ModelAndView com.captaindebug.exceptions.ExceptionsDemoController.handleExceptionArray(java.lang.NullPointerException) java.lang.IllegalStateException: Unsupported argument [java.lang.NullPointerException] for @ExceptionHandler method: public org.springframework.web.servlet.ModelAndView com.captaindebug.exceptions.ExceptionsDemoController.handleExceptionArray(java.lang.NullPointerException) at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerExceptionResolver.resolveHandlerArguments(AnnotationMethodHandlerExceptionResolver.java:241) at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerExceptionResolver.doResolveException(AnnotationMethodHandlerExceptionResolver.java:128) at org.springframework.web.servlet.handler.AbstractHandlerExceptionResolver.resolveException(AbstractHandlerExceptionResolver.java:136) at org.springframework.web.servlet.DispatcherServlet.processHandlerException(DispatcherServlet.java:987) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:811) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:549) ...which may or may not be caught in testing as it's easily missed unless you check for every exception. There are another couple of points to cover when it comes to Spring 3 MVC exception handling including the use of the @ResponseStatus annotation, but more on that later.... The full webapp sample is available at: git://github.com/roghughe/captaindebug.git See the Spring documentation for reference material. From http://www.captaindebug.com/2012/02/spring-3-mvc-exception-handlers-and.html

There’s still a lot of confusion over the difference types of managed beans provided in Java EE 6 with EJBs, CDI beans and JSF managed beans all being available. This article aims to clear up some of the differences between the them and define when to use them. A number of people assume that there is some meaning to all these different types of beans that they just don’t understand. However, the problem is down to the different APIs overlapping which is unfortunate. JSF Managed Beans, CDI Beans and EJBs JSF was initially developed with its own managed bean and dependency injection mechanism which was enhanced for JSF 2.0 to include annotation based beans. When CDI was released with Java EE 6, it was regarded as the managed bean framework for that platform and of course, EJBs outdated them all having been around for well over a decade. The problem of course is knowing which one to use and when, but they all involve the same process. Typically a class has to be identified as a managed bean, and where necessary, will need a scope,qualifiers and a name if it is to be used in JSF. What follows is a brief description of the different types of managed beans and how and when to use them. Let’s start with the simplest, JSF Managed beans. JSF Managed Beans In short, don’t use them if you are developing for Java EE 6 and using CDI. They provide a simple mechanism for dependency injection and defining backing beans for web pages, but they are far less powerful than CDI beans. They can be defined using the @javax.faces.bean.ManagedBean annotation which takes an optional name parameter. This name can be used to reference the bean from JSF pages. Scope can be applied to the bean using one of the different scopes defined in the javax.faces.bean package which include the request, session, applicaion, view and custom scopes. @ManagedBean(name="someBean") @RequestScoped public class SomeBean { .... .... } JSF beans cannot be mixed with other kinds of beans without some kind of manual coding. CDI Beans CDI is the bean management and dependency injection framework that was released as part of Java EE 6 and it includes a complete, comprehensive managed bean facility. CDI beans are far more advanced and flexible than simple JSF managed beans. They can make use of interceptors, conversation scope, Events, type safe injection, decorators, stereotypes and producer methods. To deploy CDI beans, you must place a file called beans.xml in a META-INF folder on the classpath. Once you do this, then every bean in the package becomes a CDI bean. There are a lot of features in CDI, too many to cover here, but as a quick reference for JSF-like features, you can define the scope of the CDI bean using one of the scopes defined in the javax.enterprise.context package (namely, request, conversation, session and application scopes). If you want to use the CDI bean from a JSF page, you can give it a name using the javax.inject.Named annotation. To inject a bean into another bean, you annotate the field with javax.inject.Inject annotation. @Named("someBean") @RequestScoped public class SomeBean { @Inject private SomeService someService; } Automatic injection like that defined above can be controlled through the use of Qualifiers that can help match the specific class that you want injected. If you have multiple payment types, you might add a qualifier for whether it is asynchronous or not. While you can use the @Named annotation as a qualifier, you shouldn’t as it is provided for exposing the beans in EL. CDI handles the injection of beans with mismatched scopes through the use of proxies. Because of this you can inject a request scoped bean into a session scoped bean and the reference will still be valid on each request because for each request, the proxy re-connects to a live instance of the request scoped bean. CDI also has support for interceptors, events, the new conversation scope and many other features which makes it a much better choice over JSF managed beans. EJB EJBs predate CDI beans and are in someways similar to CDI beans and in other ways very different. Primarily, the differences between CDI beans and EJBs is that EJBs are : Transactional Remote or local Able to passivate stateful beans freeing up resources Able to make use of timers Can be asynchronous The two types of EJBs are called stateless and stateful. Stateless EJBs can be thought of as thread safe single-use beans that don’t maintain any state between two web requests. Stateful EJBs do hold state and can be created and sit around for as long as they are needed until they are disposed of. Defining an EJB is simple, you just add either a javax.ejb.Stateless or javax.ejb.Stateful annotation to the class. @Stateless public class BookingService { public String makeReservation(Item Item,Customer customer) { ... ... } } Stateless beans must have a dependent scope while a stateful session bean can have any scope. By default they are transactional, but you can use the transaction attribute annotation. While EJBs and CDI beans are very different in terms of feaures, writing the code to integrate them is very similar since CDI beans can be injected into EJBs and EJBs can be injected into CDI beans. There is no need to make any distinction when injecting one into the other. Again, the different scopes are handled by CDI through the use of proxying. One exception to this is that CDI does not support the injection of remote EJBs but that can be implemented by writing a simple producer method for it. The javax.inject.Named annotation as well as any Qualifiers can be used on an EJB to match it to an injection point. When to use which bean How do you know when to use which bean? Simple. Never use JSF managed beans unless you are working in a servlet container and don’t want to try and get CDI working in Tomcat (although I have a Maven archetype for that so there’s no excuse). In general, you should use CDI beans unless you need the advanced functionality available in the EJBs such as transactional functions. You can write your own interceptor to make CDI beans transactional, but for now, its simpler to use an EJB until CDI gets transactional CDI beans which is just around the corner. If you are stuck in a servlet container and are using CDI, then either hand written transactions or your own transaction interceptor is the only option without EJBs. From http://www.andygibson.net/blog/article/comparing-jsf-beans-cdi-beans-and-ejbs/

In this post I will demonstrate how to use the propOrder property on the @XmlType annotation to control the ordering of XML elements.

PDF files aren't pleasant. The good news is that they're documented (http://www.adobe.com/devnet/pdf/pdf_reference.html). The bad news is that they're rather complex. I found four Python packages for reading PDF files. http://pybrary.net/pyPdf/ - weak http://www.swftools.org/gfx_tutorial.html - depends on binary XPDF http://blog.didierstevens.com/programs/pdf-tools/ - limited http://www.unixuser.org/~euske/python/pdfminer/ - acceptable I elected to work with PDFMiner for two reasons. (1) Pure Python, (2) Reasonably Complete. This is not, however, much of an endorsement. The implementation (while seemingly correct for my purposes) needs a fair amount of cleanup. Here's one example of remarkably poor programming. # Connect the parser and document objects. parser.set_document(doc) doc.set_parser(parser) Only one of these two is needed; the other is trivially handled as part of the setter method. Also, the package seems to rely on a huge volume of isinstance type checking. It's not clear if proper polymorphism is even possible. But some kind of filter that picked elements by type might be nicer than a lot of isinstance checks. Annotation Extraction While shabby, the good news is that PDFMiner seems to reliably extract the annotations on a PDF form. In a couple of hours, I had this example of how to read a PDF document and collect the data filled into the form. from pdfminer.pdfparser import PDFParser, PDFDocument from pdfminer.psparser import PSLiteral from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter, PDFTextExtractionNotAllowed from pdfminer.pdfdevice import PDFDevice from pdfminer.pdftypes import PDFObjRef from pdfminer.layout import LAParams, LTTextBoxHorizontal from pdfminer.converter import PDFPageAggregator from collections import defaultdict, namedtuple TextBlock= namedtuple("TextBlock", ["x", "y", "text"]) class Parser( object ): """Parse the PDF. 1. Get the annotations into the self.fields dictionary. 2. Get the text into a dictionary of text blocks. The key to the dictionary is page number (1-based). The value in the dictionary is a sequence of items in (-y, x) order. That is approximately top-to-bottom, left-to-right. """ def __init__( self ): self.fields = {} self.text= {} def load( self, open_file ): self.fields = {} self.text= {} # Create a PDF parser object associated with the file object. parser = PDFParser(open_file) # Create a PDF document object that stores the document structure. doc = PDFDocument() # Connect the parser and document objects. parser.set_document(doc) doc.set_parser(parser) # Supply the password for initialization. # (If no password is set, give an empty string.) doc.initialize('') # Check if the document allows text extraction. If not, abort. if not doc.is_extractable: raise PDFTextExtractionNotAllowed # Create a PDF resource manager object that stores shared resources. rsrcmgr = PDFResourceManager() # Set parameters for analysis. laparams = LAParams() # Create a PDF page aggregator object. device = PDFPageAggregator(rsrcmgr, laparams=laparams) # Create a PDF interpreter object. interpreter = PDFPageInterpreter(rsrcmgr, device) # Process each page contained in the document. for pgnum, page in enumerate( doc.get_pages() ): interpreter.process_page(page) if page.annots: self._build_annotations( page ) txt= self._get_text( device ) self.text[pgnum+1]= txt def _build_annotations( self, page ): for annot in page.annots.resolve(): if isinstance( annot, PDFObjRef ): annot= annot.resolve() assert annot['Type'].name == "Annot", repr(annot) if annot['Subtype'].name == "Widget": if annot['FT'].name == "Btn": assert annot['T'] not in self.fields self.fields[ annot['T'] ] = annot['V'].name elif annot['FT'].name == "Tx": assert annot['T'] not in self.fields self.fields[ annot['T'] ] = annot['V'] elif annot['FT'].name == "Ch": assert annot['T'] not in self.fields self.fields[ annot['T'] ] = annot['V'] # Alternative choices in annot['Opt'] ) else: raise Exception( "Unknown Widget" ) else: raise Exception( "Unknown Annotation" ) def _get_text( self, device ): text= [] layout = device.get_result() for obj in layout: if isinstance( obj, LTTextBoxHorizontal ): if obj.get_text().strip(): text.append( TextBlock(obj.x0, obj.y1, obj.get_text().strip()) ) text.sort( key=lambda row: (-row.y, row.x) ) return text def is_recognized( self ): """Check for Copyright as well as Revision information on each page.""" bottom_page_1 = self.text[1][-3:] bottom_page_2 = self.text[2][-3:] pg1_rev= "Rev 2011.01.17" == bottom_page_1[2].text pg2_rev= "Rev 2011.01.17" == bottom_page_2[0].text return pg1_rev and pg2_rev This gives us a dictionary of field names and values. Essentially transforming the PDF form into the same kind of data that comes from an HTML POST request. An important part is that we don't want much of the background text. Just enough to confirm the version of the form file itself. The cryptic text.sort( key=lambda row: (-row.y, row.x) ) will sort the text blocks into order from top-to-bottom and left-to-right. For the most part, a page footer will show up last. This is not guaranteed, however. In a multi-column layout, the footer can be so close to the bottom of a column that PDFMiner may put the two text blocks together. The other unfortunate part is the extremely long (and opaque) setup required to get the data from the page. Source: http://slott-softwarearchitect.blogspot.com/2012/02/pdf-reading.html