Some people have tried to argue that source code is a liability, not an asset. Apparently this “is now widely accepted” and... “this is a very strong idea that has a lot of impact across the IT industry and in the way developers view and perform their day-to-day work”. Really? The argument, as far as I can follow it, is that while engineers are paid to help design and build bridges and power plants, as developers we’re paid to “deliver business value”, and… “Source code is merely the necessary evil that’s required to create value” Source code, the software that we create, is only a means to and end. The software itself has no value, or worse it has negative value, because it creates a drag on your ability to innovate and move forward. The more code that you have, the higher your maintenance costs will be, therefore… “… the best code of all is the code that's never written.” Michael Feathers, who has a lot of smart things to say about source code, joined in on this discussion. In The Carrying-Cost of Code he says that “code is inventory. It is stuff lying around and it has substantial cost of ownership. It might do us good to consider what we can do to minimize it.” He goes so far as to suggest a goofy thought experiment where “every line of code written disappears exactly three months after it is written”. The point of this would be to get developers and the business to understand that the “costs of carrying code are real, but no one accounts for them”. Feathers reinforces the valid points about the drag that unmaintained or poorly maintained legacy code has on companies. Writing less code to solve a problem is a good thing – it’s (usually) more efficient and (usually) costs less to maintain a smaller code base. And yes there is a necessary cost to maintaining software and working with existing software and changing it. But none of this changes the fact that software is an asset If you build and operate a power plant or a bridge, you have to maintain it – just like software. And like a bridge or a power plant, a newer, more modern, better-designed, more efficient and simpler asset is better than a big, old, complicated, expensive-to-maintain one. The “software is a liability” argument seems to be that it’s not the software that’s the asset, it’s the “features and options” – the capabilities that the software provides. This is like saying that it’s not the power plant (which a company spent millions of dollars to design and engineer) that’s a valuable asset to a company, it’s the energy that it generates. It’s not the bridge – it’s the ability to drive over water. It’s not the airplane, it’s the ability to fly. Pretending that software has no value in itself is silly. Try explaining this to accountants (don’t depreciate the airplane, depreciate the ability to fly!) and IP lawyers and to investors who buy software companies for their IP. They all understand that software and the ideas embodied in it are valuable and need to be treated as assets. The ideas themselves are only worth so much, even if they’re patented. But the ideas realized in software, actualized and proven and ready to be used or (better) already being used – that’s where the real value is. And this is the value that needs to be maintained and preserved. Software is more valuable than other assets The important difference between software and other assets is that software is much more plastic than other engineering work. Software is “soft” – it can be changed easily and inexpensively and quickly. This makes software more strategically valuable than “hard” assets like a building because software can be continuously adapted and renewed in response to changing situations, and transformed to create new business opportunities. Software has to be changed to stay useful. The problem is NOT that we HAVE TO maintain software and change it to do things that it was never intended to do, to work in ways that it was never designed to, to do things that we couldn’t imagine a few years ago. This is the opportunity that software gives us – that we CAN do this. This is why Software is Eating the World. Source: http://swreflections.blogspot.com/2012/02/source-code-is-asset-not-liability.html

If you’re testing a web application, you can’t go far wrong with Selenium WebDriver. But in this web 2.0 world of ajax-y goodness, it can be a pain dealing with the asynchronous nature of modern sites. Back when all we had was web 1.0 you clicked a button and eventually you got a new page, or if you were unlucky: an error message. But now when you click links all sorts of funky things happen – some of which happen faster than others. From the user’s perspective this creates a great UI. But if you’re trying to automate testing this you can get all sorts of horrible race conditions. Thread.sleep The naive approach is to write your tests the same way you did before: you click buttons and assert that what you expected to happen actually happened. For the most part, this works. Sites are normally fast enough, even in a continuous integration environment, that by the time the test harness looks for a change it’s already happened. But then… things slow down a little and you start getting flickers - tests that sometimes pass and sometimes fail. So you add a little delay. Just 500 milliseconds should do it, while you wait for the server to respond and update the page. Then a month later it’s flickering again, so you make it 1 second. Then two… then twenty. The trouble is, each test runs at the pace that it runs at its slowest. If login normally takes 0.1 seconds, but sometimes takes 10 seconds when the environment’s overloaded – the test has to wait for 10 seconds so as not to flicker. This means even though the app often runs faster, the test has to wait just in case. Before you know it, your tests are crawling and take hours to run – you’ve lost your fast feedback loop and developers no longer trust the tests. An Example Thankfully WebDriver has a solution to this. It allows you to wait for some condition to pass, so you can use it to control the pace of your tests. To demonstrate this, I’ve created a simple web application with a login form – the source is available on github. The login takes a stupid amount of time, so the tests need to react to this so as not to introduce arbitrary waits. The application is very simple – a username and password field with an authenticate button that makes an ajax request to log the user in. If the login is successful, we update the screen to let the user know. The first thing is to write our test (obviously in the real world we’d have written the test before our production code, but its the test that’s interesting here not what we’re testing – so we’ll do it in the wrong order just this once): @Test public void authenticatesUser() { driver.get("http://localhost:8080/"); LoginPage loginPage = LoginPage.open(driver); loginPage.setUsername("admin"); loginPage.setPassword("password"); loginPage.clickAuthenticate(); Assert.assertEquals("Logged in as admin", loginPage.welcomeMessage()); } We have a page object that encapsulates the login functionality. We provide the username & password then click authenticate. Finally we check that the page has updated with the user message. But how have we dealt with the asynchronous nature of this application? WebDriverWait Through the magic of WebDriverWait we can wait for a function to return true before we continue: public void clickAuthenticate() { this.authenticateButton.click(); new WebDriverWait(driver, 30).until(accountPanelIsVisible()); } private Predicate accountPanelIsVisible() { return new Predicate() { @Override public boolean apply(WebDriver driver) { return isAccountPanelVisible(); } }; } private boolean isAccountPanelVisible() { return accountPanel.isDisplayed(); } Our clickAuthenticate method clicks the button then instructs WebDriver to wait for our condition to pass. The condition is defined via a predicate (c’mon Java where’s the closures?). The predicate is simply a method that will run to determine whether or not the condition is true yet. In this case, we delegate to the isAccountPanelVisible method on the page object. This does exactly what it says on the tin, it uses the page element to check whether it’s visible yet. Simple, no? In this way we can define a condition we want to be true before we continue. In this case, the exit condition of the clickAuthenticate method is that the asynchronous authentication process has completed. This means that tests don’t need to worry about the internal mechanics of the page – about whether the operation is asynchronous or not. The test merely specifies what to test, the page object encapsulates how to do it. Javascript It’s all well and good waiting for elements to be visible or certain text to be present, but sometimes we might want more subtle control. A good approach is to update Javascript state when an action has finished. This means that tests can inspect javascript variables to determine whether something has completed or not – allowing very clear and simple coordination between production code and test. Continuing with our login example, instead of relying on a becoming visible, we could instead have set a Javascript variable. The code in fact does both, so we can have two tests. The second looks as follows: public void authenticate() { this.authenticateButton.click(); new WebDriverWait(driver, 30).until(authenticated()); } private Predicate authenticated() { return new Predicate() { @Override public boolean apply(WebDriver driver) { return isAuthenticated(); } }; } private boolean isAuthenticated() { return (Boolean) executor().executeScript("return authenticated;"); } private JavascriptExecutor executor() { return (JavascriptExecutor) driver; } This example follows the same basic pattern as the test before, but we use a different predicate. Instead of checking whether an element is visible or not, we instead get the status of a Javascript variable. We can do this because each WebDriver also implements the JavascriptExecutor allowing us to run Javascript inside the browser within the context of the test. I.e. the script “return authenticated” runs within the browser, but the result is returned to our test. We simply inspect the state of a variable, which is false initially and set to true once the authentication process has finished. This allows us to closely coordinate our production and test code without the risk of flickering tests because of race conditions. From http://blog.activelylazy.co.uk/2012/01/29/testing-asynchronous-applications-with-webdriverwait/

Another mock recipe, this one for mocking generator methods. A Python generator is a function or method that uses the yield statement to return a series of values when iterated over (there are also generator expressions and more advanced uses of generators, but we aren't concerned about them here. A very good introduction to generators and how powerful they are is: Generator Tricks for Systems Programmers.). A generator method / function is called to return the generator object. It is the generator object that is then iterated over. The protocol method for iteration is __iter__, so we can mock this using a MagicMock. Here's an example class with an "iter" method implemented as a generator: >>> class Foo(object): ... def iter(self): ... for i in [1, 2, 3]: ... yield i ... >>> foo = Foo() >>> list(foo.iter()) [1, 2, 3] How would we mock this class, and in particular its "iter" method? To configure the values returned from the iteration (implicit in the call to list), we need to configure the iterator returned by the call to foo.iter(). >>> values = [1, 2, 3] >>> mock_foo = MagicMock() >>> iterable = mock_foo.iter.return_value >>> iterator = iter(values) >>> iterable.__iter__.return_value = iterator >>> list(mock_foo.iter()) [1, 2, 3] The above example is done step-by-step. The shorter version is: >>> mock_foo = MagicMock() >>> mock_foo.iter.return_value.__iter__.return_value = iter([1, 2, 3]) >>> list(mock_foo.iter()) [1, 2, 3] This is now also in the docs on the mock examples page. There's now quite a collection of useful mock recipes there, so even if you're an experienced mock user it is worth a browse. Source: http://www.voidspace.org.uk/python/weblog/arch_d7_2011_06_11.shtml

Previously, we covered some tips and tricks for the Grid control. In this article, we will go over a few tips and tricks for the wx.ListCtrl widget when it’s in “report” mode. Take a look at the tips below: How to create a simple ListCtrl How to sort the rows of a ListCtrl How to make the ListCtrl cells editable in place Associating objects with ListCtrl rows Alternate the row colors of a ListCtrl How to create a simple ListCtrl The list control is a pretty common widget. In Windows, you will see the list control in Windows Explorer. It has four modes: icon, small icon, list, and report. They roughly match up with icons, tiles, list, and details views in Windows Explorer respectively. We’re going to focus on the ListCtrl in Report mode because that’s the mode that most developers use it in. Here’s a simple example of how to create a list control: import wx ######################################################################## class MyForm(wx.Frame): #---------------------------------------------------------------------- def __init__(self): wx.Frame.__init__(self, None, wx.ID_ANY, "List Control Tutorial") # Add a panel so it looks the correct on all platforms panel = wx.Panel(self, wx.ID_ANY) self.index = 0 self.list_ctrl = wx.ListCtrl(panel, size=(-1,100), style=wx.LC_REPORT |wx.BORDER_SUNKEN ) self.list_ctrl.InsertColumn(0, 'Subject') self.list_ctrl.InsertColumn(1, 'Due') self.list_ctrl.InsertColumn(2, 'Location', width=125) btn = wx.Button(panel, label="Add Line") btn.Bind(wx.EVT_BUTTON, self.add_line) sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.list_ctrl, 0, wx.ALL|wx.EXPAND, 5) sizer.Add(btn, 0, wx.ALL|wx.CENTER, 5) panel.SetSizer(sizer) #---------------------------------------------------------------------- def add_line(self, event): line = "Line %s" % self.index self.list_ctrl.InsertStringItem(self.index, line) self.list_ctrl.SetStringItem(self.index, 1, "01/19/2010") self.list_ctrl.SetStringItem(self.index, 2, "USA") self.index += 1 #---------------------------------------------------------------------- # Run the program if __name__ == "__main__": app = wx.App(False) frame = MyForm() frame.Show() app.MainLoop() As you can probably tell from the code above, it’s really easy to create a ListCtrl instance. Notice that we set the style to report mode using the wx.LC_REPORT flag. To add column headers, we call the ListCtrl’s InsertColumn method and pass an integer to tell the ListCtrl which column is which and a string for the user’s convenience. Yes, the columns are zero-based, so the first column is number zero, the second column is number one, etc. The next important piece is contained in the button’s event handler, add_line, where we learn how to add rows of data to the ListCtrl. The typical method to use is the InsertStringItem method. If you wanted an image added to each row as well, then you’d use a more complicated method like InsertColumnInfo along with the InsertImageStringItem method. You can see how to use them in the wxPython demo. We’re sticking with the easy stuff in this article. Anyway, when you call InsertStringItem you give it the correct row index and a string. You use the SetStringItem method to set the data for the other columns of the row. Notice that the SetStringItem method requires three parameters: the row index, the column index and a string. Lastly, we increment the row index so we don’t overwrite anything. Now you can get out there and make your own! Let’s continue and find out how to sort rows! How to sort the rows of a ListCtrl The ListCtrl widget has had some extra scripts written for it that add functionality to the widget. These scripts are called mixins. You can read about them here. For this recipe, we’ll be using the ColumnSorterMixin mixin. The code below is a stripped down version of one of the wxPython demo examples. import wx import wx.lib.mixins.listctrl as listmix musicdata = { 0 : ("Bad English", "The Price Of Love", "Rock"), 1 : ("DNA featuring Suzanne Vega", "Tom's Diner", "Rock"), 2 : ("George Michael", "Praying For Time", "Rock"), 3 : ("Gloria Estefan", "Here We Are", "Rock"), 4 : ("Linda Ronstadt", "Don't Know Much", "Rock"), 5 : ("Michael Bolton", "How Am I Supposed To Live Without You", "Blues"), 6 : ("Paul Young", "Oh Girl", "Rock"), } ######################################################################## class TestListCtrl(wx.ListCtrl): #---------------------------------------------------------------------- def __init__(self, parent, ID=wx.ID_ANY, pos=wx.DefaultPosition, size=wx.DefaultSize, style=0): wx.ListCtrl.__init__(self, parent, ID, pos, size, style) ######################################################################## class TestListCtrlPanel(wx.Panel, listmix.ColumnSorterMixin): #---------------------------------------------------------------------- def __init__(self, parent): wx.Panel.__init__(self, parent, -1, style=wx.WANTS_CHARS) self.index = 0 self.list_ctrl = TestListCtrl(self, size=(-1,100), style=wx.LC_REPORT |wx.BORDER_SUNKEN |wx.LC_SORT_ASCENDING ) self.list_ctrl.InsertColumn(0, "Artist") self.list_ctrl.InsertColumn(1, "Title", wx.LIST_FORMAT_RIGHT) self.list_ctrl.InsertColumn(2, "Genre") items = musicdata.items() index = 0 for key, data in items: self.list_ctrl.InsertStringItem(index, data[0]) self.list_ctrl.SetStringItem(index, 1, data[1]) self.list_ctrl.SetStringItem(index, 2, data[2]) self.list_ctrl.SetItemData(index, key) index += 1 # Now that the list exists we can init the other base class, # see wx/lib/mixins/listctrl.py self.itemDataMap = musicdata listmix.ColumnSorterMixin.__init__(self, 3) self.Bind(wx.EVT_LIST_COL_CLICK, self.OnColClick, self.list_ctrl) sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.list_ctrl, 0, wx.ALL|wx.EXPAND, 5) self.SetSizer(sizer) #---------------------------------------------------------------------- # Used by the ColumnSorterMixin, see wx/lib/mixins/listctrl.py def GetListCtrl(self): return self.list_ctrl #---------------------------------------------------------------------- def OnColClick(self, event): print "column clicked" event.Skip() ######################################################################## class MyForm(wx.Frame): #---------------------------------------------------------------------- def __init__(self): wx.Frame.__init__(self, None, wx.ID_ANY, "List Control Tutorial") # Add a panel so it looks the correct on all platforms panel = TestListCtrlPanel(self) #---------------------------------------------------------------------- # Run the program if __name__ == "__main__": app = wx.App(False) frame = MyForm() frame.Show() app.MainLoop() This code is a little on the odd side in that we have inherit the mixin in the wx.Panel based class rather than the wx.ListCtrl class. You can do it either way though as long as you rearrange the code correctly. Anyway, we are going to home in on the key differences between this example and the previous one. The first difference of major importance is in the looping construct where we insert the list control’s data. Here we include the list control’s SetItemData method to include the necessary inner-workings that allow the sorting to take place. As you might have guessed, this method associates the row index with the music data dict’s key. Next we instantiate the ColumnSorterMixin and tell it how many columns there are in the list control. We could have left the EVT_LIST_COL_CLICK binding off this example as it has nothing to do with the actual sorting of the rows, but in the interest of increasing your knowledge, it was left in. All it does is show you how to catch the user’s column click event. The rest of the code is self-explanatory. If you want to know about the requirements for this mixin, especially when you have images in your rows, please see the relevant section in the source (i.e. listctrl.py). Now, wasn’t that easy? Let’s continue our journey and find out how to make the cells editable! How to make the ListCtrl cells editable in place Sometimes, the programmer will want to allow the user to click on a cell and edit it in place. This is kind of a lightweight version of the wx.grid.Grid control. Here’s an example: import wx import wx.lib.mixins.listctrl as listmix ######################################################################## class EditableListCtrl(wx.ListCtrl, listmix.TextEditMixin): ''' TextEditMixin allows any column to be edited. ''' #---------------------------------------------------------------------- def __init__(self, parent, ID=wx.ID_ANY, pos=wx.DefaultPosition, size=wx.DefaultSize, style=0): """Constructor""" wx.ListCtrl.__init__(self, parent, ID, pos, size, style) listmix.TextEditMixin.__init__(self) ######################################################################## class MyPanel(wx.Panel): """""" #---------------------------------------------------------------------- def __init__(self, parent): """Constructor""" wx.Panel.__init__(self, parent) rows = [("Ford", "Taurus", "1996", "Blue"), ("Nissan", "370Z", "2010", "Green"), ("Porche", "911", "2009", "Red") ] self.list_ctrl = EditableListCtrl(self, style=wx.LC_REPORT) self.list_ctrl.InsertColumn(0, "Make") self.list_ctrl.InsertColumn(1, "Model") self.list_ctrl.InsertColumn(2, "Year") self.list_ctrl.InsertColumn(3, "Color") index = 0 for row in rows: self.list_ctrl.InsertStringItem(index, row[0]) self.list_ctrl.SetStringItem(index, 1, row[1]) self.list_ctrl.SetStringItem(index, 2, row[2]) self.list_ctrl.SetStringItem(index, 3, row[3]) index += 1 sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.list_ctrl, 0, wx.ALL|wx.EXPAND, 5) self.SetSizer(sizer) ######################################################################## class MyFrame(wx.Frame): """""" #---------------------------------------------------------------------- def __init__(self): """Constructor""" wx.Frame.__init__(self, None, wx.ID_ANY, "Editable List Control") panel = MyPanel(self) self.Show() #---------------------------------------------------------------------- if __name__ == "__main__": app = wx.App(False) frame = MyFrame() app.MainLoop() In this script, we put the TextEditMixin in our wx.ListCtrl class instead of our wx.Panel, which is the opposite of the previous example. The mixin itself does all the heavy lifting. Again, you’ll have to check out the mixin’s source to really understand how it works. Associating objects with ListCtrl rows This subject comes up a lot: How do I associate data (i.e. objects) with my ListCtrl’s rows? Well, we’re going to find out exactly how to do that with the following code: import wx ######################################################################## class Car(object): """""" #---------------------------------------------------------------------- def __init__(self, make, model, year, color="Blue"): """Constructor""" self.make = make self.model = model self.year = year self.color = color ######################################################################## class MyPanel(wx.Panel): """""" #---------------------------------------------------------------------- def __init__(self, parent): """Constructor""" wx.Panel.__init__(self, parent) rows = [Car("Ford", "Taurus", "1996"), Car("Nissan", "370Z", "2010"), Car("Porche", "911", "2009", "Red") ] self.list_ctrl = wx.ListCtrl(self, size=(-1,100), style=wx.LC_REPORT |wx.BORDER_SUNKEN ) self.list_ctrl.Bind(wx.EVT_LIST_ITEM_SELECTED, self.onItemSelected) self.list_ctrl.InsertColumn(0, "Make") self.list_ctrl.InsertColumn(1, "Model") self.list_ctrl.InsertColumn(2, "Year") self.list_ctrl.InsertColumn(3, "Color") index = 0 self.myRowDict = {} for row in rows: self.list_ctrl.InsertStringItem(index, row.make) self.list_ctrl.SetStringItem(index, 1, row.model) self.list_ctrl.SetStringItem(index, 2, row.year) self.list_ctrl.SetStringItem(index, 3, row.color) self.myRowDict[index] = row index += 1 sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.list_ctrl, 0, wx.ALL|wx.EXPAND, 5) self.SetSizer(sizer) #---------------------------------------------------------------------- def onItemSelected(self, event): """""" currentItem = event.m_itemIndex car = self.myRowDict[currentItem] print car.make print car.model print car.color print car.year ######################################################################## class MyFrame(wx.Frame): """""" #---------------------------------------------------------------------- def __init__(self): """Constructor""" wx.Frame.__init__(self, None, wx.ID_ANY, "List Control Tutorial") panel = MyPanel(self) self.Show() #---------------------------------------------------------------------- if __name__ == "__main__": app = wx.App(False) frame = MyFrame() app.MainLoop() The list control widget actually doesn’t have a built-in way to accomplish this feat. If you want that, then you’ll want to check out the ObjectListView widget, which wraps the ListCtrl and gives it a lot more functionality. In the meantime, we’ll take a minute and go over the code above. The first piece is just a plain Car class with four attributes. Then in the MyPanel class, we create a list of Car objects that we’ll use for the ListCtrl’s data. To add the data to the ListCtrl, we use a for loop to iterate over the list. We also associate each row with a Car object using a Python dictionary. We use the row’s index for the key and the dict’s value ends up being the Car object. This allows us to access all the Car/row object’s data later on in the onItemSelected method. Let’s check that out! In onItemSelected, we grab the row’s index with the following little trick: event.m_itemIndex. Then we use that value as the key for our dictionary so that we can gain access to the Car object associated with that row. At this point, we just print out all the Car object’s attributes, but you could do whatever you want here. This basic idea could easily be extended to use a result set from a SqlAlchemy query for the ListCtrl’s data. Hopefully you get the general idea. Now if you were paying close attention, like Robin Dunn (creator of wxPython) was, then you might notice some really silly logic errors in this code. Did you find them? Well, you won’t see it unless you sort the rows, delete a row or insert a row. Do you see it now? Yes, I stupidly based the “unique” key in my dictionary on the row’s position, which will change if any of those events happen. So let’s look at a better example: import wx ######################################################################## class Car(object): """""" #---------------------------------------------------------------------- def __init__(self, make, model, year, color="Blue"): """Constructor""" self.id = id(self) self.make = make self.model = model self.year = year self.color = color ######################################################################## class MyPanel(wx.Panel): """""" #---------------------------------------------------------------------- def __init__(self, parent): """Constructor""" wx.Panel.__init__(self, parent) rows = [Car("Ford", "Taurus", "1996"), Car("Nissan", "370Z", "2010"), Car("Porche", "911", "2009", "Red") ] self.list_ctrl = wx.ListCtrl(self, size=(-1,100), style=wx.LC_REPORT |wx.BORDER_SUNKEN ) self.list_ctrl.Bind(wx.EVT_LIST_ITEM_SELECTED, self.onItemSelected) self.list_ctrl.InsertColumn(0, "Make") self.list_ctrl.InsertColumn(1, "Model") self.list_ctrl.InsertColumn(2, "Year") self.list_ctrl.InsertColumn(3, "Color") index = 0 self.myRowDict = {} for row in rows: self.list_ctrl.InsertStringItem(index, row.make) self.list_ctrl.SetStringItem(index, 1, row.model) self.list_ctrl.SetStringItem(index, 2, row.year) self.list_ctrl.SetStringItem(index, 3, row.color) self.list_ctrl.SetItemData(index, row.id) self.myRowDict[row.id] = row index += 1 sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.list_ctrl, 0, wx.ALL|wx.EXPAND, 5) self.SetSizer(sizer) #---------------------------------------------------------------------- def onItemSelected(self, event): """""" currentItem = event.m_itemIndex car = self.myRowDict[self.list_ctrl.GetItemData(currentItem)] print car.make print car.model print car.color print car.year ######################################################################## class MyFrame(wx.Frame): """""" #---------------------------------------------------------------------- def __init__(self): """Constructor""" wx.Frame.__init__(self, None, wx.ID_ANY, "List Control Tutorial") panel = MyPanel(self) self.Show() #---------------------------------------------------------------------- if __name__ == "__main__": app = wx.App(False) frame = MyFrame() app.MainLoop() In this example, we add a new attribute to our Car class that creates a unique id for each instance that is created using Python’s handy id builtin. Then in the loop where we add the data to the list control, we call the widget’s SetItemData method and give it the row index and the car instance’s unique id. Now it doesn’t matter where the row ends up because it’s had the unique id affixed to it. Finally, we have to modify the onItemSelected to get the right object. The magic happens in this code: # this code was helpfully provided by Robin Dunn car = self.myRowDict[self.list_ctrl.GetItemData(currentItem)] Cool, huh? Our last example will cover how to alternate the row colors, so let’s take a look! Alternate the row colors of a ListCtrl As this section’s title suggests, we will look at how to alternate colors of the rows of a ListCtrl. Here’s the code: import wx import wx.lib.mixins.listctrl as listmix ######################################################################## class MyPanel(wx.Panel): """""" #---------------------------------------------------------------------- def __init__(self, parent): """Constructor""" wx.Panel.__init__(self, parent) rows = [("Ford", "Taurus", "1996", "Blue"), ("Nissan", "370Z", "2010", "Green"), ("Porche", "911", "2009", "Red") ] self.list_ctrl = wx.ListCtrl(self, style=wx.LC_REPORT) self.list_ctrl.InsertColumn(0, "Make") self.list_ctrl.InsertColumn(1, "Model") self.list_ctrl.InsertColumn(2, "Year") self.list_ctrl.InsertColumn(3, "Color") index = 0 for row in rows: self.list_ctrl.InsertStringItem(index, row[0]) self.list_ctrl.SetStringItem(index, 1, row[1]) self.list_ctrl.SetStringItem(index, 2, row[2]) self.list_ctrl.SetStringItem(index, 3, row[3]) if index % 2: self.list_ctrl.SetItemBackgroundColour(index, "white") else: self.list_ctrl.SetItemBackgroundColour(index, "yellow") index += 1 sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.list_ctrl, 0, wx.ALL|wx.EXPAND, 5) self.SetSizer(sizer) ######################################################################## class MyFrame(wx.Frame): """""" #---------------------------------------------------------------------- def __init__(self): """Constructor""" wx.Frame.__init__(self, None, wx.ID_ANY, "List Control w/ Alternate Colors") panel = MyPanel(self) self.Show() #---------------------------------------------------------------------- if __name__ == "__main__": app = wx.App(False) frame = MyFrame() app.MainLoop() The code above will alternate each row’s background color. Thus you should see yellow and white rows. We do this by calling the ListCtrl instance’s SetItemBackgroundColour method. If you were using a virtual list control, then you’d want to override the OnGetItemAttr method. To see an example of the latter method, open up your copy of the wxPython demo; there’s one in there. Wrapping Up We’ve covered a lot of ground here. You should now be able to do a lot more with your wx.ListCtrl than when you started, assuming you’re new to using it, of course. Feel free to ask questions in the comments or suggest future recipes. I hope you found this helpful! Note: All examples were tested on Windows XP with Python 2.5 and wxPython 2.8.10.1. They were also tested on Windows 7 Professional with Python 2.6 Additional Reading The official wxPython wx.ListCtrl documentation The ListControls wiki page ListCtrl Tooltips wiki page The ObjectListView website The UltimateListCtrl, a pure Python implementation now included with wxPython Source Code listctrl.zip listctrl.tar Source: http://www.blog.pythonlibrary.org/2011/01/04/wxpython-wx-listctrl-tips-and-tricks/

The simplest in-memory cache implementation should support Addition of objects into cache either via key-value, or via object creation mechanism Deletion of objects from cache based on key, or object type Querying cache store to check existence of an object There are several ways to achieve this using multiple design patterns. But if we were to implement those design patterns in our applications, we would end up designing a framework similar to Enterprise Library Caching block. So to keep things fairly simple – we need a simple implementation of caching objects in-memory and this cache to be thread-safe for multi-threading applications. So for that, you can just copy this piece of code into your application and you should be all set with an in-memory cache. public static class CacheStore { /// /// In-memory cache dictionary /// private static Dictionary _cache; private static object _sync; /// /// Cache initializer /// static CacheStore() { _cache = new Dictionary(); _sync = new object(); } /// /// Check if an object exists in cache /// /// Type of object /// Name of key in cache /// True, if yes; False, otherwise public static bool Exists(string key) where T : class { Type type = typeof(T); lock (_sync) { return _cache.ContainsKey(type.Name + key); } } /// /// Check if an object exists in cache /// /// Type of object /// True, if yes; False, otherwise public static bool Exists() where T : class { Type type = typeof(T); lock (_sync) { return _cache.ContainsKey(type.Name); } } /// /// Get an object from cache /// /// Type of object /// Object from cache public static T Get() where T : class { Type type = typeof(T); lock (_sync) { if (_cache.ContainsKey(type.Name) == false) throw new ApplicationException("An object of the desired type does not exist: " + type.Name); lock (_sync) { return (T)_cache[type.Name]; } } } /// /// Get an object from cache /// /// Type of object /// Name of key in cache /// Object from cache public static T Get(string key) where T : class { Type type = typeof(T); lock (_sync) { if (_cache.ContainsKey(key + type.Name) == false) throw new ApplicationException(String.Format("An object with key '{0}' does not exists", key)); lock (_sync) { return (T)_cache[key + type.Name]; } } } /// /// Create default instance of the object and add it in cache /// /// Class whose object is to be created /// Object of the class public static T Create(string key, params object[] constructorParameters) where T : class { Type type = typeof(T); T value = (T)Activator.CreateInstance(type, constructorParameters); lock (_sync) { if (_cache.ContainsKey(key + type.Name)) throw new ApplicationException(String.Format("An object with key '{0}' already exists", key)); lock (_sync) { _cache.Add(key + type.Name, value); } } return value; } /// /// Create default instance of the object and add it in cache /// /// Class whose object is to be created /// Object of the class public static T Create(params object[] constructorParameters) where T : class { Type type = typeof(T); T value = (T)Activator.CreateInstance(type, constructorParameters); lock (_sync) { if (_cache.ContainsKey(type.Name)) throw new ApplicationException(String.Format("An object of type '{0}' already exists", type.Name)); lock (_sync) { _cache.Add(type.Name, value); } } return value; } public static void Add(string key, T value) { Type type = typeof(T); if (value.GetType() != type) throw new ApplicationException(String.Format("The type of value passed to cache {0} does not match the cache type {1} for key {2}", value.GetType().FullName, type.FullName, key)); lock (_sync) { if (_cache.ContainsKey(key + type.Name)) throw new ApplicationException(String.Format("An object with key '{0}' already exists", key)); lock (_sync) { _cache.Add(key + type.Name, value); } } } /// /// Remove an object type from cache /// /// Type of object public void Remove() { Type type = typeof(T); lock (_sync) { if (_cache.ContainsKey(type.Name) == false) throw new ApplicationException(String.Format("An object of type '{0}' does not exists in cache", type.Name)); lock (_sync) { _cache.Remove(type.Name); } } } /// /// Remove an object stored with a key from cache /// /// Type of object /// Key of the object public void Remove(string key) { Type type = typeof(T); lock (_sync) { if (_cache.ContainsKey(key + type.Name) == false) throw new ApplicationException(String.Format("An object with key '{0}' does not exists in cache", key)); lock (_sync) { _cache.Remove(key + type.Name); } } } } Every method has 2 overloads With Key as a parameter: This method adds a new key-value in the cache store for a particular object type. This also means that for a particular object (say Employee), you can have multiple cached-objects (say, multiple employees in an organization) Without Key as a parameter – This method adds a new key (type of the object) and value in the cache store. This means, for a particular object type (say ConfigurationSettings) there will single object in the cache (say, configuration value) Implementation example using CacheStore is: MonoAssemblyResolver targetAssembly = null; if (CacheStore.Exists(projMapping.TargetAssemblyPath)) { targetAssembly = CacheStore.Get(projMapping.TargetAssemblyPath); } else { targetAssembly = new MonoAssemblyResolver(projMapping.TargetAssemblyPath); CacheStore.Add(projMapping.TargetAssemblyPath, targetAssembly); } Since this uses plain-C# and is light weight, this can be used in ASP.NET MVC, Silverlight, WPF, or Windows Phone applications. So happy coding! Source: http://www.ganshani.com/2012/01/31/in-memory-cache-implementation-in-c

Emgu CV is a cross platform .Net wrapper to the Intel OpenCV image processing library. Allowing OpenCV functions to be called from .NET compatible languages such as C#, VB, VC++, IronPython etc. The wrapper can be compiled in Mono and run on Linux / Mac OS X. Unlike other wrappers such as OpenCVDotNet, SharperCV which use unsafe code, Emgu CV is written entirely in C#. The benefit is that it can be compiled in Mono and therefore is able to run on any platform Mono supports, including Linux, Solaris and Mac OS X. A lot of efforts has been spend to have a pure C# implementation since the headers have to be ported, compared with managed C++ implementation where header files can simply be included. But it is well worth it if you see Emgu CV running on Fedora 10! Plus it always gives you the comfort knowing that your code is cross-platform. Face Recognition 1- Create a Windows Form Application 2- Add a PictureBox and a Timer (and Enable it) 3- Run it on a x86 system 4- Be sure you have the OpenCV relevant dlls (included with the Emgu CV download) in the folder where you code executes. 5- Adjust the path to find the Haarcascade xml (last line of the code) using System; using System.Windows.Forms; using System.Drawing; using Emgu.CV; using Emgu.Util; using Emgu.CV.Structure; using Emgu.CV.CvEnum; namespace opencvtut { public partial class Form1 : Form { private Capture cap; private HaarCascade haar; public Form1() { InitializeComponent(); } private void timer1_Tick(object sender, EventArgs e) { using (Image nextFrame = cap.QueryFrame()) { if (nextFrame != null) { // there’s only one channel (greyscale), hence the zero index //var faces = nextFrame.DetectHaarCascade(haar)[0]; Image grayframe = nextFrame.Convert(); var faces = grayframe.DetectHaarCascade( haar, 1.4, 4, HAAR_DETECTION_TYPE.DO_CANNY_PRUNING, new Size(nextFrame.Width/8, nextFrame.Height/8) )[0]; foreach (var face in faces) { nextFrame.Draw(face.rect, new Bgr(0,double.MaxValue,0), 3); } pictureBox1.Image = nextFrame.ToBitmap(); } } } private void Form1_Load(object sender, EventArgs e) { // passing 0 gets zeroth webcam cap = new Capture(0); // adjust path to find your xml haar = new HaarCascade( “..\\..\\..\\..\\lib\\haarcascade_frontalface_alt2.xml”); } } } Source: http://blog.csharplearners.com/2012/01/30/face-recognation-c/

In previous posts I have covered how to map inheritance relationships in JAXB. This can be done by element name (via @XmlElementRef), by the xsi:type attribute, or in EclipseLink MOXy using another XML attribute (via @XmlDescriminatorNode/@XmlDescriminatorValue). In this post the type indicator will be an XML attribute/element unique to that type, and we will leverage an XmlAdapter to implement this behaviour. Input (input.xml) In this example the possible contact methods are Address and PhoneNumber. If the street attribute is present on the contact-method element we will instantiate an Address object, and if the number attribute is present we will instantiate a PhoneNumber object. Java Model Below is the domain model that will be used for this example. Customer package blog.inheritance.xmladapter; import java.util.List; import javax.xml.bind.annotation.*; @XmlRootElement @XmlAccessorType(XmlAccessType.FIELD) public class Customer { @XmlElement(name="contact-method") private List contactMethods; } ContactMethod ContactMethod and its subclasses (Address & PhoneNumber) will be handled by an XmlAdapter, so the only mapping required is @XmlJavaTypeAdapter to specify the implementation of XmlAdapter. package blog.inheritance.xmladapter; import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter; @XmlJavaTypeAdapter(ContactMethodAdapter.class) public abstract class ContactMethod { } Address package blog.inheritance.xmladapter; public class Address extends ContactMethod { protected String street; protected String city; } PhoneNumber package blog.inheritance.xmladapter; public class PhoneNumber extends ContactMethod { protected String number; } XmlAdapter (ContactMethodAdapter) The AdaptedContactMethod class has been created and represents the combined properties of ContactMethod, Address, and PhoneNumber. In a marshal operation only the properties corresponding to the type being marshalled are populated. During the unmarshal operation after the AdaptedContactMethod has been built, we can look at which properties have been populated to determine the appropriate subtype that should be returned. package blog.inheritance.xmladapter; import javax.xml.bind.annotation.XmlAttribute; import javax.xml.bind.annotation.adapters.XmlAdapter; public class ContactMethodAdapter extends XmlAdapter { @Override public AdaptedContactMethod marshal(ContactMethod contactMethod) throws Exception { if (null == contactMethod) { return null; } AdaptedContactMethod adaptedContactMethod = new AdaptedContactMethod(); if (contactMethod instanceof Address) { Address address = (Address) contactMethod; adaptedContactMethod.street = address.street; adaptedContactMethod.city = address.city; } else { PhoneNumber phoneNumber = (PhoneNumber) contactMethod; adaptedContactMethod.number = phoneNumber.number; } return adaptedContactMethod; } @Override public ContactMethod unmarshal(AdaptedContactMethod adaptedContactMethod) throws Exception { if (null == adaptedContactMethod) { return null; } if (null != adaptedContactMethod.number) { PhoneNumber phoneNumber = new PhoneNumber(); phoneNumber.number = adaptedContactMethod.number; return phoneNumber; } else { Address address = new Address(); address.street = adaptedContactMethod.street; address.city = adaptedContactMethod.city; return address; } } public static class AdaptedContactMethod { @XmlAttribute public String number; @XmlAttribute public String street; @XmlAttribute public String city; } } Demo Code The following demo code will be used for this example. We will unmarshal the input document, output the type of each object in the collection, and then marshal the objects back to XML. package blog.inheritance.xmladapter; import java.io.File; import javax.xml.bind.*; public class Demo { public static void main(String[] args) throws Exception { JAXBContext jc = JAXBContext.newInstance(Customer.class); Unmarshaller unmarshaller = jc.createUnmarshaller(); File xml = new File("src/blog/inheritance/xmladapter/input.xml"); Customer customer = (Customer) unmarshaller.unmarshal(xml); for(ContactMethod contactMethod : customer.getContactMethods()) { System.out.println(contactMethod.getClass()); } Marshaller marshaller = jc.createMarshaller(); marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true); marshaller.marshal(customer, System.out); } } Output The following is the output from running the demo code. Note how each of the instances of ContactMethod in the collection are of the appropriate sub-type. class blog.inheritance.xmladapter.PhoneNumber class blog.inheritance.xmladapter.Address class blog.inheritance.xmladapter.PhoneNumber Related Forum Questions Below are a couple of use cases that appeared on Stack Overflow that can be implemented using this approach: eclipselink/Moxy : inheritance and attribute name overloading based on type Jaxb objects with the same name Java/JAXB: Unmarshal XML attributes to specific Java object attributes From http://blog.bdoughan.com/2012/01/jaxb-and-inhertiance-using-xmladapter.html

We all know the Java platform allows us to create reusable objects in memory. However, all of those objects exist only as long as the Java virtual machine remains running. It would be nice if the objects we create could exist beyond the lifetime of the virtual machine. Well, with object serialization, you can flatten your objects and reuse them in powerful ways. Object serialization is the process of saving an object’s state to a sequence of bytes, as well as the process of rebuilding those bytes into a live object at some future time. The Java Serialization API provides a standard mechanism for developers to handle object serialization. The API is small and easy to use, provided the classes and methods are understood. By implementating java.io.Serializable, you get “automatic” serialization capability for objects of your class. No need to implement any other logic, it’ll just work. The Java runtime will use reflection to figure out how to marshal and unmarshal your objects. In earlier version of Java, reflection was very slow, and so serializaing large object graphs (e.g. in client-server RMI applications) was a bit of a performance problem. To handle this situation, the java.io.Externalizable interface was provided, which is like java.io.Serializable but with custom-written mechanisms to perform the marshalling and unmarshalling functions (you need to implement readExternal and writeExternal methods on your class). This gives you the means to get around the reflection performance bottleneck. In recent versions of Java (1.3 onwards, certainly) the performance of reflection is vastly better than it used to be, and so this is much less of a problem. I suspect you’d be hard-pressed to get a meaningful benefit from Externalizable with a modern JVM. Also, the built-in Java serialization mechanism isn’t the only one, you can get third-party replacements, such as JBoss Serialization, which is considerably quicker, and is a drop-in replacement for the default. A big downside of Externalizable is that you have to maintain this logic yourself – if you add, remove or change a field in your class, you have to change your writeExternal/readExternal methods to account for it. In summary, Externalizable is a relic of the Java 1.1 days. There’s really no need for it any more. References http://java.sun.com/developer/technicalArticles/Programming/serialization http://docs.oracle.com/javase/6/docs/api/java/io/Serializable.html http://docs.oracle.com/javase/6/docs/api/java/io/Externalizable.html From http://singztechmusings.in/marshalling-unmarshalling-java-objects-serialization-vs-externalization/

In previous posts I compared JAXB (JSR-222) to Simple and XStream when starting from Java objects. In this post I'll compare JAXB to XMLBeans when starting from an XML schema. I will use XMLBeans 2.5.0 (December 2009) which is the latest release. XML Schema (customer.xsd) Below is the XML schema that will be used to generate our domain models. Generating the Classes XMLBeans Below is the call to generate the XMLBeans classes from our XML schema. For the purposes of this example we will use the -srconly flag to generate only the source. 1 scomp -d out -srconly customer.xsd Below are all the artifacts that are generated by XMLBeans. The XML Schema Binary (XSB) files contain metadata need to perform binding and validation: com/example Address.java CustomerDocument.java PhoneNumber.java com/example/impl AddressImpl.java CustomerDocumentImpl.java PhoneNumberImpl.java schemaorg_apache_xmlbeans element/http_3A_2F_2Fwww_2Eexample_2Ecom customer.xsb javaname/com/example Address.xsb CustomerDocument.xsb PhoneNumber.xsb CustomerDocument Customer.xsb namespace/http_3A_2F_2Fwww_2Eexample_2Ecom xmlns.xsb src customer.xsd system/s16C99350D7D3A2544A7BFD5E35CA8BC8 address6f49type.xsb customer3fdddoctype.xsb customer11d7elemtype.xsb customerelement.xsb index.xsb phonenumber9c83type.xsb TypeSystemHolder.class type/http_3A_2F_2Fwww_2Eexample_2Ecom address.xsb phone_2Dnumber.xsb JAXB Below is the call to generate the JAXB classes from an XML schema: 1 xjc -d out customer.xsd Below are all the artifacts that are generated by JAXB. Note how many fewer artifacts are created: com.example Address.java Customer.java ObjectFactory.java package-info.java PhoneNumber.java Java Model - XMLBeans XMLBeans produces a set of Java interfaces that are backed by implementation classes. Below we will examine one of these pairs. Address This is one of the interfaces that is generated by XMLBeans. There are a few interesting things worth noting: This interface exposes POJO properties (line 24), and a DOM like model (line 29). This interface includes a factory (line 66). This factory is used for creating instances of the Address object (line 68), and for unmarshalling instances of Address from XML (line 75). package com.example; /** * An XML address(@http://www.example.com). * * This is a complex type. */ public interface Address extends org.apache.xmlbeans.XmlObject { public static final org.apache.xmlbeans.SchemaType type = (org.apache.xmlbeans.SchemaType) org.apache.xmlbeans.XmlBeans.typeSystemForClassLoader(Address.class.getClassLoader(), "schemaorg_apache_xmlbeans.system.s16C99350D7D3A2544A7BFD5E35CA8BC8").resolveHandle("address6f49type"); /** * Gets the "street" element */ java.lang.String getStreet(); /** * Gets (as xml) the "street" element */ org.apache.xmlbeans.XmlString xgetStreet(); /** * Sets the "street" element */ void setStreet(java.lang.String street); /** * Sets (as xml) the "street" element */ void xsetStreet(org.apache.xmlbeans.XmlString street); /** * Gets the "city" element */ java.lang.String getCity(); /** * Gets (as xml) the "city" element */ org.apache.xmlbeans.XmlString xgetCity(); /** * Sets the "city" element */ void setCity(java.lang.String city); /** * Sets (as xml) the "city" element */ void xsetCity(org.apache.xmlbeans.XmlString city); /** * A factory class with static methods for creating instances * of this type. */ public static final class Factory { public static com.example.Address newInstance() { return (com.example.Address) org.apache.xmlbeans.XmlBeans.getContextTypeLoader().newInstance( type, null ); } public static com.example.Address newInstance(org.apache.xmlbeans.XmlOptions options) { return (com.example.Address) org.apache.xmlbeans.XmlBeans.getContextTypeLoader().newInstance( type, options ); } /** @param xmlAsString the string value to parse */ public static com.example.Address parse(java.lang.String xmlAsString) throws org.apache.xmlbeans.XmlException { return (com.example.Address) org.apache.xmlbeans.XmlBeans.getContextTypeLoader().parse( xmlAsString, type, null ); } public static com.example.Address parse(java.lang.String xmlAsString, org.apache.xmlbeans.XmlOptions options) throws org.apache.xmlbeans.XmlException { return (com.example.Address) org.apache.xmlbeans.XmlBeans.getContextTypeLoader().parse( xmlAsString, type, options ); } /** @param file the file from which to load an xml document */ public static com.example.Address parse(java.io.File file) throws org.apache.xmlbeans.XmlException, java.io.IOException { return (com.example.Address) org.apache.xmlbeans.XmlBeans.getContextTypeLoader().parse( file, type, null ); } public static com.example.Address parse(java.io.File file, org.apache.xmlbeans.XmlOptions options) throws org.apache.xmlbeans.XmlException, java.io.IOException { return (com.example.Address) org.apache.xmlbeans.XmlBeans.getContextTypeLoader().parse( file, type, options ); } public static com.example.Address parse(java.net.URL u) throws org.apache.xmlbeans.XmlException, java.io.IOException { return (com.example.Address) org.apache.xmlbeans.XmlBeans.getContextTypeLoader().parse( u, type, null ); } public static com.example.Address parse(java.net.URL u, org.apache.xmlbeans.XmlOptions options) throws org.apache.xmlbeans.XmlException, java.io.IOException { return (com.example.Address) org.apache.xmlbeans.XmlBeans.getContextTypeLoader().parse( u, type, options ); } public static com.example.Address parse(java.io.InputStream is) throws org.apache.xmlbeans.XmlException, java.io.IOException { return (com.example.Address) org.apache.xmlbeans.XmlBeans.getContextTypeLoader().parse( is, type, null ); } public static com.example.Address parse(java.io.InputStream is, org.apache.xmlbeans.XmlOptions options) throws org.apache.xmlbeans.XmlException, java.io.IOException { return (com.example.Address) org.apache.xmlbeans.XmlBeans.getContextTypeLoader().parse( is, type, options ); } public static com.example.Address parse(java.io.Reader r) throws org.apache.xmlbeans.XmlException, java.io.IOException { return (com.example.Address) org.apache.xmlbeans.XmlBeans.getContextTypeLoader().parse( r, type, null ); } public static com.example.Address parse(java.io.Reader r, org.apache.xmlbeans.XmlOptions options) throws org.apache.xmlbeans.XmlException, java.io.IOException { return (com.example.Address) org.apache.xmlbeans.XmlBeans.getContextTypeLoader().parse( r, type, options ); } public static com.example.Address parse(javax.xml.stream.XMLStreamReader sr) throws org.apache.xmlbeans.XmlException { return (com.example.Address) org.apache.xmlbeans.XmlBeans.getContextTypeLoader().parse( sr, type, null ); } public static com.example.Address parse(javax.xml.stream.XMLStreamReader sr, org.apache.xmlbeans.XmlOptions options) throws org.apache.xmlbeans.XmlException { return (com.example.Address) org.apache.xmlbeans.XmlBeans.getContextTypeLoader().parse( sr, type, options ); } public static com.example.Address parse(org.w3c.dom.Node node) throws org.apache.xmlbeans.XmlException { return (com.example.Address) org.apache.xmlbeans.XmlBeans.getContextTypeLoader().parse( node, type, null ); } public static com.example.Address parse(org.w3c.dom.Node node, org.apache.xmlbeans.XmlOptions options) throws org.apache.xmlbeans.XmlException { return (com.example.Address) org.apache.xmlbeans.XmlBeans.getContextTypeLoader().parse( node, type, options ); } /** @deprecated {@link org.apache.xmlbeans.xml.stream.XMLInputStream} */ public static com.example.Address parse(org.apache.xmlbeans.xml.stream.XMLInputStream xis) throws org.apache.xmlbeans.XmlException, org.apache.xmlbeans.xml.stream.XMLStreamException { return (com.example.Address) org.apache.xmlbeans.XmlBeans.getContextTypeLoader().parse( xis, type, null ); } /** @deprecated {@link org.apache.xmlbeans.xml.stream.XMLInputStream} */ public static com.example.Address parse(org.apache.xmlbeans.xml.stream.XMLInputStream xis, org.apache.xmlbeans.XmlOptions options) throws org.apache.xmlbeans.XmlException, org.apache.xmlbeans.xml.stream.XMLStreamException { return (com.example.Address) org.apache.xmlbeans.XmlBeans.getContextTypeLoader().parse( xis, type, options ); } /** @deprecated {@link org.apache.xmlbeans.xml.stream.XMLInputStream} */ public static org.apache.xmlbeans.xml.stream.XMLInputStream newValidatingXMLInputStream(org.apache.xmlbeans.xml.stream.XMLInputStream xis) throws org.apache.xmlbeans.XmlException, org.apache.xmlbeans.xml.stream.XMLStreamException { return org.apache.xmlbeans.XmlBeans.getContextTypeLoader().newValidatingXMLInputStream( xis, type, null ); } /** @deprecated {@link org.apache.xmlbeans.xml.stream.XMLInputStream} */ public static org.apache.xmlbeans.xml.stream.XMLInputStream newValidatingXMLInputStream(org.apache.xmlbeans.xml.stream.XMLInputStream xis, org.apache.xmlbeans.XmlOptions options) throws org.apache.xmlbeans.XmlException, org.apache.xmlbeans.xml.stream.XMLStreamException { return org.apache.xmlbeans.XmlBeans.getContextTypeLoader().newValidatingXMLInputStream( xis, type, options ); } private Factory() { } // No instance of this class allowed } } AddressImpl Below is the source for the implementation class: /** * XML Type: address * Namespace: http://www.example.com * Java type: com.example.Address * * Automatically generated - do not modify. */ package com.example.impl; /** * An XML address(@http://www.example.com). * * This is a complex type. */ public class AddressImpl extends org.apache.xmlbeans.impl.values.XmlComplexContentImpl implements com.example.Address { private static final long serialVersionUID = 1L; public AddressImpl(org.apache.xmlbeans.SchemaType sType) { super(sType); } private static final javax.xml.namespace.QName STREET$0 = new javax.xml.namespace.QName("http://www.example.com", "street"); private static final javax.xml.namespace.QName CITY$2 = new javax.xml.namespace.QName("http://www.example.com", "city"); /** * Gets the "street" element */ public java.lang.String getStreet() { synchronized (monitor()) { check_orphaned(); org.apache.xmlbeans.SimpleValue target = null; target = (org.apache.xmlbeans.SimpleValue)get_store().find_element_user(STREET$0, 0); if (target == null) { return null; } return target.getStringValue(); } } /** * Gets (as xml) the "street" element */ public org.apache.xmlbeans.XmlString xgetStreet() { synchronized (monitor()) { check_orphaned(); org.apache.xmlbeans.XmlString target = null; target = (org.apache.xmlbeans.XmlString)get_store().find_element_user(STREET$0, 0); return target; } } /** * Sets the "street" element */ public void setStreet(java.lang.String street) { synchronized (monitor()) { check_orphaned(); org.apache.xmlbeans.SimpleValue target = null; target = (org.apache.xmlbeans.SimpleValue)get_store().find_element_user(STREET$0, 0); if (target == null) { target = (org.apache.xmlbeans.SimpleValue)get_store().add_element_user(STREET$0); } target.setStringValue(street); } } /** * Sets (as xml) the "street" element */ public void xsetStreet(org.apache.xmlbeans.XmlString street) { synchronized (monitor()) { check_orphaned(); org.apache.xmlbeans.XmlString target = null; target = (org.apache.xmlbeans.XmlString)get_store().find_element_user(STREET$0, 0); if (target == null) { target = (org.apache.xmlbeans.XmlString)get_store().add_element_user(STREET$0); } target.set(street); } } /** * Gets the "city" element */ public java.lang.String getCity() { synchronized (monitor()) { check_orphaned(); org.apache.xmlbeans.SimpleValue target = null; target = (org.apache.xmlbeans.SimpleValue)get_store().find_element_user(CITY$2, 0); if (target == null) { return null; } return target.getStringValue(); } } /** * Gets (as xml) the "city" element */ public org.apache.xmlbeans.XmlString xgetCity() { synchronized (monitor()) { check_orphaned(); org.apache.xmlbeans.XmlString target = null; target = (org.apache.xmlbeans.XmlString)get_store().find_element_user(CITY$2, 0); return target; } } /** * Sets the "city" element */ public void setCity(java.lang.String city) { synchronized (monitor()) { check_orphaned(); org.apache.xmlbeans.SimpleValue target = null; target = (org.apache.xmlbeans.SimpleValue)get_store().find_element_user(CITY$2, 0); if (target == null) { target = (org.apache.xmlbeans.SimpleValue)get_store().add_element_user(CITY$2); } target.setStringValue(city); } } /** * Sets (as xml) the "city" element */ public void xsetCity(org.apache.xmlbeans.XmlString city) { synchronized (monitor()) { check_orphaned(); org.apache.xmlbeans.XmlString target = null; target = (org.apache.xmlbeans.XmlString)get_store().find_element_user(CITY$2, 0); if (target == null) { target = (org.apache.xmlbeans.XmlString)get_store().add_element_user(CITY$2); } target.set(city); } } } Java Model - JAXB JAXB implementations produce annotated POJOs. The generated classes closely resemble the ones we created by hand in the comparisons to Simple and XStream. Address // // This file was generated by the JavaTM Architecture for XML Binding(JAXB) Reference Implementation, vJAXB 2.1.10 in JDK 6 // See http://java.sun.com/xml/jaxb // Any modifications to this file will be lost upon recompilation of the source schema. // Generated on: 2012.01.23 at 01:19:09 PM EST // package com.example; import javax.xml.bind.annotation.XmlAccessType; import javax.xml.bind.annotation.XmlAccessorType; import javax.xml.bind.annotation.XmlElement; import javax.xml.bind.annotation.XmlType; /** * Java class for address complex type. * * The following schema fragment specifies the expected content contained within this class. * * * * * * * * * * * * * * * */ @XmlAccessorType(XmlAccessType.FIELD) @XmlType(name = "address", propOrder = { "street", "city" }) public class Address { @XmlElement(required = true) protected String street; @XmlElement(required = true) protected String city; /** * Gets the value of the street property. * * @return * possible object is * {@link String } * */ public String getStreet() { return street; } /** * Sets the value of the street property. * * @param value * allowed object is * {@link String } * */ public void setStreet(String value) { this.street = value; } /** * Gets the value of the city property. * * @return * possible object is * {@link String } * */ public String getCity() { return city; } /** * Sets the value of the city property. * * @param value * allowed object is * {@link String } * */ public void setCity(String value) { this.city = value; } } Demo Code In the demo code we will unmarshal an XML file, add a phone number to the resulting customer object, and then marshal the customer back to XML. XMLBeans With XMLBeans the generated domain model is used to unmarshal (line 12) and marshal (line 19). The generated model also contains methods for interacting with collections (line 15), this is necessary as XMLBeans represent collection properties as arrays. package com.example; import java.io.File; import com.example.CustomerDocument.Customer; public class Demo { public static void main(String[] args) throws Exception { File xml = new File("src/com/example/input.xml"); CustomerDocument customerDocument = CustomerDocument.Factory.parse(xml); Customer customer = customerDocument.getCustomer(); PhoneNumber homePhoneNumber = customer.addNewPhoneNumber(); homePhoneNumber.setType("home"); homePhoneNumber.set("555-HOME"); customerDocument.save(System.out); } } JAXB JAXB separates the marshal/unmarshal calls into the standard runtime APIs (lines 9, 13 and 22). A java.util.List is used for collection properties (line 18). package com.example; import java.io.File; import javax.xml.bind.*; public class Demo { public static void main(String[] args) throws Exception { JAXBContext jc = JAXBContext.newInstance("com.example"); File xml = new File("src/com/example/input.xml"); Unmarshaller unmarshaller = jc.createUnmarshaller(); Customer customer = (Customer) unmarshaller.unmarshal(xml); PhoneNumber homePhoneNumber = new PhoneNumber(); homePhoneNumber.setType("home"); homePhoneNumber.setValue("555-HOME"); customer.getPhoneNumber().add(homePhoneNumber); Marshaller marshaller = jc.createMarshaller(); marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true); marshaller.marshal(customer, System.out); } } Summary Both XMLBeans and JAXB produce Java models that make it easy for developers to interact with XML. The JAXB model however is annotated POJOs which has the following advantages: JPA annotations could easily be applied to the JAXB model enabling the model to be persisted in a relational database. Once generated the JAXB model could be modified to handle changes in the XML schema, the XMLBeans model would need to be regenerated. Starting with Java SE 6 no additional compile/runtime dependencies are required for the JAXB model. There are multiple JAXB implementations available: EclipseLink MOXy, Metro, Apache JaxMe, etc. JAXB is the standard binding layer for JAX-WS (SOAP) and JAX-RS (RESTful) Web Services. From http://blog.bdoughan.com/2012/01/how-does-jaxb-compare-to-xmlbeans.html

Add following javascript function to aspx page function check() { alert("this is check value " + ''); } Add following variable declaration on server side i.e aspx.cs public string checkvalue ="indrnilhafa";

I was looking at the pycon US 2012 website when I stumbled upon the huge list of sponsors, which is really impressive. It got me thinking. Are all of these companies using python? If so, which ones are the best companies to work for? If it was up to you, and location and money wasn't a factor, what company would you work for and why? If you already work at one of these companies, can you share what it is you use python for, and what it is like working there? Here is a list of companies that use Python, in no particular order (most of these are pycon US 2012 sponsors). If I missed a company, please let me know. DropBox Heroku Google surveymonkey nebula nasuni microsoft gondor facebook eventbrite new relic zeomega CashStar.com linode dotcloud ccp games revolution systems canonical bit.ly activestate caktus group disqus leapfrog online spotify snoball evite plaidcloud mozilla lab305 walt disney animation studios white oak technologies aldebaran robotics cloud foundry stratasan myyearbook.com threadless cisco kontagent toast driven accense technology net-ng truveris kelly creative tech aarki freshbooks wisertogether bitbucket eucalyptus fwix imaginary landscape cox media group openstack devsar emma shining panda vocollect bigdoor reddit dreamhost Red hat Quora Yelp mixpanel justin.tv YouTube Digg Urban Airship Rackspace To parcipate in this discussion, check out the Survey.

WADL (Web Application Description Language) is to REST what WSDL is to SOAP. The mere existence of this language causes a lot of controversy (see: Do we need WADL? and To WADL or not to WADL). I can think of few legitimate use cases for using WADL, but if you are here already, you are probably not seeking for yet another discussion. So let us move forward to the WADL itself. In principle WADL is similar to WSDL, but the structure of the language is much different. Whilst WSDL defines a flat list of messages and operations either consuming or producing some of them, WADL emphasizes the hierarchical nature of RESTful web services. In REST, the primary artifact is the resource. Each resource (noun) is represented as an URI. Every resource can define both CRUD operations (verbs, implemented as HTTP methods) and nested resources. The nested resource has a strong relationship with a parent resource, typically representing an ownership. A simple example would be http://example.com/api/books resource representing a list of books. You can (HTTP) GET this resource, meaning to retrieve the whole list. You can also GET the http://example.com/api/books/7 resource, fetching the details of 7th book inside books resource. Or you can even PUT new version or DELETE the resource altogether using the same URI. You are not limited to a single level of nesting: GETting http://example.com/api/books/7/reviews?page=2&size=10 will retrieve the second page (up to 10 items) of reviews of 7th book. Obviously you can also place other resources next to books, like http://example.com/api/readers The requirement arose to formally and precisely describe every available resource, method, request and response, just like WSDL guys were able to do. WADL is one of the options to describe “available URIs", although some believe that well-written REST service should be self-descriptive (see HATEOAS). Nevertheless here is a simple, empty WADL document: Nothing fancy here. Note that the tag defines base API address. All named resources, which we are just about to add, are relative to this address. Also you can define several tags to describe more than one APIs. So, let's add a simple resource: This defines resource under http://example.com/api/books with two methods possible: GET to retrieve the whole list and POST to create (add) new item. Depending on your requirements you might want to allow DELETE method as well (to delete all items), and it is the responsibility of WADL to document what is allowed. Remember our example at the beginning: /books/7? Obviously 7 is just an example and we won't declare every possible book id in WADL. Instead there is a handy placeholder syntax:There are two important aspects you should note: first, The {bookId} place-holder was used in place of nested resource. Secondly, to make it clear, we are documenting this place-holder using tag. We will see soon how it can be used in combination with methods. Just to make sure you are still with me, the document above describes GET /books and GET /books/some_id resources. The web service is getting complex, however it describes quite a lot of operations. First of all GET /books/42/reviews is a valid operation. But the interesting part is the nested tag. As you can see we can describe parameters of each method independently. In our case optional query parameters (as opposed to template parameters used previously for URI place-holders) were defined. This gives the client additional knowledge about acceptable page and size query parameters. This means that /books/7/reviews?page=2&size=10 is a valid resource identifier. And did I mention that every resource, method and parameter can have documentation attached as per the WADL specification? We will stop here and only mention about remaining pieces of WADL. First of all, as you have probably guessed so far, there is also a child tag possible for each . Both request and response can define exact grammar (e.g. in XML Schema) that either the request or the response must follow. The response can also document possible HTTP response codes. But since we will be using the knowledge you have gained so far in a code-first application, I intentionally left the definition. WADL is agile and it allows you to define as little (or as much) information as you need. So we know the basics of WADL, now we would like to use it, maybe as a consumer or as a producer in a Java-based application. Fortunately there is a wadl.xsd XML Schema description of the language itself, which we can use to generate JAXB-annotated POJOs to work with (using xjc tool in the JDK): $ wget http://www.w3.org/Submission/wadl/wadl.xsd $ xjc wadl.xsd And there it... hangs! The life of a software developer is full of challenges and non-trivial problems. And sometimes it is just an annoying network filter that makes suspicious packets (together with half hour of your life) disappear. It is not hard to spot the problem, once you recall that article written around 2008: W3C’s Excessive DTD Traffic: Accessing xml.xsd from the browser returns an HTML page instantly, but xjc tool waits forever. Downloading this file locally and correcting the schemaLocation attribute in wadl.xsd helped. It's always the little things... $ xjc wadl.xsd parsing a schema... compiling a schema... net/java/dev/wadl/_2009/_02/Application.java net/java/dev/wadl/_2009/_02/Doc.java net/java/dev/wadl/_2009/_02/Grammars.java net/java/dev/wadl/_2009/_02/HTTPMethods.java net/java/dev/wadl/_2009/_02/Include.java net/java/dev/wadl/_2009/_02/Link.java net/java/dev/wadl/_2009/_02/Method.java net/java/dev/wadl/_2009/_02/ObjectFactory.java net/java/dev/wadl/_2009/_02/Option.java net/java/dev/wadl/_2009/_02/Param.java net/java/dev/wadl/_2009/_02/ParamStyle.java net/java/dev/wadl/_2009/_02/Representation.java net/java/dev/wadl/_2009/_02/Request.java net/java/dev/wadl/_2009/_02/Resource.java net/java/dev/wadl/_2009/_02/ResourceType.java net/java/dev/wadl/_2009/_02/Resources.java net/java/dev/wadl/_2009/_02/Response.java net/java/dev/wadl/_2009/_02/package-info.java Since we'll be using these classes in a maven based project (and I hate committing generated classes to source repository), let's move xjc execution to maven lifecycle: org.codehaus.mojo jaxb2-maven-plugin 1.3 net.java.dev.jaxb2-commons jaxb-fluent-api 2.0.1 com.sun.xml jaxb-xjc xjc -Xfluent-api bindings.xjb net.java.dev.wadl Well, pom.xml isn't the most concise format ever... Never mind, this will generate WADL XML classes during every build, before the source code is compiled. I also love the fluent-api plugin that adds with*() methods along with ordinary setters, returning this to allow chaining. Pretty convenient. Finally we define more pleasant package name for generated artifacts (if you find net.java.dev.wadl._2009._02 package name pleasant enough, you can skip this step) and add Wadl prefix to all generated classes bindings.xjb file: We are now ready to produce and consume WADL in XML format using JAXB and POJO classes. Equipped with that knowledge and the foundation we are ready to develop some interesting library – which will be the subject of the next article. From http://nurkiewicz.blogspot.com/2012/01/gentle-introduction-to-wadl-in-java.html

Overview Relative encoding is another data compression algorithm. While run-length encoding, bitmap encoding and diagram and pattern substitution were trying to reduce repeating data, with relative encoding the goal is a bit different. Indeed run-length encoding was searching for long runs of repeating elements, while pattern substitution and bitmap encoding were trying to “map” where the repetitions happen to occur. The only problem with these algorithms is that the input stream of data is not always constructed out of repeating elements. It is clear that if the input stream contains many repeating elements there must be some way of reducing them. However that doesn’t mean that we cannot compress data if there are no repetitions. It all depends on the data. Let’s say we have the following stream to compress. 1, 2, 3, 4, 5, 6, 7 It's hard to imagine how this stream of data can be compressed. The same problem may occur when trying to compress the alphabet. Indeed the letters of the alphabet are the very base of words so it is the minimal part for word construction and therefore hard to compress. Fortunately this isn’t true always. An algorithm that tries to deal with non-repeating data is relative encoding. Let’s see the following input stream – years from a given decade (the 90′s). 1991, 1991, 1999, 1998, 1991, 1993, 1992, 1992 Here we have 39 characters and we can reduce them. A natural approach is to remove the leading “19” as we humans often do. 91, 91, 99, 98, 91, 93, 92, 92 Now we have a shorter string, but we can go even further by keeping only the first year. All other years will as relative to this year. 91, 0, 8, 7, 0, 2, 1, 1 Now the volume of transferred data is reduced a lot (from 39 to 16 – more than 50%). However there are some questions we need to answer first, because the stream wont always be formatted in such a pretty way. How about the next character stream? 91, 94, 95, 95, 98, 100, 101, 102, 105, 110 We see that the value 100 is somehow in the middle of the interval and it is handy to use it as a base value for the relative encoding. Thus the stream above will become: -9, -6, -5, -5, -2, 100, 1, 2, 5, 10 The problem is that we can’t always decide which value will be the base value so easily. What if the data was dispersed in a different way: 96, 97, 98, 99, 100, 101, 102, 103, 999, 1000, 1001, 1002 Now the value of “100” isn’t useful, because compressing the stream will get something like this: -4, -3, -2, -1, 100, 1, 2, 3, 899, 900, 901, 902 To group the relative values around “some” base values will be far more handy. (-4, -3, -2, -1, 100, 1, 2, 3) (-1, 1000, 1, 2) However, to decide which value will be the base value isn’t that easy. Also the encoding format is not so trivial. On the other hand, this type of encoding can be useful in some specific cases as we can see below. Implementation The implementation of this algorithm depends on the specific task and the format of the data stream. Assuming that we have to transfer the stream of years in JSON from a web server to a browser, here’s a short PHP snippet. // JSON: [1991,1991,1999,1998,1999,1998,1995,1997,1994,1993] $years = array(1991,1991,1999,1998,1999,1998,1995,1997,1994,1993); function relative_encoding($input) { $output = array(); $inputLength = count($input); $base = $input[0]; $output[] = $base; for ($i = 1; $i < $inputLength; $i++) { $output[] = $input[$i] - $base; } return $output; } // JSON: [1991,0,8,7,8,7,4,6,3,2] echo json_encode(relative_encoding($years)); Application This algorithm may be very useful in many cases, such as this one: there are plenty of map applications around the web. Some products such as Google Maps, Yahoo! Maps, Bing Maps are quite famous, while there are also very useful open source projects like OpenStreetMap. The web sites using these apps number in the thousands. A typical use case is to transfer lots of Geo coordinates from a web server to a browser using JSON. Indeed any GEO point on Earth is relative to the point (0,0), which is located near the west coast of Africa, however on large zoom levels, when there are tons of markers we can transfer the information with relative encoding. For instance the following diagram shows San Francisco with some markers on it. The coordinates are relative to the point (0,0) on Earth. Map markers can be relative to the (0, 0) point on Earth, which can occasionally be useless. Far more useful may be to encode those markers, relative to the center of the city, thus we can save some space. Relative encoding can be useful for map markers on a large zoom level, however this type of compression can be tricky. For example, when dragging the map and updating the marker array. On the other hand, we must group markers if we have to load more than one city. That’s why we must be careful when implementing it. But it can be very useful – for instance on initial load of the map we can reduce data and speed up the load time. The thing is that with relative encoding we can save only changes to base value (data) – something like version control systems and thus reducing data transfer and load. Here’s a graphical example. In the first case on the diagram below we can see that each item is stored on its own. It doesn’t depend on the adjacent items and it can be completely independent of them. However we can keep full info only for the first item and any other item will be relative to it, like on the diagram bellow. Source: http://www.stoimen.com/blog/2012/01/30/computer-algorithms-data-compression-with-relative-encoding/

Recently, we had a project where we helped a client index a corpus of Chinese language documents in Solr. We have asked Dan Funk, a committer to Project Blacklight to provide a guest blog post for us on the details of how to approach indexing Chinese, particularly when you are a non-speaker. Take it away, Dan! Indexing Chinese in Solr Prologue (Including thanks, and some vital orientation) Before I start, I’d like to lay some thanks on a few people who helped me muddle through indexing a language I can’t speak, and having me come off looking like a pro. Wiley Kestner (@prairie_dogg) sat for hours giving me tips and pointers about Chinese. Christopher Ball helped me quickly put an excellent and professional face on my work by using the Blacklight project. And Eric Pugh (@dep4b) provided some much needed mentoring – helping me see a way forward in what I initially believed was an intractable problem. If you don’t read Chinese or have not worked with it before, here are few things you should know: Chinese words are frequently made up of more than one character, and words are not separated by spaces. (read this as “Tokenization is a big problem.”) Spoken Chinese is completely different from written Chinese, so don’t stress about the multitude of dialects when you are indexing. There are two common types of written Chinese: Standardized and Traditional. Since Traditional can be converted to Standardized fairly easily, the focus of this document is on Standardized. Traditional text has many more characters and thus the potential for deeper subtler meanings. Though traditionally written from top to bottom, right to left, it is far more common to see Chinese written from left to right – particularly on the web. Don’t depend on your documents being in UTF – you are far more likely to encounter GB2312 encoding. A great method for testing relevancy in a language you don’t know is to use a Judgment List, please see Eric Pugh’s presentation here for more information. My Best Advice: Ok, here are two most important pieces of advice I can give you: #1: Separate your Chinese text into its own field(s). That is to say, don’t try and index multiple languages in the same field. If your Lucene/Solr field structure is complicated, add a second core with duplicate field names. Why? A. You set yourself up for handling additional languages fluidly and effectively. B. You can use the best indexer available for each language (see advice #2) C. You improve overall performance because the indexes are smaller and tighter. D. You remove confusing, and likely false, results in a language the end user does not understand. #2: Use the CJK or Paoding analyzers for your Chinese Text. There is some great documentation out there for CJK, but if you would like to give Paoding a shot, here are some directions to help get you up and running: 1. Don’t use the binary distribution. It won’t work with the latest versions of Solr. Instead, grab the source: dan@maus:~$ cd code dan@maus:~/code $ svn co http://paoding.googlecode.com/svn/trunk/ paoding-analysis 2. Compile it with Ant. dan@maus:~/code/paoding-analysis $ cd paoding-analysis dan@maus: ant … Building jar: paoding-analysis.jar 3. Build a modified Solr war file. The Paoding analyzer, while brilliant at analyzing Chinese text, was not originally built to work well in a web deployed environment, and depends heavily on file paths to get to its built in dictionaries. To correct for this, you will need to inject the analyzer and it’s configuration files into your solr war file. I tested this approach with apache-solr-3.4 doing the following: dan@maus:~$ mkdir temp dan@maus:~$ cd temp dan@maus:~/tmp$ unzip /usr/local/apache-solr-3.4.0/dist/apache-solr-3.4.0.war dan@maus:~/tmp$ cp ~/code/paoding-analysis/paoding-analysis.jar WEB-INF/lib/ dan@maus:~/tmp$ cp ~/code/paoding-analysis/classes/*.properties WEB-INF/ dan@maus:~/tmp$ zip -r * apache-solr-3.4.0-paoding.war 4. Update your Solr configuration and add support for a paoding string. 5. Copy over Paoding’s dictionary files into your solr home directory. dan@maus:~/solr_home$ cp ~/code/paoding-analysis/dic my_solr_home 6. Set an environment variable to let the Paoding Analyzer know where to find the dictionary files: dan@maus:~/solr_home$ java -DPAODING_DIC_HOME=./dic -jar start.jar Choosing the right Analyzer Now that I’ve recommended Paoding and CJK, let me back that up with some details. Below I delve just a little more into the structure of Chinese text, and then run through a comparison of the available tokenizers to help give you an idea of their differences. The Structure of Chinese Text Most languages uses spaces to separate their words. A common misconception is that Chinese words are its characters – but this is the case only a fraction of the time. Take 的 (de) for example. It is the single most common character in Standard Chinese by far. It has little use on its own, but when placed with other characters it can mean: 我的my; 高的high, tall; 是的 that’s it, that’s right; 是…的one who…; 目的 goal, true, real; 的确 certainly In short, you can’t search for the characters individually as if they all carry the same weight or the relevance of the search results will be embarrassingly reduced. What Analyzers are available? Let me introduce you to the options, then follow up with some comparisons that will show off how the tokenizing will actually differ … To my knowledge what follows is a complete list of the open source options available for parsing, indexing and searching Chinese characters in Solr/Lucene. While commercial options definitely exist, they were not a part of this comparison. Method Pros Cons Default Solr setup No new configuration required, and roughly supports multiple languages. Tokenized on spaces – but will shift to character tokenization for Chinese text. See previous section for why this is problematic. CJK Thoughtfully parses Chinese characters – understands that character groups alter meaning. Ships with and is part of Solr’s default configuration. Does not use a dictionary, depends largely on an n-gram based algorithm that creates all possible groupings of pairs of symbols in the text. Smart Chinese Uses a dictionary to pull out characters. Ships with solr as an add-on package. The dictionary is minimal and handles general cases well, but many nuances of the language are lost. It requires a custom Solr configuration. Paoding Uses a large set of dictionaries, and provides exceptionally good search results across a multitude of contexts. Can be very difficult to configure and setup – almost all documentation is written in Chinese. Does not ship with Solr, and must be built from source to work correctly with the latest stable Solr versions. About the Sample Document Set: A set of 12 documents were loaded into Lucene. The first 10 are about “types of fish” and are based on a quick google search of the same. The 11th document is a wikipedia article on Hồ Chí Minh , and the 12th document is about a person whose name begins Hồ Chí. Example 1: 爬蟲 爬蟲 means “Reptile”. 爬 : [pá] crawl, climb, 蟲 : [chóng] The traditional form of 虫. meaning worm, paired with 书 to mean insect. So here is a case where we have a traditional character*, and a paired set of characters that have an alternate meaning from what they mean separately. In this table the “T1”, “T2” … represent the terms parsed out by the various analyzers. In the example below the string “爬蟲” is split into two tokens by the default solr setup, but remain a single token in CJK. Method T1 T2 Hits Default Solr setup 爬 蟲 2 hits (doc 8 and doc 3) * 爬蟲類:”reptile” – a good hit. * 爬岩鳅: “Beaufortia loach” – bad hit. - Even more problematic, is that your highlighting will identify these as two seperate hits, even when it gets it right. CJK 爬蟲 1 hit (doc 8 ) It gets the right document. But this is because CJK always groups by 2, we will see it fall short on the next example. Smart Chinese 爬 蟲 2 hits (doc 8 and doc 3) Paoding 爬蟲 1 hit (doc 8 ) * Note: A second run, replacing the traditional symbol 蟲 with the standardized 虫 symbol does not match any documents in the test set, though it would have been correct to do so. The ICU Project provides an API that would perform this conversion. Example 2: 胡志明 Hồ Chí Minh was profoundly important leader in Vietnam. However, divide these characters up and you might get “A recklessly clear magazine.” 胡志明 means “Hồ Chí Minh”. 胡 : [Hu] “recklessly” 胡说 nonsense (F鬍) (=胡子 húzi) beard (F衚) 胡同 hútòng lane 志 : [zhì] (=意志 yìzhì) will, (=标志 biāozhì) mark; 同志 tóngzhì comrade, (F誌) 杂志 zázhì magazine 明 : [míng] bright, clear, distinct, next (day or year), ; 明白 míngbai clear, understand And if you randomly pair (in the case of CJK), you just get sounds, common pairings used in names. Method T1 T2 T3 Hits Default Solr setup 胡 志 明 6 Hits: 胡志明 (Hồ Chí Minh) 胡 志 (Ho Chi) 明目 (eyesight) 眼目 (eyes) 杂志中的 (magazines) 起明显 (the aparent) CJK 胡志 志明 2 Hits 胡志明 (Hồ Chí Minh) 胡志 (Ho Chi) Smart Chinese 胡 志 明 4 Hits: 胡志明 (Hồ Chí Minh) 胡 志 (Ho Chi) 明目 (eyesight) 眼目 (eyes) Paoding 胡志明 胡志明 (Hồ Chí Minh) In Conclusion It is possible for you to index Chinese, even if you don’t speak it. The largest problem you will face is in correctly parsing the text, but there are several effective tools that help solve the problem. I would strongly discourage you from indexing Chinese content with Solr’s default settings. You will not get good results. If you need to quickly add support for Chinese to an existing project, I highly recommend using the CJK analyzer. However, if you have a discerning audience, a specialized area, or the need to enhance the quality of your results over time (by expanding on the included dictionaries) then Paoding is an excellent choice. Resources and References http://www.zein.se/patrick/3000char.html – The most common Chinese characters in order of frequency http://translate.google.com/ – A fantastic way to quickly translate a few characters or a whole page of text. http://site.icu-project.org/ – Provides an API for converting from Traditional to Standardized Chinese Characters. Source: http://www.opensourceconnections.com/2011/12/23/indexing-chinese-in-solr/

When a subclass violates the Liskov Substitution Principle, or uses only part of a superclass, it is a warning sign that composition can simplify the design. Refactoring to composition transform the superclass into an object of its own, which becomes the collaborator of the class under refactoring. Instead of inheriting every public method, the object will just expose the strictly needed methods. This refactoring is one of the most underused in the PHP world. Don't be afraid to try out composition when you see duplicated code. Why composition? The elimination of duplication through inheritance presents some issues. First, inheritance can be exploited just for code reuse instead of for establishing semantic relationships. Abstract classes with names such as VehicleAbstract, extended by Vehicle, are artificial constructs that do not represent anything in the problem domain. Moreover, inheritance exposes every public method of the superclass, possibly violating encapsulation. It's only a matter of time before someone calls a method which was not supposed to be available. The third problem is related to unit testing, and the duplication of test code. Should we test just the subclasses behavior? Or should we test also the inherited features? In the latter case, we will duplicate test code. Inheritance and delegation (also known as composition) are the two basic relationships between classes in OOP. They are equivalent from a theoretical, functional point of view - but so is a Turing Machine or the whitespace language. Steps Create a field in the subclass, and initialize it to $this. It will contain the collaborator. Change the methods in the subclass to use the delegate field. Methods which are inherited may need to be introduced as a delegation to parent. Remove the subclass declaration, and replace the delegate with a new instance of the superclass. Throughout the refactoring, the tests should always pass. This refactoring is crucial as it opens up further possibilities: for example, Dependency Injection performed on the collaborator, or the extraction of an interface containing the public methods called by the former subclass. Example We start form the end of the Pull Up Method example: we want to transform the NewsFeedItem superclass into a collaborator with the same behavior. assertEquals("Hello, world! -- giorgiosironi", $post->__toString()); } public function testALinkShowsItsAuthor() { $link = new Link("http://en.wikipedia.com", "giorgiosironi"); $this->assertEquals("http://en.wikipedia.com -- giorgiosironi", $link->__toString()); } } abstract class NewsFeedItem { /** * @var string references the author's Twitter username */ protected $author; /** * @return string an HTML printable version */ public function __toString() { return $this->displayedText() . " -- $this->author"; } /** * @return string */ protected abstract function displayedText(); } class Post extends NewsFeedItem { private $text; public function __construct($text, $author) { $this->text = $text; $this->author = $author; } protected function displayedText() { return $this->text; } } class Link extends NewsFeedItem { private $url; public function __construct($url, $author) { $this->url = $url; $this->author = $author; } protected function displayedText() { return "url\">$this->url"; } } Public methods cannot be inherited from a collaborator, so as a preliminary step they must be delegated to it. class Post extends NewsFeedItem { private $text; public function __construct($text, $author) { $this->text = $text; $this->author = $author; } protected function displayedText() { return $this->text; } } class Link extends NewsFeedItem { private $url; public function __construct($url, $author) { $this->url = $url; $this->author = $author; } protected function displayedText() { return "url\">$this->url"; } public function __toString() { return parent::__toString(); } } Deciding a name for the role of the collaborator is an important step. It is very likely to change with respect to a name that follows LSP and is used for a superclass. We choose Format, since the parent models a way to print out the author and content fields. We also extract a method, display(), in the superclass, to split the formatting behavior from the wiring to the fields. We plan to use display() as a collaborator, while __toString() was made for inheritance and will be discontinued. abstract class NewsFeedItem { /** * @var string references the author's Twitter username */ protected $author; /** * @return string an HTML printable version */ public function __toString() { return $this->display($this->displayedText(), $this->author); } public function display($text, $author) { return "$text -- $author"; } /** * @return string */ protected abstract function displayedText(); } class Post extends NewsFeedItem { private $text; private $format; public function __construct($text, $author) { $this->text = $text; $this->author = $author; $this->format = $this; } protected function displayedText() { return $this->text; } public function __toString() { return parent::__toString(); } } class Link extends NewsFeedItem { private $url; private $format; public function __construct($url, $author) { $this->url = $url; $this->author = $author; $this->format = $this; } protected function displayedText() { return "url\">$this->url"; } public function __toString() { return parent::__toString(); } } We can start using the delegate instead of parent, and of relying on inheritance. __toString() is the only point where we have to intervene: class Post extends NewsFeedItem { private $text; private $format; public function __construct($text, $author) { $this->text = $text; $this->author = $author; $this->format = $this; } protected function displayedText() { return $this->text; } public function __toString() { return $this->format->display($this->displayedText(), $this->author); } } class Link extends NewsFeedItem { private $url; private $format; public function __construct($url, $author) { $this->url = $url; $this->author = $author; $this->format = $this; } protected function displayedText() { return "url\">$this->url"; } public function __toString() { return $this->format->display($this->displayedText(), $this->author); } } Now we can eliminate abstract and the abstract method in the superclass, plus the extends keyword in the subclasses. This means now $this->format would be initialized to an instance of TextSignedByAuthorFormat, which is the new name for NewsFeedItem. We also have to push down $this->author. class TextSignedByAuthorFormat { /** * @return string an HTML printable version */ public function __toString() { return $this->display($this->displayedText(), $this->author); } public function display($text, $author) { return "$text -- $author"; } } class Post { private $text; private $author; private $format; public function __construct($text, $author) { $this->text = $text; $this->author = $author; $this->format = new TextSignedByAuthorFormat(); } protected function displayedText() { return $this->text; } public function __toString() { return $this->format->display($this->displayedText(), $this->author); } } class Link { private $url; private $author; private $format; public function __construct($url, $author) { $this->url = $url; $this->author = $author; $this->format = new TextSignedByAuthorFormat(); } protected function displayedText() { return "url\">$this->url"; } public function __toString() { return $this->format->display($this->displayedText(), $this->author); } } Finally, we can simplify part of the code. We delete the __toString() on TextSignedByAuthorFormat which is dead code; and inline the displayedMethod() on Post, which served the inheritance-based solution but now is an unnecessary indirection. class TextSignedByAuthorFormat { public function display($text, $author) { return "$text -- $author"; } } class Post { private $text; private $author; private $format; public function __construct($text, $author) { $this->text = $text; $this->author = $author; $this->format = new TextSignedByAuthorFormat(); } public function __toString() { return $this->format->display($this->text, $this->author); } } class Link { private $url; private $author; private $format; public function __construct($url, $author) { $this->url = $url; $this->author = $author; $this->format = new TextSignedByAuthorFormat(); } protected function displayedText() { return "url\">$this->url"; } public function __toString() { return $this->format->display($this->displayedText(), $this->author); } } There are many further steps we could make: inject the TextSignedByAuthorFormat object. Consequently, if the logic in the collaborator expands we can refactor tests to use a Test Double. Move $this->author into the format. Apply Extract Interface (Format should be the name), to be able to support multiple output formats. Another implementation could place a link on the author too, or could strip all or some of the tags for displaying in a RSS or in a tweet.

If you need to schedule jobs in Java, it is fairly common in the industry to use Quartz directly or via Spring integration, but you might want to think twice.

I have been inserting log4j entries into a mongodb database and each entry has been given an ISODate timestamp: "timestamp" : ISODate("2012-01-17T22:30:19.839Z") To create a mapping for this, I had to manually add the timestamp as Spring Roo did not allow timestamp to be used as it was a reserved word. So I manually added: @DateTimeFormat(style="MM/dd/yyyy") private java.util.Date timestamp; But I started getting the following error: Invalid style specification: MM/dd/yyyy The stack trace for that error was: org.joda.time.format.DateTimeFormat.createFormatterForStyle(DateTimeFormat.java:702) org.joda.time.format.DateTimeFormat.patternForStyle(DateTimeFormat.java:212) com.comcast.uivr.web.LoggingController_Roo_Controller.ajc$interMethod$com_comcast_uivr_web_LoggingController_Roo_Controller$com_comcast_uivr_web_LoggingController$addDateTimeFormatPatterns(LoggingController_Roo_Controller.aj:98) com.comcast.uivr.web.LoggingController.ajc$interMethodDispatch2$com_comcast_uivr_web$addDateTimeFormatPatterns(LoggingController.java:1) com.comcast.uivr.web.LoggingController_Roo_Controller.ajc$interMethodDispatch1$com_comcast_uivr_web_LoggingController_Roo_Controller$com_comcast_uivr_web_LoggingController$addDateTimeFormatPatterns(LoggingController_Roo_Controller.aj) com.comcast.uivr.web.LoggingController_Roo_Controller.ajc$interMethod$com_comcast_uivr_web_LoggingController_Roo_Controller$com_comcast_uivr_web_LoggingController$list(LoggingController_Roo_Controller.aj:66) com.comcast.uivr.web.LoggingController.list(LoggingController.java:1) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) java.lang.reflect.Method.invoke(Method.java:597) org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:212) org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126) org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96) org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617) org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578) org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80) org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:900) org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:827) org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882) org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:778) javax.servlet.http.HttpServlet.service(HttpServlet.java:617) javax.servlet.http.HttpServlet.service(HttpServlet.java:717) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77) org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88) org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857) org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) java.lang.Thread.run(Thread.java:662) To fix this I attempted to add the ISO date format for the @DateTimeFormat @DateTimeFormat(style="yyyyMMdd'T'HHmmss.SSSZ") private java.util.Date timestamp; Which still did not work and had the error. To resolve this I shitched to use ISO.DATE_TIME as the style: @DateTimeFormat(iso=ISO.DATE_TIME) private java.util.Date timestamp; From http://www.baselogic.com/blog/development/springframework/mapping-mongodb-isodate-spring-roo-entity/

Right. Let’s have a look at the massive technical implications of the Fix Puppet idea. As I mentioned in my earlier blogpost, in order to fix puppet in a sensible way, we’ll have to review all, and overhaul some of the underlying infrastructure that allows it all to run. The interlinks and dependencies between all the parts are a little tricky to visualise. So, here’s a picture. Anything in red needs attention, and the stuff in green *just works*. Things in blue are install stages, and these are what we’re working on making perfect. Right, so we’ve basically got a directed graph, representing the steps and stages that have to happen to a new machine before users can log in. The steps taken to build a machine, roughly look like this: Unbox. Plug in. Configure Netboot. Hand MAC Address to DHCP server and assign a hostname. Client PXEBoots. Client downloads a preseed file. Client installs itself. Client Reboots. Puppet runs on First Boot. Puppet completes. Client Reboots again. Users login That’s about it, really. The first 4 steps are a hell of a lot easier with the support and co-operation of the supplier. It’s nice to have systems preconfigured to PXE boot as the BIOS default, and even cooler if they can send the MAC addresses as labels on each physical machine. If we’re going to build out a new infrastructure, we’re going to need to review and reinstall the servers that provide this infrastructure, before we can build any workstations. I’m a massive massive fan of puppet, and believe that it should be used for the configuration of all servers and workstations. As such, I didn’t want to rebuild anything without using puppet, so the first step, had to be getting puppet working again. So, without further ado, let’s take a look at the Puppet portion of this, well, one of them. My predecessor saw fit that all nodes should be defined with puppet-dashboard, which is itself, a fine piece of software, but I think more for reporting than specification. Initially, at least, I rebuilt the puppet manifest from a known-good configuration. Namely the base configs I wrote for a blogpost about a year ago; base configs that I’m going to update soon. I’m a bit of an old fashioned puppet user. I like my nodes defined in nodes.pp, not some External Node Classifier service. Reason being, I like to be able to look in one place and find exactly what I want. It’s not a massive ballache to clone down the puppet git repo, make a change and push it back up. In fact, it’s better than having a web interface for your node classifications, because git provides you with an intrinsic log of what was changed, and it’s easy to revert to an old version, because everything’s stored in source control. You can also test what you’re about to do, because again, it’s just a source control repo. I’m a fan of having Jenkins run a few sanity checks on your puppet repo, but that’s a digression for another blogpost. I’m not going to go into great depth about how to install DHCP and DNS, and how to make it work with puppet, at least, not here. What I will say, though is that Puppet Module Tool is the most fantastically easy way to generate boilerplate modules for puppet. All you need to do is run puppet-module generate tomoconnor-dhcp and you get a full puppet module folder called tomoconnor-dhcp which contains all the structure according to the best practice guidelines. Excellent. As part of the review process, it became quite apparent that Bind9 has no sensible admin/management interface, or at least, there wasn’t one installed, and frankly, anything that has such horrific config files should be shot. Having had good experience and results using PowerDNS in the past, we decided that this would be a valid upgrade from BIND. PowerDNS relies on a SQL backend for storing the record data in. You can use either MySQL or PostgreSQL, or possibly some others. Since MySQL can be a bitch, and is, to all serious purposes, a toy database, Postgres seems like a better choice. 9.1 is stable, and there are deb package available for it. 9.1 also does hot-standby replication, which is a miracle, because Postgres replication used to be a massive pain in the testicles. There were, initially some mysterious problems with the TFTPd server being generally crappy, mostly regarding timeouts, which was because the storage of the TFTP data was on a painfully slow disk. Moving it from there to the NFS mount dramatically increased performance and stopped TFTP going crazy. In the TFTP'd config, there's a block for configuring the boot options of the preseed install. This is how PXE hands over the details of the preseed server, and the classes of preseed file to run (basically, which modules) label lucid_ws menu label ^2) Auto Install Ubuntu Lucid WorkStation text help Start hands off install of a workstation. endtext menu default kernel ubuntu-1004-installer/amd64/linux append tasks=standard pkgsel/language-pack-patterns= pkgsel/install-language-support=false vga=normal initrd=ubuntu-1004-installer/amd64/initrd.gz -- quiet auto debian-installer/country=GB debian-installer/language=en debian-installer/keymap=us debian-installer/locale=en_GB.UTF8 netcfg/choose_interface=eth0 netcfg/get_hostname=ubuntu netcfg/get_domain=installdomain.wibblesplat.com url=http://autoserver/d-i/lucid/preseed.cfg classes=wibblesplat;workstation DEBCONF_DEBUG=1 Initially, the Preseed files contained all sorts of crazy hacky shit in the d-i late-command setting. late-command is cool. It’s basically the last thing to run before the first reboot when you build a new debian/ubuntu system. You can tell it to do all sorts of stuff in there. You probably shouldn’t, though. Especially when what you’re doing in there is better done elsewhere. The previous Preseed file contained a whole bunch of “inject these source files into /etc/apt/sources.list”, which is utter bullshit, because you can do exactly the same thing with d-i local repositories, which does the same thing, only far far cleaner. That’s not to say that my refactored preseed files don’t use late-command at all. I’ve chosen to insert some lines into /etc/rc.local on the freshly built system that ensures a puppet run at first boot. On the preseed server, there’s a file called “firstboot.sh” which gets dropped into /usr/local/bin by way of a wget command in late-command. The next thing that happens in late-command is a line to remove “exit 0” from /etc/rc.local and replace it with a thing that calls “/usr/local/bin/firstboot.sh” When firstboot runs, it runs puppet, checks for sanity, and then removes itself from /etc/rc.local. The code to actually do that looks like this: d-i preseed/late_command string \ wget -q -O /target/root/firstboot.sh http://autoserver/d-i/bin/firstboot.sh && \ chmod +x /target/root/firstboot.sh && \ sed -i 's_exit 0_sh /root/firstboot.sh_' /target/etc/rc.local This relies on having something on http://autoserver that is basically just apache hosting some files for the preseeder to retrieve during installation. Cool huh? That ensures that the first thing that happens once the new machine has been built and rebooted, is a puppet run. Some stuff we do here relies on our hand-rolled deb packages, which are stored in our own, internal APT repo. We’ve also got an APT cache, created and maintained by apt-cacher-ng, which at least means that when you’re rebuilding systems frequently, that all the packages you would otherwise download from archive.ubuntu.com come straight over the LAN. The major problem initially with this was the speed, or lack of. It certainly wasn’t performing anywhere near speeds you’d expect from a 1GE LAN, and the reason was again, slow disks. Moving the apt-cache files to the NFS highspeed storage again helped performance. If we struggle in future, I’m going to look at a SSD cache for this, but I think that the performance of the SAS/SATA disks on massively parallel storage provided by our NFS servers will be adequate for the forseeable future. Next up, the Puppetmaster. Again, I was pretty keen on building this from scratch, but using puppet itself to configure it’s own master. Sounds pretty counter-intuitive, right? But the puppet client can bootstrap the master quite easily by using files as it’s source. The first step is to clone down the latest puppet manifests from git, so you either need to git export elsewhere, or install git-core. Your choice. Once you’ve got those, all you need to do is install puppet-client, and run: puppet apply /path/to/your/manifests/site.pp If you’ve written the manifests right, and you’ve got your master defined as a node, you should find that puppet will install puppetmaster, and so on, and then you get a ready and working puppetmaster that just configured itself. I used puppet-module tool to generate modules for the following services/items: “applications” - which actually contains a bunch of custom/proprietary application install rules, a declassified example is there’s a googlechrome.pp file that installs chrome from a PPA. Other modules: dhcp, kernel, ldap, network, nfs, nscd, ntp, nvidia, postgres, powerdns and ssmtp. As is the trend with puppet, and modern DevOps, a vast majority of the code in the entire manifest repository has been gleaned and researched from other puppet modules on github. Acknowledgement is in place where it’s due, and the working copies we’re using are frequently forked on github from the original. It’s great, this, actually. If you search on PuppetForge http://forge.puppetlabs.com/ the array of modules available is staggering. It makes bootstrapping a new manifest set remarkably quick and easy. The NFS module contains a bunch of requirements for mounting NFS shares, and the definitions for an NFS share to be mounted. All pretty simple stuff, but modularised for ease of use. I’m particularly proud of the postgres module which has a master class, and a slave class, which installs and configures the required files and packages to enable streaming hot-standby replication on Postgres9.1 I will release the declassified fork of this soon. I’m going to wrap this post up here. It’s a massively long one, and there’s still lots more left to write. Source: tomoconnor.eu/blogish/low-level-infrastructure-puppet-dns-and-dhcp/

In this post, you'll find a quick, 7-line code block of JavaScript that you can use to covert dates to the MM/DD/YYYY format.

When I first sat down to redesign my personal site I knew that I wanted to incorporate HTML5 Canvas somewhere in the layout. The problem was that I hadn't worked with canvas before and had to start from scratch. I went through the pain of learning every aspect of adding text, drawing shapes, importing image, etc... before I found the amazing canvas framework Processing.JS The content of this article was originally posted in Joey Cadle Allgaier's blog. For those who don't quite fully grasp what HTML5 Canvas check out the W3Schools entry for the element before reading any further, but it's basically an element that defines graphics. Canvas Basics Adding a canvas element is as simple as adding the below markup. The canvas element alone acts as a block level element with all children hidden without the use of javascript to draw text, objects, images, etc... Please note that HTML5 markup and the canvas element is only support by modern browsers such as Firefox (1.5+), Safari (1.3+), Chrome, Opera (9+), and Internet Explorer (9+). Obviously we don't want to go adding canvas elements without some type of alternative display for browsers that do not support canvas rendering. Thankfully all graphics rendered via canvas are layered above any markup contained within the element. Here's how we degrade canvas so that browsers such as Internet Explorer 8 know they need to stop being lazy and upgrade to a more modern browser. First we'll add a link to an HTML5 element shiv for any user with a browser later than IE9 in the portion of our document, adding the element to our stack of recognized html markup: Now let's update our canvas element to target non-modern browsers: Please upgrade your browser to something newer, like Google Chrome The above markup lets anyone using a non-modern browser that they should probably upgrade their browser. You can put substitute text with an image if you want. For instance, any visitor to this site using a browser that doesn't support HTML5 Canvas is met with a standard JPEG logo as opposed to the canvas alternative. CSS Styling Canvas It's always good practice to style your canvas element as until drawing has been accomplished the styling will act as a kind of start screen. While we're at it, we'll also style the child within our canvas element. Styling the child elements inside of your canvas is important so that in non modern browsers we're taking up the same amount of space. #myCanvas, #myCanvas p { width: 460px; height: 250px; background-color: #f5f5f5; color: #555; text-align: center; } In modern browsers our canvas element now displays exactly as we styled it, and non-modern browsers also show a similar styling but with a note for the user to upgrade their browser. Take note that css such as text coloring and background coloring is only useful until our canvas element is initialized. Once initialized the things we draw onto our canvas can not be styled via css. Now that we've covered the basics, how do we go about drawing to the canvas? We could use modern javascript to draw to the canvas, but in this article we're going learn how to use the javascript framework Processing.JS to handle all our drawing. Getting Started With Processing.JS Processing.JS is a port of the Processing Visual Programming Language developed by Ben Fry and Casey Reas designed for use on the web. You develop code using the processing language and processing.js transforms those actions into canvas elements. You can download the latest version of Processing.JS at their website: http://processingjs.org. Let's get started by adding a link to processing.js in the portion of our document: Processing.JS now adds functionality for us to reference our canvas element to a file (file-type: .pde) in which all of our processing code exists. Let's reference our code by adding the "data-processing-sources" attribute to our canvas element: Please upgrade your browser to something newer, like Google Chrome Now all we have to do is create the source file referenced, (in this case "myProcessingCode.pde") and add our Processing code. Writing Processing Code We're going to cover a few basic drawing methods such as shapes, text, and images, but before that I want to go over the two core functions of Processing.JS: setup, and draw. The setup function contains all of the code we want to run when our canvas is initialized. Most importantly this is where you set such key values such as our canvas element's size and framerate. Let's go ahead and set the size, framerate and background of our canvas element: void setup() { size(500, 250); background(245); framerate(30); } In the above code we're telling Processing.JS to set the size of our canvas to a width/height of 500/250 and to set the background of our canvas to an rgb value of 245, 245, 245 (#F5F5F5) and to set our canvas framerate (essential to looping, which we'll discuss later), all at canvas initialization. Note that all Processing functions are designated with "void" and in this case Processing.js recognizes the setup function as the function to be ran at intialization of our canvas. Adding a custom function is simple: void myFunction() { // do something here } Our initial setup function sets values to what our css styling is for background-color and size and our canvas now mimics what we saw before adding any processing code. Now we'll add a 50x50 pink rectangle with a 1 pixel white stroke to a random position of our canvas using by modifying our setup function. void setup() { size(500, 250); background(245); framerate(30); color pink = #ffb5b5; color white = #ffffff; fill(pink); stroke(white); int positionX = int(floor(random(20, 408))); // 20 pixel left and right padding int positionY = int(floor(random(20, 158))); // 20 pixel top and bottom padding rect(positionX, positionY, 50, 50); // x, y, width, height } Going over each function we see that we first declare some color variables using the following syntax: color myColor = #hexvalue; Always assign complicated colors to color variables so that we can link them to methods such as fill, background, and stroke. You can forego the use of hex values and instead use rgb values as so color myColor = color(255, 181 , 181);. Next we declare our fill by using the fill() method. The color value we assign to this method will be the fill color of any shape method we then call. This also applies to our stroke() method. If you do not call fill() and stroke() before declaring the shape the shape will have a default fill color of white and a default 1 pixel stroke of black. If you don't want to fill or stroke the next shape drawn you can do so by replacing fill() and stroke() with noFill() and/or noStroke() methods. noFill(); // the next shape will not be filled noStroke(); // the next shape will not be stroked We can also declare if we want our shape to be antialiased or "smoothed" (no smoothing set by default) or change the weight of our stroke (default stroke weight is 1px) by calling the smooth() and strokeWeight() methods: smooth(); // antialias our shape strokeWeight(10); // set the stroke weight to 10 pixels We now declare a positionX and positionY variable to randomize where our rectangle should appear by using the data method int, since we know our value will be an integer, and we'll make use of the random() method. The random method can and will return a floated value so we'll use the floor() method to round the number returned by random() down. int myInt = int(floor(random(start, end)) Note that in my example I know that the size the canvas is 500x250 so to ensure that my rectangle is positioned at least 20 pixels from the border of the canvas edges my starting value is 20 and my ending value is 500 (canvas size) minus 40 (20 pixel left/right padding) minus 52 (width/height of rectangle including 1pixel stroke) for the position of X and 250(canvas size) minus ... for the Y position. You, however, can use any value you see fit or declare a non random value like so int positionX = 10; The only thing left is to create our shape. We've chosen to create a rectangle by using the rect() method: rect(x, y, width, height); Remember declaring just a bare rect() without setting any fill() and/or stroke() will result in a shape you can't see, so don't forget to call those methods as stated earlier. If you hate rectangles you can change the shape by changing the rect() method to ellipse(), line(), point(), quad(), arc(), or triangle() ellipse(x, y, width, height); line(xStart, yStart, xEnd, yEnd); // doesn't auto stroke point(x, y); // doesn't auto stroke quad(x1, y1, x2, y2, x3, y3, x4, y4); // x,y position of each corner of a rectangle arc(x, y, width, height, start[radian], stop[radian]); // PI radians with or without math operators eg: PI, PI/2, TWO_PI-PI, PI+TWO_PI, etc... triangle(x1, y1, x2, y2, x3, y3); // x,y of each point of a triangle For more 2D shape methods (including curves) check the reference section of the Processing.JS website. Shapes via SVG If you're familiar with SVG and are constantly working with it you'll want to know that you can define shapes via an SVG file by using PShape Datatype: PShape mySVG; // set the PShape datatype to the mySVG variable mySVG = loadShape("mySVGfile.svg"); // load your .svg file using loadShape(); smooth(); // antialias the shape shape(mySVG, x, y, width, height); Note you must always load your svg file using the loadShape() method before calling the shape() method. Adding Text Processing.JS provides us with the PFont Datatype and the methods loadFont(), textFont(), and text() methods. Font loading in canvas can be a bit complex as, despite the ease of the loadFont() method, font support for canvas varies across browsers. Firefox supports canvas fonts the best, but has a pre-defined list of fonts. As of now it's best to use a surely installed font (such as Arial) or to use the PFont_list() method to check the fonts a user has available to load. For more information see the Processing.JS reference to PFont_list(). Let's add some text to our canvas: void setup() { size(500, 250); background(245); framerate(30); color pink = #ffb5b5; color white = #ffffff; fill(pink); stroke(white); int positionX = int(floor(random(20, 408))); int positionY = int(floor(random(20, 158))); rect(positionX, positionY, 50, 50); fill(64); // color the text #404040 rgb(64, 64, 64) PFont fontArial = loadFont("arial"); // load the Arial font textFont(fontArial, 32); // set the font-size of fontArial to 32 point text("Joey Cadle Rocks!", 110, 60); // Add the text to canvas at x, y position } The draw() Function and Image Loading Processing.JS's draw() function is where most of your drawing should take place. The biggest thing to note is that Processing.JS automaticly loops the draw() function at whatever frameRate() you specify in your setup(). Because of this automatic looping we're given the loop() and noLoop() methods. In this next append to our code we'll be loading images by making use of the PImage Datatype and the methods loadImage(), requestImage(), image(), and get(). Lets use the draw() function to handle our drawing from now on and let's load an image using requestImage() as opposed to loadImage() as loadImage() freezes canvas until the image is loaded while requestImage() does not. Here's a look at just the image loading code: PImage img; // PImage for preloading PImage part; // PImage for display img = requestImage("yourImage.png"); // accepted formats are .jpg, .gif, and .png part = img.get(); // get all pixels from the image image(part, 20, 20); // display the image at x, y coordinates Note that we're now going to initialize our PImage and PFont objects outside of our setup() and draw() functions so that they're accessible throughout our script. PImage img; PImage part; PFont fontArial = loadFont("arial"); void setup() { size(500, 250); background(245); frameRate(30); img = requestImage("yourImage.png"); } void draw() { background(245); part = img.get(); image(part, 20, 20); color pink = #ffb5b5; color white = #ffffff; fill(pink); stroke(white); int positionX = int(floor(random(20, 430))); int positionY = int(floor(random(20, 180))); rect(positionX, positionY, 50, 50); fill(64); textFont(fontArial, 32); text("Joey Cadle Rocks!", 110, 60); noLoop(); // Tell Processing.JS to stop looping. } Now we're getting down the heart of Processing.JS by utilizing it's two main features. Note that most methods, including noLoop() and loop() are accessible in other frameworks such as JQuery. You can do things like: $('.some_div').click(function() { loop(); } By specifying a noLoop() in our draw() function we're able to Making Use of Looping We're going to add some animation and some event listing for a mouse movement. We'll remove our rectangle and choose to move our image and text with our mouse. This can be done by calling the mouseMoved() function and making good use of the looping of our draw() function. Our ability to have our image and text follow our mouse hinges on the fact that Processing.JS consistently holds the current position of our mouse in the variables mouseX and mouseY. We'll add 5 frame delay to our movement with some simple math. PImage img; PImage part; PFont fontArial = loadFont("arial"); void setup() { size(500, 250); background(245); frameRate(30); x = 20; // set initial x position y = 20; // set initial y position mX = x; // set mouseX to above x mY = y; // set mouseY to above y delay = 5; // set the frames we want to delay movement img = requestImage("yourImage.png"); } void draw() { x += (mX - x) / delay; // reset our x with current x position minus mouseX position and delay it y += (mY - y) / delay; // reset our y with current y position minus mouseY position and delay it fontX = x + 90; // add the width of our image to ensure its to the right (in the demo case: 90) fontY = y + 40; // add the height of our image to ensure its level (in the demo case: 40) background(245); part = img.get(); image(part, x, y); // draw our image at x, y based on x,y values above. fill(64); textFont(fontArial, 32); text("Joey Cadle Rocks!", fontX, fontY); // Add the text to canvas at fontX and fontY position } void mouseMoved() { mX = mouseX; // set mX to our mouseX position mY = mouseY; // set mY to our mouseY position } Processing.JS has other built in event listeners such as the mouseClicked() and mouseDragged functions. Check out their website for a full list of listeners, but as far as we're concerned, our canvas is now animated and interactive! For a full demo check out the live example here. Conlusion This article is intended to show simplified use of Processing.JS. Do not in anyway take this article and use it to judge the limits of Processing.JS or Canvas in general. The native canvas API is incredibly powerful, as if Processing.JS, this article is just a taste of what you can achieve. Thanks for reading. Short URL: http://bit.ly/z25Lvg Source: http://joeycadle.com/blog/article/1/2012/22/01/html5-canvas-and-processing-js