The Problem Obviously, there are tons of different ways to sync databases, so why should it be described again? Let's imagine that we have an unusual situation with restrictions below: A future system will have some Head Office (HO) and a couple of Branch Offices (BOs) All offices are located in different places, and some of them have difficulties with the internet connection. It could even be a situation where the internet is available for 1-2 hours per day. Almost all vital data is created in the HO and should be presented as read-only in BOs. Data exchange should be limited with appropriate permissions (for example, if an operator has created some sensitive data in the HO for BO1, only BO1 should have access to it). HO should have access to all information that has been created or modified in BOs. According to all described points final decision to write own DB sync mechanism has been made. Basic Idea Due to connection degradation between HO and BOs, we have to sync everything within short-term sessions. Since there is no need to send information to all branches in general cases, we should be able to orchestrate data flow. Those thoughts bring us to the idea that we might implement some kind of RPC where an event occurs in one office, and it is reproduced (replayed) in another. Message queues (MQ) are a perfect solution to sync data between branches. RabbitMQ is my favorite MQ, so I will use it in this example. Also, this application will use the .NET stack which has a convenient API client implementation for RabbitMQ called EasyNetQ. High Level Application Architecture According to the idea of replaying some actions on other system instances, we should be able to divide them into single business-logic operations. The best way to achieve this it is by using the Aggregate Roots approach. The main idea is to have separated objects that are divided by domain entities, and each call to the methods of those objects is a single change to state of the business logic. For example, if we have some domain object Document and the ability to Get, Upsert, or Apply/Unapply, then we should describe its root as (pseudocode): public class DocumentRoot { public Document Get(Id) { ... } public Document Upsert(Document) { ... } public bool Apply(Id) { ... } public bool UnApply(Id) { ... } } Also, it's very important to ensure that each call will be in a transaction in order to avoid data loss. This can be achieved using simple method interception (for example Autofac + Castle.Proxy). In other worlds, the core process will look like this: Keep in mind things as entities primary keys, because data will be populated between different system instances, and we'll need to be sure that ID's will be the same. Also, collisions are possible while using simple auto-incrementing PK's, so our choice is GUID. With the help of a base repository, it's very simple to implement new GUID storage during object creation. Let's assume that we have an ExchangeInformation object that handles all data needed to restore a root call on a remote system. It will contain info about the method name, type name, input, and output params – this data can be obtained from a root interceptor. Also, it should have the list of new ID's, but it's not hard to get them too, even though we'll need to implement the UnitOfWork pattern on an ORM type to support transactions. This will allow us to place our ExchangeInformation in that UoF object (for example, within Entity Framework it's DbContext). Here is the implementation (using EF) of saving any changes in a domain within the base generic repository where the base entity looks like: public class EntityBase { public long Id { get; set; } public Guid Guid { get; set; } } public virtual void Save(T entity) { DbEntityEntry entry = Context.Entry(entity); if (entity.Guid == Guid.Empty) { try { Guid newGuid = Context.ExchangeInformation.IsExchangeRestore ? Context.ExchangeInformation.NewGuids[0] : Guid.NewGuid(); if (Context.ExchangeInformation.IsExchangeRestore) { Context.ExchangeInformation.NewGuids.RemoveAt(0); } else { Context.ExchangeInformation.NewGuids.Add(newGuid); } entity.Guid = newGuid; } catch { throw new Exception("Failed to restore exchange, no guid found"); } entry.State = EntityState.Added; return; } Context.Entry(entity).State = EntityState.Modified; } One more important note: to avoid code duplication, it's necessary to use GUID's on clients, because if they operate any other ID's we'll need to write two different implementations of any method. Big Picture After preparation completion, we can proceed with architecture design. Since every system instance should be able to send and receive new data, we can declare two RMQ topics: input and output. Also, because message flow must be orchestrated, queues for each system instance should be created within the output topic. The simplest strategy for a routing implementation is to use the branch office guide as a key. So we know how to do following at the moment : Save the source event in one office. Put this event to selected queues (selection could be made but it depends on the situation: read from the entity, call some additional method, use attributes etc.) The next step is a solution for how to make output events from one office appear in the input queue of the other office. RabbitMQ has two plugins for that: Federation and Shovel. They are quite similar, but shovel is working on a lower level and has more options to control the synchronization process, so that we'll use the second one to link queues. Shovel is very good with handling connection degradation and has lot of additional configurable options like message republishing properties, routing etc. Now it's time to combine all pieces in to single picture: Aggregators here are simple RabbitMQ consumers that handle incoming messages from other offices and launch appropriate methods. One other problem is restoring transferred params. From my point of view the best way is to use Json.Net with type serialization and restore them on a remote system instance with a small hack: private object[] GetParams(MethodInfo methodInfo, ExchangeInformation information, ExchangeMessage message) { ParameterInfo[] methodParams = methodInfo.GetParameters(); var listParams = new List>(information.InputParamsString); for (int ii = 0; ii < methodParams.Length; ii++) { var jObject = JsonConvert.DeserializeObject(information.OutputValueString); string typeName = jObject["$type"].ToString(); listParams.Add(jObject.ToObject(Type.GetType(typeName))); } return listParams.ToArray(); } Surely appropriate conditions for params count mismatch, so valid deserialization and so on are required. Conclusions The approach I've described is very easy to implement and it has lots of additional places that can be customized. For example, any other method can be executed before/instead of/after restoration on a target branch to change the logic of DOM behavior. The main issue is that collisions can occur if two BOs edit same object at the same time. Actually, it's not hard to track this situation by adding a hash to EntityBase. Nevertheless, a human's decision is needed to resolve conflicts, so a simple UI is necessary in the HO where the operator can choose which data is correct.

one of the goals of our development with our javafx application is to try to keep as much of the ui design as possible within the scene builder ui design tool , and the business logic for the application in java. one of the things that is not entirely clear is how to create a new component in scene builder and then reuse that component within other components created in scene builder. in the example below, i illustrate how to create a custom table and ‘add plan’ components, and then add them to a new scene using fxml. the first step is to create a simple component which acts as an “add plan” widget on the ui. the widget contains an anchorpane, imageview and label. for the next step, open up the fxml file and change the first “anchorpane” declaration to “ create a class which will act as both the root of the component as well as its controller. the class must extend the type that was previously defined in the fxml file. use the fxml loader to set the root and controller of the component to “this” class, and then load the component’s fxml file. /* * to change this template, choose tools | templates * and open the template in the editor. */ package com.lynden.fx.test; import com.lynden.ui.util.uiutilities; import java.io.ioexception; import javafx.event.eventhandler; import javafx.fxml.fxml; import javafx.fxml.fxmlloader; import javafx.scene.input.dragevent; import javafx.scene.input.dragboard; import javafx.scene.input.transfermode; import javafx.scene.layout.anchorpane;/** * * @author robt */ public class testbutton extends anchorpane { @fxml private anchorpane mytestbutton; public testbutton() { fxmlloader fxmlloader = new fxmlloader( getclass().getresource("/com/lynden/planning/ui/testbutton.fxml")); fxmlloader.setroot(this); fxmlloader.setcontroller(this); try { fxmlloader.load(); } catch (ioexception exception) { throw new runtimeexception(exception); } } } next, the 2nd component is a tableview which contains a collection of beans, and displays their corresponding data. as with the last component, the first declaration is changed to next, the corresponding root and controller class is created for the table component. /* * to change this template, choose tools | templates * and open the template in the editor. */ package com.lynden.fx.test; import com.lynden.fx.inboundbean; import java.io.ioexception; import javafx.event.eventhandler; import javafx.event.eventtype; import javafx.fxml.fxml; import javafx.fxml.fxmlloader; import javafx.scene.control.tableview; import javafx.scene.input.clipboardcontent; import javafx.scene.input.dragboard; import javafx.scene.input.mouseevent; import javafx.scene.input.transfermode; import javafx.scene.layout.anchorpane; /** * * @author robt */ public class testtable extends anchorpane { @fxml private tableview mytableview; public testtable() { fxmlloader fxmlloader = new fxmlloader( getclass().getresource("/com/lynden/planning/ui/testtable.fxml")); fxmlloader.setroot(this); fxmlloader.setcontroller(this); try { fxmlloader.load(); } catch (ioexception exception) { throw new runtimeexception(exception); } } } finally, a new scene can be created which includes both the testtable and testbutton components that were constructed above. unfortunately, custom components can’t be added to the scene builder palette, so they must be inserted manually into the fxml file. you just need to ensure that you have the proper import statements defined, and the testtable and testbutton components can be inserted into the fxml code just as any native javafx component. when scene builder opens the fxml file you will see that both components are displayed on the new scene that’s it, hopefully this will help others who have been looking at adding custom components to their uis that are designed with scene builder. twitter: @robterp

Intro If you want to manage your SQL Databases in Azure using tools that you’re a little more familiar and comfortable with – for example – SQL Management Studio, how do you go about connecting? You could read the help article from Microsoft, or you can follow my intuitive screen-based instructions, below: Assumptions 1. I’m assuming you have a version of SQL Management Studio already installed. I believe you’ll need at least SQL Server 2008 R2’s version or newer 2. I’m further assuming you’ve already created a SQL Database in Azure Steps to Connect SSMS to SQL Azure 1. Authenticate to the Azure Portal 2. Click on SQL Databases 3. Click on Servers 4. Click on the name of the Server you wish to connect to… 5. Click on Configure… If not already in place, click on ‘Add to the allowed IP addresses’ to add your current IP address (or specify an address you wish to connect from) and click ‘Save’ 6. Open SQL Management Studio and connect to Database services (usually comes up by default) Enter the fully qualified server name (.database.windows.net) Change to SQL Server Authentication Enter the login preferred (if a new database, the username you specified when yuo created the DB server) Enter the correct password 7. Hit the Connect button Troubleshooting Ensure you have the appropriate ports open outbound from your local network or connection (typically port 1433) Ensure you have allowed the correct public IP address you’re trying to connect from via the Azure Portal (steps 1-5 above) Ensure you are using the correct server name and user name For SSMS, this is the server name (in step 4) followed by .database.windows.net Ensure you are using SQL Server Authentication For SSMS the username format is If you forgot the password of your username, you can reset the password in the Azure Portal, in step 4, click on Dashboard: Lastly… You can click on the Database (in step 2) to see your connection options:

Advantages coming from dependency injection can be addictive. It's a lot easier to configure application structure using injections than doing all resolutions manually. It's hard to resign from it when we have some non-managed classes that are instantiated outside of the container - for example being part of other frameworks like Vaadin UI components or JPA entities. The latter are especially important when we're using Domain Driven Design. During DDD training ran by Slawek Sobotka we were talking about options to remove "bad coupling" from aggregate factories. I'm sure you'll admit, that it's better to have generic mechanism able to "energize" objects by dependencies defined by e.g. @Inject annotation than inject all necessary dependencies into particular factory and then pass them into object by constructor, builder or simple setters. Spring framework brings us two different solutions to achieve such requirement. I'll now describe them both of them. Let's start with with simpler one. It's especially powerful when we've case such as generic repository, mentioned earlier aggregate factory, or even any other factory just ensuring that we have only few places in our code where will instantiate objects outside of the container. In this case we can make use of AutowireCapableBeanFactory class. In particular we will be interested in two methods: void autowireBean(Object existingBean) Object initializeBean(Object existingBean, String beanName) The first one just populates our bean without applying specific post processors (e.g. @PostConstruct, etc). The second one additionally applies factory callbacks like setBeanName and setBeanFactory, as well as any other post processors with @PostConstruct of course. In our code it'll look like this: public abstract class GenericFactory { @Autowired private AutowireCapableBeanFactory autowireBeanFactory; public T createBean() { // creation logic autowireBeanFactory.autowireBean(createdBean); return createdBean; } } Simple and powerful - my favorite composition :) But what can we do when we have plenty places in our code where objects get born? That's for example case of building Vaadin layouts in Spring web application. Injecting custom bean configurer objects invoking autowireBean method won't be the peak of productivity. Happily Spring developers brought us @Configurable annotation. This annotation connected with aspects will configure each annotated object even if we will create it outside of the container using new operator. Like with any other aspects we can choose between load-time-waving (LTW) compile-time-waving (CTW). The first one is easier to configure but we become (due to instrumentation) dependent from application server, which can be undesirable in some cases. To use it we need to annotate our @Configuration class by @EnableLoadTimeWeaving (or add tag if you like Flintstones and XML configuration ;)) After completing configuration just annotate class by @Configurable: @Configurable public class MyCustomButton extends Button { @Autowired private MyAwesomeService myAwesomeService; // handlers making use of injected service } The second option is a little bit more complex to setup but after this it will be a lot lighter in runtime. Because now we want to load aspects till compilation we have to integrate aspectj compiler into our build. In Maven you need to add few dependencies: org.aspectj aspectjrt 1.7.3 org.springframework spring-aspects 3.2.4.RELEASE org.springframework spring-tx 3.2.4.RELEASE javax.persistence persistence-api 1.0 provided I hope you are curious why above you can see persistence-api. That was also strange for me, when I saw "can't determine annotations of missing type javax.persistence.Entity" error during aspectj compilation. The answer can be found in SpringFramework JIRA in issue SPR-6819. This happens when you configure spring-aspects as a aspectLibrary in aspectj-maven-plugin. Issue is unresolved for over three year so better get used to it :) The last thing we need to do is to include above-mentioned plugin into our plugins section. org.codehaus.mojo aspectj-maven-plugin 1.5 1.7 1.7 1.7 true org.springframework spring-aspects compile And that's all folks :)

"this is my rifle. there are many like it, but this one is mine." - rifleman’s creed there are a thousand ways to grep over files. most developers i have observed keep a separate command line open just for searching. a few use an ide that has file search built in. personally, i use a couple of vim macros. in vim, you can execute a cross-file search with something like :vimgrep /dostuff()/j ../**/*.c . i don’t know about you, but the first time i saw that syntax my brain simply refused. instead, i have the following in my .vimrc file: " opens search results in a window w/ links and highlight the matches command! -nargs=+ grep execute 'silent grep! -i -r -n --exclude *.{json,pyc} . -e ' | copen | execute 'silent /' " shift-control-* greps for the word under the cursor :nmap g :grep =expand("") the first command is just a simple alias for the above mentioned native grep. like all custom commands, it must start with a capital letter (to differentiate it from native commands). you simply type :grep foobar and it will search in your current directory through all file extensions (except .json and .pyc -- you can add more to the blacklist). it also displays the results in a nice little buffer window, which you can navigate through with normal hjkl keys, and open matches in the main editor window. the second line is a key mapping that will grep for the word currently under the cursor. you can just navigate to a word and hit leader-g to issue the grep command.

I’ve been playing with Clojure over the last few weeks and as a result I’ve been using a lot of maps to represent the data. For example if we have the following map of teams to Glicko ratings and ratings deviations: (def teams { "Man. United" {:points 1500 :rd 350} "Man. City" {:points 1450 :rd 300} }) We might want to increase Man. United’s points score by one for which we could use the update-in function: > (update-in teams ["Man. United" :points] inc) {"Man. United" {:points 1501, :rd 350}, "Man. City" {:points 1450, :rd 300} The 2nd argument to update-in is a nested associative structure i.e. a sequence of keys into the map in this instance. If we wanted to reset Man. United’s points score we could use assoc-in: > (assoc-in teams ["Man. United" :points] 1) {"Man. United" {:points 1, :rd 350}, "Man. City" {:points 1450, :rd 300} If we want to update multiple keys at once then we can chain them using the -> (thread first) macro: (-> teams (assoc-in ["Man. United" :points] 1600) (assoc-in ["Man. United" :rd] 200)) {"Man. United" {:points 1600, :rd 200}, "Man. City" {:points 1450, :rd 300} If instead of replacing just one part of the value we want to replace the whole entry we could use associnstead: > (assoc teams "Man. United" {:points 1600 :rd 300}) {"Man. United" {:points 1600, :rd 300}, "Man. City" {:points 1450, :rd 300} assoc can also be used to add a new key/value to the map. e.g. > (assoc teams "Arsenal" {:points 1500 :rd 330}) {"Man. United" {:points 1500, :rd 350}, "Arsenal" {:points 1500, :rd 330}, "Man. City" {:points 1450, :rd 300} dissoc plays the opposite role and returns a new map without the specified keys: > (dissoc teams "Man. United" "Man. City") {} And those are all the map based functions I’ve played around with so far…

A PHP template engine is a way of outputting PHP in your HTML without using PHP syntax or PHP tags. It's supposed to be used by having a PHP class that will send your HTML the variables you want to display, and the HTML simply displays this data. This means that you are forced to separate your PHP logic from the HTML output, which is great; separation in your code is what you should be aiming for. The best way to understand what a template engine does it to see the code. Below is a basic page where you might want to display a title and a list of products. getAllProducts(); ?> title; ?> description; ?> The idea of using a template engine is that you would remove the $title variable and the $products variable and put this in a class, and then call a template file to display the HTML. getAllProducts(); $this->renderHtml('all-products.html', $vars); } } ?> The renderHtml() method will pass in the variables you want to display to the template file, which will look similar to this. {{ title } {% for product in products %} {{ product.title } {{ product.description } {% else %} No products exist. {% endfor %} As you can see, there is a separation of the PHP logic and display of code. Another reason why you might use a template engine is because now you have an HTML view file, which you can give to your front-end developer to style, and a template engine is arguably easier to read than using PHP syntax. What Frameworks Currently Use Template Engines? There are a few large frameworks that currently use a template engine. Some that I've used are Laravel, Drupal 8 and Expression Engine. All of these use different template engines but the understanding of how they work is similar. Laravel - Blade Templating Laravel uses a template engine called Blade. Here is an example of how you will use blade in your HTML views. @section('sidebar') This is the master sidebar. @show @yield('content') Output a Variable {{ $val } Foreach Loop @foreach ($users as $user) {{ $user->name } @endforeach If/else Statement @if ($user->name == 'Dave') Welcome Dave! @else Welcome Guest! @endif Blade Templating Drupal - Twig The Drupal CMS uses a template engine called Twig. This was developed to make templating quicker and more secure. It works in a similar way as blade but the syntax is slightly different. Output a Variable {{ var } For Loop {% for user in users %} * {{ user.name } {% else %} No user have been found. {% endfor %} Inherit Templates {% extends "layout.html" %} {% block content %} Content of the page... {% endblock %} Twig Smarty Another populate template engine is called Smarty. Again, Smarty was developed for code and HTML separation, and to make your templates easier to read. Output a Variable {$name} Foreach Loop {foreach $users as $user} {strip} {$user.name}{$user.phone} {/strip} {/foreach} If Statement {if $name == 'Fred'} Welcome Sir. {elseif $name == 'Wilma'} Welcome Ma'am. {else} Welcome, whatever you are. {/if} Smarty Using a Template Engine The three examples of template engines I gave above are not the only template engines available. There are many, and not just for PHP. Lots of other languages can use a template engine. Even though they are different engines, they are all similar in a way; they were all developed to make front-end templating quicker and easier to read. They all mention that this forces separation of your code and makes HTML easier to read, because you don't have any PHP tags in your HTML. But when I work with these different engines, I never enjoy developing with them. I actually find that they drastically slow down my development because I now have to look up the Smarty syntax or the Twig syntax for the thing I want to do. What they mention about code separation is exactly right; you shouldn't have logic in your HTML. However, you don't need a templating system to do this. In a previous tutorial I demonstrated how you can separate your HTML from your code with a simple method using output buffering. Separate HTML From Code So, while dealing with logic separation is up to the developer to do properly, the benefit of a template language is that it enforces it. If you're worried about separation you can use an MVC framework, or just create a method, and use the ob_buffer to get the HTML contents. If a developer is going to add log-in to an HTML file, they can't complain about separation and not being able to debug easily. I also don't agree with what's mentioned about the template engines being easier to read. Take a foreach in blade for an example of the difference: name; ?> @foreach ($users as $user) {{ $user->name } @endforeach There's not much difference between these code samples, except in Blade you use: {{ $user->name } In PHP, on the other hand, you use: name; ?> I even made the PHP do more than I have to. I could have simply used the printf() function to output the div, so it would look like this: %s', $user->name); } ?> Is it really so difficult to read that we have to learn a whole new template language to output variables in your HTML? I don't think so. Benefit of Using a Template Engine In my experience, there isn't a benefit to using a template engine over native PHP templating, but there must be some benefit, or there would be no reason to use it. So, I asked this question on Google+: what is the benefit or using a template engine? One of the answers I had directed me to this post on stackoverflow explaining some of the benefits of using a template engine. Some of the benefits mentioned are: New syntax Automatic escaping Template inheritance Ease of reading for non-developers Now, I would argue that learning a new syntax is not a benefit. Having to look up the correct syntax every time I need to do a foreach loop or an if statement is going to take up so much more time than simply using PHP. Automatic escaping is a real benefit. So many developers forget to escape when outputting content on the page, and this takes care of that for you. But it's really not that difficult to create a function that will escape the content of a variable. If you want some good examples, there's a formatting file (/wp-includes/formatting.php) in WordPress with loads of escaping functions you can use in different situations. Then you make sure you escape your content before sending it to the view. Template inheritance is another good benefit for some pages, and could be used for skinning different areas of the site. The ease of reading for non-developers is the one I disagree with most. PHP is not a hard language to read. It's not even a hard language to learn. If you have a front-end developer that is changing your HTML and can't understand what you're doing with the PHP, then I would recommend that they go and learn the basics of PHP. I'm not saying they need to learn how everything works and OOP principles, but just the basics so they can identify what an if statement is, what a for loop is, and so on. The native PHP syntax is not that different from the template engine syntax, so if they can learn that, then they most certainly can learn PHP syntax. For me, the only real benefit to using a template engine is security and autoescaping the output of the HTML. I'm not here to discourage people from using template engines; I can see they have their place, but like anything, you have to use them on the right projects. It depends on your team for the project, the size of the application, who's coding the HTML and whether the content author will want to change the HTML. In some cases, template engines might be perfect for your project. Here is a good article for reasons why you should be using template engines: Template Engines In PHP. In conclusion, I'm not really a fan of these template engines. For me, they just add another layer, another language you need to learn with little benefit. So, am I going to use these in my future projects? I'm still undecided. I can see they have a place and a reason for being used, but I think things take longer when developing with them, and they're not easy to debug.

Dependency is a feature in TestNG that allows a test method to depend on a single or a group of test methods. This will help in executing a set of tests to be executed before a test method. The dependency on multiple test methods is configured for a test by providing comma separated dependent test method names to the attribute dependsOnMethods while using the Test annotation. The following example shows a test class where process() test method depends on multiple test methods start() and initi() of the same class. Code ? package com.skilledmonster.example; import org.testng.annotations.Test; /** * Example to demonstrate TestNG multiple dependency method execution * * @author Jagadeesh Motamarri * @version 1.0 */ public class MultipleDependencyTest { @Test public void start() { System.out.println("Starting the server"); } @Test(dependsOnMethods = { "start" }) public void init() { System.out.println("Initializing the data for processing!"); } @Test(dependsOnMethods = { "start", "init" }) public void process() { System.out.println("Processing the data!"); } @Test(dependsOnMethods = { "process" }) public void stop() { System.out.println("Stopping the server"); } } Output As seen in the above console output, process() method executed after start() and init() methods are executed and like wise stop() method is executed after process() method is executed. Download [GitHub]

There are two widely opposed (and often misunderstood) positions on how expensive it can be to change or fix software once it has been designed, coded, tested and implemented. One holds that it is extremely expensive to leave changes until late, that the cost of change rises exponentially. The other position is that changes should be left as late as possible, because the cost of changing software is – or at least can be – essentially flat (that’s why we call it software). Which position is right? Why should we care? And what can we do about it? Exponential Cost of Change Back in the early 1980s, Barry Boehm published some statistics (Software Engineering Economics, 1981) which showed that the cost of making a software change or fix increases significantly over time – you can see the original curve that he published here. Boehm looked at data collected from Waterfall-based projects at TRW and IBM in the 1970s, and found that the cost of making a change increases as you move from the stages of requirements analysis to architecture, design, coding, testing and deployment. A requirements mistake found and corrected while you are still defining the requirements costs almost nothing. But if you wait until after you've finished designing, coding and testing the system and delivering it to the customer, it can cost up to 100 times as much. A few caveats here. First, the cost curve is much higher in large projects (in smaller projects, the cost curve is more like 1:4 instead of 1:100). Those cases when the cost of change rises up to 100 times are rare, what Boehm calls Architecture-Breakers, where the team gets a fundamental architectural assumption wrong (scaling, performance, reliability) and doesn't find out until after customers are already using the system and running into serious operational problems. This analysis was all done on a small data sample from more than 30 years ago, when developing code was much more expensive and time-consuming and paperworky, and the tools sucked. A few other studies have been done since then that mostly back up Boehm's findings – at least the basic idea that the longer it takes for you to find out that you made a mistake, the more expensive it is to correct it. These studies have been widely referenced in books like Steve McConnell’s Code Complete, and used to justify the importance of early reviews and testing: Studies over the last 25 years have proven conclusively that it pays to do things right the first time. Unnecessary changes are expensive. Researchers at Hewlett-Packard, IBM, Hughes Aircraft, TRW, and other organizations have found that purging an error by the beginning of construction allows rework to be done 10 to 100 times less expensively than when it's done in the last part of the process, during system test or after release (Fagan 1976; Humphrey, Snyder, and Willis 1991; Leffingwell 1997; Willis et al. 1998; Grady 1999; Shull et al. 2002; Boehm and Turner 2004). In general, the principle is to find an error as close as possible to the time at which it was introduced. The longer the defect stays in the software food chain, the more damage it causes further down the chain. Since requirements are done first, requirements defects have the potential to be in the system longer and to be more expensive. Defects inserted into the software upstream also tend to have broader effects than those inserted further downstream. That also makes early defects more expensive. There’s some controversy over how accurate and complete this data is, how much we can rely on it, and how relevant it is today when we have much better development tools and many teams have moved from heavyweight sequential Waterfall development to lightweight iterative, incremental development approaches. Flattening the Cost of Changing Code The rules of the game should change with iterative and incremental development – because they have to. Boehm realized back in the 1980s that we could catch more mistakes early (and therefore reduce the cost of development) if we think about risks upfront and design and build software in increments, using what he called the Spiral Model, rather than trying to define, design and build software in a Waterfall sequence. The same ideas are behind more modern, lighter Agile development approaches. In Extreme Programming Explained (the first edition, but not the second) Kent Beck states that minimizing the cost of change is one of the goals of Extreme Programming, and that a flattened change cost curve is “the technical premise of XP”: Under certain circumstances, the exponential rise in the cost of changing software over time can be flattened. If we can flatten the curve, old assumptions about the best way to develop software no longer hold … You would make big decisions as late in the process as possible, to defer the cost of making the decisions and to have the greatest possible chance that they would be right. You would only implement what you had to, in hopes that the needs you anticipate for tomorrow wouldn't come true. You would introduce elements to the design only as they simplified existing code or made writing the next bit of code simpler. It’s important to understand that Beck doesn't say that with XP the change curve is flat. He says that these costs can be flattened if teams work toward this, leveraging key practices and principles in XP, such as: Simple Design, doing the simplest thing that works, and deferring design decisions as late as possible (YAGNI), so that the design is easy to understand and easy to change Continuous, disciplined refactoring to keep the code easy to understand and easy to change Test-First Development – writing automated tests upfront to catch coding mistakes immediately, and to build up a testing safety net to catch mistakes in the future Developers collaborating closely and constantly with the customer to confirm their understanding of what they need to build and working together in pairs to design solutions and solve problems, and catch mistakes and misunderstandings early Relying on working software over documentation to minimize the amount of paperwork that needs to be done with each change (write code, not specs) The team’s experience working incrementally and iteratively – the more that people work and think this way, the better they will get at it. All of this makes sense and sounds right, although there are no studies that back up these assertions, which is why Beck dropped this change curve discussion from the second edition of his XP book. But, by then, the idea that change could be flat with Agile development had already become accepted by many people. The Importance of Feedback Scott Amber agrees that the cost curve can be flattened in Agile development, not because of Simple Design, but because of the feedback loops that are fundamental to iterative, incremental development. Agile methods optimize feedback within the team, developers working closely together with each other and with the customer and relying on continuous face-to-face communications. Following technical practices like test-first development, pair programming and continuous integration makes these feedback loops even tighter. But what really matters is getting feedback from the people using the system – it’s only then that you know if you got it right or what you missed. The longer that it takes to design and build something and get feedback from real users, the more time and work that is required to get working software into a real customer’s hands, the higher your cost of change really is. Optimizing and streamlining this feedback loop is what is driving the lean startup approach to development: defining a minimum viable product (something that just barely does the job), getting it out to customers as quickly as you can, and then responding to user feedback through continuous deployment and A/B testing techniques until you find out what customers really want. Even Flat Change Can Still Be Expensive Even if you do everything to optimize these feedback loops and minimize your overheads, this still doesn’t mean that change will come cheap. Being fast isn’t good enough if you make too many mistakes along the way. The Post Agilist uses the example of painting a house: Assume that it costs $1,000 each time you paint the house, whether you paint it blue, red or white. The cost of change is flat. But if you have to paint it blue first, then red, then white before everyone is happy, you’re wasting time and money. “No matter how expensive or cheap the "cost of change" curve may be, the fewer changes that are made, the cheaper and faster the result will be … Planning is not a four letter word.” (However, I would like to point out that “plan” is.) Spending too much time upfront in planning and design is waste. But not spending enough time upfront to find out what you should be building and how you should be building it before you build it, and not taking the care to build it carefully, is also a waste. Change Gets More Expensive Over Time You also have to accept that the incremental cost of change will go up over the life of a system, especially once a system is being used. This is not just a technical debt problem. The more people using the system, the more people who might be impacted by the change if you get it wrong, the more careful you have to be. This means that you need to spend more time on planning and communicating changes, building and testing a roll-back capability, and roll changes out slowly using canary releases and dark launching – which add costs and delays to getting feedback. There are also more operational dependencies that you have to understand and take care of, and more data that you have to change or fix up, making changes even more difficult and expensive. If you do things right, keep a good team together and manage technical debt responsibly, these costs should rise gently over the life of a system – and if you don’t, that exponential change curve will kick in. What is the real cost of change? Is the real cost of change exponential, or is it flat? The truth is somewhere in between. There’s no reason that the cost of making a change to software has to be as high as it was 30 years ago. We can definitely do better today, with better tools and better, cheaper ways of developing software. The keys to minimizing the costs of change seem to be: Get your software into customer hands as quickly as you can. I am not convinced that any organization really needs to push out software changes 10 to 50 to 100 times a day, but you don’t want to wait months or years for feedback, either. Deliver less, but more often. And because you’re going to deliver more often, it makes sense to build a continuous delivery pipeline so that you can push changes out efficiently and with confidence. Use ideas from lean software development and maybe Kanban to identify and eliminate waste and to minimize cycle time. We know that, even with lots of upfront planning and design thinking, we won’t get everything right upfront -- this is the Waterfall fallacy. But it’s also important not to waste time and money iterating when you don’t need to. Spending enough time upfront in understanding requirements and in design to get it at least mostly right the first time can save a lot later on. Whether you’re working incrementally and iteratively, or sequentially, it makes good sense to catch mistakes early when you can, whether you do this through test-first development and pairing, or requirements workshops and code reviews -- whatever works for you.

Program Statement How to generate Hailstone Sequence in Java? Solution The Hailstone sequence of numbers can be generated from a starting positive integer, n by: If n is 1 then the sequence ends. If n is even then the next n of the sequence = n/2 If n is odd then the next n of the sequence = (3 * n) + 1 Code ? package com.skilledmonster.examples.operations; import java.util.Scanner; /** * Program to generate Hailstone Sequence. * This program reads a number from the user then displays the Hailstone sequence * for that number followed by a line that shows the number of steps taken to reach 1. * * @author Jagadeesh Motamarri * @version 1.0 */ public class HailstoneSequenceGenerator { public static void main(String[] args) { Scanner inputScanner = new Scanner(System.in); System.out.printf("Enter a Number: "); try { int number = inputScanner.nextInt(); int steps = 0; while (number != 1) { if (number % 2 == 0) { System.out.println(number + " is even, so I take half: " + number / 2); number /= 2; } else { System.out.println(number + " is odd, so I make 3n + 1: " + (number * 3 + 1)); number = number * 3 + 1; } steps++; } System.out.println("The process took " + steps + (steps < 2 ? " step" : " steps") + " to reach 1"); } catch (Exception e) { System.out.println("Not a Number!! Run your Program again "); } } } Output As shown in the console output, for interger 17, it took 12 steps to reach 1 using Hailstone Sequence. References http://en.wikipedia.org/wiki/Collatz_conjecture

A few days back, I blogged on using SignalR and Angular JS together and on Implementing SignalR stock ticker sample using Angular JS(Part 1 and Part 2). In those posts, I have used the traditional call-back model to call the functions defined in controller to modify data whenever an update is received from the server. One of the readers sent me feedback saying that we have a better way to use SignalR and Angular JS together. The way to go is using event methods defined on $rootscope object. This approach is based on publishing and subscribing events. As events can be published from anywhere and subscribed from anywhere, the source and destination will remain completely unaware of each other. Both of them have to depend on just one object, $rootScope. Official documentation on scope contains details on each method defined on $rootScope. We will be using the following methods for publishing and subscribing the events: $emit(name, args): Publishes an event with specified name with given arguments $on(name, listener): Subscribes to an event with specified name. Listener is a function containing logic to be executed once the event has occurred To manage SignalR’s client functionality, it is better to create a service, as services are singletons. There will be only one instance of the service in entire application. This behaviour of services makes it possible to have multiple SignalR client pages in the applications and they can be kept in sync without putting any extra amount of effort. Let’s modify the example discussed in the post titled Hooking up ASP.NET SignalR with Angular JS to use event model. Server hub, references and structure of the HTML page remains the same as past. The only components to be modified are Controller and Service. Service carries the responsibility to initialize a connection to the hub and call the SignalR’s server methods. Once a response is received from the server, we will broadcast an event from the service with data received. app.service('signalRSvc', function ($, $rootScope) { var proxy = null; var initialize = function () { //Getting the connection object connection = $.hubConnection(); //Creating proxy this.proxy = connection.createHubProxy('helloWorldHub'); //Starting connection connection.start(); //Publishing an event when server pushes a greeting message this.proxy.on('acceptGreet', function (message) { $rootScope.$emit("acceptGreet",message); }); }; var sendRequest = function () { //Invoking greetAll method defined in hub this.proxy.invoke('greetAll'); }; return { initialize: initialize, sendRequest: sendRequest }; }); To keep the things simple, I kept names of the server hub event and event rose using $emit the same. The names can be different. Let’s modify the controller to have a listener to the event raised by the service. Following is the implementation of the controller: function SignalRAngularCtrl($scope, signalRSvc, $rootScope) { $scope.text = ""; $scope.greetAll = function () { signalRSvc.sendRequest(); } updateGreetingMessage = function (text) { $scope.text = text; } signalRSvc.initialize(); //Updating greeting message after receiving a message through the event $scope.$parent.$on("acceptGreet", function (e,message) { $scope.$apply(function () { updateGreetingMessage(message) }); }); } Now open the modified page on multiple browsers and click the Greeting button randomly from all browsers. Messages printed on all browsers should be updated whenever the button is clicked. This behaviour is same as it was earlier. We just adopted a better approach to make it work. Happy coding!

In Java 5.0, a new addition called Reentrant Lock was made to enhance intrinsic locking capabilities. Prior to this, "synchronized" and "volatile" were the means for achieving concurrency. public synchronized void doAtomicTransfer(){ //enter synchronized block , acquire lock over this object. operation1() operation2(); } // exiting synchronized block, release lock over this object. Synchronized uses intrinsic locks or monitors. Every object in Java has an intrinsic lock associated with it. Whenever a thread tries to access a synchronized block or method, it acquires the intrinsic lock or the monitor on that object. In case of static methods, the thread acquires the lock over the class object. An intrinsic locking mechanism is a clean approach in terms of writing code, and is pretty good for most of the use-cases. So why do we need the additional feature of explicit locks? Let's discuss. An intrinsic locking mechanism can have some functional limitations, such as: 1.) It is not possible to interrupt a thread waiting to acquire a lock (lock Interruptibly). 2.) It is not possible to attempt to acquire a lock without being willing to wait for it forever (try lock). 3.) Cannot implement non-block-structured locking disciplines, as intrinsic locks must be released in the same block in which they are acquired. Aside from that, ReentrantLock supports lock polling, and interruptible lock waits that support time-out. ReentrantLock also has support for configurable fairness policy, allowing more flexible thread scheduling. Source: Stack Overflow Lets see a few of the methods implemented by ReentrantLock class (which implements Lock): void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit) throws InterruptedException; ..... Lets try and understand the use of these and see what benefits we can get. 1.) Polled and Timed Lock Acquisition Let's see some example code: public void transferMoneyWithSync(Account fromAccount, Account toAccount, float amount) throws InsufficientAmountException { synchronized (fromAccount) { // acquired lock on fromAccount Object synchronized (toAccount) { // acquired lock on toAccount Object if (amount > fromAccount.getCurrentAmount()) { throw new InsufficientAmountException( "Insufficient Balance"); } else { fromAccount.debit(amount); toAccount.credit(amount); } } } } In the transferMoney() method above, there is a possibility of deadlock when two threads A and B are trying to transfer money at almost the same time. A: transferMoney(acc1, acc2, 20); B: transferMoney(acc2, acc1 ,25); It is possible that thread A has acquired a lock on the acc1 object and is waiting to acquire a lock on the acc2 object. Meanwhile, thread B has acquired a lock on the acc2 object and is waiting for a lock on acc1. This will lead to deadlock, and the system would have to be restarted! There is, however, a way to avoid this, which is called "lock ordering." Personally, I find this a bit complex. A cleaner approach is implemented by ReentrantLock with the use of tryLock() method. This approach is called the "timed and polled lock-acquisition." It lets you regain control if you cannot acquire all the required locks, release the ones you have acquired and retry. So, using tryLock, we will attempt to acquire both locks. If we cannot attain both, we will release if one of these has been acquired, then retry. public boolean transferMoneyWithTryLock(Account fromAccount, Account toAccount, float amount) throws InsufficientAmountException, InterruptedException { // we are defining a stopTime long stopTime = System.nanoTime() + 5000; while (true) { if (fromAccount.lock.tryLock()) { try { if (toAccount.lock.tryLock()) { try { if (amount > fromAccount.getCurrentAmount()) { throw new InsufficientAmountException( "Insufficient Balance"); } else { fromAccount.debit(amount); toAccount.credit(amount); } } finally { toAccount.lock.unlock(); } } } finally { fromAccount.lock.unlock(); } } if(System.nanoTime() < stopTime) return false; Thread.sleep(100); }//while } Here we implemented a timed lock, so if the locks cannot be acquired within the specified time, the transferMoney method will return a failure notice and exit gracefully. We can also maintain time budget activities using this concept. 2.) Interruptible Lock Acquisition Interruptible lock acquisition allows locking to be used within cancellable activities. The lockInterruptibly method allows us to try and acquire a lock while being available for interruption. So, basically it allows the thread to immediately react to the interrupt signal sent to it from another thread. This can be helpful when we want to send a KILL signal to all the waiting locks. Let's see one example: Suppose we have a shared line to send messages. We would want to design it in such a way that if another thread comes and interrupts the current thread, the lock should release and perform the exit or shut down operations to cancel the current task. public boolean sendOnSharedLine(String message) throws InterruptedException{ lock.lockInterruptibly(); try{ return cancellableSendOnSharedLine(message); } finally { lock.unlock(); } } private boolean cancellableSendOnSharedLine(String message){ ....... The timed tryLock is also responsive to interruption. 3.) Non-block Structured Locking: In intrinsic locks, acquire-release pairs are block-structured. In other words, a lock is always released in the same basic block in which it was acquired, regardless of how control exits the block. Extrinsic locks allow the facility to have more explicit control. Some concepts, like Lock Strapping, can be achieved more easily using extrinsic locks. Some use cases are seen in hash-bashed collections and linked lists. 4.) Fairness: The ReentrantLock constructor offers a choice of two fairness options: create a non-fair lock or a fair lock. With fair locking, threads can acquire locks only in the order in which they were requested, whereas an unfair lock allows a lock to acquire it out of its turn. This is called barging (breaking the queue and acquiring the lock when it became available). Fair locking has a significant performance cost because of the overhead of suspending and resuming threads. There could be cases where there is a significant delay between when a suspended thread is resumed and when it actually runs. Let's see a situation: A -> holds a lock. B -> has requested and is in a suspended state waiting for A to release the lock. C -> requests the lock at the same time that A releases the lock, and has not yet gone to a suspended state. As C has not yet gone to a suspended state, there is a chance that it can acquire the lock released by A, use it, and release it before B even finishes waking up. So, in this context, unfair lock has a significant performance advantage. Intrinsic locks and extrinsic locks have the same mechanism inside for locking, so the performance improvement is purely subjective. It depends on the use cases we discussed above. Extrinsic locks give a more explicit control mechanism for better handling of deadlocks, starvation, and so on. In future articles, I will cover more use cases to exhibit extrinsic locks.

The following are top 10 methods for Java Array. They are the most voted questions from stackoverflow. 0. Decalre an array String[] aArray = new String[5]; String[] bArray = {"a","b","c", "d", "e"}; String[] cArray = new String[]{"a","b","c","d","e"}; 1. Print an array in Java int[] intArray = { 1, 2, 3, 4, 5 }; String intArrayString = Arrays.toString(intArray); // print directly will print reference value System.out.println(intArray); // [I@7150bd4d System.out.println(intArrayString); // [1, 2, 3, 4, 5] 2. Create ArrayList from array String[] stringArray = { "a", "b", "c", "d", "e" }; ArrayList arrayList = new ArrayList(Arrays.asList(stringArray)); System.out.println(arrayList); // [a, b, c, d, e] 3. Check if an array contains a certain value String[] stringArray = { "a", "b", "c", "d", "e" }; boolean b = Arrays.asList(stringArray).contains("a"); System.out.println(b); // true 4. Concatenate two arrays int[] intArray = { 1, 2, 3, 4, 5 }; int[] intArray2 = { 6, 7, 8, 9, 10 }; // Apache Commons Lang library int[] combinedIntArray = ArrayUtils.addAll(intArray, intArray2); 5. Declare array inline method(new String[]{"a", "b", "c", "d", "e"}); 6. Joins the elements of the provided array into a single String // containing the provided list of elements // Apache common lang String j = StringUtils.join(new String[] { "a", "b", "c" }, ", "); System.out.println(j); // a, b, c 7. Covnert ArrayList to Array String[] stringArray = { "a", "b", "c", "d", "e" }; ArrayList arrayList = new ArrayList(Arrays.asList(stringArray)); String[] stringArr = new String[arrayList.size()]; arrayList.toArray(stringArr); for (String s : stringArr) System.out.println(s); 8. Convert Array to Set Set set = new HashSet(Arrays.asList(stringArray)); System.out.println(set); //[d, e, b, c, a] 9. Reverse an array int[] intArray = { 1, 2, 3, 4, 5 }; ArrayUtils.reverse(intArray); System.out.println(Arrays.toString(intArray)); //[5, 4, 3, 2, 1] 10. Remove element of an array int[] intArray = { 1, 2, 3, 4, 5 }; int[] removed = ArrayUtils.removeElement(intArray, 3);//create a new array System.out.println(Arrays.toString(removed)); One more – convert int to byte array byte[] bytes = ByteBuffer.allocate(4).putInt(8).array(); for (byte t : bytes) { System.out.format("0x%x ", t); } In addition, do you know what arrays look like in memory ?

Learn about deploying and debugging Visual Studio packages.

About a year ago, we published in our blog a series of articles on the development of Visual Studio plugins in C#. We have recently revised those materials and added new sections and now invite you to have a look at the updated version of the manual as a series of articles here on DZone. The other articles in the series can be found here: Part 1 - How to Create, Debug and Deploy Visual Studio Extension Packages Part 2 - Using the Visual Studio Automation Object Model This article (the second in the series) contains an overview of the Visual Studio Automation Object Model. In it, we examine the model's overall structure and the means of obtaining access to its interfaces through DTE/DTE2 top-level objects. Several examples of utilizing elements of the model are provided. Also discussed are the issues of using the model's interfaces within multithreaded applications; an example of implementing such a mechanism for multithreaded interaction with COM interfaces in managed code is provided as well. Introduction The Visual Studio development environment is built upon the principles of automation and extensibility, providing developers with the ability to integrate almost any custom element into the IDE and allowing for an easy interaction with its default and user-created components. As the means of implementing these tasks, Visual Studio users are provided with several cross-complementing toolsets; the most basic and versatile among these is the Visual Studio Automation Object Model. The Automation Object Model is represented by a series of libraries containing a vast and well-structured API set that covers all aspects of IDE automation and the majority of its extensibility capabilities. Although, in comparison to other IDE extensibility tools, this model does not provide access to some portions of Visual Studio (this applies mostly to the extension of some IDE's features), it is nonetheless the most flexible and versatile among them. The majority of the model's interfaces are accessible from within every type of IDE extension module, which allows interacting with the environment even from an external independent process. Moreover, the model itself could be extended along with the extension of Visual Studio IDE, providing other third-party developers with an access to user-created custom components. Automation Object Model Structure The Visual Studio automation model is composed of several interconnected functional object groups covering all aspects of the development environment; it also provides capabilities for controlling and extending these groups. Accessing any of them is possible through the top-level global DTE interface (Development Tools Environment). Figure 1 shows the overall structure of the automation model and how it is divided among functionality groups. Figure 1 — Visual Studio Automation Object Model A user could extend the model in one of the following groups: Project models (implementing new project types, support for new languages); Document models (implementing new document types and document editors) Code editor level models (support for specific language constructs) Project build-level models An automation model could be extended from plug-ins of VSPackage type only. All of the automation model's interfaces could be conventionally subdivided into two large groups. The first group consists of the interfaces of the EnvDTE and Visual Studio Interop namespaces. These interfaces allow interactions with basic common components of the IDE itself, such as tool windows, editors, event handling services and so on. The second group consists of the interfaces of the specific project model. The figure above specifies this interface group as late-bound properties, i.e. these interfaces are implemented in a separate dynamically loaded library. Each standard (i.e. the one that is included in a regular Visual Studio distribution) project model, such as Visual C++ or Visual Basic, provides a separate implementation for these interfaces. Third-party developers are able to extend the automation model by adding their own custom project models and by providing an implementation of these automation interfaces. Also worth noting is that the interfaces of the 1st group, which was specified above, are universal, meaning that they could be utilized for interaction with any of the project models or Visual Studio editions, including the integrated\isolated Visual Studio shells. In this article, we will examine this group in more detail. But still, despite the model's versatility, not every group belonging to the model could be equally utilized from all the types of IDE extensions. For instance, some of the model's capabilities are inaccessible to external processes; these capabilities are tied to specific extension types, such as add-in or VSPackage. Therefore, when selecting the type for the extension to be developed, it is important to consider the functionality that this extension will require. The Microsoft.VisualStudio.Shell.Interop namespace also provides a group of COM interfaces, which can be used to extend and automate Visual Studio application from managed code. Managed Package Framework (MPF) classes, which we utilized earlier for creating a VSPackage plugin, are actually themselves based on these interfaces. Although theses interfaces are not a part of EnvDTE automation model described above, nevertheless they greatly enhance this model by providing additional functionality for VSPackage extensions, which is otherwise unavailable for extensions of other types. Obtaining References to DTE/DTE2 Objects In order to create a Visual Studio automation application, it is necessary to obtain access to the automation objects themselves in the first place. To accomplish this, first of all it is necessary to hook up the correct versions of libraries containing the required managed API wrappers in the EnvDTE namespace. Secondly, the reference to the automation model top-level object, that is the DTE2 interface, should be obtained. In the course of Visual Studio’s evolution, several of its automation objects had been modified or received some additional functionality. So, to maintain a backward compatibility with existing extension packages, new EnvDTE80, EnvDTE90, EnvDTE100 etc. namespaces were created instead of updating the interfaces from the original EnvDTE namespace. The majority of such updated interfaces from these new namespaces do maintain the same names as in the original ones, but with the addition of an ordinal number at the end of the name, for example Solution and Solution2. It is advised that these updated interfaces should be utilized when creating a new project, as they do contain the most recent functionality. It's worth noting that properties and methods of DTE2 interface usually return object references with types corresponding to the original DTE. For example, accessing dte2.Solution will return Solution and not the Solution2 as it would seem. Although these new EnvDTE80, EnvDTE90, EnvDTE100 namespaces do contain some of the updated functionality as mentioned above, still it is the EnvDTE interface that contains the majority of automation objects. Therefore, in order to possess access to all of the existing interfaces, it is necessary to link all versions of the managed COM wrapper libraries to the project, as well as to obtain the references to DTE and also to DTE2. The way of obtaining a top-level EnvDTE object reference is dependent upon the type of IDE extension being developed. Let's examine three such extension types: add-in, VSPackage and an MSVS-independent external process. Add-in Extension In the case of an add-in extension, access to the DTE interface can be obtained inside the OnConnection method, which should be implemented for the IDTExtensibility interface that provides access to the extension-environment interaction events. The OnConnection method is called at the moment when the module is loaded by the IDE. It can happen either when the environment is being loaded itself or after the extension was called for the first time in the IDE session. The example of obtaining the reference follows: public void OnConnection(object application, ext_ConnectMode connectMode, object addInInst, ref Array custom) { _dte2 = (DTE2)application; ... } An add-in module can be initialized either at the moment of IDE start-up, or when it is called for the first time in the current IDE session. So, the connectMode can be used to correctly determine the moment of initialization inside the OnConnection method. switch(connectMode) { case ext_ConnectMode.ext_cm_UISetup: ... break; case ext_ConnectMode.ext_cm_Startup: ... break; case ext_ConnectMode.ext_cm_AfterStartup: ... break; case ext_ConnectMode.ext_cm_CommandLine: ... break; } As in the example above, add-in could be loaded either simultaneously with the IDE itself (if the startup option in the add-in manager is checked), when it is called the first time or when it is called through the command line. The ext_ConnectMode.ext_cm_UISetup option is invoked only for a single time in the plug-in's overall lifetime, which is during its first initialization. This case should be used for initializing user UI elements that are to be integrated into the environment (more on this later on). If an add-in is being loaded during Visual Studio start-up (ext_ConnectMode.ext_cm_Startup), then at the moment OnConnect method receives control for the first time, it is possible that the IDE still is not fully initialized itself. In such a case, it is advised to postpone the acquisition of the DTE reference until the environment is fully loaded. The OnStartupComplete handler provided by the IDTExtensibility can be used for this. public void OnStartupComplete(ref Array custom) { ... } VSPackage Extension For VSPackage type of extension, the DTE could be obtained through the global Visual Studio service with the help of GetService method of a Package subclass: DTE dte = MyPackage.GetService(typeof(DTE)) as DTE; Please note that the GetService method could potentially return null in case Visual Studio is not fully loaded or initialized at the moment of such access, i.e. it is in the so called "zombie" state. To correctly handle this situation, it is advised that the acquisition of DTE reference should be postponed until this interface is inquired. But in case the DTE reference is required inside the Initialize method itself, the IVsShellPropertyEvents interface can be utilized (also by deriving our Package subclass from it) and then the reference could be safely obtained inside the OnShellPropertyChange handler. DTE dte; uint cookie; protected override void Initialize() { base.Initialize(); IVsShell shellService = GetService(typeof(SVsShell)) as IVsShell; if (shellService != null) ErrorHandler.ThrowOnFailure( shellService.AdviseShellPropertyChanges(this,out cookie)); ... } public int OnShellPropertyChange(int propid, object var) { // when zombie state changes to false, finish package initialization if ((int)__VSSPROPID.VSSPROPID_Zombie == propid) { if ((bool)var == false) { this.dte = GetService(typeof(SDTE)) as DTE; IVsShell shellService = GetService(typeof(SVsShell)) as IVsShell; if (shellService != null) ErrorHandler.ThrowOnFailure( shellService.UnadviseShellPropertyChanges(this.cookie) ); this.cookie = 0; } } return VSConstants.S_OK; } It should be noted that the process of VSPackage module initialization at IDE startup could vary for different Visual Studio versions. For instance, in case of VS2005 and VS2008, an attempt at accessing DTE during IDE startup will almost always result in null being returned, owning to the relative fast loading times of these versions. But, one does not simply obtain access into DTE. In Visual Studio 2010 case, it mistakenly appears that one could simply obtain an access to the DTE from inside the Initialize() method. In fact, this impression is a false one, as such a method of DTE acquisition could potentially cause the occasional appearance of "floating" errors which are hard to identify and debug, and even the DTE itself may be still uninitialized when the reference is acquired. Because of these disparities, the aforementioned acquisition method for handling IDE loading states should not be ignored on any version of Visual Studio. Independent External Process The DTE interface is a top-level abstraction for Visual Studio environment in the automation model. In order to acquire a reference to this interface from an external application, its ProgID COM identifier could be utilized; for instance, it will be "VisualStudio.DTE.10.0" for Visual Studio 2010. Consider this example of initializing a new IDE instance and when obtaining a reference to the DTE interface. // Get the ProgID for DTE 8.0. System.Type t = System.Type.GetTypeFromProgID( "VisualStudio.DTE.10.0", true); // Create a new instance of the IDE. object obj = System.Activator.CreateInstance(t, true); // Cast the instance to DTE2 and assign to variable dte. EnvDTE80.DTE2 dte = (EnvDTE80.DTE2)obj; // Show IDE Main Window dte.MainWindow.Activate(); In the example above, we've actually created a new DTE object, starting deven.exe process by the CreateInstance method. But at the same time, the GUI window of the environment will be displayed only after the Activate method is called. Next, let's review a simple example of obtaining the DTE reference from an already running Visual Studio Instance: EnvDTE80.DTE2 dte2; dte2 = (EnvDTE80.DTE2) System.Runtime.InteropServices.Marshal.GetActiveObject( "VisualStudio.DTE.10.0"); However, in case several instances of the Visual Studio are executing at the moment of our inquiry, the GetActiveObject method will return a reference to the IDE instance that was started the earliest. Let's examine a possible way of obtaining the reference to DTE from a running Visual Studio instance by the PID of its process. using EnvDTE80; using System.Diagnostics; using System.Runtime.InteropServices; using System.Runtime.InteropServices.ComTypes; [DllImport("ole32.dll")] private static extern void CreateBindCtx(int reserved, out IBindCtx ppbc); [DllImport("ole32.dll")] private static extern void GetRunningObjectTable(int reserved, out IRunningObjectTable prot); public static DTE2 GetByID(int ID) { //rot entry for visual studio running under current process. string rotEntry = String.Format("!VisualStudio.DTE.10.0:{0}", ID); IRunningObjectTable rot; GetRunningObjectTable(0, out rot); IEnumMoniker enumMoniker; rot.EnumRunning(out enumMoniker); enumMoniker.Reset(); IntPtr fetched = IntPtr.Zero; IMoniker[] moniker = new IMoniker[1]; while (enumMoniker.Next(1, moniker, fetched) == 0) { IBindCtx bindCtx; CreateBindCtx(0, out bindCtx); string displayName; moniker[0].GetDisplayName(bindCtx, null, out displayName); if (displayName == rotEntry) { object comObject; rot.GetObject(moniker[0], out comObject); return (EnvDTE80.DTE2)comObject; } } return null; } Here we've acquired the DTE interface by identifying the required instance of the IDE in the table of running COM objects (ROT, Running Object Table) by its process identifier. Now we can access the DTE for every of the executing instances of Visual Studio, for example: Process Devenv; ... //Get DTE by Process ID EnvDTE80.DTE2 dte2 = GetByID(Devenv.Id); Additionally, to acquire any project-specific interface (including custom model extensions), for example the CSharpProjects model, through a valid DTE interface, the GetObject method should be utilized: Projects projects = (Projects)dte.GetObject("CSharpProjects"); The GetObject method will return a Projects collection of regular Project objects, and each one of them will contain a reference to our project-specific properties, among other regular ones. Visual Studio Text Editor Documents An automation model represents Visual Studio text documents through the TextDocument interface. For example, C/C++ source code files are opened by the environment as text documents. The TextDocument is based upon the common automation model document interface (the Document interface), which represents files of any type opened in Visual Studio editor or designer. A reference to the text document object can be obtained through the 'Object' field of the Document object. Let's acquire a text document for the currently active (i.e. the one possessing focus) document from IDE's text editor: EnvDTE.TextDocument objTextDoc = (TextDocument)PVSStudio.DTE.ActiveDocument.Object("TextDocument"); Modifying Documents The TextSelection document allows controlling the text selection or to modify it. The methods of this interface represent the functionality of Visual Studio text editor, i.e. they allow the interaction with the text as it is presented directly by the UI. EnvDTE.TextDocument Doc = (TextDocument)PVSStudio.DTE.ActiveDocument.Object(string.Empty); Doc.Selection.SelectLine(); TextSelection Sel = Doc.Selection; int CurLine = Sel.TopPoint.Line; String Text = Sel.Text; Sel.Insert("test\r\n"); In this example, we selected a text line under the cursor, read the selected text and replaced it with a 'test' string. The TextDocument interface also allows text modification through the EditPoint interface. This interface is somewhat similar to the TextSelection, but instead of operating with the text through the editor UI, it directly manipulates text buffer data. The difference between them is that the text buffer is not influenced by such editor-specific notions as WordWrap and Virtual Spaces. It should be noted that both of these editing methods are not able to modify read-only text blocks. Let's examine the example of modifying text with EditPoint by placing additional lines at the end of current line with a cursor: objEditPt = objTextDoc.StartPoint.CreateEditPoint(); int lineNumber = objTextDoc.Selection.CurrentLine; objEditPt.LineDown(lineNumber - 1); EditPoint objEditPt2 = objTextDoc.StartPoint.CreateEditPoint(); objEditPt2.LineDown(lineNumber - 1); objEditPt2.CharRight(objEditPt2.LineLength); String line = objEditPt.GetText(objEditPt.LineLength); String newLine = line + "test"; objEditPt.ReplaceText(objEditPt2, newLine, (int)vsEPReplaceTextOptions.vsEPReplaceTextKeepMarkers); Navigating the Documents VSPackage modules are able to obtain access to a series of global services that could be used for opening and handling environment documents. These services could be acquired by the Package.GetGlobalService() method from Managed Package Framework. It should be noted that the services described here are not part of the EnvDTE model and are accessible only from a Package-type extension, and therefore they could not be utilized in other types of Visual Studio extensions. Nonetheless, they can be quite useful for handling IDE documents when they are utilized in addition to the Documents interface described earlier. Next, we'll examine these services in more detail. The IVsUIShellOpenDocument interface controls the state of documents opened in the environment. Following is the example that uses this interface to open a document through path to a file, which this document will represent: String path = "C:\Test\test.cpp"; IVsUIShellOpenDocument openDoc = Package.GetGlobalService(typeof(IVsUIShellOpenDocument)) as IVsUIShellOpenDocument; IVsWindowFrame frame; Microsoft.VisualStudio.OLE.Interop.IServiceProvider sp; IVsUIHierarchy hier; uint itemid; Guid logicalView = VSConstants.LOGVIEWID_Code; if (ErrorHandler.Failed( openDoc.OpenDocumentViaProject(path, ref logicalView, out sp, out hier, out itemid, out frame)) || frame == null) { return; } object docData; frame.GetProperty((int)__VSFPROPID.VSFPROPID_DocData, out docData); The file will be opened in a new editor or will receive focus in case it already has been opened earlier. Next, let's read a VsTextBuffer text buffer from this document we opened: // Get the VsTextBuffer VsTextBuffer buffer = docData as VsTextBuffer; if (buffer == null) { IVsTextBufferProvider bufferProvider = docData as IVsTextBufferProvider; if (bufferProvider != null) { IVsTextLines lines; ErrorHandler.ThrowOnFailure(bufferProvider.GetTextBuffer( out lines)); buffer = lines as VsTextBuffer; Debug.Assert(buffer != null, "IVsTextLines does not implement IVsTextBuffer"); if (buffer == null) { return; } } } The IVsTextManager interface controls all of the active text buffers in the environment. For example, we can navigate a text document using the NavigateToLineAndColumn method of this manager on a buffer we've acquired earlier: IVsTextManager mgr = Package.GetGlobalService(typeof(VsTextManagerClass)) as IVsTextManager; mgr.NavigateToLineAndColumn(buffer, ref logicalView, line, column, line, column); Subscribing and Handling Events Automation objects events are represented by the DTE.Events property. This element references all of the common IDE events (such as CommandEvents, SolutionEvents), as well as the events of separate environment components (project types, editors, tools etc.), also including the ones designed by third-party developers. To acquire a reference for this automation object, the GetObject method could be utilized. When subscribing to the DTE events, one should remember that this interface could be still unavailable at the moment of extension being initialized. So it is always important to consider the sequence of your extension initialization process if the access to DTE.Events is required in the Initialize() method of your extension package. The correct handling of initialization sequence will vary for different extension types, as it was described earlier. Let's acquire a reference for an events object of Visual C++ project model defined by the VCProjectEngineEvents interface and assign a handler for the removal of an element from the Solution Explorer tree: VCProjectEngineEvents m_ProjectItemsEvents = PVSStudio.DTE.Events.GetObject("VCProjectEngineEventsObject") as VCProjectEngineEvents; m_ProjectItemsEvents.ItemRemoved += new _dispVCProjectEngineEvents_ItemRemovedEventHandler( m_ProjectItemsEvents_ItemRemoved); MDI Windows Events The Events.WindowEvents property could be utilized to handle regular events of an environment MDI window. This interface permits the assignment of a separate handler for a single window (defined through the EnvDTE.Window interface) or the assignment of a common handler for all of the environment's windows. The following example contains the assignment of a handler for the event of switching between IDE windows: WindowEvents WE = PVSStudio.DTE.Events.WindowEvents; WE.WindowActivated += new _dispWindowEvents_WindowActivatedEventHandler( Package.WE_WindowActivated); The next example is the assignment of a handler for window switching to the currently active MDI window through the WindowEvents indexer: WindowEvents WE = m_dte.Events.WindowEvents[MyPackage.DTE.ActiveWindow]; WE.WindowActivated += new _dispWindowEvents_WindowActivatedEventHandler( MyPackage.WE_WindowActivated); IDE Commands Events The actual handling of the environment's commands and their extension through the automation model is covered in a separate article of this series. In this section, we will examine the handling of the events related to these commands (and not of the execution of the commands themselves). Assigning the handlers to these events is possible through the Events.CommandEvents interface. The CommandEvents property, as in the case of MDI windows events, also permits the assignment of a handler either for all of the commands or for a single one through the indexer. Let's examine the assignment of a handler for the event of a command execution being complete (i.e. when the command finishes its execution): CommandEvents CEvents = DTE.Events.CommandEvents; CEvents.AfterExecute += new _dispCommandEvents_AfterExecuteEventHandler(C_AfterExecute); But in order to assign such a handler for an individual command, it is necessary to identify this command in the first place. Each command of the environment is identified by a pair of GUID:ID, and in case of user-created commands, these values are specified directly by the developer during their integration, for example through the VSCT table. Visual Studio possesses a special debug mode that allows identifying any of the environment's commands. To activate this mode, it is required that the following key is to be added to the system registry (an example for Visual Studio 2010): [HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\10.0\General] "EnableVSIPLogging"=dword:00000001 Now, after restarting the IDE, hovering your mouse over the menu or toolbar elements with CTRL+SHIFT being simultaneously pressed (though sometime it will not work until you left-click it) will display a dialog window containing all of the command's internal identifiers. We are interested in the values of Guid and CmdID. Let's examine the handling of events for the File.NewFile command: CommandEvents CEvents = DTE.Events.CommandEvents[ "{5EFC7975-14BC-11CF-9B2B-00AA00573819}", 221]; CEvents.AfterExecute += new _dispCommandEvents_AfterExecuteEventHandler(C_AfterExecute); The handler obtained in this way will receive control only after the command execution is finished. void C_AfterExecute(string Guid, int ID, object CustomIn, object CustomOut) { ... } This handler should not be confused with an immediate handler for the execution of the command itself which could be assigned during this command's initialization (from an extension package and in case the command is user-created). Handling the IDE commands is described in a separate article that is entirely devoted to IDE commands. In concluding this section, it should be mentioned that in the process of developing our own VSPackage extension, we've encountered the necessity to store the references to interface objects containing our handler delegates (such as CommandEvents, WindowEvents etc.) on the top-level fields of our main Package subclass. The reason for this is that in the case of the handler being assigned through a function-level local variable, it is lost immediately after leaving the method. Such behavior could probably be attributed to the .NET garbage collector, although we've obtained these references from the DTE interface that definitely exists during the entire lifetime of our extension package. Handling Project and Solution Events for VSPackage Extensions Let's examine some of the interfaces from the Microsoft.VisualStudio.Shell.Interop namespace -- the ones that permit us to handle the events related to Visual Studio projects and solutions to be more precise. Although these interfaces are not a part of the EnvDTE automation model, they could be implemented by the main class of the VSPackage extension (that is the class that was inherited from the Package base class of Managed Package Framework). That is why, if you are developing the extension of this type, these interfaces conveniently supplement the basic set of interfaces provided by the DTE object. By the way, this is another argument for creating a full-fledged VSPackage plugin using MPF. The IVsSolutionEvents could be implemented by the class inherited from Package, and it is available starting from Visual Studio version 2005, and the isolated\integrated shells based applications. This interface permits you to track the loading, unloading, opening and closing of projects or even the whole solutions in the development environment by implementing such of its methods as OnAfterCloseSolution, OnBeforeCloseProject and OnQueryCloseSolution. For example: public int OnAfterLoadProject(IVsHierarchy pStubHierarchy, IVsHierarchy pRealHierarchy) { //your custom handler code return VSConstants.S_OK; } As you can see, this method takes the IVsHierarchy object as an input parameter that represents the loading project. Managing such objects will be examined in another article devoted to the interaction with the Visual Studio project model. The IVsSolutionLoadEvents interface, in a similar fashion to the interface described above, should be implemented by the Package subclass and is available to versions of Visual Studio starting from 2010 and above. This interface allows you to handle such interesting aspects as batch loading of project groups and background solution loadings (the OnBeforeLoadProjectBatch and OnBeforeBackgroundSolutionLoadBegins methods), and also to intercept the end of this background loading operation as well (the OnAfterBackgroundSolutionLoadComplete method). Such event handlers should come in handy in case your plug-in needs to execute some code immediately after its initialization, and, at the same time, the plug-in depends on projects\solutions that are loaded inside the IDE. In this case, executing such a code without waiting for the solution loading to be finished could lead to either incorrect (incomplete) results because of the incompletely formed projects tree, or even to runtime exceptions. While developing the PVS-Studio IDE plug-in, we've encountered another interesting aspect of VSPackage plug-in initialization. When one Package plug-in enters a waiting state (for instance, by displaying a dialog window to the user), further initialization of VSPackage extensions is suspended until the blocking plug-in returns. So, when handling loading and initialization inside the environment, one should always remember this possible scenario as well. And finally, I want to return one final time to the fact that for the interface methods described above to operate correctly, you should inherit your main class from theses interfaces: class MyPackage: Package, IVsSolutionLoadEvents, IVsSolutionEvents { //Implementation of Package, IVsSolutionLoadEvents, IVsSolutionEvents ... } Supporting Visual Studio Color Schemes If the extension you are developing will be integrated into the interface of the development environment, for instance, by creating custom tool windows or document MDI windows (and the most convenient way for such an integration is a VSPackage extesnion), it is advisable that the coloring of your custom UI components should match the common color scheme used by Visual Studio itself. The importance of this task was elevated with the release of Visual Studio 2012, containing two hugely opposite color themes (Dark and Light), which the user could switch "on the fly" from the IDE options window. The GetVSSysColorEx method from Visual Studio Interop interface IVsUIShell2 could be utilized to obtain an environment's color settings. This interface is available to VSPackage plugins only: IVsUIShell2 vsshell = this.GetService(typeof(SVsUIShell)) as IVsUIShell2; By passing the __VSSYSCOLOREX and __VSSYSCOLOREX3 enums to the GetVSSysColorEx method, you can get the currently selected color for any of the Visual Studio UI elements. For example, let's obtain one of the colors from the context menu's background gradient: uint Win32Color; vsshell.GetVSSysColorEx((int)__VSSYSCOLOREX3.VSCOLOR_COMMANDBAR_MENU_BACKGROUND_GRADIENTBEGIN, out Win32Color); Color BackgroundGradient1 = ColorTranslator.FromWin32((int)Win32Color); Now we can use this Color object to "paint" our custom context menus. To determine the point in the time at which the color theme of your components should be reapplied, you can, for example, utilize events of the environment command responsible for opening of IDE's settings window (Tools -> Options). How to subscribe your handlers to such an event was described earlier in this article. But if you are, for some reason, unable to utilize the IVsUIShell2 object (for instance, in case you are developing a non-VSPackage extension), but at the same time you still need to support Visual Studio color themes, then it is possible to obtain color values for your environment's various UI components directly from the system registry. We will not cover this approach in the article, but here you can download a free and open-source tool designed for Visual Studio color theme editing. The tool is written in C# and it contains all the code required for reading and modifying Visual Studio 2012 color themes from the managed code. Interacting with COM Interfaces from Within a Multithreaded Application Initially the PVS-Studio extension package had not contained any specific thread-safety mechanisms for its interaction with Visual Studio APIs. At the same time, we had been attempting to confine the interactions with these APIs within a single background thread that was created and owned by our plug-in. And such an approach functioned flawlessly for quite a long period. However, several bug reports from our users, each one containing a similar ComExeption error, prompted us to examine this issue in more detail and to implement a threading safety mechanism for our COM Interop. Although the Visual Studio automation model is not a thread-safe one, it still provides a way for interacting with multi-threaded applications. The Visual Studio application is a COM (Component Object Mode) server. For the task of handling calls from COM clients (in our case, this will be our extension package) to thread-unsafe servers, COM provides a mechanism known as an STA (single-threaded apartment) model. In the terms of COM, an apartment represents a logical container inside a process in which objects and threads share the same thread access rules. STA can hold only a single thread, but an unlimited number of objects, inside such container. Calls from other threads to such thread-unsafe objects inside STA are converted into messages and posted to a message queue. Messages are retrieved from the message queue and converted back into method calls one at a time by the thread running in the STA, so it becomes possible for only a single thread to access these unsafe objects on the server. Utilizing Apartment Mechanism Inside Managed Code The .NET Framework does not utilize COM Apartment mechanics directly. Therefore, when a managed application calls a COM object in the COM interoperation scenarios, CLR (Common Language Runtime) creates and initializes apartment container. A managed thread is able to create and enter either an MTA (multi-threaded apartment, a container that, contrary to an STA, can host several threads at the same time), or an STA, though a thread will be started as an MTA by default. The type of the apartment could be specified before thread is launched: Thread t = new Thread(ThreadProc); t.SetApartmentState(ApartmentState.STA); ... t.Start(); As an apartment type could not be changed once a thread had been started, the STAThread attribute should be used to specify the main thread of a managed application as an STA: [STAThread] static void Main(string[] args) {...} Implementing Message Filters for COM Interoperation Errors in a Managed Environment STA serializes all calls to the COM server, so one of the calling clients could potentially be blocked or even rejected when the server is busy, processing different calls or another thread is already inside the apartment container. In case the COM server rejects its client, .NET COM interop will generate a System.Runtime.InteropServices.COMException ("The message filter indicated that the application is busy"). When working on a Visual Studio module (add-in, vspackage) or a macro, the execution control usually passes into the module from the environment's main STA UI thread (such as in the case of handling events or environment state changes, etc.). Calling automation COM interfaces from this main IDE thread is safe. But if other background threads are planned to be utilized and EnvDTE COM interfaces are to be called from these background threads (as in the case of long calculations that could potentially hang the IDE's interface, if these are performed on the main UI thread), then it is advised to implement a mechanism for handling calls rejected by a server. While working on the PVS-Studio plug-in, we've often encountered these kinds of COM exceptions in situations when other third-party extensions were active inside the IDE simultaneously with the PVS-Studio plug-in. Heavy user interaction with the UI also was the usual cause for such issues. It is quite logical that these situations often resulted in simultaneous parallel calls to COM objects inside STA and consequently to the rejection of some of them. To selectively handle incoming and outgoing calls, COM provides the IMessageFilter interface. If the server implements it, all of the calls are passed to the HandleIncomingCall method, and the client is informed on the rejected calls through the RetryRejectedCall method. This in turn allows the rejected calls to be repeated, or at least to correctly present this rejection to a user (for example, by displaying a dialog with a 'server is busy' message). Following is the example of implementing the rejected call handling for a managed application: [ComImport()] [Guid("00000016-0000-0000-C000-000000000046")] [InterfaceType(ComInterfaceType.InterfaceIsIUnknown)] public interface IMessageFilter { [PreserveSig] int HandleInComingCall( int dwCallType, IntPtr hTaskCaller, int dwTickCount, IntPtr lpInterfaceInfo); [PreserveSig] int RetryRejectedCall( IntPtr hTaskCallee, int dwTickCount, int dwRejectType); [PreserveSig] int MessagePending( IntPtr hTaskCallee, int dwTickCount, int dwPendingType); } class MessageFilter : MarshalByRefObject, IDisposable, IMessageFilter { [DllImport("ole32.dll")] [PreserveSig] private static extern int CoRegisterMessageFilter( IMessageFilter lpMessageFilter, out IMessageFilter lplpMessageFilter); private IMessageFilter oldFilter; private const int SERVERCALL_ISHANDLED = 0; private const int PENDINGMSG_WAITNOPROCESS = 2; private const int SERVERCALL_RETRYLATER = 2; public MessageFilter() { //Starting IMessageFilter for COM objects int hr = MessageFilter.CoRegisterMessageFilter( (IMessageFilter)this, out this.oldFilter); System.Diagnostics.Debug.Assert(hr >= 0, "Registering COM IMessageFilter failed!"); } public void Dispose() { //disabling IMessageFilter IMessageFilter dummy; int hr = MessageFilter.CoRegisterMessageFilter(this.oldFilter, out dummy); System.Diagnostics.Debug.Assert(hr >= 0, "De-Registering COM IMessageFilter failed!") System.GC.SuppressFinalize(this); } int IMessageFilter.HandleInComingCall(int dwCallType, IntPtr threadIdCaller, int dwTickCount, IntPtr lpInterfaceInfo) { // Return the ole default (don't let the call through). return MessageFilter.SERVERCALL_ISHANDLED; } int IMessageFilter.RetryRejectedCall(IntPtr threadIDCallee, int dwTickCount, int dwRejectType) { if (dwRejectType == MessageFilter.SERVERCALL_RETRYLATER) { // Retry the thread call immediately if return >=0 & // <100. return 150; //waiting 150 mseconds until retry } // Too busy; cancel call. SERVERCALL_REJECTED return -1; //Call was rejected by callee. //(Exception from HRESULT: 0x80010001 (RPC_E_CALL_REJECTED)) } int IMessageFilter.MessagePending( IntPtr threadIDCallee, int dwTickCount, int dwPendingType) { // Perform default processing. return MessageFilter.PENDINGMSG_WAITNOPROCESS; } } Now we can utilize our MessageFilter while calling COM interfaces from a background thread: using (new MessageFilter()) { //COM-interface dependent code ... } References MSDN. Referencing Automation Assemblies and the DTE2 Object. MSDN. Functional Automation Groups. MZ-Tools. HOWTO: Use correctly the OnConnection method of a Visual Studio add-in. The Code Project. Understanding The COM Single-Threaded Apartment. MZ-Tools. HOWTO: Add an event handler from a Visual Studio add-in. Dr. eX's Blog. Using EnableVSIPLogging to identify menus and commands with VS 2005 + SP1.

Introduction This article is part of the ongoing series I’ve been writing recently, but can be read as a standalone article. I’m going to do a better job of integrating the changes documented here into the ongoing solution I’ve been building. However, considering how much time and effort I put into solving this issue, I’ve decided to document the approach independently in case it is of use to others in the interim. The Problem Defined This issue presents itself when you are dealing with disconnected/detached Entity Framework POCO objects,. as the DbContext doesn’t track changes to entities. Specifically, trouble occurs with entities participating in a many-to-many relationship, where the EF has hidden a “join table” from the model itself. The problem with detached entities is that the data context has no way of knowing what changes have been made to an object graph, without fetching the data from the data store and doing an entity-by-entity comparison – and that assuming it’s possible to fetch the same way as it was originally. In this solution, all the entities are detached, don’t use proxy types and are designed to move between WCF service boundaries. Some Inspiration There are no out-of-the-box solutions that I’m aware of which can process POCO object graphs that are detached. I did find an interesting solution called GraphDiff which is available from github and also as a NuGet package, but it didn’t work with the latest RC version of the Entity Framework (v6). I also found a very comprehensive article on how to implement a generic repository pattern with the Entity Framework, but it was unable to handle detached many-to-many relationships. In any case, I highly recommend a read of this article, it was inspiration for some of the approach I’ve ended up taking with my own design. The Approach This morning I put together a simple data model with the relationships that I wanted to support with detached entities. I’ve attached the solution with a sample schema and test data at the bottom of this article. If you prefer to open and play with it, be sue to add the Entity Framework (v6 RC) via NuGet, I’ve omitted it for file size and licensing reasons). Here’s a logical view of the model I wanted to support: Here’s the schema view from SQL Server: Here’s the Entity Model which is generated from the above SQL schema: In the spirit of punching myself in the head, I’ve elected to have one table implement an identity specification (meaning the underlying schema allocated PK ID values) whereas the other two tables the ID must be specified. Theoretically, if I can handle the entity types in a generic fashion, then this solution can scale out to larger and more complex models. The scenarios I’m specifically looking to solve in this solution with detached object graphs are as follows: Add a relationship (many-to-many) Add a relationship (FK-based) Update a related entity (many-to-many) Update a related entity (FK-based) Remove a relationship (many-to-many) Remove a relationship (FK-based) Per the above, here’s the scenarios within the context of the above data model: Add a new Secondary entity to a Primary entity Add an Other entity to a Secondary entity Update a Secondary entity by updating a Primary entity Update an Other entity from a Secondary entity (or Primary entity) Remove (but not delete!) a Secondary entity from a Primary entity Remove (but not delete) a Other entity from a Secondary entity Establishing Test Data Just to give myself a baseline, the data model is populated (by default) with the following data. This gives us some “existing entities” to query and modify. More Work for the Consumer Although I tried my best, I couldn’t come to a design which didn’t require the consuming client to do slightly more work to enable this to work properly. Unfortunately the best place for change tracking to occur with disconnected entities is with the layer making changes – be it a business layer or something downstream. To this effect, entities will need to implement a property which reflects the state of the entity (added, modified, deleted etc.). For the object graph to be updated/managed successfully, the consumer of the entities needs to set the entity state properly. This isn’t at all as bad as it sounds, but it’s not nothing. Establishing some Scaffolding After generating the data model, the first thing to be done is ensure each entity derives from the same base class. (“EntityBase”) this is used later to establish the active state of an entity when it needs to be processed. I’ve also created an enum (“ObjectState”) which is a property of the base class and a helper function which maps ObjectState to an EF EntityState. In case this isn’t clear, here’s a class view: Constructing Data Access To ensure that the usage is consistent, I’ve defined a single Data Access class, mainly to establish the pattern for handling detached object graphs. I can’t stress enough that this is not intended as a guide to an appropriate way to structure your data access – I’ll be updating my ongoing series of articles to go into more detail – this is only to articulate a design approach to handling detached object graphs. Having said all that, here’s a look at my “DataAccessor” class, which can be used with generic data access entities (by way of generics): As with my ongoing project, the Entity Framework DbContext is instantiated by this class on construction, and implements IDisposable to ensure the DbContext is disposed properly upon construction. Here’s the constructor showing the EF configuration options I’m using: public DataAccessor() { _accessor = new SampleEntities(); _accessor.Configuration.LazyLoadingEnabled = false; _accessor.Configuration.ProxyCreationEnabled = false; } Updating an Entity We start with a basic scenario to ensure that the scaffolding has been implemented properly. The scenario is to query for a Primary entity and then change a property and update the entity in the data store. [TestMethod] public void UpdateSingleEntity() { Primary existing = null; String existingValue = String.Empty; using (DataAccessor a = new DataAccessor()) { existing = a.DataContext.Primaries.Include("Secondaries").First(); Assert.IsNotNull(existing); existingValue = existing.Title; existing.Title = "Unit " + DateTime.Now.ToString("MMdd hh:mm:ss"); } using (DataAccessor b = new DataAccessor()) { existing.State = ObjectState.Modified; b.InsertOrUpdate(existing); } using (DataAccessor c = new DataAccessor()) { existing.Title = existingValue; existing.State = ObjectState.Modified; c.InsertOrUpdate(existing); } } You’ll noticed that there is nothing particularly significant here, except that the object’s State is reset toModified between operations. Updating a Many-to-Many Relationship Now things get interesting. I’m going to query for a Primary entity, then I’ll update both a property of thePrimary entity itself, and a property of one of the entity’s relationships. [TestMethod] public void UpdateManyToMany() { Primary existing = null; Secondary other = null; String existingValue = String.Empty; String existingOtherValue = String.Empty; using (DataAccessor a = new DataAccessor()) { //Note that we include the navigation property in the query existing = a.DataContext.Primaries.Include("Secondaries").First(); Assert.IsTrue(existing.Secondaries.Count() > 1, "Should be at least 1 linked item"); } //save the original description existingValue = existing.Description; //set a new dummy value (with a date/time so we can see it working) existing.Description = "Edit " + DateTime.Now.ToString("yyyyMMdd hh:mm:ss"); existing.State = ObjectState.Modified; other = existing.Secondaries.First(); //save the original value existingOtherValue = other.AlternateDescription; //set a new value other.AlternateDescription = "Edit " + DateTime.Now.ToString("yyyyMMdd hh:mm:ss"); other.State = ObjectState.Modified; //a new data access class (new DbContext) using (DataAccessor b = new DataAccessor()) { //single method to handle inserts and updates //set a breakpoint here to see the result in the DB b.InsertOrUpdate(existing); } //return the values to the original ones existing.Description = existingValue; other.AlternateDescription = existingOtherValue; existing.State = ObjectState.Modified; other.State = ObjectState.Modified; using (DataAccessor c = new DataAccessor()) { //update the entities back to normal //set a breakpoint here to see the data before it reverts back c.InsertOrUpdate(existing); } } If we actually run this unit test and set the breakpoints accordingly, you’ll see the following in the database: Database at Breakpoint #1 / Database at Breakpoint #2 Database when Unit Test completes You’ll notice at the second breakpoint that the description of the first entities have both been updated. Examining the Insert/Update Code The function exposed by the “data access” class really just passes through to another private function which does the heavy lifting. This is mainly in case we need to reuse the logic, since it essentially processes state action on attached entities. public void InsertOrUpdate(params T[] entities) where T : EntityBase { ApplyStateChanges(entities); DataContext.SaveChanges(); } Here’s the definition of the ApplyStateChanges function, which I’ll discuss below: private void ApplyStateChanges(params T[] items) where T : EntityBase { DbSet dbSet = DataContext.Set(); foreach (T item in items) { //loads related entities into the current context dbSet.Attach(item); if (item.State == ObjectState.Added || item.State == ObjectState.Modified) { dbSet.AddOrUpdate(item); } else if (item.State == ObjectState.Deleted) { dbSet.Remove(item); } foreach (DbEntityEntry entry in DataContext.ChangeTracker.Entries() .Where(c => c.Entity.State != ObjectState.Processed && c.Entity.State != ObjectState.Unchanged)) { var y = DataContext.Entry(entry.Entity); y.State = HelperFunctions.ConvertState(entry.Entity.State); entry.Entity.State = ObjectState.Processed; } } } Notes on this Implementation What this function does is to iterate through the items to be examined, attach them to the current Data Context (which also attaches their children), act on each item accordingly (add/update/remove) and then process new entities which have been added to the Data Context’s change tracker. For each newly “discovered” entity (and ignoring entities which are unchanged or have already been examined), each entity’s DbEntityEntry is set according to the entity’s ObjectState (which is set by the calling client). Doing this allows the Entity Framework to understand what actions it needs to perform on the entities when SaveChanges() is invoked later. You’ll also note that I set the entity’s state to “Processed” when it has been examined, so we don’t act on it more than once (for performance purposes). Fun note: the AddOrUpdate extension method is something I found in theSystem.Data.Entity.Migrations namespace and it acts as an ‘Upsert’ operation, inserting or updating entities depending on whether they exist or not already. Bonus! That’s it for adding and updating, believe it or not. Corresponding Unit Test The following unit test establishes the creation of a new many-to-many entity, it is then removed (by relationship) and then finally deleted altogether from the database: [TestMethod] public void AddRemoveRelationship() { Primary existing = null; using (DataAccessor a = new DataAccessor()) { existing = a.DataContext.Primaries.Include("Secondaries") .FirstOrDefault(); Assert.IsNotNull(existing); } Secondary newEntity = new Secondary(); newEntity.State = ObjectState.Added; newEntity.AlternateTitle = "Unit"; newEntity.AlternateDescription = "Test"; newEntity.SecondaryId = 1000; existing.Secondaries.Add(newEntity); using (DataAccessor a = new DataAccessor()) { //breakpoint #1 here a.InsertOrUpdate(existing); } newEntity.State = ObjectState.Unchanged; existing.State = ObjectState.Modified; using (DataAccessor b = new DataAccessor()) { //breakpoint #2 here b.RemoveEntities(existing, x => x.Secondaries, newEntity); } using (DataAccessor c = new DataAccessor()) { //breakpoint #3 here c.Delete(newEntity); } } Test Results: Pre-Test – Breakpoint #1 / Breakpoint #2 Breakpoint #3 / Post execution (new entity deleted) SQL Profile Trace Removing a Many-to-Many Relationship Now this is where it gets tricky. I’d like to have something a little more polished, but the best I have come up with to date is a separate operation on the data provider which exposes functionality akin to “remove relationship”. The fundamental problem with how the EF POCO entities work without any modifications, is when they are detached, to remove a many-to-many relationship, the relationship to be removed is physically removed from the collection. When the object graph is sent back for processing, there’s a missing related entity, and the service or data context would have to make an assumption that the omission was on purpose, not to mention that it would have to compare against data currently in the data store. To make this easier, I’ve implemented a function called “RemoveEnttiies” which alters the relationship between the parent and the child/children. The one bug catch is that you need to specify the navigation property or collection, which might make it slightly undesirable to implement generically. In any case, I’ve provided two options – with the navigation property as a string parameter or as a LINQ expression – they both do the same thing. public void RemoveEntities(T parent, Expression> expression, params T2[] children) where T : EntityBase where T2 : EntityBase { DataContext.Set().Attach(parent); ObjectContext obj = DataContext.ToObjectContext(); foreach (T2 child in children) { DataContext.Set().Attach(child); obj.ObjectStateManager.ChangeRelationshipState(parent, child, expression, EntityState.Deleted); } DataContext.SaveChanges(); } Notes on this Implementation The “ToObjectContext” is an extension method, and is akin to (DataContext as IObjectContextAdapter).ObjectContext. This is to expose a more fundamental part of the Entity Framework’s object model. We need this level of access to get to the functionality which controls relationships. For each child to be removed (note: not deleted from the physical database), we nominate the parent object, the child, the navigation property (collection) and the nature of the relationship change (delete). Note that this will NOT WORK for Foreign Key defined relationships – more on that below. To delete entities which have active relationships, you’ll need to drop the relationship before attempting to delete or else you’ll have data integrity/referential integrity errors, unless you have accounted for cascading deletion (which I haven’t). Example execution: using (DataAccessor c = new DataAccessor()) { //c.RemoveEntities(existing, "Secondaries", s); //(or can use an expression): c.RemoveEntities(existing, x => x.Secondaries, s); } Removing FK Relationships As mentioned above, you can’t just edit the relationship to remove an FK-based relationship. Instead, you have to follow the EF practice of setting the FK entity to NULL. Here’s a Unit Test which demonstrates how this is achieved: Secondary s = ExistingEntity(); using (DataAccessor c = new DataAccessor()) { s.Other = null; s.OtherId = null; s.State = ObjectState.Modified; o.State = ObjectState.Unchanged; c.InsertOrUpdate(s); } We use the same “Insert or Update’ call – being aware that you have to set the ObjectState properties accordingly. Note: I’m in the process of testing the reverse removal – i.e. what happens if you want to remove a Secondaryentity from an Other entity’s collection. Deleting Entities This is fairly straightforward, but I’ve taken a few more precautions to ensure that the entity to be deleted is valid no the server side. public void Delete(params T[] entities) where T : EntityBase { foreach (T entity in entities) { T attachedEntity = Exists(entity); if (attachedEntity != null) { var attachedEntry = DataContext.Entry(attachedEntity); attachedEntry.State = EntityState.Deleted; } } DataContext.SaveChanges(); } To understand the above, you should take a look at the implementation of the “Exists” function which essentially checks the data store and local cache to see if there is an attached representation: protected T Exists(T entity) where T : EntityBase { var objContext = ((IObjectContextAdapter)this.DataContext) .ObjectContext; var objSet = objContext.CreateObjectSet(); var entityKey = objContext.CreateEntityKey(objSet.EntitySet.Name, entity); DbSet set = DataContext.Set(); var keys = (from x in entityKey.EntityKeyValues select x.Value).ToArray(); //Remember, there can by surrogate keys, so don't assume there's //just one column/one value //If a surrogate key isn't ordered properly, the Set().Find() //method will fail, use attributes on the entity to determine the //proper order. //context.Configuration.AutoDetectChangesEnabled = false; return set.Find(keys); } This is a fairly expensive operation which is why it’s pretty much reserved for deletes and not more frequent operations. It essentially determines the target entity’s primary key and then checks whether the entity exists or not. Note: I haven’t tested this on entities with surrogate keys, but I’ll get to it at some point. If you have surrogate key tables, you can define the PK key order using attributes on the model entity, but I haven’t done this (yet). Summary This article is the culmination of about two days of heavy analysis and investigation. I’ve got a whole lot more to contribute on this topic, but for now, I felt it was worthy enough to post as-is. What you’ve got here is still incredibly rough, and I haven’t done nearly enough testing. To be honest, I was quite excited by the initial results, which is why I decided to write this post. there’s an incredibly good chance that I’ve missed something in the design and implementation, so please be aware of that. I’ll be continuing to refine this approach in my main series of articles with much cleaner implementation. In the meantime though, if any of this helps anyone out there struggling with detached entities, I hope it helps. There’s precious few articles and samples that are up to date, and very few that seem to work. This is provided without any warranty of any kind! If you find any issues please e-mail me [email protected] and I’ll attempt to refactor/debug and find ways around some of the inherent limitations. In the meantime, there are a few helpful links I’ve come across in my travels on the WWW. See below. Example Solution Files [ Files ] Note: you’ll need to add the Entity Framework v6 RC package via NuGet, I haven’t included it in the archive. Helpful Links http://blog.magnusmontin.net/2013/05/30/generic-dal-using-entity-framework/ https://github.com/refactorthis/GraphDiff http://stackoverflow.com/questions/11686225/dbset-find-method-ridiculously-slow-compared-to-singleordefault-on-id http://stackoverflow.com/questions/10381106/cannot-update-many-to-many-relationships-in-entity-framework http://stackoverflow.com/questions/8413248/how-to-save-an-updated-many-to-many-collection-on-detached-entity-framework-4-1 http://stackoverflow.com/questions/6018711/generic-way-to-check-if-entity-exists-in-entity-framework

Learn about ElasticSearch, an open source tool developed with Java. It is a Lucene-based, scalable, full-text search engine, and a data analysis tool.

Let’s look at some best practices for using another powerful class: ThreadLocal from java.lang, which is also implemented using WeakReference.

Use case: Deploy a new version of a WebApp so that all new users are sent to the new version while users with open sessions continue using the previous version (so they don’t lose their precious session state). Users of the new version can explicitly ask for the previous version in case it doesn’t work as expected, and vice versa. Benefits: Get new features to users that need them as soon as possible without affecting anybody negatively and without risking that a defect will prevent users from achieving their goal (thanks to being able to fall back to the previous version). An alternative to sticky sessions would be to use an external, shared session store (for example, Redis). But the overhead and complexity of that wasn’t worth the gains in this case. Implementation Configuration We will run two instances of our service, on two different ports, with HAProxy in front of them. We will use HAPRoxy’s health checks to trick it into believing that one of the instances is partially unwell. HAProxy will thus send it only the existing sessions, while sending all new ones to the other, “fully healthy,” instance. Notice that, even though I run both HAProxy and the instances on the same machine, they could also live on different ones. There are actually two solutions. The first one, described here, is simpler, but clients will experience a downtime of about five seconds when an instance is actually not available (crashes). The other solution doesn’t suffer from this drawback but requires the ability to reach the instance through two different ports (or perhaps hostnames) for the health checks. Both solutions are well described in HAProxy’s “architecture” document: “4.1 Soft-stop using a file on the servers” (though we will use a dynamic servlet instead of a static file) and “4.2 Soft-stop using backup servers“. Now go to GitHub to see the proof of concept implementation and read the two sections, 4.1 and 4.2, referenced above. The implementation uses a simple Java WebApp running on Tomcat and can be run either on a Linux machine or via Vagrant. Follow the instructions and information in the readme. The webapp needs to implement support for settable health checks. In our case, the servlet enables POSTs to /health/(enable|disable) to set the availability status and HEAD requests to /health that return either "OK" or "SERVICE UNAVAILABLE" based on the status. Deployment The deployment is simple: Figure out which zone, blue or green, is the current or the previous one (for example, by having recorded it into a file). (Optional.) Check that the previous instance has no active sessions and take it down. (Not implemented in the PoC.) Deploy over the previous instance (copy the new binary from a remote repository, S3 or whatever) Start the newly deployed instance and verify that it is well, record its zone as the active one. Switch over: Tell the old instance to start reporting to HAProxy that it is unavailable and the new one to report itself as healthy. Drink a cocktail and enjoy. See the actual implementation in the PoC project. Let me see! An old version of the app running in the blue zone (localhost:8080, reloaded in a browser tab where it was opened before the new deployment): PoC – old version of the app A new version of the app in the green zone (localhost:8080, in a new browser): PoC – the new version of the app A part of HAProxy’s stats page reporting about the zones (localhost:8081): PoC – HAProxy’s stats page with the old version (blue) pretending to be down so that new requests go to green Credits Thanks to my colleague Stein Kvarud for the inspiration to use HAProxy. Conclusion It is very easy to set up downtime-less blue/green deployment that doesn’t break existing sessions and provides the ability to go back to the previous version.

The seda component in Apache Camel is very similar to the direct component that I’ve presented in previous blog, but in a asynchronous manner.