There are currently two schools of thought in developing Web Services – one being the standards-based traditional approach [ SOAP ] and the other, simpler school of thought [ REST ]. This article quickly compares one with the other - REST SOAP Assumes a point-to-point communication model–not usable for distributed computing environment where message may go through one or more intermediaries Designed to handle distributed computing environments Minimal tooling/middleware is necessary. Only HTTP support is required Requires significant tooling/middleware support URL typically references the resource being accessed/deleted/updated The content of the message typically decides the operation e.g. doc-literal services Not reliable – HTTP DELETE can return OK status even if a resource is not deleted Reliable Formal description standards not in widespread use. WSDL 1.2, WADL are candidates. Well defined mechanism for describing the interface e.g. WSDL+XSD, WS-Policy Better suited for point-to-point or where the intermediary does not play a significant role Well suited for intermediated services No constraints on the payload Payload must comply with the SOAP schema Only the most well established standards apply e.g. HTTP, SSL. No established standards for other aspects. DELETE and PUT methods often disabled by firewalls, leads to security complexity. A large number of supporting standards for security, reliability, transactions. Built-in error handling (faults) No error handling Tied to the HTTP transport model Both SMTP and HTTP are valid application layer protocols used as Transport for SOAP Less verbose More verbose

It is common that most of the developers use their own logging frameworks at the time of development and that force organizations to maintain configuration for each logging framework. Normally switching from logging level from DEBUG to INFO sometimes requires a application restart in production. SLF4J is the latest logging facade helps to plug-in desired logging framework at deployment time. The article further talks about usage of the SLF4J with logback. SLF4J - Simple Logging Facade for Java API helps to plug-in desired logging implementation at deployment time. Logback - helps to change logging configuration through JMX at runtime with out restarting your applications in production. I hope this article helps to get high level overview of SLF4J/Logback and migrating your existing apps to common logging approach. SLF4J This is simple logging façade to abstract the various logging frameworks such as logback, log4j, commons-logging and default java logging implementation (java.util.logging). This primarily enables the user to inlcude desired logging framework at deployment time. It is lightweight and nearly adds a zero overhead on performance. Note that SLF4j doesn’t replace any logging framework; it is just a façade around any standard logging framework. If slf4j doesn’t find any logging framwork in classpath, by default it prints the logs in console. Logback This is an improved version of log4j and natively supports the slf4j, hence migrating from other logging frameworks such as log4j and java.util.logging is quite possible. Since the logback natively supports slf4j, the combination of using slf4j with this framework is relatively faster than the slf4j with other logging frameworks. Logging configuration can be done either in xml or groovy. *One important feature is that it exposes the configuration through JMX hence configuration (debug to info etc) can be changed via JMX console with out restarting the application. Also, it does print the artifact version as part of the exception stacktrace that may be helpful for debugging. java.lang.NullPointerException: null at com.fimt.poc.LoggingSample.(LoggingSample.java:16) [classes/:na] at com.fimt.poc.LoggingSample.main(LoggingSample.java:23) [fimt-logging-poc-1.0.jar/:1.0] **Reasons to prefer logback over log4j is well explainedhere SLF4J api usage in java classes (1) Import the Logger and LoggerFactory from org.slf4j package import org.slf4j.Logger; import org.slf4j.LoggerFactory; (2) Declare the logger class as, private final Logger logger = LoggerFactory.getLogger(LoggingSample.class); (3) Use debug, warn, info, error and trace with appropriate parameters. All methods by default takes the string as an input. logger.info("This is sample info statement"); SLF4J with Logback Include the following dependency pom.xml, it pulls its depedencies logback-core and slf4j-api in addition to logback-classic ch.qos.logback logback-classic 1.0.7 SLF4J can be used with existing logging framworks log4j, common-logging and java.util.logging (JUL). Required dependencies are mentioned below. SLF4J with Log4j Include the following dependency pom.xml, it pulls its depedencies log4jand slf4j-api in addition to slf4j-log4j12 artifact. org.slf4j slf4j-log4j12 1.7.2 SLF4J with JUL (java.util.logging) Include the following dependency pom.xml, it pulls its depedency slf4j-api in addition to slf4j-jdk14 artifact. org.slf4j slf4j-jdk14 1.7.2 Migrating existing projects logging to logback framework Step: 1 – Update existing project pom.xml Add the right dependency mentioned above. Also remove the unused log4j/commons logging dependencies. Step: 2 – Update java files with SLF4J API Scan all java files and replace log4j or java.util.logging classes in to SLF4J api classes. This can be done using the tool java -jar slf4j-migrator-1.7.2.jar . Detailed documentation and limitation is mentioned here This tool replaces the logging apis of log4j, commons-logging and java.util.logging in to SLF4J API classes. Step: 3 – Convert log4j.properties to logback.xml Logback provides a online translator which converts log4j properties in to logback.xml File Appenders: Like other logging frameworks, logback implementation also supports various file appenders. Daily file rolling: logFile.%d{yyyy-MM-dd}.log 30 Roll log files based on size: tests.%i.log.zip 1 10 5MB Layout %-4relative [%thread] %-5level %logger{35} - %msg%n

I Programmer - Anime Scrum - An Overview If you are an anime fan, and perhaps even if you are not, then you might like a new poster about Scrum - in anime style. As long as you find anime cute or something then seeing the different people involved in the Scrum methodology as anime characters might help you convey the ideas to others. ... Scrum Primer - Scrum Overview - Anime version High-resolution versions of the overview: Scrum Overview - Blue Scrum Overview - Pink Scrum Overview - Green Feel free to use it in your own material. ..." Come on! You KNOW that's awesome! :)

Last time we familiarized ourselves with ListenableFuture. I promised to introduced more advanced techniques, namely transformations and chaining. Let's start from something straightforward. Say we have our ListenableFuture which we got from some asynchronous service. We also have a simple method: Document parse(String xml) {//... We don't need String, we need Document. One way would be to simply resolve Future (wait for it) and do the processing on String. But much more elegant solution is to apply transformation once the results are available and treat our method as if was always returning ListenableFuture. This is pretty straightforward: final ListenableFuture future = //... final ListenableFuture documentFuture = Futures.transform(future, new Function() { @Override public Document apply(String contents) { return parse(contents); } }); or more readable: final Function parseFun = new Function() { @Override public Document apply(String contents) { return parse(contents); } }; final ListenableFuture future = //... final ListenableFuture documentFuture = Futures.transform(future, parseFun); Java syntax is a bit limiting, but please focus on what we just did. Futures.transform() doesn't wait for underlying ListenableFuture to apply parse() transformation. Instead, under the hood, it registers a callback, wishing to be notified whenever given future finishes. This transformation is applied dynamically and transparently for us at right moment. We still have Future, but this time wrapping Document. So let's go one step further. We also have an asynchronous, possibly long-running method that calculates relevance (whatever that is in this context) of a given Document: ListenableFuture calculateRelevance(Document pageContents) {//... Can we somehow chain it with ListenableFuture we already have? First attempt: final Function> relevanceFun = new Function>() { @Override public ListenableFuture apply(Document input) { return calculateRelevance(input); } }; final ListenableFuture future = //... final ListenableFuture documentFuture = Futures.transform(future, parseFun); final ListenableFuture> relevanceFuture = Futures.transform(documentFuture, relevanceFun); Ouch! Future of future of Double, that doesn't look good. Once we resolve outer future we need to wait for inner one as well. Definitely not elegant. Can we do better? final AsyncFunction relevanceAsyncFun = new AsyncFunction() { @Override public ListenableFuture apply(Document pageContents) throws Exception { return calculateRelevance(pageContents); } }; final ListenableFuture future = //comes from ListeningExecutorService final ListenableFuture documentFuture = Futures.transform(future, parseFun); final ListenableFuture relevanceFuture = Futures.transform(documentFuture, relevanceAsyncFun); Please look very carefully at all types and results. Notice the difference between Function and AsyncFunction. Initially we got an asynchronous method returning future of String. Later on we transformed it to seamlessly turn String into XML Document. This transformation happens asynchronously, when inner future completes. Having future of Document we would like to call a method that requires Document and returns future of Double. If we call relevanceFuture.get(), our Future object will first wait for inner task to complete and having its result (String -> Document) will wait for outer task and return Double. We can also register callbacks on relevanceFuture which will fire when outer task (calculateRelevance()) finishes. If you are still here, the are even more crazy transformations. Remember that all this happens in a loop. For each web site we got ListenableFuture which we asynchronously transformed to ListenableFuture. So in the end we work with a List>. This also means that in order to extract all the results we either have to register listener for each and every ListenableFuture or wait for each of them. Which doesn't progress us at all. But what if we could easily transform from List> to ListenableFuture>? Read carefully - from list of futures to future of list. In other words, rather than having a bunch of small futures we have one future that will complete when all child futures complete - and the results are mapped one-to-one to target list. Guess what, Guava can do this! final List> relevanceFutures = //...; final ListenableFuture> futureOfRelevance = Futures.allAsList(relevanceFutures); Of course there is no waiting here as well. Wrapper ListenableFuture> will be notified every time one of its child futures complete. The moment the last child ListenableFuture completes, outer future completes as well. Everything is event-driven and completely hidden from you. Do you think that's it? Say we would like to compute the biggest relevance in the whole set. As you probably know by now, we won't wait for a List. Instead we will register transformation from List to Double! final ListenableFuture maxRelevanceFuture = Futures.transform(futureOfRelevance, new Function, Double>() { @Override public Double apply(List relevanceList) { return Collections.max(relevanceList); } }); Finally, we can listen for completion event of maxRelevanceFuture and e.g. send results (asynchronously!) using JMS. Here is a complete code if you lost track: private Document parse(String xml) { return //... } private final Function parseFun = new Function() { @Override public Document apply(String contents) { return parse(contents); } }; private ListenableFuture calculateRelevance(Document pageContents) { return //... } final AsyncFunction relevanceAsyncFun = new AsyncFunction() { @Override public ListenableFuture apply(Document pageContents) throws Exception { return calculateRelevance(pageContents); } }; //... final ListeningExecutorService pool = MoreExecutors.listeningDecorator( Executors.newFixedThreadPool(10) ); final List> relevanceFutures = new ArrayList<>(topSites.size()); for (final URL siteUrl : topSites) { final ListenableFuture future = pool.submit(new Callable() { @Override public String call() throws Exception { return IOUtils.toString(siteUrl, StandardCharsets.UTF_8); } }); final ListenableFuture documentFuture = Futures.transform(future, parseFun); final ListenableFuture relevanceFuture = Futures.transform(documentFuture, relevanceAsyncFun); relevanceFutures.add(relevanceFuture); } final ListenableFuture> futureOfRelevance = Futures.allAsList(relevanceFutures); final ListenableFuture maxRelevanceFuture = Futures.transform(futureOfRelevance, new Function, Double>() { @Override public Double apply(List relevanceList) { return Collections.max(relevanceList); } }); Futures.addCallback(maxRelevanceFuture, new FutureCallback() { @Override public void onSuccess(Double result) { log.debug("Result: {}", result); } @Override public void onFailure(Throwable t) { log.error("Error :-(", t); } }); Was it worth it? Yes and no. Yes, because we learned some really important constructs and primitives used together with futures/promises: chaining, mapping (transforming) and reducing. The solution is beautiful in terms of CPU utilization - no waiting, blocking, etc. Remember that the biggest strength of Node.js is its "no-blocking" policy. Also in Netty futures are ubiquitous. Last but not least, it feels very functional. On the other hand, mainly due to Java syntax verbosity and lack of type inference (yes, we will jump into Scala soon) code seems very unreadable, hard to follow and maintain. Well, to some degree this holds true for all message driven systems. But as long as we don't invent better APIs and primitives, we must learn to live and take advantage of asynchronous, highly parallel computations. If you want to experiment with ListenableFuture even more, don't forget to read official documentation.

Attracting your attention with the article ‘Is MVC Pattern a Dead End for Application Development? ’ I think I have not explained in enough detail what the mechanism of MVC itself that’s why I should shed light on the MVP as well. I believe it is quite important to understand the difference between MVC and MVP and the common elements of both paradigms. To figure out the difference let’s have a look at the conditions why MVC has been widely spread out and what benefits it’s provided for application development. Let’s come back to the past. At the times when AJAX statement still had been unknown. At the times when programmers were afraid of using JavaScript in browser and that’s why had to reload the whole page. Let’s take a simple example. A user fills out the registration form and sends it to the server. Server validates entered data, determines the errors and sends the form with the filled data back to the user. Sending data to the server, in fact, we send the event with the data to the particular controller. It performs some actions with the data and transfers the control to the View. View generates the result with the HTML and sends it back to the user. What happens? Apparently the whole browser page and its server representation are the one big and distributed visual component. It has events as well as interface and implementation. Controller receives events in particular format and that’s why it is possible to send events to it from different views supporting that format. As an example it can be a login form with login and password. It can be placed for instance in the corner of each page or to be displayed on a separate page as well. MVC is advantageous to use when components are redrawn on every event received from the user. MVP was invented on a bit different conditions - when you build the application based on component framework and there is no need to constantly recreate the View. The goal of MVP is to make Views reusable. For that every View implements particular interface and events mechanism to communicate with Presenter. If to take another look at the example with the same form of entrance to the system - now it is possible to create two views – one in the corner of the application and another one within the main window and to subscribe the Presenter to both. From received event Presenter can get a sender and all data sent by the user via View interface. As a conclusion it is possible to say that MVC and MVP paradigms are quite similar to each other but their usage depends on the number of conditions. For MVC when View is refreshed every time on a particular event and for MVP when it is not required to recreate View every time. Having said this I would like to say that to use MVP as well as MVC is not enough to build up agile and scalable applications since they lead to the same problem of tight coupling between modules and require completely new approach that was described in previous article .

In this post I am going to explain about reason for InProcsessionState was not working with IIS 7.5. Problem: Yesterday in one of my PCs I got updated IIS 7.5 and suddenly after updating the IIS my ASP.NET application session state stopped working. It was a very weird behaviour and after doing some R and D I have found that my worker process per application was more then one. That was the reason for the session state was not working as “InProc” session state is using IIS memory to manage sessions. You can find more information about this in following link. http://stackoverflow.com/questions/2147578/asp-net-session-state-and-multiple-worker-processes Resolution: There are two ways of resolving this problem. 1) You can use other session state mode like State Server or SQL server which supports multiple worker processes. 2) Set your maximum number of worker process to 1 in IIS 7/7.5 application pool advance settings. I have changed the same and again it started working again. That's it. Hope you like it. Sty tuned for more...

The DLL also called Dynamic Link Library is a file that can be used dynamically by other programs. We create the DLL mainly to hide our confidential code so that we can use the code everywhere but can not see the code. When we talk about the layers or 3 and above tier application we need to understand one thing that why we create layers in our project. We only show some part of code to everyone and hide the main coding part. The benefit is we can use this code but can not see this. Advantages of creating DLL: There are many advantages of using the DLL in your Application but We are talking about the main advantages: 1. DLL is used to hide the code so anyone can use the functionality but no one can steal the code. 2. Once we create a DLL then it can be used in many applications. We just have to import that dll into our project. 3. Adding a dll into your project is very simple. It reduces time to write the code again and again. In C#.Net you need to just import dll with add reference in your project. Now we will see that how we can create a dll and then how the dll can be used in the application. Creating DLL: Step 1: Open Visual Studio and Create New Project by Clicking New->Project Step 2: Select 'Class Library' from the list and click Ok. Step 3: Now you will see the following screen Step 4: Change the name Class1. As I have changed the Name with MyCode and Write the code what you want to use in your DLL. Step 5: Now Click Build Menu -> Build Solution to complete the process. Now you can see the dll file with .dll extention inside you project folder. Now the dll is ready to use inside other application. Step 6: Create a New Project. Then Right click on Project Name -> Add Reference -> Select the dll File to add as reference into your project. Step 7: Now you need to include the namespace for that dll by using the following code. using ClassLibrary2; Step 8: Now you can access the method of that dll and the code will be hidden to you. You can access the method of dll file by using the following code. private void button1_Click(object sender, EventArgs e) { int x, y; x = Convert.ToInt32(txtNo1.Text); y = Convert.ToInt32(txtNo2.Text); MyCode ob = new MyCode(); int c = ob.sum(x, y); txtResult.Text = c.ToString(); } As in Example you can see that how we have created a dll and then how we used that dll into our project. You can also see the articles How to use Crystal Report in C#.Net.

node.js is well-know for its good connectivity with nosql databases. a less know fact is that it's also very efficient with relational databases. among the dozens orms out there in javascript, one stands out for relational databases: sequelize . it's quite easy to learn but there are not many pointers about how to organize model code with this module. here are a few tips we learned by using sequelize in a medium size project. sequelize 101 sequelize claims to supports mysql, postgresql and sqlite. the sequelize docs explain the first steps with the javascript orm. first, initialize a database connection, then a few models, without worrying about primary or foreign keys: var sequelize = new sequelize('database', 'username'[, 'password']) var project = sequelize.define('project', { title: sequelize.string, description: sequelize.text }); var task = sequelize.define('task', { title: sequelize.string, description: sequelize.text, deadline: sequelize.date }); project.hasmany(task); next, create new instances and persist them, query the database, etc. // create an instance var task = task.build({title: 'very important task'}) task.title // ==> 'very important task' // persist an instance task.save() .error(function(err) { // error callback }) .success(function() { // success callback }); // query persistence for instances var tasks = task.all({ where: ['dealdine < ?', new date()] }) .error(function(err) { // error callback }) .success(function() { // success callback }); sequelize uses promises so you can chain error and success callbacks, and it all plays well with unit tests. all that is pretty well documented, but the sequelize documentation only covers the basic usage. once you start using sequelize in real world projects, finding the right way to implement a feature gets trickier. model file structure all the examples in the sequelize documentation show all model declarations grouped in a single file. once a project reaches production size, this is not a viable approach. the alternative is to import models from a module using sequelize.import() . the problem is that relationships rely on several models. when you declare a relationship, models from both sides of the relationship must already be imported. you should not import model files from other model files because of node.js module caching policy (more on that later on); instead, you can define relationships in a standalone file. here is the file structure we've been working with: models/ index.js # import all models and creates relationships phonenumber.js task.js user.js ... and here is how the main models/index.js initializes the entire model: var sequelize = require('sequelize'); var config = require('config').database; // we use node-config to handle environments // initialize database connection var sequelize = new sequelize( config.name, config.username, config.password, config.options ); // load models var models = [ 'phonenumber', 'task', 'user' ]; models.foreach(function(model) { module.exports[model] = sequelize.import(__dirname + '/' + model); }); // describe relationships (function(m) { m.phonenumber.belongsto(m.user); m.task.belongsto(m.user); m.user.hasmany(m.task); m.user.hasmany(m.phonenumber); })(module.exports); // export connection module.exports.sequelize = sequelize; using models in code from other parts of the application, if you need a model class, require the models/index.js instead of the standalone model file. that way, you don't have to repeat the sequelize initialization. var models = require('./models'); var user = models.user; var user = user.build({ first_name: "john", last_name: "doe "}); the problem is, when you require the models/index.js file, node may use a cached version of the module... or not: from nodejs.org : multiple calls to require('foo') may not cause the module code to be executed multiple times. (...) modules are cached based on their resolved filename. since modules may resolve to a different filename based on the location of the calling module (loading from node_modules folders), it is not a guarantee that require('foo') will always return the exact same object, if it would resolve to different files. that means that using require('./models') to get the models may create more than one connection to the database. to avoid that, the models variable must be somehow singleton-esque. this can be easily achieved, if you're using a framework like expressjs , by attaching the models module to the application: app.set('models', require('./models')); and when you need to require a class of the model in a controller, use this application setting rather than a direct import: var user = app.get('models').user; accessing other models sequelize models can be extended with class and instance methods. you can add abilities to model classes, much like in a true activerecord implementation. but a problem arises when adding a method that depends on another model: how can a model access another one? // in models/user.js module.exports = function(sequelize, datatypes) { return sequelize.define('user', { first_name: datatypes.string, last_name: datatypes.string, }, { instancemethods: { counttasks: function() { // how to implement this method ? } } }); }; if the two models share a relationship, there is a way. here, one user has many tasks , that makes the task model accessible through user.associations['tasks'].target . and here is yet another problem: since sequelize doesn't use prototype-based inheritance, how can a user instance gain access to the user class? digging into the sequelize source brings the protected __factory to the light. with all this undocumented knowledge, it is now possible to write the counttasks() instance method: counttasks: function() { return this.__factory.associations['tasks'].target.count({ where: { user_id: this.id } }); } note that task.count() returns a promise, so counttasks() also returns a promise: user.counttasks().success(function(nbtasks) { // do somethig with the user task count }); extending models (a.k.a behaviors) what if you need to reuse several methods across several models? sequelize doesn't have a behavior system per se (or "concerns" in the ruby on rails terminology), although it's quite easy to implement . for now, you're condemned to import common methods before the call to sequelize.define() and use sequelize.utils._.extend() to add it to the instancemethods or classmethods object: // in models/friendlyurl.js module.exports = function(keys) { return { geturl: function() { var ret = ''; keys.foreach(function(key) { ret += this[key]; }) return ret .tolowercase() .replace(/^\s+|\s+$/g, "") // trim whitespace .replace(/[_|\s]+/g, "-") .replace(/[^a-z0-9-]+/g, "") .replace(/[-]+/g, "-") .replace(/^-+|-+$/g, ""); } }; } // in models/user.js var friendlyurlmethods = require('./friendlyurl')(['first_name', 'last_name']); module.exports = function(sequelize, datatypes) { return sequelize.define('user', { first_name: datatypes.string, last_name: datatypes.string, }, { instancemethods: sequelize.utils._.extend({}, friendlyurlmethods, { counttasks: function() { return this.__factory.associations['tasks'].target.count({ where: { user_id: this.id } }); } }); }) }; now the user model instances gain access to a geturl() method: var user = user.build({ first_name: 'john', last_name: 'doe' }); user.geturl(); // 'john_doe' a limitation of this trick is that you must define behaviors before the actual model. this forbids behaviors accessing other models. query series sequelize provides a tool called the querychainer to ease the resynchronization of queries. new sequelize.utils.querychainer() .add(user, 'find', [id]) .add(task, 'findall') .error(function(err) { /* hmm not good :> */ }) .success(function(results) { var user = results[0]; var tasks = results[1]; // do things with the results }); after using it a little, this utility turns out to be very limited. most notably, querychainer executes all queries in parallel by default. and you only get access to the results of the queries in the final callback - no way to pass values from one query to the other. we've found it much more convenient to use a generic resynchronizing module like async , which provides the wonderful async.auto() utility. this method lets you list tasks together with dependencies, and determines which task can be run in parallel, and which must be run in series. async.auto([ user: function(next) { user.find(id).complete(next); }, tasks: ['user', function(next) { tasks.findall({ where: { user_id: user.id } }).complete(next); }] ], function(err, results) { var user = results.user; var tasks = results.tasks; // do things with the results }); notice the complete() method, which is an alternative to the two success() and error() callbacks. complete() accepts a callback with the signature (err, res) , which is more natural in the node.js world, and compatible with async . prefetching one thing orms are usually good at is minimizing queries. sequelize offers a prefetching feature, allowing to group two queries in a single one using a join. for instance, if you want to retrieve a task together with the related user, write the query as follows: task.find({ where: { id: id } }, include: ['user']) .error(function(err) { // error callback }) .success(function(task) { task.getuser(); // does not trigger a new query }); this is another undocumented feature, although the documentation should be updated soon . migrations sequelize provides a migration command line utility. but because it only allows modifying the model using sequelize commands (and not calling any asynchronous method of your own ), this migration command falls short. as of now, we've been handling migrations manually using numbered sql files and a custom utility to run them in order. custom sql queries sequelize is built over database adapters, and as such provides a way to execute arbitrary sql queries against the database. here is an example: var util = require('util'); var query = 'select * from `task` ' + 'left join `user` on `task`.`userid` = `user`.`id` ' + 'where `user`.`last_name` = %s'; var escapedname = sequelize.constructor.utils.escape(last_name); querywithparams = util.format(query, escapedname); sequelize.query(querywithparams, task) .error(function(err) { // error callback }) .success(function(tasks) { task.getuser(); // does not trigger a new query }); sequelize.query() returns a promise just like other query functions. if you provide the model to use for hydration ( task in this case), the query() method returns model instances rather than a simple json. note that you must escape values by hand before concatenating them into the sql query. for strings, sequelize.constructor.utils.escape() is your friend. for integers, util.format('%d') should do the trick. conclusion is sequelize ready for prime time ? almost. the learning curve is made longer by an incomplete documentation, but most of the features required by a production-level orm are there. however, i wouldn't recommend it for production just yet if you're not ready to run on your own fork, since the rate at which prs are merged in the sequelize github repository is low.

I've already written about my experiments with Paratest. Paratest is a PHPUnit wrapper that allows you to run tests written for PHPUnit in parallel, making us of multiple processes running on the same machine. In a world where cycle time is an important metric, trading resources to get the test suite to finish earlier is a net gain; especially when you're stepping on unstable stones and run the suite very often. Rationale Our clock speeds aren't going to increase anymore; you should already know that the direction in which the CPU architectures are heading is to provide multiple cores instead. A PHPUnit process executed serially can get to 100% CPU, while executing two processes can theoretically double your performance, in essence halving the time taken for running your test suite. Moreover, while unit tests do not touch external resources, integration and end-to-end ones are relying on the response time of a database, network calls (even towards the same machine), files to be written, and so on. This means the CPU is not being fully utilized, because a percentage of the time is spent waiting for these resources to become available or to respond. The presence of an high iowait makes room for a number of processes greater than the number of cores available. In some cases, the resources you are using do not depend on the local machine, but are provided remotely in unlimited number. Facebook runs tests in parallel on multiple machines, but for an example available to any wallet check out SauceLabs: providing to you many different Selenium Servers from their cloud, so that you can run tests on N different browser instances at the same time. The thing you have to keep in mind is that tests must be totally independent from each other to be executed. Clearing the whole database before a test or counting the rows of an entire table aren't a suitable option when parallelizing, as tests would interfere with each other. Recent contributions Paratest delegates to a Runner object the execution of a test suite, and collects log files to aggregate the results into a single count of passed and failed tests (among with other statistics such as assertions, errors and time). The default runner creates a new php process for each test class (or even test method with the --functional configuration). This is fine for large scale tests that take in the order of tens of seconds to be run, but there is indeed a slow down with unit tests. This decrease in performance is noticeable whenever there is a sizeable PHPUnit bootstrap file, which is executed again for each of the tests. For this reason, I contributed a new Runner, named WrapperRunner. This Runner executes only a fixed number of processes (which I called Workers, with common terminology) and that take PHPUnit commands on their standard input: $ bin/phpunit-wrapper phpunit tests/MyTest.php ... phpunit tests/OtherTest.php ... Since the bootstrap file is require_once()d into the process, it is only executed once for each Worker, so if you have 1000 tests executed in 5 processes it will be executed 5 times instead of 1000. The tests are capable of reusing the same bootstrap because they previously ran serially after a single execution of it: if they are capable of running in parallel at all, they should share the bootstrap flawlessly. I will probably use Gearman in the future to manage workers as Behat does, but for now this is a very easy-to-install solution: only PHP code. With this contribution, Paratest gets back to a comparable time to PHPUnit for unit tests, while before it was doubling the (short) time required. The more cores we get, the more this time could go down. The WrapperRunner is currently an open PR. Another contribution from Dimitris Baltas and that I ported also to the new WrapperRunner regards providing an environment variable named TEST_TOKEN. If tests are using a shared resource such as a database table, they cannot easily run in parallel if they are working on the same data or reading globally, for example counting the documents in a Mongo collection The TEST_TOKEN shuld be used to map each process to a different instance of the shared resource: for example, a set of schemas database1, database2, ..., database5. The token is guaranteed to be given to a single test at any time in both Runners. This pull request is open too and will be merged into Paratest soon.

ListenableFuture in Guava is an attempt to define consistent API for Future objects to register completion callbacks. With the ability to add callback when Future completes, we can asynchronously and effectively respond to incoming events. If your application is highly concurrent with lots of future objects, I strongly recommend using ListenableFuture whenever you can. Technically ListenableFuture extends Future interface by adding simple void addListener(Runnable listener, Executor executor) method. That's it. If you get a hold of ListenableFuture you can register Runnable to be executed immediately when future in question completes. You must also supply Executor (ExecutorService extends it) that will be used to execute your listener - so that long-running listeners do not occupy your worker threads. Let's put that into action. We will start by refactoring our first example of web crawler to use ListenableFuture. Fortunately in case of thread pools it's just a matter of wrapping them using MoreExecutors.listeningDecorator(): ListeningExecutorService pool = MoreExecutors.listeningDecorator(Executors.newFixedThreadPool(10)); for (final URL siteUrl : topSites) { final ListenableFuture future = pool.submit(new Callable() { @Override public String call() throws Exception { return IOUtils.toString(siteUrl, StandardCharsets.UTF_8); } }); future.addListener(new Runnable() { @Override public void run() { try { final String contents = future.get(); //...process web site contents } catch (InterruptedException e) { log.error("Interrupted", e); } catch (ExecutionException e) { log.error("Exception in task", e.getCause()); } } }, MoreExecutors.sameThreadExecutor()); } There are several interesting observations to make. First of all notice how ListeningExecutorService wraps existing Executor. This is similar to ExecutorCompletionService approach. Later on we register custom Runnable to be notified when each and every task finishes. Secondly notice how ugly error handling becomes: we have to handle InterruptedException (which should technically never happen as Future is already resolved and get() will never throw it) and ExecutionException. We haven't covered that yet, but Future must somehow handle exceptions occurring during asynchronous computation. Such exceptions are wrapped in ExecutionException (thus the getCause() invocation during logging) thrown from get(). Finally notice MoreExecutors.sameThreadExecutor() being used. It's a handy abstraction which you can use every time some API wants to use an Executor/ExecutorService (presumably thread pool) while you are fine with using current thread. This is especially useful during unit testing - even if your production code uses asynchronous tasks, during tests you can run everything from the same thread. No matter how handy it is, whole code seems a bit cluttered. Fortunately there is a simple utility method in fantastic Futures class: Futures.addCallback(future, new FutureCallback() { @Override public void onSuccess(String contents) { //...process web site contents } @Override public void onFailure(Throwable throwable) { log.error("Exception in task", throwable); } }); FutureCallback is a much simpler abstraction to work with, resolves future and does exception handling for you. Also you can still supply custom thread pool for listeners if you want. If you are stuck with some legacy API that still returns Future you may try JdkFutureAdapters.listenInPoolThread() which is an adapter converting plain Future to ListenableFuture. But keep in mind that once you start using addListener(), each such adapter will require one thread exclusively to work so this solution doesn't scale at all and you should avoid it. Future future = //... ListenableFuture listenableFuture = JdkFutureAdapters.listenInPoolThread(future); Once we understand the basics we can dive deeply into biggest strength of listening futures: transformations and chaining. This is advanced stuff, you have been warned.

Starting with Spring 3.1, the new Environment and PropertySource abstractions simplify working with properties.

It didn’t take long. A few months after we released an open source continuous integration tool (Anthill) in 2001, we were asked, “It’s great that I have the build setup, now how do I deploy to the test lab?” That email started a clear transformation in our thinking. Lesson 1: Builds generally exist to be tested or released to customers.The corollary is: Continuous integration is not about build, it is about quality and checking quality generally requires a deployment or six. In 2005, we updated our AnthillPro 2.5 with a shiny new deployment capability. It worked ok, but with that generation of tool being so build oriented, it was never a clean fit. Lesson 2: Deployments are a serious challenge, and can not be bolted on to a build tool. This lesson has been reinforced over the years as we’ve watched the results of various tools tacking on deployments. In 2006, we released AnthillPro 3. Oh boy, what a change. Now deployments and other stuff you do to builds days and weeks later were their own thing. Demonstrating a stunning lack of marketing savvy, we called them “non-originating workflows” (processes that don’t make builds) and declared a new type of tool, an “Application Lifecycle Automation Server”. Today, you’d call it a Continuous Delivery tool (thank you Jez and Dave for the better name). To my knowledge, AnthillPro was the first tool to really take a build lifecycle or build pipeline seriously and had the ambition to manage from continuous integration build through to production release quickly and with a solid audit trail. That brings us to Lesson 3: When you come from the development side of the house as either an engineer or a vendor, you have a lot to learn about audit and security. Around the same time we also learned a great deal about the Dev / Ops gap. While some of our customers implemented the tool as intended, most of the time it was used either by a dev organization doing continuous integration and deployment to the early test environments or a production support / release management group that only did the “official” builds and focused on the production release. The failure of the developer initiated efforts to get to production provided a key insight. Lesson 4: When it comes to deployment automation, start with production, and work your way to the lower environments, treating them as a simpler case of the hard problem. We also learned about the limitations of a build pipeline approach in complex applications. Lesson 5: Deployments of complex systems often require a coordinated release of many builds. The need to start at production style deployments (and not care as much about the build) and the emphasis on deployment time dependencies focused our efforts on a deployment centric tool, uDeploy to compliment the pipeline approach in AnthillPro. Production and scaled usage come together to make the deployment tool a critical piece of infrastructure. If you lose a datacenter, can you recover the tool and use it to recover other applications into a backup datacenter?Lesson 6: Your release and deployment systems need to be configured with high availability and recoverability in mind. For the better part of the last decade, we’ve been trying to streamline deployments and kill off the release weekend. Many of customers have eliminated or shrunk those big events down to a more manageable size. Big releases though still can require the coordination of dozens of people across various teams. One time events like configuring the deployment tool with the password for a new database in each environment, or routine manual steps like putting eyeballs on the new production system before putting it out to the public still need to be managed. Lesson 7: Deployment is still just one part of release. With that in mind, we’ve added uRelease to the portfolio. Different people in the application delivery value chain need different tools. Developers need continuous integration and great testing. Change management needs great auditing around production deployments. Testers need their environments to be not broken, updated regularly and close enough to production to make their testing worthwhile. The patterns and needs overlap. Everyone needs deployments that work. Everyone benefits from understanding the lifecycle of a build (or other version) as well as what is in a given environment. Everyone benefits when release planning meetings are shorter. Lesson 8: No single tool is enough. An integrated toolchain that exchanges information is required. So there you go. Twelve years of tooling, eight with deployments. Summed up in six over-simplified lessons. I’m sure we’ll be updating this next year. Lesson 1: Builds generally exist to be tested or released to customers. Lesson 2: Deployments are a serious challenge, and can not be bolted on to a build tool. Lesson 3: When you come from the development side of the house as either an engineer or a vendor, you have a lot to learn about audit and security. Lesson 4: When it comes to deployment automation, start with production, and work your way to the lower environments, treating them as a simpler case of the hard problem. Lesson 5: Deployments of complex systems often require a coordinated release of many builds. Lesson 6: Release and deployment systems need to be configured with high availability and recoverability in mind. Lesson 7: Deployment is still just one part of release Lesson 8: No single tool does “DevOps”. An integrated toolchain that exchanges information is required. What are the biggest lessons you’ve learned about deployments in the last few years? Share in the comments below!

talend has several connectors to integrate sap systems. however, this guide is no introduction to talend’s sap components. instead, this guide helps to understand different alternatives to integrate sap systems with talend set up a local sap system configure talend studio for using sap components use talend’s sap wizard run a first talend job which connects to sap all further required information and example use cases for talend’s sap components should be available in the talend component guide at www.help.talend.com . if that’s not the case, please create a jira documentation ticket ( https://jira.talendforge.org/browse/doct )! now let’s take a look at different alternatives for integration of sap systems with talend. alternatives for sap integration three protocols exist for communication between sap and external programs: dynamic information and action gateway (diag): e.g. used by sap gui remote function call (rfc): a function call with input and output parameters (like a java interface) hypertext transfer protocol (http): internet standard the following alternatives are available for integrating sap systems using some of these protocols. file sap supports the direct import of files (call-transaction-program, batch-input, direct input). files have to be in a specific format to be imported. transformation and integration can be realized with talend’s various file components such as tfileinputdelimited. rfc remote function call is the proprietary sap ag interface for communication between a sap system and other sap or third-party compatible system over tcp/ip or cpi-c connections. remote function calls may be associated with sap software and abap programming, and provide a way for an external program (written in languages such as php, asp, java, or c, c++) to use data returned from the server. data transactions are not limited to getting data from the server, but can insert data into server records as well. sap can act as the client or server in an rfc call. a remote function call (rfc) is the call or remote execution of a remote function module in an external system. in the sap system, these functions are provided by the rfc interface system. the rfc interface system enables function calls between two sap systems, or between a sap system and an external system. tsapinput and tsapoutput are talend’s components to use rfcs. business application programming interface (bapi) a bapi is an object-oriented view on most data and transactions of a sap system (called “business objects”). object types of the business objects are stored in the business object repository (bor). bapis are always implemented as rfcs and therefore can be called the same way. additionally, they have the following characteristics (compared to rfcs): stable interface no view layer no exceptions, instead export parameter: “return” most business objects offer the following standard bapis: getlist getdetail change creationfromdata tsapinput and tsapoutput are talend’s components to use bapis. application link enabling (ale) application link enabling (ale) is used for asynchronous messaging between different systems via “intermediate documents (idoc)”. idoc is a sap document format for business transaction data transfers. it is used to realize distributed business processes. idoc is similar to xml in purpose, but differs in syntax. both serve the purpose of data exchange and automation in computer systems, but the idoc technology takes a different approach. while xml allows having some metadata about the document itself, an idoc is obligated to have information at its header like its creator, creation time, etc. while xml has a tag-like tree structure containing data and meta-data, idocs use a table with the data and meta-data. idocs also have a session that explains all the processes which the document passed or will pass, allowing one to debug and trace the status of the document. an idoc consists of control record (it contains the type of idoc, port of the partner, release of sap r/3 which produced the idoc, etc.) data records of different types. the number and type of segments is mostly fixed for each idoc type, but there is some flexibility (for example an sd order can have any number of items). status records containing messages such as 'idoc created', 'the recipient exists', 'idoc was successfully passed to the port', 'could not book the invoice because...' different idoc types are available to handle different types of messages. for example, the idoc format orders01 may be used for both purchase orders and order confirmations. tsapidocinput and tsapidocoutput are talend’s components to use ale / idoc. bapis can also be called asynchronously via ale. all new idocs are even based on bapis. soap web services sap supports soap web services. not just sap as java, but also sap as abap! integration can be realized with talend’s esb / web service components such as tesbrequest, tesbresponse, or tesbconsumer. installation of sap server and client installation can take about 6 to 8 hours, but it is an “all in one installation”, i.e. you can install it overnight. steps for installation: get yourself a windows 7 64 bit laptop or vm with 8+ gb ram and 50+gb free disc space get a sap community account (for free, just register): http://scn.sap.com/welcome download sap netweaver (software downloads --> sap netweaver main releases: http://www.sdn.sap.com/irj/scn/nw-downloads download current version of sap netweaver application server abap 64-bit trial install sap server: follow installation guide – a html website included in the download in root of extracted download folder (start.htm --> there click on “installation” link) install sap gui (rich client frontend): start.htm --> there click on “install sap gui” link and follow instructions download the sap jco for the operating system on which your connector is running. the sap jco is available for download from sap's website at http://service.sap.com/connectors . you must have an sapnet account to access the sap jco (if you do not already have one, contact your local sap basis administrator). usage of sap server hint: you have to use a windows user which has a password (as you need to enter windows credentials when stopping sap). if you have a windows user without a password (for instance if you use windows within a vm on your mac), sap cannot process these credentials (i.e. it cannot process an empty password field) --> change your windows password before starting sap start the management console (windows startmenu --> programs --> sap management console) start and stop the sap server (right click on “nsp” --> start / stop) default user: sap* (sap system super user) password: the one which you entered at installation of sap netweaver, e.g. admin123 usage of sap client a sap client should be used to get information about the sap system (functions, data, etc.) similarly to using e.g. mysql workbench to get information from a mysql database. sap gui (view layer) communicates with sap as abap (business logic layer). the application server communicates with the relational database (db layer). different clients are available for sap: sap gui windows sap gui java web browser external rfc-program for local development demos, sap gui windows is probably the best alternative. start sap gui windows by: clicking shortcut “windows start menu --> sap frontend --> sap logon” entering username and password clicking logon sap transactions in sap, you call sap programs via sap transaction codes. important transactions codes are for example: bapi: bapi explorer, view all sap bapi's se16: data browser, view/add table data se38: program editor here is a list of several other important transaction codes: http://www.sapdev.co.uk/tcodes/tcodes.htm installation of demo data the sap installation includes some demo data. as most people do not want to install “real” sap modules such as sap fi, sap crm or sap bi on their local system, this demo data is perfect for demos using talend’s sap connectors. to install the flight demo on a local sap system, you just have to open the abap editor (transaction: se38) and execute the program sapbc_data_generator. this program generates example data within the flight tables and does some further initializations. here is a good tutorial with more information and how to test the flight application: http://help.sap.com/saphelp_erp60_sp/helpdata/de/db/7c623cf568896be10000000a11405a/content.htm configuration of talend studio to use sap components talend’s sap components are already included in the studio. however, two further steps are required to be able to use them: copy sapjco3.dll to the directory c:/windows/system32 sap java connector jar must be added copy sapjco3.jar to the directory “talend/studio/lib/java” (re-) start talend studio check if sap library is added successfully open view “talend modules” (eclipse --> windows --> show view --> talend --> modules) sort by column “context” look for “tsap*” contexts and check if sapjco3.jar has status “installed” usage of sap components with talend studio this section describes how to use talend’s sap components and the sap wizard in general (using one specific example for calling a bapi). detailed descriptions of all sap components (for using bapis, rfcs, idocs, bw, etc.) are available in the documentation talend_components_rg_x.y.z.pdf at www.help.talend.com . connection to a sap system a connection to a sap system can be done “built-in” or via “metadata --> sap connections” (the latter only in enterprise version). using the latter has several advantages: reuse connection configuration quick check if connection to sap works wizards for retrieving functions from sap (instead of handwriting without wizard) quick test with test parameters if function works before finishing development lifecycle for a sap job development lifecycle for sap job: create connection (if not existing yet) right click on metadata --> sap connection create sap connection follow wizard sap jco version: 3 client: “001” userid: “sap*” password: “admin123” --> as you defined it while installation language: “en” hostname: “localhost” system number: “00” retrieve function (bapi / rfc) right click on created connection click on “retrieve sap function” enter search filter (e.g. bapi_fl*) click on “search” select and double click on your function (e.g bapi_flcust_getlist) you see all input, output and table parameters for this sap function click on “test in” --> here you see parameters in more detail: you now have to define which input and output parameters you want to use --> remove all other by selecting them and clicking “remove” button hint: if you do not remove an input parameter, you usually have to enter a value for it! select the output type - can be a single (single record), a table (list of records), or a structure output hint: difference between table and structure in sap: http://www.sapfans.com/forums/viewtopic.php?f=12&t=119794 if you want to do a quick test: enter values for input parameters (if there are any for your function call), then click “launch” button in this example, there is only an optional input parameter max_rows you should see data in the output fields in this example, you see the record with custname “sap ag” and street “neurottstr. 16” click “finish” button under “metadata --> sap connections --> “your connection” --> sap functions: there you can now see your function (in this example: bapi_flcust_getlist) create sap job drag&drop the created function into a job (without the wizard, you also can enter all data by hand) tsapinput component is proposed automatically. click ok to add it to your job go to “initialize input” and add parameter values in this example, there is just the parameter “max_rows” hint: the parameter value can be changed from a hardcoded value to a variable, of course (just click control space on your keyboard to get access to all available variables via code completion in your studio) go to the tsapinput component and add the desired output mapping (i.e. which values you want to process further with other components scroll to the bottom to “outputs” add the correct table / structure name (in this example: "customer_list") click on mapping (which is empty and has to be filled) click on “mapping”, then click on “…” add the wanted output columns of your sap function add the same names at the column “schema xpathqueries” (do not forget the double quotes here!) click “ok” button connect the tsapinput component to a tlogrowcomponent and synchronize the schema hint: always try out if this works before adding further logic to your job! run and test your job (you will see five rows logged (as you have configured max_rows = 5 that's it. now enjoy talend's sap components :-) best regards, kai wähner (twitter: @kaiwaehner) content from my blog: http://www.kai-waehner.de/blog/2013/03/03/sap-integration-with-talend-components-connectors-bapi-rfc-idoc-bw-soap/

I'll try to explain how I have developed the French, German and Spanish dictionaries for the NetBeans online spellchecker : we will build a French dictionary for NetBeans 7.3 (it works with 7.2 and 7.1 too). Summary : #1 expand GNU Aspell dictionaries files. #2 checkout the NetBeans 7.3.0 FCS sources from Mercurial repository. #3 open the English dictionary project and make a copy. #4 modify the copied project. #5 make the NBM file and test it. #6 (optional) sign the NBM file and submit it to the community for validation. Step 1 : expand GNU Aspell dictionaries files Download Aspell. You can get the latest Win32 installer version from ftp://ftp.gnu.org(...)Aspell-0-50-3-3-Setup.exe. Run it to install Aspell. Download the latest French dictionaries file. You can get Win32 installers from ftp://ftp.gnu.org/gnu/aspell/w32/. The latest French dictionnaries file is Aspell-fr-0.50-3-3.exe. Run it to install the French dictionaries into Aspell. You can now expand the dictionaries files you want to include in the future NetBeans plugin. We'll expand the "fr_FR" and "fr_CH" dictionaries. Go to the "dict" directory of Aspell and run the following commands (if necessary, add the Aspell "bin" directory to your PATH variable, in the Operating System or a batch script) : aspell --lang=fr_FR --master=fr_FR dump master | sort > aspell_dump_fr_FR.txt aspell --lang=fr_CH --master=fr_CH dump master | sort > aspell_dump_fr_CH.txt The first line will expand and sort the "fr_FR" dictionary to the default output. The > switch is used to save the output to a file (aspell_dump_fr_FR.txt), in the current directory. The second line does the same job with the "fr_CH" language. You'll note that the expanded dictionaries files may not be UTF-8 encoded. If necessary, re-encode them to UTF-8. You can do it with Notepad2 : open a file and go to File, Encoding, and select the UTF-8 encoding. It will encode the opened file. To finish, pack the two files into a ZIP file (you can do it with every ZIP archiver, like 7-Zip). We will call this archive "aspell-frwl.zip" : Nota n°1 : What does "expand a dictionary file" means ? Aspell dictionaries files are a set of words and affixes lists. Word lists contains basic forms of common words. Affixes are used to compute the different variations of words. By expanding a dictionary , we ask Aspell to compute a list of all words with all their variations. The result is a huge file. We need to expand dictionaries files because the NetBeans Spellchecker (seems to) use only expanded dictionaries files : it doesn't support affixes lists ;) Nota n°2 : These are the MS Windows instructions. Linux and MacOS ones should be easy to find. Step 2 : checkout the NetBeans 7.3.0 FCS sources from Mercurial repository Refer to the given tutorial : How to build NetBeans from sources, step 1 only. After that, you need at least two directories : "releases/nbbuild/" and "releases/spellchecker.dictionary_en/". If you want to free some space, you can delete the other directories. You can now start your NetBeans IDE and load the "spellchecker.dictionary_en" project. Step 3 : open the English dictionary project and make a copy NetBeans will ask you for a new project name. Choose something like "spellchecker.dictionary_fr" : Actually, the project name you have chosen is the project's folder name. NetBeans will show you the "SpellChecker English Dictionaries (0)" title for your project. Rename it (you can use the F2 key on the project's name) to "SpellChecker French Dictionaries" : You now have the two projects, English and French dictionaries plugin projects : Step 4 : modify the copied project The French project is correctly named, so we can now modify its content to target French dictionaries. Switch to the "File" project tab to show more files. Step 4.1 : empty the "external" directory's content and copy your French dictionaries archive file This directory contains the English dictionaries files. We will provide our own files. Delete the files located into the "external" directory and copy the "aspell-frwl.zip" created during step 1. Step 4.2 : edit the "nbproject/project.properties" file Replace : release.external/ispell-enwl-3.1.20.zip=modules/dict/ispell-enwl-3.1.20.zip jnlp.indirect.files=modules/dict/dictionary_en_US.description,modules/dict/dictionary_en_GB.description,modules/dict/ispell-enwl-3.1.20.zip,modules/dict/dictionary_en.description by : release.external/aspell-frwl.zip=modules/dict/aspell-frwl.zip jnlp.indirect.files=modules/dict/dictionary_fr_FR.description,modules/dict/dictionary_fr_CH.description,modules/dict/aspell-frwl.zip,modules/dict/dictionary_fr.description We have replaced references to English dictionary and locales by French ones. Step 4.3 : edit the "nbproject/project.xml" file Replace : org.netbeans.modules.spellchecker.dictionary_en by : org.netbeans.modules.spellchecker.dictionary_fr Step 4.4 : rename the "src/org/netbeans/modules/spellchecker/dictionary_en/" folder Rename the "dictionary_en" folder to "dictionary_fr". Step 4.5 : edit the "src/org/netbeans/modules/spellchecker/dictionary_fr/Bundle.properties" file Replace : OpenIDE-Module-Display-Category=Base IDE OpenIDE-Module-Long-Description=\ Provides Ispell's (ver. 3.1.20) word list for use in the online spellchecker. OpenIDE-Module-Name=Spellchecker English Dictionaries OpenIDE-Module-Short-Description=English Dictionaries for Spellchecker by something like : OpenIDE-Module-Display-Category=Base IDE OpenIDE-Module-Long-Description=\ Provides Aspell's French word list for use in the online spellchecker. OpenIDE-Module-Name=Spellchecker French Dictionaries OpenIDE-Module-Short-Description=French Dictionaries for Spellchecker This file describes your plugin. Do not hesitate to change the description fields. Step 4.6 : edit the "build.xml" file Replace : by : Step 4.7 : edit the "manifest.mf" file Replace : Manifest-Version: 1.0 OpenIDE-Module: org.netbeans.modules.spellchecker.dictionary_en XOpenIDE-Module-Layer: org/netbeans/modules/spellchecker/dictionary_en/layer.xml OpenIDE-Module-Localizing-Bundle: org/netbeans/modules/spellchecker/dictionary_en/Bundle.properties OpenIDE-Module-Specification-Version: 1.8.1 by : Manifest-Version: 1.0 OpenIDE-Module: org.netbeans.modules.spellchecker.dictionary_fr XOpenIDE-Module-Layer: org/netbeans/modules/spellchecker/dictionary_fr/layer.xml OpenIDE-Module-Localizing-Bundle: org/netbeans/modules/spellchecker/dictionary_fr/Bundle.properties OpenIDE-Module-Specification-Version: 1.0 Please note we have changed the plugin version, from 1.8.1 to 1.0. Step 4.8 : rename and edit the "release/module/dict/dictionary_en.description" file Rename the "dictionary_en.description" file to "dictionary_fr.description". After that, replace : jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/english.0 jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/english.1 jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/english.2 jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/english.3 jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/american.0 jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/american.1 jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/american.2 by : jar:nbinst:///modules/dict/aspell-frwl.zip!/aspell_dump_fr_FR.txt jar:nbinst:///modules/dict/aspell-frwl.zip!/aspell_dump_fr_CH.txt Some explanations : our plugin will contain dictionaries for the "fr", "fr_FR" and "fr_CH" locales. We have a ".description" file for each of them. In these files, we locate the word list(s) to use. So, the "fr" locale is the combination of the "fr_FR" and "fr_CH" word lists. The "fr_FR" locale uses the "fr_FR" word list, and the "fr_CH" locale uses the "fr_CH" word list. Here, we have edited the description file of the "fr" locale. Let's do it for the other ones :) Nota : The "aspell-frwl.zip!" syntax means that we enter into a ZIP file. Step 4.9 : rename and edit the "release/module/dict/dictionary_en_GB.description" file Rename the "dictionary_en_GB.description" file to "dictionary_fr_FR.description". After that, replace : jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/english.0 jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/english.1 jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/english.2 jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/english.3 jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/british.0 jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/british.1 jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/british.2 by : jar:nbinst:///modules/dict/aspell-frwl.zip!/aspell_dump_fr_FR.txt Step 4.10 : rename and edit the "release/module/dict/dictionary_en_US.description" file Rename the "dictionary_en_US.description" file to "dictionary_fr_CH.description". After that, replace : jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/english.0 jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/english.1 jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/english.2 jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/english.3 jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/american.0 jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/american.1 jar:nbinst:///modules/dict/ispell-enwl-3.1.20.zip!/american.2 by : jar:nbinst:///modules/dict/aspell-frwl.zip!/aspell_dump_fr_CH.txt Step 5 : make the NBM file and test it You can now launch the Build action to compile the project, and the Create NBM action to package your plugin into a NBM file ("build/org-netbeans-modules-spellchecker-dictionary_fr.nbm"). To test it : go to the Plugins Manager ("Tools" / "Plugins"), the "Downloaded" tab, the "Add Plugins..." button, load your NBM file and confirm : your plugin is now installed ! you can now change the spellchecker default locale, open a text file and type a letter (only !), wait a few seconds to let Netbeans generate a cache file for the selected locale (if you don't wait, the cache creation will fail and the spellchecker won't work), and try to use the spellchecker correction. This is explained in my French Dictionary Plugin page. Nota : If you don't wait the cache creation, the spellchecker won't work for the selected locale. You can go to your ".netbeans/7.x.y/var/cache/dict/" directory and delete the corresponding ".trie1" (or ".trie2") file. Restart NetBeans to let it to recreate a cache file. Step 6 : (optional) sign the NBM file and submit it to the community for validation You can now publish your plugin on the NetBeans Plugins Portal (don't forget create an account). To make it available in the NetBeans Plugins Manager (Tools / Plugins), you have to submit it to validation. Firstly, you have to sign your plugin (generate a certificate file and sign your plugin) : you'll find a tutorial at http://wiki.netbeans.org/DevFaqSignNbm. Example : keytool -genkey -storepass PASSWORD -alias ALIAS -keystore FOO.cert -validity 3651 (it will create the "FOO.cert" certificate file with a 3651 days validity, the "ALIAS" alias an the "PASSWORD" password. The tool will ask you additional information). Then, you can now upload your plugin to the NetBeans Plugins Portal and ask for a validation. Once validated, your plugin will be available in the NetBeans Plugins Manager.

In continuation of my earlier blogs on Introduction to Spring MVC and Testing DAO layer in Spring MVC, in this blog I will demonstrate how to test Service layer in Spring MVC. The objective of this demo is 2 fold, to build the Service layer using TDD and increase the code coverage during JUnit testing of Service layer. For people in hurry, get the latest code from Github and run the below command mvn clean test -Dtest=com.example.bookstore.service.AccountServiceTest Since in my earlier blog, we have already tested the DAO layer, in this blog we only need to focus on testing service layer. We need to mock the DAO layer so that we can control the behavior in Service layer and cover various scenarios. Mockito is a good framework which is used to mock a method and return known data and assert that in the JUnit. As a first step we define the AccountServiceTestContextConfiguration class with AccountServiceTest class. If you notice there are 2 beans defined in that class and we marked the as a @Configuration which shows that it is a Spring Context class. In the JUnit test we @Autowired AccountService class. And AccountServiceImpl @Autowired the AccountRepository class. When creating the Bean in the configuration file we also stubbed the AccountRepository class using Mockito, @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration public class AccountServiceTest { @Configuration static class AccountServiceTestContextConfiguration { @Bean public AccountService accountService() { return new AccountServiceImpl(); } @Bean public AccountRepository accountRepository() { return Mockito.mock(AccountRepository.class); } } //We Autowired the AccountService bean so that it is injected from the configuration @Autowired private AccountService accountService; @Autowired private AccountRepository accountRepository; During the setup of the JUnit we use Mockito mock findByUsername method to return a predefined account object as below @Before public void setup() { Account account = new AccountBuilder() { { address("Herve", "4650", "Rue de la gare", "1", null, "Belgium"); credentials("john", "secret"); name("John", "Doe"); } }.build(true); Mockito.when(accountRepository.findByUsername("john")).thenReturn(account); } Now we write the tests as below and test both the positive and negative scenarios, @Test(expected = AuthenticationException.class) public void testLoginFailure() throws AuthenticationException { accountService.login("john", "fail"); } @Test() public void testLoginSuccess() throws AuthenticationException { Account account = accountService.login("john", "secret"); assertEquals("John", account.getFirstName()); assertEquals("Doe", account.getLastName()); } } Finally we verify if the findByUsername method is called only once successfully as below in the teardown, @After public void verify() { Mockito.verify(accountRepository, VerificationModeFactory.times(1)).findByUsername(Mockito.anyString()); // This is allowed here: using container injected mocks Mockito.reset(accountRepository); } AccountService class looks as below, @Service @Transactional(readOnly = true) public class AccountServiceImpl implements AccountService { @Autowired private AccountRepository accountRepository; @Override public Account login(String username, String password) throws AuthenticationException { Account account = this.accountRepository.findByUsername(username, password); } else { throw new AuthenticationException("Wrong username/password", "invalid.username"); } return account; } } I hope this blog helped you. In my next blog, I will demo how to build a controller JUnit test. Reference: Pro Spring MVC: With Web Flow by by Marten Deinum, Koen Serneels

We all love wizards.... (Software wizards I mean). We are always happy to jump on those ''Next" buttons like we were dancing the funky chicken on our… well you get the point. So today we bring you your beloved wizard into your coding experience. Let's jump right into an example. Say you want to design a ConservativePerson class. view source print? import java.util.List; class ConservativePerson{ private boolean isVirgin; private boolean isMarried; private List children; ConservativePerson(boolean virgin, boolean married, List children) { this.isVirgin = virgin; this.isMarried = married; this.children = children; } public boolean isVirgin() { return isVirgin; } public boolean isMarried() { return isMarried; } public List getChildren() { return children; } } As such it has some constrains. He must be married before he can be... well, not a virgin. He can't be a virgin before he can have children (as far as we know). In the old days, which is basically all days until today..., you would probably define all kinds of modifiers methods for this class which will throw an exception in case of invariant invalidation such as NotMarriedException and VirginException. Not anymore. Today we will do it by using the Wizard Design Pattern. We use a fluent interface style and utilize the power of a modern IDE to create a wizard-like feeling when building a ConservativePerson object. We know, we know, stop talking and show us the code... but before we will present the wizard code we will show you it usage so you will get a grasp of what we are talking about... view source print? public class Main { public static void main(String[] args) { ConservativePersonWizardBuilder wizard = new ConservativePersonWizardBuilder(); ConservativePerson singlePerson = wizard. createConservativePerson(). whichIsSingle(). getObject(); ConservativePerson familyManPerson = wizard. createConservativePerson(). whichIsMarried(). andNotVirgin(). andHasChildNamed("Noa"). anotherChildNamed("Guy"). lastChildName("Alon"). getObject(); } } Now, it may look just like an ordinarily fluent interface, but the cool thing here is that a method is available for calling only if the current object state allows it. This means you will not be able to call the method andNotVirgin if you haven't called the method whichIsMarried. See the following set of screen shots: and after we state he is married we can: Here is the wizard code. I urge you to copy/paste it to your IDE and give it a try by building an object with it. view source print? import java.util.ArrayList; import java.util.List; public class ConservativePersonWizardBuilder { private boolean isVirgin; private boolean isMarried; private List children = new ArrayList(); public SetMarriedStep createConservativePerson(){ return new SetMarriedStep(); } class SetMarriedStep { public SetVirginStep whichIsMarried(){ isMarried = true; return new SetVirginStep(); } public FinalStep whichIsSingle(){ isMarried = false; return new FinalStep(); } } class SetVirginStep { public AddChildrenStep andNotVirgin(){ isVirgin = false; return new AddChildrenStep(); } public FinalStep butStillAVirgin(){ isVirgin = true; return new FinalStep(); } } class FinalStep { public ConservativePerson getObject(){ return new ConservativePerson(isVirgin, isMarried, children); } } class AddChildrenStep { public AddChildrenStep andHasChildNamed(String childName) { children.add(childName); return new AddChildrenStep(); } public AddChildrenStep anotherChildNamed(String childName) { children.add(childName); return new AddChildrenStep(); } public FinalStep lastChildName(String childName){ children.add(childName); return new FinalStep(); } } } As you can see the wizard consists of several steps. Each step is represented by a dedicated inner class. Each step reveals the legal available operations by its methods. Each method will then return a new step according to the change it has made. This way an attempt to create an illegal object will be detected at compile time instead of runtime. This pattern is actually being used in our production code. One example that comes to mind is the MediaJob class. This class describes a manipulation on some media files. In order to submit a job to the system, one has to create a MediaJob object. The problem is that this object has many parameters that could be assigned with contradicting values that create an illegal object state. By using the Wizard pattern, one can easily build a legal job without the need to know the entire (and complicated…) set of constrains. That is all for now. Hope you'll give it a try..... view source print

I have been recently working on a client generator in the wadl.java.net project and I was solving a bug where I wanted to change path parameter on a URI. This turned out to be quite easy with existing classes that are part of the Jersey project. First of all you need a instance of UriTemplate, depending on the very of Jersey this is in a slightly different package - you IDE's automatic import will do the work for you. String template = "http://example.com/name/{name}/age/{age}"; UriTemplate uriTemplate = new UriTemplate(template); Then you use match(...) to extract them: String uri = "http://example.com/name/Bob/age/47"; Map parameters = new HashMap<>(); // Not this method returns false if the URI doesn't match, ignored // for the purposes of the this blog. uriTemplate.match(uri, parameters); Then you can replace the parameters at will and rebuild the URI: parameters.put("name","Arnold"); UriBuilder builder = UriBuilder.fromPath(template); URI output = builder.build(parameters);

In continuation of my blog JUnit testing of Spring MVC application – Introduction, in this blog, I will show how to design and implement DAO layer for the Bookstore Spring MVC web application using Test Driven development. For people in hurry, get the latest code from Github and run the below command mvn clean test -Dtest=com.example.bookstore.repository.JpaBookRepositoryTest As a part of TDD, Write a basic CRUD (create, read, update, delete) operations on a Book DAO class com.example.bookstore.repository.JpaBookRepository. Don’t have the database wiring yet in this DAO class. Once we build the JUnit tests, we use JPA as a persistence layer. We also use H2 as a inmemory database for testing purpose. Create Book POJO class Create the JUnit test as below, public class JpaBookRepositoryTest { @Test public void testFindById() { Book book = bookRepository.findById(this.book.getId()); assertEquals(this.book.getAuthor(), book.getAuthor()); assertEquals(this.book.getDescription(), book.getDescription()); assertEquals(this.book.getIsbn(), book.getIsbn()); } @Test public void testFindByCategory() { List books = bookRepository.findByCategory(category); assertEquals(1, books.size()); for (Book book : books) { assertEquals(this.book.getCategory().getId(), category.getId()); assertEquals(this.book.getAuthor(), book.getAuthor()); assertEquals(this.book.getDescription(), book.getDescription()); assertEquals(this.book.getIsbn(), book.getIsbn()); } } @Test @Rollback(true) public void testStoreBook() { Book book = new BookBuilder() { { description("Something"); author("JohnDoe"); title("John Doe's life"); isbn("1234567890123"); category(category); } }.build(); bookRepository.storeBook(book); Book book1 = bookRepository.findById(book.getId()); assertEquals(book1.getAuthor(), book.getAuthor()); assertEquals(book1.getDescription(), book.getDescription()); assertEquals(book1.getIsbn(), book.getIsbn()); } } If you notice since the JpaBookRepository is only a skeleton class without implementation, all the tests will fail. As a next step, we need to create a Configuration and wire a datasource, and for the test purpose we will be using H2 database. And we also need to wire this back to JUnit test as below, @Configuration public class InfrastructureContextConfiguration { @Autowired private DataSource dataSource; //some more configurations.. @Bean public DataSource dataSource() { EmbeddedDatabaseBuilder builder = new EmbeddedDatabaseBuilder(); builder.setType(EmbeddedDatabaseType.H2); return builder.build(); } } //JUnit test wiring is as below @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(classes = { InfrastructureContextConfiguration.class, TestDataContextConfiguration.class }) @Transactional public class JpaBookRepositoryTest { //the test methods } Next step is to setup and teardown sample data in the JUnit test case as below, public class JpaBookRepositoryTest { @PersistenceContext private EntityManager entityManager; private Book book; private Category category; @Before public void setupData() { EntityBuilderManager.setEntityManager(entityManager); category = new CategoryBuilder() { { name("Evolution"); } }.build(); book = new BookBuilder() { { description("Richard Dawkins' brilliant reformulation of the theory of natural selection"); author("Richard Dawkins"); title("The Selfish Gene: 30th Anniversary Edition"); isbn("9780199291151"); category(category); } }.build(); } @After public void tearDown() { EntityBuilderManager.clearEntityManager(); } } Once we do the wiring, we need to implement the com.example.bookstore.repository.JpaBookRepository and use JPA to do the CRUD on the database and run the tests. The tests will succeed. Finally if you run Cobertura for this example from STS, we will get over 90% of line coverage for com.example.bookstore.repository.JpaBookRepository. In case you want to try few exercises you can implement repository for Account and User. I hope this blog helped you. In my next blog I will talk about Mochito and Implementing the Service layer.

Everyone is talking about the future of Java, we continue our journey explaining Future interface in Java. ExecutorCompletionService wrapper class tries to address one of the biggest deficiencies of Future type - no support for callbacks or any event-driven behaviour whatsoever. Let's go back for a moment to our sample asynchronous task downloading contents of a given URL: final ExecutorService pool = Executors.newFixedThreadPool(10); pool.submit(new Callable() { @Override public String call() throws Exception { try (InputStream input = url.openStream()) { return IOUtils.toString(input, StandardCharsets.UTF_8); } } }); Having such code we can easily write a simple web search engine/crawler, examining several URLs concurrently: final List topSites = Arrays.asList( "www.google.com", "www.youtube.com", "www.yahoo.com", "www.msn.com", "www.wikipedia.org", "www.baidu.com", "www.microsoft.com", "www.qq.com", "www.bing.com", "www.ask.com", "www.adobe.com", "www.taobao.com", "www.youku.com", "www.soso.com", "www.wordpress.com", "www.sohu.com", "www.windows.com", "www.163.com", "www.tudou.com", "www.amazon.com" ); final ExecutorService pool = Executors.newFixedThreadPool(5); List> contentsFutures = new ArrayList<>(topSites.size()); for (final String site : topSites) { final Future contentFuture = pool.submit(new Callable() { @Override public String call() throws Exception { return IOUtils.toString(new URL("http://" + site), StandardCharsets.UTF_8); } }); contentsFutures.add(contentFuture); } As easy as that. We simply submit separate task for each web site to a pool and wait for results. To achieve that we collect all Future objects into a collection and iterate through them: for (Future contentFuture : contentsFutures) { final String content = contentFuture.get(); //...process contents } Each call to contentFuture.get() waits until downloading given web site (remember that each Future represent one site) is finished. This works, but has a major bottleneck. Imagine you have as many threads in a pool as tasks (20 sites in that case). I think it's understandable that you want to start processing contents of web sites as soon as they arrive, no matter which one is first. Response times vary greatly so don't be surprised to find some web sites responding within a second while others need even 20 seconds. But here's the problem: after submitting all the tasks we block on an arbitrary Future. There is no guarantee that this Future will complete first. It is very likely that other Future objects already completed and are ready for processing but we keep hanging on that arbitrary, first Future. In worst case scenario, if the first submitted page is slower by an order of magnitude compared to all the others, all the results except the first one are ready for processing and idle, while we keep waiting for the first one. The obvious solution would be to sort web sites from fastest to slowest and submit them in that order. Then we would be guaranteed that Futures complete in the order in which we submitted them. But this is impractical and almost impossible in real life due to dynamic nature of web. This is where ExecutorCompletionService steps in. It is a thin wrapper around ExecutorService that "remembers" all submitted tasks and allows you to wait for the first completed, as opposed to first submitted task. In a way ExecutorCompletionService keeps a handle to all intermediate Future objects and once any of them finishes, it's returned. Crucial API method is CompletionService.take() that blocks and waits for any underlying Future to complete. Here is the submit step with ExecutorCompletionService: final ExecutorService pool = Executors.newFixedThreadPool(5); final ExecutorCompletionService completionService = new ExecutorCompletionService<>(pool); for (final String site : topSites) { completionService.submit(new Callable() { @Override public String call() throws Exception { return IOUtils.toString(new URL("http://" + site), StandardCharsets.UTF_8); } }); } Notice how we seamlessly switched to completionService. Now the retrieval step: for(int i = 0; i < topSites.size(); ++i) { final Future future = completionService.take(); try { final String content = future.get(); //...process contents } catch (ExecutionException e) { log.warn("Error while downloading", e.getCause()); } } You might be wondering why we need an extra counter? Unfortunately ExecutorCompletionService doesn't tell you how many Future objects are still there waiting so you must remember how many times to call take(). This solution feels much more robust. We process responses immediately when they are ready. take() blocks until fastest task still running finishes. And if processing takes a little bit longer and multiple responses finished, subsequent call to take() will return immediately. It's fun to observe the program when number of pool threads is as big as the number of tasks so that we begin downloading each site at the same time. You can easily see which websites have shortest response time and which respond very slowly. ExecutorCompletionService seems wonderful, but in fact it is quite limited. You cannot use it with arbitrary collection of Future objects which you happened to obtain somehow. It works only with Executor abstraction. Also there is no built in support for processing incoming results concurrently as well. If we want to parse results concurrently, we need to manually submit them to a second thread pool. In the next few articles I will show you more powerful constructs that mitigate these disadvantages.

This post comes from Miguel Angel Nieto at the MySQL Performance Blog. The MySQL optimizer makes the decision of what execution plan to use based on the information provided by the storage engines. That information is not accurate in some engines like InnoDB and they are based in statistics calculations therefore sometimes some tune is needed. In InnoDB these statistics are calculated automatically, check the following blog post for more information: http://www.mysqlperformanceblog.com/2011/10/06/when-does-innodb-update-table-statistics-and-when-it-can-bite/ There are some variables to tune how that statistics are calculated but we need to wait until the gathering process triggers again to see if there is any improvement. Usually the first step to try to get back to the previous execution plan is to force that process with ANALYZE TABLE that is usually fast enough to not cause troubles. Let’s see an example of how a simple and fast ANALYZE can cause a downtime. Waiting for table flush: In order to trigger this problem we need: - Lot of concurrency - A long running query - Run an ANALYZE TABLE on a table accessed by the long running query So first we need a long running query against table t: SELECT * FROM t WHERE c > '%c%'; Then in our efforts to get a better execution plan for another query we run ANALYZE TABLE: mysql> analyze table t; +--------+---------+----------+----------+ | Table | Op | Msg_type | Msg_text | +--------+---------+----------+----------+ | test.t | analyze | status | OK | +--------+---------+----------+----------+ 1 row in set (0.00 sec) Perfect, very fast! But then some seconds later we realize that our application is down. Let’s see the process list. I’ve removed most of the columns to make it clearer: +------+-------------------------+---------------------------------+ | Time | State | Info | | 32 | Writing to net | select * from t where c > '%0%' | | 12 | Waiting for table flush | select * from test.t where i=1 | | 12 | Waiting for table flush | select * from test.t where i=2 | | 12 | Waiting for table flush | select * from test.t where i=3 | | 11 | Waiting for table flush | select * from test.t where i=7 | | 10 | Waiting for table flush | select * from test.t where i=11 | | 11 | Waiting for table flush | select * from test.t where i=5 | | 11 | Waiting for table flush | select * from test.t where i=4 | | 11 | Waiting for table flush | select * from test.t where i=9 | | 11 | Waiting for table flush | select * from test.t where i=8 | | 11 | Waiting for table flush | select * from test.t where i=12 | | 11 | Waiting for table flush | select * from test.t where i=14 | | 10 | Waiting for table flush | select * from test.t where i=6 | | 10 | Waiting for table flush | select * from test.t where i=15 | | 10 | Waiting for table flush | select * from test.t where i=10 | [...] The ANALYZE TABLE runs perfect but after it the rest of the threads that are running a query against that table need to wait. This is because MySQL has detected that the underlying table has changed and it needs to close and reopen it using FLUSH. Therefore the table will be locked until all queries that are using that table finish. There are only two solutions to this situation, wait until the long query finishes or kill the query. Also, we have to take in account that killing a query could cause even more troubles. If we are dealing with a write query on InnoDB the rollback process could take even more time to finish than the original query. On the other hand, if the table is MyISAM there will be no rollback process so all the already updated rows can’t be recovered. This particular example is not only a problem of ANALYZE. Other commands like FLUSH TABLES, ALTER, RENAME, OPTIMIZE or REPAIR can cause threads to wait on “Waiting for tables”, “Waiting for table” and “Waiting for table flush”. Conclusion Before running an ANALYZE table or any other command listed before, check the running queries. If the table that you are going to work on is very used the recommendation is to run it during the low peak of load or a maintenance window.