this mobile application tutorial shows you how to create a user registration, login and logout backend using mongodb and mongoose. this article is part of a series of mobile application development tutorials that i have been publishing on my blog jorgeramon.me, which shows you how to create a meeting room booking mobile application. this app will be used to browse an inventory of meeting rooms and reserve rooms for conference calls and other types of events. the backend that we will create in this article will connect with the user account management screens that we built in a previous chapter of this series . this backend will consist of the following modules: router (using node.js and express ) controller data model to represent a user (using mongoose ) database (using mongodb ) the router receives http requests from the mobile application and forwards them to the controller, which in turn creates, reads, updates and deletes data models defined with the mongoose library . the user profiles and sessions information will reside in a mongodb database . the router also receives data from the controller and bundles it in http responses that it sends to the mobile app. in this article we will create the controller, model and database modules using mongoose and mongodb. we will test the controller and build the router in the next article of this series. let’s proceed to install the software that we will use to create the node js, mongodb and mongoose endpoint. installing node.js node.js is a platform for building network apps that you can use to build backend endpoints for your mobile applications. you can get node.js at node.js . this tutorial doesn’t require you to have extensive knowledge of node, but you should try to learn about it as much as you can in order to take full advantage of its capabilities. to start, i would recommend the tutorials over at node school . installing express express is a framework for building web applications with node.js. express’ installation page shows you how to install the framework. in this particular article we will only use the request routing capabilities of express. in the feature we will take advantage of other features. installing mongodb mongodb is a leading nosql database at the time of this writing. in the databases ecosystem, mongodb falls under the document databases category. these are databases where each record and its associated data is thought of as a “document”. document databases have characteristics that make them a good choice for storing unstructured data across multiple servers. there is abundant online documentation on this subject. if you want to learn more, you can start with the document databases page on mongodb’s website. to install mongodb you need to head to the downloads page on mongodb.org and grab the mongodb installer for your platform. if you haven’t worked with mongo, i recommend that at a minimum you go over mongodb’s interactive tutorial so you become familiar with it. installing mongoose mongoose is a javascript library that makes it easy to move data between your application and mongodb databases. it is a layer of abstraction that allows you to create schemas for the data that your application uses, and provides facilities for connecting to mongodb and validating, saving, updating, deleting and retrieving instances of these schemas. the picture below will give you an idea of where mongoose fits in our application’s architecture: you can find installation instructions and a very good introduction to the library on mongoose’s getting started page. designing the public interface of the controller the role of the controller module in our express backend will be to fulfill requests received from the mobile application: as at this point in this series of tutorials we are only concerned with the user registration, login and logout features of the application, we will create controller methods to handle these functions. we need the controller module to respond to the following requests: register user log on a user log off a user initiate a password reset for a user finalize a password reset for a user based on these requests, we will design a controller with the following public methods. controller.register(newuser, callback): this method will register a new user with the backend by saving the user’s profile in the mongodb database. parameters: newuser – the user to register in the database callback – a function that will receive the results of the registration attempt. returns: callback – the callback function passed in the arguments. controller.logon(email, password, callback): this method will logon a user if the supplied email and password are valid. if the logon attempt succeeds, the method will add the user’s profile to a private “session” variable in the controller. parameters: email – the user’s email. password – the user’s password. callback – a function that will receive the results of the logon attempt. returns: callback – the callback function passed in the arguments. controller.logoff(): this method will log off a user by delete the user’s profile data stored in the controller’s private “session” variable. controller.resetpassword(email, callback): this method will send the user an email containing a password reset link. the link will contain a unique identifier string that will be used in the controller.resetpasswordfinal method. parameters: email – the user’s email address. callback – a function that will receive the results of the reset password attempt. returns: callback – the callback function passed in the arguments. controller.resetpasswordfinal(email, newpassword, passwordresethash, callback): this method will reset a user’s password. parameters: email – the user’s email address. newpassword – the user’s new password. passwordresethash – a unique identifier sent to the user via email from the controller.resetpassword method. callback – a function that will receive the results of the reset password attempt. returns: callback – the callback function passed in the arguments. controller.setsession(session): this method will set the controller’s private “session” variable. parameters: session – the value for the controller’s “session” variable. controller.getsession(): this method will return a reference to the controller’s private session variable. returns: session – the internal “session” variable. creating a model using mongoose as explained in the mongoose documentation , the mongoose model automatically inherits a number of methods (such as create, save, remove and find) that allow us to store and retrieve model instances from a mongodb database. we will use mongoose’s help to create a model of a user. let’s create the user.js file in the model directory. in the file, we will define the following mongoose schema: var mongoose = require('mongoose'); var schema = mongoose.schema; var userschema = new schema({ email: string, firstname: string, lastname: string, passwordhash: string, passwordsalt: string }); module.exports = mongoose.model('user', userschema); the model’s properties are the user’s attributes we want to capture (email, first name and last name), as well as a hash of the user’s password and the salt value that we used to create the password’s hash. as you will see later, storing a password’s hash and salt will allow us to authenticate users without needing to store their passwords in our database. the apiresponse class i mentioned a class called apiresponse in the majority of the methods that make the controller’s public interface. this is a data transfer class that will help us move data out of the controller. let’s create the api-response.js file in the models directory. in the file, let’s type the following definition: var apiresponse = function (cnf) { this.success = cnf.success; this.extras = cnf.extras; }; module.exports = apiresponse; any request sent to the controller will eventually produce an apiresponse instance. as its name indicates, the success property of apiresponse will signal whether the request succeeded or not. the extras property will be a javascript object containing any additional data that the controller wants to send out as part of the response. the apimessages class when the success property of the apiresponse instance is false, the data sent in the extras property can include information about what caused the failure. we will define these causes in a class that we will call apimessages. let’s create the api-messages.js file in the models directory. we will define the apimessages class as follows: var apimessages = function () { }; apimessages.prototype.email_not_found = 0; apimessages.prototype.invalid_pwd = 1; apimessages.prototype.db_error = 2; apimessages.prototype.not_found = 3; apimessages.prototype.email_already_exists = 4; apimessages.prototype.could_not_create_user = 5; apimessages.prototype.password_reset_expired = 6; apimessages.prototype.password_reset_hash_mismatch = 7; apimessages.prototype.password_reset_email_mismatch = 8; apimessages.prototype.could_not_reset_password = 9; module.exports = apimessages; as the code indicates, we are defining the reasons that can cause a controller request to fail. they are basically the different error conditions that we anticipate can occur inside the controller. creating the userprofilemodel class the data sent in the extras property of an apiresponse instance can also include a read-only version of the user’s profile. we will create the userprofilemodel class to model this entity. instances of this class will help us pass user data from the database to the outer layers of the backend, and ultimately the mobile application, without exposing sensitive information such as the password hash and salt values. in the models folder, let’s create the user-profile.js file. then, type the userprofilemodel definition: var userprofilemodel = function(cnf) { this.email = cnf.email, this.firstname = cnf.firstname, this.lastname = cnf.lastname }; module.exports = userprofilemodel; in the model we defined three properties to hold the user’s first name, last name and email. this gives us a nice data transfer object that we can send from the controller out to the mobile app when the mobile app needs to display these data. we are not storing password information in instances of this model so there is no opportunity for this information to be pulled from the database and sent out as part of an http response. creating the controller it’s finally time to turn our attention to the controller itself. let’s create the account.js file in the controllers directory. we will declare the controller as follows: var accountcontroller = function (usermodel, session, mailer) { this.crypto = require('crypto'); this.uuid = require('node-uuid'); this.apiresponse = require('../models/api-response.js'); this.apimessages = require('../models/api-messages.js'); this.userprofilemodel = require('../models/user-profile.js'); this.usermodel = usermodel; this.session = session; this.mailer = mailer; }; module.exports = accountcontroller; notice that we are injecting three dependencies into the controller. the usermodel argument is an instance of the user mongoose class that we created a few minutes ago. as you already know, this is an object that knows how to save and retrieve user data from the mondodb database. the session argument is an object that the controller will use to store session data. the mailer argument is a helper object that the controller will use to send the password reset email to the user. what we are doing here is using a dependency injection approach by passing to the controller some of the entities it needs to do its job. this will make it really easy for us to test the controller using mock objects, without having to instance the database, session and mailer objects that we will use in production. in the next chapter of this tutorial you will see how this is done when we create the tests for the controller. we are also declaring a number of variables inside the controller. the crypto and uuid variables refer to the node.crypto and node-uuid modules, which we will use to generate password hashes and unique identifiers needed when we register and log on users. the apiresponse, apimessages and userprofile internal variables refer to the model classes with the same names that we created a few minutes ago. the session getter and setter methods let’s move on to implementing the controller’s public interface that we designed earlier. first, we will create the setter and getter methods for the session, immediately below the controller’s declaration: accountcontroller.prototype.getsession = function () { return this.session; }; accountcontroller.prototype.setsession = function (session) { this.session = session; }; we will use these methods to set or grab a reference to the controller’s session variable. the hashpassword method we will use this method to create a cryptographically-strong pseudo random hash of a password: accountcontroller.prototype.hashpassword = function (password, salt, callback) { // we use pbkdf2 to hash and iterate 10k times by default var iterations = 10000, keylen = 64; // 64 bit. this.crypto.pbkdf2(password, salt, iterations, keylen, callback); }; within hashpassword, we call crypto.pbkdf2, which uses a pseudorandom function to derive a key of the given length from the given password, salt and number of iterations. remember that we will save this hash in the database, instead of saving the password in clear text or encrypted. this is a good security measure because it’s very difficult to use the hash to obtain the original password without knowing the function used, salt, iteration and keylen values. the logon method next, we will create the logon method: accountcontroller.prototype.logon = function(email, password, callback) { var me = this; me.usermodel.findone({ email: email }, function (err, user) { if (err) { return callback(err, new me.apiresponse({ success: false, extras: { msg: me.apimessages.db_error } })); } if (user) { me.hashpassword(password, user.passwordsalt, function (err, passwordhash) { if (passwordhash == user.passwordhash) { var userprofilemodel = new me.userprofilemodel({ email: user.email, firstname: user.firstname, lastname: user.lastname }); me.session.userprofilemodel = userprofilemodel; return callback(err, new me.apiresponse({ success: true, extras: { userprofilemodel:userprofilemodel } })); } else { return callback(err, new me.apiresponse({ success: false, extras: { msg: me.apimessages.invalid_pwd } })); } }); } else { return callback(err, new me.apiresponse({ success: false, extras: { msg: me.apimessages.email_not_found } })); } }); }; inside logon we first create the me variable to hold a reference to the accountcontroller instance that we can use inside callback functions that we will create inline. next, we call the findone method of the usermodel instance to try to find a user with the same email in the mongodb database. the findmethod is provided by mongoose. remember that usermodel is an instance of the user model that we create with mongoose’s help. if the call to findone produces an error, we immediately invoke the callback argument, passing an apiresponse instance where the success property is set to false and the extra property contains a message that explains that there was a database error. if the call to findone produces a user, we proceed to hash the password provided by the user who is attempting to log on, and compare the hash to the password hash of the user that we found in the database. if the hashes are equal, it means that the user attempting to log on provided a valid password and we can move on to create a userprofile instance and save it to the controller’s session variable. we then invoke the callback function, setting the response’s success property to true and passing the userprofile instance in the extras property of the response. when the hashes don’t match, we invoke the callback function, setting the response’s success property to false and passing an “invalid password” reason in the extras property. finally, if the call to findone does not produce a user, we invoke the callback function with a response where the extras property contains a message indicating that the provided email was not found. the logoff method we will use the logoff method to terminate a user’s session: accountcontroller.prototype.logoff = function () { if (this.session.userprofilemodel) delete this.session.userprofilemodel; return; }; to terminate the session we simply destroy the userprofile instance that we previously saved in the controller’s session variable. the register method the controller’s register method allows a user to register with the application: accountcontroller.prototype.register = function (newuser, callback) { var me = this; me.usermodel.findone({ email: newuser.email }, function (err, user) { if (err) { return callback(err, new me.apiresponse({ success: false, extras: { msg: me.apimessages.db_error } })); } if (user) { return callback(err, new me.apiresponse({ success: false, extras: { msg: me.apimessages.email_already_exists } })); } else { newuser.save(function (err, user, numberaffected) { if (err) { return callback(err, new me.apiresponse({ success: false, extras: { msg: me.apimessages.db_error } })); } if (numberaffected === 1) { var userprofilemodel = new me.userprofilemodel({ email: user.email, firstname: user.firstname, lastname: user.lastname }); return callback(err, new me.apiresponse({ success: true, extras: { userprofilemodel: userprofilemodel } })); } else { return callback(err, new me.apiresponse({ success: false, extras: { msg: me.apimessages.could_not_create_user } })); } }); } }); }; the first step that we take in register is to check if a user with the same email address of the user that is attempting to register exists in the database. as we did in the logon method, if there is a database error we will immediately invoke the callback function and send out an apiresponse instance explaining that there was a database error. if we find an user that has the same email address of the user that is attempting to register, we also stop the registration process, as we cannot have two users with the same email address. in this case the extras property of the apiresponse instance that we send out contains a message explaining that the email address already exists. if we don’t find the email address in the database, we proceed to save the new user by invoking save method (inherited from mongooose) of the user class. the save method produces a numberaffected argument in its callback function. we check numberaffected to make sure that the new user was saved. if numberaffected is 1, we create a userprofile instance and send it out embedded in an apiresponse object. if numberaffected is not 1, we produce an apiresponse indicating that the registration failed. the resetpassword method the resetpassword method is the first step of the password reset workflow that we defined in the mobile application user registration, login and logout screens tutorial of this series. the method consists of the following code: accountcontroller.prototype.resetpassword = function (email, callback) { var me = this; me.usermodel.findone({ email: email }, function (err, user) { if (err) { return callback(err, new me.apiresponse({ success: false, extras: { msg: me.apimessages.db_error } })); } // save the user's email and a password reset hash in session. we will use var passwordresethash = me.uuid.v4(); me.session.passwordresethash = passwordresethash; me.session.emailwhorequestedpasswordreset = email; me.mailer.sendpasswordresethash(email, passwordresethash); return callback(err, new me.apiresponse({ success: true, extras: { passwordresethash: passwordresethash } })); }) }; in order to initiate a password reset sequence, users need to provide their email address. inside resetpassword we use the provided email address to retrieve the user’s record from the database. if the record exists, we create a unique identifier called passwordresethash, and pass this identifier and the user’s email address to the mailer object’s sendpasswordresethash method. this method sends a message to the user, containing the unique identifier and a password reset link that they can use to change their password. we will implement the mailer module in the next chapter of this tutorial. inside resetpassword we also save the password reset hash and the user’s email in the controller’s session variable so we can later compare them to the values provided by the user in the final step of the password reset process. if the database doesn’t have a record for the provided email address, we return an apiresponse whose extras property explains that the email was not found. the resetpasswordfinal method users will invoke this method when they access a special web page using the “password reset” link inside the email that they will receive after they perform the first step of the password reset process. here’s the code for the method: accountcontroller.prototype.resetpasswordfinal = function (email, newpassword, passwordresethash, callback) { var me = this; if (!me.session || !me.session.passwordresethash) { return callback(null, new me.apiresponse({ success: false, extras: { msg: me.apimessages.password_reset_expired } })); } if (me.session.passwordresethash !== passwordresethash) { return callback(null, new me.apiresponse({ success: false, extras: { msg: me.apimessages.password_reset_hash_mismatch } })); } if (me.session.emailwhorequestedpasswordreset !== email) { return callback(null, new me.apiresponse({ success: false, extras: { msg: me.apimessages.password_reset_email_mismatch } })); } var passwordsalt = this.uuid.v4(); me.hashpassword(newpassword, passwordsalt, function (err, passwordhash) { me.usermodel.update({ email: email }, { passwordhash: passwordhash, passwordsalt: passwordsalt }, function (err, numberaffected, raw) { if (err) { return callback(err, new me.apiresponse({ success: false, extras: { msg: me.apimessages.db_error } })); } if (numberaffected < 1) { return callback(err, new me.apiresponse({ success: false, extras: { msg: me.apimessages.could_not_reset_password } })); } else { return callback(err, new me.apiresponse({ success: true, extras: null })); } }); }); }; to reset their password a user will need to provide their email address and a new password, along with the password reset hash that we sent them in the password reset email generated from the resetpassword method. we will save the user from having to type the password reset hash by embedding the hash in the link inside the password reset email. in the next chapter of this series we will create the mailer class and implement the email features. inside resetpasswordfinal, we first check that the password reset hash is also saved in the controller’s session variable. if the hash does not exist, we return an apiresponse whose extras property explains that the password reset period expired. as a security measure, we want to limit the period of time during which a user can reset their password to the length of a session timeout period. if the password reset hash value stored in the session and the value supplied by the user do not match, we will assume that the user who requested the password reset and the user who is providing the new password are not the same. in such a case we return an apiresponse explaining that there is a mismatch of the hashes. the same logic applies when the email value stored in the session and the value supplied by the user do not match, in which case we return an apiresponse explaining that there is a mismatch of the email addresses. if the password reset hash and email address validations are successful, we proceed to hash the new password and save it by calling the user model’s update method, which is inherited from mongoose. the update method returns the number of records affected by the update operation. we check this value and return an apiresponse that signals to the outside world if the update operation succeeded or not. summary and next steps we just began building the backend for a meeting room booking application that we defined in the first chapter of this series . this is a mongodb and mongoose backend paired to a node.js and express web server. our focus in this article was building a controller module that will handle the user registration, login and logout features of the application. we implemented the controller’s public interface, along with a number of helper classes that will allow the controller to do its work. in the next chapter of this tutorial we will turn our attention to testing the controller, which will take us through choosing a testing library and implementing the tests for the controller’s features. make sure to sign up for miamicoder’s newsletter so you can be among the first to know when next part of this tutorial is available. download the source code download the mongodb and mongoose backend tutorial here: mongodb and mongoose backend for mobile application previous chapters of this series these are the previous parts of this series: mobile app tutorial: the meeting room booking app, part 1 mobile app tutorial: the meeting room booking app, part 2 mobile app tutorial: the meeting room booking app, part 3 mobile ui patterns – a flowchart for user registration, login and logout

First and foremost, it is important to define what AWS Activate is and what it is used for before we can take a deeper look. Exactly one year ago, Amazon created a program specifically designed for a particular group of customers that often times is in need of as much help as they can get (AKA startups). This program supports startups in their initial phase of building their businesses. This includes providing AWS credits, taking part in startup contests, and receiving benefits from third party solutions on the AWS cloud. Activate allows AWS partners that want to create a presence within the Activate community offer perks to member startups. Some of which include discounts and extended free tiers. Some startups that have attained high levels of success with AWS include Spotify, Pinterest, and Dropbox. With the big shots maintaining their places in startup stardom, Amazon has opened its doors to the next generation of innovators. As such, Amazon offers two different Activate packages. The Self-Starter package is comprised of a limited amount of each of the offerings listed above, whereas the Portfolio package includes some added bonuses along the lines of more high-profile and technical support as well as more in-depth training. On his blog AWS’ CTO, Werner Vogel, reiterated the importance of startups, “Startups will forever be a very important customer segment of AWS. They were among our first customers and along the way some amazing businesses have been built by these startups, many of which running for 100% on AWS.” “We’re excited to be a part of this global momentum in the startup ecosystem. The challenge now is to support and assist an increasing number of startups across the world.” The fun doesn’t stop there. In April of this year, AWS expanded the Activate package to offer much more than generalupport. This entailed sponsoring solution architects to take startups through step by step consultations in the fields of security, architecture and performance. Consequently, though Amazon’s professional services teams were established for customers, it was natural to have them take part in Activate. By nurturing new startups and making them rely heavily on the AWS cloud. As we can see today, companies that started with AWS 4 years ago are now worth billions of dollars. Airbnb and Dropbox, for example, now thoroughly enjoy the flexibility Amazon offers, as well as the fact that they no longer have to maintain cumbersome IT operations. Why not from the get-go? So the question is, if Amazon essentially built AWS on startups, why hasn’t Activate been around from the get-go, 6 years ago? AWS owes a great deal of its success to scalable startups that wanted and needed servers to run their businesses, yet didn’t have the initial capital to build their own data centers. No one really knows why Amazon did not provide startups back then with the kind of support they do today. However, as the market matured, it became clear that Amazon realized that an increasing number of startups could use their help. As a result, Amazon discovered that marketing their support services through Venture Capitalists and incubators around the world would include them as partners in this program and aid in marketing the service to startups of all kinds. “AWS Activate requires a special registration that allows startup customers with a valid AWS account to apply for either a self-starter package or a portfolio package. If a startup is a member of one of the accelerators, seed funds, or startup organizations that Amazon already works with, they may apply for the more exclusive AWS Activate Portfolio Package.” Learn More Incubators and Accelerators It was a natural step for Amazon to partner with accelerators all over the world with the Activate package. In addition to supporting startups, as mentioned above, these accelerators act as channels in the startup scene.At the first AWS re:Invent, Bezos jokes to his fellow investors, saying that eventually some of the investments will return to him because of how heavily the startup scene relies on Amazon. Activate and the approximately 150 accelerators across the world, including White Accel, Techstars, Appwest, and Battery Ventures, genuinely support and understand the values of the AWS service. They are happy to be able to use the Activate platform to help their startups flourish within the AWS clouds. 3rd Party Partners Aside from the accelerators, as an Amazon partner, you can enroll special offers to Activate members. For example, members that are part of the Self-Starter package may receive a 3 month free trial for Chef, whereas Portfolio members may receive a 6 month trial. Most of the partners will provide an extended free trial or credits via Activate. For instance, Trend Micro, one of Amazon’s biggest partners in the security domain, provides $2500 credit for Activate members in the Portfolio package. While there are not many partners on the list, the ones that are mentioned are very helpful and provide nice benefits for Activate members. Reviews of the program from both the partners’ and startups’ side showed that Activate is ideal for startups that have resource constraints. While members within the Self-Starter package are able to use the AWS Free Usage Tier, Portfolio members can receive anywhere from $1,000 to $15,000 in AWS Promotional Credit. The credit is maybe the most important value for these startups. Bearing in mind that Google also has their own line of packages and credit for new companies, it makes sense for AWS to start giving more life to these companies, above the free tier. Everyone has access to the free tier, these startups simply get more of it. Seems that there is no downside to participating. There is no obligation and the worst thing that can happen is that you will find that the services are great, and simply continue using them, which may result in you being locked-in to the point where you need to eventually pay. On the other hand, seems that the last announcement in April, which is actually “meet our architects”. Meaning the knowledge that Amazon’s architects share with startups in their consultation sessions help them get a better grasp on the ecosystem, as well as understand that more resource utilization is ultimately the next logical step for growth. All in all, although Amazon didn’t offer with this program 4 years ago, the AWS cloud was still the natural choice for startups. It included all of the benefits a startup can get using and online and on-demand infinite amount of resources. As a result, it is the clear choice for web scale startups. There are many reasons why Amazon only recently decided to offer free benefits to their prized potential customers. While it could have stemmed from competition from Microsoft and Google, or Amazon may want to simply show their support for their potential customers, demonstrating their cloud’s benefits at an early stage. Aside from that, Amazon understands and is built on companies with long term goals and possibilities. Therefore Amazon sees startups as a long term investment, which starts off with little risk.

When we define a table in Asciidoctor we might want to span a cell over multiple columns or rows, instead of just a single column or row. We can do this using a cell specifier with the following format: column-span.row-span+. The values for column-span and row-span define the number of columns and rows the cell must span. We put the cell specifier before the pipe symbol (|) in our table definition. In the following example Asciidoctor markup we have three tables. In the first table we span a cell over 2 columns, the second table spans a cell over 2 rows and in the final table we span a cell over both 2 columns and rows. == Table cell span .Cell spans columns |=== | Name | Description | Asciidoctor | Awesome way to write documentation // This cell spans 2 columns, indicated // by the number before the + sign. // The + sign // tells Asciidoctor to span this // cell over multiple columns. 2+| The statements above say it all |=== .Cell spans rows |=== | Name | Description // This cell spans 2 rows, // because the number after // the dot (.) specifies the number // of rows to span. The + sign // tells Asciidoctor to span this // cell over multiple rows. .2+| Asciidoctor | Awesome way to write documentation | Works on the JVM |=== .Cell spans both rows and columns |=== | Col1 | Col2 | Col 3 // We can combine the numbers for // row and column span within one // cell specifier. // The number before the dot (.) // is the number of columns to span, // the number after the dot (.) // is the number of rows to span. 2.2+| Cell spans 2 cols, 2 rows | Row 1, Col 3 | Row 2, Col 3 |=== If we transform our source to HTML we get the following tables: Written with Asciidoctor 1.5.1.

Converters are an essential building block in XAML interfaces with one simple task: converting values of one type to another. Since they have a input, usually a view model property, and an output, it would be wonderful if we could somehow chain them to create a new converter that processes all internal converters. Luckily, this is quite simple to do, but we do need to create a new converter which will hold other converters and whose implementation will iterate over nested converters. Full code can be found over at Github repository here, only interesting parts will be highlighted in this blog post. Our combining converter class is also a converter itself, but it can contain other converters inside it: [ContentProperty("Converters")] public class ChainingConverter : IValueConverter { public Collection Converters { get; set; } } Converter functions are trivially implemented and iteratively go through the converters list and apply the converter on the previous value. public object Convert(object value, Type targetType, object parameter, CultureInfo culture) { foreach (var converter in Converters) { value = converter.Convert(value, targetType, parameter, culture); } return value; } ConvertBack is implemented in the same fashion. This allows us to create new converters in XAML with the following syntax: But what if we need to send parameters to some of the converters, how can we do that when the same parameter is used throughout the ChainingConverter implementation? To provide custom parameter for individual converters, we can create a wrapper converter around existing converter and specify parameter on that wrapper. Here is a skeleton for such wrapper converter, notice that the wrapper is also a converter: [ContentProperty("Converter")] public class ParameterizedConverterWrapper : DependencyObject, IValueConverter { // IValueConverter Converter dependency property // object Parameter dependency property // object DefaultReturnValue dependency property public object Convert(object value, Type targetType, object parameter, CultureInfo culture) { if (Converter != null) return Converter.Convert(value, targetType, Parameter ?? parameter, culture); return DefaultReturnValue; } } Converter wrappers allow us to create complex converters such as this one: The final converter should be self explanatory even though you probably haven’t seen these converters before. You can see that unlike other converters, the wrapper is a dependency object which allows us to use bindings on the Parameter property since it is in fact a dependency property. More complex converters should be created from ordinary converters whenever possible, especially when working with primitive types such as bool, string, enums and null values. What’s next? The last example looked like a small DSL embedded in XAML. We could create converters that simulate flow control or conditionals. We could even create converters that switch depending on the property before it, essentially coding logic inside such converters. Whether that is desirable is debatable, but it can be done. The full code with sample application can be found at the following Github repository: MassivePixel/wp-common.

In my previous articles I compared 4 frameworks commonly used in communicating with MongoDB from the JVM and found out that in that use-case, Spring Data for MongoDB was the easiest solution. However I did make the remark that it doesn’t use the GeoJSON format to store geolocation coordinates and geometries. I tried to add GeoJSON support before, but couldn’t get the conversion to work propertly. But after some extensive searching I found out that the reason for it not working was my use of Spring Boot: its autoconfiguration for MongoDB does not support custom conversion out of the box. Luckily, the solution was simple: provide an extra configuration that extends from AbstractMongoConfiguration and import that in the Boot application. In that configuration you can override the customConversions() and add your converters. When you compare the geo classes in Spring Data and GeoJSON, I noticed that only a subset of GeoJSON geometries can be mapped on Spring Data geo classes: Point and Polygon. Spring Boot does not support LineString, MultiLineString, MultiPolygon or MultiPoint. However, in your mapped domain classes, you won’t use these normally. Creating a converter that adheres to the GeoJSON format is quite straightforward. import com.mongodb.BasicDBObject import com.mongodb.DBObject import org.springframework.core.convert.converter.Converter import org.springframework.data.convert.ReadingConverter import org.springframework.data.convert.WritingConverter import org.springframework.data.geo.Point import org.springframework.data.geo.Polygon final class GeoJsonConverters { static List> getConvertersToRegister() { return [ GeoJsonDBObjectToPointConverter.INSTANCE, GeoJsonDBObjectToPolygonConverter.INSTANCE, GeoJsonPointToDBObjectConverter.INSTANCE, GeoJsonPolygonToDBObjectConverter.INSTANCE ] } @WritingConverter static enum GeoJsonPointToDBObjectConverter implements Converter { INSTANCE; @Override DBObject convert(Point source) { return new BasicDBObject([type: 'Point', coordinates: [source.x, source.y]]) } } @ReadingConverter static enum GeoJsonDBObjectToPointConverter implements Converter { INSTANCE; @Override Point convert(DBObject source) { def coordinates = source.coordinates as double[] return new Point(coordinates[0], coordinates[1]) } } @WritingConverter static enum GeoJsonPolygonToDBObjectConverter implements Converter { INSTANCE; @Override DBObject convert(Polygon source) { def coordinates = source.points.collect { [it.x, it.y] } return new BasicDBObject([type: 'Polygon', coordinates: coordinates]) } } @ReadingConverter static enum GeoJsonDBObjectToPolygonConverter implements Converter { INSTANCE; @Override Polygon convert(DBObject source) { def coordinates = source.coordinates as double[] return new Point(coordinates[0], coordinates[1]) } } } To add those converters to the Spring context, you’ll have to override some methods in your MongoDB spring configuration class. import com.mongodb.Mongo import org.springframework.beans.factory.annotation.* import org.springframework.boot.SpringApplication import org.springframework.boot.autoconfigure.EnableAutoConfiguration import org.springframework.context.annotation.* import org.springframework.data.mongodb.config.AbstractMongoConfiguration import org.springframework.data.mongodb.core.convert.* @EnableAutoConfiguration @ComponentScan @Configuration @Import([MongoComparisonMongoConfiguration]) class MongoComparison { static void main(String[] args) { SpringApplication.run(MongoComparison, args); } } @Configuration class MongoComparisonMongoConfiguration extends AbstractMongoConfiguration { @Autowired Mongo mongo; @Value("\${spring.data.mongodb.database}") String databaseName; @Override protected String getDatabaseName() { return databaseName } @Override Mongo mongo() throws Exception { return mongo } @Override CustomConversions customConversions() { def customConverters = [] customConverters << GeoJsonConverters.convertersToRegister return new CustomConversions(customConverters.flatten()) } } As Spring Boot already provides the configuration of the Mongo instance and the name of the database, we can reuse these in the MongoDB configuration class. The custom conversions take preference over the existing ones for Point and Polygon. I’ll be writing a library this weekend to add support for all GeoJSON geometries in Spring Data for MongoDB. However, I already noticed it’ll be very hard to provide support for those in generated query methods in repositories, but with annotated queries being possible, I don’t think this will be a big issue but we’ll see.

Actually, its nothing to do with REST for PUT and POST. In general how HTTP PUT works and how POST work, is what I want to demonstrate through code. Why REST is considered, usually we get confused while developing REST API, that when to use PUT and when to use POST for an update and insert resource. Let's start with the actual definition of these methods (copied formhttp://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html) POST The POST method is used to request that the origin server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI in the Request-Line. The actual function performed by the POST method is determined by the server and is usually dependent on the Request-URI. The posted entity is subordinate to that URI in the same way that a file is subordinate to a directory containing it, a news article is subordinate to a newsgroup to which it is posted, or a record is subordinate to a database. The action performed by the POST method might not result in a resource that can be identified by a URI. In this case, either 200 (OK) or 204 (No Content) is the appropriate response status, depending on whether or not the response includes an entity that describes the result. If a resource has been created on the origin server, the response SHOULD be 201 (Created) and contain an entity which describes the status of the request and refers to the new resource, and a Location header (see section 14.30). Responses to this method are not cacheable unless the response includes appropriate Cache-Control or Expires header fields. However, the 303 (See Other) response can be used to direct the user agent to retrieve a cacheable resource. PUT The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI. If a new resource is created, the origin server MUST inform the user agent via the 201 (Created) response. If an existing resource is modified, either the 200 (OK) or 204 (No Content) response code SHOULD be sent to indicate successful completion of the request. If the resource could not be created or modified with the Request-URI, an appropriate error response SHOULD be given that reflects the nature of the problem. The recipient of the entity MUST NOT ignore any Content-* (e.g. Content-Range) headers that it does not understand or implement and MUST return a 501 (Not Implemented) response in such cases. If the request passes through a cache and the Request-URI identifies one or more currently cached entities, those entries SHOULD be treated as stale. Responses to this method are not cacheable. The fundamental difference between the POST and PUT requests is reflected in the different meaning of the Request-URI. The URI in a POST request identifies the resource that will handle the enclosed entity. That resource might be a data-accepting process, a gateway to some other protocol, or a separate entity that accepts annotations. In contrast, the URI in a PUT request identifies the entity enclosed with the request — the user agent knows what URI is intended and the server MUST NOT attempt to apply the request to some other resource. If the server desires that the request is applied to a different URI. Let's Go back to our REST example Ok, now to make it more clear in REST terms, let's consider an example of Customer and Order scenario, so we have API to create/modify/get a customer but for order, we do have to create order for customer and when we call GET /CustomerOrders API will get the customer orders. APIs we have GET /Customer/{custID} PUT /Customer/{custID} POST /Customer/{custID} (to demonstrate difference between POST and PUT, otherwise for the UC we are considering, it won't be required) POST /Order/{custID} GET /CustomerOrders/{custID} I have enabled browser cache by adding header “Cache-Control”. so lets first see the flow of PUT and GET for customer Initial load, I called PUT /Customer/1 which placed new resource on the server and then called GET /Customer/1 which returned me the customer I placed. now when I again call the GET /Customer/1 I will get the browser “Cached” instance of a customer. Now you call PUT /Customer/1 with updated values of a customer and then call GET /Customer/1, you will observe that browser makes calls to the server to get new changed values. and if you add debug point or increase the wait time you PUT, and make a parallel request for GET (Ajax), then GET request will be pending till PUT is served, so browser makes a cached instance of a resource to stale. In the case of POST, the new resource will be posted to the server, but if POST request is not served, and you request for the same resource using GET, the cached instance will be returned. Once the post is successful and you make GET call to the resource, the browser will hit the server to get a new resource. I added delay of 100 milliseconds in both PUT and POST and made request as 1) Called GET /Customer/1 multiple times to check if I am getting the cached resource. Then I called PUT, and immediately called GET, and GET was pending till PUT is served. below if the screen shot which explains it. 2) Called GET /Customer/1 multiple times to check if I am getting the cached resource. Then I called POST, and immediately called GET, and GET was served from cache. below is the screen shot which explains it. In our customer and order case, the customer should be PUT for a new customer and for updating customer as we are retrieving the customer using same resource URI but for Order, we used POST as we don’t have same URI for GET orders.

This blog post is the third in a series that cover Azure Active Directory Single Sign-On (SSO) authentication in native mobile applications. Authenticating iOS app users with Azure Active Directory How to Best handle AAD access tokens in native mobile apps Using Azure SSO tokens for Multiple AAD Resources From Native Mobile Apps(this post) Sharing Azure SSO access tokens across multiple native mobile apps. Brief Start In an enterprise context, it is highly likely that you would have multiple web services that your native mobile app needs to consume. I had exactly this scenario, where one of my clients had asked if they could maintain the same token in the background in the mobile app to use it for accessing multiple web services. I spent some time digging through the documentation and conducting some experiments to confirm some points. Therefore, this post is to share my findings on accessing multiple Azure AD resources from native mobile apps using ADAL. In the previous two posts, we looked at implementing Azure AD SSO login on native mobile apps, then we looked at how to best maintain these access tokens. This post discusses how to use Azure AD SSO tokens to manage access to multiple AAD resources. Let’s assume that we have 2 web services sitting in Azure (ie WebApi1, and WebApi2), both of which are set to use Azure AD authentication. Then, we have the native mobile app, which needs access to both web services (WebApi1, and WebApi2). Let’s look at what we can and cannot do. Cannot Use the Same Azure AD Access-Token for Multiple Resources The first thing that comes to mind is to use the same access token for multiple Azure AD resources, and that is what the client asked about. However, this is not allowed. Azure AD issues a token for certain resource (which is mapped to an Azure AD app). When we call AcquireToken(), we need to provide a resourceID, only ONE resourceID. The result would have a token that can only be used for the supplied resource (id). There are ways where you could use the same token (as we will see later in this post), but it is not recommended as it complicates operations logging, authentication process tracing, etc. Therefore it is better to look at the other options provided by Azure and the ADAL library. Use the Refresh-Token to Acquire Tokens for Multiple Resources The ADAL library supports acquiring multiple access-Tokens for multiple resources using a refresh token. This means once a user is authenticated, the ADAL’s authentication context, would be able to generate an access-token to multiple resources without authenticating the user again. This was mentioned briefly by the MSDN documentation here. The refresh token issued by Azure AD can be used to access multiple resources. For example, if you have a client application that has permission to call two web APIs, the refresh token can be used to get an access token to the other web API as well. (MSDN documentation) public async Task RefreshTokens() { var tokenEntry = await tokensRepository.GetTokens(); var authorizationParameters = new AuthorizationParameters (_controller); var result = "Refreshed an existing Token"; bool hasARefreshToken = true; if (tokenEntry == null) { var localAuthResult = await _authContext.AcquireTokenAsync ( resourceId1, clientId, new Uri (redirectUrl), authorizationParameters, UserIdentifier.AnyUser, null); tokenEntry = new Tokens { WebApi1AccessToken = localAuthResult.AccessToken, RefreshToken = localAuthResult.RefreshToken, Email = localAuthResult.UserInfo.DisplayableId, ExpiresOn = localAuthResult.ExpiresOn }; hasARefreshToken = false; result = "Acquired a new Token"; } var refreshAuthResult = await _authContext.AcquireTokenByRefreshTokenAsync(tokenEntry.RefreshToken, clientId, resourceId2); tokenEntry.WebApi2AccessToken = refreshAuthResult.AccessToken; tokenEntry.RefreshToken = refreshAuthResult.RefreshToken; tokenEntry.ExpiresOn = refreshAuthResult.ExpiresOn; if (hasARefreshToken) { // this will only be called when we try refreshing the tokens (not when we are acquiring new tokens. refreshAuthResult = await _authContext.AcquireTokenByRefreshTokenAsync (refreshAuthResult.RefreshToken, clientId, resourceId1); tokenEntry.WebApi1AccessToken = refreshAuthResult.AccessToken; tokenEntry.RefreshToken = refreshAuthResult.RefreshToken; tokenEntry.ExpiresOn = refreshAuthResult.ExpiresOn; } await tokensRepository.InsertOrUpdateAsync (tokenEntry); return result; } As you can see from above, we check if we have an access-token from previous runs, and if we do, we refresh the access-tokens for both web services. Notice how the _authContext.AcquireTokenByRefreshTokenAsync() provides an overloading parameter that takes a resourceId. This enables us to get multiple access tokens for multiple resources without having to re-authenticate the user. The rest of the code is similar to what we have seen in the previous two posts. ADAL Library Can Produce New Tokens For Other Resources In the previous two posts, we looked at ADAL library and how it uses TokenCache. Although ADAL does not support persistent caching of tokens yet on mobile apps, it still uses the TokenCache for in-memory caching. This enables ADAL library to generate new access-tokens if the context (AuthenticationContext) still exists from previous authentications. Remember in the previous post we said it is recommended to keep a reference to the authentication-context? Here it comes in handy, as it enables us to generate new access-tokens for accessing multiple Azure AD resources. var localAuthResult = await _authContext.AcquireTokenAsync ( resourceId2, clientId, new Uri (redirectUrl), authorizationParameters, UserIdentifier.AnyUser, null); Calling AcquireToken() (even with no refresh-token) would give us a new access-token to webApi2. This is due to ADAL great goodness where it checks if we have a refresh-token in-memory (managed by ADAL), then it uses that to generate a new access-token for webApi2. An alternative The third alternative option is the simplest, but not necessarily the best. In this option, we could use the same access token to consume multiple Azure AD resources. To do this, we need to use the same Azure AD app ID when setting the web application’s authentication. This requires some understanding of how the Azure AD authentication happens on our web apps. If you refer to Taiseer Joudeh’s tutorial, which we mentioned before, you will see that in our web app, we need to tell the authentication framework what’s our Authority and the Audience (Azure AD app Id). If we set up both of our web apps, to use the same Audience (Azure AD app Id), meaning that we link them both into the same Azure AD application, then we could use the same access-token to use both web services. // linking our web app authentication to an Azure AD application private void ConfigureAuth(IAppBuilder app) { app.UseWindowsAzureActiveDirectoryBearerAuthentication( new WindowsAzureActiveDirectoryBearerAuthenticationOptions { Audience = ConfigurationManager.AppSettings["Audience"], Tenant = ConfigurationManager.AppSettings["Tenant"] }); } As we said before, this is very simple and requires less code, but could cause complications in terms of security logging and maintenance. At the end of the day, it depends on your context and what you are trying to achieve. Therefore, I thought it would be worth mentioning and I will leave the judgement for you on which option you choose. Conclusions We looked at how we could use Azure AD SSO with ADAL to access multiple resources from native mobile apps. As we saw, there are three main options, and the choice could be made based on the context of your app. I hope you find this useful and if you have any questions or you need help with some development that you are doing, then just get in touch. This blog post is the third in a series that cover Azure Active Directory Single Sign-On (SSO) authentication in native mobile applications.

recently some pwc tech supremos wrote an article: agile coding in enterprise it: code small and local . subsections: moving away from the monolith why microservices? msa: a think-small approach for rapid development thinking the msa way: minimalism is a must where msa makes sense in msa, integration is the problem, not the solution conclusion msa is short for microservices architecture(s), in the above article. the article posits that microservices is the antidote to monoliths. it doesn’t mention cookie cutter scaling at all, which is another antidote to monoliths, with the right build infrastructure and devops. here’s a view of hypothetical architecture a company could deploy if they were doing microservices: w is web server. p and q don’t stand for anything in particular. here’s the same solution as cookie-cutter scaling, and the alternate (historical) choice of monolith to the right of it: the cookie cutter approach will often leverage components that are dependency injected into each other, and though monoliths might be the same today, pre 2004 they were probably hairballs of singletons (the design patten, not the springframework idiom). continuous delivery, agile? here’s one excerpt that confuses me: " … makes no sense to design and develop software over an 18-month process to accommodate all possible use cases when those use cases can change unexpectedly and the life span of code modules might be less than 18 months…. as i recall, the 18 month-delay problem was solved previously. agile methodologies principally, and continuous delivery/deployment in more recent times. it does not matter whether you’re compiling a monolith, a cookie-cutter solution, old soa services, or microservices, the 18-month fear isn’t real if you’re doing agile and/or cd. agile and cd were increasing the release cadence, and allowing the organization to pivot faster before microservices. it doesn’t matter whether you’ve got a monolith, something cookie-cutter scaled, or soa (micro or not), you’re going to be able to benefit from agile practices and devops setup that facilitates cd. in something like 30 thoughtworks client engagements since 2002, i have not seen the 18-month process at all. in fact i last encountered it in 1997 on an as/400 project, which was the last time i saw a waterfall process being championed. build(s) and trunk elsewhere there is a suggestion: “each microservice [has] its own build, to avoid trunk conflict”. that isn’t unique to microservices, of course. component based systems today also have a multiple build file (module) structure in a source tree. hopefully “trunk” mentioned is alluding to trunk based development, as i would recommend. build technologies this is a expansion on the above, and you can skip this paragraph if you want. hierarchical build systems like maven has allow you to have one build file per module (whether that’s a service or a simple jar destined for the classpath of a bigger thing). buck has a build grammar that allows for a build to grow/shrink/change based on what is being built (from implicitly shared source). maven is for the java ecosystem, while buck promises to be multi-language. both are doing multi-module builds for the sake of a composed or servicified deployment. both maven and buck are presently competing to draw the most reduced set of compile/test/deploy operations for the changes since last build for a hierarchy of modules. anyway, what is it we are striving for? what we want is to develop cheaply, and to deploy smoothly and often, without defect. we want the ability to deploy without large permanent or temporary headcount overseeing or participating in deployment. aside from development costs, and support/operation, deployment costs are a potentially big factor in total cost of ownership. what i like about cookie-cutter is the uniformity of the deployable things. the team size for deployment of such a thing doesn’t grow with the numbers of nodes that binary is being deployed to. at least, if you’re able to automate the deployment to those nodes, and have a strategy for handling the users connected to the stack at redeployment time somehow (sessions or stateless). the uniformity of the deployment is a cheapener, i think. when you have a number of dissimilar services, you might be able to minimize release personnel if you’re only doing one service. if more than one service is being updated in a particular deployment, you’re going to have to concentrate to make sure you don’t experience a multiplier effect for the participants. it is possible of course, to keep the headcount small, but the practice needed beforehand is bigger, which in turn allows for some calmness around the actual deployment. if we’ve stepped away from the project management office thinking that suggests three buggy releases a year (which is more usual than 18 month schedules of old), then we can employ continuous deployment to further eliminate personnel costs around going live. this is something that microservices does well at, but because the most adept proponents design forwards & backwards compatibility into the permutations most likely to co-exist in production. it is at least much quicker to redeploy and bounce one small service, n times than the the cookie-cutter uniform deployment.

both high availability (ha) and disaster recovery (dr) have been essential it topics. fundamentally ha is about fault tolerance relevant to the availability of an examined subject like application, database, vms, etc. while dr roots on the ability to resume operations in the aftermath of a catastrophic event. a fundamental difference of these two is that ha expects no down time and no data loss, while dr does. they are different issues and should be addressed separately. background for many it shops, either ha or dr has been a high risk and high cost item. both are essential to business continuity, while traditionally tough technical problems to solve with very significant and long-term commitments on resources. not only they are technically challenging, but a continual cost-cutting which has become an it standard practice in the past two decades makes purchasing hardware/software and constructing either ha or dr solution on premises further distant from it’s financial and technical realties. sense of urgency too often, the technical challenges and resource commitments overwhelm it and turn ha and dr into academic discussions, or symbolic items on a project checklist. at the same time, information is rapidly exploding as internet, mobility and social-network are becoming integral in our daily lives and businesses. there are progressively more data to process and store. for many businesses, the needs for ha and dr is urgent for better managing risks. and continual availability and on-demand recoverability of it are becoming increasingly critical. this is the reality, now the good news is that the recent introduction of cloud computing has fundamentally changed how an ha or dr solution can be implemented. microsoft azure is a vivid example of ha and dr solutions with significantly reduced the required financial commitment and involved technical complexities. the traditional approach by establishing redundancy and acquiring a physical dr site with long-term resources and financial commitments is now largely replaced with consumable services which can be configured in minutes by mouse-clicking and with a manageable cost structure based on usage. ha and dr have become it solutions which are financially realistic and technically feasible for businesses in all sizes. ha, redundancy, and microsoft azure lrs ha is to eliminate a single point of failure of an examined component, an application for example. it denotes a strategy to employ redundancy such that a target application can and will continue being available without downtime while experiencing a failure of hosting hardware or software. there are various and well-developed ha solutions like a hyper-v host cluster using redundant hardware to eliminate a single point of failure of hosting os or hardware, and an application cluster for eliminating a single point of failure by running the application in multiple vm instances with a synchronous state. although ha implementations may vary, the fundamental principle nevertheless remains the same. ha expects neither downtime nor data loss while experiencing an outage of a target hardware or software. ha has become dramatically simple in microsoft azure. basically, all data written to disk in microsoft azure are kept at least in the so-called lrs, locally redundant storage. lrs replicates a transaction synchronously to three different storage nodes across fault domains and upgrade domains within the same region for durability. in layman’s terms, microsoft azure by default maintains at least three copies of user data to achieve ha. dr, replication, and microsoft azure grs dr is about having a plan and backups in place to resume operations in the aftermath of a catastrophic event. unplanned outage is assumed in a dr scenario, therefore some data loss is also expected. notice that ha and dr are different business problems and addressed differently. while both ha and dr are based on applying redundancy, i.e. a source and replicas, or multiple identical nodes of an examines component like application instance, databases, or vms, there are however differences between the two. a dr solution generally employs replicas or backups, are implemented with asynchronous processes, and expects an outage of a source and with some data loss in transit while the outage occurs. while ha requires a logical representation with a real-time integrity using synchronous processes across all participating nodes, expects neither downtime nor data loss while experiencing an outage of a participating node. for a critical workload, one approach of dr is to establish geo-replication to address an outage of an entire geographic area caused by a natural disaster, for example. the concern is that a catastrophic event may impact an entire geographic area causing a datacenter where a mission critical application is being hosted becomes unavailable for an extended period of time. in microsoft azure, geo redundant storage or grs is the default and an optional setting, as shown above, while configuring a storage account. grs will queue a transaction committed to lrs as an asynchronous replication to a secondary region, a few hundreds miles away from the primary region where a storage account is originated. at the secondary region, data is also stored in lrs, i.e. made durable by replicating it to three storage nodes. specifically, a microsoft azure storage account configured with grs essentially maintains three replicas locally for high availability, and replicates the content and maintains three replicas at a secondary datacenter a few hundreds miles away for dr. so all are six copies, three locally and three remotely. all these are configured by one, yes one mouse click from a dropdown list while creating a storage account. the above is a conceptual model illustrated a data flow of grs. grs replication has little performance impact on an application since application data are committed to lrs in real-time while replication to grs is queued, i.e. asynchronously. a write to lrs is synchronous and in real-time, once committed, the changes are expected within 15 minutes to be asynchronously replicated to the secondary site. for a ra-grs storage account, in addition to one primary endpoint for read/write operations as it is in a grs, there is also one secondary endpoint as read only becomes available as shown below. the cost implications of grs or ra-grs include the additional storage and the transmission costs for egress traffic, as applicable, of the secondary datacenter. ingress traffic is free . and microsoft azure storage sla offers 99.9% availability and a cost calculator is also available. microsoft azure recovery services so far, much is about backing up or replicating data. to successfully restore, a dr plan must be put in place and ensure its availability upon a dr scenario in progress. either placing a dr plan at a primary site where the source is or a secondary site where a replica stays has some issues and concerns. keeping a dr plan at the source site where all the resources are in place and on-the-job trainings seems logical. or does it? dr is assuming a catastrophic event over an extended geographic areas where the source site is experiencing an outage. in such case, keeping a dr plan in the source site defeats the purpose. maintaining a dr plan at the secondary site is the choice then. in a dr scenario, a recovery site is to be brought on line within a expected period of time according to a dr plan, and having the dr plan right there and then at a recovery site makes all the sense. or does it? this decision introduces a number of requirements including the physical readiness, the timeliness, and the financial implications on securing and maintaining a dr plan at a remote physical facility. for a vmm server running on system center 2012 sp1 or later, an idea, reliable and straightforward way is to use azure recovery services to maintain a dr plan as shown below. and for any backup needs, using cloud as a backup site makes backing up and restoring data an anytime anywhere operation. azure site recovery vault this service essentially acts as the director of a dr process. it orchestrates and manages the protection and failover of vms in clouds managed by virtual machine manager 2012 sp1 or later. a noticeable advantage is the ability to test a recovery configuration, exercise a proactive failover and recovery, and automate recovery in the event of a site outage. the sla of site recovery services is 99.9% availability to ensure a configured dr plan is always in place with expected updates. this is a dr solution that it can implement, simulate, verify, bring online and be absolutely confident with the readiness. azure backup vault this is a reliable, scalable and inexpensive data protection solution with zero capital investment and extremely low operational expense. like other secure communication with microsoft azure, you will first upload a public certificate to microsoft azure. then download the backup agent to register a target server with the backup vault. then select what to be backed up. both microsoft azure backup sla (99.9% availability) and cost calculator are available for better assessing the solution. closing thoughts form an application’s view, ha is an on-going event while dr is an anticipation. ha and dr are different business problems and should be addressed differently. nevertheless, microsoft azure provides a single platform to gracefully address ha with lrs, dr with grs, and dr orchestration with recovery services, and all with published sla s and a predictable cost structure . going forward, it pros can now include ha and dr as a reliable, scalable and relatively inexpensive proposition by employing microsoft azure as a solution platform. call to action register at microsoft virtual academy, http://aka.ms/mva1 , and train yourself on microsoft azure by taking the track of courses. go to http://aka.ms/azure200 and acquire a free trial subscription and assess microsoft azure for ha and dr solutions. review my recommended content at http://aka.ms/recommended .

When comparing two objects, the equals method is used to return true if they are identical. Typically, this leads to the following code : if (name.equals("Jim")) { } The problem here is that whether intended or not, it is quite possible that the name value is null, in which case a null pointer exception would be thrown. A better practice is to execute the equals method of the string constant “Jim” instead : if ("Jim".equals(name)) { } Since the constant is never null, a null exception will not be thrown, and if the other value is null, the equals comparison will fail. If you are using Java 7 or above, the new Objects class has an equals static method to compare two objects while taking null values into account. if (Objects.equals(name,"Jim")) { } Alternatively if you are using a java version prior to Java 7, but using the guava library you can use the Objects class which has a static equal() method that takes two objects and handles null cases for you. It should also be noted that there are probably a number of other implementations in various libraries (i.e. Apache Commons)

This article represents command set in R programming language, which could be used to extract rows and columns from a given data frame.

When I needed to do prototyping, proof of concept or play with some new technology in free time, starting new project was always a little annoying barrier with Maven. Have to say that setting up Maven project is not hard and you can use Maven Archetypes. But Archetypes are often out of date. Who wants to play with old technologies? So I always end up wiring in dependencies I wanted to play with. Not very productive spent time. But than Spring Boot came to my way. I fell in love. In last few months I created at least 50 small playground projects, prototypes with Spring Boot. Also incorporated it at work. It’s just perfect for prototyping, learning, microservices, web, batch, enterprise, message flow or command line applications. You have to be dinosaur or be blind not to evaluate Spring Boot for your next Spring project. And when you finish evaluate it, you will go for it. I promise. I feel a need to highlight how easy is Black Box Testing of Spring Boot microservice. Black Box Testing refers to testing without any poking with application artifact. Such testing can be called also integration testing. You can also perform performance or stress testing way I am going to demonstrate. Spring Boot Microservice is usually web application with embedded Tomcat. So it is executed as JAR from command line. There is possibility to convert Spring Boot project into WAR artifact, that can be hosted on shared Servlet container. But we don’t want that now. It’s better when microservice has its own little embedded container. I used existing Spring’s REST service guide as testing target. Focus is mostly on testing project, so it is handy to use this “Hello World” REST application as example. I expect these two common tools are set up and installed on your machine: Maven 3 Git So we’ll need to download source code and install JAR artifact into our local repository. I am going to use command line to download and install the microservice. Let’s go to some directory where we download source code. Use these commands: git clone [email protected]:spring-guides/gs-rest-service.git cd gs-rest-service/complete mvn clean install If everything went OK, Spring Boot microservice JAR artifact is now installed in our local Maven repository. In serious Java development, it would be rather installed into shared repository (e.g. Artifactory, Nexus,… ). When our microservice is installed, we can focus on testing project. It is also Maven and Spring Boot based. Black box testing will be achieved by downloading the artifact from Maven repository (doesn’t matter if it is local or remote). Maven-dependency-plugin can help us this way: org.apache.maven.plugins maven-dependency-plugin copy-dependencies compile copy-dependencies gs-rest-service true It downloads microservice artifact into target/dependency directory by default. As you can see, it’s hooked to compile phase of Maven lifecycle, so that downloaded artifact is available during test phase. Artifact version is stripped from version information. We use latest version. It makes usage of JAR artifact easier during testing. Readers skilled with Maven may notice missing plugin version. Spring Boot driven project is inherited from parent Maven project called spring-boot-starter-parent. It contains versions of main Maven plugins. This is one of the Spring Boot’s opinionated aspects. I like it, because it provides stable dependencies matrix. You can change the version if you need. When we have artifact in our file system, we can start testing. We need to be able to execute JAR file from command line. I used standard JavaProcessBuilder this way: public class ProcessExecutor { public Process execute(String jarName) throws IOException { Process p = null; ProcessBuilder pb = new ProcessBuilder("java", "-jar", jarName); pb.directory(new File("target/dependency")); File log = new File("log"); pb.redirectErrorStream(true); pb.redirectOutput(Redirect.appendTo(log)); p = pb.start(); return p; } } This class executes given process JAR based on given file name. Location is hard-coded to target/dependency directory, where maven-dependency-plugin located our artifact. Standard and error outputs are redirected to file. Next class needed for testing is DTO (Data transfer object). It is simple POJO that will be used for deserialization from JSON. I use Lombok project to reduce boilerplate code needed for getters, setters, hashCode and equals. @Data @AllArgsConstructor @NoArgsConstructor public class Greeting { private long id; private String content; } Test itself looks like this: public class BlackBoxTest { private static final String RESOURCE_URL = "http://localhost:8080/greeting"; @Test public void contextLoads() throws InterruptedException, IOException { Process process = null; Greeting actualGreeting = null; try { process = new ProcessExecutor().execute("gs-rest-service.jar"); RestTemplate restTemplate = new RestTemplate(); waitForStart(restTemplate); actualGreeting = restTemplate.getForObject(RESOURCE_URL, Greeting.class); } finally { process.destroyForcibly(); } Assert.assertEquals(new Greeting(2L, "Hello, World!"), actualGreeting); } private void waitForStart(RestTemplate restTemplate) { while (true) { try { Thread.sleep(500); restTemplate.getForObject(RESOURCE_URL, String.class); return; } catch (Throwable throwable) { // ignoring errors } } } } It executes Spring Boot microservice process first and wait unit it starts. To verify if microservice is started, it sends HTTP request to URL where it’s expected. The service is ready for testing after first successful response. Microservice should send simple greeting JSON response for HTTP GET request. Deserialization from JSON into our Greeting DTO is verified at the end of the test. Source code is shared on Github.

This article represents the first Proof of Concept from series described in the previous article 4 Hands-On Approaches to Improve Your Data Access Layer Implementation and it presents how to implement Ehcache over MyBatis, how to achieve an optim configuration for it and personal opinions of the author about the chosen approach for the Data Access Layer. Throughout my research on caching over MyBatis I have discovered that Ehcache is the first option among developers when they need to implement a cache mechanism over MyBatis, using a 3rd party library. Ehcache is probably so popular because it represents an open source, java-based cache, available under an Apache 2 license. Also, it scales from in-process with one or more nodes through to a mixed in-process/out-of-process configuration with terabyte-sized caches. In addition, for those applications needing a coherent distributed cache, Ehcache uses the open source Terracotta Server Array. Last but not least, among its adopters is the Wikimedia Foundation that uses Ehcache to improve the performance of its wiki projects. Within this article, the following aspects will be addressed: 1. How will an application benefit from caching using Ehcache? Ehcache's features will be detailed in this section. 2. Hands-on implementation of the EhCachePOC project - in this section the key concepts of EhCache will be explored through a hands on implementation. 3. Summary - How has the application performance been improved after this implementation? Code of all the projects that will be implemented can be found at https://github.com/ammbra/CacherPoc or if you are interested only in the current implementation, you can access it here: https://github.com/ammbra/CacherPoc/tree/master/EhCachePoc How will an application benefit from caching using Ehcache? The time taken for an application to process a request principally depends on the speed of the CPU and main memory. In order to "speed up" your application you can perform one or more of the following: improve the algorithm performance achieve parallelisation of the computations across multiple CPUs or multiple machines upgrade the CPU speed As explained in the previous article, high availability applications should perform a small amount of actions with the database. Since the time taken to complete a computation depends principally on the rate at which data can be obtained, then the application should be able to temporarily store computations that may be reused again. Caching may be able to reduce the workload required, this means a caching mechanism should be created! Ehcache is described as : Fast and Light Weight , having a simple API and requiring only a dependency on SLF4J. Scalable to hundreds of nodes with the Terracotta Server Array, but also because provides Memory and Disk store for scalability into gigabytes Flexible because supports Object or Serializable caching; also provides LRU, LFU and FIFO cache eviction policies Standards Based having a full implementation of JSR107 JCACHE API Application Persistence Provider because it offers persistent disk store which stores data between VM restarts JMX Enabled Distributed Caching Enabler because it offers clustered caching via Terracotta and replicated caching via RMI, JGroups, or JMS Cache Server (RESTful, SOAP cache Server) Search Compatible, having a standalone and distributed search using a fluent query language Hands-on implementation of the EhCachePOC project The implementation of EhCachePoc will look as described in the diagram below: In order to test Ehcache performance through a POC(proof of concept) project the following project setup is performed: 1. Create a new Maven EJB Project from your IDE (this kind of project is platform provided by NetBeans but for those that use eclipse, here is an usefull tutorial) . In the article this project is named EhCachePOC. 2. Edit the project's pom by adding required jars : org.mybatis mybatis 3.2.6 org.mybatis.caches mybatis-ehcache 1.0.2 log4j log4j 1.2.17 net.sf.ehcache ehcache 2.7.0 org.slf4j slf4j-log4j12 1.7.5 3.Add your database connection driver, in this case apache derby: org.apache.derby derbyclient 10.11.1.1 4. Run mvn clean and mvn install commands on your project. Now the project setup is in place, let's go ahead with MyBatis implementation : 1. Configure under resources/com/tutorial/ehcachepoc/xml folder the Configuration.xml file with : 2. Create in java your own SQLSessionFactory implementation. For example, create something similar to com.tutorial.ehcachepoc.config. SQLSessionFactory : public class SQLSessionFactory { private static final SqlSessionFactory FACTORY; static { try { Reader reader = Resources.getResourceAsReader("com/tutorial/ehcachepoc/xml/Configuration.xml"); FACTORY = new SqlSessionFactoryBuilder().build(reader); } catch (Exception e){ throw new RuntimeException("Fatal Error. Cause: " + e, e); } } public static SqlSessionFactory getSqlSessionFactory() { return FACTORY; } } 3. Create the necessary bean classes, those that will map to your sql results, like Employee: public class Employee implements Serializable { private static final long serialVersionUID = 1L; private Integer id; private String firstName; private String lastName; private String adress; private Date hiringDate; private String sex; private String phone; private int positionId; private int deptId; public Employee() { } public Employee(Integer id) { this.id = id; } @Override public String toString() { return "com.tutorial.ehcachepoc.bean.Employee[ id=" + id + " ]"; } } 4. Create the IEmployeeDAO interface that will expose the ejb implementation when injected: public interface IEmployeeDAO { public List getEmployees(); } 5. Implement the above inteface and expose the implementation as a Stateless EJB (this kind of EJB preserves only its state, but there is no need to preserve its associated client state): @Stateless(name = "ehcacheDAO") @TransactionManagement(TransactionManagementType.CONTAINER) public class EmployeeDAO implements IEmployeeDAO { private static Logger logger = Logger.getLogger(EmployeeDAO.class); private SqlSessionFactory sqlSessionFactory; @PostConstruct public void init() { sqlSessionFactory = SQLSessionFactory.getSqlSessionFactory(); } @Override public List getEmployees() { logger.info("Getting employees....."); SqlSession sqlSession = sqlSessionFactory.openSession(); List results = sqlSession.selectList("retrieveEmployees"); sqlSession.close(); return results; } } 5. Create the EmployeeMapper.xml that contains the query named "retrieveEmployees" select id, first_name, last_name, hiring_date, sex, dept_id from employee If you remember the CacherPOC setup from the previously article, then you can test your implementation if you add EhCachePOC project as dependency and inject the IEmployeeDAO inside the EhCacheServlet. Your CacherPOC pom.xml file should contain : ${project.groupId} EhCachePoc ${project.version} and your servlet should look like: @WebServlet("/EhCacheServlet") public class EhCacheServlet extends HttpServlet { private static Logger logger = Logger.getLogger(EhCacheServlet.class); @EJB(beanName ="ehcacheDAO") IEmployeeDAO employeeDAO; private static final String LIST_USER = "/listEmployee.jsp"; @Override protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException { String forward= LIST_USER; List results = new ArrayList(); for (int i = 0; i < 10; i++) { for (Employee emp : employeeDAO.getEmployees()) { logger.debug(emp); results.add(emp); } try { Thread.sleep(3000); } catch (Exception e) { logger.error(e, e); } } req.setAttribute("employees", results); RequestDispatcher view = req.getRequestDispatcher(forward); view.forward(req, resp); } } Run your CacherPoc implementation to check if your Data Access Layer with MyBatis is working or download the code provided at https://github.com/ammbra/CacherPoc But if a great amount of employees is stored in database, or perhaps the retrieval of a number of 10xemployeesNo represents a lot of workload for the database. Also, can be noticed that the query from the EmployeeMapper.xml retrieves data that almost never changes (id, first_name, last_name, hiring_date, sex cannot change; the only value that might change in time is dept_id); so a caching mechanism can be used. Below is described how this can be achieved using EhCache: 1. Configure directly under the resources folder the ehcache.xml file with: This xml explains that the Memory Store is used for an LRU (Last Recently Used) caching strategy, sets the limits for the number of elements allowed for storage, their time to be idle and their time to live. The Memory Store strategy is often chosen because is fast and thread safe for use by multiple concurrent threads, being backed by LinkedHashMap. Also, all elements involved in the caching process are suitable for placement in the Memory Store. Another approach can be tried: storing cache on disk. This can be done by replacing the ehcache tag content with: diskStore path="F:\\cache" /> Unlike the memory store strategy, the disk store implementation is suitable only for elements which are serializable can be placed in the off-heap; if any non serializable elements are encountered, those will be removed and WARNING level log message emitted. The eviction is made using the LFU algorithm and it is not configurable or changeable. From persistency point of view, this method of caching allows control of the cache by the disk persistent configuration; if false or omitted, disk store will not persist between CacheManager restarts. 2. Update EmployeeMapper.xml to use the previous implemented caching strategy: select id, first_name, last_name, hiring_date, sex, dept_id from employee By adding the line and specifying on the query useCache="true" you are binding the ehcache.xml configuration to your DataAccessLayer implementation. Clean, build and redeploy both EhCachePOC and CacherPoc projects; now retrieve your employees for two times in order to allow the in-memory cache to store your values. When you run your query for the first time, your application will execute the query on the database and retrieve the results. Second time you access the employee list, your application will access the in-memory storage. Summary - How has the application performance been improved after this implementation? An application's performances depend on a multitude of factors how many times a cached piece of data can and is reduced by the application the proportion of the response time that is alleviated by caching Amdhal's law can be used to estimate the system's speed up : where P is proportion speed up and S is speed up. Let's take the application from this article as example and calculate the speed up. When the application ran the query without caching,a JDBC transaction is performed and in your log will be something similar to : INFO: 2014-11-27 18:01:30,020 [EmployeeDAO] INFO com.tutorial.hazelcastpoc.dao.EmployeeDAO:38 - Getting employees..... INFO: 2014-11-27 18:01:39,148 [JdbcTransaction] DEBUG org.apache.ibatis.transaction.jdbc.JdbcTransaction:98 - Setting autocommit to false on JDBC Connection [org.apache.derby.client.net.NetConnection40@1c374fd] INFO: 2014-11-27 18:01:39,159 [retrieveEmployees] DEBUG com.tutorial.hazelcastpoc.mapper.EmployeeMapper.retrieveEmployees:139 - ==> Preparing: select id, first_name, last_name, hiring_date, sex, dept_id from employee INFO: 2014-11-27 18:01:39,220 [retrieveEmployees] DEBUG com.tutorial.hazelcastpoc.mapper.EmployeeMapper.retrieveEmployees:139 - ==> Parameters: INFO: 2014-11-27 18:01:39,316 [retrieveEmployees] DEBUG com.tutorial.hazelcastpoc.mapper.EmployeeMapper.retrieveEmployees:139 - <== Total: 13 while running the queries with Ehcache caching the JDBC transaction is performed only once (to initialize the cache) and after that the log will look like : INFO: 2014-11-28 18:04:50,020 [EmployeeDAO] INFO com.tutorial.ehcachepoc.dao.EmployeeDAO:38 - Getting employees..... INFO: 2014-11-28 18:04:50,020 [EhCacheServlet] DEBUG com.tutorial.cacherpoc.EhCacheServlet:41 - com.tutorial.crudwithjsp.model.Employee[ id=1 ] Let's look at the time that each of our 10 times requests has scored: the first not cached version of 10 times requests took about 57 seconds and 51 milliseconds, while the cached requests scored a time of 27seconds and 86 miliseconds. In order to apply Amdhal's law for the system the following input is needed: Un-cached page time: 60 seconds Database time : 58 seconds Cache retrieval time: 28seconds Proportion: 96.6% (58/60) (P) The expected system speedup is thus: 1 / (( 1 – 0.966) + 0.966 / (58/28)) = 1 / (0.034 + 0. 966/2.07) = 2 times system speedup This result can be improved of course, but the purpose of this article was to prove that caching using Ehcache over MyBatis offers a significant improvement to what used to be available before its implementation. Learn more from: MyBatis Documentation MyBatis Ehcache Adapter EhCache website

Recently I’ve worked on a part of project where are a lot of entities. As in many other projects with the same feature there was implemented “soft delete” approach. That’s mean that when someone deletes any entity it remains in a database but a special field (e.g. ‘isDeleted’) changes its value to true. As you’ve already guessed in every SELECT operation for this kind of entities we need to apply condition: WHERE isDeleted = false It’s a little bit redundant and boring to append each time this condition to a SQL query. So I started look at solutions which could give me some elegant solution of the problem. Fortunately a colleague of mine have given me a hint how to deal with such cases. The answer is covered behind the Hibernate‘s annotation @Where. Let’s consider how we can decorate an entity with the @Where annotation to avoid extra condition in regular SQL queries: import org.hibernate.annotations.Where; import javax.persistence.*; @Entity @Table @Where(clause = "isDeleted='false'") public class Customer { @Id @GeneratedValue @Column private Integer id; @Column private String name; @Column private Boolean isDeleted; //Getters and setters } Now when you want to select Customer on JPA level you will always get only isDeleted=false records. It’s very convenient when you are working with “soft delete” or any other situation which requires permanent application of some condition. I hope it will be useful for your projects.

It has been a while since my first post about the ESP8266 (see “Cheap and Simple WiFi with ESP8266 for the FRDM Board“). The ESP8266 is a new inexpensive ($4.50) WiFi module which makes it easy to connect to the network or internet. Finally this week-end I have found the time to write up a tutorial: how to implement a WiFi web server for the ESP8266 WiFi module and the Freescale FRDM-KL25Z board: WSP8266 Web Server FRDM-KL25Z with ESP8266 WiFi Module Outline In this tutorial I’m using a Freescale FRDM-KL25Z board as a web server, using theESP8266 board. The ESP8266 is a ‘less than $4.5′ WiFi board getting more and more popular as an IoT board. There is even a way to run the ESP8266 standalone (because it has a full processor on that board). However, that development is still in the flux and rather unstable. Instead, I’m using a serial connection to the ESP8266 instead. With this, any small microcontroller can send and receive data from the internet: connect that board to a microcontroller with 3.3V, GND, Tx and Rx, and you have a W-LAN connection! I’m using in this tutorial Eclipse with GNU/GDB with Processor Expert, but with the steps in this tutorial you should be able to use any other toolchain too. As things might change in the future with different firmware on the ESP8266: the firmware I’m having on the board is version 00160901. Board Connections Since my first post on the ESP8266 I have cleaned up the wiring. The pins are as below for the ESP8266: ESP8266 Pins Because the ESP8266 can take > 200 mA, I’m using a 5-to-3.3V DC-DC converter. I measured around 70 to 90 mA, so it is not (yet) really needed, but I wanted to use it to protect to board. The ESP8266 Rx and Tx are connected to the microcontroller Tx and Rx pins. A general frustration point for the ESP8266 module is the connection oft the remaining pins. What worked for me is to connect CH_PD to 3.3V and leaving RST,GPIO0 and GPIO2 unconnected/floating. Wiring Setup with FRDM-KL25Z and ESP8266 Communication Protocol I recommend to use a logic analyzer to verify the communication between the ESP8266 and the microcontroller. My module communicates with 115200, but I see reports that other modules (other firmware) can use a different baud. The module uses an AT command send. The simplest command is to send “AT\r\n” and it responds with “AT\r\r\n\r\nOK\r\n”: AT Command Sent to ESP8266 In this tutorial I’m using a command line shell (see “A Shell for the Freedom KL25Z Board“) to have a manual mode to send commands to the module. More about this later. Project Creation You can use my project and source files available on GitHub (see link at the end of this article). Or create your own project. My project is using the Kinetis Design Studio and for the FRDM-KL25Z board (MKL25Z128VLK4). I have created a project for Processor Expert, as I’m using several components of it: Processor Expert Project For the project I have several files added: ESP8266 Project in Eclipse With the following source files: Application.c/.h: This runs the application and web server program ESP8266.c/.h: Driver for the ESP8266 Events.c/.h: Processor Expert event hooks main.c: main entry point Shell.c/.h: command line interface Sources Project and Source files are available on GitHub here: https://github.com/ErichStyger/mcuoneclipse/tree/master/Examples/KDS/FRDM-KL25Z/FRDM-KL25Z_ESP8266 Please check the latest source files on GitHub. At the time of writing this article, I’m using the following: Shell.h is the interface to command line shell: view source print? 01./* 02. * Shell.h 03. * 04. * Author: Erich Styger 05. */ 06. 07.#ifndef SHELL_H_ 08.#define SHELL_H_ 09. 10./*! 11. * \brief Shell parse routine 12. */ 13.voidSHELL_Parse(void); 14. 15./*! 16. * \brief Shell initialization 17. */ 18.voidSHELL_Init(void); 19. 20.#endif /* SHELL_H_ */ Shell.c implements the application part of the shell: view source print? 01./* 02. * Shell.c 03. * 04. * Author: Erich Styger 05. */ 06. 07.#include "Shell.h" 08.#include "CLS1.h" 09.#include "ESP8266.h" 10. 11./* table with shell parser/handler */ 12.staticconstCLS1_ParseCommandCallback CmdParserTable[] = 13.{ 14. CLS1_ParseCommand, 15. ESP_ParseCommand, 16. NULL /* sentinel */ 17.}; 18. 19.staticunsigned charlocalConsole_buf[48]; /* buffer for command line */ 20. 21.voidSHELL_Parse(void) { 22. (void)CLS1_ReadAndParseWithCommandTable(localConsole_buf, sizeof(localConsole_buf), CLS1_GetStdio(), CmdParserTable); 23.} 24. 25.voidSHELL_Init(void) { 26. localConsole_buf[0] = '\0'; /* initialize buffer */ 27.} ESP8266.h is the interface to the WiFi module: view source print? 001./* 002. * ESP8266.h 003. * 004. * Author: Erich Styger 005. */ 006. 007.#ifndef ESP8266_H_ 008.#define ESP8266_H_ 009. 010.#include "CLS1.h" 011. 012.#define ESP_DEFAULT_TIMEOUT_MS (100) 013. /*!< Default timeout value in milliseconds */ 014. 015./*! 016. * \brief Command line parser routine 017. * \param cmd Pointer to command line string 018. * \param handled Return value if command has been handled 019. * \param io Standard Shell I/O handler 020. * \return Error code, ERR_OK for no failure 021. */ 022.uint8_t ESP_ParseCommand(constunsigned char*cmd, bool *handled, constCLS1_StdIOType *io); 023. 024./*! 025. * \brief Send a string to th ESP8266 module 026. * \param str String to send, "\r\n" will be appended 027. * \param io Shell I/O handler or NULL if not used 028. * \return Error code, ERR_OK for no failure 029. */ 030.uint8_t ESP_SendStr(constuint8_t *str, CLS1_ConstStdIOType *io); 031. 032./*! 033. * \brief Used to send an AT command to the ESP8266 module 034. * \param cmd Command string to send 035. * \param rxBuf Buffer for the response, can be NULL 036. * \param rxBufSize Size of response buffer 037. * \param expectedTailStr Expected response from the module, can be NULL 038. * \param msTimeout Timeout time in milliseconds 039. * \param io Shell I/O handler or NULL if not used 040. * \return Error code, ERR_OK for no failure 041. */ 042.uint8_t ESP_SendATCommand(uint8_t *cmd, uint8_t *rxBuf, size_t rxBufSize, uint8_t *expectedTailStr, uint16_t msTimeout, constCLS1_StdIOType *io); 043. 044./*! 045. * \brief Read from the serial line from the module until a sentinel char is received 046. * \param buf 047. * \param bufSize 048. * \param sentinelChar 049. * \param timeoutMs Timeout time in milliseconds 050. * \return Error code, ERR_OK for no failure 051. */ 052.uint8_t ESP_ReadCharsUntil(uint8_t *buf, size_t bufSize, uint8_t sentinelChar, uint16_t timeoutMs); 053. 054./*! 055. * \brief Sends an AT command to test the connection 056. * \return Error code, ERR_OK for no failure 057. */ 058.uint8_t ESP_TestAT(void); 059. 060./*! 061. * \brief Restarts the ESP8266 module 062. * \param io Shell I/O handler or NULL if not used 063. * \param timeoutMs Timeout time in milliseconds 064. * \return Error code, ERR_OK for no failure 065. */ 066.uint8_t ESP_Restart(constCLS1_StdIOType *io, uint16_t timeoutMs); 067. 068./*! 069. * \brief Set the current mode of the module 070. * \param mode Where is 1=Sta, 2=AP or 3=both 071. * \return Error code, ERR_OK for no failure 072. */ 073.uint8_t ESP_SelectMode(uint8_t mode); 074. 075./*! 076. * \Brief returns the firmware version string 077. * \param fwBuf Buffer for the string 078. * \param fwBufSize Size of buffer in bytes 079. * \return Error code, ERR_OK for no failure 080. */ 081.uint8_t ESP_GetFirmwareVersionString(uint8_t *fwBuf, size_t fwBufSize); 082. 083./*! 084. * \brief Join an access point. 085. * \param ssid SSID of access point 086. * \param pwd Password of access point 087. * \param nofRetries Number of connection retries 088. * \param io Shell I/O or NULL if not used 089. * \return Error code, ERR_OK for no failure 090. */ 091.uint8_t ESP_JoinAP(constuint8_t *ssid, constuint8_t *pwd, intnofRetries, CLS1_ConstStdIOType *io); 092. 093./*! 094. * \brief Scans for an IPD message sent by the module 095. * \param msgBuf Pointer to the message buffer where to store the message 096. * \param msgBufSize Size of message buffer 097. * \param ch_id Pointer to where to store the channel/id 098. * \param size Pointer where to store the size of the message 099. * \param isGet TRUE if it is a GET message, FALSE for a POST message 100. * \param timeoutMs Error code, ERR_OK for no failure 101. * \param io 102. * \return Error code, ERR_OK for no failure 103. */ 104.uint8_t ESP_GetIPD(uint8_t *msgBuf, size_t msgBufSize, uint8_t *ch_id, uint16_t *size, bool *isGet, uint16_t timeoutMs, constCLS1_StdIOType *io); 105. 106./*! 107. * \brief Closes a connection 108. * \param channel Channel ID 109. * \param io Error code, ERR_OK for no failure 110. * \param timeoutMs Error code, ERR_OK for no failure 111. * \return Error code, ERR_OK for no failure 112. */ 113.uint8_t ESP_CloseConnection(uint8_t channel, constCLS1_StdIOType *io, uint16_t timeoutMs); 114. 115./*! 116. * \brief Used to determine if the web server is running or not. 117. * \return TRUE if web server has beens started 118. */ 119.bool ESP_IsServerOn(void); 120. 121./*! 122. * \brief Driver initialization 123. */ 124.voidESP_Init(void); 125. 126./*! 127. * \brief Driver de-initialization 128. */ 129.voidESP_Deinit(void); 130. 131.#endif /* ESP8266_H_ */ And the ESP8266 driver is in ESP8266.c which implements all the low level SPI access functions, the functional implementation and a command line shell interface: view source print? 001./* 002. * ESP8266.c 003. * 004. * Author: Erich Styger 005. */ 006. 007.#include "ESP8266.h" 008.#include "Shell.h" 009.#include "UTIL1.h" 010.#include "CLS1.h" 011.#include "AS2.h" 012.#include "WAIT1.h" 013. 014.staticbool ESP_WebServerIsOn = FALSE; 015. 016.bool ESP_IsServerOn(void) { 017. returnESP_WebServerIsOn; 018.} 019. 020.staticvoidSend(unsigned char*str) { 021. while(*str!='\0') { 022. AS2_SendChar(*str); 023. str++; 024. } 025.} 026. 027.staticvoidSkipNewLines(constunsigned char**p) { 028. while(**p=='\n'|| **p=='\r') { 029. (*p)++; /* skip new lines */ 030. } 031.} 032. 033.uint8_t ESP_ReadCharsUntil(uint8_t *buf, size_t bufSize, uint8_t sentinelChar, uint16_t timeoutMs) { 034. uint8_t ch; 035. uint8_t res = ERR_OK; 036. 037. if(bufSize<=1) { 038. returnERR_OVERRUN; /* buffer to small */ 039. } 040. buf[0] = '\0'; buf[bufSize-1] = '\0'; /* always terminate */ 041. bufSize--; 042. for(;;) { /* breaks */ 043. if(bufSize==0) { 044. res = ERR_OVERRUN; 045. break; 046. } 047. if(AS2_GetCharsInRxBuf()>0) { 048. (void)AS2_RecvChar(&ch); 049. *buf = ch; 050. buf++; 051. bufSize--; 052. if(ch==sentinelChar) { 053. *buf = '\0'; /* terminate string */ 054. break; /* sentinel found */ 055. } 056. } else{ 057. if(timeoutMs>10) { 058. WAIT1_WaitOSms(5); 059. timeoutMs -= 5; 060. } else{ 061. res = ERR_NOTAVAIL; /* timeout */ 062. break; 063. } 064. } 065. } 066. returnres; 067.} 068. 069.staticuint8_t RxResponse(unsigned char*rxBuf, size_t rxBufLength, unsigned char*expectedTail, uint16_t msTimeout) { 070. unsigned charch; 071. uint8_t res = ERR_OK; 072. unsigned char*p; 073. 074. if(rxBufLength < sizeof("x\r\n")) { 075. returnERR_OVERFLOW; /* not enough space in buffer */ 076. } 077. p = rxBuf; 078. p[0] = '\0'; 079. for(;;) { /* breaks */ 080. if(msTimeout == 0) { 081. break; /* will decide outside of loop if it is a timeout. */ 082. } elseif(rxBufLength == 0) { 083. res = ERR_OVERFLOW; /* not enough space in buffer */ 084. break; 085. } elseif(AS2_GetCharsInRxBuf() > 0) { 086.#if0 087. if(AS2_RecvChar(&ch) != ERR_OK) { 088. res = ERR_RXEMPTY; 089. break; 090. } 091.#else 092. /* might get an overrun OVERRUN_ERR error here? Ignoring error for now */ 093. (void)AS2_RecvChar(&ch); 094.#endif 095. *p++ = ch; 096. *p = '\0'; /* always terminate */ 097. rxBufLength--; 098. } elseif(expectedTail!=NULL && expectedTail[0]!='\0' 099. && UTIL1_strtailcmp(rxBuf, expectedTail) == 0) { 100. break; /* finished */ 101. } else{ 102. WAIT1_WaitOSms(1); 103. msTimeout--; 104. } 105. } /* for */ 106. if(msTimeout==0) { /* timeout! */ 107. if(expectedTail[0] != '\0'/* timeout, and we expected something: an error for sure */ 108. || rxBuf[0] == '\0'/* timeout, did not know what to expect, but received nothing? There has to be a response. */ 109. ) 110. { 111. res = ERR_FAULT; 112. } 113. } 114. returnres; 115.} 116. 117.uint8_t ESP_SendATCommand(uint8_t *cmd, uint8_t *rxBuf, size_t rxBufSize, uint8_t *expectedTailStr, uint16_t msTimeout, constCLS1_StdIOType *io) { 118. uint16_t snt; 119. uint8_t res; 120. 121. if(rxBuf!=NULL) { 122. rxBuf[0] = '\0'; 123. } 124. if(io!=NULL) { 125. CLS1_SendStr("sending>>:\r\n", io->stdOut); 126. CLS1_SendStr(cmd, io->stdOut); 127. } 128. if(AS2_SendBlock(cmd, (uint16_t)UTIL1_strlen((char*)cmd), &snt) != ERR_OK) { 129. returnERR_FAILED; 130. } 131. if(rxBuf!=NULL) { 132. res = RxResponse(rxBuf, rxBufSize, expectedTailStr, msTimeout); 133. if(io!=NULL) { 134. CLS1_SendStr("received<<:\r\n", io->stdOut); 135. CLS1_SendStr(rxBuf, io->stdOut); 136. } 137. } 138. returnres; 139.} 140. 141.uint8_t ESP_TestAT(void) { 142. /* AT */ 143. uint8_t rxBuf[sizeof("AT\r\r\n\r\nOK\r\n")]; 144. uint8_t res; 145. 146. res = ESP_SendATCommand("AT\r\n", rxBuf, sizeof(rxBuf), "AT\r\r\n\r\nOK\r\n", ESP_DEFAULT_TIMEOUT_MS, NULL); 147. returnres; 148.} 149. 150.uint8_t ESP_Restart(constCLS1_StdIOType *io, uint16_t timeoutMs) { 151. /* AT+RST */ 152. uint8_t rxBuf[sizeof("AT+RST\r\r\n\r\nOK\r\n")]; 153. uint8_t res; 154. uint8_t buf[64]; 155. 156. AS2_ClearRxBuf(); /* clear buffer */ 157. res = ESP_SendATCommand("AT+RST\r\n", rxBuf, sizeof(rxBuf), "AT+RST\r\r\n\r\nOK\r\n", ESP_DEFAULT_TIMEOUT_MS, io); 158. if(res==ERR_OK) { 159. for(;;) { 160. ESP_ReadCharsUntil(buf, sizeof(buf), '\n', 1000); 161. if(io!=NULL) { 162. CLS1_SendStr(buf, io->stdOut); 163. } 164. if(UTIL1_strncmp(buf, "ready", sizeof("ready")-1)==0) { /* wait until ready message from module */ 165. break; /* module has restarted */ 166. } 167. } 168. } 169. AS2_ClearRxBuf(); /* clear buffer */ 170. returnres; 171.} 172. 173.uint8_t ESP_CloseConnection(uint8_t channel, constCLS1_StdIOType *io, uint16_t timeoutMs) { 174. /* AT+CIPCLOSE= */ 175. uint8_t res; 176. uint8_t cmd[64]; 177. 178. UTIL1_strcpy(cmd, sizeof(cmd), "AT+CIPCLOSE="); 179. UTIL1_strcatNum8u(cmd, sizeof(cmd), channel); 180. UTIL1_strcat(cmd, sizeof(cmd), "\r\n"); 181. res = ESP_SendATCommand(cmd, NULL, 0, "Unlink\r\n", timeoutMs, io); 182. returnres; 183.} 184. 185.uint8_t ESP_SetNumberOfConnections(uint8_t nof, constCLS1_StdIOType *io, uint16_t timeoutMs) { 186. /* AT+CIPMUX=, 0: single connection, 1: multiple connections */ 187. uint8_t res; 188. uint8_t cmd[sizeof("AT+CIPMUX=12\r\n")]; 189. uint8_t rxBuf[sizeof("AT+CIPMUX=12\r\n\r\nOK\r\n")+10]; 190. 191. if(nof>1) { /* only 0 and 1 allowed */ 192. if(io!=NULL) { 193. CLS1_SendStr("Wrong number of connection parameter!\r\n", io->stdErr); 194. } 195. returnERR_FAILED; 196. } 197. UTIL1_strcpy(cmd, sizeof(cmd), "AT+CIPMUX="); 198. UTIL1_strcatNum8u(cmd, sizeof(cmd), nof); 199. UTIL1_strcat(cmd, sizeof(cmd), "\r\n"); 200. res = ESP_SendATCommand(cmd, rxBuf, sizeof(rxBuf), "OK\r\n", timeoutMs, io); 201. returnres; 202.} 203. 204.uint8_t ESP_SetServer(bool startIt, uint16_t port, constCLS1_StdIOType *io, uint16_t timeoutMs) { 205. /* AT+CIPSERVER=,, where : 0: stop, 1: start */ 206. uint8_t res; 207. uint8_t cmd[sizeof("AT+CIPSERVER=1,80\r\n\r\nOK\r\n")+sizeof("no change")]; 208. uint8_t rxBuf[sizeof("AT+CIPSERVER=1,80\r\n\r\nOK\r\n")+sizeof("no change")]; 209. 210. UTIL1_strcpy(cmd, sizeof(cmd), "AT+CIPSERVER="); 211. if(startIt) { 212. UTIL1_strcat(cmd, sizeof(cmd), "1,"); 213. } else{ 214. UTIL1_strcat(cmd, sizeof(cmd), "0,"); 215. } 216. UTIL1_strcatNum16u(cmd, sizeof(cmd), port); 217. UTIL1_strcat(cmd, sizeof(cmd), "\r\n"); 218. res = ESP_SendATCommand(cmd, rxBuf, sizeof(rxBuf), "OK\r\n", timeoutMs, io); 219. if(res!=ERR_OK) { /* accept "no change" too */ 220. UTIL1_strcpy(cmd, sizeof(cmd), "AT+CIPSERVER="); 221. if(startIt) { 222. UTIL1_strcat(cmd, sizeof(cmd), "1,"); 223. } else{ 224. UTIL1_strcat(cmd, sizeof(cmd), "0,"); 225. } 226. UTIL1_strcatNum16u(cmd, sizeof(cmd), port); 227. UTIL1_strcat(cmd, sizeof(cmd), "\r\r\nno change\r\n"); 228. if(UTIL1_strcmp(rxBuf, cmd)==0) { 229. res = ERR_OK; 230. } 231. } 232. returnres; 233.} 234. 235.uint8_t ESP_SelectMode(uint8_t mode) { 236. /* AT+CWMODE=, where is 1=Sta, 2=AP or 3=both */ 237. uint8_t txBuf[sizeof("AT+CWMODE=x\r\n")]; 238. uint8_t rxBuf[sizeof("AT+CWMODE=x\r\r\nno change\r\n")]; 239. uint8_t expected[sizeof("AT+CWMODE=x\r\r\nno change\r\n")]; 240. uint8_t res; 241. 242. if(mode<1|| mode>3) { 243. returnERR_RANGE; /* only 1, 2 or 3 */ 244. } 245. UTIL1_strcpy(txBuf, sizeof(txBuf), "AT+CWMODE="); 246. UTIL1_strcatNum16u(txBuf, sizeof(txBuf), mode); 247. UTIL1_strcat(txBuf, sizeof(txBuf), "\r\n"); 248. UTIL1_strcpy(expected, sizeof(expected), "AT+CWMODE="); 249. UTIL1_strcatNum16u(expected, sizeof(expected), mode); 250. UTIL1_strcat(expected, sizeof(expected), "\r\r\n\n"); 251. res = ESP_SendATCommand(txBuf, rxBuf, sizeof(rxBuf), expected, ESP_DEFAULT_TIMEOUT_MS, NULL); 252. if(res!=ERR_OK) { 253. /* answer could be as well "AT+CWMODE=x\r\r\nno change\r\n"!! */ 254. UTIL1_strcpy(txBuf, sizeof(txBuf), "AT+CWMODE="); 255. UTIL1_strcatNum16u(txBuf, sizeof(txBuf), mode); 256. UTIL1_strcat(txBuf, sizeof(txBuf), "\r\n"); 257. UTIL1_strcpy(expected, sizeof(expected), "AT+CWMODE="); 258. UTIL1_strcatNum16u(expected, sizeof(expected), mode); 259. UTIL1_strcat(expected, sizeof(expected), "\r\r\nno change\r\n"); 260. if(UTIL1_strcmp(rxBuf, expected)==0) { 261. res = ERR_OK; 262. } 263. } 264. returnres; 265.} 266. 267.uint8_t ESP_GetFirmwareVersionString(uint8_t *fwBuf, size_t fwBufSize) { 268. /* AT+GMR */ 269. uint8_t rxBuf[32]; 270. uint8_t res; 271. constunsigned char*p; 272. 273. res = ESP_SendATCommand("AT+GMR\r\n", rxBuf, sizeof(rxBuf), "\r\n\r\nOK\r\n", ESP_DEFAULT_TIMEOUT_MS, NULL); 274. if(res!=ERR_OK) { 275. if(UTIL1_strtailcmp(rxBuf, "\r\n\r\nOK\r\n")) { 276. res = ERR_OK; 277. } 278. } 279. if(res==ERR_OK) { 280. if(UTIL1_strncmp(rxBuf, "AT+GMR\r\r\n", sizeof("AT+GMR\r\r\n")-1)==0) { /* check for beginning of response */ 281. UTIL1_strCutTail(rxBuf, "\r\n\r\nOK\r\n"); /* cut tailing response */ 282. p = rxBuf+sizeof("AT+GMR\r\r\n")-1; /* skip beginning */ 283. UTIL1_strcpy(fwBuf, fwBufSize, p); /* copy firmware information string */ 284. } else{ 285. res = ERR_FAILED; 286. } 287. } 288. if(res!=ERR_OK) { 289. UTIL1_strcpy(fwBuf, fwBufSize, "ERROR"); /* default error */ 290. } 291. returnres; 292.} 293. 294.uint8_t ESP_GetIPAddrString(uint8_t *ipBuf, size_t ipBufSize) { 295. /* AT+CIFSR */ 296. uint8_t rxBuf[32]; 297. uint8_t res; 298. constunsigned char*p; 299. 300. res = ESP_SendATCommand("AT+CIFSR\r\n", rxBuf, sizeof(rxBuf), NULL, ESP_DEFAULT_TIMEOUT_MS, NULL); 301. if(res!=ERR_OK) { 302. if(UTIL1_strtailcmp(rxBuf, "\r\n")) { 303. res = ERR_OK; 304. } 305. } 306. if(res==ERR_OK) { 307. if(UTIL1_strncmp(rxBuf, "AT+CIFSR\r\r\n", sizeof("AT+CIFSR\r\r\n")-1)==0) { /* check for beginning of response */ 308. UTIL1_strCutTail(rxBuf, "\r\n"); /* cut tailing response */ 309. p = rxBuf+sizeof("AT+CIFSR\r\r\n")-1; /* skip beginning */ 310. SkipNewLines(&p); 311. UTIL1_strcpy(ipBuf, ipBufSize, p); /* copy IP information string */ 312. } else{ 313. res = ERR_FAILED; 314. } 315. } 316. if(res!=ERR_OK) { 317. UTIL1_strcpy(ipBuf, ipBufSize, "ERROR"); 318. } 319. returnres; 320.} 321. 322.uint8_t ESP_GetModeString(uint8_t *buf, size_t bufSize) { 323. /* AT+CWMODE? */ 324. uint8_t rxBuf[32]; 325. uint8_t res; 326. constunsigned char*p; 327. 328. res = ESP_SendATCommand("AT+CWMODE?\r\n", rxBuf, sizeof(rxBuf), "\r\n\r\nOK\r\n", ESP_DEFAULT_TIMEOUT_MS, NULL); 329. if(res==ERR_OK) { 330. if(UTIL1_strncmp(rxBuf, "AT+CWMODE?\r\r\n+CWMODE:", sizeof("AT+CWMODE?\r\r\n+CWMODE:")-1)==0) { /* check for beginning of response */ 331. UTIL1_strCutTail(rxBuf, "\r\n\r\nOK\r\n"); /* cut tailing response */ 332. p = rxBuf+sizeof("AT+CWMODE?\r\r\n+CWMODE:")-1; /* skip beginning */ 333. UTIL1_strcpy(buf, bufSize, p); /* copy information string */ 334. } else{ 335. res = ERR_FAILED; 336. } 337. } 338. if(res!=ERR_OK) { 339. UTIL1_strcpy(buf, bufSize, "ERROR"); 340. } 341. returnres; 342.} 343. 344.uint8_t ESP_GetCIPMUXString(uint8_t *cipmuxBuf, size_t cipmuxBufSize) { 345. /* AT+CIPMUX? */ 346. uint8_t rxBuf[32]; 347. uint8_t res; 348. constunsigned char*p; 349. 350. res = ESP_SendATCommand("AT+CIPMUX?\r\n", rxBuf, sizeof(rxBuf), "\r\n\r\nOK\r\n", ESP_DEFAULT_TIMEOUT_MS, NULL); 351. if(res==ERR_OK) { 352. if(UTIL1_strncmp(rxBuf, "AT+CIPMUX?\r\r\n+CIPMUX:", sizeof("AT+CIPMUX?\r\r\n+CIPMUX:")-1)==0) { /* check for beginning of response */ 353. UTIL1_strCutTail(rxBuf, "\r\n\r\nOK\r\n"); /* cut tailing response */ 354. p = rxBuf+sizeof("AT+CIPMUX?\r\r\n+CIPMUX:")-1; /* skip beginning */ 355. UTIL1_strcpy(cipmuxBuf, cipmuxBufSize, p); /* copy IP information string */ 356. } else{ 357. res = ERR_FAILED; 358. } 359. } 360. if(res!=ERR_OK) { 361. UTIL1_strcpy(cipmuxBuf, cipmuxBufSize, "ERROR"); 362. } 363. returnres; 364.} 365. 366.uint8_t ESP_GetConnectedAPString(uint8_t *apBuf, size_t apBufSize) { 367. /* AT+CWJAP? */ 368. uint8_t rxBuf[48]; 369. uint8_t res; 370. constunsigned char*p; 371. 372. res = ESP_SendATCommand("AT+CWJAP?\r\n", rxBuf, sizeof(rxBuf), "\r\n\r\nOK\r\n", ESP_DEFAULT_TIMEOUT_MS, NULL); 373. if(res==ERR_OK) { 374. if(UTIL1_strncmp(rxBuf, "AT+CWJAP?\r\r\n+CWJAP:\"", sizeof("AT+CWJAP?\r\r\n+CWJAP:\"")-1)==0) { /* check for beginning of response */ 375. UTIL1_strCutTail(rxBuf, "\"\r\n\r\nOK\r\n"); /* cut tailing response */ 376. p = rxBuf+sizeof("AT+CWJAP?\r\r\n+CWJAP:\"")-1; /* skip beginning */ 377. UTIL1_strcpy(apBuf, apBufSize, p); /* copy IP information string */ 378. } else{ 379. res = ERR_FAILED; 380. } 381. } 382. if(res!=ERR_OK) { 383. UTIL1_strcpy(apBuf, apBufSize, "ERROR"); 384. } 385. returnres; 386. 387.} 388. 389.staticuint8_t JoinAccessPoint(constuint8_t *ssid, constuint8_t *pwd, CLS1_ConstStdIOType *io) { 390. /* AT+CWJAP="","" */ 391. uint8_t txBuf[48]; 392. uint8_t rxBuf[64]; 393. uint8_t expected[48]; 394. 395. UTIL1_strcpy(txBuf, sizeof(txBuf), "AT+CWJAP=\""); 396. UTIL1_strcat(txBuf, sizeof(txBuf), ssid); 397. UTIL1_strcat(txBuf, sizeof(txBuf), "\",\""); 398. UTIL1_strcat(txBuf, sizeof(txBuf), pwd); 399. UTIL1_strcat(txBuf, sizeof(txBuf), "\"\r\n"); 400. 401. UTIL1_strcpy(expected, sizeof(expected), "AT+CWJAP=\""); 402. UTIL1_strcat(expected, sizeof(expected), ssid); 403. UTIL1_strcat(expected, sizeof(expected), "\",\""); 404. UTIL1_strcat(expected, sizeof(expected), pwd); 405. UTIL1_strcat(expected, sizeof(expected), "\"\r\r\n\r\nOK\r\n"); 406. 407. returnESP_SendATCommand(txBuf, rxBuf, sizeof(rxBuf), expected, ESP_DEFAULT_TIMEOUT_MS, io); 408.} 409. 410.uint8_t ESP_JoinAP(constuint8_t *ssid, constuint8_t *pwd, intnofRetries, CLS1_ConstStdIOType *io) { 411. uint8_t buf[32]; 412. uint8_t res; 413. 414. do{ 415. res = JoinAccessPoint(ssid, pwd, io); 416. if(res==ERR_OK) { 417. break; 418. } 419. WAIT1_WaitOSms(1000); 420. nofRetries--; 421. } while(nofRetries>0); 422. returnres; 423.} 424. 425.staticuint8_t ReadIntoIPDBuffer(uint8_t *buf, size_t bufSize, uint8_t *p, uint16_t msgSize, uint16_t msTimeout, constCLS1_StdIOType *io) { 426. uint8_t ch; 427. size_t nofInBuf; 428. inttimeout; 429. 430. nofInBuf = p-buf; 431. bufSize -= nofInBuf; /* take into account what we already have in buffer */ 432. timeout = msTimeout; 433. while(msgSize>0&& bufSize>0) { 434. if(AS2_GetCharsInRxBuf()>0) { 435. (void)AS2_RecvChar(&ch); 436. *p = ch; 437. if(io!=NULL) { /* copy on console */ 438. io->stdOut(ch); 439. } 440. p++; 441. *p = '\0'; /* terminate */ 442. nofInBuf++; msgSize--; bufSize--; 443. } else{ 444. /* check in case we recveive less characters than expected, happens for POST? */ 445. if(nofInBuf>6&& UTIL1_strncmp(&p[-6], "\r\nOK\r\n", sizeof("\r\nOK\r\n")-1)==0) { 446. break; 447. } else{ 448. timeout -= 10; 449. WAIT1_WaitOSms(10); 450. if(timeout<0) { 451. returnERR_BUSY; 452. } 453. } 454. } 455. } 456. returnERR_OK; 457.} 458. 459.uint8_t ESP_GetIPD(uint8_t *msgBuf, size_t msgBufSize, uint8_t *ch_id, uint16_t *size, bool *isGet, uint16_t timeoutMs, constCLS1_StdIOType *io) { 460. /* scan e.g. for 461. * +IPD,0,404:POST / HTTP/1.1 462. * and return ch_id (0), size (404) 463. */ 464. uint8_t res = ERR_OK; 465. constuint8_t *p; 466. bool isIPD = FALSE; 467. uint8_t cmd[24], rxBuf[48]; 468. uint16_t ipdSize; 469. 470. *ch_id = 0; *size = 0; *isGet = FALSE; /* init */ 471. for(;;) { /* breaks */ 472. res = ESP_ReadCharsUntil(msgBuf, msgBufSize, '\n', timeoutMs); 473. if(res!=ERR_OK) { 474. break; /* timeout */ 475. } 476. if(res==ERR_OK) { /* line read */ 477. if(io!=NULL) { 478. CLS1_SendStr(msgBuf, io->stdOut); /* copy on console */ 479. } 480. isIPD = UTIL1_strncmp(msgBuf, "+IPD,", sizeof("+IPD,")-1)==0; 481. if(isIPD) { /* start of IPD message */ 482. p = msgBuf+sizeof("+IPD,")-1; 483. if(UTIL1_ScanDecimal8uNumber(&p, ch_id)!=ERR_OK) { 484. if(io!=NULL) { 485. CLS1_SendStr("ERR: wrong channel?\r\n", io->stdErr); /* error on console */ 486. } 487. res = ERR_FAILED; 488. break; 489. } 490. if(*p!=',') { 491. res = ERR_FAILED; 492. break; 493. } 494. p++; /* skip comma */ 495. if(UTIL1_ScanDecimal16uNumber(&p, size)!=ERR_OK) { 496. if(io!=NULL) { 497. CLS1_SendStr("ERR: wrong size?\r\n", io->stdErr); /* error on console */ 498. } 499. res = ERR_FAILED; 500. break; 501. } 502. if(*p!=':') { 503. res = ERR_FAILED; 504. break; 505. } 506. ipdSize = p-msgBuf; /* length of "+IPD,," string */ 507. p++; /* skip ':' */ 508. if(UTIL1_strncmp(p, "GET", sizeof("GET")-1)==0) { 509. *isGet = TRUE; 510. } elseif(UTIL1_strncmp(p, "POST", sizeof("POST")-1)==0) { 511. *isGet = FALSE; 512. } else{ 513. res = ERR_FAILED; 514. } 515. while(*p!='\0') { 516. p++; /* skip to the end */ 517. } 518. /* read the rest of the message */ 519. res = ReadIntoIPDBuffer(msgBuf, msgBufSize, (uint8_t*)p, (*size)-ipdSize, ESP_DEFAULT_TIMEOUT_MS, io); 520. break; 521. } 522. } 523. } 524. returnres; 525.} 526. 527.uint8_t ESP_StartWebServer(constCLS1_StdIOType *io) { 528. uint8_t buf[32]; 529. uint8_t res; 530. 531. res = ESP_SetNumberOfConnections(1, io, ESP_DEFAULT_TIMEOUT_MS); 532. if(res!=ERR_OK) { 533. CLS1_SendStr("ERR: failed to set multiple connections.\r\n", io->stdErr); 534. returnres; 535. } 536. res = ESP_SetServer(TRUE, 80, io, ESP_DEFAULT_TIMEOUT_MS); 537. if(res!=ERR_OK) { 538. CLS1_SendStr("ERR: failed to set server.\r\n", io->stdErr); 539. returnres; 540. } 541. CLS1_SendStr("INFO: Web Server started, waiting for connection on ", io->stdOut); 542. if(ESP_GetIPAddrString(buf, sizeof(buf))==ERR_OK) { 543. CLS1_SendStr(buf, io->stdOut); 544. CLS1_SendStr(":80", io->stdOut); 545. } else{ 546. CLS1_SendStr("(ERROR!)", io->stdOut); 547. } 548. CLS1_SendStr("\r\n", io->stdOut); 549. 550. returnERR_OK; 551.} 552. 553.uint8_t ESP_SendStr(constuint8_t *str, CLS1_ConstStdIOType *io) { 554. uint8_t buf[32]; 555. uint8_t rxBuf[48]; 556. uint8_t res; 557. uint16_t timeoutMs; 558. #define RX_TIMEOUT_MS 3000 559. AS2_TComData ch; 560. 561. UTIL1_strcpy(buf, sizeof(buf), str); 562. UTIL1_strcat(buf, sizeof(buf), "\r\n"); 563. res = ESP_SendATCommand(buf, rxBuf, sizeof(rxBuf), NULL, ESP_DEFAULT_TIMEOUT_MS, io); 564. timeoutMs = 0; 565. while(timeoutMs0) { 569. (void)AS2_RecvChar(&ch); 570. CLS1_SendChar(ch); 571. } 572. } 573. returnERR_OK; 574.} 575. 576.staticuint8_t ESP_PrintHelp(constCLS1_StdIOType *io) { 577. CLS1_SendHelpStr("ESP", "ESP8200 commands\r\n", io->stdOut); 578. CLS1_SendHelpStr(" help|status", "Print help or status information\r\n", io->stdOut); 579. CLS1_SendHelpStr(" send ", "Sends a string to the module\r\n", io->stdOut); 580. CLS1_SendHelpStr(" test", "Sends a test AT command\r\n", io->stdOut); 581. CLS1_SendHelpStr(" restart", "Restart module\r\n", io->stdOut); 582. CLS1_SendHelpStr(" listAP", "List available Access Points\r\n", io->stdOut); 583. CLS1_SendHelpStr(" connectAP \"ssid\",\"pwd\"", "Connect to an Access Point\r\n", io->stdOut); 584. CLS1_SendHelpStr(" server (start|stop)", "Start or stop web server\r\n", io->stdOut); 585. returnERR_OK; 586.} 587. 588.staticuint8_t ESP_PrintStatus(constCLS1_StdIOType *io) { 589. uint8_t buf[48]; 590. 591. CLS1_SendStatusStr("ESP8266", "\r\n", io->stdOut); 592. 593. CLS1_SendStatusStr(" Webserver", ESP_WebServerIsOn?"ON\r\n":"OFF\r\n", io->stdOut); 594. 595. if(ESP_GetFirmwareVersionString(buf, sizeof(buf)) != ERR_OK) { 596. UTIL1_strcpy(buf, sizeof(buf), "FAILED\r\n"); 597. } else{ 598. UTIL1_strcat(buf, sizeof(buf), "\r\n"); 599. } 600. CLS1_SendStatusStr(" AT+GMR", buf, io->stdOut); 601. 602. if(ESP_GetModeString(buf, sizeof(buf)) != ERR_OK) { 603. UTIL1_strcpy(buf, sizeof(buf), "FAILED\r\n"); 604. } else{ 605. if(UTIL1_strcmp(buf, "1")==0) { 606. UTIL1_strcat(buf, sizeof(buf), " (device)"); 607. } elseif(UTIL1_strcmp(buf, "2")==0) { 608. UTIL1_strcat(buf, sizeof(buf), " (AP)"); 609. } elseif(UTIL1_strcmp(buf, "3")==0) { 610. UTIL1_strcat(buf, sizeof(buf), " (device+AP)"); 611. } else{ 612. UTIL1_strcat(buf, sizeof(buf), " (ERROR)"); 613. } 614. UTIL1_strcat(buf, sizeof(buf), "\r\n"); 615. } 616. CLS1_SendStatusStr(" AT+CWMODE?", buf, io->stdOut); 617. 618. if(ESP_GetIPAddrString(buf, sizeof(buf)) != ERR_OK) { 619. UTIL1_strcpy(buf, sizeof(buf), "FAILED\r\n"); 620. } else{ 621. UTIL1_strcat(buf, sizeof(buf), "\r\n"); 622. } 623. CLS1_SendStatusStr(" AT+CIFSR", buf, io->stdOut); 624. 625. if(ESP_GetConnectedAPString(buf, sizeof(buf)) != ERR_OK) { 626. UTIL1_strcpy(buf, sizeof(buf), "FAILED\r\n"); 627. } else{ 628. UTIL1_strcat(buf, sizeof(buf), "\r\n"); 629. } 630. CLS1_SendStatusStr(" AT+CWJAP?", buf, io->stdOut); 631. 632. if(ESP_GetCIPMUXString(buf, sizeof(buf)) != ERR_OK) { 633. UTIL1_strcpy(buf, sizeof(buf), "FAILED\r\n"); 634. } else{ 635. if(UTIL1_strcmp(buf, "0")==0) { 636. UTIL1_strcat(buf, sizeof(buf), " (single connection)"); 637. } elseif(UTIL1_strcmp(buf, "1")==0) { 638. UTIL1_strcat(buf, sizeof(buf), " (multiple connections)"); 639. } else{ 640. UTIL1_strcat(buf, sizeof(buf), " (ERROR)"); 641. } 642. UTIL1_strcat(buf, sizeof(buf), "\r\n"); 643. } 644. CLS1_SendStatusStr(" CIPMUX", buf, io->stdOut); 645. returnERR_OK; 646.} 647. 648.uint8_t ESP_ParseCommand(constunsigned char*cmd, bool *handled, constCLS1_StdIOType *io) { 649. uint32_t val; 650. uint8_t res; 651. constunsigned char*p; 652. uint8_t pwd[24], ssid[24]; 653. 654. if(UTIL1_strcmp((char*)cmd, CLS1_CMD_HELP)==0|| UTIL1_strcmp((char*)cmd, "ESP help")==0) { 655. *handled = TRUE; 656. res = ESP_PrintHelp(io); 657. } elseif(UTIL1_strcmp((char*)cmd, CLS1_CMD_STATUS)==0|| UTIL1_strcmp((char*)cmd, "ESP status")==0) { 658. *handled = TRUE; 659. res = ESP_PrintStatus(io); 660. } elseif(UTIL1_strncmp((char*)cmd, "ESP send ", sizeof("ESP send ")-1)==0) { 661. *handled = TRUE; 662. p = cmd+sizeof("ESP send ")-1; 663. 664. (void)ESP_SendStr(p, io); 665. } elseif(UTIL1_strcmp((char*)cmd, "ESP test")==0) { 666. *handled = TRUE; 667. if(ESP_TestAT()!=ERR_OK) { 668. CLS1_SendStr("TEST failed!\r\n", io->stdErr); 669. res = ERR_FAILED; 670. } else{ 671. CLS1_SendStr("TEST ok!\r\n", io->stdOut); 672. } 673. } elseif(UTIL1_strcmp((char*)cmd, "ESP listAP")==0) { 674. *handled = TRUE; 675. (void)ESP_SendStr("AT+CWLAP", io); 676. /* AT + CWLAP 677. response 678. + CWLAP: , , [, ] 679. OK Or Fails, the return ERROR 680. 0 OPEN 681. 1 WEP 682. 2 WPA_PSK 683. 3 WPA2_PSK 684. 4 WPA_WPA2_PSK 685. string parameter, the access point name 686. signal strength 687. 0: manually connect 1: An automatic connection 688. */ 689. returnERR_OK; 690. } elseif(UTIL1_strncmp((char*)cmd, "ESP connectAP ", sizeof("ESP connectAP ")-1)==0) { 691. *handled = TRUE; 692. p = cmd+sizeof("ESP connectAP ")-1; 693. ssid[0] = '\0'; pwd[0] = '\0'; 694. res = UTIL1_ScanDoubleQuotedString(&p, ssid, sizeof(ssid)); 695. if(res==ERR_OK && *p!='\0'&& *p==',') { 696. p++; /* skip comma */ 697. res = UTIL1_ScanDoubleQuotedString(&p, pwd, sizeof(pwd)); 698. } else{ 699. CLS1_SendStr("Comma expected between strings!\r\n", io->stdErr); 700. res = ERR_FAILED; 701. } 702. if(res==ERR_OK) { 703. res = ESP_JoinAP(ssid, pwd, 3, io); 704. } else{ 705. CLS1_SendStr("Wrong command format!\r\n", io->stdErr); 706. res = ERR_FAILED; 707. } 708. } elseif(UTIL1_strcmp((char*)cmd, "ESP server start")==0) { 709. *handled = TRUE; 710. res = ESP_StartWebServer(io); 711. ESP_WebServerIsOn = res==ERR_OK; 712. } elseif(UTIL1_strcmp((char*)cmd, "ESP server stop")==0) { 713. *handled = TRUE; 714. ESP_WebServerIsOn = FALSE; 715. } elseif(UTIL1_strcmp((char*)cmd, "ESP restart")==0) { 716. *handled = TRUE; 717. ESP_Restart(io, 2000); 718. } 719. returnres; 720.} 721. 722.voidESP_Deinit(void) { 723. /* nothing to do */ 724.} 725. 726.voidESP_Init(void) { 727. AS2_ClearRxBuf(); /* clear buffer */ 728.} The application interface in Application.h is rather short :-): view source print? 01./* 02. * Application.h 03. * 04. * Author: Erich Styger 05. */ 06. 07.#ifndef APPLICATION_H_ 08.#define APPLICATION_H_ 09. 10./*! 11. * \brief Application main routine 12. */ 13.voidAPP_Run(void); 14. 15.#endif /* APPLICATION_H_ */ The main loop of the application is Application.c, along with the application specific web server code. As the SendWebPage function contains HTML code, I’m posting it here separately: view source print? 01.staticuint8_t SendWebPage(uint8_t ch_id, bool ledIsOn, uint8_t temperature, constCLS1_StdIOType *io) { 02. staticuint8_t http[1024]; 03. uint8_t cmd[24], rxBuf[48], expected[48]; 04. uint8_t buf[16]; 05. uint8_t res = ERR_OK; 06. 07. /* construct web page content */ 08. UTIL1_strcpy(http, sizeof(http), (uint8_t*)"HTTP/1.0 200 OK\r\nContent-Type: text/html\r\nPragma: no-cache\r\n\r\n"); 09. UTIL1_strcat(http, sizeof(http), (uint8_t*)"\r\n\r\n"); 10. UTIL1_strcat(http, sizeof(http), (uint8_t*)"\r\n"); 11. UTIL1_strcat(http, sizeof(http), (uint8_t*)"Web Server using ESP8266\r\n"); 12. UTIL1_strcat(http, sizeof(http), (uint8_t*)" 13.\r\n"); 14. UTIL1_strcat(http, sizeof(http), (uint8_t*)"Temp: OC"); 17. if(ledIsOn) { 18. UTIL1_strcat(http, sizeof(http), (uint8_t*)"Red LED off"); 19. UTIL1_strcat(http, sizeof(http), (uint8_t*)" 20.Red LED on"); 21. } else{ 22. UTIL1_strcat(http, sizeof(http), (uint8_t*)"Red LED off"); 23. UTIL1_strcat(http, sizeof(http), (uint8_t*)" 24.Red LED on"); 25. } 26. UTIL1_strcat(http, sizeof(http), (uint8_t*)""); 27. UTIL1_strcat(http, sizeof(http), (uint8_t*)"\r\n\r\n"); 28. 29. UTIL1_strcpy(cmd, sizeof(cmd), "AT+CIPSEND="); /* parameters are , */ 30. UTIL1_strcatNum8u(cmd, sizeof(cmd), ch_id); 31. UTIL1_chcat(cmd, sizeof(cmd), ','); 32. UTIL1_strcatNum16u(cmd, sizeof(cmd), UTIL1_strlen(http)); 33. UTIL1_strcpy(expected, sizeof(expected), cmd); /* we expect the echo of our command */ 34. UTIL1_strcat(expected, sizeof(expected), "\r\r\n> "); /* expect "> " */ 35. UTIL1_strcat(cmd, sizeof(cmd), "\r\n"); 36. res = ESP_SendATCommand(cmd, rxBuf, sizeof(rxBuf), expected, ESP_DEFAULT_TIMEOUT_MS, io); 37. if(res!=ERR_OK) { 38. if(io!=NULL) { 39. CLS1_SendStr("INFO: TIMEOUT, closing connection!\r\n", io->stdOut); 40. } 41. } else{ 42. if(io!=NULL) { 43. CLS1_SendStr("INFO: Sending http page...\r\n", io->stdOut); 44. } 45. UTIL1_strcat(http, sizeof(http), "\r\n\r\n"); /* need to add this to end the command! */ 46. res = ESP_SendATCommand(http, NULL, 0, NULL, ESP_DEFAULT_TIMEOUT_MS, io); 47. if(res!=ERR_OK) { 48. CLS1_SendStr("Sending page failed!\r\n", io->stdErr); /* copy on console */ 49. } else{ 50. for(;;) { /* breaks */ 51. res = ESP_ReadCharsUntil(buf, sizeof(buf), '\n', 1000); 52. if(res==ERR_OK) { /* line read */ 53. if(io!=NULL) { 54. CLS1_SendStr(buf, io->stdOut); /* copy on console */ 55. } 56. } 57. if(UTIL1_strncmp(buf, "SEND OK\r\n", sizeof("SEND OK\r\n")-1)==0) { /* ok from module */ 58. break; 59. } 60. } 61. } 62. } 63. returnres; 64.} The rest of Application.c is rather simple: view source print? 01./* 02. * Application.c 03. * 04. * Author: Erich Styger 05. */ 06.#include "PE_Types.h" 07.#include "CLS1.h" 08.#include "WAIT1.h" 09.#include "Shell.h" 10.#include "UTIL1.h" 11.#include "ESP8266.h" 12.#include "LEDR.h" 13.#include "LEDG.h" 14.#include "AS2.h" 15. 16.staticuint8_t APP_EspMsgBuf[512]; /* buffer for messages from ESP8266 */ 17. 18.staticvoidWebProcess(void) { 19. uint8_t res=ERR_OK; 20. bool isGet; 21. uint8_t ch_id=0; 22. uint16_t size=0; 23. constuint8_t *p; 24. constCLS1_StdIOType *io; 25. 26. if(ESP_IsServerOn()) { 27. io = CLS1_GetStdio(); 28. res = ESP_GetIPD(APP_EspMsgBuf, sizeof(APP_EspMsgBuf), &ch_id, &size, &isGet, 1000, io); 29. if(res==ERR_OK) { 30. if(isGet) { /* GET: put web page */ 31. res = SendWebPage(ch_id, LEDR_Get()!=FALSE, 21/*dummy temperature*/, io); 32. if(res!=ERR_OK && io!=NULL) { 33. CLS1_SendStr("Sending page failed!\r\n", io->stdErr); /* copy on console */ 34. } 35. } else{ /* POST: received info */ 36. intpos; 37. 38. pos = UTIL1_strFind(APP_EspMsgBuf, "radio="); 39. if(pos!=-1) { /* found */ 40. if(UTIL1_strncmp(&APP_EspMsgBuf[pos], "radio=0", sizeof("radio=0")-1)) { 41. LEDR_On(); 42. } elseif(UTIL1_strncmp(&APP_EspMsgBuf[pos], "radio=1", sizeof("radio=1")-1)) { 43. LEDR_Off(); 44. } 45. } 46. res = SendWebPage(ch_id, LEDR_Get()!=FALSE, 20/*dummy temperature*/, io); 47. if(res!=ERR_OK && io!=NULL) { 48. CLS1_SendStr("Sending page failed!\r\n", io->stdErr); /* copy on console */ 49. } 50. } 51. CLS1_SendStr("INFO: Closing connection...\r\n", io->stdOut); 52. res = ESP_CloseConnection(ch_id, io, ESP_DEFAULT_TIMEOUT_MS); 53. } 54. } else{ /* copy messages we receive to console */ 55. while(AS2_GetCharsInRxBuf()>0) { 56. uint8_t ch; 57. 58. (void)AS2_RecvChar(&ch); 59. CLS1_SendChar(ch); 60. } 61. } 62.} 63. 64.voidAPP_Run(void) { 65. CLS1_ConstStdIOType *io; 66. 67. WAIT1_Waitms(1000); /* wait after power-on */ 68. ESP_Init(); 69. SHELL_Init(); 70. io = CLS1_GetStdio(); 71. CLS1_SendStr("\r\n------------------------------------------\r\n", io->stdOut); 72. CLS1_SendStr("ESP8266 with FRDM-KL25Z\r\n", io->stdOut); 73. CLS1_SendStr("------------------------------------------\r\n", io->stdOut); 74. CLS1_PrintPrompt(io); 75. for(;;) { 76. WebProcess(); 77. SHELL_Parse(); 78. WAIT1_Waitms(10); 79. LEDG_Neg(); 80. } 81.} In main.c I call the application part: view source print? 01./* ################################################################### 02.** Filename : main.c 03.** Project : FRDM-KL25Z_ESP8266 04.** Processor : MKL25Z128VLK4 05.** Version : Driver 01.01 06.** Compiler : GNU C Compiler 07.** Date/Time : 2014-10-15, 14:28, # CodeGen: 0 08.** Abstract : 09.** Main module. 10.** This module contains user's application code. 11.** Settings : 12.** Contents : 13.** No public methods 14.** 15.** ###################################################################*/ 16./*! 17.** @file main.c 18.** @version 01.01 19.** @brief 20.** Main module. 21.** This module contains user's application code. 22.*/ 23./*! 24.** @addtogroup main_module main module documentation 25.** @{ 26.*/ 27./* MODULE main */ 28. 29./* Including needed modules to compile this module/procedure */ 30.#include "Cpu.h" 31.#include "Events.h" 32.#include "WAIT1.h" 33.#include "UTIL1.h" 34.#include "AS1.h" 35.#include "ASerialLdd1.h" 36.#include "CLS1.h" 37.#include "CS1.h" 38.#include "AS2.h" 39.#include "ASerialLdd2.h" 40.#include "LEDR.h" 41.#include "LEDpin1.h" 42.#include "BitIoLdd1.h" 43.#include "LEDG.h" 44.#include "LEDpin2.h" 45.#include "BitIoLdd2.h" 46.#include "LEDB.h" 47.#include "LEDpin3.h" 48.#include "BitIoLdd3.h" 49./* Including shared modules, which are used for whole project */ 50.#include "PE_Types.h" 51.#include "PE_Error.h" 52.#include "PE_Const.h" 53.#include "IO_Map.h" 54./* User includes (#include below this line is not maintained by Processor Expert) */ 55.#include "Application.h" 56. 57./*lint -save -e970 Disable MISRA rule (6.3) checking. */ 58.intmain(void) 59./*lint -restore Enable MISRA rule (6.3) checking. */ 60.{ 61. /* Write your local variable definition here */ 62. 63. /*** Processor Expert internal initialization. DON'T REMOVE THIS CODE!!! ***/ 64. PE_low_level_init(); 65. /*** End of Processor Expert internal initialization. ***/ 66. 67. APP_Run(); 68. 69. /*** Don't write any code pass this line, or it will be deleted during code generation. ***/ 70. /*** RTOS startup code. Macro PEX_RTOS_START is defined by the RTOS component. DON'T MODIFY THIS CODE!!! ***/ 71. #ifdef PEX_RTOS_START 72. PEX_RTOS_START(); /* Startup of the selected RTOS. Macro is defined by the RTOS component. */ 73. #endif 74. /*** End of RTOS startup code. ***/ 75. /*** Processor Expert end of main routine. DON'T MODIFY THIS CODE!!! ***/ 76. for(;;){} 77. /*** Processor Expert end of main routine. DON'T WRITE CODE BELOW!!! ***/ 78.} /*** End of main routine. DO NOT MODIFY THIS TEXT!!! ***/ 79. 80./* END main */ 81./*! 82.** @} 83.*/ 84./* 85.** ################################################################### 86.** 87.** This file was created by Processor Expert 10.4 [05.10] 88.** for the Freescale Kinetis series of microcontrollers. 89.** 90.** ################################################################### 91.*/ Processor Expert Components In addition, I’m using several Processor Expert component which are available fromSourceForge. Processor Expert Components Wait: Busy waiting component, e.g. to wait for a few milliseconds. Utility: string manipulation and utility functions. AsynchroSerial (AS1): serial interface to the host for the shell command line interface Shell: command line shell implementation CriticalSection: for creating critical sections AsynchroSerial (AS2): serial interface to the ESP8266 module LEDR, LEDG and LEDB: Red, Green and Blue LED on the FRDM-KL25Z board AS1 is configured as UART connection (over OpenSDA) for the shell: Shell UART Settings There are no special settings for the Shell component: Shell Settings Important are the correct settings to the ESP8266 UART: 115200 baud and using the correct pins on the board connected to the Rx and Tx lines of the ESP8266. I’m using rather large input and output buffers: UART connection to ESP8266 The LED components are configured for the pins used on the board: PTB18 for red, PTB19 for green and PTD1 for blue LED. Red LED for FRDM-KL25Z Sending Commands The shell implements the command ESP send which I can use to send a string or command to the module: ESP send Note that for every command a trailing “\r\n” will be sent. So instead of using the programmatic way, the shell can be used to ‘manually’ drive a web server, at least most of the part. So I’m using command line commands below to explore how the ESP8266 module works. Using the Shell With the project (link to GitHub below), I have a serial connection and command line shell interface to the module. Compile the project and download it to the FRDM-KL25Z board and use a terminal program (I use Termite) to talk with the module. It power-up, the program shows a greeting message: Greeting Message With ‘help‘ I get a list of the available commands: Help Command The ‘status‘ command gives a system status: Status Command Output With this, I’m ready to send commands to the module :-). Connection Test To test the connection I send a simple ‘AT’ command ESP send AT AT Command Output and the module should respond with AT\r\r\n\r\nOK\r\n Module Restart Sometimes the module gets stuck. What helps is a power-on reset of the module. Another way is to send the AT+RST command to reset the module. The module will boot up and print a ‘ready’ message: Reset of the ESP8266 Access Point or Device First I need to configure if the ESP is either a device or an access point. For this, theCWMODE command is used: AT+CWMODE= where is one of: 1: ‘Sta’, ESP8266 is a device, it connects to an existing access point 2: ‘AP’, ESP8266 is an access point, so other devices can connect to it 3: ‘both’. Not really clear to me, but it seems that in this mode the device is in a hybrid mode? To have the ESP as device so it can connect to an existing access point I use AT+CWMODE=1 and the module should answer with AT+CWMODE=1\r\r\n\r\nOK\r\n or with a ‘no change': AT+CWMODE=1\r\r\nno change\r\n With AT+CWMODE? I can ask for the current mode: Retrieving Current Mode List of Access Points With AT+CWLAP I get a list of access points. It reports a list like this: AT+CWLAP +CWLAP:(0,"",0) +CWLAP:(4,"APforESP",-39) +CWLAP:(4,"iza-97497",-94) OK :!: I experienced problems with that command in an environment with lots of access points visible. In this case it seems the module hands up. Try first in a place with only a few access points. For this tutorial I have configured an access point with SSID “APforESP” which shows up in my list. The list is formatted like this + CWLAP: , , [, ] With following encoding: : 0: OPEN 1: WPA_PSK 2: WPA2_PSK 4: WPA_WPA2_PSK : the SSID (string) of the access point. : Signal strength. : 0: manually connect 1: automatic connect Connecting to Access Point To connect to an access point I use the command AT+CWJAP="","" Of course replace and with your setup. The module should report back an “OK” message, and you are connected :-). :!: The module stores the ssid and password. After power-up, the module will automatically reconnect to the Access Point. IP Address Once connected I can check the IP address I have been assigned to with AT+CIFSR which should give something like AT+CIFSR 192.168.0.111 So now I know my module IP address :-). With this I can ping my module: Pinging my ESP Module Building a Web Server Now as we hav a connection, it is time to use it to run a web server :-).What I want to serve a web page which I can use to turn on or off the LEDs on the board. Number of Connections: CIPMUX Before I start the server I need to make sure it accepts multiple connections. For this I use the following command: AT+CIPMUX=1 The parameter is either 0 (single connection), or 1 (multiple connections). For a web server I need to set it up for multiple connections. The ESP module should respond with AT+CIPMUX=1\r\n\r\nOK\r\n :info: To make it clear, I have included the ‘\r’ and ‘\n’ in the responses. Starting the Server: CIPSERVER I start the server with AT+CIPSERVER=1,80 The first parameter is either 0 (close connection) or 1 (open connection), followed by the port. I use here the standard http port (80). The module should answer with: AT+CIPSERVER=1,80\r\r\n\r\nOK\r\n or if it is already running the server with a ‘no change': AT+CIPSERVER=1,80\r\r\nno change\r\n No I have a connection open on my IP address (see above: 192.168.0.111), listening to the port I have specified (80). Connecting to the Server with Browser I enter the IP address in a web browser: http://192.168.0.111:80 For clarity I have specified the standard HTTP port (80). So if you are using a different port, make sure you specify it in the address line. Connection from FireFox The browser now sends a GET request to the module, and I will see this from the message printed out from the module: First response from Module The ‘Link’ indicates that it has established a link. IPD (IP Data?) is followed by the channel number (this will the one we have to respond to), plus the size of the following data (296 bytes in that case). As I’m not responding (yet), there will be a timeout (after about 1 minute or so), with an ‘Unlink’ message from the module: Link +IPD,0,296:GET / HTTP/1.1 Host: 192.168.0.111 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:33.0) Gecko/20100101 Firefox/33.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: de,en-US;q=0.7,en;q=0.3 Accept-Encoding: gzip, deflate Connection: keep-alive OK Unlink Unlink message Sending Data to Server: CIPSEND Now I need to respond and send data to the browser. For this I need to know the channel number, and this is provided in the IPD message from above, right after the comma: +IPD,0 To send data, I use the command AT+CIPSEND=, So I connect again with the browser, and I send 5 bytes (“hello”) with: AT+CIPSEND=0,5 The ESP8266 responds with AT+CIPSEND=0,5\r\n> Notice the ‘>’ at the end: this is my signal to send the actual data (“hello” in my case): hello The ESP8266 now resonds with a SEND OK: Data Sent However, the browser is still busy and spins around. I already thought that I did something wrong, but after the browser run into a timeout (after about one minute), my data is there! :-) Hello in Browser Closing Connection: CIPCLOSE So things *are* working :-). The trick is that I have to close the connection after I have sent the data. There is a CIPCLOSE command I can use: AT+CIPCLOSE= which I can use to close a channel. So I close the connection with AT+CIPCLOSE=0 and now the browser shows the content right away :-). Web Server Implementation So far I have used the module in command line and manual mode. This is great for exploration of the protocol, but for building the web server I need to do this programmatically. For this I run my ‘main’ loop in APP_Run(). After printing a greeting message and initializing the sub modules, it processes the web/module responses, parses the shell command line interfaces and blinks the green (LEDG) LED. view source print? 01.voidAPP_Run(void) { 02. CLS1_ConstStdIOType *io; 03. 04. WAIT1_Waitms(1000); /* wait after power-on */ 05. ESP_Init(); 06. SHELL_Init(); 07. io = CLS1_GetStdio(); 08. CLS1_SendStr("\r\n------------------------------------------\r\n", io->stdOut); 09. CLS1_SendStr("ESP8266 with FRDM-KL25Z\r\n", io->stdOut); 10. CLS1_SendStr("------------------------------------------\r\n", io->stdOut); 11. CLS1_PrintPrompt(io); 12. for(;;) { 13. WebProcess(); 14. SHELL_Parse(); 15. WAIT1_Waitms(10); 16. LEDG_Neg(); 17. } 18.} With ESP server start I start the web server: Starting the Web Server It sends the AT+CIPMUX command followed by the AT+CIPSERVER to start the server, and then listens to the port. Reading and responding messages is done in WebProcess(): view source print? 01.staticvoidWebProcess(void) { 02. uint8_t res=ERR_OK; 03. bool isGet; 04. uint8_t ch_id=0; 05. uint16_t size=0; 06. constuint8_t *p; 07. constCLS1_StdIOType *io; 08. 09. if(ESP_IsServerOn()) { 10. io = CLS1_GetStdio(); 11. res = ESP_GetIPD(APP_EspMsgBuf, sizeof(APP_EspMsgBuf), &ch_id, &size, &isGet, 1000, io); 12. if(res==ERR_OK) { 13. if(isGet) { /* GET: put web page */ 14. res = SendWebPage(ch_id, LEDR_Get()!=FALSE, 21/*dummy temperature*/, io); 15. if(res!=ERR_OK && io!=NULL) { 16. CLS1_SendStr("Sending page failed!\r\n", io->stdErr); /* copy on console */ 17. } 18. } else{ /* POST: received info */ 19. intpos; 20. 21. pos = UTIL1_strFind(APP_EspMsgBuf, "radio="); 22. if(pos!=-1) { /* found */ 23. if(UTIL1_strncmp(&APP_EspMsgBuf[pos], "radio=0", sizeof("radio=0")-1)) { 24. LEDR_On(); 25. } elseif(UTIL1_strncmp(&APP_EspMsgBuf[pos], "radio=1", sizeof("radio=1")-1)) { 26. LEDR_Off(); 27. } 28. } 29. res = SendWebPage(ch_id, LEDR_Get()!=FALSE, 20/*dummy temperature*/, io); 30. if(res!=ERR_OK && io!=NULL) { 31. CLS1_SendStr("Sending page failed!\r\n", io->stdErr); /* copy on console */ 32. } 33. } 34. CLS1_SendStr("INFO: Closing connection...\r\n", io->stdOut); 35. res = ESP_CloseConnection(ch_id, io, ESP_DEFAULT_TIMEOUT_MS); 36. } 37. } else{ /* copy messages we receive to console */ 38. while(AS2_GetCharsInRxBuf()>0) { 39. uint8_t ch; 40. 41. (void)AS2_RecvChar(&ch); 42. CLS1_SendChar(ch); 43. } 44. } 45.} If the server is not enabled, it simply copies the received messages to the console: view source print? 1.} else{ /* copy messages we receive to console */ 2. while(AS2_GetCharsInRxBuf()>0) { 3. uint8_t ch; 4. 5. (void)AS2_RecvChar(&ch); 6. CLS1_SendChar(ch); 7. } 8. } Otherwise it scans for an IPD message (ESP_GetIPD()). This function returns the whole message, the channel, the message size and if it is a GET or POST message: 1 res = ESP_GetIPD(APP_EspMsgBuf, sizeof(APP_EspMsgBuf), &ch_id, &size, &isGet, 1000, io); If it is a GET message, then it sends a HTML page to the module: 1 res = SendWebPage(ch_id, LEDR_Get()!=FALSE, 21 /*dummy temperature*/, io); This web page shows the status of the red LED on the board, a (dummy) temperature value and a button to submit new LED values: WSP8266 Web Server The HTML code for this page is constructed in SendWebPage() and sent withAT+CIPSEND: view source print? 01.staticuint8_t SendWebPage(uint8_t ch_id, bool ledIsOn, uint8_t temperature, constCLS1_StdIOType *io) { 02. staticuint8_t http[1024]; 03. uint8_t cmd[24], rxBuf[48], expected[48]; 04. uint8_t buf[16]; 05. uint8_t res = ERR_OK; 06. 07. /* construct web page content */ 08. UTIL1_strcpy(http, sizeof(http), (uint8_t*)"HTTP/1.0 200 OK\r\nContent-Type: text/html\r\nPragma: no-cache\r\n\r\n"); 09. UTIL1_strcat(http, sizeof(http), (uint8_t*)"\r\n\r\n"); 10. UTIL1_strcat(http, sizeof(http), (uint8_t*)"\r\n"); 11. UTIL1_strcat(http, sizeof(http), (uint8_t*)"Web Server using ESP8266\r\n"); 12. UTIL1_strcat(http, sizeof(http), (uint8_t*)" 13.\r\n"); 14. UTIL1_strcat(http, sizeof(http), (uint8_t*)"Temp: OC"); 17. if(ledIsOn) { 18. UTIL1_strcat(http, sizeof(http), (uint8_t*)"Red LED off"); 19. UTIL1_strcat(http, sizeof(http), (uint8_t*)" 20.Red LED on"); 21. } else{ 22. UTIL1_strcat(http, sizeof(http), (uint8_t*)"Red LED off"); 23. UTIL1_strcat(http, sizeof(http), (uint8_t*)" 24.Red LED on"); 25. } 26. UTIL1_strcat(http, sizeof(http), (uint8_t*)""); 27. UTIL1_strcat(http, sizeof(http), (uint8_t*)"\r\n\r\n"); 28. 29. UTIL1_strcpy(cmd, sizeof(cmd), "AT+CIPSEND="); /* parameters are , */ 30. UTIL1_strcatNum8u(cmd, sizeof(cmd), ch_id); 31. UTIL1_chcat(cmd, sizeof(cmd), ','); 32. UTIL1_strcatNum16u(cmd, sizeof(cmd), UTIL1_strlen(http)); 33. UTIL1_strcpy(expected, sizeof(expected), cmd); /* we expect the echo of our command */ 34. UTIL1_strcat(expected, sizeof(expected), "\r\r\n> "); /* expect "> " */ 35. UTIL1_strcat(cmd, sizeof(cmd), "\r\n"); 36. res = ESP_SendATCommand(cmd, rxBuf, sizeof(rxBuf), expected, ESP_DEFAULT_TIMEOUT_MS, io); 37. if(res!=ERR_OK) { 38. if(io!=NULL) { 39. CLS1_SendStr("INFO: TIMEOUT, closing connection!\r\n", io->stdOut); 40. } 41. } else{ 42. if(io!=NULL) { 43. CLS1_SendStr("INFO: Sending http page...\r\n", io->stdOut); 44. } 45. UTIL1_strcat(http, sizeof(http), "\r\n\r\n"); /* need to add this to end the command! */ 46. res = ESP_SendATCommand(http, NULL, 0, NULL, ESP_DEFAULT_TIMEOUT_MS, io); 47. if(res!=ERR_OK) { 48. CLS1_SendStr("Sending page failed!\r\n", io->stdErr); /* copy on console */ 49. } else{ 50. for(;;) { /* breaks */ 51. res = ESP_ReadCharsUntil(buf, sizeof(buf), '\n', 1000); 52. if(res==ERR_OK) { /* line read */ 53. if(io!=NULL) { 54. CLS1_SendStr(buf, io->stdOut); /* copy on console */ 55. } 56. } 57. if(UTIL1_strncmp(buf, "SEND OK\r\n", sizeof("SEND OK\r\n")-1)==0) { /* ok from module */ 58. break; 59. } 60. } 61. } 62. } 63. returnres; 64.} In case of a POST message (user has pressed the button), I scan for the radio element string and turn on/off the LED accordingly, and re-submit the new web page: view source print? 01.} else{ /* POST: received info */ 02. intpos; 03. 04. pos = UTIL1_strFind(APP_EspMsgBuf, "radio="); 05. if(pos!=-1) { /* found */ 06. if(UTIL1_strncmp(&APP_EspMsgBuf[pos], "radio=0", sizeof("radio=0")-1)) { 07. LEDR_On(); 08. } elseif(UTIL1_strncmp(&APP_EspMsgBuf[pos], "radio=1", sizeof("radio=1")-1)) { 09. LEDR_Off(); 10. } 11. } 12. res = SendWebPage(ch_id, LEDR_Get()!=FALSE, 20/*dummy temperature*/, io); 13. if(res!=ERR_OK && io!=NULL) { 14. CLS1_SendStr("Sending page failed!\r\n", io->stdErr); /* copy on console */ 15. } 16. } Finally, it closes the connection at the end: view source print? 1.CLS1_SendStr("INFO: Closing connection...\r\n", io->stdOut); 2. res = ESP_CloseConnection(ch_id, io, ESP_DEFAULT_TIMEOUT_MS); With this, I handle GET and POST messages and can toggle the LED on my board :-) :-). Summary It is amazing what is possible with this tiny and inexpensive ($4.50) WiFi module. The simple AT interface allows small and tiny microprocesors to connect to the internet or the local network. With all the hype around ‘Internet of Things’ this is where things very likely will end up: small nodes connecting in an easy way to the network. The processor on that ESP8266 is probably more powerful than the KL25Z (the specs and data sheets of that ESP8266 are still evolving). Or it is possible to run that module in standalone mode too which is a very interesting approach too, see the links at the end of this article. But still having an UART way to connect to the network is very useful and powerful. Other modules costs multiple times more. I expect that many vendors will come up with similar integrated modules e.g. to combine an ARM processor with the WiFi radio, similar that ESP8266 module. For sure that ESP8266 has a head start and paved the way how WiFi connectivity should work. We all will see what the future brings. Until then, that ESP8266 module is something I can use in many projects :-). The sources and project files can be found on GitHub: https://github.com/ErichStyger/mcuoneclipse/tree/master/Examples/KDS/FRDM-KL25Z/FRDM-KL25Z_ESP8266 Happy Web-Serving :-) Useful Links: http://www.electrodragon.com/w/Wi07c http://scargill.wordpress.com/category/esp8266/ https://github.com/esp8266/esp8266-webserver http://www.cse.dmu.ac.uk/~sexton/ESP8266/ http://defcon-cc.dyndns.org/wiki/ESP8266#Update http://www.xess.com/blog/esp8266-resources/

Here’s something that stumped me for a while today. I’ve got the following Linq query in my repository (this is using the ORM from DevExpress, XPO, but the basic idea is the same) internal virtual IOrderedQueryable GetMyData(string keyVal) { return (from MyEntity ent in new XPQuery(Context) where ent.Key == keyVal orderby ent.SortCol select end); } The problem I was having was in mocking the return value from this method. One cannot create an interface so I could not create a list of items to return from the mocked method. I finally hit on this magic combination of linq queries that lets me return a set built by hand for the mock. var emptyLst = new List(); var lst = (from d in emptyLst select d).AsQueryable().OrderBy(x => x.Key ); _mockRepo.Setup(r => r.MyMockedEvent).Returns(lst); This seems to work like a charm

I’m a huge Vaadin fan and I’ve created a Github workshop I can demo at conferences. A common issue with such kind of workshops is that attendees have to prepare their workstations in advance… and there’s always a significant part of them that comes with not everything ready. At this point, two options are available to the speaker: either wait for each of the attendee to finish the preparation – too bad for the people who took the time at home to do that, or start anyway – and lose the not-ready part. Given the current buzz around Docker, I thought that could be a very good way to make the workshop preparation quicker – only one step, and hasslefree – no problem regarding the quirks of your operation system. The required steps I ask the attendees are the following: Install Git Install Java, Maven and Tomcat Clone the git repo Build the project (to prepare the Maven repository) Deploy the built webapp Start Tomcat These should directly be automated into Docker. As I wasted much time getting this to work, here’s the tale of my journey in achieving this (be warned, it’s quite long). If you’ve got similar use-cases, I hope it will be useful in you getting things done faster. Starting with Docker The first step was to get to know the basics about Docker. Fortunately, I had the chance to attend a Docker workshop by David Gageot at Duchess Swiss. This included both Docker installation and basics of Dockerfile. I assume readers have likewise a basic understanding of Docker. For those who don’t, I guess browsing the Docker’s official documentation is a nice idea: Installation Dockerfile reference Building my first Dockerfile The Docker image can be built with the following command ran into the directory of the Dockerfile: $ docker build -t vaadinworkshop . The first issues one can encounter when playing with Docker the first time, is to get the following error message: Get http:///var/run/docker.sock/v1.14/containers/json: dial unix /var/run/docker.sock: no such file or directory The reason is because one didn’t export the required environment variables displayed by the boot2docker information message. If you lost the exact data, no worry, just use the shellinit boot2docker parameter: $ boot2docker shellinit Writing /Users/i303869/.docker/boot2docker-vm/ca.pem: Writing /Users/i303869/.docker/boot2docker-vm/cert.pem: Writing /Users/i303869/.docker/boot2docker-vm/key.pem: export DOCKER_HOST=tcp://192.168.59.103:2376 export DOCKER_CERT_PATH=/Users/i303869/.docker/boot2docker-vm Copy-paste the export lines above will solve the issue. These can also be set in one’s .bashrc script as it seems these values seldom change. Next in line is the following error: Get http://192.168.59.103:2376/v1.14/containers/json: malformed HTTP response "x15x03x01x00x02x02" This error message seems to be because of a mismatch between versions of the client and the server. It seems it is because of a bug on Mac OSX when upgrading. For a long term solution, reinstall Docker from scratch; for a quick fix, use the --tls flag with the docker command. As it is quite cumbersome to type it everything, one can alias it: $ alias docker="docker --tls" My last mistake when building the image comes from building the Dockerfile from a not empty directory. Docker sends every file it finds in the directory of the Dockerfile to the Docker container for build: $ docker --tls build -t vaadinworkshop . Sending build context to Docker daemon Too many kB Fix: do not try this at home and start from a directory container the Dockerfile only. Starting from scratch Dockerfiles describe images – images are built as a layered list of instructions. Docker images are designed around single inheritance: one image has to be set a single parent. An image requiring no parent starts from scratch, but Docker provides 4 base official distributions: busybox, debian, ubuntu and centos (operating systems are generally a good start). Whatever you want to achieve, it is necessary to choose the right parent. Given the requirements I set for myself (Java, Maven, Tomcat and Git), I tried to find the right starting image. Many Dockerfiles are already available online on the Docker hub. The browsing app is quite good, but to be really honest, the search can really be improved. My intention was to use the image that matched the most of my requirements, then fill the gap. I could find no image providing Git, but I thought the dgageot/maven Dockerfile would be a nice starting point. The problem is that the base image is a busybox and provides no installer out-of-the-box (apt-get, yum, whatever). For this reason, David uses a lot of curl to get Java 8 and Maven in his Dockerfiles. I foolishly thought I could use a different flavor of busybox that provides the opkg installer. After a while, I accumulated many problems, resolving one heading to another. In the end, I finally decided to use the OS I was most comfortable with and to install everything myself: FROM ubuntu:utopic Scripting Java installation Installing git, maven and tomcat packages is very straightforward (if you don’t forget to use the non-interactive options) with RUN and apt-get: RUN apt-get update && \ apt-get install -y --force-yes git maven tomcat8 Java doesn’t fall into this nice pattern, as Oracle wants you to accept the license. Nice people did however publish it to a third-party repo. Steps are the following: Add the needed package repository Configure the system to automatically accept the license Configure the system to add un-certified packages Update the list of repositories At last, install the package Also add a package for Java 8 system configuration. RUN echo "deb http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" | tee -a /etc/apt/sources.list && \ echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections && \ apt-key adv --keyserver keyserver.ubuntu.com --recv-keys EEA14886 RUN apt-get update && \ apt-get install -y --force-yes oracle-java8-installer oracle-java8-set-default Building the sources Getting the workshop’s sources and building them is quite straightforward with the following instructions: RUN git clone https://github.com/nfrankel/vaadin7-workshop.git WORKDIR /vaadin7-workshop RUN mvn package The drawback of this approach is that Maven will start from a fresh repository, and thus download the Internet the first time it is launched. At first, I wanted to mount a volume from the host to the container to share the ~/.m2/repository folder to avoid this, but I noticed this could only be done at runtime through the -v option as the VOLUME instruction cannot point to a host directory. Starting the image The simplest command to start the created Docker image is the following: $ docker run -p 8080:8080 Do not forget the port forwarding from the container to the host, 8080 for the standard HTTP port. Also, note that it’s not necessary to run the container as a daemon (with the -d option). The added value of that is that the standard output of the CMD (see below) will be redirected to the host. When running as a daemon and wanting to check the logs, one has to execute bash in the container, which requires a sequence of cumbersome manipulations. Configuring and launching Tomcat Tomcat can be launched when starting the container by just adding the following instruction to the Dockerfile: CMD ["catalina.sh", "run"] However, trying to start the container at this point will result in the following error: Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.ClassLoaderFactory validateFile WARNING: Problem with directory [/usr/share/tomcat8/common/classes], exists: [false], isDirectory: [false], canRead: [false] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.ClassLoaderFactory validateFile WARNING: Problem with directory [/usr/share/tomcat8/common], exists: [false], isDirectory: [false], canRead: [false] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.ClassLoaderFactory validateFile WARNING: Problem with directory [/usr/share/tomcat8/server/classes], exists: [false], isDirectory: [false], canRead: [false] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.ClassLoaderFactory validateFile WARNING: Problem with directory [/usr/share/tomcat8/server], exists: [false], isDirectory: [false], canRead: [false] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.ClassLoaderFactory validateFile WARNING: Problem with directory [/usr/share/tomcat8/shared/classes], exists: [false], isDirectory: [false], canRead: [false] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.ClassLoaderFactory validateFile WARNING: Problem with directory [/usr/share/tomcat8/shared], exists: [false], isDirectory: [false], canRead: [false] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.Catalina initDirs SEVERE: Cannot find specified temporary folder at /usr/share/tomcat8/temp Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.Catalina load WARNING: Unable to load server configuration from [/usr/share/tomcat8/conf/server.xml] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.Catalina initDirs SEVERE: Cannot find specified temporary folder at /usr/share/tomcat8/temp Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.Catalina load WARNING: Unable to load server configuration from [/usr/share/tomcat8/conf/server.xml] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.Catalina start SEVERE: Cannot start server. Server instance is not configured. I have no idea why, but it seems Tomcat 8 on Ubuntu is not configured in any meaningful way. Everything is available but we need some symbolic links here and there as well as creating the temp directory. This translates into the following instruction in the Dockerfile: RUN ln -s /var/lib/tomcat8/common $CATALINA_HOME/common && \ ln -s /var/lib/tomcat8/server $CATALINA_HOME/server && \ ln -s /var/lib/tomcat8/shared $CATALINA_HOME/shared && \ ln -s /etc/tomcat8 $CATALINA_HOME/conf && \ mkdir $CATALINA_HOME/temp The final trick is to connect the exploded webapp folder created by Maven to Tomcat’s webapps folder, which it looks for deployments: RUN mkdir $CATALINA_HOME/webapps && \ ln -s /vaadin7-workshop/target/workshop-7.2-1.0-SNAPSHOT/ $CATALINA_HOME/webapps/vaadinworkshop At this point, the Holy Grail is not far away, you just have to browse the URL… if only we knew what the IP was. Since running on Mac, there’s an additional VM beside the host and the container that’s involved. To get this IP, type: $ boot2docker ip The VM's Host only interface IP address is: 192.168.59.103 Now, browsing http://192.168.59.103:8080/vaadinworkshop/ will bring us to the familiar workshop screen: Developing from there Everything works fine but didn’t we just forget about one important thing, like how workshop attendees are supposed to work on the sources? Easy enough, just mount the volume when starting the container: docker run -v /Users//vaadin7-workshop:/vaadin7-workshop -p 8080:8080 vaadinworkshop Note that the host volume must be part of /Users and if on OSX, it must use boot2docker v. 1.3+. Unfortunately, it seems now is the showstopper, as mounting an empty directory from the host to the container will not make the container’s directory available from the host. On the contrary, it will empty the container’s directory given that the host’s directory doesn’t exist… It seems there’s an issue in Docker on Mac. The installation of JHipster runs into the same problem, and proposes to use the Samba Docker folder sharing project. I’m afraid I was too lazy to go further at this point. However, this taught me much about Docker, its usages and use-cases (as well as OSX integration limitations). For those who are interested, you’ll find below the Docker file. Happy Docker! FROM ubuntu:utopic MAINTAINER Nicolas Frankel # Config to get to install Java 8 w/o interaction RUN echo "deb http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" | tee -a /etc/apt/sources.list && echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections && apt-key adv --keyserver keyserver.ubuntu.com --recv-keys EEA14886 RUN apt-get update && apt-get install -y --force-yes git oracle-java8-installer oracle-java8-set-default maven tomcat8 RUN git clone https://github.com/nfrankel/vaadin7-workshop.git WORKDIR /vaadin7-workshop RUN git checkout v7.2-1 RUN mvn package ENV JAVA_HOME /usr/lib/jvm/java-8-oracle ENV CATALINA_HOME /usr/share/tomcat8 ENV PATH $PATH:$CATALINA_HOME/bin # Configure Tomcat 8 directories RUN ln -s /var/lib/tomcat8/common $CATALINA_HOME/common && ln -s /var/lib/tomcat8/server $CATALINA_HOME/server && ln -s /var/lib/tomcat8/shared $CATALINA_HOME/shared && ln -s /etc/tomcat8 $CATALINA_HOME/conf && mkdir $CATALINA_HOME/temp && mkdir $CATALINA_HOME/webapps && ln -s /vaadin7-workshop/target/workshop-7.2-1.0-SNAPSHOT/ $CATALINA_HOME/webapps/vaadinworkshop VOLUME ["/vaadin7-workshop"] CMD ["catalina.sh", "run"] # docker build -t vaadinworkshop . # docker run -v ~/vaadin7-workshop training/webapp -p 8080:8080 vaadinworkshop

MongoDB provides a lot of query selectors for filtering documents from a collection. Writing complex queries for MongoDB in Java can be tricky sometimes. Consider below data present in student_marks collection {"sid" : 1,"fname" : "Tom","lname" : "Ford","marks" : [ {"english" : 48}, {"maths" : 49}, {"science" : 50}]} {"sid" : 2,"fname" : "Tim","lname" : "Walker","marks" : [ {"english" : 35}, {"maths" : 42}, {"science" : 37}]} {"sid" : 3,"fname" : "John","lname" : "Ward","marks" : [ {"english" : 45}, {"maths" : 41}, {"science" : 37}]} If we want to get students whose last name is Ford and have obtained more than 35 marks in english then the MongoDB shell command for this will be - db.student_marks.find({$and:[{"lname":"Ford"},{"marks.english": {$gt:35}]}) The same query written in Java will look something like this - DBObject query = new BasicDBObject(); List andQuery = new ArrayList(); andQuery.add(new BasicDBObject("lname", "Ford")); andQuery.add(new BasicDBObject("marks.english", new BasicDBObject("$gt", 35))); query.put("$and", andQuery); Using MongoDB QueryBuilder we can rewrite above query as - DBObject query = new QueryBuilder() .start() .and(new QueryBuilder().start().put("lname").is("Ford").get(), new QueryBuilder().start().put("marks.english") .greaterThan(35).get()).get(); You can see that by using QueryBuilder we can write complex queries with ease. QueryBuilder class provides many methods like and, not, greaterThan, exists, etc. which helps in writing MongoDB queries more efficiently and less prone to error/mistakes. If you enjoyed this article and want to learn more about MongoDB, check out this collection of tutorials and articles on all things MongoDB.

Compared to other features, real-time search capability is undoubtedly one of the most important features in Elasticsearch. Today we’ll look closely how is provided real-time search by Elasticsearch. Real time First of all, if we need to explain the concept of real-time, in general, we can say that the delay between input and out time in the information is small at real-time systems. This means, data is taken without data accumulation, processed in real time. Today, the best solution Elasticsearch known for real-time search, when a record is added to it for storage makes it searchable in 1 second. How? As is known, the disks are able to create a risk of bottleneck for I/O operations at the data persistence step. Also some mechanisms used for prevent any loss of data increases cost of time. At this point Elasticsearch uses the file-system cache that sitting between itself and the disk for overcome the risk of bottleneck and ensure the a new document can be searched in real time. A new segment is written to the file-system cache first and only later it flushed to disk by Elasticsearch. This lightweight process of writing and opening a new segment is called a refresh in Elasticsearch. By default, all shards is refreshed automatically once every second. In this way, Elasticsearch support real-time search. Test time Above digression about the time of refresh of the shards you can bring to mind the following questions: What happens, when a new document is requested in less than 1 second time? Can be documents requested, without having to depend of the refresh period shards of managed by Elasticsearch? Short answers. Elasticsearch does not return the document. Yes. Now let’s get clarity on this issue is a simple example. hakdogan$ curl -XPUT localhost:9200/kodcucom/document/1 -d'{ > "title": "Document A" > }' We sent a document to Elasticsearch. The index name is kodcucom, type document, id value 1. The title field is only field in the document and the value of "Document A". Let’s take this document from Elasticsearch. hakdogan$ curl -XGET localhost:9200/kodcucom/document/1?pretty { "_index" : "kodcucom", "_type" : "document", "_id" : "1", "_version" : 1, "found" : true, "_source":{ "title": "Document A" } } As expected, the document was returned to us. Well, if we keep short the time between document recording and get request than default shard refresh time what will happen? Let’s see. hakdogan$ curl -XPUT localhost:9200/kodcucom/document/2 -d'{"title": "Document B"}'; curl -XGET localhost:9200/kodcucom/_search?pretty {"_index":"kodcucom","_type":"document","_id":"2","_version":1,"created":true}{ "took" : 38, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 1, "max_score" : 1.0, "hits" : [ { "_index" : "kodcucom", "_type" : "document", "_id" : "1", "_score" : 1.0, "_source":{ "title": "Document A" } } ] } } As can be seen, only the previous document was returned to us by Elasticsearch when we do concurrently create and get request. Well, how can I get the document concurrently? Let’s see. hakdogan$ curl -XPUT localhost:9200/kodcucom/document/3 -d'{"title": "Document C"}'; curl -XGET localhost:9200/kodcucom/_refresh; curl -XGET localhost:9200/kodcucom/_search?pretty {"_index":"kodcucom","_type":"document","_id":"3","_version":1,"created":true}{"_shards":{"total":10,"successful":5,"failed":0}{ "took" : 3, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 3, "max_score" : 1.0, "hits" : [ { "_index" : "kodcucom", "_type" : "document", "_id" : "1", "_score" : 1.0, "_source":{ "title": "Document A" } }, { "_index" : "kodcucom", "_type" : "document", "_id" : "2", "_score" : 1.0, "_source":{"title": "Document B"} }, { "_index" : "kodcucom", "_type" : "document", "_id" : "3", "_score" : 1.0, "_source":{"title": "Document C"} } ] } } In this command, we perform to refresh operation on kodcucom index before the search request. In this way, the document was returned to us. Auto refresh time can be changed. By setting the index.refresh_interval parameter in the configuration file. Applies to all indices in the cluster. A per-index basis by updated index setting. In addition to these, you can turn off automatic refresh. An important point to keep in mind about the refresh time of the shards, the refresh operation is costly in terms of system resources. If you wished to make changes to the auto-refresh time, this situation should be taken into account. Extension of the automatic refresh time, enables faster indexing but new documents and changes made to the existing documents will not appear in searches during specified period of time.

Nowadays it has become mandatory to Gzipping the APIs response due to huge amount of data we are sending in response. It saves network bandwidth and delivery time and off course space over the internet. While using CXF; it provides an option to use the Gzip Compression in no of ways. Blueprint Annotation Blueprint: Annotation: First you need to register the GZIPOutInterceptor in out interceptors list. For that you need to hook into CXF initialization classes. public class InterceptorManager extends AbstractFeature { private static final Logger LOGGER = Logger.getLogger( "simcore" ); private static final Interceptor< Message > GZIP = new GZIPOutInterceptor(); //private static final Interceptor< Message > GZIP = new GZIPOutInterceptor(512); /* (non-Javadoc) * @see org.apache.cxf.feature.AbstractFeature#initializeProvider(org.apache.cxf.interceptor.InterceptorProvider, org.apache.cxf.Bus) */ @Override protected void initializeProvider( InterceptorProvider provider, Bus bus ) { /** * Adding Gzip interceptor to all outbound requests/responses */ LOGGER.debug( " ############## Adding Gzip as OUT Interceptor ##############" ); provider.getOutInterceptors().add( GZIP ); } } GZIPOutInterceptor comes with an option to set the Threshold value as no of Bytes. If response size will be below this threshold value then it will not be compressed. It is extremely useful when we will be sending empty lists and status messages/codes only. Because compressing those small responses will be overhead at server side. But there is another factor which is no of users requesting the response. So set this value by thinking over all the cases in mind. @GZIP Now we can use this annotation on any of our web-services controller to implement compression on all the APIs provided in that class. @WebService @Consumes ( { MediaType.TEXT_PLAIN, MediaType.APPLICATION_XML, MediaType.APPLICATION_JSON } ) @Produces ( MediaType.APPLICATION_JSON ) @GZIP public interface WebServicesController { @GET @Path ( "/myGzipData" ) @Produces ( { MediaType.APPLICATION_JSON } ) Response getZipData( ); } Moreover we can set different parameters in Gzip annotation. @GZIP ( force = true, threshold = 512 )