DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

The Latest Languages Topics

article thumbnail
Stronger Anti Cross-Site Scripting (XSS) Filter for Java Web Apps
Here is a good and simple anti cross-site scripting (XSS) filter written for Java web applications. What it basically does is remove all suspicious strings from request parameters before returning them to the application. It’s an improvement over my previous post on the topic. You should configure it as the first filter in your chain (web.xml) and it’s generally a good idea to let it catch every request made to your site. The actual implementation consists of two classes, the actual filter is quite simple, it wraps the HTTP request object in a specialized HttpServletRequestWrapper that will perform our filtering. public class XSSFilter implements Filter { @Override public void init(FilterConfig filterConfig) throws ServletException { } @Override public void destroy() { } @Override public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException { chain.doFilter(new XSSRequestWrapper((HttpServletRequest) request), response); } } The wrapper overrides the getParameterValues(), getParameter() and getHeader() methods to execute the filtering before returning the desired field to the caller. The actual XSS checking and striping is performed in the stripXSS() private method. import java.util.regex.Pattern; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletRequestWrapper; public class XSSRequestWrapper extends HttpServletRequestWrapper { private static Pattern[] patterns = new Pattern[]{ // Script fragments Pattern.compile("", Pattern.CASE_INSENSITIVE), // src='...' Pattern.compile("src[\r\n]*=[\r\n]*\\\'(.*?)\\\'", Pattern.CASE_INSENSITIVE | Pattern.MULTILINE | Pattern.DOTALL), Pattern.compile("src[\r\n]*=[\r\n]*\\\"(.*?)\\\"", Pattern.CASE_INSENSITIVE | Pattern.MULTILINE | Pattern.DOTALL), // lonely script tags Pattern.compile("", Pattern.CASE_INSENSITIVE), Pattern.compile("", Pattern.CASE_INSENSITIVE | Pattern.MULTILINE | Pattern.DOTALL), // eval(...) Pattern.compile("eval\\((.*?)\\)", Pattern.CASE_INSENSITIVE | Pattern.MULTILINE | Pattern.DOTALL), // expression(...) Pattern.compile("expression\\((.*?)\\)", Pattern.CASE_INSENSITIVE | Pattern.MULTILINE | Pattern.DOTALL), // javascript:... Pattern.compile("javascript:", Pattern.CASE_INSENSITIVE), // vbscript:... Pattern.compile("vbscript:", Pattern.CASE_INSENSITIVE), // onload(...)=... Pattern.compile("onload(.*?)=", Pattern.CASE_INSENSITIVE | Pattern.MULTILINE | Pattern.DOTALL) }; public XSSRequestWrapper(HttpServletRequest servletRequest) { super(servletRequest); } @Override public String[] getParameterValues(String parameter) { String[] values = super.getParameterValues(parameter); if (values == null) { return null; } int count = values.length; String[] encodedValues = new String[count]; for (int i = 0; i < count; i++) { encodedValues[i] = stripXSS(values[i]); } return encodedValues; } @Override public String getParameter(String parameter) { String value = super.getParameter(parameter); return stripXSS(value); } @Override public String getHeader(String name) { String value = super.getHeader(name); return stripXSS(value); } private String stripXSS(String value) { if (value != null) { // NOTE: It's highly recommended to use the ESAPI library and uncomment the following line to // avoid encoded attacks. // value = ESAPI.encoder().canonicalize(value); // Avoid null characters value = value.replaceAll("\0", ""); // Remove all sections that match a pattern for (Pattern scriptPattern : patterns){ value = scriptPattern.matcher(value).replaceAll(""); } } return value; } } Notice the comment about the ESAPI library, I strongly recommend you check it out and try to include it in your projects. If you want to dig deeper on the topic I suggest you check out the OWASP page about XSS and RSnake’s XSS (Cross Site Scripting) Cheat Sheet.
March 31, 2012
by Ricardo Zuasti
· 285,003 Views · 7 Likes
article thumbnail
Converting a Value to String in JavaScript
In JavaScript, there are three main ways in which any value can be converted to a string. This blog post explains each way, along with its advantages and disadvantages. Three approaches for converting to string The three approaches for converting to string are: value.toString() "" + value String(value) The problem with approach #1 is that it doesn’t work if the value is null or undefined. That leaves us with approaches #2 and #3, which are basically equivalent. ""+value: The plus operator is fine for converting a value when it is surrounded by non-empty strings. As a way for converting a value to string, I find it less descriptive of one’s intentions. But that is a matter of taste, some people prefer this approach to String(value). String(value): This approach is nicely explicit: Apply the function String() to value. The only problem is that this function call will confuse some people, especially those coming from Java, because String is also a constructor. However, function and constructor produce completely different results: > String("abc") === new String("abc") false > typeof String("abc") 'string' > String("abc") instanceof String false > typeof new String("abc") 'object' > new String("abc") instanceof String true The function produces, as promised, a string (a primitive [1]). The constructor produces an instance of the type String (an object). The latter is hardly ever useful in JavaScript, which is why you can usually forget about String as a constructor and concentrate on its role as converting to string. A minor difference between ""+value and String(value) Until now you have heard that + and String() convert their “argument” to string. But how do they actually do that? It turns out that they do it in slightly different ways, but usually arrive at the same result. Converting primitives to string Both approaches use the internal ToString() operation to convert primitives to string. “Internal” means: a function specified by the ECMAScript 5.1 (§9.8) that isn’t accessible to the language itself. The following table explains how ToString() operates on primitives. Argument Result undefined "undefined" null "null" boolean value either "true" or "false" number value the number as a string, e.g. "1.765" string value no conversion necessary Converting objects to string Both approaches first convert an object to a primitive, before converting that primitive to string. However, + uses the internal ToNumber() operator (except for dates [2]), while String() uses ToString(). ToNumber(): To convert an object obj to a primitive, invoke obj.valueOf(). If the result is primitive, return that result. Otherwise, invoke obj.toString(). If the result is primitive, return that result. Otherwise, throw a TypeError. ToString(): Works the same, but invokes obj.toString() before obj.valueOf(). With the following object, you can observe the difference: var obj = { valueOf: function () { console.log("valueOf"); return {}; // not a primitive, keep going }, toString: function () { console.log("toString"); return {}; // not a primitive, keep going } }; Interaction: > "" + obj valueOf toString TypeError: Cannot convert object to primitive value > String(obj) toString valueOf TypeError: Cannot convert object to primitive value Most objects use the default implementation of valueOf() which returns this for objects. Hence, that method will always be skipped by ToNumber(). > var x = {} > x.valueOf() === x true Instances of Boolean, Number, and String wrap primitives and valueOf returns the wrapped primitive. But that still means that the final result will be the same as for toString(), even though it will have been produced in a different manner. > var n = new Number(756) > n.valueOf() === n false > n.valueOf() === 756 true Conclusion Which of the three approaches for converting to string should you choose? value.toString() can be OK, if you are sure that value will never be null or undefined. Otherwise, ""+value and String(value) are mostly equivalent. Which one people prefer is a matter of taste. I find String(value) more explicit. Related posts JavaScript values: not everything is an object [primitives versus objects] What is {} + {} in JavaScript? [explains how the + operator works] String concatenation in JavaScript [how to best concatenate many strings]
March 30, 2012
by Axel Rauschmayer
· 31,924 Views · 2 Likes
article thumbnail
You've Been Implementing main() Wrong All This Time
Since the very early days of Java (and C-like languages overall), the canonical way to start your program has been something like this: public class A { public static void main(String[] args) { new A().run(args); } public void run(String[] args) { // Your application starts here } } If you are still doing this, I’m here to tell you it’s time to stop. Letting go of ‘new’ First, install Guice in your project: com.google.inject guice 3.0 and then, modify your main method as follows: public class A { public static void main(String[] args) { Injector.getInstance(A.class).run(args); } } So, what does this buy you exactly? You will find a lot of articles explaining the various benefits of Guice, such as being able to substitute different environments on the fly, but I’m going to use a different angle in this article. Let’s start by assuming the existence of a Config class that contains various configuration parameters. I’ll just hardcode them for now and use fields to make the class smaller: public class Config { String host = "com.example.com"; int port = 1234; } This class is a singleton, it is instantiated somewhere in your main class and not used anywhere else at the moment. One day, you realize you need this instance in another class which happens to be deep in your runtime hierarchy, which we will call Deep. For example, if you put a break point in the method where you need this config object, your debugger would show you stack frames similar to this: com.example.A.main() com.example.B.f(int, String) com.example.C.g(String) com.example.Deep.h(Foo, int) The easy and wrong way to solve this problem is to make the Config instance static on some class (probably A) and access it directly from Deep. I’m hoping I don’t need to explain why this is a bad idea: not only do you want to avoid using statics, but you also want to make sure that each object is exposed only to objects that need them, and making the Config object static would make your instance visible to your entire code base. Not a good thing. The second thought is to pass the object down the stack, so you modify all the signatures as follows: com.example.A.main() com.example.B.f(int, String, Config) com.example.C.g(String, Config) com.example.Deep.h(Foo, int, Config) This is a bit better since you have severely restricted the exposure of the Config object, but note that you are still making it available to more methods than really need to: B#f and C#g have really nothing to do with this object, and a little sting of discomfort hits you when you start writing the Javadoc: public class C { ... /** * @param config This method doesn't really use this parameter, * it just passes it down so Deep#h can use it. */ public void g(String s, Config config) { Unnecessary exposure is actually not the worst part of this approach, the problem is that it changes all these signatures along the way, which is certainly undesirable in a private API and absolutely devastating in a public API. And of course, it’s absolutely not scalable: if you keep adding a parameter to your method whenever you need access to a certain object, you will soon be dealing with methods that take ten parameters, most of which they just pass down the chain. Here is how we solve this problem with dependency injection (performed by Guice in this example, but this is applicable to any library that implements JSR 330, obviously): public class Deep { @Inject private Config config; and we’re done. That’s it. You don’t need to modify the Config class in any way, nor do you need to make any change in any of the classes that separate Deep from your main class. With this, you have also minimized the exposure of the Config object to just the class that needs it. Injecting right There are various ways you can inject object into your class but I’ll just mention the two that, I think, are the most important. I just showed “field injection” in the previous paragraph, but be aware that you can also prefer to use “constructor injection”: public class Deep { private final Config config; @Inject public Deep(Config config) { this.config = config; } This time, you are adding a parameter to the constructor of your Deep class (which shouldn’t worry you too much since you will never invoke it directly, Guice will) and you assign the parameter to the field in the constructor. The benefit is that you can declare your field final. The downside, obviously, is that this approach is much more verbose. Personally, I see little point in final fields since I have hardly ever encountered a bug that was due to accidentally reassigning a field, so I tend to use field injection whenever I can. Taking it to the next level Obviously, the kind of configuration object I used as an example if not very realistic. Typically, a configuration will not hardcode values like I did and will, instead, read them from some external source. Similarly, you will want to inject objects that can’t necessarily be instantiated so early in the lifecycle of your application, such as servlet contexts, database connections, or implementations of your own interfaces. This topic itself would probably cover several chapters of a book dedicated to dependency injection, so I’ll just summarize it: not all objects can be injected this way, and one benefit of using a dependency injection framework in your code is that it will force you to think about what life cycle category your objects belong to. Having said that, if you want to find out how Guice can inject objects that get created at a later time in your application life cycle, look up the Javadoc for the Provider class. Wrapping up I hope this quick introduction to dependency injection piqued your interest and that you will consider using it in your project since it has so much more to offer than what I described in this post. If you want to learn more, I suggest starting with the excellent Guice documentation.
March 26, 2012
by Cedric Beust
· 13,444 Views
article thumbnail
Regular Expression In Python For E-Mail And Phone Number
// code contains regular expression for contact number and email address in python str='[email protected]' match=re.search(r'\w+@\w+',str) #return [email protected] num=555-555-555 match_num=re.search(r'^(\d{3}--\d{3}--\d{4})$',num) #return 555-555-5555
March 25, 2012
by Vinil Mehta
· 31,157 Views
article thumbnail
CSS3 Animated Gears
in today’s lesson, we have made animated gears with css3. the result looks very nice. i have used css3 keyframes, animation and transforms (rotate) in order to achieve this result. please pay attention – the current demo works well only in firefox and chrome / safari (webkit). here are the samples and downloadable package: live demo download in package ok, download the example files and let's start coding ! step 1. html as usual, we start with the html. there are easy div elements. index.html step 2. css here are the css styles of our animated gears. css/layout.css /* css3 keyframes */ @-webkit-keyframes ckw { 0% { -moz-transform: rotate(0deg); -webkit-transform: rotate(0deg); } 100% { -moz-transform: rotate(360deg); -webkit-transform: rotate(360deg); } } @-moz-keyframes ckw { 0% { -moz-transform: rotate(0deg); -webkit-transform: rotate(0deg); } 100% { -moz-transform: rotate(360deg); -webkit-transform: rotate(360deg); } } @-webkit-keyframes cckw { 0% { -moz-transform: rotate(360deg); -webkit-transform: rotate(360deg); } 100% { -moz-transform: rotate(0deg); -webkit-transform: rotate(0deg); } } @-moz-keyframes cckw { 0% { -moz-transform: rotate(360deg); -webkit-transform: rotate(360deg); } 100% { -moz-transform: rotate(0deg); -webkit-transform: rotate(0deg); } } /* gears */ .gear { float: none; position: absolute; text-align: center; -moz-animation-timing-function: linear; -moz-animation-iteration-count: infinite; -moz-animation-direction: normal; -moz-animation-delay: 0; -moz-animation-play-state: running; -moz-animation-fill-mode: forwards; -webkit-animation-timing-function: linear; -webkit-animation-iteration-count: infinite; -webkit-animation-direction: normal; -webkit-animation-delay: 0; -webkit-animation-play-state: running; -webkit-animation-fill-mode: forwards; } #gear1 { background: url('../images/g1.png') no-repeat 0 0; height: 85px; left: 31px; top: 45px; width: 85px; -moz-animation-name: ckw; -moz-animation-duration: 10s; -webkit-animation-name: ckw; -webkit-animation-duration: 10s; } #gear2 { background: url('../images/g2.png') no-repeat 0 0; height: 125px; left: 105px; top: 10px; width: 125px; -moz-animation-name: cckw; -moz-animation-duration: 16.84s; -webkit-animation-name: cckw; -webkit-animation-duration: 16.84s; } #gear3 { background: url('../images/g3.png') no-repeat 0 0; height: 103px; left: 149px; top: 118px; width: 103px; -moz-animation-name: ckw; -moz-animation-duration: 13.5s; -webkit-animation-name: ckw; -webkit-animation-duration: 13.5s; } #gear4 { background: url('../images/g4.png') no-repeat 0 0; height: 144px; left: 46px; top: 173px; width: 144px; -moz-animation-name: cckw; -moz-animation-duration: 20.2s; -webkit-animation-name: cckw; -webkit-animation-duration: 20.2s; } #gear5 { background: url('../images/g1.png') no-repeat 0 0; height: 85px; left: 127px; top: 292px; width: 85px; -moz-animation-name: ckw; -moz-animation-duration: 10s; -webkit-animation-name: ckw; -webkit-animation-duration: 10s; } #gear6 { background: url('../images/g2.png') no-repeat 0 0; height: 125px; left: 200px; top: 283px; width: 125px; -moz-animation-name: cckw; -moz-animation-duration: 16.84s; -webkit-animation-name: cckw; -webkit-animation-duration: 16.84s; } #gear7 { background: url('../images/g3.png') no-repeat 0 0; height: 103px; left: 277px; top: 217px; width: 103px; -moz-animation-name: ckw; -moz-animation-duration: 13.5s; -webkit-animation-name: ckw; -webkit-animation-duration: 13.5s; } step 3. images i have used these images: live demo download in package conclusion hope you enjoyed the new tutorial, don’t forget to give thanks and leave a comment good luck!
March 25, 2012
by Andrei Prikaznov
· 23,888 Views
article thumbnail
HTML5 Image Effects – Emboss
Today we continue our HTML5 canvas image filter examples. Today I would like to share with you a method of applying a Emboss effect to images. This is a pretty difficult method, but I am sure that you can repeat it. In our demo we can play with different images by adding an emboss effect to them, and we can ‘export’ our result on the image element (). Here are our demo and downloadable packages: Live Demo download in package Ok, download the example files and let's start coding ! Step 1. HTML Markup This is the markup of our demo page: index.html HTML5 Image Effects - Emboss Back to original tutorial on Script Tutorials Canvas Object Image Object Next imageApply Emboss EffectTo Image Basically – it contains just one canvas object, one image, and three ‘buttons’ (div elements). Step 2. CSS Here are our stylesheets (not so important, but anyway): css/main.css *{ margin:0; padding:0; } body { background-image:url(../images/bg.png); color:#fff; font:14px/1.3 Arial,sans-serif; } header { background-color:#212121; box-shadow: 0 -1px 2px #111111; display:block; height:70px; position:relative; width:100%; z-index:100; } header h2{ font-size:22px; font-weight:normal; left:50%; margin-left:-400px; padding:22px 0; position:absolute; width:540px; } header a.stuts,a.stuts:visited { border:none; text-decoration:none; color:#fcfcfc; font-size:14px; left:50%; line-height:31px; margin:23px 0 0 110px; position:absolute; top:0; } header .stuts span { font-size:22px; font-weight:bold; margin-left:5px; } .container { color: #000000; margin: 20px auto; overflow: hidden; position: relative; width: 1005px; } table { background-color: rgba(255, 255, 255, 0.7); } table td { border: 1px inset #888888; position: relative; text-align: center; } table td p { display: block; padding: 10px 0; } .button { cursor: pointer; height: 20px; padding: 15px 0; position: relative; text-align: center; width: 500px; -moz-user-select: none; -khtml-user-select: none; user-select: none; } Step 3. JS Finally – our JavaScript code for the Emboss effect: js/script.js // variables var canvas, ctx; var imgObj; // filter strength var strength = 0.5; // shifting matrix var matrix = [-2, -1, 0, -1, 1, 1, 0, 1, 2]; // normalize matrix function normalizeMatrix(m) { var j = 0; for (var i = 0; i < m.length; i++) { j += m[i]; } for (var i = 0; i < m.length; i++) { m[i] /= j; } return m; } // convert x-y coordinates into pixel position function convertCoordinates(x, y, w) { return x + (y * w); } // find a specified distance between two colours function findColorDiff(dif, dest, src) { return dif * dest + (1 - dif) * src; } // transform matrix function transformMatrix(img, pixels) { // create a second canvas and context to keep temp results var canvas2 = document.createElement('canvas'); var ctx2 = canvas2.getContext('2d'); ctx2.width = canvas2.width = img.width; ctx2.height = canvas2.height = img.height; // draw image ctx2.drawImage(img, 0, 0, img.width , img.height); var buffImageData = ctx2.getImageData(0, 0, canvas.width, canvas.height); var data = pixels.data; var bufferedData = buffImageData.data; // normalize matrix matrix = normalizeMatrix(matrix); var mSize = Math.sqrt(matrix.length); for (var i = 1; i < img.width - 1; i++) { for (var j = 1; j < img.height - 1; j++) { var sumR = sumG = sumB = 0; // loop through the matrix for (var h = 0; h < mSize; h++) { for (var w = 0; w < mSize; w++) { var r = convertCoordinates(i + h - 1, j + w - 1, img.width) << 2; // RGB for current pixel var currentPixel = { r: bufferedData[r], g: bufferedData[r + 1], b: bufferedData[r + 2] }; sumR += currentPixel.r * matrix[w + h * mSize]; sumG += currentPixel.g * matrix[w + h * mSize]; sumB += currentPixel.b * matrix[w + h * mSize]; } } var rf = convertCoordinates(i, j, img.width) << 2; data[rf] = findColorDiff(strength, sumR, data[rf]); data[rf + 1] = findColorDiff(strength, sumG, data[rf + 1]); data[rf + 2] = findColorDiff(strength, sumB, data[rf + 2]); } } return pixels; } // process emboss function function processEmboss() { // clear context ctx.clearRect(0, 0, canvas.width, canvas.height); // draw image ctx.drawImage(imgObj, 0, 0, imgObj.width , imgObj.height); // get image data var imageData = ctx.getImageData(0, 0, canvas.width, canvas.height); // transform image data imageData = transformMatrix(imgObj, imageData); // draw data back ctx.putImageData(imageData, 0, 0); }; $(function () { // create canvas and context objects canvas = document.getElementById('source'); ctx = canvas.getContext('2d'); // load source image imgObj = new Image(); imgObj.onload = function () { // draw image ctx.drawImage(this, 0, 0, this.width, this.height, 0, 0, canvas.width, canvas.height); } imgObj.src="images/pic1.jpg"; // different onclick handlers var iCur = 1; $('#next').click(function () { iCur++; if (iCur > 6) iCur = 1; imgObj.src="images/pic" + iCur + '.jpg'; }); $('#emboss').click(function () { processEmboss(); }); $('#toImage').click(function () { $('#img').attr('src', canvas.toDataURL('image/jpeg')); }); }); This effect requires some pretty difficult matrix transformations, so you can try to understand it, or use it as-is. Of course, our script will pass through all the pixels of the original image, and then will apply some transformations. Live Demo download in package Conclusion I hope that our demo looks fine. Today we have added a new interesting effect to our html5 application – Emboss. I will be glad to see your thanks and comments. Good luck!
March 20, 2012
by Andrei Prikaznov
· 13,181 Views
article thumbnail
PHP objects in MongoDB with Doctrine
An is equivalent to an Object-Relational Mapper, but with its targets are documents of a NoSQL database instead of table rows. No one said that a Data Mapper must always rely on a relational database as its back end. In the PHP world, probably the Doctrine ODM for MongoDB is the most successful. This followes to the opularity of Mongo, which is a transitional product between SQL and NoSQL, still based on some relational concepts like queries. Lots of features The Doctrine Mongo ODM supports mapping of objects via annotations placed in the class source code, or via external XML or YAML files. In this and in many aspects it is based on the same concepts as the Doctrine ORM: it features a Facade DocumentManager object and a Unit Of Work that batches changes to the database when objects are added to it. Moreover, two different types of relationships between objects are supported: references and embedded documents. The first is the equivalent of the classical pointer to another row which ORM always transform object references into; the second actually stores an object inside another one, like you would do with a Value Object. Thus, at least in Doctrine's case, it is easier to map objects as documents that as rows. As said before, the ODM borrows some concepts and classes from the ORM, in particular from the Doctrine\Common package which features a standard collection class. So if you have built objects mapped with the Doctrine ORM nothing changes for persisting them in MongoDB, except for the mapping metadata itself. Advantages If an ORM is sometimes a leaky abstraction, an ODM probably becomes an issue less often. It has less overhead than an ORM, since there is no schema to define and the ability to embed objects means there should be no compromises between the object model and the capabilities of the database. How many times we have renounced introducing a potential Value Object because of the difficulty in persisting it? The case for an ODM over a plain Mongo connection object is easy to make: you will still be able to use objects with proper encapsulation (like private fields and associations) and behavior (many methods) instead of extracting just a JSON package from your database. Installation A prerequisite for the ODM is the presence of the mongo extension, that can be installed via pecl. After having verified the extension is present, grab the Doctrine\Common as the 2.2.x package, and a zip of the doctrine-mongodb and doctrine-mongodb-odm projects from Github. Decompress everything into a Doctrine/ folder. After having setup autoloading for classes in Doctrine\, use this bootstrap to get a DocumentManager (the equivalent of EntityManager): use Doctrine\Common\Annotations\AnnotationReader, Doctrine\ODM\MongoDB\DocumentManager, Doctrine\MongoDB\Connection, Doctrine\ODM\MongoDB\Configuration, Doctrine\ODM\MongoDB\Mapping\Driver\AnnotationDriver; private function getADm() { $config = new Configuration(); $config->setProxyDir(__DIR__ . '/mongocache'); $config->setProxyNamespace('MongoProxies'); $config->setDefaultDB('test'); $config->setHydratorDir(__DIR__ . '/mongocache'); $config->setHydratorNamespace('MongoHydrators'); $reader = new AnnotationReader(); $config->setMetadataDriverImpl(new AnnotationDriver($reader, __DIR__ . '/Documents')); return DocumentManager::create(new Connection(), $config); } You will be able to call persist() and flush() on the DocumentManager, along with a set of other methods for querying like find() and getRepository(). Integration with an ORM We are researching a solution for versioning objects mapped with the Doctrine ORM. Doing this with a version column would be invasive, and also strange where multiple objects are involved (do you version just the root of an object graph? Duplicate the other ones when they change? How can you detect that?) The idea is taking a snapshot and putting it in a read only MongoDB instance, where all previous versions can be retrieved later for auditing (business reasons). This has been verified to be technically possible: the DocumentManager and EntityManager are totally separate object graphs, so they won't clash with each other. The only point of conflict is the annotations of model classes, since both use different version of @Id, and can see the other's annotation like @Entity and @Document while parsing. This can be solved by using aliases for all the annotations, using their parent namespace basename as a prefix: model = $model; } public function __toString() { return "Car #$this->document_id: $this->id, $this->model"; } } This make us able to save a copy of an ORM object into Mongo: $car = new Car('Ford'); $this->em->persist($car); $this->em->flush(); $this->dm->persist($car); $this->dm->flush(); var_dump($car->__toString()); $this->assertTrue(strlen($car->__toString()) > 20); The output produces by this test is: .string(38) "Car #4f61a8322f762f1121000000: 3, Ford" When retrieving the object, one of the two ids will be null as it is ignored by the ORM or ODM. I am not using the same field because I want to store multiple copies of a row, so it's id alone won't be unique. If you're interested, checkout my hack on Github. It contains the running example presented in this post. Remember to create the relational schema with: $ php doctrine.php orm:schema-tool:create before running the test with phpunit --bootstrap bootstrap.php DoubleMappingTest.php MongoDB won't need the schema setup, of course. There are still some use cases to test, like the behavior in the presence of proxies, but it seems that non-invasive approach of Data Mappers like Doctrine 2 is paying off: try mapping an object in multiple database with Active Records.
March 20, 2012
by Giorgio Sironi
· 22,482 Views
article thumbnail
Marker Interfaces in Java
Marker Interfaces in Java have special significance because of the fact that they have no methods declared in them which means that the classes implementing these interfaces don't have to override any of the methods. A few of the marker interfaces already exist in the JDK like Serializable and Cloneable. One can also create their own custom interfaces which doesn't have any method. The purpose of these interfaces is to force some kind of functionality in the classes by providing some functionality to a class if it implements the marker interface. A common question asked very frequently is about Runnable interface being marker or not. Runnable interface is not marker because Runnable interface has the public void run() method declared inside it. A very good example of marker interface is Serializable where the class implements can be used with ObjectOutputStream and ObjectInputStream classes. The Java language specification doesn't itself define the term marker interface and the term has been coined by authors, developers and designers. One common question asked is if we can create a marker interface or not and the answer is yes because of following reason: We can't create marker interface similar to Serializable or Cloneable but we can simulate the functionality by writing extra code around the custom marker interface.
March 17, 2012
by Sandeep Bhandari
· 55,659 Views · 2 Likes
article thumbnail
Display an OLE Object from a Microsoft Access Database using OLE Stripper
In database programming, it happens a lot that you need to bind a picture box to a field with type of photo or image. For example, if you want to show an Employee’s picture from Northwind.mdb database, you might want to try the following code: picEmployees.DataBindings.Add(“Image”, bsEmployees, “Photo”, true); This code works if the images are stored in the database with no OLE header or the images stored as a raw image file formats. As the pictures stored in the Northwind database in are not stored in raw image file formats and they are stored as an OLE image documents, then you have to strip off the OLE header to work with the image properly. Binding imageBinding = new Binding("Image", bsEmployees, "ImageBlob.ImageBlob", true); imageBinding.Format += new ConvertEventHandler(this.PictureFormat); private void PictureFormat(object sender, ConvertEventArgs e) { Byte[] img = (Byte[])e.Value; MemoryStream ms = new MemoryStream(); int offset = 78; ms.Write(img, offset, img.Length - offset); Bitmap bmp = new Bitmap(ms); ms.Close(); // Writes the new value back e.Value = bmp; } Fortunately, there are some overload methods in .NET Framework to take care of this mechanism, but it cannot be guaranteed whether you need to strip off the OLE object by yourself or not. For example, you can use the following technique to access the images of the Northwind.mdb that ships with Microsoft Access and they will be rendered properly. picEmployees.DataBindings.Add(“Image”, bsEmployees, “Photo”, true, DataSourceUpdateMode.Never, new Bitmap(typeof(Button), “Button.bmp”)); Unfortunately, there are some scenarios that you need a better solution. For example, the Xtreme.mdb database that ships with Crystal Reports has a photo filed that cannot be handled by the preceding methods. For these complex scenarios, you can download the OLEStripper classes from here and re-write the PictureFormat method as it is shown below: private void PictureFormat(object sender, ConvertEventArgs e) { // photoIndex is same as Employee ID int photoIndex = Convert.ToInt32(e.Value); // Read the original OLE object ReadOLE olePhoto = new ReadOLE(); string PhotoPath = olePhoto.GetOLEPhoto(photoIndex); // Strip the original OLE object StripOLE stripPhoto = new StripOLE(); string StripPhotoPath = stripPhoto.GetStripOLE(PhotoPath); FileStream PhotoStream = new FileStream(StripPhotoPath , FileMode.Open); Image EmployeePhoto = Image.FromStream(PhotoStream); e.Value = EmployeePhoto; PhotoStream.Close(); }
March 15, 2012
by Amir Ahani
· 11,140 Views
article thumbnail
All about JMS messages
JMS providers like ActiveMQ are based on the concept of passing one-directional messages between nodes and brokers asynchronously. A thorough knowledge of the type of messages that can be sent through a JMS middleware can simplify a lot your work in mapping the communication patterns to real code. The basic Message interface Some object members are shared by all messages: header fields, used to identify univocally a message and to route it to the right brokers and consumers. A dynamic map of properties which can be read programmatically by JMS brokers in order to filter or to route messages. A body, which is differentiated in the various implementations we'll see. Header fields The set of getJMS*() methods on the Message interface defines the available headers. Two of them are oriented to message identification: getJMSMessageID() contains a generated ID for identifying a message, unique at least for the current broker. All generated IDs start with the prefix 'ID:', but you can override it with the corresponding setter. getJMSCorrelationID() (and getJMSCorrelationID() as bytes) can link a message with another, usually one that has been sent previously. For example, a reply can carry the ID of the original message when put in another queue. Two to sender and recipient identification: getJMSDestination() returns a Destination object (a Topic or a Queue, or their temporary version) describing where the message was directed. getJMSReplyTo() is a Destination object where replies should be sent; it can be null of course. Three tune the delivery mechanism: getJMSDeliveryMode() can be DeliveryMode.NON_PERSISTENT or DeliveryMode.PERSISTENT; only persistent messages guarantee delivery in case of a crash of the brokers that transport it. getJMSExpiration() returns a timestamp indicating the expiration time of the message; it can be 0 on a message without a defined expiration. getJMSPriority() returns a 0-9 integer value (higher is better) defining the priority for delivery. It is only a best-effort value. While the remaining ones contain metadata: getJMSRedelivered() returns a boolean indicating if the message is being delivered again after a delivery which was not acknowledge. getJMSTimestamp() returns a long indicating the time of sending. getJMSType() defines a field for provider-specific or application-specific message types. Of these headers, only JMSCorrelationID, JMSReplyTo and JMSType have to be set when needed. The others are generated or managed by the send() and publish() methods if not specified (there are setters available for each of these headers.) Properties Generic properties with a String name can be added to messages and read with getBooleanProperty(), getStringProperty() and similar methods. The corresponding setters setBooleanProperty(), setStringProperty(), ... can be used for their addition. The reason for keeping some properties out of the content of the message (which a MapMessage can contain) is so they could be read before reaching the destination, for example in a JMS broker. The use case for this access to message properties is routing and filtering: downstream brokers and consumers may define a filter such as "I am interested only on messages on this Topic that have property X = 'value' and Y = 'value2'". Bodies All the subinterfaces of javax.jms.Message defined by the API provide different types of message bodies (while actual classes are defined by the providers and are not part of the API). Actual instantiation is then handled by the Session, which implements an Abstract Factory pattern. On the receival side, a cast is necessary for any message type (at least in the Java JMS Api), since only a generic Message is read. BytesMessage is the most basic type: it contains a sequence of uninterpreted bytes. Hence, it can in theory contain anything, but the generation and interpretation of the content is the client's job. BytesMessage m = session.createBytesMessage(); m.writeByte(65); m.writeBytes(new byte[] { 66, 68, 70 }); // on receival (cast shown only here) BytesMessage m = (BytesMessage) genericMessage; byte[] content = new byte[4]; m.readBytes(content); MapMessage defines a message containing an (unordered) set of key/value pairs, also called a map or dictionary or hash. However, the keys are String objects, while the values are primitives or Strings; since they are primitives, they shouldn't be null. MapMessage = session.createMapMessage(); m.setString('key', 'value'); // or m.setObject('key', 'value') to avoid specifying a type // on receival m.getString('key'); // or m.getObject('key') ObjectMessage wraps a generic Object for transmission. The Object should be Serializable. ObjectMessage m = session.createObjectMessage(); m.setObject(new ValueObject('field1', 42)); // on receival ValueObject vo = (ValueObject) m.getObject(); StreamMessage wraps a stream of primitive values of indefinite length. StreamMessage m = session.createStreamMessage(); m.writeBoolean(true); m.writeBoolean(false); m.writeBoolean(true); // receival System.out.println(m.readBoolean()); System.out.println(m.readBoolean()); System.out.println(m.readBoolean()); // prints true, false, true TextMessage wraps a String of any length. TextMessage m = session.createTextMessage("Contents"); // or use m.setText() afterwards // receival String text = m.getText(); Usually all messages are in a read-only phase after receival, so only getters can be called on them.
March 12, 2012
by Giorgio Sironi
· 62,018 Views · 1 Like
article thumbnail
Best Practices for Variable and Method Naming
Use short enough and long enough variable names in each scope of code. Generally length may be 1 char for loop counters, 1 word for condition/loop variables, 1-2 words for methods, 2-3 words for classes, 3-4 words for globals. Use specific names for variables, for example "value", "equals", "data", ... are not valid names for any case. Use meaningful names for variables. Variable name must define the exact explanation of its content. Don't start variables with o_, obj_, m_ etc. A variable does not need tags which states it is a variable. Obey company naming standards and write variable names consistently in application: e.g. txtUserName, lblUserName, cmbSchoolType, ... Otherwise readability will reduce and find/replace tools will be unusable. Obey programming language standards and don't use lowercase/uppercase characters inconsistently: e.g. userName, UserName, USER_NAME, m_userName, username, ... use Camel Case (aka Upper Camel Case) for classes: VelocityResponseWriter use Lower Case for packages: com.company.project.ui use Mixed Case (aka Lower Camel Case) for variables: studentName use Upper Case for constants : MAX_PARAMETER_COUNT = 100 use Camel Case for enum class names and Upper Case for enum values. don't use '_' anywhere except constants and enum values (which are constants). For example for Java, Don't reuse same variable name in the same class in different contexts: e.g. in method, constructor, class. So you can provide more simplicity for understandability and maintainability. Don't use same variable for different purposes in a method, conditional etc. Create a new and different named variable instead. This is also important for maintainability and readability. Don't use non-ASCII chars in variable names. Those may run on your platform but may not on others. Don't use too long variable names (e.g. 50 chars). Long names will bring ugly and hard-to-read code, also may not run on some compilers because of character limit. Decide and use one natural language for naming, e.g. using mixed English and German names will be inconsistent and unreadable. Use meaningful names for methods. The name must specify the exact action of the method and for most cases must start with a verb. (e.g. createPasswordHash) Obey company naming standards and write method names consistently in application: e.g. getTxtUserName(), getLblUserName(), isStudentApproved(), ... Otherwise readability will reduce and find/replace tools will be unusable. Obey programming language standards and don't use lowercase/uppercase characters inconsistently: e.g. getUserName, GetUserName, getusername, ... For example for Java, use Mixed Case for method names: getStudentSchoolType use Mixed Case for method parameters: setSchoolName(String schoolName) Use meaningful names for method parameters, so it can documentate itself in case of no documentation.
March 10, 2012
by Cagdas Basaraner
· 154,002 Views · 5 Likes
article thumbnail
All the mouse events in JavaScript
The HTML 5 specification is full of definitions of new events, but it gathers them in a long list only divided by the category of support: some must be supported by all elements, some by window and derivatives, and so on. Unfortunately this style mixes up many different kinds of events, like the multimedia (canplay) and keyboard-based ones (keyup). In this article, I have collected all events that can be generated with a mouse. In HTML 4 There are a few well-supported events ported by the previous version of the specification. Everyone would expect a browser to correctly fire these events: click: the simplest event. dblclick: fired on a double click on an HTML element. mousedown: fired when the button is pressed. mouseup: fired when the button is released. mouseover: fired when the cursor passes over an HTML element. mouseout: fired when the cursor leaves the displaying area of an HTML element; the inverse of mouseover. mousemove: fired everytime the cursor moves one pixel. It's very easy to hang the browser by targeting this event. In any case, window.addEventListener and element.addEventListener (where element is a reference to a DOM element) are the functions to call to setup event handlers. You could also use the on$eventName HTML attribute or property (e.g. onclick, onmouseover), but addEventListener is more flexible as it allows for many different listeners to work on the same event, without one silently replacing the other. HTML 5: dragging Most of the new mouse events (but not necessarily the most interesting ones) deal with drag and drop support: drag: fired frequently on a dragged element (which must define the *draggable* attribute.) dragstart: fired when the mouse is held down on an element and the movement starts. dragend: fired when the element is released. dragenter: fired when an element enters the displayed area of another one. dragleave: fired when an element exits the displayed area, as the inverse of dragenter. dragover: fired frequently when an element is over another. drop: fired when an element is released over another. drag, dragstart, and dragend are fired on the dragged element, while dragenter, dragleave, dragover and drop are fired on the target one. Here is a self-contained example of their usage: One of the common usages of the Drag and Drop API is the integration with the File API to allow for upload of files by dragging into the browser's window. HTML 5: the wheel The mousewheel event is fired upon the usage of the wheel over an element or the window. Here is an example that zooms in or out of an HTML element by modifying its size: The scroll event instead works at an higher-level of abstraction: it is not really a mouse-specific event, as it can be generated by the mouse but also by pressing scrolling keys. What the event models is a change in the visible section of an element, generated by a movement in the vertical or horizontal scrollbars: Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... Lorem ipsum dolor amet... HTML 5: right click The contextmenu event is fired upon a right click, that opens a contextual menu centered on the mouse pointer. Here is an example of interception: If you want to display an alternate contextual menu, you would also probably want to cancel the event; however, many browsers allow the user to override these manipulations and always show the original menu (think of websites hiding the View Source entry).
March 7, 2012
by Giorgio Sironi
· 79,728 Views
article thumbnail
Sketching with HTML5 Canvas and “Brush Images”
In a previous post on capturing user signatures in mobile applications, I explored how you capture user input from mouse or touch events and visualize that in a HTML5 Canvas. Inspired by activities with my daughter, I decided to take this signature capture component and make it a bit more fun & exciting. My daughter and I often draw and sketch together… whether its a magnetic sketching toy, doodling on the iPad, or using a crayon and a placemat at a local pizza joint, there is always something to draw. (Note: I never said I was actually good at drawing.) You can take that exact same signature capture example, make the canvas bigger, and then combine it with a tablet and a stylus, and you’ve got a decent sketching application. However, after doodling a bit you will quickly notice that your sketches leave something to be desired. When you are drawing on a canvas using moveTo(x,y) and lineTo(x,y), you are somewhat limited in what you can do. You have lines which can have consisten thickness, color, and opacity. You can adjust these, however in the end, they are only lines. If you switch your approach away from moveTo and lineTo, then things can get interesting with a minimal amount of changes. You can use images to create “brushes” for drawing strokes in a HTML5 canvas element and add a lot of style and depth to your sketched content. This is an approach that I’ve adapted to JavaScript from some OpenGL drawing applications that I’ve worked on in the past. Take a look at the video below to get an idea what I mean. Examining the sketches side by side, it is easy to see the difference that this makes. The variances in stroke thickness, opacity & angle add depth and style, and provide the appearance of drawing with a magic marker. It’s hard to see the subtleties in this image, so feel free to try out the apps on your own using an iPad or in a HTML5 Canvas-capable browser: Simple Sketch Using lineTo Stylistic Sketches Using a Brush Just click/touch and drag in the gray rectangle area to start drawing. Now, let’s examine how it all works. Both approaches use basic drawing techniques within the HTML5 Canvas element. If you aren’t familiar with the HTML5 Canvas, you can quickly get up to speed from the tutorials from Mozilla. moveTo, lineTo The first technique uses the canvas’s drawing context moveTo(x,y) and lineTo(x,y) to draw line segments that correspond to the mouse/touch coordinates. Think of this as playing “connect the dots” and drawing a solid line between two points. The code for this approach will look something like the following: var canvas = document.getElementById('canvas'); var context = canvas.getContext('2d'); context.beginPath(); context.moveTo(a.x, a.y); context.lineTo(b.x, b.y); context.lineTo(c.x, c.y); context.closePath(); context.stroke(); Brush Images The technique for using brush images is identical in concept to the previous example – you are drawing a line from point A to point B. However, rather than using the built-in drawing APIs, you are programmatically repeating an image (the brush) from point A to point B. First, take a look at the brush image shown below at 400% of the actual scale. It is a simple image that is a diagonal shape that is thicker and more opaque on the left side. By itself, this will just be a mark on the canvas. When you repeat this image from point A to point B, you will get a “solid” line. However the opacity and thickness will vary depending upon the angle of the stroke. Take a look at the sample below (approximated, and zoomed). The question is… how do you actually do this in JavaScript code? First, create an Image instance to be used as the brush source. brush = new Image(); brush.src="assets/brush2.png"; Once the image is loaded, the image can be drawn into the canvas’ context using the drawImage() function. The trick here is that you will need to use some trigonometry to determine how to repeat the image. In this case, you can calculate the angle and distance from the start point to the end point. Then, repeat the image based on that distance and angle. var canvas = document.getElementById('canvas'); var context = canvas.getContext('2d'); var halfBrushW = brush.width/2; var halfBrushH = brush.height/2; var start = { x:0, y:0 }; var end = { x:200, y:200 }; var distance = parseInt( Trig.distanceBetween2Points( start, end ) ); var angle = Trig.angleBetween2Points( start, end ); var x,y; for ( var z=0; (z<=distance || z==0); z++ ) { x = start.x + (Math.sin(angle) * z) - halfBrushW; y = start.y + (Math.cos(angle) * z) - halfBrushH; context.drawImage(this.brush, x, y); } For the trigonometry functions, I have a simple utility class to calculate the distance between two points, and the angle between two points. This is all based upon the good old Pythagorean theorem. var Trig = { distanceBetween2Points: function ( point1, point2 ) { var dx = point2.x - point1.x; var dy = point2.y - point1.y; return Math.sqrt( Math.pow( dx, 2 ) + Math.pow( dy, 2 ) ); }, angleBetween2Points: function ( point1, point2 ) { var dx = point2.x - point1.x; var dy = point2.y - point1.y; return Math.atan2( dx, dy ); } } The full source for both of these examples is available on github at: https://github.com/triceam/HTML5-Canvas-Brush-Sketch This example uses the twitter bootstrap UI framework, jQuery, and Modernizr. Both the lineTo.html and brush.html apps use the exact same code, which just uses a separate rendering function based upon the use case. Feel free to try out the apps on your own using an iPad or in a HTML5 Canvas-capable browser: Simple Sketch Using lineTo Stylistic Sketches Using a Brush Just click/touch and drag in the gray rectangle area to start drawing.
March 4, 2012
by Andrew Trice
· 18,151 Views
article thumbnail
Allowing Duplicate Keys in Java Collections
Java Collections allows you to add one or more elements with the same key by using the MultiValueMap class, found in the Apache org.apache.commons.collections package (http://commons.apache.org/collections/): package multihashmap.example; import java.util.Iterator; import java.util.List; import java.util.Map; import java.util.Set; import org.apache.commons.collections.map.MultiValueMap; public class MultiHashMapExample { public static void main(String[] args) { List list; MultiValueMap map = new MultiValueMap(); map.put("A", 4); map.put("A", 6); map.put("B", 7); map.put("C", 1); map.put("B", 9); map.put("A", 5); Set entrySet = map.entrySet(); Iterator it = entrySet.iterator(); System.out.println(" Object key Object value"); while (it.hasNext()) { Map.Entry mapEntry = (Map.Entry) it.next(); list = (List) map.get(mapEntry.getKey()); for (int j = 0; j < list.size(); j++) { System.out.println("\t" + mapEntry.getKey() + "\t " + list.get(j)); } } } } Since Java Core does’t come with some solutions for supporting multiple keys, using the org.apache.commons.collections seems to be a proper way to deal with multiple keys. And the output is: From http://e-blog-java.blogspot.com/2012/02/how-to-allow-duplicate-key-in-java.html
March 3, 2012
by A. Programmer
· 100,429 Views · 5 Likes
article thumbnail
HTML5 Image Effects: Sepia
Today we continue our HTML5 canvas examples. Today I want to share with you a method of applying a sepia effect to images. This is not a very difficult method, anyone can repeat it. In our demo we can play with different images by adding a sepia effect to them, as well as ‘export’ our result on the image element (). Here are our demo and downloadable package: Live Demo download in package Ok, download the example files and lets start coding ! Step 1. HTML Markup This is the markup of our demo page. Here it is: index.html HTML5 Image Effects - Sepia Back to original tutorial on Script Tutorials Canvas Object Image Object Next imageApply Sepia EffectTo Image Basically – it contains just one canvas object, one image, and three ‘buttons’ (div elements). Step 2. CSS Here are our stylesheets: css/main.css *{ margin:0; padding:0; } body { background-image:url(../images/bg.png); color:#fff; font:14px/1.3 Arial,sans-serif; } header { background-color:#212121; box-shadow: 0 -1px 2px #111111; display:block; height:70px; position:relative; width:100%; z-index:100; } header h2{ font-size:22px; font-weight:normal; left:50%; margin-left:-400px; padding:22px 0; position:absolute; width:540px; } header a.stuts,a.stuts:visited { border:none; text-decoration:none; color:#fcfcfc; font-size:14px; left:50%; line-height:31px; margin:23px 0 0 110px; position:absolute; top:0; } header .stuts span { font-size:22px; font-weight:bold; margin-left:5px; } .container { color: #000000; margin: 20px auto; overflow: hidden; position: relative; width: 1005px; } table { background-color: rgba(255, 255, 255, 0.7); } table td { border: 1px inset #888888; position: relative; text-align: center; } table td p { display: block; padding: 10px 0; } .button { cursor: pointer; height: 20px; padding: 15px 0; position: relative; text-align: center; width: 500px; -moz-user-select: none; -khtml-user-select: none; user-select: none; } Step 3. JS Finally – our Javascript code generating the sepia effect: js/script.js // variables var canvas, ctx; var imgObj; // set of sepia colors var r = [0, 0, 0, 1, 1, 2, 3, 3, 3, 4, 4, 4, 5, 5, 5, 6, 6, 7, 7, 7, 7, 8, 8, 8, 9, 9, 9, 9, 10, 10, 10, 10, 11, 11, 12, 12, 12, 12, 13, 13, 13, 14, 14, 15, 15, 16, 16, 17, 17, 17, 18, 19, 19, 20, 21, 22, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 39, 40, 41, 42, 44, 45, 47, 48, 49, 52, 54, 55, 57, 59, 60, 62, 65, 67, 69, 70, 72, 74, 77, 79, 81, 83, 86, 88, 90, 92, 94, 97, 99, 101, 103, 107, 109, 111, 112, 116, 118, 120, 124, 126, 127, 129, 133, 135, 136, 140, 142, 143, 145, 149, 150, 152, 155, 157, 159, 162, 163, 165, 167, 170, 171, 173, 176, 177, 178, 180, 183, 184, 185, 188, 189, 190, 192, 194, 195, 196, 198, 200, 201, 202, 203, 204, 206, 207, 208, 209, 211, 212, 213, 214, 215, 216, 218, 219, 219, 220, 221, 222, 223, 224, 225, 226, 227, 227, 228, 229, 229, 230, 231, 232, 232, 233, 234, 234, 235, 236, 236, 237, 238, 238, 239, 239, 240, 241, 241, 242, 242, 243, 244, 244, 245, 245, 245, 246, 247, 247, 248, 248, 249, 249, 249, 250, 251, 251, 252, 252, 252, 253, 254, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255], g = [0, 0, 1, 2, 2, 3, 5, 5, 6, 7, 8, 8, 10, 11, 11, 12, 13, 15, 15, 16, 17, 18, 18, 19, 21, 22, 22, 23, 24, 26, 26, 27, 28, 29, 31, 31, 32, 33, 34, 35, 35, 37, 38, 39, 40, 41, 43, 44, 44, 45, 46, 47, 48, 50, 51, 52, 53, 54, 56, 57, 58, 59, 60, 61, 63, 64, 65, 66, 67, 68, 69, 71, 72, 73, 74, 75, 76, 77, 79, 80, 81, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 97, 100, 101, 102, 103, 105, 106, 107, 108, 109, 111, 113, 114, 115, 117, 118, 119, 120, 122, 123, 124, 126, 127, 128, 129, 131, 132, 133, 135, 136, 137, 138, 140, 141, 142, 144, 145, 146, 148, 149, 150, 151, 153, 154, 155, 157, 158, 159, 160, 162, 163, 164, 166, 167, 168, 169, 171, 172, 173, 174, 175, 176, 177, 178, 179, 181, 182, 183, 184, 186, 186, 187, 188, 189, 190, 192, 193, 194, 195, 195, 196, 197, 199, 200, 201, 202, 202, 203, 204, 205, 206, 207, 208, 208, 209, 210, 211, 212, 213, 214, 214, 215, 216, 217, 218, 219, 219, 220, 221, 222, 223, 223, 224, 225, 226, 226, 227, 228, 228, 229, 230, 231, 232, 232, 232, 233, 234, 235, 235, 236, 236, 237, 238, 238, 239, 239, 240, 240, 241, 242, 242, 242, 243, 244, 245, 245, 246, 246, 247, 247, 248, 249, 249, 249, 250, 251, 251, 252, 252, 252, 253, 254, 255], b = [53, 53, 53, 54, 54, 54, 55, 55, 55, 56, 57, 57, 57, 58, 58, 58, 59, 59, 59, 60, 61, 61, 61, 62, 62, 63, 63, 63, 64, 65, 65, 65, 66, 66, 67, 67, 67, 68, 69, 69, 69, 70, 70, 71, 71, 72, 73, 73, 73, 74, 74, 75, 75, 76, 77, 77, 78, 78, 79, 79, 80, 81, 81, 82, 82, 83, 83, 84, 85, 85, 86, 86, 87, 87, 88, 89, 89, 90, 90, 91, 91, 93, 93, 94, 94, 95, 95, 96, 97, 98, 98, 99, 99, 100, 101, 102, 102, 103, 104, 105, 105, 106, 106, 107, 108, 109, 109, 110, 111, 111, 112, 113, 114, 114, 115, 116, 117, 117, 118, 119, 119, 121, 121, 122, 122, 123, 124, 125, 126, 126, 127, 128, 129, 129, 130, 131, 132, 132, 133, 134, 134, 135, 136, 137, 137, 138, 139, 140, 140, 141, 142, 142, 143, 144, 145, 145, 146, 146, 148, 148, 149, 149, 150, 151, 152, 152, 153, 153, 154, 155, 156, 156, 157, 157, 158, 159, 160, 160, 161, 161, 162, 162, 163, 164, 164, 165, 165, 166, 166, 167, 168, 168, 169, 169, 170, 170, 171, 172, 172, 173, 173, 174, 174, 175, 176, 176, 177, 177, 177, 178, 178, 179, 180, 180, 181, 181, 181, 182, 182, 183, 184, 184, 184, 185, 185, 186, 186, 186, 187, 188, 188, 188, 189, 189, 189, 190, 190, 191, 191, 192, 192, 193, 193, 193, 194, 194, 194, 195, 196, 196, 196, 197, 197, 197, 198, 199]; // noise value var noise = 20; function processSepia() { // get current image data var imageData = ctx.getImageData(0,0,canvas.width,canvas.height); for (var i=0; i < imageData.data.length; i+=4) { // change image colors imageData.data[i] = r[imageData.data[i]]; imageData.data[i+1] = g[imageData.data[i+1]]; imageData.data[i+2] = b[imageData.data[i+2]]; // apply noise if (noise > 0) { var noise = Math.round(noise - Math.random() * noise); for(var j=0; j<3; j++){ var iPN = noise + imageData.data[i+j]; imageData.data[i+j] = (iPN > 255) ? 255 : iPN; } } } // put image data back to context ctx.putImageData(imageData, 0, 0); }; $(function () { // create canvas and context objects canvas = document.getElementById('source'); ctx = canvas.getContext('2d'); // load source image imgObj = new Image(); imgObj.onload = function () { // draw image ctx.drawImage(this, 0, 0, this.width, this.height, 0, 0, canvas.width, canvas.height); } imgObj.src="images/pic1.jpg"; // different onclick handlers var iCur = 1; $('#next').click(function () { iCur++; if (iCur > 6) iCur = 1; imgObj.src="images/pic" + iCur + '.jpg'; }); $('#sepia').click(function () { processSepia(); }); $('#toImage').click(function () { $('#img').attr('src', canvas.toDataURL('image/jpeg')); }); }); Main idea: as we know – sepia images have their own, quite predefined colors. So our script will walk through all pixels of the original image, and change the pixel color to the corresponding sepia color. Plus, we will add a little of noise to our image. Live Demo download in package Conclusion I hope that our demo looks fine. Today we have added a new interesting effect to our html5 application – Sepia. I will be glad to see your thanks and comments. Good luck!
March 2, 2012
by Andrei Prikaznov
· 11,856 Views
article thumbnail
Multiple File Upload is Easy in HTML5
this won't be a terribly long post, nor one that is probably informative to a lot of people, but i finally got around to looking at the html5 specification for multiple file uploads (by that i mean allowing a user to pick multiple files from one file form field). i knew it existed i just never got around to actually looking at it. turns out it is incredibly simple. you take a file form field: and you add multiple to it (or multiple="multiple"). and... yeah, that's it. even better, it degrades perfectly. so in chrome it works, and your label switches from... to... i kept waiting for the "but, wait" moment and it never came. in ie where it fails to work, it simply remains a file upload control that works with one file, not many. because it's so simple and fallback is so good, i can't see really bothering to use another solution, like a flash uploader, but you could do a bit of massaging for it. so for example, consider if my form had the following label: multiple file: in ie, the user won't be able to select multiple files. while not the end of the world, it kind of bugs me that the label may mislead the user into trying something they can't do. (and as we know, the user will blame us, not ie.) so what to do? i did some basic research into how to use javascript to detect features. i found a few good examples that basically suggested creating a temporary dom item like so: var el = document.createelement("input"); but here's where i had issues. most of the links i found discussed how to detect new html form types , like range for example. these techniques worked by setting the type on the new element to the type you want to test and then immediately checking to see if the type still has the same value. but what about attributes? turns out dive into html5 has a great article on this. when you have your dom item, you can simply ask if the item exists. here is an example: function supportmultiple() { //do i support input type=file/multiple var el = document.createelement("input"); return ("multiple" in el); } so given that, i modified my mark up a bit. i changed the label in front of my form and hid the word "multiple": multiple file: i then added an init method to my body tag and used the following code: function supportmultiple() { //do i support input type=file/multiple var el = document.createelement("input"); return ("multiple" in el); } function init() { if(supportmultiple()) { document.queryselector("#multiplefilelabel").setattribute("style",""); } } and that worked like a charm. ie say just "file" whereas chrome and firefox had the new hotness. so yeah - that's a lot of writing about a simple little thing, but... dare i say it... i'm really getting jazzed up about html again! here's my entire test template if you want to cut and paste: normal text: multiple file: p.s. so how you actually process the upload depends on your server. my coldfusion readers are probably wondering how it handles this. good news, bad news. the bad news is that coldfusion can't (as far as i and other smarter peoiple know) handle this at all. if you want to do this in coldfusion 9, use the flash based multifile uploader instead. the good news is that coldfusion 10 handles it just file. be sure to use action value of "uploadall" of your cffile tag.
February 29, 2012
by Raymond Camden
· 101,569 Views
article thumbnail
Running Multiple Tomcat Instances on One Server
Here’s a brief step by step guide to running more than one instance of Tomcat on a single machine. Step 1: Install the Tomcat files Download Tomcat 5.5, 6.x, or 7.x and unzip it into an appropriate directory. I usually put it in /usr/local, so it ends up in a directory called /usr/local/apache-tomcat-5.5.17, and make a symlink named /usr/local/tomcat to that directory. When later versions come out, I can unzip them and relink, leaving the older version in case things don’t work out (which rarely if ever happens, but I’m paranoid). Step 2: Make directories for each instance For each instance of Tomcat you’re going to run, you’ll need a directory that will be CATALINA_BASE. For example, you might make them /var/tomcat/serverA and /var/tomcat/serverB. In each of these directories you need the following subdirectories: conf, logs, temp, webapps, and work. Put a server.xml and web.xml file in the conf directory. You can get these from the conf directory of the directory where you put the tomcat installation files, although of course you should tighten up your server.xml a bit. The webapps directory is where you’ll put the web applications you want to run on the particular instance of Tomcat. I like to have the Tomcat manager webapp installed on each instance, so I can play with the webapps, and see how many active sessions there are. See my instructions for configuring the Tomcat manager webapp. Step 3: Configure the ports and/or addresses for each instance Tomcat listens to at least two network ports, one for the shutdown command, and one or more for accepting requests. Two instances of Tomcat can’t listen to the same port number on the same IP address, so you will need to edit your server.xml files to change the ports they listen to. The first port to look at is the shutdown port. This is used by the command line shutdown script (actually, but the Java code it runs) to tell the Tomcat instance to shut itself down. This port is defined at the top of the server.xml file for the instance. Make sure each instance uses a different port value. The port value will normally need to be higher than 1024, and shouldn’t conflict with any other network service running on the same system. The shutdown string is the value that is sent to shut the server down. Note that Tomcat won’t accept shutdown commands that come from other machines. Unlike the other ports Tomcat listens to, the shutdown port can’t be configured to listen to its port on a different IP address. It always listens on 127.0.0.1. The other ports Tomcat listens to are configured with the elements, for instance the HTTP or JK listeners. The port attribute configures which port to listen to. Setting this to a different value on the different Tomcat instances on a machine will avoid conflict. Of course, you’ll need to configure whatever connects to that Connector to use the different port. If a web server is used as the front end using mod_jk, mod_proxy, or the like, then this is simple enough - change your web server’s configuration. In some cases you may not want to do this, for instance you may not want to use a port other than 8080 for HTTP connectors. If you want all of your Tomcat intances to use the same port number, you’ll need to use different IP addresses. The server system must be configured with multiple IP addresses, and the address attribute of the element for each Tomcat instance will be set to the appropriate IP address. Step 4: Startup Startup scripts are a whole other topic, but here’s the brief rundown. The main different from running a single Tomcat instance is you need to set CATALINA_BASE to the directory you set up for the particular instance you want to start (or stop). Here’s a typical startup routine: JAVA_HOME=/usr/java JAVA_OPTS="-Xmx800m -Xms800m" CATALINA_HOME=/usr/local/tomcat CATALINA_BASE=/var/tomcat/serverA export JAVA_HOME JAVA_OPTS CATALINA_HOME CATALINA_BASE $CATALINA_HOME/bin/catalina.sh start Source: http://kief.com/running-multiple-tomcat-instances-on-one-server.html
February 29, 2012
by Kief Morris
· 96,741 Views · 4 Likes
article thumbnail
Running JavaScript inside PHP code
v8js is a new PHP extension able to run JavaScript code inside V8, Google's JavaScript interpreter that powers for example Chrome and NodeJS. This extension is highly alpha - and its API would probably change in the months ahead. Since documentation is lacking, I invite you to repeat the discovering process I follow in this post in case you find some differences in a new version of v8js. Installation V8 must be present on the machine in order to install the extension. On a Debian/Ubuntu system, run the following: sudo apt-get install libv8-dev libv8-dbg libv8-dev will also install libv8 in its latest version. Afterwards, download and compile the extension with: sudo pecl install v8js There are no more requirements for compilation apart from the usual dependencies for PECL packages, like build-essential. After that, add: extension=v8js.so to php.ini or to a section in conf.d. $ php -m | grep v8 v8js will confirm you the extension is loaded. Some introspection PHP's reflection let us take a look at the classes and methods provided by this extension, even without documentation available. It probably hasn't been written yet, due to the unstable API. $ php -r 'var_dump(get_declared_classes());' | grep V8 string(8) "V8Object" string(10) "V8Function" string(4) "V8Js" string(13) "V8JsException" $ php -r '$class = new ReflectionClass("V8Js"); var_dump($class->getMethods());' array(5) { [0]=> &object(ReflectionMethod)#2 (2) { ["name"]=> string(11) "__construct" ["class"]=> string(4) "V8Js" } [1]=> &object(ReflectionMethod)#3 (2) { ["name"]=> string(13) "executeString" ["class"]=> string(4) "V8Js" } [2]=> &object(ReflectionMethod)#4 (2) { ["name"]=> string(19) "getPendingException" ["class"]=> string(4) "V8Js" } [3]=> &object(ReflectionMethod)#5 (2) { ["name"]=> string(17) "registerExtension" ["class"]=> string(4) "V8Js" } [4]=> &object(ReflectionMethod)#6 (2) { ["name"]=> string(13) "getExtensions" ["class"]=> string(4) "V8Js" } } $ php -r '$v8 = new V8Js(); var_dump($v8->executeString("1+2+3"));' int(6) Interesting! We have just executed our first JavaScript expression inside a PHP process. Apparently the last statement's value is returned by the executeString() method, with a rough conversion preserving the type: $ php -r '$v8 = new V8Js(); var_dump($v8->executeString("var obj = {}; obj.field = 1; obj.field++; obj.field;"));' int(2) Syntax or runtime errors are signaled with a V8JsException: $ php -r '$v8 = new V8Js(); var_dump($v8->executeString("var obj = {"));' PHP Fatal error: Uncaught exception 'V8JsException' with message 'V8Js::executeString():1: SyntaxError: Unexpected end of input' in Command line code:1 Stack trace: #0 Command line code(1): V8Js->executeString('var obj = {') #1 {main} thrown in Command line code on line 1 Let's add more difficulty The FizzBuzz kata OO solution is an example of JavaScript code creating an object and executing anonymous functions: it's a good test bench for our integration.Since evaluating a variable as the last line returns it, that is our channel of communication, supporting integers, strings, floats, booleans, arrays (not objects at this time). Meanwhile, input for JavaScript code can be embedded into the executed string. This code will output string(8) "FizzBuzz": executeString($javaScriptCode)); By changing it a bit, we can build a JSON string by backslashing the double quotes ("), and returns it in lieu of an object to communicate to the PHP process a complex result: ... var myFizzBuzz = new FizzBuzz({3 : "Fizz", 5 : "Buzz"}); "{\"15\" : \"" + myFizzBuzz.accept(15) + "\", \"5\" : \"" + myFizzBuzz.accept(5) + "\"}"; '; $v8 = new V8Js(); $result = $v8->executeString($javaScriptCode); var_dump($result); var_dump(json_decode($result)); The output is: string(33) "{"15" : "FizzBuzz", "5" : "Buzz"}" object(stdClass)#2 (2) { ["15"]=> string(8) "FizzBuzz" ["5"]=> string(4) "Buzz" } Wiring Executing an external script would be nice: it would provide better stack traces, with traceable line numbers at the JavaScript level. It would also mean we won't need backslashing for single quotes, simplifying the syntax. We cannot load scripts on the JavaScript side out of the box, due to V8 missing this functionality (Node JS adds this feature) but we can load the code on the PHP Side: // test.js $ php loadingfiles.php string(24) "test.js file was loaded." // loadingfiles.php executeString($javascriptCode); var_dump($result); Execution results in: $ php loadingfiles.php string(24) "test.js file was loaded." However, we still miss the capability of including JavaScript libraries. Conclusion There are many possible use cases for v8js, like sandboxed scripting or the integration of some code which was written for the client side. That would not be the most clean solution, but it's a Turing complete approach, so why not? After all, it's already possible to run PHP on Java or Python and Ruby on .NET. JavaScript is becoming ubiquitous, so why not providing support for it, even in a caged box?
February 29, 2012
by Giorgio Sironi
· 52,670 Views
article thumbnail
Audio in HTML 5: state of the art
HTML 5 capable browsers support multimedia files as first class citizens - HTML is not a language for describing documents anymore, but more a medium for building applications, which can leverage every bit of built-in functionality to run fast and to not weigh heavily on the shoulders of developers for maintenance. In this article, we'll see all the ways to play audio inside a modern browser via HTML 5. The element is a new tag not only able to load a sound file and play it, but also to feature a series of control buttons for playing, pausing and pausing the reproduction. Of course there are many use cases where you won't need controls, like for the background music of a game. is not more complex than : you specify a mandatory src attribute, and other options to configure the sound player where needed. The attributes shown in this example (and a few others) are: controls: when present, displays a browser-specific player to control the playback. The HTML 5 specification says that with these controls the user should be able to "begin playback, pause playback, seek to an arbitrary position in the content (if the content supports arbitrary seeking), change the volume" and a bunch of other features that I have not seen supported yet. autoplay: when present, tells the browser to begin playing the sound file as soon as it can. loop: when present, specifies the sound file should start again as soon as it finishes. muted: when present, sets the volume at 0 as a default. preload: specifies a policy for loading the sound file. "none" corresponds to no prefetching, which minimizes traffic; "metadata" prefetches only the start of the file, in order to read metadata like a song's duration. "auto" authorizes the browser to prefetch the whole file. Audio object Apparently the combination of HTML 5 and CSS 3 is a Turing complete language: however, it is more practical to code behavior of a web application in JavaScript. That's why eveyrthing you can do with an element can also be accomplished programmatically with JavaScript Audio objects: Thus via JavaScript you can control an audio element by calling a series of methods such as play() and pause(), and adjust properties like *volume*. Audio objects implement an interface named HtmlMediaElement, where you can find the list of all methods, plus modifiable and readonly properties available. For example, doorBell.seekable.length is a range, while doorBell.currentTime is an assignable property to make reproduction jump to a certain point of the file. Formats and support The real problem with HTML 5 Audio right now is its support in different browsers; not the support for and Audio - which is pretty consistent - but for the codecs to play all the different formats an audio file can come in. There are mainly two sides in the issue: Firefox, Chrome and Opera being as usual friendly to open source and open formats, and being the first to support Ogg Vorbis. Ogg is an open container for audio and video files, and Vorbis is a free lossy codec like MP3, but without any patent on it. Internet Explorer and Safari supporting MP3 instead. Let's not even talk about support on mobile browsers, which is close to non-existent with respect to codecs. By the way, for experimenting with stick to Chrome, Firefox and Opera. Fortunately, a series of elements can be embedded in an (or , equivalently) element to provide alternative formats, listed by MIME type: Encoding lots of files in different formats just to please browsers can be a pain, but it's viable in the case of small clips and sound effects.
February 27, 2012
by Giorgio Sironi
· 7,672 Views
article thumbnail
Serializing Python-Requests' Session Objects for Fun and Profit
Originally Authored By Shrikant Sharat If you haven't checked out @kennethreitz's excellent python-requests library yet, I suggest you go do that immediately. Go on, I'll wait for you. Had your candy? That is one of the most beatiful pieces of python code I've read. And its an excellent library with a very humane API. Recently, I have been using this library for a few of my company's internal projects and at a point I needed to serialize and save Session objects for later. That wasn't as straightforward as I first thought it'd be, so I am sharing my experience here. First off, let's make a simple http server which we are going to contact with python-requests. The server should be able to handle cookie based sessions and also have basic auth, as these things are handled by python-requests' Session objects on the client side. I won't discuss the code for the server here, you can get it from bitbucket. Once you have the server running, now for the client, lets do requests! import requests as req URL_ROOT = 'http://localhost:5050' def get_logged_in_session(name): session = req.session(auth=('user', 'pass')) login_response = session.post(URL_ROOT + '/login', data={'name': name}) login_response.raise_for_status() return session def get_whoami(session): response = session.get(URL_ROOT + '/whoami') response.raise_for_status() return response.text I defined two functions here. The get_logged_in_session will create a new session and login to the http server and return that session. Any subsequent requests using this sesssion will be made as if you have logged in. That's what will be tested with the get_whoami function, which will just return the response from /whoami. Lets test this out. Make sure the server.py is running and in another terminal, $ python -i client.py >>> s = get_logged_in_session('sharat') >>> get_whoami(s) u'You are sharat' >>> get_whoami(req.session(auth=('user', 'pass'))) u'You are a guest' Works perfectly. If we pass it the logged in session, it gives us the username and if we pass it a new session, it gives us a guest. Now, lets assume we have two functions, serialize_session and deserialize_session which do exactly what their names say. We can test them out by running a small test.py, as from client import get_logged_in_session, get_whoami from serializer import deserialize_session, serialize_session session = get_logged_in_session('sharat') dsession = deserialize_session(serialize_session(session)) assert get_whoami(session) == get_whoami(dsession) print 'Success' and a dummy serializer.py def serialize_session(session): return session def deserialize_session(session): return session And with that, of course, the test will not fail $ python test.py Success Serializing Now, to implement the functions in serializer.py. A simple one, would be to use pickle. Lets try import pickle as pk def serialize_session(session): return pk.dumps(session) def deserialize_session(data): return pk.loads(data) If you run test.py now, python is going to yell at you. $ python test.py Traceback (most recent call last): File "test.py", line 10, in dsession = deserialize_session(serialize_session(session)) [ ... ] raise TypeError, "can't pickle %s objects" % base.__name__ TypeError: can't pickle lock objects Oh well, it was worth a try I suppose. Update: The Session class can be made to implement the pickle protocol if you want to use pickle. Next plan I had was to pick up attributes and data from a Session object, just enough to recreate this object using the Session constructor, and serialize those attributes as a json. After all, the Session's API is very easy to use, how hard can picking attributes from it be? :) So, I dug in the sessions.py module of python-requests library. And here's what the signature of the constructor for Session objects looks like def __init__(self, headers=None, cookies=None, auth=None, timeout=None, proxies=None, hooks=None, params=None, config=None, verify=True): # ... So, if I pick up just these values, I should be able to recreate the session object. Sweet. import json import requests as req def serialize_session(session): attrs = ['headers', 'cookies', 'auth', 'timeout', 'proxies', 'hooks', 'params', 'config', 'verify'] session_data = {} for attr in attrs: session_data[attr] = getattr(session, attr) return json.dumps(session_data) def deserialize_session(data): return req.session(**json.loads(data)) And let's try this out $ python test.py Traceback (most recent call last): File "test.py", line 12, in assert get_whoami(session) == get_whoami(dsession) [ ... ] [...]requests/models.py", line 447, in send r = self.auth(self) TypeError: 'list' object is not callable Okay, that error message is very wierd. Why would anyone call a list object? Go dig in the models.py module. See this [ ... ] if isinstance(self.auth, tuple) and len(self.auth) == 2: # special-case basic HTTP auth self.auth = HTTPBasicAuth(*self.auth) # Allow auth to make its changes. r = self.auth(self) [ ... ] There. Its not a list that's being called. Not directly at least. The problem here is that the auth we are passing to session() is not a tuple. Duh! While I like it that auth is restricted to be a tuple, I wish there was a better error message for when auth is a list instead of a tuple. I personally wouldn't want it to accept a list for auth though. So, what went wrong? json does not differentiate between a tuple and a list. It only does lists. So, when serializing and deserializing, the auth tuple is turned to a list. Lets turn it back def deserialize_session(data): session_data = json.loads(data) if 'auth' in session_data: session_data['auth'] = tuple(session_data['auth']) return req.session(**session_data) And $ python test.py Traceback (most recent call last): File "test.py", line 12, in assert get_whoami(session) == get_whoami(dsession) [ ... ] File "/usr/lib/python2.7/string.py", line 493, in translate return s.translate(table, deletions) TypeError: translate() takes exactly one argument (2 given) Wait. What? Now we have an error from stdlib? This just keeps getting better and better. If this looks like something that can frustrate you, go get some coffee :) If you look at the complete stack trace, the second file from bottom, File "[...]site-packages/requests/packages/oreos/monkeys.py", line 470, in set if "" != translate(key, idmap, LegalChars): This thing seems to be calling the translate method incorrectly. With a bit of debugging and yelling at my monitor, I found out the problem and for a moment, lost my grip on reality. str.translate takes 2 arguments, but unicode.translate takes only 1. I have no idea why this is done this way but I sure as hell didn't enjoy it. The code in oreos/monkeys.py assumes that the key is a str. However, what json.loads gives you, is unicode stuff. So, we need to convert just the parts in the deserialized dict we get from json.loads which are being used by the oreos/monkeys.py, from unicode to str. Reading a bit more code around the oreos library, it didn't take long to figure out that those were the keys in the cookies dict. Lo def deserialize_session(data): session_data = json.loads(data) if 'auth' in session_data: session_data['auth'] = tuple(session_data['auth']) if 'cookies' in session_data: session_data['cookies'] = dict((key.encode(), val) for key, val in session_data['cookies'].items()) return req.session(**session_data) And so $ python test.py Success ! All the code is on a bitbucket repository. Update: Pickling can also work As Daslch pointed out in his comment on reddit, by implementing the pickle protocol on the Session class, we can get pickling to work. From the documentation, we need two methods, __getstate__ and __setstate__. Adding those methods as follows to sessions.Session class def __getstate__(self): attrs = ['headers', 'cookies', 'auth', 'timeout', 'proxies', 'hooks', 'params', 'config', 'verify'] return dict((attr, getattr(self, attr)) for attr in attrs) def __setstate__(self, state): for name, value in state.items(): setattr(self, name, value) self.poolmanager = PoolManager( num_pools=self.config.get('pool_connections'), maxsize=self.config.get('pool_maxsize') ) with this as the version of serializer.py that uses pickle, we do get a Success. The creation of new poolmanager in __setstate__ is a piece of code copied from __init__ of the same class. This should probably be turned to a method to avoid code repetition. Update 2: Created an issue about this. Update 3: This has been merged and Session objects are pickleable as of version 0.10.3. See requests history. Source: http://sharats.me/serializing-python-requests-session-objects-for-fun-and-profit.html
February 27, 2012
by Chris Smith
· 11,678 Views
  • Previous
  • ...
  • 438
  • 439
  • 440
  • 441
  • 442
  • 443
  • 444
  • 445
  • 446
  • 447
  • ...
  • Next
  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook
×