DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

The Latest Security Topics

article thumbnail
Spring Tool Suite (STS) and Groovy/Grails Tool Suite (GGTS) 3.0.0 releases
We are proud to announce that the newest major release of our Eclipse-based developer tooling is now available. This is a major release not only in terms of new features but because of other serious changes like project componentization, open-sourcing and the fact that for the first time we are making multiple distributions available, each tailored for a different kind of developer. Check out the release announcement on Martin Lippert's Blog. 100% Open Sourced – All STS features that were previously under a free commercial license, have been donated under the Eclipse Public License (EPL) at GitHub! Intelligent Repackaging - Repackaging the product itself makes identifying what tools you need, and getting started with them much easier. In the past, Groovy/Grails developers had to install several extensions manually into Eclipse to get started. Now there are two full eclipse distributions, one targeted at Spring developers, the other at Groovy/Grails developers – just download, install and go, no assembly required. Componentized projects: Componentizing allows installation and configuration flexibility – developers can install components individually into their existing, plain Eclipse Java EE installations if they wish, preserving their hard work of configuring their Eclipse IDEs just the way they like them. Downloads, more information and FAQ You can find the downloads as well as more information on the project websites for the toolsuites: Spring Tool Suite Groovy/Grails Tool Suite Installation Instructions FAQ Feedback and discussions If you have feedback or questions for us, please do not hesitate to contact us via our SpringSource Tool Suite forum. Bugs and feature requests are always welcome as tickets in our JIRA or, even better, as pull requests on GitHub.
August 22, 2012
by Pieter Humphrey
· 3,465 Views
article thumbnail
Spring Data, Spring Security and Envers integration
Learn about pros, cons, and basics of Spring security and data, plus Envers integration.
August 20, 2012
by Nicolas Fränkel
· 25,022 Views · 1 Like
article thumbnail
Method injection with Spring
Spring core comes out-of-the-box with two scopes: singletons and prototypes. Singletons implement the Singleton pattern, meaning there's only a single instance at runtime (in a JVM). Spring instantiates them during context creation, caches them in the context, and serves them from the cache when needed (or something like that). Prototypes are instantiated each time you access the context to get the bean. Problems arise when you need to inject a prototype-scoped bean in a singleton-scoped bean. Since singletons are created (and then injected) during context creation: it's the only time the Spring context is accessed and thus prototype-scoped beans are injected only once, thus defeating their purpose. In order to inejct prototypes into singletons, and side-by-syde with setter and constructor injection, Spring proposes another way for injection, called method injection. It works in the following way: since singletons are instantiated at context creation, it changes the way prototype-scoped are handled, from injection to created by an abstract method. The following snippet show the unsuccessful way to achieve injection: public class Singleton { private Prototype prototype; public Singleton(Prototype prototype) { this.prototype = prototype; } public void doSomething() { prototype.foo(); } public void doSomethingElse() { prototype.bar(); } } The next snippet displays the correct code: public abstract class Singleton { protected abstract Prototype createPrototype(); public void doSomething() { createPrototype().foo(); } public void doSomethingElse() { createPrototype().bar(); } } As you noticed, code doesn't specify the createPrototype() implementation. This responsibility is delegated to Spring, hence the following needed configuration: Note that an alternative to method injection would be to explicitly access the Spring context to get the bean yourself. It's a bad thing to do since it completely defeats the whole Inversion of Control pattern, but it works (and is essentially the only option when a nasty bug happens on the server - see below).However, using method injection has several main limitations: Spring achieves this black magic by changing bytecode. Thus, you'll need to have the CGLIB libraryon the classpath. The feature is only available by XML configuration, no annotations (see this JIRAfor more information) Finally, some application servers have bugs related to CGLIB (such as this one) To go further: Spring's documentation on method injection
July 30, 2012
by Nicolas Fränkel
· 97,962 Views · 4 Likes
article thumbnail
Set up a Nightly Build Process with Jenkins, SVN and Nexus
we wanted to set up a nightly integration build with our projects so that we could run unit and integration tests on the latest version of our applications and their underlying libraries. we have a number of libraries that are shared across multiple projects and we wanted this build to run every night and use the latest versions of those libraries even if our applications had a specific release version defined in their maven pom file. in this way we would be alerted early if someone added a change to one of the dependency libraries that could potentially break an application when the developer upgraded the dependent library in a future version of the application. the chart below illustrates our dependencies between our libraries and our applications. updating versions nightly both the crossdock-shared and messaging-shared libraries depend on the siesta framework library. the crossdock web service and crossdockmessaging applications both depend on the crossdock-shared and messaging-shared libraries. because of the dependency structure, we wanted the siestaframework library built first. the crossdock-shared and messaging-shared libraries could be built in parallel, but we didn’t want the builds for the crossdock web service and crossdockmessaging applications to begin until all the libraries had finished building. we also wanted the nightly build to tag a subversion with the build date as well as upload the artifact to our nexus “nightly build” repository. the resulting artifact would look something like siestaframework-20120720.jar also as i had mentioned, even though the crossdockmessaging app may specify in its pom file it depends on version 5.0.4 of the siestaframework library. for the purposes of the nightly build, we wanted it to use the freshly built siestaframework-nightly-20120720.jar version of the library. the first problem to tackle was getting the current date into the project’s version number. for this i started with the jenkins zentimestamp plugin . with this plugin the format of jenkin’s build_id timestamp can be changed. i used this to specify using the format of yyyymmdd for the timestamp. the next step was to get the timestamp into the version number of the project. i was able to accomplish this by using the maven versions plugin. one thing the versions plugin can do is allow you to dynamically override the version number in the pom file at build time. the code snippet from the siestaframework’s pom file is below. org.codehaus.mojo versions-maven-plugin 1.3.1 at this point the jenkins job can be configured to invoke the “versions;set” goal, passing in the new version string to use. the ${build_id} jenkins variable will have the newly formatted date string. this will produce an artifact with the name siestaframework-nightly-20120720.jar uploading artifacts to a nightly repository since this job needed to upload the artifact to a different repository from our release repository that's defined in our project pom files, the “altdeploymentrepository” property was used to pass in the location of the nightly repository. the deployment portion of the siestaframework job specifies the location of the nightly repository where ${lynden_nightly_repo} is a jenkins variable containing the nightly repo url. tagging subversion finally, the jenkins subversion tagging plugin was used to tag svn if the project was successfully built. the plugin provides a post-build action for the job with the configuration section shown below. dynamically updating dependencies so now that the main project is set up, the dependent projects are set up in a similar way, but need to be configured to use the siestaframework-nightly-20120720 of the dependency rather than whatever version they currently have specified in their pom file. this can be accomplished by changing the pom to use a property for the version number of the dependency. for example, if the snippet below was the original pom file— com.lynden siestaframework 5.0.1 —changing it to the following would allow the siestaframework version to be set dynamically: 5.0.1 com.lynden siestaframework ${siesta.version} this version can then be overriden by the jenkins job. the example below shows the jenkins configuration for the crossdock-shared build. enforcing build order the final step in this process is setting up a structure to enforce the build order of the projects. the dependencies are set up in such a way that siestaframework needs to be built first, and the crossdock-shared and messaging-shared libraries can be run concurrently once siestaframework finishes. the crossdock web service and crossdockmessaging application jobs can be run concurrently, too, but not until after both shared libraries have finished. setting up the crossdock-shared and messaging-shared jobs to be built after the siestaframework finishes is pretty straightforward. in the jenkins job configuration for both the shared libraries, the following build trigger is added: to satisfy the requirement that the apps build only after all libraries have built, i enlisted the help of the join plugin . the join plugin can be used to execute a job once all “downstream” jobs have completed. what does this mean exactly? looking at the diagram below, the crossdock-shared and the messaging-shared jobs are “downstream” from the siestaframework job. once both of these jobs complete, a join trigger can be used to start other jobs. in this case, rather than having the join trigger kick off other app jobs directly, i created a dummy join job. in this way, as we add more application builds, we don’t need to keep modifying the siestaframework job with the new application job we just added. to illustrate the configuration, siestaframework has a new post-build action (below): join-build is a jenkins job i configured that does not do anything when executed. then our crossdock web service and crossdockmessaging applications define their builds to trigger as soon as join-build has completed. in this way we are able to run builds each night that will update to the latest version of our dependencies as well as tag svn and archive the binaries to nexus. i’d love to hear feedback from anyone who is handling nightly builds via jenkins, and how they have handled the configuration and build issues.
July 25, 2012
by Rob Terpilowski
· 22,846 Views
article thumbnail
Dependency Convergence in Maven
I was running in to a problem with a Java project that occured only in IntelliJ Idea, but not on the command line, when running specific test classes in Maven. The exception stack trace had the following in it: Caused by: com.sun.jersey.api.container.ContainerException: No WebApplication provider is present That seems like an easy problem to fix - it is the exception message that is given when jersey can’t find the provider for JAX-RS. Fixing it is normally just a matter of making sure jersey-core is on the classpath to fulfill SPI requirements for JAX-RS. For some reason though this isn’t happening in IntelliJ Idea. I inspected the log output of the test run and it is quite clear that all of the jersey dependencies are on the classpath. Then it dawns me on the try running mvn dependency:tree from inside of Idea. Here is what I found: [INFO] +- org.mule.modules:mule-module-jersey:jar:3.2.1:provided [INFO] | +- com.sun.jersey:jersey-server:jar:1.6:provided [INFO] | +- com.sun.jersey:jersey-json:jar:1.6:provided [INFO] | | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:provided [INFO] | | \- org.codehaus.jackson:jackson-xc:jar:1.7.1:provided [INFO] | +- com.sun.jersey:jersey-client:jar:1.6:provided [INFO] | \- org.codehaus.jackson:jackson-jaxrs:jar:1.8.0:provided ... [INFO] +- org.jclouds.driver:jclouds-sshj:jar:1.4.0-rc.3:compile [INFO] | +- org.jclouds:jclouds-compute:jar:1.4.0-rc.3:compile [INFO] | | \- org.jclouds:jclouds-scriptbuilder:jar:1.4.0-rc.3:compile [INFO] | +- org.jclouds:jclouds-core:jar:1.4.0-rc.3:compile [INFO] | | +- net.oauth.core:oauth:jar:20100527:compile [INFO] | | +- com.sun.jersey:jersey-core:jar:1.11:compile [INFO] | | +- com.google.inject.extensions:guice-assistedinject:jar:3.0:compile Notice how I have jersey-core 1.11 coming from jclouds-core but jersey 1.6 everywhere else. That, my friends, is a dependency convergence problem. Maven with its default set of plugins (read: no maven-enforcer-plugin) does not even warn you if something like this happens. In this case, somehow jclouds-core depends directly on jersey-core and happens to resolve the dependency to the version that jclouds-core declared first before jersey-core can be resolved as a transitive dependency on mule-module-jersey. To fix the symptom, all I had to do was add the jersey-core dependency explicitely as a top level dependency in my pom: com.sun.jersey jersey-core ${jersey.version} provided But doing so only fixes the symptom, not the problem. The real problem is that the maven project I’m working on does not presently attempt to detect or resolve dependency convergence problems. This is where the maven-enforcer-plugin comes in handy. You can have the enforcer plugin run the DependencyConvergence rule agaisnt your build and have it fail when you have potential conflicts in your transitive dependencies that you haven’t resolved through exclusions or declaring direct dependencies yet. Binding the maven-enforcer-plugin to your build would look something like this: org.apache.maven.plugins maven-enforcer-plugin 1.0.1 enforce enforce validate ... I chose to bind to the validate phase since that is the first phase to be run in the maven lifecycle. Now my build fails immediately and contains very useful output that looks like the following: Dependency convergence error for org.codehaus.jackson:jackson-jaxrs:1.7.1 paths to dependency are: +-com.nodeable:server:1.0-SNAPSHOT +-org.mule.modules:mule-module-jersey:3.2.1 +-com.sun.jersey:jersey-json:1.6 +-org.codehaus.jackson:jackson-jaxrs:1.7.1 and +-com.nodeable:server:1.0-SNAPSHOT +-org.mule.modules:mule-module-jersey:3.2.1 +-org.codehaus.jackson:jackson-jaxrs:1.8.0 There are many rules you can apply besides DependencyConvergence. However, if the output from the DependencyConvergence rule looks anything like mine does presently, it might take you a while before you get around to getting your maven build to pass and conform to other rules.
July 11, 2012
by Jason Whaley
· 24,853 Views · 1 Like
article thumbnail
Using Cookies to implement a RememberMe functionality
Some web applications may need a "Remember Me" functionality. This means that, after a user login, user will have access from same machine to all its data even after session expired. This access will be possible until user does a logout. If you are using Spring and its login form, then you should use "Remember Me" functionality already implemented inside the framework. Some web frameworks also offer a type of SignIn panel which already has "remember me" built-in. But in case you have to implement "Remember Me" functionality by your own, this can be easily achieved using Cookies. Java has a Cookie class named javax.servlet.http.Cookie. Algorithm is straight-forward: your login panel must contain a "Remember Me" check after a succesfull login with "Remember Me" check selected, you can create two cookies: one to keep the value for rememberMe and one to keep a token which has to identify the logged user. For sake of security, this token must never contain user name or user password. The ideea is to generate a random id as token value. And token value aside with user id must be saved in your storage (database) whenever a login is needed, you have to look if there is any cookie saved by you, and if so and your "rememberMe" value is true, you can take the user from storage based on your token and do an automatic login. when a logout is done, you have to delete the cookie that keeps the token To add a cookie, you have to specify the maximum age of the cookie in seconds : HttpServletResponse servletResponse = ...; Cookie c = new Cookie(COOKIE_NAME, encodeString(uuid)); c.setMaxAge(365 * 24 * 60 * 60); // one year servletResponse.addCookie(c); To delete a cookie, you have to find cookie by name and set its maximum age to 0, before adding it to servlet response: HttpServletRequest servletRequest = ...; HttpServletResponse servletResponse = ... ; Cookie[] cookies = servletRequest.getCookies(); for (int i = 0; i < cookies.length; i++) { Cookie c = cookies[i]; if (c.getName().equals(COOKIE_NAME)) { c.setMaxAge(0); c.setValue(null); servletResponse.addCookie(c); } }
June 26, 2012
by Mihai Dinca - Panaitescu
· 58,965 Views · 1 Like
article thumbnail
TeamCity Build Dependencies
The subject of build dependencies is neither a trivial nor a minor one. Various build tools approach this subject from different perspectives contributing various solutions, each with its own strengths and weaknesses. Maven and Gradle users who are familiar with release and snapshot dependencies may not know about TeamCity snapshot dependencies or assume they’re somehow related to Maven (which isn’t true). TeamCity users who are familiar with artifact and snapshot dependencies may not know that adding an Artifactory plugin allows them to use artifact and build dependencies as well, on top of those provided by TeamCity. Some of the names mentioned above seem not to be established enough while others may require a discussion about their usage patterns. Having this in mind I’ve decided to explore each solution in its own blog post, setting a goal of providing enough information so that people can choose what works best. The first post explored Maven snapshot and release dependencies. This is the second post, which covers artifact and snapshot dependencies provided by TeamCity and the third and final part will cover the artifact and build dependencies provided by TeamCity Artifactory plugin. Non-Maven Dependencies While Maven-based dependencies management and artifact repositories are very common and widespread in Java, there are cases where you may still find them insufficient or inadequate for your needs. For starters, you may not be developing in Java or perhaps your build tool is not providing built-in integration with Maven repositories, as is the case with Ant (or its Gant and NAnt spin-offs), SCons, Rake or MSBuild. Secondly, snapshot Maven dependencies provide their own set of challenges covered in the previous blog post, making it harder to ensure correct snapshot dependency is used in a chain of builds. In order to address these scenarios, TeamCity provides two ways to connect dependent build configurations and their outcomes: artifact and snapshot dependencies. TeamCity Artifact Dependencies The idea of artifact dependencies in TeamCity is very simple: download the artifacts produced by an other build before the current one begins. After the artifacts are downloaded to the folder specified (checkout directory by default), your build script can use them to achieve its goals. You can find configuration details in TeamCity documentation. Naturally, this scheme is not suitable for build tools with automatic dependencies management, but it works well with build or shell scripts accepting and expecting local paths, relative to the checkout directory. Note that the copying works not only for the produced build binaries, but for any kind of binary or text files, like the TeamCity coverage report as demonstrated on the screenshot above. There is one important detail about specifying artifact dependencies and that is “Get artifacts from” configuration where you specify what type of build should files be taken from. Possible values of this field are “last successful”, “finished”, “pinned”, or “tagged build”, as well as the build number or “Build from the same chain”. While most values should be trivial to understand with “Last successful build” being the default and generally suitable option, the definition of “same chain” build is directly related to TeamCity snapshot dependencies. TeamCity Snapshot Dependencies Imagine a monolithic multi-step build process (build, test, package, deploy) which you decide to split into multiple smaller builds, invoked sequentially, forming a chain of executions. Doing so allows one to configure or trigger every chain step separately and run certain steps in parallel in order to speedup the process (like executing tests or building independent components). Most of all, it makes the overall maintenance significantly easier. However, while doing so you need to ensure every chain step uses the same consistent set of sources pulled from VCS even if newer commits are made all the while chain steps are running. That’s what TeamCity snapshot dependencies are for: they connect several build configurations into a single chain of execution, called build chain, with every step using the same set of sources, regardless of VCS updates. Note that the TeamCity use of the term “snapshot dependencies” may confuse people familiar with Maven snapshot dependencies which are two unrelated concepts. Snapshot dependencies are configured similarly to artifact dependencies. You can find configuration details in TeamCity documentation. Using Artifact and Snapshot Dependencies Together When applicable, it is recommended to define both kinds of dependencies between build configurations, as this ensures not only a consistent set of sources used throughout a chain steps but also a consistent flow of artifacts produced. Now the definition of “Build from the same chain” in artifact dependency mentioned above becomes clear, as this is the only meaningful option in this scenario. In a way, you can think of build chain steps running in isolation from VCS updates after the first sources’ “snapshot” is taken. Chain artifacts are either re-created from the same sources or passed through chain steps with artifact dependencies. This makes chain steps consistent, reproducible and always up-to-date (when applied to using chain artifacts), something that can’t be easily achieved with Maven snapshot dependencies. Build Chains Visibility in TeamCity 7.0 TeamCity 7.0 took the notion of build chains to a whole new level by providing build chains a new UI, making chain steps visible and re-runnable. Once you have snapshot dependencies defined, a new “Build Chains” tab appears in project reports, providing a visual representation of all related build chains and a way to re-run any chain step manually, using the same set of sources pulled originally. Build Chain Triggering Having build configurations connected with snapshot dependencies and, therefore, their builds grouped into build chains not only makes them more consistent regarding the sources used, it also impacts the way builds are added to the build queue: after a certain chain step is triggered, the default behavior is to add all preceding chain steps as well, keeping their respective order, in addition to the one that was triggered initially. Let me repeat it for more clarity: triggering certain chain configuration adds preceding (those to the left of it) and not subsequent (to the right of it) configurations to the build queue, although it may seem counterintuitive at first. The idea is to mark the location where chain execution stops, which is exactly the configuration that was triggered initially; it becomes the last execution step. To trigger subsequent chain steps upon VCS changes found in a chain configuration, you can add a VCS trigger with the “Trigger on changes in snapshot dependencies” option to the configuration that would be the last execution step. This configuration is then triggered whenever any of the preceding chain steps is updated, which schedules the whole chain for execution. Having this behavior in mind, you therefore need to decide which configurations are triggered automatically and which should be run manually. Usually, earlier chain steps having no impact on external environment can be triggered automatically by VCS trigger but final chain steps, potentially modifying external systems, are invoked manually after a human verification of the previous chain results. The process of running the final chain steps manually is usually referred to as “promoting” previously finished builds. Sample Build Chain: Compile, Test, Deploy Imagine three sample build configurations, "Compile", "Test" and "Deploy" connected into a build chain: "Deploy" is snapshot dependent on "Test" which is snapshot dependent on "Compile". In this sample scenario the "Compile" and "Test" configurations are triggered automatically while "Deploy" is triggered manually, following the recommendations given above. VCS changes in "Compile" configuration only trigger an execution of this chain step, while VCS changes in "Test" configuration trigger "Compile" and "Test" execution (in that order). Once a "Compile" configuration is added to the builds queue, its sources’ timestamp is recorded on the server to be used in all subsequent chain steps. If any of the chain steps is connected to a different VCS root, its sources are also pulled according to the same timestamp. Promoting Finished Builds As soon as the automatic chain execution stops (after running "Test"), you can continue it by clicking the corresponding “Run” button on the "Deploy" configuration that was not triggered (see the build chain screenshot above). Alternatively, it is possible to promote a finished "Test" build through its “Build Actions” and invoke configurations which are snapshot dependent on it – "Deploy" configuration in this case. Summary This article has provided an overview of TeamCity artifact and snapshot dependencies, build chains, how their steps are triggered and how finished builds are promoted. I hope you now have a good understanding of how it works and of when it is appropriate (or not) to use TeamCity build dependencies in addition to those provided by build tools such as Maven. Please, refer to the TeamCity documentation for more information about this subject: Dependent Build Build Chain The final blog post in the series will uncover how you can use the TeamCity Artifactory plugin in order to achieve a behavior which is similar to build chains for projects with Maven-based dependency management. Stay tuned!
May 11, 2012
by Evgeny Goldin
· 19,478 Views · 2 Likes
article thumbnail
Managing and Monitoring Drupal Sites on Windows Azure
A few weeks ago, I co-authored an article (with my colleague Rama Ramani) about how the Screen Actors Guild Awards website migrated its Drupal deployment from LAMP to Windows Azure: Azure Real World: Migrating a Drupal Site from LAMP to Windows Azure. Since then, Rama and another colleague, Jason Roth, have been working on writing up how the SAG Awards website was managed and monitored in Windows Azure. The article below is the fruit of their work…a very interesting/educational read. Overview Drupal is an open source content management system that runs on PHP. Windows Azure offers a flexible platform for hosting, managing, and scaling Drupal deployments. This paper focuses on an approach to host Drupal sites on Windows Azure, based on learning from a BPD Customer Programs Design Win engagement with the Screen Actors Guild Awards Drupal website. This paper covers guidelines and best practices for managing an existing Drupal web site in Windows Azure. For more information on how to migrate Drupal applications to Windows Azure, see Azure Real World: Migrating a Drupal Site from LAMP to Windows Azure. The target audience for this paper is Drupal administrators who have some exposure to Windows Azure. More detailed pointers to Windows Azure content is provided throughout the paper as links. Drupal Application Architecture on Windows Azure Before reviewing the management and monitoring guidelines, it is important to understand the architecture of a typical Drupal deployment on Windows Azure. First, the following diagram displays the basic architecture of Drupal running on Windows and IIS7. In the Windows Server scenario, you could have one or more machines hosting the web site in a farm. Those machines would either persist the site content to the file system or point to other network shares. For Windows Azure, the basic architecture is the same, but there are some differences. In Windows Azure the site is hosted on a web role. A web role instance is hosted on a Windows Server 2008 virtual machine within the Windows Azure datacenter. Like the web farm, you can have multiple instances running the site. But there is no persistence guarantee for the data on the file system. Because of this, much of the shared site content should be stored in Windows Azure Blob storage. This allows them to be highly available and durable. Usually, a large portion of the site caters to static content which lends well to caching. And caching can be applied in a set of places – browser level caching, CDN to cache content in the edge closer to the browser clients, caching in Azure to reduce the load on backend, etc. Finally, the database can be located in SQL Azure. The following diagram shows these differences. For monitoring and management, we will look at Drupal on Windows Azure from three perspectives: Availability: Ensure the web site does not go down and that all tiers are setup correctly. Apply best practices to ensure that the site is deployed across data centers and perform backup operations regularly. Scalability: Correctly handle changes in user load. Understand the performance characteristics of the site. Manageability: Correctly handle updates. Make code and site changes with no downtime when possible. Although some management tasks span one or more of these categories, it is still helpful to discuss Drupal management on Windows Azure within these focus areas. Availability One main goal is that the Drupal site remains running and accessible to all end-users. This involves monitoring both the site and the SQL Azure database that the site depends on. In this section, we will briefly look at monitoring and backup tasks. Other crossover areas that affect availability will be discussed in the next section on scalability. Monitoring With any application, monitoring plays an important role with managing availability. Monitoring data can reveal whether users are successfully using the site or whether computing resources are meeting the demand. Other data reveals error counts and possibly points to issues in a specific tier of the deployment. There are several monitoring tools that can be used. The Windows Azure Management Portal. Windows Azure diagnostic data. Custom monitoring scripts. System Center Operations Manager. Third party tools such as Azure Diagnostics Manager and Azure Storage Explorer. The Windows Azure Management Portal can be used to ensure that your deployments are successful and running. You can also use the portal to manage features such as Remote Desktop so that you can directly connect to machines that are running the Drupal site. Windows Azure diagnostics allows you to collect performance counters and logs off of the web role instances that are running the Drupal site. Although there are many options for configuring diagnostics in Azure, the best solution with Drupal is to use a diagnostics configuration file. The following configuration file demonstrates some basic performance counters that can monitor resources such as memory, processor utilization, and network bandwidth. For more information about setting up diagnostic configuration files, see How to Use the Windows Azure Diagnostics Configuration File. This information is stored locally on each role instance and then transferred to Windows Azure storage per a defined schedule or on-demand. See Getting Started with Storing and Viewing Diagnostic Data in Windows Azure Storage. Various monitoring tools, such as Azure Diagnostics Manager, help you to more easily analyze diagnostic data. Monitoring the performance of the machines hosting the Drupal site is only part of the story. In order to plan properly for both availability and scalability, you should also monitor site traffic, including user load patterns and trends. Standard and custom diagnostic data could contribute to this, but there are also third-party tools that monitor web traffic. For example, if you know that spikes occur in your application during certain days of the week, you could make changes to the application to handle the additional load and increase the availability of the Drupal solution. Backup Tasks To remain highly available, it is important to backup your data as a defense-in-depth strategy for disaster recovery. This is true even though SQL Azure and Windows Azure Storage both implement redundancy to prevent data loss. One obvious reason is that these services cannot prevent administrator error if data is accidentally deleted or incorrectly changed. SQL Azure does not currently have a formal backup technology, although there are many third-party tools and solutions that provide this capability. Usually the database size for a Drupal site is relatively small. In the case of SAG Awards, it was only ~100-150 MB. So performing an entire backup using any strategy was relatively fast. If your database is much larger, you might have to test various backup strategies to find the one that works best. Apart from third-party SQL Azure backup solutions, there are several strategies for obtaining a backup of your data: · Use the Drush tool and the portabledb-export command. · Periodically copy the database using the CREATE DATABASE Transact-SQL command. · Use Data-tier applications (DAC) to assist with backup and restore of the database. SQL Azure backup and data security techniques are described in more detail in the topic, Business Continuity in SQL Azure. Note that bandwidth costs accrue with any backup operation that transfers information outside of the Windows Azure datacenter. To reduce costs, you can copy the database to a database within the same datacenter. Or you can export the data-tier applications to blob storage in the same datacenter. Another potential backup task involves the files in Blob storage. If you keep a master copy of all media files uploaded to Blob storage, then you already have an on-premises backup of those files. However, if multiple administrators are loading files into Blob storage for use on the Drupal site, it is a good idea to enumerate the storage account and to download any new files to a central location. The following PHP script demonstrates how this can be done by backing up all files in Blob storage after a specified modification date. setProxy(true, 'YOUR_PROXY_IF_NEEDED', 80); $blobs = (array)$blobObj->listBlobs(AZURE_STORAGE_CONTAINER, '', '', 35000); backupBlobs($blobs, $blobObj); function backupBlobs($blobs, $blobObj) { foreach ($blobs as $blob) { if (strtotime($blob->lastmodified) >= DEFAULT_BACKUP_FROM_DATE && strtotime($blob->lastmodified) <= DEFAULT_BACKUP_TO_DATE) { $path = pathinfo($blob->name); if ($path['basename'] != '$$$.$$$') { $dir = $path['dirname']; $oldDir = getcwd(); if (handleDirectory($dir)) { chdir($dir); $blobObj->getBlob( AZURE_STORAGE_CONTAINER, $blob->name, $path['basename'] ); chdir($oldDir); } } } } } function handleDirectory($dir) { if (!checkDirExists($dir)) { return mkdir($dir, 0755, true); } return true; } function checkDirExists($dir) { if(file_exists($dir) && is_dir($dir)) { return true; } return false; } ?> This script has a dependency on the Windows Azure SDK for PHP. Also note there are several parameters that you must modify such as the storage account, secret, and backup location. As with SQL Azure, bandwidth and transaction charges apply to a backup script like this. Scalability Drupal sites on Windows Azure can scale as load increased through typical strategies of scale-up, scale-out, and caching. The following sections describe the specifics of how these strategies are implemented in Windows Azure. Typically you make scalability decisions based on monitoring and capacity planning. Monitoring can be done in staging during testing or in production with real-time load. Capacity planning factors in projections for changes in user demand. Scale Up When you configure your web role prior to deployment, you have the option of specifying the Virtual Machine (VM) size, such as Small or ExtraLarge. Each size tier adds additional memory, processing power, and network bandwidth to each instance of your web role. For cost efficiency and smaller units of scale, you can test your application under expected load to find the smallest virtual machine size that meets your requirements. The workload usually in most popular Drupal websites can be separated out into a limited set of Drupal admins making content changes and a large user base who perform mostly read-only workload. End users can be allowed to make ‘writes’, such as uploading blogs or posting in forums, but those changes are not ‘content changes’. Drupal admins are setup to operate without caching so that the writes are made directly to SQL Azure or the corresponding backend database. This workload performs well with Large or ExtraLarge VM sizes. Also, note that the VM size is closely tied to all hardware resources, so if there are many content-rich pages that are streaming content, then the VM size requirements are higher. To make changes to the Virtual Machine size setting, you must change the vmsize attribute of the WebRole element in the service definition file, ServiceDefinition.csdef. A virtual machine size change requires existing applications to be redeployed. Scale Out In addition to the size of each web role instance, you can increase or decrease the number of instances that are running the Drupal site. This spreads the web requests across more servers, enabling the site to handle more users. To change the number of running instances of your web role, see How to Scale Applications by Increasing or Decreasing the Number of Role Instances. Note that some configuration changes can cause your existing web role instances to recycle. You can choose to handle this situation by applying the configuration change and continue running. This is done by handling the RoleEnvironment.Changing event. For more information see, How to Use the RoleEnvironment.Changing Event. A common question for any Windows Azure solution is whether there is some type of built-in automatic scaling. Windows Azure does not provide a service that provides auto-scaling. However, it is possible to create a custom solution that scales Azure services using the Service Management API. For an example of this approach, see An Auto-Scaling Module for PHP Applications in Windows Azure. Caching Caching is an important strategy for scaling Drupal applications on Windows Azure. One reason for this is that SQL Azure implements throttling mechanisms to regulate the load on any one database in the cloud. Code that uses SQL Azure should have robust error handling and retry logic to account for this. For more information, see Error Messages (SQL Azure Database). Because of the potential for load-related throttling as well as for general performance improvement, it is strongly recommended to use caching. Although Windows Azure provides a Caching service, this service does not currently have interoperability with PHP. Because of this, the best solution for caching in Drupal is to use a module that uses an open-source caching technology, such as Memcached. Outside of a specific Drupal module, you can also configure Memcached to work in PHP for Windows Azure. For more information, see Running Memcached on Windows Azure for PHP. Here is also an example of how to get Memcached working in Windows Azure using a plugin: Windows Azure Memcached plugin. In a future paper, we hope to cover this architecture in more detail. For now, here are several design and management considerations related to caching. Area Consideration Design and Implementation For a technology like Memcached, will the cache be collocated (spread across all web role instances)? Or will you attempt to setup a dedicated cache ring with worker roles that only run Memcached? Configuration What memory is required and how will items in the cache be invalidated? Performance and Monitoring What mechanisms will be used to detect the performance and overall health of the cache? For ease of use and cost savings, collocation of the cache across the web role instances of the Drupal site works best. However, this assumes that there is available reserve memory on each instance to apply toward caching. It is possible to increase the virtual machine size setting to increase the amount of available memory on each machine. It is also possible to add additional web role instances to add to the overall memory of the cache while at the same time improving the ability of the web site to respond to load. It is possible to create a dedicated cache cluster in the cloud, but the steps for this are beyond the scope of this paper[RR1] . For Windows Azure Blob storage, there is also a caching feature built into the service called the Content Delivery Network (CDN). CDN provides high-bandwidth access to files in Blob storage by caching copies of the files in edge nodes around the world. Even within a single geographic region, you could see performance improvements as there are many more edge nodes than Windows Azure datacenters. For more information, see Delivering High-Bandwidth Content with the Windows Azure CDN. Manageability It is important to note that each hosted service has a Staging environment and a Production environment. This can be used to manage deployments, because you can load and test and application in staging before performing a VIP swap with production. From a manageability standpoint, Drupal has an advantage on Windows Azure in the way that site content is stored. Because the data necessary to serve pages is stored in the database and blob storage, there is no need to redeploy the application to change the content of the site. Another best practice is to use a separate storage account for diagnostic data than the one that is used for the application itself. This can improve performance and also helps to separate the cost of diagnostic monitoring from the cost of the running application. As mentioned previously, there are several tools that can assist with managing Windows Azure applications. The following table summarizes a few of these choices. Tool Description Windows Azure Management Portal The web interface of the Windows Azure management portal shows deployments, instance counts and properties, and supports many different common management and monitoring tasks. Azure Diagnostics Managerq[RR2] [JR3] A Red Gate Software product that provides advanced monitoring and management of diagnostic data. This tool can be very useful for easily analyzing the performance of the Drupal site to determine appropriate scaling decisions. Azure Storage Explorer A tool created by Neudesic for viewing Windows Azure storage account. This can be useful for viewing both diagnostic data and the files in Blob storage.
April 25, 2012
by Brian Swan
· 8,752 Views
article thumbnail
You've Been Implementing main() Wrong All This Time
Since the very early days of Java (and C-like languages overall), the canonical way to start your program has been something like this: public class A { public static void main(String[] args) { new A().run(args); } public void run(String[] args) { // Your application starts here } } If you are still doing this, I’m here to tell you it’s time to stop. Letting go of ‘new’ First, install Guice in your project: com.google.inject guice 3.0 and then, modify your main method as follows: public class A { public static void main(String[] args) { Injector.getInstance(A.class).run(args); } } So, what does this buy you exactly? You will find a lot of articles explaining the various benefits of Guice, such as being able to substitute different environments on the fly, but I’m going to use a different angle in this article. Let’s start by assuming the existence of a Config class that contains various configuration parameters. I’ll just hardcode them for now and use fields to make the class smaller: public class Config { String host = "com.example.com"; int port = 1234; } This class is a singleton, it is instantiated somewhere in your main class and not used anywhere else at the moment. One day, you realize you need this instance in another class which happens to be deep in your runtime hierarchy, which we will call Deep. For example, if you put a break point in the method where you need this config object, your debugger would show you stack frames similar to this: com.example.A.main() com.example.B.f(int, String) com.example.C.g(String) com.example.Deep.h(Foo, int) The easy and wrong way to solve this problem is to make the Config instance static on some class (probably A) and access it directly from Deep. I’m hoping I don’t need to explain why this is a bad idea: not only do you want to avoid using statics, but you also want to make sure that each object is exposed only to objects that need them, and making the Config object static would make your instance visible to your entire code base. Not a good thing. The second thought is to pass the object down the stack, so you modify all the signatures as follows: com.example.A.main() com.example.B.f(int, String, Config) com.example.C.g(String, Config) com.example.Deep.h(Foo, int, Config) This is a bit better since you have severely restricted the exposure of the Config object, but note that you are still making it available to more methods than really need to: B#f and C#g have really nothing to do with this object, and a little sting of discomfort hits you when you start writing the Javadoc: public class C { ... /** * @param config This method doesn't really use this parameter, * it just passes it down so Deep#h can use it. */ public void g(String s, Config config) { Unnecessary exposure is actually not the worst part of this approach, the problem is that it changes all these signatures along the way, which is certainly undesirable in a private API and absolutely devastating in a public API. And of course, it’s absolutely not scalable: if you keep adding a parameter to your method whenever you need access to a certain object, you will soon be dealing with methods that take ten parameters, most of which they just pass down the chain. Here is how we solve this problem with dependency injection (performed by Guice in this example, but this is applicable to any library that implements JSR 330, obviously): public class Deep { @Inject private Config config; and we’re done. That’s it. You don’t need to modify the Config class in any way, nor do you need to make any change in any of the classes that separate Deep from your main class. With this, you have also minimized the exposure of the Config object to just the class that needs it. Injecting right There are various ways you can inject object into your class but I’ll just mention the two that, I think, are the most important. I just showed “field injection” in the previous paragraph, but be aware that you can also prefer to use “constructor injection”: public class Deep { private final Config config; @Inject public Deep(Config config) { this.config = config; } This time, you are adding a parameter to the constructor of your Deep class (which shouldn’t worry you too much since you will never invoke it directly, Guice will) and you assign the parameter to the field in the constructor. The benefit is that you can declare your field final. The downside, obviously, is that this approach is much more verbose. Personally, I see little point in final fields since I have hardly ever encountered a bug that was due to accidentally reassigning a field, so I tend to use field injection whenever I can. Taking it to the next level Obviously, the kind of configuration object I used as an example if not very realistic. Typically, a configuration will not hardcode values like I did and will, instead, read them from some external source. Similarly, you will want to inject objects that can’t necessarily be instantiated so early in the lifecycle of your application, such as servlet contexts, database connections, or implementations of your own interfaces. This topic itself would probably cover several chapters of a book dedicated to dependency injection, so I’ll just summarize it: not all objects can be injected this way, and one benefit of using a dependency injection framework in your code is that it will force you to think about what life cycle category your objects belong to. Having said that, if you want to find out how Guice can inject objects that get created at a later time in your application life cycle, look up the Javadoc for the Provider class. Wrapping up I hope this quick introduction to dependency injection piqued your interest and that you will consider using it in your project since it has so much more to offer than what I described in this post. If you want to learn more, I suggest starting with the excellent Guice documentation.
March 26, 2012
by Cedric Beust
· 13,370 Views
article thumbnail
How to Secure an Apache Thrift Service
Make sure your Apache Thrift services are secure with this awesome tutorial.
February 13, 2012
by Buddhika Chamith
· 20,679 Views
article thumbnail
Visualize Maven Project Dependencies with dependency:tree and Dot Diagram Output
The dependency:tree goal of the Maven plugin dependency supports various graphical outputs from the version 2.4 up. This is how you would create a diagram showing all dependencies in the com.example group in the dot format: mvn dependency:tree -Dincludes=com.example-DappendOutput=true -DoutputType=dot -DappendOutput=true -DoutputFile=/path/to/output.dot To actually produce an image from .dot you can use one of .dot renderers, f.ex. this online dot renderer (paste into the right text box, press enter). You could also generate the output f.ex. in the graphml format & visualize it in Eclipse. From http://theholyjava.wordpress.com/2012/01/13/visualize-maven-project-dependencies-with-dependencytree-and-dot-diagram-output/
January 25, 2012
by Jakub Holý
· 31,645 Views
article thumbnail
Simplifying the Data Access Layer with Spring and Java Generics
1. Overview This is the second of a series of articles about Persistence with Spring. The previous article discussed setting up the persistence layer with Spring 3.1 and Hibernate, without using templates. This article will focus on simplifying the Data Access Layer by using a single, generified DAO, which will result in elegant data access, with no unnecessary clutter. Yes, in Java. The Persistence with Spring series: Part 1 – The Persistence Layer with Spring 3.1 and Hibernate Part 3 – The Persistence Layer with Spring 3.1 and JPA Part 4 – The Persistence Layer with Spring Data JPA Part 5 – Transaction configuration with JPA and Spring 3.1 2. The DAO mess Most production codebases have some kind of DAO layer. Usually the implementation ranges from a raw class with no inheritance to some kind of generified class, but one thing is consistent – there is always more then one. Most likely, there are as many DAOs as there are entities in the system. Also, depending on the level of generics involved, the actual implementations can vary from heavily duplicated code to almost empty, with the bulk of the logic grouped in an abstract class. 2.1. A Generic DAO Instead of having multiple implementations – one for each entity in the system – a single parametrized DAO can be used in such a way that it still takes full advantage of the type safety provided by generics. Two implementations of this concept are presented next, one for a Hibernate centric persistence layer and the other focusing on JPA. These implementation are by no means complete – only some data access methods are included, but they can be easily be made more thorough. 2.2. The Abstract Hibernate DAO public abstract class AbstractHibernateDAO< T extends Serializable > { private Class< T > clazz; @Autowired SessionFactory sessionFactory; public void setClazz( Class< T > clazzToSet ){ this.clazz = clazzToSet; } public T findOne( Long id ){ return (T) this.getCurrentSession().get( this.clazz, id ); } public List< T > findAll(){ return this.getCurrentSession() .createQuery( "from " + this.clazz.getName() ).list(); } public void save( T entity ){ this.getCurrentSession().persist( entity ); } public void update( T entity ){ this.getCurrentSession().merge( entity ); } public void delete( T entity ){ this.getCurrentSession().delete( entity ); } public void deleteById( Long entityId ){ T entity = this.getById( entityId ); this.delete( entity ); } protected Session getCurrentSession(){ return this.sessionFactory.getCurrentSession(); } } The DAO uses the Hibernate API directly, without relying on any Spring templates (such as HibernateTemplate). Using of templates, as well as management of the SessionFactory which is autowired in the DAO were covered in the previous post of the series. 2.3. The Abstract JPA DAO public abstract class AbstractJpaDAO< T extends Serializable > { private Class< T > clazz; @PersistenceContext EntityManager entityManager; public void setClazz( Class< T > clazzToSet ){ this.clazz = clazzToSet; } public T findOne( Long id ){ return this.entityManager.find( this.clazz, id ); } public List< T > findAll(){ return this.entityManager.createQuery( "from " + this.clazz.getName() ) .getResultList(); } public void save( T entity ){ this.entityManager.persist( entity ); } public void update( T entity ){ this.entityManager.merge( entity ); } public void delete( T entity ){ this.entityManager.remove( entity ); } public void deleteById( Long entityId ){ T entity = this.getById( entityId ); this.delete( entity ); } } Similar to the Hibernate DAO implementation, the Java Persistence API is used here directly, again not relying on the now deprecated Spring JpaTemplate. 2.4. The Generic DAO Now, the actual implementation of the generic DAO is as simple as it can be – it contains no logic. Its only purpose is to be injected by the Spring container in a service layer (or in whatever other type of client of the Data Access Layer): @Repository @Scope( BeanDefinition.SCOPE_PROTOTYPE ) public class GenericJpaDAO< T extends Serializable > extends AbstractJpaDAO< T > implements IGenericDAO< T >{ // } @Repository @Scope( BeanDefinition.SCOPE_PROTOTYPE ) public class GenericHibernateDAO< T extends Serializable > extends AbstractHibernateDAO< T > implements IGenericDAO< T >{ // } First, note that the generic implementation is itself parametrized – allowing the client to choose the correct parameter in a case by case basis. This will mean that the clients gets all the benefits of type safety without needing to create multiple artifacts for each entity. Second, notice the prototype scope of these generic DAO implementation. Using this scope means that the Spring container will create a new instance of the DAO each time it is requested (including on autowiring). That will allow a service to use multiple DAOs with different parameters for different entities, as needed. The reason this scope is so important is due to the way Spring initializes beans in the container. Leaving the generic DAO without a scope would mean using the default singleton scope, which would lead to a single instance of the DAO living in the container. That would obviously be majorly restrictive for any kind of more complex scenario. 3. The Service There is now a single DAO to be injected by Spring; also, the Class needs to be specified: @Service class FooService implements IFooService{ IGenericDAO< Foo > dao; @Autowired public void setDao( IGenericDAO< Foo > daoToSet ){ this.dao = daoToSet; this.dao.setClazz( Foo.class ); } // ... } Spring autowires the new DAO insteince using setter injection so that the implementation can be customized with the Class object. After this point, the DAO is fully parametrized and ready to be used by the service. 4. Conclusion This article discussed the simplification of the Data Access Layer by providing a single, reusable implementation of a generic DAO. This implementation was presented in both a Hibernate and a JPA based environment. The result is a streamlined persistence layer, with no unnecessary clutter. For a step by step introduction about setting up the Spring context using Java based configuration and the basic Maven pom for the project, see this article. The next article of the Persistence with Spring series will focus on setting up the DAL layer with Spring 3.1 and JPA. In the meantime, you can check out the full implementation in the github project. If you read this far, you should follow me on twitter here.
January 5, 2012
by Eugen Paraschiv
· 25,035 Views · 1 Like
article thumbnail
Maven's WAR Overlays: How to manage dependencies
If you're stumbling on this post looking for a way to manage dependencies with WAR overlays, please make sure you check out part one of this blog posting to get the background about how to apply and use WAR overlays in your POM file. This post picks up from the last post and it's assumed that you understand how to apply a WAR to an existing master project. Quick recap Why a quick recap? We're going to go into depth on how dependencies and libraries are applied to the master project. In order to have a WAR overlay, you must include the WAR as a dependency to the master project and must be overlayed during the war-plugin package phase. Where do the dependency libraries go? As mentioned in part one of this blog posting, files are placed in their parent's locations if it exists. What this means is that if, in your overlay WAR you have a .jsp file in /WEB-INF/welcome.jsp, the master WAR will then have a file named "welcome.jsp" in the /WEB-INF. Furthermore, if there is already a file named "welcome.jsp", that file will not be overwritten. If we extrapolate this a bit more, the libraries in the /WEB-INF/lib folder from the overlay will be placed inside the /WEB-INF/lib folder of the master project. This is great and all except when different versions of libraries exist causing runtime failures. Good news, there are ways to manage the dependencies in both the overlay WAR and the master project. How to see your main artifact's libraries? In order to determine what libraries are in which artifact, there are two simple ways. First, determine what libraries are in your master WAR. mvn clean package ls -al target/YOUR_ARTIFACT_WAR/WEB-INF/lib NOTE: adding "-DskipTests" when added to the above maven command would skip tests and would make this step a bit faster, though, it is strictly optional Secondly, add the "workDirectory" parameter to the configuration portion of your war-plugin when applying the overlay. Example: target/extract_here In your master project's POM file: ... ... org.apache.maven.plugins maven-war-plugin 2.1.1 target/overlay-war-folder com.yourcompany branded-templates ... ... This additional property will place each overlay's exploded contents into the specified folder to help debug library conflicts. How to exclude dependencies/libraries at build time? It can be a pain to manage the dependencies between each of the overlays, especially if the overlay is not under your development control. One method would be to manage the dependencies within the two POM files by utilizing the "provided" and "optional" dependencies that maven provides. However, if the overlay project can also stand as stand-alone project, this method will not work because the libraries are not considered to be provided or optional when running independently. This is common when trying to test each of the overlays independently. An additional option to control the libraries is to exclude them from the master WAR during the package phase. This is achieved by using the "excludes" parameter in each of the overlay sections. For example, to remove all of the spring libraries, you could exclude "spring*.jar" ... ... org.apache.maven.plugins maven-war-plugin 2.1.1 target/overlay-war-folder com.yourcompany branded-templates WEB-INF/lib/spring*.jar WEB-INF/lib/log4j*.jar ... ... You might start asking yourself, this is going to get complicated really quickly, and you're right, there are other options. Skinny vs Fat WAR Overlays Up until now, all of our overlays have been considered "fat" because they contain all of it's dependencies which get overlaid onto the master WAR file. There is an alternative that is more commonly known as the "skinny WAR". A skinny WAR is a combination between most of this post's tactics. NOTE: The method I am showing you is known as a "skinnier WAR" because the dependency libraries do exist in the overlay's WAR file, however, the classes will be externalized and the libraries can be removed during final packaging of the master project. Achieving the "skinny WAR" First, in the overlay project, during the package phase, tell maven to build the project's classes as an extra artifact in addition to the WAR file. This set of classes will become a jar file which will be used as a second dependency in the master project's POM. In the Overlay's POM file (add to the existing war plugin): ... ... org.apache.maven.plugins maven-war-plugin 2.1.1 true ... ... After successfully installing/deploying the overlay project, we will add the classes/jar file and the WAR dependency to the master project's POM file. In the master project's POM file: com.yourcompany branded-templates war ... ... com.yourcompany branded-templates 1.1 war com.yourcompany branded-templates 1.1 jar classes ... ... And finally, combine the excludes learned in the last example section above to exclude all .jar files from the overlay WAR. ... ... org.apache.maven.plugins maven-war-plugin 2.1.1 target/overlay-war-folder com.yourcompany branded-templates WEB-INF/lib/*.jar ... ... NOTE: Dependencies that exist in the Overlay will need to be added as dependencies to the master project because they have been removed from the overlay's /WEB-INF/lib folder. This is the main known risk of using skinny war overlays Congratulations! You now have a project with limited library conflicts and can better manager the main project and it's stability. Thank you for reading part one and this post, part two. Please feel free too leave comments or questions and I will do my best to answer them in a timely manner. From http://www.ensor.cc/2011/07/mavens-war-overlays-how-to-manage.html
December 29, 2011
by Mike Ensor
· 34,146 Views · 1 Like
article thumbnail
Basic and Digest authentication for a RESTful Service with Spring Security 3.1, part 6
This is the sixth of a series of articles about setting up a secure RESTful Web Service using Spring 3.1 and Spring Security 3.1. A previous article introduced security in the context of a RESTful service, using form-based authentication. This article will focus on configuration of Basic and Digest authentication and on configuring both protocols for the same URI mapping of the API, using Spring Security 3.1. The REST with Spring series: Part 1 – Bootstrapping a web application with Spring 3.1 and Java based Configuration Part 2 – Building a RESTful Web Service with Spring 3.1 and Java based Configuration Part 3 – Securing a RESTful Web Service with Spring Security 3.1 Part 4 – RESTful Web Service Discoverability Part 5 – REST Service Discoverability with Spring Configuration of Basic Authentication In part 3 of the series, the Spring Security configuration was done using form based authentication, which is not really ideal for a RESTful service. To start setting up basic authentication, first we remove the old custom entry point and filter from the main security element: Note how support for basic authentication has been added with a single configuration line – – which handles the creation and wiring of both the BasicAuthenticationFilter and the BasicAuthenticationEntryPoint. Satisfying the stateless constraint – getting rid of sessions One of the main constraints of the RESTful architectural style is that the client-server communication is fully stateless, as the original dissertation reads: 5.1.3 Stateless We next add a constraint to the client-server interaction: communication must be stateless in nature, as in the client-stateless-server (CSS) style of Section 3.4.3 (Figure 5-3), such that each request from client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server. Session state is therefore kept entirely on the client. The concept of Session on the server is one with a long history in Spring Security, and removing it entirely has been difficult until now, especially when configuration was done by using the namespace. However, Spring Security 3.1 augments the namespace configuration with a new stateless option for session creation, which effectively guarantees that no session will be created or used by Spring. What this new option does is completely removes all session related filters from the security filter chain, ensuring that authentication is performed for each request. Configuration of Digest Authentication Starting with the previous configuration, the filter and entry point necessary to set up digest authentication will be defined as beans. Then, the digest entry point will override the one created by behind the scenes. Finally, the custom digest filter will be introduced in the security filter chain using the after semantics of the security namespace to position it directly after the basic authentication filter. Unfortunately there is no support in the security namespace to automatically configure the digest authentication the way basic authentication can be configured with . Because of that, the necessary beans had to be defined and wired manually into the security configuration. Supporting both authentication protocols in the same RESTful service Basic or Digest authentication alone can be easily implemented in Spring Security 3.x; it is supporting both of them for the same RESTful web service, on the same URI mappings that introduces a new level of complexity into the configuration and testing of the service. Anonymous request With both basic and digest filters in the security chain, the way a anonymous request – a request containing no authentication credentials (Authorization HTTP header) – is processed by Spring Security is – the two authentication filters will find no credentials and will continue execution of the filter chain. Then, seeing how the request wasn’t authenticated, an AccessDeniedException is thrown and caught in the ExceptionTranslationFilter, which commences the digest entry point, prompting the client for credentials. The responsibilities of both the basic and digest filters are very narrow – they will continue to execute the security filter chain if they are unable to identify the type of authentication credentials in the request. It is because of this that Spring Security can have the flexibility to be configured with support for multiple authentication protocols on the same URI. When a request is made containing the correct authentication credentials – either basic or digest – that protocol will be rightly used. However, for an anonymous request, the client will get prompted only for digest authentication credentials. This is because the digest entry point is configured as the main and single entry point of the Spring Security chain; as such digest authentication can be considered the default. Request with authentication credentials A request with credentials for Basic authentication will be identified by the Authorization header starting with the prefix “Basic”. When processing such a request, the credentials will be decoded in the basic authentication filter and the request will be authorized. Similarly, a request with credentials for Digest authentication will use the prefix “Digest” for it’s Authorization header. Testing both scenarios The tests will consume the REST service by creating a new resource after authenticating with either basic or digest: @Test public void givenAuthenticatedByBasicAuth_whenAResourceIsCreated_then201IsReceived(){ // Given // When Response response = given() .auth().preemptive().basic( ADMIN_USERNAME, ADMIN_PASSWORD ) .contentType( HttpConstants.MIME_JSON ).body( new Foo( randomAlphabetic( 6 ) ) ) .post( this.paths.getFooURL() ); // Then assertThat( response.getStatusCode(), is( 201 ) ); } @Test public void givenAuthenticatedByDigestAuth_whenAResourceIsCreated_then201IsReceived(){ // Given // When Response response = given() .auth().digest( ADMIN_USERNAME, ADMIN_PASSWORD ) .contentType( HttpConstants.MIME_JSON ).body( new Foo( randomAlphabetic( 6 ) ) ) .post( this.paths.getFooURL() ); // Then assertThat( response.getStatusCode(), is( 201 ) ); } Note that the test using basic authentication adds credentials to the request preemptively, regardless if the server has challenged for authentication or not. This is to ensure that the server doesn’t need to challenge the client for credentials, because if it did, the challenge would be for Digest credentials, since that is the default. Conclusion This article covered the configuration and implementation of both Basic and Digest authentication for a RESTful service, using mostly Spring Security 3.0 namespace support as well as some new features added by Spring Security 3.1. In the next articles I will focus on OAuth authentication. In the meantime, check out the github project. From the REST with Spring series.
December 15, 2011
by Eugen Paraschiv
· 42,137 Views · 2 Likes
article thumbnail
Tackling the Circular Dependency in Java...
Let me first define what we mean by circular dependency in OOAD terms vis-a-vis Java. Suppose we have a class called A which has class B’s Object. (in UML terms A HAS B). at the same time we class B is also composed of Object of class A (in UML terms B HAS A). obviously this represents circular dependency because while creating the object of A, the compiler must know the size of B... on the other hand while creating object of B, the compiler must know the size of A. this is something like egg vs. chicken problem... this may be possible in real life situation as well. for example suppose a multi storied building has a lift. so in the UML terms, the building HAS lift... but at the same time, suppose, while constructing the lift object, we need to give it the information about the building object to access various functionalities of the Building class... for example, suppose the speed of the lift is set depending on the number of floors of the Building... hence while constructing the Lift object it must access the functionalities of the Building object which will give the number of floors the building has got...hence in UML terms the lift HAS building... so this is a sure case of circular dependency... in real java code it will look something as follows: public class Building { private Lift lift; private int floor; public Building(){ lift = new Lift(); setFloor(15); } public int getFloor(){ return floor; } public void setFloor(int floor){ this.floor = floor; } }//end of class building //class Lift public class Lift { private Building building; private int Speed; public Lift(){ building = new Building(); setSpeed(); } public void setSpeed(){ if (building.getFloor()>20){ //one set of functionalities //may be the the speed of the lift will be more this.Speed = 10; } else { //different set of functionalities //may be the speed of the lift will be less this.Speed = 5; } } public int getSpeed(){ return Speed; } }//end of class Lift As it becomes clear from the above code, that while creating the Building object it will create the Lift object, and while creating the Lift object, it will try to create a Building object to access some of its functionalities... So, ultimately it will go out of memory and we get a StackOverflow runtime exception... So how do we handle this problem in Java? We actually tackle this problem by declaring an IBuildingProxy interface and by deriving our Building class from that... the lift class, instead of Having Building object, it Has IBuildingProxy... the source code of the solution looks like the following... public interface IBuildingProxy { int getFloor(); void setFloor(int floor); } public class Building implements IBuildingProxy{ private Lift lift; private int floor; public Building(){ lift = new Lift(this); setFloor(15); } public int getFloor(){ return floor; } public void setFloor(int floor){ this.floor = floor; } } public class Lift { private IBuildingProxy building; private int Speed; public Lift(Building b){ this.building = b; setSpeed(); } private void setSpeed(){ if (building.getFloor()>20){ / /one set of functionalities //may be the the speed of the lift will be more this.Speed = 10; } else { //different set of functionalities //may be the speed of the lift will be less this.Speed = 5; } } public int getSpeed(){ return Speed; } } public class CircularDependencyTest { public static void main(String[] args){ Building b = new Building(); Lift l = new Lift(b); } } So whats the principle behind such work around... It will be clear soon... As it becomes clear from the code that Building HAS Lift... That is not a problem... Now when it comes to solve the part that Lift HAS Building, instead of the Building object, we have created an IBuildingProxy interface and we pass it to the Lift class... what it essentially means, that the building class knows the memory requirement to initialize the Lift object, and as the Lift class HAS just a proxy interface of the Building, it does not have to care for the Building's memory requirement... and that solves the problem... Hope this discussion becomes helpful for Java learners...
November 17, 2011
by Somenath Mukhopadhyay
· 32,477 Views · 2 Likes
article thumbnail
What Is CDI, How Does It Relate to @EJB And Spring?
A brief overview of dependency injection in Java EE, the difference between @Resource/@EJB and @Inject, and how does that all relate to Spring – mostly in the form of links. Context Dependency Injection (CDI, JSR 299) is a part of Java EE 6 Web Profile and itself builds on Dependency Injection for Java (JSR 330), which introduces @Inject, @Named etc. While JSR 330 is for DI only and is implemented e.g. by Guice and Spring, CDI adds various EE stuff such as @RequestScoped, interceptors/decorators, producers, eventing and a base for integration with JSF, EJBs etc. Java EE components such as EJBs have been redefined to build on top of CDI (=> @Stateless is now a CDI managed bean with additional services). A key part of CDI aside of its DI capabilities is its awarness of bean contexts and the management of bean lifecycle and dependencies within those contexts (such as @RequestScoped or @ConversationScoped). CDI is extensible – you can define new context scopes, drop-in interceptors and decorators, make other beans (e.g. from Spring) available for CDI,… . Resources to check: Contexts and Dependency Injection in Java EE 6 by Adam Bien – a very good explanation of the basics of CDI and how it differs from DI in Java EE 5 (hint: context awarness) Slideshow with a good overview of CDI and all it offers About CDI extensibility and SPIs (e.g. Seam 3 is basically a set of portable CDI extensions) Guice and Spring do not implement CDI (3/2011) – and Spring perhaps isn’t motivated to do so (it supports JSR 330, CDI would be too much work) DZone CDI Refcard may be handy CDI 1.0 vs. Spring 3.1 feature comparsion: bean definition & dependency injection: “in the area that I compared in this article [= DI], there is only little critical difference in the two technologies” (though Spring more fine-tunable) Java EE 6 (CDI / EJB 3.1) XOR Spring Core Reloaded: New projects should preferably start with pure Java EE including CDI and add Spring utilities such as JDBC/JMS when needed Oracle: CDI in the Java EE 6 Ecosystem – 62 pages slideshow, the stuff is explained more than in the previously mentioned slideshow Note: CDI 1.1 (JSR 346, Java EE 7) should have a standard way of bootstrapping it in non-EE environment (i.e. SE) From http://theholyjava.wordpress.com/2011/11/09/what-is-cdi-how-does-it-relate-to-ejb-and-spring/
November 12, 2011
by Jakub Holý
· 14,782 Views · 1 Like
article thumbnail
Securing a RESTful Web Service with Spring Security 3.1, part 3
1. Overview This is the third of a series of articles about setting up a secure RESTful Web Service using Spring 3.1 and Spring Security 3.1 with Java based configuration. This article will focus on the security configuration using Spring Security 3.1, assuming some understanding of Spring Security basics and focusing on the specifics of securing the RESTful web service. The REST with Spring series: Part 1 – Bootstrapping a web application with Spring 3.1 and Java based Configuration Part 2 – Building a RESTful Web Service with Spring 3.1 and Java based Configuration Part 4 – RESTful Web Service Discoverability Part 5 – REST Service Discoverability with Spring Part 6 – Basic and Digest authentication for a RESTful Service with Spring Security 3.1 Part 7 – REST Pagination in Spring Part 8 – Authentication against a RESTful Service with Spring Security Part 9 – ETags for REST with Spring 2. Introducing Spring Security in the web.xml The architecture of Spring Security is based entirely on servlet filters and, as such, comes before Spring MVC in regards to the processing of HTTP requests. Keeping this in mind, to begin with, a filter needs to be declared in the web.xml of the application: springSecurityFilterChain org.springframework.web.filter.DelegatingFilterProxy springSecurityFilterChain /* The filter must necessarily be named ‘springSecurityFilterChain’ to match the default bean created by Spring Security in the container. Note that the defined filter is not the actual class implementing the security logic but a DelegatingFilterProxy with the purpose of delegating the Filter’s methods to an internal bean. This is done so that the target bean can still benefit from the Spring context lifecycle and flexibility. The URL pattern used to configure the Filter is /* even though the entire web service is mapped to /api/* so that the security configuration has the option to secure other possible mappings as well, if required. 3. The security configuration Most of the configuration is done using the security namespace – for this to be enabled, the schema locations must be defined and pointed to the new 3.1 versions. The namespace is designed so that it expresses the common uses of Spring Security while still providing hooks to the underlying beans. 3.1. The basics The element is the main container element for HTTP security configuration. In the current implementation, it only secured a single mapping: /api/admin/**. Note that the mapping is relative to the root context of the web application, not to the rest servlet; this is because the entire security configuration lives in the root Spring context and not in the child context of the servlet. 3.2. The entry point In a standard web application, the authentication process may be automatically triggered when the client tries to access a secured resource without being authenticated – this is usually done by redirecting to a login page so that the user can enter credentials. However, for a RESTful Web Service this behavior doesn’t make much sense – authentication should only be done by a request to the correct URI and all other requests should simply fail with a 401 UNAUTHORIZED status code if the user is not authenticated. Spring Security handles this automatic triggering of the authentication process with the concept of an entry point; the entry point is a required part of the configuration, and can be injected via the entry-point-ref attribute of the element. Keeping in mind that this functionality doesn’t make sense in the context of the RESTful web service, the new custom entry point is defined: @Component( "restAuthenticationEntryPoint" ) public final class RestAuthenticationEntryPoint implements AuthenticationEntryPoint{ @Override public final void commence ( HttpServletRequest request, HttpServletResponse response, AuthenticationException authException ) throws IOException{ response.sendError( HttpServletResponse.SC_UNAUTHORIZED, "Unauthorized" ); } } 3.3. The login There are multiple ways to do authentication for a RESTful Web Service – one of the default Spring Security provides is form login – which uses an authentication processing filter – org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter. Since the element doesn’t automatically create this particular filter by default, it needs to be explicitly specified in the configuration, using the element at the position FORM_LOGIN_FILTER; the only required dependency for this bean is the authentication manager. Note that for a standard web application, the auto-configattribute of the element is shorthand syntax for some useful security configuration. While this may be appropriate for some very simple configurations, it doesn’t fit and should not be used for a REST API. 3.4. Authentication should return 200 instead of 301 By default, form login will answer a successful authentication request with a 301 MOVED PERMANENTLY status code; this makes sense in the context of an actual login form which needs to redirect after login. For a RESTful web service however, the desired response for a successful authentication should be 200 OK. This is done by injecting a custom authentication success handler in the form login filter, to replace the default one. The new handler implements the exact same login as the default org.springframework.security.web.authentication.SavedRequestAwareAuthenticationSuccessHandler with one notable difference – the redirect logic is removed: // Use the DefaultSavedRequest URL // final String targetUrl = savedRequest.getRedirectUrl(); // this.logger.debug( "Redirecting to DefaultSavedRequest Url: " + targetUrl ); // this.getRedirectStrategy().sendRedirect( request, response, targetUrl ); 3.5. The authentication manager and provider The authentication process uses an in-memory provider to perform authentication – this is meant to simplify the configuration as a production implementation of these artifacts is outside the scope of this post. 4. Maven and other trouble In addition to the pom.xml from the first post, as well as the one from the second post, the Spring Security maven dependencies need to be added: org.springframework.security spring-security-web ${spring-security.version} org.springframework.security spring-security-config ${spring-security.version} org.springframework spring-tx ${spring.version} org.springframework spring-aop ${spring.version} 3.2.2.RELEASE 3.1.3.RELEASE Notice that the version of the security dependencies is no longer 3.1.0.BUILD-SNAPSHOT as for the standard Spring framework, but 3.1.0.CI-SNAPSHOT. What is more, the security artifacts define dependencies to the 3.0.x versions of Spring, more specifically spring-security-web depends on spring-aop and on spring-tx version 3.0.x instead of the expected 3.1.x. To understand why this is a problem, we need to understand how the Maven conflict resolution algorithm works – in case of conflict, Maven will chose which jar to include based on the distance between the particular dependency and the root of the tree. In our case, the conflicts are the spring-aop and spring-tx jars, appearing once with version 3.0.6 and once with 3.1.0. In the case of spring-aop, it appears once as a level 1 dependency of both spring-security-web and spring-security-config with version 3.0.6 , and once as a level 2 dependency of spring-webmvc with the version 3.1.0; since the 3.0.6 versioned jar is closer to the root, it will be the one chosen by the conflict resolution mechanism. Now that we understand why it is that Maven will deploy the 3.0.6 version of the jars with the application and not the intended 3.1.0 version, we need to address the issue. The solution is to add the two dependencies, with the intended 3.1.0 versions, directly into the pom – this will shorten the distance between them and the root to 0 and will force Maven to use them first. 5. Conclusion This post covered the basic security configuration and implementation for a RESTful service using Spring Security 3.1, discussing the web.xml, the security configuration, the HTTP status codes for the authentication process and the Maven resolution of the security artifacts. In the next articles I will focus on a Java based configuration for Spring Security, integration testing of the secure API using the rest-assured library and HTTP basic authentication. In the meantime, check out the github project. If you read this far, you should follow me on twitter here. Original at Securing a RESTful Web Service with Spring Security 3.1 from the REST with Spring series.
November 9, 2011
by Eugen Paraschiv
· 112,511 Views · 2 Likes
article thumbnail
Dependency Injection for Dummies
Dependency injection is a very simple concept: if you have an object that interacts with other objects the responsibility of finding a reference to those objects at run time is moved outside of the object itself. What does it mean for an object to "interact" with other objects? Generally it means invoking methods or reading properties from those objects. So if we have a class A that invokes method Calculate on class B, we can say that A interacts with B. In the following example we show class A interacting with class B. We can equally say that A depends on class B to fulfill a responsibility. In this case, it not only invokes its method Calculate but it also creates a new instance of that class. class A { private B _b; public A { _b = new B(); } public int SomeMethod() { return (_b.Calculate() * 2); } } In the following example, on the other side, the responsibility of getting a reference to an implementation of a class of type B is moved outside of A: class A { private _b = B; public A(B b) { _b = b; } public int SomeMethod() { return _(b.Calculate * 2); } } In this case we say that a dependency (B) has been injected into A, via the constructor. Of course, you can also inject dependencies via a property (or even a regular method), like in the following example: class A { private _b = B; public B B { get { return _b; } set { _b = value; } } public int SomeMethod() { if (_b != null) return _b.RetrieveValue() * 2;} else // HANDLE THIS ERROR CASE return -1; } } So this is all there is about dependency injection. Everything else just builds on this core concept. Like for example Inversion Of Control (IoC) tools which helps you wiring together your objects at run time, injecting all dependencies as needed. So what exactly is Inversion of Control and how does it relate to Dependency Injection (DI)? I like to associate IoC to the Hollywood Principle: "Don't call us, we'll call you". IoC is a design principle where reusable generic code controls the execution of problem-specific code: it is a characteristic of many frameworks, where the application is built extending or customizing a common skeleton; you put your own classes at specific points and the framework will call you when needed. You can use an IoC container as a framework to perform Dependency Injection on your behalf: you tell the container which are the concrete implementation classes for your dependencies and the container will make sure that your constructors or setters will be called with the right objects. Therefore, IoC containers are just a convenience to simplify how dependency injection is handled. But even if you don't use one you could still manually perform dependency injection. (If you want to have a look at how an IoC container works you can jump to my mini tutorial on Ninject). Why is the concept of dependency injection important? Because by applying it, you simplify your design (separating the responsibility of using an object from the responsibility of using the object) and your code becomes much easier to test, since you can mock out the dependencies substituting them with fake (stub) objects. But that is the subject for another post.
November 8, 2011
by Stefano Ricciardi
· 11,104 Views · 28 Likes
article thumbnail
Using PowerMock to Mock Constructors
In my opinion, one of the main benefits of dependency injection is that you can inject mock and/or stub objects into your code in order to improve testability, increase test coverage and write better and more meaningful tests. There are those times, however, when you come across some legacy code that doesn’t use dependency injection and held together by composition rather than aggregation. When this happens, you have three options: Ignore the problem and not write any tests. Refactor like mad, changing everything to use dependency injection. Use PowerMock to mock constructors Obviously, option 1 isn’t a serious option, and although I’d recommend refactoring to move everything over to dependency injection, that takes time and you have to be pragmatic. That’s where PowerMock comes in... this blog demonstrates how to use PowerMock to mock a constructor, which means that when your code calls new it doesn’t create a real object, it creates a mock object. To demonstrate this idea, the first thing we need is some classes to test, which are shown below. public class AnyOldClass { public String someMethod() { return "someMethod"; } } public class UsesNewToInstantiateClass { public String createThing() { AnyOldClass myclass = new AnyOldClass(); String returnValue = myclass.someMethod(); return returnValue; } } The first class, AnyOldClass, is the class that the code instantiates by calling new. In this example, as the name suggests, it can be anything. The second class, the aptly named UsesNewToInstantiateClass, has one method, createThing(), which when called does a: AnyOldClass myclass = new AnyOldClass(); This is all pretty straight forward, so we’ll move quickly on to the PowerMock assisted JUnit test: import static org.easymock.EasyMock.expect; import static org.junit.Assert.assertEquals; import static org.powermock.api.easymock.PowerMock.expectNew; import static org.powermock.api.easymock.PowerMock.replay; import static org.powermock.api.easymock.PowerMock.verify; import org.junit.Test; import org.junit.runner.RunWith; import org.powermock.api.easymock.annotation.Mock; import org.powermock.core.classloader.annotations.PrepareForTest; import org.powermock.modules.junit4.PowerMockRunner; @RunWith(PowerMockRunner.class) @PrepareForTest(UsesNewToInstantiateClass.class) public class MockConstructorTest { @Mock private AnyOldClass anyClass; private UsesNewToInstantiateClass instance; @Test public final void testMockConstructor() throws Exception { instance = new UsesNewToInstantiateClass(); expectNew(AnyOldClass.class).andReturn(anyClass); final String expected = "MY_OTHER_RESULT"; expect(anyClass.someMethod()).andReturn(expected); replay(AnyOldClass.class, anyClass); String result = instance.createThing(); verify(AnyOldClass.class, anyClass); assertEquals(expected, result); } } Firstly, this class has the usual PowerMock additions of: @RunWith(PowerMockRunner.class) @PrepareForTest(UsesNewToInstantiateClass.class) at the top of the file plus the creation of the anyOldClass mock object. The important line of code to consider is: expectNew(AnyOldClass.class).andReturn(anyClass); This line of code tells PowerMock to expect a call to new AnyOldClass() and return our anyClass mock object. Also of interest are the calls to replay and verify. In the example above, they both have two arguments. The first, AnyOldClass.class relates to the expectNew(...) call above, whilst the second, anyClass refers to the straight forward mock call expect(anyClass.someMethod()).andReturn(expected);. There are those times when you should really let new do what it does: create a new object of the requested type. There is a body of opinion that says you can over-isolate your code when testing and that mocking everything reduces the meaning and value of a test. To me there’s no right answer to this and it’s a matter of choice. It’s fairly obvious that if your code accesses an external resource such as a database, then you’d either refactor and implement DI or use PowerMock. If your code under test doesn’t access any external resources, then it’s more of a judgement call on how much code isolation is too much? This perhaps needs some thought and may be the subject for another blog on anther day... From http://www.captaindebug.com/2011/10/using-powermock-to-mock-constructors.html
October 20, 2011
by Roger Hughes
· 73,906 Views · 2 Likes
article thumbnail
OAuth in headless applications
OAuth is a wonderful standard: it allows users to give permissions to a third-party service to use theirs accounts on a website; but it works without forcing them to share their password like a phishing website would do. The typical use of OAuth is for accessing the Facebook, Twitter or Google+ Api (social networks have lots of data to share). It works like a charm: An URL on socialnetwork.com is generated, that the user loads. A callback URL pointing to your application is attached as a GET param. Since the user is logged in (or he can log in with a simple form) on socialnetwork.com, a token is randomly generated and authorized. In case some permissions are required the user is prompted for approval. The user is redirected back to your application with the token, that now you can use to make requests. You never get to know the user's password. A full explanation of OAuth is out of the scope of this post; what I want to tackle today is how to use OAuth inside an headless application such as Jenkins, PHPUnit or Ant. Headless application? An headless application does not have a real user interface, and which can run in background for days. Consider for example Jenkins performing builds for Continuous Integration; a Selenium server; or a crawler loading web pages all day on a server machine. The point is in headless application you cannot send the user (who may be yourself) to a browser for approval at every reboot (for example because the process runs on a CI machine.) And usually, the social network can't redirect the user to the headless application with an HTTP Location header. However, in the cases we are interested in a user is needed in order to make requests to the Api. Most social networks allows you to make many kinds of Api calls only after having logged in with an user. Thus we have to split up our application in two parts: one (small) whose job is to obtain the authentication token, and one for using it as normal. Obtaining the token The assumption we make (see the relevant section at the end of the article) is that long-lived tokens are available for authorization. LinkedIn works like this, and I used my own user for authorizations to make Api calls for groups data. I integrated an example of authorization from Scribe, the simplest Java library for OAuth. You load the URL provided by Scribe in a browser, follow the website-specific authorization procedure and then paste back the verifier parameter in Scribe. However, I printed the token instead of using it immediately, and saved it in a .properties file which is ignored by Git via configuration, in order to avoid publishing it in a source code repository. Beware, a commit remains in the repository forever! import java.io.IOException; import java.util.Properties; import java.util.Scanner; import org.scribe.builder.ServiceBuilder; import org.scribe.builder.api.LinkedInApi; import org.scribe.model.OAuthRequest; import org.scribe.model.Response; import org.scribe.model.Token; import org.scribe.model.Verb; import org.scribe.model.Verifier; import org.scribe.oauth.OAuthService; public class LinkedInConfigurator { /** * @param args */ public static void main(String[] args) { try { Properties properties = LinkedInDefault.getConfiguration(); OAuthService service = new ServiceBuilder() .provider(LinkedInApi.class) .apiKey(properties.getProperty("apiKey")) .apiSecret(properties.getProperty("apiSecret")) .callback("http://localhost") .build(); Scanner in = new Scanner(System.in); System.out.println("Fetching the Request Token..."); Token requestToken = service.getRequestToken(); System.out.println("Got the Request Token!"); System.out.println(); System.out.println("Now go and authorize Scribe here:"); System.out.println(service.getAuthorizationUrl(requestToken)); System.out.println("And paste the verifier here"); System.out.print(">>"); Verifier verifier = new Verifier(in.nextLine()); System.out.println(); System.out.println("Trading the Request Token for an Access Token..."); Token accessToken = service.getAccessToken(requestToken, verifier); System.out.println("The access token is [accessToken, accessSecret]: " + accessToken); } catch (IOException e) { e.printStackTrace(); } } } The storing process is manual in my case, but you can easily save the token wherever you want. Using the token To use the access token, instantiate Scribe OAuthService again, along with the Token: you should pass to the constructor the two informations obtained from dumping it (token value and secret). Now you can create requests and pass them to the service along with the token to be signed. The headless application acquire all the permissions that the user has in the Api. import static org.junit.Assert.assertTrue; import org.scribe.model.OAuthRequest; import org.scribe.model.Response; import org.scribe.model.Token; import org.scribe.model.Verb; import org.scribe.oauth.OAuthService; public class ScribeLinkedInService implements LinkedInService { private OAuthService oauthService; private Token accessToken; public ScribeLinkedInService(OAuthService oauthService, Token accessToken) { this.oauthService = oauthService; this.accessToken = accessToken; } @Override public String getLastPostsResponse(int groupId, int numberOfPosts, long timestampToStartFrom) { OAuthRequest request = new OAuthRequest(Verb.GET, "http://api.linkedin.com/v1/groups/" + groupId + "/posts?count=" + numberOfPosts + "&modified-since=" + timestampToStartFrom); oauthService.signRequest(accessToken, request); Response response = request.send(); return response.getBody(); } } Terms Of Service and expiration Before jumping to implementation, verify that your social network of choice gives you tokens that you can legally store and do not expire in half an hour. LinkedIn is where I tested this approach and it explicitly said that if the user specifies so, token are durable and do not expire until explicitly revoked. In fact I'm still using one obtained last week for running my integration tests. Facebook by default gives you a parameter (in the redirect URL) along with the token that tells you how many seconds the token will last. However, if you ask the user for the offline_access permission in the scope parameter of the OAuth dialog, the token will have an infinite expiry time. It's not really secure for a user to relinquish the access to his account to the application forever, but I presume you're using your own user (or a dummy one) like I am with LinkedIn. Then, you're already saving your password in the browser, aren't you? Twitter does not currently expire access tokens, unless your application is suspended or the user rejects it from their settings page. Moreover, it offers a wide public portion of its Api where you do not need an authenticated user to perform requests; these tricks may not be neeeded. FourSquare does not expire tokens, but may do so in the future.
October 13, 2011
by Giorgio Sironi
· 25,497 Views · 1 Like
  • Previous
  • ...
  • 169
  • 170
  • 171
  • 172
  • 173
  • 174
  • 175
  • Next
  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook
×