Ludovic Poitou on OpenDS and Directory Server Enterprise Edition

DZone recently sat down with Ludovic Poitou, senior staff engineer at Sun Microsystems and community manager for the OpenDS project, a high performance, modular, pure Java directory server implementation.  OpenDS, which was initiated by Sun Microsystems, supports all common, directory related standards and specifications. OpenDS is feature rich, enterprise ready, and commerically supported by one of the most well known firms in the identitiy management area. Here are a few of its features:

• LDAP V3 support for core operations
• Support for a number of standard or experimental controls. Controls can be included in requests to ask the server for additional processing. Each additional processing capability is called a control
•  Support for some of Simple Authentication and Security Layer (SASL) authentication mechanism with possibility to add other mechanisms

• Server extendibility and plug ability
• Replication support
• Embedable
• Import, export, backup and restore
• OpenDS complies with password policy implementation draft.
• Already implemented and possibility to add more LDAP Extended operations
• Support for StartTLS

In the following interview, Ludovic Poitou delves into some of key features of the OpenDS project and what we can expect to see in future releases.

DZone: Hi Ludovic, can you please introduce yourself and let us know for how long have you been working on OpenDS?

Ludovic: Hi Masoud. I'm Ludovic Poitou, a senior staff engineer working for Sun Microsystems. I'm based in France, working out of the Grenoble Engineering Center. I've been working on LDAP technology for Sun since 1996 and has been involved in all Sun directory products since, first as a developer, more recently as the architect. I've been involved with OpenDS since we started the project in July 2006, and became the OpenDS community manager in September 2007.

DZone: Can you describe the overall shape of the OpenDS project and how it fits into the enterprise?

Ludovic: The OpenDS project intends to deliver a complete and comprehensive LDAP directory service. With OpenDS 1.2 released a few weeks ago, we have reached a major step, delivering a reliable, extremely efficient, fully compliant LDAPv3 directory server. The OpenDS server has all features required for the use in enterprises: security, access controls, multi-master replication providing high availability with regular hardware, monitoring, backup... And it is now also available for OpenSolaris, from the IPS package repository.

In terms of performance, we're extremely pleased with the numbers we're able to squeeze. We do all our performance tests with 10 million entries. This is our baseline. Also, we always configure 2 identical servers with multi-master replication between them, to reproduce real customers environment. So with this configuration, on an X4600 server (8 dual core AMD processors at 3GHz), with Solaris 10 update 4, a 6140 Storage using 15000 rpm disks and ZFS, an instance of OpenDS 1.3.0-build001 has been put under stress and produced the following results: Importing the 10 millions entries at a rate of 10 000 entries per second, Subtree exact search at a rate of approximately 30 000 searches per second with an average response time of 0.2 milliseconds, Modification of a single attribute at a rate of approximately 10 000 modifications per second with an average response time of 0.6 milliseconds. And when mixing the searches and modifications (at a ratio 90% / 10%), the server is able to sustain 24400 operations per seconds with an average response time of 1.2 ms.

DZone: There are at least two more directory services in the open source community, one of which is supported by Apache (ApacheDS). How does OpenDS compare with these two projects?

Ludovic: OpenDS does share one thing with ApacheDS: both projects are based on the Java platform. I think the similarities stop here. The OpenDS project has been started with some very specific customers requirements, some commercial goals and one major objective : be better than Sun Directory Server. And the project was started with the same engineers that have been working on Sun Directory Server code for over a decade now, so not only they are expert in LDAP, but they have experience in designing code for performance and reliability.

Compared to the other directory services open source projects that are C based, OpenDS has already caught up. In terms of functionality, but the project hasn't reach the same level of popularity yet, especially with Linux developers. One thing that is making a real difference with the other projects, is the ease of use. You can get on any machine with Java 1.5 or higher, an OpenDS based server installed, configured and running in less than 3 minutes and 6 clicks. Also, contrary to most open source projects, OpenDS was started with an intensive documentation effort, and the OpenDS Documentation wiki describes most of the administrative tasks required to manage an OpenDS based directory service.

DZone: A very common and recurring question about OpenDS is why Sun started a second directory server project while it hds one of the most prominent directory servers available in the market.

Ludovic: Sun Directory Server was designed in the mid 90s, and back then the requirements on directory services were very different from what they are now. To be meet some of the requirements given by our customers, we needed to redesign and recode major areas of the server. At the same time, Sun directions were to do our project development in open source. So, instead of spending years in doing due diligence for open-sourcing the current code, we decided to start a new project from scratch, with our past experience, and hopefully taking a huge step forward in term of usability and performance.

This said, here we're only talking about the code. Sun Directory Server Enterprise Edition is not discontinued, and but within the next 2 years, it will be based on OpenDS code while still providing compatibility with previous versions. You can find a comparison between OpenDS and DSEE 6 feature here

DZone: Does OpenDS provide replication and high availability?

Ludovic: So, since day one, OpenDS was designed to support multi-master replication, allowing to build highly available services for all applications.

The multi-master replication architecture is very similar to the one of Sun Directory Server. It is based on a loosely consistent model, where a change is received and applied on a single master, and then replicated to all other servers. Synchronized clocks guarantee a complete time based ordering within the service, and all servers apply the same conflict resolution rules so that over time, all servers ends with the exact same set of information.

OpenDS replication protocol has been optimized for both data-centers and replication over WAN and it is as fast as sending the change over and applying it. Overall it only takes a few milliseconds to replicate a change to the replicas, and even under sustained load, replicas are in sync with each others in less than a second.
OpenDS 2.0, planned to be released in June, has some new replication functionality, providing greater reliability and consistency of the data in a replicated topology. With "Assured Replication", the administrator can control how many servers would need to receive, and/or apply the change before the original master can return an Ok to the client application. While this is not yet fully 2 phase commit transactions for replicated services, the "Assured Replication" mode is providing a greater reliability, ensuring that a change cannot be lost even if the machine crashes right at the time the transaction is committed.

DZone: Can we use multiple instances of OpenDS to form a distributed directory server? If not do you have any plan to include this feature in next major version of OpenDS?

Ludovic: Today, OpenDS server can return referrals to alternate servers, allowing to form a distributed directory services, but referrals are actually handled by the client applications and thus it is not fully transparent for them.
OpenDS 3.0 will probably have the ability to chain requests to multiple directory servers, providing a single access point to the directory service. But as I speak, OpenDS 3.0 content is not finalized, so the feature might be in or out by then.

DZone: How would you compare the 'enterprise-readyness' of OpenLDAP, ApacheDS and OpenDS?

Ludovic: If I was an enterprise looking for an open source LDAP directory server, I would consider either OpenLDAP or OpenDS. Honestly, I haven't followed much the development of Apache DS, but I've never heard of any deal in which Apache DS was considered against OpenDS or Sun Directory Server, and I've never heard of real production site based on Apache DS.

OpenLDAP on the other hand is definitely a mature and complete LDAP based directory server, although until recently it lacked the enterprise specific features such as the ability to change the configuration, the schema or the access control rules without restarting the service. It's only with the recent version 2.4 that those features are available. The weakness of OpenLDAP in my opinion is the lack of tools, lack of decent and up to date documentation and the relative complexity for installing, configuring and tuning it for performances.

And of course OpenDS is ready for enterprise use. It is complete in term of features, has amazing performances, built-in high availability with Multi-Master Replication. OpenDS has a simple graphical user interface for basic administration tasks and monitoring, a complete user and administration documentation, and is really simple to use, making it a perfect choice for SMB but as well for large extranet directories, those with millions of entries.

DZone: Sun Microsystems provides commercial support for the OpenDS Standard Edition. What all does this include?

Ludovic: Sun OpenDS Standard Edition is the commercial product based on the OpenDS project. The license entitles for support, including telephone support, patches, hot-fixes, and for indemnification.
If you're deploying the open source version of OpenDS, you can post questions to the mailing lists or the IRC channel (#opends on irc.freenode.net) and you will surely get some advice or answer, but you will have to wait for a fix to appear in the source repository to have a resolution. If the directory service is critical for your business, you probably don't want to take this path.

Also, Sun OpenDS Standard Edition is exactly identical to OpenDS (minor some license and branding differences), but in the future, there might be some Sun specific add-ons such as a replication gateway from Sun Directory Server to OpenDS.

Ludovic: OpenDS is built on Oracle Berkeley DB Java Editiion which has the features, the scalability and performance characteristics that we need. We have defined a "backend" interface allowing us to plug additional databases in the future, or allowing developers to plug their own. An alternate backend available in OpenDS is a "Memory backend" which allows us to stress and optimize the core engine.

Recently, several of Sun and MySQL customers have expressed the need for accessing their customer data with both LDAP and SQL. Sun Directory Server Enterprise Edition 6.x has virtual directory capabilities in the Directory Proxy Server, allowing to map and access RDBMs through LDAP. But this solution is doing translation of LDAP to SQL to access the data.

DZone: How does MySQL Cluster work with OpenDS?

Ludovic: MySQL Cluster uses NDB (Network DataBase) which is an in-memory, distributed, highly available database, and customers can either access the database through the network via theMySQL engine, or through C++ API.

The OpenDS NDB backend is another implementation of the "backend" interface that goes directly to the NDB database, allowing MySQL Cluster customers to access their data either via SQL or directly via the API, or via LDAP.

This backend is still under tests and code is updated regularly, but I encourage the braves to play with it. Note that unlike the regular OpenDS builds, the OpenDS version with NDB backend expect MySQL Cluster and libraries to be already installed on the machine. 

DZone: OpenDS can be used as an embedded directory server. Do you have any examples of where this is being used?

Ludovic: OpenDS being 100% Java can be embedded in Java applications and the best example is the OpenSSO project that runs an embedded instance of OpenDS as its configuration and policy store. Some developers have wrapped OpenDS in a Web application so they can deploy it easily in their infrastructure the same way they deploy the other applications. Last time, I talked with them, they had deployed 2 instances, in Tomcat, serving 5 millions users.An quick tutorial for embedding OpenDS can be found at http://blogs.sun.com/marginNotes/entry/opends_inside

DZone: Are you back-porting DSSE features to OpenDS or are you planning to rewrite them from the beginning with some new architecture and design goals?

Ludovic: Because DSEE and OpenDS are written in different languages (DSEE is C based and OpenDS written in Java), we do not back-port DSEE features to OpenDS, but we implement the features with some new design goals. A best example for this is the grouping capabilities in OpenDS. There is a single standard grouping mechanism in LDAP: static groups defined as either GroupOfNames or GroupOfUniqueNames. Netscape Directory Server (and all its derivate servers) has dynamic groups with are defined by a filter (groupOfURLs). In OpenDS, we have defined an interface with a set of common services and both static and dynamic groups are implementing this interface, offering similar services such as the group membership back-link in the user entry.

DZone: Ludovic, Thank you for participating in this interview. Any final thoughts for our members?

Ludovic: If you've haven't looked at OpenDS, give it a try. It only requires 3 minutes and 6 clicks. Just start here http://www.opends.org/promoted-builds/latest/install/QuickSetup.jnlp.
If you have downloaded OpenDS and are using it, please let us know. Join the OpenDS community to https://opends.dev.java.net/servlets/ProjectMembershipRequest and give us feedback on what you like, what you would like to see enhanced, what is not yet part of OpenDS but would like to see. And if you would like to share more, tell us your OpenDS Story http://blogs.sun.com/Ludo/entry/what_s_your_opends_story