Announcing MongoDB Perl Driver 1.0

The Perl Driver team is thrilled to announce the release of the v1.0.0 ("v1") MongoDB Perl driver!

Goals of the Next-Generation Perl Driver

In the next-generation MongoDB drivers announcement, we described why next-generation drivers follow a common set of behavior specification documents. Whereas previous drivers were written idiosyncratically, each driver now aims to deliver similar underlying behaviors through similar APIs, while still striving to be idiomatic for its language.

For example, the v1 Perl driver implements the new MongoDB Driver CRUD API specification in MongoDB::Collection, so that CRUD operations behave similarly across drivers and the mongo shell. Likewise, it implements the Server Discovery and Monitoring and Server Selection specifications for more predictable network operations and failover.

For this next-generation Perl driver, however, we had several goals that went above and beyond cross-driver consistency. In particular, we wanted to improve the user experience by addressing some significant pain points in the "v0" driver releases.

Improved Consistency in APIs and Error Handling

In the v0 driver, many parts of the API were inconsistent from method to method, or even within methods. There was little rhyme or reason for return values, option passing style, or error handling.

For example, in the v0 driver, the update command could return "1" or could return a hash reference, depending on whether a write concern was in use. This means that a mere configuration change actually changes the return type of a method call, which could – unless checked – easily result in a runtime error.

Likewise, the v0 driver had numerous ways of reporting failure, including throwing an exception, returning a false value, returning a string value (rather than a hash reference), or returning a hash-reference that must be checked for the ok field. And even though the documentation didn't mention it, any method involving network I/O could throw an exception on a network error. These inconsistencies can lead to bugs and surprises.

The v1 API implements the CRUD API and provides consistency in option passing and return types for the most common operations users need. Other APIs have been expanded or revised similarly.

On the error handling side in the v1 driver, errors are now consistently signaled by throwing exception objects. The exception objects are described in MongoDB::Error and include structured data about the error (when appropriate) for applications that wish to inspect it.

Better Encapsulation and Abstraction

In the v0 driver, too many low-level, internal operations were exposed as part of the public API, which complicated maintenance work. The v1 API encapsulates internal operations and deprecates, removes, or privatizes most legacy low-level functions. The aim is to minimize the "public surface" exposed to end-user developers, allowing faster feature development in the future with less risk of breakage.

On the flip side, in many cases the return values of some v0 methods were simply the raw result documents sent by the MongoDB server. This leaves application developers exposed to changes in the server's return document format, fields, or semantics over time. Along with the implementation of the CRUD API, methods return result objects with methods for inspection. This protects end-users, as the driver will compensate for server-side result document changes behind consistent result object APIs.

Improved Server Compatibility and Feature Support

The MongoDB server is continually evolving and improving, but the v0 driver had not fully kept pace. The v1 driver internally implements all write operations using write commands instead of the legacy (soon to be deprecated) wire protocol for writes, which improves the reliability and granularity of error detection.

In the v1 driver, SSL support is provided via IO::Socket::SSL and users may pass it custom SSL configuration parameters, if needed. This provides full support for X509 certificate authentication and SSL server certificate validation, both of which the v0 driver lacked.

More Portability and Reduced Dependencies

The v0 driver had a large dependency tree and substantial non-portable C code, particularly for networking, SSL, and SASL authentication. The v1 driver now relies on widely-used, well-tested CPAN modules in place of custom C code everywhere except for BSON encoding/decoding. This change improves overall reliability and platform compatibility. It also lays the groundwork for a future "pure-Perl optional" driver with a pure-Perl BSON implementation for deployment environments without a compiler available.

Notably, the v1 driver switched its OO framework from Moose to Moo. This change reduces the number of CPAN module dependencies substantially and enabled several performance improvements. With this and other changes, the v1 driver trims the runtime "deep dependency tree" (i.e. including dependencies of dependencies) from about 50 CPAN libraries to about 30 – a 40% reduction.

Improved BSON Support

The Perl driver's BSON implementation is responsible for mapping typed BSON data to Perl's dynamically-typed variables (and select classes) and back again. There are inherent ambiguities translating typed data to and from Perl's generic scalar variables. When decoding a typed BSON variable to a Perl scalar, the type information is lost unless it is decoded to a blessed object to signify type.

In many instances, the v0 driver BSON implementation defaulted to losing BSON typing information, making data round-trips impossible. The v1 driver substantially overhauls BSON encoding and decoding to round-trip data correctly by default – at least to the greatest extent possible with a typeless dynamic language like Perl.

Highlights of Specific Changes

As you might imagine given these goals, the v1 driver includes a substantial number of changes, listed in exhaustive detail in the change log. Fortunately, the MongoDB::Upgrading document summarizes changes that application developers need to be aware of before upgrading to the v1 driver.

Listed below are some highlights from that document:

Configuration

In the v1 driver, MongoDB::MongoClient gains a number of new configuration options to control server selection and timeouts. In addition, client configurations are now immutable; former read/write attributes are now all read-only. Changes to options like write concern are now done directly at the database or collection level, as shown later.

One handy change is that creating a MongoDB::MongoClient object is now possible directly from the "MongoDB" namespace connect method with just the standard connection-string URI:

Options for MongoDB::MongoClient can be passed as an optional hash reference.

To avoid ambiguities in server discovery, replica sets now require an explicit replica set name. Without a replica set name, a "direct connection" will be set up, such as for sending administrative commands to a secondary in a replica set.

Some existing options (e.g. timeout) are deprecated in favor of more specific options (connect_timeout_ms); others are removed entirely where they no longer fit the new paradigms of the client API (find_master).

Lazy Connection

In the v1 driver, creating a MongoDB::MongoClient object no longer connects to a server right away. This is the new standard for all official MongoDB drivers, but might break code that expected an immediate error from new if MongoDB wasn't available:

Instead, create a client object and do the actual work in the eval block.

Failover

The change to lazy connections puts client connection, reconnection, and error handling on equal footing. Whenever progress can't be made, an error is thrown. The exact circumstances will dictate the exact MongoDB::Error object and message (e.g. timeout, network error, no primary, etc.) Generally, for any network-related error, the connection to the server is closed. If the exception is handled, the next attempt at communicating with the server will automatically attempt to reconnect. For a replica set, this means it will failover to a new primary when a new primary is ready.

Authentication

Given the immutable client configuration, lazy connections, and automatic failover, authentication is now based only on configuration options and occurs immediately when any server connection is made. This is another change for all official drivers that harmonizes the behavior of MongoDB's different authentication mechanisms, some of which could only happen at connection time and some of which could be done later.

This won't change behavior for drivers that already provided authentication credentials in the MongoDB::MongoClient constructor parameters. However, the authenticate method has been removed and code that depended on it will need to be updated.

As a side effect of harmonizing authentication credential handling, only a single set of credentials may be associated with a single client object. Code that relied on calling authenticate multiple times to authenticate to multiple databases will need to be re-written to use multiple client objects instead.

Read Preferences and Write Concerns

Read preferences and write concern are now represented as objects. As a result, they can be set at the client, database, or collection object level. Now that configuration is immutable, changing these temporarily is done by cloning an object with a modified configuration.

For example,to temporarily change the write concern for a given collection, one now uses the clone method:

BSON Encoding and Decoding

The various $MongoDB::BSON::... global variables have been removed, as BSON encoding is now encapsulated in the bson_codec attribute. As with read preferences and write concerns, it can be set per-client, per-database, or per-collection. A special with_codec method exists on MongoDB::Collection to simplify cloning with changes to BSON encoding/decoding:

As a new optimization, integers are now encoded by default to the smallest BSON integer type that fits rather than always taking up a fixed size equal to the compiled integer size of the interpreter. In the example above, on a 64-bit perl, the "42" would still be converted to BSON as a 32-bit integer rather than a 64-bit integer.

In addition, all common JSON Boolean classes will now encode correctly and Time::Moment is supported for both encoding and decoding datetimes.

Deprecations

Almost all of the existing MongoDB::Collection API was deprecated and replaced with an implementation of the MongoDB Driver CRUD API. The deprecated methods are now undocumented, but still mostly work as they used to. A handful of methods in other classes were deprecated as well. We encourage you to transition to the new API as soon as possible.

Next Steps

With the release of the v1.0.0 Perl driver, the team is turning to the next set of enhancements. Currently, we expect the v1.2.0 driver to include:

• support for (forthcoming) MongoDB 3.2 features
• a revised and enhanced GridFS API
• additional performance improvements

As excited as we are about the v1 release, we know that no software is ever without bugs. We invite you to submit bug reports, features requests, or general feedback about the MongoDB Perl driver on our JIRA issue tracker.

Want more MongoDB? Read our white paper to learn about the latest features in MongoDB 3.0:

What's New in MongoDB 3.0

About the Authors - David Golden

David has been active in open-source software for over 15 years, with particular emphasis on the Perl language and community. He is a member of the "Perl 5 Porters" that maintain the core Perl source code.

He currently works at MongoDB as a senior software engineer. Previously, David worked for 16 years as a management consultant for some of the largest North American and global financial institutions.