Over a million developers have joined DZone.

Black-Pipe Testing libmongoc and a Connected Application In C

I describe completely testing libmongoc as a black pipe; while most testing covers the public API, this procedure tests an external and internal scenario.

· DevOps Zone

The DevOps Zone is brought to you in partnership with Sonatype Nexus. The Nexus Suite helps scale your DevOps delivery with continuous component intelligence integrated into development tools, including Eclipse, IntelliJ, Jenkins, Bamboo, SonarQube and more. Schedule a demo today

The F TrainLike any network client library, libmongoc cannot be fully tested as a black box. Traditional black box tests enter some input and check the output—this only validates one side of the system at a time. But libmongoc has two sides, working in concert. One side is its public API, its structs and functions and so on. The other is its communication over the network with the MongoDB server. Only by treating it as a black pipe can we fully test its two sides.


I began thinking about black-pipe testing early this year. I was reading the libmongoc test suite in preparation for taking over the project from Christian Hergert and Jason Carey, and I came across Christian's mock_server_t struct. Test code in C does not ordinarily make lively reading, but I woke up when I saw this. Had he really written a MongoDB wire protocol server in order to test the client library?

If you know Christian Hergert's work, you know the answer. Of course he had. His mock server listened on a random TCP port, parsed the client's network messages, and sent MongoDB responses. At the time, mock_server_t used callbacks: you created a mock server with a pointer to a function that handled requests and chose how to reply. And if you think callbacks are ungainly in Javascript or Python, try them in C.

Despite its awkward API, the mock server was indispensable for certain tests. For example, Christian had a mock server that reported it only spoke wire protocol versions 10 and 11. Since the latest MongoDB protocol version is only 3, the driver does not know how to talk to such a futuristic server and should refuse to, but the only way to test that behavior is by simulating the server.

Besides the protocol-version test, Christian also used the mock server to validate the client's handling of "read preferences". That is, how the client expresses whether it wants to read from a primary server, a secondary, or some subtler criterion. A mock server is required here because a correct client and a buggy one appear the same at the API level: it is only when we test its behavior at the network layer that bugs are caught.

In these two tests I saw the two use cases for "black-pipe" testing. First, black-pipe tests simulate unusual server behavior and network events. Second, in cases where the client's API behavior can appear correct even when there are bugs at the network layer, black-pipe tests validate the network-level logic too.

F Train

Evolution: From C to Python

I had not yet taken leadership of libmongoc—I was finishing up some Python work. So, inspired by Christian's idea, I wrote a mock server in Python, called MockupDB. MockupDB is the subject of my earlier article in this series: "Testing PyMongo As A Black-Pipe."

Since I was working in my native tongue Python, I could afford to be finicky about MockupDB's interface. I didn't want callbacks, dammit, I wanted to make something nice! As I wrote in the MockupDB article, I came up with a future-based programming interface that let me neatly interleave client and server operations in a single test function:

from mockupdb import MockupDB, Command, go
from pymongo import MongoClient
def test():
   server = MockupDB(auto_ismaster={"maxWireVersion": 3})

   client = MongoClient(server.uri)
   collection = client.db.collection

   future = go(collection.insert_one, {"_id": 1})
   request = server.receives(Command({"insert": "collection"}))
   request.reply({'ok': 1})
   assert(future().inserted_id == 1)

Let's break this down. I use MockupDB's go function to start a PyMongo operation on a background thread, obtaining a handle to its future result:

future = go(collection.insert_one, {"_id": 1})

The driver sends an "insert" command to the mock server and blocks waiting for the server response. I retrieve that command from the server and validate that it has the expected format:

request = server.receives(Command({"insert": "collection"}))

MockupDB asserts that the command arrives promptly and has the right format before it returns the command to me. I reply to the client, which unblocks it and lets me retrieve the future value:

request.reply({'ok': 1})
assert(future().inserted_id == 1)

More Evolution: From Python Back to C

Once Bernie Hackett and I released PyMongo 3.0, I devoted myself to libmongoc full-time. I set to work updating its mock_server_t with the ideas I had developed in Python. I wrote an example with the API I wanted:

mock_server_t *server;
mongoc_client_t *client;
mongoc_collection_t *collection;
bson_t *document;
bson_error_t error;
future_t *future;
request_t *request;

/* protocol version 3 includes the new "insert" command */
server = mock_server_with_autoismaster (3);
mock_server_run (server);

client = mongoc_client_new_from_uri (mock_server_get_uri (server));
collection = mongoc_client_get_collection (client, "test", "collection");
document = BCON_NEW ("_id", BCON_INT64 (1));
future = future_collection_insert (collection,
                                   MONGOC_INSERT_NONE,/* flags */
                                   NULL,              /* writeConcern */

request = mock_server_receives_command (server, "test", MONGOC_QUERY_NONE,
                                        "{'insert': 'collection'}");

mock_server_replies_simple (request, "{'ok': 1}");
assert (future_get_bool (future));

future_destroy (future);
request_destroy (request);
bson_destroy (document);
mock_server_destroy (server);

Alas, C is prolix; this was as lean as I could make it. I doubt that you read that block of code. Let's focus on some key lines.

First, the mock server starts up and binds an unused port. Just like in Python, I connect a real client object to the mock server's URI:

client = mongoc_client_new_from_uri (mock_server_get_uri (server));

Now I insert a document. The client sends an "insert" command to the mock server, and blocks waiting for the response:

future = future_collection_insert (collection,
                                   MONGOC_INSERT_NONE,/* flags */
                                   NULL,              /* writeConcern */

The future_collection_insert function starts a background thread and runs the libmongoc function mongoc_collection_insert. It returns a future value, which will be resolved once the background thread completes.

Meanwhile, the mock server receives the client's "insert" command:

request = mock_server_receives_command (server,
                                        "test",            /* DB name */
                                        MONGOC_QUERY_NONE, /* no flags */
                                        "{'insert': 'collection'}");

This statement accomplishes several goals. First, it waits (using a condition variable) for the background thread to send the "insert" command. Second, it validates that the command has the proper format: its database name is "test", its flags are unset, the command itself is named "insert," and the target collection is named "collection."

The test completes when I reply to the client:

mock_server_replies_simple (request, "{'ok': 1}");
assert (future_get_bool (future));

This unblocks the background thread. The future is resolved with the return value of mongoc_collection_insert. I assert that its return value was true, meaning it succeeded. My test framework detects if future_get_bool stays blocked: this means mongoc_collection_insert is not finishing for some reason, and this too will cause my test to fail.


When I first saw Christian Hergert's mock_server_t its brilliance inspired me: To test a MongoDB client, impersonate a MongoDB server!

I wrote the MockupDB package in Python, and then I overhauled Christian's mock server in C. As I developed and used this idea over the last year, I generalized it beyond the problem of testing MongoDB drivers. What I call a "black pipe test" applies to any networked application whose API behavior and network protocol must be validated simultaneously.

The DevOps Zone is brought to you in partnership with Sonatype Nexus. Use the Nexus Suite to automate your software supply chain and ensure you're using the highest quality open source components at every step of the development lifecycle. Get Nexus today

c language ,python ,mongodb

Published at DZone with permission of A. Jesse Jiryu Davis, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}