Over a million developers have joined DZone.

Starting with Lift and CouchDB: Resolving Some CouchDB-Record Inconsistencies

DZone's Guide to

Starting with Lift and CouchDB: Resolving Some CouchDB-Record Inconsistencies

· Database Zone
Free Resource

Traditional relational databases weren’t designed for today’s customers. Learn about the world’s first NoSQL Engagement Database purpose-built for the new era of customer experience.

In recent months I have been introduced to Lift web framework by my scala-enthusiastic colleagues @ OpenCredo. The scala expressiveness and conciseness along with view-first web approach looked very interesting from the start, especially since great part of my previous experience comes from verbose Java frameworks with traditional MVC architecture.
After initial interest and research I did my first project, using a relational database and the well-known Lift-Mapper framework. Much of the data we needed to store represented documents, so I started experimenting with CouchDB. While the REST interface to Documents and Views in Couch does not require a rich mapping layer (as opposed to the richness  of SQL ORM frameworks), I still found it cumbersome to work directly with HTTP all the time in order to store/query my model objects to/from CouchDB.
One of the nice features of Lift's Record framework is it's agnostic approach to the underlying persistence mechanism, which means that it is relatively easy to plug in any storage solution. Currently the Lift-Record framework has a Record implementation for relational databases (Squeryl), as well as  Mongo-Record and CouchDB-Record implementations which gained prominence with recent increased interest in NoSQL storage. While the CouchDB-Record implementation is not complete, it has enough features to make it useful when working with CouchDB and Lift.  Using CouchDB-Record I could easily map my domain model to the underlying database, and abstract the HTTP layer when I wanted to. The CouchDB-Record implementation did not offer everything that you might expect from a typical O(R)M framework - specifically there are some missing features such as associations mapping, lazy loading, caching. However these features weren't required for this specific project, and some would argue that those are not even a features that a non-relational storage should use (there is no R in ORM if using CouchDB).
First thing to do when using the CouchDB with lift is to add database configuration to the Lift's Boot class. To do this, you need only three lines of code, as Listing 1 illustrates
Listing 1:
val couch = new Database("",5984, "lift-test")
couch.createIfNotCreated(new Http())
CouchDB.defaultDatabase = couch
To map a model object to CouchDB persistence, you'll need to extend CouchRecord class which is part of the couchdb-record framework. In addition, you define its companion object with the same name, which mixes CouchMetaRecord trait.
Listing 2 shows the sample User class mapped to the CouchDB database using Lift's Record framework
Listing 2:
import net.liftweb.record.field.{DateTimeField, PasswordField, StringField}
import net.liftweb.couchdb.CouchRecord
import com.opencredo.nicheplatform.couchdb.CouchMetaRecord
class User extends CouchRecord[User] {
  def meta = User

  object username extends StringField(this, 20)
  object fullName extends StringField(this, 100)
  object birthDate extends DateTimeField(this)
object User extends User with CouchMetaRecord[User] {
We demonstrated 2 different field types here: StringField and DateTimeField. For a full list of supported filed types in couchdb-record implementation, you can check the online documentation at http://exploring.liftweb.net/master/index-8.html.
Ok, let's persist a user now. To check that the user is actually peristed we're going to write a test, as illustrated in Listing 3:
Listing 3:
class UserTest extends FunSuite with LazyLoggable {
  // reset database and boot lift
  if (!Boot.initialized_?) {
      Http(new Database("", 5984, "lift-test").delete)
    }catch {
      case e : Exception => Console.print(e)
    val boot: Boot = new Boot()

 test("Create a new User") {
   val userRecord = User.createRecord #1
   userRecord.username.set("mrbob") #2
   userRecord.fullName.set("Bob Bobic") #3
   val user: Box[User] = userRecord.save #4
   assert(user.isDefined, user.compoundFailMsg("Failure")) #5
To create a persistable couchdb-backed document, we simple call the createRecord fuction on our user object (#1). After we set the properties we require (#2, #3), we simply save the created object (#4).
The save function persists the object (by issuing a POST request to the CouchDb server) and returns a Box. The Box will be full if the operation succeeded, in which case the Box will contain the peristed object - with assigned id and rev fields from the CouchDb server.
If the POST operation fails to create the new document the box will be empty, and will contain the Failure message (#5)

The code looks nice, so we run the test, excited about our first couch db persisted object - but the test FAILS miserably!!! What could be wrong.
The test fails at the assertion point (so at least we did the configuration right), and the message is as follows:
Failure(User not defined,Empty,Full(Failure(ok not present in reply or not true: JObject(List(JField(ok,JBool(true)), JField(id,JString(706a63a5ae0adc3325347c8a1d020160)), JField(rev,JString(1-8eaacb2ca383a965037251676b25dbd6)))),Empty,Empty)))

What does this mean? "ok not present in reply or not true", but the JSON object referenced in the rest of the message clearly has the OK message (JField(ok,JBool(true))?
I found this confusing.
What was even more confusing was that the object was actually saved to CouchDB successfully, as I could find it using the Futon web interface.
The fact that the object was actually persisted, and the error message suggested that there was a bug in response parsing code within Lift. I wasn't that familiar with the Lift code base, so after numerous tries,
I found the thread that suggested someone else was having a similar problem ( http://groups.google.com/group/liftweb/browse_thread/thread/15d17b9348d4d88e#)
What was the problem? Unfortunately, CouchDB Record implementation didn't follow the work done on other parts of lift framework. The culprit is the following code in CouchDB Record implementation ( net.liftweb.couchdb.Database.scala file):
Listing 4:
private[couchdb] object DatabaseHelpers {
  /** Handles the JSON result of an update action by parsing out id and rev, updating the given original object with the values and returning it */
  def handleUpdateResult(original: JObject)(json: JValue): Box[JObject] =
    for {
      obj <- Full(json).asA[JObject] ?~ ("update result is not a JObject: " + json)
      ok  <- Full(json \ "ok" ).asA[JField].map(_.value).asA[JBool].filter(_.value) ?~ ("ok not present in reply or not true: "+json)
      id  <- Full(json \ "id" ).asA[JField].map(_.value).asA[JString].map(_.s)    ?~ ("id not present or not a string: " + json)
      rev <- Full(json \ "rev").asA[JField].map(_.value).asA[JString].map(_.s)    ?~ ("rev not present or not a string: " + json)
    } yield updateIdAndRev(original, id, rev)

Unfortunately, this change hasn't propagated to CouchDB Record code, and at the time of the writing, there is an open ticket for this bug ( http://www.assembla.com/spaces/liftweb/tickets/961-saving-to-couchdb-doesn-t-fetch-the-document--id--property)
Since DatabaseHelpers is the private scala object, it cannot be extended - so you have 2 options:
1. Checkout liftweb, fix the source, build it and use the custom built artefacts in your project. Of course, you'll lose ability to follow future releases of liftweb until the bug is fixed - but hopefully that won't be for too long
2. Override the methods that depend on it, by injecting the correct implementation.
You can easily overwrite the post method on the Database class, which is used by CouchRecord, which all of your CouchDB-mapped objects extend:
Listing 6:
val couch = new Database("",5984, "lift-test") {
      override def post(doc: JObject): Handler[Box[JObject]] = {
        (JSONRequest(this) <<# doc) ># handleUpdateResultCorrected(doc) _
      def handleUpdateResultCorrecred(original: JObject)(json: JValue): Box[JObject] = {
        //bug-free implementation, as above
couch.createIfNotCreated(new Http())
CouchDB.defaultDatabase = couch

This solves the problem of storing new documents to CouchDB - however updates are not handled by this overridden method.
Updates are defined in the Document trait buried within the Database.scala, and this trait is then extend and used for composition quite a few times - not easy to follow, let alone to figure out the clean way to overrirde its behaviour.
You could argue that it is not a significant problem for updates, as you know the document's id and revision values up front, you do not need to confirm them from the response when it succeeds. However, you still have incorrect behaviour, and that is something I cannot live with :)
Rather then spending too much time figuring out how to cleanly override the post method in Document trait, I went down the hard route, checked out the code and fixed the bug, and from there on I used my own custom build lift-couchdb_2.9.0-2.4M3-Aleksa.jar

Now that we have basic tests passing, let's see how we can load the persisted object from CouchDB using its id.
We will continue working on the same test:
Listing 7:
test("Create a new User") {
   val userRecord = User.createRecord
   userRecord.fullName.set("Bob Bobic")
   val user: Box[User] = userRecord.save

   assert(user.isDefined, user.compoundFailMsg("User not defined"))

   assert(!user.open_!.id.get.get.isEmpty) #1
   assert(!user.open_!.rev.get.get.isEmpty) #2

   val loadedUser: Box[User] = User.fetch(user.open_!.id.get.get) #3
   assert(loadedUser.isDefined, loadedUser.compoundFailMsg("User not defined"))
   expect("Bob Bobic"){

First, we check that the ide and revision fields are now populated as expected (#1, #2).
And the we load the user using the existing id using the fetch method (#3) - it's that simple!
The assertions that follow prove that we loaded all user properties, and that we loaded the document with the latest (expected) revision.
NOTE: Because the CouchRecord's id field is optional, the get function returns the Option object, so we need another get to get to the actual id.

You can use the save function to update the existing record as well. The document to be updated must have id and rev fields set.
Listing 8:
val u2: User = loadedUser.open_!
val updatedUser: Box[User] = u2.save
expect("bobby") {
expect(loadedUser.open_!.id.get) {
And that's it. Even with few challenges from the Lift's CouchDb implementation, the persistence and fetching of Scala objects to/from CouchDB worked nicelly, using Lift's Record framework.

Learn how the world’s first NoSQL Engagement Database delivers unparalleled performance at any scale for customer experience innovation that never ends.


Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}