DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
  1. DZone
  2. Data Engineering
  3. Data
  4. Auction and Bidding Applications with Document-Based NoSQL

Auction and Bidding Applications with Document-Based NoSQL

Oren Eini user avatar by
Oren Eini
·
Dec. 18, 11 · Interview
Like (0)
Save
Tweet
Share
8.42K Views

Join the DZone community and get the full member experience.

Join For Free
in my previous post , we dealt with how to model auctions and products, this time, we are going to look at how to model bids.

before we can do that, we need to figure out how we are going to use them. as i mentioned, i am going to use ebay as the source for “application mockups”.  so i went to ebay and took a couple of screen shots.

here is the actual auction page:

image


and here is the actual bids page.

image


this tells us several things:

  • bids aren’t really accessed for the main page.
  • there is a strong likelihood that the number of bids is going to be small for most items (less than a thousand).
  • even for items with a lot of bids, we only care about the most recent ones for the most part.

this is the auction document as we have last seen it:

{
   "quantity":15,
   "product":{
      "name":"flying monkey doll",
      "colors":[
         "blue & green"
      ],
      "price":29,
      "weight":0.23
   },
   "startsat":"2011-09-01",
   "endsat":"2011-09-15"
}

the question is where are we putting the bids? one easy option would be to put all the bids inside the auction document, like so:

{
   "quantity":15,
   "product":{
      "name":"flying monkey doll",
      "colors":[
         "blue & green"
      ],
      "price":29,
      "weight":0.23
   },
   "startsat":"2011-09-01",
   "endsat":"2011-09-15",
   "bids": [
     {"bidder": "bidders/123", "amount": 0.1, "at": "2011-09-08t12:20" }
   ]
}

the problem with such an approach is that we are now forced to load the bids whenever we want to load the auction, but the main scenario is that we just need the auction details, not all of the bids details. in fact, we only need the count of bids and the winning bid, it will also fail to handle properly the scenario of high interest auction, one that has a lot of bids.

that leave us with few options. one of those indicate that we don’t really care about bids and auction as a time sensitive matter. as long as we are accepting bids, we don’t really need to give you immediate feedback. indeed, this is how most auction sites work. they give you a cached view of the data, refreshing it every 30 seconds or so. the idea is to reduce the cost of actually accepting a new bids to the minimum necessary. once the auction is closed, we can figure out who actually won and notify them.

a good design for this scenario would be a separate bid document for each bid, and a map/reduce index to get the winning bid amount and big count. something like this

     {"bidder": "bidders/123", "amount": 0.1, "at": "2011-09-08t12:20", "auction": "auctions/1234"}
     {"bidder": "bidders/234", "amount": 0.15, "at": "2011-09-08t12:21", "auction": "auctions/1234" }
     {"bidder": "bidders/123", "amount": 0.2, "at": "2011-09-08t12:22", "auction": "auctions/1234" }

and the index:

from bids in docs.bids
select new { count = 1, bid.amount, big.auction }

select result from results
group result by result.auction into g
select new 
{
   count = g.sum(x=>x.count),
   amount = g.max(x=>x.amount),
   auction = g.key
}

as you can imagine, due to the nature of ravendb’s indexes, we can cheaply insert new bids, without having to wait for the indexing to work. and we can always display the last calculated value of the auction, including what time it is stable for.

that is one model for an auction site, but another one would be a much stringer scenario, where you can’t just accept any bid. it might be a system where you are charged per bid, so accepting a known invalid bid is not allowed (if you were outbid in the meantime). how would we build such a system? we can still use the previous design, and just defer the actual billing for a later stage, but let us assume that this is a strong constraint on the system.

in this case, we can’t rely on the indexes, because we need immediately consistent information, and we need it to be cheap. with ravendb, we have the document store, which is acidly consistent. so we can do the following, store all of the bids for an auction in a single document:

{
   "auction": "auctions/1234",
   "bids": [
     {"bidder": "bidders/123", "amount": 0.1, "at": "2011-09-08t12:20", "auction": "auctions/1234"}
     {"bidder": "bidders/234", "amount": 0.15, "at": "2011-09-08t12:21", "auction": "auctions/1234" }
     {"bidder": "bidders/123", "amount": 0.2, "at": "2011-09-08t12:22", "auction": "auctions/1234" }
    ]
}

and we modify the auction document to be:

{
   "quantity":15,
   "product":{
      "name":"flying monkey doll",
      "colors":[
         "blue & green"
      ],
      "price":29,
      "weight":0.23
   },
   "startsat":"2011-09-01",
   "endsat":"2011-09-15",
   "winningbidamount": 0.2,
   "bidscount" 3
}

adding the bidscount and winningbidamount to the auction means that we can very cheaply show them to the users. because ravendb is transactional, we can actually do it like this:

using(var session = store.opensession())
{
  session.advanced.optimisticconcurrency = true;
  
  var auction = session.load<auction>("auctions/1234")
  var bids = session.load<bids>("auctions/1234/bids");
  
  bids.addnewbid(bidder, amount);
  
  auction.updatestatsfrom(bids);
  
  session.savechanges();
}

we are now guaranteed that this will either succeed completely (and we have a new winning bid), or it will fail utterly, leaving no trace. note that addnewbid will reject a bid that isn’t the higher (throw an exception), and if we have two concurrent modifications, ravendb will throw on that. both the auction and its bids are treated as a single transactional unit, just the way it should.

the final question is how to handle high interest auction, one that gather a lot of bids. we didn’t worry about it in the previous model, because that was left for ravendb to handle. in this case, since we are using a single document for the bids, we need to take care of that ourselves. there are a few things that we need to consider here:

  • bids that lost are usually of little interest.
  • we probably need to keep them around, just in case, nevertheless.

therefor, we will implement splitting for the bids document. what does this means?

whenever the number of bids in the bids document reaches 500 bids, we split the document. we take the oldest 250 bids and move them to historical bids document, and then we save.

that way, we have a set of historical documents with 250 bids each that no one is ever likely to read, but we need to keep, and we have the main bids document, which contains the most recent (and relevant bids. a high interest auction might end up looking like:

  • auctions/1234 <- auction document
  • auctions/1234/bids <- bids document
  • auctions/1234/bids/1 <- historical bids #1
  • auctions/1234/bids/2 <- historical bids #2

and that is enough for now i think, this post went on a little longer than i intended, but hopefully i was able to explain to you both the final design decisions and the process used to reach them.

thoughts?

Document application NoSQL

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • DeveloperWeek 2023: The Enterprise Community Sharing Security Best Practices
  • 5 Best Python Testing Frameworks
  • Chaos Engineering Tutorial: Comprehensive Guide With Best Practices
  • Secure APIs: Best Practices and Measures

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: