The 5-Minute Interview: Improving Processing Time With Neo4j
The 5-Minute Interview: Improving Processing Time With Neo4j
Quickly traversing relationships is right in Neo4j's wheelhouse. Marriott has used Neo4j to drastically increase the speed at which customers can search for hotel rooms.
Join the DZone community and get the full member experience.Join For Free
Built by the engineers behind Netezza and the technology behind Amazon Redshift, AnzoGraph™ is a native, Massively Parallel Processing (MPP) distributed Graph OLAP (GOLAP) database that executes queries more than 100x faster than other vendors.
When Marriott customers search for hotel rooms for certain dates in certain locations, Marriott provides pricing options that vary depending on a variety of factors – such as time of year. With a relational database, it could take up to minutes to yield pricing results, so they needed to find a faster solution.
In this week’s 5-minute interview (conducted at GraphConnect San Francisco
), Scott discusses how the Marriott team implemented Neo4j and uses it to provide accurate, rapid-fire pricing information for its 4,500 locations across the country.
Talk to us about how you use Neo4j at Marriott.
Scott Grimes: I’m the senior director of revenue management, which is the science behind determining the prices that come up on our website and on our mobile channel when you search for a hotel room in a specific city for a certain date range. We use Neo4j to power our pricing engine application, which is used by revenue managers to maintain a pricing strategy for things such as different seasons or group rates.
We also have a very sophisticated concept of “touch one, modify many.” This means we have a few fixed price points that serve as the starting point for other rate fluctuations. So a concierge access may be a certain dollar value or percentage above the regular rate, or an advance purchase discount might be a certain amount below the regular rate.
We take the information from a user that maintains seasons and rules, and transform it into fixed price points for every rate set for a certain property. This publishing process includes pre-computations and calculations which results in modified rates available for sale.
By the time a guest is searching for a certain property on specific dates, the price returned by the system is the fixed price that was already precomputed. The system takes rules or certain scenarios managed by a personal property and transforms that into a fixed price point for sale.
What made you choose to work with Neo4j?
Scott: The way we identified Neo4j and graph databases as an appropriate use case is interesting because it doesn’t fit into the six classic use cases Neo4j markets itself around, such as recommendation engines or ACL lists.
We were experiencing a pain point with our publishing process for certain properties, which were taking as long as minutes to process. We also encountered some backlogs that resulted in it taking hours to update prices. This was extremely problematic in certain scenarios, and we needed to be able to update our prices more quickly.
Some of this resulted from our relational database
, in which we had set up multiple levels of primary keys and foreign keys throughout the data model. In some scenarios, we were generating up to 30,000 SQL calls to process a single data structure.
The graph database allowed us to convert relationships into first-class entities so that we could navigate the graph much more quickly. We’ve been able to take our average processing time from over four minutes to about 13 seconds.
What have been some of the most interesting or surprising results you’ve seen while using Neo4j?
Scott: We initiated our project with Neo4j to benefit the performance of the application, but we saw some other benefits we weren’t expecting.
For example, since we implemented the application, performance has improved so substantially that we’ve actually seen a 300% growth in the volume of price changes generated by properties. There’s a qualitative factor that the performance of an application can yield better adoption of a system, which allows Marriott to be more sensitive to the market and update its pricing accordingly.
The other main benefit — which was also a bit of a surprise — is that our hardware requirements to operate the environment have been substantially reduced. With the Neo4j solution, we’re using an order of magnitude less CPU compute to process with the new system. This has allowed us to rightsize the environment and reduce our overall infrastructure cost by about 50%.
If you could start over with Neo4j, what would you do differently?
Scott: When I reflect back on our experience with Neo4j or look at some of the current best practices, I think we actually did a number of things the correct way. Neo4j was a new technology, and we had been using a traditional RDBMS solution but we wanted to explore graph databases.
We made several key architectural decisions that I would repeat if I had to start all over again. The first is that we implemented Neo4j as a cache of our RDBMS solution; we didn’t retire our relational database completely so that we could maintain a full system of record. Graph technology was new and somewhat unproven at the time, so keeping our RDBMS system on the backend allowed us to maintain security and provided us with some comfort that we could safely move forward with this new solution.
Before making the switch, we also did an eight-week proof of concept where we evaluated several different deployment implementation options. We evaluated server mode versus embedded mode, and Cypher
versus direct API versus unmanaged extensions. We were able to collaborate with the Neo4j support teams on the prototyping results to come up with a recommended deployment option, which is what we implemented.
Talk to me a bit about the other technologies you use in your application stack alongside Neo4j.
Scott: We have an Oracle RDBMS that is in concert, and we keep the data in sync between them so that Neo4j is actually implemented as a cache. If the Neo4j data was to get corrupted or lost, we could actually refresh all of the data to Neo4j from our RDBMS. This provides us with security because we still have that information stored in a database that we’ve had for many years within Marriott.
Something to keep in mind is that Marriott has about 4,500 properties within the chain, each with its own pricing strategy or structure. This means we have 4,500 independent graphs that don’t have relationships with each other.
In order to realize performance increases, we had to make sure that we had as much data in memory as possible. To do this, we created multiple clusters and divided the properties into thirds across the three different clusters. We wrote our own custom charting or partitioning component so that we know which property is on which cluster.
Now when an event happens, we can route it to the right location where the appropriate data is located. That was one of the key architectural components we had to implement, which provided a custom solution as we adopted Neo4j.
Anything else you’d like to add or say?
Scott: One of the benefits that Marriott got from being an early adopter of Neo4j is that we now have a proven technology that we have the opportunity to explore.
There are other use cases and scenarios within our company that may benefit from Neo4j, and we no longer have the anxiety associated with bringing a new technology in-house that could incur risk. We’ve done that in every use case, and now, we can look for other opportunities within Marriott where we can leverage Neo4j.
Published at DZone with permission of Rachel Howard , DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.