Primary and Post-Process Deduplication in NexentaStor

There's been a lot happening in the enterprise storage domain recently.  Last year EMC purchased DataDomain for over $2 billion and Dell just purchased Ocarina, a deduplication specialist.  DZone had a chat with Nexenta CEO Evan Powell about the growing interest in cost-saving data storage features like deduplication.  Nexenta is an open source storage software company that has had many recent successes in the market with its ZFS-based storage software.  The new Nexenta Core Platform 3.0 has already increased Nexenta's revenue by 351% thanks to affordability and cost-effective features like primary and post-process data deduplication.  

Analysts say storage now comprises 42% of all IT spending, making it the largest piece of most tech companies' budgets.  Powell has a joke proclaiming that if current rates of growth continue, in five years storage will comprise 105% of a company's budget.  Features like deduplication could slow down that trend, says Powell.  Deduplication is a data compression technique by which the storage software recognizes redundant data and maintains a single copy of the data for reference, eliminating unecessary copies.  

There are two types of deduplication that Powell describes: primary and post-process.  Primary deduplication happens the first time you write the storage, in realtime.  In Post-Process deduplication, you run the deduplication nightly and the speed of your system takes a hit while the software looks for redundant data.  Powell says that NetApp in particular, uses language in their advertisements that indicate both primary and post-process data 'dedup' capabilities, when in fact they (and Ocarina) only have post-process capability.  "You can't really run [their solutions] on primary data because it slams your array "  Nexenta's newly released 3.0 platform however, has both kinds - and its commercial version costs up to 80% less than other vendors' software.  

Nexenta has an Open Core business model with Sun's ZFS and the OpenSolaris kernel at its core.  Nexenta's platform is comparable to a Network Attached Storage Solution and it also does Unit Block Storage.  It comprises the foundational layer where the data comes to rest.  The Nexenta Core platform integrates its Solaris features with Ubuntu userland.  Their software runs on industry standard hardware and is built on top of OpenSolaris to comprise an open source network storage solution called NexentaStor and a commercial version.  The most recent release has support for ZFS deduplication and Crossbow network virtualization.    

Powell says that it took about an hour to migrate data from OpenSolaris systems over to Nexenta, whereas a migration between EMC and NetApp, for example, would take months.  This is the problem with proprietary data formats, he says.  Legacy storage systems built 15 years ago aren't flexible enough to take advantage of new trends like virtualization and cloud computing, unlike NexentaStor, which can handle plenty of VMs. 

Nexenta also sponsors the largest non-Oracle distribution of OpenSolaris at nexenta.org.