Originally Written by Steve Cellini
In the blog post announcing the GA of NuoDB Swifts Release 2.1, I said this:
Our plan is to always be shipping, with updates to 2.x being released on a quarterly basis in 2015.
And so, here we are, right on time: Welcome to NuoDB Larks Release 2.2!
Larks is another milestone in the 2.x release trajectory established over the past year to reinforce NuoDB's ability to play a central role in an enterprise's data architecture. As with previous releases, we've added important features as well as delivering improved scalability, performance, and hundreds of improvements based on customer scenarios (such as our own NuoDB-powered Drupal site, nuodb.com). Below the waterline - where it really matters - we continue to invest heavily in the infrastructure of our storage system, transaction layer, and other aspects of our distributed system.
With this release we're also highlighting a theme that you’ll see us revisiting and investing in again in coming releases: Continuous Availability with emphasis on availability, resiliency, and disaster recovery (DR).
"Availability" refers to the readiness of a system to support operations... a "highly available" system is one in which downtime is "rare". At NuoDB, we use the term "continuous availability" to connote how NuoDB's distributed architecture enables a NuoDB database to be configured at an arbitrary level of redundancy, without any level of downtime. In Swifts (NuoDB release 2.1), we showed how NuoDB's unique architecture can support simultaneous full-speed operational and analytic workloads (see our HTAP White Paper). The same architectural characteristics also enable achieving this level of availability and throughput during, say, software updates, schema changes, or database backups. Seth Proctor, NuoDB CTO, addresses these concepts in our latest white paper.
Resiliency (and Automation)
With this release, we've augmented NuoDB's architectural advantage for availability with capabilities in the area of "resiliency", which we define broadly as the system's ability to detect, diagnose, and recover from conditions that represent trending or actualized conditions of reduced throughput, availability, or errors.
While resiliency is a characteristic that touches on all aspects of the system, our database automation framework - with database templates supporting database SLAs - is key to our vision for implementing a fully managed, self-healing "Autopilot" capability for the data center. For instance: automation resiliency envisions the ability of the system to detect increasing transactional load on a database and add another NuoDB Transaction Engine in response.
In Larks, we've enhanced our automation capabilities with two important new resiliency features: an incremental enforcer retry policy, and predefined database health alarms.
Let's explain the incremental enforcer retry policy by starting with an example. Suppose your goal is to “operate a database that is minimally redundant”. You’d apply the supplied “Minimally Redundant” NuoDB automation template when you create the database. NuoDB does the rest, spinning up Transaction Engines (TEs) and Storage Managers on a minimum of two hosts. Automation also ensures that if a host goes down for any reason, another host will be automatically selected on which to start replacement TEs and Storage Managers (SMs) as necessary to sustain the database template SLA.
The new incremental enforcer retry policy mentioned above allows more fine-grained control over how NuoDB will enforce the restarting of TE and SM engines in service of an automation template SLA. By default, a TE or SM engine will be restarted as necessary to enforce the SLA. With this new policy, however, you can specify the minimum accumulated uptime that must be observed before an engine can be considered for restart, the maximum number of times an engine should be re-started, and the initial delay (in milliseconds) to wait before restarting an engine the first time. The "incremental" part of the new policy comes into play because that initial delay is doubled the second time a process must be restarted, and doubled again the third time, and so on, until the maximum retry limit is reached. This provides a powerful and flexible approach to resiliency for your particular configuration because it lets administrators decide describe when an engine on a particular node should be classified as ”failing” and thus unavailable to the database any longer. See the documentation for more info.
A related addition in Larks are Predefined alarms for NodeFailed, EnforcerNodeBackoff, and EnforcerNodeBackoffMaxRetried: You've always been able to activate alarms for database events such as NodeFailed, to help you track the health of your database and domain. Now, we've made that easier by shipping the product with these alarms already enabled.
We've made many other improvements in Automation for usability, flexibility, and to address specific user feedback. For instance, it's now easier to inject database options and Host Tag constraints into a Template at the Transaction Engine and Storage Manager level, and you can use regular expressions in NuoDB Manager to search for tags.
Disaster Recovery Preview
Larks also offers a preview of an important new NuoDB scenario: where you can configure a database to include Storage Manager engines in a separate data center or region, for data backup and recovery purposes. This provides database integrity in the event that your primary region becomes unavailable due to planned or unplanned outtages, ensuring no loss of data.
Beyond Disaster Recovery, here's a summary of the other new features and capabilities of this release:
- New SQL optimizations and statistics: The SQL optimizer, which received a major boost in Swifts Release 2.1 through the addition of cost-based heuristics, has been enhanced to use histograms to find more efficient join plans. New statistics have been added for the number of distinct values per index prefix, and the optimizer now uses “average key size” to find much more efficient table access paths. A special new optimization finds optimal plans for queries with combinations of big IN-lists that can use indexes. ORDER BY ... LIMIT queries are multiple times faster now through a query strategy that avoids sorting the whole result.
- The DELETE FROM clause has been extended to support ORDER BY ... LIMIT so that it is possible to limit the number of rows deleted.
- LAST_INSERT_ID(): A SQL new System Information Function that returns the last automatically generated ID for a column declared with GENERATED ALWAYS AS IDENTITY or GENERATED BY DEFAULT AS IDENTITY
- User defined functions (UDF), in SQL and Java
We're really excited about UDFs. This new capability makes it easy to encapsulate commonly used code into functions for re-use. A scalar UDF can be used as a list item in a select query or a view or in WHERE clause, and a table function UDF can be used in a FROM clause.
Here's an example showing the definition of a function of type TABLE in a FROM clause. This uses an INSERT INTO...SELECT... statement with a RETURN statement.
SET DELIMITER @ CREATE FUNCTION fnc_tbl_get_teams (i_PlayerId STRING) RETURNS TABLE fnc_tbl_get_teams (PlayerId STRING, year STRING, position STRING, team_name STRING, stint STRING, games_played integer,conference STRING, Division STRING) AS INSERT INTO fnc_tbl_get_teams SELECT s.playerId, s.year, s.position, t.name, s.stint, s.gamesplayed, t.conferenceid,t.divisionid FROM scoring s, teams t WHERE s.playerId = i_PlayerId AND s.year = t.year AND s.teamid = t.teamid; RETURN; END_FUNCTION; @
And here's how it might be used:
SET DELIMITER ; SELECT p.playerid,p.firstname,p.lastname,f.year,f.position,f.team_name,f.stint,f.games_played,f.conference,f.division FROM players p, fnc_tbl_get_teams(p.playerid) f WHERE p.firstnhl = 2010 and p.lastnhl = 2011 ORDER BY p.playerid, f.year, f.stint;
PLAYERID FIRSTNAME LASTNAME YEAR POSITION TEAM_NAME STINT GAMES_PLAYED CONFERENCE DIVISION --------- ----------- ------------- ----- --------- --------------------- ------ ------------- ----------- --------- adamlu01 Luke Adam 2010 L Buffalo Sabres 1 19 EC NE adamlu01 Luke Adam 2011 L Buffalo Sabres 1 52 EC NE aulieke01 Keith Aulie 2010 D Toronto Maple Leafs 1 40 EC NE aulieke01 Keith Aulie 2011 D Toronto Maple Leafs 1 17 EC NE aulieke01 Keith Aulie 2011 D Tampa Bay Lightning 2 19 EC SE
For more info on UDFs, see the doc.
Documentation and Examples
In our last release, NuoDB Swifts Release 2.1, we completely revamped our documentation. With Larks, we continue that effort with improved and refined sections for:
- C++ API
- Database Status
- Handling Error Conditions in Transactions
... and many other areas. As before, we welcome your feedback on how we're doing here.
We've also continued to make it easier to understand and evaluate NuoDB with a number of improvements to the Storefront sample application, to help those new to NuoDB learn more about what the product can do. Check it out - it's shipped with the product and can be found in the NuoDB Admin Center. A complete guided tour, which includes Storefront and other samples, can be found here.
Updates to System Requirements
- Java: When NuoDB Swifts Release 2.1 was released last Fall, we announced that we'd be deprecating support for Java 1.6. With this release, Java 1.6 is no longer supported for NuoDB hosts; please use Java 1.7 or Java 1.8. Client applications are unaffected. See the documentation for more info.
- We've added support for RHEL v7 and CentOS
- Docker: NuoDB has always worked in Docker configurations, but there are some interesting configuration and deployment considerations to take into account. NuoDB CTO Seth Proctor describes his initial experiences in an excellent blog post.
For more information about Larks, please visit the release notes.
As always, a big shout-out to our customers and partners; NuoDB Larks is a better release because of your help!
See you in three months, when NuoDB Robins hits the streets. We think you'll like it :)