DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
View Events Video Library
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Modern Digital Website Security: Prepare to face any form of malicious web activity and enable your sites to optimally serve your customers.

Containers Trend Report: Explore the current state of containers, containerization strategies, and modernizing architecture.

Low-Code Development: Learn the concepts of low code, features + use cases for professional devs, and the low-code implementation process.

E-Commerce Development Essentials: Considering starting or working on an e-commerce business? Learn how to create a backend that scales.

Related

  • Build Quicker With Zipper: Building a Ping Pong Ranking App Using TypeScript Functions
  • How to Build a URL Shortener Web App With Flask Framework
  • Auto-Scaling a Spring Boot Native App With Nomad
  • Android Cloud Apps with Azure

Trending

  • Software Architecture Design Patterns for Serverless System
  • Navigating Software Leadership in a Dynamic Era
  • From Docker Swarm to Kubernetes: Transitioning and Scaling
  • Top 11 Project Management Hacks for Software Project Managers
  1. DZone
  2. Data Engineering
  3. Big Data
  4. Database Scaling Made Simple

Database Scaling Made Simple

For every problem technology poses, it offers many possible solutions. Here's how to take much of the sting out of scaling databases.

Darren Perucci user avatar by
Darren Perucci
·
May. 24, 17 · Opinion
Like (2)
Save
Tweet
Share
6.3K Views

Join the DZone community and get the full member experience.

Join For Free

Your database scaling strategy should match the way the system is used and the extent to which the database's nodes are distributed.

Not so long ago, the solution to bulky database performance was to throw more hardware at it — faster servers and memory, more network bandwidth, and boom! Your database is back to bullet-train speeds.

Any system administrator will tell you that databases aren't what they used to be. For one thing, today's databases are much more widely distributed, reaching end points of all types: phones, tablets, controllers, sensors, even appliances. For another thing, a single database is more likely to run on multiple platforms and interface with systems of all types. Last but not least, the amount of data stored in a typical production database dwarfs the typical storage of just a few years ago.

So, you've got more data than ever that's integrating with more devices and platforms than ever and reaching more device types than ever. Add to this the modern demands of real-time analysis and zero downtime. It's enough to make a DBA consider a midlife career change.

But before you start thinking about life as a karaoke repairman, remember that for every problem technology poses, it offers many possible solutions. The trick is to find the answer for your technical predicament and apply it to best effect. The solution to many database performance glitches is scalability. The challenge is that traditional RDBMSs are easy to scale up by adding more and faster CPUs and memory, but they're notorious for not scaling out easily, which is a problem for today's widely distributed systems. Here's how to take much of the sting out of scaling databases.

The prototypical example of a scale-out system is a web app, which expands across low-cost servers as the number of users increases. By contrast, an RDBMS is designed to scale up: improve performance by adding processing power and more/faster memory.

Web apps are noted for scaling out to commodity servers as demand increases, while RDBMSs scale up by adding more and faster processor and memory, but less well as the number of users goes up. Source: Couchbase.

Forget the Backend; Analytics Now Happen in the Real-Time Data Stream

If you're still analyzing data in batch mode on the back end, you could be further behind the competition than you think. InfoWorld's Andrew C. Oliver explains in a July 25, 2015 article that systems based on streaming analytics may cost more initially, but over time they make much better use of resources. One reason real-time analytics is more efficient is that you're not re-analyzing historic data the way you do with batch-mode analysis.

In a Dr. Dobbs article from November 2012, Nikita Shamgunov distinguishes the scale-out requirements of OLTP and OLAP databases: OLTP is used for real-time transaction processing, and OLAP accommodates analysis of large amounts of aggregate data.

Online analytical processing emphasizes deep dives into large pools of aggregate data, while online transaction processing focuses on many simultaneous data transactions with fewer data demands and shorter durations. Source: Paris Technology.

Scaling OLAP databases relies on these four characteristics:

  1. Columnar indexes and compression (not specifically scaling out, but they reduce CPU cycles per node).
  2. Intra-parallel query execution (a scale-up partitioning technique that executes subqueries in parallel).
  3. Shared-nothing architecture (prevents a single point of contention).
  4. Distributed query optimization (sends a single SQL query to multiple nodes).

Here are the four considerations for ensuring OLTP databases scale well:

  1. Don't rely on columnar indexes because OLTP is real-time, and its data records are accessed randomly, which reduces the effectiveness of indexes.
  2. Parallel query execution is likewise less effective in OLTP because few queries require processing large amounts of data.
  3. Data in OLAP systems is generally loaded in batch mode, but OLTP data is ingested in real-time, so optimizing bursty traffic is less beneficial in OLTP.
  4. In OLTP, uptime is paramount; OLAP systems are more tolerant of downtime.

Simple management of SQL, NoSQL, and in-memory databases is a key feature of the Morpheus next-gen Platform as a Service software appliance built on Docker containers. Morpheus supports management of the full app and database lifecycle in a way that accommodates your unique needs. Provisioning databases, apps, and app stack components is simple and straightforward via the intuitive Morpheus interface.

The range of databases, apps, and components you can provision includes MySQL, MongoDB, Cassandra, PostgreSQL, Redis, ElasticSearch, and Memcached databases; ActiveMQ and RabbitMQ messaging; Tomcat, Java, Ruby, Go, and Python application servers and runtimes; Apache and NGINX web servers; and Jenkins, Confluence, and Nexus app stack components.

Database Scaling (geometry) Data stream app

Published at DZone with permission of Darren Perucci, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Build Quicker With Zipper: Building a Ping Pong Ranking App Using TypeScript Functions
  • How to Build a URL Shortener Web App With Flask Framework
  • Auto-Scaling a Spring Boot Native App With Nomad
  • Android Cloud Apps with Azure

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends: