DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Generative AI has transformed nearly every industry. How can you leverage GenAI to improve your productivity and efficiency?

SBOMs are essential to circumventing software supply chain attacks, and they provide visibility into various software components.

Related

  • The Beginner's Guide To Understanding Graph Databases
  • How Do You Know If a Graph Database Solves the Problem?
  • Why Is SQL Knowledge Vital for Data Scientists? A Sneak Peek
  • Deploying CockroachDB on Kubernetes using OpenEBS LocalPV

Trending

  • Multiple Stakeholder Management in Software Engineering
  • Advanced Insight Generation: Revolutionizing Data Ingestion for AI-Powered Search
  • Testing the MongoDB MCP Server Using SingleStore Kai
  • Scaling Multi-Tenant Go Apps: Choosing the Right Database Partitioning Approach
  1. DZone
  2. Data Engineering
  3. Databases
  4. Problems Being Solved With Databases — Executives' Perspectives

Problems Being Solved With Databases — Executives' Perspectives

Databases are enabling companies to use data to inform real-time decisions about their business as well as to use predictive analytics to make better informed, real-time decisions.

By 
Tom Smith user avatar
Tom Smith
DZone Core CORE ·
Feb. 08, 16 · Analysis
Likes (2)
Comment
Save
Tweet
Share
22.6K Views

Join the DZone community and get the full member experience.

Join For Free

To gather insights for DZone's Data Persistence Research Guide, scheduled for release in March, 2016, we spoke to 16 executives, from 13 companies, who develop databases and manage persistent data in their own company or help clients do so.

Here's who we talked to:

Satyen Sangani, CEO, Alation | Sam Rehman, CTO, Arxan | Andy Warfield, Co-Founder/CTO, Coho Data | Rami Chahine, V.P. Product Management and Dan Potter, CMO, Datawatch | Eric Frenkiel, Co-Founder/CEO, MemSQL | Will Shulman, CEO, MongoLab | Philip Rathle, V.P. of Product, Neo Technology | Paul Nashawaty, Product Marketing and Strategy, Progress | Joan Wrabetz, CTO, Qualisystems | Yiftach Shoolman, Co-Founder and CTO and Leena Joshi, V.P. Product Marketing, Redis Labs | Partha Seetala, CTO, Robin Systems | Dale Lutz, Co-Founder, and Paul Nalos, Database Team Lead, Safe Software | Jon Bock, VP of Product and Marketing, Snowflake Computing

The macro-trend is that more data is being analyzed in real-time. The internet of connected things enables you to see how things interact. Applications are tending to use multiple databases to provide polyglot persistence.

Here's what we heard when we asked, "What problems are being solved with databases?":

  • Huge evolution is happening very rapidly. Relational databases with big iron are long gone. We are using distributed data and accessing information from different sources. Different databases provide different access. We may use Spark SQL for transactional data and then a warehouse for mining legacy data. Each database has a different look but they all need to have holistic access.

  • People are being more predictive with analytics. Databases used to be reporting engines. Now they support predictive enabling advertisers to know an audience’s receptivity to a banner ad, a retailer can use a customer profile to provide a special offer, predictive analytics can be used to route package systems from point A to point B by helping to determine how to staff and assign drivers.

  • Companies are using databases to learn about their business faster. Safeway uses Teradata and Alation for their loyalty card to market better, provide better service, predict what people will buy, and churn. Ebay is using it to understand how to instruct the website when presenting the customer an offer—an improved user experience (UX). Learning and measuring what people are doing to provide a better customer experience (CX). We track how people are using the software to improve it—one set of techniques providing better results.

  • The ability to test before making changes without disturbing the persistent data ensures that the customer experience (CX) improves rather than deteriorates.

  • New generation databases solely operate in memory (i.e. Apache Ignite and Spark) enabling data to be read as fast as possible. We expect data to be retrieved quickly. We solve problems by retrieving data faster than before. Need to solve for persistence for analytics. This enables us to do many things much faster.

  • The ability to get instant insights into what the business is experiencing. Oil and gas IoT drill bits let the drillers know when a drill bit is about to break. Advertisers can deliver the right ad to the right customer in 10 milliseconds. Banks have real-time risk management. Real-time logistics has led to on-demand ride sourcing.

  • Traditionally stored data and answered inquiries. Mesh with governance, audit, and meta data. The difference between traditional and database applications is increasingly blurred. Virtual machines and containers enable different types of databases like data stores for virtualization. Twenty years ago there were seven to ten applications by Microsoft. Now, most large companies are doing their own application development. Support is becoming broader. We need a storage platform for traditional and new forms.

  • A general class of problems tries to get information from individual pieces of data. Some queries grab individual pieces of data. The macro-trend is that more and more data will be analyzed in real time. Start to find more intelligent ways for recommendations, fraud detection, access rights, and IoT. How to deal with volumes of sensor data. Internet of connected things—what happens when you see how things interact, you can get into all of the systems and the necessary data and avoid those that it doesn’t need. Understand connections and data relationships in the graph. As databases proliferate, how do we keep track of the data in all of the places. What’s an original and what’s a copy?

  • People are noticing that different parts of an application will have different needs. Specialized databases do specific things better (i.e. graphing the database of a social network and crawling links). Other databases scale better. The trend is towards specialization with apps using two, three, or more databases for polyglot persistence. At the end of the day, all databases store, organize, and retrieve data. Query interfaces may differ (i.e. key value stores and robust languages).

  • Mobile, IoT, and distributed applications have led to the distributed nature of data. The cloud has solved some of the problems but also resulted in more distributed data. While it would be ideal for all data to be in a single location, this is no longer realistic. You must have data accessibility across the board. This raises issues around security and privacy. Keys and credentials are critical to managing data. Security is a difficult subject. The more you can tweak and customize to protect the data, the better (i.e. access tokens). Data is critical, don’t let it get outside of control zones. Implement enterprise level policies.

  • Organizations that implement this strategy benefit from improved performance in retrieving subsets of the data from large datasets. They can also reuse data for different purposes and perform querying, exploring, and mining to create value from data. They’re ultimately creating a shared operational picture, while providing different views of the same data to different users depending on their needs.

  • There are many use cases. Twitter timeline is based on the Redis time series database.

  • On the one end there are quick data use cases with non-relational clickstream data allowing data to be shared by multiple data science teams without copying from one source to another. Since this takes time, it must be tracked and costs more to store.

Database Relational database Data science

Opinions expressed by DZone contributors are their own.

Related

  • The Beginner's Guide To Understanding Graph Databases
  • How Do You Know If a Graph Database Solves the Problem?
  • Why Is SQL Knowledge Vital for Data Scientists? A Sneak Peek
  • Deploying CockroachDB on Kubernetes using OpenEBS LocalPV

Partner Resources

×

Comments

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • [email protected]

Let's be friends: