DZone Research: Database Concerns

To gather insights on the current and future state of the database ecosystem, we talked to IT executives from 22 companies about how their clients are using databases today and how they see use and solutions changing in the future.

We asked them, "What are your biggest concerns regarding databases today?" Here's what they told us:

Proliferation

• It’s hard to track all of the new databases that come out every week. Today there are so many technologies. It’s challenging to decide on a technology that will be around over the long haul. What do I bid on? Acknowledge that you will likely be changing database technologies with a more frequent cadence than you would in the past. Automation tool that can capture needs. 
• A couple of tensions: 1) how much does the database do versus the processing layer. Do you support ML or do you need Apache Spark? 2) Data infrastructure is just getting fragmented. Some specializations will converge. Kafka will not be part of our solution. How do you tie all of these pieces together? SQL is the common glue. 3) Understanding the difference between the interface and how it is ultimately integrated. Want a database to slice and dice data in a particular way. 4) Cloud versus edge devices. A lot of use cases are just getting connected that may not be feasible to send the data back or you may not want to. Only send anomalies. How does the database handle cloud analysis versus edge analysis?
• There are a lot of databases out there. A lot of confusion. What to use and why. Choices and tradeoffs are not well defined.
• No. The more the merrier. Understand the nuances, select the right database and derive value from it. Users are becoming more mature about what to use and when. Look at DB-engines.com to help determine what the right database is for the problem you are looking to solve.
• Not understanding or aligning problems with the right kind of database for the right kind of solutions. So much activity with different databases. Many databases are not yet mature. This can lead to innovation. There is still a lot of muscle memory for relational databases. People take for granted the database is reliable. Take with a grain of caution – revisit assumptions always. Kurt Monash – “The cardinal rules of DBMS development Rule 1: Developing a good DBMS requires 5-7 years and tens of millions of dollars. That’s if things go extremely well. Rule 2: You aren’t an exception to Rule 1.”
  

Security

• It is concerning that so many developers and administrators consider security as an afterthought. For example, in 2017 tens of thousands of misconfigured MongoDB databases were subject to ransomware attacks because authentication was never switched on. Understanding the proper and secure way to configure your database is vital. Hosted database solutions provided by experts can certainly help, but there's no substitute for a good understanding of and respect for security.
• As a business security. GDPR rolling out all the security capabilities are becoming more important – encryption, access, auditing AI models, auditing test and training data.
• 1) One of my biggest concerns is how the role of the DBA will evolve with the rise of PaaS and managed service offerings. 2) I’m worried that increasing concerns about data security will erase any agility gains made through new technology advances. 3) Finally, there’s a skills gap. New database technologies are proliferating so quickly that enterprises can’t find the talent to properly take advantage of them.
• Privacy. It’s depressing to read yet another news story about a huge data breach, or a tale of more data being misused. As we collect, store, and analyze more and more data, privacy needs to be top of mind in order to reassure customers that we’re doing a good and necessary job with their data, and not simply trying to monetize it.
  

Other

• They’re fairly narrow in terms of the workloads they are optimized for. Coordinating across databases and silos is where the difficulty is. The scale and distributed nature of the data are what you need as the building block. The accent is moving from the processing to the data.
• On a broad scale nothing systemic. Technologies are dealing with bugs that can be resolved in time. We’re landing in a place where we’re doing things at scale. Use the right tool for the right job. What makes the most sense. When multi-model stuff you may not get the best of both worlds. You may have trouble with performance and scalability.
• Vendors and the big guys dictate how the environment will be. Many other smaller providers blur the boundary between Windows and Linux, this leads to challenging integration. The industry needs to get together and establish standards for getting data in and out of databases. It would be nice to have a database illustrator to operate across all platforms. It takes a tremendous amount of effort today.
• Healthy with so much activity and growth in the interest of meeting users’ needs better. Customers are getting smarter. Innovation is going on with stronger businesses that will drive more alignment. There will be consolidation across databases.
• Companies that play on the fact developers may not have sophistication with distributed systems and make blatantly false claims. Slapping some tired technology on top of old relational technology and saying its better when it’s not. This causes more project failure.
• Data patterns, data velocity, volume, and variety has evolved significantly in the last decade. General purpose databases that were popular and “did the job” in the 70s through 90s are just not sufficient anymore. Specialized data patterns and use cases require specialized architectures. Employing the same old general-purpose database for a variety of fast/big data use cases simply is not working.
• As the industry matures, the lack of standardization across technology choices will slow down the adoption of new NoSQL technologies. A lack of awareness of all that these new technologies can solve is one of the biggest concerns.
• I’m worried about the general perception that databases are a commodity -  and by extension that they aren’t innovating or even that they aren’t worth thinking about. The rise of NoSQL was 10 years ago but I still run into many people that reflexively use an inferior database or one that’s mismatched to their situation. Then they live with the limitations and that reinforces the perception. In fact, we’re in a time of enormous innovation in data management. I’d like people to know that because they can get so much more value from their data if they think about it creatively. Another facet of this is that I worry that people don’t consider the lifecycle aspects of their databases and how much vendor support stability, and service matter. I find myself in rescue situations around this, sometimes because a huge vendor won’t help their customer, sometimes because a startup is causing them too much churn. It’s gratifying to be able to help, but I hate seeing that people have gotten into trouble in the first place.

Here’s who we talked to: