To gather insights for DZone's Data Persistence Research Guide, scheduled for release in March, 2016, we spoke to 16 executives, from 13 companies, who develop databases and manage persistent data in their own company or help clients do so.
Here's who we talked to:
Satyen Sangani, CEO, Alation | Sam Rehman, CTO, Arxan | Andy Warfield, Co-Founder/CTO, Coho Data | Rami Chahine, V.P. Product Management and Dan Potter, CMO, Datawatch | Eric Frenkiel, Co-Founder/CEO, MemSQL | Will Shulman, CEO, MongoLab | Philip Rathle, V.P. of Product, Neo Technology | Paul Nashawaty, Product Marketing and Strategy, Progress | Joan Wrabetz, CTO, Qualisystems | Yiftach Shoolman, Co-Founder and CTO and Leena Joshi, V.P. Product Marketing, Redis Labs | Partha Seetala, CTO, Robin Systems | Dale Lutz, Co-Founder, and Paul Nalos, Database Team Lead, Safe Software | Jon Bock, VP of Product and Marketing, Snowflake Computing
What developers need to keep in mind when working with different databases is consistent with what they need to keep in mind when working with all technologies: use best practices that are already established, proven, and tested; don’t reinvent the wheel if you already have the right technology for the job; understand how the data will be used so it’s in the right data store and language for the required analysis; and, there’s no such thing as “one size fits all,” so don’t become too attached to a single solution.
Here's what we heard when we asked, "What do developers need to keep in mind when working with databases?":
Developers are used to looking at compiling code together. They’re spending less time writing their own code. The same philosophy applies to databases and connectivity. Use the best practices that are already established, proven, tested, and provide a quick response. Educate developers on the resources available to them. Node.JS has a large community with data sources and code snippets. We host Mongo internally so developers can reduce time to market with a solution that’s already been tested and proven.
Don’t reinvent the wheel if you already have the right technology. Look for opportunities to improve old technology. SQL used to be the traditional database software. During the growth of Big Data, SQL was seen as a legacy system, but people are seeing it still provides a lot of benefits, so they keep it as a core tool.
Have the right tool for the job. Don’t get married to the technology, but on the flip side, don’t go chasing every shiny new object.
Understand how data will be used so it is placed in the right data store and language for every type of data analysis. Not only are access methods specific to the data store, there’s a specialized language for the data you’re analyzing. Because databases and languages are specialized, you must understand the differences of each one and know the best database for the use case.
It’s not a constant language, it’s an ever-evolving language. Applications will dictate the need. We started with a standard query language. Now you need to know many different languages (e.g. SQL, Mapreduce, query languages for noSQL databases).
Developers want to experiment with new approaches. I recommend exploring: 1) JSON, 2) predictive analytics, and 3) geospatial. Predictive analytics generation one is the immediate consumption of real-time data for dashboards while generation two is feedback to business based on the analysis of data. Geospatial is the next frontier with things like ride sharing and the location of mobile phones and other devices. These three will be critical the next decade.
Know SQL, but learn as many other languages as you can. Learn what database solutions are right for the need. Understand the interplay between data. Bring more diverse technologies for analytics.
Know what databases are good at doing. What is the barometer for your data? What tech to use to generate a category; a platform with a community of support; a platform with solid projects and company behind it. Know the options for support. Ensure the platform has a global presence and a sufficient number of companies and users. Rule #1 - it takes five to seven years and millions of dollars to create a good data infrastructure. Rule #2 - there are no exceptions to rule #1. Databases are easy to build but they’re hard to build for the long-term. Do your due diligence.
There’s no such thing as "one size fits all." Each database has a purpose for being. Know the tools in the toolbox and what tool is right for the job. Always understand the tools and what tool to use for what job. Don’t chase every shiny object.
Keep an open mind. Ask the users what they need. Don’t be wed to a single solution. Think about scalability and usability. Don’t be afraid to build a front-end that's different from the backend if that fulfills the need of the project. Use microservices to protect the data. Have a clear focus around security from beginning to end.
Our goals are different than most—we want to read, query, and write everything supported by databases, especially spatial data, in a robust, high-performance way. Most database development is not as generic and focuses on specific schema and/or problem space. Be ready to embrace changes. Determining the right database for the job is exciting and challenging. We’re able to see what our customers are asking for to see what’s most needed. If you’re a developer, you’re living in the golden age of where you’ll be able to do things next year that were inconceivable this year.
Think about what data you will need and what the response time needs to be for that data. Think about your overall data strategy identifying hot, warm, and data that can wait until later. Hot and warm data will need to be served out of the fastest database possible. There will continue to be increasing demand for performance and accessibility.
As the project is being defined, you need to think about what you’re going to do with the data when the database is up and running. You might want to clone the data. Developers need to be thinking ahead regarding the need to clone or snapshot a database.
What other things do developers need to keep in mind when working on a database?