DZone Research: Developers and Databases: What You Need to Know
Developers need to know SQL and understand the different kinds of databases so they can select the right database for the problem they are working to solve.
Join the DZone community and get the full member experience.Join For Free
To gather insights on the current and future state of the database ecosystem, we talked to IT executives from 22 companies about how their clients are using databases today and how they see use and solutions changing in the future.
We asked them, "What advanced database knowledge or skills do developers need?" Here's what they told us:
- 1) May not need specialized new knowledge for our platform. 2) For more advanced build up better SQL knowledge. SQL has persisted through multiple generations. Community = software ecosystem. Signed on the trained user base. Take the complexity off of the developer.
- SQL is still the most useful and used language. Learn it and use it. Some programming is involved when writing apps to handle data for ML model or interface or display a map. NodeJS and Python for data scientists. What differs is how you access the data.
- Trend cross-functional teams with generalist developers writing database code changes. Don’t forget about the database. Learn how to be a better SQL developer. Break down the silo of just being an app developer. Be more consultative in reaching out to data architects or database administrators. Spend time educating yourself on the capabilities of data stores. Understand tools so you know the right tool to solve the problem.
- Tried and true understand a developer’s capabilities. Know SQL don’t leave it behind, think about adding some NoSQL without dropping SQL. You have a lot of leverage with people trying things and not succeeding. This is how you learn. Meetups and Forums are a great place to learn what’s working and what’s not working.
- SQL still very important. It's a core database skill. On the AI front developers have an advantage with Python. Can pick up skills. Java still popular from a code perspective from the analytics side. Node on the operations side. Locations being added to apps. Predictive stuff with location built in. Geospatial data types and functions become more mainstream.
- Machine learning, artificial intelligence, SQL, NoSQL.
- There are many kinds of developers, so it depends. Most development teams have a mix of different types of developers. My vision is actually that a typical enterprise developer shouldn’t need any advanced database knowledge — they should just think about solving the problem at hand and the data platform should handle the rest. That being said, I think that all developers, including front-end developers, benefit from understanding data organization principles, having a basic facility with SQL, and having a favorite data analysis tool that includes an ability to assess the quality of their data. All data is dirty, and serious applications need to accommodate this. If you are a data analyst, it’s time to get familiar with big data and machine learning facilities. You might not need them today, but sooner or later they’ll be part of your job.
Kinds of Databases
- 1) Understand conceptually the different kinds of databases and what are the most viable in each category. 2) Try out the technologies. Spin up a cloud sandbox and begin playing in minutes. Get hands-on with different databases. 3) Make use of all the good resources out there.
- Make sure work and investment in learning technology are able to leverage skill sets developed with new technologies. You don’t need to know something completely new. Extend what you are familiar with. There is a need for a standard that’s a descendant of SQL and MapReduce. Evolution of the learning curve. Be familiar with the general graph model at a conceptual level. Make sure you have the fundamentals down and pick up the other things as needed.
- It is important to understand the tradeoffs between consistency and availability as more and more databases become distributed in nature. This allows you to better decide what database strategy suits your needs and understand what the failure mode of your database looks like. Additionally, a deep and thorough understanding of indexing is vital to ensuring the smooth operation of your database. We'd go so far to say that thorough knowledge of indexing is a basic requirement for working effectively with databases.
- Know how to take advantage of the platform they are using. SQL on Teradata is difficult to optimize. Need to know how to optimize for the platform they are working on. Use Meet-Ups and Google to learn given the pace of change.
- Instead of advanced esoteric knowledge, developers should understand they can use multiple techniques and interfaces into the database. Unshackle the view that this is a database application. File access may make more sense augmented with streaming analytics.
- The best possible thing to do is to work on a production system for a particular database, see it in production, see how data models impact it. Think about failover, uptime, how things break, how things work. In the true sense of DevOps. While it can be painful, it will be worth it in the long term. You will end up producing and writing better code.
- Learn the operating systems and environments the databases fit on. There are whole networking technologies surrounding databases. With vast amounts of non-relational data, you need to know how to access — know systems and networking.
- It’s going to be a polyglot world. Move to data where it best fits for the use case. Learn the most efficient way to move and shape the data so it’s available as soon as possible and as transactionally correct as possible. Be well-versed in change data capture. Move changes from the database as efficiently as possible. Agility for any target you want to move data to.
- Look to see where you are going to end up as opposed to where starting and be thoughtful about that. Don’t start with the easiest. Think about growth and evolution. Not possible for developers to build apps and throw it over the wall, think about the lifecycle of the app and how you will operate it for the next three to five year. Pick solutions that are flexible and allow you to operate efficiently.
- Advanced database knowledge concepts around distributed systems. Developers don’t believe in physics. If two servers aren’t beside each other, it takes time for data to move. Don’t expect instantaneous access. Understand the physics and how distributed systems work and the computer science around managing that. Have an acknowledgment to eliminate so many classes of problems.
- Developers are required to deliver new functionality as quickly as possible to meet the ever-evolving business needs. This means that they need to learn and adopt new methodologies to development (agile development practices) as well as new tools and techniques to bring these new innovations to market faster — CI/CD, cloud etc. Databases have also evolved in the last few years. Developers need to understand the innovations that have taken place here and take advantage of it. With platforms, the power of SQL and schema flexibility of JSON are both brought together to offer ease of programmability. With deep integration in all popular programming languages (Spring framework for Java developers and Linq support for .Net), it makes Developers these days also need to think of deployment and especially scale, at the time of design and development. The choices they make at this stage has deep implications on how much the application would scale and how it can meet the needs of the business. So ease of deployment, cloud-friendly and multi-cloud considerations, seamless on-prem to on-cloud expansion, cross data center replication for 5 9s availability, accessing the same application from mobile devices all become considerations that developers need to think about when it comes to choosing the right database.
- They should probably learn about migration scripts — the scripts necessary to migrate databases from one version to the next. Other than that, if developers can write code in C++ and C#, they can write code in languages like SQL. It’s a relatively simple language to learn and there are lots of third-party tools out there that can help. If it sounds blasé, it’s not meant to be. In our 2018 Database DevOps Survey, 76% of respondents had developers in their team who work across both applications and databases. So database development is already seen as a part of modern software development and we see terms like ‘full stack developer’ being used more and more. As long as the tools they use for database development plug into and integrate with the same infrastructure they use for application development, it will become the norm.
- To implement our stack, developers can download and implement the open source stack and familiarize themselves quickly. The stack is comprised of four components: 1) A plugin-driven server agent for collecting and reporting metrics from other systems. We currently have 160+ (and growing) agents to bring metrics over. 2) A time-series database built from the ground up to handle high write and query loads. 3) Complete interface for our platform with a SQL-like language to query data. 4) A real-time streaming data processing engine which can analyze both batch and stream data to detect patterns and trigger alerts.
Here’s who we talked to:
Opinions expressed by DZone contributors are their own.