Over a million developers have joined DZone.

Making Big Data Analysis Accessible Throughout Your Company

Andy Granowitz, an engineer at Wagon, shares how to make SQL more accessible to those who may not be familiar with it and how that could increase productivity.

· Big Data Zone

Hortonworks DataFlow is an integrated platform that makes data ingestion fast, easy, and secure. Download the white paper now.  Brought to you in partnership with Hortonworks

Thanks to the likes of Google Analytics, Mixpanel, and others, it's easy to measure app engagement. The problem is these services only report on a predefined set of metrics and user segments. Also, it's hard to analyze other customer interactions, such as sales calls or support interactions, alongside app usage data. When you need more custom analysis or to combine product usage with other data sources, you'll need to query the raw data. I'm talking user IDs and timestamps.

There are a number of great modern ETL tools that centralize app engagement and other customer interaction data into a data warehouse like Redshift or BigQuery. Having all of your analytics data in Redshift and queryable by SQL is very powerful: It allows you to answer nearly any question you have of your data.

But there's a Big Problem™: Only some people know how to write SQL. Engineers and savvy analysts will be comfortable writing SQL, but less technical members of your team also need access to the data. What's the best way to make everyone at your company data driven, regardless of their SQL knowledge?

We've found three ways to to help everyone at your company benefit from SQL.

Make It Easy and Fun to Share SQL

Sharing queries can be an important, rewarding part of your team's culture.

At Wagon, we encourage this in two ways. We have a dedicated Slack channel for posting ad-hoc queries, data, and charts. Slack is a great place to show off your hard work! We also keep a repository of "source of truth" queries that are code reviewed, commented, and frequently updated. Having a place for ad-hoc sharing and more standard reports solved a lot of confusion.

Wagon SQL shared on Slack

Reuse and Modify Your Teammates' Work

Many questions can be answered with the same underlying SQL. Make it easier to browse other people's queries by having them in one place and with reasonable comments. We see a lot of people tweaking a query by changing a where clause to use a different customer ID or log line. If you're using data from another analytics service, many provide starter queries. Here are a few: Amplitude, Google Analytics, Segment, and Snowplow.

Learn some SQL

SQL isn't that hard to learn. We joke that you can double your salary by learning how JOIN works. We recommend Udemy's online SQL course or if you prefer in person, check out General Assembly's SQL bootcamp.

Hortonworks Sandbox is a personal, portable Apache Hadoop® environment that comes with dozens of interactive Hadoop and it's ecosystem tutorials and the most exciting developments from the latest HDP distribution, brought to you in partnership with Hortonworks.

big data,sql,slack,communication

Published at DZone with permission of Andy Granowitz, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}