Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Simplifying Existing Workflows

DZone's Guide to

Simplifying Existing Workflows

· Database Zone ·
Free Resource

Running out of memory? Learn how Redis Enterprise enables large dataset analysis with the highest throughput and lowest latency while reducing costs over 75%! 

Jason Harmon, Chief Platform Officer at Typeform, shares his thoughts on ways that developers can simplify and enhance overall productivity. Typeform is a Barcelona-based company that combines the experience of real-life conversations with the data collection capability of web forms. 

For developers, productivity can remain stagnant due to a lack of integration tools and processes. And for those constantly working against deadlines, it’s common for user experience (UX) to get overlooked.

Jason shares his thoughts on what's missing in today's modern workflow for developers, why speed and UX don't need to be seen as mutually exclusive, and what the future of data collection looks like.

How are you and your company involved in data collection and analysis?

Typeform is a cloud-based SaaS web application that uses a conversational approach to data collection. The approach is simple: businesses and developers can easily create forms to collect data for a variety of business uses. Our user interface offers a decluttered, one-question-at-a-time, seamless experience that gives brands and developers a way to collect meaningful data from the people they’re trying to reach.

What are the keys to a successful collection and analysis strategy?

In order to collect rich, meaningful data, both the process of collecting it and the platform itself need to be designed for real people. This entails combining the flow of real-life conversations with traditional web forms in order to create a more natural user experience. The other key to success is to ensure data collection tools can be easily integrated into other platforms so workflows are not interrupted.

How have databases changed to handle data collection and analysis?

Databases are moving toward a more natural way of interacting with people. Traditional relational databases present scaling problems in storage and retrieval of this data in a normalized fashion. As such, storing the responses as a set of objects provides a more performant way to retrieve those results.

What are the technical solutions you, or your clients, use for this process?

We use MySQL for storing the definition of the questions and ElasticSearch to store a read-only version of responses. This aids our ability to quickly retrieve and analyze responses, as well as the potential for horizontal scaling. We also use Amazon RedShift to extract deeper insights into the wide variety of use cases being solved by our product.

What are real-world problems you, or your clients, are solving (use cases)?

We’re helping our customers collect valuable information about the people they want to know more about. This can be done in a variety of ways, from procedural forms to casual interactions. Some use cases include things as simple as Trello using Typeform for company culture initiatives all the way to starting a business.

What are the most common issues you see companies having with data collection and analysis?

On one side, companies can rely too heavily on artificial intelligence (AI) and chatbots, which leads to a disengaged connection to its customers. With this approach, the human side of interactions gets lost, data isn’t meaningful, and customer insights fall through the cracks. In other cases, it’s easy to put all your focus on user experience. When competing with UX giants, startups’ backends can fall behind from devoting a majority of their time to design. There needs to be a healthy balance between the two in order to get the best results.

Where do you think the biggest opportunities are in the evolution of data collection and analysis?

The biggest opportunity right now facing data collection is providing an amazing UX. But to make the UX simplified and engaging requires a technical backend. If you have an engaging platform that draws people in, you’re going to get the results you want. On top of that, reaching people in ways that are convenient to them is going to be a major opportunity. This includes mobile phones and tablets, but also voice assistants such as Amazon Alexa and Google Home.

What are your biggest concerns regarding data collection and analysis today?

Security and privacy of stored responses is a constant challenge, especially as data collection platforms become more interconnected with other platforms. A focus for us is ensuring that we have stored and delivered responses in a secure fashion, and ensuring our own internal processes protect our customers’ data.

Another concern involves scaling and the ability to retrieve and analyze large response sets. When customer datasets tend to be on the smaller side, this presents difficulties in performance scaling.

Lastly, integrating collected responses into customers’ workflows is a constant and growing need. Customers expect the data they’ve collected to be connected to their systems of record. For example, when collecting contact information in a lead generation use case, they expect those leads to ultimately live in their CRM. This requires management of many connections to a wide variety of other systems in order to meet customers’ needs.

What skills do developers need to be proficient collecting and analyzing conversations?

For data collection, a major skill needed for developers is the ability to understand how to store widely varying data in scalable database systems. Understanding how to rationalize diverse responses into consistent data models, which can be stored in NoSQL-style databases is a critical skill.

In terms of skills needed to analyze the data collected, these require data science techniques such as machine learning (ML) and natural language processing (NLP), to provide stronger insights about the responses we’ve stored.

Collecting data in a conversational way requires excellent design and front-end/Javascript development skills. However, with heavily integrated workflows, it’s important to understand that integrating systems using backend technologies is a constant need. We use Golang for our microservices (which are provided as external APIs in some cases), as well as for building integrations with other platforms.

What have I failed to ask that you think we need to cover on this topic?

Diving a bit further into advanced use cases, and in order to provide our partners with the tools to integrate Typeform into their products, we have provided external REST APIs and Webhooks. This is often thought of as something that is added to an existing product. We believe that microservices is the core of our platform, selectively externalized as public/partner APIs ensures that developers who work with us externally have the same capabilities as developers inside Typeform. This produces a more consistent experience as we continue to connect conversational data collection to other platforms our customers consider critical.

A key aspect of understanding how to work with external developers is to create an environment where you experience what an outside developer experiences. This process which we call "drinking our own champagne" creates a healthier API design process. We’re also moving toward working more directly with our partners and ISVs on providing early access to developer tools for their input; the highest quality feedback on developer experience is best achieved through tactile feedback, actually coding against it.

Running out of memory? Never run out of memory with Redis Enterprise databaseStart your free trial today.

Topics:
database

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}