DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Data Engineering
  3. Big Data
  4. How Splice Machine Democratizes AI/ML for Business

How Splice Machine Democratizes AI/ML for Business

Companies can analyze petabytes of data and power applications in real-time. OLPP is a new way to provide the power of the lambda architecture with SQL!

Tom Smith user avatar by
Tom Smith
CORE ·
Oct. 24, 17 · News
Like (2)
Save
Tweet
Share
3.17K Views

Join the DZone community and get the full member experience.

Join For Free

It was great talking with Monte Zweben, CEO of Splice Machine, about how he and his team are making artificial intelligence (AI) and machine learning (ML) data processing easier than constructing a lambda architecture.

Monte worked on machine learning systems for NASA, serves on the advisory board of the Dean of Computer Science at Carnegie Mellon University, and was Chairman of Rocket Fuel, where he saw first-hand the difficulty of putting together the three different engines of a lambda architecture to make AI/ML computations:

  1. A batch layer for data analysis and building ML models, like Spark or Hive.
  2. A streaming engine for ingesting and analyzing streams in real-time, like Kafka.
  3. An app serving layer for low-latency computing, which looks up and updates data in milliseconds for applications, like HBase.

Specific limitations of lambda architectures that Splice Machine set out to address include:

  • Complexity: Separate systems, often open-source projects, written in different languages, that all need to be integrated and maintained to remain operational.
  • Specialized skills: Requires developers who program in multiple programming languages and distributed system paradigms (and these are not easy people to find).
  • Loose coupling: Engines are loosely coupled, so changes to data in one layer take time and effort to be communicated to another layer in order to be considered in the analysis.
  • Concurrency: Unlike operational databases designed to handle concurrent users, Lambda-based systems can make it difficult to ensure updates are made properly in the event of errors, power failures, and simultaneous updates.
  • Resilience: Pipelines often need to be restarted, resulting in high latency and missed deadlines. Companies that experience these missed deadlines are flying blind and often must act on stale data.

While big companies like Google, Facebook, and Netflix may have the computing power and data scientists necessary to do this work, few other companies have these resources. Splice Machine strives to make it simple for companies by providing an online predictive processing (OLPP) RDBMS with a lambda architecture underneath. By democratizing the lambda architecture with SQL, companies can scale and analyze petabytes of data in real-time.

A couple of examples:

  • Clearsense uses big data analysis of electronic medical records (EMRs) of patients in hospitals to predict sepsis or code blue events so that nurses can intervene and prevent these potentially deadly events from occurring.
  • Mojix, an IoT company that uses RFID to track supply chain and inventory, had a lambda architecture that was too complex. Splice Machine provided them with a platform that enables them to ingest and analyze their data in real-time as well as power their application.
  • A leading financial services company is powering a customer service application with seven petabytes of data and is able to look up and update data in milliseconds.

OLPP is a new way to provide the power of the lambda architecture but with the familiarity and ubiquity of SQL.

Machine learning Splice (platform) Data science Big data Lambda architecture Architecture

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • The Changing Face of ETL
  • The Key Assumption of Modern Work Culture
  • Apache Kafka Introduction, Installation, and Implementation Using .NET Core 6
  • The Real Democratization of AI, and Why It Has to Be Closely Monitored

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: