Over a million developers have joined DZone.

The Cloudcast #155 - Large Scale Distributed Infrastructure with Mesos

DZone's Guide to

The Cloudcast #155 - Large Scale Distributed Infrastructure with Mesos

· Cloud Zone
Free Resource

Linkerd, the open source service mesh for cloud native applications. Get the complete guide to using Linkerd and Kubernetes to build scalable, resilient applications.

Download the MP3

Date: July 31, 2014

By: Aaron Delp and Brian Gracely

Description: Aaron and Brian talk with Dave Lester (@davelester, Open Source Advocate at Twitter. Friend of @ApacheMesos, @ApacheAurora) about how Twitter manages large-scale infrastructure, an introduction to Apache Mesos and how projects like Kubernetes, Docker, Aurora are helping to define the next-generation of web-scale infrastructure management. Music Credit: Nine Inch Nails (www.nin.com)

Today’s show is dedicated to the memory of a good friend of the show, Jim Ruddy, who left us last week in a tragic car accident. Jim loved his family, life and exploring all the new technology that we cover on the show. He’ll be deeply missed by everybody that knew him.

Topic 1 - We met at OSCON last week and you introduced me to Apache Mesos. Give us some of your background and what you do at Twitter with regard to Mesos?

Topic 2 - Let’s talk about the types of challenges that Mesos attempts to solve at Twitter and why so many webscale companies are using it. (Twitter, AirBnB, CloudPhysics, eBay, Netflix, PayPal, Vimeo, etc.)

Topic 3 - As I started digging into the Mesos architecture, it’s a fairly complex set of inter-related activities (Mesos Controller, Hadoop, ZooKeeper, Docker). Can you walk us through the basics of what area function does or some common use-cases?
  • Zookeeper - like a CMDB - keeping track of configs
  • Hadoop Scheduler - collecting data to feed the Mesos scheduler?
  • Mesos Master - making scheduling decisions (using hadoop data) and pulling configs from Zookeeper?

Topic 4 - Mesos appears to be primarily focused on compute, but does it have any logic to also interact with networking elements (network segments, load-balancers, firewalls, etc.)?

Topic 5 - Is Mesos a system that could live on top of something like OpenStack, or is it primarily focused on containers or bare-metal compute clusters?

Topic 5 - Some of our listeners have heard about this thing that Google released, called Kubernetes, which they use to build their Google Compute Engine. On some level it sounds like another resource scheduler, but we also heard about Mesos + Kubernetes interaction. Are these groups solving similar problems, or are they finding new ways to use tools/frameworks that might have been used in other ways before (eg. Hadoop)?

Topic 6 - There is an event happening in August (21st-22nd) called MesosCon, in Chicago (along with LinuxCon). What are the interesting things that will be happening those days?

Linkerd, the open source service mesh for cloud native applications. Get the complete guide to using Linkerd and Kubernetes to build scalable, resilient applications.


Published at DZone with permission of Brian Gracely, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.


Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.


{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}