Over a million developers have joined DZone.

The Cloudcast #155 - Large Scale Distributed Infrastructure with Mesos

DZone's Guide to

The Cloudcast #155 - Large Scale Distributed Infrastructure with Mesos

· Cloud Zone
Free Resource

MongoDB Atlas is a database as a service that makes it easy to deploy, manage, and scale MongoDB. So you can focus on innovation, not operations. Brought to you in partnership with MongoDB.

Download the MP3

Date: July 31, 2014

By: Aaron Delp and Brian Gracely

Description: Aaron and Brian talk with Dave Lester (@davelester, Open Source Advocate at Twitter. Friend of @ApacheMesos, @ApacheAurora) about how Twitter manages large-scale infrastructure, an introduction to Apache Mesos and how projects like Kubernetes, Docker, Aurora are helping to define the next-generation of web-scale infrastructure management. Music Credit: Nine Inch Nails (www.nin.com)

Today’s show is dedicated to the memory of a good friend of the show, Jim Ruddy, who left us last week in a tragic car accident. Jim loved his family, life and exploring all the new technology that we cover on the show. He’ll be deeply missed by everybody that knew him.

Topic 1 - We met at OSCON last week and you introduced me to Apache Mesos. Give us some of your background and what you do at Twitter with regard to Mesos?

Topic 2 - Let’s talk about the types of challenges that Mesos attempts to solve at Twitter and why so many webscale companies are using it. (Twitter, AirBnB, CloudPhysics, eBay, Netflix, PayPal, Vimeo, etc.)

Topic 3 - As I started digging into the Mesos architecture, it’s a fairly complex set of inter-related activities (Mesos Controller, Hadoop, ZooKeeper, Docker). Can you walk us through the basics of what area function does or some common use-cases?
  • Zookeeper - like a CMDB - keeping track of configs
  • Hadoop Scheduler - collecting data to feed the Mesos scheduler?
  • Mesos Master - making scheduling decisions (using hadoop data) and pulling configs from Zookeeper?

Topic 4 - Mesos appears to be primarily focused on compute, but does it have any logic to also interact with networking elements (network segments, load-balancers, firewalls, etc.)?

Topic 5 - Is Mesos a system that could live on top of something like OpenStack, or is it primarily focused on containers or bare-metal compute clusters?

Topic 5 - Some of our listeners have heard about this thing that Google released, called Kubernetes, which they use to build their Google Compute Engine. On some level it sounds like another resource scheduler, but we also heard about Mesos + Kubernetes interaction. Are these groups solving similar problems, or are they finding new ways to use tools/frameworks that might have been used in other ways before (eg. Hadoop)?

Topic 6 - There is an event happening in August (21st-22nd) called MesosCon, in Chicago (along with LinuxCon). What are the interesting things that will be happening those days?

MongoDB Atlas is the best way to run MongoDB on AWS — highly secure by default, highly available, and fully elastic. Get started free. Brought to you in partnership with MongoDB.


Published at DZone with permission of Brian Gracely, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.


Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.


{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}