Over a million developers have joined DZone.

The Cloudcast #155 - Large Scale Distributed Infrastructure with Mesos

· Cloud Zone

Download the Essential Cloud Buyer’s Guide to learn important factors to consider before selecting a provider as well as buying criteria to help you make the best decision for your infrastructure needs, brought to you in partnership with Internap.

Download the MP3

Date: July 31, 2014

By: Aaron Delp and Brian Gracely

Description: Aaron and Brian talk with Dave Lester (@davelester, Open Source Advocate at Twitter. Friend of @ApacheMesos, @ApacheAurora) about how Twitter manages large-scale infrastructure, an introduction to Apache Mesos and how projects like Kubernetes, Docker, Aurora are helping to define the next-generation of web-scale infrastructure management. Music Credit: Nine Inch Nails (www.nin.com)

Today’s show is dedicated to the memory of a good friend of the show, Jim Ruddy, who left us last week in a tragic car accident. Jim loved his family, life and exploring all the new technology that we cover on the show. He’ll be deeply missed by everybody that knew him.

Topic 1 - We met at OSCON last week and you introduced me to Apache Mesos. Give us some of your background and what you do at Twitter with regard to Mesos?

Topic 2 - Let’s talk about the types of challenges that Mesos attempts to solve at Twitter and why so many webscale companies are using it. (Twitter, AirBnB, CloudPhysics, eBay, Netflix, PayPal, Vimeo, etc.)

Topic 3 - As I started digging into the Mesos architecture, it’s a fairly complex set of inter-related activities (Mesos Controller, Hadoop, ZooKeeper, Docker). Can you walk us through the basics of what area function does or some common use-cases?
  • Zookeeper - like a CMDB - keeping track of configs
  • Hadoop Scheduler - collecting data to feed the Mesos scheduler?
  • Mesos Master - making scheduling decisions (using hadoop data) and pulling configs from Zookeeper?

Topic 4 - Mesos appears to be primarily focused on compute, but does it have any logic to also interact with networking elements (network segments, load-balancers, firewalls, etc.)?

Topic 5 - Is Mesos a system that could live on top of something like OpenStack, or is it primarily focused on containers or bare-metal compute clusters?

Topic 5 - Some of our listeners have heard about this thing that Google released, called Kubernetes, which they use to build their Google Compute Engine. On some level it sounds like another resource scheduler, but we also heard about Mesos + Kubernetes interaction. Are these groups solving similar problems, or are they finding new ways to use tools/frameworks that might have been used in other ways before (eg. Hadoop)?

Topic 6 - There is an event happening in August (21st-22nd) called MesosCon, in Chicago (along with LinuxCon). What are the interesting things that will be happening those days?

The Cloud Zone is brought to you in partnership with Internap. Read Bare-Metal Cloud 101 to learn about bare-metal cloud and how it has emerged as a way to complement virtualized services.


Published at DZone with permission of Brian Gracely, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}