Container Clustering & Management: Docker Swarm, Kubernetes & Apache Mesos and Marathon

Docker Swarm:

Swarm is the native Docker clustering solution, Swarm extends the existing Docker API to make a cluster of machines look like a single Docker API.

Docker Swarm

Key Components:

  • Managers: Distribute tasks across the cluster, with one manager orchestrating the worker nodes that make up the swarm.
  • Workers: Run Docker containers assigned to them by a manager.
  • Services: An interface to a particular set of Docker containers running across the swarm.
  • Tasks: Individual Docker containers running the image, plus commands, needed by a particular service.
  • Key-value store: etcd, Consul or Zookeeper storing the swarm’s state and providing service discoverability.

Calsoft Whitepaper: File Virtualization with the Direct NFS Protocol

This paper discusses a SANergy-like file system that works on UNIX clients against UNIX servers using NFS, and provides the performance of a SAN, and the manageability of NAS. This paper was published at the NASA/IEEE Symposium on Mass Storage Systems, Maryland.

Download

When to use Docker Swarm:

Swarm has been viewed as suitable for experiments and smaller scale deployments; Docker Swarm gives you an easy way to move into container orchestration.

Kubernetes:

Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. Kubernetes was built by Google.

Kubernetes

Key Components:

  • Master: A single master handles API calls, assigns workloads and maintains configuration state.
  • Minions: The servers that run workloads and anything else that’s not on the master.
  • Pods: Units of compute power, made-up of one or a handful of containers deployed on the same host, that together perform a task; have a single IP address and flat networking within the pods.
  • Services: Front end and load balancer for pods, providing a floating IP for access to the pods that power the service, meaning that changes can happen at the pod-level while maintaining a stable interface.
  • Replication controllers: Responsible for maintaining ‘X’ replicas of the required pods.
  • Labels: key-value tags that the system use to identify pods, replication controllers and services.

 

When to use Kubernetes:

Kubernetes is well suited to medium to large clusters running complex applications

Apache Mesos and Marathon:

Mesos is a Multi-Framework orchestration solution for containers.

Mesos is less focused on running just containers. Mesos focuses on scheduling, and plugging in multiple different schedulers,  and  the result, the Hadoop and Marathon can co-exist in the same scheduling environment.

Apchae Mesos

Key Components:

  • Masters: Zookeeper manages a minimum of three master nodes and enables high availability by relying on a quorum amongst those nodes.
  • Slaves: these nodes run the tasks passed down by the framework.
  • Framework: Mesos itself knows nothing about the workloads, whereas specialist frameworks decide what to do with the resources offered to them by Mesos.

On top of Mesos, Marathon then offers a highly available framework delivering:

  • Service discovery: through a dedicated DNS service, as well as other options.
  • Load balancing: through HAProxy.
  • Constraint management: to control where in the cluster certain workloads run, maintain a set level of resources for those workloads, enable rack awareness and other constraints.
  • Applications: the long running services you want to run on the cluster; may be Docker containers but can also be other types of workload.
  • REST API: deploy, alter and destroy workloads.

 

When to use Mesos and Marathon:

Mesos and Marathon are, perhaps, a good choice if you need to run non-containerized workloads alongside containers and you want the reassurance of something that is proven at very large scale.

Conclusion

The three main options right now vary considerably in implementation. Docker’s Swarm gives you the easiest route into orchestrating a cluster of Docker hosts. Kubernetes is container-centric but focuses less on the containers themselves and more on deploying and managing services. Mesos with Marathon promises huge scale but introduces additional complexity.

To know more email: marketing@calsoftinc.com

Yogesh Patil | Calsoft Inc.

Container Ecosystem Services

Calsoft has deep expertise in containerization of Storage and Networking products. With our in-depth understanding of various containerization technologies like Docker, Kubernetes, Apache Mesos and Coreos, we have helped ISVs to design and develop solutions in and around these technologies.

 

Save

Yogesh Patil

Yogesh Patil

Senior Test Lead at Calsoft Inc.
Software QA professional with over 10 years of industry experience. Currently working with Calsoft as a Sr. QA Lead.
His experience is across various technology domains; from Systems Engineering, Distributed Computing to Storage, Virtualization and Cloud computing.
Yogesh Patil

2 thoughts on “Container Clustering & Management: Docker Swarm, Kubernetes & Apache Mesos and Marathon

  1. This blog explains the details of most popular technological details. This helps to learn about what are all the different method is there. And the working methods all of that are explained here. Informative blog.

Leave a Reply

Your email address will not be published. Required fields are marked *