Chapter 3 overview

How to Manage Your Apps Using Orchestration Tools

Learn how to deploy and manage your apps using orchestration approaches such as server orchestration (e.g., Ansible), VM orchestration (e.g., Auto Scaling Groups), container orchestration (e.g., Kubernetes), and serverless orchestration (e.g., AWS Lambda).

Key ideas you'll learn

  • Server orchestration
  • VM orchestration
  • Container orchestration
  • Serverless orchestration

Examples you'll try

  • Use Ansible to deploy app servers & Nginx
  • Use OpenTofu to deploy an ASG and ALB
  • Deploy a Dockerized app in Kubernetes
  • Deploy a serverless app with AWS Lambda

Table of contents

3.1 An Introduction to Orchestration
3.2 Server Orchestration
3.2.1 Example: Deploy an App Securely and Reliably Using Ansible
3.2.2 Example: Deploy a Load Balancer Using Ansible and Nginx
3.2.3 Example: Roll Out Updates with Ansible
3.3 VM Orchestration
3.3.1 Example: Build a VM Image Using Packer
3.3.2 Example: Deploy a VM Image in an Auto Scaling Group Using OpenTofu
3.3.3 Example: Deploy an Application Load Balancer Using OpenTofu
3.3.4 Example: Roll Out Updates with OpenTofu and Auto Scaling Groups
3.4 Container Orchestration
3.4.1 Example: A Crash Course on Docker
3.4.2 Example: Create a Docker Image for a Node.js app
3.4.3 Example: Deploy a Dockerized App with Kubernetes
3.4.4 Example: Deploy a Load Balancer with Kubernetes
3.4.5 Example: Roll Out Updates with Kubernetes
3.4.6 Example: Deploy a Kubernetes Cluster in AWS Using EKS
3.4.7 Example: Push a Docker Image to ECR
3.4.8 Example: Deploy a Dockerized App into an EKS Cluster
3.5 Serverless Orchestration
3.5.1 Example: Deploy a Serverless Function with AWS Lambda
3.5.2 Example: Create a Lambda Function URL
3.5.3 Example: Roll Out Updates with AWS Lambda
3.6 Comparing Orchestration Options
3.7 Conclusion

Related Books

Kubernetes in Action

By Marko Luksa (Manning)

recommended

With Kubernetes, users don't have to worry about which specific machine in their data center their application is running on. Each layer in their application is decoupled from other layers so they can scale, update, and maintain them independently. Kubernetes in Action teaches developers how to use Kubernetes to deploy self-healing scalable distributed applications. By the end, readers will be able to build and deploy applications in a proper way to take full advantage of the Kubernetes platform.

Cloud Native Infrastructure: Patterns for Scalable Infrastructure and Applications in a Dynamic Environment

By Justin Garrison and Kris Nova (O'Reilly)

Cloud native infrastructure is more than servers, network, and storage in the cloud—it is as much about operational hygiene as it is about elasticity and scalability. In this book, you’ll learn practices, patterns, and requirements for creating infrastructure that meets your needs, capable of managing the full life cycle of cloud native applications. Justin Garrison and Kris Nova reveal hard-earned lessons on architecting infrastructure from companies such as Google, Amazon, and Netflix. They draw inspiration from projects adopted by the Cloud Native Computing Foundation (CNCF), and provide examples of patterns seen in existing tools such as Kubernetes.

Cloud Native DevOps with Kubernetes: Building, Deploying, and Scaling Modern Applications in the Cloud

By John Arundel and Justin Domingus (O'Reilly)

Kubernetes is the operating system of the cloud native world, providing a reliable and scalable platform for running containerized workloads. In this friendly, pragmatic book, cloud experts John Arundel and Justin Domingus show you what Kubernetes can do—and what you can do with it. You’ll learn all about the Kubernetes ecosystem, and use battle-tested solutions to everyday problems. You’ll build, step by step, an example cloud native application and its supporting infrastructure, along with a development environment and continuous deployment pipeline that you can use for your own applications.

Cloud-Native Patterns: Designing Change-Tolerant Software

By Cornelia Davis (Manning)

Author Cornelia Davis likes to say that 'the cloud' is more about how you design your applications than where you deploy them. Cloud Native Patterns: Designing Change-tolerant Software is your guide to developing strong applications that thrive in the dynamic, distributed, virtual world of the cloud. This book presents a mental model for cloud-native applications, along with the patterns, practices, and tooling that sets them apart. In it, you'll find realistic examples and expert advice for working with apps, data, services, routing, and more.

Kubernetes: Up and Running: Dive into the Future of Infrastructure

By Kelsey Hightower, Brendan Burns, and Joe Beda (O'Reilly)

Legend has it that Google deploys over two billion application containers a week. How’s that possible? Google revealed the secret through a project called Kubernetes, an open source cluster orchestrator (based on its internal Borg system) that radically simplifies the task of building, deploying, and maintaining scalable distributed systems in the cloud. This practical guide shows you how Kubernetes and container technology can help you achieve new levels of velocity, agility, reliability, and efficiency. Authors Kelsey Hightower, Brendan Burns, and Joe Beda—who’ve worked on Kubernetes at Google and other organizations—explain how this system fits into the lifecycle of a distributed application. You will learn how to use tools and APIs to automate scalable distributed systems, whether it is for online services, machine-learning applications, or a cluster of Raspberry Pi computers.

Docker: Up and Running: Shipping Reliable Containers in Production

By Karl Matthias and Sean P. Kane (O'Reilly)

Docker is quickly changing the way that organizations are deploying software at scale. But understanding how Linux containers fit into your workflow--and getting the integration details right--are not trivial tasks. With this practical guide, you'll learn how to use Docker to package your applications with all of their dependencies, and then test, ship, scale, and support your containers in production. Two Lead Site Reliability Engineers at New Relic share much of what they have learned from using Docker in production since shortly after its initial release. Their goal is to help you reap the benefits of this technology while avoiding the many setbacks they experienced.

Kubernetes Patterns: Reusable Elements for Designing Cloud Native Applications

By Bilgin Ibryam and Roland Huß (O'Reilly)

The way developers design, build, and run software has changed significantly with the evolution of microservices and containers. These modern architectures use new primitives that require a different set of practices than most developers, tech leads, and architects are accustomed to. With this focused guide, Bilgin Ibryam and Roland Huß from Red Hat provide common reusable elements, patterns, principles, and practices for designing and implementing cloud-native applications on Kubernetes. Each pattern includes a description of the problem and a proposed solution with Kubernetes specifics. Many patterns are also backed by concrete code examples. This book is ideal for developers already familiar with basic Kubernetes concepts who want to learn common cloud-native patterns.

Production Kubernetes: Building Successful Application Platforms

By Josh Rosso, Rich Lander, Alex Brand, John Harris (O'Reilly)

Kubernetes has become the dominant container orchestrator, but many organizations that have recently adopted this system are still struggling to run actual production workloads. In this practical book, four software engineers from VMware bring their shared experiences running Kubernetes in production and provide insight on key challenges and best practices. The brilliance of Kubernetes is how configurable and extensible the system is, from pluggable runtimes to storage integrations. For platform engineers, software developers, infosec, network engineers, storage engineers, and others, this book examines how the path to success with Kubernetes involves a variety of technology, pattern, and abstraction considerations.

Serverless Architectures on AWS: With Examples Using AWS Lambda

By Peter Sbarski (Manning)

Serverless Architectures on AWS teaches you how to build, secure and manage serverless architectures that can power the most demanding web and mobile apps.

Other Related Resources

A Brief History of Containers: From the 1970s Till Now

By Rani Osnat (Blog post)

The history of virtual container technology since the 70s with chroot & Unix, through the rise of Docker, Kubernetes' take over, and the future of Containers.

Deployment Script Tools

Capistrano

A remote server automation and deployment tool written in Ruby.

Deployer

Deployer is a tool for deploying PHP applications to a remote server.

Mina

Really fast deployer and server automation tool.

Fabric

Fabric is a high level Python (2.7, 3.4+) library designed to execute shell commands remotely over SSH, yielding useful Python objects in return.

Shipit

Universal automation and deployment tool ⛵️.

Process Supervisor Tools

systemd

recommended used-in-book

systemd is a suite of basic building blocks for a Linux system. It provides a system and service manager that runs as PID 1 and starts the rest of the system.

supervisord

Supervisor is a client/server system that allows its users to monitor and control a number of processes on UNIX-like operating systems. It shares some of the same goals of programs like launchd, daemontools, and runit. Unlike some of these programs, it is not meant to be run as a substitute for init as “process id 1”. Instead it is meant to be used to control processes related to a project or a customer, and is meant to start like any other program at boot time.

runit

runit is a cross-platform Unix init scheme with service supervision, a replacement for sysvinit, and other init schemes. It runs on GNU/Linux, *BSD, MacOSX, Solaris, and can easily be adapted to other Unix operating systems.

PM2

recommended used-in-book

Advanced process manager for production Node.js applications. Load balancer, logs facility, startup script, micro service management, at a glance.

Load Balancer (Web Server) Tools

Apache HTTP Server

The Apache HTTP Server Project is an effort to develop and maintain an open-source HTTP server for modern operating systems including UNIX and Windows. The goal of this project is to provide a secure, efficient and extensible server that provides HTTP services in sync with the current HTTP standards.

Nginx

recommended used-in-book

Optimize, deliver, and secure apps across the entire enterprise with NGINX.

HAProxy

Reliable, High Performance TCP/HTTP Load Balancer.

Caddy

Caddy is a powerful, enterprise-ready, open source web server with automatic HTTPS written in Go.

Varnish

Varnish Cache is a web application accelerator also known as a caching HTTP reverse proxy. You install it in front of any server that speaks HTTP and configure it to cache the contents. Varnish Cache is really, really fast. It typically speeds up delivery with a factor of 300 - 1000x, depending on your architecture.

Lighttpd

lighttpd (pronounced /lighty/) is a secure, fast, compliant, and very flexible web server that has been optimized for high-performance environments. lighttpd uses memory and CPU efficiently and has lower resource use than other popular web servers. Its advanced feature-set (FastCGI, CGI, Auth, Output-Compression, URL-Rewriting and much more) make lighttpd the perfect web server for all systems, small and large. lighttpd is released under the Open Source revised BSD license.

Microsoft IIS

Internet Information Services (IIS) for Windows Server is a flexible, secure and manageable Web server for hosting anything on the Web. From media streaming to web applications, IIS's scalable and open architecture is ready to handle the most demanding tasks.

AWS Elastic Load Balancer

recommended used-in-book

Elastic Load Balancing (ELB) automatically distributes incoming application traffic across multiple targets and virtual appliances in one or more Availability Zones (AZs).

GCP Cloud Load Balancer

High performance, scalable global load balancing on Google’s worldwide network, with support for HTTP(S), TCP/SSL, UDP, and autoscaling.

Azure Load Balancer

Azure Load Balancer operates at layer 4 of the Open Systems Interconnection (OSI) model. It's the single point of contact for clients. The service distributes inbound flows that arrive at the load balancer's frontend to backend pool instances. These flows are distributed according to configured load-balancing rules and health probes. The backend pool instances can be Azure virtual machines (VMs) or virtual machine scale sets.

VM Cluster Tools

Auto Scaling Groups (ASG)

recommended used-in-book

Automatically maintain application performance and availability by dynamically scaling EC2 instances based on user-defined policies and health checks.

Managed Instance Groups

An instance group is a collection of virtual machine (VM) instances that you can manage as a single entity.

Scale Sets

Azure Virtual Machine Scale Sets let you create and manage a group of load balanced VMs. The number of VM instances can automatically increase or decrease in response to demand or a defined schedule.

Container Orchestration Tools

Kubernetes

recommended used-in-book

Kubernetes, also known as K8s, is an open source system for automating deployment, scaling, and management of containerized applications. It groups containers that make up an application into logical units for easy management and discovery. Kubernetes builds upon 15 years of experience of running production workloads at Google, combined with best-of-breed ideas and practices from the community. Planet Scale Designed on the same principles that allow Google to run billions of containers a week, Kubernetes can scale without increasing your operations team.

Amazon ECS

recommended

Amazon Elastic Container Service (Amazon ECS) provides a fully managed container service solution that’s easy to use, scalable, secure, and reliable.

Nomad

Nomad is a simple and flexible scheduler and orchestrator for managing containers and non-containerized applications across on-prem and clouds at scale.

OpenShift

A unified application development platform that lets you build, modernize, and deploy applications at scale on your choice of hybrid cloud infrastructure.

Docker Swarm

Swarm mode is an advanced feature for managing a cluster of Docker daemons. Use Swarm mode if you intend to use Swarm as a production runtime environment.

Mesos

Apache Mesos abstracts resources away from machines, enabling fault-tolerant and elastic distributed systems to easily be built and run effectively.

Kubernetes Application Configuration Tools

Helm

recommended

Helm - The Kubernetes Package Manager.

OpenTofu

recommended

The open source infrastructure as code tool.

Pulumi

Pulumi is an infrastructure management platform to automate through infrastructure as code, secure with secrets management, and manage infrastructure with AI.

Kustomize

Kubernetes native configuration management.

kapp

Deploy and view groups of Kubernetes resources as 'applications'. Apply changes safely and predictably, watching resources as they converge.

Container Registry Tools

Docker Hub

recommended used-in-book

Welcome to the world's largest container registry built for developers and open source contributors to find, use, and share their container images. Build, push and pull.

Amazon Elastic Container Registry (ECR)

recommended used-in-book

Amazon Elastic Container Registry (ECR) is a fully managed Docker container registry that makes it easy to store, share, and deploy container images.

Azure Container Registry

A registry of Docker and Open Container Initiative (OCI) images, with support for all OCI artifacts.

Google Artifact Registry

A universal package manager for all your build artifacts and dependencies. Fast, scalable, reliable and secure.

GitHub Container Registry

You can store and manage Docker and OCI images in the Container registry.

JFrog Docker Registry

Manage your Docker images within a private Docker registry, with high availability, in a single Artifactory and unlimited image pulls.

Serverless Tools

AWS Lambda

recommended used-in-book

AWS Lambda is a serverless compute service for running code without having to provision or manage servers. You pay only for the compute time you consume.

GCP Cloud Functions

Scalable pay-as-you-go Function-as-a-Service (FaaS) to run your code with zero server management.

Azure serverless

Build, deploy, and operate serverless apps with end-to-end Azure serverless solutions. Focus on solving business problems instead of managing infrastructure.

CloudFlare Workers

Deploy serverless code instantly across the globe to give it exceptional performance, reliability, and scale.

Vercel Functions

Vercel Functions allow you to run server-side code without managing a server.

AWS Fargate

recommended

AWS Fargate is a serverless compute engine for containers that works with both Amazon Elastic Container Service (ECS) and Amazon Elastic Kubernetes (EKS).

Knative

Knative is an open-source enterprise-level solution to build serverless and event driven applications.

Lambda Function URLs

recommended used-in-book

Configure a Lambda function URL to assign an HTTP(S) endpoint to your Lambda function without having to integrate with other AWS services.

Amazon API Gateway

Amazon API Gateway helps you build HTTP, REST, and WebSocket APIs with a fully managed service that makes it easy to create, publish, maintain, manage, monitor, and secure APIs.

Amazon Aurora Serverless

With Amazon Aurora Serverless, there are no DB Instances to manage. The database automatically starts, stops, and scales capacity up or down based on your application's needs.

Serverless Framework

recommended

Easily build auto-scaling, low-overhead applications on AWS Lambda, API Gateway, DynamoDB, and other managed services with the Serverless Framework.

SAM

Simplify how you build, deploy, deliver, and share serverless applications on AWS.

Comments